diff --git a/docs/diagrams/slack-error-alert.png b/docs/diagrams/slack-error-alert.png new file mode 100644 index 00000000..71bde57e Binary files /dev/null and b/docs/diagrams/slack-error-alert.png differ diff --git a/docs/diagrams/slack-info-alert.png b/docs/diagrams/slack-info-alert.png new file mode 100644 index 00000000..d6191008 Binary files /dev/null and b/docs/diagrams/slack-info-alert.png differ diff --git a/docs/get-started/index.md b/docs/get-started/index.md index 57692d69..e91b2eb9 100644 --- a/docs/get-started/index.md +++ b/docs/get-started/index.md @@ -21,7 +21,7 @@ export GITHUB_USER= To install the latest `tk` release run: -```bash +```sh curl -s https://toolkit.fluxcd.io/install.sh | sudo bash ``` @@ -58,7 +58,7 @@ kubectl cluster-info --context kind-staging Verify that your staging cluster satisfies the prerequisites with: -```text +```console $ tk check --pre ► checking prerequisites @@ -104,22 +104,9 @@ $ tk bootstrap github --owner=gitopsrun --repository=fleet-infra --path=staging- ✚ generating manifests ✔ components manifests pushed ► installing components in gitops-system namespace -namespace/gitops-system created -customresourcedefinition.apiextensions.k8s.io/gitrepositories.source.fluxcd.io created -customresourcedefinition.apiextensions.k8s.io/helmcharts.source.fluxcd.io created -customresourcedefinition.apiextensions.k8s.io/helmrepositories.source.fluxcd.io created -customresourcedefinition.apiextensions.k8s.io/kustomizations.kustomize.fluxcd.io created -customresourcedefinition.apiextensions.k8s.io/profiles.kustomize.fluxcd.io created -role.rbac.authorization.k8s.io/crd-controller-gitops-system created -rolebinding.rbac.authorization.k8s.io/crd-controller-gitops-system created -clusterrolebinding.rbac.authorization.k8s.io/cluster-reconciler-gitops-system created -service/source-controller created -deployment.apps/kustomize-controller created -deployment.apps/source-controller created -networkpolicy.networking.k8s.io/deny-ingress created -Waiting for deployment "source-controller" rollout to finish: 0 of 1 updated replicas are available... deployment "source-controller" successfully rolled out deployment "kustomize-controller" successfully rolled out +deployment "notification-controller" successfully rolled out ✔ install completed ► configuring deploy key ✔ deploy key configured @@ -208,7 +195,7 @@ git add -A && git commit -m "add staging webapp" && git push In about 30s the synchronization should start: -```text +```console $ watch tk get kustomizations ✔ gitops-system last applied revision master/35d5765a1acb9e9ce66cad7274c6fe03eee1e8eb @@ -219,7 +206,7 @@ $ watch tk get kustomizations When the synchronization finishes you can check that the webapp services are running: -```text +```console $ kubectl -n webapp get deployments,services NAME READY UP-TO-DATE AVAILABLE AGE @@ -307,7 +294,7 @@ git add -A && git commit -m "add prod webapp" && git push List git sources: -```text +```console $ tk get sources git ✔ gitops-system last fetched revision master/99072ee132abdead8b7799d7891eae2f524eb73d @@ -318,7 +305,7 @@ The kubectl equivalent is `kubectl -n gitops-system get gitrepositories`. List kustomization: -```text +```console $ tk get kustomizations ✔ gitops-system last applied revision master/99072ee132abdead8b7799d7891eae2f524eb73d @@ -341,7 +328,7 @@ git add -A && git commit -m "update prod webapp" && git push Trigger a git sync: -```text +```console $ tk sync ks gitops-system --with-source ► annotating source gitops-system @@ -358,7 +345,7 @@ The kubectl equivalent is `kubectl -n gitops-system annotate gitrepository/gitop Wait for the webapp to be upgraded: -```text +```console $ watch tk get kustomizations ✔ gitops-system last applied revision master/d751ea264d48bf0db8b588d1d08184834ac8fec9 diff --git a/docs/guides/notifications.md b/docs/guides/notifications.md new file mode 100644 index 00000000..4ba92203 --- /dev/null +++ b/docs/guides/notifications.md @@ -0,0 +1,101 @@ +# Setup Notifications + +When operating a cluster, different teams may wish to receive notifications about +the status of their GitOps pipelines. +For example, the on-call team would receive alerts about reconciliation +failures in the cluster, while the dev team may wish to be alerted when a new version +of an app was deployed and if the deployment is healthy. + +## Prerequisites + +* [Get started guide](../get-started/index.md) + +The GitOps toolkit controllers emit Kubernetes events whenever a resource status changes. +You can use the [notification-controller](../components/notification/controller.md) +to forward these events to Slack, Microsoft Teams, Discord or Rocket chart. +The notification controller is part of the default toolkit installation. + +## Define a provider + +First create a secret with your Slack incoming webhook: + +```sh +kubectl -n gitops-system create secret generic slack-url \ +--from-literal=address=https://hooks.slack.com/services/YOUR/SLACK/WEBHOOK +``` + +Note that the secret must contain an `address` field, +it can be a Slack, Microsoft Teams, Discord or Rocket webhook URL. + +Create a notification provider for Slack by referencing the above secret: + +```yaml +apiVersion: notification.fluxcd.io/v1alpha1 +kind: Provider +metadata: + name: slack + namespace: gitops-system +spec: + type: slack + channel: general + secretRef: + name: slack-url +``` + +The provider type can be `slack`, `msteams`, `discord`, `rocket` or `webhook`. + +When type `webhook` is specified, the notification controller will post the incoming +[event](../components/notification/event.md) in JSON format to the webhook address. +This way you can create custom handlers that can store the events in +Elasticsearch, CloudWatch, Stackdriver, etc. + +## Define an alert + +Create an alert definition for the webapp kustomizations: + +```yaml +apiVersion: notification.fluxcd.io/v1alpha1 +kind: Alert +metadata: + name: on-call-webapp + namespace: gitops-system +spec: + providerRef: + name: slack + eventSeverity: info + eventSources: + - kind: Kustomization + name: webapp-backend + - kind: Kustomization + name: webapp-frontend +``` + +Apply the above files or commit them to the `fleet-infra` repository. + +To verify that the alert has been acknowledge by the notification controller do: + +```console +$ kubectl -n gitops-system get alerts + +NAME READY STATUS AGE +on-call-webapp True Initialized 1m +``` + +Multiple alerts can be used to send notifications to different channels or Slack organizations. + +The event severity can be set to `info` or `error`. +When the severity is set to `error`, the kustomize controller will alert on any error +encountered during the reconciliation process. +This includes kustomize build and validation errors, +apply errors and health check failures. + +![error alert](../diagrams/slack-error-alert.png) + +When the verbosity is set to `info`, the controller will alert if: + +* a Kubernetes object was created, updated or deleted +* heath checks are passing +* a dependency is delaying the execution +* an error occurs + +![info alert](../diagrams/slack-info-alert.png) diff --git a/mkdocs.yml b/mkdocs.yml index a60a69c0..a02ea240 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -34,10 +34,13 @@ markdown_extensions: - pymdownx.superfences: highlight_code: true - pymdownx.tabbed + - pymdownx.tilde nav: - Introduction: index.md - Get Started: get-started/index.md + - Guides: + - Setup Notifications: guides/notifications.md - Toolkit Components: - Source Controller: - Overview: components/source/controller.md @@ -50,7 +53,7 @@ nav: - Kustomize API Reference: components/kustomize/api.md - Notification Controller: - Overview: components/notification/controller.md - - Profile CRD: components/notification/profile.md + - Provider CRD: components/notification/provider.md - Alert CRD: components/notification/alert.md - Event: components/notification/event.md - Notification API Reference: components/notification/api.md