@ -108,17 +108,18 @@ Depending on what you want to do, some of the following bits might be your first
- Check out [how to contribute](CONTRIBUTING.md) to the project
### Upcoming Events
- 14 Dec 2020 - [The Power of GitOps with Flux and Flagger with Leigh Capili](https://www.meetup.com/GitOps-Community/events/274924513/)
- 11 Jan 2021 - [Helm + GitOps = ⚡️⚡️⚡️ with Scott Rigby](https://www.meetup.com/GitOps-Community/events/275348736/)
- 25 Jan 2021 - [GitOps Core Concepts & How to Teach Your Teams with Leigh Capili](https://www.meetup.com/GitOps-Community/events/275625806/)
### Featured Talks
- 14 Dec 2020 - [The Power of GitOps with Flux and Flagger (GitOps Hands-On) with Leigh Capili](https://youtu.be/cB7iXeNLteE)
- 30 Nov 2020 - [The Power of GitOps with Flux 2 - Part 3 with Leigh Capili](https://youtu.be/N_K5g7o9JKg)
- 24 Nov 2020 - [Flux CD v2 with GitOps Toolkit - Kubernetes Deployment and Sync Mechanism](https://youtu.be/R6OeIgb7lUI)
- 02 Nov 2020 - [The Power of GitOps with Flux & GitOps Toolkit - Part 2 with Leigh Capili](https://youtu.be/fC2YCxQRUwU)
- 28 Oct 2020 - [The Kubelist Podcast: Flux with Michael Bridgen](https://www.heavybit.com/library/podcasts/the-kubelist-podcast/ep-5-flux-with-michael-bridgen-of-weaveworks/)
- 19 Oct 2020 - [The Power of GitOps with Flux & GitOps Toolkit - Part 1 with Leigh Capili](https://youtu.be/0v5bjysXTL8)
- 30 Nov 2020 - [The Power of GitOps with Flux 2 - Part 3 with Leigh Capili](https://youtu.be/N_K5g7o9JKg)
- 12 Oct 2020 - [Rawkode Live: Introduction to GitOps Toolkit with Stefan Prodan](https://youtu.be/HqTzuOBP0eY)
- 4 Sep 2020 - [KubeCon Europe: The road to Flux v2 and Progressive Delivery with Stefan Prodan & Hidde Beydals](https://youtu.be/8v94nUkXsxU)
- 25 June 2020 - [Cloud Native Nordics: Introduction to GitOps & GitOps Toolkit with Alexis Richardson & Stefan Prodan](https://youtu.be/qQBtSkgl7tI)
- 04 Sep 2020 - [KubeCon Europe: The road to Flux v2 and Progressive Delivery with Stefan Prodan & Hidde Beydals](https://youtu.be/8v94nUkXsxU)
- 25 Jun 2020 - [Cloud Native Nordics: Introduction to GitOps & GitOps Toolkit with Alexis Richardson & Stefan Prodan](https://youtu.be/qQBtSkgl7tI)
returnfmt.Errorf("%s is an invalid project name for gitlab.\nIt can contain only letters, digits, emojis, '_', '.', dash, space. It must start with letter, digit, emoji or '_'.",glRepository)
--components strings list of components, accepts comma-separated values (default [source-controller,kustomize-controller,helm-controller,notification-controller])
--components strings list of components, accepts comma-separated values (default [source-controller,kustomize-controller,helm-controller,notification-controller])
--components strings list of components, accepts comma-separated values (default [source-controller,kustomize-controller,helm-controller,notification-controller])
--components strings list of components, accepts comma-separated values (default [source-controller,kustomize-controller,helm-controller,notification-controller])
--components-extra strings list of components in addition to those supplied or defaulted, accepts comma-separated values
GitOps is a way of managing your infrastructure and applications so that whole system is described declaratively and version controlled (most likely in a Git repository), and having an automated process that ensures that the deployed environment matches the state specified in a repository.
For more information, take a look at ["What is GitOps?"](https://www.gitops.tech/#what-is-gitops).
## Sources
A *Source* defines the origin of a source and the requirements to obtain
it (e.g. credentials, version selectors). For example, the latest `1.x` tag
available from a Git repository over SSH.
Sources produce an artifact that is consumed by other Flux elements to perform
actions, like applying the contents of the artifact on the cluster. A source
may be shared by multiple consumers to deduplicate configuration and/or storage.
The origin of the source is checked for changes on a defined interval, if
there is a newer version available that matches the criteria, a new artifact
is produced.
All sources are specified as Custom Resources in a Kubernetes cluster, examples
of sources are `GitRepository`, `HelmRepository` and `Bucket` resources.
For more information, take a look at [the source controller documentation](../components/source/source.md).
## Reconciliation
Reconciliation refers to ensuring that a given state (e.g application running in the cluster, infrastructure) matches a desired state declaratively defined somewhere (e.g a git repository). There are various examples of these in flux e.g:
- HelmRelease reconciliation: ensures the state of the Helm release matches what is defined in the resource, performs a release if this is not the case (including revision changes of a HelmChart resource).
- Bucket reconciliation: downloads and archives the contents of the declared bucket on a given interval and stores this as an artifact, records the observed revision of the artifact and the artifact itself in the status of resource.
- [Kustomization](#kustomization) reconciliation: ensures the state of the application deployed on a cluster matches resources contained in a git repository.
## Kustomization
The kustomization represents a local set of Kubernetes resources that Flux is supposed to reconcile in the cluster. The reconciliation runs every one minute by default but this can be specified in the kustomization. If you make any changes to the cluster using `kubectl edit` or `kubectl patch`, it will be promptly reverted. You either suspend the reconciliation or push your changes to a Git repository.
For more information, take a look at [this documentation](../components/kustomize/kustomization.md).
## Bootstrap
The process of installing the Flux components in a complete GitOps way is called a bootstrap. The manifests are applied to the cluster, a `GitRepository` and `Kustomization` are created for the Flux components, and the manifests are pushed to an existing Git repository (or a new one is created). Flux can manage itself just as it manages other resources.
The bootstrap is done using the `flux` CLI `flux bootstrap`.
For more information, take a look at [the documentation for the bootstrap command](../cmd/flux_bootstrap.md).
that can create repositories (check all permissions under `repo`).
that can create repositories (check all permissions under `repo`) to enable Flux do this.
Export your GitHub personal access token and username:
@ -51,24 +61,13 @@ To configure your shell to load `flux` [bash completions](../cmd/flux_completion
[`zsh`](../cmd/flux_completion_zsh.md), [`fish`](../cmd/flux_completion_fish.md), and [`powershell`](../cmd/flux_completion_powershell.md) are also supported with their own sub-commands.
## GitOps workflow
You'll be using a dedicated Git repository e.g. `fleet-infra` to manage one or more Kubernetes clusters.
This guide assumes that you have two clusters, one for staging and one for production.
Using the Flux CLI you'll do the following:
- configure each cluster to synchronise with a directory inside the fleet repository
- register app sources (git repositories) that contain plain Kubernetes manifests or Kustomize overlays
- configure app deployments on both clusters (pre-releases on staging, semver releases on production)
## Install Flux components
## Staging bootstrap
Create the staging cluster using Kubernetes kind or set the kubectl context to an existing cluster:
Create the cluster using Kubernetes kind or set the kubectl context to an existing cluster:
```sh
kind create cluster --name staging
kubectl cluster-info --context kind-staging
kind create cluster
kubectl cluster-info
```
Verify that your staging cluster satisfies the prerequisites with:
@ -88,17 +87,18 @@ flux bootstrap github \
--owner=$GITHUB_USER \
--repository=fleet-infra \
--branch=main \
--path=staging-cluster \
--path=./clusters/my-cluster \
--personal
```
!!! hint "ARM"
When deploying to a Kubernetes cluster with ARM architecture,
you can use `--arch=arm` for ARMv7 32-bit container images
and `--arch=arm64` for ARMv8 64-bit container images.
!!! hint "Multi-arch images"
The component images are published as [multi-arch container images](https://docs.docker.com/docker-for-mac/multi-arch/)
with support for Linux `amd64`, `arm64` and `armv7` (e.g. 32bit Raspberry Pi)
architectures.
The bootstrap command creates a repository if one doesn't exist, and
commits the manifests for the Flux components to the default branch at the specified path.
The bootstrap command creates a repository if one doesn't exist,
commits the manifests for the Flux components to the default branch at the specified path,
and installs the Flux components.
Then it configures the target cluster to synchronize with the specified path inside the repository.
If you wish to create the repository under a GitHub organization:
Please ensure that you enable workload identity for your cluster, create a GCP service account that has access to the container registry and create an IAM policy binding between the GCP service account and the Kubernetes service account so that the pods created by the cronjob can access GCP APIs and get the token.
Take a look at [this guide](https://cloud.google.com/kubernetes-engine/docs/how-to/workload-identity)
The access token for GCR expires hourly.
Considering this limitation, one needs to ensure the credentials are being
refreshed before expiration so that the controller can rely on them for
authentication.
The solution proposed is to create a cronjob that runs every 45 minutes which would
re-create the `docker-registry` secret using a new token.
Edit and save the following snippet to a file
`./clusters/my-cluster/gcr-sync.yaml`, commit and push it to git.
From [Google documentation on authenticating container registry](https://cloud.google.com/container-registry/docs/advanced-authentication#json-key)
> A user-managed key-pair that you can use as a credential for a service account. Because the credential is long-lived, it is the least secure option of all the available authentication methods. When possible, use an access token or another available authentication method to reduce the risk of unauthorized access to your artifacts. If you must use a service account key, ensure that you follow best practices for managing credentials.
Json keys doesn't expire so we don't need a cronjob, we just need to create the secret and reference it in the ImagePolicy.
First, create a json key file by following this [documentation](https://cloud.google.com/container-registry/docs/advanced-authentication). Grant the service account the role of `Container Registry Service Agent` so that it can access GCR and download the json file.
Then create a secret, encrypt it using [Mozilla SOPS](mozilla-sops.md) or [Sealed Secrets](sealed-secrets.md) , commit and push the encypted file to git.
The secret referenced in the provider is expected to contain a [personal access token](https://docs.github.com/en/github/authenticating-to-github/creating-a-personal-access-token)
to authenticate with the GitHub API.
By now the fleet-infra repository should have a similar directory structure.
```
fleet-infra
└── clusters/
└── my-cluster/
├── flux-system/
│ ├── gotk-components.yaml
│ ├── gotk-sync.yaml
│ └── kustomization.yaml
├── podinfo/
│ ├── namespace.yaml
│ ├── deployment.yaml
│ ├── hpa.yaml
│ ├── service.yaml
│ └── kustomization.yaml
├── podinfo-kustomization.yaml
└── podinfo-notification.yaml
```
If podinfo is deployed and the health checks pass you should get a successful status in
your forked podinfo repository.
If everything is setup correctly there should now be a green check-mark next to the latest commit.
Clicking the check-mark should show a detailed view.
| GitHub | GitLab |
| ------------- | ------------- |
| ![commit status GitHub successful](../_files/commit-status-github-success.png) | ![commit status GitLab successful](../_files/commit-status-gitlab-success.png) |
Generate error
A deployment failure can be forced by setting an invalid image tag in the podinfo deployment.
```yaml
apiVersion: v1
kind: Secret
metadata:
name: github
namespace: flux-system
data:
token: <token>
apiVersion: apps/v1
kind: Deployment
spec:
template:
spec:
containers:
- name: podinfod
image: ghcr.io/stefanprodan/podinfo:fake
```
After the commit has been reconciled it should return a failed commit status.
This is where the health check in the Kustomization comes into play together
with the timeout. The health check is used to asses the health of the Kustomization.
A failed commit status will not be sent until the health check timeout. Setting
a lower timeout will give feedback faster, but may sometimes not allow enough time
for a new application to deploy.
| GitHub | GitLab |
| ------------- | ------------- |
| ![commit status GitHub failure](../_files/commit-status-github-failure.png) | ![commit status GitLab failure](../_files/commit-status-gitlab-failure.png) |
### Status changes
The provider will continuously receive events as they happen, and multiple events may
be received for the same commit hash. The git providers are configured to only update
the status if the status has changed. This is to avoid spamming the commit status
history with the same status over and over again.
There is an aspect of state fullness that needs to be considered, compared to the other
notification providers, as the events are stored by the git provider. This means that
the status of a commit can change over time. Initially a deployment may be healthy, resulting
in a successful status. Down the line the application, and the health check, may start failing
due to the amount of traffic it receives or external dependencies no longer being available.
The change in the health check would cause the status to go from successful to failed.
It is important to keep this in mind when building any automation tools that deals with the
status, and consider the fact that receiving a successful status once does not mean it will
@ -84,20 +84,20 @@ Depending on what you want to do, some of the following bits might be your first
- And if you are completely new to Flux v2 and the GitOps Toolkit, take a look at our [Get Started guide](get-started/index.md) and give us feedback
- Check out [how to contribute](contributing/index.md) to the project
### Featured Talks
### Upcoming Events
- 25 Jan 2021 - [GitOps Core Concepts & How to Teach Your Teams with Leigh Capili](https://www.meetup.com/GitOps-Community/events/275625806/)
### Featured Talks
- 11 Jan 2021 - [Helm + GitOps = ⚡️⚡️⚡️ with Scott Rigby](https://youtu.be/YG8jMFrYQvM)
- 14 Dec 2020 - [The Power of GitOps with Flux and Flagger (GitOps Hands-On) with Leigh Capili](https://youtu.be/cB7iXeNLteE)
- 30 Nov 2020 - [The Power of GItOps with Flux 2 - Part 3 with Leigh Capili](https://youtu.be/N_K5g7o9JKg)
- 24 Nov 2020 - [Flux CD v2 with GitOps Toolkit - Kubernetes Deployment and Sync Mechanism](https://youtu.be/R6OeIgb7lUI)
- 19 Oct 2020 - [The Power of GitOps with Flux & GitOps Toolkit - Part 2 with Leigh Capili](https://youtu.be/fC2YCxQRUwU)
- 02 Nov 2020 - [The Power of GitOps with Flux & GitOps Toolkit - Part 2 with Leigh Capili](https://youtu.be/fC2YCxQRUwU)
- 28 Oct 2020 - [The Kubelist Podcast: Flux with Michael Bridgen](https://www.heavybit.com/library/podcasts/the-kubelist-podcast/ep-5-flux-with-michael-bridgen-of-weaveworks/)
- 19 Oct 2020 - [The Power of GitOps with Flux & GitOps Toolkit - Part 1 with Leigh Capili](https://youtu.be/0v5bjysXTL8)
- 12 Oct 2020 - [Rawkode Live: Introduction to GitOps Toolkit with Stefan Prodan](https://youtu.be/HqTzuOBP0eY)
- 4 Sep 2020 - [KubeCon Europe: The road to Flux v2 and Progressive Delivery with Stefan Prodan & Hidde Beydals](https://youtu.be/8v94nUkXsxU)
- 04 Sep 2020 - [KubeCon Europe: The road to Flux v2 and Progressive Delivery with Stefan Prodan & Hidde Beydals](https://youtu.be/8v94nUkXsxU)
- 25 June 2020 - [Cloud Native Nordics: Introduction to GitOps & GitOps Toolkit with Alexis Richardson & Stefan Prodan](https://youtu.be/qQBtSkgl7tI)
- 7 May 2020 - [GitOps Days - Community Special: GitOps Toolkit Experimentation with Stefan Prodan](https://youtu.be/WHzxunv4DKk?t=6521)
### Upcoming Events
- 12-13 Nov 2020 - [GitOps Days EMEA](https://www.gitopsdays.com/) with talks and workshops on migrating to Flux v2 and Helm Controller
- 19 Nov 2020 - [KubeCon NA: Progressive Delivery Techniques with Flagger and Flux v2 with Stefan Prodan](https://kccncna20.sched.com/event/1b04f8408b49976b843a5d0019cb8112)
- 07 May 2020 - [GitOps Days - Community Special: GitOps Toolkit Experimentation with Stefan Prodan](https://youtu.be/WHzxunv4DKk?t=6521)