You cannot select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

20 KiB

Raw Blame History

Installation

This guide walks you through setting up Flux v2 (hereafter: "Flux") to manage one or more Kubernetes clusters.

Prerequisites

You will need a Kubernetes cluster version 1.16 or newer and kubectl version 1.18 or newer.

Install the Flux CLI

With Homebrew:

brew install fluxcd/tap/flux

With Bash:

curl -s https://toolkit.fluxcd.io/install.sh | sudo bash

# enable completions in ~/.bash_profile
. <(flux completion bash)

Command-line completion for zsh, fish, and powershell are also supported with their own sub-commands.

Binaries for macOS, Windows and Linux AMD64/ARM are available for download on the release page.

Verify that your cluster satisfies the prerequisites with:

flux check --pre

Bootstrap

Using the flux bootstrap command you can install Flux on a Kubernetes cluster and configure it to manage itself from a Git repository.

The bootstrap creates a Git repository if one doesn't exist and commits the Flux components manifests to the main branch. Then it configures the target cluster to synchronize with that repository by setting up SSH deploy keys.

If the Flux components are present on the cluster, the bootstrap command will perform an upgrade if needed. The bootstrap is idempotent, it's safe to run the command as many times as you want.

You can choose what components to install and for which cluster with:

flux bootstrap <GIT-PROVIDER> \
  --components=source-controller,kustomize-controller,helm-controller,notification-controller \
  --components-extra=image-reflector-controller,image-automation-controller \
  --path=clusters/my-cluster

!!! hint "Multi-arch images" The component images are published as multi-arch container images with support for Linux amd64, arm64 and armv7 (e.g. 32bit Raspberry Pi) architectures.

If you wish to install a specific version, use the Flux release tag e.g. --version=v0.9.0.

If you wish to deploy the Flux components onto tainted Kubernetes nodes, you can specify the toleration keys with --toleration-keys=node.kubernetes.io/dedicated-to-flux.

With --path you can configure the directory which will be used to reconcile the target cluster. To control multiple clusters from the same Git repository, you have to set a unique path per cluster e.g. clusters/staging and clusters/production:

./clusters/
├── staging # <- path=clusters/staging
│   └── flux-system # <- namespace dir generated by bootstrap
│       ├── gotk-components.yaml
│       ├── gotk-sync.yaml
│       └── kustomization.yaml
└── production-cluster # <- path=clusters/production
    └── flux-system

After running bootstrap you can place Kubernetes YAMLs inside a dir under path e.g. clusters/staging/my-app, and Flux will reconcile them on your cluster.

!!! hint "Change the default branch" If you wish to change the branch to something else than main, create the repository manually, push a branch to origin and then use flux bootstrap <GIT-PROVIDER> --branch=your-branch.

For examples on how you can structure your Git repository see:

GitHub and GitHub Enterprise

Generate a personal access token that can create repositories by checking all permissions under repo.

Export your GitHub personal access token as an environment variable:

export GITHUB_TOKEN=<your-token>

Run the bootstrap for a repository on your personal GitHub account:

flux bootstrap github \
  --owner=my-github-username \
  --repository=my-repository \
  --path=clusters/my-cluster \
  --personal

!!! hint "Deploy key" The bootstrap command creates an SSH key which it stores as a secret in the Kubernetes cluster. The key is also used to create a deploy key in the GitHub repository. The new deploy key will be linked to the personal access token used to authenticate. Removing the personal access token will also remove the deploy key.

Run the bootstrap for a repository owned by a GitHub organization:

flux bootstrap github \
  --owner=my-github-organization \
  --repository=my-repository \
  --team=team1-slug \
  --team=team2-slug \
  --path=clusters/my-cluster

When you specify a list of teams, those teams will be granted maintainer access to the repository.

To run the bootstrap for a repository hosted on GitHub Enterprise, you have to specify your GitHub hostname:

flux bootstrap github \
  --hostname=my-github-enterprise.com \
  --ssh-hostname=my-github-enterprise.com \
  --owner=my-github-organization \
  --repository=my-repository \
  --branch=main \
  --path=clusters/my-cluster

If your GitHub Enterprise has SSH access disabled, you can use HTTPS and token authentication with:

flux bootstrap github \
  --token-auth \
  --hostname=my-github-enterprise.com \
  --owner=my-github-organization \
  --repository=my-repository \
  --branch=main \
  --path=clusters/my-cluster

GitLab and GitLab Enterprise

Generate a personal access token that grants complete read/write access to the GitLab API.

Export your GitLab personal access token as an environment variable:

export GITLAB_TOKEN=<your-token>

Run the bootstrap for a repository on your personal GitLab account:

flux bootstrap gitlab \
  --owner=my-gitlab-username \
  --repository=my-repository \
  --branch=master \
  --path=clusters/my-cluster \
  --token-auth \
  --personal

To run the bootstrap for a repository using deploy keys for authentication, you have to specify the SSH hostname:

flux bootstrap gitlab \
  --ssh-hostname=gitlab.com \
  --owner=my-gitlab-username \
  --repository=my-repository \
  --branch=master \
  --path=clusters/my-cluster

!!! hint "Authentication" When providing the --ssh-hostname, a read-only (SSH) deploy key will be added to your repository, otherwise your GitLab personal token will be used to authenticate against the HTTPS endpoint instead.

Run the bootstrap for a repository owned by a GitLab (sub)group:

flux bootstrap gitlab \
  --owner=my-gitlab-group/my-gitlab-subgroup \
  --repository=my-repository \
  --branch=master \
  --path=clusters/my-cluster

To run the bootstrap for a repository hosted on GitLab on-prem or enterprise, you have to specify your GitLab hostname:

flux bootstrap gitlab \
  --hostname=my-gitlab.com \
  --token-auth \
  --owner=my-gitlab-group \
  --repository=my-repository \
  --branch=master \
  --path=clusters/my-cluster

Air-gapped Environments

To bootstrap Flux on air-gapped environments without access to github.com and ghcr.io, first you'll need to download the flux binary, and the container images from a computer with access to internet.

List all container images:

$ flux install --export | grep ghcr.io

image: ghcr.io/fluxcd/helm-controller:v0.8.0
image: ghcr.io/fluxcd/kustomize-controller:v0.9.0
image: ghcr.io/fluxcd/notification-controller:v0.9.0
image: ghcr.io/fluxcd/source-controller:v0.9.0

Pull the images locally and push them to your container registry:

docker pull ghcr.io/fluxcd/source-controller:v0.9.0
docker tag ghcr.io/fluxcd/source-controller:v0.9.0 registry.internal/fluxcd/source-controller:v0.9.0
docker push registry.internal/fluxcd/source-controller:v0.9.0

Copy flux binary to a computer with access to your air-gapped cluster, and create the pull secret in the flux-system namespace:

kubectl create ns flux-system

kubectl -n flux-system create secret generic regcred \
    --from-file=.dockerconfigjson=/.docker/config.json \
    --type=kubernetes.io/dockerconfigjson

Finally, bootstrap Flux using the images from your private registry:

flux bootstrap <GIT-PROVIDER> \
  --registry=registry.internal/fluxcd \
  --image-pull-secret=regcred \
  --hostname=my-git-server.internal

Note that when running flux bootstrap without specifying a --version, the CLI will use the manifests embedded in its binary instead of downloading them from GitHub. You can determine which version you'll be installing, with flux --version.

Generic Git Server

For other Git providers such as Bitbucket, Gogs, Gitea, Azure DevOps, etc you can manually setup the repository and deploy key.

Create a Git repository and clone it locally:

git clone ssh://<host>/<org>/my-repository
cd my-repository

Create a directory inside the repository:

mkdir -p ./clusters/my-cluster/flux-system

Generate the Flux manifests with:

flux install \
  --export > ./clusters/my-cluster/flux-system/gotk-components.yaml

Commit and push the manifest to the master branch:

git add -A && git commit -m "add components" && git push

Apply the manifests on your cluster:

kubectl apply -f ./clusters/my-cluster/flux-system/gotk-components.yaml

Verify that the controllers have started:

flux check

Create a GitRepository object on your cluster by specifying the SSH address of your repo:

flux create source git flux-system \
  --url=ssh://git@<host>/<org>/<repository> \
  --ssh-key-algorithm=ecdsa \
  --ssh-ecdsa-curve=p521 \
  --branch=master \
  --interval=1m

You will be prompted to add a deploy key to your repository. If you don't specify the SSH algorithm, then flux will generate an RSA 2048 bits key.

!!! hint "Azure DevOps" Azure DevOps requires a non-default Git implementation (libgit2) to be enabled, so that the Git v2 protocol is supported. Note that this implementation does not support shallow cloning, and it is therefore advised to only resort to this option if a connection fails with the default configuration.

Azure DevOps [only supports RSA SSH keys](https://developercommunity.visualstudio.com/t/support-non-rsa-keys-for-ssh-authentication/365980),
you cannot use elliptic curve SSH keys like ecdsa or ed25519.

Here is how to specify the `libgit2` implementation and generate a proper RSA key:

```sh
flux create source git flux-system \
  --git-implementation=libgit2 \
  --ssh-key-algorithm=rsa \
  --ssh-rsa-bits=4096 \
  --url=ssh://git@ssh.dev.azure.com/v3/<org>/<project>/<repository> \
  --branch=main \
  --interval=1m
```

This config uses the `main` branch, but your repo may be older and need to specify `master` instead.

Note that unlike `git`, Flux does not support the
["shorter" scp-like syntax for the SSH protocol](https://git-scm.com/book/en/v2/Git-on-the-Server-The-Protocols#_the_ssh_protocol)
(e.g. `ssh.dev.azure.com:v3`).
Use the [RFC 3986 compatible syntax](https://tools.ietf.org/html/rfc3986#section-3) instead: `ssh.dev.azure.com/v3`.

The `flux create source git` command will prompt you to add a deploy key to your repository, but Azure DevOps
[does not support repository or org-specific deploy keys](https://developercommunity.visualstudio.com/t/allow-the-creation-of-ssh-deploy-keys-for-vsts-hos/365747).
You may add the deploy key to a user's personal SSH keys being mindful that removing them from the repo may revoke Flux's access.
As an alternative, create a machine-user whose sole purpose is to store credentials for automation.
Using a machine-user also has the benefit of being able to be read-only or restricted to specific repositories if that is needed.

If you wish to use Git over HTTPS, then generate a personal access token and supply it as the password:

```sh
flux create source git flux-system \
  --git-implementation=libgit2 \
  --url=https://dev.azure.com/<org>/<project>/_git/<repository> \
  --branch=master \
  --username=git \
  --password=${AZ_PAT_TOKEN} \
  --interval=1m
```

Please consult the [Azure DevOps documentation](https://docs.microsoft.com/en-us/azure/devops/organizations/accounts/use-personal-access-tokens-to-authenticate?view=azure-devops&tabs=preview-page)
on how to generate personal access tokens for Git repositories.
Azure DevOps PAT's always have an expiration date, so be sure to have some process for renewing or updating these tokens.
Similar to the lack of repo-specific deploy keys, a user needs to generate a user-specific PAT.
If you are using a machine-user, you can generate a PAT or simply use the machine-user's password which does not expire.

If your Git server supports basic auth, you can set the URL to HTTPS and specify the credentials with:

flux create source git flux-system \
  --url=https://<host>/<org>/my-repository \
  --username=my-username \
  --password=my-password \
  --branch=master \
  --interval=1m

Create a Kustomization object on your cluster:

flux create kustomization flux-system \
  --source=flux-system \
  --path="./clusters/my-cluster" \
  --prune=true \
  --interval=10m

Export both objects, generate a kustomization.yaml, commit and push the manifests to Git:

flux export source git flux-system \
  > ./clusters/my-cluster/flux-system/gotk-sync.yaml

flux export kustomization flux-system \
  >> ./clusters/my-cluster/flux-system/gotk-sync.yaml

cd ./clusters/my-cluster/flux-system && kustomize create --autodetect

git add -A && git commit -m "add sync manifests" && git push

To upgrade the Flux components to a newer version, download the latest flux binary, run the install command and commit the changes:

flux install \
  --export > ./clusters/my-cluster/flux-system/gotk-components.yaml

git add -A && git commit -m "update flux" && git push

The source-controller will pull the changes on the cluster, then the kustomize-controller will perform a rolling update of all Flux components including itself.

Bootstrap with Terraform

The bootstrap procedure can be implemented with Terraform using the Flux provider published on registry.terraform.io.

The provider consists of two data sources (flux_install and flux_sync) for generating the Kubernetes manifests that can be used to install or upgrade Flux:

data "flux_install" "main" {
  target_path    = "clusters/my-cluster"
  network_policy = false
  version        = "latest"
}

data "flux_sync" "main" {
  target_path = "clusters/my-cluster"
  url         = "https://github.com/${var.github_owner}/${var.repository_name}"
  branch      = "main"
}

For more details on how to use the Terraform provider please see fluxcd/terraform-provider-flux.

Customize Flux manifests

You can customize the Flux components in the Git repository where you've run bootstrap with Kustomize patches.

First clone the repository locally and generate a kustomization.yaml file with:

cd ./clusters/production && kustomize create --autodetect

Assuming you want to add custom annotations and labels to the Flux controllers in clusters/production. Create a Kustomize patch and set the metadata for source-controller and kustomize-controller pods:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: source-controller
  namespace: flux-system
spec:
  template:
    metadata:
      annotations:
        custom: annotation
      labels:
        custom: label
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: kustomize-controller
  namespace: flux-system
spec:
  template:
    metadata:
      annotations:
        custom: annotation
      labels:
        custom: label

Save the above file as flux-system-patch.yaml inside the clusters/production dir.

Edit clusters/production/kustomization.yaml and add the patch:

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
  - flux-system
patchesStrategicMerge:
  - flux-system-patch.yaml

Push the changes to main branch:

git add -A && git commit -m "add production metadata" && git push

Flux will detect the change and will update itself on the production cluster.

Dev install

For testing purposes you can install Flux without storing its manifests in a Git repository:

flux install

Or using kubectl:

kubectl apply -f https://github.com/fluxcd/flux2/releases/latest/download/install.yaml

Then you can register Git repositories and reconcile them on your cluster:

flux create source git podinfo \
  --url=https://github.com/stefanprodan/podinfo \
  --tag-semver=">=4.0.0" \
  --interval=1m

flux create kustomization podinfo-default \
  --source=podinfo \
  --path="./kustomize" \
  --prune=true \
  --validation=client \
  --interval=10m \
  --health-check="Deployment/podinfo.default" \
  --health-check-timeout=2m

You can register Helm repositories and create Helm releases:

flux create source helm bitnami \
  --interval=1h \
  --url=https://charts.bitnami.com/bitnami

flux create helmrelease nginx \
  --interval=1h \
  --release-name=nginx-ingress-controller \
  --target-namespace=kube-system \
  --source=HelmRepository/bitnami \
  --chart=nginx-ingress-controller \
  --chart-version="5.x.x"

Upgrade

!!! note "Patch versions" It is safe and advised to use the latest PATCH version when upgrading to a new MINOR version.

Update Flux CLI to the latest release with brew upgrade fluxcd/tap/flux or by downloading the binary from GitHub.

Verify that you are running the latest version with:

flux --version

Bootstrap upgrade

If you've used the bootstrap procedure to deploy Flux, then rerun the bootstrap command for each cluster using the same arguments as before:

flux bootstrap github \
  --owner=my-github-username \
  --repository=my-repository \
  --branch=main \
  --path=clusters/my-cluster \
  --personal

The above command will clone the repository, it will update the components manifest in <path>/flux-system/gotk-components.yaml and it will push the changes to the remote branch.

Tell Flux to pull the manifests from Git and upgrade itself with:

flux reconcile source git flux-system

Verify that the controllers have been upgrade with:

flux check

!!! hint "Automated upgrades" You can automate the components manifest update with GitHub Actions and open a PR when there is a new Flux version available. For more details please see Flux GitHub Action docs.

Terraform upgrade

Update the Flux provider to the latest release and run terraform apply.

Tell Flux to upgrade itself in-cluster or wait for it to pull the latest commit from Git:

kubectl annotate --overwrite gitrepository/flux-system reconcile.fluxcd.io/requestedAt="$(date +%s)"

In-cluster upgrade

If you've installed Flux directly on the cluster, then rerun the install command:

flux install

The above command will apply the new manifests on your cluster. You can verify that the controllers have been upgraded to the latest version with flux check.

If you've installed Flux directly on the cluster with kubectl, then rerun the command using the latest manifests from the main branch:

kustomize build https://github.com/fluxcd/flux2/manifests/install?ref=main | kubectl apply -f-

Uninstall

You can uninstall Flux with:

flux uninstall --namespace=flux-system

The above command performs the following operations:

deletes Flux components (deployments and services)
deletes Flux network policies
deletes Flux RBAC (service accounts, cluster roles and cluster role bindings)
removes the Kubernetes finalizers from Flux custom resources
deletes Flux custom resource definitions and custom resources
deletes the namespace where Flux was installed

If you've installed Flux in a namespace that you wish to preserve, you can skip the namespace deletion with:

flux uninstall --namespace=infra --keep-namespace

!!! hint Note that the uninstall command will not remove any Kubernetes objects or Helm releases that were reconciled on the cluster by Flux.

20 KiB Raw Blame History