diff --git a/docs/guides/image-update.md b/docs/guides/image-update.md index ae347f6e..bd8cffff 100644 --- a/docs/guides/image-update.md +++ b/docs/guides/image-update.md @@ -857,11 +857,11 @@ so that it can access GCR and download the json file. Then create a secret, encrypt it using [Mozilla SOPS](mozilla-sops.md) or [Sealed Secrets](sealed-secrets.md) , commit and push the encypted file to git. -``` - kubectl create secret docker-registry \ - --docker-server= \ # e.g gcr.io - --docker-username=_json_key \ - --docker-password="$(cat )" +```sh +kubectl create secret docker-registry \ + --docker-server= \ # e.g gcr.io + --docker-username=_json_key \ + --docker-password="$(cat )" ``` ### Azure Container Registry @@ -874,15 +874,7 @@ Note that the resulting ImagePullSecret for Flux could also be specified by Pods #### Generating Tokens for Managed Identities [short-lived] -With [AAD Pod-Identity](https://azure.github.io/aad-pod-identity/docs/), we can create Pods that have their own -cloud credentials for accessing Azure services like ACR. - -Your cluster should have `--enable-managed-identity` configured. -This software can be [installed via Helm](https://azure.github.io/aad-pod-identity/docs/getting-started/installation/) not managed by Azure. -Use Flux's `HelmRepository` and `HelmRelease` object to manage the aad-pod-identity installation from a bootstrap repository. - -!!! As an alternative to Helm, the `--enable-aad-pod-identity` flag for the `az aks create` is currently in Preview. - Follow the Azure guide for [Creating an AKS cluster with AAD Pod Identity](https://docs.microsoft.com/en-us/azure/aks/use-azure-ad-pod-identity) if you would like to enable this feature with the Azure CLI. +As a pre-requisite, your AKS cluster will need [AAD Pod Identity](../use-cases/azure.md#aad-pod-identity) installed. Once we have AAD Pod Identity installed, we can create a Deployment that frequently refreshes an image pull secret into our desired Namespace. @@ -937,7 +929,8 @@ folder into your own repository or use a git submodule to vendor it if preferred #### Using Static Credentials [long-lived] -!!! Using a static credential requires a Secrets management solution compatible with your GitOps workflow. +!!! info + Using a static credential requires a Secrets management solution compatible with your GitOps workflow. Follow the official Azure documentation for [Creating an Image Pull Secret for ACR](https://docs.microsoft.com/en-us/azure/container-registry/container-registry-auth-kubernetes). @@ -949,4 +942,5 @@ Update the `ImageRepository.spec.secretRef` to point to it. It is also possible to create [Repository Scoped Tokens](https://docs.microsoft.com/en-us/azure/container-registry/container-registry-repository-scoped-permissions). -!!! Note that this feature is in preview and does have limitations. +!!! warning + Repository Scoped Tokens are in preview and do have limitations. diff --git a/docs/guides/installation.md b/docs/guides/installation.md index 82df44e8..99cf53ba 100644 --- a/docs/guides/installation.md +++ b/docs/guides/installation.md @@ -336,22 +336,35 @@ If you don't specify the SSH algorithm, then `flux` will generate an RSA 2048 bi Note that this implementation does not support shallow cloning, and it is therefore advised to only resort to this option if a connection fails with the default configuration. - If you are using Azure DevOps you need to specify a different Git implementation than the default: + Azure DevOps [only supports RSA SSH keys](https://developercommunity.visualstudio.com/t/support-non-rsa-keys-for-ssh-authentication/365980), + you cannot use elliptic curve SSH keys like ecdsa or ed25519. + + Here is how to specify the `libgit2` implementation and generate a proper RSA key: ```sh flux create source git flux-system \ --git-implementation=libgit2 \ + --ssh-key-algorithm=rsa \ + --ssh-rsa-bits=4096 \ --url=ssh://git@ssh.dev.azure.com/v3/// \ - --branch=master \ + --branch=main \ --interval=1m ``` + This config uses the `main` branch, but your repo may be older and need to specify `master` instead. + Note that unlike `git`, Flux does not support the ["shorter" scp-like syntax for the SSH protocol](https://git-scm.com/book/en/v2/Git-on-the-Server-The-Protocols#_the_ssh_protocol) (e.g. `ssh.dev.azure.com:v3`). Use the [RFC 3986 compatible syntax](https://tools.ietf.org/html/rfc3986#section-3) instead: `ssh.dev.azure.com/v3`. - If you wish to use Git over HTTPS, then generated a personal access token and supply it as the password: + The `flux create source git` command will prompt you to add a deploy key to your repository, but Azure DevOps + [does not support repository or org-specific deploy keys](https://developercommunity.visualstudio.com/t/allow-the-creation-of-ssh-deploy-keys-for-vsts-hos/365747). + You may add the deploy key to a user's personal SSH keys being mindful that removing them from the repo may revoke Flux's access. + As an alternative, create a machine-user whose sole purpose is to store credentials for automation. + Using a machine-user also has the benefit of being able to be read-only or restricted to specific repositories if that is needed. + + If you wish to use Git over HTTPS, then generate a personal access token and supply it as the password: ```sh flux create source git flux-system \ @@ -359,12 +372,15 @@ If you don't specify the SSH algorithm, then `flux` will generate an RSA 2048 bi --url=https://dev.azure.com///_git/ \ --branch=master \ --username=git \ - --password=token \ + --password=${AZ_PAT_TOKEN} \ --interval=1m ``` Please consult the [Azure DevOps documentation](https://docs.microsoft.com/en-us/azure/devops/organizations/accounts/use-personal-access-tokens-to-authenticate?view=azure-devops&tabs=preview-page) on how to generate personal access tokens for Git repositories. + Azure DevOps PAT's always have an expiration date, so be sure to have some process for renewing or updating these tokens. + Similar to the lack of repo-specific deploy keys, a user needs to generate a user-specific PAT. + If you are using a machine-user, you can generate a PAT or simply use the machine-user's password which does not expire. If your Git server supports basic auth, you can set the URL to HTTPS and specify the credentials with: diff --git a/docs/guides/mozilla-sops.md b/docs/guides/mozilla-sops.md index c39fec0d..7d2f6499 100644 --- a/docs/guides/mozilla-sops.md +++ b/docs/guides/mozilla-sops.md @@ -213,9 +213,92 @@ IAM Role example: #### Azure -When using Azure Key Vault you need to authenticate the kustomize controller either by passing -[Service Principal credentials as environment variables](https://github.com/mozilla/sops#encrypting-using-azure-key-vault) -or with [add-pod-identity](https://github.com/Azure/aad-pod-identity). +When using Azure Key Vault you need to authenticate kustomize-controller either with [add-pod-identity](../use-cases/azure.md#aad-pod-identity) +or by passing [Service Principal credentials as environment variables](https://github.com/mozilla/sops#encrypting-using-azure-key-vault). + +Create the Azure Key-Vault: + +```sh +export VAULT_NAME="fluxcd-$(uuidgen | tr -d - | head -c 16)" +export KEY_NAME="sops-cluster0" + +az keyvault create --name "${VAULT_NAME}" +az keyvault key create --name "${KEY_NAME}" \ + --vault-name "${VAULT_NAME}" + --protection software \ + --ops encrypt decrypt +az keyvault key show --name "${KEY_NAME}" \ + --vault-name "${VAULT_NAME}" \ + --query key.kid +``` + +If using AAD Pod-Identity, create an identity within Azure to bind against, then create an `AzureIdentity` object to match: + +```yaml +# Create an identity in Azure and assign it a role to access Key Vault (note: the identity's resourceGroup should match the desired Key Vault): +# az identity create -n sops-akv-decryptor +# az role assignment create --role "Key Vault Crypto User" --assignee-object-id "$(az identity show -n sops-akv-decryptor -o tsv --query principalId)" +# Fetch the clientID and resourceID to configure the AzureIdentity spec below: +# az identity show -n sops-akv-decryptor -otsv --query clientId +# az identity show -n sops-akv-decryptor -otsv --query resourceId +--- +apiVersion: aadpodidentity.k8s.io/v1 +kind: AzureIdentity +metadata: + name: sops-akv-decryptor # kustomize-controller label will match this name + namespace: flux-system +spec: + clientID: 58027844-6b86-424b-9888-b5ae2dc28b4f + resourceID: /subscriptions/8c69185e-55f9-4d00-8e71-a1b1bb1386a1/resourcegroups/stealthybox/providers/Microsoft.ManagedIdentity/userAssignedIdentities/sops-akv-decryptor + type: 0 # user-managed identity +``` + +[Customize your Flux Manifests](../guides/installation.md#customize-flux-manifests) so that kustomize-controller has the proper credentials. +Patch the kustomize-controller Pod template so that the label matches the `AzureIdentity` name. +Additionally, the SOPS specific environment variable `AZURE_AUTH_METHOD=msi` to activate the proper auth method within kustomize-controller. + +```yaml +apiVersion: apps/v1 +kind: Deployment +metadata: + name: kustomize-controller + namespace: flux-system +spec: + template: + metadata: + labels: + aadpodidbinding: sops-akv-decryptor # match the AzureIdentity name + spec: + containers: + - name: manager + env: + - name: AZURE_AUTH_METHOD + value: msi +``` + +Alternatively, if using a Service Principal stored in a K8s Secret, patch the Pod's envFrom to reference the `AZURE_TENANT_ID`/`AZURE_CLIENT_ID`/`AZURE_CLIENT_SECRET` +fields from your Secret. + +```yaml +apiVersion: apps/v1 +kind: Deployment +metadata: + name: kustomize-controller + namespace: flux-system +spec: + template: + spec: + containers: + - name: manager + envFrom: + - secretRef: + name: sops-akv-decryptor-service-principal +``` + +At this point, kustomize-controller is now authorized to decrypt values in SOPS encrypted files from your Sources via the related Key Vault. + +See Mozilla's guide to [Encrypting Using Azure Key Vault](https://github.com/mozilla/sops#encrypting-using-azure-key-vault) to get started +committing encrypted files to your Git Repository or other Sources. #### Google Cloud diff --git a/docs/index.md b/docs/index.md index 18770995..f4921f80 100644 --- a/docs/index.md +++ b/docs/index.md @@ -48,7 +48,7 @@ the API. ## Where do I start? -!!!hint "Get started with Flux v2!" +!!! hint "Get started with Flux v2!" Following this [guide](get-started/index.md) will just take a couple of minutes to complete: After installing the `flux` CLI and running a couple of very simple commands, you will have a GitOps workflow setup which involves a staging and a production cluster. diff --git a/docs/use-cases/azure.md b/docs/use-cases/azure.md new file mode 100644 index 00000000..b1791102 --- /dev/null +++ b/docs/use-cases/azure.md @@ -0,0 +1,112 @@ +# Using Flux on Azure + +## AKS Cluster Options + +It's important to follow some guidelines when installing Flux on AKS. + +### CNI and Network Policy + +Previously, there has been an issue with Flux and Network Policy on AKS. ([Upstream Azure Issue](https://github.com/Azure/AKS/issues/2031)) ([Flux Issue](https://github.com/fluxcd/flux2/issues/703)) +If you ensure your AKS cluster is upgraded, and your Nodes have been restarted with the most recent Node images, this could +resolve flux reconciliation failures where source-controller is unreachable. +Using `--network-plugin=azure --network-policy=calico` has been tested to work properly. +This issue only affects you if you are using `--network-policy` on AKS, which is not a default option. + +!!! warning + AKS `--network-policy` is currently in Preview + +### AAD Pod-Identity + +Depending on the features you are interested in using with Flux, you may want to install AAD Pod Identity. +With [AAD Pod-Identity](https://azure.github.io/aad-pod-identity/docs/), we can create Pods that have their own +cloud credentials for accessing Azure services like Azure Container Registry(ACR) and Azure Key Vault(AKV). + +If you do not use AAD Pod-Identity, you'll need to manage and store Service Principal credentials in K8s Secrets, to integrate Flux +with other Azure Services. + +As a pre-requisite, your cluster must have `--enable-managed-identity` configured. + +This software can be [installed via Helm](https://azure.github.io/aad-pod-identity/docs/getting-started/installation/) (unmanaged by Azure). +Use Flux's `HelmRepository` and `HelmRelease` object to manage the aad-pod-identity installation from a bootstrap repository and keep it up to date. + +!!! note + As an alternative to Helm, the `--enable-aad-pod-identity` flag for the `az aks create` is currently in Preview. + Follow the Azure guide for [Creating an AKS cluster with AAD Pod Identity](https://docs.microsoft.com/en-us/azure/aks/use-azure-ad-pod-identity) if you would like to enable this feature with the Azure CLI. + +### Cluster Creation + +!!! info + When working with the Azure CLI, it can help to set a default `location`, `group`, and `acr`. + See `az configure --help`, `az configure --list-defaults`, and `az configure --defaults key=value` + +The following creates an AKS cluster with some minimal configuration that will work well with Flux: + +```sh +az aks create \ + --network-plugin="azure" \ + --network-policy="calico" \ + --enable-managed-identity \ + --enable-pod-identity \ + --name="my-cluster" +``` + +## Flux Installation with Azure DevOps Repos + +Ensure you can login to [https://dev.azure.com](dev.azure.com) for your proper organization, and create a new repo to hold your +flux install and other necessary config. + +There is no bootstrap provider currently for Azure DevOps Repos, +but you can clone your Azure Repo, then use the [Generic Git Server](../guides/installation.md#generic-git-server) +guide to manually bootstrap Flux. (It must be a Git repo; TFVC Repos are not supported by source-controller) +Take note of the Azure DevOps specific section within the guide. + +If you use the generated SSH deploy key from `flux create source git`, ensure it is an RSA key (not an elliptic curve). +Make sure to use the `libgit2` provider for all `GitRepository` objects fetching from Azure Repos since they use Git Protocol v2. + +Whether you're using the generated SSH deploy key or a Personal Access Token, the credentials used by +Flux will need to be owned by an Azure DevOps User with access to the repo. +Consider creating a machine-user and granting it granular permissions to access what's needed. +This allows changing user access without affecting Flux. +Since PAT's expire on Azure DevOps, using a machine-user's login password to authenticate with HTTPS and `libgit2` +can be a good option that avoids the need to renew the credential while also having the benefit of more granular permissions. + +## Helm Repositories on Azure Container Registry + +The Flux `HelmRepository` object currently supports [Chart Repositories](https://helm.sh/docs/topics/chart_repository/) +as well as fetching `HelmCharts` from paths in `GitRepository` sources. + +Azure Container Registry has a sub-command ([`az acr helm`](https://docs.microsoft.com/en-us/cli/azure/acr/helm)) for working with +ACR-Hosted Chart Repositories, but it is deprecated. +If you are using these deprecated Azure Chart Repositories, you can use Flux `HelmRepository` objects with them. + +[Newer ACR Helm documentation](https://docs.microsoft.com/en-us/azure/container-registry/container-registry-helm-repos) suggests +using ACR as an experimental [Helm OCI Registry](https://helm.sh/docs/topics/registries/). +This will not work with Flux, because using Charts from OCI Registries is not yet supported. + +## Secrets Management with SOPS and Azure Key Vault + +You will need to create an Azure Key Vault and bind a credential such as a Service Principal or Managed Identity to it. +If you want to use Managed Identities, install or enable [AAD Pod Identity](#aad-pod-identity). + +Patch kustomize-controller with the proper Azure credentials, so that it may access your Azure Key Vault, and then begin +committing SOPS encrypted files to the Git repository with the proper Azure Key Vault configuration. + +See the [Mozilla SOPS Azure Guide](../guides/azure.md) for further detail. + +## Image Updates with Azure Container Registry + +You will need to create an ACR registry and bind a credential such as a Service Principal or Managed Identity to it. +If you want to use Managed Identities, install or enable [AAD Pod Identity](#aad-pod-identity). + +You may need to update your Flux install to include additional components: +```sh +flux install \ + --components-extra="image-reflector-controller,image-automation-controller" \ + --export > ./clusters/my-cluster/flux-system/gotk-components.yaml +``` + +Follow the [Image Update Automation Guide](../guides/image-update.md) and see the +[ACR specific section](../guides/image-update.md#azure-container-registry) for more details. + +Your AKS cluster's configuration can also be updated to [allow the kubelets to pull images from ACR](https://docs.microsoft.com/en-us/azure/aks/cluster-container-registry-integration) +without ImagePullSecrets as an optional, complimentary step. diff --git a/mkdocs.yml b/mkdocs.yml index d9714ddb..feab9507 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -64,6 +64,8 @@ nav: - Mozilla SOPS: guides/mozilla-sops.md - Automate image updates to Git: guides/image-update.md - Sortable image tags to use with automation: guides/sortable-image-tags.md + - Use Cases: + - Azure: use-cases/azure.md - Toolkit Components: - Overview: components/index.md - Source Controller: