Merge pull request #4082 from fluxcd/backport-4077-to-release/v2.0.x

[release/v2.0.x] build(deps): bump the ci group with 2 updates

.github/workflows/action.yaml

@@ -1,29 +0,0 @@
-name: test-gh-action
-
-on:
-  pull_request:
-    paths:
-      - 'action/**'
-  push:
-    paths:
-      - 'action/**'
-    branches:
-      - 'main'
-      - 'release/**'
-
-permissions: read-all
-
-jobs:
-  actions:
-    strategy:
-      fail-fast: false
-      matrix:
-        version: [ubuntu-latest, macos-latest, windows-latest]
-
-    runs-on: ${{ matrix.version }}
-    name: action on ${{ matrix.version }}
-    steps:
-      - name: Checkout
-        uses: actions/checkout@c85c95e3d7251135ab7dc9ce3241c5835cc595a9 # v3.5.3
-      - name: Setup flux
-        uses: ./action

action/README.md

@@ -1,22 +1,216 @@
 # Flux GitHub Action
 
-To install the latest Flux CLI on Linux, macOS or Windows GitHub runners:
+Usage:
 
 ```yaml
-steps:
-  - name: Setup Flux CLI
-    uses: fluxcd/flux2/action@main
-    with:
-      version: 'latest'
-  - name: Run Flux CLI
-    run: flux version --client
+    steps:
+      - name: Setup Flux CLI
+        uses: fluxcd/flux2/action@main
+      - name: Run Flux commands
+        run: flux -v
 ```
 
-The Flux GitHub Action can be used to automate various tasks in CI, such as:
+The latest stable version of the `flux` binary is downloaded from
+GitHub [releases](https://github.com/fluxcd/flux2/releases)
+and placed at `/usr/local/bin/flux`.
 
-- [Automate Flux upgrades on clusters via Pull Requests](https://fluxcd.io/flux/flux-gh-action/#automate-flux-updates)
-- [Push Kubernetes manifests to container registries](https://fluxcd.io/flux/flux-gh-action/#push-kubernetes-manifests-to-container-registries)
-- [Run end-to-end testing with Flux and Kubernetes Kind](https://fluxcd.io/flux/flux-gh-action/#end-to-end-testing)
+Note that this action can only be used on GitHub **Linux** runners.
+You can change the arch (defaults to `amd64`) with:
 
-For more information, please see the [Flux GitHub Action documentation](/flux/flux-gh-action.md).
+```yaml
+    steps:
+      - name: Setup Flux CLI
+        uses: fluxcd/flux2/action@main
+        with:
+          arch: arm64 # can be amd64, arm64 or arm
+```
 
+You can download a specific version with:
+
+```yaml
+    steps:
+      - name: Setup Flux CLI
+        uses: fluxcd/flux2/action@main
+        with:
+          version: 0.32.0
+```
+
+You can also authenticate against the GitHub API using GitHub Actions' `GITHUB_TOKEN` secret.
+
+For more information, please [read about the GitHub token secret](https://docs.github.com/en/actions/security-guides/automatic-token-authentication#about-the-github_token-secret).
+
+```yaml
+    steps:
+      - name: Setup Flux CLI
+        uses: fluxcd/flux2/action@main
+        with:
+          token: ${{ secrets.GITHUB_TOKEN }}
+```
+
+This is useful if you are seeing failures on shared runners, those failures are usually API limits being hit.
+
+### Automate Flux updates
+
+Example workflow for updating Flux's components generated with `flux bootstrap --path=clusters/production`:
+
+```yaml
+name: update-flux
+
+on:
+  workflow_dispatch:
+  schedule:
+    - cron: "0 * * * *"
+
+permissions:
+  contents: write
+  pull-requests: write
+
+jobs:
+  components:
+    runs-on: ubuntu-latest
+    steps:
+      - name: Check out code
+        uses: actions/checkout@v3
+      - name: Setup Flux CLI
+        uses: fluxcd/flux2/action@main
+      - name: Check for updates
+        id: update
+        run: |
+          flux install \
+            --export > ./clusters/production/flux-system/gotk-components.yaml
+
+          VERSION="$(flux -v)"
+          echo "flux_version=$VERSION" >> $GITHUB_OUTPUT
+      - name: Create Pull Request
+        uses: peter-evans/create-pull-request@v4
+        with:
+            token: ${{ secrets.GITHUB_TOKEN }}
+            branch: update-flux
+            commit-message: Update to ${{ steps.update.outputs.flux_version }}
+            title: Update to ${{ steps.update.outputs.flux_version }}
+            body: |
+              ${{ steps.update.outputs.flux_version }}
+```
+
+### Push Kubernetes manifests to container registries
+
+Example workflow for publishing Kubernetes manifests bundled as OCI artifacts to GitHub Container Registry:
+
+```yaml
+name: push-artifact-staging
+
+on:
+  push:
+    branches:
+      - 'main'
+
+permissions:
+  packages: write # needed for ghcr.io access
+
+env:
+  OCI_REPO: "oci://ghcr.io/my-org/manifests/${{ github.event.repository.name }}"
+
+jobs:
+  kubernetes:
+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout
+        uses: actions/checkout@v3
+      - name: Setup Flux CLI
+        uses: fluxcd/flux2/action@main
+      - name: Login to GHCR
+        uses: docker/login-action@v2
+        with:
+          registry: ghcr.io
+          username: ${{ github.actor }}
+          password: ${{ secrets.GITHUB_TOKEN }}
+      - name: Generate manifests
+        run: |
+          kustomize build ./manifests/staging > ./deploy/app.yaml
+      - name: Push manifests
+        run: |
+          flux push artifact $OCI_REPO:$(git rev-parse --short HEAD) \
+            --path="./deploy" \
+            --source="$(git config --get remote.origin.url)" \
+            --revision="$(git branch --show-current)@sha1:$(git rev-parse HEAD)"
+      - name: Deploy manifests to staging
+        run: |
+          flux tag artifact $OCI_REPO:$(git rev-parse --short HEAD) --tag staging
+```
+
+### Push and sign Kubernetes manifests to container registries
+
+Example workflow for publishing Kubernetes manifests bundled as OCI artifacts
+which are signed with Cosign and GitHub OIDC:
+
+```yaml
+name: push-sign-artifact
+
+on:
+  push:
+    branches:
+      - 'main'
+
+permissions:
+  packages: write # needed for ghcr.io access
+  id-token: write # needed for keyless signing
+
+env:
+  OCI_REPO: "oci://ghcr.io/my-org/manifests/${{ github.event.repository.name }}"
+
+jobs:
+  kubernetes:
+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout
+        uses: actions/checkout@v3
+      - name: Setup Flux CLI
+        uses: fluxcd/flux2/action@main
+      - name: Setup Cosign
+        uses: sigstore/cosign-installer@main
+      - name: Login to GHCR
+        uses: docker/login-action@v2
+        with:
+          registry: ghcr.io
+          username: ${{ github.actor }}
+          password: ${{ secrets.GITHUB_TOKEN }}
+      - name: Push and sign manifests
+        run: |
+          digest_url=$(flux push artifact \
+          $OCI_REPO:$(git rev-parse --short HEAD) \
+          --path="./manifests" \
+          --source="$(git config --get remote.origin.url)" \
+          --revision="$(git branch --show-current)@sha1:$(git rev-parse HEAD)" |\
+          jq -r '. | .repository + "@" + .digest')
+
+          cosign sign $digest_url
+```
+
+### End-to-end testing
+
+Example workflow for running Flux in Kubernetes Kind:
+
+```yaml
+name: e2e
+
+on:
+  push:
+    branches:
+      - '*'
+
+jobs:
+  kubernetes:
+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout
+        uses: actions/checkout@v3
+      - name: Setup Flux CLI
+        uses: fluxcd/flux2/action@main
+      - name: Setup Kubernetes Kind
+        uses: engineerd/setup-kind@v0.5.0
+      - name: Install Flux in Kubernetes Kind
+        run: flux install
+```
+
+A complete e2e testing workflow is available here
+[flux2-kustomize-helm-example](https://github.com/fluxcd/flux2-kustomize-helm-example/blob/main/.github/workflows/e2e.yaml)

action/action.yml

@@ -1,120 +1,64 @@
 name: Setup Flux CLI
-description: A GitHub Action for installing the Flux CLI
-author: Flux project
+description: A GitHub Action for running Flux commands
+author: Stefan Prodan
 branding:
   color: blue
   icon: command
 inputs:
   version:
-    description: "Flux version e.g. 2.0.0 (defaults to latest stable release)"
+    description: "Flux version e.g. 0.8.0 (defaults to latest stable release)"
     required: false
   arch:
     description: "arch can be amd64, arm64 or arm"
-    required: false
-    deprecationMessage: "No longer required, action will now detect runner arch."
+    required: true
+    default: "amd64"
   bindir:
-    description: "Alternative location for the Flux binary, defaults to path relative to $RUNNER_TOOL_CACHE."
+    description: "Optional location of the Flux binary. Will not use sudo if set. Updates System Path."
     required: false
   token:
-    description: "Token used to authentication against the GitHub.com API. Defaults to the token from the GitHub context of the workflow."
+    description: "GitHub Token used to authentication against the API (generally only needed to prevent quota limit errors)"
     required: false
 runs:
   using: composite
   steps:
-    - name: "Download the binary to the runner's cache dir"
+    - name: "Download flux binary to tmp"
       shell: bash
       run: |
+        ARCH=${{ inputs.arch }}
         VERSION=${{ inputs.version }}
-
         TOKEN=${{ inputs.token }}
-        if [[ -z "$TOKEN" ]]; then
-          TOKEN=${{ github.token }}
-        fi
 
-        if [[ -z "$VERSION" ]] || [[ "$VERSION" = "latest" ]]; then
-          VERSION=$(curl -fsSL -H "Authorization: token ${TOKEN}" https://api.github.com/repos/fluxcd/flux2/releases/latest | grep tag_name | cut -d '"' -f 4)
-        fi
-        if [[ -z "$VERSION" ]]; then
-          echo "Unable to determine Flux CLI version"
-          exit 1
-        fi
-        if [[ $VERSION = v* ]]; then
-          VERSION="${VERSION:1}"
-        fi
-
-        OS=$(echo "${RUNNER_OS}" | tr '[:upper:]' '[:lower:]')
-        if [[ "$OS" == "macos" ]]; then
-          OS="darwin"
-        fi
-
-        ARCH=$(echo "${RUNNER_ARCH}" | tr '[:upper:]' '[:lower:]')
-        if [[ "$ARCH" == "x64" ]]; then
-          ARCH="amd64"
-        elif [[ "$ARCH" == "x86" ]]; then
-          ARCH="386"
-        fi
-
-        FLUX_EXEC_FILE="flux"
-        if [[ "$OS" == "windows" ]]; then
-            FLUX_EXEC_FILE="${FLUX_EXEC_FILE}.exe"
-        fi
-
-        FLUX_TOOL_DIR=${{ inputs.bindir }}
-        if [[ -z "$FLUX_TOOL_DIR" ]]; then
-          FLUX_TOOL_DIR="${RUNNER_TOOL_CACHE}/flux2/${VERSION}/${OS}/${ARCH}"
-        fi
-        if [[ ! -x "$FLUX_TOOL_DIR/FLUX_EXEC_FILE" ]]; then
-          DL_DIR="$(mktemp -dt flux2-XXXXXX)"
-          trap 'rm -rf $DL_DIR' EXIT
-
-          echo "Downloading flux ${VERSION} for ${OS}/${ARCH}"
-          FLUX_TARGET_FILE="flux_${VERSION}_${OS}_${ARCH}.tar.gz"
-          if [[ "$OS" == "windows" ]]; then
-            FLUX_TARGET_FILE="flux_${VERSION}_${OS}_${ARCH}.zip"
-          fi
-
-          FLUX_CHECKSUMS_FILE="flux_${VERSION}_checksums.txt"
-
-          FLUX_DOWNLOAD_URL="https://github.com/fluxcd/flux2/releases/download/v${VERSION}/"
-
-          curl -fsSL -o "$DL_DIR/$FLUX_TARGET_FILE" "$FLUX_DOWNLOAD_URL/$FLUX_TARGET_FILE"
-          curl -fsSL -o "$DL_DIR/$FLUX_CHECKSUMS_FILE" "$FLUX_DOWNLOAD_URL/$FLUX_CHECKSUMS_FILE"
-
-          echo "Verifying checksum"
-          sum=""
-          if command -v openssl > /dev/null; then
-            sum=$(openssl sha256 "$DL_DIR/$FLUX_TARGET_FILE" | awk '{print $2}')
-          elif command -v sha256sum > /dev/null; then
-            sum=$(sha256sum "$DL_DIR/$FLUX_TARGET_FILE" | awk '{print $1}')
-          fi
-
-          if [[ -z "$sum" ]]; then
-            echo "Neither openssl nor sha256sum found. Cannot calculate checksum."
-            exit 1
-          fi
-
-          expected_sum=$(grep " $FLUX_TARGET_FILE\$" "$DL_DIR/$FLUX_CHECKSUMS_FILE" | awk '{print $1}')
-          if [ "$sum" != "$expected_sum" ]; then
-            echo "SHA sum of ${FLUX_TARGET_FILE} does not match. Aborting."
-            exit 1
-          fi
-
-          echo "Installing flux to ${FLUX_TOOL_DIR}"
-          mkdir -p "$FLUX_TOOL_DIR"
-        
-          if [[ "$OS" == "windows" ]]; then
-            unzip "$DL_DIR/$FLUX_TARGET_FILE" "$FLUX_EXEC_FILE" -d "$FLUX_TOOL_DIR"
+        if [ -z "${VERSION}" ]; then
+          if [ -n "${TOKEN}" ]; then
+            VERSION_SLUG=$(curl https://api.github.com/repos/fluxcd/flux2/releases/latest --silent --location --header "Authorization: token ${TOKEN}" | grep tag_name)
           else
-            tar xzf "$DL_DIR/$FLUX_TARGET_FILE" -C "$FLUX_TOOL_DIR" $FLUX_EXEC_FILE
+            # With no GITHUB_TOKEN you will experience occasional failures due to rate limiting
+            # Ref: https://github.com/fluxcd/flux2/issues/3509#issuecomment-1400820992
+            VERSION_SLUG=$(curl https://api.github.com/repos/fluxcd/flux2/releases/latest --silent --location | grep tag_name)
           fi
 
-          chmod +x "$FLUX_TOOL_DIR/$FLUX_EXEC_FILE"
+          VERSION=$(echo "${VERSION_SLUG}" | sed -E 's/.*"([^"]+)".*/\1/' | cut -c 2-)
         fi
 
-        echo "Adding flux to path"
-        echo "$FLUX_TOOL_DIR" >> "$GITHUB_PATH"
-
-    - name: "Print installed flux version"
+        BIN_URL="https://github.com/fluxcd/flux2/releases/download/v${VERSION}/flux_${VERSION}_linux_${ARCH}.tar.gz"
+        curl --silent --fail --location "${BIN_URL}" --output /tmp/flux.tar.gz
+        mkdir -p /tmp/flux
+        tar -C /tmp/flux/ -zxvf /tmp/flux.tar.gz
+    - name: "Copy Flux binary to execute location"
+      shell: bash
+      run: |
+        BINDIR=${{ inputs.bindir }}
+        if [ -z "${BINDIR}" ]; then
+          sudo cp /tmp/flux/flux /usr/local/bin
+        else
+          cp /tmp/flux/flux "${BINDIR}"
+          echo "${BINDIR}" >> $GITHUB_PATH
+        fi
+    - name: "Cleanup tmp"
+      shell: bash
+      run: |
+        rm -rf /tmp/flux/ /tmp/flux.tar.gz
+    - name: "Verify correct installation of binary"
       shell: bash
       run: |
         flux -v

rfcs/0006-git-repo-passwordless-auth/README.md

@@ -1,209 +0,0 @@
-# RFC-0006 Passwordless authentication for Git repositories
-
-**Status:** provisional
-
-**Creation date:** 2023-31-07
-
-## Summary
-
-Flux should provide a mechanism to authenticate against Git repositories without
-the use of passwords. This RFC proposes the use of alternative authentication
-methods like OIDC, OAuth2 and IAM to access Git repositories hosted on specific
-Git SaaS platforms and cloud providers.
-
-## Motivation
-
-At the moment, Flux supports HTTP basic and bearer authentication. Users are
-required to create a Secret containing the username and the password/bearer
-token, which is then referred to in the GitRepository using `.spec.secretRef`.
-
-While this works fine, it has a couple of drawbacks:
-* Scalability: Each new GitRepository potentially warrants another credentials
-pair, which doesn't scale well in big organizations with hundreds of
-repositories with different owners, increasing the risk of mismanagement and
-leaks.
-* Identity: A username is associated with an actual human. But often, the
-repository belongs to a team of 2 or more people. This leads to a problem where
-teams have to decide whose credentials should Flux use for authentication.
-
-These problems exist not due to flaws in Flux, but because of the inherent
-nature of password based authentication.
-
-With support for OIDC, OAuth2 and IAM based authentication, we can eliminate
-these problems:
-* Scalability: Since OIDC is fully handled by the cloud provider, it eliminates
-any user involvement in managing credentials. For OAuth2 and IAM, users do need
-to provide certain information like the ID of the resource, private key, etc.
-but these are still a better alternative to passwords since the same resource
-can be reused by multiple teams with different members.
-* Identity: Since all the above authentication methods are associated with a
-virtual resource independent of a user, it solves the problem of a single person
-being tied to automation that several people are involved in.
-
-### Goals
-
-* Integrate with major cloud providers' OIDC and IAM offerings to provide a
-seamless way of Git repository authentication.
-* Integrate with major Git SaaS providers to support their app based OAuth2
-mechanism.
-
-### Non-Goals
-* Replace the existing basic and bearer authentication API.
-
-## Proposal
-
-A new string field `.spec.provider` shall be added to the GitRepository API.
-The field will be an enum with the following variants:
-* `azure`
-* `github`
-* `gcp`
-
-> AWS CodeCommit is not supported as it does not support authentication via IAM
-Roles without the use of https://github.com/aws/git-remote-codecommit.
-
-By default, it will be blank, which indicates that the user wants to
-authenticate via HTTP basic/bearer auth or SSH.
-
-### Azure
-
-Git repositories hosted on Azure Devops can be accessed by Flux using OIDC if
-the cluster running Flux is hosted on AKS with [managed identity](https://learn.microsoft.com/en-us/azure/devops/integrate/get-started/authentication/service-principal-managed-identity?view=azure-devops)
-enabled. The managed identity associated with the cluster must have sufficient
-permissions to be able to access Azure Devops resources. This enables Flux to
-access the Git repository without the need for any credentials.
-
-```yaml
-apiVersion: source.toolkit.fluxcd.io/v1
-kind: GitRepository
-metadata:
-  name: azure-devops
-spec:
-  interval: 1m
-  url: https://dev.azure.com/<org>/<project>/_git/<repository>
-  ref:
-    branch: master
-  # notice the lack of secretRef
-  provider: azure
-```
-
-### GCP
-
-Git repositories hosted on Google Cloud Source Repositories can be accessed by
-Flux via a [GCP Service Account](https://cloud.google.com/iam/docs/service-account-overview).
-The Service Account must have sufficient permissions to be able to access Google
-Cloud Source Repositories and its credentials should be specified in the secret
-referred to in `.spec.secretRef`.
-
-```yaml
-apiVersion: source.toolkit.fluxcd.io/v1
-kind: GitRepository
-metadata:
-  name: gcp-repo
-spec:
-  interval: 1m
-  url: https://source.developers.google.com/p/<project>/r/<repository>
-  ref:
-    branch: master
-  provider: gcp
-  secretRef:
-    name: gcp-sa
----
-kind: Secret
-metadata:
-  name: gcp-sa
-stringData:
-  gcpServiceAccount: |
-    {
-      "type": "service_account",
-      "project_id": "my-google-project",
-      "private_key_id": "REDACTED",
-      "private_key": "-----BEGIN PRIVATE KEY-----\nREDACTED\n-----END PRIVATE KEY-----\n",
-      "client_email": "<service-account-id>@my-google-project.iam.gserviceaccount.com",
-      "client_id": "REDACTED",
-      "auth_uri": "https://accounts.google.com/o/oauth2/auth",
-      "token_uri": "https://oauth2.googleapis.com/token",
-      "auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs",
-      "client_x509_cert_url": "https://www.googleapis.com/robot/v1/metadata/x509/<service-account-id>%40my-google-project.iam.gserviceaccount.com"
-    }
-```
-
-### GitHub
-
-Git repositories hosted on GitHub can be accessed via [GitHub Apps](https://docs.github.com/en/apps/overview).
-This allows users to create a single resource from which they can access all
-their GitHub repositories. The app must have sufficient permissions to be able
-to access repositories. The app's ID, private key and installation ID should
-be mentioned in the Secret referred to by `.spec.secretRef`. GitHub Enterprise
-users will also need to mention their GitHub API URL in the Secret.
-
-```yaml
-apiVersion: source.toolkit.fluxcd.io/v1
-kind: GitRepository
-metadata:
-  name: github-repo
-spec:
-  interval: 1m
-  url: https://github.com/<org>/<repository>
-  ref:
-    branch: master
-  provider: github
-  secretRef:
-    name: github-app
----
-kind: Secret
-metadata:
-  name: gcp-sa
-stringData:
-  githubAppID: <app-id>
-  githubInstallationID: <installation-id>
-  githubPrivateKey: |
-    <PEM-private-key>
-  githubApiURl: <github-enterprise-api-url> #optional, required only for GitHub Enterprise users
-```
-
-## Design Details
-
-### Azure
-
-If `.spec.provider` is set to `azure`, Flux controllers will reach out to
-[Azure IMDS (Instance Metadata Service)](https://learn.microsoft.com/en-us/azure/active-directory/managed-identities-azure-resources/how-to-use-vm-token#get-a-token-using-go)
-to get an access token. This [access token will be then used as a bearer token](https://learn.microsoft.com/en-us/azure/devops/integrate/get-started/authentication/service-principal-managed-identity?view=azure-devops#q-can-i-use-a-service-principal-to-do-git-operations-like-clone-a-repo)
-to perform HTTP bearer authentication.
-
-### GCP
-
-If `.spec.provider` is set to `gcp`, Flux controllers will get the Service
-Account credentials from the specified Secret and use
-[`google.CredentialsFromJSON`](https://pkg.go.dev/golang.org/x/oauth2/google#CredentialsFromJSON)
-to fetch the access token. This access token will be then used as the password
-and the `client_email` as the username to perform HTTP basic authentication.
-
-### GitHub
-
-If `.spec.provider` is set to `github`, Flux controllers will get the app
-details from the specified Secret and use it to [generate an app installation
-token](https://docs.github.com/en/apps/creating-github-apps/authenticating-with-a-github-app/generating-an-installation-access-token-for-a-github-app).
-This token is then used as the password and [`x-access-token` as the username](https://docs.github.com/en/apps/creating-github-apps/registering-a-github-app/choosing-permissions-for-a-github-app#choosing-permissions-for-git-access)
-to perform HTTP basic authentication.
-
-### Token Caching and Refreshing
-
-To avoid calling the upstream API for a token during every reconciliation, Flux
-controllers shall cache the token after fetching it. Since GitHub tokens
-self-expire, the cache shall automatically evict the token after it has expired,
-triggering a fetch of a fresh token.
-For GCP, the [`TokenSource`](https://pkg.go.dev/golang.org/x/oauth2@v0.10.0#TokenSource)
-object will be cached, since it automatically handles refreshing an expired
-token and always returns a valid token. Since a `TokenSource` never expires, it
-need not be evicted from the cache.
-While Azure's managed identities subsystem caches the token, it is
-[recommended for the consumer application to implement their own caching](https://learn.microsoft.com/en-us/azure/active-directory/managed-identities-azure-resources/how-to-use-vm-token#token-caching)
-as well.
-The caches for all three providers are separate, i.e. there shall exist a
-dedicated cache for each provider.
-
-Since the proposed authentication methods for GitHub and GCP involve some form
-of credentials stored in a Kubernetes Secret, the cache key can be the Secret's
-`<namespace/name>`. Since authentication for Azure is configured directly via
-the source-controller Deployment, the token can just be stored in a global
-variable, which is refreshed whenever the token expires.