Merge pull request #3339 from fluxcd/update-deps

Update dependencies

.github/dependabot.yml

@@ -0,0 +1,9 @@
+version: 2
+
+updates:
+  - package-ecosystem: "github-actions"
+    directory: "/"
+    labels: ["area/build"]
+    schedule:
+      # by default this will be on a monday.
+      interval: "weekly"

.github/runners/README.md

@@ -1,24 +1,32 @@
 # Flux ARM64 GitHub runners
 
-The Flux ARM64 end-to-end tests run on Equinix instances provisioned with Docker and GitHub self-hosted runners.
+The Flux ARM64 end-to-end tests run on Equinix Metal instances provisioned with Docker and GitHub self-hosted runners.
 
 ## Current instances
 
-| Runner        | Instance            | Region |
-|---------------|---------------------|--------|
-| equinix-arm-1 | flux-equinix-arm-01 | AMS1   |
-| equinix-arm-2 | flux-equinix-arm-01 | AMS1   |
-| equinix-arm-3 | flux-equinix-arm-01 | AMS1   |
-| equinix-arm-4 | flux-equinix-arm-02 | DFW2   |
-| equinix-arm-5 | flux-equinix-arm-02 | DFW2   |
-| equinix-arm-6 | flux-equinix-arm-02 | DFW2   |
+| Repository                  | Runner           | Instance               | Location      |
+|-----------------------------|------------------|------------------------|---------------|
+| flux2                       | equinix-arm-dc-1 | flux-equinix-arm-dc-01 | Washington DC |
+| flux2                       | equinix-arm-dc-2 | flux-equinix-arm-dc-01 | Washington DC |
+| flux2                       | equinix-arm-da-1 | flux-equinix-arm-da-01 | Dallas        |
+| flux2                       | equinix-arm-da-2 | flux-equinix-arm-da-01 | Dallas        |
+| source-controller           | equinix-arm-dc-1 | flux-equinix-arm-dc-01 | Washington DC |
+| source-controller           | equinix-arm-da-1 | flux-equinix-arm-da-01 | Dallas        |
+| image-automation-controller | equinix-arm-dc-1 | flux-equinix-arm-dc-01 | Washington DC |
+| image-automation-controller | equinix-arm-da-1 | flux-equinix-arm-da-01 | Dallas        |
+
+Instance spec:
+- Ampere Altra Q80-30 80-core processor @ 2.8GHz
+- 2 x 960GB NVME
+- 256GB RAM
+- 2 x 25Gbps
 
 ## Instance setup
 
 In order to add a new runner to the GitHub Actions pool,
 first create a server on Equinix with the following configuration:
-- Type: c2.large.arm
-- OS: Ubuntu 20.04
+- Type: `c3.large.arm64`
+- OS: `Ubuntu 22.04 LTS`
 
 ### Install prerequisites
 
@@ -54,14 +62,14 @@ sudo ./prereq.sh
 
 - Retrieve the GitHub runner token from the repository [settings page](https://github.com/fluxcd/flux2/settings/actions/runners/new?arch=arm64&os=linux)
 
-- Create 3 directories `runner1`, `runner2`, `runner3`
+- Create two directories `flux2-01`, `flux2-02`
 
 - In each dir run:
 ```shell
 curl -sL https://raw.githubusercontent.com/fluxcd/flux2/main/.github/runners/runner-setup.sh > runner-setup.sh \
   && chmod +x ./runner-setup.sh
 
-./runner-setup.sh equinix-arm-<NUMBER> <TOKEN>
+./runner-setup.sh equinix-arm-<NUMBER> <TOKEN> <REPO>
 ```
 
 - Reboot the instance

.github/runners/prereq.sh

@@ -18,11 +18,11 @@
 
 set -eu
 
-KIND_VERSION=0.14.0
+KIND_VERSION=0.17.0
 KUBECTL_VERSION=1.24.0
-KUSTOMIZE_VERSION=4.5.4
-HELM_VERSION=3.8.2
-GITHUB_RUNNER_VERSION=2.291.1
+KUSTOMIZE_VERSION=4.5.7
+HELM_VERSION=3.10.1
+GITHUB_RUNNER_VERSION=2.298.2
 PACKAGES="apt-transport-https ca-certificates software-properties-common build-essential libssl-dev gnupg lsb-release jq pkg-config"
 
 # install prerequisites
@@ -31,6 +31,10 @@ apt-get update \
   && apt-get clean \
   && rm -rf /var/lib/apt/lists/*
 
+# fix Kubernetes DNS resolution
+rm /etc/resolv.conf
+cat "/run/systemd/resolve/stub-resolv.conf" | sed '/search/d' > /etc/resolv.conf
+
 # install docker
 curl -fsSL https://get.docker.com -o get-docker.sh \
   && chmod +x get-docker.sh

.github/runners/runner-setup.sh

@@ -22,7 +22,7 @@ RUNNER_NAME=$1
 REPOSITORY_TOKEN=$2
 REPOSITORY_URL=${3:-https://github.com/fluxcd/flux2}
 
-GITHUB_RUNNER_VERSION=2.285.1
+GITHUB_RUNNER_VERSION=2.298.2
 
 # download runner
 curl -o actions-runner-linux-arm64.tar.gz -L https://github.com/actions/runner/releases/download/v${GITHUB_RUNNER_VERSION}/actions-runner-linux-arm64-${GITHUB_RUNNER_VERSION}.tar.gz \

.github/workflows/README.md

@@ -0,0 +1,50 @@
+# Flux GitHub Workflows
+
+## End-to-end Testing
+
+The e2e workflows run a series of tests to ensure that the Flux CLI and
+the GitOps Toolkit controllers work well all together.
+The tests are written in Go, Bash, Make and Terraform.
+
+| Workflow           | Jobs                 | Runner         | Role                                          |
+|--------------------|----------------------|----------------|-----------------------------------------------|
+| e2e.yaml           | e2e-amd64-kubernetes | GitHub Ubuntu  | integration testing with Kubernetes Kind<br/> |
+| e2e-arm64.yaml     | e2e-arm64-kubernetes | Equinix Ubuntu | integration testing with Kubernetes Kind<br/> |
+| e2e-bootstrap.yaml | e2e-boostrap-github  | GitHub Ubuntu  | integration testing with GitHub API<br/>      |
+| e2e-azure.yaml     | e2e-amd64-aks        | GitHub Ubuntu  | integration testing with Azure API<br/>       |
+| scan.yaml          | scan-fossa           | GitHub Ubuntu  | license scanning<br/>                         |
+| scan.yaml          | scan-snyk            | GitHub Ubuntu  | vulnerability scanning<br/>                   |
+| scan.yaml          | scan-codeql          | GitHub Ubuntu  | vulnerability scanning<br/>                   |
+
+## Components Update
+
+The components update workflow scans the GitOps Toolkit controller repositories for new releases,
+amd when it finds a new controller version, the workflow performs the following steps:
+- Updates the controller API package version in `go.mod`.
+- Patches the controller CRDs version in the `manifests/crds` overlay.
+- Patches the controller Deployment version in `manifests/bases` overlay.
+- Opens a Pull Request against the `main` branch.
+- Triggers the e2e test suite to run for the opened PR.
+
+
+| Workflow    | Jobs              | Runner        | Role                                                |
+|-------------|-------------------|---------------|-----------------------------------------------------|
+| update.yaml | update-components | GitHub Ubuntu | update the GitOps Toolkit APIs and controllers<br/> |
+
+## Release
+
+The release workflow is triggered by a semver Git tag and performs the following steps:
+- Generates the Flux install manifests (YAML).
+- Generates the OpenAPI validation schemas for the GitOps Toolkit CRDs (JSON).
+- Generates a Software Bill of Materials (SPDX JSON).
+- Builds the Flux CLI binaries and the multi-arch container images.
+- Pushes the container images to GitHub Container Registry and DockerHub.
+- Signs the sbom, the binaries checksum and the container images with Cosign and GitHub OIDC.
+- Uploads the sbom, binaries, checksums and install manifests to GitHub Releases.
+- Pushes the install manifests as OCI artifacts to GitHub Container Registry and DockerHub.
+- Signs the OCI artifacts with Cosign and GitHub OIDC.
+
+| Workflow     | Jobs                   | Runner        | Role                                                 |
+|--------------|------------------------|---------------|------------------------------------------------------|
+| release.yaml | release-flux-cli       | GitHub Ubuntu | build, push and sign the CLI release artifacts<br/>  |
+| release.yaml | release-flux-manifests | GitHub Ubuntu | build, push and sign the Flux install manifests<br/> |

.github/workflows/e2e-arm64.yaml

@@ -3,33 +3,97 @@ name: e2e-arm64
 on:
   workflow_dispatch:
   push:
-    branches: [ main, update-components ]
+    branches: [ main, update-components, e2e-arm64* ]
+
+permissions:
+  contents: read
 
 jobs:
-  test:
+  e2e-arm64-kubernetes:
     # Hosted on Equinix
     # Docs: https://github.com/fluxcd/flux2/tree/main/.github/runners
     runs-on: [self-hosted, Linux, ARM64, equinix]
+    strategy:
+      matrix:
+        # Keep this list up-to-date with https://endoflife.date/kubernetes
+        KUBERNETES_VERSION: [ 1.23.13, 1.24.7, 1.25.3 ]
     steps:
       - name: Checkout
-        uses: actions/checkout@v3
+        uses: actions/checkout@93ea575cb5d8a053eaa0ac8fa3b40d7e05a33cc8
       - name: Setup Go
-        uses: actions/setup-go@v3
+        uses: actions/setup-go@c4a742cab115ed795e34d4513e2cf7d472deb55f
         with:
-          go-version: 1.18.x
+          go-version: 1.19.x
       - name: Prepare
         id: prep
         run: |
-          echo ::set-output name=CLUSTER::arm64-${GITHUB_SHA:0:7}-$(date +%s)
-          echo ::set-output name=CONTEXT::kind-arm64-${GITHUB_SHA:0:7}-$(date +%s)
+          ID=${GITHUB_SHA:0:7}-${{ matrix.KUBERNETES_VERSION }}-$(date +%s)
+          echo "CLUSTER=arm64-${ID}" >> $GITHUB_OUTPUT
       - name: Build
         run: |
           make build
       - name: Setup Kubernetes Kind
         run: |
-          kind create cluster --name ${{ steps.prep.outputs.CLUSTER }} --kubeconfig=/tmp/${{ steps.prep.outputs.CLUSTER }}
+          kind create cluster \
+          --wait 5m \
+          --name ${{ steps.prep.outputs.CLUSTER }} \
+          --kubeconfig=/tmp/${{ steps.prep.outputs.CLUSTER }} \
+          --image=kindest/node:v${{ matrix.KUBERNETES_VERSION }}
       - name: Run e2e tests
         run: TEST_KUBECONFIG=/tmp/${{ steps.prep.outputs.CLUSTER }} make e2e
+      - name: Run multi-tenancy tests
+        env:
+          KUBECONFIG: /tmp/${{ steps.prep.outputs.CLUSTER }}
+        run: |
+          ./bin/flux install
+          ./bin/flux create source git flux-system \
+          --interval=15m \
+          --url=https://github.com/fluxcd/flux2-multi-tenancy \
+          --branch=main \
+          --ignore-paths="./clusters/**/flux-system/"
+          ./bin/flux create kustomization flux-system \
+          --interval=15m \
+          --source=flux-system \
+          --path=./clusters/staging
+          kubectl -n flux-system wait kustomization/tenants --for=condition=ready --timeout=5m
+          kubectl -n apps wait kustomization/dev-team --for=condition=ready --timeout=1m
+          kubectl -n apps wait helmrelease/podinfo --for=condition=ready --timeout=1m
+      - name: Run monitoring tests
+        # Keep this test in sync with https://fluxcd.io/flux/guides/monitoring/
+        env:
+          KUBECONFIG: /tmp/${{ steps.prep.outputs.CLUSTER }}
+        run: |
+          ./bin/flux create source git flux-monitoring \
+          --interval=30m \
+          --url=https://github.com/fluxcd/flux2 \
+          --branch=${GITHUB_REF#refs/heads/}
+          ./bin/flux create kustomization kube-prometheus-stack \
+          --interval=1h \
+          --prune \
+          --source=flux-monitoring \
+          --path="./manifests/monitoring/kube-prometheus-stack" \
+          --health-check-timeout=5m \
+          --wait
+          ./bin/flux create kustomization monitoring-config \
+          --depends-on=kube-prometheus-stack \
+          --interval=1h \
+          --prune=true \
+          --source=flux-monitoring \
+          --path="./manifests/monitoring/monitoring-config" \
+          --health-check-timeout=1m \
+          --wait
+          kubectl -n flux-system wait kustomization/kube-prometheus-stack --for=condition=ready --timeout=5m
+          kubectl -n flux-system wait kustomization/monitoring-config --for=condition=ready --timeout=5m
+          kubectl -n monitoring wait helmrelease/kube-prometheus-stack --for=condition=ready --timeout=1m
+      - name: Debug failure
+        if: failure()
+        env:
+          KUBECONFIG: /tmp/${{ steps.prep.outputs.CLUSTER }}
+        run: |
+          kubectl -n flux-system get all
+          kubectl -n flux-system describe po 
+          kubectl -n flux-system logs deploy/source-controller
+          kubectl -n flux-system logs deploy/kustomize-controller
       - name: Cleanup
         if: always()
         run: |

.github/workflows/e2e-azure.yaml

@@ -7,31 +7,31 @@ on:
   push:
     branches: [ azure* ]
 
+permissions:
+  contents: read
+
 jobs:
-  e2e:
-    runs-on: ubuntu-latest
+  e2e-amd64-aks:
+    runs-on: ubuntu-22.04
     steps:
       - name: Checkout
-        uses: actions/checkout@v3
+        uses: actions/checkout@93ea575cb5d8a053eaa0ac8fa3b40d7e05a33cc8
       - name: Restore Go cache
-        uses: actions/cache@v3
+        uses: actions/cache@9b0c1fce7a93df8e3bb8926b0d6e9d89e92f20a7
         with:
           path: ~/go/pkg/mod
           key: ${{ runner.os }}-go1.18-${{ hashFiles('**/go.sum') }}
           restore-keys: |
             ${{ runner.os }}-go1.18-
       - name: Setup Go
-        uses: actions/setup-go@v2
+        uses: actions/setup-go@c4a742cab115ed795e34d4513e2cf7d472deb55f
         with:
-          go-version: 1.18.x
+          go-version: 1.19.x
       - name: Install libgit2
         run: |
-          sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 648ACFD622F3D138
-          sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 0E98404D386FA1D9
-          echo "deb http://deb.debian.org/debian unstable main" | sudo tee -a /etc/apt/sources.list
-          echo "deb-src http://deb.debian.org/debian unstable main" | sudo tee -a /etc/apt/sources.list
+          echo "deb http://archive.ubuntu.com/ubuntu/ kinetic universe" | sudo tee -a /etc/apt/sources.list
           sudo apt-get update
-          sudo apt-get install -y --allow-downgrades libgit2-dev/unstable zlib1g-dev/unstable libssh2-1-dev/unstable libpcre3-dev/unstable
+          sudo apt-get install -y -t kinetic libgit2-dev=1.3.0+dfsg.1-3ubuntu1
       - name: Setup Flux CLI
         run: |
           make build
@@ -44,9 +44,9 @@ jobs:
           mkdir -p $HOME/.local/bin
           mv sops-v3.7.1.linux $HOME/.local/bin/sops
       - name: Setup Terraform
-        uses: hashicorp/setup-terraform@v1
+        uses: hashicorp/setup-terraform@633666f66e0061ca3b725c73b2ec20cd13a8fdd1  # v2
         with:
-          terraform_version: 1.0.7
+          terraform_version: 1.2.8
           terraform_wrapper: false
       - name: Setup Azure CLI
         run: |

.github/workflows/e2e-bootstrap.yaml

@@ -1,34 +1,38 @@
-name: bootstrap
+name: e2e-bootstrap
 
 on:
+  workflow_dispatch:
   push:
     branches: [ main ]
   pull_request:
     branches: [ main ]
 
+permissions:
+  contents: read
+
 jobs:
-  github:
+  e2e-boostrap-github:
     runs-on: ubuntu-latest
-    if: github.event_name != 'pull_request' || github.event.pull_request.head.repo.full_name == github.repository
+    if: (github.event_name != 'pull_request' || github.event.pull_request.head.repo.full_name == github.repository) && github.actor != 'dependabot[bot]'
     steps:
       - name: Checkout
-        uses: actions/checkout@v3
+        uses: actions/checkout@93ea575cb5d8a053eaa0ac8fa3b40d7e05a33cc8
       - name: Restore Go cache
-        uses: actions/cache@v3
+        uses: actions/cache@9b0c1fce7a93df8e3bb8926b0d6e9d89e92f20a7
         with:
           path: ~/go/pkg/mod
           key: ${{ runner.os }}-go1.18-${{ hashFiles('**/go.sum') }}
           restore-keys: |
             ${{ runner.os }}-go1.18-
       - name: Setup Go
-        uses: actions/setup-go@v3
+        uses: actions/setup-go@c4a742cab115ed795e34d4513e2cf7d472deb55f
         with:
-          go-version: 1.18.x
+          go-version: 1.19.x
       - name: Setup Kubernetes
-        uses: engineerd/setup-kind@v0.5.0
+        uses: engineerd/setup-kind@aa272fe2a7309878ffc2a81c56cfe3ef108ae7d0 # v0.5.0
         with:
-          version: v0.11.1
-          image: kindest/node:v1.21.1@sha256:69860bda5563ac81e3c0057d654b5253219618a22ec3a346306239bba8cfa1a6
+          version: v0.16.0
+          image: kindest/node:v1.25.2@sha256:9be91e9e9cdf116809841fc77ebdb8845443c4c72fe5218f3ae9eb57fdb4bace
       - name: Setup Kustomize
         uses: fluxcd/pkg//actions/kustomize@main
       - name: Build

.github/workflows/e2e.yaml

@@ -1,13 +1,17 @@
 name: e2e
 
 on:
+  workflow_dispatch:
   push:
     branches: [ main ]
   pull_request:
     branches: [ main, oci ]
 
+permissions:
+  contents: read
+
 jobs:
-  kind:
+  e2e-amd64-kubernetes:
     runs-on: ubuntu-latest
     services:
       registry:
@@ -16,20 +20,20 @@ jobs:
           - 5000:5000
     steps:
       - name: Checkout
-        uses: actions/checkout@v3
+        uses: actions/checkout@93ea575cb5d8a053eaa0ac8fa3b40d7e05a33cc8
       - name: Restore Go cache
-        uses: actions/cache@v3
+        uses: actions/cache@9b0c1fce7a93df8e3bb8926b0d6e9d89e92f20a7
         with:
           path: ~/go/pkg/mod
           key: ${{ runner.os }}-go1.18-${{ hashFiles('**/go.sum') }}
           restore-keys: |
             ${{ runner.os }}-go1.18-
       - name: Setup Go
-        uses: actions/setup-go@v3
+        uses: actions/setup-go@c4a742cab115ed795e34d4513e2cf7d472deb55f
         with:
-          go-version: 1.18.x
+          go-version: 1.19.x
       - name: Setup Kubernetes
-        uses: engineerd/setup-kind@v0.5.0
+        uses: engineerd/setup-kind@aa272fe2a7309878ffc2a81c56cfe3ef108ae7d0 # v0.5.0
         with:
           version: v0.11.1
           image: kindest/node:v1.20.7

.github/workflows/release.yaml

@@ -5,41 +5,43 @@ on:
     tags: [ 'v*' ]
 
 permissions:
-  contents: write # needed to write releases
-  id-token: write # needed for keyless signing
-  packages: write # needed for ghcr access
+  contents: read
 
 jobs:
-  goreleaser:
+  release-flux-cli:
     runs-on: ubuntu-latest
+    permissions:
+      contents: write # needed to write releases
+      id-token: write # needed for keyless signing
+      packages: write # needed for ghcr access
     steps:
       - name: Checkout
-        uses: actions/checkout@v3
+        uses: actions/checkout@93ea575cb5d8a053eaa0ac8fa3b40d7e05a33cc8
       - name: Unshallow
         run: git fetch --prune --unshallow
       - name: Setup Go
-        uses: actions/setup-go@v3
+        uses: actions/setup-go@c4a742cab115ed795e34d4513e2cf7d472deb55f
         with:
-          go-version: 1.18.x
+          go-version: 1.19.x
       - name: Setup QEMU
-        uses: docker/setup-qemu-action@v2
+        uses: docker/setup-qemu-action@e81a89b1732b9c48d79cd809d8d81d79c4647a18 # v2
       - name: Setup Docker Buildx
         id: buildx
-        uses: docker/setup-buildx-action@v2
+        uses: docker/setup-buildx-action@8c0edbc76e98fa90f69d9a2c020dcb50019dc325  # v2
       - name: Setup Syft
-        uses: anchore/sbom-action/download-syft@v0
+        uses: anchore/sbom-action/download-syft@06e109483e6aa305a2b2395eabae554e51530e1d # v0.13.1
       - name: Setup Cosign
-        uses: sigstore/cosign-installer@main
+        uses: sigstore/cosign-installer@7bca8b41164994a7dc93749d266e2f1db492f8a2
       - name: Setup Kustomize
         uses: fluxcd/pkg//actions/kustomize@main
       - name: Login to GitHub Container Registry
-        uses: docker/login-action@v2
+        uses: docker/login-action@f4ef78c080cd8ba55a85445d5b36e214a81df20a # v2
         with:
           registry: ghcr.io
           username: fluxcdbot
           password: ${{ secrets.GHCR_TOKEN }}
       - name: Login to Docker Hub
-        uses: docker/login-action@v2
+        uses: docker/login-action@f4ef78c080cd8ba55a85445d5b36e214a81df20a  # v2
         with:
           username: fluxcdbot
           password: ${{ secrets.DOCKER_FLUXCD_PASSWORD }}
@@ -51,10 +53,8 @@ jobs:
       - name: Build CRDs
         run: |
           kustomize build manifests/crds > all-crds.yaml
-      # Pinned to commit before https://github.com/fluxcd/pkg/pull/189 due to
-      # introduction faulty behavior.
       - name: Generate OpenAPI JSON schemas from CRDs
-        uses: fluxcd/pkg//actions/crdjsonschema@49e26aa2ee9e734c3233c560253fd9542afe18ae
+        uses: fluxcd/pkg//actions/crdjsonschema@main
         with:
           crd: all-crds.yaml
           output: schemas
@@ -73,7 +73,7 @@ jobs:
         env:
           GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
       - name: Run GoReleaser
-        uses: goreleaser/goreleaser-action@v3
+        uses: goreleaser/goreleaser-action@b508e2e3ef3b19d4e4146d4f8fb3ba9db644a757 # v3
         with:
           version: latest
           args: release --release-notes=output/notes.md --skip-validate
@@ -81,3 +81,69 @@ jobs:
           GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
           HOMEBREW_TAP_GITHUB_TOKEN: ${{ secrets.BOT_GITHUB_TOKEN }}
           AUR_BOT_SSH_PRIVATE_KEY: ${{ secrets.AUR_BOT_SSH_PRIVATE_KEY }}
+  release-flux-manifests:
+    runs-on: ubuntu-latest
+    needs: release-flux-cli
+    permissions:
+      id-token: write
+      packages: write
+    steps:
+      - uses: actions/checkout@93ea575cb5d8a053eaa0ac8fa3b40d7e05a33cc8
+      - name: Setup Kustomize
+        uses: fluxcd/pkg/actions/kustomize@main
+      - name: Setup Flux CLI
+        uses: ./action/
+      - name: Prepare
+        id: prep
+        run: |
+          VERSION=$(flux version --client | awk '{ print $NF }')
+          echo ::set-output name=VERSION::${VERSION}
+      - name: Login to GHCR
+        uses: docker/login-action@f4ef78c080cd8ba55a85445d5b36e214a81df20a # v2
+        with:
+          registry: ghcr.io
+          username: fluxcdbot
+          password: ${{ secrets.GHCR_TOKEN }}
+      - name: Login to DockerHub
+        uses: docker/login-action@f4ef78c080cd8ba55a85445d5b36e214a81df20a # v2
+        with:
+          username: fluxcdbot
+          password: ${{ secrets.DOCKER_FLUXCD_PASSWORD }}
+      - name: Push manifests to GHCR
+        run: |
+          mkdir -p ./ghcr.io/flux-system
+          flux install --registry=ghcr.io/fluxcd \
+          --components-extra=image-reflector-controller,image-automation-controller \
+          --export > ./ghcr.io/flux-system/gotk-components.yaml
+          
+          cd ./ghcr.io && flux push artifact \
+          oci://ghcr.io/fluxcd/flux-manifests:${{ steps.prep.outputs.VERSION }} \
+          --path="./flux-system" \
+          --source=${{ github.repositoryUrl }} \
+          --revision="${{ github.ref_name }}/${{ github.sha }}"
+      - name: Push manifests to DockerHub
+        run: |
+          mkdir -p ./docker.io/flux-system
+          flux install --registry=docker.io/fluxcd \
+          --components-extra=image-reflector-controller,image-automation-controller \
+          --export > ./docker.io/flux-system/gotk-components.yaml
+          
+          cd ./docker.io && flux push artifact \
+          oci://docker.io/fluxcd/flux-manifests:${{ steps.prep.outputs.VERSION }} \
+          --path="./flux-system" \
+          --source=${{ github.repositoryUrl }} \
+          --revision="${{ github.ref_name }}/${{ github.sha }}"
+      - uses: sigstore/cosign-installer@7cc35d7fdbe70d4278a0c96779081e6fac665f88 # v2.8.0
+      - name: Sign manifests
+        env:
+          COSIGN_EXPERIMENTAL: 1
+        run: |
+          cosign sign ghcr.io/fluxcd/flux-manifests:${{ steps.prep.outputs.VERSION }}
+          cosign sign docker.io/fluxcd/flux-manifests:${{ steps.prep.outputs.VERSION }}
+      - name: Tag manifests
+        run: |
+          flux tag artifact oci://ghcr.io/fluxcd/flux-manifests:${{ steps.prep.outputs.VERSION }} \
+          --tag latest
+
+          flux tag artifact oci://docker.io/fluxcd/flux-manifests:${{ steps.prep.outputs.VERSION }} \
+          --tag latest

.github/workflows/scan.yaml

@@ -1,6 +1,7 @@
 name: scan
 
 on:
+  workflow_dispatch:
   push:
     branches: [ main ]
   pull_request:
@@ -9,56 +10,62 @@ on:
     - cron: '18 10 * * 3'
 
 permissions:
-  contents: read # for actions/checkout to fetch code
-  security-events: write # for codeQL to write security events
+  contents: read
 
 jobs:
-  fossa:
-    name: FOSSA
+  scan-fossa:
     runs-on: ubuntu-latest
+    if: github.actor != 'dependabot[bot]'
     steps:
-      - uses: actions/checkout@v3
+      - uses: actions/checkout@93ea575cb5d8a053eaa0ac8fa3b40d7e05a33cc8
       - name: Run FOSSA scan and upload build data
-        uses: fossa-contrib/fossa-action@v1
+        uses: fossa-contrib/fossa-action@6cffaa064112e1cf9b5798c6224f9487dc1ec316 # v1
         with:
           # FOSSA Push-Only API Token
           fossa-api-key: 5ee8bf422db1471e0bcf2bcb289185de
           github-token: ${{ github.token }}
 
-  snyk:
-    name: Snyk
+  scan-snyk:
     runs-on: ubuntu-latest
-    if: github.event_name != 'pull_request' || github.event.pull_request.head.repo.full_name == github.repository
+    permissions:
+      security-events: write
+    if: (github.event_name != 'pull_request' || github.event.pull_request.head.repo.full_name == github.repository) && github.actor != 'dependabot[bot]'
     steps:
-      - uses: actions/checkout@v3
+      - uses: actions/checkout@93ea575cb5d8a053eaa0ac8fa3b40d7e05a33cc8
       - name: Setup Kustomize
         uses: fluxcd/pkg//actions/kustomize@main
       - name: Build manifests
         run: |
           make cmd/flux/.manifests.done
       - name: Run Snyk to check for vulnerabilities
-        uses: snyk/actions/golang@master
+        uses: snyk/actions/golang@a8dd587d8a94f5663fa3d67d51abd0cc66aff244 # v0.3.0
         continue-on-error: true
         env:
           SNYK_TOKEN: ${{ secrets.SNYK_TOKEN }}
         with:
           args: --sarif-file-output=snyk.sarif
       - name: Upload result to GitHub Code Scanning
-        uses: github/codeql-action/upload-sarif@v1
+        uses: github/codeql-action/upload-sarif@678fc3afe258fb2e0cdc165ccf77b85719de7b3c # v2
         with:
           sarif_file: snyk.sarif
 
-  codeql:
-    name: CodeQL
+  scan-codeql:
     runs-on: ubuntu-latest
+    permissions:
+      security-events: write
+    if: github.actor != 'dependabot[bot]'
     steps:
       - name: Checkout repository
-        uses: actions/checkout@v3
+        uses: actions/checkout@93ea575cb5d8a053eaa0ac8fa3b40d7e05a33cc8
+      - name: Set up Go
+        uses: actions/setup-go@c4a742cab115ed795e34d4513e2cf7d472deb55f
+        with:
+          go-version: 1.19.x
       - name: Initialize CodeQL
-        uses: github/codeql-action/init@v2
+        uses: github/codeql-action/init@678fc3afe258fb2e0cdc165ccf77b85719de7b3c # v2
         with:
           languages: go
       - name: Autobuild
-        uses: github/codeql-action/autobuild@v2
+        uses: github/codeql-action/autobuild@678fc3afe258fb2e0cdc165ccf77b85719de7b3c # v2
       - name: Perform CodeQL Analysis
-        uses: github/codeql-action/analyze@v2
+        uses: github/codeql-action/analyze@678fc3afe258fb2e0cdc165ccf77b85719de7b3c # v2

.github/workflows/update.yaml

@@ -1,4 +1,4 @@
-name: Update Components
+name: update
 
 on:
   workflow_dispatch:
@@ -7,16 +7,22 @@ on:
   push:
     branches: [main]
 
+permissions:
+  contents: read
+
 jobs:
   update-components:
     runs-on: ubuntu-latest
+    permissions:
+      contents: write
+      pull-requests: write
     steps:
       - name: Check out code
-        uses: actions/checkout@v3
+        uses: actions/checkout@93ea575cb5d8a053eaa0ac8fa3b40d7e05a33cc8
       - name: Setup Go
-        uses: actions/setup-go@v3
+        uses: actions/setup-go@c4a742cab115ed795e34d4513e2cf7d472deb55f
         with:
-          go-version: 1.18.x
+          go-version: 1.19.x
       - name: Update component versions
         id: update
         run: |
@@ -69,7 +75,7 @@ jobs:
 
       - name: Create Pull Request
         id: cpr
-        uses: peter-evans/create-pull-request@v3
+        uses: peter-evans/create-pull-request@b4d51739f96fca8047ad065eccef63442d8e99f7 # v4
         with:
           token: ${{ secrets.BOT_GITHUB_TOKEN }}
           commit-message: |

CONTRIBUTING.md

@@ -59,7 +59,7 @@ This project is composed of:
 ### Understanding the code
 
 To get started with developing controllers, you might want to review
-[our guide](https://fluxcd.io/docs/gitops-toolkit/source-watcher/) which
+[our guide](https://fluxcd.io/flux/gitops-toolkit/source-watcher/) which
 walks you through writing a short and concise controller that watches out
 for source changes.
 
@@ -67,7 +67,7 @@ for source changes.
 
 Prerequisites:
 
-* go >= 1.17
+* go >= 1.19
 * kubectl >= 1.20
 * kustomize >= 4.4
 * coreutils (on Mac OS)

Dockerfile

@@ -3,7 +3,7 @@ FROM alpine:3.16 as builder
 RUN apk add --no-cache ca-certificates curl
 
 ARG ARCH=linux/amd64
-ARG KUBECTL_VER=1.24.1
+ARG KUBECTL_VER=1.25.4
 
 RUN curl -sL https://storage.googleapis.com/kubernetes-release/release/v${KUBECTL_VER}/bin/${ARCH}/kubectl \
     -o /usr/local/bin/kubectl && chmod +x /usr/local/bin/kubectl && \
@@ -11,10 +11,6 @@ RUN curl -sL https://storage.googleapis.com/kubernetes-release/release/v${KUBECT
 
 FROM alpine:3.16 as flux-cli
 
-# Create minimal nsswitch.conf file to prioritize the usage of /etc/hosts over DNS queries.
-# https://github.com/gliderlabs/docker-alpine/issues/367#issuecomment-354316460
-RUN [ ! -e /etc/nsswitch.conf ] && echo 'hosts: files dns' > /etc/nsswitch.conf
-
 RUN apk add --no-cache ca-certificates
 
 COPY --from=builder /usr/local/bin/kubectl /usr/local/bin/

Makefile

@@ -17,8 +17,8 @@ rwildcard=$(foreach d,$(wildcard $(addsuffix *,$(1))),$(call rwildcard,$(d)/,$(2
 all: test build
 
 tidy:
-	go mod tidy -compat=1.17
-	cd tests/azure && go mod tidy -compat=1.17
+	go mod tidy -compat=1.19
+	cd tests/azure && go mod tidy -compat=1.19
 
 fmt:
 	go fmt ./...

README.md

@@ -1,14 +1,13 @@
 # Flux version 2
 
-[![CII Best Practices](https://bestpractices.coreinfrastructure.org/projects/4782/badge)](https://bestpractices.coreinfrastructure.org/projects/4782)
-[![e2e](https://github.com/fluxcd/flux2/workflows/e2e/badge.svg)](https://github.com/fluxcd/flux2/actions)
-[![report](https://goreportcard.com/badge/github.com/fluxcd/flux2)](https://goreportcard.com/report/github.com/fluxcd/flux2)
-[![license](https://img.shields.io/github/license/fluxcd/flux2.svg)](https://github.com/fluxcd/flux2/blob/main/LICENSE)
 [![release](https://img.shields.io/github/release/fluxcd/flux2/all.svg)](https://github.com/fluxcd/flux2/releases)
+[![CII Best Practices](https://bestpractices.coreinfrastructure.org/projects/4782/badge)](https://bestpractices.coreinfrastructure.org/projects/4782)
+[![FOSSA Status](https://app.fossa.com/api/projects/custom%2B162%2Fgithub.com%2Ffluxcd%2Fflux2.svg?type=shield)](https://app.fossa.com/projects/custom%2B162%2Fgithub.com%2Ffluxcd%2Fflux2?ref=badge_shield)
+[![Artifact HUB](https://img.shields.io/endpoint?url=https://artifacthub.io/badge/repository/flux2)](https://artifacthub.io/packages/helm/fluxcd-community/flux2)
 
 Flux is a tool for keeping Kubernetes clusters in sync with sources of
-configuration (like Git repositories), and automating updates to
-configuration when there is new code to deploy.
+configuration (like Git repositories and OCI artifacts),
+and automating updates to configuration when there is new code to deploy.
 
 Flux version 2 ("v2") is built from the ground up to use Kubernetes'
 API extension system, and to integrate with Prometheus and other core
@@ -20,18 +19,19 @@ Flux v2 is constructed with the [GitOps Toolkit](#gitops-toolkit), a
 set of composable APIs and specialized tools for building Continuous
 Delivery on top of Kubernetes.
 
-Flux is a Cloud Native Computing Foundation ([CNCF](https://www.cncf.io/)) project.
+Flux is a Cloud Native Computing Foundation ([CNCF](https://www.cncf.io/)) project, used in
+production by various [organisations](https://fluxcd.io/adopters) and [cloud providers](https://fluxcd.io/ecosystem).
 
 ## Quickstart and documentation
 
-To get started check out this [guide](https://fluxcd.io/docs/get-started/)
+To get started check out this [guide](https://fluxcd.io/flux/get-started/)
 on how to bootstrap Flux on Kubernetes and deploy a sample application in a GitOps manner.
 
 For more comprehensive documentation, see the following guides:
-- [Ways of structuring your repositories](https://fluxcd.io/docs/guides/repository-structure/)
-- [Manage Helm Releases](https://fluxcd.io/docs/guides/helmreleases/)
-- [Automate image updates to Git](https://fluxcd.io/docs/guides/image-update/)  
-- [Manage Kubernetes secrets with Mozilla SOPS](https://fluxcd.io/docs/guides/mozilla-sops/)  
+- [Ways of structuring your repositories](https://fluxcd.io/flux/guides/repository-structure/)
+- [Manage Helm Releases](https://fluxcd.io/flux/guides/helmreleases/)
+- [Automate image updates to Git](https://fluxcd.io/flux/guides/image-update/)  
+- [Manage Kubernetes secrets with Mozilla SOPS](https://fluxcd.io/flux/guides/mozilla-sops/)  
 
 If you need help, please refer to our **[Support page](https://fluxcd.io/support/)**.
 
@@ -46,27 +46,28 @@ automation tooling.
 
 You can use the toolkit to extend Flux, or to build your own systems
 for continuous delivery -- see [the developer
-guides](https://fluxcd.io/docs/gitops-toolkit/source-watcher/).
+guides](https://fluxcd.io/flux/gitops-toolkit/source-watcher/).
 
 ### Components
 
-- [Source Controller](https://fluxcd.io/docs/components/source/)
-    - [GitRepository CRD](https://fluxcd.io/docs/components/source/gitrepositories/)
-    - [HelmRepository CRD](https://fluxcd.io/docs/components/source/helmrepositories/)
-    - [HelmChart CRD](https://fluxcd.io/docs/components/source/helmcharts/)
-    - [Bucket CRD](https://fluxcd.io/docs/components/source/buckets/)
-- [Kustomize Controller](https://fluxcd.io/docs/components/kustomize/)
-    - [Kustomization CRD](https://fluxcd.io/docs/components/kustomize/kustomization/)
-- [Helm Controller](https://fluxcd.io/docs/components/helm/)
-    - [HelmRelease CRD](https://fluxcd.io/docs/components/helm/helmreleases/)
-- [Notification Controller](https://fluxcd.io/docs/components/notification/)
-    - [Provider CRD](https://fluxcd.io/docs/components/notification/provider/)
-    - [Alert CRD](https://fluxcd.io/docs/components/notification/alert/)
-    - [Receiver CRD](https://fluxcd.io/docs/components/notification/receiver/)
-- [Image Automation Controllers](https://fluxcd.io/docs/components/image/)
-  - [ImageRepository CRD](https://fluxcd.io/docs/components/image/imagerepositories/)
-  - [ImagePolicy CRD](https://fluxcd.io/docs/components/image/imagepolicies/)
-  - [ImageUpdateAutomation CRD](https://fluxcd.io/docs/components/image/imageupdateautomations/)
+- [Source Controller](https://fluxcd.io/flux/components/source/)
+    - [GitRepository CRD](https://fluxcd.io/flux/components/source/gitrepositories/)
+    - [OCIRepository CRD](https://fluxcd.io/flux/components/source/ocirepositories/)
+    - [HelmRepository CRD](https://fluxcd.io/flux/components/source/helmrepositories/)
+    - [HelmChart CRD](https://fluxcd.io/flux/components/source/helmcharts/)
+    - [Bucket CRD](https://fluxcd.io/flux/components/source/buckets/)
+- [Kustomize Controller](https://fluxcd.io/flux/components/kustomize/)
+    - [Kustomization CRD](https://fluxcd.io/flux/components/kustomize/kustomization/)
+- [Helm Controller](https://fluxcd.io/flux/components/helm/)
+    - [HelmRelease CRD](https://fluxcd.io/flux/components/helm/helmreleases/)
+- [Notification Controller](https://fluxcd.io/flux/components/notification/)
+    - [Provider CRD](https://fluxcd.io/flux/components/notification/provider/)
+    - [Alert CRD](https://fluxcd.io/flux/components/notification/alert/)
+    - [Receiver CRD](https://fluxcd.io/flux/components/notification/receiver/)
+- [Image Automation Controllers](https://fluxcd.io/flux/components/image/)
+  - [ImageRepository CRD](https://fluxcd.io/flux/components/image/imagerepositories/)
+  - [ImagePolicy CRD](https://fluxcd.io/flux/components/image/imagepolicies/)
+  - [ImageUpdateAutomation CRD](https://fluxcd.io/flux/components/image/imageupdateautomations/)
   
 ## Community
 
@@ -74,18 +75,19 @@ Need help or want to contribute? Please see the links below. The Flux project is
 new contributors and there are a multitude of ways to get involved.
 
 - Getting Started?
-    - Look at our [Get Started guide](https://fluxcd.io/docs/get-started/) and give us feedback
+    - Look at our [Get Started guide](https://fluxcd.io/flux/get-started/) and give us feedback
 - Need help?
-    - First: Ask questions on our [GH Discussions page](https://github.com/fluxcd/flux2/discussions)
-    - Second: Talk to us in the #flux channel on [CNCF Slack](https://slack.cncf.io/)
+    - First: Ask questions on our [GH Discussions page](https://github.com/fluxcd/flux2/discussions).
+    - Second: Talk to us in the #flux channel on [CNCF Slack](https://slack.cncf.io/).
     - Please follow our [Support Guidelines](https://fluxcd.io/support/)
       (in short: be nice, be respectful of volunteers' time, understand that maintainers and
       contributors cannot respond to all DMs, and keep discussions in the public #flux channel as much as possible).
 - Have feature proposals or want to contribute?
-    - Propose features on our [GH Discussions page](https://github.com/fluxcd/flux2/discussions)
-    - Join our upcoming dev meetings ([meeting access and agenda](https://docs.google.com/document/d/1l_M0om0qUEN_NNiGgpqJ2tvsF2iioHkaARDeh6b70B0/view))
+    - Propose features on our [GitHub Discussions page](https://github.com/fluxcd/flux2/discussions).
+    - Join our upcoming dev meetings ([meeting access and agenda](https://docs.google.com/document/d/1l_M0om0qUEN_NNiGgpqJ2tvsF2iioHkaARDeh6b70B0/view)).
     - [Join the flux-dev mailing list](https://lists.cncf.io/g/cncf-flux-dev).
-    - Check out [how to contribute](CONTRIBUTING.md) to the project
+    - Check out [how to contribute](CONTRIBUTING.md) to the project.
+    - Check out the [project roadmap](https://fluxcd.io/roadmap/).
 
 ### Events
 

cmd/flux/bootstrap_bitbucket_server.go

@@ -22,14 +22,14 @@ import (
 	"os"
 	"time"
 
-	"github.com/go-git/go-git/v5/plumbing/transport/http"
+	"github.com/fluxcd/pkg/git"
+	"github.com/fluxcd/pkg/git/gogit"
 	"github.com/spf13/cobra"
 
-	"github.com/fluxcd/flux2/internal/bootstrap"
-	"github.com/fluxcd/flux2/internal/bootstrap/git/gogit"
-	"github.com/fluxcd/flux2/internal/bootstrap/provider"
 	"github.com/fluxcd/flux2/internal/flags"
 	"github.com/fluxcd/flux2/internal/utils"
+	"github.com/fluxcd/flux2/pkg/bootstrap"
+	"github.com/fluxcd/flux2/pkg/bootstrap/provider"
 	"github.com/fluxcd/flux2/pkg/manifestgen"
 	"github.com/fluxcd/flux2/pkg/manifestgen/install"
 	"github.com/fluxcd/flux2/pkg/manifestgen/sourcesecret"
@@ -171,10 +171,16 @@ func bootstrapBServerCmdRun(cmd *cobra.Command, args []string) error {
 		return fmt.Errorf("failed to create temporary working dir: %w", err)
 	}
 	defer os.RemoveAll(tmpDir)
-	gitClient := gogit.New(tmpDir, &http.BasicAuth{
-		Username: user,
-		Password: bitbucketToken,
+
+	gitClient, err := gogit.NewClient(tmpDir, &git.AuthOptions{
+		Transport: git.HTTPS,
+		Username:  user,
+		Password:  bitbucketToken,
+		CAFile:    caBundle,
 	})
+	if err != nil {
+		return err
+	}
 
 	// Install manifest config
 	installOptions := install.Options{
@@ -212,19 +218,18 @@ func bootstrapBServerCmdRun(cmd *cobra.Command, args []string) error {
 			secretOpts.Username = bServerArgs.username
 		}
 		secretOpts.Password = bitbucketToken
-
-		if bootstrapArgs.caFile != "" {
-			secretOpts.CAFilePath = bootstrapArgs.caFile
-		}
+		secretOpts.CAFile = caBundle
 	} else {
+		keypair, err := sourcesecret.LoadKeyPairFromPath(bootstrapArgs.privateKeyFile, gitArgs.password)
+		if err != nil {
+			return err
+		}
+		secretOpts.Keypair = keypair
 		secretOpts.PrivateKeyAlgorithm = sourcesecret.PrivateKeyAlgorithm(bootstrapArgs.keyAlgorithm)
 		secretOpts.RSAKeyBits = int(bootstrapArgs.keyRSABits)
 		secretOpts.ECDSACurve = bootstrapArgs.keyECDSACurve.Curve
-		secretOpts.SSHHostname = bServerArgs.hostname
 
-		if bootstrapArgs.privateKeyFile != "" {
-			secretOpts.PrivateKeyPath = bootstrapArgs.privateKeyFile
-		}
+		secretOpts.SSHHostname = bServerArgs.hostname
 		if bootstrapArgs.sshHostname != "" {
 			secretOpts.SSHHostname = bootstrapArgs.sshHostname
 		}
@@ -243,19 +248,24 @@ func bootstrapBServerCmdRun(cmd *cobra.Command, args []string) error {
 		RecurseSubmodules: bootstrapArgs.recurseSubmodules,
 	}
 
+	entityList, err := bootstrap.LoadEntityListFromPath(bootstrapArgs.gpgKeyRingPath)
+	if err != nil {
+		return err
+	}
+
 	// Bootstrap config
+
 	bootstrapOpts := []bootstrap.GitProviderOption{
 		bootstrap.WithProviderRepository(bServerArgs.owner, bServerArgs.repository, bServerArgs.personal),
 		bootstrap.WithBranch(bootstrapArgs.branch),
 		bootstrap.WithBootstrapTransportType("https"),
-		bootstrap.WithAuthor(bootstrapArgs.authorName, bootstrapArgs.authorEmail),
+		bootstrap.WithSignature(bootstrapArgs.authorName, bootstrapArgs.authorEmail),
 		bootstrap.WithCommitMessageAppendix(bootstrapArgs.commitMessageAppendix),
 		bootstrap.WithProviderTeamPermissions(mapTeamSlice(bServerArgs.teams, bServerDefaultPermission)),
 		bootstrap.WithReadWriteKeyPermissions(bServerArgs.readWriteKey),
 		bootstrap.WithKubeconfig(kubeconfigArgs, kubeclientOptions),
 		bootstrap.WithLogger(logger),
-		bootstrap.WithCABundle(caBundle),
-		bootstrap.WithGitCommitSigning(bootstrapArgs.gpgKeyRingPath, bootstrapArgs.gpgPassphrase, bootstrapArgs.gpgKeyID),
+		bootstrap.WithGitCommitSigning(entityList, bootstrapArgs.gpgPassphrase, bootstrapArgs.gpgKeyID),
 	}
 	if bootstrapArgs.sshHostname != "" {
 		bootstrapOpts = append(bootstrapOpts, bootstrap.WithSSHHostname(bootstrapArgs.sshHostname))

cmd/flux/bootstrap_git.go

@@ -24,21 +24,19 @@ import (
 	"strings"
 	"time"
 
-	"github.com/go-git/go-git/v5/plumbing/transport"
-	"github.com/go-git/go-git/v5/plumbing/transport/http"
-	"github.com/go-git/go-git/v5/plumbing/transport/ssh"
 	"github.com/manifoldco/promptui"
 	"github.com/spf13/cobra"
 	corev1 "k8s.io/api/core/v1"
 
-	"github.com/fluxcd/flux2/internal/bootstrap"
-	"github.com/fluxcd/flux2/internal/bootstrap/git/gogit"
 	"github.com/fluxcd/flux2/internal/flags"
 	"github.com/fluxcd/flux2/internal/utils"
+	"github.com/fluxcd/flux2/pkg/bootstrap"
 	"github.com/fluxcd/flux2/pkg/manifestgen"
 	"github.com/fluxcd/flux2/pkg/manifestgen/install"
 	"github.com/fluxcd/flux2/pkg/manifestgen/sourcesecret"
 	"github.com/fluxcd/flux2/pkg/manifestgen/sync"
+	"github.com/fluxcd/pkg/git"
+	"github.com/fluxcd/pkg/git/gogit"
 )
 
 var bootstrapGitCmd = &cobra.Command{
@@ -62,6 +60,12 @@ command will perform an upgrade if needed.`,
 
   # Run bootstrap for a Git repository with a private key and password
   flux bootstrap git --url=ssh://git@example.com/repository.git --private-key-file=<path/to/private.key> --password=<password>
+
+  # Run bootstrap for a Git repository on AWS CodeCommit
+  flux bootstrap git --url=ssh://<SSH-Key-ID>@git-codecommit.<region>.amazonaws.com/v1/repos/<repository> --private-key-file=<path/to/private.key> --password=<SSH-passphrase>
+
+  # Run bootstrap for a Git repository on Azure Devops
+  flux bootstrap git --url=ssh://git@ssh.dev.azure.com/v3/<org>/<project>/<repository> --ssh-key-algorithm=rsa --ssh-rsa-bits=4096
 `,
 	RunE: bootstrapGitCmdRun,
 }
@@ -116,9 +120,22 @@ func bootstrapGitCmdRun(cmd *cobra.Command, args []string) error {
 	if err != nil {
 		return err
 	}
-	gitAuth, err := transportForURL(repositoryURL)
-	if err != nil {
-		return err
+
+	if strings.Contains(repositoryURL.Hostname(), "git-codecommit") && strings.Contains(repositoryURL.Hostname(), "amazonaws.com") {
+		if repositoryURL.Scheme == string(git.SSH) {
+			if repositoryURL.User == nil {
+				return fmt.Errorf("invalid AWS CodeCommit url: ssh username should be specified in the url")
+			}
+			if repositoryURL.User.Username() == git.DefaultPublicKeyAuthUser {
+				return fmt.Errorf("invalid AWS CodeCommit url: ssh username should be the SSH key ID for the provided private key")
+			}
+			if bootstrapArgs.privateKeyFile == "" {
+				return fmt.Errorf("private key file is required for bootstrapping against AWS CodeCommit using ssh")
+			}
+		}
+		if repositoryURL.Scheme == string(git.HTTPS) && !bootstrapArgs.tokenAuth {
+			return fmt.Errorf("--token-auth=true must be specified for using a HTTPS AWS CodeCommit url")
+		}
 	}
 
 	ctx, cancel := context.WithTimeout(context.Background(), rootArgs.timeout)
@@ -145,7 +162,28 @@ func bootstrapGitCmdRun(cmd *cobra.Command, args []string) error {
 		return fmt.Errorf("failed to create temporary working dir: %w", err)
 	}
 	defer os.RemoveAll(tmpDir)
-	gitClient := gogit.New(tmpDir, gitAuth)
+
+	var caBundle []byte
+	if bootstrapArgs.caFile != "" {
+		var err error
+		caBundle, err = os.ReadFile(bootstrapArgs.caFile)
+		if err != nil {
+			return fmt.Errorf("unable to read TLS CA file: %w", err)
+		}
+	}
+	authOpts, err := getAuthOpts(repositoryURL, caBundle)
+	if err != nil {
+		return fmt.Errorf("failed to create authentication options for %s: %w", repositoryURL.String(), err)
+	}
+
+	clientOpts := []gogit.ClientOption{gogit.WithDiskStorage()}
+	if gitArgs.insecureHttpAllowed {
+		clientOpts = append(clientOpts, gogit.WithInsecureCredentialsOverHTTP())
+	}
+	gitClient, err := gogit.NewClient(tmpDir, authOpts, clientOpts...)
+	if err != nil {
+		return fmt.Errorf("failed to create a Git client: %w", err)
+	}
 
 	// Install manifest config
 	installOptions := install.Options{
@@ -179,10 +217,7 @@ func bootstrapGitCmdRun(cmd *cobra.Command, args []string) error {
 	if bootstrapArgs.tokenAuth {
 		secretOpts.Username = gitArgs.username
 		secretOpts.Password = gitArgs.password
-
-		if bootstrapArgs.caFile != "" {
-			secretOpts.CAFilePath = bootstrapArgs.caFile
-		}
+		secretOpts.CAFile = caBundle
 
 		// Remove port of the given host when not syncing over HTTP/S to not assume port for protocol
 		// This _might_ be overwritten later on by e.g. --ssh-hostname
@@ -192,7 +227,9 @@ func bootstrapGitCmdRun(cmd *cobra.Command, args []string) error {
 
 		// Configure repository URL to match auth config for sync.
 		repositoryURL.User = nil
-		repositoryURL.Scheme = "https"
+		if !gitArgs.insecureHttpAllowed {
+			repositoryURL.Scheme = "https"
+		}
 	} else {
 		secretOpts.PrivateKeyAlgorithm = sourcesecret.PrivateKeyAlgorithm(bootstrapArgs.keyAlgorithm)
 		secretOpts.Password = gitArgs.password
@@ -211,9 +248,12 @@ func bootstrapGitCmdRun(cmd *cobra.Command, args []string) error {
 		if bootstrapArgs.sshHostname != "" {
 			repositoryURL.Host = bootstrapArgs.sshHostname
 		}
-		if bootstrapArgs.privateKeyFile != "" {
-			secretOpts.PrivateKeyPath = bootstrapArgs.privateKeyFile
+
+		keypair, err := sourcesecret.LoadKeyPairFromPath(bootstrapArgs.privateKeyFile, gitArgs.password)
+		if err != nil {
+			return err
 		}
+		secretOpts.Keypair = keypair
 
 		// Configure last as it depends on the config above.
 		secretOpts.SSHHostname = repositoryURL.Host
@@ -233,26 +273,21 @@ func bootstrapGitCmdRun(cmd *cobra.Command, args []string) error {
 		RecurseSubmodules: bootstrapArgs.recurseSubmodules,
 	}
 
-	var caBundle []byte
-	if bootstrapArgs.caFile != "" {
-		var err error
-		caBundle, err = os.ReadFile(bootstrapArgs.caFile)
-		if err != nil {
-			return fmt.Errorf("unable to read TLS CA file: %w", err)
-		}
+	entityList, err := bootstrap.LoadEntityListFromPath(bootstrapArgs.gpgKeyRingPath)
+	if err != nil {
+		return err
 	}
 
 	// Bootstrap config
 	bootstrapOpts := []bootstrap.GitOption{
 		bootstrap.WithRepositoryURL(gitArgs.url),
 		bootstrap.WithBranch(bootstrapArgs.branch),
-		bootstrap.WithAuthor(bootstrapArgs.authorName, bootstrapArgs.authorEmail),
+		bootstrap.WithSignature(bootstrapArgs.authorName, bootstrapArgs.authorEmail),
 		bootstrap.WithCommitMessageAppendix(bootstrapArgs.commitMessageAppendix),
 		bootstrap.WithKubeconfig(kubeconfigArgs, kubeclientOptions),
 		bootstrap.WithPostGenerateSecretFunc(promptPublicKey),
 		bootstrap.WithLogger(logger),
-		bootstrap.WithCABundle(caBundle),
-		bootstrap.WithGitCommitSigning(bootstrapArgs.gpgKeyRingPath, bootstrapArgs.gpgPassphrase, bootstrapArgs.gpgKeyID),
+		bootstrap.WithGitCommitSigning(entityList, bootstrapArgs.gpgPassphrase, bootstrapArgs.gpgKeyID),
 	}
 
 	// Setup bootstrapper with constructed configs
@@ -265,28 +300,45 @@ func bootstrapGitCmdRun(cmd *cobra.Command, args []string) error {
 	return bootstrap.Run(ctx, b, manifestsBase, installOptions, secretOpts, syncOpts, rootArgs.pollInterval, rootArgs.timeout)
 }
 
-// transportForURL constructs a transport.AuthMethod based on the scheme
+// getAuthOpts retruns a AuthOptions based on the scheme
 // of the given URL and the configured flags. If the protocol equals
 // "ssh" but no private key is configured, authentication using the local
 // SSH-agent is attempted.
-func transportForURL(u *url.URL) (transport.AuthMethod, error) {
+func getAuthOpts(u *url.URL, caBundle []byte) (*git.AuthOptions, error) {
 	switch u.Scheme {
 	case "http":
 		if !gitArgs.insecureHttpAllowed {
 			return nil, fmt.Errorf("scheme http is insecure, pass --allow-insecure-http=true to allow it")
 		}
-		return &http.BasicAuth{
-			Username: gitArgs.username,
-			Password: gitArgs.password,
+		return &git.AuthOptions{
+			Transport: git.HTTP,
+			Username:  gitArgs.username,
+			Password:  gitArgs.password,
 		}, nil
 	case "https":
-		return &http.BasicAuth{
-			Username: gitArgs.username,
-			Password: gitArgs.password,
+		return &git.AuthOptions{
+			Transport: git.HTTPS,
+			Username:  gitArgs.username,
+			Password:  gitArgs.password,
+			CAFile:    caBundle,
 		}, nil
 	case "ssh":
 		if bootstrapArgs.privateKeyFile != "" {
-			return ssh.NewPublicKeysFromFile(u.User.Username(), bootstrapArgs.privateKeyFile, gitArgs.password)
+			pk, err := os.ReadFile(bootstrapArgs.privateKeyFile)
+			if err != nil {
+				return nil, err
+			}
+			kh, err := sourcesecret.ScanHostKey(u.Host)
+			if err != nil {
+				return nil, err
+			}
+			return &git.AuthOptions{
+				Transport:  git.SSH,
+				Username:   u.User.Username(),
+				Password:   gitArgs.password,
+				Identity:   pk,
+				KnownHosts: kh,
+			}, nil
 		}
 		return nil, nil
 	default:

cmd/flux/bootstrap_github.go

@@ -22,14 +22,14 @@ import (
 	"os"
 	"time"
 
-	"github.com/go-git/go-git/v5/plumbing/transport/http"
+	"github.com/fluxcd/pkg/git"
+	"github.com/fluxcd/pkg/git/gogit"
 	"github.com/spf13/cobra"
 
-	"github.com/fluxcd/flux2/internal/bootstrap"
-	"github.com/fluxcd/flux2/internal/bootstrap/git/gogit"
-	"github.com/fluxcd/flux2/internal/bootstrap/provider"
 	"github.com/fluxcd/flux2/internal/flags"
 	"github.com/fluxcd/flux2/internal/utils"
+	"github.com/fluxcd/flux2/pkg/bootstrap"
+	"github.com/fluxcd/flux2/pkg/bootstrap/provider"
 	"github.com/fluxcd/flux2/pkg/manifestgen"
 	"github.com/fluxcd/flux2/pkg/manifestgen/install"
 	"github.com/fluxcd/flux2/pkg/manifestgen/sourcesecret"
@@ -161,16 +161,21 @@ func bootstrapGitHubCmdRun(cmd *cobra.Command, args []string) error {
 		return err
 	}
 
-	// Lazy go-git repository
 	tmpDir, err := manifestgen.MkdirTempAbs("", "flux-bootstrap-")
 	if err != nil {
 		return fmt.Errorf("failed to create temporary working dir: %w", err)
 	}
 	defer os.RemoveAll(tmpDir)
-	gitClient := gogit.New(tmpDir, &http.BasicAuth{
-		Username: githubArgs.owner,
-		Password: ghToken,
+
+	gitClient, err := gogit.NewClient(tmpDir, &git.AuthOptions{
+		Transport: git.HTTPS,
+		Username:  githubArgs.owner,
+		Password:  ghToken,
+		CAFile:    caBundle,
 	})
+	if err != nil {
+		return err
+	}
 
 	// Install manifest config
 	installOptions := install.Options{
@@ -204,16 +209,13 @@ func bootstrapGitHubCmdRun(cmd *cobra.Command, args []string) error {
 	if bootstrapArgs.tokenAuth {
 		secretOpts.Username = "git"
 		secretOpts.Password = ghToken
-
-		if bootstrapArgs.caFile != "" {
-			secretOpts.CAFilePath = bootstrapArgs.caFile
-		}
+		secretOpts.CAFile = caBundle
 	} else {
 		secretOpts.PrivateKeyAlgorithm = sourcesecret.PrivateKeyAlgorithm(bootstrapArgs.keyAlgorithm)
 		secretOpts.RSAKeyBits = int(bootstrapArgs.keyRSABits)
 		secretOpts.ECDSACurve = bootstrapArgs.keyECDSACurve.Curve
-		secretOpts.SSHHostname = githubArgs.hostname
 
+		secretOpts.SSHHostname = githubArgs.hostname
 		if bootstrapArgs.sshHostname != "" {
 			secretOpts.SSHHostname = bootstrapArgs.sshHostname
 		}
@@ -232,19 +234,23 @@ func bootstrapGitHubCmdRun(cmd *cobra.Command, args []string) error {
 		RecurseSubmodules: bootstrapArgs.recurseSubmodules,
 	}
 
+	entityList, err := bootstrap.LoadEntityListFromPath(bootstrapArgs.gpgKeyRingPath)
+	if err != nil {
+		return err
+	}
+
 	// Bootstrap config
 	bootstrapOpts := []bootstrap.GitProviderOption{
 		bootstrap.WithProviderRepository(githubArgs.owner, githubArgs.repository, githubArgs.personal),
 		bootstrap.WithBranch(bootstrapArgs.branch),
 		bootstrap.WithBootstrapTransportType("https"),
-		bootstrap.WithAuthor(bootstrapArgs.authorName, bootstrapArgs.authorEmail),
+		bootstrap.WithSignature(bootstrapArgs.authorName, bootstrapArgs.authorEmail),
 		bootstrap.WithCommitMessageAppendix(bootstrapArgs.commitMessageAppendix),
 		bootstrap.WithProviderTeamPermissions(mapTeamSlice(githubArgs.teams, ghDefaultPermission)),
 		bootstrap.WithReadWriteKeyPermissions(githubArgs.readWriteKey),
 		bootstrap.WithKubeconfig(kubeconfigArgs, kubeclientOptions),
 		bootstrap.WithLogger(logger),
-		bootstrap.WithCABundle(caBundle),
-		bootstrap.WithGitCommitSigning(bootstrapArgs.gpgKeyRingPath, bootstrapArgs.gpgPassphrase, bootstrapArgs.gpgKeyID),
+		bootstrap.WithGitCommitSigning(entityList, bootstrapArgs.gpgPassphrase, bootstrapArgs.gpgKeyID),
 	}
 	if bootstrapArgs.sshHostname != "" {
 		bootstrapOpts = append(bootstrapOpts, bootstrap.WithSSHHostname(bootstrapArgs.sshHostname))

cmd/flux/bootstrap_gitlab.go

@@ -24,14 +24,14 @@ import (
 	"strings"
 	"time"
 
-	"github.com/go-git/go-git/v5/plumbing/transport/http"
+	"github.com/fluxcd/pkg/git"
+	"github.com/fluxcd/pkg/git/gogit"
 	"github.com/spf13/cobra"
 
-	"github.com/fluxcd/flux2/internal/bootstrap"
-	"github.com/fluxcd/flux2/internal/bootstrap/git/gogit"
-	"github.com/fluxcd/flux2/internal/bootstrap/provider"
 	"github.com/fluxcd/flux2/internal/flags"
 	"github.com/fluxcd/flux2/internal/utils"
+	"github.com/fluxcd/flux2/pkg/bootstrap"
+	"github.com/fluxcd/flux2/pkg/bootstrap/provider"
 	"github.com/fluxcd/flux2/pkg/manifestgen"
 	"github.com/fluxcd/flux2/pkg/manifestgen/install"
 	"github.com/fluxcd/flux2/pkg/manifestgen/sourcesecret"
@@ -178,10 +178,16 @@ func bootstrapGitLabCmdRun(cmd *cobra.Command, args []string) error {
 		return fmt.Errorf("failed to create temporary working dir: %w", err)
 	}
 	defer os.RemoveAll(tmpDir)
-	gitClient := gogit.New(tmpDir, &http.BasicAuth{
-		Username: gitlabArgs.owner,
-		Password: glToken,
+
+	gitClient, err := gogit.NewClient(tmpDir, &git.AuthOptions{
+		Transport: git.HTTPS,
+		Username:  gitlabArgs.owner,
+		Password:  glToken,
+		CAFile:    caBundle,
 	})
+	if err != nil {
+		return err
+	}
 
 	// Install manifest config
 	installOptions := install.Options{
@@ -215,19 +221,18 @@ func bootstrapGitLabCmdRun(cmd *cobra.Command, args []string) error {
 	if bootstrapArgs.tokenAuth {
 		secretOpts.Username = "git"
 		secretOpts.Password = glToken
-
-		if bootstrapArgs.caFile != "" {
-			secretOpts.CAFilePath = bootstrapArgs.caFile
-		}
+		secretOpts.CAFile = caBundle
 	} else {
+		keypair, err := sourcesecret.LoadKeyPairFromPath(bootstrapArgs.privateKeyFile, gitArgs.password)
+		if err != nil {
+			return err
+		}
+		secretOpts.Keypair = keypair
 		secretOpts.PrivateKeyAlgorithm = sourcesecret.PrivateKeyAlgorithm(bootstrapArgs.keyAlgorithm)
 		secretOpts.RSAKeyBits = int(bootstrapArgs.keyRSABits)
 		secretOpts.ECDSACurve = bootstrapArgs.keyECDSACurve.Curve
-		secretOpts.SSHHostname = gitlabArgs.hostname
 
-		if bootstrapArgs.privateKeyFile != "" {
-			secretOpts.PrivateKeyPath = bootstrapArgs.privateKeyFile
-		}
+		secretOpts.SSHHostname = gitlabArgs.hostname
 		if bootstrapArgs.sshHostname != "" {
 			secretOpts.SSHHostname = bootstrapArgs.sshHostname
 		}
@@ -246,19 +251,23 @@ func bootstrapGitLabCmdRun(cmd *cobra.Command, args []string) error {
 		RecurseSubmodules: bootstrapArgs.recurseSubmodules,
 	}
 
+	entityList, err := bootstrap.LoadEntityListFromPath(bootstrapArgs.gpgKeyRingPath)
+	if err != nil {
+		return err
+	}
+
 	// Bootstrap config
 	bootstrapOpts := []bootstrap.GitProviderOption{
 		bootstrap.WithProviderRepository(gitlabArgs.owner, gitlabArgs.repository, gitlabArgs.personal),
 		bootstrap.WithBranch(bootstrapArgs.branch),
 		bootstrap.WithBootstrapTransportType("https"),
-		bootstrap.WithAuthor(bootstrapArgs.authorName, bootstrapArgs.authorEmail),
+		bootstrap.WithSignature(bootstrapArgs.authorName, bootstrapArgs.authorEmail),
 		bootstrap.WithCommitMessageAppendix(bootstrapArgs.commitMessageAppendix),
 		bootstrap.WithProviderTeamPermissions(mapTeamSlice(gitlabArgs.teams, glDefaultPermission)),
 		bootstrap.WithReadWriteKeyPermissions(gitlabArgs.readWriteKey),
 		bootstrap.WithKubeconfig(kubeconfigArgs, kubeclientOptions),
 		bootstrap.WithLogger(logger),
-		bootstrap.WithCABundle(caBundle),
-		bootstrap.WithGitCommitSigning(bootstrapArgs.gpgKeyRingPath, bootstrapArgs.gpgPassphrase, bootstrapArgs.gpgKeyID),
+		bootstrap.WithGitCommitSigning(entityList, bootstrapArgs.gpgPassphrase, bootstrapArgs.gpgKeyID),
 	}
 	if bootstrapArgs.sshHostname != "" {
 		bootstrapOpts = append(bootstrapOpts, bootstrap.WithSSHHostname(bootstrapArgs.sshHostname))

cmd/flux/build_artifact.go

@@ -19,19 +19,24 @@ package main
 import (
 	"fmt"
 	"os"
+	"strings"
 
 	"github.com/spf13/cobra"
 
 	oci "github.com/fluxcd/pkg/oci/client"
+	"github.com/fluxcd/pkg/sourceignore"
 )
 
 var buildArtifactCmd = &cobra.Command{
 	Use:   "artifact",
 	Short: "Build artifact",
-	Long:  `The build artifact command creates a tgz file with the manifests from the given directory.`,
+	Long:  `The build artifact command creates a tgz file with the manifests from the given directory or a single manifest file.`,
 	Example: `  # Build the given manifests directory into an artifact
   flux build artifact --path ./path/to/local/manifests --output ./path/to/artifact.tgz
 
+  # Build the given single manifest file into an artifact
+  flux build artifact --path ./path/to/local/manifest.yaml --output ./path/to/artifact.tgz
+
   # List the files bundled in the artifact
   tar -ztvf ./path/to/artifact.tgz
 `,
@@ -39,15 +44,20 @@ var buildArtifactCmd = &cobra.Command{
 }
 
 type buildArtifactFlags struct {
-	output string
-	path   string
+	output      string
+	path        string
+	ignorePaths []string
 }
 
+var excludeOCI = append(strings.Split(sourceignore.ExcludeVCS, ","), strings.Split(sourceignore.ExcludeExt, ",")...)
+
 var buildArtifactArgs buildArtifactFlags
 
 func init() {
 	buildArtifactCmd.Flags().StringVar(&buildArtifactArgs.path, "path", "", "Path to the directory where the Kubernetes manifests are located.")
 	buildArtifactCmd.Flags().StringVarP(&buildArtifactArgs.output, "output", "o", "artifact.tgz", "Path to where the artifact tgz file should be written.")
+	buildArtifactCmd.Flags().StringSliceVar(&buildArtifactArgs.ignorePaths, "ignore-paths", excludeOCI, "set paths to ignore in .gitignore format")
+
 	buildCmd.AddCommand(buildArtifactCmd)
 }
 
@@ -56,14 +66,14 @@ func buildArtifactCmdRun(cmd *cobra.Command, args []string) error {
 		return fmt.Errorf("invalid path %q", buildArtifactArgs.path)
 	}
 
-	if fs, err := os.Stat(buildArtifactArgs.path); err != nil || !fs.IsDir() {
-		return fmt.Errorf("invalid path '%s', must point to an existing directory", buildArtifactArgs.path)
+	if _, err := os.Stat(buildArtifactArgs.path); err != nil {
+		return fmt.Errorf("invalid path '%s', must point to an existing directory or file", buildArtifactArgs.path)
 	}
 
 	logger.Actionf("building artifact from %s", buildArtifactArgs.path)
 
 	ociClient := oci.NewLocalClient()
-	if err := ociClient.Build(buildArtifactArgs.output, buildArtifactArgs.path); err != nil {
+	if err := ociClient.Build(buildArtifactArgs.output, buildArtifactArgs.path, buildArtifactArgs.ignorePaths); err != nil {
 		return fmt.Errorf("bulding artifact failed, error: %w", err)
 	}
 

cmd/flux/build_kustomization.go

@@ -40,7 +40,11 @@ It is possible to specify a Flux kustomization file using --kustomization-file.`
 flux build kustomization my-app --path ./path/to/local/manifests
 
 # Build using a local flux kustomization file
-flux build kustomization my-app --path ./path/to/local/manifests --kustomization-file ./path/to/local/my-app.yaml`,
+flux build kustomization my-app --path ./path/to/local/manifests --kustomization-file ./path/to/local/my-app.yaml
+
+# Build in dry-run mode without connecting to the cluster.
+# Note that variable substitutions from Secrets and ConfigMaps are skipped in dry-run mode.
+flux build kustomization my-app --path ./path/to/local/manifests --kustomization-file ./path/to/local/my-app.yaml --dry-run`,
 	ValidArgsFunction: resourceNamesCompletionFunc(kustomizev1.GroupVersion.WithKind(kustomizev1.KustomizationKind)),
 	RunE:              buildKsCmdRun,
 }
@@ -48,6 +52,7 @@ flux build kustomization my-app --path ./path/to/local/manifests --kustomization
 type buildKsFlags struct {
 	kustomizationFile string
 	path              string
+	dryRun            bool
 }
 
 var buildKsArgs buildKsFlags
@@ -55,10 +60,11 @@ var buildKsArgs buildKsFlags
 func init() {
 	buildKsCmd.Flags().StringVar(&buildKsArgs.path, "path", "", "Path to the manifests location.")
 	buildKsCmd.Flags().StringVar(&buildKsArgs.kustomizationFile, "kustomization-file", "", "Path to the Flux Kustomization YAML file.")
+	buildKsCmd.Flags().BoolVar(&buildKsArgs.dryRun, "dry-run", false, "Dry run mode.")
 	buildCmd.AddCommand(buildKsCmd)
 }
 
-func buildKsCmdRun(cmd *cobra.Command, args []string) error {
+func buildKsCmdRun(cmd *cobra.Command, args []string) (err error) {
 	if len(args) < 1 {
 		return fmt.Errorf("%s name is required", kustomizationType.humanKind)
 	}
@@ -72,13 +78,22 @@ func buildKsCmdRun(cmd *cobra.Command, args []string) error {
 		return fmt.Errorf("invalid resource path %q", buildKsArgs.path)
 	}
 
+	if buildKsArgs.dryRun && buildKsArgs.kustomizationFile == "" {
+		return fmt.Errorf("dry-run mode requires a kustomization file")
+	}
+
 	if buildKsArgs.kustomizationFile != "" {
 		if fs, err := os.Stat(buildKsArgs.kustomizationFile); os.IsNotExist(err) || fs.IsDir() {
 			return fmt.Errorf("invalid kustomization file %q", buildKsArgs.kustomizationFile)
 		}
 	}
 
-	builder, err := build.NewBuilder(kubeconfigArgs, kubeclientOptions, name, buildKsArgs.path, build.WithTimeout(rootArgs.timeout), build.WithKustomizationFile(buildKsArgs.kustomizationFile))
+	builder, err := build.NewBuilder(name, buildKsArgs.path,
+		build.WithClientConfig(kubeconfigArgs, kubeclientOptions),
+		build.WithTimeout(rootArgs.timeout),
+		build.WithKustomizationFile(buildKsArgs.kustomizationFile),
+		build.WithDryRun(buildKsArgs.dryRun),
+	)
 	if err != nil {
 		return err
 	}

cmd/flux/build_kustomization_test.go

@@ -142,6 +142,12 @@ spec:
 			resultFile: "./testdata/build-kustomization/podinfo-with-var-substitution-result.yaml",
 			assertFunc: "assertGoldenTemplateFile",
 		},
+		{
+			name:       "build deployment and configmap with var substitution in dry-run mode",
+			args:       "build kustomization podinfo --kustomization-file ./testdata/build-kustomization/podinfo.yaml --path ./testdata/build-kustomization/var-substitution --dry-run",
+			resultFile: "./testdata/build-kustomization/podinfo-with-var-substitution-result.yaml",
+			assertFunc: "assertGoldenTemplateFile",
+		},
 	}
 
 	tmpl := map[string]string{

cmd/flux/create.go

@@ -79,12 +79,12 @@ type upsertable interface {
 // want to update. The mutate function is nullary -- you mutate a
 // value in the closure, e.g., by doing this:
 //
-//     var existing Value
-//     existing.Name = name
-//     existing.Namespace = ns
-//     upsert(ctx, client, valueAdapter{&value}, func() error {
-//       value.Spec = onePreparedEarlier
-//     })
+//	var existing Value
+//	existing.Name = name
+//	existing.Namespace = ns
+//	upsert(ctx, client, valueAdapter{&value}, func() error {
+//	  value.Spec = onePreparedEarlier
+//	})
 func (names apiType) upsert(ctx context.Context, kubeClient client.Client, object upsertable, mutate func() error) (types.NamespacedName, error) {
 	nsname := types.NamespacedName{
 		Namespace: object.GetNamespace(),

cmd/flux/create_kustomization.go

@@ -169,7 +169,7 @@ func createKsCmdRun(cmd *cobra.Command, args []string) error {
 	}
 
 	if kustomizationArgs.kubeConfigSecretRef != "" {
-		kustomization.Spec.KubeConfig = &kustomizev1.KubeConfig{
+		kustomization.Spec.KubeConfig = &meta.KubeConfigReference{
 			SecretRef: meta.SecretKeyReference{
 				Name: kustomizationArgs.kubeConfigSecretRef,
 			},

cmd/flux/create_secret_git.go

@@ -21,6 +21,7 @@ import (
 	"crypto/elliptic"
 	"fmt"
 	"net/url"
+	"os"
 
 	"github.com/spf13/cobra"
 	corev1 "k8s.io/api/core/v1"
@@ -135,8 +136,12 @@ func createSecretGitCmdRun(cmd *cobra.Command, args []string) error {
 	}
 	switch u.Scheme {
 	case "ssh":
+		keypair, err := sourcesecret.LoadKeyPairFromPath(secretGitArgs.privateKeyFile, secretGitArgs.password)
+		if err != nil {
+			return err
+		}
+		opts.Keypair = keypair
 		opts.SSHHostname = u.Host
-		opts.PrivateKeyPath = secretGitArgs.privateKeyFile
 		opts.PrivateKeyAlgorithm = sourcesecret.PrivateKeyAlgorithm(secretGitArgs.keyAlgorithm)
 		opts.RSAKeyBits = int(secretGitArgs.rsaBits)
 		opts.ECDSACurve = secretGitArgs.ecdsaCurve.Curve
@@ -147,7 +152,13 @@ func createSecretGitCmdRun(cmd *cobra.Command, args []string) error {
 		}
 		opts.Username = secretGitArgs.username
 		opts.Password = secretGitArgs.password
-		opts.CAFilePath = secretGitArgs.caFile
+		if secretGitArgs.caFile != "" {
+			caBundle, err := os.ReadFile(secretGitArgs.caFile)
+			if err != nil {
+				return fmt.Errorf("unable to read TLS CA file: %w", err)
+			}
+			opts.CAFile = caBundle
+		}
 	default:
 		return fmt.Errorf("git URL scheme '%s' not supported, can be: ssh, http and https", u.Scheme)
 	}

cmd/flux/create_secret_helm.go

@@ -18,6 +18,8 @@ package main
 
 import (
 	"context"
+	"fmt"
+	"os"
 
 	"github.com/spf13/cobra"
 	corev1 "k8s.io/api/core/v1"
@@ -74,15 +76,34 @@ func createSecretHelmCmdRun(cmd *cobra.Command, args []string) error {
 		return err
 	}
 
+	caBundle := []byte{}
+	if secretHelmArgs.caFile != "" {
+		var err error
+		caBundle, err = os.ReadFile(secretHelmArgs.caFile)
+		if err != nil {
+			return fmt.Errorf("unable to read TLS CA file: %w", err)
+		}
+	}
+
+	var certFile, keyFile []byte
+	if secretHelmArgs.certFile != "" && secretHelmArgs.keyFile != "" {
+		if certFile, err = os.ReadFile(secretHelmArgs.certFile); err != nil {
+			return fmt.Errorf("failed to read cert file: %w", err)
+		}
+		if keyFile, err = os.ReadFile(secretHelmArgs.keyFile); err != nil {
+			return fmt.Errorf("failed to read key file: %w", err)
+		}
+	}
+
 	opts := sourcesecret.Options{
-		Name:         name,
-		Namespace:    *kubeconfigArgs.Namespace,
-		Labels:       labels,
-		Username:     secretHelmArgs.username,
-		Password:     secretHelmArgs.password,
-		CAFilePath:   secretHelmArgs.caFile,
-		CertFilePath: secretHelmArgs.certFile,
-		KeyFilePath:  secretHelmArgs.keyFile,
+		Name:      name,
+		Namespace: *kubeconfigArgs.Namespace,
+		Labels:    labels,
+		Username:  secretHelmArgs.username,
+		Password:  secretHelmArgs.password,
+		CAFile:    caBundle,
+		CertFile:  certFile,
+		KeyFile:   keyFile,
 	}
 	secret, err := sourcesecret.Generate(opts)
 	if err != nil {

cmd/flux/create_secret_tls.go

@@ -18,6 +18,8 @@ package main
 
 import (
 	"context"
+	"fmt"
+	"os"
 
 	"github.com/spf13/cobra"
 	"github.com/spf13/pflag"
@@ -73,13 +75,32 @@ func createSecretTLSCmdRun(cmd *cobra.Command, args []string) error {
 		return err
 	}
 
+	caBundle := []byte{}
+	if secretTLSArgs.caFile != "" {
+		var err error
+		caBundle, err = os.ReadFile(secretTLSArgs.caFile)
+		if err != nil {
+			return fmt.Errorf("unable to read TLS CA file: %w", err)
+		}
+	}
+
+	var certFile, keyFile []byte
+	if secretTLSArgs.certFile != "" && secretTLSArgs.keyFile != "" {
+		if certFile, err = os.ReadFile(secretTLSArgs.certFile); err != nil {
+			return fmt.Errorf("failed to read cert file: %w", err)
+		}
+		if keyFile, err = os.ReadFile(secretTLSArgs.keyFile); err != nil {
+			return fmt.Errorf("failed to read key file: %w", err)
+		}
+	}
+
 	opts := sourcesecret.Options{
-		Name:         name,
-		Namespace:    *kubeconfigArgs.Namespace,
-		Labels:       labels,
-		CAFilePath:   secretTLSArgs.caFile,
-		CertFilePath: secretTLSArgs.certFile,
-		KeyFilePath:  secretTLSArgs.keyFile,
+		Name:      name,
+		Namespace: *kubeconfigArgs.Namespace,
+		Labels:    labels,
+		CAFile:    caBundle,
+		CertFile:  certFile,
+		KeyFile:   keyFile,
 	}
 	secret, err := sourcesecret.Generate(opts)
 	if err != nil {

cmd/flux/create_source_git.go

@@ -259,16 +259,26 @@ func createSourceGitCmdRun(cmd *cobra.Command, args []string) error {
 		}
 		switch u.Scheme {
 		case "ssh":
+			keypair, err := sourcesecret.LoadKeyPairFromPath(sourceGitArgs.privateKeyFile, sourceGitArgs.password)
+			if err != nil {
+				return err
+			}
+			secretOpts.Keypair = keypair
 			secretOpts.SSHHostname = u.Host
-			secretOpts.PrivateKeyPath = sourceGitArgs.privateKeyFile
 			secretOpts.PrivateKeyAlgorithm = sourcesecret.PrivateKeyAlgorithm(sourceGitArgs.keyAlgorithm)
 			secretOpts.RSAKeyBits = int(sourceGitArgs.keyRSABits)
 			secretOpts.ECDSACurve = sourceGitArgs.keyECDSACurve.Curve
 			secretOpts.Password = sourceGitArgs.password
 		case "https":
+			if sourceGitArgs.caFile != "" {
+				caBundle, err := os.ReadFile(sourceGitArgs.caFile)
+				if err != nil {
+					return fmt.Errorf("unable to read TLS CA file: %w", err)
+				}
+				secretOpts.CAFile = caBundle
+			}
 			secretOpts.Username = sourceGitArgs.username
 			secretOpts.Password = sourceGitArgs.password
-			secretOpts.CAFilePath = sourceGitArgs.caFile
 		case "http":
 			logger.Warningf("insecure configuration: credentials configured for an HTTP URL")
 			secretOpts.Username = sourceGitArgs.username

cmd/flux/create_source_helm.go

@@ -168,6 +168,25 @@ func createSourceHelmCmdRun(cmd *cobra.Command, args []string) error {
 		return err
 	}
 
+	caBundle := []byte{}
+	if sourceHelmArgs.caFile != "" {
+		var err error
+		caBundle, err = os.ReadFile(sourceHelmArgs.caFile)
+		if err != nil {
+			return fmt.Errorf("unable to read TLS CA file: %w", err)
+		}
+	}
+
+	var certFile, keyFile []byte
+	if sourceHelmArgs.certFile != "" && sourceHelmArgs.keyFile != "" {
+		if certFile, err = os.ReadFile(sourceHelmArgs.certFile); err != nil {
+			return fmt.Errorf("failed to read cert file: %w", err)
+		}
+		if keyFile, err = os.ReadFile(sourceHelmArgs.keyFile); err != nil {
+			return fmt.Errorf("failed to read key file: %w", err)
+		}
+	}
+
 	logger.Generatef("generating HelmRepository source")
 	if sourceHelmArgs.secretRef == "" {
 		secretName := fmt.Sprintf("helm-%s", name)
@@ -176,9 +195,9 @@ func createSourceHelmCmdRun(cmd *cobra.Command, args []string) error {
 			Namespace:    *kubeconfigArgs.Namespace,
 			Username:     sourceHelmArgs.username,
 			Password:     sourceHelmArgs.password,
-			CertFilePath: sourceHelmArgs.certFile,
-			KeyFilePath:  sourceHelmArgs.keyFile,
-			CAFilePath:   sourceHelmArgs.caFile,
+			CAFile:       caBundle,
+			CertFile:     certFile,
+			KeyFile:      keyFile,
 			ManifestFile: sourcesecret.MakeDefaultOptions().ManifestFile,
 		}
 		secret, err := sourcesecret.Generate(secretOpts)

cmd/flux/create_source_oci.go

@@ -60,6 +60,7 @@ type sourceOCIRepositoryFlags struct {
 	certSecretRef  string
 	ignorePaths    []string
 	provider       flags.SourceOCIProvider
+	insecure       bool
 }
 
 var sourceOCIRepositoryArgs = newSourceOCIFlags()
@@ -80,6 +81,7 @@ func init() {
 	createSourceOCIRepositoryCmd.Flags().StringVar(&sourceOCIRepositoryArgs.serviceAccount, "service-account", "", "the name of the Kubernetes service account that refers to an image pull secret")
 	createSourceOCIRepositoryCmd.Flags().StringVar(&sourceOCIRepositoryArgs.certSecretRef, "cert-ref", "", "the name of a secret to use for TLS certificates")
 	createSourceOCIRepositoryCmd.Flags().StringSliceVar(&sourceOCIRepositoryArgs.ignorePaths, "ignore-paths", nil, "set paths to ignore resources (can specify multiple paths with commas: path1,path2)")
+	createSourceOCIRepositoryCmd.Flags().BoolVar(&sourceOCIRepositoryArgs.insecure, "insecure", false, "for when connecting to a non-TLS registries over plain HTTP")
 
 	createSourceCmd.AddCommand(createSourceOCIRepositoryCmd)
 }
@@ -115,6 +117,7 @@ func createSourceOCIRepositoryCmdRun(cmd *cobra.Command, args []string) error {
 		Spec: sourcev1.OCIRepositorySpec{
 			Provider: sourceOCIRepositoryArgs.provider.String(),
 			URL:      sourceOCIRepositoryArgs.url,
+			Insecure: sourceOCIRepositoryArgs.insecure,
 			Interval: metav1.Duration{
 				Duration: createArgs.interval,
 			},

cmd/flux/diff_artifact.go

@@ -0,0 +1,111 @@
+/*
+Copyright 2022 The Flux authors
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+
+package main
+
+import (
+	"context"
+	"fmt"
+	"os"
+
+	"github.com/fluxcd/flux2/internal/flags"
+	oci "github.com/fluxcd/pkg/oci/client"
+	sourcev1 "github.com/fluxcd/source-controller/api/v1beta2"
+	"github.com/spf13/cobra"
+)
+
+var diffArtifactCmd = &cobra.Command{
+	Use:   "artifact",
+	Short: "Diff Artifact",
+	Long:  `The diff artifact command computes the diff between the remote OCI artifact and a local directory or file`,
+	Example: `# Check if local files differ from remote
+flux diff artifact oci://ghcr.io/stefanprodan/manifests:podinfo:6.2.0 --path=./kustomize`,
+	RunE: diffArtifactCmdRun,
+}
+
+type diffArtifactFlags struct {
+	path        string
+	creds       string
+	provider    flags.SourceOCIProvider
+	ignorePaths []string
+}
+
+var diffArtifactArgs = newDiffArtifactArgs()
+
+func newDiffArtifactArgs() diffArtifactFlags {
+	return diffArtifactFlags{
+		provider: flags.SourceOCIProvider(sourcev1.GenericOCIProvider),
+	}
+}
+
+func init() {
+	diffArtifactCmd.Flags().StringVar(&diffArtifactArgs.path, "path", "", "path to the directory where the Kubernetes manifests are located")
+	diffArtifactCmd.Flags().StringVar(&diffArtifactArgs.creds, "creds", "", "credentials for OCI registry in the format <username>[:<password>] if --provider is generic")
+	diffArtifactCmd.Flags().Var(&diffArtifactArgs.provider, "provider", sourceOCIRepositoryArgs.provider.Description())
+	diffArtifactCmd.Flags().StringSliceVar(&diffArtifactArgs.ignorePaths, "ignore-paths", excludeOCI, "set paths to ignore in .gitignore format")
+	diffCmd.AddCommand(diffArtifactCmd)
+}
+
+func diffArtifactCmdRun(cmd *cobra.Command, args []string) error {
+	if len(args) < 1 {
+		return fmt.Errorf("artifact URL is required")
+	}
+	ociURL := args[0]
+
+	if diffArtifactArgs.path == "" {
+		return fmt.Errorf("invalid path %q", diffArtifactArgs.path)
+	}
+
+	url, err := oci.ParseArtifactURL(ociURL)
+	if err != nil {
+		return err
+	}
+
+	if _, err := os.Stat(diffArtifactArgs.path); err != nil {
+		return fmt.Errorf("invalid path '%s', must point to an existing directory or file", diffArtifactArgs.path)
+	}
+
+	ctx, cancel := context.WithTimeout(context.Background(), rootArgs.timeout)
+	defer cancel()
+
+	ociClient := oci.NewLocalClient()
+
+	if diffArtifactArgs.provider.String() == sourcev1.GenericOCIProvider && diffArtifactArgs.creds != "" {
+		logger.Actionf("logging in to registry with credentials")
+		if err := ociClient.LoginWithCredentials(diffArtifactArgs.creds); err != nil {
+			return fmt.Errorf("could not login with credentials: %w", err)
+		}
+	}
+
+	if diffArtifactArgs.provider.String() != sourcev1.GenericOCIProvider {
+		logger.Actionf("logging in to registry with provider credentials")
+		ociProvider, err := diffArtifactArgs.provider.ToOCIProvider()
+		if err != nil {
+			return fmt.Errorf("provider not supported: %w", err)
+		}
+
+		if err := ociClient.LoginWithProvider(ctx, url, ociProvider); err != nil {
+			return fmt.Errorf("error during login with provider: %w", err)
+		}
+	}
+
+	if err := ociClient.Diff(ctx, url, diffArtifactArgs.path, diffArtifactArgs.ignorePaths); err != nil {
+		return err
+	}
+
+	logger.Successf("no changes detected")
+	return nil
+}

cmd/flux/diff_artifact_test.go

@@ -0,0 +1,109 @@
+//go:build unit
+// +build unit
+
+/*
+Copyright 2021 The Flux authors
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+
+package main
+
+import (
+	"context"
+	"fmt"
+	"testing"
+	"time"
+
+	"github.com/distribution/distribution/v3/configuration"
+	"github.com/distribution/distribution/v3/registry"
+	_ "github.com/distribution/distribution/v3/registry/auth/htpasswd"
+	_ "github.com/distribution/distribution/v3/registry/storage/driver/inmemory"
+	"github.com/phayes/freeport"
+	ctrl "sigs.k8s.io/controller-runtime"
+)
+
+var dockerReg string
+
+func setupRegistryServer(ctx context.Context) error {
+	// Registry config
+	config := &configuration.Configuration{}
+	port, err := freeport.GetFreePort()
+	if err != nil {
+		return fmt.Errorf("failed to get free port: %s", err)
+	}
+
+	dockerReg = fmt.Sprintf("localhost:%d", port)
+	config.HTTP.Addr = fmt.Sprintf("127.0.0.1:%d", port)
+	config.HTTP.DrainTimeout = time.Duration(10) * time.Second
+	config.Storage = map[string]configuration.Parameters{"inmemory": map[string]interface{}{}}
+	dockerRegistry, err := registry.NewRegistry(ctx, config)
+	if err != nil {
+		return fmt.Errorf("failed to create docker registry: %w", err)
+	}
+
+	// Start Docker registry
+	go dockerRegistry.ListenAndServe()
+
+	return nil
+}
+
+func TestDiffArtifact(t *testing.T) {
+	tests := []struct {
+		name     string
+		url      string
+		argsTpl  string
+		pushFile string
+		diffFile string
+		assert   assertFunc
+	}{
+		{
+			name:     "should not fail if there is no diff",
+			url:      "oci://%s/podinfo:1.0.0",
+			argsTpl:  "diff artifact  %s --path=%s",
+			pushFile: "./testdata/diff-artifact/deployment.yaml",
+			diffFile: "./testdata/diff-artifact/deployment.yaml",
+			assert:   assertGoldenFile("testdata/diff-artifact/success.golden"),
+		},
+		{
+			name:     "should fail if there is a diff",
+			url:      "oci://%s/podinfo:2.0.0",
+			argsTpl:  "diff artifact %s --path=%s",
+			pushFile: "./testdata/diff-artifact/deployment.yaml",
+			diffFile: "./testdata/diff-artifact/deployment-diff.yaml",
+			assert:   assertError("the remote artifact contents differs from the local one"),
+		},
+	}
+
+	ctx := ctrl.SetupSignalHandler()
+	err := setupRegistryServer(ctx)
+	if err != nil {
+		panic(fmt.Sprintf("failed to start docker registry: %s", err))
+	}
+
+	for _, tt := range tests {
+		t.Run(tt.name, func(t *testing.T) {
+			tt.url = fmt.Sprintf(tt.url, dockerReg)
+			_, err := executeCommand("push artifact " + tt.url + " --path=" + tt.pushFile + " --source=test --revision=test")
+			if err != nil {
+				t.Fatalf(fmt.Errorf("failed to push image: %w", err).Error())
+			}
+
+			cmd := cmdTestCase{
+				args:   fmt.Sprintf(tt.argsTpl, tt.url, tt.diffFile),
+				assert: tt.assert,
+			}
+			cmd.runTestCmd(t)
+		})
+	}
+}

cmd/flux/diff_kustomization.go

@@ -77,12 +77,21 @@ func diffKsCmdRun(cmd *cobra.Command, args []string) error {
 		}
 	}
 
-	var builder *build.Builder
-	var err error
+	var (
+		builder *build.Builder
+		err     error
+	)
 	if diffKsArgs.progressBar {
-		builder, err = build.NewBuilder(kubeconfigArgs, kubeclientOptions, name, diffKsArgs.path, build.WithTimeout(rootArgs.timeout), build.WithKustomizationFile(diffKsArgs.kustomizationFile), build.WithProgressBar())
+		builder, err = build.NewBuilder(name, diffKsArgs.path,
+			build.WithClientConfig(kubeconfigArgs, kubeclientOptions),
+			build.WithTimeout(rootArgs.timeout),
+			build.WithKustomizationFile(diffKsArgs.kustomizationFile),
+			build.WithProgressBar())
 	} else {
-		builder, err = build.NewBuilder(kubeconfigArgs, kubeclientOptions, name, diffKsArgs.path, build.WithTimeout(rootArgs.timeout), build.WithKustomizationFile(diffKsArgs.kustomizationFile))
+		builder, err = build.NewBuilder(name, diffKsArgs.path,
+			build.WithClientConfig(kubeconfigArgs, kubeclientOptions),
+			build.WithTimeout(rootArgs.timeout),
+			build.WithKustomizationFile(diffKsArgs.kustomizationFile))
 	}
 
 	if err != nil {

cmd/flux/diff_kustomization_test.go

@@ -97,7 +97,7 @@ func TestDiffKustomization(t *testing.T) {
 		"fluxns": allocateNamespace("flux-system"),
 	}
 
-	b, _ := build.NewBuilder(kubeconfigArgs, kubeclientOptions, "podinfo", "")
+	b, _ := build.NewBuilder("podinfo", "", build.WithClientConfig(kubeconfigArgs, kubeclientOptions))
 
 	resourceManager, err := b.Manager()
 	if err != nil {

cmd/flux/get.go

@@ -162,7 +162,9 @@ func (get getCommand) run(cmd *cobra.Command, args []string) error {
 	}
 
 	if get.list.len() == 0 {
-		if !getAll {
+		if len(args) > 0 {
+			logger.Failuref("%s object '%s' not found in '%s' namespace", get.kind, args[0], *kubeconfigArgs.Namespace)
+		} else if !getAll {
 			logger.Failuref("no %s objects found in %s namespace", get.kind, *kubeconfigArgs.Namespace)
 		}
 		return nil
@@ -212,7 +214,6 @@ func getRowsToPrint(getAll bool, list summarisable) ([][]string, error) {
 	return rows, nil
 }
 
-//
 // watch starts a client-side watch of one or more resources.
 func (get *getCommand) watch(ctx context.Context, kubeClient client.WithWatch, cmd *cobra.Command, args []string, listOpts []client.ListOption) error {
 	w, err := kubeClient.Watch(ctx, get.list.asClientList(), listOpts...)

cmd/flux/list_artifact.go

@@ -20,6 +20,8 @@ import (
 	"context"
 	"fmt"
 
+	"github.com/fluxcd/flux2/internal/flags"
+	sourcev1 "github.com/fluxcd/source-controller/api/v1beta2"
 	"github.com/spf13/cobra"
 
 	oci "github.com/fluxcd/pkg/oci/client"
@@ -27,11 +29,26 @@ import (
 	"github.com/fluxcd/flux2/pkg/printers"
 )
 
+type listArtifactFlags struct {
+	semverFilter string
+	regexFilter  string
+	creds        string
+	provider     flags.SourceOCIProvider
+}
+
+var listArtifactArgs = newListArtifactFlags()
+
+func newListArtifactFlags() listArtifactFlags {
+	return listArtifactFlags{
+		provider: flags.SourceOCIProvider(sourcev1.GenericOCIProvider),
+	}
+}
+
 var listArtifactsCmd = &cobra.Command{
 	Use:   "artifacts",
 	Short: "list artifacts",
 	Long: `The list command fetches the tags and their metadata from a remote OCI repository.
-The command uses the credentials from '~/.docker/config.json'.`,
+The command can read the credentials from '~/.docker/config.json' but they can also be passed with --creds. It can also login to a supported provider with the --provider flag.`,
 	Example: `  # List the artifacts stored in an OCI repository
   flux list artifact oci://ghcr.io/org/config/app
 `,
@@ -39,6 +56,11 @@ The command uses the credentials from '~/.docker/config.json'.`,
 }
 
 func init() {
+	listArtifactsCmd.Flags().StringVar(&listArtifactArgs.semverFilter, "filter-semver", "", "filter tags returned from the oci repository using semver")
+	listArtifactsCmd.Flags().StringVar(&listArtifactArgs.regexFilter, "filter-regex", "", "filter tags returned from the oci repository using regex")
+	listArtifactsCmd.Flags().StringVar(&listArtifactArgs.creds, "creds", "", "credentials for OCI registry in the format <username>[:<password>] if --provider is generic")
+	listArtifactsCmd.Flags().Var(&listArtifactArgs.provider, "provider", listArtifactArgs.provider.Description())
+
 	listCmd.AddCommand(listArtifactsCmd)
 }
 
@@ -51,13 +73,38 @@ func listArtifactsCmdRun(cmd *cobra.Command, args []string) error {
 	ctx, cancel := context.WithTimeout(context.Background(), rootArgs.timeout)
 	defer cancel()
 
-	ociClient := oci.NewLocalClient()
 	url, err := oci.ParseArtifactURL(ociURL)
 	if err != nil {
 		return err
 	}
 
-	metas, err := ociClient.List(ctx, url)
+	ociClient := oci.NewLocalClient()
+
+	if listArtifactArgs.provider.String() == sourcev1.GenericOCIProvider && listArtifactArgs.creds != "" {
+		logger.Actionf("logging in to registry with credentials")
+		if err := ociClient.LoginWithCredentials(listArtifactArgs.creds); err != nil {
+			return fmt.Errorf("could not login with credentials: %w", err)
+		}
+	}
+
+	if listArtifactArgs.provider.String() != sourcev1.GenericOCIProvider {
+		logger.Actionf("logging in to registry with provider credentials")
+		ociProvider, err := listArtifactArgs.provider.ToOCIProvider()
+		if err != nil {
+			return fmt.Errorf("provider not supported: %w", err)
+		}
+
+		if err := ociClient.LoginWithProvider(ctx, url, ociProvider); err != nil {
+			return fmt.Errorf("error during login with provider: %w", err)
+		}
+	}
+
+	opts := oci.ListOptions{
+		RegexFilter:  listArtifactArgs.regexFilter,
+		SemverFilter: listArtifactArgs.semverFilter,
+	}
+
+	metas, err := ociClient.List(ctx, url, opts)
 	if err != nil {
 		return err
 	}

cmd/flux/logs.go

@@ -252,7 +252,7 @@ func logRequest(ctx context.Context, request rest.ResponseWrapper, w io.Writer)
 
 	scanner := bufio.NewScanner(stream)
 
-	const logTmpl = "{{.Timestamp}} {{.Level}} {{.Kind}}{{if .Name}}/{{.Name}}.{{.Namespace}}{{end}} - {{.Message}} {{.Error}}\n"
+	const logTmpl = "{{.Timestamp}} {{.Level}} {{or .Kind .ControllerKind}}{{if .Name}}/{{.Name}}.{{.Namespace}}{{end}} - {{.Message}} {{.Error}}\n"
 	t, err := template.New("log").Parse(logTmpl)
 	if err != nil {
 		return fmt.Errorf("unable to create template, err: %s", err)
@@ -278,7 +278,7 @@ func logRequest(ctx context.Context, request rest.ResponseWrapper, w io.Writer)
 
 func filterPrintLog(t *template.Template, l *ControllerLogEntry, w io.Writer) {
 	if (logsArgs.logLevel == "" || logsArgs.logLevel == l.Level) &&
-		(logsArgs.kind == "" || strings.EqualFold(logsArgs.kind, l.Kind)) &&
+		(logsArgs.kind == "" || strings.EqualFold(logsArgs.kind, l.Kind) || strings.EqualFold(logsArgs.kind, l.ControllerKind)) &&
 		(logsArgs.name == "" || strings.EqualFold(logsArgs.name, l.Name)) &&
 		(logsArgs.allNamespaces || strings.EqualFold(*kubeconfigArgs.Namespace, l.Namespace)) {
 		err := t.Execute(w, l)
@@ -289,12 +289,13 @@ func filterPrintLog(t *template.Template, l *ControllerLogEntry, w io.Writer) {
 }
 
 type ControllerLogEntry struct {
-	Timestamp string         `json:"ts"`
-	Level     flags.LogLevel `json:"level"`
-	Message   string         `json:"msg"`
-	Error     string         `json:"error,omitempty"`
-	Logger    string         `json:"logger"`
-	Kind      string         `json:"reconciler kind,omitempty"`
-	Name      string         `json:"name,omitempty"`
-	Namespace string         `json:"namespace,omitempty"`
+	Timestamp      string         `json:"ts"`
+	Level          flags.LogLevel `json:"level"`
+	Message        string         `json:"msg"`
+	Error          string         `json:"error,omitempty"`
+	Logger         string         `json:"logger"`
+	Kind           string         `json:"reconciler kind,omitempty"`
+	ControllerKind string         `json:"controllerKind,omitempty"`
+	Name           string         `json:"name,omitempty"`
+	Namespace      string         `json:"namespace,omitempty"`
 }

cmd/flux/logs_test.go

@@ -170,12 +170,13 @@ func TestLogRequest(t *testing.T) {
 	}
 }
 
-var testPodLogs = `{"level":"info","ts":"2022-08-02T12:55:34.419Z","logger":"controller.gitrepository","msg":"no changes since last reconcilation: observed revision","reconciler group":"source.toolkit.fluxcd.io","reconciler kind":"GitRepository","name":"podinfo","namespace":"default"}
-{"level":"error","ts":"2022-08-02T12:56:04.679Z","logger":"controller.gitrepository","msg":"no changes since last reconcilation: observed revision","reconciler group":"source.toolkit.fluxcd.io","reconciler kind":"GitRepository","name":"flux-system","namespace":"flux-system"}
+var testPodLogs = `{"level":"info","ts":"2022-08-02T12:55:34.419Z","msg":"no changes since last reconcilation: observed revision","controller":"gitrepository","controllerGroup":"source.toolkit.fluxcd.io","controllerKind":"GitRepository","gitRepository":{"name":"podinfo","namespace":"default"},"namespace":"default","name":"podinfo","reconcileID":"5ef9b2ef-4ea5-47b7-b887-a247cafc1bce"}
+{"level":"error","ts":"2022-08-02T12:56:04.679Z","logger":"controller.gitrepository","msg":"no changes since last reconcilation: observed revision","controllerGroup":"source.toolkit.fluxcd.io","controllerKind":"GitRepository","gitRepository":{"name":"podinfo","namespace":"flux-system"},"name":"flux-system","namespace":"flux-system","reconcileID":"543ef9b2ef-4ea5-47b7-b887-a247cafc1bce"}
 {"level":"error","ts":"2022-08-02T12:56:34.961Z","logger":"controller.kustomization","msg":"no changes since last reconcilation: observed revision","reconciler group":"kustomize.toolkit.fluxcd.io","reconciler kind":"Kustomization","name":"flux-system","namespace":"flux-system"}
 {"level":"info","ts":"2022-08-02T12:56:34.961Z","logger":"controller.kustomization","msg":"no changes since last reconcilation: observed revision","reconciler group":"kustomize.toolkit.fluxcd.io","reconciler kind":"Kustomization","name":"podinfo","namespace":"default"}
 {"level":"info","ts":"2022-08-02T12:56:34.961Z","logger":"controller.gitrepository","msg":"no changes since last reconcilation: observed revision","reconciler group":"source.toolkit.fluxcd.io","reconciler kind":"GitRepository","name":"podinfo","namespace":"default"}
-{"level":"error","ts":"2022-08-02T12:56:34.961Z","logger":"controller.kustomization","msg":"no changes since last reconcilation: observed revision","reconciler group":"kustomize.toolkit.fluxcd.io","reconciler kind":"Kustomization","name":"podinfo","namespace":"flux-system"}`
+{"level":"error","ts":"2022-08-02T12:56:34.961Z","logger":"controller.kustomization","msg":"no changes since last reconcilation: observed revision","reconciler group":"kustomize.toolkit.fluxcd.io","reconciler kind":"Kustomization","name":"podinfo","namespace":"flux-system"}
+`
 
 type testResponseMapper struct {
 }

cmd/flux/main_test.go

@@ -288,36 +288,38 @@ func assertGoldenFile(goldenFile string) assertFunc {
 // is pre-processed with the specified templateValues.
 func assertGoldenTemplateFile(goldenFile string, templateValues map[string]string) assertFunc {
 	goldenFileContents, fileErr := os.ReadFile(goldenFile)
-	return func(output string, err error) error {
-		if fileErr != nil {
-			return fmt.Errorf("Error reading golden file '%s': %s", goldenFile, fileErr)
-		}
-		var expectedOutput string
-		if len(templateValues) > 0 {
-			expectedOutput, err = executeTemplate(string(goldenFileContents), templateValues)
-			if err != nil {
-				return fmt.Errorf("Error executing golden template file '%s': %s", goldenFile, err)
+	return assert(
+		assertSuccess(),
+		func(output string, err error) error {
+			if fileErr != nil {
+				return fmt.Errorf("Error reading golden file '%s': %s", goldenFile, fileErr)
 			}
-		} else {
-			expectedOutput = string(goldenFileContents)
-		}
-		if assertErr := assertGoldenValue(expectedOutput)(output, err); assertErr != nil {
-			// Update the golden files if comparison fails and the update flag is set.
-			if *update && output != "" {
-				// Skip update if there are template values.
-				if len(templateValues) > 0 {
-					fmt.Println("NOTE: -update flag passed but golden template files can't be updated, please update it manually")
-				} else {
-					if err := os.WriteFile(goldenFile, []byte(output), 0644); err != nil {
-						return fmt.Errorf("failed to update golden file '%s': %v", goldenFile, err)
-					}
-					return nil
+			var expectedOutput string
+			if len(templateValues) > 0 {
+				expectedOutput, err = executeTemplate(string(goldenFileContents), templateValues)
+				if err != nil {
+					return fmt.Errorf("Error executing golden template file '%s': %s", goldenFile, err)
 				}
+			} else {
+				expectedOutput = string(goldenFileContents)
 			}
-			return fmt.Errorf("Mismatch from golden file '%s': %v", goldenFile, assertErr)
-		}
-		return nil
-	}
+			if assertErr := assertGoldenValue(expectedOutput)(output, err); assertErr != nil {
+				// Update the golden files if comparison fails and the update flag is set.
+				if *update && output != "" {
+					// Skip update if there are template values.
+					if len(templateValues) > 0 {
+						fmt.Println("NOTE: -update flag passed but golden template files can't be updated, please update it manually")
+					} else {
+						if err := os.WriteFile(goldenFile, []byte(output), 0644); err != nil {
+							return fmt.Errorf("failed to update golden file '%s': %v", goldenFile, err)
+						}
+						return nil
+					}
+				}
+				return fmt.Errorf("Mismatch from golden file '%s': %v", goldenFile, assertErr)
+			}
+			return nil
+		})
 }
 
 type TestClusterMode int
@@ -341,7 +343,6 @@ type cmdTestCase struct {
 
 func (cmd *cmdTestCase) runTestCmd(t *testing.T) {
 	actual, testErr := executeCommand(cmd.args)
-
 	// If the cmd error is a change, discard it
 	if isChangeError(testErr) {
 		testErr = nil
@@ -383,15 +384,51 @@ func executeCommand(cmd string) (string, error) {
 	return result, err
 }
 
+// resetCmdArgs resets the flags for various cmd
+// Note: this will also clear default value of the flags set in init()
 func resetCmdArgs() {
+	*kubeconfigArgs.Namespace = rootArgs.defaults.Namespace
+	alertArgs = alertFlags{}
+	alertProviderArgs = alertProviderFlags{}
+	bootstrapArgs = NewBootstrapFlags()
+	bServerArgs = bServerFlags{}
+	buildKsArgs = buildKsFlags{}
+	checkArgs = checkFlags{}
 	createArgs = createFlags{}
+	deleteArgs = deleteFlags{}
+	diffKsArgs = diffKsFlags{}
+	exportArgs = exportFlags{}
 	getArgs = GetFlags{}
+	gitArgs = gitFlags{}
+	githubArgs = githubFlags{}
+	gitlabArgs = gitlabFlags{}
+	helmReleaseArgs = helmReleaseFlags{
+		reconcileStrategy: "ChartVersion",
+	}
+	imagePolicyArgs = imagePolicyFlags{}
+	imageRepoArgs = imageRepoFlags{}
+	imageUpdateArgs = imageUpdateFlags{}
+	kustomizationArgs = NewKustomizationFlags()
+	receiverArgs = receiverFlags{}
+	resumeArgs = ResumeFlags{}
+	rhrArgs = reconcileHelmReleaseFlags{}
+	rksArgs = reconcileKsFlags{}
+	secretGitArgs = NewSecretGitFlags()
+	secretHelmArgs = secretHelmFlags{}
+	secretTLSArgs = secretTLSFlags{}
+	sourceBucketArgs = sourceBucketFlags{}
+	sourceGitArgs = newSourceGitFlags()
 	sourceHelmArgs = sourceHelmFlags{}
 	sourceOCIRepositoryArgs = sourceOCIRepositoryFlags{}
-	sourceGitArgs = sourceGitFlags{}
-	sourceBucketArgs = sourceBucketFlags{}
-	secretGitArgs = NewSecretGitFlags()
-	*kubeconfigArgs.Namespace = rootArgs.defaults.Namespace
+	suspendArgs = SuspendFlags{}
+	tenantArgs = tenantFlags{}
+	traceArgs = traceFlags{}
+	treeKsArgs = TreeKsFlags{}
+	uninstallArgs = uninstallFlags{}
+	versionArgs = versionFlags{
+		output: "yaml",
+	}
+
 }
 
 func isChangeError(err error) bool {

cmd/flux/pull_artifact.go

@@ -21,6 +21,8 @@ import (
 	"fmt"
 	"os"
 
+	"github.com/fluxcd/flux2/internal/flags"
+	sourcev1 "github.com/fluxcd/source-controller/api/v1beta2"
 	"github.com/spf13/cobra"
 
 	oci "github.com/fluxcd/pkg/oci/client"
@@ -30,7 +32,7 @@ var pullArtifactCmd = &cobra.Command{
 	Use:   "artifact",
 	Short: "Pull artifact",
 	Long: `The pull artifact command downloads and extracts the OCI artifact content to the given path.
-The pull command uses the credentials from '~/.docker/config.json'.`,
+The command can read the credentials from '~/.docker/config.json' but they can also be passed with --creds. It can also login to a supported provider with the --provider flag.`,
 	Example: `  # Pull an OCI artifact created by flux from GHCR
   flux pull artifact oci://ghcr.io/org/manifests/app:v0.0.1 --output ./path/to/local/manifests
 `,
@@ -38,13 +40,23 @@ The pull command uses the credentials from '~/.docker/config.json'.`,
 }
 
 type pullArtifactFlags struct {
-	output string
+	output   string
+	creds    string
+	provider flags.SourceOCIProvider
 }
 
-var pullArtifactArgs pullArtifactFlags
+var pullArtifactArgs = newPullArtifactFlags()
+
+func newPullArtifactFlags() pullArtifactFlags {
+	return pullArtifactFlags{
+		provider: flags.SourceOCIProvider(sourcev1.GenericOCIProvider),
+	}
+}
 
 func init() {
 	pullArtifactCmd.Flags().StringVarP(&pullArtifactArgs.output, "output", "o", "", "path where the artifact content should be extracted.")
+	pullArtifactCmd.Flags().StringVar(&pullArtifactArgs.creds, "creds", "", "credentials for OCI registry in the format <username>[:<password>] if --provider is generic")
+	pullArtifactCmd.Flags().Var(&pullArtifactArgs.provider, "provider", sourceOCIRepositoryArgs.provider.Description())
 	pullCmd.AddCommand(pullArtifactCmd)
 }
 
@@ -62,7 +74,6 @@ func pullArtifactCmdRun(cmd *cobra.Command, args []string) error {
 		return fmt.Errorf("invalid output path %s", pullArtifactArgs.output)
 	}
 
-	ociClient := oci.NewLocalClient()
 	url, err := oci.ParseArtifactURL(ociURL)
 	if err != nil {
 		return err
@@ -71,6 +82,27 @@ func pullArtifactCmdRun(cmd *cobra.Command, args []string) error {
 	ctx, cancel := context.WithTimeout(context.Background(), rootArgs.timeout)
 	defer cancel()
 
+	ociClient := oci.NewLocalClient()
+
+	if pullArtifactArgs.provider.String() == sourcev1.GenericOCIProvider && pullArtifactArgs.creds != "" {
+		logger.Actionf("logging in to registry with credentials")
+		if err := ociClient.LoginWithCredentials(pullArtifactArgs.creds); err != nil {
+			return fmt.Errorf("could not login with credentials: %w", err)
+		}
+	}
+
+	if pullArtifactArgs.provider.String() != sourcev1.GenericOCIProvider {
+		logger.Actionf("logging in to registry with provider credentials")
+		ociProvider, err := pullArtifactArgs.provider.ToOCIProvider()
+		if err != nil {
+			return fmt.Errorf("provider not supported: %w", err)
+		}
+
+		if err := ociClient.LoginWithProvider(ctx, url, ociProvider); err != nil {
+			return fmt.Errorf("error during login with provider: %w", err)
+		}
+	}
+
 	logger.Actionf("pulling artifact from %s", url)
 
 	meta, err := ociClient.Pull(ctx, url, pullArtifactArgs.output)

cmd/flux/push_artifact.go

@@ -21,6 +21,8 @@ import (
 	"fmt"
 	"os"
 
+	"github.com/fluxcd/flux2/internal/flags"
+	sourcev1 "github.com/fluxcd/source-controller/api/v1beta2"
 	"github.com/spf13/cobra"
 
 	oci "github.com/fluxcd/pkg/oci/client"
@@ -29,8 +31,8 @@ import (
 var pushArtifactCmd = &cobra.Command{
 	Use:   "artifact",
 	Short: "Push artifact",
-	Long: `The push artifact command creates a tarball from the given directory and uploads the artifact to an OCI repository.
-The command uses the credentials from '~/.docker/config.json'.`,
+	Long: `The push artifact command creates a tarball from the given directory or the single file and uploads the artifact to an OCI repository.
+The command can read the credentials from '~/.docker/config.json' but they can also be passed with --creds. It can also login to a supported provider with the --provider flag.`,
 	Example: `  # Push manifests to GHCR using the short Git SHA as the OCI artifact tag
   echo $GITHUB_PAT | docker login ghcr.io --username flux --password-stdin
   flux push artifact oci://ghcr.io/org/config/app:$(git rev-parse --short HEAD) \
@@ -38,28 +40,64 @@ The command uses the credentials from '~/.docker/config.json'.`,
 	--source="$(git config --get remote.origin.url)" \
 	--revision="$(git branch --show-current)/$(git rev-parse HEAD)"
 
+  # Push single manifest file to GHCR using the short Git SHA as the OCI artifact tag
+  echo $GITHUB_PAT | docker login ghcr.io --username flux --password-stdin
+  flux push artifact oci://ghcr.io/org/config/app:$(git rev-parse --short HEAD) \
+	--path="./path/to/local/manifest.yaml" \
+	--source="$(git config --get remote.origin.url)" \
+	--revision="$(git branch --show-current)/$(git rev-parse HEAD)"
+
   # Push manifests to Docker Hub using the Git tag as the OCI artifact tag
   echo $DOCKER_PAT | docker login --username flux --password-stdin
   flux push artifact oci://docker.io/org/app-config:$(git tag --points-at HEAD) \
 	--path="./path/to/local/manifests" \
 	--source="$(git config --get remote.origin.url)" \
 	--revision="$(git tag --points-at HEAD)/$(git rev-parse HEAD)"
+
+  # Login directly to the registry provider
+  # You might need to export the following variable if you use local config files for AWS:
+  # export AWS_SDK_LOAD_CONFIG=1
+  flux push artifact oci://<account>.dkr.ecr.<region>.amazonaws.com/foo:v1:$(git tag --points-at HEAD) \
+	--path="./path/to/local/manifests" \
+	--source="$(git config --get remote.origin.url)" \
+	--revision="$(git tag --points-at HEAD)/$(git rev-parse HEAD)" \
+	--provider aws
+
+  # Or pass credentials directly
+  flux push artifact oci://docker.io/org/app-config:$(git tag --points-at HEAD) \
+	--path="./path/to/local/manifests" \
+	--source="$(git config --get remote.origin.url)" \
+	--revision="$(git tag --points-at HEAD)/$(git rev-parse HEAD)" \
+	--creds flux:$DOCKER_PAT
 `,
 	RunE: pushArtifactCmdRun,
 }
 
 type pushArtifactFlags struct {
-	path     string
-	source   string
-	revision string
+	path        string
+	source      string
+	revision    string
+	creds       string
+	provider    flags.SourceOCIProvider
+	ignorePaths []string
 }
 
-var pushArtifactArgs pushArtifactFlags
+var pushArtifactArgs = newPushArtifactFlags()
+
+func newPushArtifactFlags() pushArtifactFlags {
+	return pushArtifactFlags{
+		provider: flags.SourceOCIProvider(sourcev1.GenericOCIProvider),
+	}
+}
 
 func init() {
 	pushArtifactCmd.Flags().StringVar(&pushArtifactArgs.path, "path", "", "path to the directory where the Kubernetes manifests are located")
 	pushArtifactCmd.Flags().StringVar(&pushArtifactArgs.source, "source", "", "the source address, e.g. the Git URL")
 	pushArtifactCmd.Flags().StringVar(&pushArtifactArgs.revision, "revision", "", "the source revision in the format '<branch|tag>/<commit-sha>'")
+	pushArtifactCmd.Flags().StringVar(&pushArtifactArgs.creds, "creds", "", "credentials for OCI registry in the format <username>[:<password>] if --provider is generic")
+	pushArtifactCmd.Flags().Var(&pushArtifactArgs.provider, "provider", pushArtifactArgs.provider.Description())
+	pushArtifactCmd.Flags().StringSliceVar(&pushArtifactArgs.ignorePaths, "ignore-paths", excludeOCI, "set paths to ignore in .gitignore format")
+
 	pushCmd.AddCommand(pushArtifactCmd)
 }
 
@@ -81,14 +119,13 @@ func pushArtifactCmdRun(cmd *cobra.Command, args []string) error {
 		return fmt.Errorf("invalid path %q", pushArtifactArgs.path)
 	}
 
-	ociClient := oci.NewLocalClient()
 	url, err := oci.ParseArtifactURL(ociURL)
 	if err != nil {
 		return err
 	}
 
-	if fs, err := os.Stat(pushArtifactArgs.path); err != nil || !fs.IsDir() {
-		return fmt.Errorf("invalid path %q", pushArtifactArgs.path)
+	if _, err := os.Stat(pushArtifactArgs.path); err != nil {
+		return fmt.Errorf("invalid path '%s', must point to an existing directory or file", buildArtifactArgs.path)
 	}
 
 	meta := oci.Metadata{
@@ -99,9 +136,30 @@ func pushArtifactCmdRun(cmd *cobra.Command, args []string) error {
 	ctx, cancel := context.WithTimeout(context.Background(), rootArgs.timeout)
 	defer cancel()
 
+	ociClient := oci.NewLocalClient()
+
+	if pushArtifactArgs.provider.String() == sourcev1.GenericOCIProvider && pushArtifactArgs.creds != "" {
+		logger.Actionf("logging in to registry with credentials")
+		if err := ociClient.LoginWithCredentials(pushArtifactArgs.creds); err != nil {
+			return fmt.Errorf("could not login with credentials: %w", err)
+		}
+	}
+
+	if pushArtifactArgs.provider.String() != sourcev1.GenericOCIProvider {
+		logger.Actionf("logging in to registry with provider credentials")
+		ociProvider, err := pushArtifactArgs.provider.ToOCIProvider()
+		if err != nil {
+			return fmt.Errorf("provider not supported: %w", err)
+		}
+
+		if err := ociClient.LoginWithProvider(ctx, url, ociProvider); err != nil {
+			return fmt.Errorf("error during login with provider: %w", err)
+		}
+	}
+
 	logger.Actionf("pushing artifact to %s", url)
 
-	digest, err := ociClient.Push(ctx, url, pushArtifactArgs.path, meta)
+	digest, err := ociClient.Push(ctx, url, pushArtifactArgs.path, meta, pushArtifactArgs.ignorePaths)
 	if err != nil {
 		return fmt.Errorf("pushing artifact failed: %w", err)
 	}
@@ -109,5 +167,4 @@ func pushArtifactCmdRun(cmd *cobra.Command, args []string) error {
 	logger.Successf("artifact successfully pushed to %s", digest)
 
 	return nil
-
 }

cmd/flux/tag_artifact.go

@@ -20,6 +20,8 @@ import (
 	"context"
 	"fmt"
 
+	"github.com/fluxcd/flux2/internal/flags"
+	sourcev1 "github.com/fluxcd/source-controller/api/v1beta2"
 	"github.com/spf13/cobra"
 
 	oci "github.com/fluxcd/pkg/oci/client"
@@ -29,7 +31,7 @@ var tagArtifactCmd = &cobra.Command{
 	Use:   "artifact",
 	Short: "Tag artifact",
 	Long: `The tag artifact command creates tags for the given OCI artifact.
-The command uses the credentials from '~/.docker/config.json'.`,
+The command can read the credentials from '~/.docker/config.json' but they can also be passed with --creds. It can also login to a supported provider with the --provider flag.`,
 	Example: `  # Tag an artifact version as latest
   flux tag artifact oci://ghcr.io/org/manifests/app:v0.0.1 --tag latest
 `,
@@ -37,13 +39,23 @@ The command uses the credentials from '~/.docker/config.json'.`,
 }
 
 type tagArtifactFlags struct {
-	tags []string
+	tags     []string
+	creds    string
+	provider flags.SourceOCIProvider
 }
 
-var tagArtifactArgs tagArtifactFlags
+var tagArtifactArgs = newTagArtifactFlags()
+
+func newTagArtifactFlags() tagArtifactFlags {
+	return tagArtifactFlags{
+		provider: flags.SourceOCIProvider(sourcev1.GenericOCIProvider),
+	}
+}
 
 func init() {
 	tagArtifactCmd.Flags().StringSliceVar(&tagArtifactArgs.tags, "tag", nil, "tag name")
+	tagArtifactCmd.Flags().StringVar(&tagArtifactArgs.creds, "creds", "", "credentials for OCI registry in the format <username>[:<password>] if --provider is generic")
+	tagArtifactCmd.Flags().Var(&tagArtifactArgs.provider, "provider", tagArtifactArgs.provider.Description())
 	tagCmd.AddCommand(tagArtifactCmd)
 }
 
@@ -57,7 +69,6 @@ func tagArtifactCmdRun(cmd *cobra.Command, args []string) error {
 		return fmt.Errorf("--tag is required")
 	}
 
-	ociClient := oci.NewLocalClient()
 	url, err := oci.ParseArtifactURL(ociURL)
 	if err != nil {
 		return err
@@ -66,6 +77,27 @@ func tagArtifactCmdRun(cmd *cobra.Command, args []string) error {
 	ctx, cancel := context.WithTimeout(context.Background(), rootArgs.timeout)
 	defer cancel()
 
+	ociClient := oci.NewLocalClient()
+
+	if tagArtifactArgs.provider.String() == sourcev1.GenericOCIProvider && tagArtifactArgs.creds != "" {
+		logger.Actionf("logging in to registry with credentials")
+		if err := ociClient.LoginWithCredentials(tagArtifactArgs.creds); err != nil {
+			return fmt.Errorf("could not login with credentials: %w", err)
+		}
+	}
+
+	if tagArtifactArgs.provider.String() != sourcev1.GenericOCIProvider {
+		logger.Actionf("logging in to registry with provider credentials")
+		ociProvider, err := tagArtifactArgs.provider.ToOCIProvider()
+		if err != nil {
+			return fmt.Errorf("provider not supported: %w", err)
+		}
+
+		if err := ociClient.LoginWithProvider(ctx, url, ociProvider); err != nil {
+			return fmt.Errorf("error during login with provider: %w", err)
+		}
+	}
+
 	logger.Actionf("tagging artifact")
 
 	for _, tag := range tagArtifactArgs.tags {

cmd/flux/testdata/diff-artifact/deployment-diff.yaml

@@ -0,0 +1,78 @@
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  labels:
+      kustomize.toolkit.fluxcd.io/name: podinfo
+      kustomize.toolkit.fluxcd.io/namespace: {{ .fluxns }}
+  name: podinfo-diff
+  namespace: default
+spec:
+  minReadySeconds: 3
+  revisionHistoryLimit: 5
+  progressDeadlineSeconds: 60
+  strategy:
+    rollingUpdate:
+      maxUnavailable: 0
+    type: RollingUpdate
+  selector:
+    matchLabels:
+      app: podinfo
+  template:
+    metadata:
+      annotations:
+        prometheus.io/scrape: "true"
+        prometheus.io/port: "9797"
+      labels:
+        app: podinfo
+    spec:
+      containers:
+      - name: podinfod
+        image: ghcr.io/stefanprodan/podinfo:6.0.10
+        imagePullPolicy: IfNotPresent
+        ports:
+        - name: http
+          containerPort: 9898
+          protocol: TCP
+        - name: http-metrics
+          containerPort: 9797
+          protocol: TCP
+        - name: grpc
+          containerPort: 9999
+          protocol: TCP
+        command:
+        - ./podinfo
+        - --port=9898
+        - --port-metrics=9797
+        - --grpc-port=9999
+        - --grpc-service-name=podinfo
+        - --level=info
+        - --random-delay=false
+        - --random-error=false
+        env:
+        - name: PODINFO_UI_COLOR
+          value: "#34577c"
+        livenessProbe:
+          exec:
+            command:
+            - podcli
+            - check
+            - http
+            - localhost:9898/healthz
+          initialDelaySeconds: 5
+          timeoutSeconds: 5
+        readinessProbe:
+          exec:
+            command:
+            - podcli
+            - check
+            - http
+            - localhost:9898/readyz
+          initialDelaySeconds: 5
+          timeoutSeconds: 5
+        resources:
+          limits:
+            cpu: 2000m
+            memory: 512Mi
+          requests:
+            cpu: 100m
+            memory: 64Mi

cmd/flux/testdata/diff-artifact/deployment.yaml

@@ -0,0 +1,78 @@
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  labels:
+      kustomize.toolkit.fluxcd.io/name: podinfo
+      kustomize.toolkit.fluxcd.io/namespace: {{ .fluxns }}
+  name: podinfo
+  namespace: default
+spec:
+  minReadySeconds: 3
+  revisionHistoryLimit: 5
+  progressDeadlineSeconds: 60
+  strategy:
+    rollingUpdate:
+      maxUnavailable: 0
+    type: RollingUpdate
+  selector:
+    matchLabels:
+      app: podinfo
+  template:
+    metadata:
+      annotations:
+        prometheus.io/scrape: "true"
+        prometheus.io/port: "9797"
+      labels:
+        app: podinfo
+    spec:
+      containers:
+      - name: podinfod
+        image: ghcr.io/stefanprodan/podinfo:6.0.10
+        imagePullPolicy: IfNotPresent
+        ports:
+        - name: http
+          containerPort: 9898
+          protocol: TCP
+        - name: http-metrics
+          containerPort: 9797
+          protocol: TCP
+        - name: grpc
+          containerPort: 9999
+          protocol: TCP
+        command:
+        - ./podinfo
+        - --port=9898
+        - --port-metrics=9797
+        - --grpc-port=9999
+        - --grpc-service-name=podinfo
+        - --level=info
+        - --random-delay=false
+        - --random-error=false
+        env:
+        - name: PODINFO_UI_COLOR
+          value: "#34577c"
+        livenessProbe:
+          exec:
+            command:
+            - podcli
+            - check
+            - http
+            - localhost:9898/healthz
+          initialDelaySeconds: 5
+          timeoutSeconds: 5
+        readinessProbe:
+          exec:
+            command:
+            - podcli
+            - check
+            - http
+            - localhost:9898/readyz
+          initialDelaySeconds: 5
+          timeoutSeconds: 5
+        resources:
+          limits:
+            cpu: 2000m
+            memory: 512Mi
+          requests:
+            cpu: 100m
+            memory: 64Mi

cmd/flux/testdata/diff-artifact/success.golden

@@ -0,0 +1 @@
+✔ no changes detected

cmd/flux/testdata/export/helm-repo.yaml

@@ -6,6 +6,7 @@ metadata:
   namespace: {{ .fluxns }}
 spec:
   interval: 5m0s
+  provider: generic
   timeout: 1m0s
   url: https://stefanprodan.github.io/podinfo
 

cmd/flux/testdata/oci/create_source_oci.golden

@@ -2,4 +2,4 @@
 ✔ OCIRepository created
 ◎ waiting for OCIRepository reconciliation
 ✔ OCIRepository reconciliation completed
-✔ fetched revision: dbdb109711ffb3be77504d2670dbe13c24dd63d8d7f1fb489d350e5bfe930dd3
+✔ fetched revision: 6.1.6/dbdb109711ffb3be77504d2670dbe13c24dd63d8d7f1fb489d350e5bfe930dd3

cmd/flux/testdata/oci/get_oci.golden

@@ -1,2 +1,2 @@
-NAME 	REVISION                                                        	SUSPENDED	READY	MESSAGE                                                                                       
-thrfg	dbdb109711ffb3be77504d2670dbe13c24dd63d8d7f1fb489d350e5bfe930dd3	False    	True 	stored artifact for digest 'dbdb109711ffb3be77504d2670dbe13c24dd63d8d7f1fb489d350e5bfe930dd3'	
+NAME 	REVISION                                                              	SUSPENDED	READY	MESSAGE                                                                                             
+thrfg	6.1.6/dbdb109711ffb3be77504d2670dbe13c24dd63d8d7f1fb489d350e5bfe930dd3	False    	True 	stored artifact for digest '6.1.6/dbdb109711ffb3be77504d2670dbe13c24dd63d8d7f1fb489d350e5bfe930dd3'	

cmd/flux/testdata/oci/reconcile_oci.golden

@@ -1,4 +1,4 @@
 ► annotating OCIRepository thrfg in {{ .ns }} namespace
 ✔ OCIRepository annotated
 ◎ waiting for OCIRepository reconciliation
-✔ fetched revision dbdb109711ffb3be77504d2670dbe13c24dd63d8d7f1fb489d350e5bfe930dd3
+✔ fetched revision 6.1.6/dbdb109711ffb3be77504d2670dbe13c24dd63d8d7f1fb489d350e5bfe930dd3

cmd/flux/testdata/oci/resume_oci.golden

@@ -2,4 +2,4 @@
 ✔ source oci resumed
 ◎ waiting for OCIRepository reconciliation
 ✔ OCIRepository reconciliation completed
-✔ fetched revision dbdb109711ffb3be77504d2670dbe13c24dd63d8d7f1fb489d350e5bfe930dd3
+✔ fetched revision 6.1.6/dbdb109711ffb3be77504d2670dbe13c24dd63d8d7f1fb489d350e5bfe930dd3

cmd/flux/uninstall.go

@@ -22,21 +22,9 @@ import (
 
 	"github.com/manifoldco/promptui"
 	"github.com/spf13/cobra"
-	appsv1 "k8s.io/api/apps/v1"
-	corev1 "k8s.io/api/core/v1"
-	networkingv1 "k8s.io/api/networking/v1"
-	rbacv1 "k8s.io/api/rbac/v1"
-	apiextensionsv1 "k8s.io/apiextensions-apiserver/pkg/apis/apiextensions/v1"
-	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
-	"sigs.k8s.io/controller-runtime/pkg/client"
 
 	"github.com/fluxcd/flux2/internal/utils"
-	"github.com/fluxcd/flux2/pkg/manifestgen"
-	helmv2 "github.com/fluxcd/helm-controller/api/v2beta1"
-	autov1 "github.com/fluxcd/image-automation-controller/api/v1beta1"
-	imagev1 "github.com/fluxcd/image-reflector-controller/api/v1beta1"
-	kustomizev1 "github.com/fluxcd/kustomize-controller/api/v1beta2"
-	sourcev1 "github.com/fluxcd/source-controller/api/v1beta2"
+	"github.com/fluxcd/flux2/pkg/uninstall"
 )
 
 var uninstallCmd = &cobra.Command{
@@ -90,265 +78,18 @@ func uninstallCmdRun(cmd *cobra.Command, args []string) error {
 	}
 
 	logger.Actionf("deleting components in %s namespace", *kubeconfigArgs.Namespace)
-	uninstallComponents(ctx, kubeClient, *kubeconfigArgs.Namespace, uninstallArgs.dryRun)
+	uninstall.Components(ctx, logger, kubeClient, *kubeconfigArgs.Namespace, uninstallArgs.dryRun)
 
 	logger.Actionf("deleting toolkit.fluxcd.io finalizers in all namespaces")
-	uninstallFinalizers(ctx, kubeClient, uninstallArgs.dryRun)
+	uninstall.Finalizers(ctx, logger, kubeClient, uninstallArgs.dryRun)
 
 	logger.Actionf("deleting toolkit.fluxcd.io custom resource definitions")
-	uninstallCustomResourceDefinitions(ctx, kubeClient, uninstallArgs.dryRun)
+	uninstall.CustomResourceDefinitions(ctx, logger, kubeClient, uninstallArgs.dryRun)
 
 	if !uninstallArgs.keepNamespace {
-		uninstallNamespace(ctx, kubeClient, *kubeconfigArgs.Namespace, uninstallArgs.dryRun)
+		uninstall.Namespace(ctx, logger, kubeClient, *kubeconfigArgs.Namespace, uninstallArgs.dryRun)
 	}
 
 	logger.Successf("uninstall finished")
 	return nil
 }
-
-func uninstallComponents(ctx context.Context, kubeClient client.Client, namespace string, dryRun bool) {
-	opts, dryRunStr := getDeleteOptions(dryRun)
-	selector := client.MatchingLabels{manifestgen.PartOfLabelKey: manifestgen.PartOfLabelValue}
-	{
-		var list appsv1.DeploymentList
-		if err := kubeClient.List(ctx, &list, client.InNamespace(namespace), selector); err == nil {
-			for _, r := range list.Items {
-				if err := kubeClient.Delete(ctx, &r, opts); err != nil {
-					logger.Failuref("Deployment/%s/%s deletion failed: %s", r.Namespace, r.Name, err.Error())
-				} else {
-					logger.Successf("Deployment/%s/%s deleted %s", r.Namespace, r.Name, dryRunStr)
-				}
-			}
-		}
-	}
-	{
-		var list corev1.ServiceList
-		if err := kubeClient.List(ctx, &list, client.InNamespace(namespace), selector); err == nil {
-			for _, r := range list.Items {
-				if err := kubeClient.Delete(ctx, &r, opts); err != nil {
-					logger.Failuref("Service/%s/%s deletion failed: %s", r.Namespace, r.Name, err.Error())
-				} else {
-					logger.Successf("Service/%s/%s deleted %s", r.Namespace, r.Name, dryRunStr)
-				}
-			}
-		}
-	}
-	{
-		var list networkingv1.NetworkPolicyList
-		if err := kubeClient.List(ctx, &list, client.InNamespace(namespace), selector); err == nil {
-			for _, r := range list.Items {
-				if err := kubeClient.Delete(ctx, &r, opts); err != nil {
-					logger.Failuref("NetworkPolicy/%s/%s deletion failed: %s", r.Namespace, r.Name, err.Error())
-				} else {
-					logger.Successf("NetworkPolicy/%s/%s deleted %s", r.Namespace, r.Name, dryRunStr)
-				}
-			}
-		}
-	}
-	{
-		var list corev1.ServiceAccountList
-		if err := kubeClient.List(ctx, &list, client.InNamespace(namespace), selector); err == nil {
-			for _, r := range list.Items {
-				if err := kubeClient.Delete(ctx, &r, opts); err != nil {
-					logger.Failuref("ServiceAccount/%s/%s deletion failed: %s", r.Namespace, r.Name, err.Error())
-				} else {
-					logger.Successf("ServiceAccount/%s/%s deleted %s", r.Namespace, r.Name, dryRunStr)
-				}
-			}
-		}
-	}
-	{
-		var list rbacv1.ClusterRoleList
-		if err := kubeClient.List(ctx, &list, selector); err == nil {
-			for _, r := range list.Items {
-				if err := kubeClient.Delete(ctx, &r, opts); err != nil {
-					logger.Failuref("ClusterRole/%s deletion failed: %s", r.Name, err.Error())
-				} else {
-					logger.Successf("ClusterRole/%s deleted %s", r.Name, dryRunStr)
-				}
-			}
-		}
-	}
-	{
-		var list rbacv1.ClusterRoleBindingList
-		if err := kubeClient.List(ctx, &list, selector); err == nil {
-			for _, r := range list.Items {
-				if err := kubeClient.Delete(ctx, &r, opts); err != nil {
-					logger.Failuref("ClusterRoleBinding/%s deletion failed: %s", r.Name, err.Error())
-				} else {
-					logger.Successf("ClusterRoleBinding/%s deleted %s", r.Name, dryRunStr)
-				}
-			}
-		}
-	}
-}
-
-func uninstallFinalizers(ctx context.Context, kubeClient client.Client, dryRun bool) {
-	opts, dryRunStr := getUpdateOptions(dryRun)
-	{
-		var list sourcev1.GitRepositoryList
-		if err := kubeClient.List(ctx, &list, client.InNamespace("")); err == nil {
-			for _, r := range list.Items {
-				r.Finalizers = []string{}
-				if err := kubeClient.Update(ctx, &r, opts); err != nil {
-					logger.Failuref("%s/%s/%s removing finalizers failed: %s", r.Kind, r.Namespace, r.Name, err.Error())
-				} else {
-					logger.Successf("%s/%s/%s finalizers deleted %s", r.Kind, r.Namespace, r.Name, dryRunStr)
-				}
-			}
-		}
-	}
-	{
-		var list sourcev1.HelmRepositoryList
-		if err := kubeClient.List(ctx, &list, client.InNamespace("")); err == nil {
-			for _, r := range list.Items {
-				r.Finalizers = []string{}
-				if err := kubeClient.Update(ctx, &r, opts); err != nil {
-					logger.Failuref("%s/%s/%s removing finalizers failed: %s", r.Kind, r.Namespace, r.Name, err.Error())
-				} else {
-					logger.Successf("%s/%s/%s finalizers deleted %s", r.Kind, r.Namespace, r.Name, dryRunStr)
-				}
-			}
-		}
-	}
-	{
-		var list sourcev1.HelmChartList
-		if err := kubeClient.List(ctx, &list, client.InNamespace("")); err == nil {
-			for _, r := range list.Items {
-				r.Finalizers = []string{}
-				if err := kubeClient.Update(ctx, &r, opts); err != nil {
-					logger.Failuref("%s/%s/%s removing finalizers failed: %s", r.Kind, r.Namespace, r.Name, err.Error())
-				} else {
-					logger.Successf("%s/%s/%s finalizers deleted %s", r.Kind, r.Namespace, r.Name, dryRunStr)
-				}
-			}
-		}
-	}
-	{
-		var list sourcev1.BucketList
-		if err := kubeClient.List(ctx, &list, client.InNamespace("")); err == nil {
-			for _, r := range list.Items {
-				r.Finalizers = []string{}
-				if err := kubeClient.Update(ctx, &r, opts); err != nil {
-					logger.Failuref("%s/%s/%s removing finalizers failed: %s", r.Kind, r.Namespace, r.Name, err.Error())
-				} else {
-					logger.Successf("%s/%s/%s finalizers deleted %s", r.Kind, r.Namespace, r.Name, dryRunStr)
-				}
-			}
-		}
-	}
-	{
-		var list kustomizev1.KustomizationList
-		if err := kubeClient.List(ctx, &list, client.InNamespace("")); err == nil {
-			for _, r := range list.Items {
-				r.Finalizers = []string{}
-				if err := kubeClient.Update(ctx, &r, opts); err != nil {
-					logger.Failuref("%s/%s/%s removing finalizers failed: %s", r.Kind, r.Namespace, r.Name, err.Error())
-				} else {
-					logger.Successf("%s/%s/%s finalizers deleted %s", r.Kind, r.Namespace, r.Name, dryRunStr)
-				}
-			}
-		}
-	}
-	{
-		var list helmv2.HelmReleaseList
-		if err := kubeClient.List(ctx, &list, client.InNamespace("")); err == nil {
-			for _, r := range list.Items {
-				r.Finalizers = []string{}
-				if err := kubeClient.Update(ctx, &r, opts); err != nil {
-					logger.Failuref("%s/%s/%s removing finalizers failed: %s", r.Kind, r.Namespace, r.Name, err.Error())
-				} else {
-					logger.Successf("%s/%s/%s finalizers deleted %s", r.Kind, r.Namespace, r.Name, dryRunStr)
-				}
-			}
-		}
-	}
-	{
-		var list imagev1.ImagePolicyList
-		if err := kubeClient.List(ctx, &list, client.InNamespace("")); err == nil {
-			for _, r := range list.Items {
-				r.Finalizers = []string{}
-				if err := kubeClient.Update(ctx, &r, opts); err != nil {
-					logger.Failuref("%s/%s/%s removing finalizers failed: %s", r.Kind, r.Namespace, r.Name, err.Error())
-				} else {
-					logger.Successf("%s/%s/%s finalizers deleted %s", r.Kind, r.Namespace, r.Name, dryRunStr)
-				}
-			}
-		}
-	}
-	{
-		var list imagev1.ImageRepositoryList
-		if err := kubeClient.List(ctx, &list, client.InNamespace("")); err == nil {
-			for _, r := range list.Items {
-				r.Finalizers = []string{}
-				if err := kubeClient.Update(ctx, &r, opts); err != nil {
-					logger.Failuref("%s/%s/%s removing finalizers failed: %s", r.Kind, r.Namespace, r.Name, err.Error())
-				} else {
-					logger.Successf("%s/%s/%s finalizers deleted %s", r.Kind, r.Namespace, r.Name, dryRunStr)
-				}
-			}
-		}
-	}
-	{
-		var list autov1.ImageUpdateAutomationList
-		if err := kubeClient.List(ctx, &list, client.InNamespace("")); err == nil {
-			for _, r := range list.Items {
-				r.Finalizers = []string{}
-				if err := kubeClient.Update(ctx, &r, opts); err != nil {
-					logger.Failuref("%s/%s/%s removing finalizers failed: %s", r.Kind, r.Namespace, r.Name, err.Error())
-				} else {
-					logger.Successf("%s/%s/%s finalizers deleted %s", r.Kind, r.Namespace, r.Name, dryRunStr)
-				}
-			}
-		}
-	}
-}
-
-func uninstallCustomResourceDefinitions(ctx context.Context, kubeClient client.Client, dryRun bool) {
-	opts, dryRunStr := getDeleteOptions(dryRun)
-	selector := client.MatchingLabels{manifestgen.PartOfLabelKey: manifestgen.PartOfLabelValue}
-	{
-		var list apiextensionsv1.CustomResourceDefinitionList
-		if err := kubeClient.List(ctx, &list, selector); err == nil {
-			for _, r := range list.Items {
-				if err := kubeClient.Delete(ctx, &r, opts); err != nil {
-					logger.Failuref("CustomResourceDefinition/%s deletion failed: %s", r.Name, err.Error())
-				} else {
-					logger.Successf("CustomResourceDefinition/%s deleted %s", r.Name, dryRunStr)
-				}
-			}
-		}
-	}
-}
-
-func uninstallNamespace(ctx context.Context, kubeClient client.Client, namespace string, dryRun bool) {
-	opts, dryRunStr := getDeleteOptions(dryRun)
-	ns := corev1.Namespace{ObjectMeta: metav1.ObjectMeta{Name: namespace}}
-	if err := kubeClient.Delete(ctx, &ns, opts); err != nil {
-		logger.Failuref("Namespace/%s deletion failed: %s", namespace, err.Error())
-	} else {
-		logger.Successf("Namespace/%s deleted %s", namespace, dryRunStr)
-	}
-}
-
-func getDeleteOptions(dryRun bool) (*client.DeleteOptions, string) {
-	opts := &client.DeleteOptions{}
-	var dryRunStr string
-	if dryRun {
-		client.DryRunAll.ApplyToDelete(opts)
-		dryRunStr = "(dry run)"
-	}
-
-	return opts, dryRunStr
-}
-
-func getUpdateOptions(dryRun bool) (*client.UpdateOptions, string) {
-	opts := &client.UpdateOptions{}
-	var dryRunStr string
-	if dryRun {
-		client.DryRunAll.ApplyToUpdate(opts)
-		dryRunStr = "(dry run)"
-	}
-
-	return opts, dryRunStr
-}

cmd/flux/version_utils.go

@@ -18,6 +18,7 @@ package main
 
 import (
 	"fmt"
+	"strings"
 
 	"github.com/fluxcd/flux2/internal/utils"
 	"github.com/fluxcd/flux2/pkg/manifestgen/install"
@@ -28,6 +29,10 @@ func getVersion(input string) (string, error) {
 		return rootArgs.defaults.Version, nil
 	}
 
+	if input != install.MakeDefaultOptions().Version && !strings.HasPrefix(input, "v") {
+		return "", fmt.Errorf("targeted version '%s' must be prefixed with 'v'", input)
+	}
+
 	if isEmbeddedVersion(input) {
 		return input, nil
 	}

docs/_redirects

@@ -1,18 +1,18 @@
 # individual rules
-/core-concepts          https://fluxcd.io/docs/concepts                     301!
+/core-concepts          https://fluxcd.io/flux/concepts                     301!
 /contributing           https://fluxcd.io/contributing                      301!
 /install.sh             https://fluxcd.io/install.sh                        301!
 
 # refer to https://github.com/fluxcd/flux2/discussions/367
-/dev-guides/*           https://fluxcd.io/docs/gitops-toolkit/:splat        301!
+/dev-guides/*           https://fluxcd.io/flux/gitops-toolkit/:splat        301!
 
 
 # this is how things looked in the navbar anyway..?
-/guides/faq-migration                   https://fluxcd.io/docs/migration/faq-migration                  301!
-/guides/flux-v1-automation-migration    https://fluxcd.io/docs/migration/flux-v1-automation-migration   301!
-/guides/flux-v1-migration               https://fluxcd.io/docs/migration/flux-v1-migration              301!
-/guides/helm-operator-migration         https://fluxcd.io/docs/migration/helm-operator-migration        301!
+/guides/faq-migration                   https://fluxcd.io/flux/migration/faq-migration                  301!
+/guides/flux-v1-automation-migration    https://fluxcd.io/flux/migration/flux-v1-automation-migration   301!
+/guides/flux-v1-migration               https://fluxcd.io/flux/migration/flux-v1-migration              301!
+/guides/helm-operator-migration         https://fluxcd.io/flux/migration/helm-operator-migration        301!
 
 
 # catch all
-/*                      https://fluxcd.io/docs/:splat                       301!
+/*                      https://fluxcd.io/flux/:splat                       301!

go.mod

@@ -1,179 +1,218 @@
 module github.com/fluxcd/flux2
 
-go 1.17
+go 1.18
 
 require (
 	github.com/Masterminds/semver/v3 v3.1.1
-	github.com/ProtonMail/go-crypto v0.0.0-20220517143526-88bb52951d5b
+	github.com/ProtonMail/go-crypto v0.0.0-20221026131551-cf6655e29de4
 	github.com/cyphar/filepath-securejoin v0.2.3
-	github.com/fluxcd/go-git-providers v0.6.0
-	github.com/fluxcd/helm-controller/api v0.22.2
-	github.com/fluxcd/image-automation-controller/api v0.24.1
-	github.com/fluxcd/image-reflector-controller/api v0.20.0
-	github.com/fluxcd/kustomize-controller/api v0.27.0
-	github.com/fluxcd/notification-controller/api v0.25.1
-	github.com/fluxcd/pkg/apis/meta v0.14.2
-	github.com/fluxcd/pkg/kustomize v0.5.2
-	github.com/fluxcd/pkg/oci v0.3.0
-	github.com/fluxcd/pkg/runtime v0.16.2
-	github.com/fluxcd/pkg/ssa v0.17.0
-	github.com/fluxcd/pkg/ssh v0.5.0
-	github.com/fluxcd/pkg/untar v0.1.0
-	github.com/fluxcd/pkg/version v0.1.0
-	github.com/fluxcd/source-controller/api v0.26.1
-	github.com/go-git/go-git/v5 v5.4.2
+	github.com/distribution/distribution/v3 v3.0.0-20221119093643-85d4039064cc
+	github.com/fluxcd/go-git-providers v0.11.0
+	github.com/fluxcd/go-git/v5 v5.0.0-20221104190732-329fd6659b10
+	github.com/fluxcd/helm-controller/api v0.27.0
+	github.com/fluxcd/image-automation-controller/api v0.27.0
+	github.com/fluxcd/image-reflector-controller/api v0.23.0
+	github.com/fluxcd/kustomize-controller/api v0.31.0
+	github.com/fluxcd/notification-controller/api v0.29.0
+	github.com/fluxcd/pkg/apis/meta v0.18.0
+	github.com/fluxcd/pkg/git v0.7.0
+	github.com/fluxcd/pkg/git/gogit v0.2.0
+	github.com/fluxcd/pkg/kustomize v0.10.0
+	github.com/fluxcd/pkg/oci v0.15.0
+	github.com/fluxcd/pkg/runtime v0.24.0
+	github.com/fluxcd/pkg/sourceignore v0.3.0
+	github.com/fluxcd/pkg/ssa v0.22.0
+	github.com/fluxcd/pkg/ssh v0.7.0
+	github.com/fluxcd/pkg/untar v0.2.0
+	github.com/fluxcd/pkg/version v0.2.0
+	github.com/fluxcd/source-controller/api v0.32.1
 	github.com/gonvenience/bunt v1.3.4
 	github.com/gonvenience/ytbx v1.4.4
-	github.com/google/go-cmp v0.5.8
-	github.com/google/go-containerregistry v0.10.0
-	github.com/hashicorp/go-multierror v1.1.1
-	github.com/homeport/dyff v1.5.4
+	github.com/google/go-cmp v0.5.9
+	github.com/google/go-containerregistry v0.12.1
+	github.com/homeport/dyff v1.5.6
 	github.com/lucasb-eyer/go-colorful v1.2.0
 	github.com/manifoldco/promptui v0.9.0
 	github.com/mattn/go-shellwords v1.0.12
 	github.com/olekukonko/tablewriter v0.0.5
-	github.com/onsi/gomega v1.19.0
-	github.com/spf13/cobra v1.4.0
+	github.com/onsi/gomega v1.24.1
+	github.com/phayes/freeport v0.0.0-20220201140144-74d24b5ae9f5
+	github.com/spf13/cobra v1.6.1
 	github.com/spf13/pflag v1.0.5
 	github.com/theckman/yacspin v0.13.12
-	golang.org/x/crypto v0.0.0-20220525230936-793ad666bf5e
-	golang.org/x/term v0.0.0-20220526004731-065cf7ba2467
-	k8s.io/api v0.24.3
-	k8s.io/apiextensions-apiserver v0.24.3
-	k8s.io/apimachinery v0.24.3
-	k8s.io/cli-runtime v0.24.1
-	k8s.io/client-go v0.24.3
-	k8s.io/kubectl v0.24.1
-	sigs.k8s.io/cli-utils v0.32.0
-	sigs.k8s.io/controller-runtime v0.11.2
+	golang.org/x/crypto v0.3.0
+	golang.org/x/term v0.2.0
+	k8s.io/api v0.25.4
+	k8s.io/apiextensions-apiserver v0.25.4
+	k8s.io/apimachinery v0.25.4
+	k8s.io/cli-runtime v0.25.4
+	k8s.io/client-go v0.25.4
+	k8s.io/kubectl v0.25.4
+	sigs.k8s.io/cli-utils v0.34.0
+	sigs.k8s.io/controller-runtime v0.13.1
 	sigs.k8s.io/kustomize/api v0.12.1
 	sigs.k8s.io/kustomize/kyaml v0.13.9
 	sigs.k8s.io/yaml v1.3.0
 )
 
+// Fix CVE-2022-32149
+replace golang.org/x/text => golang.org/x/text v0.4.0
+
 // Fix CVE-2022-28948
 replace gopkg.in/yaml.v3 => gopkg.in/yaml.v3 v3.0.1
 
 require (
 	cloud.google.com/go v0.99.0 // indirect
+	github.com/Azure/azure-sdk-for-go/sdk/azcore v1.2.0 // indirect
+	github.com/Azure/azure-sdk-for-go/sdk/azidentity v1.2.0 // indirect
+	github.com/Azure/azure-sdk-for-go/sdk/internal v1.0.0 // indirect
 	github.com/Azure/go-ansiterm v0.0.0-20210617225240-d185dfc1b5a1 // indirect
 	github.com/Azure/go-autorest v14.2.0+incompatible // indirect
-	github.com/Azure/go-autorest/autorest v0.11.24 // indirect
-	github.com/Azure/go-autorest/autorest/adal v0.9.18 // indirect
+	github.com/Azure/go-autorest/autorest v0.11.27 // indirect
+	github.com/Azure/go-autorest/autorest/adal v0.9.20 // indirect
 	github.com/Azure/go-autorest/autorest/date v0.3.0 // indirect
 	github.com/Azure/go-autorest/logger v0.2.1 // indirect
 	github.com/Azure/go-autorest/tracing v0.6.0 // indirect
+	github.com/AzureAD/microsoft-authentication-library-for-go v0.7.0 // indirect
 	github.com/BurntSushi/toml v1.0.0 // indirect
-	github.com/MakeNowJust/heredoc v0.0.0-20170808103936-bb23615498cd // indirect
-	github.com/Microsoft/go-winio v0.5.2 // indirect
+	github.com/MakeNowJust/heredoc v1.0.0 // indirect
+	github.com/Microsoft/go-winio v0.6.0 // indirect
+	github.com/Shopify/logrus-bugsnag v0.0.0-20171204204709-577dee27f20d // indirect
 	github.com/acomagu/bufpipe v1.0.3 // indirect
+	github.com/aws/aws-sdk-go v1.44.137 // indirect
 	github.com/beorn7/perks v1.0.1 // indirect
+	github.com/bshuster-repo/logrus-logstash-hook v1.0.0 // indirect
+	github.com/bugsnag/bugsnag-go v0.0.0-20141110184014-b1d153021fcd // indirect
+	github.com/bugsnag/osext v0.0.0-20130617224835-0dd3f918b21b // indirect
+	github.com/bugsnag/panicwrap v0.0.0-20151223152923-e2c28503fcd0 // indirect
 	github.com/cespare/xxhash/v2 v2.1.2 // indirect
-	github.com/chai2010/gettext-go v0.0.0-20160711120539-c6fed771bfd5 // indirect
+	github.com/chai2010/gettext-go v1.0.2 // indirect
 	github.com/chzyer/readline v0.0.0-20180603132655-2972be24d48e // indirect
-	github.com/containerd/stargz-snapshotter/estargz v0.11.4 // indirect
+	github.com/cloudflare/circl v1.3.0 // indirect
+	github.com/containerd/stargz-snapshotter/estargz v0.12.1 // indirect
 	github.com/cpuguy83/go-md2man/v2 v2.0.2 // indirect
 	github.com/davecgh/go-spew v1.1.1 // indirect
-	github.com/docker/cli v20.10.17+incompatible // indirect
+	github.com/docker/cli v20.10.20+incompatible // indirect
 	github.com/docker/distribution v2.8.1+incompatible // indirect
-	github.com/docker/docker v20.10.17+incompatible // indirect
-	github.com/docker/docker-credential-helpers v0.6.4 // indirect
-	github.com/drone/envsubst/v2 v2.0.0-20210730161058-179042472c46 // indirect
-	github.com/emicklei/go-restful v2.15.0+incompatible // indirect
-	github.com/emirpasic/gods v1.12.0 // indirect
-	github.com/evanphx/json-patch v4.12.0+incompatible // indirect
-	github.com/exponent-io/jsonpath v0.0.0-20151013193312-d6023ce2651d // indirect
+	github.com/docker/docker v20.10.20+incompatible // indirect
+	github.com/docker/docker-credential-helpers v0.7.0 // indirect
+	github.com/docker/go-events v0.0.0-20190806004212-e31b211e4f1c // indirect
+	github.com/docker/go-metrics v0.0.1 // indirect
+	github.com/docker/libtrust v0.0.0-20150114040149-fa567046d9b1 // indirect
+	github.com/drone/envsubst v1.0.3 // indirect
+	github.com/emicklei/go-restful/v3 v3.10.0 // indirect
+	github.com/emirpasic/gods v1.18.1 // indirect
+	github.com/evanphx/json-patch v5.6.0+incompatible // indirect
+	github.com/evanphx/json-patch/v5 v5.6.0 // indirect
+	github.com/exponent-io/jsonpath v0.0.0-20210407135951-1de76d718b3f // indirect
 	github.com/fatih/color v1.13.0 // indirect
-	github.com/fluxcd/pkg/apis/acl v0.0.3 // indirect
-	github.com/fluxcd/pkg/apis/kustomize v0.4.2 // indirect
-	github.com/fsnotify/fsnotify v1.5.1 // indirect
-	github.com/go-errors/errors v1.0.1 // indirect
+	github.com/felixge/httpsnoop v1.0.1 // indirect
+	github.com/fluxcd/pkg/apis/acl v0.1.0 // indirect
+	github.com/fluxcd/pkg/apis/kustomize v0.7.0 // indirect
+	github.com/fluxcd/pkg/tar v0.2.0 // indirect
+	github.com/fsnotify/fsnotify v1.6.0 // indirect
+	github.com/go-errors/errors v1.4.2 // indirect
 	github.com/go-git/gcfg v1.5.0 // indirect
 	github.com/go-git/go-billy/v5 v5.3.1 // indirect
+	github.com/go-git/go-git/v5 v5.4.2 // indirect
 	github.com/go-logr/logr v1.2.3 // indirect
 	github.com/go-openapi/jsonpointer v0.19.5 // indirect
 	github.com/go-openapi/jsonreference v0.20.0 // indirect
-	github.com/go-openapi/swag v0.21.1 // indirect
+	github.com/go-openapi/swag v0.22.3 // indirect
 	github.com/gogo/protobuf v1.3.2 // indirect
-	github.com/golang-jwt/jwt/v4 v4.4.1 // indirect
+	github.com/golang-jwt/jwt/v4 v4.4.2 // indirect
 	github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da // indirect
 	github.com/golang/protobuf v1.5.2 // indirect
-	github.com/gonvenience/neat v1.3.10 // indirect
+	github.com/gomodule/redigo v1.8.2 // indirect
+	github.com/gonvenience/neat v1.3.11 // indirect
 	github.com/gonvenience/term v1.0.2 // indirect
 	github.com/gonvenience/text v1.0.7 // indirect
-	github.com/gonvenience/wrap v1.1.1 // indirect
-	github.com/google/btree v1.0.1 // indirect
+	github.com/gonvenience/wrap v1.1.2 // indirect
+	github.com/google/btree v1.1.2 // indirect
 	github.com/google/gnostic v0.6.9 // indirect
-	github.com/google/go-github/v42 v42.0.0 // indirect
+	github.com/google/go-github/v47 v47.1.0 // indirect
 	github.com/google/go-querystring v1.1.0 // indirect
 	github.com/google/gofuzz v1.2.0 // indirect
 	github.com/google/shlex v0.0.0-20191202100458-e7afc7fbc510 // indirect
 	github.com/google/uuid v1.3.0 // indirect
+	github.com/gorilla/handlers v1.5.1 // indirect
+	github.com/gorilla/mux v1.8.0 // indirect
 	github.com/gregjones/httpcache v0.0.0-20190611155906-901d90724c79 // indirect
-	github.com/hashicorp/errwrap v1.0.0 // indirect
+	github.com/hashicorp/errwrap v1.1.0 // indirect
 	github.com/hashicorp/go-cleanhttp v0.5.2 // indirect
+	github.com/hashicorp/go-multierror v1.1.1 // indirect
 	github.com/hashicorp/go-retryablehttp v0.7.1 // indirect
-	github.com/imdario/mergo v0.3.12 // indirect
-	github.com/inconshreveable/mousetrap v1.0.0 // indirect
+	github.com/hashicorp/golang-lru v0.5.4 // indirect
+	github.com/imdario/mergo v0.3.13 // indirect
+	github.com/inconshreveable/mousetrap v1.0.1 // indirect
 	github.com/jbenet/go-context v0.0.0-20150711004518-d14ea06fba99 // indirect
+	github.com/jmespath/go-jmespath v0.4.0 // indirect
 	github.com/josharian/intern v1.0.0 // indirect
 	github.com/json-iterator/go v1.1.12 // indirect
-	github.com/kevinburke/ssh_config v0.0.0-20201106050909-4977a11b4351 // indirect
-	github.com/klauspost/compress v1.15.4 // indirect
+	github.com/kevinburke/ssh_config v1.2.0 // indirect
+	github.com/klauspost/compress v1.15.11 // indirect
+	github.com/kylelemons/godebug v1.1.0 // indirect
 	github.com/liggitt/tabwriter v0.0.0-20181228230101-89fcab3d43de // indirect
 	github.com/mailru/easyjson v0.7.7 // indirect
 	github.com/mattn/go-ciede2000 v0.0.0-20170301095244-782e8c62fec3 // indirect
 	github.com/mattn/go-colorable v0.1.12 // indirect
 	github.com/mattn/go-isatty v0.0.14 // indirect
 	github.com/mattn/go-runewidth v0.0.13 // indirect
-	github.com/matttproud/golang_protobuf_extensions v1.0.2-0.20181231171920-c182affec369 // indirect
+	github.com/matttproud/golang_protobuf_extensions v1.0.4 // indirect
 	github.com/mitchellh/go-homedir v1.1.0 // indirect
 	github.com/mitchellh/go-ps v1.0.0 // indirect
-	github.com/mitchellh/go-wordwrap v1.0.0 // indirect
+	github.com/mitchellh/go-wordwrap v1.0.1 // indirect
 	github.com/mitchellh/hashstructure v1.1.0 // indirect
 	github.com/moby/spdystream v0.2.0 // indirect
-	github.com/moby/term v0.0.0-20210619224110-3f7ff695adc6 // indirect
+	github.com/moby/term v0.0.0-20221105221325-4eb28fa6025c // indirect
 	github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect
 	github.com/modern-go/reflect2 v1.0.2 // indirect
 	github.com/monochromegane/go-gitignore v0.0.0-20200626010858-205db1a8cc00 // indirect
 	github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 // indirect
 	github.com/opencontainers/go-digest v1.0.0 // indirect
-	github.com/opencontainers/image-spec v1.0.3-0.20220114050600-8b9d41f48198 // indirect
+	github.com/opencontainers/image-spec v1.1.0-rc2 // indirect
 	github.com/peterbourgon/diskv v2.0.1+incompatible // indirect
+	github.com/pkg/browser v0.0.0-20210911075715-681adbf594b8 // indirect
 	github.com/pkg/errors v0.9.1 // indirect
-	github.com/prometheus/client_golang v1.12.1 // indirect
-	github.com/prometheus/client_model v0.2.0 // indirect
-	github.com/prometheus/common v0.32.1 // indirect
-	github.com/prometheus/procfs v0.7.3 // indirect
+	github.com/prometheus/client_golang v1.14.0 // indirect
+	github.com/prometheus/client_model v0.3.0 // indirect
+	github.com/prometheus/common v0.37.0 // indirect
+	github.com/prometheus/procfs v0.8.0 // indirect
 	github.com/rivo/uniseg v0.2.0 // indirect
 	github.com/russross/blackfriday v1.6.0 // indirect
 	github.com/russross/blackfriday/v2 v2.1.0 // indirect
 	github.com/sergi/go-diff v1.2.0 // indirect
 	github.com/sirupsen/logrus v1.9.0 // indirect
+	github.com/skeema/knownhosts v1.1.0 // indirect
 	github.com/texttheater/golang-levenshtein v1.0.1 // indirect
 	github.com/vbatts/tar-split v0.11.2 // indirect
 	github.com/virtuald/go-ordered-json v0.0.0-20170621173500-b18e6e673d74 // indirect
-	github.com/xanzy/go-gitlab v0.58.0 // indirect
-	github.com/xanzy/ssh-agent v0.3.0 // indirect
+	github.com/xanzy/go-gitlab v0.74.0 // indirect
+	github.com/xanzy/ssh-agent v0.3.2 // indirect
 	github.com/xlab/treeprint v1.1.0 // indirect
-	go.starlark.net v0.0.0-20200306205701-8dd3e2ee1dd5 // indirect
-	golang.org/x/net v0.0.0-20220708220712-1185a9018129 // indirect
-	golang.org/x/oauth2 v0.0.0-20220718184931-c8730f7fcb92 // indirect
-	golang.org/x/sync v0.0.0-20220513210516-0976fa681c29 // indirect
-	golang.org/x/sys v0.0.0-20220715151400-c0bba94af5f8 // indirect
-	golang.org/x/text v0.3.7 // indirect
-	golang.org/x/time v0.0.0-20220411224347-583f2d630306 // indirect
+	github.com/yvasiyarov/go-metrics v0.0.0-20140926110328-57bccd1ccd43 // indirect
+	github.com/yvasiyarov/gorelic v0.0.0-20141212073537-a9bba5b9ab50 // indirect
+	github.com/yvasiyarov/newrelic_platform_go v0.0.0-20140908184405-b21fdbd4370f // indirect
+	go.starlark.net v0.0.0-20221028183056-acb66ad56dd2 // indirect
+	golang.org/x/mod v0.6.0 // indirect
+	golang.org/x/net v0.2.0 // indirect
+	golang.org/x/oauth2 v0.2.0 // indirect
+	golang.org/x/sync v0.1.0 // indirect
+	golang.org/x/sys v0.2.0 // indirect
+	golang.org/x/text v0.4.0 // indirect
+	golang.org/x/time v0.2.0 // indirect
+	golang.org/x/tools v0.1.12 // indirect
 	gomodules.xyz/jsonpatch/v2 v2.2.0 // indirect
 	google.golang.org/appengine v1.6.7 // indirect
-	google.golang.org/protobuf v1.28.0 // indirect
+	google.golang.org/protobuf v1.28.1 // indirect
 	gopkg.in/inf.v0 v0.9.1 // indirect
 	gopkg.in/warnings.v0 v0.1.2 // indirect
 	gopkg.in/yaml.v2 v2.4.0 // indirect
 	gopkg.in/yaml.v3 v3.0.1 // indirect
-	k8s.io/component-base v0.24.3 // indirect
-	k8s.io/klog/v2 v2.60.1 // indirect
-	k8s.io/kube-openapi v0.0.0-20220413171646-5e7f5fdc6da6 // indirect
-	k8s.io/utils v0.0.0-20220210201930-3a6ce19ff2f9 // indirect
-	sigs.k8s.io/json v0.0.0-20220525155127-227cbc7cc124 // indirect
-	sigs.k8s.io/structured-merge-diff/v4 v4.2.1 // indirect
+	k8s.io/component-base v0.25.4 // indirect
+	k8s.io/klog/v2 v2.80.1 // indirect
+	k8s.io/kube-openapi v0.0.0-20221110221610-a28e98eb7c70 // indirect
+	k8s.io/utils v0.0.0-20221108210102-8e77b1f39fe2 // indirect
+	sigs.k8s.io/json v0.0.0-20221116044647-bc3834ca7abd // indirect
+	sigs.k8s.io/structured-merge-diff/v4 v4.2.3 // indirect
 )

internal/bootstrap/git/commit_options.go

@@ -1,42 +0,0 @@
-package git
-
-// Option is a some configuration that modifies options for a commit.
-type Option interface {
-	// ApplyToCommit applies this configuration to a given commit option.
-	ApplyToCommit(*CommitOptions)
-}
-
-// CommitOptions contains options for making a commit.
-type CommitOptions struct {
-	*GPGSigningInfo
-}
-
-// GPGSigningInfo contains information for signing a commit.
-type GPGSigningInfo struct {
-	KeyRingPath string
-	Passphrase  string
-	KeyID       string
-}
-
-type GpgSigningOption struct {
-	*GPGSigningInfo
-}
-
-func (w GpgSigningOption) ApplyToCommit(in *CommitOptions) {
-	in.GPGSigningInfo = w.GPGSigningInfo
-}
-
-func WithGpgSigningOption(path, passphrase, keyID string) Option {
-	// Return nil if no path is set, even if other options are configured.
-	if path == "" {
-		return GpgSigningOption{}
-	}
-
-	return GpgSigningOption{
-		GPGSigningInfo: &GPGSigningInfo{
-			KeyRingPath: path,
-			Passphrase:  passphrase,
-			KeyID:       keyID,
-		},
-	}
-}

internal/bootstrap/git/git.go

@@ -1,52 +0,0 @@
-/*
-Copyright 2021 The Flux authors
-
-Licensed under the Apache License, Version 2.0 (the "License");
-you may not use this file except in compliance with the License.
-You may obtain a copy of the License at
-
-    http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing, software
-distributed under the License is distributed on an "AS IS" BASIS,
-WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-See the License for the specific language governing permissions and
-limitations under the License.
-*/
-
-package git
-
-import (
-	"context"
-	"errors"
-	"io"
-)
-
-var (
-	ErrNoGitRepository = errors.New("no git repository")
-	ErrNoStagedFiles   = errors.New("no staged files")
-)
-
-type Author struct {
-	Name  string
-	Email string
-}
-
-type Commit struct {
-	Author
-	Hash    string
-	Message string
-}
-
-// Git is an interface for basic Git operations on a single branch of a
-// remote repository.
-type Git interface {
-	Init(url, branch string) (bool, error)
-	Clone(ctx context.Context, url, branch string, caBundle []byte) (bool, error)
-	Write(path string, reader io.Reader) error
-	Commit(message Commit, options ...Option) (string, error)
-	Push(ctx context.Context, caBundle []byte) error
-	Status() (bool, error)
-	Head() (string, error)
-	Path() string
-}

internal/bootstrap/git/gogit/gogit.go

@@ -1,296 +0,0 @@
-/*
-Copyright 2021 The Flux authors
-
-Licensed under the Apache License, Version 2.0 (the "License");
-you may not use this file except in compliance with the License.
-You may obtain a copy of the License at
-
-    http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing, software
-distributed under the License is distributed on an "AS IS" BASIS,
-WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-See the License for the specific language governing permissions and
-limitations under the License.
-*/
-
-package gogit
-
-import (
-	"context"
-	"fmt"
-	"io"
-	"os"
-	"path/filepath"
-	"strings"
-	"time"
-
-	"github.com/ProtonMail/go-crypto/openpgp"
-	gogit "github.com/go-git/go-git/v5"
-	"github.com/go-git/go-git/v5/config"
-	"github.com/go-git/go-git/v5/plumbing"
-	"github.com/go-git/go-git/v5/plumbing/object"
-	"github.com/go-git/go-git/v5/plumbing/transport"
-
-	"github.com/fluxcd/flux2/internal/bootstrap/git"
-)
-
-type GoGit struct {
-	path       string
-	auth       transport.AuthMethod
-	repository *gogit.Repository
-}
-
-type CommitOptions struct {
-	GpgKeyPath       string
-	GpgKeyPassphrase string
-	KeyID            string
-}
-
-func New(path string, auth transport.AuthMethod) *GoGit {
-	return &GoGit{
-		path: path,
-		auth: auth,
-	}
-}
-
-func (g *GoGit) Init(url, branch string) (bool, error) {
-	if g.repository != nil {
-		return false, nil
-	}
-
-	r, err := gogit.PlainInit(g.path, false)
-	if err != nil {
-		return false, err
-	}
-	if _, err = r.CreateRemote(&config.RemoteConfig{
-		Name: gogit.DefaultRemoteName,
-		URLs: []string{url},
-	}); err != nil {
-		return false, err
-	}
-	branchRef := plumbing.NewBranchReferenceName(branch)
-	if err = r.CreateBranch(&config.Branch{
-		Name:   branch,
-		Remote: gogit.DefaultRemoteName,
-		Merge:  branchRef,
-	}); err != nil {
-		return false, err
-	}
-	// PlainInit assumes the initial branch to always be master, we can
-	// overwrite this by setting the reference of the Storer to a new
-	// symbolic reference (as there are no commits yet) that points
-	// the HEAD to our new branch.
-	if err = r.Storer.SetReference(plumbing.NewSymbolicReference(plumbing.HEAD, branchRef)); err != nil {
-		return false, err
-	}
-
-	g.repository = r
-	return true, nil
-}
-
-func (g *GoGit) Clone(ctx context.Context, url, branch string, caBundle []byte) (bool, error) {
-	branchRef := plumbing.NewBranchReferenceName(branch)
-	r, err := gogit.PlainCloneContext(ctx, g.path, false, &gogit.CloneOptions{
-		URL:           url,
-		Auth:          g.auth,
-		RemoteName:    gogit.DefaultRemoteName,
-		ReferenceName: branchRef,
-		SingleBranch:  true,
-
-		NoCheckout: false,
-		Progress:   nil,
-		Tags:       gogit.NoTags,
-		CABundle:   caBundle,
-	})
-	if err != nil {
-		if err == transport.ErrEmptyRemoteRepository || isRemoteBranchNotFoundErr(err, branchRef.String()) {
-			return g.Init(url, branch)
-		}
-		return false, err
-	}
-
-	g.repository = r
-	return true, nil
-}
-
-func (g *GoGit) Write(path string, reader io.Reader) error {
-	if g.repository == nil {
-		return git.ErrNoGitRepository
-	}
-
-	wt, err := g.repository.Worktree()
-	if err != nil {
-		return err
-	}
-
-	f, err := wt.Filesystem.Create(path)
-	if err != nil {
-		return err
-	}
-	defer f.Close()
-
-	_, err = io.Copy(f, reader)
-	return err
-}
-
-func (g *GoGit) Commit(message git.Commit, opts ...git.Option) (string, error) {
-	if g.repository == nil {
-		return "", git.ErrNoGitRepository
-	}
-
-	wt, err := g.repository.Worktree()
-	if err != nil {
-		return "", err
-	}
-
-	status, err := wt.Status()
-	if err != nil {
-		return "", err
-	}
-
-	// apply the options
-	options := &git.CommitOptions{}
-	for _, opt := range opts {
-		opt.ApplyToCommit(options)
-	}
-
-	// go-git has [a bug](https://github.com/go-git/go-git/issues/253)
-	// whereby it thinks broken symlinks to absolute paths are
-	// modified. There's no circumstance in which we want to commit a
-	// change to a broken symlink: so, detect and skip those.
-	var changed bool
-	for file, _ := range status {
-		abspath := filepath.Join(g.path, file)
-		info, err := os.Lstat(abspath)
-		if err != nil {
-			return "", fmt.Errorf("checking if %s is a symlink: %w", file, err)
-		}
-		if info.Mode()&os.ModeSymlink > 0 {
-			// symlinks are OK; broken symlinks are probably a result
-			// of the bug mentioned above, but not of interest in any
-			// case.
-			if _, err := os.Stat(abspath); os.IsNotExist(err) {
-				continue
-			}
-		}
-		_, _ = wt.Add(file)
-		changed = true
-	}
-
-	if !changed {
-		head, err := g.repository.Head()
-		if err != nil {
-			return "", err
-		}
-		return head.Hash().String(), git.ErrNoStagedFiles
-	}
-
-	commitOpts := &gogit.CommitOptions{
-		Author: &object.Signature{
-			Name:  message.Name,
-			Email: message.Email,
-			When:  time.Now(),
-		},
-	}
-
-	if options.GPGSigningInfo != nil {
-		entity, err := getOpenPgpEntity(*options.GPGSigningInfo)
-		if err != nil {
-			return "", err
-		}
-
-		commitOpts.SignKey = entity
-	}
-
-	commit, err := wt.Commit(message.Message, commitOpts)
-	if err != nil {
-		return "", err
-	}
-	return commit.String(), nil
-}
-
-func (g *GoGit) Push(ctx context.Context, caBundle []byte) error {
-	if g.repository == nil {
-		return git.ErrNoGitRepository
-	}
-
-	return g.repository.PushContext(ctx, &gogit.PushOptions{
-		RemoteName: gogit.DefaultRemoteName,
-		Auth:       g.auth,
-		Progress:   nil,
-		CABundle:   caBundle,
-	})
-}
-
-func (g *GoGit) Status() (bool, error) {
-	if g.repository == nil {
-		return false, git.ErrNoGitRepository
-	}
-	wt, err := g.repository.Worktree()
-	if err != nil {
-		return false, err
-	}
-	status, err := wt.Status()
-	if err != nil {
-		return false, err
-	}
-	return status.IsClean(), nil
-}
-
-func (g *GoGit) Head() (string, error) {
-	if g.repository == nil {
-		return "", git.ErrNoGitRepository
-	}
-	head, err := g.repository.Head()
-	if err != nil {
-		return "", err
-	}
-	return head.Hash().String(), nil
-}
-
-func (g *GoGit) Path() string {
-	return g.path
-}
-
-func isRemoteBranchNotFoundErr(err error, ref string) bool {
-	return strings.Contains(err.Error(), fmt.Sprintf("couldn't find remote ref %q", ref))
-}
-
-func getOpenPgpEntity(info git.GPGSigningInfo) (*openpgp.Entity, error) {
-	r, err := os.Open(info.KeyRingPath)
-	if err != nil {
-		return nil, fmt.Errorf("unable to open GPG key ring: %w", err)
-	}
-
-	entityList, err := openpgp.ReadKeyRing(r)
-	if err != nil {
-		return nil, err
-	}
-
-	if len(entityList) == 0 {
-		return nil, fmt.Errorf("empty GPG key ring")
-	}
-
-	var entity *openpgp.Entity
-	if info.KeyID != "" {
-		for _, ent := range entityList {
-			if ent.PrimaryKey.KeyIdString() == info.KeyID {
-				entity = ent
-			}
-		}
-
-		if entity == nil {
-			return nil, fmt.Errorf("no GPG private key matching key id '%s' found", info.KeyID)
-		}
-	} else {
-		entity = entityList[0]
-	}
-
-	err = entity.PrivateKey.Decrypt([]byte(info.Passphrase))
-	if err != nil {
-		return nil, fmt.Errorf("unable to decrypt GPG private key: %w", err)
-	}
-
-	return entity, nil
-}

internal/bootstrap/git/gogit/gogit_test.go

@@ -1,67 +0,0 @@
-//go:build unit
-// +build unit
-
-package gogit
-
-import (
-	"testing"
-
-	"github.com/fluxcd/flux2/internal/bootstrap/git"
-)
-
-func TestGetOpenPgpEntity(t *testing.T) {
-	tests := []struct {
-		name       string
-		keyPath    string
-		passphrase string
-		id         string
-		expectErr  bool
-	}{
-		{
-			name:       "no default key id given",
-			keyPath:    "testdata/private.key",
-			passphrase: "flux",
-			id:         "",
-			expectErr:  false,
-		},
-		{
-			name:       "key id given",
-			keyPath:    "testdata/private.key",
-			passphrase: "flux",
-			id:         "0619327DBD777415",
-			expectErr:  false,
-		},
-		{
-			name:       "wrong key id",
-			keyPath:    "testdata/private.key",
-			passphrase: "flux",
-			id:         "0619327DBD777416",
-			expectErr:  true,
-		},
-		{
-			name:       "wrong password",
-			keyPath:    "testdata/private.key",
-			passphrase: "fluxe",
-			id:         "0619327DBD777415",
-			expectErr:  true,
-		},
-	}
-
-	for _, tt := range tests {
-		t.Run(tt.name, func(t *testing.T) {
-			gpgInfo := git.GPGSigningInfo{
-				KeyRingPath: tt.keyPath,
-				Passphrase:  tt.passphrase,
-				KeyID:       tt.id,
-			}
-
-			_, err := getOpenPgpEntity(gpgInfo)
-			if err != nil && !tt.expectErr {
-				t.Errorf("unexpected error: %s", err)
-			}
-			if err == nil && tt.expectErr {
-				t.Errorf("expected error when %s", tt.name)
-			}
-		})
-	}
-}

internal/build/build.go

@@ -42,8 +42,8 @@ import (
 
 	kustomizev1 "github.com/fluxcd/kustomize-controller/api/v1beta2"
 	"github.com/fluxcd/pkg/kustomize"
-	"github.com/fluxcd/pkg/kustomize/filesys"
 	runclient "github.com/fluxcd/pkg/runtime/client"
+	"sigs.k8s.io/kustomize/kyaml/filesys"
 
 	"github.com/fluxcd/flux2/internal/utils"
 )
@@ -76,10 +76,13 @@ type Builder struct {
 	kustomization *kustomizev1.Kustomization
 	timeout       time.Duration
 	spinner       *yacspin.Spinner
+	dryRun        bool
 }
 
+// BuilderOptionFunc is a function that configures a Builder
 type BuilderOptionFunc func(b *Builder) error
 
+// WithKustomizationFile sets the kustomization file
 func WithKustomizationFile(file string) BuilderOptionFunc {
 	return func(b *Builder) error {
 		b.kustomizationFile = file
@@ -87,6 +90,7 @@ func WithKustomizationFile(file string) BuilderOptionFunc {
 	}
 }
 
+// WithTimeout sets the timeout for the builder
 func WithTimeout(timeout time.Duration) BuilderOptionFunc {
 	return func(b *Builder) error {
 		b.timeout = timeout
@@ -116,24 +120,47 @@ func WithProgressBar() BuilderOptionFunc {
 	}
 }
 
+// WithClientConfig sets the client configuration
+func WithClientConfig(rcg *genericclioptions.ConfigFlags, clientOpts *runclient.Options) BuilderOptionFunc {
+	return func(b *Builder) error {
+		kubeClient, err := utils.KubeClient(rcg, clientOpts)
+		if err != nil {
+			return err
+		}
+
+		restMapper, err := rcg.ToRESTMapper()
+		if err != nil {
+			return err
+		}
+		b.client = kubeClient
+		b.restMapper = restMapper
+		b.namespace = *rcg.Namespace
+		return nil
+	}
+}
+
+// WithDryRun sets the dry-run flag
+func WithDryRun(dryRun bool) BuilderOptionFunc {
+	return func(b *Builder) error {
+		b.dryRun = dryRun
+		return nil
+	}
+}
+
 // NewBuilder returns a new Builder
-// to dp : create functional options
-func NewBuilder(rcg *genericclioptions.ConfigFlags, clientOpts *runclient.Options, name, resources string, opts ...BuilderOptionFunc) (*Builder, error) {
-	kubeClient, err := utils.KubeClient(rcg, clientOpts)
-	if err != nil {
-		return nil, err
-	}
-
-	restMapper, err := rcg.ToRESTMapper()
-	if err != nil {
-		return nil, err
-	}
-
+// It takes a kustomization name and a path to the resources
+// It also takes a list of BuilderOptionFunc to configure the builder
+// One of the options is WithClientConfig, that must be provided for the builder to work
+// with the k8s cluster
+// One other option is WithKustomizationFile, that must be provided for the builder to work
+// with a local kustomization file. If the kustomization file is not provided, the builder
+// will try to retrieve the kustomization object from the k8s cluster.
+// WithDryRun sets the dry-run flag, and needs to be provided if the builder is used for
+// a dry-run. This flag works in conjunction with WithKustomizationFile, because the
+// kustomization object is not retrieved from the k8s cluster when the dry-run flag is set.
+func NewBuilder(name, resources string, opts ...BuilderOptionFunc) (*Builder, error) {
 	b := &Builder{
-		client:        kubeClient,
-		restMapper:    restMapper,
 		name:          name,
-		namespace:     *rcg.Namespace,
 		resourcesPath: resources,
 	}
 
@@ -147,6 +174,14 @@ func NewBuilder(rcg *genericclioptions.ConfigFlags, clientOpts *runclient.Option
 		b.timeout = defaultTimeout
 	}
 
+	if b.dryRun && b.kustomizationFile == "" {
+		return nil, fmt.Errorf("kustomization file is required for dry-run")
+	}
+
+	if !b.dryRun && b.client == nil {
+		return nil, fmt.Errorf("client is required for live run")
+	}
+
 	return b, nil
 }
 
@@ -188,7 +223,7 @@ func (b *Builder) build() (m resmap.ResMap, err error) {
 	defer cancel()
 
 	// Get the kustomization object
-	k := &kustomizev1.Kustomization{}
+	var k *kustomizev1.Kustomization
 	if b.kustomizationFile != "" {
 		k, err = b.unMarshallKustomization()
 		if err != nil {
@@ -273,7 +308,7 @@ func (b *Builder) generate(kustomization kustomizev1.Kustomization, dirPath stri
 	if err != nil {
 		return "", err
 	}
-	gen := kustomize.NewGenerator(unstructured.Unstructured{Object: data})
+	gen := kustomize.NewGenerator("", unstructured.Unstructured{Object: data})
 
 	// acuire the lock
 	b.mu.Lock()
@@ -283,17 +318,13 @@ func (b *Builder) generate(kustomization kustomizev1.Kustomization, dirPath stri
 }
 
 func (b *Builder) do(ctx context.Context, kustomization kustomizev1.Kustomization, dirPath string) (resmap.ResMap, error) {
-	// TODO(hidde): provide option to enforce FS boundaries of local build
-	fs, err := filesys.MakeFsOnDiskSecureBuild("/")
-	if err != nil {
-		return nil, fmt.Errorf("kustomization build failed: %w", err)
-	}
+	fs := filesys.MakeFsOnDisk()
 
 	// acuire the lock
 	b.mu.Lock()
 	defer b.mu.Unlock()
 
-	m, err := kustomize.BuildKustomization(fs, dirPath)
+	m, err := kustomize.Build(fs, dirPath)
 	if err != nil {
 		return nil, fmt.Errorf("kustomize build failed: %w", err)
 	}
@@ -305,7 +336,7 @@ func (b *Builder) do(ctx context.Context, kustomization kustomizev1.Kustomizatio
 			if err != nil {
 				return nil, err
 			}
-			outRes, err := kustomize.SubstituteVariables(ctx, b.client, unstructured.Unstructured{Object: data}, res)
+			outRes, err := kustomize.SubstituteVariables(ctx, b.client, unstructured.Unstructured{Object: data}, res, b.dryRun)
 			if err != nil {
 				return nil, fmt.Errorf("var substitution failed for '%s': %w", res.GetName(), err)
 			}

internal/build/diff.go

@@ -27,20 +27,22 @@ import (
 	"sort"
 	"strings"
 
-	"github.com/fluxcd/flux2/pkg/printers"
-	kustomizev1 "github.com/fluxcd/kustomize-controller/api/v1beta2"
-	"github.com/fluxcd/pkg/ssa"
 	"github.com/gonvenience/bunt"
 	"github.com/gonvenience/ytbx"
 	"github.com/google/go-cmp/cmp"
-	"github.com/hashicorp/go-multierror"
 	"github.com/homeport/dyff/pkg/dyff"
 	"github.com/lucasb-eyer/go-colorful"
 	"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured"
 	"k8s.io/apimachinery/pkg/runtime/schema"
+	"k8s.io/apimachinery/pkg/util/errors"
 	"sigs.k8s.io/cli-utils/pkg/kstatus/polling"
 	"sigs.k8s.io/cli-utils/pkg/object"
 	"sigs.k8s.io/yaml"
+
+	"github.com/fluxcd/pkg/ssa"
+
+	"github.com/fluxcd/flux2/pkg/printers"
+	kustomizev1 "github.com/fluxcd/kustomize-controller/api/v1beta2"
 )
 
 func (b *Builder) Manager() (*ssa.ResourceManager, error) {
@@ -85,7 +87,7 @@ func (b *Builder) Diff() (string, bool, error) {
 		}
 	}
 
-	var diffErrs error
+	var diffErrs []error
 	// create an inventory of objects to be reconciled
 	newInventory := newInventory()
 	for _, obj := range objects {
@@ -97,7 +99,7 @@ func (b *Builder) Diff() (string, bool, error) {
 		change, liveObject, mergedObject, err := resourceManager.Diff(ctx, obj, diffOptions)
 		if err != nil {
 			// gather errors and continue, as we want to see all the diffs
-			diffErrs = multierror.Append(diffErrs, err)
+			diffErrs = append(diffErrs, err)
 			continue
 		}
 
@@ -135,7 +137,7 @@ func (b *Builder) Diff() (string, bool, error) {
 		b.spinner.Message("processing inventory")
 	}
 
-	if b.kustomization.Spec.Prune && diffErrs == nil {
+	if b.kustomization.Spec.Prune && len(diffErrs) == 0 {
 		oldStatus := b.kustomization.Status.DeepCopy()
 		if oldStatus.Inventory != nil {
 			diffObjects, err := diffInventory(oldStatus.Inventory, newInventory)
@@ -155,7 +157,7 @@ func (b *Builder) Diff() (string, bool, error) {
 		}
 	}
 
-	return output.String(), createdOrDrifted, diffErrs
+	return output.String(), createdOrDrifted, errors.Reduce(errors.Flatten(errors.NewAggregate(diffErrs)))
 }
 
 func writeYamls(liveObject, mergedObject *unstructured.Unstructured) (string, string, string, error) {

internal/flags/source_oci_provider.go

@@ -21,6 +21,7 @@ import (
 	"strings"
 
 	"github.com/fluxcd/flux2/internal/utils"
+	"github.com/fluxcd/pkg/oci"
 	sourcev1 "github.com/fluxcd/source-controller/api/v1beta2"
 )
 
@@ -31,6 +32,13 @@ var supportedSourceOCIProviders = []string{
 	sourcev1.GoogleOCIProvider,
 }
 
+var sourceOCIProvidersToOCIProvider = map[string]oci.Provider{
+	sourcev1.GenericOCIProvider: oci.ProviderGeneric,
+	sourcev1.AmazonOCIProvider:  oci.ProviderAWS,
+	sourcev1.AzureOCIProvider:   oci.ProviderAzure,
+	sourcev1.GoogleOCIProvider:  oci.ProviderGCP,
+}
+
 type SourceOCIProvider string
 
 func (p *SourceOCIProvider) String() string {
@@ -60,3 +68,12 @@ func (p *SourceOCIProvider) Description() string {
 		strings.Join(supportedSourceOCIProviders, ", "),
 	)
 }
+
+func (p *SourceOCIProvider) ToOCIProvider() (oci.Provider, error) {
+	value, ok := sourceOCIProvidersToOCIProvider[p.String()]
+	if !ok {
+		return 0, fmt.Errorf("no mapping between source OCI provider %s and OCI provider", p.String())
+	}
+
+	return value, nil
+}

manifests/bases/helm-controller/kustomization.yaml

@@ -1,9 +1,11 @@
 apiVersion: kustomize.config.k8s.io/v1beta1
 kind: Kustomization
 resources:
-- https://github.com/fluxcd/helm-controller/releases/download/v0.22.2/helm-controller.crds.yaml
-- https://github.com/fluxcd/helm-controller/releases/download/v0.22.2/helm-controller.deployment.yaml
+- https://github.com/fluxcd/helm-controller/releases/download/v0.27.0/helm-controller.crds.yaml
+- https://github.com/fluxcd/helm-controller/releases/download/v0.27.0/helm-controller.deployment.yaml
 - account.yaml
+transformers:
+- labels.yaml
 patchesJson6902:
 - target:
     group: apps

manifests/bases/helm-controller/labels.yaml

@@ -0,0 +1,9 @@
+apiVersion: builtin
+kind: LabelTransformer
+metadata:
+  name: labels
+labels:
+  app.kubernetes.io/component: helm-controller
+fieldSpecs:
+  - path: metadata/labels
+    create: true

manifests/bases/image-automation-controller/kustomization.yaml

@@ -1,9 +1,11 @@
 apiVersion: kustomize.config.k8s.io/v1beta1
 kind: Kustomization
 resources:
-- https://github.com/fluxcd/image-automation-controller/releases/download/v0.24.1/image-automation-controller.crds.yaml
-- https://github.com/fluxcd/image-automation-controller/releases/download/v0.24.1/image-automation-controller.deployment.yaml
+- https://github.com/fluxcd/image-automation-controller/releases/download/v0.27.0/image-automation-controller.crds.yaml
+- https://github.com/fluxcd/image-automation-controller/releases/download/v0.27.0/image-automation-controller.deployment.yaml
 - account.yaml
+transformers:
+- labels.yaml
 patchesJson6902:
 - target:
     group: apps

manifests/bases/image-automation-controller/labels.yaml

@@ -0,0 +1,9 @@
+apiVersion: builtin
+kind: LabelTransformer
+metadata:
+  name: labels
+labels:
+  app.kubernetes.io/component: image-automation-controller
+fieldSpecs:
+  - path: metadata/labels
+    create: true

manifests/bases/image-reflector-controller/kustomization.yaml

@@ -1,9 +1,11 @@
 apiVersion: kustomize.config.k8s.io/v1beta1
 kind: Kustomization
 resources:
-- https://github.com/fluxcd/image-reflector-controller/releases/download/v0.20.0/image-reflector-controller.crds.yaml
-- https://github.com/fluxcd/image-reflector-controller/releases/download/v0.20.0/image-reflector-controller.deployment.yaml
+- https://github.com/fluxcd/image-reflector-controller/releases/download/v0.23.0/image-reflector-controller.crds.yaml
+- https://github.com/fluxcd/image-reflector-controller/releases/download/v0.23.0/image-reflector-controller.deployment.yaml
 - account.yaml
+transformers:
+- labels.yaml
 patchesJson6902:
 - target:
     group: apps

manifests/bases/image-reflector-controller/labels.yaml

@@ -0,0 +1,9 @@
+apiVersion: builtin
+kind: LabelTransformer
+metadata:
+  name: labels
+labels:
+  app.kubernetes.io/component: image-reflector-controller
+fieldSpecs:
+  - path: metadata/labels
+    create: true

manifests/bases/kustomize-controller/kustomization.yaml

@@ -1,9 +1,11 @@
 apiVersion: kustomize.config.k8s.io/v1beta1
 kind: Kustomization
 resources:
-- https://github.com/fluxcd/kustomize-controller/releases/download/v0.27.0/kustomize-controller.crds.yaml
-- https://github.com/fluxcd/kustomize-controller/releases/download/v0.27.0/kustomize-controller.deployment.yaml
+- https://github.com/fluxcd/kustomize-controller/releases/download/v0.31.0/kustomize-controller.crds.yaml
+- https://github.com/fluxcd/kustomize-controller/releases/download/v0.31.0/kustomize-controller.deployment.yaml
 - account.yaml
+transformers:
+- labels.yaml
 patchesJson6902:
 - target:
     group: apps
@@ -11,4 +13,3 @@ patchesJson6902:
     kind: Deployment
     name: kustomize-controller
   path: patch.yaml
-

manifests/bases/kustomize-controller/labels.yaml

@@ -0,0 +1,9 @@
+apiVersion: builtin
+kind: LabelTransformer
+metadata:
+  name: labels
+labels:
+  app.kubernetes.io/component: kustomize-controller
+fieldSpecs:
+  - path: metadata/labels
+    create: true

manifests/bases/notification-controller/kustomization.yaml

@@ -1,9 +1,11 @@
 apiVersion: kustomize.config.k8s.io/v1beta1
 kind: Kustomization
 resources:
-- https://github.com/fluxcd/notification-controller/releases/download/v0.25.1/notification-controller.crds.yaml
-- https://github.com/fluxcd/notification-controller/releases/download/v0.25.1/notification-controller.deployment.yaml
+- https://github.com/fluxcd/notification-controller/releases/download/v0.29.0/notification-controller.crds.yaml
+- https://github.com/fluxcd/notification-controller/releases/download/v0.29.0/notification-controller.deployment.yaml
 - account.yaml
+transformers:
+- labels.yaml
 patchesJson6902:
   - target:
       group: apps

manifests/bases/notification-controller/labels.yaml

@@ -0,0 +1,9 @@
+apiVersion: builtin
+kind: LabelTransformer
+metadata:
+  name: labels
+labels:
+  app.kubernetes.io/component: notification-controller
+fieldSpecs:
+  - path: metadata/labels
+    create: true

manifests/bases/source-controller/kustomization.yaml

@@ -1,9 +1,11 @@
 apiVersion: kustomize.config.k8s.io/v1beta1
 kind: Kustomization
 resources:
-- https://github.com/fluxcd/source-controller/releases/download/v0.26.1/source-controller.crds.yaml
-- https://github.com/fluxcd/source-controller/releases/download/v0.26.1/source-controller.deployment.yaml
+- https://github.com/fluxcd/source-controller/releases/download/v0.32.1/source-controller.crds.yaml
+- https://github.com/fluxcd/source-controller/releases/download/v0.32.1/source-controller.deployment.yaml
 - account.yaml
+transformers:
+- labels.yaml
 patchesJson6902:
 - target:
     group: apps
@@ -11,4 +13,3 @@ patchesJson6902:
     kind: Deployment
     name: source-controller
   path: patch.yaml
-

manifests/bases/source-controller/labels.yaml

@@ -0,0 +1,9 @@
+apiVersion: builtin
+kind: LabelTransformer
+metadata:
+  name: labels
+labels:
+  app.kubernetes.io/component: source-controller
+fieldSpecs:
+  - path: metadata/labels
+    create: true

manifests/crds/kustomization.yaml

@@ -1,9 +1,9 @@
 apiVersion: kustomize.config.k8s.io/v1beta1
 kind: Kustomization
 resources:
-- https://github.com/fluxcd/source-controller/releases/download/v0.26.1/source-controller.crds.yaml
-- https://github.com/fluxcd/kustomize-controller/releases/download/v0.27.0/kustomize-controller.crds.yaml
-- https://github.com/fluxcd/helm-controller/releases/download/v0.22.2/helm-controller.crds.yaml
-- https://github.com/fluxcd/notification-controller/releases/download/v0.25.1/notification-controller.crds.yaml
-- https://github.com/fluxcd/image-reflector-controller/releases/download/v0.20.0/image-reflector-controller.crds.yaml
-- https://github.com/fluxcd/image-automation-controller/releases/download/v0.24.1/image-automation-controller.crds.yaml
+- https://github.com/fluxcd/source-controller/releases/download/v0.32.1/source-controller.crds.yaml
+- https://github.com/fluxcd/kustomize-controller/releases/download/v0.31.0/kustomize-controller.crds.yaml
+- https://github.com/fluxcd/helm-controller/releases/download/v0.27.0/helm-controller.crds.yaml
+- https://github.com/fluxcd/notification-controller/releases/download/v0.29.0/notification-controller.crds.yaml
+- https://github.com/fluxcd/image-reflector-controller/releases/download/v0.23.0/image-reflector-controller.crds.yaml
+- https://github.com/fluxcd/image-automation-controller/releases/download/v0.27.0/image-automation-controller.crds.yaml

manifests/install/kustomization.yaml

@@ -13,3 +13,14 @@ resources:
   - ../policies
 transformers:
   - labels.yaml
+images:
+  - name: fluxcd/source-controller
+    newName: ghcr.io/fluxcd/source-controller
+  - name: fluxcd/kustomize-controller
+    newName: ghcr.io/fluxcd/kustomize-controller
+  - name: fluxcd/helm-controller
+    newName: ghcr.io/fluxcd/helm-controller
+  - name: fluxcd/image-reflector-controller
+    newName: ghcr.io/fluxcd/image-reflector-controller
+  - name: fluxcd/image-automation-controller
+    newName: ghcr.io/fluxcd/image-automation-controller

manifests/integrations/eventhub-credentials-sync/_cronjobs/generic/secret-azure-credentials.yaml

@@ -10,5 +10,5 @@ metadata:
 type: Opaque
 # This is just a example secret, you should never store secrets in git.
 # One way forward can be to use sealed-secrets or SOPS
-# https://fluxcd.io/docs/guides/sealed-secrets/
-# https://fluxcd.io/docs/guides/mozilla-sops/
+# https://fluxcd.io/flux/guides/sealed-secrets/
+# https://fluxcd.io/flux/guides/mozilla-sops/

manifests/integrations/registry-credentials-sync/aws/config-patches.yaml

@@ -24,8 +24,8 @@ metadata:
 ## If not using IRSA, set the AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY environment variables
 ## Store these values in a Secret and load them in the container using envFrom.
 ## For managing this secret via GitOps, consider using SOPS or SealedSecrets and add that manifest in a resource file for this kustomize build.
-##   https://fluxcd.io/docs/guides/mozilla-sops/
-##   https://fluxcd.io/docs/guides/sealed-secrets/
+##   https://fluxcd.io/flux/guides/mozilla-sops/
+##   https://fluxcd.io/flux/guides/sealed-secrets/
 # ---
 # apiVersion: apps/v1
 # kind: Deployment

manifests/monitoring/kube-prometheus-stack/release.yaml

@@ -6,11 +6,13 @@ spec:
   interval: 5m
   chart:
     spec:
-      version: "35.x"
+      version: "41.x"
       chart: kube-prometheus-stack
       sourceRef:
         kind: HelmRepository
         name: prometheus-community
+      verify:
+        provider: cosign
       interval: 60m
   install:
     crds: Create

manifests/monitoring/kube-prometheus-stack/repository.yaml

@@ -4,4 +4,5 @@ metadata:
   name: prometheus-community
 spec:
   interval: 120m
-  url: https://prometheus-community.github.io/helm-charts
+  type: oci
+  url: oci://ghcr.io/prometheus-community/charts

manifests/monitoring/monitoring-config/dashboards/control-plane.json

@@ -548,7 +548,7 @@
       "steppedLine": false,
       "targets": [
         {
-          "expr": "rate(go_memstats_alloc_bytes_total{namespace=\"$namespace\",pod=~\".*-controller-.*\"}[1m])",
+          "expr": "sum(container_memory_working_set_bytes{namespace=\"$namespace\",container!=\"POD\",container!=\"\",pod=~\".*-controller-.*\"}) by (pod)",
           "hide": false,
           "interval": "",
           "legendFormat": "{{pod}}",

manifests/monitoring/monitoring-config/podmonitor.yaml

@@ -23,3 +23,8 @@ spec:
           - image-reflector-controller
   podMetricsEndpoints:
     - port: http-prom
+      relabelings:
+        # https://github.com/prometheus-operator/prometheus-operator/issues/4816
+        - sourceLabels: [__meta_kubernetes_pod_phase]
+          action: keep
+          regex: Running

pkg/bootstrap/bootstrap_plain_git.go

@@ -19,12 +19,14 @@ package bootstrap
 import (
 	"context"
 	"fmt"
+	"io"
 	"os"
 	"path/filepath"
 	"strings"
 	"time"
 
-	gogit "github.com/go-git/go-git/v5"
+	"github.com/ProtonMail/go-crypto/openpgp"
+	gogit "github.com/fluxcd/go-git/v5"
 	corev1 "k8s.io/api/core/v1"
 	"k8s.io/apimachinery/pkg/runtime/schema"
 	"k8s.io/apimachinery/pkg/util/wait"
@@ -38,7 +40,6 @@ import (
 	"github.com/fluxcd/pkg/kustomize/filesys"
 	runclient "github.com/fluxcd/pkg/runtime/client"
 
-	"github.com/fluxcd/flux2/internal/bootstrap/git"
 	"github.com/fluxcd/flux2/internal/utils"
 	"github.com/fluxcd/flux2/pkg/log"
 	"github.com/fluxcd/flux2/pkg/manifestgen/install"
@@ -46,28 +47,29 @@ import (
 	"github.com/fluxcd/flux2/pkg/manifestgen/sourcesecret"
 	"github.com/fluxcd/flux2/pkg/manifestgen/sync"
 	"github.com/fluxcd/flux2/pkg/status"
+	"github.com/fluxcd/pkg/git"
+	"github.com/fluxcd/pkg/git/repository"
 )
 
 type PlainGitBootstrapper struct {
-	url      string
-	branch   string
-	caBundle []byte
+	url    string
+	branch string
 
-	author                git.Author
+	signature             git.Signature
 	commitMessageAppendix string
 
-	gpgKeyRingPath string
-	gpgPassphrase  string
-	gpgKeyID       string
+	gpgKeyRing    openpgp.EntityList
+	gpgPassphrase string
+	gpgKeyID      string
 
 	restClientGetter  genericclioptions.RESTClientGetter
 	restClientOptions *runclient.Options
 
 	postGenerateSecret []PostGenerateSecretFunc
 
-	git    git.Git
-	kube   client.Client
-	logger log.Logger
+	gitClient repository.Client
+	kube      client.Client
+	logger    log.Logger
 }
 
 type GitOption interface {
@@ -94,10 +96,10 @@ func (o postGenerateSecret) applyGit(b *PlainGitBootstrapper) {
 	b.postGenerateSecret = append(b.postGenerateSecret, PostGenerateSecretFunc(o))
 }
 
-func NewPlainGitProvider(git git.Git, kube client.Client, opts ...GitOption) (*PlainGitBootstrapper, error) {
+func NewPlainGitProvider(git repository.Client, kube client.Client, opts ...GitOption) (*PlainGitBootstrapper, error) {
 	b := &PlainGitBootstrapper{
-		git:  git,
-		kube: kube,
+		gitClient: git,
+		kube:      kube,
 	}
 	for _, opt := range opts {
 		opt.applyGit(b)
@@ -107,7 +109,7 @@ func NewPlainGitProvider(git git.Git, kube client.Client, opts ...GitOption) (*P
 
 func (b *PlainGitBootstrapper) ReconcileComponents(ctx context.Context, manifestsBase string, options install.Options, _ sourcesecret.Options) error {
 	// Clone if not already
-	if _, err := b.git.Status(); err != nil {
+	if _, err := b.gitClient.Head(); err != nil {
 		if err != git.ErrNoGitRepository {
 			return err
 		}
@@ -115,7 +117,17 @@ func (b *PlainGitBootstrapper) ReconcileComponents(ctx context.Context, manifest
 		b.logger.Actionf("cloning branch %q from Git repository %q", b.branch, b.url)
 		var cloned bool
 		if err = retry(1, 2*time.Second, func() (err error) {
-			cloned, err = b.git.Clone(ctx, b.url, b.branch, b.caBundle)
+			_, err = b.gitClient.Clone(ctx, b.url, repository.CloneOptions{
+				CheckoutStrategy: repository.CheckoutStrategy{
+					Branch: b.branch,
+				},
+			})
+			if err != nil {
+				b.logger.Warningf(" clone failure: %s", err)
+			}
+			if err == nil {
+				cloned = true
+			}
 			return
 		}); err != nil {
 			return fmt.Errorf("failed to clone repository: %w", err)
@@ -133,28 +145,33 @@ func (b *PlainGitBootstrapper) ReconcileComponents(ctx context.Context, manifest
 	}
 	b.logger.Successf("generated component manifests")
 
-	// Write manifest to Git repository
-	if err = b.git.Write(manifests.Path, strings.NewReader(manifests.Content)); err != nil {
-		return fmt.Errorf("failed to write manifest %q: %w", manifests.Path, err)
+	// Write generated files and make a commit
+	var signer *openpgp.Entity
+	if b.gpgKeyRing != nil {
+		signer, err = getOpenPgpEntity(b.gpgKeyRing, b.gpgPassphrase, b.gpgKeyID)
+		if err != nil {
+			return fmt.Errorf("failed to generate OpenPGP entity: %w", err)
+		}
 	}
-
-	// Git commit generated
-	gpgOpts := git.WithGpgSigningOption(b.gpgKeyRingPath, b.gpgPassphrase, b.gpgKeyID)
 	commitMsg := fmt.Sprintf("Add Flux %s component manifests", options.Version)
 	if b.commitMessageAppendix != "" {
 		commitMsg = commitMsg + "\n\n" + b.commitMessageAppendix
 	}
-	commit, err := b.git.Commit(git.Commit{
-		Author:  b.author,
+
+	commit, err := b.gitClient.Commit(git.Commit{
+		Author:  b.signature,
 		Message: commitMsg,
-	}, gpgOpts)
+	}, repository.WithFiles(map[string]io.Reader{
+		manifests.Path: strings.NewReader(manifests.Content),
+	}), repository.WithSigner(signer))
 	if err != nil && err != git.ErrNoStagedFiles {
 		return fmt.Errorf("failed to commit sync manifests: %w", err)
 	}
+
 	if err == nil {
 		b.logger.Successf("committed sync manifests to %q (%q)", b.branch, commit)
 		b.logger.Actionf("pushing component manifests to %q", b.url)
-		if err = b.git.Push(ctx, b.caBundle); err != nil {
+		if err = b.gitClient.Push(ctx); err != nil {
 			return fmt.Errorf("failed to push manifests: %w", err)
 		}
 	} else {
@@ -165,16 +182,16 @@ func (b *PlainGitBootstrapper) ReconcileComponents(ctx context.Context, manifest
 	if mustInstallManifests(ctx, b.kube, options.Namespace) {
 		b.logger.Actionf("installing components in %q namespace", options.Namespace)
 
-		componentsYAML := filepath.Join(b.git.Path(), manifests.Path)
+		componentsYAML := filepath.Join(b.gitClient.Path(), manifests.Path)
 		kfile := filepath.Join(filepath.Dir(componentsYAML), konfig.DefaultKustomizationFileName())
 		if _, err := os.Stat(kfile); err == nil {
 			// Apply the components and their patches
-			if _, err := utils.Apply(ctx, b.restClientGetter, b.restClientOptions, b.git.Path(), kfile); err != nil {
+			if _, err := utils.Apply(ctx, b.restClientGetter, b.restClientOptions, b.gitClient.Path(), kfile); err != nil {
 				return err
 			}
 		} else {
 			// Apply the CRDs and controllers
-			if _, err := utils.Apply(ctx, b.restClientGetter, b.restClientOptions, b.git.Path(), componentsYAML); err != nil {
+			if _, err := utils.Apply(ctx, b.restClientGetter, b.restClientOptions, b.gitClient.Path(), componentsYAML); err != nil {
 				return err
 			}
 		}
@@ -195,7 +212,7 @@ func (b *PlainGitBootstrapper) ReconcileSourceSecret(ctx context.Context, option
 	}
 
 	// Return early if exists and no custom config is passed
-	if ok && len(options.CAFilePath+options.PrivateKeyPath+options.Username+options.Password) == 0 {
+	if ok && options.Keypair == nil && len(options.CAFile) == 0 && len(options.Username+options.Password) == 0 {
 		b.logger.Successf("source secret up to date")
 		return nil
 	}
@@ -236,12 +253,19 @@ func (b *PlainGitBootstrapper) ReconcileSyncConfig(ctx context.Context, options
 	}
 
 	// Clone if not already
-	if _, err := b.git.Status(); err != nil {
+	if _, err := b.gitClient.Head(); err != nil {
 		if err == git.ErrNoGitRepository {
 			b.logger.Actionf("cloning branch %q from Git repository %q", b.branch, b.url)
 			var cloned bool
 			if err = retry(1, 2*time.Second, func() (err error) {
-				cloned, err = b.git.Clone(ctx, b.url, b.branch, b.caBundle)
+				_, err = b.gitClient.Clone(ctx, b.url, repository.CloneOptions{
+					CheckoutStrategy: repository.CheckoutStrategy{
+						Branch: b.branch,
+					},
+				})
+				if err == nil {
+					cloned = true
+				}
 				return
 			}); err != nil {
 				return fmt.Errorf("failed to clone repository: %w", err)
@@ -259,41 +283,47 @@ func (b *PlainGitBootstrapper) ReconcileSyncConfig(ctx context.Context, options
 	if err != nil {
 		return fmt.Errorf("sync manifests generation failed: %w", err)
 	}
-	if err = b.git.Write(manifests.Path, strings.NewReader(manifests.Content)); err != nil {
-		return fmt.Errorf("failed to write manifest %q: %w", manifests.Path, err)
-	}
 
 	// Create secure Kustomize FS
-	fs, err := filesys.MakeFsOnDiskSecureBuild(b.git.Path())
+	fs, err := filesys.MakeFsOnDiskSecureBuild(b.gitClient.Path())
 	if err != nil {
 		return fmt.Errorf("failed to initialize Kustomize file system: %w", err)
 	}
 
+	if err = fs.WriteFile(filepath.Join(b.gitClient.Path(), manifests.Path), []byte(manifests.Content)); err != nil {
+		return err
+	}
+
 	// Generate Kustomization
 	kusManifests, err := kustomization.Generate(kustomization.Options{
 		FileSystem: fs,
-		BaseDir:    b.git.Path(),
+		BaseDir:    b.gitClient.Path(),
 		TargetPath: filepath.Dir(manifests.Path),
 	})
 	if err != nil {
 		return fmt.Errorf("%s generation failed: %w", konfig.DefaultKustomizationFileName(), err)
 	}
-	if err = b.git.Write(kusManifests.Path, strings.NewReader(kusManifests.Content)); err != nil {
-		return fmt.Errorf("failed to write manifest %q: %w", kusManifests.Path, err)
-	}
 	b.logger.Successf("generated sync manifests")
 
-	// Git commit generated
-	gpgOpts := git.WithGpgSigningOption(b.gpgKeyRingPath, b.gpgPassphrase, b.gpgKeyID)
+	// Write generated files and make a commit
+	var signer *openpgp.Entity
+	if b.gpgKeyRing != nil {
+		signer, err = getOpenPgpEntity(b.gpgKeyRing, b.gpgPassphrase, b.gpgKeyID)
+		if err != nil {
+			return fmt.Errorf("failed to generate OpenPGP entity: %w", err)
+		}
+	}
 	commitMsg := fmt.Sprintf("Add Flux sync manifests")
 	if b.commitMessageAppendix != "" {
 		commitMsg = commitMsg + "\n\n" + b.commitMessageAppendix
 	}
-	commit, err := b.git.Commit(git.Commit{
-		Author:  b.author,
-		Message: commitMsg,
-	}, gpgOpts)
 
+	commit, err := b.gitClient.Commit(git.Commit{
+		Author:  b.signature,
+		Message: commitMsg,
+	}, repository.WithFiles(map[string]io.Reader{
+		kusManifests.Path: strings.NewReader(kusManifests.Content),
+	}), repository.WithSigner(signer))
 	if err != nil && err != git.ErrNoStagedFiles {
 		return fmt.Errorf("failed to commit sync manifests: %w", err)
 	}
@@ -301,18 +331,22 @@ func (b *PlainGitBootstrapper) ReconcileSyncConfig(ctx context.Context, options
 	if err == nil {
 		b.logger.Successf("committed sync manifests to %q (%q)", b.branch, commit)
 		b.logger.Actionf("pushing sync manifests to %q", b.url)
-		err = b.git.Push(ctx, b.caBundle)
+		err = b.gitClient.Push(ctx)
 		if err != nil {
 			if strings.HasPrefix(err.Error(), gogit.ErrNonFastForwardUpdate.Error()) {
 				b.logger.Waitingf("git conflict detected, retrying with a fresh clone")
-				if err := os.RemoveAll(b.git.Path()); err != nil {
+				if err := os.RemoveAll(b.gitClient.Path()); err != nil {
 					return fmt.Errorf("failed to remove tmp dir: %w", err)
 				}
-				if err := os.Mkdir(b.git.Path(), 0o700); err != nil {
+				if err := os.Mkdir(b.gitClient.Path(), 0o700); err != nil {
 					return fmt.Errorf("failed to recreate tmp dir: %w", err)
 				}
 				if err = retry(1, 2*time.Second, func() (err error) {
-					_, err = b.git.Clone(ctx, b.url, b.branch, b.caBundle)
+					_, err = b.gitClient.Clone(ctx, b.url, repository.CloneOptions{
+						CheckoutStrategy: repository.CheckoutStrategy{
+							Branch: b.branch,
+						},
+					})
 					return
 				}); err != nil {
 					return fmt.Errorf("failed to clone repository: %w", err)
@@ -327,7 +361,7 @@ func (b *PlainGitBootstrapper) ReconcileSyncConfig(ctx context.Context, options
 
 	// Apply to cluster
 	b.logger.Actionf("applying sync manifests")
-	if _, err := utils.Apply(ctx, b.restClientGetter, b.restClientOptions, b.git.Path(), filepath.Join(b.git.Path(), kusManifests.Path)); err != nil {
+	if _, err := utils.Apply(ctx, b.restClientGetter, b.restClientOptions, b.gitClient.Path(), filepath.Join(b.gitClient.Path(), kusManifests.Path)); err != nil {
 		return err
 	}
 
@@ -337,7 +371,7 @@ func (b *PlainGitBootstrapper) ReconcileSyncConfig(ctx context.Context, options
 }
 
 func (b *PlainGitBootstrapper) ReportKustomizationHealth(ctx context.Context, options sync.Options, pollInterval, timeout time.Duration) error {
-	head, err := b.git.Head()
+	head, err := b.gitClient.Head()
 	if err != nil {
 		return err
 	}
@@ -389,3 +423,42 @@ func (b *PlainGitBootstrapper) ReportComponentsHealth(ctx context.Context, insta
 	b.logger.Successf("all components are healthy")
 	return nil
 }
+
+func getOpenPgpEntity(keyRing openpgp.EntityList, passphrase, keyID string) (*openpgp.Entity, error) {
+	if len(keyRing) == 0 {
+		return nil, fmt.Errorf("empty GPG key ring")
+	}
+
+	var entity *openpgp.Entity
+	if keyID != "" {
+		if strings.HasPrefix(keyID, "0x") {
+			keyID = strings.TrimPrefix(keyID, "0x")
+		}
+		if len(keyID) != 16 {
+			return nil, fmt.Errorf("invalid GPG key id length; expected %d, got %d", 16, len(keyID))
+		}
+		keyID = strings.ToUpper(keyID)
+
+		for _, ent := range keyRing {
+			if ent.PrimaryKey.KeyIdString() == keyID {
+				entity = ent
+			}
+		}
+
+		if entity == nil {
+			return nil, fmt.Errorf("no GPG keyring matching key id '%s' found", keyID)
+		}
+		if entity.PrivateKey == nil {
+			return nil, fmt.Errorf("keyring does not contain private key for key id '%s'", keyID)
+		}
+	} else {
+		entity = keyRing[0]
+	}
+
+	err := entity.PrivateKey.Decrypt([]byte(passphrase))
+	if err != nil {
+		return nil, fmt.Errorf("unable to decrypt GPG private key: %w", err)
+	}
+
+	return entity, nil
+}

pkg/bootstrap/bootstrap_provider.go

@@ -29,10 +29,10 @@ import (
 
 	"github.com/fluxcd/go-git-providers/gitprovider"
 
-	"github.com/fluxcd/flux2/internal/bootstrap/git"
-	"github.com/fluxcd/flux2/internal/bootstrap/provider"
+	"github.com/fluxcd/flux2/pkg/bootstrap/provider"
 	"github.com/fluxcd/flux2/pkg/manifestgen/sourcesecret"
 	"github.com/fluxcd/flux2/pkg/manifestgen/sync"
+	"github.com/fluxcd/pkg/git/repository"
 )
 
 type GitProviderBootstrapper struct {
@@ -62,11 +62,12 @@ type GitProviderBootstrapper struct {
 	provider gitprovider.Client
 }
 
-func NewGitProviderBootstrapper(git git.Git, provider gitprovider.Client, kube client.Client, opts ...GitProviderOption) (*GitProviderBootstrapper, error) {
+func NewGitProviderBootstrapper(git repository.Client, provider gitprovider.Client,
+	kube client.Client, opts ...GitProviderOption) (*GitProviderBootstrapper, error) {
 	b := &GitProviderBootstrapper{
 		PlainGitBootstrapper: &PlainGitBootstrapper{
-			git:  git,
-			kube: kube,
+			gitClient: git,
+			kube:      kube,
 		},
 		bootstrapTransportType: "https",
 		syncTransportType:      "ssh",
@@ -298,11 +299,8 @@ func (b *GitProviderBootstrapper) reconcileOrgRepository(ctx context.Context) (g
 
 	var changed bool
 	if b.reconcile {
-		// Set default branch before calling Reconcile due to bug described
-		// above.
-		repoInfo.DefaultBranch = repo.Get().DefaultBranch
 		if err = retry(1, 2*time.Second, func() (err error) {
-			repo, changed, err = b.provider.OrgRepositories().Reconcile(ctx, repoRef, repoInfo)
+			changed, err = repo.Reconcile(ctx)
 			return
 		}); err != nil {
 			return nil, fmt.Errorf("failed to reconcile Git repository %q: %w", repoRef.String(), err)
@@ -373,12 +371,9 @@ func (b *GitProviderBootstrapper) reconcileUserRepository(ctx context.Context) (
 	}
 
 	if b.reconcile {
-		// Set default branch before calling Reconcile due to bug described
-		// above.
-		repoInfo.DefaultBranch = repo.Get().DefaultBranch
 		var changed bool
 		if err = retry(1, 2*time.Second, func() (err error) {
-			repo, changed, err = b.provider.UserRepositories().Reconcile(ctx, repoRef, repoInfo)
+			changed, err = repo.Reconcile(ctx)
 			return
 		}); err != nil {
 			return nil, fmt.Errorf("failed to reconcile Git repository %q: %w", repoRef.String(), err)

pkg/bootstrap/options.go

@@ -17,11 +17,15 @@ limitations under the License.
 package bootstrap
 
 import (
+	"fmt"
+	"os"
+
 	"k8s.io/cli-runtime/pkg/genericclioptions"
 
+	"github.com/ProtonMail/go-crypto/openpgp"
+	"github.com/fluxcd/pkg/git"
 	runclient "github.com/fluxcd/pkg/runtime/client"
 
-	"github.com/fluxcd/flux2/internal/bootstrap/git"
 	"github.com/fluxcd/flux2/pkg/log"
 )
 
@@ -44,42 +48,28 @@ func (o branchOption) applyGitProvider(b *GitProviderBootstrapper) {
 	o.applyGit(b.PlainGitBootstrapper)
 }
 
-func WithAuthor(name, email string) Option {
-	return authorOption{
+func WithSignature(name, email string) Option {
+	return signatureOption{
 		Name:  name,
 		Email: email,
 	}
 }
 
-type authorOption git.Author
+type signatureOption git.Signature
 
-func (o authorOption) applyGit(b *PlainGitBootstrapper) {
+func (o signatureOption) applyGit(b *PlainGitBootstrapper) {
 	if o.Name != "" {
-		b.author.Name = o.Name
+		b.signature.Name = o.Name
 	}
 	if o.Email != "" {
-		b.author.Email = o.Email
+		b.signature.Email = o.Email
 	}
 }
 
-func (o authorOption) applyGitProvider(b *GitProviderBootstrapper) {
+func (o signatureOption) applyGitProvider(b *GitProviderBootstrapper) {
 	o.applyGit(b.PlainGitBootstrapper)
 }
 
-func WithCABundle(b []byte) Option {
-	return caBundleOption(b)
-}
-
-type caBundleOption []byte
-
-func (o caBundleOption) applyGit(b *PlainGitBootstrapper) {
-	b.caBundle = o
-}
-
-func (o caBundleOption) applyGitProvider(b *GitProviderBootstrapper) {
-	b.caBundle = o
-}
-
 func WithCommitMessageAppendix(appendix string) Option {
 	return commitMessageAppendixOption(appendix)
 }
@@ -131,22 +121,22 @@ func (o loggerOption) applyGitProvider(b *GitProviderBootstrapper) {
 	b.logger = o.logger
 }
 
-func WithGitCommitSigning(path, passphrase, keyID string) Option {
+func WithGitCommitSigning(gpgKeyRing openpgp.EntityList, passphrase, keyID string) Option {
 	return gitCommitSigningOption{
-		gpgKeyRingPath: path,
-		gpgPassphrase:  passphrase,
-		gpgKeyID:       keyID,
+		gpgKeyRing:    gpgKeyRing,
+		gpgPassphrase: passphrase,
+		gpgKeyID:      keyID,
 	}
 }
 
 type gitCommitSigningOption struct {
-	gpgKeyRingPath string
-	gpgPassphrase  string
-	gpgKeyID       string
+	gpgKeyRing    openpgp.EntityList
+	gpgPassphrase string
+	gpgKeyID      string
 }
 
 func (o gitCommitSigningOption) applyGit(b *PlainGitBootstrapper) {
-	b.gpgKeyRingPath = o.gpgKeyRingPath
+	b.gpgKeyRing = o.gpgKeyRing
 	b.gpgPassphrase = o.gpgPassphrase
 	b.gpgKeyID = o.gpgKeyID
 }
@@ -154,3 +144,18 @@ func (o gitCommitSigningOption) applyGit(b *PlainGitBootstrapper) {
 func (o gitCommitSigningOption) applyGitProvider(b *GitProviderBootstrapper) {
 	o.applyGit(b.PlainGitBootstrapper)
 }
+
+func LoadEntityListFromPath(path string) (openpgp.EntityList, error) {
+	if path == "" {
+		return nil, nil
+	}
+	r, err := os.Open(path)
+	if err != nil {
+		return nil, fmt.Errorf("unable to open GPG key ring: %w", err)
+	}
+	entityList, err := openpgp.ReadKeyRing(r)
+	if err != nil {
+		return nil, fmt.Errorf("unable to read GPG key ring: %w", err)
+	}
+	return entityList, nil
+}

pkg/log/nop.go

@@ -1,5 +1,5 @@
 /*
-Copyright 2021 The Flux authors
+Copyright 2022 The Flux authors
 
 Licensed under the Apache License, Version 2.0 (the "License");
 you may not use this file except in compliance with the License.
@@ -14,22 +14,18 @@ See the License for the specific language governing permissions and
 limitations under the License.
 */
 
-package test
+package log
 
-import (
-	"context"
-	"os/exec"
-	"strings"
-)
+type NopLogger struct{}
 
-type whichTerraform struct{}
+func (NopLogger) Actionf(format string, a ...interface{}) {}
 
-func (w *whichTerraform) ExecPath(ctx context.Context) (string, error) {
-	cmd := exec.CommandContext(ctx, "which", "terraform")
-	output, err := cmd.Output()
-	if err != nil {
-		return "", err
-	}
-	path := strings.TrimSuffix(string(output), "\n")
-	return path, nil
-}
+func (NopLogger) Generatef(format string, a ...interface{}) {}
+
+func (NopLogger) Waitingf(format string, a ...interface{}) {}
+
+func (NopLogger) Successf(format string, a ...interface{}) {}
+
+func (NopLogger) Warningf(format string, a ...interface{}) {}
+
+func (NopLogger) Failuref(format string, a ...interface{}) {}

pkg/manifestgen/sourcesecret/options.go

@@ -18,6 +18,8 @@ package sourcesecret
 
 import (
 	"crypto/elliptic"
+
+	"github.com/fluxcd/pkg/ssh"
 )
 
 type PrivateKeyAlgorithm string
@@ -48,12 +50,12 @@ type Options struct {
 	PrivateKeyAlgorithm PrivateKeyAlgorithm
 	RSAKeyBits          int
 	ECDSACurve          elliptic.Curve
-	PrivateKeyPath      string
+	Keypair             *ssh.KeyPair
 	Username            string
 	Password            string
-	CAFilePath          string
-	CertFilePath        string
-	KeyFilePath         string
+	CAFile              []byte
+	CertFile            []byte
+	KeyFile             []byte
 	TargetPath          string
 	ManifestFile        string
 }
@@ -64,12 +66,11 @@ func MakeDefaultOptions() Options {
 		Namespace:           "flux-system",
 		Labels:              map[string]string{},
 		PrivateKeyAlgorithm: RSAPrivateKeyAlgorithm,
-		PrivateKeyPath:      "",
 		Username:            "",
 		Password:            "",
-		CAFilePath:          "",
-		CertFilePath:        "",
-		KeyFilePath:         "",
+		CAFile:              []byte{},
+		CertFile:            []byte{},
+		KeyFile:             []byte{},
 		ManifestFile:        "secret.yaml",
 	}
 }

pkg/manifestgen/sourcesecret/sourcesecret.go

@@ -66,10 +66,8 @@ func Generate(options Options) (*manifestgen.Manifest, error) {
 	switch {
 	case options.Username != "" && options.Password != "":
 		// noop
-	case len(options.PrivateKeyPath) > 0:
-		if keypair, err = loadKeyPair(options.PrivateKeyPath, options.Password); err != nil {
-			return nil, err
-		}
+	case options.Keypair != nil:
+		keypair = options.Keypair
 	case len(options.PrivateKeyAlgorithm) > 0:
 		if keypair, err = generateKeyPair(options); err != nil {
 			return nil, err
@@ -78,28 +76,11 @@ func Generate(options Options) (*manifestgen.Manifest, error) {
 
 	var hostKey []byte
 	if keypair != nil {
-		if hostKey, err = scanHostKey(options.SSHHostname); err != nil {
+		if hostKey, err = ScanHostKey(options.SSHHostname); err != nil {
 			return nil, err
 		}
 	}
 
-	var caFile []byte
-	if options.CAFilePath != "" {
-		if caFile, err = os.ReadFile(options.CAFilePath); err != nil {
-			return nil, fmt.Errorf("failed to read CA file: %w", err)
-		}
-	}
-
-	var certFile, keyFile []byte
-	if options.CertFilePath != "" && options.KeyFilePath != "" {
-		if certFile, err = os.ReadFile(options.CertFilePath); err != nil {
-			return nil, fmt.Errorf("failed to read cert file: %w", err)
-		}
-		if keyFile, err = os.ReadFile(options.KeyFilePath); err != nil {
-			return nil, fmt.Errorf("failed to read key file: %w", err)
-		}
-	}
-
 	var dockerCfgJson []byte
 	if options.Registry != "" {
 		dockerCfgJson, err = generateDockerConfigJson(options.Registry, options.Username, options.Password)
@@ -108,7 +89,7 @@ func Generate(options Options) (*manifestgen.Manifest, error) {
 		}
 	}
 
-	secret := buildSecret(keypair, hostKey, caFile, certFile, keyFile, dockerCfgJson, options)
+	secret := buildSecret(keypair, hostKey, options.CAFile, options.CertFile, options.KeyFile, dockerCfgJson, options)
 	b, err := yaml.Marshal(secret)
 	if err != nil {
 		return nil, err
@@ -120,6 +101,35 @@ func Generate(options Options) (*manifestgen.Manifest, error) {
 	}, nil
 }
 
+func LoadKeyPairFromPath(path, password string) (*ssh.KeyPair, error) {
+	if path == "" {
+		return nil, nil
+	}
+
+	b, err := os.ReadFile(path)
+	if err != nil {
+		return nil, fmt.Errorf("failed to open private key file: %w", err)
+	}
+	return LoadKeyPair(b, password)
+}
+
+func LoadKeyPair(privateKey []byte, password string) (*ssh.KeyPair, error) {
+	var ppk cryptssh.Signer
+	var err error
+	if password != "" {
+		ppk, err = cryptssh.ParsePrivateKeyWithPassphrase(privateKey, []byte(password))
+	} else {
+		ppk, err = cryptssh.ParsePrivateKey(privateKey)
+	}
+	if err != nil {
+		return nil, err
+	}
+	return &ssh.KeyPair{
+		PublicKey:  cryptssh.MarshalAuthorizedKey(ppk.PublicKey()),
+		PrivateKey: privateKey,
+	}, nil
+}
+
 func buildSecret(keypair *ssh.KeyPair, hostKey, caFile, certFile, keyFile, dockerCfg []byte, options Options) (secret corev1.Secret) {
 	secret.TypeMeta = metav1.TypeMeta{
 		APIVersion: "v1",
@@ -143,16 +153,16 @@ func buildSecret(keypair *ssh.KeyPair, hostKey, caFile, certFile, keyFile, docke
 		secret.StringData[PasswordSecretKey] = options.Password
 	}
 
-	if caFile != nil {
+	if len(caFile) != 0 {
 		secret.StringData[CAFileSecretKey] = string(caFile)
 	}
 
-	if certFile != nil && keyFile != nil {
+	if len(certFile) != 0 && len(keyFile) != 0 {
 		secret.StringData[CertFileSecretKey] = string(certFile)
 		secret.StringData[KeyFileSecretKey] = string(keyFile)
 	}
 
-	if keypair != nil && hostKey != nil {
+	if keypair != nil && len(hostKey) != 0 {
 		secret.StringData[PrivateKeySecretKey] = string(keypair.PrivateKey)
 		secret.StringData[PublicKeySecretKey] = string(keypair.PublicKey)
 		secret.StringData[KnownHostsSecretKey] = string(hostKey)
@@ -165,29 +175,6 @@ func buildSecret(keypair *ssh.KeyPair, hostKey, caFile, certFile, keyFile, docke
 	return
 }
 
-func loadKeyPair(path string, password string) (*ssh.KeyPair, error) {
-	b, err := os.ReadFile(path)
-	if err != nil {
-		return nil, fmt.Errorf("failed to open private key file: %w", err)
-	}
-
-	var ppk cryptssh.Signer
-	if password != "" {
-		ppk, err = cryptssh.ParsePrivateKeyWithPassphrase(b, []byte(password))
-	} else {
-		ppk, err = cryptssh.ParsePrivateKey(b)
-	}
-
-	if err != nil {
-		return nil, err
-	}
-
-	return &ssh.KeyPair{
-		PublicKey:  cryptssh.MarshalAuthorizedKey(ppk.PublicKey()),
-		PrivateKey: b,
-	}, nil
-}
-
 func generateKeyPair(options Options) (*ssh.KeyPair, error) {
 	var keyGen ssh.KeyPairGenerator
 	switch options.PrivateKeyAlgorithm {
@@ -207,7 +194,7 @@ func generateKeyPair(options Options) (*ssh.KeyPair, error) {
 	return pair, nil
 }
 
-func scanHostKey(host string) ([]byte, error) {
+func ScanHostKey(host string) ([]byte, error) {
 	if _, _, err := net.SplitHostPort(host); err != nil {
 		// Assume we are dealing with a hostname without a port,
 		// append the default SSH port as this is required for

pkg/manifestgen/sourcesecret/sourcesecret_test.go

@@ -48,7 +48,7 @@ func Test_passwordLoadKeyPair(t *testing.T) {
 			pk, _ := os.ReadFile(tt.privateKeyPath)
 			ppk, _ := os.ReadFile(tt.publicKeyPath)
 
-			got, err := loadKeyPair(tt.privateKeyPath, tt.password)
+			got, err := LoadKeyPair(pk, tt.password)
 			if err != nil {
 				t.Errorf("loadKeyPair() error = %v", err)
 				return
@@ -67,24 +67,13 @@ func Test_passwordLoadKeyPair(t *testing.T) {
 func Test_PasswordlessLoadKeyPair(t *testing.T) {
 	for algo, privateKey := range testdata.PEMBytes {
 		t.Run(algo, func(t *testing.T) {
-			f, err := os.CreateTemp("", "test-private-key-")
-			if err != nil {
-				t.Fatalf("unable to create temporary file. err: %s", err)
-			}
-			defer os.Remove(f.Name())
-
-			if _, err = f.Write(privateKey); err != nil {
-				t.Fatalf("unable to write private key to file. err: %s", err)
-			}
-
-			got, err := loadKeyPair(f.Name(), "")
+			got, err := LoadKeyPair(privateKey, "")
 			if err != nil {
 				t.Errorf("loadKeyPair() error = %v", err)
 				return
 			}
 
-			pk, _ := os.ReadFile(f.Name())
-			if !reflect.DeepEqual(got.PrivateKey, pk) {
+			if !reflect.DeepEqual(got.PrivateKey, privateKey) {
 				t.Errorf("PrivateKey %s != %s", got.PrivateKey, string(privateKey))
 			}
 

pkg/uninstall/uninstall.go

@@ -0,0 +1,372 @@
+/*
+Copyright 2022 The Flux authors
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+
+package uninstall
+
+import (
+	"context"
+
+	appsv1 "k8s.io/api/apps/v1"
+	corev1 "k8s.io/api/core/v1"
+	networkingv1 "k8s.io/api/networking/v1"
+	rbacv1 "k8s.io/api/rbac/v1"
+	apiextensionsv1 "k8s.io/apiextensions-apiserver/pkg/apis/apiextensions/v1"
+	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
+	"k8s.io/apimachinery/pkg/util/errors"
+	"sigs.k8s.io/controller-runtime/pkg/client"
+
+	"github.com/fluxcd/flux2/pkg/log"
+	"github.com/fluxcd/flux2/pkg/manifestgen"
+	helmv2 "github.com/fluxcd/helm-controller/api/v2beta1"
+	autov1 "github.com/fluxcd/image-automation-controller/api/v1beta1"
+	imagev1 "github.com/fluxcd/image-reflector-controller/api/v1beta1"
+	kustomizev1 "github.com/fluxcd/kustomize-controller/api/v1beta2"
+	notificationv1 "github.com/fluxcd/notification-controller/api/v1beta1"
+	sourcev1 "github.com/fluxcd/source-controller/api/v1beta2"
+)
+
+// Components removes all Kubernetes components that are part of Flux excluding the CRDs and namespace.
+func Components(ctx context.Context, logger log.Logger, kubeClient client.Client, namespace string, dryRun bool) error {
+	var aggregateErr []error
+	opts, dryRunStr := getDeleteOptions(dryRun)
+	selector := client.MatchingLabels{manifestgen.PartOfLabelKey: manifestgen.PartOfLabelValue}
+	{
+		var list appsv1.DeploymentList
+		if err := kubeClient.List(ctx, &list, client.InNamespace(namespace), selector); err == nil {
+			for _, r := range list.Items {
+				if err := kubeClient.Delete(ctx, &r, opts); err != nil {
+					logger.Failuref("Deployment/%s/%s deletion failed: %s", r.Namespace, r.Name, err.Error())
+					aggregateErr = append(aggregateErr, err)
+				} else {
+					logger.Successf("Deployment/%s/%s deleted %s", r.Namespace, r.Name, dryRunStr)
+				}
+			}
+		}
+	}
+	{
+		var list corev1.ServiceList
+		if err := kubeClient.List(ctx, &list, client.InNamespace(namespace), selector); err == nil {
+			for _, r := range list.Items {
+				if err := kubeClient.Delete(ctx, &r, opts); err != nil {
+					logger.Failuref("Service/%s/%s deletion failed: %s", r.Namespace, r.Name, err.Error())
+					aggregateErr = append(aggregateErr, err)
+				} else {
+					logger.Successf("Service/%s/%s deleted %s", r.Namespace, r.Name, dryRunStr)
+				}
+			}
+		}
+	}
+	{
+		var list networkingv1.NetworkPolicyList
+		if err := kubeClient.List(ctx, &list, client.InNamespace(namespace), selector); err == nil {
+			for _, r := range list.Items {
+				if err := kubeClient.Delete(ctx, &r, opts); err != nil {
+					logger.Failuref("NetworkPolicy/%s/%s deletion failed: %s", r.Namespace, r.Name, err.Error())
+					aggregateErr = append(aggregateErr, err)
+				} else {
+					logger.Successf("NetworkPolicy/%s/%s deleted %s", r.Namespace, r.Name, dryRunStr)
+				}
+			}
+		}
+	}
+	{
+		var list corev1.ServiceAccountList
+		if err := kubeClient.List(ctx, &list, client.InNamespace(namespace), selector); err == nil {
+			for _, r := range list.Items {
+				if err := kubeClient.Delete(ctx, &r, opts); err != nil {
+					logger.Failuref("ServiceAccount/%s/%s deletion failed: %s", r.Namespace, r.Name, err.Error())
+					aggregateErr = append(aggregateErr, err)
+				} else {
+					logger.Successf("ServiceAccount/%s/%s deleted %s", r.Namespace, r.Name, dryRunStr)
+				}
+			}
+		}
+	}
+	{
+		var list rbacv1.ClusterRoleList
+		if err := kubeClient.List(ctx, &list, selector); err == nil {
+			for _, r := range list.Items {
+				if err := kubeClient.Delete(ctx, &r, opts); err != nil {
+					logger.Failuref("ClusterRole/%s deletion failed: %s", r.Name, err.Error())
+					aggregateErr = append(aggregateErr, err)
+				} else {
+					logger.Successf("ClusterRole/%s deleted %s", r.Name, dryRunStr)
+				}
+			}
+		}
+	}
+	{
+		var list rbacv1.ClusterRoleBindingList
+		if err := kubeClient.List(ctx, &list, selector); err == nil {
+			for _, r := range list.Items {
+				if err := kubeClient.Delete(ctx, &r, opts); err != nil {
+					logger.Failuref("ClusterRoleBinding/%s deletion failed: %s", r.Name, err.Error())
+					aggregateErr = append(aggregateErr, err)
+				} else {
+					logger.Successf("ClusterRoleBinding/%s deleted %s", r.Name, dryRunStr)
+				}
+			}
+		}
+	}
+
+	return errors.Reduce(errors.Flatten(errors.NewAggregate(aggregateErr)))
+}
+
+// Finalizers removes all finalizes on Kubernetes components that have been added by a Flux controller.
+func Finalizers(ctx context.Context, logger log.Logger, kubeClient client.Client, dryRun bool) error {
+	var aggregateErr []error
+	opts, dryRunStr := getUpdateOptions(dryRun)
+	{
+		var list sourcev1.GitRepositoryList
+		if err := kubeClient.List(ctx, &list, client.InNamespace("")); err == nil {
+			for _, r := range list.Items {
+				r.Finalizers = []string{}
+				if err := kubeClient.Update(ctx, &r, opts); err != nil {
+					logger.Failuref("%s/%s/%s removing finalizers failed: %s", r.Kind, r.Namespace, r.Name, err.Error())
+					aggregateErr = append(aggregateErr, err)
+				} else {
+					logger.Successf("%s/%s/%s finalizers deleted %s", r.Kind, r.Namespace, r.Name, dryRunStr)
+				}
+			}
+		}
+	}
+	{
+		var list sourcev1.OCIRepositoryList
+		if err := kubeClient.List(ctx, &list, client.InNamespace("")); err == nil {
+			for _, r := range list.Items {
+				r.Finalizers = []string{}
+				if err := kubeClient.Update(ctx, &r, opts); err != nil {
+					logger.Failuref("%s/%s/%s removing finalizers failed: %s", r.Kind, r.Namespace, r.Name, err.Error())
+					aggregateErr = append(aggregateErr, err)
+				} else {
+					logger.Successf("%s/%s/%s finalizers deleted %s", r.Kind, r.Namespace, r.Name, dryRunStr)
+				}
+			}
+		}
+	}
+	{
+		var list sourcev1.HelmRepositoryList
+		if err := kubeClient.List(ctx, &list, client.InNamespace("")); err == nil {
+			for _, r := range list.Items {
+				r.Finalizers = []string{}
+				if err := kubeClient.Update(ctx, &r, opts); err != nil {
+					logger.Failuref("%s/%s/%s removing finalizers failed: %s", r.Kind, r.Namespace, r.Name, err.Error())
+					aggregateErr = append(aggregateErr, err)
+				} else {
+					logger.Successf("%s/%s/%s finalizers deleted %s", r.Kind, r.Namespace, r.Name, dryRunStr)
+				}
+			}
+		}
+	}
+	{
+		var list sourcev1.HelmChartList
+		if err := kubeClient.List(ctx, &list, client.InNamespace("")); err == nil {
+			for _, r := range list.Items {
+				r.Finalizers = []string{}
+				if err := kubeClient.Update(ctx, &r, opts); err != nil {
+					logger.Failuref("%s/%s/%s removing finalizers failed: %s", r.Kind, r.Namespace, r.Name, err.Error())
+					aggregateErr = append(aggregateErr, err)
+				} else {
+					logger.Successf("%s/%s/%s finalizers deleted %s", r.Kind, r.Namespace, r.Name, dryRunStr)
+				}
+			}
+		}
+	}
+	{
+		var list sourcev1.BucketList
+		if err := kubeClient.List(ctx, &list, client.InNamespace("")); err == nil {
+			for _, r := range list.Items {
+				r.Finalizers = []string{}
+				if err := kubeClient.Update(ctx, &r, opts); err != nil {
+					logger.Failuref("%s/%s/%s removing finalizers failed: %s", r.Kind, r.Namespace, r.Name, err.Error())
+					aggregateErr = append(aggregateErr, err)
+				} else {
+					logger.Successf("%s/%s/%s finalizers deleted %s", r.Kind, r.Namespace, r.Name, dryRunStr)
+				}
+			}
+		}
+	}
+	{
+		var list kustomizev1.KustomizationList
+		if err := kubeClient.List(ctx, &list, client.InNamespace("")); err == nil {
+			for _, r := range list.Items {
+				r.Finalizers = []string{}
+				if err := kubeClient.Update(ctx, &r, opts); err != nil {
+					logger.Failuref("%s/%s/%s removing finalizers failed: %s", r.Kind, r.Namespace, r.Name, err.Error())
+					aggregateErr = append(aggregateErr, err)
+				} else {
+					logger.Successf("%s/%s/%s finalizers deleted %s", r.Kind, r.Namespace, r.Name, dryRunStr)
+				}
+			}
+		}
+	}
+	{
+		var list helmv2.HelmReleaseList
+		if err := kubeClient.List(ctx, &list, client.InNamespace("")); err == nil {
+			for _, r := range list.Items {
+				r.Finalizers = []string{}
+				if err := kubeClient.Update(ctx, &r, opts); err != nil {
+					logger.Failuref("%s/%s/%s removing finalizers failed: %s", r.Kind, r.Namespace, r.Name, err.Error())
+					aggregateErr = append(aggregateErr, err)
+				} else {
+					logger.Successf("%s/%s/%s finalizers deleted %s", r.Kind, r.Namespace, r.Name, dryRunStr)
+				}
+			}
+		}
+	}
+	{
+		var list notificationv1.AlertList
+		if err := kubeClient.List(ctx, &list, client.InNamespace("")); err == nil {
+			for _, r := range list.Items {
+				r.Finalizers = []string{}
+				if err := kubeClient.Update(ctx, &r, opts); err != nil {
+					logger.Failuref("%s/%s/%s removing finalizers failed: %s", r.Kind, r.Namespace, r.Name, err.Error())
+					aggregateErr = append(aggregateErr, err)
+				} else {
+					logger.Successf("%s/%s/%s finalizers deleted %s", r.Kind, r.Namespace, r.Name, dryRunStr)
+				}
+			}
+		}
+	}
+	{
+		var list notificationv1.ProviderList
+		if err := kubeClient.List(ctx, &list, client.InNamespace("")); err == nil {
+			for _, r := range list.Items {
+				r.Finalizers = []string{}
+				if err := kubeClient.Update(ctx, &r, opts); err != nil {
+					logger.Failuref("%s/%s/%s removing finalizers failed: %s", r.Kind, r.Namespace, r.Name, err.Error())
+					aggregateErr = append(aggregateErr, err)
+				} else {
+					logger.Successf("%s/%s/%s finalizers deleted %s", r.Kind, r.Namespace, r.Name, dryRunStr)
+				}
+			}
+		}
+	}
+	{
+		var list notificationv1.ReceiverList
+		if err := kubeClient.List(ctx, &list, client.InNamespace("")); err == nil {
+			for _, r := range list.Items {
+				r.Finalizers = []string{}
+				if err := kubeClient.Update(ctx, &r, opts); err != nil {
+					logger.Failuref("%s/%s/%s removing finalizers failed: %s", r.Kind, r.Namespace, r.Name, err.Error())
+					aggregateErr = append(aggregateErr, err)
+				} else {
+					logger.Successf("%s/%s/%s finalizers deleted %s", r.Kind, r.Namespace, r.Name, dryRunStr)
+				}
+			}
+		}
+	}
+	{
+		var list imagev1.ImagePolicyList
+		if err := kubeClient.List(ctx, &list, client.InNamespace("")); err == nil {
+			for _, r := range list.Items {
+				r.Finalizers = []string{}
+				if err := kubeClient.Update(ctx, &r, opts); err != nil {
+					logger.Failuref("%s/%s/%s removing finalizers failed: %s", r.Kind, r.Namespace, r.Name, err.Error())
+					aggregateErr = append(aggregateErr, err)
+				} else {
+					logger.Successf("%s/%s/%s finalizers deleted %s", r.Kind, r.Namespace, r.Name, dryRunStr)
+				}
+			}
+		}
+	}
+	{
+		var list imagev1.ImageRepositoryList
+		if err := kubeClient.List(ctx, &list, client.InNamespace("")); err == nil {
+			for _, r := range list.Items {
+				r.Finalizers = []string{}
+				if err := kubeClient.Update(ctx, &r, opts); err != nil {
+					logger.Failuref("%s/%s/%s removing finalizers failed: %s", r.Kind, r.Namespace, r.Name, err.Error())
+					aggregateErr = append(aggregateErr, err)
+				} else {
+					logger.Successf("%s/%s/%s finalizers deleted %s", r.Kind, r.Namespace, r.Name, dryRunStr)
+				}
+			}
+		}
+	}
+	{
+		var list autov1.ImageUpdateAutomationList
+		if err := kubeClient.List(ctx, &list, client.InNamespace("")); err == nil {
+			for _, r := range list.Items {
+				r.Finalizers = []string{}
+				if err := kubeClient.Update(ctx, &r, opts); err != nil {
+					logger.Failuref("%s/%s/%s removing finalizers failed: %s", r.Kind, r.Namespace, r.Name, err.Error())
+					aggregateErr = append(aggregateErr, err)
+				} else {
+					logger.Successf("%s/%s/%s finalizers deleted %s", r.Kind, r.Namespace, r.Name, dryRunStr)
+				}
+			}
+		}
+	}
+	return errors.Reduce(errors.Flatten(errors.NewAggregate(aggregateErr)))
+}
+
+// CustomResourceDefinitions removes all Kubernetes CRDs that are a part of Flux.
+func CustomResourceDefinitions(ctx context.Context, logger log.Logger, kubeClient client.Client, dryRun bool) error {
+	var aggregateErr []error
+	opts, dryRunStr := getDeleteOptions(dryRun)
+	selector := client.MatchingLabels{manifestgen.PartOfLabelKey: manifestgen.PartOfLabelValue}
+	{
+		var list apiextensionsv1.CustomResourceDefinitionList
+		if err := kubeClient.List(ctx, &list, selector); err == nil {
+			for _, r := range list.Items {
+				if err := kubeClient.Delete(ctx, &r, opts); err != nil {
+					logger.Failuref("CustomResourceDefinition/%s deletion failed: %s", r.Name, err.Error())
+					aggregateErr = append(aggregateErr, err)
+				} else {
+					logger.Successf("CustomResourceDefinition/%s deleted %s", r.Name, dryRunStr)
+				}
+			}
+		}
+	}
+	return errors.Reduce(errors.Flatten(errors.NewAggregate(aggregateErr)))
+}
+
+// Namespace removes the namespace Flux is installed in.
+func Namespace(ctx context.Context, logger log.Logger, kubeClient client.Client, namespace string, dryRun bool) error {
+	var aggregateErr []error
+	opts, dryRunStr := getDeleteOptions(dryRun)
+	ns := corev1.Namespace{ObjectMeta: metav1.ObjectMeta{Name: namespace}}
+	if err := kubeClient.Delete(ctx, &ns, opts); err != nil {
+		logger.Failuref("Namespace/%s deletion failed: %s", namespace, err.Error())
+		aggregateErr = append(aggregateErr, err)
+	} else {
+		logger.Successf("Namespace/%s deleted %s", namespace, dryRunStr)
+	}
+	return errors.Reduce(errors.Flatten(errors.NewAggregate(aggregateErr)))
+}
+
+func getDeleteOptions(dryRun bool) (*client.DeleteOptions, string) {
+	opts := &client.DeleteOptions{}
+	var dryRunStr string
+	if dryRun {
+		client.DryRunAll.ApplyToDelete(opts)
+		dryRunStr = "(dry run)"
+	}
+
+	return opts, dryRunStr
+}
+
+func getUpdateOptions(dryRun bool) (*client.UpdateOptions, string) {
+	opts := &client.UpdateOptions{}
+	var dryRunStr string
+	if dryRun {
+		client.DryRunAll.ApplyToUpdate(opts)
+		dryRunStr = "(dry run)"
+	}
+
+	return opts, dryRunStr
+}

rfcs/0002-helm-oci/README.md

@@ -4,7 +4,7 @@
 
 **Creation date:** 2022-03-30
 
-**Last update:** 2022-06-07
+**Last update:** 2022-10-20
 
 ## Summary
 
@@ -22,6 +22,7 @@ they do today for container images.
 ### Goals
 
 - Add support for fetching Helm charts stored as OCI artifacts with minimal API changes to Flux.
+- Add support for verifying the authenticity of Helm OCI charts signed with Cosign.
 - Make it easy for users to switch from [HTTP/S Helm repositories](https://github.com/helm/helm-www/blob/416fabea6ffab8dc156b6a0c5eb5e8df5f5ef7dc/content/en/docs/topics/chart_repository.md)
   to OCI repositories.
 
@@ -32,12 +33,109 @@ they do today for container images.
 ## Proposal
 
 Introduce an optional field called `type` to the `HelmRepository` spec.
-
 When not specified, the `spec.type` field defaults to `default` which preserve the current `HelmRepository` API behaviour.
-
 When the `spec.type` field is set to `oci`, the `spec.url` field must be prefixed with `oci://` (to follow the Helm conventions).
 For `oci://` URLs, source-controller will use the Helm SDK and the `oras` library to connect to the OCI remote storage.
-For authentication, the controller will use Kubernetes secrets of `kubernetes.io/dockerconfigjson` type.
+
+Introduce an optional field called `provider` for
+[context-based authorization](https://fluxcd.io/flux/security/contextual-authorization/)
+to AWS, Azure and Google Cloud. The `spec.provider` is ignored when `spec.type` is set to `default`.
+
+### Pull charts from private repositories
+
+#### Basic auth
+
+For private repositories hosted on GitHub, Quay, self-hosted Docker Registry and others,
+the credentials can be supplied with:
+
+```yaml
+apiVersion: source.toolkit.fluxcd.io/v1beta2
+kind: HelmRepository
+metadata:
+  name: <repo-name>
+spec:
+  type: oci
+  secretRef:
+    name: regcred
+```
+
+The `secretRef` points to a Kubernetes secret in the same namespace as the `HelmRepository`.
+The [secret type](https://kubernetes.io/docs/concepts/configuration/secret/#secret-types)
+must be `kubernetes.io/dockerconfigjson`:
+
+```shell
+kubectl create secret docker-registry regcred \
+  --docker-server=<your-registry-server> \
+  --docker-username=<your-name> \
+  --docker-password=<your-pword>
+```
+
+#### OIDC auth
+
+When Flux runs on AKS, EKS or GKE, an IAM role (that grants read-only access to ACR, ECR or GCR)
+can be used to bind the `source-controller` to the IAM role.
+
+```yaml
+apiVersion: source.toolkit.fluxcd.io/v1beta2
+kind: HelmRepository
+metadata:
+  name: <repo-name>
+spec:
+  type: oci
+  provider: azure
+```
+
+The provider accepts the following values: `generic`, `aws`, `azure` and `gcp`. When the provider is
+not specified, it defaults to `generic`. When the provider is set to `aws`, `azure` or `gcp`, the
+controller will use a specific cloud SDK for authentication purposes.
+
+If both `spec.secretRef` and a non-generic provider are present in the definition,
+the controller will use the static credentials from the referenced secret.
+
+### Verify Helm charts
+
+To verify the authenticity of the Helm OCI charts, Flux will use the Sigstore Go SDK and implement verification
+for artifacts which were either signed with keys generated by Cosign or signed using the Cosign
+[keyless method](https://github.com/sigstore/cosign/blob/main/KEYLESS.md).
+
+To enable signature verification, the Cosign public keys can be supplied with:
+
+```yaml
+apiVersion: source.toolkit.fluxcd.io/v1beta2
+kind: HelmChart
+metadata:
+  name: <chart-name>
+spec:
+  verify:
+    provider: cosign
+    secretRef:
+      name: cosign-public-keys
+```
+
+Note that the Kubernetes secret containing the Cosign public keys, must use `.pub` extension:
+
+```yaml
+apiVersion: v1
+kind: Secret
+metadata:
+  name: cosign-public-keys
+type: Opaque
+stringData:
+  key1.pub: <pub-key-1>
+  key2.pub: <pub-key-2>
+```
+
+For verifying public Helm charts which are signed using the keyless method,
+the `spec.verify.secretRef` field must be omitted:
+
+```yaml
+spec:
+  verify:
+    provider: cosign
+```
+
+When using the keyless method, Flux will verify the signatures in the Rekor
+transparency log instance hosted at [rekor.sigstore.dev](https://rekor.sigstore.dev/).
 
 ### User Stories
 
@@ -176,8 +274,10 @@ The feature is enabled by default.
 
 * **2022-05-19** Partially implemented by [source-controller#690](https://github.com/fluxcd/source-controller/pull/690)
 * **2022-06-06** First implementation released with [flux2 v0.31.0](https://github.com/fluxcd/flux2/releases/tag/v0.31.0)
+* **2022-08-11** Resolve chart dependencies from OCI released with [flux2 v0.32.0](https://github.com/fluxcd/flux2/releases/tag/v0.32.0)
+* **2022-08-29** Contextual login for AWS, Azure and GCP released with [flux2 v0.33.0](https://github.com/fluxcd/flux2/releases/tag/v0.33.0)
+* **2022-10-21** Verifying Helm charts with Cosign released with [flux2 v0.36.0](https://github.com/fluxcd/flux2/releases/tag/v0.36.0)
 
 ### TODOs
 
-* [Resolve chart dependencies from OCI](https://github.com/fluxcd/source-controller/issues/722)
 * [Add support for container registries with self-signed TLS certs](https://github.com/fluxcd/source-controller/issues/723)

rfcs/0003-kubernetes-oci/README.md

@@ -1,10 +1,10 @@
 # RFC-0003 Flux OCI support for Kubernetes manifests
 
-**Status:** implementable
+**Status:** implemented
 
 **Creation date:** 2022-03-31
 
-**Last update:** 2022-08-02
+**Last update:** 2022-09-29
 
 ## Summary
 
@@ -124,16 +124,27 @@ spec:
     semver: "6.0.x"
 ```
 
-To verify the authenticity of an artifact, the Sigstore cosign public key can be supplied with:
+### Layer selection
+
+By default, Flux assumes that the first layer of the OCI artifact contains the Kubernetes configuration.
+For multi-layer artifacts created by other tools than Flux CLI
+(e.g. [oras](https://github.com/oras-project/oras),
+[crane](https://github.com/google/go-containerregistry/tree/main/cmd/crane)),
+users can specify the [media type](https://github.com/opencontainers/image-spec/blob/v1.0.2/media-types.md) of the layer
+which contains the tarball with Kubernetes manifests.
 
 ```yaml
 spec:
-  verify:
-    provider: cosign
-    secretRef:
-      name: cosign-key
+  layerSelector:
+    mediaType: "application/deployment.content.v1.tar+gzip"
 ```
 
+If the layer selector matches more than one layer,
+the first layer matching the specified media type will be used.
+Note that Flux requires that the OCI layer is
+[compressed](https://github.com/opencontainers/image-spec/blob/v1.0.2/layer.md#gzip-media-types)
+in the `tar+gzip` format.
+
 ### Pull artifacts from private repositories
 
 For authentication purposes, Flux users can choose between supplying static credentials with Kubernetes secrets
@@ -203,6 +214,34 @@ controller will use a specific cloud SDK for authentication purposes. If both `s
 a non-generic provider are present in the definition, the controller will use the static credentials
 from the referenced secret.
 
+### Verify artifacts
+
+To verify the authenticity of the OCI artifacts, Flux will use the Sigstore Go SDK and implement verification
+for artifacts which were either signed with keys generated by Cosign or signed using the Cosign
+[keyless method](https://github.com/sigstore/cosign/blob/main/KEYLESS.md).
+
+To enable signature verification, the Cosign public key can be supplied with:
+
+```yaml
+spec:
+  verify:
+    provider: cosign
+    secretRef:
+      name: cosign-key
+```
+
+For verifying public artifacts which are signed using the keyless method,
+the `spec.verify.secretRef` field must be omitted:
+
+```yaml
+spec:
+  verify:
+    provider: cosign
+```
+
+When using the keyless method, Flux will verify the signatures in the Rekor
+transparency log instance hosted at [rekor.sigstore.dev](https://rekor.sigstore.dev/).
+
 ### Reconcile artifacts
 
 The `OCIRepository` can be used as a drop-in replacement for `GitRepository` and `Bucket` sources.
@@ -359,7 +398,7 @@ The Flux CLI will produce OCI artifacts with the following format:
   "config": {
     "mediaType": "application/vnd.docker.container.image.v1+json",
     "size": 233,
-    "digest": "sha256:e7c52109f8e375176a888fd571dc0e0b40ed8a80d9301208474a2a906b0a2dcc"
+    "digest": "sha256:3b6cdcc7adcc9a84d3214ee1c029543789d90b5ae69debe9efa3f66e982875de"
   },
   "layers": [
     {
@@ -369,14 +408,16 @@ The Flux CLI will produce OCI artifacts with the following format:
     }
   ],
   "annotations": {
-    "source.toolkit.fluxcd.io/revision": "6.1.6/450796ddb2ab6724ee1cc32a4be56da032d1cca0",
-    "source.toolkit.fluxcd.io/url": "https://github.com/stefanprodan/podinfo.git"
+    "org.opencontainers.image.created": "2022-08-08T12:31:41+03:00",
+    "org.opencontainers.image.revision": "6.1.8/b3b00fe35424a45d373bf4c7214178bc36fd7872",
+    "org.opencontainers.image.source": "https://github.com/stefanprodan/podinfo.git"
   }
 }
 ```
 
 The source-controller will extract the first layer from the OCI artifact, and will repackage it
-as an internal `sourcev1.Artifact`. The internal artifact revision will be set to the OCI SHA256 digest:
+as an internal `sourcev1.Artifact`. The internal artifact revision will be set to the OCI SHA256 digest
+and the OpenContainers annotation will be copied to the internal artifact metadata:
 
 ```yaml
 apiVersion: source.toolkit.fluxcd.io/v1beta2
@@ -400,6 +441,10 @@ status:
   artifact:
     checksum: d7e924b4882e55b97627355c7b3d2e711e9b54303afa2f50c25377f4df66a83b
     lastUpdateTime: "2022-06-22T09:14:21Z"
+    metadata:
+      org.opencontainers.image.created: "2022-08-08T12:31:41+03:00"
+      org.opencontainers.image.revision: 6.1.8/b3b00fe35424a45d373bf4c7214178bc36fd7872
+      org.opencontainers.image.source: https://github.com/stefanprodan/podinfo.git
     path: ocirepository/oci/podinfo/3b6cdcc7adcc9a84d3214ee1c029543789d90b5ae69debe9efa3f66e982875de.tar.gz
     revision: 3b6cdcc7adcc9a84d3214ee1c029543789d90b5ae69debe9efa3f66e982875de
     size: 1105
@@ -424,3 +469,10 @@ status:
 ### Enabling the feature
 
 The feature is enabled by default.
+
+## Implementation History
+
+* **2022-08-08** Partially implemented by [source-controller#788](https://github.com/fluxcd/source-controller/pull/788)
+* **2022-08-11** First implementation released with [flux2 v0.32.0](https://github.com/fluxcd/flux2/releases/tag/v0.32.0)
+* **2022-08-29** Select layer by OCI media type released with [flux2 v0.33.0](https://github.com/fluxcd/flux2/releases/tag/v0.33.0)
+* **2022-09-29** Verifying OCI artifacts with Cosign released with [flux2 v0.35.0](https://github.com/fluxcd/flux2/releases/tag/v0.35.0)

rfcs/0004-insecure-http/README.md

@@ -0,0 +1,219 @@
+# RFC-0004 Block insecure HTTP connections across Flux
+
+**Status:** implementable
+
+**Creation Date:** 2022-09-08
+
+**Last update:** 2022-10-21
+
+## Summary
+
+Flux should have a consistent way of disabling insecure HTTP connections.
+
+At the controller level, a flag should be present which would disable all outgoing HTTP connections.
+At the object level, a field should be provided which would enable the use of non-TLS endpoints.
+
+If the use of a non-TLS endpoint is not supported, reconciliation will fail and the object will be marked
+as stalled, signalling that human intervention is required.
+
+## Motivation
+
+Today the use of non-TLS based connections is inconsistent across Flux controllers.
+
+Controllers that deal only with `http` and `https` schemes have no way to block use of the `http` scheme at controller-level.
+Some Flux objects provide a `.spec.insecure` field to enable the use of non-TLS based endpoints, but they don't clearly notify
+users when the option is not supported (e.g. Azure/GCP Buckets).
+
+### Goals
+
+* Provide a flag across relevant Flux controllers which disables all outgoing HTTP connections.
+* Add a field which enables the use of non-TLS endpoints to appropriate Flux objects.
+* Provide a way for users to be made aware that their use of non-TLS endpoints is not supported if that is the case.
+
+### Non-Goals
+
+* Break Flux's current behavior of allowing HTTP connections.
+* Change in behavior of communication between Flux components.
+
+## Proposal
+
+### Controllers
+
+Flux users should be able to enforce that controllers are using HTTPS connections only.
+This shall be enabled by adding a new boolean flag `--insecure-allow-http` to the following controllers:
+* source-controller
+* notification-controller
+* image-automation-controller
+* image-reflector-controller
+
+The default value of this flag shall be `true`. This would ensure that there is no breaking change with controllers
+still being able to access non-TLS endpoints. To disable this behavior and enforce the use of HTTPS connections, users would
+have to explicitly pass the flag to the controller:
+
+```yaml
+spec:
+  template:
+    spec:
+      containers:
+      - name: manager
+        image: fluxcd/source-controller
+        args:
+          - --watch-all-namespaces
+          - --log-level=info
+          - --log-encoding=json
+          - --enable-leader-election
+          - --storage-path=/data
+          - --storage-adv-addr=source-controller.$(RUNTIME_NAMESPACE).svc.cluster.local.
+          - --insecure-allow-http=false
+```
+
+**Note:** The flag shall not be added to the following controllers:
+* kustomize-controller: This flag is excluded from this controller, as the upstream `kubenetes-sigs/kustomize` project
+does not support disabling HTTP connections while fetching resources from remote bases. We can revisit this if the
+upstream project adds support for this at a later point in time.
+* helm-controller: This flag does not serve a purpose in this controller, as the controller does not make any HTTP calls.
+Furthermore although both controllers can also do remote applies, serving `kube-apiserver` over plain
+HTTP is disabled by default. While technically this can be enabled, the option for this configuration was also disabled
+quite a while back (ref: https://github.com/kubernetes/kubernetes/pull/65830/).
+
+### Objects
+
+Some Flux objects, like `GitRepository`, provide a field for specifying a URL, and the URL would contain the scheme.
+In such cases, the scheme can be used for inferring the transport type of the connection and consequently,
+whether to use HTTP or HTTPS connections for that object.
+But there are a few objects that don't allow such behavior, for example:
+
+* `ImageRepository`: It provides a field, `.spec.image`, which is used for specifying the address of the image present on
+a container registry. But any address containing a scheme is considered invalid and HTTPS is the default transport used.
+This prevents users from using images present on insecure registries.
+* OCI `HelmRepository`: When using an OCI registry as a Helm repository, the `.spec.url` is expected to begin with `oci://`.
+Since the scheme part of the URL is used to specify the type of `HelmRepository`, there is no way for users to specify
+that the registry is hosted at a non-TLS endpoint.
+
+For such objects, we shall introduce a new boolean field `.spec.insecure`, which shall be `false` by default. Users that
+need their object to point to an HTTP endpoint, can set this to `true`.
+
+### CLI
+
+The Flux CLI offers several commands for creating Flux specific resources. Some of these commands may involve specifying
+an endpoint such as creating an `OCIRepository`:
+
+```sh
+ flux create source oci podinfo \
+    --url=oci://ghcr.io/stefanprodan/manifests/podinfo \
+    --tag=6.1.6 \
+    --interval=10m
+```
+
+Since these commands essentially create object definitions, the CLI should offer a boolean flag `--insecure`
+for the required commands, which will be used for specifying the value of `.spec.insecure` of such objects.
+
+> Note: This flag should not be confused with `--insecure-skip-tls-verify` which is meant to skip TLS verification
+> when using an HTTPS connection.
+
+### Precedence & Validity
+
+Objects with `.spec.insecure` as `true ` will only be allowed if HTTP connections are allowed at the controller level.
+Similarly, an object can have `.spec.insecure` as `true` only if the Saas/Cloud provider allows HTTP connections.
+For example, using a `Bucket` with its `.spec.provider` set to `azure` would be invalid since Azure doesn't allow
+HTTP connections.
+
+### User Stories
+
+#### Story 1
+
+> As a cluster admin of a multi-tenant cluster, I want to ensure all controllers access endpoints using only HTTPS
+> regardless of tenants' object definitions.
+
+Apply a `kustomize` patch which prevents the use of HTTP connections:
+
+```yaml
+apiVersion: kustomize.config.k8s.io/v1beta1
+kind: Kustomization
+resources:
+  - gotk-components.yaml
+  - gotk-sync.yaml
+patches:
+  - patch: |
+      - op: add
+        path: /spec/template/spec/containers/0/args/-
+        value: --insecure-allow-http=false
+    target:
+      kind: Deployment
+      name: "(source-controller|notification-controller|image-reflector-controller|image-automation-controller)"
+  # Since the above flag is not available in kustomize-controller for reasons explained in a previous section,
+  # we disable Kustomize remote builds by disallowing use of remote bases. This ensures that kustomize-controller
+  # won't initiate any plain HTTP connections.
+  - patch: |
+      - op: add
+        path: /spec/template/spec/containers/0/args/-
+        value: --no-remote-bases=true
+    target:
+      kind: Deployment
+      name: kustomize-controller
+```
+
+#### Story 2
+
+> As an application developer, I'm trying to debug a new image pushed to my local registry which
+> is not served over HTTPS.
+
+Modify the object spec to use HTTP connections explicitly:
+```yaml
+apiVersion: image.toolkit.fluxcd.io/v1beta1
+kind: ImageRepository
+metadata:
+  name: podinfo
+  namespace: flux-system
+spec:
+  image: kind-registry:5000/stefanprodan/podinfo
+  interval: 1m0s
+  insecure: true
+```
+
+### Alternatives
+
+Instead of adding a flag, we can instruct users to make use of Kyverno policies to enforce that
+all objects have `.spec.insecure` as `false` and any URLs present in the definition don't have `http`
+as the scheme. This is less attractive, as this would ask users to install another software and prevent
+Flux multi-tenancy from being standalone.
+
+## Design Details
+
+If a controller is started with `--insecure-allow-http=false`, any URL in a Flux object which has `http`
+as the scheme will result in an unsuccessful reconciliation and the following condition will be added to the object's
+`.status.conditions`:
+
+```yaml
+status:
+  conditions:
+  - lastTransitionTime: "2022-09-06T09:14:21Z"
+    message: "Use of insecure HTTP connections isn't allowed for this controller"
+    observedGeneration: 1
+    reason: InsecureConnectionsDisallowed
+    status: "True"
+    type: Stalled
+```
+
+Similarly, if an object has `.spec.insecure` as `true` but the Cloud provider doesn't allow HTTP connections,
+the reconciliation will fail and the following condition will be added to the object's `.status.conditions`:
+
+```yaml
+status:
+  conditions:
+  - lastTransitionTime: "2022-09-06T09:14:21Z"
+    message: "Use of insecure HTTP connections isn't allowed for Azure Storage"
+    observedGeneration: 1
+    reason: UnsupportedConnectionType
+    status: "True"
+    type: Stalled
+```
+
+If an object has `.spec.insecure` as `true`, the registry client or bucket client shall be created with the use
+of HTTP connections enabled explicitly.
+
+## Implementation History
+
+**2022-08-12** Allow defining OCI sources for non-TLS container registries with `flux create source oci --insecure`
+released with [flux2 v0.34.0](https://github.com/fluxcd/flux2/releases/tag/v0.34.0)
+

rfcs/0005-artifact-revision-and-digest/README.md

@@ -0,0 +1,362 @@
+# RFC-0005 Artifact `Revision` format and introduction of `Digest`
+
+**Status:** implementable
+
+**Creation date:** 2022-10-20
+
+**Last update:** 2022-11-16
+
+## Summary
+
+This RFC proposes to establish a canonical `Revision` format for an `Artifact`
+which points to a specific revision represented as a checksum (e.g. an OCI
+manifest digest or Git commit SHA) of a named pointer (e.g. an OCI repository
+name or Git tag). In addition, it proposes to include the algorithm name (e.g.
+`sha256`) as a prefix to an advertised checksum for an `Artifact` and
+further referring to it as a `Digest`, deprecating the `Checksum` field.
+
+## Motivation
+
+The current `Artifact` type's `Revision` field format is not "officially"
+standardized (albeit assumed throughout our code bases), and has mostly been
+derived from `GitRepository` which uses `/` as a separator between the named
+pointer (a Git branch or tag) and a specific (SHA-1, or theoretical SHA-256)
+revision.
+
+Since the introduction of `OCIRepository` and with the recent changes around
+`HelmChart` objects to allow the consumption of charts from OCI registries,
+this could be seen as outdated or confusing due to the format differing from
+the canonical format used by OCI, which is `<name>@<algo>:<checksum>` (the
+part after `@` formally known as a ["digest"][digest-spec]) to refer to a
+specific version of an OCI manifest.
+
+While also taking note that Git does not have an official canonical format for
+e.g. branch references at a specific commit, and `/` has less of a symbolic
+meaning than `@`, which could be interpreted as "`<branch>` _at_
+`<commit SHA>`".
+
+In addition, with the introduction of algorithm prefixes for an `Artifact`'s
+checksum, it would be possible to add support and allow user configuration of
+other algorithms than SHA-256. For example SHA-384 and SHA-512, or the more
+performant (parallelizable) [BLAKE3][].
+
+Besides this, it would make it easier to implement a client that can verify the
+checksum without having to resort to an assumed format or guessing
+method based on the length of it, and allows for a more robust solution in
+which it can continue to calculate against the algorithm of a previous
+configuration.
+
+The inclusion of the `Artifact`'s algorithm prefix has been proposed before in
+[source-controller#855](https://github.com/fluxcd/source-controller/issues/855),
+with supportive response from Core Maintainers.
+
+### Goals
+
+- Establish a canonical format to refer to an `Artifact`'s `Revision` field
+  which consists of a named pointer and a specific checksum reference.
+- Allow easier verification of the `Artifact`'s checksum by including an
+  alias for the algorithm.
+- Deprecate the `Artifact`'s `Checksum` field in favor of the `Digest` field.
+- Allow configuration of the algorithm used to calculate the checksum of an
+  `Artifact`.
+- Allow configuration of algorithms other than SHA-256 to calculate the
+  `Digest` of an `Artifact`.
+- Allow compatibility with SemVer name references which might contain an `@`
+  symbol already (e.g. `package@v1.0.0@sha256:...`, as opposed to OCI's
+  `name:v1.0.0@sha256:...`).
+
+### Non-Goals
+
+- Define a canonical format for an `Artifact`'s `Revision` field which contains
+  a named pointer and a different reference than a checksum.
+
+## Proposal
+
+### Establish an Artifact Revision format
+
+Change the format of the `Revision` field of the `source.toolkit.fluxcd.io`
+Group's `Artifact` type across all `Source` kinds to contain an `@` separator
+opposed to `/`, and include the algorithm alias as a prefix to the checksum
+(creating a "digest").
+
+```text
+[ <named pointer> ] [ [ "@" ] <algo> ":" <checksum> ]
+```
+
+Where `<named pointer>` is the name of e.g. a Git branch or OCI repository
+name, `<checksum>` is the exact revision (e.g. a Git commit SHA or OCI manifest
+digest), and `<algo>` is the alias of the algorithm used to calculate the
+checksum (e.g. `sha256`). In case only a named pointer or digest is advertised,
+the `@` is omitted.
+
+For a `GitRepository`'s `Artifact` pointing towards an SHA-1 Git commit on
+branch `main`, the `Revision` field value would become:
+
+```text
+main@sha1:1eabc9a41ca088515cab83f1cce49eb43e84b67f
+```
+
+For a `GitRepository`'s `Artifact` pointing towards a specific SHA-1 Git commit
+without a defined branch or tag, the `Revision` field value would become:
+
+```text
+sha1:1eabc9a41ca088515cab83f1cce49eb43e84b67f
+```
+
+For a `Bucket`'s `Artifact` with a revision based on an SHA-256 calculation of
+a list of object keys and their etags, the `Revision` field value would become:
+
+```text
+sha256:8fb62a09c9e48ace5463bf940dc15e85f525be4f230e223bbceef6e13024110c
+```
+
+For a `HelmChart`'s `Artifact` pointing towards a Helm chart version, the
+`Revision` field value would become:
+
+```text
+1.2.3
+```
+
+### Introduce a `Digest` field
+
+Introduce a new field to the `source.toolkit.fluxcd.io` Group's `Artifact` type
+across all `Source` kinds called `Digest`, containing the checksum of the file
+advertised in the `Path`, and alias of the algorithm used to calculate it
+(creating a ["digest"][digest-spec]).
+
+```text
+<algo> ":" <checksum>
+```
+
+For a `GitRepository` `Artifact`'s checksum calculated using SHA-256, the
+`Digest` field value would become:
+
+```text
+sha256:1111f92aba67995f108b3ee3ffdc00edcfe206b11fbbb459c8ef4c4a8209fca8
+```
+
+#### Deprecate the `Checksum` field
+
+In favor of the `Digest` field, the `Checksum` field of the `source.toolkit.fluxcd.io`
+Group's `Artifact` type across all `Source` kinds is deprecated, and removed in
+a future version.
+
+### User Stories
+
+#### Artifact revision verification
+
+> As a user of the source-controller, I want to be able to see the exact
+> revision of an Artifact that is being used, so that I can verify that it
+> matches the expected revision at a remote source.
+
+For a Source kind that has an `Artifact` with a `Revision` which contains a
+checksum, the field value can be retrieved using the Kubernetes API. For
+example:
+
+```console
+$ kubectl get gitrepository -o jsonpath='{.status.artifact.revision}' <name>
+main@sha1:1eabc9a41ca088515cab83f1cce49eb43e84b67f
+```
+
+#### Artifact checksum verification
+
+> As a user of the source-controller, I want to be able to verify the checksum
+> of an Artifact.
+
+For a Source kind with an `Artifact` the digest consisting of the algorithm
+alias and checksum is advertised in the `Digest` field, and can be retrieved
+using the Kubernetes API. For example:
+
+```console
+$ kubectl get gitrepository -o jsonpath='{.status.artifact.digest}' <name>
+sha256:1111f92aba67995f108b3ee3ffdc00edcfe206b11fbbb459c8ef4c4a8209fca8
+```
+
+#### Artifact checksum algorithm configuration
+
+> As a user of the source-controller, I want to be able to configure the
+> algorithm used to calculate the checksum of an Artifact.
+
+The source-controller binary accepts a `--artifact-digest-algo` flag which
+configures the algorithm used to calculate the checksum of an `Artifact`.
+The default value is `sha256`, but can be changed to `sha384`, `sha512`
+or `blake3`.
+
+When set, newly advertised `Artifact`'s `Digest` fields will be calculated
+using the configured algorithm. For previous `Artifact`'s that were set using
+a previous configuration, the `Artifact`'s `Digest` field will be recalculated
+using the advertised algorithm.
+
+#### Artifact revisions in notifications
+
+> As a user of the notification-controller, I want to be able to see the
+> exact revision a notification is referring to.
+
+The notification-controller can use the revision for a Source's `Artifact`
+attached as an annotation to an `Event`, and correctly parses the value field
+when attempting to extract e.g. a Git commit digest from an event for a
+`GitRepository`. As currently already applicable for the `/` separator.
+
+> As a user of the notification-controller, I want to be able to observe what
+> commit has been applied on my (supported) Git provider.
+
+The notification-controller can use the revision attached as an annotation to
+an `Event`, and is capable of extracting the correct reference for a Git
+provider integration (e.g. GitHub, GitLab) to construct a payload. For example,
+extracting `1eabc9a41ca088515cab83f1cce49eb43e84b67f` from
+`main@sha1:1eabc9a41ca088515cab83f1cce49eb43e84b67f`.
+
+#### Artifact revisions in listed views
+
+> As a Flux CLI user, I want to see the current revision of my Source in a
+> listed overview.
+
+By running `flux get source <kind>`, the listed view of Sources would show a
+truncated version of the checksum in the `Revision` field.
+
+```console
+$ flux get source gitrepository
+NAME            REVISION              SUSPENDED       READY   MESSAGE                                                                      
+flux-monitoring main@sha1:1eabc9a4    False           True    stored artifact for revision 'main@sha1:1eabc9a41ca088515cab83f1cce49eb43e84b67f'
+
+$ flux get source oci
+NAME            REVISION               SUSPENDED       READY   MESSAGE                                                                                              
+apps-source     local@sha256:e5fa481b  False           True    stored artifact for digest 'local@sha256:e5fa481bb17327bd269927d0a223862d243d76c89fe697ea8c9adefc47c47e17'
+
+$ flux get source bucket
+NAME            REVISION         SUSPENDED       READY   MESSAGE                                                                                              
+apps-source     sha256:e3b0c442  False           True    stored artifact for revision 'sha256:8fb62a09c9e48ace5463bf940dc15e85f525be4f230e223bbceef6e13024110c'
+```
+
+### Alternatives
+
+The two main alternatives around the `Revision` parts in this RFC are to either
+keep the current field value formats as is, or to invent another format. Given
+the [motivation](#motivation) for this RFC outlines the reasoning for not
+keeping the current `Revision` format, and the proposed is a commonly known
+format. Neither strike as a better alternative.
+
+For the changes related to `Checksum` and `Digest`, the alternative is to keep
+the current field name as is, and only change the field value format. However,
+given the naming of the field is more accurate with the introduction of the
+algorithm alias, and now is the time to make last (breaking) changes to the
+API. This does not strike as a better alternative.
+
+## Design Details
+
+### Artifact Revision format
+
+For an `Artifact`'s `Revision` which contains a checksum referring to an exact
+revision, the checksum within the value MUST be appended with an alias for the
+algorithm separated by `:` (e.g. `sha256:...`), further referred to as a
+"digest". The algorithm alias and checksum of the digest MUST be lowercase and
+alphanumeric.
+
+For an `Artifact`'s `Revision` which contains a digest and a named pointer,
+it MUST be prefixed with `@`, and appended at the end of the `Revision` value.
+The named pointer MAY contain arbitrary characters, including but not limited
+to `/` and `@`.
+
+#### Format
+
+```text
+[ <named pointer> ] [ [ "@" ] <algo> ":" <checksum> ]
+```
+
+Where `[ ]` indicates an optional element, `" "` a literal string, and `< >`
+a variable.
+
+#### Parsing
+
+When parsing the `Revision` field value of an `Artifact` to extract the digest,
+the value after the last `@` is considered to be the digest. The remaining
+value on the left side is considered to be the named pointer, which MAY contain
+an additional `@` separator if applicable for the domain of the `Source`
+implementation.
+
+#### Truncation
+
+When truncating the `Revision` field value of an `Artifact` to display in a
+view with limited space, the `<checksum>` of the digest MAY be truncated to
+7 or more characters. The `<algo>` of the digest MUST NOT be truncated.
+In addition, a digest MUST always contain the full length checksum for the
+algorithm.
+
+#### Backwards compatibility
+
+To allow backwards compatability in the notification-controller, Flux CLI and
+other applicable components, the `Revision` new field value format could be
+detected by the presence of the `@` or `:` characters. Falling back to their
+current behaviour if not present, phasing out the old format in a future
+release.
+
+### Artifact Digest
+
+The `Artifact`'s `Digest` field advertises the checksum of the file in the
+`URL`. The checksum within the value MUST be appended with an alias for the
+algorithm separated by `:` (e.g. `sha256:...`). This follows the
+[digest format][go-digest] of OCI.
+
+#### Format
+
+```text
+<algo> ":" <checksum>
+```
+
+Where `" "` indicates a literal string, and `< >` a variable.
+
+#### Library
+
+The library used for calculating the `Digest` field value is
+`github.com/opencontainers/go-digest`. This library is stable and extensible,
+and used by various OCI libraries which we already depend on.
+
+#### Calculation
+
+The checksum in the `Digest` field value MUST be calculated using the canonical
+algorithm [set at runtime](#configuration).
+
+#### Configuration
+
+The algorithm used for calculating the `Digest` field value MAY be configured
+using the `--artifact-digest-algo` flag of the source-controller binary. The
+default value is `sha256`, but can be changed to `sha384`, `sha512` or
+`blake3`.
+
+**Note:** availability of BLAKE3 is at present dependent on an explicit import
+of `github.com/opencontainers/go-digest/blake3`.
+
+When the provided algorithm is NOT supported, the source-controller MUST
+fail to start.
+
+When the configured algorithm changes, the `Digest` MAY be recalculated to
+update the value.
+
+#### Verification
+
+The checksum of a downloaded artifact MUST be verified against the `Digest`
+field value. If the checksum does not match, the verification MUST fail.
+
+### Deprecation of Checksum
+
+The `Artifact`'s `Checksum` field is deprecated and MUST be removed in a
+future release. The `Digest` field MUST be used instead.
+
+#### Backwards compatibility
+
+To allow backwards compatability, the source-controller could continue
+to advertise the checksum part of a `Digest` in the `Checksum` field until
+the field is removed.
+
+## Implementation History
+
+<!--
+Major milestones in the lifecycle of the RFC such as:
+- The first Flux release where an initial version of the RFC was available.
+- The version of Flux where the RFC graduated to general availability.
+- The version of Flux where the RFC was retired or superseded.
+-->
+
+[BLAKE3]: https://github.com/BLAKE3-team/BLAKE3
+[digest-spec]: https://github.com/opencontainers/image-spec/blob/main/descriptor.md#digests
+[go-digest]: https://pkg.go.dev/github.com/opencontainers/go-digest#hdr-Basics

tests/azure/README.md

@@ -5,7 +5,7 @@ Azure services are actually working now and in the future.
 
 ## Architecture
 
-The tests are run with the help of pre configured Azure subscriptions and Azure DevOps organization. Access to thse accounts are currently limited to
+The tests are run with the help of pre-configured Azure subscriptions and Azure DevOps organization. Access to those accounts are currently limited to
 Flux maintainers.
 * [Azure Subscription](https://portal.azure.com/#@weaveworksendtoend.onmicrosoft.com/resource/subscriptions/71e8dce4-9af6-405a-8e96-425f5d3c302b/overview)
 * [Azure DevOps organization](https://dev.azure.com/flux-azure/)
@@ -15,17 +15,17 @@ state in Azure. They should all be placed in the [same container](https://portal
 but use different keys.
 
 The [shared](./terraform/shared) Terraform creates long running cheaper infrastructure that is used across all tests. This includes a Key Vault which
-contains an ssh key and Azure DevOps Personal Access Token which cant be created automatically. It also includes an Azure Container Registry which the
+contains an ssh key and Azure DevOps Personal Access Token which cannot be created automatically. It also includes an Azure Container Registry which the
 forked [podinfo](https://dev.azure.com/flux-azure/e2e/_git/podinfo) repository pushes an Helm Chart and Docker image to.
 
 The [aks](./terraform/aks) Terraform creates the AKS cluster and related resources to run the tests. It creates the AKS cluster, Azure DevOps
-repositories, Key Vault Key for Sops, and Azure EventHub. The resources should be created and destroyed before and after every test run. Currently
+repositories, Key Vault Key for Sops, and Azure EventHub. The resources should be created and destroyed before and after every test run. Currently,
 the same state is reused between runs to make sure that resources are left running after each test run.
 
 ## Tests
 
-Each test run is intiated by running `terraform apply` on the aks Terraform, it does this by using the library [terraform-exec](github.com/hashicorp/terraform-exec).
-It then reads the output of the Terraform to get credentials and ssh keys, this means that a lot of the communication with the Azure API is offest to
+Each test run is initiated by running `terraform apply` on the aks Terraform, it does this by using the library [terraform-exec](github.com/hashicorp/terraform-exec).
+It then reads the output of the Terraform to get credentials and ssh keys, this means that a lot of the communication with the Azure API is offset to
 Terraform instead of requiring it to be implemented in the test.
 
 The following tests are currently implemented: