Refer to authorisation model in RFC-0001

Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>

.github/runners/README.md

@@ -1,42 +1,72 @@
-# Flux GitHub runners
+# Flux ARM64 GitHub runners
 
-How to provision GitHub Actions self-hosted runners for Flux conformance testing.
+The Flux ARM64 end-to-end tests run on Equinix instances provisioned with Docker and GitHub self-hosted runners.
 
-## ARM64 Instance specs
+## Current instances
+
+| Runner        | Instance            | Region |
+|---------------|---------------------|--------|
+| equinix-arm-1 | flux-equinix-arm-01 | AMS1   |
+| equinix-arm-2 | flux-equinix-arm-01 | AMS1   |
+| equinix-arm-3 | flux-equinix-arm-01 | AMS1   |
+| equinix-arm-4 | flux-equinix-arm-02 | DFW2   |
+| equinix-arm-5 | flux-equinix-arm-02 | DFW2   |
+| equinix-arm-6 | flux-equinix-arm-02 | DFW2   |
+
+## Instance setup
 
 In order to add a new runner to the GitHub Actions pool,
-first create an instance on Oracle Cloud with the following configuration:
+first create a server on Equinix with the following configuration:
-- OS: Canonical Ubuntu 20.04
+- Type: c2.large.arm
-- Shape: VM.Standard.A1.Flex
+- OS: Ubuntu 20.04
-- OCPU Count: 2 
-- Memory (GB): 12
-- Network Bandwidth (Gbps): 2
-- Local Disk: Block Storage Only  
 
-Note that the instance image source must be **Canonical Ubuntu** instead of the default Oracle Linux.
+### Install prerequisites
-
-## ARM64 Instance setup
 
 - SSH into a newly created instance
 ```shell
-ssh ubuntu@<instance-public-IP>
+ssh root@<instance-public-IP>
 ``` 
-- Create the action runner dir
+
+- Create the ubuntu user
 ```shell
-mkdir -p actions-runner && cd actions-runner
+adduser ubuntu
+usermod -aG sudo ubuntu
+su - ubuntu
 ```
-- Download the provisioning script
+
+- Create the prerequisites dir
 ```shell
-curl -sL https://raw.githubusercontent.com/fluxcd/flux2/main/.github/runners/arm64.sh > arm64.sh \
+mkdir -p prereq && cd prereq
-  && chmod +x ./arm64.sh
 ```
+
+- Download the prerequisites script
+```shell
+curl -sL https://raw.githubusercontent.com/fluxcd/flux2/main/.github/runners/prereq.sh > prereq.sh \
+  && chmod +x ./prereq.sh
+```
+
+- Install the prerequisites
+```shell
+sudo ./prereq.sh
+```
+
+### Install runners
+
 - Retrieve the GitHub runner token from the repository [settings page](https://github.com/fluxcd/flux2/settings/actions/runners/new?arch=arm64&os=linux)
-- Run the provisioning script passing the token as the first argument
+
+- Create 3 directories `runner1`, `runner2`, `runner3`
+
+- In each dir run:
 ```shell
-sudo ./arm64.sh <TOKEN>
+curl -sL https://raw.githubusercontent.com/fluxcd/flux2/main/.github/runners/runner-setup.sh > runner-setup.sh \
+  && chmod +x ./runner-setup.sh
+
+./runner-setup.sh equinix-arm-<NUMBER> <TOKEN>
 ```
+
 - Reboot the instance
 ```shell
 sudo reboot
-```  
+```
+
 - Navigate to the GitHub repository [runners page](https://github.com/fluxcd/flux2/settings/actions/runners) and check the runner status

.github/runners/prereq.sh

@@ -14,19 +14,14 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
-# This script is meant to be run locally and in CI to validate the Kubernetes
+# This script installs the prerequisites for running Flux end-to-end tests with Docker and GitHub self-hosted runners.
-# manifests (including Flux custom resources) before changes are merged into
-# the branch synced by Flux in-cluster.
 
 set -eu
 
-REPOSITORY_TOKEN=$1
-REPOSITORY_URL=${2:-https://github.com/fluxcd/flux2}
-
 KIND_VERSION=0.11.1
 KUBECTL_VERSION=1.21.2
 KUSTOMIZE_VERSION=4.1.3
-GITHUB_RUNNER_VERSION=2.278.0
+GITHUB_RUNNER_VERSION=2.285.1
 PACKAGES="apt-transport-https ca-certificates software-properties-common build-essential libssl-dev gnupg lsb-release jq"
 
 # install prerequisites
@@ -64,10 +59,3 @@ curl -o actions-runner-linux-arm64.tar.gz -L https://github.com/actions/runner/r
 
 # install runner dependencies
 ./bin/installdependencies.sh
-
-# register runner with GitHub
-sudo -u ubuntu ./config.sh --unattended --url ${REPOSITORY_URL} --token ${REPOSITORY_TOKEN}
-
-# start runner
-./svc.sh install
-./svc.sh start

.github/runners/runner-setup.sh

@@ -0,0 +1,37 @@
+#!/usr/bin/env bash
+
+# Copyright 2021 The Flux authors. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# This script installs a GitHub self-hosted ARM64 runner for running Flux end-to-end tests.
+
+set -eu
+
+RUNNER_NAME=$1
+REPOSITORY_TOKEN=$2
+REPOSITORY_URL=${3:-https://github.com/fluxcd/flux2}
+
+GITHUB_RUNNER_VERSION=2.285.1
+
+# download runner
+curl -o actions-runner-linux-arm64.tar.gz -L https://github.com/actions/runner/releases/download/v${GITHUB_RUNNER_VERSION}/actions-runner-linux-arm64-${GITHUB_RUNNER_VERSION}.tar.gz \
+  && tar xzf actions-runner-linux-arm64.tar.gz \
+  && rm actions-runner-linux-arm64.tar.gz
+
+# register runner with GitHub
+./config.sh --unattended --url ${REPOSITORY_URL} --token ${REPOSITORY_TOKEN} --name ${RUNNER_NAME}
+
+# start runner
+sudo ./svc.sh install
+sudo ./svc.sh start

.github/workflows/e2e-arm64.yaml

@@ -3,14 +3,13 @@ name: e2e-arm64
 on:
   workflow_dispatch:
   push:
-    branches: [ main, update-components, arm64-e2e ]
+    branches: [ main, update-components, equinix-runners ]
 
 jobs:
-  ampere:
+  test:
-    # Runner info
+    # Hosted on Equinix
-    # Owner: Stefan Prodan
     # Docs: https://github.com/fluxcd/flux2/tree/main/.github/runners
-    runs-on: [self-hosted, Linux, ARM64]
+    runs-on: [self-hosted, Linux, ARM64, equinix]
     steps:
       - name: Checkout
         uses: actions/checkout@v2

.github/workflows/e2e.yaml

@@ -2,7 +2,7 @@ name: e2e
 
 on:
   push:
-    branches: [ main ]
+    branches: [ main, e2e* ]
   pull_request:
     branches: [ main ]
 
@@ -35,7 +35,7 @@ jobs:
           version: "1.21.x"
       - name: Setup Calico for network policy
         run: |
-          kubectl apply -f https://docs.projectcalico.org/v3.16/manifests/calico.yaml
+          kubectl apply -f https://docs.projectcalico.org/v3.20/manifests/calico.yaml
           kubectl -n kube-system set env daemonset/calico-node FELIX_IGNORELOOSERPF=true
       - name: Setup Kustomize
         uses: fluxcd/pkg//actions/kustomize@main

MAINTAINERS

@@ -12,9 +12,9 @@ should.
 
 In alphabetical order:
 
-Aurel Canciu, Sortlist <aurel@sortlist.com> (github: @relu, slack: relu)
+Aurel Canciu, NexHealth <aurel.canciu@nexhealth.com> (github: @relu, slack: relu)
 Hidde Beydals, Weaveworks <hidde@weave.works> (github: @hiddeco, slack: hidde)
-Max Jonas Werner, D2iQ <mwerner@d2iq.com> (github: @makkes, slack: max)
+Max Jonas Werner, D2iQ <max@e13.dev> (github: @makkes, slack: max)
 Philip Laine, Xenit <philip.laine@xenit.se> (github: @phillebaba, slack: phillebaba)
 Stefan Prodan, Weaveworks <stefan@weave.works> (github: @stefanprodan, slack: stefanprodan)
 Sunny, Weaveworks <sunny@weave.works> (github: @darkowlzz, slack: darkowlzz)

cmd/flux/trace.go

@@ -40,7 +40,7 @@ import (
 
 var traceCmd = &cobra.Command{
 	Use:   "trace [name]",
-	Short: "trace an in-cluster object throughout the GitOps delivery pipeline",
+	Short: "Trace an in-cluster object throughout the GitOps delivery pipeline",
 	Long: `The trace command shows how an object is managed by Flux,
 from which source and revision it comes, and what's the latest reconciliation status.'`,
 	Example: `  # Trace a Kubernetes Deployment

go.mod

@@ -7,10 +7,10 @@ require (
 	github.com/ProtonMail/go-crypto v0.0.0-20210428141323-04723f9f07d7
 	github.com/cyphar/filepath-securejoin v0.2.2
 	github.com/fluxcd/go-git-providers v0.4.0
-	github.com/fluxcd/helm-controller/api v0.14.0
+	github.com/fluxcd/helm-controller/api v0.14.1
 	github.com/fluxcd/image-automation-controller/api v0.18.0
 	github.com/fluxcd/image-reflector-controller/api v0.14.0
-	github.com/fluxcd/kustomize-controller/api v0.18.1
+	github.com/fluxcd/kustomize-controller/api v0.18.2
 	github.com/fluxcd/notification-controller/api v0.19.0
 	github.com/fluxcd/pkg/apis/meta v0.10.1
 	github.com/fluxcd/pkg/runtime v0.12.2
@@ -18,7 +18,7 @@ require (
 	github.com/fluxcd/pkg/ssh v0.0.5
 	github.com/fluxcd/pkg/untar v0.0.5
 	github.com/fluxcd/pkg/version v0.0.1
-	github.com/fluxcd/source-controller/api v0.19.0
+	github.com/fluxcd/source-controller/api v0.19.2
 	github.com/go-errors/errors v1.4.0 // indirect
 	github.com/go-git/go-git/v5 v5.4.2
 	github.com/google/go-cmp v0.5.6

go.sum

@@ -225,20 +225,20 @@ github.com/fatih/color v1.7.0/go.mod h1:Zm6kSWBoL9eyXnKyktHP6abPY2pDugNf5Kwzbycv
 github.com/felixge/httpsnoop v1.0.1/go.mod h1:m8KPJKqk1gH5J9DgRY2ASl2lWCfGKXixSwevea8zH2U=
 github.com/fluxcd/go-git-providers v0.4.0 h1:hdGGRDCNphf9FRrk297lorhwHWcST74R7cGAOZTRtSU=
 github.com/fluxcd/go-git-providers v0.4.0/go.mod h1:enIPrXnSOBxahS6rngohpG3d/QZ3yjjy/w+agbp97ZI=
-github.com/fluxcd/helm-controller/api v0.14.0 h1:I6gyOaEcUUEk0cuXgKhvmvgN833LxdZ3dngTnMDii7w=
+github.com/fluxcd/helm-controller/api v0.14.1 h1:aAWaYZxTI68SD1R2SpNJh8+hm9oBeIOa9nW4YX5qYjM=
-github.com/fluxcd/helm-controller/api v0.14.0/go.mod h1:3wDrDVSH/3yH31PzfXzYCCf8OP70eg82tlQU1+QGsS0=
+github.com/fluxcd/helm-controller/api v0.14.1/go.mod h1:NkfZ5ugs9EUUPSGHfAnNs295mf8sVKG0842aL6cFzMM=
 github.com/fluxcd/image-automation-controller/api v0.18.0 h1:/FMwAJ31s4xk5XL7AKYWT0Z9f+GpLbd55e2I1m6jg/o=
 github.com/fluxcd/image-automation-controller/api v0.18.0/go.mod h1:XqrRtH4aFOY1llgf4wx8VcSmzLsdV/xWcrPbdZjvvLg=
 github.com/fluxcd/image-reflector-controller/api v0.14.0 h1:JlwwpQENkGSxtAC7VXykpOqObsupO61easXu30jpvb0=
 github.com/fluxcd/image-reflector-controller/api v0.14.0/go.mod h1:ew7uL5XzPH3QTfNxt3BAklDh9ONrR5I2m3D7ST0zE9E=
-github.com/fluxcd/kustomize-controller/api v0.18.1 h1:cssv85lh5RlDDEFvMv/4InR/7SzqX/+rVMYA3GqpnFI=
+github.com/fluxcd/kustomize-controller/api v0.18.2 h1:rGu9R6PMFw3x0S6tVj/ZS54sJWW6/cdWe0Gga09e1AY=
-github.com/fluxcd/kustomize-controller/api v0.18.1/go.mod h1:xGHBIzVXepzm2/0iQJJSbCIRY0Ahq5AgbYsVojpRyX0=
+github.com/fluxcd/kustomize-controller/api v0.18.2/go.mod h1:psXYbb3lbSCbakC715fzdpUnuILra00N++t/S1UUDRM=
 github.com/fluxcd/notification-controller/api v0.19.0 h1:mSsDj30T4v/9aL2GoMY616p+6nIifD1nrZiBD/rUi8U=
 github.com/fluxcd/notification-controller/api v0.19.0/go.mod h1:SkB3tLOXouLN6PAceNCsJrJaawHt+WiUVfUSIYcpwjs=
 github.com/fluxcd/pkg/apis/acl v0.0.1 h1:biCgZMjpDSv3Q4mZPikUJILx3t2MuNXR4Oa5jRQxaNQ=
 github.com/fluxcd/pkg/apis/acl v0.0.1/go.mod h1:y3qOXUFObVWk7jzOjubMnr/u18j1kCeSi6olycnxr/E=
-github.com/fluxcd/pkg/apis/kustomize v0.2.0 h1:jhu2QHvs+j3Zo9rR6w8hkO3LSC6h3M37zY5ejufOmxY=
+github.com/fluxcd/pkg/apis/kustomize v0.3.0 h1:o7o8hHIWh+1kyx/E21b2tZpCD4tX8NRijbGioAZXSBU=
-github.com/fluxcd/pkg/apis/kustomize v0.2.0/go.mod h1:gEl+W5cVykCC3RfrCaqe+Pz+j4lKl2aeR4dxsom/zII=
+github.com/fluxcd/pkg/apis/kustomize v0.3.0/go.mod h1:bkeOkDpm2PQamZT+RLs20ZOt3TgOdLKWmi9kVxmcdmM=
 github.com/fluxcd/pkg/apis/meta v0.10.0/go.mod h1:CW9X9ijMTpNe7BwnokiUOrLl/h13miwVr/3abEQLbKE=
 github.com/fluxcd/pkg/apis/meta v0.10.1 h1:zISenRlqNG7WK8TP3HxZTvv+1Z7JZOUIQvZrOr6pQ2w=
 github.com/fluxcd/pkg/apis/meta v0.10.1/go.mod h1:yUblM2vg+X8TE3A2VvJfdhkGmg+uqBlSPkLk7dxi0UM=
@@ -252,8 +252,9 @@ github.com/fluxcd/pkg/untar v0.0.5 h1:UGI3Ch1UIEIaqQvMicmImL1s9npQa64DJ/ozqHKB7g
 github.com/fluxcd/pkg/untar v0.0.5/go.mod h1:O6V9+rtl8c1mHBafgqFlJN6zkF1HS5SSYn7RpQJ/nfw=
 github.com/fluxcd/pkg/version v0.0.1 h1:/8asQoDXSThz3csiwi4Qo8Zb6blAxLXbtxNgeMJ9bCg=
 github.com/fluxcd/pkg/version v0.0.1/go.mod h1:WAF4FEEA9xyhngF8TDxg3UPu5fA1qhEYV8Pmi2Il01Q=
-github.com/fluxcd/source-controller/api v0.19.0 h1:D4hc/ROhcl7iJdgeVhmM6B7WkDqwtQKnvpl04n+LcNg=
 github.com/fluxcd/source-controller/api v0.19.0/go.mod h1:rUqw0LmoCyGUoElmtLqHc8O35WAQUd8LdgdyDA+kNs4=
+github.com/fluxcd/source-controller/api v0.19.2 h1:+C1s9KslnkA/3SIfvEkFDIesaEpzLKFkumCUYOx5lz8=
+github.com/fluxcd/source-controller/api v0.19.2/go.mod h1:rUqw0LmoCyGUoElmtLqHc8O35WAQUd8LdgdyDA+kNs4=
 github.com/flynn/go-shlex v0.0.0-20150515145356-3f9db97f8568/go.mod h1:xEzjJPgXI435gkrCt3MPfRiAkVrwSbHsst4LCFVfpJc=
 github.com/form3tech-oss/jwt-go v3.2.2+incompatible/go.mod h1:pbq4aXjuKjdthFRnoDwaVPLA+WlJuPGy+QneDUgJi2k=
 github.com/form3tech-oss/jwt-go v3.2.3+incompatible h1:7ZaBxOI7TMoYBfyA3cQHErNNyAWIKUMIwqxEtgHOs5c=

manifests/bases/helm-controller/kustomization.yaml

@@ -1,8 +1,8 @@
 apiVersion: kustomize.config.k8s.io/v1beta1
 kind: Kustomization
 resources:
-- https://github.com/fluxcd/helm-controller/releases/download/v0.14.0/helm-controller.crds.yaml
+- https://github.com/fluxcd/helm-controller/releases/download/v0.14.1/helm-controller.crds.yaml
-- https://github.com/fluxcd/helm-controller/releases/download/v0.14.0/helm-controller.deployment.yaml
+- https://github.com/fluxcd/helm-controller/releases/download/v0.14.1/helm-controller.deployment.yaml
 - account.yaml
 patchesJson6902:
 - target:

manifests/bases/kustomize-controller/kustomization.yaml

@@ -1,8 +1,8 @@
 apiVersion: kustomize.config.k8s.io/v1beta1
 kind: Kustomization
 resources:
-- https://github.com/fluxcd/kustomize-controller/releases/download/v0.18.1/kustomize-controller.crds.yaml
+- https://github.com/fluxcd/kustomize-controller/releases/download/v0.18.2/kustomize-controller.crds.yaml
-- https://github.com/fluxcd/kustomize-controller/releases/download/v0.18.1/kustomize-controller.deployment.yaml
+- https://github.com/fluxcd/kustomize-controller/releases/download/v0.18.2/kustomize-controller.deployment.yaml
 - account.yaml
 patchesJson6902:
 - target:

manifests/bases/source-controller/kustomization.yaml

@@ -1,8 +1,8 @@
 apiVersion: kustomize.config.k8s.io/v1beta1
 kind: Kustomization
 resources:
-- https://github.com/fluxcd/source-controller/releases/download/v0.19.0/source-controller.crds.yaml
+- https://github.com/fluxcd/source-controller/releases/download/v0.19.2/source-controller.crds.yaml
-- https://github.com/fluxcd/source-controller/releases/download/v0.19.0/source-controller.deployment.yaml
+- https://github.com/fluxcd/source-controller/releases/download/v0.19.2/source-controller.deployment.yaml
 - account.yaml
 patchesJson6902:
 - target:

manifests/crds/kustomization.yaml

@@ -1,9 +1,9 @@
 apiVersion: kustomize.config.k8s.io/v1beta1
 kind: Kustomization
 resources:
-- https://github.com/fluxcd/source-controller/releases/download/v0.19.0/source-controller.crds.yaml
+- https://github.com/fluxcd/source-controller/releases/download/v0.19.2/source-controller.crds.yaml
-- https://github.com/fluxcd/kustomize-controller/releases/download/v0.18.1/kustomize-controller.crds.yaml
+- https://github.com/fluxcd/kustomize-controller/releases/download/v0.18.2/kustomize-controller.crds.yaml
-- https://github.com/fluxcd/helm-controller/releases/download/v0.14.0/helm-controller.crds.yaml
+- https://github.com/fluxcd/helm-controller/releases/download/v0.14.1/helm-controller.crds.yaml
 - https://github.com/fluxcd/notification-controller/releases/download/v0.19.0/notification-controller.crds.yaml
 - https://github.com/fluxcd/image-reflector-controller/releases/download/v0.14.0/image-reflector-controller.crds.yaml
 - https://github.com/fluxcd/image-automation-controller/releases/download/v0.18.0/image-automation-controller.crds.yaml

manifests/monitoring/grafana/dashboards/control-plane.json

@@ -8,22 +8,30 @@
         "hide": true,
         "iconColor": "rgba(0, 211, 255, 1)",
         "name": "Annotations & Alerts",
+        "target": {
+          "limit": 100,
+          "matchAny": false,
+          "tags": [],
+          "type": "dashboard"
+        },
         "type": "dashboard"
       }
     ]
   },
   "editable": true,
+  "fiscalYearStartMonth": 0,
   "gnetId": null,
   "graphTooltip": 0,
-  "iteration": 1596541513301,
+  "id": 29,
+  "iteration": 1639041352219,
   "links": [],
+  "liveNow": false,
   "panels": [
     {
       "datasource": "${DS_PROMETHEUS}",
       "description": "",
       "fieldConfig": {
         "defaults": {
-          "custom": {},
           "decimals": 0,
           "mappings": [],
           "thresholds": {
@@ -62,9 +70,10 @@
           "fields": "",
           "values": false
         },
+        "text": {},
         "textMode": "value"
       },
-      "pluginVersion": "7.1.1",
+      "pluginVersion": "8.2.3",
       "targets": [
         {
           "expr": "sum(go_info{namespace=\"$namespace\",pod=~\".*-controller-.*\"})",
@@ -83,7 +92,6 @@
       "description": "",
       "fieldConfig": {
         "defaults": {
-          "custom": {},
           "mappings": [],
           "thresholds": {
             "mode": "absolute",
@@ -125,9 +133,10 @@
           "fields": "",
           "values": false
         },
+        "text": {},
         "textMode": "auto"
       },
-      "pluginVersion": "7.1.1",
+      "pluginVersion": "8.2.3",
       "targets": [
         {
           "expr": "max(workqueue_longest_running_processor_seconds{namespace=\"$namespace\",pod=~\".*-controller-.*\"})",
@@ -147,7 +156,6 @@
       "description": "",
       "fieldConfig": {
         "defaults": {
-          "custom": {},
           "mappings": [],
           "thresholds": {
             "mode": "absolute",
@@ -187,9 +195,10 @@
           "values": false
         },
         "showThresholdLabels": false,
-        "showThresholdMarkers": true
+        "showThresholdMarkers": true,
+        "text": {}
       },
-      "pluginVersion": "7.1.1",
+      "pluginVersion": "8.2.3",
       "targets": [
         {
           "expr": "sum(go_memstats_alloc_bytes{namespace=\"$namespace\",pod=~\".*-controller-.*\"})",
@@ -208,9 +217,6 @@
       "description": "",
       "fieldConfig": {
         "defaults": {
-          "custom": {
-            "align": null
-          },
           "mappings": [],
           "thresholds": {
             "mode": "absolute",
@@ -251,9 +257,10 @@
           "fields": "",
           "values": false
         },
+        "text": {},
         "textMode": "auto"
       },
-      "pluginVersion": "7.1.1",
+      "pluginVersion": "8.2.3",
       "targets": [
         {
           "expr": "sum(rate(rest_client_requests_total{namespace=\"$namespace\",pod=~\".*-controller-.*\"}[1m]))",
@@ -267,131 +274,6 @@
       "title": "API Requests",
       "type": "stat"
     },
-    {
-      "collapsed": false,
-      "datasource": "${DS_PROMETHEUS}",
-      "gridPos": {
-        "h": 1,
-        "w": 24,
-        "x": 0,
-        "y": 5
-      },
-      "id": 15,
-      "panels": [],
-      "title": "Resource Usage",
-      "type": "row"
-    },
-    {
-      "aliasColors": {},
-      "bars": false,
-      "dashLength": 10,
-      "dashes": false,
-      "datasource": "${DS_PROMETHEUS}",
-      "description": "",
-      "fieldConfig": {
-        "defaults": {
-          "custom": {}
-        },
-        "overrides": []
-      },
-      "fill": 1,
-      "fillGradient": 0,
-      "gridPos": {
-        "h": 8,
-        "w": 12,
-        "x": 0,
-        "y": 6
-      },
-      "hiddenSeries": false,
-      "id": 8,
-      "legend": {
-        "alignAsTable": true,
-        "avg": true,
-        "current": true,
-        "max": false,
-        "min": false,
-        "rightSide": false,
-        "show": true,
-        "total": false,
-        "values": true
-      },
-      "lines": true,
-      "linewidth": 1,
-      "nullPointMode": "null",
-      "percentage": false,
-      "pluginVersion": "7.1.1",
-      "pointradius": 2,
-      "points": false,
-      "renderer": "flot",
-      "seriesOverrides": [],
-      "spaceLength": 10,
-      "stack": false,
-      "steppedLine": false,
-      "targets": [
-        {
-          "expr": "histogram_quantile(0.50, sum(rate(rest_client_request_latency_seconds_bucket{namespace=\"$namespace\"}[5m])) by (le))",
-          "interval": "",
-          "legendFormat": "P50",
-          "refId": "A"
-        },
-        {
-          "expr": "histogram_quantile(0.90, sum(rate(rest_client_request_latency_seconds_bucket{namespace=\"$namespace\"}[5m])) by (le))",
-          "hide": true,
-          "interval": "",
-          "legendFormat": "P90",
-          "refId": "B"
-        },
-        {
-          "expr": "histogram_quantile(0.99, sum(rate(rest_client_request_latency_seconds_bucket{namespace=\"$namespace\"}[5m])) by (le))",
-          "hide": false,
-          "interval": "",
-          "legendFormat": "P99",
-          "refId": "C"
-        }
-      ],
-      "thresholds": [],
-      "timeFrom": null,
-      "timeRegions": [],
-      "timeShift": null,
-      "title": "Kubernetes API Requests Duration",
-      "tooltip": {
-        "shared": true,
-        "sort": 0,
-        "value_type": "individual"
-      },
-      "type": "graph",
-      "xaxis": {
-        "buckets": null,
-        "mode": "time",
-        "name": null,
-        "show": true,
-        "values": []
-      },
-      "yaxes": [
-        {
-          "$$hashKey": "object:912",
-          "format": "s",
-          "label": null,
-          "logBase": 1,
-          "max": null,
-          "min": null,
-          "show": true
-        },
-        {
-          "$$hashKey": "object:913",
-          "format": "short",
-          "label": null,
-          "logBase": 1,
-          "max": null,
-          "min": null,
-          "show": true
-        }
-      ],
-      "yaxis": {
-        "align": false,
-        "alignLevel": null
-      }
-    },
     {
       "aliasColors": {},
       "bars": false,
@@ -400,19 +282,13 @@
       "datasource": "${DS_PROMETHEUS}",
       "decimals": null,
       "description": "",
-      "fieldConfig": {
-        "defaults": {
-          "custom": {}
-        },
-        "overrides": []
-      },
       "fill": 1,
       "fillGradient": 0,
       "gridPos": {
         "h": 8,
-        "w": 12,
+        "w": 24,
-        "x": 12,
+        "x": 0,
-        "y": 6
+        "y": 5
       },
       "hiddenSeries": false,
       "id": 21,
@@ -430,8 +306,11 @@
       "lines": true,
       "linewidth": 1,
       "nullPointMode": "null",
+      "options": {
+        "alertThreshold": true
+      },
       "percentage": false,
-      "pluginVersion": "7.1.1",
+      "pluginVersion": "8.2.3",
       "pointradius": 2,
       "points": false,
       "renderer": "flot",
@@ -499,18 +378,30 @@
         "alignLevel": null
       }
     },
+    {
+      "collapsed": false,
+      "datasource": "${DS_PROMETHEUS}",
+      "fieldConfig": {
+        "defaults": {},
+        "overrides": []
+      },
+      "gridPos": {
+        "h": 1,
+        "w": 24,
+        "x": 0,
+        "y": 13
+      },
+      "id": 15,
+      "panels": [],
+      "title": "Resource Usage",
+      "type": "row"
+    },
     {
       "aliasColors": {},
       "bars": false,
       "dashLength": 10,
       "dashes": false,
       "datasource": "${DS_PROMETHEUS}",
-      "fieldConfig": {
-        "defaults": {
-          "custom": {}
-        },
-        "overrides": []
-      },
       "fill": 1,
       "fillGradient": 0,
       "gridPos": {
@@ -534,8 +425,11 @@
       "lines": true,
       "linewidth": 1,
       "nullPointMode": "null",
+      "options": {
+        "alertThreshold": true
+      },
       "percentage": false,
-      "pluginVersion": "7.1.1",
+      "pluginVersion": "8.2.3",
       "pointradius": 2,
       "points": false,
       "renderer": "flot",
@@ -600,12 +494,6 @@
       "dashLength": 10,
       "dashes": false,
       "datasource": "${DS_PROMETHEUS}",
-      "fieldConfig": {
-        "defaults": {
-          "custom": {}
-        },
-        "overrides": []
-      },
       "fill": 1,
       "fillGradient": 0,
       "gridPos": {
@@ -629,8 +517,11 @@
       "lines": true,
       "linewidth": 1,
       "nullPointMode": "null",
+      "options": {
+        "alertThreshold": true
+      },
       "percentage": false,
-      "pluginVersion": "7.1.1",
+      "pluginVersion": "8.2.3",
       "pointradius": 2,
       "points": false,
       "renderer": "flot",
@@ -694,6 +585,10 @@
     {
       "collapsed": false,
       "datasource": "${DS_PROMETHEUS}",
+      "fieldConfig": {
+        "defaults": {},
+        "overrides": []
+      },
       "gridPos": {
         "h": 1,
         "w": 24,
@@ -711,12 +606,6 @@
       "dashLength": 10,
       "dashes": false,
       "datasource": "${DS_PROMETHEUS}",
-      "fieldConfig": {
-        "defaults": {
-          "custom": {}
-        },
-        "overrides": []
-      },
       "fill": 1,
       "fillGradient": 0,
       "gridPos": {
@@ -741,8 +630,11 @@
       "lines": true,
       "linewidth": 1,
       "nullPointMode": "null",
+      "options": {
+        "alertThreshold": true
+      },
       "percentage": false,
-      "pluginVersion": "7.1.1",
+      "pluginVersion": "8.2.3",
       "pointradius": 2,
       "points": false,
       "renderer": "flot",
@@ -810,12 +702,6 @@
       "datasource": "${DS_PROMETHEUS}",
       "decimals": 2,
       "description": "",
-      "fieldConfig": {
-        "defaults": {
-          "custom": {}
-        },
-        "overrides": []
-      },
       "fill": 1,
       "fillGradient": 0,
       "gridPos": {
@@ -841,7 +727,7 @@
       "linewidth": 1,
       "nullPointMode": "null",
       "percentage": false,
-      "pluginVersion": "7.1.1",
+      "pluginVersion": "8.2.3",
       "pointradius": 2,
       "points": false,
       "renderer": "flot",
@@ -916,12 +802,6 @@
       "datasource": "${DS_PROMETHEUS}",
       "decimals": 2,
       "description": "",
-      "fieldConfig": {
-        "defaults": {
-          "custom": {}
-        },
-        "overrides": []
-      },
       "fill": 1,
       "fillGradient": 0,
       "gridPos": {
@@ -947,7 +827,7 @@
       "linewidth": 1,
       "nullPointMode": "null",
       "percentage": false,
-      "pluginVersion": "7.1.1",
+      "pluginVersion": "8.2.3",
       "pointradius": 2,
       "points": false,
       "renderer": "flot",
@@ -1017,6 +897,10 @@
     {
       "collapsed": false,
       "datasource": "${DS_PROMETHEUS}",
+      "fieldConfig": {
+        "defaults": {},
+        "overrides": []
+      },
       "gridPos": {
         "h": 1,
         "w": 24,
@@ -1034,12 +918,6 @@
       "dashLength": 10,
       "dashes": false,
       "datasource": "${DS_PROMETHEUS}",
-      "fieldConfig": {
-        "defaults": {
-          "custom": {}
-        },
-        "overrides": []
-      },
       "fill": 1,
       "fillGradient": 0,
       "gridPos": {
@@ -1065,7 +943,7 @@
       "linewidth": 1,
       "nullPointMode": "null as zero",
       "percentage": false,
-      "pluginVersion": "7.1.1",
+      "pluginVersion": "8.2.3",
       "pointradius": 2,
       "points": false,
       "renderer": "flot",
@@ -1147,12 +1025,6 @@
       "datasource": "${DS_PROMETHEUS}",
       "decimals": 2,
       "description": "",
-      "fieldConfig": {
-        "defaults": {
-          "custom": {}
-        },
-        "overrides": []
-      },
       "fill": 1,
       "fillGradient": 0,
       "gridPos": {
@@ -1178,7 +1050,7 @@
       "linewidth": 1,
       "nullPointMode": "null",
       "percentage": false,
-      "pluginVersion": "7.1.1",
+      "pluginVersion": "8.2.3",
       "pointradius": 2,
       "points": false,
       "renderer": "flot",
@@ -1253,12 +1125,6 @@
       "datasource": "${DS_PROMETHEUS}",
       "decimals": 2,
       "description": "",
-      "fieldConfig": {
-        "defaults": {
-          "custom": {}
-        },
-        "overrides": []
-      },
       "fill": 1,
       "fillGradient": 0,
       "gridPos": {
@@ -1284,7 +1150,7 @@
       "linewidth": 1,
       "nullPointMode": "null",
       "percentage": false,
-      "pluginVersion": "7.1.1",
+      "pluginVersion": "8.2.3",
       "pointradius": 2,
       "points": false,
       "renderer": "flot",
@@ -1353,7 +1219,7 @@
     }
   ],
   "refresh": "10s",
-  "schemaVersion": 26,
+  "schemaVersion": 31,
   "style": "light",
   "tags": [
     "flux"
@@ -1366,6 +1232,8 @@
           "text": "Prometheus",
           "value": "Prometheus"
         },
+        "description": null,
+        "error": null,
         "hide": 2,
         "includeAll": false,
         "label": null,
@@ -1387,19 +1255,23 @@
         },
         "datasource": "${DS_PROMETHEUS}",
         "definition": "workqueue_work_duration_seconds_count",
+        "description": null,
+        "error": null,
         "hide": 0,
         "includeAll": false,
         "label": null,
         "multi": false,
         "name": "namespace",
         "options": [],
-        "query": "workqueue_work_duration_seconds_count",
+        "query": {
+          "query": "workqueue_work_duration_seconds_count",
+          "refId": "Prometheus-namespace-Variable-Query"
+        },
         "refresh": 2,
         "regex": "/.*namespace=\"([^\"]*).*/",
         "skipUrlSync": false,
         "sort": 0,
         "tagValuesQuery": "",
-        "tags": [],
         "tagsQuery": "",
         "type": "query",
         "useTags": false
@@ -1426,5 +1298,5 @@
   "timezone": "",
   "title": "Flux Control Plane",
   "uid": "flux-control-plane",
-  "version": 1
+  "version": 2
-}
+}

manifests/monitoring/kube-prometheus-stack/release.yaml

@@ -6,6 +6,7 @@ spec:
   interval: 5m
   chart:
     spec:
+      version: 23.2.0
       chart: kube-prometheus-stack
       sourceRef:
         kind: HelmRepository

manifests/scripts/bundle.sh

@@ -16,8 +16,8 @@
 
 set -e
 
-IN_PATH=${1:-"$(git rev-parse --show-toplevel)/manifests"}
+IN_PATH=${1:-"$(realpath $(dirname "${BASH_SOURCE[0]}")/../..)/manifests"}
-OUT_PATH=${2:-"$(git rev-parse --show-toplevel)/cmd/flux/manifests"}
+OUT_PATH=${2:-"$(realpath $(dirname "${BASH_SOURCE[0]}")/../..)/cmd/flux/manifests"}
 TAR=${3}
 
 info() {

rfcs/0004-multi-tenancy/README.md

@@ -0,0 +1,225 @@
+# RFC-0004 Flux Multi-Tenancy
+
+**Status:** provisional
+
+**Creation date:** 2021-11-15
+
+**Last update:** 2021-12-17
+
+## Summary
+
+This RFC explains the mechanisms available in Flux for implementing multi-tenancy, defines two
+models for multi-tenancy, and gives reference implementations for those models.
+
+## Motivation
+
+To this point, the Flux project has provided [examples of multi-tenancy][mt], but not explained exactly
+how they relate to Flux's authorisation model. This RFC explains two multi-tenancy implementations,
+their security properties, and how they are implemented within the authorisation model
+as defined in [RFC-0001](https://github.com/fluxcd/flux2/pull/2212).
+
+### Goals
+
+- Explain the mechanisms available in Flux for supporting multi-tenancy
+- Define two models for multi-tenancy, "soft multi-tenancy" and "hard multi-tenancy".
+- Explain when each model is appropriate.
+- Describe a reference implementation of each model with Flux.
+
+### Non-Goals
+
+- Give an exhaustive account of multi-tenancy implementations in general.
+- Provide an [end-to-end workflow][mt] of
+  how to set up multi-tenancy with Flux.
+
+## Introduction
+
+Flux allows different organizations and/or teams to share the same Kubernetes control plane; this is
+referred to as "multi-tenancy". To make this safe, Flux supports segmentation and isolation of
+resources by using namespaces and role-based access control ("RBAC"), and integrating with
+Kubernetes Cluster API.
+
+The following subsections explain the existing mechanisms used for safe multi-tenancy.
+
+### Flux's authorisation model
+
+Flux defers to Kubernetes' native RBAC to specify which operations are authorised when processing
+the custom resources in the Flux API. By default, this means operations are constrained by the
+service account under which the controllers run, which (again, by default) has the `cluster-admin`
+role bound to it. This is convenient for a deployment in which all users are trusted.
+
+In a multi-tenant deployment, each tenant needs to be restricted in the operations that can be done
+on their behalf. Since tenants control Flux via its API objects, this becomes a matter of attaching
+RBAC rules to Flux API objects. There are two mechanisms that do this, "impersonation" and "remote
+apply".
+
+#### Impersonation
+
+The Kustomize controller and Helm controller both apply arbitrary sets of Kubernetes configuration
+to a cluster. These controllers are subject to authorisation on two counts:
+
+ - when accessing Kubernetes resources that are needed for a
+   particular "apply" operation -- for example, a secret referenced in
+   the field `.spec.valuesFrom` in a `HelmRelease`;
+ - when creating, watching, updating and deleting Kubernetes resources
+   in the process of applying a piece of configuration.
+
+To give users control over this authorisation, these two controllers will _impersonate_ (assume the
+identity of) a service account mentioned in the apply specification (e.g., the field
+`.spec.serviceAccountName` in a [`Kustomization` object][kcsa]
+or in a [`HelmRelease` object][hcsa]) for both accessing resources and applying configuration.
+This lets a user constrain the operations mentioned above with RBAC.
+
+As stated in [RFC-0003](https://github.com/fluxcd/flux2/pull/2093),
+the platform admins can configure Flux to enforce service account impersonation
+by setting a default service account name when `.spec.serviceAccountName` is not specified.
+
+#### Remote apply
+
+The Kustomize controller and Helm controller are able to apply a set of configuration to a cluster
+other than the cluster in which they run. If the specification [refers to a secret containing a
+"kubeconfig" file][kubeconfig], the controller will construct a client using that kubeconfig, then
+the client used to apply the specified set of configuration. The effect of this is that the
+configuration will be applied as the user given in the kubeconfig; often this is a user with the
+`cluster-admin` role bound to it, but not necessarily so.
+
+## Assumptions made by the multi-tenancy models
+
+### User Roles
+
+The tenancy models assume two types of user: platform admins and tenants.
+Besides installing Flux, all the other operations (deploy applications, configure ingress, policies, etc)
+do not require users to have direct access to the Kubernetes API. Flux acts as a proxy between users and
+the Kubernetes API, using Git as source of truth for the cluster desired state. Changes to the clusters
+and workloads configuration can be made in a collaborative manner, where the various teams responsible for
+the delivery process propose, review and approve changes via pull request workflows.
+
+#### Platform Admins
+
+The platform admins have unrestricted access to Kubernetes API.
+They are responsible for installing Flux and granting Flux
+access to the sources (Git, Helm, OCI repositories) that make up the cluster(s) control plane desired state.
+The repository(s) owned by the platform admins are reconciled on the cluster(s) by Flux, under
+the [cluster-admin](https://kubernetes.io/docs/reference/access-authn-authz/rbac/#user-facing-roles)
+Kubernetes cluster role.
+
+Example of operations performed by platform admins:
+
+- Bootstrap Flux onto cluster(s).
+- Extend the Kubernetes API with custom resource definitions and validation webhooks.
+- Configure various controllers for ingress, storage, logging, monitoring, progressive delivery, etc.
+- Set up namespaces for tenants and define their level of access with Kubernetes RBAC.
+- Onboard tenants by registering their Git repositories with Flux.
+
+#### Tenants
+
+The tenants have restricted access to the cluster(s) according to the Kubernetes RBAC configured
+by the platform admins. The repositories owned by tenants are reconciled on the cluster(s) by Flux,
+under the Kubernetes account(s) assigned by platform admins.
+
+Example of operations performed by tenants:
+
+- Register their sources with Flux (`GitRepositories`, `HelmRepositories` and `Buckets`).
+- Deploy workload(s) into their namespace(s) using Flux custom resources (`Kustomizations` and `HelmReleases`).
+- Automate application updates using Flux custom resources (`ImageRepositories`, `ImagePolicies` and `ImageUpdateAutomations`).
+- Configure the release pipeline(s) using Flagger custom resources (`Canaries` and `MetricsTemplates`).
+- Setup webhooks and alerting for their release pipeline(s) using Flux custom resources (`Receivers` and `Alerts`).
+
+## Tenancy Models
+
+The Kubernetes tenancy models supported by Flux are: soft multi-tenancy and hard multi-tenancy.
+
+For an overview of the Kubernetes multi-tenant architecture please consult the following documentation:
+
+- [Three Tenancy Models For Kubernetes](https://kubernetes.io/blog/2021/04/15/three-tenancy-models-for-kubernetes/)
+- [GKE multi-tenancy overview](https://cloud.google.com/kubernetes-engine/docs/concepts/multitenancy-overview)
+- [EKS multi-tenancy best practices](https://aws.github.io/aws-eks-best-practices/security/docs/multitenancy/)
+
+### Soft Multi-Tenancy
+
+With soft multi-tenancy, the platform admins use Kubernetes constructs such as namespaces, accounts,
+roles and role bindings to create a logical separation between tenants.
+
+When Flux deploys workloads from a repository belonging to a tenant, it uses the Kubernetes account assigned to that
+tenant to perform the git-to-cluster reconciliation. By leveraging Kubernetes RBAC, Flux can ensure
+that the operations performed by tenants are restricted to their namespaces.
+
+Note that with this model, tenants share cluster-wide resources such as
+`ClusterRoles`, `CustomResourceDefinitions`, `IngressClasses`, `StorageClasses`,
+and they cannot create or alter these resources.
+If a tenant adds a cluster-scoped resource definition to their repository,
+Flux will fail the git-to-cluster reconciliation due to Kubernetes RBAC restrictions.
+
+To restrict the reconciliation of tenant's sources, a Kubernetes service account name can be specified 
+in Flux `Kustomizations` and `HelmReleases` under `.spec.serviceAccountName`. Please consult the Flux  
+documentation for more details:
+
+- [Kustomization API: Role-based access control][kcsa]
+- [HelmRelease API: Role-based access control][hcsa]
+- [Flux multi-tenancy example repository][mt]
+
+Note that with soft multi-tenancy, true tenant isolation requires security measures beyond Kubernetes RBAC.
+Please refer to the Kubernetes [security considerations documentation](https://kubernetes.io/blog/2021/04/15/three-tenancy-models-for-kubernetes/#security-considerations)
+for more details on how to harden shared clusters.
+
+#### Tenants Onboarding
+
+When onboarding tenants, platform admins have the option to assign namespaces, set
+permissions and register the tenants main repositories onto clusters.
+
+The Flux CLI offers an easy way of generating all the Kubernetes manifests needed to onboard tenants:
+
+- `flux create tenant` command generates namespaces, service accounts and Kubernetes RBAC
+  with restricted access to the cluster resources, given tenants access only to their namespaces.
+- `flux create secret git` command generates SSH keys used by Flux to clone the tenants repositories.
+- `flux create source git` command generates the configuration that tells Flux which repositories belong to tenants.
+- `flux create kustomization` command generates the configuration that tells Flux how to reconcile the manifests found in the tenants repositories.
+
+Once the tenants main repositories are registered on the cluster(s), the tenants can configure their app delivery
+in Git using Kubernetes namespace-scoped resources such as `Deployments`, `Services`, Flagger `Canaries`,
+Flux `GitRepositories`, `Kustomizations`, `HelmRepositories`, `HelmReleases`, `ImageUpdateAutomations`,
+`Alerts`, `Receivers`, etc.
+
+### Hard Multi-Tenancy
+
+With hard multi-tenancy, the platform admins create dedicated clusters for each tenant.
+
+When the tenants's clusters are created with Kubernetes Cluster API, the Flux instance
+installed on the management cluster is responsible for reconciling the cluster
+definitions belonging to tenants.
+
+To enable GitOps for the tenant's clusters, the platform admins can configure the Flux instance running on the
+management cluster to connect to the tenant's cluster using the kubeconfig generated by the Cluster API provider
+or by creating kubeconfig secrets for the clusters created by other means than Cluster API.
+
+To configure Flux reconciliation of remote clusters, a Kubernetes secret containing a `kubeConfig` can be specified
+in Flux `Kustomizations` and `HelmReleases` under `.spec.kubeConfig.secretRef`. Please consult the Flux API
+documentation for more details:
+
+- [Kustomization API: Remote Clusters](https://fluxcd.io/docs/components/kustomize/kustomization/#remote-clusters--cluster-api)
+- [HelmRelease API: Remote Clusters](https://fluxcd.io/docs/components/helm/helmreleases/#remote-clusters--cluster-api)
+
+Note that with hard multi-tenancy, tenants have full access to cluster-wide resources, so they have the option
+to manage Flux independently of platform admins, by deploying a Flux instance on each cluster.
+
+#### Caveats
+
+When using a Kubernetes Cluster API provider, the `kubeConfig` secret is automatically generated and Flux can
+make use of it without any manual actions. For clusters created by other means than Cluster API, the
+platform team has to create the `kubeConfig` secrets to allow Flux access to the remote clusters.
+
+As of Flux v0.24 (Nov 2021), we don't provide any guidance for cluster admins on how to generate the `kubeConfig` secrets.
+
+## Implementation History
+
+- Soft multi-tenancy based on service account impersonation was first released in flux2 **v0.0.1**.
+- Generating namespaces and RBAC for defining tenants with `flux create tenant` was first released in flux2 **v0.1.0**.
+- Hard multi-tenancy based on remote cluster reconciliation was first released in flux2 **v0.2.0**.
+- Soft multi-tenancy end-to-end workflow example was first published on 27 Nov 2020 at
+  [fluxcd/flux2-multi-tenancy](https://github.com/fluxcd/flux2-multi-tenancy).
+- Soft multi-tenancy [CVE-2021-41254](https://github.com/fluxcd/kustomize-controller/security/advisories/GHSA-35rf-v2jv-gfg7)
+  "Privilege escalation to cluster admin on multi-tenant environments" was fixed in flux2 **v0.15.0**.
+
+[mt]: https://github.com/fluxcd/flux2-multi-tenancy/tree/v0.1.0
+[kcsa]: https://fluxcd.io/docs/components/kustomize/kustomization/#role-based-access-control
+[hcsa]: https://fluxcd.io/docs/components/helm/helmreleases/#role-based-access-control
+[kubeconfig]: https://fluxcd.io/docs/components/kustomize/api/#kustomize.toolkit.fluxcd.io/v1beta2.KubeConfig

rfcs/README.md

@@ -0,0 +1,45 @@
+# Flux RFCs
+
+In many cases, new features and enhancements are proposed on [flux2/discussions](https://github.com/fluxcd/flux2/discussions).
+A proposal is discussed in public by maintainers, contributors, users and other interested parties.
+After some form of consensus is reached between participants, the proposed changes go through the
+pull request process where the implementation details are reviewed, approved or rejected by maintainers.
+
+Some proposals may be **substantial**, and for these we ask for a design process to be followed
+so that all stakeholders can be confident about the direction Flux is evolving in.
+
+The "RFC" (request for comments) process is intended to provide a consistent and
+controlled path for substantial changes to enter Flux.
+
+Examples of substantial changes:
+
+- API additions (new kinds of resources, new relationships between existing APIs)
+- API breaking changes (new required fields, field removals)
+- Security related changes (Flux controllers permissions, tenant isolation and impersonation)
+- Impactful UX changes (new required inputs to the bootstrap process)
+- Drop capabilities (sunset an existing integration with an external service due to security concerns)
+
+## RFC Process
+
+- Before submitting an RFC please discuss the proposal with the Flux community.
+  Start a discussion on GitHub and ask for feedback at the weekly dev meeting.
+  You must find a maintainer willing to sponsor the RFC.
+- Submit an RFC by opening a pull request using [RFC-0000](RFC-0000/README.md) as template.
+- The sponsor will assign the PR to themselves, will label the PR with `area/RFC` and
+  will request other maintainers to begin the review process.
+- Integrate feedback by adding commits without overriding the history.
+- At least two maintainers have to approve the proposal before it can be merged.
+  Approvers must be satisfied that an
+  [appropriate level of consensus](https://github.com/fluxcd/community/blob/main/GOVERNANCE.md#decision-guidelines)
+  has been reached.
+- Before the merge, an RFC number is assigned by the sponsor and the PR branch must be rebased with main.
+- Once merged, the proposal may be implemented in Flux.
+  The progress could be tracked using the RFC number (used as prefix for issues and PRs).
+- After the proposal implementation is available in a release candidate or final release,
+  the RFC should be updated with the Flux version added to the "Implementation History" section.
+- During the implementation phase, the RFC could be discarded due to security or performance concerns.
+  In this case, the RFC "Implementation History" should state the rejection motives.
+  Ultimately the decision on the feasibility of a particular implementation,
+  resides with the maintainers that reviewed the code changes.
+- A new RFC could be summited with the scope of replacing an RFC rejected during implementation.
+  The new RFC must come with a solution for the rejection motives of the previous RFC.

rfcs/RFC-0000/README.md

@@ -0,0 +1,92 @@
+# RFC-NNNN Title
+
+<!--
+The title must be short and descriptive.
+-->
+
+**Status:** provisional
+
+<!--
+Status represents the current state of the RFC.
+Must be one of `provisional`, `implementable`, `implemented`, `deferred`, `rejected`, `withdrawn`, or `replaced`.
+-->
+
+**Creation date:** YYYY-MM-DD
+
+**Last update:** YYYY-MM-DD
+
+## Summary
+
+<!--
+One paragraph explanation of the proposed feature or enhancement.
+-->
+
+## Motivation
+
+<!--
+This section is for explicitly listing the motivation, goals, and non-goals of
+this RFC. Describe why the change is important and the benefits to users.
+-->
+
+### Goals
+
+<!--
+List the specific goals of this RFC. What is it trying to achieve? How will we
+know that this has succeeded?
+-->
+
+### Non-Goals
+
+<!--
+What is out of scope for this RFC? Listing non-goals helps to focus discussion
+and make progress.
+-->
+
+## Proposal
+
+<!--
+This is where we get down to the specifics of what the proposal actually is.
+This should have enough detail that reviewers can understand exactly what
+you're proposing, but should not include things like API designs or
+implementation.
+
+If the RFC goal is to document best practices,
+then this section can be replaced with the the actual documentation.
+-->
+
+### User Stories
+
+<!--
+Optional if existing discussions and/or issues are linked in the motivation section.
+-->
+
+### Alternatives
+
+<!--
+List plausible alternatives to the proposal and explain why the proposal is superior.
+
+This is a good place to incorporate suggestions made during discussion of the RFC.
+-->
+
+## Design Details
+
+<!--
+This section should contain enough information that the specifics of your
+change are understandable. This may include API specs and code snippets.
+
+The design details should address at least the following questions:
+- How can this feature be enabled / disabled?
+- Does enabling the feature change any default behavior?
+- Can the feature be disabled once it has been enabled?
+- How can an operator determine if the feature is in use?
+- Are there any drawbacks when enabling this feature?
+-->
+
+## Implementation History
+
+<!--
+Major milestones in the lifecycle of the RFC such as:
+- The first Flux release where an initial version of the RFC was available.
+- The version of Flux where the RFC graduated to general availability.
+- The version of Flux where the RFC was retired or superseded.
+-->