Compare commits
4 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
eb2eb004ed | ||
|
|
c3e5b18b8c | ||
|
|
edf15894f8 | ||
|
|
74878a9aef |
9
go.mod
9
go.mod
@@ -14,8 +14,9 @@ require (
|
||||
github.com/fluxcd/notification-controller/api v0.21.0
|
||||
github.com/fluxcd/pkg/apis/kustomize v0.3.1 // indirect
|
||||
github.com/fluxcd/pkg/apis/meta v0.10.2
|
||||
github.com/fluxcd/pkg/kustomize v0.0.2
|
||||
github.com/fluxcd/pkg/runtime v0.12.4
|
||||
github.com/fluxcd/pkg/ssa v0.11.0
|
||||
github.com/fluxcd/pkg/ssa v0.12.0
|
||||
github.com/fluxcd/pkg/ssh v0.3.1
|
||||
github.com/fluxcd/pkg/untar v0.0.5
|
||||
github.com/fluxcd/pkg/version v0.0.1
|
||||
@@ -45,12 +46,8 @@ require (
|
||||
sigs.k8s.io/cli-utils v0.27.0
|
||||
sigs.k8s.io/controller-runtime v0.11.0
|
||||
sigs.k8s.io/kustomize/api v0.10.1
|
||||
sigs.k8s.io/yaml v1.3.0
|
||||
)
|
||||
|
||||
require (
|
||||
github.com/fluxcd/pkg/kustomize v0.0.2
|
||||
sigs.k8s.io/kustomize/kyaml v0.13.0
|
||||
sigs.k8s.io/yaml v1.3.0
|
||||
)
|
||||
|
||||
require (
|
||||
|
||||
4
go.sum
4
go.sum
@@ -245,8 +245,8 @@ github.com/fluxcd/pkg/kustomize v0.0.2/go.mod h1:AFwnf3OqQmpTCuwCARTGpPRMBf0ZFJN
|
||||
github.com/fluxcd/pkg/runtime v0.12.3/go.mod h1:imJ2xYy/d4PbSinX2IefmZk+iS2c1P5fY0js8mCE4SM=
|
||||
github.com/fluxcd/pkg/runtime v0.12.4 h1:gA27RG/+adN2/7Qe03PB46Iwmye/MusPCpuS4zQ2fW0=
|
||||
github.com/fluxcd/pkg/runtime v0.12.4/go.mod h1:gspNvhAqodZgSmK1ZhMtvARBf/NGAlxmaZaIOHkJYsc=
|
||||
github.com/fluxcd/pkg/ssa v0.11.0 h1:ejEMlHPsbXMP8BJQx3+0PwoBgJur0mHiPcMNKcFwoOE=
|
||||
github.com/fluxcd/pkg/ssa v0.11.0/go.mod h1:S+qig7BTOxop0c134y8Yv8/iQST4Kt7S2xXiFkP4VMA=
|
||||
github.com/fluxcd/pkg/ssa v0.12.0 h1:7nF4UigU9Zk/9P/nbzUP3ah8IRC+BpB64O9iu5VnvEo=
|
||||
github.com/fluxcd/pkg/ssa v0.12.0/go.mod h1:S+qig7BTOxop0c134y8Yv8/iQST4Kt7S2xXiFkP4VMA=
|
||||
github.com/fluxcd/pkg/ssh v0.3.1 h1:iQw07bkX2rScactX8WYv+uMDsufFOlg8M3fV2TNs244=
|
||||
github.com/fluxcd/pkg/ssh v0.3.1/go.mod h1:Sebfv4Um51PvomuYdMvKRncQW5dtKhZ5J5TA+wiHNSQ=
|
||||
github.com/fluxcd/pkg/untar v0.0.5 h1:UGI3Ch1UIEIaqQvMicmImL1s9npQa64DJ/ozqHKB7gk=
|
||||
|
||||
206
rfcs/0003-multi-tenancy-mode/README.md
Normal file
206
rfcs/0003-multi-tenancy-mode/README.md
Normal file
@@ -0,0 +1,206 @@
|
||||
# RFC-0003 Flux Multi-Tenancy Mode
|
||||
|
||||
**Status:** provisional
|
||||
|
||||
**Creation date:** 2021-11-16
|
||||
|
||||
**Last update:** 2022-02-03
|
||||
|
||||
## Summary
|
||||
|
||||
For multi-tenant environments, we want to offer an easy way of configuring Flux to enforce tenant isolation
|
||||
(as defined by the Soft Multi-Tenancy model from RFC-0001).
|
||||
|
||||
When running in the multi-tenant mode, Flux will lock down access to sources (as defined by RFC-0002),
|
||||
and will use the tenant service account instead of defaulting to `cluster-admin`.
|
||||
|
||||
From an end-user perspective, the multi-tenancy mode means that:
|
||||
|
||||
- Platform admins have to create a Kubernetes service account and RBAC in each namespace where
|
||||
Flux performs source-to-cluster reconciliation on behalf of tenants.
|
||||
By default, Flux will have no permissions to reconcile the tenants sources onto clusters.
|
||||
- Source owners have to specify with which tenants they wish to share their sources.
|
||||
By default, nothing is shared between tenants.
|
||||
|
||||
## Motivation
|
||||
|
||||
As of [version 0.26](https://github.com/fluxcd/flux2/releases/tag/v0.26.0) (Feb 2022),
|
||||
configuring Flux for soft multi-tenancy requires platform admins to:
|
||||
- Deny cross-namespace access to Flux custom resources by setting the `--no-cross-namespace-refs` flag.
|
||||
- Enforce impersonation by setting a default service account with the `--default-service-account` flag.
|
||||
|
||||
Instead of using a Kustomize patch to lock down Flux as descried in the
|
||||
[multi-tenancy lockdown documentation](https://fluxcd.io/docs/installation/#multi-tenancy-lockdown),
|
||||
we could extend `flux install` and `flux bootstrap` and offer a flag to configure Flux with multi-tenancy enforcements.
|
||||
|
||||
### Goals
|
||||
|
||||
- Enforce service account impersonation for source-to-cluster reconciliation.
|
||||
- Enforce ACLs for cross-namespace access to sources.
|
||||
|
||||
### Non-Goals
|
||||
|
||||
- Enforce tenant's workload isolation with network policies and pod security standards as described
|
||||
[here](https://kubernetes.io/blog/2021/04/15/three-tenancy-models-for-kubernetes/#security-considerations).
|
||||
|
||||
## Proposal
|
||||
|
||||
### User Stories
|
||||
|
||||
#### Story 1
|
||||
|
||||
> As a platform admin, I want to install Flux with lowest privilege/permission level possible.
|
||||
|
||||
#### Story 2
|
||||
|
||||
> As a platform admin, I want to give tenants full control over their assigned namespaces.
|
||||
> So that tenants could use their own repositories and manager the app delivery with Flux.
|
||||
|
||||
#### Story 3
|
||||
|
||||
> As a platform admin, I want to prevent tenants from changing the cluster-wide configuration.
|
||||
> If a tenant adds to their repository a cluster-scoped resource such as a namespace or cluster role,
|
||||
> Flux should reject the change and notify the tenant that this operation is not allowed.
|
||||
|
||||
### Multi-tenant Bootstrap
|
||||
|
||||
When bootstrapping Flux, platform admins should have the option to lock down Flux for multi-tenant environments e.g.:
|
||||
|
||||
```shell
|
||||
flux bootstrap --security-profile=multi-tenant
|
||||
```
|
||||
|
||||
The security profile flag accepts two values: `single-tenant` and `multi-tenant`.
|
||||
Platform admins may switch between the two modes at any time, either by rerunning bootstrap
|
||||
or by patching the Flux manifests in Git.
|
||||
|
||||
The `multi-tenant` profile is just a shortcut to setting the following container args in the Flux deployment manifests:
|
||||
|
||||
```yaml
|
||||
containers:
|
||||
- name: manager
|
||||
args:
|
||||
- --default-service-account=flux
|
||||
- --enable-source-acl=true
|
||||
```
|
||||
|
||||
And for disabling cross-namespace references when using the notification API:
|
||||
|
||||
```yaml
|
||||
kind: Deployment
|
||||
metadata:
|
||||
name: notification-controller
|
||||
spec:
|
||||
template:
|
||||
spec:
|
||||
containers:
|
||||
- name: manager
|
||||
args:
|
||||
- --no-cross-namespace-refs=true
|
||||
```
|
||||
|
||||
When running in the `multi-tenant` mode, Flux behaves differently:
|
||||
|
||||
- The source-to-cluster reconciliation no longer runs under the service account of
|
||||
the Flux controllers. The controller service account, is only used to impersonate
|
||||
the service account specified in the Flux custom resources (`Kustomizations`, `HelmReleases`).
|
||||
- When no service account name is specified in a Flux custom resource,
|
||||
a default will be used e.g. `system:serviceaccount:<tenant-namespace>:flux`.
|
||||
- When a Flux custom resource (`Kustomizations`, `HelmReleases`, `ImagePolicies`, `ImageUpdateAutomations`)
|
||||
refers to a source in a different namespace, access is granted based the source access control list.
|
||||
If no ACL is defined for a source, cross-namespace access is denied.
|
||||
- When a Flux notification (`Alerts`, `Receivers`)
|
||||
refers to a resource in a different namespace, access is denied.
|
||||
|
||||
### Tenants Onboarding
|
||||
|
||||
When onboarding tenants, platform admins should have the option to assign namespaces, set
|
||||
permissions and register the tenants repositories onto clusters in a declarative manner.
|
||||
|
||||
The Flux CLI offers an easy way of generating all the Kubernetes manifests needed to onboard tenants:
|
||||
|
||||
- `flux create tenant` command generates namespaces, service accounts and Kubernetes RBAC
|
||||
with restricted access to the cluster resources, given tenants access only to their namespaces.
|
||||
- `flux create secret git` command generates SSH keys used by Flux to clone the tenants repositories.
|
||||
- `flux create source git` command generates the configuration that tells Flux which repositories belong to tenants.
|
||||
- `flux create kustomization` command generates the configuration that tells Flux how to reconcile the manifests found in the tenants repositories.
|
||||
|
||||
All the above commands have an `--export` flag for generating the Kubernetes resources in YAML format.
|
||||
The platform admins should place the generated manifests in the repository that defines the cluster(s) desired state.
|
||||
|
||||
Here is an example of the generated manifests:
|
||||
|
||||
```yaml
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: Namespace
|
||||
metadata:
|
||||
name: tenant1
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: ServiceAccount
|
||||
metadata:
|
||||
name: flux
|
||||
namespace: tenant1
|
||||
---
|
||||
apiVersion: rbac.authorization.k8s.io/v1
|
||||
kind: RoleBinding
|
||||
metadata:
|
||||
name: flux
|
||||
namespace: tenant1
|
||||
roleRef:
|
||||
apiGroup: rbac.authorization.k8s.io
|
||||
kind: ClusterRole
|
||||
name: cluster-admin
|
||||
subjects:
|
||||
- kind: ServiceAccount
|
||||
name: flux
|
||||
namespace: tenant1
|
||||
---
|
||||
apiVersion: source.toolkit.fluxcd.io/v1beta1
|
||||
kind: GitRepository
|
||||
metadata:
|
||||
name: tenant1
|
||||
namespace: tenant1
|
||||
spec:
|
||||
interval: 5m0s
|
||||
ref:
|
||||
branch: main
|
||||
secretRef:
|
||||
name: tenant1-git-auth
|
||||
url: ssh://git@github.com/org/tenant1
|
||||
---
|
||||
apiVersion: kustomize.toolkit.fluxcd.io/v1beta2
|
||||
kind: Kustomization
|
||||
metadata:
|
||||
name: tenant1
|
||||
namespace: tenant1
|
||||
spec:
|
||||
interval: 10m0s
|
||||
path: ./
|
||||
prune: true
|
||||
serviceAccountName: flux
|
||||
sourceRef:
|
||||
kind: GitRepository
|
||||
name: tenant1
|
||||
```
|
||||
|
||||
Note that the [cluster-admin](https://kubernetes.io/docs/reference/access-authn-authz/rbac/#user-facing-roles)
|
||||
role is used in a `RoleBinding`, this only gives full control over every resource in the role binding's namespace.
|
||||
|
||||
Once the tenants repositories are registered on the cluster(s), the tenants can configure their app delivery
|
||||
in Git using Kubernetes namespace-scoped resources such as `Deployments`, `Services`, Flagger `Canaries`,
|
||||
Flux `Kustomizations`, `HelmReleases`, `ImageUpdateAutomations`, `Alerts`, `Receivers`, etc.
|
||||
|
||||
## Alternatives
|
||||
|
||||
Instead of introducing the security profile flag to `flux bootstrap`,
|
||||
we could document how to patch each controller deployment with Kustomize as described in the
|
||||
[multi-tenancy lockdown documentation](https://fluxcd.io/docs/installation/#multi-tenancy-lockdown).
|
||||
|
||||
Having an easy way of locking down Flux with a single flag, make users aware of the security implications
|
||||
and improves the user experience.
|
||||
|
||||
## Implementation History
|
||||
|
||||
- Disabling cross-namespace access and providing a default service account was first released in flux2 **v0.26.0**.
|
||||
@@ -4,14 +4,14 @@ go 1.17
|
||||
|
||||
require (
|
||||
github.com/Azure/azure-event-hubs-go/v3 v3.3.13
|
||||
github.com/fluxcd/helm-controller/api v0.15.0
|
||||
github.com/fluxcd/image-automation-controller/api v0.19.0
|
||||
github.com/fluxcd/image-reflector-controller/api v0.15.0
|
||||
github.com/fluxcd/kustomize-controller/api v0.19.1
|
||||
github.com/fluxcd/notification-controller/api v0.20.1
|
||||
github.com/fluxcd/helm-controller/api v0.16.0
|
||||
github.com/fluxcd/image-automation-controller/api v0.20.0
|
||||
github.com/fluxcd/image-reflector-controller/api v0.16.0
|
||||
github.com/fluxcd/kustomize-controller/api v0.20.0
|
||||
github.com/fluxcd/notification-controller/api v0.21.0
|
||||
github.com/fluxcd/pkg/apis/meta v0.10.2
|
||||
github.com/fluxcd/pkg/runtime v0.12.3
|
||||
github.com/fluxcd/source-controller/api v0.20.1
|
||||
github.com/fluxcd/pkg/runtime v0.12.4
|
||||
github.com/fluxcd/source-controller/api v0.21.1
|
||||
github.com/hashicorp/terraform-exec v0.14.0
|
||||
github.com/libgit2/git2go/v31 v31.6.1
|
||||
github.com/microsoft/azure-devops-go-api/azuredevops v1.0.0-b5
|
||||
|
||||
@@ -198,26 +198,28 @@ github.com/evanphx/json-patch v4.12.0+incompatible h1:4onqiflcdA9EOZ4RxV643DvftH
|
||||
github.com/evanphx/json-patch v4.12.0+incompatible/go.mod h1:50XU6AFN0ol/bzJsmQLiYLvXMP4fmwYFNcr97nuDLSk=
|
||||
github.com/fatih/color v1.7.0/go.mod h1:Zm6kSWBoL9eyXnKyktHP6abPY2pDugNf5KwzbycvMj4=
|
||||
github.com/felixge/httpsnoop v1.0.1/go.mod h1:m8KPJKqk1gH5J9DgRY2ASl2lWCfGKXixSwevea8zH2U=
|
||||
github.com/fluxcd/helm-controller/api v0.15.0 h1:1uei4JWf5cOEbixj8d5mZ3EMruuarR8yCSiPIsaotKo=
|
||||
github.com/fluxcd/helm-controller/api v0.15.0/go.mod h1:/OeNzk18BVa7UmhGphENJdD/2GerKpMeDSxY8QYlO0o=
|
||||
github.com/fluxcd/image-automation-controller/api v0.19.0 h1:XR2yBR3RxB6i1mS6ZpqgbEnuV23s9q4JfkyKLyOTViU=
|
||||
github.com/fluxcd/image-automation-controller/api v0.19.0/go.mod h1:e9hAvFZT5y1X6NaSNUHXkabpMkPA3Z1bDr3yea8gMzE=
|
||||
github.com/fluxcd/image-reflector-controller/api v0.15.0 h1:2XUKXLhWjbS7X8k1Ur/LJaIv2C8kbpErB46yw4Xmf4U=
|
||||
github.com/fluxcd/image-reflector-controller/api v0.15.0/go.mod h1:SPUqO4kodOglDFpZ+GhW/XBhKo71mWIqFRc+oT0jCfc=
|
||||
github.com/fluxcd/kustomize-controller/api v0.19.1 h1:71E9/7WNQN7aNVhTvfweyOEPwOLVnohAhcR0qMnA67g=
|
||||
github.com/fluxcd/kustomize-controller/api v0.19.1/go.mod h1:q0AA6fxVlm8fvXZEaqSMMw8ANPharpywBve7dlcARhk=
|
||||
github.com/fluxcd/notification-controller/api v0.20.1 h1:iK1+icG0ESuSUki6O9tL/4uxPx6eymYwaFxGprlKSQA=
|
||||
github.com/fluxcd/notification-controller/api v0.20.1/go.mod h1:YFW/YQ6kScEzpnuKgvOJWak+9zGyF3FJ73kKsSQ4LDk=
|
||||
github.com/fluxcd/helm-controller/api v0.16.0 h1:VXlt0EZVHgz2ZX1kUuKQAaQWNj1cC3PIKMRR/0aq07g=
|
||||
github.com/fluxcd/helm-controller/api v0.16.0/go.mod h1:/OeNzk18BVa7UmhGphENJdD/2GerKpMeDSxY8QYlO0o=
|
||||
github.com/fluxcd/image-automation-controller/api v0.20.0 h1:Z+lxqif0KwccsuNOBZq5M96RXCPxtm4Xt8siC1kR6H8=
|
||||
github.com/fluxcd/image-automation-controller/api v0.20.0/go.mod h1:XhLYccGUbmJvTTpJ1jAFKZHr2e1GNXy0T85ZBO50mik=
|
||||
github.com/fluxcd/image-reflector-controller/api v0.16.0 h1:1O1YdoK7LsJgWLyvfZTSbvQcUQCBcgJ573HA0arlQQY=
|
||||
github.com/fluxcd/image-reflector-controller/api v0.16.0/go.mod h1:OIe3mSXc3OwQiNbiQ9vNXWYtNif31hc7WAbZWlFUUnc=
|
||||
github.com/fluxcd/kustomize-controller/api v0.20.0 h1:Vw+2qCxeHMv0y1mfiBgVrMfcfFevBMrRfLEdfPTrb40=
|
||||
github.com/fluxcd/kustomize-controller/api v0.20.0/go.mod h1:5MdpzJVx8+KiDIRv37zLme992BAOCgE0v1n+NOgs1lo=
|
||||
github.com/fluxcd/notification-controller/api v0.21.0 h1:D5B3TH5YtSww0SyvW1Ru5xWsh0MgHQanC/a1t3CvXq0=
|
||||
github.com/fluxcd/notification-controller/api v0.21.0/go.mod h1:gA9/j0kjh7VDuUC2Cubr9twxOdzb/0+ojcE9Lzsc9ug=
|
||||
github.com/fluxcd/pkg/apis/acl v0.0.3 h1:Lw0ZHdpnO4G7Zy9KjrzwwBmDZQuy4qEjaU/RvA6k1lc=
|
||||
github.com/fluxcd/pkg/apis/acl v0.0.3/go.mod h1:XPts6lRJ9C9fIF9xVWofmQwftvhY25n1ps7W9xw0XLU=
|
||||
github.com/fluxcd/pkg/apis/kustomize v0.3.1 h1:wmb5D9e1+Rr3/5O3235ERuj+h2VKUArVfYYk68QKP+w=
|
||||
github.com/fluxcd/pkg/apis/kustomize v0.3.1/go.mod h1:k2HSRd68UwgNmOYBPOd6WbX6a2MH2X/Jeh7e3s3PFPc=
|
||||
github.com/fluxcd/pkg/apis/meta v0.10.2 h1:pnDBBEvfs4HaKiVAYgz+e/AQ8dLvcgmVfSeBroZ/KKI=
|
||||
github.com/fluxcd/pkg/apis/meta v0.10.2/go.mod h1:KQ2er9xa6koy7uoPMZjIjNudB5p4tXs+w0GO6fRcy7I=
|
||||
github.com/fluxcd/pkg/runtime v0.12.3 h1:h21AZ3YG5MAP7DxFF9hfKrP+vFzys2L7CkUbPFjbP/0=
|
||||
github.com/fluxcd/pkg/runtime v0.12.3/go.mod h1:imJ2xYy/d4PbSinX2IefmZk+iS2c1P5fY0js8mCE4SM=
|
||||
github.com/fluxcd/source-controller/api v0.20.1 h1:BfYw1gNHykiCVFNtDz3atcf3Vph+arfuveKmouI98wE=
|
||||
github.com/fluxcd/source-controller/api v0.20.1/go.mod h1:Ab2qDmAUz6ZCp8UaHYLYzxyFrC1FQqEqjxiROb/Rdiw=
|
||||
github.com/fluxcd/pkg/runtime v0.12.4 h1:gA27RG/+adN2/7Qe03PB46Iwmye/MusPCpuS4zQ2fW0=
|
||||
github.com/fluxcd/pkg/runtime v0.12.4/go.mod h1:gspNvhAqodZgSmK1ZhMtvARBf/NGAlxmaZaIOHkJYsc=
|
||||
github.com/fluxcd/source-controller/api v0.21.0/go.mod h1:Ab2qDmAUz6ZCp8UaHYLYzxyFrC1FQqEqjxiROb/Rdiw=
|
||||
github.com/fluxcd/source-controller/api v0.21.1 h1:7X39YQHmB1vmIBrHxU+YAqxwtdC9Zh+tdtMKREW3xiQ=
|
||||
github.com/fluxcd/source-controller/api v0.21.1/go.mod h1:Ab2qDmAUz6ZCp8UaHYLYzxyFrC1FQqEqjxiROb/Rdiw=
|
||||
github.com/flynn/go-shlex v0.0.0-20150515145356-3f9db97f8568/go.mod h1:xEzjJPgXI435gkrCt3MPfRiAkVrwSbHsst4LCFVfpJc=
|
||||
github.com/form3tech-oss/jwt-go v3.2.2+incompatible/go.mod h1:pbq4aXjuKjdthFRnoDwaVPLA+WlJuPGy+QneDUgJi2k=
|
||||
github.com/form3tech-oss/jwt-go v3.2.3+incompatible h1:7ZaBxOI7TMoYBfyA3cQHErNNyAWIKUMIwqxEtgHOs5c=
|
||||
|
||||
Reference in New Issue
Block a user