flux2/rfcs/kubernetes-oci/README.md

RFC-xxxx Flux OCI support for Kubernetes manifests

Status: provisional

Creation date: 2022-03-31

Last update: 2022-03-31

Summary

Flux should be able to distribute and reconcile Kubernetes configuration packaged as OCI artifacts.

On the client-side, the Flux CLI should offer a command for packaging Kubernetes configs into an OCI artifact and pushing the artifact to a container registry using the Docker config file and the Docker credential helpers for authentication.

On the server-side, the Flux source-controller should offer a dedicated API Kind for defining how OCI artifacts are pulled from container registries and how the artifact's authenticity can be verified. Flux should be able to work with any type of artifact even if it's not created with the Flux CLI.

Motivation

Given that OCI registries are evolving into a generic artifact storage solution, we should extend Flux to allow fetching Kubernetes manifests and related configs from container registries similar to how Flux works with Git and Bucket storage.

With OCI support, Flux users can automate artifact updates to Git in the same way they do today for container images.

Goals

• Add support to the Flux CLI for packaging Kubernetes manifests and related configs into OCI artifacts.
• Add support to Flux source-controller for fetching configs stored as OCI artifacts.
• Make it easy for users to switch from Git repositories and Buckets to OCI repositories.

Non-Goals

• Introduce a new OCI media type for artifacts containing Kubernetes manifests.

Proposal

Push artifacts

Flux users should be able to package a local directory containing Kubernetes configs into a tarball and push the archive to a container registry as an OCI artifact.

flux push artifact docker.io/org/app-config:v1.0.0 -f ./deploy

Flux CLI with produce artifacts of type application/vnd.oci.image.config.v1+json. The directory pointed to by -f is archived and compressed in the tar+gzip format and the layer media type is set to application/vnd.oci.image.layer.v1.tar+gzip.

A proof-of-concept CLI implementation for distributing Kubernetes configs as OCI artifacts is available at kustomizer.dev.

Pull artifacts

Flux users should be able to define a source for pulling manifests inside the cluster from an OCI repository.

apiVersion: source.toolkit.fluxcd.io/v1beta2
kind: OCIRepository
metadata:
  name: app-config
  namespace: flux-system
spec:
  interval: 10m
  url: docker.io/org/app-config
  ref:
    tag: v1.0.0

An OCIRepository can refer to an artifact by tag, digest or semver range:

spec:
  ref:
    # one of
    tag: "latest"
    digest: "sha256:45b23dee08af5e43a7fea6c4cf9c25ccf269ee113168c19722f87876677c5cb2"
    semver: "6.0.x"

For private repositories, the credentials can be supplied with:

spec:
  secretRef:
    name: regcred

The secretRef points to a Kubernetes secret in the same namespace as the OCIRepository, the secret type must be kubernetes.io/dockerconfigjson.

To verify the authenticity of an artifact, the Sigstore cosign public key can be supplied with:

spec:
  verify:
    provider: cosign
    secretRef:
      name: cosign-key

Reconcile artifacts

The OCIRepository can be used as a drop-in replacement for GitRepository and Bucket sources. For example a Flux Kustomization can refer to an OCIRepository and reconcile the manifests found in the OCI artifact:

apiVersion: kustomize.toolkit.fluxcd.io/v1beta2
kind: Kustomization
metadata:
  name: app
  namespace: flux-system
spec:
  interval: 10m
  sourceRef:
    kind: OCIRepository
    name: app-config
  path: ./

User Stories

TODO

Alternatives

TODO

Design Details

TODO

Enabling the feature

The feature is enabled by default.