10 KiB
RFC-NNNN Artifact Revision format and introduction of digests
Status: provisional
Creation date: 2022-10-20
Last update: 2022-10-21
Summary
This RFC proposes to establish a canonical format for an Artifact which
points to a specific checksum (e.g. an OCI manifest digest or Git commit SHA)
of a named pointer (e.g. an OCI image tag or Git tag). In addition, it proposes
to include the algorithm name (e.g. sha256) as a prefix to any advertised
checksum in an Artifact and further referring to it as a Digest as opposed
to a Checksum.
Motivation
The current Artifact type's Revision field format is not "officially"
standardized (albeit assumed throughout our code bases), and has mostly been
derived from GitRepository which uses / as a delimiter between the named
pointer (a Git branch or tag) and a specific (SHA-1, or theoretical SHA-256)
revision.
Since the introduction of OCIRepository and with the recent changes around
HelmChart objects to allow the consumption of charts from OCI registries.
This could be seen as outdated or confusing due to the format differing from
the canonical format used by OCI, which is <tag>@<algo>:<checksum> (the
part after @ formally known as a "digest") to refer to a specific version
of a tagged OCI manifest.
While also taking note that Git does not have an official canonical format for
e.g. branch references at a specific commit, and / has less of a symbolic
meaning than @. Which could be interpreted as "<branch> at
<commit SHA>".
In addition, with the introduction of algorithm prefixes for an Artifact's
checksum. It would be possible to add support and allow user configuration of
other algorithms than SHA-256. For example SHA-384 and SHA-512, or the more
performant (parallelizable) BLAKE3.
Besides this, it would make it easier to implement a client that can verify the checksum without having to resort to an assumed format or guessing method based on the length of it, and allows for a more robust solution in which it can continue to calculate against the algorithm of a previous configuration.
The inclusion of the Artifact's algorithm prefix has been proposed before in
source-controller#855,
with supportive response from Core Maintainers.
Goals
- Establish a canonical format to refer to an
Artifact'sRevisionfield which consists of a named pointer and a specific checksum reference. - Allow easier verification of the
Artifact's checksum by including an alias for the algorithm. - Allow configuration of the algorithm used to calculate the checksum of an
Artifact. - Allow configuration of BLAKE3 as an alternative to SHA for calculating checksums. This has promising performance improvements over SHA-256, which could allow for performance improvements in large scale environments.
- Allow compatability with SemVer name references which might contain an
@symbol already (e.g.package@v1.0.0@sha256:..., as opposed to OCI'stag:v1.0.0@sha256:...).
Non-Goals
- Define a canonical format for an
Artifact'sRevisionfield which contains a named pointer and a different reference than a checksum.
Proposal
Establish an Artifact Revision format
Change the format of the Revision field of the source.toolkit.fluxcd.io
Group's Artifact type across all Source kinds to contain an @ delimiter
opposed to /, and include the algorithm alias as a prefix to the checksum
(creating a "digest").
[ <named pointer> ] [ [ "@" ] <algo> ":" <checksum> ]
Where <named pointer> is the name of e.g. a Git branch or OCI tag,
<checksum> is the exact revision (e.g. a Git commit SHA or OCI manifest
digest), and <algo> is the alias of the algorithm used to calculate the
checksum (e.g. sha256). In case only a named pointer or digest is advertised,
the @ is omitted.
For a GitRepository's Artifact pointing towards an SHA-256 Git commit on
branch main, the Revision field value would become:
main@sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
For a GitRepository's Artifact pointing towards a specific SHA-1 Git commit
without a defined branch or tag, the Revision field value would become:
sha1:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
For a Bucket's Artifact with a revision based on an SHA-256 calculation of
a list of object keys and their etags, the Revision field value would become:
sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
Change the Artifact Checksum to a Digest
Change the format of the Checksum field of the source.toolkit.fluxcd.io
Group's Artifact to Digest, and include the alias of the algorithm used to
calculate the Artifact checksum as a prefix (creating a "digest").
<algo> ":" <checksum>
For a GitRepository Artifact's checksum calculated using SHA-256, the
Digest field value would become:
sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
User Stories
Artifact revision verification
As a user of the source-controller, I want to be able to see the exact revision of an Artifact that is being used, so that I can verify that it matches the expected revision at a remote source.
For a Source kind that has an Artifact with a Revision which contains a
checksum, the field value can be retrieved using the Kubernetes API. For
example:
$ kubectl get gitrepository -o jsonpath='{.status.artifact.revision}' <name>
main@sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
Artifact checksum verification
As a user of the source-controller, I want to be able to verify the checksum of an Artifact.
For a Source kind with an Aritfact the digest consisting of the algorithm
alias and checksum is advertised in the Digest field, and can be retrieved
using the Kubernetes API. For example:
$ kubectl get gitrepository -o jsonpath='{.status.artifact.digest}' <name>
sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
Artifact checksum algorithm configuration
As a user of the source-controller, I want to be able to configure the algorithm used to calculate the checksum of an Artifact.
The source-controller binary accepts a --artifact-digest-algo flag which
configures the algorithm used to calculate the checksum of an Artifact.
The default value is sha256, but can be changed to sha384, sha512
or blake3.
When set, newly advertised Artifact's Digest fields will be calculated
using the configured algorithm. For previous Artifact's that were set using
a previous configuration, the Artifact's Digest field will be calculated
using the advertised algorithm.
Artifact revisions in notifications
As a user of the notification-controller, I want to be able to see the exact revision a notification is referring to.
The notification-controller can use the revision for a Source's Artifact
attached as an annotation to an Event, and correctly parses the value field
when attempting to extract e.g. a Git commit digest from an event for a
GitRepository. As currently already applicable for the / delimiter.
As a user of the notification-controller, I want to be able to observe what commit has been applied on my (supported) Git provider.
The notification-controller can use the revision attached as an annotation to
an Event, and is capable of extracting the correct reference for a Git
provider integration (e.g. GitHub, GitLab) to construct a payload. For example,
extracting e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
from main@sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855.
Artifact revisions in listed views
As a Flux CLI user, I want to see the current revision of my Source in a listed overview.
By running flux get source <kind>, the listed view of Sources would show a
truncated version of the checksum in the Revision field.
$ flux get source gitrepository
NAME REVISION SUSPENDED READY MESSAGE
flux-monitoring main@sha1:6f6c0979 False True stored artifact for revision 'main@sha1:6f6c0979809c12ce4aa687fb42be913f5dc78a75'
$ flux get source oci
NAME REVISION SUSPENDED READY MESSAGE
apps-source local@sha256:b1ad9be6 False True stored artifact for digest 'local@sha256:b1ad9be6fe5fefc76a93f462ef2be1295fa6693d57e9d783780af99cd7234dc8'
$ flux get source bucket
NAME REVISION SUSPENDED READY MESSAGE
apps-source sha256:e3b0c442 False True stored artifact for revision 'sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855'
Alternatives
The two main alternatives around the Revision parts in this RFC are to either
keep the current field value formats as is, or to invent another format. Given
the motivation for this RFC outlines the reasoning for not
keeping the current Revision format, and the proposed is a commonly known
format. Neither strike as a better alternative.
For the changes related to Checksum and Digest, the alternative is to keep
the current field name as is, and only change the field value format. However,
given the naming of the field is more correct with the introduction of the
algorithm alias, and now is the time to make last (breaking) changes to the
API. This does not strike as a better alternative.