Mirrors/flux2
1
0
Fork 0
mirror of synced 2026-02-06 19:05:55 +00:00
Code Issues Packages Projects Releases Wiki Activity

Separate image tags howto from migration howto

Browse Source 
The Flux v1 migration how-to flows better if the section on how to set
builds up to tag images in the right way is its own document. It's a
lot to skim past when you don't need it, and (since it's a different
layer of yak hair) something you might want to figure out first if you
do need it.

Signed-off-by: Michael Bridgen <michael@weave.works>
This commit is contained in:
Michael Bridgen
2021-02-08 12:34:47 +00:00
committed by Hidde Beydals
parent 15f8e6369b
commit 2a3a4456c1
2 changed files with 168 additions and 99 deletions
Download Patch File Download Diff File Expand all files Collapse all files

117
docs/guides/flux-v1-automation-migration.md
View File

@@ -300,26 +300,13 @@ example, the prefix is `semver:`:
fluxcd.io/tag.app: semver:^5.0 fluxcd.io/tag.app: semver:^5.0
``` ```
These are the prefixes supported in Flux v1: These are the prefixes supported in Flux v1, and what to use in Flux v2:
| Flux v1 prefix | Meaning | | Flux v1 prefix | Meaning | Flux v2 equivalent |
|------------------|-------------| |------------------|-------------|--------------------|
| `glob:` | Filter for tags matching the glob pattern, then select the newest by build time | | `glob:` | Filter for tags matching the glob pattern, then select the newest by build time | [Use timestamped tags](#how-to-use-timestamps-in-image-tags) |
| `regex:` | Filter for tags matching the regular expression, then select the newest by build time | | `regex:` | Filter for tags matching the regular expression, then select the newest by build time |[Use timestamped tags](#how-to-use-timestamp-in-image-tags) |
| `semver:` | Filter for tags that represent versions, and select the highest version in the given range | | `semver:` | Filter for tags that represent versions, and select the highest version in the given range | [Use semver ordering](#how-to-use-semver-image-tags) |
Flux v2 does not support selecting the lastest image by build time, because that requires the
container config file for each image to be fetched, and this operation is subject to strict rate
limiting by image registries (e.g., by [DockerHub][dockerhub-rates]). The suggested way to select by
image build time is to put a timestamp in each image tag, as explained below.
If you are using ...
| Flux v1 prefix | Flux v2 equivalent |
|-----------------|--------------------|
| glob: | [Use timestamped tags](#how-to-use-timestamps-in-image-tags) |
| regex: | [Use timestamped tags](#how-to-use-timestamp-in-image-tags) |
| semver: | [Use semver ordering](#how-to-use-semver-image-tags) |
#### How to use timestamps in image tags #### How to use timestamps in image tags
@@ -330,84 +317,18 @@ This is a change from Flux v1, in which the build time was fetched from each ima
didn't need to be included in the image tag. Therefore, this is likely to require a change to your didn't need to be included in the image tag. Therefore, this is likely to require a change to your
build process. build process.
##### Example of a build process with timestamp tagging The guide [How to make sortable image tags][image-tags-guide] explains how to change your build
process to tag images with a timestamp. This will mean Flux v2 can sort the tags to find the most
Here is an example of a [GitHub Actions job][gha-syntax] that creates a "build ID" with the git recently built image.
branch, SHA1, and a timestamp, and uses it as a tag when building an image:
```yaml
jobs:
build-push:
env:
IMAGE: org/my-app
runs-on: ubuntu-latest
steps:
- name: Generate build ID
id: prep
run: |
branch=${GITHUB_REF##*/}
sha=${GITHUB_SHA::8}
ts=$(date +%s)
echo "::set-output name=BUILD_ID::${branch}-${sha}-${ts}"
# These are prerequisites for the docker build step
- name: Set up QEMU
uses: docker/setup-qemu-action@v1
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v1
- name: Login to DockerHub
uses: docker/login-action@v1
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
- name: Publish multi-arch container image
uses: docker/build-push-action@v2
with:
push: true
context: .
file: ./Dockerfile
tags: |
${{ env.IMAGE }}:${{ steps.prep.outputs.BUILD_ID }}
```
##### Formats and alternatives
The important properties for sorting alphabetically to work well are that the parts of the timestamp
go from most significant to least (e.g., the year down to the second), and that the output is always
the same number of characters.
Image tags often show in user interfaces, so readability matters. Here are some alternatives:
```bash
$ # seconds-since-epoch (used in the example above)
$ date +%s
1611840548
$ # date and time (remember ':' is not allowed in a tag)
$ date +%F.%H%M%S
2021-01-28.133158
```
Alternatively, you can use a stable serial number as part of the tag. Some CI platforms will
provide a build number in an environment variable, but that may not be reliable to use as a serial
number -- check the platform documentation.
A commit count can be a reasonable stand-in for a serial number, if you build an image per commit,
and you don't rewrite the branch in question.
```bash
$ # commits in branch
$ git --rev-list --count HEAD
1504
```
Beware: this will not give a useful number if you have a shallow clone.
##### Filtering the tags in an `ImagePolicy` ##### Filtering the tags in an `ImagePolicy`
The recommended format for image tags using a timestamp is:
<branch>-<sha1>-<timestamp>
The timestamp (or serial number) is the part of the tag that you want to order on. The SHA1 is there The timestamp (or serial number) is the part of the tag that you want to order on. The SHA1 is there
so you can track an image back to the commit from which it was built. You don't need the branch for so you can trace an image back to the commit from which it was built. You don't need the branch for
sorting, but you may want to include only builds from a specific branch. sorting, but you may want to include only builds from a specific branch.
Say you want to filter for only images that are from `main` branch, and pick the most recent. Your Say you want to filter for only images that are from `main` branch, and pick the most recent. Your
@@ -567,8 +488,8 @@ The value `flux-system:my-app-policy` names the policy that selects the desired
This works in the same way for `DaemonSet` and `CronJob` manifests. For `HelmRelease` manifests, put This works in the same way for `DaemonSet` and `CronJob` manifests. For `HelmRelease` manifests, put
the marker alongside the part of the `values` that has the image tag. If the image tag is a separate the marker alongside the part of the `values` that has the image tag. If the image tag is a separate
field, you can put `:tag` on the end of the name, to replace the value with just the selected field, you can put `:tag` on the end of the name, to replace the value with just the selected
image's tag. The [image automation guide][image-auto-guide] has examples for `HelmRelease` and other image's tag. The [image automation guide][image-update-tute-custom] has examples for `HelmRelease`
custom resources. and other custom resources.
## Controlling automation ## Controlling automation
@@ -708,16 +629,14 @@ spec:
**The Flux v1 migration guide has a branch where you don't put gotk-components.yaml in the repo** **The Flux v1 migration guide has a branch where you don't put gotk-components.yaml in the repo**
[image-update-tute]: https://toolkit.fluxcd.io/guides/image-update/ [image-update-tute]: https://toolkit.fluxcd.io/guides/image-update/
[dockerhub-rates]: https://docs.docker.com/docker-hub/billing/faq/#pull-rate-limiting-faqs
[gha-syntax]: https://docs.github.com/en/actions/reference/workflow-syntax-for-github-actions
[imagepolicy-ref]: https://toolkit.fluxcd.io/components/image/imagepolicies/ [imagepolicy-ref]: https://toolkit.fluxcd.io/components/image/imagepolicies/
[helm-auto]: https://docs.fluxcd.io/en/1.21.1/references/helm-operator-integration/#automated-image-detection). [helm-auto]: https://docs.fluxcd.io/en/1.21.1/references/helm-operator-integration/#automated-image-detection).
[image-auto-guide]: https://toolkit.fluxcd.io/guides/image-update/#configure-image-update-for-custom-resources [image-update-tute-custom]: https://toolkit.fluxcd.io/guides/image-update/#configure-image-update-for-custom-resources
[flux-v1-migration]: ../flux-v1-migration/ [flux-v1-migration]: ../flux-v1-migration/
[install-cli]: https://toolkit.fluxcd.io/get-started/#install-the-flux-cli [install-cli]: https://toolkit.fluxcd.io/get-started/#install-the-flux-cli
[image-auto-guide]: ../image-update/
[flux-bootstrap]: https://toolkit.fluxcd.io/guides/installation/#bootstrap [flux-bootstrap]: https://toolkit.fluxcd.io/guides/installation/#bootstrap
[github-pat]: https://docs.github.com/en/github/authenticating-to-github/creating-a-personal-access-token [github-pat]: https://docs.github.com/en/github/authenticating-to-github/creating-a-personal-access-token
[auto-object-ref]: https://toolkit.fluxcd.io/components/image/imageupdateautomations/ [auto-object-ref]: https://toolkit.fluxcd.io/components/image/imageupdateautomations/
[image-update-tute-creds]: https://toolkit.fluxcd.io/guides/image-update/#configure-image-scanning [image-update-tute-creds]: https://toolkit.fluxcd.io/guides/image-update/#configure-image-scanning
[image-update-tute-clouds]: https://toolkit.fluxcd.io/guides/image-update/#imagerepository-cloud-providers-authenticatio [image-update-tute-clouds]: https://toolkit.fluxcd.io/guides/image-update/#imagerepository-cloud-providers-authenticatio
[image-tags-guide]: ../sortable-image-tags/

150
docs/guides/sortable-image-tags.md Normal file
View File

@@ -0,0 +1,150 @@
<!-- -*- fill-column: 100 -*- -->
# How to make sortable image tags to use with automation
Flux v2 does not support selecting the lastest image by build time. Obtaining the build time needs
the container config for each image, and fetching that is subject to strict rate limiting by image
registries (e.g., by [DockerHub][dockerhub-rates]).
This guide explains how to construct image tags so that the most recent image has the tag that comes
last in alphabetical order. The technique suggested is to put a timestamp or serial number in each
image tag.
## Formats and alternatives
The important properties for sorting alphabetically are that the parts of the timestamp go from most
significant to least (e.g., the year down to the second), and that the output is always the same
number of characters.
Image tags are often shown in user interfaces, so readability matters. Here are some alternatives:
```bash
$ # seconds-since-epoch (used in the example above)
$ date +%s
1611840548
$ # date and time (remember ':' is not allowed in a tag)
$ date +%F.%H%M%S
2021-01-28.133158
```
Alternatively, you can use a stable serial number as part of the tag. Some CI platforms will
provide a build number in an environment variable, but that may not be reliable to use as a serial
number -- check the platform documentation.
A commit count can be a reasonable stand-in for a serial number, if you build an image per commit,
and you don't rewrite the branch in question.
```bash
$ # commits in branch
$ git --rev-list --count HEAD
1504
```
Beware: this will not give a useful number if you have a shallow clone.
### Other things to include in the image tag
It is also handy to quickly trace an image to the branch and commit of its source code. Including
the branch also means you can filter for images from a particular branch.
In sum, a useful tag format is
<branch>-<sha1>-<timestamp>
The branch and tag will usually be made available in a CI platform as environment variables. See
- [CircleCI's built-in variables `CIRCLE_BRANCH` and `CIRCLE_SHA1`][circle-ci-env]
- [GitHub Actions' `GITHUB_REF` and `GITHUB_SHA`][github-actions-env]
- [Travis CI's `TRAVIS_BRANCH` and `TRAVIS_COMMIT`][travis-env].
## Example of a build process with timestamp tagging
Here is an example of a [GitHub Actions job][gha-syntax] that creates a "build ID" with the git
branch, SHA1, and a timestamp, and uses it as a tag when building an image:
```yaml
jobs:
build-push:
env:
IMAGE: org/my-app
runs-on: ubuntu-latest
steps:
- name: Generate build ID
id: prep
run: |
branch=${GITHUB_REF##*/}
sha=${GITHUB_SHA::8}
ts=$(date +%s)
echo "::set-output name=BUILD_ID::${branch}-${sha}-${ts}"
# These are prerequisites for the docker build step
- name: Set up QEMU
uses: docker/setup-qemu-action@v1
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v1
- name: Login to DockerHub
uses: docker/login-action@v1
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
- name: Build and publish container image with tag
uses: docker/build-push-action@v2
with:
push: true
context: .
file: ./Dockerfile
tags: |
${{ env.IMAGE }}:${{ steps.prep.outputs.BUILD_ID }}
```
## Using in an `ImagePolicy` object
When creating an `ImagePolicy` object, you will need to extract just the timestamp part of the tag,
using the `tagFilter` field. You can filter for a particular branch to restrict images to only those
built from that branch.
Here is an example that filters for only images built from `main` branch, and selects the most
recent according the timestamp (created with `date +%s`):
```
apiVersion: image.toolkit.fluxcd.io/v1alpha1
kind: ImagePolicy
metadata:
name: image-repo-policy
namespace: flux-system
spec:
imageRepositoryRef:
name: image-repo
filterTags:
pattern: '^main-[a-f0-9]+-(?P<ts>[0-9]+)'
extract: '$ts'
policy:
alphabetical:
order: asc
```
If you don't care about the branch, that part can be a wildcard in the pattern:
```
apiVersion: image.toolkit.fluxcd.io/v1alpha1
kind: ImagePolicy
metadata:
name: image-repo-policy
namespace: flux-system
spec:
imageRepositoryRef:
name: image-repo
filterTags:
pattern: '^.+-[a-f0-9]+-(?P<ts>[0-9]+)'
extract: '$ts'
policy:
alphabetical:
order: asc
```
[circle-ci-env]: https://circleci.com/docs/2.0/env-vars/#built-in-environment-variables
[github-actions-env]: https://docs.github.com/en/actions/reference/environment-variables#default-environment-variables
[travis-env]: https://docs.travis-ci.com/user/environment-variables/#default-environment-variables
[dockerhub-rates]: https://docs.docker.com/docker-hub/billing/faq/#pull-rate-limiting-faqs
[gha-syntax]: https://docs.github.com/en/actions/reference/workflow-syntax-for-github-actions