1
0
mirror of synced 2026-03-02 11:36:56 +00:00

Compare commits

..

7 Commits

Author SHA1 Message Date
Stefan Prodan
9efc4986f9 Merge pull request #1553 from fluxcd/bootstrap-libgit2-test
Add libgit2 test to bootstrap workflow
2021-06-22 18:49:38 +03:00
Stefan Prodan
fdc366ec1c Increase wait time for the helm e2e test
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2021-06-22 18:37:04 +03:00
Stefan Prodan
d455db444c Add libgit2 test to bootstrap workflow
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2021-06-22 18:24:10 +03:00
Stefan Prodan
17418c9858 Merge pull request #1554 from fluxcd/update-components
Update toolkit components
2021-06-22 18:21:13 +03:00
fluxcdbot
5da7eb7d0d Update toolkit components
- source-controller to v0.15.2
  https://github.com/fluxcd/source-controller/blob/v0.15.2/CHANGELOG.md
- image-automation-controller to v0.13.0
  https://github.com/fluxcd/image-automation-controller/blob/v0.13.0/CHANGELOG.md

Signed-off-by: GitHub <noreply@github.com>
2021-06-22 14:00:42 +00:00
Stefan Prodan
9da977ee7a Merge pull request #1547 from fluxcd/gh-amr64-runner
Add provisioning script for ARM64 GitHub runners
2021-06-21 09:41:44 +03:00
Stefan Prodan
8e4b2ead9b Add provisioning script for ARM64 GitHub runners
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2021-06-19 14:58:57 +03:00
12 changed files with 136 additions and 174 deletions

42
.github/runners/README.md vendored Normal file
View File

@@ -0,0 +1,42 @@
# Flux GitHub runners
How to provision GitHub Actions self-hosted runners for Flux conformance testing.
## ARM64 Instance specs
In order to add a new runner to the GitHub Actions pool,
first create an instance on Oracle Cloud with the following configuration:
- OS: Canonical Ubuntu 20.04
- Shape: VM.Standard.A1.Flex
- OCPU Count: 2
- Memory (GB): 12
- Network Bandwidth (Gbps): 2
- Local Disk: Block Storage Only
Note that the instance image source must be **Canonical Ubuntu** instead of the default Oracle Linux.
## ARM64 Instance setup
- SSH into a newly created instance
```shell
ssh ubuntu@<instance-public-IP>
```
- Create the action runner dir
```shell
mkdir -p actions-runner && cd actions-runner
```
- Download the provisioning script
```shell
curl -sL https://raw.githubusercontent.com/fluxcd/flux2/main/.github/runners/arm64.sh > arm64.sh \
&& chmod +x ./arm64.sh
```
- Retrieve the GitHub runner token from the repository [settings page](https://github.com/fluxcd/flux2/settings/actions/runners/new?arch=arm64&os=linux)
- Run the provisioning script passing the token as the first argument
```shell
sudo ./arm64.sh <TOKEN>
```
- Reboot the instance
```shell
sudo reboot
```
- Navigate to the GitHub repository [runners page](https://github.com/fluxcd/flux2/settings/actions/runners) and check the runner status

73
.github/runners/arm64.sh vendored Executable file
View File

@@ -0,0 +1,73 @@
#!/usr/bin/env bash
# Copyright 2021 The Flux authors. All rights reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# This script is meant to be run locally and in CI to validate the Kubernetes
# manifests (including Flux custom resources) before changes are merged into
# the branch synced by Flux in-cluster.
set -eu
REPOSITORY_TOKEN=$1
REPOSITORY_URL=${2:-https://github.com/fluxcd/flux2}
KIND_VERSION=0.11.1
KUBECTL_VERSION=1.21.2
KUSTOMIZE_VERSION=4.1.3
GITHUB_RUNNER_VERSION=2.278.0
PACKAGES="apt-transport-https ca-certificates software-properties-common build-essential libssl-dev gnupg lsb-release jq"
# install prerequisites
apt-get update \
&& apt-get install -y -q ${PACKAGES} \
&& apt-get clean \
&& rm -rf /var/lib/apt/lists/*
# install docker
curl -fsSL https://get.docker.com -o get-docker.sh \
&& chmod +x get-docker.sh
./get-docker.sh
systemctl enable docker.service
systemctl enable containerd.service
usermod -aG docker ubuntu
# install kind
curl -Lo ./kind https://kind.sigs.k8s.io/dl/v${KIND_VERSION}/kind-linux-arm64
install -o root -g root -m 0755 kind /usr/local/bin/kind
# install kubectl
curl -LO "https://dl.k8s.io/release/v${KUBECTL_VERSION}/bin/linux/arm64/kubectl"
install -o root -g root -m 0755 kubectl /usr/local/bin/kubectl
# install kustomize
curl -Lo ./kustomize.tar.gz https://github.com/kubernetes-sigs/kustomize/releases/download/kustomize%2Fv${KUSTOMIZE_VERSION}/kustomize_v${KUSTOMIZE_VERSION}_linux_arm64.tar.gz \
&& tar -zxvf kustomize.tar.gz \
&& rm kustomize.tar.gz
install -o root -g root -m 0755 kustomize /usr/local/bin/kustomize
# download runner
curl -o actions-runner-linux-arm64.tar.gz -L https://github.com/actions/runner/releases/download/v${GITHUB_RUNNER_VERSION}/actions-runner-linux-arm64-${GITHUB_RUNNER_VERSION}.tar.gz \
&& tar xzf actions-runner-linux-arm64.tar.gz \
&& rm actions-runner-linux-arm64.tar.gz
# install runner dependencies
./bin/installdependencies.sh
# register runner with GitHub
sudo -u ubuntu ./config.sh --unattended --url ${REPOSITORY_URL} --token ${REPOSITORY_TOKEN}
# start runner
./svc.sh install
./svc.sh start

View File

@@ -61,6 +61,13 @@ jobs:
--team=team-z
env:
GITHUB_TOKEN: ${{ secrets.GITPROVIDER_BOT_TOKEN }}
- name: libgit2
run: |
/tmp/flux create source git test-libgit2 \
--url=ssh://git@github.com/fluxcd-testing/${{ steps.vars.outputs.test_repo_name }} \
--git-implementation=libgit2 \
--secret-ref=flux-system \
--branch=main
- name: uninstall
run: |
/tmp/flux uninstall -s --keep-namespace

View File

@@ -9,9 +9,7 @@ jobs:
ampere:
# Runner info
# Owner: Stefan Prodan
# VM: Oracle Cloud VM.Standard.A1.Flex 4CPU 24GB RAM
# OS: Linux 5.4.0-1045-oracle #49-Ubuntu SMP aarch64
# Packages: docker, kind, kubectl, kustomize
# Docs: https://github.com/fluxcd/flux2/tree/main/.github/runners
runs-on: [self-hosted, Linux, ARM64]
steps:
- name: Checkout

View File

@@ -193,7 +193,7 @@ jobs:
/tmp/flux create kustomization flux-system \
--source=flux-system \
--path=./clusters/staging
kubectl -n flux-system wait kustomization/apps --for=condition=ready --timeout=2m
kubectl -n flux-system wait kustomization/apps --for=condition=ready --timeout=5m
- name: flux check
run: |
/tmp/flux check

View File

@@ -1,39 +0,0 @@
/*
Copyright 2021 The Flux authors
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package main
import (
"github.com/spf13/cobra"
)
var encryptCmd = &cobra.Command{
Use: "encrypt",
Short: "Encrypt secrets using SOPS",
Long: "The encrypt sub-commands initialise and manage Secret encryption using SOPS.",
}
type encryptFlags struct {
export bool
}
var encryptArgs encryptFlags
func init() {
encryptCmd.PersistentFlags().BoolVar(&encryptArgs.export, "export", false, "export in YAML format to stdout")
rootCmd.AddCommand(encryptCmd)
}

View File

@@ -1,113 +0,0 @@
/*
Copyright 2021 The Flux authors
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package main
import (
"context"
"fmt"
"io/ioutil"
"os"
"path/filepath"
"filippo.io/age"
"github.com/fluxcd/flux2/internal/utils"
"github.com/go-git/go-git/v5"
"github.com/spf13/cobra"
corev1 "k8s.io/api/core/v1"
v1 "k8s.io/apimachinery/pkg/apis/meta/v1"
)
var encryptInitCmd = &cobra.Command{
Use: "init",
Short: "Init SOPS encryption with age identity",
Long: "The encryption init command creates a new age identity and writes a .sops.yaml file to the current working directory.",
Example: ` # Init SOPS encryption with a new age identity
flux encryption init`,
RunE: encryptInitCmdRun,
}
func init() {
encryptCmd.AddCommand(encryptInitCmd)
}
func encryptInitCmdRun(cmd *cobra.Command, args []string) error {
// Confirm our current path is in a Git repository
path, err := os.Getwd()
if err != nil {
return err
}
if _, err := git.PlainOpen(path); err != nil {
if err == git.ErrRepositoryNotExists {
err = fmt.Errorf("'%s' is not in a Git repository", path)
}
return err
}
// Abort early if .sops.yaml already exists
sopsCfgPath := filepath.Join(path, ".sops.yaml")
if _, err := os.Stat(sopsCfgPath); err == nil || os.IsExist(err) {
return fmt.Errorf("'%s' already contains a .sops.yaml config", path)
}
// Generate a new identity
i, err := age.GenerateX25519Identity()
if err != nil {
return err
}
logger.Successf("Generated identity %s", i.Recipient().String())
// Attempt to configure identity in .sops.yaml
const sopsCfg = `creation_rules:
- path_regex: .*.yaml
encrypted_regex: ^(data|stringData)$
age: %s
`
if err := ioutil.WriteFile(sopsCfgPath, []byte(fmt.Sprintf(sopsCfg, i.Recipient().String())), 0644); err != nil {
logger.Failuref("Failed to write recipient to .sops.yaml file")
return err
}
logger.Successf("Configured recipient in .sops.yaml file")
// Init client
ctx, cancel := context.WithTimeout(context.Background(), rootArgs.timeout)
defer cancel()
kubeClient, err := utils.KubeClient(rootArgs.kubeconfig, rootArgs.kubecontext)
if err != nil {
return err
}
// Create a secret
secret := &corev1.Secret{
ObjectMeta: v1.ObjectMeta{
Name: "sops-age",
Namespace: rootArgs.namespace,
},
StringData: map[string]string{
"flux-auto.age": i.String(),
},
}
if err := kubeClient.Create(ctx, secret); err != nil {
return err
}
logger.Successf(`Secret '%s' with private key created`, secret.Name)
// TODO(hidde): lookup kustomize based on path ref? Do direct cluster mutation? (Preferably not!)
// Feels something is missing in general to provide a user experience improving bridge between "die hard"
// `--export` and "please do not do this" direct-apply-to-cluster.
return nil
}

5
go.mod
View File

@@ -3,12 +3,11 @@ module github.com/fluxcd/flux2
go 1.16
require (
filippo.io/age v1.0.0-rc.3
github.com/Masterminds/semver/v3 v3.1.0
github.com/cyphar/filepath-securejoin v0.2.2
github.com/fluxcd/go-git-providers v0.1.1
github.com/fluxcd/helm-controller/api v0.11.1
github.com/fluxcd/image-automation-controller/api v0.12.0
github.com/fluxcd/image-automation-controller/api v0.13.0
github.com/fluxcd/image-reflector-controller/api v0.10.0
github.com/fluxcd/kustomize-controller/api v0.13.0
github.com/fluxcd/notification-controller/api v0.15.0
@@ -17,7 +16,7 @@ require (
github.com/fluxcd/pkg/ssh v0.0.5
github.com/fluxcd/pkg/untar v0.0.5
github.com/fluxcd/pkg/version v0.0.1
github.com/fluxcd/source-controller/api v0.15.1
github.com/fluxcd/source-controller/api v0.15.2
github.com/go-git/go-git/v5 v5.4.2
github.com/google/go-containerregistry v0.2.0
github.com/manifoldco/promptui v0.7.0

13
go.sum
View File

@@ -24,9 +24,6 @@ cloud.google.com/go/storage v1.0.0/go.mod h1:IhtSnM/ZTZV8YYJWCY8RULGVqBDmpoyjwiy
cloud.google.com/go/storage v1.5.0/go.mod h1:tpKbwo567HUNpVclU5sGELwQWBDZ8gh0ZeosJ0Rtdos=
cloud.google.com/go/storage v1.6.0/go.mod h1:N7U0C8pVQ/+NIKOBQyamJIeKQKkZ+mxpohlUTyfDhBk=
dmitri.shuralyov.com/gpu/mtl v0.0.0-20190408044501-666a987793e9/go.mod h1:H6x//7gZCb22OMCxBHrMx7a5I7Hp++hsVxbQ4BYO7hU=
filippo.io/age v1.0.0-rc.3 h1:8JjuJ5ffGKDmC4SS0zoyQxZROZX75so768b7AjulKLw=
filippo.io/age v1.0.0-rc.3/go.mod h1:UjINLBMeA60aGZkHCGsmDzKcaXoTTzpvrqQM+Vo3YHU=
filippo.io/edwards25519 v1.0.0-beta.3/go.mod h1:X+pm78QAUPtFLi1z9PYIlS/bdDnvbCOGKtZ+ACWEf7o=
github.com/Azure/azure-sdk-for-go v35.0.0+incompatible/go.mod h1:9XXNKU+eRnpl9moKnB4QOLf1HestfXbmab5FXxiDBjc=
github.com/Azure/azure-sdk-for-go v38.0.0+incompatible/go.mod h1:9XXNKU+eRnpl9moKnB4QOLf1HestfXbmab5FXxiDBjc=
github.com/Azure/azure-sdk-for-go v42.3.0+incompatible/go.mod h1:9XXNKU+eRnpl9moKnB4QOLf1HestfXbmab5FXxiDBjc=
@@ -206,8 +203,8 @@ github.com/fluxcd/go-git-providers v0.1.1 h1:R4VafMOo1IlfEZcImApCeElge/HajhFvRzD
github.com/fluxcd/go-git-providers v0.1.1/go.mod h1:nRgNpHZmZhrsyNSma1JcAhjUG9xrqMGJcIUr9K7M7vk=
github.com/fluxcd/helm-controller/api v0.11.1 h1:7oanAnhcRdqrnALDGpZcg5iuDqwCv+jNTmGvSrgDyCo=
github.com/fluxcd/helm-controller/api v0.11.1/go.mod h1:nt5YdVS+jWXDSbP3gAX3HII6oX+8ahrHD6og2ZVsnN4=
github.com/fluxcd/image-automation-controller/api v0.12.0 h1:TM37onbW9vqcwPZgcavrXka9OF+JmK2+m69V8GYRWIM=
github.com/fluxcd/image-automation-controller/api v0.12.0/go.mod h1:f4KeYUdaQD8hfANsEOAdIxoloIBIY7UUHKM99h/swqg=
github.com/fluxcd/image-automation-controller/api v0.13.0 h1:o9ezxdkbCaxmbj3fQX0L8yJu8T2CJTKHN+aBT+JYGQA=
github.com/fluxcd/image-automation-controller/api v0.13.0/go.mod h1:WDBKSufHazCf065F9qPWtVPUN8kU4MdM5rVC1MoZtpk=
github.com/fluxcd/image-reflector-controller/api v0.10.0 h1:uZsf5X0Djko2yJMNdKn516nKCiSBFecE8c+O66lrgyI=
github.com/fluxcd/image-reflector-controller/api v0.10.0/go.mod h1:X4qZ11pfA5w1ajbkYbWmQ3hBW3gzCyIjhU87AvV6o2A=
github.com/fluxcd/kustomize-controller/api v0.13.0 h1:bAbaNinlgxu7xflQHR/DLyf3u8V98pyT+U+d1+VM67c=
@@ -227,9 +224,8 @@ github.com/fluxcd/pkg/untar v0.0.5 h1:UGI3Ch1UIEIaqQvMicmImL1s9npQa64DJ/ozqHKB7g
github.com/fluxcd/pkg/untar v0.0.5/go.mod h1:O6V9+rtl8c1mHBafgqFlJN6zkF1HS5SSYn7RpQJ/nfw=
github.com/fluxcd/pkg/version v0.0.1 h1:/8asQoDXSThz3csiwi4Qo8Zb6blAxLXbtxNgeMJ9bCg=
github.com/fluxcd/pkg/version v0.0.1/go.mod h1:WAF4FEEA9xyhngF8TDxg3UPu5fA1qhEYV8Pmi2Il01Q=
github.com/fluxcd/source-controller/api v0.14.0/go.mod h1:P1pIkaoIsiCJ/NLC7IBXPb9XEime9NvA1WN4hZu2Of4=
github.com/fluxcd/source-controller/api v0.15.1 h1:ajJHCP3e3AST13nOJrM2ax2mqTS2vSjj7jkedhbMds0=
github.com/fluxcd/source-controller/api v0.15.1/go.mod h1:P1pIkaoIsiCJ/NLC7IBXPb9XEime9NvA1WN4hZu2Of4=
github.com/fluxcd/source-controller/api v0.15.2 h1:MRhh/MScEdGqGS/dYff8iEF2XhNr3ruvhGjTYnWT3ag=
github.com/fluxcd/source-controller/api v0.15.2/go.mod h1:P1pIkaoIsiCJ/NLC7IBXPb9XEime9NvA1WN4hZu2Of4=
github.com/flynn/go-shlex v0.0.0-20150515145356-3f9db97f8568/go.mod h1:xEzjJPgXI435gkrCt3MPfRiAkVrwSbHsst4LCFVfpJc=
github.com/form3tech-oss/jwt-go v3.2.2+incompatible h1:TcekIExNqud5crz4xD2pavyTgWiPvpYe4Xau31I0PRk=
github.com/form3tech-oss/jwt-go v3.2.2+incompatible/go.mod h1:pbq4aXjuKjdthFRnoDwaVPLA+WlJuPGy+QneDUgJi2k=
@@ -802,7 +798,6 @@ golang.org/x/crypto v0.0.0-20191206172530-e9b2fee46413/go.mod h1:LzIPMQfyMNhhGPh
golang.org/x/crypto v0.0.0-20200220183623-bac4c82f6975/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
golang.org/x/crypto v0.0.0-20201002170205-7f63de1d35b0/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
golang.org/x/crypto v0.0.0-20201221181555-eec23a3978ad/go.mod h1:jdWPYTVW3xRLrWPugEBEK3UY2ZEsg3UU495nc5E+M+I=
golang.org/x/crypto v0.0.0-20210220033148-5ea612d1eb83/go.mod h1:jdWPYTVW3xRLrWPugEBEK3UY2ZEsg3UU495nc5E+M+I=
golang.org/x/crypto v0.0.0-20210322153248-0c34fe9e7dc2/go.mod h1:T9bdIzuCu7OtxOm1hfPfRQxPLYneinmdGuTeoZ9dtd4=
golang.org/x/crypto v0.0.0-20210421170649-83a5a9bb288b h1:7mWr3k41Qtv8XlltBkDkl8LoP3mpSgBW8BUoxtEdbXg=

View File

@@ -1,8 +1,8 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- https://github.com/fluxcd/image-automation-controller/releases/download/v0.12.0/image-automation-controller.crds.yaml
- https://github.com/fluxcd/image-automation-controller/releases/download/v0.12.0/image-automation-controller.deployment.yaml
- https://github.com/fluxcd/image-automation-controller/releases/download/v0.13.0/image-automation-controller.crds.yaml
- https://github.com/fluxcd/image-automation-controller/releases/download/v0.13.0/image-automation-controller.deployment.yaml
- account.yaml
patchesJson6902:
- target:

View File

@@ -1,8 +1,8 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- https://github.com/fluxcd/source-controller/releases/download/v0.15.1/source-controller.crds.yaml
- https://github.com/fluxcd/source-controller/releases/download/v0.15.1/source-controller.deployment.yaml
- https://github.com/fluxcd/source-controller/releases/download/v0.15.2/source-controller.crds.yaml
- https://github.com/fluxcd/source-controller/releases/download/v0.15.2/source-controller.deployment.yaml
- account.yaml
patchesJson6902:
- target:

View File

@@ -1,9 +1,9 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- https://github.com/fluxcd/source-controller/releases/download/v0.15.1/source-controller.crds.yaml
- https://github.com/fluxcd/source-controller/releases/download/v0.15.2/source-controller.crds.yaml
- https://github.com/fluxcd/kustomize-controller/releases/download/v0.13.0/kustomize-controller.crds.yaml
- https://github.com/fluxcd/helm-controller/releases/download/v0.11.1/helm-controller.crds.yaml
- https://github.com/fluxcd/notification-controller/releases/download/v0.15.0/notification-controller.crds.yaml
- https://github.com/fluxcd/image-reflector-controller/releases/download/v0.10.0/image-reflector-controller.crds.yaml
- https://github.com/fluxcd/image-automation-controller/releases/download/v0.12.0/image-automation-controller.crds.yaml
- https://github.com/fluxcd/image-automation-controller/releases/download/v0.13.0/image-automation-controller.crds.yaml