Merge pull request #2191 from fluxcd/equinix-runners

Run the ARM64 e2e tests on Equinix hardware

.github/runners/README.md

@@ -1,42 +1,72 @@
-# Flux GitHub runners
+# Flux ARM64 GitHub runners
 
-How to provision GitHub Actions self-hosted runners for Flux conformance testing.
+The Flux ARM64 end-to-end tests run on Equinix instances provisioned with Docker and GitHub self-hosted runners.
 
-## ARM64 Instance specs
+## Current instances
 
-In order to add a new runner to the GitHub Actions pool,
+| Runner        | Instance            | Region |
-first create an instance on Oracle Cloud with the following configuration:
+|---------------|---------------------|--------|
-- OS: Canonical Ubuntu 20.04
+| equinix-arm-1 | flux-equinix-arm-01 | AMS1   |
-- Shape: VM.Standard.A1.Flex
+| equinix-arm-2 | flux-equinix-arm-01 | AMS1   |
-- OCPU Count: 2 
+| equinix-arm-3 | flux-equinix-arm-01 | AMS1   |
-- Memory (GB): 12
+| equinix-arm-4 | flux-equinix-arm-02 | DFW2   |
-- Network Bandwidth (Gbps): 2
+| equinix-arm-5 | flux-equinix-arm-02 | DFW2   |
-- Local Disk: Block Storage Only  
+| equinix-arm-6 | flux-equinix-arm-02 | DFW2   |
+
+## Instance setup
 
-Note that the instance image source must be **Canonical Ubuntu** instead of the default Oracle Linux.
+In order to add a new runner to the GitHub Actions pool,
+first create a server on Equinix with the following configuration:
+- Type: c2.large.arm
+- OS: Ubuntu 20.04
 
-## ARM64 Instance setup
+### Install prerequisites
 
 - SSH into a newly created instance
 ```shell
-ssh ubuntu@<instance-public-IP>
+ssh root@<instance-public-IP>
+``` 
+
+- Create the ubuntu user
+```shell
+adduser ubuntu
+usermod -aG sudo ubuntu
+su - ubuntu
+```
+
+- Create the prerequisites dir
+```shell
+mkdir -p prereq && cd prereq
 ```
-- Create the action runner dir
+
+- Download the prerequisites script
 ```shell
-mkdir -p actions-runner && cd actions-runner
+curl -sL https://raw.githubusercontent.com/fluxcd/flux2/main/.github/runners/prereq.sh > prereq.sh \
+  && chmod +x ./prereq.sh
 ```
-- Download the provisioning script
+
+- Install the prerequisites
 ```shell
-curl -sL https://raw.githubusercontent.com/fluxcd/flux2/main/.github/runners/arm64.sh > arm64.sh \
+sudo ./prereq.sh
-  && chmod +x ./arm64.sh
 ```
+
+### Install runners
+
 - Retrieve the GitHub runner token from the repository [settings page](https://github.com/fluxcd/flux2/settings/actions/runners/new?arch=arm64&os=linux)
-- Run the provisioning script passing the token as the first argument
+
+- Create 3 directories `runner1`, `runner2`, `runner3`
+
+- In each dir run:
 ```shell
-sudo ./arm64.sh <TOKEN>
+curl -sL https://raw.githubusercontent.com/fluxcd/flux2/main/.github/runners/runner-setup.sh > runner-setup.sh \
+  && chmod +x ./runner-setup.sh
+
+./runner-setup.sh equinix-arm-<NUMBER> <TOKEN>
 ```
+
 - Reboot the instance
 ```shell
 sudo reboot
 ```
+
 - Navigate to the GitHub repository [runners page](https://github.com/fluxcd/flux2/settings/actions/runners) and check the runner status

.github/runners/prereq.sh

@@ -14,19 +14,14 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
-# This script is meant to be run locally and in CI to validate the Kubernetes
+# This script installs the prerequisites for running Flux end-to-end tests with Docker and GitHub self-hosted runners.
-# manifests (including Flux custom resources) before changes are merged into
-# the branch synced by Flux in-cluster.
 
 set -eu
 
-REPOSITORY_TOKEN=$1
-REPOSITORY_URL=${2:-https://github.com/fluxcd/flux2}
-
 KIND_VERSION=0.11.1
 KUBECTL_VERSION=1.21.2
 KUSTOMIZE_VERSION=4.1.3
-GITHUB_RUNNER_VERSION=2.278.0
+GITHUB_RUNNER_VERSION=2.285.1
 PACKAGES="apt-transport-https ca-certificates software-properties-common build-essential libssl-dev gnupg lsb-release jq"
 
 # install prerequisites
@@ -64,10 +59,3 @@ curl -o actions-runner-linux-arm64.tar.gz -L https://github.com/actions/runner/r
 
 # install runner dependencies
 ./bin/installdependencies.sh
-
-# register runner with GitHub
-sudo -u ubuntu ./config.sh --unattended --url ${REPOSITORY_URL} --token ${REPOSITORY_TOKEN}
-
-# start runner
-./svc.sh install
-./svc.sh start

.github/runners/runner-setup.sh

@@ -0,0 +1,37 @@
+#!/usr/bin/env bash
+
+# Copyright 2021 The Flux authors. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# This script installs a GitHub self-hosted ARM64 runner for running Flux end-to-end tests.
+
+set -eu
+
+RUNNER_NAME=$1
+REPOSITORY_TOKEN=$2
+REPOSITORY_URL=${3:-https://github.com/fluxcd/flux2}
+
+GITHUB_RUNNER_VERSION=2.285.1
+
+# download runner
+curl -o actions-runner-linux-arm64.tar.gz -L https://github.com/actions/runner/releases/download/v${GITHUB_RUNNER_VERSION}/actions-runner-linux-arm64-${GITHUB_RUNNER_VERSION}.tar.gz \
+  && tar xzf actions-runner-linux-arm64.tar.gz \
+  && rm actions-runner-linux-arm64.tar.gz
+
+# register runner with GitHub
+./config.sh --unattended --url ${REPOSITORY_URL} --token ${REPOSITORY_TOKEN} --name ${RUNNER_NAME}
+
+# start runner
+sudo ./svc.sh install
+sudo ./svc.sh start

.github/workflows/e2e-arm64.yaml

@@ -3,14 +3,13 @@ name: e2e-arm64
 on:
   workflow_dispatch:
   push:
-    branches: [ main, update-components, arm64-e2e ]
+    branches: [ main, update-components, equinix-runners ]
 
 jobs:
-  ampere:
+  test:
-    # Runner info
+    # Hosted on Equinix
-    # Owner: Stefan Prodan
     # Docs: https://github.com/fluxcd/flux2/tree/main/.github/runners
-    runs-on: [self-hosted, Linux, ARM64]
+    runs-on: [self-hosted, Linux, ARM64, equinix]
     steps:
       - name: Checkout
         uses: actions/checkout@v2