Add default value of githubWebhookServer.syncPeriod to chart (#622)

* Add default value of `githubWebhookServer.syncPeriod` to chart

* Bump chart version

* Update README.md

.dockerignore

@@ -0,0 +1,12 @@
+Makefile
+acceptance
+runner
+hack
+test-assets
+config
+charts
+.github
+.envrc
+*.md
+*.txt
+*.sh

.github/ISSUE_TEMPLATE/bug_report.md

@@ -0,0 +1,36 @@
+---
+name: Bug report
+about: Create a report to help us improve
+title: ''
+assignees: ''
+
+---
+
+**Describe the bug**
+A clear and concise description of what the bug is.
+
+**Checks**
+
+- [ ] My actions-runner-controller version (v0.x.y) does support the feature
+- [ ] I'm using an unreleased version of the controller I built from HEAD of the default branch
+
+**To Reproduce**
+Steps to reproduce the behavior:
+1. Go to '...'
+2. Click on '....'
+3. Scroll down to '....'
+4. See error
+
+**Expected behavior**
+A clear and concise description of what you expected to happen.
+
+**Screenshots**
+If applicable, add screenshots to help explain your problem.
+
+**Environment (please complete the following information):**
+ - Controller Version [e.g. 0.18.2]
+ - Deployment Method [e.g. Helm and Kustomize ]
+ - Helm Chart Version [e.g. 0.11.0, if applicable]
+
+**Additional context**
+Add any other context about the problem here.

.github/ISSUE_TEMPLATE/feature_request.md

@@ -0,0 +1,19 @@
+---
+name: Feature request
+about: Suggest an idea for this project
+title: ''
+assignees: ''
+
+---
+
+**Is your feature request related to a problem? Please describe.**
+A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
+
+**Describe the solution you'd like**
+A clear and concise description of what you want to happen.
+
+**Describe alternatives you've considered**
+A clear and concise description of any alternative solutions or features you've considered.
+
+**Additional context**
+Add any other context or screenshots about the feature request here.

.github/RELEASE_NOTE_TEMPLATE.md

@@ -0,0 +1,34 @@
+# Release Note Template
+
+This is the template of actions-runner-controller's release notes.
+
+Whenever a new release is made, I start by manually copy-pasting this template onto the GitHub UI for creating the release.
+
+I then walk-through all the changes, take sometime to think abount best one-sentence explanations to tell the users about changes, write it all,
+and click the publish button.
+
+If you think you can improve future release notes in any way, please do submit a pull request to change the template below.
+
+Note that even though it looks like a Go template, I don't use any templating to generate the changelog.
+It's just that I'm used to reading and intepreting Go template by myself, not a computer program :)
+
+**Title**:
+
+```
+v{{ .Version }}: {{ .TitlesOfImportantChanges }}
+```
+
+**Body**:
+
+```
+**CAUTION:** If you're using the Helm chart, beware to review changes to CRDs and do manually upgrade CRDs! Helm installs CRDs only on installing a chart. It doesn't automatically upgrade CRDs. Otherwise you end up with troubles like #427, #467, and #468. Please refer to the [UPGRADING](charts/actions-runner-controller/docs/UPGRADING.md) docs for the latest process.
+
+This release includes the following changes from contributors. Thank you!
+
+- @{{ .GitHubUser }} fixed {{ .Feature }} to not break when ... (#{{ .PullRequestNumber }})
+- @{{ .GitHubUser }} enhanced {{ .Feature }} to ... (#{{ .PullRequestNumber }})
+- @{{ .GitHubUser }} added {{ .Feature }} for ... (#{{ .PullRequestNumber }})
+- @{{ .GitHubUser }} fixed {{ .Topic }} in the documentation so that ... (#{{ .PullRequestNumber }})
+- @{{ .GitHubUser }} added {{ .Topic }} to the documentation (#{{ .PullRequestNumber }})
+- @{{ .GitHubUser }} improved the documentation about {{ .Topic }} to also cover ... (#{{ .PullRequestNumber }})
+```

.github/stale.yml

@@ -0,0 +1,66 @@
+# Configuration for probot-stale - https://github.com/probot/stale
+
+# Number of days of inactivity before an Issue or Pull Request becomes stale
+daysUntilStale: 30
+
+# Number of days of inactivity before an Issue or Pull Request with the stale label is closed.
+# Set to false to disable. If disabled, issues still need to be closed manually, but will remain marked as stale.
+daysUntilClose: 14
+
+# Only issues or pull requests with all of these labels are check if stale. Defaults to `[]` (disabled)
+onlyLabels: []
+
+# Issues or Pull Requests with these labels will never be considered stale. Set to `[]` to disable
+exemptLabels:
+  - pinned
+  - security
+  - enhancement
+  - refactor
+  - documentation
+  - chore
+  - needs-investigation
+  - bug
+
+# Set to true to ignore issues in a project (defaults to false)
+exemptProjects: false
+
+# Set to true to ignore issues in a milestone (defaults to false)
+exemptMilestones: false
+
+# Set to true to ignore issues with an assignee (defaults to false)
+exemptAssignees: false
+
+# Label to use when marking as stale
+staleLabel: stale
+
+# Comment to post when marking as stale. Set to `false` to disable
+markComment: >
+  This issue has been automatically marked as stale because it has not had
+  recent activity. It will be closed if no further activity occurs. Thank you
+  for your contributions.
+
+# Comment to post when removing the stale label.
+# unmarkComment: >
+#   Your comment here.
+
+# Comment to post when closing a stale Issue or Pull Request.
+# closeComment: >
+#   Your comment here.
+
+# Limit the number of actions per hour, from 1-30. Default is 30
+limitPerRun: 30
+
+# Limit to only `issues` or `pulls`
+# only: issues
+
+# Optionally, specify configuration settings that are specific to just 'issues' or 'pulls':
+# pulls:
+#   daysUntilStale: 30
+#   markComment: >
+#     This pull request has been automatically marked as stale because it has not had
+#     recent activity. It will be closed if no further activity occurs. Thank you
+#     for your contributions.
+
+# issues:
+#   exemptLabels:
+#     - confirmed

.github/workflows/build-and-release-runners.yml

@@ -6,32 +6,38 @@ on:
       - '**'
     paths:
       - 'runner/**'
-      - .github/workflows/build-runner.yml
+      - .github/workflows/build-and-release-runners.yml
   push:
     branches:
       - master
     paths:
       - runner/patched/*
       - runner/Dockerfile
-      - runner/dindrunner.Dockerfile
+      - runner/Dockerfile.ubuntu.1804
+      - runner/Dockerfile.dindrunner
       - runner/entrypoint.sh
-      - .github/workflows/build-runner.yml
+      - .github/workflows/build-and-release-runners.yml
 
 jobs:
   build:
     runs-on: ubuntu-latest
-    name: Build ${{ matrix.name }}
+    name: Build ${{ matrix.name }}-ubuntu-${{ matrix.os-version }}
     strategy:
       matrix:
         include:
           - name: actions-runner
+            os-version: 20.04
             dockerfile: Dockerfile
+          - name: actions-runner
+            os-version: 18.04
+            dockerfile: Dockerfile.ubuntu.1804
           - name: actions-runner-dind
-            dockerfile: dindrunner.Dockerfile
+            os-version: 20.04
+            dockerfile: Dockerfile.dindrunner
     env:
-      RUNNER_VERSION: 2.276.1
+      RUNNER_VERSION: 2.278.0
       DOCKER_VERSION: 19.03.12
-      DOCKERHUB_USERNAME: ${{ github.repository_owner }}
+      DOCKERHUB_USERNAME: ${{ secrets.DOCKER_USER }}
     steps:
       - name: Set outputs
         id: vars
@@ -50,21 +56,68 @@ jobs:
 
       - name: Login to DockerHub
         uses: docker/login-action@v1
-        if: ${{ github.event_name == 'push' }}
+        if: ${{ github.event_name == 'push' || github.event_name == 'release' }}
         with:
-          username: ${{ github.repository_owner }}
+          username: ${{ secrets.DOCKER_USER }}
           password: ${{ secrets.DOCKER_ACCESS_TOKEN }}
 
-      - name: Build and Push
+      - name: Build and Push Versioned Tags
         uses: docker/build-push-action@v2
         with:
           context: ./runner
           file: ./runner/${{ matrix.dockerfile }}
           platforms: linux/amd64,linux/arm64
+          push: ${{ github.event_name != 'pull_request' }}
+          build-args: |
+            RUNNER_VERSION=${{ env.RUNNER_VERSION }}
+            DOCKER_VERSION=${{ env.DOCKER_VERSION }}
+          tags: |
+            ${{ env.DOCKERHUB_USERNAME }}/${{ matrix.name }}:v${{ env.RUNNER_VERSION }}-ubuntu-${{ matrix.os-version }}
+            ${{ env.DOCKERHUB_USERNAME }}/${{ matrix.name }}:v${{ env.RUNNER_VERSION }}-ubuntu-${{ matrix.os-version }}-${{ steps.vars.outputs.sha_short }}
+
+  latest-tags:
+    if: ${{ github.event_name == 'push' || github.event_name == 'release' }}
+    runs-on: ubuntu-latest
+    name: Build ${{ matrix.name }}-latest
+    strategy:
+      matrix:
+        include:
+          - name: actions-runner
+            dockerfile: Dockerfile
+          - name: actions-runner-dind
+            dockerfile: Dockerfile.dindrunner
+    env:
+      RUNNER_VERSION: 2.277.1
+      DOCKER_VERSION: 19.03.12
+      DOCKERHUB_USERNAME: ${{ secrets.DOCKER_USER }}
+    steps:
+
+      - name: Checkout
+        uses: actions/checkout@v2
+
+      - name: Set up QEMU
+        uses: docker/setup-qemu-action@v1
+
+      - name: Set up Docker Buildx
+        uses: docker/setup-buildx-action@v1
+        with:
+          version: latest
+
+      - name: Login to DockerHub
+        uses: docker/login-action@v1
+        with:
+          username: ${{ secrets.DOCKER_USER }}
+          password: ${{ secrets.DOCKER_ACCESS_TOKEN }}
+
+      - name: Build and Push Latest Tag
+        uses: docker/build-push-action@v2
+        with:
+          context: ./runner
+          file: ./runner/${{ matrix.dockerfile }}
+          platforms: linux/amd64,linux/arm64
+          push: true
           build-args: |
             RUNNER_VERSION=${{ env.RUNNER_VERSION }}
             DOCKER_VERSION=${{ env.DOCKER_VERSION }}
           tags: |
-            ${{ env.DOCKERHUB_USERNAME }}/${{ matrix.name }}:v${{ env.RUNNER_VERSION }}
-            ${{ env.DOCKERHUB_USERNAME }}/${{ matrix.name }}:v${{ env.RUNNER_VERSION }}-${{ steps.vars.outputs.sha_short }}
             ${{ env.DOCKERHUB_USERNAME }}/${{ matrix.name }}:latest

.github/workflows/on-push-lint-charts.yml

@@ -4,9 +4,11 @@ on:
   push:
     paths:
       - 'charts/**'
+      - '!charts/actions-runner-controller/docs/**'
+      - '!charts/actions-runner-controller/*.md'
       - '.github/**'
+      - '!.github/*.md'
   workflow_dispatch:
-
 env:
   KUBE_SCORE_VERSION: 1.10.0
   HELM_VERSION: v3.4.1

.github/workflows/on-push-master-publish-chart.yml

@@ -7,7 +7,9 @@ on:
       - main # assume that the branch name may change in future
     paths:
       - 'charts/**'
+      - '!charts/actions-runner-controller/docs/**'
       - '.github/**'
+      - '!**.md'
   workflow_dispatch:
 
 env:

.github/workflows/release.yml

@@ -7,7 +7,7 @@ jobs:
     runs-on: ubuntu-latest
     name: Release
     env:
-      DOCKERHUB_USERNAME: ${{ github.repository_owner }}
+      DOCKERHUB_USERNAME: ${{ secrets.DOCKER_USER }}
     steps:
       - name: Set outputs
         id: vars
@@ -47,7 +47,7 @@ jobs:
       - name: Login to DockerHub
         uses: docker/login-action@v1
         with:
-          username: ${{ github.repository_owner }}
+          username: ${{ secrets.DOCKER_USER }}
           password: ${{ secrets.DOCKER_ACCESS_TOKEN }}
 
       - name: Build and Push

.github/workflows/test.yaml

@@ -6,6 +6,9 @@ on:
       - master
     paths-ignore:
     - 'runner/**'
+    - .github/workflows/build-and-release-runners.yml
+    - '*.md'
+    - '.gitignore'
 
 jobs:
   test:

.github/workflows/wip.yml

@@ -4,13 +4,15 @@ on:
       - master
     paths-ignore:
       - "runner/**"
+      - "**.md"
+      - ".gitignore"
 
 jobs:
   build:
     runs-on: ubuntu-latest
     name: release-latest
     env:
-      DOCKERHUB_USERNAME: ${{ github.repository_owner }}
+      DOCKERHUB_USERNAME: ${{ secrets.DOCKER_USER }}
     steps:
       - name: Checkout
         uses: actions/checkout@v2
@@ -27,7 +29,7 @@ jobs:
       - name: Login to DockerHub
         uses: docker/login-action@v1
         with:
-          username: ${{ github.repository_owner }}
+          username: ${{ secrets.DOCKER_USER }}
           password: ${{ secrets.DOCKER_ACCESS_TOKEN }}
 
       # Considered unstable builds

CONTRIBUTING.md

@@ -0,0 +1,8 @@
+# Contributing
+
+### Helm Version Bumps
+
+**Chart Version :** When bumping the chart version follow semantic versioning https://semver.org/<br />
+**App Version :** When bumping the app version you will also need to bump the chart version too. Again, follow semantic versioning when bumping the chart.
+
+To determine if you need to bump the MAJOR, MINOR or PATCH versions you will need to review the changes between the previous app version and the new app version and / or ask for a maintainer to advise.

Makefile

@@ -1,5 +1,17 @@
-NAME ?= summerwind/actions-runner-controller
+ifdef DOCKER_USER
+	NAME ?= ${DOCKER_USER}/actions-runner-controller
+else
+	NAME ?= summerwind/actions-runner-controller
+endif
+DOCKER_USER ?= $(shell echo ${NAME} | cut -d / -f1)
 VERSION ?= latest
+RUNNER_NAME ?= ${DOCKER_USER}/actions-runner
+RUNNER_TAG  ?= ${VERSION}
+TEST_REPO ?= ${DOCKER_USER}/actions-runner-controller
+TEST_ORG ?=
+TEST_ORG_REPO ?=
+SYNC_PERIOD ?= 5m
+
 # From https://github.com/VictoriaMetrics/operator/pull/44
 YAML_DROP=$(YQ) delete --inplace
 YAML_DROP_PREFIX=spec.validation.openAPIV3Schema.properties.spec.properties
@@ -14,6 +26,8 @@ else
 GOBIN=$(shell go env GOBIN)
 endif
 
+TEST_ASSETS=$(PWD)/test-assets
+
 # default list of platforms for which multiarch image is built
 ifeq (${PLATFORMS}, )
 	export PLATFORMS="linux/amd64,linux/arm64"
@@ -22,8 +36,8 @@ endif
 # if IMG_RESULT is unspecified, by default the image will be pushed to registry
 ifeq (${IMG_RESULT}, load)
 	export PUSH_ARG="--load"
-    # if load is specified, image will be built only for the build machine architecture.
-    export PLATFORMS="local"
+	# if load is specified, image will be built only for the build machine architecture.
+	export PLATFORMS="local"
 else ifeq (${IMG_RESULT}, cache)
 	# if cache is specified, image will only be available in the build cache, it won't be pushed or loaded
 	# therefore no PUSH_ARG will be specified
@@ -37,6 +51,13 @@ all: manager
 test: generate fmt vet manifests
 	go test ./... -coverprofile cover.out
 
+test-with-deps: kube-apiserver etcd kubectl
+	# See https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/envtest#pkg-constants
+	TEST_ASSET_KUBE_APISERVER=$(KUBE_APISERVER_BIN) \
+	TEST_ASSET_ETCD=$(ETCD_BIN) \
+	TEST_ASSET_KUBECTL=$(KUBECTL_BIN) \
+	  make test
+
 # Build manager binary
 manager: generate fmt vet
 	go build -o bin/manager main.go
@@ -96,12 +117,9 @@ generate: controller-gen
 	$(CONTROLLER_GEN) object:headerFile=./hack/boilerplate.go.txt paths="./..."
 
 # Build the docker image
-docker-build: test
+docker-build:
 	docker build . -t ${NAME}:${VERSION}
-
-# Push the docker image
-docker-push:
-	docker push ${NAME}:${VERSION}
+	docker build runner -t ${RUNNER_NAME}:${RUNNER_TAG} --build-arg TARGETPLATFORM=$(shell arch)
 
 docker-buildx:
 	export DOCKER_CLI_EXPERIMENTAL=enabled
@@ -115,6 +133,11 @@ docker-buildx:
 		-f Dockerfile \
 		. ${PUSH_ARG}
 
+# Push the docker image
+docker-push:
+	docker push ${NAME}:${VERSION}
+	docker push ${RUNNER_NAME}:${RUNNER_TAG}
+
 # Generate the release manifest file
 release: manifests
 	cd config/manager && kustomize edit set image controller=${NAME}:${VERSION}
@@ -126,19 +149,41 @@ release/clean:
 	rm -rf release
 
 .PHONY: acceptance
-acceptance: release/clean docker-build docker-push release
-	ACCEPTANCE_TEST_SECRET_TYPE=token make acceptance/kind acceptance/setup acceptance/tests acceptance/teardown
-	ACCEPTANCE_TEST_SECRET_TYPE=app make acceptance/kind acceptance/setup acceptance/tests acceptance/teardown
-	ACCEPTANCE_TEST_DEPLOYMENT_TOOL=helm ACCEPTANCE_TEST_SECRET_TYPE=token make acceptance/kind acceptance/setup acceptance/tests acceptance/teardown
-	ACCEPTANCE_TEST_DEPLOYMENT_TOOL=helm ACCEPTANCE_TEST_SECRET_TYPE=app make acceptance/kind acceptance/setup acceptance/tests acceptance/teardown
+acceptance: release/clean acceptance/pull docker-build release
+	ACCEPTANCE_TEST_SECRET_TYPE=token make acceptance/run
+	ACCEPTANCE_TEST_SECRET_TYPE=app make acceptance/run
+	ACCEPTANCE_TEST_DEPLOYMENT_TOOL=helm ACCEPTANCE_TEST_SECRET_TYPE=token make acceptance/run
+	ACCEPTANCE_TEST_DEPLOYMENT_TOOL=helm ACCEPTANCE_TEST_SECRET_TYPE=app make acceptance/run
+
+acceptance/run: acceptance/kind acceptance/load acceptance/setup acceptance/deploy acceptance/tests acceptance/teardown
 
 acceptance/kind:
-	kind create cluster --name acceptance
+	kind create cluster --name acceptance --config acceptance/kind.yaml
+
+# Set TMPDIR to somewhere under $HOME when you use docker installed with Ubuntu snap
+# Otherwise `load docker-image` fail while running `docker save`.
+# See https://kind.sigs.k8s.io/docs/user/known-issues/#docker-installed-with-snap
+acceptance/load:
+	kind load docker-image ${NAME}:${VERSION} --name acceptance
+	kind load docker-image quay.io/brancz/kube-rbac-proxy:v0.10.0 --name acceptance
+	kind load docker-image ${RUNNER_NAME}:${RUNNER_TAG} --name acceptance
+	kind load docker-image docker:dind --name acceptance
+	kind load docker-image quay.io/jetstack/cert-manager-controller:v1.0.4 --name acceptance
+	kind load docker-image quay.io/jetstack/cert-manager-cainjector:v1.0.4 --name acceptance
+	kind load docker-image quay.io/jetstack/cert-manager-webhook:v1.0.4 --name acceptance
 	kubectl cluster-info --context kind-acceptance
 
+# Pull the docker images for acceptance
+acceptance/pull:
+	docker pull quay.io/brancz/kube-rbac-proxy:v0.10.0
+	docker pull docker:dind
+	docker pull quay.io/jetstack/cert-manager-controller:v1.0.4
+	docker pull quay.io/jetstack/cert-manager-cainjector:v1.0.4
+	docker pull quay.io/jetstack/cert-manager-webhook:v1.0.4
+
 acceptance/setup:
 	kubectl apply --validate=false -f https://github.com/jetstack/cert-manager/releases/download/v1.0.4/cert-manager.yaml	#kubectl create namespace actions-runner-system
-	kubectl -n cert-manager wait deploy/cert-manager-cainjector --for condition=available --timeout 60s
+	kubectl -n cert-manager wait deploy/cert-manager-cainjector --for condition=available --timeout 90s
 	kubectl -n cert-manager wait deploy/cert-manager-webhook --for condition=available --timeout 60s
 	kubectl -n cert-manager wait deploy/cert-manager --for condition=available --timeout 60s
 	kubectl create namespace actions-runner-system || true
@@ -148,8 +193,12 @@ acceptance/setup:
 acceptance/teardown:
 	kind delete cluster --name acceptance
 
-acceptance/tests:
+acceptance/deploy:
+	NAME=${NAME} DOCKER_USER=${DOCKER_USER} VERSION=${VERSION} RUNNER_NAME=${RUNNER_NAME} RUNNER_TAG=${RUNNER_TAG} TEST_REPO=${TEST_REPO} \
+	TEST_ORG=${TEST_ORG} TEST_ORG_REPO=${TEST_ORG_REPO} SYNC_PERIOD=${SYNC_PERIOD} \
 	acceptance/deploy.sh
+
+acceptance/tests:
 	acceptance/checks.sh
 
 # Upload release file to GitHub.
@@ -191,3 +240,77 @@ ifeq (, $(wildcard $(GOBIN)/yq))
 	}
 endif
 YQ=$(GOBIN)/yq
+
+OS_NAME := $(shell uname -s | tr A-Z a-z)
+
+# find or download etcd
+etcd:
+ifeq (, $(shell which etcd))
+ifeq (, $(wildcard $(TEST_ASSETS)/etcd))
+	@{ \
+	set -xe ;\
+	INSTALL_TMP_DIR=$$(mktemp -d) ;\
+	cd $$INSTALL_TMP_DIR ;\
+	wget https://github.com/kubernetes-sigs/kubebuilder/releases/download/v2.3.2/kubebuilder_2.3.2_$(OS_NAME)_amd64.tar.gz ;\
+	mkdir -p $(TEST_ASSETS) ;\
+	tar zxvf kubebuilder_2.3.2_$(OS_NAME)_amd64.tar.gz ;\
+	mv kubebuilder_2.3.2_$(OS_NAME)_amd64/bin/etcd $(TEST_ASSETS)/etcd ;\
+	mv kubebuilder_2.3.2_$(OS_NAME)_amd64/bin/kube-apiserver $(TEST_ASSETS)/kube-apiserver ;\
+	mv kubebuilder_2.3.2_$(OS_NAME)_amd64/bin/kubectl $(TEST_ASSETS)/kubectl ;\
+	rm -rf $$INSTALL_TMP_DIR ;\
+	}
+ETCD_BIN=$(TEST_ASSETS)/etcd
+else
+ETCD_BIN=$(TEST_ASSETS)/etcd
+endif
+else
+ETCD_BIN=$(shell which etcd)
+endif
+
+# find or download kube-apiserver
+kube-apiserver:
+ifeq (, $(shell which kube-apiserver))
+ifeq (, $(wildcard $(TEST_ASSETS)/kube-apiserver))
+	@{ \
+	set -xe ;\
+	INSTALL_TMP_DIR=$$(mktemp -d) ;\
+	cd $$INSTALL_TMP_DIR ;\
+	wget https://github.com/kubernetes-sigs/kubebuilder/releases/download/v2.3.2/kubebuilder_2.3.2_$(OS_NAME)_amd64.tar.gz ;\
+	mkdir -p $(TEST_ASSETS) ;\
+	tar zxvf kubebuilder_2.3.2_$(OS_NAME)_amd64.tar.gz ;\
+	mv kubebuilder_2.3.2_$(OS_NAME)_amd64/bin/etcd $(TEST_ASSETS)/etcd ;\
+	mv kubebuilder_2.3.2_$(OS_NAME)_amd64/bin/kube-apiserver $(TEST_ASSETS)/kube-apiserver ;\
+	mv kubebuilder_2.3.2_$(OS_NAME)_amd64/bin/kubectl $(TEST_ASSETS)/kubectl ;\
+	rm -rf $$INSTALL_TMP_DIR ;\
+	}
+KUBE_APISERVER_BIN=$(TEST_ASSETS)/kube-apiserver
+else
+KUBE_APISERVER_BIN=$(TEST_ASSETS)/kube-apiserver
+endif
+else
+KUBE_APISERVER_BIN=$(shell which kube-apiserver)
+endif
+
+# find or download kubectl
+kubectl:
+ifeq (, $(shell which kubectl))
+ifeq (, $(wildcard $(TEST_ASSETS)/kubectl))
+	@{ \
+	set -xe ;\
+	INSTALL_TMP_DIR=$$(mktemp -d) ;\
+	cd $$INSTALL_TMP_DIR ;\
+	wget https://github.com/kubernetes-sigs/kubebuilder/releases/download/v2.3.2/kubebuilder_2.3.2_$(OS_NAME)_amd64.tar.gz ;\
+	mkdir -p $(TEST_ASSETS) ;\
+	tar zxvf kubebuilder_2.3.2_$(OS_NAME)_amd64.tar.gz ;\
+	mv kubebuilder_2.3.2_$(OS_NAME)_amd64/bin/etcd $(TEST_ASSETS)/etcd ;\
+	mv kubebuilder_2.3.2_$(OS_NAME)_amd64/bin/kube-apiserver $(TEST_ASSETS)/kube-apiserver ;\
+	mv kubebuilder_2.3.2_$(OS_NAME)_amd64/bin/kubectl $(TEST_ASSETS)/kubectl ;\
+	rm -rf $$INSTALL_TMP_DIR ;\
+	}
+KUBECTL_BIN=$(TEST_ASSETS)/kubectl
+else
+KUBECTL_BIN=$(TEST_ASSETS)/kubectl
+endif
+else
+KUBECTL_BIN=$(shell which kubectl)
+endif

README.md

@@ -8,25 +8,28 @@ ToC:
 
 - [Motivation](#motivation)
 - [Installation](#installation)
-  - [GitHub Enterprise support](#github-enterprise-support)
-- [Setting up authentication with GitHub API](#setting-up-authentication-with-github-api)
-  - [Using GitHub App](#using-github-app)
-  - [Using Personal AccessToken ](#using-personal-access-token)
+  - [GitHub Enterprise Support](#github-enterprise-support)
+- [Setting Up Authentication with GitHub API](#setting-up-authentication-with-github-api)
+  - [Deploying Using GitHub App Authentication](#deploying-using-github-app-authentication)
+  - [Deploying Using PAT Authentication](#deploying-using-pat-authentication)
 - [Usage](#usage)
   - [Repository Runners](#repository-runners)
   - [Organization Runners](#organization-runners)
+  - [Enterprise Runners](#enterprise-runners)
   - [Runner Deployments](#runnerdeployments)
-    - [Autoscaling](#autoscaling)
-      - [Faster Autoscaling with GitHub Webhook](#faster-autoscaling-with-github-webhook)
+    - [Note on scaling to/from 0](#note-on-scaling-tofrom-0)
+  - [Autoscaling](#autoscaling)
+    - [Faster Autoscaling with GitHub Webhook](#faster-autoscaling-with-github-webhook)
+    - [Autoscaling to/from 0](#autoscaling-tofrom-0)
+    - [Scheduled Overrides](#scheduled-overrides)
   - [Runner with DinD](#runner-with-dind)
-  - [Additional tweaks](#additional-tweaks)
-  - [Runner labels](#runner-labels)
-  - [Runer groups](#runner-groups)
-  - [Using EKS IAM role for service accounts](#using-eks-iam-role-for-service-accounts)
-  - [Software installed in the runner image](#software-installed-in-the-runner-image)
-  - [Common errors](#common-errors)
-- [Developing](#developing)
-- [Alternatives](#alternatives)
+  - [Additional Tweaks](#additional-tweaks)
+  - [Runner Labels](#runner-labels)
+  - [Runner Groups](#runner-groups)
+  - [Using IRSA (IAM Roles for Service Accounts) in EKS](#using-irsa-iam-roles-for-service-accounts-in-eks)
+  - [Software Installed in the Runner Image](#software-installed-in-the-runner-image)
+  - [Common Errors](#common-errors)
+- [Contributing](#contributing)
 
 ## Motivation
 
@@ -42,94 +45,83 @@ actions-runner-controller uses [cert-manager](https://cert-manager.io/docs/insta
 
 Install the custom resource and actions-runner-controller with `kubectl` or `helm`. This will create actions-runner-system namespace in your Kubernetes and deploy the required resources.
 
-`kubectl`:
+**Kubectl Deployment:**
 
-```
-# REPLACE "v0.16.1" with the latest release
-kubectl apply -f https://github.com/summerwind/actions-runner-controller/releases/download/v0.16.1/actions-runner-controller.yaml
+```shell
+# REPLACE "v0.18.2" with the version you wish to deploy
+kubectl apply -f https://github.com/actions-runner-controller/actions-runner-controller/releases/download/v0.18.2/actions-runner-controller.yaml
 ```
 
-`helm`:
+**Helm Deployment:**
 
-```
-helm repo add actions-runner-controller https://summerwind.github.io/actions-runner-controller
-helm upgrade --install -n actions-runner-system actions-runner-controller/actions-runner-controller
+__**Note: For all configuration options for the Helm chart see the chart's [README](./charts/actions-runner-controller/README.md)
+
+```shell
+helm repo add actions-runner-controller https://actions-runner-controller.github.io/actions-runner-controller
+helm upgrade --install --namespace actions-runner-system --create-namespace \ 
+             --wait actions-runner-controller actions-runner-controller/actions-runner-controller
 ```
 
-### Github Enterprise support
+### GitHub Enterprise Support
 
-If you use either Github Enterprise Cloud or Server, you can use **actions-runner-controller**  with those, too.
-Authentication works same way as with public Github (repo and organization level).
-The minimum version of Github Enterprise Server is 3.0.0 (or rc1/rc2).
-In most cases maintainers do not have environment where to test changes and are reliant on the community for testing.
+The solution supports both GitHub Enterprise Cloud and Server editions as well as regular GitHub. Both PAT (personal access token) and GitHub App authentication works for installations that will be deploying either repository level and / or organization level runners. If you need to deploy enterprise level runners then you are restricted to PAT based authentication as GitHub doesn't support GitHub App based authentication for enterprise runners currently.
 
+If you are deploying this solution into a GitHub Enterprise Server environment then you will need version >= [3.0.0](https://docs.github.com/en/enterprise-server@3.0/admin/release-notes#3.0.0).
+
+When deploying the solution for a GitHub Enterprise Server environment you need to provide an additional environment variable as part of the controller deployment:
 
 ```shell
 kubectl set env deploy controller-manager -c manager GITHUB_ENTERPRISE_URL=<GHEC/S URL> --namespace actions-runner-system
 ```
 
-#### Enterprise runners usage
+__**Note: The repository maintainers do not have an enterprise environment (cloud or server). Support for the enterprise specific feature set is community driven and on a best effort basis. PRs from the community are welcomed to add features and maintain support.**__
 
-In order to use enterprise runners you must have Admin access to Github Enterprise and you should do Personal Access Token (PAT)
-with `enterprise:admin` access. Enterprise runners are not possible to run with Github APP or any other permission.
+## Setting Up Authentication with GitHub API
 
-When you use enterprise runners those will get access to Github Organisations. However, access to the repositories is **NOT**
-allowed by default. Each Github Organisation must allow Enterprise runner groups to be used in repositories.
-This is needed only one time and is permanent after that.
+There are two ways for actions-runner-controller to authenticate with the GitHub API (only 1 can be configured at a time however):
 
-Example:
+1. Using a GitHub App (not supported for enterprise level runners due to lack of support from GitHub)
+2. Using a PAT
 
-```yaml
-apiVersion: actions.summerwind.dev/v1alpha1
-kind: RunnerDeployment
-metadata:
-  name: ghe-runner-deployment
-spec:
-  replicas: 2
-  template:
-    spec:
-      enterprise: your-enterprise-name
-      dockerdWithinRunnerContainer: true
-      resources:
-        limits:
-          cpu: "4000m"
-          memory: "2Gi"
-        requests:
-          cpu: "200m"
-          memory: "200Mi"
-      volumeMounts:
-      - mountPath: /runner
-        name: runner
-      volumes:
-      - name: runner
-        emptyDir: {}
+Functionality wise, there isn't much of a difference between the 2 authentication methods. The primarily benefit of authenticating via a GitHub App is an [increased API quota](https://docs.github.com/en/developers/apps/rate-limits-for-github-apps).
 
-```
+If you are deploying the solution for a GitHub Enterprise Server environment you are able to [configure your rate limiting settings](https://docs.github.com/en/enterprise-server@3.0/admin/configuration/configuring-rate-limits) making the main benefit irrelevant. If you're deploying the solution for a GitHub Enterprise Cloud or regular GitHub environment and you run into rate limiting issues, consider deploying the solution using the GitHub App authentication method instead.
 
-## Setting up authentication with GitHub API
+### Deploying Using GitHub App Authentication
 
-There are two ways for actions-runner-controller to authenticate with the GitHub API:
+You can create a GitHub App for either your user account or any organization, below are the app permissions required for each supported type of runner:
 
-1. Using GitHub App.
-2. Using Personal Access Token.
+_Note: Links are provided further down to create an app for your logged in user account or an organisation with the permissions for all runner types set in each link's query string_
 
-Regardless of which authentication method you use, the same permissions are required, those permissions are:
-- Repository: Administration (read/write)
-- Repository: Actions (read)
-- Organization: Self-hosted runners (read/write)
+**Required Permissions for Repository Runners:**<br />
+**Repository Permissions**
 
+* Actions (read)
+* Administration (read / write)
+* Metadata (read)
 
-**NOTE: It is extremely important to only follow one of the sections below and not both.**
+**Required Permissions for Organisation Runners:**<br />
+**Repository Permissions**
 
-### Using GitHub App
+* Actions (read)
+* Metadata (read)
 
-You can create a GitHub App for either your account or any organization. If you want to create a GitHub App for your account, open the following link to the creation page, enter any unique name in the "GitHub App name" field, and hit the "Create GitHub App" button at the bottom of the page.
+**Organization Permissions**
+* Self-hosted runners (read / write)
 
-- [Create GitHub Apps on your account](https://github.com/settings/apps/new?url=http://github.com/summerwind/actions-runner-controller&webhook_active=false&public=false&administration=write&actions=read)
+_Note: All API routes mapped to their permissions can be found [here](https://docs.github.com/en/rest/reference/permissions-required-for-github-apps) if you wish to review_
+
+---
+
+**Setup Steps**
+
+If you want to create a GitHub App for your account, open the following link to the creation page, enter any unique name in the "GitHub App name" field, and hit the "Create GitHub App" button at the bottom of the page.
+
+- [Create GitHub Apps on your account](https://github.com/settings/apps/new?url=http://github.com/actions-runner-controller/actions-runner-controller&webhook_active=false&public=false&administration=write&actions=read)
 
 If you want to create a GitHub App for your organization, replace the `:org` part of the following URL with your organization name before opening it. Then enter any unique name in the "GitHub App name" field, and hit the "Create GitHub App" button at the bottom of the page to create a GitHub App.
 
-- [Create GitHub Apps on your organization](https://github.com/organizations/:org/settings/apps/new?url=http://github.com/summerwind/actions-runner-controller&webhook_active=false&public=false&administration=write&organization_self_hosted_runners=write&actions=read)
+- [Create GitHub Apps on your organization](https://github.com/organizations/:org/settings/apps/new?url=http://github.com/actions-runner-controller/actions-runner-controller&webhook_active=false&public=false&administration=write&organization_self_hosted_runners=write&actions=read)
 
 You will see an *App ID* on the page of the GitHub App you created as follows, the value of this App ID will be used later.
 
@@ -148,8 +140,11 @@ When the installation is complete, you will be taken to a URL in one of the foll
 - `https://github.com/settings/installations/${INSTALLATION_ID}`
 - `https://github.com/organizations/eventreactor/settings/installations/${INSTALLATION_ID}`
 
+
 Finally, register the App ID (`APP_ID`), Installation ID (`INSTALLATION_ID`), and downloaded private key file (`PRIVATE_KEY_FILE_PATH`) to Kubernetes as Secret.
 
+**Kubectl Deployment:**
+
 ```shell
 $ kubectl create secret generic controller-manager \
     -n actions-runner-system \
@@ -158,19 +153,41 @@ $ kubectl create secret generic controller-manager \
     --from-file=github_app_private_key=${PRIVATE_KEY_FILE_PATH}
 ```
 
-### Using Personal Access Token
+**Helm Deployment:**
 
-From an account that has `admin` privileges for the repository, create a [personal access token](https://github.com/settings/tokens) with `repo` scope. This token is used to register a self-hosted runner by *actions-runner-controller*.
+Configure your values.yaml, see the chart's [README](./charts/actions-runner-controller/README.md) for deploying the secret via Helm
 
-Self-hosted runners in GitHub can either be connected to a single repository, or to a GitHub organization (so they are available to all repositories in the organization). This token is used to register a self-hosted runner by *actions-runner-controller*.
+### Deploying Using PAT Authentication
 
-For adding a runner to a repository, the token should have `repo` scope. If the runner should be added to an organization, the token should have `admin:org` scope. Note that to use a Personal Access Token, you must issue the token with an account that has `admin` privileges (on the repository and/or the organization).
+Personal Access Tokens can be used to register a self-hosted runner by *actions-runner-controller*.
 
-Open the Create Token page from the following link, grant the `repo` and/or `admin:org` scope, and press the "Generate Token" button at the bottom of the page to create the token.
+Log-in to a GitHub account that has `admin` privileges for the repository, and [create a personal access token](https://github.com/settings/tokens/new) with the appropriate scopes listed below:
 
-- [Create personal access token](https://github.com/settings/tokens/new)
+**Required Scopes for Repository Runners**
 
-Register the created token (`GITHUB_TOKEN`) as a Kubernetes secret.
+* repo (Full control)
+
+**Required Scopes for Organization Runners**
+
+* repo (Full control)
+* admin:org (Full control)
+* admin:public_key (read:public_key)
+* admin:repo_hook (read:repo_hook)
+* admin:org_hook (Full control)
+* notifications (Full control)
+* workflow (Full control)
+
+**Required Scopes for Enterprise Runners**
+
+* enterprise:admin (Full control)
+
+_Note: When you deploy enterprise runners they will get access to organisations, however, access to the repositories themselves is **NOT** allowed by default. Each GitHub organisation must allow enterprise runner groups to be used in repositories as an initial one time configuration step, this  only needs to be done once after which it is permanent for that runner group._
+
+---
+
+Once you have created the appropriate token, deploy it as a secret to your Kubernetes cluster that you are going to deploy the solution on:
+
+**Kubectl Deployment:**
 
 ```shell
 kubectl create secret generic controller-manager \
@@ -178,16 +195,25 @@ kubectl create secret generic controller-manager \
     --from-literal=github_token=${GITHUB_TOKEN}
 ```
 
+**Helm Deployment:**
+
+Configure your values.yaml, see the chart's [README](./charts/actions-runner-controller/README.md) for deploying the secret via Helm
+
 ## Usage
 
+[GitHub self-hosted runners can be deployed at various levels in a management hierarchy](https://docs.github.com/en/actions/hosting-your-own-runners/about-self-hosted-runners#about-self-hosted-runners):
+- The repository level
+- The organization level
+- The enterprise level
+
 There are two ways to use this controller:
 
 - Manage runners one by one with `Runner`.
 - Manage a set of runners with `RunnerDeployment`.
 
-### Repository runners
+### Repository Runners
 
-To launch a single self-hosted runner, you need to create a manifest file includes *Runner* resource as follows. This example launches a self-hosted runner with name *example-runner* for the *summerwind/actions-runner-controller* repository.
+To launch a single self-hosted runner, you need to create a manifest file includes `Runner` resource as follows. This example launches a self-hosted runner with name *example-runner* for the *actions-runner-controller/actions-runner-controller* repository.
 
 ```yaml
 # runner.yaml
@@ -245,6 +271,22 @@ spec:
 
 Now you can see the runner on the organization level (if you have organization owner permissions).
 
+### Enterprise Runners
+
+To add the runner to an enterprise, you only need to replace the `repository` field with `enterprise`, so the runner will register itself to the enterprise.
+
+```yaml
+# runner.yaml
+apiVersion: actions.summerwind.dev/v1alpha1
+kind: Runner
+metadata:
+  name: example-enterprise-runner
+spec:
+  enterprise: your-enterprise-name
+```
+
+Now you can see the runner on the enterprise level (if you have enterprise access permissions).
+
 ### RunnerDeployments
 
 There are `RunnerReplicaSet` and `RunnerDeployment` that corresponds to `ReplicaSet` and `Deployment` but for `Runner`.
@@ -268,7 +310,7 @@ spec:
 Apply the manifest file to your cluster:
 
 ```shell
-$ kubectl apply -f runner.yaml
+$ kubectl apply -f runnerdeployment.yaml
 runnerdeployment.actions.summerwind.dev/example-runnerdeploy created
 ```
 
@@ -281,11 +323,64 @@ example-runnerdeploy2475h595fr   mumoshu/actions-runner-controller-ci   Running
 example-runnerdeploy2475ht2qbr   mumoshu/actions-runner-controller-ci   Running
 ```
 
-#### Autoscaling
+##### Note on scaling to/from 0
 
-`RunnerDeployment` can scale the number of runners between `minReplicas` and `maxReplicas` fields, depending on pending workflow runs.
+> This is a documentation about a unreleased version of actions-runner-controller.
+>
+> It would be great if you could try building the latest controller image following https://github.com/actions-runner-controller/actions-runner-controller#contributing if you are eager to test it early and help
+> developers by reporting any bugs :smile:
 
-In the below example, `actions-runner` checks for pending workflow runs for each sync period, and scale to e.g. 3 if there're 3 pending jobs at sync time.
+You can either delete the runner deployment, or update it to have `replicas: 0`, so that there will be 0 runner pods in the cluster. This, in combination with e.g. `cluster-autoscaler`, enables you to save your infrastructure cost when there's no need to run Actions jobs.
+
+```yaml
+# runnerdeployment.yaml
+apiVersion: actions.summerwind.dev/v1alpha1
+kind: RunnerDeployment
+metadata:
+  name: example-runnerdeploy
+spec:
+  replicas: 0
+```
+
+The implication of setting `replicas: 0` instead of deleting the runner deployment is that you can let GitHub Actions queue jobs until there will be one or more runners. See [#465](https://github.com/actions-runner-controller/actions-runner-controller/pull/465) for more information.
+
+Also note that the controller creates a "registration-only" runner per RunnerReplicaSet on it's being scaled to zero,
+and retains it until there are one or more runners available.
+
+This, in combination with a correctly configured HorizontalRunnerAutoscaler, allows you to automatically [scale to/from 0](#autoscaling-tofrom-0)
+
+### Autoscaling
+
+__**IMPORTANT : Due to limitations / a bug with GitHub's [routing engine](https://docs.github.com/en/actions/hosting-your-own-runners/using-self-hosted-runners-in-a-workflow#routing-precedence-for-self-hosted-runners) autoscaling does NOT work correctly with RunnerDeployments that target the enterprise level. Scaling activity works as expected however jobs fail to get assigned to the scaled out replicas. This was explored in issue [#470](https://github.com/actions-runner-controller/actions-runner-controller/issues/470). Once GitHub resolves the issue with their backend service we expect the solution to be able to support autoscaled enterprise runnerdeploments without any additional changes.**__
+
+A `RunnerDeployment` (excluding enterprise runners) can scale the number of runners between `minReplicas` and `maxReplicas` fields based the chosen scaling metric as defined in the `metrics` attribute
+
+**Scaling Metrics**
+
+**TotalNumberOfQueuedAndInProgressWorkflowRuns**
+
+In the below example, `actions-runner` will poll GitHub for all pending workflows with the poll period defined by the sync period configuration. It will then scale to e.g. 3 if there're 3 pending jobs at sync time.
+With this scaling metric we are required to define a list of repositories within our metric.
+
+The scale out performance is controlled via the manager containers startup `--sync-period` argument. The default value is set to 10 minutes to prevent default deployments rate limiting themselves from the GitHub API.
+
+**Kustomize Config :** The period can be customised in the `config/default/manager_auth_proxy_patch.yaml` patch<br />
+
+**Benefits of this metric**
+1. Supports named repositories allowing you to restrict the runner to a specified set of repositories server side.
+2. Scales the runner count based on the actual queue depth of the jobs meaning a more 1:1 scaling of runners to queued jobs (caveat, see drawback #4)
+3. Like all scaling metrics, you can manage workflow allocation to the RunnerDeployment through the use of [GitHub labels](#runner-labels).
+
+**Drawbacks of this metric**
+1. Repositories must be named within the scaling metric, maintaining a list of repositories may not be viable in larger environments or self-serve environments.
+2. May not scale quick enough for some users needs. This metric is pull based and so the queue depth is polled as configured by the sync period, as a result scaling performance is bound by this sync period meaning there is a lag to scaling activity.
+3. Relatively large amounts of API requests required to maintain this metric, you may run in API rate limiting issues depending on the size of your environment and how aggressive your sync period configuration is
+4. The GitHub API doesn't provide a way to filter workflow jobs to just those targeting self-hosted runners. If your environment's workflows target both self-hosted and GitHub hosted runners then the queue depth this metric scales against isn't a true 1:1 mapping of queue depth to required runner count. As a result of this, this metric may scale too aggressively for your actual self-hosted runner count needs.
+
+
+Example `RunnerDeployment` backed by a `HorizontalRunnerAutoscaler`:
+
+_Important!!! We no longer include the attribute `replicas` in our `RunnerDeployment` if we are configuring autoscaling!_
 
 ```yaml
 apiVersion: actions.summerwind.dev/v1alpha1
@@ -312,38 +407,35 @@ spec:
     - summerwind/actions-runner-controller
 ```
 
-The scale out performance is controlled via the manager containers startup `--sync-period` argument. The default value is 10 minutes to prevent unconfigured deployments rate limiting themselves from the GitHub API. The period can be customised in the `config/default/manager_auth_proxy_patch.yaml` patch for those that are building the solution via the kustomize setup.
-
-Additionally, the autoscaling feature has an anti-flapping option that prevents periodic loop of scaling up and down.
-By default, it doesn't scale down until the grace period of 10 minutes passes after a scale up. The grace period can be configured by setting `scaleDownDelaySecondsAfterScaleUp`:
+Additionally, the `HorizontalRunnerAutoscaler` also has an anti-flapping option that prevents periodic loop of scaling up and down.
+By default, it doesn't scale down until the grace period of 10 minutes passes after a scale up. The grace period can be configured however by adding the setting `scaleDownDelaySecondsAfterScaleOut` in the `HorizontalRunnerAutoscaler` `spec`:
 
 ```yaml
-apiVersion: actions.summerwind.dev/v1alpha1
-kind: RunnerDeployment
-metadata:
-  name: example-runner-deployment
 spec:
-  template:
-    spec:
-      repository: summerwind/actions-runner-controller
----
-apiVersion: actions.summerwind.dev/v1alpha1
-kind: HorizontalRunnerAutoscaler
-metadata:
-  name: example-runner-deployment-autoscaler
-spec:
-  scaleTargetRef:
-    name: example-runner-deployment
-  minReplicas: 1
-  maxReplicas: 3
   scaleDownDelaySecondsAfterScaleOut: 60
-  metrics:
-  - type: TotalNumberOfQueuedAndInProgressWorkflowRuns
-    repositoryNames:
-    - summerwind/actions-runner-controller
 ```
 
-If you do not want to manage an explicit list of repositories to scale, an alternate autoscaling scheme that can be applied is the PercentageRunnersBusy scheme. The number of desired pods are evaulated by checking how many runners are currently busy and applying a scaleup or scale down factor if certain thresholds are met. By setting the metric type to PercentageRunnersBusy, the HorizontalRunnerAutoscaler will query github for the number of busy runners which live in the RunnerDeployment namespace. Scaleup and scaledown thresholds are the percentage of busy runners at which the number of desired runners are re-evaluated. Scaleup and scaledown factors are the multiplicative factor applied to the current number of runners used to calculate the number of desired runners. This scheme is also especially useful if you want multiple controllers in various clusters, each responsible for scaling their own runner pods per namespace.
+**PercentageRunnersBusy**
+
+The `HorizontalRunnerAutoscaler` will poll GitHub based on the configuration sync period for the number of busy runners which live in the RunnerDeployment's namespace and scale based on the settings
+
+**Kustomize Config :** The period can be customised in the `config/default/manager_auth_proxy_patch.yaml` patch<br />
+
+**Benefits of this metric**
+1. Supports named repositories server side the same as the `TotalNumberOfQueuedAndInProgressWorkflowRuns` metric [#313](https://github.com/actions-runner-controller/actions-runner-controller/pull/313)
+2. Supports GitHub organization wide scaling without maintaining an explicit list of repositories, this is especially useful for those that are working at a larger scale. [#223](https://github.com/actions-runner-controller/actions-runner-controller/pull/223)
+3. Like all scaling metrics, you can manage workflow allocation to the RunnerDeployment through the use of [GitHub labels](#runner-labels)
+4. Supports scaling desired runner count on both a percentage increase / decrease basis as well as on a fixed increase / decrease count basis [#223](https://github.com/actions-runner-controller/actions-runner-controller/pull/223) [#315](https://github.com/actions-runner-controller/actions-runner-controller/pull/315)
+
+**Drawbacks of this metric**
+1. May not scale quick enough for some users needs. This metric is pull based and so the number of busy runners are polled as configured by the sync period, as a result scaling performance is bound by this sync period meaning there is a lag to scaling activity.
+2. We are scaling up and down based on indicative information rather than a count of the actual number of queued jobs and so the desired runner count is likely to under provision new runners or overprovision them relative to actual job queue depth, this may or may not be a problem for you.
+
+
+Examples of each scaling type implemented with a `RunnerDeployment` backed by a `HorizontalRunnerAutoscaler`:
+
+
+_Important!!! We no longer include the attribute `replicas` in our `RunnerDeployment` if we are configuring autoscaling!_
 
 ```yaml
 ---
@@ -356,26 +448,53 @@ spec:
     name: example-runner-deployment
   minReplicas: 1
   maxReplicas: 3
-  scaleDownDelaySecondsAfterScaleOut: 60
   metrics:
   - type: PercentageRunnersBusy
-    scaleUpThreshold: '0.75'
-    scaleDownThreshold: '0.3'
-    scaleUpFactor: '1.4'
-    scaleDownFactor: '0.7'
+    scaleUpThreshold: '0.75'    # The percentage of busy runners at which the number of desired runners are re-evaluated to scale up
+    scaleDownThreshold: '0.3'   # The percentage of busy runners at which the number of desired runners are re-evaluated to scale down
+    scaleUpFactor: '1.4'        # The scale up multiplier factor applied to desired count
+    scaleDownFactor: '0.7'      # The scale down multiplier factor applied to desired count
+```
+
+```yaml
+---
+apiVersion: actions.summerwind.dev/v1alpha1
+kind: HorizontalRunnerAutoscaler
+metadata:
+  name: example-runner-deployment-autoscaler
+spec:
+  scaleTargetRef:
+    name: example-runner-deployment
+  minReplicas: 1
+  maxReplicas: 3
+  metrics:
+  - type: PercentageRunnersBusy
+    scaleUpThreshold: '0.75'    # The percentage of busy runners at which the number of desired runners are re-evaluated to scale up
+    scaleDownThreshold: '0.3'   # The percentage of busy runners at which the number of desired runners are re-evaluated to scale down
+    ScaleUpAdjustment: '2'      # The scale up runner count added to desired count
+    ScaleDownAdjustment: '1'    # The scale down runner count subtracted from the desired count
+```
+
+Like the previous metric, the scale down factor respects the anti-flapping configuration is applied to the `HorizontalRunnerAutoscaler` as mentioned previously:
+
+```yaml
+spec:
+  scaleDownDelaySecondsAfterScaleOut: 60
 ```
 
 #### Faster Autoscaling with GitHub Webhook
 
+__**IMPORTANT : Due to missing webhook events, webhook based scaling is not available for enterprise level RunnerDeployments. This was explored in issue [#470](https://github.com/actions-runner-controller/actions-runner-controller/issues/470).**__
+
 > This feature is an ADVANCED feature which may require more work to set up.
 > Please get prepared to put some time and effort to learn and leverage this feature!
 
 `actions-runner-controller` has an optional Webhook server that receives GitHub Webhook events and scale
-[`RunnerDeployment`s](#runnerdeployments) by updating corresponding [`HorizontalRunnerAutoscaler`s](#autoscaling).
+[`RunnerDeployments`](#runnerdeployments) by updating corresponding [`HorizontalRunnerAutoscalers`](#autoscaling).
 
 Today, the Webhook server can be configured to respond GitHub `check_run`, `pull_request`, and `push` events
 by scaling up the matching `HorizontalRunnerAutoscaler` by N replica(s), where `N` is configurable within
-`HorizontalRunerAutoscaler`'s `Spec`.
+`HorizontalRunnerAutoscaler's` `Spec`.
 
 More concretely, you can configure the targeted GitHub event types and the `N` in
 `scaleUpTriggers`:
@@ -385,7 +504,7 @@ kind: HorizontalRunnerAutoscaler
 spec:
   scaleTargetRef:
     name: myrunners
-  scaleUpTrigggers:
+  scaleUpTriggers:
   - githubEvent:
       checkRun:
         types: ["created"]
@@ -404,7 +523,7 @@ In contrast, the standard autoscaling requires you to wait next sync period to a
 insufficient runners. You can definitely shorten the sync period to make the standard autoscaling more responsive.
 But doing so eventually result in the controller not functional due to GitHub API rate limit.
 
-> You can learn the implementation details in #282
+> You can learn the implementation details in [#282](https://github.com/actions-runner-controller/actions-runner-controller/pull/282)
 
 To enable this feature, you firstly need to install the webhook server.
 
@@ -444,7 +563,7 @@ kind: HorizontalRunnerAutoscaler
 spec:
   scaleTargetRef:
     name: myrunners
-  scaleUpTrigggers:
+  scaleUpTriggers:
   - githubEvent:
       checkRun:
         types: ["created"]
@@ -466,7 +585,7 @@ kind: HorizontalRunnerAutoscaler
 spec:
   scaleTargetRef:
     name: myrunners
-  scaleUpTrigggers:
+  scaleUpTriggers:
   - githubEvent:
       pullRequest:
         types: ["synchronize"]
@@ -477,6 +596,107 @@ spec:
 
 See ["activity types"](https://docs.github.com/en/actions/reference/events-that-trigger-workflows#pull_request) for the list of valid values for `scaleUpTriggers[].githubEvent.pullRequest.types`.
 
+
+#### Autoscaling to/from 0
+
+> This is a documentation about a unreleased version of actions-runner-controller.
+>
+> It would be great if you could try building the latest controller image following https://github.com/actions-runner-controller/actions-runner-controller#contributing if you are eager to test it early and help
+> developers by reporting any bugs :smile:
+
+Previously, we've discussed about [how to scale a RunnerDeployment to/from 0](#note-on-scaling-tofrom-0)
+
+To automate the process of scaling to/from 0, you can use `HorizontalRunnerAutoscaler` with a caveat.
+
+That is, you need to choose one of the following configuration for metrics and triggers:
+
+- `TotalNumberOfQueuedAndInProgressWorkflowRuns`
+- `PercentageRunnersBusy` + `TotalNumberOfQueuedAndInProgressWorkflowRuns`
+- `PercentageRunnersBusy` + Webhook-based autoscaling
+
+This is due to that `PercentageRunnersBusy`, by its definition, needs one or more GitHub runners that can become `busy`, which cannot happen at all when you have 0 active runners.
+
+If and only if HorizontalRunnerAutoscaler is configured to have a secondary metric of `TotalNumberOfQueuedAndInProgressWorkflowRuns` and the controller sees the primary metric of `PercentageRunnersBusy` returned 0 desired replicas, it uses the secondary metric for calculating the desired replicas once again.
+
+A correctly configured `TotalNumberOfQueuedAndInProgressWorkflowRuns` can return non-zero desired replicas even when there are no runners other than [registration-only runners](#note-on-scaling-tofrom-0), hence the `PercentageRunnersBusy` + `TotalNumberOfQueuedAndInProgressWorkflowRuns` configuration makes scaling from zero possible.
+
+Similarly, Webhook-based autoscaling works regardless of there are active runners, hence `PercentageRunnersBusy` + Webhook-based autoscaling configuration makes scaling from zero, too.
+
+#### Scheduled Overrides
+
+> This is a documentation about a unreleased version of actions-runner-controller.
+>
+> It would be great if you could try building the latest controller image following https://github.com/actions-runner-controller/actions-runner-controller#contributing if you are eager to test it early and help
+> developers by reporting any bugs :smile:
+
+`Scheduled Overrides` allows you to configure HorizontalRunnerAutoscaler so that its Spec gets updated only during a certain period of time.
+
+usually, this feature is used for following scenarios:
+
+- You want to pay for your infrastructure cost running runners only in business hours
+- You want to prepare for scheduled spikes in workloads
+
+For the first scenario, you might consider configuration like the below:
+
+```
+apiVersion: actions.summerwind.dev/v1alpha1
+kind: HorizontalRunnerAutoscaler
+metadata:
+  name: example-runner-deployment-autoscaler
+spec:
+  scaleTargetRef:
+    name: example-runner-deployment
+  scheduledOverrides:
+  # Override minReplicas to 0 only between 0am sat to 0am mon
+  - startTime: "2021-05-01T00:00:00+09:00"
+    endTime: "2021-05-03T00:00:00+09:00"
+    recurrenceRule:
+      frequency: Weekly
+      untilTime: "2022-05-01T00:00:00+09:00"
+    minReplicas: 0
+  minReplicas: 1
+```
+
+For the second scenario, you might consider something like the below:
+
+```
+apiVersion: actions.summerwind.dev/v1alpha1
+kind: HorizontalRunnerAutoscaler
+metadata:
+  name: example-runner-deployment-autoscaler
+spec:
+  scaleTargetRef:
+    name: example-runner-deployment
+  scheduledOverrides:
+  # Override minReplicas to 100 only between 2021-06-01T00:00:00+09:00 and 2021-06-03T00:00:00+09:00
+  - startTime: "2021-06-01T00:00:00+09:00"
+    endTime: "2021-06-03T00:00:00+09:00"
+    minReplicas: 100
+  minReplicas: 1
+```
+
+The most basic usage of this feature is actually the second scenario mentioned above.
+A scheduled override without `recurrenceRule` is considered a one-off override, that is active between `startTime` and `endTime`. In the second scenario, it overrides `minReplicas` to `100` only between `2021-06-01T00:00:00+09:00` and `2021-06-03T00:00:00+09:00`.
+
+A scheduled override with `recurrenceRule` is considered a recurring override. A recurring override is initially active between `startTime` and `endTime`, and then it repeatedly get activated after a certain period of time denoted by `frequency`.
+
+`frequecy` can take one of the following values:
+
+- `Daily`
+- `Weekly`
+- `Monthly`
+- `Yearly`
+
+By default, a scheduled override repeats forever. If you want it to repeat until a specific point in time, define `untilTime`. The controller create the last recurrence of the override until the recurrence's `startTime` is equal or earlier than `untilTime`.
+
+Do note that you have enough slack for `untilTime`, so that a delayed or offline `actions-runner-controller` is much less likely to miss the last recurrence. For example, you might want to set `untilTime` to `M` minutes after the last recurrence's `startTime`, so that `actions-runner-controller` being offline up to `M` minutes doesn't miss the last recurrence.
+
+**Combining Multiple Scheduled Overrides**:
+
+In case you have a more complex scenarios, try writing two or more entries under `scheduledOverrides`.
+
+The earlier entry is prioritized higher than later entries. So you usually define one-time overrides in the top of your list, then yearly, monthly, weekly, and lastly daily overrides.
+
 ### Runner with DinD
 
 When using default runner, runner pod starts up 2 containers: runner and DinD (Docker-in-Docker). This might create issues if there's `LimitRange` set to namespace.
@@ -499,7 +719,7 @@ spec:
 
 This also helps with resources, as you don't need to give resources separately to docker and runner.
 
-### Additional tweaks
+### Additional Tweaks
 
 You can pass details through the spec selector. Here's an eg. of what you may like to do:
 
@@ -512,10 +732,22 @@ metadata:
 spec:
   replicas: 2
   template:
+    metadata:
+      annotations:
+        cluster-autoscaler.kubernetes.io/safe-to-evict: "true"
     spec:
       nodeSelector:
         node-role.kubernetes.io/test: ""
-
+      
+      securityContext:
+        #All level/role/type/user values will vary based on your SELinux policies.
+        #See https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux_atomic_host/7/html/container_security_guide/docker_selinux_security_policy for information about SELinux with containers
+        seLinuxOptions: 
+          level: "s0"
+          role: "system_r"
+          type: "super_t"
+          user: "system_u"
+          
       tolerations:
       - effect: NoSchedule
         key: node-role.kubernetes.io/test
@@ -531,20 +763,37 @@ spec:
         requests:
           cpu: "2.0"
           memory: "4Gi"
-
       # Timeout after a node crashed or became unreachable to evict your pods somewhere else (default 5mins)
       tolerations:
         - key: "node.kubernetes.io/unreachable"
           operator: "Exists"
           effect: "NoExecute"
           tolerationSeconds: 10
-
+      # true (default) = The runner restarts after running jobs, to ensure a clean and reproducible build environment
+      # false = The runner is persistent across jobs and doesn't automatically restart
+      # This directly controls the behaviour of `--once` flag provided to the github runner
+      ephemeral: false 
+      # true (default) = A privileged docker sidecar container is included in the runner pod.
+      # false = A docker sidecar container is not included in the runner pod and you can't use docker.
       # If set to false, there are no privileged container and you cannot use docker.
       dockerEnabled: false
-      # If set to true, runner pod container only 1 container that's expected to be able to run docker, too.
-      # image summerwind/actions-runner-dind or custom one should be used with true -value
-      dockerdWithinRunnerContainer: false
-      # Valid if dockerdWithinRunnerContainer is not true
+      # Optional Docker containers network MTU
+      # If your network card MTU is smaller than Docker's default 1500, you might encounter Docker networking issues.
+      # To fix these issues, you should setup Docker MTU smaller than or equal to that on the outgoing network card.
+      # More information:
+      # - https://mlohr.com/docker-mtu/
+      dockerMTU: 1500
+      # Optional Docker registry mirror
+      # Docker Hub has enabled rate-limiting for free plans.
+      # To avoid disruptions in your CI/CD pipelines, you might want to setup an external or on-premises Docker registry mirror.
+      # More information:
+      # - https://docs.docker.com/docker-hub/download-rate-limit/
+      # - https://cloud.google.com/container-registry/docs/pulling-cached-images
+      dockerRegistryMirror: https://mirror.gcr.io/
+      # false (default) = Docker support is provided by a sidecar container deployed in the runner pod.
+      # true = No docker sidecar container is deployed in the runner pod but docker can be used within the runner container instead. The image summerwind/actions-runner-dind is used by default.
+      dockerdWithinRunnerContainer: true
+      # Docker sidecar container image tweaks examples below, only applicable if dockerdWithinRunnerContainer = false
       dockerdContainerResources:
         limits:
           cpu: "4.0"
@@ -552,6 +801,7 @@ spec:
         requests:
           cpu: "2.0"
           memory: "4Gi"
+      # Additional N number of sidecar containers
       sidecarContainers:
         - name: mysql
           image: mysql:5.7
@@ -560,13 +810,28 @@ spec:
               value: abcd1234
           securityContext:
             runAsUser: 0
-      # if workDir is not specified, the default working directory is /runner/_work
-      # this setting allows you to customize the working directory location
+      # workDir if not specified (default = /runner/_work)
+      # You can customise this setting allowing you to change the default working directory location
       # for example, the below setting is the same as on the ubuntu-18.04 image
       workDir: /home/runner/work
+      # You can mount some of the shared volumes to the dind container using dockerVolumeMounts, like any other volume mounting.
+      # NOTE: in case you want to use an hostPath like the following example, make sure that Kubernetes doesn't schedule more than one runner
+      # per physical host. You can achieve that by setting pod anti-affinity rules and/or resource requests/limits.
+      volumes:
+        - name: docker-extra
+          hostPath:
+            path: /mnt/docker-extra
+            type: DirectoryOrCreate
+      dockerVolumeMounts:
+        - mountPath: /var/lib/docker
+          name: docker-extra
+      # Optional name of the container runtime configuration that should be used for pods.
+      # This must match the name of a RuntimeClass resource available on the cluster.
+      # More info: https://kubernetes.io/docs/concepts/containers/runtime-class
+      runtimeClassName: "runc"
 ```
 
-### Runner labels
+### Runner Labels
 
 To run a workflow job on a self-hosted runner, you can use the following syntax in your workflow:
 
@@ -605,7 +870,7 @@ Note that if you specify `self-hosted` in your workflow, then this will run your
 
 ### Runner Groups
 
-Runner groups can be used to limit which repositories are able to use the GitHub Runner at an Organisation level. Runner groups have to be [created in GitHub first](https://docs.github.com/en/actions/hosting-your-own-runners/managing-access-to-self-hosted-runners-using-groups) before they can be referenced.
+Runner groups can be used to limit which repositories are able to use the GitHub Runner at an organization level. Runner groups have to be [created in GitHub first](https://docs.github.com/en/actions/hosting-your-own-runners/managing-access-to-self-hosted-runners-using-groups) before they can be referenced.
 
 To add the runner to the group `NewGroup`, specify the group in your `Runner` or `RunnerDeployment` spec.
 
@@ -622,15 +887,14 @@ spec:
       group: NewGroup
 ```
 
-### Using EKS IAM role for service accounts
+### Using IRSA (IAM Roles for Service Accounts) in EKS
 
-`actions-runner-controller` v0.15.0 or later has support for EKS IAM role for service accounts.
+`actions-runner-controller` v0.15.0 or later has support for IRSA in EKS.
 
 As similar as for regular pods and deployments, you firstly need an existing service account with the IAM role associated.
 Create one using e.g. `eksctl`. You can refer to [the EKS documentation](https://docs.aws.amazon.com/eks/latest/userguide/iam-roles-for-service-accounts.html) for more details.
 
-Once you set up the service account, all you need is to add `serviceAccountName` and `fsGroup` to any pods that uses
-the IAM-role enabled service account.
+Once you set up the service account, all you need is to add `serviceAccountName` and `fsGroup` to any pods that uses the IAM-role enabled service account.
 
 For `RunnerDeployment`, you can set those two fields under the runner spec at `RunnerDeployment.Spec.Template`:
 
@@ -645,17 +909,21 @@ spec:
       repository: USER/REO
       serviceAccountName: my-service-account
       securityContext:
-        fsGroup: 1447
+        fsGroup: 1000
 ```
 
-### Software installed in the runner image
+### Software Installed in the Runner Image
 
-The GitHub hosted runners include a large amount of pre-installed software packages. For Ubuntu 18.04, this list can be found at <https://github.com/actions/virtual-environments/blob/master/images/linux/Ubuntu1804-README.md>
+**Cloud Tooling**<br />
+The project supports being deployed on the various cloud Kubernetes platforms (e.g. EKS), it does not however aim to go beyond that. No cloud specific tooling is bundled in the base runner, this is an active decision to keep the overhead of maintaining the solution manageable.
 
-The container image is based on Ubuntu 18.04, but it does not contain all of the software installed on the GitHub runners. It contains the following subset of packages from the GitHub runners:
+**Bundled Software**<br />
+The GitHub hosted runners include a large amount of pre-installed software packages. GitHub maintain a list in README files at <https://github.com/actions/virtual-environments/tree/main/images/linux>
+
+This solution maintains a few runner images with `latest` aligning with GitHub's Ubuntu version. Older images are maintained whilst GitHub also provides them as an option. These images do not contain all of the software installed on the GitHub runners. It contains the following subset of packages from the GitHub runners:
 
 - Basic CLI packages
-- git (2.26)
+- git
 - docker
 - build-essentials
 
@@ -694,10 +962,68 @@ spec:
 **Solutions**<br />
 Your base64'ed PAT token has a new line at the end, it needs to be created without a `\n` added
 * `echo -n $TOKEN | base64`
-* Create the secret as described in the docs using the shell and documeneted flags
+* Create the secret as described in the docs using the shell and documented flags
 
-# Developing
+#### Runner coming up before network available
 
+If you're running your action runners on a service mesh like Istio, you might
+have problems with runner configuration accompanied by logs like:
+
+```
+....
+runner Starting Runner listener with startup type: service
+runner Started listener process
+runner An error occurred: Not configured
+runner Runner listener exited with error code 2
+runner Runner listener exit with retryable error, re-launch runner in 5 seconds.
+....
+```
+
+This is because the `istio-proxy` has not completed configuring itself when the
+configuration script tries to communicate with the network.
+
+**Solution**<br />
+
+> This feature is experimental and will be dropped once maintainers think that
+> everyone has already migrated to use Istio's `holdApplicationUntilProxyStarts` ([istio/istio#11130](https://github.com/istio/istio/issues/11130)).
+>
+> Please read the discussion in #592 for more information.
+
+You can add a delay to the entrypoint script by setting the `STARTUP_DELAY` environment
+variable. This will cause the script to sleep `STARTUP_DELAY` seconds.
+
+*Example `Runner` with a 2 second startup delay:*
+```yaml
+apiVersion: actions.summerwind.dev/v1alpha1
+kind: Runner
+metadata:
+  name: example-runner-with-sleep
+spec:
+  env:
+    - name: STARTUP_DELAY
+      value: "2" # Remember! env var values must be strings.  
+```
+
+*Example `RunnerDeployment` with a 2 second startup delay:*
+```yaml
+apiVersion: actions.summerwind.dev/v1alpha1
+kind: RunnerDeployment
+metadata:
+  name: example-runnerdeployment-with-sleep
+spec:
+  template:
+    spec:
+      env:
+        - name: STARTUP_DELAY
+          value: "2" # Remember! env var values must be strings.
+```
+
+
+# Contributing
+
+For more details about any requirements or process, please check out [Getting Started with Contributing](CONTRIBUTING.md).
+
+**The Controller**<br />
 If you'd like to modify the controller to fork or contribute, I'd suggest using the following snippet for running
 the acceptance test:
 
@@ -705,14 +1031,23 @@ the acceptance test:
 # This sets `VERSION` envvar to some appropriate value
 . hack/make-env.sh
 
-NAME=$DOCKER_USER/actions-runner-controller \
+DOCKER_USER=*** \
   GITHUB_TOKEN=*** \
   APP_ID=*** \
   PRIVATE_KEY_FILE_PATH=path/to/pem/file \
   INSTALLATION_ID=*** \
-  make docker-build docker-push acceptance
+  make acceptance
 ```
 
+> **Notes for Ubuntu 20.04+ users**
+>
+> If you're using Ubuntu 20.04 or greater, you might have installed `docker` with `snap`.
+>
+> If you want to stick with `snap`-provided `docker`, do not forget to set `TMPDIR` to
+> somewhere under `$HOME`.
+> Otherwise `kind load docker-image` fail while running `docker save`.
+> See https://kind.sigs.k8s.io/docs/user/known-issues/#docker-installed-with-snap for more information.
+
 Please follow the instructions explained in [Using Personal Access Token](#using-personal-access-token) to obtain
 `GITHUB_TOKEN`, and those in [Using GitHub App](#using-github-app) to obtain `APP_ID`, `INSTALLATION_ID`, and
 `PRIAVTE_KEY_FILE_PATH`.
@@ -721,27 +1056,103 @@ The test creates a one-off `kind` cluster, deploys `cert-manager` and `actions-r
 creates a `RunnerDeployment` custom resource for a public Git repository to confirm that the
 controller is able to bring up a runner pod with the actions runner registration token installed.
 
+**Rerunning a failed test**
+
+When one of tests run by `make acceptance` failed, you'd probably like to rerun only the failed one.
+
+It can be done by `make acceptance/run` and by setting the combination of `ACCEPTANCE_TEST_DEPLOYMENT_TOOL` and `ACCEPTANCE_TEST_SECRET_TYPE` values that failed.
+
+In the example below, we rerun the test for the combination `ACCEPTANCE_TEST_DEPLOYMENT_TOOL=helm ACCEPTANCE_TEST_SECRET_TYPE=token` only:
+
+```
+DOCKER_USER=*** \
+  GITHUB_TOKEN=*** \
+  APP_ID=*** \
+  PRIVATE_KEY_FILE_PATH=path/to/pem/file \
+  INSTALLATION_ID=*** \
+  ACCEPTANCE_TEST_DEPLOYMENT_TOOL=helm ACCEPTANCE_TEST_SECRET_TYPE=token \
+  make acceptance/run
+```
+
+**Testing in a non-kind cluster**
+
 If you prefer to test in a non-kind cluster, you can instead run:
 
 ```shell script
 KUBECONFIG=path/to/kubeconfig \
-NAME=$DOCKER_USER/actions-runner-controller \
+  DOCKER_USER=*** \
   GITHUB_TOKEN=*** \
   APP_ID=*** \
   PRIVATE_KEY_FILE_PATH=path/to/pem/file \
   INSTALLATION_ID=*** \
   ACCEPTANCE_TEST_SECRET_TYPE=token \
-  make docker-build docker-push \
-       acceptance/setup acceptance/tests
+  make docker-build acceptance/setup \
+       acceptance/deploy \
+       acceptance/tests
 ```
-# Alternatives
 
-The following is a list of alternative solutions that may better fit you depending on your use-case:
+**Development Tips**
 
-- <https://github.com/evryfs/github-actions-runner-operator/>
+Rerunning the whole acceptance test suite from scratch on every little change to the controller, the runner, and the chart would be counter-productive.
 
-Although the situation can change over time, as of writing this sentence, the benefits of using `actions-runner-controller` over the alternatives are:
+To make your development cycle faster, use the below command to update deploy and update all the three:
 
-- `actions-runner-controller` has the ability to autoscale runners based on number of pending/progressing jobs (#99)
-- `actions-runner-controller` is able to gracefully stop runners (#103)
-- `actions-runner-controller` has ARM support
+```
+# Let assume we have all other envvars like DOCKER_USER, GITHUB_TOKEN already set,
+# The below command will (re)build `actions-runner-controller:controller1` and `actions-runner:runner1`,
+# load those into kind nodes, and then rerun kubectl or helm to install/upgrade the controller,
+# and finally upgrade the runner deployment to use the new runner image.
+#
+# As helm 3 and kubectl is unable to recreate a pod when no tag change,
+# you either need to bump VERSION and RUNNER_TAG on each run,
+# or manually run `kubectl delete pod $POD` on respective pods for changes to actually take effect.
+VERSION=controller1 \
+  RUNNER_TAG=runner1 \
+  make docker-build acceptance/load acceptance/deploy
+```
+
+If you've already deployed actions-runner-controller and only want to recreate pods to use the newer image, you can run:
+
+```
+NAME=$DOCKER_USER/actions-runner-controller \
+  make docker-build acceptance/load && \
+  kubectl -n actions-runner-system delete po $(kubectl -n actions-runner-system get po -ojsonpath={.items[*].metadata.name})
+```
+
+Similarly, if you'd like to recreate runner pods with the newer runner image,
+
+```
+NAME=$DOCKER_USER/actions-runner make \
+  -C runner docker-{build,push}-ubuntu && \
+  (kubectl get po -ojsonpath={.items[*].metadata.name} | xargs -n1 kubectl delete po)
+```
+
+**Runner Tests**<br />
+A set of example pipelines (./acceptance/pipelines) are provided in this repository which you can use to validate your runners are working as expected. When raising a PR please run the relevant suites to prove your change hasn't broken anything.
+
+**Running Ginkgo Tests**
+
+You can run the integration test suite that is written in Ginkgo with:
+
+```bash
+make test-with-deps
+```
+
+This will firstly install a few binaries required to setup the integration test environment and then runs `go test` to start the Ginkgo test.
+
+If you don't want to use `make`, like when you're running tests from your IDE, install required binaries to `/usr/local/kubebuilder/bin`. That's the directory in which controller-runtime's `envtest` framework locates the binaries.
+
+```bash
+sudo mkdir -p /usr/local/kubebuilder/bin
+make kube-apiserver etcd
+sudo mv test-assets/{etcd,kube-apiserver} /usr/local/kubebuilder/bin/
+go test -v -run TestAPIs github.com/summerwind/actions-runner-controller/controllers
+```
+
+To run Ginkgo tests selectively, set the pattern of target test names to `GINKGO_FOCUS`.
+All the Ginkgo test that matches `GINKGO_FOCUS` will be run.
+
+```bash
+GINKGO_FOCUS='[It] should create a new Runner resource from the specified template, add a another Runner on replicas increased, and removes all the replicas when set to 0' \
+  go test -v -run TestAPIs github.com/summerwind/actions-runner-controller/controllers
+```

acceptance/checks.sh

@@ -12,6 +12,9 @@ done
 
 echo Found runner ${runner_name}.
 
+# Wait a bit to make sure the runner pod is created before looking for it.
+sleep 2
+
 pod_name=
 
 while [ -z "${pod_name}" ]; do
@@ -24,6 +27,6 @@ echo Found pod ${pod_name}.
 
 echo Waiting for pod ${runner_name} to become ready... 1>&2
 
-kubectl wait pod/${runner_name} --for condition=ready --timeout 180s
+kubectl wait pod/${runner_name} --for condition=ready --timeout 270s
 
 echo All tests passed. 1>&2

acceptance/deploy.sh

@@ -4,10 +4,14 @@ set -e
 
 tpe=${ACCEPTANCE_TEST_SECRET_TYPE}
 
+VALUES_FILE=${VALUES_FILE:-$(dirname $0)/values.yaml}
+
 if [ "${tpe}" == "token" ]; then
-  kubectl create secret generic controller-manager \
-	  -n actions-runner-system \
-	  --from-literal=github_token=${GITHUB_TOKEN:?GITHUB_TOKEN must not be empty}
+  if ! kubectl get secret controller-manager -n actions-runner-system >/dev/null; then
+    kubectl create secret generic controller-manager \
+      -n actions-runner-system \
+      --from-literal=github_token=${GITHUB_TOKEN:?GITHUB_TOKEN must not be empty}
+  fi
 elif [ "${tpe}" == "app" ]; then
   kubectl create secret generic controller-manager \
     -n actions-runner-system \
@@ -26,17 +30,37 @@ if [ "${tool}" == "helm" ]; then
     charts/actions-runner-controller \
     -n actions-runner-system \
     --create-namespace \
-    --set syncPeriod=5m
-  kubectl -n actions-runner-system wait deploy/actions-runner-controller --for condition=available
+    --set syncPeriod=${SYNC_PERIOD} \
+    --set authSecret.create=false \
+    --set image.repository=${NAME} \
+    --set image.tag=${VERSION} \
+    -f ${VALUES_FILE}
+  kubectl -n actions-runner-system wait deploy/actions-runner-controller --for condition=available --timeout 60s
 else
   kubectl apply \
     -n actions-runner-system \
     -f release/actions-runner-controller.yaml
-  kubectl -n actions-runner-system wait deploy/controller-manager --for condition=available --timeout 60s
+  kubectl -n actions-runner-system wait deploy/controller-manager --for condition=available --timeout 120s
 fi
 
 # Adhocly wait for some time until actions-runner-controller's admission webhook gets ready
 sleep 20
 
-kubectl apply \
-  -f acceptance/testdata/runnerdeploy.yaml
+if [ -n "${TEST_REPO}" ]; then
+  cat acceptance/testdata/runnerdeploy.yaml | envsubst | kubectl apply -f -
+  cat acceptance/testdata/hra.yaml | envsubst | kubectl apply -f -
+else
+  echo 'Skipped deploying runnerdeployment and hra. Set TEST_REPO to "yourorg/yourrepo" to deploy.'
+fi
+
+if [ -n "${TEST_ORG}" ]; then
+  cat acceptance/testdata/org.runnerdeploy.yaml | envsubst | kubectl apply -f -
+
+  if [ -n "${TEST_ORG_REPO}" ]; then
+    cat acceptance/testdata/org.hra.yaml | envsubst | kubectl apply -f -
+  else
+    echo 'Skipped deploying organizational hra. Set TEST_ORG_REPO to "yourorg/yourrepo" to deploy.'
+  fi
+else
+  echo 'Skipped deploying organizational runnerdeployment. Set TEST_ORG to deploy.'
+fi

acceptance/kind.yaml

@@ -0,0 +1,10 @@
+apiVersion: kind.x-k8s.io/v1alpha4
+kind: Cluster
+nodes:
+- role: control-plane
+  extraPortMappings:
+  - containerPort: 31000
+    hostPort: 31000
+    listenAddress: "0.0.0.0"
+    protocol: tcp
+#- role: worker

acceptance/pipelines/eks-integration-tests.yaml

@@ -0,0 +1,36 @@
+name: EKS Integration Tests
+
+on:
+  workflow_dispatch:
+
+env:
+  IRSA_ROLE_ARN:
+  ASSUME_ROLE_ARN: 
+  AWS_REGION: 
+
+jobs:
+  assume-role-in-runner-test:
+    runs-on: ['self-hosted', 'Linux']
+    steps:
+      - name: Test aws-actions/configure-aws-credentials Action
+        uses: aws-actions/configure-aws-credentials@v1
+        with:
+          aws-region: ${{ env.AWS_REGION }}
+          role-to-assume: ${{ env.ASSUME_ROLE_ARN }}
+          role-duration-seconds: 900
+  assume-role-in-container-test:
+    runs-on: ['self-hosted', 'Linux']
+    container: 
+      image: amazon/aws-cli
+      env:
+        AWS_WEB_IDENTITY_TOKEN_FILE: /var/run/secrets/eks.amazonaws.com/serviceaccount/token
+        AWS_ROLE_ARN: ${{ env.IRSA_ROLE_ARN }}
+      volumes:
+        - /var/run/secrets/eks.amazonaws.com/serviceaccount/token:/var/run/secrets/eks.amazonaws.com/serviceaccount/token
+    steps:
+      - name: Test aws-actions/configure-aws-credentials Action in container
+        uses: aws-actions/configure-aws-credentials@v1
+        with:
+          aws-region: ${{ env.AWS_REGION }}
+          role-to-assume: ${{ env.ASSUME_ROLE_ARN }}
+          role-duration-seconds: 900

acceptance/pipelines/runner-integration-tests.yaml

@@ -0,0 +1,83 @@
+name: Runner Integration Tests
+
+on:
+  workflow_dispatch:
+
+env:
+  ImageOS: ubuntu18 # Used by ruby/setup-ruby action | Update me for the runner OS version you are testing against
+
+jobs:
+  run-step-in-container-test:
+    runs-on: ['self-hosted', 'Linux']
+    container: 
+      image: alpine
+    steps:
+      - name: Test we are working in the container
+        run: |
+          if [[ $(sed -n '2p' < /etc/os-release | cut -d "=" -f2) != "alpine" ]]; then
+              echo "::error ::Failed OS detection test, could not match /etc/os-release with alpine. Are we really running in the container?"
+              echo "/etc/os-release below:"
+              cat /etc/os-release
+              exit 1
+          fi
+  setup-python-test:
+    runs-on: ['self-hosted', 'Linux']
+    steps:
+      - name: Print native Python environment
+        run: |
+          which python
+          python --version
+      - uses: actions/setup-python@v2
+        with:
+          python-version: 3.9
+      - name: Test actions/setup-python works
+        run: |
+          VERSION=$(python --version 2>&1 | cut -d ' ' -f2 | cut -d '.' -f1-2)
+          if [[ $VERSION != '3.9' ]]; then
+            echo "Python version detected : $(python --version 2>&1)"
+            echo "::error ::Detected python failed setup version test, could not match version with version specified in the setup action"
+            exit 1
+          else
+            echo "Python version detected : $(python --version 2>&1)"
+          fi
+  setup-node-test:
+    runs-on: ['self-hosted', 'Linux']
+    steps:
+      - uses: actions/setup-node@v2
+        with:
+          node-version: '12'
+      - name: Test actions/setup-node works 
+        run: |
+          VERSION=$(node --version | cut -c 2- | cut -d '.' -f1)
+          if [[ $VERSION != '12' ]]; then
+            echo "Node version detected : $(node --version 2>&1)"
+            echo "::error ::Detected node failed setup version test, could not match version with version specified in the setup action"
+            exit 1
+          else
+            echo "Node version detected : $(node --version 2>&1)"
+          fi
+  setup-ruby-test:
+    runs-on: ['self-hosted', 'Linux']
+    steps:
+      - uses: ruby/setup-ruby@v1
+        with:
+          ruby-version: 3.0
+          bundler-cache: true
+      - name: Test ruby/setup-ruby works 
+        run: |
+          VERSION=$(ruby --version | cut -d ' ' -f2 | cut -d '.' -f1-2)
+          if [[ $VERSION != '3.0' ]]; then
+              echo "Ruby version detected : $(ruby --version 2>&1)"
+              echo "::error ::Detected ruby failed setup version test, could not match version with version specified in the setup action"
+              exit 1
+          else
+              echo "Ruby version detected : $(ruby --version 2>&1)"
+          fi
+  python-shell-test:
+    runs-on: ['self-hosted', 'Linux']
+    steps:      
+      - name: Test Python shell works
+        run: |
+          import os
+          print(os.environ['PATH'])
+        shell: python

acceptance/testdata/hra.yaml

@@ -0,0 +1,25 @@
+apiVersion: actions.summerwind.dev/v1alpha1
+kind: HorizontalRunnerAutoscaler
+metadata:
+  name: actions-runner-aos-autoscaler
+spec:
+  scaleTargetRef:
+    name: example-runnerdeploy
+  scaleUpTriggers:
+  - githubEvent:
+      checkRun:
+        types: ["created"]
+        status: "queued"
+    amount: 1
+    duration: "1m"
+  minReplicas: 0
+  maxReplicas: 5
+  metrics:
+  - type: PercentageRunnersBusy
+    scaleUpThreshold: '0.75'
+    scaleDownThreshold: '0.3'
+    scaleUpFactor: '2'
+    scaleDownFactor: '0.5'
+  - type: TotalNumberOfQueuedAndInProgressWorkflowRuns
+    repositoryNames:
+    - ${TEST_REPO}

acceptance/testdata/org.hra.yaml

@@ -0,0 +1,35 @@
+apiVersion: actions.summerwind.dev/v1alpha1
+kind: HorizontalRunnerAutoscaler
+metadata:
+  name: org
+spec:
+  scaleTargetRef:
+    name: org-runnerdeploy
+  scaleUpTriggers:
+  - githubEvent:
+      checkRun:
+        types: ["created"]
+        status: "queued"
+    amount: 1
+    duration: "1m"
+  scheduledOverrides:
+  - startTime: "2021-05-11T16:05:00+09:00"
+    endTime: "2021-05-11T16:40:00+09:00"
+    minReplicas: 2
+  - startTime: "2021-05-01T00:00:00+09:00"
+    endTime: "2021-05-03T00:00:00+09:00"
+    recurrenceRule:
+      frequency: Weekly
+      untilTime: "2022-05-01T00:00:00+09:00"
+    minReplicas: 0
+  minReplicas: 0
+  maxReplicas: 5
+  metrics:
+  - type: PercentageRunnersBusy
+    scaleUpThreshold: '0.75'
+    scaleDownThreshold: '0.3'
+    scaleUpFactor: '2'
+    scaleDownFactor: '0.5'
+  - type: TotalNumberOfQueuedAndInProgressWorkflowRuns
+    repositoryNames:
+    - ${TEST_ORG_REPO}

acceptance/testdata/org.runnerdeploy.yaml

@@ -0,0 +1,37 @@
+apiVersion: actions.summerwind.dev/v1alpha1
+kind: RunnerDeployment
+metadata:
+  name: org-runnerdeploy
+spec:
+#  replicas: 1
+  template:
+    spec:
+      organization: ${TEST_ORG}
+
+      #
+      # Custom runner image
+      #
+      image: ${RUNNER_NAME}:${RUNNER_TAG}
+      imagePullPolicy: IfNotPresent
+
+      #
+      # dockerd within runner container
+      #
+      ## Replace `mumoshu/actions-runner-dind:dev` with your dind image
+      #dockerdWithinRunnerContainer: true
+      #image: mumoshu/actions-runner-dind:dev
+
+      #
+      # Set the MTU used by dockerd-managed network interfaces (including docker-build-ubuntu)
+      #
+      #dockerMTU: 1450
+
+      #Runner group
+      # labels:
+      # - "mylabel 1"
+      # - "mylabel 2"
+
+      #
+      # Non-standard working directory
+      #
+      # workDir: "/"

acceptance/testdata/runnerdeploy.yaml

@@ -6,4 +6,32 @@ spec:
 #  replicas: 1
   template:
     spec:
-      repository: mumoshu/actions-runner-controller-ci
+      repository: ${TEST_REPO}
+
+      #
+      # Custom runner image
+      #
+      image: ${RUNNER_NAME}:${RUNNER_TAG}
+      imagePullPolicy: IfNotPresent
+
+      #
+      # dockerd within runner container
+      #
+      ## Replace `mumoshu/actions-runner-dind:dev` with your dind image
+      #dockerdWithinRunnerContainer: true
+      #image: mumoshu/actions-runner-dind:dev
+
+      #
+      # Set the MTU used by dockerd-managed network interfaces (including docker-build-ubuntu)
+      #
+      #dockerMTU: 1450
+
+      #Runner group
+      # labels:
+      # - "mylabel 1"
+      # - "mylabel 2"
+
+      #
+      # Non-standard working directory
+      #
+      # workDir: "/"

acceptance/values.yaml

@@ -0,0 +1,20 @@
+# Set actions-runner-controller settings for testing
+githubAPICacheDuration: 10s
+githubWebhookServer:
+  enabled: true
+  labels: {}
+  replicaCount: 1
+  syncPeriod: 10m
+  secret:
+    create: true
+    name: "github-webhook-server"
+    ### GitHub Webhook Configuration
+    #github_webhook_secret_token: ""
+  service:
+    type: NodePort
+    ports:
+      - port: 80
+        targetPort: http
+        protocol: TCP
+        name: http
+        nodePort: 31000

api/v1alpha1/horizontalrunnerautoscaler_types.go

@@ -54,6 +54,12 @@ type HorizontalRunnerAutoscalerSpec struct {
 	ScaleUpTriggers []ScaleUpTrigger `json:"scaleUpTriggers,omitempty"`
 
 	CapacityReservations []CapacityReservation `json:"capacityReservations,omitempty" patchStrategy:"merge" patchMergeKey:"name"`
+
+	// ScheduledOverrides is the list of ScheduledOverride.
+	// It can be used to override a few fields of HorizontalRunnerAutoscalerSpec on schedule.
+	// The earlier a scheduled override is, the higher it is prioritized.
+	// +optional
+	ScheduledOverrides []ScheduledOverride `json:"scheduledOverrides,omitempty"`
 }
 
 type ScaleUpTrigger struct {
@@ -72,6 +78,12 @@ type GitHubEventScaleUpTriggerSpec struct {
 type CheckRunSpec struct {
 	Types  []string `json:"types,omitempty"`
 	Status string   `json:"status,omitempty"`
+
+	// Names is a list of GitHub Actions glob patterns.
+	// Any check_run event whose name matches one of patterns in the list can trigger autoscaling.
+	// Note that check_run name seem to equal to the job name you've defined in your actions workflow yaml file.
+	// So it is very likely that you can utilize this to trigger depending on the job.
+	Names []string `json:"names,omitempty"`
 }
 
 // https://docs.github.com/en/actions/reference/events-that-trigger-workflows#pull_request
@@ -138,6 +150,40 @@ type MetricSpec struct {
 	ScaleDownAdjustment int `json:"scaleDownAdjustment,omitempty"`
 }
 
+// ScheduledOverride can be used to override a few fields of HorizontalRunnerAutoscalerSpec on schedule.
+// A schedule can optionally be recurring, so that the correspoding override happens every day, week, month, or year.
+type ScheduledOverride struct {
+	// StartTime is the time at which the first override starts.
+	StartTime metav1.Time `json:"startTime"`
+
+	// EndTime is the time at which the first override ends.
+	EndTime metav1.Time `json:"endTime"`
+
+	// MinReplicas is the number of runners while overriding.
+	// If omitted, it doesn't override minReplicas.
+	// +optional
+	// +nullable
+	// +kubebuilder:validation:Minimum=0
+	MinReplicas *int `json:"minReplicas,omitempty"`
+
+	// +optional
+	RecurrenceRule RecurrenceRule `json:"recurrenceRule,omitempty"`
+}
+
+type RecurrenceRule struct {
+	// Frequency is the name of a predefined interval of each recurrence.
+	// The valid values are "Daily", "Weekly", "Monthly", and "Yearly".
+	// If empty, the corresponding override happens only once.
+	// +optional
+	// +kubebuilder:validation:Enum=Daily;Weekly;Monthly;Yearly
+	Frequency string `json:"frequency,omitempty"`
+
+	// UntilTime is the time of the final recurrence.
+	// If empty, the schedule recurs forever.
+	// +optional
+	UntilTime metav1.Time `json:"untilTime,omitempty"`
+}
+
 type HorizontalRunnerAutoscalerStatus struct {
 	// ObservedGeneration is the most recent generation observed for the target. It corresponds to e.g.
 	// RunnerDeployment's generation, which is updated on mutation by the API Server.
@@ -150,10 +196,16 @@ type HorizontalRunnerAutoscalerStatus struct {
 	DesiredReplicas *int `json:"desiredReplicas,omitempty"`
 
 	// +optional
+	// +nullable
 	LastSuccessfulScaleOutTime *metav1.Time `json:"lastSuccessfulScaleOutTime,omitempty"`
 
 	// +optional
 	CacheEntries []CacheEntry `json:"cacheEntries,omitempty"`
+
+	// ScheduledOverridesSummary is the summary of active and upcoming scheduled overrides to be shown in e.g. a column of a `kubectl get hra` output
+	// for observability.
+	// +optional
+	ScheduledOverridesSummary *string `json:"scheduledOverridesSummary,omitempty"`
 }
 
 const CacheEntryKeyDesiredReplicas = "desiredReplicas"
@@ -169,6 +221,7 @@ type CacheEntry struct {
 // +kubebuilder:printcolumn:JSONPath=".spec.minReplicas",name=Min,type=number
 // +kubebuilder:printcolumn:JSONPath=".spec.maxReplicas",name=Max,type=number
 // +kubebuilder:printcolumn:JSONPath=".status.desiredReplicas",name=Desired,type=number
+// +kubebuilder:printcolumn:JSONPath=".status.scheduledOverridesSummary",name=Schedule,type=string
 
 // HorizontalRunnerAutoscaler is the Schema for the horizontalrunnerautoscaler API
 type HorizontalRunnerAutoscaler struct {

api/v1alpha1/runner_types.go

@@ -19,6 +19,8 @@ package v1alpha1
 import (
 	"errors"
 
+	"k8s.io/apimachinery/pkg/api/resource"
+
 	corev1 "k8s.io/api/core/v1"
 	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
 )
@@ -43,11 +45,16 @@ type RunnerSpec struct {
 	// +optional
 	Group string `json:"group,omitempty"`
 
+	// +optional
+	Ephemeral *bool `json:"ephemeral,omitempty"`
+
 	// +optional
 	Containers []corev1.Container `json:"containers,omitempty"`
 	// +optional
 	DockerdContainerResources corev1.ResourceRequirements `json:"dockerdContainerResources,omitempty"`
 	// +optional
+	DockerVolumeMounts []corev1.VolumeMount `json:"dockerVolumeMounts,omitempty"`
+	// +optional
 	Resources corev1.ResourceRequirements `json:"resources,omitempty"`
 	// +optional
 	VolumeMounts []corev1.VolumeMount `json:"volumeMounts,omitempty"`
@@ -92,6 +99,19 @@ type RunnerSpec struct {
 	DockerdWithinRunnerContainer *bool `json:"dockerdWithinRunnerContainer,omitempty"`
 	// +optional
 	DockerEnabled *bool `json:"dockerEnabled,omitempty"`
+	// +optional
+	DockerMTU *int64 `json:"dockerMTU,omitempty"`
+	// +optional
+	DockerRegistryMirror *string `json:"dockerRegistryMirror,omitempty"`
+	// +optional
+	HostAliases []corev1.HostAlias `json:"hostAliases,omitempty"`
+	// +optional
+	VolumeSizeLimit *resource.Quantity `json:"volumeSizeLimit,omitempty"`
+
+	// RuntimeClassName is the container runtime configuration that containers should run under.
+	// More info: https://kubernetes.io/docs/concepts/containers/runtime-class
+	// +optional
+	RuntimeClassName *string `json:"runtimeClassName,omitempty"`
 }
 
 // ValidateRepository validates repository field.
@@ -119,10 +139,17 @@ func (rs *RunnerSpec) ValidateRepository() error {
 
 // RunnerStatus defines the observed state of Runner
 type RunnerStatus struct {
+	// +optional
 	Registration RunnerStatusRegistration `json:"registration"`
-	Phase        string                   `json:"phase"`
-	Reason       string                   `json:"reason"`
-	Message      string                   `json:"message"`
+	// +optional
+	Phase string `json:"phase,omitempty"`
+	// +optional
+	Reason string `json:"reason,omitempty"`
+	// +optional
+	Message string `json:"message,omitempty"`
+	// +optional
+	// +nullable
+	LastRegistrationCheckTime *metav1.Time `json:"lastRegistrationCheckTime,omitempty"`
 }
 
 // RunnerStatusRegistration contains runner registration status
@@ -142,6 +169,7 @@ type RunnerStatusRegistration struct {
 // +kubebuilder:printcolumn:JSONPath=".spec.repository",name=Repository,type=string
 // +kubebuilder:printcolumn:JSONPath=".spec.labels",name=Labels,type=string
 // +kubebuilder:printcolumn:JSONPath=".status.phase",name=Status,type=string
+// +kubebuilder:printcolumn:name="Age",type="date",JSONPath=".metadata.creationTimestamp"
 
 // Runner is the Schema for the runners API
 type Runner struct {

api/v1alpha1/runner_webhook.go

@@ -34,7 +34,7 @@ func (r *Runner) SetupWebhookWithManager(mgr ctrl.Manager) error {
 		Complete()
 }
 
-// +kubebuilder:webhook:path=/mutate-actions-summerwind-dev-v1alpha1-runner,verbs=create;update,mutating=true,failurePolicy=fail,groups=actions.summerwind.dev,resources=runners,versions=v1alpha1,name=mutate.runner.actions.summerwind.dev
+// +kubebuilder:webhook:path=/mutate-actions-summerwind-dev-v1alpha1-runner,verbs=create;update,mutating=true,failurePolicy=fail,groups=actions.summerwind.dev,resources=runners,versions=v1alpha1,name=mutate.runner.actions.summerwind.dev,sideEffects=None
 
 var _ webhook.Defaulter = &Runner{}
 
@@ -43,7 +43,7 @@ func (r *Runner) Default() {
 	// Nothing to do.
 }
 
-// +kubebuilder:webhook:path=/validate-actions-summerwind-dev-v1alpha1-runner,verbs=create;update,mutating=false,failurePolicy=fail,groups=actions.summerwind.dev,resources=runners,versions=v1alpha1,name=validate.runner.actions.summerwind.dev
+// +kubebuilder:webhook:path=/validate-actions-summerwind-dev-v1alpha1-runner,verbs=create;update,mutating=false,failurePolicy=fail,groups=actions.summerwind.dev,resources=runners,versions=v1alpha1,name=validate.runner.actions.summerwind.dev,sideEffects=None
 
 var _ webhook.Validator = &Runner{}
 

api/v1alpha1/runnerdeployment_types.go

@@ -25,30 +25,54 @@ const (
 	AutoscalingMetricTypePercentageRunnersBusy                        = "PercentageRunnersBusy"
 )
 
-// RunnerReplicaSetSpec defines the desired state of RunnerDeployment
+// RunnerDeploymentSpec defines the desired state of RunnerDeployment
 type RunnerDeploymentSpec struct {
 	// +optional
 	// +nullable
 	Replicas *int `json:"replicas,omitempty"`
 
-	Template RunnerTemplate `json:"template"`
+	// +optional
+	// +nullable
+	Selector *metav1.LabelSelector `json:"selector"`
+	Template RunnerTemplate        `json:"template"`
 }
 
 type RunnerDeploymentStatus struct {
-	AvailableReplicas int `json:"availableReplicas"`
-	ReadyReplicas     int `json:"readyReplicas"`
+	// See K8s deployment controller code for reference
+	// https://github.com/kubernetes/kubernetes/blob/ea0764452222146c47ec826977f49d7001b0ea8c/pkg/controller/deployment/sync.go#L487-L505
 
-	// Replicas is the total number of desired, non-terminated and latest pods to be set for the primary RunnerSet
+	// AvailableReplicas is the total number of available runners which have been successfully registered to GitHub and still running.
+	// This corresponds to the sum of status.availableReplicas of all the runner replica sets.
+	// +optional
+	AvailableReplicas *int `json:"availableReplicas"`
+
+	// ReadyReplicas is the total number of available runners which have been successfully registered to GitHub and still running.
+	// This corresponds to the sum of status.readyReplicas of all the runner replica sets.
+	// +optional
+	ReadyReplicas *int `json:"readyReplicas"`
+
+	// ReadyReplicas is the total number of available runners which have been successfully registered to GitHub and still running.
+	// This corresponds to status.replicas of the runner replica set that has the desired template hash.
+	// +optional
+	UpdatedReplicas *int `json:"updatedReplicas"`
+
+	// DesiredReplicas is the total number of desired, non-terminated and latest pods to be set for the primary RunnerSet
 	// This doesn't include outdated pods while upgrading the deployment and replacing the runnerset.
 	// +optional
-	Replicas *int `json:"desiredReplicas,omitempty"`
+	DesiredReplicas *int `json:"desiredReplicas"`
+
+	// Replicas is the total number of replicas
+	// +optional
+	Replicas *int `json:"replicas"`
 }
 
 // +kubebuilder:object:root=true
 // +kubebuilder:subresource:status
 // +kubebuilder:printcolumn:JSONPath=".spec.replicas",name=Desired,type=number
-// +kubebuilder:printcolumn:JSONPath=".status.availableReplicas",name=Current,type=number
-// +kubebuilder:printcolumn:JSONPath=".status.readyReplicas",name=Ready,type=number
+// +kubebuilder:printcolumn:JSONPath=".status.replicas",name=Current,type=number
+// +kubebuilder:printcolumn:JSONPath=".status.updatedReplicas",name=Up-To-Date,type=number
+// +kubebuilder:printcolumn:JSONPath=".status.availableReplicas",name=Available,type=number
+// +kubebuilder:printcolumn:name="Age",type="date",JSONPath=".metadata.creationTimestamp"
 
 // RunnerDeployment is the Schema for the runnerdeployments API
 type RunnerDeployment struct {

api/v1alpha1/runnerreplicaset_types.go

@@ -26,12 +26,26 @@ type RunnerReplicaSetSpec struct {
 	// +nullable
 	Replicas *int `json:"replicas,omitempty"`
 
-	Template RunnerTemplate `json:"template"`
+	// +optional
+	// +nullable
+	Selector *metav1.LabelSelector `json:"selector"`
+	Template RunnerTemplate        `json:"template"`
 }
 
 type RunnerReplicaSetStatus struct {
-	AvailableReplicas int `json:"availableReplicas"`
-	ReadyReplicas     int `json:"readyReplicas"`
+	// See K8s replicaset controller code for reference
+	// https://github.com/kubernetes/kubernetes/blob/ea0764452222146c47ec826977f49d7001b0ea8c/pkg/controller/replicaset/replica_set_utils.go#L101-L106
+
+	// Replicas is the number of runners that are created and still being managed by this runner replica set.
+	// +optional
+	Replicas *int `json:"replicas"`
+
+	// ReadyReplicas is the number of runners that are created and Runnning.
+	ReadyReplicas *int `json:"readyReplicas"`
+
+	// AvailableReplicas is the number of runners that are created and Runnning.
+	// This is currently same as ReadyReplicas but perserved for future use.
+	AvailableReplicas *int `json:"availableReplicas"`
 }
 
 type RunnerTemplate struct {
@@ -43,8 +57,9 @@ type RunnerTemplate struct {
 // +kubebuilder:object:root=true
 // +kubebuilder:subresource:status
 // +kubebuilder:printcolumn:JSONPath=".spec.replicas",name=Desired,type=number
-// +kubebuilder:printcolumn:JSONPath=".status.availableReplicas",name=Current,type=number
+// +kubebuilder:printcolumn:JSONPath=".status.replicas",name=Current,type=number
 // +kubebuilder:printcolumn:JSONPath=".status.readyReplicas",name=Ready,type=number
+// +kubebuilder:printcolumn:name="Age",type="date",JSONPath=".metadata.creationTimestamp"
 
 // RunnerReplicaSet is the Schema for the runnerreplicasets API
 type RunnerReplicaSet struct {

api/v1alpha1/runnerreplicaset_webhook.go

@@ -34,7 +34,7 @@ func (r *RunnerReplicaSet) SetupWebhookWithManager(mgr ctrl.Manager) error {
 		Complete()
 }
 
-// +kubebuilder:webhook:path=/mutate-actions-summerwind-dev-v1alpha1-runnerreplicaset,verbs=create;update,mutating=true,failurePolicy=fail,groups=actions.summerwind.dev,resources=runnerreplicasets,versions=v1alpha1,name=mutate.runnerreplicaset.actions.summerwind.dev
+// +kubebuilder:webhook:path=/mutate-actions-summerwind-dev-v1alpha1-runnerreplicaset,verbs=create;update,mutating=true,failurePolicy=fail,groups=actions.summerwind.dev,resources=runnerreplicasets,versions=v1alpha1,name=mutate.runnerreplicaset.actions.summerwind.dev,sideEffects=None
 
 var _ webhook.Defaulter = &RunnerReplicaSet{}
 
@@ -43,7 +43,7 @@ func (r *RunnerReplicaSet) Default() {
 	// Nothing to do.
 }
 
-// +kubebuilder:webhook:path=/validate-actions-summerwind-dev-v1alpha1-runnerreplicaset,verbs=create;update,mutating=false,failurePolicy=fail,groups=actions.summerwind.dev,resources=runnerreplicasets,versions=v1alpha1,name=validate.runnerreplicaset.actions.summerwind.dev
+// +kubebuilder:webhook:path=/validate-actions-summerwind-dev-v1alpha1-runnerreplicaset,verbs=create;update,mutating=false,failurePolicy=fail,groups=actions.summerwind.dev,resources=runnerreplicasets,versions=v1alpha1,name=validate.runnerreplicaset.actions.summerwind.dev,sideEffects=None
 
 var _ webhook.Validator = &RunnerReplicaSet{}
 

api/v1alpha1/zz_generated.deepcopy.go

@@ -22,6 +22,7 @@ package v1alpha1
 
 import (
 	"k8s.io/api/core/v1"
+	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
 	"k8s.io/apimachinery/pkg/runtime"
 )
 
@@ -65,6 +66,11 @@ func (in *CheckRunSpec) DeepCopyInto(out *CheckRunSpec) {
 		*out = make([]string, len(*in))
 		copy(*out, *in)
 	}
+	if in.Names != nil {
+		in, out := &in.Names, &out.Names
+		*out = make([]string, len(*in))
+		copy(*out, *in)
+	}
 }
 
 // DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new CheckRunSpec.
@@ -206,6 +212,13 @@ func (in *HorizontalRunnerAutoscalerSpec) DeepCopyInto(out *HorizontalRunnerAuto
 			(*in)[i].DeepCopyInto(&(*out)[i])
 		}
 	}
+	if in.ScheduledOverrides != nil {
+		in, out := &in.ScheduledOverrides, &out.ScheduledOverrides
+		*out = make([]ScheduledOverride, len(*in))
+		for i := range *in {
+			(*in)[i].DeepCopyInto(&(*out)[i])
+		}
+	}
 }
 
 // DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new HorizontalRunnerAutoscalerSpec.
@@ -237,6 +250,11 @@ func (in *HorizontalRunnerAutoscalerStatus) DeepCopyInto(out *HorizontalRunnerAu
 			(*in)[i].DeepCopyInto(&(*out)[i])
 		}
 	}
+	if in.ScheduledOverridesSummary != nil {
+		in, out := &in.ScheduledOverridesSummary, &out.ScheduledOverridesSummary
+		*out = new(string)
+		**out = **in
+	}
 }
 
 // DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new HorizontalRunnerAutoscalerStatus.
@@ -309,6 +327,22 @@ func (in *PushSpec) DeepCopy() *PushSpec {
 	return out
 }
 
+// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.
+func (in *RecurrenceRule) DeepCopyInto(out *RecurrenceRule) {
+	*out = *in
+	in.UntilTime.DeepCopyInto(&out.UntilTime)
+}
+
+// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new RecurrenceRule.
+func (in *RecurrenceRule) DeepCopy() *RecurrenceRule {
+	if in == nil {
+		return nil
+	}
+	out := new(RecurrenceRule)
+	in.DeepCopyInto(out)
+	return out
+}
+
 // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.
 func (in *Runner) DeepCopyInto(out *Runner) {
 	*out = *in
@@ -403,6 +437,11 @@ func (in *RunnerDeploymentSpec) DeepCopyInto(out *RunnerDeploymentSpec) {
 		*out = new(int)
 		**out = **in
 	}
+	if in.Selector != nil {
+		in, out := &in.Selector, &out.Selector
+		*out = new(metav1.LabelSelector)
+		(*in).DeepCopyInto(*out)
+	}
 	in.Template.DeepCopyInto(&out.Template)
 }
 
@@ -419,6 +458,26 @@ func (in *RunnerDeploymentSpec) DeepCopy() *RunnerDeploymentSpec {
 // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.
 func (in *RunnerDeploymentStatus) DeepCopyInto(out *RunnerDeploymentStatus) {
 	*out = *in
+	if in.AvailableReplicas != nil {
+		in, out := &in.AvailableReplicas, &out.AvailableReplicas
+		*out = new(int)
+		**out = **in
+	}
+	if in.ReadyReplicas != nil {
+		in, out := &in.ReadyReplicas, &out.ReadyReplicas
+		*out = new(int)
+		**out = **in
+	}
+	if in.UpdatedReplicas != nil {
+		in, out := &in.UpdatedReplicas, &out.UpdatedReplicas
+		*out = new(int)
+		**out = **in
+	}
+	if in.DesiredReplicas != nil {
+		in, out := &in.DesiredReplicas, &out.DesiredReplicas
+		*out = new(int)
+		**out = **in
+	}
 	if in.Replicas != nil {
 		in, out := &in.Replicas, &out.Replicas
 		*out = new(int)
@@ -474,7 +533,7 @@ func (in *RunnerReplicaSet) DeepCopyInto(out *RunnerReplicaSet) {
 	out.TypeMeta = in.TypeMeta
 	in.ObjectMeta.DeepCopyInto(&out.ObjectMeta)
 	in.Spec.DeepCopyInto(&out.Spec)
-	out.Status = in.Status
+	in.Status.DeepCopyInto(&out.Status)
 }
 
 // DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new RunnerReplicaSet.
@@ -535,6 +594,11 @@ func (in *RunnerReplicaSetSpec) DeepCopyInto(out *RunnerReplicaSetSpec) {
 		*out = new(int)
 		**out = **in
 	}
+	if in.Selector != nil {
+		in, out := &in.Selector, &out.Selector
+		*out = new(metav1.LabelSelector)
+		(*in).DeepCopyInto(*out)
+	}
 	in.Template.DeepCopyInto(&out.Template)
 }
 
@@ -551,6 +615,21 @@ func (in *RunnerReplicaSetSpec) DeepCopy() *RunnerReplicaSetSpec {
 // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.
 func (in *RunnerReplicaSetStatus) DeepCopyInto(out *RunnerReplicaSetStatus) {
 	*out = *in
+	if in.Replicas != nil {
+		in, out := &in.Replicas, &out.Replicas
+		*out = new(int)
+		**out = **in
+	}
+	if in.ReadyReplicas != nil {
+		in, out := &in.ReadyReplicas, &out.ReadyReplicas
+		*out = new(int)
+		**out = **in
+	}
+	if in.AvailableReplicas != nil {
+		in, out := &in.AvailableReplicas, &out.AvailableReplicas
+		*out = new(int)
+		**out = **in
+	}
 }
 
 // DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new RunnerReplicaSetStatus.
@@ -571,6 +650,11 @@ func (in *RunnerSpec) DeepCopyInto(out *RunnerSpec) {
 		*out = make([]string, len(*in))
 		copy(*out, *in)
 	}
+	if in.Ephemeral != nil {
+		in, out := &in.Ephemeral, &out.Ephemeral
+		*out = new(bool)
+		**out = **in
+	}
 	if in.Containers != nil {
 		in, out := &in.Containers, &out.Containers
 		*out = make([]v1.Container, len(*in))
@@ -579,6 +663,13 @@ func (in *RunnerSpec) DeepCopyInto(out *RunnerSpec) {
 		}
 	}
 	in.DockerdContainerResources.DeepCopyInto(&out.DockerdContainerResources)
+	if in.DockerVolumeMounts != nil {
+		in, out := &in.DockerVolumeMounts, &out.DockerVolumeMounts
+		*out = make([]v1.VolumeMount, len(*in))
+		for i := range *in {
+			(*in)[i].DeepCopyInto(&(*out)[i])
+		}
+	}
 	in.Resources.DeepCopyInto(&out.Resources)
 	if in.VolumeMounts != nil {
 		in, out := &in.VolumeMounts, &out.VolumeMounts
@@ -678,6 +769,33 @@ func (in *RunnerSpec) DeepCopyInto(out *RunnerSpec) {
 		*out = new(bool)
 		**out = **in
 	}
+	if in.DockerMTU != nil {
+		in, out := &in.DockerMTU, &out.DockerMTU
+		*out = new(int64)
+		**out = **in
+	}
+	if in.DockerRegistryMirror != nil {
+		in, out := &in.DockerRegistryMirror, &out.DockerRegistryMirror
+		*out = new(string)
+		**out = **in
+	}
+	if in.HostAliases != nil {
+		in, out := &in.HostAliases, &out.HostAliases
+		*out = make([]v1.HostAlias, len(*in))
+		for i := range *in {
+			(*in)[i].DeepCopyInto(&(*out)[i])
+		}
+	}
+	if in.VolumeSizeLimit != nil {
+		in, out := &in.VolumeSizeLimit, &out.VolumeSizeLimit
+		x := (*in).DeepCopy()
+		*out = &x
+	}
+	if in.RuntimeClassName != nil {
+		in, out := &in.RuntimeClassName, &out.RuntimeClassName
+		*out = new(string)
+		**out = **in
+	}
 }
 
 // DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new RunnerSpec.
@@ -694,6 +812,10 @@ func (in *RunnerSpec) DeepCopy() *RunnerSpec {
 func (in *RunnerStatus) DeepCopyInto(out *RunnerStatus) {
 	*out = *in
 	in.Registration.DeepCopyInto(&out.Registration)
+	if in.LastRegistrationCheckTime != nil {
+		in, out := &in.LastRegistrationCheckTime, &out.LastRegistrationCheckTime
+		*out = (*in).DeepCopy()
+	}
 }
 
 // DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new RunnerStatus.
@@ -779,3 +901,26 @@ func (in *ScaleUpTrigger) DeepCopy() *ScaleUpTrigger {
 	in.DeepCopyInto(out)
 	return out
 }
+
+// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.
+func (in *ScheduledOverride) DeepCopyInto(out *ScheduledOverride) {
+	*out = *in
+	in.StartTime.DeepCopyInto(&out.StartTime)
+	in.EndTime.DeepCopyInto(&out.EndTime)
+	if in.MinReplicas != nil {
+		in, out := &in.MinReplicas, &out.MinReplicas
+		*out = new(int)
+		**out = **in
+	}
+	in.RecurrenceRule.DeepCopyInto(&out.RecurrenceRule)
+}
+
+// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new ScheduledOverride.
+func (in *ScheduledOverride) DeepCopy() *ScheduledOverride {
+	if in == nil {
+		return nil
+	}
+	out := new(ScheduledOverride)
+	in.DeepCopyInto(out)
+	return out
+}

charts/actions-runner-controller/.helmignore

@@ -21,3 +21,5 @@
 .idea/
 *.tmproj
 .vscode/
+# Docs
+docs/

charts/actions-runner-controller/Chart.yaml

@@ -15,17 +15,16 @@ type: application
 # This is the chart version. This version number should be incremented each time you make changes
 # to the chart and its templates, including the app version.
 # Versions are expected to follow Semantic Versioning (https://semver.org/)
-version: 0.5.1
+version: 0.12.2
 
-home: https://github.com/summerwind/actions-runner-controller
+# Used as the default manager tag value when no tag property is provided in the values.yaml
+appVersion: 0.19.0
+
+home: https://github.com/actions-runner-controller/actions-runner-controller
 
 sources:
-    - https://github.com/summerwind/actions-runner-controller
+  - https://github.com/actions-runner-controller/actions-runner-controller
 
 maintainers:
-    - name: summerwind
-      email: contact@summerwind.jp
-      url: https://github.com/summerwind
-    - name: funkypenguin
-      email: davidy@funkypenguin.co.nz
-      url: https://www.funkypenguin.co.nz
+  - name: actions-runner-controller
+    url: https://github.com/actions-runner-controller

charts/actions-runner-controller/README.md

@@ -0,0 +1,82 @@
+## Docs
+
+All additional docs are kept in the `docs/` folder, this README is solely for documenting the values.yaml keys and values
+
+## Values
+
+_The values are documented as of HEAD_
+
+_Default values are the defaults set in the charts values.yaml, some properties have default configurations in the code for when the property is omitted or invalid_
+
+| Key                                                      | Description                                                                                                                | Default                                                              |
+|----------------------------------------------------------|----------------------------------------------------------------------------------------------------------------------------|----------------------------------------------------------------------|
+| `labels`                                                 | Set labels to apply to all resources in the chart                                                                          |                                                                      |
+| `replicaCount`                                           | Set the number of controller pods                                                                                          | 1                                                                    |
+| `syncPeriod`                                             | Set the period in which the controler reconciles the desired runners count                                                 | 10m                                                                  |
+| `githubAPICacheDuration`                                 | Set the cache period for API calls                                                                                         |                                                                      |
+| `logLevel`                                               | Set the log level of the controller container                                                                              |                                                                      |
+| `authSecret.create`                                      | Deploy the controller auth secret                                                                                          | true                                                                 |
+| `authSecret.name`                                        | Set the name of the auth secret                                                                                            | controller-manager                                                   |
+| `authSecret.github_app_id`                               | The ID of your GitHub App. **This can't be set at the same time as `authSecret.github_token`**                             |                                                                      |
+| `authSecret.github_app_installation_id`                  | The ID of your GitHub App installation. **This can't be set at the same time as `authSecret.github_token`**                |                                                                      |
+| `authSecret.github_app_private_key`                      | The multiline string of your GitHub App's private key. **This can't be set at the same time as `authSecret.github_token`** |                                                                      |
+| `authSecret.github_token`                                | Your chosen GitHub PAT token. **This can't be set at the same time as the `authSecret.github_app_*`**                      |                                                                      |
+| `image.repository`                                       | The "repository/image" of the controller container                                                                         | summerwind/actions-runner-controller                                 |
+| `image.tag`                                              | The tag of the controller container                                                                                        |                                                                      |
+| `image.dindSidecarRepositoryAndTag`                      | The "repository/image" of the dind sidecar container                                                                       | docker:dind                                                          |
+| `image.pullPolicy`                                       | The pull policy of the controller image                                                                                    | IfNotPresent                                                         |
+| `metrics.serviceMonitor`                                 | Deploy serviceMonitor kind for for use with prometheus-operator CRDs                                                       | false                                                                |
+| `metrics.port`                                           | Set port of metrics service                                                                                                | 8443                                                                 |
+| `metrics.proxy.enabled`                                  | Deploy kube-rbac-proxy container in controller pod                                                                         | true                                                                 |
+| `metrics.proxy.image.repository`                         | The "repository/image" of the kube-proxy container                                                                         | quay.io/brancz/kube-rbac-proxy                                       |
+| `metrics.proxy.image.tag`                                | The tag of the kube-proxy image to use when pulling the container                                                          | v0.10.0                                                              |
+| `imagePullSecrets`                                       | Specifies the secret to be used when pulling the controller pod containers                                                 |                                                                      |
+| `fullNameOverride`                                       | Override the full resource names	                                                                                        |                                                                      |
+| `nameOverride`                                           | Override the resource name prefix	                                                                                        |                                                                      |
+| `serviceAccont.annotations`                              | Set annotations to the service account                                                                                     |                                                                      |
+| `serviceAccount.create`                                  | Deploy the controller pod under a service account                                                                          | true                                                                 |
+| `podAnnotations`                                         | Set annotations for the controller pod                                                                                     |                                                                      |
+| `podLabels`                                              | Set labels for the controller pod                                                                                          |                                                                      |
+| `serviceAccount.name`                                    | Set the name of the service account                                                                                        |                                                                      |
+| `securityContext`                                        | Set the security context for each container in the controller pod                                                          |                                                                      |
+| `podSecurityContext`                                     | Set the security context to controller pod                                                                                 |                                                                      |
+| `service.port`                                           | Set controller service type                                                                                                |                                                                      |
+| `service.type`                                           | Set controller service ports                                                                                               |                                                                      |
+| `topologySpreadConstraints`                              | Set the controller pod topologySpreadConstraints                                                                           |                                                                      |
+| `nodeSelector`                                           | Set the controller pod nodeSelector                                                                                        |                                                                      |
+| `resources`                                              | Set the controller pod resources                                                                                           |                                                                      |
+| `affinity`                                               | Set the controller pod affinity rules                                                                                      |                                                                      |
+| `tolerations`                                            | Set the controller pod tolerations                                                                                         |                                                                      |
+| `env`                                                    | Set environment variables for the controller container                                                                     |                                                                      |
+| `priorityClassName`                                      | Set the controller pod priorityClassName                                                                                   |                                                                      |
+| `scope.watchNamespace`                                   | Tells the controller which namespace to watch if `scope.singleNamespace` is true                                           |                                                                      |
+| `scope.singleNamespace`                                  | Limit the controller to watch a single namespace                                                                           | false                                                                |
+| `githubWebhookServer.logLevel`                           | Set the log level of the githubWebhookServer container                                                                     |                                                                      |
+| `githubWebhookServer.replicaCount`                       | Set the number of webhook server pods                                                                                      | 1                                                                    |
+| `githubWebhookServer.syncPeriod`                         | Set the period in which the controller reconciles the resources                                                            | 10m                                                                  |
+| `githubWebhookServer.enabled`                            | Deploy the webhook server pod                                                                                              | false                                                                |
+| `githubWebhookServer.secret.create`                      | Deploy the webhook hook secret                                                                                             | true                                                                 |
+| `githubWebhookServer.secret.name`                        | Set the name of the webhook hook secret                                                                                    | github-webhook-server                                                |
+| `githubWebhookServer.secret.github_webhook_secret_token` | Set the webhook secret token value                                                                                         |                                                                      |
+| `githubWebhookServer.imagePullSecrets`                   | Specifies the secret to be used when pulling the githubWebhookServer pod containers                                        |                                                                      |
+| `githubWebhookServer.nameOveride`                        | Override the resource name prefix	                                                                                        |                                                                      |
+| `githubWebhookServer.fullNameOveride`                    | Override the full resource names	                                                                                        |                                                                      |
+| `githubWebhookServer.serviceAccount.create`              | Deploy the githubWebhookServer under a service account                                                                     | true                                                                 |
+| `githubWebhookServer.serviceAccount.annotations`         | Set annotations for the service account                                                                                    |                                                                      |
+| `githubWebhookServer.serviceAccount.name`                | Set the service account name                                                                                               |                                                                      |
+| `githubWebhookServer.podAnnotations`                     | Set annotations for the githubWebhookServer pod                                                                            |                                                                      |
+| `githubWebhookServer.podLabels`                          | Set labels for the githubWebhookServer pod                                                                                 |                                                                      |
+| `githubWebhookServer.podSecurityContext`                 | Set the security context to githubWebhookServer pod                                                                        |                                                                      |
+| `githubWebhookServer.securityContext`                    | Set the security context for each container in the githubWebhookServer pod                                                 |                                                                      |
+| `githubWebhookServer.resources`                          | Set the githubWebhookServer pod resources                                                                                  |                                                                      |
+| `githubWebhookServer.topologySpreadConstraints`          | Set the githubWebhookServer pod topologySpreadConstraints                                                                  |                                                                      |
+| `githubWebhookServer.nodeSelector`                       | Set the githubWebhookServer pod nodeSelector                                                                               |                                                                      |
+| `githubWebhookServer.tolerations`                        | Set the githubWebhookServer pod tolerations                                                                                |                                                                      |
+| `githubWebhookServer.affinity`                           | Set the githubWebhookServer pod affinity rules                                                                             |                                                                      |
+| `githubWebhookServer.priorityClassName`                  | Set the githubWebhookServer pod priorityClassName                                                                          |                                                                      |
+| `githubWebhookServer.service.type`                       | Set githubWebhookServer service type                                                                                       |                                                                      |
+| `githubWebhookServer.service.ports`                      | Set githubWebhookServer service ports                                                                                      | `[{"port":80, "targetPort:"http", "protocol":"TCP", "name":"http"}]` |
+| `githubWebhookServer.ingress.enabled`                    | Deploy an ingress kind for the githubWebhookServer                                                                         | false                                                                |
+| `githubWebhookServer.ingress.annotations`                | Set annotations for the ingress kind                                                                                       |                                                                      |
+| `githubWebhookServer.ingress.hosts`                      | Set hosts configuration for ingress                                                                                        | `[{"host": "chart-example.local", "paths": []}]`                     |
+| `githubWebhookServer.ingress.tls`                        | Set tls configuration for ingress                                                                                          |                                                                      |

charts/actions-runner-controller/ci/ci-values.yaml

@@ -22,6 +22,9 @@ resources:
     cpu: 100m
     memory: 128Mi  
 
+authSecret:
+  create: false
+
 # Set the following to true to create a dummy secret, allowing the manager pod to start
 # This is only useful in CI
 createDummySecret: true

charts/actions-runner-controller/crds/actions.summerwind.dev_horizontalrunnerautoscalers.yaml

@@ -18,6 +18,9 @@ spec:
   - JSONPath: .status.desiredReplicas
     name: Desired
     type: number
+  - JSONPath: .status.scheduledOverridesSummary
+    name: Schedule
+    type: string
   group: actions.summerwind.dev
   names:
     kind: HorizontalRunnerAutoscaler
@@ -148,6 +151,17 @@ spec:
                       checkRun:
                         description: https://docs.github.com/en/actions/reference/events-that-trigger-workflows#check_run
                         properties:
+                          names:
+                            description: Names is a list of GitHub Actions glob patterns.
+                              Any check_run event whose name matches one of patterns
+                              in the list can trigger autoscaling. Note that check_run
+                              name seem to equal to the job name you've defined in
+                              your actions workflow yaml file. So it is very likely
+                              that you can utilize this to trigger depending on the
+                              job.
+                            items:
+                              type: string
+                            type: array
                           status:
                             type: string
                           types:
@@ -174,6 +188,56 @@ spec:
                     type: object
                 type: object
               type: array
+            scheduledOverrides:
+              description: ScheduledOverrides is the list of ScheduledOverride. It
+                can be used to override a few fields of HorizontalRunnerAutoscalerSpec
+                on schedule. The earlier a scheduled override is, the higher it is
+                prioritized.
+              items:
+                description: ScheduledOverride can be used to override a few fields
+                  of HorizontalRunnerAutoscalerSpec on schedule. A schedule can optionally
+                  be recurring, so that the correspoding override happens every day,
+                  week, month, or year.
+                properties:
+                  endTime:
+                    description: EndTime is the time at which the first override ends.
+                    format: date-time
+                    type: string
+                  minReplicas:
+                    description: MinReplicas is the number of runners while overriding.
+                      If omitted, it doesn't override minReplicas.
+                    minimum: 0
+                    nullable: true
+                    type: integer
+                  recurrenceRule:
+                    properties:
+                      frequency:
+                        description: Frequency is the name of a predefined interval
+                          of each recurrence. The valid values are "Daily", "Weekly",
+                          "Monthly", and "Yearly". If empty, the corresponding override
+                          happens only once.
+                        enum:
+                        - Daily
+                        - Weekly
+                        - Monthly
+                        - Yearly
+                        type: string
+                      untilTime:
+                        description: UntilTime is the time of the final recurrence.
+                          If empty, the schedule recurs forever.
+                        format: date-time
+                        type: string
+                    type: object
+                  startTime:
+                    description: StartTime is the time at which the first override
+                      starts.
+                    format: date-time
+                    type: string
+                required:
+                - endTime
+                - startTime
+                type: object
+              type: array
           type: object
         status:
           properties:
@@ -196,6 +260,7 @@ spec:
               type: integer
             lastSuccessfulScaleOutTime:
               format: date-time
+              nullable: true
               type: string
             observedGeneration:
               description: ObservedGeneration is the most recent generation observed
@@ -203,6 +268,11 @@ spec:
                 which is updated on mutation by the API Server.
               format: int64
               type: integer
+            scheduledOverridesSummary:
+              description: ScheduledOverridesSummary is the summary of active and
+                upcoming scheduled overrides to be shown in e.g. a column of a `kubectl
+                get hra` output for observability.
+              type: string
           type: object
       type: object
   version: v1alpha1

charts/actions-runner-controller/crds/actions.summerwind.dev_runnerdeployments.yaml

@@ -10,12 +10,18 @@ spec:
     - JSONPath: .spec.replicas
       name: Desired
       type: number
-    - JSONPath: .status.availableReplicas
+    - JSONPath: .status.replicas
       name: Current
       type: number
-    - JSONPath: .status.readyReplicas
-      name: Ready
+    - JSONPath: .status.updatedReplicas
+      name: Up-To-Date
       type: number
+    - JSONPath: .status.availableReplicas
+      name: Available
+      type: number
+    - JSONPath: .metadata.creationTimestamp
+      name: Age
+      type: date
   group: actions.summerwind.dev
   names:
     kind: RunnerDeployment
@@ -38,11 +44,42 @@ spec:
         metadata:
           type: object
         spec:
-          description: RunnerReplicaSetSpec defines the desired state of RunnerDeployment
+          description: RunnerDeploymentSpec defines the desired state of RunnerDeployment
           properties:
             replicas:
               nullable: true
               type: integer
+            selector:
+              description: A label selector is a label query over a set of resources. The result of matchLabels and matchExpressions are ANDed. An empty label selector matches all objects. A null label selector matches no objects.
+              nullable: true
+              properties:
+                matchExpressions:
+                  description: matchExpressions is a list of label selector requirements. The requirements are ANDed.
+                  items:
+                    description: A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.
+                    properties:
+                      key:
+                        description: key is the label key that the selector applies to.
+                        type: string
+                      operator:
+                        description: operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.
+                        type: string
+                      values:
+                        description: values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.
+                        items:
+                          type: string
+                        type: array
+                    required:
+                      - key
+                      - operator
+                    type: object
+                  type: array
+                matchLabels:
+                  additionalProperties:
+                    type: string
+                  description: matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is "key", the operator is "In", and the values array contains only "value". The requirements are ANDed.
+                  type: object
+              type: object
             template:
               properties:
                 metadata:
@@ -402,6 +439,38 @@ spec:
                       type: array
                     dockerEnabled:
                       type: boolean
+                    dockerMTU:
+                      format: int64
+                      type: integer
+                    dockerRegistryMirror:
+                      type: string
+                    dockerVolumeMounts:
+                      items:
+                        description: VolumeMount describes a mounting of a Volume within a container.
+                        properties:
+                          mountPath:
+                            description: Path within the container at which the volume should be mounted.  Must not contain ':'.
+                            type: string
+                          mountPropagation:
+                            description: mountPropagation determines how mounts are propagated from the host to container and the other way around. When not set, MountPropagationNone is used. This field is beta in 1.10.
+                            type: string
+                          name:
+                            description: This must match the Name of a Volume.
+                            type: string
+                          readOnly:
+                            description: Mounted read-only if true, read-write otherwise (false or unspecified). Defaults to false.
+                            type: boolean
+                          subPath:
+                            description: Path within the volume from which the container's volume should be mounted. Defaults to "" (volume's root).
+                            type: string
+                          subPathExpr:
+                            description: Expanded path within the volume from which the container's volume should be mounted. Behaves similarly to SubPath but environment variable references $(VAR_NAME) are expanded using the container's environment. Defaults to "" (volume's root). SubPathExpr and SubPath are mutually exclusive. This field is beta in 1.15.
+                            type: string
+                        required:
+                          - mountPath
+                          - name
+                        type: object
+                      type: array
                     dockerdContainerResources:
                       description: ResourceRequirements describes the compute resource requirements.
                       properties:
@@ -537,6 +606,8 @@ spec:
                             type: object
                         type: object
                       type: array
+                    ephemeral:
+                      type: boolean
                     ephemeralContainers:
                       items:
                         description: An EphemeralContainer is a container that may be added temporarily to an existing pod for user-initiated activities such as debugging. Ephemeral containers have no resource or scheduling guarantees, and they will not be restarted when they exit or when a pod is removed or restarted. If an ephemeral container causes a pod to exceed its resource allocation, the pod may be evicted. Ephemeral containers may not be added by directly updating the pod spec. They must be added via the pod's ephemeralcontainers subresource, and they will appear in the pod spec once added. This is an alpha feature enabled by the EphemeralContainers feature flag.
@@ -546,6 +617,20 @@ spec:
                       type: array
                     group:
                       type: string
+                    hostAliases:
+                      items:
+                        description: HostAlias holds the mapping between IP and hostnames that will be injected as an entry in the pod's hosts file.
+                        properties:
+                          hostnames:
+                            description: Hostnames for the above IP address.
+                            items:
+                              type: string
+                            type: array
+                          ip:
+                            description: IP address of the host file entry.
+                            type: string
+                        type: object
+                      type: array
                     image:
                       type: string
                     imagePullPolicy:
@@ -603,6 +688,9 @@ spec:
                           description: 'Requests describes the minimum amount of compute resources required. If Requests is omitted for a container, it defaults to Limits if that is explicitly specified, otherwise to an implementation-defined value. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/'
                           type: object
                       type: object
+                    runtimeClassName:
+                      description: 'RuntimeClassName is the container runtime configuration that containers should run under. More info: https://kubernetes.io/docs/concepts/containers/runtime-class'
+                      type: string
                     securityContext:
                       description: PodSecurityContext holds pod-level security attributes and common container settings. Some fields are also present in container.securityContext.  Field values of container.securityContext take precedence over field values of PodSecurityContext.
                       properties:
@@ -734,6 +822,12 @@ spec:
                           - name
                         type: object
                       type: array
+                    volumeSizeLimit:
+                      anyOf:
+                        - type: integer
+                        - type: string
+                      pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
+                      x-kubernetes-int-or-string: true
                     volumes:
                       items:
                         description: Volume represents a named volume in a pod that may be accessed by any container in the pod.
@@ -1546,15 +1640,20 @@ spec:
         status:
           properties:
             availableReplicas:
+              description: AvailableReplicas is the total number of available runners which have been successfully registered to GitHub and still running. This corresponds to the sum of status.availableReplicas of all the runner replica sets.
               type: integer
             desiredReplicas:
-              description: Replicas is the total number of desired, non-terminated and latest pods to be set for the primary RunnerSet This doesn't include outdated pods while upgrading the deployment and replacing the runnerset.
+              description: DesiredReplicas is the total number of desired, non-terminated and latest pods to be set for the primary RunnerSet This doesn't include outdated pods while upgrading the deployment and replacing the runnerset.
               type: integer
             readyReplicas:
+              description: ReadyReplicas is the total number of available runners which have been successfully registered to GitHub and still running. This corresponds to the sum of status.readyReplicas of all the runner replica sets.
+              type: integer
+            replicas:
+              description: Replicas is the total number of replicas
+              type: integer
+            updatedReplicas:
+              description: ReadyReplicas is the total number of available runners which have been successfully registered to GitHub and still running. This corresponds to status.replicas of the runner replica set that has the desired template hash.
               type: integer
-          required:
-            - availableReplicas
-            - readyReplicas
           type: object
       type: object
   version: v1alpha1

charts/actions-runner-controller/crds/actions.summerwind.dev_runnerreplicasets.yaml

@@ -10,12 +10,15 @@ spec:
     - JSONPath: .spec.replicas
       name: Desired
       type: number
-    - JSONPath: .status.availableReplicas
+    - JSONPath: .status.replicas
       name: Current
       type: number
     - JSONPath: .status.readyReplicas
       name: Ready
       type: number
+    - JSONPath: .metadata.creationTimestamp
+      name: Age
+      type: date
   group: actions.summerwind.dev
   names:
     kind: RunnerReplicaSet
@@ -43,6 +46,37 @@ spec:
             replicas:
               nullable: true
               type: integer
+            selector:
+              description: A label selector is a label query over a set of resources. The result of matchLabels and matchExpressions are ANDed. An empty label selector matches all objects. A null label selector matches no objects.
+              nullable: true
+              properties:
+                matchExpressions:
+                  description: matchExpressions is a list of label selector requirements. The requirements are ANDed.
+                  items:
+                    description: A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.
+                    properties:
+                      key:
+                        description: key is the label key that the selector applies to.
+                        type: string
+                      operator:
+                        description: operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.
+                        type: string
+                      values:
+                        description: values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.
+                        items:
+                          type: string
+                        type: array
+                    required:
+                      - key
+                      - operator
+                    type: object
+                  type: array
+                matchLabels:
+                  additionalProperties:
+                    type: string
+                  description: matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is "key", the operator is "In", and the values array contains only "value". The requirements are ANDed.
+                  type: object
+              type: object
             template:
               properties:
                 metadata:
@@ -402,6 +436,38 @@ spec:
                       type: array
                     dockerEnabled:
                       type: boolean
+                    dockerMTU:
+                      format: int64
+                      type: integer
+                    dockerRegistryMirror:
+                      type: string
+                    dockerVolumeMounts:
+                      items:
+                        description: VolumeMount describes a mounting of a Volume within a container.
+                        properties:
+                          mountPath:
+                            description: Path within the container at which the volume should be mounted.  Must not contain ':'.
+                            type: string
+                          mountPropagation:
+                            description: mountPropagation determines how mounts are propagated from the host to container and the other way around. When not set, MountPropagationNone is used. This field is beta in 1.10.
+                            type: string
+                          name:
+                            description: This must match the Name of a Volume.
+                            type: string
+                          readOnly:
+                            description: Mounted read-only if true, read-write otherwise (false or unspecified). Defaults to false.
+                            type: boolean
+                          subPath:
+                            description: Path within the volume from which the container's volume should be mounted. Defaults to "" (volume's root).
+                            type: string
+                          subPathExpr:
+                            description: Expanded path within the volume from which the container's volume should be mounted. Behaves similarly to SubPath but environment variable references $(VAR_NAME) are expanded using the container's environment. Defaults to "" (volume's root). SubPathExpr and SubPath are mutually exclusive. This field is beta in 1.15.
+                            type: string
+                        required:
+                          - mountPath
+                          - name
+                        type: object
+                      type: array
                     dockerdContainerResources:
                       description: ResourceRequirements describes the compute resource requirements.
                       properties:
@@ -537,6 +603,8 @@ spec:
                             type: object
                         type: object
                       type: array
+                    ephemeral:
+                      type: boolean
                     ephemeralContainers:
                       items:
                         description: An EphemeralContainer is a container that may be added temporarily to an existing pod for user-initiated activities such as debugging. Ephemeral containers have no resource or scheduling guarantees, and they will not be restarted when they exit or when a pod is removed or restarted. If an ephemeral container causes a pod to exceed its resource allocation, the pod may be evicted. Ephemeral containers may not be added by directly updating the pod spec. They must be added via the pod's ephemeralcontainers subresource, and they will appear in the pod spec once added. This is an alpha feature enabled by the EphemeralContainers feature flag.
@@ -546,6 +614,20 @@ spec:
                       type: array
                     group:
                       type: string
+                    hostAliases:
+                      items:
+                        description: HostAlias holds the mapping between IP and hostnames that will be injected as an entry in the pod's hosts file.
+                        properties:
+                          hostnames:
+                            description: Hostnames for the above IP address.
+                            items:
+                              type: string
+                            type: array
+                          ip:
+                            description: IP address of the host file entry.
+                            type: string
+                        type: object
+                      type: array
                     image:
                       type: string
                     imagePullPolicy:
@@ -603,6 +685,9 @@ spec:
                           description: 'Requests describes the minimum amount of compute resources required. If Requests is omitted for a container, it defaults to Limits if that is explicitly specified, otherwise to an implementation-defined value. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/'
                           type: object
                       type: object
+                    runtimeClassName:
+                      description: 'RuntimeClassName is the container runtime configuration that containers should run under. More info: https://kubernetes.io/docs/concepts/containers/runtime-class'
+                      type: string
                     securityContext:
                       description: PodSecurityContext holds pod-level security attributes and common container settings. Some fields are also present in container.securityContext.  Field values of container.securityContext take precedence over field values of PodSecurityContext.
                       properties:
@@ -734,6 +819,12 @@ spec:
                           - name
                         type: object
                       type: array
+                    volumeSizeLimit:
+                      anyOf:
+                        - type: integer
+                        - type: string
+                      pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
+                      x-kubernetes-int-or-string: true
                     volumes:
                       items:
                         description: Volume represents a named volume in a pod that may be accessed by any container in the pod.
@@ -1546,8 +1637,13 @@ spec:
         status:
           properties:
             availableReplicas:
+              description: AvailableReplicas is the number of runners that are created and Runnning. This is currently same as ReadyReplicas but perserved for future use.
               type: integer
             readyReplicas:
+              description: ReadyReplicas is the number of runners that are created and Runnning.
+              type: integer
+            replicas:
+              description: Replicas is the number of runners that are created and still being managed by this runner replica set.
               type: integer
           required:
             - availableReplicas

charts/actions-runner-controller/crds/actions.summerwind.dev_runners.yaml

@@ -22,6 +22,9 @@ spec:
     - JSONPath: .status.phase
       name: Status
       type: string
+    - JSONPath: .metadata.creationTimestamp
+      name: Age
+      type: date
   group: actions.summerwind.dev
   names:
     kind: Runner
@@ -398,6 +401,38 @@ spec:
               type: array
             dockerEnabled:
               type: boolean
+            dockerMTU:
+              format: int64
+              type: integer
+            dockerRegistryMirror:
+              type: string
+            dockerVolumeMounts:
+              items:
+                description: VolumeMount describes a mounting of a Volume within a container.
+                properties:
+                  mountPath:
+                    description: Path within the container at which the volume should be mounted.  Must not contain ':'.
+                    type: string
+                  mountPropagation:
+                    description: mountPropagation determines how mounts are propagated from the host to container and the other way around. When not set, MountPropagationNone is used. This field is beta in 1.10.
+                    type: string
+                  name:
+                    description: This must match the Name of a Volume.
+                    type: string
+                  readOnly:
+                    description: Mounted read-only if true, read-write otherwise (false or unspecified). Defaults to false.
+                    type: boolean
+                  subPath:
+                    description: Path within the volume from which the container's volume should be mounted. Defaults to "" (volume's root).
+                    type: string
+                  subPathExpr:
+                    description: Expanded path within the volume from which the container's volume should be mounted. Behaves similarly to SubPath but environment variable references $(VAR_NAME) are expanded using the container's environment. Defaults to "" (volume's root). SubPathExpr and SubPath are mutually exclusive. This field is beta in 1.15.
+                    type: string
+                required:
+                  - mountPath
+                  - name
+                type: object
+              type: array
             dockerdContainerResources:
               description: ResourceRequirements describes the compute resource requirements.
               properties:
@@ -533,6 +568,8 @@ spec:
                     type: object
                 type: object
               type: array
+            ephemeral:
+              type: boolean
             ephemeralContainers:
               items:
                 description: An EphemeralContainer is a container that may be added temporarily to an existing pod for user-initiated activities such as debugging. Ephemeral containers have no resource or scheduling guarantees, and they will not be restarted when they exit or when a pod is removed or restarted. If an ephemeral container causes a pod to exceed its resource allocation, the pod may be evicted. Ephemeral containers may not be added by directly updating the pod spec. They must be added via the pod's ephemeralcontainers subresource, and they will appear in the pod spec once added. This is an alpha feature enabled by the EphemeralContainers feature flag.
@@ -542,6 +579,20 @@ spec:
               type: array
             group:
               type: string
+            hostAliases:
+              items:
+                description: HostAlias holds the mapping between IP and hostnames that will be injected as an entry in the pod's hosts file.
+                properties:
+                  hostnames:
+                    description: Hostnames for the above IP address.
+                    items:
+                      type: string
+                    type: array
+                  ip:
+                    description: IP address of the host file entry.
+                    type: string
+                type: object
+              type: array
             image:
               type: string
             imagePullPolicy:
@@ -599,6 +650,9 @@ spec:
                   description: 'Requests describes the minimum amount of compute resources required. If Requests is omitted for a container, it defaults to Limits if that is explicitly specified, otherwise to an implementation-defined value. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/'
                   type: object
               type: object
+            runtimeClassName:
+              description: 'RuntimeClassName is the container runtime configuration that containers should run under. More info: https://kubernetes.io/docs/concepts/containers/runtime-class'
+              type: string
             securityContext:
               description: PodSecurityContext holds pod-level security attributes and common container settings. Some fields are also present in container.securityContext.  Field values of container.securityContext take precedence over field values of PodSecurityContext.
               properties:
@@ -730,6 +784,12 @@ spec:
                   - name
                 type: object
               type: array
+            volumeSizeLimit:
+              anyOf:
+                - type: integer
+                - type: string
+              pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
+              x-kubernetes-int-or-string: true
             volumes:
               items:
                 description: Volume represents a named volume in a pod that may be accessed by any container in the pod.
@@ -1538,6 +1598,10 @@ spec:
         status:
           description: RunnerStatus defines the observed state of Runner
           properties:
+            lastRegistrationCheckTime:
+              format: date-time
+              nullable: true
+              type: string
             message:
               type: string
             phase:
@@ -1566,11 +1630,6 @@ spec:
                 - expiresAt
                 - token
               type: object
-          required:
-            - message
-            - phase
-            - reason
-            - registration
           type: object
       type: object
   version: v1alpha1

charts/actions-runner-controller/docs/UPGRADING.md

@@ -0,0 +1,40 @@
+## Upgrading
+
+This project makes extensive use of CRDs to provide much of its functionality. Helm unfortunately does not support [managing](https://helm.sh/docs/chart_best_practices/custom_resource_definitions/) CRDs by design:
+
+_The full breakdown as to how they came to this decision and why they have taken the approach they have for dealing with CRDs can be found in [Helm Improvement Proposal 11](https://github.com/helm/community/blob/main/hips/hip-0011.md)_
+
+```
+There is no support at this time for upgrading or deleting CRDs using Helm. This was an explicit decision after much 
+community discussion due to the danger for unintentional data loss. Furthermore, there is currently no community 
+consensus around how to handle CRDs and their lifecycle. As this evolves, Helm will add support for those use cases.
+```
+
+Helm will do an initial install of CRDs but it will not touch them afterwards (update or delete).
+
+Additionally, because the project leverages CRDs so extensively you **MUST** run the matching controller app container with its matching CRDs i.e. always redeploy your CRDs if you are changing the app version.
+
+Due to the above you can't just do a `helm upgrade` to release the latest version of the chart, the best practice steps are recorded below:
+
+## Steps
+
+1. Upgrade CRDs
+
+```shell
+ # REMEMBER TO UPDATE THE CHART_VERSION TO RELEVANT CHART VERISON!!!!
+CHART_VERSION=0.11.0
+
+curl -L https://github.com/actions-runner-controller/actions-runner-controller/releases/download/actions-runner-controller-${CHART_VERSION}/actions-runner-controller-${CHART_VERSION}.tgz | tar zxv --strip 1 actions-runner-controller/crds
+
+kubectl apply -f crds/
+```
+
+2. Upgrade the Helm release
+
+```shell
+helm upgrade --install \
+  --namespace actions-runner-system \
+  --version ${CHART_VERSION} \
+  actions-runner-controller/actions-runner-controller \
+  actions-runner-controller
+```

charts/actions-runner-controller/templates/NOTES.txt

@@ -1,8 +1,8 @@
 1. Get the application URL by running these commands:
-{{- if .Values.ingress.enabled }}
-{{- range $host := .Values.ingress.hosts }}
+{{- if .Values.githubWebhookServer.ingress.enabled }}
+{{- range $host := .Values.githubWebhookServer.ingress.hosts }}
   {{- range .paths }}
-  http{{ if $.Values.ingress.tls }}s{{ end }}://{{ $host.host }}{{ . }}
+  http{{ if $.Values.githubWebhookServer.ingress.tls }}s{{ end }}://{{ $host.host }}{{ . }}
   {{- end }}
 {{- end }}
 {{- else if contains "NodePort" .Values.service.type }}

charts/actions-runner-controller/templates/_github_webhook_server_helpers.tpl

@@ -47,6 +47,14 @@ Create the name of the service account to use
 {{- end }}
 {{- end }}
 
+{{- define "actions-runner-controller-github-webhook-server.secretName" -}}
+{{- default (include "actions-runner-controller-github-webhook-server.fullname" .) .Values.githubWebhookServer.secret.name }}
+{{- end }}
+
 {{- define "actions-runner-controller-github-webhook-server.roleName" -}}
 {{- include "actions-runner-controller-github-webhook-server.fullname" . }}
 {{- end }}
+
+{{- define "actions-runner-controller-github-webhook-server.serviceMonitorName" -}}
+{{- include "actions-runner-controller-github-webhook-server.fullname" . | trunc 47 }}-service-monitor
+{{- end }}

charts/actions-runner-controller/templates/_helpers.tpl

@@ -64,6 +64,10 @@ Create the name of the service account to use
 {{- end }}
 {{- end }}
 
+{{- define "actions-runner-controller.secretName" -}}
+{{- default (include "actions-runner-controller.fullname" .) .Values.authSecret.name -}}
+{{- end }}
+
 {{- define "actions-runner-controller.leaderElectionRoleName" -}}
 {{- include "actions-runner-controller.fullname" . }}-leader-election
 {{- end }}
@@ -88,10 +92,14 @@ Create the name of the service account to use
 {{- include "actions-runner-controller.fullname" . | trunc 55 }}-webhook
 {{- end }}
 
-{{- define "actions-runner-controller.authProxyServiceName" -}}
+{{- define "actions-runner-controller.metricsServiceName" -}}
 {{- include "actions-runner-controller.fullname" . | trunc 47 }}-metrics-service
 {{- end }}
 
+{{- define "actions-runner-controller.serviceMonitorName" -}}
+{{- include "actions-runner-controller.fullname" . | trunc 47 }}-service-monitor
+{{- end }}
+
 {{- define "actions-runner-controller.selfsignedIssuerName" -}}
 {{- include "actions-runner-controller.fullname" . }}-selfsigned-issuer
 {{- end }}

charts/actions-runner-controller/templates/auth_proxy_role.yaml

@@ -1,3 +1,4 @@
+{{- if .Values.metrics.proxy.enabled }}
 apiVersion: rbac.authorization.k8s.io/v1
 kind: ClusterRole
 metadata:
@@ -11,3 +12,4 @@ rules:
   resources:
   - subjectaccessreviews
   verbs: ["create"]
+{{- end }}

charts/actions-runner-controller/templates/auth_proxy_role_binding.yaml

@@ -1,3 +1,4 @@
+{{- if .Values.metrics.proxy.enabled }}
 apiVersion: rbac.authorization.k8s.io/v1
 kind: ClusterRoleBinding
 metadata:
@@ -10,3 +11,4 @@ subjects:
 - kind: ServiceAccount
   name: {{ include "actions-runner-controller.serviceAccountName" . }}
   namespace: {{ .Release.Namespace }}
+{{- end }}

charts/actions-runner-controller/templates/certificate.yaml

@@ -5,7 +5,7 @@ apiVersion: cert-manager.io/v1
 kind: Issuer
 metadata:
   name: {{ include "actions-runner-controller.selfsignedIssuerName" . }}
-  namespace: {{ .Namespace }}
+  namespace: {{ .Release.Namespace }}
 spec:
   selfSigned: {}
 ---
@@ -13,7 +13,7 @@ apiVersion: cert-manager.io/v1
 kind: Certificate
 metadata:
   name: {{ include "actions-runner-controller.servingCertName" . }}
-  namespace: {{ .Namespace }}
+  namespace: {{ .Release.Namespace }}
 spec:
   dnsNames:
   - {{ include "actions-runner-controller.webhookServiceName" . }}.{{ .Release.Namespace }}.svc

charts/actions-runner-controller/templates/controller.metrics.service.yaml

@@ -3,12 +3,12 @@ kind: Service
 metadata:
   labels:
     {{- include "actions-runner-controller.labels" . | nindent 4 }}
-  name: {{ include "actions-runner-controller.authProxyServiceName" . }}
+  name: {{ include "actions-runner-controller.metricsServiceName" . }}
   namespace: {{ .Release.Namespace }}
 spec:
   ports:
-  - name: https
-    port: 8443
-    targetPort: https
+  - name: metrics-port
+    port: {{ .Values.metrics.port }}
+    targetPort: metrics-port
   selector:
     {{- include "actions-runner-controller.selectorLabels" . | nindent 4 }}

charts/actions-runner-controller/templates/controller.metrics.serviceMonitor.yaml

@@ -0,0 +1,15 @@
+{{- if .Values.metrics.serviceMonitor }}
+apiVersion: monitoring.coreos.com/v1
+kind: ServiceMonitor
+metadata:
+  labels:
+    {{- include "actions-runner-controller.labels" . | nindent 4 }}
+  name: {{ include "actions-runner-controller.serviceMonitorName" . }}
+spec:
+  endpoints:
+    - path: /metrics
+      port: metrics-port
+  selector:
+    matchLabels:
+      {{- include "actions-runner-controller.selectorLabels" . | nindent 6 }}
+{{- end }}

charts/actions-runner-controller/templates/deployment.yaml

@@ -6,6 +6,7 @@ metadata:
   labels:
     {{- include "actions-runner-controller.labels" . | nindent 4 }}
 spec:
+  replicas: {{ .Values.replicaCount }}
   selector:
     matchLabels:
       {{- include "actions-runner-controller.selectorLabels" . | nindent 6 }}
@@ -17,6 +18,9 @@ spec:
       {{- end }}
       labels:
         {{- include "actions-runner-controller.selectorLabels" . | nindent 8 }}
+      {{- with .Values.podLabels }}
+        {{- toYaml . | nindent 8 }}
+      {{- end }}
     spec:
       {{- with .Values.imagePullSecrets }}
       imagePullSecrets:
@@ -30,10 +34,21 @@ spec:
       {{- end }}
       containers:
       - args:
-        - "--metrics-addr=127.0.0.1:8080"
+        {{- $metricsHost := .Values.metrics.proxy.enabled | ternary "127.0.0.1" "0.0.0.0" }}
+        {{- $metricsPort := .Values.metrics.proxy.enabled | ternary "8080" .Values.metrics.port }}
+        - "--metrics-addr={{ $metricsHost }}:{{ $metricsPort }}"
         - "--enable-leader-election"
         - "--sync-period={{ .Values.syncPeriod }}"
         - "--docker-image={{ .Values.image.dindSidecarRepositoryAndTag }}"
+        {{- if .Values.scope.singleNamespace }}
+        - "--watch-namespace={{ default .Release.Namespace .Values.scope.watchNamespace }}"
+        {{- end }}
+        {{- if .Values.githubAPICacheDuration }}
+        - "--github-api-cache-duration={{ .Values.githubAPICacheDuration }}"
+        {{- end }}
+        {{- if .Values.logLevel }}
+        - "--log-level={{ .Values.logLevel }}"
+        {{- end }}
         command:
         - "/manager"
         env:
@@ -41,19 +56,19 @@ spec:
           valueFrom:
             secretKeyRef:
               key: github_token
-              name: controller-manager
+              name: {{ include "actions-runner-controller.secretName" . }}
               optional: true
         - name: GITHUB_APP_ID
           valueFrom:
             secretKeyRef:
               key: github_app_id
-              name: controller-manager
+              name: {{ include "actions-runner-controller.secretName" . }}
               optional: true
         - name: GITHUB_APP_INSTALLATION_ID
           valueFrom:
             secretKeyRef:
               key: github_app_installation_id
-              name: controller-manager
+              name: {{ include "actions-runner-controller.secretName" . }}
               optional: true
         - name: GITHUB_APP_PRIVATE_KEY
           value: /etc/actions-runner-controller/github_app_private_key
@@ -68,39 +83,46 @@ spec:
         - containerPort: 9443
           name: webhook-server
           protocol: TCP
+        {{- if not .Values.metrics.proxy.enabled }}
+        - containerPort: {{ .Values.metrics.port }}
+          name: metrics-port
+          protocol: TCP
+        {{- end }}
         resources:
           {{- toYaml .Values.resources | nindent 12 }}
         securityContext:
-          {{- toYaml .Values.securityContext | nindent 12 }}          
+          {{- toYaml .Values.securityContext | nindent 12 }}
         volumeMounts:
         - mountPath: "/etc/actions-runner-controller"
-          name: controller-manager
+          name: secret
           readOnly: true
         - mountPath: /tmp
-          name: tmp         
+          name: tmp
         - mountPath: /tmp/k8s-webhook-server/serving-certs
           name: cert
           readOnly: true
+      {{- if .Values.metrics.proxy.enabled }}
       - args:
-        - "--secure-listen-address=0.0.0.0:8443"
+        - "--secure-listen-address=0.0.0.0:{{ .Values.metrics.port }}"
         - "--upstream=http://127.0.0.1:8080/"
         - "--logtostderr=true"
         - "--v=10"
-        image: "{{ .Values.kube_rbac_proxy.image.repository }}:{{ .Values.kube_rbac_proxy.image.tag }}"
+        image: "{{ .Values.metrics.proxy.image.repository }}:{{ .Values.metrics.proxy.image.tag }}"
         name: kube-rbac-proxy
         imagePullPolicy: {{ .Values.image.pullPolicy }}
         ports:
-        - containerPort: 8443
-          name: https
+        - containerPort: {{ .Values.metrics.port }}
+          name: metrics-port
         resources:
-          {{- toYaml .Values.resources | nindent 12 }}   
+          {{- toYaml .Values.resources | nindent 12 }}
         securityContext:
-          {{- toYaml .Values.securityContext | nindent 12 }}     
+          {{- toYaml .Values.securityContext | nindent 12 }}
+      {{- end }}
       terminationGracePeriodSeconds: 10
       volumes:
-      - name: controller-manager
+      - name: secret
         secret:
-          secretName: controller-manager
+          secretName: {{ include "actions-runner-controller.secretName" . }}
       - name: cert
         secret:
           defaultMode: 420
@@ -119,3 +141,7 @@ spec:
       tolerations:
         {{- toYaml . | nindent 8 }}
       {{- end }}
+      {{- with .Values.topologySpreadConstraints }}
+      topologySpreadConstraints:
+        {{- toYaml . | nindent 8 }}
+      {{- end }}

charts/actions-runner-controller/templates/githubwebhook.deployment.yaml

@@ -7,6 +7,7 @@ metadata:
   labels:
     {{- include "actions-runner-controller.labels" . | nindent 4 }}
 spec:
+  replicas: {{ .Values.githubWebhookServer.replicaCount }}
   selector:
     matchLabels:
       {{- include "actions-runner-controller-github-webhook-server.selectorLabels" . | nindent 6 }}
@@ -18,6 +19,9 @@ spec:
       {{- end }}
       labels:
         {{- include "actions-runner-controller-github-webhook-server.selectorLabels" . | nindent 8 }}
+      {{- with .Values.githubWebhookServer.podLabels }}
+        {{- toYaml . | nindent 8 }}
+      {{- end }}
     spec:
       {{- with .Values.githubWebhookServer.imagePullSecrets }}
       imagePullSecrets:
@@ -31,9 +35,13 @@ spec:
       {{- end }}
       containers:
       - args:
-        - "--metrics-addr=127.0.0.1:8080"
-        - "--enable-leader-election"
+        {{- $metricsHost := .Values.metrics.proxy.enabled | ternary "127.0.0.1" "0.0.0.0" }}
+        {{- $metricsPort := .Values.metrics.proxy.enabled | ternary "8080" .Values.metrics.port }}
+        - "--metrics-addr={{ $metricsHost }}:{{ $metricsPort }}"
         - "--sync-period={{ .Values.githubWebhookServer.syncPeriod }}"
+        {{- if .Values.githubWebhookServer.logLevel }}
+        - "--log-level={{ .Values.githubWebhookServer.logLevel }}"
+        {{- end }}
         command:
         - "/github-webhook-server"
         env:
@@ -41,43 +49,46 @@ spec:
           valueFrom:
             secretKeyRef:
               key: github_webhook_secret_token
-              name: github-webhook-server
+              name: {{ include "actions-runner-controller-github-webhook-server.secretName" . }}
               optional: true
         {{- range $key, $val := .Values.githubWebhookServer.env }}
         - name: {{ $key }}
           value: {{ $val | quote }}
         {{- end }}
-        image: "{{ .Values.githubWebhookServer.image.repository }}:{{ .Values.githubWebhookServer.image.tag | default (cat "v" .Chart.AppVersion | replace " " "") }}"
+        image: "{{ .Values.image.repository }}:{{ .Values.image.tag  | default (cat "v" .Chart.AppVersion | replace " " "") }}"
         name: github-webhook-server
         imagePullPolicy: {{ .Values.image.pullPolicy }}
         ports:
         - containerPort: 8000
-          name: github-webhook-server
+          name: http
           protocol: TCP
+        {{- if not .Values.metrics.proxy.enabled }}
+        - containerPort: {{ .Values.metrics.port }}
+          name: metrics-port
+          protocol: TCP
+        {{- end }}
         resources:
           {{- toYaml .Values.githubWebhookServer.resources | nindent 12 }}
         securityContext:
           {{- toYaml .Values.githubWebhookServer.securityContext | nindent 12 }}
+      {{- if .Values.metrics.proxy.enabled }}
       - args:
-        - "--secure-listen-address=0.0.0.0:8443"
+        - "--secure-listen-address=0.0.0.0:{{ .Values.metrics.port }}"
         - "--upstream=http://127.0.0.1:8080/"
         - "--logtostderr=true"
         - "--v=10"
-        image: "{{ .Values.kube_rbac_proxy.image.repository }}:{{ .Values.kube_rbac_proxy.image.tag }}"
+        image: "{{ .Values.metrics.proxy.image.repository }}:{{ .Values.metrics.proxy.image.tag }}"
         name: kube-rbac-proxy
         imagePullPolicy: {{ .Values.image.pullPolicy }}
         ports:
-        - containerPort: 8443
-          name: https
+        - containerPort: {{ .Values.metrics.port }}
+          name: metrics-port
         resources:
-          {{- toYaml .Values.resources | nindent 12 }}   
+          {{- toYaml .Values.resources | nindent 12 }}
         securityContext:
-          {{- toYaml .Values.securityContext | nindent 12 }}     
+          {{- toYaml .Values.securityContext | nindent 12 }}
+      {{- end }}
       terminationGracePeriodSeconds: 10
-      volumes:
-      - name: github-webhook-server
-        secret:
-          secretName: github-webhook-server
       {{- with .Values.githubWebhookServer.nodeSelector }}
       nodeSelector:
         {{- toYaml . | nindent 8 }}
@@ -90,4 +101,8 @@ spec:
       tolerations:
         {{- toYaml . | nindent 8 }}
       {{- end }}
+      {{- with .Values.githubWebhookServer.topologySpreadConstraints }}
+      topologySpreadConstraints:
+        {{- toYaml . | nindent 8 }}
+      {{- end }}
 {{- end }}

charts/actions-runner-controller/templates/githubwebhook.ingress.yaml

@@ -0,0 +1,41 @@
+{{- if .Values.githubWebhookServer.ingress.enabled -}}
+{{- $fullName := include "actions-runner-controller-github-webhook-server.fullname" . -}}
+{{- $svcPort := (index .Values.githubWebhookServer.service.ports 0).port -}}
+{{- if semverCompare ">=1.14-0" .Capabilities.KubeVersion.GitVersion -}}
+apiVersion: networking.k8s.io/v1beta1
+{{- else -}}
+apiVersion: extensions/v1beta1
+{{- end }}
+kind: Ingress
+metadata:
+  name: {{ $fullName }}
+  labels:
+    {{- include "actions-runner-controller.labels" . | nindent 4 }}
+  {{- with .Values.githubWebhookServer.ingress.annotations }}
+  annotations:
+    {{- toYaml . | nindent 4 }}
+  {{- end }}
+spec:
+  {{- if .Values.githubWebhookServer.ingress.tls }}
+  tls:
+    {{- range .Values.githubWebhookServer.ingress.tls }}
+    - hosts:
+        {{- range .hosts }}
+        - {{ . | quote }}
+        {{- end }}
+      secretName: {{ .secretName }}
+    {{- end }}
+  {{- end }}
+  rules:
+    {{- range .Values.githubWebhookServer.ingress.hosts }}
+    - host: {{ .host | quote }}
+      http:
+        paths:
+          {{- range .paths }}
+          - path: {{ .path }}
+            backend:
+              serviceName: {{ $fullName }}
+              servicePort: {{ $svcPort }}
+          {{- end }}
+    {{- end }}
+  {{- end }}

charts/actions-runner-controller/templates/githubwebhook.role_binding.yaml

@@ -0,0 +1,14 @@
+{{- if .Values.githubWebhookServer.enabled }}
+apiVersion: rbac.authorization.k8s.io/v1
+kind: ClusterRoleBinding
+metadata:
+  name: {{ include "actions-runner-controller-github-webhook-server.roleName" . }}
+roleRef:
+  apiGroup: rbac.authorization.k8s.io
+  kind: ClusterRole
+  name: {{ include "actions-runner-controller-github-webhook-server.roleName" . }}
+subjects:
+  - kind: ServiceAccount
+    name: {{ include "actions-runner-controller-github-webhook-server.serviceAccountName" . }}
+    namespace: {{ .Release.Namespace }}
+{{- end }}

charts/actions-runner-controller/templates/githubwebhook.secrets.yaml

@@ -1,16 +1,16 @@
 {{- if .Values.githubWebhookServer.enabled }}
-{{- if .Values.githubWebhookServer.secret.enabled }}
+{{- if .Values.githubWebhookServer.secret.create }}
 apiVersion: v1
 kind: Secret
 metadata:
-  name: github-webhook-server
+  name: {{ include "actions-runner-controller-github-webhook-server.secretName" . }}
   namespace: {{ .Release.Namespace }}
   labels:
     {{- include "actions-runner-controller.labels" . | nindent 4 }}
 type: Opaque
 data:
-{{- range $k, $v := .Values.githubWebhookServer.secret }}
-  {{ $k }}: {{ $v | toString | b64enc }}
-{{- end }} 
+{{- if .Values.githubWebhookServer.secret.github_webhook_secret_token }}
+  github_webhook_secret_token: {{ .Values.githubWebhookServer.secret.github_webhook_secret_token | toString | b64enc }}
+{{- end }}
 {{- end }}
 {{- end }}

charts/actions-runner-controller/templates/githubwebhook.service.yaml

@@ -12,6 +12,11 @@ spec:
     {{ range $_, $port := .Values.githubWebhookServer.service.ports -}}
     - {{ $port | toYaml | nindent 6 }}
     {{- end }}
+    {{- if .Values.metrics.serviceMonitor }}
+    - name: metrics-port
+      port: {{ .Values.metrics.port }}
+      targetPort: metrics-port
+    {{- end }}
   selector:
     {{- include "actions-runner-controller-github-webhook-server.selectorLabels" . | nindent 4 }}
 {{- end }}

charts/actions-runner-controller/templates/githubwebhook.serviceMonitor.yaml

@@ -0,0 +1,15 @@
+{{- if and .Values.githubWebhookServer.enabled .Values.metrics.serviceMonitor }}
+apiVersion: monitoring.coreos.com/v1
+kind: ServiceMonitor
+metadata:
+  labels:
+    {{- include "actions-runner-controller.labels" . | nindent 4 }}
+  name: {{ include "actions-runner-controller-github-webhook-server.serviceMonitorName" . }}
+spec:
+  endpoints:
+    - path: /metrics
+      port: metrics-port
+  selector:
+    matchLabels:
+      {{- include "actions-runner-controller-github-webhook-server.selectorLabels" . | nindent 6 }}
+{{- end }}

charts/actions-runner-controller/templates/manager_secrets.yaml

@@ -1,14 +1,23 @@
-{{- if or .Values.authSecret.enabled }}
+{{- if .Values.authSecret.create }}
 apiVersion: v1
 kind: Secret
 metadata:
-  name: controller-manager
+  name: {{ include "actions-runner-controller.secretName" . }}
   namespace: {{ .Release.Namespace }}
   labels:
     {{- include "actions-runner-controller.labels" . | nindent 4 }}
 type: Opaque
 data:
-{{- range $k, $v := .Values.authSecret }}
-  {{ $k }}: {{ $v | toString | b64enc }}
-{{- end }} 
-{{- end }}
+{{- if .Values.authSecret.github_app_id }}
+  github_app_id: {{ .Values.authSecret.github_app_id | toString | b64enc }}
+{{- end }}
+{{- if .Values.authSecret.github_app_installation_id }}
+  github_app_installation_id: {{ .Values.authSecret.github_app_installation_id | toString | b64enc }}
+{{- end }}
+{{- if .Values.authSecret.github_app_private_key }}
+  github_app_private_key: {{ .Values.authSecret.github_app_private_key | toString | b64enc }}
+{{- end }}
+{{- if .Values.authSecret.github_token }}
+  github_token: {{ .Values.authSecret.github_token | toString | b64enc }}
+{{- end }}
+{{- end }}

charts/actions-runner-controller/templates/webhook_configs.yaml

@@ -9,7 +9,6 @@ metadata:
     cert-manager.io/inject-ca-from: {{ .Release.Namespace }}/{{ include "actions-runner-controller.servingCertName" . }}
 webhooks:
 - clientConfig:
-    caBundle: Cg==
     service:
       name: {{ include "actions-runner-controller.webhookServiceName" . }}
       namespace: {{ .Release.Namespace }}
@@ -26,8 +25,8 @@ webhooks:
     - UPDATE
     resources:
     - runners
+  sideEffects: None
 - clientConfig:
-    caBundle: Cg==
     service:
       name: {{ include "actions-runner-controller.webhookServiceName" . }}
       namespace: {{ .Release.Namespace }}
@@ -44,8 +43,8 @@ webhooks:
     - UPDATE
     resources:
     - runnerdeployments
+  sideEffects: None
 - clientConfig:
-    caBundle: Cg==
     service:
       name: {{ include "actions-runner-controller.webhookServiceName" . }}
       namespace: {{ .Release.Namespace }}
@@ -62,7 +61,7 @@ webhooks:
     - UPDATE
     resources:
     - runnerreplicasets
-
+  sideEffects: None
 ---
 apiVersion: admissionregistration.k8s.io/v1beta1
 kind: ValidatingWebhookConfiguration
@@ -73,7 +72,6 @@ metadata:
     cert-manager.io/inject-ca-from: {{ .Release.Namespace }}/{{ include "actions-runner-controller.servingCertName" . }}
 webhooks:
 - clientConfig:
-    caBundle: Cg==
     service:
       name: {{ include "actions-runner-controller.webhookServiceName" . }}
       namespace: {{ .Release.Namespace }}
@@ -90,8 +88,8 @@ webhooks:
     - UPDATE
     resources:
     - runners
+  sideEffects: None
 - clientConfig:
-    caBundle: Cg==
     service:
       name: {{ include "actions-runner-controller.webhookServiceName" . }}
       namespace: {{ .Release.Namespace }}
@@ -108,8 +106,8 @@ webhooks:
     - UPDATE
     resources:
     - runnerdeployments
+  sideEffects: None
 - clientConfig:
-    caBundle: Cg==
     service:
       name: {{ include "actions-runner-controller.webhookServiceName" . }}
       namespace: {{ .Release.Namespace }}
@@ -126,3 +124,4 @@ webhooks:
     - UPDATE
     resources:
     - runnerreplicasets
+  sideEffects: None

charts/actions-runner-controller/values.yaml

@@ -8,10 +8,16 @@ replicaCount: 1
 
 syncPeriod: 10m
 
+# The controller tries its best not to repeat the duplicate GitHub API call
+# within this duration.
+# Defaults to syncPeriod - 10s.
+#githubAPICacheDuration: 30s
+
 # Only 1 authentication method can be deployed at a time
 # Uncomment the configuration you are applying and fill in the details
 authSecret:
-  enabled: false
+  create: true
+  name: "controller-manager"
   ### GitHub Apps Configuration
   #github_app_id: ""
   #github_app_installation_id: ""
@@ -21,16 +27,9 @@ authSecret:
 
 image:
   repository: summerwind/actions-runner-controller
-  # Overrides the manager image tag whose default is the chart appVersion if the tag key is commented out
-  tag: "latest"
   dindSidecarRepositoryAndTag: "docker:dind"
   pullPolicy: IfNotPresent
 
-kube_rbac_proxy:
-  image:
-    repository: quay.io/brancz/kube-rbac-proxy
-    tag: v0.8.0
-
 imagePullSecrets: []
 nameOverride: ""
 fullnameOverride: ""
@@ -46,10 +45,14 @@ serviceAccount:
 
 podAnnotations: {}
 
-podSecurityContext: {}
+podLabels: {}
+
+podSecurityContext:
+  {}
   # fsGroup: 2000
 
-securityContext: {}
+securityContext:
+  {}
   # capabilities:
   #   drop:
   #   - ALL
@@ -61,20 +64,17 @@ service:
   type: ClusterIP
   port: 443
 
-ingress:
-  enabled: false
-  annotations: {}
-    # kubernetes.io/ingress.class: nginx
-    # kubernetes.io/tls-acme: "true"
-  hosts:
-    - host: chart-example.local
-      paths: []
-  tls: []
-  #  - secretName: chart-example-tls
-  #    hosts:
-  #      - chart-example.local
+metrics:
+  serviceMonitor: false
+  port: 8443
+  proxy:
+    enabled: true
+    image:
+      repository: quay.io/brancz/kube-rbac-proxy
+      tag: v0.10.0
 
-resources: {}
+resources:
+  {}
   # We usually recommend not to specify default resources and to leave this as a conscious
   # choice for the user. This also increases chances charts run on environments with little
   # resources, such as Minikube. If you do want to specify resources, uncomment the following
@@ -86,13 +86,6 @@ resources: {}
   #   cpu: 100m
   #   memory: 128Mi
 
-autoscaling:
-  enabled: false
-  minReplicas: 1
-  maxReplicas: 100
-  targetCPUUtilizationPercentage: 80
-  # targetMemoryUtilizationPercentage: 80
-
 nodeSelector: {}
 
 tolerations: []
@@ -104,25 +97,28 @@ affinity: {}
 # PriorityClass: system-cluster-critical
 priorityClassName: ""
 
-env: {}
+env:
+  {}
   # http_proxy: "proxy.com:8080"
   # https_proxy: "proxy.com:8080"
   # no_proxy: ""
 
+scope:
+  # If true, the controller will only watch custom resources in a single namespace
+  singleNamespace: false
+  # If `scope.singleNamespace=true`, the controller will only watch custom resources in this namespace
+  # The default value is "", which means the namespace of the controller
+  watchNamespace: ""
+
 githubWebhookServer:
   enabled: false
-  labels: {}
   replicaCount: 1
   syncPeriod: 10m
   secret:
-    enabled: false
+    create: true
+    name: "github-webhook-server"
     ### GitHub Webhook Configuration
     #github_webhook_secret_token: ""
-  image:
-    repository: summerwind/actions-runner-controller
-    # Overrides the manager image tag whose default is the chart appVersion if the tag key is commented out
-    tag: "latest"
-    pullPolicy: IfNotPresent
   imagePullSecrets: []
   nameOverride: ""
   fullnameOverride: ""
@@ -135,6 +131,7 @@ githubWebhookServer:
     # If not set and create is true, a name is generated using the fullname template
     name: ""
   podAnnotations: {}
+  podLabels: {}
   podSecurityContext: {}
   # fsGroup: 2000
   securityContext: {}
@@ -144,10 +141,23 @@ githubWebhookServer:
   affinity: {}
   priorityClassName: ""
   service:
-    type: NodePort
+    type: ClusterIP
     ports:
-    - port: 80
-      targetPort: 8000
-      protocol: TCP
-      name: http
-      #nodePort: someFixedPortForUseWithTerraformCdkCfnEtc
+      - port: 80
+        targetPort: http
+        protocol: TCP
+        name: http
+        #nodePort: someFixedPortForUseWithTerraformCdkCfnEtc
+  ingress:
+    enabled: false
+    annotations:
+      {}
+      # kubernetes.io/ingress.class: nginx
+      # kubernetes.io/tls-acme: "true"
+    hosts:
+      - host: chart-example.local
+        paths: []
+    tls: []
+    #  - secretName: chart-example-tls
+    #    hosts:
+    #      - chart-example.local

cmd/githubwebhookserver/main.go

@@ -27,6 +27,7 @@ import (
 
 	actionsv1alpha1 "github.com/summerwind/actions-runner-controller/api/v1alpha1"
 	"github.com/summerwind/actions-runner-controller/controllers"
+	zaplib "go.uber.org/zap"
 	"k8s.io/apimachinery/pkg/runtime"
 	clientgoscheme "k8s.io/client-go/kubernetes/scheme"
 	_ "k8s.io/client-go/plugin/pkg/client/auth/exec"
@@ -42,6 +43,13 @@ var (
 	setupLog = ctrl.Log.WithName("setup")
 )
 
+const (
+	logLevelDebug = "debug"
+	logLevelInfo  = "info"
+	logLevelWarn  = "warn"
+	logLevelError = "error"
+)
+
 func init() {
 	_ = clientgoscheme.AddToScheme(scheme)
 
@@ -63,6 +71,7 @@ func main() {
 
 		enableLeaderElection bool
 		syncPeriod           time.Duration
+		logLevel             string
 	)
 
 	webhookSecretToken = os.Getenv("GITHUB_WEBHOOK_SECRET_TOKEN")
@@ -73,6 +82,7 @@ func main() {
 	flag.BoolVar(&enableLeaderElection, "enable-leader-election", false,
 		"Enable leader election for controller manager. Enabling this will ensure there is only one active controller manager.")
 	flag.DurationVar(&syncPeriod, "sync-period", 10*time.Minute, "Determines the minimum frequency at which K8s resources managed by this controller are reconciled. When you use autoscaling, set to a lower value like 10 minute, because this corresponds to the minimum time to react on demand change")
+	flag.StringVar(&logLevel, "log-level", logLevelDebug, `The verbosity of the logging. Valid values are "debug", "info", "warn", "error". Defaults to "debug".`)
 	flag.Parse()
 
 	if webhookSecretToken == "" {
@@ -86,7 +96,19 @@ func main() {
 	}
 
 	logger := zap.New(func(o *zap.Options) {
-		o.Development = true
+		switch logLevel {
+		case logLevelDebug:
+			o.Development = true
+		case logLevelInfo:
+			lvl := zaplib.NewAtomicLevelAt(zaplib.InfoLevel)
+			o.Level = &lvl
+		case logLevelWarn:
+			lvl := zaplib.NewAtomicLevelAt(zaplib.WarnLevel)
+			o.Level = &lvl
+		case logLevelError:
+			lvl := zaplib.NewAtomicLevelAt(zaplib.ErrorLevel)
+			o.Level = &lvl
+		}
 	})
 
 	ctrl.SetLogger(logger)
@@ -110,7 +132,7 @@ func main() {
 		Recorder:       nil,
 		Scheme:         mgr.GetScheme(),
 		SecretKeyBytes: []byte(webhookSecretToken),
-		WatchNamespace: watchNamespace,
+		Namespace:      watchNamespace,
 	}
 
 	if err = hraGitHubWebhook.SetupWithManager(mgr); err != nil {

config/crd/bases/actions.summerwind.dev_horizontalrunnerautoscalers.yaml

@@ -18,6 +18,9 @@ spec:
   - JSONPath: .status.desiredReplicas
     name: Desired
     type: number
+  - JSONPath: .status.scheduledOverridesSummary
+    name: Schedule
+    type: string
   group: actions.summerwind.dev
   names:
     kind: HorizontalRunnerAutoscaler
@@ -148,6 +151,17 @@ spec:
                       checkRun:
                         description: https://docs.github.com/en/actions/reference/events-that-trigger-workflows#check_run
                         properties:
+                          names:
+                            description: Names is a list of GitHub Actions glob patterns.
+                              Any check_run event whose name matches one of patterns
+                              in the list can trigger autoscaling. Note that check_run
+                              name seem to equal to the job name you've defined in
+                              your actions workflow yaml file. So it is very likely
+                              that you can utilize this to trigger depending on the
+                              job.
+                            items:
+                              type: string
+                            type: array
                           status:
                             type: string
                           types:
@@ -174,6 +188,56 @@ spec:
                     type: object
                 type: object
               type: array
+            scheduledOverrides:
+              description: ScheduledOverrides is the list of ScheduledOverride. It
+                can be used to override a few fields of HorizontalRunnerAutoscalerSpec
+                on schedule. The earlier a scheduled override is, the higher it is
+                prioritized.
+              items:
+                description: ScheduledOverride can be used to override a few fields
+                  of HorizontalRunnerAutoscalerSpec on schedule. A schedule can optionally
+                  be recurring, so that the correspoding override happens every day,
+                  week, month, or year.
+                properties:
+                  endTime:
+                    description: EndTime is the time at which the first override ends.
+                    format: date-time
+                    type: string
+                  minReplicas:
+                    description: MinReplicas is the number of runners while overriding.
+                      If omitted, it doesn't override minReplicas.
+                    minimum: 0
+                    nullable: true
+                    type: integer
+                  recurrenceRule:
+                    properties:
+                      frequency:
+                        description: Frequency is the name of a predefined interval
+                          of each recurrence. The valid values are "Daily", "Weekly",
+                          "Monthly", and "Yearly". If empty, the corresponding override
+                          happens only once.
+                        enum:
+                        - Daily
+                        - Weekly
+                        - Monthly
+                        - Yearly
+                        type: string
+                      untilTime:
+                        description: UntilTime is the time of the final recurrence.
+                          If empty, the schedule recurs forever.
+                        format: date-time
+                        type: string
+                    type: object
+                  startTime:
+                    description: StartTime is the time at which the first override
+                      starts.
+                    format: date-time
+                    type: string
+                required:
+                - endTime
+                - startTime
+                type: object
+              type: array
           type: object
         status:
           properties:
@@ -196,6 +260,7 @@ spec:
               type: integer
             lastSuccessfulScaleOutTime:
               format: date-time
+              nullable: true
               type: string
             observedGeneration:
               description: ObservedGeneration is the most recent generation observed
@@ -203,6 +268,11 @@ spec:
                 which is updated on mutation by the API Server.
               format: int64
               type: integer
+            scheduledOverridesSummary:
+              description: ScheduledOverridesSummary is the summary of active and
+                upcoming scheduled overrides to be shown in e.g. a column of a `kubectl
+                get hra` output for observability.
+              type: string
           type: object
       type: object
   version: v1alpha1

config/crd/bases/actions.summerwind.dev_runnerdeployments.yaml

@@ -10,12 +10,18 @@ spec:
     - JSONPath: .spec.replicas
       name: Desired
       type: number
-    - JSONPath: .status.availableReplicas
+    - JSONPath: .status.replicas
       name: Current
       type: number
-    - JSONPath: .status.readyReplicas
-      name: Ready
+    - JSONPath: .status.updatedReplicas
+      name: Up-To-Date
       type: number
+    - JSONPath: .status.availableReplicas
+      name: Available
+      type: number
+    - JSONPath: .metadata.creationTimestamp
+      name: Age
+      type: date
   group: actions.summerwind.dev
   names:
     kind: RunnerDeployment
@@ -38,11 +44,42 @@ spec:
         metadata:
           type: object
         spec:
-          description: RunnerReplicaSetSpec defines the desired state of RunnerDeployment
+          description: RunnerDeploymentSpec defines the desired state of RunnerDeployment
           properties:
             replicas:
               nullable: true
               type: integer
+            selector:
+              description: A label selector is a label query over a set of resources. The result of matchLabels and matchExpressions are ANDed. An empty label selector matches all objects. A null label selector matches no objects.
+              nullable: true
+              properties:
+                matchExpressions:
+                  description: matchExpressions is a list of label selector requirements. The requirements are ANDed.
+                  items:
+                    description: A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.
+                    properties:
+                      key:
+                        description: key is the label key that the selector applies to.
+                        type: string
+                      operator:
+                        description: operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.
+                        type: string
+                      values:
+                        description: values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.
+                        items:
+                          type: string
+                        type: array
+                    required:
+                      - key
+                      - operator
+                    type: object
+                  type: array
+                matchLabels:
+                  additionalProperties:
+                    type: string
+                  description: matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is "key", the operator is "In", and the values array contains only "value". The requirements are ANDed.
+                  type: object
+              type: object
             template:
               properties:
                 metadata:
@@ -402,6 +439,38 @@ spec:
                       type: array
                     dockerEnabled:
                       type: boolean
+                    dockerMTU:
+                      format: int64
+                      type: integer
+                    dockerRegistryMirror:
+                      type: string
+                    dockerVolumeMounts:
+                      items:
+                        description: VolumeMount describes a mounting of a Volume within a container.
+                        properties:
+                          mountPath:
+                            description: Path within the container at which the volume should be mounted.  Must not contain ':'.
+                            type: string
+                          mountPropagation:
+                            description: mountPropagation determines how mounts are propagated from the host to container and the other way around. When not set, MountPropagationNone is used. This field is beta in 1.10.
+                            type: string
+                          name:
+                            description: This must match the Name of a Volume.
+                            type: string
+                          readOnly:
+                            description: Mounted read-only if true, read-write otherwise (false or unspecified). Defaults to false.
+                            type: boolean
+                          subPath:
+                            description: Path within the volume from which the container's volume should be mounted. Defaults to "" (volume's root).
+                            type: string
+                          subPathExpr:
+                            description: Expanded path within the volume from which the container's volume should be mounted. Behaves similarly to SubPath but environment variable references $(VAR_NAME) are expanded using the container's environment. Defaults to "" (volume's root). SubPathExpr and SubPath are mutually exclusive. This field is beta in 1.15.
+                            type: string
+                        required:
+                          - mountPath
+                          - name
+                        type: object
+                      type: array
                     dockerdContainerResources:
                       description: ResourceRequirements describes the compute resource requirements.
                       properties:
@@ -537,6 +606,8 @@ spec:
                             type: object
                         type: object
                       type: array
+                    ephemeral:
+                      type: boolean
                     ephemeralContainers:
                       items:
                         description: An EphemeralContainer is a container that may be added temporarily to an existing pod for user-initiated activities such as debugging. Ephemeral containers have no resource or scheduling guarantees, and they will not be restarted when they exit or when a pod is removed or restarted. If an ephemeral container causes a pod to exceed its resource allocation, the pod may be evicted. Ephemeral containers may not be added by directly updating the pod spec. They must be added via the pod's ephemeralcontainers subresource, and they will appear in the pod spec once added. This is an alpha feature enabled by the EphemeralContainers feature flag.
@@ -546,6 +617,20 @@ spec:
                       type: array
                     group:
                       type: string
+                    hostAliases:
+                      items:
+                        description: HostAlias holds the mapping between IP and hostnames that will be injected as an entry in the pod's hosts file.
+                        properties:
+                          hostnames:
+                            description: Hostnames for the above IP address.
+                            items:
+                              type: string
+                            type: array
+                          ip:
+                            description: IP address of the host file entry.
+                            type: string
+                        type: object
+                      type: array
                     image:
                       type: string
                     imagePullPolicy:
@@ -603,6 +688,9 @@ spec:
                           description: 'Requests describes the minimum amount of compute resources required. If Requests is omitted for a container, it defaults to Limits if that is explicitly specified, otherwise to an implementation-defined value. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/'
                           type: object
                       type: object
+                    runtimeClassName:
+                      description: 'RuntimeClassName is the container runtime configuration that containers should run under. More info: https://kubernetes.io/docs/concepts/containers/runtime-class'
+                      type: string
                     securityContext:
                       description: PodSecurityContext holds pod-level security attributes and common container settings. Some fields are also present in container.securityContext.  Field values of container.securityContext take precedence over field values of PodSecurityContext.
                       properties:
@@ -734,6 +822,12 @@ spec:
                           - name
                         type: object
                       type: array
+                    volumeSizeLimit:
+                      anyOf:
+                        - type: integer
+                        - type: string
+                      pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
+                      x-kubernetes-int-or-string: true
                     volumes:
                       items:
                         description: Volume represents a named volume in a pod that may be accessed by any container in the pod.
@@ -1546,15 +1640,20 @@ spec:
         status:
           properties:
             availableReplicas:
+              description: AvailableReplicas is the total number of available runners which have been successfully registered to GitHub and still running. This corresponds to the sum of status.availableReplicas of all the runner replica sets.
               type: integer
             desiredReplicas:
-              description: Replicas is the total number of desired, non-terminated and latest pods to be set for the primary RunnerSet This doesn't include outdated pods while upgrading the deployment and replacing the runnerset.
+              description: DesiredReplicas is the total number of desired, non-terminated and latest pods to be set for the primary RunnerSet This doesn't include outdated pods while upgrading the deployment and replacing the runnerset.
               type: integer
             readyReplicas:
+              description: ReadyReplicas is the total number of available runners which have been successfully registered to GitHub and still running. This corresponds to the sum of status.readyReplicas of all the runner replica sets.
+              type: integer
+            replicas:
+              description: Replicas is the total number of replicas
+              type: integer
+            updatedReplicas:
+              description: ReadyReplicas is the total number of available runners which have been successfully registered to GitHub and still running. This corresponds to status.replicas of the runner replica set that has the desired template hash.
               type: integer
-          required:
-            - availableReplicas
-            - readyReplicas
           type: object
       type: object
   version: v1alpha1

config/crd/bases/actions.summerwind.dev_runnerreplicasets.yaml

@@ -10,12 +10,15 @@ spec:
     - JSONPath: .spec.replicas
       name: Desired
       type: number
-    - JSONPath: .status.availableReplicas
+    - JSONPath: .status.replicas
       name: Current
       type: number
     - JSONPath: .status.readyReplicas
       name: Ready
       type: number
+    - JSONPath: .metadata.creationTimestamp
+      name: Age
+      type: date
   group: actions.summerwind.dev
   names:
     kind: RunnerReplicaSet
@@ -43,6 +46,37 @@ spec:
             replicas:
               nullable: true
               type: integer
+            selector:
+              description: A label selector is a label query over a set of resources. The result of matchLabels and matchExpressions are ANDed. An empty label selector matches all objects. A null label selector matches no objects.
+              nullable: true
+              properties:
+                matchExpressions:
+                  description: matchExpressions is a list of label selector requirements. The requirements are ANDed.
+                  items:
+                    description: A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.
+                    properties:
+                      key:
+                        description: key is the label key that the selector applies to.
+                        type: string
+                      operator:
+                        description: operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.
+                        type: string
+                      values:
+                        description: values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.
+                        items:
+                          type: string
+                        type: array
+                    required:
+                      - key
+                      - operator
+                    type: object
+                  type: array
+                matchLabels:
+                  additionalProperties:
+                    type: string
+                  description: matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is "key", the operator is "In", and the values array contains only "value". The requirements are ANDed.
+                  type: object
+              type: object
             template:
               properties:
                 metadata:
@@ -402,6 +436,38 @@ spec:
                       type: array
                     dockerEnabled:
                       type: boolean
+                    dockerMTU:
+                      format: int64
+                      type: integer
+                    dockerRegistryMirror:
+                      type: string
+                    dockerVolumeMounts:
+                      items:
+                        description: VolumeMount describes a mounting of a Volume within a container.
+                        properties:
+                          mountPath:
+                            description: Path within the container at which the volume should be mounted.  Must not contain ':'.
+                            type: string
+                          mountPropagation:
+                            description: mountPropagation determines how mounts are propagated from the host to container and the other way around. When not set, MountPropagationNone is used. This field is beta in 1.10.
+                            type: string
+                          name:
+                            description: This must match the Name of a Volume.
+                            type: string
+                          readOnly:
+                            description: Mounted read-only if true, read-write otherwise (false or unspecified). Defaults to false.
+                            type: boolean
+                          subPath:
+                            description: Path within the volume from which the container's volume should be mounted. Defaults to "" (volume's root).
+                            type: string
+                          subPathExpr:
+                            description: Expanded path within the volume from which the container's volume should be mounted. Behaves similarly to SubPath but environment variable references $(VAR_NAME) are expanded using the container's environment. Defaults to "" (volume's root). SubPathExpr and SubPath are mutually exclusive. This field is beta in 1.15.
+                            type: string
+                        required:
+                          - mountPath
+                          - name
+                        type: object
+                      type: array
                     dockerdContainerResources:
                       description: ResourceRequirements describes the compute resource requirements.
                       properties:
@@ -537,6 +603,8 @@ spec:
                             type: object
                         type: object
                       type: array
+                    ephemeral:
+                      type: boolean
                     ephemeralContainers:
                       items:
                         description: An EphemeralContainer is a container that may be added temporarily to an existing pod for user-initiated activities such as debugging. Ephemeral containers have no resource or scheduling guarantees, and they will not be restarted when they exit or when a pod is removed or restarted. If an ephemeral container causes a pod to exceed its resource allocation, the pod may be evicted. Ephemeral containers may not be added by directly updating the pod spec. They must be added via the pod's ephemeralcontainers subresource, and they will appear in the pod spec once added. This is an alpha feature enabled by the EphemeralContainers feature flag.
@@ -546,6 +614,20 @@ spec:
                       type: array
                     group:
                       type: string
+                    hostAliases:
+                      items:
+                        description: HostAlias holds the mapping between IP and hostnames that will be injected as an entry in the pod's hosts file.
+                        properties:
+                          hostnames:
+                            description: Hostnames for the above IP address.
+                            items:
+                              type: string
+                            type: array
+                          ip:
+                            description: IP address of the host file entry.
+                            type: string
+                        type: object
+                      type: array
                     image:
                       type: string
                     imagePullPolicy:
@@ -603,6 +685,9 @@ spec:
                           description: 'Requests describes the minimum amount of compute resources required. If Requests is omitted for a container, it defaults to Limits if that is explicitly specified, otherwise to an implementation-defined value. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/'
                           type: object
                       type: object
+                    runtimeClassName:
+                      description: 'RuntimeClassName is the container runtime configuration that containers should run under. More info: https://kubernetes.io/docs/concepts/containers/runtime-class'
+                      type: string
                     securityContext:
                       description: PodSecurityContext holds pod-level security attributes and common container settings. Some fields are also present in container.securityContext.  Field values of container.securityContext take precedence over field values of PodSecurityContext.
                       properties:
@@ -734,6 +819,12 @@ spec:
                           - name
                         type: object
                       type: array
+                    volumeSizeLimit:
+                      anyOf:
+                        - type: integer
+                        - type: string
+                      pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
+                      x-kubernetes-int-or-string: true
                     volumes:
                       items:
                         description: Volume represents a named volume in a pod that may be accessed by any container in the pod.
@@ -1546,8 +1637,13 @@ spec:
         status:
           properties:
             availableReplicas:
+              description: AvailableReplicas is the number of runners that are created and Runnning. This is currently same as ReadyReplicas but perserved for future use.
               type: integer
             readyReplicas:
+              description: ReadyReplicas is the number of runners that are created and Runnning.
+              type: integer
+            replicas:
+              description: Replicas is the number of runners that are created and still being managed by this runner replica set.
               type: integer
           required:
             - availableReplicas

config/crd/bases/actions.summerwind.dev_runners.yaml

@@ -22,6 +22,9 @@ spec:
     - JSONPath: .status.phase
       name: Status
       type: string
+    - JSONPath: .metadata.creationTimestamp
+      name: Age
+      type: date
   group: actions.summerwind.dev
   names:
     kind: Runner
@@ -398,6 +401,38 @@ spec:
               type: array
             dockerEnabled:
               type: boolean
+            dockerMTU:
+              format: int64
+              type: integer
+            dockerRegistryMirror:
+              type: string
+            dockerVolumeMounts:
+              items:
+                description: VolumeMount describes a mounting of a Volume within a container.
+                properties:
+                  mountPath:
+                    description: Path within the container at which the volume should be mounted.  Must not contain ':'.
+                    type: string
+                  mountPropagation:
+                    description: mountPropagation determines how mounts are propagated from the host to container and the other way around. When not set, MountPropagationNone is used. This field is beta in 1.10.
+                    type: string
+                  name:
+                    description: This must match the Name of a Volume.
+                    type: string
+                  readOnly:
+                    description: Mounted read-only if true, read-write otherwise (false or unspecified). Defaults to false.
+                    type: boolean
+                  subPath:
+                    description: Path within the volume from which the container's volume should be mounted. Defaults to "" (volume's root).
+                    type: string
+                  subPathExpr:
+                    description: Expanded path within the volume from which the container's volume should be mounted. Behaves similarly to SubPath but environment variable references $(VAR_NAME) are expanded using the container's environment. Defaults to "" (volume's root). SubPathExpr and SubPath are mutually exclusive. This field is beta in 1.15.
+                    type: string
+                required:
+                  - mountPath
+                  - name
+                type: object
+              type: array
             dockerdContainerResources:
               description: ResourceRequirements describes the compute resource requirements.
               properties:
@@ -533,6 +568,8 @@ spec:
                     type: object
                 type: object
               type: array
+            ephemeral:
+              type: boolean
             ephemeralContainers:
               items:
                 description: An EphemeralContainer is a container that may be added temporarily to an existing pod for user-initiated activities such as debugging. Ephemeral containers have no resource or scheduling guarantees, and they will not be restarted when they exit or when a pod is removed or restarted. If an ephemeral container causes a pod to exceed its resource allocation, the pod may be evicted. Ephemeral containers may not be added by directly updating the pod spec. They must be added via the pod's ephemeralcontainers subresource, and they will appear in the pod spec once added. This is an alpha feature enabled by the EphemeralContainers feature flag.
@@ -542,6 +579,20 @@ spec:
               type: array
             group:
               type: string
+            hostAliases:
+              items:
+                description: HostAlias holds the mapping between IP and hostnames that will be injected as an entry in the pod's hosts file.
+                properties:
+                  hostnames:
+                    description: Hostnames for the above IP address.
+                    items:
+                      type: string
+                    type: array
+                  ip:
+                    description: IP address of the host file entry.
+                    type: string
+                type: object
+              type: array
             image:
               type: string
             imagePullPolicy:
@@ -599,6 +650,9 @@ spec:
                   description: 'Requests describes the minimum amount of compute resources required. If Requests is omitted for a container, it defaults to Limits if that is explicitly specified, otherwise to an implementation-defined value. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/'
                   type: object
               type: object
+            runtimeClassName:
+              description: 'RuntimeClassName is the container runtime configuration that containers should run under. More info: https://kubernetes.io/docs/concepts/containers/runtime-class'
+              type: string
             securityContext:
               description: PodSecurityContext holds pod-level security attributes and common container settings. Some fields are also present in container.securityContext.  Field values of container.securityContext take precedence over field values of PodSecurityContext.
               properties:
@@ -730,6 +784,12 @@ spec:
                   - name
                 type: object
               type: array
+            volumeSizeLimit:
+              anyOf:
+                - type: integer
+                - type: string
+              pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
+              x-kubernetes-int-or-string: true
             volumes:
               items:
                 description: Volume represents a named volume in a pod that may be accessed by any container in the pod.
@@ -1538,6 +1598,10 @@ spec:
         status:
           description: RunnerStatus defines the observed state of Runner
           properties:
+            lastRegistrationCheckTime:
+              format: date-time
+              nullable: true
+              type: string
             message:
               type: string
             phase:
@@ -1566,11 +1630,6 @@ spec:
                 - expiresAt
                 - token
               type: object
-          required:
-            - message
-            - phase
-            - reason
-            - registration
           type: object
       type: object
   version: v1alpha1

config/default/manager_auth_proxy_patch.yaml

@@ -10,7 +10,7 @@ spec:
     spec:
       containers:
       - name: kube-rbac-proxy
-        image: quay.io/brancz/kube-rbac-proxy:v0.8.0
+        image: quay.io/brancz/kube-rbac-proxy:v0.10.0
         args:
         - "--secure-listen-address=0.0.0.0:8443"
         - "--upstream=http://127.0.0.1:8080/"

config/manager/kustomization.yaml

@@ -4,5 +4,5 @@ apiVersion: kustomize.config.k8s.io/v1beta1
 kind: Kustomization
 images:
 - name: controller
-  newName: summerwind/actions-runner-controller
-  newTag: latest
+  newName: mumoshu/actions-runner-controller
+  newTag: dev

config/webhook/manifests.yaml

@@ -24,6 +24,7 @@ webhooks:
     - UPDATE
     resources:
     - runners
+  sideEffects: None
 - clientConfig:
     caBundle: Cg==
     service:
@@ -60,6 +61,7 @@ webhooks:
     - UPDATE
     resources:
     - runnerreplicasets
+  sideEffects: None
 
 ---
 apiVersion: admissionregistration.k8s.io/v1beta1
@@ -86,6 +88,7 @@ webhooks:
     - UPDATE
     resources:
     - runners
+  sideEffects: None
 - clientConfig:
     caBundle: Cg==
     service:
@@ -122,3 +125,4 @@ webhooks:
     - UPDATE
     resources:
     - runnerreplicasets
+  sideEffects: None

controllers/autoscaling.go

@@ -10,6 +10,8 @@ import (
 	"time"
 
 	"github.com/summerwind/actions-runner-controller/api/v1alpha1"
+	kerrors "k8s.io/apimachinery/pkg/api/errors"
+	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
 	"sigs.k8s.io/controller-runtime/pkg/client"
 )
 
@@ -32,7 +34,7 @@ func getValueAvailableAt(now time.Time, from, to *time.Time, reservedValue int)
 	return &reservedValue
 }
 
-func (r *HorizontalRunnerAutoscalerReconciler) getDesiredReplicasFromCache(hra v1alpha1.HorizontalRunnerAutoscaler) *int {
+func (r *HorizontalRunnerAutoscalerReconciler) fetchSuggestedReplicasFromCache(hra v1alpha1.HorizontalRunnerAutoscaler) *int {
 	var entry *v1alpha1.CacheEntry
 
 	for i := range hra.Status.CacheEntries {
@@ -61,7 +63,7 @@ func (r *HorizontalRunnerAutoscalerReconciler) getDesiredReplicasFromCache(hra v
 	return nil
 }
 
-func (r *HorizontalRunnerAutoscalerReconciler) determineDesiredReplicas(rd v1alpha1.RunnerDeployment, hra v1alpha1.HorizontalRunnerAutoscaler) (*int, error) {
+func (r *HorizontalRunnerAutoscalerReconciler) suggestDesiredReplicas(rd v1alpha1.RunnerDeployment, hra v1alpha1.HorizontalRunnerAutoscaler) (*int, error) {
 	if hra.Spec.MinReplicas == nil {
 		return nil, fmt.Errorf("horizontalrunnerautoscaler %s/%s is missing minReplicas", hra.Namespace, hra.Name)
 	} else if hra.Spec.MaxReplicas == nil {
@@ -69,19 +71,68 @@ func (r *HorizontalRunnerAutoscalerReconciler) determineDesiredReplicas(rd v1alp
 	}
 
 	metrics := hra.Spec.Metrics
-	if len(metrics) == 0 || metrics[0].Type == v1alpha1.AutoscalingMetricTypeTotalNumberOfQueuedAndInProgressWorkflowRuns {
-		return r.calculateReplicasByQueuedAndInProgressWorkflowRuns(rd, hra)
-	} else if metrics[0].Type == v1alpha1.AutoscalingMetricTypePercentageRunnersBusy {
-		return r.calculateReplicasByPercentageRunnersBusy(rd, hra)
-	} else {
-		return nil, fmt.Errorf("validting autoscaling metrics: unsupported metric type %q", metrics[0].Type)
+	numMetrics := len(metrics)
+	if numMetrics == 0 {
+		if len(hra.Spec.ScaleUpTriggers) == 0 {
+			return r.suggestReplicasByQueuedAndInProgressWorkflowRuns(rd, hra, nil)
+		}
+
+		return nil, nil
+	} else if numMetrics > 2 {
+		return nil, fmt.Errorf("Too many autoscaling metrics configured: It must be 0 to 2, but got %d", numMetrics)
 	}
+
+	primaryMetric := metrics[0]
+	primaryMetricType := primaryMetric.Type
+
+	var (
+		suggested *int
+		err       error
+	)
+
+	switch primaryMetricType {
+	case v1alpha1.AutoscalingMetricTypeTotalNumberOfQueuedAndInProgressWorkflowRuns:
+		suggested, err = r.suggestReplicasByQueuedAndInProgressWorkflowRuns(rd, hra, &primaryMetric)
+	case v1alpha1.AutoscalingMetricTypePercentageRunnersBusy:
+		suggested, err = r.suggestReplicasByPercentageRunnersBusy(rd, hra, primaryMetric)
+	default:
+		return nil, fmt.Errorf("validting autoscaling metrics: unsupported metric type %q", primaryMetric)
+	}
+
+	if err != nil {
+		return nil, err
+	}
+
+	if suggested != nil && *suggested > 0 {
+		return suggested, nil
+	}
+
+	if len(metrics) == 1 {
+		// This is never supposed to happen but anyway-
+		// Fall-back to `minReplicas + capacityReservedThroughWebhook`.
+		return nil, nil
+	}
+
+	// At this point, we are sure that there are exactly 2 Metrics entries.
+
+	fallbackMetric := metrics[1]
+	fallbackMetricType := fallbackMetric.Type
+
+	if primaryMetricType != v1alpha1.AutoscalingMetricTypePercentageRunnersBusy ||
+		fallbackMetricType != v1alpha1.AutoscalingMetricTypeTotalNumberOfQueuedAndInProgressWorkflowRuns {
+
+		return nil, fmt.Errorf(
+			"invalid HRA Spec: Metrics[0] of %s cannot be combined with Metrics[1] of %s: The only allowed combination is 0=PercentageRunnersBusy and 1=TotalNumberOfQueuedAndInProgressWorkflowRuns",
+			primaryMetricType, fallbackMetricType,
+		)
+	}
+
+	return r.suggestReplicasByQueuedAndInProgressWorkflowRuns(rd, hra, &fallbackMetric)
 }
 
-func (r *HorizontalRunnerAutoscalerReconciler) calculateReplicasByQueuedAndInProgressWorkflowRuns(rd v1alpha1.RunnerDeployment, hra v1alpha1.HorizontalRunnerAutoscaler) (*int, error) {
+func (r *HorizontalRunnerAutoscalerReconciler) suggestReplicasByQueuedAndInProgressWorkflowRuns(rd v1alpha1.RunnerDeployment, hra v1alpha1.HorizontalRunnerAutoscaler, metrics *v1alpha1.MetricSpec) (*int, error) {
 
 	var repos [][]string
-	metrics := hra.Spec.Metrics
 	repoID := rd.Spec.Template.Spec.Repository
 	if repoID == "" {
 		orgName := rd.Spec.Template.Spec.Organization
@@ -89,11 +140,18 @@ func (r *HorizontalRunnerAutoscalerReconciler) calculateReplicasByQueuedAndInPro
 			return nil, fmt.Errorf("asserting runner deployment spec to detect bug: spec.template.organization should not be empty on this code path")
 		}
 
-		if len(metrics[0].RepositoryNames) == 0 {
+		// In case it's an organizational runners deployment without any scaling metrics defined,
+		// we assume that the desired replicas should always be `minReplicas + capacityReservedThroughWebhook`.
+		// See https://github.com/summerwind/actions-runner-controller/issues/377#issuecomment-793372693
+		if metrics == nil {
+			return nil, nil
+		}
+
+		if len(metrics.RepositoryNames) == 0 {
 			return nil, errors.New("validating autoscaling metrics: spec.autoscaling.metrics[].repositoryNames is required and must have one more more entries for organizational runner deployment")
 		}
 
-		for _, repoName := range metrics[0].RepositoryNames {
+		for _, repoName := range metrics.RepositoryNames {
 			repos = append(repos, []string{orgName, repoName})
 		}
 	} else {
@@ -163,28 +221,10 @@ func (r *HorizontalRunnerAutoscalerReconciler) calculateReplicasByQueuedAndInPro
 		}
 	}
 
-	minReplicas := *hra.Spec.MinReplicas
-	maxReplicas := *hra.Spec.MaxReplicas
 	necessaryReplicas := queued + inProgress
 
-	var desiredReplicas int
-
-	if necessaryReplicas < minReplicas {
-		desiredReplicas = minReplicas
-	} else if necessaryReplicas > maxReplicas {
-		desiredReplicas = maxReplicas
-	} else {
-		desiredReplicas = necessaryReplicas
-	}
-
-	rd.Status.Replicas = &desiredReplicas
-	replicas := desiredReplicas
-
 	r.Log.V(1).Info(
-		"Calculated desired replicas",
-		"computed_replicas_desired", desiredReplicas,
-		"spec_replicas_min", minReplicas,
-		"spec_replicas_max", maxReplicas,
+		fmt.Sprintf("Suggested desired replicas of %d by TotalNumberOfQueuedAndInProgressWorkflowRuns", necessaryReplicas),
 		"workflow_runs_completed", completed,
 		"workflow_runs_in_progress", inProgress,
 		"workflow_runs_queued", queued,
@@ -194,14 +234,11 @@ func (r *HorizontalRunnerAutoscalerReconciler) calculateReplicasByQueuedAndInPro
 		"horizontal_runner_autoscaler", hra.Name,
 	)
 
-	return &replicas, nil
+	return &necessaryReplicas, nil
 }
 
-func (r *HorizontalRunnerAutoscalerReconciler) calculateReplicasByPercentageRunnersBusy(rd v1alpha1.RunnerDeployment, hra v1alpha1.HorizontalRunnerAutoscaler) (*int, error) {
+func (r *HorizontalRunnerAutoscalerReconciler) suggestReplicasByPercentageRunnersBusy(rd v1alpha1.RunnerDeployment, hra v1alpha1.HorizontalRunnerAutoscaler, metrics v1alpha1.MetricSpec) (*int, error) {
 	ctx := context.Background()
-	minReplicas := *hra.Spec.MinReplicas
-	maxReplicas := *hra.Spec.MaxReplicas
-	metrics := hra.Spec.Metrics[0]
 	scaleUpThreshold := defaultScaleUpThreshold
 	scaleDownThreshold := defaultScaleDownThreshold
 	scaleUpFactor := defaultScaleUpFactor
@@ -259,9 +296,30 @@ func (r *HorizontalRunnerAutoscalerReconciler) calculateReplicasByPercentageRunn
 
 	// return the list of runners in namespace. Horizontal Runner Autoscaler should only be responsible for scaling resources in its own ns.
 	var runnerList v1alpha1.RunnerList
-	if err := r.List(ctx, &runnerList, client.InNamespace(rd.Namespace)); err != nil {
+
+	var opts []client.ListOption
+
+	opts = append(opts, client.InNamespace(rd.Namespace))
+
+	selector, err := metav1.LabelSelectorAsSelector(getSelector(&rd))
+	if err != nil {
 		return nil, err
 	}
+
+	opts = append(opts, client.MatchingLabelsSelector{Selector: selector})
+
+	r.Log.V(2).Info("Finding runners with selector", "ns", rd.Namespace)
+
+	if err := r.List(
+		ctx,
+		&runnerList,
+		opts...,
+	); err != nil {
+		if !kerrors.IsNotFound(err) {
+			return nil, err
+		}
+	}
+
 	runnerMap := make(map[string]struct{})
 	for _, items := range runnerList.Items {
 		runnerMap[items.Name] = struct{}{}
@@ -282,45 +340,62 @@ func (r *HorizontalRunnerAutoscalerReconciler) calculateReplicasByPercentageRunn
 	if err != nil {
 		return nil, err
 	}
-	numRunners := len(runnerList.Items)
-	numRunnersBusy := 0
+
+	var desiredReplicasBefore int
+
+	if v := rd.Spec.Replicas; v == nil {
+		desiredReplicasBefore = 1
+	} else {
+		desiredReplicasBefore = *v
+	}
+
+	var (
+		numRunners           int
+		numRunnersRegistered int
+		numRunnersBusy       int
+	)
+
+	numRunners = len(runnerList.Items)
+
 	for _, runner := range runners {
-		if _, ok := runnerMap[*runner.Name]; ok && runner.GetBusy() {
-			numRunnersBusy++
+		if _, ok := runnerMap[*runner.Name]; ok {
+			numRunnersRegistered++
+
+			if runner.GetBusy() {
+				numRunnersBusy++
+			}
 		}
 	}
 
 	var desiredReplicas int
-	fractionBusy := float64(numRunnersBusy) / float64(numRunners)
+	fractionBusy := float64(numRunnersBusy) / float64(desiredReplicasBefore)
 	if fractionBusy >= scaleUpThreshold {
 		if scaleUpAdjustment > 0 {
-			desiredReplicas = numRunners + scaleUpAdjustment
+			desiredReplicas = desiredReplicasBefore + scaleUpAdjustment
 		} else {
-			desiredReplicas = int(math.Ceil(float64(numRunners) * scaleUpFactor))
+			desiredReplicas = int(math.Ceil(float64(desiredReplicasBefore) * scaleUpFactor))
 		}
 	} else if fractionBusy < scaleDownThreshold {
 		if scaleDownAdjustment > 0 {
-			desiredReplicas = numRunners - scaleDownAdjustment
+			desiredReplicas = desiredReplicasBefore - scaleDownAdjustment
 		} else {
-			desiredReplicas = int(float64(numRunners) * scaleDownFactor)
+			desiredReplicas = int(float64(desiredReplicasBefore) * scaleDownFactor)
 		}
 	} else {
 		desiredReplicas = *rd.Spec.Replicas
 	}
 
-	if desiredReplicas < minReplicas {
-		desiredReplicas = minReplicas
-	} else if desiredReplicas > maxReplicas {
-		desiredReplicas = maxReplicas
-	}
+	// NOTES for operators:
+	//
+	// - num_runners can be as twice as large as replicas_desired_before while
+	//   the runnerdeployment controller is replacing RunnerReplicaSet for runner update.
 
 	r.Log.V(1).Info(
-		"Calculated desired replicas",
-		"computed_replicas_desired", desiredReplicas,
-		"spec_replicas_min", minReplicas,
-		"spec_replicas_max", maxReplicas,
-		"current_replicas", rd.Spec.Replicas,
+		fmt.Sprintf("Suggested desired replicas of %d by PercentageRunnersBusy", desiredReplicas),
+		"replicas_desired_before", desiredReplicasBefore,
+		"replicas_desired", desiredReplicas,
 		"num_runners", numRunners,
+		"num_runners_registered", numRunnersRegistered,
 		"num_runners_busy", numRunnersBusy,
 		"namespace", hra.Namespace,
 		"runner_deployment", rd.Name,
@@ -330,8 +405,5 @@ func (r *HorizontalRunnerAutoscalerReconciler) calculateReplicasByPercentageRunn
 		"repository", repository,
 	)
 
-	rd.Status.Replicas = &desiredReplicas
-	replicas := desiredReplicas
-
-	return &replicas, nil
+	return &desiredReplicas, nil
 }

controllers/autoscaling_test.go

@@ -209,7 +209,7 @@ func TestDetermineDesiredReplicas_RepositoryRunner(t *testing.T) {
 					Replicas: tc.fixed,
 				},
 				Status: v1alpha1.RunnerDeploymentStatus{
-					Replicas: tc.sReplicas,
+					DesiredReplicas: tc.sReplicas,
 				},
 			}
 
@@ -224,7 +224,12 @@ func TestDetermineDesiredReplicas_RepositoryRunner(t *testing.T) {
 				},
 			}
 
-			got, err := h.computeReplicas(rd, hra)
+			minReplicas, _, _, err := h.getMinReplicas(log, metav1Now.Time, hra)
+			if err != nil {
+				t.Fatalf("unexpected error: %v", err)
+			}
+
+			got, _, _, err := h.computeReplicasWithCache(log, metav1Now.Time, rd, hra, minReplicas)
 			if err != nil {
 				if tc.err == "" {
 					t.Fatalf("unexpected error: expected none, got %v", err)
@@ -234,12 +239,8 @@ func TestDetermineDesiredReplicas_RepositoryRunner(t *testing.T) {
 				return
 			}
 
-			if got == nil {
-				t.Fatalf("unexpected value of rs.Spec.Replicas: nil")
-			}
-
-			if *got != tc.want {
-				t.Errorf("%d: incorrect desired replicas: want %d, got %d", i, tc.want, *got)
+			if got != tc.want {
+				t.Errorf("%d: incorrect desired replicas: want %d, got %d", i, tc.want, got)
 			}
 		})
 	}
@@ -424,6 +425,8 @@ func TestDetermineDesiredReplicas_OrganizationalRunner(t *testing.T) {
 		_ = v1alpha1.AddToScheme(scheme)
 
 		t.Run(fmt.Sprintf("case %d", i), func(t *testing.T) {
+			t.Helper()
+
 			server := fake.NewServer(
 				fake.WithListRepositoryWorkflowRunsResponse(200, tc.workflowRuns, tc.workflowRuns_queued, tc.workflowRuns_in_progress),
 				fake.WithListWorkflowJobsResponse(200, tc.workflowJobs),
@@ -443,7 +446,17 @@ func TestDetermineDesiredReplicas_OrganizationalRunner(t *testing.T) {
 					Name: "testrd",
 				},
 				Spec: v1alpha1.RunnerDeploymentSpec{
+					Selector: &metav1.LabelSelector{
+						MatchLabels: map[string]string{
+							"foo": "bar",
+						},
+					},
 					Template: v1alpha1.RunnerTemplate{
+						ObjectMeta: metav1.ObjectMeta{
+							Labels: map[string]string{
+								"foo": "bar",
+							},
+						},
 						Spec: v1alpha1.RunnerSpec{
 							Organization: tc.org,
 						},
@@ -451,7 +464,7 @@ func TestDetermineDesiredReplicas_OrganizationalRunner(t *testing.T) {
 					Replicas: tc.fixed,
 				},
 				Status: v1alpha1.RunnerDeploymentStatus{
-					Replicas: tc.sReplicas,
+					DesiredReplicas: tc.sReplicas,
 				},
 			}
 
@@ -475,7 +488,12 @@ func TestDetermineDesiredReplicas_OrganizationalRunner(t *testing.T) {
 				},
 			}
 
-			got, err := h.computeReplicas(rd, hra)
+			minReplicas, _, _, err := h.getMinReplicas(log, metav1Now.Time, hra)
+			if err != nil {
+				t.Fatalf("unexpected error: %v", err)
+			}
+
+			got, _, _, err := h.computeReplicasWithCache(log, metav1Now.Time, rd, hra, minReplicas)
 			if err != nil {
 				if tc.err == "" {
 					t.Fatalf("unexpected error: expected none, got %v", err)
@@ -485,12 +503,8 @@ func TestDetermineDesiredReplicas_OrganizationalRunner(t *testing.T) {
 				return
 			}
 
-			if got == nil {
-				t.Fatalf("unexpected value of rs.Spec.Replicas: nil, wanted %v", tc.want)
-			}
-
-			if *got != tc.want {
-				t.Errorf("%d: incorrect desired replicas: want %d, got %d", i, tc.want, *got)
+			if got != tc.want {
+				t.Errorf("%d: incorrect desired replicas: want %d, got %d", i, tc.want, got)
 			}
 		})
 	}

controllers/horizontal_runner_autoscaler_webhook.go

@@ -20,11 +20,13 @@ import (
 	"context"
 	"fmt"
 	"io/ioutil"
+	"net/http"
+	"strings"
+	"time"
+
 	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
 	"k8s.io/apimachinery/pkg/types"
-	"net/http"
 	"sigs.k8s.io/controller-runtime/pkg/reconcile"
-	"time"
 
 	"github.com/go-logr/logr"
 	gogithub "github.com/google/go-github/v33/github"
@@ -52,10 +54,11 @@ type HorizontalRunnerAutoscalerGitHubWebhook struct {
 	// the administrator is generated and specified in GitHub Web UI.
 	SecretKeyBytes []byte
 
-	// WatchNamespace is the namespace to watch for HorizontalRunnerAutoscaler's to be
+	// Namespace is the namespace to watch for HorizontalRunnerAutoscaler's to be
 	// scaled on Webhook.
 	// Set to empty for letting it watch for all namespaces.
-	WatchNamespace string
+	Namespace string
+	Name      string
 }
 
 func (autoscaler *HorizontalRunnerAutoscalerGitHubWebhook) Reconcile(request reconcile.Request) (reconcile.Result, error) {
@@ -93,6 +96,12 @@ func (autoscaler *HorizontalRunnerAutoscalerGitHubWebhook) Handle(w http.Respons
 		}
 	}()
 
+	// respond ok to GET / e.g. for health check
+	if r.Method == http.MethodGet {
+		fmt.Fprintln(w, "webhook server is running")
+		return
+	}
+
 	var payload []byte
 
 	if len(autoscaler.SecretKeyBytes) > 0 {
@@ -126,30 +135,54 @@ func (autoscaler *HorizontalRunnerAutoscalerGitHubWebhook) Handle(w http.Respons
 
 	var target *ScaleTarget
 
-	autoscaler.Log.Info("processing webhook event", "eventType", webhookType)
+	log := autoscaler.Log.WithValues(
+		"event", webhookType,
+		"hookID", r.Header.Get("X-GitHub-Hook-ID"),
+		"delivery", r.Header.Get("X-GitHub-Delivery"),
+	)
 
 	switch e := event.(type) {
 	case *gogithub.PushEvent:
 		target, err = autoscaler.getScaleUpTarget(
 			context.TODO(),
-			*e.Repo.Name,
-			*e.Repo.Organization,
+			log,
+			e.Repo.GetName(),
+			e.Repo.Owner.GetLogin(),
+			e.Repo.Owner.GetType(),
 			autoscaler.MatchPushEvent(e),
 		)
 	case *gogithub.PullRequestEvent:
 		target, err = autoscaler.getScaleUpTarget(
 			context.TODO(),
-			*e.Repo.Name,
-			*e.Repo.Organization.Name,
+			log,
+			e.Repo.GetName(),
+			e.Repo.Owner.GetLogin(),
+			e.Repo.Owner.GetType(),
 			autoscaler.MatchPullRequestEvent(e),
 		)
+
+		if pullRequest := e.PullRequest; pullRequest != nil {
+			log = log.WithValues(
+				"pullRequest.base.ref", e.PullRequest.Base.GetRef(),
+				"action", e.GetAction(),
+			)
+		}
 	case *gogithub.CheckRunEvent:
 		target, err = autoscaler.getScaleUpTarget(
 			context.TODO(),
-			*e.Repo.Name,
-			*e.Org.Name,
+			log,
+			e.Repo.GetName(),
+			e.Repo.Owner.GetLogin(),
+			e.Repo.Owner.GetType(),
 			autoscaler.MatchCheckRunEvent(e),
 		)
+
+		if checkRun := e.GetCheckRun(); checkRun != nil {
+			log = log.WithValues(
+				"checkRun.status", checkRun.GetStatus(),
+				"action", e.GetAction(),
+			)
+		}
 	case *gogithub.PingEvent:
 		ok = true
 
@@ -158,42 +191,44 @@ func (autoscaler *HorizontalRunnerAutoscalerGitHubWebhook) Handle(w http.Respons
 		msg := "pong"
 
 		if written, err := w.Write([]byte(msg)); err != nil {
-			autoscaler.Log.Error(err, "failed writing http response", "msg", msg, "written", written)
+			log.Error(err, "failed writing http response", "msg", msg, "written", written)
 		}
 
-		autoscaler.Log.Info("received ping event")
+		log.Info("received ping event")
 
 		return
 	default:
-		autoscaler.Log.Info("unknown event type", "eventType", webhookType)
+		log.Info("unknown event type", "eventType", webhookType)
 
 		return
 	}
 
 	if err != nil {
-		autoscaler.Log.Error(err, "handling check_run event")
+		log.Error(err, "handling check_run event")
 
 		return
 	}
 
 	if target == nil {
-		msg := "no horizontalrunnerautoscaler to scale for this github event"
+		log.Info(
+			"Scale target not found. If this is unexpected, ensure that there is exactly one repository-wide or organizational runner deployment that matches this webhook event",
+		)
 
-		autoscaler.Log.Info(msg, "eventType", webhookType)
+		msg := "no horizontalrunnerautoscaler to scale for this github event"
 
 		ok = true
 
 		w.WriteHeader(http.StatusOK)
 
 		if written, err := w.Write([]byte(msg)); err != nil {
-			autoscaler.Log.Error(err, "failed writing http response", "msg", msg, "written", written)
+			log.Error(err, "failed writing http response", "msg", msg, "written", written)
 		}
 
 		return
 	}
 
 	if err := autoscaler.tryScaleUp(context.TODO(), target); err != nil {
-		autoscaler.Log.Error(err, "could not scale up")
+		log.Error(err, "could not scale up")
 
 		return
 	}
@@ -207,12 +242,12 @@ func (autoscaler *HorizontalRunnerAutoscalerGitHubWebhook) Handle(w http.Respons
 	autoscaler.Log.Info(msg)
 
 	if written, err := w.Write([]byte(msg)); err != nil {
-		autoscaler.Log.Error(err, "failed writing http response", "msg", msg, "written", written)
+		log.Error(err, "failed writing http response", "msg", msg, "written", written)
 	}
 }
 
 func (autoscaler *HorizontalRunnerAutoscalerGitHubWebhook) findHRAsByKey(ctx context.Context, value string) ([]v1alpha1.HorizontalRunnerAutoscaler, error) {
-	ns := autoscaler.WatchNamespace
+	ns := autoscaler.Namespace
 
 	var defaultListOpts []client.ListOption
 
@@ -226,6 +261,10 @@ func (autoscaler *HorizontalRunnerAutoscalerGitHubWebhook) findHRAsByKey(ctx con
 		opts := append([]client.ListOption{}, defaultListOpts...)
 		opts = append(opts, client.MatchingFields{scaleTargetKey: value})
 
+		if autoscaler.Namespace != "" {
+			opts = append(opts, client.InNamespace(autoscaler.Namespace))
+		}
+
 		var hraList v1alpha1.HorizontalRunnerAutoscalerList
 
 		if err := autoscaler.List(ctx, &hraList, opts...); err != nil {
@@ -292,28 +331,64 @@ func (autoscaler *HorizontalRunnerAutoscalerGitHubWebhook) getScaleTarget(ctx co
 		return nil, err
 	}
 
+	autoscaler.Log.V(1).Info(fmt.Sprintf("Found %d HRAs by key", len(hras)), "key", name)
+
 	targets := autoscaler.searchScaleTargets(hras, f)
 
-	if len(targets) != 1 {
+	n := len(targets)
+
+	if n == 0 {
+		return nil, nil
+	}
+
+	if n > 1 {
+		var scaleTargetIDs []string
+
+		for _, t := range targets {
+			scaleTargetIDs = append(scaleTargetIDs, t.HorizontalRunnerAutoscaler.Name)
+		}
+
+		autoscaler.Log.Info(
+			"Found too many scale targets: "+
+				"It must be exactly one to avoid ambiguity. "+
+				"Either set Namespace for the webhook-based autoscaler to let it only find HRAs in the namespace, "+
+				"or update Repository or Organization fields in your RunnerDeployment resources to fix the ambiguity.",
+			"scaleTargets", strings.Join(scaleTargetIDs, ","))
+
 		return nil, nil
 	}
 
 	return &targets[0], nil
 }
 
-func (autoscaler *HorizontalRunnerAutoscalerGitHubWebhook) getScaleUpTarget(ctx context.Context, repoNameFromWebhook, orgNameFromWebhook string, f func(v1alpha1.ScaleUpTrigger) bool) (*ScaleTarget, error) {
-	if target, err := autoscaler.getScaleTarget(ctx, repoNameFromWebhook, f); err != nil {
+func (autoscaler *HorizontalRunnerAutoscalerGitHubWebhook) getScaleUpTarget(ctx context.Context, log logr.Logger, repo, owner, ownerType string, f func(v1alpha1.ScaleUpTrigger) bool) (*ScaleTarget, error) {
+	repositoryRunnerKey := owner + "/" + repo
+
+	if target, err := autoscaler.getScaleTarget(ctx, repositoryRunnerKey, f); err != nil {
+		log.Info("finding repository-wide runner", "repository", repositoryRunnerKey)
 		return nil, err
 	} else if target != nil {
-		autoscaler.Log.Info("scale up target is repository-wide runners", "repository", repoNameFromWebhook)
+		log.Info("scale up target is repository-wide runners", "repository", repo)
 		return target, nil
 	}
 
-	if target, err := autoscaler.getScaleTarget(ctx, orgNameFromWebhook, f); err != nil {
+	if ownerType == "User" {
+		log.V(1).Info("no repository runner found", "organization", owner)
+
+		return nil, nil
+	}
+
+	if target, err := autoscaler.getScaleTarget(ctx, owner, f); err != nil {
+		log.Info("finding organizational runner", "organization", owner)
 		return nil, err
 	} else if target != nil {
-		autoscaler.Log.Info("scale up target is organizational runners", "repository", orgNameFromWebhook)
+		log.Info("scale up target is organizational runners", "organization", owner)
 		return target, nil
+	} else {
+		log.V(1).Info("no repository runner or organizational runner found",
+			"repository", repositoryRunnerKey,
+			"organization", owner,
+		)
 	}
 
 	return nil, nil
@@ -324,8 +399,6 @@ func (autoscaler *HorizontalRunnerAutoscalerGitHubWebhook) tryScaleUp(ctx contex
 		return nil
 	}
 
-	log := autoscaler.Log.WithValues("horizontalrunnerautoscaler", target.HorizontalRunnerAutoscaler.Name)
-
 	copy := target.HorizontalRunnerAutoscaler.DeepCopy()
 
 	amount := 1
@@ -334,22 +407,40 @@ func (autoscaler *HorizontalRunnerAutoscalerGitHubWebhook) tryScaleUp(ctx contex
 		amount = target.ScaleUpTrigger.Amount
 	}
 
-	copy.Spec.CapacityReservations = append(copy.Spec.CapacityReservations, v1alpha1.CapacityReservation{
+	capacityReservations := getValidCapacityReservations(copy)
+
+	copy.Spec.CapacityReservations = append(capacityReservations, v1alpha1.CapacityReservation{
 		ExpirationTime: metav1.Time{Time: time.Now().Add(target.ScaleUpTrigger.Duration.Duration)},
 		Replicas:       amount,
 	})
 
-	if err := autoscaler.Client.Update(ctx, copy); err != nil {
-		log.Error(err, "Failed to update horizontalrunnerautoscaler resource")
-
-		return err
+	if err := autoscaler.Client.Patch(ctx, copy, client.MergeFrom(&target.HorizontalRunnerAutoscaler)); err != nil {
+		return fmt.Errorf("patching horizontalrunnerautoscaler to add capacity reservation: %w", err)
 	}
 
 	return nil
 }
 
+func getValidCapacityReservations(autoscaler *v1alpha1.HorizontalRunnerAutoscaler) []v1alpha1.CapacityReservation {
+	var capacityReservations []v1alpha1.CapacityReservation
+
+	now := time.Now()
+
+	for _, reservation := range autoscaler.Spec.CapacityReservations {
+		if reservation.ExpirationTime.Time.After(now) {
+			capacityReservations = append(capacityReservations, reservation)
+		}
+	}
+
+	return capacityReservations
+}
+
 func (autoscaler *HorizontalRunnerAutoscalerGitHubWebhook) SetupWithManager(mgr ctrl.Manager) error {
 	name := "webhookbasedautoscaler"
+	if autoscaler.Name != "" {
+		name = autoscaler.Name
+	}
+
 	autoscaler.Recorder = mgr.GetEventRecorderFor(name)
 
 	if err := mgr.GetFieldIndexer().IndexField(&v1alpha1.HorizontalRunnerAutoscaler{}, scaleTargetKey, func(rawObj runtime.Object) []string {

controllers/horizontal_runner_autoscaler_webhook_on_check_run.go

@@ -3,6 +3,7 @@ package controllers
 import (
 	"github.com/google/go-github/v33/github"
 	"github.com/summerwind/actions-runner-controller/api/v1alpha1"
+	"github.com/summerwind/actions-runner-controller/pkg/actionsglob"
 )
 
 func (autoscaler *HorizontalRunnerAutoscalerGitHubWebhook) MatchCheckRunEvent(event *github.CheckRunEvent) func(scaleUpTrigger v1alpha1.ScaleUpTrigger) bool {
@@ -27,6 +28,16 @@ func (autoscaler *HorizontalRunnerAutoscalerGitHubWebhook) MatchCheckRunEvent(ev
 			return false
 		}
 
+		if checkRun := event.CheckRun; checkRun != nil && len(cr.Names) > 0 {
+			for _, pat := range cr.Names {
+				if r := actionsglob.Match(pat, checkRun.GetName()); r {
+					return true
+				}
+			}
+
+			return false
+		}
+
 		return true
 	}
 }

controllers/horizontal_runner_autoscaler_webhook_test.go

@@ -9,13 +9,16 @@ import (
 	actionsv1alpha1 "github.com/summerwind/actions-runner-controller/api/v1alpha1"
 	"io"
 	"io/ioutil"
+	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
 	"k8s.io/apimachinery/pkg/runtime"
 	clientgoscheme "k8s.io/client-go/kubernetes/scheme"
 	"net/http"
 	"net/http/httptest"
 	"net/url"
+	"os"
 	"sigs.k8s.io/controller-runtime/pkg/client/fake"
 	"testing"
+	"time"
 )
 
 var (
@@ -27,21 +30,37 @@ func init() {
 	_ = actionsv1alpha1.AddToScheme(sc)
 }
 
-func TestWebhookCheckRun(t *testing.T) {
+func TestOrgWebhookCheckRun(t *testing.T) {
+	f, err := os.Open("testdata/org_webhook_check_run_payload.json")
+	if err != nil {
+		t.Fatalf("could not open the fixture: %s", err)
+	}
+	defer f.Close()
+	var e github.CheckRunEvent
+	if err := json.NewDecoder(f).Decode(&e); err != nil {
+		t.Fatalf("invalid json: %s", err)
+	}
 	testServer(t,
 		"check_run",
-		&github.CheckRunEvent{
-			CheckRun: &github.CheckRun{
-				Status: github.String("queued"),
-			},
-			Repo: &github.Repository{
-				Name: github.String("myorg/myrepo"),
-			},
-			Org: &github.Organization{
-				Name: github.String("myorg"),
-			},
-			Action: github.String("created"),
-		},
+		&e,
+		200,
+		"no horizontalrunnerautoscaler to scale for this github event",
+	)
+}
+
+func TestRepoWebhookCheckRun(t *testing.T) {
+	f, err := os.Open("testdata/repo_webhook_check_run_payload.json")
+	if err != nil {
+		t.Fatalf("could not open the fixture: %s", err)
+	}
+	defer f.Close()
+	var e github.CheckRunEvent
+	if err := json.NewDecoder(f).Decode(&e); err != nil {
+		t.Fatalf("invalid json: %s", err)
+	}
+	testServer(t,
+		"check_run",
+		&e,
 		200,
 		"no horizontalrunnerautoscaler to scale for this github event",
 	)
@@ -94,6 +113,56 @@ func TestWebhookPing(t *testing.T) {
 	)
 }
 
+func TestGetRequest(t *testing.T) {
+	hra := HorizontalRunnerAutoscalerGitHubWebhook{}
+	request, _ := http.NewRequest(http.MethodGet, "/", nil)
+	recorder := httptest.ResponseRecorder{}
+
+	hra.Handle(&recorder, request)
+	response := recorder.Result()
+
+	if response.StatusCode != http.StatusOK {
+		t.Errorf("want %d, got %d", http.StatusOK, response.StatusCode)
+	}
+}
+
+func TestGetValidCapacityReservations(t *testing.T) {
+	now := time.Now()
+
+	hra := &actionsv1alpha1.HorizontalRunnerAutoscaler{
+		Spec: actionsv1alpha1.HorizontalRunnerAutoscalerSpec{
+			CapacityReservations: []actionsv1alpha1.CapacityReservation{
+				{
+					ExpirationTime: metav1.Time{Time: now.Add(-time.Second)},
+					Replicas:       1,
+				},
+				{
+					ExpirationTime: metav1.Time{Time: now},
+					Replicas:       2,
+				},
+				{
+					ExpirationTime: metav1.Time{Time: now.Add(time.Second)},
+					Replicas:       3,
+				},
+			},
+		},
+	}
+
+	revs := getValidCapacityReservations(hra)
+
+	var count int
+
+	for _, r := range revs {
+		count += r.Replicas
+	}
+
+	want := 3
+
+	if count != want {
+		t.Errorf("want %d, got %d", want, count)
+	}
+}
+
 func installTestLogger(webhook *HorizontalRunnerAutoscalerGitHubWebhook) *bytes.Buffer {
 	logs := &bytes.Buffer{}
 

controllers/horizontalrunnerautoscaler_controller.go

@@ -18,8 +18,12 @@ package controllers
 
 import (
 	"context"
+	"fmt"
+	"reflect"
 	"time"
 
+	corev1 "k8s.io/api/core/v1"
+
 	"github.com/summerwind/actions-runner-controller/github"
 	"k8s.io/apimachinery/pkg/types"
 
@@ -29,10 +33,10 @@ import (
 	ctrl "sigs.k8s.io/controller-runtime"
 	"sigs.k8s.io/controller-runtime/pkg/client"
 
-	corev1 "k8s.io/api/core/v1"
 	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
 
 	"github.com/summerwind/actions-runner-controller/api/v1alpha1"
+	"github.com/summerwind/actions-runner-controller/controllers/metrics"
 )
 
 const (
@@ -48,8 +52,11 @@ type HorizontalRunnerAutoscalerReconciler struct {
 	Scheme       *runtime.Scheme
 
 	CacheDuration time.Duration
+	Name          string
 }
 
+const defaultReplicas = 1
+
 // +kubebuilder:rbac:groups=actions.summerwind.dev,resources=runnerdeployments,verbs=get;list;watch;update;patch
 // +kubebuilder:rbac:groups=actions.summerwind.dev,resources=horizontalrunnerautoscalers,verbs=get;list;watch;create;update;patch;delete
 // +kubebuilder:rbac:groups=actions.summerwind.dev,resources=horizontalrunnerautoscalers/finalizers,verbs=get;list;watch;create;update;patch;delete
@@ -69,6 +76,8 @@ func (r *HorizontalRunnerAutoscalerReconciler) Reconcile(req ctrl.Request) (ctrl
 		return ctrl.Result{}, nil
 	}
 
+	metrics.SetHorizontalRunnerAutoscalerSpec(hra.ObjectMeta, hra.Spec)
+
 	var rd v1alpha1.RunnerDeployment
 	if err := r.Get(ctx, types.NamespacedName{
 		Namespace: req.Namespace,
@@ -81,80 +90,50 @@ func (r *HorizontalRunnerAutoscalerReconciler) Reconcile(req ctrl.Request) (ctrl
 		return ctrl.Result{}, nil
 	}
 
-	var replicas *int
-
-	replicasFromCache := r.getDesiredReplicasFromCache(hra)
-
-	if replicasFromCache != nil {
-		replicas = replicasFromCache
-	} else {
-		var err error
-
-		replicas, err = r.computeReplicas(rd, hra)
-		if err != nil {
-			r.Recorder.Event(&hra, corev1.EventTypeNormal, "RunnerAutoscalingFailure", err.Error())
-
-			log.Error(err, "Could not compute replicas")
-
-			return ctrl.Result{}, err
-		}
-	}
-
-	const defaultReplicas = 1
-
-	currentDesiredReplicas := getIntOrDefault(rd.Spec.Replicas, defaultReplicas)
-	newDesiredReplicas := getIntOrDefault(replicas, defaultReplicas)
-
 	now := time.Now()
 
-	for _, reservation := range hra.Spec.CapacityReservations {
-		if reservation.ExpirationTime.Time.After(now) {
-			newDesiredReplicas += reservation.Replicas
-		}
+	minReplicas, active, upcoming, err := r.getMinReplicas(log, now, hra)
+	if err != nil {
+		log.Error(err, "Could not compute min replicas")
+
+		return ctrl.Result{}, err
 	}
 
-	if hra.Spec.MaxReplicas != nil && *hra.Spec.MaxReplicas < newDesiredReplicas {
-		newDesiredReplicas = *hra.Spec.MaxReplicas
+	newDesiredReplicas, computedReplicas, computedReplicasFromCache, err := r.computeReplicasWithCache(log, now, rd, hra, minReplicas)
+	if err != nil {
+		r.Recorder.Event(&hra, corev1.EventTypeNormal, "RunnerAutoscalingFailure", err.Error())
+
+		log.Error(err, "Could not compute replicas")
+
+		return ctrl.Result{}, err
 	}
 
+	currentDesiredReplicas := getIntOrDefault(rd.Spec.Replicas, defaultReplicas)
+
 	// Please add more conditions that we can in-place update the newest runnerreplicaset without disruption
 	if currentDesiredReplicas != newDesiredReplicas {
 		copy := rd.DeepCopy()
 		copy.Spec.Replicas = &newDesiredReplicas
 
-		if err := r.Client.Update(ctx, copy); err != nil {
-			log.Error(err, "Failed to update runnerderployment resource")
-
-			return ctrl.Result{}, err
+		if err := r.Client.Patch(ctx, copy, client.MergeFrom(&rd)); err != nil {
+			return ctrl.Result{}, fmt.Errorf("patching runnerdeployment to have %d replicas: %w", newDesiredReplicas, err)
 		}
 	}
 
-	var updated *v1alpha1.HorizontalRunnerAutoscaler
+	updated := hra.DeepCopy()
 
-	if hra.Status.DesiredReplicas == nil || *hra.Status.DesiredReplicas != *replicas {
-		updated = hra.DeepCopy()
-
-		if (hra.Status.DesiredReplicas == nil && *replicas > 1) ||
-			(hra.Status.DesiredReplicas != nil && *replicas > *hra.Status.DesiredReplicas) {
+	if hra.Status.DesiredReplicas == nil || *hra.Status.DesiredReplicas != newDesiredReplicas {
+		if (hra.Status.DesiredReplicas == nil && newDesiredReplicas > 1) ||
+			(hra.Status.DesiredReplicas != nil && newDesiredReplicas > *hra.Status.DesiredReplicas) {
 
 			updated.Status.LastSuccessfulScaleOutTime = &metav1.Time{Time: time.Now()}
 		}
 
-		updated.Status.DesiredReplicas = replicas
+		updated.Status.DesiredReplicas = &newDesiredReplicas
 	}
 
-	if replicasFromCache == nil {
-		if updated == nil {
-			updated = hra.DeepCopy()
-		}
-
-		var cacheEntries []v1alpha1.CacheEntry
-
-		for _, ent := range updated.Status.CacheEntries {
-			if ent.ExpirationTime.Before(&metav1.Time{Time: now}) {
-				cacheEntries = append(cacheEntries, ent)
-			}
-		}
+	if computedReplicasFromCache == nil {
+		cacheEntries := getValidCacheEntries(updated, now)
 
 		var cacheDuration time.Duration
 
@@ -164,26 +143,65 @@ func (r *HorizontalRunnerAutoscalerReconciler) Reconcile(req ctrl.Request) (ctrl
 			cacheDuration = 10 * time.Minute
 		}
 
-		updated.Status.CacheEntries = append(updated.Status.CacheEntries, v1alpha1.CacheEntry{
+		updated.Status.CacheEntries = append(cacheEntries, v1alpha1.CacheEntry{
 			Key:            v1alpha1.CacheEntryKeyDesiredReplicas,
-			Value:          *replicas,
+			Value:          computedReplicas,
 			ExpirationTime: metav1.Time{Time: time.Now().Add(cacheDuration)},
 		})
 	}
 
-	if updated != nil {
-		if err := r.Status().Update(ctx, updated); err != nil {
-			log.Error(err, "Failed to update horizontalrunnerautoscaler status")
+	var overridesSummary string
 
-			return ctrl.Result{}, err
+	if (active != nil && upcoming == nil) || (active != nil && upcoming != nil && active.Period.EndTime.Before(upcoming.Period.StartTime)) {
+		after := defaultReplicas
+		if hra.Spec.MinReplicas != nil && *hra.Spec.MinReplicas >= 0 {
+			after = *hra.Spec.MinReplicas
+		}
+
+		overridesSummary = fmt.Sprintf("min=%d time=%s", after, active.Period.EndTime)
+	}
+
+	if active == nil && upcoming != nil || (active != nil && upcoming != nil && active.Period.EndTime.After(upcoming.Period.StartTime)) {
+		if upcoming.ScheduledOverride.MinReplicas != nil {
+			overridesSummary = fmt.Sprintf("min=%d time=%s", *upcoming.ScheduledOverride.MinReplicas, upcoming.Period.StartTime)
+		}
+	}
+
+	if overridesSummary != "" {
+		updated.Status.ScheduledOverridesSummary = &overridesSummary
+	} else {
+		updated.Status.ScheduledOverridesSummary = nil
+	}
+
+	if !reflect.DeepEqual(hra.Status, updated.Status) {
+		metrics.SetHorizontalRunnerAutoscalerStatus(updated.ObjectMeta, updated.Status)
+
+		if err := r.Status().Patch(ctx, updated, client.MergeFrom(&hra)); err != nil {
+			return ctrl.Result{}, fmt.Errorf("patching horizontalrunnerautoscaler status: %w", err)
 		}
 	}
 
 	return ctrl.Result{}, nil
 }
 
+func getValidCacheEntries(hra *v1alpha1.HorizontalRunnerAutoscaler, now time.Time) []v1alpha1.CacheEntry {
+	var cacheEntries []v1alpha1.CacheEntry
+
+	for _, ent := range hra.Status.CacheEntries {
+		if ent.ExpirationTime.After(now) {
+			cacheEntries = append(cacheEntries, ent)
+		}
+	}
+
+	return cacheEntries
+}
+
 func (r *HorizontalRunnerAutoscalerReconciler) SetupWithManager(mgr ctrl.Manager) error {
 	name := "horizontalrunnerautoscaler-controller"
+	if r.Name != "" {
+		name = r.Name
+	}
+
 	r.Recorder = mgr.GetEventRecorderFor(name)
 
 	return ctrl.NewControllerManagedBy(mgr).
@@ -192,14 +210,132 @@ func (r *HorizontalRunnerAutoscalerReconciler) SetupWithManager(mgr ctrl.Manager
 		Complete(r)
 }
 
-func (r *HorizontalRunnerAutoscalerReconciler) computeReplicas(rd v1alpha1.RunnerDeployment, hra v1alpha1.HorizontalRunnerAutoscaler) (*int, error) {
-	var computedReplicas *int
+type Override struct {
+	ScheduledOverride v1alpha1.ScheduledOverride
+	Period            Period
+}
 
-	replicas, err := r.determineDesiredReplicas(rd, hra)
-	if err != nil {
-		return nil, err
+func (r *HorizontalRunnerAutoscalerReconciler) matchScheduledOverrides(log logr.Logger, now time.Time, hra v1alpha1.HorizontalRunnerAutoscaler) (*int, *Override, *Override, error) {
+	var minReplicas *int
+	var active, upcoming *Override
+
+	for _, o := range hra.Spec.ScheduledOverrides {
+		log.V(1).Info(
+			"Checking scheduled override",
+			"now", now,
+			"startTime", o.StartTime,
+			"endTime", o.EndTime,
+			"frequency", o.RecurrenceRule.Frequency,
+			"untilTime", o.RecurrenceRule.UntilTime,
+		)
+
+		a, u, err := MatchSchedule(
+			now, o.StartTime.Time, o.EndTime.Time,
+			RecurrenceRule{
+				Frequency: o.RecurrenceRule.Frequency,
+				UntilTime: o.RecurrenceRule.UntilTime.Time,
+			},
+		)
+		if err != nil {
+			return minReplicas, nil, nil, err
+		}
+
+		// Use the first when there are two or more active scheduled overrides,
+		// as the spec defines that the earlier scheduled override is prioritized higher than later ones.
+		if a != nil && active == nil {
+			active = &Override{Period: *a, ScheduledOverride: o}
+
+			if o.MinReplicas != nil {
+				minReplicas = o.MinReplicas
+
+				log.V(1).Info(
+					"Found active scheduled override",
+					"activeStartTime", a.StartTime,
+					"activeEndTime", a.EndTime,
+					"activeMinReplicas", minReplicas,
+				)
+			}
+		}
+
+		if u != nil && (upcoming == nil || u.StartTime.Before(upcoming.Period.StartTime)) {
+			upcoming = &Override{Period: *u, ScheduledOverride: o}
+
+			log.V(1).Info(
+				"Found upcoming scheduled override",
+				"upcomingStartTime", u.StartTime,
+				"upcomingEndTime", u.EndTime,
+				"upcomingMinReplicas", o.MinReplicas,
+			)
+		}
 	}
 
+	return minReplicas, active, upcoming, nil
+}
+
+func (r *HorizontalRunnerAutoscalerReconciler) getMinReplicas(log logr.Logger, now time.Time, hra v1alpha1.HorizontalRunnerAutoscaler) (int, *Override, *Override, error) {
+	minReplicas := defaultReplicas
+	if hra.Spec.MinReplicas != nil && *hra.Spec.MinReplicas >= 0 {
+		minReplicas = *hra.Spec.MinReplicas
+	}
+
+	m, active, upcoming, err := r.matchScheduledOverrides(log, now, hra)
+	if err != nil {
+		return 0, nil, nil, err
+	} else if m != nil {
+		minReplicas = *m
+	}
+
+	return minReplicas, active, upcoming, nil
+}
+
+func (r *HorizontalRunnerAutoscalerReconciler) computeReplicasWithCache(log logr.Logger, now time.Time, rd v1alpha1.RunnerDeployment, hra v1alpha1.HorizontalRunnerAutoscaler, minReplicas int) (int, int, *int, error) {
+	var suggestedReplicas int
+
+	suggestedReplicasFromCache := r.fetchSuggestedReplicasFromCache(hra)
+
+	var cached *int
+
+	if suggestedReplicasFromCache != nil {
+		cached = suggestedReplicasFromCache
+
+		if cached == nil {
+			suggestedReplicas = minReplicas
+		} else {
+			suggestedReplicas = *cached
+		}
+	} else {
+		v, err := r.suggestDesiredReplicas(rd, hra)
+		if err != nil {
+			return 0, 0, nil, err
+		}
+
+		if v == nil {
+			suggestedReplicas = minReplicas
+		} else {
+			suggestedReplicas = *v
+		}
+	}
+
+	var reserved int
+
+	for _, reservation := range hra.Spec.CapacityReservations {
+		if reservation.ExpirationTime.Time.After(now) {
+			reserved += reservation.Replicas
+		}
+	}
+
+	newDesiredReplicas := suggestedReplicas + reserved
+
+	if newDesiredReplicas < minReplicas {
+		newDesiredReplicas = minReplicas
+	} else if hra.Spec.MaxReplicas != nil && newDesiredReplicas > *hra.Spec.MaxReplicas {
+		newDesiredReplicas = *hra.Spec.MaxReplicas
+	}
+
+	//
+	// Delay scaling-down for ScaleDownDelaySecondsAfterScaleUp or DefaultScaleDownDelay
+	//
+
 	var scaleDownDelay time.Duration
 
 	if hra.Spec.ScaleDownDelaySecondsAfterScaleUp != nil {
@@ -208,17 +344,50 @@ func (r *HorizontalRunnerAutoscalerReconciler) computeReplicas(rd v1alpha1.Runne
 		scaleDownDelay = DefaultScaleDownDelay
 	}
 
-	now := time.Now()
+	var scaleDownDelayUntil *time.Time
 
 	if hra.Status.DesiredReplicas == nil ||
-		*hra.Status.DesiredReplicas < *replicas ||
-		hra.Status.LastSuccessfulScaleOutTime == nil ||
-		hra.Status.LastSuccessfulScaleOutTime.Add(scaleDownDelay).Before(now) {
+		*hra.Status.DesiredReplicas < newDesiredReplicas ||
+		hra.Status.LastSuccessfulScaleOutTime == nil {
 
-		computedReplicas = replicas
+	} else if hra.Status.LastSuccessfulScaleOutTime != nil {
+		t := hra.Status.LastSuccessfulScaleOutTime.Add(scaleDownDelay)
+
+		// ScaleDownDelay is not passed
+		if t.After(now) {
+			scaleDownDelayUntil = &t
+			newDesiredReplicas = *hra.Status.DesiredReplicas
+		}
 	} else {
-		computedReplicas = hra.Status.DesiredReplicas
+		newDesiredReplicas = *hra.Status.DesiredReplicas
 	}
 
-	return computedReplicas, nil
+	//
+	// Logs various numbers for monitoring and debugging purpose
+	//
+
+	kvs := []interface{}{
+		"suggested", suggestedReplicas,
+		"reserved", reserved,
+		"min", minReplicas,
+	}
+
+	if cached != nil {
+		kvs = append(kvs, "cached", *cached)
+	}
+
+	if scaleDownDelayUntil != nil {
+		kvs = append(kvs, "last_scale_up_time", *hra.Status.LastSuccessfulScaleOutTime)
+		kvs = append(kvs, "scale_down_delay_until", scaleDownDelayUntil)
+	}
+
+	if maxReplicas := hra.Spec.MaxReplicas; maxReplicas != nil {
+		kvs = append(kvs, "max", *maxReplicas)
+	}
+
+	log.V(1).Info(fmt.Sprintf("Calculated desired replicas of %d", newDesiredReplicas),
+		kvs...,
+	)
+
+	return newDesiredReplicas, suggestedReplicas, suggestedReplicasFromCache, nil
 }

controllers/horizontalrunnerautoscaler_controller_test.go

@@ -0,0 +1,49 @@
+package controllers
+
+import (
+	"github.com/google/go-cmp/cmp"
+	actionsv1alpha1 "github.com/summerwind/actions-runner-controller/api/v1alpha1"
+	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
+	"testing"
+	"time"
+)
+
+func TestGetValidCacheEntries(t *testing.T) {
+	now := time.Now()
+
+	hra := &actionsv1alpha1.HorizontalRunnerAutoscaler{
+		Status: actionsv1alpha1.HorizontalRunnerAutoscalerStatus{
+			CacheEntries: []actionsv1alpha1.CacheEntry{
+				{
+					Key:            "foo",
+					Value:          1,
+					ExpirationTime: metav1.Time{Time: now.Add(-time.Second)},
+				},
+				{
+					Key:            "foo",
+					Value:          2,
+					ExpirationTime: metav1.Time{Time: now},
+				},
+				{
+					Key:            "foo",
+					Value:          3,
+					ExpirationTime: metav1.Time{Time: now.Add(time.Second)},
+				},
+			},
+		},
+	}
+
+	revs := getValidCacheEntries(hra, now)
+
+	counts := map[string]int{}
+
+	for _, r := range revs {
+		counts[r.Key] += r.Value
+	}
+
+	want := map[string]int{"foo": 3}
+
+	if d := cmp.Diff(want, counts); d != "" {
+		t.Errorf("%s", d)
+	}
+}

controllers/metrics/horizontalrunnerautoscaler.go

@@ -0,0 +1,67 @@
+package metrics
+
+import (
+	"github.com/prometheus/client_golang/prometheus"
+	"github.com/summerwind/actions-runner-controller/api/v1alpha1"
+	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
+)
+
+const (
+	hraName      = "horizontalrunnerautoscaler"
+	hraNamespace = "namespace"
+)
+
+var (
+	horizontalRunnerAutoscalerMetrics = []prometheus.Collector{
+		horizontalRunnerAutoscalerMinReplicas,
+		horizontalRunnerAutoscalerMaxReplicas,
+		horizontalRunnerAutoscalerDesiredReplicas,
+	}
+)
+
+var (
+	horizontalRunnerAutoscalerMinReplicas = prometheus.NewGaugeVec(
+		prometheus.GaugeOpts{
+			Name: "horizontalrunnerautoscaler_spec_min_replicas",
+			Help: "minReplicas of HorizontalRunnerAutoscaler",
+		},
+		[]string{hraName, hraNamespace},
+	)
+	horizontalRunnerAutoscalerMaxReplicas = prometheus.NewGaugeVec(
+		prometheus.GaugeOpts{
+			Name: "horizontalrunnerautoscaler_spec_max_replicas",
+			Help: "maxReplicas of HorizontalRunnerAutoscaler",
+		},
+		[]string{hraName, hraNamespace},
+	)
+	horizontalRunnerAutoscalerDesiredReplicas = prometheus.NewGaugeVec(
+		prometheus.GaugeOpts{
+			Name: "horizontalrunnerautoscaler_status_desired_replicas",
+			Help: "desiredReplicas of HorizontalRunnerAutoscaler",
+		},
+		[]string{hraName, hraNamespace},
+	)
+)
+
+func SetHorizontalRunnerAutoscalerSpec(o metav1.ObjectMeta, spec v1alpha1.HorizontalRunnerAutoscalerSpec) {
+	labels := prometheus.Labels{
+		hraName:      o.Name,
+		hraNamespace: o.Namespace,
+	}
+	if spec.MaxReplicas != nil {
+		horizontalRunnerAutoscalerMaxReplicas.With(labels).Set(float64(*spec.MaxReplicas))
+	}
+	if spec.MinReplicas != nil {
+		horizontalRunnerAutoscalerMinReplicas.With(labels).Set(float64(*spec.MinReplicas))
+	}
+}
+
+func SetHorizontalRunnerAutoscalerStatus(o metav1.ObjectMeta, status v1alpha1.HorizontalRunnerAutoscalerStatus) {
+	labels := prometheus.Labels{
+		hraName:      o.Name,
+		hraNamespace: o.Namespace,
+	}
+	if status.DesiredReplicas != nil {
+		horizontalRunnerAutoscalerDesiredReplicas.With(labels).Set(float64(*status.DesiredReplicas))
+	}
+}

controllers/metrics/metrics.go

@@ -0,0 +1,14 @@
+// Package metrics provides the metrics of custom resources such as HRA.
+//
+// This depends on the metrics exporter of kubebuilder.
+// See https://book.kubebuilder.io/reference/metrics.html for details.
+package metrics
+
+import (
+	"sigs.k8s.io/controller-runtime/pkg/metrics"
+)
+
+func init() {
+	metrics.Registry.MustRegister(runnerDeploymentMetrics...)
+	metrics.Registry.MustRegister(horizontalRunnerAutoscalerMetrics...)
+}

controllers/metrics/runnerdeployment.go

@@ -0,0 +1,37 @@
+package metrics
+
+import (
+	"github.com/prometheus/client_golang/prometheus"
+	"github.com/summerwind/actions-runner-controller/api/v1alpha1"
+)
+
+const (
+	rdName      = "runnerdeployment"
+	rdNamespace = "namespace"
+)
+
+var (
+	runnerDeploymentMetrics = []prometheus.Collector{
+		runnerDeploymentReplicas,
+	}
+)
+
+var (
+	runnerDeploymentReplicas = prometheus.NewGaugeVec(
+		prometheus.GaugeOpts{
+			Name: "runnerdeployment_spec_replicas",
+			Help: "replicas of RunnerDeployment",
+		},
+		[]string{rdName, rdNamespace},
+	)
+)
+
+func SetRunnerDeployment(rd v1alpha1.RunnerDeployment) {
+	labels := prometheus.Labels{
+		rdName:      rd.Name,
+		rdNamespace: rd.Namespace,
+	}
+	if rd.Spec.Replicas != nil {
+		runnerDeploymentReplicas.With(labels).Set(float64(*rd.Spec.Replicas))
+	}
+}

controllers/runner_controller.go

@@ -20,11 +20,13 @@ import (
 	"context"
 	"errors"
 	"fmt"
-	gogithub "github.com/google/go-github/v33/github"
-	"github.com/summerwind/actions-runner-controller/hash"
 	"strings"
 	"time"
 
+	gogithub "github.com/google/go-github/v33/github"
+	"github.com/summerwind/actions-runner-controller/hash"
+	"k8s.io/apimachinery/pkg/util/wait"
+
 	"github.com/go-logr/logr"
 	kerrors "k8s.io/apimachinery/pkg/api/errors"
 	"k8s.io/apimachinery/pkg/runtime"
@@ -46,17 +48,23 @@ const (
 	LabelKeyPodTemplateHash = "pod-template-hash"
 
 	retryDelayOnGitHubAPIRateLimitError = 30 * time.Second
+
+	// This is an annotation internal to actions-runner-controller and can change in backward-incompatible ways
+	annotationKeyRegistrationOnly = "actions-runner-controller/registration-only"
 )
 
 // RunnerReconciler reconciles a Runner object
 type RunnerReconciler struct {
 	client.Client
-	Log          logr.Logger
-	Recorder     record.EventRecorder
-	Scheme       *runtime.Scheme
-	GitHubClient *github.Client
-	RunnerImage  string
-	DockerImage  string
+	Log                         logr.Logger
+	Recorder                    record.EventRecorder
+	Scheme                      *runtime.Scheme
+	GitHubClient                *github.Client
+	RunnerImage                 string
+	DockerImage                 string
+	Name                        string
+	RegistrationRecheckInterval time.Duration
+	RegistrationRecheckJitter   time.Duration
 }
 
 // +kubebuilder:rbac:groups=actions.summerwind.dev,resources=runners,verbs=get;list;watch;create;update;patch;delete
@@ -129,8 +137,8 @@ func (r *RunnerReconciler) Reconcile(req ctrl.Request) (ctrl.Result, error) {
 			newRunner := runner.DeepCopy()
 			newRunner.ObjectMeta.Finalizers = finalizers
 
-			if err := r.Update(ctx, newRunner); err != nil {
-				log.Error(err, "Failed to update runner")
+			if err := r.Patch(ctx, newRunner, client.MergeFrom(&runner)); err != nil {
+				log.Error(err, "Failed to update runner for finalizer removal")
 				return ctrl.Result{}, err
 			}
 
@@ -140,6 +148,34 @@ func (r *RunnerReconciler) Reconcile(req ctrl.Request) (ctrl.Result, error) {
 		return ctrl.Result{}, nil
 	}
 
+	registrationOnly := metav1.HasAnnotation(runner.ObjectMeta, annotationKeyRegistrationOnly)
+	if registrationOnly && runner.Status.Phase != "" {
+		// At this point we are sure that the registration-only runner has successfully configured and
+		// is of `offline` status, because we set runner.Status.Phase to that of the runner pod only after
+		// successful registration.
+
+		var pod corev1.Pod
+		if err := r.Get(ctx, req.NamespacedName, &pod); err != nil {
+			if !kerrors.IsNotFound(err) {
+				log.Info(fmt.Sprintf("Retrying soon as we failed to get registration-only runner pod: %v", err))
+
+				return ctrl.Result{Requeue: true}, nil
+			}
+		} else if err := r.Delete(ctx, &pod); err != nil {
+			if !kerrors.IsNotFound(err) {
+				log.Info(fmt.Sprintf("Retrying soon as we failed to delete registration-only runner pod: %v", err))
+
+				return ctrl.Result{Requeue: true}, nil
+			}
+		}
+
+		log.Info("Successfully deleted egistration-only runner pod to free node and cluster resource")
+
+		// Return here to not recreate the deleted pod, because recreating it is the waste of cluster and node resource,
+		// and also defeats the original purpose of scale-from/to-zero we're trying to implement by using the registration-only runner.
+		return ctrl.Result{}, nil
+	}
+
 	var pod corev1.Pod
 	if err := r.Get(ctx, req.NamespacedName, &pod); err != nil {
 		if !kerrors.IsNotFound(err) {
@@ -159,31 +195,25 @@ func (r *RunnerReconciler) Reconcile(req ctrl.Request) (ctrl.Result, error) {
 		}
 
 		if err := r.Create(ctx, &newPod); err != nil {
+			if kerrors.IsAlreadyExists(err) {
+				// Gracefully handle pod-already-exists errors due to informer cache delay.
+				// Without this we got a few errors like the below on new runner pod:
+				// 2021-03-16T00:23:10.116Z        ERROR   controller-runtime.controller   Reconciler error      {"controller": "runner-controller", "request": "default/example-runnerdeploy-b2g2g-j4mcp", "error": "pods \"example-runnerdeploy-b2g2g-j4mcp\" already exists"}
+				log.Info(
+					"Failed to create pod due to AlreadyExists error. Probably this pod has been already created in previous reconcilation but is still not in the informer cache. Will retry on pod created. If it doesn't repeat, there's no problem",
+				)
+
+				return ctrl.Result{}, nil
+			}
+
 			log.Error(err, "Failed to create pod resource")
+
 			return ctrl.Result{}, err
 		}
 
 		r.Recorder.Event(&runner, corev1.EventTypeNormal, "PodCreated", fmt.Sprintf("Created pod '%s'", newPod.Name))
 		log.Info("Created runner pod", "repository", runner.Spec.Repository)
 	} else {
-		// If pod has ended up succeeded we need to restart it
-		// Happens e.g. when dind is in runner and run completes
-		restart := pod.Status.Phase == corev1.PodSucceeded
-
-		if !restart && runner.Status.Phase != string(pod.Status.Phase) {
-			updated := runner.DeepCopy()
-			updated.Status.Phase = string(pod.Status.Phase)
-			updated.Status.Reason = pod.Status.Reason
-			updated.Status.Message = pod.Status.Message
-
-			if err := r.Status().Update(ctx, updated); err != nil {
-				log.Error(err, "Failed to update runner status")
-				return ctrl.Result{}, err
-			}
-
-			return ctrl.Result{}, nil
-		}
-
 		if !pod.ObjectMeta.DeletionTimestamp.IsZero() {
 			deletionTimeout := 1 * time.Minute
 			currentTime := time.Now()
@@ -191,7 +221,7 @@ func (r *RunnerReconciler) Reconcile(req ctrl.Request) (ctrl.Result, error) {
 
 			if deletionDidTimeout {
 				log.Info(
-					"Pod failed to delete itself in a timely manner. "+
+					fmt.Sprintf("Failed to delete pod within %s. ", deletionTimeout)+
 						"This is typically the case when a Kubernetes node became unreachable "+
 						"and the kube controller started evicting nodes. Forcefully deleting the pod to not get stuck.",
 					"podDeletionTimestamp", pod.DeletionTimestamp,
@@ -220,18 +250,35 @@ func (r *RunnerReconciler) Reconcile(req ctrl.Request) (ctrl.Result, error) {
 			}
 		}
 
-		if pod.Status.Phase == corev1.PodRunning {
-			for _, status := range pod.Status.ContainerStatuses {
-				if status.Name != containerName {
-					continue
-				}
+		// If pod has ended up succeeded we need to restart it
+		// Happens e.g. when dind is in runner and run completes
+		stopped := pod.Status.Phase == corev1.PodSucceeded
 
-				if status.State.Terminated != nil && status.State.Terminated.ExitCode == 0 {
-					restart = true
+		if !stopped {
+			if pod.Status.Phase == corev1.PodRunning {
+				for _, status := range pod.Status.ContainerStatuses {
+					if status.Name != containerName {
+						continue
+					}
+
+					if status.State.Terminated != nil && status.State.Terminated.ExitCode == 0 {
+						stopped = true
+					}
 				}
 			}
 		}
 
+		restart := stopped
+
+		if registrationOnly && stopped {
+			restart = false
+
+			log.Info(
+				"Observed that registration-only runner for scaling-from-zero has successfully stopped. " +
+					"Unlike other pods, this one will be recreated only when runner spec changes.",
+			)
+		}
+
 		if updated, err := r.updateRegistrationToken(ctx, runner); err != nil {
 			return ctrl.Result{}, err
 		} else if updated {
@@ -244,64 +291,199 @@ func (r *RunnerReconciler) Reconcile(req ctrl.Request) (ctrl.Result, error) {
 			return ctrl.Result{}, err
 		}
 
-		notRegistered := false
+		if registrationOnly {
+			newPod.Spec.Containers[0].Env = append(
+				newPod.Spec.Containers[0].Env,
+				corev1.EnvVar{
+					Name:  "RUNNER_REGISTRATION_ONLY",
+					Value: "true",
+				},
+			)
+		}
 
-		runnerBusy, err := r.GitHubClient.IsRunnerBusy(ctx, runner.Spec.Enterprise, runner.Spec.Organization, runner.Spec.Repository, runner.Name)
-		if err != nil {
-			var e *github.RunnerNotFound
-			if errors.As(err, &e) {
-				log.V(1).Info("Failed to check if runner is busy. Either this runner has never been successfully registered to GitHub or it still needs more time.", "runnerName", runner.Name)
+		var registrationRecheckDelay time.Duration
 
-				notRegistered = true
-			} else {
-				var e *gogithub.RateLimitError
-				if errors.As(err, &e) {
-					// We log the underlying error when we failed calling GitHub API to list or unregisters,
-					// or the runner is still busy.
-					log.Error(
-						err,
-						fmt.Sprintf(
-							"Failed to check if runner is busy due to Github API rate limit. Retrying in %s to avoid excessive GitHub API calls",
-							retryDelayOnGitHubAPIRateLimitError,
-						),
+		// all checks done below only decide whether a restart is needed
+		// if a restart was already decided before, there is no need for the checks
+		// saving API calls and scary log messages
+		if !restart {
+			registrationCheckInterval := time.Minute
+			if r.RegistrationRecheckInterval > 0 {
+				registrationCheckInterval = r.RegistrationRecheckInterval
+			}
+
+			// We want to call ListRunners GitHub Actions API only once per runner per minute.
+			// This if block, in conjunction with:
+			//   return ctrl.Result{RequeueAfter: registrationRecheckDelay}, nil
+			// achieves that.
+			if lastCheckTime := runner.Status.LastRegistrationCheckTime; lastCheckTime != nil {
+				nextCheckTime := lastCheckTime.Add(registrationCheckInterval)
+				now := time.Now()
+
+				// Requeue scheduled by RequeueAfter can happen a bit earlier (like dozens of milliseconds)
+				// so to avoid excessive, in-effective retry, we heuristically ignore the remaining delay in case it is
+				// shorter than 1s
+				requeueAfter := nextCheckTime.Sub(now) - time.Second
+				if requeueAfter > 0 {
+					log.Info(
+						fmt.Sprintf("Skipped registration check because it's deferred until %s. Retrying in %s at latest", nextCheckTime, requeueAfter),
+						"lastRegistrationCheckTime", lastCheckTime,
+						"registrationCheckInterval", registrationCheckInterval,
 					)
 
-					return ctrl.Result{RequeueAfter: retryDelayOnGitHubAPIRateLimitError}, err
+					// Without RequeueAfter, the controller may not retry on scheduled. Instead, it must wait until the
+					// next sync period passes, which can be too much later than nextCheckTime.
+					//
+					// We need to requeue on this reconcilation even though we have already scheduled the initial
+					// requeue previously with `return ctrl.Result{RequeueAfter: registrationRecheckDelay}, nil`.
+					// Apparently, the workqueue used by controller-runtime seems to deduplicate and resets the delay on
+					// other requeues- so the initial scheduled requeue may have been reset due to requeue on
+					// spec/status change.
+					return ctrl.Result{RequeueAfter: requeueAfter}, nil
+				}
+			}
+
+			notFound := false
+			offline := false
+
+			runnerBusy, err := r.GitHubClient.IsRunnerBusy(ctx, runner.Spec.Enterprise, runner.Spec.Organization, runner.Spec.Repository, runner.Name)
+
+			currentTime := time.Now()
+
+			if err != nil {
+				var notFoundException *github.RunnerNotFound
+				var offlineException *github.RunnerOffline
+				if errors.As(err, &notFoundException) {
+					notFound = true
+				} else if errors.As(err, &offlineException) {
+					offline = true
+				} else {
+					var e *gogithub.RateLimitError
+					if errors.As(err, &e) {
+						// We log the underlying error when we failed calling GitHub API to list or unregisters,
+						// or the runner is still busy.
+						log.Error(
+							err,
+							fmt.Sprintf(
+								"Failed to check if runner is busy due to Github API rate limit. Retrying in %s to avoid excessive GitHub API calls",
+								retryDelayOnGitHubAPIRateLimitError,
+							),
+						)
+
+						return ctrl.Result{RequeueAfter: retryDelayOnGitHubAPIRateLimitError}, err
+					}
+
+					return ctrl.Result{}, err
+				}
+			}
+
+			// See the `newPod` function called above for more information
+			// about when this hash changes.
+			curHash := pod.Labels[LabelKeyPodTemplateHash]
+			newHash := newPod.Labels[LabelKeyPodTemplateHash]
+
+			if !runnerBusy && curHash != newHash {
+				restart = true
+			}
+
+			registrationTimeout := 10 * time.Minute
+			durationAfterRegistrationTimeout := currentTime.Sub(pod.CreationTimestamp.Add(registrationTimeout))
+			registrationDidTimeout := durationAfterRegistrationTimeout > 0
+
+			if notFound {
+				if registrationDidTimeout {
+					log.Info(
+						"Runner failed to register itself to GitHub in timely manner. "+
+							"Recreating the pod to see if it resolves the issue. "+
+							"CAUTION: If you see this a lot, you should investigate the root cause. "+
+							"See https://github.com/summerwind/actions-runner-controller/issues/288",
+						"podCreationTimestamp", pod.CreationTimestamp,
+						"currentTime", currentTime,
+						"configuredRegistrationTimeout", registrationTimeout,
+					)
+
+					restart = true
+				} else {
+					log.V(1).Info(
+						"Runner pod exists but we failed to check if runner is busy. Apparently it still needs more time.",
+						"runnerName", runner.Name,
+					)
+				}
+			} else if offline {
+				if registrationOnly {
+					log.Info(
+						"Observed that registration-only runner for scaling-from-zero has successfully been registered.",
+						"podCreationTimestamp", pod.CreationTimestamp,
+						"currentTime", currentTime,
+						"configuredRegistrationTimeout", registrationTimeout,
+					)
+				} else if registrationDidTimeout {
+					log.Info(
+						"Already existing GitHub runner still appears offline . "+
+							"Recreating the pod to see if it resolves the issue. "+
+							"CAUTION: If you see this a lot, you should investigate the root cause. ",
+						"podCreationTimestamp", pod.CreationTimestamp,
+						"currentTime", currentTime,
+						"configuredRegistrationTimeout", registrationTimeout,
+					)
+
+					restart = true
+				} else {
+					log.V(1).Info(
+						"Runner pod exists but the GitHub runner appears to be still offline. Waiting for runner to get online ...",
+						"runnerName", runner.Name,
+					)
+				}
+			}
+
+			if (notFound || (offline && !registrationOnly)) && !registrationDidTimeout {
+				registrationRecheckJitter := 10 * time.Second
+				if r.RegistrationRecheckJitter > 0 {
+					registrationRecheckJitter = r.RegistrationRecheckJitter
 				}
 
-				return ctrl.Result{}, err
+				registrationRecheckDelay = registrationCheckInterval + wait.Jitter(registrationRecheckJitter, 0.1)
 			}
 		}
 
-		// See the `newPod` function called above for more information
-		// about when this hash changes.
-		curHash := pod.Labels[LabelKeyPodTemplateHash]
-		newHash := newPod.Labels[LabelKeyPodTemplateHash]
-
-		if !runnerBusy && curHash != newHash {
-			restart = true
-		}
-
-		registrationTimeout := 10 * time.Minute
-		currentTime := time.Now()
-		registrationDidTimeout := currentTime.Sub(pod.CreationTimestamp.Add(registrationTimeout)) > 0
-
-		if notRegistered && registrationDidTimeout {
-			log.Info(
-				"Runner failed to register itself to GitHub in timely manner. "+
-					"Recreating the pod to see if it resolves the issue. "+
-					"CAUTION: If you see this a lot, you should investigate the root cause. "+
-					"See https://github.com/summerwind/actions-runner-controller/issues/288",
-				"podCreationTimestamp", pod.CreationTimestamp,
-				"currentTime", currentTime,
-				"configuredRegistrationTimeout", registrationTimeout,
-			)
-
-			restart = true
-		}
-
 		// Don't do anything if there's no need to restart the runner
 		if !restart {
+			// This guard enables us to update runner.Status.Phase to `Running` only after
+			// the runner is registered to GitHub.
+			if registrationRecheckDelay > 0 {
+				log.V(1).Info(fmt.Sprintf("Rechecking the runner registration in %s", registrationRecheckDelay))
+
+				updated := runner.DeepCopy()
+				updated.Status.LastRegistrationCheckTime = &metav1.Time{Time: time.Now()}
+
+				if err := r.Status().Patch(ctx, updated, client.MergeFrom(&runner)); err != nil {
+					log.Error(err, "Failed to update runner status for LastRegistrationCheckTime")
+					return ctrl.Result{}, err
+				}
+
+				return ctrl.Result{RequeueAfter: registrationRecheckDelay}, nil
+			}
+
+			if runner.Status.Phase != string(pod.Status.Phase) {
+				if pod.Status.Phase == corev1.PodRunning {
+					// Seeing this message, you can expect the runner to become `Running` soon.
+					log.Info(
+						"Runner appears to have registered and running.",
+						"podCreationTimestamp", pod.CreationTimestamp,
+					)
+				}
+
+				updated := runner.DeepCopy()
+				updated.Status.Phase = string(pod.Status.Phase)
+				updated.Status.Reason = pod.Status.Reason
+				updated.Status.Message = pod.Status.Message
+
+				if err := r.Status().Patch(ctx, updated, client.MergeFrom(&runner)); err != nil {
+					log.Error(err, "Failed to update runner status for Phase/Reason/Message")
+					return ctrl.Result{}, err
+				}
+			}
+
 			return ctrl.Result{}, nil
 		}
 
@@ -369,8 +551,8 @@ func (r *RunnerReconciler) updateRegistrationToken(ctx context.Context, runner v
 		ExpiresAt:    metav1.NewTime(rt.GetExpiresAt().Time),
 	}
 
-	if err := r.Status().Update(ctx, updated); err != nil {
-		log.Error(err, "Failed to update runner status")
+	if err := r.Status().Patch(ctx, updated, client.MergeFrom(&runner)); err != nil {
+		log.Error(err, "Failed to update runner status for Registration")
 		return false, err
 	}
 
@@ -382,9 +564,11 @@ func (r *RunnerReconciler) updateRegistrationToken(ctx context.Context, runner v
 
 func (r *RunnerReconciler) newPod(runner v1alpha1.Runner) (corev1.Pod, error) {
 	var (
-		privileged      bool = true
-		dockerdInRunner bool = runner.Spec.DockerdWithinRunnerContainer != nil && *runner.Spec.DockerdWithinRunnerContainer
-		dockerEnabled   bool = runner.Spec.DockerEnabled == nil || *runner.Spec.DockerEnabled
+		privileged                bool = true
+		dockerdInRunner           bool = runner.Spec.DockerdWithinRunnerContainer != nil && *runner.Spec.DockerdWithinRunnerContainer
+		dockerEnabled             bool = runner.Spec.DockerEnabled == nil || *runner.Spec.DockerEnabled
+		ephemeral                 bool = runner.Spec.Ephemeral == nil || *runner.Spec.Ephemeral
+		dockerdInRunnerPrivileged bool = dockerdInRunner
 	)
 
 	runnerImage := runner.Spec.Image
@@ -443,6 +627,18 @@ func (r *RunnerReconciler) newPod(runner v1alpha1.Runner) (corev1.Pod, error) {
 			Name:  "RUNNER_WORKDIR",
 			Value: workDir,
 		},
+		{
+			Name:  "RUNNER_EPHEMERAL",
+			Value: fmt.Sprintf("%v", ephemeral),
+		},
+	}
+
+	if metav1.HasAnnotation(runner.ObjectMeta, annotationKeyRegistrationOnly) {
+		env = append(env, corev1.EnvVar{
+			Name:  "RUNNER_REGISTRATION_ONLY",
+			Value: "true",
+		},
+		)
 	}
 
 	env = append(env, runner.Spec.Env...)
@@ -479,6 +675,15 @@ func (r *RunnerReconciler) newPod(runner v1alpha1.Runner) (corev1.Pod, error) {
 		r.GitHubClient.GithubBaseURL,
 	)
 
+	var seLinuxOptions *corev1.SELinuxOptions
+	if runner.Spec.SecurityContext != nil {
+		seLinuxOptions = runner.Spec.SecurityContext.SELinuxOptions
+		if seLinuxOptions != nil {
+			privileged = false
+			dockerdInRunnerPrivileged = false
+		}
+	}
+
 	pod := corev1.Pod{
 		ObjectMeta: metav1.ObjectMeta{
 			Name:        runner.Name,
@@ -497,7 +702,7 @@ func (r *RunnerReconciler) newPod(runner v1alpha1.Runner) (corev1.Pod, error) {
 					EnvFrom:         runner.Spec.EnvFrom,
 					SecurityContext: &corev1.SecurityContext{
 						// Runner need to run privileged if it contains DinD
-						Privileged: runner.Spec.DockerdWithinRunnerContainer,
+						Privileged: &dockerdInRunnerPrivileged,
 					},
 					Resources: runner.Spec.Resources,
 				},
@@ -505,45 +710,81 @@ func (r *RunnerReconciler) newPod(runner v1alpha1.Runner) (corev1.Pod, error) {
 		},
 	}
 
-	if !dockerdInRunner && dockerEnabled {
-		runnerVolumeName := "runner"
-		runnerVolumeMountPath := "/runner"
-
-		pod.Spec.Volumes = []corev1.Volume{
+	if mtu := runner.Spec.DockerMTU; mtu != nil && dockerdInRunner {
+		pod.Spec.Containers[0].Env = append(pod.Spec.Containers[0].Env, []corev1.EnvVar{
 			{
+				Name:  "MTU",
+				Value: fmt.Sprintf("%d", *runner.Spec.DockerMTU),
+			},
+		}...)
+	}
+
+	if mirror := runner.Spec.DockerRegistryMirror; mirror != nil && dockerdInRunner {
+		pod.Spec.Containers[0].Env = append(pod.Spec.Containers[0].Env, []corev1.EnvVar{
+			{
+				Name:  "DOCKER_REGISTRY_MIRROR",
+				Value: *runner.Spec.DockerRegistryMirror,
+			},
+		}...)
+	}
+
+	//
+	// /runner must be generated on runtime from /runnertmp embedded in the container image.
+	//
+	// When you're NOT using dindWithinRunner=true,
+	// it must also be shared with the dind container as it seems like required to run docker steps.
+	//
+
+	runnerVolumeName := "runner"
+	runnerVolumeMountPath := "/runner"
+	runnerVolumeEmptyDir := &corev1.EmptyDirVolumeSource{}
+
+	if runner.Spec.VolumeSizeLimit != nil {
+		runnerVolumeEmptyDir.SizeLimit = runner.Spec.VolumeSizeLimit
+	}
+
+	pod.Spec.Volumes = append(pod.Spec.Volumes,
+		corev1.Volume{
+			Name: runnerVolumeName,
+			VolumeSource: corev1.VolumeSource{
+				EmptyDir: runnerVolumeEmptyDir,
+			},
+		},
+	)
+
+	pod.Spec.Containers[0].VolumeMounts = append(pod.Spec.Containers[0].VolumeMounts,
+		corev1.VolumeMount{
+			Name:      runnerVolumeName,
+			MountPath: runnerVolumeMountPath,
+		},
+	)
+
+	if !dockerdInRunner && dockerEnabled {
+		pod.Spec.Volumes = append(pod.Spec.Volumes,
+			corev1.Volume{
 				Name: "work",
 				VolumeSource: corev1.VolumeSource{
 					EmptyDir: &corev1.EmptyDirVolumeSource{},
 				},
 			},
-			{
-				Name: runnerVolumeName,
-				VolumeSource: corev1.VolumeSource{
-					EmptyDir: &corev1.EmptyDirVolumeSource{},
-				},
-			},
-			{
+			corev1.Volume{
 				Name: "certs-client",
 				VolumeSource: corev1.VolumeSource{
 					EmptyDir: &corev1.EmptyDirVolumeSource{},
 				},
 			},
-		}
-		pod.Spec.Containers[0].VolumeMounts = []corev1.VolumeMount{
-			{
+		)
+		pod.Spec.Containers[0].VolumeMounts = append(pod.Spec.Containers[0].VolumeMounts,
+			corev1.VolumeMount{
 				Name:      "work",
 				MountPath: workDir,
 			},
-			{
-				Name:      runnerVolumeName,
-				MountPath: runnerVolumeMountPath,
-			},
-			{
+			corev1.VolumeMount{
 				Name:      "certs-client",
 				MountPath: "/certs/client",
 				ReadOnly:  true,
 			},
-		}
+		)
 		pod.Spec.Containers[0].Env = append(pod.Spec.Containers[0].Env, []corev1.EnvVar{
 			{
 				Name:  "DOCKER_HOST",
@@ -558,23 +799,31 @@ func (r *RunnerReconciler) newPod(runner v1alpha1.Runner) (corev1.Pod, error) {
 				Value: "/certs/client",
 			},
 		}...)
-		pod.Spec.Containers = append(pod.Spec.Containers, corev1.Container{
-			Name:  "docker",
-			Image: r.DockerImage,
-			VolumeMounts: []corev1.VolumeMount{
-				{
-					Name:      "work",
-					MountPath: workDir,
-				},
-				{
-					Name:      runnerVolumeName,
-					MountPath: runnerVolumeMountPath,
-				},
-				{
-					Name:      "certs-client",
-					MountPath: "/certs/client",
-				},
+
+		// Determine the volume mounts assigned to the docker sidecar. In case extra mounts are included in the RunnerSpec, append them to the standard
+		// set of mounts. See https://github.com/summerwind/actions-runner-controller/issues/435 for context.
+		dockerVolumeMounts := []corev1.VolumeMount{
+			{
+				Name:      "work",
+				MountPath: workDir,
 			},
+			{
+				Name:      runnerVolumeName,
+				MountPath: runnerVolumeMountPath,
+			},
+			{
+				Name:      "certs-client",
+				MountPath: "/certs/client",
+			},
+		}
+		if extraDockerVolumeMounts := runner.Spec.DockerVolumeMounts; extraDockerVolumeMounts != nil {
+			dockerVolumeMounts = append(dockerVolumeMounts, extraDockerVolumeMounts...)
+		}
+
+		pod.Spec.Containers = append(pod.Spec.Containers, corev1.Container{
+			Name:         "docker",
+			Image:        r.DockerImage,
+			VolumeMounts: dockerVolumeMounts,
 			Env: []corev1.EnvVar{
 				{
 					Name:  "DOCKER_TLS_CERTDIR",
@@ -582,11 +831,32 @@ func (r *RunnerReconciler) newPod(runner v1alpha1.Runner) (corev1.Pod, error) {
 				},
 			},
 			SecurityContext: &corev1.SecurityContext{
-				Privileged: &privileged,
+				Privileged:     &privileged,
+				SELinuxOptions: seLinuxOptions,
 			},
 			Resources: runner.Spec.DockerdContainerResources,
 		})
 
+		if mtu := runner.Spec.DockerMTU; mtu != nil {
+			pod.Spec.Containers[1].Env = append(pod.Spec.Containers[1].Env, []corev1.EnvVar{
+				// See https://docs.docker.com/engine/security/rootless/
+				{
+					Name:  "DOCKERD_ROOTLESS_ROOTLESSKIT_MTU",
+					Value: fmt.Sprintf("%d", *runner.Spec.DockerMTU),
+				},
+			}...)
+
+			pod.Spec.Containers[1].Args = append(pod.Spec.Containers[1].Args,
+				"--mtu",
+				fmt.Sprintf("%d", *runner.Spec.DockerMTU),
+			)
+		}
+
+		if mirror := runner.Spec.DockerRegistryMirror; mirror != nil {
+			pod.Spec.Containers[1].Args = append(pod.Spec.Containers[1].Args,
+				fmt.Sprintf("--registry-mirror=%s", *runner.Spec.DockerRegistryMirror),
+			)
+		}
 	}
 
 	if len(runner.Spec.Containers) != 0 {
@@ -647,6 +917,14 @@ func (r *RunnerReconciler) newPod(runner v1alpha1.Runner) (corev1.Pod, error) {
 		pod.Spec.TerminationGracePeriodSeconds = runner.Spec.TerminationGracePeriodSeconds
 	}
 
+	if len(runner.Spec.HostAliases) != 0 {
+		pod.Spec.HostAliases = runner.Spec.HostAliases
+	}
+
+	if runner.Spec.RuntimeClassName != nil {
+		pod.Spec.RuntimeClassName = runner.Spec.RuntimeClassName
+	}
+
 	if err := ctrl.SetControllerReference(&runner, &pod, r.Scheme); err != nil {
 		return pod, err
 	}
@@ -656,6 +934,9 @@ func (r *RunnerReconciler) newPod(runner v1alpha1.Runner) (corev1.Pod, error) {
 
 func (r *RunnerReconciler) SetupWithManager(mgr ctrl.Manager) error {
 	name := "runner-controller"
+	if r.Name != "" {
+		name = r.Name
+	}
 
 	r.Recorder = mgr.GetEventRecorderFor(name)
 

controllers/runnerdeployment_controller.go

@@ -20,6 +20,7 @@ import (
 	"context"
 	"fmt"
 	"hash/fnv"
+	"reflect"
 	"sort"
 	"time"
 
@@ -37,10 +38,12 @@ import (
 	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
 
 	"github.com/summerwind/actions-runner-controller/api/v1alpha1"
+	"github.com/summerwind/actions-runner-controller/controllers/metrics"
 )
 
 const (
-	LabelKeyRunnerTemplateHash = "runner-template-hash"
+	LabelKeyRunnerTemplateHash   = "runner-template-hash"
+	LabelKeyRunnerDeploymentName = "runner-deployment-name"
 
 	runnerSetOwnerKey = ".metadata.controller"
 )
@@ -48,9 +51,11 @@ const (
 // RunnerDeploymentReconciler reconciles a Runner object
 type RunnerDeploymentReconciler struct {
 	client.Client
-	Log      logr.Logger
-	Recorder record.EventRecorder
-	Scheme   *runtime.Scheme
+	Log                logr.Logger
+	Recorder           record.EventRecorder
+	Scheme             *runtime.Scheme
+	CommonRunnerLabels []string
+	Name               string
 }
 
 // +kubebuilder:rbac:groups=actions.summerwind.dev,resources=runnerdeployments,verbs=get;list;watch;create;update;patch;delete
@@ -73,6 +78,8 @@ func (r *RunnerDeploymentReconciler) Reconcile(req ctrl.Request) (ctrl.Result, e
 		return ctrl.Result{}, nil
 	}
 
+	metrics.SetRunnerDeployment(rd)
+
 	var myRunnerReplicaSetList v1alpha1.RunnerReplicaSetList
 	if err := r.List(ctx, &myRunnerReplicaSetList, client.InNamespace(req.Namespace), client.MatchingFields{runnerSetOwnerKey: req.Name}); err != nil {
 		return ctrl.Result{}, err
@@ -141,6 +148,28 @@ func (r *RunnerDeploymentReconciler) Reconcile(req ctrl.Request) (ctrl.Result, e
 		return ctrl.Result{RequeueAfter: 5 * time.Second}, nil
 	}
 
+	if !reflect.DeepEqual(newestSet.Spec.Selector, desiredRS.Spec.Selector) {
+		updateSet := newestSet.DeepCopy()
+		updateSet.Spec = *desiredRS.Spec.DeepCopy()
+
+		// A selector update change doesn't trigger replicaset replacement,
+		// but we still need to update the existing replicaset with it.
+		// Otherwise selector-based runner query will never work on replicasets created before the controller v0.17.0
+		// See https://github.com/summerwind/actions-runner-controller/pull/355#discussion_r585379259
+		if err := r.Client.Update(ctx, updateSet); err != nil {
+			log.Error(err, "Failed to update runnerreplicaset resource")
+
+			return ctrl.Result{}, err
+		}
+
+		// At this point, we are already sure that there's no need to create a new replicaset
+		// as the runner template hash is not changed.
+		//
+		// But we still need to requeue for the (possibly rare) cases that there are still old replicasets that needs
+		// to be cleaned up.
+		return ctrl.Result{RequeueAfter: 5 * time.Second}, nil
+	}
+
 	const defaultReplicas = 1
 
 	currentDesiredReplicas := getIntOrDefault(newestSet.Spec.Replicas, defaultReplicas)
@@ -159,18 +188,35 @@ func (r *RunnerDeploymentReconciler) Reconcile(req ctrl.Request) (ctrl.Result, e
 		return ctrl.Result{}, err
 	}
 
-	// Do we old runner replica sets that should eventually deleted?
+	// Do we have old runner replica sets that should eventually deleted?
 	if len(oldSets) > 0 {
-		readyReplicas := newestSet.Status.ReadyReplicas
+		var readyReplicas int
+		if newestSet.Status.ReadyReplicas != nil {
+			readyReplicas = *newestSet.Status.ReadyReplicas
+		}
 
-		if readyReplicas < currentDesiredReplicas {
-			log.WithValues("runnerreplicaset", types.NamespacedName{
+		oldSetsCount := len(oldSets)
+
+		logWithDebugInfo := log.WithValues(
+			"newest_runnerreplicaset", types.NamespacedName{
 				Namespace: newestSet.Namespace,
 				Name:      newestSet.Name,
-			}).
-				Info("Waiting until the newest runner replica set to be 100% available")
+			},
+			"newest_runnerreplicaset_replicas_ready", readyReplicas,
+			"newest_runnerreplicaset_replicas_desired", currentDesiredReplicas,
+			"old_runnerreplicasets_count", oldSetsCount,
+		)
 
-			return ctrl.Result{RequeueAfter: 10 * time.Second}, nil
+		if readyReplicas < currentDesiredReplicas {
+			logWithDebugInfo.
+				Info("Waiting until the newest runnerreplicaset to be 100% available")
+
+			return ctrl.Result{}, nil
+		}
+
+		if oldSetsCount > 0 {
+			logWithDebugInfo.
+				Info("The newest runnerreplicaset is 100% available. Deleting old runnerreplicasets")
 		}
 
 		for i := range oldSets {
@@ -188,14 +234,49 @@ func (r *RunnerDeploymentReconciler) Reconcile(req ctrl.Request) (ctrl.Result, e
 		}
 	}
 
-	if rd.Spec.Replicas == nil && desiredRS.Spec.Replicas != nil {
+	var replicaSets []v1alpha1.RunnerReplicaSet
+
+	replicaSets = append(replicaSets, *newestSet)
+	replicaSets = append(replicaSets, oldSets...)
+
+	var totalCurrentReplicas, totalStatusAvailableReplicas, updatedReplicas int
+
+	for _, rs := range replicaSets {
+		var current, available int
+
+		if rs.Status.Replicas != nil {
+			current = *rs.Status.Replicas
+		}
+
+		if rs.Status.AvailableReplicas != nil {
+			available = *rs.Status.AvailableReplicas
+		}
+
+		totalCurrentReplicas += current
+		totalStatusAvailableReplicas += available
+	}
+
+	if newestSet.Status.Replicas != nil {
+		updatedReplicas = *newestSet.Status.Replicas
+	}
+
+	var status v1alpha1.RunnerDeploymentStatus
+
+	status.AvailableReplicas = &totalStatusAvailableReplicas
+	status.ReadyReplicas = &totalStatusAvailableReplicas
+	status.DesiredReplicas = &newDesiredReplicas
+	status.Replicas = &totalCurrentReplicas
+	status.UpdatedReplicas = &updatedReplicas
+
+	if !reflect.DeepEqual(rd.Status, status) {
 		updated := rd.DeepCopy()
-		updated.Status.Replicas = desiredRS.Spec.Replicas
+		updated.Status = status
 
-		if err := r.Status().Update(ctx, updated); err != nil {
-			log.Error(err, "Failed to update runnerdeployment status")
-
-			return ctrl.Result{}, err
+		if err := r.Status().Patch(ctx, updated, client.MergeFrom(&rd)); err != nil {
+			log.Info("Failed to patch runnerdeployment status. Retrying immediately", "error", err.Error())
+			return ctrl.Result{
+				Requeue: true,
+			}, nil
 		}
 	}
 
@@ -256,28 +337,94 @@ func CloneAndAddLabel(labels map[string]string, labelKey, labelValue string) map
 	return newLabels
 }
 
-func (r *RunnerDeploymentReconciler) newRunnerReplicaSet(rd v1alpha1.RunnerDeployment) (*v1alpha1.RunnerReplicaSet, error) {
-	newRSTemplate := *rd.Spec.Template.DeepCopy()
-	templateHash := ComputeHash(&newRSTemplate)
-	// Add template hash label to selector.
-	labels := CloneAndAddLabel(rd.Spec.Template.Labels, LabelKeyRunnerTemplateHash, templateHash)
+// Clones the given selector and returns a new selector with the given key and value added.
+// Returns the given selector, if labelKey is empty.
+//
+// Proudly copied from k8s.io/kubernetes/pkg/util/labels.CloneSelectorAndAddLabel
+func CloneSelectorAndAddLabel(selector *metav1.LabelSelector, labelKey, labelValue string) *metav1.LabelSelector {
+	if labelKey == "" {
+		// Don't need to add a label.
+		return selector
+	}
 
-	newRSTemplate.Labels = labels
+	// Clone.
+	newSelector := new(metav1.LabelSelector)
+
+	newSelector.MatchLabels = make(map[string]string)
+	if selector.MatchLabels != nil {
+		for key, val := range selector.MatchLabels {
+			newSelector.MatchLabels[key] = val
+		}
+	}
+	newSelector.MatchLabels[labelKey] = labelValue
+
+	if selector.MatchExpressions != nil {
+		newMExps := make([]metav1.LabelSelectorRequirement, len(selector.MatchExpressions))
+		for i, me := range selector.MatchExpressions {
+			newMExps[i].Key = me.Key
+			newMExps[i].Operator = me.Operator
+			if me.Values != nil {
+				newMExps[i].Values = make([]string, len(me.Values))
+				copy(newMExps[i].Values, me.Values)
+			} else {
+				newMExps[i].Values = nil
+			}
+		}
+		newSelector.MatchExpressions = newMExps
+	} else {
+		newSelector.MatchExpressions = nil
+	}
+
+	return newSelector
+}
+
+func (r *RunnerDeploymentReconciler) newRunnerReplicaSet(rd v1alpha1.RunnerDeployment) (*v1alpha1.RunnerReplicaSet, error) {
+	return newRunnerReplicaSet(&rd, r.CommonRunnerLabels, r.Scheme)
+}
+
+func getSelector(rd *v1alpha1.RunnerDeployment) *metav1.LabelSelector {
+	selector := rd.Spec.Selector
+	if selector == nil {
+		selector = &metav1.LabelSelector{MatchLabels: map[string]string{LabelKeyRunnerDeploymentName: rd.Name}}
+	}
+
+	return selector
+}
+
+func newRunnerReplicaSet(rd *v1alpha1.RunnerDeployment, commonRunnerLabels []string, scheme *runtime.Scheme) (*v1alpha1.RunnerReplicaSet, error) {
+	newRSTemplate := *rd.Spec.Template.DeepCopy()
+
+	for _, l := range commonRunnerLabels {
+		newRSTemplate.Spec.Labels = append(newRSTemplate.Spec.Labels, l)
+	}
+
+	templateHash := ComputeHash(&newRSTemplate)
+
+	// Add template hash label to selector.
+	newRSTemplate.ObjectMeta.Labels = CloneAndAddLabel(newRSTemplate.ObjectMeta.Labels, LabelKeyRunnerTemplateHash, templateHash)
+
+	// This label selector is used by default when rd.Spec.Selector is empty.
+	newRSTemplate.ObjectMeta.Labels = CloneAndAddLabel(newRSTemplate.ObjectMeta.Labels, LabelKeyRunnerDeploymentName, rd.Name)
+
+	selector := getSelector(rd)
+
+	newRSSelector := CloneSelectorAndAddLabel(selector, LabelKeyRunnerTemplateHash, templateHash)
 
 	rs := v1alpha1.RunnerReplicaSet{
 		TypeMeta: metav1.TypeMeta{},
 		ObjectMeta: metav1.ObjectMeta{
 			GenerateName: rd.ObjectMeta.Name + "-",
 			Namespace:    rd.ObjectMeta.Namespace,
-			Labels:       labels,
+			Labels:       newRSTemplate.ObjectMeta.Labels,
 		},
 		Spec: v1alpha1.RunnerReplicaSetSpec{
 			Replicas: rd.Spec.Replicas,
+			Selector: newRSSelector,
 			Template: newRSTemplate,
 		},
 	}
 
-	if err := ctrl.SetControllerReference(&rd, &rs, r.Scheme); err != nil {
+	if err := ctrl.SetControllerReference(rd, &rs, scheme); err != nil {
 		return &rs, err
 	}
 
@@ -286,6 +433,10 @@ func (r *RunnerDeploymentReconciler) newRunnerReplicaSet(rd v1alpha1.RunnerDeplo
 
 func (r *RunnerDeploymentReconciler) SetupWithManager(mgr ctrl.Manager) error {
 	name := "runnerdeployment-controller"
+	if r.Name != "" {
+		name = r.Name
+	}
+
 	r.Recorder = mgr.GetEventRecorderFor(name)
 
 	if err := mgr.GetFieldIndexer().IndexField(&v1alpha1.RunnerReplicaSet{}, runnerSetOwnerKey, func(rawObj runtime.Object) []string {

controllers/runnerdeployment_controller_test.go

@@ -2,8 +2,13 @@ package controllers
 
 import (
 	"context"
+	"fmt"
+	"testing"
 	"time"
 
+	"github.com/google/go-cmp/cmp"
+	"k8s.io/apimachinery/pkg/runtime"
+
 	corev1 "k8s.io/api/core/v1"
 	"k8s.io/apimachinery/pkg/types"
 	"k8s.io/client-go/kubernetes/scheme"
@@ -18,6 +23,103 @@ import (
 	actionsv1alpha1 "github.com/summerwind/actions-runner-controller/api/v1alpha1"
 )
 
+func TestNewRunnerReplicaSet(t *testing.T) {
+	scheme := runtime.NewScheme()
+	if err := actionsv1alpha1.AddToScheme(scheme); err != nil {
+		t.Fatalf("%v", err)
+	}
+
+	r := &RunnerDeploymentReconciler{
+		CommonRunnerLabels: []string{"dev"},
+		Scheme:             scheme,
+	}
+	rd := actionsv1alpha1.RunnerDeployment{
+		ObjectMeta: metav1.ObjectMeta{
+			Name: "example",
+		},
+		Spec: actionsv1alpha1.RunnerDeploymentSpec{
+			Selector: &metav1.LabelSelector{
+				MatchLabels: map[string]string{
+					"foo": "bar",
+				},
+			},
+			Template: actionsv1alpha1.RunnerTemplate{
+				ObjectMeta: metav1.ObjectMeta{
+					Labels: map[string]string{
+						"foo": "bar",
+					},
+				},
+				Spec: actionsv1alpha1.RunnerSpec{
+					Labels: []string{"project1"},
+				},
+			},
+		},
+	}
+
+	rs, err := r.newRunnerReplicaSet(rd)
+	if err != nil {
+		t.Fatalf("%v", err)
+	}
+
+	if val, ok := rs.Labels["foo"]; ok {
+		if val != "bar" {
+			t.Errorf("foo label does not have bar but %v", val)
+		}
+	} else {
+		t.Errorf("foo label does not exist")
+	}
+
+	hash1, ok := rs.Labels[LabelKeyRunnerTemplateHash]
+	if !ok {
+		t.Errorf("missing runner-template-hash label")
+	}
+
+	runnerLabel := []string{"project1", "dev"}
+	if d := cmp.Diff(runnerLabel, rs.Spec.Template.Spec.Labels); d != "" {
+		t.Errorf("%s", d)
+	}
+
+	rd2 := rd.DeepCopy()
+	rd2.Spec.Template.Spec.Labels = []string{"project2"}
+
+	rs2, err := r.newRunnerReplicaSet(*rd2)
+	if err != nil {
+		t.Fatalf("%v", err)
+	}
+
+	hash2, ok := rs2.Labels[LabelKeyRunnerTemplateHash]
+	if !ok {
+		t.Errorf("missing runner-template-hash label")
+	}
+
+	if hash1 == hash2 {
+		t.Errorf(
+			"runner replica sets from runner deployments with varying labels must have different template hash, but got %s and %s",
+			hash1, hash2,
+		)
+	}
+
+	rd3 := rd.DeepCopy()
+	rd3.Spec.Template.Labels["foo"] = "baz"
+
+	rs3, err := r.newRunnerReplicaSet(*rd3)
+	if err != nil {
+		t.Fatalf("%v", err)
+	}
+
+	hash3, ok := rs3.Labels[LabelKeyRunnerTemplateHash]
+	if !ok {
+		t.Errorf("missing runner-template-hash label")
+	}
+
+	if hash1 == hash3 {
+		t.Errorf(
+			"runner replica sets from runner deployments with varying meta labels must have different template hash, but got %s and %s",
+			hash1, hash3,
+		)
+	}
+}
+
 // SetupDeploymentTest will set up a testing environment.
 // This includes:
 // * creating a Namespace to be used during the test
@@ -37,7 +139,9 @@ func SetupDeploymentTest(ctx context.Context) *corev1.Namespace {
 		err := k8sClient.Create(ctx, ns)
 		Expect(err).NotTo(HaveOccurred(), "failed to create test namespace")
 
-		mgr, err := ctrl.NewManager(cfg, ctrl.Options{})
+		mgr, err := ctrl.NewManager(cfg, ctrl.Options{
+			Namespace: ns.Name,
+		})
 		Expect(err).NotTo(HaveOccurred(), "failed to create manager")
 
 		controller := &RunnerDeploymentReconciler{
@@ -45,6 +149,7 @@ func SetupDeploymentTest(ctx context.Context) *corev1.Namespace {
 			Scheme:   scheme.Scheme,
 			Log:      logf.Log,
 			Recorder: mgr.GetEventRecorderFor("runnerreplicaset-controller"),
+			Name:     "runnerdeployment-" + ns.Name,
 		}
 		err = controller.SetupWithManager(mgr)
 		Expect(err).NotTo(HaveOccurred(), "failed to setup controller")
@@ -74,7 +179,7 @@ var _ = Context("Inside of a new namespace", func() {
 	Describe("when no existing resources exist", func() {
 
 		It("should create a new RunnerReplicaSet resource from the specified template, add a another RunnerReplicaSet on template modification, and eventually removes old runnerreplicasets", func() {
-			name := "example-runnerdeploy"
+			name := "example-runnerdeploy-1"
 
 			{
 				rs := &actionsv1alpha1.RunnerDeployment{
@@ -84,9 +189,19 @@ var _ = Context("Inside of a new namespace", func() {
 					},
 					Spec: actionsv1alpha1.RunnerDeploymentSpec{
 						Replicas: intPtr(1),
+						Selector: &metav1.LabelSelector{
+							MatchLabels: map[string]string{
+								"foo": "bar",
+							},
+						},
 						Template: actionsv1alpha1.RunnerTemplate{
+							ObjectMeta: metav1.ObjectMeta{
+								Labels: map[string]string{
+									"foo": "bar",
+								},
+							},
 							Spec: actionsv1alpha1.RunnerSpec{
-								Repository: "foo/bar",
+								Repository: "test/valid",
 								Image:      "bar",
 								Env: []corev1.EnvVar{
 									{Name: "FOO", Value: "FOOVALUE"},
@@ -103,29 +218,25 @@ var _ = Context("Inside of a new namespace", func() {
 				runnerSets := actionsv1alpha1.RunnerReplicaSetList{Items: []actionsv1alpha1.RunnerReplicaSet{}}
 
 				Eventually(
-					func() int {
-						err := k8sClient.List(ctx, &runnerSets, client.InNamespace(ns.Name))
+					func() (int, error) {
+						selector, err := metav1.LabelSelectorAsSelector(rs.Spec.Selector)
 						if err != nil {
-							logf.Log.Error(err, "list runner sets")
+							return 0, err
 						}
-
-						return len(runnerSets.Items)
-					},
-					time.Second*5, time.Millisecond*500).Should(BeEquivalentTo(1))
-
-				Eventually(
-					func() int {
-						err := k8sClient.List(ctx, &runnerSets, client.InNamespace(ns.Name))
+						err = k8sClient.List(
+							ctx,
+							&runnerSets,
+							client.InNamespace(ns.Name),
+							client.MatchingLabelsSelector{Selector: selector},
+						)
 						if err != nil {
-							logf.Log.Error(err, "list runner sets")
+							return 0, err
+						}
+						if len(runnerSets.Items) != 1 {
+							return 0, fmt.Errorf("runnerreplicasets is not 1 but %d", len(runnerSets.Items))
 						}
 
-						if len(runnerSets.Items) == 0 {
-							logf.Log.Info("No runnerreplicasets exist yet")
-							return -1
-						}
-
-						return *runnerSets.Items[0].Spec.Replicas
+						return *runnerSets.Items[0].Spec.Replicas, nil
 					},
 					time.Second*5, time.Millisecond*500).Should(BeEquivalentTo(1))
 			}
@@ -134,13 +245,12 @@ var _ = Context("Inside of a new namespace", func() {
 				// We wrap the update in the Eventually block to avoid the below error that occurs due to concurrent modification
 				// made by the controller to update .Status.AvailableReplicas and .Status.ReadyReplicas
 				//   Operation cannot be fulfilled on runnersets.actions.summerwind.dev "example-runnerset": the object has been modified; please apply your changes to the latest version and try again
+				var rd actionsv1alpha1.RunnerDeployment
 				Eventually(func() error {
-					var rd actionsv1alpha1.RunnerDeployment
-
 					err := k8sClient.Get(ctx, types.NamespacedName{Namespace: ns.Name, Name: name}, &rd)
-
-					Expect(err).NotTo(HaveOccurred(), "failed to get test RunnerReplicaSet resource")
-
+					if err != nil {
+						return fmt.Errorf("failed to get test RunnerReplicaSet resource: %v\n", err)
+					}
 					rd.Spec.Replicas = intPtr(2)
 
 					return k8sClient.Update(ctx, &rd)
@@ -150,27 +260,222 @@ var _ = Context("Inside of a new namespace", func() {
 				runnerSets := actionsv1alpha1.RunnerReplicaSetList{Items: []actionsv1alpha1.RunnerReplicaSet{}}
 
 				Eventually(
-					func() int {
-						err := k8sClient.List(ctx, &runnerSets, client.InNamespace(ns.Name))
+					func() (int, error) {
+						selector, err := metav1.LabelSelectorAsSelector(rd.Spec.Selector)
 						if err != nil {
-							logf.Log.Error(err, "list runner sets")
+							return 0, err
+						}
+						err = k8sClient.List(
+							ctx,
+							&runnerSets,
+							client.InNamespace(ns.Name),
+							client.MatchingLabelsSelector{Selector: selector},
+						)
+						if err != nil {
+							return 0, err
+						}
+						if len(runnerSets.Items) != 1 {
+							return 0, fmt.Errorf("runnerreplicasets is not 1 but %d", len(runnerSets.Items))
 						}
 
-						return len(runnerSets.Items)
-					},
-					time.Second*5, time.Millisecond*500).Should(BeEquivalentTo(1))
-
-				Eventually(
-					func() int {
-						err := k8sClient.List(ctx, &runnerSets, client.InNamespace(ns.Name))
-						if err != nil {
-							logf.Log.Error(err, "list runner sets")
-						}
-
-						return *runnerSets.Items[0].Spec.Replicas
+						return *runnerSets.Items[0].Spec.Replicas, nil
 					},
 					time.Second*5, time.Millisecond*500).Should(BeEquivalentTo(2))
 			}
 		})
+
+		It("should create a new RunnerReplicaSet resource from the specified template without labels and selector, add a another RunnerReplicaSet on template modification, and eventually removes old runnerreplicasets", func() {
+			name := "example-runnerdeploy-2"
+
+			{
+				rs := &actionsv1alpha1.RunnerDeployment{
+					ObjectMeta: metav1.ObjectMeta{
+						Name:      name,
+						Namespace: ns.Name,
+					},
+					Spec: actionsv1alpha1.RunnerDeploymentSpec{
+						Replicas: intPtr(1),
+						Template: actionsv1alpha1.RunnerTemplate{
+							Spec: actionsv1alpha1.RunnerSpec{
+								Repository: "test/valid",
+								Image:      "bar",
+								Env: []corev1.EnvVar{
+									{Name: "FOO", Value: "FOOVALUE"},
+								},
+							},
+						},
+					},
+				}
+
+				err := k8sClient.Create(ctx, rs)
+
+				Expect(err).NotTo(HaveOccurred(), "failed to create test RunnerReplicaSet resource")
+
+				runnerSets := actionsv1alpha1.RunnerReplicaSetList{Items: []actionsv1alpha1.RunnerReplicaSet{}}
+
+				Eventually(
+					func() (int, error) {
+						selector, err := metav1.LabelSelectorAsSelector(rs.Spec.Selector)
+						if err != nil {
+							return 0, err
+						}
+						err = k8sClient.List(
+							ctx,
+							&runnerSets,
+							client.InNamespace(ns.Name),
+							client.MatchingLabelsSelector{Selector: selector},
+						)
+						if err != nil {
+							return 0, err
+						}
+						if len(runnerSets.Items) != 1 {
+							return 0, fmt.Errorf("runnerreplicasets is not 1 but %d", len(runnerSets.Items))
+						}
+
+						return *runnerSets.Items[0].Spec.Replicas, nil
+					},
+					time.Second*5, time.Millisecond*500).Should(BeEquivalentTo(1))
+			}
+
+			{
+				// We wrap the update in the Eventually block to avoid the below error that occurs due to concurrent modification
+				// made by the controller to update .Status.AvailableReplicas and .Status.ReadyReplicas
+				//   Operation cannot be fulfilled on runnersets.actions.summerwind.dev "example-runnerset": the object has been modified; please apply your changes to the latest version and try again
+				var rd actionsv1alpha1.RunnerDeployment
+				Eventually(func() error {
+					err := k8sClient.Get(ctx, types.NamespacedName{Namespace: ns.Name, Name: name}, &rd)
+					if err != nil {
+						return fmt.Errorf("failed to get test RunnerReplicaSet resource: %v\n", err)
+					}
+					rd.Spec.Replicas = intPtr(2)
+
+					return k8sClient.Update(ctx, &rd)
+				},
+					time.Second*1, time.Millisecond*500).Should(BeNil())
+
+				runnerSets := actionsv1alpha1.RunnerReplicaSetList{Items: []actionsv1alpha1.RunnerReplicaSet{}}
+
+				Eventually(
+					func() (int, error) {
+						selector, err := metav1.LabelSelectorAsSelector(rd.Spec.Selector)
+						if err != nil {
+							return 0, err
+						}
+						err = k8sClient.List(
+							ctx,
+							&runnerSets,
+							client.InNamespace(ns.Name),
+							client.MatchingLabelsSelector{Selector: selector},
+						)
+						if err != nil {
+							return 0, err
+						}
+						if len(runnerSets.Items) != 1 {
+							return 0, fmt.Errorf("runnerreplicasets is not 1 but %d", len(runnerSets.Items))
+						}
+
+						return *runnerSets.Items[0].Spec.Replicas, nil
+					},
+					time.Second*5, time.Millisecond*500).Should(BeEquivalentTo(2))
+			}
+		})
+
+		It("should adopt RunnerReplicaSet created before 0.18.0 to have Spec.Selector", func() {
+			name := "example-runnerdeploy-2"
+
+			{
+				rd := &actionsv1alpha1.RunnerDeployment{
+					ObjectMeta: metav1.ObjectMeta{
+						Name:      name,
+						Namespace: ns.Name,
+					},
+					Spec: actionsv1alpha1.RunnerDeploymentSpec{
+						Replicas: intPtr(1),
+						Template: actionsv1alpha1.RunnerTemplate{
+							Spec: actionsv1alpha1.RunnerSpec{
+								Repository: "test/valid",
+								Image:      "bar",
+								Env: []corev1.EnvVar{
+									{Name: "FOO", Value: "FOOVALUE"},
+								},
+							},
+						},
+					},
+				}
+
+				createRDErr := k8sClient.Create(ctx, rd)
+				Expect(createRDErr).NotTo(HaveOccurred(), "failed to create test RunnerReplicaSet resource")
+
+				Eventually(
+					func() (int, error) {
+						runnerSets := actionsv1alpha1.RunnerReplicaSetList{Items: []actionsv1alpha1.RunnerReplicaSet{}}
+
+						err := k8sClient.List(
+							ctx,
+							&runnerSets,
+							client.InNamespace(ns.Name),
+						)
+						if err != nil {
+							return 0, err
+						}
+
+						return len(runnerSets.Items), nil
+					},
+					time.Second*1, time.Millisecond*500).Should(BeEquivalentTo(1))
+
+				var rs17 *actionsv1alpha1.RunnerReplicaSet
+
+				Consistently(
+					func() (*metav1.LabelSelector, error) {
+						runnerSets := actionsv1alpha1.RunnerReplicaSetList{Items: []actionsv1alpha1.RunnerReplicaSet{}}
+
+						err := k8sClient.List(
+							ctx,
+							&runnerSets,
+							client.InNamespace(ns.Name),
+						)
+						if err != nil {
+							return nil, err
+						}
+						if len(runnerSets.Items) != 1 {
+							return nil, fmt.Errorf("runnerreplicasets is not 1 but %d", len(runnerSets.Items))
+						}
+
+						rs17 = &runnerSets.Items[0]
+
+						return runnerSets.Items[0].Spec.Selector, nil
+					},
+					time.Second*1, time.Millisecond*500).Should(Not(BeNil()))
+
+				// We simulate the old, pre 0.18.0 RunnerReplicaSet by updating it.
+				// I've tried to use controllerutil.Set{Owner,Controller}Reference and k8sClient.Create(rs17)
+				// but it didn't work due to missing RD UID, where UID is generated on K8s API server on k8sCLient.Create(rd)
+				rs17.Spec.Selector = nil
+
+				updateRSErr := k8sClient.Update(ctx, rs17)
+				Expect(updateRSErr).NotTo(HaveOccurred())
+
+				Eventually(
+					func() (*metav1.LabelSelector, error) {
+						runnerSets := actionsv1alpha1.RunnerReplicaSetList{Items: []actionsv1alpha1.RunnerReplicaSet{}}
+
+						err := k8sClient.List(
+							ctx,
+							&runnerSets,
+							client.InNamespace(ns.Name),
+						)
+						if err != nil {
+							return nil, err
+						}
+						if len(runnerSets.Items) != 1 {
+							return nil, fmt.Errorf("runnerreplicasets is not 1 but %d", len(runnerSets.Items))
+						}
+
+						return runnerSets.Items[0].Spec.Selector, nil
+					},
+					time.Second*1, time.Millisecond*500).Should(Not(BeNil()))
+			}
+		})
+
 	})
 })

controllers/runnerreplicaset_controller.go

@@ -20,9 +20,11 @@ import (
 	"context"
 	"errors"
 	"fmt"
-	gogithub "github.com/google/go-github/v33/github"
+	"reflect"
 	"time"
 
+	gogithub "github.com/google/go-github/v33/github"
+
 	"github.com/go-logr/logr"
 	kerrors "k8s.io/apimachinery/pkg/api/errors"
 	"k8s.io/apimachinery/pkg/runtime"
@@ -44,6 +46,7 @@ type RunnerReplicaSetReconciler struct {
 	Recorder     record.EventRecorder
 	Scheme       *runtime.Scheme
 	GitHubClient *github.Client
+	Name         string
 }
 
 // +kubebuilder:rbac:groups=actions.summerwind.dev,resources=runnerreplicasets,verbs=get;list;watch;create;update;patch;delete
@@ -66,8 +69,18 @@ func (r *RunnerReplicaSetReconciler) Reconcile(req ctrl.Request) (ctrl.Result, e
 		return ctrl.Result{}, nil
 	}
 
+	selector, err := metav1.LabelSelectorAsSelector(rs.Spec.Selector)
+	if err != nil {
+		return ctrl.Result{}, err
+	}
+	// Get the Runners managed by the target RunnerReplicaSet
 	var allRunners v1alpha1.RunnerList
-	if err := r.List(ctx, &allRunners, client.InNamespace(req.Namespace)); err != nil {
+	if err := r.List(
+		ctx,
+		&allRunners,
+		client.InNamespace(req.Namespace),
+		client.MatchingLabelsSelector{Selector: selector},
+	); err != nil {
 		if !kerrors.IsNotFound(err) {
 			return ctrl.Result{}, err
 		}
@@ -75,16 +88,24 @@ func (r *RunnerReplicaSetReconciler) Reconcile(req ctrl.Request) (ctrl.Result, e
 
 	var myRunners []v1alpha1.Runner
 
-	var available, ready int
+	var (
+		current   int
+		ready     int
+		available int
+	)
 
 	for _, r := range allRunners.Items {
-		if metav1.IsControlledBy(&r, &rs) {
+		// This guard is required to avoid the RunnerReplicaSet created by the controller v0.17.0 or before
+		// to not treat all the runners in the namespace as its children.
+		if metav1.IsControlledBy(&r, &rs) && !metav1.HasAnnotation(r.ObjectMeta, annotationKeyRegistrationOnly) {
 			myRunners = append(myRunners, r)
 
-			available += 1
+			current += 1
 
 			if r.Status.Phase == string(corev1.PodRunning) {
 				ready += 1
+				// available is currently the same as ready, as we don't yet have minReadySeconds for runners
+				available += 1
 			}
 		}
 	}
@@ -97,22 +118,92 @@ func (r *RunnerReplicaSetReconciler) Reconcile(req ctrl.Request) (ctrl.Result, e
 		desired = 1
 	}
 
-	log.V(0).Info("debug", "desired", desired, "available", available)
+	registrationOnlyRunnerNsName := req.NamespacedName
+	registrationOnlyRunnerNsName.Name = registrationOnlyRunnerNameFor(rs.Name)
+	registrationOnlyRunner := v1alpha1.Runner{}
+	registrationOnlyRunnerExists := false
+	if err := r.Get(
+		ctx,
+		registrationOnlyRunnerNsName,
+		&registrationOnlyRunner,
+	); err != nil {
+		if !kerrors.IsNotFound(err) {
+			return ctrl.Result{}, err
+		}
+	} else {
+		registrationOnlyRunnerExists = true
+	}
 
-	if available > desired {
-		n := available - desired
+	// On scale to zero, we must have fully registered registration-only runner before we start deleting other runners, hence `desired == 0`
+	// On scale from zero, we must retain the registratoin-only runner until one or more other runners get registered, hence `registrationOnlyRunnerExists && available == 0`.
+	// On RunnerReplicaSet creation, it have always 0 replics and no registration-only runner.
+	// In this case We don't need to bother creating a registration-only runner which gets deleted soon after we have 1 or more available repolicas,
+	// hence it's not `available == 0`, but `registrationOnlyRunnerExists && available == 0`.
+	// See https://github.com/actions-runner-controller/actions-runner-controller/issues/516
+	registrationOnlyRunnerNeeded := desired == 0 || (registrationOnlyRunnerExists && current == 0)
 
-		// get runners that are currently not busy
-		var notBusy []v1alpha1.Runner
-		for _, runner := range myRunners {
+	if registrationOnlyRunnerNeeded {
+		if registrationOnlyRunnerExists {
+			if registrationOnlyRunner.Status.Phase == "" {
+				log.Info("Still waiting for the registration-only runner to be registered")
+
+				return ctrl.Result{}, nil
+			}
+		} else {
+			// A registration-only runner does not exist and is needed, hence create it.
+
+			runnerForScaleFromToZero, err := r.newRunner(rs)
+			if err != nil {
+				return ctrl.Result{}, fmt.Errorf("failed to create runner for scale from/to zero: %v", err)
+			}
+
+			runnerForScaleFromToZero.ObjectMeta.Name = registrationOnlyRunnerNsName.Name
+			runnerForScaleFromToZero.ObjectMeta.GenerateName = ""
+			runnerForScaleFromToZero.ObjectMeta.Labels = nil
+			metav1.SetMetaDataAnnotation(&runnerForScaleFromToZero.ObjectMeta, annotationKeyRegistrationOnly, "true")
+
+			if err := r.Client.Create(ctx, &runnerForScaleFromToZero); err != nil {
+				log.Error(err, "Failed to create runner for scale from/to zero")
+
+				return ctrl.Result{}, err
+			}
+
+			// We can continue to deleting runner pods only after the
+			// registration-only runner gets registered.
+			return ctrl.Result{}, nil
+		}
+	} else {
+		// A registration-only runner exists and is not needed, hence delete it.
+		if registrationOnlyRunnerExists {
+			if err := r.Client.Delete(ctx, &registrationOnlyRunner); err != nil {
+				log.Error(err, "Retrying soon because we failed to delete registration-only runner")
+
+				return ctrl.Result{Requeue: true}, nil
+			}
+		}
+	}
+
+	if current > desired {
+		n := current - desired
+
+		log.V(0).Info(fmt.Sprintf("Deleting %d runners", n), "desired", desired, "current", current, "ready", ready)
+
+		// get runners that are currently offline/not busy/timed-out to register
+		var deletionCandidates []v1alpha1.Runner
+
+		for _, runner := range allRunners.Items {
 			busy, err := r.GitHubClient.IsRunnerBusy(ctx, runner.Spec.Enterprise, runner.Spec.Organization, runner.Spec.Repository, runner.Name)
 			if err != nil {
 				notRegistered := false
+				offline := false
 
-				var e *github.RunnerNotFound
-				if errors.As(err, &e) {
-					log.V(1).Info("Failed to check if runner is busy. Either this runner has never been successfully registered to GitHub or has not managed yet to, and therefore we prioritize it for deletion", "runnerName", runner.Name)
+				var notFoundException *github.RunnerNotFound
+				var offlineException *github.RunnerOffline
+				if errors.As(err, &notFoundException) {
+					log.V(1).Info("Failed to check if runner is busy. Either this runner has never been successfully registered to GitHub or it still needs more time.", "runnerName", runner.Name)
 					notRegistered = true
+				} else if errors.As(err, &offlineException) {
+					offline = true
 				} else {
 					var e *gogithub.RateLimitError
 					if errors.As(err, &e) {
@@ -139,7 +230,7 @@ func (r *RunnerReplicaSetReconciler) Reconcile(req ctrl.Request) (ctrl.Result, e
 				if notRegistered && registrationDidTimeout {
 					log.Info(
 						"Runner failed to register itself to GitHub in timely manner. "+
-							"Recreating the pod to see if it resolves the issue. "+
+							"Marking the runner for scale down. "+
 							"CAUTION: If you see this a lot, you should investigate the root cause. "+
 							"See https://github.com/summerwind/actions-runner-controller/issues/288",
 						"runnerCreationTimestamp", runner.CreationTimestamp,
@@ -147,29 +238,38 @@ func (r *RunnerReplicaSetReconciler) Reconcile(req ctrl.Request) (ctrl.Result, e
 						"configuredRegistrationTimeout", registrationTimeout,
 					)
 
-					notBusy = append(notBusy, runner)
+					deletionCandidates = append(deletionCandidates, runner)
+				}
+
+				// offline runners should always be a great target for scale down
+				if offline {
+					deletionCandidates = append(deletionCandidates, runner)
 				}
 			} else if !busy {
-				notBusy = append(notBusy, runner)
+				deletionCandidates = append(deletionCandidates, runner)
 			}
 		}
 
-		if len(notBusy) < n {
-			n = len(notBusy)
+		if len(deletionCandidates) < n {
+			n = len(deletionCandidates)
 		}
 
+		log.V(0).Info(fmt.Sprintf("Deleting %d runner(s)", n), "desired", desired, "current", current, "ready", ready)
+
 		for i := 0; i < n; i++ {
-			if err := r.Client.Delete(ctx, &notBusy[i]); client.IgnoreNotFound(err) != nil {
+			if err := r.Client.Delete(ctx, &deletionCandidates[i]); client.IgnoreNotFound(err) != nil {
 				log.Error(err, "Failed to delete runner resource")
 
 				return ctrl.Result{}, err
 			}
 
-			r.Recorder.Event(&rs, corev1.EventTypeNormal, "RunnerDeleted", fmt.Sprintf("Deleted runner '%s'", myRunners[i].Name))
-			log.Info("Deleted runner", "runnerreplicaset", rs.ObjectMeta.Name)
+			r.Recorder.Event(&rs, corev1.EventTypeNormal, "RunnerDeleted", fmt.Sprintf("Deleted runner '%s'", deletionCandidates[i].Name))
+			log.Info("Deleted runner")
 		}
-	} else if desired > available {
-		n := desired - available
+	} else if desired > current {
+		n := desired - current
+
+		log.V(0).Info(fmt.Sprintf("Creating %d runner(s)", n), "desired", desired, "available", current, "ready", ready)
 
 		for i := 0; i < n; i++ {
 			newRunner, err := r.newRunner(rs)
@@ -187,14 +287,21 @@ func (r *RunnerReplicaSetReconciler) Reconcile(req ctrl.Request) (ctrl.Result, e
 		}
 	}
 
-	if rs.Status.AvailableReplicas != available || rs.Status.ReadyReplicas != ready {
-		updated := rs.DeepCopy()
-		updated.Status.AvailableReplicas = available
-		updated.Status.ReadyReplicas = ready
+	var status v1alpha1.RunnerReplicaSetStatus
 
-		if err := r.Status().Update(ctx, updated); err != nil {
-			log.Error(err, "Failed to update runner status")
-			return ctrl.Result{}, err
+	status.Replicas = &current
+	status.AvailableReplicas = &available
+	status.ReadyReplicas = &ready
+
+	if !reflect.DeepEqual(rs.Status, status) {
+		updated := rs.DeepCopy()
+		updated.Status = status
+
+		if err := r.Status().Patch(ctx, updated, client.MergeFrom(&rs)); err != nil {
+			log.Info("Failed to update runnerreplicaset status. Retrying immediately", "error", err.Error())
+			return ctrl.Result{
+				Requeue: true,
+			}, nil
 		}
 	}
 
@@ -222,6 +329,10 @@ func (r *RunnerReplicaSetReconciler) newRunner(rs v1alpha1.RunnerReplicaSet) (v1
 
 func (r *RunnerReplicaSetReconciler) SetupWithManager(mgr ctrl.Manager) error {
 	name := "runnerreplicaset-controller"
+	if r.Name != "" {
+		name = r.Name
+	}
+
 	r.Recorder = mgr.GetEventRecorderFor(name)
 
 	return ctrl.NewControllerManagedBy(mgr).
@@ -230,3 +341,7 @@ func (r *RunnerReplicaSetReconciler) SetupWithManager(mgr ctrl.Manager) error {
 		Named(name).
 		Complete(r)
 }
+
+func registrationOnlyRunnerNameFor(rsName string) string {
+	return rsName + "-registration-only"
+}

controllers/runnerreplicaset_controller_test.go

@@ -2,15 +2,14 @@ package controllers
 
 import (
 	"context"
+	"fmt"
 	"math/rand"
 	"net/http/httptest"
 	"time"
 
-	"github.com/google/go-github/v33/github"
 	corev1 "k8s.io/api/core/v1"
 	"k8s.io/apimachinery/pkg/types"
 	"k8s.io/client-go/kubernetes/scheme"
-	"k8s.io/utils/pointer"
 	ctrl "sigs.k8s.io/controller-runtime"
 	logf "sigs.k8s.io/controller-runtime/pkg/log"
 
@@ -47,7 +46,9 @@ func SetupTest(ctx context.Context) *corev1.Namespace {
 		err := k8sClient.Create(ctx, ns)
 		Expect(err).NotTo(HaveOccurred(), "failed to create test namespace")
 
-		mgr, err := ctrl.NewManager(cfg, ctrl.Options{})
+		mgr, err := ctrl.NewManager(cfg, ctrl.Options{
+			Namespace: ns.Name,
+		})
 		Expect(err).NotTo(HaveOccurred(), "failed to create manager")
 
 		runnersList = fake.NewRunnersList()
@@ -60,6 +61,7 @@ func SetupTest(ctx context.Context) *corev1.Namespace {
 			Log:          logf.Log,
 			Recorder:     mgr.GetEventRecorderFor("runnerreplicaset-controller"),
 			GitHubClient: ghClient,
+			Name:         "runnerreplicaset-" + ns.Name,
 		}
 		err = controller.SetupWithManager(mgr)
 		Expect(err).NotTo(HaveOccurred(), "failed to setup controller")
@@ -114,9 +116,19 @@ var _ = Context("Inside of a new namespace", func() {
 					},
 					Spec: actionsv1alpha1.RunnerReplicaSetSpec{
 						Replicas: intPtr(1),
+						Selector: &metav1.LabelSelector{
+							MatchLabels: map[string]string{
+								"foo": "bar",
+							},
+						},
 						Template: actionsv1alpha1.RunnerTemplate{
+							ObjectMeta: metav1.ObjectMeta{
+								Labels: map[string]string{
+									"foo": "bar",
+								},
+							},
 							Spec: actionsv1alpha1.RunnerSpec{
-								Repository: "foo/bar",
+								Repository: "test/valid",
 								Image:      "bar",
 								Env: []corev1.EnvVar{
 									{Name: "FOO", Value: "FOOVALUE"},
@@ -134,20 +146,29 @@ var _ = Context("Inside of a new namespace", func() {
 
 				Eventually(
 					func() int {
-						err := k8sClient.List(ctx, &runners, client.InNamespace(ns.Name))
+						selector, err := metav1.LabelSelectorAsSelector(
+							&metav1.LabelSelector{
+								MatchLabels: map[string]string{
+									"foo": "bar",
+								},
+							},
+						)
+						if err != nil {
+							logf.Log.Error(err, "failed to create labelselector")
+							return -1
+						}
+						err = k8sClient.List(
+							ctx,
+							&runners,
+							client.InNamespace(ns.Name),
+							client.MatchingLabelsSelector{Selector: selector},
+						)
 						if err != nil {
 							logf.Log.Error(err, "list runners")
+							return -1
 						}
 
-						for i, runner := range runners.Items {
-							runnersList.Add(&github.Runner{
-								ID:     pointer.Int64Ptr(int64(i) + 1),
-								Name:   pointer.StringPtr(runner.Name),
-								OS:     pointer.StringPtr("linux"),
-								Status: pointer.StringPtr("online"),
-								Busy:   pointer.BoolPtr(false),
-							})
-						}
+						runnersList.Sync(runners.Items)
 
 						return len(runners.Items)
 					},
@@ -175,20 +196,28 @@ var _ = Context("Inside of a new namespace", func() {
 
 				Eventually(
 					func() int {
-						err := k8sClient.List(ctx, &runners, client.InNamespace(ns.Name))
+						selector, err := metav1.LabelSelectorAsSelector(
+							&metav1.LabelSelector{
+								MatchLabels: map[string]string{
+									"foo": "bar",
+								},
+							},
+						)
+						if err != nil {
+							logf.Log.Error(err, "failed to create labelselector")
+							return -1
+						}
+						err = k8sClient.List(
+							ctx,
+							&runners,
+							client.InNamespace(ns.Name),
+							client.MatchingLabelsSelector{Selector: selector},
+						)
 						if err != nil {
 							logf.Log.Error(err, "list runners")
 						}
 
-						for i, runner := range runners.Items {
-							runnersList.Add(&github.Runner{
-								ID:     pointer.Int64Ptr(int64(i) + 1),
-								Name:   pointer.StringPtr(runner.Name),
-								OS:     pointer.StringPtr("linux"),
-								Status: pointer.StringPtr("online"),
-								Busy:   pointer.BoolPtr(false),
-							})
-						}
+						runnersList.Sync(runners.Items)
 
 						return len(runners.Items)
 					},
@@ -216,21 +245,36 @@ var _ = Context("Inside of a new namespace", func() {
 
 				Eventually(
 					func() int {
-						err := k8sClient.List(ctx, &runners, client.InNamespace(ns.Name))
-						if err != nil {
-							logf.Log.Error(err, "list runners")
+						selector, err := metav1.LabelSelectorAsSelector(&metav1.LabelSelector{
+							MatchLabels: map[string]string{
+								"foo": "bar",
+							},
+						})
+						Expect(err).ToNot(HaveOccurred())
+
+						var regOnly actionsv1alpha1.Runner
+						if err := k8sClient.Get(ctx, types.NamespacedName{Namespace: ns.Name, Name: registrationOnlyRunnerNameFor(name)}, &regOnly); err != nil {
+							logf.Log.Info(fmt.Sprintf("Failed getting registration-only runner in test: %v", err))
+							return -1
+						} else {
+							updated := regOnly.DeepCopy()
+							updated.Status.Phase = "Completed"
+
+							if err := k8sClient.Status().Patch(ctx, updated, client.MergeFrom(&regOnly)); err != nil {
+								logf.Log.Info(fmt.Sprintf("Failed updating registration-only runner in test: %v", err))
+								return -1
+							}
+
+							runnersList.AddOffline([]actionsv1alpha1.Runner{*updated})
 						}
 
-						for i, runner := range runners.Items {
-							runnersList.Add(&github.Runner{
-								ID:     pointer.Int64Ptr(int64(i) + 1),
-								Name:   pointer.StringPtr(runner.Name),
-								OS:     pointer.StringPtr("linux"),
-								Status: pointer.StringPtr("online"),
-								Busy:   pointer.BoolPtr(false),
-							})
+						if err := k8sClient.List(ctx, &runners, client.InNamespace(ns.Name), client.MatchingLabelsSelector{Selector: selector}); err != nil {
+							logf.Log.Error(err, "list runners")
+							return -1
 						}
 
+						runnersList.Sync(runners.Items)
+
 						return len(runners.Items)
 					},
 					time.Second*5, time.Millisecond*500).Should(BeEquivalentTo(0))

controllers/schedule.go

@@ -0,0 +1,122 @@
+package controllers
+
+import (
+	"fmt"
+	"time"
+
+	"github.com/teambition/rrule-go"
+)
+
+type RecurrenceRule struct {
+	Frequency string
+	UntilTime time.Time
+}
+
+type Period struct {
+	StartTime time.Time
+	EndTime   time.Time
+}
+
+func (r *Period) String() string {
+	if r == nil {
+		return ""
+	}
+
+	return r.StartTime.Format(time.RFC3339) + "-" + r.EndTime.Format(time.RFC3339)
+}
+
+func MatchSchedule(now time.Time, startTime, endTime time.Time, recurrenceRule RecurrenceRule) (*Period, *Period, error) {
+	return calculateActiveAndUpcomingRecurringPeriods(
+		now,
+		startTime,
+		endTime,
+		recurrenceRule.Frequency,
+		recurrenceRule.UntilTime,
+	)
+}
+
+func calculateActiveAndUpcomingRecurringPeriods(now, startTime, endTime time.Time, frequency string, untilTime time.Time) (*Period, *Period, error) {
+	var freqValue rrule.Frequency
+
+	var freqDurationDay int
+	var freqDurationMonth int
+	var freqDurationYear int
+
+	switch frequency {
+	case "Daily":
+		freqValue = rrule.DAILY
+		freqDurationDay = 1
+	case "Weekly":
+		freqValue = rrule.WEEKLY
+		freqDurationDay = 7
+	case "Monthly":
+		freqValue = rrule.MONTHLY
+		freqDurationMonth = 1
+	case "Yearly":
+		freqValue = rrule.YEARLY
+		freqDurationYear = 1
+	case "":
+		if now.Before(startTime) {
+			return nil, &Period{StartTime: startTime, EndTime: endTime}, nil
+		}
+
+		if now.Before(endTime) {
+			return &Period{StartTime: startTime, EndTime: endTime}, nil, nil
+		}
+
+		return nil, nil, nil
+	default:
+		return nil, nil, fmt.Errorf(`invalid freq %q: It must be one of "Daily", "Weekly", "Monthly", and "Yearly"`, frequency)
+	}
+
+	freqDurationLater := time.Date(
+		now.Year()+freqDurationYear,
+		time.Month(int(now.Month())+freqDurationMonth),
+		now.Day()+freqDurationDay,
+		now.Hour(), now.Minute(), now.Second(), now.Nanosecond(), now.Location(),
+	)
+
+	freqDuration := freqDurationLater.Sub(now)
+
+	overrideDuration := endTime.Sub(startTime)
+	if overrideDuration > freqDuration {
+		return nil, nil, fmt.Errorf("override's duration %s must be equal to sor shorter than the duration implied by freq %q (%s)", overrideDuration, frequency, freqDuration)
+	}
+
+	rrule, err := rrule.NewRRule(rrule.ROption{
+		Freq:    freqValue,
+		Dtstart: startTime,
+		Until:   untilTime,
+	})
+	if err != nil {
+		return nil, nil, err
+	}
+
+	overrideDurationBefore := now.Add(-overrideDuration + 1)
+	activeOverrideStarts := rrule.Between(overrideDurationBefore, now, true)
+
+	var active *Period
+
+	if len(activeOverrideStarts) > 1 {
+		return nil, nil, fmt.Errorf("[bug] unexpted number of active overrides found: %v", activeOverrideStarts)
+	} else if len(activeOverrideStarts) == 1 {
+		active = &Period{
+			StartTime: activeOverrideStarts[0],
+			EndTime:   activeOverrideStarts[0].Add(overrideDuration),
+		}
+	}
+
+	oneSecondLater := now.Add(1)
+	upcomingOverrideStarts := rrule.Between(oneSecondLater, freqDurationLater, true)
+
+	var next *Period
+
+	if len(upcomingOverrideStarts) > 0 {
+		next = &Period{
+			StartTime: upcomingOverrideStarts[0],
+			EndTime:   upcomingOverrideStarts[0].Add(overrideDuration),
+		}
+	}
+
+	return active, next, nil
+}

controllers/schedule_test.go

@@ -0,0 +1,607 @@
+package controllers
+
+import (
+	"testing"
+	"time"
+)
+
+func TestCalculateActiveAndUpcomingRecurringPeriods(t *testing.T) {
+	type recurrence struct {
+		Start string
+		End   string
+		Freq  string
+		Until string
+	}
+
+	type testcase struct {
+		now string
+
+		recurrence recurrence
+
+		wantActive   string
+		wantUpcoming string
+	}
+
+	check := func(t *testing.T, tc testcase) {
+		t.Helper()
+
+		_, err := time.Parse(time.RFC3339, "2021-05-08T00:00:00Z")
+		if err != nil {
+			t.Fatal(err)
+		}
+
+		now, err := time.Parse(time.RFC3339, tc.now)
+		if err != nil {
+			t.Fatal(err)
+		}
+
+		active, upcoming, err := parseAndMatchRecurringPeriod(now, tc.recurrence.Start, tc.recurrence.End, tc.recurrence.Freq, tc.recurrence.Until)
+		if err != nil {
+			t.Fatal(err)
+		}
+
+		if active.String() != tc.wantActive {
+			t.Errorf("unexpected active: want %q, got %q", tc.wantActive, active)
+		}
+
+		if upcoming.String() != tc.wantUpcoming {
+			t.Errorf("unexpected upcoming: want %q, got %q", tc.wantUpcoming, upcoming)
+		}
+	}
+
+	t.Run("onetime override about to start", func(t *testing.T) {
+		t.Helper()
+
+		check(t, testcase{
+			recurrence: recurrence{
+				Start: "2021-05-01T00:00:00+09:00",
+				End:   "2021-05-03T00:00:00+09:00",
+			},
+
+			now: "2021-04-30T23:59:59+09:00",
+
+			wantActive:   "",
+			wantUpcoming: "2021-05-01T00:00:00+09:00-2021-05-03T00:00:00+09:00",
+		})
+	})
+
+	t.Run("onetime override started", func(t *testing.T) {
+		t.Helper()
+
+		check(t, testcase{
+			recurrence: recurrence{
+				Start: "2021-05-01T00:00:00+09:00",
+				End:   "2021-05-03T00:00:00+09:00",
+			},
+
+			now: "2021-05-01T00:00:00+09:00",
+
+			wantActive:   "2021-05-01T00:00:00+09:00-2021-05-03T00:00:00+09:00",
+			wantUpcoming: "",
+		})
+	})
+
+	t.Run("onetime override about to end", func(t *testing.T) {
+		t.Helper()
+
+		check(t, testcase{
+			recurrence: recurrence{
+				Start: "2021-05-01T00:00:00+09:00",
+				End:   "2021-05-03T00:00:00+09:00",
+			},
+
+			now: "2021-05-02T23:59:59+09:00",
+
+			wantActive:   "2021-05-01T00:00:00+09:00-2021-05-03T00:00:00+09:00",
+			wantUpcoming: "",
+		})
+	})
+
+	t.Run("onetime override ended", func(t *testing.T) {
+		t.Helper()
+
+		check(t, testcase{
+			recurrence: recurrence{
+				Start: "2021-05-01T00:00:00+09:00",
+				End:   "2021-05-03T00:00:00+09:00",
+			},
+
+			now: "2021-05-03T00:00:00+09:00",
+
+			wantActive:   "",
+			wantUpcoming: "",
+		})
+	})
+
+	t.Run("weekly override about to start", func(t *testing.T) {
+		t.Helper()
+
+		check(t, testcase{
+			recurrence: recurrence{
+				Start: "2021-05-01T00:00:00+09:00",
+				End:   "2021-05-03T00:00:00+09:00",
+				Freq:  "Weekly",
+				Until: "2022-05-01T00:00:00+09:00",
+			},
+
+			now: "2021-04-30T23:59:59+09:00",
+
+			wantActive:   "",
+			wantUpcoming: "2021-05-01T00:00:00+09:00-2021-05-03T00:00:00+09:00",
+		})
+	})
+
+	t.Run("weekly override started", func(t *testing.T) {
+		t.Helper()
+
+		check(t, testcase{
+			recurrence: recurrence{
+				Start: "2021-05-01T00:00:00+09:00",
+				End:   "2021-05-03T00:00:00+09:00",
+				Freq:  "Weekly",
+				Until: "2022-05-01T00:00:00+09:00",
+			},
+
+			now: "2021-05-01T00:00:00+09:00",
+
+			wantActive:   "2021-05-01T00:00:00+09:00-2021-05-03T00:00:00+09:00",
+			wantUpcoming: "2021-05-08T00:00:00+09:00-2021-05-10T00:00:00+09:00",
+		})
+	})
+
+	t.Run("weekly override about to end", func(t *testing.T) {
+		t.Helper()
+
+		check(t, testcase{
+			recurrence: recurrence{
+				Start: "2021-05-01T00:00:00+09:00",
+				End:   "2021-05-03T00:00:00+09:00",
+				Freq:  "Weekly",
+				Until: "2022-05-01T00:00:00+09:00",
+			},
+
+			now: "2021-05-02T23:59:59+09:00",
+
+			wantActive:   "2021-05-01T00:00:00+09:00-2021-05-03T00:00:00+09:00",
+			wantUpcoming: "2021-05-08T00:00:00+09:00-2021-05-10T00:00:00+09:00",
+		})
+	})
+
+	t.Run("weekly override ended", func(t *testing.T) {
+		t.Helper()
+
+		check(t, testcase{
+			recurrence: recurrence{
+				Start: "2021-05-01T00:00:00+09:00",
+				End:   "2021-05-03T00:00:00+09:00",
+				Freq:  "Weekly",
+				Until: "2022-05-01T00:00:00+09:00",
+			},
+
+			now: "2021-05-03T00:00:00+09:00",
+
+			wantActive:   "",
+			wantUpcoming: "2021-05-08T00:00:00+09:00-2021-05-10T00:00:00+09:00",
+		})
+	})
+
+	t.Run("weekly override reccurrence about to start", func(t *testing.T) {
+		t.Helper()
+
+		check(t, testcase{
+			recurrence: recurrence{
+				Start: "2021-05-01T00:00:00+09:00",
+				End:   "2021-05-03T00:00:00+09:00",
+				Freq:  "Weekly",
+				Until: "2022-05-01T00:00:00+09:00",
+			},
+
+			now: "2021-05-07T23:59:59+09:00",
+
+			wantActive:   "",
+			wantUpcoming: "2021-05-08T00:00:00+09:00-2021-05-10T00:00:00+09:00",
+		})
+	})
+
+	t.Run("weekly override reccurrence started", func(t *testing.T) {
+		t.Helper()
+
+		check(t, testcase{
+			recurrence: recurrence{
+				Start: "2021-05-01T00:00:00+09:00",
+				End:   "2021-05-03T00:00:00+09:00",
+				Freq:  "Weekly",
+				Until: "2022-05-01T00:00:00+09:00",
+			},
+
+			now: "2021-05-08T00:00:00+09:00",
+
+			wantActive:   "2021-05-08T00:00:00+09:00-2021-05-10T00:00:00+09:00",
+			wantUpcoming: "2021-05-15T00:00:00+09:00-2021-05-17T00:00:00+09:00",
+		})
+	})
+
+	t.Run("weekly override reccurrence about to end", func(t *testing.T) {
+		t.Helper()
+
+		check(t, testcase{
+			recurrence: recurrence{
+				Start: "2021-05-01T00:00:00+09:00",
+				End:   "2021-05-03T00:00:00+09:00",
+				Freq:  "Weekly",
+				Until: "2022-05-01T00:00:00+09:00",
+			},
+
+			now: "2021-05-09T23:59:59+09:00",
+
+			wantActive:   "2021-05-08T00:00:00+09:00-2021-05-10T00:00:00+09:00",
+			wantUpcoming: "2021-05-15T00:00:00+09:00-2021-05-17T00:00:00+09:00",
+		})
+	})
+
+	t.Run("weekly override reccurrence ended", func(t *testing.T) {
+		t.Helper()
+
+		check(t, testcase{
+			recurrence: recurrence{
+				Start: "2021-05-01T00:00:00+09:00",
+				End:   "2021-05-03T00:00:00+09:00",
+				Freq:  "Weekly",
+				Until: "2022-05-01T00:00:00+09:00",
+			},
+
+			now: "2021-05-10T00:00:00+09:00",
+
+			wantActive:   "",
+			wantUpcoming: "2021-05-15T00:00:00+09:00-2021-05-17T00:00:00+09:00",
+		})
+	})
+
+	t.Run("weekly override's last reccurrence about to start", func(t *testing.T) {
+		t.Helper()
+
+		check(t, testcase{
+			recurrence: recurrence{
+				Start: "2021-05-01T00:00:00+09:00",
+				End:   "2021-05-03T00:00:00+09:00",
+				Freq:  "Weekly",
+				Until: "2022-05-01T00:00:00+09:00",
+			},
+
+			now: "2022-04-29T23:59:59+09:00",
+
+			wantActive:   "",
+			wantUpcoming: "2022-04-30T00:00:00+09:00-2022-05-02T00:00:00+09:00",
+		})
+	})
+
+	t.Run("weekly override reccurrence started", func(t *testing.T) {
+		t.Helper()
+
+		check(t, testcase{
+			recurrence: recurrence{
+				Start: "2021-05-01T00:00:00+09:00",
+				End:   "2021-05-03T00:00:00+09:00",
+				Freq:  "Weekly",
+				Until: "2022-05-01T00:00:00+09:00",
+			},
+
+			now: "2022-04-30T00:00:00+09:00",
+
+			wantActive:   "2022-04-30T00:00:00+09:00-2022-05-02T00:00:00+09:00",
+			wantUpcoming: "",
+		})
+	})
+
+	t.Run("weekly override reccurrence about to end", func(t *testing.T) {
+		t.Helper()
+
+		check(t, testcase{
+			recurrence: recurrence{
+				Start: "2021-05-01T00:00:00+09:00",
+				End:   "2021-05-03T00:00:00+09:00",
+				Freq:  "Weekly",
+				Until: "2022-05-01T00:00:00+09:00",
+			},
+
+			now: "2022-05-01T23:59:59+09:00",
+
+			wantActive:   "2022-04-30T00:00:00+09:00-2022-05-02T00:00:00+09:00",
+			wantUpcoming: "",
+		})
+	})
+
+	t.Run("weekly override reccurrence ended", func(t *testing.T) {
+		t.Helper()
+
+		check(t, testcase{
+			recurrence: recurrence{
+				Start: "2021-05-01T00:00:00+09:00",
+				End:   "2021-05-03T00:00:00+09:00",
+				Freq:  "Weekly",
+				Until: "2022-05-01T00:00:00+09:00",
+			},
+
+			now: "2022-05-02T00:00:00+09:00",
+
+			wantActive:   "",
+			wantUpcoming: "",
+		})
+	})
+
+	t.Run("weekly override repeated forever started", func(t *testing.T) {
+		t.Helper()
+
+		check(t, testcase{
+			recurrence: recurrence{
+				Start: "2021-05-01T00:00:00+09:00",
+				End:   "2021-05-03T00:00:00+09:00",
+				Freq:  "Weekly",
+			},
+
+			now: "2021-05-08T00:00:00+09:00",
+
+			wantActive:   "2021-05-08T00:00:00+09:00-2021-05-10T00:00:00+09:00",
+			wantUpcoming: "2021-05-15T00:00:00+09:00-2021-05-17T00:00:00+09:00",
+		})
+	})
+
+	t.Run("monthly override started", func(t *testing.T) {
+		t.Helper()
+
+		check(t, testcase{
+			recurrence: recurrence{
+				Start: "2021-05-01T00:00:00+09:00",
+				End:   "2021-05-03T00:00:00+09:00",
+				Freq:  "Monthly",
+				Until: "2022-05-01T00:00:00+09:00",
+			},
+
+			now: "2021-05-01T00:00:00+09:00",
+
+			wantActive:   "2021-05-01T00:00:00+09:00-2021-05-03T00:00:00+09:00",
+			wantUpcoming: "2021-06-01T00:00:00+09:00-2021-06-03T00:00:00+09:00",
+		})
+	})
+
+	t.Run("monthly override recurrence started", func(t *testing.T) {
+		t.Helper()
+
+		check(t, testcase{
+			recurrence: recurrence{
+				Start: "2021-05-01T00:00:00+09:00",
+				End:   "2021-05-03T00:00:00+09:00",
+				Freq:  "Monthly",
+				Until: "2022-05-01T00:00:00+09:00",
+			},
+
+			now: "2021-06-01T00:00:00+09:00",
+
+			wantActive:   "2021-06-01T00:00:00+09:00-2021-06-03T00:00:00+09:00",
+			wantUpcoming: "2021-07-01T00:00:00+09:00-2021-07-03T00:00:00+09:00",
+		})
+	})
+
+	t.Run("monthly override's last reccurence about to start", func(t *testing.T) {
+		t.Helper()
+
+		check(t, testcase{
+			recurrence: recurrence{
+				Start: "2021-05-01T00:00:00+09:00",
+				End:   "2021-05-03T00:00:00+09:00",
+				Freq:  "Monthly",
+				Until: "2022-05-01T00:00:00+09:00",
+			},
+
+			now: "2022-04-30T23:59:59+09:00",
+
+			wantActive:   "",
+			wantUpcoming: "2022-05-01T00:00:00+09:00-2022-05-03T00:00:00+09:00",
+		})
+	})
+
+	t.Run("monthly override's last reccurence started", func(t *testing.T) {
+		t.Helper()
+
+		check(t, testcase{
+			recurrence: recurrence{
+				Start: "2021-05-01T00:00:00+09:00",
+				End:   "2021-05-03T00:00:00+09:00",
+				Freq:  "Monthly",
+				Until: "2022-05-01T00:00:00+09:00",
+			},
+
+			now: "2022-05-01T00:00:00+09:00",
+
+			wantActive:   "2022-05-01T00:00:00+09:00-2022-05-03T00:00:00+09:00",
+			wantUpcoming: "",
+		})
+	})
+
+	t.Run("monthly override's last reccurence started", func(t *testing.T) {
+		t.Helper()
+
+		check(t, testcase{
+			recurrence: recurrence{
+				Start: "2021-05-01T00:00:00+09:00",
+				End:   "2021-05-03T00:00:00+09:00",
+				Freq:  "Monthly",
+				Until: "2022-05-01T00:00:00+09:00",
+			},
+
+			now: "2022-05-01T00:00:01+09:00",
+
+			wantActive:   "2022-05-01T00:00:00+09:00-2022-05-03T00:00:00+09:00",
+			wantUpcoming: "",
+		})
+	})
+
+	t.Run("monthly override's last reccurence ending", func(t *testing.T) {
+		t.Helper()
+
+		check(t, testcase{
+			recurrence: recurrence{
+				Start: "2021-05-01T00:00:00+09:00",
+				End:   "2021-05-03T00:00:00+09:00",
+				Freq:  "Monthly",
+				Until: "2022-05-01T00:00:00+09:00",
+			},
+
+			now: "2022-05-02T23:59:59+09:00",
+
+			wantActive:   "2022-05-01T00:00:00+09:00-2022-05-03T00:00:00+09:00",
+			wantUpcoming: "",
+		})
+	})
+
+	t.Run("monthly override's last reccurence ended", func(t *testing.T) {
+		t.Helper()
+
+		check(t, testcase{
+			recurrence: recurrence{
+				Start: "2021-05-01T00:00:00+09:00",
+				End:   "2021-05-03T00:00:00+09:00",
+				Freq:  "Monthly",
+				Until: "2022-05-01T00:00:00+09:00",
+			},
+
+			now: "2022-05-03T00:00:00+09:00",
+
+			wantActive:   "",
+			wantUpcoming: "",
+		})
+	})
+
+	t.Run("yearly override started", func(t *testing.T) {
+		t.Helper()
+
+		check(t, testcase{
+			recurrence: recurrence{
+				Start: "2021-05-01T00:00:00+09:00",
+				End:   "2021-05-03T00:00:00+09:00",
+				Freq:  "Yearly",
+				Until: "2022-05-01T00:00:00+09:00",
+			},
+
+			now: "2021-05-01T00:00:00+09:00",
+
+			wantActive:   "2021-05-01T00:00:00+09:00-2021-05-03T00:00:00+09:00",
+			wantUpcoming: "2022-05-01T00:00:00+09:00-2022-05-03T00:00:00+09:00",
+		})
+	})
+
+	t.Run("yearly override reccurrence started", func(t *testing.T) {
+		t.Helper()
+
+		check(t, testcase{
+			recurrence: recurrence{
+				Start: "2021-05-01T00:00:00+09:00",
+				End:   "2021-05-03T00:00:00+09:00",
+				Freq:  "Yearly",
+				Until: "2023-05-01T00:00:00+09:00",
+			},
+
+			now: "2022-05-01T00:00:00+09:00",
+
+			wantActive:   "2022-05-01T00:00:00+09:00-2022-05-03T00:00:00+09:00",
+			wantUpcoming: "2023-05-01T00:00:00+09:00-2023-05-03T00:00:00+09:00",
+		})
+	})
+
+	t.Run("yearly override's last recurrence about to start", func(t *testing.T) {
+		t.Helper()
+
+		check(t, testcase{
+			recurrence: recurrence{
+				Start: "2021-05-01T00:00:00+09:00",
+				End:   "2021-05-03T00:00:00+09:00",
+				Freq:  "Yearly",
+				Until: "2023-05-01T00:00:00+09:00",
+			},
+
+			now: "2023-04-30T23:59:59+09:00",
+
+			wantActive:   "",
+			wantUpcoming: "2023-05-01T00:00:00+09:00-2023-05-03T00:00:00+09:00",
+		})
+	})
+
+	t.Run("yearly override's last recurrence started", func(t *testing.T) {
+		t.Helper()
+
+		check(t, testcase{
+			recurrence: recurrence{
+				Start: "2021-05-01T00:00:00+09:00",
+				End:   "2021-05-03T00:00:00+09:00",
+				Freq:  "Yearly",
+				Until: "2023-05-01T00:00:00+09:00",
+			},
+
+			now: "2023-05-01T00:00:00+09:00",
+
+			wantActive:   "2023-05-01T00:00:00+09:00-2023-05-03T00:00:00+09:00",
+			wantUpcoming: "",
+		})
+	})
+
+	t.Run("yearly override's last recurrence ending", func(t *testing.T) {
+		t.Helper()
+
+		check(t, testcase{
+			recurrence: recurrence{
+				Start: "2021-05-01T00:00:00+09:00",
+				End:   "2021-05-03T00:00:00+09:00",
+				Freq:  "Yearly",
+				Until: "2023-05-01T00:00:00+09:00",
+			},
+
+			now: "2023-05-02T23:23:59+09:00",
+
+			wantActive:   "2023-05-01T00:00:00+09:00-2023-05-03T00:00:00+09:00",
+			wantUpcoming: "",
+		})
+	})
+
+	t.Run("yearly override's last recurrence ended", func(t *testing.T) {
+		t.Helper()
+
+		check(t, testcase{
+			recurrence: recurrence{
+				Start: "2021-05-01T00:00:00+09:00",
+				End:   "2021-05-03T00:00:00+09:00",
+				Freq:  "Yearly",
+				Until: "2023-05-01T00:00:00+09:00",
+			},
+
+			now: "2023-05-03T00:00:00+09:00",
+
+			wantActive:   "",
+			wantUpcoming: "",
+		})
+	})
+}
+
+func parseAndMatchRecurringPeriod(now time.Time, start, end, frequency, until string) (*Period, *Period, error) {
+	startTime, err := time.Parse(time.RFC3339, start)
+	if err != nil {
+		return nil, nil, err
+	}
+
+	endTime, err := time.Parse(time.RFC3339, end)
+	if err != nil {
+		return nil, nil, err
+	}
+
+	var untilTime time.Time
+
+	if until != "" {
+		ut, err := time.Parse(time.RFC3339, until)
+		if err != nil {
+			return nil, nil, err
+		}
+
+		untilTime = ut
+	}
+
+	return MatchSchedule(now, startTime, endTime, RecurrenceRule{Frequency: frequency, UntilTime: untilTime})
+}

controllers/suite_test.go

@@ -55,9 +55,17 @@ func TestAPIs(t *testing.T) {
 var _ = BeforeSuite(func(done Done) {
 	logf.SetLogger(zap.LoggerTo(GinkgoWriter, true))
 
+	var apiServerFlags []string
+
+	apiServerFlags = append(apiServerFlags, envtest.DefaultKubeAPIServerFlags...)
+	// Avoids the following error:
+	// 2021-03-19T15:14:11.673+0900    ERROR   controller-runtime.controller   Reconciler error      {"controller": "testns-tvjzjrunner", "request": "testns-gdnyx/example-runnerdeploy-zps4z-j5562", "error": "Pod \"example-runnerdeploy-zps4z-j5562\" is invalid: [spec.containers[1].image: Required value, spec.containers[1].securityContext.privileged: Forbidden: disallowed by cluster policy]"}
+	apiServerFlags = append(apiServerFlags, "--allow-privileged=true")
+
 	By("bootstrapping test environment")
 	testEnv = &envtest.Environment{
-		CRDDirectoryPaths: []string{filepath.Join("..", "config", "crd", "bases")},
+		CRDDirectoryPaths:  []string{filepath.Join("..", "config", "crd", "bases")},
+		KubeAPIServerFlags: apiServerFlags,
 	}
 
 	var err error

controllers/testdata/org_webhook_check_run_payload.json

@@ -0,0 +1,373 @@
+{
+  "action": "created",
+  "check_run": {
+    "id": 1234567890,
+    "node_id": "ABCDEFGHIJKLMNOPQRSTUVWXYZ",
+    "head_sha": "1234567890123456789012345678901234567890",
+    "external_id": "92058b04-f16a-5035-546c-cae3ad5e2f5f",
+    "url": "https://api.github.com/repos/MYORG/MYREPO/check-runs/123467890",
+    "html_url": "https://github.com/MYORG/MYREPO/runs/123467890",
+    "details_url": "https://github.com/MYORG/MYREPO/runs/123467890",
+    "status": "queued",
+    "conclusion": null,
+    "started_at": "2021-02-18T06:16:31Z",
+    "completed_at": null,
+    "output": {
+      "title": null,
+      "summary": null,
+      "text": null,
+      "annotations_count": 0,
+      "annotations_url": "https://api.github.com/repos/MYORG/MYREPO/check-runs/123467890/annotations"
+    },
+    "name": "validate",
+    "check_suite": {
+      "id": 1234567890,
+      "node_id": "ABCDEFGHIJKLMNOPQRSTUVWXYZ",
+      "head_branch": "MYNAME/actions-runner-controller-webhook",
+      "head_sha": "1234567890123456789012345678901234567890",
+      "status": "queued",
+      "conclusion": null,
+      "url": "https://api.github.com/repos/MYORG/MYREPO/check-suites/1234567890",
+      "before": "1234567890123456789012345678901234567890",
+      "after": "1234567890123456789012345678901234567890",
+      "pull_requests": [
+        {
+          "url": "https://api.github.com/repos/MYORG/MYREPO/pulls/2033",
+          "id": 1234567890,
+          "number": 1234567890,
+          "head": {
+            "ref": "feature",
+            "sha": "1234567890123456789012345678901234567890",
+            "repo": {
+              "id": 1234567890,
+              "url": "https://api.github.com/repos/MYORG/MYREPO",
+              "name": "MYREPO"
+            }
+          },
+          "base": {
+            "ref": "master",
+            "sha": "1234567890123456789012345678901234567890",
+            "repo": {
+              "id": 1234567890,
+              "url": "https://api.github.com/repos/MYORG/MYREPO",
+              "name": "MYREPO"
+            }
+          }
+        }
+      ],
+      "app": {
+        "id": 1234567890,
+        "slug": "github-actions",
+        "node_id": "ABCDEFGHIJKLMNOPQRSTUVWXYZ",
+        "owner": {
+          "login": "github",
+          "id": 1234567890,
+          "node_id": "ABCDEFGHIJKLMNOPQRSTUVWXYZ",
+          "avatar_url": "https://avatars.githubusercontent.com/u/123467890?v=4",
+          "gravatar_id": "",
+          "url": "https://api.github.com/users/github",
+          "html_url": "https://github.com/github",
+          "followers_url": "https://api.github.com/users/github/followers",
+          "following_url": "https://api.github.com/users/github/following{/other_user}",
+          "gists_url": "https://api.github.com/users/github/gists{/gist_id}",
+          "starred_url": "https://api.github.com/users/github/starred{/owner}{/repo}",
+          "subscriptions_url": "https://api.github.com/users/github/subscriptions",
+          "organizations_url": "https://api.github.com/users/github/orgs",
+          "repos_url": "https://api.github.com/users/github/repos",
+          "events_url": "https://api.github.com/users/github/events{/privacy}",
+          "received_events_url": "https://api.github.com/users/github/received_events",
+          "type": "Organization",
+          "site_admin": false
+        },
+        "name": "GitHub Actions",
+        "description": "Automate your workflow from idea to production",
+        "external_url": "https://help.github.com/en/actions",
+        "html_url": "https://github.com/apps/github-actions",
+        "created_at": "2018-07-30T09:30:17Z",
+        "updated_at": "2019-12-10T19:04:12Z",
+        "permissions": {
+          "actions": "write",
+          "checks": "write",
+          "contents": "write",
+          "deployments": "write",
+          "issues": "write",
+          "metadata": "read",
+          "organization_packages": "write",
+          "packages": "write",
+          "pages": "write",
+          "pull_requests": "write",
+          "repository_hooks": "write",
+          "repository_projects": "write",
+          "security_events": "write",
+          "statuses": "write",
+          "vulnerability_alerts": "read"
+        },
+        "events": [
+          "check_run",
+          "check_suite",
+          "create",
+          "delete",
+          "deployment",
+          "deployment_status",
+          "fork",
+          "gollum",
+          "issues",
+          "issue_comment",
+          "label",
+          "milestone",
+          "page_build",
+          "project",
+          "project_card",
+          "project_column",
+          "public",
+          "pull_request",
+          "pull_request_review",
+          "pull_request_review_comment",
+          "push",
+          "registry_package",
+          "release",
+          "repository",
+          "repository_dispatch",
+          "status",
+          "watch",
+          "workflow_dispatch",
+          "workflow_run"
+        ]
+      },
+      "created_at": "2021-02-18T06:15:32Z",
+      "updated_at": "2021-02-18T06:16:31Z"
+    },
+    "app": {
+    "id": 1234567890,
+    "slug": "github-actions",
+    "node_id": "ABCDEFGHIJKLMNOPQRSTUVWXYZ",
+    "owner": {
+      "login": "github",
+      "id": 1234567890,
+      "node_id": "ABCDEFGHIJKLMNOPQRSTUVWXYZ",
+      "avatar_url": "https://avatars.githubusercontent.com/u/1234567890?v=4",
+      "gravatar_id": "",
+      "url": "https://api.github.com/users/github",
+      "html_url": "https://github.com/github",
+      "followers_url": "https://api.github.com/users/github/followers",
+      "following_url": "https://api.github.com/users/github/following{/other_user}",
+      "gists_url": "https://api.github.com/users/github/gists{/gist_id}",
+      "starred_url": "https://api.github.com/users/github/starred{/owner}{/repo}",
+      "subscriptions_url": "https://api.github.com/users/github/subscriptions",
+      "organizations_url": "https://api.github.com/users/github/orgs",
+      "repos_url": "https://api.github.com/users/github/repos",
+      "events_url": "https://api.github.com/users/github/events{/privacy}",
+      "received_events_url": "https://api.github.com/users/github/received_events",
+      "type": "Organization",
+      "site_admin": false
+    },
+    "name": "GitHub Actions",
+    "description": "Automate your workflow from idea to production",
+    "external_url": "https://help.github.com/en/actions",
+    "html_url": "https://github.com/apps/github-actions",
+    "created_at": "2018-07-30T09:30:17Z",
+    "updated_at": "2019-12-10T19:04:12Z",
+    "permissions": {
+      "actions": "write",
+      "checks": "write",
+      "contents": "write",
+      "deployments": "write",
+      "issues": "write",
+      "metadata": "read",
+      "organization_packages": "write",
+      "packages": "write",
+      "pages": "write",
+      "pull_requests": "write",
+      "repository_hooks": "write",
+      "repository_projects": "write",
+      "security_events": "write",
+      "statuses": "write",
+      "vulnerability_alerts": "read"
+    },
+    "events": [
+      "check_run",
+      "check_suite",
+      "create",
+      "delete",
+      "deployment",
+      "deployment_status",
+      "fork",
+      "gollum",
+      "issues",
+      "issue_comment",
+      "label",
+      "milestone",
+      "page_build",
+      "project",
+      "project_card",
+      "project_column",
+      "public",
+      "pull_request",
+      "pull_request_review",
+      "pull_request_review_comment",
+      "push",
+      "registry_package",
+      "release",
+      "repository",
+      "repository_dispatch",
+      "status",
+      "watch",
+      "workflow_dispatch",
+      "workflow_run"
+    ]
+  },
+  "pull_requests": [
+    {
+      "url": "https://api.github.com/repos/MYORG/MYREPO/pulls/1234567890",
+      "id": 1234567890,
+      "number": 1234567890,
+      "head": {
+        "ref": "feature",
+        "sha": "1234567890123456789012345678901234567890",
+        "repo": {
+          "id": 1234567890,
+          "url": "https://api.github.com/repos/MYORG/MYREPO",
+          "name": "MYREPO"
+        }
+      },
+      "base": {
+        "ref": "master",
+        "sha": "1234567890123456789012345678901234567890",
+        "repo": {
+          "id": 1234567890,
+          "url": "https://api.github.com/repos/MYORG/MYREPO",
+          "name": "MYREPO"
+        }
+      }
+    }
+  ]
+},
+"repository": {
+    "id": 1234567890,
+    "node_id": "ABCDEFGHIJKLMNOPQRSTUVWXYZ",
+    "name": "MYREPO",
+    "full_name": "MYORG/MYREPO",
+    "private": true,
+    "owner": {
+      "login": "MYORG",
+      "id": 1234567890,
+      "node_id": "ABCDEFGHIJKLMNOPQRSTUVWXYZ",
+      "avatar_url": "https://avatars.githubusercontent.com/u/1234567890?v=4",
+      "gravatar_id": "",
+      "url": "https://api.github.com/users/MYORG",
+      "html_url": "https://github.com/MYORG",
+      "followers_url": "https://api.github.com/users/MYORG/followers",
+      "following_url": "https://api.github.com/users/MYORG/following{/other_user}",
+      "gists_url": "https://api.github.com/users/MYORG/gists{/gist_id}",
+      "starred_url": "https://api.github.com/users/MYORG/starred{/owner}{/repo}",
+      "subscriptions_url": "https://api.github.com/users/MYORG/subscriptions",
+      "organizations_url": "https://api.github.com/users/MYORG/orgs",
+      "repos_url": "https://api.github.com/users/MYORG/repos",
+      "events_url": "https://api.github.com/users/MYORG/events{/privacy}",
+      "received_events_url": "https://api.github.com/users/MYORG/received_events",
+      "type": "Organization",
+      "site_admin": false
+    },
+    "html_url": "https://github.com/MYORG/MYREPO",
+    "description": "MYREPO",
+    "fork": false,
+    "url": "https://api.github.com/repos/MYORG/MYREPO",
+    "forks_url": "https://api.github.com/repos/MYORG/MYREPO/forks",
+    "keys_url": "https://api.github.com/repos/MYORG/MYREPO/keys{/key_id}",
+    "collaborators_url": "https://api.github.com/repos/MYORG/MYREPO/collaborators{/collaborator}",
+    "teams_url": "https://api.github.com/repos/MYORG/MYREPO/teams",
+    "hooks_url": "https://api.github.com/repos/MYORG/MYREPO/hooks",
+    "issue_events_url": "https://api.github.com/repos/MYORG/MYREPO/issues/events{/number}",
+    "events_url": "https://api.github.com/repos/MYORG/MYREPO/events",
+    "assignees_url": "https://api.github.com/repos/MYORG/MYREPO/assignees{/user}",
+    "branches_url": "https://api.github.com/repos/MYORG/MYREPO/branches{/branch}",
+    "tags_url": "https://api.github.com/repos/MYORG/MYREPO/tags",
+    "blobs_url": "https://api.github.com/repos/MYORG/MYREPO/git/blobs{/sha}",
+    "git_tags_url": "https://api.github.com/repos/MYORG/MYREPO/git/tags{/sha}",
+    "git_refs_url": "https://api.github.com/repos/MYORG/MYREPO/git/refs{/sha}",
+    "trees_url": "https://api.github.com/repos/MYORG/MYREPO/git/trees{/sha}",
+    "statuses_url": "https://api.github.com/repos/MYORG/MYREPO/statuses/{sha}",
+    "languages_url": "https://api.github.com/repos/MYORG/MYREPO/languages",
+    "stargazers_url": "https://api.github.com/repos/MYORG/MYREPO/stargazers",
+    "contributors_url": "https://api.github.com/repos/MYORG/MYREPO/contributors",
+    "subscribers_url": "https://api.github.com/repos/MYORG/MYREPO/subscribers",
+    "subscription_url": "https://api.github.com/repos/MYORG/MYREPO/subscription",
+    "commits_url": "https://api.github.com/repos/MYORG/MYREPO/commits{/sha}",
+    "git_commits_url": "https://api.github.com/repos/MYORG/MYREPO/git/commits{/sha}",
+    "comments_url": "https://api.github.com/repos/MYORG/MYREPO/comments{/number}",
+    "issue_comment_url": "https://api.github.com/repos/MYORG/MYREPO/issues/comments{/number}",
+    "contents_url": "https://api.github.com/repos/MYORG/MYREPO/contents/{+path}",
+    "compare_url": "https://api.github.com/repos/MYORG/MYREPO/compare/{base}...{head}",
+    "merges_url": "https://api.github.com/repos/MYORG/MYREPO/merges",
+    "archive_url": "https://api.github.com/repos/MYORG/MYREPO/{archive_format}{/ref}",
+    "downloads_url": "https://api.github.com/repos/MYORG/MYREPO/downloads",
+    "issues_url": "https://api.github.com/repos/MYORG/MYREPO/issues{/number}",
+    "pulls_url": "https://api.github.com/repos/MYORG/MYREPO/pulls{/number}",
+    "milestones_url": "https://api.github.com/repos/MYORG/MYREPO/milestones{/number}",
+    "notifications_url": "https://api.github.com/repos/MYORG/MYREPO/notifications{?since,all,participating}",
+    "labels_url": "https://api.github.com/repos/MYORG/MYREPO/labels{/name}",
+    "releases_url": "https://api.github.com/repos/MYORG/MYREPO/releases{/id}",
+    "deployments_url": "https://api.github.com/repos/MYORG/MYREPO/deployments",
+    "created_at": "2017-08-10T02:21:10Z",
+    "updated_at": "2021-02-18T04:40:55Z",
+    "pushed_at": "2021-02-18T06:15:30Z",
+    "git_url": "git://github.com/MYORG/MYREPO.git",
+    "ssh_url": "git@github.com:MYORG/MYREPO.git",
+    "clone_url": "https://github.com/MYORG/MYREPO.git",
+    "svn_url": "https://github.com/MYORG/MYREPO",
+    "homepage": null,
+    "size": 30782,
+    "stargazers_count": 2,
+    "watchers_count": 2,
+    "language": "Shell",
+    "has_issues": false,
+    "has_projects": true,
+    "has_downloads": true,
+    "has_wiki": false,
+    "has_pages": false,
+    "forks_count": 0,
+    "mirror_url": null,
+    "archived": false,
+    "disabled": false,
+    "open_issues_count": 6,
+    "license": null,
+    "forks": 0,
+    "open_issues": 6,
+    "watchers": 2,
+    "default_branch": "master"
+  },
+  "organization": {
+    "login": "MYORG",
+    "id": 1234567890,
+    "node_id": "ABCDEFGHIJKLMNOPQRSTUVWXYZ",
+    "url": "https://api.github.com/orgs/MYORG",
+    "repos_url": "https://api.github.com/orgs/MYORG/repos",
+    "events_url": "https://api.github.com/orgs/MYORG/events",
+    "hooks_url": "https://api.github.com/orgs/MYORG/hooks",
+    "issues_url": "https://api.github.com/orgs/MYORG/issues",
+    "members_url": "https://api.github.com/orgs/MYORG/members{/member}",
+    "public_members_url": "https://api.github.com/orgs/MYORG/public_members{/member}",
+    "avatar_url": "https://avatars.githubusercontent.com/u/1234567890?v=4",
+    "description": ""
+  },
+  "sender": {
+    "login": "MYNAME",
+    "id": 1234567890,
+    "node_id": "ABCDEFGHIJKLMNOPQRSTUVWXYZ",
+    "avatar_url": "https://avatars.githubusercontent.com/u/1234567890?v=4",
+    "gravatar_id": "",
+    "url": "https://api.github.com/users/MYNAME",
+    "html_url": "https://github.com/MYNAME",
+    "followers_url": "https://api.github.com/users/MYNAME/followers",
+    "following_url": "https://api.github.com/users/MYNAME/following{/other_user}",
+    "gists_url": "https://api.github.com/users/MYNAME/gists{/gist_id}",
+    "starred_url": "https://api.github.com/users/MYNAME/starred{/owner}{/repo}",
+    "subscriptions_url": "https://api.github.com/users/MYNAME/subscriptions",
+    "organizations_url": "https://api.github.com/users/MYNAME/orgs",
+    "repos_url": "https://api.github.com/users/MYNAME/repos",
+    "events_url": "https://api.github.com/users/MYNAME/events{/privacy}",
+    "received_events_url": "https://api.github.com/users/MYNAME/received_events",
+    "type": "User",
+    "site_admin": false
+  }
+}

controllers/testdata/repo_webhook_check_run_payload.json

@@ -0,0 +1,360 @@
+{
+  "action": "completed",
+  "check_run": {
+    "id": 1949438388,
+    "node_id": "ABCDEFGHIJKLMNOPQRSTUVWXYZ",
+    "head_sha": "1234567890123456789012345678901234567890",
+    "external_id": "ca395085-040a-526b-2ce8-bdc85f692774",
+    "url": "https://api.github.com/repos/MYORG/MYREPO/check-runs/123467890",
+    "html_url": "https://github.com/MYORG/MYREPO/runs/123467890",
+    "details_url": "https://github.com/MYORG/MYREPO/runs/123467890",
+    "status": "queued",
+    "conclusion": null,
+    "started_at": "2021-02-18T06:16:31Z",
+    "completed_at": null,
+    "output": {
+      "title": null,
+      "summary": null,
+      "text": null,
+      "annotations_count": 0,
+      "annotations_url": "https://api.github.com/repos/MYORG/MYREPO/check-runs/123467890/annotations"
+    },
+    "name": "build",
+    "name": "validate",
+    "check_suite": {
+      "id": 1234567890,
+      "node_id": "ABCDEFGHIJKLMNOPQRSTUVWXYZ",
+      "head_branch": "MYNAME/actions-runner-controller-webhook",
+      "head_sha": "1234567890123456789012345678901234567890",
+      "status": "queued",
+      "conclusion": null,
+      "url": "https://api.github.com/repos/MYORG/MYREPO/check-suites/1234567890",
+      "before": "1234567890123456789012345678901234567890",
+      "after": "1234567890123456789012345678901234567890",
+      "pull_requests": [
+        {
+          "url": "https://api.github.com/repos/MYORG/MYREPO/pulls/2033",
+          "id": 1234567890,
+          "number": 1234567890,
+          "head": {
+            "ref": "feature",
+            "sha": "1234567890123456789012345678901234567890",
+            "repo": {
+              "id": 1234567890,
+              "url": "https://api.github.com/repos/MYORG/MYREPO",
+              "name": "MYREPO"
+            }
+          },
+          "base": {
+            "ref": "master",
+            "sha": "1234567890123456789012345678901234567890",
+            "repo": {
+              "id": 1234567890,
+              "url": "https://api.github.com/repos/MYORG/MYREPO",
+              "name": "MYREPO"
+            }
+          }
+        }
+      ],
+      "app": {
+        "id": 1234567890,
+        "slug": "github-actions",
+        "node_id": "ABCDEFGHIJKLMNOPQRSTUVWXYZ",
+        "owner": {
+          "login": "github",
+          "id": 1234567890,
+          "node_id": "ABCDEFGHIJKLMNOPQRSTUVWXYZ",
+          "avatar_url": "https://avatars.githubusercontent.com/u/123467890?v=4",
+          "gravatar_id": "",
+          "url": "https://api.github.com/users/github",
+          "html_url": "https://github.com/github",
+          "followers_url": "https://api.github.com/users/github/followers",
+          "following_url": "https://api.github.com/users/github/following{/other_user}",
+          "gists_url": "https://api.github.com/users/github/gists{/gist_id}",
+          "starred_url": "https://api.github.com/users/github/starred{/owner}{/repo}",
+          "subscriptions_url": "https://api.github.com/users/github/subscriptions",
+          "organizations_url": "https://api.github.com/users/github/orgs",
+          "repos_url": "https://api.github.com/users/github/repos",
+          "events_url": "https://api.github.com/users/github/events{/privacy}",
+          "received_events_url": "https://api.github.com/users/github/received_events",
+          "type": "Organization",
+          "site_admin": false
+        },
+        "name": "GitHub Actions",
+        "description": "Automate your workflow from idea to production",
+        "external_url": "https://help.github.com/en/actions",
+        "html_url": "https://github.com/apps/github-actions",
+        "created_at": "2018-07-30T09:30:17Z",
+        "updated_at": "2019-12-10T19:04:12Z",
+        "permissions": {
+          "actions": "write",
+          "checks": "write",
+          "contents": "write",
+          "deployments": "write",
+          "issues": "write",
+          "metadata": "read",
+          "organization_packages": "write",
+          "packages": "write",
+          "pages": "write",
+          "pull_requests": "write",
+          "repository_hooks": "write",
+          "repository_projects": "write",
+          "security_events": "write",
+          "statuses": "write",
+          "vulnerability_alerts": "read"
+        },
+        "events": [
+          "check_run",
+          "check_suite",
+          "create",
+          "delete",
+          "deployment",
+          "deployment_status",
+          "fork",
+          "gollum",
+          "issues",
+          "issue_comment",
+          "label",
+          "milestone",
+          "page_build",
+          "project",
+          "project_card",
+          "project_column",
+          "public",
+          "pull_request",
+          "pull_request_review",
+          "pull_request_review_comment",
+          "push",
+          "registry_package",
+          "release",
+          "repository",
+          "repository_dispatch",
+          "status",
+          "watch",
+          "workflow_dispatch",
+          "workflow_run"
+        ]
+      },
+      "created_at": "2021-02-18T06:15:32Z",
+      "updated_at": "2021-02-18T06:16:31Z"
+    },
+    "app": {
+      "id": 1234567890,
+      "slug": "github-actions",
+      "node_id": "ABCDEFGHIJKLMNOPQRSTUVWXYZ",
+      "owner": {
+        "login": "github",
+        "id": 1234567890,
+        "node_id": "ABCDEFGHIJKLMNOPQRSTUVWXYZ",
+        "avatar_url": "https://avatars.githubusercontent.com/u/1234567890?v=4",
+        "gravatar_id": "",
+        "url": "https://api.github.com/users/github",
+        "html_url": "https://github.com/github",
+        "followers_url": "https://api.github.com/users/github/followers",
+        "following_url": "https://api.github.com/users/github/following{/other_user}",
+        "gists_url": "https://api.github.com/users/github/gists{/gist_id}",
+        "starred_url": "https://api.github.com/users/github/starred{/owner}{/repo}",
+        "subscriptions_url": "https://api.github.com/users/github/subscriptions",
+        "organizations_url": "https://api.github.com/users/github/orgs",
+        "repos_url": "https://api.github.com/users/github/repos",
+        "events_url": "https://api.github.com/users/github/events{/privacy}",
+        "received_events_url": "https://api.github.com/users/github/received_events",
+        "type": "Organization",
+        "site_admin": false
+      },
+      "name": "GitHub Actions",
+      "description": "Automate your workflow from idea to production",
+      "external_url": "https://help.github.com/en/actions",
+      "html_url": "https://github.com/apps/github-actions",
+      "created_at": "2018-07-30T09:30:17Z",
+      "updated_at": "2019-12-10T19:04:12Z",
+      "permissions": {
+        "actions": "write",
+        "checks": "write",
+        "contents": "write",
+        "deployments": "write",
+        "issues": "write",
+        "metadata": "read",
+        "organization_packages": "write",
+        "packages": "write",
+        "pages": "write",
+        "pull_requests": "write",
+        "repository_hooks": "write",
+        "repository_projects": "write",
+        "security_events": "write",
+        "statuses": "write",
+        "vulnerability_alerts": "read"
+      },
+      "events": [
+        "check_run",
+        "check_suite",
+        "create",
+        "delete",
+        "deployment",
+        "deployment_status",
+        "fork",
+        "gollum",
+        "issues",
+        "issue_comment",
+        "label",
+        "milestone",
+        "page_build",
+        "project",
+        "project_card",
+        "project_column",
+        "public",
+        "pull_request",
+        "pull_request_review",
+        "pull_request_review_comment",
+        "push",
+        "registry_package",
+        "release",
+        "repository",
+        "repository_dispatch",
+        "status",
+        "watch",
+        "workflow_dispatch",
+        "workflow_run"
+      ]
+    },
+    "pull_requests": [
+      {
+        "url": "https://api.github.com/repos/MYORG/MYREPO/pulls/1234567890",
+        "id": 1234567890,
+        "number": 1234567890,
+        "head": {
+          "ref": "feature",
+          "sha": "1234567890123456789012345678901234567890",
+          "repo": {
+            "id": 1234567890,
+            "url": "https://api.github.com/repos/MYORG/MYREPO",
+            "name": "MYREPO"
+          }
+        },
+        "base": {
+          "ref": "master",
+          "sha": "1234567890123456789012345678901234567890",
+          "repo": {
+            "id": 1234567890,
+            "url": "https://api.github.com/repos/MYORG/MYREPO",
+            "name": "MYREPO"
+          }
+        }
+      }
+    ]
+  },
+  "repository": {
+    "id": 1234567890,
+    "node_id": "ABCDEFGHIJKLMNOPQRSTUVWXYZ",
+    "name": "MYREPO",
+    "full_name": "MYORG/MYREPO",
+    "private": true,
+    "owner": {
+      "login": "MYUSER",
+      "id": 1234567890,
+      "node_id": "ABCDEFGHIJKLMNOPQRSTUVWXYZ",
+      "avatar_url": "https://avatars.githubusercontent.com/u/1234567890?v=4",
+      "gravatar_id": "",
+      "url": "https://api.github.com/users/MYUSER",
+      "html_url": "https://github.com/MYUSER",
+      "followers_url": "https://api.github.com/users/MYUSER/followers",
+      "following_url": "https://api.github.com/users/MYUSER/following{/other_user}",
+      "gists_url": "https://api.github.com/users/MYUSER/gists{/gist_id}",
+      "starred_url": "https://api.github.com/users/MYUSER/starred{/owner}{/repo}",
+      "subscriptions_url": "https://api.github.com/users/MYUSER/subscriptions",
+      "organizations_url": "https://api.github.com/users/MYUSER/orgs",
+      "repos_url": "https://api.github.com/users/MYUSER/repos",
+      "events_url": "https://api.github.com/users/MYUSER/events{/privacy}",
+      "received_events_url": "https://api.github.com/users/MYUSER/received_events",
+      "type": "User",
+      "site_admin": false
+    },
+    "html_url": "https://github.com/MYUSER/MYREPO",
+    "description": null,
+    "fork": false,
+    "url": "https://api.github.com/repos/MYUSER/MYREPO",
+    "forks_url": "https://api.github.com/repos/MYUSER/MYREPO/forks",
+    "keys_url": "https://api.github.com/repos/MYUSER/MYREPO/keys{/key_id}",
+    "collaborators_url": "https://api.github.com/repos/MYUSER/MYREPO/collaborators{/collaborator}",
+    "teams_url": "https://api.github.com/repos/MYUSER/MYREPO/teams",
+    "hooks_url": "https://api.github.com/repos/MYUSER/MYREPO/hooks",
+    "issue_events_url": "https://api.github.com/repos/MYUSER/MYREPO/issues/events{/number}",
+    "events_url": "https://api.github.com/repos/MYUSER/MYREPO/events",
+    "assignees_url": "https://api.github.com/repos/MYUSER/MYREPO/assignees{/user}",
+    "branches_url": "https://api.github.com/repos/MYUSER/MYREPO/branches{/branch}",
+    "tags_url": "https://api.github.com/repos/MYUSER/MYREPO/tags",
+    "blobs_url": "https://api.github.com/repos/MYUSER/MYREPO/git/blobs{/sha}",
+    "git_tags_url": "https://api.github.com/repos/MYUSER/MYREPO/git/tags{/sha}",
+    "git_refs_url": "https://api.github.com/repos/MYUSER/MYREPO/git/refs{/sha}",
+    "trees_url": "https://api.github.com/repos/MYUSER/MYREPO/git/trees{/sha}",
+    "statuses_url": "https://api.github.com/repos/MYUSER/MYREPO/statuses/{sha}",
+    "languages_url": "https://api.github.com/repos/MYUSER/MYREPO/languages",
+    "stargazers_url": "https://api.github.com/repos/MYUSER/MYREPO/stargazers",
+    "contributors_url": "https://api.github.com/repos/MYUSER/MYREPO/contributors",
+    "subscribers_url": "https://api.github.com/repos/MYUSER/MYREPO/subscribers",
+    "subscription_url": "https://api.github.com/repos/MYUSER/MYREPO/subscription",
+    "commits_url": "https://api.github.com/repos/MYUSER/MYREPO/commits{/sha}",
+    "git_commits_url": "https://api.github.com/repos/MYUSER/MYREPO/git/commits{/sha}",
+    "comments_url": "https://api.github.com/repos/MYUSER/MYREPO/comments{/number}",
+    "issue_comment_url": "https://api.github.com/repos/MYUSER/MYREPO/issues/comments{/number}",
+    "contents_url": "https://api.github.com/repos/MYUSER/MYREPO/contents/{+path}",
+    "compare_url": "https://api.github.com/repos/MYUSER/MYREPO/compare/{base}...{head}",
+    "merges_url": "https://api.github.com/repos/MYUSER/MYREPO/merges",
+    "archive_url": "https://api.github.com/repos/MYUSER/MYREPO/{archive_format}{/ref}",
+    "downloads_url": "https://api.github.com/repos/MYUSER/MYREPO/downloads",
+    "issues_url": "https://api.github.com/repos/MYUSER/MYREPO/issues{/number}",
+    "pulls_url": "https://api.github.com/repos/MYUSER/MYREPO/pulls{/number}",
+    "milestones_url": "https://api.github.com/repos/MYUSER/MYREPO/milestones{/number}",
+    "notifications_url": "https://api.github.com/repos/MYUSER/MYREPO/notifications{?since,all,participating}",
+    "labels_url": "https://api.github.com/repos/MYUSER/MYREPO/labels{/name}",
+    "releases_url": "https://api.github.com/repos/MYUSER/MYREPO/releases{/id}",
+    "deployments_url": "https://api.github.com/repos/MYUSER/MYREPO/deployments",
+    "created_at": "2021-02-18T06:16:31Z",
+    "updated_at": "2021-02-18T06:16:31Z",
+    "pushed_at": "2021-02-18T06:16:31Z",
+    "git_url": "git://github.com/MYUSER/MYREPO.git",
+    "ssh_url": "git@github.com:MYUSER/MYREPO.git",
+    "clone_url": "https://github.com/MYUSER/MYREPO.git",
+    "svn_url": "https://github.com/MYUSER/MYREPO",
+    "homepage": null,
+    "size": 4,
+    "stargazers_count": 0,
+    "watchers_count": 0,
+    "language": null,
+    "has_issues": true,
+    "has_projects": true,
+    "has_downloads": true,
+    "has_wiki": true,
+    "has_pages": false,
+    "forks_count": 0,
+    "mirror_url": null,
+    "archived": false,
+    "disabled": false,
+    "open_issues_count": 0,
+    "license": null,
+    "forks": 0,
+    "open_issues": 0,
+    "watchers": 0,
+    "default_branch": "main"
+  },
+  "sender": {
+    "login": "MYUSER",
+    "id": 1234567890,
+    "node_id": "ABCDEFGHIJKLMNOPQRSTUVWXYZ",
+    "avatar_url": "https://avatars.githubusercontent.com/u/1234567890?v=4",
+    "gravatar_id": "",
+    "url": "https://api.github.com/users/MYUSER",
+    "html_url": "https://github.com/MYUSER",
+    "followers_url": "https://api.github.com/users/MYUSER/followers",
+    "following_url": "https://api.github.com/users/MYUSER/following{/other_user}",
+    "gists_url": "https://api.github.com/users/MYUSER/gists{/gist_id}",
+    "starred_url": "https://api.github.com/users/MYUSER/starred{/owner}{/repo}",
+    "subscriptions_url": "https://api.github.com/users/MYUSER/subscriptions",
+    "organizations_url": "https://api.github.com/users/MYUSER/orgs",
+    "repos_url": "https://api.github.com/users/MYUSER/repos",
+    "events_url": "https://api.github.com/users/MYUSER/events{/privacy}",
+    "received_events_url": "https://api.github.com/users/MYUSER/received_events",
+    "type": "User",
+    "site_admin": false
+  }
+}

github/fake/runners.go

@@ -6,6 +6,8 @@ import (
 	"net/http/httptest"
 	"strconv"
 
+	"github.com/summerwind/actions-runner-controller/api/v1alpha1"
+
 	"github.com/google/go-github/v33/github"
 	"github.com/gorilla/mux"
 )
@@ -64,6 +66,32 @@ func (r *RunnersList) handleRemove() http.HandlerFunc {
 	}
 }
 
+func (r *RunnersList) Sync(runners []v1alpha1.Runner) {
+	r.runners = nil
+
+	for i, want := range runners {
+		r.Add(&github.Runner{
+			ID:     github.Int64(int64(i)),
+			Name:   github.String(want.Name),
+			OS:     github.String("linux"),
+			Status: github.String("online"),
+			Busy:   github.Bool(false),
+		})
+	}
+}
+
+func (r *RunnersList) AddOffline(runners []v1alpha1.Runner) {
+	for i, want := range runners {
+		r.Add(&github.Runner{
+			ID:     github.Int64(int64(1000 + i)),
+			Name:   github.String(want.Name),
+			OS:     github.String("linux"),
+			Status: github.String("offline"),
+			Busy:   github.Bool(false),
+		})
+	}
+}
+
 func exists(runners []*github.Runner, runner *github.Runner) bool {
 	for _, r := range runners {
 		if *r.Name == *runner.Name {

github/github.go

@@ -5,6 +5,7 @@ import (
 	"fmt"
 	"net/http"
 	"net/url"
+	"os"
 	"strings"
 	"sync"
 	"time"
@@ -39,10 +40,20 @@ func (c *Config) NewClient() (*Client, error) {
 	if len(c.Token) > 0 {
 		transport = oauth2.NewClient(context.Background(), oauth2.StaticTokenSource(&oauth2.Token{AccessToken: c.Token})).Transport
 	} else {
-		tr, err := ghinstallation.NewKeyFromFile(http.DefaultTransport, c.AppID, c.AppInstallationID, c.AppPrivateKey)
-		if err != nil {
-			return nil, fmt.Errorf("authentication failed: %v", err)
+		var tr *ghinstallation.Transport
+
+		if _, err := os.Stat(c.AppPrivateKey); err == nil {
+			tr, err = ghinstallation.NewKeyFromFile(http.DefaultTransport, c.AppID, c.AppInstallationID, c.AppPrivateKey)
+			if err != nil {
+				return nil, fmt.Errorf("authentication failed: using private key at %s: %v", c.AppPrivateKey, err)
+			}
+		} else {
+			tr, err = ghinstallation.New(http.DefaultTransport, c.AppID, c.AppInstallationID, []byte(c.AppPrivateKey))
+			if err != nil {
+				return nil, fmt.Errorf("authentication failed: using private key of size %d (%s...): %v", len(c.AppPrivateKey), strings.Split(c.AppPrivateKey, "\n")[0], err)
+			}
 		}
+
 		if len(c.EnterpriseURL) > 0 {
 			githubAPIURL, err := getEnterpriseApiUrl(c.EnterpriseURL)
 			if err != nil {
@@ -85,7 +96,10 @@ func (c *Client) GetRegistrationToken(ctx context.Context, enterprise, org, repo
 	key := getRegistrationKey(org, repo, enterprise)
 	rt, ok := c.regTokens[key]
 
-	if ok && rt.GetExpiresAt().After(time.Now()) {
+	// we like to give runners a chance that are just starting up and may miss the expiration date by a bit
+	runnerStartupTimeout := 3 * time.Minute
+
+	if ok && rt.GetExpiresAt().After(time.Now().Add(runnerStartupTimeout)) {
 		return rt, nil
 	}
 
@@ -144,7 +158,7 @@ func (c *Client) ListRunners(ctx context.Context, enterprise, org, repo string)
 
 	var runners []*github.Runner
 
-	opts := github.ListOptions{PerPage: 10}
+	opts := github.ListOptions{PerPage: 100}
 	for {
 		list, res, err := c.listRunners(ctx, enterprise, owner, repo, &opts)
 
@@ -208,8 +222,6 @@ func (c *Client) listRunners(ctx context.Context, enterprise, org, repo string,
 }
 
 func (c *Client) ListRepositoryWorkflowRuns(ctx context.Context, user string, repoName string) ([]*github.WorkflowRun, error) {
-	c.Client.Actions.ListRepositoryWorkflowRuns(ctx, user, repoName, nil)
-
 	queued, err := c.listRepositoryWorkflowRuns(ctx, user, repoName, "queued")
 	if err != nil {
 		return nil, fmt.Errorf("listing queued workflow runs: %w", err)
@@ -229,8 +241,6 @@ func (c *Client) ListRepositoryWorkflowRuns(ctx context.Context, user string, re
 }
 
 func (c *Client) listRepositoryWorkflowRuns(ctx context.Context, user string, repoName, status string) ([]*github.WorkflowRun, error) {
-	c.Client.Actions.ListRepositoryWorkflowRuns(ctx, user, repoName, nil)
-
 	var workflowRuns []*github.WorkflowRun
 
 	opts := github.ListWorkflowRunsOptions{
@@ -310,6 +320,14 @@ func (e *RunnerNotFound) Error() string {
 	return fmt.Sprintf("runner %q not found", e.runnerName)
 }
 
+type RunnerOffline struct {
+	runnerName string
+}
+
+func (e *RunnerOffline) Error() string {
+	return fmt.Sprintf("runner %q offline", e.runnerName)
+}
+
 func (r *Client) IsRunnerBusy(ctx context.Context, enterprise, org, repo, name string) (bool, error) {
 	runners, err := r.ListRunners(ctx, enterprise, org, repo)
 	if err != nil {
@@ -318,6 +336,9 @@ func (r *Client) IsRunnerBusy(ctx context.Context, enterprise, org, repo, name s
 
 	for _, runner := range runners {
 		if runner.GetName() == name {
+			if runner.GetStatus() == "offline" {
+				return false, &RunnerOffline{runnerName: name}
+			}
 			return runner.GetBusy(), nil
 		}
 	}

go.mod

@@ -6,6 +6,7 @@ require (
 	github.com/bradleyfalzon/ghinstallation v1.1.1
 	github.com/davecgh/go-spew v1.1.1
 	github.com/go-logr/logr v0.1.0
+	github.com/google/go-cmp v0.3.1
 	github.com/google/go-github/v33 v33.0.1-0.20210204004227-319dcffb518a
 	github.com/gorilla/mux v1.8.0
 	github.com/kelseyhightower/envconfig v1.4.0
@@ -13,10 +14,11 @@ require (
 	github.com/onsi/gomega v1.5.0
 	github.com/prometheus/client_golang v0.9.2
 	github.com/stretchr/testify v1.4.0 // indirect
+	github.com/teambition/rrule-go v1.6.2
+	go.uber.org/zap v1.9.1
 	golang.org/x/oauth2 v0.0.0-20190604053449-0f29369cfe45
 	k8s.io/api v0.0.0-20190918155943-95b840bb6a1f
 	k8s.io/apimachinery v0.0.0-20190913080033-27d36303b655
 	k8s.io/client-go v0.0.0-20190918160344-1fbdaa4c8d90
-	k8s.io/utils v0.0.0-20190801114015-581e00157fb1
 	sigs.k8s.io/controller-runtime v0.4.0
 )

go.sum

@@ -112,7 +112,6 @@ github.com/golang/protobuf v1.3.2 h1:6nsPYzhq5kReh6QImI3k5qWzO4PEbvbIW2cwSfR/6xs
 github.com/golang/protobuf v1.3.2/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
 github.com/google/btree v0.0.0-20180813153112-4030bb1f1f0c/go.mod h1:lNA+9X1NB3Zf8V7Ke586lFgjr2dZNuvo3lPJSGZ5JPQ=
 github.com/google/go-cmp v0.2.0/go.mod h1:oXzfMopK8JAjlY9xF4vHSVASa0yLyX7SntLO5aqRK0M=
-github.com/google/go-cmp v0.3.0 h1:crn/baboCvb5fXaQ0IJ1SGTsTVrWpDsCWC8EGETZijY=
 github.com/google/go-cmp v0.3.0/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU=
 github.com/google/go-cmp v0.3.1 h1:Xye71clBPdm5HgqGwUkwhbynsUJZhDbS20FvLhQ2izg=
 github.com/google/go-cmp v0.3.1/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU=
@@ -216,7 +215,6 @@ github.com/prometheus/procfs v0.0.0-20181204211112-1dc9a6cbc91a h1:9a8MnZMP0X2nL
 github.com/prometheus/procfs v0.0.0-20181204211112-1dc9a6cbc91a/go.mod h1:c3At6R/oaqEKCNdg8wHV1ftS6bRYblBhIjjI8uT2IGk=
 github.com/remyoudompheng/bigfft v0.0.0-20170806203942-52369c62f446/go.mod h1:uYEyJGbgTkfkS4+E/PavXkNJcbFIpEtjt2B0KDQ5+9M=
 github.com/russross/blackfriday v1.5.2/go.mod h1:JO/DiYxRf+HjHt06OyowR9PTA263kcR/rfWxYHBV53g=
-github.com/sirupsen/logrus v1.4.2 h1:SPIRibHv4MatM3XXNO2BJeFLZwZ2LvZgfQ5+UNI2im4=
 github.com/sirupsen/logrus v1.4.2/go.mod h1:tLMulIdttU9McNUspp0xgXVQah82FyeX6MwdIuYE2rE=
 github.com/soheilhy/cmux v0.1.3/go.mod h1:IM3LyeVVIOuxMH7sFAkER9+bJ4dT7Ms6E4xg4kGIyLM=
 github.com/spf13/afero v1.1.2/go.mod h1:j4pytiNVoe2o6bmDsKpLACNPDBIoEAkihy7loJ1B0CQ=
@@ -233,10 +231,11 @@ github.com/stretchr/objx v0.1.1/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+
 github.com/stretchr/objx v0.2.0/go.mod h1:qt09Ya8vawLte6SNmTgCsAVtYtaKzEcn8ATUoHMkEqE=
 github.com/stretchr/testify v0.0.0-20151208002404-e3a8ff8ce365/go.mod h1:a8OnRcib4nhh0OaRAV+Yts87kKdq0PP7pXfy6kDkUVs=
 github.com/stretchr/testify v1.2.2/go.mod h1:a8OnRcib4nhh0OaRAV+Yts87kKdq0PP7pXfy6kDkUVs=
-github.com/stretchr/testify v1.3.0 h1:TivCn/peBQ7UY8ooIcPgZFpTNSz0Q2U6UrFlUfqbe0Q=
 github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI=
 github.com/stretchr/testify v1.4.0 h1:2E4SXV/wtOkTonXsotYi4li6zVWxYlZuYNCXe9XRJyk=
 github.com/stretchr/testify v1.4.0/go.mod h1:j7eGeouHqKxXV5pUuKE4zz7dFj8WfuZ+81PSLYec5m4=
+github.com/teambition/rrule-go v1.6.2 h1:keZiiijltBxYUuhQaySAEGyIFR0UOkAd7i+u6FM5/+I=
+github.com/teambition/rrule-go v1.6.2/go.mod h1:mBJ1Ht5uboJ6jexKdNUJg2NcwP8uUMNvStWXlJD3MvU=
 github.com/tmc/grpc-websocket-proxy v0.0.0-20170815181823-89b8d40f7ca8/go.mod h1:ncp9v5uamzpCO7NfCPTXjqaC+bZgJeR0sMTm6dMHP7U=
 github.com/ugorji/go/codec v0.0.0-20181204163529-d75b2dcb6bc8/go.mod h1:VFNgLljTbGfSG7qAOspJ7OScBnGdDN/yBr0sguwnwf0=
 github.com/xiang90/probing v0.0.0-20160813154853-07dd2e8dfe18/go.mod h1:UETIi67q53MR2AWcXfiuqkDkRtnGDLqkBTpCHuJHxtU=

main.go

@@ -20,12 +20,14 @@ import (
 	"flag"
 	"fmt"
 	"os"
+	"strings"
 	"time"
 
 	"github.com/kelseyhightower/envconfig"
 	actionsv1alpha1 "github.com/summerwind/actions-runner-controller/api/v1alpha1"
 	"github.com/summerwind/actions-runner-controller/controllers"
 	"github.com/summerwind/actions-runner-controller/github"
+	zaplib "go.uber.org/zap"
 	"k8s.io/apimachinery/pkg/runtime"
 	clientgoscheme "k8s.io/client-go/kubernetes/scheme"
 	_ "k8s.io/client-go/plugin/pkg/client/auth/gcp"
@@ -37,11 +39,16 @@ import (
 const (
 	defaultRunnerImage = "summerwind/actions-runner:latest"
 	defaultDockerImage = "docker:dind"
+
+	logLevelDebug = "debug"
+	logLevelInfo  = "info"
+	logLevelWarn  = "warn"
+	logLevelError = "error"
 )
 
 var (
-	scheme   = runtime.NewScheme()
-	setupLog = ctrl.Log.WithName("setup")
+	scheme = runtime.NewScheme()
+	log    = ctrl.Log.WithName("actions-runner-controller")
 )
 
 func init() {
@@ -60,8 +67,14 @@ func main() {
 		enableLeaderElection bool
 		syncPeriod           time.Duration
 
+		gitHubAPICacheDuration time.Duration
+
 		runnerImage string
 		dockerImage string
+		namespace   string
+		logLevel    string
+
+		commonRunnerLabels commaSeparatedStringSlice
 	)
 
 	var c github.Config
@@ -79,11 +92,27 @@ func main() {
 	flag.Int64Var(&c.AppID, "github-app-id", c.AppID, "The application ID of GitHub App.")
 	flag.Int64Var(&c.AppInstallationID, "github-app-installation-id", c.AppInstallationID, "The installation ID of GitHub App.")
 	flag.StringVar(&c.AppPrivateKey, "github-app-private-key", c.AppPrivateKey, "The path of a private key file to authenticate as a GitHub App")
-	flag.DurationVar(&syncPeriod, "sync-period", 10*time.Minute, "Determines the minimum frequency at which K8s resources managed by this controller are reconciled. When you use autoscaling, set to a lower value like 10 minute, because this corresponds to the minimum time to react on demand change")
+	flag.DurationVar(&gitHubAPICacheDuration, "github-api-cache-duration", 0, "The duration until the GitHub API cache expires. Setting this to e.g. 10m results in the controller tries its best not to make the same API call within 10m to reduce the chance of being rate-limited. Defaults to mostly the same value as sync-period. If you're tweaking this in order to make autoscaling more responsive, you'll probably want to tweak sync-period, too")
+	flag.DurationVar(&syncPeriod, "sync-period", 10*time.Minute, "Determines the minimum frequency at which K8s resources managed by this controller are reconciled. When you use autoscaling, set to a lower value like 10 minute, because this corresponds to the minimum time to react on demand change. . If you're tweaking this in order to make autoscaling more responsive, you'll probably want to tweak github-api-cache-duration, too")
+	flag.Var(&commonRunnerLabels, "common-runner-labels", "Runner labels in the K1=V1,K2=V2,... format that are inherited all the runners created by the controller. See https://github.com/summerwind/actions-runner-controller/issues/321 for more information")
+	flag.StringVar(&namespace, "watch-namespace", "", "The namespace to watch for custom resources. Set to empty for letting it watch for all namespaces.")
+	flag.StringVar(&logLevel, "log-level", logLevelDebug, `The verbosity of the logging. Valid values are "debug", "info", "warn", "error". Defaults to "debug".`)
 	flag.Parse()
 
 	logger := zap.New(func(o *zap.Options) {
-		o.Development = true
+		switch logLevel {
+		case logLevelDebug:
+			o.Development = true
+		case logLevelInfo:
+			lvl := zaplib.NewAtomicLevelAt(zaplib.InfoLevel)
+			o.Level = &lvl
+		case logLevelWarn:
+			lvl := zaplib.NewAtomicLevelAt(zaplib.WarnLevel)
+			o.Level = &lvl
+		case logLevelError:
+			lvl := zaplib.NewAtomicLevelAt(zaplib.ErrorLevel)
+			o.Level = &lvl
+		}
 	})
 
 	ghClient, err = c.NewClient()
@@ -100,15 +129,16 @@ func main() {
 		LeaderElection:     enableLeaderElection,
 		Port:               9443,
 		SyncPeriod:         &syncPeriod,
+		Namespace:          namespace,
 	})
 	if err != nil {
-		setupLog.Error(err, "unable to start manager")
+		log.Error(err, "unable to start manager")
 		os.Exit(1)
 	}
 
 	runnerReconciler := &controllers.RunnerReconciler{
 		Client:       mgr.GetClient(),
-		Log:          ctrl.Log.WithName("controllers").WithName("Runner"),
+		Log:          log.WithName("runner"),
 		Scheme:       mgr.GetScheme(),
 		GitHubClient: ghClient,
 		RunnerImage:  runnerImage,
@@ -116,63 +146,99 @@ func main() {
 	}
 
 	if err = runnerReconciler.SetupWithManager(mgr); err != nil {
-		setupLog.Error(err, "unable to create controller", "controller", "Runner")
+		log.Error(err, "unable to create controller", "controller", "Runner")
 		os.Exit(1)
 	}
 
 	runnerSetReconciler := &controllers.RunnerReplicaSetReconciler{
 		Client:       mgr.GetClient(),
-		Log:          ctrl.Log.WithName("controllers").WithName("RunnerReplicaSet"),
+		Log:          log.WithName("runnerreplicaset"),
 		Scheme:       mgr.GetScheme(),
 		GitHubClient: ghClient,
 	}
 
 	if err = runnerSetReconciler.SetupWithManager(mgr); err != nil {
-		setupLog.Error(err, "unable to create controller", "controller", "RunnerReplicaSet")
+		log.Error(err, "unable to create controller", "controller", "RunnerReplicaSet")
 		os.Exit(1)
 	}
 
 	runnerDeploymentReconciler := &controllers.RunnerDeploymentReconciler{
-		Client: mgr.GetClient(),
-		Log:    ctrl.Log.WithName("controllers").WithName("RunnerDeployment"),
-		Scheme: mgr.GetScheme(),
+		Client:             mgr.GetClient(),
+		Log:                log.WithName("runnerdeployment"),
+		Scheme:             mgr.GetScheme(),
+		CommonRunnerLabels: commonRunnerLabels,
 	}
 
 	if err = runnerDeploymentReconciler.SetupWithManager(mgr); err != nil {
-		setupLog.Error(err, "unable to create controller", "controller", "RunnerDeployment")
+		log.Error(err, "unable to create controller", "controller", "RunnerDeployment")
 		os.Exit(1)
 	}
 
+	if gitHubAPICacheDuration == 0 {
+		gitHubAPICacheDuration = syncPeriod - 10*time.Second
+	}
+
+	if gitHubAPICacheDuration < 0 {
+		gitHubAPICacheDuration = 0
+	}
+
+	log.Info(
+		"Initializing actions-runner-controller",
+		"github-api-cahce-duration", gitHubAPICacheDuration,
+		"sync-period", syncPeriod,
+		"runner-image", runnerImage,
+		"docker-image", dockerImage,
+		"common-runnner-labels", commonRunnerLabels,
+		"watch-namespace", namespace,
+	)
+
 	horizontalRunnerAutoscaler := &controllers.HorizontalRunnerAutoscalerReconciler{
 		Client:        mgr.GetClient(),
-		Log:           ctrl.Log.WithName("controllers").WithName("HorizontalRunnerAutoscaler"),
+		Log:           log.WithName("horizontalrunnerautoscaler"),
 		Scheme:        mgr.GetScheme(),
 		GitHubClient:  ghClient,
-		CacheDuration: syncPeriod - 10*time.Second,
+		CacheDuration: gitHubAPICacheDuration,
 	}
 
 	if err = horizontalRunnerAutoscaler.SetupWithManager(mgr); err != nil {
-		setupLog.Error(err, "unable to create controller", "controller", "HorizontalRunnerAutoscaler")
+		log.Error(err, "unable to create controller", "controller", "HorizontalRunnerAutoscaler")
 		os.Exit(1)
 	}
 
 	if err = (&actionsv1alpha1.Runner{}).SetupWebhookWithManager(mgr); err != nil {
-		setupLog.Error(err, "unable to create webhook", "webhook", "Runner")
+		log.Error(err, "unable to create webhook", "webhook", "Runner")
 		os.Exit(1)
 	}
 	if err = (&actionsv1alpha1.RunnerDeployment{}).SetupWebhookWithManager(mgr); err != nil {
-		setupLog.Error(err, "unable to create webhook", "webhook", "RunnerDeployment")
+		log.Error(err, "unable to create webhook", "webhook", "RunnerDeployment")
 		os.Exit(1)
 	}
 	if err = (&actionsv1alpha1.RunnerReplicaSet{}).SetupWebhookWithManager(mgr); err != nil {
-		setupLog.Error(err, "unable to create webhook", "webhook", "RunnerReplicaSet")
+		log.Error(err, "unable to create webhook", "webhook", "RunnerReplicaSet")
 		os.Exit(1)
 	}
 	// +kubebuilder:scaffold:builder
 
-	setupLog.Info("starting manager")
+	log.Info("starting manager")
 	if err := mgr.Start(ctrl.SetupSignalHandler()); err != nil {
-		setupLog.Error(err, "problem running manager")
+		log.Error(err, "problem running manager")
 		os.Exit(1)
 	}
 }
+
+type commaSeparatedStringSlice []string
+
+func (s *commaSeparatedStringSlice) String() string {
+	return fmt.Sprintf("%v", *s)
+}
+
+func (s *commaSeparatedStringSlice) Set(value string) error {
+	for _, v := range strings.Split(value, ",") {
+		if v == "" {
+			continue
+		}
+
+		*s = append(*s, v)
+	}
+	return nil
+}

pkg/actionsglob/README.md

@@ -0,0 +1,8 @@
+This package is an implementation of glob that is intended to simulate the behaviour of 
+https://github.com/actions/toolkit/tree/master/packages/glob in many cases.
+
+This isn't a complete reimplementation of the referenced nodejs package.
+
+Differences:
+
+- This package doesn't implement `**`

pkg/actionsglob/actionsglob.go

@@ -0,0 +1,78 @@
+package actionsglob
+
+import (
+	"fmt"
+	"strings"
+)
+
+func Match(pat string, s string) bool {
+	if len(pat) == 0 {
+		panic(fmt.Sprintf("unexpected length of pattern: %d", len(pat)))
+	}
+
+	var inverse bool
+
+	if pat[0] == '!' {
+		pat = pat[1:]
+		inverse = true
+	}
+
+	tokens := strings.SplitAfter(pat, "*")
+
+	var wildcardInHead bool
+
+	for i := 0; i < len(tokens); i++ {
+		p := tokens[i]
+
+		if p == "" {
+			s = ""
+			break
+		}
+
+		if p == "*" {
+			if i == len(tokens)-1 {
+				s = ""
+				break
+			}
+
+			wildcardInHead = true
+
+			continue
+		}
+
+		wildcardInTail := p[len(p)-1] == '*'
+		if wildcardInTail {
+			p = p[:len(p)-1]
+		}
+
+		subs := strings.SplitN(s, p, 2)
+
+		if len(subs) == 0 {
+			break
+		}
+
+		if subs[0] != "" {
+			if !wildcardInHead {
+				break
+			}
+		}
+
+		if subs[1] != "" {
+			if !wildcardInTail {
+				break
+			}
+		}
+
+		s = subs[1]
+
+		wildcardInHead = wildcardInTail
+	}
+
+	r := s == ""
+
+	if inverse {
+		r = !r
+	}
+
+	return r
+}

pkg/actionsglob/match_test.go

@@ -0,0 +1,214 @@
+package actionsglob
+
+import (
+	"testing"
+)
+
+func TestMatch(t *testing.T) {
+	type testcase struct {
+		Pattern, Target string
+		Want            bool
+	}
+
+	run := func(t *testing.T, tc testcase) {
+		t.Helper()
+
+		got := Match(tc.Pattern, tc.Target)
+
+		if got != tc.Want {
+			t.Errorf("%s against %s: want %v, got %v", tc.Pattern, tc.Target, tc.Want, got)
+		}
+	}
+
+	t.Run("foo == foo", func(t *testing.T) {
+		run(t, testcase{
+			Pattern: "foo",
+			Target:  "foo",
+			Want:    true,
+		})
+	})
+
+	t.Run("!foo == foo", func(t *testing.T) {
+		run(t, testcase{
+			Pattern: "!foo",
+			Target:  "foo",
+			Want:    false,
+		})
+	})
+
+	t.Run("foo == foo1", func(t *testing.T) {
+		run(t, testcase{
+			Pattern: "foo",
+			Target:  "foo1",
+			Want:    false,
+		})
+	})
+
+	t.Run("!foo == foo1", func(t *testing.T) {
+		run(t, testcase{
+			Pattern: "!foo",
+			Target:  "foo1",
+			Want:    true,
+		})
+	})
+
+	t.Run("*foo == foo", func(t *testing.T) {
+		run(t, testcase{
+			Pattern: "*foo",
+			Target:  "foo",
+			Want:    true,
+		})
+	})
+
+	t.Run("!*foo == foo", func(t *testing.T) {
+		run(t, testcase{
+			Pattern: "!*foo",
+			Target:  "foo",
+			Want:    false,
+		})
+	})
+
+	t.Run("*foo == 1foo", func(t *testing.T) {
+		run(t, testcase{
+			Pattern: "*foo",
+			Target:  "1foo",
+			Want:    true,
+		})
+	})
+
+	t.Run("!*foo == 1foo", func(t *testing.T) {
+		run(t, testcase{
+			Pattern: "!*foo",
+			Target:  "1foo",
+			Want:    false,
+		})
+	})
+
+	t.Run("*foo == foo1", func(t *testing.T) {
+		run(t, testcase{
+			Pattern: "*foo",
+			Target:  "foo1",
+			Want:    false,
+		})
+	})
+
+	t.Run("!*foo == foo1", func(t *testing.T) {
+		run(t, testcase{
+			Pattern: "!*foo",
+			Target:  "foo1",
+			Want:    true,
+		})
+	})
+
+	t.Run("*foo* == foo1", func(t *testing.T) {
+		run(t, testcase{
+			Pattern: "*foo*",
+			Target:  "foo1",
+			Want:    true,
+		})
+	})
+
+	t.Run("!*foo* == foo1", func(t *testing.T) {
+		run(t, testcase{
+			Pattern: "!*foo*",
+			Target:  "foo1",
+			Want:    false,
+		})
+	})
+
+	t.Run("*foo == foobar", func(t *testing.T) {
+		run(t, testcase{
+			Pattern: "*foo",
+			Target:  "foobar",
+			Want:    false,
+		})
+	})
+
+	t.Run("!*foo == foobar", func(t *testing.T) {
+		run(t, testcase{
+			Pattern: "!*foo",
+			Target:  "foobar",
+			Want:    true,
+		})
+	})
+
+	t.Run("*foo* == foobar", func(t *testing.T) {
+		run(t, testcase{
+			Pattern: "*foo*",
+			Target:  "foobar",
+			Want:    true,
+		})
+	})
+
+	t.Run("!*foo* == foobar", func(t *testing.T) {
+		run(t, testcase{
+			Pattern: "!*foo*",
+			Target:  "foobar",
+			Want:    false,
+		})
+	})
+
+	t.Run("foo* == foo", func(t *testing.T) {
+		run(t, testcase{
+			Pattern: "foo*",
+			Target:  "foo",
+			Want:    true,
+		})
+	})
+
+	t.Run("!foo* == foo", func(t *testing.T) {
+		run(t, testcase{
+			Pattern: "!foo*",
+			Target:  "foo",
+			Want:    false,
+		})
+	})
+
+	t.Run("foo* == foobar", func(t *testing.T) {
+		run(t, testcase{
+			Pattern: "foo*",
+			Target:  "foobar",
+			Want:    true,
+		})
+	})
+
+	t.Run("!foo* == foobar", func(t *testing.T) {
+		run(t, testcase{
+			Pattern: "!foo*",
+			Target:  "foobar",
+			Want:    false,
+		})
+	})
+
+	t.Run("foo (* == foo ( 1 / 2 )", func(t *testing.T) {
+		run(t, testcase{
+			Pattern: "foo (*",
+			Target:  "foo ( 1 / 2 )",
+			Want:    true,
+		})
+	})
+
+	t.Run("!foo (* == foo ( 1 / 2 )", func(t *testing.T) {
+		run(t, testcase{
+			Pattern: "!foo (*",
+			Target:  "foo ( 1 / 2 )",
+			Want:    false,
+		})
+	})
+
+	t.Run("actions-*-metrics == actions-workflow-metrics", func(t *testing.T) {
+		run(t, testcase{
+			Pattern: "actions-*-metrics",
+			Target:  "actions-workflow-metrics",
+			Want:    true,
+		})
+	})
+
+	t.Run("!actions-*-metrics == actions-workflow-metrics", func(t *testing.T) {
+		run(t, testcase{
+			Pattern: "!actions-*-metrics",
+			Target:  "actions-workflow-metrics",
+			Want:    false,
+		})
+	})
+}

runner/Dockerfile

@@ -1,64 +1,68 @@
-FROM ubuntu:18.04
+FROM ubuntu:20.04
 
 ARG TARGETPLATFORM
-ARG RUNNER_VERSION=2.274.2
+ARG RUNNER_VERSION=2.278.0
 ARG DOCKER_VERSION=19.03.12
 
 RUN test -n "$TARGETPLATFORM" || (echo "TARGETPLATFORM must be set" && false)
 
 ENV DEBIAN_FRONTEND=noninteractive
 RUN apt update -y \
-  && apt install -y software-properties-common \
-  && add-apt-repository -y ppa:git-core/ppa \
-  && apt update -y \
-  && apt install -y --no-install-recommends \
-  build-essential \
-  curl \
-  ca-certificates \
-  dnsutils \
-  ftp \
-  git \
-  iproute2 \
-  iputils-ping \
-  jq \
-  libunwind8 \
-  locales \
-  netcat \
-  openssh-client \
-  parallel \
-  rsync \
-  shellcheck \
-  sudo \
-  telnet \
-  time \
-  tzdata \
-  unzip \
-  upx \
-  wget \
-  zip \
-  zstd \
-  && rm -rf /var/lib/apt/lists/*
+    && apt install -y software-properties-common \
+    && add-apt-repository -y ppa:git-core/ppa \
+    && apt update -y \
+    && apt install -y --no-install-recommends \
+    build-essential \
+    curl \
+    ca-certificates \
+    dnsutils \
+    ftp \
+    git \
+    iproute2 \
+    iputils-ping \
+    jq \
+    libunwind8 \
+    locales \
+    netcat \
+    openssh-client \
+    parallel \
+    python3-pip \
+    rsync \
+    shellcheck \
+    sudo \
+    telnet \
+    time \
+    tzdata \
+    unzip \
+    upx \
+    wget \
+    zip \
+    zstd \
+    && ln -sf /usr/bin/python3 /usr/bin/python \
+    && ln -sf /usr/bin/pip3 /usr/bin/pip \
+    && rm -rf /var/lib/apt/lists/*
 
 RUN export ARCH=$(echo ${TARGETPLATFORM} | cut -d / -f2) \
-  && curl -L -o /usr/local/bin/dumb-init https://github.com/Yelp/dumb-init/releases/download/v1.2.2/dumb-init_1.2.2_${ARCH} \
-  && chmod +x /usr/local/bin/dumb-init
+    && curl -L -o /usr/local/bin/dumb-init https://github.com/Yelp/dumb-init/releases/download/v1.2.2/dumb-init_1.2.2_${ARCH} \
+    && chmod +x /usr/local/bin/dumb-init
 
 # Docker download supports arm64 as aarch64 & amd64 as x86_64
 RUN set -vx; \
-  export ARCH=$(echo ${TARGETPLATFORM} | cut -d / -f2) \
-  && if [ "$ARCH" = "arm64" ]; then export ARCH=aarch64 ; fi \
-  && if [ "$ARCH" = "amd64" ]; then export ARCH=x86_64 ; fi \
-  && curl -L -o docker.tgz https://download.docker.com/linux/static/stable/${ARCH}/docker-${DOCKER_VERSION}.tgz \
-  && tar zxvf docker.tgz \
-  && install -o root -g root -m 755 docker/docker /usr/local/bin/docker \
-  && rm -rf docker docker.tgz \
-  && adduser --disabled-password --gecos "" --uid 1000 runner \
-  && groupadd docker \
-  && usermod -aG sudo runner \
-  && usermod -aG docker runner \
-  && echo "%sudo   ALL=(ALL:ALL) NOPASSWD:ALL" > /etc/sudoers
+    export ARCH=$(echo ${TARGETPLATFORM} | cut -d / -f2) \
+    && if [ "$ARCH" = "arm64" ]; then export ARCH=aarch64 ; fi \
+    && if [ "$ARCH" = "amd64" ]; then export ARCH=x86_64 ; fi \
+    && curl -L -o docker.tgz https://download.docker.com/linux/static/stable/${ARCH}/docker-${DOCKER_VERSION}.tgz \
+    && tar zxvf docker.tgz \
+    && install -o root -g root -m 755 docker/docker /usr/local/bin/docker \
+    && rm -rf docker docker.tgz \
+    && adduser --disabled-password --gecos "" --uid 1000 runner \
+    && groupadd docker \
+    && usermod -aG sudo runner \
+    && usermod -aG docker runner \
+    && echo "%sudo   ALL=(ALL:ALL) NOPASSWD:ALL" > /etc/sudoers
 
 ENV RUNNER_ASSETS_DIR=/runnertmp
+ENV HOME=/home/runner
 
 # Runner download supports amd64 as x64. Externalstmp is needed for making mount points work inside DinD.
 #
@@ -66,25 +70,28 @@ ENV RUNNER_ASSETS_DIR=/runnertmp
 # It is installed after installdependencies.sh and before removing /var/lib/apt/lists
 # to avoid rerunning apt-update on its own.
 RUN export ARCH=$(echo ${TARGETPLATFORM} | cut -d / -f2) \
-  && if [ "$ARCH" = "amd64" ]; then export ARCH=x64 ; fi \
-  && mkdir -p "$RUNNER_ASSETS_DIR" \
-  && cd "$RUNNER_ASSETS_DIR" \
-  && curl -L -o runner.tar.gz https://github.com/actions/runner/releases/download/v${RUNNER_VERSION}/actions-runner-linux-${ARCH}-${RUNNER_VERSION}.tar.gz \
-  && tar xzf ./runner.tar.gz \
-  && rm runner.tar.gz \
-  && ./bin/installdependencies.sh \
-  && mv ./externals ./externalstmp \
-  && apt-get install -y libyaml-dev \
-  && rm -rf /var/lib/apt/lists/*
+    && if [ "$ARCH" = "amd64" -o "$ARCH" = "x86_64" ]; then export ARCH=x64 ; fi \
+    && mkdir -p "$RUNNER_ASSETS_DIR" \
+    && cd "$RUNNER_ASSETS_DIR" \
+    && curl -L -o runner.tar.gz https://github.com/actions/runner/releases/download/v${RUNNER_VERSION}/actions-runner-linux-${ARCH}-${RUNNER_VERSION}.tar.gz \
+    && tar xzf ./runner.tar.gz \
+    && rm runner.tar.gz \
+    && ./bin/installdependencies.sh \
+    && mv ./externals ./externalstmp \
+    && apt-get install -y libyaml-dev \
+    && rm -rf /var/lib/apt/lists/*
 
 RUN echo AGENT_TOOLSDIRECTORY=/opt/hostedtoolcache > .env \
-  && mkdir /opt/hostedtoolcache \
-  && chgrp runner /opt/hostedtoolcache \
-  && chmod g+rwx /opt/hostedtoolcache
+    && mkdir /opt/hostedtoolcache \
+    && chgrp docker /opt/hostedtoolcache \
+    && chmod g+rwx /opt/hostedtoolcache
 
 COPY entrypoint.sh /
 COPY --chown=runner:docker patched $RUNNER_ASSETS_DIR/patched
 
+# Add the Python "User Script Directory" to the PATH
+ENV PATH="${PATH}:${HOME}/.local/bin"
+
 USER runner
 ENTRYPOINT ["/usr/local/bin/dumb-init", "--"]
 CMD ["/entrypoint.sh"]

runner/Dockerfile.dindrunner

@@ -1,11 +1,11 @@
 FROM ubuntu:20.04
 
 ENV DEBIAN_FRONTEND=noninteractive
-# Dev + DinD dependencies
-RUN apt update \
+RUN apt update -y \
     && apt install -y software-properties-common \
     && add-apt-repository -y ppa:git-core/ppa \
-    && apt install -y \
+    && apt update -y \
+    && apt install -y --no-install-recommends \
     build-essential \
     curl \
     ca-certificates \
@@ -13,18 +13,21 @@ RUN apt update \
     ftp \
     git \
     iproute2 \
-    iptables \
     iputils-ping \
+    iptables \
     jq \
     libunwind8 \
     locales \
     netcat \
+    net-tools \
     openssh-client \
     parallel \
+    python3-pip \
     rsync \
     shellcheck \
-    sudo \
     supervisor \
+    software-properties-common \
+    sudo \
     telnet \
     time \
     tzdata \
@@ -33,7 +36,9 @@ RUN apt update \
     wget \
     zip \
     zstd \
-    && rm -rf /var/lib/apt/list/*
+    && ln -sf /usr/bin/python3 /usr/bin/python \
+    && ln -sf /usr/bin/pip3 /usr/bin/pip \
+    && rm -rf /var/lib/apt/lists/*
 
 # Runner user
 RUN adduser --disabled-password --gecos "" --uid 1000 runner \
@@ -43,7 +48,7 @@ RUN adduser --disabled-password --gecos "" --uid 1000 runner \
     && echo "%sudo   ALL=(ALL:ALL) NOPASSWD:ALL" > /etc/sudoers
 
 ARG TARGETPLATFORM
-ARG RUNNER_VERSION=2.274.1
+ARG RUNNER_VERSION=2.278.0
 ARG DOCKER_CHANNEL=stable
 ARG DOCKER_VERSION=19.03.13
 ARG DEBUG=false
@@ -69,6 +74,7 @@ RUN export ARCH=$(echo ${TARGETPLATFORM} | cut -d / -f2) \
 	docker --version
 
 ENV RUNNER_ASSETS_DIR=/runnertmp
+ENV HOME=/home/runner
 
 # Runner download supports amd64 as x64
 #
@@ -76,9 +82,9 @@ ENV RUNNER_ASSETS_DIR=/runnertmp
 # It is installed after installdependencies.sh and before removing /var/lib/apt/lists
 # to avoid rerunning apt-update on its own.
 RUN export ARCH=$(echo ${TARGETPLATFORM} | cut -d / -f2) \
-    && if [ "$ARCH" = "amd64" ]; then export ARCH=x64 ; fi \
+    && if [ "$ARCH" = "amd64" -o "$ARCH" = "x86_64" ]; then export ARCH=x64 ; fi \
     && mkdir -p "$RUNNER_ASSETS_DIR" \
-     && cd "$RUNNER_ASSETS_DIR" \
+    && cd "$RUNNER_ASSETS_DIR" \
     && curl -L -o runner.tar.gz https://github.com/actions/runner/releases/download/v${RUNNER_VERSION}/actions-runner-linux-${ARCH}-${RUNNER_VERSION}.tar.gz \
     && tar xzf ./runner.tar.gz \
     && rm runner.tar.gz \
@@ -87,9 +93,9 @@ RUN export ARCH=$(echo ${TARGETPLATFORM} | cut -d / -f2) \
     && rm -rf /var/lib/apt/lists/*
 
 RUN echo AGENT_TOOLSDIRECTORY=/opt/hostedtoolcache > /runner.env \
-  && mkdir /opt/hostedtoolcache \
-  && chgrp runner /opt/hostedtoolcache \
-  && chmod g+rwx /opt/hostedtoolcache
+    && mkdir /opt/hostedtoolcache \
+    && chgrp docker /opt/hostedtoolcache \
+    && chmod g+rwx /opt/hostedtoolcache
 
 COPY modprobe startup.sh /usr/local/bin/
 COPY supervisor/ /etc/supervisor/conf.d/
@@ -106,6 +112,9 @@ VOLUME /var/lib/docker
 
 COPY --chown=runner:docker patched $RUNNER_ASSETS_DIR/patched
 
+# Add the Python "User Script Directory" to the PATH
+ENV PATH="${PATH}:${HOME}/.local/bin"
+
 # No group definition, as that makes it harder to run docker.
 USER runner
 

runner/Dockerfile.ubuntu.1804

@@ -0,0 +1,97 @@
+FROM ubuntu:18.04
+
+ARG TARGETPLATFORM
+ARG RUNNER_VERSION=2.278.0
+ARG DOCKER_VERSION=19.03.12
+
+RUN test -n "$TARGETPLATFORM" || (echo "TARGETPLATFORM must be set" && false)
+
+ENV DEBIAN_FRONTEND=noninteractive
+RUN apt update -y \
+    && apt install -y software-properties-common \
+    && add-apt-repository -y ppa:git-core/ppa \
+    && apt update -y \
+    && apt install -y --no-install-recommends \
+    build-essential \
+    curl \
+    ca-certificates \
+    dnsutils \
+    ftp \
+    git \
+    iproute2 \
+    iputils-ping \
+    jq \
+    libunwind8 \
+    locales \
+    netcat \
+    openssh-client \
+    parallel \
+    python3-pip \
+    rsync \
+    shellcheck \
+    sudo \
+    telnet \
+    time \
+    tzdata \
+    unzip \
+    upx \
+    wget \
+    zip \
+    zstd \
+    && ln -sf /usr/bin/python3 /usr/bin/python \
+    && ln -sf /usr/bin/pip3 /usr/bin/pip \
+    && rm -rf /var/lib/apt/lists/*
+
+RUN export ARCH=$(echo ${TARGETPLATFORM} | cut -d / -f2) \
+    && curl -L -o /usr/local/bin/dumb-init https://github.com/Yelp/dumb-init/releases/download/v1.2.2/dumb-init_1.2.2_${ARCH} \
+    && chmod +x /usr/local/bin/dumb-init
+
+# Docker download supports arm64 as aarch64 & amd64 as x86_64
+RUN set -vx; \
+    export ARCH=$(echo ${TARGETPLATFORM} | cut -d / -f2) \
+    && if [ "$ARCH" = "arm64" ]; then export ARCH=aarch64 ; fi \
+    && if [ "$ARCH" = "amd64" ]; then export ARCH=x86_64 ; fi \
+    && curl -L -o docker.tgz https://download.docker.com/linux/static/stable/${ARCH}/docker-${DOCKER_VERSION}.tgz \
+    && tar zxvf docker.tgz \
+    && install -o root -g root -m 755 docker/docker /usr/local/bin/docker \
+    && rm -rf docker docker.tgz \
+    && adduser --disabled-password --gecos "" --uid 1000 runner \
+    && groupadd docker \
+    && usermod -aG sudo runner \
+    && usermod -aG docker runner \
+    && echo "%sudo   ALL=(ALL:ALL) NOPASSWD:ALL" > /etc/sudoers
+
+ENV RUNNER_ASSETS_DIR=/runnertmp
+ENV HOME=/home/runner
+
+# Runner download supports amd64 as x64. Externalstmp is needed for making mount points work inside DinD.
+#
+# libyaml-dev is required for ruby/setup-ruby action.
+# It is installed after installdependencies.sh and before removing /var/lib/apt/lists
+# to avoid rerunning apt-update on its own.
+RUN export ARCH=$(echo ${TARGETPLATFORM} | cut -d / -f2) \
+    && if [ "$ARCH" = "amd64" -o "$ARCH" = "x86_64" ]; then export ARCH=x64 ; fi \
+    && mkdir -p "$RUNNER_ASSETS_DIR" \
+    && cd "$RUNNER_ASSETS_DIR" \
+    && curl -L -o runner.tar.gz https://github.com/actions/runner/releases/download/v${RUNNER_VERSION}/actions-runner-linux-${ARCH}-${RUNNER_VERSION}.tar.gz \
+    && tar xzf ./runner.tar.gz \
+    && rm runner.tar.gz \
+    && ./bin/installdependencies.sh \
+    && mv ./externals ./externalstmp \
+    && apt-get install -y libyaml-dev \
+    && rm -rf /var/lib/apt/lists/*
+
+RUN echo AGENT_TOOLSDIRECTORY=/opt/hostedtoolcache > .env \
+    && mkdir /opt/hostedtoolcache \
+    && chgrp docker /opt/hostedtoolcache \
+    && chmod g+rwx /opt/hostedtoolcache
+
+COPY entrypoint.sh /
+COPY --chown=runner:docker patched $RUNNER_ASSETS_DIR/patched
+
+# Add the Python "User Script Directory" to the PATH
+ENV PATH="${PATH}:${HOME}/.local/bin"
+
+USER runner
+ENTRYPOINT ["/usr/local/bin/dumb-init", "--"]
+CMD ["/entrypoint.sh"]

runner/Makefile

@@ -2,7 +2,7 @@ NAME ?= summerwind/actions-runner
 DIND_RUNNER_NAME ?= ${NAME}-dind
 TAG ?= latest
 
-RUNNER_VERSION ?= 2.274.2
+RUNNER_VERSION ?= 2.278.0
 DOCKER_VERSION ?= 19.03.12
 
 # default list of platforms for which multiarch image is built
@@ -22,16 +22,15 @@ else
 	export PUSH_ARG="--push"
 endif
 
-docker-build:
+docker-build-ubuntu:
 	docker build --build-arg TARGETPLATFORM=amd64 --build-arg RUNNER_VERSION=${RUNNER_VERSION} --build-arg DOCKER_VERSION=${DOCKER_VERSION} -t ${NAME}:${TAG} .
-	docker build --build-arg TARGETPLATFORM=amd64 --build-arg RUNNER_VERSION=${RUNNER_VERSION} --build-arg DOCKER_VERSION=${DOCKER_VERSION} -t ${DIND_RUNNER_NAME}:${TAG} -f dindrunner.Dockerfile .
+	docker build --build-arg TARGETPLATFORM=amd64 --build-arg RUNNER_VERSION=${RUNNER_VERSION} --build-arg DOCKER_VERSION=${DOCKER_VERSION} -t ${DIND_RUNNER_NAME}:${TAG} -f Dockerfile.dindrunner .
 
-
-docker-push:
+docker-push-ubuntu:
 	docker push ${NAME}:${TAG}
 	docker push ${DIND_RUNNER_NAME}:${TAG}
 
-docker-buildx:
+docker-buildx-ubuntu:
 	export DOCKER_CLI_EXPERIMENTAL=enabled
 	@if ! docker buildx ls | grep -q container-builder; then\
 		docker buildx create --platform ${PLATFORMS} --name container-builder --use;\
@@ -46,5 +45,5 @@ docker-buildx:
 		--build-arg RUNNER_VERSION=${RUNNER_VERSION} \
 		--build-arg DOCKER_VERSION=${DOCKER_VERSION} \
 		-t "${DIND_RUNNER_NAME}:latest" \
-		-f dindrunner.Dockerfile \
+		-f Dockerfile.dindrunner \
 		. ${PUSH_ARG}

runner/entrypoint.sh

@@ -1,5 +1,10 @@
 #!/bin/bash
 
+if [ ! -z "${STARTUP_DELAY}" ]; then
+  echo "Delaying startup by ${STARTUP_DELAY} seconds" 1>&2
+  sleep ${STARTUP_DELAY}
+fi
+
 if [ -z "${GITHUB_URL}" ]; then
   echo "Working with public GitHub" 1>&2
   GITHUB_URL="https://github.com/"
@@ -29,21 +34,13 @@ else
   exit 1
 fi
 
-if [ -n "${RUNNER_WORKDIR}" ]; then
-  WORKDIR_ARG="--work ${RUNNER_WORKDIR}"
-fi
-
-if [ -n "${RUNNER_LABELS}" ]; then
-  LABEL_ARG="--labels ${RUNNER_LABELS}"
-fi
-
 if [ -z "${RUNNER_TOKEN}" ]; then
   echo "RUNNER_TOKEN must be set" 1>&2
   exit 1
 fi
 
-if [ -z "${RUNNER_REPO}" ] && [ -n "${RUNNER_ORG}" ] && [ -n "${RUNNER_GROUP}" ];then
-  RUNNER_GROUP_ARG="--runnergroup ${RUNNER_GROUP}"
+if [ -z "${RUNNER_REPO}" ] && [ -n "${RUNNER_GROUP}" ];then
+  RUNNER_GROUPS=${RUNNER_GROUP}
 fi
 
 # Hack due to https://github.com/summerwind/actions-runner-controller/issues/252#issuecomment-758338483
@@ -56,7 +53,45 @@ sudo chown -R runner:docker /runner
 mv /runnertmp/* /runner/
 
 cd /runner
-./config.sh --unattended --replace --name "${RUNNER_NAME}" --url "${GITHUB_URL}${ATTACH}" --token "${RUNNER_TOKEN}" ${RUNNER_GROUP_ARG} ${LABEL_ARG} ${WORKDIR_ARG}
+./config.sh --unattended --replace \
+  --name "${RUNNER_NAME}" \
+  --url "${GITHUB_URL}${ATTACH}" \
+  --token "${RUNNER_TOKEN}" \
+  --runnergroup "${RUNNER_GROUPS}" \
+  --labels "${RUNNER_LABELS}" \
+  --work "${RUNNER_WORKDIR}"
+
+if [ -f /runner/.runner ]; then
+  echo Runner has successfully been configured with the following data.
+  cat /runner/.runner
+  # Note: the `.runner` file's content should be something like the below:
+  #
+  # $ cat /runner/.runner
+  # {
+  # "agentId": 117, #=> corresponds to the ID of the runner
+  # "agentName": "THE_RUNNER_POD_NAME",
+  # "poolId": 1,
+  # "poolName": "Default",
+  # "serverUrl": "https://pipelines.actions.githubusercontent.com/SOME_RANDOM_ID",
+  # "gitHubUrl": "https://github.com/USER/REPO",
+  # "workFolder": "/some/work/dir" #=> corresponds to Runner.Spec.WorkDir
+  # }
+  #
+  # Especially `agentId` is important, as other than listing all the runners in the repo,
+  # this is the only change we could get the exact runnner ID which can be useful for further
+  # GitHub API call like the below. Note that 171 is the agentId seen above.
+  #   curl \
+  #     -H "Accept: application/vnd.github.v3+json" \
+  #     -H "Authorization: bearer ${GITHUB_TOKEN}"
+  #     https://api.github.com/repos/USER/REPO/actions/runners/171
+fi
+
+if [ -n "${RUNNER_REGISTRATION_ONLY}" ]; then
+  echo
+  echo "This runner is configured to be registration-only. Existing without starting the runner service..."
+  exit 0
+fi
+
 mkdir ./externals
 # Hack due to the DinD volumes
 mv ./externalstmp/* ./externals/
@@ -67,5 +102,10 @@ for f in runsvc.sh RunnerService.js; do
   sudo mv {patched,bin}/${f}
 done
 
+args=()
+if [ "${RUNNER_EPHEMERAL}" != "false" ]; then
+  args+=(--once)
+fi
+
 unset RUNNER_NAME RUNNER_REPO RUNNER_TOKEN
-exec ./bin/runsvc.sh --once
+exec ./bin/runsvc.sh "${args[@]}"