feat: Support for scaling from/to zero (#465)

This is an attempt to support scaling from/to zero.

The basic idea is that we create a one-off "registration-only" runner pod on RunnerReplicaSet being scaled to zero, so that there is one "offline" runner, which enables GitHub Actions to queue jobs instead of discarding those.

GitHub Actions seems to immediately throw away the new job when there are no runners at all. Generally, having runners of any status, `busy`, `idle`, or `offline` would prevent GitHub actions from failing jobs. But retaining `busy` or `idle` runners means that we need to keep runner pods running, which conflicts with our desired to scale to/from zero, hence we retain `offline` runners.

In this change, I enhanced the runnerreplicaset controller to create a registration-only runner on very beginning of its reconciliation logic, only when a runnerreplicaset is scaled to zero. The runner controller creates the registration-only runner pod, waits for it to become "offline", and then removes the runner pod. The runner on GitHub stays `offline`, until the runner resource on K8s is deleted. As we remove the registration-only runner pod as soon as it registers, this doesn't block cluster-autoscaler.

Related to #447

runner/entrypoint.sh

@@ -56,6 +56,37 @@ cd /runner
   --labels "${RUNNER_LABELS}" \
   --work "${RUNNER_WORKDIR}"
 
+if [ -f /runner/.runner ]; then
+  echo Runner has successfully been configured with the following data.
+  cat /runner/.runner
+  # Note: the `.runner` file's content should be something like the below:
+  #
+  # $ cat /runner/.runner
+  # {
+  # "agentId": 117, #=> corresponds to the ID of the runner
+  # "agentName": "THE_RUNNER_POD_NAME",
+  # "poolId": 1,
+  # "poolName": "Default",
+  # "serverUrl": "https://pipelines.actions.githubusercontent.com/SOME_RANDOM_ID",
+  # "gitHubUrl": "https://github.com/USER/REPO",
+  # "workFolder": "/some/work/dir" #=> corresponds to Runner.Spec.WorkDir
+  # }
+  #
+  # Especially `agentId` is important, as other than listing all the runners in the repo,
+  # this is the only change we could get the exact runnner ID which can be useful for further
+  # GitHub API call like the below. Note that 171 is the agentId seen above.
+  #   curl \
+  #     -H "Accept: application/vnd.github.v3+json" \
+  #     -H "Authorization: bearer ${GITHUB_TOKEN}"
+  #     https://api.github.com/repos/USER/REPO/actions/runners/171
+fi
+
+if [ -n "${RUNNER_REGISTRATION_ONLY}" ]; then
+  echo
+  echo "This runner is configured to be registration-only. Existing without starting the runner service..."
+  exit 0
+fi
+
 mkdir ./externals
 # Hack due to the DinD volumes
 mv ./externalstmp/* ./externals/