Compare commits

...

35 Commits

Author SHA1 Message Date
Moto Ishizawa
f1556ff060 Merge pull request #63 from summerwind/runner-v2.267.0
Update runner to v2.267.0
2020-06-28 12:00:24 +09:00
Moto Ishizawa
8c35cab1a4 Update runner to v2.267.0 2020-06-28 11:59:16 +09:00
KUOKA Yusuke
5bb2694349 feat: Repository-wide RunnerDeployment Autoscaling (#57)
* feat: Repository-wide RunnerDeployment Autoscaling

This adds `maxReplicas` and `minReplicas` to the RunnerDeploymentSpec. If and only if both fields are set, the controller computes and sets desired `replicas` automatically depending on the demand.

The number of demanded runner replicas is computed by `queued workflow runs + in_progress workflow runs` for the repository. The support for organizational runners is not included.

Ref https://github.com/summerwind/actions-runner-controller/issues/10
2020-06-27 17:26:46 +09:00
Moto Ishizawa
512cae68a1 Merge pull request #59 from vitali-raikov/patch-1
Add an extra permission to README for organization self hosted runners
2020-06-02 22:49:29 +09:00
Vitali Raikov
905ed18824 Add an extra permission for organization self hosted runners 2020-06-02 16:16:26 +03:00
Moto Ishizawa
6ce35ced7a Merge pull request #56 from summerwind/runner-v2.263.0
Update runner to v2.263.0
2020-05-25 22:58:43 +09:00
Moto Ishizawa
d09b14b629 Update runner to v2.263.0 2020-05-24 20:45:16 +09:00
Moto Ishizawa
eaabf1fc8c Merge pull request #55 from summerwind/runner-v2.262.1
Update runner to v2.262.1
2020-05-13 22:12:26 +09:00
Moto Ishizawa
553dda65a4 Update runner to v2.262.1 2020-05-13 22:10:23 +09:00
Moto Ishizawa
d446893890 Merge pull request #54 from summerwind/runner-v2.262.0
Update runner to v2.262.0
2020-05-13 21:59:30 +09:00
Moto Ishizawa
46883eb742 Update runner to v2.262.0 2020-05-12 22:46:06 +09:00
Moto Ishizawa
390f2a62d9 Merge pull request #50 from summerwind/runner-validation-webhook
Add validation webhooks
2020-05-08 22:39:13 +09:00
naka-gawa
1555651325 fix typo readme 2020-05-06 12:25:30 +09:00
Moto Ishizawa
e7445e286f Merge pull request #52 from reiniertimmer/add-git-unzip-to-runner
Add dependencies from GitHub virtual environment to runner
2020-05-04 22:34:38 +09:00
Reinier Timmer
79655989d0 Add additional software packages to runner image 2020-05-04 06:55:16 +00:00
Moto Ishizawa
55323c3754 Update installation section 2020-05-02 16:57:19 +09:00
Moto Ishizawa
f80c3c1928 Set volume to pod properly 2020-05-01 08:51:25 +09:00
Moto Ishizawa
9a86812214 Add manifests for validation webhook 2020-04-30 22:12:39 +09:00
Moto Ishizawa
e889eaeb04 Add validation webhooks 2020-04-30 22:11:59 +09:00
Reinier Timmer
b96979888c fix delete pod when runner failed to register 2020-04-29 14:23:58 +09:00
Moto Ishizawa
7df119e470 Merge pull request #44 from reiniertimmer/organization-runners
support for organization runners & labels
2020-04-28 22:38:35 +09:00
Reinier Timmer
966e0dca37 fix merge conflict on README 2020-04-28 11:24:59 +02:00
Reinier Timmer
8c42b317ec updated documentation 2020-04-28 11:15:41 +02:00
Reinier Timmer
9f57f52e36 organization and repository are now exclusive 2020-04-28 11:14:31 +02:00
Reinier Timmer
8c5b776807 support runner labels 2020-04-28 11:14:31 +02:00
Reinier Timmer
2567f6ee4e omit empty repository from runner spec 2020-04-28 11:14:31 +02:00
Reinier Timmer
eca3cc7941 add organization info to runner status 2020-04-28 11:14:31 +02:00
Reinier Timmer
75d15ee91b backwards compatibility of dockerfile 2020-04-28 11:14:31 +02:00
Reinier Timmer
fb35dd4131 support for organization runners 2020-04-28 11:14:31 +02:00
Moto Ishizawa
d1429beaa6 Merge pull request #45 from summerwind/share-work-dir
Share runner's working directory with docker sidecar
2020-04-25 10:08:04 +09:00
Moto Ishizawa
3b8ea2991c Share runner's working directory with docker sidecar 2020-04-24 22:36:27 +09:00
Moto Ishizawa
6bdfb5cff6 Merge pull request #41 from arhue/improve-readme
Improve readme
2020-04-23 19:59:28 +09:00
Varun Priolkar
df5eed52d0 Specify language in additional tweaks code block 2020-04-23 12:57:56 +05:30
Varun Priolkar
5f271a3050 Fix code block on additional tweak section 2020-04-23 12:57:15 +05:30
Varun Priolkar
6997cc97c6 Improve readme 2020-04-23 12:53:46 +05:30
27 changed files with 1509 additions and 141 deletions

214
README.md
View File

@@ -4,19 +4,30 @@ This controller operates self-hosted runners for GitHub Actions on your Kubernet
## Motivation
[GitHub Actions](https://github.com/features/actions) is very useful as a tool for automating development. GitHub Actions job is run in the cloud by default, but you may want to run your jobs in your environment. [Self-hosted runner](https://github.com/actions/runner) can be used for such use cases, but requires the provision of a virtual machine instance and configuration. If you already have a Kubernetes cluster, you'll want to run the self-hosted runner on top of it.
[GitHub Actions](https://github.com/features/actions) is a very useful tool for automating development. GitHub Actions jobs are run in the cloud by default, but you may want to run your jobs in your environment. [Self-hosted runner](https://github.com/actions/runner) can be used for such use cases, but requires the provisioning and configuration of a virtual machine instance. Instead if you already have a Kubernetes cluster, it makes more sense to run the self-hosted runner on top of it.
*actions-runner-controller* makes that possible. Just create a *Runner* resource on your Kubernetes, and it will run and operate the self-hosted runner of the specified repository. Combined with Kubernetes RBAC, you can also build simple Self-hosted runners as a Service.
**actions-runner-controller** makes that possible. Just create a *Runner* resource on your Kubernetes, and it will run and operate the self-hosted runner for the specified repository. Combined with Kubernetes RBAC, you can also build simple Self-hosted runners as a Service.
## Installation
First, install *actions-runner-controller* with a manifest file. This will create a *actions-runner-system* namespace in your Kubernetes and deploy the required resources.
actions-runner-controller uses [cert-manager](https://cert-manager.io/docs/installation/kubernetes/) for certificate management of Admission Webhook. Make sure you have already installed cert-manager before you install. The installation instructions for cert-manager can be found below.
- [Installing cert-manager on Kubernetes](https://cert-manager.io/docs/installation/kubernetes/)
Install the custom resource and actions-runner-controller itself. This will create actions-runner-system namespace in your Kubernetes and deploy the required resources.
```
$ kubectl apply -f https://github.com/summerwind/actions-runner-controller/releases/latest/download/actions-runner-controller.yaml
```
Next, set up a GitHub App or personal access token for *actions-runner-controller* to access the GitHub API.
## Setting up authentication with GitHub API
There are two ways for actions-runner-controller to authenticate with the the GitHub API:
1. Using GitHub App.
2. Using Personal Access Token.
**NOTE: It is extremely important to only follow one of the sections below and not both.**
### Using GitHub App
@@ -26,7 +37,7 @@ You can create a GitHub App for either your account or any organization. If you
If you want to create a GitHub App for your organization, replace the `:org` part of the following URL with your organization name before opening it. Then enter any unique name in the "GitHub App name" field, and hit the "Create GitHub App" button at the bottom of the page to create a GitHub App.
- [Create GitHub Apps on your organization](https://github.com/organizations/:org/settings/apps/new?url=http://github.com/summerwind/actions-runner-controller&webhook_active=false&public=false&administration=write)
- [Create GitHub Apps on your organization](https://github.com/organizations/:org/settings/apps/new?url=http://github.com/summerwind/actions-runner-controller&webhook_active=false&public=false&administration=write&organization_self_hosted_runners=write)
You will see an *App ID* on the page of the GitHub App you created as follows, the value of this App ID will be used later.
@@ -55,13 +66,15 @@ $ kubectl create secret generic controller-manager \
--from-file=github_app_private_key=${PRIVATE_KEY_FILE_PATH}
```
### Using personal access token
### Using Personal Access Token
Next, from an account that has `admin` privileges for the repository, create a [personal access token](https://github.com/settings/tokens) with `repo` scope. This token is used to register a self-hosted runner by *actions-runner-controller*.
From an account that has `admin` privileges for the repository, create a [personal access token](https://github.com/settings/tokens) with `repo` scope. This token is used to register a self-hosted runner by *actions-runner-controller*.
To use a Personal Access Token, you must issue the token with an account that has `admin` privileges.
Self-hosted runners in GitHub can either be connected to a single repository, or to a GitHub organization (so they are available to all repositories in the organization). This token is used to register a self-hosted runner by *actions-runner-controller*.
Open the Create Token page from the following link, grant the `repo` scope, and press the "Generate Token" button at the bottom of the page to create the token.
For adding a runner to a repository, the token should have `repo` scope. If the runner should be added to an organization, the token should have `admin:org` scope. Note that to use a Personal Access Token, you must issue the token with an account that has `admin` privileges (on the repository and/or the organization).
Open the Create Token page from the following link, grant the `repo` and/or `admin:org` scope, and press the "Generate Token" button at the bottom of the page to create the token.
- [Create personal access token](https://github.com/settings/tokens/new)
@@ -75,14 +88,14 @@ $ kubectl create secret generic controller-manager \
## Usage
There's generally two ways to use this controller:
There are two ways to use this controller:
- Manage runners one by one with `Runner`
- Manage a set of runners with `RunnerDeployment`
- Manage runners one by one with `Runner`.
- Manage a set of runners with `RunnerDeployment`.
### Runners
### Repository runners
To launch a single Self-hosted runner, you need to create a manifest file includes *Runner* resource as follows. This example launches a self-hosted runner with name *example-runner* for the *summerwind/actions-runner-controller* repository.
To launch a single self-hosted runner, you need to create a manifest file includes *Runner* resource as follows. This example launches a self-hosted runner with name *example-runner* for the *summerwind/actions-runner-controller* repository.
```
# runner.yaml
@@ -124,9 +137,25 @@ The runner you created has been registered to your repository.
Now your can use your self-hosted runner. See the [official documentation](https://help.github.com/en/actions/automating-your-workflow-with-github-actions/using-self-hosted-runners-in-a-workflow) on how to run a job with it.
### Organization Runners
To add the runner to an organization, you only need to replace the `repository` field with `organization`, so the runner will register itself to the organization.
```
# runner.yaml
apiVersion: actions.summerwind.dev/v1alpha1
kind: Runner
metadata:
name: example-org-runner
spec:
organization: your-organization-name
```
Now you can see the runner on the organization level (if you have organization owner permissions).
### RunnerDeployments
There's also `RunnerReplicaSet` and `RunnerDeployment` that corresponds to `ReplicaSet` and `Deployment` but for `Runner`.
There are `RunnerReplicaSet` and `RunnerDeployment` that corresponds to `ReplicaSet` and `Deployment` but for `Runner`.
You usually need only `RunnerDeployment` rather than `RunnerReplicaSet` as the former is for managing the latter.
@@ -151,12 +180,163 @@ $ kubectl apply -f runner.yaml
runnerdeployment.actions.summerwind.dev/example-runnerdeploy created
```
You can see that 2 runners has been created as specified by `replicas: 2`:
You can see that 2 runners have been created as specified by `replicas: 2`:
```
$ kubectl get runners
NAME REPOSITORY STATUS
NAME REPOSITORY STATUS
example-runnerdeploy2475h595fr mumoshu/actions-runner-controller-ci Running
example-runnerdeploy2475ht2qbr mumoshu/actions-runner-controller-ci Running
```
#### Autoscaling
`RunnerDeployment` can scale number of runners between `minReplicas` and `maxReplicas` fields, depending on pending workflow runs.
In the below example, `actions-runner` checks for pending workflow runs for each sync period, and scale to e.g. 3 if there're 3 pending jobs at sync time.
```
apiVersion: actions.summerwind.dev/v1alpha1
kind: RunnerDeployment
metadata:
name: summerwind-actions-runner-controller
spec:
minReplicas: 1
maxReplicas: 3
template:
spec:
repository: summerwind/actions-runner-controller
```
Please also note that the sync period is set to 10 minutes by default and it's configurable via `--sync-period` flag.
Additionally, the autoscaling feature has an anti-flapping option that prevents periodic loop of scaling up and down.
By default, it doesn't scale down until the grace period of 10 minutes passes after a scale up. The grace period can be configured by setting `scaleDownDelaySecondsAfterScaleUp`:
```
apiVersion: actions.summerwind.dev/v1alpha1
kind: RunnerDeployment
metadata:
name: summerwind-actions-runner-controller
spec:
minReplicas: 1
maxReplicas: 3
scaleDownDelaySecondsAfterScaleUp: 1m
template:
spec:
repository: summerwind/actions-runner-controller
```
## Additional tweaks
You can pass details through the spec selector. Here's an eg. of what you may like to do:
```yaml
apiVersion: actions.summerwind.dev/v1alpha1
kind: RunnerDeployment
metadata:
name: actions-runner
namespace: default
spec:
replicas: 2
template:
spec:
nodeSelector:
node-role.kubernetes.io/test: ""
tolerations:
- effect: NoSchedule
key: node-role.kubernetes.io/test
operator: Exists
repository: mumoshu/actions-runner-controller-ci
ImagePullPolicy: Always
image: custom-image/actions-runner:latest
resources:
limits:
cpu: "4.0"
memory: "8Gi"
requests:
cpu: "2.0"
memory: "4Gi"
sidecarContainers:
- name: mysql
image: mysql:5.7
env:
- name: MYSQL_ROOT_PASSWORD
value: abcd1234
securityContext:
runAsUser: 0
```
## Runner labels
To run a workflow job on a self-hosted runner, you can use the following syntax in your workflow:
```yaml
jobs:
release:
runs-on: self-hosted
```
When you have multiple kinds of self-hosted runners, you can distinguish between them using labels. In order to do so, you can specify one or more labels in your `Runner` or `RunnerDeployment` spec.
```yaml
# runnerdeployment.yaml
apiVersion: actions.summerwind.dev/v1alpha1
kind: RunnerDeployment
metadata:
name: custom-runner
spec:
replicas: 1
template:
spec:
repository: summerwind/actions-runner-controller
labels:
- custom-runner
```
Once this spec is applied, you can observe the labels for your runner from the repository or organization in the GitHub settings page for the repository or organization. You can now select a specific runner from your workflow by using the label in `runs-on`:
```yaml
jobs:
release:
runs-on: custom-runner
```
Note that if you specify `self-hosted` in your worlflow, then this will run your job on _any_ self-hosted runner, regardless of the labels that they have.
## Softeware installed in the runner image
The GitHub hosted runners include a large amount of pre-installed software packages. For Ubuntu 18.04, this list can be found at https://github.com/actions/virtual-environments/blob/master/images/linux/Ubuntu1804-README.md
The container image is based on Ubuntu 18.04, but it does not contain all of the software installed on the GitHub runners. It contains the following subset of packages from the GitHub runners:
* Basic CLI packages
* git (2.26)
* docker
* build-essentials
The virtual environments from GitHub contain a lot more software packages (different versions of Java, Node.js, Golang, .NET, etc) which are not provided in the runner image. Most of these have dedicated setup actions which allow the tools to be installed on-demand in a workflow, for example: `actions/setup-java` or `actions/setup-node`
If there is a need to include packages in the runner image for which there is no setup action, then this can be achieved by building a custom container image for the runner. The easiest way is to start with the `summerwind/actions-runner` image and installing the extra dependencies directly in the docker image:
```yaml
FROM summerwind/actions-runner:v2.169.1
RUN sudo apt update -y \
&& apt install YOUR_PACKAGE
&& rm -rf /var/lib/apt/lists/*
```
You can then configure the runner to use a custom docker image by configuring the `image` field of a `Runner` or `RunnerDeployment`:
```yaml
apiVersion: actions.summerwind.dev/v1alpha1
kind: Runner
metadata:
name: custom-runner
spec:
repository: summerwind/actions-runner-controller
image: YOUR_CUSTOM_DOCKER_IMAGE
```

View File

@@ -17,15 +17,24 @@ limitations under the License.
package v1alpha1
import (
"errors"
corev1 "k8s.io/api/core/v1"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
)
// RunnerSpec defines the desired state of Runner
type RunnerSpec struct {
// +kubebuilder:validation:MinLength=3
// +optional
// +kubebuilder:validation:Pattern=`^[^/]+$`
Organization string `json:"organization,omitempty"`
// +optional
// +kubebuilder:validation:Pattern=`^[^/]+/[^/]+$`
Repository string `json:"repository"`
Repository string `json:"repository,omitempty"`
// +optional
Labels []string `json:"labels,omitempty"`
// +optional
Containers []corev1.Container `json:"containers,omitempty"`
@@ -68,6 +77,19 @@ type RunnerSpec struct {
TerminationGracePeriodSeconds *int64 `json:"terminationGracePeriodSeconds,omitempty"`
}
// ValidateRepository validates repository field.
func (rs *RunnerSpec) ValidateRepository() error {
// Organization and repository are both exclusive.
if len(rs.Organization) == 0 && len(rs.Repository) == 0 {
return errors.New("Spec needs organization or repository")
}
if len(rs.Organization) > 0 && len(rs.Repository) > 0 {
return errors.New("Spec cannot have both organization and repository")
}
return nil
}
// RunnerStatus defines the observed state of Runner
type RunnerStatus struct {
Registration RunnerStatusRegistration `json:"registration"`
@@ -76,15 +98,20 @@ type RunnerStatus struct {
Message string `json:"message"`
}
// RunnerStatusRegistration contains runner registration status
type RunnerStatusRegistration struct {
Repository string `json:"repository"`
Token string `json:"token"`
ExpiresAt metav1.Time `json:"expiresAt"`
Organization string `json:"organization,omitempty"`
Repository string `json:"repository,omitempty"`
Labels []string `json:"labels,omitempty"`
Token string `json:"token"`
ExpiresAt metav1.Time `json:"expiresAt"`
}
// +kubebuilder:object:root=true
// +kubebuilder:subresource:status
// +kubebuilder:printcolumn:JSONPath=".spec.organization",name=Organization,type=string
// +kubebuilder:printcolumn:JSONPath=".spec.repository",name=Repository,type=string
// +kubebuilder:printcolumn:JSONPath=".spec.labels",name=Labels,type=string
// +kubebuilder:printcolumn:JSONPath=".status.phase",name=Status,type=string
// Runner is the Schema for the runners API

View File

@@ -0,0 +1,84 @@
/*
Copyright 2020 The actions-runner-controller authors.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package v1alpha1
import (
apierrors "k8s.io/apimachinery/pkg/api/errors"
"k8s.io/apimachinery/pkg/runtime"
"k8s.io/apimachinery/pkg/util/validation/field"
ctrl "sigs.k8s.io/controller-runtime"
logf "sigs.k8s.io/controller-runtime/pkg/runtime/log"
"sigs.k8s.io/controller-runtime/pkg/webhook"
)
// log is for logging in this package.
var runnerLog = logf.Log.WithName("runner-resource")
func (r *Runner) SetupWebhookWithManager(mgr ctrl.Manager) error {
return ctrl.NewWebhookManagedBy(mgr).
For(r).
Complete()
}
// +kubebuilder:webhook:path=/mutate-actions-summerwind-dev-v1alpha1-runner,verbs=create;update,mutating=true,failurePolicy=fail,groups=actions.summerwind.dev,resources=runners,versions=v1alpha1,name=mutate.runner.actions.summerwind.dev
var _ webhook.Defaulter = &Runner{}
// Default implements webhook.Defaulter so a webhook will be registered for the type
func (r *Runner) Default() {
// Nothing to do.
}
// +kubebuilder:webhook:path=/validate-actions-summerwind-dev-v1alpha1-runner,verbs=create;update,mutating=false,failurePolicy=fail,groups=actions.summerwind.dev,resources=runners,versions=v1alpha1,name=validate.runner.actions.summerwind.dev
var _ webhook.Validator = &Runner{}
// ValidateCreate implements webhook.Validator so a webhook will be registered for the type
func (r *Runner) ValidateCreate() error {
runnerLog.Info("validate resource to be created", "name", r.Name)
return r.Validate()
}
// ValidateUpdate implements webhook.Validator so a webhook will be registered for the type
func (r *Runner) ValidateUpdate(old runtime.Object) error {
runnerLog.Info("validate resource to be updated", "name", r.Name)
return r.Validate()
}
// ValidateDelete implements webhook.Validator so a webhook will be registered for the type
func (r *Runner) ValidateDelete() error {
return nil
}
// Validate validates resource spec.
func (r *Runner) Validate() error {
var (
errList field.ErrorList
err error
)
err = r.Spec.ValidateRepository()
if err != nil {
errList = append(errList, field.Invalid(field.NewPath("spec", "repository"), r.Spec.Repository, err.Error()))
}
if len(errList) > 0 {
return apierrors.NewInvalid(r.GroupVersionKind().GroupKind(), r.Name, errList)
}
return nil
}

View File

@@ -22,7 +22,21 @@ import (
// RunnerReplicaSetSpec defines the desired state of RunnerDeployment
type RunnerDeploymentSpec struct {
Replicas *int `json:"replicas"`
// +optional
Replicas *int `json:"replicas,omitempty"`
// MinReplicas is the minimum number of replicas the deployment is allowed to scale
// +optional
MinReplicas *int `json:"minReplicas,omitempty"`
// MinReplicas is the maximum number of replicas the deployment is allowed to scale
// +optional
MaxReplicas *int `json:"maxReplicas,omitempty"`
// ScaleDownDelaySecondsAfterScaleUp is the approximate delay for a scale down followed by a scale up
// Used to prevent flapping (down->up->down->... loop)
// +optional
ScaleDownDelaySecondsAfterScaleUp *int `json:"scaleDownDelaySecondsAfterScaleOut,omitempty"`
Template RunnerTemplate `json:"template"`
}
@@ -30,6 +44,14 @@ type RunnerDeploymentSpec struct {
type RunnerDeploymentStatus struct {
AvailableReplicas int `json:"availableReplicas"`
ReadyReplicas int `json:"readyReplicas"`
// Replicas is the total number of desired, non-terminated and latest pods to be set for the primary RunnerSet
// This doesn't include outdated pods while upgrading the deployment and replacing the runnerset.
// +optional
Replicas *int `json:"desiredReplicas,omitempty"`
// +optional
LastSuccessfulScaleOutTime *metav1.Time `json:"lastSuccessfulScaleOutTime,omitempty"`
}
// +kubebuilder:object:root=true

View File

@@ -0,0 +1,84 @@
/*
Copyright 2020 The actions-runner-controller authors.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package v1alpha1
import (
apierrors "k8s.io/apimachinery/pkg/api/errors"
"k8s.io/apimachinery/pkg/runtime"
"k8s.io/apimachinery/pkg/util/validation/field"
ctrl "sigs.k8s.io/controller-runtime"
logf "sigs.k8s.io/controller-runtime/pkg/runtime/log"
"sigs.k8s.io/controller-runtime/pkg/webhook"
)
// log is for logging in this package.
var runenrDeploymentLog = logf.Log.WithName("runnerdeployment-resource")
func (r *RunnerDeployment) SetupWebhookWithManager(mgr ctrl.Manager) error {
return ctrl.NewWebhookManagedBy(mgr).
For(r).
Complete()
}
// +kubebuilder:webhook:path=/mutate-actions-summerwind-dev-v1alpha1-runnerdeployment,verbs=create;update,mutating=true,failurePolicy=fail,groups=actions.summerwind.dev,resources=runnerdeployments,versions=v1alpha1,name=mutate.runnerdeployment.actions.summerwind.dev
var _ webhook.Defaulter = &RunnerDeployment{}
// Default implements webhook.Defaulter so a webhook will be registered for the type
func (r *RunnerDeployment) Default() {
// Nothing to do.
}
// +kubebuilder:webhook:path=/validate-actions-summerwind-dev-v1alpha1-runnerdeployment,verbs=create;update,mutating=false,failurePolicy=fail,groups=actions.summerwind.dev,resources=runnerdeployments,versions=v1alpha1,name=validate.runnerdeployment.actions.summerwind.dev
var _ webhook.Validator = &RunnerDeployment{}
// ValidateCreate implements webhook.Validator so a webhook will be registered for the type
func (r *RunnerDeployment) ValidateCreate() error {
runenrDeploymentLog.Info("validate resource to be created", "name", r.Name)
return r.Validate()
}
// ValidateUpdate implements webhook.Validator so a webhook will be registered for the type
func (r *RunnerDeployment) ValidateUpdate(old runtime.Object) error {
runenrDeploymentLog.Info("validate resource to be updated", "name", r.Name)
return r.Validate()
}
// ValidateDelete implements webhook.Validator so a webhook will be registered for the type
func (r *RunnerDeployment) ValidateDelete() error {
return nil
}
// Validate validates resource spec.
func (r *RunnerDeployment) Validate() error {
var (
errList field.ErrorList
err error
)
err = r.Spec.Template.Spec.ValidateRepository()
if err != nil {
errList = append(errList, field.Invalid(field.NewPath("spec", "template", "spec", "repository"), r.Spec.Template.Spec.Repository, err.Error()))
}
if len(errList) > 0 {
return apierrors.NewInvalid(r.GroupVersionKind().GroupKind(), r.Name, errList)
}
return nil
}

View File

@@ -0,0 +1,84 @@
/*
Copyright 2020 The actions-runner-controller authors.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package v1alpha1
import (
apierrors "k8s.io/apimachinery/pkg/api/errors"
"k8s.io/apimachinery/pkg/runtime"
"k8s.io/apimachinery/pkg/util/validation/field"
ctrl "sigs.k8s.io/controller-runtime"
logf "sigs.k8s.io/controller-runtime/pkg/runtime/log"
"sigs.k8s.io/controller-runtime/pkg/webhook"
)
// log is for logging in this package.
var runnerReplicaSetLog = logf.Log.WithName("runnerreplicaset-resource")
func (r *RunnerReplicaSet) SetupWebhookWithManager(mgr ctrl.Manager) error {
return ctrl.NewWebhookManagedBy(mgr).
For(r).
Complete()
}
// +kubebuilder:webhook:path=/mutate-actions-summerwind-dev-v1alpha1-runnerreplicaset,verbs=create;update,mutating=true,failurePolicy=fail,groups=actions.summerwind.dev,resources=runnerreplicasets,versions=v1alpha1,name=mutate.runnerreplicaset.actions.summerwind.dev
var _ webhook.Defaulter = &RunnerReplicaSet{}
// Default implements webhook.Defaulter so a webhook will be registered for the type
func (r *RunnerReplicaSet) Default() {
// Nothing to do.
}
// +kubebuilder:webhook:path=/validate-actions-summerwind-dev-v1alpha1-runnerreplicaset,verbs=create;update,mutating=false,failurePolicy=fail,groups=actions.summerwind.dev,resources=runnerreplicasets,versions=v1alpha1,name=validate.runnerreplicaset.actions.summerwind.dev
var _ webhook.Validator = &RunnerReplicaSet{}
// ValidateCreate implements webhook.Validator so a webhook will be registered for the type
func (r *RunnerReplicaSet) ValidateCreate() error {
runnerReplicaSetLog.Info("validate resource to be created", "name", r.Name)
return r.Validate()
}
// ValidateUpdate implements webhook.Validator so a webhook will be registered for the type
func (r *RunnerReplicaSet) ValidateUpdate(old runtime.Object) error {
runnerReplicaSetLog.Info("validate resource to be updated", "name", r.Name)
return r.Validate()
}
// ValidateDelete implements webhook.Validator so a webhook will be registered for the type
func (r *RunnerReplicaSet) ValidateDelete() error {
return nil
}
// Validate validates resource spec.
func (r *RunnerReplicaSet) Validate() error {
var (
errList field.ErrorList
err error
)
err = r.Spec.Template.Spec.ValidateRepository()
if err != nil {
errList = append(errList, field.Invalid(field.NewPath("spec", "template", "spec", "repository"), r.Spec.Template.Spec.Repository, err.Error()))
}
if len(errList) > 0 {
return apierrors.NewInvalid(r.GroupVersionKind().GroupKind(), r.Name, errList)
}
return nil
}

View File

@@ -22,7 +22,7 @@ package v1alpha1
import (
"k8s.io/api/core/v1"
runtime "k8s.io/apimachinery/pkg/runtime"
"k8s.io/apimachinery/pkg/runtime"
)
// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.
@@ -58,7 +58,7 @@ func (in *RunnerDeployment) DeepCopyInto(out *RunnerDeployment) {
out.TypeMeta = in.TypeMeta
in.ObjectMeta.DeepCopyInto(&out.ObjectMeta)
in.Spec.DeepCopyInto(&out.Spec)
out.Status = in.Status
in.Status.DeepCopyInto(&out.Status)
}
// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new RunnerDeployment.
@@ -119,6 +119,21 @@ func (in *RunnerDeploymentSpec) DeepCopyInto(out *RunnerDeploymentSpec) {
*out = new(int)
**out = **in
}
if in.MinReplicas != nil {
in, out := &in.MinReplicas, &out.MinReplicas
*out = new(int)
**out = **in
}
if in.MaxReplicas != nil {
in, out := &in.MaxReplicas, &out.MaxReplicas
*out = new(int)
**out = **in
}
if in.ScaleDownDelaySecondsAfterScaleUp != nil {
in, out := &in.ScaleDownDelaySecondsAfterScaleUp, &out.ScaleDownDelaySecondsAfterScaleUp
*out = new(int)
**out = **in
}
in.Template.DeepCopyInto(&out.Template)
}
@@ -135,6 +150,15 @@ func (in *RunnerDeploymentSpec) DeepCopy() *RunnerDeploymentSpec {
// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.
func (in *RunnerDeploymentStatus) DeepCopyInto(out *RunnerDeploymentStatus) {
*out = *in
if in.Replicas != nil {
in, out := &in.Replicas, &out.Replicas
*out = new(int)
**out = **in
}
if in.LastSuccessfulScaleOutTime != nil {
in, out := &in.LastSuccessfulScaleOutTime, &out.LastSuccessfulScaleOutTime
*out = (*in).DeepCopy()
}
}
// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new RunnerDeploymentStatus.
@@ -277,6 +301,11 @@ func (in *RunnerReplicaSetStatus) DeepCopy() *RunnerReplicaSetStatus {
// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.
func (in *RunnerSpec) DeepCopyInto(out *RunnerSpec) {
*out = *in
if in.Labels != nil {
in, out := &in.Labels, &out.Labels
*out = make([]string, len(*in))
copy(*out, *in)
}
if in.Containers != nil {
in, out := &in.Containers, &out.Containers
*out = make([]v1.Container, len(*in))
@@ -404,6 +433,11 @@ func (in *RunnerStatus) DeepCopy() *RunnerStatus {
// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.
func (in *RunnerStatusRegistration) DeepCopyInto(out *RunnerStatusRegistration) {
*out = *in
if in.Labels != nil {
in, out := &in.Labels, &out.Labels
*out = make([]string, len(*in))
copy(*out, *in)
}
in.ExpiresAt.DeepCopyInto(&out.ExpiresAt)
}

View File

@@ -46,8 +46,21 @@ spec:
spec:
description: RunnerReplicaSetSpec defines the desired state of RunnerDeployment
properties:
maxReplicas:
description: MinReplicas is the maximum number of replicas the deployment
is allowed to scale
type: integer
minReplicas:
description: MinReplicas is the minimum number of replicas the deployment
is allowed to scale
type: integer
replicas:
type: integer
scaleDownDelaySecondsAfterScaleOut:
description: ScaleDownDelaySecondsAfterScaleUp is the approximate delay
for a scale down followed by a scale up Used to prevent flapping (down->up->down->...
loop)
type: integer
template:
properties:
metadata:
@@ -4111,12 +4124,18 @@ spec:
- name
type: object
type: array
labels:
items:
type: string
type: array
nodeSelector:
additionalProperties:
type: string
type: object
organization:
pattern: ^[^/]+$
type: string
repository:
minLength: 3
pattern: ^[^/]+/[^/]+$
type: string
resources:
@@ -6708,18 +6727,23 @@ spec:
- name
type: object
type: array
required:
- repository
type: object
type: object
required:
- replicas
- template
type: object
status:
properties:
availableReplicas:
type: integer
desiredReplicas:
description: Replicas is the total number of desired, non-terminated
and latest pods to be set for the primary RunnerSet This doesn't include
outdated pods while upgrading the deployment and replacing the runnerset.
type: integer
lastSuccessfulScaleOutTime:
format: date-time
type: string
readyReplicas:
type: integer
required:

View File

@@ -4111,12 +4111,18 @@ spec:
- name
type: object
type: array
labels:
items:
type: string
type: array
nodeSelector:
additionalProperties:
type: string
type: object
organization:
pattern: ^[^/]+$
type: string
repository:
minLength: 3
pattern: ^[^/]+/[^/]+$
type: string
resources:
@@ -6708,8 +6714,6 @@ spec:
- name
type: object
type: array
required:
- repository
type: object
type: object
required:

View File

@@ -9,9 +9,15 @@ metadata:
name: runners.actions.summerwind.dev
spec:
additionalPrinterColumns:
- JSONPath: .spec.organization
name: Organization
type: string
- JSONPath: .spec.repository
name: Repository
type: string
- JSONPath: .spec.labels
name: Labels
type: string
- JSONPath: .status.phase
name: Status
type: string
@@ -3854,12 +3860,18 @@ spec:
- name
type: object
type: array
labels:
items:
type: string
type: array
nodeSelector:
additionalProperties:
type: string
type: object
organization:
pattern: ^[^/]+$
type: string
repository:
minLength: 3
pattern: ^[^/]+/[^/]+$
type: string
resources:
@@ -6292,8 +6304,6 @@ spec:
- name
type: object
type: array
required:
- repository
type: object
status:
description: RunnerStatus defines the observed state of Runner
@@ -6305,17 +6315,23 @@ spec:
reason:
type: string
registration:
description: RunnerStatusRegistration contains runner registration status
properties:
expiresAt:
format: date-time
type: string
labels:
items:
type: string
type: array
organization:
type: string
repository:
type: string
token:
type: string
required:
- expiresAt
- repository
- token
type: object
required:

View File

@@ -17,9 +17,9 @@ bases:
- ../rbac
- ../manager
# [WEBHOOK] To enable webhook, uncomment all the sections with [WEBHOOK] prefix including the one in crd/kustomization.yaml
#- ../webhook
- ../webhook
# [CERTMANAGER] To enable cert-manager, uncomment all sections with 'CERTMANAGER'. 'WEBHOOK' components are required.
#- ../certmanager
- ../certmanager
# [PROMETHEUS] To enable prometheus monitor, uncomment all sections with 'PROMETHEUS'.
#- ../prometheus
@@ -36,39 +36,39 @@ patchesStrategicMerge:
#- manager_prometheus_metrics_patch.yaml
# [WEBHOOK] To enable webhook, uncomment all the sections with [WEBHOOK] prefix including the one in crd/kustomization.yaml
#- manager_webhook_patch.yaml
- manager_webhook_patch.yaml
# [CERTMANAGER] To enable cert-manager, uncomment all sections with 'CERTMANAGER'.
# Uncomment 'CERTMANAGER' sections in crd/kustomization.yaml to enable the CA injection in the admission webhooks.
# 'CERTMANAGER' needs to be enabled to use ca injection
#- webhookcainjection_patch.yaml
- webhookcainjection_patch.yaml
# the following config is for teaching kustomize how to do var substitution
vars:
# [CERTMANAGER] To enable cert-manager, uncomment all sections with 'CERTMANAGER' prefix.
#- name: CERTIFICATE_NAMESPACE # namespace of the certificate CR
# objref:
# kind: Certificate
# group: cert-manager.io
# version: v1alpha2
# name: serving-cert # this name should match the one in certificate.yaml
# fieldref:
# fieldpath: metadata.namespace
#- name: CERTIFICATE_NAME
# objref:
# kind: Certificate
# group: cert-manager.io
# version: v1alpha2
# name: serving-cert # this name should match the one in certificate.yaml
#- name: SERVICE_NAMESPACE # namespace of the service
# objref:
# kind: Service
# version: v1
# name: webhook-service
# fieldref:
# fieldpath: metadata.namespace
#- name: SERVICE_NAME
# objref:
# kind: Service
# version: v1
# name: webhook-service
- name: CERTIFICATE_NAMESPACE # namespace of the certificate CR
objref:
kind: Certificate
group: cert-manager.io
version: v1alpha2
name: serving-cert # this name should match the one in certificate.yaml
fieldref:
fieldpath: metadata.namespace
- name: CERTIFICATE_NAME
objref:
kind: Certificate
group: cert-manager.io
version: v1alpha2
name: serving-cert # this name should match the one in certificate.yaml
- name: SERVICE_NAMESPACE # namespace of the service
objref:
kind: Service
version: v1
name: webhook-service
fieldref:
fieldpath: metadata.namespace
- name: SERVICE_NAME
objref:
kind: Service
version: v1
name: webhook-service

View File

@@ -0,0 +1,124 @@
---
apiVersion: admissionregistration.k8s.io/v1beta1
kind: MutatingWebhookConfiguration
metadata:
creationTimestamp: null
name: mutating-webhook-configuration
webhooks:
- clientConfig:
caBundle: Cg==
service:
name: webhook-service
namespace: system
path: /mutate-actions-summerwind-dev-v1alpha1-runner
failurePolicy: Fail
name: mutate.runner.actions.summerwind.dev
rules:
- apiGroups:
- actions.summerwind.dev
apiVersions:
- v1alpha1
operations:
- CREATE
- UPDATE
resources:
- runners
- clientConfig:
caBundle: Cg==
service:
name: webhook-service
namespace: system
path: /mutate-actions-summerwind-dev-v1alpha1-runnerdeployment
failurePolicy: Fail
name: mutate.runnerdeployment.actions.summerwind.dev
rules:
- apiGroups:
- actions.summerwind.dev
apiVersions:
- v1alpha1
operations:
- CREATE
- UPDATE
resources:
- runnerdeployments
- clientConfig:
caBundle: Cg==
service:
name: webhook-service
namespace: system
path: /mutate-actions-summerwind-dev-v1alpha1-runnerreplicaset
failurePolicy: Fail
name: mutate.runnerreplicaset.actions.summerwind.dev
rules:
- apiGroups:
- actions.summerwind.dev
apiVersions:
- v1alpha1
operations:
- CREATE
- UPDATE
resources:
- runnerreplicasets
---
apiVersion: admissionregistration.k8s.io/v1beta1
kind: ValidatingWebhookConfiguration
metadata:
creationTimestamp: null
name: validating-webhook-configuration
webhooks:
- clientConfig:
caBundle: Cg==
service:
name: webhook-service
namespace: system
path: /validate-actions-summerwind-dev-v1alpha1-runner
failurePolicy: Fail
name: validate.runner.actions.summerwind.dev
rules:
- apiGroups:
- actions.summerwind.dev
apiVersions:
- v1alpha1
operations:
- CREATE
- UPDATE
resources:
- runners
- clientConfig:
caBundle: Cg==
service:
name: webhook-service
namespace: system
path: /validate-actions-summerwind-dev-v1alpha1-runnerdeployment
failurePolicy: Fail
name: validate.runnerdeployment.actions.summerwind.dev
rules:
- apiGroups:
- actions.summerwind.dev
apiVersions:
- v1alpha1
operations:
- CREATE
- UPDATE
resources:
- runnerdeployments
- clientConfig:
caBundle: Cg==
service:
name: webhook-service
namespace: system
path: /validate-actions-summerwind-dev-v1alpha1-runnerreplicaset
failurePolicy: Fail
name: validate.runnerreplicaset.actions.summerwind.dev
rules:
- apiGroups:
- actions.summerwind.dev
apiVersions:
- v1alpha1
operations:
- CREATE
- UPDATE
resources:
- runnerreplicasets

View File

@@ -0,0 +1,92 @@
package controllers
import (
"context"
"fmt"
"github.com/summerwind/actions-runner-controller/api/v1alpha1"
"strings"
)
type NotSupported struct {
}
var _ error = NotSupported{}
func (e NotSupported) Error() string {
return "Autoscaling is currently supported only when spec.repository is set"
}
func (r *RunnerDeploymentReconciler) determineDesiredReplicas(rd v1alpha1.RunnerDeployment) (*int, error) {
if rd.Spec.Replicas != nil {
return nil, fmt.Errorf("bug: determineDesiredReplicas should not be called for deplomeny with specific replicas")
} else if rd.Spec.MinReplicas == nil {
return nil, fmt.Errorf("runnerdeployment %s/%s is missing minReplicas", rd.Namespace, rd.Name)
} else if rd.Spec.MaxReplicas == nil {
return nil, fmt.Errorf("runnerdeployment %s/%s is missing maxReplicas", rd.Namespace, rd.Name)
}
var replicas int
repoID := rd.Spec.Template.Spec.Repository
if repoID == "" {
return nil, NotSupported{}
}
repo := strings.Split(repoID, "/")
user, repoName := repo[0], repo[1]
list, _, err := r.GitHubClient.Actions.ListRepositoryWorkflowRuns(context.TODO(), user, repoName, nil)
if err != nil {
return nil, err
}
var total, inProgress, queued, completed, unknown int
for _, r := range list.WorkflowRuns {
total++
// In May 2020, there are only 3 statuses.
// Follow the below links for more details:
// - https://developer.github.com/v3/actions/workflow-runs/#list-repository-workflow-runs
// - https://developer.github.com/v3/checks/runs/#create-a-check-run
switch r.GetStatus() {
case "completed":
completed++
case "in_progress":
inProgress++
case "queued":
queued++
default:
unknown++
}
}
minReplicas := *rd.Spec.MinReplicas
maxReplicas := *rd.Spec.MaxReplicas
necessaryReplicas := queued + inProgress
var desiredReplicas int
if necessaryReplicas < minReplicas {
desiredReplicas = minReplicas
} else if necessaryReplicas > maxReplicas {
desiredReplicas = maxReplicas
} else {
desiredReplicas = necessaryReplicas
}
rd.Status.Replicas = &desiredReplicas
replicas = desiredReplicas
r.Log.V(1).Info(
"Calculated desired replicas",
"computed_replicas_desired", desiredReplicas,
"spec_replicas_min", minReplicas,
"spec_replicas_max", maxReplicas,
"workflow_runs_completed", completed,
"workflow_runs_in_progress", inProgress,
"workflow_runs_queued", queued,
"workflow_runs_unknown", unknown,
)
return &replicas, nil
}

View File

@@ -0,0 +1,199 @@
package controllers
import (
"fmt"
"github.com/summerwind/actions-runner-controller/api/v1alpha1"
"github.com/summerwind/actions-runner-controller/github"
"github.com/summerwind/actions-runner-controller/github/fake"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/apimachinery/pkg/runtime"
clientgoscheme "k8s.io/client-go/kubernetes/scheme"
"net/http/httptest"
"net/url"
"sigs.k8s.io/controller-runtime/pkg/log/zap"
"testing"
)
func newGithubClient(server *httptest.Server) *github.Client {
client, err := github.NewClientWithAccessToken("token")
if err != nil {
panic(err)
}
baseURL, err := url.Parse(server.URL + "/")
if err != nil {
panic(err)
}
client.Client.BaseURL = baseURL
return client
}
func TestDetermineDesiredReplicas_RepositoryRunner(t *testing.T) {
intPtr := func(v int) *int {
return &v
}
metav1Now := metav1.Now()
testcases := []struct {
repo string
org string
fixed *int
max *int
min *int
sReplicas *int
sTime *metav1.Time
workflowRuns string
want int
err string
}{
// 3 demanded, max at 3
{
repo: "test/valid",
min: intPtr(2),
max: intPtr(3),
workflowRuns: `{"total_count": 4, "workflow_runs":[{"status":"queued"}, {"status":"in_progress"}, {"status":"in_progress"}, {"status":"completed"}]}"`,
want: 3,
},
// 2 demanded, max at 3, currently 3, delay scaling down due to grace period
{
repo: "test/valid",
min: intPtr(2),
max: intPtr(3),
sReplicas: intPtr(3),
sTime: &metav1Now,
workflowRuns: `{"total_count": 4, "workflow_runs":[{"status":"queued"}, {"status":"in_progress"}, {"status":"completed"}]}"`,
want: 3,
},
// 3 demanded, max at 2
{
repo: "test/valid",
min: intPtr(2),
max: intPtr(2),
workflowRuns: `{"total_count": 4, "workflow_runs":[{"status":"queued"}, {"status":"in_progress"}, {"status":"in_progress"}, {"status":"completed"}]}"`,
want: 2,
},
// 2 demanded, min at 2
{
repo: "test/valid",
min: intPtr(2),
max: intPtr(3),
workflowRuns: `{"total_count": 3, "workflow_runs":[{"status":"queued"}, {"status":"in_progress"}, {"status":"completed"}]}"`,
want: 2,
},
// 1 demanded, min at 2
{
repo: "test/valid",
min: intPtr(2),
max: intPtr(3),
workflowRuns: `{"total_count": 2, "workflow_runs":[{"status":"queued"}, {"status":"completed"}]}"`,
want: 2,
},
// 1 demanded, min at 2
{
repo: "test/valid",
min: intPtr(2),
max: intPtr(3),
workflowRuns: `{"total_count": 2, "workflow_runs":[{"status":"in_progress"}, {"status":"completed"}]}"`,
want: 2,
},
// 1 demanded, min at 1
{
repo: "test/valid",
min: intPtr(1),
max: intPtr(3),
workflowRuns: `{"total_count": 2, "workflow_runs":[{"status":"queued"}, {"status":"completed"}]}"`,
want: 1,
},
// 1 demanded, min at 1
{
repo: "test/valid",
min: intPtr(1),
max: intPtr(3),
workflowRuns: `{"total_count": 2, "workflow_runs":[{"status":"in_progress"}, {"status":"completed"}]}"`,
want: 1,
},
// fixed at 3
{
repo: "test/valid",
fixed: intPtr(3),
want: 3,
},
// org runner, fixed at 3
{
org: "test",
fixed: intPtr(3),
want: 3,
},
// org runner, 1 demanded, min at 1
{
org: "test",
min: intPtr(1),
max: intPtr(3),
workflowRuns: `{"total_count": 2, "workflow_runs":[{"status":"in_progress"}, {"status":"completed"}]}"`,
err: "Autoscaling is currently supported only when spec.repository is set",
},
}
for i := range testcases {
tc := testcases[i]
log := zap.New(func(o *zap.Options) {
o.Development = true
})
scheme := runtime.NewScheme()
_ = clientgoscheme.AddToScheme(scheme)
_ = v1alpha1.AddToScheme(scheme)
t.Run(fmt.Sprintf("case %d", i), func(t *testing.T) {
server := fake.NewServer(fake.WithListRepositoryWorkflowRunsResponse(200, tc.workflowRuns))
defer server.Close()
client := newGithubClient(server)
r := &RunnerDeploymentReconciler{
Log: log,
GitHubClient: client,
Scheme: scheme,
}
rd := v1alpha1.RunnerDeployment{
TypeMeta: metav1.TypeMeta{},
Spec: v1alpha1.RunnerDeploymentSpec{
Template: v1alpha1.RunnerTemplate{
Spec: v1alpha1.RunnerSpec{
Repository: tc.repo,
},
},
Replicas: tc.fixed,
MaxReplicas: tc.max,
MinReplicas: tc.min,
},
Status: v1alpha1.RunnerDeploymentStatus{
Replicas: tc.sReplicas,
LastSuccessfulScaleOutTime: tc.sTime,
},
}
rs, err := r.newRunnerReplicaSetWithAutoscaling(rd)
if err != nil {
if tc.err == "" {
t.Fatalf("unexpected error: expected none, got %v", err)
} else if err.Error() != tc.err {
t.Fatalf("unexpected error: expected %v, got %v", tc.err, err)
}
return
}
got := rs.Spec.Replicas
if got == nil {
t.Fatalf("unexpected value of rs.Spec.Replicas: nil")
}
if *got != tc.want {
t.Errorf("%d: incorrect desired replicas: want %d, got %d", i, tc.want, *got)
}
})
}
}

View File

@@ -20,6 +20,7 @@ import (
"context"
"fmt"
"reflect"
"strings"
"github.com/go-logr/logr"
"k8s.io/apimachinery/pkg/api/errors"
@@ -65,6 +66,12 @@ func (r *RunnerReconciler) Reconcile(req ctrl.Request) (ctrl.Result, error) {
return ctrl.Result{}, client.IgnoreNotFound(err)
}
err := runner.Validate()
if err != nil {
log.Info("Failed to validate runner spec", "error", err.Error())
return ctrl.Result{}, nil
}
if runner.ObjectMeta.DeletionTimestamp.IsZero() {
finalizers, added := addFinalizer(runner.ObjectMeta.Finalizers)
@@ -83,14 +90,18 @@ func (r *RunnerReconciler) Reconcile(req ctrl.Request) (ctrl.Result, error) {
finalizers, removed := removeFinalizer(runner.ObjectMeta.Finalizers)
if removed {
ok, err := r.unregisterRunner(ctx, runner.Spec.Repository, runner.Name)
if err != nil {
log.Error(err, "Failed to unregister runner")
return ctrl.Result{}, err
}
if len(runner.Status.Registration.Token) > 0 {
ok, err := r.unregisterRunner(ctx, runner.Spec.Organization, runner.Spec.Repository, runner.Name)
if err != nil {
log.Error(err, "Failed to unregister runner")
return ctrl.Result{}, err
}
if !ok {
log.V(1).Info("Runner no longer exists on GitHub")
if !ok {
log.V(1).Info("Runner no longer exists on GitHub")
}
} else {
log.V(1).Info("Runner was never registered on GitHub")
}
newRunner := runner.DeepCopy()
@@ -101,14 +112,14 @@ func (r *RunnerReconciler) Reconcile(req ctrl.Request) (ctrl.Result, error) {
return ctrl.Result{}, err
}
log.Info("Removed runner from GitHub", "repository", runner.Spec.Repository)
log.Info("Removed runner from GitHub", "repository", runner.Spec.Repository, "organization", runner.Spec.Organization)
}
return ctrl.Result{}, nil
}
if !runner.IsRegisterable() {
rt, err := r.GitHubClient.GetRegistrationToken(ctx, runner.Spec.Repository, runner.Name)
rt, err := r.GitHubClient.GetRegistrationToken(ctx, runner.Spec.Organization, runner.Spec.Repository, runner.Name)
if err != nil {
r.Recorder.Event(&runner, corev1.EventTypeWarning, "FailedUpdateRegistrationToken", "Updating registration token failed")
log.Error(err, "Failed to get new registration token")
@@ -117,9 +128,11 @@ func (r *RunnerReconciler) Reconcile(req ctrl.Request) (ctrl.Result, error) {
updated := runner.DeepCopy()
updated.Status.Registration = v1alpha1.RunnerStatusRegistration{
Repository: runner.Spec.Repository,
Token: rt.GetToken(),
ExpiresAt: metav1.NewTime(rt.GetExpiresAt().Time),
Organization: runner.Spec.Organization,
Repository: runner.Spec.Repository,
Labels: runner.Spec.Labels,
Token: rt.GetToken(),
ExpiresAt: metav1.NewTime(rt.GetExpiresAt().Time),
}
if err := r.Status().Update(ctx, updated); err != nil {
@@ -212,8 +225,8 @@ func (r *RunnerReconciler) Reconcile(req ctrl.Request) (ctrl.Result, error) {
return ctrl.Result{}, nil
}
func (r *RunnerReconciler) unregisterRunner(ctx context.Context, repo, name string) (bool, error) {
runners, err := r.GitHubClient.ListRunners(ctx, repo)
func (r *RunnerReconciler) unregisterRunner(ctx context.Context, org, repo, name string) (bool, error) {
runners, err := r.GitHubClient.ListRunners(ctx, org, repo)
if err != nil {
return false, err
}
@@ -230,7 +243,7 @@ func (r *RunnerReconciler) unregisterRunner(ctx context.Context, repo, name stri
return false, nil
}
if err := r.GitHubClient.RemoveRunner(ctx, repo, id); err != nil {
if err := r.GitHubClient.RemoveRunner(ctx, org, repo, id); err != nil {
return false, err
}
@@ -253,10 +266,18 @@ func (r *RunnerReconciler) newPod(runner v1alpha1.Runner) (corev1.Pod, error) {
Name: "RUNNER_NAME",
Value: runner.Name,
},
{
Name: "RUNNER_ORG",
Value: runner.Spec.Organization,
},
{
Name: "RUNNER_REPO",
Value: runner.Spec.Repository,
},
{
Name: "RUNNER_LABELS",
Value: strings.Join(runner.Spec.Labels, ","),
},
{
Name: "RUNNER_TOKEN",
Value: runner.Status.Registration.Token,
@@ -281,6 +302,10 @@ func (r *RunnerReconciler) newPod(runner v1alpha1.Runner) (corev1.Pod, error) {
Env: env,
EnvFrom: runner.Spec.EnvFrom,
VolumeMounts: []corev1.VolumeMount{
{
Name: "work",
MountPath: "/runner/_work",
},
{
Name: "docker",
MountPath: "/var/run",
@@ -295,6 +320,10 @@ func (r *RunnerReconciler) newPod(runner v1alpha1.Runner) (corev1.Pod, error) {
Name: "docker",
Image: r.DockerImage,
VolumeMounts: []corev1.VolumeMount{
{
Name: "work",
MountPath: "/runner/_work",
},
{
Name: "docker",
MountPath: "/var/run",
@@ -306,7 +335,13 @@ func (r *RunnerReconciler) newPod(runner v1alpha1.Runner) (corev1.Pod, error) {
},
},
Volumes: []corev1.Volume{
corev1.Volume{
{
Name: "work",
VolumeSource: corev1.VolumeSource{
EmptyDir: &corev1.EmptyDirVolumeSource{},
},
},
{
Name: "docker",
VolumeSource: corev1.VolumeSource{
EmptyDir: &corev1.EmptyDirVolumeSource{},
@@ -325,7 +360,7 @@ func (r *RunnerReconciler) newPod(runner v1alpha1.Runner) (corev1.Pod, error) {
}
if len(runner.Spec.Volumes) != 0 {
pod.Spec.Volumes = append(runner.Spec.Volumes, runner.Spec.Volumes...)
pod.Spec.Volumes = append(pod.Spec.Volumes, runner.Spec.Volumes...)
}
if len(runner.Spec.InitContainers) != 0 {
pod.Spec.InitContainers = append(pod.Spec.InitContainers, runner.Spec.InitContainers...)

View File

@@ -20,10 +20,12 @@ import (
"context"
"fmt"
"hash/fnv"
"k8s.io/apimachinery/pkg/types"
"sort"
"time"
"github.com/summerwind/actions-runner-controller/github"
"k8s.io/apimachinery/pkg/types"
"github.com/davecgh/go-spew/spew"
"github.com/go-logr/logr"
"k8s.io/apimachinery/pkg/runtime"
@@ -47,9 +49,10 @@ const (
// RunnerDeploymentReconciler reconciles a Runner object
type RunnerDeploymentReconciler struct {
client.Client
Log logr.Logger
Recorder record.EventRecorder
Scheme *runtime.Scheme
GitHubClient *github.Client
Log logr.Logger
Recorder record.EventRecorder
Scheme *runtime.Scheme
}
// +kubebuilder:rbac:groups=actions.summerwind.dev,resources=runnerdeployments,verbs=get;list;watch;create;update;patch;delete
@@ -94,15 +97,19 @@ func (r *RunnerDeploymentReconciler) Reconcile(req ctrl.Request) (ctrl.Result, e
oldSets = myRunnerReplicaSets[1:]
}
desiredRS, err := r.newRunnerReplicaSet(rd)
desiredRS, err := r.newRunnerReplicaSetWithAutoscaling(rd)
if err != nil {
if _, ok := err.(NotSupported); ok {
r.Recorder.Event(&rd, corev1.EventTypeNormal, "RunnerReplicaSetAutoScaleNotSupported", err.Error())
}
log.Error(err, "Could not create runnerreplicaset")
return ctrl.Result{}, err
}
if newestSet == nil {
if err := r.Client.Create(ctx, &desiredRS); err != nil {
if err := r.Client.Create(ctx, desiredRS); err != nil {
log.Error(err, "Failed to create runnerreplicaset resource")
return ctrl.Result{}, err
@@ -118,7 +125,7 @@ func (r *RunnerDeploymentReconciler) Reconcile(req ctrl.Request) (ctrl.Result, e
return ctrl.Result{}, nil
}
desiredTemplateHash, ok := getTemplateHash(&desiredRS)
desiredTemplateHash, ok := getTemplateHash(desiredRS)
if !ok {
log.Info("Failed to get template hash of desired runnerreplicaset resource. It must be in an invalid state. Please manually delete the runnerreplicaset so that it is recreated")
@@ -126,7 +133,7 @@ func (r *RunnerDeploymentReconciler) Reconcile(req ctrl.Request) (ctrl.Result, e
}
if newestTemplateHash != desiredTemplateHash {
if err := r.Client.Create(ctx, &desiredRS); err != nil {
if err := r.Client.Create(ctx, desiredRS); err != nil {
log.Error(err, "Failed to create runnerreplicaset resource")
return ctrl.Result{}, err
@@ -184,6 +191,23 @@ func (r *RunnerDeploymentReconciler) Reconcile(req ctrl.Request) (ctrl.Result, e
}
}
if rd.Spec.Replicas == nil && desiredRS.Spec.Replicas != nil {
updated := rd.DeepCopy()
updated.Status.Replicas = desiredRS.Spec.Replicas
if (rd.Status.Replicas == nil && *desiredRS.Spec.Replicas > 1) ||
(rd.Status.Replicas != nil && *desiredRS.Spec.Replicas > *rd.Status.Replicas) {
updated.Status.LastSuccessfulScaleOutTime = &metav1.Time{Time: time.Now()}
}
if err := r.Status().Update(ctx, updated); err != nil {
log.Error(err, "Failed to update runnerdeployment status")
return ctrl.Result{}, err
}
}
return ctrl.Result{}, nil
}
@@ -241,7 +265,7 @@ func CloneAndAddLabel(labels map[string]string, labelKey, labelValue string) map
return newLabels
}
func (r *RunnerDeploymentReconciler) newRunnerReplicaSet(rd v1alpha1.RunnerDeployment) (v1alpha1.RunnerReplicaSet, error) {
func (r *RunnerDeploymentReconciler) newRunnerReplicaSet(rd v1alpha1.RunnerDeployment, computedReplicas *int) (*v1alpha1.RunnerReplicaSet, error) {
newRSTemplate := *rd.Spec.Template.DeepCopy()
templateHash := ComputeHash(&newRSTemplate)
// Add template hash label to selector.
@@ -262,11 +286,15 @@ func (r *RunnerDeploymentReconciler) newRunnerReplicaSet(rd v1alpha1.RunnerDeplo
},
}
if err := ctrl.SetControllerReference(&rd, &rs, r.Scheme); err != nil {
return rs, err
if computedReplicas != nil {
rs.Spec.Replicas = computedReplicas
}
return rs, nil
if err := ctrl.SetControllerReference(&rd, &rs, r.Scheme); err != nil {
return &rs, err
}
return &rs, nil
}
func (r *RunnerDeploymentReconciler) SetupWithManager(mgr ctrl.Manager) error {
@@ -293,3 +321,36 @@ func (r *RunnerDeploymentReconciler) SetupWithManager(mgr ctrl.Manager) error {
Owns(&v1alpha1.RunnerReplicaSet{}).
Complete(r)
}
func (r *RunnerDeploymentReconciler) newRunnerReplicaSetWithAutoscaling(rd v1alpha1.RunnerDeployment) (*v1alpha1.RunnerReplicaSet, error) {
var computedReplicas *int
if rd.Spec.Replicas == nil {
replicas, err := r.determineDesiredReplicas(rd)
if err != nil {
return nil, err
}
var scaleDownDelay time.Duration
if rd.Spec.ScaleDownDelaySecondsAfterScaleUp != nil {
scaleDownDelay = time.Duration(*rd.Spec.ScaleDownDelaySecondsAfterScaleUp) * time.Second
} else {
scaleDownDelay = 10 * time.Minute
}
now := time.Now()
if rd.Status.Replicas == nil ||
*rd.Status.Replicas < *replicas ||
rd.Status.LastSuccessfulScaleOutTime == nil ||
rd.Status.LastSuccessfulScaleOutTime.Add(scaleDownDelay).Before(now) {
computedReplicas = replicas
} else {
computedReplicas = rd.Status.Replicas
}
}
return r.newRunnerReplicaSet(rd, computedReplicas)
}

View File

@@ -31,7 +31,14 @@ func (h *handler) ServeHTTP(w http.ResponseWriter, req *http.Request) {
fmt.Fprintf(w, h.Body)
}
func NewServer() *httptest.Server {
// NewServer creates a fake server for running unit tests
func NewServer(opts ...Option) *httptest.Server {
var responses FixedResponses
for _, o := range opts {
o(&responses)
}
routes := map[string]handler{
// For CreateRegistrationToken
"/repos/test/valid/actions/runners/registration-token": handler{
@@ -46,6 +53,18 @@ func NewServer() *httptest.Server {
Status: http.StatusBadRequest,
Body: "",
},
"/orgs/test/actions/runners/registration-token": handler{
Status: http.StatusCreated,
Body: fmt.Sprintf("{\"token\": \"%s\", \"expires_at\": \"%s\"}", RegistrationToken, time.Now().Add(time.Hour*1).Format(time.RFC3339)),
},
"/orgs/invalid/actions/runners/registration-token": handler{
Status: http.StatusOK,
Body: fmt.Sprintf("{\"token\": \"%s\", \"expires_at\": \"%s\"}", RegistrationToken, time.Now().Add(time.Hour*1).Format(time.RFC3339)),
},
"/orgs/error/actions/runners/registration-token": handler{
Status: http.StatusBadRequest,
Body: "",
},
// For ListRunners
"/repos/test/valid/actions/runners": handler{
@@ -60,6 +79,18 @@ func NewServer() *httptest.Server {
Status: http.StatusBadRequest,
Body: "",
},
"/orgs/test/actions/runners": handler{
Status: http.StatusOK,
Body: RunnersListBody,
},
"/orgs/invalid/actions/runners": handler{
Status: http.StatusNoContent,
Body: "",
},
"/orgs/error/actions/runners": handler{
Status: http.StatusBadRequest,
Body: "",
},
// For RemoveRunner
"/repos/test/valid/actions/runners/1": handler{
@@ -74,6 +105,21 @@ func NewServer() *httptest.Server {
Status: http.StatusBadRequest,
Body: "",
},
"/orgs/test/actions/runners/1": handler{
Status: http.StatusNoContent,
Body: "",
},
"/orgs/invalid/actions/runners/1": handler{
Status: http.StatusOK,
Body: "",
},
"/orgs/error/actions/runners/1": handler{
Status: http.StatusBadRequest,
Body: "",
},
// For auto-scaling based on the number of queued(pending) workflow runs
"/repos/test/valid/actions/runs": responses.listRepositoryWorkflowRuns.handler(),
}
mux := http.NewServeMux()

28
github/fake/options.go Normal file
View File

@@ -0,0 +1,28 @@
package fake
type FixedResponses struct {
listRepositoryWorkflowRuns FixedResponse
}
type FixedResponse struct {
Status int
Body string
}
func (r FixedResponse) handler() handler {
return handler{
Status: r.Status,
Body: r.Body,
}
}
type Option func(responses *FixedResponses)
func WithListRepositoryWorkflowRunsResponse(status int, body string) Option {
return func(r *FixedResponses) {
r.listRepositoryWorkflowRuns = FixedResponse{
Status: status,
Body: body,
}
}
}

View File

@@ -2,7 +2,6 @@ package github
import (
"context"
"errors"
"fmt"
"net/http"
"strings"
@@ -14,6 +13,7 @@ import (
"golang.org/x/oauth2"
)
// Client wraps GitHub client with some additional
type Client struct {
*github.Client
regTokens map[string]*github.RegistrationToken
@@ -27,14 +27,16 @@ func NewClient(appID, installationID int64, privateKeyPath string) (*Client, err
return nil, fmt.Errorf("authentication failed: %v", err)
}
gh := github.NewClient(&http.Client{Transport: tr})
return &Client{
Client: github.NewClient(&http.Client{Transport: tr}),
Client: gh,
regTokens: map[string]*github.RegistrationToken{},
mu: sync.Mutex{},
}, nil
}
// NewClient returns a client authenticated with personal access token.
// NewClientWithAccessToken returns a client authenticated with personal access token.
func NewClientWithAccessToken(token string) (*Client, error) {
tc := oauth2.NewClient(context.Background(), oauth2.StaticTokenSource(
&oauth2.Token{AccessToken: token},
@@ -48,22 +50,25 @@ func NewClientWithAccessToken(token string) (*Client, error) {
}
// GetRegistrationToken returns a registration token tied with the name of repository and runner.
func (c *Client) GetRegistrationToken(ctx context.Context, repository, name string) (*github.RegistrationToken, error) {
func (c *Client) GetRegistrationToken(ctx context.Context, org, repo, name string) (*github.RegistrationToken, error) {
c.mu.Lock()
defer c.mu.Unlock()
owner, repo, err := splitOwnerAndRepo(repository)
if err != nil {
return nil, err
}
key := fmt.Sprintf("%s/%s", repo, name)
key := getRegistrationKey(org, repo)
rt, ok := c.regTokens[key]
if ok && rt.GetExpiresAt().After(time.Now().Add(-10*time.Minute)) {
return rt, nil
}
rt, res, err := c.Client.Actions.CreateRegistrationToken(ctx, owner, repo)
owner, repo, err := getOwnerAndRepo(org, repo)
if err != nil {
return rt, err
}
rt, res, err := c.createRegistrationToken(ctx, owner, repo)
if err != nil {
return nil, fmt.Errorf("failed to create registration token: %v", err)
}
@@ -81,13 +86,15 @@ func (c *Client) GetRegistrationToken(ctx context.Context, repository, name stri
}
// RemoveRunner removes a runner with specified runner ID from repocitory.
func (c *Client) RemoveRunner(ctx context.Context, repository string, runnerID int64) error {
owner, repo, err := splitOwnerAndRepo(repository)
func (c *Client) RemoveRunner(ctx context.Context, org, repo string, runnerID int64) error {
owner, repo, err := getOwnerAndRepo(org, repo)
if err != nil {
return err
}
res, err := c.Client.Actions.RemoveRunner(ctx, owner, repo, runnerID)
res, err := c.removeRunner(ctx, owner, repo, runnerID)
if err != nil {
return fmt.Errorf("failed to remove runner: %v", err)
}
@@ -99,18 +106,20 @@ func (c *Client) RemoveRunner(ctx context.Context, repository string, runnerID i
return nil
}
// ListRunners returns a list of runners of specified repository name.
func (c *Client) ListRunners(ctx context.Context, repository string) ([]*github.Runner, error) {
var runners []*github.Runner
// ListRunners returns a list of runners of specified owner/repository name.
func (c *Client) ListRunners(ctx context.Context, org, repo string) ([]*github.Runner, error) {
owner, repo, err := getOwnerAndRepo(org, repo)
owner, repo, err := splitOwnerAndRepo(repository)
if err != nil {
return runners, err
return nil, err
}
var runners []*github.Runner
opts := github.ListOptions{PerPage: 10}
for {
list, res, err := c.Client.Actions.ListRunners(ctx, owner, repo, &opts)
list, res, err := c.listRunners(ctx, owner, repo, &opts)
if err != nil {
return runners, fmt.Errorf("failed to remove runner: %v", err)
}
@@ -137,11 +146,56 @@ func (c *Client) cleanup() {
}
}
// splitOwnerAndRepo splits specified repository name to the owner and repo name.
// wrappers for github functions (switch between organization/repository mode)
// so the calling functions don't need to switch and their code is a bit cleaner
func (c *Client) createRegistrationToken(ctx context.Context, owner, repo string) (*github.RegistrationToken, *github.Response, error) {
if len(repo) > 0 {
return c.Client.Actions.CreateRegistrationToken(ctx, owner, repo)
} else {
return CreateOrganizationRegistrationToken(ctx, c, owner)
}
}
func (c *Client) removeRunner(ctx context.Context, owner, repo string, runnerID int64) (*github.Response, error) {
if len(repo) > 0 {
return c.Client.Actions.RemoveRunner(ctx, owner, repo, runnerID)
} else {
return RemoveOrganizationRunner(ctx, c, owner, runnerID)
}
}
func (c *Client) listRunners(ctx context.Context, owner, repo string, opts *github.ListOptions) (*github.Runners, *github.Response, error) {
if len(repo) > 0 {
return c.Client.Actions.ListRunners(ctx, owner, repo, opts)
} else {
return ListOrganizationRunners(ctx, c, owner, opts)
}
}
// Validates owner and repo arguments. Both are optional, but at least one should be specified
func getOwnerAndRepo(org, repo string) (string, string, error) {
if len(repo) > 0 {
return splitOwnerAndRepo(repo)
}
if len(org) > 0 {
return org, "", nil
}
return "", "", fmt.Errorf("organization and repository are both empty")
}
func getRegistrationKey(org, repo string) string {
if len(org) > 0 {
return org
} else {
return repo
}
}
func splitOwnerAndRepo(repo string) (string, string, error) {
chunk := strings.Split(repo, "/")
if len(chunk) != 2 {
return "", "", errors.New("invalid repository name")
return "", "", fmt.Errorf("invalid repository name: '%s'", repo)
}
return chunk[0], chunk[1], nil
}

95
github/github_beta.go Normal file
View File

@@ -0,0 +1,95 @@
package github
// this contains BETA API clients, that are currently not (yet) in go-github
// once these functions have been added there, they can be removed from here
// code was reused from https://github.com/google/go-github
import (
"context"
"fmt"
"net/url"
"reflect"
"github.com/google/go-github/v31/github"
"github.com/google/go-querystring/query"
)
// CreateOrganizationRegistrationToken creates a token that can be used to add a self-hosted runner on an organization.
//
// GitHub API docs: https://developer.github.com/v3/actions/self-hosted-runners/#create-a-registration-token-for-an-organization
func CreateOrganizationRegistrationToken(ctx context.Context, client *Client, owner string) (*github.RegistrationToken, *github.Response, error) {
u := fmt.Sprintf("orgs/%v/actions/runners/registration-token", owner)
req, err := client.NewRequest("POST", u, nil)
if err != nil {
return nil, nil, err
}
registrationToken := new(github.RegistrationToken)
resp, err := client.Do(ctx, req, registrationToken)
if err != nil {
return nil, resp, err
}
return registrationToken, resp, nil
}
// ListOrganizationRunners lists all the self-hosted runners for an organization.
//
// GitHub API docs: https://developer.github.com/v3/actions/self-hosted-runners/#list-self-hosted-runners-for-an-organization
func ListOrganizationRunners(ctx context.Context, client *Client, owner string, opts *github.ListOptions) (*github.Runners, *github.Response, error) {
u := fmt.Sprintf("orgs/%v/actions/runners", owner)
u, err := addOptions(u, opts)
if err != nil {
return nil, nil, err
}
req, err := client.NewRequest("GET", u, nil)
if err != nil {
return nil, nil, err
}
runners := &github.Runners{}
resp, err := client.Do(ctx, req, &runners)
if err != nil {
return nil, resp, err
}
return runners, resp, nil
}
// RemoveOrganizationRunner forces the removal of a self-hosted runner in a repository using the runner id.
//
// GitHub API docs: https://developer.github.com/v3/actions/self_hosted_runners/#remove-a-self-hosted-runner
func RemoveOrganizationRunner(ctx context.Context, client *Client, owner string, runnerID int64) (*github.Response, error) {
u := fmt.Sprintf("orgs/%v/actions/runners/%v", owner, runnerID)
req, err := client.NewRequest("DELETE", u, nil)
if err != nil {
return nil, err
}
return client.Do(ctx, req, nil)
}
// addOptions adds the parameters in opt as URL query parameters to s. opt
// must be a struct whose fields may contain "url" tags.
func addOptions(s string, opts interface{}) (string, error) {
v := reflect.ValueOf(opts)
if v.Kind() == reflect.Ptr && v.IsNil() {
return s, nil
}
u, err := url.Parse(s)
if err != nil {
return s, err
}
qs, err := query.Values(opts)
if err != nil {
return s, err
}
u.RawQuery = qs.Encode()
return u.String(), nil
}

View File

@@ -36,18 +36,22 @@ func TestMain(m *testing.M) {
func TestGetRegistrationToken(t *testing.T) {
tests := []struct {
org string
repo string
token string
err bool
}{
{repo: "test/valid", token: fake.RegistrationToken, err: false},
{repo: "test/invalid", token: "", err: true},
{repo: "test/error", token: "", err: true},
{org: "test", repo: "valid", token: fake.RegistrationToken, err: false},
{org: "test", repo: "invalid", token: "", err: true},
{org: "test", repo: "error", token: "", err: true},
{org: "test", repo: "", token: fake.RegistrationToken, err: false},
{org: "invalid", repo: "", token: "", err: true},
{org: "error", repo: "", token: "", err: true},
}
client := newTestClient()
for i, tt := range tests {
rt, err := client.GetRegistrationToken(context.Background(), tt.repo, "test")
rt, err := client.GetRegistrationToken(context.Background(), tt.org, tt.repo, "test")
if !tt.err && err != nil {
t.Errorf("[%d] unexpected error: %v", i, err)
}
@@ -59,18 +63,22 @@ func TestGetRegistrationToken(t *testing.T) {
func TestListRunners(t *testing.T) {
tests := []struct {
org string
repo string
length int
err bool
}{
{repo: "test/valid", length: 2, err: false},
{repo: "test/invalid", length: 0, err: true},
{repo: "test/error", length: 0, err: true},
{org: "test", repo: "valid", length: 2, err: false},
{org: "test", repo: "invalid", length: 0, err: true},
{org: "test", repo: "error", length: 0, err: true},
{org: "test", repo: "", length: 2, err: false},
{org: "invalid", repo: "", length: 0, err: true},
{org: "error", repo: "", length: 0, err: true},
}
client := newTestClient()
for i, tt := range tests {
runners, err := client.ListRunners(context.Background(), tt.repo)
runners, err := client.ListRunners(context.Background(), tt.org, tt.repo)
if !tt.err && err != nil {
t.Errorf("[%d] unexpected error: %v", i, err)
}
@@ -82,17 +90,21 @@ func TestListRunners(t *testing.T) {
func TestRemoveRunner(t *testing.T) {
tests := []struct {
org string
repo string
err bool
}{
{repo: "test/valid", err: false},
{repo: "test/invalid", err: true},
{repo: "test/error", err: true},
{org: "test", repo: "valid", err: false},
{org: "test", repo: "invalid", err: true},
{org: "test", repo: "error", err: true},
{org: "test", repo: "", err: false},
{org: "invalid", repo: "", err: true},
{org: "error", repo: "", err: true},
}
client := newTestClient()
for i, tt := range tests {
err := client.RemoveRunner(context.Background(), tt.repo, int64(1))
err := client.RemoveRunner(context.Background(), tt.org, tt.repo, int64(1))
if !tt.err && err != nil {
t.Errorf("[%d] unexpected error: %v", i, err)
}

2
go.mod
View File

@@ -7,6 +7,7 @@ require (
github.com/davecgh/go-spew v1.1.1
github.com/go-logr/logr v0.1.0
github.com/google/go-github/v31 v31.0.0
github.com/google/go-querystring v1.0.0
github.com/onsi/ginkgo v1.8.0
github.com/onsi/gomega v1.5.0
github.com/stretchr/testify v1.4.0 // indirect
@@ -14,5 +15,6 @@ require (
k8s.io/api v0.0.0-20190918155943-95b840bb6a1f
k8s.io/apimachinery v0.0.0-20190913080033-27d36303b655
k8s.io/client-go v0.0.0-20190918160344-1fbdaa4c8d90
k8s.io/klog v0.4.0
sigs.k8s.io/controller-runtime v0.4.0
)

1
go.sum
View File

@@ -116,6 +116,7 @@ github.com/google/go-cmp v0.3.0 h1:crn/baboCvb5fXaQ0IJ1SGTsTVrWpDsCWC8EGETZijY=
github.com/google/go-cmp v0.3.0/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU=
github.com/google/go-cmp v0.3.1 h1:Xye71clBPdm5HgqGwUkwhbynsUJZhDbS20FvLhQ2izg=
github.com/google/go-cmp v0.3.1/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU=
github.com/google/go-github v17.0.0+incompatible h1:N0LgJ1j65A7kfXrZnUDaYCs/Sf4rEjNlfyDHW9dolSY=
github.com/google/go-github/v29 v29.0.2 h1:opYN6Wc7DOz7Ku3Oh4l7prmkOMwEcQxpFtxdU8N8Pts=
github.com/google/go-github/v29 v29.0.2/go.mod h1:CHKiKKPHJ0REzfwc14QMklvtHwCveD0PxlMjLlzAM5E=
github.com/google/go-github/v31 v31.0.0 h1:JJUxlP9lFK+ziXKimTCprajMApV1ecWD4NB6CCb0plo=

24
main.go
View File

@@ -21,6 +21,7 @@ import (
"fmt"
"os"
"strconv"
"time"
actionsv1alpha1 "github.com/summerwind/actions-runner-controller/api/v1alpha1"
"github.com/summerwind/actions-runner-controller/controllers"
@@ -57,6 +58,7 @@ func main() {
metricsAddr string
enableLeaderElection bool
syncPeriod time.Duration
runnerImage string
dockerImage string
@@ -76,6 +78,7 @@ func main() {
flag.Int64Var(&ghAppID, "github-app-id", 0, "The application ID of GitHub App.")
flag.Int64Var(&ghAppInstallationID, "github-app-installation-id", 0, "The installation ID of GitHub App.")
flag.StringVar(&ghAppPrivateKey, "github-app-private-key", "", "The path of a private key file to authenticate as a GitHub App")
flag.DurationVar(&syncPeriod, "sync-period", 10*time.Minute, "Determines the minimum frequency at which K8s resources managed by this controller are reconciled. When you use autoscaling, set to a lower value like 10 minute, because this corresponds to the minimum time to react on demand change")
flag.Parse()
if ghToken == "" {
@@ -133,6 +136,7 @@ func main() {
MetricsBindAddress: metricsAddr,
LeaderElection: enableLeaderElection,
Port: 9443,
SyncPeriod: &syncPeriod,
})
if err != nil {
setupLog.Error(err, "unable to start manager")
@@ -165,15 +169,29 @@ func main() {
}
runnerDeploymentReconciler := &controllers.RunnerDeploymentReconciler{
Client: mgr.GetClient(),
Log: ctrl.Log.WithName("controllers").WithName("RunnerDeployment"),
Scheme: mgr.GetScheme(),
Client: mgr.GetClient(),
Log: ctrl.Log.WithName("controllers").WithName("RunnerDeployment"),
Scheme: mgr.GetScheme(),
GitHubClient: ghClient,
}
if err = runnerDeploymentReconciler.SetupWithManager(mgr); err != nil {
setupLog.Error(err, "unable to create controller", "controller", "RunnerDeployment")
os.Exit(1)
}
if err = (&actionsv1alpha1.Runner{}).SetupWebhookWithManager(mgr); err != nil {
setupLog.Error(err, "unable to create webhook", "webhook", "Runner")
os.Exit(1)
}
if err = (&actionsv1alpha1.RunnerDeployment{}).SetupWebhookWithManager(mgr); err != nil {
setupLog.Error(err, "unable to create webhook", "webhook", "RunnerDeployment")
os.Exit(1)
}
if err = (&actionsv1alpha1.RunnerReplicaSet{}).SetupWebhookWithManager(mgr); err != nil {
setupLog.Error(err, "unable to create webhook", "webhook", "RunnerReplicaSet")
os.Exit(1)
}
// +kubebuilder:scaffold:builder
setupLog.Info("starting manager")

View File

@@ -3,9 +3,40 @@ FROM ubuntu:18.04
ARG RUNNER_VERSION
ARG DOCKER_VERSION
RUN apt update \
&& apt install sudo curl ca-certificates -y --no-install-recommends \
&& curl -L -o docker.tgz https://download.docker.com/linux/static/stable/x86_64/docker-${DOCKER_VERSION}.tgz \
ENV DEBIAN_FRONTEND=noninteractive
RUN apt update -y \
&& apt install -y software-properties-common \
&& add-apt-repository -y ppa:git-core/ppa \
&& apt update -y \
&& apt install -y --no-install-recommends \
build-essential \
curl \
ca-certificates \
dnsutils \
ftp \
git \
iproute2 \
iputils-ping \
jq \
libunwind8 \
locales \
netcat \
openssh-client \
parallel \
rsync \
shellcheck \
sudo \
telnet \
time \
tzdata \
unzip \
upx \
wget \
zip \
zstd \
&& rm -rf /var/lib/apt/lists/*
RUN curl -L -o docker.tgz https://download.docker.com/linux/static/stable/x86_64/docker-${DOCKER_VERSION}.tgz \
&& tar zxvf docker.tgz \
&& install -o root -g root -m 755 docker/docker /usr/local/bin/docker \
&& rm -rf docker docker.tgz \
@@ -20,7 +51,8 @@ RUN mkdir -p /runner \
&& curl -L -o runner.tar.gz https://github.com/actions/runner/releases/download/v${RUNNER_VERSION}/actions-runner-linux-x64-${RUNNER_VERSION}.tar.gz \
&& tar xzf ./runner.tar.gz \
&& rm runner.tar.gz \
&& ./bin/installdependencies.sh
&& ./bin/installdependencies.sh \
&& rm -rf /var/lib/apt/lists/*
COPY entrypoint.sh /runner

View File

@@ -1,6 +1,6 @@
NAME ?= summerwind/actions-runner
RUNNER_VERSION ?= 2.169.1
RUNNER_VERSION ?= 2.267.0
DOCKER_VERSION ?= 19.03.8
docker-build:

View File

@@ -5,18 +5,28 @@ if [ -z "${RUNNER_NAME}" ]; then
exit 1
fi
if [ -z "${RUNNER_REPO}" ]; then
echo "RUNNER_REPO must be set" 1>&2
if [ -n "${RUNNER_ORG}" -a -n "${RUNNER_REPO}" ]; then
ATTACH="${RUNNER_ORG}/${RUNNER_REPO}"
elif [ -n "${RUNNER_ORG}" ]; then
ATTACH="${RUNNER_ORG}"
elif [ -n "${RUNNER_REPO}" ]; then
ATTACH="${RUNNER_REPO}"
else
echo "At least one of RUNNER_ORG or RUNNER_REPO must be set" 1>&2
exit 1
fi
if [ -n "${RUNNER_LABELS}" ]; then
LABEL_ARG="--labels ${RUNNER_LABELS}"
fi
if [ -z "${RUNNER_TOKEN}" ]; then
echo "RUNNER_TOKEN must be set" 1>&2
exit 1
fi
cd /runner
./config.sh --unattended --replace --name "${RUNNER_NAME}" --url "https://github.com/${RUNNER_REPO}" --token "${RUNNER_TOKEN}"
./config.sh --unattended --replace --name "${RUNNER_NAME}" --url "https://github.com/${ATTACH}" --token "${RUNNER_TOKEN}" ${LABEL_ARG}
unset RUNNER_NAME RUNNER_REPO RUNNER_TOKEN
exec ./run.sh --once