Compare commits

...

66 Commits

Author SHA1 Message Date
Tingluo Huang
c645de9aee Delete docker-image.yml 2021-12-13 23:25:57 -05:00
TingluoHuang
0e4f76ec4e . 2021-10-28 22:59:19 -04:00
TingluoHuang
af18df4621 . 2021-10-28 21:39:55 -04:00
TingluoHuang
5215d95637 . 2021-10-28 21:31:52 -04:00
TingluoHuang
e750eb7e38 . 2021-10-28 21:20:48 -04:00
TingluoHuang
ca1f621077 . 2021-10-28 21:07:52 -04:00
TingluoHuang
80d0b58f3c . 2021-10-28 19:57:37 -04:00
TingluoHuang
11ff2be7e9 . 2021-10-28 19:48:42 -04:00
TingluoHuang
3ce763338d . 2021-10-28 19:40:25 -04:00
TingluoHuang
a45c0278e6 . 2021-10-28 19:18:21 -04:00
TingluoHuang
658d36c1bc . 2021-10-28 19:01:29 -04:00
TingluoHuang
ca3b803237 . 2021-10-28 18:50:44 -04:00
TingluoHuang
4fa691f73e . 2021-10-28 18:40:09 -04:00
TingluoHuang
dfcfae49e5 . 2021-10-14 16:57:24 -04:00
TingluoHuang
1235dc1cea . 2021-10-14 16:53:58 -04:00
TingluoHuang
cc0d0bed90 . 2021-10-14 16:49:54 -04:00
TingluoHuang
0fac863568 . 2021-10-14 16:46:45 -04:00
TingluoHuang
42e7359f5c . 2021-10-14 16:40:42 -04:00
TingluoHuang
5639175ecb . 2021-10-14 16:33:36 -04:00
TingluoHuang
7128998d77 . 2021-10-14 16:29:59 -04:00
TingluoHuang
f37e9f80a6 . 2021-10-14 15:33:18 -04:00
TingluoHuang
0fa08423d2 . 2021-10-14 15:28:59 -04:00
TingluoHuang
029106a1dc . 2021-10-14 15:11:17 -04:00
TingluoHuang
493a2a0bf7 . 2021-10-14 15:02:29 -04:00
TingluoHuang
43f983486e . 2021-10-14 14:59:34 -04:00
TingluoHuang
f6053b616c . 2021-10-14 14:55:12 -04:00
TingluoHuang
4f4608b710 . 2021-10-14 14:52:45 -04:00
TingluoHuang
28686c40d2 . 2021-10-14 14:50:28 -04:00
TingluoHuang
ce1679bb6f . 2021-10-14 14:48:42 -04:00
TingluoHuang
0a7611b0b5 podman 2021-10-14 14:45:09 -04:00
TingluoHuang
b3fee33a92 ref_* context. 2021-10-13 09:58:07 -04:00
Ferenc Hammerl
d83ef5549e Keep env vars alphabetical 2021-10-13 09:57:37 -04:00
TingluoHuang
fe6719d120 c 2021-10-13 09:57:37 -04:00
Raphael Cruzeiro
400b2d879c Makes the user keychains available to the service (#847)
Without creating a session, the service is not able to access the keychains for the user specified under `UserName`. This causes any workflow that deals with code signing to fail as the only keychain loaded with be the system one. This should fix #350
2021-10-06 15:37:45 -04:00
Thomas Boop
c4b6d288d4 fix ephemeral runner upgrade on mac/linux (#1403) 2021-10-05 10:15:19 +02:00
Julio Barba
0699597876 Use Actions Service health and api.github.com endpoints after connection failure on Actions Server and Hosted (#1385) 2021-09-30 13:40:34 -04:00
Thomas Boop
a592b14ae3 Runner 2.283.2 Release (#1389) 2021-09-29 15:49:40 -04:00
Thomas Boop
04269f7b1b Handle keeping previous OSX versions more smoothly on Mac (#1381)
* Handle macOS upgrade smoothly

* cleanup

* misc cleanup

* final updates

* Update src/Misc/layoutbin/update.sh.template

Co-authored-by: Patrick Ellis <319655+pje@users.noreply.github.com>

* Update src/Misc/layoutbin/update.sh.template

Co-authored-by: Patrick Ellis <319655+pje@users.noreply.github.com>

* Upload telemetry and default to old method as needed

* minor fix

* add one more bit of logging

* some more telemetry

* quote variables to handle spaces

* tiny fix for ubuntu

* remove version and move telemetry to diag

* use full path

Co-authored-by: Patrick Ellis <319655+pje@users.noreply.github.com>
2021-09-29 15:49:31 -04:00
Ferenc Hammerl
e89d2e84bd Stop-Commands: stopToken restrictions (#1371)
* Prevent stopTokens that are workflow commands

Co-authored-by: Thomas Boop <52323235+thboop@users.noreply.github.com>

* Check context for env var too

* Accept true, 1 and $true instead of just "true"

* Setup ExpressionValues in tests

* Update src/Runner.Common/Constants.cs

Co-authored-by: Thomas Boop <52323235+thboop@users.noreply.github.com>

* Separate success and fail tests for invalid token

* Fix envcontext for tests

Co-authored-by: Thomas Boop <52323235+thboop@users.noreply.github.com>
2021-09-29 14:44:01 -04:00
Thomas Boop
afe7066e39 only cleanup runner local files on success (#1384) 2021-09-28 18:55:28 -04:00
Ferenc Hammerl
da79ef4acb Fix unconfiguring of runner after group changes (#1359)
* Ignore agentpool when unconfiguring the runner

Runner names and IDs are unique within a ServiceHost
They don't need to be included when unconfiguring the runner.

* Use -1 instead of 0 to highlight how it is ignored

* Use overloads and 0 instead of -1

Using 0 seems to be the convention

* Fix typo calling the wrong method
2021-09-22 15:04:43 +02:00
Tingluo Huang
5afb52b272 Update the comment about the --once in Constants.cs (#1360)
* Update Constants.cs

* feedback.

* Update src/Runner.Listener/Runner.cs

Co-authored-by: Thomas Boop <52323235+thboop@users.noreply.github.com>

Co-authored-by: Thomas Boop <52323235+thboop@users.noreply.github.com>
2021-09-21 21:31:48 +00:00
Thomas Boop
cf87c55557 Don't retry 422 (#1352) 2021-09-21 09:59:21 -04:00
Ferenc Hammerl
43fa351980 Update telemetry (#1355)
* Track "pause-logging"

* Bump release version
2021-09-20 15:54:20 +02:00
Ferenc Hammerl
ecfc2cc9e9 Prepare 2.283.0 release (#1351)
* Update releaseNote.md

* Update runnerversion

* Update releaseNote.md

Co-authored-by: Patrick Ellis <319655+pje@users.noreply.github.com>

Co-authored-by: Patrick Ellis <319655+pje@users.noreply.github.com>
2021-09-20 14:59:37 +02:00
Ferenc Hammerl
740fb43731 Generic telemetry (#1321)
* Add generateIdTokenUrl as an env var

* Add generateIdTokenUrl to env vars

* Add basic telemetry class and submit it on jobcompleted

* Use constructor overload

* Rename telemetry to jobTelemetry

* Rename telemetry file

* Make JobTelemetryType a string

* Collect telemetry

* Remove debugger

* Update src/Runner.Worker/ActionCommandManager.cs

Co-authored-by: Thomas Boop <52323235+thboop@users.noreply.github.com>

* Use same JobTelemetry for all contexts

* Mask telemetry data

* Mask in JobRunner instead

* Empty line

* Change method signature
Returning with a List suggests we clone it and that the
original doesn't change..

* Update launch.json

Co-authored-by: Thomas Boop <52323235+thboop@users.noreply.github.com>
2021-09-20 14:44:50 +02:00
Patrick Ellis
f259e5706f Ephemeral runner deletes local .runner,.credentials files after completion (#1344)
Closes #1337
2021-09-16 11:00:27 -04:00
Ferenc Hammerl
5d84918ed5 Add name to runner context (#1312)
* Add generateIdTokenUrl as an env var

* Add generateIdTokenUrl to env vars

* Add name and runner_group to context

* No longer add runner-group

* Update runner name if needed

* Get interface instead of concrete class

* Check for nulls on ReservedAgent

* Avoid loading setting file unnecesseraly

* Only check agentName once

* Use Trace.Error when can't update settings

* Better equals and exception handling

* Update JobDispatcher.cs

* Add tests and null check
2021-09-16 15:25:51 +02:00
Julio Barba
881c521005 Revert "Recreate VssConnection on retry (#1316)" (#1343)
This reverts commit 4359dd605b.
2021-09-15 13:21:50 -04:00
Patrick Ellis
176e7f5208 Trim trailing whitespace in all md and yml files (#1329)
* Trim non-significant trailing whitespace, add final newlines to md,yml files

* Add .editorconfig with basic whitespace conventions
2021-09-15 13:35:25 +02:00
Jacob Wallraff
b6d46c148a Add attempt number to GitHub context (#1302)
* Add attempt number to GitHub context

* Change context name

* Changing order
2021-09-15 11:00:53 +02:00
Thomas Boop
38e33bb8e3 Update network.md 2021-09-14 15:28:30 -04:00
Patrick Ellis
404b3418b7 Prepare 2.282.0 release (#1327) 2021-09-13 13:56:47 -04:00
Tingluo Huang
7ffd9af644 Support --ephemeral flag (#660)
This optional flag will configure the runner to only take one job, and let the service un-configure the runner after that job finishes.
2021-09-13 11:28:09 -04:00
Thomas Boop
1b69c279f5 Networking TSG (#1325)
* Update Network Troubleshooting doc

* fix list

* Update network.md
2021-09-13 09:53:20 +02:00
Liviu Ionescu
567870dbb8 Avoid ConsoleColor.White, it is unreadable on light themes (#1295) (#1319)
* Avoid white, it is unreadable on light themes (#1295)

* remove ', ConsoleColor.White' from banner

* remove ', ConsoleColor.White' from prompt

* cleanups
2021-09-13 07:50:52 +00:00
Tingluo Huang
72fa2a8a0d Wait for job record updated before running steps. (#1320)
* Wait for job record updated before running steps.

* only oidc
2021-09-09 21:55:15 -04:00
Julio Barba
4359dd605b Recreate VssConnection on retry (#1316) 2021-09-09 19:09:17 -04:00
dependabot[bot]
aab936d081 Bump path-parse in /src/Misc/expressionFunc/hashFiles (#1256)
Bumps [path-parse](https://github.com/jbgutierrez/path-parse) from 1.0.6 to 1.0.7.
- [Release notes](https://github.com/jbgutierrez/path-parse/releases)
- [Commits](https://github.com/jbgutierrez/path-parse/commits/v1.0.7)

---
updated-dependencies:
- dependency-name: path-parse
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-09-01 16:48:41 -04:00
Nancy Hsieh
777ce5a0dc ignore empty arrays in jq select (#1269)
* ignore empty arrays in jq select

* Update delete.sh

Co-authored-by: Ferenc Hammerl <31069338+fhammerl@users.noreply.github.com>
2021-09-01 20:39:45 +00:00
Tingluo Huang
1a62162708 Prepare 2.281.1 runner release. (#1305) 2021-09-01 16:15:08 -04:00
Thomas Boop
9a829995e0 Show More Step Information in composite Actions (#1279) 2021-09-01 16:04:27 -04:00
Vladimir Panteleev
c5ce52641c Allow setting default severity to "notice" (#1213) 2021-09-01 16:01:23 -04:00
Nick Fields
e82725b580 Update error to say 'uninstall' not 'unconfigure' (#1179)
* Update error to say 'uninstall' not 'unconfigure'

* Say uninstall service in *nix config error msgs

Co-authored-by: Ferenc Hammerl <31069338+fhammerl@users.noreply.github.com>
2021-09-01 16:00:19 -04:00
Daniel Asztalos
0464f77de3 Typo fixed (#1289) 2021-09-01 15:59:18 -04:00
Tingluo Huang
1fc159e0df Temporary fix for macOS runner upgrade crash loop. (#1304) 2021-09-01 15:39:17 -04:00
96 changed files with 29752 additions and 305 deletions

8
.editorconfig Normal file
View File

@@ -0,0 +1,8 @@
# https://editorconfig.org/
[*]
insert_final_newline = true # ensure all files end with a single newline
trim_trailing_whitespace = true # attempt to remove trailing whitespace on save
[*.md]
trim_trailing_whitespace = false # in markdown, "two trailing spaces" is unfortunately meaningful; it means `<br>`

View File

@@ -2,17 +2,19 @@
### Common things that can cause the runner to not working properly
- Bug in the runner or the dotnet framework that causes actions runner can't make Http request in a certain network environment.
- A bug in the runner or the dotnet framework that causes the actions runner to be unable to make Http requests in a certain network environment.
- Proxy/Firewall block certain HTTP method, like it block all POST and PUT calls which the runner will use to upload logs.
- A Proxy or Firewall may block certain HTTP method, such as blocking all POST and PUT calls which the runner will use to upload logs.
- Proxy/Firewall only allows requests with certain user-agent to pass through and the actions runner user-agent is not in the allow list.
- A Proxy or Firewall may only allows requests with certain user-agent to pass through and the actions runner user-agent is not in the allow list.
- Proxy try to decrypt and exam HTTPS traffic for security purpose but cause the actions-runner to fail to finish SSL handshake due to the lack of trusting proxy's CA.
- A Proxy try to decrypt and exam HTTPS traffic for security purpose but cause the actions-runner to fail to finish SSL handshake due to the lack of trusting proxy's CA.
- Proxy try to modify the HTTPS request (like add or change some http headers) and causes the request become incompatible with the Actions Service (ASP.NetCore), Ex: [Nginx](https://github.com/dotnet/aspnetcore/issues/17081)
- The SSL handshake may fail if the client and server do not support the same TLS version, or the same cipher suites.
- Firewall rules that block action runner from accessing certain hosts, ex: `*.github.com`, `*.actions.githubusercontent.com`, etc.
- A Proxy may try to modify the HTTPS request (like add or change some http headers) and causes the request become incompatible with the Actions Service (ASP.NetCore), Ex: [Nginx](https://github.com/dotnet/aspnetcore/issues/17081)
- Firewall rules that block action runner from accessing certain hosts, ex: `*.github.com`, `*.actions.githubusercontent.com`, etc
### Identify and solve these problems
@@ -30,3 +32,31 @@ Use a 3rd party tool to make the same requests as the runner did would be a good
If the 3rd party tool is also experiencing the same error as the runner does, then you might want to contact your network administrator for help.
Otherwise, contact GitHub customer support or log an issue at https://github.com/actions/runner
### Troubleshooting: Why can't I configure a runner?
If you are having trouble connecting, try these steps:
1. Validate you can reach our endpoints from your web browser. If not, double check your local network connection
- For hosted Github:
- https://api.github.com/
- https://vstoken.actions.githubusercontent.com/_apis/health
- https://pipelines.actions.githubusercontent.com/_apis/health
- For GHES/GHAE
- https://myGHES.com/_services/vstoken/_apis/health
- https://myGHES.com/_services/pipelines/_apis/health
- https://myGHES.com/api/v3
2. Validate you can reach those endpoints in powershell core
- The runner runs on .net core, lets validate the local settings for that stack
- Open up `pwsh`
- Run the command using the urls above `Invoke-WebRequest {url}`
3. If not, get a packet trace using a tool like wireshark and start looking at the TLS handshake.
- If you see a Client Hello followed by a Server RST:
- You may need to configure your TLS settings to use the correct version
- You should support TLS version 1.2 or later
- You may need to configure your TLS settings to have up to date cipher suites, this may be solved by system updates and patches.
- Most notably, on windows server 2012 make sure [the tls cipher suite update](https://support.microsoft.com/en-us/topic/update-adds-new-tls-cipher-suites-and-changes-cipher-suite-priorities-in-windows-8-1-and-windows-server-2012-r2-8e395e43-c8ef-27d8-b60c-0fc57d526d94) is installed
- Your firewall, proxy or network configuration may be blocking the connection
- You will want to reach out to whoever is in charge of your network with these pcap files to further troubleshoot
- If you see a failure later in the handshake:
- Try the fix in the [SSLCert Fix](./sslcert.md)

74
job.yml Normal file
View File

@@ -0,0 +1,74 @@
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: pod-admin
namespace: default
rules:
- apiGroups: [""]
resources: ["pods", "pods/log", "pods/attach", "pods/exec"]
verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: default-pod-admin
namespace: default
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: pod-admin
subjects:
- kind: ServiceAccount
name: default
namespace: default
---
apiVersion: batch/v1
kind: Job
metadata:
namespace: default
name: actions-runners
spec:
template:
spec:
# hostNetwork: true
volumes:
- name: runner-working
emptyDir: {}
containers:
- name: k8srunner
image: huangtingluo/kube-runner:v0
imagePullPolicy: Always
volumeMounts:
- mountPath: /actions-runner/_work
name: runner-working
env:
- name: GITHUB_PAT
value: ghp_
- name: RUNNER_CONFIG_URL
value: https://github.com/bbq-beets/ting-test
- name: K8S_NODE_NAME
valueFrom:
fieldRef:
fieldPath: spec.nodeName
- name: K8S_POD_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
- name: K8S_POD_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
- name: K8S_POD_IP
valueFrom:
fieldRef:
fieldPath: status.podIP
- name: K8S_POD_SERVICE_ACCOUNT
valueFrom:
fieldRef:
fieldPath: spec.serviceAccountName
restartPolicy: Never
backoffLimit: 1
completions: 1
parallelism: 1

View File

@@ -2,7 +2,7 @@
## Bugs
- Fixed an issue where GHES runners fail to download public docker images (#1199)
- Fixed an issue where ephemeral runners did not restart after upgrading (#1396)
## Misc

View File

@@ -51,7 +51,7 @@ fi
# Ensure offline
#--------------------------------------
runner_status=$(curl -s -X GET ${base_api_url}/${runner_scope}/actions/runners?per_page=100 -H "accept: application/vnd.github.everest-preview+json" -H "authorization: token ${RUNNER_CFG_PAT}" \
| jq -M -j ".runners | .[] | [select(.name == \"${runner_name}\")] | .[0].status")
| jq -M -j ".runners | .[] | select(.name == \"${runner_name}\") | .status")
if [ -z "${runner_status}" ]; then
fatal "Could not find runner with name ${runner_name}"
@@ -67,7 +67,7 @@ fi
# Get id of runner to remove
#--------------------------------------
runner_id=$(curl -s -X GET ${base_api_url}/${runner_scope}/actions/runners?per_page=100 -H "accept: application/vnd.github.everest-preview+json" -H "authorization: token ${RUNNER_CFG_PAT}" \
| jq -M -j ".runners | .[] | [select(.name == \"${runner_name}\")] | .[0].id")
| jq -M -j ".runners | .[] | select(.name == \"${runner_name}\") | .id")
if [ -z "${runner_id}" ]; then
fatal "Could not find runner with name ${runner_name}"

78
src/Dockerfile Normal file
View File

@@ -0,0 +1,78 @@
FROM mcr.microsoft.com/dotnet/sdk:3.1 AS Build
# ENV RUNNER_CONFIG_URL=""
# ENV GITHUB_PAT=""
# ENV RUNNER_NAME=""
# ENV RUNNER_GROUP=""
# ENV RUNNER_LABELS=""
# ENV GITHUB_RUNNER_SCOPE=""
# ENV GITHUB_SERVER_URL=""
# ENV GITHUB_API_URL=""
# ENV K8S_HOST_IP=""
RUN apt-get update --fix-missing \
&& apt-get install -y --no-install-recommends \
curl \
# jq \
# git \
apt-utils \
apt-transport-https \
unzip \
net-tools\
gnupg2\
&& apt-get clean \
&& rm -rf /var/lib/apt/lists/*
# Install kubectl
# RUN curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key add - && \
# echo "deb https://apt.kubernetes.io/ kubernetes-xenial main" | tee -a /etc/apt/sources.list.d/kubernetes.list && \
# apt-get update && apt-get -y install --no-install-recommends kubectl
# Install docker
# RUN curl -fsSL https://get.docker.com -o get-docker.sh
# RUN sh get-docker.sh
# Allow runner to run as root
# ENV RUNNER_ALLOW_RUNASROOT=1
# Directory for runner to operate in
RUN mkdir /actions-runner
RUN mkdir /actions-runner/src
WORKDIR /actions-runner/src
COPY ./ /actions-runner/src
RUN /actions-runner/src/dev.sh l
FROM mcr.microsoft.com/dotnet/core/runtime-deps:3.1
ENV RUNNER_CONFIG_URL=""
ENV GITHUB_PAT=""
RUN apt-get update --fix-missing \
&& apt-get install -y --no-install-recommends \
curl \
# jq \
# git \
# apt-utils \
# apt-transport-https \
# unzip \
# net-tools\
gnupg2\
&& apt-get clean \
&& rm -rf /var/lib/apt/lists/*
# Install kubectl
RUN curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key add - && \
echo "deb https://apt.kubernetes.io/ kubernetes-xenial main" | tee -a /etc/apt/sources.list.d/kubernetes.list && \
apt-get update && apt-get -y install --no-install-recommends kubectl
# Allow runner to run as root
ENV RUNNER_ALLOW_RUNASROOT=1
# Directory for runner to operate in
RUN mkdir /actions-runner
WORKDIR /actions-runner
COPY --from=Build /actions-runner/_layout /actions-runner
ENTRYPOINT ["./entrypoint.sh"]

View File

@@ -0,0 +1,3 @@
dist/
lib/
node_modules/

View File

@@ -0,0 +1,59 @@
{
"plugins": ["jest", "@typescript-eslint"],
"extends": ["plugin:github/es6"],
"parser": "@typescript-eslint/parser",
"parserOptions": {
"ecmaVersion": 9,
"sourceType": "module",
"project": "./tsconfig.json"
},
"rules": {
"eslint-comments/no-use": "off",
"import/no-namespace": "off",
"no-console": "off",
"no-unused-vars": "off",
"@typescript-eslint/no-unused-vars": "error",
"@typescript-eslint/explicit-member-accessibility": ["error", {"accessibility": "no-public"}],
"@typescript-eslint/no-require-imports": "error",
"@typescript-eslint/array-type": "error",
"@typescript-eslint/await-thenable": "error",
"@typescript-eslint/ban-ts-ignore": "error",
"camelcase": "off",
"@typescript-eslint/camelcase": "error",
"@typescript-eslint/class-name-casing": "error",
"@typescript-eslint/explicit-function-return-type": ["error", {"allowExpressions": true}],
"@typescript-eslint/func-call-spacing": ["error", "never"],
"@typescript-eslint/generic-type-naming": ["error", "^[A-Z][A-Za-z]*$"],
"@typescript-eslint/no-array-constructor": "error",
"@typescript-eslint/no-empty-interface": "error",
"@typescript-eslint/no-explicit-any": "error",
"@typescript-eslint/no-extraneous-class": "error",
"@typescript-eslint/no-for-in-array": "error",
"@typescript-eslint/no-inferrable-types": "error",
"@typescript-eslint/no-misused-new": "error",
"@typescript-eslint/no-namespace": "error",
"@typescript-eslint/no-non-null-assertion": "warn",
"@typescript-eslint/no-object-literal-type-assertion": "error",
"@typescript-eslint/no-unnecessary-qualifier": "error",
"@typescript-eslint/no-unnecessary-type-assertion": "error",
"@typescript-eslint/no-useless-constructor": "error",
"@typescript-eslint/no-var-requires": "error",
"@typescript-eslint/prefer-for-of": "warn",
"@typescript-eslint/prefer-function-type": "warn",
"@typescript-eslint/prefer-includes": "error",
"@typescript-eslint/prefer-interface": "error",
"@typescript-eslint/prefer-string-starts-ends-with": "error",
"@typescript-eslint/promise-function-async": "error",
"@typescript-eslint/require-array-sort-compare": "error",
"@typescript-eslint/restrict-plus-operands": "error",
"semi": "off",
"@typescript-eslint/semi": ["error", "never"],
"@typescript-eslint/type-annotation-spacing": "error",
"@typescript-eslint/unbound-method": "error"
},
"env": {
"node": true,
"es6": true,
"jest/globals": true
}
}

View File

@@ -0,0 +1,3 @@
dist/
lib/
node_modules/

View File

@@ -0,0 +1,11 @@
{
"printWidth": 80,
"tabWidth": 2,
"useTabs": false,
"semi": false,
"singleQuote": true,
"trailingComma": "none",
"bracketSpacing": false,
"arrowParens": "avoid",
"parser": "typescript"
}

View File

@@ -0,0 +1 @@
To update kubeInnerHandler under `Misc/layoutbin` run `npm install && npm run all`

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,36 @@
{
"name": "kubeInnerHandler",
"version": "1.0.0",
"description": "GitHub Actions",
"main": "lib/kubeInnerHandler.js",
"scripts": {
"build": "tsc",
"format": "prettier --write **/*.ts",
"format-check": "prettier --check **/*.ts",
"lint": "eslint src/**/*.ts",
"pack": "ncc build -o ../../layoutbin/kubeInnerHandler",
"all": "npm run build && npm run format && npm run lint && npm run pack"
},
"repository": {
"type": "git",
"url": "git+https://github.com/actions/runner.git"
},
"keywords": [
"actions"
],
"author": "GitHub Actions",
"license": "MIT",
"dependencies": {
"@actions/exec": "^1.1.0",
"@actions/core": "^1.6.0"
},
"devDependencies": {
"@types/node": "^12.7.12",
"@typescript-eslint/parser": "^2.8.0",
"@zeit/ncc": "^0.20.5",
"eslint": "^6.8.0",
"eslint-plugin-github": "^2.0.0",
"prettier": "^1.19.1",
"typescript": "^3.6.4"
}
}

View File

@@ -0,0 +1,49 @@
import * as exec from '@actions/exec'
import * as core from '@actions/core'
import * as events from 'events'
import * as readline from 'readline'
async function run(): Promise<void> {
let input = ''
const rl = readline.createInterface({
input: process.stdin
})
rl.on('line', line => {
core.debug(`Line from STDIN: ${line}`)
input = line
})
await events.once(rl, 'close')
core.debug(input)
const execInput = JSON.parse(input)
core.debug(JSON.stringify(execInput))
// podman exec -i --workdir /__w/canary/canary
// -e GITHUB_JOB -e GITHUB_REF -e GITHUB_SHA -e GITHUB_REPOSITORY
// -e GITHUB_REPOSITORY_OWNER -e GITHUB_RUN_ID -e GITHUB_RUN_NUMBER
// -e GITHUB_RETENTION_DAYS -e GITHUB_RUN_ATTEMPT -e GITHUB_ACTOR
// -e GITHUB_WORKFLOW -e GITHUB_HEAD_REF -e GITHUB_BASE_REF -e GITHUB_EVENT_NAME
// -e GITHUB_SERVER_URL -e GITHUB_API_URL -e GITHUB_GRAPHQL_URL
// -e GITHUB_WORKSPACE -e GITHUB_ACTION -e GITHUB_EVENT_PATH -e GITHUB_ACTION_REPOSITORY
// -e GITHUB_ACTION_REF -e GITHUB_PATH -e GITHUB_ENV -e RUNNER_DEBUG
// -e RUNNER_OS -e RUNNER_NAME -e RUNNER_TOOL_CACHE
// -e RUNNER_TEMP -e RUNNER_WORKSPACE
// eccdf520697a035599d6e8c8dc801f004fdd3797cdce88f590aba3669a88d9bc sh -e /__w/_temp/d3b30383-719c-4e76-a16f-8f85443352be.sh
const execArgs = []
const args = (<string>execInput.arguments).split(' ')
core.debug(JSON.stringify(args))
execArgs.push(...args)
core.debug(JSON.stringify(execArgs))
await exec.exec(execInput.fileName, execArgs, {
env: execInput.environmentVariables
})
}
run()

View File

@@ -0,0 +1,12 @@
{
"compilerOptions": {
"target": "es6", /* Specify ECMAScript target version: 'ES3' (default), 'ES5', 'ES2015', 'ES2016', 'ES2017', 'ES2018', 'ES2019' or 'ESNEXT'. */
"module": "commonjs", /* Specify module code generation: 'none', 'commonjs', 'amd', 'system', 'umd', 'es2015', or 'ESNext'. */
"outDir": "./lib", /* Redirect output structure to the directory. */
"rootDir": "./src", /* Specify the root directory of input files. Use to control the output directory structure with --outDir. */
"strict": true, /* Enable all strict type-checking options. */
"noImplicitAny": true, /* Raise error on expressions and declarations with an implied 'any' type. */
"esModuleInterop": true /* Enables emit interoperability between CommonJS and ES Modules via creation of namespace objects for all imports. Implies 'allowSyntheticDefaultImports'. */
},
"exclude": ["node_modules", "**/*.test.ts"]
}

View File

@@ -0,0 +1,3 @@
dist/
lib/
node_modules/

View File

@@ -0,0 +1,59 @@
{
"plugins": ["jest", "@typescript-eslint"],
"extends": ["plugin:github/es6"],
"parser": "@typescript-eslint/parser",
"parserOptions": {
"ecmaVersion": 9,
"sourceType": "module",
"project": "./tsconfig.json"
},
"rules": {
"eslint-comments/no-use": "off",
"import/no-namespace": "off",
"no-console": "off",
"no-unused-vars": "off",
"@typescript-eslint/no-unused-vars": "error",
"@typescript-eslint/explicit-member-accessibility": ["error", {"accessibility": "no-public"}],
"@typescript-eslint/no-require-imports": "error",
"@typescript-eslint/array-type": "error",
"@typescript-eslint/await-thenable": "error",
"@typescript-eslint/ban-ts-ignore": "error",
"camelcase": "off",
"@typescript-eslint/camelcase": "error",
"@typescript-eslint/class-name-casing": "error",
"@typescript-eslint/explicit-function-return-type": ["error", {"allowExpressions": true}],
"@typescript-eslint/func-call-spacing": ["error", "never"],
"@typescript-eslint/generic-type-naming": ["error", "^[A-Z][A-Za-z]*$"],
"@typescript-eslint/no-array-constructor": "error",
"@typescript-eslint/no-empty-interface": "error",
"@typescript-eslint/no-explicit-any": "error",
"@typescript-eslint/no-extraneous-class": "error",
"@typescript-eslint/no-for-in-array": "error",
"@typescript-eslint/no-inferrable-types": "error",
"@typescript-eslint/no-misused-new": "error",
"@typescript-eslint/no-namespace": "error",
"@typescript-eslint/no-non-null-assertion": "warn",
"@typescript-eslint/no-object-literal-type-assertion": "error",
"@typescript-eslint/no-unnecessary-qualifier": "error",
"@typescript-eslint/no-unnecessary-type-assertion": "error",
"@typescript-eslint/no-useless-constructor": "error",
"@typescript-eslint/no-var-requires": "error",
"@typescript-eslint/prefer-for-of": "warn",
"@typescript-eslint/prefer-function-type": "warn",
"@typescript-eslint/prefer-includes": "error",
"@typescript-eslint/prefer-interface": "error",
"@typescript-eslint/prefer-string-starts-ends-with": "error",
"@typescript-eslint/promise-function-async": "error",
"@typescript-eslint/require-array-sort-compare": "error",
"@typescript-eslint/restrict-plus-operands": "error",
"semi": "off",
"@typescript-eslint/semi": ["error", "never"],
"@typescript-eslint/type-annotation-spacing": "error",
"@typescript-eslint/unbound-method": "error"
},
"env": {
"node": true,
"es6": true,
"jest/globals": true
}
}

View File

@@ -0,0 +1,3 @@
dist/
lib/
node_modules/

View File

@@ -0,0 +1,11 @@
{
"printWidth": 80,
"tabWidth": 2,
"useTabs": false,
"semi": false,
"singleQuote": true,
"trailingComma": "none",
"bracketSpacing": false,
"arrowParens": "avoid",
"parser": "typescript"
}

View File

@@ -0,0 +1 @@
To update kubectlHandler under `Misc/layoutbin` run `npm install && npm run all`

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,36 @@
{
"name": "kubectlHandler",
"version": "1.0.0",
"description": "GitHub Actions",
"main": "lib/kubectlHandler.js",
"scripts": {
"build": "tsc",
"format": "prettier --write **/*.ts",
"format-check": "prettier --check **/*.ts",
"lint": "eslint src/**/*.ts",
"pack": "ncc build -o ../../layoutbin/kubectlHandler",
"all": "npm run build && npm run format && npm run lint && npm run pack"
},
"repository": {
"type": "git",
"url": "git+https://github.com/actions/runner.git"
},
"keywords": [
"actions"
],
"author": "GitHub Actions",
"license": "MIT",
"dependencies": {
"@actions/exec": "^1.1.0",
"@actions/core": "^1.6.0"
},
"devDependencies": {
"@types/node": "^12.7.12",
"@typescript-eslint/parser": "^2.8.0",
"@zeit/ncc": "^0.20.5",
"eslint": "^6.8.0",
"eslint-plugin-github": "^2.0.0",
"prettier": "^1.19.1",
"typescript": "^3.6.4"
}
}

View File

@@ -0,0 +1,156 @@
import * as exec from '@actions/exec'
import * as core from '@actions/core'
import * as events from 'events'
import * as readline from 'readline'
async function run(): Promise<void> {
let input = ''
const rl = readline.createInterface({
input: process.stdin
})
rl.on('line', line => {
core.debug(`Line from STDIN: ${line}`)
input = line
})
await events.once(rl, 'close')
core.debug(input)
const inputJson = JSON.parse(input)
core.debug(JSON.stringify(inputJson))
const command = inputJson.command
if (command === 'Create') {
const creationInput = inputJson.creationInput
core.debug(JSON.stringify(creationInput))
const containers = creationInput.containers
const jobContainer = containers[0]
// const networkName = 'actions_podman_network'
// // podman network create {network} -> track and return `network` for ${{job.container.network}}
// await exec.exec('podman', ['network', 'create', networkName])
const containerImage = `${jobContainer.containerImage}`
// podman pull docker.io/library/{image}
// await exec.exec('podman', ['pull', containerImage])
// kubectl run e088c842be1f46b394212618408aaba0_node1016jessie_6196c9
// --image=node:10.16-jessie
// -- tail -f /dev/null
const runArgs = ['run', 'job-container']
// runArgs.push(`--workdir=${jobContainer.containerWorkDirectory}`)
// runArgs.push(`--network=${networkName}`)
// for (const mountVolume of jobContainer.mountVolumes) {
// runArgs.push(
// `-v=${mountVolume.sourceVolumePath}:${mountVolume.targetVolumePath}`
// )
// }
runArgs.push(`--image=${containerImage}`)
runArgs.push(`--`)
runArgs.push(`tail`)
runArgs.push(`-f`)
runArgs.push(`/dev/null`)
core.debug(JSON.stringify(runArgs))
// const containerId = await exec.getExecOutput('podman', [
// 'create',
// // `--workdir ${jobContainer.containerWorkDirectory}`,
// `--network=${networkName}`,
// // `-v=/Users/ting/Desktop/runner/_layout/_work:/__w`,
// `--entrypoint=${jobContainer.containerEntryPoint}`,
// `${containerImage}`,
// `${jobContainer.containerEntryPointArgs}`
// ])
await exec.exec('kubectl', runArgs)
// get PATH inside the container
const waitArgs = ['wait', '--for=condition=Ready', 'pod/job-container']
await exec.exec('kubectl', waitArgs)
// output containerId for ${{job.container.id}}
// copy over node.js
const cpNodeArgs = [
'cp',
'/actions-runner/externals/node12/bin',
'job-container:/__runner_util/'
]
await exec.exec('kubectl', cpNodeArgs)
// copy over innerhandler
const cpKubeInnerArgs = [
'cp',
'/actions-runner/bin/kubeInnerHandler',
'job-container:/__runner_util/kubeInnerHandler'
]
await exec.exec('kubectl', cpKubeInnerArgs)
// copy over _work
const cpWorkArgs = ['cp', '/actions-runner/_work', 'job-container:/__w/']
await exec.exec('kubectl', cpWorkArgs)
const creationOutput = {
JobContainerId: 'job-container',
Network: 'job-container'
}
const output = JSON.stringify({CreationOutput: creationOutput})
core.debug(output)
process.stderr.write(
`___CONTAINER_ENGINE_HANDLER_OUTPUT___${output}___CONTAINER_ENGINE_HANDLER_OUTPUT___`
)
} else if (command === 'Remove') {
const removeInput = inputJson.removeInput
core.debug(JSON.stringify(removeInput))
// const jobContainerId = removeInput.jobContainerId
// await exec.exec('kubectl', ['delete', 'pod', jobContainerId, '--force'])
// await exec.exec('podman', ['network', 'rm', '-f', network])
} else if (command === 'Exec') {
const execInput = inputJson.execInput
core.debug(JSON.stringify(execInput))
// podman exec -i --workdir /__w/canary/canary
// -e GITHUB_JOB -e GITHUB_REF -e GITHUB_SHA -e GITHUB_REPOSITORY
// -e GITHUB_REPOSITORY_OWNER -e GITHUB_RUN_ID -e GITHUB_RUN_NUMBER
// -e GITHUB_RETENTION_DAYS -e GITHUB_RUN_ATTEMPT -e GITHUB_ACTOR
// -e GITHUB_WORKFLOW -e GITHUB_HEAD_REF -e GITHUB_BASE_REF -e GITHUB_EVENT_NAME
// -e GITHUB_SERVER_URL -e GITHUB_API_URL -e GITHUB_GRAPHQL_URL
// -e GITHUB_WORKSPACE -e GITHUB_ACTION -e GITHUB_EVENT_PATH -e GITHUB_ACTION_REPOSITORY
// -e GITHUB_ACTION_REF -e GITHUB_PATH -e GITHUB_ENV -e RUNNER_DEBUG
// -e RUNNER_OS -e RUNNER_NAME -e RUNNER_TOOL_CACHE
// -e RUNNER_TEMP -e RUNNER_WORKSPACE
// eccdf520697a035599d6e8c8dc801f004fdd3797cdce88f590aba3669a88d9bc sh -e /__w/_temp/d3b30383-719c-4e76-a16f-8f85443352be.sh
const cpTempArgs = [
'cp',
'/actions-runner/_work/_temp',
'job-container:/__w/'
]
await exec.exec('kubectl', cpTempArgs)
const execArgs = ['exec']
execArgs.push(execInput.jobContainer.containerId)
execArgs.push('-i')
execArgs.push('-t')
execArgs.push('--')
execArgs.push('/__runner_util/node')
execArgs.push('/__runner_util/kubeInnerHandler')
core.debug(JSON.stringify(execArgs))
await exec.exec('kubectl', execArgs, {
input: Buffer.from(JSON.stringify(execInput))
})
}
}
run()

View File

@@ -0,0 +1,12 @@
{
"compilerOptions": {
"target": "es6", /* Specify ECMAScript target version: 'ES3' (default), 'ES5', 'ES2015', 'ES2016', 'ES2017', 'ES2018', 'ES2019' or 'ESNEXT'. */
"module": "commonjs", /* Specify module code generation: 'none', 'commonjs', 'amd', 'system', 'umd', 'es2015', or 'ESNext'. */
"outDir": "./lib", /* Redirect output structure to the directory. */
"rootDir": "./src", /* Specify the root directory of input files. Use to control the output directory structure with --outDir. */
"strict": true, /* Enable all strict type-checking options. */
"noImplicitAny": true, /* Raise error on expressions and declarations with an implied 'any' type. */
"esModuleInterop": true /* Enables emit interoperability between CommonJS and ES Modules via creation of namespace objects for all imports. Implies 'allowSyntheticDefaultImports'. */
},
"exclude": ["node_modules", "**/*.test.ts"]
}

View File

@@ -0,0 +1,3 @@
dist/
lib/
node_modules/

View File

@@ -0,0 +1,59 @@
{
"plugins": ["jest", "@typescript-eslint"],
"extends": ["plugin:github/es6"],
"parser": "@typescript-eslint/parser",
"parserOptions": {
"ecmaVersion": 9,
"sourceType": "module",
"project": "./tsconfig.json"
},
"rules": {
"eslint-comments/no-use": "off",
"import/no-namespace": "off",
"no-console": "off",
"no-unused-vars": "off",
"@typescript-eslint/no-unused-vars": "error",
"@typescript-eslint/explicit-member-accessibility": ["error", {"accessibility": "no-public"}],
"@typescript-eslint/no-require-imports": "error",
"@typescript-eslint/array-type": "error",
"@typescript-eslint/await-thenable": "error",
"@typescript-eslint/ban-ts-ignore": "error",
"camelcase": "off",
"@typescript-eslint/camelcase": "error",
"@typescript-eslint/class-name-casing": "error",
"@typescript-eslint/explicit-function-return-type": ["error", {"allowExpressions": true}],
"@typescript-eslint/func-call-spacing": ["error", "never"],
"@typescript-eslint/generic-type-naming": ["error", "^[A-Z][A-Za-z]*$"],
"@typescript-eslint/no-array-constructor": "error",
"@typescript-eslint/no-empty-interface": "error",
"@typescript-eslint/no-explicit-any": "error",
"@typescript-eslint/no-extraneous-class": "error",
"@typescript-eslint/no-for-in-array": "error",
"@typescript-eslint/no-inferrable-types": "error",
"@typescript-eslint/no-misused-new": "error",
"@typescript-eslint/no-namespace": "error",
"@typescript-eslint/no-non-null-assertion": "warn",
"@typescript-eslint/no-object-literal-type-assertion": "error",
"@typescript-eslint/no-unnecessary-qualifier": "error",
"@typescript-eslint/no-unnecessary-type-assertion": "error",
"@typescript-eslint/no-useless-constructor": "error",
"@typescript-eslint/no-var-requires": "error",
"@typescript-eslint/prefer-for-of": "warn",
"@typescript-eslint/prefer-function-type": "warn",
"@typescript-eslint/prefer-includes": "error",
"@typescript-eslint/prefer-interface": "error",
"@typescript-eslint/prefer-string-starts-ends-with": "error",
"@typescript-eslint/promise-function-async": "error",
"@typescript-eslint/require-array-sort-compare": "error",
"@typescript-eslint/restrict-plus-operands": "error",
"semi": "off",
"@typescript-eslint/semi": ["error", "never"],
"@typescript-eslint/type-annotation-spacing": "error",
"@typescript-eslint/unbound-method": "error"
},
"env": {
"node": true,
"es6": true,
"jest/globals": true
}
}

View File

@@ -0,0 +1,3 @@
dist/
lib/
node_modules/

View File

@@ -0,0 +1,11 @@
{
"printWidth": 80,
"tabWidth": 2,
"useTabs": false,
"semi": false,
"singleQuote": true,
"trailingComma": "none",
"bracketSpacing": false,
"arrowParens": "avoid",
"parser": "typescript"
}

View File

@@ -0,0 +1 @@
To update podmanHandler under `Misc/layoutbin` run `npm install && npm run all`

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,36 @@
{
"name": "podmanHandler",
"version": "1.0.0",
"description": "GitHub Actions",
"main": "lib/podmanHandler.js",
"scripts": {
"build": "tsc",
"format": "prettier --write **/*.ts",
"format-check": "prettier --check **/*.ts",
"lint": "eslint src/**/*.ts",
"pack": "ncc build -o ../../layoutbin/podmanHandler",
"all": "npm run build && npm run format && npm run lint && npm run pack"
},
"repository": {
"type": "git",
"url": "git+https://github.com/actions/runner.git"
},
"keywords": [
"actions"
],
"author": "GitHub Actions",
"license": "MIT",
"dependencies": {
"@actions/exec": "^1.1.0",
"@actions/core": "^1.6.0"
},
"devDependencies": {
"@types/node": "^12.7.12",
"@typescript-eslint/parser": "^2.8.0",
"@zeit/ncc": "^0.20.5",
"eslint": "^6.8.0",
"eslint-plugin-github": "^2.0.0",
"prettier": "^1.19.1",
"typescript": "^3.6.4"
}
}

View File

@@ -0,0 +1,150 @@
import * as exec from '@actions/exec'
import * as core from '@actions/core'
import * as events from 'events'
import * as readline from 'readline'
async function run(): Promise<void> {
let input = ''
const rl = readline.createInterface({
input: process.stdin
})
rl.on('line', line => {
core.debug(`Line from STDIN: ${line}`)
input = line
})
await events.once(rl, 'close')
core.debug(input)
const inputJson = JSON.parse(input)
core.debug(JSON.stringify(inputJson))
const command = inputJson.command
if (command === 'Create') {
const creationInput = inputJson.creationInput
core.debug(JSON.stringify(creationInput))
const containers = creationInput.containers
const jobContainer = containers[0]
const networkName = 'actions_podman_network'
// podman network create {network} -> track and return `network` for ${{job.container.network}}
await exec.exec('podman', ['network', 'create', networkName])
const containerImage = `docker.io/library/${jobContainer.containerImage}`
// podman pull docker.io/library/{image}
await exec.exec('podman', ['pull', containerImage])
// podman create --name e088c842be1f46b394212618408aaba0_node1016jessie_6196c9
// --label fa4e14
// --workdir /__w/canary/canary
// --network github_network_f98a6e1e96e74d919d814c165641cba3
// -e "HOME=/github/home" -e GITHUB_ACTIONS=true -e CI=true
// -v "/var/run/docker.sock":"/var/run/docker.sock"
// -v "/home/runner/work":"/__w"
// -v "/home/runner/runners/2.283.2/externals":"/__e":ro
// -v "/home/runner/work/_temp":"/__w/_temp"
// -v "/home/runner/work/_actions":"/__w/_actions"
// -v "/opt/hostedtoolcache":"/__t"
// -v "/home/runner/work/_temp/_github_home":"/github/home"
// -v "/home/runner/work/_temp/_github_workflow":"/github/workflow"
// --entrypoint "tail" node:10.16-jessie "-f" "/dev/null"
const creatArgs = ['create']
creatArgs.push(`--workdir=${jobContainer.containerWorkDirectory}`)
creatArgs.push(`--network=${networkName}`)
for (const mountVolume of jobContainer.mountVolumes) {
creatArgs.push(
`-v=${mountVolume.sourceVolumePath}:${mountVolume.targetVolumePath}`
)
}
creatArgs.push(`--entrypoint=tail`)
creatArgs.push(containerImage)
creatArgs.push(`-f`)
creatArgs.push(`/dev/null`)
core.debug(JSON.stringify(creatArgs))
// const containerId = await exec.getExecOutput('podman', [
// 'create',
// // `--workdir ${jobContainer.containerWorkDirectory}`,
// `--network=${networkName}`,
// // `-v=/Users/ting/Desktop/runner/_layout/_work:/__w`,
// `--entrypoint=${jobContainer.containerEntryPoint}`,
// `${containerImage}`,
// `${jobContainer.containerEntryPointArgs}`
// ])
const containerId = await exec.getExecOutput('podman', creatArgs)
core.debug(JSON.stringify(containerId))
// podman start {containerId}
await exec.exec('podman', ['start', containerId.stdout.trim()])
// get PATH inside the container
// output containerId for ${{job.container.id}}
const creationOutput = {
JobContainerId: containerId.stdout.trim(),
Network: networkName
}
const output = JSON.stringify({CreationOutput: creationOutput})
core.debug(output)
process.stderr.write(
`___CONTAINER_ENGINE_HANDLER_OUTPUT___${output}___CONTAINER_ENGINE_HANDLER_OUTPUT___`
)
} else if (command === 'Remove') {
const removeInput = inputJson.removeInput
core.debug(JSON.stringify(removeInput))
const jobContainerId = removeInput.jobContainerId
const network = removeInput.network
await exec.exec('podman', ['rm', '-f', jobContainerId])
await exec.exec('podman', ['network', 'rm', '-f', network])
} else if (command === 'Exec') {
const execInput = inputJson.execInput
core.debug(JSON.stringify(execInput))
// podman exec -i --workdir /__w/canary/canary
// -e GITHUB_JOB -e GITHUB_REF -e GITHUB_SHA -e GITHUB_REPOSITORY
// -e GITHUB_REPOSITORY_OWNER -e GITHUB_RUN_ID -e GITHUB_RUN_NUMBER
// -e GITHUB_RETENTION_DAYS -e GITHUB_RUN_ATTEMPT -e GITHUB_ACTOR
// -e GITHUB_WORKFLOW -e GITHUB_HEAD_REF -e GITHUB_BASE_REF -e GITHUB_EVENT_NAME
// -e GITHUB_SERVER_URL -e GITHUB_API_URL -e GITHUB_GRAPHQL_URL
// -e GITHUB_WORKSPACE -e GITHUB_ACTION -e GITHUB_EVENT_PATH -e GITHUB_ACTION_REPOSITORY
// -e GITHUB_ACTION_REF -e GITHUB_PATH -e GITHUB_ENV -e RUNNER_DEBUG
// -e RUNNER_OS -e RUNNER_NAME -e RUNNER_TOOL_CACHE
// -e RUNNER_TEMP -e RUNNER_WORKSPACE
// eccdf520697a035599d6e8c8dc801f004fdd3797cdce88f590aba3669a88d9bc sh -e /__w/_temp/d3b30383-719c-4e76-a16f-8f85443352be.sh
const execArgs = ['exec']
execArgs.push('-i')
execArgs.push(`--workdir=${execInput.workingDirectory}`)
for (const envKey of execInput.environmentKeys) {
execArgs.push(`-e=${envKey}`)
}
execArgs.push(execInput.jobContainer.containerId)
execArgs.push(execInput.fileName)
const args = (<string>execInput.arguments).split(' ')
core.debug(JSON.stringify(args))
execArgs.push(...args)
core.debug(JSON.stringify(execArgs))
await exec.exec('podman', execArgs)
}
await exec.exec('podman', ['network', 'ls'])
await exec.exec('podman', ['ps', '-a'])
}
run()

View File

@@ -0,0 +1,12 @@
{
"compilerOptions": {
"target": "es6", /* Specify ECMAScript target version: 'ES3' (default), 'ES5', 'ES2015', 'ES2016', 'ES2017', 'ES2018', 'ES2019' or 'ESNEXT'. */
"module": "commonjs", /* Specify module code generation: 'none', 'commonjs', 'amd', 'system', 'umd', 'es2015', or 'ESNext'. */
"outDir": "./lib", /* Redirect output structure to the directory. */
"rootDir": "./src", /* Specify the root directory of input files. Use to control the output directory structure with --outDir. */
"strict": true, /* Enable all strict type-checking options. */
"noImplicitAny": true, /* Raise error on expressions and declarations with an implied 'any' type. */
"esModuleInterop": true /* Enables emit interoperability between CommonJS and ES Modules via creation of namespace objects for all imports. Implies 'allowSyntheticDefaultImports'. */
},
"exclude": ["node_modules", "**/*.test.ts"]
}

View File

@@ -1947,9 +1947,9 @@
"dev": true
},
"path-parse": {
"version": "1.0.6",
"resolved": "https://registry.npmjs.org/path-parse/-/path-parse-1.0.6.tgz",
"integrity": "sha512-GSmOT2EbHrINBf9SR7CDELwlJ8AENk3Qn7OikK4nFYAu3Ote2+JYNVvkpAEQm3/TLNEJFD/xZJjzyxg3KBWOzw==",
"version": "1.0.7",
"resolved": "https://registry.npmjs.org/path-parse/-/path-parse-1.0.7.tgz",
"integrity": "sha512-LDJzPVEEEPR+y48z93A0Ed0yXb8pAByGWo/k5YYdYgpY2/2EsOsksJrq7lOHxryrVOn1ejG6oAp8ahvOIQD8sw==",
"dev": true
},
"path-type": {

View File

@@ -25,5 +25,7 @@
</dict>
<key>ProcessType</key>
<string>Interactive</string>
<key>SessionCreate</key>
<true/>
</dict>
</plist>

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,49 @@
// Job container creation
// podman network create {network} -> track and return `network` for ${{job.container.network}}
// podman pull docker.io/library/{image}
// podman create --name e088c842be1f46b394212618408aaba0_node1016jessie_6196c9
// --label fa4e14
// --workdir /__w/canary/canary
// --network github_network_f98a6e1e96e74d919d814c165641cba3
// -e "HOME=/github/home" -e GITHUB_ACTIONS=true -e CI=true
// -v "/var/run/docker.sock":"/var/run/docker.sock"
// -v "/home/runner/work":"/__w"
// -v "/home/runner/runners/2.283.2/externals":"/__e":ro
// -v "/home/runner/work/_temp":"/__w/_temp"
// -v "/home/runner/work/_actions":"/__w/_actions"
// -v "/opt/hostedtoolcache":"/__t"
// -v "/home/runner/work/_temp/_github_home":"/github/home"
// -v "/home/runner/work/_temp/_github_workflow":"/github/workflow"
// --entrypoint "tail" node:10.16-jessie "-f" "/dev/null"
// podman start {containerId}
// get PATH inside the container
// output containerId for ${{job.container.id}}
// Job container stop
// podman rm --force {containerId}
// podman network rm {network}
// Run step
// podman exec -i --workdir /__w/canary/canary
// -e GITHUB_JOB -e GITHUB_REF -e GITHUB_SHA -e GITHUB_REPOSITORY
// -e GITHUB_REPOSITORY_OWNER -e GITHUB_RUN_ID -e GITHUB_RUN_NUMBER
// -e GITHUB_RETENTION_DAYS -e GITHUB_RUN_ATTEMPT -e GITHUB_ACTOR
// -e GITHUB_WORKFLOW -e GITHUB_HEAD_REF -e GITHUB_BASE_REF -e GITHUB_EVENT_NAME
// -e GITHUB_SERVER_URL -e GITHUB_API_URL -e GITHUB_GRAPHQL_URL
// -e GITHUB_WORKSPACE -e GITHUB_ACTION -e GITHUB_EVENT_PATH -e GITHUB_ACTION_REPOSITORY
// -e GITHUB_ACTION_REF -e GITHUB_PATH -e GITHUB_ENV -e RUNNER_DEBUG
// -e RUNNER_OS -e RUNNER_NAME -e RUNNER_TOOL_CACHE
// -e RUNNER_TEMP -e RUNNER_WORKSPACE
// eccdf520697a035599d6e8c8dc801f004fdd3797cdce88f590aba3669a88d9bc sh -e /__w/_temp/d3b30383-719c-4e76-a16f-8f85443352be.sh

File diff suppressed because it is too large Load Diff

View File

@@ -18,6 +18,8 @@ downloadrunnerversion=_DOWNLOAD_RUNNER_VERSION_
logfile="_UPDATE_LOG_"
restartinteractiverunner=_RESTART_INTERACTIVE_RUNNER_
telemetryfile="$rootfolder/_diag/.telemetry"
# log user who run the script
date "+[%F %T-%4N] --------whoami--------" >> "$logfile" 2>&1
whoami >> "$logfile" 2>&1
@@ -118,6 +120,104 @@ then
exit 1
fi
# fix upgrade issue with macOS when running as a service
attemptedtargetedfix=0
currentplatform=$(uname | awk '{print tolower($0)}')
if [[ "$currentplatform" == 'darwin' && restartinteractiverunner -eq 0 ]]; then
# We needed a fix for https://github.com/actions/runner/issues/743
# We will recreate the ./externals/node12/bin/node of the past runner version that launched the runnerlistener service
# Otherwise mac gatekeeper kills the processes we spawn on creation as we are running a process with no backing file
# We need the pid for the nodejs loop, get that here, its the parent of the runner C# pid
# assumption here is only one process is invoking rootfolder/runsvc.sh
procgroup=$(ps x -o pgid,command | grep "$rootfolder/runsvc.sh" | grep -v grep | awk '{print $1}')
if [[ $? -eq 0 && -n "$procgroup" ]]
then
# inspect the open file handles to find the node process
# we can't actually inspect the process using ps because it uses relative paths and doesn't follow symlinks
path=$(lsof -a -g "$procgroup" -F n | grep node12/bin/node | grep externals | tail -1 | cut -c2-)
if [[ $? -eq 0 && -n "$path" ]]
then
# trim the last 5 characters of the path '/node'
trimmedpath=$(dirname "$path")
if [[ $? -eq 0 && -n "$trimmedpath" ]]
then
attemptedtargetedfix=1
# Create the path if it does not exist
if [[ ! -e "$path" ]]
then
date "+[%F %T-%4N] Creating fallback node at path $path" >> "$logfile" 2>&1
mkdir -p "$trimmedpath"
cp "$rootfolder/externals/node12/bin/node" "$path"
else
date "+[%F %T-%4N] Path for fallback node exists, skipping creating $path" >> "$logfile" 2>&1
fi
else
date "+[%F %T-%4N] DarwinRunnerUpgrade: Failed to trim runner path. TrimmedPath: $trimmedpath, path: $path, pgid: $procgroup, root: $rootfolder" >> "$logfile" 2>&1
date "+[%F %T-%4N] DarwinRunnerUpgrade: Failed to trim runner path. TrimmedPath: $trimmedpath, path: $path, pgid: $procgroup, root: $rootfolder" >> "$telemetryfile" 2>&1
fi
else
date "+[%F %T-%4N] DarwinRunnerUpgrade: Failed to find runner path. Path: $path, pgid: $procgroup, root: $rootfolder" >> "$logfile" 2>&1
date "+[%F %T-%4N] DarwinRunnerUpgrade: Failed to find runner path. Path: $path, pgid: $procgroup, root: $rootfolder" >> "$telemetryfile" 2>&1
fi
else
date "+[%F %T-%4N] DarwinRunnerUpgrade: Failed to find runner pgid. pgid: $procgroup, root: $rootfolder" >> "$logfile" 2>&1
date "+[%F %T-%4N] DarwinRunnerUpgrade: Failed to find runner pgid. pgid: $procgroup, root: $rootfolder" >> "$telemetryfile" 2>&1
fi
if [ $attemptedtargetedfix -eq 0 ]
then
date "+[%F %T-%4N] DarwinRunnerUpgrade: Defaulting to old macOS service fix" >> "$logfile" 2>&1
date "+[%F %T-%4N] DarwinRunnerUpgrade: Defaulting to old macOS service fix" >> "$telemetryfile" 2>&1
if [[ ! -e "$rootfolder/externals.2.280.3/node12/bin/node" ]]
then
mkdir -p "$rootfolder/externals.2.280.3/node12/bin"
cp "$rootfolder/externals/node12/bin/node" "$rootfolder/externals.2.280.3/node12/bin/node"
fi
if [[ ! -e "$rootfolder/externals.2.280.2/node12/bin/node" ]]
then
mkdir -p "$rootfolder/externals.2.280.2/node12/bin"
cp "$rootfolder/externals/node12/bin/node" "$rootfolder/externals.2.280.2/node12/bin/node"
fi
if [[ ! -e "$rootfolder/externals.2.280.1/node12/bin/node" ]]
then
mkdir -p "$rootfolder/externals.2.280.1/node12/bin"
cp "$rootfolder/externals/node12/bin/node" "$rootfolder/externals.2.280.1/node12/bin/node"
fi
# GHES 3.2
if [[ ! -e "$rootfolder/externals.2.279.0/node12/bin/node" ]]
then
mkdir -p "$rootfolder/externals.2.279.0/node12/bin"
cp "$rootfolder/externals/node12/bin/node" "$rootfolder/externals.2.279.0/node12/bin/node"
fi
# GHES 3.1.2 or later
if [[ ! -e "$rootfolder/externals.2.278.0/node12/bin/node" ]]
then
mkdir -p "$rootfolder/externals.2.278.0/node12/bin"
cp "$rootfolder/externals/node12/bin/node" "$rootfolder/externals.2.278.0/node12/bin/node"
fi
# GHES 3.1.0
if [[ ! -e "$rootfolder/externals.2.276.1/node12/bin/node" ]]
then
mkdir -p "$rootfolder/externals.2.276.1/node12/bin"
cp "$rootfolder/externals/node12/bin/node" "$rootfolder/externals.2.276.1/node12/bin/node"
fi
# GHES 3.0
if [[ ! -e "$rootfolder/externals.2.273.5/node12/bin/node" ]]
then
mkdir -p "$rootfolder/externals.2.273.5/node12/bin"
cp "$rootfolder/externals/node12/bin/node" "$rootfolder/externals.2.273.5/node12/bin/node"
fi
fi
fi
date "+[%F %T-%4N] Update succeed" >> "$logfile"
# rename the update log file with %logfile%.succeed/.failed/succeedneedrestart

View File

@@ -0,0 +1,68 @@
#!/bin/bash
set -euo pipefail
function fatal() {
echo "error: $1" >&2
exit 1
}
[ -n "${GITHUB_PAT:-""}" ] || fatal "GITHUB_PAT variable must be set"
[ -n "${RUNNER_CONFIG_URL:-""}" ] || fatal "RUNNER_CONFIG_URL variable must be set"
# [ -n "${RUNNER_NAME:-""}" ] || fatal "RUNNER_NAME variable must be set"
# if [ -n "${RUNNER_NAME}" ]; then
# # Use container id to gen unique runner name if name not provide
# CONTAINER_ID=$(cat /proc/self/cgroup | head -n 1 | tr '/' '\n' | tail -1 | cut -c1-12)
# RUNNER_NAME="actions-runner-${CONTAINER_ID}"
# fi
# if the scope has a slash, it's a repo runner
# orgs_or_repos="orgs"
# if [[ "$GITHUB_RUNNER_SCOPE" == *\/* ]]; then
# orgs_or_repos="repos"
# fi
# RUNNER_REG_URL="${GITHUB_SERVER_URL:=https://github.com}/${GITHUB_RUNNER_SCOPE}"
# echo "Runner Name : ${RUNNER_NAME}"
echo "Registration URL : ${RUNNER_CONFIG_URL}"
# echo "GitHub API URL : ${GITHUB_API_URL:=https://api.github.com}"
# echo "Runner Labels : ${RUNNER_LABELS:=""}"
# TODO: if api url is not default, validate it ends in /api/v3
# RUNNER_LABELS_ARG=""
# if [ -n "${RUNNER_LABELS}" ]; then
# RUNNER_LABELS_ARG="--labels ${RUNNER_LABELS}"
# fi
# RUNNER_GROUP_ARG=""
# if [ -n "${RUNNER_GROUP}" ]; then
# RUNNER_GROUP_ARG="--runnergroup ${RUNNER_GROUP}"
# fi
# if [ -n "${K8S_HOST_IP}" ]; then
# export http_proxy=http://$K8S_HOST_IP:9090
# fi
# curl -v -s -X POST ${GITHUB_API_URL}/${orgs_or_repos}/${GITHUB_RUNNER_SCOPE}/actions/runners/registration-token -H "authorization: token $GITHUB_PAT" -H "accept: application/vnd.github.everest-preview+json"
# Generate registration token
# RUNNER_REG_TOKEN=$(curl -s -X POST ${GITHUB_API_URL}/${orgs_or_repos}/${GITHUB_RUNNER_SCOPE}/actions/runners/registration-token -H "authorization: token $GITHUB_PAT" -H "accept: application/vnd.github.everest-preview+json" | jq -r '.token')
# Create the runner and configure it
./config.sh --unattended --url $RUNNER_CONFIG_URL --pat $GITHUB_PAT --replace --ephemeral
# while (! docker version ); do
# # Docker takes a few seconds to initialize
# echo "Waiting for Docker to launch..."
# sleep 1
# done
# unset env
unset RUNNER_CONFIG_URL
unset GITHUB_PAT
# Run it
./run.sh

View File

@@ -43,6 +43,21 @@ else
else
sleep 5
fi
elif [[ $returnCode == 4 ]]; then
if [ ! -x "$(command -v sleep)" ]; then
if [ ! -x "$(command -v ping)" ]; then
COUNT="0"
while [[ $COUNT != 5000 ]]; do
echo "SLEEP" > /dev/null
COUNT=$[$COUNT+1]
done
else
ping -c 5 127.0.0.1 > /dev/null
fi
else
sleep 5
fi
"$DIR"/bin/Runner.Listener run $*
else
exit $returnCode
fi

View File

@@ -33,6 +33,9 @@ namespace GitHub.Runner.Common
[DataMember(EmitDefaultValue = false)]
public string PoolName { get; set; }
[DataMember(EmitDefaultValue = false)]
public bool Ephemeral { get; set; }
[DataMember(EmitDefaultValue = false)]
public string ServerUrl { get; set; }

View File

@@ -26,6 +26,7 @@ namespace GitHub.Runner.Common
Certificates,
Options,
SetupInfo,
Telemetry
}
public static class Constants
@@ -125,9 +126,10 @@ namespace GitHub.Runner.Common
{
public static readonly string Check = "check";
public static readonly string Commit = "commit";
public static readonly string Ephemeral = "ephemeral";
public static readonly string Help = "help";
public static readonly string Replace = "replace";
public static readonly string Once = "once";
public static readonly string Once = "once"; // Keep this around since customers still relies on it
public static readonly string RunAsService = "runasservice";
public static readonly string Unattended = "unattended";
public static readonly string Version = "version";
@@ -153,6 +155,7 @@ namespace GitHub.Runner.Common
public static readonly string LowDiskSpace = "LOW_DISK_SPACE";
public static readonly string UnsupportedCommand = "UNSUPPORTED_COMMAND";
public static readonly string UnsupportedCommandMessageDisabled = "The `{0}` command is disabled. Please upgrade to using Environment Files or opt into unsecure command execution by setting the `ACTIONS_ALLOW_UNSECURE_COMMANDS` environment variable to `true`. For more information see: https://github.blog/changelog/2020-10-01-github-actions-deprecating-set-env-and-add-path-commands/";
public static readonly string UnsupportedStopCommandTokenDisabled = "You cannot use a endToken that is an empty string, the string 'pause-logging', or another workflow command. For more information see: https://docs.github.com/en/actions/learn-github-actions/workflow-commands-for-github-actions#example-stopping-and-starting-workflow-commands or opt into insecure command execution by setting the `ACTIONS_ALLOW_UNSECURE_STOPCOMMAND_TOKENS` environment variable to `true`.";
}
public static class RunnerEvent
@@ -212,6 +215,7 @@ namespace GitHub.Runner.Common
// Keep alphabetical
//
public static readonly string AllowUnsupportedCommands = "ACTIONS_ALLOW_UNSECURE_COMMANDS";
public static readonly string AllowUnsupportedStopCommandTokens = "ACTIONS_ALLOW_UNSECURE_STOPCOMMAND_TOKENS";
public static readonly string RunnerDebug = "ACTIONS_RUNNER_DEBUG";
public static readonly string StepDebug = "ACTIONS_STEP_DEBUG";
}

View File

@@ -343,6 +343,12 @@ namespace GitHub.Runner.Common
".setup_info");
break;
case WellKnownConfigFile.Telemetry:
path = Path.Combine(
GetDirectory(WellKnownDirectory.Diag),
".telemetry");
break;
default:
throw new NotSupportedException($"Unexpected well known config file: '{configFile}'");
}

View File

@@ -2,8 +2,11 @@
using System;
using System.Collections.Generic;
using System.IO;
using System.Net.Http;
using System.Threading;
using System.Threading.Tasks;
using GitHub.Runner.Sdk;
using GitHub.Services.Common;
using GitHub.Services.WebApi;
namespace GitHub.Runner.Common
@@ -36,6 +39,9 @@ namespace GitHub.Runner.Common
{
_connection = jobConnection;
int attemptCount = 5;
var configurationStore = HostContext.GetService<IConfigurationStore>();
var runnerSettings = configurationStore.GetSettings();
while (!_connection.HasAuthenticated && attemptCount-- > 0)
{
try
@@ -45,8 +51,13 @@ namespace GitHub.Runner.Common
}
catch (Exception ex) when (attemptCount > 0)
{
Trace.Info($"Catch exception during connect. {attemptCount} attemp left.");
Trace.Info($"Catch exception during connect. {attemptCount} attempts left.");
Trace.Error(ex);
if (runnerSettings.IsHostedServer)
{
await CheckNetworkEndpointsAsync();
}
}
await Task.Delay(100);
@@ -56,6 +67,52 @@ namespace GitHub.Runner.Common
_hasConnection = true;
}
private async Task CheckNetworkEndpointsAsync()
{
try
{
Trace.Info("Requesting Actions Service health endpoint status");
using (var httpClientHandler = HostContext.CreateHttpClientHandler())
using (var actionsClient = new HttpClient(httpClientHandler))
{
var baseUri = new Uri(_connection.Uri.GetLeftPart(UriPartial.Authority));
actionsClient.DefaultRequestHeaders.UserAgent.AddRange(HostContext.UserAgents);
// Call the _apis/health endpoint
var response = await actionsClient.GetAsync(new Uri(baseUri, "_apis/health"));
Trace.Info($"Actions health status code: {response.StatusCode}");
}
}
catch (Exception ex)
{
// Log error, but continue as this call is best-effort
Trace.Info($"Actions Service health endpoint failed due to {ex.GetType().Name}");
Trace.Error(ex);
}
try
{
Trace.Info("Requesting Github API endpoint status");
// This is a dotcom public API... just call it directly
using (var httpClientHandler = HostContext.CreateHttpClientHandler())
using (var gitHubClient = new HttpClient(httpClientHandler))
{
gitHubClient.DefaultRequestHeaders.UserAgent.AddRange(HostContext.UserAgents);
// Call the api.github.com endpoint
var response = await gitHubClient.GetAsync("https://api.github.com");
Trace.Info($"api.github.com status code: {response.StatusCode}");
}
}
catch (Exception ex)
{
// Log error, but continue as this call is best-effort
Trace.Info($"Github API endpoint failed due to {ex.GetType().Name}");
Trace.Error(ex);
}
}
private void CheckConnection()
{
if (!_hasConnection)

View File

@@ -15,6 +15,7 @@ namespace GitHub.Runner.Common
[ServiceLocator(Default = typeof(JobServerQueue))]
public interface IJobServerQueue : IRunnerService, IThrottlingReporter
{
TaskCompletionSource<int> JobRecordUpdated { get; }
event EventHandler<ThrottlingEventArgs> JobServerQueueThrottling;
Task ShutdownAsync();
void Start(Pipelines.AgentJobRequestMessage jobRequest);
@@ -62,8 +63,11 @@ namespace GitHub.Runner.Common
private IJobServer _jobServer;
private Task[] _allDequeueTasks;
private readonly TaskCompletionSource<int> _jobCompletionSource = new TaskCompletionSource<int>();
private readonly TaskCompletionSource<int> _jobRecordUpdated = new TaskCompletionSource<int>();
private bool _queueInProcess = false;
public TaskCompletionSource<int> JobRecordUpdated => _jobRecordUpdated;
public event EventHandler<ThrottlingEventArgs> JobServerQueueThrottling;
// Web console dequeue will start with process queue every 250ms for the first 60*4 times (~60 seconds).
@@ -455,6 +459,14 @@ namespace GitHub.Runner.Common
{
Trace.Verbose("Cleanup buffered timeline record for timeline: {0}.", update.TimelineId);
}
if (!_jobRecordUpdated.Task.IsCompleted &&
update.PendingRecords.Any(x => x.Id == _jobTimelineRecordId && x.State != null))
{
// We have changed the state of the job
Trace.Info("Job timeline record has been updated for the first time.");
_jobRecordUpdated.TrySetResult(0);
}
}
catch (Exception ex)
{

View File

@@ -29,8 +29,10 @@ namespace GitHub.Runner.Common
// Configuration
Task<TaskAgent> AddAgentAsync(Int32 agentPoolId, TaskAgent agent);
Task DeleteAgentAsync(int agentPoolId, int agentId);
Task DeleteAgentAsync(int agentId);
Task<List<TaskAgentPool>> GetAgentPoolsAsync(string agentPoolName = null, TaskAgentPoolType poolType = TaskAgentPoolType.Automation);
Task<List<TaskAgent>> GetAgentsAsync(int agentPoolId, string agentName = null);
Task<List<TaskAgent>> GetAgentsAsync(string agentName);
Task<TaskAgent> ReplaceAgentAsync(int agentPoolId, TaskAgent agent);
// messagequeue
@@ -252,6 +254,11 @@ namespace GitHub.Runner.Common
return _genericTaskAgentClient.GetAgentsAsync(agentPoolId, agentName, false);
}
public Task<List<TaskAgent>> GetAgentsAsync(string agentName)
{
return GetAgentsAsync(0, agentName); // search in all all agentPools
}
public Task<TaskAgent> ReplaceAgentAsync(int agentPoolId, TaskAgent agent)
{
CheckConnection(RunnerConnectionType.Generic);
@@ -264,6 +271,11 @@ namespace GitHub.Runner.Common
return _genericTaskAgentClient.DeleteAgentAsync(agentPoolId, agentId);
}
public Task DeleteAgentAsync(int agentId)
{
return DeleteAgentAsync(0, agentId); // agentPool is ignored server side
}
//-----------------------------------------------------------------
// MessageQueue
//-----------------------------------------------------------------

View File

@@ -164,9 +164,8 @@ namespace GitHub.Runner.Common
if (!Silent)
{
Console.WriteLine();
Console.ForegroundColor = ConsoleColor.White;
Console.WriteLine($"# {message}");
Console.ResetColor();
Console.WriteLine($"# {message}");
Console.WriteLine();
}
}
@@ -177,9 +176,8 @@ namespace GitHub.Runner.Common
{
Console.ForegroundColor = ConsoleColor.Green;
Console.Write("√ ");
Console.ForegroundColor = ConsoleColor.White;
Console.WriteLine(message);
Console.ResetColor();
Console.WriteLine(message);
}
}

View File

@@ -29,10 +29,11 @@ namespace GitHub.Runner.Listener
{
Constants.Runner.CommandLine.Flags.Check,
Constants.Runner.CommandLine.Flags.Commit,
Constants.Runner.CommandLine.Flags.Ephemeral,
Constants.Runner.CommandLine.Flags.Help,
Constants.Runner.CommandLine.Flags.Once,
Constants.Runner.CommandLine.Flags.Replace,
Constants.Runner.CommandLine.Flags.RunAsService,
Constants.Runner.CommandLine.Flags.Once,
Constants.Runner.CommandLine.Flags.Unattended,
Constants.Runner.CommandLine.Flags.Version
};
@@ -66,7 +67,9 @@ namespace GitHub.Runner.Listener
public bool Help => TestFlag(Constants.Runner.CommandLine.Flags.Help);
public bool Unattended => TestFlag(Constants.Runner.CommandLine.Flags.Unattended);
public bool Version => TestFlag(Constants.Runner.CommandLine.Flags.Version);
public bool Ephemeral => TestFlag(Constants.Runner.CommandLine.Flags.Ephemeral);
// Keep this around since customers still relies on it
public bool RunOnce => TestFlag(Constants.Runner.CommandLine.Flags.Once);
// Constructor.

View File

@@ -22,6 +22,7 @@ namespace GitHub.Runner.Listener.Configuration
bool IsConfigured();
Task ConfigureAsync(CommandSettings command);
Task UnconfigureAsync(CommandSettings command);
void DeleteLocalRunnerConfig();
RunnerSettings LoadSettings();
}
@@ -65,18 +66,18 @@ namespace GitHub.Runner.Listener.Configuration
public async Task ConfigureAsync(CommandSettings command)
{
_term.WriteLine();
_term.WriteLine("--------------------------------------------------------------------------------", ConsoleColor.White);
_term.WriteLine("| ____ _ _ _ _ _ _ _ _ |", ConsoleColor.White);
_term.WriteLine("| / ___(_) |_| | | |_ _| |__ / \\ ___| |_(_) ___ _ __ ___ |", ConsoleColor.White);
_term.WriteLine("| | | _| | __| |_| | | | | '_ \\ / _ \\ / __| __| |/ _ \\| '_ \\/ __| |", ConsoleColor.White);
_term.WriteLine("| | |_| | | |_| _ | |_| | |_) | / ___ \\ (__| |_| | (_) | | | \\__ \\ |", ConsoleColor.White);
_term.WriteLine("| \\____|_|\\__|_| |_|\\__,_|_.__/ /_/ \\_\\___|\\__|_|\\___/|_| |_|___/ |", ConsoleColor.White);
_term.WriteLine("| |", ConsoleColor.White);
_term.Write("| ", ConsoleColor.White);
_term.WriteLine("--------------------------------------------------------------------------------");
_term.WriteLine("| ____ _ _ _ _ _ _ _ _ |");
_term.WriteLine("| / ___(_) |_| | | |_ _| |__ / \\ ___| |_(_) ___ _ __ ___ |");
_term.WriteLine("| | | _| | __| |_| | | | | '_ \\ / _ \\ / __| __| |/ _ \\| '_ \\/ __| |");
_term.WriteLine("| | |_| | | |_| _ | |_| | |_) | / ___ \\ (__| |_| | (_) | | | \\__ \\ |");
_term.WriteLine("| \\____|_|\\__|_| |_|\\__,_|_.__/ /_/ \\_\\___|\\__|_|\\___/|_| |_|___/ |");
_term.WriteLine("| |");
_term.Write("| ");
_term.Write("Self-hosted runner registration", ConsoleColor.Cyan);
_term.WriteLine(" |", ConsoleColor.White);
_term.WriteLine("| |", ConsoleColor.White);
_term.WriteLine("--------------------------------------------------------------------------------", ConsoleColor.White);
_term.WriteLine(" |");
_term.WriteLine("| |");
_term.WriteLine("--------------------------------------------------------------------------------");
Trace.Info(nameof(ConfigureAsync));
if (IsConfigured())
@@ -195,6 +196,7 @@ namespace GitHub.Runner.Listener.Configuration
TaskAgent agent;
while (true)
{
runnerSettings.Ephemeral = command.Ephemeral;
runnerSettings.AgentName = command.GetRunnerName();
_term.WriteLine();
@@ -211,7 +213,7 @@ namespace GitHub.Runner.Listener.Configuration
if (command.GetReplace())
{
// Update existing agent with new PublicKey, agent version.
agent = UpdateExistingAgent(agent, publicKey, userLabels);
agent = UpdateExistingAgent(agent, publicKey, userLabels, runnerSettings.Ephemeral);
try
{
@@ -234,7 +236,7 @@ namespace GitHub.Runner.Listener.Configuration
else
{
// Create a new agent.
agent = CreateNewAgent(runnerSettings.AgentName, publicKey, userLabels);
agent = CreateNewAgent(runnerSettings.AgentName, publicKey, userLabels, runnerSettings.Ephemeral);
try
{
@@ -328,6 +330,38 @@ namespace GitHub.Runner.Listener.Configuration
#endif
}
// Delete .runner and .credentials files
public void DeleteLocalRunnerConfig()
{
bool isConfigured = _store.IsConfigured();
bool hasCredentials = _store.HasCredentials();
//delete credential config files
var currentAction = "Removing .credentials";
if (hasCredentials)
{
_store.DeleteCredential();
var keyManager = HostContext.GetService<IRSAKeyManager>();
keyManager.DeleteKey();
_term.WriteSuccessMessage("Removed .credentials");
}
else
{
_term.WriteLine("Does not exist. Skipping " + currentAction);
}
//delete settings config file
currentAction = "Removing .runner";
if (isConfigured)
{
_store.DeleteSettings();
_term.WriteSuccessMessage("Removed .runner");
}
else
{
_term.WriteLine("Does not exist. Skipping " + currentAction);
}
}
public async Task UnconfigureAsync(CommandSettings command)
{
string currentAction = string.Empty;
@@ -347,12 +381,9 @@ namespace GitHub.Runner.Listener.Configuration
_term.WriteLine();
_term.WriteSuccessMessage("Runner service removed");
#elif OS_LINUX
// unconfig system D service first
throw new Exception("Unconfigure service first");
#elif OS_OSX
// unconfig osx service first
throw new Exception("Unconfigure service first");
#else
// unconfig systemd or osx service first
throw new Exception("Uninstall service first");
#endif
}
@@ -384,7 +415,7 @@ namespace GitHub.Runner.Listener.Configuration
// Determine the service deployment type based on connection data. (Hosted/OnPremises)
await _runnerServer.ConnectAsync(new Uri(settings.ServerUrl), creds);
var agents = await _runnerServer.GetAgentsAsync(settings.PoolId, settings.AgentName);
var agents = await _runnerServer.GetAgentsAsync(settings.AgentName);
Trace.Verbose("Returns {0} agents", agents.Count);
TaskAgent agent = agents.FirstOrDefault();
if (agent == null)
@@ -393,7 +424,7 @@ namespace GitHub.Runner.Listener.Configuration
}
else
{
await _runnerServer.DeleteAgentAsync(settings.PoolId, settings.AgentId);
await _runnerServer.DeleteAgentAsync(settings.AgentId);
_term.WriteLine();
_term.WriteSuccessMessage("Runner removed successfully");
@@ -404,31 +435,7 @@ namespace GitHub.Runner.Listener.Configuration
_term.WriteLine("Cannot connect to server, because config files are missing. Skipping removing runner from the server.");
}
//delete credential config files
currentAction = "Removing .credentials";
if (hasCredentials)
{
_store.DeleteCredential();
var keyManager = HostContext.GetService<IRSAKeyManager>();
keyManager.DeleteKey();
_term.WriteSuccessMessage("Removed .credentials");
}
else
{
_term.WriteLine("Does not exist. Skipping " + currentAction);
}
//delete settings config file
currentAction = "Removing .runner";
if (isConfigured)
{
_store.DeleteSettings();
_term.WriteSuccessMessage("Removed .runner");
}
else
{
_term.WriteLine("Does not exist. Skipping " + currentAction);
}
DeleteLocalRunnerConfig();
}
catch (Exception)
{
@@ -459,7 +466,7 @@ namespace GitHub.Runner.Listener.Configuration
}
private TaskAgent UpdateExistingAgent(TaskAgent agent, RSAParameters publicKey, ISet<string> userLabels)
private TaskAgent UpdateExistingAgent(TaskAgent agent, RSAParameters publicKey, ISet<string> userLabels, bool ephemeral)
{
ArgUtil.NotNull(agent, nameof(agent));
agent.Authorization = new TaskAgentAuthorization
@@ -470,6 +477,8 @@ namespace GitHub.Runner.Listener.Configuration
// update should replace the existing labels
agent.Version = BuildConstants.RunnerPackage.Version;
agent.OSDescription = RuntimeInformation.OSDescription;
agent.Ephemeral = ephemeral;
agent.MaxParallelism = 1;
agent.Labels.Clear();
@@ -485,7 +494,7 @@ namespace GitHub.Runner.Listener.Configuration
return agent;
}
private TaskAgent CreateNewAgent(string agentName, RSAParameters publicKey, ISet<string> userLabels)
private TaskAgent CreateNewAgent(string agentName, RSAParameters publicKey, ISet<string> userLabels, bool ephemeral)
{
TaskAgent agent = new TaskAgent(agentName)
{
@@ -496,6 +505,7 @@ namespace GitHub.Runner.Listener.Configuration
MaxParallelism = 1,
Version = BuildConstants.RunnerPackage.Version,
OSDescription = RuntimeInformation.OSDescription,
Ephemeral = ephemeral,
};
agent.Labels.Add(new AgentLabel("self-hosted", LabelType.System));

View File

@@ -85,7 +85,7 @@ namespace GitHub.Runner.Listener.Configuration
while (true)
{
// Write the message prompt.
_terminal.Write($"{description} ", ConsoleColor.White);
_terminal.Write($"{description} ");
if(!string.IsNullOrEmpty(defaultValue))
{

View File

@@ -36,7 +36,10 @@ namespace GitHub.Runner.Listener
{
private readonly Lazy<Dictionary<long, TaskResult>> _localRunJobResult = new Lazy<Dictionary<long, TaskResult>>();
private int _poolId;
RunnerSettings _runnerSetting;
IConfigurationStore _configurationStore;
RunnerSettings _runnerSettings;
private static readonly string _workerProcessName = $"Runner.Worker{IOUtil.ExeExtension}";
// this is not thread-safe
@@ -54,9 +57,9 @@ namespace GitHub.Runner.Listener
base.Initialize(hostContext);
// get pool id from config
var configurationStore = hostContext.GetService<IConfigurationStore>();
_runnerSetting = configurationStore.GetSettings();
_poolId = _runnerSetting.PoolId;
_configurationStore = hostContext.GetService<IConfigurationStore>();
_runnerSettings = _configurationStore.GetSettings();
_poolId = _runnerSettings.PoolId;
int channelTimeoutSeconds;
if (!int.TryParse(Environment.GetEnvironmentVariable("GITHUB_ACTIONS_RUNNER_CHANNEL_TIMEOUT") ?? string.Empty, out channelTimeoutSeconds))
@@ -661,13 +664,15 @@ namespace GitHub.Runner.Listener
try
{
request = await runnerServer.RenewAgentRequestAsync(poolId, requestId, lockToken, orchestrationId, token);
Trace.Info($"Successfully renew job request {requestId}, job is valid till {request.LockedUntil.Value}");
if (!firstJobRequestRenewed.Task.IsCompleted)
{
// fire first renew succeed event.
firstJobRequestRenewed.TrySetResult(0);
// Update settings if the runner name has been changed server-side
UpdateAgentNameIfNeeded(request.ReservedAgent?.Name);
}
if (encounteringError > 0)
@@ -767,6 +772,27 @@ namespace GitHub.Runner.Listener
}
}
private void UpdateAgentNameIfNeeded(string agentName)
{
var isNewAgentName = !string.Equals(_runnerSettings.AgentName, agentName, StringComparison.Ordinal);
if (!isNewAgentName || string.IsNullOrEmpty(agentName))
{
return;
}
_runnerSettings.AgentName = agentName;
try
{
_configurationStore.SaveSettings(_runnerSettings);
}
catch (Exception ex)
{
Trace.Error("Cannot update the settings file:");
Trace.Error(ex);
}
}
// Best effort upload any logs for this job.
private async Task TryUploadUnfinishedLogs(Pipelines.AgentJobRequestMessage message)
{

View File

@@ -233,8 +233,14 @@ namespace GitHub.Runner.Listener
Trace.Info($"Set runner startup type - {startType}");
HostContext.StartupType = startType;
if (command.RunOnce)
{
_term.WriteLine("Warning: '--once' is going to be deprecated in the future, please consider using '--ephemeral' during runner registration.", ConsoleColor.Yellow);
_term.WriteLine("https://docs.github.com/en/actions/hosting-your-own-runners/autoscaling-with-self-hosted-runners#using-ephemeral-runners-for-autoscaling", ConsoleColor.Yellow);
}
// Run the runner interactively or as service
return await RunAsync(settings, command.RunOnce);
return await RunAsync(settings, command.RunOnce || settings.Ephemeral);
}
else
{
@@ -310,6 +316,9 @@ namespace GitHub.Runner.Listener
IJobDispatcher jobDispatcher = null;
CancellationTokenSource messageQueueLoopTokenSource = CancellationTokenSource.CreateLinkedTokenSource(HostContext.RunnerShutdownToken);
// Should we try to cleanup ephemeral runners
bool runOnceJobCompleted = false;
try
{
var notification = HostContext.GetService<IJobNotification>();
@@ -371,6 +380,7 @@ namespace GitHub.Runner.Listener
Task completeTask = await Task.WhenAny(getNextMessage, jobDispatcher.RunOnceJobCompleted.Task);
if (completeTask == jobDispatcher.RunOnceJobCompleted.Task)
{
runOnceJobCompleted = true;
Trace.Info("Job has finished at backend, the runner will exit since it is running under onetime use mode.");
Trace.Info("Stop message queue looping.");
messageQueueLoopTokenSource.Cancel();
@@ -466,10 +476,24 @@ namespace GitHub.Runner.Listener
await jobDispatcher.ShutdownAsync();
}
//TODO: make sure we don't mask more important exception
await _listener.DeleteSessionAsync();
try
{
await _listener.DeleteSessionAsync();
}
catch (Exception ex) when (runOnce)
{
// ignore exception during delete session for ephemeral runner since the runner might already be deleted from the server side
// and the delete session call will ends up with 401.
Trace.Info($"Ignore any exception during DeleteSession for an ephemeral runner. {ex}");
}
messageQueueLoopTokenSource.Dispose();
if (settings.Ephemeral && runOnceJobCompleted)
{
var configManager = HostContext.GetService<IConfigurationManager>();
configManager.DeleteLocalRunnerConfig();
}
}
}
catch (TaskAgentAccessTokenExpiredException)
@@ -512,7 +536,9 @@ Config Options:
--labels string Extra labels in addition to the default: 'self-hosted,{Constants.Runner.Platform},{Constants.Runner.PlatformArchitecture}'
--work string Relative runner work directory (default {Constants.Path.WorkDirectory})
--replace Replace any existing runner with the same name (default false)
--pat GitHub personal access token used for checking network connectivity when executing `.{separator}run.{ext} --check`");
--pat GitHub personal access token used for checking network connectivity when executing `.{separator}run.{ext} --check`
--ephemeral Configure the runner to only take one job and then let the service un-configure the runner after the job finishes (default false)");
#if OS_WINDOWS
_term.WriteLine($@" --runasservice Run the runner as a service");
_term.WriteLine($@" --windowslogonaccount string Account to run the service as. Requires runasservice");

View File

@@ -74,10 +74,12 @@ namespace GitHub.Runner.Listener
await jobDispatcher.WaitAsync(token);
Trace.Info($"All running job has exited.");
// We need to keep runner backup around for macOS until we fixed https://github.com/actions/runner/issues/743
#if !OS_OSX
// delete runner backup
DeletePreviousVersionRunnerBackup(token);
Trace.Info($"Delete old version runner backup.");
#endif
// generate update script from template
await UpdateRunnerUpdateStateAsync("Generate and execute update script.");
@@ -96,7 +98,7 @@ namespace GitHub.Runner.Listener
invokeScript.Start();
Trace.Info($"Update script start running");
await UpdateRunnerUpdateStateAsync("Runner will exit shortly for update, should back online within 10 seconds.");
await UpdateRunnerUpdateStateAsync("Runner will exit shortly for update, should be back online within 10 seconds.");
return true;
}

View File

@@ -1,7 +1,5 @@
using GitHub.DistributedTask.Pipelines;
using GitHub.DistributedTask.Pipelines.ContextData;
using GitHub.DistributedTask.Pipelines.ContextData;
using GitHub.DistributedTask.WebApi;
using GitHub.Runner.Common.Util;
using GitHub.Runner.Worker.Container;
using System;
using System.Collections.Generic;
@@ -110,11 +108,18 @@ namespace GitHub.Runner.Worker
// Stop command
if (string.Equals(actionCommand.Command, _stopCommand, StringComparison.OrdinalIgnoreCase))
{
context.Output(input);
context.Debug("Paused processing commands until '##[{actionCommand.Data}]' is received");
ValidateStopToken(context, actionCommand.Data);
_stopToken = actionCommand.Data;
_stopProcessCommand = true;
_registeredCommands.Add(_stopToken);
if (_stopToken.Length > 6)
{
HostContext.SecretMasker.AddValue(_stopToken);
}
context.Output(input);
context.Debug("Paused processing commands until the token you called ::stopCommands:: with is received");
return true;
}
// Found command
@@ -148,7 +153,42 @@ namespace GitHub.Runner.Worker
return true;
}
internal static bool EnhancedAnnotationsEnabled(IExecutionContext context) {
private void ValidateStopToken(IExecutionContext context, string stopToken)
{
#if OS_WINDOWS
var envContext = context.ExpressionValues["env"] as DictionaryContextData;
#else
var envContext = context.ExpressionValues["env"] as CaseSensitiveDictionaryContextData;
#endif
var allowUnsecureStopCommandTokens = false;
allowUnsecureStopCommandTokens = StringUtil.ConvertToBoolean(Environment.GetEnvironmentVariable(Constants.Variables.Actions.AllowUnsupportedStopCommandTokens));
if (!allowUnsecureStopCommandTokens && envContext.ContainsKey(Constants.Variables.Actions.AllowUnsupportedStopCommandTokens))
{
allowUnsecureStopCommandTokens = StringUtil.ConvertToBoolean(envContext[Constants.Variables.Actions.AllowUnsupportedStopCommandTokens].ToString());
}
bool isTokenInvalid = _registeredCommands.Contains(stopToken)
|| string.IsNullOrEmpty(stopToken)
|| string.Equals(stopToken, "pause-logging", StringComparison.OrdinalIgnoreCase);
if (isTokenInvalid)
{
var telemetry = new JobTelemetry
{
Message = $"Invoked ::stopCommand:: with token: [{stopToken}]",
Type = JobTelemetryType.ActionCommand
};
context.JobTelemetry.Add(telemetry);
}
if (isTokenInvalid && !allowUnsecureStopCommandTokens)
{
throw new Exception(Constants.Runner.UnsupportedStopCommandTokenDisabled);
}
}
internal static bool EnhancedAnnotationsEnabled(IExecutionContext context)
{
return context.Global.Variables.GetBoolean("DistributedTask.EnhancedAnnotations") ?? false;
}
}

View File

@@ -633,7 +633,12 @@ namespace GitHub.Runner.Worker
}
catch (Exception ex) when (!executionContext.CancellationToken.IsCancellationRequested) // Do not retry if the run is canceled.
{
if (attempt < 3)
// UnresolvableActionDownloadInfoException is a 422 client error, don't retry
// Some possible cases are:
// * Repo is rate limited
// * Repo or tag doesn't exist, or isn't public
// * Policy validation failed
if (attempt < 3 && !(ex is WebApi.UnresolvableActionDownloadInfoException))
{
executionContext.Output($"Failed to resolve action download info. Error: {ex.Message}");
executionContext.Debug(ex.ToString());
@@ -649,6 +654,7 @@ namespace GitHub.Runner.Worker
// Some possible cases are:
// * Repo is rate limited
// * Repo or tag doesn't exist, or isn't public
// * Policy validation failed
if (ex is WebApi.UnresolvableActionDownloadInfoException)
{
throw;

View File

@@ -54,7 +54,7 @@ namespace GitHub.Runner.Worker.Container
_pathMappings.Add(new PathMapping(hostContext.GetDirectory(WellKnownDirectory.Externals), "/__e"));
if (this.IsJobContainer)
{
this.MountVolumes.Add(new MountVolume("/var/run/docker.sock", "/var/run/docker.sock"));
// this.MountVolumes.Add(new MountVolume("/var/run/docker.sock", "/var/run/docker.sock"));
}
#endif
if (container.Ports?.Count > 0)

View File

@@ -12,9 +12,88 @@ using GitHub.Runner.Sdk;
using GitHub.DistributedTask.Pipelines.ContextData;
using Microsoft.Win32;
using GitHub.DistributedTask.Pipelines.ObjectTemplating;
using System.Threading.Channels;
using GitHub.Services.WebApi;
using System.Text;
using System.Runtime.Serialization;
namespace GitHub.Runner.Worker
{
[DataContract]
public class ContainerEngineHandlerInput
{
[DataMember]
public string Command { get; set; }
[DataMember]
public ContainersCreationInput CreationInput { get; set; }
[DataMember]
public JobContainerExecInput ExecInput { get; set; }
[DataMember]
public ContainersRemoveInput RemoveInput { get; set; }
}
[DataContract]
public class ContainersCreationInput
{
[DataMember]
public List<ContainerInfo> Containers { get; set; }
}
[DataContract]
public class JobContainerExecInput
{
[DataMember]
public ContainerInfo JobContainer { get; set; }
[DataMember]
public string WorkingDirectory { get; set; }
[DataMember]
public string FileName { get; set; }
[DataMember]
public string Arguments { get; set; }
[DataMember]
public List<string> EnvironmentKeys { get; set; }
[DataMember]
public Dictionary<string, string> EnvironmentVariables { get; set; }
}
[DataContract]
public class ContainersRemoveInput
{
[DataMember]
public string Network { get; set; }
[DataMember]
public string JobContainerId { get; set; }
}
[DataContract]
public class ContainersCreationOutput
{
[DataMember]
public string Network { get; set; }
[DataMember]
public string JobContainerId { get; set; }
}
[DataContract]
public class ContainerEngineHandlerOutput
{
[DataMember]
public ContainersCreationOutput CreationOutput { get; set; }
}
[ServiceLocator(Default = typeof(ContainerOperationProvider))]
public interface IContainerOperationProvider : IRunnerService
{
@@ -24,25 +103,57 @@ namespace GitHub.Runner.Worker
public class ContainerOperationProvider : RunnerService, IContainerOperationProvider
{
private IDockerCommandManager _dockerManager;
private IDockerCommandManager _dockerManager = null;
public override void Initialize(IHostContext hostContext)
{
base.Initialize(hostContext);
_dockerManager = HostContext.GetService<IDockerCommandManager>();
// _dockerManager = HostContext.GetService<IDockerCommandManager>();
}
public async Task StartContainersAsync(IExecutionContext executionContext, object data)
{
Trace.Entering();
if (!Constants.Runner.Platform.Equals(Constants.OSPlatform.Linux))
{
throw new NotSupportedException("Container operations are only supported on Linux runners");
}
// if (!Constants.Runner.Platform.Equals(Constants.OSPlatform.Linux))
// {
// throw new NotSupportedException("Container operations are only supported on Linux runners");
// }
ArgUtil.NotNull(executionContext, nameof(executionContext));
List<ContainerInfo> containers = data as List<ContainerInfo>;
ArgUtil.NotNull(containers, nameof(containers));
foreach (var container in containers)
{
if (container.IsJobContainer)
{
// Configure job container - Mount workspace and tools, set up environment, and start long running process
var githubContext = executionContext.ExpressionValues["github"] as GitHubContext;
ArgUtil.NotNull(githubContext, nameof(githubContext));
var workingDirectory = githubContext["workspace"] as StringContextData;
ArgUtil.NotNullOrEmpty(workingDirectory, nameof(workingDirectory));
container.MountVolumes.Add(new MountVolume(HostContext.GetDirectory(WellKnownDirectory.Work), container.TranslateToContainerPath(HostContext.GetDirectory(WellKnownDirectory.Work))));
container.MountVolumes.Add(new MountVolume(HostContext.GetDirectory(WellKnownDirectory.Externals), container.TranslateToContainerPath(HostContext.GetDirectory(WellKnownDirectory.Externals)), true));
container.MountVolumes.Add(new MountVolume(HostContext.GetDirectory(WellKnownDirectory.Temp), container.TranslateToContainerPath(HostContext.GetDirectory(WellKnownDirectory.Temp))));
// container.MountVolumes.Add(new MountVolume(HostContext.GetDirectory(WellKnownDirectory.Actions), container.TranslateToContainerPath(HostContext.GetDirectory(WellKnownDirectory.Actions))));
container.MountVolumes.Add(new MountVolume(HostContext.GetDirectory(WellKnownDirectory.Tools), container.TranslateToContainerPath(HostContext.GetDirectory(WellKnownDirectory.Tools))));
var tempHomeDirectory = Path.Combine(HostContext.GetDirectory(WellKnownDirectory.Temp), "_github_home");
Directory.CreateDirectory(tempHomeDirectory);
container.MountVolumes.Add(new MountVolume(tempHomeDirectory, "/github/home"));
container.AddPathTranslateMapping(tempHomeDirectory, "/github/home");
container.ContainerEnvironmentVariables["HOME"] = container.TranslateToContainerPath(tempHomeDirectory);
var tempWorkflowDirectory = Path.Combine(HostContext.GetDirectory(WellKnownDirectory.Temp), "_github_workflow");
Directory.CreateDirectory(tempWorkflowDirectory);
container.MountVolumes.Add(new MountVolume(tempWorkflowDirectory, "/github/workflow"));
container.AddPathTranslateMapping(tempWorkflowDirectory, "/github/workflow");
container.ContainerWorkDirectory = container.TranslateToContainerPath(workingDirectory);
container.ContainerEntryPoint = "tail";
container.ContainerEntryPointArgs = "-f /dev/null";
}
}
var postJobStep = new JobExtensionRunner(runAsync: this.StopContainersAsync,
condition: $"{PipelineTemplateConstants.Always}()",
displayName: "Stop containers",
@@ -51,9 +162,71 @@ namespace GitHub.Runner.Worker
executionContext.Debug($"Register post job cleanup for stopping/deleting containers.");
executionContext.RegisterPostJobStep(postJobStep);
// Check whether we are inside a container.
// Our container feature requires to map working directory from host to the container.
// If we are already inside a container, we will not able to find out the real working direcotry path on the host.
var podManHandler = Path.Combine(HostContext.GetDirectory(WellKnownDirectory.Bin), "kubectlHandler", "index.js");
if (File.Exists(podManHandler))
{
var podmanInput = new ContainerEngineHandlerInput()
{
Command = "Create",
CreationInput = new ContainersCreationInput()
{
Containers = containers
}
};
ContainerEngineHandlerOutput podmanOutput = null;
using (var processInvoker = HostContext.CreateService<IProcessInvoker>())
{
var redirectStandardIn = Channel.CreateUnbounded<string>(new UnboundedChannelOptions() { SingleReader = true, SingleWriter = true });
redirectStandardIn.Writer.TryWrite(JsonUtility.ToString(podmanInput));
processInvoker.OutputDataReceived += delegate (object sender, ProcessDataReceivedEventArgs message)
{
executionContext.Output(message.Data);
};
processInvoker.ErrorDataReceived += delegate (object sender, ProcessDataReceivedEventArgs message)
{
executionContext.Output(message.Data);
if (podmanOutput == null && message.Data.IndexOf("___CONTAINER_ENGINE_HANDLER_OUTPUT___") >= 0)
{
try
{
podmanOutput = JsonUtility.FromString<ContainerEngineHandlerOutput>(message.Data.Replace("___CONTAINER_ENGINE_HANDLER_OUTPUT___", ""));
}
catch (Exception ex)
{
executionContext.Error(ex);
}
}
};
// Execute the process. Exit code 0 should always be returned.
// A non-zero exit code indicates infrastructural failure.
// Task failure should be communicated over STDOUT using ## commands.
await processInvoker.ExecuteAsync(workingDirectory: HostContext.GetDirectory(WellKnownDirectory.Bin),
fileName: Path.Combine(HostContext.GetDirectory(WellKnownDirectory.Externals), "node12", "bin", $"node{IOUtil.ExeExtension}"),
arguments: podManHandler,
environment: null,
requireExitCodeZero: false,
outputEncoding: Encoding.UTF8,
killProcessOnCancel: false,
redirectStandardIn: redirectStandardIn,
cancellationToken: executionContext.CancellationToken);
}
if (podmanOutput != null)
{
executionContext.JobContext.Container["network"] = new StringContextData(podmanOutput.CreationOutput.Network);
executionContext.JobContext.Container["id"] = new StringContextData(podmanOutput.CreationOutput.JobContainerId);
executionContext.Global.Container.ContainerId = podmanOutput.CreationOutput.JobContainerId;
}
}
else
{
// Check whether we are inside a container.
// Our container feature requires to map working directory from host to the container.
// If we are already inside a container, we will not able to find out the real working direcotry path on the host.
#if OS_WINDOWS
// service CExecSvc is Container Execution Agent.
ServiceController[] scServices = ServiceController.GetServices();
@@ -62,11 +235,11 @@ namespace GitHub.Runner.Worker
throw new NotSupportedException("Container feature is not supported when runner is already running inside container.");
}
#else
var initProcessCgroup = File.ReadLines("/proc/1/cgroup");
if (initProcessCgroup.Any(x => x.IndexOf(":/docker/", StringComparison.OrdinalIgnoreCase) >= 0))
{
throw new NotSupportedException("Container feature is not supported when runner is already running inside container.");
}
var initProcessCgroup = File.ReadLines("/proc/1/cgroup");
if (initProcessCgroup.Any(x => x.IndexOf(":/docker/", StringComparison.OrdinalIgnoreCase) >= 0))
{
throw new NotSupportedException("Container feature is not supported when runner is already running inside container.");
}
#endif
#if OS_WINDOWS
@@ -90,68 +263,69 @@ namespace GitHub.Runner.Worker
}
#endif
// Check docker client/server version
executionContext.Output("##[group]Checking docker version");
DockerVersion dockerVersion = await _dockerManager.DockerVersion(executionContext);
executionContext.Output("##[endgroup]");
// Check docker client/server version
executionContext.Output("##[group]Checking docker version");
DockerVersion dockerVersion = await _dockerManager.DockerVersion(executionContext);
executionContext.Output("##[endgroup]");
ArgUtil.NotNull(dockerVersion.ServerVersion, nameof(dockerVersion.ServerVersion));
ArgUtil.NotNull(dockerVersion.ClientVersion, nameof(dockerVersion.ClientVersion));
ArgUtil.NotNull(dockerVersion.ServerVersion, nameof(dockerVersion.ServerVersion));
ArgUtil.NotNull(dockerVersion.ClientVersion, nameof(dockerVersion.ClientVersion));
#if OS_WINDOWS
Version requiredDockerEngineAPIVersion = new Version(1, 30); // Docker-EE version 17.6
#else
Version requiredDockerEngineAPIVersion = new Version(1, 35); // Docker-CE version 17.12
Version requiredDockerEngineAPIVersion = new Version(1, 35); // Docker-CE version 17.12
#endif
if (dockerVersion.ServerVersion < requiredDockerEngineAPIVersion)
{
throw new NotSupportedException($"Min required docker engine API server version is '{requiredDockerEngineAPIVersion}', your docker ('{_dockerManager.DockerPath}') server version is '{dockerVersion.ServerVersion}'");
}
if (dockerVersion.ClientVersion < requiredDockerEngineAPIVersion)
{
throw new NotSupportedException($"Min required docker engine API client version is '{requiredDockerEngineAPIVersion}', your docker ('{_dockerManager.DockerPath}') client version is '{dockerVersion.ClientVersion}'");
}
// Clean up containers left by previous runs
executionContext.Output("##[group]Clean up resources from previous jobs");
var staleContainers = await _dockerManager.DockerPS(executionContext, $"--all --quiet --no-trunc --filter \"label={_dockerManager.DockerInstanceLabel}\"");
foreach (var staleContainer in staleContainers)
{
int containerRemoveExitCode = await _dockerManager.DockerRemove(executionContext, staleContainer);
if (containerRemoveExitCode != 0)
if (dockerVersion.ServerVersion < requiredDockerEngineAPIVersion)
{
executionContext.Warning($"Delete stale containers failed, docker rm fail with exit code {containerRemoveExitCode} for container {staleContainer}");
throw new NotSupportedException($"Min required docker engine API server version is '{requiredDockerEngineAPIVersion}', your docker ('{_dockerManager.DockerPath}') server version is '{dockerVersion.ServerVersion}'");
}
if (dockerVersion.ClientVersion < requiredDockerEngineAPIVersion)
{
throw new NotSupportedException($"Min required docker engine API client version is '{requiredDockerEngineAPIVersion}', your docker ('{_dockerManager.DockerPath}') client version is '{dockerVersion.ClientVersion}'");
}
}
int networkPruneExitCode = await _dockerManager.DockerNetworkPrune(executionContext);
if (networkPruneExitCode != 0)
{
executionContext.Warning($"Delete stale container networks failed, docker network prune fail with exit code {networkPruneExitCode}");
}
executionContext.Output("##[endgroup]");
// Clean up containers left by previous runs
executionContext.Output("##[group]Clean up resources from previous jobs");
var staleContainers = await _dockerManager.DockerPS(executionContext, $"--all --quiet --no-trunc --filter \"label={_dockerManager.DockerInstanceLabel}\"");
foreach (var staleContainer in staleContainers)
{
int containerRemoveExitCode = await _dockerManager.DockerRemove(executionContext, staleContainer);
if (containerRemoveExitCode != 0)
{
executionContext.Warning($"Delete stale containers failed, docker rm fail with exit code {containerRemoveExitCode} for container {staleContainer}");
}
}
// Create local docker network for this job to avoid port conflict when multiple runners run on same machine.
// All containers within a job join the same network
executionContext.Output("##[group]Create local container network");
var containerNetwork = $"github_network_{Guid.NewGuid().ToString("N")}";
await CreateContainerNetworkAsync(executionContext, containerNetwork);
executionContext.JobContext.Container["network"] = new StringContextData(containerNetwork);
executionContext.Output("##[endgroup]");
int networkPruneExitCode = await _dockerManager.DockerNetworkPrune(executionContext);
if (networkPruneExitCode != 0)
{
executionContext.Warning($"Delete stale container networks failed, docker network prune fail with exit code {networkPruneExitCode}");
}
executionContext.Output("##[endgroup]");
foreach (var container in containers)
{
container.ContainerNetwork = containerNetwork;
await StartContainerAsync(executionContext, container);
}
// Create local docker network for this job to avoid port conflict when multiple runners run on same machine.
// All containers within a job join the same network
executionContext.Output("##[group]Create local container network");
var containerNetwork = $"github_network_{Guid.NewGuid().ToString("N")}";
await CreateContainerNetworkAsync(executionContext, containerNetwork);
executionContext.JobContext.Container["network"] = new StringContextData(containerNetwork);
executionContext.Output("##[endgroup]");
executionContext.Output("##[group]Waiting for all services to be ready");
foreach (var container in containers.Where(c => !c.IsJobContainer))
{
await ContainerHealthcheck(executionContext, container);
foreach (var container in containers)
{
container.ContainerNetwork = containerNetwork;
await StartContainerAsync(executionContext, container);
}
executionContext.Output("##[group]Waiting for all services to be ready");
foreach (var container in containers.Where(c => !c.IsJobContainer))
{
await ContainerHealthcheck(executionContext, container);
}
executionContext.Output("##[endgroup]");
}
executionContext.Output("##[endgroup]");
}
public async Task StopContainersAsync(IExecutionContext executionContext, object data)
@@ -162,12 +336,69 @@ namespace GitHub.Runner.Worker
List<ContainerInfo> containers = data as List<ContainerInfo>;
ArgUtil.NotNull(containers, nameof(containers));
foreach (var container in containers)
var podManHandler = Path.Combine(HostContext.GetDirectory(WellKnownDirectory.Bin), "kubectlHandler", "index.js");
if (File.Exists(podManHandler))
{
await StopContainerAsync(executionContext, container);
var podmanInput = new ContainerEngineHandlerInput()
{
Command = "Remove",
RemoveInput = new ContainersRemoveInput()
{
Network = executionContext.JobContext.Container["network"].ToString(),
JobContainerId = executionContext.JobContext.Container["id"].ToString()
}
};
ContainerEngineHandlerOutput podmanOutput = null;
using (var processInvoker = HostContext.CreateService<IProcessInvoker>())
{
var redirectStandardIn = Channel.CreateUnbounded<string>(new UnboundedChannelOptions() { SingleReader = true, SingleWriter = true });
redirectStandardIn.Writer.TryWrite(JsonUtility.ToString(podmanInput));
processInvoker.OutputDataReceived += delegate (object sender, ProcessDataReceivedEventArgs message)
{
executionContext.Output(message.Data);
};
processInvoker.ErrorDataReceived += delegate (object sender, ProcessDataReceivedEventArgs message)
{
executionContext.Output(message.Data);
if (podmanOutput == null && message.Data.IndexOf("___CONTAINER_ENGINE_HANDLER_OUTPUT___") >= 0)
{
try
{
podmanOutput = JsonUtility.FromString<ContainerEngineHandlerOutput>(message.Data.Replace("___CONTAINER_ENGINE_HANDLER_OUTPUT___", ""));
}
catch (Exception ex)
{
executionContext.Error(ex);
}
}
};
// Execute the process. Exit code 0 should always be returned.
// A non-zero exit code indicates infrastructural failure.
// Task failure should be communicated over STDOUT using ## commands.
await processInvoker.ExecuteAsync(workingDirectory: HostContext.GetDirectory(WellKnownDirectory.Work),
fileName: Path.Combine(HostContext.GetDirectory(WellKnownDirectory.Externals), "node12", "bin", $"node{IOUtil.ExeExtension}"),
arguments: podManHandler,
environment: null,
requireExitCodeZero: false,
outputEncoding: Encoding.UTF8,
killProcessOnCancel: false,
redirectStandardIn: redirectStandardIn,
cancellationToken: executionContext.CancellationToken);
}
}
else
{
foreach (var container in containers)
{
await StopContainerAsync(executionContext, container);
}
// Remove the container network
await RemoveContainerNetworkAsync(executionContext, containers.First().ContainerNetwork);
}
// Remove the container network
await RemoveContainerNetworkAsync(executionContext, containers.First().ContainerNetwork);
}
private async Task StartContainerAsync(IExecutionContext executionContext, ContainerInfo container)

View File

@@ -52,6 +52,7 @@ namespace GitHub.Runner.Worker
Dictionary<string, VariableValue> JobOutputs { get; }
ActionsEnvironmentReference ActionsEnvironment { get; }
List<ActionsStepTelemetry> ActionsStepsTelemetry { get; }
List<JobTelemetry> JobTelemetry { get; }
DictionaryContextData ExpressionValues { get; }
IList<IFunctionInfo> ExpressionFunctions { get; }
JobContext JobContext { get; }
@@ -150,6 +151,7 @@ namespace GitHub.Runner.Worker
public ActionsEnvironmentReference ActionsEnvironment { get; private set; }
public List<ActionsStepTelemetry> ActionsStepsTelemetry { get; private set; }
public List<JobTelemetry> JobTelemetry { get; private set; }
public DictionaryContextData ExpressionValues { get; } = new DictionaryContextData();
public IList<IFunctionInfo> ExpressionFunctions { get; } = new List<IFunctionInfo>();
@@ -294,6 +296,7 @@ namespace GitHub.Runner.Worker
child.ContextName = contextName;
child.EmbeddedId = embeddedId;
child.SiblingScopeName = siblingScopeName;
child.JobTelemetry = JobTelemetry;
if (intraActionState == null)
{
child.IntraActionState = new Dictionary<string, string>(StringComparer.OrdinalIgnoreCase);
@@ -650,6 +653,8 @@ namespace GitHub.Runner.Worker
// ActionsStepTelemetry
ActionsStepsTelemetry = new List<ActionsStepTelemetry>();
JobTelemetry = new List<JobTelemetry>();
// Service container info
Global.ServiceContainers = new List<ContainerInfo>();
@@ -980,18 +985,6 @@ namespace GitHub.Runner.Worker
context.Write(null, message);
}
public static void WriteDetails(this IExecutionContext context, string message)
{
if (context.IsEmbedded)
{
context.Debug(message);
}
else
{
context.Output(message);
}
}
// Do not add a format string overload. See comment on ExecutionContext.Write().
public static void Command(this IExecutionContext context, string message)
{

View File

@@ -23,9 +23,13 @@ namespace GitHub.Runner.Worker
"job",
"path",
"ref",
"ref_name",
"ref_protected",
"ref_type",
"repository",
"repository_owner",
"retention_days",
"run_attempt",
"run_id",
"run_number",
"server_url",
@@ -38,9 +42,16 @@ namespace GitHub.Runner.Worker
{
foreach (var data in this)
{
if (_contextEnvAllowlist.Contains(data.Key) && data.Value is StringContextData value)
if (_contextEnvAllowlist.Contains(data.Key))
{
yield return new KeyValuePair<string, string>($"GITHUB_{data.Key.ToUpperInvariant()}", value);
if (data.Value is StringContextData value)
{
yield return new KeyValuePair<string, string>($"GITHUB_{data.Key.ToUpperInvariant()}", value);
}
else if (data.Value is BooleanContextData booleanValue)
{
yield return new KeyValuePair<string, string>($"GITHUB_{data.Key.ToUpperInvariant()}", booleanValue.ToString());
}
}
}
}

View File

@@ -50,8 +50,8 @@ namespace GitHub.Runner.Worker.Handlers
var dockerFile = Path.Combine(ActionDirectory, Data.Image);
ArgUtil.File(dockerFile, nameof(Data.Image));
ExecutionContext.WriteDetails(ExecutionContext.IsEmbedded ? "Building docker image" : $"##[group]Building docker image");
ExecutionContext.WriteDetails($"Dockerfile for action: '{dockerFile}'.");
ExecutionContext.Output($"##[group]Building docker image");
ExecutionContext.Output($"Dockerfile for action: '{dockerFile}'.");
var imageName = $"{dockerManager.DockerInstanceLabel}:{ExecutionContext.Id.ToString("N")}";
var buildExitCode = await dockerManager.DockerBuild(
ExecutionContext,
@@ -59,7 +59,7 @@ namespace GitHub.Runner.Worker.Handlers
dockerFile,
Directory.GetParent(dockerFile).FullName,
imageName);
ExecutionContext.WriteDetails(ExecutionContext.IsEmbedded ? "" : "##[endgroup]");
ExecutionContext.Output("##[endgroup]");
if (buildExitCode != 0)
{

View File

@@ -82,7 +82,7 @@ namespace GitHub.Runner.Worker.Handlers
if (stage == ActionRunStage.Post)
{
ExecutionContext.WriteDetails($"Post job cleanup.");
ExecutionContext.Output($"Post job cleanup.");
return;
}
@@ -118,30 +118,30 @@ namespace GitHub.Runner.Worker.Handlers
groupName = "Action details";
}
ExecutionContext.WriteDetails(ExecutionContext.IsEmbedded ? groupName : $"##[group]{groupName}");
ExecutionContext.Output($"##[group]{groupName}");
if (this.Inputs?.Count > 0)
{
ExecutionContext.WriteDetails("with:");
ExecutionContext.Output("with:");
foreach (var input in this.Inputs)
{
if (!string.IsNullOrEmpty(input.Value))
{
ExecutionContext.WriteDetails($" {input.Key}: {input.Value}");
ExecutionContext.Output($" {input.Key}: {input.Value}");
}
}
}
if (this.Environment?.Count > 0)
{
ExecutionContext.WriteDetails("env:");
ExecutionContext.Output("env:");
foreach (var env in this.Environment)
{
ExecutionContext.WriteDetails($" {env.Key}: {env.Value}");
ExecutionContext.Output($" {env.Key}: {env.Value}");
}
}
ExecutionContext.WriteDetails(ExecutionContext.IsEmbedded ? "" : "##[endgroup]");
ExecutionContext.Output("##[endgroup]");
}
public override void Initialize(IHostContext hostContext)

View File

@@ -40,7 +40,7 @@ namespace GitHub.Runner.Worker.Handlers
firstLine = firstLine.Substring(0, firstNewLine);
}
ExecutionContext.WriteDetails(ExecutionContext.IsEmbedded ? $"Run {firstLine}" : $"##[group]Run {firstLine}");
ExecutionContext.Output($"##[group]Run {firstLine}");
}
else
{
@@ -51,7 +51,7 @@ namespace GitHub.Runner.Worker.Handlers
foreach (var line in multiLines)
{
// Bright Cyan color
ExecutionContext.WriteDetails($"\x1b[36;1m{line}\x1b[0m");
ExecutionContext.Output($"\x1b[36;1m{line}\x1b[0m");
}
string argFormat;
@@ -110,23 +110,23 @@ namespace GitHub.Runner.Worker.Handlers
if (!string.IsNullOrEmpty(shellCommandPath))
{
ExecutionContext.WriteDetails($"shell: {shellCommandPath} {argFormat}");
ExecutionContext.Output($"shell: {shellCommandPath} {argFormat}");
}
else
{
ExecutionContext.WriteDetails($"shell: {shellCommand} {argFormat}");
ExecutionContext.Output($"shell: {shellCommand} {argFormat}");
}
if (this.Environment?.Count > 0)
{
ExecutionContext.WriteDetails("env:");
ExecutionContext.Output("env:");
foreach (var env in this.Environment)
{
ExecutionContext.WriteDetails($" {env.Key}: {env.Value}");
ExecutionContext.Output($" {env.Key}: {env.Value}");
}
}
ExecutionContext.WriteDetails(ExecutionContext.IsEmbedded ? "" : "##[endgroup]");
ExecutionContext.Output("##[endgroup]");
}
public async Task RunAsync(ActionRunStage stage)

View File

@@ -21,6 +21,8 @@ namespace GitHub.Runner.Worker.Handlers
event EventHandler<ProcessDataReceivedEventArgs> OutputDataReceived;
event EventHandler<ProcessDataReceivedEventArgs> ErrorDataReceived;
IExecutionContext ExecutionContext { get; set; }
string ResolvePathForStepHost(string path);
Task<string> DetermineNodeRuntimeVersion(IExecutionContext executionContext);
@@ -53,6 +55,8 @@ namespace GitHub.Runner.Worker.Handlers
public event EventHandler<ProcessDataReceivedEventArgs> OutputDataReceived;
public event EventHandler<ProcessDataReceivedEventArgs> ErrorDataReceived;
public IExecutionContext ExecutionContext { get; set; }
public string ResolvePathForStepHost(string path)
{
return path;
@@ -99,6 +103,8 @@ namespace GitHub.Runner.Worker.Handlers
public event EventHandler<ProcessDataReceivedEventArgs> OutputDataReceived;
public event EventHandler<ProcessDataReceivedEventArgs> ErrorDataReceived;
public IExecutionContext ExecutionContext { get; set; }
public string ResolvePathForStepHost(string path)
{
// make sure container exist.
@@ -174,69 +180,138 @@ namespace GitHub.Runner.Worker.Handlers
ArgUtil.NotNull(Container, nameof(Container));
ArgUtil.NotNullOrEmpty(Container.ContainerId, nameof(Container.ContainerId));
var dockerManager = HostContext.GetService<IDockerCommandManager>();
string dockerClientPath = dockerManager.DockerPath;
// Usage: docker exec [OPTIONS] CONTAINER COMMAND [ARG...]
IList<string> dockerCommandArgs = new List<string>();
dockerCommandArgs.Add($"exec");
// [OPTIONS]
dockerCommandArgs.Add($"-i");
dockerCommandArgs.Add($"--workdir {workingDirectory}");
foreach (var env in environment)
var podManHandler = Path.Combine(HostContext.GetDirectory(WellKnownDirectory.Bin), "kubectlHandler", "index.js");
if (File.Exists(podManHandler))
{
// e.g. -e MY_SECRET maps the value into the exec'ed process without exposing
// the value directly in the command
dockerCommandArgs.Add($"-e {env.Key}");
var podmanInput = new ContainerEngineHandlerInput()
{
Command = "Exec",
ExecInput = new JobContainerExecInput()
{
JobContainer = this.Container,
WorkingDirectory = workingDirectory,
FileName = fileName,
Arguments = arguments,
EnvironmentKeys = environment.Keys.ToList(),
EnvironmentVariables = environment.ToDictionary(x => x.Key, y => y.Value)
}
};
// make sure all env are using container path
foreach (var envKey in environment.Keys.ToList())
{
environment[envKey] = this.Container.TranslateToContainerPath(environment[envKey]);
}
// ContainerEngineHandlerOutput podmanOutput = null;
using (var processInvoker = HostContext.CreateService<IProcessInvoker>())
{
var redirectStandardIn = Channel.CreateUnbounded<string>(new UnboundedChannelOptions() { SingleReader = true, SingleWriter = true });
redirectStandardIn.Writer.TryWrite(JsonUtility.ToString(podmanInput));
// processInvoker.OutputDataReceived += delegate (object sender, ProcessDataReceivedEventArgs message)
// {
// ExecutionContext.Output(message.Data);
// };
// processInvoker.ErrorDataReceived += delegate (object sender, ProcessDataReceivedEventArgs message)
// {
// executionContext.Output(message.Data);
// if (podmanOutput == null && message.Data.IndexOf("___CONTAINER_ENGINE_HANDLER_OUTPUT___") >= 0)
// {
// try
// {
// podmanOutput = JsonUtility.FromString<ContainerEngineHandlerOutput>(message.Data.Replace("___CONTAINER_ENGINE_HANDLER_OUTPUT___", ""));
// }
// catch (Exception ex)
// {
// executionContext.Error(ex);
// }
// }
// };
processInvoker.OutputDataReceived += OutputDataReceived;
processInvoker.ErrorDataReceived += ErrorDataReceived;
// Execute the process. Exit code 0 should always be returned.
// A non-zero exit code indicates infrastructural failure.
// Task failure should be communicated over STDOUT using ## commands.
return await processInvoker.ExecuteAsync(workingDirectory: HostContext.GetDirectory(WellKnownDirectory.Work),
fileName: Path.Combine(HostContext.GetDirectory(WellKnownDirectory.Externals), "node12", "bin", $"node{IOUtil.ExeExtension}"),
arguments: podManHandler,
environment: environment,
requireExitCodeZero: requireExitCodeZero,
outputEncoding: Encoding.UTF8,
killProcessOnCancel: killProcessOnCancel,
redirectStandardIn: redirectStandardIn,
cancellationToken: cancellationToken);
}
}
if (!string.IsNullOrEmpty(PrependPath))
else
{
// Prepend tool paths to container's PATH
var fullPath = !string.IsNullOrEmpty(Container.ContainerRuntimePath) ? $"{PrependPath}:{Container.ContainerRuntimePath}" : PrependPath;
dockerCommandArgs.Add($"-e PATH=\"{fullPath}\"");
}
var dockerManager = HostContext.GetService<IDockerCommandManager>();
string dockerClientPath = dockerManager.DockerPath;
// CONTAINER
dockerCommandArgs.Add($"{Container.ContainerId}");
// Usage: docker exec [OPTIONS] CONTAINER COMMAND [ARG...]
IList<string> dockerCommandArgs = new List<string>();
dockerCommandArgs.Add($"exec");
// COMMAND
dockerCommandArgs.Add(fileName);
// [OPTIONS]
dockerCommandArgs.Add($"-i");
dockerCommandArgs.Add($"--workdir {workingDirectory}");
foreach (var env in environment)
{
// e.g. -e MY_SECRET maps the value into the exec'ed process without exposing
// the value directly in the command
dockerCommandArgs.Add($"-e {env.Key}");
}
if (!string.IsNullOrEmpty(PrependPath))
{
// Prepend tool paths to container's PATH
var fullPath = !string.IsNullOrEmpty(Container.ContainerRuntimePath) ? $"{PrependPath}:{Container.ContainerRuntimePath}" : PrependPath;
dockerCommandArgs.Add($"-e PATH=\"{fullPath}\"");
}
// [ARG...]
dockerCommandArgs.Add(arguments);
// CONTAINER
dockerCommandArgs.Add($"{Container.ContainerId}");
string dockerCommandArgstring = string.Join(" ", dockerCommandArgs);
// COMMAND
dockerCommandArgs.Add(fileName);
// make sure all env are using container path
foreach (var envKey in environment.Keys.ToList())
{
environment[envKey] = this.Container.TranslateToContainerPath(environment[envKey]);
}
// [ARG...]
dockerCommandArgs.Add(arguments);
using (var processInvoker = HostContext.CreateService<IProcessInvoker>())
{
processInvoker.OutputDataReceived += OutputDataReceived;
processInvoker.ErrorDataReceived += ErrorDataReceived;
string dockerCommandArgstring = string.Join(" ", dockerCommandArgs);
// make sure all env are using container path
foreach (var envKey in environment.Keys.ToList())
{
environment[envKey] = this.Container.TranslateToContainerPath(environment[envKey]);
}
using (var processInvoker = HostContext.CreateService<IProcessInvoker>())
{
processInvoker.OutputDataReceived += OutputDataReceived;
processInvoker.ErrorDataReceived += ErrorDataReceived;
#if OS_WINDOWS
// It appears that node.exe outputs UTF8 when not in TTY mode.
outputEncoding = Encoding.UTF8;
#else
// Let .NET choose the default.
outputEncoding = null;
// Let .NET choose the default.
outputEncoding = null;
#endif
return await processInvoker.ExecuteAsync(workingDirectory: HostContext.GetDirectory(WellKnownDirectory.Work),
fileName: dockerClientPath,
arguments: dockerCommandArgstring,
environment: environment,
requireExitCodeZero: requireExitCodeZero,
outputEncoding: outputEncoding,
killProcessOnCancel: killProcessOnCancel,
redirectStandardIn: null,
inheritConsoleHandler: inheritConsoleHandler,
cancellationToken: cancellationToken);
return await processInvoker.ExecuteAsync(workingDirectory: HostContext.GetDirectory(WellKnownDirectory.Work),
fileName: dockerClientPath,
arguments: dockerCommandArgstring,
environment: environment,
requireExitCodeZero: requireExitCodeZero,
outputEncoding: outputEncoding,
killProcessOnCancel: killProcessOnCancel,
redirectStandardIn: null,
inheritConsoleHandler: inheritConsoleHandler,
cancellationToken: cancellationToken);
}
}
}
}

View File

@@ -350,6 +350,7 @@ namespace GitHub.Runner.Worker
case "":
case "ERROR":
case "WARNING":
case "NOTICE":
break;
default:
throw new ArgumentException($"Matcher '{_owner}' contains unexpected default severity '{_severity}'");

View File

@@ -7,6 +7,7 @@ using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Text;
using System.Threading;
using System.Threading.Tasks;
using System.Net.Http;
@@ -106,6 +107,9 @@ namespace GitHub.Runner.Worker
jobContext.SetRunnerContext("os", VarUtil.OS);
var runnerSettings = HostContext.GetService<IConfigurationStore>().GetSettings();
jobContext.SetRunnerContext("name", runnerSettings.AgentName);
string toolsDirectory = HostContext.GetDirectory(WellKnownDirectory.Tools);
Directory.CreateDirectory(toolsDirectory);
jobContext.SetRunnerContext("tool_cache", toolsDirectory);
@@ -145,6 +149,16 @@ namespace GitHub.Runner.Worker
Trace.Verbose($"Job steps: '{string.Join(", ", jobSteps.Select(x => x.DisplayName))}'");
HostContext.WritePerfCounter($"WorkerJobInitialized_{message.RequestId.ToString()}");
if (systemConnection.Data.TryGetValue("GenerateIdTokenUrl", out var generateIdTokenUrl) &&
!string.IsNullOrEmpty(generateIdTokenUrl))
{
// Server won't issue ID_TOKEN for non-inprogress job.
// If the job is trying to use OIDC feature, we want the job to be marked as in-progress before running any customer's steps as much as we can.
// Timeline record update background process runs every 500ms, so delay 1000ms is enough for most of the cases
Trace.Info($"Waiting for job to be marked as started.");
await Task.WhenAny(_jobServerQueue.JobRecordUpdated.Task, Task.Delay(1000));
}
// Run all job steps
Trace.Info("Run all job steps.");
var stepsRunner = HostContext.GetService<IStepsRunner>();
@@ -215,8 +229,15 @@ namespace GitHub.Runner.Worker
return result;
}
// Load any upgrade telemetry
LoadFromTelemetryFile(jobContext.JobTelemetry);
// Make sure we don't submit secrets as telemetry
MaskTelemetrySecrets(jobContext.JobTelemetry);
Trace.Info("Raising job completed event.");
var jobCompletedEvent = new JobCompletedEvent(message.RequestId, message.JobId, result, jobContext.JobOutputs, jobContext.ActionsEnvironment, jobContext.ActionsStepsTelemetry);
var jobCompletedEvent = new JobCompletedEvent(message.RequestId, message.JobId, result, jobContext.JobOutputs, jobContext.ActionsEnvironment, jobContext.ActionsStepsTelemetry, jobContext.JobTelemetry);
var completeJobRetryLimit = 5;
var exceptions = new List<Exception>();
@@ -260,6 +281,38 @@ namespace GitHub.Runner.Worker
throw new AggregateException(exceptions);
}
private void MaskTelemetrySecrets(List<JobTelemetry> jobTelemetry)
{
foreach (var telemetryItem in jobTelemetry)
{
telemetryItem.Message = HostContext.SecretMasker.MaskSecrets(telemetryItem.Message);
}
}
private void LoadFromTelemetryFile(List<JobTelemetry> jobTelemetry)
{
try
{
var telemetryFilePath = HostContext.GetConfigFile(WellKnownConfigFile.Telemetry);
if (File.Exists(telemetryFilePath))
{
var telemetryData = File.ReadAllText(telemetryFilePath, Encoding.UTF8);
var telemetry = new JobTelemetry
{
Message = $"Runner File Telemetry:\n{telemetryData}",
Type = JobTelemetryType.General
};
jobTelemetry.Add(telemetry);
IOUtil.DeleteFile(telemetryFilePath);
}
}
catch (Exception e)
{
Trace.Error("Error when trying to load telemetry from telemetry file");
Trace.Error(e);
}
}
private async Task ShutdownQueue(bool throwOnFailure)
{
if (_jobServerQueue != null)

View File

@@ -155,6 +155,19 @@ namespace GitHub.DistributedTask.WebApi
this.ActionsStepsTelemetry = actionsStepsTelemetry;
}
public JobCompletedEvent(
Int64 requestId,
Guid jobId,
TaskResult result,
Dictionary<String, VariableValue> outputs,
ActionsEnvironmentReference actionsEnvironment,
List<ActionsStepTelemetry> actionsStepsTelemetry,
List<JobTelemetry> jobTelemetry)
: this(requestId, jobId, result, outputs, actionsEnvironment, actionsStepsTelemetry)
{
this.JobTelemetry = jobTelemetry;
}
[DataMember(EmitDefaultValue = false)]
public Int64 RequestId
{
@@ -189,6 +202,13 @@ namespace GitHub.DistributedTask.WebApi
get;
set;
}
[DataMember(EmitDefaultValue = false)]
public List<JobTelemetry> JobTelemetry
{
get;
set;
}
}
[DataContract]

View File

@@ -0,0 +1,17 @@
using System.Runtime.Serialization;
namespace GitHub.DistributedTask.WebApi
{
/// <summary>
/// Information about a job run on the runner
/// </summary>
[DataContract]
public class JobTelemetry
{
[DataMember(EmitDefaultValue = false)]
public string Message { get; set; }
[DataMember(EmitDefaultValue = false)]
public JobTelemetryType Type { get; set; }
}
}

View File

@@ -0,0 +1,13 @@
using System.Runtime.Serialization;
namespace GitHub.DistributedTask.WebApi
{
public enum JobTelemetryType
{
[EnumMember]
General = 0,
[EnumMember]
ActionCommand = 1,
}
}

View File

@@ -24,6 +24,7 @@ namespace GitHub.DistributedTask.WebApi
this.OSDescription = referenceToBeCloned.OSDescription;
this.ProvisioningState = referenceToBeCloned.ProvisioningState;
this.AccessPoint = referenceToBeCloned.AccessPoint;
this.Ephemeral = referenceToBeCloned.Ephemeral;
if (referenceToBeCloned.m_links != null)
{
@@ -81,6 +82,16 @@ namespace GitHub.DistributedTask.WebApi
set;
}
/// <summary>
/// Signifies that this Agent can only run one job and will be removed by the server after that one job finish.
/// </summary>
[DataMember]
public bool? Ephemeral
{
get;
set;
}
/// <summary>
/// Whether or not the agent is online.
/// </summary>

View File

@@ -161,7 +161,8 @@ namespace GitHub.Runner.Common.Tests.Listener.Configuration
"--work", _expectedWorkFolder,
"--auth", _expectedAuthType,
"--token", _expectedToken,
"--labels", userLabels
"--labels", userLabels,
"--ephemeral",
});
trace.Info("Constructed.");
_store.Setup(x => x.IsConfigured()).Returns(false);
@@ -179,6 +180,7 @@ namespace GitHub.Runner.Common.Tests.Listener.Configuration
Assert.True(s.AgentName.Equals(_expectedAgentName));
Assert.True(s.PoolId.Equals(_secondRunnerGroupId));
Assert.True(s.WorkFolder.Equals(_expectedWorkFolder));
Assert.True(s.Ephemeral.Equals(true));
// validate GetAgentPoolsAsync gets called twice with automation pool type
_runnerServer.Verify(x => x.GetAgentPoolsAsync(It.IsAny<string>(), It.Is<TaskAgentPoolType>(p => p == TaskAgentPoolType.Automation)), Times.Exactly(2));

View File

@@ -264,6 +264,170 @@ namespace GitHub.Runner.Common.Tests.Listener
}
}
[Fact]
[Trait("Level", "L0")]
[Trait("Category", "Runner")]
public async void RenewJobRequestNewAgentNameUpdatesSettings()
{
//Arrange
using (var hc = new TestHostContext(this))
{
var count = 0;
var oldName = "OldName";
var newName = "NewName";
var oldSettings = new RunnerSettings { AgentName = oldName };
var reservedAgent = new TaskAgentReference { Name = newName };
var trace = hc.GetTrace(nameof(DispatcherRenewJobRequestStopOnJobTokenExpiredExceptions));
TaskCompletionSource<int> firstJobRequestRenewed = new TaskCompletionSource<int>();
CancellationTokenSource cancellationTokenSource = new CancellationTokenSource();
var request = new Mock<TaskAgentJobRequest>();
request.Object.ReservedAgent = reservedAgent;
PropertyInfo lockUntilProperty = request.Object.GetType().GetProperty("LockedUntil", BindingFlags.Instance | BindingFlags.NonPublic | BindingFlags.Public);
Assert.NotNull(lockUntilProperty);
lockUntilProperty.SetValue(request.Object, DateTime.UtcNow.AddMinutes(5));
hc.SetSingleton<IRunnerServer>(_runnerServer.Object);
hc.SetSingleton<IConfigurationStore>(_configurationStore.Object);
_configurationStore.Setup(x => x.GetSettings()).Returns(oldSettings);
_runnerServer.Setup(x => x.RenewAgentRequestAsync(It.IsAny<int>(), It.IsAny<long>(), It.IsAny<Guid>(), It.IsAny<string>(), It.IsAny<CancellationToken>()))
.Returns(() =>
{
count++;
if (count < 5)
{
return Task.FromResult<TaskAgentJobRequest>(request.Object);
}
else if (count == 5 || count == 6 || count == 7)
{
throw new TimeoutException("");
}
else
{
cancellationTokenSource.Cancel();
return Task.FromResult<TaskAgentJobRequest>(request.Object);
}
});
var jobDispatcher = new JobDispatcher();
jobDispatcher.Initialize(hc);
// Act
await jobDispatcher.RenewJobRequestAsync(0, 0, Guid.Empty, Guid.NewGuid().ToString(), firstJobRequestRenewed, cancellationTokenSource.Token);
// Assert
_configurationStore.Verify(x => x.SaveSettings(It.Is<RunnerSettings>(settings => settings.AgentName == newName)), Times.Once);
}
}
[Fact]
[Trait("Level", "L0")]
[Trait("Category", "Runner")]
public async void RenewJobRequestSameAgentNameIgnored()
{
//Arrange
using (var hc = new TestHostContext(this))
{
var count = 0;
var oldName = "OldName";
var newName = "OldName";
var oldSettings = new RunnerSettings { AgentName = oldName };
var reservedAgent = new TaskAgentReference { Name = newName };
var trace = hc.GetTrace(nameof(DispatcherRenewJobRequestStopOnJobTokenExpiredExceptions));
TaskCompletionSource<int> firstJobRequestRenewed = new TaskCompletionSource<int>();
CancellationTokenSource cancellationTokenSource = new CancellationTokenSource();
var request = new Mock<TaskAgentJobRequest>();
request.Object.ReservedAgent = reservedAgent;
PropertyInfo lockUntilProperty = request.Object.GetType().GetProperty("LockedUntil", BindingFlags.Instance | BindingFlags.NonPublic | BindingFlags.Public);
Assert.NotNull(lockUntilProperty);
lockUntilProperty.SetValue(request.Object, DateTime.UtcNow.AddMinutes(5));
hc.SetSingleton<IRunnerServer>(_runnerServer.Object);
hc.SetSingleton<IConfigurationStore>(_configurationStore.Object);
_configurationStore.Setup(x => x.GetSettings()).Returns(oldSettings);
_runnerServer.Setup(x => x.RenewAgentRequestAsync(It.IsAny<int>(), It.IsAny<long>(), It.IsAny<Guid>(), It.IsAny<string>(), It.IsAny<CancellationToken>()))
.Returns(() =>
{
count++;
if (count < 5)
{
return Task.FromResult<TaskAgentJobRequest>(request.Object);
}
else if (count == 5 || count == 6 || count == 7)
{
throw new TimeoutException("");
}
else
{
cancellationTokenSource.Cancel();
return Task.FromResult<TaskAgentJobRequest>(request.Object);
}
});
var jobDispatcher = new JobDispatcher();
jobDispatcher.Initialize(hc);
// Act
await jobDispatcher.RenewJobRequestAsync(0, 0, Guid.Empty, Guid.NewGuid().ToString(), firstJobRequestRenewed, cancellationTokenSource.Token);
// Assert
_configurationStore.Verify(x => x.SaveSettings(It.IsAny<RunnerSettings>()), Times.Never);
}
}
[Fact]
[Trait("Level", "L0")]
[Trait("Category", "Runner")]
public async void RenewJobRequestNullAgentNameIgnored()
{
//Arrange
using (var hc = new TestHostContext(this))
{
var count = 0;
var oldName = "OldName";
var oldSettings = new RunnerSettings { AgentName = oldName };
var trace = hc.GetTrace(nameof(DispatcherRenewJobRequestStopOnJobTokenExpiredExceptions));
TaskCompletionSource<int> firstJobRequestRenewed = new TaskCompletionSource<int>();
CancellationTokenSource cancellationTokenSource = new CancellationTokenSource();
var request = new Mock<TaskAgentJobRequest>();
PropertyInfo lockUntilProperty = request.Object.GetType().GetProperty("LockedUntil", BindingFlags.Instance | BindingFlags.NonPublic | BindingFlags.Public);
Assert.NotNull(lockUntilProperty);
lockUntilProperty.SetValue(request.Object, DateTime.UtcNow.AddMinutes(5));
hc.SetSingleton<IRunnerServer>(_runnerServer.Object);
hc.SetSingleton<IConfigurationStore>(_configurationStore.Object);
_configurationStore.Setup(x => x.GetSettings()).Returns(oldSettings);
_runnerServer.Setup(x => x.RenewAgentRequestAsync(It.IsAny<int>(), It.IsAny<long>(), It.IsAny<Guid>(), It.IsAny<string>(), It.IsAny<CancellationToken>()))
.Returns(() =>
{
count++;
if (count < 5)
{
return Task.FromResult<TaskAgentJobRequest>(request.Object);
}
else if (count == 5 || count == 6 || count == 7)
{
throw new TimeoutException("");
}
else
{
cancellationTokenSource.Cancel();
return Task.FromResult<TaskAgentJobRequest>(request.Object);
}
});
var jobDispatcher = new JobDispatcher();
jobDispatcher.Initialize(hc);
// Act
await jobDispatcher.RenewJobRequestAsync(0, 0, Guid.Empty, Guid.NewGuid().ToString(), firstJobRequestRenewed, cancellationTokenSource.Token);
// Assert
_configurationStore.Verify(x => x.SaveSettings(It.IsAny<RunnerSettings>()), Times.Never);
}
}
[Fact]
[Trait("Level", "L0")]
[Trait("Category", "Runner")]

View File

@@ -149,6 +149,9 @@ namespace GitHub.Runner.Common.Tests.Listener
_messageListener.Verify(x => x.CreateSessionAsync(It.IsAny<CancellationToken>()), Times.Once());
_messageListener.Verify(x => x.DeleteSessionAsync(), Times.Once());
_messageListener.Verify(x => x.DeleteMessageAsync(It.IsAny<TaskAgentMessage>()), Times.AtLeastOnce());
// verify that we didn't try to delete local settings file (since we're not ephemeral)
_configurationManager.Verify(x => x.DeleteLocalRunnerConfig(), Times.Never());
}
}
}
@@ -243,7 +246,8 @@ namespace GitHub.Runner.Common.Tests.Listener
runner.Initialize(hc);
var settings = new RunnerSettings
{
PoolId = 43242
PoolId = 43242,
Ephemeral = true
};
var message = new TaskAgentMessage()
@@ -294,7 +298,7 @@ namespace GitHub.Runner.Common.Tests.Listener
_configStore.Setup(x => x.IsServiceConfigured()).Returns(false);
//Act
var command = new CommandSettings(hc, new string[] { "run", "--once" });
var command = new CommandSettings(hc, new string[] { "run" });
Task<int> runnerTask = runner.ExecuteCommand(command);
//Assert
@@ -311,6 +315,9 @@ namespace GitHub.Runner.Common.Tests.Listener
_messageListener.Verify(x => x.CreateSessionAsync(It.IsAny<CancellationToken>()), Times.Once());
_messageListener.Verify(x => x.DeleteSessionAsync(), Times.Once());
_messageListener.Verify(x => x.DeleteMessageAsync(It.IsAny<TaskAgentMessage>()), Times.AtLeastOnce());
// verify that we did try to delete local settings file (since we're ephemeral)
_configurationManager.Verify(x => x.DeleteLocalRunnerConfig(), Times.Once());
}
}
@@ -332,7 +339,8 @@ namespace GitHub.Runner.Common.Tests.Listener
runner.Initialize(hc);
var settings = new RunnerSettings
{
PoolId = 43242
PoolId = 43242,
Ephemeral = true
};
var message1 = new TaskAgentMessage()
@@ -390,7 +398,7 @@ namespace GitHub.Runner.Common.Tests.Listener
_configStore.Setup(x => x.IsServiceConfigured()).Returns(false);
//Act
var command = new CommandSettings(hc, new string[] { "run", "--once" });
var command = new CommandSettings(hc, new string[] { "run" });
Task<int> runnerTask = runner.ExecuteCommand(command);
//Assert
@@ -431,7 +439,8 @@ namespace GitHub.Runner.Common.Tests.Listener
var settings = new RunnerSettings
{
PoolId = 43242,
AgentId = 5678
AgentId = 5678,
Ephemeral = true
};
var message1 = new TaskAgentMessage()
@@ -475,7 +484,7 @@ namespace GitHub.Runner.Common.Tests.Listener
_configStore.Setup(x => x.IsServiceConfigured()).Returns(false);
//Act
var command = new CommandSettings(hc, new string[] { "run", "--once" });
var command = new CommandSettings(hc, new string[] { "run" });
Task<int> runnerTask = runner.ExecuteCommand(command);
//Assert

View File

@@ -2,6 +2,7 @@
using System.Collections.Generic;
using System.IO;
using System.Runtime.CompilerServices;
using GitHub.DistributedTask.Pipelines.ContextData;
using GitHub.DistributedTask.WebApi;
using GitHub.Runner.Worker;
using GitHub.Runner.Worker.Container;
@@ -83,6 +84,7 @@ namespace GitHub.Runner.Common.Tests.Worker
{
using (TestHostContext hc = CreateTestContext())
{
_ec.Setup(x => x.ExpressionValues).Returns(GetExpressionValues());
_ec.Setup(x => x.Write(It.IsAny<string>(), It.IsAny<string>()))
.Returns((string tag, string line) =>
{
@@ -105,6 +107,88 @@ namespace GitHub.Runner.Common.Tests.Worker
}
}
[Theory]
[InlineData("stop-commands", "1")]
[InlineData("", "1")]
[InlineData("set-env", "1")]
[InlineData("stop-commands", "true")]
[InlineData("", "true")]
[InlineData("set-env", "true")]
[Trait("Level", "L0")]
[Trait("Category", "Worker")]
public void StopProcessCommand__AllowsInvalidStopTokens__IfEnvVarIsSet(string invalidToken, string allowUnsupportedStopCommandTokens)
{
using (TestHostContext hc = CreateTestContext())
{
_ec.Object.Global.EnvironmentVariables = new Dictionary<string, string>();
var expressionValues = new DictionaryContextData
{
["env"] =
#if OS_WINDOWS
new DictionaryContextData{ { Constants.Variables.Actions.AllowUnsupportedStopCommandTokens, new StringContextData(allowUnsupportedStopCommandTokens) }}
#else
new CaseSensitiveDictionaryContextData{ { Constants.Variables.Actions.AllowUnsupportedStopCommandTokens, new StringContextData(allowUnsupportedStopCommandTokens) }}
#endif
};
_ec.Setup(x => x.ExpressionValues).Returns(expressionValues);
_ec.Setup(x => x.JobTelemetry).Returns(new List<JobTelemetry>());
Assert.True(_commandManager.TryProcessCommand(_ec.Object, $"::stop-commands::{invalidToken}", null));
}
}
[Theory]
[InlineData("stop-commands")]
[InlineData("")]
[InlineData("set-env")]
[Trait("Level", "L0")]
[Trait("Category", "Worker")]
public void StopProcessCommand__FailOnInvalidStopTokens(string invalidToken)
{
using (TestHostContext hc = CreateTestContext())
{
_ec.Object.Global.EnvironmentVariables = new Dictionary<string, string>();
_ec.Setup(x => x.ExpressionValues).Returns(GetExpressionValues());
_ec.Setup(x => x.JobTelemetry).Returns(new List<JobTelemetry>());
Assert.Throws<Exception>(() => _commandManager.TryProcessCommand(_ec.Object, $"::stop-commands::{invalidToken}", null));
}
}
[Fact]
[Trait("Level", "L0")]
[Trait("Category", "Worker")]
public void StopProcessCommandAcceptsValidToken()
{
var validToken = "randomToken";
using (TestHostContext hc = CreateTestContext())
{
_ec.Setup(x => x.ExpressionValues).Returns(GetExpressionValues());
Assert.True(_commandManager.TryProcessCommand(_ec.Object, $"::stop-commands::{validToken}", null));
Assert.False(_commandManager.TryProcessCommand(_ec.Object, "##[set-env name=foo]bar", null));
Assert.True(_commandManager.TryProcessCommand(_ec.Object, $"::{validToken}::", null));
Assert.True(_commandManager.TryProcessCommand(_ec.Object, "##[set-env name=foo]bar", null));
}
}
[Fact]
[Trait("Level", "L0")]
[Trait("Category", "Worker")]
public void StopProcessCommandMasksValidTokenForEntireRun()
{
var validToken = "randomToken";
using (TestHostContext hc = CreateTestContext())
{
_ec.Setup(x => x.ExpressionValues).Returns(GetExpressionValues());
Assert.True(_commandManager.TryProcessCommand(_ec.Object, $"::stop-commands::{validToken}", null));
Assert.False(_commandManager.TryProcessCommand(_ec.Object, "##[set-env name=foo]bar", null));
Assert.Equal("***", hc.SecretMasker.MaskSecrets(validToken));
Assert.True(_commandManager.TryProcessCommand(_ec.Object, $"::{validToken}::", null));
Assert.True(_commandManager.TryProcessCommand(_ec.Object, "##[set-env name=foo]bar", null));
Assert.Equal("***", hc.SecretMasker.MaskSecrets(validToken));
}
}
[Fact]
[Trait("Level", "L0")]
[Trait("Category", "Worker")]
@@ -202,7 +286,7 @@ namespace GitHub.Runner.Common.Tests.Worker
return 1;
});
var registeredCommands = new HashSet<string>(new string[1]{ "warning" });
var registeredCommands = new HashSet<string>(new string[1] { "warning" });
ActionCommand command;
// Columns when lines are different
@@ -375,5 +459,19 @@ namespace GitHub.Runner.Common.Tests.Worker
return hostContext;
}
private DictionaryContextData GetExpressionValues()
{
return new DictionaryContextData
{
["env"] =
#if OS_WINDOWS
new DictionaryContextData()
#else
new CaseSensitiveDictionaryContextData()
#endif
};
}
}
}

View File

@@ -392,6 +392,35 @@ namespace GitHub.Runner.Common.Tests.Worker
Assert.Equal("not-working", match.Message);
}
[Fact]
[Trait("Level", "L0")]
[Trait("Category", "Worker")]
public void Matcher_MultiplePatterns_DefaultSeverityNotice()
{
var config = JsonUtility.FromString<IssueMatchersConfig>(@"
{
""problemMatcher"": [
{
""owner"": ""myMatcher"",
""severity"": ""notice"",
""pattern"": [
{
""regexp"": ""^(.+)$"",
""message"": 1
}
]
}
]
}
");
config.Validate();
var matcher = new IssueMatcher(config.Matchers[0], TimeSpan.FromSeconds(1));
var match = matcher.Match("just-a-notice");
Assert.Equal("notice", match.Severity);
Assert.Equal("just-a-notice", match.Message);
}
[Fact]
[Trait("Level", "L0")]
[Trait("Category", "Worker")]

View File

@@ -2,10 +2,10 @@
<Project ToolsVersion="14.0" DefaultTargets="Build"
xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<Target Name="GenerateConstant">
<Exec Command="git rev-parse HEAD" ConsoleToMSBuild="true">
<!-- <Exec Command="git rev-parse HEAD" ConsoleToMSBuild="true">
<Output TaskParameter="ConsoleOutput" PropertyName="GitInfoCommitHash" />
</Exec>
<Message Text="Building $(Product): $(GitInfoCommitHash) --- $(PackageRuntime)" Importance="high"/>
</Exec> -->
<Message Text="Building $(Product): --- $(PackageRuntime)" Importance="high"/>
<ItemGroup>
<BuildConstants Include="namespace GitHub.Runner.Sdk"/>
@@ -14,7 +14,7 @@
<BuildConstants Include="%20%20%20%20{"/>
<BuildConstants Include="%20%20%20%20%20%20%20%20public static class Source"/>
<BuildConstants Include="%20%20%20%20%20%20%20%20{"/>
<BuildConstants Include="%20%20%20%20%20%20%20%20%20%20%20%20public static readonly string CommitHash = %22$(GitInfoCommitHash)%22%3B"/>
<BuildConstants Include="%20%20%20%20%20%20%20%20%20%20%20%20public static readonly string CommitHash = %22dfcfae49e59b6dc3c2bb5295c649b33c4b49c964%22%3B"/>
<BuildConstants Include="%20%20%20%20%20%20%20%20}%0A"/>
<BuildConstants Include="%20%20%20%20%20%20%20%20public static class RunnerPackage"/>
<BuildConstants Include="%20%20%20%20%20%20%20%20{"/>
@@ -27,7 +27,6 @@
<WriteLinesToFile File="Runner.Sdk/BuildConstants.cs" Lines="@(BuildConstants)" Overwrite="true" />
<Exec Command="git update-index --assume-unchanged ./Runner.Sdk/BuildConstants.cs" ConsoleToMSBuild="true" />
</Target>
<ItemGroup>

View File

@@ -1 +1 @@
2.281.0
2.283.3