Compare commits

..

64 Commits

Author SHA1 Message Date
Ferenc Hammerl
c75a77df66 Update releaseVersion 2021-11-29 17:22:57 +01:00
Ferenc Hammerl
6332f9a42f Prepare for runner 2.285.0 release (#1520) 2021-11-29 16:07:56 +00:00
Tingluo Huang
5b8ff174c6 Add telemetry around runner update process. (#1497)
* Add telemetry around runner update process.

* .

* .

* .
2021-11-22 18:27:57 -05:00
Tingluo Huang
e3e977fd84 Support node.js 16 and bump node.js 12 version. (#1439)
* Support node.js 16 and bump node.js 12 version.

* L0
2021-11-18 15:25:33 -05:00
Ferenc Hammerl
4dc8a09db3 Only execute post for actions that have one (#1481)
* Only execute post for actions that have one

* Revert haspost check

* Remove launch commit

* Remove comment

* Restore whitespace

* Restore wspace
2021-11-18 17:56:13 +01:00
Laura Yu
dcc5d34ad1 Add secret source to start job step (#1411)
* Add secret source to start job step

WIP

* Update to use GetGitHubContext to grab source info

* Update JobExtensionL0.cs

* Update JobExtension.cs

* Update JobExtension.cs
2021-11-17 17:09:38 -05:00
eric sciple
3e34fb10c1 improve telemetry to better diagnose runner configuration issues (#1487) 2021-11-15 13:42:57 -06:00
Tingluo Huang
23a693aa2c Update README.md 2021-11-09 14:19:46 -05:00
Tingluo Huang
eb36db8ff9 Try to delete portable-net45+win8 from all projects (#1470)
* Update Runner.Common.csproj

* Update Runner.Listener.csproj

* Update Runner.PluginHost.csproj

* Update Runner.Plugins.csproj

* Update Runner.Sdk.csproj

* Update Runner.Worker.csproj

* Update Sdk.csproj

* Update Test.csproj
2021-11-09 10:53:53 -05:00
Julio Barba
85e1927754 Prepare for runner 2.284.0 release (#1448) 2021-11-01 11:16:21 -04:00
Julio Barba
b6dbf42746 Improve retry handling based on feedback (#1447) 2021-10-29 16:04:34 -04:00
Ferenc Hammerl
67ba8a7d42 Support Conditional Steps in Composite Actions (#1438)
* conditional support for composite actions

* Fix Conditional function evaluation

* Push launch.json temporarily

* Revert "Push launch.json temporarily"

* rename context

* Cleanup comments

* fix success/failure functions to run based on pre/main steps

* idea of step_status

* change to use steps context, WIP

* add inputs to possible if condition expressions

* use action_status

* pr cleanup

* Added right stages

* Test on stage in conditional functions

* Fix naming and formatting

* Fix tests

* Add success and failure L0s

* Remove comment

* Remove whitespace

* Undo formatting

* Add L0 for step-if parsing

* Add ADR

Co-authored-by: Thomas Boop <thboop@github.com>
2021-10-29 15:45:42 +02:00
Ferenc Hammerl
e4f9e6ae26 Log current runner version in terminal (#1441) 2021-10-29 14:23:26 +02:00
Thomas Boop
854d5e3bf3 Fix an issue where nested local composite actions did not correctly register post steps (#1433)
* Always register post steps for local actions

* Register post steps along with their conditions

* remove debug code

Co-authored-by: Ferenc Hammerl <fhammerl@github.com>
2021-10-27 15:31:58 +02:00
Thomas Boop
57dec28f68 Cleanup Older versions on MacOS now that we recreate node versions as needed (#1410)
* Cleanup old version update code

* fix template

* fix indents
2021-10-19 10:15:37 -04:00
Tingluo Huang
55a861f089 Expose GITHUB_REF_* as environment variable (#1314)
* Keep env vars alphabetical

* ref_* context.

Co-authored-by: Ferenc Hammerl <31069338+fhammerl@users.noreply.github.com>
2021-10-18 22:22:34 -04:00
jeremyd2019
51b2031cbf Add arch to runner context (#1372)
Fixes #1185
2021-10-13 23:49:26 -04:00
Raphael Cruzeiro
400b2d879c Makes the user keychains available to the service (#847)
Without creating a session, the service is not able to access the keychains for the user specified under `UserName`. This causes any workflow that deals with code signing to fail as the only keychain loaded with be the system one. This should fix #350
2021-10-06 15:37:45 -04:00
Thomas Boop
c4b6d288d4 fix ephemeral runner upgrade on mac/linux (#1403) 2021-10-05 10:15:19 +02:00
Julio Barba
0699597876 Use Actions Service health and api.github.com endpoints after connection failure on Actions Server and Hosted (#1385) 2021-09-30 13:40:34 -04:00
Thomas Boop
a592b14ae3 Runner 2.283.2 Release (#1389) 2021-09-29 15:49:40 -04:00
Thomas Boop
04269f7b1b Handle keeping previous OSX versions more smoothly on Mac (#1381)
* Handle macOS upgrade smoothly

* cleanup

* misc cleanup

* final updates

* Update src/Misc/layoutbin/update.sh.template

Co-authored-by: Patrick Ellis <319655+pje@users.noreply.github.com>

* Update src/Misc/layoutbin/update.sh.template

Co-authored-by: Patrick Ellis <319655+pje@users.noreply.github.com>

* Upload telemetry and default to old method as needed

* minor fix

* add one more bit of logging

* some more telemetry

* quote variables to handle spaces

* tiny fix for ubuntu

* remove version and move telemetry to diag

* use full path

Co-authored-by: Patrick Ellis <319655+pje@users.noreply.github.com>
2021-09-29 15:49:31 -04:00
Ferenc Hammerl
e89d2e84bd Stop-Commands: stopToken restrictions (#1371)
* Prevent stopTokens that are workflow commands

Co-authored-by: Thomas Boop <52323235+thboop@users.noreply.github.com>

* Check context for env var too

* Accept true, 1 and $true instead of just "true"

* Setup ExpressionValues in tests

* Update src/Runner.Common/Constants.cs

Co-authored-by: Thomas Boop <52323235+thboop@users.noreply.github.com>

* Separate success and fail tests for invalid token

* Fix envcontext for tests

Co-authored-by: Thomas Boop <52323235+thboop@users.noreply.github.com>
2021-09-29 14:44:01 -04:00
Thomas Boop
afe7066e39 only cleanup runner local files on success (#1384) 2021-09-28 18:55:28 -04:00
Ferenc Hammerl
da79ef4acb Fix unconfiguring of runner after group changes (#1359)
* Ignore agentpool when unconfiguring the runner

Runner names and IDs are unique within a ServiceHost
They don't need to be included when unconfiguring the runner.

* Use -1 instead of 0 to highlight how it is ignored

* Use overloads and 0 instead of -1

Using 0 seems to be the convention

* Fix typo calling the wrong method
2021-09-22 15:04:43 +02:00
Tingluo Huang
5afb52b272 Update the comment about the --once in Constants.cs (#1360)
* Update Constants.cs

* feedback.

* Update src/Runner.Listener/Runner.cs

Co-authored-by: Thomas Boop <52323235+thboop@users.noreply.github.com>

Co-authored-by: Thomas Boop <52323235+thboop@users.noreply.github.com>
2021-09-21 21:31:48 +00:00
Thomas Boop
cf87c55557 Don't retry 422 (#1352) 2021-09-21 09:59:21 -04:00
Ferenc Hammerl
43fa351980 Update telemetry (#1355)
* Track "pause-logging"

* Bump release version
2021-09-20 15:54:20 +02:00
Ferenc Hammerl
ecfc2cc9e9 Prepare 2.283.0 release (#1351)
* Update releaseNote.md

* Update runnerversion

* Update releaseNote.md

Co-authored-by: Patrick Ellis <319655+pje@users.noreply.github.com>

Co-authored-by: Patrick Ellis <319655+pje@users.noreply.github.com>
2021-09-20 14:59:37 +02:00
Ferenc Hammerl
740fb43731 Generic telemetry (#1321)
* Add generateIdTokenUrl as an env var

* Add generateIdTokenUrl to env vars

* Add basic telemetry class and submit it on jobcompleted

* Use constructor overload

* Rename telemetry to jobTelemetry

* Rename telemetry file

* Make JobTelemetryType a string

* Collect telemetry

* Remove debugger

* Update src/Runner.Worker/ActionCommandManager.cs

Co-authored-by: Thomas Boop <52323235+thboop@users.noreply.github.com>

* Use same JobTelemetry for all contexts

* Mask telemetry data

* Mask in JobRunner instead

* Empty line

* Change method signature
Returning with a List suggests we clone it and that the
original doesn't change..

* Update launch.json

Co-authored-by: Thomas Boop <52323235+thboop@users.noreply.github.com>
2021-09-20 14:44:50 +02:00
Patrick Ellis
f259e5706f Ephemeral runner deletes local .runner,.credentials files after completion (#1344)
Closes #1337
2021-09-16 11:00:27 -04:00
Ferenc Hammerl
5d84918ed5 Add name to runner context (#1312)
* Add generateIdTokenUrl as an env var

* Add generateIdTokenUrl to env vars

* Add name and runner_group to context

* No longer add runner-group

* Update runner name if needed

* Get interface instead of concrete class

* Check for nulls on ReservedAgent

* Avoid loading setting file unnecesseraly

* Only check agentName once

* Use Trace.Error when can't update settings

* Better equals and exception handling

* Update JobDispatcher.cs

* Add tests and null check
2021-09-16 15:25:51 +02:00
Julio Barba
881c521005 Revert "Recreate VssConnection on retry (#1316)" (#1343)
This reverts commit 4359dd605b.
2021-09-15 13:21:50 -04:00
Patrick Ellis
176e7f5208 Trim trailing whitespace in all md and yml files (#1329)
* Trim non-significant trailing whitespace, add final newlines to md,yml files

* Add .editorconfig with basic whitespace conventions
2021-09-15 13:35:25 +02:00
Jacob Wallraff
b6d46c148a Add attempt number to GitHub context (#1302)
* Add attempt number to GitHub context

* Change context name

* Changing order
2021-09-15 11:00:53 +02:00
Thomas Boop
38e33bb8e3 Update network.md 2021-09-14 15:28:30 -04:00
Patrick Ellis
404b3418b7 Prepare 2.282.0 release (#1327) 2021-09-13 13:56:47 -04:00
Tingluo Huang
7ffd9af644 Support --ephemeral flag (#660)
This optional flag will configure the runner to only take one job, and let the service un-configure the runner after that job finishes.
2021-09-13 11:28:09 -04:00
Thomas Boop
1b69c279f5 Networking TSG (#1325)
* Update Network Troubleshooting doc

* fix list

* Update network.md
2021-09-13 09:53:20 +02:00
Liviu Ionescu
567870dbb8 Avoid ConsoleColor.White, it is unreadable on light themes (#1295) (#1319)
* Avoid white, it is unreadable on light themes (#1295)

* remove ', ConsoleColor.White' from banner

* remove ', ConsoleColor.White' from prompt

* cleanups
2021-09-13 07:50:52 +00:00
Tingluo Huang
72fa2a8a0d Wait for job record updated before running steps. (#1320)
* Wait for job record updated before running steps.

* only oidc
2021-09-09 21:55:15 -04:00
Julio Barba
4359dd605b Recreate VssConnection on retry (#1316) 2021-09-09 19:09:17 -04:00
dependabot[bot]
aab936d081 Bump path-parse in /src/Misc/expressionFunc/hashFiles (#1256)
Bumps [path-parse](https://github.com/jbgutierrez/path-parse) from 1.0.6 to 1.0.7.
- [Release notes](https://github.com/jbgutierrez/path-parse/releases)
- [Commits](https://github.com/jbgutierrez/path-parse/commits/v1.0.7)

---
updated-dependencies:
- dependency-name: path-parse
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-09-01 16:48:41 -04:00
Nancy Hsieh
777ce5a0dc ignore empty arrays in jq select (#1269)
* ignore empty arrays in jq select

* Update delete.sh

Co-authored-by: Ferenc Hammerl <31069338+fhammerl@users.noreply.github.com>
2021-09-01 20:39:45 +00:00
Tingluo Huang
1a62162708 Prepare 2.281.1 runner release. (#1305) 2021-09-01 16:15:08 -04:00
Thomas Boop
9a829995e0 Show More Step Information in composite Actions (#1279) 2021-09-01 16:04:27 -04:00
Vladimir Panteleev
c5ce52641c Allow setting default severity to "notice" (#1213) 2021-09-01 16:01:23 -04:00
Nick Fields
e82725b580 Update error to say 'uninstall' not 'unconfigure' (#1179)
* Update error to say 'uninstall' not 'unconfigure'

* Say uninstall service in *nix config error msgs

Co-authored-by: Ferenc Hammerl <31069338+fhammerl@users.noreply.github.com>
2021-09-01 16:00:19 -04:00
Daniel Asztalos
0464f77de3 Typo fixed (#1289) 2021-09-01 15:59:18 -04:00
Tingluo Huang
1fc159e0df Temporary fix for macOS runner upgrade crash loop. (#1304) 2021-09-01 15:39:17 -04:00
Ferenc Hammerl
3615fb6923 Runner 2.281.0 (#1298)
* Add generateIdTokenUrl as an env var

* Add generateIdTokenUrl to env vars

* Update runnerversion

* Remove old relese notes

* Update releaseNote.md
2021-08-30 18:57:24 +02:00
Ferenc Hammerl
f61dcad5bb Don't try to login to ghcr.io with GHES tokens (#1291)
* Don't try GHXX tokens for ghcr.io login

* Explain hosted / onpremise in comment

* Nitfix variable name
2021-08-30 11:52:12 +02:00
Tingluo Huang
62d568674c Add ACTIONS_ID_TOKEN_REQUEST_URL/Token to script as well. (#1287) 2021-08-26 13:29:02 -04:00
Ferenc Hammerl
07c00f6a8a PowerShell secret masking (#1258)
* Trim pwsh special chars when masking secrets

* Add pwsh valueEncoder

* Explain regex

* Update ValueEncoders.cs

* Add tests for pwsh color codes in secrets

* Formatting

* Group tests into theories

* Split secret on PS chars and mask for them

* Clean up comments

* Remove unused unittest

* Rename escape methods
2021-08-25 23:07:19 +02:00
Tingluo Huang
05b84297b7 Add extra env for the Token log-in action is going to use to request ID_TOKEN. (#1270) 2021-08-23 14:50:35 -04:00
Thomas Boop
04679b56a9 Runner 2.280.3 Release (#1276) 2021-08-19 08:40:11 -04:00
Thomas Boop
d2ca24fa43 For Main Steps, just run the step, don't check condition (#1273)
* For Main Steps, just run the step, don't check condition

* fix whitespace

* pr feedback
2021-08-18 16:40:25 -04:00
Thomas Boop
abdaacfa6e Runner release 2.280.2 (#1259)
* Runner release 2.280.2

* update

* update
2021-08-12 12:55:45 -04:00
Thomas Boop
53fd7161e2 send path when resolving actions (#1250) 2021-08-11 09:48:32 -04:00
Ferenc Hammerl
ce68f3b167 Allow the use of flags in scripts/create-latest-svc.sh in a backwards compatible way (#1220)
* Use flags in svc creation script

* Refactor regex and add comments

* Fix indentation and typo in user matching

* Consistency use flags in automation scripts

* Update documentation to reflect new usage

* Make example more readable

* Remove test echos from script

* Remove test echo

* Format scripts and remove test script

* Remove tar

* Use getopts and single letter flags

* Update docs to show flag usage

* Update usage of create svc

* Revert svc to not use flags

* Revert delete script

* Update docs

* Readd deleted comments
2021-08-09 10:22:19 +02:00
Thomas Boop
e2c7329292 Release notes for 2.280.1 runner (#1244) 2021-08-04 13:28:32 -04:00
Thomas Boop
22a9d89772 Correctly set post step step context (#1243) 2021-08-04 11:39:22 -04:00
Thomas Boop
3851acd0cf fix continue on error (#1238) 2021-08-03 17:44:58 -04:00
Tingluo Huang
aab4aca8f7 Finish job when worker crashed with IOException. (#1239) 2021-08-03 16:21:39 -04:00
93 changed files with 1782 additions and 376 deletions

8
.editorconfig Normal file
View File

@@ -0,0 +1,8 @@
# https://editorconfig.org/
[*]
insert_final_newline = true # ensure all files end with a single newline
trim_trailing_whitespace = true # attempt to remove trailing whitespace on save
[*.md]
trim_trailing_whitespace = false # in markdown, "two trailing spaces" is unfortunately meaningful; it means `<br>`

6
.gitattributes vendored
View File

@@ -20,7 +20,7 @@
#
# Merging from the command prompt will add diff markers to the files if there
# are conflicts (Merging from VS is not affected by the settings below, in VS
# the diff markers are never inserted). Diff markers may cause the following
# the diff markers are never inserted). Diff markers may cause the following
# file extensions to fail to load in VS. An alternative would be to treat
# these files as binary and thus will always conflict and require user
# intervention with every merge. To do so, just uncomment the entries below
@@ -70,9 +70,9 @@
###############################################################################
# diff behavior for common document formats
#
#
# Convert binary document formats to text before diffing them. This feature
# is only available from the command line. Turn it on by uncommenting the
# is only available from the command line. Turn it on by uncommenting the
# entries below.
###############################################################################
*.doc diff=astextplain

View File

@@ -24,4 +24,4 @@ If applicable, add a code snippet.
**Additional information**
Add any other context about the feature here.
NOTE: if the feature request has been agreed upon then the assignee will create an ADR. See docs/adrs/README.md
NOTE: if the feature request has been agreed upon then the assignee will create an ADR. See docs/adrs/README.md

View File

@@ -7,12 +7,12 @@ on:
- main
- releases/*
paths-ignore:
- '**.md'
- '**.md'
pull_request:
branches:
- '*'
paths-ignore:
- '**.md'
- '**.md'
jobs:
build:

View File

@@ -28,7 +28,7 @@ jobs:
# languages: go, javascript, csharp, python, cpp, java
- name: Manual build
run : |
run : |
./dev.sh layout Release linux-x64
working-directory: src

View File

@@ -5,7 +5,7 @@ on:
push:
paths:
- releaseVersion
jobs:
check:
if: startsWith(github.ref, 'refs/heads/releases/') || github.ref == 'refs/heads/main'
@@ -13,8 +13,8 @@ jobs:
steps:
- uses: actions/checkout@v2
# Make sure ./releaseVersion match ./src/runnerversion
# Query GitHub release ensure version is not used
# Make sure ./releaseVersion match ./src/runnerversion
# Query GitHub release ensure version is not used
- name: Check version
uses: actions/github-script@0.3.0
with:
@@ -42,7 +42,7 @@ jobs:
throw e
}
}
build:
needs: check
outputs:
@@ -152,7 +152,7 @@ jobs:
releaseNote = releaseNote.replace(/<LINUX_ARM64_SHA>/g, '${{needs.build.outputs.linux-arm64-sha}}')
console.log(releaseNote)
core.setOutput('version', runnerVersion);
core.setOutput('note', releaseNote);
core.setOutput('note', releaseNote);
# Create GitHub release
- uses: actions/create-release@master
id: createRelease

2
.vscode/launch.json vendored
View File

@@ -54,4 +54,4 @@
"requireExactSource": false,
},
],
}
}

View File

@@ -5,7 +5,7 @@
# GitHub Actions Runner
[![Actions Status](https://github.com/actions/runner/workflows/Runner%20CI/badge.svg)](https://github.com/actions/runner/actions)
[![Runner E2E Test](https://github.com/actions/runner/workflows/Runner%20E2E%20Test/badge.svg)](https://github.com/actions/runner/actions)
[![Runner E2E Test](https://github.com/actions-canary/actions-runner-e2e/actions/workflows/runner_e2etest.yml/badge.svg)](https://github.com/actions-canary/actions-runner-e2e/actions/workflows/runner_e2etest.yml)
The runner is the application that runs a job from a GitHub Actions workflow. It is used by GitHub Actions in the [hosted virtual environments](https://github.com/actions/virtual-environments), or you can [self-host the runner](https://help.github.com/en/actions/automating-your-workflow-with-github-actions/about-self-hosted-runners) in your own environment.

View File

@@ -0,0 +1,71 @@
# ADR 1438: Support Conditionals In Composite Actions
**Date**: 2021-10-13
**Status**: Accepted
## Context
We recently shipped composite actions, which allows you to reuse individual steps inside an action.
However, one of the [most requested features](https://github.com/actions/runner/issues/834) has been a way to support the `if` keyword.
### Goals
- We want to keep consistent with current behavior
- We want to support conditionals via the `if` keyword
- Our built in functions like `success` should be implementable without calling them, for example you can do `job.status == success` rather then `success()` currently.
### How does composite currently work?
Currently, we have limited conditional support in composite actions for `pre` and `post` steps.
These are based on the `job status`, and support keywords like `always()`, `failed()`, `success()` and `cancelled()`.
However, generic or main steps do **not** support conditionals.
By default, in a regular workflow, a step runs on the `success()` condition. Which looks at the **job** **status**, sees if it is successful and runs.
By default, in a composite action, main steps run until a single step fails in that composite action, then the composite action is halted early. It does **not** care about the job status.
Pre, and post steps in composite actions use the job status to determine if they should run.
### How do we go forward?
Well, if we think about what composite actions are currently doing when invoking main steps, they are checking if the current composite action is successful.
Lets formalize that concept into a "real" idea.
- We will add an `action_status` field to the github context to mimic the [job's context status](https://docs.github.com/en/actions/learn-github-actions/contexts#job-context).
- We have an existing concept that does this `action_path` which is only set for composite actions on the github context.
- In a composite action during a main step, the `success()` function will check if `action_status == success`, rather then `job_status == success`. Failure will work the same way.
- Pre and post steps in composite actions will not change, they will continue to check the job status.
### Nested Scenario
For nested composite actions, we will follow the existing behavior, you only care about your current composite action, not any parents.
For example, lets imagine a scenario with a simple nested composite action
```
- Job
- Regular Step
- Composite Action
- runs: exit 1
- if: always()
uses: A child composite action
- if: success()
runs: echo "this should print"
- runs: echo "this should also print"
- if: success()
runs: echo "this will not print as the current composite action has failed already"
```
The child composite actions steps should run in this example, the child composite action has not yet failed, so it should run all steps until a step fails. This is consistent with how a composite action currently works in production if the main job fails but a composite action is invoked with `if:always()` or `if: failure()`
### Other options explored
We could add the `current_step_status` to the job context rather then `__status` to the steps context, however this comes with two major downsides:
- We need to support the field for every type of step, because its non trivial to remove a field from the job context once it has been added (its readonly)
- For all actions besides composite it would only every be `success`
- Its weird to have a `current_step` value on the job context
- We also explored a `__status` on the steps context.
- The `__` is required to prevent us from colliding with a step with id: status
- This felt wrong because the naming was not smooth, and did not fit into current conventions.
### Consequences
- github context has a new field for the status of the current composite action.
- We support conditional's in composite actions
- We keep the existing behavior for all users, but allow them to expand that functionality.

View File

@@ -21,16 +21,33 @@ export RUNNER_CFG_PAT=yourPAT
:point_right: [Sample script here](../scripts/create-latest-svc.sh) :point_left:
Run as a one-liner. NOTE: replace with yourorg/yourrepo (repo level) or just yourorg (org level)
Run as a one-liner. NOTE: replace with yourorg/yourrepo (repo level) or just yourorg (org level)
```bash
curl -s https://raw.githubusercontent.com/actions/runner/main/scripts/create-latest-svc.sh | bash -s yourorg/yourrepo
```
You can call the script with additional arguments:
```bash
# Usage:
# export RUNNER_CFG_PAT=<yourPAT>
# ./create-latest-svc -s scope -g [ghe_domain] -n [name] -u [user] -l [labels]
# -s required scope: repo (:owner/:repo) or org (:organization)
# -g optional ghe_hostname: the fully qualified domain name of your GitHub Enterprise Server deployment
# -n optional name of the runner, defaults to hostname
# -u optional user svc will run as, defaults to current
# -l optional list of labels (split by comma) applied on the runner"
```
Use `--` to pass any number of optional named parameters:
```
curl -s https://raw.githubusercontent.com/actions/runner/main/scripts/create-latest-svc.sh | bash -s -- -s myorg/myrepo -n myname -l label1,label2
```
### Why can't I use a container?
The runner is installed as a service using `systemd` and `systemctl`. Docker does not support `systemd` for service configuration on a container.
## Uninstall running as service
## Uninstall running as service
**Scenario**: Run on a machine or VM ([not container](#why-cant-i-use-a-container)) which automates:
@@ -40,7 +57,7 @@ The runner is installed as a service using `systemd` and `systemctl`. Docker doe
:point_right: [Sample script here](../scripts/remove-svc.sh) :point_left:
Repo level one liner. NOTE: replace with yourorg/yourrepo (repo level) or just yourorg (org level)
Repo level one liner. NOTE: replace with yourorg/yourrepo (repo level) or just yourorg (org level)
```bash
curl -s https://raw.githubusercontent.com/actions/runner/main/scripts/remove-svc.sh | bash -s yourorg/yourrepo
```

View File

@@ -18,16 +18,16 @@ Make sure the runner has access to actions service for GitHub.com or GitHub Ente
- DNS lookup for api.github.com or myGHES.com using dotnet
- Ping api.github.com or myGHES.com using dotnet
- Make HTTP GET to https://api.github.com or https://myGHES.com/api/v3 using dotnet, check response headers contains `X-GitHub-Request-Id`
- Make HTTP GET to https://api.github.com or https://myGHES.com/api/v3 using dotnet, check response headers contains `X-GitHub-Request-Id`
---
- DNS lookup for vstoken.actions.githubusercontent.com using dotnet
- Ping vstoken.actions.githubusercontent.com using dotnet
- Make HTTP GET to https://vstoken.actions.githubusercontent.com/_apis/health or https://myGHES.com/_services/vstoken/_apis/health using dotnet, check response headers contains `x-vss-e2eid`
- Make HTTP GET to https://vstoken.actions.githubusercontent.com/_apis/health or https://myGHES.com/_services/vstoken/_apis/health using dotnet, check response headers contains `x-vss-e2eid`
---
- DNS lookup for pipelines.actions.githubusercontent.com using dotnet
- Ping pipelines.actions.githubusercontent.com using dotnet
- Make HTTP GET to https://pipelines.actions.githubusercontent.com/_apis/health or https://myGHES.com/_services/pipelines/_apis/health using dotnet, check response headers contains `x-vss-e2eid`
- Make HTTP POST to https://pipelines.actions.githubusercontent.com/_apis/health or https://myGHES.com/_services/pipelines/_apis/health using dotnet, check response headers contains `x-vss-e2eid`
- Make HTTP GET to https://pipelines.actions.githubusercontent.com/_apis/health or https://myGHES.com/_services/pipelines/_apis/health using dotnet, check response headers contains `x-vss-e2eid`
- Make HTTP POST to https://pipelines.actions.githubusercontent.com/_apis/health or https://myGHES.com/_services/pipelines/_apis/health using dotnet, check response headers contains `x-vss-e2eid`
## How to fix the issue?
@@ -42,4 +42,4 @@ Make sure the runner has access to actions service for GitHub.com or GitHub Ente
## Still not working?
Contact GitHub customer service or log an issue at https://github.com/actions/runner if you think it's a runner issue.
Contact GitHub customer service or log an issue at https://github.com/actions/runner if you think it's a runner issue.

View File

@@ -31,4 +31,4 @@ The test also set environment variable `GIT_TRACE=1` and `GIT_CURL_VERBOSE=1` be
## Still not working?
Contact GitHub customer service or log an issue at https://github.com/actions/runner if you think it's a runner issue.
Contact GitHub customer service or log an issue at https://github.com/actions/runner if you think it's a runner issue.

View File

@@ -13,7 +13,7 @@ Even the runner is configured to GitHub Enterprise Server, the runner can still
- DNS lookup for api.github.com using dotnet
- Ping api.github.com using dotnet
- Make HTTP GET to https://api.github.com using dotnet, check response headers contains `X-GitHub-Request-Id`
- Make HTTP GET to https://api.github.com using dotnet, check response headers contains `X-GitHub-Request-Id`
## How to fix the issue?
@@ -23,4 +23,4 @@ Even the runner is configured to GitHub Enterprise Server, the runner can still
## Still not working?
Contact GitHub customer service or log an issue at https://github.com/actions/runner if you think it's a runner issue.
Contact GitHub customer service or log an issue at https://github.com/actions/runner if you think it's a runner issue.

View File

@@ -2,17 +2,19 @@
### Common things that can cause the runner to not working properly
- Bug in the runner or the dotnet framework that causes actions runner can't make Http request in a certain network environment.
- A bug in the runner or the dotnet framework that causes the actions runner to be unable to make Http requests in a certain network environment.
- Proxy/Firewall block certain HTTP method, like it block all POST and PUT calls which the runner will use to upload logs.
- A Proxy or Firewall may block certain HTTP method, such as blocking all POST and PUT calls which the runner will use to upload logs.
- Proxy/Firewall only allows requests with certain user-agent to pass through and the actions runner user-agent is not in the allow list.
- A Proxy or Firewall may only allows requests with certain user-agent to pass through and the actions runner user-agent is not in the allow list.
- Proxy try to decrypt and exam HTTPS traffic for security purpose but cause the actions-runner to fail to finish SSL handshake due to the lack of trusting proxy's CA.
- A Proxy try to decrypt and exam HTTPS traffic for security purpose but cause the actions-runner to fail to finish SSL handshake due to the lack of trusting proxy's CA.
- Proxy try to modify the HTTPS request (like add or change some http headers) and causes the request become incompatible with the Actions Service (ASP.NetCore), Ex: [Nginx](https://github.com/dotnet/aspnetcore/issues/17081)
- The SSL handshake may fail if the client and server do not support the same TLS version, or the same cipher suites.
- Firewall rules that block action runner from accessing certain hosts, ex: `*.github.com`, `*.actions.githubusercontent.com`, etc.
- A Proxy may try to modify the HTTPS request (like add or change some http headers) and causes the request become incompatible with the Actions Service (ASP.NetCore), Ex: [Nginx](https://github.com/dotnet/aspnetcore/issues/17081)
- Firewall rules that block action runner from accessing certain hosts, ex: `*.github.com`, `*.actions.githubusercontent.com`, etc
### Identify and solve these problems
@@ -23,10 +25,38 @@ Use a 3rd party tool to make the same requests as the runner did would be a good
- Use `nslookup` to check DNS
- Use `ping` to check Ping
- Use `traceroute`, `tracepath`, or `tracert` to check the network route between the runner and the Actions service
- Use `traceroute`, `tracepath`, or `tracert` to check the network route between the runner and the Actions service
- Use `curl -v` to check the network stack, good for verifying default certificate/proxy settings.
- Use `Invoke-WebRequest` from `pwsh` (`PowerShell Core`) to check the dotnet network stack, good for verifying bugs in the dotnet framework.
If the 3rd party tool is also experiencing the same error as the runner does, then you might want to contact your network administrator for help.
Otherwise, contact GitHub customer support or log an issue at https://github.com/actions/runner
Otherwise, contact GitHub customer support or log an issue at https://github.com/actions/runner
### Troubleshooting: Why can't I configure a runner?
If you are having trouble connecting, try these steps:
1. Validate you can reach our endpoints from your web browser. If not, double check your local network connection
- For hosted Github:
- https://api.github.com/
- https://vstoken.actions.githubusercontent.com/_apis/health
- https://pipelines.actions.githubusercontent.com/_apis/health
- For GHES/GHAE
- https://myGHES.com/_services/vstoken/_apis/health
- https://myGHES.com/_services/pipelines/_apis/health
- https://myGHES.com/api/v3
2. Validate you can reach those endpoints in powershell core
- The runner runs on .net core, lets validate the local settings for that stack
- Open up `pwsh`
- Run the command using the urls above `Invoke-WebRequest {url}`
3. If not, get a packet trace using a tool like wireshark and start looking at the TLS handshake.
- If you see a Client Hello followed by a Server RST:
- You may need to configure your TLS settings to use the correct version
- You should support TLS version 1.2 or later
- You may need to configure your TLS settings to have up to date cipher suites, this may be solved by system updates and patches.
- Most notably, on windows server 2012 make sure [the tls cipher suite update](https://support.microsoft.com/en-us/topic/update-adds-new-tls-cipher-suites-and-changes-cipher-suite-priorities-in-windows-8-1-and-windows-server-2012-r2-8e395e43-c8ef-27d8-b60c-0fc57d526d94) is installed
- Your firewall, proxy or network configuration may be blocking the connection
- You will want to reach out to whoever is in charge of your network with these pcap files to further troubleshoot
- If you see a failure later in the handshake:
- Try the fix in the [SSLCert Fix](./sslcert.md)

View File

@@ -27,4 +27,4 @@ All javascript base Actions will get executed by the built-in `node` at `<runner
## Still not working?
Contact GitHub customer service or log an issue at https://github.com/actions/runner if you think it's a runner issue.
Contact GitHub customer service or log an issue at https://github.com/actions/runner if you think it's a runner issue.

View File

@@ -12,7 +12,7 @@ As long as your certificate is generated properly, most of the issues should be
> !!! DO NOT SKIP SSL CERT VALIDATION !!!
> !!! IT IS A BAD SECURITY PRACTICE !!!
### Download SSL certificate chain
### Download SSL certificate chain
Depends on how your SSL server certificate gets configured, you might need to download the whole certificate chain from a machine that has trusted the SSL certificate's CA.
@@ -28,7 +28,7 @@ The actions runner is a dotnet core application which will follow how dotnet loa
You can get full details documentation at [here](https://docs.microsoft.com/en-us/dotnet/standard/security/cross-platform-cryptography#x509store)
In short:
In short:
- Windows: Load from Windows certificate store.
- Linux: Load from OpenSSL CA cert bundle.
- macOS: Load from macOS KeyChain.
@@ -43,13 +43,13 @@ To let the runner trusts your CA certificate, you will need to:
1. RedHat: https://www.redhat.com/sysadmin/ca-certificates-cli
2. Ubuntu: http://manpages.ubuntu.com/manpages/focal/man8/update-ca-certificates.8.html
3. Google search: "trust ca certificate on [linux distribution]"
4. If all approaches failed, set environment variable `SSL_CERT_FILE` to the CA bundle `.pem` file we get.
4. If all approaches failed, set environment variable `SSL_CERT_FILE` to the CA bundle `.pem` file we get.
> To verify cert gets installed properly on Linux, you can try use `curl -v https://sitewithsslissue.com` and `pwsh -Command \"Invoke-WebRequest -Uri https://sitewithsslissue.com\"`
### Trust CA certificate for Git CLI
Git uses various CA bundle file depends on your operation system.
- Git packaged the CA bundle file within the Git installation on Windows
- Git packaged the CA bundle file within the Git installation on Windows
- Git use OpenSSL certificate CA bundle file on Linux and macOS
You can check where Git check CA file by running:

View File

@@ -12,7 +12,7 @@ Issues in this repository should be for the runner application. Note that the V
## Enhancements and Feature Requests
We ask that before significant effort is put into code changes, that we have agreement on taking the change before time is invested in code changes.
We ask that before significant effort is put into code changes, that we have agreement on taking the change before time is invested in code changes.
1. Create a feature request. Once agreed we will take the enhancement
2. Create an ADR to agree on the details of the change.
@@ -46,9 +46,9 @@ Tip: Make sure your job can run on this runner. The easiest way is to set `runs-
## Development Life Cycle
If you're using VS Code, you can follow [these](contribute/vscode.md) steps instead.
If you're using VS Code, you can follow [these](contribute/vscode.md) steps instead.
### To Build, Test, Layout
### To Build, Test, Layout
Navigate to the `src` directory and run the following command:

View File

@@ -4,7 +4,7 @@ These examples use VS Code, but the idea should be similar across all IDEs as lo
## Configure
To successfully start the runner, you need to register it using a repository and a runner registration token.
Run `Configure` first to build the source code and set up the runner in `_layout`.
Run `Configure` first to build the source code and set up the runner in `_layout`.
Once it's done creating `_layout`, it asks for the url of your repository and your token in the terminal.
Check [Quickstart](../contribute.md#quickstart-run-a-job-from-a-real-repository) if you don't know how to get this token.
@@ -34,7 +34,7 @@ All the configs below can be found in `.vscode/launch.json`.
If you launch `Run` or `Run [build]`, it starts a process called `Runner.Listener`.
This process will receive any job queued on this repository if the job runs on matching labels (e.g `runs-on: self-hosted`).
Once a job is received, a `Runner.Listener` starts a new process of `Runner.Worker`.
Once a job is received, a `Runner.Listener` starts a new process of `Runner.Worker`.
Since this is a diferent process, you can't use the same debugger session debug it.
Instead, a parallel debugging session has to be started, using a different launch config.
Luckily, VS Code supports multiple parallel debugging sessions.
@@ -45,7 +45,7 @@ Because the worker process is usually started by the listener instead of an IDE,
For this reason, `Runner.Worker` can be configured to wait for a debugger to be attached before it begins any actual work.
Set the environment variable `GITHUB_ACTIONS_RUNNER_ATTACH_DEBUGGER` to `true` or `1` to enable this wait.
All worker processes now will wait 20 seconds before they start working on their task.
All worker processes now will wait 20 seconds before they start working on their task.
This gives enough time to attach a debugger by running `Debug Worker`.
If for some reason you have multiple workers running, run the launch config `Attach` instead.

View File

@@ -58,4 +58,4 @@ Authentication in a workflow run to github.com can be accomplished by using the
Hosted runner authentication differs from self-hosted authentication in that runners do not undergo a registration process, but instead, the hosted runners get the OAuth token directly by reading the `.credentials` file. The scope of this particular token is limited for a given workflow job execution, and the token is revoked as soon as the job is finished.
![Hosted runner config and start](../res/hosted-config-start.png)
![Hosted runner config and start](../res/hosted-config-start.png)

View File

@@ -27,7 +27,7 @@ Dependencies is missing for Dotnet Core 3.0
Execute ./bin/installdependencies.sh to install any missing Dotnet Core 3.0 dependencies.
```
You can easily correct the problem by executing `./bin/installdependencies.sh`.
The `installdependencies.sh` script should install all required dependencies on all supported Linux versions
The `installdependencies.sh` script should install all required dependencies on all supported Linux versions
> Note: The `installdependencies.sh` script will try to use the default package management mechanism on your Linux flavor (ex. `yum`/`apt-get`/`apt`).
### Full dependencies list
@@ -35,15 +35,15 @@ The `installdependencies.sh` script should install all required dependencies on
Debian based OS (Debian, Ubuntu, Linux Mint)
- liblttng-ust0
- libkrb5-3
- libkrb5-3
- zlib1g
- libssl1.1, libssl1.0.2 or libssl1.0.0
- libicu63, libicu60, libicu57 or libicu55
Fedora based OS (Fedora, Red Hat Enterprise Linux, CentOS, Oracle Linux 7)
- lttng-ust
- openssl-libs
- lttng-ust
- openssl-libs
- krb5-libs
- zlib
- libicu

View File

@@ -5,7 +5,7 @@
## Supported Versions
- macOS High Sierra (10.13) and later versions
## Apple Silicon M1
The runner is currently not supported on devices with an Apple M1 chip.

View File

@@ -1,14 +1,19 @@
## Features
- Print source of secret in runs (Actions/Dependabot/None) #1411
- Support node.js 16 and bump node.js 12 version #1439
## Bugs
- Fixed a bug where composite actions did not respect `continue-on-error` (#1238)
- Fixed a bug where composite actions post steps did not have the correct step context (#1243)
- Fix a bug where local node action would crash in post-steps #1481
## Misc
- Correctly finish Job when worker crashes with IO Exceptions (#1239)
- Add telemetry around runner update process. #1497
- Improve telemetry to better diagnose runner configuration issues #1487
- Clean up dependencies #1470
## Windows x64
We recommend configuring the runner in a root folder of the Windows drive (e.g. "C:\actions-runner"). This will help avoid issues related to service identity folder permissions and long file path restrictions on Windows.
@@ -20,7 +25,7 @@ mkdir \actions-runner ; cd \actions-runner
# Download the latest runner package
Invoke-WebRequest -Uri https://github.com/actions/runner/releases/download/v<RUNNER_VERSION>/actions-runner-win-x64-<RUNNER_VERSION>.zip -OutFile actions-runner-win-x64-<RUNNER_VERSION>.zip
# Extract the installer
Add-Type -AssemblyName System.IO.Compression.FileSystem ;
Add-Type -AssemblyName System.IO.Compression.FileSystem ;
[System.IO.Compression.ZipFile]::ExtractToDirectory("$PWD\actions-runner-win-x64-<RUNNER_VERSION>.zip", "$PWD")
```

View File

@@ -1 +1 @@
2.280.1
2.285.0

View File

@@ -1,4 +1,4 @@
# Sample scripts for self-hosted runners
Here are some examples to work from if you'd like to automate your use of self-hosted runners.
See the docs [here](../docs/automate.md).
See the docs [here](../docs/automate.md).

View File

@@ -2,36 +2,68 @@
set -e
#
# Downloads latest releases (not pre-release) runner
# Configures as a service
#
# Examples:
# RUNNER_CFG_PAT=<yourPAT> ./create-latest-svc.sh myuser/myrepo my.ghe.deployment.net
# RUNNER_CFG_PAT=<yourPAT> ./create-latest-svc.sh myorg my.ghe.deployment.net
#
# Usage:
# export RUNNER_CFG_PAT=<yourPAT>
# ./create-latest-svc scope [ghe_domain] [name] [user] [labels]
#
# scope required repo (:owner/:repo) or org (:organization)
# ghe_domain optional the fully qualified domain name of your GitHub Enterprise Server deployment
# name optional defaults to hostname
# user optional user svc will run as. defaults to current
# labels optional list of labels (split by comma) applied on the runner
#
# Notes:
# PATS over envvars are more secure
# Downloads latest runner release (not pre-release)
# Configures it as a service more secure
# Should be used on VMs and not containers
# Works on OSX and Linux
# Assumes x64 arch
#
# See EXAMPLES below
runner_scope=${1}
ghe_hostname=${2}
runner_name=${3:-$(hostname)}
svc_user=${4:-$USER}
labels=${5}
flags_found=false
while getopts 's:g:n:u:l:' opt; do
flags_found=true
case $opt in
s)
runner_scope=$OPTARG
;;
g)
ghe_hostname=$OPTARG
;;
n)
runner_name=$OPTARG
;;
u)
svc_user=$OPTARG
;;
l)
labels=$OPTARG
;;
*)
echo "
Runner Service Installer
Examples:
RUNNER_CFG_PAT=<yourPAT> ./create-latest-svc.sh myuser/myrepo my.ghe.deployment.net
RUNNER_CFG_PAT=<yourPAT> ./create-latest-svc.sh -s myorg -u user_name -l label1,label2
Usage:
export RUNNER_CFG_PAT=<yourPAT>
./create-latest-svc scope [ghe_domain] [name] [user] [labels]
-s required scope: repo (:owner/:repo) or org (:organization)
-g optional ghe_hostname: the fully qualified domain name of your GitHub Enterprise Server deployment
-n optional name of the runner, defaults to hostname
-u optional user svc will run as, defaults to current
-l optional list of labels (split by comma) applied on the runner"
exit 0
;;
esac
done
shift "$((OPTIND - 1))"
if ! "$flags_found"; then
runner_scope=${1}
ghe_hostname=${2}
runner_name=${3:-$(hostname)}
svc_user=${4:-$USER}
labels=${5}
fi
# apply defaults
runner_name=${runner_name:-$(hostname)}
svc_user=${svc_user:-$USER}
echo "Configuring runner @ ${runner_scope}"
sudo echo
@@ -142,7 +174,7 @@ echo
echo "Configuring as a service ..."
prefix=""
if [ "${runner_plat}" == "linux" ]; then
prefix="sudo "
prefix="sudo "
fi
${prefix}./svc.sh install ${svc_user}

View File

@@ -51,7 +51,7 @@ fi
# Ensure offline
#--------------------------------------
runner_status=$(curl -s -X GET ${base_api_url}/${runner_scope}/actions/runners?per_page=100 -H "accept: application/vnd.github.everest-preview+json" -H "authorization: token ${RUNNER_CFG_PAT}" \
| jq -M -j ".runners | .[] | [select(.name == \"${runner_name}\")] | .[0].status")
| jq -M -j ".runners | .[] | select(.name == \"${runner_name}\") | .status")
if [ -z "${runner_status}" ]; then
fatal "Could not find runner with name ${runner_name}"
@@ -67,7 +67,7 @@ fi
# Get id of runner to remove
#--------------------------------------
runner_id=$(curl -s -X GET ${base_api_url}/${runner_scope}/actions/runners?per_page=100 -H "accept: application/vnd.github.everest-preview+json" -H "authorization: token ${RUNNER_CFG_PAT}" \
| jq -M -j ".runners | .[] | [select(.name == \"${runner_name}\")] | .[0].id")
| jq -M -j ".runners | .[] | select(.name == \"${runner_name}\") | .id")
if [ -z "${runner_id}" ]; then
fatal "Could not find runner with name ${runner_name}"

View File

@@ -1947,9 +1947,9 @@
"dev": true
},
"path-parse": {
"version": "1.0.6",
"resolved": "https://registry.npmjs.org/path-parse/-/path-parse-1.0.6.tgz",
"integrity": "sha512-GSmOT2EbHrINBf9SR7CDELwlJ8AENk3Qn7OikK4nFYAu3Ote2+JYNVvkpAEQm3/TLNEJFD/xZJjzyxg3KBWOzw==",
"version": "1.0.7",
"resolved": "https://registry.npmjs.org/path-parse/-/path-parse-1.0.7.tgz",
"integrity": "sha512-LDJzPVEEEPR+y48z93A0Ed0yXb8pAByGWo/k5YYdYgpY2/2EsOsksJrq7lOHxryrVOn1ejG6oAp8ahvOIQD8sw==",
"dev": true
},
"path-type": {

View File

@@ -3,7 +3,8 @@ PACKAGERUNTIME=$1
PRECACHE=$2
NODE_URL=https://nodejs.org/dist
NODE12_VERSION="12.13.1"
NODE12_VERSION="12.22.7"
NODE16_VERSION="16.13.0"
get_abs_path() {
# exploits the fact that pwd will print abs path when no args
@@ -126,6 +127,8 @@ function acquireExternalTool() {
if [[ "$PACKAGERUNTIME" == "win-x64" || "$PACKAGERUNTIME" == "win-x86" ]]; then
acquireExternalTool "$NODE_URL/v${NODE12_VERSION}/$PACKAGERUNTIME/node.exe" node12/bin
acquireExternalTool "$NODE_URL/v${NODE12_VERSION}/$PACKAGERUNTIME/node.lib" node12/bin
acquireExternalTool "$NODE_URL/v${NODE16_VERSION}/$PACKAGERUNTIME/node.exe" node16/bin
acquireExternalTool "$NODE_URL/v${NODE16_VERSION}/$PACKAGERUNTIME/node.lib" node16/bin
if [[ "$PRECACHE" != "" ]]; then
acquireExternalTool "https://github.com/microsoft/vswhere/releases/download/2.6.7/vswhere.exe" vswhere
fi
@@ -134,18 +137,23 @@ fi
# Download the external tools only for OSX.
if [[ "$PACKAGERUNTIME" == "osx-x64" ]]; then
acquireExternalTool "$NODE_URL/v${NODE12_VERSION}/node-v${NODE12_VERSION}-darwin-x64.tar.gz" node12 fix_nested_dir
acquireExternalTool "$NODE_URL/v${NODE16_VERSION}/node-v${NODE16_VERSION}-darwin-x64.tar.gz" node16 fix_nested_dir
fi
# Download the external tools for Linux PACKAGERUNTIMEs.
if [[ "$PACKAGERUNTIME" == "linux-x64" ]]; then
acquireExternalTool "$NODE_URL/v${NODE12_VERSION}/node-v${NODE12_VERSION}-linux-x64.tar.gz" node12 fix_nested_dir
acquireExternalTool "https://vstsagenttools.blob.core.windows.net/tools/nodejs/${NODE12_VERSION}/alpine/x64/node-${NODE12_VERSION}-alpine-x64.tar.gz" node12_alpine
acquireExternalTool "https://vstsagenttools.blob.core.windows.net/tools/nodejs/${NODE12_VERSION}/alpine/x64/node-v${NODE12_VERSION}-alpine-x64.tar.gz" node12_alpine
acquireExternalTool "$NODE_URL/v${NODE16_VERSION}/node-v${NODE16_VERSION}-linux-x64.tar.gz" node16 fix_nested_dir
acquireExternalTool "https://vstsagenttools.blob.core.windows.net/tools/nodejs/${NODE16_VERSION}/alpine/x64/node-v${NODE16_VERSION}-alpine-x64.tar.gz" node16_alpine
fi
if [[ "$PACKAGERUNTIME" == "linux-arm64" ]]; then
acquireExternalTool "$NODE_URL/v${NODE12_VERSION}/node-v${NODE12_VERSION}-linux-arm64.tar.gz" node12 fix_nested_dir
acquireExternalTool "$NODE_URL/v${NODE16_VERSION}/node-v${NODE16_VERSION}-linux-arm64.tar.gz" node16 fix_nested_dir
fi
if [[ "$PACKAGERUNTIME" == "linux-arm" ]]; then
acquireExternalTool "$NODE_URL/v${NODE12_VERSION}/node-v${NODE12_VERSION}-linux-armv7l.tar.gz" node12 fix_nested_dir
acquireExternalTool "$NODE_URL/v${NODE16_VERSION}/node-v${NODE16_VERSION}-linux-armv7l.tar.gz" node16 fix_nested_dir
fi

View File

@@ -25,5 +25,7 @@
</dict>
<key>ProcessType</key>
<string>Interactive</string>
<key>SessionCreate</key>
<true/>
</dict>
</plist>

View File

@@ -18,6 +18,8 @@ downloadrunnerversion=_DOWNLOAD_RUNNER_VERSION_
logfile="_UPDATE_LOG_"
restartinteractiverunner=_RESTART_INTERACTIVE_RUNNER_
telemetryfile="$rootfolder/_diag/.telemetry"
# log user who run the script
date "+[%F %T-%4N] --------whoami--------" >> "$logfile" 2>&1
whoami >> "$logfile" 2>&1
@@ -118,6 +120,58 @@ then
exit 1
fi
# fix upgrade issue with macOS when running as a service
attemptedtargetedfix=0
currentplatform=$(uname | awk '{print tolower($0)}')
if [[ "$currentplatform" == 'darwin' && restartinteractiverunner -eq 0 ]]; then
# We needed a fix for https://github.com/actions/runner/issues/743
# We will recreate the ./externals/node12/bin/node of the past runner version that launched the runnerlistener service
# Otherwise mac gatekeeper kills the processes we spawn on creation as we are running a process with no backing file
# We need the pid for the nodejs loop, get that here, its the parent of the runner C# pid
# assumption here is only one process is invoking rootfolder/runsvc.sh
procgroup=$(ps x -o pgid,command | grep "$rootfolder/runsvc.sh" | grep -v grep | awk '{print $1}')
if [[ $? -eq 0 && -n "$procgroup" ]]
then
# inspect the open file handles to find the node process
# we can't actually inspect the process using ps because it uses relative paths and doesn't follow symlinks
path=$(lsof -a -g "$procgroup" -F n | grep node12/bin/node | grep externals | tail -1 | cut -c2-)
if [[ $? -eq 0 && -n "$path" ]]
then
# trim the last 5 characters of the path '/node'
trimmedpath=$(dirname "$path")
if [[ $? -eq 0 && -n "$trimmedpath" ]]
then
attemptedtargetedfix=1
# Create the path if it does not exist
if [[ ! -e "$path" ]]
then
date "+[%F %T-%4N] Creating fallback node at path $path" >> "$logfile" 2>&1
mkdir -p "$trimmedpath"
cp "$rootfolder/externals/node12/bin/node" "$path"
else
date "+[%F %T-%4N] Path for fallback node exists, skipping creating $path" >> "$logfile" 2>&1
fi
else
date "+[%F %T-%4N] DarwinRunnerUpgrade: Failed to trim runner path. TrimmedPath: $trimmedpath, path: $path, pgid: $procgroup, root: $rootfolder" >> "$logfile" 2>&1
date "+[%F %T-%4N] DarwinRunnerUpgrade: Failed to trim runner path. TrimmedPath: $trimmedpath, path: $path, pgid: $procgroup, root: $rootfolder" >> "$telemetryfile" 2>&1
fi
else
date "+[%F %T-%4N] DarwinRunnerUpgrade: Failed to find runner path. Path: $path, pgid: $procgroup, root: $rootfolder" >> "$logfile" 2>&1
date "+[%F %T-%4N] DarwinRunnerUpgrade: Failed to find runner path. Path: $path, pgid: $procgroup, root: $rootfolder" >> "$telemetryfile" 2>&1
fi
else
runproc=$(ps x -o pgid,command | grep "run.sh" | grep -v grep | awk '{print $1}')
if [[ $? -eq 0 && -n "$runproc" ]]
then
date "+[%F %T-%4N] Running as ephemeral using run.sh, no need to recreate node folder" >> "$logfile" 2>&1
else
date "+[%F %T-%4N] DarwinRunnerUpgrade: Failed to find runner pgid. pgid: $procgroup, root: $rootfolder" >> "$logfile" 2>&1
date "+[%F %T-%4N] DarwinRunnerUpgrade: Failed to find runner pgid. pgid: $procgroup, root: $rootfolder" >> "$telemetryfile" 2>&1
fi
fi
fi
date "+[%F %T-%4N] Update succeed" >> "$logfile"
# rename the update log file with %logfile%.succeed/.failed/succeedneedrestart

View File

@@ -43,6 +43,21 @@ else
else
sleep 5
fi
elif [[ $returnCode == 4 ]]; then
if [ ! -x "$(command -v sleep)" ]; then
if [ ! -x "$(command -v ping)" ]; then
COUNT="0"
while [[ $COUNT != 5000 ]]; do
echo "SLEEP" > /dev/null
COUNT=$[$COUNT+1]
done
else
ping -c 5 127.0.0.1 > /dev/null
fi
else
sleep 5
fi
"$DIR"/bin/Runner.Listener run $*
else
exit $returnCode
fi

View File

@@ -33,6 +33,9 @@ namespace GitHub.Runner.Common
[DataMember(EmitDefaultValue = false)]
public string PoolName { get; set; }
[DataMember(EmitDefaultValue = false)]
public bool Ephemeral { get; set; }
[DataMember(EmitDefaultValue = false)]
public string ServerUrl { get; set; }

View File

@@ -26,6 +26,7 @@ namespace GitHub.Runner.Common
Certificates,
Options,
SetupInfo,
Telemetry
}
public static class Constants
@@ -125,9 +126,10 @@ namespace GitHub.Runner.Common
{
public static readonly string Check = "check";
public static readonly string Commit = "commit";
public static readonly string Ephemeral = "ephemeral";
public static readonly string Help = "help";
public static readonly string Replace = "replace";
public static readonly string Once = "once";
public static readonly string Once = "once"; // Keep this around since customers still relies on it
public static readonly string RunAsService = "runasservice";
public static readonly string Unattended = "unattended";
public static readonly string Version = "version";
@@ -153,6 +155,7 @@ namespace GitHub.Runner.Common
public static readonly string LowDiskSpace = "LOW_DISK_SPACE";
public static readonly string UnsupportedCommand = "UNSUPPORTED_COMMAND";
public static readonly string UnsupportedCommandMessageDisabled = "The `{0}` command is disabled. Please upgrade to using Environment Files or opt into unsecure command execution by setting the `ACTIONS_ALLOW_UNSECURE_COMMANDS` environment variable to `true`. For more information see: https://github.blog/changelog/2020-10-01-github-actions-deprecating-set-env-and-add-path-commands/";
public static readonly string UnsupportedStopCommandTokenDisabled = "You cannot use a endToken that is an empty string, the string 'pause-logging', or another workflow command. For more information see: https://docs.github.com/en/actions/learn-github-actions/workflow-commands-for-github-actions#example-stopping-and-starting-workflow-commands or opt into insecure command execution by setting the `ACTIONS_ALLOW_UNSECURE_STOPCOMMAND_TOKENS` environment variable to `true`.";
}
public static class RunnerEvent
@@ -212,6 +215,7 @@ namespace GitHub.Runner.Common
// Keep alphabetical
//
public static readonly string AllowUnsupportedCommands = "ACTIONS_ALLOW_UNSECURE_COMMANDS";
public static readonly string AllowUnsupportedStopCommandTokens = "ACTIONS_ALLOW_UNSECURE_STOPCOMMAND_TOKENS";
public static readonly string RunnerDebug = "ACTIONS_RUNNER_DEBUG";
public static readonly string StepDebug = "ACTIONS_STEP_DEBUG";
}

View File

@@ -90,6 +90,8 @@ namespace GitHub.Runner.Common
this.SecretMasker.AddValueEncoder(ValueEncoders.UriDataEscape);
this.SecretMasker.AddValueEncoder(ValueEncoders.XmlDataEscape);
this.SecretMasker.AddValueEncoder(ValueEncoders.TrimDoubleQuotes);
this.SecretMasker.AddValueEncoder(ValueEncoders.PowerShellPreAmpersandEscape);
this.SecretMasker.AddValueEncoder(ValueEncoders.PowerShellPostAmpersandEscape);
// Create the trace manager.
if (string.IsNullOrEmpty(logFile))
@@ -198,9 +200,17 @@ namespace GitHub.Runner.Common
if (credData != null &&
credData.Data.TryGetValue("clientId", out var clientId))
{
_userAgents.Add(new ProductInfoHeaderValue($"RunnerId", clientId));
_userAgents.Add(new ProductInfoHeaderValue("ClientId", clientId));
}
}
var runnerFile = GetConfigFile(WellKnownConfigFile.Runner);
if (File.Exists(runnerFile))
{
var runnerSettings = IOUtil.LoadObject<RunnerSettings>(runnerFile);
_userAgents.Add(new ProductInfoHeaderValue("RunnerId", runnerSettings.AgentId.ToString(CultureInfo.InvariantCulture)));
_userAgents.Add(new ProductInfoHeaderValue("GroupId", runnerSettings.PoolId.ToString(CultureInfo.InvariantCulture)));
}
}
public string GetDirectory(WellKnownDirectory directory)
@@ -340,6 +350,12 @@ namespace GitHub.Runner.Common
GetDirectory(WellKnownDirectory.Root),
".setup_info");
break;
case WellKnownConfigFile.Telemetry:
path = Path.Combine(
GetDirectory(WellKnownDirectory.Diag),
".telemetry");
break;
default:
throw new NotSupportedException($"Unexpected well known config file: '{configFile}'");

View File

@@ -2,8 +2,11 @@
using System;
using System.Collections.Generic;
using System.IO;
using System.Net.Http;
using System.Threading;
using System.Threading.Tasks;
using GitHub.Runner.Sdk;
using GitHub.Services.Common;
using GitHub.Services.WebApi;
namespace GitHub.Runner.Common
@@ -35,7 +38,11 @@ namespace GitHub.Runner.Common
public async Task ConnectAsync(VssConnection jobConnection)
{
_connection = jobConnection;
int attemptCount = 5;
int totalAttempts = 5;
int attemptCount = totalAttempts;
var configurationStore = HostContext.GetService<IConfigurationStore>();
var runnerSettings = configurationStore.GetSettings();
while (!_connection.HasAuthenticated && attemptCount-- > 0)
{
try
@@ -45,17 +52,71 @@ namespace GitHub.Runner.Common
}
catch (Exception ex) when (attemptCount > 0)
{
Trace.Info($"Catch exception during connect. {attemptCount} attemp left.");
Trace.Info($"Catch exception during connect. {attemptCount} attempts left.");
Trace.Error(ex);
if (runnerSettings.IsHostedServer)
{
await CheckNetworkEndpointsAsync(attemptCount);
}
}
await Task.Delay(100);
int attempt = totalAttempts - attemptCount;
TimeSpan backoff = BackoffTimerHelper.GetExponentialBackoff(attempt, TimeSpan.FromMilliseconds(100), TimeSpan.FromSeconds(3.2), TimeSpan.FromMilliseconds(100));
await Task.Delay(backoff);
}
_taskClient = _connection.GetClient<TaskHttpClient>();
_hasConnection = true;
}
private async Task CheckNetworkEndpointsAsync(int attemptsLeft)
{
try
{
Trace.Info("Requesting Actions Service health endpoint status");
using (var httpClientHandler = HostContext.CreateHttpClientHandler())
using (var actionsClient = new HttpClient(httpClientHandler))
{
var baseUri = new Uri(_connection.Uri.GetLeftPart(UriPartial.Authority));
actionsClient.DefaultRequestHeaders.UserAgent.AddRange(HostContext.UserAgents);
// Call the _apis/health endpoint, and include how many attempts are left as a URL query for easy tracking
var response = await actionsClient.GetAsync(new Uri(baseUri, $"_apis/health?_internalRunnerAttemptsLeft={attemptsLeft}"));
Trace.Info($"Actions health status code: {response.StatusCode}");
}
}
catch (Exception ex)
{
// Log error, but continue as this call is best-effort
Trace.Info($"Actions Service health endpoint failed due to {ex.GetType().Name}");
Trace.Error(ex);
}
try
{
Trace.Info("Requesting Github API endpoint status");
// This is a dotcom public API... just call it directly
using (var httpClientHandler = HostContext.CreateHttpClientHandler())
using (var gitHubClient = new HttpClient(httpClientHandler))
{
gitHubClient.DefaultRequestHeaders.UserAgent.AddRange(HostContext.UserAgents);
// Call the api.github.com endpoint, and include how many attempts are left as a URL query for easy tracking
var response = await gitHubClient.GetAsync($"https://api.github.com?_internalRunnerAttemptsLeft={attemptsLeft}");
Trace.Info($"api.github.com status code: {response.StatusCode}");
}
}
catch (Exception ex)
{
// Log error, but continue as this call is best-effort
Trace.Info($"Github API endpoint failed due to {ex.GetType().Name}");
Trace.Error(ex);
}
}
private void CheckConnection()
{
if (!_hasConnection)

View File

@@ -15,6 +15,7 @@ namespace GitHub.Runner.Common
[ServiceLocator(Default = typeof(JobServerQueue))]
public interface IJobServerQueue : IRunnerService, IThrottlingReporter
{
TaskCompletionSource<int> JobRecordUpdated { get; }
event EventHandler<ThrottlingEventArgs> JobServerQueueThrottling;
Task ShutdownAsync();
void Start(Pipelines.AgentJobRequestMessage jobRequest);
@@ -62,8 +63,11 @@ namespace GitHub.Runner.Common
private IJobServer _jobServer;
private Task[] _allDequeueTasks;
private readonly TaskCompletionSource<int> _jobCompletionSource = new TaskCompletionSource<int>();
private readonly TaskCompletionSource<int> _jobRecordUpdated = new TaskCompletionSource<int>();
private bool _queueInProcess = false;
public TaskCompletionSource<int> JobRecordUpdated => _jobRecordUpdated;
public event EventHandler<ThrottlingEventArgs> JobServerQueueThrottling;
// Web console dequeue will start with process queue every 250ms for the first 60*4 times (~60 seconds).
@@ -287,11 +291,11 @@ namespace GitHub.Runner.Common
{
await _jobServer.AppendTimelineRecordFeedAsync(_scopeIdentifier, _hubName, _planId, _jobTimelineId, _jobTimelineRecordId, stepRecordId, batch.Select(logLine => logLine.Line).ToList(), batch[0].LineNumber.Value, default(CancellationToken));
}
else
else
{
await _jobServer.AppendTimelineRecordFeedAsync(_scopeIdentifier, _hubName, _planId, _jobTimelineId, _jobTimelineRecordId, stepRecordId, batch.Select(logLine => logLine.Line).ToList(), default(CancellationToken));
}
if (_firstConsoleOutputs)
{
HostContext.WritePerfCounter($"WorkerJobServerQueueAppendFirstConsoleOutput_{_planId.ToString()}");
@@ -455,6 +459,14 @@ namespace GitHub.Runner.Common
{
Trace.Verbose("Cleanup buffered timeline record for timeline: {0}.", update.TimelineId);
}
if (!_jobRecordUpdated.Task.IsCompleted &&
update.PendingRecords.Any(x => x.Id == _jobTimelineRecordId && x.State != null))
{
// We have changed the state of the job
Trace.Info("Job timeline record has been updated for the first time.");
_jobRecordUpdated.TrySetResult(0);
}
}
catch (Exception ex)
{

View File

@@ -5,7 +5,6 @@
<OutputType>Library</OutputType>
<RuntimeIdentifiers>win-x64;win-x86;linux-x64;linux-arm64;linux-arm;osx-x64</RuntimeIdentifiers>
<TargetLatestRuntimePatch>true</TargetLatestRuntimePatch>
<AssetTargetFallback>portable-net45+win8</AssetTargetFallback>
<NoWarn>NU1701;NU1603</NoWarn>
<Version>$(Version)</Version>
<TieredCompilationQuickJit>true</TieredCompilationQuickJit>

View File

@@ -29,8 +29,10 @@ namespace GitHub.Runner.Common
// Configuration
Task<TaskAgent> AddAgentAsync(Int32 agentPoolId, TaskAgent agent);
Task DeleteAgentAsync(int agentPoolId, int agentId);
Task DeleteAgentAsync(int agentId);
Task<List<TaskAgentPool>> GetAgentPoolsAsync(string agentPoolName = null, TaskAgentPoolType poolType = TaskAgentPoolType.Automation);
Task<List<TaskAgent>> GetAgentsAsync(int agentPoolId, string agentName = null);
Task<List<TaskAgent>> GetAgentsAsync(string agentName);
Task<TaskAgent> ReplaceAgentAsync(int agentPoolId, TaskAgent agent);
// messagequeue
@@ -49,7 +51,7 @@ namespace GitHub.Runner.Common
Task<PackageMetadata> GetPackageAsync(string packageType, string platform, string version, bool includeToken, CancellationToken cancellationToken);
// agent update
Task<TaskAgent> UpdateAgentUpdateStateAsync(int agentPoolId, int agentId, string currentState);
Task<TaskAgent> UpdateAgentUpdateStateAsync(int agentPoolId, int agentId, string currentState, string trace);
}
public sealed class RunnerServer : RunnerService, IRunnerServer
@@ -252,6 +254,11 @@ namespace GitHub.Runner.Common
return _genericTaskAgentClient.GetAgentsAsync(agentPoolId, agentName, false);
}
public Task<List<TaskAgent>> GetAgentsAsync(string agentName)
{
return GetAgentsAsync(0, agentName); // search in all all agentPools
}
public Task<TaskAgent> ReplaceAgentAsync(int agentPoolId, TaskAgent agent)
{
CheckConnection(RunnerConnectionType.Generic);
@@ -264,6 +271,11 @@ namespace GitHub.Runner.Common
return _genericTaskAgentClient.DeleteAgentAsync(agentPoolId, agentId);
}
public Task DeleteAgentAsync(int agentId)
{
return DeleteAgentAsync(0, agentId); // agentPool is ignored server side
}
//-----------------------------------------------------------------
// MessageQueue
//-----------------------------------------------------------------
@@ -329,25 +341,10 @@ namespace GitHub.Runner.Common
return _genericTaskAgentClient.GetPackageAsync(packageType, platform, version, includeToken, cancellationToken: cancellationToken);
}
public Task<TaskAgent> UpdateAgentUpdateStateAsync(int agentPoolId, int agentId, string currentState)
public Task<TaskAgent> UpdateAgentUpdateStateAsync(int agentPoolId, int agentId, string currentState, string trace)
{
CheckConnection(RunnerConnectionType.Generic);
return _genericTaskAgentClient.UpdateAgentUpdateStateAsync(agentPoolId, agentId, currentState);
}
//-----------------------------------------------------------------
// Runner Auth Url
//-----------------------------------------------------------------
public Task<string> GetRunnerAuthUrlAsync(int runnerPoolId, int runnerId)
{
CheckConnection(RunnerConnectionType.MessageQueue);
return _messageTaskAgentClient.GetAgentAuthUrlAsync(runnerPoolId, runnerId);
}
public Task ReportRunnerAuthUrlErrorAsync(int runnerPoolId, int runnerId, string error)
{
CheckConnection(RunnerConnectionType.MessageQueue);
return _messageTaskAgentClient.ReportAgentAuthUrlMigrationErrorAsync(runnerPoolId, runnerId, error);
return _genericTaskAgentClient.UpdateAgentUpdateStateAsync(agentPoolId, agentId, currentState, trace);
}
}
}

View File

@@ -164,9 +164,8 @@ namespace GitHub.Runner.Common
if (!Silent)
{
Console.WriteLine();
Console.ForegroundColor = ConsoleColor.White;
Console.WriteLine($"# {message}");
Console.ResetColor();
Console.WriteLine($"# {message}");
Console.WriteLine();
}
}
@@ -177,9 +176,8 @@ namespace GitHub.Runner.Common
{
Console.ForegroundColor = ConsoleColor.Green;
Console.Write("√ ");
Console.ForegroundColor = ConsoleColor.White;
Console.WriteLine(message);
Console.ResetColor();
Console.WriteLine(message);
}
}

View File

@@ -29,10 +29,11 @@ namespace GitHub.Runner.Listener
{
Constants.Runner.CommandLine.Flags.Check,
Constants.Runner.CommandLine.Flags.Commit,
Constants.Runner.CommandLine.Flags.Ephemeral,
Constants.Runner.CommandLine.Flags.Help,
Constants.Runner.CommandLine.Flags.Once,
Constants.Runner.CommandLine.Flags.Replace,
Constants.Runner.CommandLine.Flags.RunAsService,
Constants.Runner.CommandLine.Flags.Once,
Constants.Runner.CommandLine.Flags.Unattended,
Constants.Runner.CommandLine.Flags.Version
};
@@ -66,7 +67,9 @@ namespace GitHub.Runner.Listener
public bool Help => TestFlag(Constants.Runner.CommandLine.Flags.Help);
public bool Unattended => TestFlag(Constants.Runner.CommandLine.Flags.Unattended);
public bool Version => TestFlag(Constants.Runner.CommandLine.Flags.Version);
public bool Ephemeral => TestFlag(Constants.Runner.CommandLine.Flags.Ephemeral);
// Keep this around since customers still relies on it
public bool RunOnce => TestFlag(Constants.Runner.CommandLine.Flags.Once);
// Constructor.

View File

@@ -22,6 +22,7 @@ namespace GitHub.Runner.Listener.Configuration
bool IsConfigured();
Task ConfigureAsync(CommandSettings command);
Task UnconfigureAsync(CommandSettings command);
void DeleteLocalRunnerConfig();
RunnerSettings LoadSettings();
}
@@ -65,18 +66,18 @@ namespace GitHub.Runner.Listener.Configuration
public async Task ConfigureAsync(CommandSettings command)
{
_term.WriteLine();
_term.WriteLine("--------------------------------------------------------------------------------", ConsoleColor.White);
_term.WriteLine("| ____ _ _ _ _ _ _ _ _ |", ConsoleColor.White);
_term.WriteLine("| / ___(_) |_| | | |_ _| |__ / \\ ___| |_(_) ___ _ __ ___ |", ConsoleColor.White);
_term.WriteLine("| | | _| | __| |_| | | | | '_ \\ / _ \\ / __| __| |/ _ \\| '_ \\/ __| |", ConsoleColor.White);
_term.WriteLine("| | |_| | | |_| _ | |_| | |_) | / ___ \\ (__| |_| | (_) | | | \\__ \\ |", ConsoleColor.White);
_term.WriteLine("| \\____|_|\\__|_| |_|\\__,_|_.__/ /_/ \\_\\___|\\__|_|\\___/|_| |_|___/ |", ConsoleColor.White);
_term.WriteLine("| |", ConsoleColor.White);
_term.Write("| ", ConsoleColor.White);
_term.WriteLine("--------------------------------------------------------------------------------");
_term.WriteLine("| ____ _ _ _ _ _ _ _ _ |");
_term.WriteLine("| / ___(_) |_| | | |_ _| |__ / \\ ___| |_(_) ___ _ __ ___ |");
_term.WriteLine("| | | _| | __| |_| | | | | '_ \\ / _ \\ / __| __| |/ _ \\| '_ \\/ __| |");
_term.WriteLine("| | |_| | | |_| _ | |_| | |_) | / ___ \\ (__| |_| | (_) | | | \\__ \\ |");
_term.WriteLine("| \\____|_|\\__|_| |_|\\__,_|_.__/ /_/ \\_\\___|\\__|_|\\___/|_| |_|___/ |");
_term.WriteLine("| |");
_term.Write("| ");
_term.Write("Self-hosted runner registration", ConsoleColor.Cyan);
_term.WriteLine(" |", ConsoleColor.White);
_term.WriteLine("| |", ConsoleColor.White);
_term.WriteLine("--------------------------------------------------------------------------------", ConsoleColor.White);
_term.WriteLine(" |");
_term.WriteLine("| |");
_term.WriteLine("--------------------------------------------------------------------------------");
Trace.Info(nameof(ConfigureAsync));
if (IsConfigured())
@@ -117,6 +118,7 @@ namespace GitHub.Runner.Listener.Configuration
try
{
// Determine the service deployment type based on connection data. (Hosted/OnPremises)
// Hosted usually means github.com or localhost, while OnPremises means GHES or GHAE
runnerSettings.IsHostedServer = runnerSettings.GitHubUrl == null || UrlUtil.IsHostedServer(new UriBuilder(runnerSettings.GitHubUrl));
// Warn if the Actions server url and GHES server url has different Host
@@ -194,6 +196,7 @@ namespace GitHub.Runner.Listener.Configuration
TaskAgent agent;
while (true)
{
runnerSettings.Ephemeral = command.Ephemeral;
runnerSettings.AgentName = command.GetRunnerName();
_term.WriteLine();
@@ -210,7 +213,7 @@ namespace GitHub.Runner.Listener.Configuration
if (command.GetReplace())
{
// Update existing agent with new PublicKey, agent version.
agent = UpdateExistingAgent(agent, publicKey, userLabels);
agent = UpdateExistingAgent(agent, publicKey, userLabels, runnerSettings.Ephemeral);
try
{
@@ -233,7 +236,7 @@ namespace GitHub.Runner.Listener.Configuration
else
{
// Create a new agent.
agent = CreateNewAgent(runnerSettings.AgentName, publicKey, userLabels);
agent = CreateNewAgent(runnerSettings.AgentName, publicKey, userLabels, runnerSettings.Ephemeral);
try
{
@@ -327,6 +330,38 @@ namespace GitHub.Runner.Listener.Configuration
#endif
}
// Delete .runner and .credentials files
public void DeleteLocalRunnerConfig()
{
bool isConfigured = _store.IsConfigured();
bool hasCredentials = _store.HasCredentials();
//delete credential config files
var currentAction = "Removing .credentials";
if (hasCredentials)
{
_store.DeleteCredential();
var keyManager = HostContext.GetService<IRSAKeyManager>();
keyManager.DeleteKey();
_term.WriteSuccessMessage("Removed .credentials");
}
else
{
_term.WriteLine("Does not exist. Skipping " + currentAction);
}
//delete settings config file
currentAction = "Removing .runner";
if (isConfigured)
{
_store.DeleteSettings();
_term.WriteSuccessMessage("Removed .runner");
}
else
{
_term.WriteLine("Does not exist. Skipping " + currentAction);
}
}
public async Task UnconfigureAsync(CommandSettings command)
{
string currentAction = string.Empty;
@@ -346,12 +381,9 @@ namespace GitHub.Runner.Listener.Configuration
_term.WriteLine();
_term.WriteSuccessMessage("Runner service removed");
#elif OS_LINUX
// unconfig system D service first
throw new Exception("Unconfigure service first");
#elif OS_OSX
// unconfig osx service first
throw new Exception("Unconfigure service first");
#else
// unconfig systemd or osx service first
throw new Exception("Uninstall service first");
#endif
}
@@ -383,7 +415,7 @@ namespace GitHub.Runner.Listener.Configuration
// Determine the service deployment type based on connection data. (Hosted/OnPremises)
await _runnerServer.ConnectAsync(new Uri(settings.ServerUrl), creds);
var agents = await _runnerServer.GetAgentsAsync(settings.PoolId, settings.AgentName);
var agents = await _runnerServer.GetAgentsAsync(settings.AgentName);
Trace.Verbose("Returns {0} agents", agents.Count);
TaskAgent agent = agents.FirstOrDefault();
if (agent == null)
@@ -392,7 +424,7 @@ namespace GitHub.Runner.Listener.Configuration
}
else
{
await _runnerServer.DeleteAgentAsync(settings.PoolId, settings.AgentId);
await _runnerServer.DeleteAgentAsync(settings.AgentId);
_term.WriteLine();
_term.WriteSuccessMessage("Runner removed successfully");
@@ -403,31 +435,7 @@ namespace GitHub.Runner.Listener.Configuration
_term.WriteLine("Cannot connect to server, because config files are missing. Skipping removing runner from the server.");
}
//delete credential config files
currentAction = "Removing .credentials";
if (hasCredentials)
{
_store.DeleteCredential();
var keyManager = HostContext.GetService<IRSAKeyManager>();
keyManager.DeleteKey();
_term.WriteSuccessMessage("Removed .credentials");
}
else
{
_term.WriteLine("Does not exist. Skipping " + currentAction);
}
//delete settings config file
currentAction = "Removing .runner";
if (isConfigured)
{
_store.DeleteSettings();
_term.WriteSuccessMessage("Removed .runner");
}
else
{
_term.WriteLine("Does not exist. Skipping " + currentAction);
}
DeleteLocalRunnerConfig();
}
catch (Exception)
{
@@ -458,7 +466,7 @@ namespace GitHub.Runner.Listener.Configuration
}
private TaskAgent UpdateExistingAgent(TaskAgent agent, RSAParameters publicKey, ISet<string> userLabels)
private TaskAgent UpdateExistingAgent(TaskAgent agent, RSAParameters publicKey, ISet<string> userLabels, bool ephemeral)
{
ArgUtil.NotNull(agent, nameof(agent));
agent.Authorization = new TaskAgentAuthorization
@@ -469,6 +477,8 @@ namespace GitHub.Runner.Listener.Configuration
// update should replace the existing labels
agent.Version = BuildConstants.RunnerPackage.Version;
agent.OSDescription = RuntimeInformation.OSDescription;
agent.Ephemeral = ephemeral;
agent.MaxParallelism = 1;
agent.Labels.Clear();
@@ -484,7 +494,7 @@ namespace GitHub.Runner.Listener.Configuration
return agent;
}
private TaskAgent CreateNewAgent(string agentName, RSAParameters publicKey, ISet<string> userLabels)
private TaskAgent CreateNewAgent(string agentName, RSAParameters publicKey, ISet<string> userLabels, bool ephemeral)
{
TaskAgent agent = new TaskAgent(agentName)
{
@@ -495,6 +505,7 @@ namespace GitHub.Runner.Listener.Configuration
MaxParallelism = 1,
Version = BuildConstants.RunnerPackage.Version,
OSDescription = RuntimeInformation.OSDescription,
Ephemeral = ephemeral,
};
agent.Labels.Add(new AgentLabel("self-hosted", LabelType.System));

View File

@@ -85,7 +85,7 @@ namespace GitHub.Runner.Listener.Configuration
while (true)
{
// Write the message prompt.
_terminal.Write($"{description} ", ConsoleColor.White);
_terminal.Write($"{description} ");
if(!string.IsNullOrEmpty(defaultValue))
{

View File

@@ -36,7 +36,10 @@ namespace GitHub.Runner.Listener
{
private readonly Lazy<Dictionary<long, TaskResult>> _localRunJobResult = new Lazy<Dictionary<long, TaskResult>>();
private int _poolId;
RunnerSettings _runnerSetting;
IConfigurationStore _configurationStore;
RunnerSettings _runnerSettings;
private static readonly string _workerProcessName = $"Runner.Worker{IOUtil.ExeExtension}";
// this is not thread-safe
@@ -54,9 +57,9 @@ namespace GitHub.Runner.Listener
base.Initialize(hostContext);
// get pool id from config
var configurationStore = hostContext.GetService<IConfigurationStore>();
_runnerSetting = configurationStore.GetSettings();
_poolId = _runnerSetting.PoolId;
_configurationStore = hostContext.GetService<IConfigurationStore>();
_runnerSettings = _configurationStore.GetSettings();
_poolId = _runnerSettings.PoolId;
int channelTimeoutSeconds;
if (!int.TryParse(Environment.GetEnvironmentVariable("GITHUB_ACTIONS_RUNNER_CHANNEL_TIMEOUT") ?? string.Empty, out channelTimeoutSeconds))
@@ -661,13 +664,15 @@ namespace GitHub.Runner.Listener
try
{
request = await runnerServer.RenewAgentRequestAsync(poolId, requestId, lockToken, orchestrationId, token);
Trace.Info($"Successfully renew job request {requestId}, job is valid till {request.LockedUntil.Value}");
if (!firstJobRequestRenewed.Task.IsCompleted)
{
// fire first renew succeed event.
firstJobRequestRenewed.TrySetResult(0);
// Update settings if the runner name has been changed server-side
UpdateAgentNameIfNeeded(request.ReservedAgent?.Name);
}
if (encounteringError > 0)
@@ -767,6 +772,27 @@ namespace GitHub.Runner.Listener
}
}
private void UpdateAgentNameIfNeeded(string agentName)
{
var isNewAgentName = !string.Equals(_runnerSettings.AgentName, agentName, StringComparison.Ordinal);
if (!isNewAgentName || string.IsNullOrEmpty(agentName))
{
return;
}
_runnerSettings.AgentName = agentName;
try
{
_configurationStore.SaveSettings(_runnerSettings);
}
catch (Exception ex)
{
Trace.Error("Cannot update the settings file:");
Trace.Error(ex);
}
}
// Best effort upload any logs for this job.
private async Task TryUploadUnfinishedLogs(Pipelines.AgentJobRequestMessage message)
{

View File

@@ -5,7 +5,6 @@
<OutputType>Exe</OutputType>
<RuntimeIdentifiers>win-x64;win-x86;linux-x64;linux-arm64;linux-arm;osx-x64</RuntimeIdentifiers>
<TargetLatestRuntimePatch>true</TargetLatestRuntimePatch>
<AssetTargetFallback>portable-net45+win8</AssetTargetFallback>
<NoWarn>NU1701;NU1603</NoWarn>
<Version>$(Version)</Version>
<TieredCompilationQuickJit>true</TieredCompilationQuickJit>

View File

@@ -214,7 +214,7 @@ namespace GitHub.Runner.Listener
var startupTypeAsString = command.GetStartupType();
if (string.IsNullOrEmpty(startupTypeAsString) && configuredAsService)
{
// We need try our best to make the startup type accurate
// We need try our best to make the startup type accurate
// The problem is coming from runner autoupgrade, which result an old version service host binary but a newer version runner binary
// At that time the servicehost won't pass --startuptype to Runner.Listener while the runner is actually running as service.
// We will guess the startup type only when the runner is configured as service and the guess will based on whether STDOUT/STDERR/STDIN been redirect or not
@@ -233,8 +233,14 @@ namespace GitHub.Runner.Listener
Trace.Info($"Set runner startup type - {startType}");
HostContext.StartupType = startType;
if (command.RunOnce)
{
_term.WriteLine("Warning: '--once' is going to be deprecated in the future, please consider using '--ephemeral' during runner registration.", ConsoleColor.Yellow);
_term.WriteLine("https://docs.github.com/en/actions/hosting-your-own-runners/autoscaling-with-self-hosted-runners#using-ephemeral-runners-for-autoscaling", ConsoleColor.Yellow);
}
// Run the runner interactively or as service
return await RunAsync(settings, command.RunOnce);
return await RunAsync(settings, command.RunOnce || settings.Ephemeral);
}
else
{
@@ -306,10 +312,15 @@ namespace GitHub.Runner.Listener
}
HostContext.WritePerfCounter("SessionCreated");
_term.WriteLine($"Current runner version: '{BuildConstants.RunnerPackage.Version}'");
_term.WriteLine($"{DateTime.UtcNow:u}: Listening for Jobs");
IJobDispatcher jobDispatcher = null;
CancellationTokenSource messageQueueLoopTokenSource = CancellationTokenSource.CreateLinkedTokenSource(HostContext.RunnerShutdownToken);
// Should we try to cleanup ephemeral runners
bool runOnceJobCompleted = false;
try
{
var notification = HostContext.GetService<IJobNotification>();
@@ -371,6 +382,7 @@ namespace GitHub.Runner.Listener
Task completeTask = await Task.WhenAny(getNextMessage, jobDispatcher.RunOnceJobCompleted.Task);
if (completeTask == jobDispatcher.RunOnceJobCompleted.Task)
{
runOnceJobCompleted = true;
Trace.Info("Job has finished at backend, the runner will exit since it is running under onetime use mode.");
Trace.Info("Stop message queue looping.");
messageQueueLoopTokenSource.Cancel();
@@ -466,10 +478,24 @@ namespace GitHub.Runner.Listener
await jobDispatcher.ShutdownAsync();
}
//TODO: make sure we don't mask more important exception
await _listener.DeleteSessionAsync();
try
{
await _listener.DeleteSessionAsync();
}
catch (Exception ex) when (runOnce)
{
// ignore exception during delete session for ephemeral runner since the runner might already be deleted from the server side
// and the delete session call will ends up with 401.
Trace.Info($"Ignore any exception during DeleteSession for an ephemeral runner. {ex}");
}
messageQueueLoopTokenSource.Dispose();
if (settings.Ephemeral && runOnceJobCompleted)
{
var configManager = HostContext.GetService<IConfigurationManager>();
configManager.DeleteLocalRunnerConfig();
}
}
}
catch (TaskAgentAccessTokenExpiredException)
@@ -512,7 +538,9 @@ Config Options:
--labels string Extra labels in addition to the default: 'self-hosted,{Constants.Runner.Platform},{Constants.Runner.PlatformArchitecture}'
--work string Relative runner work directory (default {Constants.Path.WorkDirectory})
--replace Replace any existing runner with the same name (default false)
--pat GitHub personal access token used for checking network connectivity when executing `.{separator}run.{ext} --check`");
--pat GitHub personal access token used for checking network connectivity when executing `.{separator}run.{ext} --check`
--ephemeral Configure the runner to only take one job and then let the service un-configure the runner after the job finishes (default false)");
#if OS_WINDOWS
_term.WriteLine($@" --runasservice Run the runner as a service");
_term.WriteLine($@" --windowslogonaccount string Account to run the service as. Requires runasservice");

View File

@@ -13,6 +13,7 @@ using GitHub.Services.WebApi;
using GitHub.Services.Common;
using GitHub.Runner.Common;
using GitHub.Runner.Sdk;
using System.Text;
namespace GitHub.Runner.Listener
{
@@ -63,23 +64,25 @@ namespace GitHub.Runner.Listener
// Print console line that warn user not shutdown runner.
await UpdateRunnerUpdateStateAsync("Runner update in progress, do not shutdown runner.");
await UpdateRunnerUpdateStateAsync($"Downloading {_targetPackage.Version} runner");
await UpdateRunnerUpdateStateAsync($"Downloading {_targetPackage.Version} runner", $"RunnerPlatform: {_targetPackage.Platform}");
await DownloadLatestRunner(token);
var downloadTrace = await DownloadLatestRunner(token);
Trace.Info($"Download latest runner and unzip into runner root.");
// wait till all running job finish
await UpdateRunnerUpdateStateAsync("Waiting for current job finish running.");
await UpdateRunnerUpdateStateAsync("Waiting for current job finish running.", downloadTrace);
await jobDispatcher.WaitAsync(token);
Trace.Info($"All running job has exited.");
// We need to keep runner backup around for macOS until we fixed https://github.com/actions/runner/issues/743
// delete runner backup
var stopWatch = Stopwatch.StartNew();
DeletePreviousVersionRunnerBackup(token);
Trace.Info($"Delete old version runner backup.");
stopWatch.Stop();
// generate update script from template
await UpdateRunnerUpdateStateAsync("Generate and execute update script.");
await UpdateRunnerUpdateStateAsync("Generate and execute update script.", $"DeleteRunnerBackupTime: {stopWatch.ElapsedMilliseconds}ms");
string updateScript = GenerateUpdateScript(restartInteractiveRunner);
Trace.Info($"Generate update script into: {updateScript}");
@@ -96,7 +99,7 @@ namespace GitHub.Runner.Listener
invokeScript.Start();
Trace.Info($"Update script start running");
await UpdateRunnerUpdateStateAsync("Runner will exit shortly for update, should back online within 10 seconds.");
await UpdateRunnerUpdateStateAsync("Runner will exit shortly for update, should be back online within 10 seconds.", $"RestartInteractiveRunner: {restartInteractiveRunner}");
return true;
}
@@ -150,8 +153,10 @@ namespace GitHub.Runner.Listener
/// </summary>
/// <param name="token"></param>
/// <returns></returns>
private async Task DownloadLatestRunner(CancellationToken token)
private async Task<string> DownloadLatestRunner(CancellationToken token)
{
var traceStringBuilder = new StringBuilder();
traceStringBuilder.AppendLine($"DownloadUrl: {_targetPackage.DownloadUrl}");
string latestRunnerDirectory = Path.Combine(HostContext.GetDirectory(WellKnownDirectory.Work), Constants.Path.UpdateDirectory);
IOUtil.DeleteDirectory(latestRunnerDirectory, token);
Directory.CreateDirectory(latestRunnerDirectory);
@@ -160,6 +165,7 @@ namespace GitHub.Runner.Listener
string archiveFile = null;
bool downloadSucceeded = false;
var stopWatch = Stopwatch.StartNew();
try
{
// Download the runner, using multiple attempts in order to be resilient against any networking/CDN issues
@@ -210,6 +216,7 @@ namespace GitHub.Runner.Listener
try
{
Trace.Info($"Download runner: begin download");
long downloadSize = 0;
//open zip stream in async mode
using (HttpClient httpClient = new HttpClient(HostContext.CreateHttpClientHandler()))
@@ -228,11 +235,16 @@ namespace GitHub.Runner.Listener
//81920 is the default used by System.IO.Stream.CopyTo and is under the large object heap threshold (85k).
await result.CopyToAsync(fs, 81920, downloadCts.Token);
await fs.FlushAsync(downloadCts.Token);
downloadSize = fs.Length;
}
}
Trace.Info($"Download runner: finished download");
downloadSucceeded = true;
stopWatch.Stop();
traceStringBuilder.AppendLine($"PackageDownloadTime: {stopWatch.ElapsedMilliseconds}ms");
traceStringBuilder.AppendLine($"Attempts: {attempt}");
traceStringBuilder.AppendLine($"PackageSize: {downloadSize / 1024 / 1024}MB");
break;
}
catch (OperationCanceledException) when (token.IsCancellationRequested)
@@ -257,6 +269,7 @@ namespace GitHub.Runner.Listener
throw new TaskCanceledException($"Runner package '{archiveFile}' failed after {Constants.RunnerDownloadRetryMaxAttempts} download attempts");
}
stopWatch.Restart();
// If we got this far, we know that we've successfully downloaded the runner package
// Validate Hash Matches if it is provided
using (FileStream stream = File.OpenRead(archiveFile))
@@ -320,7 +333,9 @@ namespace GitHub.Runner.Listener
throw new NotSupportedException($"{archiveFile}");
}
stopWatch.Stop();
Trace.Info($"Finished getting latest runner package at: {latestRunnerDirectory}.");
traceStringBuilder.AppendLine($"PackageExtractTime: {stopWatch.ElapsedMilliseconds}ms");
}
finally
{
@@ -340,6 +355,7 @@ namespace GitHub.Runner.Listener
}
}
stopWatch.Restart();
// copy latest runner into runner root folder
// copy bin from _work/_update -> bin.version under root
string binVersionDir = Path.Combine(HostContext.GetDirectory(WellKnownDirectory.Root), $"{Constants.Path.BinDirectory}.{_targetPackage.Version}");
@@ -365,6 +381,11 @@ namespace GitHub.Runner.Listener
IOUtil.DeleteFile(destination);
file.CopyTo(destination, true);
}
stopWatch.Stop();
traceStringBuilder.AppendLine($"CopyRunnerToRootTime: {stopWatch.ElapsedMilliseconds}ms");
return traceStringBuilder.ToString();
}
private void DeletePreviousVersionRunnerBackup(CancellationToken token)
@@ -484,13 +505,18 @@ namespace GitHub.Runner.Listener
return updateScript;
}
private async Task UpdateRunnerUpdateStateAsync(string currentState)
private async Task UpdateRunnerUpdateStateAsync(string currentState, string trace = "")
{
_terminal.WriteLine(currentState);
if (!string.IsNullOrEmpty(trace))
{
Trace.Info(trace);
}
try
{
await _runnerServer.UpdateAgentUpdateStateAsync(_poolId, _agentId, currentState);
await _runnerServer.UpdateAgentUpdateStateAsync(_poolId, _agentId, currentState, trace);
}
catch (VssResourceNotFoundException)
{

View File

@@ -5,7 +5,6 @@
<OutputType>Exe</OutputType>
<RuntimeIdentifiers>win-x64;win-x86;linux-x64;linux-arm64;linux-arm;osx-x64</RuntimeIdentifiers>
<TargetLatestRuntimePatch>true</TargetLatestRuntimePatch>
<AssetTargetFallback>portable-net45+win8</AssetTargetFallback>
<NoWarn>NU1701;NU1603</NoWarn>
<Version>$(Version)</Version>
<TieredCompilationQuickJit>true</TieredCompilationQuickJit>

View File

@@ -5,7 +5,6 @@
<OutputType>Library</OutputType>
<RuntimeIdentifiers>win-x64;win-x86;linux-x64;linux-arm64;linux-arm;osx-x64</RuntimeIdentifiers>
<TargetLatestRuntimePatch>true</TargetLatestRuntimePatch>
<AssetTargetFallback>portable-net45+win8</AssetTargetFallback>
<NoWarn>NU1701;NU1603</NoWarn>
<Version>$(Version)</Version>
<TieredCompilationQuickJit>true</TieredCompilationQuickJit>

View File

@@ -5,7 +5,6 @@
<OutputType>Library</OutputType>
<RuntimeIdentifiers>win-x64;win-x86;linux-x64;linux-arm64;linux-arm;osx-x64</RuntimeIdentifiers>
<TargetLatestRuntimePatch>true</TargetLatestRuntimePatch>
<AssetTargetFallback>portable-net45+win8</AssetTargetFallback>
<NoWarn>NU1701;NU1603</NoWarn>
<Version>$(Version)</Version>
<TieredCompilationQuickJit>true</TieredCompilationQuickJit>

View File

@@ -1,7 +1,5 @@
using GitHub.DistributedTask.Pipelines;
using GitHub.DistributedTask.Pipelines.ContextData;
using GitHub.DistributedTask.Pipelines.ContextData;
using GitHub.DistributedTask.WebApi;
using GitHub.Runner.Common.Util;
using GitHub.Runner.Worker.Container;
using System;
using System.Collections.Generic;
@@ -110,11 +108,18 @@ namespace GitHub.Runner.Worker
// Stop command
if (string.Equals(actionCommand.Command, _stopCommand, StringComparison.OrdinalIgnoreCase))
{
context.Output(input);
context.Debug("Paused processing commands until '##[{actionCommand.Data}]' is received");
ValidateStopToken(context, actionCommand.Data);
_stopToken = actionCommand.Data;
_stopProcessCommand = true;
_registeredCommands.Add(_stopToken);
if (_stopToken.Length > 6)
{
HostContext.SecretMasker.AddValue(_stopToken);
}
context.Output(input);
context.Debug("Paused processing commands until the token you called ::stopCommands:: with is received");
return true;
}
// Found command
@@ -148,7 +153,42 @@ namespace GitHub.Runner.Worker
return true;
}
internal static bool EnhancedAnnotationsEnabled(IExecutionContext context) {
private void ValidateStopToken(IExecutionContext context, string stopToken)
{
#if OS_WINDOWS
var envContext = context.ExpressionValues["env"] as DictionaryContextData;
#else
var envContext = context.ExpressionValues["env"] as CaseSensitiveDictionaryContextData;
#endif
var allowUnsecureStopCommandTokens = false;
allowUnsecureStopCommandTokens = StringUtil.ConvertToBoolean(Environment.GetEnvironmentVariable(Constants.Variables.Actions.AllowUnsupportedStopCommandTokens));
if (!allowUnsecureStopCommandTokens && envContext.ContainsKey(Constants.Variables.Actions.AllowUnsupportedStopCommandTokens))
{
allowUnsecureStopCommandTokens = StringUtil.ConvertToBoolean(envContext[Constants.Variables.Actions.AllowUnsupportedStopCommandTokens].ToString());
}
bool isTokenInvalid = _registeredCommands.Contains(stopToken)
|| string.IsNullOrEmpty(stopToken)
|| string.Equals(stopToken, "pause-logging", StringComparison.OrdinalIgnoreCase);
if (isTokenInvalid)
{
var telemetry = new JobTelemetry
{
Message = $"Invoked ::stopCommand:: with token: [{stopToken}]",
Type = JobTelemetryType.ActionCommand
};
context.JobTelemetry.Add(telemetry);
}
if (isTokenInvalid && !allowUnsecureStopCommandTokens)
{
throw new Exception(Constants.Runner.UnsupportedStopCommandTokenDisabled);
}
}
internal static bool EnhancedAnnotationsEnabled(IExecutionContext context)
{
return context.Global.Variables.GetBoolean("DistributedTask.EnhancedAnnotations") ?? false;
}
}
@@ -252,7 +292,7 @@ namespace GitHub.Runner.Worker
public const String Name = "name";
}
private string[] _setEnvBlockList =
private string[] _setEnvBlockList =
{
"NODE_OPTIONS"
};
@@ -353,7 +393,7 @@ namespace GitHub.Runner.Worker
public Type ExtensionType => typeof(IActionCommandExtension);
public void ProcessCommand(IExecutionContext context, string line, ActionCommand command, ContainerInfo container)
{
{
var allowUnsecureCommands = false;
bool.TryParse(Environment.GetEnvironmentVariable(Constants.Variables.Actions.AllowUnsupportedCommands), out allowUnsecureCommands);
@@ -542,11 +582,11 @@ namespace GitHub.Runner.Worker
command.Properties.TryGetValue(IssueCommandProperties.Line, out string line);
command.Properties.TryGetValue(IssueCommandProperties.Column, out string column);
if (!ActionCommandManager.EnhancedAnnotationsEnabled(context))
if (!ActionCommandManager.EnhancedAnnotationsEnabled(context))
{
context.Debug("Enhanced Annotations not enabled on the server. The 'title', 'end_line', and 'end_column' fields are unsupported.");
}
Issue issue = new Issue()
{
Category = "General",
@@ -598,7 +638,7 @@ namespace GitHub.Runner.Worker
context.AddIssue(issue);
}
public static void ValidateLinesAndColumns(ActionCommand command, IExecutionContext context)
public static void ValidateLinesAndColumns(ActionCommand command, IExecutionContext context)
{
command.Properties.TryGetValue(IssueCommandProperties.Line, out string line);
command.Properties.TryGetValue(IssueCommandProperties.EndLine, out string endLine);
@@ -627,28 +667,28 @@ namespace GitHub.Runner.Worker
column = endColumn;
}
if (!hasStartLine && hasColumn)
if (!hasStartLine && hasColumn)
{
context.Debug($"Invalid {command.Command} command value. '{IssueCommandProperties.Column}' and '{IssueCommandProperties.EndColumn}' can only be set if '{IssueCommandProperties.Line}' value is provided.");
command.Properties.Remove(IssueCommandProperties.Column);
command.Properties.Remove(IssueCommandProperties.EndColumn);
}
if (hasEndLine && line != endLine && hasColumn)
if (hasEndLine && line != endLine && hasColumn)
{
context.Debug($"Invalid {command.Command} command value. '{IssueCommandProperties.Column}' and '{IssueCommandProperties.EndColumn}' cannot be set if '{IssueCommandProperties.Line}' and '{IssueCommandProperties.EndLine}' are different values.");
command.Properties.Remove(IssueCommandProperties.Column);
command.Properties.Remove(IssueCommandProperties.EndColumn);
}
if (hasStartLine && hasEndLine && endLineNumber < lineNumber)
if (hasStartLine && hasEndLine && endLineNumber < lineNumber)
{
context.Debug($"Invalid {command.Command} command value. '{IssueCommandProperties.EndLine}' cannot be less than '{IssueCommandProperties.Line}'.");
command.Properties.Remove(IssueCommandProperties.Line);
command.Properties.Remove(IssueCommandProperties.EndLine);
}
if (hasStartColumn && hasEndColumn && endColumnNumber < columnNumber)
if (hasStartColumn && hasEndColumn && endColumnNumber < columnNumber)
{
context.Debug($"Invalid {command.Command} command value. '{IssueCommandProperties.EndColumn}' cannot be less than '{IssueCommandProperties.Column}'.");
command.Properties.Remove(IssueCommandProperties.Column);

View File

@@ -267,6 +267,19 @@ namespace GitHub.Runner.Worker
_cachedEmbeddedPostSteps[parentStepId].Push(clonedAction);
}
}
else if (depth > 0)
{
// if we're in a composite action and haven't loaded the local action yet
// we assume it has a post step
if (!_cachedEmbeddedPostSteps.ContainsKey(parentStepId))
{
// If we haven't done so already, add the parent to the post steps
_cachedEmbeddedPostSteps[parentStepId] = new Stack<Pipelines.ActionStep>();
}
// Clone action so we can modify the condition without affecting the original
var clonedAction = action.Clone() as Pipelines.ActionStep;
_cachedEmbeddedPostSteps[parentStepId].Push(clonedAction);
}
}
}
@@ -430,7 +443,7 @@ namespace GitHub.Runner.Worker
{
for (var i = 0; i < compositeAction.Steps.Count; i++)
{
// Store Id's for later load actions
// Load stored Ids for later load actions
compositeAction.Steps[i].Id = _cachedEmbeddedStepIds[action.Id][i];
if (string.IsNullOrEmpty(executionContext.Global.Variables.Get("DistributedTask.EnableCompositeActions")) && compositeAction.Steps[i].Reference.Type != Pipelines.ActionSourceType.Script)
{
@@ -438,6 +451,16 @@ namespace GitHub.Runner.Worker
}
}
}
else
{
_cachedEmbeddedStepIds[action.Id] = new List<Guid>();
foreach (var compositeStep in compositeAction.Steps)
{
var guid = Guid.NewGuid();
compositeStep.Id = guid;
_cachedEmbeddedStepIds[action.Id].Add(guid);
}
}
}
else
{
@@ -610,6 +633,7 @@ namespace GitHub.Runner.Worker
{
NameWithOwner = repositoryReference.Name,
Ref = repositoryReference.Ref,
Path = repositoryReference.Path,
};
})
.ToList();
@@ -632,7 +656,12 @@ namespace GitHub.Runner.Worker
}
catch (Exception ex) when (!executionContext.CancellationToken.IsCancellationRequested) // Do not retry if the run is canceled.
{
if (attempt < 3)
// UnresolvableActionDownloadInfoException is a 422 client error, don't retry
// Some possible cases are:
// * Repo is rate limited
// * Repo or tag doesn't exist, or isn't public
// * Policy validation failed
if (attempt < 3 && !(ex is WebApi.UnresolvableActionDownloadInfoException))
{
executionContext.Output($"Failed to resolve action download info. Error: {ex.Message}");
executionContext.Debug(ex.ToString());
@@ -648,6 +677,7 @@ namespace GitHub.Runner.Worker
// Some possible cases are:
// * Repo is rate limited
// * Repo or tag doesn't exist, or isn't public
// * Policy validation failed
if (ex is WebApi.UnresolvableActionDownloadInfoException)
{
throw;
@@ -1027,7 +1057,6 @@ namespace GitHub.Runner.Worker
}
}
// TODO: remove once we remove the DistributedTask.EnableCompositeActions FF
foreach (var step in compositeAction.Steps)
{
if (string.IsNullOrEmpty(executionContext.Global.Variables.Get("DistributedTask.EnableCompositeActions")) && step.Reference.Type != Pipelines.ActionSourceType.Script)
@@ -1170,6 +1199,8 @@ namespace GitHub.Runner.Worker
public string Pre { get; set; }
public string Post { get; set; }
public string NodeVersion { get; set; }
}
public sealed class PluginActionExecutionData : ActionExecutionData

View File

@@ -451,7 +451,8 @@ namespace GitHub.Runner.Worker
};
}
}
else if (string.Equals(usingToken.Value, "node12", StringComparison.OrdinalIgnoreCase))
else if (string.Equals(usingToken.Value, "node12", StringComparison.OrdinalIgnoreCase)||
string.Equals(usingToken.Value, "node16", StringComparison.OrdinalIgnoreCase))
{
if (string.IsNullOrEmpty(mainToken?.Value))
{
@@ -461,6 +462,7 @@ namespace GitHub.Runner.Worker
{
return new NodeJSActionExecutionData()
{
NodeVersion = usingToken.Value,
Script = mainToken.Value,
Pre = preToken?.Value,
InitCondition = preIfToken?.Value ?? "always()",
@@ -490,7 +492,7 @@ namespace GitHub.Runner.Worker
}
else
{
throw new ArgumentOutOfRangeException($"'using: {usingToken.Value}' is not supported, use 'docker' or 'node12' instead.");
throw new ArgumentOutOfRangeException($"'using: {usingToken.Value}' is not supported, use 'docker', 'node12' or 'node16' instead.");
}
}
else if (pluginToken != null)

View File

@@ -494,7 +494,8 @@ namespace GitHub.Runner.Worker
private void UpdateRegistryAuthForGitHubToken(IExecutionContext executionContext, ContainerInfo container)
{
var registryIsTokenCompatible = container.RegistryServer.Equals("ghcr.io", StringComparison.OrdinalIgnoreCase) || container.RegistryServer.Equals("containers.pkg.github.com", StringComparison.OrdinalIgnoreCase);
if (!registryIsTokenCompatible)
var isFallbackTokenFromHostedGithub = HostContext.GetService<IConfigurationStore>().GetSettings().IsHostedServer;
if (!registryIsTokenCompatible || !isFallbackTokenFromHostedGithub)
{
return;
}

View File

@@ -40,6 +40,7 @@ namespace GitHub.Runner.Worker
string ScopeName { get; }
string SiblingScopeName { get; }
string ContextName { get; }
ActionRunStage Stage { get; }
Task ForceCompleted { get; }
TaskResult? Result { get; set; }
TaskResult? Outcome { get; set; }
@@ -52,6 +53,7 @@ namespace GitHub.Runner.Worker
Dictionary<string, VariableValue> JobOutputs { get; }
ActionsEnvironmentReference ActionsEnvironment { get; }
List<ActionsStepTelemetry> ActionsStepsTelemetry { get; }
List<JobTelemetry> JobTelemetry { get; }
DictionaryContextData ExpressionValues { get; }
IList<IFunctionInfo> ExpressionFunctions { get; }
JobContext JobContext { get; }
@@ -61,7 +63,7 @@ namespace GitHub.Runner.Worker
// Only job level ExecutionContext has PostJobSteps
Stack<IStep> PostJobSteps { get; }
HashSet<Guid> EmbeddedStepsWithPostRegistered{ get; }
Dictionary<Guid, string> EmbeddedStepsWithPostRegistered { get; }
// Keep track of embedded steps states
Dictionary<Guid, Dictionary<string, string>> EmbeddedIntraActionState { get; }
@@ -75,8 +77,8 @@ namespace GitHub.Runner.Worker
// Initialize
void InitializeJob(Pipelines.AgentJobRequestMessage message, CancellationToken token);
void CancelToken();
IExecutionContext CreateChild(Guid recordId, string displayName, string refName, string scopeName, string contextName, Dictionary<string, string> intraActionState = null, int? recordOrder = null, IPagingLogger logger = null, bool isEmbedded = false, CancellationTokenSource cancellationTokenSource = null, Guid embeddedId = default(Guid), string siblingScopeName = null);
IExecutionContext CreateEmbeddedChild(string scopeName, string contextName, Guid embeddedId, Dictionary<string, string> intraActionState = null, string siblingScopeName = null);
IExecutionContext CreateChild(Guid recordId, string displayName, string refName, string scopeName, string contextName, ActionRunStage stage, Dictionary<string, string> intraActionState = null, int? recordOrder = null, IPagingLogger logger = null, bool isEmbedded = false, CancellationTokenSource cancellationTokenSource = null, Guid embeddedId = default(Guid), string siblingScopeName = null);
IExecutionContext CreateEmbeddedChild(string scopeName, string contextName, Guid embeddedId, ActionRunStage stage, Dictionary<string, string> intraActionState = null, string siblingScopeName = null);
// logging
long Write(string tag, string message);
@@ -143,6 +145,7 @@ namespace GitHub.Runner.Worker
public string ScopeName { get; private set; }
public string SiblingScopeName { get; private set; }
public string ContextName { get; private set; }
public ActionRunStage Stage { get; private set; }
public Task ForceCompleted => _forceCompleted.Task;
public CancellationToken CancellationToken => _cancellationTokenSource.Token;
public Dictionary<string, string> IntraActionState { get; private set; }
@@ -150,6 +153,7 @@ namespace GitHub.Runner.Worker
public ActionsEnvironmentReference ActionsEnvironment { get; private set; }
public List<ActionsStepTelemetry> ActionsStepsTelemetry { get; private set; }
public List<JobTelemetry> JobTelemetry { get; private set; }
public DictionaryContextData ExpressionValues { get; } = new DictionaryContextData();
public IList<IFunctionInfo> ExpressionFunctions { get; } = new List<IFunctionInfo>();
@@ -166,7 +170,7 @@ namespace GitHub.Runner.Worker
public HashSet<Guid> StepsWithPostRegistered { get; private set; }
// Only job level ExecutionContext has EmbeddedStepsWithPostRegistered
public HashSet<Guid> EmbeddedStepsWithPostRegistered { get; private set; }
public Dictionary<Guid, string> EmbeddedStepsWithPostRegistered { get; private set; }
public Dictionary<Guid, Dictionary<string, string>> EmbeddedIntraActionState { get; private set; }
@@ -263,12 +267,19 @@ namespace GitHub.Runner.Worker
string siblingScopeName = null;
if (this.IsEmbedded)
{
if (step is IActionRunner actionRunner && !Root.EmbeddedStepsWithPostRegistered.Add(actionRunner.Action.Id))
if (step is IActionRunner actionRunner)
{
Trace.Info($"'post' of '{actionRunner.DisplayName}' already push to child post step stack.");
if (Root.EmbeddedStepsWithPostRegistered.ContainsKey(actionRunner.Action.Id))
{
Trace.Info($"'post' of '{actionRunner.DisplayName}' already push to child post step stack.");
}
else
{
Root.EmbeddedStepsWithPostRegistered[actionRunner.Action.Id] = actionRunner.Condition;
}
return;
}
return;
}
}
else if (step is IActionRunner actionRunner && !Root.StepsWithPostRegistered.Add(actionRunner.Action.Id))
{
Trace.Info($"'post' of '{actionRunner.DisplayName}' already push to post step stack.");
@@ -283,7 +294,7 @@ namespace GitHub.Runner.Worker
Root.PostJobSteps.Push(step);
}
public IExecutionContext CreateChild(Guid recordId, string displayName, string refName, string scopeName, string contextName, Dictionary<string, string> intraActionState = null, int? recordOrder = null, IPagingLogger logger = null, bool isEmbedded = false, CancellationTokenSource cancellationTokenSource = null, Guid embeddedId = default(Guid), string siblingScopeName = null)
public IExecutionContext CreateChild(Guid recordId, string displayName, string refName, string scopeName, string contextName, ActionRunStage stage, Dictionary<string, string> intraActionState = null, int? recordOrder = null, IPagingLogger logger = null, bool isEmbedded = false, CancellationTokenSource cancellationTokenSource = null, Guid embeddedId = default(Guid), string siblingScopeName = null)
{
Trace.Entering();
@@ -292,8 +303,10 @@ namespace GitHub.Runner.Worker
child.Global = Global;
child.ScopeName = scopeName;
child.ContextName = contextName;
child.Stage = stage;
child.EmbeddedId = embeddedId;
child.SiblingScopeName = siblingScopeName;
child.JobTelemetry = JobTelemetry;
if (intraActionState == null)
{
child.IntraActionState = new Dictionary<string, string>(StringComparer.OrdinalIgnoreCase);
@@ -341,9 +354,9 @@ namespace GitHub.Runner.Worker
/// An embedded execution context shares the same record ID, record name, logger,
/// and a linked cancellation token.
/// </summary>
public IExecutionContext CreateEmbeddedChild(string scopeName, string contextName, Guid embeddedId, Dictionary<string, string> intraActionState = null, string siblingScopeName = null)
public IExecutionContext CreateEmbeddedChild(string scopeName, string contextName, Guid embeddedId, ActionRunStage stage, Dictionary<string, string> intraActionState = null, string siblingScopeName = null)
{
return Root.CreateChild(_record.Id, _record.Name, _record.Id.ToString("N"), scopeName, contextName, logger: _logger, isEmbedded: true, cancellationTokenSource: CancellationTokenSource.CreateLinkedTokenSource(_cancellationTokenSource.Token), intraActionState: intraActionState, embeddedId: embeddedId, siblingScopeName: siblingScopeName);
return Root.CreateChild(_record.Id, _record.Name, _record.Id.ToString("N"), scopeName, contextName, stage, logger: _logger, isEmbedded: true, cancellationTokenSource: CancellationTokenSource.CreateLinkedTokenSource(_cancellationTokenSource.Token), intraActionState: intraActionState, embeddedId: embeddedId, siblingScopeName: siblingScopeName);
}
public void Start(string currentOperation = null)
@@ -540,8 +553,8 @@ namespace GitHub.Runner.Worker
}
_record.WarningCount++;
}
else if (issue.Type == IssueType.Notice)
}
else if (issue.Type == IssueType.Notice)
{
// tracking line number for each issue in log file
@@ -650,6 +663,8 @@ namespace GitHub.Runner.Worker
// ActionsStepTelemetry
ActionsStepsTelemetry = new List<ActionsStepTelemetry>();
JobTelemetry = new List<JobTelemetry>();
// Service container info
Global.ServiceContainers = new List<ContainerInfo>();
@@ -711,10 +726,10 @@ namespace GitHub.Runner.Worker
StepsWithPostRegistered = new HashSet<Guid>();
// EmbeddedStepsWithPostRegistered for job ExecutionContext
EmbeddedStepsWithPostRegistered = new HashSet<Guid>();
EmbeddedStepsWithPostRegistered = new Dictionary<Guid, string>();
// EmbeddedIntraActionState for job ExecutionContext
EmbeddedIntraActionState = new Dictionary<Guid, Dictionary<string,string>>();
EmbeddedIntraActionState = new Dictionary<Guid, Dictionary<string, string>>();
// Job timeline record.
InitializeTimelineRecord(
@@ -932,7 +947,7 @@ namespace GitHub.Runner.Worker
}
var newGuid = Guid.NewGuid();
return CreateChild(newGuid, displayName, newGuid.ToString("N"), null, null, intraActionState, _childTimelineRecordOrder - Root.PostJobSteps.Count, siblingScopeName: siblingScopeName);
return CreateChild(newGuid, displayName, newGuid.ToString("N"), null, null, ActionRunStage.Post, intraActionState, _childTimelineRecordOrder - Root.PostJobSteps.Count, siblingScopeName: siblingScopeName);
}
}
@@ -965,7 +980,7 @@ namespace GitHub.Runner.Worker
// Do not add a format string overload. See comment on ExecutionContext.Write().
public static void InfrastructureError(this IExecutionContext context, string message)
{
context.AddIssue(new Issue() { Type = IssueType.Error, Message = message, IsInfrastructureIssue = true});
context.AddIssue(new Issue() { Type = IssueType.Error, Message = message, IsInfrastructureIssue = true });
}
// Do not add a format string overload. See comment on ExecutionContext.Write().
@@ -980,18 +995,6 @@ namespace GitHub.Runner.Worker
context.Write(null, message);
}
public static void WriteDetails(this IExecutionContext context, string message)
{
if (context.IsEmbedded)
{
context.Debug(message);
}
else
{
context.Output(message);
}
}
// Do not add a format string overload. See comment on ExecutionContext.Write().
public static void Command(this IExecutionContext context, string message)
{

View File

@@ -24,8 +24,19 @@ namespace GitHub.Runner.Worker.Expressions
ArgUtil.NotNull(templateContext, nameof(templateContext));
var executionContext = templateContext.State[nameof(IExecutionContext)] as IExecutionContext;
ArgUtil.NotNull(executionContext, nameof(executionContext));
ActionResult jobStatus = executionContext.JobContext.Status ?? ActionResult.Success;
return jobStatus == ActionResult.Failure;
// Decide based on 'action_status' for composite MAIN steps and 'job.status' for pre, post and job-level steps
var isCompositeMainStep = executionContext.IsEmbedded && executionContext.Stage == ActionRunStage.Main;
if (isCompositeMainStep)
{
ActionResult actionStatus = EnumUtil.TryParse<ActionResult>(executionContext.GetGitHubContext("action_status")) ?? ActionResult.Success;
return actionStatus == ActionResult.Failure;
}
else
{
ActionResult jobStatus = executionContext.JobContext.Status ?? ActionResult.Success;
return jobStatus == ActionResult.Failure;
}
}
}
}

View File

@@ -24,8 +24,19 @@ namespace GitHub.Runner.Worker.Expressions
ArgUtil.NotNull(templateContext, nameof(templateContext));
var executionContext = templateContext.State[nameof(IExecutionContext)] as IExecutionContext;
ArgUtil.NotNull(executionContext, nameof(executionContext));
ActionResult jobStatus = executionContext.JobContext.Status ?? ActionResult.Success;
return jobStatus == ActionResult.Success;
// Decide based on 'action_status' for composite MAIN steps and 'job.status' for pre, post and job-level steps
var isCompositeMainStep = executionContext.IsEmbedded && executionContext.Stage == ActionRunStage.Main;
if (isCompositeMainStep)
{
ActionResult actionStatus = EnumUtil.TryParse<ActionResult>(executionContext.GetGitHubContext("action_status")) ?? ActionResult.Success;
return actionStatus == ActionResult.Success;
}
else
{
ActionResult jobStatus = executionContext.JobContext.Status ?? ActionResult.Success;
return jobStatus == ActionResult.Success;
}
}
}
}

View File

@@ -23,9 +23,13 @@ namespace GitHub.Runner.Worker
"job",
"path",
"ref",
"ref_name",
"ref_protected",
"ref_type",
"repository",
"repository_owner",
"retention_days",
"run_attempt",
"run_id",
"run_number",
"server_url",
@@ -38,9 +42,16 @@ namespace GitHub.Runner.Worker
{
foreach (var data in this)
{
if (_contextEnvAllowlist.Contains(data.Key) && data.Value is StringContextData value)
if (_contextEnvAllowlist.Contains(data.Key))
{
yield return new KeyValuePair<string, string>($"GITHUB_{data.Key.ToUpperInvariant()}", value);
if (data.Value is StringContextData value)
{
yield return new KeyValuePair<string, string>($"GITHUB_{data.Key.ToUpperInvariant()}", value);
}
else if (data.Value is BooleanContextData booleanValue)
{
yield return new KeyValuePair<string, string>($"GITHUB_{data.Key.ToUpperInvariant()}", booleanValue.ToString());
}
}
}
}

View File

@@ -50,8 +50,9 @@ namespace GitHub.Runner.Worker.Handlers
// Only register post steps for steps that actually ran
foreach (var step in Data.PostSteps.ToList())
{
if (ExecutionContext.Root.EmbeddedStepsWithPostRegistered.Contains(step.Id))
if (ExecutionContext.Root.EmbeddedStepsWithPostRegistered.ContainsKey(step.Id))
{
step.Condition = ExecutionContext.Root.EmbeddedStepsWithPostRegistered[step.Id];
steps.Add(step);
}
else
@@ -124,7 +125,7 @@ namespace GitHub.Runner.Worker.Handlers
{
ArgUtil.NotNull(step, step.DisplayName);
var stepId = $"__{Guid.NewGuid()}";
step.ExecutionContext = ExecutionContext.CreateEmbeddedChild(childScopeName, stepId, Guid.NewGuid());
step.ExecutionContext = ExecutionContext.CreateEmbeddedChild(childScopeName, stepId, Guid.NewGuid(), stage);
embeddedSteps.Add(step);
}
}
@@ -143,7 +144,7 @@ namespace GitHub.Runner.Worker.Handlers
step.Stage = stage;
step.Condition = stepData.Condition;
ExecutionContext.Root.EmbeddedIntraActionState.TryGetValue(step.Action.Id, out var intraActionState);
step.ExecutionContext = ExecutionContext.CreateEmbeddedChild(childScopeName, stepData.ContextName, step.Action.Id, intraActionState: intraActionState, siblingScopeName: siblingScopeName);
step.ExecutionContext = ExecutionContext.CreateEmbeddedChild(childScopeName, stepData.ContextName, step.Action.Id, stage, intraActionState: intraActionState, siblingScopeName: siblingScopeName);
step.ExecutionContext.ExpressionValues["inputs"] = inputsData;
if (!String.IsNullOrEmpty(ExecutionContext.SiblingScopeName))
{
@@ -241,6 +242,10 @@ namespace GitHub.Runner.Worker.Handlers
step.ExecutionContext.ExpressionFunctions.Add(new FunctionInfo<FailureFunction>(PipelineTemplateConstants.Failure, 0, 0));
step.ExecutionContext.ExpressionFunctions.Add(new FunctionInfo<SuccessFunction>(PipelineTemplateConstants.Success, 0, 0));
// Set action_status to the success of the current composite action
var actionResult = ExecutionContext.Result?.ToActionResult() ?? ActionResult.Success;
step.ExecutionContext.SetGitHubContext("action_status", actionResult.ToString());
// Initialize env context
Trace.Info("Initialize Env context for embedded step");
#if OS_WINDOWS
@@ -304,7 +309,7 @@ namespace GitHub.Runner.Worker.Handlers
// Mark job as cancelled
ExecutionContext.Root.Result = TaskResult.Canceled;
ExecutionContext.Root.JobContext.Status = ExecutionContext.Root.Result?.ToActionResult();
step.ExecutionContext.Debug($"Re-evaluate condition on job cancellation for step: '{step.DisplayName}'.");
var conditionReTestTraceWriter = new ConditionTraceWriter(Trace, null); // host tracing only
var conditionReTestResult = false;
@@ -388,6 +393,7 @@ namespace GitHub.Runner.Worker.Handlers
{
await RunStepAsync(step);
}
}
finally
{
@@ -403,12 +409,6 @@ namespace GitHub.Runner.Worker.Handlers
{
Trace.Info($"Update job result with current composite step result '{step.ExecutionContext.Result}'.");
ExecutionContext.Result = TaskResultUtil.MergeTaskResults(ExecutionContext.Result, step.ExecutionContext.Result.Value);
// We should run cleanup even if one of the cleanup step fails
if (stage != ActionRunStage.Post)
{
break;
}
}
}
}

View File

@@ -50,8 +50,8 @@ namespace GitHub.Runner.Worker.Handlers
var dockerFile = Path.Combine(ActionDirectory, Data.Image);
ArgUtil.File(dockerFile, nameof(Data.Image));
ExecutionContext.WriteDetails(ExecutionContext.IsEmbedded ? "Building docker image" : $"##[group]Building docker image");
ExecutionContext.WriteDetails($"Dockerfile for action: '{dockerFile}'.");
ExecutionContext.Output($"##[group]Building docker image");
ExecutionContext.Output($"Dockerfile for action: '{dockerFile}'.");
var imageName = $"{dockerManager.DockerInstanceLabel}:{ExecutionContext.Id.ToString("N")}";
var buildExitCode = await dockerManager.DockerBuild(
ExecutionContext,
@@ -59,7 +59,7 @@ namespace GitHub.Runner.Worker.Handlers
dockerFile,
Directory.GetParent(dockerFile).FullName,
imageName);
ExecutionContext.WriteDetails(ExecutionContext.IsEmbedded ? "" : "##[endgroup]");
ExecutionContext.Output("##[endgroup]");
if (buildExitCode != 0)
{
@@ -217,6 +217,7 @@ namespace GitHub.Runner.Worker.Handlers
if (systemConnection.Data.TryGetValue("GenerateIdTokenUrl", out var generateIdTokenUrl) && !string.IsNullOrEmpty(generateIdTokenUrl))
{
Environment["ACTIONS_ID_TOKEN_REQUEST_URL"] = generateIdTokenUrl;
Environment["ACTIONS_ID_TOKEN_REQUEST_TOKEN"] = systemConnection.Authorization.Parameters[EndpointAuthorizationParameters.AccessToken];
}
foreach (var variable in this.Environment)

View File

@@ -82,7 +82,7 @@ namespace GitHub.Runner.Worker.Handlers
if (stage == ActionRunStage.Post)
{
ExecutionContext.WriteDetails($"Post job cleanup.");
ExecutionContext.Output($"Post job cleanup.");
return;
}
@@ -118,30 +118,30 @@ namespace GitHub.Runner.Worker.Handlers
groupName = "Action details";
}
ExecutionContext.WriteDetails(ExecutionContext.IsEmbedded ? groupName : $"##[group]{groupName}");
ExecutionContext.Output($"##[group]{groupName}");
if (this.Inputs?.Count > 0)
{
ExecutionContext.WriteDetails("with:");
ExecutionContext.Output("with:");
foreach (var input in this.Inputs)
{
if (!string.IsNullOrEmpty(input.Value))
{
ExecutionContext.WriteDetails($" {input.Key}: {input.Value}");
ExecutionContext.Output($" {input.Key}: {input.Value}");
}
}
}
if (this.Environment?.Count > 0)
{
ExecutionContext.WriteDetails("env:");
ExecutionContext.Output("env:");
foreach (var env in this.Environment)
{
ExecutionContext.WriteDetails($" {env.Key}: {env.Value}");
ExecutionContext.Output($" {env.Key}: {env.Value}");
}
}
ExecutionContext.WriteDetails(ExecutionContext.IsEmbedded ? "" : "##[endgroup]");
ExecutionContext.Output("##[endgroup]");
}
public override void Initialize(IHostContext hostContext)

View File

@@ -56,6 +56,7 @@ namespace GitHub.Runner.Worker.Handlers
if (systemConnection.Data.TryGetValue("GenerateIdTokenUrl", out var generateIdTokenUrl) && !string.IsNullOrEmpty(generateIdTokenUrl))
{
Environment["ACTIONS_ID_TOKEN_REQUEST_URL"] = generateIdTokenUrl;
Environment["ACTIONS_ID_TOKEN_REQUEST_TOKEN"] = systemConnection.Authorization.Parameters[EndpointAuthorizationParameters.AccessToken];
}
// Resolve the target script.
@@ -82,7 +83,7 @@ namespace GitHub.Runner.Worker.Handlers
HasPreStep = Data.HasPre,
HasPostStep = Data.HasPost,
IsEmbedded = ExecutionContext.IsEmbedded,
Type = "node12"
Type = Data.NodeVersion
};
ExecutionContext.Root.ActionsStepsTelemetry.Add(telemetry);
}
@@ -98,7 +99,7 @@ namespace GitHub.Runner.Worker.Handlers
workingDirectory = HostContext.GetDirectory(WellKnownDirectory.Work);
}
var nodeRuntimeVersion = await StepHost.DetermineNodeRuntimeVersion(ExecutionContext);
var nodeRuntimeVersion = await StepHost.DetermineNodeRuntimeVersion(ExecutionContext, Data.NodeVersion);
string file = Path.Combine(HostContext.GetDirectory(WellKnownDirectory.Externals), nodeRuntimeVersion, "bin", $"node{IOUtil.ExeExtension}");
// Format the arguments passed to node.

View File

@@ -40,7 +40,7 @@ namespace GitHub.Runner.Worker.Handlers
firstLine = firstLine.Substring(0, firstNewLine);
}
ExecutionContext.WriteDetails(ExecutionContext.IsEmbedded ? $"Run {firstLine}" : $"##[group]Run {firstLine}");
ExecutionContext.Output($"##[group]Run {firstLine}");
}
else
{
@@ -51,7 +51,7 @@ namespace GitHub.Runner.Worker.Handlers
foreach (var line in multiLines)
{
// Bright Cyan color
ExecutionContext.WriteDetails($"\x1b[36;1m{line}\x1b[0m");
ExecutionContext.Output($"\x1b[36;1m{line}\x1b[0m");
}
string argFormat;
@@ -110,23 +110,23 @@ namespace GitHub.Runner.Worker.Handlers
if (!string.IsNullOrEmpty(shellCommandPath))
{
ExecutionContext.WriteDetails($"shell: {shellCommandPath} {argFormat}");
ExecutionContext.Output($"shell: {shellCommandPath} {argFormat}");
}
else
{
ExecutionContext.WriteDetails($"shell: {shellCommand} {argFormat}");
ExecutionContext.Output($"shell: {shellCommand} {argFormat}");
}
if (this.Environment?.Count > 0)
{
ExecutionContext.WriteDetails("env:");
ExecutionContext.Output("env:");
foreach (var env in this.Environment)
{
ExecutionContext.WriteDetails($" {env.Key}: {env.Value}");
ExecutionContext.Output($" {env.Key}: {env.Value}");
}
}
ExecutionContext.WriteDetails(ExecutionContext.IsEmbedded ? "" : "##[endgroup]");
ExecutionContext.Output("##[endgroup]");
}
public async Task RunAsync(ActionRunStage stage)
@@ -147,7 +147,8 @@ namespace GitHub.Runner.Worker.Handlers
// Add Telemetry to JobContext to send with JobCompleteMessage
if (stage == ActionRunStage.Main)
{
var telemetry = new ActionsStepTelemetry {
var telemetry = new ActionsStepTelemetry
{
IsEmbedded = ExecutionContext.IsEmbedded,
Type = "run",
};
@@ -276,6 +277,13 @@ namespace GitHub.Runner.Worker.Handlers
fileName = node12;
}
#endif
var systemConnection = ExecutionContext.Global.Endpoints.Single(x => string.Equals(x.Name, WellKnownServiceEndpointNames.SystemVssConnection, StringComparison.OrdinalIgnoreCase));
if (systemConnection.Data.TryGetValue("GenerateIdTokenUrl", out var generateIdTokenUrl) && !string.IsNullOrEmpty(generateIdTokenUrl))
{
Environment["ACTIONS_ID_TOKEN_REQUEST_URL"] = generateIdTokenUrl;
Environment["ACTIONS_ID_TOKEN_REQUEST_TOKEN"] = systemConnection.Authorization.Parameters[EndpointAuthorizationParameters.AccessToken];
}
ExecutionContext.Debug($"{fileName} {arguments}");
using (var stdoutManager = new OutputManager(ExecutionContext, ActionCommandManager))

View File

@@ -23,7 +23,7 @@ namespace GitHub.Runner.Worker.Handlers
string ResolvePathForStepHost(string path);
Task<string> DetermineNodeRuntimeVersion(IExecutionContext executionContext);
Task<string> DetermineNodeRuntimeVersion(IExecutionContext executionContext, string preferredVersion);
Task<int> ExecuteAsync(string workingDirectory,
string fileName,
@@ -58,9 +58,9 @@ namespace GitHub.Runner.Worker.Handlers
return path;
}
public Task<string> DetermineNodeRuntimeVersion(IExecutionContext executionContext)
public Task<string> DetermineNodeRuntimeVersion(IExecutionContext executionContext, string preferredVersion)
{
return Task.FromResult<string>("node12");
return Task.FromResult<string>(preferredVersion);
}
public async Task<int> ExecuteAsync(string workingDirectory,
@@ -123,7 +123,7 @@ namespace GitHub.Runner.Worker.Handlers
}
}
public async Task<string> DetermineNodeRuntimeVersion(IExecutionContext executionContext)
public async Task<string> DetermineNodeRuntimeVersion(IExecutionContext executionContext, string preferredVersion)
{
// Best effort to determine a compatible node runtime
// There may be more variation in which libraries are linked than just musl/glibc,
@@ -148,14 +148,14 @@ namespace GitHub.Runner.Worker.Handlers
var msg = $"JavaScript Actions in Alpine containers are only supported on x64 Linux runners. Detected {os} {arch}";
throw new NotSupportedException(msg);
}
nodeExternal = "node12_alpine";
nodeExternal = $"{preferredVersion}_alpine";
executionContext.Debug($"Container distribution is alpine. Running JavaScript Action with external tool: {nodeExternal}");
return nodeExternal;
}
}
}
// Optimistically use the default
nodeExternal = "node12";
nodeExternal = preferredVersion;
executionContext.Debug($"Running JavaScript Action with default external tool: {nodeExternal}");
return nodeExternal;
}

View File

@@ -350,6 +350,7 @@ namespace GitHub.Runner.Worker
case "":
case "ERROR":
case "WARNING":
case "NOTICE":
break;
default:
throw new ArgumentException($"Matcher '{_owner}' contains unexpected default severity '{_severity}'");

View File

@@ -55,7 +55,7 @@ namespace GitHub.Runner.Worker
ArgUtil.NotNull(message, nameof(message));
// Create a new timeline record for 'Set up job'
IExecutionContext context = jobContext.CreateChild(Guid.NewGuid(), "Set up job", $"{nameof(JobExtension)}_Init", null, null);
IExecutionContext context = jobContext.CreateChild(Guid.NewGuid(), "Set up job", $"{nameof(JobExtension)}_Init", null, null, ActionRunStage.Pre);
List<IStep> preJobSteps = new List<IStep>();
List<IStep> jobSteps = new List<IStep>();
@@ -142,6 +142,12 @@ namespace GitHub.Runner.Worker
Trace.Error(ex);
}
var secretSource = context.GetGitHubContext("secret_source");
if (!string.IsNullOrEmpty(secretSource))
{
context.Output($"Secret source: {secretSource}");
}
var repoFullName = context.GetGitHubContext("repository");
ArgUtil.NotNull(repoFullName, nameof(repoFullName));
context.Debug($"Primary repository: {repoFullName}");
@@ -306,13 +312,13 @@ namespace GitHub.Runner.Worker
JobExtensionRunner extensionStep = step as JobExtensionRunner;
ArgUtil.NotNull(extensionStep, extensionStep.DisplayName);
Guid stepId = Guid.NewGuid();
extensionStep.ExecutionContext = jobContext.CreateChild(stepId, extensionStep.DisplayName, null, null, stepId.ToString("N"));
extensionStep.ExecutionContext = jobContext.CreateChild(stepId, extensionStep.DisplayName, null, null, stepId.ToString("N"), ActionRunStage.Pre);
}
else if (step is IActionRunner actionStep)
{
ArgUtil.NotNull(actionStep, step.DisplayName);
Guid stepId = Guid.NewGuid();
actionStep.ExecutionContext = jobContext.CreateChild(stepId, actionStep.DisplayName, stepId.ToString("N"), null, null, intraActionStates[actionStep.Action.Id]);
actionStep.ExecutionContext = jobContext.CreateChild(stepId, actionStep.DisplayName, stepId.ToString("N"), null, null, ActionRunStage.Pre, intraActionStates[actionStep.Action.Id]);
}
}
@@ -323,7 +329,7 @@ namespace GitHub.Runner.Worker
{
ArgUtil.NotNull(actionStep, step.DisplayName);
intraActionStates.TryGetValue(actionStep.Action.Id, out var intraActionState);
actionStep.ExecutionContext = jobContext.CreateChild(actionStep.Action.Id, actionStep.DisplayName, actionStep.Action.Name, null, actionStep.Action.ContextName, intraActionState);
actionStep.ExecutionContext = jobContext.CreateChild(actionStep.Action.Id, actionStep.DisplayName, actionStep.Action.Name, null, actionStep.Action.ContextName, ActionRunStage.Main, intraActionState);
}
}
@@ -394,7 +400,7 @@ namespace GitHub.Runner.Worker
ArgUtil.NotNull(jobContext, nameof(jobContext));
// create a new timeline record node for 'Finalize job'
IExecutionContext context = jobContext.CreateChild(Guid.NewGuid(), "Complete job", $"{nameof(JobExtension)}_Final", null, null);
IExecutionContext context = jobContext.CreateChild(Guid.NewGuid(), "Complete job", $"{nameof(JobExtension)}_Final", null, null, ActionRunStage.Post);
using (var register = jobContext.CancellationToken.Register(() => { context.CancelToken(); }))
{
try

View File

@@ -7,6 +7,7 @@ using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Text;
using System.Threading;
using System.Threading.Tasks;
using System.Net.Http;
@@ -105,6 +106,10 @@ namespace GitHub.Runner.Worker
}
jobContext.SetRunnerContext("os", VarUtil.OS);
jobContext.SetRunnerContext("arch", VarUtil.OSArchitecture);
var runnerSettings = HostContext.GetService<IConfigurationStore>().GetSettings();
jobContext.SetRunnerContext("name", runnerSettings.AgentName);
string toolsDirectory = HostContext.GetDirectory(WellKnownDirectory.Tools);
Directory.CreateDirectory(toolsDirectory);
@@ -145,6 +150,16 @@ namespace GitHub.Runner.Worker
Trace.Verbose($"Job steps: '{string.Join(", ", jobSteps.Select(x => x.DisplayName))}'");
HostContext.WritePerfCounter($"WorkerJobInitialized_{message.RequestId.ToString()}");
if (systemConnection.Data.TryGetValue("GenerateIdTokenUrl", out var generateIdTokenUrl) &&
!string.IsNullOrEmpty(generateIdTokenUrl))
{
// Server won't issue ID_TOKEN for non-inprogress job.
// If the job is trying to use OIDC feature, we want the job to be marked as in-progress before running any customer's steps as much as we can.
// Timeline record update background process runs every 500ms, so delay 1000ms is enough for most of the cases
Trace.Info($"Waiting for job to be marked as started.");
await Task.WhenAny(_jobServerQueue.JobRecordUpdated.Task, Task.Delay(1000));
}
// Run all job steps
Trace.Info("Run all job steps.");
var stepsRunner = HostContext.GetService<IStepsRunner>();
@@ -215,8 +230,15 @@ namespace GitHub.Runner.Worker
return result;
}
// Load any upgrade telemetry
LoadFromTelemetryFile(jobContext.JobTelemetry);
// Make sure we don't submit secrets as telemetry
MaskTelemetrySecrets(jobContext.JobTelemetry);
Trace.Info("Raising job completed event.");
var jobCompletedEvent = new JobCompletedEvent(message.RequestId, message.JobId, result, jobContext.JobOutputs, jobContext.ActionsEnvironment, jobContext.ActionsStepsTelemetry);
var jobCompletedEvent = new JobCompletedEvent(message.RequestId, message.JobId, result, jobContext.JobOutputs, jobContext.ActionsEnvironment, jobContext.ActionsStepsTelemetry, jobContext.JobTelemetry);
var completeJobRetryLimit = 5;
var exceptions = new List<Exception>();
@@ -260,6 +282,38 @@ namespace GitHub.Runner.Worker
throw new AggregateException(exceptions);
}
private void MaskTelemetrySecrets(List<JobTelemetry> jobTelemetry)
{
foreach (var telemetryItem in jobTelemetry)
{
telemetryItem.Message = HostContext.SecretMasker.MaskSecrets(telemetryItem.Message);
}
}
private void LoadFromTelemetryFile(List<JobTelemetry> jobTelemetry)
{
try
{
var telemetryFilePath = HostContext.GetConfigFile(WellKnownConfigFile.Telemetry);
if (File.Exists(telemetryFilePath))
{
var telemetryData = File.ReadAllText(telemetryFilePath, Encoding.UTF8);
var telemetry = new JobTelemetry
{
Message = $"Runner File Telemetry:\n{telemetryData}",
Type = JobTelemetryType.General
};
jobTelemetry.Add(telemetry);
IOUtil.DeleteFile(telemetryFilePath);
}
}
catch (Exception e)
{
Trace.Error("Error when trying to load telemetry from telemetry file");
Trace.Error(e);
}
}
private async Task ShutdownQueue(bool throwOnFailure)
{
if (_jobServerQueue != null)

View File

@@ -5,7 +5,6 @@
<OutputType>Exe</OutputType>
<RuntimeIdentifiers>win-x64;win-x86;linux-x64;linux-arm64;linux-arm;osx-x64</RuntimeIdentifiers>
<TargetLatestRuntimePatch>true</TargetLatestRuntimePatch>
<AssetTargetFallback>portable-net45+win8</AssetTargetFallback>
<NoWarn>NU1701;NU1603</NoWarn>
<Version>$(Version)</Version>
<TieredCompilationQuickJit>true</TieredCompilationQuickJit>

View File

@@ -46,7 +46,7 @@
"runs": {
"one-of": [
"container-runs",
"node12-runs",
"node-runs",
"plugin-runs",
"composite-runs"
]
@@ -80,7 +80,7 @@
"loose-value-type": "string"
}
},
"node12-runs": {
"node-runs": {
"mapping": {
"properties": {
"using": "non-empty-string",
@@ -112,10 +112,10 @@
"item-type": "composite-step"
}
},
"composite-step":{
"composite-step": {
"one-of": [
"run-step",
"uses-step"
"uses-step"
]
},
"run-step": {
@@ -123,6 +123,7 @@
"properties": {
"name": "string-steps-context",
"id": "non-empty-string",
"if": "step-if",
"run": {
"type": "string-steps-context",
"required": true
@@ -141,6 +142,7 @@
"properties": {
"name": "string-steps-context",
"id": "non-empty-string",
"if": "step-if",
"uses": {
"type": "non-empty-string",
"required": true
@@ -216,6 +218,24 @@
"loose-value-type": "string"
}
},
"step-if": {
"context": [
"github",
"inputs",
"strategy",
"matrix",
"steps",
"job",
"runner",
"env",
"always(0,0)",
"failure(0,0)",
"cancelled(0,0)",
"success(0,0)",
"hashFiles(1,255)"
],
"string": {}
},
"step-with": {
"context": [
"github",
@@ -234,4 +254,4 @@
}
}
}
}
}

View File

@@ -768,6 +768,7 @@ namespace GitHub.DistributedTask.WebApi
/// <param name="poolId"></param>
/// <param name="agentId"></param>
/// <param name="currentState"></param>
/// <param name="updateTrace"></param>
/// <param name="userState"></param>
/// <param name="cancellationToken">The cancellation token to cancel operation.</param>
[EditorBrowsable(EditorBrowsableState.Never)]
@@ -775,6 +776,7 @@ namespace GitHub.DistributedTask.WebApi
int poolId,
int agentId,
string currentState,
string updateTrace,
object userState = null,
CancellationToken cancellationToken = default)
{
@@ -784,6 +786,7 @@ namespace GitHub.DistributedTask.WebApi
List<KeyValuePair<string, string>> queryParams = new List<KeyValuePair<string, string>>();
queryParams.Add("currentState", currentState);
queryParams.Add("updateTrace", updateTrace);
return SendAsync<TaskAgent>(
httpMethod,
@@ -794,65 +797,5 @@ namespace GitHub.DistributedTask.WebApi
userState: userState,
cancellationToken: cancellationToken);
}
/// <summary>
/// [Preview API]
/// </summary>
/// <param name="poolId"></param>
/// <param name="agentId"></param>
/// <param name="userState"></param>
/// <param name="cancellationToken">The cancellation token to cancel operation.</param>
public Task<String> GetAgentAuthUrlAsync(
int poolId,
int agentId,
object userState = null,
CancellationToken cancellationToken = default)
{
HttpMethod httpMethod = new HttpMethod("GET");
Guid locationId = new Guid("a82a119c-1e46-44b6-8d75-c82a79cf975b");
object routeValues = new { poolId = poolId, agentId = agentId };
return SendAsync<String>(
httpMethod,
locationId,
routeValues: routeValues,
version: new ApiResourceVersion(6.0, 1),
userState: userState,
cancellationToken: cancellationToken);
}
/// <summary>
/// [Preview API]
/// </summary>
/// <param name="poolId"></param>
/// <param name="agentId"></param>
/// <param name="error"></param>
/// <param name="userState"></param>
/// <param name="cancellationToken">The cancellation token to cancel operation.</param>
[EditorBrowsable(EditorBrowsableState.Never)]
public virtual async Task ReportAgentAuthUrlMigrationErrorAsync(
int poolId,
int agentId,
string error,
object userState = null,
CancellationToken cancellationToken = default)
{
HttpMethod httpMethod = new HttpMethod("POST");
Guid locationId = new Guid("a82a119c-1e46-44b6-8d75-c82a79cf975b");
object routeValues = new { poolId = poolId, agentId = agentId };
HttpContent content = new ObjectContent<string>(error, new VssJsonMediaTypeFormatter(true));
using (HttpResponseMessage response = await SendAsync(
httpMethod,
locationId,
routeValues: routeValues,
version: new ApiResourceVersion(6.0, 1),
userState: userState,
cancellationToken: cancellationToken,
content: content).ConfigureAwait(false))
{
return;
}
}
}
}

View File

@@ -2,6 +2,7 @@
using System.ComponentModel;
using System.Security;
using System.Text;
using System.Text.RegularExpressions;
using Newtonsoft.Json;
namespace GitHub.DistributedTask.Logging
@@ -80,6 +81,65 @@ namespace GitHub.DistributedTask.Logging
return trimmed;
}
public static String PowerShellPreAmpersandEscape(String value)
{
// if the secret is passed to PS as a command and it causes an error, sections of it can be surrounded by color codes
// or printed individually.
// The secret secretpart1&secretpart2&secretpart3 would be split into 2 sections:
// 'secretpart1&secretpart2&' and 'secretpart3'. This method masks for the first section.
// The secret secretpart1&+secretpart2&secretpart3 would be split into 2 sections:
// 'secretpart1&+' and (no 's') 'ecretpart2&secretpart3'. This method masks for the first section.
var trimmed = string.Empty;
if (!string.IsNullOrEmpty(value) && value.Contains("&"))
{
var secretSection = string.Empty;
if (value.Contains("&+"))
{
secretSection = value.Substring(0, value.IndexOf("&+") + "&+".Length);
}
else
{
secretSection = value.Substring(0, value.LastIndexOf("&") + "&".Length);
}
// Don't mask short secrets
if (secretSection.Length >= 6)
{
trimmed = secretSection;
}
}
return trimmed;
}
public static String PowerShellPostAmpersandEscape(String value)
{
var trimmed = string.Empty;
if (!string.IsNullOrEmpty(value) && value.Contains("&"))
{
var secretSection = string.Empty;
if (value.Contains("&+"))
{
// +1 to skip the letter that got colored
secretSection = value.Substring(value.IndexOf("&+") + "&+".Length + 1);
}
else
{
secretSection = value.Substring(value.LastIndexOf("&") + "&".Length);
}
if (secretSection.Length >= 6)
{
trimmed = secretSection;
}
}
return trimmed;
}
private static string Base64StringEscapeShift(String value, int shift)
{
var bytes = Encoding.UTF8.GetBytes(value);

View File

@@ -638,6 +638,7 @@ namespace GitHub.DistributedTask.Pipelines.ObjectTemplating
new NamedValueInfo<NoOperationNamedValue>(PipelineTemplateConstants.Matrix),
new NamedValueInfo<NoOperationNamedValue>(PipelineTemplateConstants.Steps),
new NamedValueInfo<NoOperationNamedValue>(PipelineTemplateConstants.GitHub),
new NamedValueInfo<NoOperationNamedValue>(PipelineTemplateConstants.Inputs),
new NamedValueInfo<NoOperationNamedValue>(PipelineTemplateConstants.Job),
new NamedValueInfo<NoOperationNamedValue>(PipelineTemplateConstants.Runner),
new NamedValueInfo<NoOperationNamedValue>(PipelineTemplateConstants.Env),

View File

@@ -18,5 +18,12 @@ namespace GitHub.DistributedTask.WebApi
get;
set;
}
[DataMember]
public string Path
{
get;
set;
}
}
}

View File

@@ -153,6 +153,19 @@ namespace GitHub.DistributedTask.WebApi
{
this.ActionsEnvironment = actionsEnvironment;
this.ActionsStepsTelemetry = actionsStepsTelemetry;
}
public JobCompletedEvent(
Int64 requestId,
Guid jobId,
TaskResult result,
Dictionary<String, VariableValue> outputs,
ActionsEnvironmentReference actionsEnvironment,
List<ActionsStepTelemetry> actionsStepsTelemetry,
List<JobTelemetry> jobTelemetry)
: this(requestId, jobId, result, outputs, actionsEnvironment, actionsStepsTelemetry)
{
this.JobTelemetry = jobTelemetry;
}
[DataMember(EmitDefaultValue = false)]
@@ -189,6 +202,13 @@ namespace GitHub.DistributedTask.WebApi
get;
set;
}
[DataMember(EmitDefaultValue = false)]
public List<JobTelemetry> JobTelemetry
{
get;
set;
}
}
[DataContract]

View File

@@ -0,0 +1,17 @@
using System.Runtime.Serialization;
namespace GitHub.DistributedTask.WebApi
{
/// <summary>
/// Information about a job run on the runner
/// </summary>
[DataContract]
public class JobTelemetry
{
[DataMember(EmitDefaultValue = false)]
public string Message { get; set; }
[DataMember(EmitDefaultValue = false)]
public JobTelemetryType Type { get; set; }
}
}

View File

@@ -0,0 +1,13 @@
using System.Runtime.Serialization;
namespace GitHub.DistributedTask.WebApi
{
public enum JobTelemetryType
{
[EnumMember]
General = 0,
[EnumMember]
ActionCommand = 1,
}
}

View File

@@ -24,6 +24,7 @@ namespace GitHub.DistributedTask.WebApi
this.OSDescription = referenceToBeCloned.OSDescription;
this.ProvisioningState = referenceToBeCloned.ProvisioningState;
this.AccessPoint = referenceToBeCloned.AccessPoint;
this.Ephemeral = referenceToBeCloned.Ephemeral;
if (referenceToBeCloned.m_links != null)
{
@@ -81,6 +82,16 @@ namespace GitHub.DistributedTask.WebApi
set;
}
/// <summary>
/// Signifies that this Agent can only run one job and will be removed by the server after that one job finish.
/// </summary>
[DataMember]
public bool? Ephemeral
{
get;
set;
}
/// <summary>
/// Whether or not the agent is online.
/// </summary>

View File

@@ -5,7 +5,6 @@
<OutputType>Library</OutputType>
<RuntimeIdentifiers>win-x64;win-x86;linux-x64;linux-arm64;linux-arm;osx-x64</RuntimeIdentifiers>
<TargetLatestRuntimePatch>true</TargetLatestRuntimePatch>
<AssetTargetFallback>portable-net45+win8</AssetTargetFallback>
<NoWarn>NU1701;NU1603</NoWarn>
<Version>$(Version)</Version>
<DefineConstants>TRACE</DefineConstants>

View File

@@ -112,6 +112,36 @@ namespace GitHub.Runner.Common.Tests
}
}
[Theory]
[InlineData("secret&secret&secret", "secret&secret&\x0033[96msecret\x0033[0m", "***\x0033[96m***\x0033[0m")]
[InlineData("secret&secret+secret", "secret&\x0033[96msecret+secret\x0033[0m", "***\x0033[96m***\x0033[0m")]
[InlineData("secret+secret&secret", "secret+secret&\x0033[96msecret\x0033[0m", "***\x0033[96m***\x0033[0m")]
[InlineData("secret&secret&+secretsecret", "secret&secret&+\x0033[96ms\x0033[0mecretsecret", "***\x0033[96ms\x0033[0m***")]
[InlineData("secret&+secret&secret", "secret&+\x0033[96ms\x0033[0mecret&secret", "***\x0033[96ms\x0033[0m***")]
[InlineData("secret&+secret&+secret", "secret&+\x0033[96ms\x0033[0mecret&+secret", "***\x0033[96ms\x0033[0m***")]
[InlineData("secret&+secret&secret&+secret", "secret&+\x0033[96ms\x0033[0mecret&secret&+secret", "***\x0033[96ms\x0033[0m***")]
[Trait("Level", "L0")]
[Trait("Category", "Common")]
public void SecretSectionMasking(string secret, string rawOutput, string maskedOutput)
{
try
{
// Arrange.
Setup();
// Act.
_hc.SecretMasker.AddValue(secret);
// Assert.
Assert.Equal(maskedOutput, _hc.SecretMasker.MaskSecrets(rawOutput));
}
finally
{
// Cleanup.
Teardown();
}
}
[Fact]
[Trait("Level", "L0")]
[Trait("Category", "Common")]

View File

@@ -161,7 +161,8 @@ namespace GitHub.Runner.Common.Tests.Listener.Configuration
"--work", _expectedWorkFolder,
"--auth", _expectedAuthType,
"--token", _expectedToken,
"--labels", userLabels
"--labels", userLabels,
"--ephemeral",
});
trace.Info("Constructed.");
_store.Setup(x => x.IsConfigured()).Returns(false);
@@ -179,6 +180,7 @@ namespace GitHub.Runner.Common.Tests.Listener.Configuration
Assert.True(s.AgentName.Equals(_expectedAgentName));
Assert.True(s.PoolId.Equals(_secondRunnerGroupId));
Assert.True(s.WorkFolder.Equals(_expectedWorkFolder));
Assert.True(s.Ephemeral.Equals(true));
// validate GetAgentPoolsAsync gets called twice with automation pool type
_runnerServer.Verify(x => x.GetAgentPoolsAsync(It.IsAny<string>(), It.Is<TaskAgentPoolType>(p => p == TaskAgentPoolType.Automation)), Times.Exactly(2));

View File

@@ -264,6 +264,170 @@ namespace GitHub.Runner.Common.Tests.Listener
}
}
[Fact]
[Trait("Level", "L0")]
[Trait("Category", "Runner")]
public async void RenewJobRequestNewAgentNameUpdatesSettings()
{
//Arrange
using (var hc = new TestHostContext(this))
{
var count = 0;
var oldName = "OldName";
var newName = "NewName";
var oldSettings = new RunnerSettings { AgentName = oldName };
var reservedAgent = new TaskAgentReference { Name = newName };
var trace = hc.GetTrace(nameof(DispatcherRenewJobRequestStopOnJobTokenExpiredExceptions));
TaskCompletionSource<int> firstJobRequestRenewed = new TaskCompletionSource<int>();
CancellationTokenSource cancellationTokenSource = new CancellationTokenSource();
var request = new Mock<TaskAgentJobRequest>();
request.Object.ReservedAgent = reservedAgent;
PropertyInfo lockUntilProperty = request.Object.GetType().GetProperty("LockedUntil", BindingFlags.Instance | BindingFlags.NonPublic | BindingFlags.Public);
Assert.NotNull(lockUntilProperty);
lockUntilProperty.SetValue(request.Object, DateTime.UtcNow.AddMinutes(5));
hc.SetSingleton<IRunnerServer>(_runnerServer.Object);
hc.SetSingleton<IConfigurationStore>(_configurationStore.Object);
_configurationStore.Setup(x => x.GetSettings()).Returns(oldSettings);
_runnerServer.Setup(x => x.RenewAgentRequestAsync(It.IsAny<int>(), It.IsAny<long>(), It.IsAny<Guid>(), It.IsAny<string>(), It.IsAny<CancellationToken>()))
.Returns(() =>
{
count++;
if (count < 5)
{
return Task.FromResult<TaskAgentJobRequest>(request.Object);
}
else if (count == 5 || count == 6 || count == 7)
{
throw new TimeoutException("");
}
else
{
cancellationTokenSource.Cancel();
return Task.FromResult<TaskAgentJobRequest>(request.Object);
}
});
var jobDispatcher = new JobDispatcher();
jobDispatcher.Initialize(hc);
// Act
await jobDispatcher.RenewJobRequestAsync(0, 0, Guid.Empty, Guid.NewGuid().ToString(), firstJobRequestRenewed, cancellationTokenSource.Token);
// Assert
_configurationStore.Verify(x => x.SaveSettings(It.Is<RunnerSettings>(settings => settings.AgentName == newName)), Times.Once);
}
}
[Fact]
[Trait("Level", "L0")]
[Trait("Category", "Runner")]
public async void RenewJobRequestSameAgentNameIgnored()
{
//Arrange
using (var hc = new TestHostContext(this))
{
var count = 0;
var oldName = "OldName";
var newName = "OldName";
var oldSettings = new RunnerSettings { AgentName = oldName };
var reservedAgent = new TaskAgentReference { Name = newName };
var trace = hc.GetTrace(nameof(DispatcherRenewJobRequestStopOnJobTokenExpiredExceptions));
TaskCompletionSource<int> firstJobRequestRenewed = new TaskCompletionSource<int>();
CancellationTokenSource cancellationTokenSource = new CancellationTokenSource();
var request = new Mock<TaskAgentJobRequest>();
request.Object.ReservedAgent = reservedAgent;
PropertyInfo lockUntilProperty = request.Object.GetType().GetProperty("LockedUntil", BindingFlags.Instance | BindingFlags.NonPublic | BindingFlags.Public);
Assert.NotNull(lockUntilProperty);
lockUntilProperty.SetValue(request.Object, DateTime.UtcNow.AddMinutes(5));
hc.SetSingleton<IRunnerServer>(_runnerServer.Object);
hc.SetSingleton<IConfigurationStore>(_configurationStore.Object);
_configurationStore.Setup(x => x.GetSettings()).Returns(oldSettings);
_runnerServer.Setup(x => x.RenewAgentRequestAsync(It.IsAny<int>(), It.IsAny<long>(), It.IsAny<Guid>(), It.IsAny<string>(), It.IsAny<CancellationToken>()))
.Returns(() =>
{
count++;
if (count < 5)
{
return Task.FromResult<TaskAgentJobRequest>(request.Object);
}
else if (count == 5 || count == 6 || count == 7)
{
throw new TimeoutException("");
}
else
{
cancellationTokenSource.Cancel();
return Task.FromResult<TaskAgentJobRequest>(request.Object);
}
});
var jobDispatcher = new JobDispatcher();
jobDispatcher.Initialize(hc);
// Act
await jobDispatcher.RenewJobRequestAsync(0, 0, Guid.Empty, Guid.NewGuid().ToString(), firstJobRequestRenewed, cancellationTokenSource.Token);
// Assert
_configurationStore.Verify(x => x.SaveSettings(It.IsAny<RunnerSettings>()), Times.Never);
}
}
[Fact]
[Trait("Level", "L0")]
[Trait("Category", "Runner")]
public async void RenewJobRequestNullAgentNameIgnored()
{
//Arrange
using (var hc = new TestHostContext(this))
{
var count = 0;
var oldName = "OldName";
var oldSettings = new RunnerSettings { AgentName = oldName };
var trace = hc.GetTrace(nameof(DispatcherRenewJobRequestStopOnJobTokenExpiredExceptions));
TaskCompletionSource<int> firstJobRequestRenewed = new TaskCompletionSource<int>();
CancellationTokenSource cancellationTokenSource = new CancellationTokenSource();
var request = new Mock<TaskAgentJobRequest>();
PropertyInfo lockUntilProperty = request.Object.GetType().GetProperty("LockedUntil", BindingFlags.Instance | BindingFlags.NonPublic | BindingFlags.Public);
Assert.NotNull(lockUntilProperty);
lockUntilProperty.SetValue(request.Object, DateTime.UtcNow.AddMinutes(5));
hc.SetSingleton<IRunnerServer>(_runnerServer.Object);
hc.SetSingleton<IConfigurationStore>(_configurationStore.Object);
_configurationStore.Setup(x => x.GetSettings()).Returns(oldSettings);
_runnerServer.Setup(x => x.RenewAgentRequestAsync(It.IsAny<int>(), It.IsAny<long>(), It.IsAny<Guid>(), It.IsAny<string>(), It.IsAny<CancellationToken>()))
.Returns(() =>
{
count++;
if (count < 5)
{
return Task.FromResult<TaskAgentJobRequest>(request.Object);
}
else if (count == 5 || count == 6 || count == 7)
{
throw new TimeoutException("");
}
else
{
cancellationTokenSource.Cancel();
return Task.FromResult<TaskAgentJobRequest>(request.Object);
}
});
var jobDispatcher = new JobDispatcher();
jobDispatcher.Initialize(hc);
// Act
await jobDispatcher.RenewJobRequestAsync(0, 0, Guid.Empty, Guid.NewGuid().ToString(), firstJobRequestRenewed, cancellationTokenSource.Token);
// Assert
_configurationStore.Verify(x => x.SaveSettings(It.IsAny<RunnerSettings>()), Times.Never);
}
}
[Fact]
[Trait("Level", "L0")]
[Trait("Category", "Runner")]

View File

@@ -149,6 +149,9 @@ namespace GitHub.Runner.Common.Tests.Listener
_messageListener.Verify(x => x.CreateSessionAsync(It.IsAny<CancellationToken>()), Times.Once());
_messageListener.Verify(x => x.DeleteSessionAsync(), Times.Once());
_messageListener.Verify(x => x.DeleteMessageAsync(It.IsAny<TaskAgentMessage>()), Times.AtLeastOnce());
// verify that we didn't try to delete local settings file (since we're not ephemeral)
_configurationManager.Verify(x => x.DeleteLocalRunnerConfig(), Times.Never());
}
}
}
@@ -243,7 +246,8 @@ namespace GitHub.Runner.Common.Tests.Listener
runner.Initialize(hc);
var settings = new RunnerSettings
{
PoolId = 43242
PoolId = 43242,
Ephemeral = true
};
var message = new TaskAgentMessage()
@@ -294,7 +298,7 @@ namespace GitHub.Runner.Common.Tests.Listener
_configStore.Setup(x => x.IsServiceConfigured()).Returns(false);
//Act
var command = new CommandSettings(hc, new string[] { "run", "--once" });
var command = new CommandSettings(hc, new string[] { "run" });
Task<int> runnerTask = runner.ExecuteCommand(command);
//Assert
@@ -311,6 +315,9 @@ namespace GitHub.Runner.Common.Tests.Listener
_messageListener.Verify(x => x.CreateSessionAsync(It.IsAny<CancellationToken>()), Times.Once());
_messageListener.Verify(x => x.DeleteSessionAsync(), Times.Once());
_messageListener.Verify(x => x.DeleteMessageAsync(It.IsAny<TaskAgentMessage>()), Times.AtLeastOnce());
// verify that we did try to delete local settings file (since we're ephemeral)
_configurationManager.Verify(x => x.DeleteLocalRunnerConfig(), Times.Once());
}
}
@@ -332,7 +339,8 @@ namespace GitHub.Runner.Common.Tests.Listener
runner.Initialize(hc);
var settings = new RunnerSettings
{
PoolId = 43242
PoolId = 43242,
Ephemeral = true
};
var message1 = new TaskAgentMessage()
@@ -390,7 +398,7 @@ namespace GitHub.Runner.Common.Tests.Listener
_configStore.Setup(x => x.IsServiceConfigured()).Returns(false);
//Act
var command = new CommandSettings(hc, new string[] { "run", "--once" });
var command = new CommandSettings(hc, new string[] { "run" });
Task<int> runnerTask = runner.ExecuteCommand(command);
//Assert
@@ -431,7 +439,8 @@ namespace GitHub.Runner.Common.Tests.Listener
var settings = new RunnerSettings
{
PoolId = 43242,
AgentId = 5678
AgentId = 5678,
Ephemeral = true
};
var message1 = new TaskAgentMessage()
@@ -475,7 +484,7 @@ namespace GitHub.Runner.Common.Tests.Listener
_configStore.Setup(x => x.IsServiceConfigured()).Returns(false);
//Act
var command = new CommandSettings(hc, new string[] { "run", "--once" });
var command = new CommandSettings(hc, new string[] { "run" });
Task<int> runnerTask = runner.ExecuteCommand(command);
//Assert

View File

@@ -2,6 +2,7 @@
using System.Collections.Generic;
using System.IO;
using System.Runtime.CompilerServices;
using GitHub.DistributedTask.Pipelines.ContextData;
using GitHub.DistributedTask.WebApi;
using GitHub.Runner.Worker;
using GitHub.Runner.Worker.Container;
@@ -83,6 +84,7 @@ namespace GitHub.Runner.Common.Tests.Worker
{
using (TestHostContext hc = CreateTestContext())
{
_ec.Setup(x => x.ExpressionValues).Returns(GetExpressionValues());
_ec.Setup(x => x.Write(It.IsAny<string>(), It.IsAny<string>()))
.Returns((string tag, string line) =>
{
@@ -105,6 +107,88 @@ namespace GitHub.Runner.Common.Tests.Worker
}
}
[Theory]
[InlineData("stop-commands", "1")]
[InlineData("", "1")]
[InlineData("set-env", "1")]
[InlineData("stop-commands", "true")]
[InlineData("", "true")]
[InlineData("set-env", "true")]
[Trait("Level", "L0")]
[Trait("Category", "Worker")]
public void StopProcessCommand__AllowsInvalidStopTokens__IfEnvVarIsSet(string invalidToken, string allowUnsupportedStopCommandTokens)
{
using (TestHostContext hc = CreateTestContext())
{
_ec.Object.Global.EnvironmentVariables = new Dictionary<string, string>();
var expressionValues = new DictionaryContextData
{
["env"] =
#if OS_WINDOWS
new DictionaryContextData{ { Constants.Variables.Actions.AllowUnsupportedStopCommandTokens, new StringContextData(allowUnsupportedStopCommandTokens) }}
#else
new CaseSensitiveDictionaryContextData{ { Constants.Variables.Actions.AllowUnsupportedStopCommandTokens, new StringContextData(allowUnsupportedStopCommandTokens) }}
#endif
};
_ec.Setup(x => x.ExpressionValues).Returns(expressionValues);
_ec.Setup(x => x.JobTelemetry).Returns(new List<JobTelemetry>());
Assert.True(_commandManager.TryProcessCommand(_ec.Object, $"::stop-commands::{invalidToken}", null));
}
}
[Theory]
[InlineData("stop-commands")]
[InlineData("")]
[InlineData("set-env")]
[Trait("Level", "L0")]
[Trait("Category", "Worker")]
public void StopProcessCommand__FailOnInvalidStopTokens(string invalidToken)
{
using (TestHostContext hc = CreateTestContext())
{
_ec.Object.Global.EnvironmentVariables = new Dictionary<string, string>();
_ec.Setup(x => x.ExpressionValues).Returns(GetExpressionValues());
_ec.Setup(x => x.JobTelemetry).Returns(new List<JobTelemetry>());
Assert.Throws<Exception>(() => _commandManager.TryProcessCommand(_ec.Object, $"::stop-commands::{invalidToken}", null));
}
}
[Fact]
[Trait("Level", "L0")]
[Trait("Category", "Worker")]
public void StopProcessCommandAcceptsValidToken()
{
var validToken = "randomToken";
using (TestHostContext hc = CreateTestContext())
{
_ec.Setup(x => x.ExpressionValues).Returns(GetExpressionValues());
Assert.True(_commandManager.TryProcessCommand(_ec.Object, $"::stop-commands::{validToken}", null));
Assert.False(_commandManager.TryProcessCommand(_ec.Object, "##[set-env name=foo]bar", null));
Assert.True(_commandManager.TryProcessCommand(_ec.Object, $"::{validToken}::", null));
Assert.True(_commandManager.TryProcessCommand(_ec.Object, "##[set-env name=foo]bar", null));
}
}
[Fact]
[Trait("Level", "L0")]
[Trait("Category", "Worker")]
public void StopProcessCommandMasksValidTokenForEntireRun()
{
var validToken = "randomToken";
using (TestHostContext hc = CreateTestContext())
{
_ec.Setup(x => x.ExpressionValues).Returns(GetExpressionValues());
Assert.True(_commandManager.TryProcessCommand(_ec.Object, $"::stop-commands::{validToken}", null));
Assert.False(_commandManager.TryProcessCommand(_ec.Object, "##[set-env name=foo]bar", null));
Assert.Equal("***", hc.SecretMasker.MaskSecrets(validToken));
Assert.True(_commandManager.TryProcessCommand(_ec.Object, $"::{validToken}::", null));
Assert.True(_commandManager.TryProcessCommand(_ec.Object, "##[set-env name=foo]bar", null));
Assert.Equal("***", hc.SecretMasker.MaskSecrets(validToken));
}
}
[Fact]
[Trait("Level", "L0")]
[Trait("Category", "Worker")]
@@ -202,15 +286,15 @@ namespace GitHub.Runner.Common.Tests.Worker
return 1;
});
var registeredCommands = new HashSet<string>(new string[1]{ "warning" });
var registeredCommands = new HashSet<string>(new string[1] { "warning" });
ActionCommand command;
// Columns when lines are different
ActionCommand.TryParseV2("::warning line=1,endLine=2,col=1,endColumn=2::this is a warning", registeredCommands, out command);
Assert.Equal("1", command.Properties["col"]);
IssueCommandExtension.ValidateLinesAndColumns(command, _ec.Object);
Assert.False(command.Properties.ContainsKey("col"));
// No lines with columns
ActionCommand.TryParseV2("::warning col=1,endColumn=2::this is a warning", registeredCommands, out command);
Assert.Equal("1", command.Properties["col"]);
@@ -375,5 +459,19 @@ namespace GitHub.Runner.Common.Tests.Worker
return hostContext;
}
private DictionaryContextData GetExpressionValues()
{
return new DictionaryContextData
{
["env"] =
#if OS_WINDOWS
new DictionaryContextData()
#else
new CaseSensitiveDictionaryContextData()
#endif
};
}
}
}

View File

@@ -965,7 +965,7 @@ namespace GitHub.Runner.Common.Tests.Worker
};
//Act
var result = await _actionManager.PrepareActionsAsync(_ec.Object, actions);
var result = await _actionManager.PrepareActionsAsync(_ec.Object, actions);
//Assert
Assert.Equal(1, result.PreStepTracker.Count);
@@ -1288,7 +1288,7 @@ runs:
[Fact]
[Trait("Level", "L0")]
[Trait("Category", "Worker")]
public void LoadsNodeActionDefinition()
public void LoadsNode12ActionDefinition()
{
try
{
@@ -1344,8 +1344,78 @@ runs:
Assert.True(string.IsNullOrEmpty(inputDefaults["entryPoint"]));
Assert.NotNull(definition.Data.Execution); // execution
Assert.NotNull((definition.Data.Execution as NodeJSActionExecutionData));
Assert.NotNull(definition.Data.Execution as NodeJSActionExecutionData);
Assert.Equal("task.js", (definition.Data.Execution as NodeJSActionExecutionData).Script);
Assert.Equal("node12", (definition.Data.Execution as NodeJSActionExecutionData).NodeVersion);
}
finally
{
Teardown();
}
}
[Fact]
[Trait("Level", "L0")]
[Trait("Category", "Worker")]
public void LoadsNode16ActionDefinition()
{
try
{
// Arrange.
Setup();
const string Content = @"
# Container action
name: 'Hello World'
description: 'Greet the world and record the time'
author: 'GitHub'
inputs:
greeting: # id of input
description: 'The greeting we choose - will print ""{greeting}, World!"" on stdout'
required: true
default: 'Hello'
entryPoint: # id of input
description: 'optional docker entrypoint overwrite.'
required: false
outputs:
time: # id of output
description: 'The time we did the greeting'
icon: 'hello.svg' # vector art to display in the GitHub Marketplace
color: 'green' # optional, decorates the entry in the GitHub Marketplace
runs:
using: 'node16'
main: 'task.js'
";
Pipelines.ActionStep instance;
string directory;
CreateAction(yamlContent: Content, instance: out instance, directory: out directory);
// Act.
Definition definition = _actionManager.LoadAction(_ec.Object, instance);
// Assert.
Assert.NotNull(definition);
Assert.Equal(directory, definition.Directory);
Assert.NotNull(definition.Data);
Assert.NotNull(definition.Data.Inputs); // inputs
Dictionary<string, string> inputDefaults = new Dictionary<string, string>(StringComparer.OrdinalIgnoreCase);
foreach (var input in definition.Data.Inputs)
{
var name = input.Key.AssertString("key").Value;
var value = input.Value.AssertScalar("value").ToString();
_hc.GetTrace().Info($"Default: {name} = {value}");
inputDefaults[name] = value;
}
Assert.Equal(2, inputDefaults.Count);
Assert.True(inputDefaults.ContainsKey("greeting"));
Assert.Equal("Hello", inputDefaults["greeting"]);
Assert.True(string.IsNullOrEmpty(inputDefaults["entryPoint"]));
Assert.NotNull(definition.Data.Execution); // execution
Assert.NotNull(definition.Data.Execution as NodeJSActionExecutionData);
Assert.Equal("task.js", (definition.Data.Execution as NodeJSActionExecutionData).Script);
Assert.Equal("node16", (definition.Data.Execution as NodeJSActionExecutionData).NodeVersion);
}
finally
{

View File

@@ -408,6 +408,50 @@ namespace GitHub.Runner.Common.Tests.Worker
var nodeAction = result.Execution as NodeJSActionExecutionData;
Assert.Equal("main.js", nodeAction.Script);
Assert.Equal("node12", nodeAction.NodeVersion);
}
finally
{
Teardown();
}
}
[Fact]
[Trait("Level", "L0")]
[Trait("Category", "Worker")]
public void Load_Node16Action()
{
try
{
//Arrange
Setup();
var actionManifest = new ActionManifestManager();
actionManifest.Initialize(_hc);
//Act
var result = actionManifest.Load(_ec.Object, Path.Combine(TestUtil.GetTestDataPath(), "node16action.yml"));
//Assert
Assert.Equal("Hello World", result.Name);
Assert.Equal("Greet the world and record the time", result.Description);
Assert.Equal(2, result.Inputs.Count);
Assert.Equal("greeting", result.Inputs[0].Key.AssertString("key").Value);
Assert.Equal("Hello", result.Inputs[0].Value.AssertString("value").Value);
Assert.Equal("entryPoint", result.Inputs[1].Key.AssertString("key").Value);
Assert.Equal("", result.Inputs[1].Value.AssertString("value").Value);
Assert.Equal(1, result.Deprecated.Count);
Assert.True(result.Deprecated.ContainsKey("greeting"));
result.Deprecated.TryGetValue("greeting", out string value);
Assert.Equal("This property has been deprecated", value);
Assert.Equal(ActionExecutionType.NodeJS, result.Execution.ExecutionType);
var nodeAction = result.Execution as NodeJSActionExecutionData;
Assert.Equal("main.js", nodeAction.Script);
Assert.Equal("node16", nodeAction.NodeVersion);
}
finally
{
@@ -626,6 +670,32 @@ namespace GitHub.Runner.Common.Tests.Worker
{
Teardown();
}
}
[Fact]
[Trait("Level", "L0")]
[Trait("Category", "Worker")]
public void Load_ConditionalCompositeAction()
{
try
{
//Arrange
Setup();
var actionManifest = new ActionManifestManager();
actionManifest.Initialize(_hc);
//Act
var result = actionManifest.Load(_ec.Object, Path.Combine(TestUtil.GetTestDataPath(), "conditional_composite_action.yml"));
//Assert
Assert.Equal("Conditional Composite", result.Name);
Assert.Equal(ActionExecutionType.Composite, result.Execution.ExecutionType);
}
finally
{
Teardown();
}
}
[Fact]

View File

@@ -193,9 +193,9 @@ namespace GitHub.Runner.Common.Tests.Worker
// Act.
jobContext.InitializeJob(jobRequest, CancellationToken.None);
var action1 = jobContext.CreateChild(Guid.NewGuid(), "action_1", "action_1", null, null);
var action1 = jobContext.CreateChild(Guid.NewGuid(), "action_1", "action_1", null, null, 0);
action1.IntraActionState["state"] = "1";
var action2 = jobContext.CreateChild(Guid.NewGuid(), "action_2", "action_2", null, null);
var action2 = jobContext.CreateChild(Guid.NewGuid(), "action_2", "action_2", null, null, 0);
action2.IntraActionState["state"] = "2";
@@ -291,8 +291,8 @@ namespace GitHub.Runner.Common.Tests.Worker
// Act.
jobContext.InitializeJob(jobRequest, CancellationToken.None);
var action1 = jobContext.CreateChild(Guid.NewGuid(), "action_1_pre", "action_1_pre", null, null);
var action2 = jobContext.CreateChild(Guid.NewGuid(), "action_1_main", "action_1_main", null, null);
var action1 = jobContext.CreateChild(Guid.NewGuid(), "action_1_pre", "action_1_pre", null, null, 0);
var action2 = jobContext.CreateChild(Guid.NewGuid(), "action_1_main", "action_1_main", null, null, 0);
var actionId = Guid.NewGuid();
var postRunner1 = hc.CreateService<IActionRunner>();

View File

@@ -105,6 +105,36 @@ namespace GitHub.Runner.Common.Tests.Worker.Expressions
}
}
[Theory]
[Trait("Level", "L0")]
[Trait("Category", "Worker")]
[InlineData(ActionResult.Failure, ActionResult.Failure, true)]
[InlineData(ActionResult.Failure, ActionResult.Success, false)]
[InlineData(ActionResult.Success, ActionResult.Failure, true)]
[InlineData(ActionResult.Success, ActionResult.Success, false)]
[InlineData(ActionResult.Success, null, false)]
public void FailureFunctionComposite(ActionResult jobStatus, ActionResult? actionStatus, bool expected)
{
using (TestHostContext hc = CreateTestContext())
{
// Arrange
var executionContext = InitializeExecutionContext(hc);
executionContext.Setup(x => x.GetGitHubContext("action_status")).Returns(actionStatus.ToString());
executionContext.Setup( x=> x.IsEmbedded).Returns(true);
executionContext.Setup( x=> x.Stage).Returns(ActionRunStage.Main);
_jobContext.Status = jobStatus;
// Act.
bool actual = Evaluate("failure()");
// Assert.
Assert.Equal(expected, actual);
}
}
[Fact]
[Trait("Level", "L0")]
[Trait("Category", "Worker")]
@@ -134,12 +164,43 @@ namespace GitHub.Runner.Common.Tests.Worker.Expressions
}
}
[Theory]
[Trait("Level", "L0")]
[Trait("Category", "Worker")]
[InlineData(ActionResult.Failure, ActionResult.Failure, false)]
[InlineData(ActionResult.Failure, ActionResult.Success, true)]
[InlineData(ActionResult.Success, ActionResult.Failure, false)]
[InlineData(ActionResult.Success, ActionResult.Success, true)]
[InlineData(ActionResult.Success, null, true)]
public void SuccessFunctionComposite(ActionResult jobStatus, ActionResult? actionStatus, bool expected)
{
using (TestHostContext hc = CreateTestContext())
{
// Arrange
var executionContext = InitializeExecutionContext(hc);
executionContext.Setup(x => x.GetGitHubContext("action_status")).Returns(actionStatus.ToString());
executionContext.Setup( x=> x.IsEmbedded).Returns(true);
executionContext.Setup( x=> x.Stage).Returns(ActionRunStage.Main);
_jobContext.Status = jobStatus;
// Act.
bool actual = Evaluate("success()");
// Assert.
Assert.Equal(expected, actual);
}
}
private TestHostContext CreateTestContext([CallerMemberName] String testName = "")
{
return new TestHostContext(this, testName);
}
private void InitializeExecutionContext(TestHostContext hc)
private Mock<IExecutionContext> InitializeExecutionContext(TestHostContext hc)
{
_jobContext = new JobContext();
@@ -149,6 +210,8 @@ namespace GitHub.Runner.Common.Tests.Worker.Expressions
_templateContext = new TemplateContext();
_templateContext.State[nameof(IExecutionContext)] = executionContext.Object;
return executionContext;
}
private bool Evaluate(string expression)

View File

@@ -392,6 +392,35 @@ namespace GitHub.Runner.Common.Tests.Worker
Assert.Equal("not-working", match.Message);
}
[Fact]
[Trait("Level", "L0")]
[Trait("Category", "Worker")]
public void Matcher_MultiplePatterns_DefaultSeverityNotice()
{
var config = JsonUtility.FromString<IssueMatchersConfig>(@"
{
""problemMatcher"": [
{
""owner"": ""myMatcher"",
""severity"": ""notice"",
""pattern"": [
{
""regexp"": ""^(.+)$"",
""message"": 1
}
]
}
]
}
");
config.Validate();
var matcher = new IssueMatcher(config.Matchers[0], TimeSpan.FromSeconds(1));
var match = matcher.Match("just-a-notice");
Assert.Equal("notice", match.Severity);
Assert.Equal("just-a-notice", match.Message);
}
[Fact]
[Trait("Level", "L0")]
[Trait("Category", "Worker")]

View File

@@ -102,6 +102,7 @@ namespace GitHub.Runner.Common.Tests.Worker
_message = new Pipelines.AgentJobRequestMessage(plan, timeline, jobId, "test", "test", null, null, null, new Dictionary<string, VariableValue>(), new List<MaskHint>(), new Pipelines.JobResources(), new Pipelines.ContextData.DictionaryContextData(), new Pipelines.WorkspaceOptions(), steps, null, null, null, null);
GitHubContext github = new GitHubContext();
github["repository"] = new Pipelines.ContextData.StringContextData("actions/runner");
github["secret_source"] = new Pipelines.ContextData.StringContextData("Actions");
_message.ContextData.Add("github", github);
hc.SetSingleton(_actionManager.Object);

View File

@@ -0,0 +1,109 @@
using System;
using System.Collections.Generic;
using System.Runtime.CompilerServices;
using System.Threading.Tasks;
using Moq;
using Xunit;
using GitHub.Runner.Worker;
using GitHub.Runner.Worker.Handlers;
using GitHub.Runner.Worker.Container;
namespace GitHub.Runner.Common.Tests.Worker
{
public sealed class StepHostL0
{
private Mock<IExecutionContext> _ec;
private Mock<IDockerCommandManager> _dc;
private TestHostContext CreateTestContext([CallerMemberName] String testName = "")
{
var hc = new TestHostContext(this, testName);
_ec = new Mock<IExecutionContext>();
_ec.SetupAllProperties();
_ec.Setup(x => x.Global).Returns(new GlobalContext { WriteDebug = true });
var trace = hc.GetTrace();
_ec.Setup(x => x.Write(It.IsAny<string>(), It.IsAny<string>())).Callback((string tag, string message) => { trace.Info($"[{tag}]{message}"); });
_dc = new Mock<IDockerCommandManager>();
hc.SetSingleton(_dc.Object);
return hc;
}
[Fact]
[Trait("Level", "L0")]
[Trait("Category", "Worker")]
public async Task DetermineNodeRuntimeVersionInContainerAsync()
{
using (TestHostContext hc = CreateTestContext())
{
// Arrange.
var sh = new ContainerStepHost();
sh.Initialize(hc);
sh.Container = new ContainerInfo() { ContainerId = "1234abcd" };
_dc.Setup(d => d.DockerExec(_ec.Object, It.IsAny<string>(), It.IsAny<string>(), It.IsAny<string>(), It.IsAny<List<string>>()))
.ReturnsAsync(0);
// Act.
var nodeVersion = await sh.DetermineNodeRuntimeVersion(_ec.Object, "node12");
// Assert.
Assert.Equal("node12", nodeVersion);
}
}
[Fact]
[Trait("Level", "L0")]
[Trait("Category", "Worker")]
public async Task DetermineNodeRuntimeVersionInAlpineContainerAsync()
{
using (TestHostContext hc = CreateTestContext())
{
// Arrange.
var sh = new ContainerStepHost();
sh.Initialize(hc);
sh.Container = new ContainerInfo() { ContainerId = "1234abcd" };
_dc.Setup(d => d.DockerExec(_ec.Object, It.IsAny<string>(), It.IsAny<string>(), It.IsAny<string>(), It.IsAny<List<string>>()))
.Callback((IExecutionContext ec, string id, string options, string command, List<string> output) =>
{
output.Add("alpine");
})
.ReturnsAsync(0);
// Act.
var nodeVersion = await sh.DetermineNodeRuntimeVersion(_ec.Object, "node16");
// Assert.
Assert.Equal("node16_alpine", nodeVersion);
}
}
[Fact]
[Trait("Level", "L0")]
[Trait("Category", "Worker")]
public async Task DetermineNodeRuntimeVersionInUnknowContainerAsync()
{
using (TestHostContext hc = CreateTestContext())
{
// Arrange.
var sh = new ContainerStepHost();
sh.Initialize(hc);
sh.Container = new ContainerInfo() { ContainerId = "1234abcd" };
_dc.Setup(d => d.DockerExec(_ec.Object, It.IsAny<string>(), It.IsAny<string>(), It.IsAny<string>(), It.IsAny<List<string>>()))
.Callback((IExecutionContext ec, string id, string options, string command, List<string> output) =>
{
output.Add("github");
})
.ReturnsAsync(0);
// Act.
var nodeVersion = await sh.DetermineNodeRuntimeVersion(_ec.Object, "node16");
// Assert.
Assert.Equal("node16", nodeVersion);
}
}
}
}

View File

@@ -3,7 +3,6 @@
<TargetFramework>netcoreapp3.1</TargetFramework>
<RuntimeIdentifiers>win-x64;win-x86;linux-x64;linux-arm64;linux-arm;osx-x64</RuntimeIdentifiers>
<TargetLatestRuntimePatch>true</TargetLatestRuntimePatch>
<AssetTargetFallback>portable-net45+win8</AssetTargetFallback>
<NoWarn>NU1701;NU1603;NU1603;xUnit2013;</NoWarn>
</PropertyGroup>

View File

@@ -0,0 +1,49 @@
name: 'Conditional Composite'
description: 'Test composite run step conditionals'
inputs:
exit-code:
description: 'Action fails if set to non-zero'
default: '0'
outputs:
default:
description: "Did step run with default?"
value: ${{ steps.default-conditional.outputs.default }}
success:
description: "Did step run with success?"
value: ${{ steps.success-conditional.outputs.success }}
failure:
description: "Did step run with failure?"
value: ${{ steps.failure-conditional.outputs.failure }}
always:
description: "Did step run with always?"
value: ${{ steps.always-conditional.outputs.always }}
runs:
using: "composite"
steps:
- run: exit ${{ inputs.exit-code }}
shell: bash
- run: echo "::set-output name=default::true"
id: default-conditional
shell: bash
- run: echo "::set-output name=success::true"
id: success-conditional
shell: bash
if: success()
- run: echo "::set-output name=failure::true"
id: failure-conditional
shell: bash
if: failure()
- run: echo "::set-output name=always::true"
id: always-conditional
shell: bash
if: always()
- run: echo "failed"
shell: bash
if: ${{ inputs.exit-code == 1 && failure() }}

View File

@@ -0,0 +1,20 @@
name: 'Hello World'
description: 'Greet the world and record the time'
author: 'Test Corporation'
inputs:
greeting: # id of input
description: 'The greeting we choose - will print ""{greeting}, World!"" on stdout'
required: true
default: 'Hello'
deprecationMessage: 'This property has been deprecated'
entryPoint: # id of input
description: 'optional docker entrypoint overwrite.'
required: false
outputs:
time: # id of output
description: 'The time we did the greeting'
icon: 'hello.svg' # vector art to display in the GitHub Marketplace
color: 'green' # optional, decorates the entry in the GitHub Marketplace
runs:
using: 'node16'
main: 'main.js'

View File

@@ -1 +1 @@
2.280.1
2.285.0