mirror of
https://github.com/actions/runner.git
synced 2025-12-10 12:36:23 +00:00
Compare commits
28 Commits
releases/m
...
v2.281.1
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
c8caf59bb7 | ||
|
|
77c5a664ed | ||
|
|
b6aa01fabc | ||
|
|
3615fb6923 | ||
|
|
f61dcad5bb | ||
|
|
62d568674c | ||
|
|
07c00f6a8a | ||
|
|
05b84297b7 | ||
|
|
04679b56a9 | ||
|
|
d2ca24fa43 | ||
|
|
abdaacfa6e | ||
|
|
53fd7161e2 | ||
|
|
ce68f3b167 | ||
|
|
e2c7329292 | ||
|
|
22a9d89772 | ||
|
|
3851acd0cf | ||
|
|
aab4aca8f7 | ||
|
|
5af7b87074 | ||
|
|
110eb3a5de | ||
|
|
bd1341e580 | ||
|
|
85ce33b1d3 | ||
|
|
92ec3d0f29 | ||
|
|
4e95d0d6ad | ||
|
|
5281434f3f | ||
|
|
e9a8bf29df | ||
|
|
a65331e887 | ||
|
|
908a082527 | ||
|
|
10ba74f59b |
92
docs/adrs/1144-composite-actions.md
Normal file
92
docs/adrs/1144-composite-actions.md
Normal file
@@ -0,0 +1,92 @@
|
|||||||
|
**Date**: 2021-06-10
|
||||||
|
|
||||||
|
**Status**: Accepted
|
||||||
|
|
||||||
|
## Context
|
||||||
|
|
||||||
|
We released [composite run steps](https://github.com/actions/runner/pull/554) last year which started our journey of reusing steps across different workflow files. To continue that journey, we want to expand composite run steps into composite actions.
|
||||||
|
|
||||||
|
We want to support the `uses` steps from workflows in composite actions, including:
|
||||||
|
- Container actions
|
||||||
|
- Javascript actions
|
||||||
|
- Other Composite actions (up to a limit of course!)
|
||||||
|
- The pre and post steps these actions can generate
|
||||||
|
|
||||||
|
## Guiding Principles
|
||||||
|
|
||||||
|
- Composite Actions should function as a single step or action, no matter how many steps it is composed of or how many levels of recursion it has
|
||||||
|
- In the future we may add a configurable option to make this no longer the case
|
||||||
|
- A workflow author should not need to understand the inner workings of a composite action in order to use it
|
||||||
|
- Composite actions should leverage inputs to get values they need, they will not have full access to the `context` objects. The secrets context will **not** be available to composite actions, users will need to pass these values in as an input.
|
||||||
|
- Other Actions should **just work** inside a composite action, without any code changes
|
||||||
|
|
||||||
|
## Decisions
|
||||||
|
|
||||||
|
### Composite Recursion Limit
|
||||||
|
|
||||||
|
- We will start with supporting a recursion limit of `10` composite actions deep
|
||||||
|
- We are free to bump this limit in the future, the code will be written to just require updating a variable. If the graph evaluates beyond the recursion limit, the job will fail in the pre-job phase (The `Set up job` step).
|
||||||
|
- A composite actions interface is its inputs and outputs, nothing else is carried over when invoking recursively.
|
||||||
|
|
||||||
|
### Pre/Post Steps in nested Actions
|
||||||
|
|
||||||
|
- We do not plan on adding the ability to configure a customizable pre or post step for composite actions at this time. However, we will execute the pre and post steps of any actions referenced in a composite action.
|
||||||
|
- Composite actions will generate a single pre-step and post-step for the entire composite action, even if there are multiple pre-steps and post-steps in the referenced actions.
|
||||||
|
- These steps will execute following the same ordering rules we have today, first to run has their pre step run first and their post step run last.
|
||||||
|
- For example, if you had a composite action with two pre steps and two posts steps:
|
||||||
|
|
||||||
|
```
|
||||||
|
- uses: action1
|
||||||
|
- uses: composite1
|
||||||
|
- uses: action2
|
||||||
|
```
|
||||||
|
|
||||||
|
The order of execution would be:
|
||||||
|
|
||||||
|
```
|
||||||
|
- prestep-action1
|
||||||
|
- prestep-composite1
|
||||||
|
- prestep-composite1-first-action-referenced
|
||||||
|
- prestep-composite1-second-action-referenced
|
||||||
|
- prestep-action2
|
||||||
|
- the job steps
|
||||||
|
- poststep-action2
|
||||||
|
- poststep-composite1
|
||||||
|
- poststep-composite1-the-second-action-referenced
|
||||||
|
- poststep-composite1-first-action-referenced
|
||||||
|
- poststep-action1
|
||||||
|
```
|
||||||
|
|
||||||
|
#### Set-state
|
||||||
|
|
||||||
|
- While the composite action has an individual combined pre/post action, the `set-state` command will not be shared.
|
||||||
|
- If the `set-state` command is used during a composite step, only the action that originally called `set-state` will have access to the env variable during the post run step.
|
||||||
|
- This prevents multiple actions that set the same state from interfering with the execution of another action's post step.
|
||||||
|
|
||||||
|
### Resolve Action Endpoint changes
|
||||||
|
|
||||||
|
- The resolve actions endpoint will now validate policy to ensure that the given workflow run has access to download that action.
|
||||||
|
- Older GHES/GHAE customers with newer runners will be locked out of composite uses steps until they upgrade their instance.
|
||||||
|
|
||||||
|
### Local actions
|
||||||
|
- Local actions will expand the tree, perform policy checks, and download actions Just in Time when the step is running.
|
||||||
|
- Like current local actions, we will not support presteps. If an action is running local, by the time we know that, the time to run presteps have already passed.
|
||||||
|
|
||||||
|
### If, continue-on-error, timeout-minutes - Not being considered at this time
|
||||||
|
|
||||||
|
- `if`, `continue-on-error`, `timeout-minutes` could be supported in composite run/uses steps. These values were not originally supported in our composite run steps implementation.
|
||||||
|
- Browsing the community forums and runner repo, there hasn't been a lot of noise asking for these features, so we will hold off on them.
|
||||||
|
- These values passed as input into the composite action will **not** be carried over as input into the individual steps the composite action runs.
|
||||||
|
|
||||||
|
### Defaults - Not being considered at this time
|
||||||
|
|
||||||
|
- In actions, we have the idea of [defaults](https://docs.github.com/en/actions/reference/workflow-syntax-for-github-actions#defaultsrun) , which allow you to specify a shell and working directory in one location, rather then on each step.
|
||||||
|
- However, `shell` is currently required in composite run steps
|
||||||
|
- In regular run steps, it is optional, and defaults to a different value based on the OS.
|
||||||
|
- We want to prioritize the right experience for the consumer, and make the action author continue to explicitly set these values. We can consider improving this experience in the future.
|
||||||
|
|
||||||
|
## Consequences
|
||||||
|
|
||||||
|
- Workflows are now more reusable across multiple workflow files
|
||||||
|
- Composite actions implement most of the existing workflow run steps, with room to expand these in the future
|
||||||
|
- Feature flags will control this rollout
|
||||||
@@ -26,6 +26,23 @@ Run as a one-liner. NOTE: replace with yourorg/yourrepo (repo level) or just you
|
|||||||
curl -s https://raw.githubusercontent.com/actions/runner/main/scripts/create-latest-svc.sh | bash -s yourorg/yourrepo
|
curl -s https://raw.githubusercontent.com/actions/runner/main/scripts/create-latest-svc.sh | bash -s yourorg/yourrepo
|
||||||
```
|
```
|
||||||
|
|
||||||
|
You can call the script with additional arguments:
|
||||||
|
```bash
|
||||||
|
# Usage:
|
||||||
|
# export RUNNER_CFG_PAT=<yourPAT>
|
||||||
|
# ./create-latest-svc -s scope -g [ghe_domain] -n [name] -u [user] -l [labels]
|
||||||
|
# -s required scope: repo (:owner/:repo) or org (:organization)
|
||||||
|
# -g optional ghe_hostname: the fully qualified domain name of your GitHub Enterprise Server deployment
|
||||||
|
# -n optional name of the runner, defaults to hostname
|
||||||
|
# -u optional user svc will run as, defaults to current
|
||||||
|
# -l optional list of labels (split by comma) applied on the runner"
|
||||||
|
```
|
||||||
|
|
||||||
|
Use `--` to pass any number of optional named parameters:
|
||||||
|
|
||||||
|
```
|
||||||
|
curl -s https://raw.githubusercontent.com/actions/runner/main/scripts/create-latest-svc.sh | bash -s -- -s myorg/myrepo -n myname -l label1,label2
|
||||||
|
```
|
||||||
### Why can't I use a container?
|
### Why can't I use a container?
|
||||||
|
|
||||||
The runner is installed as a service using `systemd` and `systemctl`. Docker does not support `systemd` for service configuration on a container.
|
The runner is installed as a service using `systemd` and `systemctl`. Docker does not support `systemd` for service configuration on a container.
|
||||||
|
|||||||
@@ -1,22 +1,17 @@
|
|||||||
## Features
|
## Features
|
||||||
|
|
||||||
- Add Job Message size to both Worker and Listener logs for debugging (#1100)
|
- Allow setting default severity to "notice" (#1213)
|
||||||
- Add notice annotation level (in addition to error and warning) and support more annotation fields (#1175)
|
- Show More Step Information in composite Actions (#1279)
|
||||||
|
|
||||||
## Bugs
|
## Bugs
|
||||||
|
|
||||||
- Remove the `NODE_ICU_DATA` environment variable that may cause conflicts with node within the runner. (#1060)
|
- Temporary fix for macOS runner upgrade crash loop. (#1304)
|
||||||
- Handle cancelled jobs better to prevent orphaned processes (#1083)
|
- Fixed an issue where GHES runners fail to download public docker images (#1199)
|
||||||
- No longer fail to remove a `systemd` service with `svc.sh uninstall` if the script had previously been run from the wrong location (#1135)
|
|
||||||
- Send `SIGKILL` to the runner listener if it doesn't respond to `SIGINT` for 30 seconds
|
|
||||||
- Match runner group name when configuring even if there's only a single runner group
|
|
||||||
|
|
||||||
|
|
||||||
## Misc
|
## Misc
|
||||||
- Fix automation links in documentation (#1089)
|
|
||||||
- Improve developer and first contributor experience by improving tooling for VS Code (#1101, #1117, #1119, #1132)
|
|
||||||
- Fix bug where linux users are not able to run remove-svc.sh as root (#1127)
|
|
||||||
|
|
||||||
|
- Update error to say 'uninstall' not 'unconfigure' (#1179)
|
||||||
|
- Typo fixed (#1289)
|
||||||
|
|
||||||
## Windows x64
|
## Windows x64
|
||||||
We recommend configuring the runner in a root folder of the Windows drive (e.g. "C:\actions-runner"). This will help avoid issues related to service identity folder permissions and long file path restrictions on Windows.
|
We recommend configuring the runner in a root folder of the Windows drive (e.g. "C:\actions-runner"). This will help avoid issues related to service identity folder permissions and long file path restrictions on Windows.
|
||||||
|
|||||||
@@ -1 +1 @@
|
|||||||
<Update to ./src/runnerversion when creating release>
|
2.281.1
|
||||||
|
|||||||
@@ -2,36 +2,68 @@
|
|||||||
|
|
||||||
set -e
|
set -e
|
||||||
|
|
||||||
#
|
|
||||||
# Downloads latest releases (not pre-release) runner
|
|
||||||
# Configures as a service
|
|
||||||
#
|
|
||||||
# Examples:
|
|
||||||
# RUNNER_CFG_PAT=<yourPAT> ./create-latest-svc.sh myuser/myrepo my.ghe.deployment.net
|
|
||||||
# RUNNER_CFG_PAT=<yourPAT> ./create-latest-svc.sh myorg my.ghe.deployment.net
|
|
||||||
#
|
|
||||||
# Usage:
|
|
||||||
# export RUNNER_CFG_PAT=<yourPAT>
|
|
||||||
# ./create-latest-svc scope [ghe_domain] [name] [user] [labels]
|
|
||||||
#
|
|
||||||
# scope required repo (:owner/:repo) or org (:organization)
|
|
||||||
# ghe_domain optional the fully qualified domain name of your GitHub Enterprise Server deployment
|
|
||||||
# name optional defaults to hostname
|
|
||||||
# user optional user svc will run as. defaults to current
|
|
||||||
# labels optional list of labels (split by comma) applied on the runner
|
|
||||||
#
|
|
||||||
# Notes:
|
# Notes:
|
||||||
# PATS over envvars are more secure
|
# PATS over envvars are more secure
|
||||||
|
# Downloads latest runner release (not pre-release)
|
||||||
|
# Configures it as a service more secure
|
||||||
# Should be used on VMs and not containers
|
# Should be used on VMs and not containers
|
||||||
# Works on OSX and Linux
|
# Works on OSX and Linux
|
||||||
# Assumes x64 arch
|
# Assumes x64 arch
|
||||||
#
|
# See EXAMPLES below
|
||||||
|
|
||||||
runner_scope=${1}
|
flags_found=false
|
||||||
ghe_hostname=${2}
|
|
||||||
runner_name=${3:-$(hostname)}
|
while getopts 's:g:n:u:l:' opt; do
|
||||||
svc_user=${4:-$USER}
|
flags_found=true
|
||||||
labels=${5}
|
|
||||||
|
case $opt in
|
||||||
|
s)
|
||||||
|
runner_scope=$OPTARG
|
||||||
|
;;
|
||||||
|
g)
|
||||||
|
ghe_hostname=$OPTARG
|
||||||
|
;;
|
||||||
|
n)
|
||||||
|
runner_name=$OPTARG
|
||||||
|
;;
|
||||||
|
u)
|
||||||
|
svc_user=$OPTARG
|
||||||
|
;;
|
||||||
|
l)
|
||||||
|
labels=$OPTARG
|
||||||
|
;;
|
||||||
|
*)
|
||||||
|
echo "
|
||||||
|
Runner Service Installer
|
||||||
|
Examples:
|
||||||
|
RUNNER_CFG_PAT=<yourPAT> ./create-latest-svc.sh myuser/myrepo my.ghe.deployment.net
|
||||||
|
RUNNER_CFG_PAT=<yourPAT> ./create-latest-svc.sh -s myorg -u user_name -l label1,label2
|
||||||
|
Usage:
|
||||||
|
export RUNNER_CFG_PAT=<yourPAT>
|
||||||
|
./create-latest-svc scope [ghe_domain] [name] [user] [labels]
|
||||||
|
-s required scope: repo (:owner/:repo) or org (:organization)
|
||||||
|
-g optional ghe_hostname: the fully qualified domain name of your GitHub Enterprise Server deployment
|
||||||
|
-n optional name of the runner, defaults to hostname
|
||||||
|
-u optional user svc will run as, defaults to current
|
||||||
|
-l optional list of labels (split by comma) applied on the runner"
|
||||||
|
exit 0
|
||||||
|
;;
|
||||||
|
esac
|
||||||
|
done
|
||||||
|
|
||||||
|
shift "$((OPTIND - 1))"
|
||||||
|
|
||||||
|
if ! "$flags_found"; then
|
||||||
|
runner_scope=${1}
|
||||||
|
ghe_hostname=${2}
|
||||||
|
runner_name=${3:-$(hostname)}
|
||||||
|
svc_user=${4:-$USER}
|
||||||
|
labels=${5}
|
||||||
|
fi
|
||||||
|
|
||||||
|
# apply defaults
|
||||||
|
runner_name=${runner_name:-$(hostname)}
|
||||||
|
svc_user=${svc_user:-$USER}
|
||||||
|
|
||||||
echo "Configuring runner @ ${runner_scope}"
|
echo "Configuring runner @ ${runner_scope}"
|
||||||
sudo echo
|
sudo echo
|
||||||
@@ -142,7 +174,7 @@ echo
|
|||||||
echo "Configuring as a service ..."
|
echo "Configuring as a service ..."
|
||||||
prefix=""
|
prefix=""
|
||||||
if [ "${runner_plat}" == "linux" ]; then
|
if [ "${runner_plat}" == "linux" ]; then
|
||||||
prefix="sudo "
|
prefix="sudo "
|
||||||
fi
|
fi
|
||||||
|
|
||||||
${prefix}./svc.sh install ${svc_user}
|
${prefix}./svc.sh install ${svc_user}
|
||||||
|
|||||||
18
src/Misc/expressionFunc/hashFiles/package-lock.json
generated
18
src/Misc/expressionFunc/hashFiles/package-lock.json
generated
@@ -1291,9 +1291,9 @@
|
|||||||
}
|
}
|
||||||
},
|
},
|
||||||
"glob-parent": {
|
"glob-parent": {
|
||||||
"version": "5.1.1",
|
"version": "5.1.2",
|
||||||
"resolved": "https://registry.npmjs.org/glob-parent/-/glob-parent-5.1.1.tgz",
|
"resolved": "https://registry.npmjs.org/glob-parent/-/glob-parent-5.1.2.tgz",
|
||||||
"integrity": "sha512-FnI+VGOpnlGHWZxthPGR+QhR78fuiK0sNLkHQv+bL9fQi57lNNdquIbna/WrfROrolq8GK5Ek6BiMwqL/voRYQ==",
|
"integrity": "sha512-AOIgSQCepiJYwP3ARnGx+5VnTu2HBYdzbGP45eLw1vr3zB3vZLeyed1sC9hnbcOc9/SrMyM5RPQrkGz4aS9Zow==",
|
||||||
"dev": true,
|
"dev": true,
|
||||||
"requires": {
|
"requires": {
|
||||||
"is-glob": "^4.0.1"
|
"is-glob": "^4.0.1"
|
||||||
@@ -1374,9 +1374,9 @@
|
|||||||
"dev": true
|
"dev": true
|
||||||
},
|
},
|
||||||
"hosted-git-info": {
|
"hosted-git-info": {
|
||||||
"version": "2.8.8",
|
"version": "2.8.9",
|
||||||
"resolved": "https://registry.npmjs.org/hosted-git-info/-/hosted-git-info-2.8.8.tgz",
|
"resolved": "https://registry.npmjs.org/hosted-git-info/-/hosted-git-info-2.8.9.tgz",
|
||||||
"integrity": "sha512-f/wzC2QaWBs7t9IYqB4T3sR1xviIViXJRJTWBlx2Gf3g0Xi5vI7Yy4koXQ1c9OYDGHN9sBy1DQ2AB8fqZBWhUg==",
|
"integrity": "sha512-mxIDAb9Lsm6DoOJ7xH+5+X4y1LU/4Hi50L9C5sIswK3JzULS4bwk1FvjdBgvYR4bzT4tuUQiC15FE2f5HbLvYw==",
|
||||||
"dev": true
|
"dev": true
|
||||||
},
|
},
|
||||||
"iconv-lite": {
|
"iconv-lite": {
|
||||||
@@ -1683,9 +1683,9 @@
|
|||||||
}
|
}
|
||||||
},
|
},
|
||||||
"lodash": {
|
"lodash": {
|
||||||
"version": "4.17.19",
|
"version": "4.17.21",
|
||||||
"resolved": "https://registry.npmjs.org/lodash/-/lodash-4.17.19.tgz",
|
"resolved": "https://registry.npmjs.org/lodash/-/lodash-4.17.21.tgz",
|
||||||
"integrity": "sha512-JNvd8XER9GQX0v2qJgsaN/mzFCNA5BRe/j8JN9d+tWyGLSodKQHKFicdwNYzWwI3wjRnaKPsGj1XkBjx/F96DQ==",
|
"integrity": "sha512-v2kDEe57lecTulaDIuNTPy3Ry4gLGJ6Z1O3vE1krgXZNrsQ+LFTGHVxVjcXPs17LhbZVGedAJv8XZ1tvj5FvSg==",
|
||||||
"dev": true
|
"dev": true
|
||||||
},
|
},
|
||||||
"lodash.unescape": {
|
"lodash.unescape": {
|
||||||
|
|||||||
@@ -94,7 +94,6 @@ then
|
|||||||
fi
|
fi
|
||||||
}
|
}
|
||||||
|
|
||||||
# libssl version prefer: libssl1.1 -> libssl1.0.2 -> libssl1.0.0
|
|
||||||
apt_get_with_fallbacks libssl1.1$ libssl1.0.2$ libssl1.0.0$
|
apt_get_with_fallbacks libssl1.1$ libssl1.0.2$ libssl1.0.0$
|
||||||
if [ $? -ne 0 ]
|
if [ $? -ne 0 ]
|
||||||
then
|
then
|
||||||
@@ -103,8 +102,7 @@ then
|
|||||||
exit 1
|
exit 1
|
||||||
fi
|
fi
|
||||||
|
|
||||||
# libicu version prefer: libicu66 -> libicu63 -> libicu60 -> libicu57 -> libicu55 -> libicu52
|
apt_get_with_fallbacks libicu72 libicu71 libicu70 libicu69 libicu68 libicu67 libicu66 libicu65 libicu63 libicu60 libicu57 libicu55 libicu52
|
||||||
apt_get_with_fallbacks libicu66 libicu63 libicu60 libicu57 libicu55 libicu52
|
|
||||||
if [ $? -ne 0 ]
|
if [ $? -ne 0 ]
|
||||||
then
|
then
|
||||||
echo "'$apt_get' failed with exit code '$?'"
|
echo "'$apt_get' failed with exit code '$?'"
|
||||||
|
|||||||
@@ -118,6 +118,43 @@ then
|
|||||||
exit 1
|
exit 1
|
||||||
fi
|
fi
|
||||||
|
|
||||||
|
# fix upgrade issue with macOS
|
||||||
|
currentplatform=$(uname | awk '{print tolower($0)}')
|
||||||
|
if [[ "$currentplatform" == 'darwin' ]]; then
|
||||||
|
# need a short-term fix for https://github.com/actions/runner/issues/743
|
||||||
|
# we will recreate all the ./externals/node12/bin/node of the past 5 versions
|
||||||
|
# v2.280.3 v2.280.2 v2.280.1 v2.279.0 v2.278.0
|
||||||
|
if [[ ! -e "$rootfolder/externals.2.280.3/node12/bin/node" ]]
|
||||||
|
then
|
||||||
|
mkdir -p "$rootfolder/externals.2.280.3/node12/bin"
|
||||||
|
cp "$rootfolder/externals/node12/bin/node" "$rootfolder/externals.2.280.3/node12/bin/node"
|
||||||
|
fi
|
||||||
|
|
||||||
|
if [[ ! -e "$rootfolder/externals.2.280.2/node12/bin/node" ]]
|
||||||
|
then
|
||||||
|
mkdir -p "$rootfolder/externals.2.280.2/node12/bin"
|
||||||
|
cp "$rootfolder/externals/node12/bin/node" "$rootfolder/externals.2.280.2/node12/bin/node"
|
||||||
|
fi
|
||||||
|
|
||||||
|
if [[ ! -e "$rootfolder/externals.2.280.1/node12/bin/node" ]]
|
||||||
|
then
|
||||||
|
mkdir -p "$rootfolder/externals.2.280.1/node12/bin"
|
||||||
|
cp "$rootfolder/externals/node12/bin/node" "$rootfolder/externals.2.280.1/node12/bin/node"
|
||||||
|
fi
|
||||||
|
|
||||||
|
if [[ ! -e "$rootfolder/externals.2.279.0/node12/bin/node" ]]
|
||||||
|
then
|
||||||
|
mkdir -p "$rootfolder/externals.2.279.0/node12/bin"
|
||||||
|
cp "$rootfolder/externals/node12/bin/node" "$rootfolder/externals.2.279.0/node12/bin/node"
|
||||||
|
fi
|
||||||
|
|
||||||
|
if [[ ! -e "$rootfolder/externals.2.278.0/node12/bin/node" ]]
|
||||||
|
then
|
||||||
|
mkdir -p "$rootfolder/externals.2.278.0/node12/bin"
|
||||||
|
cp "$rootfolder/externals/node12/bin/node" "$rootfolder/externals.2.278.0/node12/bin/node"
|
||||||
|
fi
|
||||||
|
fi
|
||||||
|
|
||||||
date "+[%F %T-%4N] Update succeed" >> "$logfile"
|
date "+[%F %T-%4N] Update succeed" >> "$logfile"
|
||||||
|
|
||||||
# rename the update log file with %logfile%.succeed/.failed/succeedneedrestart
|
# rename the update log file with %logfile%.succeed/.failed/succeedneedrestart
|
||||||
|
|||||||
@@ -90,6 +90,8 @@ namespace GitHub.Runner.Common
|
|||||||
this.SecretMasker.AddValueEncoder(ValueEncoders.UriDataEscape);
|
this.SecretMasker.AddValueEncoder(ValueEncoders.UriDataEscape);
|
||||||
this.SecretMasker.AddValueEncoder(ValueEncoders.XmlDataEscape);
|
this.SecretMasker.AddValueEncoder(ValueEncoders.XmlDataEscape);
|
||||||
this.SecretMasker.AddValueEncoder(ValueEncoders.TrimDoubleQuotes);
|
this.SecretMasker.AddValueEncoder(ValueEncoders.TrimDoubleQuotes);
|
||||||
|
this.SecretMasker.AddValueEncoder(ValueEncoders.PowerShellPreAmpersandEscape);
|
||||||
|
this.SecretMasker.AddValueEncoder(ValueEncoders.PowerShellPostAmpersandEscape);
|
||||||
|
|
||||||
// Create the trace manager.
|
// Create the trace manager.
|
||||||
if (string.IsNullOrEmpty(logFile))
|
if (string.IsNullOrEmpty(logFile))
|
||||||
|
|||||||
@@ -117,6 +117,7 @@ namespace GitHub.Runner.Listener.Configuration
|
|||||||
try
|
try
|
||||||
{
|
{
|
||||||
// Determine the service deployment type based on connection data. (Hosted/OnPremises)
|
// Determine the service deployment type based on connection data. (Hosted/OnPremises)
|
||||||
|
// Hosted usually means github.com or localhost, while OnPremises means GHES or GHAE
|
||||||
runnerSettings.IsHostedServer = runnerSettings.GitHubUrl == null || UrlUtil.IsHostedServer(new UriBuilder(runnerSettings.GitHubUrl));
|
runnerSettings.IsHostedServer = runnerSettings.GitHubUrl == null || UrlUtil.IsHostedServer(new UriBuilder(runnerSettings.GitHubUrl));
|
||||||
|
|
||||||
// Warn if the Actions server url and GHES server url has different Host
|
// Warn if the Actions server url and GHES server url has different Host
|
||||||
@@ -186,7 +187,7 @@ namespace GitHub.Runner.Listener.Configuration
|
|||||||
}
|
}
|
||||||
else
|
else
|
||||||
{
|
{
|
||||||
Trace.Info("Found a self-hosted runner group with id {1} and name {2}", agentPool.Id, agentPool.Name);
|
Trace.Info($"Found a self-hosted runner group with id {agentPool.Id} and name {agentPool.Name}");
|
||||||
runnerSettings.PoolId = agentPool.Id;
|
runnerSettings.PoolId = agentPool.Id;
|
||||||
runnerSettings.PoolName = agentPool.Name;
|
runnerSettings.PoolName = agentPool.Name;
|
||||||
}
|
}
|
||||||
@@ -346,12 +347,9 @@ namespace GitHub.Runner.Listener.Configuration
|
|||||||
|
|
||||||
_term.WriteLine();
|
_term.WriteLine();
|
||||||
_term.WriteSuccessMessage("Runner service removed");
|
_term.WriteSuccessMessage("Runner service removed");
|
||||||
#elif OS_LINUX
|
#else
|
||||||
// unconfig system D service first
|
// unconfig systemd or osx service first
|
||||||
throw new Exception("Unconfigure service first");
|
throw new Exception("Uninstall service first");
|
||||||
#elif OS_OSX
|
|
||||||
// unconfig osx service first
|
|
||||||
throw new Exception("Unconfigure service first");
|
|
||||||
#endif
|
#endif
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -507,7 +507,20 @@ namespace GitHub.Runner.Listener
|
|||||||
{
|
{
|
||||||
detailInfo = string.Join(Environment.NewLine, workerOutput);
|
detailInfo = string.Join(Environment.NewLine, workerOutput);
|
||||||
Trace.Info($"Return code {returnCode} indicate worker encounter an unhandled exception or app crash, attach worker stdout/stderr to JobRequest result.");
|
Trace.Info($"Return code {returnCode} indicate worker encounter an unhandled exception or app crash, attach worker stdout/stderr to JobRequest result.");
|
||||||
await LogWorkerProcessUnhandledException(message, detailInfo);
|
|
||||||
|
var jobServer = HostContext.GetService<IJobServer>();
|
||||||
|
VssCredentials jobServerCredential = VssUtil.GetVssCredential(systemConnection);
|
||||||
|
VssConnection jobConnection = VssUtil.CreateConnection(systemConnection.Url, jobServerCredential);
|
||||||
|
await jobServer.ConnectAsync(jobConnection);
|
||||||
|
|
||||||
|
await LogWorkerProcessUnhandledException(jobServer, message, detailInfo);
|
||||||
|
|
||||||
|
// Go ahead to finish the job with result 'Failed' if the STDERR from worker is System.IO.IOException, since it typically means we are running out of disk space.
|
||||||
|
if (detailInfo.Contains(typeof(System.IO.IOException).ToString(), StringComparison.OrdinalIgnoreCase))
|
||||||
|
{
|
||||||
|
Trace.Info($"Finish job with result 'Failed' due to IOException.");
|
||||||
|
await ForceFailJob(jobServer, message);
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
TaskResult result = TaskResultUtil.TranslateFromReturnCode(returnCode);
|
TaskResult result = TaskResultUtil.TranslateFromReturnCode(returnCode);
|
||||||
@@ -915,53 +928,16 @@ namespace GitHub.Runner.Listener
|
|||||||
}
|
}
|
||||||
|
|
||||||
// log an error issue to job level timeline record
|
// log an error issue to job level timeline record
|
||||||
private async Task LogWorkerProcessUnhandledException(Pipelines.AgentJobRequestMessage message, string errorMessage)
|
private async Task LogWorkerProcessUnhandledException(IJobServer jobServer, Pipelines.AgentJobRequestMessage message, string errorMessage)
|
||||||
{
|
{
|
||||||
try
|
try
|
||||||
{
|
{
|
||||||
var systemConnection = message.Resources.Endpoints.SingleOrDefault(x => string.Equals(x.Name, WellKnownServiceEndpointNames.SystemVssConnection));
|
|
||||||
ArgUtil.NotNull(systemConnection, nameof(systemConnection));
|
|
||||||
|
|
||||||
var jobServer = HostContext.GetService<IJobServer>();
|
|
||||||
VssCredentials jobServerCredential = VssUtil.GetVssCredential(systemConnection);
|
|
||||||
VssConnection jobConnection = VssUtil.CreateConnection(systemConnection.Url, jobServerCredential);
|
|
||||||
|
|
||||||
/* Below is the legacy 'OnPremises' code that is currently unused by the runner
|
|
||||||
ToDo: re-implement code as appropriate once GHES support is added.
|
|
||||||
// Make sure SystemConnection Url match Config Url base for OnPremises server
|
|
||||||
if (!message.Variables.ContainsKey(Constants.Variables.System.ServerType) ||
|
|
||||||
string.Equals(message.Variables[Constants.Variables.System.ServerType]?.Value, "OnPremises", StringComparison.OrdinalIgnoreCase))
|
|
||||||
{
|
|
||||||
try
|
|
||||||
{
|
|
||||||
Uri result = null;
|
|
||||||
Uri configUri = new Uri(_runnerSetting.ServerUrl);
|
|
||||||
if (Uri.TryCreate(new Uri(configUri.GetComponents(UriComponents.SchemeAndServer, UriFormat.Unescaped)), jobServerUrl.PathAndQuery, out result))
|
|
||||||
{
|
|
||||||
//replace the schema and host portion of messageUri with the host from the
|
|
||||||
//server URI (which was set at config time)
|
|
||||||
jobServerUrl = result;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
catch (InvalidOperationException ex)
|
|
||||||
{
|
|
||||||
//cannot parse the Uri - not a fatal error
|
|
||||||
Trace.Error(ex);
|
|
||||||
}
|
|
||||||
catch (UriFormatException ex)
|
|
||||||
{
|
|
||||||
//cannot parse the Uri - not a fatal error
|
|
||||||
Trace.Error(ex);
|
|
||||||
}
|
|
||||||
} */
|
|
||||||
|
|
||||||
await jobServer.ConnectAsync(jobConnection);
|
|
||||||
|
|
||||||
var timeline = await jobServer.GetTimelineAsync(message.Plan.ScopeIdentifier, message.Plan.PlanType, message.Plan.PlanId, message.Timeline.Id, CancellationToken.None);
|
var timeline = await jobServer.GetTimelineAsync(message.Plan.ScopeIdentifier, message.Plan.PlanType, message.Plan.PlanId, message.Timeline.Id, CancellationToken.None);
|
||||||
|
|
||||||
ArgUtil.NotNull(timeline, nameof(timeline));
|
ArgUtil.NotNull(timeline, nameof(timeline));
|
||||||
|
|
||||||
TimelineRecord jobRecord = timeline.Records.FirstOrDefault(x => x.Id == message.JobId && x.RecordType == "Job");
|
TimelineRecord jobRecord = timeline.Records.FirstOrDefault(x => x.Id == message.JobId && x.RecordType == "Job");
|
||||||
ArgUtil.NotNull(jobRecord, nameof(jobRecord));
|
ArgUtil.NotNull(jobRecord, nameof(jobRecord));
|
||||||
|
|
||||||
var unhandledExceptionIssue = new Issue() { Type = IssueType.Error, Message = errorMessage };
|
var unhandledExceptionIssue = new Issue() { Type = IssueType.Error, Message = errorMessage };
|
||||||
unhandledExceptionIssue.Data[Constants.Runner.InternalTelemetryIssueDataKey] = Constants.Runner.WorkerCrash;
|
unhandledExceptionIssue.Data[Constants.Runner.InternalTelemetryIssueDataKey] = Constants.Runner.WorkerCrash;
|
||||||
jobRecord.ErrorCount++;
|
jobRecord.ErrorCount++;
|
||||||
@@ -975,6 +951,21 @@ namespace GitHub.Runner.Listener
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// raise job completed event to fail the job.
|
||||||
|
private async Task ForceFailJob(IJobServer jobServer, Pipelines.AgentJobRequestMessage message)
|
||||||
|
{
|
||||||
|
try
|
||||||
|
{
|
||||||
|
var jobCompletedEvent = new JobCompletedEvent(message.RequestId, message.JobId, TaskResult.Failed);
|
||||||
|
await jobServer.RaisePlanEventAsync<JobCompletedEvent>(message.Plan.ScopeIdentifier, message.Plan.PlanType, message.Plan.PlanId, jobCompletedEvent, CancellationToken.None);
|
||||||
|
}
|
||||||
|
catch (Exception ex)
|
||||||
|
{
|
||||||
|
Trace.Error("Fail to raise JobCompletedEvent back to service.");
|
||||||
|
Trace.Error(ex);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
private class WorkerDispatcher : IDisposable
|
private class WorkerDispatcher : IDisposable
|
||||||
{
|
{
|
||||||
public long RequestId { get; }
|
public long RequestId { get; }
|
||||||
|
|||||||
@@ -74,10 +74,12 @@ namespace GitHub.Runner.Listener
|
|||||||
await jobDispatcher.WaitAsync(token);
|
await jobDispatcher.WaitAsync(token);
|
||||||
Trace.Info($"All running job has exited.");
|
Trace.Info($"All running job has exited.");
|
||||||
|
|
||||||
|
// We need to keep runner backup around for macOS until we fixed https://github.com/actions/runner/issues/743
|
||||||
|
#if !OS_OSX
|
||||||
// delete runner backup
|
// delete runner backup
|
||||||
DeletePreviousVersionRunnerBackup(token);
|
DeletePreviousVersionRunnerBackup(token);
|
||||||
Trace.Info($"Delete old version runner backup.");
|
Trace.Info($"Delete old version runner backup.");
|
||||||
|
#endif
|
||||||
// generate update script from template
|
// generate update script from template
|
||||||
await UpdateRunnerUpdateStateAsync("Generate and execute update script.");
|
await UpdateRunnerUpdateStateAsync("Generate and execute update script.");
|
||||||
|
|
||||||
@@ -96,7 +98,7 @@ namespace GitHub.Runner.Listener
|
|||||||
invokeScript.Start();
|
invokeScript.Start();
|
||||||
Trace.Info($"Update script start running");
|
Trace.Info($"Update script start running");
|
||||||
|
|
||||||
await UpdateRunnerUpdateStateAsync("Runner will exit shortly for update, should back online within 10 seconds.");
|
await UpdateRunnerUpdateStateAsync("Runner will exit shortly for update, should be back online within 10 seconds.");
|
||||||
|
|
||||||
return true;
|
return true;
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -295,8 +295,21 @@ namespace GitHub.Runner.Worker
|
|||||||
{
|
{
|
||||||
throw new Exception("Required field 'name' is missing in ##[save-state] command.");
|
throw new Exception("Required field 'name' is missing in ##[save-state] command.");
|
||||||
}
|
}
|
||||||
|
// Embedded steps (composite) keep track of the state at the root level
|
||||||
context.IntraActionState[stateName] = command.Data;
|
if (context.IsEmbedded)
|
||||||
|
{
|
||||||
|
var id = context.EmbeddedId;
|
||||||
|
if (!context.Root.EmbeddedIntraActionState.ContainsKey(id))
|
||||||
|
{
|
||||||
|
context.Root.EmbeddedIntraActionState[id] = new Dictionary<string, string>(StringComparer.OrdinalIgnoreCase);
|
||||||
|
}
|
||||||
|
context.Root.EmbeddedIntraActionState[id][stateName] = command.Data;
|
||||||
|
}
|
||||||
|
// Otherwise modify the ExecutionContext
|
||||||
|
else
|
||||||
|
{
|
||||||
|
context.IntraActionState[stateName] = command.Data;
|
||||||
|
}
|
||||||
context.Debug($"Save intra-action state {stateName} = {command.Data}");
|
context.Debug($"Save intra-action state {stateName} = {command.Data}");
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -37,7 +37,10 @@ namespace GitHub.Runner.Worker
|
|||||||
public interface IActionManager : IRunnerService
|
public interface IActionManager : IRunnerService
|
||||||
{
|
{
|
||||||
Dictionary<Guid, ContainerInfo> CachedActionContainers { get; }
|
Dictionary<Guid, ContainerInfo> CachedActionContainers { get; }
|
||||||
Task<PrepareResult> PrepareActionsAsync(IExecutionContext executionContext, IEnumerable<Pipelines.JobStep> steps);
|
Dictionary<Guid, List<Pipelines.ActionStep>> CachedEmbeddedPreSteps { get; }
|
||||||
|
Dictionary<Guid, List<Guid>> CachedEmbeddedStepIds { get; }
|
||||||
|
Dictionary<Guid, Stack<Pipelines.ActionStep>> CachedEmbeddedPostSteps { get; }
|
||||||
|
Task<PrepareResult> PrepareActionsAsync(IExecutionContext executionContext, IEnumerable<Pipelines.JobStep> steps, Guid rootStepId = default(Guid));
|
||||||
Definition LoadAction(IExecutionContext executionContext, Pipelines.ActionStep action);
|
Definition LoadAction(IExecutionContext executionContext, Pipelines.ActionStep action);
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -48,10 +51,20 @@ namespace GitHub.Runner.Worker
|
|||||||
//81920 is the default used by System.IO.Stream.CopyTo and is under the large object heap threshold (85k).
|
//81920 is the default used by System.IO.Stream.CopyTo and is under the large object heap threshold (85k).
|
||||||
private const int _defaultCopyBufferSize = 81920;
|
private const int _defaultCopyBufferSize = 81920;
|
||||||
private const string _dotcomApiUrl = "https://api.github.com";
|
private const string _dotcomApiUrl = "https://api.github.com";
|
||||||
private readonly Dictionary<Guid, ContainerInfo> _cachedActionContainers = new Dictionary<Guid, ContainerInfo>();
|
|
||||||
|
|
||||||
|
private readonly Dictionary<Guid, ContainerInfo> _cachedActionContainers = new Dictionary<Guid, ContainerInfo>();
|
||||||
public Dictionary<Guid, ContainerInfo> CachedActionContainers => _cachedActionContainers;
|
public Dictionary<Guid, ContainerInfo> CachedActionContainers => _cachedActionContainers;
|
||||||
public async Task<PrepareResult> PrepareActionsAsync(IExecutionContext executionContext, IEnumerable<Pipelines.JobStep> steps)
|
|
||||||
|
private readonly Dictionary<Guid, List<Pipelines.ActionStep>> _cachedEmbeddedPreSteps = new Dictionary<Guid, List<Pipelines.ActionStep>>();
|
||||||
|
public Dictionary<Guid, List<Pipelines.ActionStep>> CachedEmbeddedPreSteps => _cachedEmbeddedPreSteps;
|
||||||
|
|
||||||
|
private readonly Dictionary<Guid, List<Guid>> _cachedEmbeddedStepIds = new Dictionary<Guid, List<Guid>>();
|
||||||
|
public Dictionary<Guid, List<Guid>> CachedEmbeddedStepIds => _cachedEmbeddedStepIds;
|
||||||
|
|
||||||
|
private readonly Dictionary<Guid, Stack<Pipelines.ActionStep>> _cachedEmbeddedPostSteps = new Dictionary<Guid, Stack<Pipelines.ActionStep>>();
|
||||||
|
public Dictionary<Guid, Stack<Pipelines.ActionStep>> CachedEmbeddedPostSteps => _cachedEmbeddedPostSteps;
|
||||||
|
|
||||||
|
public async Task<PrepareResult> PrepareActionsAsync(IExecutionContext executionContext, IEnumerable<Pipelines.JobStep> steps, Guid rootStepId = default(Guid))
|
||||||
{
|
{
|
||||||
// Assert inputs
|
// Assert inputs
|
||||||
ArgUtil.NotNull(executionContext, nameof(executionContext));
|
ArgUtil.NotNull(executionContext, nameof(executionContext));
|
||||||
@@ -64,10 +77,30 @@ namespace GitHub.Runner.Worker
|
|||||||
PreStepTracker = new Dictionary<Guid, IActionRunner>()
|
PreStepTracker = new Dictionary<Guid, IActionRunner>()
|
||||||
};
|
};
|
||||||
var containerSetupSteps = new List<JobExtensionRunner>();
|
var containerSetupSteps = new List<JobExtensionRunner>();
|
||||||
IOUtil.DeleteDirectory(HostContext.GetDirectory(WellKnownDirectory.Actions), executionContext.CancellationToken);
|
var depth = 0;
|
||||||
|
// We are running at the start of a job
|
||||||
|
if (rootStepId == default(Guid))
|
||||||
|
{
|
||||||
|
IOUtil.DeleteDirectory(HostContext.GetDirectory(WellKnownDirectory.Actions), executionContext.CancellationToken);
|
||||||
|
}
|
||||||
|
// We are running mid job due to a local composite action
|
||||||
|
else
|
||||||
|
{
|
||||||
|
if (!_cachedEmbeddedStepIds.ContainsKey(rootStepId))
|
||||||
|
{
|
||||||
|
_cachedEmbeddedStepIds[rootStepId] = new List<Guid>();
|
||||||
|
foreach (var compositeStep in steps)
|
||||||
|
{
|
||||||
|
var guid = Guid.NewGuid();
|
||||||
|
compositeStep.Id = guid;
|
||||||
|
_cachedEmbeddedStepIds[rootStepId].Add(guid);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
depth = 1;
|
||||||
|
}
|
||||||
IEnumerable<Pipelines.ActionStep> actions = steps.OfType<Pipelines.ActionStep>();
|
IEnumerable<Pipelines.ActionStep> actions = steps.OfType<Pipelines.ActionStep>();
|
||||||
executionContext.Output("Prepare all required actions");
|
executionContext.Output("Prepare all required actions");
|
||||||
var result = await PrepareActionsRecursiveAsync(executionContext, state, actions, 0);
|
var result = await PrepareActionsRecursiveAsync(executionContext, state, actions, depth, rootStepId);
|
||||||
if (state.ImagesToPull.Count > 0)
|
if (state.ImagesToPull.Count > 0)
|
||||||
{
|
{
|
||||||
foreach (var imageToPull in result.ImagesToPull)
|
foreach (var imageToPull in result.ImagesToPull)
|
||||||
@@ -103,7 +136,7 @@ namespace GitHub.Runner.Worker
|
|||||||
return new PrepareResult(containerSetupSteps, result.PreStepTracker);
|
return new PrepareResult(containerSetupSteps, result.PreStepTracker);
|
||||||
}
|
}
|
||||||
|
|
||||||
private async Task<PrepareActionsState> PrepareActionsRecursiveAsync(IExecutionContext executionContext, PrepareActionsState state, IEnumerable<Pipelines.ActionStep> actions, Int32 depth = 0)
|
private async Task<PrepareActionsState> PrepareActionsRecursiveAsync(IExecutionContext executionContext, PrepareActionsState state, IEnumerable<Pipelines.ActionStep> actions, Int32 depth = 0, Guid parentStepId = default(Guid))
|
||||||
{
|
{
|
||||||
ArgUtil.NotNull(executionContext, nameof(executionContext));
|
ArgUtil.NotNull(executionContext, nameof(executionContext));
|
||||||
if (depth > Constants.CompositeActionsMaxDepth)
|
if (depth > Constants.CompositeActionsMaxDepth)
|
||||||
@@ -187,24 +220,51 @@ namespace GitHub.Runner.Worker
|
|||||||
state.ImagesToBuildInfo[setupInfo.Container.ActionRepository] = setupInfo.Container;
|
state.ImagesToBuildInfo[setupInfo.Container.ActionRepository] = setupInfo.Container;
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
else if(setupInfo != null && setupInfo.Steps != null && setupInfo.Steps.Count > 0)
|
else if (setupInfo != null && setupInfo.Steps != null && setupInfo.Steps.Count > 0)
|
||||||
{
|
{
|
||||||
state = await PrepareActionsRecursiveAsync(executionContext, state, setupInfo.Steps, depth + 1);
|
state = await PrepareActionsRecursiveAsync(executionContext, state, setupInfo.Steps, depth + 1, action.Id);
|
||||||
}
|
}
|
||||||
var repoAction = action.Reference as Pipelines.RepositoryPathReference;
|
var repoAction = action.Reference as Pipelines.RepositoryPathReference;
|
||||||
if (repoAction.RepositoryType != Pipelines.PipelineConstants.SelfAlias)
|
if (repoAction.RepositoryType != Pipelines.PipelineConstants.SelfAlias)
|
||||||
{
|
{
|
||||||
var definition = LoadAction(executionContext, action);
|
var definition = LoadAction(executionContext, action);
|
||||||
// TODO: Support pre's in composite actions
|
if (definition.Data.Execution.HasPre)
|
||||||
if (definition.Data.Execution.HasPre && depth < 1)
|
|
||||||
{
|
{
|
||||||
var actionRunner = HostContext.CreateService<IActionRunner>();
|
|
||||||
actionRunner.Action = action;
|
|
||||||
actionRunner.Stage = ActionRunStage.Pre;
|
|
||||||
actionRunner.Condition = definition.Data.Execution.InitCondition;
|
|
||||||
|
|
||||||
Trace.Info($"Add 'pre' execution for {action.Id}");
|
Trace.Info($"Add 'pre' execution for {action.Id}");
|
||||||
state.PreStepTracker[action.Id] = actionRunner;
|
// Root Step
|
||||||
|
if (depth < 1)
|
||||||
|
{
|
||||||
|
var actionRunner = HostContext.CreateService<IActionRunner>();
|
||||||
|
actionRunner.Action = action;
|
||||||
|
actionRunner.Stage = ActionRunStage.Pre;
|
||||||
|
actionRunner.Condition = definition.Data.Execution.InitCondition;
|
||||||
|
state.PreStepTracker[action.Id] = actionRunner;
|
||||||
|
}
|
||||||
|
// Embedded Step
|
||||||
|
else
|
||||||
|
{
|
||||||
|
if (!_cachedEmbeddedPreSteps.ContainsKey(parentStepId))
|
||||||
|
{
|
||||||
|
_cachedEmbeddedPreSteps[parentStepId] = new List<Pipelines.ActionStep>();
|
||||||
|
}
|
||||||
|
// Clone action so we can modify the condition without affecting the original
|
||||||
|
var clonedAction = action.Clone() as Pipelines.ActionStep;
|
||||||
|
clonedAction.Condition = definition.Data.Execution.InitCondition;
|
||||||
|
_cachedEmbeddedPreSteps[parentStepId].Add(clonedAction);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
if (definition.Data.Execution.HasPost && depth > 0)
|
||||||
|
{
|
||||||
|
if (!_cachedEmbeddedPostSteps.ContainsKey(parentStepId))
|
||||||
|
{
|
||||||
|
// If we haven't done so already, add the parent to the post steps
|
||||||
|
_cachedEmbeddedPostSteps[parentStepId] = new Stack<Pipelines.ActionStep>();
|
||||||
|
}
|
||||||
|
// Clone action so we can modify the condition without affecting the original
|
||||||
|
var clonedAction = action.Clone() as Pipelines.ActionStep;
|
||||||
|
clonedAction.Condition = definition.Data.Execution.CleanupCondition;
|
||||||
|
_cachedEmbeddedPostSteps[parentStepId].Push(clonedAction);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -355,6 +415,29 @@ namespace GitHub.Runner.Worker
|
|||||||
Trace.Verbose($"Details: {StringUtil.ConvertToJson(compositeAction?.Steps)}");
|
Trace.Verbose($"Details: {StringUtil.ConvertToJson(compositeAction?.Steps)}");
|
||||||
Trace.Info($"Load: {compositeAction.Outputs?.Count ?? 0} number of outputs");
|
Trace.Info($"Load: {compositeAction.Outputs?.Count ?? 0} number of outputs");
|
||||||
Trace.Info($"Details: {StringUtil.ConvertToJson(compositeAction?.Outputs)}");
|
Trace.Info($"Details: {StringUtil.ConvertToJson(compositeAction?.Outputs)}");
|
||||||
|
|
||||||
|
if (CachedEmbeddedPreSteps.TryGetValue(action.Id, out var preSteps))
|
||||||
|
{
|
||||||
|
compositeAction.PreSteps = preSteps;
|
||||||
|
}
|
||||||
|
|
||||||
|
if (CachedEmbeddedPostSteps.TryGetValue(action.Id, out var postSteps))
|
||||||
|
{
|
||||||
|
compositeAction.PostSteps = postSteps;
|
||||||
|
}
|
||||||
|
|
||||||
|
if (_cachedEmbeddedStepIds.ContainsKey(action.Id))
|
||||||
|
{
|
||||||
|
for (var i = 0; i < compositeAction.Steps.Count; i++)
|
||||||
|
{
|
||||||
|
// Store Id's for later load actions
|
||||||
|
compositeAction.Steps[i].Id = _cachedEmbeddedStepIds[action.Id][i];
|
||||||
|
if (string.IsNullOrEmpty(executionContext.Global.Variables.Get("DistributedTask.EnableCompositeActions")) && compositeAction.Steps[i].Reference.Type != Pipelines.ActionSourceType.Script)
|
||||||
|
{
|
||||||
|
throw new Exception("`uses:` keyword is not currently supported.");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
}
|
}
|
||||||
else
|
else
|
||||||
{
|
{
|
||||||
@@ -527,6 +610,7 @@ namespace GitHub.Runner.Worker
|
|||||||
{
|
{
|
||||||
NameWithOwner = repositoryReference.Name,
|
NameWithOwner = repositoryReference.Name,
|
||||||
Ref = repositoryReference.Ref,
|
Ref = repositoryReference.Ref,
|
||||||
|
Path = repositoryReference.Path,
|
||||||
};
|
};
|
||||||
})
|
})
|
||||||
.ToList();
|
.ToList();
|
||||||
@@ -620,7 +704,7 @@ namespace GitHub.Runner.Worker
|
|||||||
// make sure we get a clean folder ready to use.
|
// make sure we get a clean folder ready to use.
|
||||||
IOUtil.DeleteDirectory(destDirectory, executionContext.CancellationToken);
|
IOUtil.DeleteDirectory(destDirectory, executionContext.CancellationToken);
|
||||||
Directory.CreateDirectory(destDirectory);
|
Directory.CreateDirectory(destDirectory);
|
||||||
executionContext.Output($"Download action repository '{downloadInfo.NameWithOwner}@{downloadInfo.Ref}'");
|
executionContext.Output($"Download action repository '{downloadInfo.NameWithOwner}@{downloadInfo.Ref}' (SHA:{downloadInfo.ResolvedSha})");
|
||||||
}
|
}
|
||||||
|
|
||||||
await DownloadRepositoryActionAsync(executionContext, downloadInfo, destDirectory);
|
await DownloadRepositoryActionAsync(executionContext, downloadInfo, destDirectory);
|
||||||
@@ -928,20 +1012,30 @@ namespace GitHub.Runner.Worker
|
|||||||
}
|
}
|
||||||
else if (actionDefinitionData.Execution.ExecutionType == ActionExecutionType.Composite)
|
else if (actionDefinitionData.Execution.ExecutionType == ActionExecutionType.Composite)
|
||||||
{
|
{
|
||||||
// TODO: we need to generate unique Id's for composite steps
|
|
||||||
Trace.Info($"Loading Composite steps");
|
Trace.Info($"Loading Composite steps");
|
||||||
var compositeAction = actionDefinitionData.Execution as CompositeActionExecutionData;
|
var compositeAction = actionDefinitionData.Execution as CompositeActionExecutionData;
|
||||||
setupInfo.Steps = compositeAction.Steps;
|
setupInfo.Steps = compositeAction.Steps;
|
||||||
|
|
||||||
|
// cache steps ids if not done so already
|
||||||
|
if (!_cachedEmbeddedStepIds.ContainsKey(repositoryAction.Id))
|
||||||
|
{
|
||||||
|
_cachedEmbeddedStepIds[repositoryAction.Id] = new List<Guid>();
|
||||||
|
foreach (var compositeStep in compositeAction.Steps)
|
||||||
|
{
|
||||||
|
var guid = Guid.NewGuid();
|
||||||
|
compositeStep.Id = guid;
|
||||||
|
_cachedEmbeddedStepIds[repositoryAction.Id].Add(guid);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TODO: remove once we remove the DistributedTask.EnableCompositeActions FF
|
||||||
foreach (var step in compositeAction.Steps)
|
foreach (var step in compositeAction.Steps)
|
||||||
{
|
{
|
||||||
step.Id = Guid.NewGuid();
|
|
||||||
if (string.IsNullOrEmpty(executionContext.Global.Variables.Get("DistributedTask.EnableCompositeActions")) && step.Reference.Type != Pipelines.ActionSourceType.Script)
|
if (string.IsNullOrEmpty(executionContext.Global.Variables.Get("DistributedTask.EnableCompositeActions")) && step.Reference.Type != Pipelines.ActionSourceType.Script)
|
||||||
{
|
{
|
||||||
throw new Exception("`uses:` keyword is not currently supported.");
|
throw new Exception("`uses:` keyword is not currently supported.");
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
return setupInfo;
|
return setupInfo;
|
||||||
}
|
}
|
||||||
else
|
else
|
||||||
@@ -1102,9 +1196,11 @@ namespace GitHub.Runner.Worker
|
|||||||
public sealed class CompositeActionExecutionData : ActionExecutionData
|
public sealed class CompositeActionExecutionData : ActionExecutionData
|
||||||
{
|
{
|
||||||
public override ActionExecutionType ExecutionType => ActionExecutionType.Composite;
|
public override ActionExecutionType ExecutionType => ActionExecutionType.Composite;
|
||||||
public override bool HasPre => false;
|
public override bool HasPre => PreSteps.Count > 0;
|
||||||
public override bool HasPost => false;
|
public override bool HasPost => PostSteps.Count > 0;
|
||||||
|
public List<Pipelines.ActionStep> PreSteps { get; set; }
|
||||||
public List<Pipelines.ActionStep> Steps { get; set; }
|
public List<Pipelines.ActionStep> Steps { get; set; }
|
||||||
|
public Stack<Pipelines.ActionStep> PostSteps { get; set; }
|
||||||
public MappingToken Outputs { get; set; }
|
public MappingToken Outputs { get; set; }
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -1168,7 +1264,7 @@ namespace GitHub.Runner.Worker
|
|||||||
public class ActionSetupInfo
|
public class ActionSetupInfo
|
||||||
{
|
{
|
||||||
public ActionContainer Container { get; set; }
|
public ActionContainer Container { get; set; }
|
||||||
public List<Pipelines.ActionStep> Steps {get; set;}
|
public List<Pipelines.ActionStep> Steps { get; set; }
|
||||||
}
|
}
|
||||||
|
|
||||||
public class PrepareActionsState
|
public class PrepareActionsState
|
||||||
|
|||||||
@@ -480,6 +480,10 @@ namespace GitHub.Runner.Worker
|
|||||||
return new CompositeActionExecutionData()
|
return new CompositeActionExecutionData()
|
||||||
{
|
{
|
||||||
Steps = steps.Cast<Pipelines.ActionStep>().ToList(),
|
Steps = steps.Cast<Pipelines.ActionStep>().ToList(),
|
||||||
|
PreSteps = new List<Pipelines.ActionStep>(),
|
||||||
|
PostSteps = new Stack<Pipelines.ActionStep>(),
|
||||||
|
InitCondition = "always()",
|
||||||
|
CleanupCondition = "always()",
|
||||||
Outputs = outputs
|
Outputs = outputs
|
||||||
};
|
};
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -82,6 +82,28 @@ namespace GitHub.Runner.Worker
|
|||||||
ActionExecutionData handlerData = definition.Data?.Execution;
|
ActionExecutionData handlerData = definition.Data?.Execution;
|
||||||
ArgUtil.NotNull(handlerData, nameof(handlerData));
|
ArgUtil.NotNull(handlerData, nameof(handlerData));
|
||||||
|
|
||||||
|
List<JobExtensionRunner> localActionContainerSetupSteps = null;
|
||||||
|
// Handle Composite Local Actions
|
||||||
|
// Need to download and expand the tree of referenced actions
|
||||||
|
if (handlerData.ExecutionType == ActionExecutionType.Composite &&
|
||||||
|
handlerData is CompositeActionExecutionData compositeHandlerData &&
|
||||||
|
Stage == ActionRunStage.Main &&
|
||||||
|
Action.Reference is Pipelines.RepositoryPathReference localAction &&
|
||||||
|
string.Equals(localAction.RepositoryType, Pipelines.PipelineConstants.SelfAlias, StringComparison.OrdinalIgnoreCase))
|
||||||
|
{
|
||||||
|
var actionManager = HostContext.GetService<IActionManager>();
|
||||||
|
var prepareResult = await actionManager.PrepareActionsAsync(ExecutionContext, compositeHandlerData.Steps, ExecutionContext.Id);
|
||||||
|
|
||||||
|
// Reload definition since post may exist now (from embedded steps that were JIT downloaded)
|
||||||
|
definition = taskManager.LoadAction(ExecutionContext, Action);
|
||||||
|
ArgUtil.NotNull(definition, nameof(definition));
|
||||||
|
handlerData = definition.Data?.Execution;
|
||||||
|
ArgUtil.NotNull(handlerData, nameof(handlerData));
|
||||||
|
|
||||||
|
// Save container setup steps so we can reference them later
|
||||||
|
localActionContainerSetupSteps = prepareResult.ContainerSetupSteps;
|
||||||
|
}
|
||||||
|
|
||||||
if (handlerData.HasPre &&
|
if (handlerData.HasPre &&
|
||||||
Action.Reference is Pipelines.RepositoryPathReference repoAction &&
|
Action.Reference is Pipelines.RepositoryPathReference repoAction &&
|
||||||
string.Equals(repoAction.RepositoryType, Pipelines.PipelineConstants.SelfAlias, StringComparison.OrdinalIgnoreCase))
|
string.Equals(repoAction.RepositoryType, Pipelines.PipelineConstants.SelfAlias, StringComparison.OrdinalIgnoreCase))
|
||||||
@@ -249,7 +271,8 @@ namespace GitHub.Runner.Worker
|
|||||||
inputs,
|
inputs,
|
||||||
environment,
|
environment,
|
||||||
ExecutionContext.Global.Variables,
|
ExecutionContext.Global.Variables,
|
||||||
actionDirectory: definition.Directory);
|
actionDirectory: definition.Directory,
|
||||||
|
localActionContainerSetupSteps: localActionContainerSetupSteps);
|
||||||
|
|
||||||
// Print out action details
|
// Print out action details
|
||||||
handler.PrintActionDetails(Stage);
|
handler.PrintActionDetails(Stage);
|
||||||
|
|||||||
47
src/Runner.Worker/ConditionTraceWriter.cs
Normal file
47
src/Runner.Worker/ConditionTraceWriter.cs
Normal file
@@ -0,0 +1,47 @@
|
|||||||
|
using System;
|
||||||
|
using System.Text;
|
||||||
|
using GitHub.DistributedTask.Pipelines;
|
||||||
|
using GitHub.Runner.Common;
|
||||||
|
using GitHub.Runner.Sdk;
|
||||||
|
using ObjectTemplating = GitHub.DistributedTask.ObjectTemplating;
|
||||||
|
|
||||||
|
namespace GitHub.Runner.Worker
|
||||||
|
{
|
||||||
|
public sealed class ConditionTraceWriter : ObjectTemplating::ITraceWriter
|
||||||
|
{
|
||||||
|
private readonly IExecutionContext _executionContext;
|
||||||
|
private readonly Tracing _trace;
|
||||||
|
private readonly StringBuilder _traceBuilder = new StringBuilder();
|
||||||
|
|
||||||
|
public string Trace => _traceBuilder.ToString();
|
||||||
|
|
||||||
|
public ConditionTraceWriter(Tracing trace, IExecutionContext executionContext)
|
||||||
|
{
|
||||||
|
ArgUtil.NotNull(trace, nameof(trace));
|
||||||
|
_trace = trace;
|
||||||
|
_executionContext = executionContext;
|
||||||
|
}
|
||||||
|
|
||||||
|
public void Error(string format, params Object[] args)
|
||||||
|
{
|
||||||
|
var message = StringUtil.Format(format, args);
|
||||||
|
_trace.Error(message);
|
||||||
|
_executionContext?.Debug(message);
|
||||||
|
}
|
||||||
|
|
||||||
|
public void Info(string format, params Object[] args)
|
||||||
|
{
|
||||||
|
var message = StringUtil.Format(format, args);
|
||||||
|
_trace.Info(message);
|
||||||
|
_executionContext?.Debug(message);
|
||||||
|
_traceBuilder.AppendLine(message);
|
||||||
|
}
|
||||||
|
|
||||||
|
public void Verbose(string format, params Object[] args)
|
||||||
|
{
|
||||||
|
var message = StringUtil.Format(format, args);
|
||||||
|
_trace.Verbose(message);
|
||||||
|
_executionContext?.Debug(message);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -494,7 +494,8 @@ namespace GitHub.Runner.Worker
|
|||||||
private void UpdateRegistryAuthForGitHubToken(IExecutionContext executionContext, ContainerInfo container)
|
private void UpdateRegistryAuthForGitHubToken(IExecutionContext executionContext, ContainerInfo container)
|
||||||
{
|
{
|
||||||
var registryIsTokenCompatible = container.RegistryServer.Equals("ghcr.io", StringComparison.OrdinalIgnoreCase) || container.RegistryServer.Equals("containers.pkg.github.com", StringComparison.OrdinalIgnoreCase);
|
var registryIsTokenCompatible = container.RegistryServer.Equals("ghcr.io", StringComparison.OrdinalIgnoreCase) || container.RegistryServer.Equals("containers.pkg.github.com", StringComparison.OrdinalIgnoreCase);
|
||||||
if (!registryIsTokenCompatible)
|
var isFallbackTokenFromHostedGithub = HostContext.GetService<IConfigurationStore>().GetSettings().IsHostedServer;
|
||||||
|
if (!registryIsTokenCompatible || !isFallbackTokenFromHostedGithub)
|
||||||
{
|
{
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -36,7 +36,9 @@ namespace GitHub.Runner.Worker
|
|||||||
public interface IExecutionContext : IRunnerService
|
public interface IExecutionContext : IRunnerService
|
||||||
{
|
{
|
||||||
Guid Id { get; }
|
Guid Id { get; }
|
||||||
|
Guid EmbeddedId { get; }
|
||||||
string ScopeName { get; }
|
string ScopeName { get; }
|
||||||
|
string SiblingScopeName { get; }
|
||||||
string ContextName { get; }
|
string ContextName { get; }
|
||||||
Task ForceCompleted { get; }
|
Task ForceCompleted { get; }
|
||||||
TaskResult? Result { get; set; }
|
TaskResult? Result { get; set; }
|
||||||
@@ -49,6 +51,7 @@ namespace GitHub.Runner.Worker
|
|||||||
Dictionary<string, string> IntraActionState { get; }
|
Dictionary<string, string> IntraActionState { get; }
|
||||||
Dictionary<string, VariableValue> JobOutputs { get; }
|
Dictionary<string, VariableValue> JobOutputs { get; }
|
||||||
ActionsEnvironmentReference ActionsEnvironment { get; }
|
ActionsEnvironmentReference ActionsEnvironment { get; }
|
||||||
|
List<ActionsStepTelemetry> ActionsStepsTelemetry { get; }
|
||||||
DictionaryContextData ExpressionValues { get; }
|
DictionaryContextData ExpressionValues { get; }
|
||||||
IList<IFunctionInfo> ExpressionFunctions { get; }
|
IList<IFunctionInfo> ExpressionFunctions { get; }
|
||||||
JobContext JobContext { get; }
|
JobContext JobContext { get; }
|
||||||
@@ -58,6 +61,10 @@ namespace GitHub.Runner.Worker
|
|||||||
|
|
||||||
// Only job level ExecutionContext has PostJobSteps
|
// Only job level ExecutionContext has PostJobSteps
|
||||||
Stack<IStep> PostJobSteps { get; }
|
Stack<IStep> PostJobSteps { get; }
|
||||||
|
HashSet<Guid> EmbeddedStepsWithPostRegistered{ get; }
|
||||||
|
|
||||||
|
// Keep track of embedded steps states
|
||||||
|
Dictionary<Guid, Dictionary<string, string>> EmbeddedIntraActionState { get; }
|
||||||
|
|
||||||
bool EchoOnActionCommand { get; set; }
|
bool EchoOnActionCommand { get; set; }
|
||||||
|
|
||||||
@@ -68,8 +75,8 @@ namespace GitHub.Runner.Worker
|
|||||||
// Initialize
|
// Initialize
|
||||||
void InitializeJob(Pipelines.AgentJobRequestMessage message, CancellationToken token);
|
void InitializeJob(Pipelines.AgentJobRequestMessage message, CancellationToken token);
|
||||||
void CancelToken();
|
void CancelToken();
|
||||||
IExecutionContext CreateChild(Guid recordId, string displayName, string refName, string scopeName, string contextName, Dictionary<string, string> intraActionState = null, int? recordOrder = null, IPagingLogger logger = null, bool isEmbedded = false, CancellationTokenSource cancellationTokenSource = null);
|
IExecutionContext CreateChild(Guid recordId, string displayName, string refName, string scopeName, string contextName, Dictionary<string, string> intraActionState = null, int? recordOrder = null, IPagingLogger logger = null, bool isEmbedded = false, CancellationTokenSource cancellationTokenSource = null, Guid embeddedId = default(Guid), string siblingScopeName = null);
|
||||||
IExecutionContext CreateEmbeddedChild(string scopeName, string contextName);
|
IExecutionContext CreateEmbeddedChild(string scopeName, string contextName, Guid embeddedId, Dictionary<string, string> intraActionState = null, string siblingScopeName = null);
|
||||||
|
|
||||||
// logging
|
// logging
|
||||||
long Write(string tag, string message);
|
long Write(string tag, string message);
|
||||||
@@ -132,7 +139,9 @@ namespace GitHub.Runner.Worker
|
|||||||
private long _totalThrottlingDelayInMilliseconds = 0;
|
private long _totalThrottlingDelayInMilliseconds = 0;
|
||||||
|
|
||||||
public Guid Id => _record.Id;
|
public Guid Id => _record.Id;
|
||||||
|
public Guid EmbeddedId { get; private set; }
|
||||||
public string ScopeName { get; private set; }
|
public string ScopeName { get; private set; }
|
||||||
|
public string SiblingScopeName { get; private set; }
|
||||||
public string ContextName { get; private set; }
|
public string ContextName { get; private set; }
|
||||||
public Task ForceCompleted => _forceCompleted.Task;
|
public Task ForceCompleted => _forceCompleted.Task;
|
||||||
public CancellationToken CancellationToken => _cancellationTokenSource.Token;
|
public CancellationToken CancellationToken => _cancellationTokenSource.Token;
|
||||||
@@ -140,6 +149,7 @@ namespace GitHub.Runner.Worker
|
|||||||
public Dictionary<string, VariableValue> JobOutputs { get; private set; }
|
public Dictionary<string, VariableValue> JobOutputs { get; private set; }
|
||||||
|
|
||||||
public ActionsEnvironmentReference ActionsEnvironment { get; private set; }
|
public ActionsEnvironmentReference ActionsEnvironment { get; private set; }
|
||||||
|
public List<ActionsStepTelemetry> ActionsStepsTelemetry { get; private set; }
|
||||||
public DictionaryContextData ExpressionValues { get; } = new DictionaryContextData();
|
public DictionaryContextData ExpressionValues { get; } = new DictionaryContextData();
|
||||||
public IList<IFunctionInfo> ExpressionFunctions { get; } = new List<IFunctionInfo>();
|
public IList<IFunctionInfo> ExpressionFunctions { get; } = new List<IFunctionInfo>();
|
||||||
|
|
||||||
@@ -155,6 +165,11 @@ namespace GitHub.Runner.Worker
|
|||||||
// Only job level ExecutionContext has StepsWithPostRegistered
|
// Only job level ExecutionContext has StepsWithPostRegistered
|
||||||
public HashSet<Guid> StepsWithPostRegistered { get; private set; }
|
public HashSet<Guid> StepsWithPostRegistered { get; private set; }
|
||||||
|
|
||||||
|
// Only job level ExecutionContext has EmbeddedStepsWithPostRegistered
|
||||||
|
public HashSet<Guid> EmbeddedStepsWithPostRegistered { get; private set; }
|
||||||
|
|
||||||
|
public Dictionary<Guid, Dictionary<string, string>> EmbeddedIntraActionState { get; private set; }
|
||||||
|
|
||||||
public bool EchoOnActionCommand { get; set; }
|
public bool EchoOnActionCommand { get; set; }
|
||||||
|
|
||||||
// An embedded execution context shares the same record ID, record name, and logger
|
// An embedded execution context shares the same record ID, record name, and logger
|
||||||
@@ -245,23 +260,30 @@ namespace GitHub.Runner.Worker
|
|||||||
|
|
||||||
public void RegisterPostJobStep(IStep step)
|
public void RegisterPostJobStep(IStep step)
|
||||||
{
|
{
|
||||||
// TODO: Remove when we support composite post job steps
|
string siblingScopeName = null;
|
||||||
if (this.IsEmbedded)
|
if (this.IsEmbedded)
|
||||||
{
|
{
|
||||||
throw new Exception("Composite actions do not currently support post steps");
|
if (step is IActionRunner actionRunner && !Root.EmbeddedStepsWithPostRegistered.Add(actionRunner.Action.Id))
|
||||||
|
{
|
||||||
}
|
Trace.Info($"'post' of '{actionRunner.DisplayName}' already push to child post step stack.");
|
||||||
if (step is IActionRunner actionRunner && !Root.StepsWithPostRegistered.Add(actionRunner.Action.Id))
|
}
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
else if (step is IActionRunner actionRunner && !Root.StepsWithPostRegistered.Add(actionRunner.Action.Id))
|
||||||
{
|
{
|
||||||
Trace.Info($"'post' of '{actionRunner.DisplayName}' already push to post step stack.");
|
Trace.Info($"'post' of '{actionRunner.DisplayName}' already push to post step stack.");
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
|
if (step is IActionRunner runner)
|
||||||
|
{
|
||||||
|
siblingScopeName = runner.Action.ContextName;
|
||||||
|
}
|
||||||
|
|
||||||
step.ExecutionContext = Root.CreatePostChild(step.DisplayName, IntraActionState);
|
step.ExecutionContext = Root.CreatePostChild(step.DisplayName, IntraActionState, siblingScopeName);
|
||||||
Root.PostJobSteps.Push(step);
|
Root.PostJobSteps.Push(step);
|
||||||
}
|
}
|
||||||
|
|
||||||
public IExecutionContext CreateChild(Guid recordId, string displayName, string refName, string scopeName, string contextName, Dictionary<string, string> intraActionState = null, int? recordOrder = null, IPagingLogger logger = null, bool isEmbedded = false, CancellationTokenSource cancellationTokenSource = null)
|
public IExecutionContext CreateChild(Guid recordId, string displayName, string refName, string scopeName, string contextName, Dictionary<string, string> intraActionState = null, int? recordOrder = null, IPagingLogger logger = null, bool isEmbedded = false, CancellationTokenSource cancellationTokenSource = null, Guid embeddedId = default(Guid), string siblingScopeName = null)
|
||||||
{
|
{
|
||||||
Trace.Entering();
|
Trace.Entering();
|
||||||
|
|
||||||
@@ -270,6 +292,8 @@ namespace GitHub.Runner.Worker
|
|||||||
child.Global = Global;
|
child.Global = Global;
|
||||||
child.ScopeName = scopeName;
|
child.ScopeName = scopeName;
|
||||||
child.ContextName = contextName;
|
child.ContextName = contextName;
|
||||||
|
child.EmbeddedId = embeddedId;
|
||||||
|
child.SiblingScopeName = siblingScopeName;
|
||||||
if (intraActionState == null)
|
if (intraActionState == null)
|
||||||
{
|
{
|
||||||
child.IntraActionState = new Dictionary<string, string>(StringComparer.OrdinalIgnoreCase);
|
child.IntraActionState = new Dictionary<string, string>(StringComparer.OrdinalIgnoreCase);
|
||||||
@@ -317,9 +341,9 @@ namespace GitHub.Runner.Worker
|
|||||||
/// An embedded execution context shares the same record ID, record name, logger,
|
/// An embedded execution context shares the same record ID, record name, logger,
|
||||||
/// and a linked cancellation token.
|
/// and a linked cancellation token.
|
||||||
/// </summary>
|
/// </summary>
|
||||||
public IExecutionContext CreateEmbeddedChild(string scopeName, string contextName)
|
public IExecutionContext CreateEmbeddedChild(string scopeName, string contextName, Guid embeddedId, Dictionary<string, string> intraActionState = null, string siblingScopeName = null)
|
||||||
{
|
{
|
||||||
return Root.CreateChild(_record.Id, _record.Name, _record.Id.ToString("N"), scopeName, contextName, logger: _logger, isEmbedded: true, cancellationTokenSource: CancellationTokenSource.CreateLinkedTokenSource(_cancellationTokenSource.Token));
|
return Root.CreateChild(_record.Id, _record.Name, _record.Id.ToString("N"), scopeName, contextName, logger: _logger, isEmbedded: true, cancellationTokenSource: CancellationTokenSource.CreateLinkedTokenSource(_cancellationTokenSource.Token), intraActionState: intraActionState, embeddedId: embeddedId, siblingScopeName: siblingScopeName);
|
||||||
}
|
}
|
||||||
|
|
||||||
public void Start(string currentOperation = null)
|
public void Start(string currentOperation = null)
|
||||||
@@ -375,7 +399,7 @@ namespace GitHub.Runner.Worker
|
|||||||
|
|
||||||
_logger.End();
|
_logger.End();
|
||||||
|
|
||||||
// Skip if generated context name. Generated context names start with "__". After M271-ish the server will never send an empty context name.
|
// Skip if generated context name. Generated context names start with "__". After 3.2 the server will never send an empty context name.
|
||||||
if (!string.IsNullOrEmpty(ContextName) && !ContextName.StartsWith("__", StringComparison.Ordinal))
|
if (!string.IsNullOrEmpty(ContextName) && !ContextName.StartsWith("__", StringComparison.Ordinal))
|
||||||
{
|
{
|
||||||
Global.StepsContext.SetOutcome(ScopeName, ContextName, (Outcome ?? Result ?? TaskResult.Succeeded).ToActionResult());
|
Global.StepsContext.SetOutcome(ScopeName, ContextName, (Outcome ?? Result ?? TaskResult.Succeeded).ToActionResult());
|
||||||
@@ -438,7 +462,7 @@ namespace GitHub.Runner.Worker
|
|||||||
{
|
{
|
||||||
ArgUtil.NotNullOrEmpty(name, nameof(name));
|
ArgUtil.NotNullOrEmpty(name, nameof(name));
|
||||||
|
|
||||||
// Skip if generated context name. Generated context names start with "__". After M271-ish the server will never send an empty context name.
|
// Skip if generated context name. Generated context names start with "__". After 3.2 the server will never send an empty context name.
|
||||||
if (string.IsNullOrEmpty(ContextName) || ContextName.StartsWith("__", StringComparison.Ordinal))
|
if (string.IsNullOrEmpty(ContextName) || ContextName.StartsWith("__", StringComparison.Ordinal))
|
||||||
{
|
{
|
||||||
reference = null;
|
reference = null;
|
||||||
@@ -623,6 +647,9 @@ namespace GitHub.Runner.Worker
|
|||||||
// Actions environment
|
// Actions environment
|
||||||
ActionsEnvironment = message.ActionsEnvironment;
|
ActionsEnvironment = message.ActionsEnvironment;
|
||||||
|
|
||||||
|
// ActionsStepTelemetry
|
||||||
|
ActionsStepsTelemetry = new List<ActionsStepTelemetry>();
|
||||||
|
|
||||||
// Service container info
|
// Service container info
|
||||||
Global.ServiceContainers = new List<ContainerInfo>();
|
Global.ServiceContainers = new List<ContainerInfo>();
|
||||||
|
|
||||||
@@ -683,6 +710,12 @@ namespace GitHub.Runner.Worker
|
|||||||
// StepsWithPostRegistered for job ExecutionContext
|
// StepsWithPostRegistered for job ExecutionContext
|
||||||
StepsWithPostRegistered = new HashSet<Guid>();
|
StepsWithPostRegistered = new HashSet<Guid>();
|
||||||
|
|
||||||
|
// EmbeddedStepsWithPostRegistered for job ExecutionContext
|
||||||
|
EmbeddedStepsWithPostRegistered = new HashSet<Guid>();
|
||||||
|
|
||||||
|
// EmbeddedIntraActionState for job ExecutionContext
|
||||||
|
EmbeddedIntraActionState = new Dictionary<Guid, Dictionary<string,string>>();
|
||||||
|
|
||||||
// Job timeline record.
|
// Job timeline record.
|
||||||
InitializeTimelineRecord(
|
InitializeTimelineRecord(
|
||||||
timelineId: message.Timeline.Id,
|
timelineId: message.Timeline.Id,
|
||||||
@@ -889,7 +922,7 @@ namespace GitHub.Runner.Worker
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
private IExecutionContext CreatePostChild(string displayName, Dictionary<string, string> intraActionState)
|
private IExecutionContext CreatePostChild(string displayName, Dictionary<string, string> intraActionState, string siblingScopeName = null)
|
||||||
{
|
{
|
||||||
if (!_expandedForPostJob)
|
if (!_expandedForPostJob)
|
||||||
{
|
{
|
||||||
@@ -899,7 +932,7 @@ namespace GitHub.Runner.Worker
|
|||||||
}
|
}
|
||||||
|
|
||||||
var newGuid = Guid.NewGuid();
|
var newGuid = Guid.NewGuid();
|
||||||
return CreateChild(newGuid, displayName, newGuid.ToString("N"), null, null, intraActionState, _childTimelineRecordOrder - Root.PostJobSteps.Count);
|
return CreateChild(newGuid, displayName, newGuid.ToString("N"), null, null, intraActionState, _childTimelineRecordOrder - Root.PostJobSteps.Count, siblingScopeName: siblingScopeName);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -11,6 +11,7 @@ using GitHub.DistributedTask.Pipelines.ContextData;
|
|||||||
using GitHub.DistributedTask.Pipelines.ObjectTemplating;
|
using GitHub.DistributedTask.Pipelines.ObjectTemplating;
|
||||||
using GitHub.DistributedTask.WebApi;
|
using GitHub.DistributedTask.WebApi;
|
||||||
using GitHub.Runner.Common;
|
using GitHub.Runner.Common;
|
||||||
|
using GitHub.Runner.Common.Util;
|
||||||
using GitHub.Runner.Sdk;
|
using GitHub.Runner.Sdk;
|
||||||
using GitHub.Runner.Worker;
|
using GitHub.Runner.Worker;
|
||||||
using GitHub.Runner.Worker.Expressions;
|
using GitHub.Runner.Worker.Expressions;
|
||||||
@@ -34,8 +35,67 @@ namespace GitHub.Runner.Worker.Handlers
|
|||||||
Trace.Entering();
|
Trace.Entering();
|
||||||
ArgUtil.NotNull(ExecutionContext, nameof(ExecutionContext));
|
ArgUtil.NotNull(ExecutionContext, nameof(ExecutionContext));
|
||||||
ArgUtil.NotNull(Inputs, nameof(Inputs));
|
ArgUtil.NotNull(Inputs, nameof(Inputs));
|
||||||
ArgUtil.NotNull(Data.Steps, nameof(Data.Steps));
|
|
||||||
|
|
||||||
|
List<Pipelines.ActionStep> steps;
|
||||||
|
|
||||||
|
if (stage == ActionRunStage.Pre)
|
||||||
|
{
|
||||||
|
ArgUtil.NotNull(Data.PreSteps, nameof(Data.PreSteps));
|
||||||
|
steps = Data.PreSteps;
|
||||||
|
}
|
||||||
|
else if (stage == ActionRunStage.Post)
|
||||||
|
{
|
||||||
|
ArgUtil.NotNull(Data.PostSteps, nameof(Data.PostSteps));
|
||||||
|
steps = new List<Pipelines.ActionStep>();
|
||||||
|
// Only register post steps for steps that actually ran
|
||||||
|
foreach (var step in Data.PostSteps.ToList())
|
||||||
|
{
|
||||||
|
if (ExecutionContext.Root.EmbeddedStepsWithPostRegistered.Contains(step.Id))
|
||||||
|
{
|
||||||
|
steps.Add(step);
|
||||||
|
}
|
||||||
|
else
|
||||||
|
{
|
||||||
|
Trace.Info($"Skipping executing post step id: {step.Id}, name: ${step.DisplayName}");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
else
|
||||||
|
{
|
||||||
|
ArgUtil.NotNull(Data.Steps, nameof(Data.Steps));
|
||||||
|
steps = Data.Steps;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Add Telemetry to JobContext to send with JobCompleteMessage
|
||||||
|
if (stage == ActionRunStage.Main)
|
||||||
|
{
|
||||||
|
var hasRunsStep = false;
|
||||||
|
var hasUsesStep = false;
|
||||||
|
foreach (var step in steps)
|
||||||
|
{
|
||||||
|
if (step.Reference.Type == Pipelines.ActionSourceType.Script)
|
||||||
|
{
|
||||||
|
hasRunsStep = true;
|
||||||
|
}
|
||||||
|
else
|
||||||
|
{
|
||||||
|
hasUsesStep = true;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
var pathReference = Action as Pipelines.RepositoryPathReference;
|
||||||
|
var telemetry = new ActionsStepTelemetry {
|
||||||
|
Ref = GetActionRef(),
|
||||||
|
HasPreStep = Data.HasPre,
|
||||||
|
HasPostStep = Data.HasPost,
|
||||||
|
IsEmbedded = ExecutionContext.IsEmbedded,
|
||||||
|
Type = "composite",
|
||||||
|
HasRunsStep = hasRunsStep,
|
||||||
|
HasUsesStep = hasUsesStep,
|
||||||
|
StepCount = steps.Count
|
||||||
|
};
|
||||||
|
ExecutionContext.Root.ActionsStepsTelemetry.Add(telemetry);
|
||||||
|
}
|
||||||
|
|
||||||
try
|
try
|
||||||
{
|
{
|
||||||
// Inputs of the composite step
|
// Inputs of the composite step
|
||||||
@@ -45,7 +105,7 @@ namespace GitHub.Runner.Worker.Handlers
|
|||||||
inputsData[i.Key] = new StringContextData(i.Value);
|
inputsData[i.Key] = new StringContextData(i.Value);
|
||||||
}
|
}
|
||||||
|
|
||||||
// Temporary hack until after M271-ish. After M271-ish the server will never send an empty
|
// Temporary hack until after 3.2. After 3.2 the server will never send an empty
|
||||||
// context name. Generated context names start with "__"
|
// context name. Generated context names start with "__"
|
||||||
var childScopeName = ExecutionContext.GetFullyQualifiedContextName();
|
var childScopeName = ExecutionContext.GetFullyQualifiedContextName();
|
||||||
if (string.IsNullOrEmpty(childScopeName))
|
if (string.IsNullOrEmpty(childScopeName))
|
||||||
@@ -55,15 +115,44 @@ namespace GitHub.Runner.Worker.Handlers
|
|||||||
|
|
||||||
// Create embedded steps
|
// Create embedded steps
|
||||||
var embeddedSteps = new List<IStep>();
|
var embeddedSteps = new List<IStep>();
|
||||||
foreach (Pipelines.ActionStep stepData in Data.Steps)
|
|
||||||
|
// If we need to setup containers beforehand, do it
|
||||||
|
// only relevant for local composite actions that need to JIT download/setup containers
|
||||||
|
if (LocalActionContainerSetupSteps != null && LocalActionContainerSetupSteps.Count > 0)
|
||||||
{
|
{
|
||||||
|
foreach (var step in LocalActionContainerSetupSteps)
|
||||||
|
{
|
||||||
|
ArgUtil.NotNull(step, step.DisplayName);
|
||||||
|
var stepId = $"__{Guid.NewGuid()}";
|
||||||
|
step.ExecutionContext = ExecutionContext.CreateEmbeddedChild(childScopeName, stepId, Guid.NewGuid());
|
||||||
|
embeddedSteps.Add(step);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
foreach (Pipelines.ActionStep stepData in steps)
|
||||||
|
{
|
||||||
|
// Compute child sibling scope names for post steps
|
||||||
|
// We need to use the main's scope to keep step context correct, makes inputs flow correctly
|
||||||
|
string siblingScopeName = null;
|
||||||
|
if (!String.IsNullOrEmpty(ExecutionContext.SiblingScopeName) && stage == ActionRunStage.Post)
|
||||||
|
{
|
||||||
|
siblingScopeName = $"{ExecutionContext.SiblingScopeName}.{stepData.ContextName}";
|
||||||
|
}
|
||||||
|
|
||||||
var step = HostContext.CreateService<IActionRunner>();
|
var step = HostContext.CreateService<IActionRunner>();
|
||||||
step.Action = stepData;
|
step.Action = stepData;
|
||||||
step.Stage = stage;
|
step.Stage = stage;
|
||||||
step.Condition = stepData.Condition;
|
step.Condition = stepData.Condition;
|
||||||
step.ExecutionContext = ExecutionContext.CreateEmbeddedChild(childScopeName, stepData.ContextName);
|
ExecutionContext.Root.EmbeddedIntraActionState.TryGetValue(step.Action.Id, out var intraActionState);
|
||||||
|
step.ExecutionContext = ExecutionContext.CreateEmbeddedChild(childScopeName, stepData.ContextName, step.Action.Id, intraActionState: intraActionState, siblingScopeName: siblingScopeName);
|
||||||
step.ExecutionContext.ExpressionValues["inputs"] = inputsData;
|
step.ExecutionContext.ExpressionValues["inputs"] = inputsData;
|
||||||
step.ExecutionContext.ExpressionValues["steps"] = ExecutionContext.Global.StepsContext.GetScope(childScopeName);
|
if (!String.IsNullOrEmpty(ExecutionContext.SiblingScopeName))
|
||||||
|
{
|
||||||
|
step.ExecutionContext.ExpressionValues["steps"] = ExecutionContext.Global.StepsContext.GetScope(ExecutionContext.SiblingScopeName);
|
||||||
|
}
|
||||||
|
else
|
||||||
|
{
|
||||||
|
step.ExecutionContext.ExpressionValues["steps"] = ExecutionContext.Global.StepsContext.GetScope(childScopeName);
|
||||||
|
}
|
||||||
|
|
||||||
// Shallow copy github context
|
// Shallow copy github context
|
||||||
var gitHubContext = step.ExecutionContext.ExpressionValues["github"] as GitHubContext;
|
var gitHubContext = step.ExecutionContext.ExpressionValues["github"] as GitHubContext;
|
||||||
@@ -78,13 +167,12 @@ namespace GitHub.Runner.Worker.Handlers
|
|||||||
}
|
}
|
||||||
|
|
||||||
// Run embedded steps
|
// Run embedded steps
|
||||||
await RunStepsAsync(embeddedSteps);
|
await RunStepsAsync(embeddedSteps, stage);
|
||||||
|
|
||||||
// Set outputs
|
// Set outputs
|
||||||
ExecutionContext.ExpressionValues["inputs"] = inputsData;
|
ExecutionContext.ExpressionValues["inputs"] = inputsData;
|
||||||
ExecutionContext.ExpressionValues["steps"] = ExecutionContext.Global.StepsContext.GetScope(childScopeName);
|
ExecutionContext.ExpressionValues["steps"] = ExecutionContext.Global.StepsContext.GetScope(childScopeName);
|
||||||
ProcessOutputs();
|
ProcessOutputs();
|
||||||
ExecutionContext.Global.StepsContext.ClearScope(childScopeName);
|
|
||||||
}
|
}
|
||||||
catch (Exception ex)
|
catch (Exception ex)
|
||||||
{
|
{
|
||||||
@@ -138,7 +226,7 @@ namespace GitHub.Runner.Worker.Handlers
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
private async Task RunStepsAsync(List<IStep> embeddedSteps)
|
private async Task RunStepsAsync(List<IStep> embeddedSteps, ActionRunStage stage)
|
||||||
{
|
{
|
||||||
ArgUtil.NotNull(embeddedSteps, nameof(embeddedSteps));
|
ArgUtil.NotNull(embeddedSteps, nameof(embeddedSteps));
|
||||||
|
|
||||||
@@ -148,6 +236,10 @@ namespace GitHub.Runner.Worker.Handlers
|
|||||||
|
|
||||||
// Add Expression Functions
|
// Add Expression Functions
|
||||||
step.ExecutionContext.ExpressionFunctions.Add(new FunctionInfo<HashFilesFunction>(PipelineTemplateConstants.HashFiles, 1, byte.MaxValue));
|
step.ExecutionContext.ExpressionFunctions.Add(new FunctionInfo<HashFilesFunction>(PipelineTemplateConstants.HashFiles, 1, byte.MaxValue));
|
||||||
|
step.ExecutionContext.ExpressionFunctions.Add(new FunctionInfo<AlwaysFunction>(PipelineTemplateConstants.Always, 0, 0));
|
||||||
|
step.ExecutionContext.ExpressionFunctions.Add(new FunctionInfo<CancelledFunction>(PipelineTemplateConstants.Cancelled, 0, 0));
|
||||||
|
step.ExecutionContext.ExpressionFunctions.Add(new FunctionInfo<FailureFunction>(PipelineTemplateConstants.Failure, 0, 0));
|
||||||
|
step.ExecutionContext.ExpressionFunctions.Add(new FunctionInfo<SuccessFunction>(PipelineTemplateConstants.Success, 0, 0));
|
||||||
|
|
||||||
// Initialize env context
|
// Initialize env context
|
||||||
Trace.Info("Initialize Env context for embedded step");
|
Trace.Info("Initialize Env context for embedded step");
|
||||||
@@ -178,16 +270,17 @@ namespace GitHub.Runner.Worker.Handlers
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
var actionStep = step as IActionRunner;
|
|
||||||
|
|
||||||
try
|
try
|
||||||
{
|
{
|
||||||
// Evaluate and merge embedded-step env
|
if (step is IActionRunner actionStep)
|
||||||
var templateEvaluator = step.ExecutionContext.ToPipelineTemplateEvaluator();
|
|
||||||
var actionEnvironment = templateEvaluator.EvaluateStepEnvironment(actionStep.Action.Environment, step.ExecutionContext.ExpressionValues, step.ExecutionContext.ExpressionFunctions, Common.Util.VarUtil.EnvironmentVariableKeyComparer);
|
|
||||||
foreach (var env in actionEnvironment)
|
|
||||||
{
|
{
|
||||||
envContext[env.Key] = new StringContextData(env.Value ?? string.Empty);
|
// Evaluate and merge embedded-step env
|
||||||
|
var templateEvaluator = step.ExecutionContext.ToPipelineTemplateEvaluator();
|
||||||
|
var actionEnvironment = templateEvaluator.EvaluateStepEnvironment(actionStep.Action.Environment, step.ExecutionContext.ExpressionValues, step.ExecutionContext.ExpressionFunctions, Common.Util.VarUtil.EnvironmentVariableKeyComparer);
|
||||||
|
foreach (var env in actionEnvironment)
|
||||||
|
{
|
||||||
|
envContext[env.Key] = new StringContextData(env.Value ?? string.Empty);
|
||||||
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
catch (Exception ex)
|
catch (Exception ex)
|
||||||
@@ -198,13 +291,133 @@ namespace GitHub.Runner.Worker.Handlers
|
|||||||
step.ExecutionContext.Complete(TaskResult.Failed);
|
step.ExecutionContext.Complete(TaskResult.Failed);
|
||||||
}
|
}
|
||||||
|
|
||||||
await RunStepAsync(step);
|
// Register Callback
|
||||||
|
CancellationTokenRegistration? jobCancelRegister = null;
|
||||||
|
try
|
||||||
|
{
|
||||||
|
// For main steps just run the action
|
||||||
|
if (stage == ActionRunStage.Main)
|
||||||
|
{
|
||||||
|
await RunStepAsync(step);
|
||||||
|
}
|
||||||
|
// We need to evaluate conditions for pre/post steps
|
||||||
|
else
|
||||||
|
{
|
||||||
|
// Register job cancellation call back only if job cancellation token not been fire before each step run
|
||||||
|
if (!ExecutionContext.Root.CancellationToken.IsCancellationRequested)
|
||||||
|
{
|
||||||
|
// Test the condition again. The job was canceled after the condition was originally evaluated.
|
||||||
|
jobCancelRegister = ExecutionContext.Root.CancellationToken.Register(() =>
|
||||||
|
{
|
||||||
|
// Mark job as cancelled
|
||||||
|
ExecutionContext.Root.Result = TaskResult.Canceled;
|
||||||
|
ExecutionContext.Root.JobContext.Status = ExecutionContext.Root.Result?.ToActionResult();
|
||||||
|
|
||||||
|
step.ExecutionContext.Debug($"Re-evaluate condition on job cancellation for step: '{step.DisplayName}'.");
|
||||||
|
var conditionReTestTraceWriter = new ConditionTraceWriter(Trace, null); // host tracing only
|
||||||
|
var conditionReTestResult = false;
|
||||||
|
if (HostContext.RunnerShutdownToken.IsCancellationRequested)
|
||||||
|
{
|
||||||
|
step.ExecutionContext.Debug($"Skip Re-evaluate condition on runner shutdown.");
|
||||||
|
}
|
||||||
|
else
|
||||||
|
{
|
||||||
|
try
|
||||||
|
{
|
||||||
|
var templateEvaluator = step.ExecutionContext.ToPipelineTemplateEvaluator(conditionReTestTraceWriter);
|
||||||
|
var condition = new BasicExpressionToken(null, null, null, step.Condition);
|
||||||
|
conditionReTestResult = templateEvaluator.EvaluateStepIf(condition, step.ExecutionContext.ExpressionValues, step.ExecutionContext.ExpressionFunctions, step.ExecutionContext.ToExpressionState());
|
||||||
|
}
|
||||||
|
catch (Exception ex)
|
||||||
|
{
|
||||||
|
// Cancel the step since we get exception while re-evaluate step condition
|
||||||
|
Trace.Info("Caught exception from expression when re-test condition on job cancellation.");
|
||||||
|
step.ExecutionContext.Error(ex);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
if (!conditionReTestResult)
|
||||||
|
{
|
||||||
|
// Cancel the step
|
||||||
|
Trace.Info("Cancel current running step.");
|
||||||
|
step.ExecutionContext.CancelToken();
|
||||||
|
}
|
||||||
|
});
|
||||||
|
}
|
||||||
|
else
|
||||||
|
{
|
||||||
|
if (ExecutionContext.Root.Result != TaskResult.Canceled)
|
||||||
|
{
|
||||||
|
// Mark job as cancelled
|
||||||
|
ExecutionContext.Root.Result = TaskResult.Canceled;
|
||||||
|
ExecutionContext.Root.JobContext.Status = ExecutionContext.Root.Result?.ToActionResult();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
// Evaluate condition
|
||||||
|
step.ExecutionContext.Debug($"Evaluating condition for step: '{step.DisplayName}'");
|
||||||
|
var conditionTraceWriter = new ConditionTraceWriter(Trace, step.ExecutionContext);
|
||||||
|
var conditionResult = false;
|
||||||
|
var conditionEvaluateError = default(Exception);
|
||||||
|
if (HostContext.RunnerShutdownToken.IsCancellationRequested)
|
||||||
|
{
|
||||||
|
step.ExecutionContext.Debug($"Skip evaluate condition on runner shutdown.");
|
||||||
|
}
|
||||||
|
else
|
||||||
|
{
|
||||||
|
try
|
||||||
|
{
|
||||||
|
var templateEvaluator = step.ExecutionContext.ToPipelineTemplateEvaluator(conditionTraceWriter);
|
||||||
|
var condition = new BasicExpressionToken(null, null, null, step.Condition);
|
||||||
|
conditionResult = templateEvaluator.EvaluateStepIf(condition, step.ExecutionContext.ExpressionValues, step.ExecutionContext.ExpressionFunctions, step.ExecutionContext.ToExpressionState());
|
||||||
|
}
|
||||||
|
catch (Exception ex)
|
||||||
|
{
|
||||||
|
Trace.Info("Caught exception from expression.");
|
||||||
|
Trace.Error(ex);
|
||||||
|
conditionEvaluateError = ex;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if (!conditionResult && conditionEvaluateError == null)
|
||||||
|
{
|
||||||
|
// Condition is false
|
||||||
|
Trace.Info("Skipping step due to condition evaluation.");
|
||||||
|
step.ExecutionContext.Result = TaskResult.Skipped;
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
else if (conditionEvaluateError != null)
|
||||||
|
{
|
||||||
|
// Condition error
|
||||||
|
step.ExecutionContext.Error(conditionEvaluateError);
|
||||||
|
step.ExecutionContext.Result = TaskResult.Failed;
|
||||||
|
ExecutionContext.Result = TaskResult.Failed;
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
else
|
||||||
|
{
|
||||||
|
await RunStepAsync(step);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
finally
|
||||||
|
{
|
||||||
|
if (jobCancelRegister != null)
|
||||||
|
{
|
||||||
|
jobCancelRegister?.Dispose();
|
||||||
|
jobCancelRegister = null;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
// Check failed or canceled
|
// Check failed or canceled
|
||||||
if (step.ExecutionContext.Result == TaskResult.Failed || step.ExecutionContext.Result == TaskResult.Canceled)
|
if (step.ExecutionContext.Result == TaskResult.Failed || step.ExecutionContext.Result == TaskResult.Canceled)
|
||||||
{
|
{
|
||||||
ExecutionContext.Result = step.ExecutionContext.Result;
|
Trace.Info($"Update job result with current composite step result '{step.ExecutionContext.Result}'.");
|
||||||
break;
|
ExecutionContext.Result = TaskResultUtil.MergeTaskResults(ExecutionContext.Result, step.ExecutionContext.Result.Value);
|
||||||
|
|
||||||
|
// We should run cleanup even if one of the cleanup step fails
|
||||||
|
if (stage != ActionRunStage.Post)
|
||||||
|
{
|
||||||
|
break;
|
||||||
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -1,4 +1,4 @@
|
|||||||
using System.Collections.Generic;
|
using System.Collections.Generic;
|
||||||
using System.IO;
|
using System.IO;
|
||||||
using System.Threading.Tasks;
|
using System.Threading.Tasks;
|
||||||
using System;
|
using System;
|
||||||
@@ -69,6 +69,20 @@ namespace GitHub.Runner.Worker.Handlers
|
|||||||
Data.Image = imageName;
|
Data.Image = imageName;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
string type = Action.Type == Pipelines.ActionSourceType.Repository ? "Dockerfile" : "DockerHub";
|
||||||
|
// Add Telemetry to JobContext to send with JobCompleteMessage
|
||||||
|
if (stage == ActionRunStage.Main)
|
||||||
|
{
|
||||||
|
var telemetry = new ActionsStepTelemetry {
|
||||||
|
Ref = GetActionRef(),
|
||||||
|
HasPreStep = Data.HasPre,
|
||||||
|
HasPostStep = Data.HasPost,
|
||||||
|
IsEmbedded = ExecutionContext.IsEmbedded,
|
||||||
|
Type = type
|
||||||
|
};
|
||||||
|
ExecutionContext.Root.ActionsStepsTelemetry.Add(telemetry);
|
||||||
|
}
|
||||||
|
|
||||||
// run container
|
// run container
|
||||||
var container = new ContainerInfo(HostContext)
|
var container = new ContainerInfo(HostContext)
|
||||||
{
|
{
|
||||||
@@ -200,6 +214,11 @@ namespace GitHub.Runner.Worker.Handlers
|
|||||||
{
|
{
|
||||||
Environment["ACTIONS_CACHE_URL"] = cacheUrl;
|
Environment["ACTIONS_CACHE_URL"] = cacheUrl;
|
||||||
}
|
}
|
||||||
|
if (systemConnection.Data.TryGetValue("GenerateIdTokenUrl", out var generateIdTokenUrl) && !string.IsNullOrEmpty(generateIdTokenUrl))
|
||||||
|
{
|
||||||
|
Environment["ACTIONS_ID_TOKEN_REQUEST_URL"] = generateIdTokenUrl;
|
||||||
|
Environment["ACTIONS_ID_TOKEN_REQUEST_TOKEN"] = systemConnection.Authorization.Parameters[EndpointAuthorizationParameters.AccessToken];
|
||||||
|
}
|
||||||
|
|
||||||
foreach (var variable in this.Environment)
|
foreach (var variable in this.Environment)
|
||||||
{
|
{
|
||||||
|
|||||||
@@ -20,6 +20,7 @@ namespace GitHub.Runner.Worker.Handlers
|
|||||||
IStepHost StepHost { get; set; }
|
IStepHost StepHost { get; set; }
|
||||||
Dictionary<string, string> Inputs { get; set; }
|
Dictionary<string, string> Inputs { get; set; }
|
||||||
string ActionDirectory { get; set; }
|
string ActionDirectory { get; set; }
|
||||||
|
List<JobExtensionRunner> LocalActionContainerSetupSteps { get; set; }
|
||||||
Task RunAsync(ActionRunStage stage);
|
Task RunAsync(ActionRunStage stage);
|
||||||
void PrintActionDetails(ActionRunStage stage);
|
void PrintActionDetails(ActionRunStage stage);
|
||||||
}
|
}
|
||||||
@@ -41,9 +42,44 @@ namespace GitHub.Runner.Worker.Handlers
|
|||||||
public IStepHost StepHost { get; set; }
|
public IStepHost StepHost { get; set; }
|
||||||
public Dictionary<string, string> Inputs { get; set; }
|
public Dictionary<string, string> Inputs { get; set; }
|
||||||
public string ActionDirectory { get; set; }
|
public string ActionDirectory { get; set; }
|
||||||
|
public List<JobExtensionRunner> LocalActionContainerSetupSteps { get; set; }
|
||||||
|
|
||||||
|
public virtual string GetActionRef()
|
||||||
|
{
|
||||||
|
if (Action.Type == Pipelines.ActionSourceType.ContainerRegistry)
|
||||||
|
{
|
||||||
|
var registryAction = Action as Pipelines.ContainerRegistryReference;
|
||||||
|
return registryAction.Image;
|
||||||
|
}
|
||||||
|
else if (Action.Type == Pipelines.ActionSourceType.Repository)
|
||||||
|
{
|
||||||
|
var repoAction = Action as Pipelines.RepositoryPathReference;
|
||||||
|
if (string.Equals(repoAction.RepositoryType, Pipelines.PipelineConstants.SelfAlias, StringComparison.OrdinalIgnoreCase))
|
||||||
|
{
|
||||||
|
return repoAction.Path;
|
||||||
|
}
|
||||||
|
else
|
||||||
|
{
|
||||||
|
if (string.IsNullOrEmpty(repoAction.Path))
|
||||||
|
{
|
||||||
|
return $"{repoAction.Name}@{repoAction.Ref}";
|
||||||
|
}
|
||||||
|
else
|
||||||
|
{
|
||||||
|
return $"{repoAction.Name}/{repoAction.Path}@{repoAction.Ref}";
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
else
|
||||||
|
{
|
||||||
|
// this should never happen
|
||||||
|
Trace.Error($"Can't generate ref for {Action.Type.ToString()}");
|
||||||
|
}
|
||||||
|
return "";
|
||||||
|
}
|
||||||
public virtual void PrintActionDetails(ActionRunStage stage)
|
public virtual void PrintActionDetails(ActionRunStage stage)
|
||||||
{
|
{
|
||||||
|
|
||||||
if (stage == ActionRunStage.Post)
|
if (stage == ActionRunStage.Post)
|
||||||
{
|
{
|
||||||
ExecutionContext.Output($"Post job cleanup.");
|
ExecutionContext.Output($"Post job cleanup.");
|
||||||
|
|||||||
@@ -19,7 +19,8 @@ namespace GitHub.Runner.Worker.Handlers
|
|||||||
Dictionary<string, string> inputs,
|
Dictionary<string, string> inputs,
|
||||||
Dictionary<string, string> environment,
|
Dictionary<string, string> environment,
|
||||||
Variables runtimeVariables,
|
Variables runtimeVariables,
|
||||||
string actionDirectory);
|
string actionDirectory,
|
||||||
|
List<JobExtensionRunner> localActionContainerSetupSteps);
|
||||||
}
|
}
|
||||||
|
|
||||||
public sealed class HandlerFactory : RunnerService, IHandlerFactory
|
public sealed class HandlerFactory : RunnerService, IHandlerFactory
|
||||||
@@ -32,7 +33,8 @@ namespace GitHub.Runner.Worker.Handlers
|
|||||||
Dictionary<string, string> inputs,
|
Dictionary<string, string> inputs,
|
||||||
Dictionary<string, string> environment,
|
Dictionary<string, string> environment,
|
||||||
Variables runtimeVariables,
|
Variables runtimeVariables,
|
||||||
string actionDirectory)
|
string actionDirectory,
|
||||||
|
List<JobExtensionRunner> localActionContainerSetupSteps)
|
||||||
{
|
{
|
||||||
// Validate args.
|
// Validate args.
|
||||||
Trace.Entering();
|
Trace.Entering();
|
||||||
@@ -84,6 +86,7 @@ namespace GitHub.Runner.Worker.Handlers
|
|||||||
handler.StepHost = stepHost;
|
handler.StepHost = stepHost;
|
||||||
handler.Inputs = inputs;
|
handler.Inputs = inputs;
|
||||||
handler.ActionDirectory = actionDirectory;
|
handler.ActionDirectory = actionDirectory;
|
||||||
|
handler.LocalActionContainerSetupSteps = localActionContainerSetupSteps;
|
||||||
return handler;
|
return handler;
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -53,6 +53,11 @@ namespace GitHub.Runner.Worker.Handlers
|
|||||||
{
|
{
|
||||||
Environment["ACTIONS_CACHE_URL"] = cacheUrl;
|
Environment["ACTIONS_CACHE_URL"] = cacheUrl;
|
||||||
}
|
}
|
||||||
|
if (systemConnection.Data.TryGetValue("GenerateIdTokenUrl", out var generateIdTokenUrl) && !string.IsNullOrEmpty(generateIdTokenUrl))
|
||||||
|
{
|
||||||
|
Environment["ACTIONS_ID_TOKEN_REQUEST_URL"] = generateIdTokenUrl;
|
||||||
|
Environment["ACTIONS_ID_TOKEN_REQUEST_TOKEN"] = systemConnection.Authorization.Parameters[EndpointAuthorizationParameters.AccessToken];
|
||||||
|
}
|
||||||
|
|
||||||
// Resolve the target script.
|
// Resolve the target script.
|
||||||
string target = null;
|
string target = null;
|
||||||
@@ -69,6 +74,20 @@ namespace GitHub.Runner.Worker.Handlers
|
|||||||
target = Data.Post;
|
target = Data.Post;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// Add Telemetry to JobContext to send with JobCompleteMessage
|
||||||
|
if (stage == ActionRunStage.Main)
|
||||||
|
{
|
||||||
|
var telemetry = new ActionsStepTelemetry
|
||||||
|
{
|
||||||
|
Ref = GetActionRef(),
|
||||||
|
HasPreStep = Data.HasPre,
|
||||||
|
HasPostStep = Data.HasPost,
|
||||||
|
IsEmbedded = ExecutionContext.IsEmbedded,
|
||||||
|
Type = "node12"
|
||||||
|
};
|
||||||
|
ExecutionContext.Root.ActionsStepsTelemetry.Add(telemetry);
|
||||||
|
}
|
||||||
|
|
||||||
ArgUtil.NotNullOrEmpty(target, nameof(target));
|
ArgUtil.NotNullOrEmpty(target, nameof(target));
|
||||||
target = Path.Combine(ActionDirectory, target);
|
target = Path.Combine(ActionDirectory, target);
|
||||||
ArgUtil.File(target, nameof(target));
|
ArgUtil.File(target, nameof(target));
|
||||||
|
|||||||
@@ -23,18 +23,6 @@ namespace GitHub.Runner.Worker.Handlers
|
|||||||
|
|
||||||
public override void PrintActionDetails(ActionRunStage stage)
|
public override void PrintActionDetails(ActionRunStage stage)
|
||||||
{
|
{
|
||||||
// We don't want to display the internal workings if composite (similar/equivalent information can be found in debug)
|
|
||||||
void writeDetails(string message)
|
|
||||||
{
|
|
||||||
if (ExecutionContext.IsEmbedded)
|
|
||||||
{
|
|
||||||
ExecutionContext.Debug(message);
|
|
||||||
}
|
|
||||||
else
|
|
||||||
{
|
|
||||||
ExecutionContext.Output(message);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
if (stage == ActionRunStage.Post)
|
if (stage == ActionRunStage.Post)
|
||||||
{
|
{
|
||||||
@@ -52,7 +40,7 @@ namespace GitHub.Runner.Worker.Handlers
|
|||||||
firstLine = firstLine.Substring(0, firstNewLine);
|
firstLine = firstLine.Substring(0, firstNewLine);
|
||||||
}
|
}
|
||||||
|
|
||||||
writeDetails(ExecutionContext.IsEmbedded ? $"Run {firstLine}" : $"##[group]Run {firstLine}");
|
ExecutionContext.Output($"##[group]Run {firstLine}");
|
||||||
}
|
}
|
||||||
else
|
else
|
||||||
{
|
{
|
||||||
@@ -63,7 +51,7 @@ namespace GitHub.Runner.Worker.Handlers
|
|||||||
foreach (var line in multiLines)
|
foreach (var line in multiLines)
|
||||||
{
|
{
|
||||||
// Bright Cyan color
|
// Bright Cyan color
|
||||||
writeDetails($"\x1b[36;1m{line}\x1b[0m");
|
ExecutionContext.Output($"\x1b[36;1m{line}\x1b[0m");
|
||||||
}
|
}
|
||||||
|
|
||||||
string argFormat;
|
string argFormat;
|
||||||
@@ -122,23 +110,23 @@ namespace GitHub.Runner.Worker.Handlers
|
|||||||
|
|
||||||
if (!string.IsNullOrEmpty(shellCommandPath))
|
if (!string.IsNullOrEmpty(shellCommandPath))
|
||||||
{
|
{
|
||||||
writeDetails($"shell: {shellCommandPath} {argFormat}");
|
ExecutionContext.Output($"shell: {shellCommandPath} {argFormat}");
|
||||||
}
|
}
|
||||||
else
|
else
|
||||||
{
|
{
|
||||||
writeDetails($"shell: {shellCommand} {argFormat}");
|
ExecutionContext.Output($"shell: {shellCommand} {argFormat}");
|
||||||
}
|
}
|
||||||
|
|
||||||
if (this.Environment?.Count > 0)
|
if (this.Environment?.Count > 0)
|
||||||
{
|
{
|
||||||
writeDetails("env:");
|
ExecutionContext.Output("env:");
|
||||||
foreach (var env in this.Environment)
|
foreach (var env in this.Environment)
|
||||||
{
|
{
|
||||||
writeDetails($" {env.Key}: {env.Value}");
|
ExecutionContext.Output($" {env.Key}: {env.Value}");
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
writeDetails(ExecutionContext.IsEmbedded ? "" : "##[endgroup]");
|
ExecutionContext.Output("##[endgroup]");
|
||||||
}
|
}
|
||||||
|
|
||||||
public async Task RunAsync(ActionRunStage stage)
|
public async Task RunAsync(ActionRunStage stage)
|
||||||
@@ -156,6 +144,17 @@ namespace GitHub.Runner.Worker.Handlers
|
|||||||
var githubContext = ExecutionContext.ExpressionValues["github"] as GitHubContext;
|
var githubContext = ExecutionContext.ExpressionValues["github"] as GitHubContext;
|
||||||
ArgUtil.NotNull(githubContext, nameof(githubContext));
|
ArgUtil.NotNull(githubContext, nameof(githubContext));
|
||||||
|
|
||||||
|
// Add Telemetry to JobContext to send with JobCompleteMessage
|
||||||
|
if (stage == ActionRunStage.Main)
|
||||||
|
{
|
||||||
|
var telemetry = new ActionsStepTelemetry
|
||||||
|
{
|
||||||
|
IsEmbedded = ExecutionContext.IsEmbedded,
|
||||||
|
Type = "run",
|
||||||
|
};
|
||||||
|
ExecutionContext.Root.ActionsStepsTelemetry.Add(telemetry);
|
||||||
|
}
|
||||||
|
|
||||||
var tempDirectory = HostContext.GetDirectory(WellKnownDirectory.Temp);
|
var tempDirectory = HostContext.GetDirectory(WellKnownDirectory.Temp);
|
||||||
|
|
||||||
Inputs.TryGetValue("script", out var contents);
|
Inputs.TryGetValue("script", out var contents);
|
||||||
@@ -278,6 +277,13 @@ namespace GitHub.Runner.Worker.Handlers
|
|||||||
fileName = node12;
|
fileName = node12;
|
||||||
}
|
}
|
||||||
#endif
|
#endif
|
||||||
|
var systemConnection = ExecutionContext.Global.Endpoints.Single(x => string.Equals(x.Name, WellKnownServiceEndpointNames.SystemVssConnection, StringComparison.OrdinalIgnoreCase));
|
||||||
|
if (systemConnection.Data.TryGetValue("GenerateIdTokenUrl", out var generateIdTokenUrl) && !string.IsNullOrEmpty(generateIdTokenUrl))
|
||||||
|
{
|
||||||
|
Environment["ACTIONS_ID_TOKEN_REQUEST_URL"] = generateIdTokenUrl;
|
||||||
|
Environment["ACTIONS_ID_TOKEN_REQUEST_TOKEN"] = systemConnection.Authorization.Parameters[EndpointAuthorizationParameters.AccessToken];
|
||||||
|
}
|
||||||
|
|
||||||
ExecutionContext.Debug($"{fileName} {arguments}");
|
ExecutionContext.Debug($"{fileName} {arguments}");
|
||||||
|
|
||||||
using (var stdoutManager = new OutputManager(ExecutionContext, ActionCommandManager))
|
using (var stdoutManager = new OutputManager(ExecutionContext, ActionCommandManager))
|
||||||
|
|||||||
@@ -350,6 +350,7 @@ namespace GitHub.Runner.Worker
|
|||||||
case "":
|
case "":
|
||||||
case "ERROR":
|
case "ERROR":
|
||||||
case "WARNING":
|
case "WARNING":
|
||||||
|
case "NOTICE":
|
||||||
break;
|
break;
|
||||||
default:
|
default:
|
||||||
throw new ArgumentException($"Matcher '{_owner}' contains unexpected default severity '{_severity}'");
|
throw new ArgumentException($"Matcher '{_owner}' contains unexpected default severity '{_severity}'");
|
||||||
|
|||||||
@@ -216,7 +216,7 @@ namespace GitHub.Runner.Worker
|
|||||||
}
|
}
|
||||||
|
|
||||||
Trace.Info("Raising job completed event.");
|
Trace.Info("Raising job completed event.");
|
||||||
var jobCompletedEvent = new JobCompletedEvent(message.RequestId, message.JobId, result, jobContext.JobOutputs, jobContext.ActionsEnvironment);
|
var jobCompletedEvent = new JobCompletedEvent(message.RequestId, message.JobId, result, jobContext.JobOutputs, jobContext.ActionsEnvironment, jobContext.ActionsStepsTelemetry);
|
||||||
|
|
||||||
var completeJobRetryLimit = 5;
|
var completeJobRetryLimit = 5;
|
||||||
var exceptions = new List<Exception>();
|
var exceptions = new List<Exception>();
|
||||||
|
|||||||
@@ -354,43 +354,5 @@ namespace GitHub.Runner.Worker
|
|||||||
|
|
||||||
executionContext.Complete(result, resultCode: resultCode);
|
executionContext.Complete(result, resultCode: resultCode);
|
||||||
}
|
}
|
||||||
|
|
||||||
private sealed class ConditionTraceWriter : ObjectTemplating::ITraceWriter
|
|
||||||
{
|
|
||||||
private readonly IExecutionContext _executionContext;
|
|
||||||
private readonly Tracing _trace;
|
|
||||||
private readonly StringBuilder _traceBuilder = new StringBuilder();
|
|
||||||
|
|
||||||
public string Trace => _traceBuilder.ToString();
|
|
||||||
|
|
||||||
public ConditionTraceWriter(Tracing trace, IExecutionContext executionContext)
|
|
||||||
{
|
|
||||||
ArgUtil.NotNull(trace, nameof(trace));
|
|
||||||
_trace = trace;
|
|
||||||
_executionContext = executionContext;
|
|
||||||
}
|
|
||||||
|
|
||||||
public void Error(string format, params Object[] args)
|
|
||||||
{
|
|
||||||
var message = StringUtil.Format(format, args);
|
|
||||||
_trace.Error(message);
|
|
||||||
_executionContext?.Debug(message);
|
|
||||||
}
|
|
||||||
|
|
||||||
public void Info(string format, params Object[] args)
|
|
||||||
{
|
|
||||||
var message = StringUtil.Format(format, args);
|
|
||||||
_trace.Info(message);
|
|
||||||
_executionContext?.Debug(message);
|
|
||||||
_traceBuilder.AppendLine(message);
|
|
||||||
}
|
|
||||||
|
|
||||||
public void Verbose(string format, params Object[] args)
|
|
||||||
{
|
|
||||||
var message = StringUtil.Format(format, args);
|
|
||||||
_trace.Verbose(message);
|
|
||||||
_executionContext?.Debug(message);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -2,6 +2,7 @@
|
|||||||
using System.ComponentModel;
|
using System.ComponentModel;
|
||||||
using System.Security;
|
using System.Security;
|
||||||
using System.Text;
|
using System.Text;
|
||||||
|
using System.Text.RegularExpressions;
|
||||||
using Newtonsoft.Json;
|
using Newtonsoft.Json;
|
||||||
|
|
||||||
namespace GitHub.DistributedTask.Logging
|
namespace GitHub.DistributedTask.Logging
|
||||||
@@ -80,6 +81,65 @@ namespace GitHub.DistributedTask.Logging
|
|||||||
return trimmed;
|
return trimmed;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
public static String PowerShellPreAmpersandEscape(String value)
|
||||||
|
{
|
||||||
|
// if the secret is passed to PS as a command and it causes an error, sections of it can be surrounded by color codes
|
||||||
|
// or printed individually.
|
||||||
|
|
||||||
|
// The secret secretpart1&secretpart2&secretpart3 would be split into 2 sections:
|
||||||
|
// 'secretpart1&secretpart2&' and 'secretpart3'. This method masks for the first section.
|
||||||
|
|
||||||
|
// The secret secretpart1&+secretpart2&secretpart3 would be split into 2 sections:
|
||||||
|
// 'secretpart1&+' and (no 's') 'ecretpart2&secretpart3'. This method masks for the first section.
|
||||||
|
|
||||||
|
var trimmed = string.Empty;
|
||||||
|
if (!string.IsNullOrEmpty(value) && value.Contains("&"))
|
||||||
|
{
|
||||||
|
var secretSection = string.Empty;
|
||||||
|
if (value.Contains("&+"))
|
||||||
|
{
|
||||||
|
secretSection = value.Substring(0, value.IndexOf("&+") + "&+".Length);
|
||||||
|
}
|
||||||
|
else
|
||||||
|
{
|
||||||
|
secretSection = value.Substring(0, value.LastIndexOf("&") + "&".Length);
|
||||||
|
}
|
||||||
|
|
||||||
|
// Don't mask short secrets
|
||||||
|
if (secretSection.Length >= 6)
|
||||||
|
{
|
||||||
|
trimmed = secretSection;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
return trimmed;
|
||||||
|
}
|
||||||
|
|
||||||
|
public static String PowerShellPostAmpersandEscape(String value)
|
||||||
|
{
|
||||||
|
var trimmed = string.Empty;
|
||||||
|
if (!string.IsNullOrEmpty(value) && value.Contains("&"))
|
||||||
|
{
|
||||||
|
var secretSection = string.Empty;
|
||||||
|
if (value.Contains("&+"))
|
||||||
|
{
|
||||||
|
// +1 to skip the letter that got colored
|
||||||
|
secretSection = value.Substring(value.IndexOf("&+") + "&+".Length + 1);
|
||||||
|
}
|
||||||
|
else
|
||||||
|
{
|
||||||
|
secretSection = value.Substring(value.LastIndexOf("&") + "&".Length);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (secretSection.Length >= 6)
|
||||||
|
{
|
||||||
|
trimmed = secretSection;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
return trimmed;
|
||||||
|
}
|
||||||
|
|
||||||
private static string Base64StringEscapeShift(String value, int shift)
|
private static string Base64StringEscapeShift(String value, int shift)
|
||||||
{
|
{
|
||||||
var bytes = Encoding.UTF8.GetBytes(value);
|
var bytes = Encoding.UTF8.GetBytes(value);
|
||||||
|
|||||||
@@ -18,5 +18,12 @@ namespace GitHub.DistributedTask.WebApi
|
|||||||
get;
|
get;
|
||||||
set;
|
set;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
[DataMember]
|
||||||
|
public string Path
|
||||||
|
{
|
||||||
|
get;
|
||||||
|
set;
|
||||||
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
36
src/Sdk/DTWebApi/WebApi/ActionsStepTelemetry.cs
Normal file
36
src/Sdk/DTWebApi/WebApi/ActionsStepTelemetry.cs
Normal file
@@ -0,0 +1,36 @@
|
|||||||
|
using System.Runtime.Serialization;
|
||||||
|
|
||||||
|
namespace GitHub.DistributedTask.WebApi
|
||||||
|
{
|
||||||
|
/// <summary>
|
||||||
|
/// Information about a step run on the runner
|
||||||
|
/// </summary>
|
||||||
|
[DataContract]
|
||||||
|
public class ActionsStepTelemetry
|
||||||
|
{
|
||||||
|
|
||||||
|
[DataMember(EmitDefaultValue = false)]
|
||||||
|
public string Ref { get; set; }
|
||||||
|
|
||||||
|
[DataMember(EmitDefaultValue = false)]
|
||||||
|
public string Type { get; set; }
|
||||||
|
|
||||||
|
[DataMember(EmitDefaultValue = false)]
|
||||||
|
public bool? HasRunsStep { get; set; }
|
||||||
|
|
||||||
|
[DataMember(EmitDefaultValue = false)]
|
||||||
|
public bool? HasUsesStep { get; set; }
|
||||||
|
|
||||||
|
[DataMember(EmitDefaultValue = false)]
|
||||||
|
public bool IsEmbedded { get; set; }
|
||||||
|
|
||||||
|
[DataMember(EmitDefaultValue = false)]
|
||||||
|
public bool? HasPreStep { get; set; }
|
||||||
|
|
||||||
|
[DataMember(EmitDefaultValue = false)]
|
||||||
|
public bool? HasPostStep { get; set; }
|
||||||
|
|
||||||
|
[DataMember(EmitDefaultValue = false)]
|
||||||
|
public int? StepCount { get; set; }
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -142,6 +142,19 @@ namespace GitHub.DistributedTask.WebApi
|
|||||||
this.ActionsEnvironment = actionsEnvironment;
|
this.ActionsEnvironment = actionsEnvironment;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
public JobCompletedEvent(
|
||||||
|
Int64 requestId,
|
||||||
|
Guid jobId,
|
||||||
|
TaskResult result,
|
||||||
|
Dictionary<String, VariableValue> outputs,
|
||||||
|
ActionsEnvironmentReference actionsEnvironment,
|
||||||
|
List<ActionsStepTelemetry> actionsStepsTelemetry)
|
||||||
|
: this(requestId, jobId, result, outputs)
|
||||||
|
{
|
||||||
|
this.ActionsEnvironment = actionsEnvironment;
|
||||||
|
this.ActionsStepsTelemetry = actionsStepsTelemetry;
|
||||||
|
}
|
||||||
|
|
||||||
[DataMember(EmitDefaultValue = false)]
|
[DataMember(EmitDefaultValue = false)]
|
||||||
public Int64 RequestId
|
public Int64 RequestId
|
||||||
{
|
{
|
||||||
@@ -169,6 +182,13 @@ namespace GitHub.DistributedTask.WebApi
|
|||||||
get;
|
get;
|
||||||
set;
|
set;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
[DataMember(EmitDefaultValue = false)]
|
||||||
|
public List<ActionsStepTelemetry> ActionsStepsTelemetry
|
||||||
|
{
|
||||||
|
get;
|
||||||
|
set;
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
[DataContract]
|
[DataContract]
|
||||||
|
|||||||
@@ -112,6 +112,36 @@ namespace GitHub.Runner.Common.Tests
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
[Theory]
|
||||||
|
[InlineData("secret&secret&secret", "secret&secret&\x0033[96msecret\x0033[0m", "***\x0033[96m***\x0033[0m")]
|
||||||
|
[InlineData("secret&secret+secret", "secret&\x0033[96msecret+secret\x0033[0m", "***\x0033[96m***\x0033[0m")]
|
||||||
|
[InlineData("secret+secret&secret", "secret+secret&\x0033[96msecret\x0033[0m", "***\x0033[96m***\x0033[0m")]
|
||||||
|
[InlineData("secret&secret&+secretsecret", "secret&secret&+\x0033[96ms\x0033[0mecretsecret", "***\x0033[96ms\x0033[0m***")]
|
||||||
|
[InlineData("secret&+secret&secret", "secret&+\x0033[96ms\x0033[0mecret&secret", "***\x0033[96ms\x0033[0m***")]
|
||||||
|
[InlineData("secret&+secret&+secret", "secret&+\x0033[96ms\x0033[0mecret&+secret", "***\x0033[96ms\x0033[0m***")]
|
||||||
|
[InlineData("secret&+secret&secret&+secret", "secret&+\x0033[96ms\x0033[0mecret&secret&+secret", "***\x0033[96ms\x0033[0m***")]
|
||||||
|
[Trait("Level", "L0")]
|
||||||
|
[Trait("Category", "Common")]
|
||||||
|
public void SecretSectionMasking(string secret, string rawOutput, string maskedOutput)
|
||||||
|
{
|
||||||
|
try
|
||||||
|
{
|
||||||
|
// Arrange.
|
||||||
|
Setup();
|
||||||
|
|
||||||
|
// Act.
|
||||||
|
_hc.SecretMasker.AddValue(secret);
|
||||||
|
|
||||||
|
// Assert.
|
||||||
|
Assert.Equal(maskedOutput, _hc.SecretMasker.MaskSecrets(rawOutput));
|
||||||
|
}
|
||||||
|
finally
|
||||||
|
{
|
||||||
|
// Cleanup.
|
||||||
|
Teardown();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
[Fact]
|
[Fact]
|
||||||
[Trait("Level", "L0")]
|
[Trait("Level", "L0")]
|
||||||
[Trait("Category", "Common")]
|
[Trait("Category", "Common")]
|
||||||
|
|||||||
@@ -877,6 +877,7 @@ namespace GitHub.Runner.Common.Tests.Worker
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
};
|
};
|
||||||
|
_hc.EnqueueInstance<IActionRunner>(new Mock<IActionRunner>().Object);
|
||||||
|
|
||||||
//Act
|
//Act
|
||||||
var steps = (await _actionManager.PrepareActionsAsync(_ec.Object, actions)).ContainerSetupSteps;
|
var steps = (await _actionManager.PrepareActionsAsync(_ec.Object, actions)).ContainerSetupSteps;
|
||||||
@@ -967,8 +968,7 @@ namespace GitHub.Runner.Common.Tests.Worker
|
|||||||
var result = await _actionManager.PrepareActionsAsync(_ec.Object, actions);
|
var result = await _actionManager.PrepareActionsAsync(_ec.Object, actions);
|
||||||
|
|
||||||
//Assert
|
//Assert
|
||||||
// TODO: Update this test
|
Assert.Equal(1, result.PreStepTracker.Count);
|
||||||
Assert.Equal(0, result.PreStepTracker.Count);
|
|
||||||
|
|
||||||
}
|
}
|
||||||
finally
|
finally
|
||||||
|
|||||||
@@ -61,8 +61,8 @@ namespace GitHub.Runner.Common.Tests.Worker
|
|||||||
_actionRunner.Action = action;
|
_actionRunner.Action = action;
|
||||||
|
|
||||||
Dictionary<string, string> finialInputs = new Dictionary<string, string>();
|
Dictionary<string, string> finialInputs = new Dictionary<string, string>();
|
||||||
_handlerFactory.Setup(x => x.Create(It.IsAny<IExecutionContext>(), It.IsAny<ActionStepDefinitionReference>(), It.IsAny<IStepHost>(), It.IsAny<ActionExecutionData>(), It.IsAny<Dictionary<string, string>>(), It.IsAny<Dictionary<string, string>>(), It.IsAny<Variables>(), It.IsAny<string>()))
|
_handlerFactory.Setup(x => x.Create(It.IsAny<IExecutionContext>(), It.IsAny<ActionStepDefinitionReference>(), It.IsAny<IStepHost>(), It.IsAny<ActionExecutionData>(), It.IsAny<Dictionary<string, string>>(), It.IsAny<Dictionary<string, string>>(), It.IsAny<Variables>(), It.IsAny<string>(), It.IsAny<List<JobExtensionRunner>>()))
|
||||||
.Callback((IExecutionContext executionContext, Pipelines.ActionStepDefinitionReference actionReference, IStepHost stepHost, ActionExecutionData data, Dictionary<string, string> inputs, Dictionary<string, string> environment, Variables runtimeVariables, string taskDirectory) =>
|
.Callback((IExecutionContext executionContext, Pipelines.ActionStepDefinitionReference actionReference, IStepHost stepHost, ActionExecutionData data, Dictionary<string, string> inputs, Dictionary<string, string> environment, Variables runtimeVariables, string taskDirectory, List<JobExtensionRunner> localActionContainerSetupSteps) =>
|
||||||
{
|
{
|
||||||
finialInputs = inputs;
|
finialInputs = inputs;
|
||||||
})
|
})
|
||||||
@@ -107,8 +107,8 @@ namespace GitHub.Runner.Common.Tests.Worker
|
|||||||
_actionRunner.Action = action;
|
_actionRunner.Action = action;
|
||||||
|
|
||||||
Dictionary<string, string> finialInputs = new Dictionary<string, string>();
|
Dictionary<string, string> finialInputs = new Dictionary<string, string>();
|
||||||
_handlerFactory.Setup(x => x.Create(It.IsAny<IExecutionContext>(), It.IsAny<ActionStepDefinitionReference>(), It.IsAny<IStepHost>(), It.IsAny<ActionExecutionData>(), It.IsAny<Dictionary<string, string>>(), It.IsAny<Dictionary<string, string>>(), It.IsAny<Variables>(), It.IsAny<string>()))
|
_handlerFactory.Setup(x => x.Create(It.IsAny<IExecutionContext>(), It.IsAny<ActionStepDefinitionReference>(), It.IsAny<IStepHost>(), It.IsAny<ActionExecutionData>(), It.IsAny<Dictionary<string, string>>(), It.IsAny<Dictionary<string, string>>(), It.IsAny<Variables>(), It.IsAny<string>(), It.IsAny<List<JobExtensionRunner>>()))
|
||||||
.Callback((IExecutionContext executionContext, Pipelines.ActionStepDefinitionReference actionReference, IStepHost stepHost, ActionExecutionData data, Dictionary<string, string> inputs, Dictionary<string, string> environment, Variables runtimeVariables, string taskDirectory) =>
|
.Callback((IExecutionContext executionContext, Pipelines.ActionStepDefinitionReference actionReference, IStepHost stepHost, ActionExecutionData data, Dictionary<string, string> inputs, Dictionary<string, string> environment, Variables runtimeVariables, string taskDirectory, List<JobExtensionRunner> localActionContainerSetupSteps) =>
|
||||||
{
|
{
|
||||||
finialInputs = inputs;
|
finialInputs = inputs;
|
||||||
})
|
})
|
||||||
@@ -308,8 +308,8 @@ namespace GitHub.Runner.Common.Tests.Worker
|
|||||||
_actionRunner.Action = action;
|
_actionRunner.Action = action;
|
||||||
|
|
||||||
Dictionary<string, string> finialInputs = new Dictionary<string, string>();
|
Dictionary<string, string> finialInputs = new Dictionary<string, string>();
|
||||||
_handlerFactory.Setup(x => x.Create(It.IsAny<IExecutionContext>(), It.IsAny<ActionStepDefinitionReference>(), It.IsAny<IStepHost>(), It.IsAny<ActionExecutionData>(), It.IsAny<Dictionary<string, string>>(), It.IsAny<Dictionary<string, string>>(), It.IsAny<Variables>(), It.IsAny<string>()))
|
_handlerFactory.Setup(x => x.Create(It.IsAny<IExecutionContext>(), It.IsAny<ActionStepDefinitionReference>(), It.IsAny<IStepHost>(), It.IsAny<ActionExecutionData>(), It.IsAny<Dictionary<string, string>>(), It.IsAny<Dictionary<string, string>>(), It.IsAny<Variables>(), It.IsAny<string>(), It.IsAny<List<JobExtensionRunner>>()))
|
||||||
.Callback((IExecutionContext executionContext, Pipelines.ActionStepDefinitionReference actionReference, IStepHost stepHost, ActionExecutionData data, Dictionary<string, string> inputs, Dictionary<string, string> environment, Variables runtimeVariables, string taskDirectory) =>
|
.Callback((IExecutionContext executionContext, Pipelines.ActionStepDefinitionReference actionReference, IStepHost stepHost, ActionExecutionData data, Dictionary<string, string> inputs, Dictionary<string, string> environment, Variables runtimeVariables, string taskDirectory, List<JobExtensionRunner> localActionContainerSetupSteps) =>
|
||||||
{
|
{
|
||||||
finialInputs = inputs;
|
finialInputs = inputs;
|
||||||
})
|
})
|
||||||
@@ -359,8 +359,8 @@ namespace GitHub.Runner.Common.Tests.Worker
|
|||||||
_actionRunner.Action = action;
|
_actionRunner.Action = action;
|
||||||
|
|
||||||
Dictionary<string, string> finialInputs = new Dictionary<string, string>();
|
Dictionary<string, string> finialInputs = new Dictionary<string, string>();
|
||||||
_handlerFactory.Setup(x => x.Create(It.IsAny<IExecutionContext>(), It.IsAny<ActionStepDefinitionReference>(), It.IsAny<IStepHost>(), It.IsAny<ActionExecutionData>(), It.IsAny<Dictionary<string, string>>(), It.IsAny<Dictionary<string, string>>(), It.IsAny<Variables>(), It.IsAny<string>()))
|
_handlerFactory.Setup(x => x.Create(It.IsAny<IExecutionContext>(), It.IsAny<ActionStepDefinitionReference>(), It.IsAny<IStepHost>(), It.IsAny<ActionExecutionData>(), It.IsAny<Dictionary<string, string>>(), It.IsAny<Dictionary<string, string>>(), It.IsAny<Variables>(), It.IsAny<string>(), It.IsAny<List<JobExtensionRunner>>()))
|
||||||
.Callback((IExecutionContext executionContext, Pipelines.ActionStepDefinitionReference actionReference, IStepHost stepHost, ActionExecutionData data, Dictionary<string, string> inputs, Dictionary<string, string> environment, Variables runtimeVariables, string taskDirectory) =>
|
.Callback((IExecutionContext executionContext, Pipelines.ActionStepDefinitionReference actionReference, IStepHost stepHost, ActionExecutionData data, Dictionary<string, string> inputs, Dictionary<string, string> environment, Variables runtimeVariables, string taskDirectory, List<JobExtensionRunner> localActionContainerSetupSteps) =>
|
||||||
{
|
{
|
||||||
finialInputs = inputs;
|
finialInputs = inputs;
|
||||||
})
|
})
|
||||||
|
|||||||
@@ -392,6 +392,35 @@ namespace GitHub.Runner.Common.Tests.Worker
|
|||||||
Assert.Equal("not-working", match.Message);
|
Assert.Equal("not-working", match.Message);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
[Fact]
|
||||||
|
[Trait("Level", "L0")]
|
||||||
|
[Trait("Category", "Worker")]
|
||||||
|
public void Matcher_MultiplePatterns_DefaultSeverityNotice()
|
||||||
|
{
|
||||||
|
var config = JsonUtility.FromString<IssueMatchersConfig>(@"
|
||||||
|
{
|
||||||
|
""problemMatcher"": [
|
||||||
|
{
|
||||||
|
""owner"": ""myMatcher"",
|
||||||
|
""severity"": ""notice"",
|
||||||
|
""pattern"": [
|
||||||
|
{
|
||||||
|
""regexp"": ""^(.+)$"",
|
||||||
|
""message"": 1
|
||||||
|
}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
");
|
||||||
|
config.Validate();
|
||||||
|
var matcher = new IssueMatcher(config.Matchers[0], TimeSpan.FromSeconds(1));
|
||||||
|
|
||||||
|
var match = matcher.Match("just-a-notice");
|
||||||
|
Assert.Equal("notice", match.Severity);
|
||||||
|
Assert.Equal("just-a-notice", match.Message);
|
||||||
|
}
|
||||||
|
|
||||||
[Fact]
|
[Fact]
|
||||||
[Trait("Level", "L0")]
|
[Trait("Level", "L0")]
|
||||||
[Trait("Category", "Worker")]
|
[Trait("Category", "Worker")]
|
||||||
|
|||||||
@@ -141,7 +141,7 @@ namespace GitHub.Runner.Common.Tests.Worker
|
|||||||
var jobExtension = new JobExtension();
|
var jobExtension = new JobExtension();
|
||||||
jobExtension.Initialize(hc);
|
jobExtension.Initialize(hc);
|
||||||
|
|
||||||
_actionManager.Setup(x => x.PrepareActionsAsync(It.IsAny<IExecutionContext>(), It.IsAny<IEnumerable<Pipelines.JobStep>>()))
|
_actionManager.Setup(x => x.PrepareActionsAsync(It.IsAny<IExecutionContext>(), It.IsAny<IEnumerable<Pipelines.JobStep>>(), It.IsAny<Guid>()))
|
||||||
.Returns(Task.FromResult(new PrepareResult(new List<JobExtensionRunner>(), new Dictionary<Guid, IActionRunner>())));
|
.Returns(Task.FromResult(new PrepareResult(new List<JobExtensionRunner>(), new Dictionary<Guid, IActionRunner>())));
|
||||||
|
|
||||||
List<IStep> result = await jobExtension.InitializeJob(_jobEc, _message);
|
List<IStep> result = await jobExtension.InitializeJob(_jobEc, _message);
|
||||||
@@ -176,7 +176,7 @@ namespace GitHub.Runner.Common.Tests.Worker
|
|||||||
var jobExtension = new JobExtension();
|
var jobExtension = new JobExtension();
|
||||||
jobExtension.Initialize(hc);
|
jobExtension.Initialize(hc);
|
||||||
|
|
||||||
_actionManager.Setup(x => x.PrepareActionsAsync(It.IsAny<IExecutionContext>(), It.IsAny<IEnumerable<Pipelines.JobStep>>()))
|
_actionManager.Setup(x => x.PrepareActionsAsync(It.IsAny<IExecutionContext>(), It.IsAny<IEnumerable<Pipelines.JobStep>>(), It.IsAny<Guid>()))
|
||||||
.Returns(Task.FromResult(new PrepareResult(new List<JobExtensionRunner>() { new JobExtensionRunner(null, "", "prepare1", null), new JobExtensionRunner(null, "", "prepare2", null) }, new Dictionary<Guid, IActionRunner>())));
|
.Returns(Task.FromResult(new PrepareResult(new List<JobExtensionRunner>() { new JobExtensionRunner(null, "", "prepare1", null), new JobExtensionRunner(null, "", "prepare2", null) }, new Dictionary<Guid, IActionRunner>())));
|
||||||
|
|
||||||
List<IStep> result = await jobExtension.InitializeJob(_jobEc, _message);
|
List<IStep> result = await jobExtension.InitializeJob(_jobEc, _message);
|
||||||
|
|||||||
@@ -1 +1 @@
|
|||||||
2.279.0
|
2.281.1
|
||||||
|
|||||||
Reference in New Issue
Block a user