Compare commits

...

23 Commits

Author SHA1 Message Date
Thomas Boop
2247f2e3ea 2.160.1 Runner Release Notes (#171)
* 2.160.1 Runner Release Notes

* Minor verbiage updates to be consistent
2019-11-05 15:43:40 -05:00
Thomas Boop
81fe044a22 Cache Feature Bug: Job Container stops before post actions are run breaking Cache (#167)
* stop job container after all post actions.

* c

* c
2019-11-05 15:43:05 -05:00
Thomas Boop
3ba55f86a9 add-mask is leaking a secret in master if debug or ::echo::on is set (#158)
* Output after processing command to avoid leaking mask

* Remove extra noise output from echo changes

* Omit Echoing of add-mask command

* avoid echoing on debug/warning/error
2019-11-04 14:31:21 -05:00
eric sciple
61f9d3f656 cherry picks from master (#156)
* Update dotnet install script (#155)

* root search pattern for hashfiles and allow forward slash on windows (#151)
2019-10-30 14:46:40 -04:00
Thomas Boop
08b9f6e045 Merge release 160.0 into master (#153)
* Update to Version 2.160.0 (#144)

* Revert "remove issue generation on warning/error commands (#137)" (#147)

* Revert "remove issue generation on warning/error commands (#137)"

This reverts commit 53da198867.

* Updated Release notes

* Users/thboop/port directory changes (#152)

* Clear action cache for local runner

* update release notes for actions directory cache changes
2019-10-28 15:45:27 -04:00
eric sciple
0129e8111f Clear action cache for local runner (#148) 2019-10-28 11:56:12 -04:00
eric sciple
ccca13ac07 restrict hashFiles to basic globbing and globstar (#150) 2019-10-28 11:52:22 -04:00
Julio Barba
82e9857d4f Implement new echo behavior and command (#139)
* Remove controlling echoing by command

* Add 'echo on' and 'echo off' action commands

* PR feedback and add L0 tests

* Register new command

* Eric's PR feedback

* Tweak logging a bit

* Rename EchoOnActionCommandSuccess -> EchoOnActionCommand

* More PR reaction

* Make warning messages in Action Commands not rely on context from echo commands
2019-10-25 10:38:56 -04:00
Tingluo Huang
afd233b735 consume dotnet core 3.0 (#127)
* consume dotnet core 3.0

* update linux dependency doc.
2019-10-24 16:52:29 -04:00
Thomas Boop
83be145bfd Update build.yml 2019-10-24 14:05:10 -04:00
Thomas Boop
6e20aceaff Update build.yml 2019-10-24 14:00:56 -04:00
Lucas Killgore
e89148e33e Don't retry uploads when the http status code response from the server is in the 400's (#131)
* Don't retry uploads when the http status code
response from the server is in the 400's

* Don't retry on fast-fail

* Feedback from code review

* Always try to attach any uploaded files to the build
Don't fast-fail on 'Conflict'

* Add dispose

* Refactored upload code.
Called out specialized 'Conflict' logic.

* Added typed exception.
2019-10-23 12:48:10 -04:00
Thomas Boop
f1c58c33b6 Default to pwsh with powershell as a fallback (#142)
* Default to pwsh with powershell as a fallback

* Update releaseNote.md

* clean up code
2019-10-23 11:24:02 -04:00
Thomas Boop
7b158fff05 Revert "Need to update the release pool (#138)" (#141)
This reverts commit 3a43d853be.
2019-10-22 13:41:55 -04:00
Thomas Boop
6225b22870 Update to Version 2.159.2 (#140) 2019-10-22 13:14:58 -04:00
Thomas Boop
53da198867 remove issue generation on warning/error commands (#137) 2019-10-21 16:19:57 -04:00
Thomas Boop
3a43d853be Need to update the release pool (#138) 2019-10-21 15:32:57 -04:00
Thomas Boop
9d7b0a2405 set default shell to powershell in windows (#135)
* set default shell to powershell in windows

* Update error checking

* Update error message
2019-10-21 14:08:12 -04:00
Thomas Boop
05f0b938ac update version to 2.159.1 (#133)
* update version to 2.159.1

* Update releaseNote.md
2019-10-17 17:05:16 -04:00
Tingluo Huang
f5f14d4811 Runner register labels during configuration (#130)
* Runners will add os and architecture labels during registration

* support github.localhost for dev.
2019-10-17 16:33:43 -04:00
Tingluo Huang
2f261f2c31 Update build.yml 2019-10-14 16:14:22 -04:00
Thomas Boop
c3b11b36e9 Remove unneeded base64 encoders (#128)
* Remove unneeded base64 encoders

* update comment
2019-10-14 10:47:41 -04:00
Tingluo Huang
8777686c11 L0 for HostContext default masker. (#129) 2019-10-14 10:47:20 -04:00
55 changed files with 798 additions and 850 deletions

View File

@@ -4,6 +4,7 @@ on:
push: push:
branches: branches:
- master - master
- releases/*
pull_request: pull_request:
branches: branches:
- '*' - '*'
@@ -19,7 +20,7 @@ jobs:
- os: macOS-latest - os: macOS-latest
devScript: ./dev.sh devScript: ./dev.sh
- os: windows-latest - os: windows-latest
devScript: dev.cmd devScript: ./dev
runs-on: ${{ matrix.os }} runs-on: ${{ matrix.os }}
steps: steps:
- uses: actions/checkout@v1 - uses: actions/checkout@v1

View File

@@ -5,36 +5,55 @@
## Supported Distributions and Versions ## Supported Distributions and Versions
x64 x64
- Red Hat Enterprise Linux 6 (see note 1), 7 - Red Hat Enterprise Linux 7
- CentOS 6 (see note 1), 7 - CentOS 7
- Oracle Linux 7 - Oracle Linux 7
- Fedora 28, 27 - Fedora 29+
- Debian 9, 8.7 or later versions - Debian 9+
- Ubuntu 18.04, Ubuntu 16.04, Ubuntu 14.04 - Ubuntu 16.04+
- Linux Mint 18, 17 - Linux Mint 18+
- openSUSE 42.3 or later versions - openSUSE 15+
- SUSE Enterprise Linux (SLES) 12 SP2 or later versions - SUSE Enterprise Linux (SLES) 12 SP2+
ARM32 (see note 2) ## Install .Net Core 3.x Linux Dependencies
- Debian 9 or later versions
- Ubuntu 18.04 or later versions
> Note 1: Red Hat Enterprise Linux 6 and CentOS 6 require installing the specialized "rhel.6-x64" agent package The `./config.sh` will check .Net Core 3.x dependencies during runner configuration.
> Note 2: ARM instruction set [ARMv7](https://en.wikipedia.org/wiki/List_of_ARM_microarchitectures) or above is required, you can get your device's information by executing `uname -a`
## Install .Net Core 2.x Linux Dependencies
The `./config.sh` will check .Net Core 2.x dependencies during agent configuration.
You might see something like this which indicate a dependency's missing. You might see something like this which indicate a dependency's missing.
```bash ```bash
./config.sh ./config.sh
libunwind.so.8 => not found libunwind.so.8 => not found
libunwind-x86_64.so.8 => not found libunwind-x86_64.so.8 => not found
Dependencies is missing for Dotnet Core 2.1 Dependencies is missing for Dotnet Core 3.0
Execute ./bin/installdependencies.sh to install any missing Dotnet Core 2.1 dependencies. Execute ./bin/installdependencies.sh to install any missing Dotnet Core 3.0 dependencies.
``` ```
You can easily correct the problem by executing `./bin/installdependencies.sh`. You can easily correct the problem by executing `./bin/installdependencies.sh`.
The `installdependencies.sh` script should install all required dependencies on all supported Linux versions The `installdependencies.sh` script should install all required dependencies on all supported Linux versions
> Note: The `installdependencies.sh` script will try to use the default package management mechanism on your Linux flavor (ex. `yum`/`apt-get`/`apt`). You might need to deal with error coming from the package management mechanism related to your setup, like [#1353](https://github.com/Microsoft/vsts-agent/issues/1353) > Note: The `installdependencies.sh` script will try to use the default package management mechanism on your Linux flavor (ex. `yum`/`apt-get`/`apt`). You might need to deal with error coming from the package management mechanism related to your setup, like [#1353](https://github.com/Microsoft/vsts-agent/issues/1353)
### Full dependencies list
Debian based OS (Debian, Ubuntu, Linux Mint)
- liblttng-ust0
- libkrb5-3
- zlib1g
- libssl1.1, libssl1.0.2 or libssl1.0.0
- libicu63, libicu60, libicu57 or libicu55
Fedora based OS (Fedora, Redhat, Centos, Oracle Linux 7)
- lttng-ust
- openssl-libs
- krb5-libs
- zlib
- libicu
SUSE based OS (OpenSUSE, SUSE Enterprise)
- lttng-ust
- libopenssl1_1
- krb5
- zlib
- libicu60_2
## [More .Net Core Prerequisites Information](https://docs.microsoft.com/en-us/dotnet/core/linux-prerequisites?tabs=netcore2x) ## [More .Net Core Prerequisites Information](https://docs.microsoft.com/en-us/dotnet/core/linux-prerequisites?tabs=netcore2x)

View File

@@ -1,16 +1,15 @@
## Features ## Features
- Runner config auth via GitHub.com. (#107) (#117) - Added commands to enable or disable echoing of commands (#139)
- Adding wrapper action to support post job cleanup, adding checkout v1.1 (#91)
- Improving terminal experience (#110)
- Add runner support for cache action. (#120)
## Bugs ## Bugs
- Set GITHUB_ACTIONS in containers. (#119) - Do not retry uploads on 4xx Errors for Artifact Upload Service (#131)
- Fix issue data column/col mismatch. (#122) - Actions cache no longer incorrectly caches the action if the tag was updated for self hosted runners (#148)
- Disabled echoing of commands on add-mask, debug, warning and error commands (#158)
- HashFile now is correctly configured to only support basic globbing and globstar (#149)
- HashFile now sets a default root and handles Windows paths correctly (#151)
## Misc ## Misc
- Use GitHub actions for CI/PR (#112) - N/A
- Code Cleanup (#123) (#124) (#125)
## Agent Downloads ## Agent Downloads

View File

@@ -37,10 +37,7 @@
.PARAMETER SharedRuntime .PARAMETER SharedRuntime
This parameter is obsolete and may be removed in a future version of this script. This parameter is obsolete and may be removed in a future version of this script.
The recommended alternative is '-Runtime dotnet'. The recommended alternative is '-Runtime dotnet'.
Default: false
Installs just the shared runtime bits, not the entire SDK. Installs just the shared runtime bits, not the entire SDK.
This is equivalent to specifying `-Runtime dotnet`.
.PARAMETER Runtime .PARAMETER Runtime
Installs just a shared runtime, not the entire SDK. Installs just a shared runtime, not the entire SDK.
Possible values: Possible values:
@@ -77,11 +74,15 @@
Skips installing non-versioned files if they already exist, such as dotnet.exe. Skips installing non-versioned files if they already exist, such as dotnet.exe.
.PARAMETER NoCdn .PARAMETER NoCdn
Disable downloading from the Azure CDN, and use the uncached feed directly. Disable downloading from the Azure CDN, and use the uncached feed directly.
.PARAMETER JSonFile
Determines the SDK version from a user specified global.json file
Note: global.json must have a value for 'SDK:Version'
#> #>
[cmdletbinding()] [cmdletbinding()]
param( param(
[string]$Channel="LTS", [string]$Channel="LTS",
[string]$Version="Latest", [string]$Version="Latest",
[string]$JSonFile,
[string]$InstallDir="<auto>", [string]$InstallDir="<auto>",
[string]$Architecture="<auto>", [string]$Architecture="<auto>",
[ValidateSet("dotnet", "aspnetcore", "windowsdesktop", IgnoreCase = $false)] [ValidateSet("dotnet", "aspnetcore", "windowsdesktop", IgnoreCase = $false)]
@@ -258,7 +259,6 @@ function GetHTTPResponse([Uri] $Uri)
}) })
} }
function Get-Latest-Version-Info([string]$AzureFeed, [string]$Channel, [bool]$Coherent) { function Get-Latest-Version-Info([string]$AzureFeed, [string]$Channel, [bool]$Coherent) {
Say-Invocation $MyInvocation Say-Invocation $MyInvocation
@@ -304,20 +304,64 @@ function Get-Latest-Version-Info([string]$AzureFeed, [string]$Channel, [bool]$Co
return $VersionInfo return $VersionInfo
} }
function Parse-Jsonfile-For-Version([string]$JSonFile) {
function Get-Specific-Version-From-Version([string]$AzureFeed, [string]$Channel, [string]$Version) {
Say-Invocation $MyInvocation Say-Invocation $MyInvocation
switch ($Version.ToLower()) { If (-Not (Test-Path $JSonFile)) {
{ $_ -eq "latest" } { throw "Unable to find '$JSonFile'"
$LatestVersionInfo = Get-Latest-Version-Info -AzureFeed $AzureFeed -Channel $Channel -Coherent $False exit 0
return $LatestVersionInfo.Version }
try {
$JSonContent = Get-Content($JSonFile) -Raw | ConvertFrom-Json | Select-Object -expand "sdk" -ErrorAction SilentlyContinue
}
catch {
throw "Json file unreadable: '$JSonFile'"
exit 0
}
if ($JSonContent) {
try {
$JSonContent.PSObject.Properties | ForEach-Object {
$PropertyName = $_.Name
if ($PropertyName -eq "version") {
$Version = $_.Value
Say-Verbose "Version = $Version"
}
}
} }
{ $_ -eq "coherent" } { catch {
$LatestVersionInfo = Get-Latest-Version-Info -AzureFeed $AzureFeed -Channel $Channel -Coherent $True throw "Unable to parse the SDK node in '$JSonFile'"
return $LatestVersionInfo.Version exit 0
} }
default { return $Version } }
else {
throw "Unable to find the SDK node in '$JSonFile'"
exit 0
}
If ($Version -eq $null) {
throw "Unable to find the SDK:version node in '$JSonFile'"
exit 0
}
return $Version
}
function Get-Specific-Version-From-Version([string]$AzureFeed, [string]$Channel, [string]$Version, [string]$JSonFile) {
Say-Invocation $MyInvocation
if (-not $JSonFile) {
switch ($Version.ToLower()) {
{ $_ -eq "latest" } {
$LatestVersionInfo = Get-Latest-Version-Info -AzureFeed $AzureFeed -Channel $Channel -Coherent $False
return $LatestVersionInfo.Version
}
{ $_ -eq "coherent" } {
$LatestVersionInfo = Get-Latest-Version-Info -AzureFeed $AzureFeed -Channel $Channel -Coherent $True
return $LatestVersionInfo.Version
}
default { return $Version }
}
}
else {
return Parse-Jsonfile-For-Version $JSonFile
} }
} }
@@ -382,23 +426,6 @@ function Resolve-Installation-Path([string]$InstallDir) {
return $InstallDir return $InstallDir
} }
function Get-Version-Info-From-Version-File([string]$InstallRoot, [string]$RelativePathToVersionFile) {
Say-Invocation $MyInvocation
$VersionFile = Join-Path -Path $InstallRoot -ChildPath $RelativePathToVersionFile
Say-Verbose "Local version file: $VersionFile"
if (Test-Path $VersionFile) {
$VersionText = cat $VersionFile
Say-Verbose "Local version file text: $VersionText"
return Get-Version-Info-From-Version-Text $VersionText
}
Say-Verbose "Local version file not found."
return $null
}
function Is-Dotnet-Package-Installed([string]$InstallRoot, [string]$RelativePathToPackage, [string]$SpecificVersion) { function Is-Dotnet-Package-Installed([string]$InstallRoot, [string]$RelativePathToPackage, [string]$SpecificVersion) {
Say-Invocation $MyInvocation Say-Invocation $MyInvocation
@@ -534,7 +561,7 @@ function Prepend-Sdk-InstallRoot-To-Path([string]$InstallRoot, [string]$BinFolde
} }
$CLIArchitecture = Get-CLIArchitecture-From-Architecture $Architecture $CLIArchitecture = Get-CLIArchitecture-From-Architecture $Architecture
$SpecificVersion = Get-Specific-Version-From-Version -AzureFeed $AzureFeed -Channel $Channel -Version $Version $SpecificVersion = Get-Specific-Version-From-Version -AzureFeed $AzureFeed -Channel $Channel -Version $Version -JSonFile $JSonFile
$DownloadLink = Get-Download-Link -AzureFeed $AzureFeed -SpecificVersion $SpecificVersion -CLIArchitecture $CLIArchitecture $DownloadLink = Get-Download-Link -AzureFeed $AzureFeed -SpecificVersion $SpecificVersion -CLIArchitecture $CLIArchitecture
$LegacyDownloadLink = Get-LegacyDownload-Link -AzureFeed $AzureFeed -SpecificVersion $SpecificVersion -CLIArchitecture $CLIArchitecture $LegacyDownloadLink = Get-LegacyDownload-Link -AzureFeed $AzureFeed -SpecificVersion $SpecificVersion -CLIArchitecture $CLIArchitecture

View File

@@ -435,11 +435,52 @@ get_latest_version_info() {
return $? return $?
} }
# args:
# json_file - $1
parse_jsonfile_for_version() {
eval $invocation
local json_file="$1"
if [ ! -f "$json_file" ]; then
say_err "Unable to find \`$json_file\`"
return 1
fi
sdk_section=$(cat $json_file | awk '/"sdk"/,/}/')
if [ -z "$sdk_section" ]; then
say_err "Unable to parse the SDK node in \`$json_file\`"
return 1
fi
sdk_list=$(echo $sdk_section | awk -F"[{}]" '{print $2}')
sdk_list=${sdk_list//[\" ]/}
sdk_list=${sdk_list//,/$'\n'}
sdk_list="$(echo -e "${sdk_list}" | tr -d '[[:space:]]')"
local version_info=""
while read -r line; do
IFS=:
while read -r key value; do
if [[ "$key" == "version" ]]; then
version_info=$value
fi
done <<< "$line"
done <<< "$sdk_list"
if [ -z "$version_info" ]; then
say_err "Unable to find the SDK:version node in \`$json_file\`"
return 1
fi
echo "$version_info"
return 0
}
# args: # args:
# azure_feed - $1 # azure_feed - $1
# channel - $2 # channel - $2
# normalized_architecture - $3 # normalized_architecture - $3
# version - $4 # version - $4
# json_file - $5
get_specific_version_from_version() { get_specific_version_from_version() {
eval $invocation eval $invocation
@@ -447,27 +488,35 @@ get_specific_version_from_version() {
local channel="$2" local channel="$2"
local normalized_architecture="$3" local normalized_architecture="$3"
local version="$(to_lowercase "$4")" local version="$(to_lowercase "$4")"
local json_file="$5"
case "$version" in if [ -z "$json_file" ]; then
latest) case "$version" in
local version_info latest)
version_info="$(get_latest_version_info "$azure_feed" "$channel" "$normalized_architecture" false)" || return 1 local version_info
say_verbose "get_specific_version_from_version: version_info=$version_info" version_info="$(get_latest_version_info "$azure_feed" "$channel" "$normalized_architecture" false)" || return 1
echo "$version_info" | get_version_from_version_info say_verbose "get_specific_version_from_version: version_info=$version_info"
return 0 echo "$version_info" | get_version_from_version_info
;; return 0
coherent) ;;
local version_info coherent)
version_info="$(get_latest_version_info "$azure_feed" "$channel" "$normalized_architecture" true)" || return 1 local version_info
say_verbose "get_specific_version_from_version: version_info=$version_info" version_info="$(get_latest_version_info "$azure_feed" "$channel" "$normalized_architecture" true)" || return 1
echo "$version_info" | get_version_from_version_info say_verbose "get_specific_version_from_version: version_info=$version_info"
return 0 echo "$version_info" | get_version_from_version_info
;; return 0
*) ;;
echo "$version" *)
return 0 echo "$version"
;; return 0
esac ;;
esac
else
local version_info
version_info="$(parse_jsonfile_for_version "$json_file")" || return 1
echo "$version_info"
return 0
fi
} }
# args: # args:
@@ -558,24 +607,6 @@ resolve_installation_path() {
return 0 return 0
} }
# args:
# install_root - $1
get_installed_version_info() {
eval $invocation
local install_root="$1"
local version_file="$(combine_paths "$install_root" "$local_version_file_relative_path")"
say_verbose "Local version file: $version_file"
if [ ! -z "$version_file" ] | [ -r "$version_file" ]; then
local version_info="$(cat "$version_file")"
echo "$version_info"
return 0
fi
say_verbose "Local version file not found."
return 0
}
# args: # args:
# relative_or_absolute_path - $1 # relative_or_absolute_path - $1
get_absolute_path() { get_absolute_path() {
@@ -724,7 +755,7 @@ calculate_vars() {
normalized_architecture="$(get_normalized_architecture_from_architecture "$architecture")" normalized_architecture="$(get_normalized_architecture_from_architecture "$architecture")"
say_verbose "normalized_architecture=$normalized_architecture" say_verbose "normalized_architecture=$normalized_architecture"
specific_version="$(get_specific_version_from_version "$azure_feed" "$channel" "$normalized_architecture" "$version")" specific_version="$(get_specific_version_from_version "$azure_feed" "$channel" "$normalized_architecture" "$version" "$json_file")"
say_verbose "specific_version=$specific_version" say_verbose "specific_version=$specific_version"
if [ -z "$specific_version" ]; then if [ -z "$specific_version" ]; then
say_err "Could not resolve version information." say_err "Could not resolve version information."
@@ -826,6 +857,7 @@ temporary_file_template="${TMPDIR:-/tmp}/dotnet.XXXXXXXXX"
channel="LTS" channel="LTS"
version="Latest" version="Latest"
json_file=""
install_dir="<auto>" install_dir="<auto>"
architecture="<auto>" architecture="<auto>"
dry_run=false dry_run=false
@@ -912,6 +944,10 @@ do
runtime_id="$1" runtime_id="$1"
non_dynamic_parameters+=" $name "\""$1"\""" non_dynamic_parameters+=" $name "\""$1"\"""
;; ;;
--jsonfile|-[Jj][Ss]on[Ff]ile)
shift
json_file="$1"
;;
--skip-non-versioned-files|-[Ss]kip[Nn]on[Vv]ersioned[Ff]iles) --skip-non-versioned-files|-[Ss]kip[Nn]on[Vv]ersioned[Ff]iles)
override_non_versioned_files=false override_non_versioned_files=false
non_dynamic_parameters+=" $name" non_dynamic_parameters+=" $name"
@@ -953,22 +989,25 @@ do
echo " Possible values:" echo " Possible values:"
echo " - dotnet - the Microsoft.NETCore.App shared runtime" echo " - dotnet - the Microsoft.NETCore.App shared runtime"
echo " - aspnetcore - the Microsoft.AspNetCore.App shared runtime" echo " - aspnetcore - the Microsoft.AspNetCore.App shared runtime"
echo " --skip-non-versioned-files Skips non-versioned files if they already exist, such as the dotnet executable."
echo " -SkipNonVersionedFiles"
echo " --dry-run,-DryRun Do not perform installation. Display download link." echo " --dry-run,-DryRun Do not perform installation. Display download link."
echo " --no-path, -NoPath Do not set PATH for the current process." echo " --no-path, -NoPath Do not set PATH for the current process."
echo " --verbose,-Verbose Display diagnostics information." echo " --verbose,-Verbose Display diagnostics information."
echo " --azure-feed,-AzureFeed Azure feed location. Defaults to $azure_feed, This parameter typically is not changed by the user." echo " --azure-feed,-AzureFeed Azure feed location. Defaults to $azure_feed, This parameter typically is not changed by the user."
echo " --uncached-feed,-UncachedFeed Uncached feed location. This parameter typically is not changed by the user." echo " --uncached-feed,-UncachedFeed Uncached feed location. This parameter typically is not changed by the user."
echo " --no-cdn,-NoCdn Disable downloading from the Azure CDN, and use the uncached feed directly."
echo " --feed-credential,-FeedCredential Azure feed shared access token. This parameter typically is not specified." echo " --feed-credential,-FeedCredential Azure feed shared access token. This parameter typically is not specified."
echo " --skip-non-versioned-files Skips non-versioned files if they already exist, such as the dotnet executable."
echo " -SkipNonVersionedFiles"
echo " --no-cdn,-NoCdn Disable downloading from the Azure CDN, and use the uncached feed directly."
echo " --jsonfile <JSONFILE> Determines the SDK version from a user specified global.json file."
echo " Note: global.json must have a value for 'SDK:Version'"
echo " --runtime-id Installs the .NET Tools for the given platform (use linux-x64 for portable linux)." echo " --runtime-id Installs the .NET Tools for the given platform (use linux-x64 for portable linux)."
echo " -RuntimeId" echo " -RuntimeId"
echo " -?,--?,-h,--help,-Help Shows this help message" echo " -?,--?,-h,--help,-Help Shows this help message"
echo "" echo ""
echo "Obsolete parameters:" echo "Obsolete parameters:"
echo " --shared-runtime The recommended alternative is '--runtime dotnet'." echo " --shared-runtime The recommended alternative is '--runtime dotnet'."
echo " -SharedRuntime Installs just the shared runtime bits, not the entire SDK." echo " This parameter is obsolete and may be removed in a future version of this script."
echo " Installs just the shared runtime bits, not the entire SDK."
echo "" echo ""
echo "Install Location:" echo "Install Location:"
echo " Location is chosen in following order:" echo " Location is chosen in following order:"

View File

@@ -60,10 +60,9 @@ then
print_errormessage print_errormessage
exit 1 exit 1
fi fi
# ubuntu 18 uses libcurl4 # libissl version prefer: libssl1.1 -> libssl1.0.2 -> libssl1.0.0
# ubuntu 14, 16 and other linux use libcurl3 apt install -y libssl1.1$ || apt install -y libssl1.0.2$ || apt install -y libssl1.0.0$
apt install -y libcurl3 || apt install -y libcurl4
if [ $? -ne 0 ] if [ $? -ne 0 ]
then then
echo "'apt' failed with exit code '$?'" echo "'apt' failed with exit code '$?'"
@@ -71,18 +70,8 @@ then
exit 1 exit 1
fi fi
# debian 9 use libssl1.0.2 # libicu version prefer: libicu63 -> libicu60 -> libicu57 -> libicu55 -> libicu52
# other debian linux use libssl1.0.0 apt install -y libicu63 || apt install -y libicu60 || apt install -y libicu57 || apt install -y libicu55 || apt install -y libicu52
apt install -y libssl1.0.0 || apt install -y libssl1.0.2
if [ $? -ne 0 ]
then
echo "'apt' failed with exit code '$?'"
print_errormessage
exit 1
fi
# libicu version prefer: libicu52 -> libicu55 -> libicu57 -> libicu60
apt install -y libicu52 || apt install -y libicu55 || apt install -y libicu57 || apt install -y libicu60
if [ $? -ne 0 ] if [ $? -ne 0 ]
then then
echo "'apt' failed with exit code '$?'" echo "'apt' failed with exit code '$?'"
@@ -101,9 +90,8 @@ then
exit 1 exit 1
fi fi
# ubuntu 18 uses libcurl4 # libissl version prefer: libssl1.1 -> libssl1.0.2 -> libssl1.0.0
# ubuntu 14, 16 and other linux use libcurl3 apt-get install -y libssl1.1$ || apt-get install -y libssl1.0.2$ || apt install -y libssl1.0.0$
apt-get install -y libcurl3 || apt-get install -y libcurl4
if [ $? -ne 0 ] if [ $? -ne 0 ]
then then
echo "'apt-get' failed with exit code '$?'" echo "'apt-get' failed with exit code '$?'"
@@ -111,18 +99,8 @@ then
exit 1 exit 1
fi fi
# debian 9 use libssl1.0.2 # libicu version prefer: libicu63 -> libicu60 -> libicu57 -> libicu55 -> libicu52
# other debian linux use libssl1.0.0 apt-get install -y libicu63 || apt-get install -y libicu60 || apt install -y libicu57 || apt install -y libicu55 || apt install -y libicu52
apt-get install -y libssl1.0.0 || apt install -y libssl1.0.2
if [ $? -ne 0 ]
then
echo "'apt-get' failed with exit code '$?'"
print_errormessage
exit 1
fi
# libicu version prefer: libicu52 -> libicu55 -> libicu57 -> libicu60
apt-get install -y libicu52 || apt install -y libicu55 || apt install -y libicu57 || apt install -y libicu60
if [ $? -ne 0 ] if [ $? -ne 0 ]
then then
echo "'apt-get' failed with exit code '$?'" echo "'apt-get' failed with exit code '$?'"
@@ -149,46 +127,7 @@ then
command -v dnf command -v dnf
if [ $? -eq 0 ] if [ $? -eq 0 ]
then then
useCompatSsl=0 dnf install -y lttng-ust openssl-libs krb5-libs zlib libicu
grep -i 'fedora release 28' /etc/fedora-release
if [ $? -eq 0 ]
then
useCompatSsl=1
else
grep -i 'fedora release 27' /etc/fedora-release
if [ $? -eq 0 ]
then
useCompatSsl=1
else
grep -i 'fedora release 26' /etc/fedora-release
if [ $? -eq 0 ]
then
useCompatSsl=1
fi
fi
fi
if [ $useCompatSsl -eq 1 ]
then
echo "Use compat-openssl10-devel instead of openssl-devel for Fedora 27/28 (dotnet core requires openssl 1.0.x)"
dnf install -y compat-openssl10
if [ $? -ne 0 ]
then
echo "'dnf' failed with exit code '$?'"
print_errormessage
exit 1
fi
else
dnf install -y openssl-libs
if [ $? -ne 0 ]
then
echo "'dnf' failed with exit code '$?'"
print_errormessage
exit 1
fi
fi
dnf install -y lttng-ust libcurl krb5-libs zlib libicu
if [ $? -ne 0 ] if [ $? -ne 0 ]
then then
echo "'dnf' failed with exit code '$?'" echo "'dnf' failed with exit code '$?'"
@@ -204,22 +143,13 @@ then
command -v yum command -v yum
if [ $? -eq 0 ] if [ $? -eq 0 ]
then then
yum install -y openssl-libs libcurl krb5-libs zlib libicu yum install -y lttng-ust openssl-libs krb5-libs zlib libicu
if [ $? -ne 0 ] if [ $? -ne 0 ]
then then
echo "'yum' failed with exit code '$?'" echo "'yum' failed with exit code '$?'"
print_errormessage print_errormessage
exit 1 exit 1
fi fi
# install lttng-ust separately since it's not part of offical package repository
yum install -y wget && wget -P /etc/yum.repos.d/ https://packages.efficios.com/repo.files/EfficiOS-RHEL7-x86-64.repo && rpmkeys --import https://packages.efficios.com/rhel/repo.key && yum updateinfo && yum install -y lttng-ust
if [ $? -ne 0 ]
then
echo "'lttng-ust' installation failed with exit code '$?'"
print_errormessage
exit 1
fi
else else
echo "Can not find 'yum'" echo "Can not find 'yum'"
print_errormessage print_errormessage
@@ -230,13 +160,14 @@ then
# we might on OpenSUSE # we might on OpenSUSE
OSTYPE=$(grep ID_LIKE /etc/os-release | cut -f2 -d=) OSTYPE=$(grep ID_LIKE /etc/os-release | cut -f2 -d=)
echo $OSTYPE echo $OSTYPE
if [ $OSTYPE == '"suse"' ] echo $OSTYPE | grep "suse"
if [ $? -eq 0 ]
then then
echo "The current OS is SUSE based" echo "The current OS is SUSE based"
command -v zypper command -v zypper
if [ $? -eq 0 ] if [ $? -eq 0 ]
then then
zypper -n install lttng-ust libopenssl1_0_0 libcurl4 krb5 zlib libicu52_1 zypper -n install lttng-ust libopenssl1_1 krb5 zlib libicu60_2
if [ $? -ne 0 ] if [ $? -ne 0 ]
then then
echo "'zypper' failed with exit code '$?'" echo "'zypper' failed with exit code '$?'"

View File

@@ -8,7 +8,7 @@ if [ $user_id -eq 0 -a -z "$AGENT_ALLOW_RUNASROOT" ]; then
exit 1 exit 1
fi fi
# Check dotnet core 2.1 dependencies for Linux # Check dotnet core 3.0 dependencies for Linux
if [[ (`uname` == "Linux") ]] if [[ (`uname` == "Linux") ]]
then then
command -v ldd > /dev/null command -v ldd > /dev/null
@@ -20,29 +20,22 @@ then
ldd ./bin/libcoreclr.so | grep 'not found' ldd ./bin/libcoreclr.so | grep 'not found'
if [ $? -eq 0 ]; then if [ $? -eq 0 ]; then
echo "Dependencies is missing for Dotnet Core 2.1" echo "Dependencies is missing for Dotnet Core 3.0"
echo "Execute ./bin/installdependencies.sh to install any missing Dotnet Core 2.1 dependencies." echo "Execute ./bin/installdependencies.sh to install any missing Dotnet Core 3.0 dependencies."
exit 1 exit 1
fi fi
ldd ./bin/System.Security.Cryptography.Native.OpenSsl.so | grep 'not found' ldd ./bin/System.Security.Cryptography.Native.OpenSsl.so | grep 'not found'
if [ $? -eq 0 ]; then if [ $? -eq 0 ]; then
echo "Dependencies is missing for Dotnet Core 2.1" echo "Dependencies is missing for Dotnet Core 3.0"
echo "Execute ./bin/installdependencies.sh to install any missing Dotnet Core 2.1 dependencies." echo "Execute ./bin/installdependencies.sh to install any missing Dotnet Core 3.0 dependencies."
exit 1 exit 1
fi fi
ldd ./bin/System.IO.Compression.Native.so | grep 'not found' ldd ./bin/System.IO.Compression.Native.so | grep 'not found'
if [ $? -eq 0 ]; then if [ $? -eq 0 ]; then
echo "Dependencies is missing for Dotnet Core 2.1" echo "Dependencies is missing for Dotnet Core 3.0"
echo "Execute ./bin/installdependencies.sh to install any missing Dotnet Core 2.1 dependencies." echo "Execute ./bin/installdependencies.sh to install any missing Dotnet Core 3.0 dependencies."
exit 1
fi
ldd ./bin/System.Net.Http.Native.so | grep 'not found'
if [ $? -eq 0 ]; then
echo "Dependencies is missing for Dotnet Core 2.1"
echo "Execute ./bin/installdependencies.sh to install any missing Dotnet Core 2.1 dependencies."
exit 1 exit 1
fi fi
@@ -59,8 +52,8 @@ then
libpath=${LD_LIBRARY_PATH:-} libpath=${LD_LIBRARY_PATH:-}
$LDCONFIG_COMMAND -NXv ${libpath//:/} 2>&1 | grep libicu >/dev/null 2>&1 $LDCONFIG_COMMAND -NXv ${libpath//:/} 2>&1 | grep libicu >/dev/null 2>&1
if [ $? -ne 0 ]; then if [ $? -ne 0 ]; then
echo "Libicu's dependencies is missing for Dotnet Core 2.1" echo "Libicu's dependencies is missing for Dotnet Core 3.0"
echo "Execute ./bin/installdependencies.sh to install any missing Dotnet Core 2.1 dependencies." echo "Execute ./bin/installdependencies.sh to install any missing Dotnet Core 3.0 dependencies."
exit 1 exit 1
fi fi
fi fi

View File

@@ -1,73 +0,0 @@
using System;
using System.Collections.Generic;
using System.Linq;
using System.Threading;
using System.Threading.Tasks;
using GitHub.Runner.Common.Util;
using GitHub.Runner.Sdk;
namespace GitHub.Runner.Common.Capabilities
{
[ServiceLocator(Default = typeof(CapabilitiesManager))]
public interface ICapabilitiesManager : IRunnerService
{
Task<Dictionary<string, string>> GetCapabilitiesAsync(RunnerSettings settings, CancellationToken token);
}
public sealed class CapabilitiesManager : RunnerService, ICapabilitiesManager
{
public async Task<Dictionary<string, string>> GetCapabilitiesAsync(RunnerSettings settings, CancellationToken cancellationToken)
{
Trace.Entering();
ArgUtil.NotNull(settings, nameof(settings));
// Initialize a dictionary of capabilities.
var capabilities = new Dictionary<string, string>(StringComparer.OrdinalIgnoreCase);
if (settings.SkipCapabilitiesScan)
{
Trace.Info("Skip capabilities scan.");
return capabilities;
}
// Get the providers.
var extensionManager = HostContext.GetService<IExtensionManager>();
IEnumerable<ICapabilitiesProvider> providers =
extensionManager
.GetExtensions<ICapabilitiesProvider>()
?.OrderBy(x => x.Order);
// Add each capability returned from each provider.
foreach (ICapabilitiesProvider provider in providers ?? new ICapabilitiesProvider[0])
{
foreach (Capability capability in await provider.GetCapabilitiesAsync(settings, cancellationToken) ?? new List<Capability>())
{
// Make sure we mask secrets in capabilities values.
capabilities[capability.Name] = HostContext.SecretMasker.MaskSecrets(capability.Value);
}
}
return capabilities;
}
}
public interface ICapabilitiesProvider : IExtension
{
int Order { get; }
Task<List<Capability>> GetCapabilitiesAsync(RunnerSettings settings, CancellationToken cancellationToken);
}
public sealed class Capability
{
public string Name { get; }
public string Value { get; }
public Capability(string name, string value)
{
ArgUtil.NotNullOrEmpty(name, nameof(name));
Name = name;
Value = value ?? string.Empty;
}
}
}

View File

@@ -1,86 +0,0 @@
using GitHub.Runner.Common.Util;
using GitHub.Runner.Sdk;
using Microsoft.Win32;
using System;
using System.Collections.Generic;
using System.Threading;
using System.Threading.Tasks;
namespace GitHub.Runner.Common.Capabilities
{
public sealed class RunnerCapabilitiesProvider : RunnerService, ICapabilitiesProvider
{
public Type ExtensionType => typeof(ICapabilitiesProvider);
public int Order => 99; // Process last to override prior.
public Task<List<Capability>> GetCapabilitiesAsync(RunnerSettings settings, CancellationToken cancellationToken)
{
ArgUtil.NotNull(settings, nameof(settings));
var capabilities = new List<Capability>();
Add(capabilities, "Runner.Name", settings.AgentName ?? string.Empty);
Add(capabilities, "Runner.OS", VarUtil.OS);
Add(capabilities, "Runner.OSArchitecture", VarUtil.OSArchitecture);
#if OS_WINDOWS
Add(capabilities, "Runner.OSVersion", GetOSVersionString());
#endif
Add(capabilities, "InteractiveSession", (HostContext.StartupType != StartupType.Service).ToString());
Add(capabilities, "Runner.Version", BuildConstants.RunnerPackage.Version);
Add(capabilities, "Runner.ComputerName", Environment.MachineName ?? string.Empty);
Add(capabilities, "Runner.HomeDirectory", HostContext.GetDirectory(WellKnownDirectory.Root));
return Task.FromResult(capabilities);
}
private void Add(List<Capability> capabilities, string name, string value)
{
Trace.Info($"Adding '{name}': '{value}'");
capabilities.Add(new Capability(name, value));
}
private object GetHklmValue(string keyName, string valueName)
{
keyName = $@"HKEY_LOCAL_MACHINE\{keyName}";
object value = Registry.GetValue(keyName, valueName, defaultValue: null);
if (object.ReferenceEquals(value, null))
{
Trace.Info($"Key name '{keyName}', value name '{valueName}' is null.");
return null;
}
Trace.Info($"Key name '{keyName}', value name '{valueName}': '{value}'");
return value;
}
private string GetOSVersionString()
{
// Do not use System.Environment.OSVersion.Version to resolve the OS version number.
// It leverages the GetVersionEx function which may report an incorrect version
// depending on the app's manifest. For details, see:
// https://msdn.microsoft.com/library/windows/desktop/ms724451(v=vs.85).aspx
// Attempt to retrieve the major/minor version from the new registry values added in
// in Windows 10.
//
// The registry value "CurrentVersion" is unreliable in Windows 10. It contains the
// value "6.3" instead of "10.0".
object major = GetHklmValue(@"SOFTWARE\Microsoft\Windows NT\CurrentVersion", "CurrentMajorVersionNumber");
object minor = GetHklmValue(@"SOFTWARE\Microsoft\Windows NT\CurrentVersion", "CurrentMinorVersionNumber");
string majorMinorString;
if (major != null && minor != null)
{
majorMinorString = StringUtil.Format("{0}.{1}", major, minor);
}
else
{
// Fallback to the registry value "CurrentVersion".
majorMinorString = GetHklmValue(@"SOFTWARE\Microsoft\Windows NT\CurrentVersion", "CurrentVersion") as string;
}
// Opted to use the registry value "CurrentBuildNumber" over "CurrentBuild". Based on brief
// internet investigation, the only difference appears to be that on Windows XP "CurrentBuild"
// was unreliable and "CurrentBuildNumber" was the correct choice.
string build = GetHklmValue(@"SOFTWARE\Microsoft\Windows NT\CurrentVersion", "CurrentBuildNumber") as string;
return StringUtil.Format("{0}.{1}", majorMinorString, build);
}
}
}

View File

@@ -39,10 +39,6 @@ namespace GitHub.Runner.Common
var extensions = new List<IExtension>(); var extensions = new List<IExtension>();
switch (typeof(T).FullName) switch (typeof(T).FullName)
{ {
// Listener capabilities providers.
case "GitHub.Runner.Common.Capabilities.ICapabilitiesProvider":
Add<T>(extensions, "GitHub.Runner.Common.Capabilities.RunnerCapabilitiesProvider, Runner.Common");
break;
// Action command extensions. // Action command extensions.
case "GitHub.Runner.Worker.IActionCommandExtension": case "GitHub.Runner.Worker.IActionCommandExtension":
Add<T>(extensions, "GitHub.Runner.Worker.InternalPluginSetRepoPathCommandExtension, Runner.Worker"); Add<T>(extensions, "GitHub.Runner.Worker.InternalPluginSetRepoPathCommandExtension, Runner.Worker");
@@ -58,6 +54,7 @@ namespace GitHub.Runner.Common
Add<T>(extensions, "GitHub.Runner.Worker.DebugCommandExtension, Runner.Worker"); Add<T>(extensions, "GitHub.Runner.Worker.DebugCommandExtension, Runner.Worker");
Add<T>(extensions, "GitHub.Runner.Worker.GroupCommandExtension, Runner.Worker"); Add<T>(extensions, "GitHub.Runner.Worker.GroupCommandExtension, Runner.Worker");
Add<T>(extensions, "GitHub.Runner.Worker.EndGroupCommandExtension, Runner.Worker"); Add<T>(extensions, "GitHub.Runner.Worker.EndGroupCommandExtension, Runner.Worker");
Add<T>(extensions, "GitHub.Runner.Worker.EchoCommandExtension, Runner.Worker");
break; break;
default: default:
// This should never happen. // This should never happen.

View File

@@ -84,9 +84,6 @@ namespace GitHub.Runner.Common
this.SecretMasker.AddValueEncoder(ValueEncoders.Base64StringEscape); this.SecretMasker.AddValueEncoder(ValueEncoders.Base64StringEscape);
this.SecretMasker.AddValueEncoder(ValueEncoders.Base64StringEscapeShift1); this.SecretMasker.AddValueEncoder(ValueEncoders.Base64StringEscapeShift1);
this.SecretMasker.AddValueEncoder(ValueEncoders.Base64StringEscapeShift2); this.SecretMasker.AddValueEncoder(ValueEncoders.Base64StringEscapeShift2);
this.SecretMasker.AddValueEncoder(ValueEncoders.Base64StringEscapeShift3);
this.SecretMasker.AddValueEncoder(ValueEncoders.Base64StringEscapeShift4);
this.SecretMasker.AddValueEncoder(ValueEncoders.Base64StringEscapeShift5);
this.SecretMasker.AddValueEncoder(ValueEncoders.ExpressionStringEscape); this.SecretMasker.AddValueEncoder(ValueEncoders.ExpressionStringEscape);
this.SecretMasker.AddValueEncoder(ValueEncoders.JsonStringEscape); this.SecretMasker.AddValueEncoder(ValueEncoders.JsonStringEscape);
this.SecretMasker.AddValueEncoder(ValueEncoders.UriDataEscape); this.SecretMasker.AddValueEncoder(ValueEncoders.UriDataEscape);

View File

@@ -1,13 +1,14 @@
<Project Sdk="Microsoft.NET.Sdk"> <Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup> <PropertyGroup>
<TargetFramework>netcoreapp2.2</TargetFramework> <TargetFramework>netcoreapp3.0</TargetFramework>
<OutputType>Library</OutputType> <OutputType>Library</OutputType>
<RuntimeIdentifiers>win-x64;win-x86;linux-x64;linux-arm;rhel.6-x64;osx-x64</RuntimeIdentifiers> <RuntimeIdentifiers>win-x64;win-x86;linux-x64;linux-arm;rhel.6-x64;osx-x64</RuntimeIdentifiers>
<TargetLatestRuntimePatch>true</TargetLatestRuntimePatch> <TargetLatestRuntimePatch>true</TargetLatestRuntimePatch>
<AssetTargetFallback>portable-net45+win8</AssetTargetFallback> <AssetTargetFallback>portable-net45+win8</AssetTargetFallback>
<NoWarn>NU1701;NU1603</NoWarn> <NoWarn>NU1701;NU1603</NoWarn>
<Version>$(Version)</Version> <Version>$(Version)</Version>
<TieredCompilationQuickJit>true</TieredCompilationQuickJit>
</PropertyGroup> </PropertyGroup>
<ItemGroup> <ItemGroup>

View File

@@ -31,7 +31,7 @@ namespace GitHub.Runner.Common
Task DeleteAgentAsync(int agentPoolId, int agentId); Task DeleteAgentAsync(int agentPoolId, int agentId);
Task<List<TaskAgentPool>> GetAgentPoolsAsync(string agentPoolName = null, TaskAgentPoolType poolType = TaskAgentPoolType.Automation); Task<List<TaskAgentPool>> GetAgentPoolsAsync(string agentPoolName = null, TaskAgentPoolType poolType = TaskAgentPoolType.Automation);
Task<List<TaskAgent>> GetAgentsAsync(int agentPoolId, string agentName = null); Task<List<TaskAgent>> GetAgentsAsync(int agentPoolId, string agentName = null);
Task<TaskAgent> UpdateAgentAsync(int agentPoolId, TaskAgent agent); Task<TaskAgent> ReplaceAgentAsync(int agentPoolId, TaskAgent agent);
// messagequeue // messagequeue
Task<TaskAgentSession> CreateAgentSessionAsync(Int32 poolId, TaskAgentSession session, CancellationToken cancellationToken); Task<TaskAgentSession> CreateAgentSessionAsync(Int32 poolId, TaskAgentSession session, CancellationToken cancellationToken);
@@ -257,7 +257,7 @@ namespace GitHub.Runner.Common
return _genericTaskAgentClient.GetAgentsAsync(agentPoolId, agentName, false); return _genericTaskAgentClient.GetAgentsAsync(agentPoolId, agentName, false);
} }
public Task<TaskAgent> UpdateAgentAsync(int agentPoolId, TaskAgent agent) public Task<TaskAgent> ReplaceAgentAsync(int agentPoolId, TaskAgent agent)
{ {
CheckConnection(RunnerConnectionType.Generic); CheckConnection(RunnerConnectionType.Generic);
return _genericTaskAgentClient.ReplaceAgentAsync(agentPoolId, agent); return _genericTaskAgentClient.ReplaceAgentAsync(agentPoolId, agent);

View File

@@ -1,24 +1,19 @@
using GitHub.DistributedTask.WebApi; using GitHub.DistributedTask.WebApi;
using GitHub.Runner.Common.Capabilities;
using GitHub.Runner.Common.Util; using GitHub.Runner.Common.Util;
using GitHub.Services.Common; using GitHub.Services.Common;
using GitHub.Services.OAuth; using GitHub.Services.OAuth;
using GitHub.Services.WebApi; using GitHub.Services.WebApi;
using System; using System;
using System.Collections.Generic; using System.Collections.Generic;
using System.Diagnostics;
using System.IO; using System.IO;
using System.Linq; using System.Linq;
using System.Security.Cryptography; using System.Security.Cryptography;
using System.Security.Principal;
using System.Threading;
using System.Threading.Tasks; using System.Threading.Tasks;
using System.Runtime.InteropServices; using System.Runtime.InteropServices;
using GitHub.Runner.Common; using GitHub.Runner.Common;
using GitHub.Runner.Sdk; using GitHub.Runner.Sdk;
using System.Net.Http; using System.Net.Http;
using System.Net.Http.Headers; using System.Net.Http.Headers;
using System.Text;
namespace GitHub.Runner.Listener.Configuration namespace GitHub.Runner.Listener.Configuration
{ {
@@ -168,7 +163,8 @@ namespace GitHub.Runner.Listener.Configuration
{ {
// Get the URL // Get the URL
var inputUrl = command.GetUrl(); var inputUrl = command.GetUrl();
if (!inputUrl.Contains("github.com", StringComparison.OrdinalIgnoreCase)) if (!inputUrl.Contains("github.com", StringComparison.OrdinalIgnoreCase) &&
!inputUrl.Contains("github.localhost", StringComparison.OrdinalIgnoreCase))
{ {
runnerSettings.ServerUrl = inputUrl; runnerSettings.ServerUrl = inputUrl;
// Get the credentials // Get the credentials
@@ -238,11 +234,8 @@ namespace GitHub.Runner.Listener.Configuration
{ {
runnerSettings.AgentName = command.GetAgentName(); runnerSettings.AgentName = command.GetAgentName();
// Get the system capabilities.
Dictionary<string, string> systemCapabilities = await HostContext.GetService<ICapabilitiesManager>().GetCapabilitiesAsync(runnerSettings, CancellationToken.None);
_term.WriteLine(); _term.WriteLine();
var agents = await _runnerServer.GetAgentsAsync(runnerSettings.PoolId, runnerSettings.AgentName); var agents = await _runnerServer.GetAgentsAsync(runnerSettings.PoolId, runnerSettings.AgentName);
Trace.Verbose("Returns {0} agents", agents.Count); Trace.Verbose("Returns {0} agents", agents.Count);
agent = agents.FirstOrDefault(); agent = agents.FirstOrDefault();
@@ -251,12 +244,12 @@ namespace GitHub.Runner.Listener.Configuration
_term.WriteLine("A runner exists with the same name", ConsoleColor.Yellow); _term.WriteLine("A runner exists with the same name", ConsoleColor.Yellow);
if (command.GetReplace()) if (command.GetReplace())
{ {
// Update existing agent with new PublicKey, agent version and SystemCapabilities. // Update existing agent with new PublicKey, agent version.
agent = UpdateExistingAgent(agent, publicKey, systemCapabilities); agent = UpdateExistingAgent(agent, publicKey);
try try
{ {
agent = await _runnerServer.UpdateAgentAsync(runnerSettings.PoolId, agent); agent = await _runnerServer.ReplaceAgentAsync(runnerSettings.PoolId, agent);
_term.WriteSuccessMessage("Successfully replaced the runner"); _term.WriteSuccessMessage("Successfully replaced the runner");
break; break;
} }
@@ -275,7 +268,7 @@ namespace GitHub.Runner.Listener.Configuration
else else
{ {
// Create a new agent. // Create a new agent.
agent = CreateNewAgent(runnerSettings.AgentName, publicKey, systemCapabilities); agent = CreateNewAgent(runnerSettings.AgentName, publicKey);
try try
{ {
@@ -435,7 +428,7 @@ namespace GitHub.Runner.Listener.Configuration
{ {
ArgUtil.Equal(RunMode.Normal, HostContext.RunMode, nameof(HostContext.RunMode)); ArgUtil.Equal(RunMode.Normal, HostContext.RunMode, nameof(HostContext.RunMode));
string currentAction = string.Empty; string currentAction = string.Empty;
_term.WriteSection("Runner removal"); _term.WriteSection("Runner removal");
try try
@@ -499,7 +492,7 @@ namespace GitHub.Runner.Listener.Configuration
else else
{ {
await _runnerServer.DeleteAgentAsync(settings.PoolId, settings.AgentId); await _runnerServer.DeleteAgentAsync(settings.PoolId, settings.AgentId);
_term.WriteLine(); _term.WriteLine();
_term.WriteSuccessMessage("Runner removed successfully"); _term.WriteSuccessMessage("Runner removed successfully");
} }
@@ -573,7 +566,7 @@ namespace GitHub.Runner.Listener.Configuration
} }
private TaskAgent UpdateExistingAgent(TaskAgent agent, RSAParameters publicKey, Dictionary<string, string> systemCapabilities) private TaskAgent UpdateExistingAgent(TaskAgent agent, RSAParameters publicKey)
{ {
ArgUtil.NotNull(agent, nameof(agent)); ArgUtil.NotNull(agent, nameof(agent));
agent.Authorization = new TaskAgentAuthorization agent.Authorization = new TaskAgentAuthorization
@@ -585,15 +578,14 @@ namespace GitHub.Runner.Listener.Configuration
agent.Version = BuildConstants.RunnerPackage.Version; agent.Version = BuildConstants.RunnerPackage.Version;
agent.OSDescription = RuntimeInformation.OSDescription; agent.OSDescription = RuntimeInformation.OSDescription;
foreach (KeyValuePair<string, string> capability in systemCapabilities) agent.Labels.Add("self-hosted");
{ agent.Labels.Add(VarUtil.OS);
agent.SystemCapabilities[capability.Key] = capability.Value ?? string.Empty; agent.Labels.Add(VarUtil.OSArchitecture);
}
return agent; return agent;
} }
private TaskAgent CreateNewAgent(string agentName, RSAParameters publicKey, Dictionary<string, string> systemCapabilities) private TaskAgent CreateNewAgent(string agentName, RSAParameters publicKey)
{ {
TaskAgent agent = new TaskAgent(agentName) TaskAgent agent = new TaskAgent(agentName)
{ {
@@ -606,10 +598,9 @@ namespace GitHub.Runner.Listener.Configuration
OSDescription = RuntimeInformation.OSDescription, OSDescription = RuntimeInformation.OSDescription,
}; };
foreach (KeyValuePair<string, string> capability in systemCapabilities) agent.Labels.Add("self-hosted");
{ agent.Labels.Add(VarUtil.OS);
agent.SystemCapabilities[capability.Key] = capability.Value ?? string.Empty; agent.Labels.Add(VarUtil.OSArchitecture);
}
return agent; return agent;
} }
@@ -638,7 +629,7 @@ namespace GitHub.Runner.Listener.Configuration
private async Task<GitHubAuthResult> GetTenantCredential(string githubUrl, string githubToken) private async Task<GitHubAuthResult> GetTenantCredential(string githubUrl, string githubToken)
{ {
var gitHubUrl = new UriBuilder(githubUrl); var gitHubUrl = new UriBuilder(githubUrl);
var githubApiUrl = $"https://api.github.com/repos/{gitHubUrl.Path.Trim('/')}/actions-runners/registration"; var githubApiUrl = $"https://api.{gitHubUrl.Host}/repos/{gitHubUrl.Path.Trim('/')}/actions-runners/registration";
using (var httpClientHandler = HostContext.CreateHttpClientHandler()) using (var httpClientHandler = HostContext.CreateHttpClientHandler())
using (var httpClient = new HttpClient(httpClientHandler)) using (var httpClient = new HttpClient(httpClientHandler))
{ {

View File

@@ -1,7 +1,5 @@
using GitHub.DistributedTask.WebApi; using GitHub.DistributedTask.WebApi;
using GitHub.Runner.Common.Capabilities;
using GitHub.Runner.Listener.Configuration; using GitHub.Runner.Listener.Configuration;
using GitHub.Runner.Common.Util;
using GitHub.Services.Common; using GitHub.Services.Common;
using System; using System;
using System.Collections.Generic; using System.Collections.Generic;
@@ -10,7 +8,6 @@ using System.Threading.Tasks;
using System.Security.Cryptography; using System.Security.Cryptography;
using System.IO; using System.IO;
using System.Text; using System.Text;
using GitHub.Services.WebApi;
using GitHub.Services.OAuth; using GitHub.Services.OAuth;
using System.Diagnostics; using System.Diagnostics;
using System.Runtime.InteropServices; using System.Runtime.InteropServices;
@@ -59,9 +56,6 @@ namespace GitHub.Runner.Listener
var serverUrl = _settings.ServerUrl; var serverUrl = _settings.ServerUrl;
Trace.Info(_settings); Trace.Info(_settings);
// Capabilities.
Dictionary<string, string> systemCapabilities = await HostContext.GetService<ICapabilitiesManager>().GetCapabilitiesAsync(_settings, token);
// Create connection. // Create connection.
Trace.Info("Loading Credentials"); Trace.Info("Loading Credentials");
var credMgr = HostContext.GetService<ICredentialManager>(); var credMgr = HostContext.GetService<ICredentialManager>();
@@ -75,7 +69,7 @@ namespace GitHub.Runner.Listener
OSDescription = RuntimeInformation.OSDescription, OSDescription = RuntimeInformation.OSDescription,
}; };
string sessionName = $"{Environment.MachineName ?? "RUNNER"}"; string sessionName = $"{Environment.MachineName ?? "RUNNER"}";
var taskAgentSession = new TaskAgentSession(sessionName, agent, systemCapabilities); var taskAgentSession = new TaskAgentSession(sessionName, agent);
string errorMessage = string.Empty; string errorMessage = string.Empty;
bool encounteringError = false; bool encounteringError = false;

View File

@@ -1,13 +1,15 @@
<Project Sdk="Microsoft.NET.Sdk"> <Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup> <PropertyGroup>
<TargetFramework>netcoreapp2.2</TargetFramework> <TargetFramework>netcoreapp3.0</TargetFramework>
<OutputType>Exe</OutputType> <OutputType>Exe</OutputType>
<RuntimeIdentifiers>win-x64;win-x86;linux-x64;linux-arm;rhel.6-x64;osx-x64</RuntimeIdentifiers> <RuntimeIdentifiers>win-x64;win-x86;linux-x64;linux-arm;rhel.6-x64;osx-x64</RuntimeIdentifiers>
<TargetLatestRuntimePatch>true</TargetLatestRuntimePatch> <TargetLatestRuntimePatch>true</TargetLatestRuntimePatch>
<AssetTargetFallback>portable-net45+win8</AssetTargetFallback> <AssetTargetFallback>portable-net45+win8</AssetTargetFallback>
<NoWarn>NU1701;NU1603</NoWarn> <NoWarn>NU1701;NU1603</NoWarn>
<Version>$(Version)</Version> <Version>$(Version)</Version>
<TieredCompilationQuickJit>true</TieredCompilationQuickJit>
<PublishReadyToRun>true</PublishReadyToRun>
</PropertyGroup> </PropertyGroup>
<ItemGroup> <ItemGroup>

View File

@@ -1,13 +1,15 @@
<Project Sdk="Microsoft.NET.Sdk"> <Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup> <PropertyGroup>
<TargetFramework>netcoreapp2.2</TargetFramework> <TargetFramework>netcoreapp3.0</TargetFramework>
<OutputType>Exe</OutputType> <OutputType>Exe</OutputType>
<RuntimeIdentifiers>win-x64;win-x86;linux-x64;linux-arm;rhel.6-x64;osx-x64</RuntimeIdentifiers> <RuntimeIdentifiers>win-x64;win-x86;linux-x64;linux-arm;rhel.6-x64;osx-x64</RuntimeIdentifiers>
<TargetLatestRuntimePatch>true</TargetLatestRuntimePatch> <TargetLatestRuntimePatch>true</TargetLatestRuntimePatch>
<AssetTargetFallback>portable-net45+win8</AssetTargetFallback> <AssetTargetFallback>portable-net45+win8</AssetTargetFallback>
<NoWarn>NU1701;NU1603</NoWarn> <NoWarn>NU1701;NU1603</NoWarn>
<Version>$(Version)</Version> <Version>$(Version)</Version>
<TieredCompilationQuickJit>true</TieredCompilationQuickJit>
<PublishReadyToRun>true</PublishReadyToRun>
</PropertyGroup> </PropertyGroup>
<ItemGroup> <ItemGroup>

View File

@@ -236,15 +236,15 @@ namespace GitHub.Runner.Plugins.Artifact
// try upload all files for the first time. // try upload all files for the first time.
UploadResult uploadResult = await ParallelUploadAsync(context, files, maxConcurrentUploads, _uploadCancellationTokenSource.Token); UploadResult uploadResult = await ParallelUploadAsync(context, files, maxConcurrentUploads, _uploadCancellationTokenSource.Token);
if (uploadResult.FailedFiles.Count == 0) if (uploadResult.RetryFiles.Count == 0)
{ {
// all files have been upload succeed. // all files have been upload succeed.
context.Output("File upload succeed."); context.Output("File upload complete.");
return uploadResult.TotalFileSizeUploaded; return uploadResult.TotalFileSizeUploaded;
} }
else else
{ {
context.Output($"{uploadResult.FailedFiles.Count} files failed to upload, retry these files after a minute."); context.Output($"{uploadResult.RetryFiles.Count} files failed to upload, retry these files after a minute.");
} }
// Delay 1 min then retry failed files. // Delay 1 min then retry failed files.
@@ -255,13 +255,13 @@ namespace GitHub.Runner.Plugins.Artifact
} }
// Retry upload all failed files. // Retry upload all failed files.
context.Output($"Start retry {uploadResult.FailedFiles.Count} failed files upload."); context.Output($"Start retry {uploadResult.RetryFiles.Count} failed files upload.");
UploadResult retryUploadResult = await ParallelUploadAsync(context, uploadResult.FailedFiles, maxConcurrentUploads, _uploadCancellationTokenSource.Token); UploadResult retryUploadResult = await ParallelUploadAsync(context, uploadResult.RetryFiles, maxConcurrentUploads, _uploadCancellationTokenSource.Token);
if (retryUploadResult.FailedFiles.Count == 0) if (retryUploadResult.RetryFiles.Count == 0)
{ {
// all files have been upload succeed after retry. // all files have been upload succeed after retry.
context.Output("File upload succeed after retry."); context.Output("File upload complete after retry.");
return uploadResult.TotalFileSizeUploaded + retryUploadResult.TotalFileSizeUploaded; return uploadResult.TotalFileSizeUploaded + retryUploadResult.TotalFileSizeUploaded;
} }
else else
@@ -465,75 +465,61 @@ namespace GitHub.Runner.Plugins.Artifact
using (FileStream fs = File.Open(fileToUpload, FileMode.Open, FileAccess.Read, FileShare.Read)) using (FileStream fs = File.Open(fileToUpload, FileMode.Open, FileAccess.Read, FileShare.Read))
{ {
string itemPath = (_containerPath.TrimEnd('/') + "/" + fileToUpload.Remove(0, _sourceParentDirectory.Length + 1)).Replace('\\', '/'); string itemPath = (_containerPath.TrimEnd('/') + "/" + fileToUpload.Remove(0, _sourceParentDirectory.Length + 1)).Replace('\\', '/');
uploadTimer.Restart(); bool failAndExit = false;
bool catchExceptionDuringUpload = false;
HttpResponseMessage response = null;
try try
{ {
response = await _fileContainerHttpClient.UploadFileAsync(_containerId, itemPath, fs, _projectId, cancellationToken: token, chunkSize: 4 * 1024 * 1024); uploadTimer.Restart();
using (HttpResponseMessage response = await _fileContainerHttpClient.UploadFileAsync(_containerId, itemPath, fs, _projectId, cancellationToken: token, chunkSize: 4 * 1024 * 1024))
{
if (response == null || response.StatusCode != HttpStatusCode.Created)
{
context.Output($"Unable to copy file to server StatusCode={response?.StatusCode}: {response?.ReasonPhrase}. Source file path: {fileToUpload}. Target server path: {itemPath}");
if (response?.StatusCode == HttpStatusCode.Conflict)
{
// fail upload task but continue with any other files
context.Error($"Error '{fileToUpload}' has already been uploaded.");
}
else if (_fileContainerHttpClient.IsFastFailResponse(response))
{
// Fast fail: we received an http status code where we should abandon our efforts
context.Output($"Cannot continue uploading files, so draining upload queue of {_fileUploadQueue.Count} items.");
DrainUploadQueue(context);
failedFiles.Clear();
failAndExit = true;
throw new UploadFailedException($"Critical failure uploading '{fileToUpload}'");
}
else
{
context.Debug($"Adding '{fileToUpload}' to retry list.");
failedFiles.Add(fileToUpload);
}
throw new UploadFailedException($"Http failure response '{response?.StatusCode}': '{response?.ReasonPhrase}' while uploading '{fileToUpload}'");
}
uploadTimer.Stop();
context.Debug($"File: '{fileToUpload}' took {uploadTimer.ElapsedMilliseconds} milliseconds to finish upload");
uploadedSize += fs.Length;
OutputLogForFile(context, fileToUpload, $"Detail upload trace for file: {itemPath}", context.Debug);
}
} }
catch (OperationCanceledException) when (token.IsCancellationRequested) catch (OperationCanceledException) when (token.IsCancellationRequested)
{ {
context.Output($"File upload has been cancelled during upload file: '{fileToUpload}'."); context.Output($"File upload has been cancelled during upload file: '{fileToUpload}'.");
if (response != null)
{
response.Dispose();
response = null;
}
throw; throw;
} }
catch (Exception ex) catch (Exception ex)
{ {
catchExceptionDuringUpload = true;
context.Output($"Fail to upload '{fileToUpload}' due to '{ex.Message}'."); context.Output($"Fail to upload '{fileToUpload}' due to '{ex.Message}'.");
context.Output(ex.ToString()); context.Output(ex.ToString());
}
uploadTimer.Stop(); OutputLogForFile(context, fileToUpload, $"Detail upload trace for file that fail to upload: {itemPath}", context.Output);
if (catchExceptionDuringUpload || (response != null && response.StatusCode != HttpStatusCode.Created))
{ if (failAndExit)
if (response != null)
{ {
context.Output($"Unable to copy file to server StatusCode={response.StatusCode}: {response.ReasonPhrase}. Source file path: {fileToUpload}. Target server path: {itemPath}"); context.Debug("Exiting upload.");
throw;
} }
// output detail upload trace for the file.
ConcurrentQueue<string> logQueue;
if (_fileUploadTraceLog.TryGetValue(itemPath, out logQueue))
{
context.Output($"Detail upload trace for file that fail to upload: {itemPath}");
string message;
while (logQueue.TryDequeue(out message))
{
context.Output(message);
}
}
// tracking file that failed to upload.
failedFiles.Add(fileToUpload);
}
else
{
context.Debug($"File: '{fileToUpload}' took {uploadTimer.ElapsedMilliseconds} milliseconds to finish upload");
uploadedSize += fs.Length;
// debug detail upload trace for the file.
ConcurrentQueue<string> logQueue;
if (_fileUploadTraceLog.TryGetValue(itemPath, out logQueue))
{
context.Debug($"Detail upload trace for file: {itemPath}");
string message;
while (logQueue.TryDequeue(out message))
{
context.Debug(message);
}
}
}
if (response != null)
{
response.Dispose();
response = null;
} }
} }
@@ -590,6 +576,30 @@ namespace GitHub.Runner.Plugins.Artifact
} }
} }
private void DrainUploadQueue(RunnerActionPluginExecutionContext context)
{
while (_fileUploadQueue.TryDequeue(out string fileToUpload))
{
context.Debug($"Clearing upload queue: '{fileToUpload}'");
Interlocked.Increment(ref _uploadFilesProcessed);
}
}
private void OutputLogForFile(RunnerActionPluginExecutionContext context, string itemPath, string logDescription, Action<string> log)
{
// output detail upload trace for the file.
ConcurrentQueue<string> logQueue;
if (_fileUploadTraceLog.TryGetValue(itemPath, out logQueue))
{
log(logDescription);
string message;
while (logQueue.TryDequeue(out message))
{
log(message);
}
}
}
private void UploadFileTraceReportReceived(object sender, ReportTraceEventArgs e) private void UploadFileTraceReportReceived(object sender, ReportTraceEventArgs e)
{ {
ConcurrentQueue<string> logQueue = _fileUploadTraceLog.GetOrAdd(e.File, new ConcurrentQueue<string>()); ConcurrentQueue<string> logQueue = _fileUploadTraceLog.GetOrAdd(e.File, new ConcurrentQueue<string>());
@@ -607,22 +617,22 @@ namespace GitHub.Runner.Plugins.Artifact
{ {
public UploadResult() public UploadResult()
{ {
FailedFiles = new List<string>(); RetryFiles = new List<string>();
TotalFileSizeUploaded = 0; TotalFileSizeUploaded = 0;
} }
public UploadResult(List<string> failedFiles, long totalFileSizeUploaded) public UploadResult(List<string> retryFiles, long totalFileSizeUploaded)
{ {
FailedFiles = failedFiles; RetryFiles = retryFiles ?? new List<string>();
TotalFileSizeUploaded = totalFileSizeUploaded; TotalFileSizeUploaded = totalFileSizeUploaded;
} }
public List<string> FailedFiles { get; set; } public List<string> RetryFiles { get; set; }
public long TotalFileSizeUploaded { get; set; } public long TotalFileSizeUploaded { get; set; }
public void AddUploadResult(UploadResult resultToAdd) public void AddUploadResult(UploadResult resultToAdd)
{ {
this.FailedFiles.AddRange(resultToAdd.FailedFiles); this.RetryFiles.AddRange(resultToAdd.RetryFiles);
this.TotalFileSizeUploaded += resultToAdd.TotalFileSizeUploaded; this.TotalFileSizeUploaded += resultToAdd.TotalFileSizeUploaded;
} }
} }
@@ -657,4 +667,19 @@ namespace GitHub.Runner.Plugins.Artifact
this.FailedFiles.AddRange(resultToAdd.FailedFiles); this.FailedFiles.AddRange(resultToAdd.FailedFiles);
} }
} }
public class UploadFailedException : Exception
{
public UploadFailedException()
: base()
{ }
public UploadFailedException(string message)
: base(message)
{ }
public UploadFailedException(string message, Exception inner)
: base(message, inner)
{ }
}
} }

View File

@@ -74,17 +74,22 @@ namespace GitHub.Runner.Plugins.Artifact
context.Output($"Uploading artifact '{artifactName}' from '{fullPath}' for run #{buildId}"); context.Output($"Uploading artifact '{artifactName}' from '{fullPath}' for run #{buildId}");
FileContainerServer fileContainerHelper = new FileContainerServer(context.VssConnection, projectId, containerId, artifactName); FileContainerServer fileContainerHelper = new FileContainerServer(context.VssConnection, projectId, containerId, artifactName);
long size = await fileContainerHelper.CopyToContainerAsync(context, fullPath, token);
var propertiesDictionary = new Dictionary<string, string>(StringComparer.OrdinalIgnoreCase); var propertiesDictionary = new Dictionary<string, string>(StringComparer.OrdinalIgnoreCase);
propertiesDictionary.Add("artifactsize", size.ToString()); try
{
string fileContainerFullPath = StringUtil.Format($"#/{containerId}/{artifactName}"); long size = await fileContainerHelper.CopyToContainerAsync(context, fullPath, token);
context.Output($"Uploaded '{fullPath}' to server"); propertiesDictionary.Add("artifactsize", size.ToString());
context.Output($"Uploaded '{size}' bytes from '{fullPath}' to server");
BuildServer buildHelper = new BuildServer(context.VssConnection); }
string jobId = context.Variables.GetValueOrDefault(WellKnownDistributedTaskVariables.JobId).Value ?? string.Empty; // if any of the results were successful, make sure to attach them to the build
var artifact = await buildHelper.AssociateArtifact(projectId, buildId, jobId, artifactName, ArtifactResourceTypes.Container, fileContainerFullPath, propertiesDictionary, token); finally
context.Output($"Associated artifact {artifactName} ({artifact.Id}) with run #{buildId}"); {
string fileContainerFullPath = StringUtil.Format($"#/{containerId}/{artifactName}");
BuildServer buildHelper = new BuildServer(context.VssConnection);
string jobId = context.Variables.GetValueOrDefault(WellKnownDistributedTaskVariables.JobId).Value ?? string.Empty;
var artifact = await buildHelper.AssociateArtifact(projectId, buildId, jobId, artifactName, ArtifactResourceTypes.Container, fileContainerFullPath, propertiesDictionary, token);
context.Output($"Associated artifact {artifactName} ({artifact.Id}) with run #{buildId}");
}
} }
} }
} }

View File

@@ -1,13 +1,14 @@
<Project Sdk="Microsoft.NET.Sdk"> <Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup> <PropertyGroup>
<TargetFramework>netcoreapp2.2</TargetFramework> <TargetFramework>netcoreapp3.0</TargetFramework>
<OutputType>Library</OutputType> <OutputType>Library</OutputType>
<RuntimeIdentifiers>win-x64;win-x86;linux-x64;linux-arm;rhel.6-x64;osx-x64</RuntimeIdentifiers> <RuntimeIdentifiers>win-x64;win-x86;linux-x64;linux-arm;rhel.6-x64;osx-x64</RuntimeIdentifiers>
<TargetLatestRuntimePatch>true</TargetLatestRuntimePatch> <TargetLatestRuntimePatch>true</TargetLatestRuntimePatch>
<AssetTargetFallback>portable-net45+win8</AssetTargetFallback> <AssetTargetFallback>portable-net45+win8</AssetTargetFallback>
<NoWarn>NU1701;NU1603</NoWarn> <NoWarn>NU1701;NU1603</NoWarn>
<Version>$(Version)</Version> <Version>$(Version)</Version>
<TieredCompilationQuickJit>true</TieredCompilationQuickJit>
</PropertyGroup> </PropertyGroup>
<ItemGroup> <ItemGroup>

View File

@@ -70,13 +70,6 @@ namespace GitHub.Runner.Sdk
VssClientHttpRequestSettings.Default.UserAgent = headerValues; VssClientHttpRequestSettings.Default.UserAgent = headerValues;
#if OS_LINUX || OS_OSX
// The .NET Core 2.1 runtime switched its HTTP default from HTTP 1.1 to HTTP 2.
// This causes problems with some versions of the Curl handler.
// See GitHub issue https://github.com/dotnet/corefx/issues/32376
VssClientHttpRequestSettings.Default.UseHttp11 = true;
#endif
var certSetting = GetCertConfiguration(); var certSetting = GetCertConfiguration();
if (certSetting != null) if (certSetting != null)
{ {

View File

@@ -1,13 +1,14 @@
<Project Sdk="Microsoft.NET.Sdk"> <Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup> <PropertyGroup>
<TargetFramework>netcoreapp2.2</TargetFramework> <TargetFramework>netcoreapp3.0</TargetFramework>
<OutputType>Library</OutputType> <OutputType>Library</OutputType>
<RuntimeIdentifiers>win-x64;win-x86;linux-x64;linux-arm;rhel.6-x64;osx-x64</RuntimeIdentifiers> <RuntimeIdentifiers>win-x64;win-x86;linux-x64;linux-arm;rhel.6-x64;osx-x64</RuntimeIdentifiers>
<TargetLatestRuntimePatch>true</TargetLatestRuntimePatch> <TargetLatestRuntimePatch>true</TargetLatestRuntimePatch>
<AssetTargetFallback>portable-net45+win8</AssetTargetFallback> <AssetTargetFallback>portable-net45+win8</AssetTargetFallback>
<NoWarn>NU1701;NU1603</NoWarn> <NoWarn>NU1701;NU1603</NoWarn>
<Version>$(Version)</Version> <Version>$(Version)</Version>
<TieredCompilationQuickJit>true</TieredCompilationQuickJit>
</PropertyGroup> </PropertyGroup>
<ItemGroup> <ItemGroup>

View File

@@ -27,13 +27,6 @@ namespace GitHub.Runner.Sdk
VssClientHttpRequestSettings.Default.UserAgent = headerValues; VssClientHttpRequestSettings.Default.UserAgent = headerValues;
VssClientHttpRequestSettings.Default.ClientCertificateManager = clientCert; VssClientHttpRequestSettings.Default.ClientCertificateManager = clientCert;
#if OS_LINUX || OS_OSX
// The .NET Core 2.1 runtime switched its HTTP default from HTTP 1.1 to HTTP 2.
// This causes problems with some versions of the Curl handler.
// See GitHub issue https://github.com/dotnet/corefx/issues/32376
VssClientHttpRequestSettings.Default.UseHttp11 = true;
#endif
VssHttpMessageHandler.DefaultWebProxy = proxy; VssHttpMessageHandler.DefaultWebProxy = proxy;
} }

View File

@@ -73,7 +73,7 @@ namespace GitHub.Runner.Worker
return false; return false;
} }
// process action command in serialize oreder. // process action command in serialize order.
lock (_commandSerializeLock) lock (_commandSerializeLock)
{ {
if (_stopProcessCommand) if (_stopProcessCommand)
@@ -107,26 +107,22 @@ namespace GitHub.Runner.Worker
} }
else if (_commandExtensions.TryGetValue(actionCommand.Command, out IActionCommandExtension extension)) else if (_commandExtensions.TryGetValue(actionCommand.Command, out IActionCommandExtension extension))
{ {
bool omitEcho; if (context.EchoOnActionCommand && !extension.OmitEcho)
{
context.Output(input);
}
try try
{ {
extension.ProcessCommand(context, input, actionCommand, out omitEcho); extension.ProcessCommand(context, input, actionCommand);
} }
catch (Exception ex) catch (Exception ex)
{ {
omitEcho = true; var commandInformation = extension.OmitEcho ? extension.Command : input;
context.Output(input); context.Error($"Unable to process command '{commandInformation}' successfully.");
context.Error($"Unable to process command '{input}' successfully.");
context.Error(ex); context.Error(ex);
context.CommandResult = TaskResult.Failed; context.CommandResult = TaskResult.Failed;
} }
if (!omitEcho)
{
context.Output(input);
context.Debug($"Processed command");
}
} }
else else
{ {
@@ -142,17 +138,19 @@ namespace GitHub.Runner.Worker
public interface IActionCommandExtension : IExtension public interface IActionCommandExtension : IExtension
{ {
string Command { get; } string Command { get; }
bool OmitEcho { get; }
void ProcessCommand(IExecutionContext context, string line, ActionCommand command, out bool omitEcho); void ProcessCommand(IExecutionContext context, string line, ActionCommand command);
} }
public sealed class InternalPluginSetRepoPathCommandExtension : RunnerService, IActionCommandExtension public sealed class InternalPluginSetRepoPathCommandExtension : RunnerService, IActionCommandExtension
{ {
public string Command => "internal-set-repo-path"; public string Command => "internal-set-repo-path";
public bool OmitEcho => false;
public Type ExtensionType => typeof(IActionCommandExtension); public Type ExtensionType => typeof(IActionCommandExtension);
public void ProcessCommand(IExecutionContext context, string line, ActionCommand command, out bool omitEcho) public void ProcessCommand(IExecutionContext context, string line, ActionCommand command)
{ {
if (!command.Properties.TryGetValue(SetRepoPathCommandProperties.repoFullName, out string repoFullName) || string.IsNullOrEmpty(repoFullName)) if (!command.Properties.TryGetValue(SetRepoPathCommandProperties.repoFullName, out string repoFullName) || string.IsNullOrEmpty(repoFullName))
{ {
@@ -166,8 +164,6 @@ namespace GitHub.Runner.Worker
var directoryManager = HostContext.GetService<IPipelineDirectoryManager>(); var directoryManager = HostContext.GetService<IPipelineDirectoryManager>();
var trackingConfig = directoryManager.UpdateRepositoryDirectory(context, repoFullName, command.Data, StringUtil.ConvertToBoolean(workspaceRepo)); var trackingConfig = directoryManager.UpdateRepositoryDirectory(context, repoFullName, command.Data, StringUtil.ConvertToBoolean(workspaceRepo));
omitEcho = true;
} }
private static class SetRepoPathCommandProperties private static class SetRepoPathCommandProperties
@@ -180,10 +176,11 @@ namespace GitHub.Runner.Worker
public sealed class SetEnvCommandExtension : RunnerService, IActionCommandExtension public sealed class SetEnvCommandExtension : RunnerService, IActionCommandExtension
{ {
public string Command => "set-env"; public string Command => "set-env";
public bool OmitEcho => false;
public Type ExtensionType => typeof(IActionCommandExtension); public Type ExtensionType => typeof(IActionCommandExtension);
public void ProcessCommand(IExecutionContext context, string line, ActionCommand command, out bool omitEcho) public void ProcessCommand(IExecutionContext context, string line, ActionCommand command)
{ {
if (!command.Properties.TryGetValue(SetEnvCommandProperties.Name, out string envName) || string.IsNullOrEmpty(envName)) if (!command.Properties.TryGetValue(SetEnvCommandProperties.Name, out string envName) || string.IsNullOrEmpty(envName))
{ {
@@ -192,9 +189,7 @@ namespace GitHub.Runner.Worker
context.EnvironmentVariables[envName] = command.Data; context.EnvironmentVariables[envName] = command.Data;
context.SetEnvContext(envName, command.Data); context.SetEnvContext(envName, command.Data);
context.Output(line);
context.Debug($"{envName}='{command.Data}'"); context.Debug($"{envName}='{command.Data}'");
omitEcho = true;
} }
private static class SetEnvCommandProperties private static class SetEnvCommandProperties
@@ -206,10 +201,11 @@ namespace GitHub.Runner.Worker
public sealed class SetOutputCommandExtension : RunnerService, IActionCommandExtension public sealed class SetOutputCommandExtension : RunnerService, IActionCommandExtension
{ {
public string Command => "set-output"; public string Command => "set-output";
public bool OmitEcho => false;
public Type ExtensionType => typeof(IActionCommandExtension); public Type ExtensionType => typeof(IActionCommandExtension);
public void ProcessCommand(IExecutionContext context, string line, ActionCommand command, out bool omitEcho) public void ProcessCommand(IExecutionContext context, string line, ActionCommand command)
{ {
if (!command.Properties.TryGetValue(SetOutputCommandProperties.Name, out string outputName) || string.IsNullOrEmpty(outputName)) if (!command.Properties.TryGetValue(SetOutputCommandProperties.Name, out string outputName) || string.IsNullOrEmpty(outputName))
{ {
@@ -217,9 +213,7 @@ namespace GitHub.Runner.Worker
} }
context.SetOutput(outputName, command.Data, out var reference); context.SetOutput(outputName, command.Data, out var reference);
context.Output(line);
context.Debug($"{reference}='{command.Data}'"); context.Debug($"{reference}='{command.Data}'");
omitEcho = true;
} }
private static class SetOutputCommandProperties private static class SetOutputCommandProperties
@@ -231,10 +225,11 @@ namespace GitHub.Runner.Worker
public sealed class SaveStateCommandExtension : RunnerService, IActionCommandExtension public sealed class SaveStateCommandExtension : RunnerService, IActionCommandExtension
{ {
public string Command => "save-state"; public string Command => "save-state";
public bool OmitEcho => false;
public Type ExtensionType => typeof(IActionCommandExtension); public Type ExtensionType => typeof(IActionCommandExtension);
public void ProcessCommand(IExecutionContext context, string line, ActionCommand command, out bool omitEcho) public void ProcessCommand(IExecutionContext context, string line, ActionCommand command)
{ {
if (!command.Properties.TryGetValue(SaveStateCommandProperties.Name, out string stateName) || string.IsNullOrEmpty(stateName)) if (!command.Properties.TryGetValue(SaveStateCommandProperties.Name, out string stateName) || string.IsNullOrEmpty(stateName))
{ {
@@ -243,7 +238,6 @@ namespace GitHub.Runner.Worker
context.IntraActionState[stateName] = command.Data; context.IntraActionState[stateName] = command.Data;
context.Debug($"Save intra-action state {stateName} = {command.Data}"); context.Debug($"Save intra-action state {stateName} = {command.Data}");
omitEcho = true;
} }
private static class SaveStateCommandProperties private static class SaveStateCommandProperties
@@ -255,49 +249,53 @@ namespace GitHub.Runner.Worker
public sealed class AddMaskCommandExtension : RunnerService, IActionCommandExtension public sealed class AddMaskCommandExtension : RunnerService, IActionCommandExtension
{ {
public string Command => "add-mask"; public string Command => "add-mask";
public bool OmitEcho => true;
public Type ExtensionType => typeof(IActionCommandExtension); public Type ExtensionType => typeof(IActionCommandExtension);
public void ProcessCommand(IExecutionContext context, string line, ActionCommand command, out bool omitEcho) public void ProcessCommand(IExecutionContext context, string line, ActionCommand command)
{ {
if (string.IsNullOrWhiteSpace(command.Data)) if (string.IsNullOrWhiteSpace(command.Data))
{ {
context.Warning("Can't add secret mask for empty string."); context.Warning("Can't add secret mask for empty string in ##[add-mask] command.");
} }
else else
{ {
if (context.EchoOnActionCommand)
{
context.Output($"::{Command}::***");
}
HostContext.SecretMasker.AddValue(command.Data); HostContext.SecretMasker.AddValue(command.Data);
Trace.Info($"Add new secret mask with length of {command.Data.Length}"); Trace.Info($"Add new secret mask with length of {command.Data.Length}");
} }
omitEcho = true;
} }
} }
public sealed class AddPathCommandExtension : RunnerService, IActionCommandExtension public sealed class AddPathCommandExtension : RunnerService, IActionCommandExtension
{ {
public string Command => "add-path"; public string Command => "add-path";
public bool OmitEcho => false;
public Type ExtensionType => typeof(IActionCommandExtension); public Type ExtensionType => typeof(IActionCommandExtension);
public void ProcessCommand(IExecutionContext context, string line, ActionCommand command, out bool omitEcho) public void ProcessCommand(IExecutionContext context, string line, ActionCommand command)
{ {
ArgUtil.NotNullOrEmpty(command.Data, "path"); ArgUtil.NotNullOrEmpty(command.Data, "path");
context.PrependPath.RemoveAll(x => string.Equals(x, command.Data, StringComparison.CurrentCulture)); context.PrependPath.RemoveAll(x => string.Equals(x, command.Data, StringComparison.CurrentCulture));
context.PrependPath.Add(command.Data); context.PrependPath.Add(command.Data);
omitEcho = false;
} }
} }
public sealed class AddMatcherCommandExtension : RunnerService, IActionCommandExtension public sealed class AddMatcherCommandExtension : RunnerService, IActionCommandExtension
{ {
public string Command => "add-matcher"; public string Command => "add-matcher";
public bool OmitEcho => false;
public Type ExtensionType => typeof(IActionCommandExtension); public Type ExtensionType => typeof(IActionCommandExtension);
public void ProcessCommand(IExecutionContext context, string line, ActionCommand command, out bool omitEcho) public void ProcessCommand(IExecutionContext context, string line, ActionCommand command)
{ {
omitEcho = false;
var file = command.Data; var file = command.Data;
// File is required // File is required
@@ -339,26 +337,26 @@ namespace GitHub.Runner.Worker
public sealed class RemoveMatcherCommandExtension : RunnerService, IActionCommandExtension public sealed class RemoveMatcherCommandExtension : RunnerService, IActionCommandExtension
{ {
public string Command => "remove-matcher"; public string Command => "remove-matcher";
public bool OmitEcho => false;
public Type ExtensionType => typeof(IActionCommandExtension); public Type ExtensionType => typeof(IActionCommandExtension);
public void ProcessCommand(IExecutionContext context, string line, ActionCommand command, out bool omitEcho) public void ProcessCommand(IExecutionContext context, string line, ActionCommand command)
{ {
omitEcho = false;
command.Properties.TryGetValue(RemoveMatcherCommandProperties.Owner, out string owner); command.Properties.TryGetValue(RemoveMatcherCommandProperties.Owner, out string owner);
var file = command.Data; var file = command.Data;
// Owner and file are mutually exclusive // Owner and file are mutually exclusive
if (!string.IsNullOrEmpty(owner) && !string.IsNullOrEmpty(file)) if (!string.IsNullOrEmpty(owner) && !string.IsNullOrEmpty(file))
{ {
context.Warning("Either specify a matcher owner name or a file path. Both values cannot be set."); context.Warning("Either specify an owner name or a file path in ##[remove-matcher] command. Both values cannot be set.");
return; return;
} }
// Owner or file is required // Owner or file is required
if (string.IsNullOrEmpty(owner) && string.IsNullOrEmpty(file)) if (string.IsNullOrEmpty(owner) && string.IsNullOrEmpty(file))
{ {
context.Warning("Either a matcher owner name or a file path must be specified."); context.Warning("Either an owner name or a file path must be specified in ##[remove-matcher] command.");
return; return;
} }
@@ -407,12 +405,12 @@ namespace GitHub.Runner.Worker
public sealed class DebugCommandExtension : RunnerService, IActionCommandExtension public sealed class DebugCommandExtension : RunnerService, IActionCommandExtension
{ {
public string Command => "debug"; public string Command => "debug";
public bool OmitEcho => true;
public Type ExtensionType => typeof(IActionCommandExtension); public Type ExtensionType => typeof(IActionCommandExtension);
public void ProcessCommand(IExecutionContext context, string inputLine, ActionCommand command, out bool omitEcho) public void ProcessCommand(IExecutionContext context, string inputLine, ActionCommand command)
{ {
omitEcho = true;
context.Debug(command.Data); context.Debug(command.Data);
} }
} }
@@ -435,12 +433,12 @@ namespace GitHub.Runner.Worker
{ {
public abstract IssueType Type { get; } public abstract IssueType Type { get; }
public abstract string Command { get; } public abstract string Command { get; }
public bool OmitEcho => true;
public Type ExtensionType => typeof(IActionCommandExtension); public Type ExtensionType => typeof(IActionCommandExtension);
public void ProcessCommand(IExecutionContext context, string inputLine, ActionCommand command, out bool omitEcho) public void ProcessCommand(IExecutionContext context, string inputLine, ActionCommand command)
{ {
omitEcho = true;
command.Properties.TryGetValue(IssueCommandProperties.File, out string file); command.Properties.TryGetValue(IssueCommandProperties.File, out string file);
command.Properties.TryGetValue(IssueCommandProperties.Line, out string line); command.Properties.TryGetValue(IssueCommandProperties.Line, out string line);
command.Properties.TryGetValue(IssueCommandProperties.Column, out string column); command.Properties.TryGetValue(IssueCommandProperties.Column, out string column);
@@ -515,13 +513,42 @@ namespace GitHub.Runner.Worker
public abstract class GroupingCommandExtension : RunnerService, IActionCommandExtension public abstract class GroupingCommandExtension : RunnerService, IActionCommandExtension
{ {
public abstract string Command { get; } public abstract string Command { get; }
public bool OmitEcho => false;
public Type ExtensionType => typeof(IActionCommandExtension); public Type ExtensionType => typeof(IActionCommandExtension);
public void ProcessCommand(IExecutionContext context, string line, ActionCommand command, out bool omitEcho) public void ProcessCommand(IExecutionContext context, string line, ActionCommand command)
{ {
var data = this is GroupCommandExtension ? command.Data : string.Empty; var data = this is GroupCommandExtension ? command.Data : string.Empty;
context.Output($"##[{Command}]{data}"); context.Output($"##[{Command}]{data}");
omitEcho = true; }
}
public sealed class EchoCommandExtension : RunnerService, IActionCommandExtension
{
public string Command => "echo";
public bool OmitEcho => false;
public Type ExtensionType => typeof(IActionCommandExtension);
public void ProcessCommand(IExecutionContext context, string line, ActionCommand command)
{
ArgUtil.NotNullOrEmpty(command.Data, "value");
switch (command.Data.Trim().ToUpperInvariant())
{
case "ON":
context.EchoOnActionCommand = true;
context.Debug("Setting echo command value to 'on'");
break;
case "OFF":
context.EchoOnActionCommand = false;
context.Debug("Setting echo command value to 'off'");
break;
default:
throw new Exception($"Invalid echo command value. Possible values can be: 'on', 'off'. Current value is: '{command.Data}'.");
break;
}
} }
} }
} }

View File

@@ -58,6 +58,9 @@ namespace GitHub.Runner.Worker
executionContext.Warning("The 'PREVIEW_ACTION_TOKEN' secret is depreciated. Please remove it from the repository's secrets"); executionContext.Warning("The 'PREVIEW_ACTION_TOKEN' secret is depreciated. Please remove it from the repository's secrets");
} }
// Clear the cache (local runner)
IOUtil.DeleteDirectory(HostContext.GetDirectory(WellKnownDirectory.Actions), executionContext.CancellationToken);
foreach (var action in actions) foreach (var action in actions)
{ {
if (action.Reference.Type == Pipelines.ActionSourceType.ContainerRegistry) if (action.Reference.Type == Pipelines.ActionSourceType.ContainerRegistry)
@@ -445,7 +448,7 @@ namespace GitHub.Runner.Worker
} }
else else
{ {
// make sure we get an clean folder ready to use. // make sure we get a clean folder ready to use.
IOUtil.DeleteDirectory(destDirectory, executionContext.CancellationToken); IOUtil.DeleteDirectory(destDirectory, executionContext.CancellationToken);
Directory.CreateDirectory(destDirectory); Directory.CreateDirectory(destDirectory);
executionContext.Output($"Download action repository '{repositoryReference.Name}@{repositoryReference.Ref}'"); executionContext.Output($"Download action repository '{repositoryReference.Name}@{repositoryReference.Ref}'");

View File

@@ -94,7 +94,21 @@ namespace GitHub.Runner.Worker
{ {
postDisplayName = $"Post {this.DisplayName}"; postDisplayName = $"Post {this.DisplayName}";
} }
ExecutionContext.RegisterPostJobAction(postDisplayName, handlerData.CleanupCondition, Action);
var repositoryReference = Action.Reference as RepositoryPathReference;
var pathString = string.IsNullOrEmpty(repositoryReference.Path) ? string.Empty : $"/{repositoryReference.Path}";
var repoString = string.IsNullOrEmpty(repositoryReference.Ref) ? $"{repositoryReference.Name}{pathString}" :
$"{repositoryReference.Name}{pathString}@{repositoryReference.Ref}";
ExecutionContext.Debug($"Register post job cleanup for action: {repoString}");
var actionRunner = HostContext.CreateService<IActionRunner>();
actionRunner.Action = Action;
actionRunner.Stage = ActionRunStage.Post;
actionRunner.Condition = handlerData.CleanupCondition;
actionRunner.DisplayName = postDisplayName;
ExecutionContext.RegisterPostJobStep($"{actionRunner.Action.Name}_post", actionRunner);
} }
IStepHost stepHost = HostContext.CreateService<IDefaultStepHost>(); IStepHost stepHost = HostContext.CreateService<IDefaultStepHost>();

View File

@@ -11,6 +11,7 @@ using GitHub.Runner.Common;
using GitHub.Runner.Sdk; using GitHub.Runner.Sdk;
using GitHub.DistributedTask.Pipelines.ContextData; using GitHub.DistributedTask.Pipelines.ContextData;
using Microsoft.Win32; using Microsoft.Win32;
using GitHub.DistributedTask.Pipelines.ObjectTemplating;
namespace GitHub.Runner.Worker namespace GitHub.Runner.Worker
{ {
@@ -38,6 +39,14 @@ namespace GitHub.Runner.Worker
List<ContainerInfo> containers = data as List<ContainerInfo>; List<ContainerInfo> containers = data as List<ContainerInfo>;
ArgUtil.NotNull(containers, nameof(containers)); ArgUtil.NotNull(containers, nameof(containers));
var postJobStep = new JobExtensionRunner(runAsync: this.StopContainersAsync,
condition: $"{PipelineTemplateConstants.Always}()",
displayName: "Stop containers",
data: data);
executionContext.Debug($"Register post job cleanup for stoping/deleting containers.");
executionContext.RegisterPostJobStep(nameof(StopContainersAsync), postJobStep);
// Check whether we are inside a container. // Check whether we are inside a container.
// Our container feature requires to map working directory from host to the container. // Our container feature requires to map working directory from host to the container.
// If we are already inside a container, we will not able to find out the real working direcotry path on the host. // If we are already inside a container, we will not able to find out the real working direcotry path on the host.

View File

@@ -2,19 +2,9 @@
using System.Collections.Generic; using System.Collections.Generic;
using System.IO; using System.IO;
using System.IO.Compression; using System.IO.Compression;
using System.Text;
using System.Runtime.InteropServices;
using GitHub.DistributedTask.WebApi; using GitHub.DistributedTask.WebApi;
using GitHub.Runner.Common.Util;
using GitHub.Runner.Worker;
using GitHub.Runner.Common.Capabilities;
using GitHub.Services.WebApi;
using Microsoft.Win32;
using System.Diagnostics;
using System.Linq; using System.Linq;
using System.Collections.ObjectModel;
using System.Globalization; using System.Globalization;
using System.Threading;
using System.Threading.Tasks; using System.Threading.Tasks;
using Pipelines = GitHub.DistributedTask.Pipelines; using Pipelines = GitHub.DistributedTask.Pipelines;
using GitHub.Runner.Common; using GitHub.Runner.Common;

View File

@@ -12,14 +12,14 @@ using GitHub.Services.WebApi;
using GitHub.DistributedTask.Pipelines; using GitHub.DistributedTask.Pipelines;
using GitHub.DistributedTask.Pipelines.ContextData; using GitHub.DistributedTask.Pipelines.ContextData;
using GitHub.DistributedTask.WebApi; using GitHub.DistributedTask.WebApi;
using Pipelines = GitHub.DistributedTask.Pipelines;
using ObjectTemplating = GitHub.DistributedTask.ObjectTemplating;
using GitHub.Runner.Common.Util; using GitHub.Runner.Common.Util;
using GitHub.Runner.Common; using GitHub.Runner.Common;
using GitHub.Runner.Sdk; using GitHub.Runner.Sdk;
using Newtonsoft.Json; using Newtonsoft.Json;
using System.Text; using System.Text;
using System.Collections; using System.Collections;
using ObjectTemplating = GitHub.DistributedTask.ObjectTemplating;
using Pipelines = GitHub.DistributedTask.Pipelines;
namespace GitHub.Runner.Worker namespace GitHub.Runner.Worker
{ {
@@ -62,6 +62,8 @@ namespace GitHub.Runner.Worker
// Only job level ExecutionContext has PostJobSteps // Only job level ExecutionContext has PostJobSteps
Stack<IStep> PostJobSteps { get; } Stack<IStep> PostJobSteps { get; }
bool EchoOnActionCommand { get; set; }
// Initialize // Initialize
void InitializeJob(Pipelines.AgentJobRequestMessage message, CancellationToken token); void InitializeJob(Pipelines.AgentJobRequestMessage message, CancellationToken token);
void CancelToken(); void CancelToken();
@@ -96,7 +98,7 @@ namespace GitHub.Runner.Worker
// others // others
void ForceTaskComplete(); void ForceTaskComplete();
void RegisterPostJobAction(string displayName, string condition, Pipelines.ActionStep action); void RegisterPostJobStep(string refName, IStep step);
} }
public sealed class ExecutionContext : RunnerService, IExecutionContext public sealed class ExecutionContext : RunnerService, IExecutionContext
@@ -153,6 +155,8 @@ namespace GitHub.Runner.Worker
// Only job level ExecutionContext has PostJobSteps // Only job level ExecutionContext has PostJobSteps
public Stack<IStep> PostJobSteps { get; private set; } public Stack<IStep> PostJobSteps { get; private set; }
public bool EchoOnActionCommand { get; set; }
public TaskResult? Result public TaskResult? Result
{ {
@@ -236,27 +240,10 @@ namespace GitHub.Runner.Worker
}); });
} }
public void RegisterPostJobAction(string displayName, string condition, Pipelines.ActionStep action) public void RegisterPostJobStep(string refName, IStep step)
{ {
if (action.Reference.Type != ActionSourceType.Repository) step.ExecutionContext = Root.CreatePostChild(step.DisplayName, refName, IntraActionState);
{ Root.PostJobSteps.Push(step);
throw new NotSupportedException("Only action that has `action.yml` can define post job execution.");
}
var repositoryReference = action.Reference as RepositoryPathReference;
var pathString = string.IsNullOrEmpty(repositoryReference.Path) ? string.Empty : $"/{repositoryReference.Path}";
var repoString = string.IsNullOrEmpty(repositoryReference.Ref) ? $"{repositoryReference.Name}{pathString}" :
$"{repositoryReference.Name}{pathString}@{repositoryReference.Ref}";
this.Debug($"Register post job cleanup for action: {repoString}");
var actionRunner = HostContext.CreateService<IActionRunner>();
actionRunner.Action = action;
actionRunner.Stage = ActionRunStage.Post;
actionRunner.Condition = condition;
actionRunner.DisplayName = displayName;
actionRunner.ExecutionContext = Root.CreatePostChild(displayName, $"{actionRunner.Action.Name}_post", IntraActionState);
Root.PostJobSteps.Push(actionRunner);
} }
public IExecutionContext CreateChild(Guid recordId, string displayName, string refName, string scopeName, string contextName, Dictionary<string, string> intraActionState = null, int? recordOrder = null) public IExecutionContext CreateChild(Guid recordId, string displayName, string refName, string scopeName, string contextName, Dictionary<string, string> intraActionState = null, int? recordOrder = null)
@@ -292,6 +279,7 @@ namespace GitHub.Runner.Worker
child.PrependPath = PrependPath; child.PrependPath = PrependPath;
child.Container = Container; child.Container = Container;
child.ServiceContainers = ServiceContainers; child.ServiceContainers = ServiceContainers;
child.EchoOnActionCommand = EchoOnActionCommand;
if (recordOrder != null) if (recordOrder != null)
{ {
@@ -704,6 +692,9 @@ namespace GitHub.Runner.Worker
_logger = HostContext.CreateService<IPagingLogger>(); _logger = HostContext.CreateService<IPagingLogger>();
_logger.Setup(_mainTimelineId, _record.Id); _logger.Setup(_mainTimelineId, _record.Id);
// Initialize 'echo on action command success' property, default to false, unless Step_Debug is set
EchoOnActionCommand = Variables.Step_Debug ?? false;
// Verbosity (from GitHub.Step_Debug). // Verbosity (from GitHub.Step_Debug).
WriteDebug = Variables.Step_Debug ?? false; WriteDebug = Variables.Step_Debug ?? false;

View File

@@ -60,10 +60,16 @@ namespace GitHub.Runner.Worker.Handlers
if (string.IsNullOrEmpty(shell)) if (string.IsNullOrEmpty(shell))
{ {
#if OS_WINDOWS #if OS_WINDOWS
shellCommand = "cmd"; shellCommand = "pwsh";
if(validateShellOnHost) if(validateShellOnHost)
{ {
shellCommandPath = System.Environment.GetEnvironmentVariable("ComSpec"); shellCommandPath = WhichUtil.Which(shellCommand, require: false, Trace);
if (string.IsNullOrEmpty(shellCommandPath))
{
shellCommand = "powershell";
Trace.Info($"Defaulting to {shellCommand}");
shellCommandPath = WhichUtil.Which(shellCommand, require: true, Trace);
}
} }
#else #else
shellCommand = "sh"; shellCommand = "sh";
@@ -143,9 +149,15 @@ namespace GitHub.Runner.Worker.Handlers
if (string.IsNullOrEmpty(shell)) if (string.IsNullOrEmpty(shell))
{ {
#if OS_WINDOWS #if OS_WINDOWS
shellCommand = "cmd"; shellCommand = "pwsh";
commandPath = System.Environment.GetEnvironmentVariable("ComSpec"); commandPath = WhichUtil.Which(shellCommand, require: false, Trace);
ArgUtil.NotNullOrEmpty(commandPath, "%ComSpec%"); if (string.IsNullOrEmpty(commandPath))
{
shellCommand = "powershell";
Trace.Info($"Defaulting to {shellCommand}");
commandPath = WhichUtil.Which(shellCommand, require: true, Trace);
}
ArgUtil.NotNullOrEmpty(commandPath, "Default Shell");
#else #else
shellCommand = "sh"; shellCommand = "sh";
commandPath = WhichUtil.Which("bash", false, Trace) ?? WhichUtil.Which("sh", true, Trace); commandPath = WhichUtil.Which("bash", false, Trace) ?? WhichUtil.Which("sh", true, Trace);

View File

@@ -110,9 +110,7 @@ namespace GitHub.Runner.Worker
} }
} }
// Build up 3 lists of steps, pre-job, job, post-job // Build up 2 lists of steps, pre-job, job
var postJobStepsBuilder = new Stack<IStep>();
// Download actions not already in the cache // Download actions not already in the cache
Trace.Info("Downloading actions"); Trace.Info("Downloading actions");
var actionManager = HostContext.GetService<IActionManager>(); var actionManager = HostContext.GetService<IActionManager>();
@@ -134,10 +132,6 @@ namespace GitHub.Runner.Worker
condition: $"{PipelineTemplateConstants.Success}()", condition: $"{PipelineTemplateConstants.Success}()",
displayName: "Initialize containers", displayName: "Initialize containers",
data: (object)containers)); data: (object)containers));
postJobStepsBuilder.Push(new JobExtensionRunner(runAsync: containerProvider.StopContainersAsync,
condition: $"{PipelineTemplateConstants.Always}()",
displayName: "Stop containers",
data: (object)containers));
} }
// Add action steps // Add action steps
@@ -187,33 +181,9 @@ namespace GitHub.Runner.Worker
} }
} }
// Add post-job steps
Trace.Info("Adding post-job steps");
while (postJobStepsBuilder.Count > 0)
{
postJobSteps.Add(postJobStepsBuilder.Pop());
}
// Create execution context for post-job steps
foreach (var step in postJobSteps)
{
if (step is JobExtensionRunner)
{
JobExtensionRunner extensionStep = step as JobExtensionRunner;
ArgUtil.NotNull(extensionStep, extensionStep.DisplayName);
Guid stepId = Guid.NewGuid();
extensionStep.ExecutionContext = jobContext.CreateChild(stepId, extensionStep.DisplayName, stepId.ToString("N"), null, null);
}
}
List<IStep> steps = new List<IStep>(); List<IStep> steps = new List<IStep>();
steps.AddRange(preJobSteps); steps.AddRange(preJobSteps);
steps.AddRange(jobSteps); steps.AddRange(jobSteps);
steps.AddRange(postJobSteps);
// Start agent log plugin host process
// var logPlugin = HostContext.GetService<IAgentLogPlugin>();
// await logPlugin.StartAsync(context, steps, jobContext.CancellationToken);
// Prepare for orphan process cleanup // Prepare for orphan process cleanup
_processCleanup = jobContext.Variables.GetBoolean("process.clean") ?? true; _processCleanup = jobContext.Variables.GetBoolean("process.clean") ?? true;

View File

@@ -1,13 +1,15 @@
<Project Sdk="Microsoft.NET.Sdk"> <Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup> <PropertyGroup>
<TargetFramework>netcoreapp2.2</TargetFramework> <TargetFramework>netcoreapp3.0</TargetFramework>
<OutputType>Exe</OutputType> <OutputType>Exe</OutputType>
<RuntimeIdentifiers>win-x64;win-x86;linux-x64;linux-arm;rhel.6-x64;osx-x64</RuntimeIdentifiers> <RuntimeIdentifiers>win-x64;win-x86;linux-x64;linux-arm;rhel.6-x64;osx-x64</RuntimeIdentifiers>
<TargetLatestRuntimePatch>true</TargetLatestRuntimePatch> <TargetLatestRuntimePatch>true</TargetLatestRuntimePatch>
<AssetTargetFallback>portable-net45+win8</AssetTargetFallback> <AssetTargetFallback>portable-net45+win8</AssetTargetFallback>
<NoWarn>NU1701;NU1603</NoWarn> <NoWarn>NU1701;NU1603</NoWarn>
<Version>$(Version)</Version> <Version>$(Version)</Version>
<TieredCompilationQuickJit>true</TieredCompilationQuickJit>
<PublishReadyToRun>true</PublishReadyToRun>
</PropertyGroup> </PropertyGroup>
<ItemGroup> <ItemGroup>

View File

@@ -56,7 +56,7 @@ namespace GitHub.DistributedTask.Expressions2
// Attempt to match a named-value or index operator. // Attempt to match a named-value or index operator.
// Note, do not push children of the index operator. // Note, do not push children of the index operator.
if (node is NamedValue || node is Index) if (node is NamedValue || node is Sdk.Operators.Index)
{ {
// Lazy initialize the pattern segments // Lazy initialize the pattern segments
if (segmentedPatterns is null) if (segmentedPatterns is null)
@@ -201,7 +201,7 @@ namespace GitHub.DistributedTask.Expressions2
result.Push(node); result.Push(node);
} }
// Node is an index // Node is an index
else if (node is Index index) else if (node is Sdk.Operators.Index index)
{ {
while (true) while (true)
{ {
@@ -218,7 +218,7 @@ namespace GitHub.DistributedTask.Expressions2
break; break;
} }
// Parameter 0 is an index // Parameter 0 is an index
else if (parameter0 is Index index2) else if (parameter0 is Sdk.Operators.Index index2)
{ {
index = index2; index = index2;
} }

View File

@@ -5,6 +5,7 @@ using System.Text;
using Minimatch; using Minimatch;
using System.IO; using System.IO;
using System.Security.Cryptography; using System.Security.Cryptography;
using GitHub.DistributedTask.Expressions2.Sdk;
using GitHub.DistributedTask.Pipelines.ContextData; using GitHub.DistributedTask.Pipelines.ContextData;
using GitHub.DistributedTask.Pipelines.ObjectTemplating; using GitHub.DistributedTask.Pipelines.ObjectTemplating;
namespace GitHub.DistributedTask.Expressions2.Sdk.Functions namespace GitHub.DistributedTask.Expressions2.Sdk.Functions
@@ -28,29 +29,50 @@ namespace GitHub.DistributedTask.Expressions2.Sdk.Functions
string searchRoot = workspaceData.Value; string searchRoot = workspaceData.Value;
string pattern = Parameters[0].Evaluate(context).ConvertToString(); string pattern = Parameters[0].Evaluate(context).ConvertToString();
// Convert slashes on Windows
if (s_isWindows)
{
pattern = pattern.Replace('\\', '/');
}
// Root the pattern
if (!Path.IsPathRooted(pattern))
{
var patternRoot = s_isWindows ? searchRoot.Replace('\\', '/').TrimEnd('/') : searchRoot.TrimEnd('/');
pattern = string.Concat(patternRoot, "/", pattern);
}
// Get all files
context.Trace.Info($"Search root directory: '{searchRoot}'"); context.Trace.Info($"Search root directory: '{searchRoot}'");
context.Trace.Info($"Search pattern: '{pattern}'"); context.Trace.Info($"Search pattern: '{pattern}'");
var files = Directory.GetFiles(searchRoot, "*", SearchOption.AllDirectories).OrderBy(x => x).ToList(); var files = Directory.GetFiles(searchRoot, "*", SearchOption.AllDirectories)
.Select(x => s_isWindows ? x.Replace('\\', '/') : x)
.OrderBy(x => x, StringComparer.Ordinal)
.ToList();
if (files.Count == 0) if (files.Count == 0)
{ {
throw new ArgumentException($"'hashFiles({pattern})' failed. Directory '{searchRoot}' is empty"); throw new ArgumentException($"hashFiles('{ExpressionUtility.StringEscape(pattern)}') failed. Directory '{searchRoot}' is empty");
} }
else else
{ {
context.Trace.Info($"Found {files.Count} files"); context.Trace.Info($"Found {files.Count} files");
} }
// Match
var matcher = new Minimatcher(pattern, s_minimatchOptions); var matcher = new Minimatcher(pattern, s_minimatchOptions);
files = matcher.Filter(files).ToList(); files = matcher.Filter(files)
.Select(x => s_isWindows ? x.Replace('/', '\\') : x)
.ToList();
if (files.Count == 0) if (files.Count == 0)
{ {
throw new ArgumentException($"'hashFiles({pattern})' failed. Search pattern '{pattern}' doesn't match any file under '{searchRoot}'"); throw new ArgumentException($"hashFiles('{ExpressionUtility.StringEscape(pattern)}') failed. Search pattern '{pattern}' doesn't match any file under '{searchRoot}'");
} }
else else
{ {
context.Trace.Info($"{files.Count} matches to hash"); context.Trace.Info($"{files.Count} matches to hash");
} }
// Hash each file
List<byte> filesSha256 = new List<byte>(); List<byte> filesSha256 = new List<byte>();
foreach (var file in files) foreach (var file in files)
{ {
@@ -64,6 +86,7 @@ namespace GitHub.DistributedTask.Expressions2.Sdk.Functions
} }
} }
// Hash the hashes
using (SHA256 sha256hash = SHA256.Create()) using (SHA256 sha256hash = SHA256.Create())
{ {
var hashBytes = sha256hash.ComputeHash(filesSha256.ToArray()); var hashBytes = sha256hash.ComputeHash(filesSha256.ToArray());
@@ -83,11 +106,17 @@ namespace GitHub.DistributedTask.Expressions2.Sdk.Functions
} }
} }
private static readonly bool s_isWindows = Environment.OSVersion.Platform != PlatformID.Unix && Environment.OSVersion.Platform != PlatformID.MacOSX;
// Only support basic globbing (* ? and []) and globstar (**)
private static readonly Options s_minimatchOptions = new Options private static readonly Options s_minimatchOptions = new Options
{ {
Dot = true, Dot = true,
NoBrace = true, NoBrace = true,
NoCase = Environment.OSVersion.Platform != PlatformID.Unix && Environment.OSVersion.Platform != PlatformID.MacOSX NoCase = s_isWindows,
NoComment = true,
NoExt = true,
NoNegate = true,
}; };
} }
} }

View File

@@ -154,7 +154,7 @@ namespace GitHub.DistributedTask.Expressions2.Tokens
{ {
case TokenKind.StartIndex: // "[" case TokenKind.StartIndex: // "["
case TokenKind.Dereference: // "." case TokenKind.Dereference: // "."
return new Index(); return new Sdk.Operators.Index();
case TokenKind.LogicalOperator: case TokenKind.LogicalOperator:
switch (RawValue) switch (RawValue)

View File

@@ -17,10 +17,16 @@ namespace GitHub.DistributedTask.Logging
return Convert.ToBase64String(Encoding.UTF8.GetBytes(value)); return Convert.ToBase64String(Encoding.UTF8.GetBytes(value));
} }
// Base64 is 6 bytes -> char // Base64 is 6 bits -> char
// A byte is 8 bits
// When end user doing somthing like base64(user:password) // When end user doing somthing like base64(user:password)
// The length of the leading content will cause different base64 encoding result on the password // The length of the leading content will cause different base64 encoding result on the password
// So we add base64(value - 1/2/3/4/5 bytes) as secret as well. // So we add base64(value shifted 1 and two bytes) as secret as well.
// B1 B2 B3 B4 B5 B6 B7
// 000000|00 0000|0000 00|000000| 000000|00 0000|0000 00|000000|
// Char1 Char2 Char3 Char4
// See the above, the first byte has a character beginning at index 0, the second byte has a character beginning at index 4, the third byte has a character beginning at index 2 and then the pattern repeats
// We register byte offsets for all these possible values
public static String Base64StringEscapeShift1(String value) public static String Base64StringEscapeShift1(String value)
{ {
return Base64StringEscapeShift(value, 1); return Base64StringEscapeShift(value, 1);
@@ -31,21 +37,6 @@ namespace GitHub.DistributedTask.Logging
return Base64StringEscapeShift(value, 2); return Base64StringEscapeShift(value, 2);
} }
public static String Base64StringEscapeShift3(String value)
{
return Base64StringEscapeShift(value, 3);
}
public static String Base64StringEscapeShift4(String value)
{
return Base64StringEscapeShift(value, 4);
}
public static String Base64StringEscapeShift5(String value)
{
return Base64StringEscapeShift(value, 5);
}
public static String ExpressionStringEscape(String value) public static String ExpressionStringEscape(String value)
{ {
return Expressions.ExpressionUtil.StringEscape(value); return Expressions.ExpressionUtil.StringEscape(value);

View File

@@ -55,14 +55,9 @@ namespace GitHub.DistributedTask.WebApi
m_properties = new PropertiesCollection(agentToBeCloned.m_properties); m_properties = new PropertiesCollection(agentToBeCloned.m_properties);
} }
if (agentToBeCloned.m_systemCapabilities != null && agentToBeCloned.m_systemCapabilities.Count > 0) if (agentToBeCloned.m_labels != null && agentToBeCloned.m_labels.Count > 0)
{ {
m_systemCapabilities = new Dictionary<String, String>(agentToBeCloned.m_systemCapabilities, StringComparer.OrdinalIgnoreCase); m_labels = new HashSet<string>(agentToBeCloned.m_labels, StringComparer.OrdinalIgnoreCase);
}
if (agentToBeCloned.m_userCapabilities != null && agentToBeCloned.m_userCapabilities.Count > 0)
{
m_userCapabilities = new Dictionary<String, String>(agentToBeCloned.m_userCapabilities, StringComparer.OrdinalIgnoreCase);
} }
if (agentToBeCloned.PendingUpdate != null) if (agentToBeCloned.PendingUpdate != null)
@@ -152,32 +147,17 @@ namespace GitHub.DistributedTask.WebApi
} }
/// <summary> /// <summary>
/// System-defined capabilities supported by this agent's host. /// The labels of the runner
/// </summary> /// </summary>
public IDictionary<String, String> SystemCapabilities public ISet<string> Labels
{ {
get get
{ {
if (m_systemCapabilities == null) if (m_labels == null)
{ {
m_systemCapabilities = new Dictionary<String, String>(StringComparer.OrdinalIgnoreCase); m_labels = new HashSet<string>(StringComparer.OrdinalIgnoreCase);
} }
return m_systemCapabilities; return m_labels;
}
}
/// <summary>
/// User-defined capabilities supported by this agent's host.
/// </summary>
public IDictionary<String, String> UserCapabilities
{
get
{
if (m_userCapabilities == null)
{
m_userCapabilities = new Dictionary<String, String>(StringComparer.OrdinalIgnoreCase);
}
return m_userCapabilities;
} }
} }
@@ -214,10 +194,7 @@ namespace GitHub.DistributedTask.WebApi
[DataMember(IsRequired = false, EmitDefaultValue = false, Name = "Properties")] [DataMember(IsRequired = false, EmitDefaultValue = false, Name = "Properties")]
private PropertiesCollection m_properties; private PropertiesCollection m_properties;
[DataMember(IsRequired = false, EmitDefaultValue = false, Name = "SystemCapabilities")] [DataMember(IsRequired = false, EmitDefaultValue = false, Name = "Labels")]
private Dictionary<String, String> m_systemCapabilities; private HashSet<string> m_labels;
[DataMember(IsRequired = false, EmitDefaultValue = false, Name = "UserCapabilities")]
private Dictionary<String, String> m_userCapabilities;
} }
} }

View File

@@ -246,8 +246,7 @@ namespace GitHub.DistributedTask.WebApi
PlanType = hubName, PlanType = hubName,
ScopeId = scopeIdentifier, ScopeId = scopeIdentifier,
PlanId = planId, PlanId = planId,
JobId = jobId, JobId = jobId
Demands = demands,
}; };
return QueueAgentRequestByPoolAsync(poolId, request, userState, cancellationToken); return QueueAgentRequestByPoolAsync(poolId, request, userState, cancellationToken);

View File

@@ -35,7 +35,6 @@ namespace GitHub.DistributedTask.WebApi
this.PoolId = requestToBeCloned.PoolId; this.PoolId = requestToBeCloned.PoolId;
this.JobId = requestToBeCloned.JobId; this.JobId = requestToBeCloned.JobId;
this.JobName = requestToBeCloned.JobName; this.JobName = requestToBeCloned.JobName;
this.Demands = new List<Demand>(requestToBeCloned.Demands ?? new Demand[0]);
this.LockToken = requestToBeCloned.LockToken; this.LockToken = requestToBeCloned.LockToken;
this.ExpectedDuration = requestToBeCloned.ExpectedDuration; this.ExpectedDuration = requestToBeCloned.ExpectedDuration;
this.OrchestrationId = requestToBeCloned.OrchestrationId; this.OrchestrationId = requestToBeCloned.OrchestrationId;
@@ -68,6 +67,11 @@ namespace GitHub.DistributedTask.WebApi
{ {
this.AgentSpecification = new JObject(requestToBeCloned.AgentSpecification); this.AgentSpecification = new JObject(requestToBeCloned.AgentSpecification);
} }
if (requestToBeCloned.Labels != null)
{
this.Labels = new HashSet<string>(requestToBeCloned.Labels, StringComparer.OrdinalIgnoreCase);
}
} }
/// <summary> /// <summary>
@@ -229,6 +233,7 @@ namespace GitHub.DistributedTask.WebApi
/// </summary> /// </summary>
/// <value></value> /// <value></value>
[DataMember(Order = 16, EmitDefaultValue = false)] [DataMember(Order = 16, EmitDefaultValue = false)]
[Obsolete("No more demands, use labels", true)]
public IList<Demand> Demands public IList<Demand> Demands
{ {
get; get;
@@ -386,6 +391,13 @@ namespace GitHub.DistributedTask.WebApi
set; set;
} }
[DataMember(Order = 33, EmitDefaultValue = false)]
public ISet<string> Labels
{
get;
set;
}
[IgnoreDataMember] [IgnoreDataMember]
internal Guid? LockToken internal Guid? LockToken
{ {

View File

@@ -1,5 +1,4 @@
using System; using System;
using System.Collections.Generic;
using System.Runtime.Serialization; using System.Runtime.Serialization;
namespace GitHub.DistributedTask.WebApi namespace GitHub.DistributedTask.WebApi
@@ -27,29 +26,6 @@ namespace GitHub.DistributedTask.WebApi
this.OwnerName = ownerName; this.OwnerName = ownerName;
} }
/// <summary>
/// Initializes a new <c>TaskAgentSession</c> isntance with the specified owner name, agent, and capabilities.
/// </summary>
/// <param name="ownerName">The name of the owner for this session. This should typically be the agent machine</param>
/// <param name="agent">The target agent for the session</param>
/// <param name="systemCapabilities">A collection of capabilities to publish on session creation</param>
public TaskAgentSession(
String ownerName,
TaskAgentReference agent,
IDictionary<String, String> systemCapabilities)
{
this.Agent = agent;
this.OwnerName = ownerName;
foreach (var capability in systemCapabilities)
{
if (capability.Value != null)
{
this.SystemCapabilities.Add(capability.Key, capability.Value);
}
}
}
/// <summary> /// <summary>
/// Gets the unique identifier for this session. /// Gets the unique identifier for this session.
/// </summary> /// </summary>
@@ -89,33 +65,5 @@ namespace GitHub.DistributedTask.WebApi
get; get;
set; set;
} }
/// <summary>
/// Gets the collection of system capabilities used for this session.
/// </summary>
public IDictionary<String, String> SystemCapabilities
{
get
{
if (m_systemCapabilities == null)
{
m_systemCapabilities = new Dictionary<String, String>(StringComparer.OrdinalIgnoreCase);
}
return m_systemCapabilities;
}
}
[OnSerializing]
private void OnSerializing(StreamingContext context)
{
if (m_systemCapabilities?.Count == 0)
{
m_systemCapabilities = null;
}
}
[DataMember(IsRequired = false, EmitDefaultValue = false, Name = "SystemCapabilities")]
private IDictionary<String, String> m_systemCapabilities;
} }
} }

View File

@@ -1,7 +1,7 @@
<Project Sdk="Microsoft.NET.Sdk"> <Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup> <PropertyGroup>
<TargetFramework>netcoreapp2.2</TargetFramework> <TargetFramework>netcoreapp3.0</TargetFramework>
<OutputType>Library</OutputType> <OutputType>Library</OutputType>
<RuntimeIdentifiers>win-x64;win-x86;linux-x64;linux-arm;rhel.6-x64;osx-x64</RuntimeIdentifiers> <RuntimeIdentifiers>win-x64;win-x86;linux-x64;linux-arm;rhel.6-x64;osx-x64</RuntimeIdentifiers>
<TargetLatestRuntimePatch>true</TargetLatestRuntimePatch> <TargetLatestRuntimePatch>true</TargetLatestRuntimePatch>
@@ -11,6 +11,7 @@
<DefineConstants>NETSTANDARD;NET_STANDARD;TRACE</DefineConstants> <DefineConstants>NETSTANDARD;NET_STANDARD;TRACE</DefineConstants>
<LangVersion>7.3</LangVersion> <LangVersion>7.3</LangVersion>
<AllowUnsafeBlocks>true</AllowUnsafeBlocks> <AllowUnsafeBlocks>true</AllowUnsafeBlocks>
<TieredCompilationQuickJit>true</TieredCompilationQuickJit>
</PropertyGroup> </PropertyGroup>
<ItemGroup> <ItemGroup>
@@ -21,11 +22,7 @@
<PackageReference Include="System.Security.Cryptography.Cng" Version="4.4.0" /> <PackageReference Include="System.Security.Cryptography.Cng" Version="4.4.0" />
<PackageReference Include="System.Security.Cryptography.Pkcs" Version="4.4.0" /> <PackageReference Include="System.Security.Cryptography.Pkcs" Version="4.4.0" />
<PackageReference Include="System.Security.Cryptography.ProtectedData" Version="4.4.0" /> <PackageReference Include="System.Security.Cryptography.ProtectedData" Version="4.4.0" />
<PackageReference Include="System.Data.SqlClient" Version="4.4.0" />
<PackageReference Include="System.Configuration.ConfigurationManager" Version="4.4.0" />
<PackageReference Include="Microsoft.IdentityModel.Clients.ActiveDirectory" Version="3.19.4" /> <PackageReference Include="Microsoft.IdentityModel.Clients.ActiveDirectory" Version="3.19.4" />
<PackageReference Include="Microsoft.ApplicationInsights" Version="2.9.1" />
<PackageReference Include="WindowsAzure.Storage" Version="8.7.0" />
<PackageReference Include="Minimatch" Version="2.0.0" /> <PackageReference Include="Minimatch" Version="2.0.0" />
<PackageReference Include="YamlDotNet.Signed" Version="5.3.0" /> <PackageReference Include="YamlDotNet.Signed" Version="5.3.0" />
</ItemGroup> </ItemGroup>

View File

@@ -397,6 +397,11 @@ namespace GitHub.Services.FileContainer.Client
{ {
break; break;
} }
else if (IsFastFailResponse(response))
{
FileUploadTrace(itemPath, $"Chunk '{currentChunk}' attempt '{attempt}' of file '{itemPath}' received non-success status code {response.StatusCode} for sending request and cannot continue.");
break;
}
else else
{ {
FileUploadTrace(itemPath, $"Chunk '{currentChunk}' attempt '{attempt}' of file '{itemPath}' received non-success status code {response.StatusCode} for sending request."); FileUploadTrace(itemPath, $"Chunk '{currentChunk}' attempt '{attempt}' of file '{itemPath}' received non-success status code {response.StatusCode} for sending request.");
@@ -538,6 +543,17 @@ namespace GitHub.Services.FileContainer.Client
cancellationToken); cancellationToken);
} }
public bool IsFastFailResponse(HttpResponseMessage response)
{
int statusCode = (int)response?.StatusCode;
return statusCode >= 400 && statusCode <= 499;
}
protected override bool ShouldThrowError(HttpResponseMessage response)
{
return !response.IsSuccessStatusCode && !IsFastFailResponse(response);
}
private async Task<HttpResponseMessage> ContainerGetRequestAsync( private async Task<HttpResponseMessage> ContainerGetRequestAsync(
Int64 containerId, Int64 containerId,
String itemPath, String itemPath,

View File

@@ -1,5 +1,4 @@
using GitHub.Runner.Common.Capabilities; using GitHub.Runner.Worker;
using GitHub.Runner.Worker;
using System; using System;
using System.Collections.Generic; using System.Collections.Generic;
using System.Linq; using System.Linq;
@@ -21,12 +20,12 @@ namespace GitHub.Runner.Common.Tests
manager.Initialize(tc); manager.Initialize(tc);
// Act. // Act.
List<ICapabilitiesProvider> extensions = manager.GetExtensions<ICapabilitiesProvider>(); List<IActionCommandExtension> extensions = manager.GetExtensions<IActionCommandExtension>();
// Assert. // Assert.
Assert.True( Assert.True(
extensions.Any(x => x is RunnerCapabilitiesProvider), extensions.Any(x => x is SetEnvCommandExtension),
$"Expected {nameof(RunnerCapabilitiesProvider)} extension to be returned as a job extension."); $"Expected {nameof(SetEnvCommandExtension)} extension to be returned as a job extension.");
} }
} }
@@ -42,9 +41,9 @@ namespace GitHub.Runner.Common.Tests
manager.Initialize(tc); manager.Initialize(tc);
// Act/Assert. // Act/Assert.
AssertContains<GitHub.Runner.Common.Capabilities.ICapabilitiesProvider>( AssertContains<GitHub.Runner.Worker.IActionCommandExtension>(
manager, manager,
concreteType: typeof(GitHub.Runner.Common.Capabilities.RunnerCapabilitiesProvider)); concreteType: typeof(GitHub.Runner.Worker.SetEnvCommandExtension));
} }
} }

View File

@@ -1,7 +1,9 @@
using GitHub.Runner.Common.Util; using GitHub.Runner.Common.Util;
using System;
using System.IO; using System.IO;
using System.Reflection; using System.Reflection;
using System.Runtime.CompilerServices; using System.Runtime.CompilerServices;
using System.Text;
using System.Threading; using System.Threading;
using Xunit; using Xunit;
@@ -67,6 +69,44 @@ namespace GitHub.Runner.Common.Tests
} }
} }
[Fact]
[Trait("Level", "L0")]
[Trait("Category", "Common")]
public void DefaultSecretMaskers()
{
try
{
// Arrange.
Setup();
// Act.
_hc.SecretMasker.AddValue("Password123!");
_hc.SecretMasker.AddValue("Pass\"word\"123!");
_hc.SecretMasker.AddValue("Pass word 123!");
_hc.SecretMasker.AddValue("Pass<word>123!");
_hc.SecretMasker.AddValue("Pass'word'123!");
// Assert.
Assert.Equal("123***123", _hc.SecretMasker.MaskSecrets("123Password123!123"));
Assert.Equal("password123", _hc.SecretMasker.MaskSecrets("password123"));
Assert.Equal("123***123", _hc.SecretMasker.MaskSecrets("123Pass\\\"word\\\"123!123"));
Assert.Equal("123***123", _hc.SecretMasker.MaskSecrets("123Pass%20word%20123%21123"));
Assert.Equal("123***123", _hc.SecretMasker.MaskSecrets("123Pass&lt;word&gt;123!123"));
Assert.Equal("123***123", _hc.SecretMasker.MaskSecrets("123Pass''word''123!123"));
Assert.Equal("OlBh***", _hc.SecretMasker.MaskSecrets(Convert.ToBase64String(Encoding.UTF8.GetBytes($":Password123!"))));
Assert.Equal("YTpQ***", _hc.SecretMasker.MaskSecrets(Convert.ToBase64String(Encoding.UTF8.GetBytes($"a:Password123!"))));
Assert.Equal("YWI6***", _hc.SecretMasker.MaskSecrets(Convert.ToBase64String(Encoding.UTF8.GetBytes($"ab:Password123!"))));
Assert.Equal("YWJjOlBh***", _hc.SecretMasker.MaskSecrets(Convert.ToBase64String(Encoding.UTF8.GetBytes($"abc:Password123!"))));
Assert.Equal("YWJjZDpQ***", _hc.SecretMasker.MaskSecrets(Convert.ToBase64String(Encoding.UTF8.GetBytes($"abcd:Password123!"))));
Assert.Equal("YWJjZGU6***", _hc.SecretMasker.MaskSecrets(Convert.ToBase64String(Encoding.UTF8.GetBytes($"abcde:Password123!"))));
}
finally
{
// Cleanup.
Teardown();
}
}
public void Setup([CallerMemberName] string testName = "") public void Setup([CallerMemberName] string testName = "")
{ {
_tokenSource = new CancellationTokenSource(); _tokenSource = new CancellationTokenSource();

View File

@@ -1,79 +0,0 @@
using GitHub.Runner.Common.Capabilities;
using GitHub.Runner.Listener.Configuration;
using Moq;
using System;
using System.Collections.Generic;
using System.Linq;
using System.Threading;
using System.Threading.Tasks;
using Xunit;
namespace GitHub.Runner.Common.Tests.Listener
{
public sealed class AgentCapabilitiesProviderTestL0
{
[Fact]
[Trait("Level", "L0")]
[Trait("Category", "Agent")]
public async void TestGetCapabilities()
{
using (var hc = new TestHostContext(this))
using (var tokenSource = new CancellationTokenSource())
{
Mock<IConfigurationManager> configurationManager = new Mock<IConfigurationManager>();
hc.SetSingleton<IConfigurationManager>(configurationManager.Object);
// Arrange
var provider = new RunnerCapabilitiesProvider();
provider.Initialize(hc);
var settings = new RunnerSettings() { AgentName = "IAmAgent007" };
// Act
List<Capability> capabilities = await provider.GetCapabilitiesAsync(settings, tokenSource.Token);
// Assert
Assert.NotNull(capabilities);
Capability runnerNameCapability = capabilities.SingleOrDefault(x => string.Equals(x.Name, "Runner.Name", StringComparison.Ordinal));
Assert.NotNull(runnerNameCapability);
Assert.Equal("IAmAgent007", runnerNameCapability.Value);
}
}
[Fact]
[Trait("Level", "L0")]
[Trait("Category", "Agent")]
public async void TestInteractiveSessionCapability()
{
using (var hc = new TestHostContext(this))
using (var tokenSource = new CancellationTokenSource())
{
hc.StartupType = StartupType.AutoStartup;
await VerifyInteractiveSessionCapability(hc, tokenSource.Token, true);
hc.StartupType = StartupType.Service;
await VerifyInteractiveSessionCapability(hc, tokenSource.Token, false);
hc.StartupType = StartupType.Manual;
await VerifyInteractiveSessionCapability(hc, tokenSource.Token, true);
}
}
private async Task VerifyInteractiveSessionCapability(IHostContext hc, CancellationToken token, bool expectedValue)
{
// Arrange
var provider = new RunnerCapabilitiesProvider();
provider.Initialize(hc);
var settings = new RunnerSettings() { AgentName = "IAmAgent007" };
// Act
List<Capability> capabilities = await provider.GetCapabilitiesAsync(settings, token);
// Assert
Assert.NotNull(capabilities);
Capability iSessionCapability = capabilities.SingleOrDefault(x => string.Equals(x.Name, "InteractiveSession", StringComparison.Ordinal));
Assert.NotNull(iSessionCapability);
bool.TryParse(iSessionCapability.Value, out bool isInteractive);
Assert.Equal(expectedValue, isInteractive);
}
}
}

View File

@@ -1,6 +1,5 @@
using GitHub.DistributedTask.WebApi; using GitHub.DistributedTask.WebApi;
using GitHub.Runner.Listener; using GitHub.Runner.Listener;
using GitHub.Runner.Common.Capabilities;
using GitHub.Runner.Listener.Configuration; using GitHub.Runner.Listener.Configuration;
using GitHub.Runner.Common.Util; using GitHub.Runner.Common.Util;
using GitHub.Services.WebApi; using GitHub.Services.WebApi;
@@ -40,8 +39,6 @@ namespace GitHub.Runner.Common.Tests.Listener.Configuration
#endif #endif
private Mock<IRSAKeyManager> _rsaKeyManager; private Mock<IRSAKeyManager> _rsaKeyManager;
private ICapabilitiesManager _capabilitiesManager;
// private DeploymentGroupAgentConfigProvider _deploymentGroupAgentConfigProvider;
private string _expectedToken = "expectedToken"; private string _expectedToken = "expectedToken";
private string _expectedServerUrl = "https://localhost"; private string _expectedServerUrl = "https://localhost";
private string _expectedAgentName = "expectedAgentName"; private string _expectedAgentName = "expectedAgentName";
@@ -78,8 +75,6 @@ namespace GitHub.Runner.Common.Tests.Listener.Configuration
_serviceControlManager = new Mock<ILinuxServiceControlManager>(); _serviceControlManager = new Mock<ILinuxServiceControlManager>();
#endif #endif
_capabilitiesManager = new CapabilitiesManager();
var expectedAgent = new TaskAgent(_expectedAgentName) { Id = 1 }; var expectedAgent = new TaskAgent(_expectedAgentName) { Id = 1 };
var expectedDeploymentMachine = new DeploymentMachine() { Agent = expectedAgent, Id = _expectedDeploymentMachineId }; var expectedDeploymentMachine = new DeploymentMachine() { Agent = expectedAgent, Id = _expectedDeploymentMachineId };
expectedAgent.Authorization = new TaskAgentAuthorization expectedAgent.Authorization = new TaskAgentAuthorization
@@ -127,7 +122,7 @@ namespace GitHub.Runner.Common.Tests.Listener.Configuration
_agentServer.Setup(x => x.GetAgentsAsync(It.IsAny<int>(), It.IsAny<string>())).Returns(Task.FromResult(expectedAgents)); _agentServer.Setup(x => x.GetAgentsAsync(It.IsAny<int>(), It.IsAny<string>())).Returns(Task.FromResult(expectedAgents));
_agentServer.Setup(x => x.AddAgentAsync(It.IsAny<int>(), It.IsAny<TaskAgent>())).Returns(Task.FromResult(expectedAgent)); _agentServer.Setup(x => x.AddAgentAsync(It.IsAny<int>(), It.IsAny<TaskAgent>())).Returns(Task.FromResult(expectedAgent));
_agentServer.Setup(x => x.UpdateAgentAsync(It.IsAny<int>(), It.IsAny<TaskAgent>())).Returns(Task.FromResult(expectedAgent)); _agentServer.Setup(x => x.ReplaceAgentAsync(It.IsAny<int>(), It.IsAny<TaskAgent>())).Returns(Task.FromResult(expectedAgent));
rsa = new RSACryptoServiceProvider(2048); rsa = new RSACryptoServiceProvider(2048);
@@ -143,8 +138,6 @@ namespace GitHub.Runner.Common.Tests.Listener.Configuration
tc.SetSingleton<IExtensionManager>(_extnMgr.Object); tc.SetSingleton<IExtensionManager>(_extnMgr.Object);
tc.SetSingleton<IRunnerServer>(_agentServer.Object); tc.SetSingleton<IRunnerServer>(_agentServer.Object);
tc.SetSingleton<ILocationServer>(_locationServer.Object); tc.SetSingleton<ILocationServer>(_locationServer.Object);
// tc.SetSingleton<IDeploymentGroupServer>(_machineGroupServer.Object);
tc.SetSingleton<ICapabilitiesManager>(_capabilitiesManager);
tc.SetSingleton<IRunnerWebProxy>(_runnerWebProxy.Object); tc.SetSingleton<IRunnerWebProxy>(_runnerWebProxy.Object);
tc.SetSingleton<IRunnerCertificateManager>(_cert.Object); tc.SetSingleton<IRunnerCertificateManager>(_cert.Object);
@@ -208,12 +201,7 @@ namespace GitHub.Runner.Common.Tests.Listener.Configuration
// validate GetAgentPoolsAsync gets called once with automation pool type // validate GetAgentPoolsAsync gets called once with automation pool type
_agentServer.Verify(x => x.GetAgentPoolsAsync(It.IsAny<string>(), It.Is<TaskAgentPoolType>(p => p == TaskAgentPoolType.Automation)), Times.Once); _agentServer.Verify(x => x.GetAgentPoolsAsync(It.IsAny<string>(), It.Is<TaskAgentPoolType>(p => p == TaskAgentPoolType.Automation)), Times.Once);
// validate GetAgentPoolsAsync not called with deployment pool type _agentServer.Verify(x => x.AddAgentAsync(It.IsAny<int>(), It.Is<TaskAgent>(a => a.Labels.Contains("self-hosted") && a.Labels.Contains(VarUtil.OS) && a.Labels.Contains(VarUtil.OSArchitecture))), Times.Once);
_agentServer.Verify(x => x.GetAgentPoolsAsync(It.IsAny<string>(), It.Is<TaskAgentPoolType>(p => p == TaskAgentPoolType.Deployment)), Times.Never);
// For build and release agent / deployment pool, tags logic should not get trigger;
// _machineGroupServer.Verify(x =>
// x.UpdateDeploymentTargetsAsync(It.IsAny<Guid>(), It.IsAny<int>(), It.IsAny<List<DeploymentMachine>>()), Times.Never);
} }
} }
} }

View File

@@ -2,7 +2,6 @@
using GitHub.Services.Common; using GitHub.Services.Common;
using GitHub.Services.WebApi; using GitHub.Services.WebApi;
using GitHub.Runner.Listener; using GitHub.Runner.Listener;
using GitHub.Runner.Common.Capabilities;
using GitHub.Runner.Listener.Configuration; using GitHub.Runner.Listener.Configuration;
using Moq; using Moq;
using System; using System;
@@ -21,7 +20,6 @@ namespace GitHub.Runner.Common.Tests.Listener
private Mock<IConfigurationManager> _config; private Mock<IConfigurationManager> _config;
private Mock<IRunnerServer> _agentServer; private Mock<IRunnerServer> _agentServer;
private Mock<ICredentialManager> _credMgr; private Mock<ICredentialManager> _credMgr;
private Mock<ICapabilitiesManager> _capabilitiesManager;
public MessageListenerL0() public MessageListenerL0()
{ {
@@ -30,7 +28,6 @@ namespace GitHub.Runner.Common.Tests.Listener
_config.Setup(x => x.LoadSettings()).Returns(_settings); _config.Setup(x => x.LoadSettings()).Returns(_settings);
_agentServer = new Mock<IRunnerServer>(); _agentServer = new Mock<IRunnerServer>();
_credMgr = new Mock<ICredentialManager>(); _credMgr = new Mock<ICredentialManager>();
_capabilitiesManager = new Mock<ICapabilitiesManager>();
} }
private TestHostContext CreateTestContext([CallerMemberName] String testName = "") private TestHostContext CreateTestContext([CallerMemberName] String testName = "")
@@ -39,7 +36,6 @@ namespace GitHub.Runner.Common.Tests.Listener
tc.SetSingleton<IConfigurationManager>(_config.Object); tc.SetSingleton<IConfigurationManager>(_config.Object);
tc.SetSingleton<IRunnerServer>(_agentServer.Object); tc.SetSingleton<IRunnerServer>(_agentServer.Object);
tc.SetSingleton<ICredentialManager>(_credMgr.Object); tc.SetSingleton<ICredentialManager>(_credMgr.Object);
tc.SetSingleton<ICapabilitiesManager>(_capabilitiesManager.Object);
return tc; return tc;
} }
@@ -62,8 +58,6 @@ namespace GitHub.Runner.Common.Tests.Listener
tokenSource.Token)) tokenSource.Token))
.Returns(Task.FromResult(expectedSession)); .Returns(Task.FromResult(expectedSession));
_capabilitiesManager.Setup(x => x.GetCapabilitiesAsync(_settings, It.IsAny<CancellationToken>())).Returns(Task.FromResult(new Dictionary<string, string>()));
_credMgr.Setup(x => x.LoadCredentials()).Returns(new VssCredentials()); _credMgr.Setup(x => x.LoadCredentials()).Returns(new VssCredentials());
// Act. // Act.
@@ -106,8 +100,6 @@ namespace GitHub.Runner.Common.Tests.Listener
tokenSource.Token)) tokenSource.Token))
.Returns(Task.FromResult(expectedSession)); .Returns(Task.FromResult(expectedSession));
_capabilitiesManager.Setup(x => x.GetCapabilitiesAsync(_settings, It.IsAny<CancellationToken>())).Returns(Task.FromResult(new Dictionary<string, string>()));
_credMgr.Setup(x => x.LoadCredentials()).Returns(new VssCredentials()); _credMgr.Setup(x => x.LoadCredentials()).Returns(new VssCredentials());
// Act. // Act.
@@ -153,8 +145,6 @@ namespace GitHub.Runner.Common.Tests.Listener
tokenSource.Token)) tokenSource.Token))
.Returns(Task.FromResult(expectedSession)); .Returns(Task.FromResult(expectedSession));
_capabilitiesManager.Setup(x => x.GetCapabilitiesAsync(_settings, It.IsAny<CancellationToken>())).Returns(Task.FromResult(new Dictionary<string, string>()));
_credMgr.Setup(x => x.LoadCredentials()).Returns(new VssCredentials()); _credMgr.Setup(x => x.LoadCredentials()).Returns(new VssCredentials());
// Act. // Act.

View File

@@ -1,5 +1,4 @@
using GitHub.Runner.Listener; using GitHub.Runner.Listener;
using GitHub.Runner.Common.Capabilities;
using GitHub.Runner.Listener.Configuration; using GitHub.Runner.Listener.Configuration;
using GitHub.Runner.Worker; using GitHub.Runner.Worker;
using GitHub.Runner.Worker.Handlers; using GitHub.Runner.Worker.Handlers;
@@ -44,7 +43,6 @@ namespace GitHub.Runner.Common.Tests
typeof(IHostContext), typeof(IHostContext),
typeof(ITraceManager), typeof(ITraceManager),
typeof(IThrottlingReporter), typeof(IThrottlingReporter),
typeof(ICapabilitiesProvider)
}; };
Validate( Validate(
assembly: typeof(IHostContext).GetTypeInfo().Assembly, assembly: typeof(IHostContext).GetTypeInfo().Assembly,

View File

@@ -5,6 +5,7 @@ using GitHub.DistributedTask.WebApi;
using GitHub.Runner.Worker; using GitHub.Runner.Worker;
using Moq; using Moq;
using Xunit; using Xunit;
using Pipelines = GitHub.DistributedTask.Pipelines;
namespace GitHub.Runner.Common.Tests.Worker namespace GitHub.Runner.Common.Tests.Worker
{ {
@@ -146,5 +147,159 @@ namespace GitHub.Runner.Common.Tests.Worker
Assert.True(commandManager.TryProcessCommand(_ec.Object, "##[set-env name=foo]bar")); Assert.True(commandManager.TryProcessCommand(_ec.Object, "##[set-env name=foo]bar"));
} }
} }
[Fact]
[Trait("Level", "L0")]
[Trait("Category", "Worker")]
public void EchoProcessCommand()
{
using (TestHostContext _hc = new TestHostContext(this))
{
var extensionManager = new Mock<IExtensionManager>();
var echoCommand = new EchoCommandExtension();
echoCommand.Initialize(_hc);
extensionManager.Setup(x => x.GetExtensions<IActionCommandExtension>())
.Returns(new List<IActionCommandExtension>() { echoCommand });
_hc.SetSingleton<IExtensionManager>(extensionManager.Object);
Mock<IExecutionContext> _ec = new Mock<IExecutionContext>();
_ec.Setup(x => x.Write(It.IsAny<string>(), It.IsAny<string>()))
.Returns((string tag, string line) =>
{
_hc.GetTrace().Info($"{tag} {line}");
return 1;
});
_ec.SetupAllProperties();
ActionCommandManager commandManager = new ActionCommandManager();
commandManager.Initialize(_hc);
Assert.False(_ec.Object.EchoOnActionCommand);
Assert.True(commandManager.TryProcessCommand(_ec.Object, "::echo::on"));
Assert.True(_ec.Object.EchoOnActionCommand);
Assert.True(commandManager.TryProcessCommand(_ec.Object, "::echo::off"));
Assert.False(_ec.Object.EchoOnActionCommand);
Assert.True(commandManager.TryProcessCommand(_ec.Object, "::echo::ON"));
Assert.True(_ec.Object.EchoOnActionCommand);
Assert.True(commandManager.TryProcessCommand(_ec.Object, "::echo::Off "));
Assert.False(_ec.Object.EchoOnActionCommand);
}
}
[Fact]
[Trait("Level", "L0")]
[Trait("Category", "Worker")]
public void EchoProcessCommandDebugOn()
{
using (TestHostContext _hc = new TestHostContext(this))
{
// Set up a few things
// 1. Job request message (with ACTIONS_STEP_DEBUG = true)
TaskOrchestrationPlanReference plan = new TaskOrchestrationPlanReference();
TimelineReference timeline = new TimelineReference();
JobEnvironment environment = new JobEnvironment();
environment.SystemConnection = new ServiceEndpoint();
List<TaskInstance> tasks = new List<TaskInstance>();
Guid JobId = Guid.NewGuid();
string jobName = "some job name";
var jobRequest = Pipelines.AgentJobRequestMessageUtil.Convert(new AgentJobRequestMessage(plan, timeline, JobId, jobName, jobName, environment, tasks));
jobRequest.Resources.Repositories.Add(new Pipelines.RepositoryResource()
{
Alias = Pipelines.PipelineConstants.SelfAlias,
Id = "github",
Version = "sha1"
});
jobRequest.ContextData["github"] = new Pipelines.ContextData.DictionaryContextData();
jobRequest.Variables["ACTIONS_STEP_DEBUG"] = "true";
// Some service dependencies
var jobServerQueue = new Mock<IJobServerQueue>();
jobServerQueue.Setup(x => x.QueueTimelineRecordUpdate(It.IsAny<Guid>(), It.IsAny<TimelineRecord>()));
_hc.SetSingleton(jobServerQueue.Object);
var extensionManager = new Mock<IExtensionManager>();
var echoCommand = new EchoCommandExtension();
echoCommand.Initialize(_hc);
extensionManager.Setup(x => x.GetExtensions<IActionCommandExtension>())
.Returns(new List<IActionCommandExtension>() { echoCommand });
_hc.SetSingleton<IExtensionManager>(extensionManager.Object);
var configurationStore = new Mock<IConfigurationStore>();
configurationStore.Setup(x => x.GetSettings()).Returns(new RunnerSettings());
_hc.SetSingleton(configurationStore.Object);
var pagingLogger = new Mock<IPagingLogger>();
_hc.EnqueueInstance(pagingLogger.Object);
ActionCommandManager commandManager = new ActionCommandManager();
commandManager.Initialize(_hc);
var _ec = new Runner.Worker.ExecutionContext();
_ec.Initialize(_hc);
// Initialize the job (to exercise logic that sets EchoOnActionCommand)
_ec.InitializeJob(jobRequest, System.Threading.CancellationToken.None);
_ec.Complete();
Assert.True(_ec.EchoOnActionCommand);
Assert.True(commandManager.TryProcessCommand(_ec, "::echo::off"));
Assert.False(_ec.EchoOnActionCommand);
Assert.True(commandManager.TryProcessCommand(_ec, "::echo::on"));
Assert.True(_ec.EchoOnActionCommand);
}
}
[Fact]
[Trait("Level", "L0")]
[Trait("Category", "Worker")]
public void EchoProcessCommandInvalid()
{
using (TestHostContext _hc = new TestHostContext(this))
{
var extensionManager = new Mock<IExtensionManager>();
var echoCommand = new EchoCommandExtension();
echoCommand.Initialize(_hc);
extensionManager.Setup(x => x.GetExtensions<IActionCommandExtension>())
.Returns(new List<IActionCommandExtension>() { echoCommand });
_hc.SetSingleton<IExtensionManager>(extensionManager.Object);
Mock<IExecutionContext> _ec = new Mock<IExecutionContext>();
_ec.Setup(x => x.Write(It.IsAny<string>(), It.IsAny<string>()))
.Returns((string tag, string line) =>
{
_hc.GetTrace().Info($"{tag} {line}");
return 1;
});
_ec.SetupAllProperties();
ActionCommandManager commandManager = new ActionCommandManager();
commandManager.Initialize(_hc);
// Echo commands below are considered "processed", but are invalid
// 1. Invalid echo value
Assert.True(commandManager.TryProcessCommand(_ec.Object, "::echo::invalid"));
Assert.Equal(TaskResult.Failed, _ec.Object.CommandResult);
Assert.False(_ec.Object.EchoOnActionCommand);
// 2. No value
Assert.True(commandManager.TryProcessCommand(_ec.Object, "::echo::"));
Assert.Equal(TaskResult.Failed, _ec.Object.CommandResult);
Assert.False(_ec.Object.EchoOnActionCommand);
}
}
} }
} }

View File

@@ -233,6 +233,7 @@ namespace GitHub.Runner.Common.Tests.Worker
} }
} }
/*
[Fact] [Fact]
[Trait("Level", "L0")] [Trait("Level", "L0")]
[Trait("Category", "Worker")] [Trait("Category", "Worker")]
@@ -272,6 +273,7 @@ namespace GitHub.Runner.Common.Tests.Worker
Teardown(); Teardown();
} }
} }
*/
[Fact] [Fact]
[Trait("Level", "L0")] [Trait("Level", "L0")]
@@ -306,6 +308,7 @@ namespace GitHub.Runner.Common.Tests.Worker
} }
} }
/*
#if OS_LINUX #if OS_LINUX
[Fact] [Fact]
[Trait("Level", "L0")] [Trait("Level", "L0")]
@@ -772,6 +775,7 @@ namespace GitHub.Runner.Common.Tests.Worker
Teardown(); Teardown();
} }
} }
*/
[Fact] [Fact]
[Trait("Level", "L0")] [Trait("Level", "L0")]

View File

@@ -206,8 +206,22 @@ namespace GitHub.Runner.Common.Tests.Worker
var action2 = jobContext.CreateChild(Guid.NewGuid(), "action_2", "action_2", null, null); var action2 = jobContext.CreateChild(Guid.NewGuid(), "action_2", "action_2", null, null);
action2.IntraActionState["state"] = "2"; action2.IntraActionState["state"] = "2";
action1.RegisterPostJobAction("post1", "always()", new Pipelines.ActionStep() { Name = "post1", DisplayName = "Test 1", Reference = new Pipelines.RepositoryPathReference() { Name = "actions/action" } });
action2.RegisterPostJobAction("post2", "always()", new Pipelines.ActionStep() { Name = "post2", DisplayName = "Test 2", Reference = new Pipelines.RepositoryPathReference() { Name = "actions/action" } }); var postRunner1 = hc.CreateService<IActionRunner>();
postRunner1.Action = new Pipelines.ActionStep() { Name = "post1", DisplayName = "Test 1", Reference = new Pipelines.RepositoryPathReference() { Name = "actions/action" } };
postRunner1.Stage = ActionRunStage.Post;
postRunner1.Condition = "always()";
postRunner1.DisplayName = "post1";
var postRunner2 = hc.CreateService<IActionRunner>();
postRunner2.Action = new Pipelines.ActionStep() { Name = "post2", DisplayName = "Test 2", Reference = new Pipelines.RepositoryPathReference() { Name = "actions/action" } };
postRunner2.Stage = ActionRunStage.Post;
postRunner2.Condition = "always()";
postRunner2.DisplayName = "post2";
action1.RegisterPostJobStep("post1", postRunner1);
action2.RegisterPostJobStep("post2", postRunner2);
Assert.NotNull(jobContext.JobSteps); Assert.NotNull(jobContext.JobSteps);
Assert.NotNull(jobContext.PostJobSteps); Assert.NotNull(jobContext.PostJobSteps);

View File

@@ -1,6 +1,6 @@
<Project Sdk="Microsoft.NET.Sdk"> <Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup> <PropertyGroup>
<TargetFramework>netcoreapp2.2</TargetFramework> <TargetFramework>netcoreapp3.0</TargetFramework>
<RuntimeIdentifiers>win-x64;win-x86;linux-x64;linux-arm;rhel.6-x64;osx-x64</RuntimeIdentifiers> <RuntimeIdentifiers>win-x64;win-x86;linux-x64;linux-arm;rhel.6-x64;osx-x64</RuntimeIdentifiers>
<TargetLatestRuntimePatch>true</TargetLatestRuntimePatch> <TargetLatestRuntimePatch>true</TargetLatestRuntimePatch>
<AssetTargetFallback>portable-net45+win8</AssetTargetFallback> <AssetTargetFallback>portable-net45+win8</AssetTargetFallback>

View File

@@ -16,7 +16,7 @@ LAYOUT_DIR="$SCRIPT_DIR/../_layout"
DOWNLOAD_DIR="$SCRIPT_DIR/../_downloads/netcore2x" DOWNLOAD_DIR="$SCRIPT_DIR/../_downloads/netcore2x"
PACKAGE_DIR="$SCRIPT_DIR/../_package" PACKAGE_DIR="$SCRIPT_DIR/../_package"
DOTNETSDK_ROOT="$SCRIPT_DIR/../_dotnetsdk" DOTNETSDK_ROOT="$SCRIPT_DIR/../_dotnetsdk"
DOTNETSDK_VERSION="2.2.300" DOTNETSDK_VERSION="3.0.100"
DOTNETSDK_INSTALLDIR="$DOTNETSDK_ROOT/$DOTNETSDK_VERSION" DOTNETSDK_INSTALLDIR="$DOTNETSDK_ROOT/$DOTNETSDK_VERSION"
RUNNER_VERSION=$(cat runnerversion) RUNNER_VERSION=$(cat runnerversion)

View File

@@ -1,5 +1,5 @@
{ {
"sdk": { "sdk": {
"version": "2.2.300" "version": "3.0.100"
} }
} }

View File

@@ -1 +1 @@
2.159.0 2.160.1