Compare commits

...

42 Commits

Author SHA1 Message Date
Julio Barba
b061ec410f Revert: Switch publish agent package task to direct to new pool 2019-12-02 16:54:19 -05:00
Julio Barba
2b63b9c379 Prepare 2.162.0 runner release (#204) 2019-12-02 16:16:35 -05:00
Julio Barba
d93fb70a3e Fix PrepareActions_DownloadActionFromGraph test (#205) 2019-12-02 16:03:42 -05:00
Julio Barba
4ce1bfb58a Switch publish agent package task to direct to new pool 2019-12-02 14:51:46 -05:00
Julio Barba
7a6d9dc5c8 Implement new runner service name convention (#193)
* Limit service name to 80 characters
* Add L0 tests
* New service name convention
* Make RepoOrOrgName a computed property
* Add service name sanitizing logic with L0 test
2019-11-27 14:44:29 -05:00
Julio Barba
de29a39d14 Support downloading/publishing artifacts from Pipelines endpoint (#188)
* Support downloading/publishing artifacts from Pipelines endpoint
* Remove `Path` from everywhere
* Remove unused JobId argument
* PR feedback
* More PR feedback
2019-11-25 13:30:44 -05:00
eric sciple
7d505f7f77 problem matcher default severity (#203) 2019-11-22 13:21:46 -05:00
Thomas Boop
159e4c506a Updated Release Notes to use CLI Downloads (#196)
* Updated Release Notes to use CLI Downloads

* Add Config and Run steps

* Specify Root Drive for Windows

* Remove unactionable steps from readme config
2019-11-19 16:27:36 -05:00
David Kale
45c19eb7cb 150: Support more cpu architectures (#184)
* Cross compile for win-x86, linux-arm, linux-arm64

* Build with actions instead

* Remove win-x86

* Preserve CURRENT_PLATFORM in dev.sh

* build.yaml

* Fix formatting. Remove piplines

* Use 4 space indent consistently

* x32 -> x86

* TEMP: Only test when platform === target runtime

Fix arm64 node externals url

* win-x86 externals

* Temporarily bench rhel

* Add RHEL6, skip L0 on arm for now

* Add stub for downloading new node externals when they are ready

* Remove RHEL6

* Package based on new runtime names

* Remove unused rhel from matrix includes

* Update release, add packages

* RID typo

* Cant cross test arm on x64 hosts

* New arch is a feature

Dont release x86 until we have an e2e test machine

* Fix version

* Get version from file to avoid exec error during package on x64 host for arm package

* Update Release Notes for 2.161.0 (#195)

* More cleanup

* Update release notes
2019-11-13 11:26:06 -05:00
Thomas Boop
9ba971592b V 2.160.2 Release notes and version bump (#190) 2019-11-11 15:45:37 -05:00
Thomas Boop
b27cfb18e6 2.160.1 Runner Release Notes (#171)
* 2.160.1 Runner Release Notes

* Minor verbiage updates to be consistent
2019-11-11 15:45:13 -05:00
Thomas Boop
ced4c2ca50 add-mask is leaking a secret in master if debug or ::echo::on is set (#158)
* Output after processing command to avoid leaking mask

* Remove extra noise output from echo changes

* Omit Echoing of add-mask command

* avoid echoing on debug/warning/error
2019-11-11 15:40:18 -05:00
Thomas Boop
54f21c641f Update dependency docs for OSX and Windows (#162)
* Update Dependency docs for .net 3.0 depedencies

* Update Supported Windows Versions

* Update Supported Mac OS link

* Update docs/start/envosx.md

Fix typo in OSX Version

Co-Authored-By: Lucas Costi <lucascosti@users.noreply.github.com>
2019-11-11 13:43:02 -05:00
Thomas Boop
c5cbac9796 Runner fails to run as a service on windows: Disable Delay Signing on the Runner Service (#185)
* test release

* Reverse test release changes

* Remove Unused Public Keys from Runner Service
2019-11-08 09:49:20 -05:00
eric sciple
8911283cdb fix problem matcher to treat fromPath as a file path (#183) 2019-11-07 11:26:29 -05:00
Eilon Lipton
76078b5c44 Add SLN file and make projects build in VS (#173)
* Add SLN file and make projects build in VS

- Added new ActionsRunner.sln file with all the CSPROJ's in it
- Added Directory.Build.props that gets auto-included in all CSPROJ files under it
- Made default runtime platforms for Windows and Linux (to be 64bit) so that you don't have to specify it in order to build

* Remove extra invalid parens
2019-11-06 16:57:46 -05:00
Eilon Lipton
ec9cb6c68d Fix build warnings in Test project (#178)
The build warnings were of these type (mostly reported by xUnit's Code Analyzers):
- Fixed wrong parameter order in xUnit assertions (can lead to poor error reporting in test failures)
- Unused code was removed
- Correct assertions were used (e.g. Assert.True/.Contains/.EndsWith)
- Public non-test methods on test classes were made private
2019-11-06 08:52:51 -05:00
Eilon Lipton
bcac4557a0 Fix Runner.Worker build warnings (#174)
Most of these warnings show up on only certain build OSes because of #ifdefs in the code. The fix is to suppress these warnings.
2019-11-06 08:47:56 -05:00
Eilon Lipton
19580bdaf8 Update contribute.md (#175)
Small cleanup of contribution guide
2019-11-05 17:21:50 -05:00
Eilon Lipton
96d3288553 Remove unused dreamlifter section (#177) 2019-11-05 17:21:00 -05:00
Tingluo Huang
5b6f9d3b93 Stop job container after all post actions. (#165)
* stop job container after all post actions.

* c

* c
2019-11-04 13:19:21 -05:00
eric sciple
51581ac865 root search pattern for hashfiles and allow forward slash on windows (#151) 2019-10-29 13:23:30 -04:00
eric sciple
e7dd2c6cc2 Update dotnet install script (#155) 2019-10-29 13:13:58 -04:00
Thomas Boop
08b9f6e045 Merge release 160.0 into master (#153)
* Update to Version 2.160.0 (#144)

* Revert "remove issue generation on warning/error commands (#137)" (#147)

* Revert "remove issue generation on warning/error commands (#137)"

This reverts commit 53da198867.

* Updated Release notes

* Users/thboop/port directory changes (#152)

* Clear action cache for local runner

* update release notes for actions directory cache changes
2019-10-28 15:45:27 -04:00
eric sciple
0129e8111f Clear action cache for local runner (#148) 2019-10-28 11:56:12 -04:00
eric sciple
ccca13ac07 restrict hashFiles to basic globbing and globstar (#150) 2019-10-28 11:52:22 -04:00
Julio Barba
82e9857d4f Implement new echo behavior and command (#139)
* Remove controlling echoing by command

* Add 'echo on' and 'echo off' action commands

* PR feedback and add L0 tests

* Register new command

* Eric's PR feedback

* Tweak logging a bit

* Rename EchoOnActionCommandSuccess -> EchoOnActionCommand

* More PR reaction

* Make warning messages in Action Commands not rely on context from echo commands
2019-10-25 10:38:56 -04:00
Tingluo Huang
afd233b735 consume dotnet core 3.0 (#127)
* consume dotnet core 3.0

* update linux dependency doc.
2019-10-24 16:52:29 -04:00
Thomas Boop
83be145bfd Update build.yml 2019-10-24 14:05:10 -04:00
Thomas Boop
6e20aceaff Update build.yml 2019-10-24 14:00:56 -04:00
Lucas Killgore
e89148e33e Don't retry uploads when the http status code response from the server is in the 400's (#131)
* Don't retry uploads when the http status code
response from the server is in the 400's

* Don't retry on fast-fail

* Feedback from code review

* Always try to attach any uploaded files to the build
Don't fast-fail on 'Conflict'

* Add dispose

* Refactored upload code.
Called out specialized 'Conflict' logic.

* Added typed exception.
2019-10-23 12:48:10 -04:00
Thomas Boop
f1c58c33b6 Default to pwsh with powershell as a fallback (#142)
* Default to pwsh with powershell as a fallback

* Update releaseNote.md

* clean up code
2019-10-23 11:24:02 -04:00
Thomas Boop
7b158fff05 Revert "Need to update the release pool (#138)" (#141)
This reverts commit 3a43d853be.
2019-10-22 13:41:55 -04:00
Thomas Boop
6225b22870 Update to Version 2.159.2 (#140) 2019-10-22 13:14:58 -04:00
Thomas Boop
53da198867 remove issue generation on warning/error commands (#137) 2019-10-21 16:19:57 -04:00
Thomas Boop
3a43d853be Need to update the release pool (#138) 2019-10-21 15:32:57 -04:00
Thomas Boop
9d7b0a2405 set default shell to powershell in windows (#135)
* set default shell to powershell in windows

* Update error checking

* Update error message
2019-10-21 14:08:12 -04:00
Thomas Boop
05f0b938ac update version to 2.159.1 (#133)
* update version to 2.159.1

* Update releaseNote.md
2019-10-17 17:05:16 -04:00
Tingluo Huang
f5f14d4811 Runner register labels during configuration (#130)
* Runners will add os and architecture labels during registration

* support github.localhost for dev.
2019-10-17 16:33:43 -04:00
Tingluo Huang
2f261f2c31 Update build.yml 2019-10-14 16:14:22 -04:00
Thomas Boop
c3b11b36e9 Remove unneeded base64 encoders (#128)
* Remove unneeded base64 encoders

* update comment
2019-10-14 10:47:41 -04:00
Tingluo Huang
8777686c11 L0 for HostContext default masker. (#129) 2019-10-14 10:47:20 -04:00
112 changed files with 3098 additions and 1473 deletions

View File

@@ -4,6 +4,7 @@ on:
push:
branches:
- master
- releases/*
pull_request:
branches:
- '*'
@@ -12,14 +13,28 @@ jobs:
build:
strategy:
matrix:
os: [ubuntu-latest, windows-latest, macOS-latest]
runtime: [ linux-x64, linux-arm64, linux-arm, win-x64, osx-x64 ]
include:
- os: ubuntu-latest
- runtime: linux-x64
os: ubuntu-latest
devScript: ./dev.sh
- os: macOS-latest
- runtime: linux-arm64
os: ubuntu-latest
devScript: ./dev.sh
- os: windows-latest
devScript: dev.cmd
- runtime: linux-arm
os: ubuntu-latest
devScript: ./dev.sh
- runtime: osx-x64
os: macOS-latest
devScript: ./dev.sh
- runtime: win-x64
os: windows-latest
devScript: ./dev
runs-on: ${{ matrix.os }}
steps:
- uses: actions/checkout@v1
@@ -27,7 +42,7 @@ jobs:
# Build runner layout
- name: Build & Layout Release
run: |
${{ matrix.devScript }} layout Release
${{ matrix.devScript }} layout Release ${{ matrix.runtime }}
working-directory: src
# Run tests
@@ -35,6 +50,7 @@ jobs:
run: |
${{ matrix.devScript }} test
working-directory: src
if: matrix.runtime != 'linux-arm64' && matrix.runtime != 'linux-arm'
# Create runner package tar.gz/zip
- name: Package Release
@@ -48,5 +64,5 @@ jobs:
if: github.event_name != 'pull_request'
uses: actions/upload-artifact@v1
with:
name: runner-package-${{ matrix.os }}
name: runner-package-${{ matrix.runtime }}
path: _package

1
.gitignore vendored
View File

@@ -3,7 +3,6 @@
**/libs
**/*.xproj
**/*.xproj.user
**/*.sln
**/.vs
**/.vscode
**/*.error

73
ActionsRunner.sln Normal file
View File

@@ -0,0 +1,73 @@

Microsoft Visual Studio Solution File, Format Version 12.00
# Visual Studio Version 16
VisualStudioVersion = 16.0.29411.138
MinimumVisualStudioVersion = 10.0.40219.1
Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "Runner.Common", "src\Runner.Common\Runner.Common.csproj", "{084289A3-CD7A-42E0-9219-4348B4B7E19B}"
EndProject
Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "Runner.Listener", "src\Runner.Listener\Runner.Listener.csproj", "{7D461AEE-BF2A-4855-BD96-56921160B36A}"
EndProject
Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "Runner.PluginHost", "src\Runner.PluginHost\Runner.PluginHost.csproj", "{D0320EB1-CB6D-4179-BFDC-2F2B664A370C}"
EndProject
Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "Runner.Plugins", "src\Runner.Plugins\Runner.Plugins.csproj", "{C23AFD6F-4DCD-4243-BC61-865BE31B9168}"
EndProject
Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "Runner.Sdk", "src\Runner.Sdk\Runner.Sdk.csproj", "{D0484633-DA97-4C34-8E47-1DADE212A57A}"
EndProject
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "RunnerService", "src\Runner.Service\Windows\RunnerService.csproj", "{D12EBD71-0464-46D0-8394-40BCFBA0A6F2}"
EndProject
Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "Runner.Worker", "src\Runner.Worker\Runner.Worker.csproj", "{C2F5B9FA-2621-411F-8EB2-273ED276F503}"
EndProject
Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "Sdk", "src\Sdk\Sdk.csproj", "{D2EE812B-E4DF-49BB-AE87-12BC49949B5F}"
EndProject
Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "Test", "src\Test\Test.csproj", "{C932061F-F6A1-4F1E-B854-A6C6B30DC3EF}"
EndProject
Global
GlobalSection(SolutionConfigurationPlatforms) = preSolution
Debug|Any CPU = Debug|Any CPU
Release|Any CPU = Release|Any CPU
EndGlobalSection
GlobalSection(ProjectConfigurationPlatforms) = postSolution
{084289A3-CD7A-42E0-9219-4348B4B7E19B}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{084289A3-CD7A-42E0-9219-4348B4B7E19B}.Debug|Any CPU.Build.0 = Debug|Any CPU
{084289A3-CD7A-42E0-9219-4348B4B7E19B}.Release|Any CPU.ActiveCfg = Release|Any CPU
{084289A3-CD7A-42E0-9219-4348B4B7E19B}.Release|Any CPU.Build.0 = Release|Any CPU
{7D461AEE-BF2A-4855-BD96-56921160B36A}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{7D461AEE-BF2A-4855-BD96-56921160B36A}.Debug|Any CPU.Build.0 = Debug|Any CPU
{7D461AEE-BF2A-4855-BD96-56921160B36A}.Release|Any CPU.ActiveCfg = Release|Any CPU
{7D461AEE-BF2A-4855-BD96-56921160B36A}.Release|Any CPU.Build.0 = Release|Any CPU
{D0320EB1-CB6D-4179-BFDC-2F2B664A370C}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{D0320EB1-CB6D-4179-BFDC-2F2B664A370C}.Debug|Any CPU.Build.0 = Debug|Any CPU
{D0320EB1-CB6D-4179-BFDC-2F2B664A370C}.Release|Any CPU.ActiveCfg = Release|Any CPU
{D0320EB1-CB6D-4179-BFDC-2F2B664A370C}.Release|Any CPU.Build.0 = Release|Any CPU
{C23AFD6F-4DCD-4243-BC61-865BE31B9168}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{C23AFD6F-4DCD-4243-BC61-865BE31B9168}.Debug|Any CPU.Build.0 = Debug|Any CPU
{C23AFD6F-4DCD-4243-BC61-865BE31B9168}.Release|Any CPU.ActiveCfg = Release|Any CPU
{C23AFD6F-4DCD-4243-BC61-865BE31B9168}.Release|Any CPU.Build.0 = Release|Any CPU
{D0484633-DA97-4C34-8E47-1DADE212A57A}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{D0484633-DA97-4C34-8E47-1DADE212A57A}.Debug|Any CPU.Build.0 = Debug|Any CPU
{D0484633-DA97-4C34-8E47-1DADE212A57A}.Release|Any CPU.ActiveCfg = Release|Any CPU
{D0484633-DA97-4C34-8E47-1DADE212A57A}.Release|Any CPU.Build.0 = Release|Any CPU
{D12EBD71-0464-46D0-8394-40BCFBA0A6F2}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{D12EBD71-0464-46D0-8394-40BCFBA0A6F2}.Debug|Any CPU.Build.0 = Debug|Any CPU
{D12EBD71-0464-46D0-8394-40BCFBA0A6F2}.Release|Any CPU.ActiveCfg = Release|Any CPU
{D12EBD71-0464-46D0-8394-40BCFBA0A6F2}.Release|Any CPU.Build.0 = Release|Any CPU
{C2F5B9FA-2621-411F-8EB2-273ED276F503}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{C2F5B9FA-2621-411F-8EB2-273ED276F503}.Debug|Any CPU.Build.0 = Debug|Any CPU
{C2F5B9FA-2621-411F-8EB2-273ED276F503}.Release|Any CPU.ActiveCfg = Release|Any CPU
{C2F5B9FA-2621-411F-8EB2-273ED276F503}.Release|Any CPU.Build.0 = Release|Any CPU
{D2EE812B-E4DF-49BB-AE87-12BC49949B5F}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{D2EE812B-E4DF-49BB-AE87-12BC49949B5F}.Debug|Any CPU.Build.0 = Debug|Any CPU
{D2EE812B-E4DF-49BB-AE87-12BC49949B5F}.Release|Any CPU.ActiveCfg = Release|Any CPU
{D2EE812B-E4DF-49BB-AE87-12BC49949B5F}.Release|Any CPU.Build.0 = Release|Any CPU
{C932061F-F6A1-4F1E-B854-A6C6B30DC3EF}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{C932061F-F6A1-4F1E-B854-A6C6B30DC3EF}.Debug|Any CPU.Build.0 = Debug|Any CPU
{C932061F-F6A1-4F1E-B854-A6C6B30DC3EF}.Release|Any CPU.ActiveCfg = Release|Any CPU
{C932061F-F6A1-4F1E-B854-A6C6B30DC3EF}.Release|Any CPU.Build.0 = Release|Any CPU
EndGlobalSection
GlobalSection(SolutionProperties) = preSolution
HideSolutionNode = FALSE
EndGlobalSection
GlobalSection(ExtensibilityGlobals) = postSolution
SolutionGuid = {4A831DDA-3860-45E5-930E-BB3A7833AE80}
EndGlobalSection
EndGlobal

View File

@@ -16,5 +16,17 @@
"platform": "linux-x64",
"version": "<RUNNER_VERSION>",
"downloadUrl": "https://githubassets.azureedge.net/runners/<RUNNER_VERSION>/actions-runner-linux-x64-<RUNNER_VERSION>.tar.gz"
},
{
"name": "actions-runner-linux-arm64-<RUNNER_VERSION>.tar.gz",
"platform": "linux-arm64",
"version": "<RUNNER_VERSION>",
"downloadUrl": "https://githubassets.azureedge.net/runners/<RUNNER_VERSION>/actions-runner-linux-arm64-<RUNNER_VERSION>.tar.gz"
},
{
"name": "actions-runner-linux-arm-<RUNNER_VERSION>.tar.gz",
"platform": "linux-arm",
"version": "<RUNNER_VERSION>",
"downloadUrl": "https://githubassets.azureedge.net/runners/<RUNNER_VERSION>/actions-runner-linux-arm-<RUNNER_VERSION>.tar.gz"
}
]

View File

@@ -11,6 +11,8 @@ stages:
# Steps template for windows platform
- template: windows.template.yml
parameters:
targetRuntime: win-x64
# Package dotnet core windows dependency (VC++ Redistributable)
- powershell: |
@@ -28,13 +30,13 @@ stages:
displayName: Package UCRT
# Create agent package zip
- script: dev.cmd package Release
- script: dev.cmd package Release win-x64
workingDirectory: src
displayName: Package Release
# Upload agent package zip as build artifact
- task: PublishBuildArtifacts@1
displayName: Publish Artifact (Windows)
displayName: Publish Artifact (Windows x64)
inputs:
pathToPublish: _package
artifactName: runners
@@ -50,22 +52,76 @@ stages:
# Steps template for non-windows platform
- template: nonwindows.template.yml
parameters:
targetRuntime: linux-x64
# Create agent package zip
- script: ./dev.sh package Release
- script: ./dev.sh package Release linux-x64
workingDirectory: src
displayName: Package Release
# Upload agent package zip as build artifact
- task: PublishBuildArtifacts@1
displayName: Publish Artifact (Linux)
displayName: Publish Artifact (Linux x64)
inputs:
pathToPublish: _package
artifactName: runners
artifactType: container
################################################################################
- job: build_osx_agent
- job: build_linux_agent_arm64
################################################################################
displayName: Linux Agent (arm64)
pool:
vmImage: ubuntu-16.04
steps:
# Steps template for non-windows platform
- template: nonwindows.template.yml
parameters:
targetRuntime: linux-arm64
# Create agent package zip
- script: ./dev.sh package Release linux-arm64
workingDirectory: src
displayName: Package Release
# Upload agent package zip as build artifact
- task: PublishBuildArtifacts@1
displayName: Publish Artifact (Linux ARM64)
inputs:
pathToPublish: _package
artifactName: runners
artifactType: container
################################################################################
- job: build_linux_agent_arm
################################################################################
displayName: Linux Agent (arm)
pool:
vmImage: ubuntu-16.04
steps:
# Steps template for non-windows platform
- template: nonwindows.template.yml
parameters:
targetRuntime: linux-arm
# Create agent package zip
- script: ./dev.sh package Release linux-arm
workingDirectory: src
displayName: Package Release
# Upload agent package zip as build artifact
- task: PublishBuildArtifacts@1
displayName: Publish Artifact (Linux ARM)
inputs:
pathToPublish: _package
artifactName: runners
artifactType: container
################################################################################
- job: build_osx_agent_x64
################################################################################
displayName: macOS Agent (x64)
pool:
@@ -74,15 +130,17 @@ stages:
# Steps template for non-windows platform
- template: nonwindows.template.yml
parameters:
targetRuntime: osx-x64
# Create agent package zip
- script: ./dev.sh package Release
- script: ./dev.sh package Release osx-x64
workingDirectory: src
displayName: Package Release
# Upload agent package zip as build artifact
- task: PublishBuildArtifacts@1
displayName: Publish Artifact (OSX)
displayName: Publish Artifact (OSX x64)
inputs:
pathToPublish: _package
artifactName: runners

View File

@@ -1,12 +1,12 @@
# Contribute (Dev)
# Contribution guide for developers
## Dev Dependencies
## Required Dev Dependencies
![Win](res/win_sm.png) Git for Windows [Install Here](https://git-scm.com/downloads) (needed for dev sh script)
## Build, Test, Layout
## To Build, Test, Layout
From src:
Navigate to the `src` directory and run the following command:
![Win](res/win_sm.png) `dev {command}`
@@ -14,13 +14,12 @@ From src:
**Commands:**
`layout` (`l`): Run first time to create a full agent layout in {root}/_layout
* `layout` (`l`): Run first time to create a full agent layout in `{root}/_layout`
* `build` (`b`): Build everything and update agent layout folder
* `test` (`t`): Build agent binaries and run unit tests
`build` (`b`): build everything and update agent layout folder
Sample developer flow:
`test` (`t`): build agent binaries and run unit tests
Normal dev flow:
```bash
git clone https://github.com/actions/runner
cd ./src
@@ -37,5 +36,5 @@ cd ./src
## Styling
We use the dotnet foundation and CoreCLR style guidelines [located here](
We use the .NET Foundation and CoreCLR style guidelines [located here](
https://github.com/dotnet/corefx/blob/master/Documentation/coding-guidelines/coding-style.md)

View File

@@ -5,36 +5,55 @@
## Supported Distributions and Versions
x64
- Red Hat Enterprise Linux 6 (see note 1), 7
- CentOS 6 (see note 1), 7
- Red Hat Enterprise Linux 7
- CentOS 7
- Oracle Linux 7
- Fedora 28, 27
- Debian 9, 8.7 or later versions
- Ubuntu 18.04, Ubuntu 16.04, Ubuntu 14.04
- Linux Mint 18, 17
- openSUSE 42.3 or later versions
- SUSE Enterprise Linux (SLES) 12 SP2 or later versions
- Fedora 29+
- Debian 9+
- Ubuntu 16.04+
- Linux Mint 18+
- openSUSE 15+
- SUSE Enterprise Linux (SLES) 12 SP2+
ARM32 (see note 2)
- Debian 9 or later versions
- Ubuntu 18.04 or later versions
## Install .Net Core 3.x Linux Dependencies
> Note 1: Red Hat Enterprise Linux 6 and CentOS 6 require installing the specialized "rhel.6-x64" agent package
> Note 2: ARM instruction set [ARMv7](https://en.wikipedia.org/wiki/List_of_ARM_microarchitectures) or above is required, you can get your device's information by executing `uname -a`
## Install .Net Core 2.x Linux Dependencies
The `./config.sh` will check .Net Core 2.x dependencies during agent configuration.
The `./config.sh` will check .Net Core 3.x dependencies during runner configuration.
You might see something like this which indicate a dependency's missing.
```bash
./config.sh
libunwind.so.8 => not found
libunwind-x86_64.so.8 => not found
Dependencies is missing for Dotnet Core 2.1
Execute ./bin/installdependencies.sh to install any missing Dotnet Core 2.1 dependencies.
Dependencies is missing for Dotnet Core 3.0
Execute ./bin/installdependencies.sh to install any missing Dotnet Core 3.0 dependencies.
```
You can easily correct the problem by executing `./bin/installdependencies.sh`.
The `installdependencies.sh` script should install all required dependencies on all supported Linux versions
> Note: The `installdependencies.sh` script will try to use the default package management mechanism on your Linux flavor (ex. `yum`/`apt-get`/`apt`). You might need to deal with error coming from the package management mechanism related to your setup, like [#1353](https://github.com/Microsoft/vsts-agent/issues/1353)
### Full dependencies list
Debian based OS (Debian, Ubuntu, Linux Mint)
- liblttng-ust0
- libkrb5-3
- zlib1g
- libssl1.1, libssl1.0.2 or libssl1.0.0
- libicu63, libicu60, libicu57 or libicu55
Fedora based OS (Fedora, Redhat, Centos, Oracle Linux 7)
- lttng-ust
- openssl-libs
- krb5-libs
- zlib
- libicu
SUSE based OS (OpenSUSE, SUSE Enterprise)
- lttng-ust
- libopenssl1_1
- krb5
- zlib
- libicu60_2
## [More .Net Core Prerequisites Information](https://docs.microsoft.com/en-us/dotnet/core/linux-prerequisites?tabs=netcore2x)

View File

@@ -4,7 +4,7 @@
## Supported Versions
- macOS Sierra (10.12) and later versions
- macOS High Sierra (10.13) and later versions
## [More .Net Core Prerequisites Information](https://docs.microsoft.com/en-us/dotnet/core/macos-prerequisites?tabs=netcore2x)
## [More .Net Core Prerequisites Information](https://docs.microsoft.com/en-us/dotnet/core/macos-prerequisites?tabs=netcore30)

View File

@@ -5,8 +5,8 @@
- Windows 7 64-bit
- Windows 8.1 64-bit
- Windows 10 64-bit
- Windows Server 2008 R2 SP1 64-bit
- Windows Server 2012 R2 64-bit
- Windows Server 2016 64-bit
- Windows Server 2019 64-bit
## [More .Net Core Prerequisites Information](https://docs.microsoft.com/en-us/dotnet/core/windows-prerequisites?tabs=netcore2x)
## [More .Net Core Prerequisites Information](https://docs.microsoft.com/en-us/dotnet/core/windows-prerequisites?tabs=netcore30)

View File

@@ -1,14 +1,18 @@
parameters:
targetRuntime: ''
steps:
# Build agent layout
- script: ./dev.sh layout Release
- script: ./dev.sh layout Release ${{ parameters.targetRuntime }}
workingDirectory: src
displayName: Build & Layout Release
displayName: Build & Layout Release ${{ parameters.targetRuntime }}
# Run test
- script: ./dev.sh test
workingDirectory: src
displayName: Test
condition: and(ne('${{ parameters.targetRuntime }}', 'linux-arm64'), ne('${{ parameters.targetRuntime }}', 'linux-arm'))
# # Publish test results
# - task: PublishTestResults@2

View File

@@ -1,44 +1,69 @@
## Features
- Runner config auth via GitHub.com. (#107) (#117)
- Adding wrapper action to support post job cleanup, adding checkout v1.1 (#91)
- Improving terminal experience (#110)
- Add runner support for cache action. (#120)
- Added the "severity" keyword to allow action authors to set the default severity for problem matchers (#203)
## Bugs
- Set GITHUB_ACTIONS in containers. (#119)
- Fix issue data column/col mismatch. (#122)
- Fixed generated self-hosted runner names to never go over 80 characters (helps Windows customers) (#193)
- Fixed `PrepareActions_DownloadActionFromGraph` test by pointing to an active Actions repository (#205)
## Misc
- Use GitHub actions for CI/PR (#112)
- Code Cleanup (#123) (#124) (#125)
## Agent Downloads
| | Package |
| ------- | ----------------------------------------------------------------------------------------------------------- |
| Windows x64 | [actions-runner-win-x64-<RUNNER_VERSION>.zip](https://githubassets.azureedge.net/runners/<RUNNER_VERSION>/actions-runner-win-x64-<RUNNER_VERSION>.zip) |
| macOS | [actions-runner-osx-x64-<RUNNER_VERSION>.tar.gz](https://githubassets.azureedge.net/runners/<RUNNER_VERSION>/actions-runner-osx-x64-<RUNNER_VERSION>.tar.gz) |
| Linux x64 | [actions-runner-linux-x64-<RUNNER_VERSION>.tar.gz](https://githubassets.azureedge.net/runners/<RUNNER_VERSION>/actions-runner-linux-x64-<RUNNER_VERSION>.tar.gz) |
After Download:
- Updated the publish and download artifact actions to use the v2 endpoint (#188)
- Updated the service name on self-hosted runner name to include repository or organization information (#193)
## Windows x64
``` bash
C:\> mkdir myagent && cd myagent
C:\myagent> Add-Type -AssemblyName System.IO.Compression.FileSystem ; [System.IO.Compression.ZipFile]::ExtractToDirectory("$HOME\Downloads\actions-runner-win-x64-<RUNNER_VERSION>.zip", "$PWD")
We recommend configuring the runner under "<DRIVE>:\actions-runner". This will help avoid issues related to service identity folder permissions and long file path restrictions on Windows
```
// Create a folder under the drive root
mkdir \actions-runner ; cd \actions-runner
// Download the latest runner package
Invoke-WebRequest -Uri https://githubassets.azureedge.net/runners/<RUNNER_VERSION>/actions-runner-win-x64-<RUNNER_VERSION>.zip -OutFile actions-runner-win-x64-<RUNNER_VERSION>.zip
// Extract the installer
Add-Type -AssemblyName System.IO.Compression.FileSystem ;
[System.IO.Compression.ZipFile]::ExtractToDirectory("$HOME\Downloads\actions-runner-win-x64-<RUNNER_VERSION>.zip", "$PWD")
```
## OSX
``` bash
~/$ mkdir myagent && cd myagent
~/myagent$ tar xzf ~/Downloads/actions-runner-osx-x64-<RUNNER_VERSION>.tar.gz
// Create a folder
mkdir actions-runner && cd actions-runner
// Download the latest runner package
curl -O https://githubassets.azureedge.net/runners/<RUNNER_VERSION>/actions-runner-osx-x64-<RUNNER_VERSION>.tar.gz
// Extract the installer
tar xzf ./actions-runner-osx-x64-<RUNNER_VERSION>.tar.gz
```
## Linux x64
``` bash
~/$ mkdir myagent && cd myagent
~/myagent$ tar xzf ~/Downloads/actions-runner-linux-x64-<RUNNER_VERSION>.tar.gz
// Create a folder
mkdir actions-runner && cd actions-runner
// Download the latest runner package
curl -O https://githubassets.azureedge.net/runners/<RUNNER_VERSION>/actions-runner-linux-x64-<RUNNER_VERSION>.tar.gz
// Extract the installer
tar xzf ./actions-runner-linux-x64-<RUNNER_VERSION>.tar.gz
```
## Linux arm64 (Pre-release)
``` bash
// Create a folder
mkdir actions-runner && cd actions-runner
// Download the latest runner package
curl -O https://githubassets.azureedge.net/runners/<RUNNER_VERSION>/actions-runner-linux-arm64-<RUNNER_VERSION>.tar.gz
// Extract the installer
tar xzf ./actions-runner-linux-arm64-<RUNNER_VERSION>.tar.gz
```
## Linux arm (Pre-release)
``` bash
// Create a folder
mkdir actions-runner && cd actions-runner
// Download the latest runner package
curl -O https://githubassets.azureedge.net/runners/<RUNNER_VERSION>/actions-runner-linux-arm-<RUNNER_VERSION>.tar.gz
// Extract the installer
tar xzf ./actions-runner-linux-arm-<RUNNER_VERSION>.tar.gz
```
## Using your self hosted runner
For additional details about configuring, running, or shutting down the runner please check out our [product docs.](https://help.github.com/en/actions/automating-your-workflow-with-github-actions/adding-self-hosted-runners)

49
src/Directory.Build.props Normal file
View File

@@ -0,0 +1,49 @@
<Project ToolsVersion="15.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<!-- Detect OS for build -->
<PropertyGroup>
<BUILD_OS Condition="'$([System.Runtime.InteropServices.RuntimeInformation]::IsOSPlatform($([System.Runtime.InteropServices.OSPlatform]::Windows)))' == 'true'">Windows</BUILD_OS>
<BUILD_OS Condition="'$([System.Runtime.InteropServices.RuntimeInformation]::IsOSPlatform($([System.Runtime.InteropServices.OSPlatform]::OSX)))' == 'true'">OSX</BUILD_OS>
<BUILD_OS Condition="'$([System.Runtime.InteropServices.RuntimeInformation]::IsOSPlatform($([System.Runtime.InteropServices.OSPlatform]::Linux)))' == 'true'">Linux</BUILD_OS>
</PropertyGroup>
<!-- Set OS vars -->
<PropertyGroup Condition="'$(BUILD_OS)' == 'Windows'">
<DefineConstants>$(DefineConstants);OS_WINDOWS</DefineConstants>
</PropertyGroup>
<PropertyGroup Condition="'$(BUILD_OS)' == 'OSX'">
<DefineConstants>$(DefineConstants);OS_OSX</DefineConstants>
</PropertyGroup>
<PropertyGroup Condition="'$(BUILD_OS)' == 'Linux'">
<DefineConstants>$(DefineConstants);OS_LINUX</DefineConstants>
</PropertyGroup>
<!-- Set Platform/bitness vars -->
<PropertyGroup Condition="'$(BUILD_OS)' == 'Windows' AND ('$(PackageRuntime)' == 'win-x64' OR '$(PackageRuntime)' == '')">
<DefineConstants>$(DefineConstants);X64</DefineConstants>
</PropertyGroup>
<PropertyGroup Condition="'$(BUILD_OS)' == 'Windows' AND '$(PackageRuntime)' == 'win-x86'">
<DefineConstants>$(DefineConstants);X86</DefineConstants>
</PropertyGroup>
<PropertyGroup Condition="'$(BUILD_OS)' == 'OSX'">
<DefineConstants>$(DefineConstants);X64</DefineConstants>
</PropertyGroup>
<PropertyGroup Condition="'$(BUILD_OS)' == 'Linux' AND ('$(PackageRuntime)' == 'linux-x64' OR '$(PackageRuntime)' == '')">
<DefineConstants>$(DefineConstants);X64</DefineConstants>
</PropertyGroup>
<PropertyGroup Condition="'$(BUILD_OS)' == 'Linux' AND '$(PackageRuntime)' == 'linux-arm'">
<DefineConstants>$(DefineConstants);ARM</DefineConstants>
</PropertyGroup>
<PropertyGroup Condition="'$(BUILD_OS)' == 'Linux' AND '$(PackageRuntime)' == 'linux-arm64'">
<DefineConstants>$(DefineConstants);ARM64</DefineConstants>
</PropertyGroup>
<!-- Set TRACE/DEBUG vars -->
<PropertyGroup>
<DefineConstants>$(DefineConstants);TRACE</DefineConstants>
</PropertyGroup>
<PropertyGroup Condition="'$(Configuration)' == 'Debug'">
<DefineConstants>$(DefineConstants);DEBUG</DefineConstants>
</PropertyGroup>
</Project>

View File

@@ -37,10 +37,7 @@
.PARAMETER SharedRuntime
This parameter is obsolete and may be removed in a future version of this script.
The recommended alternative is '-Runtime dotnet'.
Default: false
Installs just the shared runtime bits, not the entire SDK.
This is equivalent to specifying `-Runtime dotnet`.
.PARAMETER Runtime
Installs just a shared runtime, not the entire SDK.
Possible values:
@@ -77,11 +74,15 @@
Skips installing non-versioned files if they already exist, such as dotnet.exe.
.PARAMETER NoCdn
Disable downloading from the Azure CDN, and use the uncached feed directly.
.PARAMETER JSonFile
Determines the SDK version from a user specified global.json file
Note: global.json must have a value for 'SDK:Version'
#>
[cmdletbinding()]
param(
[string]$Channel="LTS",
[string]$Version="Latest",
[string]$JSonFile,
[string]$InstallDir="<auto>",
[string]$Architecture="<auto>",
[ValidateSet("dotnet", "aspnetcore", "windowsdesktop", IgnoreCase = $false)]
@@ -258,7 +259,6 @@ function GetHTTPResponse([Uri] $Uri)
})
}
function Get-Latest-Version-Info([string]$AzureFeed, [string]$Channel, [bool]$Coherent) {
Say-Invocation $MyInvocation
@@ -304,10 +304,50 @@ function Get-Latest-Version-Info([string]$AzureFeed, [string]$Channel, [bool]$Co
return $VersionInfo
}
function Get-Specific-Version-From-Version([string]$AzureFeed, [string]$Channel, [string]$Version) {
function Parse-Jsonfile-For-Version([string]$JSonFile) {
Say-Invocation $MyInvocation
If (-Not (Test-Path $JSonFile)) {
throw "Unable to find '$JSonFile'"
exit 0
}
try {
$JSonContent = Get-Content($JSonFile) -Raw | ConvertFrom-Json | Select-Object -expand "sdk" -ErrorAction SilentlyContinue
}
catch {
throw "Json file unreadable: '$JSonFile'"
exit 0
}
if ($JSonContent) {
try {
$JSonContent.PSObject.Properties | ForEach-Object {
$PropertyName = $_.Name
if ($PropertyName -eq "version") {
$Version = $_.Value
Say-Verbose "Version = $Version"
}
}
}
catch {
throw "Unable to parse the SDK node in '$JSonFile'"
exit 0
}
}
else {
throw "Unable to find the SDK node in '$JSonFile'"
exit 0
}
If ($Version -eq $null) {
throw "Unable to find the SDK:version node in '$JSonFile'"
exit 0
}
return $Version
}
function Get-Specific-Version-From-Version([string]$AzureFeed, [string]$Channel, [string]$Version, [string]$JSonFile) {
Say-Invocation $MyInvocation
if (-not $JSonFile) {
switch ($Version.ToLower()) {
{ $_ -eq "latest" } {
$LatestVersionInfo = Get-Latest-Version-Info -AzureFeed $AzureFeed -Channel $Channel -Coherent $False
@@ -319,6 +359,10 @@ function Get-Specific-Version-From-Version([string]$AzureFeed, [string]$Channel,
}
default { return $Version }
}
}
else {
return Parse-Jsonfile-For-Version $JSonFile
}
}
function Get-Download-Link([string]$AzureFeed, [string]$SpecificVersion, [string]$CLIArchitecture) {
@@ -382,23 +426,6 @@ function Resolve-Installation-Path([string]$InstallDir) {
return $InstallDir
}
function Get-Version-Info-From-Version-File([string]$InstallRoot, [string]$RelativePathToVersionFile) {
Say-Invocation $MyInvocation
$VersionFile = Join-Path -Path $InstallRoot -ChildPath $RelativePathToVersionFile
Say-Verbose "Local version file: $VersionFile"
if (Test-Path $VersionFile) {
$VersionText = cat $VersionFile
Say-Verbose "Local version file text: $VersionText"
return Get-Version-Info-From-Version-Text $VersionText
}
Say-Verbose "Local version file not found."
return $null
}
function Is-Dotnet-Package-Installed([string]$InstallRoot, [string]$RelativePathToPackage, [string]$SpecificVersion) {
Say-Invocation $MyInvocation
@@ -534,7 +561,7 @@ function Prepend-Sdk-InstallRoot-To-Path([string]$InstallRoot, [string]$BinFolde
}
$CLIArchitecture = Get-CLIArchitecture-From-Architecture $Architecture
$SpecificVersion = Get-Specific-Version-From-Version -AzureFeed $AzureFeed -Channel $Channel -Version $Version
$SpecificVersion = Get-Specific-Version-From-Version -AzureFeed $AzureFeed -Channel $Channel -Version $Version -JSonFile $JSonFile
$DownloadLink = Get-Download-Link -AzureFeed $AzureFeed -SpecificVersion $SpecificVersion -CLIArchitecture $CLIArchitecture
$LegacyDownloadLink = Get-LegacyDownload-Link -AzureFeed $AzureFeed -SpecificVersion $SpecificVersion -CLIArchitecture $CLIArchitecture

View File

@@ -435,11 +435,52 @@ get_latest_version_info() {
return $?
}
# args:
# json_file - $1
parse_jsonfile_for_version() {
eval $invocation
local json_file="$1"
if [ ! -f "$json_file" ]; then
say_err "Unable to find \`$json_file\`"
return 1
fi
sdk_section=$(cat $json_file | awk '/"sdk"/,/}/')
if [ -z "$sdk_section" ]; then
say_err "Unable to parse the SDK node in \`$json_file\`"
return 1
fi
sdk_list=$(echo $sdk_section | awk -F"[{}]" '{print $2}')
sdk_list=${sdk_list//[\" ]/}
sdk_list=${sdk_list//,/$'\n'}
sdk_list="$(echo -e "${sdk_list}" | tr -d '[[:space:]]')"
local version_info=""
while read -r line; do
IFS=:
while read -r key value; do
if [[ "$key" == "version" ]]; then
version_info=$value
fi
done <<< "$line"
done <<< "$sdk_list"
if [ -z "$version_info" ]; then
say_err "Unable to find the SDK:version node in \`$json_file\`"
return 1
fi
echo "$version_info"
return 0
}
# args:
# azure_feed - $1
# channel - $2
# normalized_architecture - $3
# version - $4
# json_file - $5
get_specific_version_from_version() {
eval $invocation
@@ -447,7 +488,9 @@ get_specific_version_from_version() {
local channel="$2"
local normalized_architecture="$3"
local version="$(to_lowercase "$4")"
local json_file="$5"
if [ -z "$json_file" ]; then
case "$version" in
latest)
local version_info
@@ -468,6 +511,12 @@ get_specific_version_from_version() {
return 0
;;
esac
else
local version_info
version_info="$(parse_jsonfile_for_version "$json_file")" || return 1
echo "$version_info"
return 0
fi
}
# args:
@@ -558,24 +607,6 @@ resolve_installation_path() {
return 0
}
# args:
# install_root - $1
get_installed_version_info() {
eval $invocation
local install_root="$1"
local version_file="$(combine_paths "$install_root" "$local_version_file_relative_path")"
say_verbose "Local version file: $version_file"
if [ ! -z "$version_file" ] | [ -r "$version_file" ]; then
local version_info="$(cat "$version_file")"
echo "$version_info"
return 0
fi
say_verbose "Local version file not found."
return 0
}
# args:
# relative_or_absolute_path - $1
get_absolute_path() {
@@ -724,7 +755,7 @@ calculate_vars() {
normalized_architecture="$(get_normalized_architecture_from_architecture "$architecture")"
say_verbose "normalized_architecture=$normalized_architecture"
specific_version="$(get_specific_version_from_version "$azure_feed" "$channel" "$normalized_architecture" "$version")"
specific_version="$(get_specific_version_from_version "$azure_feed" "$channel" "$normalized_architecture" "$version" "$json_file")"
say_verbose "specific_version=$specific_version"
if [ -z "$specific_version" ]; then
say_err "Could not resolve version information."
@@ -826,6 +857,7 @@ temporary_file_template="${TMPDIR:-/tmp}/dotnet.XXXXXXXXX"
channel="LTS"
version="Latest"
json_file=""
install_dir="<auto>"
architecture="<auto>"
dry_run=false
@@ -912,6 +944,10 @@ do
runtime_id="$1"
non_dynamic_parameters+=" $name "\""$1"\"""
;;
--jsonfile|-[Jj][Ss]on[Ff]ile)
shift
json_file="$1"
;;
--skip-non-versioned-files|-[Ss]kip[Nn]on[Vv]ersioned[Ff]iles)
override_non_versioned_files=false
non_dynamic_parameters+=" $name"
@@ -953,22 +989,25 @@ do
echo " Possible values:"
echo " - dotnet - the Microsoft.NETCore.App shared runtime"
echo " - aspnetcore - the Microsoft.AspNetCore.App shared runtime"
echo " --skip-non-versioned-files Skips non-versioned files if they already exist, such as the dotnet executable."
echo " -SkipNonVersionedFiles"
echo " --dry-run,-DryRun Do not perform installation. Display download link."
echo " --no-path, -NoPath Do not set PATH for the current process."
echo " --verbose,-Verbose Display diagnostics information."
echo " --azure-feed,-AzureFeed Azure feed location. Defaults to $azure_feed, This parameter typically is not changed by the user."
echo " --uncached-feed,-UncachedFeed Uncached feed location. This parameter typically is not changed by the user."
echo " --no-cdn,-NoCdn Disable downloading from the Azure CDN, and use the uncached feed directly."
echo " --feed-credential,-FeedCredential Azure feed shared access token. This parameter typically is not specified."
echo " --skip-non-versioned-files Skips non-versioned files if they already exist, such as the dotnet executable."
echo " -SkipNonVersionedFiles"
echo " --no-cdn,-NoCdn Disable downloading from the Azure CDN, and use the uncached feed directly."
echo " --jsonfile <JSONFILE> Determines the SDK version from a user specified global.json file."
echo " Note: global.json must have a value for 'SDK:Version'"
echo " --runtime-id Installs the .NET Tools for the given platform (use linux-x64 for portable linux)."
echo " -RuntimeId"
echo " -?,--?,-h,--help,-Help Shows this help message"
echo ""
echo "Obsolete parameters:"
echo " --shared-runtime The recommended alternative is '--runtime dotnet'."
echo " -SharedRuntime Installs just the shared runtime bits, not the entire SDK."
echo " This parameter is obsolete and may be removed in a future version of this script."
echo " Installs just the shared runtime bits, not the entire SDK."
echo ""
echo "Install Location:"
echo " Location is chosen in following order:"

View File

@@ -123,9 +123,9 @@ function acquireExternalTool() {
}
# Download the external tools only for Windows.
if [[ "$PACKAGERUNTIME" == "win-x64" ]]; then
acquireExternalTool "$NODE_URL/v${NODE12_VERSION}/win-x64/node.exe" node12/bin
acquireExternalTool "$NODE_URL/v${NODE12_VERSION}/win-x64/node.lib" node12/bin
if [[ "$PACKAGERUNTIME" == "win-x64" || "$PACKAGERUNTIME" == "win-x86" ]]; then
acquireExternalTool "$NODE_URL/v${NODE12_VERSION}/$PACKAGERUNTIME/node.exe" node12/bin
acquireExternalTool "$NODE_URL/v${NODE12_VERSION}/$PACKAGERUNTIME/node.lib" node12/bin
if [[ "$PRECACHE" != "" ]]; then
acquireExternalTool "https://github.com/microsoft/vswhere/releases/download/2.6.7/vswhere.exe" vswhere
fi
@@ -136,13 +136,23 @@ if [[ "$PACKAGERUNTIME" == "osx-x64" ]]; then
acquireExternalTool "$NODE_URL/v${NODE12_VERSION}/node-v${NODE12_VERSION}-darwin-x64.tar.gz" node12 fix_nested_dir
fi
# Download the external tools common across Linux PACKAGERUNTIMEs (excluding OSX).
if [[ "$PACKAGERUNTIME" == "linux-x64" || "$PACKAGERUNTIME" == "rhel.6-x64" ]]; then
# Download the external tools for Linux PACKAGERUNTIMEs.
if [[ "$PACKAGERUNTIME" == "linux-x64" ]]; then
acquireExternalTool "$NODE_URL/v${NODE12_VERSION}/node-v${NODE12_VERSION}-linux-x64.tar.gz" node12 fix_nested_dir
# TODO: Repath this blob to use a consistent version format (_ vs .)
acquireExternalTool "https://vstsagenttools.blob.core.windows.net/tools/nodejs/12_4_0/alpine/node-v${NODE12_VERSION}-alpine.tar.gz" node12_alpine
# acquireExternalTool "https://vstsagenttools.blob.core.windows.net/tools/nodejs/12.13.0/alpine/x64/node-v${NODE12_VERSION}-alpine-x64.tar.gz" node12_alpine
fi
if [[ "$PACKAGERUNTIME" == "linux-arm64" ]]; then
acquireExternalTool "$NODE_URL/v${NODE12_VERSION}/node-v${NODE12_VERSION}-linux-arm64.tar.gz" node12 fix_nested_dir
# TODO: alpine node runtime for arm64(8)
# acquireExternalTool "https://vstsagenttools.blob.core.windows.net/tools/nodejs/12.13.0/alpine/arm64/node-v${NODE12_VERSION}-alpine-arm64.tar.gz" node12_alpine
fi
if [[ "$PACKAGERUNTIME" == "linux-arm" ]]; then
acquireExternalTool "$NODE_URL/v${NODE12_VERSION}/node-v${NODE12_VERSION}-linux-armv7l.tar.gz" node12 fix_nested_dir
# TODO: alpine node runtime for arm32(7)
# Need to set up custom gcc toolchain to cross compile on x64 ubuntu for armv7 (per https://github.com/nodejs/node/blob/master/BUILDING.md)
# acquireExternalTool "https://vstsagenttools.blob.core.windows.net/tools/nodejs/12.13.0/alpine/arm/node-v${NODE12_VERSION}-alpine-arm.tar.gz" node12_alpine
fi

View File

@@ -61,9 +61,8 @@ then
exit 1
fi
# ubuntu 18 uses libcurl4
# ubuntu 14, 16 and other linux use libcurl3
apt install -y libcurl3 || apt install -y libcurl4
# libissl version prefer: libssl1.1 -> libssl1.0.2 -> libssl1.0.0
apt install -y libssl1.1$ || apt install -y libssl1.0.2$ || apt install -y libssl1.0.0$
if [ $? -ne 0 ]
then
echo "'apt' failed with exit code '$?'"
@@ -71,18 +70,8 @@ then
exit 1
fi
# debian 9 use libssl1.0.2
# other debian linux use libssl1.0.0
apt install -y libssl1.0.0 || apt install -y libssl1.0.2
if [ $? -ne 0 ]
then
echo "'apt' failed with exit code '$?'"
print_errormessage
exit 1
fi
# libicu version prefer: libicu52 -> libicu55 -> libicu57 -> libicu60
apt install -y libicu52 || apt install -y libicu55 || apt install -y libicu57 || apt install -y libicu60
# libicu version prefer: libicu63 -> libicu60 -> libicu57 -> libicu55 -> libicu52
apt install -y libicu63 || apt install -y libicu60 || apt install -y libicu57 || apt install -y libicu55 || apt install -y libicu52
if [ $? -ne 0 ]
then
echo "'apt' failed with exit code '$?'"
@@ -101,9 +90,8 @@ then
exit 1
fi
# ubuntu 18 uses libcurl4
# ubuntu 14, 16 and other linux use libcurl3
apt-get install -y libcurl3 || apt-get install -y libcurl4
# libissl version prefer: libssl1.1 -> libssl1.0.2 -> libssl1.0.0
apt-get install -y libssl1.1$ || apt-get install -y libssl1.0.2$ || apt install -y libssl1.0.0$
if [ $? -ne 0 ]
then
echo "'apt-get' failed with exit code '$?'"
@@ -111,18 +99,8 @@ then
exit 1
fi
# debian 9 use libssl1.0.2
# other debian linux use libssl1.0.0
apt-get install -y libssl1.0.0 || apt install -y libssl1.0.2
if [ $? -ne 0 ]
then
echo "'apt-get' failed with exit code '$?'"
print_errormessage
exit 1
fi
# libicu version prefer: libicu52 -> libicu55 -> libicu57 -> libicu60
apt-get install -y libicu52 || apt install -y libicu55 || apt install -y libicu57 || apt install -y libicu60
# libicu version prefer: libicu63 -> libicu60 -> libicu57 -> libicu55 -> libicu52
apt-get install -y libicu63 || apt-get install -y libicu60 || apt install -y libicu57 || apt install -y libicu55 || apt install -y libicu52
if [ $? -ne 0 ]
then
echo "'apt-get' failed with exit code '$?'"
@@ -149,46 +127,7 @@ then
command -v dnf
if [ $? -eq 0 ]
then
useCompatSsl=0
grep -i 'fedora release 28' /etc/fedora-release
if [ $? -eq 0 ]
then
useCompatSsl=1
else
grep -i 'fedora release 27' /etc/fedora-release
if [ $? -eq 0 ]
then
useCompatSsl=1
else
grep -i 'fedora release 26' /etc/fedora-release
if [ $? -eq 0 ]
then
useCompatSsl=1
fi
fi
fi
if [ $useCompatSsl -eq 1 ]
then
echo "Use compat-openssl10-devel instead of openssl-devel for Fedora 27/28 (dotnet core requires openssl 1.0.x)"
dnf install -y compat-openssl10
if [ $? -ne 0 ]
then
echo "'dnf' failed with exit code '$?'"
print_errormessage
exit 1
fi
else
dnf install -y openssl-libs
if [ $? -ne 0 ]
then
echo "'dnf' failed with exit code '$?'"
print_errormessage
exit 1
fi
fi
dnf install -y lttng-ust libcurl krb5-libs zlib libicu
dnf install -y lttng-ust openssl-libs krb5-libs zlib libicu
if [ $? -ne 0 ]
then
echo "'dnf' failed with exit code '$?'"
@@ -204,22 +143,13 @@ then
command -v yum
if [ $? -eq 0 ]
then
yum install -y openssl-libs libcurl krb5-libs zlib libicu
yum install -y lttng-ust openssl-libs krb5-libs zlib libicu
if [ $? -ne 0 ]
then
echo "'yum' failed with exit code '$?'"
print_errormessage
exit 1
fi
# install lttng-ust separately since it's not part of offical package repository
yum install -y wget && wget -P /etc/yum.repos.d/ https://packages.efficios.com/repo.files/EfficiOS-RHEL7-x86-64.repo && rpmkeys --import https://packages.efficios.com/rhel/repo.key && yum updateinfo && yum install -y lttng-ust
if [ $? -ne 0 ]
then
echo "'lttng-ust' installation failed with exit code '$?'"
print_errormessage
exit 1
fi
else
echo "Can not find 'yum'"
print_errormessage
@@ -230,13 +160,14 @@ then
# we might on OpenSUSE
OSTYPE=$(grep ID_LIKE /etc/os-release | cut -f2 -d=)
echo $OSTYPE
if [ $OSTYPE == '"suse"' ]
echo $OSTYPE | grep "suse"
if [ $? -eq 0 ]
then
echo "The current OS is SUSE based"
command -v zypper
if [ $? -eq 0 ]
then
zypper -n install lttng-ust libopenssl1_0_0 libcurl4 krb5 zlib libicu52_1
zypper -n install lttng-ust libopenssl1_1 krb5 zlib libicu60_2
if [ $? -ne 0 ]
then
echo "'zypper' failed with exit code '$?'"

View File

@@ -8,7 +8,7 @@ if [ $user_id -eq 0 -a -z "$AGENT_ALLOW_RUNASROOT" ]; then
exit 1
fi
# Check dotnet core 2.1 dependencies for Linux
# Check dotnet core 3.0 dependencies for Linux
if [[ (`uname` == "Linux") ]]
then
command -v ldd > /dev/null
@@ -20,29 +20,22 @@ then
ldd ./bin/libcoreclr.so | grep 'not found'
if [ $? -eq 0 ]; then
echo "Dependencies is missing for Dotnet Core 2.1"
echo "Execute ./bin/installdependencies.sh to install any missing Dotnet Core 2.1 dependencies."
echo "Dependencies is missing for Dotnet Core 3.0"
echo "Execute ./bin/installdependencies.sh to install any missing Dotnet Core 3.0 dependencies."
exit 1
fi
ldd ./bin/System.Security.Cryptography.Native.OpenSsl.so | grep 'not found'
if [ $? -eq 0 ]; then
echo "Dependencies is missing for Dotnet Core 2.1"
echo "Execute ./bin/installdependencies.sh to install any missing Dotnet Core 2.1 dependencies."
echo "Dependencies is missing for Dotnet Core 3.0"
echo "Execute ./bin/installdependencies.sh to install any missing Dotnet Core 3.0 dependencies."
exit 1
fi
ldd ./bin/System.IO.Compression.Native.so | grep 'not found'
if [ $? -eq 0 ]; then
echo "Dependencies is missing for Dotnet Core 2.1"
echo "Execute ./bin/installdependencies.sh to install any missing Dotnet Core 2.1 dependencies."
exit 1
fi
ldd ./bin/System.Net.Http.Native.so | grep 'not found'
if [ $? -eq 0 ]; then
echo "Dependencies is missing for Dotnet Core 2.1"
echo "Execute ./bin/installdependencies.sh to install any missing Dotnet Core 2.1 dependencies."
echo "Dependencies is missing for Dotnet Core 3.0"
echo "Execute ./bin/installdependencies.sh to install any missing Dotnet Core 3.0 dependencies."
exit 1
fi
@@ -59,8 +52,8 @@ then
libpath=${LD_LIBRARY_PATH:-}
$LDCONFIG_COMMAND -NXv ${libpath//:/} 2>&1 | grep libicu >/dev/null 2>&1
if [ $? -ne 0 ]; then
echo "Libicu's dependencies is missing for Dotnet Core 2.1"
echo "Execute ./bin/installdependencies.sh to install any missing Dotnet Core 2.1 dependencies."
echo "Libicu's dependencies is missing for Dotnet Core 3.0"
echo "Execute ./bin/installdependencies.sh to install any missing Dotnet Core 3.0 dependencies."
exit 1
fi
fi

View File

@@ -1,73 +0,0 @@
using System;
using System.Collections.Generic;
using System.Linq;
using System.Threading;
using System.Threading.Tasks;
using GitHub.Runner.Common.Util;
using GitHub.Runner.Sdk;
namespace GitHub.Runner.Common.Capabilities
{
[ServiceLocator(Default = typeof(CapabilitiesManager))]
public interface ICapabilitiesManager : IRunnerService
{
Task<Dictionary<string, string>> GetCapabilitiesAsync(RunnerSettings settings, CancellationToken token);
}
public sealed class CapabilitiesManager : RunnerService, ICapabilitiesManager
{
public async Task<Dictionary<string, string>> GetCapabilitiesAsync(RunnerSettings settings, CancellationToken cancellationToken)
{
Trace.Entering();
ArgUtil.NotNull(settings, nameof(settings));
// Initialize a dictionary of capabilities.
var capabilities = new Dictionary<string, string>(StringComparer.OrdinalIgnoreCase);
if (settings.SkipCapabilitiesScan)
{
Trace.Info("Skip capabilities scan.");
return capabilities;
}
// Get the providers.
var extensionManager = HostContext.GetService<IExtensionManager>();
IEnumerable<ICapabilitiesProvider> providers =
extensionManager
.GetExtensions<ICapabilitiesProvider>()
?.OrderBy(x => x.Order);
// Add each capability returned from each provider.
foreach (ICapabilitiesProvider provider in providers ?? new ICapabilitiesProvider[0])
{
foreach (Capability capability in await provider.GetCapabilitiesAsync(settings, cancellationToken) ?? new List<Capability>())
{
// Make sure we mask secrets in capabilities values.
capabilities[capability.Name] = HostContext.SecretMasker.MaskSecrets(capability.Value);
}
}
return capabilities;
}
}
public interface ICapabilitiesProvider : IExtension
{
int Order { get; }
Task<List<Capability>> GetCapabilitiesAsync(RunnerSettings settings, CancellationToken cancellationToken);
}
public sealed class Capability
{
public string Name { get; }
public string Value { get; }
public Capability(string name, string value)
{
ArgUtil.NotNullOrEmpty(name, nameof(name));
Name = name;
Value = value ?? string.Empty;
}
}
}

View File

@@ -1,86 +0,0 @@
using GitHub.Runner.Common.Util;
using GitHub.Runner.Sdk;
using Microsoft.Win32;
using System;
using System.Collections.Generic;
using System.Threading;
using System.Threading.Tasks;
namespace GitHub.Runner.Common.Capabilities
{
public sealed class RunnerCapabilitiesProvider : RunnerService, ICapabilitiesProvider
{
public Type ExtensionType => typeof(ICapabilitiesProvider);
public int Order => 99; // Process last to override prior.
public Task<List<Capability>> GetCapabilitiesAsync(RunnerSettings settings, CancellationToken cancellationToken)
{
ArgUtil.NotNull(settings, nameof(settings));
var capabilities = new List<Capability>();
Add(capabilities, "Runner.Name", settings.AgentName ?? string.Empty);
Add(capabilities, "Runner.OS", VarUtil.OS);
Add(capabilities, "Runner.OSArchitecture", VarUtil.OSArchitecture);
#if OS_WINDOWS
Add(capabilities, "Runner.OSVersion", GetOSVersionString());
#endif
Add(capabilities, "InteractiveSession", (HostContext.StartupType != StartupType.Service).ToString());
Add(capabilities, "Runner.Version", BuildConstants.RunnerPackage.Version);
Add(capabilities, "Runner.ComputerName", Environment.MachineName ?? string.Empty);
Add(capabilities, "Runner.HomeDirectory", HostContext.GetDirectory(WellKnownDirectory.Root));
return Task.FromResult(capabilities);
}
private void Add(List<Capability> capabilities, string name, string value)
{
Trace.Info($"Adding '{name}': '{value}'");
capabilities.Add(new Capability(name, value));
}
private object GetHklmValue(string keyName, string valueName)
{
keyName = $@"HKEY_LOCAL_MACHINE\{keyName}";
object value = Registry.GetValue(keyName, valueName, defaultValue: null);
if (object.ReferenceEquals(value, null))
{
Trace.Info($"Key name '{keyName}', value name '{valueName}' is null.");
return null;
}
Trace.Info($"Key name '{keyName}', value name '{valueName}': '{value}'");
return value;
}
private string GetOSVersionString()
{
// Do not use System.Environment.OSVersion.Version to resolve the OS version number.
// It leverages the GetVersionEx function which may report an incorrect version
// depending on the app's manifest. For details, see:
// https://msdn.microsoft.com/library/windows/desktop/ms724451(v=vs.85).aspx
// Attempt to retrieve the major/minor version from the new registry values added in
// in Windows 10.
//
// The registry value "CurrentVersion" is unreliable in Windows 10. It contains the
// value "6.3" instead of "10.0".
object major = GetHklmValue(@"SOFTWARE\Microsoft\Windows NT\CurrentVersion", "CurrentMajorVersionNumber");
object minor = GetHklmValue(@"SOFTWARE\Microsoft\Windows NT\CurrentVersion", "CurrentMinorVersionNumber");
string majorMinorString;
if (major != null && minor != null)
{
majorMinorString = StringUtil.Format("{0}.{1}", major, minor);
}
else
{
// Fallback to the registry value "CurrentVersion".
majorMinorString = GetHklmValue(@"SOFTWARE\Microsoft\Windows NT\CurrentVersion", "CurrentVersion") as string;
}
// Opted to use the registry value "CurrentBuildNumber" over "CurrentBuild". Based on brief
// internet investigation, the only difference appears to be that on Windows XP "CurrentBuild"
// was unreliable and "CurrentBuildNumber" was the correct choice.
string build = GetHklmValue(@"SOFTWARE\Microsoft\Windows NT\CurrentVersion", "CurrentBuildNumber") as string;
return StringUtil.Format("{0}.{1}", majorMinorString, build);
}
}
}

View File

@@ -1,6 +1,8 @@
using GitHub.Runner.Common.Util;
using GitHub.Runner.Sdk;
using System;
using System.IO;
using System.Linq;
using System.Runtime.Serialization;
using System.Text;
using System.Threading;
@@ -51,6 +53,34 @@ namespace GitHub.Runner.Common
[DataMember(EmitDefaultValue = false)]
public string MonitorSocketAddress { get; set; }
/// <summary>
// Computed property for convenience. Can either return:
// 1. If runner was configured at the repo level, returns something like: "myorg/myrepo"
// 2. If runner was configured at the org level, returns something like: "myorg"
/// </summary>
public string RepoOrOrgName
{
get
{
Uri accountUri = new Uri(this.ServerUrl);
string repoOrOrgName = string.Empty;
if (accountUri.Host.EndsWith(".githubusercontent.com", StringComparison.OrdinalIgnoreCase))
{
Uri gitHubUrl = new Uri(this.GitHubUrl);
// Use the "NWO part" from the GitHub URL path
repoOrOrgName = gitHubUrl.AbsolutePath.Trim('/');
}
else
{
repoOrOrgName = accountUri.AbsolutePath.Split('/', StringSplitOptions.RemoveEmptyEntries).FirstOrDefault();
}
return repoOrOrgName;
}
}
}
[DataContract]

View File

@@ -39,10 +39,6 @@ namespace GitHub.Runner.Common
var extensions = new List<IExtension>();
switch (typeof(T).FullName)
{
// Listener capabilities providers.
case "GitHub.Runner.Common.Capabilities.ICapabilitiesProvider":
Add<T>(extensions, "GitHub.Runner.Common.Capabilities.RunnerCapabilitiesProvider, Runner.Common");
break;
// Action command extensions.
case "GitHub.Runner.Worker.IActionCommandExtension":
Add<T>(extensions, "GitHub.Runner.Worker.InternalPluginSetRepoPathCommandExtension, Runner.Worker");
@@ -58,6 +54,7 @@ namespace GitHub.Runner.Common
Add<T>(extensions, "GitHub.Runner.Worker.DebugCommandExtension, Runner.Worker");
Add<T>(extensions, "GitHub.Runner.Worker.GroupCommandExtension, Runner.Worker");
Add<T>(extensions, "GitHub.Runner.Worker.EndGroupCommandExtension, Runner.Worker");
Add<T>(extensions, "GitHub.Runner.Worker.EchoCommandExtension, Runner.Worker");
break;
default:
// This should never happen.

View File

@@ -84,9 +84,6 @@ namespace GitHub.Runner.Common
this.SecretMasker.AddValueEncoder(ValueEncoders.Base64StringEscape);
this.SecretMasker.AddValueEncoder(ValueEncoders.Base64StringEscapeShift1);
this.SecretMasker.AddValueEncoder(ValueEncoders.Base64StringEscapeShift2);
this.SecretMasker.AddValueEncoder(ValueEncoders.Base64StringEscapeShift3);
this.SecretMasker.AddValueEncoder(ValueEncoders.Base64StringEscapeShift4);
this.SecretMasker.AddValueEncoder(ValueEncoders.Base64StringEscapeShift5);
this.SecretMasker.AddValueEncoder(ValueEncoders.ExpressionStringEscape);
this.SecretMasker.AddValueEncoder(ValueEncoders.JsonStringEscape);
this.SecretMasker.AddValueEncoder(ValueEncoders.UriDataEscape);

View File

@@ -1,13 +1,14 @@
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFramework>netcoreapp2.2</TargetFramework>
<TargetFramework>netcoreapp3.0</TargetFramework>
<OutputType>Library</OutputType>
<RuntimeIdentifiers>win-x64;win-x86;linux-x64;linux-arm;rhel.6-x64;osx-x64</RuntimeIdentifiers>
<RuntimeIdentifiers>win-x64;win-x86;linux-x64;linux-arm64;linux-arm;osx-x64</RuntimeIdentifiers>
<TargetLatestRuntimePatch>true</TargetLatestRuntimePatch>
<AssetTargetFallback>portable-net45+win8</AssetTargetFallback>
<NoWarn>NU1701;NU1603</NoWarn>
<Version>$(Version)</Version>
<TieredCompilationQuickJit>true</TieredCompilationQuickJit>
</PropertyGroup>
<ItemGroup>
@@ -27,42 +28,4 @@
<DebugType>portable</DebugType>
</PropertyGroup>
<PropertyGroup Condition="'$([System.Runtime.InteropServices.RuntimeInformation]::IsOSPlatform($([System.Runtime.InteropServices.OSPlatform]::Windows)))' == 'true' AND '$(PackageRuntime)' == 'win-x64'">
<DefineConstants>OS_WINDOWS;X64;TRACE</DefineConstants>
</PropertyGroup>
<PropertyGroup Condition="'$([System.Runtime.InteropServices.RuntimeInformation]::IsOSPlatform($([System.Runtime.InteropServices.OSPlatform]::Windows)))' == 'true' AND '$(PackageRuntime)' == 'win-x86'">
<DefineConstants>OS_WINDOWS;X86;TRACE</DefineConstants>
</PropertyGroup>
<PropertyGroup Condition="'$([System.Runtime.InteropServices.RuntimeInformation]::IsOSPlatform($([System.Runtime.InteropServices.OSPlatform]::Windows)))' == 'true' AND '$(Configuration)' == 'Debug' AND '$(PackageRuntime)' == 'win-x64'">
<DefineConstants>OS_WINDOWS;X64;DEBUG;TRACE</DefineConstants>
</PropertyGroup>
<PropertyGroup Condition="'$([System.Runtime.InteropServices.RuntimeInformation]::IsOSPlatform($([System.Runtime.InteropServices.OSPlatform]::Windows)))' == 'true' AND '$(Configuration)' == 'Debug' AND '$(PackageRuntime)' == 'win-x86'">
<DefineConstants>OS_WINDOWS;X86;DEBUG;TRACE</DefineConstants>
</PropertyGroup>
<PropertyGroup Condition="'$([System.Runtime.InteropServices.RuntimeInformation]::IsOSPlatform($([System.Runtime.InteropServices.OSPlatform]::OSX)))' == 'true'">
<DefineConstants>OS_OSX;X64;TRACE</DefineConstants>
</PropertyGroup>
<PropertyGroup Condition="'$([System.Runtime.InteropServices.RuntimeInformation]::IsOSPlatform($([System.Runtime.InteropServices.OSPlatform]::OSX)))' == 'true' AND '$(Configuration)' == 'Debug'">
<DefineConstants>OS_OSX;DEBUG;X64;TRACE</DefineConstants>
</PropertyGroup>
<PropertyGroup Condition="'$([System.Runtime.InteropServices.RuntimeInformation]::IsOSPlatform($([System.Runtime.InteropServices.OSPlatform]::Linux)))' == 'true' AND '$(PackageRuntime)' == 'linux-x64'">
<DefineConstants>OS_LINUX;X64;TRACE</DefineConstants>
</PropertyGroup>
<PropertyGroup Condition="'$([System.Runtime.InteropServices.RuntimeInformation]::IsOSPlatform($([System.Runtime.InteropServices.OSPlatform]::Linux)))' == 'true' AND '$(PackageRuntime)' == 'rhel.6-x64'">
<DefineConstants>OS_LINUX;OS_RHEL6;X64;TRACE</DefineConstants>
</PropertyGroup>
<PropertyGroup Condition="'$([System.Runtime.InteropServices.RuntimeInformation]::IsOSPlatform($([System.Runtime.InteropServices.OSPlatform]::Linux)))' == 'true' AND '$(PackageRuntime)' == 'linux-arm'">
<DefineConstants>OS_LINUX;ARM;TRACE</DefineConstants>
</PropertyGroup>
<PropertyGroup Condition="'$([System.Runtime.InteropServices.RuntimeInformation]::IsOSPlatform($([System.Runtime.InteropServices.OSPlatform]::Linux)))' == 'true' AND '$(Configuration)' == 'Debug' AND '$(PackageRuntime)' == 'linux-x64'">
<DefineConstants>OS_LINUX;X64;DEBUG;TRACE</DefineConstants>
</PropertyGroup>
<PropertyGroup Condition="'$([System.Runtime.InteropServices.RuntimeInformation]::IsOSPlatform($([System.Runtime.InteropServices.OSPlatform]::Linux)))' == 'true' AND '$(Configuration)' == 'Debug' AND '$(PackageRuntime)' == 'rhel.6-x64'">
<DefineConstants>OS_LINUX;OS_RHEL6;X64;DEBUG;TRACE</DefineConstants>
</PropertyGroup>
<PropertyGroup Condition="'$([System.Runtime.InteropServices.RuntimeInformation]::IsOSPlatform($([System.Runtime.InteropServices.OSPlatform]::Linux)))' == 'true' AND '$(Configuration)' == 'Debug' AND '$(PackageRuntime)' == 'linux-arm'">
<DefineConstants>OS_LINUX;ARM;DEBUG;TRACE</DefineConstants>
</PropertyGroup>
</Project>

View File

@@ -31,7 +31,7 @@ namespace GitHub.Runner.Common
Task DeleteAgentAsync(int agentPoolId, int agentId);
Task<List<TaskAgentPool>> GetAgentPoolsAsync(string agentPoolName = null, TaskAgentPoolType poolType = TaskAgentPoolType.Automation);
Task<List<TaskAgent>> GetAgentsAsync(int agentPoolId, string agentName = null);
Task<TaskAgent> UpdateAgentAsync(int agentPoolId, TaskAgent agent);
Task<TaskAgent> ReplaceAgentAsync(int agentPoolId, TaskAgent agent);
// messagequeue
Task<TaskAgentSession> CreateAgentSessionAsync(Int32 poolId, TaskAgentSession session, CancellationToken cancellationToken);
@@ -257,7 +257,7 @@ namespace GitHub.Runner.Common
return _genericTaskAgentClient.GetAgentsAsync(agentPoolId, agentName, false);
}
public Task<TaskAgent> UpdateAgentAsync(int agentPoolId, TaskAgent agent)
public Task<TaskAgent> ReplaceAgentAsync(int agentPoolId, TaskAgent agent)
{
CheckConnection(RunnerConnectionType.Generic);
return _genericTaskAgentClient.ReplaceAgentAsync(agentPoolId, agent);

View File

@@ -54,6 +54,8 @@ namespace GitHub.Runner.Common.Util
return "X64";
case Constants.Architecture.Arm:
return "ARM";
case Constants.Architecture.Arm64:
return "ARM64";
default:
throw new NotSupportedException(); // Should never reach here.
}

View File

@@ -1,24 +1,19 @@
using GitHub.DistributedTask.WebApi;
using GitHub.Runner.Common.Capabilities;
using GitHub.Runner.Common.Util;
using GitHub.Services.Common;
using GitHub.Services.OAuth;
using GitHub.Services.WebApi;
using System;
using System.Collections.Generic;
using System.Diagnostics;
using System.IO;
using System.Linq;
using System.Security.Cryptography;
using System.Security.Principal;
using System.Threading;
using System.Threading.Tasks;
using System.Runtime.InteropServices;
using GitHub.Runner.Common;
using GitHub.Runner.Sdk;
using System.Net.Http;
using System.Net.Http.Headers;
using System.Text;
namespace GitHub.Runner.Listener.Configuration
{
@@ -168,7 +163,8 @@ namespace GitHub.Runner.Listener.Configuration
{
// Get the URL
var inputUrl = command.GetUrl();
if (!inputUrl.Contains("github.com", StringComparison.OrdinalIgnoreCase))
if (!inputUrl.Contains("github.com", StringComparison.OrdinalIgnoreCase) &&
!inputUrl.Contains("github.localhost", StringComparison.OrdinalIgnoreCase))
{
runnerSettings.ServerUrl = inputUrl;
// Get the credentials
@@ -238,9 +234,6 @@ namespace GitHub.Runner.Listener.Configuration
{
runnerSettings.AgentName = command.GetAgentName();
// Get the system capabilities.
Dictionary<string, string> systemCapabilities = await HostContext.GetService<ICapabilitiesManager>().GetCapabilitiesAsync(runnerSettings, CancellationToken.None);
_term.WriteLine();
var agents = await _runnerServer.GetAgentsAsync(runnerSettings.PoolId, runnerSettings.AgentName);
@@ -251,12 +244,12 @@ namespace GitHub.Runner.Listener.Configuration
_term.WriteLine("A runner exists with the same name", ConsoleColor.Yellow);
if (command.GetReplace())
{
// Update existing agent with new PublicKey, agent version and SystemCapabilities.
agent = UpdateExistingAgent(agent, publicKey, systemCapabilities);
// Update existing agent with new PublicKey, agent version.
agent = UpdateExistingAgent(agent, publicKey);
try
{
agent = await _runnerServer.UpdateAgentAsync(runnerSettings.PoolId, agent);
agent = await _runnerServer.ReplaceAgentAsync(runnerSettings.PoolId, agent);
_term.WriteSuccessMessage("Successfully replaced the runner");
break;
}
@@ -275,7 +268,7 @@ namespace GitHub.Runner.Listener.Configuration
else
{
// Create a new agent.
agent = CreateNewAgent(runnerSettings.AgentName, publicKey, systemCapabilities);
agent = CreateNewAgent(runnerSettings.AgentName, publicKey);
try
{
@@ -573,7 +566,7 @@ namespace GitHub.Runner.Listener.Configuration
}
private TaskAgent UpdateExistingAgent(TaskAgent agent, RSAParameters publicKey, Dictionary<string, string> systemCapabilities)
private TaskAgent UpdateExistingAgent(TaskAgent agent, RSAParameters publicKey)
{
ArgUtil.NotNull(agent, nameof(agent));
agent.Authorization = new TaskAgentAuthorization
@@ -585,15 +578,14 @@ namespace GitHub.Runner.Listener.Configuration
agent.Version = BuildConstants.RunnerPackage.Version;
agent.OSDescription = RuntimeInformation.OSDescription;
foreach (KeyValuePair<string, string> capability in systemCapabilities)
{
agent.SystemCapabilities[capability.Key] = capability.Value ?? string.Empty;
}
agent.Labels.Add("self-hosted");
agent.Labels.Add(VarUtil.OS);
agent.Labels.Add(VarUtil.OSArchitecture);
return agent;
}
private TaskAgent CreateNewAgent(string agentName, RSAParameters publicKey, Dictionary<string, string> systemCapabilities)
private TaskAgent CreateNewAgent(string agentName, RSAParameters publicKey)
{
TaskAgent agent = new TaskAgent(agentName)
{
@@ -606,10 +598,9 @@ namespace GitHub.Runner.Listener.Configuration
OSDescription = RuntimeInformation.OSDescription,
};
foreach (KeyValuePair<string, string> capability in systemCapabilities)
{
agent.SystemCapabilities[capability.Key] = capability.Value ?? string.Empty;
}
agent.Labels.Add("self-hosted");
agent.Labels.Add(VarUtil.OS);
agent.Labels.Add(VarUtil.OSArchitecture);
return agent;
}
@@ -638,7 +629,7 @@ namespace GitHub.Runner.Listener.Configuration
private async Task<GitHubAuthResult> GetTenantCredential(string githubUrl, string githubToken)
{
var gitHubUrl = new UriBuilder(githubUrl);
var githubApiUrl = $"https://api.github.com/repos/{gitHubUrl.Path.Trim('/')}/actions-runners/registration";
var githubApiUrl = $"https://api.{gitHubUrl.Host}/repos/{gitHubUrl.Path.Trim('/')}/actions-runners/registration";
using (var httpClientHandler = HostContext.CreateHttpClientHandler())
using (var httpClient = new HttpClient(httpClientHandler))
{

View File

@@ -12,8 +12,8 @@ namespace GitHub.Runner.Listener.Configuration
public class OsxServiceControlManager : ServiceControlManager, ILinuxServiceControlManager
{
// This is the name you would see when you do `systemctl list-units | grep runner`
private const string _svcNamePattern = "actions.runner.{0}.{1}.{2}";
private const string _svcDisplayPattern = "GitHub Actions Runner ({0}.{1}.{2})";
private const string _svcNamePattern = "actions.runner.{0}.{1}";
private const string _svcDisplayPattern = "GitHub Actions Runner ({0}.{1})";
private const string _shTemplate = "darwin.svc.sh.template";
private const string _svcShName = "svc.sh";

View File

@@ -1,5 +1,6 @@
using System;
using System.Linq;
using System.Text.RegularExpressions;
using GitHub.Runner.Common;
using GitHub.Runner.Common.Util;
using GitHub.Runner.Sdk;
@@ -37,25 +38,38 @@ namespace GitHub.Runner.Listener.Configuration
serviceName = string.Empty;
serviceDisplayName = string.Empty;
Uri accountUri = new Uri(settings.ServerUrl);
string accountName = string.Empty;
if (accountUri.Host.EndsWith(".githubusercontent.com", StringComparison.OrdinalIgnoreCase))
if (string.IsNullOrEmpty(settings.RepoOrOrgName))
{
accountName = accountUri.AbsolutePath.Split('/', StringSplitOptions.RemoveEmptyEntries).FirstOrDefault();
}
else
{
accountName = accountUri.Host.Split('.').FirstOrDefault();
throw new InvalidOperationException($"Cannot find GitHub repository/organization name from server url: '{settings.ServerUrl}'");
}
if (string.IsNullOrEmpty(accountName))
// For the service name, replace any characters outside of the alpha-numeric set and ".", "_", "-" with "-"
Regex regex = new Regex(@"[^0-9a-zA-Z._\-]");
string repoOrOrgName = regex.Replace(settings.RepoOrOrgName, "-");
serviceName = StringUtil.Format(serviceNamePattern, repoOrOrgName, settings.AgentName);
if (serviceName.Length > 80)
{
throw new InvalidOperationException($"Cannot find GitHub organization name from server url: '{settings.ServerUrl}'");
Trace.Verbose($"Calculated service name is too long (> 80 chars). Trying again by calculating a shorter name.");
int exceededCharLength = serviceName.Length - 80;
string repoOrOrgNameSubstring = StringUtil.SubstringPrefix(repoOrOrgName, 45);
exceededCharLength -= repoOrOrgName.Length - repoOrOrgNameSubstring.Length;
string runnerNameSubstring = settings.AgentName;
// Only trim runner name if it's really necessary
if (exceededCharLength > 0)
{
runnerNameSubstring = StringUtil.SubstringPrefix(settings.AgentName, settings.AgentName.Length - exceededCharLength);
}
serviceName = StringUtil.Format(serviceNamePattern, accountName, settings.PoolName, settings.AgentName);
serviceDisplayName = StringUtil.Format(serviceDisplayNamePattern, accountName, settings.PoolName, settings.AgentName);
serviceName = StringUtil.Format(serviceNamePattern, repoOrOrgNameSubstring, runnerNameSubstring);
}
serviceDisplayName = StringUtil.Format(serviceDisplayNamePattern, repoOrOrgName, settings.AgentName);
Trace.Info($"Service name '{serviceName}' display name '{serviceDisplayName}' will be used for service configuration.");
}

View File

@@ -13,8 +13,8 @@ namespace GitHub.Runner.Listener.Configuration
public class SystemDControlManager : ServiceControlManager, ILinuxServiceControlManager
{
// This is the name you would see when you do `systemctl list-units | grep runner`
private const string _svcNamePattern = "actions.runner.{0}.{1}.{2}.service";
private const string _svcDisplayPattern = "GitHub Actions Runner ({0}.{1}.{2})";
private const string _svcNamePattern = "actions.runner.{0}.{1}.service";
private const string _svcDisplayPattern = "GitHub Actions Runner ({0}.{1})";
private const string _shTemplate = "systemd.svc.sh.template";
private const string _shName = "svc.sh";

View File

@@ -15,8 +15,8 @@ namespace GitHub.Runner.Listener.Configuration
{
public const string WindowsServiceControllerName = "RunnerService.exe";
private const string ServiceNamePattern = "actionsrunner.{0}.{1}.{2}";
private const string ServiceDisplayNamePattern = "GitHub Actions Runner ({0}.{1}.{2})";
private const string ServiceNamePattern = "actions.runner.{0}.{1}";
private const string ServiceDisplayNamePattern = "GitHub Actions Runner ({0}.{1})";
private INativeWindowsServiceHelper _windowsServiceHelper;
private ITerminal _term;

View File

@@ -1,7 +1,5 @@
using GitHub.DistributedTask.WebApi;
using GitHub.Runner.Common.Capabilities;
using GitHub.Runner.Listener.Configuration;
using GitHub.Runner.Common.Util;
using GitHub.Services.Common;
using System;
using System.Collections.Generic;
@@ -10,7 +8,6 @@ using System.Threading.Tasks;
using System.Security.Cryptography;
using System.IO;
using System.Text;
using GitHub.Services.WebApi;
using GitHub.Services.OAuth;
using System.Diagnostics;
using System.Runtime.InteropServices;
@@ -59,9 +56,6 @@ namespace GitHub.Runner.Listener
var serverUrl = _settings.ServerUrl;
Trace.Info(_settings);
// Capabilities.
Dictionary<string, string> systemCapabilities = await HostContext.GetService<ICapabilitiesManager>().GetCapabilitiesAsync(_settings, token);
// Create connection.
Trace.Info("Loading Credentials");
var credMgr = HostContext.GetService<ICredentialManager>();
@@ -75,7 +69,7 @@ namespace GitHub.Runner.Listener
OSDescription = RuntimeInformation.OSDescription,
};
string sessionName = $"{Environment.MachineName ?? "RUNNER"}";
var taskAgentSession = new TaskAgentSession(sessionName, agent, systemCapabilities);
var taskAgentSession = new TaskAgentSession(sessionName, agent);
string errorMessage = string.Empty;
bool encounteringError = false;

View File

@@ -1,13 +1,15 @@
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFramework>netcoreapp2.2</TargetFramework>
<TargetFramework>netcoreapp3.0</TargetFramework>
<OutputType>Exe</OutputType>
<RuntimeIdentifiers>win-x64;win-x86;linux-x64;linux-arm;rhel.6-x64;osx-x64</RuntimeIdentifiers>
<RuntimeIdentifiers>win-x64;win-x86;linux-x64;linux-arm64;linux-arm;osx-x64</RuntimeIdentifiers>
<TargetLatestRuntimePatch>true</TargetLatestRuntimePatch>
<AssetTargetFallback>portable-net45+win8</AssetTargetFallback>
<NoWarn>NU1701;NU1603</NoWarn>
<Version>$(Version)</Version>
<TieredCompilationQuickJit>true</TieredCompilationQuickJit>
<PublishReadyToRun>true</PublishReadyToRun>
</PropertyGroup>
<ItemGroup>
@@ -29,42 +31,4 @@
<DebugType>portable</DebugType>
</PropertyGroup>
<PropertyGroup Condition="'$([System.Runtime.InteropServices.RuntimeInformation]::IsOSPlatform($([System.Runtime.InteropServices.OSPlatform]::Windows)))' == 'true' AND '$(PackageRuntime)' == 'win-x64'">
<DefineConstants>OS_WINDOWS;X64;TRACE</DefineConstants>
</PropertyGroup>
<PropertyGroup Condition="'$([System.Runtime.InteropServices.RuntimeInformation]::IsOSPlatform($([System.Runtime.InteropServices.OSPlatform]::Windows)))' == 'true' AND '$(PackageRuntime)' == 'win-x86'">
<DefineConstants>OS_WINDOWS;X86;TRACE</DefineConstants>
</PropertyGroup>
<PropertyGroup Condition="'$([System.Runtime.InteropServices.RuntimeInformation]::IsOSPlatform($([System.Runtime.InteropServices.OSPlatform]::Windows)))' == 'true' AND '$(Configuration)' == 'Debug' AND '$(PackageRuntime)' == 'win-x64'">
<DefineConstants>OS_WINDOWS;X64;DEBUG;TRACE</DefineConstants>
</PropertyGroup>
<PropertyGroup Condition="'$([System.Runtime.InteropServices.RuntimeInformation]::IsOSPlatform($([System.Runtime.InteropServices.OSPlatform]::Windows)))' == 'true' AND '$(Configuration)' == 'Debug' AND '$(PackageRuntime)' == 'win-x86'">
<DefineConstants>OS_WINDOWS;X86;DEBUG;TRACE</DefineConstants>
</PropertyGroup>
<PropertyGroup Condition="'$([System.Runtime.InteropServices.RuntimeInformation]::IsOSPlatform($([System.Runtime.InteropServices.OSPlatform]::OSX)))' == 'true'">
<DefineConstants>OS_OSX;X64;TRACE</DefineConstants>
</PropertyGroup>
<PropertyGroup Condition="'$([System.Runtime.InteropServices.RuntimeInformation]::IsOSPlatform($([System.Runtime.InteropServices.OSPlatform]::OSX)))' == 'true' AND '$(Configuration)' == 'Debug'">
<DefineConstants>OS_OSX;DEBUG;X64;TRACE</DefineConstants>
</PropertyGroup>
<PropertyGroup Condition="'$([System.Runtime.InteropServices.RuntimeInformation]::IsOSPlatform($([System.Runtime.InteropServices.OSPlatform]::Linux)))' == 'true' AND '$(PackageRuntime)' == 'linux-x64'">
<DefineConstants>OS_LINUX;X64;TRACE</DefineConstants>
</PropertyGroup>
<PropertyGroup Condition="'$([System.Runtime.InteropServices.RuntimeInformation]::IsOSPlatform($([System.Runtime.InteropServices.OSPlatform]::Linux)))' == 'true' AND '$(PackageRuntime)' == 'rhel.6-x64'">
<DefineConstants>OS_LINUX;OS_RHEL6;X64;TRACE</DefineConstants>
</PropertyGroup>
<PropertyGroup Condition="'$([System.Runtime.InteropServices.RuntimeInformation]::IsOSPlatform($([System.Runtime.InteropServices.OSPlatform]::Linux)))' == 'true' AND '$(PackageRuntime)' == 'linux-arm'">
<DefineConstants>OS_LINUX;ARM;TRACE</DefineConstants>
</PropertyGroup>
<PropertyGroup Condition="'$([System.Runtime.InteropServices.RuntimeInformation]::IsOSPlatform($([System.Runtime.InteropServices.OSPlatform]::Linux)))' == 'true' AND '$(Configuration)' == 'Debug' AND '$(PackageRuntime)' == 'linux-x64'">
<DefineConstants>OS_LINUX;X64;DEBUG;TRACE</DefineConstants>
</PropertyGroup>
<PropertyGroup Condition="'$([System.Runtime.InteropServices.RuntimeInformation]::IsOSPlatform($([System.Runtime.InteropServices.OSPlatform]::Linux)))' == 'true' AND '$(Configuration)' == 'Debug' AND '$(PackageRuntime)' == 'rhel.6-x64'">
<DefineConstants>OS_LINUX;OS_RHEL6;X64;DEBUG;TRACE</DefineConstants>
</PropertyGroup>
<PropertyGroup Condition="'$([System.Runtime.InteropServices.RuntimeInformation]::IsOSPlatform($([System.Runtime.InteropServices.OSPlatform]::Linux)))' == 'true' AND '$(Configuration)' == 'Debug' AND '$(PackageRuntime)' == 'linux-arm'">
<DefineConstants>OS_LINUX;ARM;DEBUG;TRACE</DefineConstants>
</PropertyGroup>
</Project>

View File

@@ -1,13 +1,15 @@
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFramework>netcoreapp2.2</TargetFramework>
<TargetFramework>netcoreapp3.0</TargetFramework>
<OutputType>Exe</OutputType>
<RuntimeIdentifiers>win-x64;win-x86;linux-x64;linux-arm;rhel.6-x64;osx-x64</RuntimeIdentifiers>
<RuntimeIdentifiers>win-x64;win-x86;linux-x64;linux-arm64;linux-arm;osx-x64</RuntimeIdentifiers>
<TargetLatestRuntimePatch>true</TargetLatestRuntimePatch>
<AssetTargetFallback>portable-net45+win8</AssetTargetFallback>
<NoWarn>NU1701;NU1603</NoWarn>
<Version>$(Version)</Version>
<TieredCompilationQuickJit>true</TieredCompilationQuickJit>
<PublishReadyToRun>true</PublishReadyToRun>
</PropertyGroup>
<ItemGroup>
@@ -22,42 +24,4 @@
<DebugType>portable</DebugType>
</PropertyGroup>
<PropertyGroup Condition="'$([System.Runtime.InteropServices.RuntimeInformation]::IsOSPlatform($([System.Runtime.InteropServices.OSPlatform]::Windows)))' == 'true' AND '$(PackageRuntime)' == 'win-x64'">
<DefineConstants>OS_WINDOWS;X64;TRACE</DefineConstants>
</PropertyGroup>
<PropertyGroup Condition="'$([System.Runtime.InteropServices.RuntimeInformation]::IsOSPlatform($([System.Runtime.InteropServices.OSPlatform]::Windows)))' == 'true' AND '$(PackageRuntime)' == 'win-x86'">
<DefineConstants>OS_WINDOWS;X86;TRACE</DefineConstants>
</PropertyGroup>
<PropertyGroup Condition="'$([System.Runtime.InteropServices.RuntimeInformation]::IsOSPlatform($([System.Runtime.InteropServices.OSPlatform]::Windows)))' == 'true' AND '$(Configuration)' == 'Debug' AND '$(PackageRuntime)' == 'win-x64'">
<DefineConstants>OS_WINDOWS;X64;DEBUG;TRACE</DefineConstants>
</PropertyGroup>
<PropertyGroup Condition="'$([System.Runtime.InteropServices.RuntimeInformation]::IsOSPlatform($([System.Runtime.InteropServices.OSPlatform]::Windows)))' == 'true' AND '$(Configuration)' == 'Debug' AND '$(PackageRuntime)' == 'win-x86'">
<DefineConstants>OS_WINDOWS;X86;DEBUG;TRACE</DefineConstants>
</PropertyGroup>
<PropertyGroup Condition="'$([System.Runtime.InteropServices.RuntimeInformation]::IsOSPlatform($([System.Runtime.InteropServices.OSPlatform]::OSX)))' == 'true'">
<DefineConstants>OS_OSX;X64;TRACE</DefineConstants>
</PropertyGroup>
<PropertyGroup Condition="'$([System.Runtime.InteropServices.RuntimeInformation]::IsOSPlatform($([System.Runtime.InteropServices.OSPlatform]::OSX)))' == 'true' AND '$(Configuration)' == 'Debug'">
<DefineConstants>OS_OSX;DEBUG;X64;TRACE</DefineConstants>
</PropertyGroup>
<PropertyGroup Condition="'$([System.Runtime.InteropServices.RuntimeInformation]::IsOSPlatform($([System.Runtime.InteropServices.OSPlatform]::Linux)))' == 'true' AND '$(PackageRuntime)' == 'linux-x64'">
<DefineConstants>OS_LINUX;X64;TRACE</DefineConstants>
</PropertyGroup>
<PropertyGroup Condition="'$([System.Runtime.InteropServices.RuntimeInformation]::IsOSPlatform($([System.Runtime.InteropServices.OSPlatform]::Linux)))' == 'true' AND '$(PackageRuntime)' == 'rhel.6-x64'">
<DefineConstants>OS_LINUX;OS_RHEL6;X64;TRACE</DefineConstants>
</PropertyGroup>
<PropertyGroup Condition="'$([System.Runtime.InteropServices.RuntimeInformation]::IsOSPlatform($([System.Runtime.InteropServices.OSPlatform]::Linux)))' == 'true' AND '$(PackageRuntime)' == 'linux-arm'">
<DefineConstants>OS_LINUX;ARM;TRACE</DefineConstants>
</PropertyGroup>
<PropertyGroup Condition="'$([System.Runtime.InteropServices.RuntimeInformation]::IsOSPlatform($([System.Runtime.InteropServices.OSPlatform]::Linux)))' == 'true' AND '$(Configuration)' == 'Debug' AND '$(PackageRuntime)' == 'linux-x64'">
<DefineConstants>OS_LINUX;X64;DEBUG;TRACE</DefineConstants>
</PropertyGroup>
<PropertyGroup Condition="'$([System.Runtime.InteropServices.RuntimeInformation]::IsOSPlatform($([System.Runtime.InteropServices.OSPlatform]::Linux)))' == 'true' AND '$(Configuration)' == 'Debug' AND '$(PackageRuntime)' == 'rhel.6-x64'">
<DefineConstants>OS_LINUX;OS_RHEL6;X64;DEBUG;TRACE</DefineConstants>
</PropertyGroup>
<PropertyGroup Condition="'$([System.Runtime.InteropServices.RuntimeInformation]::IsOSPlatform($([System.Runtime.InteropServices.OSPlatform]::Linux)))' == 'true' AND '$(Configuration)' == 'Debug' AND '$(PackageRuntime)' == 'linux-arm'">
<DefineConstants>OS_LINUX;ARM;DEBUG;TRACE</DefineConstants>
</PropertyGroup>
</Project>

View File

@@ -50,28 +50,61 @@ namespace GitHub.Runner.Plugins.Artifact
throw new ArgumentException($"Run Id is not an Int32: {buildIdStr}");
}
context.Output($"Download artifact '{artifactName}' to: '{targetPath}'");
// Determine whether to call Pipelines or Build endpoint to publish artifact based on variable setting
string usePipelinesArtifactEndpointVar = context.Variables.GetValueOrDefault("Runner.UseActionsArtifactsApis")?.Value;
bool.TryParse(usePipelinesArtifactEndpointVar, out bool usePipelinesArtifactEndpoint);
string containerPath;
long containerId;
context.Output($"Downloading artifact '{artifactName}' to: '{targetPath}'");
if (usePipelinesArtifactEndpoint)
{
context.Debug("Downloading artifact using v2 endpoint");
// Definition ID is a dummy value only used by HTTP client routing purposes
int definitionId = 1;
var pipelinesHelper = new PipelinesServer(context.VssConnection);
var actionsStorageArtifact = await pipelinesHelper.GetActionsStorageArtifact(definitionId, buildId, artifactName, token);
if (actionsStorageArtifact == null)
{
throw new Exception($"The actions storage artifact for '{artifactName}' could not be found, or is no longer available");
}
containerPath = actionsStorageArtifact.Name; // In actions storage artifacts, name equals the path
containerId = actionsStorageArtifact.ContainerId;
}
else
{
context.Debug("Downloading artifact using v1 endpoint");
BuildServer buildHelper = new BuildServer(context.VssConnection);
BuildArtifact buildArtifact = await buildHelper.GetArtifact(projectId, buildId, artifactName, token);
if (string.Equals(buildArtifact.Resource.Type, "Container", StringComparison.OrdinalIgnoreCase))
if (string.Equals(buildArtifact.Resource.Type, "Container", StringComparison.OrdinalIgnoreCase) ||
// Artifact was published by Pipelines endpoint, check new type here to handle rollback scenario
string.Equals(buildArtifact.Resource.Type, "Actions_Storage", StringComparison.OrdinalIgnoreCase))
{
string containerUrl = buildArtifact.Resource.Data;
string[] parts = containerUrl.Split(new[] { '/' }, 3);
if (parts.Length < 3 || !long.TryParse(parts[1], out long containerId))
if (parts.Length < 3 || !long.TryParse(parts[1], out containerId))
{
throw new ArgumentOutOfRangeException($"Invalid container url '{containerUrl}' for artifact '{buildArtifact.Name}'");
}
string containerPath = parts[2];
FileContainerServer fileContainerServer = new FileContainerServer(context.VssConnection, projectId, containerId, containerPath);
await fileContainerServer.DownloadFromContainerAsync(context, targetPath, token);
containerPath = parts[2];
}
else
{
throw new NotSupportedException($"Invalid artifact type: {buildArtifact.Resource.Type}");
}
}
FileContainerServer fileContainerServer = new FileContainerServer(context.VssConnection, projectId, containerId, containerPath);
await fileContainerServer.DownloadFromContainerAsync(context, targetPath, token);
context.Output("Artifact download finished.");
}

View File

@@ -236,15 +236,15 @@ namespace GitHub.Runner.Plugins.Artifact
// try upload all files for the first time.
UploadResult uploadResult = await ParallelUploadAsync(context, files, maxConcurrentUploads, _uploadCancellationTokenSource.Token);
if (uploadResult.FailedFiles.Count == 0)
if (uploadResult.RetryFiles.Count == 0)
{
// all files have been upload succeed.
context.Output("File upload succeed.");
context.Output("File upload complete.");
return uploadResult.TotalFileSizeUploaded;
}
else
{
context.Output($"{uploadResult.FailedFiles.Count} files failed to upload, retry these files after a minute.");
context.Output($"{uploadResult.RetryFiles.Count} files failed to upload, retry these files after a minute.");
}
// Delay 1 min then retry failed files.
@@ -255,13 +255,13 @@ namespace GitHub.Runner.Plugins.Artifact
}
// Retry upload all failed files.
context.Output($"Start retry {uploadResult.FailedFiles.Count} failed files upload.");
UploadResult retryUploadResult = await ParallelUploadAsync(context, uploadResult.FailedFiles, maxConcurrentUploads, _uploadCancellationTokenSource.Token);
context.Output($"Start retry {uploadResult.RetryFiles.Count} failed files upload.");
UploadResult retryUploadResult = await ParallelUploadAsync(context, uploadResult.RetryFiles, maxConcurrentUploads, _uploadCancellationTokenSource.Token);
if (retryUploadResult.FailedFiles.Count == 0)
if (retryUploadResult.RetryFiles.Count == 0)
{
// all files have been upload succeed after retry.
context.Output("File upload succeed after retry.");
context.Output("File upload complete after retry.");
return uploadResult.TotalFileSizeUploaded + retryUploadResult.TotalFileSizeUploaded;
}
else
@@ -465,76 +465,62 @@ namespace GitHub.Runner.Plugins.Artifact
using (FileStream fs = File.Open(fileToUpload, FileMode.Open, FileAccess.Read, FileShare.Read))
{
string itemPath = (_containerPath.TrimEnd('/') + "/" + fileToUpload.Remove(0, _sourceParentDirectory.Length + 1)).Replace('\\', '/');
uploadTimer.Restart();
bool catchExceptionDuringUpload = false;
HttpResponseMessage response = null;
bool failAndExit = false;
try
{
response = await _fileContainerHttpClient.UploadFileAsync(_containerId, itemPath, fs, _projectId, cancellationToken: token, chunkSize: 4 * 1024 * 1024);
uploadTimer.Restart();
using (HttpResponseMessage response = await _fileContainerHttpClient.UploadFileAsync(_containerId, itemPath, fs, _projectId, cancellationToken: token, chunkSize: 4 * 1024 * 1024))
{
if (response == null || response.StatusCode != HttpStatusCode.Created)
{
context.Output($"Unable to copy file to server StatusCode={response?.StatusCode}: {response?.ReasonPhrase}. Source file path: {fileToUpload}. Target server path: {itemPath}");
if (response?.StatusCode == HttpStatusCode.Conflict)
{
// fail upload task but continue with any other files
context.Error($"Error '{fileToUpload}' has already been uploaded.");
}
else if (_fileContainerHttpClient.IsFastFailResponse(response))
{
// Fast fail: we received an http status code where we should abandon our efforts
context.Output($"Cannot continue uploading files, so draining upload queue of {_fileUploadQueue.Count} items.");
DrainUploadQueue(context);
failedFiles.Clear();
failAndExit = true;
throw new UploadFailedException($"Critical failure uploading '{fileToUpload}'");
}
else
{
context.Debug($"Adding '{fileToUpload}' to retry list.");
failedFiles.Add(fileToUpload);
}
throw new UploadFailedException($"Http failure response '{response?.StatusCode}': '{response?.ReasonPhrase}' while uploading '{fileToUpload}'");
}
uploadTimer.Stop();
context.Debug($"File: '{fileToUpload}' took {uploadTimer.ElapsedMilliseconds} milliseconds to finish upload");
uploadedSize += fs.Length;
OutputLogForFile(context, fileToUpload, $"Detail upload trace for file: {itemPath}", context.Debug);
}
}
catch (OperationCanceledException) when (token.IsCancellationRequested)
{
context.Output($"File upload has been cancelled during upload file: '{fileToUpload}'.");
if (response != null)
{
response.Dispose();
response = null;
}
throw;
}
catch (Exception ex)
{
catchExceptionDuringUpload = true;
context.Output($"Fail to upload '{fileToUpload}' due to '{ex.Message}'.");
context.Output(ex.ToString());
}
uploadTimer.Stop();
if (catchExceptionDuringUpload || (response != null && response.StatusCode != HttpStatusCode.Created))
{
if (response != null)
{
context.Output($"Unable to copy file to server StatusCode={response.StatusCode}: {response.ReasonPhrase}. Source file path: {fileToUpload}. Target server path: {itemPath}");
}
OutputLogForFile(context, fileToUpload, $"Detail upload trace for file that fail to upload: {itemPath}", context.Output);
// output detail upload trace for the file.
ConcurrentQueue<string> logQueue;
if (_fileUploadTraceLog.TryGetValue(itemPath, out logQueue))
if (failAndExit)
{
context.Output($"Detail upload trace for file that fail to upload: {itemPath}");
string message;
while (logQueue.TryDequeue(out message))
{
context.Output(message);
context.Debug("Exiting upload.");
throw;
}
}
// tracking file that failed to upload.
failedFiles.Add(fileToUpload);
}
else
{
context.Debug($"File: '{fileToUpload}' took {uploadTimer.ElapsedMilliseconds} milliseconds to finish upload");
uploadedSize += fs.Length;
// debug detail upload trace for the file.
ConcurrentQueue<string> logQueue;
if (_fileUploadTraceLog.TryGetValue(itemPath, out logQueue))
{
context.Debug($"Detail upload trace for file: {itemPath}");
string message;
while (logQueue.TryDequeue(out message))
{
context.Debug(message);
}
}
}
if (response != null)
{
response.Dispose();
response = null;
}
}
Interlocked.Increment(ref _uploadFilesProcessed);
@@ -590,6 +576,30 @@ namespace GitHub.Runner.Plugins.Artifact
}
}
private void DrainUploadQueue(RunnerActionPluginExecutionContext context)
{
while (_fileUploadQueue.TryDequeue(out string fileToUpload))
{
context.Debug($"Clearing upload queue: '{fileToUpload}'");
Interlocked.Increment(ref _uploadFilesProcessed);
}
}
private void OutputLogForFile(RunnerActionPluginExecutionContext context, string itemPath, string logDescription, Action<string> log)
{
// output detail upload trace for the file.
ConcurrentQueue<string> logQueue;
if (_fileUploadTraceLog.TryGetValue(itemPath, out logQueue))
{
log(logDescription);
string message;
while (logQueue.TryDequeue(out message))
{
log(message);
}
}
}
private void UploadFileTraceReportReceived(object sender, ReportTraceEventArgs e)
{
ConcurrentQueue<string> logQueue = _fileUploadTraceLog.GetOrAdd(e.File, new ConcurrentQueue<string>());
@@ -607,22 +617,22 @@ namespace GitHub.Runner.Plugins.Artifact
{
public UploadResult()
{
FailedFiles = new List<string>();
RetryFiles = new List<string>();
TotalFileSizeUploaded = 0;
}
public UploadResult(List<string> failedFiles, long totalFileSizeUploaded)
public UploadResult(List<string> retryFiles, long totalFileSizeUploaded)
{
FailedFiles = failedFiles;
RetryFiles = retryFiles ?? new List<string>();
TotalFileSizeUploaded = totalFileSizeUploaded;
}
public List<string> FailedFiles { get; set; }
public List<string> RetryFiles { get; set; }
public long TotalFileSizeUploaded { get; set; }
public void AddUploadResult(UploadResult resultToAdd)
{
this.FailedFiles.AddRange(resultToAdd.FailedFiles);
this.RetryFiles.AddRange(resultToAdd.RetryFiles);
this.TotalFileSizeUploaded += resultToAdd.TotalFileSizeUploaded;
}
}
@@ -657,4 +667,19 @@ namespace GitHub.Runner.Plugins.Artifact
this.FailedFiles.AddRange(resultToAdd.FailedFiles);
}
}
public class UploadFailedException : Exception
{
public UploadFailedException()
: base()
{ }
public UploadFailedException(string message)
: base(message)
{ }
public UploadFailedException(string message, Exception inner)
: base(message, inner)
{ }
}
}

View File

@@ -0,0 +1,60 @@
using System;
using System.IO;
using System.Threading;
using System.Threading.Tasks;
using GitHub.Actions.Pipelines.WebApi;
using GitHub.Services.WebApi;
using GitHub.Runner.Sdk;
using Pipelines = GitHub.Actions.Pipelines.WebApi;
namespace GitHub.Runner.Plugins.Artifact
{
// A client wrapper interacting with Pipelines's Artifact API
public class PipelinesServer
{
private readonly PipelinesHttpClient _pipelinesHttpClient;
public PipelinesServer(VssConnection connection)
{
ArgUtil.NotNull(connection, nameof(connection));
_pipelinesHttpClient = connection.GetClient<PipelinesHttpClient>();
}
// Associate the specified Actions Storage artifact with a pipeline
public async Task<Pipelines.ActionsStorageArtifact> AssociateActionsStorageArtifactAsync(
int pipelineId,
int runId,
long containerId,
string name,
long size,
CancellationToken cancellationToken = default(CancellationToken))
{
CreateArtifactParameters parameters = new CreateActionsStorageArtifactParameters()
{
Name = name,
ContainerId = containerId,
Size = size
};
return await _pipelinesHttpClient.CreateArtifactAsync(
parameters,
pipelineId,
runId,
cancellationToken: cancellationToken) as Pipelines.ActionsStorageArtifact;
}
// Get named Actions Storage artifact for a pipeline
public async Task<Pipelines.ActionsStorageArtifact> GetActionsStorageArtifact(
int pipelineId,
int runId,
string name,
CancellationToken cancellationToken)
{
return await _pipelinesHttpClient.GetArtifactAsync(
pipelineId,
runId,
name,
cancellationToken: cancellationToken) as Pipelines.ActionsStorageArtifact;
}
}
}

View File

@@ -68,23 +68,60 @@ namespace GitHub.Runner.Plugins.Artifact
string containerIdStr = context.Variables.GetValueOrDefault(BuildVariables.ContainerId)?.Value ?? string.Empty;
if (!long.TryParse(containerIdStr, out long containerId))
{
throw new ArgumentException($"Container Id is not a Int64: {containerIdStr}");
throw new ArgumentException($"Container Id is not an Int64: {containerIdStr}");
}
context.Output($"Uploading artifact '{artifactName}' from '{fullPath}' for run #{buildId}");
FileContainerServer fileContainerHelper = new FileContainerServer(context.VssConnection, projectId, containerId, artifactName);
long size = await fileContainerHelper.CopyToContainerAsync(context, fullPath, token);
var propertiesDictionary = new Dictionary<string, string>(StringComparer.OrdinalIgnoreCase);
long size = 0;
try
{
size = await fileContainerHelper.CopyToContainerAsync(context, fullPath, token);
propertiesDictionary.Add("artifactsize", size.ToString());
string fileContainerFullPath = StringUtil.Format($"#/{containerId}/{artifactName}");
context.Output($"Uploaded '{fullPath}' to server");
context.Output($"Uploaded '{size}' bytes from '{fullPath}' to server");
}
// if any of the results were successful, make sure to attach them to the build
finally
{
// Determine whether to call Pipelines or Build endpoint to publish artifact based on variable setting
string usePipelinesArtifactEndpointVar = context.Variables.GetValueOrDefault("Runner.UseActionsArtifactsApis")?.Value;
bool.TryParse(usePipelinesArtifactEndpointVar, out bool usePipelinesArtifactEndpoint);
if (usePipelinesArtifactEndpoint)
{
// Definition ID is a dummy value only used by HTTP client routing purposes
int definitionId = 1;
PipelinesServer pipelinesHelper = new PipelinesServer(context.VssConnection);
var artifact = await pipelinesHelper.AssociateActionsStorageArtifactAsync(
definitionId,
buildId,
containerId,
artifactName,
size,
token);
context.Output($"Associated artifact {artifactName} ({artifact.ContainerId}) with run #{buildId}");
context.Debug($"Associated artifact using v2 endpoint");
}
else
{
string fileContainerFullPath = StringUtil.Format($"#/{containerId}/{artifactName}");
BuildServer buildHelper = new BuildServer(context.VssConnection);
string jobId = context.Variables.GetValueOrDefault(WellKnownDistributedTaskVariables.JobId).Value ?? string.Empty;
var artifact = await buildHelper.AssociateArtifact(projectId, buildId, jobId, artifactName, ArtifactResourceTypes.Container, fileContainerFullPath, propertiesDictionary, token);
context.Output($"Associated artifact {artifactName} ({artifact.Id}) with run #{buildId}");
context.Debug($"Associated artifact using v1 endpoint");
}
}
}
}
}

View File

@@ -1,13 +1,14 @@
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFramework>netcoreapp2.2</TargetFramework>
<TargetFramework>netcoreapp3.0</TargetFramework>
<OutputType>Library</OutputType>
<RuntimeIdentifiers>win-x64;win-x86;linux-x64;linux-arm;rhel.6-x64;osx-x64</RuntimeIdentifiers>
<RuntimeIdentifiers>win-x64;win-x86;linux-x64;linux-arm64;linux-arm;osx-x64</RuntimeIdentifiers>
<TargetLatestRuntimePatch>true</TargetLatestRuntimePatch>
<AssetTargetFallback>portable-net45+win8</AssetTargetFallback>
<NoWarn>NU1701;NU1603</NoWarn>
<Version>$(Version)</Version>
<TieredCompilationQuickJit>true</TieredCompilationQuickJit>
</PropertyGroup>
<ItemGroup>
@@ -19,42 +20,4 @@
<DebugType>portable</DebugType>
</PropertyGroup>
<PropertyGroup Condition="'$([System.Runtime.InteropServices.RuntimeInformation]::IsOSPlatform($([System.Runtime.InteropServices.OSPlatform]::Windows)))' == 'true' AND '$(PackageRuntime)' == 'win-x64'">
<DefineConstants>OS_WINDOWS;X64;TRACE</DefineConstants>
</PropertyGroup>
<PropertyGroup Condition="'$([System.Runtime.InteropServices.RuntimeInformation]::IsOSPlatform($([System.Runtime.InteropServices.OSPlatform]::Windows)))' == 'true' AND '$(PackageRuntime)' == 'win-x86'">
<DefineConstants>OS_WINDOWS;X86;TRACE</DefineConstants>
</PropertyGroup>
<PropertyGroup Condition="'$([System.Runtime.InteropServices.RuntimeInformation]::IsOSPlatform($([System.Runtime.InteropServices.OSPlatform]::Windows)))' == 'true' AND '$(Configuration)' == 'Debug' AND '$(PackageRuntime)' == 'win-x64'">
<DefineConstants>OS_WINDOWS;X64;DEBUG;TRACE</DefineConstants>
</PropertyGroup>
<PropertyGroup Condition="'$([System.Runtime.InteropServices.RuntimeInformation]::IsOSPlatform($([System.Runtime.InteropServices.OSPlatform]::Windows)))' == 'true' AND '$(Configuration)' == 'Debug' AND '$(PackageRuntime)' == 'win-x86'">
<DefineConstants>OS_WINDOWS;X86;DEBUG;TRACE</DefineConstants>
</PropertyGroup>
<PropertyGroup Condition="'$([System.Runtime.InteropServices.RuntimeInformation]::IsOSPlatform($([System.Runtime.InteropServices.OSPlatform]::OSX)))' == 'true'">
<DefineConstants>OS_OSX;X64;TRACE</DefineConstants>
</PropertyGroup>
<PropertyGroup Condition="'$([System.Runtime.InteropServices.RuntimeInformation]::IsOSPlatform($([System.Runtime.InteropServices.OSPlatform]::OSX)))' == 'true' AND '$(Configuration)' == 'Debug'">
<DefineConstants>OS_OSX;DEBUG;X64;TRACE</DefineConstants>
</PropertyGroup>
<PropertyGroup Condition="'$([System.Runtime.InteropServices.RuntimeInformation]::IsOSPlatform($([System.Runtime.InteropServices.OSPlatform]::Linux)))' == 'true' AND '$(PackageRuntime)' == 'linux-x64'">
<DefineConstants>OS_LINUX;X64;TRACE</DefineConstants>
</PropertyGroup>
<PropertyGroup Condition="'$([System.Runtime.InteropServices.RuntimeInformation]::IsOSPlatform($([System.Runtime.InteropServices.OSPlatform]::Linux)))' == 'true' AND '$(PackageRuntime)' == 'rhel.6-x64'">
<DefineConstants>OS_LINUX;OS_RHEL6;X64;TRACE</DefineConstants>
</PropertyGroup>
<PropertyGroup Condition="'$([System.Runtime.InteropServices.RuntimeInformation]::IsOSPlatform($([System.Runtime.InteropServices.OSPlatform]::Linux)))' == 'true' AND '$(PackageRuntime)' == 'linux-arm'">
<DefineConstants>OS_LINUX;ARM;TRACE</DefineConstants>
</PropertyGroup>
<PropertyGroup Condition="'$([System.Runtime.InteropServices.RuntimeInformation]::IsOSPlatform($([System.Runtime.InteropServices.OSPlatform]::Linux)))' == 'true' AND '$(Configuration)' == 'Debug' AND '$(PackageRuntime)' == 'linux-x64'">
<DefineConstants>OS_LINUX;X64;DEBUG;TRACE</DefineConstants>
</PropertyGroup>
<PropertyGroup Condition="'$([System.Runtime.InteropServices.RuntimeInformation]::IsOSPlatform($([System.Runtime.InteropServices.OSPlatform]::Linux)))' == 'true' AND '$(Configuration)' == 'Debug' AND '$(PackageRuntime)' == 'rhel.6-x64'">
<DefineConstants>OS_LINUX;OS_RHEL6;X64;DEBUG;TRACE</DefineConstants>
</PropertyGroup>
<PropertyGroup Condition="'$([System.Runtime.InteropServices.RuntimeInformation]::IsOSPlatform($([System.Runtime.InteropServices.OSPlatform]::Linux)))' == 'true' AND '$(Configuration)' == 'Debug' AND '$(PackageRuntime)' == 'linux-arm'">
<DefineConstants>OS_LINUX;ARM;DEBUG;TRACE</DefineConstants>
</PropertyGroup>
</Project>

View File

@@ -70,13 +70,6 @@ namespace GitHub.Runner.Sdk
VssClientHttpRequestSettings.Default.UserAgent = headerValues;
#if OS_LINUX || OS_OSX
// The .NET Core 2.1 runtime switched its HTTP default from HTTP 1.1 to HTTP 2.
// This causes problems with some versions of the Curl handler.
// See GitHub issue https://github.com/dotnet/corefx/issues/32376
VssClientHttpRequestSettings.Default.UseHttp11 = true;
#endif
var certSetting = GetCertConfiguration();
if (certSetting != null)
{

View File

@@ -1,13 +1,14 @@
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFramework>netcoreapp2.2</TargetFramework>
<TargetFramework>netcoreapp3.0</TargetFramework>
<OutputType>Library</OutputType>
<RuntimeIdentifiers>win-x64;win-x86;linux-x64;linux-arm;rhel.6-x64;osx-x64</RuntimeIdentifiers>
<RuntimeIdentifiers>win-x64;win-x86;linux-x64;linux-arm64;linux-arm;osx-x64</RuntimeIdentifiers>
<TargetLatestRuntimePatch>true</TargetLatestRuntimePatch>
<AssetTargetFallback>portable-net45+win8</AssetTargetFallback>
<NoWarn>NU1701;NU1603</NoWarn>
<Version>$(Version)</Version>
<TieredCompilationQuickJit>true</TieredCompilationQuickJit>
</PropertyGroup>
<ItemGroup>
@@ -24,42 +25,4 @@
<DebugType>portable</DebugType>
</PropertyGroup>
<PropertyGroup Condition="'$([System.Runtime.InteropServices.RuntimeInformation]::IsOSPlatform($([System.Runtime.InteropServices.OSPlatform]::Windows)))' == 'true' AND '$(PackageRuntime)' == 'win-x64'">
<DefineConstants>OS_WINDOWS;X64;TRACE</DefineConstants>
</PropertyGroup>
<PropertyGroup Condition="'$([System.Runtime.InteropServices.RuntimeInformation]::IsOSPlatform($([System.Runtime.InteropServices.OSPlatform]::Windows)))' == 'true' AND '$(PackageRuntime)' == 'win-x86'">
<DefineConstants>OS_WINDOWS;X86;TRACE</DefineConstants>
</PropertyGroup>
<PropertyGroup Condition="'$([System.Runtime.InteropServices.RuntimeInformation]::IsOSPlatform($([System.Runtime.InteropServices.OSPlatform]::Windows)))' == 'true' AND '$(Configuration)' == 'Debug' AND '$(PackageRuntime)' == 'win-x64'">
<DefineConstants>OS_WINDOWS;X64;DEBUG;TRACE</DefineConstants>
</PropertyGroup>
<PropertyGroup Condition="'$([System.Runtime.InteropServices.RuntimeInformation]::IsOSPlatform($([System.Runtime.InteropServices.OSPlatform]::Windows)))' == 'true' AND '$(Configuration)' == 'Debug' AND '$(PackageRuntime)' == 'win-x86'">
<DefineConstants>OS_WINDOWS;X86;DEBUG;TRACE</DefineConstants>
</PropertyGroup>
<PropertyGroup Condition="'$([System.Runtime.InteropServices.RuntimeInformation]::IsOSPlatform($([System.Runtime.InteropServices.OSPlatform]::OSX)))' == 'true'">
<DefineConstants>OS_OSX;X64;TRACE</DefineConstants>
</PropertyGroup>
<PropertyGroup Condition="'$([System.Runtime.InteropServices.RuntimeInformation]::IsOSPlatform($([System.Runtime.InteropServices.OSPlatform]::OSX)))' == 'true' AND '$(Configuration)' == 'Debug'">
<DefineConstants>OS_OSX;DEBUG;X64;TRACE</DefineConstants>
</PropertyGroup>
<PropertyGroup Condition="'$([System.Runtime.InteropServices.RuntimeInformation]::IsOSPlatform($([System.Runtime.InteropServices.OSPlatform]::Linux)))' == 'true' AND '$(PackageRuntime)' == 'linux-x64'">
<DefineConstants>OS_LINUX;X64;TRACE</DefineConstants>
</PropertyGroup>
<PropertyGroup Condition="'$([System.Runtime.InteropServices.RuntimeInformation]::IsOSPlatform($([System.Runtime.InteropServices.OSPlatform]::Linux)))' == 'true' AND '$(PackageRuntime)' == 'rhel.6-x64'">
<DefineConstants>OS_LINUX;OS_RHEL6;X64;TRACE</DefineConstants>
</PropertyGroup>
<PropertyGroup Condition="'$([System.Runtime.InteropServices.RuntimeInformation]::IsOSPlatform($([System.Runtime.InteropServices.OSPlatform]::Linux)))' == 'true' AND '$(PackageRuntime)' == 'linux-arm'">
<DefineConstants>OS_LINUX;ARM;TRACE</DefineConstants>
</PropertyGroup>
<PropertyGroup Condition="'$([System.Runtime.InteropServices.RuntimeInformation]::IsOSPlatform($([System.Runtime.InteropServices.OSPlatform]::Linux)))' == 'true' AND '$(Configuration)' == 'Debug' AND '$(PackageRuntime)' == 'linux-x64'">
<DefineConstants>OS_LINUX;X64;DEBUG;TRACE</DefineConstants>
</PropertyGroup>
<PropertyGroup Condition="'$([System.Runtime.InteropServices.RuntimeInformation]::IsOSPlatform($([System.Runtime.InteropServices.OSPlatform]::Linux)))' == 'true' AND '$(Configuration)' == 'Debug' AND '$(PackageRuntime)' == 'rhel.6-x64'">
<DefineConstants>OS_LINUX;OS_RHEL6;X64;DEBUG;TRACE</DefineConstants>
</PropertyGroup>
<PropertyGroup Condition="'$([System.Runtime.InteropServices.RuntimeInformation]::IsOSPlatform($([System.Runtime.InteropServices.OSPlatform]::Linux)))' == 'true' AND '$(Configuration)' == 'Debug' AND '$(PackageRuntime)' == 'linux-arm'">
<DefineConstants>OS_LINUX;ARM;DEBUG;TRACE</DefineConstants>
</PropertyGroup>
</Project>

View File

@@ -122,5 +122,10 @@ namespace GitHub.Runner.Sdk
return format;
}
}
public static string SubstringPrefix(string value, int count)
{
return value?.Substring(0, Math.Min(value.Length, count));
}
}
}

View File

@@ -27,13 +27,6 @@ namespace GitHub.Runner.Sdk
VssClientHttpRequestSettings.Default.UserAgent = headerValues;
VssClientHttpRequestSettings.Default.ClientCertificateManager = clientCert;
#if OS_LINUX || OS_OSX
// The .NET Core 2.1 runtime switched its HTTP default from HTTP 1.1 to HTTP 2.
// This causes problems with some versions of the Curl handler.
// See GitHub issue https://github.com/dotnet/corefx/issues/32376
VssClientHttpRequestSettings.Default.UseHttp11 = true;
#endif
VssHttpMessageHandler.DefaultWebProxy = proxy;
}

View File

@@ -9,9 +9,8 @@
<AppDesignerFolder>Properties</AppDesignerFolder>
<RootNamespace>RunnerService</RootNamespace>
<AssemblyName>RunnerService</AssemblyName>
<SignAssembly>true</SignAssembly>
<AssemblyOriginatorKeyFile>FinalPublicKey.snk</AssemblyOriginatorKeyFile>
<DelaySign>true</DelaySign>
<SignAssembly>false</SignAssembly>
<DelaySign>false</DelaySign>
<TargetFrameworkVersion>v4.5</TargetFrameworkVersion>
<FileAlignment>512</FileAlignment>
<AutoGenerateBindingRedirects>true</AutoGenerateBindingRedirects>
@@ -64,7 +63,6 @@
</ItemGroup>
<ItemGroup>
<None Include="App.config" />
<None Include="FinalPublicKey.snk" />
</ItemGroup>
<ItemGroup>
<EmbeddedResource Include="Resource.resx">

View File

@@ -73,7 +73,7 @@ namespace GitHub.Runner.Worker
return false;
}
// process action command in serialize oreder.
// process action command in serialize order.
lock (_commandSerializeLock)
{
if (_stopProcessCommand)
@@ -107,26 +107,22 @@ namespace GitHub.Runner.Worker
}
else if (_commandExtensions.TryGetValue(actionCommand.Command, out IActionCommandExtension extension))
{
bool omitEcho;
if (context.EchoOnActionCommand && !extension.OmitEcho)
{
context.Output(input);
}
try
{
extension.ProcessCommand(context, input, actionCommand, out omitEcho);
extension.ProcessCommand(context, input, actionCommand);
}
catch (Exception ex)
{
omitEcho = true;
context.Output(input);
context.Error($"Unable to process command '{input}' successfully.");
var commandInformation = extension.OmitEcho ? extension.Command : input;
context.Error($"Unable to process command '{commandInformation}' successfully.");
context.Error(ex);
context.CommandResult = TaskResult.Failed;
}
if (!omitEcho)
{
context.Output(input);
context.Debug($"Processed command");
}
}
else
{
@@ -142,17 +138,19 @@ namespace GitHub.Runner.Worker
public interface IActionCommandExtension : IExtension
{
string Command { get; }
bool OmitEcho { get; }
void ProcessCommand(IExecutionContext context, string line, ActionCommand command, out bool omitEcho);
void ProcessCommand(IExecutionContext context, string line, ActionCommand command);
}
public sealed class InternalPluginSetRepoPathCommandExtension : RunnerService, IActionCommandExtension
{
public string Command => "internal-set-repo-path";
public bool OmitEcho => false;
public Type ExtensionType => typeof(IActionCommandExtension);
public void ProcessCommand(IExecutionContext context, string line, ActionCommand command, out bool omitEcho)
public void ProcessCommand(IExecutionContext context, string line, ActionCommand command)
{
if (!command.Properties.TryGetValue(SetRepoPathCommandProperties.repoFullName, out string repoFullName) || string.IsNullOrEmpty(repoFullName))
{
@@ -166,8 +164,6 @@ namespace GitHub.Runner.Worker
var directoryManager = HostContext.GetService<IPipelineDirectoryManager>();
var trackingConfig = directoryManager.UpdateRepositoryDirectory(context, repoFullName, command.Data, StringUtil.ConvertToBoolean(workspaceRepo));
omitEcho = true;
}
private static class SetRepoPathCommandProperties
@@ -180,10 +176,11 @@ namespace GitHub.Runner.Worker
public sealed class SetEnvCommandExtension : RunnerService, IActionCommandExtension
{
public string Command => "set-env";
public bool OmitEcho => false;
public Type ExtensionType => typeof(IActionCommandExtension);
public void ProcessCommand(IExecutionContext context, string line, ActionCommand command, out bool omitEcho)
public void ProcessCommand(IExecutionContext context, string line, ActionCommand command)
{
if (!command.Properties.TryGetValue(SetEnvCommandProperties.Name, out string envName) || string.IsNullOrEmpty(envName))
{
@@ -192,9 +189,7 @@ namespace GitHub.Runner.Worker
context.EnvironmentVariables[envName] = command.Data;
context.SetEnvContext(envName, command.Data);
context.Output(line);
context.Debug($"{envName}='{command.Data}'");
omitEcho = true;
}
private static class SetEnvCommandProperties
@@ -206,10 +201,11 @@ namespace GitHub.Runner.Worker
public sealed class SetOutputCommandExtension : RunnerService, IActionCommandExtension
{
public string Command => "set-output";
public bool OmitEcho => false;
public Type ExtensionType => typeof(IActionCommandExtension);
public void ProcessCommand(IExecutionContext context, string line, ActionCommand command, out bool omitEcho)
public void ProcessCommand(IExecutionContext context, string line, ActionCommand command)
{
if (!command.Properties.TryGetValue(SetOutputCommandProperties.Name, out string outputName) || string.IsNullOrEmpty(outputName))
{
@@ -217,9 +213,7 @@ namespace GitHub.Runner.Worker
}
context.SetOutput(outputName, command.Data, out var reference);
context.Output(line);
context.Debug($"{reference}='{command.Data}'");
omitEcho = true;
}
private static class SetOutputCommandProperties
@@ -231,10 +225,11 @@ namespace GitHub.Runner.Worker
public sealed class SaveStateCommandExtension : RunnerService, IActionCommandExtension
{
public string Command => "save-state";
public bool OmitEcho => false;
public Type ExtensionType => typeof(IActionCommandExtension);
public void ProcessCommand(IExecutionContext context, string line, ActionCommand command, out bool omitEcho)
public void ProcessCommand(IExecutionContext context, string line, ActionCommand command)
{
if (!command.Properties.TryGetValue(SaveStateCommandProperties.Name, out string stateName) || string.IsNullOrEmpty(stateName))
{
@@ -243,7 +238,6 @@ namespace GitHub.Runner.Worker
context.IntraActionState[stateName] = command.Data;
context.Debug($"Save intra-action state {stateName} = {command.Data}");
omitEcho = true;
}
private static class SaveStateCommandProperties
@@ -255,49 +249,53 @@ namespace GitHub.Runner.Worker
public sealed class AddMaskCommandExtension : RunnerService, IActionCommandExtension
{
public string Command => "add-mask";
public bool OmitEcho => true;
public Type ExtensionType => typeof(IActionCommandExtension);
public void ProcessCommand(IExecutionContext context, string line, ActionCommand command, out bool omitEcho)
public void ProcessCommand(IExecutionContext context, string line, ActionCommand command)
{
if (string.IsNullOrWhiteSpace(command.Data))
{
context.Warning("Can't add secret mask for empty string.");
context.Warning("Can't add secret mask for empty string in ##[add-mask] command.");
}
else
{
if (context.EchoOnActionCommand)
{
context.Output($"::{Command}::***");
}
HostContext.SecretMasker.AddValue(command.Data);
Trace.Info($"Add new secret mask with length of {command.Data.Length}");
}
omitEcho = true;
}
}
public sealed class AddPathCommandExtension : RunnerService, IActionCommandExtension
{
public string Command => "add-path";
public bool OmitEcho => false;
public Type ExtensionType => typeof(IActionCommandExtension);
public void ProcessCommand(IExecutionContext context, string line, ActionCommand command, out bool omitEcho)
public void ProcessCommand(IExecutionContext context, string line, ActionCommand command)
{
ArgUtil.NotNullOrEmpty(command.Data, "path");
context.PrependPath.RemoveAll(x => string.Equals(x, command.Data, StringComparison.CurrentCulture));
context.PrependPath.Add(command.Data);
omitEcho = false;
}
}
public sealed class AddMatcherCommandExtension : RunnerService, IActionCommandExtension
{
public string Command => "add-matcher";
public bool OmitEcho => false;
public Type ExtensionType => typeof(IActionCommandExtension);
public void ProcessCommand(IExecutionContext context, string line, ActionCommand command, out bool omitEcho)
public void ProcessCommand(IExecutionContext context, string line, ActionCommand command)
{
omitEcho = false;
var file = command.Data;
// File is required
@@ -339,26 +337,26 @@ namespace GitHub.Runner.Worker
public sealed class RemoveMatcherCommandExtension : RunnerService, IActionCommandExtension
{
public string Command => "remove-matcher";
public bool OmitEcho => false;
public Type ExtensionType => typeof(IActionCommandExtension);
public void ProcessCommand(IExecutionContext context, string line, ActionCommand command, out bool omitEcho)
public void ProcessCommand(IExecutionContext context, string line, ActionCommand command)
{
omitEcho = false;
command.Properties.TryGetValue(RemoveMatcherCommandProperties.Owner, out string owner);
var file = command.Data;
// Owner and file are mutually exclusive
if (!string.IsNullOrEmpty(owner) && !string.IsNullOrEmpty(file))
{
context.Warning("Either specify a matcher owner name or a file path. Both values cannot be set.");
context.Warning("Either specify an owner name or a file path in ##[remove-matcher] command. Both values cannot be set.");
return;
}
// Owner or file is required
if (string.IsNullOrEmpty(owner) && string.IsNullOrEmpty(file))
{
context.Warning("Either a matcher owner name or a file path must be specified.");
context.Warning("Either an owner name or a file path must be specified in ##[remove-matcher] command.");
return;
}
@@ -407,12 +405,12 @@ namespace GitHub.Runner.Worker
public sealed class DebugCommandExtension : RunnerService, IActionCommandExtension
{
public string Command => "debug";
public bool OmitEcho => true;
public Type ExtensionType => typeof(IActionCommandExtension);
public void ProcessCommand(IExecutionContext context, string inputLine, ActionCommand command, out bool omitEcho)
public void ProcessCommand(IExecutionContext context, string inputLine, ActionCommand command)
{
omitEcho = true;
context.Debug(command.Data);
}
}
@@ -435,12 +433,12 @@ namespace GitHub.Runner.Worker
{
public abstract IssueType Type { get; }
public abstract string Command { get; }
public bool OmitEcho => true;
public Type ExtensionType => typeof(IActionCommandExtension);
public void ProcessCommand(IExecutionContext context, string inputLine, ActionCommand command, out bool omitEcho)
public void ProcessCommand(IExecutionContext context, string inputLine, ActionCommand command)
{
omitEcho = true;
command.Properties.TryGetValue(IssueCommandProperties.File, out string file);
command.Properties.TryGetValue(IssueCommandProperties.Line, out string line);
command.Properties.TryGetValue(IssueCommandProperties.Column, out string column);
@@ -515,13 +513,41 @@ namespace GitHub.Runner.Worker
public abstract class GroupingCommandExtension : RunnerService, IActionCommandExtension
{
public abstract string Command { get; }
public bool OmitEcho => false;
public Type ExtensionType => typeof(IActionCommandExtension);
public void ProcessCommand(IExecutionContext context, string line, ActionCommand command, out bool omitEcho)
public void ProcessCommand(IExecutionContext context, string line, ActionCommand command)
{
var data = this is GroupCommandExtension ? command.Data : string.Empty;
context.Output($"##[{Command}]{data}");
omitEcho = true;
}
}
public sealed class EchoCommandExtension : RunnerService, IActionCommandExtension
{
public string Command => "echo";
public bool OmitEcho => false;
public Type ExtensionType => typeof(IActionCommandExtension);
public void ProcessCommand(IExecutionContext context, string line, ActionCommand command)
{
ArgUtil.NotNullOrEmpty(command.Data, "value");
switch (command.Data.Trim().ToUpperInvariant())
{
case "ON":
context.EchoOnActionCommand = true;
context.Debug("Setting echo command value to 'on'");
break;
case "OFF":
context.EchoOnActionCommand = false;
context.Debug("Setting echo command value to 'off'");
break;
default:
throw new Exception($"Invalid echo command value. Possible values can be: 'on', 'off'. Current value is: '{command.Data}'.");
}
}
}
}

View File

@@ -58,6 +58,9 @@ namespace GitHub.Runner.Worker
executionContext.Warning("The 'PREVIEW_ACTION_TOKEN' secret is depreciated. Please remove it from the repository's secrets");
}
// Clear the cache (local runner)
IOUtil.DeleteDirectory(HostContext.GetDirectory(WellKnownDirectory.Actions), executionContext.CancellationToken);
foreach (var action in actions)
{
if (action.Reference.Type == Pipelines.ActionSourceType.ContainerRegistry)
@@ -445,7 +448,7 @@ namespace GitHub.Runner.Worker
}
else
{
// make sure we get an clean folder ready to use.
// make sure we get a clean folder ready to use.
IOUtil.DeleteDirectory(destDirectory, executionContext.CancellationToken);
Directory.CreateDirectory(destDirectory);
executionContext.Output($"Download action repository '{repositoryReference.Name}@{repositoryReference.Ref}'");

View File

@@ -94,7 +94,21 @@ namespace GitHub.Runner.Worker
{
postDisplayName = $"Post {this.DisplayName}";
}
ExecutionContext.RegisterPostJobAction(postDisplayName, handlerData.CleanupCondition, Action);
var repositoryReference = Action.Reference as RepositoryPathReference;
var pathString = string.IsNullOrEmpty(repositoryReference.Path) ? string.Empty : $"/{repositoryReference.Path}";
var repoString = string.IsNullOrEmpty(repositoryReference.Ref) ? $"{repositoryReference.Name}{pathString}" :
$"{repositoryReference.Name}{pathString}@{repositoryReference.Ref}";
ExecutionContext.Debug($"Register post job cleanup for action: {repoString}");
var actionRunner = HostContext.CreateService<IActionRunner>();
actionRunner.Action = Action;
actionRunner.Stage = ActionRunStage.Post;
actionRunner.Condition = handlerData.CleanupCondition;
actionRunner.DisplayName = postDisplayName;
ExecutionContext.RegisterPostJobStep($"{actionRunner.Action.Name}_post", actionRunner);
}
IStepHost stepHost = HostContext.CreateService<IDefaultStepHost>();

View File

@@ -276,7 +276,9 @@ namespace GitHub.Runner.Worker.Container
return await ExecuteDockerCommandAsync(context, "exec", $"{options} {containerId} {command}", context.CancellationToken);
}
#pragma warning disable CS1998 // Async method lacks 'await' operators and will run synchronously (method has async logic on only certain platforms)
public async Task<int> DockerExec(IExecutionContext context, string containerId, string options, string command, List<string> output)
#pragma warning restore CS1998 // Async method lacks 'await' operators and will run synchronously
{
ArgUtil.NotNull(output, nameof(output));
@@ -337,7 +339,9 @@ namespace GitHub.Runner.Worker.Container
return ExecuteDockerCommandAsync(context, command, options, null, cancellationToken);
}
#pragma warning disable CS1998 // Async method lacks 'await' operators and will run synchronously (method has async logic on only certain platforms)
private async Task<int> ExecuteDockerCommandAsync(IExecutionContext context, string command, string options, IDictionary<string, string> environment, EventHandler<ProcessDataReceivedEventArgs> stdoutDataReceived, EventHandler<ProcessDataReceivedEventArgs> stderrDataReceived, CancellationToken cancellationToken = default(CancellationToken))
#pragma warning restore CS1998 // Async method lacks 'await' operators and will run synchronously
{
string arg = $"{command} {options}".Trim();
context.Command($"{DockerPath} {arg}");
@@ -362,7 +366,9 @@ namespace GitHub.Runner.Worker.Container
#endif
}
#pragma warning disable CS1998 // Async method lacks 'await' operators and will run synchronously (method has async logic on only certain platforms)
private async Task<int> ExecuteDockerCommandAsync(IExecutionContext context, string command, string options, string workingDirectory, CancellationToken cancellationToken = default(CancellationToken))
#pragma warning restore CS1998 // Async method lacks 'await' operators and will run synchronously
{
string arg = $"{command} {options}".Trim();
context.Command($"{DockerPath} {arg}");

View File

@@ -11,6 +11,7 @@ using GitHub.Runner.Common;
using GitHub.Runner.Sdk;
using GitHub.DistributedTask.Pipelines.ContextData;
using Microsoft.Win32;
using GitHub.DistributedTask.Pipelines.ObjectTemplating;
namespace GitHub.Runner.Worker
{
@@ -38,6 +39,14 @@ namespace GitHub.Runner.Worker
List<ContainerInfo> containers = data as List<ContainerInfo>;
ArgUtil.NotNull(containers, nameof(containers));
var postJobStep = new JobExtensionRunner(runAsync: this.StopContainersAsync,
condition: $"{PipelineTemplateConstants.Always}()",
displayName: "Stop containers",
data: data);
executionContext.Debug($"Register post job cleanup for stoping/deleting containers.");
executionContext.RegisterPostJobStep(nameof(StopContainersAsync), postJobStep);
// Check whether we are inside a container.
// Our container feature requires to map working directory from host to the container.
// If we are already inside a container, we will not able to find out the real working direcotry path on the host.
@@ -48,9 +57,6 @@ namespace GitHub.Runner.Worker
{
throw new NotSupportedException("Container feature is not supported when runner is already running inside container.");
}
#elif OS_RHEL6
// Red Hat and CentOS 6 do not support the container feature
throw new NotSupportedException("Runner does not support the container feature on Red Hat Enterprise Linux 6 or CentOS 6.");
#else
var initProcessCgroup = File.ReadLines("/proc/1/cgroup");
if (initProcessCgroup.Any(x => x.IndexOf(":/docker/", StringComparison.OrdinalIgnoreCase) >= 0))

View File

@@ -2,19 +2,9 @@
using System.Collections.Generic;
using System.IO;
using System.IO.Compression;
using System.Text;
using System.Runtime.InteropServices;
using GitHub.DistributedTask.WebApi;
using GitHub.Runner.Common.Util;
using GitHub.Runner.Worker;
using GitHub.Runner.Common.Capabilities;
using GitHub.Services.WebApi;
using Microsoft.Win32;
using System.Diagnostics;
using System.Linq;
using System.Collections.ObjectModel;
using System.Globalization;
using System.Threading;
using System.Threading.Tasks;
using Pipelines = GitHub.DistributedTask.Pipelines;
using GitHub.Runner.Common;
@@ -41,7 +31,9 @@ namespace GitHub.Runner.Worker
public sealed class DiagnosticLogManager : RunnerService, IDiagnosticLogManager
{
private static string DateTimeFormat = "yyyyMMdd-HHmmss";
#pragma warning disable CS1998 // Async method lacks 'await' operators and will run synchronously (method has async logic on only certain platforms)
public async Task UploadDiagnosticLogsAsync(IExecutionContext executionContext,
#pragma warning restore CS1998 // Async method lacks 'await' operators and will run synchronously
IExecutionContext parentContext,
Pipelines.AgentJobRequestMessage message,
DateTime jobStartTimeUtc)

View File

@@ -12,14 +12,14 @@ using GitHub.Services.WebApi;
using GitHub.DistributedTask.Pipelines;
using GitHub.DistributedTask.Pipelines.ContextData;
using GitHub.DistributedTask.WebApi;
using Pipelines = GitHub.DistributedTask.Pipelines;
using ObjectTemplating = GitHub.DistributedTask.ObjectTemplating;
using GitHub.Runner.Common.Util;
using GitHub.Runner.Common;
using GitHub.Runner.Sdk;
using Newtonsoft.Json;
using System.Text;
using System.Collections;
using ObjectTemplating = GitHub.DistributedTask.ObjectTemplating;
using Pipelines = GitHub.DistributedTask.Pipelines;
namespace GitHub.Runner.Worker
{
@@ -62,6 +62,8 @@ namespace GitHub.Runner.Worker
// Only job level ExecutionContext has PostJobSteps
Stack<IStep> PostJobSteps { get; }
bool EchoOnActionCommand { get; set; }
// Initialize
void InitializeJob(Pipelines.AgentJobRequestMessage message, CancellationToken token);
void CancelToken();
@@ -96,7 +98,7 @@ namespace GitHub.Runner.Worker
// others
void ForceTaskComplete();
void RegisterPostJobAction(string displayName, string condition, Pipelines.ActionStep action);
void RegisterPostJobStep(string refName, IStep step);
}
public sealed class ExecutionContext : RunnerService, IExecutionContext
@@ -153,6 +155,8 @@ namespace GitHub.Runner.Worker
// Only job level ExecutionContext has PostJobSteps
public Stack<IStep> PostJobSteps { get; private set; }
public bool EchoOnActionCommand { get; set; }
public TaskResult? Result
{
@@ -236,27 +240,10 @@ namespace GitHub.Runner.Worker
});
}
public void RegisterPostJobAction(string displayName, string condition, Pipelines.ActionStep action)
public void RegisterPostJobStep(string refName, IStep step)
{
if (action.Reference.Type != ActionSourceType.Repository)
{
throw new NotSupportedException("Only action that has `action.yml` can define post job execution.");
}
var repositoryReference = action.Reference as RepositoryPathReference;
var pathString = string.IsNullOrEmpty(repositoryReference.Path) ? string.Empty : $"/{repositoryReference.Path}";
var repoString = string.IsNullOrEmpty(repositoryReference.Ref) ? $"{repositoryReference.Name}{pathString}" :
$"{repositoryReference.Name}{pathString}@{repositoryReference.Ref}";
this.Debug($"Register post job cleanup for action: {repoString}");
var actionRunner = HostContext.CreateService<IActionRunner>();
actionRunner.Action = action;
actionRunner.Stage = ActionRunStage.Post;
actionRunner.Condition = condition;
actionRunner.DisplayName = displayName;
actionRunner.ExecutionContext = Root.CreatePostChild(displayName, $"{actionRunner.Action.Name}_post", IntraActionState);
Root.PostJobSteps.Push(actionRunner);
step.ExecutionContext = Root.CreatePostChild(step.DisplayName, refName, IntraActionState);
Root.PostJobSteps.Push(step);
}
public IExecutionContext CreateChild(Guid recordId, string displayName, string refName, string scopeName, string contextName, Dictionary<string, string> intraActionState = null, int? recordOrder = null)
@@ -292,6 +279,7 @@ namespace GitHub.Runner.Worker
child.PrependPath = PrependPath;
child.Container = Container;
child.ServiceContainers = ServiceContainers;
child.EchoOnActionCommand = EchoOnActionCommand;
if (recordOrder != null)
{
@@ -704,6 +692,9 @@ namespace GitHub.Runner.Worker
_logger = HostContext.CreateService<IPagingLogger>();
_logger.Setup(_mainTimelineId, _record.Id);
// Initialize 'echo on action command success' property, default to false, unless Step_Debug is set
EchoOnActionCommand = Variables.Step_Debug ?? false;
// Verbosity (from GitHub.Step_Debug).
WriteDebug = Variables.Step_Debug ?? false;

View File

@@ -22,7 +22,9 @@ namespace GitHub.Runner.Worker.Handlers
{
public ContainerActionExecutionData Data { get; set; }
#pragma warning disable CS1998 // Async method lacks 'await' operators and will run synchronously (method has async logic on only certain platforms)
public async Task RunAsync(ActionRunStage stage)
#pragma warning restore CS1998 // Async method lacks 'await' operators and will run synchronously
{
// Validate args.
Trace.Entering();

View File

@@ -263,15 +263,17 @@ namespace GitHub.Runner.Worker.Handlers
// Root using fromPath
if (!string.IsNullOrWhiteSpace(match.FromPath) && !Path.IsPathRooted(file))
{
file = Path.Combine(match.FromPath, file);
var fromDirectory = Path.GetDirectoryName(match.FromPath);
if (!string.IsNullOrWhiteSpace(fromDirectory))
{
file = Path.Combine(fromDirectory, file);
}
}
// Root using system.defaultWorkingDirectory
// Root using workspace
if (!Path.IsPathRooted(file))
{
var githubContext = _executionContext.ExpressionValues["github"] as GitHubContext;
ArgUtil.NotNull(githubContext, nameof(githubContext));
var workspace = githubContext["workspace"].ToString();
var workspace = _executionContext.GetGitHubContext("workspace");
ArgUtil.NotNullOrEmpty(workspace, "workspace");
file = Path.Combine(workspace, file);
@@ -297,7 +299,7 @@ namespace GitHub.Runner.Worker.Handlers
}
else
{
// prefer `/` on all platforms
// Prefer `/` on all platforms
issue.Data["file"] = file.Substring(repositoryPath.Length).TrimStart(Path.DirectorySeparatorChar).Replace(Path.DirectorySeparatorChar, Path.AltDirectorySeparatorChar);
}
}

View File

@@ -60,10 +60,16 @@ namespace GitHub.Runner.Worker.Handlers
if (string.IsNullOrEmpty(shell))
{
#if OS_WINDOWS
shellCommand = "cmd";
shellCommand = "pwsh";
if(validateShellOnHost)
{
shellCommandPath = System.Environment.GetEnvironmentVariable("ComSpec");
shellCommandPath = WhichUtil.Which(shellCommand, require: false, Trace);
if (string.IsNullOrEmpty(shellCommandPath))
{
shellCommand = "powershell";
Trace.Info($"Defaulting to {shellCommand}");
shellCommandPath = WhichUtil.Which(shellCommand, require: true, Trace);
}
}
#else
shellCommand = "sh";
@@ -143,9 +149,15 @@ namespace GitHub.Runner.Worker.Handlers
if (string.IsNullOrEmpty(shell))
{
#if OS_WINDOWS
shellCommand = "cmd";
commandPath = System.Environment.GetEnvironmentVariable("ComSpec");
ArgUtil.NotNullOrEmpty(commandPath, "%ComSpec%");
shellCommand = "pwsh";
commandPath = WhichUtil.Which(shellCommand, require: false, Trace);
if (string.IsNullOrEmpty(commandPath))
{
shellCommand = "powershell";
Trace.Info($"Defaulting to {shellCommand}");
commandPath = WhichUtil.Which(shellCommand, require: true, Trace);
}
ArgUtil.NotNullOrEmpty(commandPath, "Default Shell");
#else
shellCommand = "sh";
commandPath = WhichUtil.Which("bash", false, Trace) ?? WhichUtil.Which("sh", true, Trace);

View File

@@ -20,6 +20,7 @@ namespace GitHub.Runner.Worker
public sealed class IssueMatcher
{
private string _defaultSeverity;
private string _owner;
private IssuePattern[] _patterns;
private IssueMatch[] _state;
@@ -27,6 +28,7 @@ namespace GitHub.Runner.Worker
public IssueMatcher(IssueMatcherConfig config, TimeSpan timeout)
{
_owner = config.Owner;
_defaultSeverity = config.Severity;
_patterns = config.Patterns.Select(x => new IssuePattern(x , timeout)).ToArray();
Reset();
}
@@ -37,13 +39,26 @@ namespace GitHub.Runner.Worker
{
if (_owner == null)
{
_owner = String.Empty;
_owner = string.Empty;
}
return _owner;
}
}
public string DefaultSeverity
{
get
{
if (_defaultSeverity == null)
{
_defaultSeverity = string.Empty;
}
return _defaultSeverity;
}
}
public IssueMatch Match(string line)
{
// Single pattern
@@ -54,7 +69,7 @@ namespace GitHub.Runner.Worker
if (regexMatch.Success)
{
return new IssueMatch(null, pattern, regexMatch.Groups);
return new IssueMatch(null, pattern, regexMatch.Groups, DefaultSeverity);
}
return null;
@@ -95,7 +110,7 @@ namespace GitHub.Runner.Worker
}
// Return
return new IssueMatch(runningMatch, pattern, regexMatch.Groups);
return new IssueMatch(runningMatch, pattern, regexMatch.Groups, DefaultSeverity);
}
// Not the last pattern
else
@@ -169,7 +184,7 @@ namespace GitHub.Runner.Worker
public sealed class IssueMatch
{
public IssueMatch(IssueMatch runningMatch, IssuePattern pattern, GroupCollection groups)
public IssueMatch(IssueMatch runningMatch, IssuePattern pattern, GroupCollection groups, string defaultSeverity = null)
{
File = runningMatch?.File ?? GetValue(groups, pattern.File);
Line = runningMatch?.Line ?? GetValue(groups, pattern.Line);
@@ -178,6 +193,11 @@ namespace GitHub.Runner.Worker
Code = runningMatch?.Code ?? GetValue(groups, pattern.Code);
Message = runningMatch?.Message ?? GetValue(groups, pattern.Message);
FromPath = runningMatch?.FromPath ?? GetValue(groups, pattern.FromPath);
if (string.IsNullOrEmpty(Severity) && !string.IsNullOrEmpty(defaultSeverity))
{
Severity = defaultSeverity;
}
}
public string File { get; }
@@ -256,6 +276,9 @@ namespace GitHub.Runner.Worker
[DataMember(Name = "owner")]
private string _owner;
[DataMember(Name = "severity")]
private string _severity;
[DataMember(Name = "pattern")]
private IssuePatternConfig[] _patterns;
@@ -265,7 +288,7 @@ namespace GitHub.Runner.Worker
{
if (_owner == null)
{
_owner = String.Empty;
_owner = string.Empty;
}
return _owner;
@@ -277,6 +300,24 @@ namespace GitHub.Runner.Worker
}
}
public string Severity
{
get
{
if (_severity == null)
{
_severity = string.Empty;
}
return _severity;
}
set
{
_severity = value;
}
}
public IssuePatternConfig[] Patterns
{
get
@@ -303,6 +344,17 @@ namespace GitHub.Runner.Worker
throw new ArgumentException("Owner must not be empty");
}
// Validate severity
switch ((_severity ?? string.Empty).ToUpperInvariant())
{
case "":
case "ERROR":
case "WARNING":
break;
default:
throw new ArgumentException($"Matcher '{_owner}' contains unexpected default severity '{_severity}'");
}
// Validate at least one pattern
if (_patterns == null || _patterns.Length == 0)
{

View File

@@ -110,9 +110,7 @@ namespace GitHub.Runner.Worker
}
}
// Build up 3 lists of steps, pre-job, job, post-job
var postJobStepsBuilder = new Stack<IStep>();
// Build up 2 lists of steps, pre-job, job
// Download actions not already in the cache
Trace.Info("Downloading actions");
var actionManager = HostContext.GetService<IActionManager>();
@@ -134,10 +132,6 @@ namespace GitHub.Runner.Worker
condition: $"{PipelineTemplateConstants.Success}()",
displayName: "Initialize containers",
data: (object)containers));
postJobStepsBuilder.Push(new JobExtensionRunner(runAsync: containerProvider.StopContainersAsync,
condition: $"{PipelineTemplateConstants.Always}()",
displayName: "Stop containers",
data: (object)containers));
}
// Add action steps
@@ -187,33 +181,9 @@ namespace GitHub.Runner.Worker
}
}
// Add post-job steps
Trace.Info("Adding post-job steps");
while (postJobStepsBuilder.Count > 0)
{
postJobSteps.Add(postJobStepsBuilder.Pop());
}
// Create execution context for post-job steps
foreach (var step in postJobSteps)
{
if (step is JobExtensionRunner)
{
JobExtensionRunner extensionStep = step as JobExtensionRunner;
ArgUtil.NotNull(extensionStep, extensionStep.DisplayName);
Guid stepId = Guid.NewGuid();
extensionStep.ExecutionContext = jobContext.CreateChild(stepId, extensionStep.DisplayName, stepId.ToString("N"), null, null);
}
}
List<IStep> steps = new List<IStep>();
steps.AddRange(preJobSteps);
steps.AddRange(jobSteps);
steps.AddRange(postJobSteps);
// Start agent log plugin host process
// var logPlugin = HostContext.GetService<IAgentLogPlugin>();
// await logPlugin.StartAsync(context, steps, jobContext.CancellationToken);
// Prepare for orphan process cleanup
_processCleanup = jobContext.Variables.GetBoolean("process.clean") ?? true;

View File

@@ -1,13 +1,15 @@
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFramework>netcoreapp2.2</TargetFramework>
<TargetFramework>netcoreapp3.0</TargetFramework>
<OutputType>Exe</OutputType>
<RuntimeIdentifiers>win-x64;win-x86;linux-x64;linux-arm;rhel.6-x64;osx-x64</RuntimeIdentifiers>
<RuntimeIdentifiers>win-x64;win-x86;linux-x64;linux-arm64;linux-arm;osx-x64</RuntimeIdentifiers>
<TargetLatestRuntimePatch>true</TargetLatestRuntimePatch>
<AssetTargetFallback>portable-net45+win8</AssetTargetFallback>
<NoWarn>NU1701;NU1603</NoWarn>
<Version>$(Version)</Version>
<TieredCompilationQuickJit>true</TieredCompilationQuickJit>
<PublishReadyToRun>true</PublishReadyToRun>
</PropertyGroup>
<ItemGroup>
@@ -33,42 +35,4 @@
<DebugType>portable</DebugType>
</PropertyGroup>
<PropertyGroup Condition="'$([System.Runtime.InteropServices.RuntimeInformation]::IsOSPlatform($([System.Runtime.InteropServices.OSPlatform]::Windows)))' == 'true' AND '$(PackageRuntime)' == 'win-x64'">
<DefineConstants>OS_WINDOWS;X64;TRACE</DefineConstants>
</PropertyGroup>
<PropertyGroup Condition="'$([System.Runtime.InteropServices.RuntimeInformation]::IsOSPlatform($([System.Runtime.InteropServices.OSPlatform]::Windows)))' == 'true' AND '$(PackageRuntime)' == 'win-x86'">
<DefineConstants>OS_WINDOWS;X86;TRACE</DefineConstants>
</PropertyGroup>
<PropertyGroup Condition="'$([System.Runtime.InteropServices.RuntimeInformation]::IsOSPlatform($([System.Runtime.InteropServices.OSPlatform]::Windows)))' == 'true' AND '$(Configuration)' == 'Debug' AND '$(PackageRuntime)' == 'win-x64'">
<DefineConstants>OS_WINDOWS;X64;DEBUG;TRACE</DefineConstants>
</PropertyGroup>
<PropertyGroup Condition="'$([System.Runtime.InteropServices.RuntimeInformation]::IsOSPlatform($([System.Runtime.InteropServices.OSPlatform]::Windows)))' == 'true' AND '$(Configuration)' == 'Debug' AND '$(PackageRuntime)' == 'win-x86'">
<DefineConstants>OS_WINDOWS;X86;DEBUG;TRACE</DefineConstants>
</PropertyGroup>
<PropertyGroup Condition="'$([System.Runtime.InteropServices.RuntimeInformation]::IsOSPlatform($([System.Runtime.InteropServices.OSPlatform]::OSX)))' == 'true'">
<DefineConstants>OS_OSX;X64;TRACE</DefineConstants>
</PropertyGroup>
<PropertyGroup Condition="'$([System.Runtime.InteropServices.RuntimeInformation]::IsOSPlatform($([System.Runtime.InteropServices.OSPlatform]::OSX)))' == 'true' AND '$(Configuration)' == 'Debug'">
<DefineConstants>OS_OSX;DEBUG;X64;TRACE</DefineConstants>
</PropertyGroup>
<PropertyGroup Condition="'$([System.Runtime.InteropServices.RuntimeInformation]::IsOSPlatform($([System.Runtime.InteropServices.OSPlatform]::Linux)))' == 'true' AND '$(PackageRuntime)' == 'linux-x64'">
<DefineConstants>OS_LINUX;X64;TRACE</DefineConstants>
</PropertyGroup>
<PropertyGroup Condition="'$([System.Runtime.InteropServices.RuntimeInformation]::IsOSPlatform($([System.Runtime.InteropServices.OSPlatform]::Linux)))' == 'true' AND '$(PackageRuntime)' == 'rhel.6-x64'">
<DefineConstants>OS_LINUX;OS_RHEL6;X64;TRACE</DefineConstants>
</PropertyGroup>
<PropertyGroup Condition="'$([System.Runtime.InteropServices.RuntimeInformation]::IsOSPlatform($([System.Runtime.InteropServices.OSPlatform]::Linux)))' == 'true' AND '$(PackageRuntime)' == 'linux-arm'">
<DefineConstants>OS_LINUX;ARM;TRACE</DefineConstants>
</PropertyGroup>
<PropertyGroup Condition="'$([System.Runtime.InteropServices.RuntimeInformation]::IsOSPlatform($([System.Runtime.InteropServices.OSPlatform]::Linux)))' == 'true' AND '$(Configuration)' == 'Debug' AND '$(PackageRuntime)' == 'linux-x64'">
<DefineConstants>OS_LINUX;X64;DEBUG;TRACE</DefineConstants>
</PropertyGroup>
<PropertyGroup Condition="'$([System.Runtime.InteropServices.RuntimeInformation]::IsOSPlatform($([System.Runtime.InteropServices.OSPlatform]::Linux)))' == 'true' AND '$(Configuration)' == 'Debug' AND '$(PackageRuntime)' == 'rhel.6-x64'">
<DefineConstants>OS_LINUX;OS_RHEL6;X64;DEBUG;TRACE</DefineConstants>
</PropertyGroup>
<PropertyGroup Condition="'$([System.Runtime.InteropServices.RuntimeInformation]::IsOSPlatform($([System.Runtime.InteropServices.OSPlatform]::Linux)))' == 'true' AND '$(Configuration)' == 'Debug' AND '$(PackageRuntime)' == 'linux-arm'">
<DefineConstants>OS_LINUX;ARM;DEBUG;TRACE</DefineConstants>
</PropertyGroup>
</Project>

View File

@@ -56,7 +56,7 @@ namespace GitHub.DistributedTask.Expressions2
// Attempt to match a named-value or index operator.
// Note, do not push children of the index operator.
if (node is NamedValue || node is Index)
if (node is NamedValue || node is Sdk.Operators.Index)
{
// Lazy initialize the pattern segments
if (segmentedPatterns is null)
@@ -201,7 +201,7 @@ namespace GitHub.DistributedTask.Expressions2
result.Push(node);
}
// Node is an index
else if (node is Index index)
else if (node is Sdk.Operators.Index index)
{
while (true)
{
@@ -218,7 +218,7 @@ namespace GitHub.DistributedTask.Expressions2
break;
}
// Parameter 0 is an index
else if (parameter0 is Index index2)
else if (parameter0 is Sdk.Operators.Index index2)
{
index = index2;
}

View File

@@ -5,6 +5,7 @@ using System.Text;
using Minimatch;
using System.IO;
using System.Security.Cryptography;
using GitHub.DistributedTask.Expressions2.Sdk;
using GitHub.DistributedTask.Pipelines.ContextData;
using GitHub.DistributedTask.Pipelines.ObjectTemplating;
namespace GitHub.DistributedTask.Expressions2.Sdk.Functions
@@ -28,29 +29,50 @@ namespace GitHub.DistributedTask.Expressions2.Sdk.Functions
string searchRoot = workspaceData.Value;
string pattern = Parameters[0].Evaluate(context).ConvertToString();
// Convert slashes on Windows
if (s_isWindows)
{
pattern = pattern.Replace('\\', '/');
}
// Root the pattern
if (!Path.IsPathRooted(pattern))
{
var patternRoot = s_isWindows ? searchRoot.Replace('\\', '/').TrimEnd('/') : searchRoot.TrimEnd('/');
pattern = string.Concat(patternRoot, "/", pattern);
}
// Get all files
context.Trace.Info($"Search root directory: '{searchRoot}'");
context.Trace.Info($"Search pattern: '{pattern}'");
var files = Directory.GetFiles(searchRoot, "*", SearchOption.AllDirectories).OrderBy(x => x).ToList();
var files = Directory.GetFiles(searchRoot, "*", SearchOption.AllDirectories)
.Select(x => s_isWindows ? x.Replace('\\', '/') : x)
.OrderBy(x => x, StringComparer.Ordinal)
.ToList();
if (files.Count == 0)
{
throw new ArgumentException($"'hashFiles({pattern})' failed. Directory '{searchRoot}' is empty");
throw new ArgumentException($"hashFiles('{ExpressionUtility.StringEscape(pattern)}') failed. Directory '{searchRoot}' is empty");
}
else
{
context.Trace.Info($"Found {files.Count} files");
}
// Match
var matcher = new Minimatcher(pattern, s_minimatchOptions);
files = matcher.Filter(files).ToList();
files = matcher.Filter(files)
.Select(x => s_isWindows ? x.Replace('/', '\\') : x)
.ToList();
if (files.Count == 0)
{
throw new ArgumentException($"'hashFiles({pattern})' failed. Search pattern '{pattern}' doesn't match any file under '{searchRoot}'");
throw new ArgumentException($"hashFiles('{ExpressionUtility.StringEscape(pattern)}') failed. Search pattern '{pattern}' doesn't match any file under '{searchRoot}'");
}
else
{
context.Trace.Info($"{files.Count} matches to hash");
}
// Hash each file
List<byte> filesSha256 = new List<byte>();
foreach (var file in files)
{
@@ -64,6 +86,7 @@ namespace GitHub.DistributedTask.Expressions2.Sdk.Functions
}
}
// Hash the hashes
using (SHA256 sha256hash = SHA256.Create())
{
var hashBytes = sha256hash.ComputeHash(filesSha256.ToArray());
@@ -83,11 +106,17 @@ namespace GitHub.DistributedTask.Expressions2.Sdk.Functions
}
}
private static readonly bool s_isWindows = Environment.OSVersion.Platform != PlatformID.Unix && Environment.OSVersion.Platform != PlatformID.MacOSX;
// Only support basic globbing (* ? and []) and globstar (**)
private static readonly Options s_minimatchOptions = new Options
{
Dot = true,
NoBrace = true,
NoCase = Environment.OSVersion.Platform != PlatformID.Unix && Environment.OSVersion.Platform != PlatformID.MacOSX
NoCase = s_isWindows,
NoComment = true,
NoExt = true,
NoNegate = true,
};
}
}

View File

@@ -154,7 +154,7 @@ namespace GitHub.DistributedTask.Expressions2.Tokens
{
case TokenKind.StartIndex: // "["
case TokenKind.Dereference: // "."
return new Index();
return new Sdk.Operators.Index();
case TokenKind.LogicalOperator:
switch (RawValue)

View File

@@ -17,10 +17,16 @@ namespace GitHub.DistributedTask.Logging
return Convert.ToBase64String(Encoding.UTF8.GetBytes(value));
}
// Base64 is 6 bytes -> char
// Base64 is 6 bits -> char
// A byte is 8 bits
// When end user doing somthing like base64(user:password)
// The length of the leading content will cause different base64 encoding result on the password
// So we add base64(value - 1/2/3/4/5 bytes) as secret as well.
// So we add base64(value shifted 1 and two bytes) as secret as well.
// B1 B2 B3 B4 B5 B6 B7
// 000000|00 0000|0000 00|000000| 000000|00 0000|0000 00|000000|
// Char1 Char2 Char3 Char4
// See the above, the first byte has a character beginning at index 0, the second byte has a character beginning at index 4, the third byte has a character beginning at index 2 and then the pattern repeats
// We register byte offsets for all these possible values
public static String Base64StringEscapeShift1(String value)
{
return Base64StringEscapeShift(value, 1);
@@ -31,21 +37,6 @@ namespace GitHub.DistributedTask.Logging
return Base64StringEscapeShift(value, 2);
}
public static String Base64StringEscapeShift3(String value)
{
return Base64StringEscapeShift(value, 3);
}
public static String Base64StringEscapeShift4(String value)
{
return Base64StringEscapeShift(value, 4);
}
public static String Base64StringEscapeShift5(String value)
{
return Base64StringEscapeShift(value, 5);
}
public static String ExpressionStringEscape(String value)
{
return Expressions.ExpressionUtil.StringEscape(value);

View File

@@ -55,14 +55,9 @@ namespace GitHub.DistributedTask.WebApi
m_properties = new PropertiesCollection(agentToBeCloned.m_properties);
}
if (agentToBeCloned.m_systemCapabilities != null && agentToBeCloned.m_systemCapabilities.Count > 0)
if (agentToBeCloned.m_labels != null && agentToBeCloned.m_labels.Count > 0)
{
m_systemCapabilities = new Dictionary<String, String>(agentToBeCloned.m_systemCapabilities, StringComparer.OrdinalIgnoreCase);
}
if (agentToBeCloned.m_userCapabilities != null && agentToBeCloned.m_userCapabilities.Count > 0)
{
m_userCapabilities = new Dictionary<String, String>(agentToBeCloned.m_userCapabilities, StringComparer.OrdinalIgnoreCase);
m_labels = new HashSet<string>(agentToBeCloned.m_labels, StringComparer.OrdinalIgnoreCase);
}
if (agentToBeCloned.PendingUpdate != null)
@@ -152,32 +147,17 @@ namespace GitHub.DistributedTask.WebApi
}
/// <summary>
/// System-defined capabilities supported by this agent's host.
/// The labels of the runner
/// </summary>
public IDictionary<String, String> SystemCapabilities
public ISet<string> Labels
{
get
{
if (m_systemCapabilities == null)
if (m_labels == null)
{
m_systemCapabilities = new Dictionary<String, String>(StringComparer.OrdinalIgnoreCase);
m_labels = new HashSet<string>(StringComparer.OrdinalIgnoreCase);
}
return m_systemCapabilities;
}
}
/// <summary>
/// User-defined capabilities supported by this agent's host.
/// </summary>
public IDictionary<String, String> UserCapabilities
{
get
{
if (m_userCapabilities == null)
{
m_userCapabilities = new Dictionary<String, String>(StringComparer.OrdinalIgnoreCase);
}
return m_userCapabilities;
return m_labels;
}
}
@@ -214,10 +194,7 @@ namespace GitHub.DistributedTask.WebApi
[DataMember(IsRequired = false, EmitDefaultValue = false, Name = "Properties")]
private PropertiesCollection m_properties;
[DataMember(IsRequired = false, EmitDefaultValue = false, Name = "SystemCapabilities")]
private Dictionary<String, String> m_systemCapabilities;
[DataMember(IsRequired = false, EmitDefaultValue = false, Name = "UserCapabilities")]
private Dictionary<String, String> m_userCapabilities;
[DataMember(IsRequired = false, EmitDefaultValue = false, Name = "Labels")]
private HashSet<string> m_labels;
}
}

View File

@@ -246,8 +246,7 @@ namespace GitHub.DistributedTask.WebApi
PlanType = hubName,
ScopeId = scopeIdentifier,
PlanId = planId,
JobId = jobId,
Demands = demands,
JobId = jobId
};
return QueueAgentRequestByPoolAsync(poolId, request, userState, cancellationToken);

View File

@@ -35,7 +35,6 @@ namespace GitHub.DistributedTask.WebApi
this.PoolId = requestToBeCloned.PoolId;
this.JobId = requestToBeCloned.JobId;
this.JobName = requestToBeCloned.JobName;
this.Demands = new List<Demand>(requestToBeCloned.Demands ?? new Demand[0]);
this.LockToken = requestToBeCloned.LockToken;
this.ExpectedDuration = requestToBeCloned.ExpectedDuration;
this.OrchestrationId = requestToBeCloned.OrchestrationId;
@@ -68,6 +67,11 @@ namespace GitHub.DistributedTask.WebApi
{
this.AgentSpecification = new JObject(requestToBeCloned.AgentSpecification);
}
if (requestToBeCloned.Labels != null)
{
this.Labels = new HashSet<string>(requestToBeCloned.Labels, StringComparer.OrdinalIgnoreCase);
}
}
/// <summary>
@@ -229,6 +233,7 @@ namespace GitHub.DistributedTask.WebApi
/// </summary>
/// <value></value>
[DataMember(Order = 16, EmitDefaultValue = false)]
[Obsolete("No more demands, use labels", true)]
public IList<Demand> Demands
{
get;
@@ -386,6 +391,13 @@ namespace GitHub.DistributedTask.WebApi
set;
}
[DataMember(Order = 33, EmitDefaultValue = false)]
public ISet<string> Labels
{
get;
set;
}
[IgnoreDataMember]
internal Guid? LockToken
{

View File

@@ -1,5 +1,4 @@
using System;
using System.Collections.Generic;
using System.Runtime.Serialization;
namespace GitHub.DistributedTask.WebApi
@@ -27,29 +26,6 @@ namespace GitHub.DistributedTask.WebApi
this.OwnerName = ownerName;
}
/// <summary>
/// Initializes a new <c>TaskAgentSession</c> isntance with the specified owner name, agent, and capabilities.
/// </summary>
/// <param name="ownerName">The name of the owner for this session. This should typically be the agent machine</param>
/// <param name="agent">The target agent for the session</param>
/// <param name="systemCapabilities">A collection of capabilities to publish on session creation</param>
public TaskAgentSession(
String ownerName,
TaskAgentReference agent,
IDictionary<String, String> systemCapabilities)
{
this.Agent = agent;
this.OwnerName = ownerName;
foreach (var capability in systemCapabilities)
{
if (capability.Value != null)
{
this.SystemCapabilities.Add(capability.Key, capability.Value);
}
}
}
/// <summary>
/// Gets the unique identifier for this session.
/// </summary>
@@ -89,33 +65,5 @@ namespace GitHub.DistributedTask.WebApi
get;
set;
}
/// <summary>
/// Gets the collection of system capabilities used for this session.
/// </summary>
public IDictionary<String, String> SystemCapabilities
{
get
{
if (m_systemCapabilities == null)
{
m_systemCapabilities = new Dictionary<String, String>(StringComparer.OrdinalIgnoreCase);
}
return m_systemCapabilities;
}
}
[OnSerializing]
private void OnSerializing(StreamingContext context)
{
if (m_systemCapabilities?.Count == 0)
{
m_systemCapabilities = null;
}
}
[DataMember(IsRequired = false, EmitDefaultValue = false, Name = "SystemCapabilities")]
private IDictionary<String, String> m_systemCapabilities;
}
}

View File

@@ -0,0 +1,44 @@
using System.Runtime.Serialization;
using GitHub.Services.WebApi;
namespace GitHub.Actions.Pipelines.WebApi
{
[DataContract]
public class ActionsStorageArtifact : Artifact
{
public ActionsStorageArtifact()
: base(ArtifactType.Actions_Storage)
{
}
/// <summary>
/// File Container ID
/// </summary>
[DataMember]
public long ContainerId
{
get;
set;
}
/// <summary>
/// Size of the file in bytes
/// </summary>
[DataMember]
public long Size
{
get;
set;
}
/// <summary>
/// Signed content url for downloading the artifact
/// </summary>
[DataMember]
public SignedUrl SignedContent
{
get;
set;
}
}
}

View File

@@ -0,0 +1,46 @@
using System.Runtime.Serialization;
using GitHub.Actions.Pipelines.WebApi.Contracts;
using Newtonsoft.Json;
namespace GitHub.Actions.Pipelines.WebApi
{
[DataContract]
[KnownType(typeof(ActionsStorageArtifact))]
[JsonConverter(typeof(ArtifactJsonConverter))]
public class Artifact
{
public Artifact(ArtifactType type)
{
Type = type;
}
/// <summary>
/// The type of the artifact.
/// </summary>
[DataMember]
public ArtifactType Type
{
get;
}
/// <summary>
/// The name of the artifact.
/// </summary>
[DataMember]
public string Name
{
get;
set;
}
/// <summary>
/// Self-referential url
/// </summary>
[DataMember]
public string Url
{
get;
set;
}
}
}

View File

@@ -0,0 +1,85 @@
using System;
using System.Reflection;
using GitHub.Services.WebApi;
using Newtonsoft.Json;
using Newtonsoft.Json.Linq;
using Newtonsoft.Json.Serialization;
namespace GitHub.Actions.Pipelines.WebApi
{
public abstract class ArtifactBaseJsonConverter<T> : VssSecureJsonConverter where T : class
{
public override bool CanConvert(Type objectType)
{
return typeof(T).GetTypeInfo().IsAssignableFrom(objectType.GetTypeInfo());
}
// by returning false, the converter doesn't take part in writes
// which means we use the default serialization logic
public override bool CanWrite
{
get
{
return false;
}
}
protected abstract T Create(Type objectType);
protected abstract T Create(ArtifactType type);
public override object ReadJson(
JsonReader reader,
Type objectType,
object existingValue,
JsonSerializer serializer)
{
if (reader.TokenType != JsonToken.StartObject)
{
return null;
}
var contract = serializer.ContractResolver.ResolveContract(objectType) as JsonObjectContract;
if (contract == null)
{
return existingValue;
}
// if objectType is one of our known types, we can ignore the type property
T targetObject = Create(objectType);
// read the data into a JObject so we can look at it
var value = JObject.Load(reader);
if (targetObject == null)
{
// use the Type property
var typeProperty = contract.Properties.GetClosestMatchProperty("Type");
if (typeProperty == null)
{
// we don't know the type. just bail
return existingValue;
}
if (!value.TryGetValue(typeProperty.PropertyName, StringComparison.OrdinalIgnoreCase, out var typeValue))
{
// a type property exists on the contract, but the JObject has no value for it
return existingValue;
}
var type = UnknownEnum.Parse<ArtifactType>(typeValue.ToString());
targetObject = Create(type);
}
if (targetObject != null)
{
using (var objectReader = value.CreateReader())
{
serializer.Populate(objectReader, targetObject);
}
}
return targetObject;
}
}
}

View File

@@ -0,0 +1,29 @@
using System;
namespace GitHub.Actions.Pipelines.WebApi.Contracts
{
public class ArtifactJsonConverter : ArtifactBaseJsonConverter<Artifact>
{
protected override Artifact Create(Type objectType)
{
if (objectType == typeof(ActionsStorageArtifact))
{
return new ActionsStorageArtifact();
}
else
{
return null;
}
}
protected override Artifact Create(ArtifactType type)
{
if (type == ArtifactType.Actions_Storage)
{
return new ActionsStorageArtifact();
}
return null;
}
}
}

View File

@@ -0,0 +1,14 @@
using System;
using System.Runtime.Serialization;
using Newtonsoft.Json;
namespace GitHub.Actions.Pipelines.WebApi
{
[DataContract]
[JsonConverter(typeof(ArtifactTypeEnumJsonConverter))]
public enum ArtifactType
{
Unknown = 0,
Actions_Storage = 1
}
}

View File

@@ -0,0 +1,25 @@
using Newtonsoft.Json;
using Newtonsoft.Json.Serialization;
using System;
using System.Net.Http.Formatting;
namespace GitHub.Actions.Pipelines.WebApi
{
public class ArtifactTypeEnumJsonConverter : UnknownEnumJsonConverter
{
//json.net v12 exposes a "NamingStrategy" member that can do all this. We are at json.net v10 which only supports camel case.
//This is a poor man's way to fake it
public override void WriteJson(JsonWriter writer, object enumValue, JsonSerializer serializer)
{
var value = (ArtifactType)enumValue;
if (value == ArtifactType.Actions_Storage)
{
writer.WriteValue("actions_storage");
}
else
{
base.WriteJson(writer, enumValue, serializer);
}
}
}
}

View File

@@ -0,0 +1,33 @@
using System.Runtime.Serialization;
namespace GitHub.Actions.Pipelines.WebApi
{
[DataContract]
public class CreateActionsStorageArtifactParameters : CreateArtifactParameters
{
public CreateActionsStorageArtifactParameters()
: base(ArtifactType.Actions_Storage)
{
}
/// <summary>
/// the id of the file container
/// </summary>
[DataMember]
public long ContainerId
{
get;
set;
}
/// <summary>
/// Size of the file in bytes
/// </summary>
[DataMember]
public long Size
{
get;
set;
}
}
}

View File

@@ -0,0 +1,35 @@
using System.Runtime.Serialization;
using Newtonsoft.Json;
namespace GitHub.Actions.Pipelines.WebApi
{
[DataContract]
[KnownType(typeof(CreateActionsStorageArtifactParameters))]
[JsonConverter(typeof(CreateArtifactParametersJsonConverter))]
public class CreateArtifactParameters
{
protected CreateArtifactParameters(ArtifactType type)
{
Type = type;
}
/// <summary>
/// The type of the artifact.
/// </summary>
[DataMember]
public ArtifactType Type
{
get;
}
/// <summary>
/// The name of the artifact.
/// </summary>
[DataMember]
public string Name
{
get;
set;
}
}
}

View File

@@ -0,0 +1,29 @@
using System;
namespace GitHub.Actions.Pipelines.WebApi
{
public class CreateArtifactParametersJsonConverter : ArtifactBaseJsonConverter<CreateArtifactParameters>
{
protected override CreateArtifactParameters Create(Type objectType)
{
if (objectType == typeof(CreateActionsStorageArtifactParameters))
{
return new CreateActionsStorageArtifactParameters();
}
else
{
return null;
}
}
protected override CreateArtifactParameters Create(ArtifactType type)
{
if (type == ArtifactType.Actions_Storage)
{
return new CreateActionsStorageArtifactParameters();
}
return null;
}
}
}

View File

@@ -0,0 +1,16 @@
using System;
using System.ComponentModel;
namespace GitHub.Actions.Pipelines.WebApi
{
/// <summary>
/// $expand options for GetArtifact and ListArtifacts.
/// </summary>
[TypeConverter(typeof(KnownFlagsEnumTypeConverter))]
[Flags]
public enum GetArtifactExpandOptions
{
None = 0,
SignedContent = 1,
}
}

View File

@@ -0,0 +1,105 @@
using System;
using System.Collections.Generic;
using System.Diagnostics;
using System.Globalization;
using System.Linq;
using System.Runtime.Serialization;
using GitHub.Services.Common;
namespace GitHub.Actions.Pipelines.WebApi
{
public static class FlagsEnum
{
public static TEnum ParseKnownFlags<TEnum>(string stringValue) where TEnum : System.Enum
{
return (TEnum)ParseKnownFlags(typeof(TEnum), stringValue);
}
/// <summary>
/// Parse known enum flags in a comma-separated string. Unknown flags are ignored. Allows for degraded compatibility without serializing enums to integers.
/// </summary>
/// <remarks>
/// Case insensitive. Both standard and EnumMemberAttribute names are parsed.
/// </remarks>
/// <exception cref="NullReferenceException">Thrown if stringValue is null.</exception>
/// <exception cref="ArgumentException">Thrown if a flag name is empty.</exception>
public static object ParseKnownFlags(Type enumType, string stringValue)
{
ArgumentUtility.CheckForNull(enumType, nameof(enumType));
if (!enumType.IsEnum)
{
throw new ArgumentException(PipelinesWebApiResources.FlagEnumTypeRequired());
}
// Check for the flags attribute in debug. Skip this reflection in release.
Debug.Assert(enumType.GetCustomAttributes(typeof(FlagsAttribute), inherit: false).Any(), "FlagsEnum only intended for enums with the Flags attribute.");
// The exception types below are based on Enum.TryParseEnum (http://index/?query=TryParseEnum&rightProject=mscorlib&file=system%5Cenum.cs&rightSymbol=bhaeh2vnegwo)
if (stringValue == null)
{
throw new ArgumentNullException(stringValue);
}
if (String.IsNullOrWhiteSpace(stringValue))
{
throw new ArgumentException(PipelinesWebApiResources.NonEmptyEnumElementsRequired(stringValue));
}
if (UInt64.TryParse(stringValue, NumberStyles.AllowLeadingSign, CultureInfo.InvariantCulture, out ulong ulongValue))
{
return Enum.Parse(enumType, stringValue);
}
var enumNames = Enum.GetNames(enumType).ToHashSet(name => name, StringComparer.OrdinalIgnoreCase);
var enumMemberMappings = new Lazy<IDictionary<string, string>>(() =>
{
IDictionary<string, string> mappings = null;
foreach (var field in enumType.GetFields())
{
if (field.GetCustomAttributes(typeof(EnumMemberAttribute), false).FirstOrDefault() is EnumMemberAttribute enumMemberAttribute)
{
if (mappings == null)
{
mappings = new Dictionary<string, string>(StringComparer.OrdinalIgnoreCase);
}
mappings.Add(enumMemberAttribute.Value, field.GetValue(null).ToString());
}
}
return mappings;
});
var values = stringValue.Split(s_enumSeparatorCharArray);
var matches = new List<string>();
for (int i = 0; i < values.Length; i++)
{
string value = values[i].Trim();
if (String.IsNullOrEmpty(value))
{
throw new ArgumentException(PipelinesWebApiResources.NonEmptyEnumElementsRequired(stringValue));
}
if (enumNames.Contains(value))
{
matches.Add(value);
}
else if (enumMemberMappings.Value != null && enumMemberMappings.Value.TryGetValue(value, out string matchingValue))
{
matches.Add(matchingValue);
}
}
if (!matches.Any())
{
return Enum.Parse(enumType, "0");
}
string matchesString = String.Join(", ", matches);
return Enum.Parse(enumType, matchesString, ignoreCase: true);
}
private static readonly char[] s_enumSeparatorCharArray = new char[] { ',' };
}
}

View File

@@ -0,0 +1,273 @@
/*
* ---------------------------------------------------------
* Copyright(C) Microsoft Corporation. All rights reserved.
* ---------------------------------------------------------
*
* ---------------------------------------------------------
* Generated file, DO NOT EDIT
* ---------------------------------------------------------
*
* See following wiki page for instructions on how to regenerate:
* https://aka.ms/azure-devops-client-generation
*
* Configuration file:
* actions\client\webapi\clientgeneratorconfigs\pipelines.genclient.json
*/
using System;
using System.Collections.Generic;
using System.Collections.Specialized;
using System.Globalization;
using System.IO;
using System.IO.Compression;
using System.Linq;
using System.Net.Http;
using System.Net.Http.Formatting;
using System.Threading;
using System.Threading.Tasks;
using GitHub.Services.Common;
using GitHub.Services.WebApi;
namespace GitHub.Actions.Pipelines.WebApi
{
[ResourceArea(PipelinesArea.IdString)]
public abstract class PipelinesHttpClientBase : VssHttpClientBase
{
public PipelinesHttpClientBase(Uri baseUrl, VssCredentials credentials)
: base(baseUrl, credentials)
{
}
public PipelinesHttpClientBase(Uri baseUrl, VssCredentials credentials, VssHttpRequestSettings settings)
: base(baseUrl, credentials, settings)
{
}
public PipelinesHttpClientBase(Uri baseUrl, VssCredentials credentials, params DelegatingHandler[] handlers)
: base(baseUrl, credentials, handlers)
{
}
public PipelinesHttpClientBase(Uri baseUrl, VssCredentials credentials, VssHttpRequestSettings settings, params DelegatingHandler[] handlers)
: base(baseUrl, credentials, settings, handlers)
{
}
public PipelinesHttpClientBase(Uri baseUrl, HttpMessageHandler pipeline, bool disposeHandler)
: base(baseUrl, pipeline, disposeHandler)
{
}
/// <summary>
/// [Preview API] Associates an artifact with a run.
/// </summary>
/// <param name="createArtifactParameters"></param>
/// <param name="pipelineId">The ID of the pipeline.</param>
/// <param name="runId">The ID of the run.</param>
/// <param name="userState"></param>
/// <param name="cancellationToken">The cancellation token to cancel operation.</param>
public virtual Task<Artifact> CreateArtifactAsync(
CreateArtifactParameters createArtifactParameters,
int pipelineId,
int runId,
object userState = null,
CancellationToken cancellationToken = default)
{
HttpMethod httpMethod = new HttpMethod("POST");
Guid locationId = new Guid("85023071-bd5e-4438-89b0-2a5bf362a19d");
object routeValues = new { pipelineId = pipelineId, runId = runId };
HttpContent content = new ObjectContent<CreateArtifactParameters>(createArtifactParameters, new VssJsonMediaTypeFormatter(true));
return SendAsync<Artifact>(
httpMethod,
locationId,
routeValues: routeValues,
version: new ApiResourceVersion(6.0, 1),
userState: userState,
cancellationToken: cancellationToken,
content: content);
}
/// <summary>
/// [Preview API] Associates an artifact with a run.
/// </summary>
/// <param name="createArtifactParameters"></param>
/// <param name="project">Project ID or project name</param>
/// <param name="pipelineId">The ID of the pipeline.</param>
/// <param name="runId">The ID of the run.</param>
/// <param name="userState"></param>
/// <param name="cancellationToken">The cancellation token to cancel operation.</param>
public virtual Task<Artifact> CreateArtifactAsync(
CreateArtifactParameters createArtifactParameters,
string project,
int pipelineId,
int runId,
object userState = null,
CancellationToken cancellationToken = default)
{
HttpMethod httpMethod = new HttpMethod("POST");
Guid locationId = new Guid("85023071-bd5e-4438-89b0-2a5bf362a19d");
object routeValues = new { project = project, pipelineId = pipelineId, runId = runId };
HttpContent content = new ObjectContent<CreateArtifactParameters>(createArtifactParameters, new VssJsonMediaTypeFormatter(true));
return SendAsync<Artifact>(
httpMethod,
locationId,
routeValues: routeValues,
version: new ApiResourceVersion(6.0, 1),
userState: userState,
cancellationToken: cancellationToken,
content: content);
}
/// <summary>
/// [Preview API] Associates an artifact with a run.
/// </summary>
/// <param name="createArtifactParameters"></param>
/// <param name="project">Project ID</param>
/// <param name="pipelineId">The ID of the pipeline.</param>
/// <param name="runId">The ID of the run.</param>
/// <param name="userState"></param>
/// <param name="cancellationToken">The cancellation token to cancel operation.</param>
public virtual Task<Artifact> CreateArtifactAsync(
CreateArtifactParameters createArtifactParameters,
Guid project,
int pipelineId,
int runId,
object userState = null,
CancellationToken cancellationToken = default)
{
HttpMethod httpMethod = new HttpMethod("POST");
Guid locationId = new Guid("85023071-bd5e-4438-89b0-2a5bf362a19d");
object routeValues = new { project = project, pipelineId = pipelineId, runId = runId };
HttpContent content = new ObjectContent<CreateArtifactParameters>(createArtifactParameters, new VssJsonMediaTypeFormatter(true));
return SendAsync<Artifact>(
httpMethod,
locationId,
routeValues: routeValues,
version: new ApiResourceVersion(6.0, 1),
userState: userState,
cancellationToken: cancellationToken,
content: content);
}
/// <summary>
/// [Preview API] Get a specific artifact
/// </summary>
/// <param name="project">Project ID or project name</param>
/// <param name="pipelineId"></param>
/// <param name="runId"></param>
/// <param name="artifactName"></param>
/// <param name="expand"></param>
/// <param name="userState"></param>
/// <param name="cancellationToken">The cancellation token to cancel operation.</param>
public virtual Task<Artifact> GetArtifactAsync(
string project,
int pipelineId,
int runId,
string artifactName,
GetArtifactExpandOptions? expand = null,
object userState = null,
CancellationToken cancellationToken = default)
{
HttpMethod httpMethod = new HttpMethod("GET");
Guid locationId = new Guid("85023071-bd5e-4438-89b0-2a5bf362a19d");
object routeValues = new { project = project, pipelineId = pipelineId, runId = runId };
List<KeyValuePair<string, string>> queryParams = new List<KeyValuePair<string, string>>();
queryParams.Add("artifactName", artifactName);
if (expand != null)
{
queryParams.Add("$expand", expand.Value.ToString());
}
return SendAsync<Artifact>(
httpMethod,
locationId,
routeValues: routeValues,
version: new ApiResourceVersion(6.0, 1),
queryParameters: queryParams,
userState: userState,
cancellationToken: cancellationToken);
}
/// <summary>
/// [Preview API] Get a specific artifact
/// </summary>
/// <param name="project">Project ID</param>
/// <param name="pipelineId"></param>
/// <param name="runId"></param>
/// <param name="artifactName"></param>
/// <param name="expand"></param>
/// <param name="userState"></param>
/// <param name="cancellationToken">The cancellation token to cancel operation.</param>
public virtual Task<Artifact> GetArtifactAsync(
Guid project,
int pipelineId,
int runId,
string artifactName,
GetArtifactExpandOptions? expand = null,
object userState = null,
CancellationToken cancellationToken = default)
{
HttpMethod httpMethod = new HttpMethod("GET");
Guid locationId = new Guid("85023071-bd5e-4438-89b0-2a5bf362a19d");
object routeValues = new { project = project, pipelineId = pipelineId, runId = runId };
List<KeyValuePair<string, string>> queryParams = new List<KeyValuePair<string, string>>();
queryParams.Add("artifactName", artifactName);
if (expand != null)
{
queryParams.Add("$expand", expand.Value.ToString());
}
return SendAsync<Artifact>(
httpMethod,
locationId,
routeValues: routeValues,
version: new ApiResourceVersion(6.0, 1),
queryParameters: queryParams,
userState: userState,
cancellationToken: cancellationToken);
}
/// <summary>
/// [Preview API] Get a specific artifact
/// </summary>
/// <param name="pipelineId"></param>
/// <param name="runId"></param>
/// <param name="artifactName"></param>
/// <param name="expand"></param>
/// <param name="userState"></param>
/// <param name="cancellationToken">The cancellation token to cancel operation.</param>
public virtual Task<Artifact> GetArtifactAsync(
int pipelineId,
int runId,
string artifactName,
GetArtifactExpandOptions? expand = null,
object userState = null,
CancellationToken cancellationToken = default)
{
HttpMethod httpMethod = new HttpMethod("GET");
Guid locationId = new Guid("85023071-bd5e-4438-89b0-2a5bf362a19d");
object routeValues = new { pipelineId = pipelineId, runId = runId };
List<KeyValuePair<string, string>> queryParams = new List<KeyValuePair<string, string>>();
queryParams.Add("artifactName", artifactName);
if (expand != null)
{
queryParams.Add("$expand", expand.Value.ToString());
}
return SendAsync<Artifact>(
httpMethod,
locationId,
routeValues: routeValues,
version: new ApiResourceVersion(6.0, 1),
queryParameters: queryParams,
userState: userState,
cancellationToken: cancellationToken);
}
}
}

View File

@@ -0,0 +1,44 @@
using System;
using System.ComponentModel;
using System.Globalization;
namespace GitHub.Actions.Pipelines.WebApi
{
/// <summary>
/// Parses known enum flags in a comma-separated string. Unknown flags are ignored. Allows for degraded compatibility without serializing enums to integer values.
/// </summary>
/// <remarks>
/// Case insensitive. Both standard and EnumMemberAttribute names are parsed.
/// json deserialization doesn't happen for query parameters :)
/// </remarks>
public class KnownFlagsEnumTypeConverter : EnumConverter
{
public KnownFlagsEnumTypeConverter(Type type)
: base(type)
{
}
public override bool CanConvertFrom(ITypeDescriptorContext context, Type sourceType)
{
return sourceType == typeof(string) || base.CanConvertFrom(context, sourceType);
}
/// <exception cref="FormatException">Thrown if a flag name is empty.</exception>
public override object ConvertFrom(ITypeDescriptorContext context, CultureInfo culture, object value)
{
if (value is string stringValue)
{
try
{
return FlagsEnum.ParseKnownFlags(EnumType, stringValue);
}
catch (Exception ex)
{
// Matches the exception type thrown by EnumTypeConverter.
throw new FormatException(PipelinesWebApiResources.InvalidFlagsEnumValue(stringValue, EnumType), ex);
}
}
return base.ConvertFrom(context, culture, value);
}
}
}

View File

@@ -0,0 +1,36 @@
using System;
using System.Net.Http;
using GitHub.Services.Common;
using GitHub.Services.WebApi;
namespace GitHub.Actions.Pipelines.WebApi
{
[ResourceArea(PipelinesArea.IdString)]
public class PipelinesHttpClient : PipelinesHttpClientBase
{
public PipelinesHttpClient(Uri baseUrl, VssCredentials credentials)
: base(baseUrl, credentials)
{
}
public PipelinesHttpClient(Uri baseUrl, VssCredentials credentials, VssHttpRequestSettings settings)
: base(baseUrl, credentials, settings)
{
}
public PipelinesHttpClient(Uri baseUrl, VssCredentials credentials, params DelegatingHandler[] handlers)
: base(baseUrl, credentials, handlers)
{
}
public PipelinesHttpClient(Uri baseUrl, VssCredentials credentials, VssHttpRequestSettings settings, params DelegatingHandler[] handlers)
: base(baseUrl, credentials, settings, handlers)
{
}
public PipelinesHttpClient(Uri baseUrl, HttpMessageHandler pipeline, bool disposeHandler)
: base(baseUrl, pipeline, disposeHandler)
{
}
}
}

View File

@@ -0,0 +1,74 @@
using System;
namespace GitHub.Actions.Pipelines.WebApi
{
public static class PipelinesArea
{
public const string Name = "pipelines";
public const string IdString = "2e0bf237-8973-4ec9-a581-9c3d679d1776";
public static readonly Guid Id = new Guid(PipelinesArea.IdString);
}
public static class PipelinesResources
{
public static class Artifacts
{
public const string Name = "artifacts";
public static readonly Guid Id = new Guid("85023071-BD5E-4438-89B0-2A5BF362A19D");
}
public static class PipelineOrgs
{
public const string Name = "orgs";
public static readonly Guid Id = new Guid("CD70BA1A-D59A-4E0B-9934-97998159CCC8");
}
public static class Logs
{
public const string Name = "logs";
public static readonly Guid Id = new Guid("fb1b6d27-3957-43d5-a14b-a2d70403e545");
}
public static class Pipelines
{
public const string Name = "pipelines";
public static readonly Guid Id = new Guid("28e1305e-2afe-47bf-abaf-cbb0e6a91988");
}
public static class Reputations
{
public const string Name = "reputations";
public static readonly Guid Id = new Guid("ABA353B0-46FB-4885-88C5-391C6B6382B3");
}
public static class Runs
{
public const string Name = "runs";
public static readonly Guid Id = new Guid("7859261e-d2e9-4a68-b820-a5d84cc5bb3d");
}
public static class SignalR
{
public const string Name = "signalr";
public static readonly Guid Id = new Guid("1FFE4916-AC72-4566-ADD0-9BAB31E44FCF");
}
public static class SignedArtifactsContent
{
public const string Name = "signedartifactscontent";
public static readonly Guid Id = new Guid("6B2AC16F-CD00-4DF9-A13B-3A1CC8AFB188");
}
public static class SignedLogContent
{
public const string Name = "signedlogcontent";
public static readonly Guid Id = new Guid("74f99e32-e2c4-44f4-93dc-dec0bca530a5");
}
public static class SignalRLive
{
public const string Name = "live";
public static readonly Guid Id = new Guid("C41B3775-6D50-48BD-B261-42DA7F0F1BA0");
}
}
}

View File

@@ -0,0 +1,45 @@
using System;
using System.Linq;
using System.Runtime.Serialization;
namespace GitHub.Actions.Pipelines.WebApi
{
public static class UnknownEnum
{
public static T Parse<T>(string stringValue)
{
return (T)Parse(typeof(T), stringValue);
}
public static object Parse(Type enumType, string stringValue)
{
var underlyingType = Nullable.GetUnderlyingType(enumType);
enumType = underlyingType != null ? underlyingType : enumType;
var names = Enum.GetNames(enumType);
if (!string.IsNullOrEmpty(stringValue))
{
var match = names.FirstOrDefault(name => string.Equals(name, stringValue, StringComparison.OrdinalIgnoreCase));
if (match != null)
{
return Enum.Parse(enumType, match);
}
// maybe we have an enum member with an EnumMember attribute specifying a custom name
foreach (var field in enumType.GetFields())
{
var enumMemberAttribute = field.GetCustomAttributes(typeof(EnumMemberAttribute), false).FirstOrDefault() as EnumMemberAttribute;
if (enumMemberAttribute != null && string.Equals(enumMemberAttribute.Value, stringValue, StringComparison.OrdinalIgnoreCase))
{
// we already have the field, no need to do enum.parse on it
return field.GetValue(null);
}
}
}
return Enum.Parse(enumType, UnknownName);
}
private const string UnknownName = "Unknown";
}
}

View File

@@ -0,0 +1,53 @@
using System;
using System.Linq;
using Newtonsoft.Json;
using Newtonsoft.Json.Converters;
namespace GitHub.Actions.Pipelines.WebApi
{
public class UnknownEnumJsonConverter : StringEnumConverter
{
public UnknownEnumJsonConverter()
{
this.CamelCaseText = true;
}
public override bool CanConvert(Type objectType)
{
// we require one member to be named "Unknown"
return objectType.IsEnum && Enum.GetNames(objectType).Any(name => string.Equals(name, UnknownName, StringComparison.OrdinalIgnoreCase));
}
public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer)
{
// Newtonsoft doesn't call CanConvert if you specify the converter using a JsonConverter attribute
// they just assume you know what you're doing :)
if (!CanConvert(objectType))
{
// if there's no Unknown value, fall back to the StringEnumConverter behavior
return base.ReadJson(reader, objectType, existingValue, serializer);
}
if (reader.TokenType == JsonToken.Integer)
{
var intValue = Convert.ToInt32(reader.Value);
var values = (int[])Enum.GetValues(objectType);
if (values.Contains(intValue))
{
return Enum.Parse(objectType, intValue.ToString());
}
}
if (reader.TokenType == JsonToken.String)
{
var stringValue = reader.Value.ToString();
return UnknownEnum.Parse(objectType, stringValue);
}
// we know there's an Unknown value because CanConvert returned true
return Enum.Parse(objectType, UnknownName);
}
private const string UnknownName = "Unknown";
}
}

View File

@@ -0,0 +1,26 @@
using System.Globalization;
namespace GitHub.Actions.Pipelines.WebApi
{
public static class PipelinesWebApiResources
{
public static string FlagEnumTypeRequired()
{
const string Format = @"Invalid type. An enum type with the Flags attribute must be supplied.";
return Format;
}
public static string InvalidFlagsEnumValue(object arg0, object arg1)
{
const string Format = @"'{0}' is not a valid value for {1}";
return string.Format(CultureInfo.CurrentCulture, Format, arg0, arg1);
}
public static string NonEmptyEnumElementsRequired(object arg0)
{
const string Format = @"Each comma separated enum value must be non-empty: {0}";
return string.Format(CultureInfo.CurrentCulture, Format, arg0);
}
}
}

View File

@@ -1,9 +1,9 @@
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFramework>netcoreapp2.2</TargetFramework>
<TargetFramework>netcoreapp3.0</TargetFramework>
<OutputType>Library</OutputType>
<RuntimeIdentifiers>win-x64;win-x86;linux-x64;linux-arm;rhel.6-x64;osx-x64</RuntimeIdentifiers>
<RuntimeIdentifiers>win-x64;win-x86;linux-x64;linux-arm64;linux-arm;osx-x64</RuntimeIdentifiers>
<TargetLatestRuntimePatch>true</TargetLatestRuntimePatch>
<AssetTargetFallback>portable-net45+win8</AssetTargetFallback>
<NoWarn>NU1701;NU1603</NoWarn>
@@ -11,6 +11,7 @@
<DefineConstants>NETSTANDARD;NET_STANDARD;TRACE</DefineConstants>
<LangVersion>7.3</LangVersion>
<AllowUnsafeBlocks>true</AllowUnsafeBlocks>
<TieredCompilationQuickJit>true</TieredCompilationQuickJit>
</PropertyGroup>
<ItemGroup>
@@ -21,11 +22,7 @@
<PackageReference Include="System.Security.Cryptography.Cng" Version="4.4.0" />
<PackageReference Include="System.Security.Cryptography.Pkcs" Version="4.4.0" />
<PackageReference Include="System.Security.Cryptography.ProtectedData" Version="4.4.0" />
<PackageReference Include="System.Data.SqlClient" Version="4.4.0" />
<PackageReference Include="System.Configuration.ConfigurationManager" Version="4.4.0" />
<PackageReference Include="Microsoft.IdentityModel.Clients.ActiveDirectory" Version="3.19.4" />
<PackageReference Include="Microsoft.ApplicationInsights" Version="2.9.1" />
<PackageReference Include="WindowsAzure.Storage" Version="8.7.0" />
<PackageReference Include="Minimatch" Version="2.0.0" />
<PackageReference Include="YamlDotNet.Signed" Version="5.3.0" />
</ItemGroup>

View File

@@ -0,0 +1,18 @@
using System;
using System.Runtime.Serialization;
namespace GitHub.Services.WebApi
{
/// <summary>
/// A signed url allowing limited-time anonymous access to private resources.
/// </summary>
[DataContract]
public class SignedUrl
{
[DataMember]
public string Url { get; set; }
[DataMember]
public DateTime SignatureExpires { get; set; }
}
}

View File

@@ -397,6 +397,11 @@ namespace GitHub.Services.FileContainer.Client
{
break;
}
else if (IsFastFailResponse(response))
{
FileUploadTrace(itemPath, $"Chunk '{currentChunk}' attempt '{attempt}' of file '{itemPath}' received non-success status code {response.StatusCode} for sending request and cannot continue.");
break;
}
else
{
FileUploadTrace(itemPath, $"Chunk '{currentChunk}' attempt '{attempt}' of file '{itemPath}' received non-success status code {response.StatusCode} for sending request.");
@@ -538,6 +543,17 @@ namespace GitHub.Services.FileContainer.Client
cancellationToken);
}
public bool IsFastFailResponse(HttpResponseMessage response)
{
int statusCode = (int)response?.StatusCode;
return statusCode >= 400 && statusCode <= 499;
}
protected override bool ShouldThrowError(HttpResponseMessage response)
{
return !response.IsSuccessStatusCode && !IsFastFailResponse(response);
}
private async Task<HttpResponseMessage> ContainerGetRequestAsync(
Int64 containerId,
String itemPath,

View File

@@ -47,8 +47,8 @@ namespace GitHub.Runner.Common.Tests
});
// Assert.
Assert.Equal(hc.SecretMasker.MaskSecrets("secret value 1"), "***");
Assert.Equal(hc.SecretMasker.MaskSecrets("secret value 2"), "***");
Assert.Equal("***", hc.SecretMasker.MaskSecrets("secret value 1"));
Assert.Equal("***", hc.SecretMasker.MaskSecrets("secret value 2"));
}
}
@@ -90,9 +90,9 @@ namespace GitHub.Runner.Common.Tests
trace.Info("Args: {0}", clp.Args.Count);
Assert.True(clp.Args.Count == 2);
Assert.True(clp.Args.ContainsKey("arg1"));
Assert.Equal(clp.Args["arg1"], "arg1val");
Assert.Equal("arg1val", clp.Args["arg1"]);
Assert.True(clp.Args.ContainsKey("arg2"));
Assert.Equal(clp.Args["arg2"], "arg2val");
Assert.Equal("arg2val", clp.Args["arg2"]);
}
}
@@ -113,8 +113,8 @@ namespace GitHub.Runner.Common.Tests
trace.Info("Args: {0}", clp.Flags.Count);
Assert.True(clp.Flags.Count == 2);
Assert.True(clp.Flags.Contains("flag1"));
Assert.True(clp.Flags.Contains("flag2"));
Assert.Contains("flag1", clp.Flags);
Assert.Contains("flag2", clp.Flags);
}
}

View File

@@ -18,7 +18,7 @@ namespace GitHub.Runner.Common.Tests
"win-x86",
"linux-x64",
"linux-arm",
"rhel.6-x64",
"linux-arm64",
"osx-x64"
};

View File

@@ -36,10 +36,10 @@ namespace GitHub.Runner.Common.Tests.Worker.Container
// Assert
Assert.NotNull(result0);
Assert.Equal(result0.Count, 0);
Assert.Equal(0, result0.Count);
Assert.NotNull(result1);
Assert.Equal(result1.Count, 1);
Assert.Equal(1, result1.Count);
var result1Port80Mapping = result1.Find(pm =>
string.Equals(pm.ContainerPort, "80") &&
string.Equals(pm.HostPort, "32881") &&
@@ -48,10 +48,10 @@ namespace GitHub.Runner.Common.Tests.Worker.Container
Assert.NotNull(result1Port80Mapping);
Assert.NotNull(result1Empty);
Assert.Equal(result1Empty.Count, 0);
Assert.Equal(0, result1Empty.Count);
Assert.NotNull(result2);
Assert.Equal(result2.Count, 2);
Assert.Equal(2, result2.Count);
var result2Port80Mapping = result2.Find(pm =>
string.Equals(pm.ContainerPort, "80") &&
string.Equals(pm.HostPort, "32881") &&

View File

@@ -1,5 +1,4 @@
using GitHub.Runner.Common.Capabilities;
using GitHub.Runner.Worker;
using GitHub.Runner.Worker;
using System;
using System.Collections.Generic;
using System.Linq;
@@ -21,12 +20,12 @@ namespace GitHub.Runner.Common.Tests
manager.Initialize(tc);
// Act.
List<ICapabilitiesProvider> extensions = manager.GetExtensions<ICapabilitiesProvider>();
List<IActionCommandExtension> extensions = manager.GetExtensions<IActionCommandExtension>();
// Assert.
Assert.True(
extensions.Any(x => x is RunnerCapabilitiesProvider),
$"Expected {nameof(RunnerCapabilitiesProvider)} extension to be returned as a job extension.");
extensions.Any(x => x is SetEnvCommandExtension),
$"Expected {nameof(SetEnvCommandExtension)} extension to be returned as a job extension.");
}
}
@@ -42,9 +41,9 @@ namespace GitHub.Runner.Common.Tests
manager.Initialize(tc);
// Act/Assert.
AssertContains<GitHub.Runner.Common.Capabilities.ICapabilitiesProvider>(
AssertContains<GitHub.Runner.Worker.IActionCommandExtension>(
manager,
concreteType: typeof(GitHub.Runner.Common.Capabilities.RunnerCapabilitiesProvider));
concreteType: typeof(GitHub.Runner.Worker.SetEnvCommandExtension));
}
}

View File

@@ -1,7 +1,9 @@
using GitHub.Runner.Common.Util;
using System;
using System.IO;
using System.Reflection;
using System.Runtime.CompilerServices;
using System.Text;
using System.Threading;
using Xunit;
@@ -67,7 +69,45 @@ namespace GitHub.Runner.Common.Tests
}
}
public void Setup([CallerMemberName] string testName = "")
[Fact]
[Trait("Level", "L0")]
[Trait("Category", "Common")]
public void DefaultSecretMaskers()
{
try
{
// Arrange.
Setup();
// Act.
_hc.SecretMasker.AddValue("Password123!");
_hc.SecretMasker.AddValue("Pass\"word\"123!");
_hc.SecretMasker.AddValue("Pass word 123!");
_hc.SecretMasker.AddValue("Pass<word>123!");
_hc.SecretMasker.AddValue("Pass'word'123!");
// Assert.
Assert.Equal("123***123", _hc.SecretMasker.MaskSecrets("123Password123!123"));
Assert.Equal("password123", _hc.SecretMasker.MaskSecrets("password123"));
Assert.Equal("123***123", _hc.SecretMasker.MaskSecrets("123Pass\\\"word\\\"123!123"));
Assert.Equal("123***123", _hc.SecretMasker.MaskSecrets("123Pass%20word%20123%21123"));
Assert.Equal("123***123", _hc.SecretMasker.MaskSecrets("123Pass&lt;word&gt;123!123"));
Assert.Equal("123***123", _hc.SecretMasker.MaskSecrets("123Pass''word''123!123"));
Assert.Equal("OlBh***", _hc.SecretMasker.MaskSecrets(Convert.ToBase64String(Encoding.UTF8.GetBytes($":Password123!"))));
Assert.Equal("YTpQ***", _hc.SecretMasker.MaskSecrets(Convert.ToBase64String(Encoding.UTF8.GetBytes($"a:Password123!"))));
Assert.Equal("YWI6***", _hc.SecretMasker.MaskSecrets(Convert.ToBase64String(Encoding.UTF8.GetBytes($"ab:Password123!"))));
Assert.Equal("YWJjOlBh***", _hc.SecretMasker.MaskSecrets(Convert.ToBase64String(Encoding.UTF8.GetBytes($"abc:Password123!"))));
Assert.Equal("YWJjZDpQ***", _hc.SecretMasker.MaskSecrets(Convert.ToBase64String(Encoding.UTF8.GetBytes($"abcd:Password123!"))));
Assert.Equal("YWJjZGU6***", _hc.SecretMasker.MaskSecrets(Convert.ToBase64String(Encoding.UTF8.GetBytes($"abcde:Password123!"))));
}
finally
{
// Cleanup.
Teardown();
}
}
private void Setup([CallerMemberName] string testName = "")
{
_tokenSource = new CancellationTokenSource();
_hc = new HostContext(
@@ -75,7 +115,7 @@ namespace GitHub.Runner.Common.Tests
logFile: Path.Combine(Path.GetDirectoryName(Assembly.GetEntryAssembly().Location), $"trace_{nameof(HostContextL0)}_{testName}.log"));
}
public void Teardown()
private void Teardown()
{
_hc?.Dispose();
_tokenSource?.Dispose();

View File

@@ -52,7 +52,7 @@ namespace GitHub.Runner.Common.Tests
// Assert.
Assert.Equal("some agent", actual);
Assert.Equal(string.Empty, Environment.GetEnvironmentVariable("ACTIONS_RUNNER_INPUT_AGENT") ?? string.Empty); // Should remove.
Assert.Equal(hc.SecretMasker.MaskSecrets("some agent"), "some agent");
Assert.Equal("some agent", hc.SecretMasker.MaskSecrets("some agent"));
}
finally
{
@@ -80,7 +80,7 @@ namespace GitHub.Runner.Common.Tests
// Assert.
Assert.Equal("some secret token value", actual);
Assert.Equal(string.Empty, Environment.GetEnvironmentVariable("ACTIONS_RUNNER_INPUT_TOKEN") ?? string.Empty); // Should remove.
Assert.Equal(hc.SecretMasker.MaskSecrets("some secret token value"), "***");
Assert.Equal("***", hc.SecretMasker.MaskSecrets("some secret token value"));
}
finally
{
@@ -250,7 +250,7 @@ namespace GitHub.Runner.Common.Tests
bool actual = command.Unattended;
// Assert.
Assert.Equal(true, actual);
Assert.True(actual);
Assert.Equal(string.Empty, Environment.GetEnvironmentVariable("ACTIONS_RUNNER_INPUT_UNATTENDED") ?? string.Empty); // Should remove.
}
finally
@@ -720,7 +720,7 @@ namespace GitHub.Runner.Common.Tests
var command = new CommandSettings(hc, args: new string[] { "badcommand" });
// Assert.
Assert.True(command.Validate().Contains("badcommand"));
Assert.Contains("badcommand", command.Validate());
}
}
@@ -735,7 +735,7 @@ namespace GitHub.Runner.Common.Tests
var command = new CommandSettings(hc, args: new string[] { "--badflag" });
// Assert.
Assert.True(command.Validate().Contains("badflag"));
Assert.Contains("badflag", command.Validate());
}
}
@@ -750,7 +750,7 @@ namespace GitHub.Runner.Common.Tests
var command = new CommandSettings(hc, args: new string[] { "--badargname", "bad arg value" });
// Assert.
Assert.True(command.Validate().Contains("badargname"));
Assert.Contains("badargname", command.Validate());
}
}

View File

@@ -1,79 +0,0 @@
using GitHub.Runner.Common.Capabilities;
using GitHub.Runner.Listener.Configuration;
using Moq;
using System;
using System.Collections.Generic;
using System.Linq;
using System.Threading;
using System.Threading.Tasks;
using Xunit;
namespace GitHub.Runner.Common.Tests.Listener
{
public sealed class AgentCapabilitiesProviderTestL0
{
[Fact]
[Trait("Level", "L0")]
[Trait("Category", "Agent")]
public async void TestGetCapabilities()
{
using (var hc = new TestHostContext(this))
using (var tokenSource = new CancellationTokenSource())
{
Mock<IConfigurationManager> configurationManager = new Mock<IConfigurationManager>();
hc.SetSingleton<IConfigurationManager>(configurationManager.Object);
// Arrange
var provider = new RunnerCapabilitiesProvider();
provider.Initialize(hc);
var settings = new RunnerSettings() { AgentName = "IAmAgent007" };
// Act
List<Capability> capabilities = await provider.GetCapabilitiesAsync(settings, tokenSource.Token);
// Assert
Assert.NotNull(capabilities);
Capability runnerNameCapability = capabilities.SingleOrDefault(x => string.Equals(x.Name, "Runner.Name", StringComparison.Ordinal));
Assert.NotNull(runnerNameCapability);
Assert.Equal("IAmAgent007", runnerNameCapability.Value);
}
}
[Fact]
[Trait("Level", "L0")]
[Trait("Category", "Agent")]
public async void TestInteractiveSessionCapability()
{
using (var hc = new TestHostContext(this))
using (var tokenSource = new CancellationTokenSource())
{
hc.StartupType = StartupType.AutoStartup;
await VerifyInteractiveSessionCapability(hc, tokenSource.Token, true);
hc.StartupType = StartupType.Service;
await VerifyInteractiveSessionCapability(hc, tokenSource.Token, false);
hc.StartupType = StartupType.Manual;
await VerifyInteractiveSessionCapability(hc, tokenSource.Token, true);
}
}
private async Task VerifyInteractiveSessionCapability(IHostContext hc, CancellationToken token, bool expectedValue)
{
// Arrange
var provider = new RunnerCapabilitiesProvider();
provider.Initialize(hc);
var settings = new RunnerSettings() { AgentName = "IAmAgent007" };
// Act
List<Capability> capabilities = await provider.GetCapabilitiesAsync(settings, token);
// Assert
Assert.NotNull(capabilities);
Capability iSessionCapability = capabilities.SingleOrDefault(x => string.Equals(x.Name, "InteractiveSession", StringComparison.Ordinal));
Assert.NotNull(iSessionCapability);
bool.TryParse(iSessionCapability.Value, out bool isInteractive);
Assert.Equal(expectedValue, isInteractive);
}
}
}

View File

@@ -1,6 +1,5 @@
using GitHub.DistributedTask.WebApi;
using GitHub.Runner.Listener;
using GitHub.Runner.Common.Capabilities;
using GitHub.Runner.Listener.Configuration;
using GitHub.Runner.Common.Util;
using GitHub.Services.WebApi;
@@ -40,16 +39,10 @@ namespace GitHub.Runner.Common.Tests.Listener.Configuration
#endif
private Mock<IRSAKeyManager> _rsaKeyManager;
private ICapabilitiesManager _capabilitiesManager;
// private DeploymentGroupAgentConfigProvider _deploymentGroupAgentConfigProvider;
private string _expectedToken = "expectedToken";
private string _expectedServerUrl = "https://localhost";
private string _expectedAgentName = "expectedAgentName";
private string _expectedPoolName = "poolName";
private string _expectedCollectionName = "testCollectionName";
private string _expectedProjectName = "testProjectName";
private string _expectedProjectId = "edf3f94e-d251-49df-bfce-602d6c967409";
private string _expectedMachineGroupName = "testMachineGroupName";
private string _expectedAuthType = "pat";
private string _expectedWorkFolder = "_work";
private int _expectedPoolId = 1;
@@ -78,8 +71,6 @@ namespace GitHub.Runner.Common.Tests.Listener.Configuration
_serviceControlManager = new Mock<ILinuxServiceControlManager>();
#endif
_capabilitiesManager = new CapabilitiesManager();
var expectedAgent = new TaskAgent(_expectedAgentName) { Id = 1 };
var expectedDeploymentMachine = new DeploymentMachine() { Agent = expectedAgent, Id = _expectedDeploymentMachineId };
expectedAgent.Authorization = new TaskAgentAuthorization
@@ -127,7 +118,7 @@ namespace GitHub.Runner.Common.Tests.Listener.Configuration
_agentServer.Setup(x => x.GetAgentsAsync(It.IsAny<int>(), It.IsAny<string>())).Returns(Task.FromResult(expectedAgents));
_agentServer.Setup(x => x.AddAgentAsync(It.IsAny<int>(), It.IsAny<TaskAgent>())).Returns(Task.FromResult(expectedAgent));
_agentServer.Setup(x => x.UpdateAgentAsync(It.IsAny<int>(), It.IsAny<TaskAgent>())).Returns(Task.FromResult(expectedAgent));
_agentServer.Setup(x => x.ReplaceAgentAsync(It.IsAny<int>(), It.IsAny<TaskAgent>())).Returns(Task.FromResult(expectedAgent));
rsa = new RSACryptoServiceProvider(2048);
@@ -143,8 +134,6 @@ namespace GitHub.Runner.Common.Tests.Listener.Configuration
tc.SetSingleton<IExtensionManager>(_extnMgr.Object);
tc.SetSingleton<IRunnerServer>(_agentServer.Object);
tc.SetSingleton<ILocationServer>(_locationServer.Object);
// tc.SetSingleton<IDeploymentGroupServer>(_machineGroupServer.Object);
tc.SetSingleton<ICapabilitiesManager>(_capabilitiesManager);
tc.SetSingleton<IRunnerWebProxy>(_runnerWebProxy.Object);
tc.SetSingleton<IRunnerCertificateManager>(_cert.Object);
@@ -208,12 +197,7 @@ namespace GitHub.Runner.Common.Tests.Listener.Configuration
// validate GetAgentPoolsAsync gets called once with automation pool type
_agentServer.Verify(x => x.GetAgentPoolsAsync(It.IsAny<string>(), It.Is<TaskAgentPoolType>(p => p == TaskAgentPoolType.Automation)), Times.Once);
// validate GetAgentPoolsAsync not called with deployment pool type
_agentServer.Verify(x => x.GetAgentPoolsAsync(It.IsAny<string>(), It.Is<TaskAgentPoolType>(p => p == TaskAgentPoolType.Deployment)), Times.Never);
// For build and release agent / deployment pool, tags logic should not get trigger;
// _machineGroupServer.Verify(x =>
// x.UpdateDeploymentTargetsAsync(It.IsAny<Guid>(), It.IsAny<int>(), It.IsAny<List<DeploymentMachine>>()), Times.Never);
_agentServer.Verify(x => x.AddAgentAsync(It.IsAny<int>(), It.Is<TaskAgent>(a => a.Labels.Contains("self-hosted") && a.Labels.Contains(VarUtil.OS) && a.Labels.Contains(VarUtil.OSArchitecture))), Times.Once);
}
}
}

View File

@@ -2,7 +2,6 @@
using GitHub.Services.Common;
using GitHub.Services.WebApi;
using GitHub.Runner.Listener;
using GitHub.Runner.Common.Capabilities;
using GitHub.Runner.Listener.Configuration;
using Moq;
using System;
@@ -21,7 +20,6 @@ namespace GitHub.Runner.Common.Tests.Listener
private Mock<IConfigurationManager> _config;
private Mock<IRunnerServer> _agentServer;
private Mock<ICredentialManager> _credMgr;
private Mock<ICapabilitiesManager> _capabilitiesManager;
public MessageListenerL0()
{
@@ -30,7 +28,6 @@ namespace GitHub.Runner.Common.Tests.Listener
_config.Setup(x => x.LoadSettings()).Returns(_settings);
_agentServer = new Mock<IRunnerServer>();
_credMgr = new Mock<ICredentialManager>();
_capabilitiesManager = new Mock<ICapabilitiesManager>();
}
private TestHostContext CreateTestContext([CallerMemberName] String testName = "")
@@ -39,7 +36,6 @@ namespace GitHub.Runner.Common.Tests.Listener
tc.SetSingleton<IConfigurationManager>(_config.Object);
tc.SetSingleton<IRunnerServer>(_agentServer.Object);
tc.SetSingleton<ICredentialManager>(_credMgr.Object);
tc.SetSingleton<ICapabilitiesManager>(_capabilitiesManager.Object);
return tc;
}
@@ -62,8 +58,6 @@ namespace GitHub.Runner.Common.Tests.Listener
tokenSource.Token))
.Returns(Task.FromResult(expectedSession));
_capabilitiesManager.Setup(x => x.GetCapabilitiesAsync(_settings, It.IsAny<CancellationToken>())).Returns(Task.FromResult(new Dictionary<string, string>()));
_credMgr.Setup(x => x.LoadCredentials()).Returns(new VssCredentials());
// Act.
@@ -106,8 +100,6 @@ namespace GitHub.Runner.Common.Tests.Listener
tokenSource.Token))
.Returns(Task.FromResult(expectedSession));
_capabilitiesManager.Setup(x => x.GetCapabilitiesAsync(_settings, It.IsAny<CancellationToken>())).Returns(Task.FromResult(new Dictionary<string, string>()));
_credMgr.Setup(x => x.LoadCredentials()).Returns(new VssCredentials());
// Act.
@@ -153,8 +145,6 @@ namespace GitHub.Runner.Common.Tests.Listener
tokenSource.Token))
.Returns(Task.FromResult(expectedSession));
_capabilitiesManager.Setup(x => x.GetCapabilitiesAsync(_settings, It.IsAny<CancellationToken>())).Returns(Task.FromResult(new Dictionary<string, string>()));
_credMgr.Setup(x => x.LoadCredentials()).Returns(new VssCredentials());
// Act.

View File

@@ -64,7 +64,7 @@ namespace GitHub.Runner.Common.Tests.Listener
string line;
while ((line = freader.ReadLine()) != null)
{
Assert.True(line.EndsWith(LogData));
Assert.EndsWith(LogData, line);
bytesWritten += logDataSize;
}
}

View File

@@ -0,0 +1,175 @@
using System;
using System.Runtime.CompilerServices;
using GitHub.Runner.Common;
using GitHub.Runner.Listener.Configuration;
using Xunit;
namespace GitHub.Runner.Common.Tests
{
public sealed class ServiceControlManagerL0
{
[Fact]
[Trait("Level", "L0")]
[Trait("Category", "Service")]
public void CalculateServiceName()
{
RunnerSettings settings = new RunnerSettings();
settings.AgentName = "thisiskindofalongrunnername1";
settings.ServerUrl = "https://example.githubusercontent.com/12345678901234567890123456789012345678901234567890";
settings.GitHubUrl = "https://github.com/myorganizationexample/myrepoexample";
string serviceNamePattern = "actions.runner.{0}.{1}";
string serviceDisplayNamePattern = "GitHub Actions Runner ({0}.{1})";
using (TestHostContext hc = CreateTestContext())
{
ServiceControlManager scm = new ServiceControlManager();
scm.Initialize(hc);
scm.CalculateServiceName(
settings,
serviceNamePattern,
serviceDisplayNamePattern,
out string serviceName,
out string serviceDisplayName);
var serviceNameParts = serviceName.Split('.');
// Verify name is 79 characters
Assert.Equal(79, serviceName.Length);
// Verify nothing has been shortened out
Assert.Equal("actions", serviceNameParts[0]);
Assert.Equal("runner", serviceNameParts[1]);
Assert.Equal("myorganizationexample-myrepoexample", serviceNameParts[2]); // '/' has been replaced with '-'
Assert.Equal("thisiskindofalongrunnername1", serviceNameParts[3]);
}
}
[Fact]
[Trait("Level", "L0")]
[Trait("Category", "Service")]
public void CalculateServiceName80Chars()
{
RunnerSettings settings = new RunnerSettings();
settings.AgentName = "thisiskindofalongrunnername12";
settings.ServerUrl = "https://example.githubusercontent.com/12345678901234567890123456789012345678901234567890";
settings.GitHubUrl = "https://github.com/myorganizationexample/myrepoexample";
string serviceNamePattern = "actions.runner.{0}.{1}";
string serviceDisplayNamePattern = "GitHub Actions Runner ({0}.{1})";
using (TestHostContext hc = CreateTestContext())
{
ServiceControlManager scm = new ServiceControlManager();
scm.Initialize(hc);
scm.CalculateServiceName(
settings,
serviceNamePattern,
serviceDisplayNamePattern,
out string serviceName,
out string serviceDisplayName);
// Verify name is still equal to 80 characters
Assert.Equal(80, serviceName.Length);
var serviceNameParts = serviceName.Split('.');
// Verify nothing has been shortened out
Assert.Equal("actions", serviceNameParts[0]);
Assert.Equal("runner", serviceNameParts[1]);
Assert.Equal("myorganizationexample-myrepoexample", serviceNameParts[2]); // '/' has been replaced with '-'
Assert.Equal("thisiskindofalongrunnername12", serviceNameParts[3]);
}
}
[Fact]
[Trait("Level", "L0")]
[Trait("Category", "Service")]
public void CalculateServiceNameLimitsServiceNameTo80Chars()
{
RunnerSettings settings = new RunnerSettings();
settings.AgentName = "thisisareallyreallylongbutstillvalidagentname";
settings.ServerUrl = "https://example.githubusercontent.com/12345678901234567890123456789012345678901234567890";
settings.GitHubUrl = "https://github.com/myreallylongorganizationexample/myreallylongrepoexample";
string serviceNamePattern = "actions.runner.{0}.{1}";
string serviceDisplayNamePattern = "GitHub Actions Runner ({0}.{1})";
using (TestHostContext hc = CreateTestContext())
{
ServiceControlManager scm = new ServiceControlManager();
scm.Initialize(hc);
scm.CalculateServiceName(
settings,
serviceNamePattern,
serviceDisplayNamePattern,
out string serviceName,
out string serviceDisplayName);
// Verify name has been shortened to 80 characters
Assert.Equal(80, serviceName.Length);
var serviceNameParts = serviceName.Split('.');
// Verify that each component has been shortened to a sensible length
Assert.Equal("actions", serviceNameParts[0]); // Never shortened
Assert.Equal("runner", serviceNameParts[1]); // Never shortened
Assert.Equal("myreallylongorganizationexample-myreallylongr", serviceNameParts[2]); // First 45 chars, '/' has been replaced with '-'
Assert.Equal("thisisareallyreally", serviceNameParts[3]); // Remainder of unused chars
}
}
// Special 'defensive' test that verifies we can gracefully handle creating service names
// in case GitHub.com changes its org/repo naming convention in the future,
// and some of these characters may be invalid for service names
// Not meant to test character set exhaustively -- it's just here to exercise the sanitizing logic
[Fact]
[Trait("Level", "L0")]
[Trait("Category", "Service")]
public void CalculateServiceNameSanitizeOutOfRangeChars()
{
RunnerSettings settings = new RunnerSettings();
settings.AgentName = "name";
settings.ServerUrl = "https://example.githubusercontent.com/12345678901234567890123456789012345678901234567890";
settings.GitHubUrl = "https://github.com/org!@$*+[]()/repo!@$*+[]()";
string serviceNamePattern = "actions.runner.{0}.{1}";
string serviceDisplayNamePattern = "GitHub Actions Runner ({0}.{1})";
using (TestHostContext hc = CreateTestContext())
{
ServiceControlManager scm = new ServiceControlManager();
scm.Initialize(hc);
scm.CalculateServiceName(
settings,
serviceNamePattern,
serviceDisplayNamePattern,
out string serviceName,
out string serviceDisplayName);
var serviceNameParts = serviceName.Split('.');
// Verify service name parts are sanitized correctly
Assert.Equal("actions", serviceNameParts[0]);
Assert.Equal("runner", serviceNameParts[1]);
Assert.Equal("org----------repo---------", serviceNameParts[2]); // Chars replaced with '-'
Assert.Equal("name", serviceNameParts[3]);
}
}
private TestHostContext CreateTestContext([CallerMemberName] string testName = "")
{
TestHostContext hc = new TestHostContext(this, testName);
return hc;
}
}
}

View File

@@ -1,5 +1,4 @@
using GitHub.Runner.Listener;
using GitHub.Runner.Common.Capabilities;
using GitHub.Runner.Listener.Configuration;
using GitHub.Runner.Worker;
using GitHub.Runner.Worker.Handlers;
@@ -44,7 +43,6 @@ namespace GitHub.Runner.Common.Tests
typeof(IHostContext),
typeof(ITraceManager),
typeof(IThrottlingReporter),
typeof(ICapabilitiesProvider)
};
Validate(
assembly: typeof(IHostContext).GetTypeInfo().Assembly,

View File

@@ -31,8 +31,8 @@ namespace GitHub.Runner.Common.Tests.Util
var connect = VssUtil.CreateConnection(new Uri("https://github.com/actions/runner"), new VssCredentials());
// Assert.
Assert.Equal(connect.Settings.MaxRetryRequest.ToString(), "10");
Assert.Equal(connect.Settings.SendTimeout.TotalSeconds.ToString(), "360");
Assert.Equal("10", connect.Settings.MaxRetryRequest.ToString());
Assert.Equal("360", connect.Settings.SendTimeout.TotalSeconds.ToString());
trace.Info("Set httpretry to 100.");
Environment.SetEnvironmentVariable("GITHUB_ACTIONS_RUNNER_HTTP_RETRY", "100");
@@ -42,8 +42,8 @@ namespace GitHub.Runner.Common.Tests.Util
connect = VssUtil.CreateConnection(new Uri("https://github.com/actions/runner"), new VssCredentials());
// Assert.
Assert.Equal(connect.Settings.MaxRetryRequest.ToString(), "10");
Assert.Equal(connect.Settings.SendTimeout.TotalSeconds.ToString(), "1200");
Assert.Equal("10", connect.Settings.MaxRetryRequest.ToString());
Assert.Equal("1200", connect.Settings.SendTimeout.TotalSeconds.ToString());
}
finally
{

View File

@@ -5,6 +5,7 @@ using GitHub.DistributedTask.WebApi;
using GitHub.Runner.Worker;
using Moq;
using Xunit;
using Pipelines = GitHub.DistributedTask.Pipelines;
namespace GitHub.Runner.Common.Tests.Worker
{
@@ -146,5 +147,159 @@ namespace GitHub.Runner.Common.Tests.Worker
Assert.True(commandManager.TryProcessCommand(_ec.Object, "##[set-env name=foo]bar"));
}
}
[Fact]
[Trait("Level", "L0")]
[Trait("Category", "Worker")]
public void EchoProcessCommand()
{
using (TestHostContext _hc = new TestHostContext(this))
{
var extensionManager = new Mock<IExtensionManager>();
var echoCommand = new EchoCommandExtension();
echoCommand.Initialize(_hc);
extensionManager.Setup(x => x.GetExtensions<IActionCommandExtension>())
.Returns(new List<IActionCommandExtension>() { echoCommand });
_hc.SetSingleton<IExtensionManager>(extensionManager.Object);
Mock<IExecutionContext> _ec = new Mock<IExecutionContext>();
_ec.Setup(x => x.Write(It.IsAny<string>(), It.IsAny<string>()))
.Returns((string tag, string line) =>
{
_hc.GetTrace().Info($"{tag} {line}");
return 1;
});
_ec.SetupAllProperties();
ActionCommandManager commandManager = new ActionCommandManager();
commandManager.Initialize(_hc);
Assert.False(_ec.Object.EchoOnActionCommand);
Assert.True(commandManager.TryProcessCommand(_ec.Object, "::echo::on"));
Assert.True(_ec.Object.EchoOnActionCommand);
Assert.True(commandManager.TryProcessCommand(_ec.Object, "::echo::off"));
Assert.False(_ec.Object.EchoOnActionCommand);
Assert.True(commandManager.TryProcessCommand(_ec.Object, "::echo::ON"));
Assert.True(_ec.Object.EchoOnActionCommand);
Assert.True(commandManager.TryProcessCommand(_ec.Object, "::echo::Off "));
Assert.False(_ec.Object.EchoOnActionCommand);
}
}
[Fact]
[Trait("Level", "L0")]
[Trait("Category", "Worker")]
public void EchoProcessCommandDebugOn()
{
using (TestHostContext _hc = new TestHostContext(this))
{
// Set up a few things
// 1. Job request message (with ACTIONS_STEP_DEBUG = true)
TaskOrchestrationPlanReference plan = new TaskOrchestrationPlanReference();
TimelineReference timeline = new TimelineReference();
JobEnvironment environment = new JobEnvironment();
environment.SystemConnection = new ServiceEndpoint();
List<TaskInstance> tasks = new List<TaskInstance>();
Guid JobId = Guid.NewGuid();
string jobName = "some job name";
var jobRequest = Pipelines.AgentJobRequestMessageUtil.Convert(new AgentJobRequestMessage(plan, timeline, JobId, jobName, jobName, environment, tasks));
jobRequest.Resources.Repositories.Add(new Pipelines.RepositoryResource()
{
Alias = Pipelines.PipelineConstants.SelfAlias,
Id = "github",
Version = "sha1"
});
jobRequest.ContextData["github"] = new Pipelines.ContextData.DictionaryContextData();
jobRequest.Variables["ACTIONS_STEP_DEBUG"] = "true";
// Some service dependencies
var jobServerQueue = new Mock<IJobServerQueue>();
jobServerQueue.Setup(x => x.QueueTimelineRecordUpdate(It.IsAny<Guid>(), It.IsAny<TimelineRecord>()));
_hc.SetSingleton(jobServerQueue.Object);
var extensionManager = new Mock<IExtensionManager>();
var echoCommand = new EchoCommandExtension();
echoCommand.Initialize(_hc);
extensionManager.Setup(x => x.GetExtensions<IActionCommandExtension>())
.Returns(new List<IActionCommandExtension>() { echoCommand });
_hc.SetSingleton<IExtensionManager>(extensionManager.Object);
var configurationStore = new Mock<IConfigurationStore>();
configurationStore.Setup(x => x.GetSettings()).Returns(new RunnerSettings());
_hc.SetSingleton(configurationStore.Object);
var pagingLogger = new Mock<IPagingLogger>();
_hc.EnqueueInstance(pagingLogger.Object);
ActionCommandManager commandManager = new ActionCommandManager();
commandManager.Initialize(_hc);
var _ec = new Runner.Worker.ExecutionContext();
_ec.Initialize(_hc);
// Initialize the job (to exercise logic that sets EchoOnActionCommand)
_ec.InitializeJob(jobRequest, System.Threading.CancellationToken.None);
_ec.Complete();
Assert.True(_ec.EchoOnActionCommand);
Assert.True(commandManager.TryProcessCommand(_ec, "::echo::off"));
Assert.False(_ec.EchoOnActionCommand);
Assert.True(commandManager.TryProcessCommand(_ec, "::echo::on"));
Assert.True(_ec.EchoOnActionCommand);
}
}
[Fact]
[Trait("Level", "L0")]
[Trait("Category", "Worker")]
public void EchoProcessCommandInvalid()
{
using (TestHostContext _hc = new TestHostContext(this))
{
var extensionManager = new Mock<IExtensionManager>();
var echoCommand = new EchoCommandExtension();
echoCommand.Initialize(_hc);
extensionManager.Setup(x => x.GetExtensions<IActionCommandExtension>())
.Returns(new List<IActionCommandExtension>() { echoCommand });
_hc.SetSingleton<IExtensionManager>(extensionManager.Object);
Mock<IExecutionContext> _ec = new Mock<IExecutionContext>();
_ec.Setup(x => x.Write(It.IsAny<string>(), It.IsAny<string>()))
.Returns((string tag, string line) =>
{
_hc.GetTrace().Info($"{tag} {line}");
return 1;
});
_ec.SetupAllProperties();
ActionCommandManager commandManager = new ActionCommandManager();
commandManager.Initialize(_hc);
// Echo commands below are considered "processed", but are invalid
// 1. Invalid echo value
Assert.True(commandManager.TryProcessCommand(_ec.Object, "::echo::invalid"));
Assert.Equal(TaskResult.Failed, _ec.Object.CommandResult);
Assert.False(_ec.Object.EchoOnActionCommand);
// 2. No value
Assert.True(commandManager.TryProcessCommand(_ec.Object, "::echo::"));
Assert.Equal(TaskResult.Failed, _ec.Object.CommandResult);
Assert.False(_ec.Object.EchoOnActionCommand);
}
}
}
}

Some files were not shown because too many files have changed in this diff Show More