Compare commits

..

60 Commits

Author SHA1 Message Date
TingluoHuang
a3c2479a29 prepare 2.165.0 runner release. 2020-01-27 22:14:06 -05:00
Thomas Boop
c45aebc9ab Update Base64 Encoders to deal with suffixes (#284)
* Update Base64 Encoders to deal with suffixes

* Set UriDataEscape to public for unit tests
2020-01-27 21:38:31 -05:00
Thomas Boop
b676ab3d33 Add ADR For Base64 Masking Improvements (#297)
* Base64 Secret Masking ADR

* slight addendums

* Update and rename 0000-base64-masking-trailing-characters.md to 0297-base64-masking-trailing-characters.md
2020-01-27 21:38:01 -05:00
Tingluo Huang
0a6bac355d include step.env as part of env context. (#300) 2020-01-27 15:54:28 -05:00
Tingluo Huang
eb78d19b17 Set both http_proxy and HTTP_PROXY env for runner/worker processes. (#298) 2020-01-27 11:04:35 -05:00
David Kale
17970ad1f9 Set http_proxy and related env vars for containers (#304)
* WIP add in basic passthrough for proxy env vars

* Add http_proxy vars after container env is created
2020-01-27 10:56:18 -05:00
Alberto Gimeno
2e0e8eb822 Change prompt message when removing a runner (#303)
* Change prompt message to be consistent with the UI/API

* Update test
2020-01-26 18:40:06 -05:00
Tingluo Huang
2a506cc556 Update 0278-env-context.md (#299) 2020-01-22 20:09:09 -05:00
Tingluo Huang
43dd34820b Follow redirect link for download runner package
GitHub release assets download link returns 302 to storage.
2020-01-21 12:23:30 -05:00
David Kale
746c9d9ec0 Trace javascript action exit code instead of user logs (#290)
* Trace javascript action exit code instead of user logs

* Debug instead of trace
2020-01-21 11:23:30 -05:00
Tingluo Huang
fa2ecfcc4c Fix page log name isn't unqiue. (#295) 2020-01-21 11:08:37 -05:00
Alberto Gimeno
c59c0e2ded Support action.yaml file (#288)
* Support action.yaml file

* L0 tests.

* l0

Co-authored-by: Tingluo Huang <tingluohuang@github.com>
2020-01-20 12:22:59 -05:00
Tingluo Huang
7a382facb3 default post-job action's condition to always(). (#293) 2020-01-20 09:28:29 -05:00
Tingluo Huang
e9ae42693f change hashFiles() expression to use @actions/glob. (#268) 2020-01-19 10:32:13 -05:00
David Kale
9cafe8c028 Update --help (#282)
* Update --help

* Tiny indent

* Remove once option
2020-01-17 13:55:11 -05:00
Thomas Boop
1484c3fb03 Add Temporary Step to fix build on windows (#285)
* Add Temporary Step to fix build on windows

* Update workflow
2020-01-16 14:19:54 -05:00
eric sciple
53d632706d adr: command input echoing (#280) 2020-01-14 14:55:17 -05:00
eric sciple
d6179242ca adr: hashFiles expression function (#279) 2020-01-14 14:54:37 -05:00
eric sciple
0da38a6924 adr: add env context (#278) 2020-01-14 14:54:29 -05:00
eric sciple
b19e5d7924 ADR: Run action shell options (#277) 2020-01-14 14:54:20 -05:00
eric sciple
80ac4a8964 ADR: problem matchers (#276) 2020-01-14 14:54:04 -05:00
eric sciple
02639a2092 translate problem matcher file to host path (#272) 2020-01-13 15:24:57 -05:00
eric sciple
a727194742 allow container to be null/empty (#266) 2020-01-12 23:02:36 -05:00
eric sciple
a9c58d7398 Handle escaped '%' in commands data section (#200) 2020-01-12 03:30:26 -05:00
Bryan MacFarlane
e15414eb5e proxy support ADR (#263) 2020-01-09 11:52:04 -05:00
Tingluo Huang
4ab1e645c3 Fix typo in error message. (#260) 2020-01-07 15:13:05 -05:00
Tingluo Huang
584f6b6ca3 upload log on runner force kill worker. (#255) 2020-01-06 13:04:23 -05:00
Tingluo Huang
abc65839f3 detect source file path without using env. (#257) 2020-01-06 12:56:15 -05:00
Tingluo Huang
06292aa118 Expose whether debug is on/off via RUNNER_DEBUG. (#253) 2020-01-03 21:23:46 -05:00
Joseph Petersen
ac1a076a3b Treat warnings as errors (#249)
* Treat warnings as errors

* fix warnings
2019-12-21 09:51:41 -05:00
Joseph Petersen
300bc67950 ignore .idea folder (#248)
created by rider
2019-12-21 09:50:07 -05:00
Tim Heuer
289c7f36a2 Minor typo (#247)
capitalize .NET
2019-12-20 20:20:44 -05:00
Julio Barba
b89d7fb8ef Remove old "v1" artifact download/publish code (#212)
* Remove old v1 artifact download/publish code
* Remove the Build2 REST API SDK
2019-12-19 16:02:00 -05:00
Josh Gross
5fd705bb84 Update LICENSE (#242) 2019-12-19 15:22:49 -05:00
Julio Barba
9e37732401 Verify that has Windows service started successfully (#236) 2019-12-19 14:34:26 -05:00
Bryan MacFarlane
6c70d53eea clean up some unneeded dockerfiles 2019-12-19 09:16:43 -05:00
Bryan MacFarlane
f791e2d512 update contributions md 2019-12-19 09:03:14 -05:00
Bryan MacFarlane
f1e36651ad build workflow ignores md changes 2019-12-19 08:54:43 -05:00
Bryan MacFarlane
be24fea81b update issue templates 2019-12-19 08:34:17 -05:00
Bryan MacFarlane
84ca2c05ce update readme 2019-12-19 08:20:39 -05:00
Bryan MacFarlane
2249560cec update readme 2019-12-18 23:15:30 -05:00
Bryan MacFarlane
2d4b821abe update readme 2019-12-18 23:13:22 -05:00
Bryan MacFarlane
371bf8e607 update readme and contributions 2019-12-18 22:49:31 -05:00
Tingluo Huang
9ba11da490 move .sln file. (#238) 2019-12-18 20:13:57 -05:00
Tingluo Huang
40302373ba Create releaseVersion (#237) 2019-12-18 15:39:26 -05:00
Tingluo Huang
9a08f7418f delete unused files. (#235) 2019-12-18 15:28:36 -05:00
Tingluo Huang
80b6038cdc consume dotnet core 3.1 in runner. (#213) 2019-12-18 15:09:03 -05:00
David Kale
70a09bc5ac shell from prependpath (#231)
* Prepend path before locating shell tool

* Join optional prepended path to path before searching it

* Use prepended path when whiching shell tool

* Addition prependPath location

* Also use prepended paths when writing out run details

* Small tweak to undo unnecessary change
2019-12-18 15:00:12 -05:00
Tingluo Huang
c6cf1eb3f1 Release runner using actions (#223)
* update runner release workflow

* trim script.

* feedback.
2019-12-18 14:56:37 -05:00
Julio Barba
50d979f1b2 Bring back tools folder fallback code (#232) 2019-12-17 18:21:13 -05:00
Tingluo Huang
91b7e7a07a delete more unused code. (#230)
* delete more unused code.

* pr feedback.
2019-12-17 16:47:14 -05:00
Tingluo Huang
d0a4a41a63 delete un-used code. (#218) 2019-12-16 17:05:26 -05:00
Julio Barba
c3c66bb14a Replace remaining Agent -> Runner references (#229) 2019-12-16 15:45:00 -05:00
Tingluo Huang
86df779fe9 expose github.run_id and github.run_number to action runtime env. (#224) 2019-12-16 15:23:55 -05:00
David Kale
1918906505 First pass (#221) 2019-12-16 14:53:19 -05:00
Julio Barba
9448135fcd Replace a few more instances Agent -> Runner (#228) 2019-12-16 11:51:08 -05:00
Tingluo Huang
f3aedd86fd Update AGENT_ALLOW_RUNASROOT to RUNNER_ALLOW_RUNASROOT (#227)
* Update config.sh

* Update run.sh
2019-12-16 11:27:46 -05:00
Mike Coutermarsh
d778f13dee Remove runner flow: Change from PAT to "deletion token" in prompt (#225)
* Updating prompt deletion token

Currently if you leave the token off the command, we're showing "Enter your personal access token:"

Which won't work. This updates prompt to "deletion token"

* Call correct function in test

* Fix command text in test
2019-12-15 21:38:42 -05:00
Tingluo Huang
9bbbca9e5d Prepare 2.163.0 runner release. 2019-12-12 15:20:39 -05:00
Tingluo Huang
2cac011558 Load and set env from .env file before creating HostContext. (#220) 2019-12-12 14:56:45 -05:00
354 changed files with 42481 additions and 7836 deletions

View File

@@ -1,10 +0,0 @@
## Runner Version and Platform
Version of your runner?
OS of the machine running the runner? OSX/Windows/Linux/...
## What's not working?
Please include error messages and screenshots.
## Runner and Worker's Diagnostic Logs
Logs are located in the runner's `_diag` folder. The runner logs are prefixed with `Runner_` and the worker logs are prefixed with `Worker_`. All sensitive information should already be masked out, but please double-check before pasting here.

34
.github/ISSUE_TEMPLATE/bug_report.md vendored Normal file
View File

@@ -0,0 +1,34 @@
---
name: Bug report
about: Create a report to help us improve
title: ''
labels: bug
assignees: ''
---
**Describe the bug**
A clear and concise description of what the bug is.
**To Reproduce**
Steps to reproduce the behavior:
1. Go to '...'
2. Run '....'
3. See error
**Expected behavior**
A clear and concise description of what you expected to happen.
## Runner Version and Platform
Version of your runner?
OS of the machine running the runner? OSX/Windows/Linux/...
## What's not working?
Please include error messages and screenshots.
## Job Log Output
If applicable, include the relevant part of the job / step log output here. All sensitive information should already be masked out, but please double-check before pasting here.
## Runner and Worker's Diagnostic Logs
If applicable, add relevant diagnostic log information. Logs are located in the runner's `_diag` folder. The runner logs are prefixed with `Runner_` and the worker logs are prefixed with `Worker_`. Each job run correlates to a worker log. All sensitive information should already be masked out, but please double-check before pasting here.

View File

@@ -0,0 +1,27 @@
---
name: Feature Request
about: Create a request to help us improve
title: ''
labels: enhancement
assignees: ''
---
Thank you 🙇‍♀ for wanting to create a feature in this repository. Before you do, please ensure you are filing the issue in the right place. Issues should only be opened on if the issue **relates to code in this repository**.
* If you have found a security issue [please submit it here](https://hackerone.com/github)
* If you have questions or issues with the service, writing workflows or actions, then please [visit the GitHub Community Forum's Actions Board](https://github.community/t5/GitHub-Actions/bd-p/actions)
* If you are having an issue or question about GitHub Actions then please [contact customer support](https://help.github.com/en/actions/automating-your-workflow-with-github-actions/about-github-actions#contacting-support)
If you have a feature request that is relevant to this repository, the runner, then please include the information below:
**Describe the enhancement**
A clear and concise description of what the features or enhancement you need.
**Code Snippet**
If applicable, add a code snippet.
**Additional information**
Add any other context about the feature here.
NOTE: if the feature request has been agreed upon then the assignee will create an ADR. See docs/adrs/README.md

View File

@@ -5,9 +5,13 @@ on:
branches:
- master
- releases/*
paths-ignore:
- '**.md'
pull_request:
branches:
- '*'
paths-ignore:
- '**.md'
jobs:
build:
@@ -39,6 +43,14 @@ jobs:
steps:
- uses: actions/checkout@v1
# Set Path workaround for https://github.com/actions/virtual-environments/issues/263
- run: |
echo "::add-path::C:\Program Files\Git\mingw64\bin"
echo "::add-path::C:\Program Files\Git\usr\bin"
echo "::add-path::C:\Program Files\Git\bin"
if: matrix.os == 'windows-latest'
name: "Temp step to Set Path for Windows"
# Build runner layout
- name: Build & Layout Release
run: |

View File

@@ -3,10 +3,47 @@ name: Runner CD
on:
push:
paths:
- src/runnerversion_block # Change this to src/runnerversion when we are ready.
- releaseVersion
jobs:
check:
if: startsWith(github.ref, 'refs/heads/releases/') || github.ref == 'refs/heads/master'
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
# Make sure ./releaseVersion match ./src/runnerversion
# Query GitHub release ensure version is not used
- name: Check version
uses: actions/github-script@0.3.0
with:
github-token: ${{secrets.GITHUB_TOKEN}}
script: |
const core = require('@actions/core')
const fs = require('fs');
const runnerVersion = fs.readFileSync('${{ github.workspace }}/src/runnerversion', 'utf8').replace(/\n$/g, '')
const releaseVersion = fs.readFileSync('${{ github.workspace }}/releaseVersion', 'utf8').replace(/\n$/g, '')
if (runnerVersion != releaseVersion) {
console.log('Request Release Version: ' + releaseVersion + '\nCurrent Runner Version: ' + runnerVersion)
core.setFailed('Version mismatch! Make sure ./releaseVersion match ./src/runnerVersion')
return
}
try {
const release = await github.repos.getReleaseByTag({
owner: '${{ github.event.repository.owner.name }}',
repo: '${{ github.event.repository.name }}',
tag: 'v' + runnerVersion
})
core.setFailed('Release with same tag already created: ' + release.data.html_url)
} catch (e) {
// We are good to create the release if release with same tag doesn't exists
if (e.status != 404) {
throw e
}
}
build:
needs: check
strategy:
matrix:
runtime: [ linux-x64, linux-arm64, linux-arm, win-x64, osx-x64 ]
@@ -52,7 +89,7 @@ jobs:
- name: Package Release
if: github.event_name != 'pull_request'
run: |
${{ matrix.devScript }} package Release
${{ matrix.devScript }} package Release ${{ matrix.runtime }}
working-directory: src
# Upload runner package tar.gz/zip as artifact.
@@ -66,14 +103,17 @@ jobs:
release:
needs: build
runs-on: linux-latest
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
# Download runner package tar.gz/zip produced by 'build' job
- name: Download Artifact
uses: actions/download-artifact@v1
with:
name: runner-packages
path: ./
# Create ReleaseNote file
- name: Create ReleaseNote
@@ -82,103 +122,74 @@ jobs:
with:
github-token: ${{secrets.GITHUB_TOKEN}}
script: |
const core = require('@actions/core')
const fs = require('fs');
// Get runner version from ./src/runnerVersion file
const versionContent = await github.repos.getContents({
owner: '${{ github.event.repository.owner.name }}',
repo: '${{ github.event.repository.name }}',
path: 'src/runnerversion'
ref: ${{ github.sha }}
})
const runnerVersion = Buffer.from(versionContent.data.content, 'base64').toString()
console.log("Runner Version ' + runnerVersion)
const runnerVersion = fs.readFileSync('${{ github.workspace }}/src/runnerversion', 'utf8').replace(/\n$/g, '')
const releaseNote = fs.readFileSync('${{ github.workspace }}/releaseNote.md', 'utf8').replace(/<RUNNER_VERSION>/g, runnerVersion)
console.log(releaseNote)
core.setOutput('version', runnerVersion);
// Query GitHub release ensure version is bumped
const latestRelease = await github.repos.getLatestRelease({
owner: '${{ github.event.repository.owner.name }}',
repo: '${{ github.event.repository.name }}'
})
console.log(latestRelease.name)
const latestReleaseVersion = latestRelease.name.substring(1)
const vLatest = latestReleaseVersion.split('.')
const vNew = runnerVersion.split('.')
let versionBumped = true
for (let i = 0; i < 3; ++i) {
var v1 = parseInt(vLatest[i], 10);
var v2 = parseInt(vNew[i], 10);
if (v2 > v1) {
console.log(runnerVersion + " > " + latestReleaseVersion + "(Latest)")
break
}
if (v1 > v2) {
versionBumped = false
core.setFailed(runnerVersion + " < " + latestReleaseVersion + "(Latest)")
break
}
}
// Generate release note
if (versionBumped) {
const releaseNoteContent = await github.repos.getContents({
owner: '${{ github.event.repository.owner.name }}',
repo: '${{ github.event.repository.name }}',
path: 'releaseNote.md'
ref: ${{ github.sha }}
})
const releaseNote = Buffer.from(releaseNoteContent.data.content, 'base64').toString().replace("<RUNNER_VERSION>", runnerVersion)
console.log(releaseNote)
core.setOutput('note', releaseNote);
}
core.setOutput('note', releaseNote);
# Create GitHub release
- uses: actions/create-release@v1
- uses: actions/create-release@master
id: createRelease
name: Create ${{ steps.releaseNote.outputs.version }} Runner Release
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:
tag_name: "v${{ steps.releaseNote.outputs.version }}"
release_name: "v${{ steps.releaseNote.outputs.version }}"
body: ${{ steps.releaseNote.outputs.note }}
body: |
${{ steps.releaseNote.outputs.note }}
prerelease: true
# Upload release assets
- name: Upload Release Asset (win-x64)
uses: actions/upload-release-asset@v1.0.1
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:
upload_url: ${{ steps.createRelease.outputs.upload_url }}
asset_path: ./actions-runner-win-x64-${{ steps.releaseNote.outputs.version }}.zip
asset_path: ${{ github.workspace }}/actions-runner-win-x64-${{ steps.releaseNote.outputs.version }}.zip
asset_name: actions-runner-win-x64-${{ steps.releaseNote.outputs.version }}.zip
asset_content_type: application/octet-stream
- name: Upload Release Asset (linux-x64)
uses: actions/upload-release-asset@v1.0.1
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:
upload_url: ${{ steps.createRelease.outputs.upload_url }}
asset_path: ./actions-runner-linux-x64-${{ steps.releaseNote.outputs.version }}.zip
asset_name: actions-runner-linux-x64-${{ steps.releaseNote.outputs.version }}.zip
asset_path: ${{ github.workspace }}/actions-runner-linux-x64-${{ steps.releaseNote.outputs.version }}.tar.gz
asset_name: actions-runner-linux-x64-${{ steps.releaseNote.outputs.version }}.tar.gz
asset_content_type: application/octet-stream
- name: Upload Release Asset (mac-x64)
- name: Upload Release Asset (osx-x64)
uses: actions/upload-release-asset@v1.0.1
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:
upload_url: ${{ steps.createRelease.outputs.upload_url }}
asset_path: ./actions-runner-mac-x64-${{ steps.releaseNote.outputs.version }}.zip
asset_name: actions-runner-mac-x64-${{ steps.releaseNote.outputs.version }}.zip
asset_path: ${{ github.workspace }}/actions-runner-osx-x64-${{ steps.releaseNote.outputs.version }}.tar.gz
asset_name: actions-runner-osx-x64-${{ steps.releaseNote.outputs.version }}.tar.gz
asset_content_type: application/octet-stream
- name: Upload Release Asset (linux-arm)
uses: actions/upload-release-asset@v1.0.1
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:
upload_url: ${{ steps.createRelease.outputs.upload_url }}
asset_path: ./actions-runner-linux-arm-${{ steps.releaseNote.outputs.version }}.zip
asset_name: actions-runner-linux-arm-${{ steps.releaseNote.outputs.version }}.zip
asset_path: ${{ github.workspace }}/actions-runner-linux-arm-${{ steps.releaseNote.outputs.version }}.tar.gz
asset_name: actions-runner-linux-arm-${{ steps.releaseNote.outputs.version }}.tar.gz
asset_content_type: application/octet-stream
- name: Upload Release Asset (linux-arm64)
uses: actions/upload-release-asset@v1.0.1
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:
upload_url: ${{ steps.createRelease.outputs.upload_url }}
asset_path: ./actions-runner-linux-arm64-${{ steps.releaseNote.outputs.version }}.zip
asset_name: actions-runner-linux-arm64-${{ steps.releaseNote.outputs.version }}.zip
asset_path: ${{ github.workspace }}/actions-runner-linux-arm64-${{ steps.releaseNote.outputs.version }}.tar.gz
asset_name: actions-runner-linux-arm64-${{ steps.releaseNote.outputs.version }}.tar.gz
asset_content_type: application/octet-stream

8
.gitignore vendored
View File

@@ -1,12 +1,19 @@
# build output
**/bin
**/obj
**/libs
**/lib
# editors
**/*.xproj
**/*.xproj.user
**/.vs
**/.vscode
**/*.error
**/*.json.pretty
.idea/
# output
node_modules
_downloads
_layout
@@ -19,4 +26,3 @@ TestLogs
#generated
src/Runner.Sdk/BuildConstants.cs

View File

@@ -1,5 +1,5 @@
The MIT License (MIT)
Copyright (c) Microsoft Corporation
Copyright (c) 2019 GitHub
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
@@ -17,4 +17,4 @@ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
SOFTWARE.

View File

@@ -1,31 +1,25 @@
# GitHub Actions Runner
<p align="center">
<img src="docs/res/github-graph.png">
</p>
# GitHub Actions Runner
[![Actions Status](https://github.com/actions/runner/workflows/Runner%20CI/badge.svg)](https://github.com/actions/runner/actions)
The runner is the application that runs a job from a GitHub Actions workflow. The runner can run on the [hosted machine pools](https://github.com/actions/virtual-environments) or run on [self-hosted environments](https://help.github.com/en/actions/automating-your-workflow-with-github-actions/about-self-hosted-runners).
## Get Started
![win](docs/res/win_sm.png) [Pre-reqs](docs/start/envwin.md) | [Download](https://github.com/actions/runner/releases/latest)
For more information about installing and using self-hosted runners, see [Adding self-hosted runners](https://help.github.com/en/actions/automating-your-workflow-with-github-actions/adding-self-hosted-runners) and [Using self-hosted runners in a workflow](https://help.github.com/en/actions/automating-your-workflow-with-github-actions/using-self-hosted-runners-in-a-workflow)
![macOS](docs/res/apple_sm.png) [Pre-reqs](docs/start/envosx.md) | [Download](https://github.com/actions/runner/releases/latest)
Runner releases:
![linux](docs/res/linux_sm.png) [Pre-reqs](docs/start/envlinux.md) | [Download](https://github.com/actions/runner/releases/latest)
![win](docs/res/win_sm.png) [Pre-reqs](docs/start/envwin.md) | [Download](https://github.com/actions/runner/releases)
**Configure:**
![macOS](docs/res/apple_sm.png) [Pre-reqs](docs/start/envosx.md) | [Download](https://github.com/actions/runner/releases)
*MacOS and Linux*
```bash
./config.sh
```
*Windows*
```bash
config.cmd
```
![linux](docs/res/linux_sm.png) [Pre-reqs](docs/start/envlinux.md) | [Download](https://github.com/actions/runner/releases)
## Contribute
For developers that want to contribute, [read here](docs/contribute.md) on how to build and test.
We accept contributions in the form of issues and pull requests. [Read more here](docs/contribute.md) before contributing.

View File

@@ -1,32 +0,0 @@
[
{
"name": "actions-runner-win-x64-<RUNNER_VERSION>.zip",
"platform": "win-x64",
"version": "<RUNNER_VERSION>",
"downloadUrl": "https://githubassets.azureedge.net/runners/<RUNNER_VERSION>/actions-runner-win-x64-<RUNNER_VERSION>.zip"
},
{
"name": "actions-runner-osx-x64-<RUNNER_VERSION>.tar.gz",
"platform": "osx-x64",
"version": "<RUNNER_VERSION>",
"downloadUrl": "https://githubassets.azureedge.net/runners/<RUNNER_VERSION>/actions-runner-osx-x64-<RUNNER_VERSION>.tar.gz"
},
{
"name": "actions-runner-linux-x64-<RUNNER_VERSION>.tar.gz",
"platform": "linux-x64",
"version": "<RUNNER_VERSION>",
"downloadUrl": "https://githubassets.azureedge.net/runners/<RUNNER_VERSION>/actions-runner-linux-x64-<RUNNER_VERSION>.tar.gz"
},
{
"name": "actions-runner-linux-arm64-<RUNNER_VERSION>.tar.gz",
"platform": "linux-arm64",
"version": "<RUNNER_VERSION>",
"downloadUrl": "https://githubassets.azureedge.net/runners/<RUNNER_VERSION>/actions-runner-linux-arm64-<RUNNER_VERSION>.tar.gz"
},
{
"name": "actions-runner-linux-arm-<RUNNER_VERSION>.tar.gz",
"platform": "linux-arm",
"version": "<RUNNER_VERSION>",
"downloadUrl": "https://githubassets.azureedge.net/runners/<RUNNER_VERSION>/actions-runner-linux-arm-<RUNNER_VERSION>.tar.gz"
}
]

View File

@@ -1,237 +0,0 @@
stages:
- stage: Build
jobs:
################################################################################
- job: build_windows_agent_x64
################################################################################
displayName: Windows Agent (x64)
pool:
vmImage: vs2017-win2016
steps:
# Steps template for windows platform
- template: windows.template.yml
parameters:
targetRuntime: win-x64
# Package dotnet core windows dependency (VC++ Redistributable)
- powershell: |
Write-Host "Downloading 'VC++ Redistributable' package."
$outDir = Join-Path -Path $env:TMP -ChildPath ([Guid]::NewGuid())
New-Item -Path $outDir -ItemType directory
$outFile = Join-Path -Path $outDir -ChildPath "ucrt.zip"
Invoke-WebRequest -Uri https://vstsagenttools.blob.core.windows.net/tools/ucrt/ucrt_x64.zip -OutFile $outFile
Write-Host "Unzipping 'VC++ Redistributable' package to agent layout."
$unzipDir = Join-Path -Path $outDir -ChildPath "unzip"
Add-Type -AssemblyName System.IO.Compression.FileSystem
[System.IO.Compression.ZipFile]::ExtractToDirectory($outFile, $unzipDir)
$agentLayoutBin = Join-Path -Path $(Build.SourcesDirectory) -ChildPath "_layout\bin"
Copy-Item -Path $unzipDir -Destination $agentLayoutBin -Force
displayName: Package UCRT
# Create agent package zip
- script: dev.cmd package Release win-x64
workingDirectory: src
displayName: Package Release
# Upload agent package zip as build artifact
- task: PublishBuildArtifacts@1
displayName: Publish Artifact (Windows x64)
inputs:
pathToPublish: _package
artifactName: runners
artifactType: container
################################################################################
- job: build_linux_agent_x64
################################################################################
displayName: Linux Agent (x64)
pool:
vmImage: ubuntu-16.04
steps:
# Steps template for non-windows platform
- template: nonwindows.template.yml
parameters:
targetRuntime: linux-x64
# Create agent package zip
- script: ./dev.sh package Release linux-x64
workingDirectory: src
displayName: Package Release
# Upload agent package zip as build artifact
- task: PublishBuildArtifacts@1
displayName: Publish Artifact (Linux x64)
inputs:
pathToPublish: _package
artifactName: runners
artifactType: container
################################################################################
- job: build_linux_agent_arm64
################################################################################
displayName: Linux Agent (arm64)
pool:
vmImage: ubuntu-16.04
steps:
# Steps template for non-windows platform
- template: nonwindows.template.yml
parameters:
targetRuntime: linux-arm64
# Create agent package zip
- script: ./dev.sh package Release linux-arm64
workingDirectory: src
displayName: Package Release
# Upload agent package zip as build artifact
- task: PublishBuildArtifacts@1
displayName: Publish Artifact (Linux ARM64)
inputs:
pathToPublish: _package
artifactName: runners
artifactType: container
################################################################################
- job: build_linux_agent_arm
################################################################################
displayName: Linux Agent (arm)
pool:
vmImage: ubuntu-16.04
steps:
# Steps template for non-windows platform
- template: nonwindows.template.yml
parameters:
targetRuntime: linux-arm
# Create agent package zip
- script: ./dev.sh package Release linux-arm
workingDirectory: src
displayName: Package Release
# Upload agent package zip as build artifact
- task: PublishBuildArtifacts@1
displayName: Publish Artifact (Linux ARM)
inputs:
pathToPublish: _package
artifactName: runners
artifactType: container
################################################################################
- job: build_osx_agent_x64
################################################################################
displayName: macOS Agent (x64)
pool:
vmImage: macOS-10.13
steps:
# Steps template for non-windows platform
- template: nonwindows.template.yml
parameters:
targetRuntime: osx-x64
# Create agent package zip
- script: ./dev.sh package Release osx-x64
workingDirectory: src
displayName: Package Release
# Upload agent package zip as build artifact
- task: PublishBuildArtifacts@1
displayName: Publish Artifact (OSX x64)
inputs:
pathToPublish: _package
artifactName: runners
artifactType: container
- stage: Release
dependsOn: Build
jobs:
################################################################################
- job: publish_agent_packages
################################################################################
displayName: Publish Agents (Windows/Linux/OSX)
pool:
name: ProductionRMAgents
steps:
# Download all agent packages from all previous phases
- task: DownloadBuildArtifacts@0
displayName: Download Agent Packages
inputs:
artifactName: runners
# Upload agent packages to Azure blob storage and refresh Azure CDN
- powershell: |
Write-Host "Preloading Azure modules." # This is for better performance, to avoid module-autoloading.
Import-Module AzureRM, AzureRM.profile, AzureRM.Storage, Azure.Storage, AzureRM.Cdn -ErrorAction Ignore -PassThru
Enable-AzureRmAlias -Scope CurrentUser
$uploadFiles = New-Object System.Collections.ArrayList
$certificateThumbprint = (Get-ItemProperty -Path "$(ServicePrincipalReg)").ServicePrincipalCertThumbprint
$clientId = (Get-ItemProperty -Path "$(ServicePrincipalReg)").ServicePrincipalClientId
Write-Host "##vso[task.setsecret]$certificateThumbprint"
Write-Host "##vso[task.setsecret]$clientId"
Login-AzureRmAccount -ServicePrincipal -CertificateThumbprint $certificateThumbprint -ApplicationId $clientId -TenantId $(GitHubTenantId)
Select-AzureRmSubscription -SubscriptionId $(GitHubSubscriptionId)
$storage = Get-AzureRmStorageAccount -ResourceGroupName githubassets -AccountName githubassets
Get-ChildItem -LiteralPath "$(System.ArtifactsDirectory)/runners" | ForEach-Object {
$versionDir = $_.Name.Trim('.zip').Trim('.tar.gz')
$versionDir = $versionDir.SubString($versionDir.LastIndexOf('-') + 1)
Write-Host "##vso[task.setvariable variable=ReleaseAgentVersion;]$versionDir"
Write-Host "Uploading $_ to BlobStorage githubassets/runners/$versionDir"
Set-AzureStorageBlobContent -Context $storage.Context -Container runners -File "$(System.ArtifactsDirectory)/runners/$_" -Blob "$versionDir/$_" -Force
$uploadFiles.Add("/runners/$versionDir/$_")
}
Write-Host "Get CDN info"
Get-AzureRmCdnEndpoint -ProfileName githubassets -ResourceGroupName githubassets
Write-Host "Purge Azure CDN Cache"
Unpublish-AzureRmCdnEndpointContent -EndpointName githubassets -ProfileName githubassets -ResourceGroupName githubassets -PurgeContent $uploadFiles
Write-Host "Pull assets through Azure CDN"
$uploadFiles | ForEach-Object {
$downloadUrl = "https://githubassets.azureedge.net" + $_
Write-Host $downloadUrl
Invoke-WebRequest -Uri $downloadUrl -OutFile $_.SubString($_.LastIndexOf('/') + 1)
}
displayName: Upload to Azure Blob
# Create agent release on Github
- powershell: |
Write-Host "Creating github release."
$releaseNotes = [System.IO.File]::ReadAllText("$(Build.SourcesDirectory)\releaseNote.md").Replace("<RUNNER_VERSION>","$(ReleaseAgentVersion)")
$releaseData = @{
tag_name = "v$(ReleaseAgentVersion)";
target_commitish = "$(Build.SourceVersion)";
name = "v$(ReleaseAgentVersion)";
body = $releaseNotes;
draft = $false;
prerelease = $true;
}
$releaseParams = @{
Uri = "https://api.github.com/repos/actions/runner/releases";
Method = 'POST';
Headers = @{
Authorization = 'Basic ' + [Convert]::ToBase64String([Text.Encoding]::ASCII.GetBytes("github:$(GithubToken)"));
}
ContentType = 'application/json';
Body = (ConvertTo-Json $releaseData -Compress)
}
[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12
$releaseCreated = Invoke-RestMethod @releaseParams
Write-Host $releaseCreated
$releaseId = $releaseCreated.id
Get-ChildItem -LiteralPath "$(System.ArtifactsDirectory)/runners" | ForEach-Object {
Write-Host "Uploading $_ as GitHub release assets"
$assetsParams = @{
Uri = "https://uploads.github.com/repos/actions/runner/releases/$releaseId/assets?name=$($_.Name)"
Method = 'POST';
Headers = @{
Authorization = 'Basic ' + [Convert]::ToBase64String([Text.Encoding]::ASCII.GetBytes("github:$(GithubToken)"));
}
ContentType = 'application/octet-stream';
Body = [System.IO.File]::ReadAllBytes($_.FullName)
}
Invoke-RestMethod @assetsParams
}
displayName: Create agent release on Github

View File

@@ -1,95 +0,0 @@
jobs:
################################################################################
- job: build_windows_x64_agent
################################################################################
displayName: Windows Agent (x64)
pool:
vmImage: vs2017-win2016
steps:
# Steps template for windows platform
- template: windows.template.yml
# Package dotnet core windows dependency (VC++ Redistributable)
- powershell: |
Write-Host "Downloading 'VC++ Redistributable' package."
$outDir = Join-Path -Path $env:TMP -ChildPath ([Guid]::NewGuid())
New-Item -Path $outDir -ItemType directory
$outFile = Join-Path -Path $outDir -ChildPath "ucrt.zip"
Invoke-WebRequest -Uri https://vstsagenttools.blob.core.windows.net/tools/ucrt/ucrt_x64.zip -OutFile $outFile
Write-Host "Unzipping 'VC++ Redistributable' package to agent layout."
$unzipDir = Join-Path -Path $outDir -ChildPath "unzip"
Add-Type -AssemblyName System.IO.Compression.FileSystem
[System.IO.Compression.ZipFile]::ExtractToDirectory($outFile, $unzipDir)
$agentLayoutBin = Join-Path -Path $(Build.SourcesDirectory) -ChildPath "_layout\bin"
Copy-Item -Path $unzipDir -Destination $agentLayoutBin -Force
displayName: Package UCRT
condition: and(succeeded(), ne(variables['build.reason'], 'PullRequest'))
# Create agent package zip
- script: dev.cmd package Release
workingDirectory: src
displayName: Package Release
condition: and(succeeded(), ne(variables['build.reason'], 'PullRequest'))
# Upload agent package zip as build artifact
- task: PublishBuildArtifacts@1
displayName: Publish Artifact (Windows x64)
condition: and(succeeded(), ne(variables['build.reason'], 'PullRequest'))
inputs:
pathToPublish: _package
artifactName: agent
artifactType: container
################################################################################
- job: build_linux_x64_agent
################################################################################
displayName: Linux Agent (x64)
pool:
vmImage: ubuntu-16.04
steps:
# Steps template for non-windows platform
- template: nonwindows.template.yml
# Create agent package zip
- script: ./dev.sh package Release
workingDirectory: src
displayName: Package Release
condition: and(succeeded(), ne(variables['build.reason'], 'PullRequest'))
# Upload agent package zip as build artifact
- task: PublishBuildArtifacts@1
displayName: Publish Artifact (Linux x64)
condition: and(succeeded(), ne(variables['build.reason'], 'PullRequest'))
inputs:
pathToPublish: _package
artifactName: agent
artifactType: container
################################################################################
- job: build_osx_agent
################################################################################
displayName: macOS Agent (x64)
pool:
vmImage: macOS-10.14
steps:
# Steps template for non-windows platform
- template: nonwindows.template.yml
# Create agent package zip
- script: ./dev.sh package Release
workingDirectory: src
displayName: Package Release
condition: and(succeeded(), ne(variables['build.reason'], 'PullRequest'))
# Upload agent package zip as build artifact
- task: PublishBuildArtifacts@1
displayName: Publish Artifact (OSX)
condition: and(succeeded(), ne(variables['build.reason'], 'PullRequest'))
inputs:
pathToPublish: _package
artifactName: agent
artifactType: container

View File

@@ -0,0 +1,61 @@
# ADR 263: Self Hosted Runner Proxies
**Date**: 2019-11-13
**Status**: Accepted
## Context
- Proxy support is required for some enterprises and organizations to start using their own self hosted runners
- While there is not a standard convention, many applications support setting proxies via the environmental variables `http_proxy`, `https_proxy`, `no_proxy`, such as curl, wget, perl, python, docker, git, R, ect
- Some of these applications use `HTTPS_PROXY` versus `https_proxy`, but most understand or primarily support the lowercase variant
## Decision
We will update the Runner to use the conventional environment variables for proxies: `http_proxy`, `https_proxy` and `no_proxy` if they are set.
These are described in detail below:
- `https_proxy` a proxy URL for all https traffic. It may contain basic authentication credentials. For example:
- http://proxy.com
- http://127.0.0.1:8080
- http://user:password@proxy.com
- `http_proxy` a proxy URL for all http traffic. It may contain basic authentication credentials. For example:
- http://proxy.com
- http://127.0.0.1:8080
- http://user:password@proxy.com
- `no_proxy` a comma seperated list of hosts that should not use the proxy. An optional port may be specified
- `google.com`
- `yahoo.com:443`
- `google.com,bing.com`
We won't use `http_proxy` for https traffic when `https_proxy` is not set, this behavior lines up with any libcurl based tools (curl, git) and wget.
Otherwise action authors and workflow users need to adjust to differences between the runner proxy convention, and tools used by their actions and scripts.
Example:
Customer set `http_proxy=http://127.0.0.1:8888` and configure the runner against `https://github.com/owner/repo`, with the `https_proxy` -> `http_proxy` fallback, the runner will connect to server without any problem. However, if user runs `git push` to `https://github.com/owner/repo`, `git` won't use the proxy since it require `https_proxy` to be set for any https traffic.
> `golang`, `node.js` and other dev tools from the linux community use `http_proxy` for both http and https traffic base on my research.
A majority of our users are using Linux where these variables are commonly required to be set by various programs. By reading these values, we simplify the process for self hosted runners to set up proxy, and expose it in a way users are already familiar with.
A password provided for a proxy will be masked in the logs.
We will support the lowercase and uppercase variants, with lowercase taking priority if both are set.
### No Proxy Format
While exact implementations are different per application on handle `no_proxy` env, most applications accept a comma separated list of hosts. Some accept wildcard characters (*). We are going to do exact case-insentive matches, and not support wildcards at this time.
For example:
- example.com will match example.com, foo.example.com, foo.bar.example.com
- foo.example.com will match bar.foo.example.com and foo.example.com
We will not support IP addresses for `no_proxy`, only hostnames.
## Consequences
1. Enterprises and organizations needing proxy support will be able to embrace self hosted runners
2. Users will need to set these environmental variables before configuring the runner in order to use a proxy when configuring
3. The runner will read from the environmental variables during config and runtime and use the provided proxy if it exists
4. Users may need to pass these environmental variables into other applications if they do not natively take these variables
5. Action authors may need to update their workflows to react to the these environment variables
6. We will document the way of setting environmental variables for runners using the environmental variables and how the runner uses them
7. Like all other secrets, users will be able to relatively easily figure out proxy password if they can modify a workflow file running on a self hosted machine

View File

@@ -0,0 +1,263 @@
# ADR 0276: Problem Matchers
**Date** 2019-06-05
**Status** Accepted
## Context
Compilation failures during a CI build should surface good error messages.
For example, the actual compile errors from the typescript compiler should bubble as issues in the UI. And not simply "tsc exited with exit code 1".
VSCode has an extensible model for solving this type of problem. VSCode allows users to configure which problems matchers to use, when scanning output. For example, a user can apply the `tsc` problem matcher to receive a rich error output experience in VSCode, when compiling their typescript project.
The problem-matcher concept fits well with "setup" actions. For example, the `setup-nodejs` action will download node.js, add it to the PATH, and register the `tsc` problem matcher. For the duration of the job, the `tsc` problem matcher will be applied against the output.
## Decision
### Registration
#### Using `##` command
`##[add-matcher]path-to-problem-matcher-config.json`
Using a `##` command allows for flexibility:
- Ad hoc scripts can register problem matchers
- Allows problem matchers to be conditionally registered
Note, if a matcher with the same name is registered a second time, it will clobber the first instance.
#### Unregister using `##` command
A way out for rare cases where scoping is a problem.
`##[remove-matcher]owner`
For the this to be usable, the `owner` needs to be discoverable. Therefore, debug print the owner on registration.
### Single line matcher
Consider the output:
```
[...]
Build FAILED.
"C:\temp\problemmatcher\myproject\ConsoleApp1\ConsoleApp1.sln" (default target) (1) ->
"C:\temp\problemmatcher\myproject\ConsoleApp1\ConsoleApp1\ConsoleApp1.csproj" (default target) (2) ->
"C:\temp\problemmatcher\myproject\ConsoleApp1\ClassLibrary1\ClassLibrary1.csproj" (default target) (3) ->
(CoreCompile target) ->
Class1.cs(16,24): warning CS0612: 'ClassLibrary1.Helpers.MyHelper.Name' is obsolete [C:\temp\problemmatcher\myproject\ConsoleApp1\ClassLibrary1\ClassLibrary1.csproj]
"C:\temp\problemmatcher\myproject\ConsoleApp1\ConsoleApp1.sln" (default target) (1) ->
"C:\temp\problemmatcher\myproject\ConsoleApp1\ConsoleApp1\ConsoleApp1.csproj" (default target) (2) ->
"C:\temp\problemmatcher\myproject\ConsoleApp1\ClassLibrary1\ClassLibrary1.csproj" (default target) (3) ->
(CoreCompile target) ->
Helpers\MyHelper.cs(16,30): error CS1002: ; expected [C:\temp\problemmatcher\myproject\ConsoleApp1\ClassLibrary1\ClassLibrary1.csproj]
1 Warning(s)
1 Error(s)
```
The below match configuration uses a regular expression to discover problem lines. And the match groups are mapped into issue-properties.
```json
"owner": "msbuild",
"pattern": [
{
"regexp": "^\\s*([^:]+)\\((\\d+),(\\d+)\\): (error|warning) ([^:]+): (.*) \\[(.+)\\]$",
"file": 1,
"line": 2,
"column": 3,
"severity": 4,
"code": 5,
"message": 6,
"fromPath": 7
}
]
```
The above output and match configuration produces the following matches:
```
line: Class1.cs(16,24): warning CS0612: 'ClassLibrary1.Helpers.MyHelper.Name' is obsolete [C:\myrepo\myproject\ConsoleApp1\ClassLibrary1\ClassLibrary1.csproj]
file: Class1.cs
line: 16
column: 24
severity: warning
code: CS0612
message: 'ClassLibrary1.Helpers.MyHelper.Name' is obsolete
fromPath: C:\myrepo\myproject\ConsoleApp1\ClassLibrary1\ClassLibrary1.csproj
```
```
line: Helpers\MyHelper.cs(16,30): error CS1002: ; expected [C:\myrepo\myproject\ConsoleApp1\ClassLibrary1\ClassLibrary1.csproj]
file: Helpers\MyHelper.cs
line: 16
column: 30
severity: error
code: CS1002
message: ; expected
fromPath: C:\myrepo\myproject\ConsoleApp1\ClassLibrary1\ClassLibrary1.csproj
```
Additionally the line will appear red in the web UI (prefix with `##[error]`).
Note, an error does not imply task failure. Exit codes communicate failure.
Note, strip color codes when evaluating regular expressions.
### Multi-line matcher
Consider the below output from ESLint in stylish mode. The file name is printed once, yet multiple error lines are printed.
```
test.js
1:0 error Missing "use strict" statement strict
5:10 error 'addOne' is defined but never used no-unused-vars
✖ 2 problems (2 errors, 0 warnings)
```
The below match configuration uses multiple regular expressions, for the multiple lines.
And the last pattern of a multiline matcher can specify the `loop` property. This allows multiple errors to be discovered.
```json
"owner": "eslint-stylish",
"pattern": [
{
"regexp": "^([^\\s].*)$",
"file": 1
},
{
"regexp": "^\\s+(\\d+):(\\d+)\\s+(error|warning|info)\\s+(.*)\\s\\s+(.*)$",
"line": 1,
"column": 2,
"severity": 3,
"message": 4,
"code": 5,
"loop": true
}
]
```
The above output and match configuration produces two matches:
```
line: 1:0 error Missing "use strict" statement strict
file: test.js
line: 1
column: 0
severity: error
message: Missing "use strict" statement
code: strict
```
```
line: 5:10 error 'addOne' is defined but never used no-unused-vars
file: test.js
line: 5
column: 10
severity: error
message: 'addOne' is defined but never used
code: no-unused-vars
```
Note, in the above example only the error line will appear red in the web UI. The \"file\" line will not appear red.
### Other details
#### Configuration `owner`
Can be used to stomp over or remove.
#### Rooting the file
The goal of the file information is to provide a hyperlink in the UI.
Solving this problem means:
- Rooting the file when unrooted:
- Use the `fromPath` if specified (assume file path)
- Use the `github.workspace` (where the repo is cloned on disk)
- Match against a repository to determine the relative path within the repo
This is a place where we diverge from VSCode. VSCode task configuration are specific to the local workspace (workspace root is known or can be specified). We're solving a more generic problem, so we need more information - specifically the `fromPath` property - in order to accurately root the path.
In order to avoid creating inaccurate hyperlinks on the error issues, the agent will verify the file exists and is in the main repository. Otherwise omit the file property from the error issue and debug trace what happened.
#### Supported severity levels
Ordinal ignore case:
- `warning`
- `error`
Coalesce empty with \"error\". For any other values, omit logging an issue and debug trace what happened.
#### Default severity level
Problem matchers are unable to interpret severity strings other than `warning` and `error`. The `severity` match group expects `warning` or `error` (case insensitive).
However some tools indicate error/warning in different ways. For example `flake8` uses codes like `E100`, `W200`, and `F300` (error, warning, fatal, respectively).
Therefore, allow a property `severity`, sibling to `owner`, which identifies the default severity for the problem matcher. This allows two problem matchers are registered - one for warnings and one for errors.
For example, given the following `flake8` output:
```
./bootcamp/settings.py:156:80: E501 line too long (94 > 79 characters)
./bootcamp/settings.py:165:5: F403 'from local_settings import *' used; unable to detect undefined names
```
Two problem matchers can be used:
```json
{
"problemMatcher": [
{
"owner": "flake8",
"pattern": [
{
"regexp": "^(.+):(\\d+):(\\d+): ([EF]\\d+) (.+)$",
"file": 1,
"line": 2,
"column": 3,
"code": 4,
"message": 5
}
]
},
{
"owner": "flake8-warnings",
"severity": "warning",
"pattern": [
{
"regexp": "^(.+):(\\d+):(\\d+): (W\\d+) (.+)$",
"file": 1,
"line": 2,
"column": 3,
"code": 4,
"message": 5
}
]
}
]
}
```
#### Mitigate regular expression denial of service (ReDos)
If a matcher exceeds a 1 second timeout when processing a line, retry up to two three times total.
After three unsuccessful attempts, warn and eject the matcher. The matcher will not run again for the duration of the job.
### Where we diverge from VSCode
- We added the `fromPath` concept for rooting paths. This is done differently in VSCode, since a task is the scope (root path well known). For us, the job is the scope.
- VSCode allows additional activation info background tasks that are always running (recompile on files changed). They allow regular expressions to define when the matcher scope begins and ends. This is an interesting concept that we could leverage to help solve our scoping problem.
## Consequences
- Setup actions should register problem matchers

View File

@@ -0,0 +1,93 @@
# ADR 0277: Run action shell option
**Date** 2019-07-09
**Status** Accepted
## Context
run-actions run scripts using a platform specific shell:
`bash -eo pipefail` on non-windows, and `cmd.exe /c /d /s` on windows
The `shell` option overwrites this to allow different flags or completely different shells/interpreters
A small example is:
```yml
jobs:
bash-job:
actions:
- run: echo "Hello"
shell: bash
python-job:
actions:
- run: print("Hello")
shell: python {0}
```
## Decision
___
### Shell option
The keyword being used is `shell`
`shell` can be either:
1. Builtins / Explicitly supported keywords. It is useful to support at least `cmd`, and `powershell` on Windows. Because `cmd my_cmd_script` and `powershell my_ps1_script` are not valid the same way many Linux/cross-platform interpreters are, e.g. `bash myscript` or `python myscript`. Those tools (and potentially others) also require the correct file extension to run, or must be run in a particular way to get the exit codes consistently, so we must have first class knowledge about them. We provide default templates for these keywords as follows:
- `cmd`: Default is: `%ComSpec% /D /E:ON /V:OFF /S /C "CALL "{0}""` where the script name is automatically appended with `.cmd` and substituted for `{0}`
- Note this is equivalent to the default Windows behavior if no shell option is given
- `pwsh`: Default is: `pwsh -command "& '{0}'"` where the script is automatically appended with `.ps1`
- `powershell`: Default is: `powershell -command "& '{0}'"` where the script is automatically appended with `.ps1`
- `bash`: Uses `bash --noprofile --norc -eo pipefail {0}`
- The default behavior on non-Windows if no shell is given is to attempt this first
- `sh`: Uses `sh -e {0}`
- This is the default behavior on non-Windows if no shell is given, AND `bash` (see above) was not located on the PATH
- `python`: `python {0}`
- **NOTE**: The exact command ran may vary by machine. We only provide default arguments and command format for the listed shell. While the above behavior is expected on hosted machines, private runners may vary. For example, `sh` (or other commands) may actually be a link to `/bin/dash`, `/bin/bash`, or other
1. A template string: `command [...options] {0} [...more_options]`
- As above, the file name of the temporary script will be templated in. This gives users more control to have options at any location relative to the script path
- The first whitespace-delimited word of the string will be interpreted as the command
- e.g. `python {0} arg1 arg2` or similar can be used if passing args is needed. Some shells will require other options after the filename for various reasons
Note that (1) simply provides defaults that are executed with the same mechanism as (2). That is:
- A temporary script file is generated, and the path to that file is templated into the string at `{0}`
- The first word of the formatted string is assumed to be a command, and we attempt to locate its full path
- The fully qualified path to the command, plus the remaining arguments, is executed
- e.g. `shell: bash` expands to `/bin/bash --noprofile --norc -eo pipefail /runner/_layout/_work/_temp/f8d4fb2b-19d9-47e6-a786-4cc538d52761.sh` on my private runner
At this time, **THE LIST OF WELL-KNOWN SHELL OPTIONS IS**:
- cmd - Windows (hosted vs2017, vs2019) only
- powershell - Windows (hosted vs2017, vs2019) only
- sh - All hosted platforms
- pwsh - All hosted platforms
- bash - All hosted platforms
- python - All hosted platforms. Can use setup-python to configure which python will be used
___
### Containers
For container jobs, `shell` should just work the same as above, transparently. We will simply `exec` the command in the job container, passing the same arguments in
___
### Exit codes / Error action preference
For builtin shells, we provide defaults that make the most sense for CI, running within Actions, and being executed by our runner
bash/sh:
- Fail-fast behavior using `set -e o pipefail` is the default for `bash` and `shell` builtins, and by default when no option is given on non-Windows platforms
- Users can opt out of fail-fast and take full control easily by providing a template string to the shell options, eg: `bash {0}`.
- sh-like shells exit with the exit code of the last command executed in a script, and is our default behavior. Thus the runner reports the status of the step as fail/succeed based on this exit code
powershell/pwsh
- Fail-fast behavior when possible. For `pwsh` and `powershell` builtins, we will prepend `$ErrorActionPreference = 'stop'` to script contents
- We append `if ((Test-Path -LiteralPath variable:\LASTEXITCODE)) { exit $LASTEXITCODE }` to powershell scripts to get Action statuses to reflect the script's last exit code
- Users can always opt out by not using the builtins, and providing a shell option like: `pwsh -File {0}`, or `powershell -Command "& '{0}'"`, depending on need
cmd
- There doesnt seem to be a way to fully opt in to fail-fast behavior other than writing your script to check each error code and respond accordingly, so we cant actually provide that behavior by default, it will be completely up to the user to write this behavior into their script
- cmd.exe will exit (return the error code to the runner) with the errorlevel of the last program it executed. This is internally consistent with the previous default behavior (sh, pwsh) and is the cmd.exe default, so we keep that behavior
## Consequences
Valid `shell` options will depend on the hosted images. We will need to maintain tight image compat
First class support for a shell will require a major version schema change to modify. We cannot remove or modify the behavior of a well-known supported option, However, adding first class support for new shells is backwards compatible. For instance, we can add a well-known `python` option, because non-well-known options would have always needed to include `{0}`, e.g. `python {0}`

View File

@@ -0,0 +1,60 @@
# ADR 0278: Env Context
**Date**: 2019-09-30
**Status**: Accepted
## Context
User wants to reference workflow variables defined in workflow yaml file for action's input, displayName and condition.
## Decision
### Add `env` context in the runner
Runner will create and populate the `env` context for every job execution using following logic:
1. On job start, create `env` context with any environment variables in the job message, these are env defined in customer's YAML file's job/workflow level `env` section.
2. Update `env` context when customer use `::set-env::` to set env at the runner level.
3. Update `env` context with step's `env` block before each step runs.
The `env` context is only available in the runner, customer can't use the `env` context in any server evaluation part, just like the `runner` context
Example yaml:
```yaml
env:
env1: 10
env2: 20
env3: 30
jobs:
build:
env:
env1: 100
env2: 200
runs-on: ubuntu-latest
steps:
- run: |
echo ${{ env.env1 }} // 1000
echo $env1 // 1000
echo $env2 // 200
echo $env3 // 30
if: env.env2 == 200 // true
name: ${{ env.env1 }}_${{ env.env2 }} //1000_200
env:
env1: 1000
```
### Don't populate the `env` context with environment variables from runner machine.
With job container and container action, the `env` context may not have the right value customer want and will cause confusion.
Ex:
```yaml
build:
runs-on: ubuntu-latest <- $USER=runner in hosted machine
container: ubuntu:16.04 <- $USER=root in container
steps:
- run: echo ${{env.USER}} <- what should customer expect this output? runner/root
- uses: docker://ubuntu:18.04
with:
args: echo ${{env.USER}} <- what should customer expect this output? runner/root
```

View File

@@ -0,0 +1,71 @@
# ADR 0279: HashFiles Expression Function
**Date**: 2019-09-30
**Status**: Accepted
## Context
First party action `actions/cache` needs a input which is an explicit `key` used for restoring and saving the cache. For packages caching, the most comment `key` might be the hash result of contents from all `package-lock.json` under `node_modules` folder.
There are serval different ways to get the hash `key` input for `actions/cache` action.
1. Customer calculate the `key` themselves from a different action, customer won't like this since it needs extra step for using cache feature
```yaml
steps:
- run: |
hash=some_linux_hash_method(file1, file2, file3)
echo ::set-output name=hash::$hash
id: createHash
- uses: actions/cache@v1
with:
key: ${{ steps.createHash.outputs.hash }}
```
2. Make the `key` input of `actions/cache` follow certain convention to calculate hash, this limited the `key` input to a certain format customer may not want.
```yaml
steps:
- uses: actions/cache@v1
with:
key: ${{ runner.os }}|${{ github.workspace }}|**/package-lock.json
```
## Decision
### Add hashFiles() function to expression engine for calculate files' hash
`hashFiles()` will only allow on runner side since it needs to read files on disk, using `hashFiles()` on any server side evaluated expression will cause runtime errors.
`hashFiles()` will only support hashing files under the `$GITHUB_WORKSPACE` since the expression evaluated on the runner, if customer use job container or container action, the runner won't have access to file system inside the container.
`hashFiles()` will only take 1 parameters:
- `hashFiles('**/package-lock.json')` // Search files under $GITHUB_WORKSPACE and calculate a hash for them
**Question: Do we need to support more than one match patterns?**
Ex: `hashFiles('**/package-lock.json', '!toolkit/core/package-lock.json', '!toolkit/io/package-lock.json')`
Answer: Only support single match pattern for GA, we can always add later.
This will help customer has better experience with the `actions/cache` action's input.
```yaml
steps:
- uses: actions/cache@v1
with:
key: ${{hashFiles('**/package-lock.json')}}-${{github.ref}}-${{runner.os}}
```
For search pattern, we will use basic globbing (`*` `?` and `[]`) and globstar (`**`).
Additional pattern details:
- Root relative paths with `github.workspace` (the main repo)
- Make `*` match files that start with `.`
- Case insensitive on Windows
- Accept `\` or `/` path separators on Windows
Hashing logic:
1. Get all files under `$GITHUB_WORKSPACE`.
2. Use search pattern filter all files to get files that matches the search pattern. (search pattern only apply to file path not folder path)
3. Sort all matched files by full file path in alphabet order.
4. Use SHA256 algorithm to hash each matched file and store hash result.
5. Use SHA256 to hash all stored files' hash results to get the final 64 chars hash result.
**Question: Should we include the folder structure info into the hash?**
Answer: No

View File

@@ -0,0 +1,30 @@
# ADR 0280: Echoing of Command Input
**Date**: 2019-11-04
**Status**: Accepted
## Context
Command echoing as a default behavior tends to clutter the user logs, so we want to swap to a system where users have to opt in to see this information.
Command outputs will still be echoed in the case there are any errors processing such commands. This is so the end user can have more context on why the command failed and help with troubleshooting.
Echo output in the user logs can be explicitly controlled by the new commands `::echo::on` and `::echo::off`. By default, echoing is enabled if `ACTIONS_STEP_DEBUG` secret is enabled, otherwise echoing is disabled.
## Decision
- The only commands that currently echo output are
- `remove-matcher`
- `add-matcher`
- `add-path`
- These will no longer echo the command, if processed successfully
- All commands echo the input when any of these conditions is fulfilled:
1. When such commands fail with an error
2. When `::echo::on` is set
3. When the `ACTIONS_STEP_DEBUG` is set, and echoing hasn't been explicitly disabled with `::echo::off`
- There are a few commands that won't be echoed, even when echo is enabled. These are (as of 2019/11/04):
- `add-mask`
- `debug`
- `warning`
- `error`
- The three commands above will not echo, either because echoing the command would leak secrets (e.g. `add-mask`), or it would not add any additional troubleshooting information to the logs (e.g. `debug`). It's expected that future commands would follow these "echo-suppressing" guidelines as well. Echo-suppressed commands are still free to output other information to the logs, as deemed fit.

View File

@@ -0,0 +1,48 @@
# ADR 0297: Base64 Masking Trailing Characters
**Date** 2020-01-21
**Status** Proposed
## Context
The Runner registers a number of Value Encoders, which mask various encodings of a provided secret. Currently, we register a 3 base64 Encoders:
- The base64 encoded secret
- The secret with the first character removed then base64 encoded
- The secret with the first two characters removed then base64 encoded
This gives us good coverage across the board for secrets and secrets with a prefix (i.e. `base64($user:$pass)`).
However, we don't have great coverage for cases where the secret has a string appended to it before it is base64 encoded (i.e.: `base64($pass\n))`).
Most notably we've seen this as a result of user error where a user accidentially appends a newline or space character before encoding their secret in base64.
## Decision
### Trim end characters
We are going to modify all existing base64 encoders to trim information before registering as a secret.
We will trim:
- `=` from the end of all base64 strings. This is a padding character that contains no information.
- Based on the number of `=`'s at the end of a base64 string, a malicious user could predict the length of the original secret modulo 3.
- If a user saw `***==`, they would know the secret could be 1,4,7,10... characters.
- If a string contains `=` we will also trim the last non-padding character from the base64 secret.
- This character can change if a string is appended to the secret before the encoding.
### Register a fourth encoder
We will also add back in the original base64 encoded secret encoder for four total encoders:
- The base64 encoded secret
- The base64 encoded secret trimmed
- The secret with the first character removed then base64 encoded and trimmed
- The secret with the first two characters removed then base64 encoded and trimmed
This allows us to fully cover the most common scenario where a user base64 encodes their secret and expects the entire thing to be masked.
This will result in us only revealing length or bit information when a prefix or suffix is added to a secret before encoding.
## Consequences
- In the case where a secret has a prefix or suffix added before base64 encoding, we may now reveal up to 20 bits of information and the length of the original string modulo 3, rather then the original 16 bits and no length information
- Secrets with a suffix appended before encoding will now be masked across the board. Previously it was only masked if it was a multiple of 3 characters
- Performance will suffer in a neglible way

19
docs/adrs/README.md Normal file
View File

@@ -0,0 +1,19 @@
# ADRs
ADR, short for "Architecture Decision Record" is a way of capturing important architectural decisions, along with their context and consequences.
This folder includes ADRs for the actions runner. ADRs are proposed in the form of a pull request, and they commonly follow this format:
* **Title**: short present tense imperative phrase, less than 50 characters, like a git commit message.
* **Status**: proposed, accepted, rejected, deprecated, superseded, etc.
* **Context**: what is the issue that we're seeing that is motivating this decision or change.
* **Decision**: what is the change that we're actually proposing or doing.
* **Consequences**: what becomes easier or more difficult to do because of this change.
---
- More information about ADRs can be found [here](https://github.com/joelparkerhenderson/architecture_decision_record).

View File

@@ -1,10 +1,31 @@
# Contribution guide for developers
# Contributions
## Required Dev Dependencies
We welcome contributions in the form of issues and pull requests. We view the contributions and the process as the same for github and external contributors.
> IMPORTANT: Building your own runner is critical for the dev inner loop process when contributing changes. However, only runners built and distributed by GitHub (releases) are supported in production. Be aware that workflows and orchestrations run service side with the runner being a remote process to run steps. For that reason, the service can pull the runner forward so customizations can be lost.
## Issues
Log issues for both bugs and enhancement requests. Logging issues are important for the open community.
Issues in this repository should be for the runner application. Note that the VM and virtual machine images (including the developer toolsets) installed on the actions hosted machine pools are located [in this repository](https://github.com/actions/virtual-environments)
## Enhancements and Feature Requests
We ask that before significant effort is put into code changes, that we have agreement on taking the change before time is invested in code changes.
1. Create a feature request. Once agreed we will take the enhancment
2. Create an ADR to agree on the details of the change.
An ADR is an Architectural Decision Record. This allows consensus on the direction forward and also serves as a record of the change and motivation. [Read more here](adrs/README.md)
## Development Life Cycle
### Required Dev Dependencies
![Win](res/win_sm.png) Git for Windows [Install Here](https://git-scm.com/downloads) (needed for dev sh script)
## To Build, Test, Layout
### To Build, Test, Layout
Navigate to the `src` directory and run the following command:
@@ -14,27 +35,27 @@ Navigate to the `src` directory and run the following command:
**Commands:**
* `layout` (`l`): Run first time to create a full agent layout in `{root}/_layout`
* `build` (`b`): Build everything and update agent layout folder
* `test` (`t`): Build agent binaries and run unit tests
* `layout` (`l`): Run first time to create a full runner layout in `{root}/_layout`
* `build` (`b`): Build everything and update runner layout folder
* `test` (`t`): Build runner binaries and run unit tests
Sample developer flow:
```bash
git clone https://github.com/actions/runner
cd ./src
./dev.(sh/cmd) layout # the agent that build from source is in {root}/_layout
./dev.(sh/cmd) layout # the runner that build from source is in {root}/_layout
<make code changes>
./dev.(sh/cmd) build # {root}/_layout will get updated
./dev.(sh/cmd) test # run all unit tests before git commit/push
```
## Editors
### Editors
[Using Visual Studio 2019](https://www.visualstudio.com/vs/)
[Using Visual Studio Code](https://code.visualstudio.com/)
[Using Visual Studio 2019](https://www.visualstudio.com/vs/)
## Styling
### Styling
We use the .NET Foundation and CoreCLR style guidelines [located here](
https://github.com/dotnet/corefx/blob/master/Documentation/coding-guidelines/coding-style.md)

View File

@@ -28,7 +28,7 @@ Execute ./bin/installdependencies.sh to install any missing Dotnet Core 3.0 depe
```
You can easily correct the problem by executing `./bin/installdependencies.sh`.
The `installdependencies.sh` script should install all required dependencies on all supported Linux versions
> Note: The `installdependencies.sh` script will try to use the default package management mechanism on your Linux flavor (ex. `yum`/`apt-get`/`apt`). You might need to deal with error coming from the package management mechanism related to your setup, like [#1353](https://github.com/Microsoft/vsts-agent/issues/1353)
> Note: The `installdependencies.sh` script will try to use the default package management mechanism on your Linux flavor (ex. `yum`/`apt-get`/`apt`).
### Full dependencies list

View File

@@ -9,4 +9,4 @@
- Windows Server 2016 64-bit
- Windows Server 2019 64-bit
## [More .Net Core Prerequisites Information](https://docs.microsoft.com/en-us/dotnet/core/windows-prerequisites?tabs=netcore30)
## [More .NET Core Prerequisites Information](https://docs.microsoft.com/en-us/dotnet/core/windows-prerequisites?tabs=netcore30)

View File

@@ -1,7 +0,0 @@
FROM mcr.microsoft.com/dotnet/core/runtime-deps:2.1
RUN apt-get update \
&& apt-get install -y --no-install-recommends \
curl \
git \
&& rm -rf /var/lib/apt/lists/*

View File

@@ -1,150 +0,0 @@
FROM centos:6
# Install dependencies
RUN yum install -y \
centos-release-SCL \
epel-release \
wget \
unzip \
&& \
rpm --import http://linuxsoft.cern.ch/cern/slc6X/x86_64/RPM-GPG-KEY-cern && \
wget -O /etc/yum.repos.d/slc6-devtoolset.repo http://linuxsoft.cern.ch/cern/devtoolset/slc6-devtoolset.repo && \
yum install -y \
"perl(Time::HiRes)" \
autoconf \
cmake \
cmake3 \
devtoolset-2-toolchain \
doxygen \
expat-devel \
gcc \
gcc-c++ \
gdb \
gettext-devel \
krb5-devel \
libedit-devel \
libidn-devel \
libmetalink-devel \
libnghttp2-devel \
libssh2-devel \
libunwind-devel \
libuuid-devel \
lttng-ust-devel \
lzma \
ncurses-devel \
openssl-devel \
perl-devel \
python-argparse \
python27 \
readline-devel \
swig \
xz \
zlib-devel \
&& \
yum clean all
# Build and install clang and lldb 3.9.1
RUN wget ftp://sourceware.org/pub/binutils/snapshots/binutils-2.29.1.tar.xz && \
wget http://releases.llvm.org/3.9.1/cfe-3.9.1.src.tar.xz && \
wget http://releases.llvm.org/3.9.1/llvm-3.9.1.src.tar.xz && \
wget http://releases.llvm.org/3.9.1/lldb-3.9.1.src.tar.xz && \
wget http://releases.llvm.org/3.9.1/compiler-rt-3.9.1.src.tar.xz && \
\
tar -xf binutils-2.29.1.tar.xz && \
tar -xf llvm-3.9.1.src.tar.xz && \
mkdir llvm-3.9.1.src/tools/clang && \
mkdir llvm-3.9.1.src/tools/lldb && \
mkdir llvm-3.9.1.src/projects/compiler-rt && \
tar -xf cfe-3.9.1.src.tar.xz --strip 1 -C llvm-3.9.1.src/tools/clang && \
tar -xf lldb-3.9.1.src.tar.xz --strip 1 -C llvm-3.9.1.src/tools/lldb && \
tar -xf compiler-rt-3.9.1.src.tar.xz --strip 1 -C llvm-3.9.1.src/projects/compiler-rt && \
rm binutils-2.29.1.tar.xz && \
rm cfe-3.9.1.src.tar.xz && \
rm lldb-3.9.1.src.tar.xz && \
rm llvm-3.9.1.src.tar.xz && \
rm compiler-rt-3.9.1.src.tar.xz && \
\
mkdir llvmbuild && \
cd llvmbuild && \
scl enable python27 devtoolset-2 \
' \
cmake3 \
-DCMAKE_CXX_COMPILER=/opt/rh/devtoolset-2/root/usr/bin/g++ \
-DCMAKE_C_COMPILER=/opt/rh/devtoolset-2/root/usr/bin/gcc \
-DCMAKE_LINKER=/opt/rh/devtoolset-2/root/usr/bin/ld \
-DCMAKE_BUILD_TYPE=Release \
-DLLVM_LIBDIR_SUFFIX=64 \
-DLLVM_ENABLE_EH=1 \
-DLLVM_ENABLE_RTTI=1 \
-DLLVM_BINUTILS_INCDIR=../binutils-2.29.1/include \
../llvm-3.9.1.src \
&& \
make -j $(($(getconf _NPROCESSORS_ONLN)+1)) && \
make install \
' && \
cd .. && \
rm -r llvmbuild && \
rm -r llvm-3.9.1.src && \
rm -r binutils-2.29.1
# Build and install curl 7.45.0
RUN wget https://curl.haxx.se/download/curl-7.45.0.tar.lzma && \
tar -xf curl-7.45.0.tar.lzma && \
rm curl-7.45.0.tar.lzma && \
cd curl-7.45.0 && \
scl enable python27 devtoolset-2 \
' \
./configure \
--disable-dict \
--disable-ftp \
--disable-gopher \
--disable-imap \
--disable-ldap \
--disable-ldaps \
--disable-libcurl-option \
--disable-manual \
--disable-pop3 \
--disable-rtsp \
--disable-smb \
--disable-smtp \
--disable-telnet \
--disable-tftp \
--enable-ipv6 \
--enable-optimize \
--enable-symbol-hiding \
--with-ca-bundle=/etc/pki/tls/certs/ca-bundle.crt \
--with-nghttp2 \
--with-gssapi \
--with-ssl \
--without-librtmp \
&& \
make install \
' && \
cd .. && \
rm -r curl-7.45.0
# Install ICU 57.1
RUN wget http://download.icu-project.org/files/icu4c/57.1/icu4c-57_1-RHEL6-x64.tgz && \
tar -xf icu4c-57_1-RHEL6-x64.tgz -C / && \
rm icu4c-57_1-RHEL6-x64.tgz
# Compile and install a version of the git that supports the features that cli repo build needs
# NOTE: The git needs to be built after the curl so that it can use the libcurl to add https
# protocol support.
RUN \
wget https://www.kernel.org/pub/software/scm/git/git-2.9.5.tar.gz && \
tar -xf git-2.9.5.tar.gz && \
rm git-2.9.5.tar.gz && \
cd git-2.9.5 && \
make configure && \
./configure --prefix=/usr/local --without-tcltk && \
make -j $(nproc --all) all && \
make install && \
cd .. && \
rm -r git-2.9.5
ENV LD_LIBRARY_PATH=/usr/local/lib

View File

@@ -1,33 +0,0 @@
parameters:
targetRuntime: ''
steps:
# Build agent layout
- script: ./dev.sh layout Release ${{ parameters.targetRuntime }}
workingDirectory: src
displayName: Build & Layout Release ${{ parameters.targetRuntime }}
# Run test
- script: ./dev.sh test
workingDirectory: src
displayName: Test
condition: and(ne('${{ parameters.targetRuntime }}', 'linux-arm64'), ne('${{ parameters.targetRuntime }}', 'linux-arm'))
# # Publish test results
# - task: PublishTestResults@2
# displayName: Publish Test Results **/*.trx
# condition: always()
# inputs:
# testRunner: VSTest
# testResultsFiles: '**/*.trx'
# testRunTitle: 'Agent Tests'
# # Upload test log
# - task: PublishBuildArtifacts@1
# displayName: Publish Test logs
# condition: always()
# inputs:
# pathToPublish: src/Test/TestLogs
# artifactName: $(System.JobId)
# artifactType: container

View File

@@ -1,13 +1,27 @@
## Features
- Added the "severity" keyword to allow action authors to set the default severity for problem matchers (#203)
- Expose whether debug is on/off via RUNNER_DEBUG. (#253)
- Upload log on runner when worker get killed due to cancellation timeout. (#255)
- Update config.sh/cmd --help documentation (#282)
- Set http_proxy and related env vars for job/service containers (#304)
- Set both http_proxy and HTTP_PROXY env for runner/worker processes. (#298)
## Bugs
- Fixed generated self-hosted runner names to never go over 80 characters (helps Windows customers) (#193)
- Fixed `PrepareActions_DownloadActionFromGraph` test by pointing to an active Actions repository (#205)
- Verify runner Windows service hash started successfully after configuration (#236)
- Detect source file path in L0 without using env. (#257)
- Handle escaped '%' in commands data section (#200)
- Allow container to be null/empty during matrix expansion (#266)
- Translate problem matcher file to host path (#272)
- Change hashFiles() expression function to use @actions/glob. (#268)
- Default post-job action's condition to always(). (#293)
- Support action.yaml file as action's entry file (#288)
- Trace javascript action exit code to debug instead of user logs (#290)
- Change prompt message when removing a runner to lines up with GitHub.com UI (#303)
- Include step.env as part of env context. (#300)
- Update Base64 Encoders to deal with suffixes (#284)
## Misc
- Updated the publish and download artifact actions to use the v2 endpoint (#188)
- Updated the service name on self-hosted runner name to include repository or organization information (#193)
- Move .sln file under ./src (#238)
- Treat warnings as errors during compile (#249)
## Windows x64
We recommend configuring the runner under "<DRIVE>:\actions-runner". This will help avoid issues related to service identity folder permissions and long file path restrictions on Windows
@@ -15,7 +29,7 @@ We recommend configuring the runner under "<DRIVE>:\actions-runner". This will h
// Create a folder under the drive root
mkdir \actions-runner ; cd \actions-runner
// Download the latest runner package
Invoke-WebRequest -Uri https://githubassets.azureedge.net/runners/<RUNNER_VERSION>/actions-runner-win-x64-<RUNNER_VERSION>.zip -OutFile actions-runner-win-x64-<RUNNER_VERSION>.zip
Invoke-WebRequest -Uri https://github.com/actions/runner/releases/download/v<RUNNER_VERSION>/actions-runner-win-x64-<RUNNER_VERSION>.zip -OutFile actions-runner-win-x64-<RUNNER_VERSION>.zip
// Extract the installer
Add-Type -AssemblyName System.IO.Compression.FileSystem ;
[System.IO.Compression.ZipFile]::ExtractToDirectory("$HOME\Downloads\actions-runner-win-x64-<RUNNER_VERSION>.zip", "$PWD")
@@ -27,7 +41,7 @@ Add-Type -AssemblyName System.IO.Compression.FileSystem ;
// Create a folder
mkdir actions-runner && cd actions-runner
// Download the latest runner package
curl -O https://githubassets.azureedge.net/runners/<RUNNER_VERSION>/actions-runner-osx-x64-<RUNNER_VERSION>.tar.gz
curl -O -L https://github.com/actions/runner/releases/download/v<RUNNER_VERSION>/actions-runner-osx-x64-<RUNNER_VERSION>.tar.gz
// Extract the installer
tar xzf ./actions-runner-osx-x64-<RUNNER_VERSION>.tar.gz
```
@@ -38,7 +52,7 @@ tar xzf ./actions-runner-osx-x64-<RUNNER_VERSION>.tar.gz
// Create a folder
mkdir actions-runner && cd actions-runner
// Download the latest runner package
curl -O https://githubassets.azureedge.net/runners/<RUNNER_VERSION>/actions-runner-linux-x64-<RUNNER_VERSION>.tar.gz
curl -O -L https://github.com/actions/runner/releases/download/v<RUNNER_VERSION>/actions-runner-linux-x64-<RUNNER_VERSION>.tar.gz
// Extract the installer
tar xzf ./actions-runner-linux-x64-<RUNNER_VERSION>.tar.gz
```
@@ -49,7 +63,7 @@ tar xzf ./actions-runner-linux-x64-<RUNNER_VERSION>.tar.gz
// Create a folder
mkdir actions-runner && cd actions-runner
// Download the latest runner package
curl -O https://githubassets.azureedge.net/runners/<RUNNER_VERSION>/actions-runner-linux-arm64-<RUNNER_VERSION>.tar.gz
curl -O -L https://github.com/actions/runner/releases/download/v<RUNNER_VERSION>/actions-runner-linux-arm64-<RUNNER_VERSION>.tar.gz
// Extract the installer
tar xzf ./actions-runner-linux-arm64-<RUNNER_VERSION>.tar.gz
```
@@ -60,7 +74,7 @@ tar xzf ./actions-runner-linux-arm64-<RUNNER_VERSION>.tar.gz
// Create a folder
mkdir actions-runner && cd actions-runner
// Download the latest runner package
curl -O https://githubassets.azureedge.net/runners/<RUNNER_VERSION>/actions-runner-linux-arm-<RUNNER_VERSION>.tar.gz
curl -O -L https://github.com/actions/runner/releases/download/v<RUNNER_VERSION>/actions-runner-linux-arm-<RUNNER_VERSION>.tar.gz
// Extract the installer
tar xzf ./actions-runner-linux-arm-<RUNNER_VERSION>.tar.gz
```

1
releaseVersion Normal file
View File

@@ -0,0 +1 @@
2.164.0

View File

@@ -3,23 +3,23 @@ Microsoft Visual Studio Solution File, Format Version 12.00
# Visual Studio Version 16
VisualStudioVersion = 16.0.29411.138
MinimumVisualStudioVersion = 10.0.40219.1
Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "Runner.Common", "src\Runner.Common\Runner.Common.csproj", "{084289A3-CD7A-42E0-9219-4348B4B7E19B}"
Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "Runner.Common", "Runner.Common\Runner.Common.csproj", "{084289A3-CD7A-42E0-9219-4348B4B7E19B}"
EndProject
Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "Runner.Listener", "src\Runner.Listener\Runner.Listener.csproj", "{7D461AEE-BF2A-4855-BD96-56921160B36A}"
Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "Runner.Listener", "Runner.Listener\Runner.Listener.csproj", "{7D461AEE-BF2A-4855-BD96-56921160B36A}"
EndProject
Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "Runner.PluginHost", "src\Runner.PluginHost\Runner.PluginHost.csproj", "{D0320EB1-CB6D-4179-BFDC-2F2B664A370C}"
Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "Runner.PluginHost", "Runner.PluginHost\Runner.PluginHost.csproj", "{D0320EB1-CB6D-4179-BFDC-2F2B664A370C}"
EndProject
Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "Runner.Plugins", "src\Runner.Plugins\Runner.Plugins.csproj", "{C23AFD6F-4DCD-4243-BC61-865BE31B9168}"
Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "Runner.Plugins", "Runner.Plugins\Runner.Plugins.csproj", "{C23AFD6F-4DCD-4243-BC61-865BE31B9168}"
EndProject
Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "Runner.Sdk", "src\Runner.Sdk\Runner.Sdk.csproj", "{D0484633-DA97-4C34-8E47-1DADE212A57A}"
Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "Runner.Sdk", "Runner.Sdk\Runner.Sdk.csproj", "{D0484633-DA97-4C34-8E47-1DADE212A57A}"
EndProject
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "RunnerService", "src\Runner.Service\Windows\RunnerService.csproj", "{D12EBD71-0464-46D0-8394-40BCFBA0A6F2}"
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "RunnerService", "Runner.Service\Windows\RunnerService.csproj", "{D12EBD71-0464-46D0-8394-40BCFBA0A6F2}"
EndProject
Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "Runner.Worker", "src\Runner.Worker\Runner.Worker.csproj", "{C2F5B9FA-2621-411F-8EB2-273ED276F503}"
Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "Runner.Worker", "Runner.Worker\Runner.Worker.csproj", "{C2F5B9FA-2621-411F-8EB2-273ED276F503}"
EndProject
Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "Sdk", "src\Sdk\Sdk.csproj", "{D2EE812B-E4DF-49BB-AE87-12BC49949B5F}"
Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "Sdk", "Sdk\Sdk.csproj", "{D2EE812B-E4DF-49BB-AE87-12BC49949B5F}"
EndProject
Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "Test", "src\Test\Test.csproj", "{C932061F-F6A1-4F1E-B854-A6C6B30DC3EF}"
Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "Test", "Test\Test.csproj", "{C932061F-F6A1-4F1E-B854-A6C6B30DC3EF}"
EndProject
Global
GlobalSection(SolutionConfigurationPlatforms) = preSolution

View File

@@ -46,4 +46,9 @@
<PropertyGroup Condition="'$(Configuration)' == 'Debug'">
<DefineConstants>$(DefineConstants);DEBUG</DefineConstants>
</PropertyGroup>
<!-- Set Treat tarnings as errors -->
<PropertyGroup>
<TreatWarningsAsErrors>true</TreatWarningsAsErrors>
</PropertyGroup>
</Project>

View File

@@ -0,0 +1,3 @@
dist/
lib/
node_modules/

View File

@@ -0,0 +1,59 @@
{
"plugins": ["jest", "@typescript-eslint"],
"extends": ["plugin:github/es6"],
"parser": "@typescript-eslint/parser",
"parserOptions": {
"ecmaVersion": 9,
"sourceType": "module",
"project": "./tsconfig.json"
},
"rules": {
"eslint-comments/no-use": "off",
"import/no-namespace": "off",
"no-console": "off",
"no-unused-vars": "off",
"@typescript-eslint/no-unused-vars": "error",
"@typescript-eslint/explicit-member-accessibility": ["error", {"accessibility": "no-public"}],
"@typescript-eslint/no-require-imports": "error",
"@typescript-eslint/array-type": "error",
"@typescript-eslint/await-thenable": "error",
"@typescript-eslint/ban-ts-ignore": "error",
"camelcase": "off",
"@typescript-eslint/camelcase": "error",
"@typescript-eslint/class-name-casing": "error",
"@typescript-eslint/explicit-function-return-type": ["error", {"allowExpressions": true}],
"@typescript-eslint/func-call-spacing": ["error", "never"],
"@typescript-eslint/generic-type-naming": ["error", "^[A-Z][A-Za-z]*$"],
"@typescript-eslint/no-array-constructor": "error",
"@typescript-eslint/no-empty-interface": "error",
"@typescript-eslint/no-explicit-any": "error",
"@typescript-eslint/no-extraneous-class": "error",
"@typescript-eslint/no-for-in-array": "error",
"@typescript-eslint/no-inferrable-types": "error",
"@typescript-eslint/no-misused-new": "error",
"@typescript-eslint/no-namespace": "error",
"@typescript-eslint/no-non-null-assertion": "warn",
"@typescript-eslint/no-object-literal-type-assertion": "error",
"@typescript-eslint/no-unnecessary-qualifier": "error",
"@typescript-eslint/no-unnecessary-type-assertion": "error",
"@typescript-eslint/no-useless-constructor": "error",
"@typescript-eslint/no-var-requires": "error",
"@typescript-eslint/prefer-for-of": "warn",
"@typescript-eslint/prefer-function-type": "warn",
"@typescript-eslint/prefer-includes": "error",
"@typescript-eslint/prefer-interface": "error",
"@typescript-eslint/prefer-string-starts-ends-with": "error",
"@typescript-eslint/promise-function-async": "error",
"@typescript-eslint/require-array-sort-compare": "error",
"@typescript-eslint/restrict-plus-operands": "error",
"semi": "off",
"@typescript-eslint/semi": ["error", "never"],
"@typescript-eslint/type-annotation-spacing": "error",
"@typescript-eslint/unbound-method": "error"
},
"env": {
"node": true,
"es6": true,
"jest/globals": true
}
}

View File

@@ -0,0 +1,3 @@
dist/
lib/
node_modules/

View File

@@ -0,0 +1,11 @@
{
"printWidth": 80,
"tabWidth": 2,
"useTabs": false,
"semi": false,
"singleQuote": true,
"trailingComma": "none",
"bracketSpacing": false,
"arrowParens": "avoid",
"parser": "typescript"
}

View File

@@ -0,0 +1 @@
To update hashFiles under `Misc/layoutbin` run `npm install && npm run all`

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,35 @@
{
"name": "hashFiles",
"version": "1.0.0",
"description": "GitHub Actions HashFiles() expression function",
"main": "lib/hashFiles.js",
"scripts": {
"build": "tsc",
"format": "prettier --write **/*.ts",
"format-check": "prettier --check **/*.ts",
"lint": "eslint src/**/*.ts",
"pack": "ncc build -o ../../layoutbin/hashFiles",
"all": "npm run build && npm run format && npm run lint && npm run pack"
},
"repository": {
"type": "git",
"url": "git+https://github.com/actions/runner.git"
},
"keywords": [
"actions"
],
"author": "GitHub Actions",
"license": "MIT",
"dependencies": {
"@actions/glob": "^0.1.0"
},
"devDependencies": {
"@types/node": "^12.7.12",
"@typescript-eslint/parser": "^2.8.0",
"@zeit/ncc": "^0.20.5",
"eslint": "^5.16.0",
"eslint-plugin-github": "^2.0.0",
"prettier": "^1.19.1",
"typescript": "^3.6.4"
}
}

View File

@@ -0,0 +1,55 @@
import * as glob from '@actions/glob'
import * as crypto from 'crypto'
import * as fs from 'fs'
import * as stream from 'stream'
import * as util from 'util'
import * as path from 'path'
async function run(): Promise<void> {
// arg0 -> node
// arg1 -> hashFiles.js
// env[followSymbolicLinks] = true/null
// env[patterns] -> glob patterns
let followSymbolicLinks = false
const matchPatterns = process.env.patterns || ''
if (process.env.followSymbolicLinks === 'true') {
console.log('Follow symbolic links')
followSymbolicLinks = true
}
console.log(`Match Pattern: ${matchPatterns}`)
let hasMatch = false
const githubWorkspace = process.cwd()
const result = crypto.createHash('sha256')
let count = 0
const globber = await glob.create(matchPatterns, {followSymbolicLinks})
for await (const file of globber.globGenerator()) {
console.log(file)
if (!file.startsWith(`${githubWorkspace}${path.sep}`)) {
console.log(`Ignore '${file}' since it is not under GITHUB_WORKSPACE.`)
continue
}
if (fs.statSync(file).isDirectory()) {
console.log(`Skip directory '${file}'.`)
continue
}
const hash = crypto.createHash('sha256')
const pipeline = util.promisify(stream.pipeline)
await pipeline(fs.createReadStream(file), hash)
result.write(hash.digest())
count++
if (!hasMatch) {
hasMatch = true
}
}
result.end()
if (hasMatch) {
console.log(`Find ${count} files to hash.`)
console.error(`__OUTPUT__${result.digest('hex')}__OUTPUT__`)
} else {
console.error(`__OUTPUT____OUTPUT__`)
}
}
run()

View File

@@ -0,0 +1,12 @@
{
"compilerOptions": {
"target": "es6", /* Specify ECMAScript target version: 'ES3' (default), 'ES5', 'ES2015', 'ES2016', 'ES2017', 'ES2018', 'ES2019' or 'ESNEXT'. */
"module": "commonjs", /* Specify module code generation: 'none', 'commonjs', 'amd', 'system', 'umd', 'es2015', or 'ESNext'. */
"outDir": "./lib", /* Redirect output structure to the directory. */
"rootDir": "./src", /* Specify the root directory of input files. Use to control the output directory structure with --outDir. */
"strict": true, /* Enable all strict type-checking options. */
"noImplicitAny": true, /* Raise error on expressions and declarations with an implied 'any' type. */
"esModuleInterop": true /* Enables emit interoperability between CommonJS and ES Modules via creation of namespace objects for all imports. Implies 'allowSyntheticDefaultImports'. */
},
"exclude": ["node_modules", "**/*.test.ts"]
}

File diff suppressed because it is too large Load Diff

View File

@@ -3,7 +3,7 @@
user_id=`id -u`
# we want to snapshot the environment of the config user
if [ $user_id -eq 0 -a -z "$AGENT_ALLOW_RUNASROOT" ]; then
if [ $user_id -eq 0 -a -z "$RUNNER_ALLOW_RUNASROOT" ]; then
echo "Must not run with sudo"
exit 1
fi

View File

@@ -2,7 +2,7 @@
# Validate not sudo
user_id=`id -u`
if [ $user_id -eq 0 -a -z "$AGENT_ALLOW_RUNASROOT" ]; then
if [ $user_id -eq 0 -a -z "$RUNNER_ALLOW_RUNASROOT" ]; then
echo "Must not run interactively with sudo"
exit 1
fi
@@ -26,8 +26,8 @@ if [[ "$1" == "localRun" ]]; then
else
"$DIR"/bin/Runner.Listener run $*
# Return code 4 means the run once agent received an update message.
# Sleep 5 seconds to wait for the update process finish and run the agent again.
# Return code 4 means the run once runner received an update message.
# Sleep 5 seconds to wait for the update process finish and run the runner again.
returnCode=$?
if [[ $returnCode == 4 ]]; then
if [ ! -x "$(command -v sleep)" ]; then

View File

@@ -3,8 +3,6 @@
<packageSources>
<!--To inherit the global NuGet package sources remove the <clear/> line below -->
<clear />
<add key="dotnet-core" value="https://www.myget.org/F/dotnet-core/api/v3/index.json" />
<add key="dotnet-buildtools" value="https://www.myget.org/F/dotnet-buildtools/api/v3/index.json" />
<add key="api.nuget.org" value="https://api.nuget.org/v3/index.json" />
</packageSources>
</configuration>

View File

@@ -9,26 +9,27 @@ namespace GitHub.Runner.Common
{
private static readonly EscapeMapping[] _escapeMappings = new[]
{
new EscapeMapping(token: "%", replacement: "%25"),
new EscapeMapping(token: ";", replacement: "%3B"),
new EscapeMapping(token: "\r", replacement: "%0D"),
new EscapeMapping(token: "\n", replacement: "%0A"),
new EscapeMapping(token: "]", replacement: "%5D"),
new EscapeMapping(token: "%", replacement: "%25"),
};
private static readonly EscapeMapping[] _escapeDataMappings = new[]
{
new EscapeMapping(token: "\r", replacement: "%0D"),
new EscapeMapping(token: "\n", replacement: "%0A"),
new EscapeMapping(token: "%", replacement: "%25"),
};
private static readonly EscapeMapping[] _escapePropertyMappings = new[]
{
new EscapeMapping(token: "%", replacement: "%25"),
new EscapeMapping(token: "\r", replacement: "%0D"),
new EscapeMapping(token: "\n", replacement: "%0A"),
new EscapeMapping(token: ":", replacement: "%3A"),
new EscapeMapping(token: ",", replacement: "%2C"),
new EscapeMapping(token: "%", replacement: "%25"),
};
private readonly Dictionary<string, string> _properties = new Dictionary<string, string>(StringComparer.OrdinalIgnoreCase);

View File

@@ -1,33 +0,0 @@
using System.Threading;
using System.Threading.Tasks;
namespace GitHub.Runner.Common
{
//Stephen Toub: http://blogs.msdn.com/b/pfxteam/archive/2012/02/11/10266920.aspx
public class AsyncManualResetEvent
{
private volatile TaskCompletionSource<bool> m_tcs = new TaskCompletionSource<bool>();
public Task WaitAsync() { return m_tcs.Task; }
public void Set()
{
var tcs = m_tcs;
Task.Factory.StartNew(s => ((TaskCompletionSource<bool>)s).TrySetResult(true),
tcs, CancellationToken.None, TaskCreationOptions.PreferFairness, TaskScheduler.Default);
tcs.Task.Wait();
}
public void Reset()
{
while (true)
{
var tcs = m_tcs;
if (!tcs.Task.IsCompleted ||
Interlocked.CompareExchange(ref m_tcs, new TaskCompletionSource<bool>(), tcs) == tcs)
return;
}
}
}
}

View File

@@ -71,15 +71,6 @@ namespace GitHub.Runner.Common
}
}
[DataContract]
public sealed class RunnerRuntimeOptions
{
#if OS_WINDOWS
[DataMember(EmitDefaultValue = false)]
public bool GitUseSecureChannel { get; set; }
#endif
}
[ServiceLocator(Default = typeof(ConfigurationStore))]
public interface IConfigurationStore : IRunnerService
{
@@ -92,9 +83,6 @@ namespace GitHub.Runner.Common
void SaveSettings(RunnerSettings settings);
void DeleteCredential();
void DeleteSettings();
RunnerRuntimeOptions GetRunnerRuntimeOptions();
void SaveRunnerRuntimeOptions(RunnerRuntimeOptions options);
void DeleteRunnerRuntimeOptions();
}
public sealed class ConfigurationStore : RunnerService, IConfigurationStore
@@ -103,11 +91,9 @@ namespace GitHub.Runner.Common
private string _configFilePath;
private string _credFilePath;
private string _serviceConfigFilePath;
private string _runtimeOptionsFilePath;
private CredentialData _creds;
private RunnerSettings _settings;
private RunnerRuntimeOptions _runtimeOptions;
public override void Initialize(IHostContext hostContext)
{
@@ -130,9 +116,6 @@ namespace GitHub.Runner.Common
_serviceConfigFilePath = hostContext.GetConfigFile(WellKnownConfigFile.Service);
Trace.Info("ServiceConfigFilePath: {0}", _serviceConfigFilePath);
_runtimeOptionsFilePath = hostContext.GetConfigFile(WellKnownConfigFile.Options);
Trace.Info("RuntimeOptionsFilePath: {0}", _runtimeOptionsFilePath);
}
public string RootFolder { get; private set; }
@@ -229,35 +212,5 @@ namespace GitHub.Runner.Common
{
IOUtil.Delete(_configFilePath, default(CancellationToken));
}
public RunnerRuntimeOptions GetRunnerRuntimeOptions()
{
if (_runtimeOptions == null && File.Exists(_runtimeOptionsFilePath))
{
_runtimeOptions = IOUtil.LoadObject<RunnerRuntimeOptions>(_runtimeOptionsFilePath);
}
return _runtimeOptions;
}
public void SaveRunnerRuntimeOptions(RunnerRuntimeOptions options)
{
Trace.Info("Saving runtime options.");
if (File.Exists(_runtimeOptionsFilePath))
{
// Delete existing runtime options file first, since the file is hidden and not able to overwrite.
Trace.Info("Delete exist runtime options file.");
IOUtil.DeleteFile(_runtimeOptionsFilePath);
}
IOUtil.SaveObject(options, _runtimeOptionsFilePath);
Trace.Info("Options Saved.");
File.SetAttributes(_runtimeOptionsFilePath, File.GetAttributes(_runtimeOptionsFilePath) | FileAttributes.Hidden);
}
public void DeleteRunnerRuntimeOptions()
{
IOUtil.Delete(_runtimeOptionsFilePath, default(CancellationToken));
}
}
}

View File

@@ -88,10 +88,6 @@ namespace GitHub.Runner.Common
public static readonly string MonitorSocketAddress = "monitorsocketaddress";
public static readonly string Name = "name";
public static readonly string Pool = "pool";
public static readonly string SslCACert = "sslcacert";
public static readonly string SslClientCert = "sslclientcert";
public static readonly string SslClientCertKey = "sslclientcertkey";
public static readonly string SslClientCertArchive = "sslclientcertarchive";
public static readonly string StartupType = "startuptype";
public static readonly string Url = "url";
public static readonly string UserName = "username";
@@ -99,14 +95,10 @@ namespace GitHub.Runner.Common
public static readonly string Work = "work";
// Secret args. Must be added to the "Secrets" getter as well.
public static readonly string Password = "password";
public static readonly string SslClientCertPassword = "sslclientcertpassword";
public static readonly string Token = "token";
public static readonly string WindowsLogonPassword = "windowslogonpassword";
public static string[] Secrets => new[]
{
Password,
SslClientCertPassword,
Token,
WindowsLogonPassword,
};
@@ -125,13 +117,10 @@ namespace GitHub.Runner.Common
public static class Flags
{
public static readonly string Commit = "commit";
public static readonly string GitUseSChannel = "gituseschannel";
public static readonly string Help = "help";
public static readonly string Replace = "replace";
public static readonly string LaunchBrowser = "launchbrowser";
public static readonly string Once = "once";
public static readonly string RunAsService = "runasservice";
public static readonly string SslSkipCertValidation = "sslskipcertvalidation";
public static readonly string Unattended = "unattended";
public static readonly string Version = "version";
}
@@ -173,7 +162,8 @@ namespace GitHub.Runner.Common
public static class Path
{
public static readonly string ActionsDirectory = "_actions";
public static readonly string ActionManifestFile = "action.yml";
public static readonly string ActionManifestYmlFile = "action.yml";
public static readonly string ActionManifestYamlFile = "action.yaml";
public static readonly string BinDirectory = "bin";
public static readonly string DiagDirectory = "_diag";
public static readonly string ExternalsDirectory = "externals";
@@ -200,6 +190,11 @@ namespace GitHub.Runner.Common
public static readonly string StepDebug = "ACTIONS_STEP_DEBUG";
}
public static class Agent
{
public static readonly string ToolsDirectory = "agent.ToolsDirectory";
}
public static class System
{
//

View File

@@ -58,8 +58,8 @@ namespace GitHub.Runner.Common
private CancellationTokenSource _runnerShutdownTokenSource = new CancellationTokenSource();
private object _perfLock = new object();
private Tracing _trace;
private Tracing _vssTrace;
private Tracing _httpTrace;
private Tracing _actionsHttpTrace;
private Tracing _netcoreHttpTrace;
private ITraceManager _traceManager;
private AssemblyLoadContext _loadContext;
private IDisposable _httpTraceSubscription;
@@ -83,6 +83,7 @@ namespace GitHub.Runner.Common
_loadContext.Unloading += LoadContext_Unloading;
this.SecretMasker.AddValueEncoder(ValueEncoders.Base64StringEscape);
this.SecretMasker.AddValueEncoder(ValueEncoders.Base64StringEscapeTrimmed);
this.SecretMasker.AddValueEncoder(ValueEncoders.Base64StringEscapeShift1);
this.SecretMasker.AddValueEncoder(ValueEncoders.Base64StringEscapeShift2);
this.SecretMasker.AddValueEncoder(ValueEncoders.ExpressionStringEscape);
@@ -117,8 +118,7 @@ namespace GitHub.Runner.Common
}
_trace = GetTrace(nameof(HostContext));
_vssTrace = GetTrace("GitHubActionsRunner"); // VisualStudioService
_actionsHttpTrace = GetTrace("GitHubActionsService");
// Enable Http trace
bool enableHttpTrace;
if (bool.TryParse(Environment.GetEnvironmentVariable("GITHUB_ACTIONS_RUNNER_HTTPTRACE"), out enableHttpTrace) && enableHttpTrace)
@@ -130,7 +130,7 @@ namespace GitHub.Runner.Common
_trace.Warning("** **");
_trace.Warning("*****************************************************************************************");
_httpTrace = GetTrace("HttpTrace");
_netcoreHttpTrace = GetTrace("HttpTrace");
_diagListenerSubscription = DiagnosticListener.AllListeners.Subscribe(this);
}
@@ -230,8 +230,9 @@ namespace GitHub.Runner.Common
break;
case WellKnownDirectory.Tools:
path = Environment.GetEnvironmentVariable("RUNNER_TOOL_CACHE");
// TODO: Coallesce to just check RUNNER_TOOL_CACHE when images stabilize
path = Environment.GetEnvironmentVariable("RUNNER_TOOL_CACHE") ?? Environment.GetEnvironmentVariable("RUNNER_TOOLSDIRECTORY") ?? Environment.GetEnvironmentVariable("AGENT_TOOLSDIRECTORY") ?? Environment.GetEnvironmentVariable(Constants.Variables.Agent.ToolsDirectory);
if (string.IsNullOrEmpty(path))
{
path = Path.Combine(
@@ -478,12 +479,12 @@ namespace GitHub.Runner.Common
void IObserver<DiagnosticListener>.OnCompleted()
{
_httpTrace.Info("DiagListeners finished transmitting data.");
_netcoreHttpTrace.Info("DiagListeners finished transmitting data.");
}
void IObserver<DiagnosticListener>.OnError(Exception error)
{
_httpTrace.Error(error);
_netcoreHttpTrace.Error(error);
}
void IObserver<DiagnosticListener>.OnNext(DiagnosticListener listener)
@@ -496,22 +497,22 @@ namespace GitHub.Runner.Common
void IObserver<KeyValuePair<string, object>>.OnCompleted()
{
_httpTrace.Info("HttpHandlerDiagnosticListener finished transmitting data.");
_netcoreHttpTrace.Info("HttpHandlerDiagnosticListener finished transmitting data.");
}
void IObserver<KeyValuePair<string, object>>.OnError(Exception error)
{
_httpTrace.Error(error);
_netcoreHttpTrace.Error(error);
}
void IObserver<KeyValuePair<string, object>>.OnNext(KeyValuePair<string, object> value)
{
_httpTrace.Info($"Trace {value.Key} event:{Environment.NewLine}{value.Value.ToString()}");
_netcoreHttpTrace.Info($"Trace {value.Key} event:{Environment.NewLine}{value.Value.ToString()}");
}
protected override void OnEventSourceCreated(EventSource source)
{
if (source.Name.Equals("Microsoft-VSS-Http"))
if (source.Name.Equals("GitHub-Actions-Http"))
{
EnableEvents(source, EventLevel.Verbose);
}
@@ -551,24 +552,24 @@ namespace GitHub.Runner.Common
{
case EventLevel.Critical:
case EventLevel.Error:
_vssTrace.Error(message);
_actionsHttpTrace.Error(message);
break;
case EventLevel.Warning:
_vssTrace.Warning(message);
_actionsHttpTrace.Warning(message);
break;
case EventLevel.Informational:
_vssTrace.Info(message);
_actionsHttpTrace.Info(message);
break;
default:
_vssTrace.Verbose(message);
_actionsHttpTrace.Verbose(message);
break;
}
}
catch (Exception ex)
{
_vssTrace.Error(ex);
_vssTrace.Info(eventData.Message);
_vssTrace.Info(string.Join(", ", eventData.Payload?.ToArray() ?? new string[0]));
_actionsHttpTrace.Error(ex);
_actionsHttpTrace.Info(eventData.Message);
_actionsHttpTrace.Info(string.Join(", ", eventData.Payload?.ToArray() ?? new string[0]));
}
}

View File

@@ -24,7 +24,6 @@ namespace GitHub.Runner.Common
private Guid _timelineId;
private Guid _timelineRecordId;
private string _pageId;
private FileStream _pageData;
private StreamWriter _pageWriter;
private int _byteCount;
@@ -40,7 +39,6 @@ namespace GitHub.Runner.Common
{
base.Initialize(hostContext);
_totalLines = 0;
_pageId = Guid.NewGuid().ToString();
_pagesFolder = Path.Combine(hostContext.GetDirectory(WellKnownDirectory.Diag), PagingFolder);
_jobServerQueue = HostContext.GetService<IJobServerQueue>();
Directory.CreateDirectory(_pagesFolder);
@@ -102,7 +100,7 @@ namespace GitHub.Runner.Common
{
EndPage();
_byteCount = 0;
_dataFileName = Path.Combine(_pagesFolder, $"{_pageId}_{++_pageCount}.log");
_dataFileName = Path.Combine(_pagesFolder, $"{_timelineId}_{_timelineRecordId}_{++_pageCount}.log");
_pageData = new FileStream(_dataFileName, FileMode.CreateNew);
_pageWriter = new StreamWriter(_pageData, System.Text.Encoding.UTF8);
}

View File

@@ -1,7 +1,7 @@
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFramework>netcoreapp3.0</TargetFramework>
<TargetFramework>netcoreapp3.1</TargetFramework>
<OutputType>Library</OutputType>
<RuntimeIdentifiers>win-x64;win-x86;linux-x64;linux-arm64;linux-arm;osx-x64</RuntimeIdentifiers>
<TargetLatestRuntimePatch>true</TargetLatestRuntimePatch>

View File

@@ -1,231 +0,0 @@
using System;
using GitHub.Runner.Common.Util;
using System.IO;
using System.Runtime.Serialization;
using GitHub.Services.Common;
using System.Security.Cryptography.X509Certificates;
using System.Net;
using System.Net.Security;
using System.Net.Http;
using GitHub.Services.WebApi;
using GitHub.Runner.Sdk;
namespace GitHub.Runner.Common
{
[ServiceLocator(Default = typeof(RunnerCertificateManager))]
public interface IRunnerCertificateManager : IRunnerService
{
bool SkipServerCertificateValidation { get; }
string CACertificateFile { get; }
string ClientCertificateFile { get; }
string ClientCertificatePrivateKeyFile { get; }
string ClientCertificateArchiveFile { get; }
string ClientCertificatePassword { get; }
IVssClientCertificateManager VssClientCertificateManager { get; }
}
public class RunnerCertificateManager : RunnerService, IRunnerCertificateManager
{
private RunnerClientCertificateManager _runnerClientCertificateManager = new RunnerClientCertificateManager();
public bool SkipServerCertificateValidation { private set; get; }
public string CACertificateFile { private set; get; }
public string ClientCertificateFile { private set; get; }
public string ClientCertificatePrivateKeyFile { private set; get; }
public string ClientCertificateArchiveFile { private set; get; }
public string ClientCertificatePassword { private set; get; }
public IVssClientCertificateManager VssClientCertificateManager => _runnerClientCertificateManager;
public override void Initialize(IHostContext hostContext)
{
base.Initialize(hostContext);
LoadCertificateSettings();
}
// This should only be called from config
public void SetupCertificate(bool skipCertValidation, string caCert, string clientCert, string clientCertPrivateKey, string clientCertArchive, string clientCertPassword)
{
Trace.Info("Setup runner certificate setting base on configuration inputs.");
if (skipCertValidation)
{
Trace.Info("Ignore SSL server certificate validation error");
SkipServerCertificateValidation = true;
VssClientHttpRequestSettings.Default.ServerCertificateValidationCallback = HttpClientHandler.DangerousAcceptAnyServerCertificateValidator;
}
if (!string.IsNullOrEmpty(caCert))
{
ArgUtil.File(caCert, nameof(caCert));
Trace.Info($"Self-Signed CA '{caCert}'");
}
if (!string.IsNullOrEmpty(clientCert))
{
ArgUtil.File(clientCert, nameof(clientCert));
ArgUtil.File(clientCertPrivateKey, nameof(clientCertPrivateKey));
ArgUtil.File(clientCertArchive, nameof(clientCertArchive));
Trace.Info($"Client cert '{clientCert}'");
Trace.Info($"Client cert private key '{clientCertPrivateKey}'");
Trace.Info($"Client cert archive '{clientCertArchive}'");
}
CACertificateFile = caCert;
ClientCertificateFile = clientCert;
ClientCertificatePrivateKeyFile = clientCertPrivateKey;
ClientCertificateArchiveFile = clientCertArchive;
ClientCertificatePassword = clientCertPassword;
_runnerClientCertificateManager.AddClientCertificate(ClientCertificateArchiveFile, ClientCertificatePassword);
}
// This should only be called from config
public void SaveCertificateSetting()
{
string certSettingFile = HostContext.GetConfigFile(WellKnownConfigFile.Certificates);
IOUtil.DeleteFile(certSettingFile);
var setting = new RunnerCertificateSetting();
if (SkipServerCertificateValidation)
{
Trace.Info($"Store Skip ServerCertificateValidation setting to '{certSettingFile}'");
setting.SkipServerCertValidation = true;
}
if (!string.IsNullOrEmpty(CACertificateFile))
{
Trace.Info($"Store CA cert setting to '{certSettingFile}'");
setting.CACert = CACertificateFile;
}
if (!string.IsNullOrEmpty(ClientCertificateFile) &&
!string.IsNullOrEmpty(ClientCertificatePrivateKeyFile) &&
!string.IsNullOrEmpty(ClientCertificateArchiveFile))
{
Trace.Info($"Store client cert settings to '{certSettingFile}'");
setting.ClientCert = ClientCertificateFile;
setting.ClientCertPrivatekey = ClientCertificatePrivateKeyFile;
setting.ClientCertArchive = ClientCertificateArchiveFile;
if (!string.IsNullOrEmpty(ClientCertificatePassword))
{
string lookupKey = Guid.NewGuid().ToString("D").ToUpperInvariant();
Trace.Info($"Store client cert private key password with lookup key {lookupKey}");
var credStore = HostContext.GetService<IRunnerCredentialStore>();
credStore.Write($"GITHUB_ACTIONS_RUNNER_CLIENT_CERT_PASSWORD_{lookupKey}", "GitHub", ClientCertificatePassword);
setting.ClientCertPasswordLookupKey = lookupKey;
}
}
if (SkipServerCertificateValidation ||
!string.IsNullOrEmpty(CACertificateFile) ||
!string.IsNullOrEmpty(ClientCertificateFile))
{
IOUtil.SaveObject(setting, certSettingFile);
File.SetAttributes(certSettingFile, File.GetAttributes(certSettingFile) | FileAttributes.Hidden);
}
}
// This should only be called from unconfig
public void DeleteCertificateSetting()
{
string certSettingFile = HostContext.GetConfigFile(WellKnownConfigFile.Certificates);
if (File.Exists(certSettingFile))
{
Trace.Info($"Load runner certificate setting from '{certSettingFile}'");
var certSetting = IOUtil.LoadObject<RunnerCertificateSetting>(certSettingFile);
if (certSetting != null && !string.IsNullOrEmpty(certSetting.ClientCertPasswordLookupKey))
{
Trace.Info("Delete client cert private key password from credential store.");
var credStore = HostContext.GetService<IRunnerCredentialStore>();
credStore.Delete($"GITHUB_ACTIONS_RUNNER_CLIENT_CERT_PASSWORD_{certSetting.ClientCertPasswordLookupKey}");
}
Trace.Info($"Delete cert setting file: {certSettingFile}");
IOUtil.DeleteFile(certSettingFile);
}
}
public void LoadCertificateSettings()
{
string certSettingFile = HostContext.GetConfigFile(WellKnownConfigFile.Certificates);
if (File.Exists(certSettingFile))
{
Trace.Info($"Load runner certificate setting from '{certSettingFile}'");
var certSetting = IOUtil.LoadObject<RunnerCertificateSetting>(certSettingFile);
ArgUtil.NotNull(certSetting, nameof(RunnerCertificateSetting));
if (certSetting.SkipServerCertValidation)
{
Trace.Info("Ignore SSL server certificate validation error");
SkipServerCertificateValidation = true;
VssClientHttpRequestSettings.Default.ServerCertificateValidationCallback = HttpClientHandler.DangerousAcceptAnyServerCertificateValidator;
}
if (!string.IsNullOrEmpty(certSetting.CACert))
{
// make sure all settings file exist
ArgUtil.File(certSetting.CACert, nameof(certSetting.CACert));
Trace.Info($"CA '{certSetting.CACert}'");
CACertificateFile = certSetting.CACert;
}
if (!string.IsNullOrEmpty(certSetting.ClientCert))
{
// make sure all settings file exist
ArgUtil.File(certSetting.ClientCert, nameof(certSetting.ClientCert));
ArgUtil.File(certSetting.ClientCertPrivatekey, nameof(certSetting.ClientCertPrivatekey));
ArgUtil.File(certSetting.ClientCertArchive, nameof(certSetting.ClientCertArchive));
Trace.Info($"Client cert '{certSetting.ClientCert}'");
Trace.Info($"Client cert private key '{certSetting.ClientCertPrivatekey}'");
Trace.Info($"Client cert archive '{certSetting.ClientCertArchive}'");
ClientCertificateFile = certSetting.ClientCert;
ClientCertificatePrivateKeyFile = certSetting.ClientCertPrivatekey;
ClientCertificateArchiveFile = certSetting.ClientCertArchive;
if (!string.IsNullOrEmpty(certSetting.ClientCertPasswordLookupKey))
{
var cerdStore = HostContext.GetService<IRunnerCredentialStore>();
ClientCertificatePassword = cerdStore.Read($"GITHUB_ACTIONS_RUNNER_CLIENT_CERT_PASSWORD_{certSetting.ClientCertPasswordLookupKey}").Password;
HostContext.SecretMasker.AddValue(ClientCertificatePassword);
}
_runnerClientCertificateManager.AddClientCertificate(ClientCertificateArchiveFile, ClientCertificatePassword);
}
}
else
{
Trace.Info("No certificate setting found.");
}
}
}
[DataContract]
internal class RunnerCertificateSetting
{
[DataMember]
public bool SkipServerCertValidation { get; set; }
[DataMember]
public string CACert { get; set; }
[DataMember]
public string ClientCert { get; set; }
[DataMember]
public string ClientCertPrivatekey { get; set; }
[DataMember]
public string ClientCertArchive { get; set; }
[DataMember]
public string ClientCertPasswordLookupKey { get; set; }
}
}

View File

@@ -1,948 +0,0 @@
using System;
using System.Collections.Concurrent;
using System.Collections.Generic;
using System.ComponentModel;
using System.Linq;
using System.Net;
using System.Runtime.InteropServices;
using System.Text;
using System.Text.RegularExpressions;
using System.Threading;
using GitHub.Runner.Common.Util;
using Newtonsoft.Json;
using System.IO;
using System.Runtime.Serialization;
using System.Security.Cryptography;
using GitHub.Runner.Sdk;
namespace GitHub.Runner.Common
{
// The purpose of this class is to store user's credential during runner configuration and retrive the credential back at runtime.
#if OS_WINDOWS
[ServiceLocator(Default = typeof(WindowsRunnerCredentialStore))]
#elif OS_OSX
[ServiceLocator(Default = typeof(MacOSRunnerCredentialStore))]
#else
[ServiceLocator(Default = typeof(LinuxRunnerCredentialStore))]
#endif
public interface IRunnerCredentialStore : IRunnerService
{
NetworkCredential Write(string target, string username, string password);
// throw exception when target not found from cred store
NetworkCredential Read(string target);
// throw exception when target not found from cred store
void Delete(string target);
}
#if OS_WINDOWS
// Windows credential store is per user.
// This is a limitation for user configure the runner run as windows service, when user's current login account is different with the service run as account.
// Ex: I login the box as domain\admin, configure the runner as windows service and run as domian\buildserver
// domain\buildserver won't read the stored credential from domain\admin's windows credential store.
// To workaround this limitation.
// Anytime we try to save a credential:
// 1. store it into current user's windows credential store
// 2. use DP-API do a machine level encrypt and store the encrypted content on disk.
// At the first time we try to read the credential:
// 1. read from current user's windows credential store, delete the DP-API encrypted backup content on disk if the windows credential store read succeed.
// 2. if credential not found in current user's windows credential store, read from the DP-API encrypted backup content on disk,
// write the credential back the current user's windows credential store and delete the backup on disk.
public sealed class WindowsRunnerCredentialStore : RunnerService, IRunnerCredentialStore
{
private string _credStoreFile;
private Dictionary<string, string> _credStore;
public override void Initialize(IHostContext hostContext)
{
base.Initialize(hostContext);
_credStoreFile = hostContext.GetConfigFile(WellKnownConfigFile.CredentialStore);
if (File.Exists(_credStoreFile))
{
_credStore = IOUtil.LoadObject<Dictionary<string, string>>(_credStoreFile);
}
else
{
_credStore = new Dictionary<string, string>(StringComparer.OrdinalIgnoreCase);
}
}
public NetworkCredential Write(string target, string username, string password)
{
Trace.Entering();
ArgUtil.NotNullOrEmpty(target, nameof(target));
ArgUtil.NotNullOrEmpty(username, nameof(username));
ArgUtil.NotNullOrEmpty(password, nameof(password));
// save to .credential_store file first, then Windows credential store
string usernameBase64 = Convert.ToBase64String(Encoding.UTF8.GetBytes(username));
string passwordBase64 = Convert.ToBase64String(Encoding.UTF8.GetBytes(password));
// Base64Username:Base64Password -> DP-API machine level encrypt -> Base64Encoding
string encryptedUsernamePassword = Convert.ToBase64String(ProtectedData.Protect(Encoding.UTF8.GetBytes($"{usernameBase64}:{passwordBase64}"), null, DataProtectionScope.LocalMachine));
Trace.Info($"Credentials for '{target}' written to credential store file.");
_credStore[target] = encryptedUsernamePassword;
// save to .credential_store file
SyncCredentialStoreFile();
// save to Windows Credential Store
return WriteInternal(target, username, password);
}
public NetworkCredential Read(string target)
{
Trace.Entering();
ArgUtil.NotNullOrEmpty(target, nameof(target));
IntPtr credPtr = IntPtr.Zero;
try
{
if (CredRead(target, CredentialType.Generic, 0, out credPtr))
{
Credential credStruct = (Credential)Marshal.PtrToStructure(credPtr, typeof(Credential));
int passwordLength = (int)credStruct.CredentialBlobSize;
string password = passwordLength > 0 ? Marshal.PtrToStringUni(credStruct.CredentialBlob, passwordLength / sizeof(char)) : String.Empty;
string username = Marshal.PtrToStringUni(credStruct.UserName);
Trace.Info($"Credentials for '{target}' read from windows credential store.");
// delete from .credential_store file since we are able to read it from windows credential store
if (_credStore.Remove(target))
{
Trace.Info($"Delete credentials for '{target}' from credential store file.");
SyncCredentialStoreFile();
}
return new NetworkCredential(username, password);
}
else
{
// Can't read from Windows Credential Store, fail back to .credential_store file
if (_credStore.ContainsKey(target) && !string.IsNullOrEmpty(_credStore[target]))
{
Trace.Info($"Credentials for '{target}' read from credential store file.");
// Base64Decode -> DP-API machine level decrypt -> Base64Username:Base64Password -> Base64Decode
string decryptedUsernamePassword = Encoding.UTF8.GetString(ProtectedData.Unprotect(Convert.FromBase64String(_credStore[target]), null, DataProtectionScope.LocalMachine));
string[] credential = decryptedUsernamePassword.Split(':');
if (credential.Length == 2 && !string.IsNullOrEmpty(credential[0]) && !string.IsNullOrEmpty(credential[1]))
{
string username = Encoding.UTF8.GetString(Convert.FromBase64String(credential[0]));
string password = Encoding.UTF8.GetString(Convert.FromBase64String(credential[1]));
// store back to windows credential store for current user
NetworkCredential creds = WriteInternal(target, username, password);
// delete from .credential_store file since we are able to write the credential to windows credential store for current user.
if (_credStore.Remove(target))
{
Trace.Info($"Delete credentials for '{target}' from credential store file.");
SyncCredentialStoreFile();
}
return creds;
}
else
{
throw new ArgumentOutOfRangeException(nameof(decryptedUsernamePassword));
}
}
throw new Win32Exception(Marshal.GetLastWin32Error(), $"CredRead throw an error for '{target}'");
}
}
finally
{
if (credPtr != IntPtr.Zero)
{
CredFree(credPtr);
}
}
}
public void Delete(string target)
{
Trace.Entering();
ArgUtil.NotNullOrEmpty(target, nameof(target));
// remove from .credential_store file
if (_credStore.Remove(target))
{
Trace.Info($"Delete credentials for '{target}' from credential store file.");
SyncCredentialStoreFile();
}
// remove from windows credential store
if (!CredDelete(target, CredentialType.Generic, 0))
{
throw new Win32Exception(Marshal.GetLastWin32Error(), $"Failed to delete credentials for {target}");
}
else
{
Trace.Info($"Credentials for '{target}' deleted from windows credential store.");
}
}
private NetworkCredential WriteInternal(string target, string username, string password)
{
// save to Windows Credential Store
Credential credential = new Credential()
{
Type = CredentialType.Generic,
Persist = (UInt32)CredentialPersist.LocalMachine,
TargetName = Marshal.StringToCoTaskMemUni(target),
UserName = Marshal.StringToCoTaskMemUni(username),
CredentialBlob = Marshal.StringToCoTaskMemUni(password),
CredentialBlobSize = (UInt32)Encoding.Unicode.GetByteCount(password),
AttributeCount = 0,
Comment = IntPtr.Zero,
Attributes = IntPtr.Zero,
TargetAlias = IntPtr.Zero
};
try
{
if (CredWrite(ref credential, 0))
{
Trace.Info($"Credentials for '{target}' written to windows credential store.");
return new NetworkCredential(username, password);
}
else
{
int error = Marshal.GetLastWin32Error();
throw new Win32Exception(error, "Failed to write credentials");
}
}
finally
{
if (credential.CredentialBlob != IntPtr.Zero)
{
Marshal.FreeCoTaskMem(credential.CredentialBlob);
}
if (credential.TargetName != IntPtr.Zero)
{
Marshal.FreeCoTaskMem(credential.TargetName);
}
if (credential.UserName != IntPtr.Zero)
{
Marshal.FreeCoTaskMem(credential.UserName);
}
}
}
private void SyncCredentialStoreFile()
{
Trace.Info("Sync in-memory credential store with credential store file.");
// delete the cred store file first anyway, since it's a readonly file.
IOUtil.DeleteFile(_credStoreFile);
// delete cred store file when all creds gone
if (_credStore.Count == 0)
{
return;
}
else
{
IOUtil.SaveObject(_credStore, _credStoreFile);
File.SetAttributes(_credStoreFile, File.GetAttributes(_credStoreFile) | FileAttributes.Hidden);
}
}
[DllImport("Advapi32.dll", EntryPoint = "CredDeleteW", CharSet = CharSet.Unicode, SetLastError = true)]
internal static extern bool CredDelete(string target, CredentialType type, int reservedFlag);
[DllImport("Advapi32.dll", EntryPoint = "CredReadW", CharSet = CharSet.Unicode, SetLastError = true)]
internal static extern bool CredRead(string target, CredentialType type, int reservedFlag, out IntPtr CredentialPtr);
[DllImport("Advapi32.dll", EntryPoint = "CredWriteW", CharSet = CharSet.Unicode, SetLastError = true)]
internal static extern bool CredWrite([In] ref Credential userCredential, [In] UInt32 flags);
[DllImport("Advapi32.dll", EntryPoint = "CredFree", SetLastError = true)]
internal static extern bool CredFree([In] IntPtr cred);
internal enum CredentialPersist : UInt32
{
Session = 0x01,
LocalMachine = 0x02
}
internal enum CredentialType : uint
{
Generic = 0x01,
DomainPassword = 0x02,
DomainCertificate = 0x03
}
[StructLayout(LayoutKind.Sequential, CharSet = CharSet.Unicode)]
internal struct Credential
{
public UInt32 Flags;
public CredentialType Type;
public IntPtr TargetName;
public IntPtr Comment;
public System.Runtime.InteropServices.ComTypes.FILETIME LastWritten;
public UInt32 CredentialBlobSize;
public IntPtr CredentialBlob;
public UInt32 Persist;
public UInt32 AttributeCount;
public IntPtr Attributes;
public IntPtr TargetAlias;
public IntPtr UserName;
}
}
#elif OS_OSX
public sealed class MacOSRunnerCredentialStore : RunnerService, IRunnerCredentialStore
{
private const string _osxRunnerCredStoreKeyChainName = "_GITHUB_ACTIONS_RUNNER_CREDSTORE_INTERNAL_";
// Keychain requires a password, but this is not intended to add security
private const string _osxRunnerCredStoreKeyChainPassword = "C46F23C36AF94B72B1EAEE32C68670A0";
private string _securityUtil;
private string _runnerCredStoreKeyChain;
public override void Initialize(IHostContext hostContext)
{
base.Initialize(hostContext);
_securityUtil = WhichUtil.Which("security", true, Trace);
_runnerCredStoreKeyChain = hostContext.GetConfigFile(WellKnownConfigFile.CredentialStore);
// Create osx key chain if it doesn't exists.
if (!File.Exists(_runnerCredStoreKeyChain))
{
List<string> securityOut = new List<string>();
List<string> securityError = new List<string>();
object outputLock = new object();
using (var p = HostContext.CreateService<IProcessInvoker>())
{
p.OutputDataReceived += delegate (object sender, ProcessDataReceivedEventArgs stdout)
{
if (!string.IsNullOrEmpty(stdout.Data))
{
lock (outputLock)
{
securityOut.Add(stdout.Data);
}
}
};
p.ErrorDataReceived += delegate (object sender, ProcessDataReceivedEventArgs stderr)
{
if (!string.IsNullOrEmpty(stderr.Data))
{
lock (outputLock)
{
securityError.Add(stderr.Data);
}
}
};
// make sure the 'security' has access to the key so we won't get prompt at runtime.
int exitCode = p.ExecuteAsync(workingDirectory: HostContext.GetDirectory(WellKnownDirectory.Root),
fileName: _securityUtil,
arguments: $"create-keychain -p {_osxRunnerCredStoreKeyChainPassword} \"{_runnerCredStoreKeyChain}\"",
environment: null,
cancellationToken: CancellationToken.None).GetAwaiter().GetResult();
if (exitCode == 0)
{
Trace.Info($"Successfully create-keychain for {_runnerCredStoreKeyChain}");
}
else
{
if (securityOut.Count > 0)
{
Trace.Error(string.Join(Environment.NewLine, securityOut));
}
if (securityError.Count > 0)
{
Trace.Error(string.Join(Environment.NewLine, securityError));
}
throw new InvalidOperationException($"'security create-keychain' failed with exit code {exitCode}.");
}
}
}
else
{
// Try unlock and lock the keychain, make sure it's still in good stage
UnlockKeyChain();
LockKeyChain();
}
}
public NetworkCredential Write(string target, string username, string password)
{
Trace.Entering();
ArgUtil.NotNullOrEmpty(target, nameof(target));
ArgUtil.NotNullOrEmpty(username, nameof(username));
ArgUtil.NotNullOrEmpty(password, nameof(password));
try
{
UnlockKeyChain();
// base64encode username + ':' + base64encode password
// OSX keychain requires you provide -s target and -a username to retrieve password
// So, we will trade both username and password as 'secret' store into keychain
string usernameBase64 = Convert.ToBase64String(Encoding.UTF8.GetBytes(username));
string passwordBase64 = Convert.ToBase64String(Encoding.UTF8.GetBytes(password));
string secretForKeyChain = $"{usernameBase64}:{passwordBase64}";
List<string> securityOut = new List<string>();
List<string> securityError = new List<string>();
object outputLock = new object();
using (var p = HostContext.CreateService<IProcessInvoker>())
{
p.OutputDataReceived += delegate (object sender, ProcessDataReceivedEventArgs stdout)
{
if (!string.IsNullOrEmpty(stdout.Data))
{
lock (outputLock)
{
securityOut.Add(stdout.Data);
}
}
};
p.ErrorDataReceived += delegate (object sender, ProcessDataReceivedEventArgs stderr)
{
if (!string.IsNullOrEmpty(stderr.Data))
{
lock (outputLock)
{
securityError.Add(stderr.Data);
}
}
};
// make sure the 'security' has access to the key so we won't get prompt at runtime.
int exitCode = p.ExecuteAsync(workingDirectory: HostContext.GetDirectory(WellKnownDirectory.Root),
fileName: _securityUtil,
arguments: $"add-generic-password -s {target} -a GITHUBACTIONSRUNNER -w {secretForKeyChain} -T \"{_securityUtil}\" \"{_runnerCredStoreKeyChain}\"",
environment: null,
cancellationToken: CancellationToken.None).GetAwaiter().GetResult();
if (exitCode == 0)
{
Trace.Info($"Successfully add-generic-password for {target} (GITHUBACTIONSRUNNER)");
}
else
{
if (securityOut.Count > 0)
{
Trace.Error(string.Join(Environment.NewLine, securityOut));
}
if (securityError.Count > 0)
{
Trace.Error(string.Join(Environment.NewLine, securityError));
}
throw new InvalidOperationException($"'security add-generic-password' failed with exit code {exitCode}.");
}
}
return new NetworkCredential(username, password);
}
finally
{
LockKeyChain();
}
}
public NetworkCredential Read(string target)
{
Trace.Entering();
ArgUtil.NotNullOrEmpty(target, nameof(target));
try
{
UnlockKeyChain();
string username;
string password;
List<string> securityOut = new List<string>();
List<string> securityError = new List<string>();
object outputLock = new object();
using (var p = HostContext.CreateService<IProcessInvoker>())
{
p.OutputDataReceived += delegate (object sender, ProcessDataReceivedEventArgs stdout)
{
if (!string.IsNullOrEmpty(stdout.Data))
{
lock (outputLock)
{
securityOut.Add(stdout.Data);
}
}
};
p.ErrorDataReceived += delegate (object sender, ProcessDataReceivedEventArgs stderr)
{
if (!string.IsNullOrEmpty(stderr.Data))
{
lock (outputLock)
{
securityError.Add(stderr.Data);
}
}
};
int exitCode = p.ExecuteAsync(workingDirectory: HostContext.GetDirectory(WellKnownDirectory.Root),
fileName: _securityUtil,
arguments: $"find-generic-password -s {target} -a GITHUBACTIONSRUNNER -w -g \"{_runnerCredStoreKeyChain}\"",
environment: null,
cancellationToken: CancellationToken.None).GetAwaiter().GetResult();
if (exitCode == 0)
{
string keyChainSecret = securityOut.First();
string[] secrets = keyChainSecret.Split(':');
if (secrets.Length == 2 && !string.IsNullOrEmpty(secrets[0]) && !string.IsNullOrEmpty(secrets[1]))
{
Trace.Info($"Successfully find-generic-password for {target} (GITHUBACTIONSRUNNER)");
username = Encoding.UTF8.GetString(Convert.FromBase64String(secrets[0]));
password = Encoding.UTF8.GetString(Convert.FromBase64String(secrets[1]));
return new NetworkCredential(username, password);
}
else
{
throw new ArgumentOutOfRangeException(nameof(keyChainSecret));
}
}
else
{
if (securityOut.Count > 0)
{
Trace.Error(string.Join(Environment.NewLine, securityOut));
}
if (securityError.Count > 0)
{
Trace.Error(string.Join(Environment.NewLine, securityError));
}
throw new InvalidOperationException($"'security find-generic-password' failed with exit code {exitCode}.");
}
}
}
finally
{
LockKeyChain();
}
}
public void Delete(string target)
{
Trace.Entering();
ArgUtil.NotNullOrEmpty(target, nameof(target));
try
{
UnlockKeyChain();
List<string> securityOut = new List<string>();
List<string> securityError = new List<string>();
object outputLock = new object();
using (var p = HostContext.CreateService<IProcessInvoker>())
{
p.OutputDataReceived += delegate (object sender, ProcessDataReceivedEventArgs stdout)
{
if (!string.IsNullOrEmpty(stdout.Data))
{
lock (outputLock)
{
securityOut.Add(stdout.Data);
}
}
};
p.ErrorDataReceived += delegate (object sender, ProcessDataReceivedEventArgs stderr)
{
if (!string.IsNullOrEmpty(stderr.Data))
{
lock (outputLock)
{
securityError.Add(stderr.Data);
}
}
};
int exitCode = p.ExecuteAsync(workingDirectory: HostContext.GetDirectory(WellKnownDirectory.Root),
fileName: _securityUtil,
arguments: $"delete-generic-password -s {target} -a GITHUBACTIONSRUNNER \"{_runnerCredStoreKeyChain}\"",
environment: null,
cancellationToken: CancellationToken.None).GetAwaiter().GetResult();
if (exitCode == 0)
{
Trace.Info($"Successfully delete-generic-password for {target} (GITHUBACTIONSRUNNER)");
}
else
{
if (securityOut.Count > 0)
{
Trace.Error(string.Join(Environment.NewLine, securityOut));
}
if (securityError.Count > 0)
{
Trace.Error(string.Join(Environment.NewLine, securityError));
}
throw new InvalidOperationException($"'security delete-generic-password' failed with exit code {exitCode}.");
}
}
}
finally
{
LockKeyChain();
}
}
private void UnlockKeyChain()
{
Trace.Entering();
ArgUtil.NotNullOrEmpty(_securityUtil, nameof(_securityUtil));
ArgUtil.NotNullOrEmpty(_runnerCredStoreKeyChain, nameof(_runnerCredStoreKeyChain));
List<string> securityOut = new List<string>();
List<string> securityError = new List<string>();
object outputLock = new object();
using (var p = HostContext.CreateService<IProcessInvoker>())
{
p.OutputDataReceived += delegate (object sender, ProcessDataReceivedEventArgs stdout)
{
if (!string.IsNullOrEmpty(stdout.Data))
{
lock (outputLock)
{
securityOut.Add(stdout.Data);
}
}
};
p.ErrorDataReceived += delegate (object sender, ProcessDataReceivedEventArgs stderr)
{
if (!string.IsNullOrEmpty(stderr.Data))
{
lock (outputLock)
{
securityError.Add(stderr.Data);
}
}
};
// make sure the 'security' has access to the key so we won't get prompt at runtime.
int exitCode = p.ExecuteAsync(workingDirectory: HostContext.GetDirectory(WellKnownDirectory.Root),
fileName: _securityUtil,
arguments: $"unlock-keychain -p {_osxRunnerCredStoreKeyChainPassword} \"{_runnerCredStoreKeyChain}\"",
environment: null,
cancellationToken: CancellationToken.None).GetAwaiter().GetResult();
if (exitCode == 0)
{
Trace.Info($"Successfully unlock-keychain for {_runnerCredStoreKeyChain}");
}
else
{
if (securityOut.Count > 0)
{
Trace.Error(string.Join(Environment.NewLine, securityOut));
}
if (securityError.Count > 0)
{
Trace.Error(string.Join(Environment.NewLine, securityError));
}
throw new InvalidOperationException($"'security unlock-keychain' failed with exit code {exitCode}.");
}
}
}
private void LockKeyChain()
{
Trace.Entering();
ArgUtil.NotNullOrEmpty(_securityUtil, nameof(_securityUtil));
ArgUtil.NotNullOrEmpty(_runnerCredStoreKeyChain, nameof(_runnerCredStoreKeyChain));
List<string> securityOut = new List<string>();
List<string> securityError = new List<string>();
object outputLock = new object();
using (var p = HostContext.CreateService<IProcessInvoker>())
{
p.OutputDataReceived += delegate (object sender, ProcessDataReceivedEventArgs stdout)
{
if (!string.IsNullOrEmpty(stdout.Data))
{
lock (outputLock)
{
securityOut.Add(stdout.Data);
}
}
};
p.ErrorDataReceived += delegate (object sender, ProcessDataReceivedEventArgs stderr)
{
if (!string.IsNullOrEmpty(stderr.Data))
{
lock (outputLock)
{
securityError.Add(stderr.Data);
}
}
};
// make sure the 'security' has access to the key so we won't get prompt at runtime.
int exitCode = p.ExecuteAsync(workingDirectory: HostContext.GetDirectory(WellKnownDirectory.Root),
fileName: _securityUtil,
arguments: $"lock-keychain \"{_runnerCredStoreKeyChain}\"",
environment: null,
cancellationToken: CancellationToken.None).GetAwaiter().GetResult();
if (exitCode == 0)
{
Trace.Info($"Successfully lock-keychain for {_runnerCredStoreKeyChain}");
}
else
{
if (securityOut.Count > 0)
{
Trace.Error(string.Join(Environment.NewLine, securityOut));
}
if (securityError.Count > 0)
{
Trace.Error(string.Join(Environment.NewLine, securityError));
}
throw new InvalidOperationException($"'security lock-keychain' failed with exit code {exitCode}.");
}
}
}
}
#else
public sealed class LinuxRunnerCredentialStore : RunnerService, IRunnerCredentialStore
{
// 'ghrunner' 128 bits iv
private readonly byte[] iv = new byte[] { 0x67, 0x68, 0x72, 0x75, 0x6e, 0x6e, 0x65, 0x72, 0x67, 0x68, 0x72, 0x75, 0x6e, 0x6e, 0x65, 0x72 };
// 256 bits key
private byte[] _symmetricKey;
private string _credStoreFile;
private Dictionary<string, Credential> _credStore;
public override void Initialize(IHostContext hostContext)
{
base.Initialize(hostContext);
_credStoreFile = hostContext.GetConfigFile(WellKnownConfigFile.CredentialStore);
if (File.Exists(_credStoreFile))
{
_credStore = IOUtil.LoadObject<Dictionary<string, Credential>>(_credStoreFile);
}
else
{
_credStore = new Dictionary<string, Credential>(StringComparer.OrdinalIgnoreCase);
}
string machineId;
if (File.Exists("/etc/machine-id"))
{
// try use machine-id as encryption key
// this helps avoid accidental information disclosure, but isn't intended for true security
machineId = File.ReadAllLines("/etc/machine-id").FirstOrDefault();
Trace.Info($"machine-id length {machineId?.Length ?? 0}.");
// machine-id doesn't exist or machine-id is not 256 bits
if (string.IsNullOrEmpty(machineId) || machineId.Length != 32)
{
Trace.Warning("Can not get valid machine id from '/etc/machine-id'.");
machineId = "43e7fe5da07740cf914b90f1dac51c2a";
}
}
else
{
// /etc/machine-id not exist
Trace.Warning("/etc/machine-id doesn't exist.");
machineId = "43e7fe5da07740cf914b90f1dac51c2a";
}
List<byte> keyBuilder = new List<byte>();
foreach (var c in machineId)
{
keyBuilder.Add(Convert.ToByte(c));
}
_symmetricKey = keyBuilder.ToArray();
}
public NetworkCredential Write(string target, string username, string password)
{
Trace.Entering();
ArgUtil.NotNullOrEmpty(target, nameof(target));
ArgUtil.NotNullOrEmpty(username, nameof(username));
ArgUtil.NotNullOrEmpty(password, nameof(password));
Trace.Info($"Store credential for '{target}' to cred store.");
Credential cred = new Credential(username, Encrypt(password));
_credStore[target] = cred;
SyncCredentialStoreFile();
return new NetworkCredential(username, password);
}
public NetworkCredential Read(string target)
{
Trace.Entering();
ArgUtil.NotNullOrEmpty(target, nameof(target));
Trace.Info($"Read credential for '{target}' from cred store.");
if (_credStore.ContainsKey(target))
{
Credential cred = _credStore[target];
if (!string.IsNullOrEmpty(cred.UserName) && !string.IsNullOrEmpty(cred.Password))
{
Trace.Info($"Return credential for '{target}' from cred store.");
return new NetworkCredential(cred.UserName, Decrypt(cred.Password));
}
}
throw new KeyNotFoundException(target);
}
public void Delete(string target)
{
Trace.Entering();
ArgUtil.NotNullOrEmpty(target, nameof(target));
if (_credStore.ContainsKey(target))
{
Trace.Info($"Delete credential for '{target}' from cred store.");
_credStore.Remove(target);
SyncCredentialStoreFile();
}
else
{
throw new KeyNotFoundException(target);
}
}
private void SyncCredentialStoreFile()
{
Trace.Entering();
Trace.Info("Sync in-memory credential store with credential store file.");
// delete cred store file when all creds gone
if (_credStore.Count == 0)
{
IOUtil.DeleteFile(_credStoreFile);
return;
}
if (!File.Exists(_credStoreFile))
{
CreateCredentialStoreFile();
}
IOUtil.SaveObject(_credStore, _credStoreFile);
}
private string Encrypt(string secret)
{
using (Aes aes = Aes.Create())
{
aes.Key = _symmetricKey;
aes.IV = iv;
// Create a decrytor to perform the stream transform.
ICryptoTransform encryptor = aes.CreateEncryptor();
// Create the streams used for encryption.
using (MemoryStream msEncrypt = new MemoryStream())
{
using (CryptoStream csEncrypt = new CryptoStream(msEncrypt, encryptor, CryptoStreamMode.Write))
{
using (StreamWriter swEncrypt = new StreamWriter(csEncrypt))
{
swEncrypt.Write(secret);
}
return Convert.ToBase64String(msEncrypt.ToArray());
}
}
}
}
private string Decrypt(string encryptedText)
{
using (Aes aes = Aes.Create())
{
aes.Key = _symmetricKey;
aes.IV = iv;
// Create a decrytor to perform the stream transform.
ICryptoTransform decryptor = aes.CreateDecryptor();
// Create the streams used for decryption.
using (MemoryStream msDecrypt = new MemoryStream(Convert.FromBase64String(encryptedText)))
{
using (CryptoStream csDecrypt = new CryptoStream(msDecrypt, decryptor, CryptoStreamMode.Read))
{
using (StreamReader srDecrypt = new StreamReader(csDecrypt))
{
// Read the decrypted bytes from the decrypting stream and place them in a string.
return srDecrypt.ReadToEnd();
}
}
}
}
}
private void CreateCredentialStoreFile()
{
File.WriteAllText(_credStoreFile, "");
File.SetAttributes(_credStoreFile, File.GetAttributes(_credStoreFile) | FileAttributes.Hidden);
// Try to lock down the .credentials_store file to the owner/group
var chmodPath = WhichUtil.Which("chmod", trace: Trace);
if (!String.IsNullOrEmpty(chmodPath))
{
var arguments = $"600 {new FileInfo(_credStoreFile).FullName}";
using (var invoker = HostContext.CreateService<IProcessInvoker>())
{
var exitCode = invoker.ExecuteAsync(HostContext.GetDirectory(WellKnownDirectory.Root), chmodPath, arguments, null, default(CancellationToken)).GetAwaiter().GetResult();
if (exitCode == 0)
{
Trace.Info("Successfully set permissions for credentials store file {0}", _credStoreFile);
}
else
{
Trace.Warning("Unable to successfully set permissions for credentials store file {0}. Received exit code {1} from {2}", _credStoreFile, exitCode, chmodPath);
}
}
}
else
{
Trace.Warning("Unable to locate chmod to set permissions for credentials store file {0}.", _credStoreFile);
}
}
}
[DataContract]
internal class Credential
{
public Credential()
{ }
public Credential(string userName, string password)
{
UserName = userName;
Password = password;
}
[DataMember(IsRequired = true)]
public string UserName { get; set; }
[DataMember(IsRequired = true)]
public string Password { get; set; }
}
#endif
}

View File

@@ -28,14 +28,10 @@ namespace GitHub.Runner.Listener
private readonly string[] validFlags =
{
Constants.Runner.CommandLine.Flags.Commit,
#if OS_WINDOWS
Constants.Runner.CommandLine.Flags.GitUseSChannel,
#endif
Constants.Runner.CommandLine.Flags.Help,
Constants.Runner.CommandLine.Flags.Replace,
Constants.Runner.CommandLine.Flags.RunAsService,
Constants.Runner.CommandLine.Flags.Once,
Constants.Runner.CommandLine.Flags.SslSkipCertValidation,
Constants.Runner.CommandLine.Flags.Unattended,
Constants.Runner.CommandLine.Flags.Version
};
@@ -45,13 +41,7 @@ namespace GitHub.Runner.Listener
Constants.Runner.CommandLine.Args.Auth,
Constants.Runner.CommandLine.Args.MonitorSocketAddress,
Constants.Runner.CommandLine.Args.Name,
Constants.Runner.CommandLine.Args.Password,
Constants.Runner.CommandLine.Args.Pool,
Constants.Runner.CommandLine.Args.SslCACert,
Constants.Runner.CommandLine.Args.SslClientCert,
Constants.Runner.CommandLine.Args.SslClientCertKey,
Constants.Runner.CommandLine.Args.SslClientCertArchive,
Constants.Runner.CommandLine.Args.SslClientCertPassword,
Constants.Runner.CommandLine.Args.StartupType,
Constants.Runner.CommandLine.Args.Token,
Constants.Runner.CommandLine.Args.Url,
@@ -73,9 +63,6 @@ namespace GitHub.Runner.Listener
public bool Unattended => TestFlag(Constants.Runner.CommandLine.Flags.Unattended);
public bool Version => TestFlag(Constants.Runner.CommandLine.Flags.Version);
#if OS_WINDOWS
public bool GitUseSChannel => TestFlag(Constants.Runner.CommandLine.Flags.GitUseSChannel);
#endif
public bool RunOnce => TestFlag(Constants.Runner.CommandLine.Flags.Once);
// Constructor.
@@ -160,13 +147,6 @@ namespace GitHub.Runner.Listener
defaultValue: false);
}
public bool GetAutoLaunchBrowser()
{
return TestFlagOrPrompt(
name: Constants.Runner.CommandLine.Flags.LaunchBrowser,
description: "Would you like to launch your browser for AAD Device Code Flow? (Y/N)",
defaultValue: true);
}
//
// Args.
//
@@ -179,24 +159,6 @@ namespace GitHub.Runner.Listener
validator: Validators.AuthSchemeValidator);
}
public string GetPassword()
{
return GetArgOrPrompt(
name: Constants.Runner.CommandLine.Args.Password,
description: "What is your GitHub password?",
defaultValue: string.Empty,
validator: Validators.NonEmptyValidator);
}
public string GetPool()
{
return GetArgOrPrompt(
name: Constants.Runner.CommandLine.Args.Pool,
description: "Enter the name of your runner pool:",
defaultValue: "default",
validator: Validators.NonEmptyValidator);
}
public string GetRunnerName()
{
return GetArgOrPrompt(
@@ -210,7 +172,7 @@ namespace GitHub.Runner.Listener
{
return GetArgOrPrompt(
name: Constants.Runner.CommandLine.Args.Token,
description: "Enter your personal access token:",
description: "What is your pool admin oauth access token?",
defaultValue: string.Empty,
validator: Validators.NonEmptyValidator);
}
@@ -219,7 +181,16 @@ namespace GitHub.Runner.Listener
{
return GetArgOrPrompt(
name: Constants.Runner.CommandLine.Args.Token,
description: "Enter runner register token:",
description: "What is your runner register token?",
defaultValue: string.Empty,
validator: Validators.NonEmptyValidator);
}
public string GetRunnerDeletionToken()
{
return GetArgOrPrompt(
name: Constants.Runner.CommandLine.Args.Token,
description: "Enter runner remove token:",
defaultValue: string.Empty,
validator: Validators.NonEmptyValidator);
}
@@ -240,15 +211,6 @@ namespace GitHub.Runner.Listener
validator: Validators.ServerUrlValidator);
}
public string GetUserName()
{
return GetArgOrPrompt(
name: Constants.Runner.CommandLine.Args.UserName,
description: "What is your GitHub username?",
defaultValue: string.Empty,
validator: Validators.NonEmptyValidator);
}
public string GetWindowsLogonAccount(string defaultValue, string descriptionMsg)
{
return GetArgOrPrompt(
@@ -287,36 +249,6 @@ namespace GitHub.Runner.Listener
return GetArg(Constants.Runner.CommandLine.Args.StartupType);
}
public bool GetSkipCertificateValidation()
{
return TestFlag(Constants.Runner.CommandLine.Flags.SslSkipCertValidation);
}
public string GetCACertificate()
{
return GetArg(Constants.Runner.CommandLine.Args.SslCACert);
}
public string GetClientCertificate()
{
return GetArg(Constants.Runner.CommandLine.Args.SslClientCert);
}
public string GetClientCertificatePrivateKey()
{
return GetArg(Constants.Runner.CommandLine.Args.SslClientCertKey);
}
public string GetClientCertificateArchrive()
{
return GetArg(Constants.Runner.CommandLine.Args.SslClientCertArchive);
}
public string GetClientCertificatePassword()
{
return GetArg(Constants.Runner.CommandLine.Args.SslClientCertPassword);
}
//
// Private helpers.
//
@@ -359,7 +291,7 @@ namespace GitHub.Runner.Listener
if (!string.IsNullOrEmpty(result))
{
// After read the arg from input commandline args, remove it from Arg dictionary,
// This will help if bad arg value passed through CommandLine arg, when ConfigurationManager ask CommandSetting the second time,
// This will help if bad arg value passed through CommandLine arg, when ConfigurationManager ask CommandSetting the second time,
// It will prompt for input instead of continue use the bad input.
_trace.Info($"Remove {name} from Arg dictionary.");
RemoveArg(name);

View File

@@ -85,54 +85,6 @@ namespace GitHub.Runner.Listener.Configuration
throw new InvalidOperationException("Cannot configure the runner because it is already configured. To reconfigure the runner, run 'config.cmd remove' or './config.sh remove' first.");
}
// Populate cert setting from commandline args
var runnerCertManager = HostContext.GetService<IRunnerCertificateManager>();
bool saveCertSetting = false;
bool skipCertValidation = command.GetSkipCertificateValidation();
string caCert = command.GetCACertificate();
string clientCert = command.GetClientCertificate();
string clientCertKey = command.GetClientCertificatePrivateKey();
string clientCertArchive = command.GetClientCertificateArchrive();
string clientCertPassword = command.GetClientCertificatePassword();
// We require all Certificate files are under agent root.
// So we can set ACL correctly when configure as service
if (!string.IsNullOrEmpty(caCert))
{
caCert = Path.Combine(HostContext.GetDirectory(WellKnownDirectory.Root), caCert);
ArgUtil.File(caCert, nameof(caCert));
}
if (!string.IsNullOrEmpty(clientCert) &&
!string.IsNullOrEmpty(clientCertKey) &&
!string.IsNullOrEmpty(clientCertArchive))
{
// Ensure all client cert pieces are there.
clientCert = Path.Combine(HostContext.GetDirectory(WellKnownDirectory.Root), clientCert);
clientCertKey = Path.Combine(HostContext.GetDirectory(WellKnownDirectory.Root), clientCertKey);
clientCertArchive = Path.Combine(HostContext.GetDirectory(WellKnownDirectory.Root), clientCertArchive);
ArgUtil.File(clientCert, nameof(clientCert));
ArgUtil.File(clientCertKey, nameof(clientCertKey));
ArgUtil.File(clientCertArchive, nameof(clientCertArchive));
}
else if (!string.IsNullOrEmpty(clientCert) ||
!string.IsNullOrEmpty(clientCertKey) ||
!string.IsNullOrEmpty(clientCertArchive))
{
// Print out which args are missing.
ArgUtil.NotNullOrEmpty(Constants.Runner.CommandLine.Args.SslClientCert, Constants.Runner.CommandLine.Args.SslClientCert);
ArgUtil.NotNullOrEmpty(Constants.Runner.CommandLine.Args.SslClientCertKey, Constants.Runner.CommandLine.Args.SslClientCertKey);
ArgUtil.NotNullOrEmpty(Constants.Runner.CommandLine.Args.SslClientCertArchive, Constants.Runner.CommandLine.Args.SslClientCertArchive);
}
if (skipCertValidation || !string.IsNullOrEmpty(caCert) || !string.IsNullOrEmpty(clientCert))
{
Trace.Info("Reset runner cert setting base on commandline args.");
(runnerCertManager as RunnerCertificateManager).SetupCertificate(skipCertValidation, caCert, clientCert, clientCertKey, clientCertArchive, clientCertPassword);
saveCertSetting = true;
}
RunnerSettings runnerSettings = new RunnerSettings();
bool isHostedServer = false;
@@ -352,31 +304,10 @@ namespace GitHub.Runner.Listener.Configuration
_store.SaveSettings(runnerSettings);
if (saveCertSetting)
{
Trace.Info("Save agent cert setting to disk.");
(runnerCertManager as RunnerCertificateManager).SaveCertificateSetting();
}
_term.WriteLine();
_term.WriteSuccessMessage("Settings Saved.");
_term.WriteLine();
bool saveRuntimeOptions = false;
var runtimeOptions = new RunnerRuntimeOptions();
#if OS_WINDOWS
if (command.GitUseSChannel)
{
saveRuntimeOptions = true;
runtimeOptions.GitUseSecureChannel = true;
}
#endif
if (saveRuntimeOptions)
{
Trace.Info("Save agent runtime options to disk.");
_store.SaveRunnerRuntimeOptions(runtimeOptions);
}
#if OS_WINDOWS
// config windows service
bool runAsService = command.GetRunAsService();
@@ -441,7 +372,7 @@ namespace GitHub.Runner.Listener.Configuration
}
else
{
var githubToken = command.GetToken();
var githubToken = command.GetRunnerDeletionToken();
GitHubAuthResult authResult = await GetTenantCredential(settings.GitHubUrl, githubToken);
creds = authResult.ToVssCredentials();
Trace.Info("cred retrieved via GitHub auth");
@@ -489,13 +420,6 @@ namespace GitHub.Runner.Listener.Configuration
currentAction = "Removing .runner";
if (isConfigured)
{
// delete agent cert setting
(HostContext.GetService<IRunnerCertificateManager>() as RunnerCertificateManager).DeleteCertificateSetting();
// delete agent runtime option
_store.DeleteRunnerRuntimeOptions();
_store.DeleteSettings();
_term.WriteSuccessMessage("Removed .runner");
}

View File

@@ -78,7 +78,7 @@ namespace GitHub.Runner.Listener.Configuration
if (string.Equals(TokenSchema, "OAuthAccessToken", StringComparison.OrdinalIgnoreCase))
{
return new VssCredentials(null, new VssOAuthAccessTokenCredential(Token), CredentialPromptType.DoNotPrompt);
return new VssCredentials(new VssOAuthAccessTokenCredential(Token), CredentialPromptType.DoNotPrompt);
}
else
{

View File

@@ -48,7 +48,7 @@ namespace GitHub.Runner.Listener.Configuration
ArgUtil.NotNullOrEmpty(token, nameof(token));
trace.Info("token retrieved: {0} chars", token.Length);
VssCredentials creds = new VssCredentials(null, new VssOAuthAccessTokenCredential(token), CredentialPromptType.DoNotPrompt);
VssCredentials creds = new VssCredentials(new VssOAuthAccessTokenCredential(token), CredentialPromptType.DoNotPrompt);
trace.Info("cred created");
return creds;

View File

@@ -6,7 +6,7 @@ using GitHub.Runner.Common;
namespace GitHub.Runner.Listener.Configuration
{
/// <summary>
/// Manages an RSA key for the agent using the most appropriate store for the target platform.
/// Manages an RSA key for the runner using the most appropriate store for the target platform.
/// </summary>
#if OS_WINDOWS
[ServiceLocator(Default = typeof(RSAEncryptedFileKeyManager))]
@@ -16,10 +16,10 @@ namespace GitHub.Runner.Listener.Configuration
public interface IRSAKeyManager : IRunnerService
{
/// <summary>
/// Creates a new <c>RSACryptoServiceProvider</c> instance for the current agent. If a key file is found then the current
/// Creates a new <c>RSACryptoServiceProvider</c> instance for the current runner. If a key file is found then the current
/// key is returned to the caller.
/// </summary>
/// <returns>An <c>RSACryptoServiceProvider</c> instance representing the key for the agent</returns>
/// <returns>An <c>RSACryptoServiceProvider</c> instance representing the key for the runner</returns>
RSACryptoServiceProvider CreateKey();
/// <summary>
@@ -30,7 +30,7 @@ namespace GitHub.Runner.Listener.Configuration
/// <summary>
/// Gets the <c>RSACryptoServiceProvider</c> instance currently stored by the key manager.
/// </summary>
/// <returns>An <c>RSACryptoServiceProvider</c> instance representing the key for the agent</returns>
/// <returns>An <c>RSACryptoServiceProvider</c> instance representing the key for the runner</returns>
/// <exception cref="CryptographicException">No key exists in the store</exception>
RSACryptoServiceProvider GetKey();
}

View File

@@ -447,7 +447,7 @@ namespace GitHub.Runner.Listener.Configuration
{
Trace.Entering();
string agentServiceExecutable = "\"" + Path.Combine(HostContext.GetDirectory(WellKnownDirectory.Bin), WindowsServiceControlManager.WindowsServiceControllerName) + "\"";
string runnerServiceExecutable = "\"" + Path.Combine(HostContext.GetDirectory(WellKnownDirectory.Bin), WindowsServiceControlManager.WindowsServiceControllerName) + "\"";
IntPtr scmHndl = IntPtr.Zero;
IntPtr svcHndl = IntPtr.Zero;
IntPtr tmpBuf = IntPtr.Zero;
@@ -468,7 +468,7 @@ namespace GitHub.Runner.Listener.Configuration
};
processInvoker.ExecuteAsync(workingDirectory: string.Empty,
fileName: agentServiceExecutable,
fileName: runnerServiceExecutable,
arguments: "init",
environment: null,
requireExitCodeZero: true,
@@ -490,7 +490,7 @@ namespace GitHub.Runner.Listener.Configuration
SERVICE_WIN32_OWN_PROCESS,
ServiceBootFlag.AutoStart,
ServiceError.Normal,
agentServiceExecutable,
runnerServiceExecutable,
null,
IntPtr.Zero,
null,
@@ -678,6 +678,17 @@ namespace GitHub.Runner.Listener.Configuration
if (service != null)
{
service.Start();
try
{
_term.WriteLine("Waiting for service to start...");
service.WaitForStatus(ServiceControllerStatus.Running, TimeSpan.FromSeconds(60));
}
catch (System.ServiceProcess.TimeoutException)
{
throw new InvalidOperationException($"Cannot start the service {serviceName} in a timely fashion.");
}
_term.WriteLine($"Service {serviceName} started successfully");
}
else

View File

@@ -43,7 +43,7 @@ namespace GitHub.Runner.Listener.Configuration
// Construct a credentials cache with a single OAuth credential for communication. The windows credential
// is explicitly set to null to ensure we never do that negotiation.
return new VssCredentials(null, agentCredential, CredentialPromptType.DoNotPrompt);
return new VssCredentials(agentCredential, CredentialPromptType.DoNotPrompt);
}
}
}

View File

@@ -38,25 +38,6 @@ namespace GitHub.Runner.Listener.Configuration
return CredentialManager.CredentialTypes.ContainsKey(value);
}
public static bool FilePathValidator(string value)
{
var directoryInfo = new DirectoryInfo(value);
if (!directoryInfo.Exists)
{
try
{
Directory.CreateDirectory(value);
}
catch (Exception)
{
return false;
}
}
return true;
}
public static bool BoolValidator(string value)
{
return string.Equals(value, "true", StringComparison.OrdinalIgnoreCase) ||

View File

@@ -568,6 +568,10 @@ namespace GitHub.Runner.Listener
{
Trace.Info("worker process has been killed.");
}
// When worker doesn't exit within cancel timeout, the runner will kill the worker process and worker won't finish upload job logs.
// The runner will try to upload these logs at this time.
await TryUploadUnfinishedLogs(message);
}
Trace.Info($"finish job request for job {message.JobId} with result: {resultOnAbandonOrCancel}");
@@ -712,6 +716,121 @@ namespace GitHub.Runner.Listener
}
}
// Best effort upload any logs for this job.
private async Task TryUploadUnfinishedLogs(Pipelines.AgentJobRequestMessage message)
{
Trace.Entering();
var logFolder = Path.Combine(HostContext.GetDirectory(WellKnownDirectory.Diag), PagingLogger.PagingFolder);
if (!Directory.Exists(logFolder))
{
return;
}
var logs = Directory.GetFiles(logFolder);
if (logs.Length == 0)
{
return;
}
try
{
var systemConnection = message.Resources.Endpoints.SingleOrDefault(x => string.Equals(x.Name, WellKnownServiceEndpointNames.SystemVssConnection));
ArgUtil.NotNull(systemConnection, nameof(systemConnection));
var jobServer = HostContext.GetService<IJobServer>();
VssCredentials jobServerCredential = VssUtil.GetVssCredential(systemConnection);
VssConnection jobConnection = VssUtil.CreateConnection(systemConnection.Url, jobServerCredential);
await jobServer.ConnectAsync(jobConnection);
var timeline = await jobServer.GetTimelineAsync(message.Plan.ScopeIdentifier, message.Plan.PlanType, message.Plan.PlanId, message.Timeline.Id, CancellationToken.None);
var updatedRecords = new List<TimelineRecord>();
var logPages = new Dictionary<Guid, Dictionary<int, string>>();
var logRecords = new Dictionary<Guid, TimelineRecord>();
foreach (var log in logs)
{
var logName = Path.GetFileNameWithoutExtension(log);
var logNameParts = logName.Split('_', StringSplitOptions.RemoveEmptyEntries);
if (logNameParts.Length != 3)
{
Trace.Warning($"log file '{log}' doesn't follow naming convension 'GUID_GUID_INT'.");
continue;
}
var logPageSeperator = logName.IndexOf('_');
var logRecordId = Guid.Empty;
var pageNumber = 0;
if (!Guid.TryParse(logNameParts[0], out Guid timelineId) || timelineId != timeline.Id)
{
Trace.Warning($"log file '{log}' is not belongs to current job");
continue;
}
if (!Guid.TryParse(logNameParts[1], out logRecordId))
{
Trace.Warning($"log file '{log}' doesn't follow naming convension 'GUID_GUID_INT'.");
continue;
}
if (!int.TryParse(logNameParts[2], out pageNumber))
{
Trace.Warning($"log file '{log}' doesn't follow naming convension 'GUID_GUID_INT'.");
continue;
}
var record = timeline.Records.FirstOrDefault(x => x.Id == logRecordId);
if (record != null)
{
if (!logPages.ContainsKey(record.Id))
{
logPages[record.Id] = new Dictionary<int, string>();
logRecords[record.Id] = record;
}
logPages[record.Id][pageNumber] = log;
}
}
foreach (var pages in logPages)
{
var record = logRecords[pages.Key];
if (record.Log == null)
{
// Create the log
record.Log = await jobServer.CreateLogAsync(message.Plan.ScopeIdentifier, message.Plan.PlanType, message.Plan.PlanId, new TaskLog(String.Format(@"logs\{0:D}", record.Id)), default(CancellationToken));
// Need to post timeline record updates to reflect the log creation
updatedRecords.Add(record.Clone());
}
for (var i = 1; i <= pages.Value.Count; i++)
{
var logFile = pages.Value[i];
// Upload the contents
using (FileStream fs = File.Open(logFile, FileMode.Open, FileAccess.Read, FileShare.ReadWrite))
{
var logUploaded = await jobServer.AppendLogContentAsync(message.Plan.ScopeIdentifier, message.Plan.PlanType, message.Plan.PlanId, record.Log.Id, fs, default(CancellationToken));
}
Trace.Info($"Uploaded unfinished log '{logFile}' for current job.");
IOUtil.DeleteFile(logFile);
}
}
if (updatedRecords.Count > 0)
{
await jobServer.UpdateTimelineRecordsAsync(message.Plan.ScopeIdentifier, message.Plan.PlanType, message.Plan.PlanId, message.Timeline.Id, updatedRecords, CancellationToken.None);
}
}
catch (Exception ex)
{
// Ignore any error during log upload since it's best effort
Trace.Error(ex);
}
}
// TODO: We need send detailInfo back to DT in order to add an issue for the job
private async Task CompleteJobRequestAsync(int poolId, Pipelines.AgentJobRequestMessage message, Guid lockToken, TaskResult result, string detailInfo = null)
{

View File

@@ -80,7 +80,7 @@ namespace GitHub.Runner.Listener
Trace.Info($"Attempt to create session.");
try
{
Trace.Info("Connecting to the Agent Server...");
Trace.Info("Connecting to the Runner Server...");
await _runnerServer.ConnectAsync(new Uri(serverUrl), creds);
Trace.Info("VssConnection created");
@@ -110,7 +110,7 @@ namespace GitHub.Runner.Listener
}
catch (TaskAgentAccessTokenExpiredException)
{
Trace.Info("Agent OAuth token has been revoked. Session creation failed.");
Trace.Info("Runner OAuth token has been revoked. Session creation failed.");
throw;
}
catch (Exception ex)
@@ -190,7 +190,7 @@ namespace GitHub.Runner.Listener
}
catch (TaskAgentAccessTokenExpiredException)
{
Trace.Info("Agent OAuth token has been revoked. Unable to pull message.");
Trace.Info("Runner OAuth token has been revoked. Unable to pull message.");
throw;
}
catch (Exception ex)
@@ -336,7 +336,7 @@ namespace GitHub.Runner.Listener
{
if (ex is TaskAgentNotFoundException)
{
Trace.Info("The agent no longer exists on the server. Stopping the runner.");
Trace.Info("The runner no longer exists on the server. Stopping the runner.");
_term.WriteError("The runner no longer exists on the server. Please reconfigure the runner.");
return false;
}
@@ -364,7 +364,7 @@ namespace GitHub.Runner.Listener
}
else if (ex is VssOAuthTokenRequestException && ex.Message.Contains("Current server time is"))
{
Trace.Info("Local clock might skewed.");
Trace.Info("Local clock might be skewed.");
_term.WriteError("The local machine's clock may be out of sync with the server time by more than five minutes. Please sync your clock with your domain or internet time and try again.");
if (_sessionCreationExceptionTracker.ContainsKey(nameof(VssOAuthTokenRequestException)))
{

View File

@@ -4,6 +4,7 @@ using GitHub.Runner.Sdk;
using System;
using System.Globalization;
using System.IO;
using System.Reflection;
using System.Runtime.InteropServices;
using System.Threading;
using System.Threading.Tasks;
@@ -14,6 +15,9 @@ namespace GitHub.Runner.Listener
{
public static int Main(string[] args)
{
// Add environment variables from .env file
LoadAndSetEnv();
using (HostContext context = new HostContext("Runner"))
{
return MainAsync(context, args).GetAwaiter().GetResult();
@@ -25,7 +29,7 @@ namespace GitHub.Runner.Listener
// 1: Terminate failure
// 2: Retriable failure
// 3: Exit for self update
public async static Task<int> MainAsync(IHostContext context, string[] args)
private async static Task<int> MainAsync(IHostContext context, string[] args)
{
Tracing trace = context.GetTrace(nameof(GitHub.Runner.Listener));
trace.Info($"Runner is built for {Constants.Runner.Platform} ({Constants.Runner.PlatformArchitecture}) - {BuildConstants.RunnerPackage.PackageName}.");
@@ -83,22 +87,6 @@ namespace GitHub.Runner.Listener
return Constants.Runner.ReturnCode.TerminatedError;
}
// Add environment variables from .env file
string envFile = Path.Combine(context.GetDirectory(WellKnownDirectory.Root), ".env");
if (File.Exists(envFile))
{
var envContents = File.ReadAllLines(envFile);
foreach (var env in envContents)
{
if (!string.IsNullOrEmpty(env) && env.IndexOf('=') > 0)
{
string envKey = env.Substring(0, env.IndexOf('='));
string envValue = env.Substring(env.IndexOf('=') + 1);
Environment.SetEnvironmentVariable(envKey, envValue);
}
}
}
// Parse the command line args.
var command = new CommandSettings(context, args);
trace.Info("Arguments parsed");
@@ -136,5 +124,34 @@ namespace GitHub.Runner.Listener
return Constants.Runner.ReturnCode.RetryableError;
}
}
private static void LoadAndSetEnv()
{
var binDir = Path.GetDirectoryName(Assembly.GetEntryAssembly().Location);
var rootDir = new DirectoryInfo(binDir).Parent.FullName;
string envFile = Path.Combine(rootDir, ".env");
if (File.Exists(envFile))
{
var envContents = File.ReadAllLines(envFile);
foreach (var env in envContents)
{
if (!string.IsNullOrEmpty(env))
{
var separatorIndex = env.IndexOf('=');
if (separatorIndex > 0)
{
string envKey = env.Substring(0, separatorIndex);
string envValue = null;
if (env.Length > separatorIndex + 1)
{
envValue = env.Substring(separatorIndex + 1);
}
Environment.SetEnvironmentVariable(envKey, envValue);
}
}
}
}
}
}
}

View File

@@ -1,7 +1,7 @@
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFramework>netcoreapp3.0</TargetFramework>
<TargetFramework>netcoreapp3.1</TargetFramework>
<OutputType>Exe</OutputType>
<RuntimeIdentifiers>win-x64;win-x86;linux-x64;linux-arm64;linux-arm;osx-x64</RuntimeIdentifiers>
<TargetLatestRuntimePatch>true</TargetLatestRuntimePatch>

View File

@@ -37,8 +37,7 @@ namespace GitHub.Runner.Listener
{
try
{
var runnerCertManager = HostContext.GetService<IRunnerCertificateManager>();
VssUtil.InitializeVssClientSettings(HostContext.UserAgent, HostContext.WebProxy, runnerCertManager.VssClientCertificateManager);
VssUtil.InitializeVssClientSettings(HostContext.UserAgent, HostContext.WebProxy);
_inConfigStage = true;
_completedCommand.Reset();
@@ -434,7 +433,7 @@ namespace GitHub.Runner.Listener
}
catch (TaskAgentAccessTokenExpiredException)
{
Trace.Info("Agent OAuth token has been revoked. Shutting down.");
Trace.Info("Runner OAuth token has been revoked. Shutting down.");
}
return Constants.Runner.ReturnCode.Success;
@@ -452,16 +451,38 @@ namespace GitHub.Runner.Listener
ext = "sh";
#endif
_term.WriteLine($@"
Commands:,
.{separator}config.{ext} Configures the runner
.{separator}config.{ext} remove Unconfigures the runner
.{separator}run.{ext} Runs the runner interactively. Does not require any options.
Commands:
.{separator}config.{ext} Configures the runner
.{separator}config.{ext} remove Unconfigures the runner
.{separator}run.{ext} Runs the runner interactively. Does not require any options.
Options:
--help Prints the help for each command
--version Prints the runner version
--commit Prints the runner commit
--help Prints the help for each command
");
Config Options:
--unattended Disable interactive prompts for missing arguments. Defaults will be used for missing options
--url string Repository to add the runner to. Required if unattended
--token string Registration token. Required if unattended
--name string Name of the runner to configure (default {Environment.MachineName ?? "myrunner"})
--work string Relative runner work directory (default {Constants.Path.WorkDirectory})
--replace Replace any existing runner with the same name (default false)");
#if OS_WINDOWS
_term.WriteLine($@" --runasservice Run the runner as a service");
_term.WriteLine($@" --windowslogonaccount string Account to run the service as. Requires runasservice");
_term.WriteLine($@" --windowslogonpassword string Password for the service account. Requires runasservice");
#endif
_term.WriteLine($@"
Examples:
Configure a runner non-interactively:
.{separator}config.{ext} --unattended --url <url> --token <token>
Configure a runner non-interactively, replacing any existing runner with the same name:
.{separator}config.{ext} --unattended --url <url> --token <token> --replace [--name <name>]");
#if OS_WINDOWS
_term.WriteLine($@" Configure a runner to run as a service:");
_term.WriteLine($@" .{separator}config.{ext} --url <url> --token <token> --runasservice");
#endif
}
}
}

View File

@@ -1,7 +1,7 @@
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFramework>netcoreapp3.0</TargetFramework>
<TargetFramework>netcoreapp3.1</TargetFramework>
<OutputType>Exe</OutputType>
<RuntimeIdentifiers>win-x64;win-x86;linux-x64;linux-arm64;linux-arm;osx-x64</RuntimeIdentifiers>
<TargetLatestRuntimePatch>true</TargetLatestRuntimePatch>

View File

@@ -1,58 +0,0 @@
using System;
using System.Collections.Generic;
using System.Threading;
using System.Threading.Tasks;
using GitHub.Runner.Sdk;
using GitHub.Services.WebApi;
using GitHub.Build.WebApi;
namespace GitHub.Runner.Plugins.Artifact
{
// A client wrapper interacting with Build's Artifact API
public class BuildServer
{
private readonly BuildHttpClient _buildHttpClient;
public BuildServer(VssConnection connection)
{
ArgUtil.NotNull(connection, nameof(connection));
_buildHttpClient = connection.GetClient<BuildHttpClient>();
}
// Associate the specified artifact with a build, along with custom data.
public async Task<BuildArtifact> AssociateArtifact(
Guid projectId,
int pipelineId,
string jobId,
string name,
string type,
string data,
Dictionary<string, string> propertiesDictionary,
CancellationToken cancellationToken = default(CancellationToken))
{
BuildArtifact artifact = new BuildArtifact()
{
Name = name,
Source = jobId,
Resource = new ArtifactResource()
{
Data = data,
Type = type,
Properties = propertiesDictionary
}
};
return await _buildHttpClient.CreateArtifactAsync(artifact, projectId, pipelineId, cancellationToken: cancellationToken);
}
// Get named artifact from a build
public async Task<BuildArtifact> GetArtifact(
Guid projectId,
int pipelineId,
string name,
CancellationToken cancellationToken)
{
return await _buildHttpClient.GetArtifactAsync(projectId, pipelineId, name, cancellationToken: cancellationToken);
}
}
}

View File

@@ -3,7 +3,6 @@ using System.Collections.Generic;
using System.IO;
using System.Threading;
using System.Threading.Tasks;
using GitHub.Build.WebApi;
using GitHub.Services.Common;
using GitHub.Runner.Sdk;
@@ -40,70 +39,31 @@ namespace GitHub.Runner.Plugins.Artifact
targetPath = Path.IsPathFullyQualified(targetPath) ? targetPath : Path.GetFullPath(Path.Combine(defaultWorkingDirectory, targetPath));
// Project ID
Guid projectId = new Guid(context.Variables.GetValueOrDefault(BuildVariables.TeamProjectId)?.Value ?? Guid.Empty.ToString());
// Build ID
string buildIdStr = context.Variables.GetValueOrDefault(BuildVariables.BuildId)?.Value ?? string.Empty;
string buildIdStr = context.Variables.GetValueOrDefault(SdkConstants.Variables.Build.BuildId)?.Value ?? string.Empty;
if (!int.TryParse(buildIdStr, out int buildId))
{
throw new ArgumentException($"Run Id is not an Int32: {buildIdStr}");
}
// Determine whether to call Pipelines or Build endpoint to publish artifact based on variable setting
string usePipelinesArtifactEndpointVar = context.Variables.GetValueOrDefault("Runner.UseActionsArtifactsApis")?.Value;
bool.TryParse(usePipelinesArtifactEndpointVar, out bool usePipelinesArtifactEndpoint);
string containerPath;
long containerId;
context.Output($"Downloading artifact '{artifactName}' to: '{targetPath}'");
if (usePipelinesArtifactEndpoint)
// Definition ID is a dummy value only used by HTTP client routing purposes
int definitionId = 1;
var pipelinesHelper = new PipelinesServer(context.VssConnection);
var actionsStorageArtifact = await pipelinesHelper.GetActionsStorageArtifact(definitionId, buildId, artifactName, token);
if (actionsStorageArtifact == null)
{
context.Debug("Downloading artifact using v2 endpoint");
// Definition ID is a dummy value only used by HTTP client routing purposes
int definitionId = 1;
var pipelinesHelper = new PipelinesServer(context.VssConnection);
var actionsStorageArtifact = await pipelinesHelper.GetActionsStorageArtifact(definitionId, buildId, artifactName, token);
if (actionsStorageArtifact == null)
{
throw new Exception($"The actions storage artifact for '{artifactName}' could not be found, or is no longer available");
}
containerPath = actionsStorageArtifact.Name; // In actions storage artifacts, name equals the path
containerId = actionsStorageArtifact.ContainerId;
}
else
{
context.Debug("Downloading artifact using v1 endpoint");
BuildServer buildHelper = new BuildServer(context.VssConnection);
BuildArtifact buildArtifact = await buildHelper.GetArtifact(projectId, buildId, artifactName, token);
if (string.Equals(buildArtifact.Resource.Type, "Container", StringComparison.OrdinalIgnoreCase) ||
// Artifact was published by Pipelines endpoint, check new type here to handle rollback scenario
string.Equals(buildArtifact.Resource.Type, "Actions_Storage", StringComparison.OrdinalIgnoreCase))
{
string containerUrl = buildArtifact.Resource.Data;
string[] parts = containerUrl.Split(new[] { '/' }, 3);
if (parts.Length < 3 || !long.TryParse(parts[1], out containerId))
{
throw new ArgumentOutOfRangeException($"Invalid container url '{containerUrl}' for artifact '{buildArtifact.Name}'");
}
containerPath = parts[2];
}
else
{
throw new NotSupportedException($"Invalid artifact type: {buildArtifact.Resource.Type}");
}
throw new Exception($"The actions storage artifact for '{artifactName}' could not be found, or is no longer available");
}
FileContainerServer fileContainerServer = new FileContainerServer(context.VssConnection, projectId, containerId, containerPath);
string containerPath = actionsStorageArtifact.Name; // In actions storage artifacts, name equals the path
long containerId = actionsStorageArtifact.ContainerId;
FileContainerServer fileContainerServer = new FileContainerServer(context.VssConnection, projectId: new Guid(), containerId, containerPath);
await fileContainerServer.DownloadFromContainerAsync(context, targetPath, token);
context.Output("Artifact download finished.");

View File

@@ -4,9 +4,7 @@ using System.IO;
using System.Linq;
using System.Threading;
using System.Threading.Tasks;
using GitHub.Build.WebApi;
using GitHub.Services.Common;
using GitHub.DistributedTask.WebApi;
using GitHub.Runner.Sdk;
namespace GitHub.Runner.Plugins.Artifact
@@ -45,11 +43,8 @@ namespace GitHub.Runner.Plugins.Artifact
throw new ArgumentException($"Artifact name is not valid: {artifactName}. It cannot contain '\\', '/', \"', ':', '<', '>', '|', '*', and '?'");
}
// Project ID
Guid projectId = new Guid(context.Variables.GetValueOrDefault(BuildVariables.TeamProjectId)?.Value ?? Guid.Empty.ToString());
// Build ID
string buildIdStr = context.Variables.GetValueOrDefault(BuildVariables.BuildId)?.Value ?? string.Empty;
string buildIdStr = context.Variables.GetValueOrDefault(SdkConstants.Variables.Build.BuildId)?.Value ?? string.Empty;
if (!int.TryParse(buildIdStr, out int buildId))
{
throw new ArgumentException($"Run Id is not an Int32: {buildIdStr}");
@@ -65,7 +60,7 @@ namespace GitHub.Runner.Plugins.Artifact
}
// Container ID
string containerIdStr = context.Variables.GetValueOrDefault(BuildVariables.ContainerId)?.Value ?? string.Empty;
string containerIdStr = context.Variables.GetValueOrDefault(SdkConstants.Variables.Build.ContainerId)?.Value ?? string.Empty;
if (!long.TryParse(containerIdStr, out long containerId))
{
throw new ArgumentException($"Container Id is not an Int64: {containerIdStr}");
@@ -73,7 +68,7 @@ namespace GitHub.Runner.Plugins.Artifact
context.Output($"Uploading artifact '{artifactName}' from '{fullPath}' for run #{buildId}");
FileContainerServer fileContainerHelper = new FileContainerServer(context.VssConnection, projectId, containerId, artifactName);
FileContainerServer fileContainerHelper = new FileContainerServer(context.VssConnection, projectId: Guid.Empty, containerId, artifactName);
var propertiesDictionary = new Dictionary<string, string>(StringComparer.OrdinalIgnoreCase);
long size = 0;
@@ -89,38 +84,20 @@ namespace GitHub.Runner.Plugins.Artifact
// if any of the results were successful, make sure to attach them to the build
finally
{
// Determine whether to call Pipelines or Build endpoint to publish artifact based on variable setting
string usePipelinesArtifactEndpointVar = context.Variables.GetValueOrDefault("Runner.UseActionsArtifactsApis")?.Value;
bool.TryParse(usePipelinesArtifactEndpointVar, out bool usePipelinesArtifactEndpoint);
// Definition ID is a dummy value only used by HTTP client routing purposes
int definitionId = 1;
if (usePipelinesArtifactEndpoint)
{
// Definition ID is a dummy value only used by HTTP client routing purposes
int definitionId = 1;
PipelinesServer pipelinesHelper = new PipelinesServer(context.VssConnection);
PipelinesServer pipelinesHelper = new PipelinesServer(context.VssConnection);
var artifact = await pipelinesHelper.AssociateActionsStorageArtifactAsync(
definitionId,
buildId,
containerId,
artifactName,
size,
token);
var artifact = await pipelinesHelper.AssociateActionsStorageArtifactAsync(
definitionId,
buildId,
containerId,
artifactName,
size,
token);
context.Output($"Associated artifact {artifactName} ({artifact.ContainerId}) with run #{buildId}");
context.Debug($"Associated artifact using v2 endpoint");
}
else
{
string fileContainerFullPath = StringUtil.Format($"#/{containerId}/{artifactName}");
BuildServer buildHelper = new BuildServer(context.VssConnection);
string jobId = context.Variables.GetValueOrDefault(WellKnownDistributedTaskVariables.JobId).Value ?? string.Empty;
var artifact = await buildHelper.AssociateArtifact(projectId, buildId, jobId, artifactName, ArtifactResourceTypes.Container, fileContainerFullPath, propertiesDictionary, token);
context.Output($"Associated artifact {artifactName} ({artifact.Id}) with run #{buildId}");
context.Debug($"Associated artifact using v1 endpoint");
}
context.Output($"Associated artifact {artifactName} ({artifact.ContainerId}) with run #{buildId}");
}
}
}

View File

@@ -79,11 +79,6 @@ namespace GitHub.Runner.Plugins.Repository.v1_0
{
// Validate args.
ArgUtil.NotNull(executionContext, nameof(executionContext));
bool useSelfSignedCACert = false;
bool useClientCert = false;
string clientCertPrivateKeyAskPassFile = null;
bool acceptUntrustedCerts = false;
executionContext.Output($"Syncing repository: {repoFullName}");
Uri repositoryUrl = new Uri($"https://github.com/{repoFullName}");
if (!repositoryUrl.IsAbsoluteUri)
@@ -112,9 +107,6 @@ namespace GitHub.Runner.Plugins.Repository.v1_0
}
}
var runnerCert = executionContext.GetCertConfiguration();
acceptUntrustedCerts = runnerCert?.SkipServerCertificateValidation ?? false;
executionContext.Debug($"repository url={repositoryUrl}");
executionContext.Debug($"targetPath={targetPath}");
executionContext.Debug($"sourceBranch={sourceBranch}");
@@ -124,12 +116,6 @@ namespace GitHub.Runner.Plugins.Repository.v1_0
executionContext.Debug($"checkoutNestedSubmodules={checkoutNestedSubmodules}");
executionContext.Debug($"fetchDepth={fetchDepth}");
executionContext.Debug($"gitLfsSupport={gitLfsSupport}");
executionContext.Debug($"acceptUntrustedCerts={acceptUntrustedCerts}");
#if OS_WINDOWS
bool schannelSslBackend = StringUtil.ConvertToBoolean(executionContext.GetRunnerContext("gituseschannel"));
executionContext.Debug($"schannelSslBackend={schannelSslBackend}");
#endif
// Initialize git command manager with additional environment variables.
Dictionary<string, string> gitEnv = new Dictionary<string, string>(StringComparer.OrdinalIgnoreCase);
@@ -164,54 +150,6 @@ namespace GitHub.Runner.Plugins.Repository.v1_0
// prepare askpass for client cert private key, if the repository's endpoint url match the runner config url
var systemConnection = executionContext.Endpoints.Single(x => string.Equals(x.Name, WellKnownServiceEndpointNames.SystemVssConnection, StringComparison.OrdinalIgnoreCase));
if (runnerCert != null && Uri.Compare(repositoryUrl, systemConnection.Url, UriComponents.SchemeAndServer, UriFormat.Unescaped, StringComparison.OrdinalIgnoreCase) == 0)
{
if (!string.IsNullOrEmpty(runnerCert.CACertificateFile))
{
useSelfSignedCACert = true;
}
if (!string.IsNullOrEmpty(runnerCert.ClientCertificateFile) &&
!string.IsNullOrEmpty(runnerCert.ClientCertificatePrivateKeyFile))
{
useClientCert = true;
// prepare askpass for client cert password
if (!string.IsNullOrEmpty(runnerCert.ClientCertificatePassword))
{
clientCertPrivateKeyAskPassFile = Path.Combine(executionContext.GetRunnerContext("temp"), $"{Guid.NewGuid()}.sh");
List<string> askPass = new List<string>();
askPass.Add("#!/bin/sh");
askPass.Add($"echo \"{runnerCert.ClientCertificatePassword}\"");
File.WriteAllLines(clientCertPrivateKeyAskPassFile, askPass);
#if !OS_WINDOWS
string toolPath = WhichUtil.Which("chmod", true);
string argLine = $"775 {clientCertPrivateKeyAskPassFile}";
executionContext.Command($"chmod {argLine}");
var processInvoker = new ProcessInvoker(executionContext);
processInvoker.OutputDataReceived += (object sender, ProcessDataReceivedEventArgs args) =>
{
if (!string.IsNullOrEmpty(args.Data))
{
executionContext.Output(args.Data);
}
};
processInvoker.ErrorDataReceived += (object sender, ProcessDataReceivedEventArgs args) =>
{
if (!string.IsNullOrEmpty(args.Data))
{
executionContext.Output(args.Data);
}
};
string workingDirectory = executionContext.GetRunnerContext("workspace");
await processInvoker.ExecuteAsync(workingDirectory, toolPath, argLine, null, true, CancellationToken.None);
#endif
}
}
}
// Check the current contents of the root folder to see if there is already a repo
// If there is a repo, see if it matches the one we are expecting to be there based on the remote fetch url
@@ -361,46 +299,6 @@ namespace GitHub.Runner.Plugins.Repository.v1_0
additionalFetchArgs.Add($"-c http.extraheader=\"AUTHORIZATION: {GenerateBasicAuthHeader(executionContext, accessToken)}\"");
}
// Prepare ignore ssl cert error config for fetch.
if (acceptUntrustedCerts)
{
additionalFetchArgs.Add($"-c http.sslVerify=false");
additionalLfsFetchArgs.Add($"-c http.sslVerify=false");
}
// Prepare self-signed CA cert config for fetch from server.
if (useSelfSignedCACert)
{
executionContext.Debug($"Use self-signed certificate '{runnerCert.CACertificateFile}' for git fetch.");
additionalFetchArgs.Add($"-c http.sslcainfo=\"{runnerCert.CACertificateFile}\"");
additionalLfsFetchArgs.Add($"-c http.sslcainfo=\"{runnerCert.CACertificateFile}\"");
}
// Prepare client cert config for fetch from server.
if (useClientCert)
{
executionContext.Debug($"Use client certificate '{runnerCert.ClientCertificateFile}' for git fetch.");
if (!string.IsNullOrEmpty(clientCertPrivateKeyAskPassFile))
{
additionalFetchArgs.Add($"-c http.sslcert=\"{runnerCert.ClientCertificateFile}\" -c http.sslkey=\"{runnerCert.ClientCertificatePrivateKeyFile}\" -c http.sslCertPasswordProtected=true -c core.askpass=\"{clientCertPrivateKeyAskPassFile}\"");
additionalLfsFetchArgs.Add($"-c http.sslcert=\"{runnerCert.ClientCertificateFile}\" -c http.sslkey=\"{runnerCert.ClientCertificatePrivateKeyFile}\" -c http.sslCertPasswordProtected=true -c core.askpass=\"{clientCertPrivateKeyAskPassFile}\"");
}
else
{
additionalFetchArgs.Add($"-c http.sslcert=\"{runnerCert.ClientCertificateFile}\" -c http.sslkey=\"{runnerCert.ClientCertificatePrivateKeyFile}\"");
additionalLfsFetchArgs.Add($"-c http.sslcert=\"{runnerCert.ClientCertificateFile}\" -c http.sslkey=\"{runnerCert.ClientCertificatePrivateKeyFile}\"");
}
}
#if OS_WINDOWS
if (schannelSslBackend)
{
executionContext.Debug("Use SChannel SslBackend for git fetch.");
additionalFetchArgs.Add("-c http.sslbackend=\"schannel\"");
additionalLfsFetchArgs.Add("-c http.sslbackend=\"schannel\"");
}
#endif
// Prepare gitlfs url for fetch and checkout
if (gitLfsSupport)
{
@@ -502,55 +400,12 @@ namespace GitHub.Runner.Plugins.Repository.v1_0
additionalSubmoduleUpdateArgs.Add($"-c http.{authorityUrl}.extraheader=\"AUTHORIZATION: {GenerateBasicAuthHeader(executionContext, accessToken)}\"");
}
// Prepare ignore ssl cert error config for fetch.
if (acceptUntrustedCerts)
{
additionalSubmoduleUpdateArgs.Add($"-c http.sslVerify=false");
}
// Prepare self-signed CA cert config for submodule update.
if (useSelfSignedCACert)
{
executionContext.Debug($"Use self-signed CA certificate '{runnerCert.CACertificateFile}' for git submodule update.");
string authorityUrl = repositoryUrl.AbsoluteUri.Replace(repositoryUrl.PathAndQuery, string.Empty);
additionalSubmoduleUpdateArgs.Add($"-c http.{authorityUrl}.sslcainfo=\"{runnerCert.CACertificateFile}\"");
}
// Prepare client cert config for submodule update.
if (useClientCert)
{
executionContext.Debug($"Use client certificate '{runnerCert.ClientCertificateFile}' for git submodule update.");
string authorityUrl = repositoryUrl.AbsoluteUri.Replace(repositoryUrl.PathAndQuery, string.Empty);
if (!string.IsNullOrEmpty(clientCertPrivateKeyAskPassFile))
{
additionalSubmoduleUpdateArgs.Add($"-c http.{authorityUrl}.sslcert=\"{runnerCert.ClientCertificateFile}\" -c http.{authorityUrl}.sslkey=\"{runnerCert.ClientCertificatePrivateKeyFile}\" -c http.{authorityUrl}.sslCertPasswordProtected=true -c core.askpass=\"{clientCertPrivateKeyAskPassFile}\"");
}
else
{
additionalSubmoduleUpdateArgs.Add($"-c http.{authorityUrl}.sslcert=\"{runnerCert.ClientCertificateFile}\" -c http.{authorityUrl}.sslkey=\"{runnerCert.ClientCertificatePrivateKeyFile}\"");
}
}
#if OS_WINDOWS
if (schannelSslBackend)
{
executionContext.Debug("Use SChannel SslBackend for git submodule update.");
additionalSubmoduleUpdateArgs.Add("-c http.sslbackend=\"schannel\"");
}
#endif
int exitCode_submoduleUpdate = await gitCommandManager.GitSubmoduleUpdate(executionContext, targetPath, fetchDepth, string.Join(" ", additionalSubmoduleUpdateArgs), checkoutNestedSubmodules, cancellationToken);
if (exitCode_submoduleUpdate != 0)
{
throw new InvalidOperationException($"Git submodule update failed with exit code: {exitCode_submoduleUpdate}");
}
}
if (useClientCert && !string.IsNullOrEmpty(clientCertPrivateKeyAskPassFile))
{
executionContext.Debug("Remove git.sslkey askpass file.");
IOUtil.DeleteFile(clientCertPrivateKeyAskPassFile);
}
}
private async Task<bool> IsRepositoryOriginUrlMatch(RunnerActionPluginExecutionContext context, GitCliManager gitCommandManager, string repositoryPath, Uri expectedRepositoryOriginUrl)

View File

@@ -65,11 +65,6 @@ namespace GitHub.Runner.Plugins.Repository.v1_1
// Validate args.
ArgUtil.NotNull(executionContext, nameof(executionContext));
Dictionary<string, string> configModifications = new Dictionary<string, string>();
bool useSelfSignedCACert = false;
bool useClientCert = false;
string clientCertPrivateKeyAskPassFile = null;
bool acceptUntrustedCerts = false;
executionContext.Output($"Syncing repository: {repoFullName}");
Uri repositoryUrl = new Uri($"https://github.com/{repoFullName}");
if (!repositoryUrl.IsAbsoluteUri)
@@ -98,9 +93,6 @@ namespace GitHub.Runner.Plugins.Repository.v1_1
}
}
var runnerCert = executionContext.GetCertConfiguration();
acceptUntrustedCerts = runnerCert?.SkipServerCertificateValidation ?? false;
executionContext.Debug($"repository url={repositoryUrl}");
executionContext.Debug($"targetPath={targetPath}");
executionContext.Debug($"sourceBranch={sourceBranch}");
@@ -110,12 +102,6 @@ namespace GitHub.Runner.Plugins.Repository.v1_1
executionContext.Debug($"checkoutNestedSubmodules={checkoutNestedSubmodules}");
executionContext.Debug($"fetchDepth={fetchDepth}");
executionContext.Debug($"gitLfsSupport={gitLfsSupport}");
executionContext.Debug($"acceptUntrustedCerts={acceptUntrustedCerts}");
#if OS_WINDOWS
bool schannelSslBackend = StringUtil.ConvertToBoolean(executionContext.GetRunnerContext("gituseschannel"));
executionContext.Debug($"schannelSslBackend={schannelSslBackend}");
#endif
// Initialize git command manager with additional environment variables.
Dictionary<string, string> gitEnv = new Dictionary<string, string>(StringComparer.OrdinalIgnoreCase);
@@ -153,54 +139,6 @@ namespace GitHub.Runner.Plugins.Repository.v1_1
// prepare askpass for client cert private key, if the repository's endpoint url match the runner config url
var systemConnection = executionContext.Endpoints.Single(x => string.Equals(x.Name, WellKnownServiceEndpointNames.SystemVssConnection, StringComparison.OrdinalIgnoreCase));
if (runnerCert != null && Uri.Compare(repositoryUrl, systemConnection.Url, UriComponents.SchemeAndServer, UriFormat.Unescaped, StringComparison.OrdinalIgnoreCase) == 0)
{
if (!string.IsNullOrEmpty(runnerCert.CACertificateFile))
{
useSelfSignedCACert = true;
}
if (!string.IsNullOrEmpty(runnerCert.ClientCertificateFile) &&
!string.IsNullOrEmpty(runnerCert.ClientCertificatePrivateKeyFile))
{
useClientCert = true;
// prepare askpass for client cert password
if (!string.IsNullOrEmpty(runnerCert.ClientCertificatePassword))
{
clientCertPrivateKeyAskPassFile = Path.Combine(executionContext.GetRunnerContext("temp"), $"{Guid.NewGuid()}.sh");
List<string> askPass = new List<string>();
askPass.Add("#!/bin/sh");
askPass.Add($"echo \"{runnerCert.ClientCertificatePassword}\"");
File.WriteAllLines(clientCertPrivateKeyAskPassFile, askPass);
#if !OS_WINDOWS
string toolPath = WhichUtil.Which("chmod", true);
string argLine = $"775 {clientCertPrivateKeyAskPassFile}";
executionContext.Command($"chmod {argLine}");
var processInvoker = new ProcessInvoker(executionContext);
processInvoker.OutputDataReceived += (object sender, ProcessDataReceivedEventArgs args) =>
{
if (!string.IsNullOrEmpty(args.Data))
{
executionContext.Output(args.Data);
}
};
processInvoker.ErrorDataReceived += (object sender, ProcessDataReceivedEventArgs args) =>
{
if (!string.IsNullOrEmpty(args.Data))
{
executionContext.Output(args.Data);
}
};
string workingDirectory = executionContext.GetRunnerContext("workspace");
await processInvoker.ExecuteAsync(workingDirectory, toolPath, argLine, null, true, CancellationToken.None);
#endif
}
}
}
// Check the current contents of the root folder to see if there is already a repo
// If there is a repo, see if it matches the one we are expecting to be there based on the remote fetch url
@@ -355,46 +293,6 @@ namespace GitHub.Runner.Plugins.Repository.v1_1
throw new InvalidOperationException($"Git config failed with exit code: {exitCode_config}");
}
// Prepare ignore ssl cert error config for fetch.
if (acceptUntrustedCerts)
{
additionalFetchArgs.Add($"-c http.sslVerify=false");
additionalLfsFetchArgs.Add($"-c http.sslVerify=false");
}
// Prepare self-signed CA cert config for fetch from server.
if (useSelfSignedCACert)
{
executionContext.Debug($"Use self-signed certificate '{runnerCert.CACertificateFile}' for git fetch.");
additionalFetchArgs.Add($"-c http.sslcainfo=\"{runnerCert.CACertificateFile}\"");
additionalLfsFetchArgs.Add($"-c http.sslcainfo=\"{runnerCert.CACertificateFile}\"");
}
// Prepare client cert config for fetch from server.
if (useClientCert)
{
executionContext.Debug($"Use client certificate '{runnerCert.ClientCertificateFile}' for git fetch.");
if (!string.IsNullOrEmpty(clientCertPrivateKeyAskPassFile))
{
additionalFetchArgs.Add($"-c http.sslcert=\"{runnerCert.ClientCertificateFile}\" -c http.sslkey=\"{runnerCert.ClientCertificatePrivateKeyFile}\" -c http.sslCertPasswordProtected=true -c core.askpass=\"{clientCertPrivateKeyAskPassFile}\"");
additionalLfsFetchArgs.Add($"-c http.sslcert=\"{runnerCert.ClientCertificateFile}\" -c http.sslkey=\"{runnerCert.ClientCertificatePrivateKeyFile}\" -c http.sslCertPasswordProtected=true -c core.askpass=\"{clientCertPrivateKeyAskPassFile}\"");
}
else
{
additionalFetchArgs.Add($"-c http.sslcert=\"{runnerCert.ClientCertificateFile}\" -c http.sslkey=\"{runnerCert.ClientCertificatePrivateKeyFile}\"");
additionalLfsFetchArgs.Add($"-c http.sslcert=\"{runnerCert.ClientCertificateFile}\" -c http.sslkey=\"{runnerCert.ClientCertificatePrivateKeyFile}\"");
}
}
#if OS_WINDOWS
if (schannelSslBackend)
{
executionContext.Debug("Use SChannel SslBackend for git fetch.");
additionalFetchArgs.Add("-c http.sslbackend=\"schannel\"");
additionalLfsFetchArgs.Add("-c http.sslbackend=\"schannel\"");
}
#endif
// Prepare gitlfs url for fetch and checkout
if (gitLfsSupport)
{
@@ -484,43 +382,6 @@ namespace GitHub.Runner.Plugins.Repository.v1_1
List<string> additionalSubmoduleUpdateArgs = new List<string>();
// Prepare ignore ssl cert error config for fetch.
if (acceptUntrustedCerts)
{
additionalSubmoduleUpdateArgs.Add($"-c http.sslVerify=false");
}
// Prepare self-signed CA cert config for submodule update.
if (useSelfSignedCACert)
{
executionContext.Debug($"Use self-signed CA certificate '{runnerCert.CACertificateFile}' for git submodule update.");
string authorityUrl = repositoryUrl.AbsoluteUri.Replace(repositoryUrl.PathAndQuery, string.Empty);
additionalSubmoduleUpdateArgs.Add($"-c http.{authorityUrl}.sslcainfo=\"{runnerCert.CACertificateFile}\"");
}
// Prepare client cert config for submodule update.
if (useClientCert)
{
executionContext.Debug($"Use client certificate '{runnerCert.ClientCertificateFile}' for git submodule update.");
string authorityUrl = repositoryUrl.AbsoluteUri.Replace(repositoryUrl.PathAndQuery, string.Empty);
if (!string.IsNullOrEmpty(clientCertPrivateKeyAskPassFile))
{
additionalSubmoduleUpdateArgs.Add($"-c http.{authorityUrl}.sslcert=\"{runnerCert.ClientCertificateFile}\" -c http.{authorityUrl}.sslkey=\"{runnerCert.ClientCertificatePrivateKeyFile}\" -c http.{authorityUrl}.sslCertPasswordProtected=true -c core.askpass=\"{clientCertPrivateKeyAskPassFile}\"");
}
else
{
additionalSubmoduleUpdateArgs.Add($"-c http.{authorityUrl}.sslcert=\"{runnerCert.ClientCertificateFile}\" -c http.{authorityUrl}.sslkey=\"{runnerCert.ClientCertificatePrivateKeyFile}\"");
}
}
#if OS_WINDOWS
if (schannelSslBackend)
{
executionContext.Debug("Use SChannel SslBackend for git submodule update.");
additionalSubmoduleUpdateArgs.Add("-c http.sslbackend=\"schannel\"");
}
#endif
int exitCode_submoduleUpdate = await gitCommandManager.GitSubmoduleUpdate(executionContext, targetPath, fetchDepth, string.Join(" ", additionalSubmoduleUpdateArgs), checkoutNestedSubmodules, cancellationToken);
if (exitCode_submoduleUpdate != 0)
{
@@ -528,12 +389,6 @@ namespace GitHub.Runner.Plugins.Repository.v1_1
}
}
if (useClientCert && !string.IsNullOrEmpty(clientCertPrivateKeyAskPassFile))
{
executionContext.Debug("Remove git.sslkey askpass file.");
IOUtil.DeleteFile(clientCertPrivateKeyAskPassFile);
}
// Set intra-task variable for post job cleanup
executionContext.SetIntraActionState("repositoryPath", targetPath);
executionContext.SetIntraActionState("modifiedgitconfig", JsonUtility.ToString(configModifications.Keys));

View File

@@ -1,7 +1,7 @@
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFramework>netcoreapp3.0</TargetFramework>
<TargetFramework>netcoreapp3.1</TargetFramework>
<OutputType>Library</OutputType>
<RuntimeIdentifiers>win-x64;win-x86;linux-x64;linux-arm64;linux-arm;osx-x64</RuntimeIdentifiers>
<TargetLatestRuntimePatch>true</TargetLatestRuntimePatch>

View File

@@ -83,21 +83,6 @@ namespace GitHub.Runner.Sdk
}
VssClientHttpRequestSettings.Default.UserAgent = headerValues;
var certSetting = GetCertConfiguration();
if (certSetting != null)
{
if (!string.IsNullOrEmpty(certSetting.ClientCertificateArchiveFile))
{
VssClientHttpRequestSettings.Default.ClientCertificateManager = new RunnerClientCertificateManager(certSetting.ClientCertificateArchiveFile, certSetting.ClientCertificatePassword);
}
if (certSetting.SkipServerCertificateValidation)
{
VssClientHttpRequestSettings.Default.ServerCertificateValidationCallback = HttpClientHandler.DangerousAcceptAnyServerCertificateValidator;
}
}
VssHttpMessageHandler.DefaultWebProxy = this.WebProxy;
ServiceEndpoint systemConnection = this.Endpoints.FirstOrDefault(e => string.Equals(e.Name, WellKnownServiceEndpointNames.SystemVssConnection, StringComparison.OrdinalIgnoreCase));
ArgUtil.NotNull(systemConnection, nameof(systemConnection));
@@ -227,40 +212,6 @@ namespace GitHub.Runner.Sdk
}
}
public RunnerCertificateSettings GetCertConfiguration()
{
bool skipCertValidation = StringUtil.ConvertToBoolean(GetRunnerContext("SkipCertValidation"));
string caFile = GetRunnerContext("CAInfo");
string clientCertFile = GetRunnerContext("ClientCert");
if (!string.IsNullOrEmpty(caFile) || !string.IsNullOrEmpty(clientCertFile) || skipCertValidation)
{
var certConfig = new RunnerCertificateSettings();
certConfig.SkipServerCertificateValidation = skipCertValidation;
certConfig.CACertificateFile = caFile;
if (!string.IsNullOrEmpty(clientCertFile))
{
certConfig.ClientCertificateFile = clientCertFile;
string clientCertKey = GetRunnerContext("ClientCertKey");
string clientCertArchive = GetRunnerContext("ClientCertArchive");
string clientCertPassword = GetRunnerContext("ClientCertPassword");
certConfig.ClientCertificatePrivateKeyFile = clientCertKey;
certConfig.ClientCertificateArchiveFile = clientCertArchive;
certConfig.ClientCertificatePassword = clientCertPassword;
certConfig.VssClientCertificateManager = new RunnerClientCertificateManager(clientCertArchive, clientCertPassword);
}
return certConfig;
}
else
{
return null;
}
}
private string Escape(string input)
{
foreach (var mapping in _commandEscapeMappings)

View File

@@ -1,7 +1,7 @@
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFramework>netcoreapp3.0</TargetFramework>
<TargetFramework>netcoreapp3.1</TargetFramework>
<OutputType>Library</OutputType>
<RuntimeIdentifiers>win-x64;win-x86;linux-x64;linux-arm64;linux-arm;osx-x64</RuntimeIdentifiers>
<TargetLatestRuntimePatch>true</TargetLatestRuntimePatch>

View File

@@ -1,40 +0,0 @@

using System.Security.Cryptography.X509Certificates;
using GitHub.Services.Common;
namespace GitHub.Runner.Sdk
{
public class RunnerCertificateSettings
{
public bool SkipServerCertificateValidation { get; set; }
public string CACertificateFile { get; set; }
public string ClientCertificateFile { get; set; }
public string ClientCertificatePrivateKeyFile { get; set; }
public string ClientCertificateArchiveFile { get; set; }
public string ClientCertificatePassword { get; set; }
public IVssClientCertificateManager VssClientCertificateManager { get; set; }
}
public class RunnerClientCertificateManager : IVssClientCertificateManager
{
private readonly X509Certificate2Collection _clientCertificates = new X509Certificate2Collection();
public X509Certificate2Collection ClientCertificates => _clientCertificates;
public RunnerClientCertificateManager()
{
}
public RunnerClientCertificateManager(string clientCertificateArchiveFile, string clientCertificatePassword)
{
AddClientCertificate(clientCertificateArchiveFile, clientCertificatePassword);
}
public void AddClientCertificate(string clientCertificateArchiveFile, string clientCertificatePassword)
{
if (!string.IsNullOrEmpty(clientCertificateArchiveFile))
{
_clientCertificates.Add(new X509Certificate2(clientCertificateArchiveFile, clientCertificatePassword));
}
}
}
}

View File

@@ -21,6 +21,7 @@ namespace GitHub.Runner.Sdk
private string _httpsProxyAddress;
private string _httpsProxyUsername;
private string _httpsProxyPassword;
private string _noProxyString;
private readonly List<ByPassInfo> _noProxyList = new List<ByPassInfo>();
private readonly HashSet<string> _noProxyUnique = new HashSet<string>(StringComparer.OrdinalIgnoreCase);
@@ -33,6 +34,7 @@ namespace GitHub.Runner.Sdk
public string HttpsProxyAddress => _httpsProxyAddress;
public string HttpsProxyUsername => _httpsProxyUsername;
public string HttpsProxyPassword => _httpsProxyPassword;
public string NoProxyString => _noProxyString;
public List<ByPassInfo> NoProxyList => _noProxyList;
@@ -71,6 +73,10 @@ namespace GitHub.Runner.Sdk
{
_httpProxyAddress = proxyHttpUri.AbsoluteUri;
// Set both environment variables since there are tools support both casing (curl, wget) and tools support only one casing (docker)
Environment.SetEnvironmentVariable("HTTP_PROXY", _httpProxyAddress);
Environment.SetEnvironmentVariable("http_proxy", _httpProxyAddress);
// the proxy url looks like http://[user:pass@]127.0.0.1:8888
var userInfo = Uri.UnescapeDataString(proxyHttpUri.UserInfo).Split(':', 2, StringSplitOptions.RemoveEmptyEntries);
if (userInfo.Length == 2)
@@ -97,6 +103,10 @@ namespace GitHub.Runner.Sdk
{
_httpsProxyAddress = proxyHttpsUri.AbsoluteUri;
// Set both environment variables since there are tools support both casing (curl, wget) and tools support only one casing (docker)
Environment.SetEnvironmentVariable("HTTPS_PROXY", _httpsProxyAddress);
Environment.SetEnvironmentVariable("https_proxy", _httpsProxyAddress);
// the proxy url looks like http://[user:pass@]127.0.0.1:8888
var userInfo = Uri.UnescapeDataString(proxyHttpsUri.UserInfo).Split(':', 2, StringSplitOptions.RemoveEmptyEntries);
if (userInfo.Length == 2)
@@ -121,6 +131,12 @@ namespace GitHub.Runner.Sdk
if (!string.IsNullOrEmpty(noProxyList))
{
_noProxyString = noProxyList;
// Set both environment variables since there are tools support both casing (curl, wget) and tools support only one casing (docker)
Environment.SetEnvironmentVariable("NO_PROXY", noProxyList);
Environment.SetEnvironmentVariable("no_proxy", noProxyList);
var noProxyListSplit = noProxyList.Split(',', StringSplitOptions.RemoveEmptyEntries);
foreach (string noProxy in noProxyListSplit)
{

View File

@@ -0,0 +1,19 @@
using System;
namespace GitHub.Runner.Sdk
{
public class SdkConstants
{
public static class Variables
{
public static class Build
{
// Legacy "build" variables historically used by the runner
// DO NOT add new variables here -- instead use either the Actions or Runner namespaces
public const String BuildId = "build.buildId";
public const String BuildNumber = "build.buildNumber";
public const String ContainerId = "build.containerId";
}
}
}
}

View File

@@ -14,7 +14,7 @@ namespace GitHub.Runner.Sdk
{
public static class VssUtil
{
public static void InitializeVssClientSettings(ProductInfoHeaderValue additionalUserAgent, IWebProxy proxy, IVssClientCertificateManager clientCert)
public static void InitializeVssClientSettings(ProductInfoHeaderValue additionalUserAgent, IWebProxy proxy)
{
var headerValues = new List<ProductInfoHeaderValue>();
headerValues.Add(additionalUserAgent);
@@ -26,7 +26,6 @@ namespace GitHub.Runner.Sdk
}
VssClientHttpRequestSettings.Default.UserAgent = headerValues;
VssClientHttpRequestSettings.Default.ClientCertificateManager = clientCert;
VssHttpMessageHandler.DefaultWebProxy = proxy;
}
@@ -83,7 +82,7 @@ namespace GitHub.Runner.Sdk
if (serviceEndpoint.Authorization.Scheme == EndpointAuthorizationSchemes.OAuth &&
serviceEndpoint.Authorization.Parameters.TryGetValue(EndpointAuthorizationParameters.AccessToken, out accessToken))
{
credentials = new VssCredentials(null, new VssOAuthAccessTokenCredential(accessToken), CredentialPromptType.DoNotPrompt);
credentials = new VssCredentials(new VssOAuthAccessTokenCredential(accessToken), CredentialPromptType.DoNotPrompt);
}
return credentials;

View File

@@ -7,7 +7,7 @@ namespace GitHub.Runner.Sdk
{
public static class WhichUtil
{
public static string Which(string command, bool require = false, ITraceWriter trace = null)
public static string Which(string command, bool require = false, ITraceWriter trace = null, string prependPath = null)
{
ArgUtil.NotNullOrEmpty(command, nameof(command));
trace?.Info($"Which: '{command}'");
@@ -17,6 +17,10 @@ namespace GitHub.Runner.Sdk
trace?.Info("PATH environment variable not defined.");
path = path ?? string.Empty;
}
if (!string.IsNullOrEmpty(prependPath))
{
path = PathUtil.PrependPath(prependPath, path);
}
string[] pathSegments = path.Split(new Char[] { Path.PathSeparator }, StringSplitOptions.RemoveEmptyEntries);
for (int i = 0; i < pathSegments.Length; i++)

View File

@@ -33,7 +33,7 @@ namespace GitHub.Runner.Worker
{
private const int _defaultFileStreamBufferSize = 4096;
//81920 is the default used by System.IO.Stream.CopyTo and is under the large object heap threshold (85k).
//81920 is the default used by System.IO.Stream.CopyTo and is under the large object heap threshold (85k).
private const int _defaultCopyBufferSize = 81920;
private readonly Dictionary<Guid, ContainerInfo> _cachedActionContainers = new Dictionary<Guid, ContainerInfo>();
@@ -198,14 +198,21 @@ namespace GitHub.Runner.Worker
Trace.Info($"Load action that reference repository from '{actionDirectory}'");
definition.Directory = actionDirectory;
string manifestFile = Path.Combine(actionDirectory, "action.yml");
string manifestFile = Path.Combine(actionDirectory, Constants.Path.ActionManifestYmlFile);
string manifestFileYaml = Path.Combine(actionDirectory, Constants.Path.ActionManifestYamlFile);
string dockerFile = Path.Combine(actionDirectory, "Dockerfile");
string dockerFileLowerCase = Path.Combine(actionDirectory, "dockerfile");
if (File.Exists(manifestFile))
if (File.Exists(manifestFile) || File.Exists(manifestFileYaml))
{
var manifestManager = HostContext.GetService<IActionManifestManager>();
definition.Data = manifestManager.Load(executionContext, manifestFile);
if (File.Exists(manifestFile))
{
definition.Data = manifestManager.Load(executionContext, manifestFile);
}
else
{
definition.Data = manifestManager.Load(executionContext, manifestFileYaml);
}
Trace.Verbose($"Action friendly name: '{definition.Data.Name}'");
Trace.Verbose($"Action description: '{definition.Data.Description}'");
@@ -314,7 +321,7 @@ namespace GitHub.Runner.Worker
else
{
var fullPath = IOUtil.ResolvePath(actionDirectory, "."); // resolve full path without access filesystem.
throw new NotSupportedException($"Can't find 'action.yml' or 'Dockerfile' under '{fullPath}'. Did you forget to run actions/checkout before running your local action?");
throw new NotSupportedException($"Can't find 'action.yml', 'action.yaml' or 'Dockerfile' under '{fullPath}'. Did you forget to run actions/checkout before running your local action?");
}
}
else if (action.Reference.Type == Pipelines.ActionSourceType.Script)
@@ -477,7 +484,7 @@ namespace GitHub.Runner.Worker
int retryCount = 0;
// Allow up to 20 * 60s for any action to be downloaded from github graph.
// Allow up to 20 * 60s for any action to be downloaded from github graph.
int timeoutSeconds = 20 * 60;
while (retryCount < 3)
{
@@ -655,12 +662,21 @@ namespace GitHub.Runner.Worker
// find the docker file or action.yml file
var dockerFile = Path.Combine(actionEntryDirectory, "Dockerfile");
var dockerFileLowerCase = Path.Combine(actionEntryDirectory, "dockerfile");
var actionManifest = Path.Combine(actionEntryDirectory, "action.yml");
if (File.Exists(actionManifest))
var actionManifest = Path.Combine(actionEntryDirectory, Constants.Path.ActionManifestYmlFile);
var actionManifestYaml = Path.Combine(actionEntryDirectory, Constants.Path.ActionManifestYamlFile);
if (File.Exists(actionManifest) || File.Exists(actionManifestYaml))
{
executionContext.Debug($"action.yml for action: '{actionManifest}'.");
var manifestManager = HostContext.GetService<IActionManifestManager>();
var actionDefinitionData = manifestManager.Load(executionContext, actionManifest);
ActionDefinitionData actionDefinitionData = null;
if (File.Exists(actionManifest))
{
actionDefinitionData = manifestManager.Load(executionContext, actionManifest);
}
else
{
actionDefinitionData = manifestManager.Load(executionContext, actionManifestYaml);
}
if (actionDefinitionData.Execution.ExecutionType == ActionExecutionType.Container)
{
@@ -720,7 +736,7 @@ namespace GitHub.Runner.Worker
else
{
var fullPath = IOUtil.ResolvePath(actionEntryDirectory, "."); // resolve full path without access filesystem.
throw new InvalidOperationException($"Can't find 'action.yml' or 'Dockerfile' under '{fullPath}'. Did you forget to run actions/checkout before running your local action?");
throw new InvalidOperationException($"Can't find 'action.yml', 'action.yaml' or 'Dockerfile' under '{fullPath}'. Did you forget to run actions/checkout before running your local action?");
}
}
}

View File

@@ -116,7 +116,7 @@ namespace GitHub.Runner.Worker
if (actionDefinition.Execution == null)
{
executionContext.Debug($"Loaded action.yml file: {StringUtil.ConvertToJson(actionDefinition)}");
throw new ArgumentException($"Top level 'run:' section is required for {manifestFile}");
throw new ArgumentException($"Top level 'runs:' section is required for {manifestFile}");
}
else
{
@@ -341,7 +341,7 @@ namespace GitHub.Runner.Worker
EntryPoint = entrypointToken?.Value,
Environment = envToken,
Cleanup = postEntrypointToken?.Value,
CleanupCondition = postIfToken?.Value
CleanupCondition = postIfToken?.Value ?? "always()"
};
}
}
@@ -357,7 +357,7 @@ namespace GitHub.Runner.Worker
{
Script = mainToken.Value,
Cleanup = postToken?.Value,
CleanupCondition = postIfToken?.Value
CleanupCondition = postIfToken?.Value ?? "always()"
};
}
}

View File

@@ -179,17 +179,15 @@ namespace GitHub.Runner.Worker
ExecutionContext.Debug("Loading env");
var environment = new Dictionary<String, String>(VarUtil.EnvironmentVariableKeyComparer);
// Apply environment set using ##[set-env] first since these are job level env
foreach (var env in ExecutionContext.EnvironmentVariables)
#if OS_WINDOWS
var envContext = ExecutionContext.ExpressionValues["env"] as DictionaryContextData;
#else
var envContext = ExecutionContext.ExpressionValues["env"] as CaseSensitiveDictionaryContextData;
#endif
// Apply environment from env context, env context contains job level env and action's evn block
foreach (var env in envContext)
{
environment[env.Key] = env.Value ?? string.Empty;
}
// Apply action's env block later.
var actionEnvironment = templateEvaluator.EvaluateStepEnvironment(Action.Environment, ExecutionContext.ExpressionValues, VarUtil.EnvironmentVariableKeyComparer);
foreach (var env in actionEnvironment)
{
environment[env.Key] = env.Value ?? string.Empty;
environment[env.Key] = env.Value.ToString();
}
// Apply action's intra-action state at last

View File

@@ -2,9 +2,9 @@
using System.Collections.Generic;
using System.IO;
using GitHub.Runner.Common.Util;
using Pipelines = GitHub.DistributedTask.Pipelines;
using GitHub.Runner.Common;
using GitHub.Runner.Sdk;
using Pipelines = GitHub.DistributedTask.Pipelines;
namespace GitHub.Runner.Worker.Container
{
@@ -19,7 +19,6 @@ namespace GitHub.Runner.Worker.Container
public ContainerInfo()
{
}
public ContainerInfo(IHostContext hostContext, Pipelines.JobContainer container, bool isJobContainer = true, string networkAlias = null)
@@ -64,6 +63,8 @@ namespace GitHub.Runner.Worker.Container
UserMountVolumes[volume] = volume;
}
}
UpdateWebProxyEnv(hostContext.WebProxy);
}
public string ContainerId { get; set; }
@@ -223,6 +224,26 @@ namespace GitHub.Runner.Worker.Container
{
_pathMappings.Insert(0, new PathMapping(hostCommonPath, containerCommonPath));
}
private void UpdateWebProxyEnv(RunnerWebProxy webProxy)
{
// Set common forms of proxy variables if configured in Runner and not set directly by container.env
if (!String.IsNullOrEmpty(webProxy.HttpProxyAddress))
{
ContainerEnvironmentVariables.TryAdd("HTTP_PROXY", webProxy.HttpProxyAddress);
ContainerEnvironmentVariables.TryAdd("http_proxy", webProxy.HttpProxyAddress);
}
if (!String.IsNullOrEmpty(webProxy.HttpsProxyAddress))
{
ContainerEnvironmentVariables.TryAdd("HTTPS_PROXY", webProxy.HttpsProxyAddress);
ContainerEnvironmentVariables.TryAdd("https_proxy", webProxy.HttpsProxyAddress);
}
if (!String.IsNullOrEmpty(webProxy.NoProxyString))
{
ContainerEnvironmentVariables.TryAdd("NO_PROXY", webProxy.NoProxyString);
ContainerEnvironmentVariables.TryAdd("no_proxy", webProxy.NoProxyString);
}
}
}
public class MountVolume

View File

@@ -276,9 +276,7 @@ namespace GitHub.Runner.Worker.Container
return await ExecuteDockerCommandAsync(context, "exec", $"{options} {containerId} {command}", context.CancellationToken);
}
#pragma warning disable CS1998 // Async method lacks 'await' operators and will run synchronously (method has async logic on only certain platforms)
public async Task<int> DockerExec(IExecutionContext context, string containerId, string options, string command, List<string> output)
#pragma warning restore CS1998 // Async method lacks 'await' operators and will run synchronously
{
ArgUtil.NotNull(output, nameof(output));
@@ -309,9 +307,10 @@ namespace GitHub.Runner.Worker.Container
}
};
#if OS_WINDOWS || OS_OSX
throw new NotSupportedException($"Container operation is only supported on Linux");
#else
if (!Constants.Runner.Platform.Equals(Constants.OSPlatform.Linux))
{
throw new NotSupportedException("Container operations are only supported on Linux runners");
}
return await processInvoker.ExecuteAsync(
workingDirectory: HostContext.GetDirectory(WellKnownDirectory.Work),
fileName: DockerPath,
@@ -320,7 +319,6 @@ namespace GitHub.Runner.Worker.Container
requireExitCodeZero: false,
outputEncoding: null,
cancellationToken: CancellationToken.None);
#endif
}
public async Task<List<string>> DockerInspect(IExecutionContext context, string dockerObject, string options)
@@ -339,9 +337,7 @@ namespace GitHub.Runner.Worker.Container
return ExecuteDockerCommandAsync(context, command, options, null, cancellationToken);
}
#pragma warning disable CS1998 // Async method lacks 'await' operators and will run synchronously (method has async logic on only certain platforms)
private async Task<int> ExecuteDockerCommandAsync(IExecutionContext context, string command, string options, IDictionary<string, string> environment, EventHandler<ProcessDataReceivedEventArgs> stdoutDataReceived, EventHandler<ProcessDataReceivedEventArgs> stderrDataReceived, CancellationToken cancellationToken = default(CancellationToken))
#pragma warning restore CS1998 // Async method lacks 'await' operators and will run synchronously
{
string arg = $"{command} {options}".Trim();
context.Command($"{DockerPath} {arg}");
@@ -351,9 +347,10 @@ namespace GitHub.Runner.Worker.Container
processInvoker.ErrorDataReceived += stderrDataReceived;
#if OS_WINDOWS || OS_OSX
throw new NotSupportedException($"Container operation is only supported on Linux");
#else
if (!Constants.Runner.Platform.Equals(Constants.OSPlatform.Linux))
{
throw new NotSupportedException("Container operations are only supported on Linux runners");
}
return await processInvoker.ExecuteAsync(
workingDirectory: context.GetGitHubContext("workspace"),
fileName: DockerPath,
@@ -363,12 +360,9 @@ namespace GitHub.Runner.Worker.Container
outputEncoding: null,
killProcessOnCancel: false,
cancellationToken: cancellationToken);
#endif
}
#pragma warning disable CS1998 // Async method lacks 'await' operators and will run synchronously (method has async logic on only certain platforms)
private async Task<int> ExecuteDockerCommandAsync(IExecutionContext context, string command, string options, string workingDirectory, CancellationToken cancellationToken = default(CancellationToken))
#pragma warning restore CS1998 // Async method lacks 'await' operators and will run synchronously
{
string arg = $"{command} {options}".Trim();
context.Command($"{DockerPath} {arg}");
@@ -384,9 +378,10 @@ namespace GitHub.Runner.Worker.Container
context.Output(message.Data);
};
#if OS_WINDOWS || OS_OSX
throw new NotSupportedException($"Container operation is only supported on Linux");
#else
if (!Constants.Runner.Platform.Equals(Constants.OSPlatform.Linux))
{
throw new NotSupportedException("Container operations are only supported on Linux runners");
}
return await processInvoker.ExecuteAsync(
workingDirectory: workingDirectory ?? context.GetGitHubContext("workspace"),
fileName: DockerPath,
@@ -397,7 +392,6 @@ namespace GitHub.Runner.Worker.Container
killProcessOnCancel: false,
redirectStandardIn: null,
cancellationToken: cancellationToken);
#endif
}
private async Task<List<string>> ExecuteDockerCommandAsync(IExecutionContext context, string command, string options)

View File

@@ -35,6 +35,10 @@ namespace GitHub.Runner.Worker
public async Task StartContainersAsync(IExecutionContext executionContext, object data)
{
Trace.Entering();
if (!Constants.Runner.Platform.Equals(Constants.OSPlatform.Linux))
{
throw new NotSupportedException("Container operations are only supported on Linux runners");
}
ArgUtil.NotNull(executionContext, nameof(executionContext));
List<ContainerInfo> containers = data as List<ContainerInfo>;
ArgUtil.NotNull(containers, nameof(containers));
@@ -44,7 +48,7 @@ namespace GitHub.Runner.Worker
displayName: "Stop containers",
data: data);
executionContext.Debug($"Register post job cleanup for stoping/deleting containers.");
executionContext.Debug($"Register post job cleanup for stopping/deleting containers.");
executionContext.RegisterPostJobStep(nameof(StopContainersAsync), postJobStep);
// Check whether we are inside a container.
@@ -125,7 +129,7 @@ namespace GitHub.Runner.Worker
executionContext.Warning($"Delete stale container networks failed, docker network prune fail with exit code {networkPruneExitCode}");
}
// Create local docker network for this job to avoid port conflict when multiple agents run on same machine.
// Create local docker network for this job to avoid port conflict when multiple runners run on same machine.
// All containers within a job join the same network
var containerNetwork = $"github_network_{Guid.NewGuid().ToString("N")}";
await CreateContainerNetworkAsync(executionContext, containerNetwork);

View File

@@ -20,6 +20,7 @@ using System.Text;
using System.Collections;
using ObjectTemplating = GitHub.DistributedTask.ObjectTemplating;
using Pipelines = GitHub.DistributedTask.Pipelines;
using GitHub.DistributedTask.Expressions2;
namespace GitHub.Runner.Worker
{
@@ -568,6 +569,12 @@ namespace GitHub.Runner.Worker
}
}
// Expression functions
if (Variables.GetBoolean("System.HashFilesV2") == true)
{
ExpressionConstants.UpdateFunction<Handlers.HashFiles>("hashFiles", 1, byte.MaxValue);
}
// Expression values
if (message.ContextData?.Count > 0)
{
@@ -610,44 +617,6 @@ namespace GitHub.Runner.Worker
// PostJobSteps for job ExecutionContext
PostJobSteps = new Stack<IStep>();
// // Certificate variables
// var agentCert = HostContext.GetService<IRunnerCertificateManager>();
// if (agentCert.SkipServerCertificateValidation)
// {
// SetRunnerContext("sslskipcertvalidation", bool.TrueString);
// }
// if (!string.IsNullOrEmpty(agentCert.CACertificateFile))
// {
// SetRunnerContext("sslcainfo", agentCert.CACertificateFile);
// }
// if (!string.IsNullOrEmpty(agentCert.ClientCertificateFile) &&
// !string.IsNullOrEmpty(agentCert.ClientCertificatePrivateKeyFile) &&
// !string.IsNullOrEmpty(agentCert.ClientCertificateArchiveFile))
// {
// SetRunnerContext("clientcertfile", agentCert.ClientCertificateFile);
// SetRunnerContext("clientcertprivatekey", agentCert.ClientCertificatePrivateKeyFile);
// SetRunnerContext("clientcertarchive", agentCert.ClientCertificateArchiveFile);
// if (!string.IsNullOrEmpty(agentCert.ClientCertificatePassword))
// {
// HostContext.SecretMasker.AddValue(agentCert.ClientCertificatePassword);
// SetRunnerContext("clientcertpassword", agentCert.ClientCertificatePassword);
// }
// }
// // Runtime option variables
// var runtimeOptions = HostContext.GetService<IConfigurationStore>().GetRunnerRuntimeOptions();
// if (runtimeOptions != null)
// {
// #if OS_WINDOWS
// if (runtimeOptions.GitUseSecureChannel)
// {
// SetRunnerContext("gituseschannel", runtimeOptions.GitUseSecureChannel.ToString());
// }
// #endif
// }
// Job timeline record.
InitializeTimelineRecord(

View File

@@ -0,0 +1,126 @@
using System;
using System.IO;
using GitHub.DistributedTask.Expressions2.Sdk;
using GitHub.DistributedTask.Pipelines.ContextData;
using GitHub.DistributedTask.Pipelines.ObjectTemplating;
using GitHub.Runner.Sdk;
using System.Reflection;
using System.Threading;
using System.Collections.Generic;
namespace GitHub.Runner.Worker.Handlers
{
public class FunctionTrace : ITraceWriter
{
private GitHub.DistributedTask.Expressions2.ITraceWriter _trace;
public FunctionTrace(GitHub.DistributedTask.Expressions2.ITraceWriter trace)
{
_trace = trace;
}
public void Info(string message)
{
_trace.Info(message);
}
public void Verbose(string message)
{
_trace.Info(message);
}
}
public sealed class HashFiles : Function
{
protected sealed override Object EvaluateCore(
EvaluationContext context,
out ResultMemory resultMemory)
{
resultMemory = null;
var templateContext = context.State as DistributedTask.ObjectTemplating.TemplateContext;
ArgUtil.NotNull(templateContext, nameof(templateContext));
templateContext.ExpressionValues.TryGetValue(PipelineTemplateConstants.GitHub, out var githubContextData);
ArgUtil.NotNull(githubContextData, nameof(githubContextData));
var githubContext = githubContextData as DictionaryContextData;
ArgUtil.NotNull(githubContext, nameof(githubContext));
githubContext.TryGetValue(PipelineTemplateConstants.Workspace, out var workspace);
var workspaceData = workspace as StringContextData;
ArgUtil.NotNull(workspaceData, nameof(workspaceData));
string githubWorkspace = workspaceData.Value;
bool followSymlink = false;
List<string> patterns = new List<string>();
var firstParameter = true;
foreach (var parameter in Parameters)
{
var parameterString = parameter.Evaluate(context).ConvertToString();
if (firstParameter)
{
firstParameter = false;
if (parameterString.StartsWith("--"))
{
if (string.Equals(parameterString, "--follow-symbolic-links", StringComparison.OrdinalIgnoreCase))
{
followSymlink = true;
continue;
}
else
{
throw new ArgumentOutOfRangeException($"Invalid glob option {parameterString}, avaliable option: '--follow-symbolic-links'.");
}
}
}
patterns.Add(parameterString);
}
context.Trace.Info($"Search root directory: '{githubWorkspace}'");
context.Trace.Info($"Search pattern: '{string.Join(", ", patterns)}'");
string binDir = Path.GetDirectoryName(Assembly.GetEntryAssembly().Location);
string runnerRoot = new DirectoryInfo(binDir).Parent.FullName;
string node = Path.Combine(runnerRoot, "externals", "node12", "bin", $"node{IOUtil.ExeExtension}");
string hashFilesScript = Path.Combine(binDir, "hashFiles");
var hashResult = string.Empty;
var p = new ProcessInvoker(new FunctionTrace(context.Trace));
p.ErrorDataReceived += ((_, data) =>
{
if (!string.IsNullOrEmpty(data.Data) && data.Data.StartsWith("__OUTPUT__") && data.Data.EndsWith("__OUTPUT__"))
{
hashResult = data.Data.Substring(10, data.Data.Length - 20);
context.Trace.Info($"Hash result: '{hashResult}'");
}
else
{
context.Trace.Info(data.Data);
}
});
p.OutputDataReceived += ((_, data) =>
{
context.Trace.Info(data.Data);
});
var env = new Dictionary<string, string>();
if (followSymlink)
{
env["followSymbolicLinks"] = "true";
}
env["patterns"] = string.Join(Environment.NewLine, patterns);
int exitCode = p.ExecuteAsync(workingDirectory: githubWorkspace,
fileName: node,
arguments: $"\"{hashFilesScript.Replace("\"", "\\\"")}\"",
environment: env,
requireExitCodeZero: false,
cancellationToken: new CancellationTokenSource(TimeSpan.FromSeconds(120)).Token).GetAwaiter().GetResult();
if (exitCode != 0)
{
throw new InvalidOperationException($"hashFiles('{ExpressionUtility.StringEscape(string.Join(", ", patterns))}') failed. Fail to hash files under directory '{githubWorkspace}'");
}
return hashResult;
}
}
}

View File

@@ -16,6 +16,8 @@ namespace GitHub.Runner.Worker
"head_ref",
"ref",
"repository",
"run_id",
"run_number",
"sha",
"workflow",
"workspace",

View File

@@ -189,13 +189,13 @@ namespace GitHub.Runner.Worker.Handlers
container.ContainerEnvironmentVariables[variable.Key] = container.TranslateToContainerPath(variable.Value);
}
using (var stdoutManager = new OutputManager(ExecutionContext, ActionCommandManager))
using (var stderrManager = new OutputManager(ExecutionContext, ActionCommandManager))
using (var stdoutManager = new OutputManager(ExecutionContext, ActionCommandManager, container))
using (var stderrManager = new OutputManager(ExecutionContext, ActionCommandManager, container))
{
var runExitCode = await dockerManger.DockerRun(ExecutionContext, container, stdoutManager.OnDataReceived, stderrManager.OnDataReceived);
ExecutionContext.Debug($"Docker Action run completed with exit code {runExitCode}");
if (runExitCode != 0)
{
ExecutionContext.Error($"Docker run failed with exit code {runExitCode}");
ExecutionContext.Result = TaskResult.Failed;
}
}

View File

@@ -62,7 +62,7 @@ namespace GitHub.Runner.Worker.Handlers
}
else if (data.ExecutionType == ActionExecutionType.Plugin)
{
// Agent plugin
// Runner plugin
handler = HostContext.CreateService<IRunnerPluginHandler>();
(handler as IRunnerPluginHandler).Data = data as PluginActionExecutionData;
}

View File

@@ -122,9 +122,9 @@ namespace GitHub.Runner.Worker.Handlers
else
{
var exitCode = await step;
ExecutionContext.Debug($"Node Action run completed with exit code {exitCode}");
if (exitCode != 0)
{
ExecutionContext.Error($"Node run failed with exit code {exitCode}");
ExecutionContext.Result = TaskResult.Failed;
}
}

View File

@@ -6,6 +6,7 @@ using System.Linq;
using System.Text.RegularExpressions;
using GitHub.Runner.Common;
using GitHub.Runner.Sdk;
using GitHub.Runner.Worker.Container;
using DTWebApi = GitHub.DistributedTask.WebApi;
namespace GitHub.Runner.Worker.Handlers
@@ -17,6 +18,7 @@ namespace GitHub.Runner.Worker.Handlers
private const string _timeoutKey = "GITHUB_ACTIONS_RUNNER_ISSUE_MATCHER_TIMEOUT";
private static readonly Regex _colorCodeRegex = new Regex(@"\x0033\[[0-9;]*m?", RegexOptions.Compiled | RegexOptions.CultureInvariant);
private readonly IActionCommandManager _commandManager;
private readonly ContainerInfo _container;
private readonly IExecutionContext _executionContext;
private readonly int _failsafe = 50;
private readonly object _matchersLock = new object();
@@ -25,10 +27,11 @@ namespace GitHub.Runner.Worker.Handlers
// Mapping that indicates whether a directory belongs to the workflow repository
private readonly Dictionary<string, string> _directoryMap = new Dictionary<string, string>();
public OutputManager(IExecutionContext executionContext, IActionCommandManager commandManager)
public OutputManager(IExecutionContext executionContext, IActionCommandManager commandManager, ContainerInfo container = null)
{
_executionContext = executionContext;
_commandManager = commandManager;
_container = container ?? executionContext.Container;
// Recursion failsafe (test override)
var failsafeString = Environment.GetEnvironmentVariable("RUNNER_TEST_GET_REPOSITORY_PATH_FAILSAFE");
@@ -257,6 +260,7 @@ namespace GitHub.Runner.Worker.Handlers
if (!string.IsNullOrWhiteSpace(match.File))
{
var file = match.File;
var translate = _container != null;
// Root using fromPath
if (!string.IsNullOrWhiteSpace(match.FromPath) && !Path.IsPathFullyQualified(file))
@@ -275,11 +279,19 @@ namespace GitHub.Runner.Worker.Handlers
ArgUtil.NotNullOrEmpty(workspace, "workspace");
file = Path.Combine(workspace, file);
translate = false;
}
// Remove relative pathing and normalize slashes
file = Path.GetFullPath(file);
// Translate to host
if (translate)
{
file = _container.TranslateToHostPath(file);
file = Path.GetFullPath(file);
}
// Check whether the file exists
if (File.Exists(file))
{

View File

@@ -2,6 +2,7 @@
using System.IO;
using System.Text;
using System.Threading.Tasks;
using System.Linq;
using GitHub.DistributedTask.Pipelines.ContextData;
using GitHub.Runner.Common;
using GitHub.Runner.Sdk;
@@ -56,6 +57,7 @@ namespace GitHub.Runner.Worker.Handlers
string shellCommand;
string shellCommandPath = null;
bool validateShellOnHost = !(StepHost is ContainerStepHost);
string prependPath = string.Join(Path.PathSeparator.ToString(), ExecutionContext.PrependPath.Reverse<string>());
Inputs.TryGetValue("shell", out var shell);
if (string.IsNullOrEmpty(shell))
{
@@ -63,19 +65,19 @@ namespace GitHub.Runner.Worker.Handlers
shellCommand = "pwsh";
if(validateShellOnHost)
{
shellCommandPath = WhichUtil.Which(shellCommand, require: false, Trace);
shellCommandPath = WhichUtil.Which(shellCommand, require: false, Trace, prependPath);
if (string.IsNullOrEmpty(shellCommandPath))
{
shellCommand = "powershell";
Trace.Info($"Defaulting to {shellCommand}");
shellCommandPath = WhichUtil.Which(shellCommand, require: true, Trace);
shellCommandPath = WhichUtil.Which(shellCommand, require: true, Trace, prependPath);
}
}
#else
shellCommand = "sh";
if (validateShellOnHost)
{
shellCommandPath = WhichUtil.Which("bash") ?? WhichUtil.Which("sh", true, Trace);
shellCommandPath = WhichUtil.Which("bash", false, Trace, prependPath) ?? WhichUtil.Which("sh", true, Trace, prependPath);
}
#endif
argFormat = ScriptHandlerHelpers.GetScriptArgumentsFormat(shellCommand);
@@ -86,7 +88,7 @@ namespace GitHub.Runner.Worker.Handlers
shellCommand = parsed.shellCommand;
if (validateShellOnHost)
{
shellCommandPath = WhichUtil.Which(parsed.shellCommand, true, Trace);
shellCommandPath = WhichUtil.Which(parsed.shellCommand, true, Trace, prependPath);
}
argFormat = $"{parsed.shellArgs}".TrimStart();
@@ -144,23 +146,24 @@ namespace GitHub.Runner.Worker.Handlers
Inputs.TryGetValue("shell", out var shell);
var isContainerStepHost = StepHost is ContainerStepHost;
string prependPath = string.Join(Path.PathSeparator.ToString(), ExecutionContext.PrependPath.Reverse<string>());
string commandPath, argFormat, shellCommand;
// Set up default command and arguments
if (string.IsNullOrEmpty(shell))
{
#if OS_WINDOWS
shellCommand = "pwsh";
commandPath = WhichUtil.Which(shellCommand, require: false, Trace);
commandPath = WhichUtil.Which(shellCommand, require: false, Trace, prependPath);
if (string.IsNullOrEmpty(commandPath))
{
shellCommand = "powershell";
Trace.Info($"Defaulting to {shellCommand}");
commandPath = WhichUtil.Which(shellCommand, require: true, Trace);
commandPath = WhichUtil.Which(shellCommand, require: true, Trace, prependPath);
}
ArgUtil.NotNullOrEmpty(commandPath, "Default Shell");
#else
shellCommand = "sh";
commandPath = WhichUtil.Which("bash", false, Trace) ?? WhichUtil.Which("sh", true, Trace);
commandPath = WhichUtil.Which("bash", false, Trace, prependPath) ?? WhichUtil.Which("sh", true, Trace, prependPath);
#endif
argFormat = ScriptHandlerHelpers.GetScriptArgumentsFormat(shellCommand);
}
@@ -169,7 +172,7 @@ namespace GitHub.Runner.Worker.Handlers
var parsed = ScriptHandlerHelpers.ParseShellOptionString(shell);
shellCommand = parsed.shellCommand;
// For non-ContainerStepHost, the command must be located on the host by Which
commandPath = WhichUtil.Which(parsed.shellCommand, !isContainerStepHost, Trace);
commandPath = WhichUtil.Which(parsed.shellCommand, !isContainerStepHost, Trace, prependPath);
argFormat = $"{parsed.shellArgs}".TrimStart();
if (string.IsNullOrEmpty(argFormat))
{

View File

@@ -141,6 +141,13 @@ namespace GitHub.Runner.Worker.Handlers
executionContext.Debug(line);
if (line.ToLower().Contains("alpine"))
{
if (!Constants.Runner.PlatformArchitecture.Equals(Constants.Architecture.X64))
{
var os = Constants.Runner.Platform.ToString();
var arch = Constants.Runner.PlatformArchitecture.ToString();
var msg = $"JavaScript Actions in Alpine containers are only supported on x64 Linux runners. Detected {os} {arch}";
throw new NotSupportedException(msg);
}
nodeExternal = "node12_alpine";
executionContext.Output($"Container distribution is alpine. Running JavaScript Action with external tool: {nodeExternal}");
return nodeExternal;

View File

@@ -107,6 +107,11 @@ namespace GitHub.Runner.Worker
return await CompleteJobAsync(jobServer, jobContext, message, TaskResult.Failed);
}
if (jobContext.WriteDebug)
{
jobContext.SetRunnerContext("debug", "1");
}
jobContext.SetRunnerContext("os", VarUtil.OS);
string toolsDirectory = HostContext.GetDirectory(WellKnownDirectory.Tools);

View File

@@ -1,7 +1,7 @@
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFramework>netcoreapp3.0</TargetFramework>
<TargetFramework>netcoreapp3.1</TargetFramework>
<OutputType>Exe</OutputType>
<RuntimeIdentifiers>win-x64;win-x86;linux-x64;linux-arm64;linux-arm;osx-x64</RuntimeIdentifiers>
<TargetLatestRuntimePatch>true</TargetLatestRuntimePatch>

View File

@@ -76,10 +76,33 @@ namespace GitHub.Runner.Worker
// Start
step.ExecutionContext.Start();
// Set GITHUB_ACTION
// Populate env context for each step
Trace.Info("Initialize Env context for step");
#if OS_WINDOWS
var envContext = new DictionaryContextData();
#else
var envContext = new CaseSensitiveDictionaryContextData();
#endif
step.ExecutionContext.ExpressionValues["env"] = envContext;
foreach (var pair in step.ExecutionContext.EnvironmentVariables)
{
envContext[pair.Key] = new StringContextData(pair.Value ?? string.Empty);
}
if (step is IActionRunner actionStep)
{
// Set GITHUB_ACTION
step.ExecutionContext.SetGitHubContext("action", actionStep.Action.Name);
// Evaluate and merge action's env block to env context
var templateTrace = step.ExecutionContext.ToTemplateTraceWriter();
var schema = new PipelineTemplateSchemaFactory().CreateSchema();
var templateEvaluator = new PipelineTemplateEvaluator(templateTrace, schema);
var actionEnvironment = templateEvaluator.EvaluateStepEnvironment(actionStep.Action.Environment, step.ExecutionContext.ExpressionValues, VarUtil.EnvironmentVariableKeyComparer);
foreach (var env in actionEnvironment)
{
envContext[env.Key] = new StringContextData(env.Value ?? string.Empty);
}
}
// Initialize scope

View File

@@ -2,7 +2,6 @@
using System.Collections.Concurrent;
using System.Collections.Generic;
using System.Linq;
using GitHub.Build.WebApi;
using GitHub.DistributedTask.WebApi;
using GitHub.DistributedTask.Logging;
using GitHub.DistributedTask.Pipelines.ContextData;
@@ -63,7 +62,7 @@ namespace GitHub.Runner.Worker
// DO NOT add file path variable to here.
// All file path variables needs to be retrive and set through ExecutionContext, so it can handle container file path translation.
public string Build_Number => Get(BuildVariables.BuildNumber);
public string Build_Number => Get(SdkConstants.Variables.Build.BuildNumber);
#if OS_WINDOWS
public bool Retain_Default_Encoding => false;

View File

@@ -40,8 +40,7 @@ namespace GitHub.Runner.Worker
// Validate args.
ArgUtil.NotNullOrEmpty(pipeIn, nameof(pipeIn));
ArgUtil.NotNullOrEmpty(pipeOut, nameof(pipeOut));
var runnerCertManager = HostContext.GetService<IRunnerCertificateManager>();
VssUtil.InitializeVssClientSettings(HostContext.UserAgent, HostContext.WebProxy, runnerCertManager.VssClientCertificateManager);
VssUtil.InitializeVssClientSettings(HostContext.UserAgent, HostContext.WebProxy);
var jobRunner = HostContext.CreateService<IJobRunner>();
using (var channel = HostContext.CreateService<IProcessChannel>())

View File

@@ -1,14 +0,0 @@
using System;
using GitHub.Services.Common;
namespace GitHub.Build.WebApi
{
public static class ArtifactResourceTypes
{
/// <summary>
/// Build container reference
/// E.g. #/2121/drop
/// </summary>
public const String Container = "Container";
}
}

View File

@@ -1,61 +0,0 @@
using System;
using System.Collections.Generic;
using System.Globalization;
using System.Linq;
using System.Net.Http;
using System.Threading;
using System.Threading.Tasks;
using GitHub.Services.Common;
using GitHub.Services.WebApi;
using GitHub.Services.WebApi.Patch;
using GitHub.Services.WebApi.Patch.Json;
namespace GitHub.Build.WebApi
{
public class BuildHttpClient : BuildHttpClientBase
{
static BuildHttpClient()
{
}
public BuildHttpClient(
Uri baseUrl,
VssCredentials credentials)
: base(baseUrl, credentials)
{
}
public BuildHttpClient(
Uri baseUrl,
VssCredentials credentials,
VssHttpRequestSettings settings)
: base(baseUrl, credentials, settings)
{
}
public BuildHttpClient(
Uri baseUrl,
VssCredentials credentials,
params DelegatingHandler[] handlers)
: base(baseUrl, credentials, handlers)
{
}
public BuildHttpClient(
Uri baseUrl,
VssCredentials credentials,
VssHttpRequestSettings settings,
params DelegatingHandler[] handlers)
: base(baseUrl, credentials, settings, handlers)
{
}
public BuildHttpClient(
Uri baseUrl,
HttpMessageHandler pipeline,
Boolean disposeHandler)
: base(baseUrl, pipeline, disposeHandler)
{
}
}
}

Some files were not shown because too many files have changed in this diff Show More