Compare commits

...

26 Commits

Author SHA1 Message Date
eric sciple
0b1d4de9b2 . 2020-01-13 15:15:06 -05:00
eric sciple
3baa5bc3e9 . 2020-01-13 15:08:35 -05:00
eric sciple
4c1a129cf8 . 2020-01-13 13:50:24 -05:00
eric sciple
9c1db9ef23 translate problem matcher file to host path 2020-01-13 13:27:58 -05:00
eric sciple
a727194742 allow container to be null/empty (#266) 2020-01-12 23:02:36 -05:00
eric sciple
a9c58d7398 Handle escaped '%' in commands data section (#200) 2020-01-12 03:30:26 -05:00
Bryan MacFarlane
e15414eb5e proxy support ADR (#263) 2020-01-09 11:52:04 -05:00
Tingluo Huang
4ab1e645c3 Fix typo in error message. (#260) 2020-01-07 15:13:05 -05:00
Tingluo Huang
584f6b6ca3 upload log on runner force kill worker. (#255) 2020-01-06 13:04:23 -05:00
Tingluo Huang
abc65839f3 detect source file path without using env. (#257) 2020-01-06 12:56:15 -05:00
Tingluo Huang
06292aa118 Expose whether debug is on/off via RUNNER_DEBUG. (#253) 2020-01-03 21:23:46 -05:00
Joseph Petersen
ac1a076a3b Treat warnings as errors (#249)
* Treat warnings as errors

* fix warnings
2019-12-21 09:51:41 -05:00
Joseph Petersen
300bc67950 ignore .idea folder (#248)
created by rider
2019-12-21 09:50:07 -05:00
Tim Heuer
289c7f36a2 Minor typo (#247)
capitalize .NET
2019-12-20 20:20:44 -05:00
Julio Barba
b89d7fb8ef Remove old "v1" artifact download/publish code (#212)
* Remove old v1 artifact download/publish code
* Remove the Build2 REST API SDK
2019-12-19 16:02:00 -05:00
Josh Gross
5fd705bb84 Update LICENSE (#242) 2019-12-19 15:22:49 -05:00
Julio Barba
9e37732401 Verify that has Windows service started successfully (#236) 2019-12-19 14:34:26 -05:00
Bryan MacFarlane
6c70d53eea clean up some unneeded dockerfiles 2019-12-19 09:16:43 -05:00
Bryan MacFarlane
f791e2d512 update contributions md 2019-12-19 09:03:14 -05:00
Bryan MacFarlane
f1e36651ad build workflow ignores md changes 2019-12-19 08:54:43 -05:00
Bryan MacFarlane
be24fea81b update issue templates 2019-12-19 08:34:17 -05:00
Bryan MacFarlane
84ca2c05ce update readme 2019-12-19 08:20:39 -05:00
Bryan MacFarlane
2249560cec update readme 2019-12-18 23:15:30 -05:00
Bryan MacFarlane
2d4b821abe update readme 2019-12-18 23:13:22 -05:00
Bryan MacFarlane
371bf8e607 update readme and contributions 2019-12-18 22:49:31 -05:00
Tingluo Huang
9ba11da490 move .sln file. (#238) 2019-12-18 20:13:57 -05:00
49 changed files with 644 additions and 1627 deletions

View File

@@ -1,10 +0,0 @@
## Runner Version and Platform
Version of your runner?
OS of the machine running the runner? OSX/Windows/Linux/...
## What's not working?
Please include error messages and screenshots.
## Runner and Worker's Diagnostic Logs
Logs are located in the runner's `_diag` folder. The runner logs are prefixed with `Runner_` and the worker logs are prefixed with `Worker_`. All sensitive information should already be masked out, but please double-check before pasting here.

34
.github/ISSUE_TEMPLATE/bug_report.md vendored Normal file
View File

@@ -0,0 +1,34 @@
---
name: Bug report
about: Create a report to help us improve
title: ''
labels: bug
assignees: ''
---
**Describe the bug**
A clear and concise description of what the bug is.
**To Reproduce**
Steps to reproduce the behavior:
1. Go to '...'
2. Run '....'
3. See error
**Expected behavior**
A clear and concise description of what you expected to happen.
## Runner Version and Platform
Version of your runner?
OS of the machine running the runner? OSX/Windows/Linux/...
## What's not working?
Please include error messages and screenshots.
## Job Log Output
If applicable, include the relevant part of the job / step log output here. All sensitive information should already be masked out, but please double-check before pasting here.
## Runner and Worker's Diagnostic Logs
If applicable, add relevant diagnostic log information. Logs are located in the runner's `_diag` folder. The runner logs are prefixed with `Runner_` and the worker logs are prefixed with `Worker_`. Each job run correlates to a worker log. All sensitive information should already be masked out, but please double-check before pasting here.

View File

@@ -0,0 +1,27 @@
---
name: Feature Request
about: Create a request to help us improve
title: ''
labels: enhancement
assignees: ''
---
Thank you 🙇‍♀ for wanting to create a feature in this repository. Before you do, please ensure you are filing the issue in the right place. Issues should only be opened on if the issue **relates to code in this repository**.
* If you have found a security issue [please submit it here](https://hackerone.com/github)
* If you have questions or issues with the service, writing workflows or actions, then please [visit the GitHub Community Forum's Actions Board](https://github.community/t5/GitHub-Actions/bd-p/actions)
* If you are having an issue or question about GitHub Actions then please [contact customer support](https://help.github.com/en/actions/automating-your-workflow-with-github-actions/about-github-actions#contacting-support)
If you have a feature request that is relevant to this repository, the runner, then please include the information below:
**Describe the enhancement**
A clear and concise description of what the features or enhancement you need.
**Code Snippet**
If applicable, add a code snippet.
**Additional information**
Add any other context about the feature here.
NOTE: if the feature request has been agreed upon then the assignee will create an ADR. See docs/adrs/README.md

View File

@@ -5,9 +5,13 @@ on:
branches:
- master
- releases/*
paths-ignore:
- '**.md'
pull_request:
branches:
- '*'
paths-ignore:
- '**.md'
jobs:
build:

7
.gitignore vendored
View File

@@ -1,12 +1,18 @@
# build output
**/bin
**/obj
**/libs
# editors
**/*.xproj
**/*.xproj.user
**/.vs
**/.vscode
**/*.error
**/*.json.pretty
.idea/
# output
node_modules
_downloads
_layout
@@ -19,4 +25,3 @@ TestLogs
#generated
src/Runner.Sdk/BuildConstants.cs

View File

@@ -1,5 +1,5 @@
The MIT License (MIT)
Copyright (c) Microsoft Corporation
Copyright (c) 2019 GitHub
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
@@ -17,4 +17,4 @@ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
SOFTWARE.

View File

@@ -1,31 +1,25 @@
# GitHub Actions Runner
<p align="center">
<img src="docs/res/github-graph.png">
</p>
# GitHub Actions Runner
[![Actions Status](https://github.com/actions/runner/workflows/Runner%20CI/badge.svg)](https://github.com/actions/runner/actions)
The runner is the application that runs a job from a GitHub Actions workflow. The runner can run on the [hosted machine pools](https://github.com/actions/virtual-environments) or run on [self-hosted environments](https://help.github.com/en/actions/automating-your-workflow-with-github-actions/about-self-hosted-runners).
## Get Started
![win](docs/res/win_sm.png) [Pre-reqs](docs/start/envwin.md) | [Download](https://github.com/actions/runner/releases/latest)
For more information about installing and using self-hosted runners, see [Adding self-hosted runners](https://help.github.com/en/actions/automating-your-workflow-with-github-actions/adding-self-hosted-runners) and [Using self-hosted runners in a workflow](https://help.github.com/en/actions/automating-your-workflow-with-github-actions/using-self-hosted-runners-in-a-workflow)
![macOS](docs/res/apple_sm.png) [Pre-reqs](docs/start/envosx.md) | [Download](https://github.com/actions/runner/releases/latest)
Runner releases:
![linux](docs/res/linux_sm.png) [Pre-reqs](docs/start/envlinux.md) | [Download](https://github.com/actions/runner/releases/latest)
![win](docs/res/win_sm.png) [Pre-reqs](docs/start/envwin.md) | [Download](https://github.com/actions/runner/releases)
**Configure:**
![macOS](docs/res/apple_sm.png) [Pre-reqs](docs/start/envosx.md) | [Download](https://github.com/actions/runner/releases)
*MacOS and Linux*
```bash
./config.sh
```
*Windows*
```bash
config.cmd
```
![linux](docs/res/linux_sm.png) [Pre-reqs](docs/start/envlinux.md) | [Download](https://github.com/actions/runner/releases)
## Contribute
For developers that want to contribute, [read here](docs/contribute.md) on how to build and test.
We accept contributions in the form of issues and pull requests. [Read more here](docs/contribute.md) before contributing.

View File

@@ -0,0 +1,61 @@
# ADR 263: Self Hosted Runner Proxies
**Date**: 2019-11-13
**Status**: Accepted
## Context
- Proxy support is required for some enterprises and organizations to start using their own self hosted runners
- While there is not a standard convention, many applications support setting proxies via the environmental variables `http_proxy`, `https_proxy`, `no_proxy`, such as curl, wget, perl, python, docker, git, R, ect
- Some of these applications use `HTTPS_PROXY` versus `https_proxy`, but most understand or primarily support the lowercase variant
## Decisions
We will update the Runner to use the conventional environment variables for proxies: `http_proxy`, `https_proxy` and `no_proxy` if they are set.
These are described in detail below:
- `https_proxy` a proxy URL for all https traffic. It may contain basic authentication credentials. For example:
- http://proxy.com
- http://127.0.0.1:8080
- http://user:password@proxy.com
- `http_proxy` a proxy URL for all http traffic. It may contain basic authentication credentials. For example:
- http://proxy.com
- http://127.0.0.1:8080
- http://user:password@proxy.com
- `no_proxy` a comma seperated list of hosts that should not use the proxy. An optional port may be specified
- `google.com`
- `yahoo.com:443`
- `google.com,bing.com`
We won't use `http_proxy` for https traffic when `https_proxy` is not set, this behavior lines up with any libcurl based tools (curl, git) and wget.
Otherwise action authors and workflow users need to adjust to differences between the runner proxy convention, and tools used by their actions and scripts.
Example:
Customer set `http_proxy=http://127.0.0.1:8888` and configure the runner against `https://github.com/owner/repo`, with the `https_proxy` -> `http_proxy` fallback, the runner will connect to server without any problem. However, if user runs `git push` to `https://github.com/owner/repo`, `git` won't use the proxy since it require `https_proxy` to be set for any https traffic.
> `golang`, `node.js` and other dev tools from the linux community use `http_proxy` for both http and https traffic base on my research.
A majority of our users are using Linux where these variables are commonly required to be set by various programs. By reading these values, we simplify the process for self hosted runners to set up proxy, and expose it in a way users are already familiar with.
A password provided for a proxy will be masked in the logs.
We will support the lowercase and uppercase variants, with lowercase taking priority if both are set.
### No Proxy Format
While exact implementations are different per application on handle `no_proxy` env, most applications accept a comma separated list of hosts. Some accept wildcard characters (*). We are going to do exact case-insentive matches, and not support wildcards at this time.
For example:
- example.com will match example.com, foo.example.com, foo.bar.example.com
- foo.example.com will match bar.foo.example.com and foo.example.com
We will not support IP addresses for `no_proxy`, only hostnames.
## Consequences
1. Enterprises and organizations needing proxy support will be able to embrace self hosted runners
2. Users will need to set these environmental variables before configuring the runner in order to use a proxy when configuring
3. The runner will read from the environmental variables during config and runtime and use the provided proxy if it exists
4. Users may need to pass these environmental variables into other applications if they do not natively take these variables
5. Action authors may need to update their workflows to react to the these environment variables
6. We will document the way of setting environmental variables for runners using the environmental variables and how the runner uses them
7. Like all other secrets, users will be able to relatively easily figure out proxy password if they can modify a workflow file running on a self hosted machine

19
docs/adrs/README.md Normal file
View File

@@ -0,0 +1,19 @@
# ADRs
ADR, short for "Architecture Decision Record" is a way of capturing important architectural decisions, along with their context and consequences.
This folder includes ADRs for the actions runner. ADRs are proposed in the form of a pull request, and they commonly follow this format:
* **Title**: short present tense imperative phrase, less than 50 characters, like a git commit message.
* **Status**: proposed, accepted, rejected, deprecated, superseded, etc.
* **Context**: what is the issue that we're seeing that is motivating this decision or change.
* **Decision**: what is the change that we're actually proposing or doing.
* **Consequences**: what becomes easier or more difficult to do because of this change.
---
- More information about ADRs can be found [here](https://github.com/joelparkerhenderson/architecture_decision_record).

View File

@@ -1,10 +1,31 @@
# Contribution guide for developers
# Contributions
## Required Dev Dependencies
We welcome contributions in the form of issues and pull requests. We view the contributions and the process as the same for github and external contributors.
> IMPORTANT: Building your own runner is critical for the dev inner loop process when contributing changes. However, only runners built and distributed by GitHub (releases) are supported in production. Be aware that workflows and orchestrations run service side with the runner being a remote process to run steps. For that reason, the service can pull the runner forward so customizations can be lost.
## Issues
Log issues for both bugs and enhancement requests. Logging issues are important for the open community.
Issues in this repository should be for the runner application. Note that the VM and virtual machine images (including the developer toolsets) installed on the actions hosted machine pools are located [in this repository](https://github.com/actions/virtual-environments)
## Enhancements and Feature Requests
We ask that before significant effort is put into code changes, that we have agreement on taking the change before time is invested in code changes.
1. Create a feature request. Once agreed we will take the enhancment
2. Create an ADR to agree on the details of the change.
An ADR is an Architectural Decision Record. This allows consensus on the direction forward and also serves as a record of the change and motivation. [Read more here](adrs/README.md)
## Development Life Cycle
### Required Dev Dependencies
![Win](res/win_sm.png) Git for Windows [Install Here](https://git-scm.com/downloads) (needed for dev sh script)
## To Build, Test, Layout
### To Build, Test, Layout
Navigate to the `src` directory and run the following command:
@@ -29,12 +50,12 @@ cd ./src
./dev.(sh/cmd) test # run all unit tests before git commit/push
```
## Editors
### Editors
[Using Visual Studio 2019](https://www.visualstudio.com/vs/)
[Using Visual Studio Code](https://code.visualstudio.com/)
[Using Visual Studio 2019](https://www.visualstudio.com/vs/)
## Styling
### Styling
We use the .NET Foundation and CoreCLR style guidelines [located here](
https://github.com/dotnet/corefx/blob/master/Documentation/coding-guidelines/coding-style.md)

View File

@@ -28,7 +28,7 @@ Execute ./bin/installdependencies.sh to install any missing Dotnet Core 3.0 depe
```
You can easily correct the problem by executing `./bin/installdependencies.sh`.
The `installdependencies.sh` script should install all required dependencies on all supported Linux versions
> Note: The `installdependencies.sh` script will try to use the default package management mechanism on your Linux flavor (ex. `yum`/`apt-get`/`apt`). You might need to deal with error coming from the package management mechanism related to your setup, like [#1353](https://github.com/Microsoft/vsts-agent/issues/1353)
> Note: The `installdependencies.sh` script will try to use the default package management mechanism on your Linux flavor (ex. `yum`/`apt-get`/`apt`).
### Full dependencies list

View File

@@ -9,4 +9,4 @@
- Windows Server 2016 64-bit
- Windows Server 2019 64-bit
## [More .Net Core Prerequisites Information](https://docs.microsoft.com/en-us/dotnet/core/windows-prerequisites?tabs=netcore30)
## [More .NET Core Prerequisites Information](https://docs.microsoft.com/en-us/dotnet/core/windows-prerequisites?tabs=netcore30)

View File

@@ -1,7 +0,0 @@
FROM mcr.microsoft.com/dotnet/core/runtime-deps:2.1
RUN apt-get update \
&& apt-get install -y --no-install-recommends \
curl \
git \
&& rm -rf /var/lib/apt/lists/*

View File

@@ -1,150 +0,0 @@
FROM centos:6
# Install dependencies
RUN yum install -y \
centos-release-SCL \
epel-release \
wget \
unzip \
&& \
rpm --import http://linuxsoft.cern.ch/cern/slc6X/x86_64/RPM-GPG-KEY-cern && \
wget -O /etc/yum.repos.d/slc6-devtoolset.repo http://linuxsoft.cern.ch/cern/devtoolset/slc6-devtoolset.repo && \
yum install -y \
"perl(Time::HiRes)" \
autoconf \
cmake \
cmake3 \
devtoolset-2-toolchain \
doxygen \
expat-devel \
gcc \
gcc-c++ \
gdb \
gettext-devel \
krb5-devel \
libedit-devel \
libidn-devel \
libmetalink-devel \
libnghttp2-devel \
libssh2-devel \
libunwind-devel \
libuuid-devel \
lttng-ust-devel \
lzma \
ncurses-devel \
openssl-devel \
perl-devel \
python-argparse \
python27 \
readline-devel \
swig \
xz \
zlib-devel \
&& \
yum clean all
# Build and install clang and lldb 3.9.1
RUN wget ftp://sourceware.org/pub/binutils/snapshots/binutils-2.29.1.tar.xz && \
wget http://releases.llvm.org/3.9.1/cfe-3.9.1.src.tar.xz && \
wget http://releases.llvm.org/3.9.1/llvm-3.9.1.src.tar.xz && \
wget http://releases.llvm.org/3.9.1/lldb-3.9.1.src.tar.xz && \
wget http://releases.llvm.org/3.9.1/compiler-rt-3.9.1.src.tar.xz && \
\
tar -xf binutils-2.29.1.tar.xz && \
tar -xf llvm-3.9.1.src.tar.xz && \
mkdir llvm-3.9.1.src/tools/clang && \
mkdir llvm-3.9.1.src/tools/lldb && \
mkdir llvm-3.9.1.src/projects/compiler-rt && \
tar -xf cfe-3.9.1.src.tar.xz --strip 1 -C llvm-3.9.1.src/tools/clang && \
tar -xf lldb-3.9.1.src.tar.xz --strip 1 -C llvm-3.9.1.src/tools/lldb && \
tar -xf compiler-rt-3.9.1.src.tar.xz --strip 1 -C llvm-3.9.1.src/projects/compiler-rt && \
rm binutils-2.29.1.tar.xz && \
rm cfe-3.9.1.src.tar.xz && \
rm lldb-3.9.1.src.tar.xz && \
rm llvm-3.9.1.src.tar.xz && \
rm compiler-rt-3.9.1.src.tar.xz && \
\
mkdir llvmbuild && \
cd llvmbuild && \
scl enable python27 devtoolset-2 \
' \
cmake3 \
-DCMAKE_CXX_COMPILER=/opt/rh/devtoolset-2/root/usr/bin/g++ \
-DCMAKE_C_COMPILER=/opt/rh/devtoolset-2/root/usr/bin/gcc \
-DCMAKE_LINKER=/opt/rh/devtoolset-2/root/usr/bin/ld \
-DCMAKE_BUILD_TYPE=Release \
-DLLVM_LIBDIR_SUFFIX=64 \
-DLLVM_ENABLE_EH=1 \
-DLLVM_ENABLE_RTTI=1 \
-DLLVM_BINUTILS_INCDIR=../binutils-2.29.1/include \
../llvm-3.9.1.src \
&& \
make -j $(($(getconf _NPROCESSORS_ONLN)+1)) && \
make install \
' && \
cd .. && \
rm -r llvmbuild && \
rm -r llvm-3.9.1.src && \
rm -r binutils-2.29.1
# Build and install curl 7.45.0
RUN wget https://curl.haxx.se/download/curl-7.45.0.tar.lzma && \
tar -xf curl-7.45.0.tar.lzma && \
rm curl-7.45.0.tar.lzma && \
cd curl-7.45.0 && \
scl enable python27 devtoolset-2 \
' \
./configure \
--disable-dict \
--disable-ftp \
--disable-gopher \
--disable-imap \
--disable-ldap \
--disable-ldaps \
--disable-libcurl-option \
--disable-manual \
--disable-pop3 \
--disable-rtsp \
--disable-smb \
--disable-smtp \
--disable-telnet \
--disable-tftp \
--enable-ipv6 \
--enable-optimize \
--enable-symbol-hiding \
--with-ca-bundle=/etc/pki/tls/certs/ca-bundle.crt \
--with-nghttp2 \
--with-gssapi \
--with-ssl \
--without-librtmp \
&& \
make install \
' && \
cd .. && \
rm -r curl-7.45.0
# Install ICU 57.1
RUN wget http://download.icu-project.org/files/icu4c/57.1/icu4c-57_1-RHEL6-x64.tgz && \
tar -xf icu4c-57_1-RHEL6-x64.tgz -C / && \
rm icu4c-57_1-RHEL6-x64.tgz
# Compile and install a version of the git that supports the features that cli repo build needs
# NOTE: The git needs to be built after the curl so that it can use the libcurl to add https
# protocol support.
RUN \
wget https://www.kernel.org/pub/software/scm/git/git-2.9.5.tar.gz && \
tar -xf git-2.9.5.tar.gz && \
rm git-2.9.5.tar.gz && \
cd git-2.9.5 && \
make configure && \
./configure --prefix=/usr/local --without-tcltk && \
make -j $(nproc --all) all && \
make install && \
cd .. && \
rm -r git-2.9.5
ENV LD_LIBRARY_PATH=/usr/local/lib

View File

@@ -3,23 +3,23 @@ Microsoft Visual Studio Solution File, Format Version 12.00
# Visual Studio Version 16
VisualStudioVersion = 16.0.29411.138
MinimumVisualStudioVersion = 10.0.40219.1
Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "Runner.Common", "src\Runner.Common\Runner.Common.csproj", "{084289A3-CD7A-42E0-9219-4348B4B7E19B}"
Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "Runner.Common", "Runner.Common\Runner.Common.csproj", "{084289A3-CD7A-42E0-9219-4348B4B7E19B}"
EndProject
Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "Runner.Listener", "src\Runner.Listener\Runner.Listener.csproj", "{7D461AEE-BF2A-4855-BD96-56921160B36A}"
Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "Runner.Listener", "Runner.Listener\Runner.Listener.csproj", "{7D461AEE-BF2A-4855-BD96-56921160B36A}"
EndProject
Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "Runner.PluginHost", "src\Runner.PluginHost\Runner.PluginHost.csproj", "{D0320EB1-CB6D-4179-BFDC-2F2B664A370C}"
Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "Runner.PluginHost", "Runner.PluginHost\Runner.PluginHost.csproj", "{D0320EB1-CB6D-4179-BFDC-2F2B664A370C}"
EndProject
Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "Runner.Plugins", "src\Runner.Plugins\Runner.Plugins.csproj", "{C23AFD6F-4DCD-4243-BC61-865BE31B9168}"
Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "Runner.Plugins", "Runner.Plugins\Runner.Plugins.csproj", "{C23AFD6F-4DCD-4243-BC61-865BE31B9168}"
EndProject
Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "Runner.Sdk", "src\Runner.Sdk\Runner.Sdk.csproj", "{D0484633-DA97-4C34-8E47-1DADE212A57A}"
Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "Runner.Sdk", "Runner.Sdk\Runner.Sdk.csproj", "{D0484633-DA97-4C34-8E47-1DADE212A57A}"
EndProject
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "RunnerService", "src\Runner.Service\Windows\RunnerService.csproj", "{D12EBD71-0464-46D0-8394-40BCFBA0A6F2}"
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "RunnerService", "Runner.Service\Windows\RunnerService.csproj", "{D12EBD71-0464-46D0-8394-40BCFBA0A6F2}"
EndProject
Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "Runner.Worker", "src\Runner.Worker\Runner.Worker.csproj", "{C2F5B9FA-2621-411F-8EB2-273ED276F503}"
Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "Runner.Worker", "Runner.Worker\Runner.Worker.csproj", "{C2F5B9FA-2621-411F-8EB2-273ED276F503}"
EndProject
Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "Sdk", "src\Sdk\Sdk.csproj", "{D2EE812B-E4DF-49BB-AE87-12BC49949B5F}"
Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "Sdk", "Sdk\Sdk.csproj", "{D2EE812B-E4DF-49BB-AE87-12BC49949B5F}"
EndProject
Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "Test", "src\Test\Test.csproj", "{C932061F-F6A1-4F1E-B854-A6C6B30DC3EF}"
Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "Test", "Test\Test.csproj", "{C932061F-F6A1-4F1E-B854-A6C6B30DC3EF}"
EndProject
Global
GlobalSection(SolutionConfigurationPlatforms) = preSolution

View File

@@ -46,4 +46,9 @@
<PropertyGroup Condition="'$(Configuration)' == 'Debug'">
<DefineConstants>$(DefineConstants);DEBUG</DefineConstants>
</PropertyGroup>
<!-- Set Treat tarnings as errors -->
<PropertyGroup>
<TreatWarningsAsErrors>true</TreatWarningsAsErrors>
</PropertyGroup>
</Project>

View File

@@ -9,26 +9,27 @@ namespace GitHub.Runner.Common
{
private static readonly EscapeMapping[] _escapeMappings = new[]
{
new EscapeMapping(token: "%", replacement: "%25"),
new EscapeMapping(token: ";", replacement: "%3B"),
new EscapeMapping(token: "\r", replacement: "%0D"),
new EscapeMapping(token: "\n", replacement: "%0A"),
new EscapeMapping(token: "]", replacement: "%5D"),
new EscapeMapping(token: "%", replacement: "%25"),
};
private static readonly EscapeMapping[] _escapeDataMappings = new[]
{
new EscapeMapping(token: "\r", replacement: "%0D"),
new EscapeMapping(token: "\n", replacement: "%0A"),
new EscapeMapping(token: "%", replacement: "%25"),
};
private static readonly EscapeMapping[] _escapePropertyMappings = new[]
{
new EscapeMapping(token: "%", replacement: "%25"),
new EscapeMapping(token: "\r", replacement: "%0D"),
new EscapeMapping(token: "\n", replacement: "%0A"),
new EscapeMapping(token: ":", replacement: "%3A"),
new EscapeMapping(token: ",", replacement: "%2C"),
new EscapeMapping(token: "%", replacement: "%25"),
};
private readonly Dictionary<string, string> _properties = new Dictionary<string, string>(StringComparer.OrdinalIgnoreCase);

View File

@@ -24,7 +24,6 @@ namespace GitHub.Runner.Common
private Guid _timelineId;
private Guid _timelineRecordId;
private string _pageId;
private FileStream _pageData;
private StreamWriter _pageWriter;
private int _byteCount;
@@ -40,7 +39,6 @@ namespace GitHub.Runner.Common
{
base.Initialize(hostContext);
_totalLines = 0;
_pageId = Guid.NewGuid().ToString();
_pagesFolder = Path.Combine(hostContext.GetDirectory(WellKnownDirectory.Diag), PagingFolder);
_jobServerQueue = HostContext.GetService<IJobServerQueue>();
Directory.CreateDirectory(_pagesFolder);
@@ -102,7 +100,7 @@ namespace GitHub.Runner.Common
{
EndPage();
_byteCount = 0;
_dataFileName = Path.Combine(_pagesFolder, $"{_pageId}_{++_pageCount}.log");
_dataFileName = Path.Combine(_pagesFolder, $"{_timelineRecordId}_{++_pageCount}.log");
_pageData = new FileStream(_dataFileName, FileMode.CreateNew);
_pageWriter = new StreamWriter(_pageData, System.Text.Encoding.UTF8);
}

View File

@@ -678,6 +678,17 @@ namespace GitHub.Runner.Listener.Configuration
if (service != null)
{
service.Start();
try
{
_term.WriteLine("Waiting for service to start...");
service.WaitForStatus(ServiceControllerStatus.Running, TimeSpan.FromSeconds(60));
}
catch (System.ServiceProcess.TimeoutException)
{
throw new InvalidOperationException($"Cannot start the service {serviceName} in a timely fashion.");
}
_term.WriteLine($"Service {serviceName} started successfully");
}
else

View File

@@ -568,6 +568,10 @@ namespace GitHub.Runner.Listener
{
Trace.Info("worker process has been killed.");
}
// When worker doesn't exit within cancel timeout, the runner will kill the worker process and worker won't finish upload job logs.
// The runner will try to upload these logs at this time.
await TryUploadUnfinishedLogs(message);
}
Trace.Info($"finish job request for job {message.JobId} with result: {resultOnAbandonOrCancel}");
@@ -712,6 +716,116 @@ namespace GitHub.Runner.Listener
}
}
// Best effort upload any logs for this job.
private async Task TryUploadUnfinishedLogs(Pipelines.AgentJobRequestMessage message)
{
Trace.Entering();
var logFolder = Path.Combine(HostContext.GetDirectory(WellKnownDirectory.Diag), PagingLogger.PagingFolder);
if (!Directory.Exists(logFolder))
{
return;
}
var logs = Directory.GetFiles(logFolder);
if (logs.Length == 0)
{
return;
}
try
{
var systemConnection = message.Resources.Endpoints.SingleOrDefault(x => string.Equals(x.Name, WellKnownServiceEndpointNames.SystemVssConnection));
ArgUtil.NotNull(systemConnection, nameof(systemConnection));
var jobServer = HostContext.GetService<IJobServer>();
VssCredentials jobServerCredential = VssUtil.GetVssCredential(systemConnection);
VssConnection jobConnection = VssUtil.CreateConnection(systemConnection.Url, jobServerCredential);
await jobServer.ConnectAsync(jobConnection);
var timeline = await jobServer.GetTimelineAsync(message.Plan.ScopeIdentifier, message.Plan.PlanType, message.Plan.PlanId, message.Timeline.Id, CancellationToken.None);
var updatedRecords = new List<TimelineRecord>();
var logPages = new Dictionary<Guid, Dictionary<int, string>>();
var logRecords = new Dictionary<Guid, TimelineRecord>();
foreach (var log in logs)
{
var logName = Path.GetFileNameWithoutExtension(log);
var logPageSeperator = logName.IndexOf('_');
var logRecordId = Guid.Empty;
var pageNumber = 0;
if (logPageSeperator < 0)
{
Trace.Warning($"log file '{log}' doesn't follow naming convension 'GUID_INT'.");
continue;
}
else
{
if (!Guid.TryParse(logName.Substring(0, logPageSeperator), out logRecordId))
{
Trace.Warning($"log file '{log}' doesn't follow naming convension 'GUID_INT'.");
continue;
}
if (!int.TryParse(logName.Substring(logPageSeperator + 1), out pageNumber))
{
Trace.Warning($"log file '{log}' doesn't follow naming convension 'GUID_INT'.");
continue;
}
}
var record = timeline.Records.FirstOrDefault(x => x.Id == logRecordId);
if (record != null)
{
if (!logPages.ContainsKey(record.Id))
{
logPages[record.Id] = new Dictionary<int, string>();
logRecords[record.Id] = record;
}
logPages[record.Id][pageNumber] = log;
}
}
foreach (var pages in logPages)
{
var record = logRecords[pages.Key];
if (record.Log == null)
{
// Create the log
record.Log = await jobServer.CreateLogAsync(message.Plan.ScopeIdentifier, message.Plan.PlanType, message.Plan.PlanId, new TaskLog(String.Format(@"logs\{0:D}", record.Id)), default(CancellationToken));
// Need to post timeline record updates to reflect the log creation
updatedRecords.Add(record.Clone());
}
for (var i = 1; i <= pages.Value.Count; i++)
{
var logFile = pages.Value[i];
// Upload the contents
using (FileStream fs = File.Open(logFile, FileMode.Open, FileAccess.Read, FileShare.ReadWrite))
{
var logUploaded = await jobServer.AppendLogContentAsync(message.Plan.ScopeIdentifier, message.Plan.PlanType, message.Plan.PlanId, record.Log.Id, fs, default(CancellationToken));
}
Trace.Info($"Uploaded unfinished log '{logFile}' for current job.");
IOUtil.DeleteFile(logFile);
}
}
if (updatedRecords.Count > 0)
{
await jobServer.UpdateTimelineRecordsAsync(message.Plan.ScopeIdentifier, message.Plan.PlanType, message.Plan.PlanId, message.Timeline.Id, updatedRecords, CancellationToken.None);
}
}
catch (Exception ex)
{
// Ignore any error during log upload since it's best effort
Trace.Error(ex);
}
}
// TODO: We need send detailInfo back to DT in order to add an issue for the job
private async Task CompleteJobRequestAsync(int poolId, Pipelines.AgentJobRequestMessage message, Guid lockToken, TaskResult result, string detailInfo = null)
{

View File

@@ -1,58 +0,0 @@
using System;
using System.Collections.Generic;
using System.Threading;
using System.Threading.Tasks;
using GitHub.Runner.Sdk;
using GitHub.Services.WebApi;
using GitHub.Build.WebApi;
namespace GitHub.Runner.Plugins.Artifact
{
// A client wrapper interacting with Build's Artifact API
public class BuildServer
{
private readonly BuildHttpClient _buildHttpClient;
public BuildServer(VssConnection connection)
{
ArgUtil.NotNull(connection, nameof(connection));
_buildHttpClient = connection.GetClient<BuildHttpClient>();
}
// Associate the specified artifact with a build, along with custom data.
public async Task<BuildArtifact> AssociateArtifact(
Guid projectId,
int pipelineId,
string jobId,
string name,
string type,
string data,
Dictionary<string, string> propertiesDictionary,
CancellationToken cancellationToken = default(CancellationToken))
{
BuildArtifact artifact = new BuildArtifact()
{
Name = name,
Source = jobId,
Resource = new ArtifactResource()
{
Data = data,
Type = type,
Properties = propertiesDictionary
}
};
return await _buildHttpClient.CreateArtifactAsync(artifact, projectId, pipelineId, cancellationToken: cancellationToken);
}
// Get named artifact from a build
public async Task<BuildArtifact> GetArtifact(
Guid projectId,
int pipelineId,
string name,
CancellationToken cancellationToken)
{
return await _buildHttpClient.GetArtifactAsync(projectId, pipelineId, name, cancellationToken: cancellationToken);
}
}
}

View File

@@ -3,7 +3,6 @@ using System.Collections.Generic;
using System.IO;
using System.Threading;
using System.Threading.Tasks;
using GitHub.Build.WebApi;
using GitHub.Services.Common;
using GitHub.Runner.Sdk;
@@ -40,70 +39,31 @@ namespace GitHub.Runner.Plugins.Artifact
targetPath = Path.IsPathFullyQualified(targetPath) ? targetPath : Path.GetFullPath(Path.Combine(defaultWorkingDirectory, targetPath));
// Project ID
Guid projectId = new Guid(context.Variables.GetValueOrDefault(BuildVariables.TeamProjectId)?.Value ?? Guid.Empty.ToString());
// Build ID
string buildIdStr = context.Variables.GetValueOrDefault(BuildVariables.BuildId)?.Value ?? string.Empty;
string buildIdStr = context.Variables.GetValueOrDefault(SdkConstants.Variables.Build.BuildId)?.Value ?? string.Empty;
if (!int.TryParse(buildIdStr, out int buildId))
{
throw new ArgumentException($"Run Id is not an Int32: {buildIdStr}");
}
// Determine whether to call Pipelines or Build endpoint to publish artifact based on variable setting
string usePipelinesArtifactEndpointVar = context.Variables.GetValueOrDefault("Runner.UseActionsArtifactsApis")?.Value;
bool.TryParse(usePipelinesArtifactEndpointVar, out bool usePipelinesArtifactEndpoint);
string containerPath;
long containerId;
context.Output($"Downloading artifact '{artifactName}' to: '{targetPath}'");
if (usePipelinesArtifactEndpoint)
// Definition ID is a dummy value only used by HTTP client routing purposes
int definitionId = 1;
var pipelinesHelper = new PipelinesServer(context.VssConnection);
var actionsStorageArtifact = await pipelinesHelper.GetActionsStorageArtifact(definitionId, buildId, artifactName, token);
if (actionsStorageArtifact == null)
{
context.Debug("Downloading artifact using v2 endpoint");
// Definition ID is a dummy value only used by HTTP client routing purposes
int definitionId = 1;
var pipelinesHelper = new PipelinesServer(context.VssConnection);
var actionsStorageArtifact = await pipelinesHelper.GetActionsStorageArtifact(definitionId, buildId, artifactName, token);
if (actionsStorageArtifact == null)
{
throw new Exception($"The actions storage artifact for '{artifactName}' could not be found, or is no longer available");
}
containerPath = actionsStorageArtifact.Name; // In actions storage artifacts, name equals the path
containerId = actionsStorageArtifact.ContainerId;
}
else
{
context.Debug("Downloading artifact using v1 endpoint");
BuildServer buildHelper = new BuildServer(context.VssConnection);
BuildArtifact buildArtifact = await buildHelper.GetArtifact(projectId, buildId, artifactName, token);
if (string.Equals(buildArtifact.Resource.Type, "Container", StringComparison.OrdinalIgnoreCase) ||
// Artifact was published by Pipelines endpoint, check new type here to handle rollback scenario
string.Equals(buildArtifact.Resource.Type, "Actions_Storage", StringComparison.OrdinalIgnoreCase))
{
string containerUrl = buildArtifact.Resource.Data;
string[] parts = containerUrl.Split(new[] { '/' }, 3);
if (parts.Length < 3 || !long.TryParse(parts[1], out containerId))
{
throw new ArgumentOutOfRangeException($"Invalid container url '{containerUrl}' for artifact '{buildArtifact.Name}'");
}
containerPath = parts[2];
}
else
{
throw new NotSupportedException($"Invalid artifact type: {buildArtifact.Resource.Type}");
}
throw new Exception($"The actions storage artifact for '{artifactName}' could not be found, or is no longer available");
}
FileContainerServer fileContainerServer = new FileContainerServer(context.VssConnection, projectId, containerId, containerPath);
string containerPath = actionsStorageArtifact.Name; // In actions storage artifacts, name equals the path
long containerId = actionsStorageArtifact.ContainerId;
FileContainerServer fileContainerServer = new FileContainerServer(context.VssConnection, projectId: new Guid(), containerId, containerPath);
await fileContainerServer.DownloadFromContainerAsync(context, targetPath, token);
context.Output("Artifact download finished.");

View File

@@ -4,9 +4,7 @@ using System.IO;
using System.Linq;
using System.Threading;
using System.Threading.Tasks;
using GitHub.Build.WebApi;
using GitHub.Services.Common;
using GitHub.DistributedTask.WebApi;
using GitHub.Runner.Sdk;
namespace GitHub.Runner.Plugins.Artifact
@@ -45,11 +43,8 @@ namespace GitHub.Runner.Plugins.Artifact
throw new ArgumentException($"Artifact name is not valid: {artifactName}. It cannot contain '\\', '/', \"', ':', '<', '>', '|', '*', and '?'");
}
// Project ID
Guid projectId = new Guid(context.Variables.GetValueOrDefault(BuildVariables.TeamProjectId)?.Value ?? Guid.Empty.ToString());
// Build ID
string buildIdStr = context.Variables.GetValueOrDefault(BuildVariables.BuildId)?.Value ?? string.Empty;
string buildIdStr = context.Variables.GetValueOrDefault(SdkConstants.Variables.Build.BuildId)?.Value ?? string.Empty;
if (!int.TryParse(buildIdStr, out int buildId))
{
throw new ArgumentException($"Run Id is not an Int32: {buildIdStr}");
@@ -65,7 +60,7 @@ namespace GitHub.Runner.Plugins.Artifact
}
// Container ID
string containerIdStr = context.Variables.GetValueOrDefault(BuildVariables.ContainerId)?.Value ?? string.Empty;
string containerIdStr = context.Variables.GetValueOrDefault(SdkConstants.Variables.Build.ContainerId)?.Value ?? string.Empty;
if (!long.TryParse(containerIdStr, out long containerId))
{
throw new ArgumentException($"Container Id is not an Int64: {containerIdStr}");
@@ -73,7 +68,7 @@ namespace GitHub.Runner.Plugins.Artifact
context.Output($"Uploading artifact '{artifactName}' from '{fullPath}' for run #{buildId}");
FileContainerServer fileContainerHelper = new FileContainerServer(context.VssConnection, projectId, containerId, artifactName);
FileContainerServer fileContainerHelper = new FileContainerServer(context.VssConnection, projectId: Guid.Empty, containerId, artifactName);
var propertiesDictionary = new Dictionary<string, string>(StringComparer.OrdinalIgnoreCase);
long size = 0;
@@ -89,38 +84,20 @@ namespace GitHub.Runner.Plugins.Artifact
// if any of the results were successful, make sure to attach them to the build
finally
{
// Determine whether to call Pipelines or Build endpoint to publish artifact based on variable setting
string usePipelinesArtifactEndpointVar = context.Variables.GetValueOrDefault("Runner.UseActionsArtifactsApis")?.Value;
bool.TryParse(usePipelinesArtifactEndpointVar, out bool usePipelinesArtifactEndpoint);
// Definition ID is a dummy value only used by HTTP client routing purposes
int definitionId = 1;
if (usePipelinesArtifactEndpoint)
{
// Definition ID is a dummy value only used by HTTP client routing purposes
int definitionId = 1;
PipelinesServer pipelinesHelper = new PipelinesServer(context.VssConnection);
PipelinesServer pipelinesHelper = new PipelinesServer(context.VssConnection);
var artifact = await pipelinesHelper.AssociateActionsStorageArtifactAsync(
definitionId,
buildId,
containerId,
artifactName,
size,
token);
var artifact = await pipelinesHelper.AssociateActionsStorageArtifactAsync(
definitionId,
buildId,
containerId,
artifactName,
size,
token);
context.Output($"Associated artifact {artifactName} ({artifact.ContainerId}) with run #{buildId}");
context.Debug($"Associated artifact using v2 endpoint");
}
else
{
string fileContainerFullPath = StringUtil.Format($"#/{containerId}/{artifactName}");
BuildServer buildHelper = new BuildServer(context.VssConnection);
string jobId = context.Variables.GetValueOrDefault(WellKnownDistributedTaskVariables.JobId).Value ?? string.Empty;
var artifact = await buildHelper.AssociateArtifact(projectId, buildId, jobId, artifactName, ArtifactResourceTypes.Container, fileContainerFullPath, propertiesDictionary, token);
context.Output($"Associated artifact {artifactName} ({artifact.Id}) with run #{buildId}");
context.Debug($"Associated artifact using v1 endpoint");
}
context.Output($"Associated artifact {artifactName} ({artifact.ContainerId}) with run #{buildId}");
}
}
}

View File

@@ -0,0 +1,19 @@
using System;
namespace GitHub.Runner.Sdk
{
public class SdkConstants
{
public static class Variables
{
public static class Build
{
// Legacy "build" variables historically used by the runner
// DO NOT add new variables here -- instead use either the Actions or Runner namespaces
public const String BuildId = "build.buildId";
public const String BuildNumber = "build.buildNumber";
public const String ContainerId = "build.containerId";
}
}
}
}

View File

@@ -116,7 +116,7 @@ namespace GitHub.Runner.Worker
if (actionDefinition.Execution == null)
{
executionContext.Debug($"Loaded action.yml file: {StringUtil.ConvertToJson(actionDefinition)}");
throw new ArgumentException($"Top level 'run:' section is required for {manifestFile}");
throw new ArgumentException($"Top level 'runs:' section is required for {manifestFile}");
}
else
{

View File

@@ -2,9 +2,9 @@
using System.Collections.Generic;
using System.IO;
using GitHub.Runner.Common.Util;
using Pipelines = GitHub.DistributedTask.Pipelines;
using GitHub.Runner.Common;
using GitHub.Runner.Sdk;
using Pipelines = GitHub.DistributedTask.Pipelines;
namespace GitHub.Runner.Worker.Container
{
@@ -19,7 +19,6 @@ namespace GitHub.Runner.Worker.Container
public ContainerInfo()
{
}
public ContainerInfo(IHostContext hostContext, Pipelines.JobContainer container, bool isJobContainer = true, string networkAlias = null)

View File

@@ -189,8 +189,8 @@ namespace GitHub.Runner.Worker.Handlers
container.ContainerEnvironmentVariables[variable.Key] = container.TranslateToContainerPath(variable.Value);
}
using (var stdoutManager = new OutputManager(ExecutionContext, ActionCommandManager))
using (var stderrManager = new OutputManager(ExecutionContext, ActionCommandManager))
using (var stdoutManager = new OutputManager(ExecutionContext, ActionCommandManager, container))
using (var stderrManager = new OutputManager(ExecutionContext, ActionCommandManager, container))
{
var runExitCode = await dockerManger.DockerRun(ExecutionContext, container, stdoutManager.OnDataReceived, stderrManager.OnDataReceived);
if (runExitCode != 0)

View File

@@ -6,6 +6,7 @@ using System.Linq;
using System.Text.RegularExpressions;
using GitHub.Runner.Common;
using GitHub.Runner.Sdk;
using GitHub.Runner.Worker.Container;
using DTWebApi = GitHub.DistributedTask.WebApi;
namespace GitHub.Runner.Worker.Handlers
@@ -17,6 +18,7 @@ namespace GitHub.Runner.Worker.Handlers
private const string _timeoutKey = "GITHUB_ACTIONS_RUNNER_ISSUE_MATCHER_TIMEOUT";
private static readonly Regex _colorCodeRegex = new Regex(@"\x0033\[[0-9;]*m?", RegexOptions.Compiled | RegexOptions.CultureInvariant);
private readonly IActionCommandManager _commandManager;
private readonly ContainerInfo _container;
private readonly IExecutionContext _executionContext;
private readonly int _failsafe = 50;
private readonly object _matchersLock = new object();
@@ -25,10 +27,11 @@ namespace GitHub.Runner.Worker.Handlers
// Mapping that indicates whether a directory belongs to the workflow repository
private readonly Dictionary<string, string> _directoryMap = new Dictionary<string, string>();
public OutputManager(IExecutionContext executionContext, IActionCommandManager commandManager)
public OutputManager(IExecutionContext executionContext, IActionCommandManager commandManager, ContainerInfo container = null)
{
_executionContext = executionContext;
_commandManager = commandManager;
_container = container ?? executionContext.Container;
// Recursion failsafe (test override)
var failsafeString = Environment.GetEnvironmentVariable("RUNNER_TEST_GET_REPOSITORY_PATH_FAILSAFE");
@@ -257,6 +260,7 @@ namespace GitHub.Runner.Worker.Handlers
if (!string.IsNullOrWhiteSpace(match.File))
{
var file = match.File;
var translate = _container != null;
// Root using fromPath
if (!string.IsNullOrWhiteSpace(match.FromPath) && !Path.IsPathFullyQualified(file))
@@ -275,11 +279,19 @@ namespace GitHub.Runner.Worker.Handlers
ArgUtil.NotNullOrEmpty(workspace, "workspace");
file = Path.Combine(workspace, file);
translate = false;
}
// Remove relative pathing and normalize slashes
file = Path.GetFullPath(file);
// Translate to host
if (translate)
{
file = _container.TranslateToHostPath(file);
file = Path.GetFullPath(file);
}
// Check whether the file exists
if (File.Exists(file))
{

View File

@@ -107,6 +107,11 @@ namespace GitHub.Runner.Worker
return await CompleteJobAsync(jobServer, jobContext, message, TaskResult.Failed);
}
if (jobContext.WriteDebug)
{
jobContext.SetRunnerContext("debug", "1");
}
jobContext.SetRunnerContext("os", VarUtil.OS);
string toolsDirectory = HostContext.GetDirectory(WellKnownDirectory.Tools);

View File

@@ -2,7 +2,6 @@
using System.Collections.Concurrent;
using System.Collections.Generic;
using System.Linq;
using GitHub.Build.WebApi;
using GitHub.DistributedTask.WebApi;
using GitHub.DistributedTask.Logging;
using GitHub.DistributedTask.Pipelines.ContextData;
@@ -63,7 +62,7 @@ namespace GitHub.Runner.Worker
// DO NOT add file path variable to here.
// All file path variables needs to be retrive and set through ExecutionContext, so it can handle container file path translation.
public string Build_Number => Get(BuildVariables.BuildNumber);
public string Build_Number => Get(SdkConstants.Variables.Build.BuildNumber);
#if OS_WINDOWS
public bool Retain_Default_Encoding => false;

View File

@@ -1,14 +0,0 @@
using System;
using GitHub.Services.Common;
namespace GitHub.Build.WebApi
{
public static class ArtifactResourceTypes
{
/// <summary>
/// Build container reference
/// E.g. #/2121/drop
/// </summary>
public const String Container = "Container";
}
}

View File

@@ -1,61 +0,0 @@
using System;
using System.Collections.Generic;
using System.Globalization;
using System.Linq;
using System.Net.Http;
using System.Threading;
using System.Threading.Tasks;
using GitHub.Services.Common;
using GitHub.Services.WebApi;
using GitHub.Services.WebApi.Patch;
using GitHub.Services.WebApi.Patch.Json;
namespace GitHub.Build.WebApi
{
public class BuildHttpClient : BuildHttpClientBase
{
static BuildHttpClient()
{
}
public BuildHttpClient(
Uri baseUrl,
VssCredentials credentials)
: base(baseUrl, credentials)
{
}
public BuildHttpClient(
Uri baseUrl,
VssCredentials credentials,
VssHttpRequestSettings settings)
: base(baseUrl, credentials, settings)
{
}
public BuildHttpClient(
Uri baseUrl,
VssCredentials credentials,
params DelegatingHandler[] handlers)
: base(baseUrl, credentials, handlers)
{
}
public BuildHttpClient(
Uri baseUrl,
VssCredentials credentials,
VssHttpRequestSettings settings,
params DelegatingHandler[] handlers)
: base(baseUrl, credentials, settings, handlers)
{
}
public BuildHttpClient(
Uri baseUrl,
HttpMessageHandler pipeline,
Boolean disposeHandler)
: base(baseUrl, pipeline, disposeHandler)
{
}
}
}

View File

@@ -1,10 +0,0 @@
using System;
namespace GitHub.Build.WebApi
{
public static class BuildResourceIds
{
public const String AreaId = "5D6898BB-45EC-463F-95F9-54D49C71752E";
public const String AreaName = "build";
}
}

View File

@@ -1,13 +0,0 @@
using System;
namespace GitHub.Build.WebApi
{
public static class BuildVariables
{
public const String TeamProjectId = "system.teamProjectId";
public const String BuildId = "build.buildId";
public const String BuildNumber = "build.buildNumber";
public const String ContainerId = "build.containerId";
}
}

View File

@@ -1,54 +0,0 @@
using System;
using System.Collections.Generic;
using System.Runtime.Serialization;
using GitHub.Services.WebApi;
namespace GitHub.Build.WebApi
{
[DataContract]
public class ArtifactResource : BaseSecuredObject
{
public ArtifactResource()
{
}
public ArtifactResource(
ISecuredObject securedObject)
: base(securedObject)
{
}
/// <summary>
/// The type of the resource: File container, version control folder, UNC path, etc.
/// </summary>
[DataMember(EmitDefaultValue = false)]
public String Type
{
get;
set;
}
/// <summary>
/// Type-specific data about the artifact.
/// </summary>
/// <remarks>
/// For example, "#/10002/5/drop", "$/drops/5", "\\myshare\myfolder\mydrops\5"
/// </remarks>
[DataMember(EmitDefaultValue = false)]
public String Data
{
get;
set;
}
/// <summary>
/// Type-specific properties of the artifact.
/// </summary>
[DataMember(IsRequired = false, EmitDefaultValue = false)]
public Dictionary<String, String> Properties
{
get;
set;
}
}
}

View File

@@ -1,63 +0,0 @@
using System;
using System.Runtime.Serialization;
using GitHub.Services.WebApi;
namespace GitHub.Build.WebApi
{
/// <summary>
/// Represents an artifact produced by a build.
/// </summary>
[DataContract]
public class BuildArtifact : BaseSecuredObject
{
public BuildArtifact()
{
}
internal BuildArtifact(
ISecuredObject securedObject)
: base(securedObject)
{
}
/// <summary>
/// The artifact ID.
/// </summary>
[DataMember]
public Int32 Id
{
get;
set;
}
/// <summary>
/// The name of the artifact.
/// </summary>
[DataMember]
public String Name
{
get;
set;
}
/// <summary>
/// The artifact source, which will be the ID of the job that produced this artifact.
/// </summary>
[DataMember]
public String Source
{
get;
set;
}
/// <summary>
/// The actual resource.
/// </summary>
[DataMember]
public ArtifactResource Resource
{
get;
set;
}
}
}

View File

@@ -1,107 +0,0 @@
/*
* ---------------------------------------------------------
* Copyright(C) Microsoft Corporation. All rights reserved.
* ---------------------------------------------------------
*/
using System;
using System.Collections.Generic;
using System.Net.Http;
using System.Threading;
using System.Threading.Tasks;
using GitHub.Services.Common;
using GitHub.Services.WebApi;
namespace GitHub.Build.WebApi
{
[ResourceArea(BuildResourceIds.AreaId)]
public abstract class BuildHttpClientBase : VssHttpClientBase
{
public BuildHttpClientBase(Uri baseUrl, VssCredentials credentials)
: base(baseUrl, credentials)
{
}
public BuildHttpClientBase(Uri baseUrl, VssCredentials credentials, VssHttpRequestSettings settings)
: base(baseUrl, credentials, settings)
{
}
public BuildHttpClientBase(Uri baseUrl, VssCredentials credentials, params DelegatingHandler[] handlers)
: base(baseUrl, credentials, handlers)
{
}
public BuildHttpClientBase(Uri baseUrl, VssCredentials credentials, VssHttpRequestSettings settings, params DelegatingHandler[] handlers)
: base(baseUrl, credentials, settings, handlers)
{
}
public BuildHttpClientBase(Uri baseUrl, HttpMessageHandler pipeline, bool disposeHandler)
: base(baseUrl, pipeline, disposeHandler)
{
}
/// <summary>
/// [Preview API] Associates an artifact with a build.
/// </summary>
/// <param name="artifact">The artifact.</param>
/// <param name="project">Project ID</param>
/// <param name="buildId">The ID of the build.</param>
/// <param name="userState"></param>
/// <param name="cancellationToken">The cancellation token to cancel operation.</param>
public virtual Task<BuildArtifact> CreateArtifactAsync(
BuildArtifact artifact,
Guid project,
int buildId,
object userState = null,
CancellationToken cancellationToken = default)
{
HttpMethod httpMethod = new HttpMethod("POST");
Guid locationId = new Guid("1db06c96-014e-44e1-ac91-90b2d4b3e984");
object routeValues = new { project = project, buildId = buildId };
HttpContent content = new ObjectContent<BuildArtifact>(artifact, new VssJsonMediaTypeFormatter(true));
return SendAsync<BuildArtifact>(
httpMethod,
locationId,
routeValues: routeValues,
version: new ApiResourceVersion(5.2, 5),
userState: userState,
cancellationToken: cancellationToken,
content: content);
}
/// <summary>
/// [Preview API] Gets a specific artifact for a build.
/// </summary>
/// <param name="project">Project ID</param>
/// <param name="buildId">The ID of the build.</param>
/// <param name="artifactName">The name of the artifact.</param>
/// <param name="userState"></param>
/// <param name="cancellationToken">The cancellation token to cancel operation.</param>
public virtual Task<BuildArtifact> GetArtifactAsync(
Guid project,
int buildId,
string artifactName,
object userState = null,
CancellationToken cancellationToken = default)
{
HttpMethod httpMethod = new HttpMethod("GET");
Guid locationId = new Guid("1db06c96-014e-44e1-ac91-90b2d4b3e984");
object routeValues = new { project = project, buildId = buildId };
List<KeyValuePair<string, string>> queryParams = new List<KeyValuePair<string, string>>();
queryParams.Add("artifactName", artifactName);
return SendAsync<BuildArtifact>(
httpMethod,
locationId,
routeValues: routeValues,
version: new ApiResourceVersion(5.2, 5),
queryParameters: queryParams,
userState: userState,
cancellationToken: cancellationToken);
}
}
}

View File

@@ -154,6 +154,11 @@ namespace GitHub.DistributedTask.Pipelines.ObjectTemplating
if (value is StringToken containerLiteral)
{
if (String.IsNullOrEmpty(containerLiteral.Value))
{
return null;
}
result.Image = containerLiteral.Value;
}
else

View File

@@ -472,7 +472,7 @@
"matrix"
],
"one-of": [
"non-empty-string",
"string",
"container-mapping"
]
},
@@ -497,10 +497,22 @@
],
"mapping": {
"loose-key-type": "non-empty-string",
"loose-value-type": "container"
"loose-value-type": "services-container"
}
},
"services-container": {
"context": [
"github",
"strategy",
"matrix"
],
"one-of": [
"non-empty-string",
"container-mapping"
]
},
"container-env": {
"mapping": {
"loose-key-type": "non-empty-string",

View File

@@ -1,344 +0,0 @@
namespace AsyncFixer
{
}
namespace GitHub.Actions.Pipelines.WebApi
{
}
namespace GitHub.Actions.Pipelines.WebApi.Contracts
{
}
namespace GitHub.Build.WebApi
{
}
namespace GitHub.DistributedTask.Expressions
{
}
namespace GitHub.DistributedTask.Expressions2
{
}
namespace GitHub.DistributedTask.Expressions2.Sdk
{
}
namespace GitHub.DistributedTask.Expressions2.Sdk.Functions
{
}
namespace GitHub.DistributedTask.Expressions2.Sdk.Operators
{
}
namespace GitHub.DistributedTask.Expressions2.Tokens
{
}
namespace GitHub.DistributedTask.Logging
{
}
namespace GitHub.DistributedTask.ObjectTemplating
{
}
namespace GitHub.DistributedTask.ObjectTemplating.Schema
{
}
namespace GitHub.DistributedTask.ObjectTemplating.Tokens
{
}
namespace GitHub.DistributedTask.Pipelines
{
}
namespace GitHub.DistributedTask.Pipelines.ContextData
{
}
namespace GitHub.DistributedTask.Pipelines.Expressions
{
}
namespace GitHub.DistributedTask.Pipelines.ObjectTemplating
{
}
namespace GitHub.DistributedTask.Pipelines.Validation
{
}
namespace GitHub.DistributedTask.WebApi
{
}
namespace GitHub.GraphProfile.WebApi
{
}
namespace GitHub.Services
{
}
namespace GitHub.Services.Account
{
}
namespace GitHub.Services.ActivityStatistic
{
}
namespace GitHub.Services.Auditing
{
}
namespace GitHub.Services.AzureFrontDoor
{
}
namespace GitHub.Services.CentralizedFeature
{
}
namespace GitHub.Services.ClientNotification
{
}
namespace GitHub.Services.Commerce
{
}
namespace GitHub.Services.Common
{
}
namespace GitHub.Services.Common.ClientStorage
{
}
namespace GitHub.Services.Common.Diagnostics
{
}
namespace GitHub.Services.Common.Internal
{
}
namespace GitHub.Services.Compliance
{
}
namespace GitHub.Services.ContentSecurityPolicy
{
}
namespace GitHub.Services.DelegatedAuthorization
{
}
namespace GitHub.Services.Directories.DirectoryService
{
}
namespace GitHub.Services.FeatureAvailability
{
}
namespace GitHub.Services.FileContainer
{
}
namespace GitHub.Services.FileContainer.Client
{
}
namespace GitHub.Services.FormInput
{
}
namespace GitHub.Services.GitHubConnector
{
}
namespace GitHub.Services.Graph
{
}
namespace GitHub.Services.Graph.Client
{
}
namespace GitHub.Services.GroupLicensingRule
{
}
namespace GitHub.Services.Health
{
}
namespace GitHub.Services.HostAcquisition
{
}
namespace GitHub.Services.Identity
{
}
namespace GitHub.Services.Identity.Client
{
}
namespace GitHub.Services.Identity.Mru
{
}
namespace GitHub.Services.IdentityPicker
{
}
namespace GitHub.Services.Invitation
{
}
namespace GitHub.Services.Licensing
{
}
namespace GitHub.Services.Location
{
}
namespace GitHub.Services.Location.Client
{
}
namespace GitHub.Services.MarketingPreferences
{
}
namespace GitHub.Services.Notification
{
}
namespace GitHub.Services.OAuth
{
}
namespace GitHub.Services.OAuthWhitelist
{
}
namespace GitHub.Services.Operations
{
}
namespace GitHub.Services.Organization
{
}
namespace GitHub.Services.PermissionLevel
{
}
namespace GitHub.Services.Profile
{
}
namespace GitHub.Services.Security
{
}
namespace GitHub.Services.ServicePrincipal
{
}
namespace GitHub.Services.Servicing
{
}
namespace GitHub.Services.Settings
{
}
namespace GitHub.Services.TokenAdmin.Client
{
}
namespace GitHub.Services.TokenRevocation
{
}
namespace GitHub.Services.Tokens
{
}
namespace GitHub.Services.Tokens.TokenAdmin.Client
{
}
namespace GitHub.Services.TokenSigningKeyLifecycle
{
}
namespace GitHub.Services.UserMapping
{
}
namespace GitHub.Services.Users
{
}
namespace GitHub.Services.WebApi
{
}
namespace GitHub.Services.WebApi.Exceptions
{
}
namespace GitHub.Services.WebApi.Internal
{
}
namespace GitHub.Services.WebApi.Jwt
{
}
namespace GitHub.Services.WebApi.Location
{
}
namespace GitHub.Services.WebApi.Patch
{
}
namespace GitHub.Services.WebApi.Patch.Json
{
}
namespace GitHub.Services.WebApi.Utilities
{
}
namespace GitHub.Services.WebApi.Utilities.Internal
{
}
namespace GitHub.Services.WebApi.Xml
{
}
namespace GitHub.Services.WebPlatform
{
}
namespace GitHub.Services.Zeus
{
}

View File

@@ -5,10 +5,8 @@ using System.Diagnostics;
using System.Linq;
using System.Reflection;
using System.Runtime.Serialization;
using System.Xml;
using System.Xml.Serialization;
using GitHub.Services.Common;
using GitHub.Services.Graph.Client;
namespace GitHub.Services.WebApi.Xml
{

View File

@@ -1,432 +0,0 @@
$ErrorActionPreference = "Stop"
$runnerRepo = Read-Host -Prompt "actions/runner repository root"
if (!(Test-Path -LiteralPath "$runnerRepo/src")) {
Write-Error "$runnerRepo should contains a /src folder"
return 1
}
$gitHubSdkFolder = Join-Path -Path "$runnerRepo/src" -ChildPath "Sdk"
$vsoRepo = $PWD
while ($true) {
if (Test-Path -LiteralPath "$vsoRepo/init.cmd") {
break;
}
else {
$vsoRepo = (Get-Item $vsoRepo).Parent.FullName
}
}
$targetFolders = @(
# "Common"
# "WebApi"
# "AadAuthentication"
# "DTContracts"
# "DTGenerated"
# "DTLogging"
# "DTExpressions"
# "DTExpressions2"
# "DTObjectTemplating"
# "DTPipelines"
# "DTWebApi"
# "Resources"
# "BuildWebApi"
# "CoreWebApi"
# "ArtifactWebApi"
# "ArtifactContentTelemetry"
# "ArtifactContent"
# "BlobStoreWebApi"
# "BlobStoreCommonTelemetry"
# "BlobStoreCommon"
)
$sourceFolders = @{
# "Vssf\Client\Common" = "Common";
# "Vssf\Client\WebApi" = "WebApi";
# "DistributedTask\Shared\Common\Contracts" = "DTContracts";
# "DistributedTask\Client\WebApi\Generated" = "DTGenerated";
# "DistributedTask\Client\WebApi\Logging" = "DTLogging";
# "DistributedTask\Client\WebApi\Expressions" = "DTExpressions";
# "Actions\Runtime\Client\WebApi\Expressions2" = "DTExpressions2";
# "Actions\Runtime\Client\WebApi\ObjectTemplating" = "DTObjectTemplating";
# "Actions\Runtime\Client\WebApi\Pipelines" = "DTPipelines";
# "DistributedTask\Client\WebApi\WebApi" = "DTWebApi";
# "..\obj\Debug.AnyCPU\Vssf.Client\MS.VS.Services.Common\EmbeddedVersionInfo.cs" = "Common\EmbeddedVersionInfo.cs";
# "Vssf\InteractiveClient\Client\Authentication\VssAadToken.cs" = "AadAuthentication";
# "Vssf\InteractiveClient\Client\Authentication\VssAadTokenProvider.cs" = "AadAuthentication";
# "Vssf\InteractiveClient\Client\Authentication\VssAadCredential.cs" = "AadAuthentication";
# "Vssf\InteractiveClient\Client\VssAadSettings.cs" = "AadAuthentication";
# "Vssf\InteractiveClient\Client\Authentication\VssFederatedCredential.cs" = "AadAuthentication";
# "Vssf\InteractiveClient\Client\Authentication\VssFederatedToken.cs" = "AadAuthentication";
# "Vssf\InteractiveClient\Client\Authentication\VssFederatedTokenProvider.cs" = "AadAuthentication";
# "Vssf\InteractiveClient\Client\Authentication\Utility\CookieUtility.cs" = "AadAuthentication";
# "Actions\Runtime\Client\WebApi\Pipelines\ObjectTemplating\workflow-v1.0.json" = "DTPipelines";
# "Tfs\Client\Build2\Api" = "BuildWebApi";
# "Tfs\Client\Core" = "CoreWebApi";
# "ArtifactServices\Client\WebApi" = "ArtifactWebApi";
# "ArtifactServices\Shared\Content.Common.Telemetry" = "ArtifactContentTelemetry";
# "ArtifactServices\Shared\Content.Common" = "ArtifactContent";
# "BlobStore\Client\WebApi" = "BlobStoreWebApi";
# "ArtifactServices\Shared\BlobStore.Common.Telemetry" = "BlobStoreCommonTelemetry";
# "ArtifactServices\Shared\BlobStore.Common" = "BlobStoreCommon";
}
$extraFiles = @(
# "BlobStoreCommon\BlobStore.Common\AzureStorageOperationTraceAdapter.cs"
# "BlobStoreCommon\BlobStore.Common\BlobIdentifierHelperExtensions.cs"
# "BlobStoreCommon\BlobStore.Common\BlobIdentifierHexConverter.cs"
# "BlobStoreCommon\BlobStore.Common\EdgeCacheUrlBuilder.cs"
# "BlobStoreCommon\BlobStore.Common\Exceptions.cs"
# "BlobStoreCommon\BlobStore.Common\IDownloader.cs"
# "BlobStoreCommon\BlobStore.Common\InternalsVisibleTo.cs"
# "BlobStoreCommon\BlobStore.Common\IUrlSigner.cs"
# "BlobStoreCommon\BlobStore.Common\ManagedParallelBlobDownloader.cs"
# "BlobStoreCommon\BlobStore.Common\NullableExtensions.cs"
# "BlobStoreCommon\BlobStore.Common\ObjectExtensions.cs"
# "BlobStoreCommon\BlobStore.Common\PerfCounters\InstrumentationManifest.cs"
# "BlobStoreCommon\BlobStore.Common\PerfCounters\InstrumentationManifestException.cs"
# "BlobStoreCommon\BlobStore.Common\PerfCounters\IPerformanceDataFacade.cs"
# "BlobStoreCommon\BlobStore.Common\PerfCounters\ManifestCounters.cs"
# "BlobStoreCommon\BlobStore.Common\PerfCounters\NoopPerfCounter.cs"
# "BlobStoreCommon\BlobStore.Common\PerfCounters\NoopPerformanceDataFacade.cs"
# "BlobStoreCommon\BlobStore.Common\PerfCounters\PerfCounter.cs"
# "BlobStoreCommon\BlobStore.Common\PerfCounters\PerfCounterSet.cs"
# "BlobStoreCommon\BlobStore.Common\PerfCounters\PerformanceDataFacade.cs"
# "BlobStoreCommon\BlobStore.Common\ReceiptSecretConstants.cs"
# "BlobStoreCommon\BlobStore.Common\SecurityDefinitions.cs"
# "BlobStoreCommon\BlobStore.Common\VsoHashAlgorithm.cs"
# "BlobStoreCommonTelemetry\BlobStore.Common.Telemetry\InternalsVisibleTo.cs"
# "ArtifactContentTelemetry\Content.Common.Telemetry\InternalsVisibleTo.cs"
# "ArtifactContentTelemetry\Content.Common.Telemetry\Telemetry\NoopClientTelemetry.cs"
# "ArtifactContentTelemetry\Content.Common.Telemetry\Telemetry\TelemetryPlatformSpecificNetFramework.cs"
# "ArtifactContentTelemetry\Content.Common.Telemetry\Telemetry\TelemetryEnvironmentHelper.cs"
# "ArtifactContent\Content.Common\AsyncEnumerator\AsyncEnumeratorExceptionWrapper.cs"
# "ArtifactContent\Content.Common\AsyncEnumerator\IEnumeratorExceptionMapper.cs"
# "ArtifactContent\Content.Common\AsyncEnumerator\AsyncEnumeratorWithCursor.cs"
# "ArtifactContent\Content.Common\Authentication\AadAcquireTokenException.cs"
# "ArtifactContent\Content.Common\Authentication\AadErrorHandlingPolicy.cs"
# "ArtifactContent\Content.Common\Authentication\CredentialProvider\CredentialProviderException.cs"
# "ArtifactContent\Content.Common\Authentication\CredentialProvider\CredentialProviderLoader.cs"
# "ArtifactContent\Content.Common\Authentication\CredentialProvider\CredentialProviderManager.cs"
# "ArtifactContent\Content.Common\Authentication\CredentialProvider\CredentialResponse.cs"
# "ArtifactContent\Content.Common\Authentication\CredentialProvider\ICredentialProvider.cs"
# "ArtifactContent\Content.Common\Authentication\CredentialProvider\PluginCredentialProvider.cs"
# "ArtifactContent\Content.Common\Authentication\LocalTokenCacheArgs.cs"
# "ArtifactContent\Content.Common\Authentication\TestableAuthenticationContext.cs"
# "ArtifactContent\Content.Common\Authentication\VsoAadConstants.cs"
# "ArtifactContent\Content.Common\Authentication\VsoCredentialHelper.cs"
# "ArtifactContent\Content.Common\AutoKillProcessHandle.cs"
# "ArtifactContent\Content.Common\ConcurrencyConsolidator.cs"
# "ArtifactContent\Content.Common\EnumUtilities.cs"
# "ArtifactContent\Content.Common\EquatableTuple.cs"
# "ArtifactContent\Content.Common\FileVersionHelpers.cs"
# "ArtifactContent\Content.Common\Histogram.cs"
# "ArtifactContent\Content.Common\InternalsVisibleTo.cs"
# "ArtifactContent\Content.Common\InUseDetection.cs"
# "ArtifactContent\Content.Common\IteratorPartition.cs"
# "ArtifactContent\Content.Common\Json\ByteArrayAsBase64JsonConvertor.cs"
# "ArtifactContent\Content.Common\Json\ByteArrayAsHexJsonConvertor.cs"
# "ArtifactContent\Content.Common\Json\JsonEnumerator.cs"
# "ArtifactContent\Content.Common\Json\JsonNormalizer.cs"
# "ArtifactContent\Content.Common\Json\JsonProperty.cs"
# "ArtifactContent\Content.Common\Json\JsonStream.cs"
# "ArtifactContent\Content.Common\Json\JsonWrite.cs"
# "ArtifactContent\Content.Common\Json\JsonWriterStream.cs"
# "ArtifactContent\Content.Common\Json\JsonWrites.cs"
# "ArtifactContent\Content.Common\Json\ULongJsonConverter.cs"
# "ArtifactContent\Content.Common\Kvp.cs"
# "ArtifactContent\Content.Common\Operations\SecureStringConverter.cs"
# "ArtifactContent\Content.Common\PagedEnumerator.cs"
# "ArtifactContent\Content.Common\PerformanceInfo.cs"
# "ArtifactContent\Content.Common\ReaderWriterLockSlimExtensions.cs"
# "ArtifactContent\Content.Common\ReadOnlySet.cs"
# "ArtifactContent\Content.Common\RetrievalOptions.cs"
# "ArtifactContent\Content.Common\ServiceInstanceTypes.cs"
# "ArtifactContent\Content.Common\ServicePointHelper.cs"
# "ArtifactContent\Content.Common\ShardableLocator.cs"
# "ArtifactContent\Content.Common\StringExtensions.cs"
# "ArtifactContent\Content.Common\ThreadLocalRandom.cs"
# "ArtifactContent\Content.Common\ThreadPoolHelper.cs"
# "ArtifactContent\Content.Common\Tracing\AppTraceListener.cs"
# "ArtifactContent\Content.Common\Tracing\ArtifactServicesTraceSource.cs"
# "ArtifactContent\Content.Common\Tracing\ConsoleMessageUtil.cs"
# "ArtifactContent\Content.Common\Tracing\ConsoleTraceListener.cs"
# "ArtifactContent\Content.Common\Tracing\FileTraceListener.cs"
# "ArtifactContent\Content.Common\Tracing\InMemoryLog.cs"
# "ArtifactContent\Content.Common\Tracing\InMemoryTraceListener.cs"
# "Common\Common\CommandLine\Argument.cs"
# "Common\Common\CommandLine\AttributeBasedOperationModeHandlerFactory.cs"
# "Common\Common\CommandLine\AttributeBasedOptionParserAdapter.cs"
# "Common\Common\CommandLine\BasicParser.cs"
# "Common\Common\CommandLine\CommandLineLexer.cs"
# "Common\Common\CommandLine\Enumerations.cs"
# "Common\Common\CommandLine\Exceptions.cs"
# "Common\Common\CommandLine\Extensions.cs"
# "Common\Common\CommandLine\IEnumerable.cs"
# "Common\Common\CommandLine\OperationHandler.cs"
# "Common\Common\CommandLine\OperationHandlerFactory.cs"
# "Common\Common\CommandLine\OperationModeAttribute.cs"
# "Common\Common\CommandLine\Option.cs"
# "Common\Common\CommandLine\OptionAttribute.cs"
# "Common\Common\CommandLine\OptionParser.cs"
# "Common\Common\CommandLine\OptionReader.cs"
# "Common\Common\CommandLine\ResponseFileOptionReader.cs"
# "Common\Common\CommandLine\Validation\DefaultValidation.cs"
# "Common\Common\CommandLine\Validation\IOptionValidation.cs"
# "Common\Common\CommandLine\Validation\OptionExistsFilter.cs"
# "Common\Common\CommandLine\Validation\OptionMustExist.cs"
# "Common\Common\CommandLine\Validation\OptionRequiresSpecificValue.cs"
# "Common\Common\CommandLine\Validation\OptionsAreMutuallyExclusive.cs"
# "Common\Common\CommandLine\Validation\OptionsAreMutuallyInclusive.cs"
# "Common\Common\CommandLine\Validation\OptionValidation.cs"
# "Common\Common\CommandLine\Validation\OptionValidationFilter.cs"
# "Common\Common\CommandLine\Validation\OptionValueFilter.cs"
# "Common\Common\CommandLine\ValueConverters\CsvCollectionConverter.cs"
# "Common\Common\CommandLine\ValueConverters\EnumConverter.cs"
# "Common\Common\CommandLine\ValueConverters\IValueConvertible.cs"
# "Common\Common\CommandLine\ValueConverters\UriConverter.cs"
# "Common\Common\CommandLine\ValueConverters\ValueConverter.cs"
# "Common\Common\ExternalProviders\IExternalProviderHttpRequester.cs"
# "Common\Common\Performance\PerformanceNativeMethods.cs"
# "Common\Common\TokenStorage\RegistryToken.cs"
# "Common\Common\TokenStorage\RegistryTokenStorage.cs"
# "Common\Common\TokenStorage\RegistryTokenStorageHelper.cs"
# "Common\Common\TokenStorage\VssTokenStorageFactory.cs"
# "Common\Common\Utility\CredentialsCacheManager.cs"
# "Common\Common\Utility\EncryptionUtility.cs"
# "Common\Common\Utility\EnumerableUtility.cs"
# "Common\Common\Utility\EnvironmentWrapper.cs"
# "Common\Common\Utility\ExceptionExtentions.cs"
# "Common\Common\Utility\NativeMethods.cs"
# "Common\Common\Utility\OSDetails.cs"
# "Common\Common\Utility\DateTimeUtility.cs"
# "Common\Common\Utility\PasswordUtility.cs"
# "Common\Common\Utility\RegistryHelper.cs"
# "Common\Common\Utility\SerializationHelper.cs"
# "Common\Common\Utility\Csv\CsvException.cs"
# "Common\Common\Utility\Csv\CsvConfiguration.cs"
# "Common\Common\Utility\Csv\CsvWriter.cs"
# "Common\Common\VssEnvironment.cs"
# "WebApi\WebApi\AssemblyAttributes.cs"
# "WebApi\WebApi\Contracts\DelegatedAuthorization\ExpiringToken.cs"
# "WebApi\WebApi\Contracts\ExternalEvent\Comparers\ExternalGitIssueComparer.cs"
# "WebApi\WebApi\Contracts\ExternalEvent\ExternalGitExtensions.cs"
# "WebApi\WebApi\Contracts\ExternalEvent\Comparers\ExternalGitPullRequestComparer.cs"
# "WebApi\WebApi\Contracts\ExternalEvent\Comparers\ExternalGitCommitComparer.cs"
# "WebApi\WebApi\Contracts\ExternalEvent\ExternalGitIssueEvent.cs"
# "WebApi\WebApi\Contracts\ExternalEvent\Comparers\ExternalGitRepoComparer.cs"
# "WebApi\WebApi\Contracts\ExternalEvent\ExternalGitCommitCommentEvent.cs"
# "WebApi\WebApi\Contracts\PermissionLevel\Client\PagedPermissionLevelAssignment.cs"
# "WebApi\WebApi\Contracts\PermissionLevel\Client\PermissionLevelAssignment.cs"
# "WebApi\WebApi\Contracts\PermissionLevel\Enumerations.cs"
# "WebApi\WebApi\Contracts\PermissionLevel\Client\PermissionLevelDefinition.cs"
# "WebApi\WebApi\Contracts\Tokens\PATAddedEvent.cs"
# "WebApi\WebApi\Contracts\Tokens\SshKeyAddedEvent.cs"
# "WebApi\WebApi\Contracts\Tokens\ExpiringTokenEvent.cs"
# "WebApi\WebApi\Contracts\DelegatedAuthorization\PATAddedEvent.cs"
# "WebApi\WebApi\Contracts\DelegatedAuthorization\SshKeyAddedEvent.cs"
# "WebApi\WebApi\Contracts\DelegatedAuthorization\ExpiringTokenEvent.cs"
# "WebApi\WebApi\Contracts\DelegatedAuthorization\Migration\DelegatedAuthMigrationStatus.cs"
# "WebApi\WebApi\Contracts\DelegatedAuthorization\Migration\DelegatedAuthorizationMigrationBase.cs"
# "WebApi\WebApi\Contracts\DelegatedAuthorization\Migration\TokenDelegatedAuthorizationAccessKeyPublicDataMigration.cs"
# "WebApi\WebApi\Contracts\DelegatedAuthorization\Migration\TokenDelegatedAuthorizationAccessMigration.cs"
# "WebApi\WebApi\Contracts\DelegatedAuthorization\Migration\TokenDelegatedAuthorizationMigration.cs"
# "WebApi\WebApi\Contracts\DelegatedAuthorization\Migration\TokenDelegatedAuthorizationAccessKeyMigration.cs"
# "WebApi\WebApi\Contracts\DelegatedAuthorization\Migration\TokenDelegatedAuthorizationRegistrationMigration.cs"
# "WebApi\WebApi\Contracts\DelegatedAuthorization\Migration\TokenDelegatedAuthorizationRegistrationRedirectLocationMigration.cs"
# "WebApi\WebApi\Contracts\DelegatedAuthorization\Migration\TokenDelegatedHostAuthorizationMigration.cs"
# "WebApi\WebApi\Contracts\OAuthWhitelist\OAuthWhitelistEntry.cs"
# "WebApi\WebApi\Contracts\TokenAdmin\PatRevokedEvent.cs"
# "WebApi\WebApi\Contracts\TokenAdmin\TokenAdministrationRevocation.cs"
# "WebApi\WebApi\Contracts\TokenAdmin\TokenAdminPagedSessionTokens.cs"
# "WebApi\WebApi\Contracts\TokenAdmin\TokenAdminRevocation.cs"
# "WebApi\WebApi\Contracts\TokenAdmin\TokenAdminRevocationRule.cs"
# "WebApi\WebApi\Exceptions\AuditLogExceptions.cs"
# "WebApi\WebApi\Exceptions\AadExceptions.cs"
# "WebApi\WebApi\Exceptions\PermissionLevelExceptions.cs"
# "WebApi\WebApi\HttpClients\CsmResourceProviderHttpClient.cs"
# "WebApi\WebApi\HttpClients\Generated\CsmResourceProviderHttpClientBase.cs"
# "WebApi\WebApi\HttpClients\Generated\OAuthWhitelistHttpClient.cs"
# "WebApi\WebApi\HttpClients\Generated\TokenAdminHttpClient.cs"
# "WebApi\WebApi\HttpClients\Generated\TokenAdministrationHttpClient.cs"
# "WebApi\WebApi\HttpClients\Generated\TokenExpirationHttpClient.cs"
# "WebApi\WebApi\HttpClients\Generated\TokenMigrationHttpClient.cs"
# "WebApi\WebApi\HttpClients\Generated\PermissionLevelHttpClient.cs"
# "WebApi\WebApi\HttpClients\CommerceHostHelperHttpClient.cs"
# "WebApi\WebApi\Utilities\DelegatedAuthComparers.cs"
# "WebApi\WebApi\Utilities\HttpHeadersExtensions.cs"
# "WebApi\WebApi\VssClientCertificateManager.cs"
# "WebApi\WebApi\VssClientEnvironment.cs"
# "WebApi\WebApi\VssSoapMediaTypeFormatter.cs"
)
$resourceFiles = @{
# "ExpressionResources" = "DistributedTask\Client\WebApi\Expressions\ExpressionResources.resx";
# "PipelineStrings" = "DistributedTask\Client\WebApi\Pipelines\PipelineStrings.resx";
# "CommonResources" = "Vssf\Client\Common\Resources.resx";
# "IdentityResources" = "Vssf\Client\WebApi\Resources\IdentityResources.resx";
# "JwtResources" = "Vssf\Client\WebApi\Resources\JwtResources.resx";
# "WebApiResources" = "Vssf\Client\WebApi\Resources\WebApiResources.resx";
# "DataImportResources" = "Vssf\Client\WebApi\Resources\DataImportResources.resx";
# "PatchResources" = "Vssf\Client\WebApi\Resources\PatchResources.resx";
# "AccountResources" = "Vssf\Client\WebApi\Resources\AccountResources.resx";
# "TemplateStrings" = "DistributedTask\Client\WebApi\ObjectTemplating\TemplateStrings.resx";
# "GraphResources" = "Vssf\Client\WebApi\Resources\GraphResources.resx";
# "FileContainerResources" = "Vssf\Client\WebApi\Resources\FileContainerResources.resx";
# "LocationResources" = "Vssf\Client\WebApi\Resources\LocationResources.resx";
# "CommerceResources" = "Vssf\Client\WebApi\Resources\CommerceResources.resx";
# "SecurityResources" = "Vssf\Client\WebApi\Resources\SecurityResources.resx";
# "WebPlatformResources" = "Vssf\Client\WebApi\Resources\WebPlatformResources.resx";
# "ZeusWebApiResources" = "Vssf\Client\WebApi\Resources\ZeusWebApiResources.resx";
# "NameResolutionResources" = "Vssf\Client\WebApi\Resources\NameResolutionResources.resx";
# "PartitioningResources" = "Vssf\Client\WebApi\Resources\PartitioningResources.resx";
# "WebApiResources" = "Tfs\Client\Core\Resources\WebApiResources.resx";
# "BlobStoreResources" = "BlobStore\Client\WebApi\Resources.resx"
# "ContentResources" = "ArtifactServices\Shared\Content.Common\Resources.resx"
# "BlobStoreCommonResources" = "ArtifactServices\Shared\BlobStore.Common\Resources.resx"
}
$resourceNamespace = @{
# "ExpressionResources" = "Microsoft.TeamFoundation.DistributedTask.Expressions";
# "PipelineStrings" = "Microsoft.TeamFoundation.DistributedTask.Pipelines";
# "CommonResources" = "Microsoft.VisualStudio.Services.Common.Internal";
# "IdentityResources" = "Microsoft.VisualStudio.Services.WebApi";
# "JwtResources" = "Microsoft.VisualStudio.Services.WebApi";
# "WebApiResources" = "Microsoft.VisualStudio.Services.WebApi";
# "DataImportResources" = "Microsoft.VisualStudio.Services.WebApi";
# "PatchResources" = "Microsoft.VisualStudio.Services.WebApi";
# "AccountResources" = "Microsoft.VisualStudio.Services.WebApi";
# "TemplateStrings" = "Microsoft.TeamFoundation.DistributedTask.ObjectTemplating";
# "GraphResources" = "Microsoft.VisualStudio.Services.WebApi";
# "FileContainerResources" = "Microsoft.VisualStudio.Services.WebApi";
# "LocationResources" = "Microsoft.VisualStudio.Services.WebApi";
# "CommerceResources" = "Microsoft.VisualStudio.Services.WebApi";
# "SecurityResources" = "Microsoft.VisualStudio.Services.WebApi";
# "WebPlatformResources" = "Microsoft.VisualStudio.Services.WebApi";
# "ZeusWebApiResources" = "Microsoft.VisualStudio.Services.WebApi";
# "NameResolutionResources" = "Microsoft.VisualStudio.Services.WebApi";
# "PartitioningResources" = "Microsoft.VisualStudio.Services.WebApi";
# "WebApiResources" = "Microsoft.TeamFoundation.Core.WebApi";
# "ContentResources" = "Microsoft.VisualStudio.Services.Content.Common";
# "BlobStoreCommonResources" = "Microsoft.VisualStudio.Services.BlobStore.Common";
# "BlobStoreResources" = "Microsoft.VisualStudio.Services.BlobStore.WebApi";
}
foreach ($folder in $targetFolders) {
Write-Host "Recreate $gitHubSdkFolder\$folder"
if (Test-Path -LiteralPath "$gitHubSdkFolder\$folder") {
Remove-Item -LiteralPath "$gitHubSdkFolder\$folder" -Force -Recurse
}
New-Item -Path $gitHubSdkFolder -Name $folder -ItemType "directory" -Force
}
foreach ($sourceFolder in $sourceFolders.Keys) {
$copySource = Join-Path -Path $vsoRepo -ChildPath $sourceFolder
$copyDest = Join-Path -Path $gitHubSdkFolder -ChildPath $sourceFolders[$sourceFolder]
Write-Host "Copy $copySource to $copyDest"
Copy-Item -Path $copySource -Destination $copyDest -Filter "*.cs" -Recurse -Force
}
Write-Host "Delete extra none NetStandard files"
foreach ($extraFile in $extraFiles) {
Remove-Item -LiteralPath "$gitHubSdkFolder\$extraFile" -Force
}
Write-Host "Generate C# file for resx files"
foreach ($resourceFile in $resourceFiles.Keys) {
Write-Host "Generate file for $resourceFile"
$stringBuilder = New-Object System.Text.StringBuilder
$file = $resourceFiles[$resourceFile]
$xml = [xml](Get-Content -LiteralPath "$vsoRepo\$file")
$null = $stringBuilder.AppendLine('using System.Globalization;')
$null = $stringBuilder.AppendLine('')
$namespace = $resourceNamespace[$resourceFile]
$null = $stringBuilder.AppendLine("namespace $namespace")
$null = $stringBuilder.AppendLine('{')
$null = $stringBuilder.AppendLine(" public static class $resourceFile")
$null = $stringBuilder.AppendLine(' {')
foreach ($data in $xml.root.data) {
$i = 0
$args = ""
$inputs = ""
while ($true) {
if ($data.value.Contains("{$i}") -or $data.value.Contains("{$i" + ":")) {
if ($i -eq 0) {
$args = "object arg$i"
$inputs = "arg$i"
}
else {
$args = $args + ", " + "object arg$i"
$inputs = $inputs + ", " + "arg$i"
}
$i++
}
else {
break
}
}
$null = $stringBuilder.AppendLine("")
$null = $stringBuilder.AppendLine(" public static string $($data.name)($($args))")
$null = $stringBuilder.AppendLine(" {")
$null = $stringBuilder.AppendLine(@"
const string Format = @"$($data.value.Replace('"', '""'))";
"@)
if ($i -eq 0) {
$null = $stringBuilder.AppendLine(" return Format;")
}
else {
$null = $stringBuilder.AppendLine(" return string.Format(CultureInfo.CurrentCulture, Format, $inputs);")
}
$null = $stringBuilder.AppendLine(" }")
}
$null = $stringBuilder.AppendLine(" }")
$null = $stringBuilder.AppendLine("}")
# Write Resources.g.cs.
$genResourceFile = Join-Path -Path $gitHubSdkFolder -ChildPath "Resources\$resourceFile.g.cs"
[System.IO.File]::WriteAllText($genResourceFile, ($stringBuilder.ToString()), ([System.Text.Encoding]::UTF8))
}
# Print out all namespaces
Write-Host "Rename namespaces:"
$namespaces = New-Object 'System.Collections.Generic.HashSet[string]'
$sourceFiles = Get-ChildItem -LiteralPath $gitHubSdkFolder -Filter "*.cs" -Recurse -Force -File
foreach ($file in $sourceFiles) {
foreach ($line in Get-Content $file.FullName) {
if ($line.StartsWith("namespace ")) {
$namespace = $line.Substring("namespace ".Length)
if ($namespaces.Add($namespace)) {
Write-Host $namespace
}
}
}
}
# Rename all namespaces to GitHub
$allSourceFiles = Get-ChildItem -LiteralPath $gitHubSdkFolder -Filter "*.cs" -Recurse -Force -File
foreach ($file in $allSourceFiles) {
$stringBuilder = New-Object System.Text.StringBuilder
foreach ($line in Get-Content $file.FullName) {
if ($line.Contains("Microsoft.VisualStudio")) {
$line = $line.Replace("Microsoft.VisualStudio", "GitHub");
}
elseif ($line.Contains("Microsoft.Azure.DevOps")) {
$line = $line.Replace("Microsoft.Azure.DevOps", "GitHub");
}
elseif ($line.Contains("Microsoft.TeamFoundation")) {
$line = $line.Replace("Microsoft.TeamFoundation", "GitHub");
}
$null = $stringBuilder.AppendLine($line)
}
[System.IO.File]::WriteAllText($file.FullName, ($stringBuilder.ToString()), ([System.Text.Encoding]::UTF8))
}
Write-Host "Done"

View File

@@ -260,7 +260,7 @@ namespace GitHub.Runner.Common.Tests
var proc = await processInvoker.ExecuteAsync("", "bash", "-c \"cat /proc/$$/oom_score_adj\"", null, false, null, false, null, false, false,
highPriorityProcess: false,
cancellationToken: tokenSource.Token);
Assert.Equal(oomScoreAdj, 500);
Assert.Equal(500, oomScoreAdj);
}
catch (OperationCanceledException)
{
@@ -293,12 +293,12 @@ namespace GitHub.Runner.Common.Tests
};
try
{
var proc = await processInvoker.ExecuteAsync("", "bash", "-c \"cat /proc/$$/oom_score_adj\"",
var proc = await processInvoker.ExecuteAsync("", "bash", "-c \"cat /proc/$$/oom_score_adj\"",
new Dictionary<string, string> { {"PIPELINE_JOB_OOMSCOREADJ", "1234"} },
false, null, false, null, false, false,
highPriorityProcess: false,
cancellationToken: tokenSource.Token);
Assert.Equal(oomScoreAdj, 1234);
Assert.Equal(1234, oomScoreAdj);
}
catch (OperationCanceledException)
{
@@ -336,7 +336,7 @@ namespace GitHub.Runner.Common.Tests
var proc = await processInvoker.ExecuteAsync("", "bash", "-c \"cat /proc/$$/oom_score_adj\"", null, false, null, false, null, false, false,
highPriorityProcess: true,
cancellationToken: tokenSource.Token);
Assert.Equal(oomScoreAdj, 123);
Assert.Equal(123, oomScoreAdj);
}
catch (OperationCanceledException)
{

View File

@@ -1,8 +1,7 @@
using GitHub.Runner.Common.Util;
using System.IO;
using System.IO;
using Xunit;
using System;
using GitHub.Runner.Sdk;
using System.Runtime.CompilerServices;
namespace GitHub.Runner.Common.Tests
{
@@ -21,9 +20,16 @@ namespace GitHub.Runner.Common.Tests
return projectDir;
}
public static string GetTestFilePath([CallerFilePath] string path = null)
{
return path;
}
public static string GetSrcPath()
{
string srcDir = Environment.GetEnvironmentVariable("GITHUB_RUNNER_SRC_DIR");
string L0dir = Path.GetDirectoryName(GetTestFilePath());
string testDir = Path.GetDirectoryName(L0dir);
string srcDir = Path.GetDirectoryName(testDir);
ArgUtil.Directory(srcDir, nameof(srcDir));
Assert.Equal(Src, Path.GetFileName(srcDir));
return srcDir;

View File

@@ -51,6 +51,19 @@ namespace GitHub.Runner.Common.Tests.Worker
Assert.True(ActionCommand.TryParse(message, commands, out verify));
Assert.True(IsEqualCommand(hc, test, verify));
message = "";
test = null;
verify = null;
//##[do-something k1=%253B=%250D=%250A=%255D;]%253B-%250D-%250A-%255D
message = "##[do-something k1=%253B=%250D=%250A=%255D;]%253B-%250D-%250A-%255D";
test = new ActionCommand("do-something")
{
Data = "%3B-%0D-%0A-%5D",
};
test.Properties.Add("k1", "%3B=%0D=%0A=%5D");
Assert.True(ActionCommand.TryParse(message, commands, out verify));
Assert.True(IsEqualCommand(hc, test, verify));
message = "";
test = null;
verify = null;
@@ -109,7 +122,7 @@ namespace GitHub.Runner.Common.Tests.Worker
message = "";
test = null;
verify = null;
//::do-something k1=%3B=%0D=%0A=%5D;::%3B-%0D-%0A-%5D
//::do-something k1=;=%2C=%0D=%0A=]=%3A,::;-%0D-%0A-]-:-,
message = "::do-something k1=;=%2C=%0D=%0A=]=%3A,::;-%0D-%0A-]-:-,";
test = new ActionCommand("do-something")
{
@@ -119,6 +132,19 @@ namespace GitHub.Runner.Common.Tests.Worker
Assert.True(ActionCommand.TryParseV2(message, commands, out verify));
Assert.True(IsEqualCommand(hc, test, verify));
message = "";
test = null;
verify = null;
//::do-something k1=;=%252C=%250D=%250A=]=%253A,::;-%250D-%250A-]-:-,
message = "::do-something k1=;=%252C=%250D=%250A=]=%253A,::;-%250D-%250A-]-:-,";
test = new ActionCommand("do-something")
{
Data = ";-%0D-%0A-]-:-,",
};
test.Properties.Add("k1", ";=%2C=%0D=%0A=]=%3A");
Assert.True(ActionCommand.TryParseV2(message, commands, out verify));
Assert.True(IsEqualCommand(hc, test, verify));
message = "";
test = null;
verify = null;

View File

@@ -57,8 +57,8 @@ namespace GitHub.Runner.Common.Tests.Worker
var steps = await _actionManager.PrepareActionsAsync(_ec.Object, actions);
//Assert
Assert.Equal((steps[0].Data as ContainerSetupInfo).StepIds[0], actionId);
Assert.Equal((steps[0].Data as ContainerSetupInfo).Container.Image, "ubuntu:16.04");
Assert.Equal(actionId, (steps[0].Data as ContainerSetupInfo).StepIds[0]);
Assert.Equal("ubuntu:16.04", (steps[0].Data as ContainerSetupInfo).Container.Image);
}
finally
{
@@ -125,7 +125,7 @@ namespace GitHub.Runner.Common.Tests.Worker
Directory.CreateDirectory(Path.GetDirectoryName(watermarkFile));
File.WriteAllText(watermarkFile, DateTime.UtcNow.ToString());
Directory.CreateDirectory(Path.Combine(Path.GetDirectoryName(watermarkFile), "notexist"));
File.Copy(Path.Combine(Environment.GetEnvironmentVariable("GITHUB_RUNNER_SRC_DIR"), "Test", TestDataFolderName, "dockerfileaction.yml"), Path.Combine(Path.GetDirectoryName(watermarkFile), "notexist", "action.yml"));
File.Copy(Path.Combine(TestUtil.GetSrcPath(), "Test", TestDataFolderName, "dockerfileaction.yml"), Path.Combine(Path.GetDirectoryName(watermarkFile), "notexist", "action.yml"));
//Act
await _actionManager.PrepareActionsAsync(_ec.Object, actions);
@@ -204,9 +204,9 @@ namespace GitHub.Runner.Common.Tests.Worker
//Act
var steps = await _actionManager.PrepareActionsAsync(_ec.Object, actions);
Assert.Equal((steps[0].Data as ContainerSetupInfo).StepIds[0], actionId);
Assert.Equal((steps[0].Data as ContainerSetupInfo).Container.WorkingDirectory, actionDir);
Assert.Equal((steps[0].Data as ContainerSetupInfo).Container.Dockerfile, Path.Combine(actionDir, "Dockerfile"));
Assert.Equal(actionId, (steps[0].Data as ContainerSetupInfo).StepIds[0]);
Assert.Equal(actionDir, (steps[0].Data as ContainerSetupInfo).Container.WorkingDirectory);
Assert.Equal(Path.Combine(actionDir, "Dockerfile"), (steps[0].Data as ContainerSetupInfo).Container.Dockerfile);
}
finally
{
@@ -245,9 +245,9 @@ namespace GitHub.Runner.Common.Tests.Worker
//Act
var steps = await _actionManager.PrepareActionsAsync(_ec.Object, actions);
Assert.Equal((steps[0].Data as ContainerSetupInfo).StepIds[0], actionId);
Assert.Equal((steps[0].Data as ContainerSetupInfo).Container.WorkingDirectory, actionDir);
Assert.Equal((steps[0].Data as ContainerSetupInfo).Container.Dockerfile, Path.Combine(actionDir, "images/cli", "Dockerfile"));
Assert.Equal(actionId, (steps[0].Data as ContainerSetupInfo).StepIds[0]);
Assert.Equal(actionDir, (steps[0].Data as ContainerSetupInfo).Container.WorkingDirectory);
Assert.Equal(Path.Combine(actionDir, "images/cli", "Dockerfile"), (steps[0].Data as ContainerSetupInfo).Container.Dockerfile);
}
finally
{
@@ -284,9 +284,9 @@ namespace GitHub.Runner.Common.Tests.Worker
//Act
var steps = await _actionManager.PrepareActionsAsync(_ec.Object, actions);
Assert.Equal((steps[0].Data as ContainerSetupInfo).StepIds[0], actionId);
Assert.Equal((steps[0].Data as ContainerSetupInfo).Container.WorkingDirectory, actionDir);
Assert.Equal((steps[0].Data as ContainerSetupInfo).Container.Dockerfile, Path.Combine(actionDir, "Dockerfile"));
Assert.Equal(actionId, (steps[0].Data as ContainerSetupInfo).StepIds[0]);
Assert.Equal(actionDir, (steps[0].Data as ContainerSetupInfo).Container.WorkingDirectory);
Assert.Equal(Path.Combine(actionDir, "Dockerfile"), (steps[0].Data as ContainerSetupInfo).Container.Dockerfile);
}
finally
{
@@ -324,9 +324,9 @@ namespace GitHub.Runner.Common.Tests.Worker
//Act
var steps = await _actionManager.PrepareActionsAsync(_ec.Object, actions);
Assert.Equal((steps[0].Data as ContainerSetupInfo).StepIds[0], actionId);
Assert.Equal((steps[0].Data as ContainerSetupInfo).Container.WorkingDirectory, actionDir);
Assert.Equal((steps[0].Data as ContainerSetupInfo).Container.Dockerfile, Path.Combine(actionDir, "images/Dockerfile"));
Assert.Equal(actionId, (steps[0].Data as ContainerSetupInfo).StepIds[0]);
Assert.Equal(actionDir, (steps[0].Data as ContainerSetupInfo).Container.WorkingDirectory);
Assert.Equal(Path.Combine(actionDir, "images/Dockerfile"), (steps[0].Data as ContainerSetupInfo).Container.Dockerfile);
}
finally
{
@@ -364,8 +364,8 @@ namespace GitHub.Runner.Common.Tests.Worker
//Act
var steps = await _actionManager.PrepareActionsAsync(_ec.Object, actions);
Assert.Equal((steps[0].Data as ContainerSetupInfo).StepIds[0], actionId);
Assert.Equal((steps[0].Data as ContainerSetupInfo).Container.Image, "ubuntu:18.04");
Assert.Equal(actionId, (steps[0].Data as ContainerSetupInfo).StepIds[0]);
Assert.Equal("ubuntu:18.04", (steps[0].Data as ContainerSetupInfo).Container.Image);
}
finally
{
@@ -403,9 +403,9 @@ namespace GitHub.Runner.Common.Tests.Worker
//Act
var steps = await _actionManager.PrepareActionsAsync(_ec.Object, actions);
Assert.Equal((steps[0].Data as ContainerSetupInfo).StepIds[0], actionId);
Assert.Equal((steps[0].Data as ContainerSetupInfo).Container.WorkingDirectory, actionDir);
Assert.Equal((steps[0].Data as ContainerSetupInfo).Container.Dockerfile, Path.Combine(actionDir, "Dockerfile"));
Assert.Equal(actionId, (steps[0].Data as ContainerSetupInfo).StepIds[0]);
Assert.Equal(actionDir, (steps[0].Data as ContainerSetupInfo).Container.WorkingDirectory);
Assert.Equal(Path.Combine(actionDir, "Dockerfile"), (steps[0].Data as ContainerSetupInfo).Container.Dockerfile);
}
finally
{
@@ -521,30 +521,30 @@ namespace GitHub.Runner.Common.Tests.Worker
var steps = await _actionManager.PrepareActionsAsync(_ec.Object, actions);
//Assert
Assert.Equal((steps[0].Data as ContainerSetupInfo).StepIds[0], actionId1);
Assert.Equal((steps[0].Data as ContainerSetupInfo).Container.Image, "ubuntu:16.04");
Assert.Equal(actionId1, (steps[0].Data as ContainerSetupInfo).StepIds[0]);
Assert.Equal("ubuntu:16.04", (steps[0].Data as ContainerSetupInfo).Container.Image);
Assert.True((steps[1].Data as ContainerSetupInfo).StepIds.Contains(actionId2));
Assert.True((steps[1].Data as ContainerSetupInfo).StepIds.Contains(actionId3));
Assert.True((steps[1].Data as ContainerSetupInfo).StepIds.Contains(actionId4));
Assert.Equal((steps[1].Data as ContainerSetupInfo).Container.Image, "ubuntu:18.04");
Assert.Contains(actionId2, (steps[1].Data as ContainerSetupInfo).StepIds);
Assert.Contains(actionId3, (steps[1].Data as ContainerSetupInfo).StepIds);
Assert.Contains(actionId4, (steps[1].Data as ContainerSetupInfo).StepIds);
Assert.Equal("ubuntu:18.04", (steps[1].Data as ContainerSetupInfo).Container.Image);
var actionDir = Path.Combine(_hc.GetDirectory(WellKnownDirectory.Actions), "TingluoHuang", "runner_L0", "repositoryactionwithdockerfile");
Assert.Equal((steps[2].Data as ContainerSetupInfo).StepIds[0], actionId5);
Assert.Equal((steps[2].Data as ContainerSetupInfo).Container.WorkingDirectory, actionDir);
Assert.Equal((steps[2].Data as ContainerSetupInfo).Container.Dockerfile, Path.Combine(actionDir, "Dockerfile"));
Assert.Equal(actionId5, (steps[2].Data as ContainerSetupInfo).StepIds[0]);
Assert.Equal(actionDir, (steps[2].Data as ContainerSetupInfo).Container.WorkingDirectory);
Assert.Equal(Path.Combine(actionDir, "Dockerfile"), (steps[2].Data as ContainerSetupInfo).Container.Dockerfile);
actionDir = Path.Combine(_hc.GetDirectory(WellKnownDirectory.Actions), "TingluoHuang", "runner_L0", "repositoryactionwithdockerfileinrelativepath");
Assert.True((steps[3].Data as ContainerSetupInfo).StepIds.Contains(actionId6));
Assert.True((steps[3].Data as ContainerSetupInfo).StepIds.Contains(actionId7));
Assert.Equal((steps[3].Data as ContainerSetupInfo).Container.WorkingDirectory, actionDir);
Assert.Equal((steps[3].Data as ContainerSetupInfo).Container.Dockerfile, Path.Combine(actionDir, "Dockerfile"));
Assert.Contains(actionId6, (steps[3].Data as ContainerSetupInfo).StepIds);
Assert.Contains(actionId7, (steps[3].Data as ContainerSetupInfo).StepIds);
Assert.Equal(actionDir, (steps[3].Data as ContainerSetupInfo).Container.WorkingDirectory);
Assert.Equal(Path.Combine(actionDir, "Dockerfile"), (steps[3].Data as ContainerSetupInfo).Container.Dockerfile);
Assert.Equal((steps[4].Data as ContainerSetupInfo).StepIds[0], actionId8);
Assert.Equal((steps[4].Data as ContainerSetupInfo).Container.WorkingDirectory, actionDir);
Assert.Equal((steps[4].Data as ContainerSetupInfo).Container.Dockerfile, Path.Combine(actionDir, "images/cli", "Dockerfile"));
Assert.Equal(actionId8, (steps[4].Data as ContainerSetupInfo).StepIds[0]);
Assert.Equal(actionDir, (steps[4].Data as ContainerSetupInfo).Container.WorkingDirectory);
Assert.Equal(Path.Combine(actionDir, "images/cli", "Dockerfile"), (steps[4].Data as ContainerSetupInfo).Container.Dockerfile);
}
finally
{
@@ -672,7 +672,7 @@ namespace GitHub.Runner.Common.Tests.Worker
name: 'Hello World'
description: 'Greet the world and record the time'
author: 'GitHub'
inputs:
inputs:
greeting: # id of input
description: 'The greeting we choose - will print ""{greeting}, World!"" on stdout'
required: true
@@ -772,7 +772,7 @@ runs:
name: 'Hello World'
description: 'Greet the world and record the time'
author: 'GitHub'
inputs:
inputs:
greeting: # id of input
description: 'The greeting we choose - will print ""{greeting}, World!"" on stdout'
required: true
@@ -871,7 +871,7 @@ runs:
name: 'Hello World'
description: 'Greet the world and record the time'
author: 'GitHub'
inputs:
inputs:
greeting: # id of input
description: 'The greeting we choose - will print ""{greeting}, World!"" on stdout'
required: true
@@ -940,7 +940,7 @@ runs:
name: 'Hello World'
description: 'Greet the world and record the time'
author: 'GitHub'
inputs:
inputs:
greeting: # id of input
description: 'The greeting we choose - will print ""{greeting}, World!"" on stdout'
required: true
@@ -1039,7 +1039,7 @@ runs:
name: 'Hello World'
description: 'Greet the world and record the time'
author: 'GitHub'
inputs:
inputs:
greeting: # id of input
description: 'The greeting we choose - will print ""{greeting}, World!"" on stdout'
required: true
@@ -1137,7 +1137,7 @@ runs:
name: 'Hello World'
description: 'Greet the world and record the time'
author: 'GitHub'
inputs:
inputs:
greeting: # id of input
description: 'The greeting we choose - will print ""{greeting}, World!"" on stdout'
required: true
@@ -1205,7 +1205,7 @@ runs:
name: 'Hello World'
description: 'Greet the world and record the time'
author: 'GitHub'
inputs:
inputs:
greeting: # id of input
description: 'The greeting we choose - will print ""{greeting}, World!"" on stdout'
required: true
@@ -1276,7 +1276,7 @@ runs:
name: 'Hello World'
description: 'Greet the world and record the time'
author: 'GitHub'
inputs:
inputs:
greeting: # id of input
description: 'The greeting we choose - will print ""{greeting}, World!"" on stdout'
required: true
@@ -1376,7 +1376,7 @@ runs:
name: 'Hello World'
description: 'Greet the world and record the time'
author: 'Test Corporation'
inputs:
inputs:
greeting: # id of input
description: 'The greeting we choose - will print ""{greeting}, World!"" on stdout'
required: true

View File

@@ -8,6 +8,7 @@ using System.Threading.Tasks;
using System.Runtime.CompilerServices;
using GitHub.Runner.Sdk;
using GitHub.Runner.Worker;
using GitHub.Runner.Worker.Container;
using GitHub.Runner.Worker.Handlers;
using Moq;
using Xunit;
@@ -748,6 +749,130 @@ namespace GitHub.Runner.Common.Tests.Worker
Environment.SetEnvironmentVariable("RUNNER_TEST_GET_REPOSITORY_PATH_FAILSAFE", "");
}
#if OS_LINUX
[Fact]
[Trait("Level", "L0")]
[Trait("Category", "Worker")]
public async void MatcherFile_JobContainer()
{
var matchers = new IssueMatchersConfig
{
Matchers =
{
new IssueMatcherConfig
{
Owner = "my-matcher-1",
Patterns = new[]
{
new IssuePatternConfig
{
Pattern = @"(.+): (.+)",
File = 1,
Message = 2,
},
},
},
},
};
var container = new ContainerInfo();
using (var hostContext = Setup(matchers: matchers, jobContainer: container))
using (_outputManager)
{
// Setup github.workspace, github.repository
var workDirectory = hostContext.GetDirectory(WellKnownDirectory.Work);
ArgUtil.NotNullOrEmpty(workDirectory, nameof(workDirectory));
Directory.CreateDirectory(workDirectory);
var workspaceDirectory = Path.Combine(workDirectory, "workspace");
Directory.CreateDirectory(workspaceDirectory);
_executionContext.Setup(x => x.GetGitHubContext("workspace")).Returns(workspaceDirectory);
_executionContext.Setup(x => x.GetGitHubContext("repository")).Returns("my-org/workflow-repo");
// Setup a git repository
await CreateRepository(hostContext, workspaceDirectory, "https://github.com/my-org/workflow-repo");
// Create test files
var file = Path.Combine(workspaceDirectory, "some-file.txt");
File.WriteAllText(file, "");
// Add translation path
container.AddPathTranslateMapping(workspaceDirectory, "/container/path/to/workspace");
// Process
Process($"/container/path/to/workspace/some-file.txt: some error 1");
Process($"some-file.txt: some error 2");
Assert.Equal(2, _issues.Count);
Assert.Equal("some error 1", _issues[0].Item1.Message);
Assert.Equal("some-file.txt", _issues[0].Item1.Data["file"]);
Assert.Equal("some error 2", _issues[1].Item1.Message);
Assert.Equal("some-file.txt", _issues[1].Item1.Data["file"]);
}
}
[Fact]
[Trait("Level", "L0")]
[Trait("Category", "Worker")]
public async void MatcherFile_StepContainer()
{
var matchers = new IssueMatchersConfig
{
Matchers =
{
new IssueMatcherConfig
{
Owner = "my-matcher-1",
Patterns = new[]
{
new IssuePatternConfig
{
Pattern = @"(.+): (.+)",
File = 1,
Message = 2,
},
},
},
},
};
var container = new ContainerInfo();
using (var hostContext = Setup(matchers: matchers, stepContainer: container))
using (_outputManager)
{
// Setup github.workspace, github.repository
var workDirectory = hostContext.GetDirectory(WellKnownDirectory.Work);
ArgUtil.NotNullOrEmpty(workDirectory, nameof(workDirectory));
Directory.CreateDirectory(workDirectory);
var workspaceDirectory = Path.Combine(workDirectory, "workspace");
Directory.CreateDirectory(workspaceDirectory);
_executionContext.Setup(x => x.GetGitHubContext("workspace")).Returns(workspaceDirectory);
_executionContext.Setup(x => x.GetGitHubContext("repository")).Returns("my-org/workflow-repo");
// Setup a git repository
await CreateRepository(hostContext, workspaceDirectory, "https://github.com/my-org/workflow-repo");
// Create test files
var file = Path.Combine(workspaceDirectory, "some-file.txt");
File.WriteAllText(file, "");
// Add translation path
container.AddPathTranslateMapping(workspaceDirectory, "/container/path/to/workspace");
// Process
Process($"/container/path/to/workspace/some-file.txt: some error 1");
Process($"some-file.txt: some error 2");
Assert.Equal(2, _issues.Count);
Assert.Equal("some error 1", _issues[0].Item1.Message);
Assert.Equal("some-file.txt", _issues[0].Item1.Data["file"]);
Assert.Equal("some error 2", _issues[1].Item1.Message);
Assert.Equal("some-file.txt", _issues[1].Item1.Data["file"]);
}
}
#endif
[Fact]
[Trait("Level", "L0")]
[Trait("Category", "Worker")]
@@ -806,7 +931,9 @@ namespace GitHub.Runner.Common.Tests.Worker
private TestHostContext Setup(
[CallerMemberName] string name = "",
IssueMatchersConfig matchers = null)
IssueMatchersConfig matchers = null,
ContainerInfo jobContainer = null,
ContainerInfo stepContainer = null)
{
matchers?.Validate();
@@ -824,6 +951,8 @@ namespace GitHub.Runner.Common.Tests.Worker
.Returns(true);
_executionContext.Setup(x => x.Variables)
.Returns(_variables);
_executionContext.Setup(x => x.Container)
.Returns(jobContainer);
_executionContext.Setup(x => x.GetMatchers())
.Returns(matchers?.Matchers ?? new List<IssueMatcherConfig>());
_executionContext.Setup(x => x.Add(It.IsAny<OnMatcherChanged>()))
@@ -856,7 +985,7 @@ namespace GitHub.Runner.Common.Tests.Worker
return false;
});
_outputManager = new OutputManager(_executionContext.Object, _commandManager.Object);
_outputManager = new OutputManager(_executionContext.Object, _commandManager.Object, stepContainer);
return hostContext;
}

View File

@@ -1,99 +0,0 @@
$ErrorActionPreference = "Stop"
# Push-Location -Path .\Sdk
try
{
# Generate Namespaces.cs
Write-Host 'Generating Namespaces.cs'
Remove-Item -Path Sdk\Namespaces.cs -ErrorAction Ignore
$namespaces = New-Object System.Collections.Generic.HashSet[string]
$output = findstr /snir /c:"^namespace " Sdk\*.cs
foreach ($outputLine in ($output.Trim().Replace("`r", "").Split("`n")))
{
$namespace = $outputLine.Trim().Split(':')[-1].Split(' ')[-1]
$namespaces.Add($namespace) | Out-Null
}
$namespaces = New-Object System.Collections.Generic.List[string]($namespaces)
$namespaces.Sort()
$content = New-Object System.Text.StringBuilder
foreach ($namespace in $namespaces)
{
$content.AppendLine("namespace $namespace") | Out-Null
$content.AppendLine("{") | Out-Null
$content.AppendLine("}") | Out-Null
$content.AppendLine("") | Out-Null
}
[System.IO.File]::WriteAllText("$pwd\Sdk\Namespaces.cs", $content.ToString(), (New-Object System.Text.UTF8Encoding($false)))
# Gather whitelist of files not to delete
Write-Host 'Gathering whitelist of files not to delete'
$whitelist = New-Object System.Collections.Generic.HashSet[string]
$whitelist.Add(((Resolve-Path -Path Sdk\Namespaces.cs).Path)) | Out-Null
foreach ($file in (Get-ChildItem -Path Sdk\DTExpressions -Recurse -Filter *.cs))
{
$whitelist.Add($file.FullName) | Out-Null
}
foreach ($file in (Get-ChildItem -Path Sdk\DTLogging -Recurse -Filter *.cs))
{
$whitelist.Add($file.FullName) | Out-Null
}
foreach ($file in (Get-ChildItem -Path Sdk\DTObjectTemplating -Recurse -Filter *.cs))
{
$whitelist.Add($file.FullName) | Out-Null
}
foreach ($file in (Get-ChildItem -Path Sdk\DTPipelines\Pipelines\ContextData -Recurse -Filter *.cs))
{
$whitelist.Add($file.FullName) | Out-Null
}
foreach ($file in (Get-ChildItem -Path Sdk\DTPipelines\Pipelines\ObjectTemplating -Recurse -Filter *.cs))
{
$whitelist.Add($file.FullName) | Out-Null
}
# Gather candidate files to delete
Write-Host 'Gathering candidate files to delete'
$candidatePaths = New-Object System.Collections.Generic.List[string]
$deletedPaths = New-Object System.Collections.Generic.List[string]
foreach ($candidateFile in (Get-ChildItem -Path Sdk -Recurse -Filter *.cs))
{
if (!$whitelist.Contains($candidateFile.FullName) -and (($candidateFile.FullName.IndexOf('\obj\')) -le 0))
{
$candidatePaths.Add($candidateFile.FullName)
}
}
while ($true)
{
$found = $false
for ($i = 0; $i -lt $candidatePaths.Count; )
{
$candidatePath = $candidatePaths[$i]
Write-Host "Checking $candidatePath"
Remove-Item -Path $candidatePath
.\dev.cmd build
if ($LASTEXITCODE -eq 0)
{
$deletedPaths.Add($candidatePath)
$candidatePaths.RemoveAt($i)
Write-Host "Successfully deleted $candidatePath"
$found = $true
}
else
{
Write-Host "Undeleting $candidatePath"
git checkout -- $candidatePath
$i++
}
}
if (!$found)
{
break;
}
}
}
finally
{
# Pop-Location
}

View File

@@ -109,13 +109,13 @@ function heading()
function build ()
{
heading "Building ..."
dotnet msbuild -t:Build -p:PackageRuntime="${RUNTIME_ID}" -p:BUILDCONFIG="${BUILD_CONFIG}" -p:RunnerVersion="${RUNNER_VERSION}" || failed build
dotnet msbuild -t:Build -p:PackageRuntime="${RUNTIME_ID}" -p:BUILDCONFIG="${BUILD_CONFIG}" -p:RunnerVersion="${RUNNER_VERSION}" ./dir.proj || failed build
}
function layout ()
{
heading "Create layout ..."
dotnet msbuild -t:layout -p:PackageRuntime="${RUNTIME_ID}" -p:BUILDCONFIG="${BUILD_CONFIG}" -p:RunnerVersion="${RUNNER_VERSION}" || failed build
dotnet msbuild -t:layout -p:PackageRuntime="${RUNTIME_ID}" -p:BUILDCONFIG="${BUILD_CONFIG}" -p:RunnerVersion="${RUNNER_VERSION}" ./dir.proj || failed build
#change execution flag to allow running with sudo
if [[ ("$CURRENT_PLATFORM" == "linux") || ("$CURRENT_PLATFORM" == "darwin") ]]; then
@@ -137,9 +137,7 @@ function runtest ()
ulimit -n 1024
fi
export GITHUB_RUNNER_SRC_DIR=${SCRIPT_DIR}
dotnet msbuild -t:test -p:PackageRuntime="${RUNTIME_ID}" -p:BUILDCONFIG="${BUILD_CONFIG}" -p:RunnerVersion="${RUNNER_VERSION}" || failed "failed tests"
dotnet msbuild -t:test -p:PackageRuntime="${RUNTIME_ID}" -p:BUILDCONFIG="${BUILD_CONFIG}" -p:RunnerVersion="${RUNNER_VERSION}" ./dir.proj || failed "failed tests"
}
function package ()