Compare commits

..

2 Commits

Author SHA1 Message Date
Tingluo Huang
e80ab803d6 c 2020-01-22 16:34:15 -05:00
Tingluo Huang
4f40817c82 Set both http_proxy and HTTP_PROXY env for runner/worker processes. 2020-01-22 11:44:49 -05:00
57 changed files with 1253 additions and 1434 deletions

View File

@@ -43,6 +43,14 @@ jobs:
steps: steps:
- uses: actions/checkout@v1 - uses: actions/checkout@v1
# Set Path workaround for https://github.com/actions/virtual-environments/issues/263
- run: |
echo "::add-path::C:\Program Files\Git\mingw64\bin"
echo "::add-path::C:\Program Files\Git\usr\bin"
echo "::add-path::C:\Program Files\Git\bin"
if: matrix.os == 'windows-latest'
name: "Temp step to Set Path for Windows"
# Build runner layout # Build runner layout
- name: Build & Layout Release - name: Build & Layout Release
run: | run: |

View File

@@ -15,7 +15,6 @@ User wants to reference workflow variables defined in workflow yaml file for act
Runner will create and populate the `env` context for every job execution using following logic: Runner will create and populate the `env` context for every job execution using following logic:
1. On job start, create `env` context with any environment variables in the job message, these are env defined in customer's YAML file's job/workflow level `env` section. 1. On job start, create `env` context with any environment variables in the job message, these are env defined in customer's YAML file's job/workflow level `env` section.
2. Update `env` context when customer use `::set-env::` to set env at the runner level. 2. Update `env` context when customer use `::set-env::` to set env at the runner level.
3. Update `env` context with step's `env` block before each step runs.
The `env` context is only available in the runner, customer can't use the `env` context in any server evaluation part, just like the `runner` context The `env` context is only available in the runner, customer can't use the `env` context in any server evaluation part, just like the `runner` context
@@ -23,25 +22,17 @@ Example yaml:
```yaml ```yaml
env: env:
env1: 10 env1: 100
env2: 20
env3: 30
jobs: jobs:
build: build:
env: env:
env1: 100
env2: 200 env2: 200
runs-on: ubuntu-latest runs-on: ubuntu-latest
steps: steps:
- run: | - run: |
echo ${{ env.env1 }} // 1000 echo ${{ env.env1 }}
echo $env1 // 1000 if: env.env2 == 200
echo $env2 // 200 name: ${{ env.env1 }}_${{ env.env2 }}
echo $env3 // 30
if: env.env2 == 200 // true
name: ${{ env.env1 }}_${{ env.env2 }} //1000_200
env:
env1: 1000
``` ```
### Don't populate the `env` context with environment variables from runner machine. ### Don't populate the `env` context with environment variables from runner machine.
@@ -57,4 +48,4 @@ build:
- uses: docker://ubuntu:18.04 - uses: docker://ubuntu:18.04
with: with:
args: echo ${{env.USER}} <- what should customer expect this output? runner/root args: echo ${{env.USER}} <- what should customer expect this output? runner/root
``` ```

View File

@@ -1,48 +0,0 @@
# ADR 0297: Base64 Masking Trailing Characters
**Date** 2020-01-21
**Status** Proposed
## Context
The Runner registers a number of Value Encoders, which mask various encodings of a provided secret. Currently, we register a 3 base64 Encoders:
- The base64 encoded secret
- The secret with the first character removed then base64 encoded
- The secret with the first two characters removed then base64 encoded
This gives us good coverage across the board for secrets and secrets with a prefix (i.e. `base64($user:$pass)`).
However, we don't have great coverage for cases where the secret has a string appended to it before it is base64 encoded (i.e.: `base64($pass\n))`).
Most notably we've seen this as a result of user error where a user accidentially appends a newline or space character before encoding their secret in base64.
## Decision
### Trim end characters
We are going to modify all existing base64 encoders to trim information before registering as a secret.
We will trim:
- `=` from the end of all base64 strings. This is a padding character that contains no information.
- Based on the number of `=`'s at the end of a base64 string, a malicious user could predict the length of the original secret modulo 3.
- If a user saw `***==`, they would know the secret could be 1,4,7,10... characters.
- If a string contains `=` we will also trim the last non-padding character from the base64 secret.
- This character can change if a string is appended to the secret before the encoding.
### Register a fourth encoder
We will also add back in the original base64 encoded secret encoder for four total encoders:
- The base64 encoded secret
- The base64 encoded secret trimmed
- The secret with the first character removed then base64 encoded and trimmed
- The secret with the first two characters removed then base64 encoded and trimmed
This allows us to fully cover the most common scenario where a user base64 encodes their secret and expects the entire thing to be masked.
This will result in us only revealing length or bit information when a prefix or suffix is added to a secret before encoding.
## Consequences
- In the case where a secret has a prefix or suffix added before base64 encoding, we may now reveal up to 20 bits of information and the length of the original string modulo 3, rather then the original 16 bits and no length information
- Secrets with a suffix appended before encoding will now be masked across the board. Previously it was only masked if it was a multiple of 3 characters
- Performance will suffer in a neglible way

View File

@@ -1,61 +0,0 @@
# Runner Authentication and Authorization
## Goals
- Support runner installs in untrusted domains.
- The account that configures or runs the runner process is not relevant for accessing GitHub resources.
- Accessing GitHub resources is done with a per-job token which expires when job completes.
- The token is granted to trusted parts of the system including the runner, actions and script steps specified by the workflow author as trusted.
- All OAuth tokens that come from the Token Service that the runner uses to access Actions Service resources are the same. It's just the scope and expiration of the token that may vary.
## Configuration
Configuring a self-hosted runner is [covered here in the documentation](https://help.github.com/en/actions/hosting-your-own-runners/adding-self-hosted-runners).
Configuration is done with the user being authenticated via a time-limited, GitHub runner registration token.
*Your credentials are never used for registering the runner with the service.*
![Self-hosted runner config](../res/self-hosted-config.png)
During configuration, an RSA public/private key pair is created, the private key is stored in file on disk. On Windows, the content is protected with DPAPI (machine level encrypted - runner only valid on that machine) and on Linux/OSX with `chmod` permissions.
Using your credentials, the runner is registered with the service by sending the public key to the service which adds that runner to the pool and stores the public key, the Token Service will generate a `clientId` associated with the public key.
## Start and Listen
After configuring the runner, the runner can be started interactively (`./run.cmd` or `./run.sh`) or as a service.
![Self-hosted runner start](../res/self-hosted-start.png)
On start, the runner listener process loads the RSA private key (on Windows decrypting with machine key DPAPI), and asks the Token Service for an OAuth token which is signed with the RSA private key.
The server then responds with an OAuth token that grants permission to access the message queue (HTTP long poll), allowing the runner to acquire the messages it will eventually run.
## Run a workflow
When a workflow is run, its labels are evaluated, it is matched to a runner and a message is placed in a queue of messages for that runner.
The runner then starts listening for jobs via the message queue HTTP long poll.
The message is encrypted with the runner's public key, stored during runner configuration.
![Runner workflow run](../res/workflow-run.png)
A workflow is queued as a result of a triggered [event](https://help.github.com/en/actions/reference/events-that-trigger-workflows). Workflows can be scheduled to [run at specific UTC times](https://help.github.com/en/actions/reference/events-that-trigger-workflows#scheduled-events-schedule) using POSIX `cron` syntax.
An [OAuth token](http://self-issued.info/docs/draft-ietf-oauth-json-web-token.html) is generated, granting limited access to the host in Actions Service associated with the github.com repository/organization.
The lifetime of the OAuth token is the lifetime of the run or at most the [job timeout (default: 6 hours)](https://help.github.com/en/actions/reference/workflow-syntax-for-github-actions#jobsjob_idtimeout-minutes), plus 10 additional minutes.
## Accessing GitHub resources
The job message sent to the runner contains the OAuth token to talk back to the Actions Service.
The runner listener parent process will spawn a runner worker process for that job and send it the job message over IPC.
The token is never persisted.
Each action is run as a unique subprocess.
The encrypted access token will be provided as an environment variable in each action subprocess.
The token is registered with the runner as a secret and scrubbed from the logs as they are written.
Authentication in a workflow run to github.com can be accomplished by using the [`GITHUB_TOKEN`](https://help.github.com/en/actions/configuring-and-managing-workflows/authenticating-with-the-github_token#about-the-github_token-secret)) secret. This token expires after 60 minutes. Please note that this token is different from the OAuth token that the runner uses to talk to the Actions Service.
## Hosted runner authentication
Hosted runner authentication differs from self-hosted authentication in that runners do not undergo a registration process, but instead, the hosted runners get the OAuth token directly by reading the `.credentials` file. The scope of this particular token is limited for a given workflow job execution, and the token is revoked as soon as the job is finished.
![Hosted runner config and start](../res/hosted-config-start.png)

Binary file not shown.

Before

Width:  |  Height:  |  Size: 31 KiB

View File

@@ -1,52 +0,0 @@
# Markup used to generate the runner auth diagrams: https://websequencediagrams.com
title Runner Configuration (self-hosted only)
note left of Runner: GitHub repo URL as input
Runner->github.com: Retrieve Actions Service access using runner registration token
github.com->Runner: Access token for Actions Service
note left of Runner: Generate RSA key pair
note left of Runner: Store encrypted RSA private key on disk
Runner->Actions Service: Register runner using Actions Service access token
note right of Runner: Runner name, RSA public key sent
note right of Actions Service: Public key stored
Actions Service->Token Service: Register runner as an app along with the RSA public key
note right of Token Service: Public key stored
Token Service->Actions Service: Client Id for the runner application
Actions Service->Runner: Client Id and Token Endpoint URL
note left of Runner: Store runner configuration info into .runner file
note left of Runner: Store Token registration info into .credentials file
title Runner Start and Running (self-hosted only)
Runner.Listener->Runner.Listener: Start
note left of Runner.Listener: Load config info from .runner
note left of Runner.Listener: Load token registration from .credentials
Runner.Listener->Token Service: Exchange OAuth token (happens every 50 mins)
note right of Runner.Listener: Construct JWT token, use Client Id signed by RSA private key
note left of Actions Service: Find corresponding RSA public key, use Client Id\nVerify JWT token's signature
Token Service->Runner.Listener: OAuth token with limited permission and valid for 50 mins
Runner.Listener->Actions Service: Connect to Actions Service with OAuth token
Actions Service->Runner.Listener: Workflow job
title Running workflow
Runner.Listener->Service (Message Queue): Get message
note right of Runner.Listener: Authenticate with exchanged OAuth token
Event->Actions Service: Queue workflow
Actions Service->Actions Service: Generate OAuth token per job
Actions Service->Actions Service: Build job message with the OAuth token
Actions Service->Actions Service: Encrypt job message with the target runner's public key
Actions Service->Service (Message Queue): Send encrypted job message to runner
Service (Message Queue)->Runner.Listener: Send job
note right of Runner.Listener: Decrypt message with runner's private key
Runner.Listener->Runner.Worker: Create worker process per job and run the job
title Runner Configuration, Start and Running (hosted only)
Machine Management Service->Runner.Listener: Construct .runner configuration file, store token in .credentials
Runner.Listener->Runner.Listener: Start
note left of Runner.Listener: Load config info from .runner
note left of Runner.Listener: Load OAuth token from .credentials
Runner.Listener->Actions Service: Connect to Actions Service with OAuth token in .credentials
Actions Service->Runner.Listener: Workflow job

Binary file not shown.

Before

Width:  |  Height:  |  Size: 98 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 43 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 46 KiB

View File

@@ -1,30 +1,17 @@
## Features ## Features
- Expose whether debug is on/off via RUNNER_DEBUG. (#253) - Remove runner flow: Change from PAT to "deletion token" in prompt (#225)
- Upload log on runner when worker get killed due to cancellation timeout. (#255) - Expose github.run_id and github.run_number to action runtime env. (#224)
- Update config.sh/cmd --help documentation (#282)
- Set http_proxy and related env vars for job/service containers (#304)
- Set both http_proxy and HTTP_PROXY env for runner/worker processes. (#298)
## Bugs ## Bugs
- Verify runner Windows service hash started successfully after configuration (#236) - Clean up error messages for container scenarios (#221)
- Detect source file path in L0 without using env. (#257) - Pick shell from prependpath (#231)
- Handle escaped '%' in commands data section (#200)
- Allow container to be null/empty during matrix expansion (#266)
- Translate problem matcher file to host path (#272)
- Change hashFiles() expression function to use @actions/glob. (#268)
- Default post-job action's condition to always(). (#293)
- Support action.yaml file as action's entry file (#288)
- Trace javascript action exit code to debug instead of user logs (#290)
- Change prompt message when removing a runner to lines up with GitHub.com UI (#303)
- Include step.env as part of env context. (#300)
- Update Base64 Encoders to deal with suffixes (#284)
## Misc ## Misc
- Move .sln file under ./src (#238) - Runner code cleanup (#218 #227, #228, #229, #230)
- Treat warnings as errors during compile (#249) - Consume dotnet core 3.1 in runner. (#213)
## Windows x64 ## Windows x64
We recommend configuring the runner in a root folder of the Windows drive (e.g. "C:\actions-runner"). This will help avoid issues related to service identity folder permissions and long file path restrictions on Windows We recommend configuring the runner under "<DRIVE>:\actions-runner". This will help avoid issues related to service identity folder permissions and long file path restrictions on Windows
``` ```
// Create a folder under the drive root // Create a folder under the drive root
mkdir \actions-runner ; cd \actions-runner mkdir \actions-runner ; cd \actions-runner
@@ -32,7 +19,7 @@ mkdir \actions-runner ; cd \actions-runner
Invoke-WebRequest -Uri https://github.com/actions/runner/releases/download/v<RUNNER_VERSION>/actions-runner-win-x64-<RUNNER_VERSION>.zip -OutFile actions-runner-win-x64-<RUNNER_VERSION>.zip Invoke-WebRequest -Uri https://github.com/actions/runner/releases/download/v<RUNNER_VERSION>/actions-runner-win-x64-<RUNNER_VERSION>.zip -OutFile actions-runner-win-x64-<RUNNER_VERSION>.zip
// Extract the installer // Extract the installer
Add-Type -AssemblyName System.IO.Compression.FileSystem ; Add-Type -AssemblyName System.IO.Compression.FileSystem ;
[System.IO.Compression.ZipFile]::ExtractToDirectory("$PWD\actions-runner-win-x64-<RUNNER_VERSION>.zip", "$PWD") [System.IO.Compression.ZipFile]::ExtractToDirectory("$HOME\Downloads\actions-runner-win-x64-<RUNNER_VERSION>.zip", "$PWD")
``` ```
## OSX ## OSX

View File

@@ -3,6 +3,8 @@
<packageSources> <packageSources>
<!--To inherit the global NuGet package sources remove the <clear/> line below --> <!--To inherit the global NuGet package sources remove the <clear/> line below -->
<clear /> <clear />
<add key="dotnet-core" value="https://www.myget.org/F/dotnet-core/api/v3/index.json" />
<add key="dotnet-buildtools" value="https://www.myget.org/F/dotnet-buildtools/api/v3/index.json" />
<add key="api.nuget.org" value="https://api.nuget.org/v3/index.json" /> <add key="api.nuget.org" value="https://api.nuget.org/v3/index.json" />
</packageSources> </packageSources>
</configuration> </configuration>

View File

@@ -136,12 +136,6 @@ namespace GitHub.Runner.Common
} }
} }
public static class RunnerEvent
{
public static readonly string Register = "register";
public static readonly string Remove = "remove";
}
public static class Pipeline public static class Pipeline
{ {
public static class Path public static class Path

View File

@@ -190,7 +190,7 @@ namespace GitHub.Runner.Listener
{ {
return GetArgOrPrompt( return GetArgOrPrompt(
name: Constants.Runner.CommandLine.Args.Token, name: Constants.Runner.CommandLine.Args.Token,
description: "Enter runner remove token:", description: "Enter runner deletion token:",
defaultValue: string.Empty, defaultValue: string.Empty,
validator: Validators.NonEmptyValidator); validator: Validators.NonEmptyValidator);
} }
@@ -291,7 +291,7 @@ namespace GitHub.Runner.Listener
if (!string.IsNullOrEmpty(result)) if (!string.IsNullOrEmpty(result))
{ {
// After read the arg from input commandline args, remove it from Arg dictionary, // After read the arg from input commandline args, remove it from Arg dictionary,
// This will help if bad arg value passed through CommandLine arg, when ConfigurationManager ask CommandSetting the second time, // This will help if bad arg value passed through CommandLine arg, when ConfigurationManager ask CommandSetting the second time,
// It will prompt for input instead of continue use the bad input. // It will prompt for input instead of continue use the bad input.
_trace.Info($"Remove {name} from Arg dictionary."); _trace.Info($"Remove {name} from Arg dictionary.");
RemoveArg(name); RemoveArg(name);

View File

@@ -1,18 +1,19 @@
using GitHub.DistributedTask.WebApi; using GitHub.DistributedTask.WebApi;
using GitHub.Runner.Common;
using GitHub.Runner.Common.Util; using GitHub.Runner.Common.Util;
using GitHub.Runner.Sdk;
using GitHub.Services.Common; using GitHub.Services.Common;
using GitHub.Services.OAuth; using GitHub.Services.OAuth;
using GitHub.Services.WebApi; using GitHub.Services.WebApi;
using System; using System;
using System.Collections.Generic; using System.Collections.Generic;
using System.IO;
using System.Linq; using System.Linq;
using System.Net.Http;
using System.Net.Http.Headers;
using System.Runtime.InteropServices;
using System.Security.Cryptography; using System.Security.Cryptography;
using System.Threading.Tasks; using System.Threading.Tasks;
using System.Runtime.InteropServices;
using GitHub.Runner.Common;
using GitHub.Runner.Sdk;
using System.Net.Http;
using System.Net.Http.Headers;
namespace GitHub.Runner.Listener.Configuration namespace GitHub.Runner.Listener.Configuration
{ {
@@ -108,7 +109,7 @@ namespace GitHub.Runner.Listener.Configuration
{ {
runnerSettings.GitHubUrl = inputUrl; runnerSettings.GitHubUrl = inputUrl;
var githubToken = command.GetRunnerRegisterToken(); var githubToken = command.GetRunnerRegisterToken();
GitHubAuthResult authResult = await GetTenantCredential(inputUrl, githubToken, Constants.RunnerEvent.Register); GitHubAuthResult authResult = await GetTenantCredential(inputUrl, githubToken);
runnerSettings.ServerUrl = authResult.TenantUrl; runnerSettings.ServerUrl = authResult.TenantUrl;
creds = authResult.ToVssCredentials(); creds = authResult.ToVssCredentials();
Trace.Info("cred retrieved via GitHub auth"); Trace.Info("cred retrieved via GitHub auth");
@@ -276,15 +277,12 @@ namespace GitHub.Runner.Listener.Configuration
throw new NotSupportedException("Message queue listen OAuth token."); throw new NotSupportedException("Message queue listen OAuth token.");
} }
// Testing agent connection, detect any potential connection issue, like local clock skew that cause OAuth token expired. // Testing agent connection, detect any protential connection issue, like local clock skew that cause OAuth token expired.
var credMgr = HostContext.GetService<ICredentialManager>(); var credMgr = HostContext.GetService<ICredentialManager>();
VssCredentials credential = credMgr.LoadCredentials(); VssCredentials credential = credMgr.LoadCredentials();
try try
{ {
await _runnerServer.ConnectAsync(new Uri(runnerSettings.ServerUrl), credential); await _runnerServer.ConnectAsync(new Uri(runnerSettings.ServerUrl), credential);
// ConnectAsync() hits _apis/connectionData which is an anonymous endpoint
// Need to hit an authenticate endpoint to trigger OAuth token exchange.
await _runnerServer.GetAgentPoolsAsync();
_term.WriteSuccessMessage("Runner connection is good"); _term.WriteSuccessMessage("Runner connection is good");
} }
catch (VssOAuthTokenRequestException ex) when (ex.Message.Contains("Current server time is")) catch (VssOAuthTokenRequestException ex) when (ex.Message.Contains("Current server time is"))
@@ -375,7 +373,7 @@ namespace GitHub.Runner.Listener.Configuration
else else
{ {
var githubToken = command.GetRunnerDeletionToken(); var githubToken = command.GetRunnerDeletionToken();
GitHubAuthResult authResult = await GetTenantCredential(settings.GitHubUrl, githubToken, Constants.RunnerEvent.Remove); GitHubAuthResult authResult = await GetTenantCredential(settings.GitHubUrl, githubToken);
creds = authResult.ToVssCredentials(); creds = authResult.ToVssCredentials();
Trace.Info("cred retrieved via GitHub auth"); Trace.Info("cred retrieved via GitHub auth");
} }
@@ -519,23 +517,17 @@ namespace GitHub.Runner.Listener.Configuration
} }
} }
private async Task<GitHubAuthResult> GetTenantCredential(string githubUrl, string githubToken, string runnerEvent) private async Task<GitHubAuthResult> GetTenantCredential(string githubUrl, string githubToken)
{ {
var gitHubUrlBuilder = new UriBuilder(githubUrl); var gitHubUrl = new UriBuilder(githubUrl);
var githubApiUrl = $"{gitHubUrlBuilder.Scheme}://api.{gitHubUrlBuilder.Host}/actions/runner-registration"; var githubApiUrl = $"https://api.{gitHubUrl.Host}/repos/{gitHubUrl.Path.Trim('/')}/actions-runners/registration";
using (var httpClientHandler = HostContext.CreateHttpClientHandler()) using (var httpClientHandler = HostContext.CreateHttpClientHandler())
using (var httpClient = new HttpClient(httpClientHandler)) using (var httpClient = new HttpClient(httpClientHandler))
{ {
httpClient.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("RemoteAuth", githubToken); httpClient.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("RemoteAuth", githubToken);
httpClient.DefaultRequestHeaders.UserAgent.Add(HostContext.UserAgent); httpClient.DefaultRequestHeaders.UserAgent.Add(HostContext.UserAgent);
httpClient.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/vnd.github.shuri-preview+json"));
var bodyObject = new Dictionary<string, string>() var response = await httpClient.PostAsync(githubApiUrl, new StringContent("", null, "application/json"));
{
{"url", githubUrl},
{"runner_event", runnerEvent}
};
var response = await httpClient.PostAsync(githubApiUrl, new StringContent(StringUtil.ConvertToJson(bodyObject), null, "application/json"));
if (response.IsSuccessStatusCode) if (response.IsSuccessStatusCode)
{ {

View File

@@ -21,7 +21,6 @@ namespace GitHub.Runner.Sdk
private string _httpsProxyAddress; private string _httpsProxyAddress;
private string _httpsProxyUsername; private string _httpsProxyUsername;
private string _httpsProxyPassword; private string _httpsProxyPassword;
private string _noProxyString;
private readonly List<ByPassInfo> _noProxyList = new List<ByPassInfo>(); private readonly List<ByPassInfo> _noProxyList = new List<ByPassInfo>();
private readonly HashSet<string> _noProxyUnique = new HashSet<string>(StringComparer.OrdinalIgnoreCase); private readonly HashSet<string> _noProxyUnique = new HashSet<string>(StringComparer.OrdinalIgnoreCase);
@@ -34,7 +33,6 @@ namespace GitHub.Runner.Sdk
public string HttpsProxyAddress => _httpsProxyAddress; public string HttpsProxyAddress => _httpsProxyAddress;
public string HttpsProxyUsername => _httpsProxyUsername; public string HttpsProxyUsername => _httpsProxyUsername;
public string HttpsProxyPassword => _httpsProxyPassword; public string HttpsProxyPassword => _httpsProxyPassword;
public string NoProxyString => _noProxyString;
public List<ByPassInfo> NoProxyList => _noProxyList; public List<ByPassInfo> NoProxyList => _noProxyList;
@@ -74,8 +72,8 @@ namespace GitHub.Runner.Sdk
_httpProxyAddress = proxyHttpUri.AbsoluteUri; _httpProxyAddress = proxyHttpUri.AbsoluteUri;
// Set both environment variables since there are tools support both casing (curl, wget) and tools support only one casing (docker) // Set both environment variables since there are tools support both casing (curl, wget) and tools support only one casing (docker)
Environment.SetEnvironmentVariable("HTTP_PROXY", _httpProxyAddress);
Environment.SetEnvironmentVariable("http_proxy", _httpProxyAddress); Environment.SetEnvironmentVariable("http_proxy", _httpProxyAddress);
Environment.SetEnvironmentVariable("HTTP_PROXY", _httpProxyAddress);
// the proxy url looks like http://[user:pass@]127.0.0.1:8888 // the proxy url looks like http://[user:pass@]127.0.0.1:8888
var userInfo = Uri.UnescapeDataString(proxyHttpUri.UserInfo).Split(':', 2, StringSplitOptions.RemoveEmptyEntries); var userInfo = Uri.UnescapeDataString(proxyHttpUri.UserInfo).Split(':', 2, StringSplitOptions.RemoveEmptyEntries);
@@ -104,8 +102,8 @@ namespace GitHub.Runner.Sdk
_httpsProxyAddress = proxyHttpsUri.AbsoluteUri; _httpsProxyAddress = proxyHttpsUri.AbsoluteUri;
// Set both environment variables since there are tools support both casing (curl, wget) and tools support only one casing (docker) // Set both environment variables since there are tools support both casing (curl, wget) and tools support only one casing (docker)
Environment.SetEnvironmentVariable("HTTPS_PROXY", _httpsProxyAddress);
Environment.SetEnvironmentVariable("https_proxy", _httpsProxyAddress); Environment.SetEnvironmentVariable("https_proxy", _httpsProxyAddress);
Environment.SetEnvironmentVariable("HTTPS_PROXY", _httpsProxyAddress);
// the proxy url looks like http://[user:pass@]127.0.0.1:8888 // the proxy url looks like http://[user:pass@]127.0.0.1:8888
var userInfo = Uri.UnescapeDataString(proxyHttpsUri.UserInfo).Split(':', 2, StringSplitOptions.RemoveEmptyEntries); var userInfo = Uri.UnescapeDataString(proxyHttpsUri.UserInfo).Split(':', 2, StringSplitOptions.RemoveEmptyEntries);
@@ -131,11 +129,9 @@ namespace GitHub.Runner.Sdk
if (!string.IsNullOrEmpty(noProxyList)) if (!string.IsNullOrEmpty(noProxyList))
{ {
_noProxyString = noProxyList;
// Set both environment variables since there are tools support both casing (curl, wget) and tools support only one casing (docker) // Set both environment variables since there are tools support both casing (curl, wget) and tools support only one casing (docker)
Environment.SetEnvironmentVariable("NO_PROXY", noProxyList);
Environment.SetEnvironmentVariable("no_proxy", noProxyList); Environment.SetEnvironmentVariable("no_proxy", noProxyList);
Environment.SetEnvironmentVariable("NO_PROXY", noProxyList);
var noProxyListSplit = noProxyList.Split(',', StringSplitOptions.RemoveEmptyEntries); var noProxyListSplit = noProxyList.Split(',', StringSplitOptions.RemoveEmptyEntries);
foreach (string noProxy in noProxyListSplit) foreach (string noProxy in noProxyListSplit)

View File

@@ -1,7 +1,6 @@
using GitHub.DistributedTask.Pipelines; using GitHub.DistributedTask.Pipelines;
using GitHub.DistributedTask.WebApi; using GitHub.DistributedTask.WebApi;
using GitHub.Runner.Common.Util; using GitHub.Runner.Common.Util;
using GitHub.Runner.Worker.Container;
using System; using System;
using System.Collections.Generic; using System.Collections.Generic;
using System.IO; using System.IO;
@@ -16,14 +15,14 @@ namespace GitHub.Runner.Worker
{ {
void EnablePluginInternalCommand(); void EnablePluginInternalCommand();
void DisablePluginInternalCommand(); void DisablePluginInternalCommand();
bool TryProcessCommand(IExecutionContext context, string input, ContainerInfo container); bool TryProcessCommand(IExecutionContext context, string input);
} }
public sealed class ActionCommandManager : RunnerService, IActionCommandManager public sealed class ActionCommandManager : RunnerService, IActionCommandManager
{ {
private const string _stopCommand = "stop-commands"; private const string _stopCommand = "stop-commands";
private readonly Dictionary<string, IActionCommandExtension> _commandExtensions = new Dictionary<string, IActionCommandExtension>(StringComparer.OrdinalIgnoreCase); private readonly Dictionary<string, IActionCommandExtension> _commandExtensions = new Dictionary<string, IActionCommandExtension>(StringComparer.OrdinalIgnoreCase);
private readonly HashSet<string> _registeredCommands = new HashSet<string>(StringComparer.OrdinalIgnoreCase); private HashSet<string> _registeredCommands = new HashSet<string>(StringComparer.OrdinalIgnoreCase);
private readonly object _commandSerializeLock = new object(); private readonly object _commandSerializeLock = new object();
private bool _stopProcessCommand = false; private bool _stopProcessCommand = false;
private string _stopToken = null; private string _stopToken = null;
@@ -59,7 +58,7 @@ namespace GitHub.Runner.Worker
_registeredCommands.Remove("internal-set-repo-path"); _registeredCommands.Remove("internal-set-repo-path");
} }
public bool TryProcessCommand(IExecutionContext context, string input, ContainerInfo container) public bool TryProcessCommand(IExecutionContext context, string input)
{ {
if (string.IsNullOrEmpty(input)) if (string.IsNullOrEmpty(input))
{ {
@@ -115,7 +114,7 @@ namespace GitHub.Runner.Worker
try try
{ {
extension.ProcessCommand(context, input, actionCommand, container); extension.ProcessCommand(context, input, actionCommand);
} }
catch (Exception ex) catch (Exception ex)
{ {
@@ -141,7 +140,7 @@ namespace GitHub.Runner.Worker
string Command { get; } string Command { get; }
bool OmitEcho { get; } bool OmitEcho { get; }
void ProcessCommand(IExecutionContext context, string line, ActionCommand command, ContainerInfo container); void ProcessCommand(IExecutionContext context, string line, ActionCommand command);
} }
public sealed class InternalPluginSetRepoPathCommandExtension : RunnerService, IActionCommandExtension public sealed class InternalPluginSetRepoPathCommandExtension : RunnerService, IActionCommandExtension
@@ -151,7 +150,7 @@ namespace GitHub.Runner.Worker
public Type ExtensionType => typeof(IActionCommandExtension); public Type ExtensionType => typeof(IActionCommandExtension);
public void ProcessCommand(IExecutionContext context, string line, ActionCommand command, ContainerInfo container) public void ProcessCommand(IExecutionContext context, string line, ActionCommand command)
{ {
if (!command.Properties.TryGetValue(SetRepoPathCommandProperties.repoFullName, out string repoFullName) || string.IsNullOrEmpty(repoFullName)) if (!command.Properties.TryGetValue(SetRepoPathCommandProperties.repoFullName, out string repoFullName) || string.IsNullOrEmpty(repoFullName))
{ {
@@ -181,7 +180,7 @@ namespace GitHub.Runner.Worker
public Type ExtensionType => typeof(IActionCommandExtension); public Type ExtensionType => typeof(IActionCommandExtension);
public void ProcessCommand(IExecutionContext context, string line, ActionCommand command, ContainerInfo container) public void ProcessCommand(IExecutionContext context, string line, ActionCommand command)
{ {
if (!command.Properties.TryGetValue(SetEnvCommandProperties.Name, out string envName) || string.IsNullOrEmpty(envName)) if (!command.Properties.TryGetValue(SetEnvCommandProperties.Name, out string envName) || string.IsNullOrEmpty(envName))
{ {
@@ -206,7 +205,7 @@ namespace GitHub.Runner.Worker
public Type ExtensionType => typeof(IActionCommandExtension); public Type ExtensionType => typeof(IActionCommandExtension);
public void ProcessCommand(IExecutionContext context, string line, ActionCommand command, ContainerInfo container) public void ProcessCommand(IExecutionContext context, string line, ActionCommand command)
{ {
if (!command.Properties.TryGetValue(SetOutputCommandProperties.Name, out string outputName) || string.IsNullOrEmpty(outputName)) if (!command.Properties.TryGetValue(SetOutputCommandProperties.Name, out string outputName) || string.IsNullOrEmpty(outputName))
{ {
@@ -230,7 +229,7 @@ namespace GitHub.Runner.Worker
public Type ExtensionType => typeof(IActionCommandExtension); public Type ExtensionType => typeof(IActionCommandExtension);
public void ProcessCommand(IExecutionContext context, string line, ActionCommand command, ContainerInfo container) public void ProcessCommand(IExecutionContext context, string line, ActionCommand command)
{ {
if (!command.Properties.TryGetValue(SaveStateCommandProperties.Name, out string stateName) || string.IsNullOrEmpty(stateName)) if (!command.Properties.TryGetValue(SaveStateCommandProperties.Name, out string stateName) || string.IsNullOrEmpty(stateName))
{ {
@@ -254,7 +253,7 @@ namespace GitHub.Runner.Worker
public Type ExtensionType => typeof(IActionCommandExtension); public Type ExtensionType => typeof(IActionCommandExtension);
public void ProcessCommand(IExecutionContext context, string line, ActionCommand command, ContainerInfo container) public void ProcessCommand(IExecutionContext context, string line, ActionCommand command)
{ {
if (string.IsNullOrWhiteSpace(command.Data)) if (string.IsNullOrWhiteSpace(command.Data))
{ {
@@ -280,7 +279,7 @@ namespace GitHub.Runner.Worker
public Type ExtensionType => typeof(IActionCommandExtension); public Type ExtensionType => typeof(IActionCommandExtension);
public void ProcessCommand(IExecutionContext context, string line, ActionCommand command, ContainerInfo container) public void ProcessCommand(IExecutionContext context, string line, ActionCommand command)
{ {
ArgUtil.NotNullOrEmpty(command.Data, "path"); ArgUtil.NotNullOrEmpty(command.Data, "path");
context.PrependPath.RemoveAll(x => string.Equals(x, command.Data, StringComparison.CurrentCulture)); context.PrependPath.RemoveAll(x => string.Equals(x, command.Data, StringComparison.CurrentCulture));
@@ -295,7 +294,7 @@ namespace GitHub.Runner.Worker
public Type ExtensionType => typeof(IActionCommandExtension); public Type ExtensionType => typeof(IActionCommandExtension);
public void ProcessCommand(IExecutionContext context, string line, ActionCommand command, ContainerInfo container) public void ProcessCommand(IExecutionContext context, string line, ActionCommand command)
{ {
var file = command.Data; var file = command.Data;
@@ -307,9 +306,9 @@ namespace GitHub.Runner.Worker
} }
// Translate file path back from container path // Translate file path back from container path
if (container != null) if (context.Container != null)
{ {
file = container.TranslateToHostPath(file); file = context.Container.TranslateToHostPath(file);
} }
// Root the path // Root the path
@@ -342,7 +341,7 @@ namespace GitHub.Runner.Worker
public Type ExtensionType => typeof(IActionCommandExtension); public Type ExtensionType => typeof(IActionCommandExtension);
public void ProcessCommand(IExecutionContext context, string line, ActionCommand command, ContainerInfo container) public void ProcessCommand(IExecutionContext context, string line, ActionCommand command)
{ {
command.Properties.TryGetValue(RemoveMatcherCommandProperties.Owner, out string owner); command.Properties.TryGetValue(RemoveMatcherCommandProperties.Owner, out string owner);
var file = command.Data; var file = command.Data;
@@ -370,9 +369,9 @@ namespace GitHub.Runner.Worker
else else
{ {
// Translate file path back from container path // Translate file path back from container path
if (container != null) if (context.Container != null)
{ {
file = container.TranslateToHostPath(file); file = context.Container.TranslateToHostPath(file);
} }
// Root the path // Root the path
@@ -410,7 +409,7 @@ namespace GitHub.Runner.Worker
public Type ExtensionType => typeof(IActionCommandExtension); public Type ExtensionType => typeof(IActionCommandExtension);
public void ProcessCommand(IExecutionContext context, string inputLine, ActionCommand command, ContainerInfo container) public void ProcessCommand(IExecutionContext context, string inputLine, ActionCommand command)
{ {
context.Debug(command.Data); context.Debug(command.Data);
} }
@@ -438,7 +437,7 @@ namespace GitHub.Runner.Worker
public Type ExtensionType => typeof(IActionCommandExtension); public Type ExtensionType => typeof(IActionCommandExtension);
public void ProcessCommand(IExecutionContext context, string inputLine, ActionCommand command, ContainerInfo container) public void ProcessCommand(IExecutionContext context, string inputLine, ActionCommand command)
{ {
command.Properties.TryGetValue(IssueCommandProperties.File, out string file); command.Properties.TryGetValue(IssueCommandProperties.File, out string file);
command.Properties.TryGetValue(IssueCommandProperties.Line, out string line); command.Properties.TryGetValue(IssueCommandProperties.Line, out string line);
@@ -455,10 +454,10 @@ namespace GitHub.Runner.Worker
{ {
issue.Category = "Code"; issue.Category = "Code";
if (container != null) if (context.Container != null)
{ {
// Translate file path back from container path // Translate file path back from container path
file = container.TranslateToHostPath(file); file = context.Container.TranslateToHostPath(file);
command.Properties[IssueCommandProperties.File] = file; command.Properties[IssueCommandProperties.File] = file;
} }
@@ -518,7 +517,7 @@ namespace GitHub.Runner.Worker
public Type ExtensionType => typeof(IActionCommandExtension); public Type ExtensionType => typeof(IActionCommandExtension);
public void ProcessCommand(IExecutionContext context, string line, ActionCommand command, ContainerInfo container) public void ProcessCommand(IExecutionContext context, string line, ActionCommand command)
{ {
var data = this is GroupCommandExtension ? command.Data : string.Empty; var data = this is GroupCommandExtension ? command.Data : string.Empty;
context.Output($"##[{Command}]{data}"); context.Output($"##[{Command}]{data}");
@@ -532,7 +531,7 @@ namespace GitHub.Runner.Worker
public Type ExtensionType => typeof(IActionCommandExtension); public Type ExtensionType => typeof(IActionCommandExtension);
public void ProcessCommand(IExecutionContext context, string line, ActionCommand command, ContainerInfo container) public void ProcessCommand(IExecutionContext context, string line, ActionCommand command)
{ {
ArgUtil.NotNullOrEmpty(command.Data, "value"); ArgUtil.NotNullOrEmpty(command.Data, "value");

View File

@@ -32,7 +32,6 @@ namespace GitHub.Runner.Worker
public sealed class ActionManifestManager : RunnerService, IActionManifestManager public sealed class ActionManifestManager : RunnerService, IActionManifestManager
{ {
private TemplateSchema _actionManifestSchema; private TemplateSchema _actionManifestSchema;
private IReadOnlyList<String> _fileTable;
public override void Initialize(IHostContext hostContext) public override void Initialize(IHostContext hostContext)
{ {
@@ -62,9 +61,6 @@ namespace GitHub.Runner.Worker
// Get the file ID // Get the file ID
var fileId = context.GetFileId(manifestFile); var fileId = context.GetFileId(manifestFile);
_fileTable = context.GetFileTable();
// Read the file
var fileContent = File.ReadAllText(manifestFile); var fileContent = File.ReadAllText(manifestFile);
using (var stringReader = new StringReader(fileContent)) using (var stringReader = new StringReader(fileContent))
{ {
@@ -269,15 +265,6 @@ namespace GitHub.Runner.Worker
} }
} }
// Add the file table
if (_fileTable?.Count > 0)
{
for (var i = 0 ; i < _fileTable.Count ; i++)
{
result.GetFileId(_fileTable[i]);
}
}
return result; return result;
} }
@@ -428,5 +415,566 @@ namespace GitHub.Runner.Worker
} }
} }
} }
/// <summary>
/// Converts a YAML file into a TemplateToken
/// </summary>
internal sealed class YamlObjectReader : IObjectReader
{
internal YamlObjectReader(
Int32? fileId,
TextReader input)
{
m_fileId = fileId;
m_parser = new Parser(input);
}
public Boolean AllowLiteral(out LiteralToken value)
{
if (EvaluateCurrent() is Scalar scalar)
{
// Tag specified
if (!string.IsNullOrEmpty(scalar.Tag))
{
// String tag
if (string.Equals(scalar.Tag, c_stringTag, StringComparison.Ordinal))
{
value = new StringToken(m_fileId, scalar.Start.Line, scalar.Start.Column, scalar.Value);
MoveNext();
return true;
}
// Not plain style
if (scalar.Style != ScalarStyle.Plain)
{
throw new NotSupportedException($"The scalar style '{scalar.Style}' on line {scalar.Start.Line} and column {scalar.Start.Column} is not valid with the tag '{scalar.Tag}'");
}
// Boolean, Float, Integer, or Null
switch (scalar.Tag)
{
case c_booleanTag:
value = ParseBoolean(scalar);
break;
case c_floatTag:
value = ParseFloat(scalar);
break;
case c_integerTag:
value = ParseInteger(scalar);
break;
case c_nullTag:
value = ParseNull(scalar);
break;
default:
throw new NotSupportedException($"Unexpected tag '{scalar.Tag}'");
}
MoveNext();
return true;
}
// Plain style, determine type using YAML 1.2 "core" schema https://yaml.org/spec/1.2/spec.html#id2804923
if (scalar.Style == ScalarStyle.Plain)
{
if (MatchNull(scalar, out var nullToken))
{
value = nullToken;
}
else if (MatchBoolean(scalar, out var booleanToken))
{
value = booleanToken;
}
else if (MatchInteger(scalar, out var numberToken) ||
MatchFloat(scalar, out numberToken))
{
value = numberToken;
}
else
{
value = new StringToken(m_fileId, scalar.Start.Line, scalar.Start.Column, scalar.Value);
}
MoveNext();
return true;
}
// Otherwise assume string
value = new StringToken(m_fileId, scalar.Start.Line, scalar.Start.Column, scalar.Value);
MoveNext();
return true;
}
value = default;
return false;
}
public Boolean AllowSequenceStart(out SequenceToken value)
{
if (EvaluateCurrent() is SequenceStart sequenceStart)
{
value = new SequenceToken(m_fileId, sequenceStart.Start.Line, sequenceStart.Start.Column);
MoveNext();
return true;
}
value = default;
return false;
}
public Boolean AllowSequenceEnd()
{
if (EvaluateCurrent() is SequenceEnd)
{
MoveNext();
return true;
}
return false;
}
public Boolean AllowMappingStart(out MappingToken value)
{
if (EvaluateCurrent() is MappingStart mappingStart)
{
value = new MappingToken(m_fileId, mappingStart.Start.Line, mappingStart.Start.Column);
MoveNext();
return true;
}
value = default;
return false;
}
public Boolean AllowMappingEnd()
{
if (EvaluateCurrent() is MappingEnd)
{
MoveNext();
return true;
}
return false;
}
/// <summary>
/// Consumes the last parsing events, which are expected to be DocumentEnd and StreamEnd.
/// </summary>
public void ValidateEnd()
{
if (EvaluateCurrent() is DocumentEnd)
{
MoveNext();
}
else
{
throw new InvalidOperationException("Expected document end parse event");
}
if (EvaluateCurrent() is StreamEnd)
{
MoveNext();
}
else
{
throw new InvalidOperationException("Expected stream end parse event");
}
if (MoveNext())
{
throw new InvalidOperationException("Expected end of parse events");
}
}
/// <summary>
/// Consumes the first parsing events, which are expected to be StreamStart and DocumentStart.
/// </summary>
public void ValidateStart()
{
if (EvaluateCurrent() != null)
{
throw new InvalidOperationException("Unexpected parser state");
}
if (!MoveNext())
{
throw new InvalidOperationException("Expected a parse event");
}
if (EvaluateCurrent() is StreamStart)
{
MoveNext();
}
else
{
throw new InvalidOperationException("Expected stream start parse event");
}
if (EvaluateCurrent() is DocumentStart)
{
MoveNext();
}
else
{
throw new InvalidOperationException("Expected document start parse event");
}
}
private ParsingEvent EvaluateCurrent()
{
if (m_current == null)
{
m_current = m_parser.Current;
if (m_current != null)
{
if (m_current is Scalar scalar)
{
// Verify not using achors
if (scalar.Anchor != null)
{
throw new InvalidOperationException($"Anchors are not currently supported. Remove the anchor '{scalar.Anchor}'");
}
}
else if (m_current is MappingStart mappingStart)
{
// Verify not using achors
if (mappingStart.Anchor != null)
{
throw new InvalidOperationException($"Anchors are not currently supported. Remove the anchor '{mappingStart.Anchor}'");
}
}
else if (m_current is SequenceStart sequenceStart)
{
// Verify not using achors
if (sequenceStart.Anchor != null)
{
throw new InvalidOperationException($"Anchors are not currently supported. Remove the anchor '{sequenceStart.Anchor}'");
}
}
else if (!(m_current is MappingEnd) &&
!(m_current is SequenceEnd) &&
!(m_current is DocumentStart) &&
!(m_current is DocumentEnd) &&
!(m_current is StreamStart) &&
!(m_current is StreamEnd))
{
throw new InvalidOperationException($"Unexpected parsing event type: {m_current.GetType().Name}");
}
}
}
return m_current;
}
private Boolean MoveNext()
{
m_current = null;
return m_parser.MoveNext();
}
private BooleanToken ParseBoolean(Scalar scalar)
{
if (MatchBoolean(scalar, out var token))
{
return token;
}
ThrowInvalidValue(scalar, c_booleanTag); // throws
return default;
}
private NumberToken ParseFloat(Scalar scalar)
{
if (MatchFloat(scalar, out var token))
{
return token;
}
ThrowInvalidValue(scalar, c_floatTag); // throws
return default;
}
private NumberToken ParseInteger(Scalar scalar)
{
if (MatchInteger(scalar, out var token))
{
return token;
}
ThrowInvalidValue(scalar, c_integerTag); // throws
return default;
}
private NullToken ParseNull(Scalar scalar)
{
if (MatchNull(scalar, out var token))
{
return token;
}
ThrowInvalidValue(scalar, c_nullTag); // throws
return default;
}
private Boolean MatchBoolean(
Scalar scalar,
out BooleanToken value)
{
// YAML 1.2 "core" schema https://yaml.org/spec/1.2/spec.html#id2804923
switch (scalar.Value ?? string.Empty)
{
case "true":
case "True":
case "TRUE":
value = new BooleanToken(m_fileId, scalar.Start.Line, scalar.Start.Column, true);
return true;
case "false":
case "False":
case "FALSE":
value = new BooleanToken(m_fileId, scalar.Start.Line, scalar.Start.Column, false);
return true;
}
value = default;
return false;
}
private Boolean MatchFloat(
Scalar scalar,
out NumberToken value)
{
// YAML 1.2 "core" schema https://yaml.org/spec/1.2/spec.html#id2804923
var str = scalar.Value;
if (!string.IsNullOrEmpty(str))
{
// Check for [-+]?(\.inf|\.Inf|\.INF)|\.nan|\.NaN|\.NAN
switch (str)
{
case ".inf":
case ".Inf":
case ".INF":
case "+.inf":
case "+.Inf":
case "+.INF":
value = new NumberToken(m_fileId, scalar.Start.Line, scalar.Start.Column, Double.PositiveInfinity);
return true;
case "-.inf":
case "-.Inf":
case "-.INF":
value = new NumberToken(m_fileId, scalar.Start.Line, scalar.Start.Column, Double.NegativeInfinity);
return true;
case ".nan":
case ".NaN":
case ".NAN":
value = new NumberToken(m_fileId, scalar.Start.Line, scalar.Start.Column, Double.NaN);
return true;
}
// Otherwise check [-+]?(\.[0-9]+|[0-9]+(\.[0-9]*)?)([eE][-+]?[0-9]+)?
// Skip leading sign
var index = str[0] == '-' || str[0] == '+' ? 1 : 0;
// Check for integer portion
var length = str.Length;
var hasInteger = false;
while (index < length && str[index] >= '0' && str[index] <= '9')
{
hasInteger = true;
index++;
}
// Check for decimal point
var hasDot = false;
if (index < length && str[index] == '.')
{
hasDot = true;
index++;
}
// Check for decimal portion
var hasDecimal = false;
while (index < length && str[index] >= '0' && str[index] <= '9')
{
hasDecimal = true;
index++;
}
// Check [-+]?(\.[0-9]+|[0-9]+(\.[0-9]*)?)
if ((hasDot && hasDecimal) || hasInteger)
{
// Check for end
if (index == length)
{
// Try parse
if (Double.TryParse(str, NumberStyles.AllowLeadingSign | NumberStyles.AllowDecimalPoint, CultureInfo.InvariantCulture, out var doubleValue))
{
value = new NumberToken(m_fileId, scalar.Start.Line, scalar.Start.Column, doubleValue);
return true;
}
// Otherwise exceeds range
else
{
ThrowInvalidValue(scalar, c_floatTag); // throws
}
}
// Check [eE][-+]?[0-9]
else if (index < length && (str[index] == 'e' || str[index] == 'E'))
{
index++;
// Skip sign
if (index < length && (str[index] == '-' || str[index] == '+'))
{
index++;
}
// Check for exponent
var hasExponent = false;
while (index < length && str[index] >= '0' && str[index] <= '9')
{
hasExponent = true;
index++;
}
// Check for end
if (hasExponent && index == length)
{
// Try parse
if (Double.TryParse(str, NumberStyles.AllowLeadingSign | NumberStyles.AllowDecimalPoint | NumberStyles.AllowExponent, CultureInfo.InvariantCulture, out var doubleValue))
{
value = new NumberToken(m_fileId, scalar.Start.Line, scalar.Start.Column, (Double)doubleValue);
return true;
}
// Otherwise exceeds range
else
{
ThrowInvalidValue(scalar, c_floatTag); // throws
}
}
}
}
}
value = default;
return false;
}
private Boolean MatchInteger(
Scalar scalar,
out NumberToken value)
{
// YAML 1.2 "core" schema https://yaml.org/spec/1.2/spec.html#id2804923
var str = scalar.Value;
if (!string.IsNullOrEmpty(str))
{
// Check for [0-9]+
var firstChar = str[0];
if (firstChar >= '0' && firstChar <= '9' &&
str.Skip(1).All(x => x >= '0' && x <= '9'))
{
// Try parse
if (Double.TryParse(str, NumberStyles.None, CultureInfo.InvariantCulture, out var doubleValue))
{
value = new NumberToken(m_fileId, scalar.Start.Line, scalar.Start.Column, doubleValue);
return true;
}
// Otherwise exceeds range
ThrowInvalidValue(scalar, c_integerTag); // throws
}
// Check for (-|+)[0-9]+
else if ((firstChar == '-' || firstChar == '+') &&
str.Length > 1 &&
str.Skip(1).All(x => x >= '0' && x <= '9'))
{
// Try parse
if (Double.TryParse(str, NumberStyles.AllowLeadingSign, CultureInfo.InvariantCulture, out var doubleValue))
{
value = new NumberToken(m_fileId, scalar.Start.Line, scalar.Start.Column, doubleValue);
return true;
}
// Otherwise exceeds range
ThrowInvalidValue(scalar, c_integerTag); // throws
}
// Check for 0x[0-9a-fA-F]+
else if (firstChar == '0' &&
str.Length > 2 &&
str[1] == 'x' &&
str.Skip(2).All(x => (x >= '0' && x <= '9') || (x >= 'a' && x <= 'f') || (x >= 'A' && x <= 'F')))
{
// Try parse
if (Int32.TryParse(str.Substring(2), NumberStyles.AllowHexSpecifier, CultureInfo.InvariantCulture, out var integerValue))
{
value = new NumberToken(m_fileId, scalar.Start.Line, scalar.Start.Column, integerValue);
return true;
}
// Otherwise exceeds range
ThrowInvalidValue(scalar, c_integerTag); // throws
}
// Check for 0o[0-9]+
else if (firstChar == '0' &&
str.Length > 2 &&
str[1] == 'o' &&
str.Skip(2).All(x => x >= '0' && x <= '7'))
{
// Try parse
var integerValue = default(Int32);
try
{
integerValue = Convert.ToInt32(str.Substring(2), 8);
}
// Otherwise exceeds range
catch (Exception)
{
ThrowInvalidValue(scalar, c_integerTag); // throws
}
value = new NumberToken(m_fileId, scalar.Start.Line, scalar.Start.Column, integerValue);
return true;
}
}
value = default;
return false;
}
private Boolean MatchNull(
Scalar scalar,
out NullToken value)
{
// YAML 1.2 "core" schema https://yaml.org/spec/1.2/spec.html#id2804923
switch (scalar.Value ?? string.Empty)
{
case "":
case "null":
case "Null":
case "NULL":
case "~":
value = new NullToken(m_fileId, scalar.Start.Line, scalar.Start.Column);
return true;
}
value = default;
return false;
}
private void ThrowInvalidValue(
Scalar scalar,
String tag)
{
throw new NotSupportedException($"The value '{scalar.Value}' on line {scalar.Start.Line} and column {scalar.Start.Column} is invalid for the type '{scalar.Tag}'");
}
private const String c_booleanTag = "tag:yaml.org,2002:bool";
private const String c_floatTag = "tag:yaml.org,2002:float";
private const String c_integerTag = "tag:yaml.org,2002:int";
private const String c_nullTag = "tag:yaml.org,2002:null";
private const String c_stringTag = "tag:yaml.org,2002:string";
private readonly Int32? m_fileId;
private readonly Parser m_parser;
private ParsingEvent m_current;
}
} }

View File

@@ -141,7 +141,9 @@ namespace GitHub.Runner.Worker
// Load the inputs. // Load the inputs.
ExecutionContext.Debug("Loading inputs"); ExecutionContext.Debug("Loading inputs");
var templateEvaluator = ExecutionContext.ToPipelineTemplateEvaluator(); var templateTrace = ExecutionContext.ToTemplateTraceWriter();
var schema = new PipelineTemplateSchemaFactory().CreateSchema();
var templateEvaluator = new PipelineTemplateEvaluator(templateTrace, schema);
var inputs = templateEvaluator.EvaluateStepInputs(Action.Inputs, ExecutionContext.ExpressionValues); var inputs = templateEvaluator.EvaluateStepInputs(Action.Inputs, ExecutionContext.ExpressionValues);
foreach (KeyValuePair<string, string> input in inputs) foreach (KeyValuePair<string, string> input in inputs)
@@ -177,15 +179,17 @@ namespace GitHub.Runner.Worker
ExecutionContext.Debug("Loading env"); ExecutionContext.Debug("Loading env");
var environment = new Dictionary<String, String>(VarUtil.EnvironmentVariableKeyComparer); var environment = new Dictionary<String, String>(VarUtil.EnvironmentVariableKeyComparer);
#if OS_WINDOWS // Apply environment set using ##[set-env] first since these are job level env
var envContext = ExecutionContext.ExpressionValues["env"] as DictionaryContextData; foreach (var env in ExecutionContext.EnvironmentVariables)
#else
var envContext = ExecutionContext.ExpressionValues["env"] as CaseSensitiveDictionaryContextData;
#endif
// Apply environment from env context, env context contains job level env and action's evn block
foreach (var env in envContext)
{ {
environment[env.Key] = env.Value.ToString(); environment[env.Key] = env.Value ?? string.Empty;
}
// Apply action's env block later.
var actionEnvironment = templateEvaluator.EvaluateStepEnvironment(Action.Environment, ExecutionContext.ExpressionValues, VarUtil.EnvironmentVariableKeyComparer);
foreach (var env in actionEnvironment)
{
environment[env.Key] = env.Value ?? string.Empty;
} }
// Apply action's intra-action state at last // Apply action's intra-action state at last
@@ -293,7 +297,8 @@ namespace GitHub.Runner.Worker
return displayName; return displayName;
} }
// Try evaluating fully // Try evaluating fully
var templateEvaluator = context.ToPipelineTemplateEvaluator(); var schema = new PipelineTemplateSchemaFactory().CreateSchema();
var templateEvaluator = new PipelineTemplateEvaluator(context.ToTemplateTraceWriter(), schema);
try try
{ {
didFullyEvaluate = templateEvaluator.TryEvaluateStepDisplayName(tokenToParse, contextData, out displayName); didFullyEvaluate = templateEvaluator.TryEvaluateStepDisplayName(tokenToParse, contextData, out displayName);

View File

@@ -63,8 +63,6 @@ namespace GitHub.Runner.Worker.Container
UserMountVolumes[volume] = volume; UserMountVolumes[volume] = volume;
} }
} }
UpdateWebProxyEnv(hostContext.WebProxy);
} }
public string ContainerId { get; set; } public string ContainerId { get; set; }
@@ -224,26 +222,6 @@ namespace GitHub.Runner.Worker.Container
{ {
_pathMappings.Insert(0, new PathMapping(hostCommonPath, containerCommonPath)); _pathMappings.Insert(0, new PathMapping(hostCommonPath, containerCommonPath));
} }
private void UpdateWebProxyEnv(RunnerWebProxy webProxy)
{
// Set common forms of proxy variables if configured in Runner and not set directly by container.env
if (!String.IsNullOrEmpty(webProxy.HttpProxyAddress))
{
ContainerEnvironmentVariables.TryAdd("HTTP_PROXY", webProxy.HttpProxyAddress);
ContainerEnvironmentVariables.TryAdd("http_proxy", webProxy.HttpProxyAddress);
}
if (!String.IsNullOrEmpty(webProxy.HttpsProxyAddress))
{
ContainerEnvironmentVariables.TryAdd("HTTPS_PROXY", webProxy.HttpsProxyAddress);
ContainerEnvironmentVariables.TryAdd("https_proxy", webProxy.HttpsProxyAddress);
}
if (!String.IsNullOrEmpty(webProxy.NoProxyString))
{
ContainerEnvironmentVariables.TryAdd("NO_PROXY", webProxy.NoProxyString);
ContainerEnvironmentVariables.TryAdd("no_proxy", webProxy.NoProxyString);
}
}
} }
public class MountVolume public class MountVolume

View File

@@ -11,7 +11,6 @@ using GitHub.Runner.Worker.Container;
using GitHub.Services.WebApi; using GitHub.Services.WebApi;
using GitHub.DistributedTask.Pipelines; using GitHub.DistributedTask.Pipelines;
using GitHub.DistributedTask.Pipelines.ContextData; using GitHub.DistributedTask.Pipelines.ContextData;
using GitHub.DistributedTask.Pipelines.ObjectTemplating;
using GitHub.DistributedTask.WebApi; using GitHub.DistributedTask.WebApi;
using GitHub.Runner.Common.Util; using GitHub.Runner.Common.Util;
using GitHub.Runner.Common; using GitHub.Runner.Common;
@@ -50,7 +49,6 @@ namespace GitHub.Runner.Worker
HashSet<string> OutputVariables { get; } HashSet<string> OutputVariables { get; }
IDictionary<String, String> EnvironmentVariables { get; } IDictionary<String, String> EnvironmentVariables { get; }
IDictionary<String, ContextScope> Scopes { get; } IDictionary<String, ContextScope> Scopes { get; }
IList<String> FileTable { get; }
StepsContext StepsContext { get; } StepsContext StepsContext { get; }
DictionaryContextData ExpressionValues { get; } DictionaryContextData ExpressionValues { get; }
List<string> PrependPath { get; } List<string> PrependPath { get; }
@@ -143,7 +141,6 @@ namespace GitHub.Runner.Worker
public HashSet<string> OutputVariables => _outputvariables; public HashSet<string> OutputVariables => _outputvariables;
public IDictionary<String, String> EnvironmentVariables { get; private set; } public IDictionary<String, String> EnvironmentVariables { get; private set; }
public IDictionary<String, ContextScope> Scopes { get; private set; } public IDictionary<String, ContextScope> Scopes { get; private set; }
public IList<String> FileTable { get; private set; }
public StepsContext StepsContext { get; private set; } public StepsContext StepsContext { get; private set; }
public DictionaryContextData ExpressionValues { get; } = new DictionaryContextData(); public DictionaryContextData ExpressionValues { get; } = new DictionaryContextData();
public bool WriteDebug { get; private set; } public bool WriteDebug { get; private set; }
@@ -269,7 +266,6 @@ namespace GitHub.Runner.Worker
} }
child.EnvironmentVariables = EnvironmentVariables; child.EnvironmentVariables = EnvironmentVariables;
child.Scopes = Scopes; child.Scopes = Scopes;
child.FileTable = FileTable;
child.StepsContext = StepsContext; child.StepsContext = StepsContext;
foreach (var pair in ExpressionValues) foreach (var pair in ExpressionValues)
{ {
@@ -573,9 +569,6 @@ namespace GitHub.Runner.Worker
} }
} }
// File table
FileTable = new List<String>(message.FileTable ?? new string[0]);
// Expression functions // Expression functions
if (Variables.GetBoolean("System.HashFilesV2") == true) if (Variables.GetBoolean("System.HashFilesV2") == true)
{ {
@@ -893,13 +886,6 @@ namespace GitHub.Runner.Worker
} }
} }
public static PipelineTemplateEvaluator ToPipelineTemplateEvaluator(this IExecutionContext context)
{
var templateTrace = context.ToTemplateTraceWriter();
var schema = new PipelineTemplateSchemaFactory().CreateSchema();
return new PipelineTemplateEvaluator(templateTrace, schema, context.FileTable);
}
public static ObjectTemplating.ITraceWriter ToTemplateTraceWriter(this IExecutionContext context) public static ObjectTemplating.ITraceWriter ToTemplateTraceWriter(this IExecutionContext context)
{ {
return new TemplateTraceWriter(context); return new TemplateTraceWriter(context);

View File

@@ -84,7 +84,7 @@ namespace GitHub.Runner.Worker.Handlers
{ {
// This does not need to be inside of a critical section. // This does not need to be inside of a critical section.
// The logging queues and command handlers are thread-safe. // The logging queues and command handlers are thread-safe.
if (_commandManager.TryProcessCommand(_executionContext, line, _container)) if (_commandManager.TryProcessCommand(_executionContext, line))
{ {
return; return;
} }

View File

@@ -78,7 +78,9 @@ namespace GitHub.Runner.Worker
// Evaluate the job-level environment variables // Evaluate the job-level environment variables
context.Debug("Evaluating job-level environment variables"); context.Debug("Evaluating job-level environment variables");
var templateEvaluator = context.ToPipelineTemplateEvaluator(); var templateTrace = context.ToTemplateTraceWriter();
var schema = new PipelineTemplateSchemaFactory().CreateSchema();
var templateEvaluator = new PipelineTemplateEvaluator(templateTrace, schema);
foreach (var token in message.EnvironmentVariables) foreach (var token in message.EnvironmentVariables)
{ {
var environmentVariables = templateEvaluator.EvaluateStepEnvironment(token, jobContext.ExpressionValues, VarUtil.EnvironmentVariableKeyComparer); var environmentVariables = templateEvaluator.EvaluateStepEnvironment(token, jobContext.ExpressionValues, VarUtil.EnvironmentVariableKeyComparer);

View File

@@ -76,36 +76,15 @@ namespace GitHub.Runner.Worker
// Start // Start
step.ExecutionContext.Start(); step.ExecutionContext.Start();
// Set GITHUB_ACTION
if (step is IActionRunner actionStep)
{
step.ExecutionContext.SetGitHubContext("action", actionStep.Action.Name);
}
// Initialize scope // Initialize scope
if (InitializeScope(step, scopeInputs)) if (InitializeScope(step, scopeInputs))
{ {
// Populate env context for each step
Trace.Info("Initialize Env context for step");
#if OS_WINDOWS
var envContext = new DictionaryContextData();
#else
var envContext = new CaseSensitiveDictionaryContextData();
#endif
step.ExecutionContext.ExpressionValues["env"] = envContext;
foreach (var pair in step.ExecutionContext.EnvironmentVariables)
{
envContext[pair.Key] = new StringContextData(pair.Value ?? string.Empty);
}
if (step is IActionRunner actionStep)
{
// Set GITHUB_ACTION
step.ExecutionContext.SetGitHubContext("action", actionStep.Action.Name);
// Evaluate and merge action's env block to env context
var templateEvaluator = step.ExecutionContext.ToPipelineTemplateEvaluator();
var actionEnvironment = templateEvaluator.EvaluateStepEnvironment(actionStep.Action.Environment, step.ExecutionContext.ExpressionValues, VarUtil.EnvironmentVariableKeyComparer);
foreach (var env in actionEnvironment)
{
envContext[env.Key] = new StringContextData(env.Value ?? string.Empty);
}
}
var expressionManager = HostContext.GetService<IExpressionManager>(); var expressionManager = HostContext.GetService<IExpressionManager>();
try try
{ {
@@ -245,7 +224,7 @@ namespace GitHub.Runner.Worker
// Set the timeout // Set the timeout
var timeoutMinutes = 0; var timeoutMinutes = 0;
var templateEvaluator = step.ExecutionContext.ToPipelineTemplateEvaluator(); var templateEvaluator = CreateTemplateEvaluator(step.ExecutionContext);
try try
{ {
timeoutMinutes = templateEvaluator.EvaluateStepTimeout(step.Timeout, step.ExecutionContext.ExpressionValues); timeoutMinutes = templateEvaluator.EvaluateStepTimeout(step.Timeout, step.ExecutionContext.ExpressionValues);
@@ -387,7 +366,7 @@ namespace GitHub.Runner.Worker
executionContext.Debug($"Initializing scope '{scope.Name}'"); executionContext.Debug($"Initializing scope '{scope.Name}'");
executionContext.ExpressionValues["steps"] = stepsContext.GetScope(scope.ParentName); executionContext.ExpressionValues["steps"] = stepsContext.GetScope(scope.ParentName);
executionContext.ExpressionValues["inputs"] = !String.IsNullOrEmpty(scope.ParentName) ? scopeInputs[scope.ParentName] : null; executionContext.ExpressionValues["inputs"] = !String.IsNullOrEmpty(scope.ParentName) ? scopeInputs[scope.ParentName] : null;
var templateEvaluator = executionContext.ToPipelineTemplateEvaluator(); var templateEvaluator = CreateTemplateEvaluator(executionContext);
var inputs = default(DictionaryContextData); var inputs = default(DictionaryContextData);
try try
{ {
@@ -443,7 +422,7 @@ namespace GitHub.Runner.Worker
executionContext.Debug($"Finalizing scope '{scope.Name}'"); executionContext.Debug($"Finalizing scope '{scope.Name}'");
executionContext.ExpressionValues["steps"] = stepsContext.GetScope(scope.Name); executionContext.ExpressionValues["steps"] = stepsContext.GetScope(scope.Name);
executionContext.ExpressionValues["inputs"] = null; executionContext.ExpressionValues["inputs"] = null;
var templateEvaluator = executionContext.ToPipelineTemplateEvaluator(); var templateEvaluator = CreateTemplateEvaluator(executionContext);
var outputs = default(DictionaryContextData); var outputs = default(DictionaryContextData);
try try
{ {
@@ -475,5 +454,12 @@ namespace GitHub.Runner.Worker
executionContext.Complete(result, resultCode: resultCode); executionContext.Complete(result, resultCode: resultCode);
} }
private PipelineTemplateEvaluator CreateTemplateEvaluator(IExecutionContext executionContext)
{
var templateTrace = executionContext.ToTemplateTraceWriter();
var schema = new PipelineTemplateSchemaFactory().CreateSchema();
return new PipelineTemplateEvaluator(templateTrace, schema);
}
} }
} }

View File

@@ -162,8 +162,8 @@ namespace GitHub.Services.Common
} }
IssuedToken token = null; IssuedToken token = null;
IssuedTokenProvider provider; IssuedTokenProvider provider = null;
if (this.Credentials.TryGetTokenProvider(request.RequestUri, out provider)) if (this.Credentials != null && this.Credentials.TryGetTokenProvider(request.RequestUri, out provider))
{ {
token = provider.CurrentToken; token = provider.CurrentToken;
} }
@@ -227,7 +227,7 @@ namespace GitHub.Services.Common
responseWrapper = new HttpResponseMessageWrapper(response); responseWrapper = new HttpResponseMessageWrapper(response);
if (!this.Credentials.IsAuthenticationChallenge(responseWrapper)) if (this.Credentials != null && !this.Credentials.IsAuthenticationChallenge(responseWrapper))
{ {
// Validate the token after it has been successfully authenticated with the server. // Validate the token after it has been successfully authenticated with the server.
if (provider != null) if (provider != null)
@@ -259,7 +259,10 @@ namespace GitHub.Services.Common
} }
// Ensure we have an appropriate token provider for the current challenge // Ensure we have an appropriate token provider for the current challenge
provider = this.Credentials.CreateTokenProvider(request.RequestUri, responseWrapper, token); if (this.Credentials != null)
{
provider = this.Credentials.CreateTokenProvider(request.RequestUri, responseWrapper, token);
}
// Make sure we don't invoke the provider in an invalid state // Make sure we don't invoke the provider in an invalid state
if (provider == null) if (provider == null)
@@ -308,7 +311,7 @@ namespace GitHub.Services.Common
// We're out of retries and the response was an auth challenge -- then the request was unauthorized // We're out of retries and the response was an auth challenge -- then the request was unauthorized
// and we will throw a strongly-typed exception with a friendly error message. // and we will throw a strongly-typed exception with a friendly error message.
if (!succeeded && response != null && this.Credentials.IsAuthenticationChallenge(responseWrapper)) if (!succeeded && response != null && (this.Credentials != null && this.Credentials.IsAuthenticationChallenge(responseWrapper)))
{ {
String message = null; String message = null;
IEnumerable<String> serviceError; IEnumerable<String> serviceError;

View File

@@ -184,7 +184,6 @@ namespace GitHub.DistributedTask.ObjectTemplating
id = FileIds.Count + 1; id = FileIds.Count + 1;
FileIds.Add(file, id); FileIds.Add(file, id);
FileNames.Add(file); FileNames.Add(file);
Memory.AddBytes(file);
} }
return id; return id;
@@ -192,12 +191,7 @@ namespace GitHub.DistributedTask.ObjectTemplating
internal String GetFileName(Int32 fileId) internal String GetFileName(Int32 fileId)
{ {
return FileNames.Count >= fileId ? FileNames[fileId - 1] : null; return FileNames[fileId - 1];
}
internal IReadOnlyList<String> GetFileTable()
{
return FileNames.AsReadOnly();
} }
private String GetErrorPrefix( private String GetErrorPrefix(
@@ -205,9 +199,9 @@ namespace GitHub.DistributedTask.ObjectTemplating
Int32? line, Int32? line,
Int32? column) Int32? column)
{ {
var fileName = fileId.HasValue ? GetFileName(fileId.Value) : null; if (fileId != null)
if (!String.IsNullOrEmpty(fileName))
{ {
var fileName = GetFileName(fileId.Value);
if (line != null && column != null) if (line != null && column != null)
{ {
return $"{fileName} {TemplateStrings.LineColumn(line, column)}:"; return $"{fileName} {TemplateStrings.LineColumn(line, column)}:";

View File

@@ -30,14 +30,14 @@ namespace GitHub.DistributedTask.ObjectTemplating.Tokens
Column = column; Column = column;
} }
[DataMember(Name = "file", EmitDefaultValue = false)] [IgnoreDataMember]
internal Int32? FileId { get; private set; } internal Int32? FileId { get; set; }
[DataMember(Name = "line", EmitDefaultValue = false)] [DataMember(Name = "line", EmitDefaultValue = false)]
internal Int32? Line { get; private set; } internal Int32? Line { get; }
[DataMember(Name = "col", EmitDefaultValue = false)] [DataMember(Name = "col", EmitDefaultValue = false)]
internal Int32? Column { get; private set; } internal Int32? Column { get; }
[DataMember(Name = "type", EmitDefaultValue = false)] [DataMember(Name = "type", EmitDefaultValue = false)]
internal Int32 Type { get; } internal Int32 Type { get; }

View File

@@ -115,12 +115,13 @@ namespace GitHub.DistributedTask.ObjectTemplating.Tokens
Object value, Object value,
JsonSerializer serializer) JsonSerializer serializer)
{ {
base.WriteJson(writer, value, serializer);
if (value is TemplateToken token) if (value is TemplateToken token)
{ {
switch (token.Type) switch (token.Type)
{ {
case TokenType.Null: case TokenType.Null:
if (token.FileId == null && token.Line == null && token.Column == null) if (token.Line == null && token.Column == null)
{ {
writer.WriteNull(); writer.WriteNull();
} }
@@ -129,17 +130,12 @@ namespace GitHub.DistributedTask.ObjectTemplating.Tokens
writer.WriteStartObject(); writer.WriteStartObject();
writer.WritePropertyName("type"); writer.WritePropertyName("type");
writer.WriteValue(token.Type); writer.WriteValue(token.Type);
if (token.FileId != null)
{
writer.WritePropertyName("file");
writer.WriteValue(token.FileId);
}
if (token.Line != null) if (token.Line != null)
{ {
writer.WritePropertyName("line"); writer.WritePropertyName("line");
writer.WriteValue(token.Line); writer.WriteValue(token.Line);
} }
if (token.Column != null) if (token.Line != null)
{ {
writer.WritePropertyName("col"); writer.WritePropertyName("col");
writer.WriteValue(token.Column); writer.WriteValue(token.Column);
@@ -150,7 +146,7 @@ namespace GitHub.DistributedTask.ObjectTemplating.Tokens
case TokenType.Boolean: case TokenType.Boolean:
var booleanToken = token as BooleanToken; var booleanToken = token as BooleanToken;
if (token.FileId == null && token.Line == null && token.Column == null) if (token.Line == null && token.Column == null)
{ {
writer.WriteValue(booleanToken.Value); writer.WriteValue(booleanToken.Value);
} }
@@ -159,17 +155,12 @@ namespace GitHub.DistributedTask.ObjectTemplating.Tokens
writer.WriteStartObject(); writer.WriteStartObject();
writer.WritePropertyName("type"); writer.WritePropertyName("type");
writer.WriteValue(token.Type); writer.WriteValue(token.Type);
if (token.FileId != null)
{
writer.WritePropertyName("file");
writer.WriteValue(token.FileId);
}
if (token.Line != null) if (token.Line != null)
{ {
writer.WritePropertyName("line"); writer.WritePropertyName("line");
writer.WriteValue(token.Line); writer.WriteValue(token.Line);
} }
if (token.Column != null) if (token.Line != null)
{ {
writer.WritePropertyName("col"); writer.WritePropertyName("col");
writer.WriteValue(token.Column); writer.WriteValue(token.Column);
@@ -182,7 +173,7 @@ namespace GitHub.DistributedTask.ObjectTemplating.Tokens
case TokenType.Number: case TokenType.Number:
var numberToken = token as NumberToken; var numberToken = token as NumberToken;
if (token.FileId == null && token.Line == null && token.Column == null) if (token.Line == null && token.Column == null)
{ {
writer.WriteValue(numberToken.Value); writer.WriteValue(numberToken.Value);
} }
@@ -191,17 +182,12 @@ namespace GitHub.DistributedTask.ObjectTemplating.Tokens
writer.WriteStartObject(); writer.WriteStartObject();
writer.WritePropertyName("type"); writer.WritePropertyName("type");
writer.WriteValue(token.Type); writer.WriteValue(token.Type);
if (token.FileId != null)
{
writer.WritePropertyName("file");
writer.WriteValue(token.FileId);
}
if (token.Line != null) if (token.Line != null)
{ {
writer.WritePropertyName("line"); writer.WritePropertyName("line");
writer.WriteValue(token.Line); writer.WriteValue(token.Line);
} }
if (token.Column != null) if (token.Line != null)
{ {
writer.WritePropertyName("col"); writer.WritePropertyName("col");
writer.WriteValue(token.Column); writer.WriteValue(token.Column);
@@ -214,7 +200,7 @@ namespace GitHub.DistributedTask.ObjectTemplating.Tokens
case TokenType.String: case TokenType.String:
var stringToken = token as StringToken; var stringToken = token as StringToken;
if (token.FileId == null && token.Line == null && token.Column == null) if (token.Line == null && token.Column == null)
{ {
writer.WriteValue(stringToken.Value); writer.WriteValue(stringToken.Value);
} }
@@ -223,17 +209,12 @@ namespace GitHub.DistributedTask.ObjectTemplating.Tokens
writer.WriteStartObject(); writer.WriteStartObject();
writer.WritePropertyName("type"); writer.WritePropertyName("type");
writer.WriteValue(token.Type); writer.WriteValue(token.Type);
if (token.FileId != null)
{
writer.WritePropertyName("file");
writer.WriteValue(token.FileId);
}
if (token.Line != null) if (token.Line != null)
{ {
writer.WritePropertyName("line"); writer.WritePropertyName("line");
writer.WriteValue(token.Line); writer.WriteValue(token.Line);
} }
if (token.Column != null) if (token.Line != null)
{ {
writer.WritePropertyName("col"); writer.WritePropertyName("col");
writer.WriteValue(token.Column); writer.WriteValue(token.Column);
@@ -249,17 +230,12 @@ namespace GitHub.DistributedTask.ObjectTemplating.Tokens
writer.WriteStartObject(); writer.WriteStartObject();
writer.WritePropertyName("type"); writer.WritePropertyName("type");
writer.WriteValue(token.Type); writer.WriteValue(token.Type);
if (token.FileId != null)
{
writer.WritePropertyName("file");
writer.WriteValue(token.FileId);
}
if (token.Line != null) if (token.Line != null)
{ {
writer.WritePropertyName("line"); writer.WritePropertyName("line");
writer.WriteValue(token.Line); writer.WriteValue(token.Line);
} }
if (token.Column != null) if (token.Line != null)
{ {
writer.WritePropertyName("col"); writer.WritePropertyName("col");
writer.WriteValue(token.Column); writer.WriteValue(token.Column);
@@ -277,17 +253,12 @@ namespace GitHub.DistributedTask.ObjectTemplating.Tokens
writer.WriteStartObject(); writer.WriteStartObject();
writer.WritePropertyName("type"); writer.WritePropertyName("type");
writer.WriteValue(token.Type); writer.WriteValue(token.Type);
if (token.FileId != null)
{
writer.WritePropertyName("file");
writer.WriteValue(token.FileId);
}
if (token.Line != null) if (token.Line != null)
{ {
writer.WritePropertyName("line"); writer.WritePropertyName("line");
writer.WriteValue(token.Line); writer.WriteValue(token.Line);
} }
if (token.Column != null) if (token.Line != null)
{ {
writer.WritePropertyName("col"); writer.WritePropertyName("col");
writer.WriteValue(token.Column); writer.WriteValue(token.Column);
@@ -302,17 +273,12 @@ namespace GitHub.DistributedTask.ObjectTemplating.Tokens
writer.WriteStartObject(); writer.WriteStartObject();
writer.WritePropertyName("type"); writer.WritePropertyName("type");
writer.WriteValue(token.Type); writer.WriteValue(token.Type);
if (token.FileId != null)
{
writer.WritePropertyName("file");
writer.WriteValue(token.FileId);
}
if (token.Line != null) if (token.Line != null)
{ {
writer.WritePropertyName("line"); writer.WritePropertyName("line");
writer.WriteValue(token.Line); writer.WriteValue(token.Line);
} }
if (token.Column != null) if (token.Line != null)
{ {
writer.WritePropertyName("col"); writer.WritePropertyName("col");
writer.WriteValue(token.Column); writer.WriteValue(token.Column);
@@ -335,17 +301,12 @@ namespace GitHub.DistributedTask.ObjectTemplating.Tokens
writer.WriteStartObject(); writer.WriteStartObject();
writer.WritePropertyName("type"); writer.WritePropertyName("type");
writer.WriteValue(token.Type); writer.WriteValue(token.Type);
if (token.FileId != null)
{
writer.WritePropertyName("file");
writer.WriteValue(token.FileId);
}
if (token.Line != null) if (token.Line != null)
{ {
writer.WritePropertyName("line"); writer.WritePropertyName("line");
writer.WriteValue(token.Line); writer.WriteValue(token.Line);
} }
if (token.Column != null) if (token.Line != null)
{ {
writer.WritePropertyName("col"); writer.WritePropertyName("col");
writer.WriteValue(token.Column); writer.WriteValue(token.Column);

View File

@@ -39,8 +39,7 @@ namespace GitHub.DistributedTask.Pipelines
DictionaryContextData contextData, DictionaryContextData contextData,
WorkspaceOptions workspaceOptions, WorkspaceOptions workspaceOptions,
IEnumerable<JobStep> steps, IEnumerable<JobStep> steps,
IEnumerable<ContextScope> scopes, IEnumerable<ContextScope> scopes)
IList<String> fileTable)
{ {
this.MessageType = JobRequestMessageTypes.PipelineAgentJobRequest; this.MessageType = JobRequestMessageTypes.PipelineAgentJobRequest;
this.Plan = plan; this.Plan = plan;
@@ -75,11 +74,6 @@ namespace GitHub.DistributedTask.Pipelines
this.ContextData[pair.Key] = pair.Value; this.ContextData[pair.Key] = pair.Value;
} }
} }
if (fileTable?.Count > 0)
{
m_fileTable = new List<String>(fileTable);
}
} }
[DataMember] [DataMember]
@@ -243,18 +237,6 @@ namespace GitHub.DistributedTask.Pipelines
} }
} }
public IList<String> FileTable
{
get
{
if (m_fileTable == null)
{
m_fileTable = new List<String>();
}
return m_fileTable;
}
}
// todo: remove after feature-flag DistributedTask.EvaluateContainerOnRunner is enabled everywhere // todo: remove after feature-flag DistributedTask.EvaluateContainerOnRunner is enabled everywhere
public void SetJobSidecarContainers(IDictionary<String, String> value) public void SetJobSidecarContainers(IDictionary<String, String> value)
{ {
@@ -363,11 +345,6 @@ namespace GitHub.DistributedTask.Pipelines
m_environmentVariables = null; m_environmentVariables = null;
} }
if (m_fileTable?.Count == 0)
{
m_fileTable = null;
}
if (m_maskHints?.Count == 0) if (m_maskHints?.Count == 0)
{ {
m_maskHints = null; m_maskHints = null;
@@ -397,9 +374,6 @@ namespace GitHub.DistributedTask.Pipelines
[DataMember(Name = "EnvironmentVariables", EmitDefaultValue = false)] [DataMember(Name = "EnvironmentVariables", EmitDefaultValue = false)]
private List<TemplateToken> m_environmentVariables; private List<TemplateToken> m_environmentVariables;
[DataMember(Name = "FileTable", EmitDefaultValue = false)]
private List<String> m_fileTable;
[DataMember(Name = "Mask", EmitDefaultValue = false)] [DataMember(Name = "Mask", EmitDefaultValue = false)]
private List<MaskHint> m_maskHints; private List<MaskHint> m_maskHints;

View File

@@ -35,19 +35,6 @@ namespace GitHub.DistributedTask.Pipelines.ContextData
throw new ArgumentException($"Unexpected type '{value?.GetType().Name}' encountered while reading '{objectDescription}'. The type '{nameof(DictionaryContextData)}' was expected."); throw new ArgumentException($"Unexpected type '{value?.GetType().Name}' encountered while reading '{objectDescription}'. The type '{nameof(DictionaryContextData)}' was expected.");
} }
[EditorBrowsable(EditorBrowsableState.Never)]
public static CaseSensitiveDictionaryContextData AssertCaseSensitiveDictionary(
this PipelineContextData value,
String objectDescription)
{
if (value is CaseSensitiveDictionaryContextData dictionary)
{
return dictionary;
}
throw new ArgumentException($"Unexpected type '{value?.GetType().Name}' encountered while reading '{objectDescription}'. The type '{nameof(CaseSensitiveDictionaryContextData)}' was expected.");
}
[EditorBrowsable(EditorBrowsableState.Never)] [EditorBrowsable(EditorBrowsableState.Never)]
public static StringContextData AssertString( public static StringContextData AssertString(
this PipelineContextData value, this PipelineContextData value,

View File

@@ -19,8 +19,7 @@ namespace GitHub.DistributedTask.Pipelines.ObjectTemplating
{ {
public PipelineTemplateEvaluator( public PipelineTemplateEvaluator(
ITraceWriter trace, ITraceWriter trace,
TemplateSchema schema, TemplateSchema schema)
IList<String> fileTable)
{ {
if (!String.Equals(schema.Version, PipelineTemplateConstants.Workflow_1_0, StringComparison.Ordinal)) if (!String.Equals(schema.Version, PipelineTemplateConstants.Workflow_1_0, StringComparison.Ordinal))
{ {
@@ -29,7 +28,6 @@ namespace GitHub.DistributedTask.Pipelines.ObjectTemplating
m_trace = trace; m_trace = trace;
m_schema = schema; m_schema = schema;
m_fileTable = fileTable;
} }
public Int32 MaxDepth => 50; public Int32 MaxDepth => 50;
@@ -326,16 +324,6 @@ namespace GitHub.DistributedTask.Pipelines.ObjectTemplating
TraceWriter = m_trace, TraceWriter = m_trace,
}; };
// Add the file table
if (m_fileTable?.Count > 0)
{
foreach (var file in m_fileTable)
{
result.GetFileId(file);
}
}
// Add named context
if (contextData != null) if (contextData != null)
{ {
foreach (var pair in contextData) foreach (var pair in contextData)
@@ -358,7 +346,6 @@ namespace GitHub.DistributedTask.Pipelines.ObjectTemplating
private readonly ITraceWriter m_trace; private readonly ITraceWriter m_trace;
private readonly TemplateSchema m_schema; private readonly TemplateSchema m_schema;
private readonly IList<String> m_fileTable;
private readonly String[] s_contextNames = new[] private readonly String[] s_contextNames = new[]
{ {
PipelineTemplateConstants.GitHub, PipelineTemplateConstants.GitHub,

View File

@@ -1,572 +0,0 @@
using System;
using System.Globalization;
using System.IO;
using System.Linq;
using GitHub.DistributedTask.ObjectTemplating;
using GitHub.DistributedTask.ObjectTemplating.Tokens;
using YamlDotNet.Core;
using YamlDotNet.Core.Events;
namespace GitHub.DistributedTask.Pipelines.ObjectTemplating
{
/// <summary>
/// Converts a YAML file into a TemplateToken
/// </summary>
public sealed class YamlObjectReader : IObjectReader
{
internal YamlObjectReader(
Int32? fileId,
TextReader input)
{
m_fileId = fileId;
m_parser = new Parser(input);
}
public Boolean AllowLiteral(out LiteralToken value)
{
if (EvaluateCurrent() is Scalar scalar)
{
// Tag specified
if (!String.IsNullOrEmpty(scalar.Tag))
{
// String tag
if (String.Equals(scalar.Tag, c_stringTag, StringComparison.Ordinal))
{
value = new StringToken(m_fileId, scalar.Start.Line, scalar.Start.Column, scalar.Value);
MoveNext();
return true;
}
// Not plain style
if (scalar.Style != ScalarStyle.Plain)
{
throw new NotSupportedException($"The scalar style '{scalar.Style}' on line {scalar.Start.Line} and column {scalar.Start.Column} is not valid with the tag '{scalar.Tag}'");
}
// Boolean, Float, Integer, or Null
switch (scalar.Tag)
{
case c_booleanTag:
value = ParseBoolean(scalar);
break;
case c_floatTag:
value = ParseFloat(scalar);
break;
case c_integerTag:
value = ParseInteger(scalar);
break;
case c_nullTag:
value = ParseNull(scalar);
break;
default:
throw new NotSupportedException($"Unexpected tag '{scalar.Tag}'");
}
MoveNext();
return true;
}
// Plain style, determine type using YAML 1.2 "core" schema https://yaml.org/spec/1.2/spec.html#id2804923
if (scalar.Style == ScalarStyle.Plain)
{
if (MatchNull(scalar, out var nullToken))
{
value = nullToken;
}
else if (MatchBoolean(scalar, out var booleanToken))
{
value = booleanToken;
}
else if (MatchInteger(scalar, out var numberToken) ||
MatchFloat(scalar, out numberToken))
{
value = numberToken;
}
else
{
value = new StringToken(m_fileId, scalar.Start.Line, scalar.Start.Column, scalar.Value);
}
MoveNext();
return true;
}
// Otherwise assume string
value = new StringToken(m_fileId, scalar.Start.Line, scalar.Start.Column, scalar.Value);
MoveNext();
return true;
}
value = default;
return false;
}
public Boolean AllowSequenceStart(out SequenceToken value)
{
if (EvaluateCurrent() is SequenceStart sequenceStart)
{
value = new SequenceToken(m_fileId, sequenceStart.Start.Line, sequenceStart.Start.Column);
MoveNext();
return true;
}
value = default;
return false;
}
public Boolean AllowSequenceEnd()
{
if (EvaluateCurrent() is SequenceEnd)
{
MoveNext();
return true;
}
return false;
}
public Boolean AllowMappingStart(out MappingToken value)
{
if (EvaluateCurrent() is MappingStart mappingStart)
{
value = new MappingToken(m_fileId, mappingStart.Start.Line, mappingStart.Start.Column);
MoveNext();
return true;
}
value = default;
return false;
}
public Boolean AllowMappingEnd()
{
if (EvaluateCurrent() is MappingEnd)
{
MoveNext();
return true;
}
return false;
}
/// <summary>
/// Consumes the last parsing events, which are expected to be DocumentEnd and StreamEnd.
/// </summary>
public void ValidateEnd()
{
if (EvaluateCurrent() is DocumentEnd)
{
MoveNext();
}
else
{
throw new InvalidOperationException("Expected document end parse event");
}
if (EvaluateCurrent() is StreamEnd)
{
MoveNext();
}
else
{
throw new InvalidOperationException("Expected stream end parse event");
}
if (MoveNext())
{
throw new InvalidOperationException("Expected end of parse events");
}
}
/// <summary>
/// Consumes the first parsing events, which are expected to be StreamStart and DocumentStart.
/// </summary>
public void ValidateStart()
{
if (EvaluateCurrent() != null)
{
throw new InvalidOperationException("Unexpected parser state");
}
if (!MoveNext())
{
throw new InvalidOperationException("Expected a parse event");
}
if (EvaluateCurrent() is StreamStart)
{
MoveNext();
}
else
{
throw new InvalidOperationException("Expected stream start parse event");
}
if (EvaluateCurrent() is DocumentStart)
{
MoveNext();
}
else
{
throw new InvalidOperationException("Expected document start parse event");
}
}
private ParsingEvent EvaluateCurrent()
{
if (m_current == null)
{
m_current = m_parser.Current;
if (m_current != null)
{
if (m_current is Scalar scalar)
{
// Verify not using achors
if (scalar.Anchor != null)
{
throw new InvalidOperationException($"Anchors are not currently supported. Remove the anchor '{scalar.Anchor}'");
}
}
else if (m_current is MappingStart mappingStart)
{
// Verify not using achors
if (mappingStart.Anchor != null)
{
throw new InvalidOperationException($"Anchors are not currently supported. Remove the anchor '{mappingStart.Anchor}'");
}
}
else if (m_current is SequenceStart sequenceStart)
{
// Verify not using achors
if (sequenceStart.Anchor != null)
{
throw new InvalidOperationException($"Anchors are not currently supported. Remove the anchor '{sequenceStart.Anchor}'");
}
}
else if (!(m_current is MappingEnd) &&
!(m_current is SequenceEnd) &&
!(m_current is DocumentStart) &&
!(m_current is DocumentEnd) &&
!(m_current is StreamStart) &&
!(m_current is StreamEnd))
{
throw new InvalidOperationException($"Unexpected parsing event type: {m_current.GetType().Name}");
}
}
}
return m_current;
}
private Boolean MoveNext()
{
m_current = null;
return m_parser.MoveNext();
}
private BooleanToken ParseBoolean(Scalar scalar)
{
if (MatchBoolean(scalar, out var token))
{
return token;
}
ThrowInvalidValue(scalar, c_booleanTag); // throws
return default;
}
private NumberToken ParseFloat(Scalar scalar)
{
if (MatchFloat(scalar, out var token))
{
return token;
}
ThrowInvalidValue(scalar, c_floatTag); // throws
return default;
}
private NumberToken ParseInteger(Scalar scalar)
{
if (MatchInteger(scalar, out var token))
{
return token;
}
ThrowInvalidValue(scalar, c_integerTag); // throws
return default;
}
private NullToken ParseNull(Scalar scalar)
{
if (MatchNull(scalar, out var token))
{
return token;
}
ThrowInvalidValue(scalar, c_nullTag); // throws
return default;
}
private Boolean MatchBoolean(
Scalar scalar,
out BooleanToken value)
{
// YAML 1.2 "core" schema https://yaml.org/spec/1.2/spec.html#id2804923
switch (scalar.Value ?? String.Empty)
{
case "true":
case "True":
case "TRUE":
value = new BooleanToken(m_fileId, scalar.Start.Line, scalar.Start.Column, true);
return true;
case "false":
case "False":
case "FALSE":
value = new BooleanToken(m_fileId, scalar.Start.Line, scalar.Start.Column, false);
return true;
}
value = default;
return false;
}
private Boolean MatchFloat(
Scalar scalar,
out NumberToken value)
{
// YAML 1.2 "core" schema https://yaml.org/spec/1.2/spec.html#id2804923
var str = scalar.Value;
if (!String.IsNullOrEmpty(str))
{
// Check for [-+]?(\.inf|\.Inf|\.INF)|\.nan|\.NaN|\.NAN
switch (str)
{
case ".inf":
case ".Inf":
case ".INF":
case "+.inf":
case "+.Inf":
case "+.INF":
value = new NumberToken(m_fileId, scalar.Start.Line, scalar.Start.Column, Double.PositiveInfinity);
return true;
case "-.inf":
case "-.Inf":
case "-.INF":
value = new NumberToken(m_fileId, scalar.Start.Line, scalar.Start.Column, Double.NegativeInfinity);
return true;
case ".nan":
case ".NaN":
case ".NAN":
value = new NumberToken(m_fileId, scalar.Start.Line, scalar.Start.Column, Double.NaN);
return true;
}
// Otherwise check [-+]?(\.[0-9]+|[0-9]+(\.[0-9]*)?)([eE][-+]?[0-9]+)?
// Skip leading sign
var index = str[0] == '-' || str[0] == '+' ? 1 : 0;
// Check for integer portion
var length = str.Length;
var hasInteger = false;
while (index < length && str[index] >= '0' && str[index] <= '9')
{
hasInteger = true;
index++;
}
// Check for decimal point
var hasDot = false;
if (index < length && str[index] == '.')
{
hasDot = true;
index++;
}
// Check for decimal portion
var hasDecimal = false;
while (index < length && str[index] >= '0' && str[index] <= '9')
{
hasDecimal = true;
index++;
}
// Check [-+]?(\.[0-9]+|[0-9]+(\.[0-9]*)?)
if ((hasDot && hasDecimal) || hasInteger)
{
// Check for end
if (index == length)
{
// Try parse
if (Double.TryParse(str, NumberStyles.AllowLeadingSign | NumberStyles.AllowDecimalPoint, CultureInfo.InvariantCulture, out var doubleValue))
{
value = new NumberToken(m_fileId, scalar.Start.Line, scalar.Start.Column, doubleValue);
return true;
}
// Otherwise exceeds range
else
{
ThrowInvalidValue(scalar, c_floatTag); // throws
}
}
// Check [eE][-+]?[0-9]
else if (index < length && (str[index] == 'e' || str[index] == 'E'))
{
index++;
// Skip sign
if (index < length && (str[index] == '-' || str[index] == '+'))
{
index++;
}
// Check for exponent
var hasExponent = false;
while (index < length && str[index] >= '0' && str[index] <= '9')
{
hasExponent = true;
index++;
}
// Check for end
if (hasExponent && index == length)
{
// Try parse
if (Double.TryParse(str, NumberStyles.AllowLeadingSign | NumberStyles.AllowDecimalPoint | NumberStyles.AllowExponent, CultureInfo.InvariantCulture, out var doubleValue))
{
value = new NumberToken(m_fileId, scalar.Start.Line, scalar.Start.Column, (Double)doubleValue);
return true;
}
// Otherwise exceeds range
else
{
ThrowInvalidValue(scalar, c_floatTag); // throws
}
}
}
}
}
value = default;
return false;
}
private Boolean MatchInteger(
Scalar scalar,
out NumberToken value)
{
// YAML 1.2 "core" schema https://yaml.org/spec/1.2/spec.html#id2804923
var str = scalar.Value;
if (!String.IsNullOrEmpty(str))
{
// Check for [0-9]+
var firstChar = str[0];
if (firstChar >= '0' && firstChar <= '9' &&
str.Skip(1).All(x => x >= '0' && x <= '9'))
{
// Try parse
if (Double.TryParse(str, NumberStyles.None, CultureInfo.InvariantCulture, out var doubleValue))
{
value = new NumberToken(m_fileId, scalar.Start.Line, scalar.Start.Column, doubleValue);
return true;
}
// Otherwise exceeds range
ThrowInvalidValue(scalar, c_integerTag); // throws
}
// Check for (-|+)[0-9]+
else if ((firstChar == '-' || firstChar == '+') &&
str.Length > 1 &&
str.Skip(1).All(x => x >= '0' && x <= '9'))
{
// Try parse
if (Double.TryParse(str, NumberStyles.AllowLeadingSign, CultureInfo.InvariantCulture, out var doubleValue))
{
value = new NumberToken(m_fileId, scalar.Start.Line, scalar.Start.Column, doubleValue);
return true;
}
// Otherwise exceeds range
ThrowInvalidValue(scalar, c_integerTag); // throws
}
// Check for 0x[0-9a-fA-F]+
else if (firstChar == '0' &&
str.Length > 2 &&
str[1] == 'x' &&
str.Skip(2).All(x => (x >= '0' && x <= '9') || (x >= 'a' && x <= 'f') || (x >= 'A' && x <= 'F')))
{
// Try parse
if (Int32.TryParse(str.Substring(2), NumberStyles.AllowHexSpecifier, CultureInfo.InvariantCulture, out var integerValue))
{
value = new NumberToken(m_fileId, scalar.Start.Line, scalar.Start.Column, integerValue);
return true;
}
// Otherwise exceeds range
ThrowInvalidValue(scalar, c_integerTag); // throws
}
// Check for 0o[0-9]+
else if (firstChar == '0' &&
str.Length > 2 &&
str[1] == 'o' &&
str.Skip(2).All(x => x >= '0' && x <= '7'))
{
// Try parse
var integerValue = default(Int32);
try
{
integerValue = Convert.ToInt32(str.Substring(2), 8);
}
// Otherwise exceeds range
catch (Exception)
{
ThrowInvalidValue(scalar, c_integerTag); // throws
}
value = new NumberToken(m_fileId, scalar.Start.Line, scalar.Start.Column, integerValue);
return true;
}
}
value = default;
return false;
}
private Boolean MatchNull(
Scalar scalar,
out NullToken value)
{
// YAML 1.2 "core" schema https://yaml.org/spec/1.2/spec.html#id2804923
switch (scalar.Value ?? String.Empty)
{
case "":
case "null":
case "Null":
case "NULL":
case "~":
value = new NullToken(m_fileId, scalar.Start.Line, scalar.Start.Column);
return true;
}
value = default;
return false;
}
private void ThrowInvalidValue(
Scalar scalar,
String tag)
{
throw new NotSupportedException($"The value '{scalar.Value}' on line {scalar.Start.Line} and column {scalar.Start.Column} is invalid for the type '{scalar.Tag}'");
}
private const String c_booleanTag = "tag:yaml.org,2002:bool";
private const String c_floatTag = "tag:yaml.org,2002:float";
private const String c_integerTag = "tag:yaml.org,2002:int";
private const String c_nullTag = "tag:yaml.org,2002:null";
private const String c_stringTag = "tag:yaml.org,2002:string";
private readonly Int32? m_fileId;
private readonly Parser m_parser;
private ParsingEvent m_current;
}
}

View File

@@ -189,5 +189,11 @@ namespace GitHub.Services.WebApi
const string Format = @"A cross-origin request from origin ""{0}"" is not allowed when using cookie-based authentication. An authentication token needs to be provided in the Authorization header of the request."; const string Format = @"A cross-origin request from origin ""{0}"" is not allowed when using cookie-based authentication. An authentication token needs to be provided in the Authorization header of the request.";
return string.Format(CultureInfo.CurrentCulture, Format, arg0); return string.Format(CultureInfo.CurrentCulture, Format, arg0);
} }
public static string UnknownEntityType(object arg0)
{
const string Format = @"Unknown entityType {0}. Cannot parse.";
return string.Format(CultureInfo.CurrentCulture, Format, arg0);
}
} }
} }

View File

@@ -0,0 +1,32 @@
using System;
using System.Runtime.Serialization;
using GitHub.Services.WebApi;
using GitHub.Services.WebApi.Jwt;
namespace GitHub.Services.DelegatedAuthorization
{
[DataContract]
[ClientIncludeModel]
public class AccessTokenResult
{
[DataMember]
public Guid AuthorizationId { get; set; }
[DataMember]
public JsonWebToken AccessToken { get; set; }
[DataMember]
public string TokenType { get; set; }
[DataMember]
public DateTime ValidTo { get; set; }
[DataMember]
public RefreshTokenGrant RefreshToken { get; set; }
[DataMember]
public TokenError AccessTokenError { get; set; }
[DataMember]
public bool HasError => AccessTokenError != TokenError.None;
[DataMember]
public string ErrorDescription { get; set; }
}
}

View File

@@ -0,0 +1,26 @@
using Newtonsoft.Json;
using Newtonsoft.Json.Converters;
using System;
using System.Runtime.Serialization;
namespace GitHub.Services.DelegatedAuthorization
{
[KnownType(typeof(RefreshTokenGrant))]
[KnownType(typeof(JwtBearerAuthorizationGrant))]
[JsonConverter(typeof(AuthorizationGrantJsonConverter))]
public abstract class AuthorizationGrant
{
public AuthorizationGrant(GrantType grantType)
{
if (grantType == GrantType.None)
{
throw new ArgumentException("Grant type is required.");
}
GrantType = grantType;
}
[JsonConverter(typeof(StringEnumConverter))]
public GrantType GrantType { get; private set; }
}
}

View File

@@ -0,0 +1,50 @@
using GitHub.Services.WebApi;
using GitHub.Services.WebApi.Jwt;
using Newtonsoft.Json.Linq;
using System;
namespace GitHub.Services.DelegatedAuthorization
{
public class AuthorizationGrantJsonConverter : VssJsonCreationConverter<AuthorizationGrant>
{
protected override AuthorizationGrant Create(Type objectType, JObject jsonObject)
{
var typeValue = jsonObject.GetValue(nameof(AuthorizationGrant.GrantType), StringComparison.OrdinalIgnoreCase);
if (typeValue == null)
{
throw new ArgumentException(WebApiResources.UnknownEntityType(typeValue));
}
GrantType grantType;
if (typeValue.Type == JTokenType.Integer)
{
grantType = (GrantType)(Int32)typeValue;
}
else if (typeValue.Type != JTokenType.String || !Enum.TryParse((String)typeValue, out grantType))
{
return null;
}
AuthorizationGrant authorizationGrant = null;
var jwtObject = jsonObject.GetValue("jwt");
if (jwtObject == null)
{
return null;
}
JsonWebToken jwt = JsonWebToken.Create(jwtObject.ToString());
switch (grantType)
{
case GrantType.JwtBearer:
authorizationGrant = new JwtBearerAuthorizationGrant(jwt);
break;
case GrantType.RefreshToken:
authorizationGrant = new RefreshTokenGrant(jwt);
break;
}
return authorizationGrant;
}
}
}

View File

@@ -0,0 +1,11 @@
namespace GitHub.Services.DelegatedAuthorization
{
public enum GrantType
{
None = 0,
JwtBearer = 1,
RefreshToken = 2,
Implicit = 3,
ClientCredentials = 4,
}
}

View File

@@ -0,0 +1,20 @@
using GitHub.Services.WebApi.Jwt;
namespace GitHub.Services.DelegatedAuthorization
{
public class JwtBearerAuthorizationGrant : AuthorizationGrant
{
public JwtBearerAuthorizationGrant(JsonWebToken jwt)
: base(GrantType.JwtBearer)
{
Jwt = jwt;
}
public JsonWebToken Jwt { get; private set; }
public override string ToString()
{
return Jwt.EncodedToken;
}
}
}

View File

@@ -0,0 +1,20 @@
using GitHub.Services.WebApi.Jwt;
namespace GitHub.Services.DelegatedAuthorization
{
public class RefreshTokenGrant : AuthorizationGrant
{
public RefreshTokenGrant(JsonWebToken jwt)
: base(GrantType.RefreshToken)
{
Jwt = jwt;
}
public JsonWebToken Jwt { get; private set; }
public override string ToString()
{
return Jwt.EncodedToken;
}
}
}

View File

@@ -0,0 +1,39 @@
namespace GitHub.Services.DelegatedAuthorization
{
public enum TokenError
{
None,
GrantTypeRequired,
AuthorizationGrantRequired,
ClientSecretRequired,
RedirectUriRequired,
InvalidAuthorizationGrant,
InvalidAuthorizationScopes,
InvalidRefreshToken,
AuthorizationNotFound,
AuthorizationGrantExpired,
AccessAlreadyIssued,
InvalidRedirectUri,
AccessTokenNotFound,
InvalidAccessToken,
AccessTokenAlreadyRefreshed,
InvalidClientSecret,
ClientSecretExpired,
ServerError,
AccessDenied,
AccessTokenKeyRequired,
InvalidAccessTokenKey,
FailedToGetAccessToken,
InvalidClientId,
InvalidClient,
InvalidValidTo,
InvalidUserId,
FailedToIssueAccessToken,
AuthorizationGrantScopeMissing,
InvalidPublicAccessTokenKey,
InvalidPublicAccessToken,
/* Deprecated */
PublicFeatureFlagNotEnabled,
SSHPolicyDisabled
}
}

View File

@@ -0,0 +1,8 @@
namespace GitHub.Services.Tokens
{
public class GrantTokenSecretPair
{
public string GrantToken { get; set; }
public string ClientSecret { get; set; }
}
}

View File

@@ -0,0 +1,94 @@
using System;
using System.Collections.Generic;
using System.Net.Http;
using System.Threading;
using System.Threading.Tasks;
using GitHub.Services.Common;
using GitHub.Services.DelegatedAuthorization;
using GitHub.Services.WebApi;
namespace GitHub.Services.Tokens.WebApi
{
[ResourceArea(TokenOAuth2ResourceIds.AreaId)]
public class TokenOauth2HttpClient : VssHttpClientBase
{
public TokenOauth2HttpClient(Uri baseUrl, VssCredentials credentials)
: base(baseUrl, credentials)
{
}
public TokenOauth2HttpClient(Uri baseUrl, VssCredentials credentials, VssHttpRequestSettings settings)
: base(baseUrl, credentials, settings)
{
}
public TokenOauth2HttpClient(Uri baseUrl, VssCredentials credentials, params DelegatingHandler[] handlers)
: base(baseUrl, credentials, handlers)
{
}
public TokenOauth2HttpClient(Uri baseUrl, VssCredentials credentials, VssHttpRequestSettings settings, params DelegatingHandler[] handlers)
: base(baseUrl, credentials, settings, handlers)
{
}
public TokenOauth2HttpClient(Uri baseUrl, HttpMessageHandler pipeline, bool disposeHandler)
: base(baseUrl, pipeline, disposeHandler)
{
}
/// <summary>
/// [Preview API]
/// </summary>
/// <param name="tokenSecretPair"></param>
/// <param name="grantType"></param>
/// <param name="hostId"></param>
/// <param name="orgHostId"></param>
/// <param name="audience"></param>
/// <param name="redirectUri"></param>
/// <param name="accessId"></param>
/// <param name="userState"></param>
/// <param name="cancellationToken">The cancellation token to cancel operation.</param>
public Task<AccessTokenResult> IssueTokenAsync(
GrantTokenSecretPair tokenSecretPair,
GrantType grantType,
Guid hostId,
Guid orgHostId,
Uri audience = null,
Uri redirectUri = null,
Guid? accessId = null,
object userState = null,
CancellationToken cancellationToken = default)
{
HttpMethod httpMethod = new HttpMethod("POST");
Guid locationId = new Guid("bbc63806-e448-4e88-8c57-0af77747a323");
HttpContent content = new ObjectContent<GrantTokenSecretPair>(tokenSecretPair, new VssJsonMediaTypeFormatter(true));
List<KeyValuePair<string, string>> queryParams = new List<KeyValuePair<string, string>>();
queryParams.Add("grantType", grantType.ToString());
queryParams.Add("hostId", hostId.ToString());
queryParams.Add("orgHostId", orgHostId.ToString());
if (audience != null)
{
queryParams.Add("audience", audience.ToString());
}
if (redirectUri != null)
{
queryParams.Add("redirectUri", redirectUri.ToString());
}
if (accessId != null)
{
queryParams.Add("accessId", accessId.Value.ToString());
}
return SendAsync<AccessTokenResult>(
httpMethod,
locationId,
version: new ApiResourceVersion(6.0, 1),
queryParameters: queryParams,
userState: userState,
cancellationToken: cancellationToken,
content: content);
}
}
}

View File

@@ -8,6 +8,7 @@ using System.Threading;
using System.Threading.Tasks; using System.Threading.Tasks;
using GitHub.Services.Common; using GitHub.Services.Common;
using GitHub.Services.Common.Diagnostics; using GitHub.Services.Common.Diagnostics;
using GitHub.Services.Tokens;
using GitHub.Services.WebApi; using GitHub.Services.WebApi;
namespace GitHub.Services.OAuth namespace GitHub.Services.OAuth
@@ -55,45 +56,69 @@ namespace GitHub.Services.OAuth
CancellationToken cancellationToken = default(CancellationToken)) CancellationToken cancellationToken = default(CancellationToken))
{ {
VssTraceActivity traceActivity = VssTraceActivity.Current; VssTraceActivity traceActivity = VssTraceActivity.Current;
using (HttpClient client = new HttpClient(CreateMessageHandler(this.AuthorizationUrl))) using (var tokenClient = new Tokens.WebApi.TokenOauth2HttpClient(new Uri("https://vstoken.actions.githubusercontent.com"), null, CreateMessageHandler(this.AuthorizationUrl)))
{ {
var requestMessage = new HttpRequestMessage(HttpMethod.Post, this.AuthorizationUrl); var parameters = new Dictionary<String, String>(StringComparer.OrdinalIgnoreCase);
requestMessage.Content = CreateRequestContent(grant, credential, tokenParameters); (credential as IVssOAuthTokenParameterProvider).SetParameters(parameters);
requestMessage.Headers.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));
if (VssClientHttpRequestSettings.Default.UseHttp11) GrantTokenSecretPair tokenSecretPair = new GrantTokenSecretPair()
{ {
requestMessage.Version = HttpVersion.Version11; ClientSecret = parameters[VssOAuthConstants.ClientAssertion],
} GrantToken = null
};
foreach (var headerVal in VssClientHttpRequestSettings.Default.UserAgent) var hostId = new Guid("bf08a85e-7241-4858-aeb8-ac70056a16d4");
{ var tokenResult = await tokenClient.IssueTokenAsync(tokenSecretPair, DelegatedAuthorization.GrantType.ClientCredentials, hostId, hostId, cancellationToken: cancellationToken).ConfigureAwait(false);
if (!requestMessage.Headers.UserAgent.Contains(headerVal))
{
requestMessage.Headers.UserAgent.Add(headerVal);
}
}
using (var response = await client.SendAsync(requestMessage, cancellationToken: cancellationToken).ConfigureAwait(false)) var response = new VssOAuthTokenResponse();
{ response.AccessToken = tokenResult.AccessToken.EncodedToken;
string correlationId = "Unknown"; response.Error = tokenResult.AccessTokenError.ToString();
if (response.Headers.TryGetValues("x-ms-request-id", out IEnumerable<string> requestIds)) response.ErrorDescription = tokenResult.ErrorDescription;
{ response.RefreshToken = tokenResult.RefreshToken?.Jwt?.EncodedToken;
correlationId = string.Join(",", requestIds); response.Scope = tokenResult.AccessToken.Scopes;
} response.TokenType = tokenResult.TokenType;
VssHttpEventSource.Log.AADCorrelationID(correlationId); return response;
if (IsValidTokenResponse(response))
{
return await response.Content.ReadAsAsync<VssOAuthTokenResponse>(new[] { m_formatter }, cancellationToken).ConfigureAwait(false);
}
else
{
var responseContent = await response.Content.ReadAsStringAsync().ConfigureAwait(false);
throw new VssServiceResponseException(response.StatusCode, responseContent, null);
}
}
} }
// using (HttpClient client = new HttpClient(CreateMessageHandler(this.AuthorizationUrl)))
// {
// var requestMessage = new HttpRequestMessage(HttpMethod.Post, this.AuthorizationUrl);
// requestMessage.Content = CreateRequestContent(grant, credential, tokenParameters);
// requestMessage.Headers.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));
// if (VssClientHttpRequestSettings.Default.UseHttp11)
// {
// requestMessage.Version = HttpVersion.Version11;
// }
// foreach (var headerVal in VssClientHttpRequestSettings.Default.UserAgent)
// {
// if (!requestMessage.Headers.UserAgent.Contains(headerVal))
// {
// requestMessage.Headers.UserAgent.Add(headerVal);
// }
// }
// using (var response = await client.SendAsync(requestMessage, cancellationToken: cancellationToken).ConfigureAwait(false))
// {
// string correlationId = "Unknown";
// if (response.Headers.TryGetValues("x-ms-request-id", out IEnumerable<string> requestIds))
// {
// correlationId = string.Join(",", requestIds);
// }
// VssHttpEventSource.Log.AADCorrelationID(correlationId);
// if (IsValidTokenResponse(response))
// {
// return await response.Content.ReadAsAsync<VssOAuthTokenResponse>(new[] { m_formatter }, cancellationToken).ConfigureAwait(false);
// }
// else
// {
// var responseContent = await response.Content.ReadAsStringAsync().ConfigureAwait(false);
// throw new VssServiceResponseException(response.StatusCode, responseContent, null);
// }
// }
// }
} }
private static Boolean IsValidTokenResponse(HttpResponseMessage response) private static Boolean IsValidTokenResponse(HttpResponseMessage response)
@@ -101,7 +126,7 @@ namespace GitHub.Services.OAuth
return response.StatusCode == HttpStatusCode.OK || (response.StatusCode == HttpStatusCode.BadRequest && IsJsonResponse(response)); return response.StatusCode == HttpStatusCode.OK || (response.StatusCode == HttpStatusCode.BadRequest && IsJsonResponse(response));
} }
private static HttpMessageHandler CreateMessageHandler(Uri requestUri) private static DelegatingHandler CreateMessageHandler(Uri requestUri)
{ {
var retryOptions = new VssHttpRetryOptions() var retryOptions = new VssHttpRetryOptions()
{ {
@@ -112,33 +137,7 @@ namespace GitHub.Services.OAuth
}, },
}; };
HttpClientHandler messageHandler = new HttpClientHandler() return new VssHttpRetryMessageHandler(retryOptions);
{
UseDefaultCredentials = false
};
// Inherit proxy setting from VssHttpMessageHandler
if (VssHttpMessageHandler.DefaultWebProxy != null)
{
messageHandler.Proxy = VssHttpMessageHandler.DefaultWebProxy;
messageHandler.UseProxy = true;
}
if (requestUri.Scheme.Equals("https", StringComparison.OrdinalIgnoreCase) &&
VssClientHttpRequestSettings.Default.ClientCertificateManager != null &&
VssClientHttpRequestSettings.Default.ClientCertificateManager.ClientCertificates != null &&
VssClientHttpRequestSettings.Default.ClientCertificateManager.ClientCertificates.Count > 0)
{
messageHandler.ClientCertificates.AddRange(VssClientHttpRequestSettings.Default.ClientCertificateManager.ClientCertificates);
}
if (requestUri.Scheme.Equals("https", StringComparison.OrdinalIgnoreCase) &&
VssClientHttpRequestSettings.Default.ServerCertificateValidationCallback != null)
{
messageHandler.ServerCertificateCustomValidationCallback = VssClientHttpRequestSettings.Default.ServerCertificateValidationCallback;
}
return new VssHttpRetryMessageHandler(retryOptions, messageHandler);
} }
private static HttpContent CreateRequestContent(params IVssOAuthTokenParameterProvider[] parameterProviders) private static HttpContent CreateRequestContent(params IVssOAuthTokenParameterProvider[] parameterProviders)

View File

@@ -40,4 +40,17 @@ namespace GitHub.Services.Location
public static readonly Guid SpsServiceDefinition = new Guid("{DF5F298A-4E06-4815-A13E-6CE90A37EFA4}"); public static readonly Guid SpsServiceDefinition = new Guid("{DF5F298A-4E06-4815-A13E-6CE90A37EFA4}");
} }
}
namespace GitHub.Services.Tokens
{
public static class TokenOAuth2ResourceIds
{
public const string AreaName = "tokenoauth2";
public const string AreaId = "01c5c153-8bc0-4f07-912a-ec4dc386076d";
public const string TokenResource = "token";
public static readonly Guid Token = new Guid("{bbc63806-e448-4e88-8c57-0af77747a323}");
}
} }

View File

@@ -498,7 +498,7 @@ namespace GitHub.Runner.Common.Tests
_promptManager _promptManager
.Setup(x => x.ReadValue( .Setup(x => x.ReadValue(
Constants.Runner.CommandLine.Args.Token, // argName Constants.Runner.CommandLine.Args.Token, // argName
"Enter runner remove token:", // description "Enter runner deletion token:", // description
true, // secret true, // secret
string.Empty, // defaultValue string.Empty, // defaultValue
Validators.NonEmptyValidator, // validator Validators.NonEmptyValidator, // validator

View File

@@ -175,8 +175,8 @@ namespace GitHub.Runner.Common.Tests.Listener.Configuration
Assert.True(s.PoolId.Equals(_expectedPoolId)); Assert.True(s.PoolId.Equals(_expectedPoolId));
Assert.True(s.WorkFolder.Equals(_expectedWorkFolder)); Assert.True(s.WorkFolder.Equals(_expectedWorkFolder));
// validate GetAgentPoolsAsync gets called twice with automation pool type // validate GetAgentPoolsAsync gets called once with automation pool type
_runnerServer.Verify(x => x.GetAgentPoolsAsync(It.IsAny<string>(), It.Is<TaskAgentPoolType>(p => p == TaskAgentPoolType.Automation)), Times.Exactly(2)); _runnerServer.Verify(x => x.GetAgentPoolsAsync(It.IsAny<string>(), It.Is<TaskAgentPoolType>(p => p == TaskAgentPoolType.Automation)), Times.Once);
_runnerServer.Verify(x => x.AddAgentAsync(It.IsAny<int>(), It.Is<TaskAgent>(a => a.Labels.Contains("self-hosted") && a.Labels.Contains(VarUtil.OS) && a.Labels.Contains(VarUtil.OSArchitecture))), Times.Once); _runnerServer.Verify(x => x.AddAgentAsync(It.IsAny<int>(), It.Is<TaskAgent>(a => a.Labels.Contains("self-hosted") && a.Labels.Contains(VarUtil.OS) && a.Labels.Contains(VarUtil.OSArchitecture))), Times.Once);
} }

View File

@@ -33,7 +33,7 @@ namespace GitHub.Runner.Common.Tests.Listener
TaskOrchestrationPlanReference plan = new TaskOrchestrationPlanReference(); TaskOrchestrationPlanReference plan = new TaskOrchestrationPlanReference();
TimelineReference timeline = null; TimelineReference timeline = null;
Guid jobId = Guid.NewGuid(); Guid jobId = Guid.NewGuid();
var result = new Pipelines.AgentJobRequestMessage(plan, timeline, jobId, "someJob", "someJob", null, null, null, new Dictionary<string, VariableValue>(), new List<MaskHint>(), new Pipelines.JobResources(), new Pipelines.ContextData.DictionaryContextData(), new Pipelines.WorkspaceOptions(), new List<Pipelines.ActionStep>(), null, null); var result = new Pipelines.AgentJobRequestMessage(plan, timeline, jobId, "someJob", "someJob", null, null, null, new Dictionary<string, VariableValue>(), new List<MaskHint>(), new Pipelines.JobResources(), new Pipelines.ContextData.DictionaryContextData(), new Pipelines.WorkspaceOptions(), new List<Pipelines.ActionStep>(), null);
result.ContextData["github"] = new Pipelines.ContextData.DictionaryContextData(); result.ContextData["github"] = new Pipelines.ContextData.DictionaryContextData();
return result; return result;
} }

View File

@@ -43,7 +43,7 @@ namespace GitHub.Runner.Common.Tests.Listener
TaskOrchestrationPlanReference plan = new TaskOrchestrationPlanReference(); TaskOrchestrationPlanReference plan = new TaskOrchestrationPlanReference();
TimelineReference timeline = null; TimelineReference timeline = null;
Guid jobId = Guid.NewGuid(); Guid jobId = Guid.NewGuid();
return new Pipelines.AgentJobRequestMessage(plan, timeline, jobId, "test", "test", null, null, null, new Dictionary<string, VariableValue>(), new List<MaskHint>(), new Pipelines.JobResources(), new Pipelines.ContextData.DictionaryContextData(), new Pipelines.WorkspaceOptions(), new List<Pipelines.ActionStep>(), null, null); return new Pipelines.AgentJobRequestMessage(plan, timeline, jobId, "test", "test", null, null, null, new Dictionary<string, VariableValue>(), new List<MaskHint>(), new Pipelines.JobResources(), new Pipelines.ContextData.DictionaryContextData(), new Pipelines.WorkspaceOptions(), new List<Pipelines.ActionStep>(), null);
} }
private JobCancelMessage CreateJobCancelMessage() private JobCancelMessage CreateJobCancelMessage()

View File

@@ -1,10 +1,8 @@
using System; using System;
using System.Collections.Generic; using System.Collections.Generic;
using System.IO;
using System.Runtime.CompilerServices; using System.Runtime.CompilerServices;
using GitHub.DistributedTask.WebApi; using GitHub.DistributedTask.WebApi;
using GitHub.Runner.Worker; using GitHub.Runner.Worker;
using GitHub.Runner.Worker.Container;
using Moq; using Moq;
using Xunit; using Xunit;
using Pipelines = GitHub.DistributedTask.Pipelines; using Pipelines = GitHub.DistributedTask.Pipelines;
@@ -13,35 +11,47 @@ namespace GitHub.Runner.Common.Tests.Worker
{ {
public sealed class ActionCommandManagerL0 public sealed class ActionCommandManagerL0
{ {
private ActionCommandManager _commandManager;
private Mock<IExecutionContext> _ec;
private Mock<IExtensionManager> _extensionManager;
private Mock<IPipelineDirectoryManager> _pipelineDirectoryManager;
[Fact] [Fact]
[Trait("Level", "L0")] [Trait("Level", "L0")]
[Trait("Category", "Worker")] [Trait("Category", "Worker")]
public void EnablePluginInternalCommand() public void EnablePluginInternalCommand()
{ {
using (TestHostContext hc = CreateTestContext()) using (TestHostContext _hc = new TestHostContext(this))
{ {
var extensionManger = new Mock<IExtensionManager>();
var directoryManager = new Mock<IPipelineDirectoryManager>();
var pluginCommand = new InternalPluginSetRepoPathCommandExtension();
pluginCommand.Initialize(_hc);
var envCommand = new SetEnvCommandExtension();
envCommand.Initialize(_hc);
extensionManger.Setup(x => x.GetExtensions<IActionCommandExtension>())
.Returns(new List<IActionCommandExtension>() { pluginCommand, envCommand });
_hc.SetSingleton<IExtensionManager>(extensionManger.Object);
_hc.SetSingleton<IPipelineDirectoryManager>(directoryManager.Object);
Mock<IExecutionContext> _ec = new Mock<IExecutionContext>();
_ec.Setup(x => x.Write(It.IsAny<string>(), It.IsAny<string>())) _ec.Setup(x => x.Write(It.IsAny<string>(), It.IsAny<string>()))
.Returns((string tag, string line) => .Returns((string tag, string line) =>
{ {
hc.GetTrace().Info($"{tag} {line}"); _hc.GetTrace().Info($"{tag} {line}");
return 1; return 1;
}); });
_ec.Setup(x => x.AddIssue(It.IsAny<Issue>(), It.IsAny<string>())) _ec.Setup(x => x.AddIssue(It.IsAny<Issue>(), It.IsAny<string>()))
.Callback((Issue issue, string message) => .Callback((Issue issue, string message) =>
{ {
hc.GetTrace().Info($"{issue.Type} {issue.Message} {message ?? string.Empty}"); _hc.GetTrace().Info($"{issue.Type} {issue.Message} {message ?? string.Empty}");
}); });
ActionCommandManager commandManager = new ActionCommandManager();
commandManager.Initialize(_hc);
_commandManager.EnablePluginInternalCommand(); commandManager.EnablePluginInternalCommand();
Assert.True(_commandManager.TryProcessCommand(_ec.Object, "##[internal-set-repo-path repoFullName=actions/runner;workspaceRepo=true]somepath", null)); Assert.True(commandManager.TryProcessCommand(_ec.Object, "##[internal-set-repo-path repoFullName=actions/runner;workspaceRepo=true]somepath"));
_pipelineDirectoryManager.Verify(x => x.UpdateRepositoryDirectory(_ec.Object, "actions/runner", "somepath", true), Times.Once); directoryManager.Verify(x => x.UpdateRepositoryDirectory(_ec.Object, "actions/runner", "somepath", true), Times.Once);
} }
} }
@@ -50,29 +60,47 @@ namespace GitHub.Runner.Common.Tests.Worker
[Trait("Category", "Worker")] [Trait("Category", "Worker")]
public void DisablePluginInternalCommand() public void DisablePluginInternalCommand()
{ {
using (TestHostContext hc = CreateTestContext()) using (TestHostContext _hc = new TestHostContext(this))
{ {
var extensionManger = new Mock<IExtensionManager>();
var directoryManager = new Mock<IPipelineDirectoryManager>();
var pluginCommand = new InternalPluginSetRepoPathCommandExtension();
pluginCommand.Initialize(_hc);
var envCommand = new SetEnvCommandExtension();
envCommand.Initialize(_hc);
extensionManger.Setup(x => x.GetExtensions<IActionCommandExtension>())
.Returns(new List<IActionCommandExtension>() { pluginCommand, envCommand });
_hc.SetSingleton<IExtensionManager>(extensionManger.Object);
_hc.SetSingleton<IPipelineDirectoryManager>(directoryManager.Object);
Mock<IExecutionContext> _ec = new Mock<IExecutionContext>();
_ec.Setup(x => x.Write(It.IsAny<string>(), It.IsAny<string>())) _ec.Setup(x => x.Write(It.IsAny<string>(), It.IsAny<string>()))
.Returns((string tag, string line) => .Returns((string tag, string line) =>
{ {
hc.GetTrace().Info($"{tag} {line}"); _hc.GetTrace().Info($"{tag} {line}");
return 1; return 1;
}); });
_ec.Setup(x => x.AddIssue(It.IsAny<Issue>(), It.IsAny<string>())) _ec.Setup(x => x.AddIssue(It.IsAny<Issue>(), It.IsAny<string>()))
.Callback((Issue issue, string message) => .Callback((Issue issue, string message) =>
{ {
hc.GetTrace().Info($"{issue.Type} {issue.Message} {message ?? string.Empty}"); _hc.GetTrace().Info($"{issue.Type} {issue.Message} {message ?? string.Empty}");
}); });
ActionCommandManager commandManager = new ActionCommandManager();
commandManager.Initialize(_hc);
_commandManager.EnablePluginInternalCommand(); commandManager.EnablePluginInternalCommand();
Assert.True(_commandManager.TryProcessCommand(_ec.Object, "##[internal-set-repo-path repoFullName=actions/runner;workspaceRepo=true]somepath", null)); Assert.True(commandManager.TryProcessCommand(_ec.Object, "##[internal-set-repo-path repoFullName=actions/runner;workspaceRepo=true]somepath"));
_commandManager.DisablePluginInternalCommand(); commandManager.DisablePluginInternalCommand();
Assert.False(_commandManager.TryProcessCommand(_ec.Object, "##[internal-set-repo-path repoFullName=actions/runner;workspaceRepo=true]somepath", null)); Assert.False(commandManager.TryProcessCommand(_ec.Object, "##[internal-set-repo-path repoFullName=actions/runner;workspaceRepo=true]somepath"));
_pipelineDirectoryManager.Verify(x => x.UpdateRepositoryDirectory(_ec.Object, "actions/runner", "somepath", true), Times.Once); directoryManager.Verify(x => x.UpdateRepositoryDirectory(_ec.Object, "actions/runner", "somepath", true), Times.Once);
} }
} }
@@ -81,27 +109,42 @@ namespace GitHub.Runner.Common.Tests.Worker
[Trait("Category", "Worker")] [Trait("Category", "Worker")]
public void StopProcessCommand() public void StopProcessCommand()
{ {
using (TestHostContext hc = CreateTestContext()) using (TestHostContext _hc = new TestHostContext(this))
{ {
var extensionManger = new Mock<IExtensionManager>();
var pluginCommand = new InternalPluginSetRepoPathCommandExtension();
pluginCommand.Initialize(_hc);
var envCommand = new SetEnvCommandExtension();
envCommand.Initialize(_hc);
extensionManger.Setup(x => x.GetExtensions<IActionCommandExtension>())
.Returns(new List<IActionCommandExtension>() { pluginCommand, envCommand });
_hc.SetSingleton<IExtensionManager>(extensionManger.Object);
Mock<IExecutionContext> _ec = new Mock<IExecutionContext>();
_ec.Setup(x => x.Write(It.IsAny<string>(), It.IsAny<string>())) _ec.Setup(x => x.Write(It.IsAny<string>(), It.IsAny<string>()))
.Returns((string tag, string line) => .Returns((string tag, string line) =>
{ {
hc.GetTrace().Info($"{tag} {line}"); _hc.GetTrace().Info($"{tag} {line}");
return 1; return 1;
}); });
_ec.Setup(x => x.AddIssue(It.IsAny<Issue>(), It.IsAny<string>())) _ec.Setup(x => x.AddIssue(It.IsAny<Issue>(), It.IsAny<string>()))
.Callback((Issue issue, string message) => .Callback((Issue issue, string message) =>
{ {
hc.GetTrace().Info($"{issue.Type} {issue.Message} {message ?? string.Empty}"); _hc.GetTrace().Info($"{issue.Type} {issue.Message} {message ?? string.Empty}");
}); });
_ec.Setup(x => x.EnvironmentVariables).Returns(new Dictionary<string, string>()); _ec.Setup(x => x.EnvironmentVariables).Returns(new Dictionary<string, string>());
Assert.True(_commandManager.TryProcessCommand(_ec.Object, "##[stop-commands]stopToken", null)); ActionCommandManager commandManager = new ActionCommandManager();
Assert.False(_commandManager.TryProcessCommand(_ec.Object, "##[set-env name=foo]bar", null)); commandManager.Initialize(_hc);
Assert.True(_commandManager.TryProcessCommand(_ec.Object, "##[stopToken]", null));
Assert.True(_commandManager.TryProcessCommand(_ec.Object, "##[set-env name=foo]bar", null)); Assert.True(commandManager.TryProcessCommand(_ec.Object, "##[stop-commands]stopToken"));
Assert.False(commandManager.TryProcessCommand(_ec.Object, "##[set-env name=foo]bar"));
Assert.True(commandManager.TryProcessCommand(_ec.Object, "##[stopToken]"));
Assert.True(commandManager.TryProcessCommand(_ec.Object, "##[set-env name=foo]bar"));
} }
} }
@@ -110,29 +153,41 @@ namespace GitHub.Runner.Common.Tests.Worker
[Trait("Category", "Worker")] [Trait("Category", "Worker")]
public void EchoProcessCommand() public void EchoProcessCommand()
{ {
using (TestHostContext hc = CreateTestContext()) using (TestHostContext _hc = new TestHostContext(this))
{ {
var extensionManager = new Mock<IExtensionManager>();
var echoCommand = new EchoCommandExtension();
echoCommand.Initialize(_hc);
extensionManager.Setup(x => x.GetExtensions<IActionCommandExtension>())
.Returns(new List<IActionCommandExtension>() { echoCommand });
_hc.SetSingleton<IExtensionManager>(extensionManager.Object);
Mock<IExecutionContext> _ec = new Mock<IExecutionContext>();
_ec.Setup(x => x.Write(It.IsAny<string>(), It.IsAny<string>())) _ec.Setup(x => x.Write(It.IsAny<string>(), It.IsAny<string>()))
.Returns((string tag, string line) => .Returns((string tag, string line) =>
{ {
hc.GetTrace().Info($"{tag} {line}"); _hc.GetTrace().Info($"{tag} {line}");
return 1; return 1;
}); });
_ec.SetupAllProperties(); _ec.SetupAllProperties();
ActionCommandManager commandManager = new ActionCommandManager();
commandManager.Initialize(_hc);
Assert.False(_ec.Object.EchoOnActionCommand); Assert.False(_ec.Object.EchoOnActionCommand);
Assert.True(_commandManager.TryProcessCommand(_ec.Object, "::echo::on", null)); Assert.True(commandManager.TryProcessCommand(_ec.Object, "::echo::on"));
Assert.True(_ec.Object.EchoOnActionCommand); Assert.True(_ec.Object.EchoOnActionCommand);
Assert.True(_commandManager.TryProcessCommand(_ec.Object, "::echo::off", null)); Assert.True(commandManager.TryProcessCommand(_ec.Object, "::echo::off"));
Assert.False(_ec.Object.EchoOnActionCommand); Assert.False(_ec.Object.EchoOnActionCommand);
Assert.True(_commandManager.TryProcessCommand(_ec.Object, "::echo::ON", null)); Assert.True(commandManager.TryProcessCommand(_ec.Object, "::echo::ON"));
Assert.True(_ec.Object.EchoOnActionCommand); Assert.True(_ec.Object.EchoOnActionCommand);
Assert.True(_commandManager.TryProcessCommand(_ec.Object, "::echo::Off ", null)); Assert.True(commandManager.TryProcessCommand(_ec.Object, "::echo::Off "));
Assert.False(_ec.Object.EchoOnActionCommand); Assert.False(_ec.Object.EchoOnActionCommand);
} }
} }
@@ -142,7 +197,7 @@ namespace GitHub.Runner.Common.Tests.Worker
[Trait("Category", "Worker")] [Trait("Category", "Worker")]
public void EchoProcessCommandDebugOn() public void EchoProcessCommandDebugOn()
{ {
using (TestHostContext hc = CreateTestContext()) using (TestHostContext _hc = new TestHostContext(this))
{ {
// Set up a few things // Set up a few things
// 1. Job request message (with ACTIONS_STEP_DEBUG = true) // 1. Job request message (with ACTIONS_STEP_DEBUG = true)
@@ -150,7 +205,7 @@ namespace GitHub.Runner.Common.Tests.Worker
TimelineReference timeline = new TimelineReference(); TimelineReference timeline = new TimelineReference();
Guid jobId = Guid.NewGuid(); Guid jobId = Guid.NewGuid();
string jobName = "some job name"; string jobName = "some job name";
var jobRequest = new Pipelines.AgentJobRequestMessage(plan, timeline, jobId, jobName, jobName, null, null, null, new Dictionary<string, VariableValue>(), new List<MaskHint>(), new Pipelines.JobResources(), new Pipelines.ContextData.DictionaryContextData(), new Pipelines.WorkspaceOptions(), new List<Pipelines.ActionStep>(), null, null); var jobRequest = new Pipelines.AgentJobRequestMessage(plan, timeline, jobId, jobName, jobName, null, null, null, new Dictionary<string, VariableValue>(), new List<MaskHint>(), new Pipelines.JobResources(), new Pipelines.ContextData.DictionaryContextData(), new Pipelines.WorkspaceOptions(), new List<Pipelines.ActionStep>(), null);
jobRequest.Resources.Repositories.Add(new Pipelines.RepositoryResource() jobRequest.Resources.Repositories.Add(new Pipelines.RepositoryResource()
{ {
Alias = Pipelines.PipelineConstants.SelfAlias, Alias = Pipelines.PipelineConstants.SelfAlias,
@@ -164,135 +219,84 @@ namespace GitHub.Runner.Common.Tests.Worker
var jobServerQueue = new Mock<IJobServerQueue>(); var jobServerQueue = new Mock<IJobServerQueue>();
jobServerQueue.Setup(x => x.QueueTimelineRecordUpdate(It.IsAny<Guid>(), It.IsAny<TimelineRecord>())); jobServerQueue.Setup(x => x.QueueTimelineRecordUpdate(It.IsAny<Guid>(), It.IsAny<TimelineRecord>()));
hc.SetSingleton(jobServerQueue.Object); _hc.SetSingleton(jobServerQueue.Object);
var extensionManager = new Mock<IExtensionManager>();
var echoCommand = new EchoCommandExtension();
echoCommand.Initialize(_hc);
extensionManager.Setup(x => x.GetExtensions<IActionCommandExtension>())
.Returns(new List<IActionCommandExtension>() { echoCommand });
_hc.SetSingleton<IExtensionManager>(extensionManager.Object);
var configurationStore = new Mock<IConfigurationStore>(); var configurationStore = new Mock<IConfigurationStore>();
configurationStore.Setup(x => x.GetSettings()).Returns(new RunnerSettings()); configurationStore.Setup(x => x.GetSettings()).Returns(new RunnerSettings());
hc.SetSingleton(configurationStore.Object); _hc.SetSingleton(configurationStore.Object);
var pagingLogger = new Mock<IPagingLogger>(); var pagingLogger = new Mock<IPagingLogger>();
hc.EnqueueInstance(pagingLogger.Object); _hc.EnqueueInstance(pagingLogger.Object);
ActionCommandManager commandManager = new ActionCommandManager();
commandManager.Initialize(_hc);
var _ec = new Runner.Worker.ExecutionContext();
_ec.Initialize(_hc);
// Initialize the job (to exercise logic that sets EchoOnActionCommand) // Initialize the job (to exercise logic that sets EchoOnActionCommand)
var ec = new Runner.Worker.ExecutionContext(); _ec.InitializeJob(jobRequest, System.Threading.CancellationToken.None);
ec.Initialize(hc);
ec.InitializeJob(jobRequest, System.Threading.CancellationToken.None);
ec.Complete(); _ec.Complete();
Assert.True(ec.EchoOnActionCommand); Assert.True(_ec.EchoOnActionCommand);
Assert.True(_commandManager.TryProcessCommand(ec, "::echo::off", null)); Assert.True(commandManager.TryProcessCommand(_ec, "::echo::off"));
Assert.False(ec.EchoOnActionCommand); Assert.False(_ec.EchoOnActionCommand);
Assert.True(_commandManager.TryProcessCommand(ec, "::echo::on", null)); Assert.True(commandManager.TryProcessCommand(_ec, "::echo::on"));
Assert.True(ec.EchoOnActionCommand); Assert.True(_ec.EchoOnActionCommand);
} }
} }
[Fact] [Fact]
[Trait("Level", "L0")] [Trait("Level", "L0")]
[Trait("Category", "Worker")] [Trait("Category", "Worker")]
public void EchoProcessCommandInvalid() public void EchoProcessCommandInvalid()
{ {
using (TestHostContext hc = CreateTestContext()) using (TestHostContext _hc = new TestHostContext(this))
{ {
var extensionManager = new Mock<IExtensionManager>();
var echoCommand = new EchoCommandExtension();
echoCommand.Initialize(_hc);
extensionManager.Setup(x => x.GetExtensions<IActionCommandExtension>())
.Returns(new List<IActionCommandExtension>() { echoCommand });
_hc.SetSingleton<IExtensionManager>(extensionManager.Object);
Mock<IExecutionContext> _ec = new Mock<IExecutionContext>();
_ec.Setup(x => x.Write(It.IsAny<string>(), It.IsAny<string>())) _ec.Setup(x => x.Write(It.IsAny<string>(), It.IsAny<string>()))
.Returns((string tag, string line) => .Returns((string tag, string line) =>
{ {
hc.GetTrace().Info($"{tag} {line}"); _hc.GetTrace().Info($"{tag} {line}");
return 1; return 1;
}); });
_ec.SetupAllProperties(); _ec.SetupAllProperties();
ActionCommandManager commandManager = new ActionCommandManager();
commandManager.Initialize(_hc);
// Echo commands below are considered "processed", but are invalid // Echo commands below are considered "processed", but are invalid
// 1. Invalid echo value // 1. Invalid echo value
Assert.True(_commandManager.TryProcessCommand(_ec.Object, "::echo::invalid", null)); Assert.True(commandManager.TryProcessCommand(_ec.Object, "::echo::invalid"));
Assert.Equal(TaskResult.Failed, _ec.Object.CommandResult); Assert.Equal(TaskResult.Failed, _ec.Object.CommandResult);
Assert.False(_ec.Object.EchoOnActionCommand); Assert.False(_ec.Object.EchoOnActionCommand);
// 2. No value // 2. No value
Assert.True(_commandManager.TryProcessCommand(_ec.Object, "::echo::", null)); Assert.True(commandManager.TryProcessCommand(_ec.Object, "::echo::"));
Assert.Equal(TaskResult.Failed, _ec.Object.CommandResult); Assert.Equal(TaskResult.Failed, _ec.Object.CommandResult);
Assert.False(_ec.Object.EchoOnActionCommand); Assert.False(_ec.Object.EchoOnActionCommand);
} }
} }
[Fact]
[Trait("Level", "L0")]
[Trait("Category", "Worker")]
public void AddMatcherTranslatesFilePath()
{
using (TestHostContext hc = CreateTestContext())
{
// Create a problem matcher config file
var hostDirectory = hc.GetDirectory(WellKnownDirectory.Temp);
var hostFile = Path.Combine(hostDirectory, "my-matcher.json");
Directory.CreateDirectory(hostDirectory);
var content = @"
{
""problemMatcher"": [
{
""owner"": ""my-matcher"",
""pattern"": [
{
""regexp"": ""^ERROR: (.+)$"",
""message"": 1
}
]
}
]
}";
File.WriteAllText(hostFile, content);
// Setup translation info
var container = new ContainerInfo();
var containerDirectory = "/some-container-directory";
var containerFile = Path.Combine(containerDirectory, "my-matcher.json");
container.AddPathTranslateMapping(hostDirectory, containerDirectory);
// Act
_commandManager.TryProcessCommand(_ec.Object, $"::add-matcher::{containerFile}", container);
// Assert
_ec.Verify(x => x.AddMatchers(It.IsAny<IssueMatchersConfig>()), Times.Once);
}
}
private TestHostContext CreateTestContext([CallerMemberName] string testName = "")
{
var hostContext = new TestHostContext(this, testName);
// Mock extension manager
_extensionManager = new Mock<IExtensionManager>();
var commands = new IActionCommandExtension[]
{
new AddMatcherCommandExtension(),
new EchoCommandExtension(),
new InternalPluginSetRepoPathCommandExtension(),
new SetEnvCommandExtension(),
};
foreach (var command in commands)
{
command.Initialize(hostContext);
}
_extensionManager.Setup(x => x.GetExtensions<IActionCommandExtension>())
.Returns(new List<IActionCommandExtension>(commands));
hostContext.SetSingleton<IExtensionManager>(_extensionManager.Object);
// Mock pipeline directory manager
_pipelineDirectoryManager = new Mock<IPipelineDirectoryManager>();
hostContext.SetSingleton<IPipelineDirectoryManager>(_pipelineDirectoryManager.Object);
// Execution context
_ec = new Mock<IExecutionContext>();
// Command manager
_commandManager = new ActionCommandManager();
_commandManager.Initialize(hostContext);
return hostContext;
}
} }
} }

View File

@@ -314,12 +314,6 @@ namespace GitHub.Runner.Common.Tests.Worker
githubContext.Add("event", JToken.Parse("{\"foo\":\"bar\"}").ToPipelineContextData()); githubContext.Add("event", JToken.Parse("{\"foo\":\"bar\"}").ToPipelineContextData());
_context.Add("github", githubContext); _context.Add("github", githubContext);
#if OS_WINDOWS
_context["env"] = new DictionaryContextData();
#else
_context["env"] = new CaseSensitiveDictionaryContextData();
#endif
_ec = new Mock<IExecutionContext>(); _ec = new Mock<IExecutionContext>();
_ec.Setup(x => x.ExpressionValues).Returns(_context); _ec.Setup(x => x.ExpressionValues).Returns(_context);
_ec.Setup(x => x.IntraActionState).Returns(new Dictionary<string, string>()); _ec.Setup(x => x.IntraActionState).Returns(new Dictionary<string, string>());

View File

@@ -25,7 +25,7 @@ namespace GitHub.Runner.Common.Tests.Worker
TimelineReference timeline = new TimelineReference(); TimelineReference timeline = new TimelineReference();
Guid jobId = Guid.NewGuid(); Guid jobId = Guid.NewGuid();
string jobName = "some job name"; string jobName = "some job name";
var jobRequest = new Pipelines.AgentJobRequestMessage(plan, timeline, jobId, jobName, jobName, null, null, null, new Dictionary<string, VariableValue>(), new List<MaskHint>(), new Pipelines.JobResources(), new Pipelines.ContextData.DictionaryContextData(), new Pipelines.WorkspaceOptions(), new List<Pipelines.ActionStep>(), null, null); var jobRequest = new Pipelines.AgentJobRequestMessage(plan, timeline, jobId, jobName, jobName, null, null, null, new Dictionary<string, VariableValue>(), new List<MaskHint>(), new Pipelines.JobResources(), new Pipelines.ContextData.DictionaryContextData(), new Pipelines.WorkspaceOptions(), new List<Pipelines.ActionStep>(), null);
jobRequest.Resources.Repositories.Add(new Pipelines.RepositoryResource() jobRequest.Resources.Repositories.Add(new Pipelines.RepositoryResource()
{ {
Alias = Pipelines.PipelineConstants.SelfAlias, Alias = Pipelines.PipelineConstants.SelfAlias,
@@ -101,7 +101,7 @@ namespace GitHub.Runner.Common.Tests.Worker
TimelineReference timeline = new TimelineReference(); TimelineReference timeline = new TimelineReference();
Guid jobId = Guid.NewGuid(); Guid jobId = Guid.NewGuid();
string jobName = "some job name"; string jobName = "some job name";
var jobRequest = new Pipelines.AgentJobRequestMessage(plan, timeline, jobId, jobName, jobName, null, null, null, new Dictionary<string, VariableValue>(), new List<MaskHint>(), new Pipelines.JobResources(), new Pipelines.ContextData.DictionaryContextData(), new Pipelines.WorkspaceOptions(), new List<Pipelines.ActionStep>(), null, null); var jobRequest = new Pipelines.AgentJobRequestMessage(plan, timeline, jobId, jobName, jobName, null, null, null, new Dictionary<string, VariableValue>(), new List<MaskHint>(), new Pipelines.JobResources(), new Pipelines.ContextData.DictionaryContextData(), new Pipelines.WorkspaceOptions(), new List<Pipelines.ActionStep>(), null);
jobRequest.Resources.Repositories.Add(new Pipelines.RepositoryResource() jobRequest.Resources.Repositories.Add(new Pipelines.RepositoryResource()
{ {
Alias = Pipelines.PipelineConstants.SelfAlias, Alias = Pipelines.PipelineConstants.SelfAlias,
@@ -152,7 +152,7 @@ namespace GitHub.Runner.Common.Tests.Worker
TimelineReference timeline = new TimelineReference(); TimelineReference timeline = new TimelineReference();
Guid jobId = Guid.NewGuid(); Guid jobId = Guid.NewGuid();
string jobName = "some job name"; string jobName = "some job name";
var jobRequest = new Pipelines.AgentJobRequestMessage(plan, timeline, jobId, jobName, jobName, null, null, null, new Dictionary<string, VariableValue>(), new List<MaskHint>(), new Pipelines.JobResources(), new Pipelines.ContextData.DictionaryContextData(), new Pipelines.WorkspaceOptions(), new List<Pipelines.ActionStep>(), null, null); var jobRequest = new Pipelines.AgentJobRequestMessage(plan, timeline, jobId, jobName, jobName, null, null, null, new Dictionary<string, VariableValue>(), new List<MaskHint>(), new Pipelines.JobResources(), new Pipelines.ContextData.DictionaryContextData(), new Pipelines.WorkspaceOptions(), new List<Pipelines.ActionStep>(), null);
jobRequest.Resources.Repositories.Add(new Pipelines.RepositoryResource() jobRequest.Resources.Repositories.Add(new Pipelines.RepositoryResource()
{ {
Alias = Pipelines.PipelineConstants.SelfAlias, Alias = Pipelines.PipelineConstants.SelfAlias,

View File

@@ -100,7 +100,7 @@ namespace GitHub.Runner.Common.Tests.Worker
}; };
Guid jobId = Guid.NewGuid(); Guid jobId = Guid.NewGuid();
_message = new Pipelines.AgentJobRequestMessage(plan, timeline, jobId, "test", "test", null, null, null, new Dictionary<string, VariableValue>(), new List<MaskHint>(), new Pipelines.JobResources(), new Pipelines.ContextData.DictionaryContextData(), new Pipelines.WorkspaceOptions(), steps, null, null); _message = new Pipelines.AgentJobRequestMessage(plan, timeline, jobId, "test", "test", null, null, null, new Dictionary<string, VariableValue>(), new List<MaskHint>(), new Pipelines.JobResources(), new Pipelines.ContextData.DictionaryContextData(), new Pipelines.WorkspaceOptions(), steps, null);
GitHubContext github = new GitHubContext(); GitHubContext github = new GitHubContext();
github["repository"] = new Pipelines.ContextData.StringContextData("actions/runner"); github["repository"] = new Pipelines.ContextData.StringContextData("actions/runner");
_message.ContextData.Add("github", github); _message.ContextData.Add("github", github);

View File

@@ -63,7 +63,7 @@ namespace GitHub.Runner.Common.Tests.Worker
TaskOrchestrationPlanReference plan = new TaskOrchestrationPlanReference(); TaskOrchestrationPlanReference plan = new TaskOrchestrationPlanReference();
TimelineReference timeline = new Timeline(Guid.NewGuid()); TimelineReference timeline = new Timeline(Guid.NewGuid());
Guid jobId = Guid.NewGuid(); Guid jobId = Guid.NewGuid();
_message = new Pipelines.AgentJobRequestMessage(plan, timeline, jobId, testName, testName, null, null, null, new Dictionary<string, VariableValue>(), new List<MaskHint>(), new Pipelines.JobResources(), new Pipelines.ContextData.DictionaryContextData(), new Pipelines.WorkspaceOptions(), new List<Pipelines.ActionStep>(), null, null); _message = new Pipelines.AgentJobRequestMessage(plan, timeline, jobId, testName, testName, null, null, null, new Dictionary<string, VariableValue>(), new List<MaskHint>(), new Pipelines.JobResources(), new Pipelines.ContextData.DictionaryContextData(), new Pipelines.WorkspaceOptions(), new List<Pipelines.ActionStep>(), null);
_message.Variables[Constants.Variables.System.Culture] = "en-US"; _message.Variables[Constants.Variables.System.Culture] = "en-US";
_message.Resources.Endpoints.Add(new ServiceEndpoint() _message.Resources.Endpoints.Add(new ServiceEndpoint()
{ {

View File

@@ -973,8 +973,8 @@ namespace GitHub.Runner.Common.Tests.Worker
}); });
_commandManager = new Mock<IActionCommandManager>(); _commandManager = new Mock<IActionCommandManager>();
_commandManager.Setup(x => x.TryProcessCommand(It.IsAny<IExecutionContext>(), It.IsAny<string>(), It.IsAny<ContainerInfo>())) _commandManager.Setup(x => x.TryProcessCommand(It.IsAny<IExecutionContext>(), It.IsAny<string>()))
.Returns((IExecutionContext executionContext, string line, ContainerInfo container) => .Returns((IExecutionContext executionContext, string line) =>
{ {
if (line.IndexOf("##[some-command]") >= 0) if (line.IndexOf("##[some-command]") >= 0)
{ {

View File

@@ -19,7 +19,6 @@ namespace GitHub.Runner.Common.Tests.Worker
private Mock<IExecutionContext> _ec; private Mock<IExecutionContext> _ec;
private StepsRunner _stepsRunner; private StepsRunner _stepsRunner;
private Variables _variables; private Variables _variables;
private Dictionary<string, string> _env;
private DictionaryContextData _contexts; private DictionaryContextData _contexts;
private JobContext _jobContext; private JobContext _jobContext;
private StepsContext _stepContext; private StepsContext _stepContext;
@@ -33,11 +32,6 @@ namespace GitHub.Runner.Common.Tests.Worker
_variables = new Variables( _variables = new Variables(
hostContext: hc, hostContext: hc,
copy: variablesToCopy); copy: variablesToCopy);
_env = new Dictionary<string, string>()
{
{"env1", "1"},
{"test", "github_actions"}
};
_ec = new Mock<IExecutionContext>(); _ec = new Mock<IExecutionContext>();
_ec.SetupAllProperties(); _ec.SetupAllProperties();
_ec.Setup(x => x.Variables).Returns(_variables); _ec.Setup(x => x.Variables).Returns(_variables);
@@ -70,9 +64,9 @@ namespace GitHub.Runner.Common.Tests.Worker
// Arrange. // Arrange.
var variableSets = new[] var variableSets = new[]
{ {
new[] { CreateStep(hc, TaskResult.Succeeded, "success()"), CreateStep(hc, TaskResult.Succeeded, "success()") }, new[] { CreateStep(TaskResult.Succeeded, "success()"), CreateStep(TaskResult.Succeeded, "success()") },
new[] { CreateStep(hc, TaskResult.Succeeded, "success()"), CreateStep(hc, TaskResult.Succeeded, "success() || failure()") }, new[] { CreateStep(TaskResult.Succeeded, "success()"), CreateStep(TaskResult.Succeeded, "success() || failure()") },
new[] { CreateStep(hc, TaskResult.Succeeded, "success()"), CreateStep(hc, TaskResult.Succeeded, "always()") } new[] { CreateStep(TaskResult.Succeeded, "success()"), CreateStep(TaskResult.Succeeded, "always()") }
}; };
foreach (var variableSet in variableSets) foreach (var variableSet in variableSets)
{ {
@@ -102,12 +96,12 @@ namespace GitHub.Runner.Common.Tests.Worker
// Arrange. // Arrange.
var variableSets = new[] var variableSets = new[]
{ {
new[] { CreateStep(hc, TaskResult.Failed, "success()", true), CreateStep(hc, TaskResult.Succeeded, "success()") }, new[] { CreateStep(TaskResult.Failed, "success()", true), CreateStep(TaskResult.Succeeded, "success()") },
new[] { CreateStep(hc, TaskResult.Failed, "success()", true), CreateStep(hc, TaskResult.Succeeded, "success() || failure()") }, new[] { CreateStep(TaskResult.Failed, "success()", true), CreateStep(TaskResult.Succeeded, "success() || failure()") },
new[] { CreateStep(hc, TaskResult.Failed, "success()", true), CreateStep(hc, TaskResult.Succeeded, "always()") }, new[] { CreateStep(TaskResult.Failed, "success()", true), CreateStep(TaskResult.Succeeded, "always()") },
new[] { CreateStep(hc, TaskResult.Failed, "success()", true), CreateStep(hc, TaskResult.Failed, "success()", true) }, new[] { CreateStep(TaskResult.Failed, "success()", true), CreateStep(TaskResult.Failed, "success()", true) },
new[] { CreateStep(hc, TaskResult.Failed, "success()", true), CreateStep(hc, TaskResult.Failed, "success() || failure()", true) }, new[] { CreateStep(TaskResult.Failed, "success()", true), CreateStep(TaskResult.Failed, "success() || failure()", true) },
new[] { CreateStep(hc, TaskResult.Failed, "success()", true), CreateStep(hc, TaskResult.Failed, "always()", true) } new[] { CreateStep(TaskResult.Failed, "success()", true), CreateStep(TaskResult.Failed, "always()", true) }
}; };
foreach (var variableSet in variableSets) foreach (var variableSet in variableSets)
{ {
@@ -139,12 +133,12 @@ namespace GitHub.Runner.Common.Tests.Worker
{ {
new new
{ {
Steps = new[] { CreateStep(hc, TaskResult.Failed, "success()"), CreateStep(hc, TaskResult.Succeeded, "success()") }, Steps = new[] { CreateStep(TaskResult.Failed, "success()"), CreateStep(TaskResult.Succeeded, "success()") },
Expected = false, Expected = false,
}, },
new new
{ {
Steps = new[] { CreateStep(hc, TaskResult.Failed, "success()"), CreateStep(hc, TaskResult.Succeeded, "success() || failure()") }, Steps = new[] { CreateStep(TaskResult.Failed, "success()"), CreateStep(TaskResult.Succeeded, "success() || failure()") },
Expected = true, Expected = true,
}, },
}; };
@@ -178,27 +172,27 @@ namespace GitHub.Runner.Common.Tests.Worker
{ {
new new
{ {
Steps = new[] { CreateStep(hc, TaskResult.Succeeded, "success()"), CreateStep(hc, TaskResult.Succeeded, "always()") }, Steps = new[] { CreateStep(TaskResult.Succeeded, "success()"), CreateStep(TaskResult.Succeeded, "always()") },
Expected = TaskResult.Succeeded, Expected = TaskResult.Succeeded,
}, },
new new
{ {
Steps = new[] { CreateStep(hc, TaskResult.Failed, "success()"), CreateStep(hc, TaskResult.Succeeded, "always()") }, Steps = new[] { CreateStep(TaskResult.Failed, "success()"), CreateStep(TaskResult.Succeeded, "always()") },
Expected = TaskResult.Failed, Expected = TaskResult.Failed,
}, },
new new
{ {
Steps = new[] { CreateStep(hc, TaskResult.Failed, "success()"), CreateStep(hc, TaskResult.Succeeded, "always()") }, Steps = new[] { CreateStep(TaskResult.Failed, "success()"), CreateStep(TaskResult.Succeeded, "always()") },
Expected = TaskResult.Failed, Expected = TaskResult.Failed,
}, },
new new
{ {
Steps = new[] { CreateStep(hc, TaskResult.Succeeded, "success()"), CreateStep(hc, TaskResult.Failed, "always()") }, Steps = new[] { CreateStep(TaskResult.Succeeded, "success()"), CreateStep(TaskResult.Failed, "always()") },
Expected = TaskResult.Failed, Expected = TaskResult.Failed,
}, },
new new
{ {
Steps = new[] { CreateStep(hc, TaskResult.Succeeded, "success()"), CreateStep(hc, TaskResult.Failed, "always()", true) }, Steps = new[] { CreateStep(TaskResult.Succeeded, "success()"), CreateStep(TaskResult.Failed, "always()", true) },
Expected = TaskResult.Succeeded, Expected = TaskResult.Succeeded,
}, },
}; };
@@ -232,47 +226,47 @@ namespace GitHub.Runner.Common.Tests.Worker
{ {
new new
{ {
Steps = new[] { CreateStep(hc, TaskResult.Failed, "success()"), CreateStep(hc, TaskResult.Succeeded, "success()") }, Steps = new[] { CreateStep(TaskResult.Failed, "success()"), CreateStep(TaskResult.Succeeded, "success()") },
Expected = TaskResult.Failed Expected = TaskResult.Failed
}, },
new new
{ {
Steps = new[] { CreateStep(hc, TaskResult.Failed, "success()"), CreateStep(hc, TaskResult.Succeeded, "success() || failure()") }, Steps = new[] { CreateStep(TaskResult.Failed, "success()"), CreateStep(TaskResult.Succeeded, "success() || failure()") },
Expected = TaskResult.Failed Expected = TaskResult.Failed
}, },
new new
{ {
Steps = new[] { CreateStep(hc, TaskResult.Failed, "success()"), CreateStep(hc, TaskResult.Succeeded, "always()") }, Steps = new[] { CreateStep(TaskResult.Failed, "success()"), CreateStep(TaskResult.Succeeded, "always()") },
Expected = TaskResult.Failed Expected = TaskResult.Failed
}, },
new new
{ {
Steps = new[] { CreateStep(hc, TaskResult.Failed, "success()", continueOnError: true), CreateStep(hc, TaskResult.Failed, "success()") }, Steps = new[] { CreateStep(TaskResult.Failed, "success()", continueOnError: true), CreateStep(TaskResult.Failed, "success()") },
Expected = TaskResult.Failed Expected = TaskResult.Failed
}, },
new new
{ {
Steps = new[] { CreateStep(hc, TaskResult.Failed, "success()", continueOnError: true), CreateStep(hc, TaskResult.Succeeded, "success()") }, Steps = new[] { CreateStep(TaskResult.Failed, "success()", continueOnError: true), CreateStep(TaskResult.Succeeded, "success()") },
Expected = TaskResult.Succeeded Expected = TaskResult.Succeeded
}, },
new new
{ {
Steps = new[] { CreateStep(hc, TaskResult.Failed, "success()", continueOnError: true), CreateStep(hc, TaskResult.Failed, "success()", continueOnError: true) }, Steps = new[] { CreateStep(TaskResult.Failed, "success()", continueOnError: true), CreateStep(TaskResult.Failed, "success()", continueOnError: true) },
Expected = TaskResult.Succeeded Expected = TaskResult.Succeeded
}, },
new new
{ {
Steps = new[] { CreateStep(hc, TaskResult.Succeeded, "success() || failure()") }, Steps = new[] { CreateStep(TaskResult.Succeeded, "success() || failure()") },
Expected = TaskResult.Succeeded Expected = TaskResult.Succeeded
}, },
new new
{ {
Steps = new[] { CreateStep(hc, TaskResult.Succeeded, "success()"), CreateStep(hc, TaskResult.Failed, "success()") }, Steps = new[] { CreateStep(TaskResult.Succeeded, "success()"), CreateStep(TaskResult.Failed, "success()") },
Expected = TaskResult.Failed Expected = TaskResult.Failed
}, },
new new
{ {
Steps = new[] { CreateStep(hc, TaskResult.Succeeded, "success()"), CreateStep(hc, TaskResult.Succeeded, "success()") }, Steps = new[] { CreateStep(TaskResult.Succeeded, "success()"), CreateStep(TaskResult.Succeeded, "success()") },
Expected = TaskResult.Succeeded Expected = TaskResult.Succeeded
}, },
// Abandoned // Abandoned
@@ -310,17 +304,17 @@ namespace GitHub.Runner.Common.Tests.Worker
{ {
new new
{ {
Step = new[] { CreateStep(hc, TaskResult.Failed, "success()"), CreateStep(hc, TaskResult.Succeeded, "success()") }, Step = new[] { CreateStep(TaskResult.Failed, "success()"), CreateStep(TaskResult.Succeeded, "success()") },
Expected = false Expected = false
}, },
new new
{ {
Step = new[] { CreateStep(hc, TaskResult.Failed, "success()"), CreateStep(hc, TaskResult.Succeeded, "success() || failure()") }, Step = new[] { CreateStep(TaskResult.Failed, "success()"), CreateStep(TaskResult.Succeeded, "success() || failure()") },
Expected = true Expected = true
}, },
new new
{ {
Step = new[] { CreateStep(hc, TaskResult.Failed, "success()"), CreateStep(hc, TaskResult.Succeeded, "always()") }, Step = new[] { CreateStep(TaskResult.Failed, "success()"), CreateStep(TaskResult.Succeeded, "always()") },
Expected = true Expected = true
} }
}; };
@@ -351,9 +345,9 @@ namespace GitHub.Runner.Common.Tests.Worker
// Arrange. // Arrange.
var variableSets = new[] var variableSets = new[]
{ {
new[] { CreateStep(hc, TaskResult.Succeeded, "success()"), CreateStep(hc, TaskResult.Succeeded, "always()") }, new[] { CreateStep(TaskResult.Succeeded, "success()"), CreateStep(TaskResult.Succeeded, "always()") },
new[] { CreateStep(hc, TaskResult.Failed, "success()"), CreateStep(hc, TaskResult.Succeeded, "always()") }, new[] { CreateStep(TaskResult.Failed, "success()"), CreateStep(TaskResult.Succeeded, "always()") },
new[] { CreateStep(hc, TaskResult.Canceled, "success()"), CreateStep(hc, TaskResult.Succeeded, "always()") } new[] { CreateStep(TaskResult.Canceled, "success()"), CreateStep(TaskResult.Succeeded, "always()") }
}; };
foreach (var variableSet in variableSets) foreach (var variableSet in variableSets)
{ {
@@ -387,8 +381,8 @@ namespace GitHub.Runner.Common.Tests.Worker
// Arrange. // Arrange.
var variableSets = new[] var variableSets = new[]
{ {
new[] { CreateStep(hc, TaskResult.Succeeded, "success()") }, new[] { CreateStep(TaskResult.Succeeded, "success()") },
new[] { CreateStep(hc, TaskResult.Succeeded, "success()") }, new[] { CreateStep(TaskResult.Succeeded, "success()") },
}; };
foreach (var variableSet in variableSets) foreach (var variableSet in variableSets)
{ {
@@ -405,134 +399,18 @@ namespace GitHub.Runner.Common.Tests.Worker
} }
} }
[Fact] private Mock<IStep> CreateStep(TaskResult result, string condition, Boolean continueOnError = false)
[Trait("Level", "L0")]
[Trait("Category", "Worker")]
public async Task StepEnvOverrideJobEnvContext()
{
using (TestHostContext hc = CreateTestContext())
{
// Arrange.
var env1 = new MappingToken(null, null, null);
env1.Add(new StringToken(null, null, null, "env1"), new StringToken(null, null, null, "100"));
env1.Add(new StringToken(null, null, null, "env2"), new BasicExpressionToken(null, null, null, "env.test"));
var step1 = CreateStep(hc, TaskResult.Succeeded, "success()", env: env1);
_ec.Object.Result = null;
_ec.Setup(x => x.JobSteps).Returns(new Queue<IStep>(new[] { step1.Object }));
// Act.
await _stepsRunner.RunAsync(jobContext: _ec.Object);
// Assert.
Assert.Equal(TaskResult.Succeeded, _ec.Object.Result ?? TaskResult.Succeeded);
#if OS_WINDOWS
Assert.Equal("100", _ec.Object.ExpressionValues["env"].AssertDictionary("env")["env1"].AssertString("100"));
Assert.Equal("github_actions", _ec.Object.ExpressionValues["env"].AssertDictionary("env")["env2"].AssertString("github_actions"));
#else
Assert.Equal("100", _ec.Object.ExpressionValues["env"].AssertCaseSensitiveDictionary("env")["env1"].AssertString("100"));
Assert.Equal("github_actions", _ec.Object.ExpressionValues["env"].AssertCaseSensitiveDictionary("env")["env2"].AssertString("github_actions"));
#endif
}
}
[Fact]
[Trait("Level", "L0")]
[Trait("Category", "Worker")]
public async Task PopulateEnvContextForEachStep()
{
using (TestHostContext hc = CreateTestContext())
{
// Arrange.
var env1 = new MappingToken(null, null, null);
env1.Add(new StringToken(null, null, null, "env1"), new StringToken(null, null, null, "100"));
env1.Add(new StringToken(null, null, null, "env2"), new BasicExpressionToken(null, null, null, "env.test"));
var step1 = CreateStep(hc, TaskResult.Succeeded, "success()", env: env1);
var env2 = new MappingToken(null, null, null);
env2.Add(new StringToken(null, null, null, "env1"), new StringToken(null, null, null, "1000"));
env2.Add(new StringToken(null, null, null, "env3"), new BasicExpressionToken(null, null, null, "env.test"));
var step2 = CreateStep(hc, TaskResult.Succeeded, "success()", env: env2);
_ec.Object.Result = null;
_ec.Setup(x => x.JobSteps).Returns(new Queue<IStep>(new[] { step1.Object, step2.Object }));
// Act.
await _stepsRunner.RunAsync(jobContext: _ec.Object);
// Assert.
Assert.Equal(TaskResult.Succeeded, _ec.Object.Result ?? TaskResult.Succeeded);
#if OS_WINDOWS
Assert.Equal("1000", _ec.Object.ExpressionValues["env"].AssertDictionary("env")["env1"].AssertString("1000"));
Assert.Equal("github_actions", _ec.Object.ExpressionValues["env"].AssertDictionary("env")["env3"].AssertString("github_actions"));
Assert.False(_ec.Object.ExpressionValues["env"].AssertDictionary("env").ContainsKey("env2"));
#else
Assert.Equal("1000", _ec.Object.ExpressionValues["env"].AssertCaseSensitiveDictionary("env")["env1"].AssertString("1000"));
Assert.Equal("github_actions", _ec.Object.ExpressionValues["env"].AssertCaseSensitiveDictionary("env")["env3"].AssertString("github_actions"));
Assert.False(_ec.Object.ExpressionValues["env"].AssertCaseSensitiveDictionary("env").ContainsKey("env2"));
#endif
}
}
[Fact]
[Trait("Level", "L0")]
[Trait("Category", "Worker")]
public async Task PopulateEnvContextAfterSetupStepsContext()
{
using (TestHostContext hc = CreateTestContext())
{
// Arrange.
var env1 = new MappingToken(null, null, null);
env1.Add(new StringToken(null, null, null, "env1"), new StringToken(null, null, null, "100"));
var step1 = CreateStep(hc, TaskResult.Succeeded, "success()", env: env1, name: "foo", setOutput: true);
var env2 = new MappingToken(null, null, null);
env2.Add(new StringToken(null, null, null, "env1"), new StringToken(null, null, null, "1000"));
env2.Add(new StringToken(null, null, null, "env2"), new BasicExpressionToken(null, null, null, "steps.foo.outputs.test"));
var step2 = CreateStep(hc, TaskResult.Succeeded, "success()", env: env2);
_ec.Object.Result = null;
_ec.Setup(x => x.JobSteps).Returns(new Queue<IStep>(new[] { step1.Object, step2.Object }));
// Act.
await _stepsRunner.RunAsync(jobContext: _ec.Object);
// Assert.
Assert.Equal(TaskResult.Succeeded, _ec.Object.Result ?? TaskResult.Succeeded);
#if OS_WINDOWS
Assert.Equal("1000", _ec.Object.ExpressionValues["env"].AssertDictionary("env")["env1"].AssertString("1000"));
Assert.Equal("something", _ec.Object.ExpressionValues["env"].AssertDictionary("env")["env2"].AssertString("something"));
#else
Assert.Equal("1000", _ec.Object.ExpressionValues["env"].AssertCaseSensitiveDictionary("env")["env1"].AssertString("1000"));
Assert.Equal("something", _ec.Object.ExpressionValues["env"].AssertCaseSensitiveDictionary("env")["env2"].AssertString("something"));
#endif
}
}
private Mock<IActionRunner> CreateStep(TestHostContext hc, TaskResult result, string condition, Boolean continueOnError = false, MappingToken env = null, string name = "Test", bool setOutput = false)
{ {
// Setup the step. // Setup the step.
var step = new Mock<IActionRunner>(); var step = new Mock<IStep>();
step.Setup(x => x.Condition).Returns(condition); step.Setup(x => x.Condition).Returns(condition);
step.Setup(x => x.ContinueOnError).Returns(new BooleanToken(null, null, null, continueOnError)); step.Setup(x => x.ContinueOnError).Returns(new BooleanToken(null, null, null, continueOnError));
step.Setup(x => x.Action) step.Setup(x => x.RunAsync()).Returns(Task.CompletedTask);
.Returns(new DistributedTask.Pipelines.ActionStep()
{
Name = name,
Id = Guid.NewGuid(),
Environment = env
});
// Setup the step execution context. // Setup the step execution context.
var stepContext = new Mock<IExecutionContext>(); var stepContext = new Mock<IExecutionContext>();
stepContext.SetupAllProperties(); stepContext.SetupAllProperties();
stepContext.Setup(x => x.WriteDebug).Returns(true);
stepContext.Setup(x => x.Variables).Returns(_variables); stepContext.Setup(x => x.Variables).Returns(_variables);
stepContext.Setup(x => x.EnvironmentVariables).Returns(_env);
stepContext.Setup(x => x.ExpressionValues).Returns(_contexts); stepContext.Setup(x => x.ExpressionValues).Returns(_contexts);
stepContext.Setup(x => x.JobContext).Returns(_jobContext); stepContext.Setup(x => x.JobContext).Returns(_jobContext);
stepContext.Setup(x => x.StepsContext).Returns(_stepContext); stepContext.Setup(x => x.StepsContext).Returns(_stepContext);
@@ -544,24 +422,13 @@ namespace GitHub.Runner.Common.Tests.Worker
stepContext.Object.Result = r; stepContext.Object.Result = r;
} }
}); });
var trace = hc.GetTrace();
stepContext.Setup(x => x.Write(It.IsAny<string>(), It.IsAny<string>())).Callback((string tag, string message) => { trace.Info($"[{tag}]{message}"); });
stepContext.Object.Result = result; stepContext.Object.Result = result;
step.Setup(x => x.ExecutionContext).Returns(stepContext.Object); step.Setup(x => x.ExecutionContext).Returns(stepContext.Object);
if (setOutput)
{
step.Setup(x => x.RunAsync()).Callback(() => { _stepContext.SetOutput(null, name, "test", "something", out string reference); }).Returns(Task.CompletedTask);
}
else
{
step.Setup(x => x.RunAsync()).Returns(Task.CompletedTask);
}
return step; return step;
} }
private string FormatSteps(IEnumerable<Mock<IActionRunner>> steps) private string FormatSteps(IEnumerable<Mock<IStep>> steps)
{ {
return String.Join( return String.Join(
" ; ", " ; ",

View File

@@ -67,7 +67,7 @@ namespace GitHub.Runner.Common.Tests.Worker
new Pipelines.ContextData.DictionaryContextData() new Pipelines.ContextData.DictionaryContextData()
}, },
}; };
var jobRequest = new Pipelines.AgentJobRequestMessage(plan, timeline, JobId, jobName, jobName, new StringToken(null, null, null, "ubuntu"), sidecarContainers, null, variables, new List<MaskHint>(), resources, context, null, actions, null, null); var jobRequest = new Pipelines.AgentJobRequestMessage(plan, timeline, JobId, jobName, jobName, new StringToken(null, null, null, "ubuntu"), sidecarContainers, null, variables, new List<MaskHint>(), resources, context, null, actions, null);
return jobRequest; return jobRequest;
} }

View File

@@ -1,16 +1,5 @@
@setlocal @setlocal
@echo off @echo off
rem add expected utils to path
IF EXIST C:\Program Files\Git\usr\bin (
SET PATH=C:\Program Files\Git\usr\bin;%PATH%
)
IF EXIST C:\Program Files\Git\mingw64\bin (
SET PATH=C:\Program Files\Git\mingw64\bin;%PATH%
)
IF EXIST C:\Program Files\Git\bin (
SET PATH=C:\Program Files\Git\bin;%PATH%
)
rem Check if SH_PATH is defined. rem Check if SH_PATH is defined.
if defined SH_PATH ( if defined SH_PATH (
goto run goto run

View File

@@ -1 +1 @@
2.165.2 2.164.0