Compare commits

...

14 Commits

Author SHA1 Message Date
TingluoHuang
7817e1a976 Prepare 2.169.0 runner release for GHES Alpha. 2020-04-08 11:32:56 -04:00
Tingluo Huang
d90273a068 Raise warning when volume mount root. (#413) 2020-04-08 11:17:54 -04:00
TingluoHuang
2cdde6cb16 fix L0 test. 2020-04-07 14:04:37 -04:00
TingluoHuang
1f52dfa636 bump dev-dependency version. 2020-04-07 14:00:37 -04:00
dependabot[bot]
83b5742278 Bump acorn from 6.4.0 to 6.4.1 in /src/Misc/expressionFunc/hashFiles (#371)
Bumps [acorn](https://github.com/acornjs/acorn) from 6.4.0 to 6.4.1.
- [Release notes](https://github.com/acornjs/acorn/releases)
- [Commits](https://github.com/acornjs/acorn/compare/6.4.0...6.4.1)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2020-04-07 13:49:48 -04:00
eric sciple
ba69b5bc93 Fix runner config IsHostedServer detection for GHES alpha (#401) 2020-04-01 17:29:57 -04:00
Tingluo Huang
0e8777ebda ADR for wrapper action (#361)
* wrapper action adr.

* rename

* updates.

* Update 0361-wrapper-action.md
2020-03-31 10:11:02 -04:00
Bryan MacFarlane
a5f06b3ec2 ADR 397: Configuration time custom labels (#397)
Since configuring self-hosted runners is commonly automated via scripts, the labels need to be able to be created during configuration. The runner currently registers the built-in labels (os, arch) during registration but does not accept labels via command line args to extend the set registered.
2020-03-30 17:46:32 -04:00
Tingluo Huang
be325f26a6 support config with GHES url. (#393) 2020-03-30 14:48:02 -04:00
Josh Soref
dec260920f spelling: deprecate (#394) 2020-03-30 07:45:06 -04:00
eric sciple
b0a1294ef5 Fix API URL for GHES (#390) 2020-03-27 00:16:02 -04:00
Tingluo Huang
3d70ef2da1 update workflow schema file. (#388) 2020-03-26 23:01:17 -04:00
eric sciple
e23d68f6e2 add github.url and github.api_url for ghes alpha (#386) 2020-03-25 15:11:52 -04:00
eric sciple
dff1024cd3 cache actions on premises (#381) 2020-03-24 21:51:37 -04:00
18 changed files with 976 additions and 490 deletions

View File

@@ -0,0 +1,75 @@
# ADR 361: Wrapper Action
**Date**: 2020-03-06
**Status**: Pending
## Context
In addition to action's regular execution, action author may wants their action has a chance to participate in:
- Job initialize
My Action will collect machine resource usage (CPU/RAM/Disk) during a workflow job execution, we need to start perf recorder at the begin of the job.
- Job cleanup
My Action will dirty local workspace or machine environment during execution, we need to cleanup these changes at the end of the job.
Ex: `actions/checkout@v2` will write `github.token` into local `.git/config` during execution, it has post job cleanup defined to undo the changes.
## Decision
### Add `pre` and `post` execution to action
Node Action Example:
```yaml
name: 'My action with pre'
description: 'My action with pre'
runs:
using: 'node12'
pre: 'setup.js'
pre-if: 'success()' // Optional
main: 'index.js'
post: 'cleanup.js'
post-if: 'success()' // Optional
```
Container Action Example:
```yaml
name: 'My action with pre'
description: 'My action with pre'
runs:
using: 'docker'
image: 'mycontainer:latest'
pre-entrypoint: 'setup.sh'
pre-if: 'success()' // Optional
entrypoint: 'entrypoint.sh'
post-entrypoint: 'cleanup.sh'
post-if: 'success()' // Optional
```
Both `pre` and `post` will has default `pre-if/post-if` sets to `always()`.
Setting `pre` to `always()` will make sure no matter what condition evaluate result the `main` gets at runtime, the `pre` has always run already.
`pre` executes in order of how the steps are defined.
`pre` will always be added to job steps list during job setup.
> Action referenced from local repository (`./my-action`) won't get `pre` setup correctly since the repository haven't checkout during job initialize.
> We can't use GitHub api to download the repository since there is a about 3 mins delay between `git push` and the new commit available to download using GitHub api.
`post` will be pushed into a `poststeps` stack lazily when the action's `pre` or `main` execution passed `if` condition check and about to run, you can't have an action that only contains a `post`, we will pop and run each `post` after all `pre` and `main` finished.
> Currently `post` works for both repository action (`org/repo@v1`) and local action (`./my-action`)
Valid action:
- only has `main`
- has `pre` and `main`
- has `main` and `post`
- has `pre`, `main` and `post`
Invalid action:
- only has `pre`
- only has `post`
- has `pre` and `post`
Potential downside of introducing `pre`:
- Extra magic wrt step order. Users should control the step order. Especially when we introduce templates.
- Eliminates the possibility to lazily download the action tarball, since `pre` always run by default, we have to download the tarball to check whether action defined a `pre`
- `pre` doesn't work with local action, we suggested customer use local action for testing their action changes, ex CI for their action, to avoid delay between `git push` and GitHub repo tarball download api.
- Condition on the `pre` can't be controlled using dynamic step outputs. `pre` executes too early.

View File

@@ -0,0 +1,56 @@
# ADR 0397: Support adding custom labels during runner config
**Date**: 2020-03-30
**Status**: Approved
## Context
Since configuring self-hosted runners is commonly automated via scripts, the labels need to be able to be created during configuration. The runner currently registers the built-in labels (os, arch) during registration but does not accept labels via command line args to extend the set registered.
See Issue: https://github.com/actions/runner/issues/262
This is another version of [ADR275](https://github.com/actions/runner/pull/275)
## Decision
This ADR proposes that we add a `--labels` option to `config`, which could be used to add custom additional labels to the configured runner.
For example, to add a single extra label the operator could run:
```bash
./config.sh --labels mylabel
```
> Note: the current runner command line parsing and envvar override algorithm only supports a single argument (key).
This would add the label `mylabel` to the runner, and enable users to select the runner in their workflow using this label:
```yaml
runs-on: [self-hosted, mylabel]
```
To add multiple labels the operator could run:
```bash
./config.sh --labels mylabel,anotherlabel
```
> Note: the current runner command line parsing and envvar override algorithm only supports a single argument (key).
This would add the label `mylabel` and `anotherlabel` to the runner, and enable users to select the runner in their workflow using this label:
```yaml
runs-on: [self-hosted, mylabel, anotherlabel]
```
It would not be possible to remove labels from an existing runner using `config.sh`, instead labels would have to be removed using the GitHub UI.
The labels argument will split on commas, trim and discard empty strings. That effectively means don't use commans in unattended config label names. Alternatively we could choose to escape commans but it's a nice to have.
## Replace
If an existing runner exists and the option to replace is chosen (interactively of via unattend as in this scenario), then the labels will be replaced / overwritten (not merged).
## Overriding built-in labels
Note that it is possible to register "built-in" hosted labels like `ubuntu-latest` and is not considered an error. This is an effective way for the org / runner admin to dictate by policy through registration that this set of runners will be used without having to edit all the workflow files now and in the future.
We will also not make other restrictions such as limiting explicitly adding os / arch labels and validating. We will assume that explicit labels were added for a reason and not restricting offers the most flexibility and future proofing / compat.
## Consequences
The ability to add custom labels to a self-hosted runner would enable most scenarios where job runner selection based on runner capabilities or characteristics are required.

View File

@@ -1,24 +1,11 @@
## Features ## Features
- Update Runner Register GitHub API URL to Support Org-level Runner (#339 #345 #352) - Runner support for GHES Alpha (#381 #386 #390 #393 $401)
- Preserve workflow file/line/column for better error messages (#356) - Allow secrets context in Container.env (#388)
- Switch to use token service instead of SPS for exchanging oauth token. (#325)
- Load and print machine setup info from .setup_info (#364)
- Expose job name as $GITHUB_JOB (#366)
- Add support for job outputs. (#365)
- Set CI=true when launch process in actions runner. (#374)
- Set steps.<id>.outcome and steps.<id>.conclusion. (#372)
- Add support for workflow/job defaults. (#369)
- Expose GITHUB_REPOSITORY_OWNER and ${{github.repository_owner}}. (#378)
## Bugs ## Bugs
- Use authenticate endpoint for testing runner connection. (#311) - Raise warning when volume mount root. (#413)
- Commands translate file path from container action (#331) - Fix typo (#394)
- Change problem matchers output to debug (#363)
- Switch hashFiles to extension function (#362)
- Add expanded volumes strings to container mounts (#384)
## Misc ## Misc
- Add runner auth documentation (#357) - N/A
## Windows x64 ## Windows x64
We recommend configuring the runner in a root folder of the Windows drive (e.g. "C:\actions-runner"). This will help avoid issues related to service identity folder permissions and long file path restrictions on Windows We recommend configuring the runner in a root folder of the Windows drive (e.g. "C:\actions-runner"). This will help avoid issues related to service identity folder permissions and long file path restrictions on Windows

View File

@@ -172,7 +172,7 @@ get_current_os_name() {
return 0 return 0
elif [ "$uname" = "FreeBSD" ]; then elif [ "$uname" = "FreeBSD" ]; then
echo "freebsd" echo "freebsd"
return 0 return 0
elif [ "$uname" = "Linux" ]; then elif [ "$uname" = "Linux" ]; then
local linux_platform_name local linux_platform_name
linux_platform_name="$(get_linux_platform_name)" || { echo "linux" && return 0 ; } linux_platform_name="$(get_linux_platform_name)" || { echo "linux" && return 0 ; }
@@ -728,11 +728,12 @@ downloadcurl() {
# Append feed_credential as late as possible before calling curl to avoid logging feed_credential # Append feed_credential as late as possible before calling curl to avoid logging feed_credential
remote_path="${remote_path}${feed_credential}" remote_path="${remote_path}${feed_credential}"
local curl_options="--retry 20 --retry-delay 2 --connect-timeout 15 -sSL -f --create-dirs "
local failed=false local failed=false
if [ -z "$out_path" ]; then if [ -z "$out_path" ]; then
curl --retry 10 -sSL -f --create-dirs "$remote_path" || failed=true curl $curl_options "$remote_path" || failed=true
else else
curl --retry 10 -sSL -f --create-dirs -o "$out_path" "$remote_path" || failed=true curl $curl_options -o "$out_path" "$remote_path" || failed=true
fi fi
if [ "$failed" = true ]; then if [ "$failed" = true ]; then
say_verbose "Curl download failed" say_verbose "Curl download failed"
@@ -748,12 +749,12 @@ downloadwget() {
# Append feed_credential as late as possible before calling wget to avoid logging feed_credential # Append feed_credential as late as possible before calling wget to avoid logging feed_credential
remote_path="${remote_path}${feed_credential}" remote_path="${remote_path}${feed_credential}"
local wget_options="--tries 20 --waitretry 2 --connect-timeout 15 "
local failed=false local failed=false
if [ -z "$out_path" ]; then if [ -z "$out_path" ]; then
wget -q --tries 10 -O - "$remote_path" || failed=true wget -q $wget_options -O - "$remote_path" || failed=true
else else
wget --tries 10 -O "$out_path" "$remote_path" || failed=true wget $wget_options -O "$out_path" "$remote_path" || failed=true
fi fi
if [ "$failed" = true ]; then if [ "$failed" = true ]; then
say_verbose "Wget download failed" say_verbose "Wget download failed"

File diff suppressed because it is too large Load Diff

View File

@@ -27,7 +27,7 @@
"@types/node": "^12.7.12", "@types/node": "^12.7.12",
"@typescript-eslint/parser": "^2.8.0", "@typescript-eslint/parser": "^2.8.0",
"@zeit/ncc": "^0.20.5", "@zeit/ncc": "^0.20.5",
"eslint": "^5.16.0", "eslint": "^6.8.0",
"eslint-plugin-github": "^2.0.0", "eslint-plugin-github": "^2.0.0",
"prettier": "^1.19.1", "prettier": "^1.19.1",
"typescript": "^3.6.4" "typescript": "^3.6.4"

View File

@@ -15,6 +15,9 @@ namespace GitHub.Runner.Common
[DataContract] [DataContract]
public sealed class RunnerSettings public sealed class RunnerSettings
{ {
[DataMember(Name = "IsHostedServer", EmitDefaultValue = false)]
private bool? _isHostedServer;
[DataMember(EmitDefaultValue = false)] [DataMember(EmitDefaultValue = false)]
public int AgentId { get; set; } public int AgentId { get; set; }
@@ -42,6 +45,21 @@ namespace GitHub.Runner.Common
[DataMember(EmitDefaultValue = false)] [DataMember(EmitDefaultValue = false)]
public string MonitorSocketAddress { get; set; } public string MonitorSocketAddress { get; set; }
[IgnoreDataMember]
public bool IsHostedServer
{
get
{
// Old runners do not have this property. Hosted runners likely don't have this property either.
return _isHostedServer ?? true;
}
set
{
_isHostedServer = value;
}
}
/// <summary> /// <summary>
// Computed property for convenience. Can either return: // Computed property for convenience. Can either return:
// 1. If runner was configured at the repo level, returns something like: "myorg/myrepo" // 1. If runner was configured at the repo level, returns something like: "myorg/myrepo"
@@ -69,6 +87,15 @@ namespace GitHub.Runner.Common
return repoOrOrgName; return repoOrOrgName;
} }
} }
[OnSerializing]
private void OnSerializing(StreamingContext context)
{
if (_isHostedServer.HasValue && _isHostedServer.Value)
{
_isHostedServer = null;
}
}
} }
[ServiceLocator(Default = typeof(ConfigurationStore))] [ServiceLocator(Default = typeof(ConfigurationStore))]

View File

@@ -86,7 +86,6 @@ namespace GitHub.Runner.Listener.Configuration
RunnerSettings runnerSettings = new RunnerSettings(); RunnerSettings runnerSettings = new RunnerSettings();
bool isHostedServer = false;
// Loop getting url and creds until you can connect // Loop getting url and creds until you can connect
ICredentialProvider credProvider = null; ICredentialProvider credProvider = null;
VssCredentials creds = null; VssCredentials creds = null;
@@ -95,8 +94,7 @@ namespace GitHub.Runner.Listener.Configuration
{ {
// Get the URL // Get the URL
var inputUrl = command.GetUrl(); var inputUrl = command.GetUrl();
if (!inputUrl.Contains("github.com", StringComparison.OrdinalIgnoreCase) && if (inputUrl.Contains("codedev.ms", StringComparison.OrdinalIgnoreCase))
!inputUrl.Contains("github.localhost", StringComparison.OrdinalIgnoreCase))
{ {
runnerSettings.ServerUrl = inputUrl; runnerSettings.ServerUrl = inputUrl;
// Get the credentials // Get the credentials
@@ -117,7 +115,7 @@ namespace GitHub.Runner.Listener.Configuration
try try
{ {
// Determine the service deployment type based on connection data. (Hosted/OnPremises) // Determine the service deployment type based on connection data. (Hosted/OnPremises)
isHostedServer = await IsHostedServer(runnerSettings.ServerUrl, creds); runnerSettings.IsHostedServer = runnerSettings.GitHubUrl == null || IsHostedServer(new UriBuilder(runnerSettings.GitHubUrl));
// Validate can connect. // Validate can connect.
await _runnerServer.ConnectAsync(new Uri(runnerSettings.ServerUrl), creds); await _runnerServer.ConnectAsync(new Uri(runnerSettings.ServerUrl), creds);
@@ -199,7 +197,7 @@ namespace GitHub.Runner.Listener.Configuration
} }
else else
{ {
// Create a new agent. // Create a new agent.
agent = CreateNewAgent(runnerSettings.AgentName, publicKey); agent = CreateNewAgent(runnerSettings.AgentName, publicKey);
try try
@@ -248,14 +246,6 @@ namespace GitHub.Runner.Listener.Configuration
{ {
UriBuilder configServerUrl = new UriBuilder(runnerSettings.ServerUrl); UriBuilder configServerUrl = new UriBuilder(runnerSettings.ServerUrl);
UriBuilder oauthEndpointUrlBuilder = new UriBuilder(agent.Authorization.AuthorizationUrl); UriBuilder oauthEndpointUrlBuilder = new UriBuilder(agent.Authorization.AuthorizationUrl);
if (!isHostedServer && Uri.Compare(configServerUrl.Uri, oauthEndpointUrlBuilder.Uri, UriComponents.SchemeAndServer, UriFormat.Unescaped, StringComparison.OrdinalIgnoreCase) != 0)
{
oauthEndpointUrlBuilder.Scheme = configServerUrl.Scheme;
oauthEndpointUrlBuilder.Host = configServerUrl.Host;
oauthEndpointUrlBuilder.Port = configServerUrl.Port;
Trace.Info($"Set oauth endpoint url's scheme://host:port component to match runner configure url's scheme://host:port: '{oauthEndpointUrlBuilder.Uri.AbsoluteUri}'.");
}
var credentialData = new CredentialData var credentialData = new CredentialData
{ {
Scheme = Constants.Configuration.OAuth, Scheme = Constants.Configuration.OAuth,
@@ -291,7 +281,7 @@ namespace GitHub.Runner.Listener.Configuration
{ {
// there are two exception messages server send that indicate clock skew. // there are two exception messages server send that indicate clock skew.
// 1. The bearer token expired on {jwt.ValidTo}. Current server time is {DateTime.UtcNow}. // 1. The bearer token expired on {jwt.ValidTo}. Current server time is {DateTime.UtcNow}.
// 2. The bearer token is not valid until {jwt.ValidFrom}. Current server time is {DateTime.UtcNow}. // 2. The bearer token is not valid until {jwt.ValidFrom}. Current server time is {DateTime.UtcNow}.
Trace.Error("Catch exception during test agent connection."); Trace.Error("Catch exception during test agent connection.");
Trace.Error(ex); Trace.Error(ex);
throw new Exception("The local machine's clock may be out of sync with the server time by more than five minutes. Please sync your clock with your domain or internet time and try again."); throw new Exception("The local machine's clock may be out of sync with the server time by more than five minutes. Please sync your clock with your domain or internet time and try again.");
@@ -381,7 +371,6 @@ namespace GitHub.Runner.Listener.Configuration
} }
// Determine the service deployment type based on connection data. (Hosted/OnPremises) // Determine the service deployment type based on connection data. (Hosted/OnPremises)
bool isHostedServer = await IsHostedServer(settings.ServerUrl, creds);
await _runnerServer.ConnectAsync(new Uri(settings.ServerUrl), creds); await _runnerServer.ConnectAsync(new Uri(settings.ServerUrl), creds);
var agents = await _runnerServer.GetAgentsAsync(settings.PoolId, settings.AgentName); var agents = await _runnerServer.GetAgentsAsync(settings.PoolId, settings.AgentName);
@@ -404,7 +393,7 @@ namespace GitHub.Runner.Listener.Configuration
_term.WriteLine("Cannot connect to server, because config files are missing. Skipping removing runner from the server."); _term.WriteLine("Cannot connect to server, because config files are missing. Skipping removing runner from the server.");
} }
//delete credential config files //delete credential config files
currentAction = "Removing .credentials"; currentAction = "Removing .credentials";
if (hasCredentials) if (hasCredentials)
{ {
@@ -418,7 +407,7 @@ namespace GitHub.Runner.Listener.Configuration
_term.WriteLine("Does not exist. Skipping " + currentAction); _term.WriteLine("Does not exist. Skipping " + currentAction);
} }
//delete settings config file //delete settings config file
currentAction = "Removing .runner"; currentAction = "Removing .runner";
if (isConfigured) if (isConfigured)
{ {
@@ -498,31 +487,26 @@ namespace GitHub.Runner.Listener.Configuration
return agent; return agent;
} }
private async Task<bool> IsHostedServer(string serverUrl, VssCredentials credentials) private bool IsHostedServer(UriBuilder gitHubUrl)
{ {
// Determine the service deployment type based on connection data. (Hosted/OnPremises) return string.Equals(gitHubUrl.Host, "github.com", StringComparison.OrdinalIgnoreCase) ||
var locationServer = HostContext.GetService<ILocationServer>(); string.Equals(gitHubUrl.Host, "www.github.com", StringComparison.OrdinalIgnoreCase) ||
VssConnection connection = VssUtil.CreateConnection(new Uri(serverUrl), credentials); string.Equals(gitHubUrl.Host, "github.localhost", StringComparison.OrdinalIgnoreCase);
await locationServer.ConnectAsync(connection);
try
{
var connectionData = await locationServer.GetConnectionDataAsync();
Trace.Info($"Server deployment type: {connectionData.DeploymentType}");
return connectionData.DeploymentType.HasFlag(DeploymentFlags.Hosted);
}
catch (Exception ex)
{
// Since the DeploymentType is Enum, deserialization exception means there is a new Enum member been added.
// It's more likely to be Hosted since OnPremises is always behind and customer can update their agent if are on-prem
Trace.Error(ex);
return true;
}
} }
private async Task<GitHubAuthResult> GetTenantCredential(string githubUrl, string githubToken, string runnerEvent) private async Task<GitHubAuthResult> GetTenantCredential(string githubUrl, string githubToken, string runnerEvent)
{ {
var githubApiUrl = "";
var gitHubUrlBuilder = new UriBuilder(githubUrl); var gitHubUrlBuilder = new UriBuilder(githubUrl);
var githubApiUrl = $"{gitHubUrlBuilder.Scheme}://api.{gitHubUrlBuilder.Host}/actions/runner-registration"; if (IsHostedServer(gitHubUrlBuilder))
{
githubApiUrl = $"{gitHubUrlBuilder.Scheme}://api.{gitHubUrlBuilder.Host}/actions/runner-registration";
}
else
{
githubApiUrl = $"{gitHubUrlBuilder.Scheme}://{gitHubUrlBuilder.Host}/api/v3/actions/runner-registration";
}
using (var httpClientHandler = HostContext.CreateHttpClientHandler()) using (var httpClientHandler = HostContext.CreateHttpClientHandler())
using (var httpClient = new HttpClient(httpClientHandler)) using (var httpClient = new HttpClient(httpClientHandler))
{ {

View File

@@ -51,15 +51,21 @@ namespace GitHub.Runner.Worker
List<JobExtensionRunner> containerSetupSteps = new List<JobExtensionRunner>(); List<JobExtensionRunner> containerSetupSteps = new List<JobExtensionRunner>();
IEnumerable<Pipelines.ActionStep> actions = steps.OfType<Pipelines.ActionStep>(); IEnumerable<Pipelines.ActionStep> actions = steps.OfType<Pipelines.ActionStep>();
// TODO: Depreciate the PREVIEW_ACTION_TOKEN // TODO: Deprecate the PREVIEW_ACTION_TOKEN
// Log even if we aren't using it to ensure users know. // Log even if we aren't using it to ensure users know.
if (!string.IsNullOrEmpty(executionContext.Variables.Get("PREVIEW_ACTION_TOKEN"))) if (!string.IsNullOrEmpty(executionContext.Variables.Get("PREVIEW_ACTION_TOKEN")))
{ {
executionContext.Warning("The 'PREVIEW_ACTION_TOKEN' secret is depreciated. Please remove it from the repository's secrets"); executionContext.Warning("The 'PREVIEW_ACTION_TOKEN' secret is deprecated. Please remove it from the repository's secrets");
} }
// Clear the cache (local runner) // Clear the cache (for self-hosted runners)
IOUtil.DeleteDirectory(HostContext.GetDirectory(WellKnownDirectory.Actions), executionContext.CancellationToken); // Note, temporarily avoid this step for the on-premises product, to avoid rate limiting.
var configurationStore = HostContext.GetService<IConfigurationStore>();
var isHostedServer = configurationStore.GetSettings().IsHostedServer;
if (isHostedServer)
{
IOUtil.DeleteDirectory(HostContext.GetDirectory(WellKnownDirectory.Actions), executionContext.CancellationToken);
}
foreach (var action in actions) foreach (var action in actions)
{ {
@@ -448,7 +454,8 @@ namespace GitHub.Runner.Worker
ArgUtil.NotNullOrEmpty(repositoryReference.Ref, nameof(repositoryReference.Ref)); ArgUtil.NotNullOrEmpty(repositoryReference.Ref, nameof(repositoryReference.Ref));
string destDirectory = Path.Combine(HostContext.GetDirectory(WellKnownDirectory.Actions), repositoryReference.Name.Replace(Path.AltDirectorySeparatorChar, Path.DirectorySeparatorChar), repositoryReference.Ref); string destDirectory = Path.Combine(HostContext.GetDirectory(WellKnownDirectory.Actions), repositoryReference.Name.Replace(Path.AltDirectorySeparatorChar, Path.DirectorySeparatorChar), repositoryReference.Ref);
if (File.Exists(destDirectory + ".completed")) string watermarkFile = destDirectory + ".completed";
if (File.Exists(watermarkFile))
{ {
executionContext.Debug($"Action '{repositoryReference.Name}@{repositoryReference.Ref}' already downloaded at '{destDirectory}'."); executionContext.Debug($"Action '{repositoryReference.Name}@{repositoryReference.Ref}' already downloaded at '{destDirectory}'.");
return; return;
@@ -498,24 +505,33 @@ namespace GitHub.Runner.Worker
using (var httpClientHandler = HostContext.CreateHttpClientHandler()) using (var httpClientHandler = HostContext.CreateHttpClientHandler())
using (var httpClient = new HttpClient(httpClientHandler)) using (var httpClient = new HttpClient(httpClientHandler))
{ {
var authToken = Environment.GetEnvironmentVariable("_GITHUB_ACTION_TOKEN"); var configurationStore = HostContext.GetService<IConfigurationStore>();
if (string.IsNullOrEmpty(authToken)) var isHostedServer = configurationStore.GetSettings().IsHostedServer;
if (isHostedServer)
{ {
// TODO: Depreciate the PREVIEW_ACTION_TOKEN var authToken = Environment.GetEnvironmentVariable("_GITHUB_ACTION_TOKEN");
authToken = executionContext.Variables.Get("PREVIEW_ACTION_TOKEN"); if (string.IsNullOrEmpty(authToken))
} {
// TODO: Deprecate the PREVIEW_ACTION_TOKEN
authToken = executionContext.Variables.Get("PREVIEW_ACTION_TOKEN");
}
if (!string.IsNullOrEmpty(authToken)) if (!string.IsNullOrEmpty(authToken))
{ {
HostContext.SecretMasker.AddValue(authToken); HostContext.SecretMasker.AddValue(authToken);
var base64EncodingToken = Convert.ToBase64String(Encoding.UTF8.GetBytes($"PAT:{authToken}")); var base64EncodingToken = Convert.ToBase64String(Encoding.UTF8.GetBytes($"PAT:{authToken}"));
httpClient.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Basic", base64EncodingToken); httpClient.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Basic", base64EncodingToken);
}
else
{
var accessToken = executionContext.GetGitHubContext("token");
var base64EncodingToken = Convert.ToBase64String(Encoding.UTF8.GetBytes($"x-access-token:{accessToken}"));
httpClient.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Basic", base64EncodingToken);
}
} }
else else
{ {
var accessToken = executionContext.GetGitHubContext("token"); // Intentionally empty. Temporary for GHES alpha release, download from dotcom unauthenticated.
var base64EncodingToken = Convert.ToBase64String(Encoding.UTF8.GetBytes($"x-access-token:{accessToken}"));
httpClient.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Basic", base64EncodingToken);
} }
httpClient.DefaultRequestHeaders.UserAgent.Add(HostContext.UserAgent); httpClient.DefaultRequestHeaders.UserAgent.Add(HostContext.UserAgent);
@@ -610,7 +626,7 @@ namespace GitHub.Runner.Worker
} }
Trace.Verbose("Create watermark file indicate action download succeed."); Trace.Verbose("Create watermark file indicate action download succeed.");
File.WriteAllText(destDirectory + ".completed", DateTime.UtcNow.ToString()); File.WriteAllText(watermarkFile, DateTime.UtcNow.ToString());
executionContext.Debug($"Archive '{archiveFile}' has been unzipped into '{destDirectory}'."); executionContext.Debug($"Archive '{archiveFile}' has been unzipped into '{destDirectory}'.");
Trace.Info("Finished getting action repository."); Trace.Info("Finished getting action repository.");

View File

@@ -47,7 +47,7 @@ namespace GitHub.Runner.Worker
condition: $"{PipelineTemplateConstants.Always}()", condition: $"{PipelineTemplateConstants.Always}()",
displayName: "Stop containers", displayName: "Stop containers",
data: data); data: data);
executionContext.Debug($"Register post job cleanup for stopping/deleting containers."); executionContext.Debug($"Register post job cleanup for stopping/deleting containers.");
executionContext.RegisterPostJobStep(nameof(StopContainersAsync), postJobStep); executionContext.RegisterPostJobStep(nameof(StopContainersAsync), postJobStep);
@@ -180,6 +180,11 @@ namespace GitHub.Runner.Worker
foreach (var volume in container.UserMountVolumes) foreach (var volume in container.UserMountVolumes)
{ {
Trace.Info($"User provided volume: {volume.Value}"); Trace.Info($"User provided volume: {volume.Value}");
var mount = new MountVolume(volume.Value);
if (string.Equals(mount.SourceVolumePath, "/", StringComparison.OrdinalIgnoreCase))
{
executionContext.Warning($"Volume mount {volume.Value} is going to mount '/' into the container which may cause file ownership change in the entire file system and cause Actions Runner to lose permission to access the disk.");
}
} }
// Pull down docker image with retry up to 3 times // Pull down docker image with retry up to 3 times

View File

@@ -10,6 +10,7 @@ namespace GitHub.Runner.Worker
{ {
"action", "action",
"actor", "actor",
"api_url", // temp for GHES alpha release
"base_ref", "base_ref",
"event_name", "event_name",
"event_path", "event_path",
@@ -21,6 +22,7 @@ namespace GitHub.Runner.Worker
"run_id", "run_id",
"run_number", "run_number",
"sha", "sha",
"url", // temp for GHES alpha release
"workflow", "workflow",
"workspace", "workspace",
}; };

View File

@@ -1,6 +1,7 @@
using System; using System;
using System.Collections.Generic; using System.Collections.Generic;
using System.Diagnostics; using System.Diagnostics;
using System.Globalization;
using System.IO; using System.IO;
using System.Linq; using System.Linq;
using System.Runtime.Serialization; using System.Runtime.Serialization;
@@ -127,6 +128,17 @@ namespace GitHub.Runner.Worker
context.SetRunnerContext("workspace", Path.Combine(_workDirectory, trackingConfig.PipelineDirectory)); context.SetRunnerContext("workspace", Path.Combine(_workDirectory, trackingConfig.PipelineDirectory));
context.SetGitHubContext("workspace", Path.Combine(_workDirectory, trackingConfig.WorkspaceDirectory)); context.SetGitHubContext("workspace", Path.Combine(_workDirectory, trackingConfig.WorkspaceDirectory));
// Temporary hack for GHES alpha
var configurationStore = HostContext.GetService<IConfigurationStore>();
var runnerSettings = configurationStore.GetSettings();
if (!runnerSettings.IsHostedServer && !string.IsNullOrEmpty(runnerSettings.GitHubUrl))
{
var url = new Uri(runnerSettings.GitHubUrl);
var portInfo = url.IsDefaultPort ? string.Empty : $":{url.Port.ToString(CultureInfo.InvariantCulture)}";
context.SetGitHubContext("url", $"{url.Scheme}://{url.Host}{portInfo}");
context.SetGitHubContext("api_url", $"{url.Scheme}://{url.Host}{portInfo}/api/v3");
}
// Evaluate the job-level environment variables // Evaluate the job-level environment variables
context.Debug("Evaluating job-level environment variables"); context.Debug("Evaluating job-level environment variables");
var templateEvaluator = context.ToPipelineTemplateEvaluator(); var templateEvaluator = context.ToPipelineTemplateEvaluator();

View File

@@ -8,6 +8,7 @@ namespace GitHub.DistributedTask.Pipelines.ObjectTemplating
{ {
public const String Always = "always"; public const String Always = "always";
public const String BooleanStepsContext = "boolean-steps-context"; public const String BooleanStepsContext = "boolean-steps-context";
public const String BooleanStrategyContext = "boolean-strategy-context";
public const String CancelTimeoutMinutes = "cancel-timeout-minutes"; public const String CancelTimeoutMinutes = "cancel-timeout-minutes";
public const String Cancelled = "cancelled"; public const String Cancelled = "cancelled";
public const String Checkout = "checkout"; public const String Checkout = "checkout";

View File

@@ -1,4 +1,4 @@
using System; using System;
using System.Globalization; using System.Globalization;
using System.IO; using System.IO;
using System.Linq; using System.Linq;
@@ -12,7 +12,7 @@ namespace GitHub.DistributedTask.Pipelines.ObjectTemplating
/// <summary> /// <summary>
/// Converts a YAML file into a TemplateToken /// Converts a YAML file into a TemplateToken
/// </summary> /// </summary>
public sealed class YamlObjectReader : IObjectReader internal sealed class YamlObjectReader : IObjectReader
{ {
internal YamlObjectReader( internal YamlObjectReader(
Int32? fileId, Int32? fileId,

View File

@@ -739,7 +739,7 @@
"container-env": { "container-env": {
"mapping": { "mapping": {
"loose-key-type": "non-empty-string", "loose-key-type": "non-empty-string",
"loose-value-type": "string" "loose-value-type": "string-runner-context"
} }
}, },

View File

@@ -37,7 +37,7 @@ namespace GitHub.Runner.Common.Tests.Listener.Configuration
private Mock<IRSAKeyManager> _rsaKeyManager; private Mock<IRSAKeyManager> _rsaKeyManager;
private string _expectedToken = "expectedToken"; private string _expectedToken = "expectedToken";
private string _expectedServerUrl = "https://localhost"; private string _expectedServerUrl = "https://codedev.ms";
private string _expectedAgentName = "expectedAgentName"; private string _expectedAgentName = "expectedAgentName";
private string _expectedPoolName = "poolName"; private string _expectedPoolName = "poolName";
private string _expectedAuthType = "pat"; private string _expectedAuthType = "pat";

View File

@@ -111,6 +111,57 @@ namespace GitHub.Runner.Common.Tests.Worker
} }
} }
[Fact]
[Trait("Level", "L0")]
[Trait("Category", "Worker")]
public async void PrepareActions_SkipDownloadActionFromGraphWhenCached_OnPremises()
{
try
{
// Arrange
Setup();
var actionId = Guid.NewGuid();
var actions = new List<Pipelines.ActionStep>
{
new Pipelines.ActionStep()
{
Name = "action",
Id = actionId,
Reference = new Pipelines.RepositoryPathReference()
{
Name = "actions/no-such-action",
Ref = "master",
RepositoryType = "GitHub"
}
}
};
_configurationStore.Object.GetSettings().IsHostedServer = false;
var actionDirectory = Path.Combine(_hc.GetDirectory(WellKnownDirectory.Actions), "actions/no-such-action", "master");
Directory.CreateDirectory(actionDirectory);
var watermarkFile = $"{actionDirectory}.completed";
File.WriteAllText(watermarkFile, DateTime.UtcNow.ToString());
var actionFile = Path.Combine(actionDirectory, "action.yml");
File.WriteAllText(actionFile, @"
name: ""no-such-action""
runs:
using: node12
main: no-such-action.js
");
var testFile = Path.Combine(actionDirectory, "test-file");
File.WriteAllText(testFile, "asdf");
// Act
await _actionManager.PrepareActionsAsync(_ec.Object, actions);
// Assert
Assert.True(File.Exists(testFile));
}
finally
{
Teardown();
}
}
[Fact] [Fact]
[Trait("Level", "L0")] [Trait("Level", "L0")]
[Trait("Category", "Worker")] [Trait("Category", "Worker")]

View File

@@ -1 +1 @@
2.168.0 2.169.0