mirror of
https://github.com/actions/runner.git
synced 2025-12-10 20:36:49 +00:00
Compare commits
5 Commits
users/tihu
...
users/juli
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
b1bb7b0279 | ||
|
|
62e700f052 | ||
|
|
d81e5099a0 | ||
|
|
28312e5637 | ||
|
|
0cba42590f |
61
docs/design/auth.md
Normal file
61
docs/design/auth.md
Normal file
@@ -0,0 +1,61 @@
|
||||
# Runner Authentication and Authorization
|
||||
|
||||
## Goals
|
||||
- Support runner installs in untrusted domains.
|
||||
- The account that configures or runs the runner process is not relevant for accessing GitHub resources.
|
||||
- Accessing GitHub resources is done with a per-job token which expires when job completes.
|
||||
- The token is granted to trusted parts of the system including the runner, actions and script steps specified by the workflow author as trusted.
|
||||
- All OAuth tokens that come from the Token Service that the runner uses to access Actions Service resources are the same. It's just the scope and expiration of the token that may vary.
|
||||
|
||||
## Configuration
|
||||
|
||||
Configuring a self-hosted runner is [covered here in the documentation](https://help.github.com/en/actions/hosting-your-own-runners/adding-self-hosted-runners).
|
||||
|
||||
Configuration is done with the user being authenticated via a time-limited, GitHub runner registration token.
|
||||
|
||||
*Your credentials are never used for registering the runner with the service.*
|
||||
|
||||

|
||||
|
||||
During configuration, an RSA public/private key pair is created, the private key is stored in file on disk. On Windows, the content is protected with DPAPI (machine level encrypted - runner only valid on that machine) and on Linux/OSX with `chmod` permissions.
|
||||
|
||||
Using your credentials, the runner is registered with the service by sending the public key to the service which adds that runner to the pool and stores the public key, the Token Service will generate a `clientId` associated with the public key.
|
||||
|
||||
## Start and Listen
|
||||
|
||||
After configuring the runner, the runner can be started interactively (`./run.cmd` or `./run.sh`) or as a service.
|
||||
|
||||

|
||||
|
||||
On start, the runner listener process loads the RSA private key (on Windows decrypting with machine key DPAPI), and asks the Token Service for an OAuth token which is signed with the RSA private key.
|
||||
The server then responds with an OAuth token that grants permission to access the message queue (HTTP long poll), allowing the runner to acquire the messages it will eventually run.
|
||||
|
||||
## Run a workflow
|
||||
|
||||
When a workflow is run, its labels are evaluated, it is matched to a runner and a message is placed in a queue of messages for that runner.
|
||||
The runner then starts listening for jobs via the message queue HTTP long poll.
|
||||
The message is encrypted with the runner's public key, stored during runner configuration.
|
||||
|
||||

|
||||
|
||||
A workflow is queued as a result of a triggered [event](https://help.github.com/en/actions/reference/events-that-trigger-workflows). Workflows can be scheduled to [run at specific UTC times](https://help.github.com/en/actions/reference/events-that-trigger-workflows#scheduled-events-schedule) using POSIX `cron` syntax.
|
||||
An [OAuth token](http://self-issued.info/docs/draft-ietf-oauth-json-web-token.html) is generated, granting limited access to the host in Actions Service associated with the github.com repository/organization.
|
||||
The lifetime of the OAuth token is the lifetime of the run or at most the [job timeout (default: 6 hours)](https://help.github.com/en/actions/reference/workflow-syntax-for-github-actions#jobsjob_idtimeout-minutes), plus 10 additional minutes.
|
||||
|
||||
## Accessing GitHub resources
|
||||
|
||||
The job message sent to the runner contains the OAuth token to talk back to the Actions Service.
|
||||
The runner listener parent process will spawn a runner worker process for that job and send it the job message over IPC.
|
||||
The token is never persisted.
|
||||
|
||||
Each action is run as a unique subprocess.
|
||||
The encrypted access token will be provided as an environment variable in each action subprocess.
|
||||
The token is registered with the runner as a secret and scrubbed from the logs as they are written.
|
||||
|
||||
Authentication in a workflow run to github.com can be accomplished by using the [`GITHUB_TOKEN`](https://help.github.com/en/actions/configuring-and-managing-workflows/authenticating-with-the-github_token#about-the-github_token-secret)) secret. This token expires after 60 minutes. Please note that this token is different from the OAuth token that the runner uses to talk to the Actions Service.
|
||||
|
||||
## Hosted runner authentication
|
||||
|
||||
Hosted runner authentication differs from self-hosted authentication in that runners do not undergo a registration process, but instead, the hosted runners get the OAuth token directly by reading the `.credentials` file. The scope of this particular token is limited for a given workflow job execution, and the token is revoked as soon as the job is finished.
|
||||
|
||||

|
||||
BIN
docs/res/hosted-config-start.png
Normal file
BIN
docs/res/hosted-config-start.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 31 KiB |
52
docs/res/runner-auth-diags.txt
Normal file
52
docs/res/runner-auth-diags.txt
Normal file
@@ -0,0 +1,52 @@
|
||||
# Markup used to generate the runner auth diagrams: https://websequencediagrams.com
|
||||
|
||||
title Runner Configuration (self-hosted only)
|
||||
|
||||
note left of Runner: GitHub repo URL as input
|
||||
Runner->github.com: Retrieve Actions Service access using runner registration token
|
||||
github.com->Runner: Access token for Actions Service
|
||||
note left of Runner: Generate RSA key pair
|
||||
note left of Runner: Store encrypted RSA private key on disk
|
||||
Runner->Actions Service: Register runner using Actions Service access token
|
||||
note right of Runner: Runner name, RSA public key sent
|
||||
note right of Actions Service: Public key stored
|
||||
Actions Service->Token Service: Register runner as an app along with the RSA public key
|
||||
note right of Token Service: Public key stored
|
||||
Token Service->Actions Service: Client Id for the runner application
|
||||
Actions Service->Runner: Client Id and Token Endpoint URL
|
||||
note left of Runner: Store runner configuration info into .runner file
|
||||
note left of Runner: Store Token registration info into .credentials file
|
||||
|
||||
title Runner Start and Running (self-hosted only)
|
||||
|
||||
Runner.Listener->Runner.Listener: Start
|
||||
note left of Runner.Listener: Load config info from .runner
|
||||
note left of Runner.Listener: Load token registration from .credentials
|
||||
Runner.Listener->Token Service: Exchange OAuth token (happens every 50 mins)
|
||||
note right of Runner.Listener: Construct JWT token, use Client Id signed by RSA private key
|
||||
note left of Actions Service: Find corresponding RSA public key, use Client Id\nVerify JWT token's signature
|
||||
Token Service->Runner.Listener: OAuth token with limited permission and valid for 50 mins
|
||||
Runner.Listener->Actions Service: Connect to Actions Service with OAuth token
|
||||
Actions Service->Runner.Listener: Workflow job
|
||||
|
||||
title Running workflow
|
||||
|
||||
Runner.Listener->Service (Message Queue): Get message
|
||||
note right of Runner.Listener: Authenticate with exchanged OAuth token
|
||||
Event->Actions Service: Queue workflow
|
||||
Actions Service->Actions Service: Generate OAuth token per job
|
||||
Actions Service->Actions Service: Build job message with the OAuth token
|
||||
Actions Service->Actions Service: Encrypt job message with the target runner's public key
|
||||
Actions Service->Service (Message Queue): Send encrypted job message to runner
|
||||
Service (Message Queue)->Runner.Listener: Send job
|
||||
note right of Runner.Listener: Decrypt message with runner's private key
|
||||
Runner.Listener->Runner.Worker: Create worker process per job and run the job
|
||||
|
||||
title Runner Configuration, Start and Running (hosted only)
|
||||
|
||||
Machine Management Service->Runner.Listener: Construct .runner configuration file, store token in .credentials
|
||||
Runner.Listener->Runner.Listener: Start
|
||||
note left of Runner.Listener: Load config info from .runner
|
||||
note left of Runner.Listener: Load OAuth token from .credentials
|
||||
Runner.Listener->Actions Service: Connect to Actions Service with OAuth token in .credentials
|
||||
Actions Service->Runner.Listener: Workflow job
|
||||
BIN
docs/res/self-hosted-config.png
Normal file
BIN
docs/res/self-hosted-config.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 98 KiB |
BIN
docs/res/self-hosted-start.png
Normal file
BIN
docs/res/self-hosted-start.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 43 KiB |
BIN
docs/res/workflow-run.png
Normal file
BIN
docs/res/workflow-run.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 46 KiB |
@@ -46,7 +46,6 @@ namespace GitHub.Runner.Common
|
||||
Add<T>(extensions, "GitHub.Runner.Worker.SetOutputCommandExtension, Runner.Worker");
|
||||
Add<T>(extensions, "GitHub.Runner.Worker.SaveStateCommandExtension, Runner.Worker");
|
||||
Add<T>(extensions, "GitHub.Runner.Worker.AddPathCommandExtension, Runner.Worker");
|
||||
Add<T>(extensions, "GitHub.Runner.Worker.RefreshTokenCommandExtension, Runner.Worker");
|
||||
Add<T>(extensions, "GitHub.Runner.Worker.AddMaskCommandExtension, Runner.Worker");
|
||||
Add<T>(extensions, "GitHub.Runner.Worker.AddMatcherCommandExtension, Runner.Worker");
|
||||
Add<T>(extensions, "GitHub.Runner.Worker.RemoveMatcherCommandExtension, Runner.Worker");
|
||||
|
||||
@@ -22,7 +22,6 @@ namespace GitHub.Runner.Common
|
||||
Task<List<TimelineRecord>> UpdateTimelineRecordsAsync(Guid scopeIdentifier, string hubName, Guid planId, Guid timelineId, IEnumerable<TimelineRecord> records, CancellationToken cancellationToken);
|
||||
Task RaisePlanEventAsync<T>(Guid scopeIdentifier, string hubName, Guid planId, T eventData, CancellationToken cancellationToken) where T : JobEvent;
|
||||
Task<Timeline> GetTimelineAsync(Guid scopeIdentifier, string hubName, Guid planId, Guid timelineId, CancellationToken cancellationToken);
|
||||
Task<GitHubToken> RefreshGitHubTokenAsync(Guid scopeIdentifier, string hubName, Guid planId, Guid jobId, CancellationToken cancellationToken);
|
||||
}
|
||||
|
||||
public sealed class JobServer : RunnerService, IJobServer
|
||||
@@ -114,11 +113,5 @@ namespace GitHub.Runner.Common
|
||||
CheckConnection();
|
||||
return _taskClient.GetTimelineAsync(scopeIdentifier, hubName, planId, timelineId, includeRecords: true, cancellationToken: cancellationToken);
|
||||
}
|
||||
|
||||
public Task<GitHubToken> RefreshGitHubTokenAsync(Guid scopeIdentifier, string hubName, Guid planId, Guid jobId, CancellationToken cancellationToken)
|
||||
{
|
||||
CheckConnection();
|
||||
return _taskClient.RefreshTokenAsync(scopeIdentifier, hubName, planId, jobId, cancellationToken: cancellationToken);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -288,30 +288,6 @@ namespace GitHub.Runner.Worker
|
||||
}
|
||||
}
|
||||
|
||||
public sealed class RefreshTokenCommandExtension : RunnerService, IActionCommandExtension
|
||||
{
|
||||
public string Command => "refresh-token";
|
||||
public bool OmitEcho => false;
|
||||
|
||||
public Type ExtensionType => typeof(IActionCommandExtension);
|
||||
|
||||
public void ProcessCommand(IExecutionContext context, string line, ActionCommand command, ContainerInfo container)
|
||||
{
|
||||
ArgUtil.NotNullOrEmpty(command.Data, "token file");
|
||||
var runnerTemp = HostContext.GetDirectory(WellKnownDirectory.Temp);
|
||||
var tempFile = Path.Combine(runnerTemp, command.Data);
|
||||
if (!tempFile.StartsWith(runnerTemp + Path.DirectorySeparatorChar))
|
||||
{
|
||||
throw new Exception($"'{command.Data}' has to be under runner temp directory '{runnerTemp}'");
|
||||
}
|
||||
|
||||
Trace.Info("Here");
|
||||
var githubToken = context.GetGitHubToken().GetAwaiter().GetResult();
|
||||
File.WriteAllText(tempFile, githubToken);
|
||||
Trace.Info("HereAgain");
|
||||
}
|
||||
}
|
||||
|
||||
public sealed class AddMatcherCommandExtension : RunnerService, IActionCommandExtension
|
||||
{
|
||||
public string Command => "add-matcher";
|
||||
|
||||
@@ -21,24 +21,11 @@ using PipelineTemplateConstants = GitHub.DistributedTask.Pipelines.ObjectTemplat
|
||||
|
||||
namespace GitHub.Runner.Worker
|
||||
{
|
||||
public class PrepareResult
|
||||
{
|
||||
public PrepareResult(List<JobExtensionRunner> containerSetupSteps, Dictionary<Guid, IActionRunner> preStepTracker)
|
||||
{
|
||||
this.ContainerSetupSteps = containerSetupSteps;
|
||||
this.PreStepTracker = preStepTracker;
|
||||
}
|
||||
|
||||
public List<JobExtensionRunner> ContainerSetupSteps { get; set; }
|
||||
|
||||
public Dictionary<Guid, IActionRunner> PreStepTracker { get; set; }
|
||||
}
|
||||
|
||||
[ServiceLocator(Default = typeof(ActionManager))]
|
||||
public interface IActionManager : IRunnerService
|
||||
{
|
||||
Dictionary<Guid, ContainerInfo> CachedActionContainers { get; }
|
||||
Task<PrepareResult> PrepareActionsAsync(IExecutionContext executionContext, IEnumerable<Pipelines.JobStep> steps);
|
||||
Task<List<JobExtensionRunner>> PrepareActionsAsync(IExecutionContext executionContext, IEnumerable<Pipelines.JobStep> steps);
|
||||
Definition LoadAction(IExecutionContext executionContext, Pipelines.ActionStep action);
|
||||
}
|
||||
|
||||
@@ -52,7 +39,7 @@ namespace GitHub.Runner.Worker
|
||||
private readonly Dictionary<Guid, ContainerInfo> _cachedActionContainers = new Dictionary<Guid, ContainerInfo>();
|
||||
|
||||
public Dictionary<Guid, ContainerInfo> CachedActionContainers => _cachedActionContainers;
|
||||
public async Task<PrepareResult> PrepareActionsAsync(IExecutionContext executionContext, IEnumerable<Pipelines.JobStep> steps)
|
||||
public async Task<List<JobExtensionRunner>> PrepareActionsAsync(IExecutionContext executionContext, IEnumerable<Pipelines.JobStep> steps)
|
||||
{
|
||||
ArgUtil.NotNull(executionContext, nameof(executionContext));
|
||||
ArgUtil.NotNull(steps, nameof(steps));
|
||||
@@ -62,7 +49,6 @@ namespace GitHub.Runner.Worker
|
||||
Dictionary<string, List<Guid>> imagesToBuild = new Dictionary<string, List<Guid>>(StringComparer.OrdinalIgnoreCase);
|
||||
Dictionary<string, ActionContainer> imagesToBuildInfo = new Dictionary<string, ActionContainer>(StringComparer.OrdinalIgnoreCase);
|
||||
List<JobExtensionRunner> containerSetupSteps = new List<JobExtensionRunner>();
|
||||
Dictionary<Guid, IActionRunner> preStepTracker = new Dictionary<Guid, IActionRunner>();
|
||||
IEnumerable<Pipelines.ActionStep> actions = steps.OfType<Pipelines.ActionStep>();
|
||||
|
||||
// TODO: Depreciate the PREVIEW_ACTION_TOKEN
|
||||
@@ -125,21 +111,6 @@ namespace GitHub.Runner.Worker
|
||||
imagesToBuildInfo[setupInfo.ActionRepository] = setupInfo;
|
||||
}
|
||||
}
|
||||
|
||||
var repoAction = action.Reference as Pipelines.RepositoryPathReference;
|
||||
if (repoAction.RepositoryType != Pipelines.PipelineConstants.SelfAlias)
|
||||
{
|
||||
var definition = LoadAction(executionContext, action);
|
||||
if (definition.Data.Execution.HasInit)
|
||||
{
|
||||
var actionRunner = HostContext.CreateService<IActionRunner>();
|
||||
actionRunner.Action = action;
|
||||
actionRunner.Stage = ActionRunStage.Pre;
|
||||
actionRunner.Condition = definition.Data.Execution.InitCondition;
|
||||
|
||||
preStepTracker[action.Id] = actionRunner;
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -176,7 +147,7 @@ namespace GitHub.Runner.Worker
|
||||
}
|
||||
#endif
|
||||
|
||||
return new PrepareResult(containerSetupSteps, preStepTracker);
|
||||
return containerSetupSteps;
|
||||
}
|
||||
|
||||
public Definition LoadAction(IExecutionContext executionContext, Pipelines.ActionStep action)
|
||||
@@ -191,7 +162,6 @@ namespace GitHub.Runner.Worker
|
||||
Data = new ActionDefinitionData()
|
||||
};
|
||||
|
||||
// bool localAction = false;
|
||||
if (action.Reference.Type == Pipelines.ActionSourceType.ContainerRegistry)
|
||||
{
|
||||
Trace.Info("Load action that reference container from registry.");
|
||||
@@ -210,7 +180,6 @@ namespace GitHub.Runner.Worker
|
||||
var repoAction = action.Reference as Pipelines.RepositoryPathReference;
|
||||
if (string.Equals(repoAction.RepositoryType, Pipelines.PipelineConstants.SelfAlias, StringComparison.OrdinalIgnoreCase))
|
||||
{
|
||||
// localAction = true;
|
||||
actionDirectory = executionContext.GetGitHubContext("workspace");
|
||||
if (!string.IsNullOrEmpty(repoAction.Path))
|
||||
{
|
||||
@@ -270,11 +239,6 @@ namespace GitHub.Runner.Worker
|
||||
Trace.Info($"Action container env: {StringUtil.ConvertToJson(containerAction.Environment)}.");
|
||||
}
|
||||
|
||||
if (!string.IsNullOrEmpty(containerAction.Init))
|
||||
{
|
||||
Trace.Info($"Action container init entrypoint: {containerAction.Init}.");
|
||||
}
|
||||
|
||||
if (!string.IsNullOrEmpty(containerAction.EntryPoint))
|
||||
{
|
||||
Trace.Info($"Action container entrypoint: {containerAction.EntryPoint}.");
|
||||
@@ -294,7 +258,6 @@ namespace GitHub.Runner.Worker
|
||||
else if (definition.Data.Execution.ExecutionType == ActionExecutionType.NodeJS)
|
||||
{
|
||||
var nodeAction = definition.Data.Execution as NodeJSActionExecutionData;
|
||||
Trace.Info($"Action init node.js file: {nodeAction.Init ?? "N/A"}.");
|
||||
Trace.Info($"Action node.js file: {nodeAction.Script}.");
|
||||
Trace.Info($"Action cleanup node.js file: {nodeAction.Cleanup ?? "N/A"}.");
|
||||
}
|
||||
@@ -809,8 +772,6 @@ namespace GitHub.Runner.Worker
|
||||
{
|
||||
public override ActionExecutionType ExecutionType => ActionExecutionType.Container;
|
||||
|
||||
public override bool HasInit => !string.IsNullOrEmpty(Init);
|
||||
public override bool HasMain => !string.IsNullOrEmpty(EntryPoint);
|
||||
public override bool HasCleanup => !string.IsNullOrEmpty(Cleanup);
|
||||
|
||||
public string Image { get; set; }
|
||||
@@ -821,8 +782,6 @@ namespace GitHub.Runner.Worker
|
||||
|
||||
public MappingToken Environment { get; set; }
|
||||
|
||||
public string Init { get; set; }
|
||||
|
||||
public string Cleanup { get; set; }
|
||||
}
|
||||
|
||||
@@ -830,14 +789,10 @@ namespace GitHub.Runner.Worker
|
||||
{
|
||||
public override ActionExecutionType ExecutionType => ActionExecutionType.NodeJS;
|
||||
|
||||
public override bool HasInit => !string.IsNullOrEmpty(Init);
|
||||
public override bool HasMain => !string.IsNullOrEmpty(Script);
|
||||
public override bool HasCleanup => !string.IsNullOrEmpty(Cleanup);
|
||||
|
||||
public string Script { get; set; }
|
||||
|
||||
public string Init { get; set; }
|
||||
|
||||
public string Cleanup { get; set; }
|
||||
}
|
||||
|
||||
@@ -845,9 +800,6 @@ namespace GitHub.Runner.Worker
|
||||
{
|
||||
public override ActionExecutionType ExecutionType => ActionExecutionType.Plugin;
|
||||
|
||||
public override bool HasInit => false;
|
||||
public override bool HasMain => true;
|
||||
|
||||
public override bool HasCleanup => !string.IsNullOrEmpty(Cleanup);
|
||||
|
||||
public string Plugin { get; set; }
|
||||
@@ -858,20 +810,16 @@ namespace GitHub.Runner.Worker
|
||||
public sealed class ScriptActionExecutionData : ActionExecutionData
|
||||
{
|
||||
public override ActionExecutionType ExecutionType => ActionExecutionType.Script;
|
||||
public override bool HasInit => false;
|
||||
public override bool HasMain => true;
|
||||
|
||||
public override bool HasCleanup => false;
|
||||
}
|
||||
|
||||
public abstract class ActionExecutionData
|
||||
{
|
||||
private string _initCondition = $"{Constants.Expressions.Always}()";
|
||||
private string _cleanupCondition = $"{Constants.Expressions.Always}()";
|
||||
|
||||
public abstract ActionExecutionType ExecutionType { get; }
|
||||
|
||||
public abstract bool HasInit { get; }
|
||||
public abstract bool HasMain { get; }
|
||||
public abstract bool HasCleanup { get; }
|
||||
|
||||
public string CleanupCondition
|
||||
@@ -879,12 +827,6 @@ namespace GitHub.Runner.Worker
|
||||
get { return _cleanupCondition; }
|
||||
set { _cleanupCondition = value; }
|
||||
}
|
||||
|
||||
public string InitCondition
|
||||
{
|
||||
get { return _initCondition; }
|
||||
set { _initCondition = value; }
|
||||
}
|
||||
}
|
||||
|
||||
public class ContainerSetupInfo
|
||||
|
||||
@@ -32,6 +32,7 @@ namespace GitHub.Runner.Worker
|
||||
public sealed class ActionManifestManager : RunnerService, IActionManifestManager
|
||||
{
|
||||
private TemplateSchema _actionManifestSchema;
|
||||
private IReadOnlyList<String> _fileTable;
|
||||
|
||||
public override void Initialize(IHostContext hostContext)
|
||||
{
|
||||
@@ -61,6 +62,9 @@ namespace GitHub.Runner.Worker
|
||||
|
||||
// Get the file ID
|
||||
var fileId = context.GetFileId(manifestFile);
|
||||
_fileTable = context.GetFileTable();
|
||||
|
||||
// Read the file
|
||||
var fileContent = File.ReadAllText(manifestFile);
|
||||
using (var stringReader = new StringReader(fileContent))
|
||||
{
|
||||
@@ -265,6 +269,15 @@ namespace GitHub.Runner.Worker
|
||||
}
|
||||
}
|
||||
|
||||
// Add the file table
|
||||
if (_fileTable?.Count > 0)
|
||||
{
|
||||
for (var i = 0 ; i < _fileTable.Count ; i++)
|
||||
{
|
||||
result.GetFileId(_fileTable[i]);
|
||||
}
|
||||
}
|
||||
|
||||
return result;
|
||||
}
|
||||
|
||||
@@ -280,9 +293,6 @@ namespace GitHub.Runner.Worker
|
||||
var envToken = default(MappingToken);
|
||||
var mainToken = default(StringToken);
|
||||
var pluginToken = default(StringToken);
|
||||
var preToken = default(StringToken);
|
||||
var preEntrypointToken = default(StringToken);
|
||||
var preIfToken = default(StringToken);
|
||||
var postToken = default(StringToken);
|
||||
var postEntrypointToken = default(StringToken);
|
||||
var postIfToken = default(StringToken);
|
||||
@@ -321,15 +331,6 @@ namespace GitHub.Runner.Worker
|
||||
case "post-if":
|
||||
postIfToken = run.Value.AssertString("post-if");
|
||||
break;
|
||||
case "pre":
|
||||
preToken = run.Value.AssertString("pre");
|
||||
break;
|
||||
case "pre-entrypoint":
|
||||
preEntrypointToken = run.Value.AssertString("pre-entrypoint");
|
||||
break;
|
||||
case "pre-if":
|
||||
preIfToken = run.Value.AssertString("pre-if");
|
||||
break;
|
||||
default:
|
||||
Trace.Info($"Ignore run property {runsKey}.");
|
||||
break;
|
||||
@@ -352,8 +353,6 @@ namespace GitHub.Runner.Worker
|
||||
Arguments = argsToken,
|
||||
EntryPoint = entrypointToken?.Value,
|
||||
Environment = envToken,
|
||||
Init = preEntrypointToken?.Value,
|
||||
InitCondition = preIfToken?.Value ?? "always()",
|
||||
Cleanup = postEntrypointToken?.Value,
|
||||
CleanupCondition = postIfToken?.Value ?? "always()"
|
||||
};
|
||||
@@ -370,8 +369,6 @@ namespace GitHub.Runner.Worker
|
||||
return new NodeJSActionExecutionData()
|
||||
{
|
||||
Script = mainToken.Value,
|
||||
Init = preToken?.Value,
|
||||
InitCondition = preIfToken?.Value ?? "always()",
|
||||
Cleanup = postToken?.Value,
|
||||
CleanupCondition = postIfToken?.Value ?? "always()"
|
||||
};
|
||||
@@ -431,566 +428,5 @@ namespace GitHub.Runner.Worker
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Converts a YAML file into a TemplateToken
|
||||
/// </summary>
|
||||
internal sealed class YamlObjectReader : IObjectReader
|
||||
{
|
||||
internal YamlObjectReader(
|
||||
Int32? fileId,
|
||||
TextReader input)
|
||||
{
|
||||
m_fileId = fileId;
|
||||
m_parser = new Parser(input);
|
||||
}
|
||||
|
||||
public Boolean AllowLiteral(out LiteralToken value)
|
||||
{
|
||||
if (EvaluateCurrent() is Scalar scalar)
|
||||
{
|
||||
// Tag specified
|
||||
if (!string.IsNullOrEmpty(scalar.Tag))
|
||||
{
|
||||
// String tag
|
||||
if (string.Equals(scalar.Tag, c_stringTag, StringComparison.Ordinal))
|
||||
{
|
||||
value = new StringToken(m_fileId, scalar.Start.Line, scalar.Start.Column, scalar.Value);
|
||||
MoveNext();
|
||||
return true;
|
||||
}
|
||||
|
||||
// Not plain style
|
||||
if (scalar.Style != ScalarStyle.Plain)
|
||||
{
|
||||
throw new NotSupportedException($"The scalar style '{scalar.Style}' on line {scalar.Start.Line} and column {scalar.Start.Column} is not valid with the tag '{scalar.Tag}'");
|
||||
}
|
||||
|
||||
// Boolean, Float, Integer, or Null
|
||||
switch (scalar.Tag)
|
||||
{
|
||||
case c_booleanTag:
|
||||
value = ParseBoolean(scalar);
|
||||
break;
|
||||
case c_floatTag:
|
||||
value = ParseFloat(scalar);
|
||||
break;
|
||||
case c_integerTag:
|
||||
value = ParseInteger(scalar);
|
||||
break;
|
||||
case c_nullTag:
|
||||
value = ParseNull(scalar);
|
||||
break;
|
||||
default:
|
||||
throw new NotSupportedException($"Unexpected tag '{scalar.Tag}'");
|
||||
}
|
||||
|
||||
MoveNext();
|
||||
return true;
|
||||
}
|
||||
|
||||
// Plain style, determine type using YAML 1.2 "core" schema https://yaml.org/spec/1.2/spec.html#id2804923
|
||||
if (scalar.Style == ScalarStyle.Plain)
|
||||
{
|
||||
if (MatchNull(scalar, out var nullToken))
|
||||
{
|
||||
value = nullToken;
|
||||
}
|
||||
else if (MatchBoolean(scalar, out var booleanToken))
|
||||
{
|
||||
value = booleanToken;
|
||||
}
|
||||
else if (MatchInteger(scalar, out var numberToken) ||
|
||||
MatchFloat(scalar, out numberToken))
|
||||
{
|
||||
value = numberToken;
|
||||
}
|
||||
else
|
||||
{
|
||||
value = new StringToken(m_fileId, scalar.Start.Line, scalar.Start.Column, scalar.Value);
|
||||
}
|
||||
|
||||
MoveNext();
|
||||
return true;
|
||||
}
|
||||
|
||||
// Otherwise assume string
|
||||
value = new StringToken(m_fileId, scalar.Start.Line, scalar.Start.Column, scalar.Value);
|
||||
MoveNext();
|
||||
return true;
|
||||
}
|
||||
|
||||
value = default;
|
||||
return false;
|
||||
}
|
||||
|
||||
public Boolean AllowSequenceStart(out SequenceToken value)
|
||||
{
|
||||
if (EvaluateCurrent() is SequenceStart sequenceStart)
|
||||
{
|
||||
value = new SequenceToken(m_fileId, sequenceStart.Start.Line, sequenceStart.Start.Column);
|
||||
MoveNext();
|
||||
return true;
|
||||
}
|
||||
|
||||
value = default;
|
||||
return false;
|
||||
}
|
||||
|
||||
public Boolean AllowSequenceEnd()
|
||||
{
|
||||
if (EvaluateCurrent() is SequenceEnd)
|
||||
{
|
||||
MoveNext();
|
||||
return true;
|
||||
}
|
||||
|
||||
return false;
|
||||
}
|
||||
|
||||
public Boolean AllowMappingStart(out MappingToken value)
|
||||
{
|
||||
if (EvaluateCurrent() is MappingStart mappingStart)
|
||||
{
|
||||
value = new MappingToken(m_fileId, mappingStart.Start.Line, mappingStart.Start.Column);
|
||||
MoveNext();
|
||||
return true;
|
||||
}
|
||||
|
||||
value = default;
|
||||
return false;
|
||||
}
|
||||
|
||||
public Boolean AllowMappingEnd()
|
||||
{
|
||||
if (EvaluateCurrent() is MappingEnd)
|
||||
{
|
||||
MoveNext();
|
||||
return true;
|
||||
}
|
||||
|
||||
return false;
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Consumes the last parsing events, which are expected to be DocumentEnd and StreamEnd.
|
||||
/// </summary>
|
||||
public void ValidateEnd()
|
||||
{
|
||||
if (EvaluateCurrent() is DocumentEnd)
|
||||
{
|
||||
MoveNext();
|
||||
}
|
||||
else
|
||||
{
|
||||
throw new InvalidOperationException("Expected document end parse event");
|
||||
}
|
||||
|
||||
if (EvaluateCurrent() is StreamEnd)
|
||||
{
|
||||
MoveNext();
|
||||
}
|
||||
else
|
||||
{
|
||||
throw new InvalidOperationException("Expected stream end parse event");
|
||||
}
|
||||
|
||||
if (MoveNext())
|
||||
{
|
||||
throw new InvalidOperationException("Expected end of parse events");
|
||||
}
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Consumes the first parsing events, which are expected to be StreamStart and DocumentStart.
|
||||
/// </summary>
|
||||
public void ValidateStart()
|
||||
{
|
||||
if (EvaluateCurrent() != null)
|
||||
{
|
||||
throw new InvalidOperationException("Unexpected parser state");
|
||||
}
|
||||
|
||||
if (!MoveNext())
|
||||
{
|
||||
throw new InvalidOperationException("Expected a parse event");
|
||||
}
|
||||
|
||||
if (EvaluateCurrent() is StreamStart)
|
||||
{
|
||||
MoveNext();
|
||||
}
|
||||
else
|
||||
{
|
||||
throw new InvalidOperationException("Expected stream start parse event");
|
||||
}
|
||||
|
||||
if (EvaluateCurrent() is DocumentStart)
|
||||
{
|
||||
MoveNext();
|
||||
}
|
||||
else
|
||||
{
|
||||
throw new InvalidOperationException("Expected document start parse event");
|
||||
}
|
||||
}
|
||||
|
||||
private ParsingEvent EvaluateCurrent()
|
||||
{
|
||||
if (m_current == null)
|
||||
{
|
||||
m_current = m_parser.Current;
|
||||
if (m_current != null)
|
||||
{
|
||||
if (m_current is Scalar scalar)
|
||||
{
|
||||
// Verify not using achors
|
||||
if (scalar.Anchor != null)
|
||||
{
|
||||
throw new InvalidOperationException($"Anchors are not currently supported. Remove the anchor '{scalar.Anchor}'");
|
||||
}
|
||||
}
|
||||
else if (m_current is MappingStart mappingStart)
|
||||
{
|
||||
// Verify not using achors
|
||||
if (mappingStart.Anchor != null)
|
||||
{
|
||||
throw new InvalidOperationException($"Anchors are not currently supported. Remove the anchor '{mappingStart.Anchor}'");
|
||||
}
|
||||
}
|
||||
else if (m_current is SequenceStart sequenceStart)
|
||||
{
|
||||
// Verify not using achors
|
||||
if (sequenceStart.Anchor != null)
|
||||
{
|
||||
throw new InvalidOperationException($"Anchors are not currently supported. Remove the anchor '{sequenceStart.Anchor}'");
|
||||
}
|
||||
}
|
||||
else if (!(m_current is MappingEnd) &&
|
||||
!(m_current is SequenceEnd) &&
|
||||
!(m_current is DocumentStart) &&
|
||||
!(m_current is DocumentEnd) &&
|
||||
!(m_current is StreamStart) &&
|
||||
!(m_current is StreamEnd))
|
||||
{
|
||||
throw new InvalidOperationException($"Unexpected parsing event type: {m_current.GetType().Name}");
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return m_current;
|
||||
}
|
||||
|
||||
private Boolean MoveNext()
|
||||
{
|
||||
m_current = null;
|
||||
return m_parser.MoveNext();
|
||||
}
|
||||
|
||||
private BooleanToken ParseBoolean(Scalar scalar)
|
||||
{
|
||||
if (MatchBoolean(scalar, out var token))
|
||||
{
|
||||
return token;
|
||||
}
|
||||
|
||||
ThrowInvalidValue(scalar, c_booleanTag); // throws
|
||||
return default;
|
||||
}
|
||||
|
||||
private NumberToken ParseFloat(Scalar scalar)
|
||||
{
|
||||
if (MatchFloat(scalar, out var token))
|
||||
{
|
||||
return token;
|
||||
}
|
||||
|
||||
ThrowInvalidValue(scalar, c_floatTag); // throws
|
||||
return default;
|
||||
}
|
||||
|
||||
private NumberToken ParseInteger(Scalar scalar)
|
||||
{
|
||||
if (MatchInteger(scalar, out var token))
|
||||
{
|
||||
return token;
|
||||
}
|
||||
|
||||
ThrowInvalidValue(scalar, c_integerTag); // throws
|
||||
return default;
|
||||
}
|
||||
|
||||
private NullToken ParseNull(Scalar scalar)
|
||||
{
|
||||
if (MatchNull(scalar, out var token))
|
||||
{
|
||||
return token;
|
||||
}
|
||||
|
||||
ThrowInvalidValue(scalar, c_nullTag); // throws
|
||||
return default;
|
||||
}
|
||||
|
||||
private Boolean MatchBoolean(
|
||||
Scalar scalar,
|
||||
out BooleanToken value)
|
||||
{
|
||||
// YAML 1.2 "core" schema https://yaml.org/spec/1.2/spec.html#id2804923
|
||||
switch (scalar.Value ?? string.Empty)
|
||||
{
|
||||
case "true":
|
||||
case "True":
|
||||
case "TRUE":
|
||||
value = new BooleanToken(m_fileId, scalar.Start.Line, scalar.Start.Column, true);
|
||||
return true;
|
||||
case "false":
|
||||
case "False":
|
||||
case "FALSE":
|
||||
value = new BooleanToken(m_fileId, scalar.Start.Line, scalar.Start.Column, false);
|
||||
return true;
|
||||
}
|
||||
|
||||
value = default;
|
||||
return false;
|
||||
}
|
||||
|
||||
private Boolean MatchFloat(
|
||||
Scalar scalar,
|
||||
out NumberToken value)
|
||||
{
|
||||
// YAML 1.2 "core" schema https://yaml.org/spec/1.2/spec.html#id2804923
|
||||
var str = scalar.Value;
|
||||
if (!string.IsNullOrEmpty(str))
|
||||
{
|
||||
// Check for [-+]?(\.inf|\.Inf|\.INF)|\.nan|\.NaN|\.NAN
|
||||
switch (str)
|
||||
{
|
||||
case ".inf":
|
||||
case ".Inf":
|
||||
case ".INF":
|
||||
case "+.inf":
|
||||
case "+.Inf":
|
||||
case "+.INF":
|
||||
value = new NumberToken(m_fileId, scalar.Start.Line, scalar.Start.Column, Double.PositiveInfinity);
|
||||
return true;
|
||||
case "-.inf":
|
||||
case "-.Inf":
|
||||
case "-.INF":
|
||||
value = new NumberToken(m_fileId, scalar.Start.Line, scalar.Start.Column, Double.NegativeInfinity);
|
||||
return true;
|
||||
case ".nan":
|
||||
case ".NaN":
|
||||
case ".NAN":
|
||||
value = new NumberToken(m_fileId, scalar.Start.Line, scalar.Start.Column, Double.NaN);
|
||||
return true;
|
||||
}
|
||||
|
||||
// Otherwise check [-+]?(\.[0-9]+|[0-9]+(\.[0-9]*)?)([eE][-+]?[0-9]+)?
|
||||
|
||||
// Skip leading sign
|
||||
var index = str[0] == '-' || str[0] == '+' ? 1 : 0;
|
||||
|
||||
// Check for integer portion
|
||||
var length = str.Length;
|
||||
var hasInteger = false;
|
||||
while (index < length && str[index] >= '0' && str[index] <= '9')
|
||||
{
|
||||
hasInteger = true;
|
||||
index++;
|
||||
}
|
||||
|
||||
// Check for decimal point
|
||||
var hasDot = false;
|
||||
if (index < length && str[index] == '.')
|
||||
{
|
||||
hasDot = true;
|
||||
index++;
|
||||
}
|
||||
|
||||
// Check for decimal portion
|
||||
var hasDecimal = false;
|
||||
while (index < length && str[index] >= '0' && str[index] <= '9')
|
||||
{
|
||||
hasDecimal = true;
|
||||
index++;
|
||||
}
|
||||
|
||||
// Check [-+]?(\.[0-9]+|[0-9]+(\.[0-9]*)?)
|
||||
if ((hasDot && hasDecimal) || hasInteger)
|
||||
{
|
||||
// Check for end
|
||||
if (index == length)
|
||||
{
|
||||
// Try parse
|
||||
if (Double.TryParse(str, NumberStyles.AllowLeadingSign | NumberStyles.AllowDecimalPoint, CultureInfo.InvariantCulture, out var doubleValue))
|
||||
{
|
||||
value = new NumberToken(m_fileId, scalar.Start.Line, scalar.Start.Column, doubleValue);
|
||||
return true;
|
||||
}
|
||||
// Otherwise exceeds range
|
||||
else
|
||||
{
|
||||
ThrowInvalidValue(scalar, c_floatTag); // throws
|
||||
}
|
||||
}
|
||||
// Check [eE][-+]?[0-9]
|
||||
else if (index < length && (str[index] == 'e' || str[index] == 'E'))
|
||||
{
|
||||
index++;
|
||||
|
||||
// Skip sign
|
||||
if (index < length && (str[index] == '-' || str[index] == '+'))
|
||||
{
|
||||
index++;
|
||||
}
|
||||
|
||||
// Check for exponent
|
||||
var hasExponent = false;
|
||||
while (index < length && str[index] >= '0' && str[index] <= '9')
|
||||
{
|
||||
hasExponent = true;
|
||||
index++;
|
||||
}
|
||||
|
||||
// Check for end
|
||||
if (hasExponent && index == length)
|
||||
{
|
||||
// Try parse
|
||||
if (Double.TryParse(str, NumberStyles.AllowLeadingSign | NumberStyles.AllowDecimalPoint | NumberStyles.AllowExponent, CultureInfo.InvariantCulture, out var doubleValue))
|
||||
{
|
||||
value = new NumberToken(m_fileId, scalar.Start.Line, scalar.Start.Column, (Double)doubleValue);
|
||||
return true;
|
||||
}
|
||||
// Otherwise exceeds range
|
||||
else
|
||||
{
|
||||
ThrowInvalidValue(scalar, c_floatTag); // throws
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
value = default;
|
||||
return false;
|
||||
}
|
||||
|
||||
private Boolean MatchInteger(
|
||||
Scalar scalar,
|
||||
out NumberToken value)
|
||||
{
|
||||
// YAML 1.2 "core" schema https://yaml.org/spec/1.2/spec.html#id2804923
|
||||
var str = scalar.Value;
|
||||
if (!string.IsNullOrEmpty(str))
|
||||
{
|
||||
// Check for [0-9]+
|
||||
var firstChar = str[0];
|
||||
if (firstChar >= '0' && firstChar <= '9' &&
|
||||
str.Skip(1).All(x => x >= '0' && x <= '9'))
|
||||
{
|
||||
// Try parse
|
||||
if (Double.TryParse(str, NumberStyles.None, CultureInfo.InvariantCulture, out var doubleValue))
|
||||
{
|
||||
value = new NumberToken(m_fileId, scalar.Start.Line, scalar.Start.Column, doubleValue);
|
||||
return true;
|
||||
}
|
||||
|
||||
// Otherwise exceeds range
|
||||
ThrowInvalidValue(scalar, c_integerTag); // throws
|
||||
}
|
||||
// Check for (-|+)[0-9]+
|
||||
else if ((firstChar == '-' || firstChar == '+') &&
|
||||
str.Length > 1 &&
|
||||
str.Skip(1).All(x => x >= '0' && x <= '9'))
|
||||
{
|
||||
// Try parse
|
||||
if (Double.TryParse(str, NumberStyles.AllowLeadingSign, CultureInfo.InvariantCulture, out var doubleValue))
|
||||
{
|
||||
value = new NumberToken(m_fileId, scalar.Start.Line, scalar.Start.Column, doubleValue);
|
||||
return true;
|
||||
}
|
||||
|
||||
// Otherwise exceeds range
|
||||
ThrowInvalidValue(scalar, c_integerTag); // throws
|
||||
}
|
||||
// Check for 0x[0-9a-fA-F]+
|
||||
else if (firstChar == '0' &&
|
||||
str.Length > 2 &&
|
||||
str[1] == 'x' &&
|
||||
str.Skip(2).All(x => (x >= '0' && x <= '9') || (x >= 'a' && x <= 'f') || (x >= 'A' && x <= 'F')))
|
||||
{
|
||||
// Try parse
|
||||
if (Int32.TryParse(str.Substring(2), NumberStyles.AllowHexSpecifier, CultureInfo.InvariantCulture, out var integerValue))
|
||||
{
|
||||
value = new NumberToken(m_fileId, scalar.Start.Line, scalar.Start.Column, integerValue);
|
||||
return true;
|
||||
}
|
||||
|
||||
// Otherwise exceeds range
|
||||
ThrowInvalidValue(scalar, c_integerTag); // throws
|
||||
}
|
||||
// Check for 0o[0-9]+
|
||||
else if (firstChar == '0' &&
|
||||
str.Length > 2 &&
|
||||
str[1] == 'o' &&
|
||||
str.Skip(2).All(x => x >= '0' && x <= '7'))
|
||||
{
|
||||
// Try parse
|
||||
var integerValue = default(Int32);
|
||||
try
|
||||
{
|
||||
integerValue = Convert.ToInt32(str.Substring(2), 8);
|
||||
}
|
||||
// Otherwise exceeds range
|
||||
catch (Exception)
|
||||
{
|
||||
ThrowInvalidValue(scalar, c_integerTag); // throws
|
||||
}
|
||||
|
||||
value = new NumberToken(m_fileId, scalar.Start.Line, scalar.Start.Column, integerValue);
|
||||
return true;
|
||||
}
|
||||
}
|
||||
|
||||
value = default;
|
||||
return false;
|
||||
}
|
||||
|
||||
private Boolean MatchNull(
|
||||
Scalar scalar,
|
||||
out NullToken value)
|
||||
{
|
||||
// YAML 1.2 "core" schema https://yaml.org/spec/1.2/spec.html#id2804923
|
||||
switch (scalar.Value ?? string.Empty)
|
||||
{
|
||||
case "":
|
||||
case "null":
|
||||
case "Null":
|
||||
case "NULL":
|
||||
case "~":
|
||||
value = new NullToken(m_fileId, scalar.Start.Line, scalar.Start.Column);
|
||||
return true;
|
||||
}
|
||||
|
||||
value = default;
|
||||
return false;
|
||||
}
|
||||
|
||||
private void ThrowInvalidValue(
|
||||
Scalar scalar,
|
||||
String tag)
|
||||
{
|
||||
throw new NotSupportedException($"The value '{scalar.Value}' on line {scalar.Start.Line} and column {scalar.Start.Column} is invalid for the type '{scalar.Tag}'");
|
||||
}
|
||||
|
||||
private const String c_booleanTag = "tag:yaml.org,2002:bool";
|
||||
private const String c_floatTag = "tag:yaml.org,2002:float";
|
||||
private const String c_integerTag = "tag:yaml.org,2002:int";
|
||||
private const String c_nullTag = "tag:yaml.org,2002:null";
|
||||
private const String c_stringTag = "tag:yaml.org,2002:string";
|
||||
private readonly Int32? m_fileId;
|
||||
private readonly Parser m_parser;
|
||||
private ParsingEvent m_current;
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -18,7 +18,6 @@ namespace GitHub.Runner.Worker
|
||||
{
|
||||
public enum ActionRunStage
|
||||
{
|
||||
Pre,
|
||||
Main,
|
||||
Post,
|
||||
}
|
||||
@@ -82,18 +81,20 @@ namespace GitHub.Runner.Worker
|
||||
ActionExecutionData handlerData = definition.Data?.Execution;
|
||||
ArgUtil.NotNull(handlerData, nameof(handlerData));
|
||||
|
||||
if (handlerData.HasInit &&
|
||||
Action.Reference is Pipelines.RepositoryPathReference repoAction &&
|
||||
string.Equals(repoAction.RepositoryType, Pipelines.PipelineConstants.SelfAlias, StringComparison.OrdinalIgnoreCase))
|
||||
{
|
||||
ExecutionContext.Warning($"`pre` execution is not supported for local action from '{repoAction.Path}'");
|
||||
}
|
||||
|
||||
// The action has post cleanup defined.
|
||||
// we need to create timeline record for them and add them to the step list that StepRunner is using
|
||||
if (handlerData.HasCleanup && (Stage == ActionRunStage.Pre || Stage == ActionRunStage.Main))
|
||||
if (handlerData.HasCleanup && Stage == ActionRunStage.Main)
|
||||
{
|
||||
string postDisplayName = $"Post {this.DisplayName}";
|
||||
string postDisplayName = null;
|
||||
if (this.DisplayName.StartsWith(PipelineTemplateConstants.RunDisplayPrefix))
|
||||
{
|
||||
postDisplayName = $"Post {this.DisplayName.Substring(PipelineTemplateConstants.RunDisplayPrefix.Length)}";
|
||||
}
|
||||
else
|
||||
{
|
||||
postDisplayName = $"Post {this.DisplayName}";
|
||||
}
|
||||
|
||||
var repositoryReference = Action.Reference as RepositoryPathReference;
|
||||
var pathString = string.IsNullOrEmpty(repositoryReference.Path) ? string.Empty : $"/{repositoryReference.Path}";
|
||||
var repoString = string.IsNullOrEmpty(repositoryReference.Ref) ? $"{repositoryReference.Name}{pathString}" :
|
||||
@@ -107,7 +108,7 @@ namespace GitHub.Runner.Worker
|
||||
actionRunner.Condition = handlerData.CleanupCondition;
|
||||
actionRunner.DisplayName = postDisplayName;
|
||||
|
||||
ExecutionContext.RegisterPostJobStep(actionRunner);
|
||||
ExecutionContext.RegisterPostJobStep($"{actionRunner.Action.Name}_post", actionRunner);
|
||||
}
|
||||
|
||||
IStepHost stepHost = HostContext.CreateService<IDefaultStepHost>();
|
||||
@@ -140,9 +141,7 @@ namespace GitHub.Runner.Worker
|
||||
|
||||
// Load the inputs.
|
||||
ExecutionContext.Debug("Loading inputs");
|
||||
var templateTrace = ExecutionContext.ToTemplateTraceWriter();
|
||||
var schema = new PipelineTemplateSchemaFactory().CreateSchema();
|
||||
var templateEvaluator = new PipelineTemplateEvaluator(templateTrace, schema);
|
||||
var templateEvaluator = ExecutionContext.ToPipelineTemplateEvaluator();
|
||||
var inputs = templateEvaluator.EvaluateStepInputs(Action.Inputs, ExecutionContext.ExpressionValues);
|
||||
|
||||
foreach (KeyValuePair<string, string> input in inputs)
|
||||
@@ -294,8 +293,7 @@ namespace GitHub.Runner.Worker
|
||||
return displayName;
|
||||
}
|
||||
// Try evaluating fully
|
||||
var schema = new PipelineTemplateSchemaFactory().CreateSchema();
|
||||
var templateEvaluator = new PipelineTemplateEvaluator(context.ToTemplateTraceWriter(), schema);
|
||||
var templateEvaluator = context.ToPipelineTemplateEvaluator();
|
||||
try
|
||||
{
|
||||
didFullyEvaluate = templateEvaluator.TryEvaluateStepDisplayName(tokenToParse, contextData, out displayName);
|
||||
|
||||
@@ -49,7 +49,7 @@ namespace GitHub.Runner.Worker
|
||||
data: data);
|
||||
|
||||
executionContext.Debug($"Register post job cleanup for stopping/deleting containers.");
|
||||
executionContext.RegisterPostJobStep(postJobStep);
|
||||
executionContext.RegisterPostJobStep(nameof(StopContainersAsync), postJobStep);
|
||||
|
||||
// Check whether we are inside a container.
|
||||
// Our container feature requires to map working directory from host to the container.
|
||||
|
||||
@@ -11,6 +11,7 @@ using GitHub.Runner.Worker.Container;
|
||||
using GitHub.Services.WebApi;
|
||||
using GitHub.DistributedTask.Pipelines;
|
||||
using GitHub.DistributedTask.Pipelines.ContextData;
|
||||
using GitHub.DistributedTask.Pipelines.ObjectTemplating;
|
||||
using GitHub.DistributedTask.WebApi;
|
||||
using GitHub.Runner.Common.Util;
|
||||
using GitHub.Runner.Common;
|
||||
@@ -21,7 +22,6 @@ using System.Collections;
|
||||
using ObjectTemplating = GitHub.DistributedTask.ObjectTemplating;
|
||||
using Pipelines = GitHub.DistributedTask.Pipelines;
|
||||
using GitHub.DistributedTask.Expressions2;
|
||||
using System.Diagnostics;
|
||||
|
||||
namespace GitHub.Runner.Worker
|
||||
{
|
||||
@@ -50,6 +50,7 @@ namespace GitHub.Runner.Worker
|
||||
HashSet<string> OutputVariables { get; }
|
||||
IDictionary<String, String> EnvironmentVariables { get; }
|
||||
IDictionary<String, ContextScope> Scopes { get; }
|
||||
IList<String> FileTable { get; }
|
||||
StepsContext StepsContext { get; }
|
||||
DictionaryContextData ExpressionValues { get; }
|
||||
List<string> PrependPath { get; }
|
||||
@@ -99,9 +100,7 @@ namespace GitHub.Runner.Worker
|
||||
|
||||
// others
|
||||
void ForceTaskComplete();
|
||||
void RegisterPostJobStep(IStep step);
|
||||
Task<string> GetGitHubToken();
|
||||
Task UpdateGitHubTokenInContext();
|
||||
void RegisterPostJobStep(string refName, IStep step);
|
||||
}
|
||||
|
||||
public sealed class ExecutionContext : RunnerService, IExecutionContext
|
||||
@@ -113,7 +112,6 @@ namespace GitHub.Runner.Worker
|
||||
private readonly object _loggerLock = new object();
|
||||
private readonly HashSet<string> _outputvariables = new HashSet<string>(StringComparer.OrdinalIgnoreCase);
|
||||
private readonly object _matchersLock = new object();
|
||||
public readonly List<Task> _getGitHubTokenTasks = new List<Task>();
|
||||
|
||||
private event OnMatcherChanged _onMatcherChanged;
|
||||
|
||||
@@ -134,13 +132,6 @@ namespace GitHub.Runner.Worker
|
||||
// only job level ExecutionContext will track throttling delay.
|
||||
private long _totalThrottlingDelayInMilliseconds = 0;
|
||||
|
||||
private Guid _jobId;
|
||||
private Guid _scopeIdentifier;
|
||||
private string _hubName;
|
||||
private Guid _planId;
|
||||
|
||||
private Stopwatch _githubTokenExpireTimer = new Stopwatch();
|
||||
|
||||
public Guid Id => _record.Id;
|
||||
public string ScopeName { get; private set; }
|
||||
public string ContextName { get; private set; }
|
||||
@@ -152,6 +143,7 @@ namespace GitHub.Runner.Worker
|
||||
public HashSet<string> OutputVariables => _outputvariables;
|
||||
public IDictionary<String, String> EnvironmentVariables { get; private set; }
|
||||
public IDictionary<String, ContextScope> Scopes { get; private set; }
|
||||
public IList<String> FileTable { get; private set; }
|
||||
public StepsContext StepsContext { get; private set; }
|
||||
public DictionaryContextData ExpressionValues { get; } = new DictionaryContextData();
|
||||
public bool WriteDebug { get; private set; }
|
||||
@@ -165,9 +157,6 @@ namespace GitHub.Runner.Worker
|
||||
// Only job level ExecutionContext has PostJobSteps
|
||||
public Stack<IStep> PostJobSteps { get; private set; }
|
||||
|
||||
// Only job level ExecutionContext has StepsWithPostRegisted
|
||||
public HashSet<Guid> StepsWithPostRegisted { get; private set; }
|
||||
|
||||
public bool EchoOnActionCommand { get; set; }
|
||||
|
||||
|
||||
@@ -253,68 +242,9 @@ namespace GitHub.Runner.Worker
|
||||
});
|
||||
}
|
||||
|
||||
public Task<string> GetGitHubToken()
|
||||
public void RegisterPostJobStep(string refName, IStep step)
|
||||
{
|
||||
Trace.Info($"Try get new GITHUB_TOKEN");
|
||||
return Root.RefreshGitHubToken();
|
||||
}
|
||||
|
||||
public async Task UpdateGitHubTokenInContext()
|
||||
{
|
||||
try
|
||||
{
|
||||
await Root.EnsureGitHubToken();
|
||||
}
|
||||
catch (Exception ex)
|
||||
{
|
||||
this.Warning($"Fail to get a new GITHUB_TOKEN, error: {ex.Message}");
|
||||
this.Debug(ex.ToString());
|
||||
}
|
||||
}
|
||||
|
||||
private async Task<string> RefreshGitHubToken()
|
||||
{
|
||||
Trace.Info("Request new GITHUB_TOKEN");
|
||||
var jobServer = HostContext.GetService<IJobServer>();
|
||||
var githubToken = await jobServer.RefreshGitHubTokenAsync(_scopeIdentifier, _hubName, _planId, _jobId, CancellationToken.None);
|
||||
|
||||
if (!string.IsNullOrEmpty(githubToken?.Token))
|
||||
{
|
||||
// register secret masker
|
||||
Trace.Info("Register secret masker for new GITHUB_TOKEN");
|
||||
HostContext.SecretMasker.AddValue(githubToken.Token);
|
||||
return githubToken.Token;
|
||||
}
|
||||
else
|
||||
{
|
||||
throw new InvalidOperationException("Get empty GTIHUB_TOKEN.");
|
||||
}
|
||||
}
|
||||
|
||||
private async Task EnsureGitHubToken()
|
||||
{
|
||||
// needs to refresh GITHUB_TOKEN every 50 mins since the token is good for 60 min by default
|
||||
if (_githubTokenExpireTimer.Elapsed.TotalMilliseconds > 10)
|
||||
{
|
||||
var githubToken = await this.RefreshGitHubToken();
|
||||
var secretsContext = ExpressionValues["secrets"] as DictionaryContextData;
|
||||
secretsContext["GITHUB_TOKEN"] = new StringContextData(githubToken);
|
||||
var githubContext = ExpressionValues["github"] as GitHubContext;
|
||||
githubContext["token"] = new StringContextData(githubToken);
|
||||
|
||||
_githubTokenExpireTimer.Restart();
|
||||
}
|
||||
}
|
||||
|
||||
public void RegisterPostJobStep(IStep step)
|
||||
{
|
||||
if (step is IActionRunner actionRunner && !Root.StepsWithPostRegisted.Add(actionRunner.Action.Id))
|
||||
{
|
||||
Trace.Info($"'post' of '{actionRunner.DisplayName}' already push to post step stack.");
|
||||
return;
|
||||
}
|
||||
|
||||
step.ExecutionContext = Root.CreatePostChild(step.DisplayName, ScopeName, ContextName, IntraActionState);
|
||||
step.ExecutionContext = Root.CreatePostChild(step.DisplayName, refName, IntraActionState);
|
||||
Root.PostJobSteps.Push(step);
|
||||
}
|
||||
|
||||
@@ -339,6 +269,7 @@ namespace GitHub.Runner.Worker
|
||||
}
|
||||
child.EnvironmentVariables = EnvironmentVariables;
|
||||
child.Scopes = Scopes;
|
||||
child.FileTable = FileTable;
|
||||
child.StepsContext = StepsContext;
|
||||
foreach (var pair in ExpressionValues)
|
||||
{
|
||||
@@ -642,6 +573,9 @@ namespace GitHub.Runner.Worker
|
||||
}
|
||||
}
|
||||
|
||||
// File table
|
||||
FileTable = new List<String>(message.FileTable ?? new string[0]);
|
||||
|
||||
// Expression functions
|
||||
if (Variables.GetBoolean("System.HashFilesV2") == true)
|
||||
{
|
||||
@@ -681,7 +615,6 @@ namespace GitHub.Runner.Worker
|
||||
|
||||
ExpressionValues["env"] = new CaseSensitiveDictionaryContextData();
|
||||
#endif
|
||||
_githubTokenExpireTimer.Start();
|
||||
|
||||
// Prepend Path
|
||||
PrependPath = new List<string>();
|
||||
@@ -692,14 +625,6 @@ namespace GitHub.Runner.Worker
|
||||
// PostJobSteps for job ExecutionContext
|
||||
PostJobSteps = new Stack<IStep>();
|
||||
|
||||
// StepsWithPostRegisted for job ExecutionContext
|
||||
StepsWithPostRegisted = new HashSet<Guid>();
|
||||
|
||||
_scopeIdentifier = message.Plan.ScopeIdentifier;
|
||||
_hubName = message.Plan.PlanType;
|
||||
_jobId = message.JobId;
|
||||
_planId = message.Plan.PlanId;
|
||||
|
||||
// Job timeline record.
|
||||
InitializeTimelineRecord(
|
||||
timelineId: message.Timeline.Id,
|
||||
@@ -900,7 +825,7 @@ namespace GitHub.Runner.Worker
|
||||
}
|
||||
}
|
||||
|
||||
private IExecutionContext CreatePostChild(string displayName, string scopeName, string contextName, Dictionary<string, string> intraActionState)
|
||||
private IExecutionContext CreatePostChild(string displayName, string refName, Dictionary<string, string> intraActionState)
|
||||
{
|
||||
if (!_expandedForPostJob)
|
||||
{
|
||||
@@ -909,8 +834,7 @@ namespace GitHub.Runner.Worker
|
||||
_childTimelineRecordOrder = _childTimelineRecordOrder * 2;
|
||||
}
|
||||
|
||||
var newGuid = Guid.NewGuid();
|
||||
return CreateChild(newGuid, displayName, newGuid.ToString("N"), scopeName, contextName, intraActionState, _childTimelineRecordOrder - Root.PostJobSteps.Count);
|
||||
return CreateChild(Guid.NewGuid(), displayName, refName, null, null, intraActionState, _childTimelineRecordOrder - Root.PostJobSteps.Count);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -969,6 +893,13 @@ namespace GitHub.Runner.Worker
|
||||
}
|
||||
}
|
||||
|
||||
public static PipelineTemplateEvaluator ToPipelineTemplateEvaluator(this IExecutionContext context)
|
||||
{
|
||||
var templateTrace = context.ToTemplateTraceWriter();
|
||||
var schema = new PipelineTemplateSchemaFactory().CreateSchema();
|
||||
return new PipelineTemplateEvaluator(templateTrace, schema, context.FileTable);
|
||||
}
|
||||
|
||||
public static ObjectTemplating.ITraceWriter ToTemplateTraceWriter(this IExecutionContext context)
|
||||
{
|
||||
return new TemplateTraceWriter(context);
|
||||
|
||||
@@ -82,10 +82,6 @@ namespace GitHub.Runner.Worker.Handlers
|
||||
container.ContainerEntryPoint = Inputs.GetValueOrDefault("entryPoint");
|
||||
}
|
||||
}
|
||||
else if (stage == ActionRunStage.Pre)
|
||||
{
|
||||
container.ContainerEntryPoint = Data.Init;
|
||||
}
|
||||
else if (stage == ActionRunStage.Post)
|
||||
{
|
||||
container.ContainerEntryPoint = Data.Cleanup;
|
||||
|
||||
@@ -60,10 +60,6 @@ namespace GitHub.Runner.Worker.Handlers
|
||||
{
|
||||
target = Data.Script;
|
||||
}
|
||||
else if (stage == ActionRunStage.Pre)
|
||||
{
|
||||
target = Data.Init;
|
||||
}
|
||||
else if (stage == ActionRunStage.Post)
|
||||
{
|
||||
target = Data.Cleanup;
|
||||
|
||||
@@ -78,9 +78,7 @@ namespace GitHub.Runner.Worker
|
||||
|
||||
// Evaluate the job-level environment variables
|
||||
context.Debug("Evaluating job-level environment variables");
|
||||
var templateTrace = context.ToTemplateTraceWriter();
|
||||
var schema = new PipelineTemplateSchemaFactory().CreateSchema();
|
||||
var templateEvaluator = new PipelineTemplateEvaluator(templateTrace, schema);
|
||||
var templateEvaluator = context.ToPipelineTemplateEvaluator();
|
||||
foreach (var token in message.EnvironmentVariables)
|
||||
{
|
||||
var environmentVariables = templateEvaluator.EvaluateStepEnvironment(token, jobContext.ExpressionValues, VarUtil.EnvironmentVariableKeyComparer);
|
||||
@@ -116,8 +114,8 @@ namespace GitHub.Runner.Worker
|
||||
// Download actions not already in the cache
|
||||
Trace.Info("Downloading actions");
|
||||
var actionManager = HostContext.GetService<IActionManager>();
|
||||
var prepareResult = await actionManager.PrepareActionsAsync(context, message.Steps);
|
||||
preJobSteps.AddRange(prepareResult.ContainerSetupSteps);
|
||||
var prepareSteps = await actionManager.PrepareActionsAsync(context, message.Steps);
|
||||
preJobSteps.AddRange(prepareSteps);
|
||||
|
||||
// Add start-container steps, record and stop-container steps
|
||||
if (jobContext.Container != null || jobContext.ServiceContainers.Count > 0)
|
||||
@@ -158,23 +156,9 @@ namespace GitHub.Runner.Worker
|
||||
|
||||
actionRunner.TryEvaluateDisplayName(contextData, context);
|
||||
jobSteps.Add(actionRunner);
|
||||
|
||||
if (prepareResult.PreStepTracker.TryGetValue(step.Id, out var preStep))
|
||||
{
|
||||
Trace.Info($"Adding pre-{action.DisplayName}.");
|
||||
preStep.TryEvaluateDisplayName(contextData, context);
|
||||
preStep.DisplayName = $"Pre {preStep.DisplayName}";
|
||||
preJobSteps.Add(preStep);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
var intraActionStates = new Dictionary<Guid, Dictionary<string, string>>();
|
||||
foreach (var preStep in prepareResult.PreStepTracker)
|
||||
{
|
||||
intraActionStates[preStep.Key] = new Dictionary<string, string>();
|
||||
}
|
||||
|
||||
// Create execution context for pre-job steps
|
||||
foreach (var step in preJobSteps)
|
||||
{
|
||||
@@ -185,12 +169,6 @@ namespace GitHub.Runner.Worker
|
||||
Guid stepId = Guid.NewGuid();
|
||||
extensionStep.ExecutionContext = jobContext.CreateChild(stepId, extensionStep.DisplayName, null, null, stepId.ToString("N"));
|
||||
}
|
||||
else if (step is IActionRunner actionStep)
|
||||
{
|
||||
ArgUtil.NotNull(actionStep, step.DisplayName);
|
||||
Guid stepId = Guid.NewGuid();
|
||||
actionStep.ExecutionContext = jobContext.CreateChild(stepId, actionStep.DisplayName, null, actionStep.Action.ScopeName, actionStep.Action.ContextName, intraActionStates[actionStep.Action.Id]);
|
||||
}
|
||||
}
|
||||
|
||||
// Create execution context for job steps
|
||||
@@ -199,14 +177,7 @@ namespace GitHub.Runner.Worker
|
||||
if (step is IActionRunner actionStep)
|
||||
{
|
||||
ArgUtil.NotNull(actionStep, step.DisplayName);
|
||||
if (intraActionStates.ContainsKey(actionStep.Action.Id))
|
||||
{
|
||||
actionStep.ExecutionContext = jobContext.CreateChild(actionStep.Action.Id, actionStep.DisplayName, actionStep.Action.Name, actionStep.Action.ScopeName, actionStep.Action.ContextName, intraActionStates[actionStep.Action.Id]);
|
||||
}
|
||||
else
|
||||
{
|
||||
actionStep.ExecutionContext = jobContext.CreateChild(actionStep.Action.Id, actionStep.DisplayName, actionStep.Action.Name, actionStep.Action.ScopeName, actionStep.Action.ContextName);
|
||||
}
|
||||
actionStep.ExecutionContext = jobContext.CreateChild(actionStep.Action.Id, actionStep.DisplayName, actionStep.Action.Name, actionStep.Action.ScopeName, actionStep.Action.ContextName);
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -75,7 +75,6 @@ namespace GitHub.Runner.Worker
|
||||
|
||||
// Start
|
||||
step.ExecutionContext.Start();
|
||||
await step.ExecutionContext.UpdateGitHubTokenInContext();
|
||||
|
||||
// Initialize scope
|
||||
if (InitializeScope(step, scopeInputs))
|
||||
@@ -99,9 +98,7 @@ namespace GitHub.Runner.Worker
|
||||
step.ExecutionContext.SetGitHubContext("action", actionStep.Action.Name);
|
||||
|
||||
// Evaluate and merge action's env block to env context
|
||||
var templateTrace = step.ExecutionContext.ToTemplateTraceWriter();
|
||||
var schema = new PipelineTemplateSchemaFactory().CreateSchema();
|
||||
var templateEvaluator = new PipelineTemplateEvaluator(templateTrace, schema);
|
||||
var templateEvaluator = step.ExecutionContext.ToPipelineTemplateEvaluator();
|
||||
var actionEnvironment = templateEvaluator.EvaluateStepEnvironment(actionStep.Action.Environment, step.ExecutionContext.ExpressionValues, VarUtil.EnvironmentVariableKeyComparer);
|
||||
foreach (var env in actionEnvironment)
|
||||
{
|
||||
@@ -248,7 +245,7 @@ namespace GitHub.Runner.Worker
|
||||
|
||||
// Set the timeout
|
||||
var timeoutMinutes = 0;
|
||||
var templateEvaluator = CreateTemplateEvaluator(step.ExecutionContext);
|
||||
var templateEvaluator = step.ExecutionContext.ToPipelineTemplateEvaluator();
|
||||
try
|
||||
{
|
||||
timeoutMinutes = templateEvaluator.EvaluateStepTimeout(step.Timeout, step.ExecutionContext.ExpressionValues);
|
||||
@@ -390,7 +387,7 @@ namespace GitHub.Runner.Worker
|
||||
executionContext.Debug($"Initializing scope '{scope.Name}'");
|
||||
executionContext.ExpressionValues["steps"] = stepsContext.GetScope(scope.ParentName);
|
||||
executionContext.ExpressionValues["inputs"] = !String.IsNullOrEmpty(scope.ParentName) ? scopeInputs[scope.ParentName] : null;
|
||||
var templateEvaluator = CreateTemplateEvaluator(executionContext);
|
||||
var templateEvaluator = executionContext.ToPipelineTemplateEvaluator();
|
||||
var inputs = default(DictionaryContextData);
|
||||
try
|
||||
{
|
||||
@@ -446,7 +443,7 @@ namespace GitHub.Runner.Worker
|
||||
executionContext.Debug($"Finalizing scope '{scope.Name}'");
|
||||
executionContext.ExpressionValues["steps"] = stepsContext.GetScope(scope.Name);
|
||||
executionContext.ExpressionValues["inputs"] = null;
|
||||
var templateEvaluator = CreateTemplateEvaluator(executionContext);
|
||||
var templateEvaluator = executionContext.ToPipelineTemplateEvaluator();
|
||||
var outputs = default(DictionaryContextData);
|
||||
try
|
||||
{
|
||||
@@ -478,12 +475,5 @@ namespace GitHub.Runner.Worker
|
||||
|
||||
executionContext.Complete(result, resultCode: resultCode);
|
||||
}
|
||||
|
||||
private PipelineTemplateEvaluator CreateTemplateEvaluator(IExecutionContext executionContext)
|
||||
{
|
||||
var templateTrace = executionContext.ToTemplateTraceWriter();
|
||||
var schema = new PipelineTemplateSchemaFactory().CreateSchema();
|
||||
return new PipelineTemplateEvaluator(templateTrace, schema);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -43,8 +43,6 @@
|
||||
"entrypoint": "non-empty-string",
|
||||
"args": "container-runs-args",
|
||||
"env": "container-runs-env",
|
||||
"pre-entrypoint": "non-empty-string",
|
||||
"pre-if": "non-empty-string",
|
||||
"post-entrypoint": "non-empty-string",
|
||||
"post-if": "non-empty-string"
|
||||
}
|
||||
@@ -69,8 +67,6 @@
|
||||
"properties": {
|
||||
"using": "non-empty-string",
|
||||
"main": "non-empty-string",
|
||||
"pre": "non-empty-string",
|
||||
"pre-if": "non-empty-string",
|
||||
"post": "non-empty-string",
|
||||
"post-if": "non-empty-string"
|
||||
}
|
||||
|
||||
@@ -317,40 +317,5 @@ namespace GitHub.DistributedTask.WebApi
|
||||
userState: userState,
|
||||
cancellationToken: cancellationToken);
|
||||
}
|
||||
|
||||
|
||||
/// <summary>
|
||||
/// [Preview API]
|
||||
/// </summary>
|
||||
/// <param name="scopeIdentifier">The project GUID to scope the request</param>
|
||||
/// <param name="hubName">The name of the server hub: "build" for the Build server or "rm" for the Release Management server</param>
|
||||
/// <param name="planId"></param>
|
||||
/// <param name="jobId"></param>
|
||||
/// <param name="userState"></param>
|
||||
/// <param name="cancellationToken">The cancellation token to cancel operation.</param>
|
||||
public virtual Task<GitHubToken> RefreshTokenAsync(
|
||||
Guid scopeIdentifier,
|
||||
string hubName,
|
||||
Guid planId,
|
||||
Guid jobId,
|
||||
object userState = null,
|
||||
CancellationToken cancellationToken = default)
|
||||
{
|
||||
HttpMethod httpMethod = new HttpMethod("GET");
|
||||
Guid locationId = new Guid("8aa8aff7-751b-496e-be8d-b7818770efb3");
|
||||
object routeValues = new { scopeIdentifier = scopeIdentifier, hubName = hubName, planId = planId };
|
||||
|
||||
List<KeyValuePair<string, string>> queryParams = new List<KeyValuePair<string, string>>();
|
||||
queryParams.Add("jobId", jobId.ToString());
|
||||
|
||||
return SendAsync<GitHubToken>(
|
||||
httpMethod,
|
||||
locationId,
|
||||
routeValues: routeValues,
|
||||
version: new ApiResourceVersion(6.0, 1),
|
||||
queryParameters: queryParams,
|
||||
userState: userState,
|
||||
cancellationToken: cancellationToken);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -184,6 +184,7 @@ namespace GitHub.DistributedTask.ObjectTemplating
|
||||
id = FileIds.Count + 1;
|
||||
FileIds.Add(file, id);
|
||||
FileNames.Add(file);
|
||||
Memory.AddBytes(file);
|
||||
}
|
||||
|
||||
return id;
|
||||
@@ -191,7 +192,12 @@ namespace GitHub.DistributedTask.ObjectTemplating
|
||||
|
||||
internal String GetFileName(Int32 fileId)
|
||||
{
|
||||
return FileNames[fileId - 1];
|
||||
return FileNames.Count >= fileId ? FileNames[fileId - 1] : null;
|
||||
}
|
||||
|
||||
internal IReadOnlyList<String> GetFileTable()
|
||||
{
|
||||
return FileNames.AsReadOnly();
|
||||
}
|
||||
|
||||
private String GetErrorPrefix(
|
||||
@@ -199,9 +205,9 @@ namespace GitHub.DistributedTask.ObjectTemplating
|
||||
Int32? line,
|
||||
Int32? column)
|
||||
{
|
||||
if (fileId != null)
|
||||
var fileName = fileId.HasValue ? GetFileName(fileId.Value) : null;
|
||||
if (!String.IsNullOrEmpty(fileName))
|
||||
{
|
||||
var fileName = GetFileName(fileId.Value);
|
||||
if (line != null && column != null)
|
||||
{
|
||||
return $"{fileName} {TemplateStrings.LineColumn(line, column)}:";
|
||||
|
||||
@@ -30,14 +30,14 @@ namespace GitHub.DistributedTask.ObjectTemplating.Tokens
|
||||
Column = column;
|
||||
}
|
||||
|
||||
[IgnoreDataMember]
|
||||
internal Int32? FileId { get; set; }
|
||||
[DataMember(Name = "file", EmitDefaultValue = false)]
|
||||
internal Int32? FileId { get; private set; }
|
||||
|
||||
[DataMember(Name = "line", EmitDefaultValue = false)]
|
||||
internal Int32? Line { get; }
|
||||
internal Int32? Line { get; private set; }
|
||||
|
||||
[DataMember(Name = "col", EmitDefaultValue = false)]
|
||||
internal Int32? Column { get; }
|
||||
internal Int32? Column { get; private set; }
|
||||
|
||||
[DataMember(Name = "type", EmitDefaultValue = false)]
|
||||
internal Int32 Type { get; }
|
||||
|
||||
@@ -115,13 +115,12 @@ namespace GitHub.DistributedTask.ObjectTemplating.Tokens
|
||||
Object value,
|
||||
JsonSerializer serializer)
|
||||
{
|
||||
base.WriteJson(writer, value, serializer);
|
||||
if (value is TemplateToken token)
|
||||
{
|
||||
switch (token.Type)
|
||||
{
|
||||
case TokenType.Null:
|
||||
if (token.Line == null && token.Column == null)
|
||||
if (token.FileId == null && token.Line == null && token.Column == null)
|
||||
{
|
||||
writer.WriteNull();
|
||||
}
|
||||
@@ -130,12 +129,17 @@ namespace GitHub.DistributedTask.ObjectTemplating.Tokens
|
||||
writer.WriteStartObject();
|
||||
writer.WritePropertyName("type");
|
||||
writer.WriteValue(token.Type);
|
||||
if (token.FileId != null)
|
||||
{
|
||||
writer.WritePropertyName("file");
|
||||
writer.WriteValue(token.FileId);
|
||||
}
|
||||
if (token.Line != null)
|
||||
{
|
||||
writer.WritePropertyName("line");
|
||||
writer.WriteValue(token.Line);
|
||||
}
|
||||
if (token.Line != null)
|
||||
if (token.Column != null)
|
||||
{
|
||||
writer.WritePropertyName("col");
|
||||
writer.WriteValue(token.Column);
|
||||
@@ -146,7 +150,7 @@ namespace GitHub.DistributedTask.ObjectTemplating.Tokens
|
||||
|
||||
case TokenType.Boolean:
|
||||
var booleanToken = token as BooleanToken;
|
||||
if (token.Line == null && token.Column == null)
|
||||
if (token.FileId == null && token.Line == null && token.Column == null)
|
||||
{
|
||||
writer.WriteValue(booleanToken.Value);
|
||||
}
|
||||
@@ -155,12 +159,17 @@ namespace GitHub.DistributedTask.ObjectTemplating.Tokens
|
||||
writer.WriteStartObject();
|
||||
writer.WritePropertyName("type");
|
||||
writer.WriteValue(token.Type);
|
||||
if (token.FileId != null)
|
||||
{
|
||||
writer.WritePropertyName("file");
|
||||
writer.WriteValue(token.FileId);
|
||||
}
|
||||
if (token.Line != null)
|
||||
{
|
||||
writer.WritePropertyName("line");
|
||||
writer.WriteValue(token.Line);
|
||||
}
|
||||
if (token.Line != null)
|
||||
if (token.Column != null)
|
||||
{
|
||||
writer.WritePropertyName("col");
|
||||
writer.WriteValue(token.Column);
|
||||
@@ -173,7 +182,7 @@ namespace GitHub.DistributedTask.ObjectTemplating.Tokens
|
||||
|
||||
case TokenType.Number:
|
||||
var numberToken = token as NumberToken;
|
||||
if (token.Line == null && token.Column == null)
|
||||
if (token.FileId == null && token.Line == null && token.Column == null)
|
||||
{
|
||||
writer.WriteValue(numberToken.Value);
|
||||
}
|
||||
@@ -182,12 +191,17 @@ namespace GitHub.DistributedTask.ObjectTemplating.Tokens
|
||||
writer.WriteStartObject();
|
||||
writer.WritePropertyName("type");
|
||||
writer.WriteValue(token.Type);
|
||||
if (token.FileId != null)
|
||||
{
|
||||
writer.WritePropertyName("file");
|
||||
writer.WriteValue(token.FileId);
|
||||
}
|
||||
if (token.Line != null)
|
||||
{
|
||||
writer.WritePropertyName("line");
|
||||
writer.WriteValue(token.Line);
|
||||
}
|
||||
if (token.Line != null)
|
||||
if (token.Column != null)
|
||||
{
|
||||
writer.WritePropertyName("col");
|
||||
writer.WriteValue(token.Column);
|
||||
@@ -200,7 +214,7 @@ namespace GitHub.DistributedTask.ObjectTemplating.Tokens
|
||||
|
||||
case TokenType.String:
|
||||
var stringToken = token as StringToken;
|
||||
if (token.Line == null && token.Column == null)
|
||||
if (token.FileId == null && token.Line == null && token.Column == null)
|
||||
{
|
||||
writer.WriteValue(stringToken.Value);
|
||||
}
|
||||
@@ -209,12 +223,17 @@ namespace GitHub.DistributedTask.ObjectTemplating.Tokens
|
||||
writer.WriteStartObject();
|
||||
writer.WritePropertyName("type");
|
||||
writer.WriteValue(token.Type);
|
||||
if (token.FileId != null)
|
||||
{
|
||||
writer.WritePropertyName("file");
|
||||
writer.WriteValue(token.FileId);
|
||||
}
|
||||
if (token.Line != null)
|
||||
{
|
||||
writer.WritePropertyName("line");
|
||||
writer.WriteValue(token.Line);
|
||||
}
|
||||
if (token.Line != null)
|
||||
if (token.Column != null)
|
||||
{
|
||||
writer.WritePropertyName("col");
|
||||
writer.WriteValue(token.Column);
|
||||
@@ -230,12 +249,17 @@ namespace GitHub.DistributedTask.ObjectTemplating.Tokens
|
||||
writer.WriteStartObject();
|
||||
writer.WritePropertyName("type");
|
||||
writer.WriteValue(token.Type);
|
||||
if (token.FileId != null)
|
||||
{
|
||||
writer.WritePropertyName("file");
|
||||
writer.WriteValue(token.FileId);
|
||||
}
|
||||
if (token.Line != null)
|
||||
{
|
||||
writer.WritePropertyName("line");
|
||||
writer.WriteValue(token.Line);
|
||||
}
|
||||
if (token.Line != null)
|
||||
if (token.Column != null)
|
||||
{
|
||||
writer.WritePropertyName("col");
|
||||
writer.WriteValue(token.Column);
|
||||
@@ -253,12 +277,17 @@ namespace GitHub.DistributedTask.ObjectTemplating.Tokens
|
||||
writer.WriteStartObject();
|
||||
writer.WritePropertyName("type");
|
||||
writer.WriteValue(token.Type);
|
||||
if (token.FileId != null)
|
||||
{
|
||||
writer.WritePropertyName("file");
|
||||
writer.WriteValue(token.FileId);
|
||||
}
|
||||
if (token.Line != null)
|
||||
{
|
||||
writer.WritePropertyName("line");
|
||||
writer.WriteValue(token.Line);
|
||||
}
|
||||
if (token.Line != null)
|
||||
if (token.Column != null)
|
||||
{
|
||||
writer.WritePropertyName("col");
|
||||
writer.WriteValue(token.Column);
|
||||
@@ -273,12 +302,17 @@ namespace GitHub.DistributedTask.ObjectTemplating.Tokens
|
||||
writer.WriteStartObject();
|
||||
writer.WritePropertyName("type");
|
||||
writer.WriteValue(token.Type);
|
||||
if (token.FileId != null)
|
||||
{
|
||||
writer.WritePropertyName("file");
|
||||
writer.WriteValue(token.FileId);
|
||||
}
|
||||
if (token.Line != null)
|
||||
{
|
||||
writer.WritePropertyName("line");
|
||||
writer.WriteValue(token.Line);
|
||||
}
|
||||
if (token.Line != null)
|
||||
if (token.Column != null)
|
||||
{
|
||||
writer.WritePropertyName("col");
|
||||
writer.WriteValue(token.Column);
|
||||
@@ -301,12 +335,17 @@ namespace GitHub.DistributedTask.ObjectTemplating.Tokens
|
||||
writer.WriteStartObject();
|
||||
writer.WritePropertyName("type");
|
||||
writer.WriteValue(token.Type);
|
||||
if (token.FileId != null)
|
||||
{
|
||||
writer.WritePropertyName("file");
|
||||
writer.WriteValue(token.FileId);
|
||||
}
|
||||
if (token.Line != null)
|
||||
{
|
||||
writer.WritePropertyName("line");
|
||||
writer.WriteValue(token.Line);
|
||||
}
|
||||
if (token.Line != null)
|
||||
if (token.Column != null)
|
||||
{
|
||||
writer.WritePropertyName("col");
|
||||
writer.WriteValue(token.Column);
|
||||
|
||||
@@ -39,7 +39,8 @@ namespace GitHub.DistributedTask.Pipelines
|
||||
DictionaryContextData contextData,
|
||||
WorkspaceOptions workspaceOptions,
|
||||
IEnumerable<JobStep> steps,
|
||||
IEnumerable<ContextScope> scopes)
|
||||
IEnumerable<ContextScope> scopes,
|
||||
IList<String> fileTable)
|
||||
{
|
||||
this.MessageType = JobRequestMessageTypes.PipelineAgentJobRequest;
|
||||
this.Plan = plan;
|
||||
@@ -74,6 +75,11 @@ namespace GitHub.DistributedTask.Pipelines
|
||||
this.ContextData[pair.Key] = pair.Value;
|
||||
}
|
||||
}
|
||||
|
||||
if (fileTable?.Count > 0)
|
||||
{
|
||||
m_fileTable = new List<String>(fileTable);
|
||||
}
|
||||
}
|
||||
|
||||
[DataMember]
|
||||
@@ -237,6 +243,18 @@ namespace GitHub.DistributedTask.Pipelines
|
||||
}
|
||||
}
|
||||
|
||||
public IList<String> FileTable
|
||||
{
|
||||
get
|
||||
{
|
||||
if (m_fileTable == null)
|
||||
{
|
||||
m_fileTable = new List<String>();
|
||||
}
|
||||
return m_fileTable;
|
||||
}
|
||||
}
|
||||
|
||||
// todo: remove after feature-flag DistributedTask.EvaluateContainerOnRunner is enabled everywhere
|
||||
public void SetJobSidecarContainers(IDictionary<String, String> value)
|
||||
{
|
||||
@@ -345,6 +363,11 @@ namespace GitHub.DistributedTask.Pipelines
|
||||
m_environmentVariables = null;
|
||||
}
|
||||
|
||||
if (m_fileTable?.Count == 0)
|
||||
{
|
||||
m_fileTable = null;
|
||||
}
|
||||
|
||||
if (m_maskHints?.Count == 0)
|
||||
{
|
||||
m_maskHints = null;
|
||||
@@ -374,6 +397,9 @@ namespace GitHub.DistributedTask.Pipelines
|
||||
[DataMember(Name = "EnvironmentVariables", EmitDefaultValue = false)]
|
||||
private List<TemplateToken> m_environmentVariables;
|
||||
|
||||
[DataMember(Name = "FileTable", EmitDefaultValue = false)]
|
||||
private List<String> m_fileTable;
|
||||
|
||||
[DataMember(Name = "Mask", EmitDefaultValue = false)]
|
||||
private List<MaskHint> m_maskHints;
|
||||
|
||||
|
||||
@@ -19,7 +19,8 @@ namespace GitHub.DistributedTask.Pipelines.ObjectTemplating
|
||||
{
|
||||
public PipelineTemplateEvaluator(
|
||||
ITraceWriter trace,
|
||||
TemplateSchema schema)
|
||||
TemplateSchema schema,
|
||||
IList<String> fileTable)
|
||||
{
|
||||
if (!String.Equals(schema.Version, PipelineTemplateConstants.Workflow_1_0, StringComparison.Ordinal))
|
||||
{
|
||||
@@ -28,6 +29,7 @@ namespace GitHub.DistributedTask.Pipelines.ObjectTemplating
|
||||
|
||||
m_trace = trace;
|
||||
m_schema = schema;
|
||||
m_fileTable = fileTable;
|
||||
}
|
||||
|
||||
public Int32 MaxDepth => 50;
|
||||
@@ -324,6 +326,16 @@ namespace GitHub.DistributedTask.Pipelines.ObjectTemplating
|
||||
TraceWriter = m_trace,
|
||||
};
|
||||
|
||||
// Add the file table
|
||||
if (m_fileTable?.Count > 0)
|
||||
{
|
||||
foreach (var file in m_fileTable)
|
||||
{
|
||||
result.GetFileId(file);
|
||||
}
|
||||
}
|
||||
|
||||
// Add named context
|
||||
if (contextData != null)
|
||||
{
|
||||
foreach (var pair in contextData)
|
||||
@@ -346,6 +358,7 @@ namespace GitHub.DistributedTask.Pipelines.ObjectTemplating
|
||||
|
||||
private readonly ITraceWriter m_trace;
|
||||
private readonly TemplateSchema m_schema;
|
||||
private readonly IList<String> m_fileTable;
|
||||
private readonly String[] s_contextNames = new[]
|
||||
{
|
||||
PipelineTemplateConstants.GitHub,
|
||||
|
||||
@@ -0,0 +1,572 @@
|
||||
using System;
|
||||
using System.Globalization;
|
||||
using System.IO;
|
||||
using System.Linq;
|
||||
using GitHub.DistributedTask.ObjectTemplating;
|
||||
using GitHub.DistributedTask.ObjectTemplating.Tokens;
|
||||
using YamlDotNet.Core;
|
||||
using YamlDotNet.Core.Events;
|
||||
|
||||
namespace GitHub.DistributedTask.Pipelines.ObjectTemplating
|
||||
{
|
||||
/// <summary>
|
||||
/// Converts a YAML file into a TemplateToken
|
||||
/// </summary>
|
||||
public sealed class YamlObjectReader : IObjectReader
|
||||
{
|
||||
internal YamlObjectReader(
|
||||
Int32? fileId,
|
||||
TextReader input)
|
||||
{
|
||||
m_fileId = fileId;
|
||||
m_parser = new Parser(input);
|
||||
}
|
||||
|
||||
public Boolean AllowLiteral(out LiteralToken value)
|
||||
{
|
||||
if (EvaluateCurrent() is Scalar scalar)
|
||||
{
|
||||
// Tag specified
|
||||
if (!String.IsNullOrEmpty(scalar.Tag))
|
||||
{
|
||||
// String tag
|
||||
if (String.Equals(scalar.Tag, c_stringTag, StringComparison.Ordinal))
|
||||
{
|
||||
value = new StringToken(m_fileId, scalar.Start.Line, scalar.Start.Column, scalar.Value);
|
||||
MoveNext();
|
||||
return true;
|
||||
}
|
||||
|
||||
// Not plain style
|
||||
if (scalar.Style != ScalarStyle.Plain)
|
||||
{
|
||||
throw new NotSupportedException($"The scalar style '{scalar.Style}' on line {scalar.Start.Line} and column {scalar.Start.Column} is not valid with the tag '{scalar.Tag}'");
|
||||
}
|
||||
|
||||
// Boolean, Float, Integer, or Null
|
||||
switch (scalar.Tag)
|
||||
{
|
||||
case c_booleanTag:
|
||||
value = ParseBoolean(scalar);
|
||||
break;
|
||||
case c_floatTag:
|
||||
value = ParseFloat(scalar);
|
||||
break;
|
||||
case c_integerTag:
|
||||
value = ParseInteger(scalar);
|
||||
break;
|
||||
case c_nullTag:
|
||||
value = ParseNull(scalar);
|
||||
break;
|
||||
default:
|
||||
throw new NotSupportedException($"Unexpected tag '{scalar.Tag}'");
|
||||
}
|
||||
|
||||
MoveNext();
|
||||
return true;
|
||||
}
|
||||
|
||||
// Plain style, determine type using YAML 1.2 "core" schema https://yaml.org/spec/1.2/spec.html#id2804923
|
||||
if (scalar.Style == ScalarStyle.Plain)
|
||||
{
|
||||
if (MatchNull(scalar, out var nullToken))
|
||||
{
|
||||
value = nullToken;
|
||||
}
|
||||
else if (MatchBoolean(scalar, out var booleanToken))
|
||||
{
|
||||
value = booleanToken;
|
||||
}
|
||||
else if (MatchInteger(scalar, out var numberToken) ||
|
||||
MatchFloat(scalar, out numberToken))
|
||||
{
|
||||
value = numberToken;
|
||||
}
|
||||
else
|
||||
{
|
||||
value = new StringToken(m_fileId, scalar.Start.Line, scalar.Start.Column, scalar.Value);
|
||||
}
|
||||
|
||||
MoveNext();
|
||||
return true;
|
||||
}
|
||||
|
||||
// Otherwise assume string
|
||||
value = new StringToken(m_fileId, scalar.Start.Line, scalar.Start.Column, scalar.Value);
|
||||
MoveNext();
|
||||
return true;
|
||||
}
|
||||
|
||||
value = default;
|
||||
return false;
|
||||
}
|
||||
|
||||
public Boolean AllowSequenceStart(out SequenceToken value)
|
||||
{
|
||||
if (EvaluateCurrent() is SequenceStart sequenceStart)
|
||||
{
|
||||
value = new SequenceToken(m_fileId, sequenceStart.Start.Line, sequenceStart.Start.Column);
|
||||
MoveNext();
|
||||
return true;
|
||||
}
|
||||
|
||||
value = default;
|
||||
return false;
|
||||
}
|
||||
|
||||
public Boolean AllowSequenceEnd()
|
||||
{
|
||||
if (EvaluateCurrent() is SequenceEnd)
|
||||
{
|
||||
MoveNext();
|
||||
return true;
|
||||
}
|
||||
|
||||
return false;
|
||||
}
|
||||
|
||||
public Boolean AllowMappingStart(out MappingToken value)
|
||||
{
|
||||
if (EvaluateCurrent() is MappingStart mappingStart)
|
||||
{
|
||||
value = new MappingToken(m_fileId, mappingStart.Start.Line, mappingStart.Start.Column);
|
||||
MoveNext();
|
||||
return true;
|
||||
}
|
||||
|
||||
value = default;
|
||||
return false;
|
||||
}
|
||||
|
||||
public Boolean AllowMappingEnd()
|
||||
{
|
||||
if (EvaluateCurrent() is MappingEnd)
|
||||
{
|
||||
MoveNext();
|
||||
return true;
|
||||
}
|
||||
|
||||
return false;
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Consumes the last parsing events, which are expected to be DocumentEnd and StreamEnd.
|
||||
/// </summary>
|
||||
public void ValidateEnd()
|
||||
{
|
||||
if (EvaluateCurrent() is DocumentEnd)
|
||||
{
|
||||
MoveNext();
|
||||
}
|
||||
else
|
||||
{
|
||||
throw new InvalidOperationException("Expected document end parse event");
|
||||
}
|
||||
|
||||
if (EvaluateCurrent() is StreamEnd)
|
||||
{
|
||||
MoveNext();
|
||||
}
|
||||
else
|
||||
{
|
||||
throw new InvalidOperationException("Expected stream end parse event");
|
||||
}
|
||||
|
||||
if (MoveNext())
|
||||
{
|
||||
throw new InvalidOperationException("Expected end of parse events");
|
||||
}
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Consumes the first parsing events, which are expected to be StreamStart and DocumentStart.
|
||||
/// </summary>
|
||||
public void ValidateStart()
|
||||
{
|
||||
if (EvaluateCurrent() != null)
|
||||
{
|
||||
throw new InvalidOperationException("Unexpected parser state");
|
||||
}
|
||||
|
||||
if (!MoveNext())
|
||||
{
|
||||
throw new InvalidOperationException("Expected a parse event");
|
||||
}
|
||||
|
||||
if (EvaluateCurrent() is StreamStart)
|
||||
{
|
||||
MoveNext();
|
||||
}
|
||||
else
|
||||
{
|
||||
throw new InvalidOperationException("Expected stream start parse event");
|
||||
}
|
||||
|
||||
if (EvaluateCurrent() is DocumentStart)
|
||||
{
|
||||
MoveNext();
|
||||
}
|
||||
else
|
||||
{
|
||||
throw new InvalidOperationException("Expected document start parse event");
|
||||
}
|
||||
}
|
||||
|
||||
private ParsingEvent EvaluateCurrent()
|
||||
{
|
||||
if (m_current == null)
|
||||
{
|
||||
m_current = m_parser.Current;
|
||||
if (m_current != null)
|
||||
{
|
||||
if (m_current is Scalar scalar)
|
||||
{
|
||||
// Verify not using achors
|
||||
if (scalar.Anchor != null)
|
||||
{
|
||||
throw new InvalidOperationException($"Anchors are not currently supported. Remove the anchor '{scalar.Anchor}'");
|
||||
}
|
||||
}
|
||||
else if (m_current is MappingStart mappingStart)
|
||||
{
|
||||
// Verify not using achors
|
||||
if (mappingStart.Anchor != null)
|
||||
{
|
||||
throw new InvalidOperationException($"Anchors are not currently supported. Remove the anchor '{mappingStart.Anchor}'");
|
||||
}
|
||||
}
|
||||
else if (m_current is SequenceStart sequenceStart)
|
||||
{
|
||||
// Verify not using achors
|
||||
if (sequenceStart.Anchor != null)
|
||||
{
|
||||
throw new InvalidOperationException($"Anchors are not currently supported. Remove the anchor '{sequenceStart.Anchor}'");
|
||||
}
|
||||
}
|
||||
else if (!(m_current is MappingEnd) &&
|
||||
!(m_current is SequenceEnd) &&
|
||||
!(m_current is DocumentStart) &&
|
||||
!(m_current is DocumentEnd) &&
|
||||
!(m_current is StreamStart) &&
|
||||
!(m_current is StreamEnd))
|
||||
{
|
||||
throw new InvalidOperationException($"Unexpected parsing event type: {m_current.GetType().Name}");
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return m_current;
|
||||
}
|
||||
|
||||
private Boolean MoveNext()
|
||||
{
|
||||
m_current = null;
|
||||
return m_parser.MoveNext();
|
||||
}
|
||||
|
||||
private BooleanToken ParseBoolean(Scalar scalar)
|
||||
{
|
||||
if (MatchBoolean(scalar, out var token))
|
||||
{
|
||||
return token;
|
||||
}
|
||||
|
||||
ThrowInvalidValue(scalar, c_booleanTag); // throws
|
||||
return default;
|
||||
}
|
||||
|
||||
private NumberToken ParseFloat(Scalar scalar)
|
||||
{
|
||||
if (MatchFloat(scalar, out var token))
|
||||
{
|
||||
return token;
|
||||
}
|
||||
|
||||
ThrowInvalidValue(scalar, c_floatTag); // throws
|
||||
return default;
|
||||
}
|
||||
|
||||
private NumberToken ParseInteger(Scalar scalar)
|
||||
{
|
||||
if (MatchInteger(scalar, out var token))
|
||||
{
|
||||
return token;
|
||||
}
|
||||
|
||||
ThrowInvalidValue(scalar, c_integerTag); // throws
|
||||
return default;
|
||||
}
|
||||
|
||||
private NullToken ParseNull(Scalar scalar)
|
||||
{
|
||||
if (MatchNull(scalar, out var token))
|
||||
{
|
||||
return token;
|
||||
}
|
||||
|
||||
ThrowInvalidValue(scalar, c_nullTag); // throws
|
||||
return default;
|
||||
}
|
||||
|
||||
private Boolean MatchBoolean(
|
||||
Scalar scalar,
|
||||
out BooleanToken value)
|
||||
{
|
||||
// YAML 1.2 "core" schema https://yaml.org/spec/1.2/spec.html#id2804923
|
||||
switch (scalar.Value ?? String.Empty)
|
||||
{
|
||||
case "true":
|
||||
case "True":
|
||||
case "TRUE":
|
||||
value = new BooleanToken(m_fileId, scalar.Start.Line, scalar.Start.Column, true);
|
||||
return true;
|
||||
case "false":
|
||||
case "False":
|
||||
case "FALSE":
|
||||
value = new BooleanToken(m_fileId, scalar.Start.Line, scalar.Start.Column, false);
|
||||
return true;
|
||||
}
|
||||
|
||||
value = default;
|
||||
return false;
|
||||
}
|
||||
|
||||
private Boolean MatchFloat(
|
||||
Scalar scalar,
|
||||
out NumberToken value)
|
||||
{
|
||||
// YAML 1.2 "core" schema https://yaml.org/spec/1.2/spec.html#id2804923
|
||||
var str = scalar.Value;
|
||||
if (!String.IsNullOrEmpty(str))
|
||||
{
|
||||
// Check for [-+]?(\.inf|\.Inf|\.INF)|\.nan|\.NaN|\.NAN
|
||||
switch (str)
|
||||
{
|
||||
case ".inf":
|
||||
case ".Inf":
|
||||
case ".INF":
|
||||
case "+.inf":
|
||||
case "+.Inf":
|
||||
case "+.INF":
|
||||
value = new NumberToken(m_fileId, scalar.Start.Line, scalar.Start.Column, Double.PositiveInfinity);
|
||||
return true;
|
||||
case "-.inf":
|
||||
case "-.Inf":
|
||||
case "-.INF":
|
||||
value = new NumberToken(m_fileId, scalar.Start.Line, scalar.Start.Column, Double.NegativeInfinity);
|
||||
return true;
|
||||
case ".nan":
|
||||
case ".NaN":
|
||||
case ".NAN":
|
||||
value = new NumberToken(m_fileId, scalar.Start.Line, scalar.Start.Column, Double.NaN);
|
||||
return true;
|
||||
}
|
||||
|
||||
// Otherwise check [-+]?(\.[0-9]+|[0-9]+(\.[0-9]*)?)([eE][-+]?[0-9]+)?
|
||||
|
||||
// Skip leading sign
|
||||
var index = str[0] == '-' || str[0] == '+' ? 1 : 0;
|
||||
|
||||
// Check for integer portion
|
||||
var length = str.Length;
|
||||
var hasInteger = false;
|
||||
while (index < length && str[index] >= '0' && str[index] <= '9')
|
||||
{
|
||||
hasInteger = true;
|
||||
index++;
|
||||
}
|
||||
|
||||
// Check for decimal point
|
||||
var hasDot = false;
|
||||
if (index < length && str[index] == '.')
|
||||
{
|
||||
hasDot = true;
|
||||
index++;
|
||||
}
|
||||
|
||||
// Check for decimal portion
|
||||
var hasDecimal = false;
|
||||
while (index < length && str[index] >= '0' && str[index] <= '9')
|
||||
{
|
||||
hasDecimal = true;
|
||||
index++;
|
||||
}
|
||||
|
||||
// Check [-+]?(\.[0-9]+|[0-9]+(\.[0-9]*)?)
|
||||
if ((hasDot && hasDecimal) || hasInteger)
|
||||
{
|
||||
// Check for end
|
||||
if (index == length)
|
||||
{
|
||||
// Try parse
|
||||
if (Double.TryParse(str, NumberStyles.AllowLeadingSign | NumberStyles.AllowDecimalPoint, CultureInfo.InvariantCulture, out var doubleValue))
|
||||
{
|
||||
value = new NumberToken(m_fileId, scalar.Start.Line, scalar.Start.Column, doubleValue);
|
||||
return true;
|
||||
}
|
||||
// Otherwise exceeds range
|
||||
else
|
||||
{
|
||||
ThrowInvalidValue(scalar, c_floatTag); // throws
|
||||
}
|
||||
}
|
||||
// Check [eE][-+]?[0-9]
|
||||
else if (index < length && (str[index] == 'e' || str[index] == 'E'))
|
||||
{
|
||||
index++;
|
||||
|
||||
// Skip sign
|
||||
if (index < length && (str[index] == '-' || str[index] == '+'))
|
||||
{
|
||||
index++;
|
||||
}
|
||||
|
||||
// Check for exponent
|
||||
var hasExponent = false;
|
||||
while (index < length && str[index] >= '0' && str[index] <= '9')
|
||||
{
|
||||
hasExponent = true;
|
||||
index++;
|
||||
}
|
||||
|
||||
// Check for end
|
||||
if (hasExponent && index == length)
|
||||
{
|
||||
// Try parse
|
||||
if (Double.TryParse(str, NumberStyles.AllowLeadingSign | NumberStyles.AllowDecimalPoint | NumberStyles.AllowExponent, CultureInfo.InvariantCulture, out var doubleValue))
|
||||
{
|
||||
value = new NumberToken(m_fileId, scalar.Start.Line, scalar.Start.Column, (Double)doubleValue);
|
||||
return true;
|
||||
}
|
||||
// Otherwise exceeds range
|
||||
else
|
||||
{
|
||||
ThrowInvalidValue(scalar, c_floatTag); // throws
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
value = default;
|
||||
return false;
|
||||
}
|
||||
|
||||
private Boolean MatchInteger(
|
||||
Scalar scalar,
|
||||
out NumberToken value)
|
||||
{
|
||||
// YAML 1.2 "core" schema https://yaml.org/spec/1.2/spec.html#id2804923
|
||||
var str = scalar.Value;
|
||||
if (!String.IsNullOrEmpty(str))
|
||||
{
|
||||
// Check for [0-9]+
|
||||
var firstChar = str[0];
|
||||
if (firstChar >= '0' && firstChar <= '9' &&
|
||||
str.Skip(1).All(x => x >= '0' && x <= '9'))
|
||||
{
|
||||
// Try parse
|
||||
if (Double.TryParse(str, NumberStyles.None, CultureInfo.InvariantCulture, out var doubleValue))
|
||||
{
|
||||
value = new NumberToken(m_fileId, scalar.Start.Line, scalar.Start.Column, doubleValue);
|
||||
return true;
|
||||
}
|
||||
|
||||
// Otherwise exceeds range
|
||||
ThrowInvalidValue(scalar, c_integerTag); // throws
|
||||
}
|
||||
// Check for (-|+)[0-9]+
|
||||
else if ((firstChar == '-' || firstChar == '+') &&
|
||||
str.Length > 1 &&
|
||||
str.Skip(1).All(x => x >= '0' && x <= '9'))
|
||||
{
|
||||
// Try parse
|
||||
if (Double.TryParse(str, NumberStyles.AllowLeadingSign, CultureInfo.InvariantCulture, out var doubleValue))
|
||||
{
|
||||
value = new NumberToken(m_fileId, scalar.Start.Line, scalar.Start.Column, doubleValue);
|
||||
return true;
|
||||
}
|
||||
|
||||
// Otherwise exceeds range
|
||||
ThrowInvalidValue(scalar, c_integerTag); // throws
|
||||
}
|
||||
// Check for 0x[0-9a-fA-F]+
|
||||
else if (firstChar == '0' &&
|
||||
str.Length > 2 &&
|
||||
str[1] == 'x' &&
|
||||
str.Skip(2).All(x => (x >= '0' && x <= '9') || (x >= 'a' && x <= 'f') || (x >= 'A' && x <= 'F')))
|
||||
{
|
||||
// Try parse
|
||||
if (Int32.TryParse(str.Substring(2), NumberStyles.AllowHexSpecifier, CultureInfo.InvariantCulture, out var integerValue))
|
||||
{
|
||||
value = new NumberToken(m_fileId, scalar.Start.Line, scalar.Start.Column, integerValue);
|
||||
return true;
|
||||
}
|
||||
|
||||
// Otherwise exceeds range
|
||||
ThrowInvalidValue(scalar, c_integerTag); // throws
|
||||
}
|
||||
// Check for 0o[0-9]+
|
||||
else if (firstChar == '0' &&
|
||||
str.Length > 2 &&
|
||||
str[1] == 'o' &&
|
||||
str.Skip(2).All(x => x >= '0' && x <= '7'))
|
||||
{
|
||||
// Try parse
|
||||
var integerValue = default(Int32);
|
||||
try
|
||||
{
|
||||
integerValue = Convert.ToInt32(str.Substring(2), 8);
|
||||
}
|
||||
// Otherwise exceeds range
|
||||
catch (Exception)
|
||||
{
|
||||
ThrowInvalidValue(scalar, c_integerTag); // throws
|
||||
}
|
||||
|
||||
value = new NumberToken(m_fileId, scalar.Start.Line, scalar.Start.Column, integerValue);
|
||||
return true;
|
||||
}
|
||||
}
|
||||
|
||||
value = default;
|
||||
return false;
|
||||
}
|
||||
|
||||
private Boolean MatchNull(
|
||||
Scalar scalar,
|
||||
out NullToken value)
|
||||
{
|
||||
// YAML 1.2 "core" schema https://yaml.org/spec/1.2/spec.html#id2804923
|
||||
switch (scalar.Value ?? String.Empty)
|
||||
{
|
||||
case "":
|
||||
case "null":
|
||||
case "Null":
|
||||
case "NULL":
|
||||
case "~":
|
||||
value = new NullToken(m_fileId, scalar.Start.Line, scalar.Start.Column);
|
||||
return true;
|
||||
}
|
||||
|
||||
value = default;
|
||||
return false;
|
||||
}
|
||||
|
||||
private void ThrowInvalidValue(
|
||||
Scalar scalar,
|
||||
String tag)
|
||||
{
|
||||
throw new NotSupportedException($"The value '{scalar.Value}' on line {scalar.Start.Line} and column {scalar.Start.Column} is invalid for the type '{scalar.Tag}'");
|
||||
}
|
||||
|
||||
private const String c_booleanTag = "tag:yaml.org,2002:bool";
|
||||
private const String c_floatTag = "tag:yaml.org,2002:float";
|
||||
private const String c_integerTag = "tag:yaml.org,2002:int";
|
||||
private const String c_nullTag = "tag:yaml.org,2002:null";
|
||||
private const String c_stringTag = "tag:yaml.org,2002:string";
|
||||
private readonly Int32? m_fileId;
|
||||
private readonly Parser m_parser;
|
||||
private ParsingEvent m_current;
|
||||
}
|
||||
}
|
||||
@@ -4,17 +4,6 @@ using System.Runtime.Serialization;
|
||||
|
||||
namespace GitHub.DistributedTask.WebApi
|
||||
{
|
||||
|
||||
[DataContract]
|
||||
public class GitHubToken
|
||||
{
|
||||
[DataMember(EmitDefaultValue = false)]
|
||||
public string Token { get; set; }
|
||||
|
||||
[DataMember(EmitDefaultValue = false)]
|
||||
public DateTime Expires_at { get; set; }
|
||||
}
|
||||
|
||||
[DataContract]
|
||||
public class Issue
|
||||
{
|
||||
|
||||
@@ -33,7 +33,7 @@ namespace GitHub.Runner.Common.Tests.Listener
|
||||
TaskOrchestrationPlanReference plan = new TaskOrchestrationPlanReference();
|
||||
TimelineReference timeline = null;
|
||||
Guid jobId = Guid.NewGuid();
|
||||
var result = new Pipelines.AgentJobRequestMessage(plan, timeline, jobId, "someJob", "someJob", null, null, null, new Dictionary<string, VariableValue>(), new List<MaskHint>(), new Pipelines.JobResources(), new Pipelines.ContextData.DictionaryContextData(), new Pipelines.WorkspaceOptions(), new List<Pipelines.ActionStep>(), null);
|
||||
var result = new Pipelines.AgentJobRequestMessage(plan, timeline, jobId, "someJob", "someJob", null, null, null, new Dictionary<string, VariableValue>(), new List<MaskHint>(), new Pipelines.JobResources(), new Pipelines.ContextData.DictionaryContextData(), new Pipelines.WorkspaceOptions(), new List<Pipelines.ActionStep>(), null, null);
|
||||
result.ContextData["github"] = new Pipelines.ContextData.DictionaryContextData();
|
||||
return result;
|
||||
}
|
||||
|
||||
@@ -43,7 +43,7 @@ namespace GitHub.Runner.Common.Tests.Listener
|
||||
TaskOrchestrationPlanReference plan = new TaskOrchestrationPlanReference();
|
||||
TimelineReference timeline = null;
|
||||
Guid jobId = Guid.NewGuid();
|
||||
return new Pipelines.AgentJobRequestMessage(plan, timeline, jobId, "test", "test", null, null, null, new Dictionary<string, VariableValue>(), new List<MaskHint>(), new Pipelines.JobResources(), new Pipelines.ContextData.DictionaryContextData(), new Pipelines.WorkspaceOptions(), new List<Pipelines.ActionStep>(), null);
|
||||
return new Pipelines.AgentJobRequestMessage(plan, timeline, jobId, "test", "test", null, null, null, new Dictionary<string, VariableValue>(), new List<MaskHint>(), new Pipelines.JobResources(), new Pipelines.ContextData.DictionaryContextData(), new Pipelines.WorkspaceOptions(), new List<Pipelines.ActionStep>(), null, null);
|
||||
}
|
||||
|
||||
private JobCancelMessage CreateJobCancelMessage()
|
||||
|
||||
@@ -150,7 +150,7 @@ namespace GitHub.Runner.Common.Tests.Worker
|
||||
TimelineReference timeline = new TimelineReference();
|
||||
Guid jobId = Guid.NewGuid();
|
||||
string jobName = "some job name";
|
||||
var jobRequest = new Pipelines.AgentJobRequestMessage(plan, timeline, jobId, jobName, jobName, null, null, null, new Dictionary<string, VariableValue>(), new List<MaskHint>(), new Pipelines.JobResources(), new Pipelines.ContextData.DictionaryContextData(), new Pipelines.WorkspaceOptions(), new List<Pipelines.ActionStep>(), null);
|
||||
var jobRequest = new Pipelines.AgentJobRequestMessage(plan, timeline, jobId, jobName, jobName, null, null, null, new Dictionary<string, VariableValue>(), new List<MaskHint>(), new Pipelines.JobResources(), new Pipelines.ContextData.DictionaryContextData(), new Pipelines.WorkspaceOptions(), new List<Pipelines.ActionStep>(), null, null);
|
||||
jobRequest.Resources.Repositories.Add(new Pipelines.RepositoryResource()
|
||||
{
|
||||
Alias = Pipelines.PipelineConstants.SelfAlias,
|
||||
|
||||
@@ -54,7 +54,7 @@ namespace GitHub.Runner.Common.Tests.Worker
|
||||
};
|
||||
|
||||
//Act
|
||||
var steps = (await _actionManager.PrepareActionsAsync(_ec.Object, actions)).ContainerSetupSteps;
|
||||
var steps = await _actionManager.PrepareActionsAsync(_ec.Object, actions);
|
||||
|
||||
//Assert
|
||||
Assert.Equal(actionId, (steps[0].Data as ContainerSetupInfo).StepIds[0]);
|
||||
@@ -164,7 +164,7 @@ namespace GitHub.Runner.Common.Tests.Worker
|
||||
};
|
||||
|
||||
//Act
|
||||
var steps = (await _actionManager.PrepareActionsAsync(_ec.Object, actions)).ContainerSetupSteps;
|
||||
var steps = await _actionManager.PrepareActionsAsync(_ec.Object, actions);
|
||||
|
||||
Assert.True(steps.Count == 0);
|
||||
}
|
||||
@@ -203,7 +203,7 @@ namespace GitHub.Runner.Common.Tests.Worker
|
||||
var actionDir = Path.Combine(_hc.GetDirectory(WellKnownDirectory.Actions), "TingluoHuang", "runner_L0", "repositoryactionwithdockerfile");
|
||||
|
||||
//Act
|
||||
var steps = (await _actionManager.PrepareActionsAsync(_ec.Object, actions)).ContainerSetupSteps;
|
||||
var steps = await _actionManager.PrepareActionsAsync(_ec.Object, actions);
|
||||
Assert.Equal(actionId, (steps[0].Data as ContainerSetupInfo).StepIds[0]);
|
||||
Assert.Equal(actionDir, (steps[0].Data as ContainerSetupInfo).Container.WorkingDirectory);
|
||||
Assert.Equal(Path.Combine(actionDir, "Dockerfile"), (steps[0].Data as ContainerSetupInfo).Container.Dockerfile);
|
||||
@@ -243,7 +243,7 @@ namespace GitHub.Runner.Common.Tests.Worker
|
||||
var actionDir = Path.Combine(_hc.GetDirectory(WellKnownDirectory.Actions), "TingluoHuang", "runner_L0", "repositoryactionwithdockerfileinrelativepath");
|
||||
|
||||
//Act
|
||||
var steps = (await _actionManager.PrepareActionsAsync(_ec.Object, actions)).ContainerSetupSteps;
|
||||
var steps = await _actionManager.PrepareActionsAsync(_ec.Object, actions);
|
||||
|
||||
Assert.Equal(actionId, (steps[0].Data as ContainerSetupInfo).StepIds[0]);
|
||||
Assert.Equal(actionDir, (steps[0].Data as ContainerSetupInfo).Container.WorkingDirectory);
|
||||
@@ -282,7 +282,7 @@ namespace GitHub.Runner.Common.Tests.Worker
|
||||
|
||||
var actionDir = Path.Combine(_hc.GetDirectory(WellKnownDirectory.Actions), "TingluoHuang", "runner_L0", "repositoryactionwithdockerfileinrelativepath");
|
||||
//Act
|
||||
var steps = (await _actionManager.PrepareActionsAsync(_ec.Object, actions)).ContainerSetupSteps;
|
||||
var steps = await _actionManager.PrepareActionsAsync(_ec.Object, actions);
|
||||
|
||||
Assert.Equal(actionId, (steps[0].Data as ContainerSetupInfo).StepIds[0]);
|
||||
Assert.Equal(actionDir, (steps[0].Data as ContainerSetupInfo).Container.WorkingDirectory);
|
||||
@@ -322,7 +322,7 @@ namespace GitHub.Runner.Common.Tests.Worker
|
||||
var actionDir = Path.Combine(_hc.GetDirectory(WellKnownDirectory.Actions), "TingluoHuang", "runner_L0", "RepositoryActionWithActionfile_DockerfileRelativePath");
|
||||
|
||||
//Act
|
||||
var steps = (await _actionManager.PrepareActionsAsync(_ec.Object, actions)).ContainerSetupSteps;
|
||||
var steps = await _actionManager.PrepareActionsAsync(_ec.Object, actions);
|
||||
|
||||
Assert.Equal(actionId, (steps[0].Data as ContainerSetupInfo).StepIds[0]);
|
||||
Assert.Equal(actionDir, (steps[0].Data as ContainerSetupInfo).Container.WorkingDirectory);
|
||||
@@ -362,7 +362,7 @@ namespace GitHub.Runner.Common.Tests.Worker
|
||||
var actionDir = Path.Combine(_hc.GetDirectory(WellKnownDirectory.Actions), "TingluoHuang", "runner_L0", "RepositoryActionWithActionfile_DockerHubImage");
|
||||
|
||||
//Act
|
||||
var steps = (await _actionManager.PrepareActionsAsync(_ec.Object, actions)).ContainerSetupSteps;
|
||||
var steps = await _actionManager.PrepareActionsAsync(_ec.Object, actions);
|
||||
|
||||
Assert.Equal(actionId, (steps[0].Data as ContainerSetupInfo).StepIds[0]);
|
||||
Assert.Equal("ubuntu:18.04", (steps[0].Data as ContainerSetupInfo).Container.Image);
|
||||
@@ -401,7 +401,7 @@ namespace GitHub.Runner.Common.Tests.Worker
|
||||
var actionDir = Path.Combine(_hc.GetDirectory(WellKnownDirectory.Actions), "TingluoHuang", "runner_L0", "RepositoryActionWithActionYamlFile_DockerHubImage");
|
||||
|
||||
//Act
|
||||
var steps = (await _actionManager.PrepareActionsAsync(_ec.Object, actions)).ContainerSetupSteps;
|
||||
var steps = await _actionManager.PrepareActionsAsync(_ec.Object, actions);
|
||||
|
||||
Assert.Equal((steps[0].Data as ContainerSetupInfo).StepIds[0], actionId);
|
||||
Assert.Equal("ubuntu:18.04", (steps[0].Data as ContainerSetupInfo).Container.Image);
|
||||
@@ -440,7 +440,7 @@ namespace GitHub.Runner.Common.Tests.Worker
|
||||
var actionDir = Path.Combine(_hc.GetDirectory(WellKnownDirectory.Actions), "TingluoHuang", "runner_L0", "repositoryactionwithactionfileanddockerfile");
|
||||
|
||||
//Act
|
||||
var steps = (await _actionManager.PrepareActionsAsync(_ec.Object, actions)).ContainerSetupSteps;
|
||||
var steps = await _actionManager.PrepareActionsAsync(_ec.Object, actions);
|
||||
|
||||
Assert.Equal(actionId, (steps[0].Data as ContainerSetupInfo).StepIds[0]);
|
||||
Assert.Equal(actionDir, (steps[0].Data as ContainerSetupInfo).Container.WorkingDirectory);
|
||||
@@ -557,7 +557,7 @@ namespace GitHub.Runner.Common.Tests.Worker
|
||||
};
|
||||
|
||||
//Act
|
||||
var steps = (await _actionManager.PrepareActionsAsync(_ec.Object, actions)).ContainerSetupSteps;
|
||||
var steps = await _actionManager.PrepareActionsAsync(_ec.Object, actions);
|
||||
|
||||
//Assert
|
||||
Assert.Equal(actionId1, (steps[0].Data as ContainerSetupInfo).StepIds[0]);
|
||||
@@ -618,7 +618,7 @@ namespace GitHub.Runner.Common.Tests.Worker
|
||||
};
|
||||
|
||||
//Act
|
||||
var steps = (await _actionManager.PrepareActionsAsync(_ec.Object, actions)).ContainerSetupSteps;
|
||||
var steps = await _actionManager.PrepareActionsAsync(_ec.Object, actions);
|
||||
|
||||
// node.js based action doesn't need any extra steps to build/pull containers.
|
||||
Assert.True(steps.Count == 0);
|
||||
@@ -629,47 +629,6 @@ namespace GitHub.Runner.Common.Tests.Worker
|
||||
}
|
||||
}
|
||||
|
||||
[Fact]
|
||||
[Trait("Level", "L0")]
|
||||
[Trait("Category", "Worker")]
|
||||
public async void PrepareActions_RepositoryActionWithInvalidWrapperActionfile_Node()
|
||||
{
|
||||
try
|
||||
{
|
||||
//Arrange
|
||||
Setup();
|
||||
var actionId = Guid.NewGuid();
|
||||
var actions = new List<Pipelines.ActionStep>
|
||||
{
|
||||
new Pipelines.ActionStep()
|
||||
{
|
||||
Name = "action",
|
||||
Id = actionId,
|
||||
Reference = new Pipelines.RepositoryPathReference()
|
||||
{
|
||||
Name = "TingluoHuang/runner_L0",
|
||||
Ref = "RepositoryActionWithInvalidWrapperActionfile_Node",
|
||||
RepositoryType = "GitHub"
|
||||
}
|
||||
}
|
||||
};
|
||||
|
||||
//Act
|
||||
try
|
||||
{
|
||||
await _actionManager.PrepareActionsAsync(_ec.Object, actions);
|
||||
}
|
||||
catch (ArgumentNullException e)
|
||||
{
|
||||
Assert.Contains("Entry javascript fils is not provided.", e.Message);
|
||||
}
|
||||
}
|
||||
finally
|
||||
{
|
||||
Teardown();
|
||||
}
|
||||
}
|
||||
|
||||
[Fact]
|
||||
[Trait("Level", "L0")]
|
||||
[Trait("Category", "Worker")]
|
||||
|
||||
@@ -63,52 +63,6 @@ namespace GitHub.Runner.Common.Tests.Worker
|
||||
}
|
||||
}
|
||||
|
||||
[Fact]
|
||||
[Trait("Level", "L0")]
|
||||
[Trait("Category", "Worker")]
|
||||
public void Load_ContainerAction_Dockerfile_Pre()
|
||||
{
|
||||
try
|
||||
{
|
||||
//Arrange
|
||||
Setup();
|
||||
|
||||
var actionManifest = new ActionManifestManager();
|
||||
actionManifest.Initialize(_hc);
|
||||
|
||||
//Act
|
||||
var result = actionManifest.Load(_ec.Object, Path.Combine(TestUtil.GetTestDataPath(), "dockerfileaction_init.yml"));
|
||||
|
||||
//Assert
|
||||
|
||||
Assert.Equal("Hello World", result.Name);
|
||||
Assert.Equal("Greet the world and record the time", result.Description);
|
||||
Assert.Equal(2, result.Inputs.Count);
|
||||
Assert.Equal("greeting", result.Inputs[0].Key.AssertString("key").Value);
|
||||
Assert.Equal("Hello", result.Inputs[0].Value.AssertString("value").Value);
|
||||
Assert.Equal("entryPoint", result.Inputs[1].Key.AssertString("key").Value);
|
||||
Assert.Equal("", result.Inputs[1].Value.AssertString("value").Value);
|
||||
|
||||
Assert.Equal(ActionExecutionType.Container, result.Execution.ExecutionType);
|
||||
|
||||
var containerAction = result.Execution as ContainerActionExecutionData;
|
||||
|
||||
Assert.Equal("Dockerfile", containerAction.Image);
|
||||
Assert.Equal("main.sh", containerAction.EntryPoint);
|
||||
Assert.Equal("init.sh", containerAction.Init);
|
||||
Assert.Equal("success()", containerAction.InitCondition);
|
||||
Assert.Equal("bzz", containerAction.Arguments[0].ToString());
|
||||
Assert.Equal("Token", containerAction.Environment[0].Key.ToString());
|
||||
Assert.Equal("foo", containerAction.Environment[0].Value.ToString());
|
||||
Assert.Equal("Url", containerAction.Environment[1].Key.ToString());
|
||||
Assert.Equal("bar", containerAction.Environment[1].Value.ToString());
|
||||
}
|
||||
finally
|
||||
{
|
||||
Teardown();
|
||||
}
|
||||
}
|
||||
|
||||
[Fact]
|
||||
[Trait("Level", "L0")]
|
||||
[Trait("Category", "Worker")]
|
||||
@@ -155,52 +109,6 @@ namespace GitHub.Runner.Common.Tests.Worker
|
||||
}
|
||||
}
|
||||
|
||||
[Fact]
|
||||
[Trait("Level", "L0")]
|
||||
[Trait("Category", "Worker")]
|
||||
public void Load_ContainerAction_Dockerfile_Pre_DefaultCondition()
|
||||
{
|
||||
try
|
||||
{
|
||||
//Arrange
|
||||
Setup();
|
||||
|
||||
var actionManifest = new ActionManifestManager();
|
||||
actionManifest.Initialize(_hc);
|
||||
|
||||
//Act
|
||||
var result = actionManifest.Load(_ec.Object, Path.Combine(TestUtil.GetTestDataPath(), "dockerfileaction_init_default.yml"));
|
||||
|
||||
//Assert
|
||||
|
||||
Assert.Equal("Hello World", result.Name);
|
||||
Assert.Equal("Greet the world and record the time", result.Description);
|
||||
Assert.Equal(2, result.Inputs.Count);
|
||||
Assert.Equal("greeting", result.Inputs[0].Key.AssertString("key").Value);
|
||||
Assert.Equal("Hello", result.Inputs[0].Value.AssertString("value").Value);
|
||||
Assert.Equal("entryPoint", result.Inputs[1].Key.AssertString("key").Value);
|
||||
Assert.Equal("", result.Inputs[1].Value.AssertString("value").Value);
|
||||
|
||||
Assert.Equal(ActionExecutionType.Container, result.Execution.ExecutionType);
|
||||
|
||||
var containerAction = result.Execution as ContainerActionExecutionData;
|
||||
|
||||
Assert.Equal("Dockerfile", containerAction.Image);
|
||||
Assert.Equal("main.sh", containerAction.EntryPoint);
|
||||
Assert.Equal("init.sh", containerAction.Init);
|
||||
Assert.Equal("always()", containerAction.InitCondition);
|
||||
Assert.Equal("bzz", containerAction.Arguments[0].ToString());
|
||||
Assert.Equal("Token", containerAction.Environment[0].Key.ToString());
|
||||
Assert.Equal("foo", containerAction.Environment[0].Value.ToString());
|
||||
Assert.Equal("Url", containerAction.Environment[1].Key.ToString());
|
||||
Assert.Equal("bar", containerAction.Environment[1].Value.ToString());
|
||||
}
|
||||
finally
|
||||
{
|
||||
Teardown();
|
||||
}
|
||||
}
|
||||
|
||||
[Fact]
|
||||
[Trait("Level", "L0")]
|
||||
[Trait("Category", "Worker")]
|
||||
@@ -413,94 +321,6 @@ namespace GitHub.Runner.Common.Tests.Worker
|
||||
}
|
||||
}
|
||||
|
||||
[Fact]
|
||||
[Trait("Level", "L0")]
|
||||
[Trait("Category", "Worker")]
|
||||
public void Load_NodeAction_Pre()
|
||||
{
|
||||
try
|
||||
{
|
||||
//Arrange
|
||||
Setup();
|
||||
|
||||
var actionManifest = new ActionManifestManager();
|
||||
actionManifest.Initialize(_hc);
|
||||
|
||||
//Act
|
||||
var result = actionManifest.Load(_ec.Object, Path.Combine(TestUtil.GetTestDataPath(), "nodeaction_init.yml"));
|
||||
|
||||
//Assert
|
||||
Assert.Equal("Hello World", result.Name);
|
||||
Assert.Equal("Greet the world and record the time", result.Description);
|
||||
Assert.Equal(2, result.Inputs.Count);
|
||||
Assert.Equal("greeting", result.Inputs[0].Key.AssertString("key").Value);
|
||||
Assert.Equal("Hello", result.Inputs[0].Value.AssertString("value").Value);
|
||||
Assert.Equal("entryPoint", result.Inputs[1].Key.AssertString("key").Value);
|
||||
Assert.Equal("", result.Inputs[1].Value.AssertString("value").Value);
|
||||
Assert.Equal(1, result.Deprecated.Count);
|
||||
|
||||
Assert.True(result.Deprecated.ContainsKey("greeting"));
|
||||
result.Deprecated.TryGetValue("greeting", out string value);
|
||||
Assert.Equal("This property has been deprecated", value);
|
||||
|
||||
Assert.Equal(ActionExecutionType.NodeJS, result.Execution.ExecutionType);
|
||||
|
||||
var nodeAction = result.Execution as NodeJSActionExecutionData;
|
||||
|
||||
Assert.Equal("main.js", nodeAction.Script);
|
||||
Assert.Equal("init.js", nodeAction.Init);
|
||||
Assert.Equal("cancelled()", nodeAction.InitCondition);
|
||||
}
|
||||
finally
|
||||
{
|
||||
Teardown();
|
||||
}
|
||||
}
|
||||
|
||||
[Fact]
|
||||
[Trait("Level", "L0")]
|
||||
[Trait("Category", "Worker")]
|
||||
public void Load_NodeAction_Init_DefaultCondition()
|
||||
{
|
||||
try
|
||||
{
|
||||
//Arrange
|
||||
Setup();
|
||||
|
||||
var actionManifest = new ActionManifestManager();
|
||||
actionManifest.Initialize(_hc);
|
||||
|
||||
//Act
|
||||
var result = actionManifest.Load(_ec.Object, Path.Combine(TestUtil.GetTestDataPath(), "nodeaction_init_default.yml"));
|
||||
|
||||
//Assert
|
||||
Assert.Equal("Hello World", result.Name);
|
||||
Assert.Equal("Greet the world and record the time", result.Description);
|
||||
Assert.Equal(2, result.Inputs.Count);
|
||||
Assert.Equal("greeting", result.Inputs[0].Key.AssertString("key").Value);
|
||||
Assert.Equal("Hello", result.Inputs[0].Value.AssertString("value").Value);
|
||||
Assert.Equal("entryPoint", result.Inputs[1].Key.AssertString("key").Value);
|
||||
Assert.Equal("", result.Inputs[1].Value.AssertString("value").Value);
|
||||
Assert.Equal(1, result.Deprecated.Count);
|
||||
|
||||
Assert.True(result.Deprecated.ContainsKey("greeting"));
|
||||
result.Deprecated.TryGetValue("greeting", out string value);
|
||||
Assert.Equal("This property has been deprecated", value);
|
||||
|
||||
Assert.Equal(ActionExecutionType.NodeJS, result.Execution.ExecutionType);
|
||||
|
||||
var nodeAction = result.Execution as NodeJSActionExecutionData;
|
||||
|
||||
Assert.Equal("main.js", nodeAction.Script);
|
||||
Assert.Equal("init.js", nodeAction.Init);
|
||||
Assert.Equal("always()", nodeAction.InitCondition);
|
||||
}
|
||||
finally
|
||||
{
|
||||
Teardown();
|
||||
}
|
||||
}
|
||||
|
||||
[Fact]
|
||||
[Trait("Level", "L0")]
|
||||
[Trait("Category", "Worker")]
|
||||
|
||||
@@ -25,7 +25,7 @@ namespace GitHub.Runner.Common.Tests.Worker
|
||||
TimelineReference timeline = new TimelineReference();
|
||||
Guid jobId = Guid.NewGuid();
|
||||
string jobName = "some job name";
|
||||
var jobRequest = new Pipelines.AgentJobRequestMessage(plan, timeline, jobId, jobName, jobName, null, null, null, new Dictionary<string, VariableValue>(), new List<MaskHint>(), new Pipelines.JobResources(), new Pipelines.ContextData.DictionaryContextData(), new Pipelines.WorkspaceOptions(), new List<Pipelines.ActionStep>(), null);
|
||||
var jobRequest = new Pipelines.AgentJobRequestMessage(plan, timeline, jobId, jobName, jobName, null, null, null, new Dictionary<string, VariableValue>(), new List<MaskHint>(), new Pipelines.JobResources(), new Pipelines.ContextData.DictionaryContextData(), new Pipelines.WorkspaceOptions(), new List<Pipelines.ActionStep>(), null, null);
|
||||
jobRequest.Resources.Repositories.Add(new Pipelines.RepositoryResource()
|
||||
{
|
||||
Alias = Pipelines.PipelineConstants.SelfAlias,
|
||||
@@ -101,7 +101,7 @@ namespace GitHub.Runner.Common.Tests.Worker
|
||||
TimelineReference timeline = new TimelineReference();
|
||||
Guid jobId = Guid.NewGuid();
|
||||
string jobName = "some job name";
|
||||
var jobRequest = new Pipelines.AgentJobRequestMessage(plan, timeline, jobId, jobName, jobName, null, null, null, new Dictionary<string, VariableValue>(), new List<MaskHint>(), new Pipelines.JobResources(), new Pipelines.ContextData.DictionaryContextData(), new Pipelines.WorkspaceOptions(), new List<Pipelines.ActionStep>(), null);
|
||||
var jobRequest = new Pipelines.AgentJobRequestMessage(plan, timeline, jobId, jobName, jobName, null, null, null, new Dictionary<string, VariableValue>(), new List<MaskHint>(), new Pipelines.JobResources(), new Pipelines.ContextData.DictionaryContextData(), new Pipelines.WorkspaceOptions(), new List<Pipelines.ActionStep>(), null, null);
|
||||
jobRequest.Resources.Repositories.Add(new Pipelines.RepositoryResource()
|
||||
{
|
||||
Alias = Pipelines.PipelineConstants.SelfAlias,
|
||||
@@ -152,7 +152,7 @@ namespace GitHub.Runner.Common.Tests.Worker
|
||||
TimelineReference timeline = new TimelineReference();
|
||||
Guid jobId = Guid.NewGuid();
|
||||
string jobName = "some job name";
|
||||
var jobRequest = new Pipelines.AgentJobRequestMessage(plan, timeline, jobId, jobName, jobName, null, null, null, new Dictionary<string, VariableValue>(), new List<MaskHint>(), new Pipelines.JobResources(), new Pipelines.ContextData.DictionaryContextData(), new Pipelines.WorkspaceOptions(), new List<Pipelines.ActionStep>(), null);
|
||||
var jobRequest = new Pipelines.AgentJobRequestMessage(plan, timeline, jobId, jobName, jobName, null, null, null, new Dictionary<string, VariableValue>(), new List<MaskHint>(), new Pipelines.JobResources(), new Pipelines.ContextData.DictionaryContextData(), new Pipelines.WorkspaceOptions(), new List<Pipelines.ActionStep>(), null, null);
|
||||
jobRequest.Resources.Repositories.Add(new Pipelines.RepositoryResource()
|
||||
{
|
||||
Alias = Pipelines.PipelineConstants.SelfAlias,
|
||||
@@ -199,20 +199,20 @@ namespace GitHub.Runner.Common.Tests.Worker
|
||||
|
||||
|
||||
var postRunner1 = hc.CreateService<IActionRunner>();
|
||||
postRunner1.Action = new Pipelines.ActionStep() { Id = Guid.NewGuid(), Name = "post1", DisplayName = "Test 1", Reference = new Pipelines.RepositoryPathReference() { Name = "actions/action" } };
|
||||
postRunner1.Action = new Pipelines.ActionStep() { Name = "post1", DisplayName = "Test 1", Reference = new Pipelines.RepositoryPathReference() { Name = "actions/action" } };
|
||||
postRunner1.Stage = ActionRunStage.Post;
|
||||
postRunner1.Condition = "always()";
|
||||
postRunner1.DisplayName = "post1";
|
||||
|
||||
|
||||
var postRunner2 = hc.CreateService<IActionRunner>();
|
||||
postRunner2.Action = new Pipelines.ActionStep() { Id = Guid.NewGuid(), Name = "post2", DisplayName = "Test 2", Reference = new Pipelines.RepositoryPathReference() { Name = "actions/action" } };
|
||||
postRunner2.Action = new Pipelines.ActionStep() { Name = "post2", DisplayName = "Test 2", Reference = new Pipelines.RepositoryPathReference() { Name = "actions/action" } };
|
||||
postRunner2.Stage = ActionRunStage.Post;
|
||||
postRunner2.Condition = "always()";
|
||||
postRunner2.DisplayName = "post2";
|
||||
|
||||
action1.RegisterPostJobStep(postRunner1);
|
||||
action2.RegisterPostJobStep(postRunner2);
|
||||
action1.RegisterPostJobStep("post1", postRunner1);
|
||||
action2.RegisterPostJobStep("post2", postRunner2);
|
||||
|
||||
Assert.NotNull(jobContext.JobSteps);
|
||||
Assert.NotNull(jobContext.PostJobSteps);
|
||||
|
||||
@@ -100,7 +100,7 @@ namespace GitHub.Runner.Common.Tests.Worker
|
||||
};
|
||||
|
||||
Guid jobId = Guid.NewGuid();
|
||||
_message = new Pipelines.AgentJobRequestMessage(plan, timeline, jobId, "test", "test", null, null, null, new Dictionary<string, VariableValue>(), new List<MaskHint>(), new Pipelines.JobResources(), new Pipelines.ContextData.DictionaryContextData(), new Pipelines.WorkspaceOptions(), steps, null);
|
||||
_message = new Pipelines.AgentJobRequestMessage(plan, timeline, jobId, "test", "test", null, null, null, new Dictionary<string, VariableValue>(), new List<MaskHint>(), new Pipelines.JobResources(), new Pipelines.ContextData.DictionaryContextData(), new Pipelines.WorkspaceOptions(), steps, null, null);
|
||||
GitHubContext github = new GitHubContext();
|
||||
github["repository"] = new Pipelines.ContextData.StringContextData("actions/runner");
|
||||
_message.ContextData.Add("github", github);
|
||||
@@ -144,7 +144,7 @@ namespace GitHub.Runner.Common.Tests.Worker
|
||||
jobExtension.Initialize(hc);
|
||||
|
||||
_actionManager.Setup(x => x.PrepareActionsAsync(It.IsAny<IExecutionContext>(), It.IsAny<IEnumerable<Pipelines.JobStep>>()))
|
||||
.Returns(Task.FromResult(new PrepareResult(new List<JobExtensionRunner>(), new Dictionary<Guid, IActionRunner>())));
|
||||
.Returns(Task.FromResult(new List<JobExtensionRunner>()));
|
||||
|
||||
List<IStep> result = await jobExtension.InitializeJob(_jobEc, _message);
|
||||
|
||||
@@ -179,7 +179,7 @@ namespace GitHub.Runner.Common.Tests.Worker
|
||||
jobExtension.Initialize(hc);
|
||||
|
||||
_actionManager.Setup(x => x.PrepareActionsAsync(It.IsAny<IExecutionContext>(), It.IsAny<IEnumerable<Pipelines.JobStep>>()))
|
||||
.Returns(Task.FromResult(new PrepareResult(new List<JobExtensionRunner>() { new JobExtensionRunner(null, "", "prepare1", null), new JobExtensionRunner(null, "", "prepare2", null) }, new Dictionary<Guid, IActionRunner>())));
|
||||
.Returns(Task.FromResult(new List<JobExtensionRunner>() { new JobExtensionRunner(null, "", "prepare1", null), new JobExtensionRunner(null, "", "prepare2", null) }));
|
||||
|
||||
List<IStep> result = await jobExtension.InitializeJob(_jobEc, _message);
|
||||
|
||||
|
||||
@@ -63,7 +63,7 @@ namespace GitHub.Runner.Common.Tests.Worker
|
||||
TaskOrchestrationPlanReference plan = new TaskOrchestrationPlanReference();
|
||||
TimelineReference timeline = new Timeline(Guid.NewGuid());
|
||||
Guid jobId = Guid.NewGuid();
|
||||
_message = new Pipelines.AgentJobRequestMessage(plan, timeline, jobId, testName, testName, null, null, null, new Dictionary<string, VariableValue>(), new List<MaskHint>(), new Pipelines.JobResources(), new Pipelines.ContextData.DictionaryContextData(), new Pipelines.WorkspaceOptions(), new List<Pipelines.ActionStep>(), null);
|
||||
_message = new Pipelines.AgentJobRequestMessage(plan, timeline, jobId, testName, testName, null, null, null, new Dictionary<string, VariableValue>(), new List<MaskHint>(), new Pipelines.JobResources(), new Pipelines.ContextData.DictionaryContextData(), new Pipelines.WorkspaceOptions(), new List<Pipelines.ActionStep>(), null, null);
|
||||
_message.Variables[Constants.Variables.System.Culture] = "en-US";
|
||||
_message.Resources.Endpoints.Add(new ServiceEndpoint()
|
||||
{
|
||||
|
||||
@@ -67,7 +67,7 @@ namespace GitHub.Runner.Common.Tests.Worker
|
||||
new Pipelines.ContextData.DictionaryContextData()
|
||||
},
|
||||
};
|
||||
var jobRequest = new Pipelines.AgentJobRequestMessage(plan, timeline, JobId, jobName, jobName, new StringToken(null, null, null, "ubuntu"), sidecarContainers, null, variables, new List<MaskHint>(), resources, context, null, actions, null);
|
||||
var jobRequest = new Pipelines.AgentJobRequestMessage(plan, timeline, JobId, jobName, jobName, new StringToken(null, null, null, "ubuntu"), sidecarContainers, null, variables, new List<MaskHint>(), resources, context, null, actions, null, null);
|
||||
return jobRequest;
|
||||
}
|
||||
|
||||
|
||||
@@ -1,27 +0,0 @@
|
||||
name: 'Hello World'
|
||||
description: 'Greet the world and record the time'
|
||||
author: 'Test Corporation'
|
||||
inputs:
|
||||
greeting: # id of input
|
||||
description: 'The greeting we choose - will print ""{greeting}, World!"" on stdout'
|
||||
required: true
|
||||
default: 'Hello'
|
||||
entryPoint: # id of input
|
||||
description: 'optional docker entrypoint overwrite.'
|
||||
required: false
|
||||
outputs:
|
||||
time: # id of output
|
||||
description: 'The time we did the greeting'
|
||||
icon: 'hello.svg' # vector art to display in the GitHub Marketplace
|
||||
color: 'green' # optional, decorates the entry in the GitHub Marketplace
|
||||
runs:
|
||||
using: 'docker'
|
||||
image: 'Dockerfile'
|
||||
args:
|
||||
- 'bzz'
|
||||
entrypoint: 'main.sh'
|
||||
env:
|
||||
Token: foo
|
||||
Url: bar
|
||||
pre-entrypoint: 'init.sh'
|
||||
pre-if: 'success()'
|
||||
@@ -1,26 +0,0 @@
|
||||
name: 'Hello World'
|
||||
description: 'Greet the world and record the time'
|
||||
author: 'Test Corporation'
|
||||
inputs:
|
||||
greeting: # id of input
|
||||
description: 'The greeting we choose - will print ""{greeting}, World!"" on stdout'
|
||||
required: true
|
||||
default: 'Hello'
|
||||
entryPoint: # id of input
|
||||
description: 'optional docker entrypoint overwrite.'
|
||||
required: false
|
||||
outputs:
|
||||
time: # id of output
|
||||
description: 'The time we did the greeting'
|
||||
icon: 'hello.svg' # vector art to display in the GitHub Marketplace
|
||||
color: 'green' # optional, decorates the entry in the GitHub Marketplace
|
||||
runs:
|
||||
using: 'docker'
|
||||
image: 'Dockerfile'
|
||||
args:
|
||||
- 'bzz'
|
||||
entrypoint: 'main.sh'
|
||||
env:
|
||||
Token: foo
|
||||
Url: bar
|
||||
pre-entrypoint: 'init.sh'
|
||||
@@ -1,22 +0,0 @@
|
||||
name: 'Hello World'
|
||||
description: 'Greet the world and record the time'
|
||||
author: 'Test Corporation'
|
||||
inputs:
|
||||
greeting: # id of input
|
||||
description: 'The greeting we choose - will print ""{greeting}, World!"" on stdout'
|
||||
required: true
|
||||
default: 'Hello'
|
||||
deprecationMessage: 'This property has been deprecated'
|
||||
entryPoint: # id of input
|
||||
description: 'optional docker entrypoint overwrite.'
|
||||
required: false
|
||||
outputs:
|
||||
time: # id of output
|
||||
description: 'The time we did the greeting'
|
||||
icon: 'hello.svg' # vector art to display in the GitHub Marketplace
|
||||
color: 'green' # optional, decorates the entry in the GitHub Marketplace
|
||||
runs:
|
||||
using: 'node12'
|
||||
main: 'main.js'
|
||||
pre: 'init.js'
|
||||
pre-if: 'cancelled()'
|
||||
@@ -1,21 +0,0 @@
|
||||
name: 'Hello World'
|
||||
description: 'Greet the world and record the time'
|
||||
author: 'Test Corporation'
|
||||
inputs:
|
||||
greeting: # id of input
|
||||
description: 'The greeting we choose - will print ""{greeting}, World!"" on stdout'
|
||||
required: true
|
||||
default: 'Hello'
|
||||
deprecationMessage: 'This property has been deprecated'
|
||||
entryPoint: # id of input
|
||||
description: 'optional docker entrypoint overwrite.'
|
||||
required: false
|
||||
outputs:
|
||||
time: # id of output
|
||||
description: 'The time we did the greeting'
|
||||
icon: 'hello.svg' # vector art to display in the GitHub Marketplace
|
||||
color: 'green' # optional, decorates the entry in the GitHub Marketplace
|
||||
runs:
|
||||
using: 'node12'
|
||||
main: 'main.js'
|
||||
pre: 'init.js'
|
||||
Reference in New Issue
Block a user