More PR feedback, standardize 'OAuth token' terms

* Add event type to credential call

* Move events to contants

.github/workflows/build.yml

@@ -43,14 +43,6 @@ jobs:
     steps:
     - uses: actions/checkout@v1
 
-    # Set Path workaround for https://github.com/actions/virtual-environments/issues/263
-    - run: |
-        echo "::add-path::C:\Program Files\Git\mingw64\bin"
-        echo "::add-path::C:\Program Files\Git\usr\bin"
-        echo "::add-path::C:\Program Files\Git\bin"
-      if: matrix.os == 'windows-latest'
-      name: "Temp step to Set Path for Windows"
-
     # Build runner layout
     - name: Build & Layout Release
       run: |

docs/design/auth.md

@@ -0,0 +1,61 @@
+# Runner Authentication and Authorization
+
+## Goals
+  - Support runner installs in untrusted domains.
+  - The account that configures or runs the runner process is not relevant for accessing GitHub resources.
+  - Accessing GitHub resources is done with a per-job token which expires when job completes.
+  - The token is granted to trusted parts of the system including the runner, actions and script steps specified by the workflow author as trusted.
+  - All OAuth tokens that come from the Token Service that the runner uses to access Actions Service resources are the same. It's just the scope and expiration of the token that may vary.
+
+## Configuration
+
+Configuring a self-hosted runner is [covered here in the documentation](https://help.github.com/en/actions/hosting-your-own-runners/adding-self-hosted-runners).
+
+Configuration is done with the user being authenticated via a time-limited, GitHub runner registration token.
+
+*Your credentials are never used for registering the runner with the service.*
+
+![Self-hosted runner config](../res/self-hosted-config.png)
+
+During configuration, an RSA public/private key pair is created, the private key is stored in file on disk. On Windows, the content is protected with DPAPI (machine level encrypted - runner only valid on that machine) and on Linux/OSX with `chmod` permissions.
+
+Using your credentials, the runner is registered with the service by sending the public key to the service which adds that runner to the pool and stores the public key, the Token Service will generate a `clientId` associated with the public key.
+
+## Start and Listen
+
+After configuring the runner, the runner can be started interactively (`./run.cmd` or `./run.sh`) or as a service.
+
+![Self-hosted runner start](../res/self-hosted-start.png)
+
+On start, the runner listener process loads the RSA private key (on Windows decrypting with machine key DPAPI), and asks the Token Service for an OAuth token which is signed with the RSA private key.
+The server then responds with an OAuth token that grants permission to access the message queue (HTTP long poll), allowing the runner to acquire the messages it will eventually run.
+
+## Run a workflow
+
+When a workflow is run, its labels are evaluated, it is matched to a runner and a message is placed in a queue of messages for that runner.  
+The runner then starts listening for jobs via the message queue HTTP long poll.  
+The message is encrypted with the runner's public key, stored during runner configuration.  
+
+![Runner workflow run](../res/workflow-run.png)
+
+A workflow is queued as a result of a triggered [event](https://help.github.com/en/actions/reference/events-that-trigger-workflows). Workflows can be scheduled to [run at specific UTC times](https://help.github.com/en/actions/reference/events-that-trigger-workflows#scheduled-events-schedule) using POSIX `cron` syntax.
+An [OAuth token](http://self-issued.info/docs/draft-ietf-oauth-json-web-token.html) is generated, granting limited access to the host in Actions Service associated with the github.com repository/organization.
+The lifetime of the OAuth token is the lifetime of the run or at most the [job timeout (default: 6 hours)](https://help.github.com/en/actions/reference/workflow-syntax-for-github-actions#jobsjob_idtimeout-minutes), plus 10 additional minutes.
+
+## Accessing GitHub resources
+
+The job message sent to the runner contains the OAuth token to talk back to the Actions Service.
+The runner listener parent process will spawn a runner worker process for that job and send it the job message over IPC.
+The token is never persisted.
+
+Each action is run as a unique subprocess.
+The encrypted access token will be provided as an environment variable in each action subprocess.  
+The token is registered with the runner as a secret and scrubbed from the logs as they are written.
+
+Authentication in a workflow run to github.com can be accomplished by using the [`GITHUB_TOKEN`](https://help.github.com/en/actions/configuring-and-managing-workflows/authenticating-with-the-github_token#about-the-github_token-secret)) secret. This token expires after 60 minutes. Please note that this token is different from the OAuth token that the runner uses to talk to the Actions Service.
+
+## Hosted runner authentication
+
+Hosted runner authentication differs from self-hosted authentication in that runners do not undergo a registration process, but instead, the hosted runners get the OAuth token directly by reading the `.credentials` file. The scope of this particular token is limited for a given workflow job execution, and the token is revoked as soon as the job is finished.
+
+![Hosted runner config and start](../res/hosted-config-start.png)

docs/res/runner-auth-diags.txt

@@ -0,0 +1,52 @@
+# Markup used to generate the runner auth diagrams: https://websequencediagrams.com
+
+title Runner Configuration (self-hosted only)
+
+note left of Runner: GitHub repo URL as input
+Runner->github.com: Retrieve Actions Service access using runner registration token
+github.com->Runner: Access token for Actions Service
+note left of Runner: Generate RSA key pair
+note left of Runner: Store encrypted RSA private key on disk
+Runner->Actions Service: Register runner using Actions Service access token
+note right of Runner: Runner name, RSA public key sent
+note right of Actions Service: Public key stored
+Actions Service->Token Service: Register runner as an app along with the RSA public key
+note right of Token Service: Public key stored
+Token Service->Actions Service: Client Id for the runner application
+Actions Service->Runner: Client Id and Token Endpoint URL
+note left of Runner: Store runner configuration info into .runner file
+note left of Runner: Store Token registration info into .credentials file
+
+title Runner Start and Running (self-hosted only)
+
+Runner.Listener->Runner.Listener: Start
+note left of Runner.Listener: Load config info from .runner
+note left of Runner.Listener: Load token registration from .credentials
+Runner.Listener->Token Service: Exchange OAuth token (happens every 50 mins)
+note right of Runner.Listener: Construct JWT token, use Client Id signed by RSA private key
+note left of Actions Service: Find corresponding RSA public key, use Client Id\nVerify JWT token's signature
+Token Service->Runner.Listener: OAuth token with limited permission and valid for 50 mins
+Runner.Listener->Actions Service: Connect to Actions Service with OAuth token
+Actions Service->Runner.Listener: Workflow job
+
+title Running workflow
+
+Runner.Listener->Service (Message Queue): Get message
+note right of Runner.Listener: Authenticate with exchanged OAuth token
+Event->Actions Service: Queue workflow
+Actions Service->Actions Service: Generate OAuth token per job
+Actions Service->Actions Service: Build job message with the OAuth token
+Actions Service->Actions Service: Encrypt job message with the target runner's public key
+Actions Service->Service (Message Queue): Send encrypted job message to runner
+Service (Message Queue)->Runner.Listener: Send job
+note right of Runner.Listener: Decrypt message with runner's private key
+Runner.Listener->Runner.Worker: Create worker process per job and run the job
+
+title Runner Configuration, Start and Running (hosted only)
+
+Machine Management Service->Runner.Listener: Construct .runner configuration file, store token in .credentials
+Runner.Listener->Runner.Listener: Start
+note left of Runner.Listener: Load config info from .runner
+note left of Runner.Listener: Load OAuth token from .credentials
+Runner.Listener->Actions Service: Connect to Actions Service with OAuth token in .credentials
+Actions Service->Runner.Listener: Workflow job

src/Runner.Common/Constants.cs

@@ -136,6 +136,12 @@ namespace GitHub.Runner.Common
             }
         }
 
+        public static class RunnerEvent
+        {
+            public static readonly string Register = "register";
+            public static readonly string Remove = "remove";
+        }
+
         public static class Pipeline
         {
             public static class Path

src/Runner.Common/HostContext.cs

@@ -83,7 +83,6 @@ namespace GitHub.Runner.Common
             _loadContext.Unloading += LoadContext_Unloading;
 
             this.SecretMasker.AddValueEncoder(ValueEncoders.Base64StringEscape);
-            this.SecretMasker.AddValueEncoder(ValueEncoders.Base64StringEscapeTrimmed);
             this.SecretMasker.AddValueEncoder(ValueEncoders.Base64StringEscapeShift1);
             this.SecretMasker.AddValueEncoder(ValueEncoders.Base64StringEscapeShift2);
             this.SecretMasker.AddValueEncoder(ValueEncoders.ExpressionStringEscape);

src/Runner.Listener/Configuration/ConfigurationManager.cs

@@ -1,19 +1,18 @@
 using GitHub.DistributedTask.WebApi;
+using GitHub.Runner.Common;
 using GitHub.Runner.Common.Util;
+using GitHub.Runner.Sdk;
 using GitHub.Services.Common;
 using GitHub.Services.OAuth;
 using GitHub.Services.WebApi;
 using System;
 using System.Collections.Generic;
-using System.IO;
 using System.Linq;
-using System.Security.Cryptography;
-using System.Threading.Tasks;
-using System.Runtime.InteropServices;
-using GitHub.Runner.Common;
-using GitHub.Runner.Sdk;
 using System.Net.Http;
 using System.Net.Http.Headers;
+using System.Runtime.InteropServices;
+using System.Security.Cryptography;
+using System.Threading.Tasks;
 
 namespace GitHub.Runner.Listener.Configuration
 {
@@ -109,7 +108,7 @@ namespace GitHub.Runner.Listener.Configuration
                 {
                     runnerSettings.GitHubUrl = inputUrl;
                     var githubToken = command.GetRunnerRegisterToken();
-                    GitHubAuthResult authResult = await GetTenantCredential(inputUrl, githubToken);
+                    GitHubAuthResult authResult = await GetTenantCredential(inputUrl, githubToken, Constants.RunnerEvent.Register);
                     runnerSettings.ServerUrl = authResult.TenantUrl;
                     creds = authResult.ToVssCredentials();
                     Trace.Info("cred retrieved via GitHub auth");
@@ -277,12 +276,15 @@ namespace GitHub.Runner.Listener.Configuration
                 throw new NotSupportedException("Message queue listen OAuth token.");
             }
 
-            // Testing agent connection, detect any protential connection issue, like local clock skew that cause OAuth token expired.
+            // Testing agent connection, detect any potential connection issue, like local clock skew that cause OAuth token expired.
             var credMgr = HostContext.GetService<ICredentialManager>();
             VssCredentials credential = credMgr.LoadCredentials();
             try
             {
                 await _runnerServer.ConnectAsync(new Uri(runnerSettings.ServerUrl), credential);
+                // ConnectAsync() hits _apis/connectionData which is an anonymous endpoint
+                // Need to hit an authenticate endpoint to trigger OAuth token exchange.
+                await _runnerServer.GetAgentPoolsAsync();
                 _term.WriteSuccessMessage("Runner connection is good");
             }
             catch (VssOAuthTokenRequestException ex) when (ex.Message.Contains("Current server time is"))
@@ -373,7 +375,7 @@ namespace GitHub.Runner.Listener.Configuration
                     else
                     {
                         var githubToken = command.GetRunnerDeletionToken();
-                        GitHubAuthResult authResult = await GetTenantCredential(settings.GitHubUrl, githubToken);
+                        GitHubAuthResult authResult = await GetTenantCredential(settings.GitHubUrl, githubToken, Constants.RunnerEvent.Remove);
                         creds = authResult.ToVssCredentials();
                         Trace.Info("cred retrieved via GitHub auth");
                     }
@@ -517,17 +519,23 @@ namespace GitHub.Runner.Listener.Configuration
             }
         }
 
-        private async Task<GitHubAuthResult> GetTenantCredential(string githubUrl, string githubToken)
+        private async Task<GitHubAuthResult> GetTenantCredential(string githubUrl, string githubToken, string runnerEvent)
         {
-            var gitHubUrl = new UriBuilder(githubUrl);
-            var githubApiUrl = $"https://api.{gitHubUrl.Host}/repos/{gitHubUrl.Path.Trim('/')}/actions-runners/registration";
+            var gitHubUrlBuilder = new UriBuilder(githubUrl);
+            var githubApiUrl = $"{gitHubUrlBuilder.Scheme}://api.{gitHubUrlBuilder.Host}/actions/runner-registration";
             using (var httpClientHandler = HostContext.CreateHttpClientHandler())
             using (var httpClient = new HttpClient(httpClientHandler))
             {
                 httpClient.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("RemoteAuth", githubToken);
                 httpClient.DefaultRequestHeaders.UserAgent.Add(HostContext.UserAgent);
-                httpClient.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/vnd.github.shuri-preview+json"));
-                var response = await httpClient.PostAsync(githubApiUrl, new StringContent("", null, "application/json"));
+
+                var bodyObject = new Dictionary<string, string>()
+                {
+                    {"url", githubUrl},
+                    {"runner_event", runnerEvent}
+                };
+
+                var response = await httpClient.PostAsync(githubApiUrl, new StringContent(StringUtil.ConvertToJson(bodyObject), null, "application/json"));
 
                 if (response.IsSuccessStatusCode)
                 {

src/Runner.Worker/ActionCommandManager.cs

@@ -1,6 +1,7 @@
 using GitHub.DistributedTask.Pipelines;
 using GitHub.DistributedTask.WebApi;
 using GitHub.Runner.Common.Util;
+using GitHub.Runner.Worker.Container;
 using System;
 using System.Collections.Generic;
 using System.IO;
@@ -15,14 +16,14 @@ namespace GitHub.Runner.Worker
     {
         void EnablePluginInternalCommand();
         void DisablePluginInternalCommand();
-        bool TryProcessCommand(IExecutionContext context, string input);
+        bool TryProcessCommand(IExecutionContext context, string input, ContainerInfo container);
     }
 
     public sealed class ActionCommandManager : RunnerService, IActionCommandManager
     {
         private const string _stopCommand = "stop-commands";
         private readonly Dictionary<string, IActionCommandExtension> _commandExtensions = new Dictionary<string, IActionCommandExtension>(StringComparer.OrdinalIgnoreCase);
-        private HashSet<string> _registeredCommands = new HashSet<string>(StringComparer.OrdinalIgnoreCase);
+        private readonly HashSet<string> _registeredCommands = new HashSet<string>(StringComparer.OrdinalIgnoreCase);
         private readonly object _commandSerializeLock = new object();
         private bool _stopProcessCommand = false;
         private string _stopToken = null;
@@ -58,7 +59,7 @@ namespace GitHub.Runner.Worker
             _registeredCommands.Remove("internal-set-repo-path");
         }
 
-        public bool TryProcessCommand(IExecutionContext context, string input)
+        public bool TryProcessCommand(IExecutionContext context, string input, ContainerInfo container)
         {
             if (string.IsNullOrEmpty(input))
             {
@@ -114,7 +115,7 @@ namespace GitHub.Runner.Worker
 
                         try
                         {
-                            extension.ProcessCommand(context, input, actionCommand);
+                            extension.ProcessCommand(context, input, actionCommand, container);
                         }
                         catch (Exception ex)
                         {
@@ -140,7 +141,7 @@ namespace GitHub.Runner.Worker
         string Command { get; }
         bool OmitEcho { get; }
 
-        void ProcessCommand(IExecutionContext context, string line, ActionCommand command);
+        void ProcessCommand(IExecutionContext context, string line, ActionCommand command, ContainerInfo container);
     }
 
     public sealed class InternalPluginSetRepoPathCommandExtension : RunnerService, IActionCommandExtension
@@ -150,7 +151,7 @@ namespace GitHub.Runner.Worker
 
         public Type ExtensionType => typeof(IActionCommandExtension);
 
-        public void ProcessCommand(IExecutionContext context, string line, ActionCommand command)
+        public void ProcessCommand(IExecutionContext context, string line, ActionCommand command, ContainerInfo container)
         {
             if (!command.Properties.TryGetValue(SetRepoPathCommandProperties.repoFullName, out string repoFullName) || string.IsNullOrEmpty(repoFullName))
             {
@@ -180,7 +181,7 @@ namespace GitHub.Runner.Worker
 
         public Type ExtensionType => typeof(IActionCommandExtension);
 
-        public void ProcessCommand(IExecutionContext context, string line, ActionCommand command)
+        public void ProcessCommand(IExecutionContext context, string line, ActionCommand command, ContainerInfo container)
         {
             if (!command.Properties.TryGetValue(SetEnvCommandProperties.Name, out string envName) || string.IsNullOrEmpty(envName))
             {
@@ -205,7 +206,7 @@ namespace GitHub.Runner.Worker
 
         public Type ExtensionType => typeof(IActionCommandExtension);
 
-        public void ProcessCommand(IExecutionContext context, string line, ActionCommand command)
+        public void ProcessCommand(IExecutionContext context, string line, ActionCommand command, ContainerInfo container)
         {
             if (!command.Properties.TryGetValue(SetOutputCommandProperties.Name, out string outputName) || string.IsNullOrEmpty(outputName))
             {
@@ -229,7 +230,7 @@ namespace GitHub.Runner.Worker
 
         public Type ExtensionType => typeof(IActionCommandExtension);
 
-        public void ProcessCommand(IExecutionContext context, string line, ActionCommand command)
+        public void ProcessCommand(IExecutionContext context, string line, ActionCommand command, ContainerInfo container)
         {
             if (!command.Properties.TryGetValue(SaveStateCommandProperties.Name, out string stateName) || string.IsNullOrEmpty(stateName))
             {
@@ -253,7 +254,7 @@ namespace GitHub.Runner.Worker
 
         public Type ExtensionType => typeof(IActionCommandExtension);
 
-        public void ProcessCommand(IExecutionContext context, string line, ActionCommand command)
+        public void ProcessCommand(IExecutionContext context, string line, ActionCommand command, ContainerInfo container)
         {
             if (string.IsNullOrWhiteSpace(command.Data))
             {
@@ -279,7 +280,7 @@ namespace GitHub.Runner.Worker
 
         public Type ExtensionType => typeof(IActionCommandExtension);
 
-        public void ProcessCommand(IExecutionContext context, string line, ActionCommand command)
+        public void ProcessCommand(IExecutionContext context, string line, ActionCommand command, ContainerInfo container)
         {
             ArgUtil.NotNullOrEmpty(command.Data, "path");
             context.PrependPath.RemoveAll(x => string.Equals(x, command.Data, StringComparison.CurrentCulture));
@@ -294,7 +295,7 @@ namespace GitHub.Runner.Worker
 
         public Type ExtensionType => typeof(IActionCommandExtension);
 
-        public void ProcessCommand(IExecutionContext context, string line, ActionCommand command)
+        public void ProcessCommand(IExecutionContext context, string line, ActionCommand command, ContainerInfo container)
         {
             var file = command.Data;
 
@@ -306,9 +307,9 @@ namespace GitHub.Runner.Worker
             }
 
             // Translate file path back from container path
-            if (context.Container != null)
+            if (container != null)
             {
-                file = context.Container.TranslateToHostPath(file);
+                file = container.TranslateToHostPath(file);
             }
 
             // Root the path
@@ -341,7 +342,7 @@ namespace GitHub.Runner.Worker
 
         public Type ExtensionType => typeof(IActionCommandExtension);
 
-        public void ProcessCommand(IExecutionContext context, string line, ActionCommand command)
+        public void ProcessCommand(IExecutionContext context, string line, ActionCommand command, ContainerInfo container)
         {
             command.Properties.TryGetValue(RemoveMatcherCommandProperties.Owner, out string owner);
             var file = command.Data;
@@ -369,9 +370,9 @@ namespace GitHub.Runner.Worker
             else
             {
                 // Translate file path back from container path
-                if (context.Container != null)
+                if (container != null)
                 {
-                    file = context.Container.TranslateToHostPath(file);
+                    file = container.TranslateToHostPath(file);
                 }
 
                 // Root the path
@@ -409,7 +410,7 @@ namespace GitHub.Runner.Worker
 
         public Type ExtensionType => typeof(IActionCommandExtension);
 
-        public void ProcessCommand(IExecutionContext context, string inputLine, ActionCommand command)
+        public void ProcessCommand(IExecutionContext context, string inputLine, ActionCommand command, ContainerInfo container)
         {
             context.Debug(command.Data);
         }
@@ -437,7 +438,7 @@ namespace GitHub.Runner.Worker
 
         public Type ExtensionType => typeof(IActionCommandExtension);
 
-        public void ProcessCommand(IExecutionContext context, string inputLine, ActionCommand command)
+        public void ProcessCommand(IExecutionContext context, string inputLine, ActionCommand command, ContainerInfo container)
         {
             command.Properties.TryGetValue(IssueCommandProperties.File, out string file);
             command.Properties.TryGetValue(IssueCommandProperties.Line, out string line);
@@ -454,10 +455,10 @@ namespace GitHub.Runner.Worker
             {
                 issue.Category = "Code";
 
-                if (context.Container != null)
+                if (container != null)
                 {
                     // Translate file path back from container path
-                    file = context.Container.TranslateToHostPath(file);
+                    file = container.TranslateToHostPath(file);
                     command.Properties[IssueCommandProperties.File] = file;
                 }
 
@@ -517,7 +518,7 @@ namespace GitHub.Runner.Worker
 
         public Type ExtensionType => typeof(IActionCommandExtension);
 
-        public void ProcessCommand(IExecutionContext context, string line, ActionCommand command)
+        public void ProcessCommand(IExecutionContext context, string line, ActionCommand command, ContainerInfo container)
         {
             var data = this is GroupCommandExtension ? command.Data : string.Empty;
             context.Output($"##[{Command}]{data}");
@@ -531,7 +532,7 @@ namespace GitHub.Runner.Worker
 
         public Type ExtensionType => typeof(IActionCommandExtension);
 
-        public void ProcessCommand(IExecutionContext context, string line, ActionCommand command)
+        public void ProcessCommand(IExecutionContext context, string line, ActionCommand command, ContainerInfo container)
         {
             ArgUtil.NotNullOrEmpty(command.Data, "value");
 

src/Runner.Worker/ActionManifestManager.cs

@@ -32,6 +32,7 @@ namespace GitHub.Runner.Worker
     public sealed class ActionManifestManager : RunnerService, IActionManifestManager
     {
         private TemplateSchema _actionManifestSchema;
+        private IReadOnlyList<String> _fileTable;
 
         public override void Initialize(IHostContext hostContext)
         {
@@ -61,6 +62,9 @@ namespace GitHub.Runner.Worker
 
                 // Get the file ID
                 var fileId = context.GetFileId(manifestFile);
+                _fileTable = context.GetFileTable();
+
+                // Read the file
                 var fileContent = File.ReadAllText(manifestFile);
                 using (var stringReader = new StringReader(fileContent))
                 {
@@ -265,6 +269,15 @@ namespace GitHub.Runner.Worker
                 }
             }
 
+            // Add the file table
+            if (_fileTable?.Count > 0)
+            {
+                for (var i = 0 ; i < _fileTable.Count ; i++)
+                {
+                    result.GetFileId(_fileTable[i]);
+                }
+            }
+
             return result;
         }
 
@@ -415,566 +428,5 @@ namespace GitHub.Runner.Worker
             }
         }
     }
-
-    /// <summary>
-    /// Converts a YAML file into a TemplateToken
-    /// </summary>
-    internal sealed class YamlObjectReader : IObjectReader
-    {
-        internal YamlObjectReader(
-            Int32? fileId,
-            TextReader input)
-        {
-            m_fileId = fileId;
-            m_parser = new Parser(input);
-        }
-
-        public Boolean AllowLiteral(out LiteralToken value)
-        {
-            if (EvaluateCurrent() is Scalar scalar)
-            {
-                // Tag specified
-                if (!string.IsNullOrEmpty(scalar.Tag))
-                {
-                    // String tag
-                    if (string.Equals(scalar.Tag, c_stringTag, StringComparison.Ordinal))
-                    {
-                        value = new StringToken(m_fileId, scalar.Start.Line, scalar.Start.Column, scalar.Value);
-                        MoveNext();
-                        return true;
-                    }
-
-                    // Not plain style
-                    if (scalar.Style != ScalarStyle.Plain)
-                    {
-                        throw new NotSupportedException($"The scalar style '{scalar.Style}' on line {scalar.Start.Line} and column {scalar.Start.Column} is not valid with the tag '{scalar.Tag}'");
-                    }
-
-                    // Boolean, Float, Integer, or Null
-                    switch (scalar.Tag)
-                    {
-                        case c_booleanTag:
-                            value = ParseBoolean(scalar);
-                            break;
-                        case c_floatTag:
-                            value = ParseFloat(scalar);
-                            break;
-                        case c_integerTag:
-                            value = ParseInteger(scalar);
-                            break;
-                        case c_nullTag:
-                            value = ParseNull(scalar);
-                            break;
-                        default:
-                            throw new NotSupportedException($"Unexpected tag '{scalar.Tag}'");
-                    }
-
-                    MoveNext();
-                    return true;
-                }
-
-                // Plain style, determine type using YAML 1.2 "core" schema https://yaml.org/spec/1.2/spec.html#id2804923
-                if (scalar.Style == ScalarStyle.Plain)
-                {
-                    if (MatchNull(scalar, out var nullToken))
-                    {
-                        value = nullToken;
-                    }
-                    else if (MatchBoolean(scalar, out var booleanToken))
-                    {
-                        value = booleanToken;
-                    }
-                    else if (MatchInteger(scalar, out var numberToken) ||
-                        MatchFloat(scalar, out numberToken))
-                    {
-                        value = numberToken;
-                    }
-                    else
-                    {
-                        value = new StringToken(m_fileId, scalar.Start.Line, scalar.Start.Column, scalar.Value);
-                    }
-
-                    MoveNext();
-                    return true;
-                }
-
-                // Otherwise assume string
-                value = new StringToken(m_fileId, scalar.Start.Line, scalar.Start.Column, scalar.Value);
-                MoveNext();
-                return true;
-            }
-
-            value = default;
-            return false;
-        }
-
-        public Boolean AllowSequenceStart(out SequenceToken value)
-        {
-            if (EvaluateCurrent() is SequenceStart sequenceStart)
-            {
-                value = new SequenceToken(m_fileId, sequenceStart.Start.Line, sequenceStart.Start.Column);
-                MoveNext();
-                return true;
-            }
-
-            value = default;
-            return false;
-        }
-
-        public Boolean AllowSequenceEnd()
-        {
-            if (EvaluateCurrent() is SequenceEnd)
-            {
-                MoveNext();
-                return true;
-            }
-
-            return false;
-        }
-
-        public Boolean AllowMappingStart(out MappingToken value)
-        {
-            if (EvaluateCurrent() is MappingStart mappingStart)
-            {
-                value = new MappingToken(m_fileId, mappingStart.Start.Line, mappingStart.Start.Column);
-                MoveNext();
-                return true;
-            }
-
-            value = default;
-            return false;
-        }
-
-        public Boolean AllowMappingEnd()
-        {
-            if (EvaluateCurrent() is MappingEnd)
-            {
-                MoveNext();
-                return true;
-            }
-
-            return false;
-        }
-
-        /// <summary>
-        /// Consumes the last parsing events, which are expected to be DocumentEnd and StreamEnd.
-        /// </summary>
-        public void ValidateEnd()
-        {
-            if (EvaluateCurrent() is DocumentEnd)
-            {
-                MoveNext();
-            }
-            else
-            {
-                throw new InvalidOperationException("Expected document end parse event");
-            }
-
-            if (EvaluateCurrent() is StreamEnd)
-            {
-                MoveNext();
-            }
-            else
-            {
-                throw new InvalidOperationException("Expected stream end parse event");
-            }
-
-            if (MoveNext())
-            {
-                throw new InvalidOperationException("Expected end of parse events");
-            }
-        }
-
-        /// <summary>
-        /// Consumes the first parsing events, which are expected to be StreamStart and DocumentStart.
-        /// </summary>
-        public void ValidateStart()
-        {
-            if (EvaluateCurrent() != null)
-            {
-                throw new InvalidOperationException("Unexpected parser state");
-            }
-
-            if (!MoveNext())
-            {
-                throw new InvalidOperationException("Expected a parse event");
-            }
-
-            if (EvaluateCurrent() is StreamStart)
-            {
-                MoveNext();
-            }
-            else
-            {
-                throw new InvalidOperationException("Expected stream start parse event");
-            }
-
-            if (EvaluateCurrent() is DocumentStart)
-            {
-                MoveNext();
-            }
-            else
-            {
-                throw new InvalidOperationException("Expected document start parse event");
-            }
-        }
-
-        private ParsingEvent EvaluateCurrent()
-        {
-            if (m_current == null)
-            {
-                m_current = m_parser.Current;
-                if (m_current != null)
-                {
-                    if (m_current is Scalar scalar)
-                    {
-                        // Verify not using achors
-                        if (scalar.Anchor != null)
-                        {
-                            throw new InvalidOperationException($"Anchors are not currently supported. Remove the anchor '{scalar.Anchor}'");
-                        }
-                    }
-                    else if (m_current is MappingStart mappingStart)
-                    {
-                        // Verify not using achors
-                        if (mappingStart.Anchor != null)
-                        {
-                            throw new InvalidOperationException($"Anchors are not currently supported. Remove the anchor '{mappingStart.Anchor}'");
-                        }
-                    }
-                    else if (m_current is SequenceStart sequenceStart)
-                    {
-                        // Verify not using achors
-                        if (sequenceStart.Anchor != null)
-                        {
-                            throw new InvalidOperationException($"Anchors are not currently supported. Remove the anchor '{sequenceStart.Anchor}'");
-                        }
-                    }
-                    else if (!(m_current is MappingEnd) &&
-                        !(m_current is SequenceEnd) &&
-                        !(m_current is DocumentStart) &&
-                        !(m_current is DocumentEnd) &&
-                        !(m_current is StreamStart) &&
-                        !(m_current is StreamEnd))
-                    {
-                        throw new InvalidOperationException($"Unexpected parsing event type: {m_current.GetType().Name}");
-                    }
-                }
-            }
-
-            return m_current;
-        }
-
-        private Boolean MoveNext()
-        {
-            m_current = null;
-            return m_parser.MoveNext();
-        }
-
-        private BooleanToken ParseBoolean(Scalar scalar)
-        {
-            if (MatchBoolean(scalar, out var token))
-            {
-                return token;
-            }
-
-            ThrowInvalidValue(scalar, c_booleanTag); // throws
-            return default;
-        }
-
-        private NumberToken ParseFloat(Scalar scalar)
-        {
-            if (MatchFloat(scalar, out var token))
-            {
-                return token;
-            }
-
-            ThrowInvalidValue(scalar, c_floatTag); // throws
-            return default;
-        }
-
-        private NumberToken ParseInteger(Scalar scalar)
-        {
-            if (MatchInteger(scalar, out var token))
-            {
-                return token;
-            }
-
-            ThrowInvalidValue(scalar, c_integerTag); // throws
-            return default;
-        }
-
-        private NullToken ParseNull(Scalar scalar)
-        {
-            if (MatchNull(scalar, out var token))
-            {
-                return token;
-            }
-
-            ThrowInvalidValue(scalar, c_nullTag); // throws
-            return default;
-        }
-
-        private Boolean MatchBoolean(
-            Scalar scalar,
-            out BooleanToken value)
-        {
-            // YAML 1.2 "core" schema https://yaml.org/spec/1.2/spec.html#id2804923
-            switch (scalar.Value ?? string.Empty)
-            {
-                case "true":
-                case "True":
-                case "TRUE":
-                    value = new BooleanToken(m_fileId, scalar.Start.Line, scalar.Start.Column, true);
-                    return true;
-                case "false":
-                case "False":
-                case "FALSE":
-                    value = new BooleanToken(m_fileId, scalar.Start.Line, scalar.Start.Column, false);
-                    return true;
-            }
-
-            value = default;
-            return false;
-        }
-
-        private Boolean MatchFloat(
-            Scalar scalar,
-            out NumberToken value)
-        {
-            // YAML 1.2 "core" schema https://yaml.org/spec/1.2/spec.html#id2804923
-            var str = scalar.Value;
-            if (!string.IsNullOrEmpty(str))
-            {
-                // Check for [-+]?(\.inf|\.Inf|\.INF)|\.nan|\.NaN|\.NAN
-                switch (str)
-                {
-                    case ".inf":
-                    case ".Inf":
-                    case ".INF":
-                    case "+.inf":
-                    case "+.Inf":
-                    case "+.INF":
-                        value = new NumberToken(m_fileId, scalar.Start.Line, scalar.Start.Column, Double.PositiveInfinity);
-                        return true;
-                    case "-.inf":
-                    case "-.Inf":
-                    case "-.INF":
-                        value = new NumberToken(m_fileId, scalar.Start.Line, scalar.Start.Column, Double.NegativeInfinity);
-                        return true;
-                    case ".nan":
-                    case ".NaN":
-                    case ".NAN":
-                        value = new NumberToken(m_fileId, scalar.Start.Line, scalar.Start.Column, Double.NaN);
-                        return true;
-                }
-
-                // Otherwise check [-+]?(\.[0-9]+|[0-9]+(\.[0-9]*)?)([eE][-+]?[0-9]+)?
-
-                // Skip leading sign
-                var index = str[0] == '-' || str[0] == '+' ? 1 : 0;
-
-                // Check for integer portion
-                var length = str.Length;
-                var hasInteger = false;
-                while (index < length && str[index] >= '0' && str[index] <= '9')
-                {
-                    hasInteger = true;
-                    index++;
-                }
-
-                // Check for decimal point
-                var hasDot = false;
-                if (index < length && str[index] == '.')
-                {
-                    hasDot = true;
-                    index++;
-                }
-
-                // Check for decimal portion
-                var hasDecimal = false;
-                while (index < length && str[index] >= '0' && str[index] <= '9')
-                {
-                    hasDecimal = true;
-                    index++;
-                }
-
-                // Check [-+]?(\.[0-9]+|[0-9]+(\.[0-9]*)?)
-                if ((hasDot && hasDecimal) || hasInteger)
-                {
-                    // Check for end
-                    if (index == length)
-                    {
-                        // Try parse
-                        if (Double.TryParse(str, NumberStyles.AllowLeadingSign | NumberStyles.AllowDecimalPoint, CultureInfo.InvariantCulture, out var doubleValue))
-                        {
-                            value = new NumberToken(m_fileId, scalar.Start.Line, scalar.Start.Column, doubleValue);
-                            return true;
-                        }
-                        // Otherwise exceeds range
-                        else
-                        {
-                            ThrowInvalidValue(scalar, c_floatTag); // throws
-                        }
-                    }
-                    // Check [eE][-+]?[0-9]
-                    else if (index < length && (str[index] == 'e' || str[index] == 'E'))
-                    {
-                        index++;
-
-                        // Skip sign
-                        if (index < length && (str[index] == '-' || str[index] == '+'))
-                        {
-                            index++;
-                        }
-
-                        // Check for exponent
-                        var hasExponent = false;
-                        while (index < length && str[index] >= '0' && str[index] <= '9')
-                        {
-                            hasExponent = true;
-                            index++;
-                        }
-
-                        // Check for end
-                        if (hasExponent && index == length)
-                        {
-                            // Try parse
-                            if (Double.TryParse(str, NumberStyles.AllowLeadingSign | NumberStyles.AllowDecimalPoint | NumberStyles.AllowExponent, CultureInfo.InvariantCulture, out var doubleValue))
-                            {
-                                value = new NumberToken(m_fileId, scalar.Start.Line, scalar.Start.Column, (Double)doubleValue);
-                                return true;
-                            }
-                            // Otherwise exceeds range
-                            else
-                            {
-                                ThrowInvalidValue(scalar, c_floatTag); // throws
-                            }
-                        }
-                    }
-                }
-            }
-
-            value = default;
-            return false;
-        }
-
-        private Boolean MatchInteger(
-            Scalar scalar,
-            out NumberToken value)
-        {
-            // YAML 1.2 "core" schema https://yaml.org/spec/1.2/spec.html#id2804923
-            var str = scalar.Value;
-            if (!string.IsNullOrEmpty(str))
-            {
-                // Check for [0-9]+
-                var firstChar = str[0];
-                if (firstChar >= '0' && firstChar <= '9' &&
-                    str.Skip(1).All(x => x >= '0' && x <= '9'))
-                {
-                    // Try parse
-                    if (Double.TryParse(str, NumberStyles.None, CultureInfo.InvariantCulture, out var doubleValue))
-                    {
-                        value = new NumberToken(m_fileId, scalar.Start.Line, scalar.Start.Column, doubleValue);
-                        return true;
-                    }
-
-                    // Otherwise exceeds range
-                    ThrowInvalidValue(scalar, c_integerTag); // throws
-                }
-                // Check for (-|+)[0-9]+
-                else if ((firstChar == '-' || firstChar == '+') &&
-                    str.Length > 1 &&
-                    str.Skip(1).All(x => x >= '0' && x <= '9'))
-                {
-                    // Try parse
-                    if (Double.TryParse(str, NumberStyles.AllowLeadingSign, CultureInfo.InvariantCulture, out var doubleValue))
-                    {
-                        value = new NumberToken(m_fileId, scalar.Start.Line, scalar.Start.Column, doubleValue);
-                        return true;
-                    }
-
-                    // Otherwise exceeds range
-                    ThrowInvalidValue(scalar, c_integerTag); // throws
-                }
-                // Check for 0x[0-9a-fA-F]+
-                else if (firstChar == '0' &&
-                    str.Length > 2 &&
-                    str[1] == 'x' &&
-                    str.Skip(2).All(x => (x >= '0' && x <= '9') || (x >= 'a' && x <= 'f') || (x >= 'A' && x <= 'F')))
-                {
-                    // Try parse
-                    if (Int32.TryParse(str.Substring(2), NumberStyles.AllowHexSpecifier, CultureInfo.InvariantCulture, out var integerValue))
-                    {
-                        value = new NumberToken(m_fileId, scalar.Start.Line, scalar.Start.Column, integerValue);
-                        return true;
-                    }
-
-                    // Otherwise exceeds range
-                    ThrowInvalidValue(scalar, c_integerTag); // throws
-                }
-                // Check for 0o[0-9]+
-                else if (firstChar == '0' &&
-                    str.Length > 2 &&
-                    str[1] == 'o' &&
-                    str.Skip(2).All(x => x >= '0' && x <= '7'))
-                {
-                    // Try parse
-                    var integerValue = default(Int32);
-                    try
-                    {
-                        integerValue = Convert.ToInt32(str.Substring(2), 8);
-                    }
-                    // Otherwise exceeds range
-                    catch (Exception)
-                    {
-                        ThrowInvalidValue(scalar, c_integerTag); // throws
-                    }
-
-                    value = new NumberToken(m_fileId, scalar.Start.Line, scalar.Start.Column, integerValue);
-                    return true;
-                }
-            }
-
-            value = default;
-            return false;
-        }
-
-        private Boolean MatchNull(
-            Scalar scalar,
-            out NullToken value)
-        {
-            // YAML 1.2 "core" schema https://yaml.org/spec/1.2/spec.html#id2804923
-            switch (scalar.Value ?? string.Empty)
-            {
-                case "":
-                case "null":
-                case "Null":
-                case "NULL":
-                case "~":
-                    value = new NullToken(m_fileId, scalar.Start.Line, scalar.Start.Column);
-                    return true;
-            }
-
-            value = default;
-            return false;
-        }
-
-        private void ThrowInvalidValue(
-            Scalar scalar,
-            String tag)
-        {
-            throw new NotSupportedException($"The value '{scalar.Value}' on line {scalar.Start.Line} and column {scalar.Start.Column} is invalid for the type '{scalar.Tag}'");
-        }
-
-        private const String c_booleanTag = "tag:yaml.org,2002:bool";
-        private const String c_floatTag = "tag:yaml.org,2002:float";
-        private const String c_integerTag = "tag:yaml.org,2002:int";
-        private const String c_nullTag = "tag:yaml.org,2002:null";
-        private const String c_stringTag = "tag:yaml.org,2002:string";
-        private readonly Int32? m_fileId;
-        private readonly Parser m_parser;
-        private ParsingEvent m_current;
-    }
 }
 

src/Runner.Worker/ActionRunner.cs

@@ -141,9 +141,7 @@ namespace GitHub.Runner.Worker
 
             // Load the inputs.
             ExecutionContext.Debug("Loading inputs");
-            var templateTrace = ExecutionContext.ToTemplateTraceWriter();
-            var schema = new PipelineTemplateSchemaFactory().CreateSchema();
-            var templateEvaluator = new PipelineTemplateEvaluator(templateTrace, schema);
+            var templateEvaluator = ExecutionContext.ToPipelineTemplateEvaluator();
             var inputs = templateEvaluator.EvaluateStepInputs(Action.Inputs, ExecutionContext.ExpressionValues);
 
             foreach (KeyValuePair<string, string> input in inputs)
@@ -295,8 +293,7 @@ namespace GitHub.Runner.Worker
                 return displayName;
             }
             // Try evaluating fully
-            var schema = new PipelineTemplateSchemaFactory().CreateSchema();
-            var templateEvaluator = new PipelineTemplateEvaluator(context.ToTemplateTraceWriter(), schema);
+            var templateEvaluator = context.ToPipelineTemplateEvaluator();
             try
             {
                 didFullyEvaluate = templateEvaluator.TryEvaluateStepDisplayName(tokenToParse, contextData, out displayName);

src/Runner.Worker/ExecutionContext.cs

@@ -11,6 +11,7 @@ using GitHub.Runner.Worker.Container;
 using GitHub.Services.WebApi;
 using GitHub.DistributedTask.Pipelines;
 using GitHub.DistributedTask.Pipelines.ContextData;
+using GitHub.DistributedTask.Pipelines.ObjectTemplating;
 using GitHub.DistributedTask.WebApi;
 using GitHub.Runner.Common.Util;
 using GitHub.Runner.Common;
@@ -49,6 +50,7 @@ namespace GitHub.Runner.Worker
         HashSet<string> OutputVariables { get; }
         IDictionary<String, String> EnvironmentVariables { get; }
         IDictionary<String, ContextScope> Scopes { get; }
+        IList<String> FileTable { get; }
         StepsContext StepsContext { get; }
         DictionaryContextData ExpressionValues { get; }
         List<string> PrependPath { get; }
@@ -141,6 +143,7 @@ namespace GitHub.Runner.Worker
         public HashSet<string> OutputVariables => _outputvariables;
         public IDictionary<String, String> EnvironmentVariables { get; private set; }
         public IDictionary<String, ContextScope> Scopes { get; private set; }
+        public IList<String> FileTable { get; private set; }
         public StepsContext StepsContext { get; private set; }
         public DictionaryContextData ExpressionValues { get; } = new DictionaryContextData();
         public bool WriteDebug { get; private set; }
@@ -266,6 +269,7 @@ namespace GitHub.Runner.Worker
             }
             child.EnvironmentVariables = EnvironmentVariables;
             child.Scopes = Scopes;
+            child.FileTable = FileTable;
             child.StepsContext = StepsContext;
             foreach (var pair in ExpressionValues)
             {
@@ -569,6 +573,9 @@ namespace GitHub.Runner.Worker
                 }
             }
 
+            // File table
+            FileTable = new List<String>(message.FileTable ?? new string[0]);
+
             // Expression functions
             if (Variables.GetBoolean("System.HashFilesV2") == true)
             {
@@ -886,6 +893,13 @@ namespace GitHub.Runner.Worker
             }
         }
 
+        public static PipelineTemplateEvaluator ToPipelineTemplateEvaluator(this IExecutionContext context)
+        {
+            var templateTrace = context.ToTemplateTraceWriter();
+            var schema = new PipelineTemplateSchemaFactory().CreateSchema();
+            return new PipelineTemplateEvaluator(templateTrace, schema, context.FileTable);
+        }
+
         public static ObjectTemplating.ITraceWriter ToTemplateTraceWriter(this IExecutionContext context)
         {
             return new TemplateTraceWriter(context);

src/Runner.Worker/Handlers/OutputManager.cs

@@ -84,7 +84,7 @@ namespace GitHub.Runner.Worker.Handlers
             {
                 // This does not need to be inside of a critical section.
                 // The logging queues and command handlers are thread-safe.
-                if (_commandManager.TryProcessCommand(_executionContext, line))
+                if (_commandManager.TryProcessCommand(_executionContext, line, _container))
                 {
                     return;
                 }

src/Runner.Worker/JobExtension.cs

@@ -78,9 +78,7 @@ namespace GitHub.Runner.Worker
 
                     // Evaluate the job-level environment variables
                     context.Debug("Evaluating job-level environment variables");
-                    var templateTrace = context.ToTemplateTraceWriter();
-                    var schema = new PipelineTemplateSchemaFactory().CreateSchema();
-                    var templateEvaluator = new PipelineTemplateEvaluator(templateTrace, schema);
+                    var templateEvaluator = context.ToPipelineTemplateEvaluator();
                     foreach (var token in message.EnvironmentVariables)
                     {
                         var environmentVariables = templateEvaluator.EvaluateStepEnvironment(token, jobContext.ExpressionValues, VarUtil.EnvironmentVariableKeyComparer);

src/Runner.Worker/StepsRunner.cs

@@ -98,9 +98,7 @@ namespace GitHub.Runner.Worker
                         step.ExecutionContext.SetGitHubContext("action", actionStep.Action.Name);
 
                         // Evaluate and merge action's env block to env context
-                        var templateTrace = step.ExecutionContext.ToTemplateTraceWriter();
-                        var schema = new PipelineTemplateSchemaFactory().CreateSchema();
-                        var templateEvaluator = new PipelineTemplateEvaluator(templateTrace, schema);
+                        var templateEvaluator = step.ExecutionContext.ToPipelineTemplateEvaluator();
                         var actionEnvironment = templateEvaluator.EvaluateStepEnvironment(actionStep.Action.Environment, step.ExecutionContext.ExpressionValues, VarUtil.EnvironmentVariableKeyComparer);
                         foreach (var env in actionEnvironment)
                         {
@@ -247,7 +245,7 @@ namespace GitHub.Runner.Worker
 
             // Set the timeout
             var timeoutMinutes = 0;
-            var templateEvaluator = CreateTemplateEvaluator(step.ExecutionContext);
+            var templateEvaluator = step.ExecutionContext.ToPipelineTemplateEvaluator();
             try
             {
                 timeoutMinutes = templateEvaluator.EvaluateStepTimeout(step.Timeout, step.ExecutionContext.ExpressionValues);
@@ -389,7 +387,7 @@ namespace GitHub.Runner.Worker
                     executionContext.Debug($"Initializing scope '{scope.Name}'");
                     executionContext.ExpressionValues["steps"] = stepsContext.GetScope(scope.ParentName);
                     executionContext.ExpressionValues["inputs"] = !String.IsNullOrEmpty(scope.ParentName) ? scopeInputs[scope.ParentName] : null;
-                    var templateEvaluator = CreateTemplateEvaluator(executionContext);
+                    var templateEvaluator = executionContext.ToPipelineTemplateEvaluator();
                     var inputs = default(DictionaryContextData);
                     try
                     {
@@ -445,7 +443,7 @@ namespace GitHub.Runner.Worker
                     executionContext.Debug($"Finalizing scope '{scope.Name}'");
                     executionContext.ExpressionValues["steps"] = stepsContext.GetScope(scope.Name);
                     executionContext.ExpressionValues["inputs"] = null;
-                    var templateEvaluator = CreateTemplateEvaluator(executionContext);
+                    var templateEvaluator = executionContext.ToPipelineTemplateEvaluator();
                     var outputs = default(DictionaryContextData);
                     try
                     {
@@ -477,12 +475,5 @@ namespace GitHub.Runner.Worker
 
             executionContext.Complete(result, resultCode: resultCode);
         }
-
-        private PipelineTemplateEvaluator CreateTemplateEvaluator(IExecutionContext executionContext)
-        {
-            var templateTrace = executionContext.ToTemplateTraceWriter();
-            var schema = new PipelineTemplateSchemaFactory().CreateSchema();
-            return new PipelineTemplateEvaluator(templateTrace, schema);
-        }
     }
 }

src/Sdk/DTLogging/Logging/ValueEncoders.cs

@@ -16,11 +16,6 @@ namespace GitHub.DistributedTask.Logging
         {
             return Convert.ToBase64String(Encoding.UTF8.GetBytes(value));
         }
-        
-        public static String Base64StringEscapeTrimmed(String value)
-        {
-            return TrimBase64End(Convert.ToBase64String(Encoding.UTF8.GetBytes(value)));
-        }
 
         // Base64 is 6 bits -> char
         // A byte is 8 bits
@@ -72,15 +67,15 @@ namespace GitHub.DistributedTask.Logging
             {
                 var shiftArray = new byte[bytes.Length - shift];
                 Array.Copy(bytes, shift, shiftArray, 0, bytes.Length - shift);
-                return TrimBase64End(Convert.ToBase64String(shiftArray));
+                return Convert.ToBase64String(shiftArray);
             }
             else
             {
-                return TrimBase64End(Convert.ToBase64String(bytes));
+                return Convert.ToBase64String(bytes);
             }
         }
 
-        public static String UriDataEscape(
+        private static String UriDataEscape(
             String value,
             Int32 maxSegmentSize)
         {
@@ -108,26 +103,5 @@ namespace GitHub.DistributedTask.Logging
 
             return result.ToString();
         }
-
-        private static String TrimBase64End(String value)
-        {
-            if (String.IsNullOrEmpty(value))
-            {
-                return String.Empty;
-            }
-            if (value.EndsWith('='))
-            {
-                var trimmed = value.TrimEnd('=');
-                if (trimmed.Length > 1)
-                {
-                    // If a base64 string ends in '=' it indicates that the base 64 character is only using 2 or 4 of the six bytes and will change if another character is added
-                    // For example 'ab' is 'YWI=' in base 64
-                    // 'abc' is 'YWJj'
-                    // We need to detect YW, not YWI so we trim the last character ('I')
-                    return trimmed.Substring(0, trimmed.Length - 1);
-                }
-            }
-            return value;
-        }
     }
 }

src/Sdk/DTObjectTemplating/ObjectTemplating/TemplateContext.cs

@@ -184,6 +184,7 @@ namespace GitHub.DistributedTask.ObjectTemplating
                 id = FileIds.Count + 1;
                 FileIds.Add(file, id);
                 FileNames.Add(file);
+                Memory.AddBytes(file);
             }
 
             return id;
@@ -191,7 +192,12 @@ namespace GitHub.DistributedTask.ObjectTemplating
 
         internal String GetFileName(Int32 fileId)
         {
-            return FileNames[fileId - 1];
+            return FileNames.Count >= fileId ? FileNames[fileId - 1] : null;
+        }
+
+        internal IReadOnlyList<String> GetFileTable()
+        {
+            return FileNames.AsReadOnly();
         }
 
         private String GetErrorPrefix(
@@ -199,9 +205,9 @@ namespace GitHub.DistributedTask.ObjectTemplating
             Int32? line,
             Int32? column)
         {
-            if (fileId != null)
+            var fileName = fileId.HasValue ? GetFileName(fileId.Value) : null;
+            if (!String.IsNullOrEmpty(fileName))
             {
-                var fileName = GetFileName(fileId.Value);
                 if (line != null && column != null)
                 {
                     return $"{fileName} {TemplateStrings.LineColumn(line, column)}:";

src/Sdk/DTObjectTemplating/ObjectTemplating/Tokens/TemplateToken.cs

@@ -30,14 +30,14 @@ namespace GitHub.DistributedTask.ObjectTemplating.Tokens
             Column = column;
         }
 
-        [IgnoreDataMember]
-        internal Int32? FileId { get; set; }
+        [DataMember(Name = "file", EmitDefaultValue = false)]
+        internal Int32? FileId { get; private set; }
 
         [DataMember(Name = "line", EmitDefaultValue = false)]
-        internal Int32? Line { get; }
+        internal Int32? Line { get; private set; }
 
         [DataMember(Name = "col", EmitDefaultValue = false)]
-        internal Int32? Column { get; }
+        internal Int32? Column { get; private set; }
 
         [DataMember(Name = "type", EmitDefaultValue = false)]
         internal Int32 Type { get; }

src/Sdk/DTObjectTemplating/ObjectTemplating/Tokens/TemplateTokenJsonConverter.cs

@@ -115,13 +115,12 @@ namespace GitHub.DistributedTask.ObjectTemplating.Tokens
             Object value,
             JsonSerializer serializer)
         {
-            base.WriteJson(writer, value, serializer);
             if (value is TemplateToken token)
             {
                 switch (token.Type)
                 {
                     case TokenType.Null:
-                        if (token.Line == null && token.Column == null)
+                        if (token.FileId == null && token.Line == null && token.Column == null)
                         {
                             writer.WriteNull();
                         }
@@ -130,12 +129,17 @@ namespace GitHub.DistributedTask.ObjectTemplating.Tokens
                             writer.WriteStartObject();
                             writer.WritePropertyName("type");
                             writer.WriteValue(token.Type);
+                            if (token.FileId != null)
+                            {
+                                writer.WritePropertyName("file");
+                                writer.WriteValue(token.FileId);
+                            }
                             if (token.Line != null)
                             {
                                 writer.WritePropertyName("line");
                                 writer.WriteValue(token.Line);
                             }
-                            if (token.Line != null)
+                            if (token.Column != null)
                             {
                                 writer.WritePropertyName("col");
                                 writer.WriteValue(token.Column);
@@ -146,7 +150,7 @@ namespace GitHub.DistributedTask.ObjectTemplating.Tokens
 
                     case TokenType.Boolean:
                         var booleanToken = token as BooleanToken;
-                        if (token.Line == null && token.Column == null)
+                        if (token.FileId == null && token.Line == null && token.Column == null)
                         {
                             writer.WriteValue(booleanToken.Value);
                         }
@@ -155,12 +159,17 @@ namespace GitHub.DistributedTask.ObjectTemplating.Tokens
                             writer.WriteStartObject();
                             writer.WritePropertyName("type");
                             writer.WriteValue(token.Type);
+                            if (token.FileId != null)
+                            {
+                                writer.WritePropertyName("file");
+                                writer.WriteValue(token.FileId);
+                            }
                             if (token.Line != null)
                             {
                                 writer.WritePropertyName("line");
                                 writer.WriteValue(token.Line);
                             }
-                            if (token.Line != null)
+                            if (token.Column != null)
                             {
                                 writer.WritePropertyName("col");
                                 writer.WriteValue(token.Column);
@@ -173,7 +182,7 @@ namespace GitHub.DistributedTask.ObjectTemplating.Tokens
 
                     case TokenType.Number:
                         var numberToken = token as NumberToken;
-                        if (token.Line == null && token.Column == null)
+                        if (token.FileId == null && token.Line == null && token.Column == null)
                         {
                             writer.WriteValue(numberToken.Value);
                         }
@@ -182,12 +191,17 @@ namespace GitHub.DistributedTask.ObjectTemplating.Tokens
                             writer.WriteStartObject();
                             writer.WritePropertyName("type");
                             writer.WriteValue(token.Type);
+                            if (token.FileId != null)
+                            {
+                                writer.WritePropertyName("file");
+                                writer.WriteValue(token.FileId);
+                            }
                             if (token.Line != null)
                             {
                                 writer.WritePropertyName("line");
                                 writer.WriteValue(token.Line);
                             }
-                            if (token.Line != null)
+                            if (token.Column != null)
                             {
                                 writer.WritePropertyName("col");
                                 writer.WriteValue(token.Column);
@@ -200,7 +214,7 @@ namespace GitHub.DistributedTask.ObjectTemplating.Tokens
 
                     case TokenType.String:
                         var stringToken = token as StringToken;
-                        if (token.Line == null && token.Column == null)
+                        if (token.FileId == null && token.Line == null && token.Column == null)
                         {
                             writer.WriteValue(stringToken.Value);
                         }
@@ -209,12 +223,17 @@ namespace GitHub.DistributedTask.ObjectTemplating.Tokens
                             writer.WriteStartObject();
                             writer.WritePropertyName("type");
                             writer.WriteValue(token.Type);
+                            if (token.FileId != null)
+                            {
+                                writer.WritePropertyName("file");
+                                writer.WriteValue(token.FileId);
+                            }
                             if (token.Line != null)
                             {
                                 writer.WritePropertyName("line");
                                 writer.WriteValue(token.Line);
                             }
-                            if (token.Line != null)
+                            if (token.Column != null)
                             {
                                 writer.WritePropertyName("col");
                                 writer.WriteValue(token.Column);
@@ -230,12 +249,17 @@ namespace GitHub.DistributedTask.ObjectTemplating.Tokens
                         writer.WriteStartObject();
                         writer.WritePropertyName("type");
                         writer.WriteValue(token.Type);
+                        if (token.FileId != null)
+                        {
+                            writer.WritePropertyName("file");
+                            writer.WriteValue(token.FileId);
+                        }
                         if (token.Line != null)
                         {
                             writer.WritePropertyName("line");
                             writer.WriteValue(token.Line);
                         }
-                        if (token.Line != null)
+                        if (token.Column != null)
                         {
                             writer.WritePropertyName("col");
                             writer.WriteValue(token.Column);
@@ -253,12 +277,17 @@ namespace GitHub.DistributedTask.ObjectTemplating.Tokens
                         writer.WriteStartObject();
                         writer.WritePropertyName("type");
                         writer.WriteValue(token.Type);
+                        if (token.FileId != null)
+                        {
+                            writer.WritePropertyName("file");
+                            writer.WriteValue(token.FileId);
+                        }
                         if (token.Line != null)
                         {
                             writer.WritePropertyName("line");
                             writer.WriteValue(token.Line);
                         }
-                        if (token.Line != null)
+                        if (token.Column != null)
                         {
                             writer.WritePropertyName("col");
                             writer.WriteValue(token.Column);
@@ -273,12 +302,17 @@ namespace GitHub.DistributedTask.ObjectTemplating.Tokens
                         writer.WriteStartObject();
                         writer.WritePropertyName("type");
                         writer.WriteValue(token.Type);
+                        if (token.FileId != null)
+                        {
+                            writer.WritePropertyName("file");
+                            writer.WriteValue(token.FileId);
+                        }
                         if (token.Line != null)
                         {
                             writer.WritePropertyName("line");
                             writer.WriteValue(token.Line);
                         }
-                        if (token.Line != null)
+                        if (token.Column != null)
                         {
                             writer.WritePropertyName("col");
                             writer.WriteValue(token.Column);
@@ -301,12 +335,17 @@ namespace GitHub.DistributedTask.ObjectTemplating.Tokens
                         writer.WriteStartObject();
                         writer.WritePropertyName("type");
                         writer.WriteValue(token.Type);
+                        if (token.FileId != null)
+                        {
+                            writer.WritePropertyName("file");
+                            writer.WriteValue(token.FileId);
+                        }
                         if (token.Line != null)
                         {
                             writer.WritePropertyName("line");
                             writer.WriteValue(token.Line);
                         }
-                        if (token.Line != null)
+                        if (token.Column != null)
                         {
                             writer.WritePropertyName("col");
                             writer.WriteValue(token.Column);

src/Sdk/DTPipelines/Pipelines/AgentJobRequestMessage.cs

@@ -39,7 +39,8 @@ namespace GitHub.DistributedTask.Pipelines
             DictionaryContextData contextData,
             WorkspaceOptions workspaceOptions,
             IEnumerable<JobStep> steps,
-            IEnumerable<ContextScope> scopes)
+            IEnumerable<ContextScope> scopes,
+            IList<String> fileTable)
         {
             this.MessageType = JobRequestMessageTypes.PipelineAgentJobRequest;
             this.Plan = plan;
@@ -74,6 +75,11 @@ namespace GitHub.DistributedTask.Pipelines
                     this.ContextData[pair.Key] = pair.Value;
                 }
             }
+
+            if (fileTable?.Count > 0)
+            {
+                m_fileTable = new List<String>(fileTable);
+            }
         }
 
         [DataMember]
@@ -237,6 +243,18 @@ namespace GitHub.DistributedTask.Pipelines
             }
         }
 
+        public IList<String> FileTable
+        {
+            get
+            {
+                if (m_fileTable == null)
+                {
+                    m_fileTable = new List<String>();
+                }
+                return m_fileTable;
+            }
+        }
+
         // todo: remove after feature-flag DistributedTask.EvaluateContainerOnRunner is enabled everywhere
         public void SetJobSidecarContainers(IDictionary<String, String> value)
         {
@@ -345,6 +363,11 @@ namespace GitHub.DistributedTask.Pipelines
                 m_environmentVariables = null;
             }
 
+            if (m_fileTable?.Count == 0)
+            {
+                m_fileTable = null;
+            }
+
             if (m_maskHints?.Count == 0)
             {
                 m_maskHints = null;
@@ -374,6 +397,9 @@ namespace GitHub.DistributedTask.Pipelines
         [DataMember(Name = "EnvironmentVariables", EmitDefaultValue = false)]
         private List<TemplateToken> m_environmentVariables;
 
+        [DataMember(Name = "FileTable", EmitDefaultValue = false)]
+        private List<String> m_fileTable;
+
         [DataMember(Name = "Mask", EmitDefaultValue = false)]
         private List<MaskHint> m_maskHints;
 

src/Sdk/DTPipelines/Pipelines/ObjectTemplating/PipelineTemplateEvaluator.cs

@@ -19,7 +19,8 @@ namespace GitHub.DistributedTask.Pipelines.ObjectTemplating
     {
         public PipelineTemplateEvaluator(
             ITraceWriter trace,
-            TemplateSchema schema)
+            TemplateSchema schema,
+            IList<String> fileTable)
         {
             if (!String.Equals(schema.Version, PipelineTemplateConstants.Workflow_1_0, StringComparison.Ordinal))
             {
@@ -28,6 +29,7 @@ namespace GitHub.DistributedTask.Pipelines.ObjectTemplating
 
             m_trace = trace;
             m_schema = schema;
+            m_fileTable = fileTable;
         }
 
         public Int32 MaxDepth => 50;
@@ -324,6 +326,16 @@ namespace GitHub.DistributedTask.Pipelines.ObjectTemplating
                 TraceWriter = m_trace,
             };
 
+            // Add the file table
+            if (m_fileTable?.Count > 0)
+            {
+                foreach (var file in m_fileTable)
+                {
+                    result.GetFileId(file);
+                }
+            }
+
+            // Add named context
             if (contextData != null)
             {
                 foreach (var pair in contextData)
@@ -346,6 +358,7 @@ namespace GitHub.DistributedTask.Pipelines.ObjectTemplating
 
         private readonly ITraceWriter m_trace;
         private readonly TemplateSchema m_schema;
+        private readonly IList<String> m_fileTable;
         private readonly String[] s_contextNames = new[]
         {
             PipelineTemplateConstants.GitHub,

src/Sdk/DTPipelines/Pipelines/ObjectTemplating/YamlObjectReader.cs

@@ -0,0 +1,572 @@
+using System;
+using System.Globalization;
+using System.IO;
+using System.Linq;
+using GitHub.DistributedTask.ObjectTemplating;
+using GitHub.DistributedTask.ObjectTemplating.Tokens;
+using YamlDotNet.Core;
+using YamlDotNet.Core.Events;
+
+namespace GitHub.DistributedTask.Pipelines.ObjectTemplating
+{
+    /// <summary>
+    /// Converts a YAML file into a TemplateToken
+    /// </summary>
+    public sealed class YamlObjectReader : IObjectReader
+    {
+        internal YamlObjectReader(
+            Int32? fileId,
+            TextReader input)
+        {
+            m_fileId = fileId;
+            m_parser = new Parser(input);
+        }
+
+        public Boolean AllowLiteral(out LiteralToken value)
+        {
+            if (EvaluateCurrent() is Scalar scalar)
+            {
+                // Tag specified
+                if (!String.IsNullOrEmpty(scalar.Tag))
+                {
+                    // String tag
+                    if (String.Equals(scalar.Tag, c_stringTag, StringComparison.Ordinal))
+                    {
+                        value = new StringToken(m_fileId, scalar.Start.Line, scalar.Start.Column, scalar.Value);
+                        MoveNext();
+                        return true;
+                    }
+
+                    // Not plain style
+                    if (scalar.Style != ScalarStyle.Plain)
+                    {
+                        throw new NotSupportedException($"The scalar style '{scalar.Style}' on line {scalar.Start.Line} and column {scalar.Start.Column} is not valid with the tag '{scalar.Tag}'");
+                    }
+
+                    // Boolean, Float, Integer, or Null
+                    switch (scalar.Tag)
+                    {
+                        case c_booleanTag:
+                            value = ParseBoolean(scalar);
+                            break;
+                        case c_floatTag:
+                            value = ParseFloat(scalar);
+                            break;
+                        case c_integerTag:
+                            value = ParseInteger(scalar);
+                            break;
+                        case c_nullTag:
+                            value = ParseNull(scalar);
+                            break;
+                        default:
+                            throw new NotSupportedException($"Unexpected tag '{scalar.Tag}'");
+                    }
+
+                    MoveNext();
+                    return true;
+                }
+
+                // Plain style, determine type using YAML 1.2 "core" schema https://yaml.org/spec/1.2/spec.html#id2804923
+                if (scalar.Style == ScalarStyle.Plain)
+                {
+                    if (MatchNull(scalar, out var nullToken))
+                    {
+                        value = nullToken;
+                    }
+                    else if (MatchBoolean(scalar, out var booleanToken))
+                    {
+                        value = booleanToken;
+                    }
+                    else if (MatchInteger(scalar, out var numberToken) ||
+                        MatchFloat(scalar, out numberToken))
+                    {
+                        value = numberToken;
+                    }
+                    else
+                    {
+                        value = new StringToken(m_fileId, scalar.Start.Line, scalar.Start.Column, scalar.Value);
+                    }
+
+                    MoveNext();
+                    return true;
+                }
+
+                // Otherwise assume string
+                value = new StringToken(m_fileId, scalar.Start.Line, scalar.Start.Column, scalar.Value);
+                MoveNext();
+                return true;
+            }
+
+            value = default;
+            return false;
+        }
+
+        public Boolean AllowSequenceStart(out SequenceToken value)
+        {
+            if (EvaluateCurrent() is SequenceStart sequenceStart)
+            {
+                value = new SequenceToken(m_fileId, sequenceStart.Start.Line, sequenceStart.Start.Column);
+                MoveNext();
+                return true;
+            }
+
+            value = default;
+            return false;
+        }
+
+        public Boolean AllowSequenceEnd()
+        {
+            if (EvaluateCurrent() is SequenceEnd)
+            {
+                MoveNext();
+                return true;
+            }
+
+            return false;
+        }
+
+        public Boolean AllowMappingStart(out MappingToken value)
+        {
+            if (EvaluateCurrent() is MappingStart mappingStart)
+            {
+                value = new MappingToken(m_fileId, mappingStart.Start.Line, mappingStart.Start.Column);
+                MoveNext();
+                return true;
+            }
+
+            value = default;
+            return false;
+        }
+
+        public Boolean AllowMappingEnd()
+        {
+            if (EvaluateCurrent() is MappingEnd)
+            {
+                MoveNext();
+                return true;
+            }
+
+            return false;
+        }
+
+        /// <summary>
+        /// Consumes the last parsing events, which are expected to be DocumentEnd and StreamEnd.
+        /// </summary>
+        public void ValidateEnd()
+        {
+            if (EvaluateCurrent() is DocumentEnd)
+            {
+                MoveNext();
+            }
+            else
+            {
+                throw new InvalidOperationException("Expected document end parse event");
+            }
+
+            if (EvaluateCurrent() is StreamEnd)
+            {
+                MoveNext();
+            }
+            else
+            {
+                throw new InvalidOperationException("Expected stream end parse event");
+            }
+
+            if (MoveNext())
+            {
+                throw new InvalidOperationException("Expected end of parse events");
+            }
+        }
+
+        /// <summary>
+        /// Consumes the first parsing events, which are expected to be StreamStart and DocumentStart.
+        /// </summary>
+        public void ValidateStart()
+        {
+            if (EvaluateCurrent() != null)
+            {
+                throw new InvalidOperationException("Unexpected parser state");
+            }
+
+            if (!MoveNext())
+            {
+                throw new InvalidOperationException("Expected a parse event");
+            }
+
+            if (EvaluateCurrent() is StreamStart)
+            {
+                MoveNext();
+            }
+            else
+            {
+                throw new InvalidOperationException("Expected stream start parse event");
+            }
+
+            if (EvaluateCurrent() is DocumentStart)
+            {
+                MoveNext();
+            }
+            else
+            {
+                throw new InvalidOperationException("Expected document start parse event");
+            }
+        }
+
+        private ParsingEvent EvaluateCurrent()
+        {
+            if (m_current == null)
+            {
+                m_current = m_parser.Current;
+                if (m_current != null)
+                {
+                    if (m_current is Scalar scalar)
+                    {
+                        // Verify not using achors
+                        if (scalar.Anchor != null)
+                        {
+                            throw new InvalidOperationException($"Anchors are not currently supported. Remove the anchor '{scalar.Anchor}'");
+                        }
+                    }
+                    else if (m_current is MappingStart mappingStart)
+                    {
+                        // Verify not using achors
+                        if (mappingStart.Anchor != null)
+                        {
+                            throw new InvalidOperationException($"Anchors are not currently supported. Remove the anchor '{mappingStart.Anchor}'");
+                        }
+                    }
+                    else if (m_current is SequenceStart sequenceStart)
+                    {
+                        // Verify not using achors
+                        if (sequenceStart.Anchor != null)
+                        {
+                            throw new InvalidOperationException($"Anchors are not currently supported. Remove the anchor '{sequenceStart.Anchor}'");
+                        }
+                    }
+                    else if (!(m_current is MappingEnd) &&
+                        !(m_current is SequenceEnd) &&
+                        !(m_current is DocumentStart) &&
+                        !(m_current is DocumentEnd) &&
+                        !(m_current is StreamStart) &&
+                        !(m_current is StreamEnd))
+                    {
+                        throw new InvalidOperationException($"Unexpected parsing event type: {m_current.GetType().Name}");
+                    }
+                }
+            }
+
+            return m_current;
+        }
+
+        private Boolean MoveNext()
+        {
+            m_current = null;
+            return m_parser.MoveNext();
+        }
+
+        private BooleanToken ParseBoolean(Scalar scalar)
+        {
+            if (MatchBoolean(scalar, out var token))
+            {
+                return token;
+            }
+
+            ThrowInvalidValue(scalar, c_booleanTag); // throws
+            return default;
+        }
+
+        private NumberToken ParseFloat(Scalar scalar)
+        {
+            if (MatchFloat(scalar, out var token))
+            {
+                return token;
+            }
+
+            ThrowInvalidValue(scalar, c_floatTag); // throws
+            return default;
+        }
+
+        private NumberToken ParseInteger(Scalar scalar)
+        {
+            if (MatchInteger(scalar, out var token))
+            {
+                return token;
+            }
+
+            ThrowInvalidValue(scalar, c_integerTag); // throws
+            return default;
+        }
+
+        private NullToken ParseNull(Scalar scalar)
+        {
+            if (MatchNull(scalar, out var token))
+            {
+                return token;
+            }
+
+            ThrowInvalidValue(scalar, c_nullTag); // throws
+            return default;
+        }
+
+        private Boolean MatchBoolean(
+            Scalar scalar,
+            out BooleanToken value)
+        {
+            // YAML 1.2 "core" schema https://yaml.org/spec/1.2/spec.html#id2804923
+            switch (scalar.Value ?? String.Empty)
+            {
+                case "true":
+                case "True":
+                case "TRUE":
+                    value = new BooleanToken(m_fileId, scalar.Start.Line, scalar.Start.Column, true);
+                    return true;
+                case "false":
+                case "False":
+                case "FALSE":
+                    value = new BooleanToken(m_fileId, scalar.Start.Line, scalar.Start.Column, false);
+                    return true;
+            }
+
+            value = default;
+            return false;
+        }
+
+        private Boolean MatchFloat(
+            Scalar scalar,
+            out NumberToken value)
+        {
+            // YAML 1.2 "core" schema https://yaml.org/spec/1.2/spec.html#id2804923
+            var str = scalar.Value;
+            if (!String.IsNullOrEmpty(str))
+            {
+                // Check for [-+]?(\.inf|\.Inf|\.INF)|\.nan|\.NaN|\.NAN
+                switch (str)
+                {
+                    case ".inf":
+                    case ".Inf":
+                    case ".INF":
+                    case "+.inf":
+                    case "+.Inf":
+                    case "+.INF":
+                        value = new NumberToken(m_fileId, scalar.Start.Line, scalar.Start.Column, Double.PositiveInfinity);
+                        return true;
+                    case "-.inf":
+                    case "-.Inf":
+                    case "-.INF":
+                        value = new NumberToken(m_fileId, scalar.Start.Line, scalar.Start.Column, Double.NegativeInfinity);
+                        return true;
+                    case ".nan":
+                    case ".NaN":
+                    case ".NAN":
+                        value = new NumberToken(m_fileId, scalar.Start.Line, scalar.Start.Column, Double.NaN);
+                        return true;
+                }
+
+                // Otherwise check [-+]?(\.[0-9]+|[0-9]+(\.[0-9]*)?)([eE][-+]?[0-9]+)?
+
+                // Skip leading sign
+                var index = str[0] == '-' || str[0] == '+' ? 1 : 0;
+
+                // Check for integer portion
+                var length = str.Length;
+                var hasInteger = false;
+                while (index < length && str[index] >= '0' && str[index] <= '9')
+                {
+                    hasInteger = true;
+                    index++;
+                }
+
+                // Check for decimal point
+                var hasDot = false;
+                if (index < length && str[index] == '.')
+                {
+                    hasDot = true;
+                    index++;
+                }
+
+                // Check for decimal portion
+                var hasDecimal = false;
+                while (index < length && str[index] >= '0' && str[index] <= '9')
+                {
+                    hasDecimal = true;
+                    index++;
+                }
+
+                // Check [-+]?(\.[0-9]+|[0-9]+(\.[0-9]*)?)
+                if ((hasDot && hasDecimal) || hasInteger)
+                {
+                    // Check for end
+                    if (index == length)
+                    {
+                        // Try parse
+                        if (Double.TryParse(str, NumberStyles.AllowLeadingSign | NumberStyles.AllowDecimalPoint, CultureInfo.InvariantCulture, out var doubleValue))
+                        {
+                            value = new NumberToken(m_fileId, scalar.Start.Line, scalar.Start.Column, doubleValue);
+                            return true;
+                        }
+                        // Otherwise exceeds range
+                        else
+                        {
+                            ThrowInvalidValue(scalar, c_floatTag); // throws
+                        }
+                    }
+                    // Check [eE][-+]?[0-9]
+                    else if (index < length && (str[index] == 'e' || str[index] == 'E'))
+                    {
+                        index++;
+
+                        // Skip sign
+                        if (index < length && (str[index] == '-' || str[index] == '+'))
+                        {
+                            index++;
+                        }
+
+                        // Check for exponent
+                        var hasExponent = false;
+                        while (index < length && str[index] >= '0' && str[index] <= '9')
+                        {
+                            hasExponent = true;
+                            index++;
+                        }
+
+                        // Check for end
+                        if (hasExponent && index == length)
+                        {
+                            // Try parse
+                            if (Double.TryParse(str, NumberStyles.AllowLeadingSign | NumberStyles.AllowDecimalPoint | NumberStyles.AllowExponent, CultureInfo.InvariantCulture, out var doubleValue))
+                            {
+                                value = new NumberToken(m_fileId, scalar.Start.Line, scalar.Start.Column, (Double)doubleValue);
+                                return true;
+                            }
+                            // Otherwise exceeds range
+                            else
+                            {
+                                ThrowInvalidValue(scalar, c_floatTag); // throws
+                            }
+                        }
+                    }
+                }
+            }
+
+            value = default;
+            return false;
+        }
+
+        private Boolean MatchInteger(
+            Scalar scalar,
+            out NumberToken value)
+        {
+            // YAML 1.2 "core" schema https://yaml.org/spec/1.2/spec.html#id2804923
+            var str = scalar.Value;
+            if (!String.IsNullOrEmpty(str))
+            {
+                // Check for [0-9]+
+                var firstChar = str[0];
+                if (firstChar >= '0' && firstChar <= '9' &&
+                    str.Skip(1).All(x => x >= '0' && x <= '9'))
+                {
+                    // Try parse
+                    if (Double.TryParse(str, NumberStyles.None, CultureInfo.InvariantCulture, out var doubleValue))
+                    {
+                        value = new NumberToken(m_fileId, scalar.Start.Line, scalar.Start.Column, doubleValue);
+                        return true;
+                    }
+
+                    // Otherwise exceeds range
+                    ThrowInvalidValue(scalar, c_integerTag); // throws
+                }
+                // Check for (-|+)[0-9]+
+                else if ((firstChar == '-' || firstChar == '+') &&
+                    str.Length > 1 &&
+                    str.Skip(1).All(x => x >= '0' && x <= '9'))
+                {
+                    // Try parse
+                    if (Double.TryParse(str, NumberStyles.AllowLeadingSign, CultureInfo.InvariantCulture, out var doubleValue))
+                    {
+                        value = new NumberToken(m_fileId, scalar.Start.Line, scalar.Start.Column, doubleValue);
+                        return true;
+                    }
+
+                    // Otherwise exceeds range
+                    ThrowInvalidValue(scalar, c_integerTag); // throws
+                }
+                // Check for 0x[0-9a-fA-F]+
+                else if (firstChar == '0' &&
+                    str.Length > 2 &&
+                    str[1] == 'x' &&
+                    str.Skip(2).All(x => (x >= '0' && x <= '9') || (x >= 'a' && x <= 'f') || (x >= 'A' && x <= 'F')))
+                {
+                    // Try parse
+                    if (Int32.TryParse(str.Substring(2), NumberStyles.AllowHexSpecifier, CultureInfo.InvariantCulture, out var integerValue))
+                    {
+                        value = new NumberToken(m_fileId, scalar.Start.Line, scalar.Start.Column, integerValue);
+                        return true;
+                    }
+
+                    // Otherwise exceeds range
+                    ThrowInvalidValue(scalar, c_integerTag); // throws
+                }
+                // Check for 0o[0-9]+
+                else if (firstChar == '0' &&
+                    str.Length > 2 &&
+                    str[1] == 'o' &&
+                    str.Skip(2).All(x => x >= '0' && x <= '7'))
+                {
+                    // Try parse
+                    var integerValue = default(Int32);
+                    try
+                    {
+                        integerValue = Convert.ToInt32(str.Substring(2), 8);
+                    }
+                    // Otherwise exceeds range
+                    catch (Exception)
+                    {
+                        ThrowInvalidValue(scalar, c_integerTag); // throws
+                    }
+
+                    value = new NumberToken(m_fileId, scalar.Start.Line, scalar.Start.Column, integerValue);
+                    return true;
+                }
+            }
+
+            value = default;
+            return false;
+        }
+
+        private Boolean MatchNull(
+            Scalar scalar,
+            out NullToken value)
+        {
+            // YAML 1.2 "core" schema https://yaml.org/spec/1.2/spec.html#id2804923
+            switch (scalar.Value ?? String.Empty)
+            {
+                case "":
+                case "null":
+                case "Null":
+                case "NULL":
+                case "~":
+                    value = new NullToken(m_fileId, scalar.Start.Line, scalar.Start.Column);
+                    return true;
+            }
+
+            value = default;
+            return false;
+        }
+
+        private void ThrowInvalidValue(
+            Scalar scalar,
+            String tag)
+        {
+            throw new NotSupportedException($"The value '{scalar.Value}' on line {scalar.Start.Line} and column {scalar.Start.Column} is invalid for the type '{scalar.Tag}'");
+        }
+
+        private const String c_booleanTag = "tag:yaml.org,2002:bool";
+        private const String c_floatTag = "tag:yaml.org,2002:float";
+        private const String c_integerTag = "tag:yaml.org,2002:int";
+        private const String c_nullTag = "tag:yaml.org,2002:null";
+        private const String c_stringTag = "tag:yaml.org,2002:string";
+        private readonly Int32? m_fileId;
+        private readonly Parser m_parser;
+        private ParsingEvent m_current;
+    }
+}

src/Test/L0/HostContextL0.cs

@@ -93,21 +93,12 @@ namespace GitHub.Runner.Common.Tests
                 Assert.Equal("123***123", _hc.SecretMasker.MaskSecrets("123Pass%20word%20123%21123"));
                 Assert.Equal("123***123", _hc.SecretMasker.MaskSecrets("123Pass<word>123!123"));
                 Assert.Equal("123***123", _hc.SecretMasker.MaskSecrets("123Pass''word''123!123"));
-                Assert.Equal("OlBh***Q==", _hc.SecretMasker.MaskSecrets(Convert.ToBase64String(Encoding.UTF8.GetBytes($":Password123!"))));
-                Assert.Equal("YTpQ***E=", _hc.SecretMasker.MaskSecrets(Convert.ToBase64String(Encoding.UTF8.GetBytes($"a:Password123!"))));
+                Assert.Equal("OlBh***", _hc.SecretMasker.MaskSecrets(Convert.ToBase64String(Encoding.UTF8.GetBytes($":Password123!"))));
+                Assert.Equal("YTpQ***", _hc.SecretMasker.MaskSecrets(Convert.ToBase64String(Encoding.UTF8.GetBytes($"a:Password123!"))));
                 Assert.Equal("YWI6***", _hc.SecretMasker.MaskSecrets(Convert.ToBase64String(Encoding.UTF8.GetBytes($"ab:Password123!"))));
-                Assert.Equal("YWJjOlBh***Q==", _hc.SecretMasker.MaskSecrets(Convert.ToBase64String(Encoding.UTF8.GetBytes($"abc:Password123!"))));
-                Assert.Equal("YWJjZDpQ***E=", _hc.SecretMasker.MaskSecrets(Convert.ToBase64String(Encoding.UTF8.GetBytes($"abcd:Password123!"))));
+                Assert.Equal("YWJjOlBh***", _hc.SecretMasker.MaskSecrets(Convert.ToBase64String(Encoding.UTF8.GetBytes($"abc:Password123!"))));
+                Assert.Equal("YWJjZDpQ***", _hc.SecretMasker.MaskSecrets(Convert.ToBase64String(Encoding.UTF8.GetBytes($"abcd:Password123!"))));
                 Assert.Equal("YWJjZGU6***", _hc.SecretMasker.MaskSecrets(Convert.ToBase64String(Encoding.UTF8.GetBytes($"abcde:Password123!"))));
-                Assert.Equal("***Og==", _hc.SecretMasker.MaskSecrets(Convert.ToBase64String(Encoding.UTF8.GetBytes($"Password123!:"))));
-                Assert.Equal("***OmE=", _hc.SecretMasker.MaskSecrets(Convert.ToBase64String(Encoding.UTF8.GetBytes($"Password123!:a"))));
-                Assert.Equal("***OmFi", _hc.SecretMasker.MaskSecrets(Convert.ToBase64String(Encoding.UTF8.GetBytes($"Password123!:ab"))));
-                Assert.Equal("***OmFiYw==", _hc.SecretMasker.MaskSecrets(Convert.ToBase64String(Encoding.UTF8.GetBytes($"Password123!:abc"))));
-                Assert.Equal("***OmFiY2Q=", _hc.SecretMasker.MaskSecrets(Convert.ToBase64String(Encoding.UTF8.GetBytes($"Password123!:abcd"))));
-                Assert.Equal("***OmFiY2Rl", _hc.SecretMasker.MaskSecrets(Convert.ToBase64String(Encoding.UTF8.GetBytes($"Password123!:abcde"))));
-                Assert.Equal("OlBh***To=", _hc.SecretMasker.MaskSecrets(Convert.ToBase64String(Encoding.UTF8.GetBytes($":Password123!:"))));
-                Assert.Equal("YTpQ***E6YQ==", _hc.SecretMasker.MaskSecrets(Convert.ToBase64String(Encoding.UTF8.GetBytes($"a:Password123!:a"))));
-                Assert.Equal("YWJjOlBh***Tph", _hc.SecretMasker.MaskSecrets(Convert.ToBase64String(Encoding.UTF8.GetBytes($"abc:Password123!:a"))));
             }
             finally
             {

src/Test/L0/Listener/Configuration/ConfigurationManagerL0.cs

@@ -175,8 +175,8 @@ namespace GitHub.Runner.Common.Tests.Listener.Configuration
                 Assert.True(s.PoolId.Equals(_expectedPoolId));
                 Assert.True(s.WorkFolder.Equals(_expectedWorkFolder));
 
-                // validate GetAgentPoolsAsync gets called once with automation pool type
-                _runnerServer.Verify(x => x.GetAgentPoolsAsync(It.IsAny<string>(), It.Is<TaskAgentPoolType>(p => p == TaskAgentPoolType.Automation)), Times.Once);
+                // validate GetAgentPoolsAsync gets called twice with automation pool type
+                _runnerServer.Verify(x => x.GetAgentPoolsAsync(It.IsAny<string>(), It.Is<TaskAgentPoolType>(p => p == TaskAgentPoolType.Automation)), Times.Exactly(2));
 
                 _runnerServer.Verify(x => x.AddAgentAsync(It.IsAny<int>(), It.Is<TaskAgent>(a => a.Labels.Contains("self-hosted") && a.Labels.Contains(VarUtil.OS) && a.Labels.Contains(VarUtil.OSArchitecture))), Times.Once);
             }

src/Test/L0/Listener/JobDispatcherL0.cs

@@ -33,7 +33,7 @@ namespace GitHub.Runner.Common.Tests.Listener
             TaskOrchestrationPlanReference plan = new TaskOrchestrationPlanReference();
             TimelineReference timeline = null;
             Guid jobId = Guid.NewGuid();
-            var result = new Pipelines.AgentJobRequestMessage(plan, timeline, jobId, "someJob", "someJob", null, null, null, new Dictionary<string, VariableValue>(), new List<MaskHint>(), new Pipelines.JobResources(), new Pipelines.ContextData.DictionaryContextData(), new Pipelines.WorkspaceOptions(), new List<Pipelines.ActionStep>(), null);
+            var result = new Pipelines.AgentJobRequestMessage(plan, timeline, jobId, "someJob", "someJob", null, null, null, new Dictionary<string, VariableValue>(), new List<MaskHint>(), new Pipelines.JobResources(), new Pipelines.ContextData.DictionaryContextData(), new Pipelines.WorkspaceOptions(), new List<Pipelines.ActionStep>(), null, null);
             result.ContextData["github"] = new Pipelines.ContextData.DictionaryContextData();
             return result;
         }

src/Test/L0/Listener/RunnerL0.cs

@@ -43,7 +43,7 @@ namespace GitHub.Runner.Common.Tests.Listener
             TaskOrchestrationPlanReference plan = new TaskOrchestrationPlanReference();
             TimelineReference timeline = null;
             Guid jobId = Guid.NewGuid();
-            return new Pipelines.AgentJobRequestMessage(plan, timeline, jobId, "test", "test", null, null, null, new Dictionary<string, VariableValue>(), new List<MaskHint>(), new Pipelines.JobResources(), new Pipelines.ContextData.DictionaryContextData(), new Pipelines.WorkspaceOptions(), new List<Pipelines.ActionStep>(), null);
+            return new Pipelines.AgentJobRequestMessage(plan, timeline, jobId, "test", "test", null, null, null, new Dictionary<string, VariableValue>(), new List<MaskHint>(), new Pipelines.JobResources(), new Pipelines.ContextData.DictionaryContextData(), new Pipelines.WorkspaceOptions(), new List<Pipelines.ActionStep>(), null, null);
         }
 
         private JobCancelMessage CreateJobCancelMessage()

src/Test/L0/Worker/ActionCommandManagerL0.cs

@@ -1,8 +1,10 @@
 using System;
 using System.Collections.Generic;
+using System.IO;
 using System.Runtime.CompilerServices;
 using GitHub.DistributedTask.WebApi;
 using GitHub.Runner.Worker;
+using GitHub.Runner.Worker.Container;
 using Moq;
 using Xunit;
 using Pipelines = GitHub.DistributedTask.Pipelines;
@@ -11,47 +13,35 @@ namespace GitHub.Runner.Common.Tests.Worker
 {
     public sealed class ActionCommandManagerL0
     {
+        private ActionCommandManager _commandManager;
+        private Mock<IExecutionContext> _ec;
+        private Mock<IExtensionManager> _extensionManager;
+        private Mock<IPipelineDirectoryManager> _pipelineDirectoryManager;
+
         [Fact]
         [Trait("Level", "L0")]
         [Trait("Category", "Worker")]
         public void EnablePluginInternalCommand()
         {
-            using (TestHostContext _hc = new TestHostContext(this))
+            using (TestHostContext hc = CreateTestContext())
             {
-                var extensionManger = new Mock<IExtensionManager>();
-                var directoryManager = new Mock<IPipelineDirectoryManager>();
-
-                var pluginCommand = new InternalPluginSetRepoPathCommandExtension();
-                pluginCommand.Initialize(_hc);
-
-                var envCommand = new SetEnvCommandExtension();
-                envCommand.Initialize(_hc);
-
-                extensionManger.Setup(x => x.GetExtensions<IActionCommandExtension>())
-                               .Returns(new List<IActionCommandExtension>() { pluginCommand, envCommand });
-                _hc.SetSingleton<IExtensionManager>(extensionManger.Object);
-                _hc.SetSingleton<IPipelineDirectoryManager>(directoryManager.Object);
-
-                Mock<IExecutionContext> _ec = new Mock<IExecutionContext>();
                 _ec.Setup(x => x.Write(It.IsAny<string>(), It.IsAny<string>()))
                    .Returns((string tag, string line) =>
                             {
-                                _hc.GetTrace().Info($"{tag} {line}");
+                                hc.GetTrace().Info($"{tag} {line}");
                                 return 1;
                             });
                 _ec.Setup(x => x.AddIssue(It.IsAny<Issue>(), It.IsAny<string>()))
                    .Callback((Issue issue, string message) =>
                    {
-                       _hc.GetTrace().Info($"{issue.Type} {issue.Message} {message ?? string.Empty}");
+                       hc.GetTrace().Info($"{issue.Type} {issue.Message} {message ?? string.Empty}");
                    });
-                ActionCommandManager commandManager = new ActionCommandManager();
-                commandManager.Initialize(_hc);
 
-                commandManager.EnablePluginInternalCommand();
+                _commandManager.EnablePluginInternalCommand();
 
-                Assert.True(commandManager.TryProcessCommand(_ec.Object, "##[internal-set-repo-path repoFullName=actions/runner;workspaceRepo=true]somepath"));
+                Assert.True(_commandManager.TryProcessCommand(_ec.Object, "##[internal-set-repo-path repoFullName=actions/runner;workspaceRepo=true]somepath", null));
 
-                directoryManager.Verify(x => x.UpdateRepositoryDirectory(_ec.Object, "actions/runner", "somepath", true), Times.Once);
+                _pipelineDirectoryManager.Verify(x => x.UpdateRepositoryDirectory(_ec.Object, "actions/runner", "somepath", true), Times.Once);
             }
         }
 
@@ -60,47 +50,29 @@ namespace GitHub.Runner.Common.Tests.Worker
         [Trait("Category", "Worker")]
         public void DisablePluginInternalCommand()
         {
-            using (TestHostContext _hc = new TestHostContext(this))
+            using (TestHostContext hc = CreateTestContext())
             {
-                var extensionManger = new Mock<IExtensionManager>();
-                var directoryManager = new Mock<IPipelineDirectoryManager>();
-
-                var pluginCommand = new InternalPluginSetRepoPathCommandExtension();
-                pluginCommand.Initialize(_hc);
-
-                var envCommand = new SetEnvCommandExtension();
-                envCommand.Initialize(_hc);
-
-                extensionManger.Setup(x => x.GetExtensions<IActionCommandExtension>())
-                               .Returns(new List<IActionCommandExtension>() { pluginCommand, envCommand });
-
-                _hc.SetSingleton<IExtensionManager>(extensionManger.Object);
-                _hc.SetSingleton<IPipelineDirectoryManager>(directoryManager.Object);
-
-                Mock<IExecutionContext> _ec = new Mock<IExecutionContext>();
                 _ec.Setup(x => x.Write(It.IsAny<string>(), It.IsAny<string>()))
                    .Returns((string tag, string line) =>
                             {
-                                _hc.GetTrace().Info($"{tag} {line}");
+                                hc.GetTrace().Info($"{tag} {line}");
                                 return 1;
                             });
                 _ec.Setup(x => x.AddIssue(It.IsAny<Issue>(), It.IsAny<string>()))
                    .Callback((Issue issue, string message) =>
                    {
-                       _hc.GetTrace().Info($"{issue.Type} {issue.Message} {message ?? string.Empty}");
+                       hc.GetTrace().Info($"{issue.Type} {issue.Message} {message ?? string.Empty}");
                    });
-                ActionCommandManager commandManager = new ActionCommandManager();
-                commandManager.Initialize(_hc);
 
-                commandManager.EnablePluginInternalCommand();
+                _commandManager.EnablePluginInternalCommand();
 
-                Assert.True(commandManager.TryProcessCommand(_ec.Object, "##[internal-set-repo-path repoFullName=actions/runner;workspaceRepo=true]somepath"));
+                Assert.True(_commandManager.TryProcessCommand(_ec.Object, "##[internal-set-repo-path repoFullName=actions/runner;workspaceRepo=true]somepath", null));
 
-                commandManager.DisablePluginInternalCommand();
+                _commandManager.DisablePluginInternalCommand();
 
-                Assert.False(commandManager.TryProcessCommand(_ec.Object, "##[internal-set-repo-path repoFullName=actions/runner;workspaceRepo=true]somepath"));
+                Assert.False(_commandManager.TryProcessCommand(_ec.Object, "##[internal-set-repo-path repoFullName=actions/runner;workspaceRepo=true]somepath", null));
 
-                directoryManager.Verify(x => x.UpdateRepositoryDirectory(_ec.Object, "actions/runner", "somepath", true), Times.Once);
+                _pipelineDirectoryManager.Verify(x => x.UpdateRepositoryDirectory(_ec.Object, "actions/runner", "somepath", true), Times.Once);
             }
         }
 
@@ -109,42 +81,27 @@ namespace GitHub.Runner.Common.Tests.Worker
         [Trait("Category", "Worker")]
         public void StopProcessCommand()
         {
-            using (TestHostContext _hc = new TestHostContext(this))
+            using (TestHostContext hc = CreateTestContext())
             {
-                var extensionManger = new Mock<IExtensionManager>();
-                var pluginCommand = new InternalPluginSetRepoPathCommandExtension();
-                pluginCommand.Initialize(_hc);
-
-                var envCommand = new SetEnvCommandExtension();
-                envCommand.Initialize(_hc);
-
-                extensionManger.Setup(x => x.GetExtensions<IActionCommandExtension>())
-                               .Returns(new List<IActionCommandExtension>() { pluginCommand, envCommand });
-                _hc.SetSingleton<IExtensionManager>(extensionManger.Object);
-
-                Mock<IExecutionContext> _ec = new Mock<IExecutionContext>();
                 _ec.Setup(x => x.Write(It.IsAny<string>(), It.IsAny<string>()))
                    .Returns((string tag, string line) =>
                             {
-                                _hc.GetTrace().Info($"{tag} {line}");
+                                hc.GetTrace().Info($"{tag} {line}");
                                 return 1;
                             });
 
                 _ec.Setup(x => x.AddIssue(It.IsAny<Issue>(), It.IsAny<string>()))
                    .Callback((Issue issue, string message) =>
                    {
-                       _hc.GetTrace().Info($"{issue.Type} {issue.Message} {message ?? string.Empty}");
+                       hc.GetTrace().Info($"{issue.Type} {issue.Message} {message ?? string.Empty}");
                    });
 
                 _ec.Setup(x => x.EnvironmentVariables).Returns(new Dictionary<string, string>());
 
-                ActionCommandManager commandManager = new ActionCommandManager();
-                commandManager.Initialize(_hc);
-
-                Assert.True(commandManager.TryProcessCommand(_ec.Object, "##[stop-commands]stopToken"));
-                Assert.False(commandManager.TryProcessCommand(_ec.Object, "##[set-env name=foo]bar"));
-                Assert.True(commandManager.TryProcessCommand(_ec.Object, "##[stopToken]"));
-                Assert.True(commandManager.TryProcessCommand(_ec.Object, "##[set-env name=foo]bar"));
+                Assert.True(_commandManager.TryProcessCommand(_ec.Object, "##[stop-commands]stopToken", null));
+                Assert.False(_commandManager.TryProcessCommand(_ec.Object, "##[set-env name=foo]bar", null));
+                Assert.True(_commandManager.TryProcessCommand(_ec.Object, "##[stopToken]", null));
+                Assert.True(_commandManager.TryProcessCommand(_ec.Object, "##[set-env name=foo]bar", null));
             }
         }
 
@@ -153,41 +110,29 @@ namespace GitHub.Runner.Common.Tests.Worker
         [Trait("Category", "Worker")]
         public void EchoProcessCommand()
         {
-            using (TestHostContext _hc = new TestHostContext(this))
+            using (TestHostContext hc = CreateTestContext())
             {
-                var extensionManager = new Mock<IExtensionManager>();
-                var echoCommand = new EchoCommandExtension();
-                echoCommand.Initialize(_hc);
-
-                extensionManager.Setup(x => x.GetExtensions<IActionCommandExtension>())
-                               .Returns(new List<IActionCommandExtension>() { echoCommand });
-                _hc.SetSingleton<IExtensionManager>(extensionManager.Object);
-
-                Mock<IExecutionContext> _ec = new Mock<IExecutionContext>();
                 _ec.Setup(x => x.Write(It.IsAny<string>(), It.IsAny<string>()))
                    .Returns((string tag, string line) =>
                             {
-                                _hc.GetTrace().Info($"{tag} {line}");
+                                hc.GetTrace().Info($"{tag} {line}");
                                 return 1;
                             });
 
                 _ec.SetupAllProperties();
 
-                ActionCommandManager commandManager = new ActionCommandManager();
-                commandManager.Initialize(_hc);
-
                 Assert.False(_ec.Object.EchoOnActionCommand);
 
-                Assert.True(commandManager.TryProcessCommand(_ec.Object, "::echo::on"));
+                Assert.True(_commandManager.TryProcessCommand(_ec.Object, "::echo::on", null));
                 Assert.True(_ec.Object.EchoOnActionCommand);
 
-                Assert.True(commandManager.TryProcessCommand(_ec.Object, "::echo::off"));
+                Assert.True(_commandManager.TryProcessCommand(_ec.Object, "::echo::off", null));
                 Assert.False(_ec.Object.EchoOnActionCommand);
 
-                Assert.True(commandManager.TryProcessCommand(_ec.Object, "::echo::ON"));
+                Assert.True(_commandManager.TryProcessCommand(_ec.Object, "::echo::ON", null));
                 Assert.True(_ec.Object.EchoOnActionCommand);
 
-                Assert.True(commandManager.TryProcessCommand(_ec.Object, "::echo::Off   "));
+                Assert.True(_commandManager.TryProcessCommand(_ec.Object, "::echo::Off   ", null));
                 Assert.False(_ec.Object.EchoOnActionCommand);
             }
         }
@@ -197,7 +142,7 @@ namespace GitHub.Runner.Common.Tests.Worker
         [Trait("Category", "Worker")]
         public void EchoProcessCommandDebugOn()
         {
-            using (TestHostContext _hc = new TestHostContext(this))
+            using (TestHostContext hc = CreateTestContext())
             {
                 // Set up a few things
                 // 1. Job request message (with ACTIONS_STEP_DEBUG = true)
@@ -205,7 +150,7 @@ namespace GitHub.Runner.Common.Tests.Worker
                 TimelineReference timeline = new TimelineReference();
                 Guid jobId = Guid.NewGuid();
                 string jobName = "some job name";
-                var jobRequest = new Pipelines.AgentJobRequestMessage(plan, timeline, jobId, jobName, jobName, null, null, null, new Dictionary<string, VariableValue>(), new List<MaskHint>(), new Pipelines.JobResources(), new Pipelines.ContextData.DictionaryContextData(), new Pipelines.WorkspaceOptions(), new List<Pipelines.ActionStep>(), null);
+                var jobRequest = new Pipelines.AgentJobRequestMessage(plan, timeline, jobId, jobName, jobName, null, null, null, new Dictionary<string, VariableValue>(), new List<MaskHint>(), new Pipelines.JobResources(), new Pipelines.ContextData.DictionaryContextData(), new Pipelines.WorkspaceOptions(), new List<Pipelines.ActionStep>(), null, null);
                 jobRequest.Resources.Repositories.Add(new Pipelines.RepositoryResource()
                 {
                     Alias = Pipelines.PipelineConstants.SelfAlias,
@@ -219,84 +164,135 @@ namespace GitHub.Runner.Common.Tests.Worker
                 var jobServerQueue = new Mock<IJobServerQueue>();
                 jobServerQueue.Setup(x => x.QueueTimelineRecordUpdate(It.IsAny<Guid>(), It.IsAny<TimelineRecord>()));
 
-                _hc.SetSingleton(jobServerQueue.Object);
-
-                var extensionManager = new Mock<IExtensionManager>();
-                var echoCommand = new EchoCommandExtension();
-                echoCommand.Initialize(_hc);
-
-                extensionManager.Setup(x => x.GetExtensions<IActionCommandExtension>())
-                               .Returns(new List<IActionCommandExtension>() { echoCommand });
-                _hc.SetSingleton<IExtensionManager>(extensionManager.Object);
+                hc.SetSingleton(jobServerQueue.Object);
 
                 var configurationStore = new Mock<IConfigurationStore>();
                 configurationStore.Setup(x => x.GetSettings()).Returns(new RunnerSettings());
-                _hc.SetSingleton(configurationStore.Object);
+                hc.SetSingleton(configurationStore.Object);
 
                 var pagingLogger = new Mock<IPagingLogger>();
-                _hc.EnqueueInstance(pagingLogger.Object);
-
-                ActionCommandManager commandManager = new ActionCommandManager();
-                commandManager.Initialize(_hc);
-
-                var _ec = new Runner.Worker.ExecutionContext();
-                _ec.Initialize(_hc);
+                hc.EnqueueInstance(pagingLogger.Object);
 
                 // Initialize the job (to exercise logic that sets EchoOnActionCommand)
-                _ec.InitializeJob(jobRequest, System.Threading.CancellationToken.None);
+                var ec = new Runner.Worker.ExecutionContext();
+                ec.Initialize(hc);
+                ec.InitializeJob(jobRequest, System.Threading.CancellationToken.None);
 
-                _ec.Complete();
+                ec.Complete();
 
-                Assert.True(_ec.EchoOnActionCommand);
+                Assert.True(ec.EchoOnActionCommand);
 
-                Assert.True(commandManager.TryProcessCommand(_ec, "::echo::off"));
-                Assert.False(_ec.EchoOnActionCommand);
+                Assert.True(_commandManager.TryProcessCommand(ec, "::echo::off", null));
+                Assert.False(ec.EchoOnActionCommand);
 
-                Assert.True(commandManager.TryProcessCommand(_ec, "::echo::on"));
-                Assert.True(_ec.EchoOnActionCommand);
+                Assert.True(_commandManager.TryProcessCommand(ec, "::echo::on", null));
+                Assert.True(ec.EchoOnActionCommand);
             }
         }
 
-
         [Fact]
         [Trait("Level", "L0")]
         [Trait("Category", "Worker")]
         public void EchoProcessCommandInvalid()
         {
-            using (TestHostContext _hc = new TestHostContext(this))
+            using (TestHostContext hc = CreateTestContext())
             {
-                var extensionManager = new Mock<IExtensionManager>();
-                var echoCommand = new EchoCommandExtension();
-                echoCommand.Initialize(_hc);
-
-                extensionManager.Setup(x => x.GetExtensions<IActionCommandExtension>())
-                               .Returns(new List<IActionCommandExtension>() { echoCommand });
-                _hc.SetSingleton<IExtensionManager>(extensionManager.Object);
-
-                Mock<IExecutionContext> _ec = new Mock<IExecutionContext>();
                 _ec.Setup(x => x.Write(It.IsAny<string>(), It.IsAny<string>()))
                    .Returns((string tag, string line) =>
                             {
-                                _hc.GetTrace().Info($"{tag} {line}");
+                                hc.GetTrace().Info($"{tag} {line}");
                                 return 1;
                             });
 
                 _ec.SetupAllProperties();
 
-                ActionCommandManager commandManager = new ActionCommandManager();
-                commandManager.Initialize(_hc);
-
                 // Echo commands below are considered "processed", but are invalid
                 // 1. Invalid echo value
-                Assert.True(commandManager.TryProcessCommand(_ec.Object, "::echo::invalid"));
+                Assert.True(_commandManager.TryProcessCommand(_ec.Object, "::echo::invalid", null));
                 Assert.Equal(TaskResult.Failed, _ec.Object.CommandResult);
                 Assert.False(_ec.Object.EchoOnActionCommand);
 
                 // 2. No value
-                Assert.True(commandManager.TryProcessCommand(_ec.Object, "::echo::"));
+                Assert.True(_commandManager.TryProcessCommand(_ec.Object, "::echo::", null));
                 Assert.Equal(TaskResult.Failed, _ec.Object.CommandResult);
                 Assert.False(_ec.Object.EchoOnActionCommand);
             }
         }
+
+        [Fact]
+        [Trait("Level", "L0")]
+        [Trait("Category", "Worker")]
+        public void AddMatcherTranslatesFilePath()
+        {
+            using (TestHostContext hc = CreateTestContext())
+            {
+                // Create a problem matcher config file
+                var hostDirectory = hc.GetDirectory(WellKnownDirectory.Temp);
+                var hostFile = Path.Combine(hostDirectory, "my-matcher.json");
+                Directory.CreateDirectory(hostDirectory);
+                var content = @"
+{
+    ""problemMatcher"": [
+        {
+            ""owner"": ""my-matcher"",
+            ""pattern"": [
+                {
+                    ""regexp"": ""^ERROR: (.+)$"",
+                    ""message"": 1
+                }
+            ]
+        }
+    ]
+}";
+                File.WriteAllText(hostFile, content);
+
+                // Setup translation info
+                var container = new ContainerInfo();
+                var containerDirectory = "/some-container-directory";
+                var containerFile = Path.Combine(containerDirectory, "my-matcher.json");
+                container.AddPathTranslateMapping(hostDirectory, containerDirectory);
+
+                // Act
+                _commandManager.TryProcessCommand(_ec.Object, $"::add-matcher::{containerFile}", container);
+
+                // Assert
+                _ec.Verify(x => x.AddMatchers(It.IsAny<IssueMatchersConfig>()), Times.Once);
+            }
+        }
+
+        private TestHostContext CreateTestContext([CallerMemberName] string testName = "")
+        {
+            var hostContext = new TestHostContext(this, testName);
+
+            // Mock extension manager
+            _extensionManager = new Mock<IExtensionManager>();
+            var commands = new IActionCommandExtension[]
+            {
+                new AddMatcherCommandExtension(),
+                new EchoCommandExtension(),
+                new InternalPluginSetRepoPathCommandExtension(),
+                new SetEnvCommandExtension(),
+            };
+            foreach (var command in commands)
+            {
+                command.Initialize(hostContext);
+            }
+            _extensionManager.Setup(x => x.GetExtensions<IActionCommandExtension>())
+                .Returns(new List<IActionCommandExtension>(commands));
+            hostContext.SetSingleton<IExtensionManager>(_extensionManager.Object);
+
+            // Mock pipeline directory manager
+            _pipelineDirectoryManager = new Mock<IPipelineDirectoryManager>();
+            hostContext.SetSingleton<IPipelineDirectoryManager>(_pipelineDirectoryManager.Object);
+
+            // Execution context
+            _ec = new Mock<IExecutionContext>();
+
+            // Command manager
+            _commandManager = new ActionCommandManager();
+            _commandManager.Initialize(hostContext);
+
+            return hostContext;
+        }
     }
 }

src/Test/L0/Worker/ExecutionContextL0.cs

@@ -25,7 +25,7 @@ namespace GitHub.Runner.Common.Tests.Worker
                 TimelineReference timeline = new TimelineReference();
                 Guid jobId = Guid.NewGuid();
                 string jobName = "some job name";
-                var jobRequest = new Pipelines.AgentJobRequestMessage(plan, timeline, jobId, jobName, jobName, null, null, null, new Dictionary<string, VariableValue>(), new List<MaskHint>(), new Pipelines.JobResources(), new Pipelines.ContextData.DictionaryContextData(), new Pipelines.WorkspaceOptions(), new List<Pipelines.ActionStep>(), null);
+                var jobRequest = new Pipelines.AgentJobRequestMessage(plan, timeline, jobId, jobName, jobName, null, null, null, new Dictionary<string, VariableValue>(), new List<MaskHint>(), new Pipelines.JobResources(), new Pipelines.ContextData.DictionaryContextData(), new Pipelines.WorkspaceOptions(), new List<Pipelines.ActionStep>(), null, null);
                 jobRequest.Resources.Repositories.Add(new Pipelines.RepositoryResource()
                 {
                     Alias = Pipelines.PipelineConstants.SelfAlias,
@@ -101,7 +101,7 @@ namespace GitHub.Runner.Common.Tests.Worker
                 TimelineReference timeline = new TimelineReference();
                 Guid jobId = Guid.NewGuid();
                 string jobName = "some job name";
-                var jobRequest = new Pipelines.AgentJobRequestMessage(plan, timeline, jobId, jobName, jobName, null, null, null, new Dictionary<string, VariableValue>(), new List<MaskHint>(), new Pipelines.JobResources(), new Pipelines.ContextData.DictionaryContextData(), new Pipelines.WorkspaceOptions(), new List<Pipelines.ActionStep>(), null);
+                var jobRequest = new Pipelines.AgentJobRequestMessage(plan, timeline, jobId, jobName, jobName, null, null, null, new Dictionary<string, VariableValue>(), new List<MaskHint>(), new Pipelines.JobResources(), new Pipelines.ContextData.DictionaryContextData(), new Pipelines.WorkspaceOptions(), new List<Pipelines.ActionStep>(), null, null);
                 jobRequest.Resources.Repositories.Add(new Pipelines.RepositoryResource()
                 {
                     Alias = Pipelines.PipelineConstants.SelfAlias,
@@ -152,7 +152,7 @@ namespace GitHub.Runner.Common.Tests.Worker
                 TimelineReference timeline = new TimelineReference();
                 Guid jobId = Guid.NewGuid();
                 string jobName = "some job name";
-                var jobRequest = new Pipelines.AgentJobRequestMessage(plan, timeline, jobId, jobName, jobName, null, null, null, new Dictionary<string, VariableValue>(), new List<MaskHint>(), new Pipelines.JobResources(), new Pipelines.ContextData.DictionaryContextData(), new Pipelines.WorkspaceOptions(), new List<Pipelines.ActionStep>(), null);
+                var jobRequest = new Pipelines.AgentJobRequestMessage(plan, timeline, jobId, jobName, jobName, null, null, null, new Dictionary<string, VariableValue>(), new List<MaskHint>(), new Pipelines.JobResources(), new Pipelines.ContextData.DictionaryContextData(), new Pipelines.WorkspaceOptions(), new List<Pipelines.ActionStep>(), null, null);
                 jobRequest.Resources.Repositories.Add(new Pipelines.RepositoryResource()
                 {
                     Alias = Pipelines.PipelineConstants.SelfAlias,

src/Test/L0/Worker/JobExtensionL0.cs

@@ -100,7 +100,7 @@ namespace GitHub.Runner.Common.Tests.Worker
             };
 
             Guid jobId = Guid.NewGuid();
-            _message = new Pipelines.AgentJobRequestMessage(plan, timeline, jobId, "test", "test", null, null, null, new Dictionary<string, VariableValue>(), new List<MaskHint>(), new Pipelines.JobResources(), new Pipelines.ContextData.DictionaryContextData(), new Pipelines.WorkspaceOptions(), steps, null);
+            _message = new Pipelines.AgentJobRequestMessage(plan, timeline, jobId, "test", "test", null, null, null, new Dictionary<string, VariableValue>(), new List<MaskHint>(), new Pipelines.JobResources(), new Pipelines.ContextData.DictionaryContextData(), new Pipelines.WorkspaceOptions(), steps, null, null);
             GitHubContext github = new GitHubContext();
             github["repository"] = new Pipelines.ContextData.StringContextData("actions/runner");
             _message.ContextData.Add("github", github);

src/Test/L0/Worker/JobRunnerL0.cs

@@ -63,7 +63,7 @@ namespace GitHub.Runner.Common.Tests.Worker
             TaskOrchestrationPlanReference plan = new TaskOrchestrationPlanReference();
             TimelineReference timeline = new Timeline(Guid.NewGuid());
             Guid jobId = Guid.NewGuid();
-            _message = new Pipelines.AgentJobRequestMessage(plan, timeline, jobId, testName, testName, null, null, null, new Dictionary<string, VariableValue>(), new List<MaskHint>(), new Pipelines.JobResources(), new Pipelines.ContextData.DictionaryContextData(), new Pipelines.WorkspaceOptions(), new List<Pipelines.ActionStep>(), null);
+            _message = new Pipelines.AgentJobRequestMessage(plan, timeline, jobId, testName, testName, null, null, null, new Dictionary<string, VariableValue>(), new List<MaskHint>(), new Pipelines.JobResources(), new Pipelines.ContextData.DictionaryContextData(), new Pipelines.WorkspaceOptions(), new List<Pipelines.ActionStep>(), null, null);
             _message.Variables[Constants.Variables.System.Culture] = "en-US";
             _message.Resources.Endpoints.Add(new ServiceEndpoint()
             {

src/Test/L0/Worker/OutputManagerL0.cs

@@ -973,8 +973,8 @@ namespace GitHub.Runner.Common.Tests.Worker
                 });
 
             _commandManager = new Mock<IActionCommandManager>();
-            _commandManager.Setup(x => x.TryProcessCommand(It.IsAny<IExecutionContext>(), It.IsAny<string>()))
-                .Returns((IExecutionContext executionContext, string line) =>
+            _commandManager.Setup(x => x.TryProcessCommand(It.IsAny<IExecutionContext>(), It.IsAny<string>(), It.IsAny<ContainerInfo>()))
+                .Returns((IExecutionContext executionContext, string line, ContainerInfo container) =>
                 {
                     if (line.IndexOf("##[some-command]") >= 0)
                     {

src/Test/L0/Worker/WorkerL0.cs

@@ -67,7 +67,7 @@ namespace GitHub.Runner.Common.Tests.Worker
                     new Pipelines.ContextData.DictionaryContextData()
                 },
             };
-            var jobRequest = new Pipelines.AgentJobRequestMessage(plan, timeline, JobId, jobName, jobName, new StringToken(null, null, null, "ubuntu"), sidecarContainers, null, variables, new List<MaskHint>(), resources, context, null, actions, null);
+            var jobRequest = new Pipelines.AgentJobRequestMessage(plan, timeline, JobId, jobName, jobName, new StringToken(null, null, null, "ubuntu"), sidecarContainers, null, variables, new List<MaskHint>(), resources, context, null, actions, null, null);
             return jobRequest;
         }
 

src/dev.cmd

@@ -1,5 +1,16 @@
 @setlocal
 @echo off
+ rem add expected utils to path
+IF EXIST C:\Program Files\Git\usr\bin (
+  SET PATH=C:\Program Files\Git\usr\bin;%PATH%
+)
+IF EXIST C:\Program Files\Git\mingw64\bin (
+  SET PATH=C:\Program Files\Git\mingw64\bin;%PATH%
+)
+IF EXIST C:\Program Files\Git\bin (
+  SET PATH=C:\Program Files\Git\bin;%PATH%
+)
+
  rem Check if SH_PATH is defined.
 if defined SH_PATH (
   goto run

src/runnerversion

@@ -1 +1 @@
-2.165.0
+2.165.2