More PR feedback, standardize 'OAuth token' terms

docs/adrs/0274-step-outcome-and-conclusion.md

@@ -1,62 +0,0 @@
-# ADR 0274: Step outcome and conclusion
-
-**Date**: 2020-01-13
-
-**Status**: Accepted
-
-## Context
-
-This ADR proposes adding `steps.<id>.outcome` and `steps.<id>.conclusion` to the steps context.
-
-This allows downstream a step to run based on whether a previous step succeeded or failed.
-
-Reminder, currently the steps contains `steps.<id>.outputs`.
-
-## Decision
-
-For steps that have completed, populate `steps.<id>.outcome` and `steps.<id>.conclusion` with one of the following values:
-
-- `success`
-- `failure`
-- `cancelled`
-- `skipped`
-
-When a continue-on-error step fails, the outcome will be `failure` even though the final conclusion is `success`.
-
-### Example
-
-```yaml
-steps:
-
-  - id: experimental
-    continue-on-error: true
-    run: ./build.sh experimental
-
-  - if: ${{ steps.experimental.outcome == 'success' }}
-    run: ./publish.sh experimental
-```
-
-### Terminology
-
-The runs API uses the term `conclusion`.
-
-Therefore we use a different term `outcome` for the value prior to continue-on-error.
-
-The following is a snippet from the runs API response payload:
-
-```json
-      "steps": [
-        {
-          "name": "Set up job",
-          "status": "completed",
-          "conclusion": "success",
-          "number": 1,
-          "started_at": "2020-01-09T11:06:16.000-05:00",
-          "completed_at": "2020-01-09T11:06:18.000-05:00"
-        },
-```
-
-## Consequences
-
-- Update runner
-- Update [docs](https://help.github.com/en/actions/automating-your-workflow-with-github-actions/contexts-and-expression-syntax-for-github-actions#steps-context)

docs/adrs/0354-runner-machine-info.md

@@ -1,35 +0,0 @@
-# ADR 354: Expose runner machine info
-
-**Date**: 2020-03-02
-
-**Status**: Pending
-
-## Context
-
-- Provide a mechanism in the runner to include extra information in `Set up job` step's log.
-  Ex: Include OS/Software info from Hosted image.
-
-## Decision
-
-The runner will look for a file `.setup_info` under the runner's root directory, The file can be a JSON with a simple schema.
-```json
-[
-  {
-    "group": "OS Detail",
-    "detail": "........"
-  },
-  {
-    "group": "Software Detail",
-    "detail": "........"
-  }
-]
-```
-The runner will use `##[group]` and `##[endgroup]` to fold all detail info into an expandable group.
-
-Both [virtual-environments](https://github.com/actions/virtual-environments) and self-hosted runners can use this mechanism to add extra logging info to the `Set up job` step's log.
-
-## Consequences
-
-1. Change the runner to best effort read/parse `.extra_setup_info` file under runner root directory.
-2. [virtual-environments](https://github.com/actions/virtual-environments) generate the file during image generation.
-3. Change MMS provisioner to properly copy the file to runner root directory at runtime.

docs/contribute.md

@@ -44,7 +44,7 @@ Sample developer flow:
 ```bash
 git clone https://github.com/actions/runner
 cd ./src
-./dev.(sh/cmd) layout # the runner that built from source is in {root}/_layout
+./dev.(sh/cmd) layout # the runner that build from source is in {root}/_layout
 <make code changes>
 ./dev.(sh/cmd) build # {root}/_layout will get updated
 ./dev.(sh/cmd) test # run all unit tests before git commit/push

releaseNote.md

@@ -1,24 +1,27 @@
 ## Features
-  - Update Runner Register GitHub API URL to Support Org-level Runner (#339 #345 #352) 
+  - Expose whether debug is on/off via RUNNER_DEBUG. (#253)
-  - Preserve workflow file/line/column for better error messages (#356)
+  - Upload log on runner when worker get killed due to cancellation timeout. (#255)
-  - Switch to use token service instead of SPS for exchanging oauth token. (#325)
+  - Update config.sh/cmd --help documentation (#282) 
-  - Load and print machine setup info from .setup_info (#364)
+  - Set http_proxy and related env vars for job/service containers (#304)
-  - Expose job name as $GITHUB_JOB (#366)
+  - Set both http_proxy and HTTP_PROXY env for runner/worker processes. (#298)
-  - Add support for job outputs. (#365) 
-  - Set CI=true when launch process in actions runner. (#374)
-  - Set steps.<id>.outcome and steps.<id>.conclusion. (#372)
-  - Add support for workflow/job defaults. (#369)
-  - Expose GITHUB_REPOSITORY_OWNER and ${{github.repository_owner}}. (#378)
 
 ## Bugs
-  - Use authenticate endpoint for testing runner connection. (#311) 
+  - Verify runner Windows service hash started successfully after configuration (#236)
-  - Commands translate file path from container action (#331)
+  - Detect source file path in L0 without using env. (#257)
-  - Change problem matchers output to debug (#363)
+  - Handle escaped '%' in commands data section (#200)
-  - Switch hashFiles to extension function (#362)
+  - Allow container to be null/empty during matrix expansion (#266)
-  - Add expanded volumes strings to container mounts (#384)
+  - Translate problem matcher file to host path (#272)
+  - Change hashFiles() expression function to use @actions/glob. (#268)
+  - Default post-job action's condition to always(). (#293)
+  - Support action.yaml file as action's entry file (#288) 
+  - Trace javascript action exit code to debug instead of user logs (#290)
+  - Change prompt message when removing a runner to lines up with GitHub.com UI (#303) 
+  - Include step.env as part of env context. (#300)
+  - Update Base64 Encoders to deal with suffixes (#284)
 
 ## Misc
-  - Add runner auth documentation (#357) 
+  - Move .sln file under ./src (#238)
+  - Treat warnings as errors during compile (#249) 
 
 ## Windows x64
 We recommend configuring the runner in a root folder of the Windows drive (e.g. "C:\actions-runner"). This will help avoid issues related to service identity folder permissions and long file path restrictions on Windows

releaseVersion

@@ -1 +1 @@
-2.168.0
+2.164.0

src/Runner.Common/ConfigurationStore.cs

@@ -78,10 +78,8 @@ namespace GitHub.Runner.Common
         bool IsServiceConfigured();
         bool HasCredentials();
         CredentialData GetCredentials();
-        CredentialData GetMigratedCredentials();
         RunnerSettings GetSettings();
         void SaveCredential(CredentialData credential);
-        void SaveMigratedCredential(CredentialData credential);
         void SaveSettings(RunnerSettings settings);
         void DeleteCredential();
         void DeleteSettings();
@@ -92,11 +90,9 @@ namespace GitHub.Runner.Common
         private string _binPath;
         private string _configFilePath;
         private string _credFilePath;
-        private string _migratedCredFilePath;
         private string _serviceConfigFilePath;
 
         private CredentialData _creds;
-        private CredentialData _migratedCreds;
         private RunnerSettings _settings;
 
         public override void Initialize(IHostContext hostContext)
@@ -118,9 +114,6 @@ namespace GitHub.Runner.Common
             _credFilePath = hostContext.GetConfigFile(WellKnownConfigFile.Credentials);
             Trace.Info("CredFilePath: {0}", _credFilePath);
 
-            _migratedCredFilePath = hostContext.GetConfigFile(WellKnownConfigFile.MigratedCredentials);
-            Trace.Info("MigratedCredFilePath: {0}", _migratedCredFilePath);
-
             _serviceConfigFilePath = hostContext.GetConfigFile(WellKnownConfigFile.Service);
             Trace.Info("ServiceConfigFilePath: {0}", _serviceConfigFilePath);
         }
@@ -130,7 +123,7 @@ namespace GitHub.Runner.Common
         public bool HasCredentials()
         {
             Trace.Info("HasCredentials()");
-            bool credsStored = (new FileInfo(_credFilePath)).Exists || (new FileInfo(_migratedCredFilePath)).Exists;
+            bool credsStored = (new FileInfo(_credFilePath)).Exists;
             Trace.Info("stored {0}", credsStored);
             return credsStored;
         }
@@ -161,16 +154,6 @@ namespace GitHub.Runner.Common
             return _creds;
         }
 
-        public CredentialData GetMigratedCredentials()
-        {
-            if (_migratedCreds == null && File.Exists(_migratedCredFilePath))
-            {
-                _migratedCreds = IOUtil.LoadObject<CredentialData>(_migratedCredFilePath);
-            }
-
-            return _migratedCreds;
-        }
-
         public RunnerSettings GetSettings()
         {
             if (_settings == null)
@@ -205,21 +188,6 @@ namespace GitHub.Runner.Common
             File.SetAttributes(_credFilePath, File.GetAttributes(_credFilePath) | FileAttributes.Hidden);
         }
 
-        public void SaveMigratedCredential(CredentialData credential)
-        {
-            Trace.Info("Saving {0} migrated credential @ {1}", credential.Scheme, _migratedCredFilePath);
-            if (File.Exists(_migratedCredFilePath))
-            {
-                // Delete existing credential file first, since the file is hidden and not able to overwrite.
-                Trace.Info("Delete exist runner migrated credential file.");
-                IOUtil.DeleteFile(_migratedCredFilePath);
-            }
-
-            IOUtil.SaveObject(credential, _migratedCredFilePath);
-            Trace.Info("Migrated Credentials Saved.");
-            File.SetAttributes(_migratedCredFilePath, File.GetAttributes(_migratedCredFilePath) | FileAttributes.Hidden);
-        }
-
         public void SaveSettings(RunnerSettings settings)
         {
             Trace.Info("Saving runner settings.");
@@ -238,7 +206,6 @@ namespace GitHub.Runner.Common
         public void DeleteCredential()
         {
             IOUtil.Delete(_credFilePath, default(CancellationToken));
-            IOUtil.Delete(_migratedCredFilePath, default(CancellationToken));
         }
 
         public void DeleteSettings()

src/Runner.Common/Constants.cs

@@ -19,13 +19,11 @@ namespace GitHub.Runner.Common
     {
         Runner,
         Credentials,
-        MigratedCredentials,
         RSACredentials,
         Service,
         CredentialStore,
         Certificates,
         Options,
-        SetupInfo,
     }
 
     public static class Constants

src/Runner.Common/HostContext.cs

@@ -281,12 +281,6 @@ namespace GitHub.Runner.Common
                         ".credentials");
                     break;
 
-                case WellKnownConfigFile.MigratedCredentials:
-                    path = Path.Combine(
-                        GetDirectory(WellKnownDirectory.Root),
-                        ".credentials_migrated");
-                    break;
-
                 case WellKnownConfigFile.RSACredentials:
                     path = Path.Combine(
                         GetDirectory(WellKnownDirectory.Root),
@@ -322,13 +316,6 @@ namespace GitHub.Runner.Common
                         GetDirectory(WellKnownDirectory.Root),
                         ".options");
                     break;
-
-                case WellKnownConfigFile.SetupInfo:
-                    path = Path.Combine(
-                        GetDirectory(WellKnownDirectory.Root),
-                        ".setup_info");
-                    break;
-
                 default:
                     throw new NotSupportedException($"Unexpected well known config file: '{configFile}'");
             }

src/Runner.Common/RunnerServer.cs

@@ -50,10 +50,6 @@ namespace GitHub.Runner.Common
 
         // agent update
         Task<TaskAgent> UpdateAgentUpdateStateAsync(int agentPoolId, int agentId, string currentState);
-
-        // runner authorization url
-        Task<string> GetRunnerAuthUrlAsync(int runnerPoolId, int runnerId);
-        Task ReportRunnerAuthUrlErrorAsync(int runnerPoolId, int runnerId, string error);
     }
 
     public sealed class RunnerServer : RunnerService, IRunnerServer
@@ -338,20 +334,5 @@ namespace GitHub.Runner.Common
             CheckConnection(RunnerConnectionType.Generic);
             return _genericTaskAgentClient.UpdateAgentUpdateStateAsync(agentPoolId, agentId, currentState);
         }
-
-        //-----------------------------------------------------------------
-        // Runner Auth Url
-        //-----------------------------------------------------------------
-        public Task<string> GetRunnerAuthUrlAsync(int runnerPoolId, int runnerId)
-        {
-            CheckConnection(RunnerConnectionType.MessageQueue);
-            return _messageTaskAgentClient.GetAgentAuthUrlAsync(runnerPoolId, runnerId);
-        }
-
-        public Task ReportRunnerAuthUrlErrorAsync(int runnerPoolId, int runnerId, string error)
-        {
-            CheckConnection(RunnerConnectionType.MessageQueue);
-            return _messageTaskAgentClient.ReportAgentAuthUrlMigrationErrorAsync(runnerPoolId, runnerId, error);
-        }
     }
 }

src/Runner.Listener/Configuration/CredentialManager.cs

@@ -13,7 +13,7 @@ namespace GitHub.Runner.Listener.Configuration
     public interface ICredentialManager : IRunnerService
     {
         ICredentialProvider GetCredentialProvider(string credType);
-        VssCredentials LoadCredentials(bool preferMigrated = true);
+        VssCredentials LoadCredentials();
     }
 
     public class CredentialManager : RunnerService, ICredentialManager
@@ -40,7 +40,7 @@ namespace GitHub.Runner.Listener.Configuration
             return creds;
         }
 
-        public VssCredentials LoadCredentials(bool preferMigrated = true)
+        public VssCredentials LoadCredentials()
         {
             IConfigurationStore store = HostContext.GetService<IConfigurationStore>();
 
@@ -50,16 +50,6 @@ namespace GitHub.Runner.Listener.Configuration
             }
 
             CredentialData credData = store.GetCredentials();
-
-            if (preferMigrated)
-            {
-                var migratedCred = store.GetMigratedCredentials();
-                if (migratedCred != null)
-                {
-                    credData = migratedCred;
-                }
-            }
-
             ICredentialProvider credProv = GetCredentialProvider(credData.Scheme);
             credProv.CredentialData = credData;
 

src/Runner.Listener/Configuration/OAuthCredential.cs

@@ -1,5 +1,6 @@
 using System;
 using GitHub.Runner.Common;
+using GitHub.Runner.Common.Util;
 using GitHub.Runner.Sdk;
 using GitHub.Services.Common;
 using GitHub.Services.OAuth;
@@ -28,7 +29,7 @@ namespace GitHub.Runner.Listener.Configuration
             var authorizationUrl = this.CredentialData.Data.GetValueOrDefault("authorizationUrl", null);
 
             // For back compat with .credential file that doesn't has 'oauthEndpointUrl' section
-            var oauthEndpointUrl = this.CredentialData.Data.GetValueOrDefault("oauthEndpointUrl", authorizationUrl);
+            var oathEndpointUrl = this.CredentialData.Data.GetValueOrDefault("oauthEndpointUrl", authorizationUrl);
 
             ArgUtil.NotNullOrEmpty(clientId, nameof(clientId));
             ArgUtil.NotNullOrEmpty(authorizationUrl, nameof(authorizationUrl));
@@ -38,7 +39,7 @@ namespace GitHub.Runner.Listener.Configuration
             var keyManager = context.GetService<IRSAKeyManager>();
             var signingCredentials = VssSigningCredentials.Create(() => keyManager.GetKey());
             var clientCredential = new VssOAuthJwtBearerClientCredential(clientId, authorizationUrl, signingCredentials);
-            var agentCredential = new VssOAuthCredential(new Uri(oauthEndpointUrl, UriKind.Absolute), VssOAuthGrant.ClientCredentials, clientCredential);
+            var agentCredential = new VssOAuthCredential(new Uri(oathEndpointUrl, UriKind.Absolute), VssOAuthGrant.ClientCredentials, clientCredential);
 
             // Construct a credentials cache with a single OAuth credential for communication. The windows credential
             // is explicitly set to null to ensure we never do that negotiation.

src/Runner.Listener/JobDispatcher.cs

@@ -18,7 +18,6 @@ namespace GitHub.Runner.Listener
     [ServiceLocator(Default = typeof(JobDispatcher))]
     public interface IJobDispatcher : IRunnerService
     {
-        bool Busy { get; }
         TaskCompletionSource<bool> RunOnceJobCompleted { get; }
         void Run(Pipelines.AgentJobRequestMessage message, bool runOnce = false);
         bool Cancel(JobCancelMessage message);
@@ -70,8 +69,6 @@ namespace GitHub.Runner.Listener
 
         public TaskCompletionSource<bool> RunOnceJobCompleted => _runOnceJobCompleted;
 
-        public bool Busy { get; private set; }
-
         public void Run(Pipelines.AgentJobRequestMessage jobRequestMessage, bool runOnce = false)
         {
             Trace.Info($"Job request {jobRequestMessage.RequestId} for plan {jobRequestMessage.Plan.PlanId} job {jobRequestMessage.JobId} received.");
@@ -250,7 +247,7 @@ namespace GitHub.Runner.Listener
                     Task completedTask = await Task.WhenAny(jobDispatch.WorkerDispatch, Task.Delay(TimeSpan.FromSeconds(45)));
                     if (completedTask != jobDispatch.WorkerDispatch)
                     {
-                        // at this point, the job execution might encounter some dead lock and even not able to be cancelled.
+                        // at this point, the job exectuion might encounter some dead lock and even not able to be canclled.
                         // no need to localize the exception string should never happen.
                         throw new InvalidOperationException($"Job dispatch process for {jobDispatch.JobId} has encountered unexpected error, the dispatch task is not able to be canceled within 45 seconds.");
                     }
@@ -299,143 +296,269 @@ namespace GitHub.Runner.Listener
 
         private async Task RunAsync(Pipelines.AgentJobRequestMessage message, WorkerDispatcher previousJobDispatch, CancellationToken jobRequestCancellationToken, CancellationToken workerCancelTimeoutKillToken)
         {
-            Busy = true;
+            if (previousJobDispatch != null)
-            try
             {
-                if (previousJobDispatch != null)
+                Trace.Verbose($"Make sure the previous job request {previousJobDispatch.JobId} has successfully finished on worker.");
+                await EnsureDispatchFinished(previousJobDispatch);
+            }
+            else
+            {
+                Trace.Verbose($"This is the first job request.");
+            }
+
+            var term = HostContext.GetService<ITerminal>();
+            term.WriteLine($"{DateTime.UtcNow:u}: Running job: {message.JobDisplayName}");
+
+            // first job request renew succeed.
+            TaskCompletionSource<int> firstJobRequestRenewed = new TaskCompletionSource<int>();
+            var notification = HostContext.GetService<IJobNotification>();
+
+            // lock renew cancellation token.
+            using (var lockRenewalTokenSource = new CancellationTokenSource())
+            using (var workerProcessCancelTokenSource = new CancellationTokenSource())
+            {
+                long requestId = message.RequestId;
+                Guid lockToken = Guid.Empty; // lockToken has never been used, keep this here of compat
+
+                // start renew job request
+                Trace.Info($"Start renew job request {requestId} for job {message.JobId}.");
+                Task renewJobRequest = RenewJobRequestAsync(_poolId, requestId, lockToken, firstJobRequestRenewed, lockRenewalTokenSource.Token);
+
+                // wait till first renew succeed or job request is canceled
+                // not even start worker if the first renew fail
+                await Task.WhenAny(firstJobRequestRenewed.Task, renewJobRequest, Task.Delay(-1, jobRequestCancellationToken));
+
+                if (renewJobRequest.IsCompleted)
                 {
-                    Trace.Verbose($"Make sure the previous job request {previousJobDispatch.JobId} has successfully finished on worker.");
+                    // renew job request task complete means we run out of retry for the first job request renew.
-                    await EnsureDispatchFinished(previousJobDispatch);
+                    Trace.Info($"Unable to renew job request for job {message.JobId} for the first time, stop dispatching job to worker.");
-                }
+                    return;
-                else
-                {
-                    Trace.Verbose($"This is the first job request.");
                 }
 
-                var term = HostContext.GetService<ITerminal>();
+                if (jobRequestCancellationToken.IsCancellationRequested)
-                term.WriteLine($"{DateTime.UtcNow:u}: Running job: {message.JobDisplayName}");
-
-                // first job request renew succeed.
-                TaskCompletionSource<int> firstJobRequestRenewed = new TaskCompletionSource<int>();
-                var notification = HostContext.GetService<IJobNotification>();
-
-                // lock renew cancellation token.
-                using (var lockRenewalTokenSource = new CancellationTokenSource())
-                using (var workerProcessCancelTokenSource = new CancellationTokenSource())
                 {
-                    long requestId = message.RequestId;
+                    Trace.Info($"Stop renew job request for job {message.JobId}.");
-                    Guid lockToken = Guid.Empty; // lockToken has never been used, keep this here of compat
+                    // stop renew lock
+                    lockRenewalTokenSource.Cancel();
+                    // renew job request should never blows up.
+                    await renewJobRequest;
 
-                    // start renew job request
+                    // complete job request with result Cancelled
-                    Trace.Info($"Start renew job request {requestId} for job {message.JobId}.");
+                    await CompleteJobRequestAsync(_poolId, message, lockToken, TaskResult.Canceled);
-                    Task renewJobRequest = RenewJobRequestAsync(_poolId, requestId, lockToken, firstJobRequestRenewed, lockRenewalTokenSource.Token);
+                    return;
+                }
 
-                    // wait till first renew succeed or job request is canceled
+                HostContext.WritePerfCounter($"JobRequestRenewed_{requestId.ToString()}");
-                    // not even start worker if the first renew fail
-                    await Task.WhenAny(firstJobRequestRenewed.Task, renewJobRequest, Task.Delay(-1, jobRequestCancellationToken));
 
-                    if (renewJobRequest.IsCompleted)
+                Task<int> workerProcessTask = null;
+                object _outputLock = new object();
+                List<string> workerOutput = new List<string>();
+                using (var processChannel = HostContext.CreateService<IProcessChannel>())
+                using (var processInvoker = HostContext.CreateService<IProcessInvoker>())
+                {
+                    // Start the process channel.
+                    // It's OK if StartServer bubbles an execption after the worker process has already started.
+                    // The worker will shutdown after 30 seconds if it hasn't received the job message.
+                    processChannel.StartServer(
+                        // Delegate to start the child process.
+                        startProcess: (string pipeHandleOut, string pipeHandleIn) =>
+                        {
+                            // Validate args.
+                            ArgUtil.NotNullOrEmpty(pipeHandleOut, nameof(pipeHandleOut));
+                            ArgUtil.NotNullOrEmpty(pipeHandleIn, nameof(pipeHandleIn));
+
+                            // Save STDOUT from worker, worker will use STDOUT report unhandle exception.
+                            processInvoker.OutputDataReceived += delegate (object sender, ProcessDataReceivedEventArgs stdout)
+                            {
+                                if (!string.IsNullOrEmpty(stdout.Data))
+                                {
+                                    lock (_outputLock)
+                                    {
+                                        workerOutput.Add(stdout.Data);
+                                    }
+                                }
+                            };
+
+                            // Save STDERR from worker, worker will use STDERR on crash.
+                            processInvoker.ErrorDataReceived += delegate (object sender, ProcessDataReceivedEventArgs stderr)
+                            {
+                                if (!string.IsNullOrEmpty(stderr.Data))
+                                {
+                                    lock (_outputLock)
+                                    {
+                                        workerOutput.Add(stderr.Data);
+                                    }
+                                }
+                            };
+
+                            // Start the child process.
+                            HostContext.WritePerfCounter("StartingWorkerProcess");
+                            var assemblyDirectory = HostContext.GetDirectory(WellKnownDirectory.Bin);
+                            string workerFileName = Path.Combine(assemblyDirectory, _workerProcessName);
+                            workerProcessTask = processInvoker.ExecuteAsync(
+                                workingDirectory: assemblyDirectory,
+                                fileName: workerFileName,
+                                arguments: "spawnclient " + pipeHandleOut + " " + pipeHandleIn,
+                                environment: null,
+                                requireExitCodeZero: false,
+                                outputEncoding: null,
+                                killProcessOnCancel: true,
+                                redirectStandardIn: null,
+                                inheritConsoleHandler: false,
+                                keepStandardInOpen: false,
+                                highPriorityProcess: true,
+                                cancellationToken: workerProcessCancelTokenSource.Token);
+                        });
+
+                    // Send the job request message.
+                    // Kill the worker process if sending the job message times out. The worker
+                    // process may have successfully received the job message.
+                    try
                     {
-                        // renew job request task complete means we run out of retry for the first job request renew.
+                        Trace.Info($"Send job request message to worker for job {message.JobId}.");
-                        Trace.Info($"Unable to renew job request for job {message.JobId} for the first time, stop dispatching job to worker.");
+                        HostContext.WritePerfCounter($"RunnerSendingJobToWorker_{message.JobId}");
-                        return;
+                        using (var csSendJobRequest = new CancellationTokenSource(_channelTimeout))
+                        {
+                            await processChannel.SendAsync(
+                                messageType: MessageType.NewJobRequest,
+                                body: JsonUtility.ToString(message),
+                                cancellationToken: csSendJobRequest.Token);
+                        }
                     }
-
+                    catch (OperationCanceledException)
-                    if (jobRequestCancellationToken.IsCancellationRequested)
                     {
+                        // message send been cancelled.
+                        // timeout 30 sec. kill worker.
+                        Trace.Info($"Job request message sending for job {message.JobId} been cancelled, kill running worker.");
+                        workerProcessCancelTokenSource.Cancel();
+                        try
+                        {
+                            await workerProcessTask;
+                        }
+                        catch (OperationCanceledException)
+                        {
+                            Trace.Info("worker process has been killed.");
+                        }
+
                         Trace.Info($"Stop renew job request for job {message.JobId}.");
                         // stop renew lock
                         lockRenewalTokenSource.Cancel();
                         // renew job request should never blows up.
                         await renewJobRequest;
 
-                        // complete job request with result Cancelled
+                        // not finish the job request since the job haven't run on worker at all, we will not going to set a result to server.
-                        await CompleteJobRequestAsync(_poolId, message, lockToken, TaskResult.Canceled);
                         return;
                     }
 
-                    HostContext.WritePerfCounter($"JobRequestRenewed_{requestId.ToString()}");
+                    // we get first jobrequest renew succeed and start the worker process with the job message.
+                    // send notification to machine provisioner.
+                    var systemConnection = message.Resources.Endpoints.SingleOrDefault(x => string.Equals(x.Name, WellKnownServiceEndpointNames.SystemVssConnection, StringComparison.OrdinalIgnoreCase));
+                    var accessToken = systemConnection?.Authorization?.Parameters["AccessToken"];
+                    notification.JobStarted(message.JobId, accessToken, systemConnection.Url);
 
-                    Task<int> workerProcessTask = null;
+                    HostContext.WritePerfCounter($"SentJobToWorker_{requestId.ToString()}");
-                    object _outputLock = new object();
+
-                    List<string> workerOutput = new List<string>();
+                    try
-                    using (var processChannel = HostContext.CreateService<IProcessChannel>())
-                    using (var processInvoker = HostContext.CreateService<IProcessInvoker>())
                     {
-                        // Start the process channel.
+                        TaskResult resultOnAbandonOrCancel = TaskResult.Succeeded;
-                        // It's OK if StartServer bubbles an execption after the worker process has already started.
+                        // wait for renewlock, worker process or cancellation token been fired.
-                        // The worker will shutdown after 30 seconds if it hasn't received the job message.
+                        var completedTask = await Task.WhenAny(renewJobRequest, workerProcessTask, Task.Delay(-1, jobRequestCancellationToken));
-                        processChannel.StartServer(
+                        if (completedTask == workerProcessTask)
-                            // Delegate to start the child process.
+                        {
-                            startProcess: (string pipeHandleOut, string pipeHandleIn) =>
+                            // worker finished successfully, complete job request with result, attach unhandled exception reported by worker, stop renew lock, job has finished.
+                            int returnCode = await workerProcessTask;
+                            Trace.Info($"Worker finished for job {message.JobId}. Code: " + returnCode);
+
+                            string detailInfo = null;
+                            if (!TaskResultUtil.IsValidReturnCode(returnCode))
                             {
-                                // Validate args.
+                                detailInfo = string.Join(Environment.NewLine, workerOutput);
-                                ArgUtil.NotNullOrEmpty(pipeHandleOut, nameof(pipeHandleOut));
+                                Trace.Info($"Return code {returnCode} indicate worker encounter an unhandled exception or app crash, attach worker stdout/stderr to JobRequest result.");
-                                ArgUtil.NotNullOrEmpty(pipeHandleIn, nameof(pipeHandleIn));
+                                await LogWorkerProcessUnhandledException(message, detailInfo);
+                            }
 
-                                // Save STDOUT from worker, worker will use STDOUT report unhandle exception.
+                            TaskResult result = TaskResultUtil.TranslateFromReturnCode(returnCode);
-                                processInvoker.OutputDataReceived += delegate (object sender, ProcessDataReceivedEventArgs stdout)
+                            Trace.Info($"finish job request for job {message.JobId} with result: {result}");
-                                    {
+                            term.WriteLine($"{DateTime.UtcNow:u}: Job {message.JobDisplayName} completed with result: {result}");
-                                        if (!string.IsNullOrEmpty(stdout.Data))
-                                        {
-                                            lock (_outputLock)
-                                            {
-                                                workerOutput.Add(stdout.Data);
-                                            }
-                                        }
-                                    };
 
-                                // Save STDERR from worker, worker will use STDERR on crash.
+                            Trace.Info($"Stop renew job request for job {message.JobId}.");
-                                processInvoker.ErrorDataReceived += delegate (object sender, ProcessDataReceivedEventArgs stderr)
+                            // stop renew lock
-                                    {
+                            lockRenewalTokenSource.Cancel();
-                                        if (!string.IsNullOrEmpty(stderr.Data))
+                            // renew job request should never blows up.
-                                        {
+                            await renewJobRequest;
-                                            lock (_outputLock)
-                                            {
-                                                workerOutput.Add(stderr.Data);
-                                            }
-                                        }
-                                    };
 
-                                // Start the child process.
+                            // complete job request
-                                HostContext.WritePerfCounter("StartingWorkerProcess");
+                            await CompleteJobRequestAsync(_poolId, message, lockToken, result, detailInfo);
-                                var assemblyDirectory = HostContext.GetDirectory(WellKnownDirectory.Bin);
-                                string workerFileName = Path.Combine(assemblyDirectory, _workerProcessName);
-                                workerProcessTask = processInvoker.ExecuteAsync(
-                                    workingDirectory: assemblyDirectory,
-                                    fileName: workerFileName,
-                                    arguments: "spawnclient " + pipeHandleOut + " " + pipeHandleIn,
-                                    environment: null,
-                                    requireExitCodeZero: false,
-                                    outputEncoding: null,
-                                    killProcessOnCancel: true,
-                                    redirectStandardIn: null,
-                                    inheritConsoleHandler: false,
-                                    keepStandardInOpen: false,
-                                    highPriorityProcess: true,
-                                    cancellationToken: workerProcessCancelTokenSource.Token);
-                            });
 
-                        // Send the job request message.
+                            // print out unhandled exception happened in worker after we complete job request.
-                        // Kill the worker process if sending the job message times out. The worker
+                            // when we run out of disk space, report back to server has higher priority.
-                        // process may have successfully received the job message.
+                            if (!string.IsNullOrEmpty(detailInfo))
+                            {
+                                Trace.Error("Unhandled exception happened in worker:");
+                                Trace.Error(detailInfo);
+                            }
+
+                            return;
+                        }
+                        else if (completedTask == renewJobRequest)
+                        {
+                            resultOnAbandonOrCancel = TaskResult.Abandoned;
+                        }
+                        else
+                        {
+                            resultOnAbandonOrCancel = TaskResult.Canceled;
+                        }
+
+                        // renew job request completed or job request cancellation token been fired for RunAsync(jobrequestmessage)
+                        // cancel worker gracefully first, then kill it after worker cancel timeout
                         try
                         {
-                            Trace.Info($"Send job request message to worker for job {message.JobId}.");
+                            Trace.Info($"Send job cancellation message to worker for job {message.JobId}.");
-                            HostContext.WritePerfCounter($"RunnerSendingJobToWorker_{message.JobId}");
+                            using (var csSendCancel = new CancellationTokenSource(_channelTimeout))
-                            using (var csSendJobRequest = new CancellationTokenSource(_channelTimeout))
                             {
+                                var messageType = MessageType.CancelRequest;
+                                if (HostContext.RunnerShutdownToken.IsCancellationRequested)
+                                {
+                                    switch (HostContext.RunnerShutdownReason)
+                                    {
+                                        case ShutdownReason.UserCancelled:
+                                            messageType = MessageType.RunnerShutdown;
+                                            break;
+                                        case ShutdownReason.OperatingSystemShutdown:
+                                            messageType = MessageType.OperatingSystemShutdown;
+                                            break;
+                                    }
+                                }
+
                                 await processChannel.SendAsync(
-                                    messageType: MessageType.NewJobRequest,
+                                    messageType: messageType,
-                                    body: JsonUtility.ToString(message),
+                                    body: string.Empty,
-                                    cancellationToken: csSendJobRequest.Token);
+                                    cancellationToken: csSendCancel.Token);
                             }
                         }
                         catch (OperationCanceledException)
                         {
                             // message send been cancelled.
-                            // timeout 30 sec. kill worker.
+                            Trace.Info($"Job cancel message sending for job {message.JobId} been cancelled, kill running worker.");
-                            Trace.Info($"Job request message sending for job {message.JobId} been cancelled, kill running worker.");
+                            workerProcessCancelTokenSource.Cancel();
+                            try
+                            {
+                                await workerProcessTask;
+                            }
+                            catch (OperationCanceledException)
+                            {
+                                Trace.Info("worker process has been killed.");
+                            }
+                        }
+
+                        // wait worker to exit 
+                        // if worker doesn't exit within timeout, then kill worker.
+                        completedTask = await Task.WhenAny(workerProcessTask, Task.Delay(-1, workerCancelTimeoutKillToken));
+
+                        // worker haven't exit within cancellation timeout.
+                        if (completedTask != workerProcessTask)
+                        {
+                            Trace.Info($"worker process for job {message.JobId} haven't exit within cancellation timout, kill running worker.");
                             workerProcessCancelTokenSource.Cancel();
                             try
                             {
@@ -446,165 +569,31 @@ namespace GitHub.Runner.Listener
                                 Trace.Info("worker process has been killed.");
                             }
 
-                            Trace.Info($"Stop renew job request for job {message.JobId}.");
+                            // When worker doesn't exit within cancel timeout, the runner will kill the worker process and worker won't finish upload job logs.
-                            // stop renew lock
+                            // The runner will try to upload these logs at this time.
-                            lockRenewalTokenSource.Cancel();
+                            await TryUploadUnfinishedLogs(message);
-                            // renew job request should never blows up.
-                            await renewJobRequest;
-
-                            // not finish the job request since the job haven't run on worker at all, we will not going to set a result to server.
-                            return;
                         }
 
-                        // we get first jobrequest renew succeed and start the worker process with the job message.
+                        Trace.Info($"finish job request for job {message.JobId} with result: {resultOnAbandonOrCancel}");
-                        // send notification to machine provisioner.
+                        term.WriteLine($"{DateTime.UtcNow:u}: Job {message.JobDisplayName} completed with result: {resultOnAbandonOrCancel}");
-                        var systemConnection = message.Resources.Endpoints.SingleOrDefault(x => string.Equals(x.Name, WellKnownServiceEndpointNames.SystemVssConnection, StringComparison.OrdinalIgnoreCase));
+                        // complete job request with cancel result, stop renew lock, job has finished.
-                        var accessToken = systemConnection?.Authorization?.Parameters["AccessToken"];
-                        notification.JobStarted(message.JobId, accessToken, systemConnection.Url);
 
-                        HostContext.WritePerfCounter($"SentJobToWorker_{requestId.ToString()}");
+                        Trace.Info($"Stop renew job request for job {message.JobId}.");
+                        // stop renew lock
+                        lockRenewalTokenSource.Cancel();
+                        // renew job request should never blows up.
+                        await renewJobRequest;
 
-                        try
+                        // complete job request
-                        {
+                        await CompleteJobRequestAsync(_poolId, message, lockToken, resultOnAbandonOrCancel);
-                            TaskResult resultOnAbandonOrCancel = TaskResult.Succeeded;
+                    }
-                            // wait for renewlock, worker process or cancellation token been fired.
+                    finally
-                            var completedTask = await Task.WhenAny(renewJobRequest, workerProcessTask, Task.Delay(-1, jobRequestCancellationToken));
+                    {
-                            if (completedTask == workerProcessTask)
+                        // This should be the last thing to run so we don't notify external parties until actually finished
-                            {
+                        await notification.JobCompleted(message.JobId);
-                                // worker finished successfully, complete job request with result, attach unhandled exception reported by worker, stop renew lock, job has finished.
-                                int returnCode = await workerProcessTask;
-                                Trace.Info($"Worker finished for job {message.JobId}. Code: " + returnCode);
-
-                                string detailInfo = null;
-                                if (!TaskResultUtil.IsValidReturnCode(returnCode))
-                                {
-                                    detailInfo = string.Join(Environment.NewLine, workerOutput);
-                                    Trace.Info($"Return code {returnCode} indicate worker encounter an unhandled exception or app crash, attach worker stdout/stderr to JobRequest result.");
-                                    await LogWorkerProcessUnhandledException(message, detailInfo);
-                                }
-
-                                TaskResult result = TaskResultUtil.TranslateFromReturnCode(returnCode);
-                                Trace.Info($"finish job request for job {message.JobId} with result: {result}");
-                                term.WriteLine($"{DateTime.UtcNow:u}: Job {message.JobDisplayName} completed with result: {result}");
-
-                                Trace.Info($"Stop renew job request for job {message.JobId}.");
-                                // stop renew lock
-                                lockRenewalTokenSource.Cancel();
-                                // renew job request should never blows up.
-                                await renewJobRequest;
-
-                                // complete job request
-                                await CompleteJobRequestAsync(_poolId, message, lockToken, result, detailInfo);
-
-                                // print out unhandled exception happened in worker after we complete job request.
-                                // when we run out of disk space, report back to server has higher priority.
-                                if (!string.IsNullOrEmpty(detailInfo))
-                                {
-                                    Trace.Error("Unhandled exception happened in worker:");
-                                    Trace.Error(detailInfo);
-                                }
-
-                                return;
-                            }
-                            else if (completedTask == renewJobRequest)
-                            {
-                                resultOnAbandonOrCancel = TaskResult.Abandoned;
-                            }
-                            else
-                            {
-                                resultOnAbandonOrCancel = TaskResult.Canceled;
-                            }
-
-                            // renew job request completed or job request cancellation token been fired for RunAsync(jobrequestmessage)
-                            // cancel worker gracefully first, then kill it after worker cancel timeout
-                            try
-                            {
-                                Trace.Info($"Send job cancellation message to worker for job {message.JobId}.");
-                                using (var csSendCancel = new CancellationTokenSource(_channelTimeout))
-                                {
-                                    var messageType = MessageType.CancelRequest;
-                                    if (HostContext.RunnerShutdownToken.IsCancellationRequested)
-                                    {
-                                        switch (HostContext.RunnerShutdownReason)
-                                        {
-                                            case ShutdownReason.UserCancelled:
-                                                messageType = MessageType.RunnerShutdown;
-                                                break;
-                                            case ShutdownReason.OperatingSystemShutdown:
-                                                messageType = MessageType.OperatingSystemShutdown;
-                                                break;
-                                        }
-                                    }
-
-                                    await processChannel.SendAsync(
-                                        messageType: messageType,
-                                        body: string.Empty,
-                                        cancellationToken: csSendCancel.Token);
-                                }
-                            }
-                            catch (OperationCanceledException)
-                            {
-                                // message send been cancelled.
-                                Trace.Info($"Job cancel message sending for job {message.JobId} been cancelled, kill running worker.");
-                                workerProcessCancelTokenSource.Cancel();
-                                try
-                                {
-                                    await workerProcessTask;
-                                }
-                                catch (OperationCanceledException)
-                                {
-                                    Trace.Info("worker process has been killed.");
-                                }
-                            }
-
-                            // wait worker to exit 
-                            // if worker doesn't exit within timeout, then kill worker.
-                            completedTask = await Task.WhenAny(workerProcessTask, Task.Delay(-1, workerCancelTimeoutKillToken));
-
-                            // worker haven't exit within cancellation timeout.
-                            if (completedTask != workerProcessTask)
-                            {
-                                Trace.Info($"worker process for job {message.JobId} haven't exit within cancellation timout, kill running worker.");
-                                workerProcessCancelTokenSource.Cancel();
-                                try
-                                {
-                                    await workerProcessTask;
-                                }
-                                catch (OperationCanceledException)
-                                {
-                                    Trace.Info("worker process has been killed.");
-                                }
-
-                                // When worker doesn't exit within cancel timeout, the runner will kill the worker process and worker won't finish upload job logs.
-                                // The runner will try to upload these logs at this time.
-                                await TryUploadUnfinishedLogs(message);
-                            }
-
-                            Trace.Info($"finish job request for job {message.JobId} with result: {resultOnAbandonOrCancel}");
-                            term.WriteLine($"{DateTime.UtcNow:u}: Job {message.JobDisplayName} completed with result: {resultOnAbandonOrCancel}");
-                            // complete job request with cancel result, stop renew lock, job has finished.
-
-                            Trace.Info($"Stop renew job request for job {message.JobId}.");
-                            // stop renew lock
-                            lockRenewalTokenSource.Cancel();
-                            // renew job request should never blows up.
-                            await renewJobRequest;
-
-                            // complete job request
-                            await CompleteJobRequestAsync(_poolId, message, lockToken, resultOnAbandonOrCancel);
-                        }
-                        finally
-                        {
-                            // This should be the last thing to run so we don't notify external parties until actually finished
-                            await notification.JobCompleted(message.JobId);
-                        }
                     }
                 }
             }
-            finally
-            {
-                Busy = false;
-            }
         }
 
         public async Task RenewJobRequestAsync(int poolId, long requestId, Guid lockToken, TaskCompletionSource<int> firstJobRequestRenewed, CancellationToken token)

src/Runner.Listener/MessageListener.cs

@@ -13,10 +13,7 @@ using System.Diagnostics;
 using System.Runtime.InteropServices;
 using GitHub.Runner.Common;
 using GitHub.Runner.Sdk;
-using GitHub.Services.WebApi;
-using System.Runtime.CompilerServices;
 
-[assembly: InternalsVisibleTo("Test")]
 namespace GitHub.Runner.Listener
 {
     [ServiceLocator(Default = typeof(MessageListener))]
@@ -35,30 +32,18 @@ namespace GitHub.Runner.Listener
         private ITerminal _term;
         private IRunnerServer _runnerServer;
         private TaskAgentSession _session;
-        private ICredentialManager _credMgr;
-        private IConfigurationStore _configStore;
         private TimeSpan _getNextMessageRetryInterval;
         private readonly TimeSpan _sessionCreationRetryInterval = TimeSpan.FromSeconds(30);
         private readonly TimeSpan _sessionConflictRetryLimit = TimeSpan.FromMinutes(4);
         private readonly TimeSpan _clockSkewRetryLimit = TimeSpan.FromMinutes(30);
         private readonly Dictionary<string, int> _sessionCreationExceptionTracker = new Dictionary<string, int>();
 
-        // Whether load credentials from .credentials_migrated file
-        internal bool _useMigratedCredentials;
-
-        // need to check auth url if there is only .credentials and auth schema is OAuth
-        internal bool _needToCheckAuthorizationUrlUpdate;
-        internal Task<VssCredentials> _authorizationUrlMigrationBackgroundTask;
-        internal Task _authorizationUrlRollbackReattemptDelayBackgroundTask;
-
         public override void Initialize(IHostContext hostContext)
         {
             base.Initialize(hostContext);
 
             _term = HostContext.GetService<ITerminal>();
             _runnerServer = HostContext.GetService<IRunnerServer>();
-            _credMgr = HostContext.GetService<ICredentialManager>();
-            _configStore = HostContext.GetService<IConfigurationStore>();
         }
 
         public async Task<Boolean> CreateSessionAsync(CancellationToken token)
@@ -73,8 +58,8 @@ namespace GitHub.Runner.Listener
 
             // Create connection.
             Trace.Info("Loading Credentials");
-            _useMigratedCredentials = !StringUtil.ConvertToBoolean(Environment.GetEnvironmentVariable("GITHUB_ACTIONS_RUNNER_SPSAUTHURL"));
+            var credMgr = HostContext.GetService<ICredentialManager>();
-            VssCredentials creds = _credMgr.LoadCredentials(_useMigratedCredentials);
+            VssCredentials creds = credMgr.LoadCredentials();
 
             var agent = new TaskAgentReference
             {
@@ -89,17 +74,6 @@ namespace GitHub.Runner.Listener
             string errorMessage = string.Empty;
             bool encounteringError = false;
 
-            var originalCreds = _configStore.GetCredentials();
-            var migratedCreds = _configStore.GetMigratedCredentials();
-            if (migratedCreds == null)
-            {
-                _useMigratedCredentials = false;
-                if (originalCreds.Scheme == Constants.Configuration.OAuth)
-                {
-                    _needToCheckAuthorizationUrlUpdate = true;
-                }
-            }
-
             while (true)
             {
                 token.ThrowIfCancellationRequested();
@@ -109,7 +83,7 @@ namespace GitHub.Runner.Listener
                     Trace.Info("Connecting to the Runner Server...");
                     await _runnerServer.ConnectAsync(new Uri(serverUrl), creds);
                     Trace.Info("VssConnection created");
-
+                    
                     _term.WriteLine();
                     _term.WriteSuccessMessage("Connected to GitHub");
                     _term.WriteLine();
@@ -127,12 +101,6 @@ namespace GitHub.Runner.Listener
                         encounteringError = false;
                     }
 
-                    if (_needToCheckAuthorizationUrlUpdate)
-                    {
-                        // start background task try to get new authorization url
-                        _authorizationUrlMigrationBackgroundTask = GetNewOAuthAuthorizationSetting(token);
-                    }
-
                     return true;
                 }
                 catch (OperationCanceledException) when (token.IsCancellationRequested)
@@ -152,21 +120,8 @@ namespace GitHub.Runner.Listener
 
                     if (!IsSessionCreationExceptionRetriable(ex))
                     {
-                        if (_useMigratedCredentials)
+                        _term.WriteError($"Failed to create session. {ex.Message}");
-                        {
+                        return false;
-                            // migrated credentials might cause lose permission during permission check,
-                            // we will force to use original credential and try again
-                            _useMigratedCredentials = false;
-                            var reattemptBackoff = BackoffTimerHelper.GetRandomBackoff(TimeSpan.FromHours(24), TimeSpan.FromHours(36));
-                            _authorizationUrlRollbackReattemptDelayBackgroundTask = HostContext.Delay(reattemptBackoff, token); // retry migrated creds in 24-36 hours.
-                            creds = _credMgr.LoadCredentials(false);
-                            Trace.Error("Fallback to original credentials and try again.");
-                        }
-                        else
-                        {
-                            _term.WriteError($"Failed to create session. {ex.Message}");
-                            return false;
-                        }
                     }
 
                     if (!encounteringError) //print the message only on the first error
@@ -227,51 +182,6 @@ namespace GitHub.Runner.Listener
                         encounteringError = false;
                         continuousError = 0;
                     }
-
-                    if (_needToCheckAuthorizationUrlUpdate &&
-                        _authorizationUrlMigrationBackgroundTask?.IsCompleted == true)
-                    {
-                        if (HostContext.GetService<IJobDispatcher>().Busy ||
-                            HostContext.GetService<ISelfUpdater>().Busy)
-                        {
-                            Trace.Info("Job or runner updates in progress, update credentials next time.");
-                        }
-                        else
-                        {
-                            try
-                            {
-                                var newCred = await _authorizationUrlMigrationBackgroundTask;
-                                await _runnerServer.ConnectAsync(new Uri(_settings.ServerUrl), newCred);
-                                Trace.Info("Updated connection to use migrated credential for next GetMessage call.");
-                                _useMigratedCredentials = true;
-                                _authorizationUrlMigrationBackgroundTask = null;
-                                _needToCheckAuthorizationUrlUpdate = false;
-                            }
-                            catch (Exception ex)
-                            {
-                                Trace.Error("Fail to refresh connection with new authorization url.");
-                                Trace.Error(ex);
-                            }
-                        }
-                    }
-
-                    if (_authorizationUrlRollbackReattemptDelayBackgroundTask?.IsCompleted == true)
-                    {
-                        try
-                        {
-                            // we rolled back to use original creds about 2 days before, now it's a good time to try migrated creds again.
-                            Trace.Info("Re-attempt to use migrated credential");
-                            var migratedCreds = _credMgr.LoadCredentials();
-                            await _runnerServer.ConnectAsync(new Uri(_settings.ServerUrl), migratedCreds);
-                            _useMigratedCredentials = true;
-                            _authorizationUrlRollbackReattemptDelayBackgroundTask = null;
-                        }
-                        catch (Exception ex)
-                        {
-                            Trace.Error("Fail to refresh connection with new authorization url on rollback reattempt.");
-                            Trace.Error(ex);
-                        }
-                    }
                 }
                 catch (OperationCanceledException) when (token.IsCancellationRequested)
                 {
@@ -295,21 +205,7 @@ namespace GitHub.Runner.Listener
                     }
                     else if (!IsGetNextMessageExceptionRetriable(ex))
                     {
-                        if (_useMigratedCredentials)
+                        throw;
-                        {
-                            // migrated credentials might cause lose permission during permission check,
-                            // we will force to use original credential and try again
-                            _useMigratedCredentials = false;
-                            var reattemptBackoff = BackoffTimerHelper.GetRandomBackoff(TimeSpan.FromHours(24), TimeSpan.FromHours(36));
-                            _authorizationUrlRollbackReattemptDelayBackgroundTask = HostContext.Delay(reattemptBackoff, token); // retry migrated creds in 24-36 hours.
-                            var originalCreds = _credMgr.LoadCredentials(false);
-                            await _runnerServer.ConnectAsync(new Uri(_settings.ServerUrl), originalCreds);
-                            Trace.Error("Fallback to original credentials and try again.");
-                        }
-                        else
-                        {
-                            throw;
-                        }
                     }
                     else
                     {
@@ -501,80 +397,5 @@ namespace GitHub.Runner.Listener
                 return true;
             }
         }
-
-        private async Task<VssCredentials> GetNewOAuthAuthorizationSetting(CancellationToken token)
-        {
-            Trace.Info("Start checking oauth authorization url update.");
-            while (true)
-            {
-                var backoff = BackoffTimerHelper.GetRandomBackoff(TimeSpan.FromMinutes(30), TimeSpan.FromMinutes(45));
-                await HostContext.Delay(backoff, token);
-
-                try
-                {
-                    var migratedAuthorizationUrl = await _runnerServer.GetRunnerAuthUrlAsync(_settings.PoolId, _settings.AgentId);
-                    if (!string.IsNullOrEmpty(migratedAuthorizationUrl))
-                    {
-                        var credData = _configStore.GetCredentials();
-                        var clientId = credData.Data.GetValueOrDefault("clientId", null);
-                        var currentAuthorizationUrl = credData.Data.GetValueOrDefault("authorizationUrl", null);
-                        Trace.Info($"Current authorization url: {currentAuthorizationUrl}, new authorization url: {migratedAuthorizationUrl}");
-
-                        if (string.Equals(currentAuthorizationUrl, migratedAuthorizationUrl, StringComparison.OrdinalIgnoreCase))
-                        {
-                            // We don't need to update credentials.
-                            Trace.Info("No needs to update authorization url");
-                            await Task.Delay(TimeSpan.FromMilliseconds(-1), token);
-                        }
-
-                        var keyManager = HostContext.GetService<IRSAKeyManager>();
-                        var signingCredentials = VssSigningCredentials.Create(() => keyManager.GetKey());
-
-                        var migratedClientCredential = new VssOAuthJwtBearerClientCredential(clientId, migratedAuthorizationUrl, signingCredentials);
-                        var migratedRunnerCredential = new VssOAuthCredential(new Uri(migratedAuthorizationUrl, UriKind.Absolute), VssOAuthGrant.ClientCredentials, migratedClientCredential);
-
-                        Trace.Info("Try connect service with Token Service OAuth endpoint.");
-                        var runnerServer = HostContext.CreateService<IRunnerServer>();
-                        await runnerServer.ConnectAsync(new Uri(_settings.ServerUrl), migratedRunnerCredential);
-                        await runnerServer.GetAgentPoolsAsync();
-                        Trace.Info($"Successfully connected service with new authorization url.");
-
-                        var migratedCredData = new CredentialData
-                        {
-                            Scheme = Constants.Configuration.OAuth,
-                            Data =
-                            {
-                                { "clientId", clientId },
-                                { "authorizationUrl", migratedAuthorizationUrl },
-                                { "oauthEndpointUrl", migratedAuthorizationUrl },
-                            },
-                        };
-
-                        _configStore.SaveMigratedCredential(migratedCredData);
-                        return migratedRunnerCredential;
-                    }
-                    else
-                    {
-                        Trace.Verbose("No authorization url updates");
-                    }
-                }
-                catch (Exception ex)
-                {
-                    Trace.Error("Fail to get/test new authorization url.");
-                    Trace.Error(ex);
-
-                    try
-                    {
-                        await _runnerServer.ReportRunnerAuthUrlErrorAsync(_settings.PoolId, _settings.AgentId, ex.ToString());
-                    }
-                    catch (Exception e)
-                    {
-                        // best effort
-                        Trace.Error("Fail to report the migration error");
-                        Trace.Error(e);
-                    }
-                }
-            }
-        }
     }
 }

src/Runner.Listener/SelfUpdater.cs

@@ -17,7 +17,6 @@ namespace GitHub.Runner.Listener
     [ServiceLocator(Default = typeof(SelfUpdater))]
     public interface ISelfUpdater : IRunnerService
     {
-        bool Busy { get; }
         Task<bool> SelfUpdate(AgentRefreshMessage updateMessage, IJobDispatcher jobDispatcher, bool restartInteractiveRunner, CancellationToken token);
     }
 
@@ -32,8 +31,6 @@ namespace GitHub.Runner.Listener
         private int _poolId;
         private int _agentId;
 
-        public bool Busy { get; private set; }
-
         public override void Initialize(IHostContext hostContext)
         {
             base.Initialize(hostContext);
@@ -48,60 +45,52 @@ namespace GitHub.Runner.Listener
 
         public async Task<bool> SelfUpdate(AgentRefreshMessage updateMessage, IJobDispatcher jobDispatcher, bool restartInteractiveRunner, CancellationToken token)
         {
-            Busy = true;
+            if (!await UpdateNeeded(updateMessage.TargetVersion, token))
-            try
             {
-                if (!await UpdateNeeded(updateMessage.TargetVersion, token))
+                Trace.Info($"Can't find available update package.");
-                {
+                return false;
-                    Trace.Info($"Can't find available update package.");
+            }
-                    return false;
-                }
 
-                Trace.Info($"An update is available.");
+            Trace.Info($"An update is available.");
 
-                // Print console line that warn user not shutdown runner.
+            // Print console line that warn user not shutdown runner.
-                await UpdateRunnerUpdateStateAsync("Runner update in progress, do not shutdown runner.");
+            await UpdateRunnerUpdateStateAsync("Runner update in progress, do not shutdown runner.");
-                await UpdateRunnerUpdateStateAsync($"Downloading {_targetPackage.Version} runner");
+            await UpdateRunnerUpdateStateAsync($"Downloading {_targetPackage.Version} runner");
 
-                await DownloadLatestRunner(token);
+            await DownloadLatestRunner(token);
-                Trace.Info($"Download latest runner and unzip into runner root.");
+            Trace.Info($"Download latest runner and unzip into runner root.");
 
-                // wait till all running job finish
+            // wait till all running job finish
-                await UpdateRunnerUpdateStateAsync("Waiting for current job finish running.");
+            await UpdateRunnerUpdateStateAsync("Waiting for current job finish running.");
 
-                await jobDispatcher.WaitAsync(token);
+            await jobDispatcher.WaitAsync(token);
-                Trace.Info($"All running job has exited.");
+            Trace.Info($"All running job has exited.");
 
-                // delete runner backup
+            // delete runner backup
-                DeletePreviousVersionRunnerBackup(token);
+            DeletePreviousVersionRunnerBackup(token);
-                Trace.Info($"Delete old version runner backup.");
+            Trace.Info($"Delete old version runner backup.");
 
-                // generate update script from template
+            // generate update script from template
-                await UpdateRunnerUpdateStateAsync("Generate and execute update script.");
+            await UpdateRunnerUpdateStateAsync("Generate and execute update script.");
 
-                string updateScript = GenerateUpdateScript(restartInteractiveRunner);
+            string updateScript = GenerateUpdateScript(restartInteractiveRunner);
-                Trace.Info($"Generate update script into: {updateScript}");
+            Trace.Info($"Generate update script into: {updateScript}");
 
-                // kick off update script
+            // kick off update script
-                Process invokeScript = new Process();
+            Process invokeScript = new Process();
 #if OS_WINDOWS
             invokeScript.StartInfo.FileName = WhichUtil.Which("cmd.exe", trace: Trace);
             invokeScript.StartInfo.Arguments = $"/c \"{updateScript}\"";
 #elif (OS_OSX || OS_LINUX)
-                invokeScript.StartInfo.FileName = WhichUtil.Which("bash", trace: Trace);
+            invokeScript.StartInfo.FileName = WhichUtil.Which("bash", trace: Trace);
-                invokeScript.StartInfo.Arguments = $"\"{updateScript}\"";
+            invokeScript.StartInfo.Arguments = $"\"{updateScript}\"";
 #endif
-                invokeScript.Start();
+            invokeScript.Start();
-                Trace.Info($"Update script start running");
+            Trace.Info($"Update script start running");
 
-                await UpdateRunnerUpdateStateAsync("Runner will exit shortly for update, should back online within 10 seconds.");
+            await UpdateRunnerUpdateStateAsync("Runner will exit shortly for update, should back online within 10 seconds.");
 
-                return true;
+            return true;
-            }
-            finally
-            {
-                Busy = false;
-            }
         }
 
         private async Task<bool> UpdateNeeded(string targetVersion, CancellationToken token)

src/Runner.Sdk/ProcessInvoker.cs

@@ -271,14 +271,6 @@ namespace GitHub.Runner.Sdk
             // Indicate GitHub Actions process.
             _proc.StartInfo.Environment["GITHUB_ACTIONS"] = "true";
 
-            // Set CI=true when no one else already set it.
-            // CI=true is common set in most CI provider in GitHub
-            if (!_proc.StartInfo.Environment.ContainsKey("CI") &&
-                Environment.GetEnvironmentVariable("CI") == null)
-            {
-                _proc.StartInfo.Environment["CI"] = "true";
-            }
-
             // Hook up the events.
             _proc.EnableRaisingEvents = true;
             _proc.Exited += ProcessExitedHandler;

src/Runner.Worker/ActionManifestManager.cs

@@ -22,11 +22,11 @@ namespace GitHub.Runner.Worker
     {
         ActionDefinitionData Load(IExecutionContext executionContext, string manifestFile);
 
-        List<string> EvaluateContainerArguments(IExecutionContext executionContext, SequenceToken token, IDictionary<string, PipelineContextData> extraExpressionValues);
+        List<string> EvaluateContainerArguments(IExecutionContext executionContext, SequenceToken token, IDictionary<string, PipelineContextData> contextData);
 
-        Dictionary<string, string> EvaluateContainerEnvironment(IExecutionContext executionContext, MappingToken token, IDictionary<string, PipelineContextData> extraExpressionValues);
+        Dictionary<string, string> EvaluateContainerEnvironment(IExecutionContext executionContext, MappingToken token, IDictionary<string, PipelineContextData> contextData);
 
-        string EvaluateDefaultInput(IExecutionContext executionContext, string inputName, TemplateToken token);
+        string EvaluateDefaultInput(IExecutionContext executionContext, string inputName, TemplateToken token, IDictionary<string, PipelineContextData> contextData);
     }
 
     public sealed class ActionManifestManager : RunnerService, IActionManifestManager
@@ -54,7 +54,7 @@ namespace GitHub.Runner.Worker
 
         public ActionDefinitionData Load(IExecutionContext executionContext, string manifestFile)
         {
-            var context = CreateContext(executionContext);
+            var context = CreateContext(executionContext, null);
             ActionDefinitionData actionDefinition = new ActionDefinitionData();
             try
             {
@@ -133,13 +133,13 @@ namespace GitHub.Runner.Worker
         public List<string> EvaluateContainerArguments(
             IExecutionContext executionContext,
             SequenceToken token,
-            IDictionary<string, PipelineContextData> extraExpressionValues)
+            IDictionary<string, PipelineContextData> contextData)
         {
             var result = new List<string>();
 
             if (token != null)
             {
-                var context = CreateContext(executionContext, extraExpressionValues);
+                var context = CreateContext(executionContext, contextData);
                 try
                 {
                     var evaluateResult = TemplateEvaluator.Evaluate(context, "container-runs-args", token, 0, null, omitHeader: true);
@@ -172,13 +172,13 @@ namespace GitHub.Runner.Worker
         public Dictionary<string, string> EvaluateContainerEnvironment(
             IExecutionContext executionContext,
             MappingToken token,
-            IDictionary<string, PipelineContextData> extraExpressionValues)
+            IDictionary<string, PipelineContextData> contextData)
         {
             var result = new Dictionary<string, string>(StringComparer.OrdinalIgnoreCase);
 
             if (token != null)
             {
-                var context = CreateContext(executionContext, extraExpressionValues);
+                var context = CreateContext(executionContext, contextData);
                 try
                 {
                     var evaluateResult = TemplateEvaluator.Evaluate(context, "container-runs-env", token, 0, null, omitHeader: true);
@@ -216,12 +216,13 @@ namespace GitHub.Runner.Worker
         public string EvaluateDefaultInput(
             IExecutionContext executionContext,
             string inputName,
-            TemplateToken token)
+            TemplateToken token,
+            IDictionary<string, PipelineContextData> contextData)
         {
             string result = "";
             if (token != null)
             {
-                var context = CreateContext(executionContext);
+                var context = CreateContext(executionContext, contextData);
                 try
                 {
                     var evaluateResult = TemplateEvaluator.Evaluate(context, "input-default-context", token, 0, null, omitHeader: true);
@@ -246,7 +247,7 @@ namespace GitHub.Runner.Worker
 
         private TemplateContext CreateContext(
             IExecutionContext executionContext,
-            IDictionary<string, PipelineContextData> extraExpressionValues = null)
+            IDictionary<string, PipelineContextData> contextData)
         {
             var result = new TemplateContext
             {
@@ -260,27 +261,14 @@ namespace GitHub.Runner.Worker
                 TraceWriter = executionContext.ToTemplateTraceWriter(),
             };
 
-            // Expression values from execution context
+            if (contextData?.Count > 0)
-            foreach (var pair in executionContext.ExpressionValues)
             {
-                result.ExpressionValues[pair.Key] = pair.Value;
+                foreach (var pair in contextData)
-            }
-
-            // Extra expression values
-            if (extraExpressionValues?.Count > 0)
-            {
-                foreach (var pair in extraExpressionValues)
                 {
                     result.ExpressionValues[pair.Key] = pair.Value;
                 }
             }
 
-            // Expression functions from execution context
-            foreach (var item in executionContext.ExpressionFunctions)
-            {
-                result.ExpressionFunctions.Add(item);
-            }
-
             // Add the file table
             if (_fileTable?.Count > 0)
             {

src/Runner.Worker/ActionRunner.cs

@@ -26,7 +26,7 @@ namespace GitHub.Runner.Worker
     public interface IActionRunner : IStep, IRunnerService
     {
         ActionRunStage Stage { get; set; }
-        bool TryEvaluateDisplayName(DictionaryContextData contextData, IExecutionContext context);
+        Boolean TryEvaluateDisplayName(DictionaryContextData contextData, IExecutionContext context);
         Pipelines.ActionStep Action { get; set; }
     }
 
@@ -142,7 +142,7 @@ namespace GitHub.Runner.Worker
             // Load the inputs.
             ExecutionContext.Debug("Loading inputs");
             var templateEvaluator = ExecutionContext.ToPipelineTemplateEvaluator();
-            var inputs = templateEvaluator.EvaluateStepInputs(Action.Inputs, ExecutionContext.ExpressionValues, ExecutionContext.ExpressionFunctions);
+            var inputs = templateEvaluator.EvaluateStepInputs(Action.Inputs, ExecutionContext.ExpressionValues);
 
             foreach (KeyValuePair<string, string> input in inputs)
             {
@@ -162,7 +162,13 @@ namespace GitHub.Runner.Worker
                     string key = input.Key.AssertString("action input name").Value;
                     if (!inputs.ContainsKey(key))
                     {
-                        inputs[key] = manifestManager.EvaluateDefaultInput(ExecutionContext, key, input.Value);
+                        var evaluateContext = new Dictionary<string, PipelineContextData>(StringComparer.OrdinalIgnoreCase);
+                        foreach (var data in ExecutionContext.ExpressionValues)
+                        {
+                            evaluateContext[data.Key] = data.Value;
+                        }
+
+                        inputs[key] = manifestManager.EvaluateDefaultInput(ExecutionContext, key, input.Value, evaluateContext);
                     }
                 }
             }
@@ -287,14 +293,10 @@ namespace GitHub.Runner.Worker
                 return displayName;
             }
             // Try evaluating fully
+            var templateEvaluator = context.ToPipelineTemplateEvaluator();
             try
             {
-                if (tokenToParse.CheckHasRequiredContext(contextData, context.ExpressionFunctions))
+                didFullyEvaluate = templateEvaluator.TryEvaluateStepDisplayName(tokenToParse, contextData, out displayName);
-                {
-                    var templateEvaluator = context.ToPipelineTemplateEvaluator();
-                    displayName = templateEvaluator.EvaluateStepDisplayName(tokenToParse, contextData, context.ExpressionFunctions);
-                    didFullyEvaluate = true;
-                }
             }
             catch (TemplateValidationException e)
             {

src/Runner.Worker/Container/ContainerInfo.cs

@@ -61,7 +61,6 @@ namespace GitHub.Runner.Worker.Container
                 foreach (var volume in container.Volumes)
                 {
                     UserMountVolumes[volume] = volume;
-                    MountVolumes.Add(new MountVolume(volume));
                 }
             }
 

src/Runner.Worker/Container/DockerCommandManager.cs

@@ -130,13 +130,6 @@ namespace GitHub.Runner.Worker.Container
             // Watermark for GitHub Action environment
             dockerOptions.Add("-e GITHUB_ACTIONS=true");
 
-            // Set CI=true when no one else already set it.
-            // CI=true is common set in most CI provider in GitHub
-            if (!container.ContainerEnvironmentVariables.ContainsKey("CI"))
-            {
-                dockerOptions.Add("-e CI=true");
-            }
-
             foreach (var volume in container.MountVolumes)
             {
                 // replace `"` with `\"` and add `"{0}"` to all path.
@@ -196,13 +189,6 @@ namespace GitHub.Runner.Worker.Container
             // Watermark for GitHub Action environment
             dockerOptions.Add("-e GITHUB_ACTIONS=true");
 
-            // Set CI=true when no one else already set it.
-            // CI=true is common set in most CI provider in GitHub
-            if (!container.ContainerEnvironmentVariables.ContainsKey("CI"))
-            {
-                dockerOptions.Add("-e CI=true");
-            }
-
             if (!string.IsNullOrEmpty(container.ContainerEntryPoint))
             {
                 dockerOptions.Add($"--entrypoint \"{container.ContainerEntryPoint}\"");

src/Runner.Worker/ExecutionContext.cs

@@ -1,15 +1,14 @@
 using System;
-using System.Collections;
 using System.Collections.Generic;
 using System.Collections.Specialized;
 using System.Globalization;
 using System.IO;
 using System.Linq;
-using System.Text;
 using System.Threading;
 using System.Threading.Tasks;
 using System.Web;
-using GitHub.DistributedTask.Expressions2;
+using GitHub.Runner.Worker.Container;
+using GitHub.Services.WebApi;
 using GitHub.DistributedTask.Pipelines;
 using GitHub.DistributedTask.Pipelines.ContextData;
 using GitHub.DistributedTask.Pipelines.ObjectTemplating;
@@ -17,11 +16,12 @@ using GitHub.DistributedTask.WebApi;
 using GitHub.Runner.Common.Util;
 using GitHub.Runner.Common;
 using GitHub.Runner.Sdk;
-using GitHub.Runner.Worker.Container;
-using GitHub.Services.WebApi;
 using Newtonsoft.Json;
+using System.Text;
+using System.Collections;
 using ObjectTemplating = GitHub.DistributedTask.ObjectTemplating;
 using Pipelines = GitHub.DistributedTask.Pipelines;
+using GitHub.DistributedTask.Expressions2;
 
 namespace GitHub.Runner.Worker
 {
@@ -39,7 +39,6 @@ namespace GitHub.Runner.Worker
         string ContextName { get; }
         Task ForceCompleted { get; }
         TaskResult? Result { get; set; }
-        TaskResult? Outcome { get; set; }
         string ResultCode { get; set; }
         TaskResult? CommandResult { get; set; }
         CancellationToken CancellationToken { get; }
@@ -48,14 +47,12 @@ namespace GitHub.Runner.Worker
         PlanFeatures Features { get; }
         Variables Variables { get; }
         Dictionary<string, string> IntraActionState { get; }
-        IDictionary<String, IDictionary<String, String>> JobDefaults { get; }
+        HashSet<string> OutputVariables { get; }
-        Dictionary<string, VariableValue> JobOutputs { get; }
         IDictionary<String, String> EnvironmentVariables { get; }
         IDictionary<String, ContextScope> Scopes { get; }
         IList<String> FileTable { get; }
         StepsContext StepsContext { get; }
         DictionaryContextData ExpressionValues { get; }
-        IList<IFunctionInfo> ExpressionFunctions { get; }
         List<string> PrependPath { get; }
         ContainerInfo Container { get; set; }
         List<ContainerInfo> ServiceContainers { get; }
@@ -113,6 +110,7 @@ namespace GitHub.Runner.Worker
         private readonly TimelineRecord _record = new TimelineRecord();
         private readonly Dictionary<Guid, TimelineRecord> _detailRecords = new Dictionary<Guid, TimelineRecord>();
         private readonly object _loggerLock = new object();
+        private readonly HashSet<string> _outputvariables = new HashSet<string>(StringComparer.OrdinalIgnoreCase);
         private readonly object _matchersLock = new object();
 
         private event OnMatcherChanged _onMatcherChanged;
@@ -142,14 +140,12 @@ namespace GitHub.Runner.Worker
         public List<ServiceEndpoint> Endpoints { get; private set; }
         public Variables Variables { get; private set; }
         public Dictionary<string, string> IntraActionState { get; private set; }
-        public IDictionary<String, IDictionary<String, String>> JobDefaults { get; private set; }
+        public HashSet<string> OutputVariables => _outputvariables;
-        public Dictionary<string, VariableValue> JobOutputs { get; private set; }
         public IDictionary<String, String> EnvironmentVariables { get; private set; }
         public IDictionary<String, ContextScope> Scopes { get; private set; }
         public IList<String> FileTable { get; private set; }
         public StepsContext StepsContext { get; private set; }
         public DictionaryContextData ExpressionValues { get; } = new DictionaryContextData();
-        public IList<IFunctionInfo> ExpressionFunctions { get; } = new List<IFunctionInfo>();
         public bool WriteDebug { get; private set; }
         public List<string> PrependPath { get; private set; }
         public ContainerInfo Container { get; set; }
@@ -176,8 +172,6 @@ namespace GitHub.Runner.Worker
             }
         }
 
-        public TaskResult? Outcome { get; set; }
-
         public TaskResult? CommandResult { get; set; }
 
         private string ContextType => _record.RecordType;
@@ -274,7 +268,6 @@ namespace GitHub.Runner.Worker
                 child.IntraActionState = intraActionState;
             }
             child.EnvironmentVariables = EnvironmentVariables;
-            child.JobDefaults = JobDefaults;
             child.Scopes = Scopes;
             child.FileTable = FileTable;
             child.StepsContext = StepsContext;
@@ -282,10 +275,6 @@ namespace GitHub.Runner.Worker
             {
                 child.ExpressionValues[pair.Key] = pair.Value;
             }
-            foreach (var item in ExpressionFunctions)
-            {
-                child.ExpressionFunctions.Add(item);
-            }
             child._cancellationTokenSource = new CancellationTokenSource();
             child.WriteDebug = WriteDebug;
             child._parentExecutionContext = this;
@@ -358,12 +347,6 @@ namespace GitHub.Runner.Worker
 
             _logger.End();
 
-            if (!string.IsNullOrEmpty(ContextName))
-            {
-                StepsContext.SetOutcome(ScopeName, ContextName, (Outcome ?? Result ?? TaskResult.Succeeded).ToActionResult().ToString());
-                StepsContext.SetConclusion(ScopeName, ContextName, (Result ?? TaskResult.Succeeded).ToActionResult().ToString());
-            }
-
             return Result.Value;
         }
 
@@ -574,12 +557,6 @@ namespace GitHub.Runner.Worker
             // Environment variables shared across all actions
             EnvironmentVariables = new Dictionary<string, string>(VarUtil.EnvironmentVariableKeyComparer);
 
-            // Job defaults shared across all actions
-            JobDefaults = new Dictionary<string, IDictionary<string, string>>(StringComparer.OrdinalIgnoreCase);
-
-            // Job Outputs
-            JobOutputs = new Dictionary<string, VariableValue>(StringComparer.OrdinalIgnoreCase);
-
             // Service container info
             ServiceContainers = new List<ContainerInfo>();
 
@@ -599,6 +576,12 @@ namespace GitHub.Runner.Worker
             // File table
             FileTable = new List<String>(message.FileTable ?? new string[0]);
 
+            // Expression functions
+            if (Variables.GetBoolean("System.HashFilesV2") == true)
+            {
+                ExpressionConstants.UpdateFunction<Handlers.HashFiles>("hashFiles", 1, byte.MaxValue);
+            }
+
             // Expression values
             if (message.ContextData?.Count > 0)
             {
@@ -616,13 +599,8 @@ namespace GitHub.Runner.Worker
             var githubAccessToken = new StringContextData(Variables.Get("system.github.token"));
             var base64EncodedToken = Convert.ToBase64String(Encoding.UTF8.GetBytes($"x-access-token:{githubAccessToken}"));
             HostContext.SecretMasker.AddValue(base64EncodedToken);
-            var githubJob = Variables.Get("system.github.job");
             var githubContext = new GitHubContext();
             githubContext["token"] = githubAccessToken;
-            if (!string.IsNullOrEmpty(githubJob))
-            {
-                githubContext["job"] = new StringContextData(githubJob);
-            }
             var githubDictionary = ExpressionValues["github"].AssertDictionary("github");
             foreach (var pair in githubDictionary)
             {
@@ -758,7 +736,7 @@ namespace GitHub.Runner.Worker
                 var owners = config.Matchers.Select(x => $"'{x.Owner}'");
                 var joinedOwners = string.Join(", ", owners);
                 // todo: loc
-                this.Debug($"Added matchers: {joinedOwners}. Problem matchers scan action output for known warning or error strings and report these inline.");
+                this.Output($"Added matchers: {joinedOwners}. Problem matchers scan action output for known warning or error strings and report these inline.");
             }
         }
 
@@ -800,7 +778,7 @@ namespace GitHub.Runner.Worker
                 owners = removedMatchers.Select(x => $"'{x.Owner}'");
                 var joinedOwners = string.Join(", ", owners);
                 // todo: loc
-                this.Debug($"Removed matchers: {joinedOwners}");
+                this.Output($"Removed matchers: {joinedOwners}");
             }
         }
 
@@ -915,19 +893,11 @@ namespace GitHub.Runner.Worker
             }
         }
 
-        public static IEnumerable<KeyValuePair<string, object>> ToExpressionState(this IExecutionContext context)
+        public static PipelineTemplateEvaluator ToPipelineTemplateEvaluator(this IExecutionContext context)
         {
-            return new[] { new KeyValuePair<string, object>(nameof(IExecutionContext), context) };
+            var templateTrace = context.ToTemplateTraceWriter();
-        }
+            var schema = new PipelineTemplateSchemaFactory().CreateSchema();
-
+            return new PipelineTemplateEvaluator(templateTrace, schema, context.FileTable);
-        public static PipelineTemplateEvaluator ToPipelineTemplateEvaluator(this IExecutionContext context, ObjectTemplating.ITraceWriter traceWriter = null)
-        {
-            if (traceWriter == null)
-            {
-                traceWriter = context.ToTemplateTraceWriter();
-            }
-            var schema = PipelineTemplateSchemaFactory.GetSchema();
-            return new PipelineTemplateEvaluator(traceWriter, schema, context.FileTable);
         }
 
         public static ObjectTemplating.ITraceWriter ToTemplateTraceWriter(this IExecutionContext context)
@@ -942,7 +912,6 @@ namespace GitHub.Runner.Worker
 
         internal TemplateTraceWriter(IExecutionContext executionContext)
         {
-            ArgUtil.NotNull(executionContext, nameof(executionContext));
             _executionContext = executionContext;
         }
 

src/Runner.Worker/ExpressionFunctions/HashFiles.cs

@@ -8,9 +8,28 @@ using System.Reflection;
 using System.Threading;
 using System.Collections.Generic;
 
-namespace GitHub.Runner.Worker.Expressions
+namespace GitHub.Runner.Worker.Handlers
 {
-    public sealed class HashFilesFunction : Function
+    public class FunctionTrace : ITraceWriter
+    {
+        private GitHub.DistributedTask.Expressions2.ITraceWriter _trace;
+
+        public FunctionTrace(GitHub.DistributedTask.Expressions2.ITraceWriter trace)
+        {
+            _trace = trace;
+        }
+        public void Info(string message)
+        {
+            _trace.Info(message);
+        }
+
+        public void Verbose(string message)
+        {
+            _trace.Info(message);
+        }
+    }
+
+    public sealed class HashFiles : Function
     {
         protected sealed override Object EvaluateCore(
             EvaluationContext context,
@@ -63,7 +82,7 @@ namespace GitHub.Runner.Worker.Expressions
             string node = Path.Combine(runnerRoot, "externals", "node12", "bin", $"node{IOUtil.ExeExtension}");
             string hashFilesScript = Path.Combine(binDir, "hashFiles");
             var hashResult = string.Empty;
-            var p = new ProcessInvoker(new HashFilesTrace(context.Trace));
+            var p = new ProcessInvoker(new FunctionTrace(context.Trace));
             p.ErrorDataReceived += ((_, data) =>
             {
                 if (!string.IsNullOrEmpty(data.Data) && data.Data.StartsWith("__OUTPUT__") && data.Data.EndsWith("__OUTPUT__"))
@@ -103,24 +122,5 @@ namespace GitHub.Runner.Worker.Expressions
 
             return hashResult;
         }
-
-        private sealed class HashFilesTrace : ITraceWriter
-        {
-            private GitHub.DistributedTask.Expressions2.ITraceWriter _trace;
-
-            public HashFilesTrace(GitHub.DistributedTask.Expressions2.ITraceWriter trace)
-            {
-                _trace = trace;
-            }
-            public void Info(string message)
-            {
-                _trace.Info(message);
-            }
-
-            public void Verbose(string message)
-            {
-                _trace.Info(message);
-            }
-        }
     }
 }

src/Runner.Worker/ExpressionManager.cs

@@ -0,0 +1,162 @@
+using System;
+using System.Collections;
+using System.Collections.Generic;
+using System.Linq;
+using System.Text;
+using GitHub.DistributedTask.Expressions2;
+using GitHub.DistributedTask.Expressions2.Sdk;
+using GitHub.DistributedTask.WebApi;
+using GitHub.Runner.Common;
+using GitHub.Runner.Common.Util;
+using GitHub.Runner.Sdk;
+using ObjectTemplating = GitHub.DistributedTask.ObjectTemplating;
+using PipelineTemplateConstants = GitHub.DistributedTask.Pipelines.ObjectTemplating.PipelineTemplateConstants;
+
+namespace GitHub.Runner.Worker
+{
+    [ServiceLocator(Default = typeof(ExpressionManager))]
+    public interface IExpressionManager : IRunnerService
+    {
+        ConditionResult Evaluate(IExecutionContext context, string condition, bool hostTracingOnly = false);
+    }
+
+    public sealed class ExpressionManager : RunnerService, IExpressionManager
+    {
+        public ConditionResult Evaluate(IExecutionContext executionContext, string condition, bool hostTracingOnly = false)
+        {
+            ArgUtil.NotNull(executionContext, nameof(executionContext));
+
+            ConditionResult result = new ConditionResult();
+            var expressionTrace = new TraceWriter(Trace, hostTracingOnly ? null : executionContext);
+            var tree = Parse(executionContext, expressionTrace, condition);
+            var expressionResult = tree.Evaluate(expressionTrace, HostContext.SecretMasker, state: executionContext, options: null);
+            result.Value = expressionResult.IsTruthy;
+            result.Trace = expressionTrace.Trace;
+
+            return result;
+        }
+
+        private static IExpressionNode Parse(IExecutionContext executionContext, TraceWriter expressionTrace, string condition)
+        {
+            ArgUtil.NotNull(executionContext, nameof(executionContext));
+
+            if (string.IsNullOrWhiteSpace(condition))
+            {
+                condition = $"{PipelineTemplateConstants.Success}()";
+            }
+
+            var parser = new ExpressionParser();
+            var namedValues = executionContext.ExpressionValues.Keys.Select(x => new NamedValueInfo<ContextValueNode>(x)).ToArray();
+            var functions = new IFunctionInfo[]
+            {
+                new FunctionInfo<AlwaysNode>(name: Constants.Expressions.Always, minParameters: 0, maxParameters: 0),
+                new FunctionInfo<CancelledNode>(name: Constants.Expressions.Cancelled, minParameters: 0, maxParameters: 0),
+                new FunctionInfo<FailureNode>(name: Constants.Expressions.Failure, minParameters: 0, maxParameters: 0),
+                new FunctionInfo<SuccessNode>(name: Constants.Expressions.Success, minParameters: 0, maxParameters: 0),
+            };
+            return parser.CreateTree(condition, expressionTrace, namedValues, functions) ?? new SuccessNode();
+        }
+
+        private sealed class TraceWriter : DistributedTask.Expressions2.ITraceWriter
+        {
+            private readonly IExecutionContext _executionContext;
+            private readonly Tracing _trace;
+            private readonly StringBuilder _traceBuilder = new StringBuilder();
+
+            public string Trace => _traceBuilder.ToString();
+
+            public TraceWriter(Tracing trace, IExecutionContext executionContext)
+            {
+                ArgUtil.NotNull(trace, nameof(trace));
+                _trace = trace;
+                _executionContext = executionContext;
+            }
+
+            public void Info(string message)
+            {
+                _trace.Info(message);
+                _executionContext?.Debug(message);
+                _traceBuilder.AppendLine(message);
+            }
+
+            public void Verbose(string message)
+            {
+                _trace.Verbose(message);
+                _executionContext?.Debug(message);
+            }
+        }
+
+        private sealed class AlwaysNode : Function
+        {
+            protected override Object EvaluateCore(EvaluationContext context, out ResultMemory resultMemory)
+            {
+                resultMemory = null;
+                return true;
+            }
+        }
+
+        private sealed class CancelledNode : Function
+        {
+            protected sealed override object EvaluateCore(EvaluationContext evaluationContext, out ResultMemory resultMemory)
+            {
+                resultMemory = null;
+                var executionContext = evaluationContext.State as IExecutionContext;
+                ArgUtil.NotNull(executionContext, nameof(executionContext));
+                ActionResult jobStatus = executionContext.JobContext.Status ?? ActionResult.Success;
+                return jobStatus == ActionResult.Cancelled;
+            }
+        }
+
+        private sealed class FailureNode : Function
+        {
+            protected sealed override object EvaluateCore(EvaluationContext evaluationContext, out ResultMemory resultMemory)
+            {
+                resultMemory = null;
+                var executionContext = evaluationContext.State as IExecutionContext;
+                ArgUtil.NotNull(executionContext, nameof(executionContext));
+                ActionResult jobStatus = executionContext.JobContext.Status ?? ActionResult.Success;
+                return jobStatus == ActionResult.Failure;
+            }
+        }
+
+        private sealed class SuccessNode : Function
+        {
+            protected sealed override object EvaluateCore(EvaluationContext evaluationContext, out ResultMemory resultMemory)
+            {
+                resultMemory = null;
+                var executionContext = evaluationContext.State as IExecutionContext;
+                ArgUtil.NotNull(executionContext, nameof(executionContext));
+                ActionResult jobStatus = executionContext.JobContext.Status ?? ActionResult.Success;
+                return jobStatus == ActionResult.Success;
+            }
+        }
+
+        private sealed class ContextValueNode : NamedValue
+        {
+            protected override Object EvaluateCore(EvaluationContext evaluationContext, out ResultMemory resultMemory)
+            {
+                resultMemory = null;
+                var jobContext = evaluationContext.State as IExecutionContext;
+                ArgUtil.NotNull(jobContext, nameof(jobContext));
+                return jobContext.ExpressionValues[Name];
+            }
+        }
+    }
+
+    public class ConditionResult
+    {
+        public ConditionResult(bool value = false, string trace = null)
+        {
+            this.Value = value;
+            this.Trace = trace;
+        }
+
+        public bool Value { get; set; }
+        public string Trace { get; set; }
+
+        public static implicit operator ConditionResult(bool value)
+        {
+            return new ConditionResult(value);
+        }
+    }
+}

src/Runner.Worker/Expressions/AlwaysFunction.cs

@@ -1,25 +0,0 @@
-using System;
-using System.Collections;
-using System.Collections.Generic;
-using System.Linq;
-using System.Text;
-using GitHub.DistributedTask.Expressions2;
-using GitHub.DistributedTask.Expressions2.Sdk;
-using GitHub.DistributedTask.WebApi;
-using GitHub.Runner.Common;
-using GitHub.Runner.Common.Util;
-using GitHub.Runner.Sdk;
-using ObjectTemplating = GitHub.DistributedTask.ObjectTemplating;
-using PipelineTemplateConstants = GitHub.DistributedTask.Pipelines.ObjectTemplating.PipelineTemplateConstants;
-
-namespace GitHub.Runner.Worker.Expressions
-{
-    public sealed class AlwaysFunction : Function
-    {
-        protected override Object EvaluateCore(EvaluationContext context, out ResultMemory resultMemory)
-        {
-            resultMemory = null;
-            return true;
-        }
-    }
-}

src/Runner.Worker/Expressions/CancelledFunction.cs

@@ -1,31 +0,0 @@
-using System;
-using System.Collections;
-using System.Collections.Generic;
-using System.Linq;
-using System.Text;
-using GitHub.DistributedTask.Expressions2;
-using GitHub.DistributedTask.Expressions2.Sdk;
-using GitHub.DistributedTask.ObjectTemplating;
-using GitHub.DistributedTask.WebApi;
-using GitHub.Runner.Common;
-using GitHub.Runner.Common.Util;
-using GitHub.Runner.Sdk;
-using ObjectTemplating = GitHub.DistributedTask.ObjectTemplating;
-using PipelineTemplateConstants = GitHub.DistributedTask.Pipelines.ObjectTemplating.PipelineTemplateConstants;
-
-namespace GitHub.Runner.Worker.Expressions
-{
-    public sealed class CancelledFunction : Function
-    {
-        protected sealed override object EvaluateCore(EvaluationContext evaluationContext, out ResultMemory resultMemory)
-        {
-            resultMemory = null;
-            var templateContext = evaluationContext.State as TemplateContext;
-            ArgUtil.NotNull(templateContext, nameof(templateContext));
-            var executionContext = templateContext.State[nameof(IExecutionContext)] as IExecutionContext;
-            ArgUtil.NotNull(executionContext, nameof(executionContext));
-            ActionResult jobStatus = executionContext.JobContext.Status ?? ActionResult.Success;
-            return jobStatus == ActionResult.Cancelled;
-        }
-    }
-}

src/Runner.Worker/Expressions/FailureFunction.cs

@@ -1,31 +0,0 @@
-using System;
-using System.Collections;
-using System.Collections.Generic;
-using System.Linq;
-using System.Text;
-using GitHub.DistributedTask.Expressions2;
-using GitHub.DistributedTask.Expressions2.Sdk;
-using GitHub.DistributedTask.ObjectTemplating;
-using GitHub.DistributedTask.WebApi;
-using GitHub.Runner.Common;
-using GitHub.Runner.Common.Util;
-using GitHub.Runner.Sdk;
-using ObjectTemplating = GitHub.DistributedTask.ObjectTemplating;
-using PipelineTemplateConstants = GitHub.DistributedTask.Pipelines.ObjectTemplating.PipelineTemplateConstants;
-
-namespace GitHub.Runner.Worker.Expressions
-{
-    public sealed class FailureFunction : Function
-    {
-        protected sealed override object EvaluateCore(EvaluationContext evaluationContext, out ResultMemory resultMemory)
-        {
-            resultMemory = null;
-            var templateContext = evaluationContext.State as TemplateContext;
-            ArgUtil.NotNull(templateContext, nameof(templateContext));
-            var executionContext = templateContext.State[nameof(IExecutionContext)] as IExecutionContext;
-            ArgUtil.NotNull(executionContext, nameof(executionContext));
-            ActionResult jobStatus = executionContext.JobContext.Status ?? ActionResult.Success;
-            return jobStatus == ActionResult.Failure;
-        }
-    }
-}

src/Runner.Worker/Expressions/SuccessFunction.cs

@@ -1,31 +0,0 @@
-using System;
-using System.Collections;
-using System.Collections.Generic;
-using System.Linq;
-using System.Text;
-using GitHub.DistributedTask.Expressions2;
-using GitHub.DistributedTask.Expressions2.Sdk;
-using GitHub.DistributedTask.ObjectTemplating;
-using GitHub.DistributedTask.WebApi;
-using GitHub.Runner.Common;
-using GitHub.Runner.Common.Util;
-using GitHub.Runner.Sdk;
-using ObjectTemplating = GitHub.DistributedTask.ObjectTemplating;
-using PipelineTemplateConstants = GitHub.DistributedTask.Pipelines.ObjectTemplating.PipelineTemplateConstants;
-
-namespace GitHub.Runner.Worker.Expressions
-{
-    public sealed class SuccessFunction : Function
-    {
-        protected sealed override object EvaluateCore(EvaluationContext evaluationContext, out ResultMemory resultMemory)
-        {
-            resultMemory = null;
-            var templateContext = evaluationContext.State as TemplateContext;
-            ArgUtil.NotNull(templateContext, nameof(templateContext));
-            var executionContext = templateContext.State[nameof(IExecutionContext)] as IExecutionContext;
-            ArgUtil.NotNull(executionContext, nameof(executionContext));
-            ActionResult jobStatus = executionContext.JobContext.Status ?? ActionResult.Success;
-            return jobStatus == ActionResult.Success;
-        }
-    }
-}

src/Runner.Worker/GitHubContext.cs

@@ -14,10 +14,8 @@ namespace GitHub.Runner.Worker
             "event_name",
             "event_path",
             "head_ref",
-            "job",
             "ref",
             "repository",
-            "repository_owner",
             "run_id",
             "run_number",
             "sha",

src/Runner.Worker/Handlers/ContainerActionHandler.cs

@@ -97,14 +97,14 @@ namespace GitHub.Runner.Worker.Handlers
                 }
             }
 
-            var extraExpressionValues = new Dictionary<string, PipelineContextData>(StringComparer.OrdinalIgnoreCase);
+            var evaluateContext = new Dictionary<string, PipelineContextData>(StringComparer.OrdinalIgnoreCase);
-            extraExpressionValues["inputs"] = inputsContext;
+            evaluateContext["inputs"] = inputsContext;
 
             var manifestManager = HostContext.GetService<IActionManifestManager>();
             if (Data.Arguments != null)
             {
                 container.ContainerEntryPointArgs = "";
-                var evaluatedArgs = manifestManager.EvaluateContainerArguments(ExecutionContext, Data.Arguments, extraExpressionValues);
+                var evaluatedArgs = manifestManager.EvaluateContainerArguments(ExecutionContext, Data.Arguments, evaluateContext);
                 foreach (var arg in evaluatedArgs)
                 {
                     if (!string.IsNullOrEmpty(arg))
@@ -124,7 +124,7 @@ namespace GitHub.Runner.Worker.Handlers
 
             if (Data.Environment != null)
             {
-                var evaluatedEnv = manifestManager.EvaluateContainerEnvironment(ExecutionContext, Data.Environment, extraExpressionValues);
+                var evaluatedEnv = manifestManager.EvaluateContainerEnvironment(ExecutionContext, Data.Environment, evaluateContext);
                 foreach (var env in evaluatedEnv)
                 {
                     if (!this.Environment.ContainsKey(env.Key))

src/Runner.Worker/Handlers/ScriptHandler.cs

@@ -58,21 +58,12 @@ namespace GitHub.Runner.Worker.Handlers
             string shellCommandPath = null;
             bool validateShellOnHost = !(StepHost is ContainerStepHost);
             string prependPath = string.Join(Path.PathSeparator.ToString(), ExecutionContext.PrependPath.Reverse<string>());
-            string shell = null;
+            Inputs.TryGetValue("shell", out var shell);
-            if (!Inputs.TryGetValue("shell", out shell) || string.IsNullOrEmpty(shell))
-            {
-                // TODO: figure out how defaults interact with template later
-                // for now, we won't check job.defaults if we are inside a template.
-                if (string.IsNullOrEmpty(ExecutionContext.ScopeName) && ExecutionContext.JobDefaults.TryGetValue("run", out var runDefaults))
-                {
-                    runDefaults.TryGetValue("shell", out shell);
-                }
-            }
             if (string.IsNullOrEmpty(shell))
             {
 #if OS_WINDOWS
                 shellCommand = "pwsh";
-                if (validateShellOnHost)
+                if(validateShellOnHost)
                 {
                     shellCommandPath = WhichUtil.Which(shellCommand, require: false, Trace, prependPath);
                     if (string.IsNullOrEmpty(shellCommandPath))
@@ -148,36 +139,11 @@ namespace GitHub.Runner.Worker.Handlers
             Inputs.TryGetValue("script", out var contents);
             contents = contents ?? string.Empty;
 
-            string workingDirectory = null;
+            Inputs.TryGetValue("workingDirectory", out var workingDirectory);
-            if (!Inputs.TryGetValue("workingDirectory", out workingDirectory))
-            {
-                // TODO: figure out how defaults interact with template later
-                // for now, we won't check job.defaults if we are inside a template.
-                if (string.IsNullOrEmpty(ExecutionContext.ScopeName) && ExecutionContext.JobDefaults.TryGetValue("run", out var runDefaults))
-                {
-                    if (runDefaults.TryGetValue("working-directory", out workingDirectory))
-                    {
-                        ExecutionContext.Debug("Overwrite 'working-directory' base on job defaults.");
-                    }
-                }
-            }
             var workspaceDir = githubContext["workspace"] as StringContextData;
             workingDirectory = Path.Combine(workspaceDir, workingDirectory ?? string.Empty);
 
-            string shell = null;
+            Inputs.TryGetValue("shell", out var shell);
-            if (!Inputs.TryGetValue("shell", out shell) || string.IsNullOrEmpty(shell))
-            {
-                // TODO: figure out how defaults interact with template later
-                // for now, we won't check job.defaults if we are inside a template.
-                if (string.IsNullOrEmpty(ExecutionContext.ScopeName) && ExecutionContext.JobDefaults.TryGetValue("run", out var runDefaults))
-                {
-                    if (runDefaults.TryGetValue("shell", out shell))
-                    {
-                        ExecutionContext.Debug("Overwrite 'shell' base on job defaults.");
-                    }
-                }
-            }
-
             var isContainerStepHost = StepHost is ContainerStepHost;
 
             string prependPath = string.Join(Path.PathSeparator.ToString(), ExecutionContext.PrependPath.Reverse<string>());

src/Runner.Worker/JobExtension.cs

@@ -3,11 +3,8 @@ using System.Collections.Generic;
 using System.Diagnostics;
 using System.IO;
 using System.Linq;
-using System.Runtime.Serialization;
 using System.Threading.Tasks;
 using GitHub.DistributedTask.Expressions2;
-using GitHub.DistributedTask.ObjectTemplating.Tokens;
-using GitHub.DistributedTask.Pipelines.ContextData;
 using GitHub.DistributedTask.Pipelines.ObjectTemplating;
 using GitHub.DistributedTask.WebApi;
 using GitHub.Runner.Common;
@@ -17,16 +14,6 @@ using Pipelines = GitHub.DistributedTask.Pipelines;
 
 namespace GitHub.Runner.Worker
 {
-    [DataContract]
-    public class SetupInfo
-    {
-        [DataMember]
-        public string Group { get; set; }
-
-        [DataMember]
-        public string Detail { get; set; }
-    }
-
     [ServiceLocator(Default = typeof(JobExtension))]
 
     public interface IJobExtension : IRunnerService
@@ -62,44 +49,6 @@ namespace GitHub.Runner.Worker
                     context.Start();
                     context.Debug($"Starting: Set up job");
                     context.Output($"Current runner version: '{BuildConstants.RunnerPackage.Version}'");
-
-                    var setupInfoFile = HostContext.GetConfigFile(WellKnownConfigFile.SetupInfo);
-                    if (File.Exists(setupInfoFile))
-                    {
-                        Trace.Info($"Load machine setup info from {setupInfoFile}");
-                        try
-                        {
-                            var setupInfo = IOUtil.LoadObject<List<SetupInfo>>(setupInfoFile);
-                            if (setupInfo?.Count > 0)
-                            {
-                                foreach (var info in setupInfo)
-                                {
-                                    if (!string.IsNullOrEmpty(info?.Detail))
-                                    {
-                                        var groupName = info.Group;
-                                        if (string.IsNullOrEmpty(groupName))
-                                        {
-                                            groupName = "Machine Setup Info";
-                                        }
-
-                                        context.Output($"##[group]{groupName}");
-                                        var multiLines = info.Detail.Replace("\r\n", "\n").TrimEnd('\n').Split('\n');
-                                        foreach (var line in multiLines)
-                                        {
-                                            context.Output(line);
-                                        }
-                                        context.Output("##[endgroup]");
-                                    }
-                                }
-                            }
-                        }
-                        catch (Exception ex)
-                        {
-                            context.Output($"Fail to load and print machine setup info: {ex.Message}");
-                            Trace.Error(ex);
-                        }
-                    }
-
                     var repoFullName = context.GetGitHubContext("repository");
                     ArgUtil.NotNull(repoFullName, nameof(repoFullName));
                     context.Debug($"Primary repository: {repoFullName}");
@@ -132,7 +81,7 @@ namespace GitHub.Runner.Worker
                     var templateEvaluator = context.ToPipelineTemplateEvaluator();
                     foreach (var token in message.EnvironmentVariables)
                     {
-                        var environmentVariables = templateEvaluator.EvaluateStepEnvironment(token, jobContext.ExpressionValues, jobContext.ExpressionFunctions, VarUtil.EnvironmentVariableKeyComparer);
+                        var environmentVariables = templateEvaluator.EvaluateStepEnvironment(token, jobContext.ExpressionValues, VarUtil.EnvironmentVariableKeyComparer);
                         foreach (var pair in environmentVariables)
                         {
                             context.EnvironmentVariables[pair.Key] = pair.Value ?? string.Empty;
@@ -142,7 +91,7 @@ namespace GitHub.Runner.Worker
 
                     // Evaluate the job container
                     context.Debug("Evaluating job container");
-                    var container = templateEvaluator.EvaluateJobContainer(message.JobContainer, jobContext.ExpressionValues, jobContext.ExpressionFunctions);
+                    var container = templateEvaluator.EvaluateJobContainer(message.JobContainer, jobContext.ExpressionValues);
                     if (container != null)
                     {
                         jobContext.Container = new Container.ContainerInfo(HostContext, container);
@@ -150,7 +99,7 @@ namespace GitHub.Runner.Worker
 
                     // Evaluate the job service containers
                     context.Debug("Evaluating job service containers");
-                    var serviceContainers = templateEvaluator.EvaluateJobServiceContainers(message.JobServiceContainers, jobContext.ExpressionValues, jobContext.ExpressionFunctions);
+                    var serviceContainers = templateEvaluator.EvaluateJobServiceContainers(message.JobServiceContainers, jobContext.ExpressionValues);
                     if (serviceContainers?.Count > 0)
                     {
                         foreach (var pair in serviceContainers)
@@ -161,26 +110,6 @@ namespace GitHub.Runner.Worker
                         }
                     }
 
-                    // Evaluate the job defaults
-                    context.Debug("Evaluating job defaults");
-                    foreach (var token in message.Defaults)
-                    {
-                        var defaults = token.AssertMapping("defaults");
-                        if (defaults.Any(x => string.Equals(x.Key.AssertString("defaults key").Value, "run", StringComparison.OrdinalIgnoreCase)))
-                        {
-                            context.JobDefaults["run"] = new Dictionary<string, string>(StringComparer.OrdinalIgnoreCase);
-                            var defaultsRun = defaults.First(x => string.Equals(x.Key.AssertString("defaults key").Value, "run", StringComparison.OrdinalIgnoreCase));
-                            var jobDefaults = templateEvaluator.EvaluateJobDefaultsRun(defaultsRun.Value, jobContext.ExpressionValues, jobContext.ExpressionFunctions);
-                            foreach (var pair in jobDefaults)
-                            {
-                                if (!string.IsNullOrEmpty(pair.Value))
-                                {
-                                    context.JobDefaults["run"][pair.Key] = pair.Value;
-                                }
-                            }
-                        }
-                    }
-
                     // Build up 2 lists of steps, pre-job, job
                     // Download actions not already in the cache
                     Trace.Info("Downloading actions");
@@ -313,58 +242,6 @@ namespace GitHub.Runner.Worker
                     context.Start();
                     context.Debug("Starting: Complete job");
 
-                    // Evaluate job outputs
-                    if (message.JobOutputs != null && message.JobOutputs.Type != TokenType.Null)
-                    {
-                        try
-                        {
-                            context.Output($"Evaluate and set job outputs");
-
-                            // Populate env context for each step
-                            Trace.Info("Initialize Env context for evaluating job outputs");
-#if OS_WINDOWS
-                            var envContext = new DictionaryContextData();
-#else
-                            var envContext = new CaseSensitiveDictionaryContextData();
-#endif
-                            context.ExpressionValues["env"] = envContext;
-                            foreach (var pair in context.EnvironmentVariables)
-                            {
-                                envContext[pair.Key] = new StringContextData(pair.Value ?? string.Empty);
-                            }
-
-                            Trace.Info("Initialize steps context for evaluating job outputs");
-                            context.ExpressionValues["steps"] = context.StepsContext.GetScope(context.ScopeName);
-
-                            var templateEvaluator = context.ToPipelineTemplateEvaluator();
-                            var outputs = templateEvaluator.EvaluateJobOutput(message.JobOutputs, context.ExpressionValues, context.ExpressionFunctions);
-                            foreach (var output in outputs)
-                            {
-                                if (string.IsNullOrEmpty(output.Value))
-                                {
-                                    context.Debug($"Skip output '{output.Key}' since it's empty");
-                                    continue;
-                                }
-
-                                if (!string.Equals(output.Value, HostContext.SecretMasker.MaskSecrets(output.Value)))
-                                {
-                                    context.Warning($"Skip output '{output.Key}' since it may contain secret.");
-                                    continue;
-                                }
-
-                                context.Output($"Set output '{output.Key}'");
-                                jobContext.JobOutputs[output.Key] = output.Value;
-                            }
-                        }
-                        catch (Exception ex)
-                        {
-                            context.Result = TaskResult.Failed;
-                            context.Error($"Fail to evaluate job outputs");
-                            context.Error(ex);
-                            jobContext.Result = TaskResultUtil.MergeTaskResults(jobContext.Result, TaskResult.Failed);
-                        }
-                    }
-
                     if (context.Variables.GetBoolean(Constants.Variables.Actions.RunnerDebug) ?? false)
                     {
                         Trace.Info("Support log upload starting.");

src/Runner.Worker/JobRunner.cs

@@ -231,7 +231,7 @@ namespace GitHub.Runner.Worker
             }
 
             Trace.Info("Raising job completed event.");
-            var jobCompletedEvent = new JobCompletedEvent(message.RequestId, message.JobId, result, jobContext.JobOutputs);
+            var jobCompletedEvent = new JobCompletedEvent(message.RequestId, message.JobId, result);
 
             var completeJobRetryLimit = 5;
             var exceptions = new List<Exception>();

src/Runner.Worker/StepsContext.cs

@@ -56,22 +56,13 @@ namespace GitHub.Runner.Worker
             }
         }
 
-        public void SetConclusion(
+        public void SetResult(
             string scopeName,
             string stepName,
-            string conclusion)
+            string result)
         {
             var step = GetStep(scopeName, stepName);
-            step["conclusion"] = new StringContextData(conclusion);
+            step["result"] = new StringContextData(result);
-        }
-
-        public void SetOutcome(
-            string scopeName,
-            string stepName,
-            string outcome)
-        {
-            var step = GetStep(scopeName, stepName);
-            step["outcome"] = new StringContextData(outcome);
         }
 
         private DictionaryContextData GetStep(string scopeName, string stepName)

src/Runner.Worker/StepsRunner.cs

@@ -1,6 +1,8 @@
+using GitHub.DistributedTask.WebApi;
+using Pipelines = GitHub.DistributedTask.Pipelines;
+using GitHub.Runner.Common.Util;
 using System;
 using System.Collections.Generic;
-using System.Text;
 using System.Threading;
 using System.Threading.Tasks;
 using GitHub.DistributedTask.Expressions2;
@@ -8,13 +10,8 @@ using GitHub.DistributedTask.ObjectTemplating.Tokens;
 using GitHub.DistributedTask.Pipelines;
 using GitHub.DistributedTask.Pipelines.ContextData;
 using GitHub.DistributedTask.Pipelines.ObjectTemplating;
-using GitHub.DistributedTask.WebApi;
 using GitHub.Runner.Common;
-using GitHub.Runner.Common.Util;
 using GitHub.Runner.Sdk;
-using GitHub.Runner.Worker.Expressions;
-using ObjectTemplating = GitHub.DistributedTask.ObjectTemplating;
-using Pipelines = GitHub.DistributedTask.Pipelines;
 
 namespace GitHub.Runner.Worker
 {
@@ -66,7 +63,11 @@ namespace GitHub.Runner.Worker
                 }
 
                 var step = jobContext.JobSteps.Dequeue();
-                var nextStep = jobContext.JobSteps.Count > 0 ? jobContext.JobSteps.Peek() : null;
+                IStep nextStep = null;
+                if (jobContext.JobSteps.Count > 0)
+                {
+                    nextStep = jobContext.JobSteps.Peek();
+                }
 
                 Trace.Info($"Processing step: DisplayName='{step.DisplayName}'");
                 ArgUtil.NotNull(step.ExecutionContext, nameof(step.ExecutionContext));
@@ -75,13 +76,6 @@ namespace GitHub.Runner.Worker
                 // Start
                 step.ExecutionContext.Start();
 
-                // Expression functions
-                step.ExecutionContext.ExpressionFunctions.Add(new FunctionInfo<AlwaysFunction>(PipelineTemplateConstants.Always, 0, 0));
-                step.ExecutionContext.ExpressionFunctions.Add(new FunctionInfo<CancelledFunction>(PipelineTemplateConstants.Cancelled, 0, 0));
-                step.ExecutionContext.ExpressionFunctions.Add(new FunctionInfo<FailureFunction>(PipelineTemplateConstants.Failure, 0, 0));
-                step.ExecutionContext.ExpressionFunctions.Add(new FunctionInfo<SuccessFunction>(PipelineTemplateConstants.Success, 0, 0));
-                step.ExecutionContext.ExpressionFunctions.Add(new FunctionInfo<HashFilesFunction>(PipelineTemplateConstants.HashFiles, 1, byte.MaxValue));
-
                 // Initialize scope
                 if (InitializeScope(step, scopeInputs))
                 {
@@ -105,13 +99,14 @@ namespace GitHub.Runner.Worker
 
                         // Evaluate and merge action's env block to env context
                         var templateEvaluator = step.ExecutionContext.ToPipelineTemplateEvaluator();
-                        var actionEnvironment = templateEvaluator.EvaluateStepEnvironment(actionStep.Action.Environment, step.ExecutionContext.ExpressionValues, step.ExecutionContext.ExpressionFunctions, VarUtil.EnvironmentVariableKeyComparer);
+                        var actionEnvironment = templateEvaluator.EvaluateStepEnvironment(actionStep.Action.Environment, step.ExecutionContext.ExpressionValues, VarUtil.EnvironmentVariableKeyComparer);
                         foreach (var env in actionEnvironment)
                         {
                             envContext[env.Key] = new StringContextData(env.Value ?? string.Empty);
                         }
                     }
 
+                    var expressionManager = HostContext.GetService<IExpressionManager>();
                     try
                     {
                         // Register job cancellation call back only if job cancellation token not been fire before each step run
@@ -125,29 +120,28 @@ namespace GitHub.Runner.Worker
                                 jobContext.JobContext.Status = jobContext.Result?.ToActionResult();
 
                                 step.ExecutionContext.Debug($"Re-evaluate condition on job cancellation for step: '{step.DisplayName}'.");
-                                var conditionReTestTraceWriter = new ConditionTraceWriter(Trace, null); // host tracing only
+                                ConditionResult conditionReTestResult;
-                                var conditionReTestResult = false;
                                 if (HostContext.RunnerShutdownToken.IsCancellationRequested)
                                 {
                                     step.ExecutionContext.Debug($"Skip Re-evaluate condition on runner shutdown.");
+                                    conditionReTestResult = false;
                                 }
                                 else
                                 {
                                     try
                                     {
-                                        var templateEvaluator = step.ExecutionContext.ToPipelineTemplateEvaluator(conditionReTestTraceWriter);
+                                        conditionReTestResult = expressionManager.Evaluate(step.ExecutionContext, step.Condition, hostTracingOnly: true);
-                                        var condition = new BasicExpressionToken(null, null, null, step.Condition);
-                                        conditionReTestResult = templateEvaluator.EvaluateStepIf(condition, step.ExecutionContext.ExpressionValues, step.ExecutionContext.ExpressionFunctions, step.ExecutionContext.ToExpressionState());
                                     }
                                     catch (Exception ex)
                                     {
                                         // Cancel the step since we get exception while re-evaluate step condition.
                                         Trace.Info("Caught exception from expression when re-test condition on job cancellation.");
                                         step.ExecutionContext.Error(ex);
+                                        conditionReTestResult = false;
                                     }
                                 }
 
-                                if (!conditionReTestResult)
+                                if (!conditionReTestResult.Value)
                                 {
                                     // Cancel the step.
                                     Trace.Info("Cancel current running step.");
@@ -167,35 +161,34 @@ namespace GitHub.Runner.Worker
 
                         // Evaluate condition.
                         step.ExecutionContext.Debug($"Evaluating condition for step: '{step.DisplayName}'");
-                        var conditionTraceWriter = new ConditionTraceWriter(Trace, step.ExecutionContext);
+                        Exception conditionEvaluateError = null;
-                        var conditionResult = false;
+                        ConditionResult conditionResult;
-                        var conditionEvaluateError = default(Exception);
                         if (HostContext.RunnerShutdownToken.IsCancellationRequested)
                         {
                             step.ExecutionContext.Debug($"Skip evaluate condition on runner shutdown.");
+                            conditionResult = false;
                         }
                         else
                         {
                             try
                             {
-                                var templateEvaluator = step.ExecutionContext.ToPipelineTemplateEvaluator(conditionTraceWriter);
+                                conditionResult = expressionManager.Evaluate(step.ExecutionContext, step.Condition);
-                                var condition = new BasicExpressionToken(null, null, null, step.Condition);
-                                conditionResult = templateEvaluator.EvaluateStepIf(condition, step.ExecutionContext.ExpressionValues, step.ExecutionContext.ExpressionFunctions, step.ExecutionContext.ToExpressionState());
                             }
                             catch (Exception ex)
                             {
                                 Trace.Info("Caught exception from expression.");
                                 Trace.Error(ex);
+                                conditionResult = false;
                                 conditionEvaluateError = ex;
                             }
                         }
 
                         // no evaluate error but condition is false
-                        if (!conditionResult && conditionEvaluateError == null)
+                        if (!conditionResult.Value && conditionEvaluateError == null)
                         {
                             // Condition == false
                             Trace.Info("Skipping step due to condition evaluation.");
-                            CompleteStep(step, nextStep, TaskResult.Skipped, resultCode: conditionTraceWriter.Trace);
+                            CompleteStep(step, nextStep, TaskResult.Skipped, resultCode: conditionResult.Trace);
                         }
                         else if (conditionEvaluateError != null)
                         {
@@ -255,7 +248,7 @@ namespace GitHub.Runner.Worker
             var templateEvaluator = step.ExecutionContext.ToPipelineTemplateEvaluator();
             try
             {
-                timeoutMinutes = templateEvaluator.EvaluateStepTimeout(step.Timeout, step.ExecutionContext.ExpressionValues, step.ExecutionContext.ExpressionFunctions);
+                timeoutMinutes = templateEvaluator.EvaluateStepTimeout(step.Timeout, step.ExecutionContext.ExpressionValues);
             }
             catch (Exception ex)
             {
@@ -346,7 +339,7 @@ namespace GitHub.Runner.Worker
                 var continueOnError = false;
                 try
                 {
-                    continueOnError = templateEvaluator.EvaluateStepContinueOnError(step.ContinueOnError, step.ExecutionContext.ExpressionValues, step.ExecutionContext.ExpressionFunctions);
+                    continueOnError = templateEvaluator.EvaluateStepContinueOnError(step.ContinueOnError, step.ExecutionContext.ExpressionValues);
                 }
                 catch (Exception ex)
                 {
@@ -358,7 +351,6 @@ namespace GitHub.Runner.Worker
 
                 if (continueOnError)
                 {
-                    step.ExecutionContext.Outcome = step.ExecutionContext.Result;
                     step.ExecutionContext.Result = TaskResult.Succeeded;
                     Trace.Info($"Updated step result (continue on error)");
                 }
@@ -399,7 +391,7 @@ namespace GitHub.Runner.Worker
                     var inputs = default(DictionaryContextData);
                     try
                     {
-                        inputs = templateEvaluator.EvaluateStepScopeInputs(scope.Inputs, executionContext.ExpressionValues, executionContext.ExpressionFunctions);
+                        inputs = templateEvaluator.EvaluateStepScopeInputs(scope.Inputs, executionContext.ExpressionValues);
                     }
                     catch (Exception ex)
                     {
@@ -455,7 +447,7 @@ namespace GitHub.Runner.Worker
                     var outputs = default(DictionaryContextData);
                     try
                     {
-                        outputs = templateEvaluator.EvaluateStepScopeOutputs(scope.Outputs, executionContext.ExpressionValues, executionContext.ExpressionFunctions);
+                        outputs = templateEvaluator.EvaluateStepScopeOutputs(scope.Outputs, executionContext.ExpressionValues);
                     }
                     catch (Exception ex)
                     {
@@ -483,43 +475,5 @@ namespace GitHub.Runner.Worker
 
             executionContext.Complete(result, resultCode: resultCode);
         }
-
-        private sealed class ConditionTraceWriter : ObjectTemplating::ITraceWriter
-        {
-            private readonly IExecutionContext _executionContext;
-            private readonly Tracing _trace;
-            private readonly StringBuilder _traceBuilder = new StringBuilder();
-
-            public string Trace => _traceBuilder.ToString();
-
-            public ConditionTraceWriter(Tracing trace, IExecutionContext executionContext)
-            {
-                ArgUtil.NotNull(trace, nameof(trace));
-                _trace = trace;
-                _executionContext = executionContext;
-            }
-
-            public void Error(string format, params Object[] args)
-            {
-                var message = StringUtil.Format(format, args);
-                _trace.Error(message);
-                _executionContext?.Debug(message);
-            }
-
-            public void Info(string format, params Object[] args)
-            {
-                var message = StringUtil.Format(format, args);
-                _trace.Info(message);
-                _executionContext?.Debug(message);
-                _traceBuilder.AppendLine(message);
-            }
-
-            public void Verbose(string format, params Object[] args)
-            {
-                var message = StringUtil.Format(format, args);
-                _trace.Verbose(message);
-                _executionContext?.Debug(message);
-            }
-        }
     }
 }

src/Runner.Worker/action_yaml.json

@@ -90,9 +90,10 @@
                 "github",
                 "strategy",
                 "matrix",
+                "steps",
                 "job",
                 "runner",
-                "hashFiles(1,255)"
+                "env"
             ],
             "string": {}
         },

src/Sdk/DTExpressions2/Expressions2/ExpressionConstants.cs

@@ -5,7 +5,7 @@ using GitHub.DistributedTask.Expressions2.Sdk.Functions;
 
 namespace GitHub.DistributedTask.Expressions2
 {
-    internal static class ExpressionConstants
+    public static class ExpressionConstants
     {
         static ExpressionConstants()
         {
@@ -15,7 +15,7 @@ namespace GitHub.DistributedTask.Expressions2
             AddFunction<Join>("join", 1, 2);
             AddFunction<StartsWith>("startsWith", 2, 2);
             AddFunction<ToJson>("toJson", 1, 1);
-            AddFunction<FromJson>("fromJson", 1, 1);
+            AddFunction<HashFiles>("hashFiles", 1, 1);
         }
 
         private static void AddFunction<T>(String name, Int32 minParameters, Int32 maxParameters)
@@ -24,6 +24,12 @@ namespace GitHub.DistributedTask.Expressions2
             WellKnownFunctions.Add(name, new FunctionInfo<T>(name, minParameters, maxParameters));
         }
 
+        public static void UpdateFunction<T>(String name, Int32 minParameters, Int32 maxParameters)
+                    where T : Function, new()
+        {
+            WellKnownFunctions[name] = new FunctionInfo<T>(name, minParameters, maxParameters);
+        }
+
         internal static readonly String False = "false";
         internal static readonly String Infinity = "Infinity";
         internal static readonly Int32 MaxDepth = 50;

src/Sdk/DTExpressions2/Expressions2/Sdk/Functions/FromJson.cs

@@ -1,24 +0,0 @@
-using System;
-using System.IO;
-using GitHub.DistributedTask.Pipelines.ContextData;
-using Newtonsoft.Json;
-using Newtonsoft.Json.Linq;
-
-namespace GitHub.DistributedTask.Expressions2.Sdk.Functions
-{
-    internal sealed class FromJson : Function
-    {
-        protected sealed override Object EvaluateCore(
-            EvaluationContext context,
-            out ResultMemory resultMemory)
-        {
-            resultMemory = null;
-            var json = Parameters[0].Evaluate(context).ConvertToString();
-            using (var stringReader = new StringReader(json))
-            using (var jsonReader = new JsonTextReader(stringReader) { DateParseHandling = DateParseHandling.None, FloatParseHandling = FloatParseHandling.Double })
-            {
-                var token = JToken.ReadFrom(jsonReader);
-                return token.ToPipelineContextData();
-            }
-        }
-    }}

src/Sdk/DTExpressions2/Expressions2/Sdk/Functions/HashFiles.cs

@@ -0,0 +1,122 @@
+using System;
+using System.Collections.Generic;
+using System.Linq;
+using System.Text;
+using Minimatch;
+using System.IO;
+using System.Security.Cryptography;
+using GitHub.DistributedTask.Expressions2.Sdk;
+using GitHub.DistributedTask.Pipelines.ContextData;
+using GitHub.DistributedTask.Pipelines.ObjectTemplating;
+namespace GitHub.DistributedTask.Expressions2.Sdk.Functions
+{
+    internal sealed class HashFiles : Function
+    {
+        protected sealed override Object EvaluateCore(
+            EvaluationContext context,
+            out ResultMemory resultMemory)
+        {
+            resultMemory = null;
+
+            // hashFiles() only works on the runner and only works with files under GITHUB_WORKSPACE
+            // Since GITHUB_WORKSPACE is set by runner, I am using that as the fact of this code runs on server or runner.
+            if (context.State is ObjectTemplating.TemplateContext templateContext &&
+                templateContext.ExpressionValues.TryGetValue(PipelineTemplateConstants.GitHub, out var githubContextData) &&
+                githubContextData is DictionaryContextData githubContext &&
+                githubContext.TryGetValue(PipelineTemplateConstants.Workspace, out var workspace) == true &&
+                workspace is StringContextData workspaceData)
+            {
+                string searchRoot = workspaceData.Value;
+                string pattern = Parameters[0].Evaluate(context).ConvertToString();
+
+                // Convert slashes on Windows
+                if (s_isWindows)
+                {
+                    pattern = pattern.Replace('\\', '/');
+                }
+
+                // Root the pattern
+                if (!Path.IsPathRooted(pattern))
+                {
+                    var patternRoot = s_isWindows ? searchRoot.Replace('\\', '/').TrimEnd('/') : searchRoot.TrimEnd('/');
+                    pattern = string.Concat(patternRoot, "/", pattern);
+                }
+
+                // Get all files
+                context.Trace.Info($"Search root directory: '{searchRoot}'");
+                context.Trace.Info($"Search pattern: '{pattern}'");
+                var files = Directory.GetFiles(searchRoot, "*", SearchOption.AllDirectories)
+                    .Select(x => s_isWindows ? x.Replace('\\', '/') : x)
+                    .OrderBy(x => x, StringComparer.Ordinal)
+                    .ToList();
+                if (files.Count == 0)
+                {
+                    throw new ArgumentException($"hashFiles('{ExpressionUtility.StringEscape(pattern)}') failed. Directory '{searchRoot}' is empty");
+                }
+                else
+                {
+                    context.Trace.Info($"Found {files.Count} files");
+                }
+
+                // Match
+                var matcher = new Minimatcher(pattern, s_minimatchOptions);
+                files = matcher.Filter(files)
+                    .Select(x => s_isWindows ? x.Replace('/', '\\') : x)
+                    .ToList();
+                if (files.Count == 0)
+                {
+                    throw new ArgumentException($"hashFiles('{ExpressionUtility.StringEscape(pattern)}') failed. Search pattern '{pattern}' doesn't match any file under '{searchRoot}'");
+                }
+                else
+                {
+                    context.Trace.Info($"{files.Count} matches to hash");
+                }
+
+                // Hash each file
+                List<byte> filesSha256 = new List<byte>();
+                foreach (var file in files)
+                {
+                    context.Trace.Info($"Hash {file}");
+                    using (SHA256 sha256hash = SHA256.Create())
+                    {
+                        using (var fileStream = File.OpenRead(file))
+                        {
+                            filesSha256.AddRange(sha256hash.ComputeHash(fileStream));
+                        }
+                    }
+                }
+
+                // Hash the hashes
+                using (SHA256 sha256hash = SHA256.Create())
+                {
+                    var hashBytes = sha256hash.ComputeHash(filesSha256.ToArray());
+                    StringBuilder hashString = new StringBuilder();
+                    for (int i = 0; i < hashBytes.Length; i++)
+                    {
+                        hashString.Append(hashBytes[i].ToString("x2"));
+                    }
+                    var result = hashString.ToString();
+                    context.Trace.Info($"Final hash result: '{result}'");
+                    return result;
+                }
+            }
+            else
+            {
+                throw new InvalidOperationException("'hashfiles' expression function is only supported under runner context.");
+            }
+        }
+
+        private static readonly bool s_isWindows = Environment.OSVersion.Platform != PlatformID.Unix && Environment.OSVersion.Platform != PlatformID.MacOSX;
+
+        // Only support basic globbing (* ? and []) and globstar (**)
+        private static readonly Options s_minimatchOptions = new Options
+        {
+            Dot = true,
+            NoBrace = true,
+            NoCase = s_isWindows,
+            NoComment = true,
+            NoExt = true,
+            NoNegate = true,
+        };
+    }
+}

src/Sdk/DTGenerated/Generated/TaskAgentHttpClientBase.cs

@@ -779,65 +779,5 @@ namespace GitHub.DistributedTask.WebApi
                 userState: userState,
                 cancellationToken: cancellationToken);
         }
-
-        /// <summary>
-        /// [Preview API]
-        /// </summary>
-        /// <param name="poolId"></param>
-        /// <param name="agentId"></param>
-        /// <param name="userState"></param>
-        /// <param name="cancellationToken">The cancellation token to cancel operation.</param>
-        public Task<String> GetAgentAuthUrlAsync(
-            int poolId,
-            int agentId,
-            object userState = null,
-            CancellationToken cancellationToken = default)
-        {
-            HttpMethod httpMethod = new HttpMethod("GET");
-            Guid locationId = new Guid("a82a119c-1e46-44b6-8d75-c82a79cf975b");
-            object routeValues = new { poolId = poolId, agentId = agentId };
-
-            return SendAsync<String>(
-                httpMethod,
-                locationId,
-                routeValues: routeValues,
-                version: new ApiResourceVersion(6.0, 1),
-                userState: userState,
-                cancellationToken: cancellationToken);
-        }
-
-        /// <summary>
-        /// [Preview API]
-        /// </summary>
-        /// <param name="poolId"></param>
-        /// <param name="agentId"></param>
-        /// <param name="error"></param>
-        /// <param name="userState"></param>
-        /// <param name="cancellationToken">The cancellation token to cancel operation.</param>
-        [EditorBrowsable(EditorBrowsableState.Never)]
-        public virtual async Task ReportAgentAuthUrlMigrationErrorAsync(
-            int poolId,
-            int agentId,
-            string error,
-            object userState = null,
-            CancellationToken cancellationToken = default)
-        {
-            HttpMethod httpMethod = new HttpMethod("POST");
-            Guid locationId = new Guid("a82a119c-1e46-44b6-8d75-c82a79cf975b");
-            object routeValues = new { poolId = poolId, agentId = agentId };
-            HttpContent content = new ObjectContent<string>(error, new VssJsonMediaTypeFormatter(true));
-
-            using (HttpResponseMessage response = await SendAsync(
-                httpMethod,
-                locationId,
-                routeValues: routeValues,
-                version: new ApiResourceVersion(6.0, 1),
-                userState: userState,
-                cancellationToken: cancellationToken,
-                content: content).ConfigureAwait(false))
-            {
-                return;
-            }
-        }
     }
 }

src/Sdk/DTObjectTemplating/ObjectTemplating/Schema/Definition.cs

@@ -1,5 +1,4 @@
 using System;
-using System.Collections.Generic;
 using System.Linq;
 using GitHub.DistributedTask.ObjectTemplating.Tokens;
 
@@ -23,27 +22,10 @@ namespace GitHub.DistributedTask.ObjectTemplating.Schema
                 {
                     var context = definition[i].Value.AssertSequence($"{TemplateConstants.Context}");
                     definition.RemoveAt(i);
-                    var readerContext = new HashSet<String>(StringComparer.OrdinalIgnoreCase);
+                    Context = context
-                    var evaluatorContext = new HashSet<String>(StringComparer.OrdinalIgnoreCase);
+                        .Select(x => x.AssertString($"{TemplateConstants.Context} item").Value)
-                    foreach (TemplateToken item in context)
+                        .Distinct()
-                    {
+                        .ToArray();
-                        var itemStr = item.AssertString($"{TemplateConstants.Context} item").Value;
-                        readerContext.Add(itemStr);
-
-                        // Remove min/max parameter info
-                        var paramIndex = itemStr.IndexOf('(');
-                        if (paramIndex > 0)
-                        {
-                            evaluatorContext.Add(String.Concat(itemStr.Substring(0, paramIndex + 1), ")"));
-                        }
-                        else
-                        {
-                            evaluatorContext.Add(itemStr);
-                        }
-                    }
-
-                    ReaderContext = readerContext.ToArray();
-                    EvaluatorContext = evaluatorContext.ToArray();
                 }
                 else if (String.Equals(definitionKey.Value, TemplateConstants.Description, StringComparison.Ordinal))
                 {
@@ -58,17 +40,7 @@ namespace GitHub.DistributedTask.ObjectTemplating.Schema
 
         internal abstract DefinitionType DefinitionType { get; }
 
-        /// <summary>
+        internal String[] Context { get; private set; } = new String[0];
-        /// Used by the template reader to determine allowed expression values and functions.
-        /// Also used by the template reader to validate function min/max parameters.
-        /// </summary>
-        internal String[] ReaderContext { get; private set; } = new String[0];
-
-        /// <summary>
-        /// Used by the template evaluator to determine allowed expression values and functions.
-        /// The min/max parameter info is omitted.
-        /// </summary>
-        internal String[] EvaluatorContext { get; private set; } = new String[0];
 
         internal abstract void Validate(
             TemplateSchema schema,

src/Sdk/DTObjectTemplating/ObjectTemplating/Schema/MappingDefinition.cs

@@ -30,7 +30,8 @@ namespace GitHub.DistributedTask.ObjectTemplating.Schema
                                     foreach (var propertiesPair in properties)
                                     {
                                         var propertyName = propertiesPair.Key.AssertString($"{TemplateConstants.Definition} {TemplateConstants.Mapping} {TemplateConstants.Properties} key");
-                                        Properties.Add(propertyName.Value, new PropertyValue(propertiesPair.Value));
+                                        var propertyValue = propertiesPair.Value.AssertString($"{TemplateConstants.Definition} {TemplateConstants.Mapping} {TemplateConstants.Properties} value");
+                                        Properties.Add(propertyName.Value, new PropertyValue(propertyValue.Value));
                                     }
                                     break;
 
@@ -84,7 +85,7 @@ namespace GitHub.DistributedTask.ObjectTemplating.Schema
                 }
                 else
                 {
-                    throw new ArgumentException($"Property '{TemplateConstants.LooseKeyType}' is defined but '{TemplateConstants.LooseValueType}' is not defined on '{name}'");
+                    throw new ArgumentException($"Property '{TemplateConstants.LooseKeyType}' is defined but '{TemplateConstants.LooseValueType}' is not defined");
                 }
             }
             // Otherwise validate loose value type not be defined
@@ -94,21 +95,16 @@ namespace GitHub.DistributedTask.ObjectTemplating.Schema
             }
 
             // Lookup each property
-            foreach (var property in Properties)
+            foreach (var property in Properties.Values)
             {
-                if (String.IsNullOrEmpty(property.Value.Type))
+                schema.GetDefinition(property.Type);
-                {
-                    throw new ArgumentException($"Type not specified for the '{property.Key}' property on the '{name}' type");
-                }
-
-                schema.GetDefinition(property.Value.Type);
             }
 
             if (!String.IsNullOrEmpty(Inherits))
             {
                 var inherited = schema.GetDefinition(Inherits);
 
-                if (inherited.ReaderContext.Length > 0)
+                if (inherited.Context.Length > 0)
                 {
                     throw new NotSupportedException($"Property '{TemplateConstants.Context}' is not supported on inhertied definitions");
                 }

src/Sdk/DTObjectTemplating/ObjectTemplating/Schema/OneOfDefinition.cs

@@ -62,7 +62,7 @@ namespace GitHub.DistributedTask.ObjectTemplating.Schema
             {
                 var nestedDefinition = schema.GetDefinition(nestedType);
 
-                if (nestedDefinition.ReaderContext.Length > 0)
+                if (nestedDefinition.Context.Length > 0)
                 {
                     throw new ArgumentException($"'{name}' is a one-of definition and references another definition that defines context. This is currently not supported.");
                 }

src/Sdk/DTObjectTemplating/ObjectTemplating/Schema/PropertyValue.cs

@@ -1,40 +1,18 @@
 using System;
-using GitHub.DistributedTask.ObjectTemplating.Tokens;
 
 namespace GitHub.DistributedTask.ObjectTemplating.Schema
 {
     internal sealed class PropertyValue
     {
-        internal PropertyValue(TemplateToken token)
+        internal PropertyValue()
         {
-            if (token is StringToken stringToken)
+        }
-            {
+
-                Type = stringToken.Value;
+        internal PropertyValue(String type)
-            }
+        {
-            else
+            Type = type;
-            {
-                var mapping = token.AssertMapping($"{TemplateConstants.MappingPropertyValue}");
-                foreach (var mappingPair in mapping)
-                {
-                    var mappingKey = mappingPair.Key.AssertString($"{TemplateConstants.MappingPropertyValue} key");
-                    switch (mappingKey.Value)
-                    {
-                        case TemplateConstants.Type:
-                            Type = mappingPair.Value.AssertString($"{TemplateConstants.MappingPropertyValue} {TemplateConstants.Type}").Value;
-                            break;
-                        case TemplateConstants.Required:
-                            Required = mappingPair.Value.AssertBoolean($"{TemplateConstants.MappingPropertyValue} {TemplateConstants.Required}").Value;
-                            break;
-                        default:
-                            mappingKey.AssertUnexpectedValue($"{TemplateConstants.MappingPropertyValue} key");
-                            break;
-                    }
-                }
-            }
         }
 
         internal String Type { get; set; }
-
-        internal Boolean Required { get; set; }
     }
 }

src/Sdk/DTObjectTemplating/ObjectTemplating/Schema/TemplateSchema.cs

@@ -312,8 +312,8 @@ namespace GitHub.DistributedTask.ObjectTemplating.Schema
 
                     // template-schema
                     mappingDefinition = new MappingDefinition();
-                    mappingDefinition.Properties.Add(TemplateConstants.Version, new PropertyValue(new StringToken(null, null, null, TemplateConstants.NonEmptyString)));
+                    mappingDefinition.Properties.Add(TemplateConstants.Version, new PropertyValue(TemplateConstants.NonEmptyString));
-                    mappingDefinition.Properties.Add(TemplateConstants.Definitions, new PropertyValue(new StringToken(null, null, null, TemplateConstants.Definitions)));
+                    mappingDefinition.Properties.Add(TemplateConstants.Definitions, new PropertyValue(TemplateConstants.Definitions));
                     schema.Definitions.Add(TemplateConstants.TemplateSchema, mappingDefinition);
 
                     // definitions
@@ -335,9 +335,9 @@ namespace GitHub.DistributedTask.ObjectTemplating.Schema
 
                     // null-definition
                     mappingDefinition = new MappingDefinition();
-                    mappingDefinition.Properties.Add(TemplateConstants.Description, new PropertyValue(new StringToken(null, null, null, TemplateConstants.String)));
+                    mappingDefinition.Properties.Add(TemplateConstants.Description, new PropertyValue(TemplateConstants.String));
-                    mappingDefinition.Properties.Add(TemplateConstants.Context, new PropertyValue(new StringToken(null, null, null, TemplateConstants.SequenceOfNonEmptyString)));
+                    mappingDefinition.Properties.Add(TemplateConstants.Context, new PropertyValue(TemplateConstants.SequenceOfNonEmptyString));
-                    mappingDefinition.Properties.Add(TemplateConstants.Null, new PropertyValue(new StringToken(null, null, null, TemplateConstants.NullDefinitionProperties)));
+                    mappingDefinition.Properties.Add(TemplateConstants.Null, new PropertyValue(TemplateConstants.NullDefinitionProperties));
                     schema.Definitions.Add(TemplateConstants.NullDefinition, mappingDefinition);
 
                     // null-definition-properties
@@ -346,9 +346,9 @@ namespace GitHub.DistributedTask.ObjectTemplating.Schema
 
                     // boolean-definition
                     mappingDefinition = new MappingDefinition();
-                    mappingDefinition.Properties.Add(TemplateConstants.Description, new PropertyValue(new StringToken(null, null, null, TemplateConstants.String)));
+                    mappingDefinition.Properties.Add(TemplateConstants.Description, new PropertyValue(TemplateConstants.String));
-                    mappingDefinition.Properties.Add(TemplateConstants.Context, new PropertyValue(new StringToken(null, null, null, TemplateConstants.SequenceOfNonEmptyString)));
+                    mappingDefinition.Properties.Add(TemplateConstants.Context, new PropertyValue(TemplateConstants.SequenceOfNonEmptyString));
-                    mappingDefinition.Properties.Add(TemplateConstants.Boolean, new PropertyValue(new StringToken(null, null, null, TemplateConstants.BooleanDefinitionProperties)));
+                    mappingDefinition.Properties.Add(TemplateConstants.Boolean, new PropertyValue(TemplateConstants.BooleanDefinitionProperties));
                     schema.Definitions.Add(TemplateConstants.BooleanDefinition, mappingDefinition);
 
                     // boolean-definition-properties
@@ -357,9 +357,9 @@ namespace GitHub.DistributedTask.ObjectTemplating.Schema
 
                     // number-definition
                     mappingDefinition = new MappingDefinition();
-                    mappingDefinition.Properties.Add(TemplateConstants.Description, new PropertyValue(new StringToken(null, null, null, TemplateConstants.String)));
+                    mappingDefinition.Properties.Add(TemplateConstants.Description, new PropertyValue(TemplateConstants.String));
-                    mappingDefinition.Properties.Add(TemplateConstants.Context, new PropertyValue(new StringToken(null, null, null, TemplateConstants.SequenceOfNonEmptyString)));
+                    mappingDefinition.Properties.Add(TemplateConstants.Context, new PropertyValue(TemplateConstants.SequenceOfNonEmptyString));
-                    mappingDefinition.Properties.Add(TemplateConstants.Number, new PropertyValue(new StringToken(null, null, null, TemplateConstants.NumberDefinitionProperties)));
+                    mappingDefinition.Properties.Add(TemplateConstants.Number, new PropertyValue(TemplateConstants.NumberDefinitionProperties));
                     schema.Definitions.Add(TemplateConstants.NumberDefinition, mappingDefinition);
 
                     // number-definition-properties
@@ -368,68 +368,55 @@ namespace GitHub.DistributedTask.ObjectTemplating.Schema
 
                     // string-definition
                     mappingDefinition = new MappingDefinition();
-                    mappingDefinition.Properties.Add(TemplateConstants.Description, new PropertyValue(new StringToken(null, null, null, TemplateConstants.String)));
+                    mappingDefinition.Properties.Add(TemplateConstants.Description, new PropertyValue(TemplateConstants.String));
-                    mappingDefinition.Properties.Add(TemplateConstants.Context, new PropertyValue(new StringToken(null, null, null, TemplateConstants.SequenceOfNonEmptyString)));
+                    mappingDefinition.Properties.Add(TemplateConstants.Context, new PropertyValue(TemplateConstants.SequenceOfNonEmptyString));
-                    mappingDefinition.Properties.Add(TemplateConstants.String, new PropertyValue(new StringToken(null, null, null, TemplateConstants.StringDefinitionProperties)));
+                    mappingDefinition.Properties.Add(TemplateConstants.String, new PropertyValue(TemplateConstants.StringDefinitionProperties));
                     schema.Definitions.Add(TemplateConstants.StringDefinition, mappingDefinition);
 
                     // string-definition-properties
                     mappingDefinition = new MappingDefinition();
-                    mappingDefinition.Properties.Add(TemplateConstants.Constant, new PropertyValue(new StringToken(null, null, null, TemplateConstants.NonEmptyString)));
+                    mappingDefinition.Properties.Add(TemplateConstants.Constant, new PropertyValue(TemplateConstants.NonEmptyString));
-                    mappingDefinition.Properties.Add(TemplateConstants.IgnoreCase, new PropertyValue(new StringToken(null, null, null,TemplateConstants.Boolean)));
+                    mappingDefinition.Properties.Add(TemplateConstants.IgnoreCase, new PropertyValue(TemplateConstants.Boolean));
-                    mappingDefinition.Properties.Add(TemplateConstants.RequireNonEmpty, new PropertyValue(new StringToken(null, null, null, TemplateConstants.Boolean)));
+                    mappingDefinition.Properties.Add(TemplateConstants.RequireNonEmpty, new PropertyValue(TemplateConstants.Boolean));
                     schema.Definitions.Add(TemplateConstants.StringDefinitionProperties, mappingDefinition);
 
                     // sequence-definition
                     mappingDefinition = new MappingDefinition();
-                    mappingDefinition.Properties.Add(TemplateConstants.Description, new PropertyValue(new StringToken(null, null, null, TemplateConstants.String)));
+                    mappingDefinition.Properties.Add(TemplateConstants.Description, new PropertyValue(TemplateConstants.String));
-                    mappingDefinition.Properties.Add(TemplateConstants.Context, new PropertyValue(new StringToken(null, null, null, TemplateConstants.SequenceOfNonEmptyString)));
+                    mappingDefinition.Properties.Add(TemplateConstants.Context, new PropertyValue(TemplateConstants.SequenceOfNonEmptyString));
-                    mappingDefinition.Properties.Add(TemplateConstants.Sequence, new PropertyValue(new StringToken(null, null, null, TemplateConstants.SequenceDefinitionProperties)));
+                    mappingDefinition.Properties.Add(TemplateConstants.Sequence, new PropertyValue(TemplateConstants.SequenceDefinitionProperties));
                     schema.Definitions.Add(TemplateConstants.SequenceDefinition, mappingDefinition);
 
                     // sequence-definition-properties
                     mappingDefinition = new MappingDefinition();
-                    mappingDefinition.Properties.Add(TemplateConstants.ItemType, new PropertyValue(new StringToken(null, null, null, TemplateConstants.NonEmptyString)));
+                    mappingDefinition.Properties.Add(TemplateConstants.ItemType, new PropertyValue(TemplateConstants.NonEmptyString));
                     schema.Definitions.Add(TemplateConstants.SequenceDefinitionProperties, mappingDefinition);
 
                     // mapping-definition
                     mappingDefinition = new MappingDefinition();
-                    mappingDefinition.Properties.Add(TemplateConstants.Description, new PropertyValue(new StringToken(null, null, null, TemplateConstants.String)));
+                    mappingDefinition.Properties.Add(TemplateConstants.Description, new PropertyValue(TemplateConstants.String));
-                    mappingDefinition.Properties.Add(TemplateConstants.Context, new PropertyValue(new StringToken(null, null, null, TemplateConstants.SequenceOfNonEmptyString)));
+                    mappingDefinition.Properties.Add(TemplateConstants.Context, new PropertyValue(TemplateConstants.SequenceOfNonEmptyString));
-                    mappingDefinition.Properties.Add(TemplateConstants.Mapping, new PropertyValue(new StringToken(null, null, null, TemplateConstants.MappingDefinitionProperties)));
+                    mappingDefinition.Properties.Add(TemplateConstants.Mapping, new PropertyValue(TemplateConstants.MappingDefinitionProperties));
                     schema.Definitions.Add(TemplateConstants.MappingDefinition, mappingDefinition);
 
                     // mapping-definition-properties
                     mappingDefinition = new MappingDefinition();
-                    mappingDefinition.Properties.Add(TemplateConstants.Properties, new PropertyValue(new StringToken(null, null, null, TemplateConstants.Properties)));
+                    mappingDefinition.Properties.Add(TemplateConstants.Properties, new PropertyValue(TemplateConstants.Properties));
-                    mappingDefinition.Properties.Add(TemplateConstants.LooseKeyType, new PropertyValue(new StringToken(null, null, null, TemplateConstants.NonEmptyString)));
+                    mappingDefinition.Properties.Add(TemplateConstants.LooseKeyType, new PropertyValue(TemplateConstants.NonEmptyString));
-                    mappingDefinition.Properties.Add(TemplateConstants.LooseValueType, new PropertyValue(new StringToken(null, null, null, TemplateConstants.NonEmptyString)));
+                    mappingDefinition.Properties.Add(TemplateConstants.LooseValueType, new PropertyValue(TemplateConstants.NonEmptyString));
                     schema.Definitions.Add(TemplateConstants.MappingDefinitionProperties, mappingDefinition);
 
                     // properties
                     mappingDefinition = new MappingDefinition();
                     mappingDefinition.LooseKeyType = TemplateConstants.NonEmptyString;
-                    mappingDefinition.LooseValueType = TemplateConstants.PropertyValue;
+                    mappingDefinition.LooseValueType = TemplateConstants.NonEmptyString;
                     schema.Definitions.Add(TemplateConstants.Properties, mappingDefinition);
 
-                    // property-value
-                    oneOfDefinition = new OneOfDefinition();
-                    oneOfDefinition.OneOf.Add(TemplateConstants.NonEmptyString);
-                    oneOfDefinition.OneOf.Add(TemplateConstants.MappingPropertyValue);
-                    schema.Definitions.Add(TemplateConstants.PropertyValue, oneOfDefinition);
-
-                    // mapping-property-value
-                    mappingDefinition = new MappingDefinition();
-                    mappingDefinition.Properties.Add(TemplateConstants.Type, new PropertyValue(new StringToken(null, null, null, TemplateConstants.NonEmptyString)));
-                    mappingDefinition.Properties.Add(TemplateConstants.Required, new PropertyValue(new StringToken(null, null, null, TemplateConstants.Boolean)));
-                    schema.Definitions.Add(TemplateConstants.MappingPropertyValue, mappingDefinition);
-
-
                     // one-of-definition
                     mappingDefinition = new MappingDefinition();
-                    mappingDefinition.Properties.Add(TemplateConstants.Description, new PropertyValue(new StringToken(null, null, null, TemplateConstants.String)));
+                    mappingDefinition.Properties.Add(TemplateConstants.Description, new PropertyValue(TemplateConstants.String));
-                    mappingDefinition.Properties.Add(TemplateConstants.Context, new PropertyValue(new StringToken(null, null, null, TemplateConstants.SequenceOfNonEmptyString)));
+                    mappingDefinition.Properties.Add(TemplateConstants.Context, new PropertyValue(TemplateConstants.SequenceOfNonEmptyString));
-                    mappingDefinition.Properties.Add(TemplateConstants.OneOf, new PropertyValue(new StringToken(null, null, null, TemplateConstants.SequenceOfNonEmptyString)));
+                    mappingDefinition.Properties.Add(TemplateConstants.OneOf, new PropertyValue(TemplateConstants.SequenceOfNonEmptyString));
                     schema.Definitions.Add(TemplateConstants.OneOfDefinition, mappingDefinition);
 
                     // non-empty-string
@@ -490,4 +477,4 @@ namespace GitHub.DistributedTask.ObjectTemplating.Schema
         private static readonly Regex s_definitionNameRegex = new Regex("^[a-zA-Z_][a-zA-Z0-9_-]*$", RegexOptions.Compiled);
         private static TemplateSchema s_schema;
     }
-}
+}

src/Sdk/DTObjectTemplating/ObjectTemplating/TemplateConstants.cs

@@ -22,11 +22,9 @@ namespace GitHub.DistributedTask.ObjectTemplating
         internal const String ItemType = "item-type";
         internal const String LooseKeyType = "loose-key-type";
         internal const String LooseValueType = "loose-value-type";
-        internal const String MaxConstant = "MAX";
         internal const String Mapping = "mapping";
         internal const String MappingDefinition = "mapping-definition";
         internal const String MappingDefinitionProperties = "mapping-definition-properties";
-        internal const String MappingPropertyValue = "mapping-property-value";
         internal const String NonEmptyString = "non-empty-string";
         internal const String Null = "null";
         internal const String NullDefinition = "null-definition";
@@ -37,9 +35,7 @@ namespace GitHub.DistributedTask.ObjectTemplating
         internal const String OneOf = "one-of";
         internal const String OneOfDefinition = "one-of-definition";
         internal const String OpenExpression = "${{";
-        internal const String PropertyValue = "property-value";
         internal const String Properties = "properties";
-        internal const String Required = "required";
         internal const String RequireNonEmpty = "require-non-empty";
         internal const String Scalar = "scalar";
         internal const String ScalarDefinition = "scalar-definition";
@@ -47,7 +43,6 @@ namespace GitHub.DistributedTask.ObjectTemplating
         internal const String Sequence = "sequence";
         internal const String SequenceDefinition = "sequence-definition";
         internal const String SequenceDefinitionProperties = "sequence-definition-properties";
-        internal const String Type = "type";
         internal const String SequenceOfNonEmptyString = "sequence-of-non-empty-string";
         internal const String String = "string";
         internal const String StringDefinition = "string-definition";

src/Sdk/DTObjectTemplating/ObjectTemplating/TemplateEvaluator.cs

@@ -47,16 +47,7 @@ namespace GitHub.DistributedTask.ObjectTemplating
             var evaluator = new TemplateEvaluator(context, template, removeBytes);
             try
             {
-                var availableContext = new HashSet<String>(StringComparer.OrdinalIgnoreCase);
+                var availableContext = new HashSet<String>(context.ExpressionValues.Keys);
-                foreach (var key in context.ExpressionValues.Keys)
-                {
-                    availableContext.Add(key);
-                }
-                foreach (var function in context.ExpressionFunctions)
-                {
-                    availableContext.Add($"{function.Name}()");
-                }
-
                 var definitionInfo = new DefinitionInfo(context.Schema, type, availableContext);
                 result = evaluator.Evaluate(definitionInfo);
 
@@ -191,14 +182,12 @@ namespace GitHub.DistributedTask.ObjectTemplating
             }
 
             var keys = new HashSet<String>(StringComparer.OrdinalIgnoreCase);
-            var hasExpressionKey = false;
 
             while (m_unraveler.AllowScalar(definition.Expand, out ScalarToken nextKeyScalar))
             {
                 // Expression
                 if (nextKeyScalar is ExpressionToken)
                 {
-                    hasExpressionKey = true;
                     var anyDefinition = new DefinitionInfo(definition, TemplateConstants.Any);
                     mapping.Add(nextKeyScalar, Evaluate(anyDefinition));
                     continue;
@@ -279,19 +268,6 @@ namespace GitHub.DistributedTask.ObjectTemplating
                 String listToDeDuplicate = String.Join(", ", nonDuplicates);
                 m_context.Error(mapping, TemplateStrings.UnableToDetermineOneOf(listToDeDuplicate));
             }
-            else if (mappingDefinitions.Count == 1 && !hasExpressionKey)
-            {
-                foreach (var property in mappingDefinitions[0].Properties)
-                {
-                    if (property.Value.Required)
-                    {
-                        if (!keys.Contains(property.Key))
-                        {
-                            m_context.Error(mapping, $"Required property is missing: {property.Key}");
-                        }
-                    }
-                }
-            }
 
             m_unraveler.ReadMappingEnd();
         }
@@ -402,13 +378,14 @@ namespace GitHub.DistributedTask.ObjectTemplating
                 Definition = m_schema.GetDefinition(name);
 
                 // Determine whether to expand
-                m_allowedContext = Definition.EvaluatorContext;
+                if (Definition.Context.Length > 0)
-                if (Definition.EvaluatorContext.Length > 0)
                 {
+                    m_allowedContext = Definition.Context;
                     Expand = m_availableContext.IsSupersetOf(m_allowedContext);
                 }
                 else
                 {
+                    m_allowedContext = new String[0];
                     Expand = false;
                 }
             }
@@ -424,9 +401,9 @@ namespace GitHub.DistributedTask.ObjectTemplating
                 Definition = m_schema.GetDefinition(name);
 
                 // Determine whether to expand
-                if (Definition.EvaluatorContext.Length > 0)
+                if (Definition.Context.Length > 0)
                 {
-                    m_allowedContext = new HashSet<String>(parent.m_allowedContext.Concat(Definition.EvaluatorContext), StringComparer.OrdinalIgnoreCase).ToArray();
+                    m_allowedContext = new HashSet<String>(parent.m_allowedContext.Concat(Definition.Context)).ToArray();
                     Expand = m_availableContext.IsSupersetOf(m_allowedContext);
                 }
                 else

src/Sdk/DTObjectTemplating/ObjectTemplating/TemplateException.cs

@@ -49,14 +49,6 @@ namespace GitHub.DistributedTask.ObjectTemplating
             m_errors = new List<TemplateValidationError>(errors ?? Enumerable.Empty<TemplateValidationError>());
         }
 
-        public TemplateValidationException(
-            String message,
-            IEnumerable<TemplateValidationError> errors)
-            : this(message)
-        {
-            m_errors = new List<TemplateValidationError>(errors ?? Enumerable.Empty<TemplateValidationError>());
-        }
-
         public TemplateValidationException(String message)
             : base(message)
         {

src/Sdk/DTObjectTemplating/ObjectTemplating/TemplateReader.cs

@@ -178,15 +178,14 @@ namespace GitHub.DistributedTask.ObjectTemplating
             }
 
             var keys = new HashSet<String>(StringComparer.OrdinalIgnoreCase);
-            var hasExpressionKey = false;
 
             while (m_objectReader.AllowLiteral(out LiteralToken rawLiteral))
             {
                 var nextKeyScalar = ParseScalar(rawLiteral, definition.AllowedContext);
+
                 // Expression
                 if (nextKeyScalar is ExpressionToken)
                 {
-                    hasExpressionKey = true;
                     // Legal
                     if (definition.AllowedContext.Length > 0)
                     {
@@ -281,19 +280,7 @@ namespace GitHub.DistributedTask.ObjectTemplating
                 String listToDeDuplicate = String.Join(", ", nonDuplicates);
                 m_context.Error(mapping, TemplateStrings.UnableToDetermineOneOf(listToDeDuplicate));
             }
-            else if (mappingDefinitions.Count == 1 && !hasExpressionKey)
+
-            {
-                foreach (var property in mappingDefinitions[0].Properties)
-                {
-                    if (property.Value.Required)
-                    {
-                        if (!keys.Contains(property.Key))
-                        {
-                            m_context.Error(mapping, $"Required property is missing: {property.Key}");
-                        }
-                    }
-                }
-            }
             ExpectMappingEnd();
         }
 
@@ -780,8 +767,15 @@ namespace GitHub.DistributedTask.ObjectTemplating
                 // Lookup the definition
                 Definition = m_schema.GetDefinition(name);
 
-                // Record allowed context
+                // Determine whether to expand
-                AllowedContext = Definition.ReaderContext;
+                if (Definition.Context.Length > 0)
+                {
+                    AllowedContext = Definition.Context;
+                }
+                else
+                {
+                    AllowedContext = new String[0];
+                }
             }
 
             public DefinitionInfo(
@@ -793,10 +787,10 @@ namespace GitHub.DistributedTask.ObjectTemplating
                 // Lookup the definition
                 Definition = m_schema.GetDefinition(name);
 
-                // Record allowed context
+                // Determine whether to expand
-                if (Definition.ReaderContext.Length > 0)
+                if (Definition.Context.Length > 0)
                 {
-                    AllowedContext = new HashSet<String>(parent.AllowedContext.Concat(Definition.ReaderContext), StringComparer.OrdinalIgnoreCase).ToArray();
+                    AllowedContext = new HashSet<String>(parent.AllowedContext.Concat(Definition.Context)).ToArray();
                 }
                 else
                 {

src/Sdk/DTObjectTemplating/ObjectTemplating/TemplateValidationErrors.cs

@@ -2,7 +2,6 @@
 using System.Collections;
 using System.Collections.Generic;
 using System.ComponentModel;
-using System.Linq;
 using System.Runtime.Serialization;
 
 namespace GitHub.DistributedTask.ObjectTemplating
@@ -42,7 +41,7 @@ namespace GitHub.DistributedTask.ObjectTemplating
         {
             for (int i = 0; i < 50; i++)
             {
-                String message = !String.IsNullOrEmpty(messagePrefix) ? $"{messagePrefix} {ex.Message}" : ex.ToString();
+                String message = !String.IsNullOrEmpty(messagePrefix) ? $"{messagePrefix} {ex.Message}" : ex.Message;
                 Add(new TemplateValidationError(message));
                 if (ex.InnerException == null)
                 {
@@ -89,23 +88,6 @@ namespace GitHub.DistributedTask.ObjectTemplating
             }
         }
 
-        /// <summary>
-        /// Throws <c ref="TemplateValidationException" /> if any errors.
-        /// <param name="prefix">The error message prefix</param>
-        /// </summary>
-        public void Check(String prefix)
-        {
-            if (String.IsNullOrEmpty(prefix))
-            {
-                this.Check();
-            }
-            else if (m_errors.Count > 0)
-            {
-                var message = $"{prefix.Trim()} {String.Join(",", m_errors.Select(e => e.Message))}";
-                throw new TemplateValidationException(message, m_errors);
-            }
-        }
-
         public void Clear()
         {
             m_errors.Clear();

src/Sdk/DTObjectTemplating/ObjectTemplating/Tokens/ExpressionToken.cs

@@ -1,9 +1,7 @@
 using System;
-using System.Collections.Generic;
 using System.ComponentModel;
-using System.Globalization;
+using System.Linq;
 using System.Runtime.Serialization;
-using System.Text.RegularExpressions;
 using GitHub.DistributedTask.Expressions2;
 using GitHub.DistributedTask.Expressions2.Sdk;
 using GitHub.Services.WebApi.Internal;
@@ -37,29 +35,11 @@ namespace GitHub.DistributedTask.ObjectTemplating.Tokens
             String[] allowedContext,
             out Exception ex)
         {
-            // Create dummy named values and functions
+            // Create dummy allowed contexts
-            var namedValues = new List<INamedValueInfo>();
+            INamedValueInfo[] namedValues = null;
-            var functions = new List<IFunctionInfo>();
             if (allowedContext?.Length > 0)
             {
-                foreach (var contextItem in allowedContext)
+                namedValues = allowedContext.Select(x => new NamedValueInfo<ContextValueNode>(x)).ToArray();
-                {
-                    var match = s_function.Match(contextItem);
-                    if (match.Success)
-                    {
-                        var functionName = match.Groups[1].Value;
-                        var minParameters = Int32.Parse(match.Groups[2].Value, NumberStyles.None, CultureInfo.InvariantCulture);
-                        var maxParametersRaw = match.Groups[3].Value;
-                        var maxParameters = String.Equals(maxParametersRaw, TemplateConstants.MaxConstant, StringComparison.Ordinal)
-                            ? Int32.MaxValue
-                            : Int32.Parse(maxParametersRaw, NumberStyles.None, CultureInfo.InvariantCulture);
-                        functions.Add(new FunctionInfo<DummyFunction>(functionName, minParameters, maxParameters));
-                    }
-                    else
-                    {
-                        namedValues.Add(new NamedValueInfo<ContextValueNode>(contextItem));
-                    }
-                }
             }
 
             // Parse
@@ -67,7 +47,7 @@ namespace GitHub.DistributedTask.ObjectTemplating.Tokens
             ExpressionNode root = null;
             try
             {
-                root = new ExpressionParser().CreateTree(expression, null, namedValues, functions) as ExpressionNode;
+                root = new ExpressionParser().CreateTree(expression, null, namedValues, null) as ExpressionNode;
 
                 result = true;
                 ex = null;
@@ -80,18 +60,5 @@ namespace GitHub.DistributedTask.ObjectTemplating.Tokens
 
             return result;
         }
-
-        private sealed class DummyFunction : Function
-        {
-            protected override Object EvaluateCore(
-                EvaluationContext context,
-                out ResultMemory resultMemory)
-            {
-                resultMemory = null;
-                return null;
-            }
-        }
-
-        private static readonly Regex s_function = new Regex(@"^([a-zA-Z0-9_]+)\(([0-9]+),([0-9]+|MAX)\)$", RegexOptions.Compiled);
     }
 }

src/Sdk/DTObjectTemplating/ObjectTemplating/Tokens/TemplateTokenExtensions.cs

@@ -1,8 +1,5 @@
 using System;
 using System.Collections.Generic;
-using System.Linq;
-using GitHub.DistributedTask.Expressions2;
-using GitHub.DistributedTask.Expressions2.Sdk;
 
 namespace GitHub.DistributedTask.ObjectTemplating.Tokens
 {
@@ -109,43 +106,6 @@ namespace GitHub.DistributedTask.ObjectTemplating.Tokens
             throw new ArgumentException($"Error while reading '{objectDescription}'. Unexpected value '{literal.ToString()}'");
         }
 
-        /// <summary>
-        /// Traverses the token and checks whether all required expression values
-        /// and functions are provided.
-        /// </summary>
-        public static bool CheckHasRequiredContext(
-            this TemplateToken token,
-            IReadOnlyObject expressionValues,
-            IList<IFunctionInfo> expressionFunctions)
-        {
-            var expressionTokens = token.Traverse()
-                .OfType<BasicExpressionToken>()
-                .ToArray();
-            var parser = new ExpressionParser();
-            foreach (var expressionToken in expressionTokens)
-            {
-                var tree = parser.ValidateSyntax(expressionToken.Expression, null);
-                foreach (var node in tree.Traverse())
-                {
-                    if (node is NamedValue namedValue)
-                    {
-                        if (expressionValues?.Keys.Any(x => string.Equals(x, namedValue.Name, StringComparison.OrdinalIgnoreCase)) != true)
-                        {
-                            return false;
-                        }
-                    }
-                    else if (node is Function function &&
-                        !ExpressionConstants.WellKnownFunctions.ContainsKey(function.Name) &&
-                        expressionFunctions?.Any(x => string.Equals(x.Name, function.Name, StringComparison.OrdinalIgnoreCase)) != true)
-                    {
-                        return false;
-                    }
-                }
-            }
-
-            return true;
-        }
-
         /// <summary>
         /// Returns all tokens (depth first)
         /// </summary>

src/Sdk/DTPipelines/Pipelines/AgentJobRequestMessage.cs

@@ -40,9 +40,7 @@ namespace GitHub.DistributedTask.Pipelines
             WorkspaceOptions workspaceOptions,
             IEnumerable<JobStep> steps,
             IEnumerable<ContextScope> scopes,
-            IList<String> fileTable,
+            IList<String> fileTable)
-            TemplateToken jobOutputs,
-            IList<TemplateToken> defaults)
         {
             this.MessageType = JobRequestMessageTypes.PipelineAgentJobRequest;
             this.Plan = plan;
@@ -54,7 +52,6 @@ namespace GitHub.DistributedTask.Pipelines
             this.Timeline = timeline;
             this.Resources = jobResources;
             this.Workspace = workspaceOptions;
-            this.JobOutputs = jobOutputs;
 
             m_variables = new Dictionary<String, VariableValue>(variables, StringComparer.OrdinalIgnoreCase);
             m_maskHints = new List<MaskHint>(maskHints);
@@ -70,11 +67,6 @@ namespace GitHub.DistributedTask.Pipelines
                 m_environmentVariables = new List<TemplateToken>(environmentVariables);
             }
 
-            if (defaults?.Count > 0)
-            {
-                m_defaults = new List<TemplateToken>(defaults);
-            }
-
             this.ContextData = new Dictionary<String, PipelineContextData>(StringComparer.OrdinalIgnoreCase);
             if (contextData?.Count > 0)
             {
@@ -146,13 +138,6 @@ namespace GitHub.DistributedTask.Pipelines
             private set;
         }
 
-        [DataMember(EmitDefaultValue = false)]
-        public TemplateToken JobOutputs
-        {
-            get;
-            private set;
-        }
-
         [DataMember]
         public Int64 RequestId
         {
@@ -219,21 +204,6 @@ namespace GitHub.DistributedTask.Pipelines
             }
         }
 
-        /// <summary>
-        /// Gets the hierarchy of defaults to overlay, last wins.
-        /// </summary>
-        public IList<TemplateToken> Defaults
-        {
-            get
-            {
-                if (m_defaults == null)
-                {
-                    m_defaults = new List<TemplateToken>();
-                }
-                return m_defaults;
-            }
-        }
-
         /// <summary>
         /// Gets the collection of variables associated with the current context.
         /// </summary>
@@ -273,9 +243,6 @@ namespace GitHub.DistributedTask.Pipelines
             }
         }
 
-        /// <summary>
-        /// Gets the table of files used when parsing the pipeline (e.g. yaml files)
-        /// </summary>
         public IList<String> FileTable
         {
             get
@@ -396,11 +363,6 @@ namespace GitHub.DistributedTask.Pipelines
                 m_environmentVariables = null;
             }
 
-            if (m_defaults?.Count == 0)
-            {
-                m_defaults = null;
-            }
-
             if (m_fileTable?.Count == 0)
             {
                 m_fileTable = null;
@@ -435,9 +397,6 @@ namespace GitHub.DistributedTask.Pipelines
         [DataMember(Name = "EnvironmentVariables", EmitDefaultValue = false)]
         private List<TemplateToken> m_environmentVariables;
 
-        [DataMember(Name = "Defaults", EmitDefaultValue = false)]
-        private List<TemplateToken> m_defaults;
-
         [DataMember(Name = "FileTable", EmitDefaultValue = false)]
         private List<String> m_fileTable;
 

src/Sdk/DTPipelines/Pipelines/ObjectTemplating/PipelineTemplateConstants.cs

@@ -14,7 +14,6 @@ namespace GitHub.DistributedTask.Pipelines.ObjectTemplating
         public const String Clean = "clean";
         public const String Container = "container";
         public const String ContinueOnError = "continue-on-error";
-        public const String Defaults = "defaults";
         public const String Env = "env";
         public const String Event = "event";
         public const String EventPattern = "github.event";
@@ -24,18 +23,13 @@ namespace GitHub.DistributedTask.Pipelines.ObjectTemplating
         public const String FetchDepth = "fetch-depth";
         public const String GeneratedId = "generated-id";
         public const String GitHub = "github";
-        public const String HashFiles = "hashFiles";
         public const String Id = "id";
         public const String If = "if";
         public const String Image = "image";
         public const String Include = "include";
         public const String Inputs = "inputs";
         public const String Job = "job";
-        public const String JobDefaultsRun = "job-defaults-run";
-        public const String JobIfResult = "job-if-result";
-        public const String JobOutputs = "job-outputs";
         public const String Jobs = "jobs";
-        public const String Labels = "labels";
         public const String Lfs = "lfs";
         public const String Matrix = "matrix";
         public const String MaxParallel = "max-parallel";
@@ -62,7 +56,6 @@ namespace GitHub.DistributedTask.Pipelines.ObjectTemplating
         public const String Shell = "shell";
         public const String Skipped = "skipped";
         public const String StepEnv = "step-env";
-        public const String StepIfResult = "step-if-result";
         public const String Steps = "steps";
         public const String StepsScopeInputs = "steps-scope-inputs";
         public const String StepsScopeOutputs = "steps-scope-outputs";

src/Sdk/DTPipelines/Pipelines/ObjectTemplating/PipelineTemplateConverter.cs

@@ -16,20 +16,6 @@ namespace GitHub.DistributedTask.Pipelines.ObjectTemplating
 {
     internal static class PipelineTemplateConverter
     {
-        internal static Boolean ConvertToIfResult(
-            TemplateContext context,
-            TemplateToken ifResult)
-        {
-            var expression = ifResult.Traverse().FirstOrDefault(x => x is ExpressionToken);
-            if (expression != null)
-            {
-                throw new ArgumentException($"Unexpected type '{expression.GetType().Name}' encountered while reading 'if'.");
-            }
-
-            var evaluationResult = EvaluationResult.CreateIntermediateResult(null, ifResult);
-            return evaluationResult.IsTruthy;
-        }
-
         internal static Boolean? ConvertToStepContinueOnError(
             TemplateContext context,
             TemplateToken token,

src/Sdk/DTPipelines/Pipelines/ObjectTemplating/PipelineTemplateEvaluator.cs

@@ -4,7 +4,7 @@ using System.ComponentModel;
 using System.Linq;
 using System.Threading;
 using GitHub.DistributedTask.Expressions2;
-using GitHub.DistributedTask.Expressions2.Sdk.Functions;
+using GitHub.DistributedTask.Expressions2.Sdk;
 using GitHub.DistributedTask.ObjectTemplating;
 using GitHub.DistributedTask.ObjectTemplating.Schema;
 using GitHub.DistributedTask.ObjectTemplating.Tokens;
@@ -14,9 +14,6 @@ using ITraceWriter = GitHub.DistributedTask.ObjectTemplating.ITraceWriter;
 
 namespace GitHub.DistributedTask.Pipelines.ObjectTemplating
 {
-    /// <summary>
-    /// Evaluates parts of the workflow DOM. For example, a job strategy or step inputs.
-    /// </summary>
     [EditorBrowsable(EditorBrowsableState.Never)]
     public class PipelineTemplateEvaluator
     {
@@ -53,14 +50,13 @@ namespace GitHub.DistributedTask.Pipelines.ObjectTemplating
 
         public DictionaryContextData EvaluateStepScopeInputs(
             TemplateToken token,
-            DictionaryContextData contextData,
+            DictionaryContextData contextData)
-            IList<IFunctionInfo> expressionFunctions)
         {
             var result = default(DictionaryContextData);
 
             if (token != null && token.Type != TokenType.Null)
             {
-                var context = CreateContext(contextData, expressionFunctions);
+                var context = CreateContext(contextData);
                 try
                 {
                     token = TemplateEvaluator.Evaluate(context, PipelineTemplateConstants.StepsScopeInputs, token, 0, null, omitHeader: true);
@@ -80,14 +76,13 @@ namespace GitHub.DistributedTask.Pipelines.ObjectTemplating
 
         public DictionaryContextData EvaluateStepScopeOutputs(
             TemplateToken token,
-            DictionaryContextData contextData,
+            DictionaryContextData contextData)
-            IList<IFunctionInfo> expressionFunctions)
         {
             var result = default(DictionaryContextData);
 
             if (token != null && token.Type != TokenType.Null)
             {
-                var context = CreateContext(contextData, expressionFunctions);
+                var context = CreateContext(contextData);
                 try
                 {
                     token = TemplateEvaluator.Evaluate(context, PipelineTemplateConstants.StepsScopeOutputs, token, 0, null, omitHeader: true);
@@ -107,14 +102,13 @@ namespace GitHub.DistributedTask.Pipelines.ObjectTemplating
 
         public Boolean EvaluateStepContinueOnError(
             TemplateToken token,
-            DictionaryContextData contextData,
+            DictionaryContextData contextData)
-            IList<IFunctionInfo> expressionFunctions)
         {
             var result = default(Boolean?);
 
             if (token != null && token.Type != TokenType.Null)
             {
-                var context = CreateContext(contextData, expressionFunctions);
+                var context = CreateContext(contextData);
                 try
                 {
                     token = TemplateEvaluator.Evaluate(context, PipelineTemplateConstants.BooleanStepsContext, token, 0, null, omitHeader: true);
@@ -132,44 +126,16 @@ namespace GitHub.DistributedTask.Pipelines.ObjectTemplating
             return result ?? false;
         }
 
-        public String EvaluateStepDisplayName(
-            TemplateToken token,
-            DictionaryContextData contextData,
-            IList<IFunctionInfo> expressionFunctions)
-        {
-            var result = default(String);
-
-            if (token != null && token.Type != TokenType.Null)
-            {
-                var context = CreateContext(contextData, expressionFunctions);
-                try
-                {
-                    token = TemplateEvaluator.Evaluate(context, PipelineTemplateConstants.StringStepsContext, token, 0, null, omitHeader: true);
-                    context.Errors.Check();
-                    result = PipelineTemplateConverter.ConvertToStepDisplayName(context, token);
-                }
-                catch (Exception ex) when (!(ex is TemplateValidationException))
-                {
-                    context.Errors.Add(ex);
-                }
-
-                context.Errors.Check();
-            }
-
-            return result;
-        }
-
         public Dictionary<String, String> EvaluateStepEnvironment(
             TemplateToken token,
             DictionaryContextData contextData,
-            IList<IFunctionInfo> expressionFunctions,
             StringComparer keyComparer)
         {
             var result = default(Dictionary<String, String>);
 
             if (token != null && token.Type != TokenType.Null)
             {
-                var context = CreateContext(contextData, expressionFunctions);
+                var context = CreateContext(contextData);
                 try
                 {
                     token = TemplateEvaluator.Evaluate(context, PipelineTemplateConstants.StepEnv, token, 0, null, omitHeader: true);
@@ -187,44 +153,15 @@ namespace GitHub.DistributedTask.Pipelines.ObjectTemplating
             return result ?? new Dictionary<String, String>(keyComparer);
         }
 
-        public Boolean EvaluateStepIf(
-            TemplateToken token,
-            DictionaryContextData contextData,
-            IList<IFunctionInfo> expressionFunctions,
-            IEnumerable<KeyValuePair<String, Object>> expressionState)
-        {
-            var result = default(Boolean?);
-
-            if (token != null && token.Type != TokenType.Null)
-            {
-                var context = CreateContext(contextData, expressionFunctions, expressionState);
-                try
-                {
-                    token = TemplateEvaluator.Evaluate(context, PipelineTemplateConstants.StepIfResult, token, 0, null, omitHeader: true);
-                    context.Errors.Check();
-                    result = PipelineTemplateConverter.ConvertToIfResult(context, token);
-                }
-                catch (Exception ex) when (!(ex is TemplateValidationException))
-                {
-                    context.Errors.Add(ex);
-                }
-
-                context.Errors.Check();
-            }
-
-            return result ?? throw new InvalidOperationException("Step if cannot be null");
-        }
-
         public Dictionary<String, String> EvaluateStepInputs(
             TemplateToken token,
-            DictionaryContextData contextData,
+            DictionaryContextData contextData)
-            IList<IFunctionInfo> expressionFunctions)
         {
             var result = default(Dictionary<String, String>);
 
             if (token != null && token.Type != TokenType.Null)
             {
-                var context = CreateContext(contextData, expressionFunctions);
+                var context = CreateContext(contextData);
                 try
                 {
                     token = TemplateEvaluator.Evaluate(context, PipelineTemplateConstants.StepWith, token, 0, null, omitHeader: true);
@@ -244,14 +181,13 @@ namespace GitHub.DistributedTask.Pipelines.ObjectTemplating
 
         public Int32 EvaluateStepTimeout(
             TemplateToken token,
-            DictionaryContextData contextData,
+            DictionaryContextData contextData)
-            IList<IFunctionInfo> expressionFunctions)
         {
             var result = default(Int32?);
 
             if (token != null && token.Type != TokenType.Null)
             {
-                var context = CreateContext(contextData, expressionFunctions);
+                var context = CreateContext(contextData);
                 try
                 {
                     token = TemplateEvaluator.Evaluate(context, PipelineTemplateConstants.NumberStepsContext, token, 0, null, omitHeader: true);
@@ -271,14 +207,13 @@ namespace GitHub.DistributedTask.Pipelines.ObjectTemplating
 
         public JobContainer EvaluateJobContainer(
             TemplateToken token,
-            DictionaryContextData contextData,
+            DictionaryContextData contextData)
-            IList<IFunctionInfo> expressionFunctions)
         {
             var result = default(JobContainer);
 
             if (token != null && token.Type != TokenType.Null)
             {
-                var context = CreateContext(contextData, expressionFunctions);
+                var context = CreateContext(contextData);
                 try
                 {
                     token = TemplateEvaluator.Evaluate(context, PipelineTemplateConstants.Container, token, 0, null, omitHeader: true);
@@ -296,90 +231,15 @@ namespace GitHub.DistributedTask.Pipelines.ObjectTemplating
             return result;
         }
 
-        public Dictionary<String, String> EvaluateJobOutput(
-            TemplateToken token,
-            DictionaryContextData contextData,
-            IList<IFunctionInfo> expressionFunctions)
-        {
-            var result = default(Dictionary<String, String>);
-
-            if (token != null && token.Type != TokenType.Null)
-            {
-                var context = CreateContext(contextData, expressionFunctions);
-                try
-                {
-                    token = TemplateEvaluator.Evaluate(context, PipelineTemplateConstants.JobOutputs, token, 0, null, omitHeader: true);
-                    context.Errors.Check();
-                    result = new Dictionary<String, String>(StringComparer.OrdinalIgnoreCase);
-                    var mapping = token.AssertMapping("outputs");
-                    foreach (var pair in mapping)
-                    {
-                        // Literal key
-                        var key = pair.Key.AssertString("output key");
-
-                        // Literal value
-                        var value = pair.Value.AssertString("output value");
-                        result[key.Value] = value.Value;
-                    }
-                }
-                catch (Exception ex) when (!(ex is TemplateValidationException))
-                {
-                    context.Errors.Add(ex);
-                }
-
-                context.Errors.Check();
-            }
-
-            return result;
-        }
-
-        public Dictionary<String, String> EvaluateJobDefaultsRun(
-            TemplateToken token,
-            DictionaryContextData contextData,
-            IList<IFunctionInfo> expressionFunctions)
-        {
-            var result = default(Dictionary<String, String>);
-
-            if (token != null && token.Type != TokenType.Null)
-            {
-                var context = CreateContext(contextData, expressionFunctions);
-                try
-                {
-                    token = TemplateEvaluator.Evaluate(context, PipelineTemplateConstants.JobDefaultsRun, token, 0, null, omitHeader: true);
-                    context.Errors.Check();
-                    result = new Dictionary<String, String>(StringComparer.OrdinalIgnoreCase);
-                    var mapping = token.AssertMapping("defaults run");
-                    foreach (var pair in mapping)
-                    {
-                        // Literal key
-                        var key = pair.Key.AssertString("defaults run key");
-
-                        // Literal value
-                        var value = pair.Value.AssertString("defaults run value");
-                        result[key.Value] = value.Value;
-                    }
-                }
-                catch (Exception ex) when (!(ex is TemplateValidationException))
-                {
-                    context.Errors.Add(ex);
-                }
-
-                context.Errors.Check();
-            }
-
-            return result;
-        }
-
         public IList<KeyValuePair<String, JobContainer>> EvaluateJobServiceContainers(
             TemplateToken token,
-            DictionaryContextData contextData,
+            DictionaryContextData contextData)
-            IList<IFunctionInfo> expressionFunctions)
         {
             var result = default(List<KeyValuePair<String, JobContainer>>);
 
             if (token != null && token.Type != TokenType.Null)
             {
-                var context = CreateContext(contextData, expressionFunctions);
+                var context = CreateContext(contextData);
                 try
                 {
                     token = TemplateEvaluator.Evaluate(context, PipelineTemplateConstants.Services, token, 0, null, omitHeader: true);
@@ -397,10 +257,62 @@ namespace GitHub.DistributedTask.Pipelines.ObjectTemplating
             return result;
         }
 
-        private TemplateContext CreateContext(
+        public Boolean TryEvaluateStepDisplayName(
+            TemplateToken token,
             DictionaryContextData contextData,
-            IList<IFunctionInfo> expressionFunctions,
+            out String stepName)
-            IEnumerable<KeyValuePair<String, Object>> expressionState = null)
+        {
+            stepName = default(String);
+            var context = CreateContext(contextData);
+
+            if (token != null && token.Type != TokenType.Null)
+            {
+                // We should only evaluate basic expressions if we are sure we have context on all the Named Values and functions
+                // Otherwise return and use a default name
+                if (token is BasicExpressionToken expressionToken)
+                {
+                    ExpressionNode root = null;
+                    try
+                    {
+                        root = new ExpressionParser().ValidateSyntax(expressionToken.Expression, null) as ExpressionNode;
+                    }
+                    catch (Exception exception)
+                    {
+                        context.Errors.Add(exception);
+                        context.Errors.Check();
+                    }
+                    foreach (var node in root.Traverse())
+                    {
+                        if (node is NamedValue namedValue && !contextData.ContainsKey(namedValue.Name))
+                        {
+                            return false;
+                        }
+                        else if (node is Function function &&
+                            !context.ExpressionFunctions.Any(item => String.Equals(item.Name, function.Name)) &&
+                            !ExpressionConstants.WellKnownFunctions.ContainsKey(function.Name))
+                        {
+                            return false;
+                        }
+                    }
+                }
+
+                try
+                {
+                    token = TemplateEvaluator.Evaluate(context, PipelineTemplateConstants.StringStepsContext, token, 0, null, omitHeader: true);
+                    context.Errors.Check();
+                    stepName = PipelineTemplateConverter.ConvertToStepDisplayName(context, token);
+                }
+                catch (Exception ex) when (!(ex is TemplateValidationException))
+                {
+                    context.Errors.Add(ex);
+                }
+
+                context.Errors.Check();
+            }
+            return true;
+        }
+
+        private TemplateContext CreateContext(DictionaryContextData contextData)
         {
             var result = new TemplateContext
             {
@@ -423,7 +335,7 @@ namespace GitHub.DistributedTask.Pipelines.ObjectTemplating
                 }
             }
 
-            // Add named values
+            // Add named context
             if (contextData != null)
             {
                 foreach (var pair in contextData)
@@ -432,46 +344,14 @@ namespace GitHub.DistributedTask.Pipelines.ObjectTemplating
                 }
             }
 
-            // Add functions
+            // Compat for new agent against old server
-            var functionNames = new HashSet<String>(StringComparer.OrdinalIgnoreCase);
+            foreach (var name in s_contextNames)
-            if (expressionFunctions?.Count > 0)
-            {
-                foreach (var function in expressionFunctions)
-                {
-                    result.ExpressionFunctions.Add(function);
-                    functionNames.Add(function.Name);
-                }
-            }
-
-            // Add missing expression values and expression functions.
-            // This solves the following problems:
-            //   - Compat for new agent against old server (new contexts not sent down in job message)
-            //   - Evaluating early when all referenced contexts are available, even though all allowed
-            //     contexts may not yet be available. For example, evaluating step display name can often
-            //     be performed early.
-            foreach (var name in s_expressionValueNames)
             {
                 if (!result.ExpressionValues.ContainsKey(name))
                 {
                     result.ExpressionValues[name] = null;
                 }
             }
-            foreach (var name in s_expressionFunctionNames)
-            {
-                if (!functionNames.Contains(name))
-                {
-                    result.ExpressionFunctions.Add(new FunctionInfo<NoOperation>(name, 0, Int32.MaxValue));
-                }
-            }
-
-            // Add state
-            if (expressionState != null)
-            {
-                foreach (var pair in expressionState)
-                {
-                    result.State[pair.Key] = pair.Value;
-                }
-            }
 
             return result;
         }
@@ -479,13 +359,11 @@ namespace GitHub.DistributedTask.Pipelines.ObjectTemplating
         private readonly ITraceWriter m_trace;
         private readonly TemplateSchema m_schema;
         private readonly IList<String> m_fileTable;
-        private readonly String[] s_expressionValueNames = new[]
+        private readonly String[] s_contextNames = new[]
         {
             PipelineTemplateConstants.GitHub,
-            PipelineTemplateConstants.Needs,
             PipelineTemplateConstants.Strategy,
             PipelineTemplateConstants.Matrix,
-            PipelineTemplateConstants.Needs,
             PipelineTemplateConstants.Secrets,
             PipelineTemplateConstants.Steps,
             PipelineTemplateConstants.Inputs,
@@ -493,13 +371,5 @@ namespace GitHub.DistributedTask.Pipelines.ObjectTemplating
             PipelineTemplateConstants.Runner,
             PipelineTemplateConstants.Env,
         };
-        private readonly String[] s_expressionFunctionNames = new[]
-        {
-            PipelineTemplateConstants.Always,
-            PipelineTemplateConstants.Cancelled,
-            PipelineTemplateConstants.Failure,
-            PipelineTemplateConstants.HashFiles,
-            PipelineTemplateConstants.Success,
-        };
     }
 }

src/Sdk/DTPipelines/Pipelines/ObjectTemplating/PipelineTemplateSchemaFactory.cs

@@ -2,35 +2,25 @@
 using System.ComponentModel;
 using System.IO;
 using System.Reflection;
-using System.Threading;
-using System.Threading.Tasks;
 using GitHub.DistributedTask.ObjectTemplating.Schema;
 
 namespace GitHub.DistributedTask.Pipelines.ObjectTemplating
 {
     [EditorBrowsable(EditorBrowsableState.Never)]
-    public static class PipelineTemplateSchemaFactory
+    public sealed class PipelineTemplateSchemaFactory
     {
-        public static TemplateSchema GetSchema()
+        public TemplateSchema CreateSchema()
         {
-            if (s_schema == null)
+            var assembly = Assembly.GetExecutingAssembly();
+            var json = default(String);
+            using (var stream = assembly.GetManifestResourceStream("GitHub.DistributedTask.Pipelines.ObjectTemplating.workflow-v1.0.json"))
+            using (var streamReader = new StreamReader(stream))
             {
-                var assembly = Assembly.GetExecutingAssembly();
+                json = streamReader.ReadToEnd();
-                var json = default(String);
-                using (var stream = assembly.GetManifestResourceStream("GitHub.DistributedTask.Pipelines.ObjectTemplating.workflow-v1.0.json"))
-                using (var streamReader = new StreamReader(stream))
-                {
-                    json = streamReader.ReadToEnd();
-                }
-
-                var objectReader = new JsonObjectReader(null, json);
-                var schema = TemplateSchema.Load(objectReader);
-                Interlocked.CompareExchange(ref s_schema, schema, null);
             }
 
-            return s_schema;
+            var objectReader = new JsonObjectReader(null, json);
+            return TemplateSchema.Load(objectReader);
         }
-
-        private static TemplateSchema s_schema;
     }
 }

src/Sdk/DTPipelines/workflow-v1.0.json

@@ -9,7 +9,6 @@
         "properties": {
           "on": "any",
           "name": "string",
-          "defaults": "workflow-defaults",
           "env": "workflow-env",
           "jobs": "jobs"
         }
@@ -38,7 +37,6 @@
     "steps-scope-input-value": {
       "context": [
         "github",
-        "needs",
         "strategy",
         "matrix",
         "secrets",
@@ -66,7 +64,6 @@
     "steps-scope-output-value": {
       "context": [
         "github",
-        "needs",
         "strategy",
         "matrix",
         "secrets",
@@ -91,7 +88,6 @@
       "description": "Default input values for a steps template",
       "context": [
         "github",
-        "needs",
         "strategy",
         "matrix"
       ],
@@ -114,7 +110,6 @@
       "description": "Output values for a steps template",
       "context": [
         "github",
-        "needs",
         "strategy",
         "matrix",
         "secrets",
@@ -126,23 +121,6 @@
       "string": {}
     },
 
-    "workflow-defaults": {
-      "mapping": {
-        "properties": {
-          "run": "workflow-defaults-run"
-        }
-      }
-    },
-
-    "workflow-defaults-run": {
-      "mapping": {
-        "properties": {
-          "shell": "non-empty-string",
-          "working-directory": "non-empty-string"
-        }
-      }
-    },
-
     "workflow-env": {
       "context": [
         "github",
@@ -165,21 +143,16 @@
       "mapping": {
         "properties": {
           "needs": "needs",
-          "if": "job-if",
+          "if": "string",
           "strategy": "strategy",
           "name": "string-strategy-context",
-          "runs-on": {
+          "runs-on": "runs-on",
-            "type": "runs-on",
-            "required": true
-          },
           "timeout-minutes": "number-strategy-context",
           "cancel-timeout-minutes": "number-strategy-context",
-          "continue-on-error": "boolean-strategy-context",
+          "continue-on-error": "boolean",
           "container": "container",
           "services": "services",
           "env": "job-env",
-          "outputs": "job-outputs",
-          "defaults": "job-defaults",
           "steps": "steps"
         }
       }
@@ -192,41 +165,9 @@
       ]
     },
 
-    "job-if": {
-      "context": [
-        "github",
-        "needs",
-        "always(0,0)",
-        "failure(0,MAX)",
-        "cancelled(0,0)",
-        "success(0,MAX)"
-      ],
-      "string": {}
-    },
-
-    "job-if-result": {
-      "context": [
-        "github",
-        "needs",
-        "always(0,0)",
-        "failure(0,MAX)",
-        "cancelled(0,0)",
-        "success(0,MAX)"
-      ],
-      "one-of": [
-        "null",
-        "boolean",
-        "number",
-        "string",
-        "sequence",
-        "mapping"
-      ]
-    },
-
     "strategy": {
       "context": [
-        "github",
+        "github"
-        "needs"
       ],
       "mapping": {
         "properties": {
@@ -291,24 +232,25 @@
     "runs-on": {
       "context": [
         "github",
-        "needs",
         "strategy",
         "matrix"
       ],
       "one-of": [
-        "non-empty-string",
+        "runs-on-string",
-        "sequence-of-non-empty-string",
         "runs-on-mapping"
       ]
     },
 
+    "runs-on-string": {
+      "string": {
+        "require-non-empty": true
+      }
+    },
+
     "runs-on-mapping": {
       "mapping": {
         "properties": {
-          "pool": {
+          "pool": "non-empty-string"
-            "type": "non-empty-string",
-            "required": true
-          }
         }
       }
     },
@@ -316,10 +258,9 @@
     "job-env": {
       "context": [
         "github",
-        "needs",
+        "secrets",
         "strategy",
-        "matrix",
+        "matrix"
-        "secrets"
       ],
       "mapping": {
         "loose-key-type": "non-empty-string",
@@ -327,37 +268,6 @@
       }
     },
 
-    "job-defaults": {
-      "mapping": {
-        "properties": {
-          "run": "job-defaults-run"
-        }
-      }
-    },
-
-    "job-defaults-run": {
-      "context": [
-        "github",
-        "strategy",
-        "matrix",
-        "needs",
-        "env"
-      ],
-      "mapping": {
-        "properties": {
-          "shell": "non-empty-string",
-          "working-directory": "non-empty-string"
-        }
-      }
-    },
-
-    "job-outputs": {
-      "mapping": {
-        "loose-key-type": "non-empty-string",
-        "loose-value-type": "string-runner-context"
-      }
-    },
-
     "steps": {
       "sequence": {
         "item-type": "steps-item"
@@ -391,12 +301,9 @@
         "properties": {
           "name": "string-steps-context",
           "id": "non-empty-string",
-          "if": "step-if",
+          "if": "string",
           "timeout-minutes": "number-steps-context",
-          "run": {
+          "run": "string-steps-context",
-            "type": "string-steps-context",
-            "required": true
-          },
           "continue-on-error": "boolean-steps-context",
           "env": "step-env",
           "working-directory": "string-steps-context",
@@ -410,12 +317,9 @@
         "properties": {
           "name": "string-steps-context-in-template",
           "id": "non-empty-string",
-          "if": "step-if-in-template",
+          "if": "string",
           "timeout-minutes": "number-steps-context-in-template",
-          "run": {
+          "run": "string-steps-context-in-template",
-            "type": "string-steps-context-in-template",
-            "required": true
-          },
           "continue-on-error": "boolean-steps-context-in-template",
           "env": "step-env-in-template",
           "working-directory": "string-steps-context-in-template",
@@ -429,13 +333,10 @@
         "properties": {
           "name": "string-steps-context",
           "id": "non-empty-string",
-          "if": "step-if",
+          "if": "string",
           "continue-on-error": "boolean-steps-context",
           "timeout-minutes": "number-steps-context",
-          "uses": {
+          "uses": "non-empty-string",
-            "type": "non-empty-string",
-            "required": true
-          },
           "with": "step-with",
           "env": "step-env"
         }
@@ -447,109 +348,16 @@
         "properties": {
           "name": "string-steps-context-in-template",
           "id": "non-empty-string",
-          "if": "step-if-in-template",
+          "if": "string",
           "continue-on-error": "boolean-steps-context-in-template",
           "timeout-minutes": "number-steps-context-in-template",
-          "uses": {
+          "uses": "non-empty-string",
-            "type": "non-empty-string",
-            "required": true
-          },
           "with": "step-with-in-template",
           "env": "step-env-in-template"
         }
       }
     },
 
-    "step-if": {
-      "context": [
-        "github",
-        "needs",
-        "strategy",
-        "matrix",
-        "steps",
-        "job",
-        "runner",
-        "env",
-        "always(0,0)",
-        "failure(0,0)",
-        "cancelled(0,0)",
-        "success(0,0)",
-        "hashFiles(1,255)"
-      ],
-      "string": {}
-    },
-
-    "step-if-in-template": {
-      "context": [
-        "github",
-        "needs",
-        "strategy",
-        "matrix",
-        "steps",
-        "inputs",
-        "job",
-        "runner",
-        "env",
-        "always(0,0)",
-        "failure(0,0)",
-        "cancelled(0,0)",
-        "success(0,0)",
-        "hashFiles(1,255)"
-      ],
-      "string": {}
-    },
-
-    "step-if-result": {
-      "context": [
-        "github",
-        "strategy",
-        "matrix",
-        "steps",
-        "job",
-        "runner",
-        "env",
-        "always(0,0)",
-        "failure(0,0)",
-        "cancelled(0,0)",
-        "success(0,0)",
-        "hashFiles(1,255)"
-      ],
-      "one-of": [
-        "null",
-        "boolean",
-        "number",
-        "string",
-        "sequence",
-        "mapping"
-      ]
-    },
-
-    "step-if-result-in-template": {
-      "context": [
-        "github",
-        "strategy",
-        "matrix",
-        "steps",
-        "inputs",
-        "job",
-        "runner",
-        "env",
-        "always(0,0)",
-        "failure(0,0)",
-        "cancelled(0,0)",
-        "success(0,0)",
-        "hashFiles(1,255)"
-      ],
-      "one-of": [
-        "null",
-        "boolean",
-        "number",
-        "string",
-        "sequence",
-        "mapping"
-      ]
-    },
-
     "steps-template-reference": {
       "mapping": {
         "properties": {
@@ -573,7 +381,6 @@
     "steps-template-reference-inputs": {
       "context": [
         "github",
-        "needs",
         "strategy",
         "matrix",
         "secrets",
@@ -591,7 +398,6 @@
     "steps-template-reference-inputs-in-template": {
       "context": [
         "github",
-        "needs",
         "strategy",
         "matrix",
         "secrets",
@@ -610,15 +416,13 @@
     "step-env": {
       "context": [
         "github",
-        "needs",
         "strategy",
         "matrix",
         "secrets",
         "steps",
         "job",
         "runner",
-        "env",
+        "env"
-        "hashFiles(1,255)"
       ],
       "mapping": {
         "loose-key-type": "non-empty-string",
@@ -629,7 +433,6 @@
     "step-env-in-template": {
       "context": [
         "github",
-        "needs",
         "strategy",
         "matrix",
         "secrets",
@@ -637,8 +440,7 @@
         "inputs",
         "job",
         "runner",
-        "env",
+        "env"
-        "hashFiles(1,255)"
       ],
       "mapping": {
         "loose-key-type": "non-empty-string",
@@ -649,35 +451,13 @@
     "step-with": {
       "context": [
         "github",
-        "needs",
         "strategy",
         "matrix",
         "secrets",
         "steps",
         "job",
         "runner",
-        "env",
+        "env"
-        "hashFiles(1,255)"
-      ],
-      "mapping": {
-        "loose-key-type": "non-empty-string",
-        "loose-value-type": "string"
-      }
-    },
-
-    "step-with-in-template": {
-      "context": [
-        "github",
-        "needs",
-        "strategy",
-        "matrix",
-        "secrets",
-        "steps",
-        "inputs",
-        "job",
-        "runner",
-        "env",
-        "hashFiles(1,255)"
       ],
       "mapping": {
         "loose-key-type": "non-empty-string",
@@ -688,7 +468,6 @@
     "container": {
       "context": [
         "github",
-        "needs",
         "strategy",
         "matrix"
       ],
@@ -713,7 +492,6 @@
     "services": {
       "context": [
         "github",
-        "needs",
         "strategy",
         "matrix"
       ],
@@ -726,7 +504,6 @@
     "services-container": {
       "context": [
         "github",
-        "needs",
         "strategy",
         "matrix"
       ],
@@ -743,6 +520,24 @@
       }
     },
 
+    "step-with-in-template": {
+      "context": [
+        "github",
+        "strategy",
+        "matrix",
+        "secrets",
+        "steps",
+        "inputs",
+        "job",
+        "runner",
+        "env"
+      ],
+      "mapping": {
+        "loose-key-type": "non-empty-string",
+        "loose-value-type": "string"
+      }
+    },
+
     "non-empty-string": {
       "string": {
         "require-non-empty": true
@@ -755,20 +550,9 @@
       }
     },
 
-    "boolean-strategy-context": {
-      "context": [
-        "github",
-        "needs",
-        "strategy",
-        "matrix"
-      ],
-      "boolean": {}
-    },
-
     "number-strategy-context": {
       "context": [
         "github",
-        "needs",
         "strategy",
         "matrix"
       ],
@@ -778,7 +562,6 @@
     "string-strategy-context": {
       "context": [
         "github",
-        "needs",
         "strategy",
         "matrix"
       ],
@@ -788,15 +571,13 @@
     "boolean-steps-context": {
       "context": [
         "github",
-        "needs",
         "strategy",
         "matrix",
         "secrets",
         "steps",
         "job",
         "runner",
-        "env",
+        "env"
-        "hashFiles(1,255)"
       ],
       "boolean": {}
     },
@@ -804,7 +585,6 @@
     "boolean-steps-context-in-template": {
       "context": [
         "github",
-        "needs",
         "strategy",
         "matrix",
         "secrets",
@@ -812,8 +592,7 @@
         "inputs",
         "job",
         "runner",
-        "env",
+        "env"
-        "hashFiles(1,255)"
       ],
       "boolean": {}
     },
@@ -821,15 +600,13 @@
     "number-steps-context": {
       "context": [
         "github",
-        "needs",
         "strategy",
         "matrix",
         "secrets",
         "steps",
         "job",
         "runner",
-        "env",
+        "env"
-        "hashFiles(1,255)"
       ],
       "number": {}
     },
@@ -837,7 +614,6 @@
     "number-steps-context-in-template": {
       "context": [
         "github",
-        "needs",
         "strategy",
         "matrix",
         "secrets",
@@ -845,16 +621,14 @@
         "inputs",
         "job",
         "runner",
-        "env",
+        "env"
-        "hashFiles(1,255)"
       ],
       "number": {}
     },
 
-    "string-runner-context": {
+    "string-steps-context": {
       "context": [
         "github",
-        "needs",
         "strategy",
         "matrix",
         "secrets",
@@ -866,26 +640,9 @@
       "string": {}
     },
 
-    "string-steps-context": {
-      "context": [
-        "github",
-        "needs",
-        "strategy",
-        "matrix",
-        "secrets",
-        "steps",
-        "job",
-        "runner",
-        "env",
-        "hashFiles(1,255)"
-      ],
-      "string": {}
-    },
-
     "string-steps-context-in-template": {
       "context": [
         "github",
-        "needs",
         "strategy",
         "matrix",
         "secrets",
@@ -893,8 +650,7 @@
         "inputs",
         "job",
         "runner",
-        "env",
+        "env"
-        "hashFiles(1,255)"
       ],
       "string": {}
     }

src/Sdk/DTWebApi/WebApi/JobEvent.cs

@@ -31,7 +31,7 @@ namespace GitHub.DistributedTask.WebApi
         }
 
         protected JobEvent(
-            String name,
+            String name, 
             Guid jobId)
         {
             this.Name = name;
@@ -123,12 +123,11 @@ namespace GitHub.DistributedTask.WebApi
             Int64 requestId,
             Guid jobId,
             TaskResult result,
-            Dictionary<String, VariableValue> outputs)
+            IDictionary<String, VariableValue> outputVariables)
             : base(JobEventTypes.JobCompleted, jobId)
         {
             this.RequestId = requestId;
             this.Result = result;
-            this.Outputs = outputs;
         }
 
         [DataMember(EmitDefaultValue = false)]
@@ -144,13 +143,6 @@ namespace GitHub.DistributedTask.WebApi
             get;
             set;
         }
-
-        [DataMember(EmitDefaultValue = false)]
-        public IDictionary<String, VariableValue> Outputs
-        {
-            get;
-            set;
-        }
     }
 
     [DataContract]
@@ -161,9 +153,9 @@ namespace GitHub.DistributedTask.WebApi
         }
 
         protected TaskEvent(
-            string name,
+            string name, 
-            Guid jobId,
+            Guid jobId, 
-            Guid taskId)
+            Guid taskId) 
             : base(name, jobId)
         {
             TaskId = taskId;
@@ -193,9 +185,9 @@ namespace GitHub.DistributedTask.WebApi
         }
 
         public override Object ReadJson(
-            JsonReader reader,
+            JsonReader reader, 
-            Type objectType,
+            Type objectType, 
-            Object existingValue,
+            Object existingValue, 
             JsonSerializer serializer)
         {
             var eventObject = JObject.Load(reader);

src/Sdk/DTWebApi/WebApi/TaskResourceIds.cs

@@ -260,8 +260,5 @@ namespace GitHub.DistributedTask.WebApi
         public static readonly Guid CheckpointResourcesLocationId = new Guid(CheckpointResourcesLocationIdString);
         public const String CheckpointResourcesResource = "references";
 
-        public static readonly Guid RunnerAuthUrl = new Guid("{A82A119C-1E46-44B6-8D75-C82A79CF975B}");
-        public const string RunnerAuthUrlResource = "authurl";
-
     }
 }

src/Test/L0/Listener/JobDispatcherL0.cs

@@ -33,7 +33,7 @@ namespace GitHub.Runner.Common.Tests.Listener
             TaskOrchestrationPlanReference plan = new TaskOrchestrationPlanReference();
             TimelineReference timeline = null;
             Guid jobId = Guid.NewGuid();
-            var result = new Pipelines.AgentJobRequestMessage(plan, timeline, jobId, "someJob", "someJob", null, null, null, new Dictionary<string, VariableValue>(), new List<MaskHint>(), new Pipelines.JobResources(), new Pipelines.ContextData.DictionaryContextData(), new Pipelines.WorkspaceOptions(), new List<Pipelines.ActionStep>(), null, null, null, null);
+            var result = new Pipelines.AgentJobRequestMessage(plan, timeline, jobId, "someJob", "someJob", null, null, null, new Dictionary<string, VariableValue>(), new List<MaskHint>(), new Pipelines.JobResources(), new Pipelines.ContextData.DictionaryContextData(), new Pipelines.WorkspaceOptions(), new List<Pipelines.ActionStep>(), null, null);
             result.ContextData["github"] = new Pipelines.ContextData.DictionaryContextData();
             return result;
         }

src/Test/L0/Listener/RunnerL0.cs

@@ -43,7 +43,7 @@ namespace GitHub.Runner.Common.Tests.Listener
             TaskOrchestrationPlanReference plan = new TaskOrchestrationPlanReference();
             TimelineReference timeline = null;
             Guid jobId = Guid.NewGuid();
-            return new Pipelines.AgentJobRequestMessage(plan, timeline, jobId, "test", "test", null, null, null, new Dictionary<string, VariableValue>(), new List<MaskHint>(), new Pipelines.JobResources(), new Pipelines.ContextData.DictionaryContextData(), new Pipelines.WorkspaceOptions(), new List<Pipelines.ActionStep>(), null, null, null, null);
+            return new Pipelines.AgentJobRequestMessage(plan, timeline, jobId, "test", "test", null, null, null, new Dictionary<string, VariableValue>(), new List<MaskHint>(), new Pipelines.JobResources(), new Pipelines.ContextData.DictionaryContextData(), new Pipelines.WorkspaceOptions(), new List<Pipelines.ActionStep>(), null, null);
         }
 
         private JobCancelMessage CreateJobCancelMessage()

src/Test/L0/ProcessInvokerL0.cs

@@ -8,7 +8,6 @@ using Xunit;
 using GitHub.Runner.Common.Util;
 using System.Threading.Channels;
 using GitHub.Runner.Sdk;
-using System.Linq;
 
 namespace GitHub.Runner.Common.Tests
 {
@@ -82,102 +81,6 @@ namespace GitHub.Runner.Common.Tests
             }
         }
 
-        [Fact]
-        [Trait("Level", "L0")]
-        [Trait("Category", "Common")]
-        public async Task SetCIEnv()
-        {
-            using (TestHostContext hc = new TestHostContext(this))
-            {
-                var existingCI = Environment.GetEnvironmentVariable("CI");
-                try
-                {
-                    // Clear out CI and make sure process invoker sets it.
-                    Environment.SetEnvironmentVariable("CI", null);
-
-                    Tracing trace = hc.GetTrace();
-
-                    Int32 exitCode = -1;
-                    var processInvoker = new ProcessInvokerWrapper();
-                    processInvoker.Initialize(hc);
-                    var stdout = new List<string>();
-                    var stderr = new List<string>();
-                    processInvoker.OutputDataReceived += (object sender, ProcessDataReceivedEventArgs e) =>
-                    {
-                        trace.Info(e.Data);
-                        stdout.Add(e.Data);
-                    };
-                    processInvoker.ErrorDataReceived += (object sender, ProcessDataReceivedEventArgs e) =>
-                    {
-                        trace.Info(e.Data);
-                        stderr.Add(e.Data);
-                    };
-#if OS_WINDOWS
-                    exitCode = await processInvoker.ExecuteAsync("", "cmd.exe", "/c \"echo %CI%\"", null, CancellationToken.None);
-#else
-                    exitCode = await processInvoker.ExecuteAsync("", "bash", "-c \"echo $CI\"", null, CancellationToken.None);
-#endif
-
-                    trace.Info("Exit Code: {0}", exitCode);
-                    Assert.Equal(0, exitCode);
-
-                    Assert.Equal("true", stdout.First(x => !string.IsNullOrWhiteSpace(x)));
-                }
-                finally
-                {
-                    Environment.SetEnvironmentVariable("CI", existingCI);
-                }
-            }
-        }
-
-        [Fact]
-        [Trait("Level", "L0")]
-        [Trait("Category", "Common")]
-        public async Task KeepExistingCIEnv()
-        {
-            using (TestHostContext hc = new TestHostContext(this))
-            {
-                var existingCI = Environment.GetEnvironmentVariable("CI");
-                try
-                {
-                    // Clear out CI and make sure process invoker sets it.
-                    Environment.SetEnvironmentVariable("CI", null);
-
-                    Tracing trace = hc.GetTrace();
-
-                    Int32 exitCode = -1;
-                    var processInvoker = new ProcessInvokerWrapper();
-                    processInvoker.Initialize(hc);
-                    var stdout = new List<string>();
-                    var stderr = new List<string>();
-                    processInvoker.OutputDataReceived += (object sender, ProcessDataReceivedEventArgs e) =>
-                    {
-                        trace.Info(e.Data);
-                        stdout.Add(e.Data);
-                    };
-                    processInvoker.ErrorDataReceived += (object sender, ProcessDataReceivedEventArgs e) =>
-                    {
-                        trace.Info(e.Data);
-                        stderr.Add(e.Data);
-                    };
-#if OS_WINDOWS
-                    exitCode = await processInvoker.ExecuteAsync("", "cmd.exe", "/c \"echo %CI%\"", new Dictionary<string, string>() { { "CI", "false" } }, CancellationToken.None);
-#else
-                    exitCode = await processInvoker.ExecuteAsync("", "bash", "-c \"echo $CI\"", new Dictionary<string, string>() { { "CI", "false" } }, CancellationToken.None);
-#endif
-
-                    trace.Info("Exit Code: {0}", exitCode);
-                    Assert.Equal(0, exitCode);
-
-                    Assert.Equal("false", stdout.First(x => !string.IsNullOrWhiteSpace(x)));
-                }
-                finally
-                {
-                    Environment.SetEnvironmentVariable("CI", existingCI);
-                }
-            }
-        }
-
 #if !OS_WINDOWS
         //Run a process that normally takes 20sec to finish and cancel it.
         [Fact]

src/Test/L0/TestHostContext.cs

@@ -198,7 +198,7 @@ namespace GitHub.Runner.Common.Tests
 
                 case WellKnownDirectory.Tools:
                     path = Environment.GetEnvironmentVariable("RUNNER_TOOL_CACHE");
-
+                    
                     if (string.IsNullOrEmpty(path))
                     {
                         path = Path.Combine(
@@ -279,13 +279,6 @@ namespace GitHub.Runner.Common.Tests
                         GetDirectory(WellKnownDirectory.Root),
                         ".options");
                     break;
-
-                case WellKnownConfigFile.SetupInfo:
-                    path = Path.Combine(
-                        GetDirectory(WellKnownDirectory.Root),
-                        ".setup_info");
-                    break;
-
                 default:
                     throw new NotSupportedException($"Unexpected well known config file: '{configFile}'");
             }

src/Test/L0/Worker/ActionCommandManagerL0.cs

@@ -150,7 +150,7 @@ namespace GitHub.Runner.Common.Tests.Worker
                 TimelineReference timeline = new TimelineReference();
                 Guid jobId = Guid.NewGuid();
                 string jobName = "some job name";
-                var jobRequest = new Pipelines.AgentJobRequestMessage(plan, timeline, jobId, jobName, jobName, null, null, null, new Dictionary<string, VariableValue>(), new List<MaskHint>(), new Pipelines.JobResources(), new Pipelines.ContextData.DictionaryContextData(), new Pipelines.WorkspaceOptions(), new List<Pipelines.ActionStep>(), null, null, null, null);
+                var jobRequest = new Pipelines.AgentJobRequestMessage(plan, timeline, jobId, jobName, jobName, null, null, null, new Dictionary<string, VariableValue>(), new List<MaskHint>(), new Pipelines.JobResources(), new Pipelines.ContextData.DictionaryContextData(), new Pipelines.WorkspaceOptions(), new List<Pipelines.ActionStep>(), null, null);
                 jobRequest.Resources.Repositories.Add(new Pipelines.RepositoryResource()
                 {
                     Alias = Pipelines.PipelineConstants.SelfAlias,

src/Test/L0/Worker/ActionManagerL0.cs

@@ -1,6 +1,4 @@
-using GitHub.DistributedTask.Expressions2;
+using GitHub.DistributedTask.ObjectTemplating.Tokens;
-using GitHub.DistributedTask.ObjectTemplating.Tokens;
-using GitHub.DistributedTask.Pipelines.ContextData;
 using GitHub.DistributedTask.WebApi;
 using GitHub.Runner.Common.Util;
 using GitHub.Runner.Worker;
@@ -1602,8 +1600,6 @@ runs:
             _ec = new Mock<IExecutionContext>();
             _ec.Setup(x => x.CancellationToken).Returns(_ecTokenSource.Token);
             _ec.Setup(x => x.Variables).Returns(new Variables(_hc, new Dictionary<string, VariableValue>()));
-            _ec.Setup(x => x.ExpressionValues).Returns(new DictionaryContextData());
-            _ec.Setup(x => x.ExpressionFunctions).Returns(new List<IFunctionInfo>());
             _ec.Setup(x => x.Write(It.IsAny<string>(), It.IsAny<string>())).Callback((string tag, string message) => { _hc.GetTrace().Info($"[{tag}]{message}"); });
             _ec.Setup(x => x.AddIssue(It.IsAny<Issue>(), It.IsAny<string>())).Callback((Issue issue, string message) => { _hc.GetTrace().Info($"[{issue.Type}]{issue.Message ?? message}"); });
             _ec.Setup(x => x.GetGitHubContext("workspace")).Returns(Path.Combine(_workFolder, "actions", "actions"));

src/Test/L0/Worker/ActionManifestManagerL0.cs

@@ -1,9 +1,7 @@
-using GitHub.DistributedTask.Expressions2;
 using GitHub.DistributedTask.ObjectTemplating.Tokens;
 using GitHub.DistributedTask.Pipelines.ContextData;
 using GitHub.DistributedTask.WebApi;
 using GitHub.Runner.Worker;
-using GitHub.Runner.Worker.Expressions;
 using Moq;
 using System;
 using System.Collections.Generic;
@@ -535,26 +533,26 @@ namespace GitHub.Runner.Common.Tests.Worker
                 var actionManifest = new ActionManifestManager();
                 actionManifest.Initialize(_hc);
 
-                _ec.Object.ExpressionValues["github"] = new DictionaryContextData
+                var githubContext = new DictionaryContextData();
-                {
+                githubContext.Add("ref", new StringContextData("refs/heads/master"));
-                    { "ref", new StringContextData("refs/heads/master") },
+
-                };
+                var evaluateContext = new Dictionary<string, PipelineContextData>(StringComparer.OrdinalIgnoreCase);
-                _ec.Object.ExpressionValues["strategy"] = new DictionaryContextData();
+                evaluateContext["github"] = githubContext;
-                _ec.Object.ExpressionValues["matrix"] = new DictionaryContextData();
+                evaluateContext["strategy"] = new DictionaryContextData();
-                _ec.Object.ExpressionValues["steps"] = new DictionaryContextData();
+                evaluateContext["matrix"] = new DictionaryContextData();
-                _ec.Object.ExpressionValues["job"] = new DictionaryContextData();
+                evaluateContext["steps"] = new DictionaryContextData();
-                _ec.Object.ExpressionValues["runner"] = new DictionaryContextData();
+                evaluateContext["job"] = new DictionaryContextData();
-                _ec.Object.ExpressionValues["env"] = new DictionaryContextData();
+                evaluateContext["runner"] = new DictionaryContextData();
-                _ec.Object.ExpressionFunctions.Add(new FunctionInfo<HashFilesFunction>("hashFiles", 1, 255));
+                evaluateContext["env"] = new DictionaryContextData();
 
                 //Act
-                var result = actionManifest.EvaluateDefaultInput(_ec.Object, "testInput", new StringToken(null, null, null, "defaultValue"));
+                var result = actionManifest.EvaluateDefaultInput(_ec.Object, "testInput", new StringToken(null, null, null, "defaultValue"), evaluateContext);
 
                 //Assert
                 Assert.Equal("defaultValue", result);
 
                 //Act
-                result = actionManifest.EvaluateDefaultInput(_ec.Object, "testInput", new BasicExpressionToken(null, null, null, "github.ref"));
+                result = actionManifest.EvaluateDefaultInput(_ec.Object, "testInput", new BasicExpressionToken(null, null, null, "github.ref"), evaluateContext);
 
                 //Assert
                 Assert.Equal("refs/heads/master", result);
@@ -577,8 +575,6 @@ namespace GitHub.Runner.Common.Tests.Worker
             _ec.Setup(x => x.WriteDebug).Returns(true);
             _ec.Setup(x => x.CancellationToken).Returns(_ecTokenSource.Token);
             _ec.Setup(x => x.Variables).Returns(new Variables(_hc, new Dictionary<string, VariableValue>()));
-            _ec.Setup(x => x.ExpressionValues).Returns(new DictionaryContextData());
-            _ec.Setup(x => x.ExpressionFunctions).Returns(new List<IFunctionInfo>());
             _ec.Setup(x => x.Write(It.IsAny<string>(), It.IsAny<string>())).Callback((string tag, string message) => { _hc.GetTrace().Info($"{tag}{message}"); });
             _ec.Setup(x => x.AddIssue(It.IsAny<Issue>(), It.IsAny<string>())).Callback((Issue issue, string message) => { _hc.GetTrace().Info($"[{issue.Type}]{issue.Message ?? message}"); });
         }

src/Test/L0/Worker/ActionRunnerL0.cs

@@ -1,5 +1,4 @@
-using GitHub.DistributedTask.Expressions2;
+using GitHub.DistributedTask.ObjectTemplating.Tokens;
-using GitHub.DistributedTask.ObjectTemplating.Tokens;
 using GitHub.DistributedTask.Pipelines;
 using GitHub.DistributedTask.Pipelines.ContextData;
 using GitHub.DistributedTask.WebApi;
@@ -323,7 +322,6 @@ namespace GitHub.Runner.Common.Tests.Worker
 
             _ec = new Mock<IExecutionContext>();
             _ec.Setup(x => x.ExpressionValues).Returns(_context);
-            _ec.Setup(x => x.ExpressionFunctions).Returns(new List<IFunctionInfo>());
             _ec.Setup(x => x.IntraActionState).Returns(new Dictionary<string, string>());
             _ec.Setup(x => x.EnvironmentVariables).Returns(new Dictionary<string, string>());
             _ec.Setup(x => x.SetGitHubContext(It.IsAny<string>(), It.IsAny<string>()));

src/Test/L0/Worker/ExecutionContextL0.cs

@@ -25,7 +25,7 @@ namespace GitHub.Runner.Common.Tests.Worker
                 TimelineReference timeline = new TimelineReference();
                 Guid jobId = Guid.NewGuid();
                 string jobName = "some job name";
-                var jobRequest = new Pipelines.AgentJobRequestMessage(plan, timeline, jobId, jobName, jobName, null, null, null, new Dictionary<string, VariableValue>(), new List<MaskHint>(), new Pipelines.JobResources(), new Pipelines.ContextData.DictionaryContextData(), new Pipelines.WorkspaceOptions(), new List<Pipelines.ActionStep>(), null, null, null, null);
+                var jobRequest = new Pipelines.AgentJobRequestMessage(plan, timeline, jobId, jobName, jobName, null, null, null, new Dictionary<string, VariableValue>(), new List<MaskHint>(), new Pipelines.JobResources(), new Pipelines.ContextData.DictionaryContextData(), new Pipelines.WorkspaceOptions(), new List<Pipelines.ActionStep>(), null, null);
                 jobRequest.Resources.Repositories.Add(new Pipelines.RepositoryResource()
                 {
                     Alias = Pipelines.PipelineConstants.SelfAlias,
@@ -101,7 +101,7 @@ namespace GitHub.Runner.Common.Tests.Worker
                 TimelineReference timeline = new TimelineReference();
                 Guid jobId = Guid.NewGuid();
                 string jobName = "some job name";
-                var jobRequest = new Pipelines.AgentJobRequestMessage(plan, timeline, jobId, jobName, jobName, null, null, null, new Dictionary<string, VariableValue>(), new List<MaskHint>(), new Pipelines.JobResources(), new Pipelines.ContextData.DictionaryContextData(), new Pipelines.WorkspaceOptions(), new List<Pipelines.ActionStep>(), null, null, null, null);
+                var jobRequest = new Pipelines.AgentJobRequestMessage(plan, timeline, jobId, jobName, jobName, null, null, null, new Dictionary<string, VariableValue>(), new List<MaskHint>(), new Pipelines.JobResources(), new Pipelines.ContextData.DictionaryContextData(), new Pipelines.WorkspaceOptions(), new List<Pipelines.ActionStep>(), null, null);
                 jobRequest.Resources.Repositories.Add(new Pipelines.RepositoryResource()
                 {
                     Alias = Pipelines.PipelineConstants.SelfAlias,
@@ -152,7 +152,7 @@ namespace GitHub.Runner.Common.Tests.Worker
                 TimelineReference timeline = new TimelineReference();
                 Guid jobId = Guid.NewGuid();
                 string jobName = "some job name";
-                var jobRequest = new Pipelines.AgentJobRequestMessage(plan, timeline, jobId, jobName, jobName, null, null, null, new Dictionary<string, VariableValue>(), new List<MaskHint>(), new Pipelines.JobResources(), new Pipelines.ContextData.DictionaryContextData(), new Pipelines.WorkspaceOptions(), new List<Pipelines.ActionStep>(), null, null, null, null);
+                var jobRequest = new Pipelines.AgentJobRequestMessage(plan, timeline, jobId, jobName, jobName, null, null, null, new Dictionary<string, VariableValue>(), new List<MaskHint>(), new Pipelines.JobResources(), new Pipelines.ContextData.DictionaryContextData(), new Pipelines.WorkspaceOptions(), new List<Pipelines.ActionStep>(), null, null);
                 jobRequest.Resources.Repositories.Add(new Pipelines.RepositoryResource()
                 {
                     Alias = Pipelines.PipelineConstants.SelfAlias,

src/Test/L0/Worker/ExpressionManagerL0.cs

@@ -1,20 +1,20 @@
 using System;
 using System.Collections.Generic;
 using System.Runtime.CompilerServices;
-using GitHub.DistributedTask.Expressions2;
-using GitHub.DistributedTask.ObjectTemplating;
-using GitHub.DistributedTask.Pipelines.ObjectTemplating;
 using GitHub.DistributedTask.WebApi;
 using GitHub.Runner.Worker;
-using GitHub.Runner.Worker.Expressions;
 using Moq;
 using Xunit;
+using GitHub.DistributedTask.Expressions2;
+using GitHub.DistributedTask.Pipelines.ContextData;
 
-namespace GitHub.Runner.Common.Tests.Worker.Expressions
+namespace GitHub.Runner.Common.Tests.Worker
 {
-    public sealed class ConditionFunctionsL0
+    public sealed class ExpressionManagerL0
     {
-        private TemplateContext _templateContext;
+        private Mock<IExecutionContext> _ec;
+        private ExpressionManager _expressionManager;
+        private DictionaryContextData _expressions;
         private JobContext _jobContext;
 
         [Fact]
@@ -38,7 +38,7 @@ namespace GitHub.Runner.Common.Tests.Worker.Expressions
                     _jobContext.Status = variableSet.JobStatus;
 
                     // Act.
-                    bool actual = Evaluate("always()");
+                    bool actual = _expressionManager.Evaluate(_ec.Object, "always()").Value;
 
                     // Assert.
                     Assert.Equal(variableSet.Expected, actual);
@@ -68,7 +68,7 @@ namespace GitHub.Runner.Common.Tests.Worker.Expressions
                     _jobContext.Status = variableSet.JobStatus;
 
                     // Act.
-                    bool actual = Evaluate("cancelled()");
+                    bool actual = _expressionManager.Evaluate(_ec.Object, "cancelled()").Value;
 
                     // Assert.
                     Assert.Equal(variableSet.Expected, actual);
@@ -97,7 +97,7 @@ namespace GitHub.Runner.Common.Tests.Worker.Expressions
                     _jobContext.Status = variableSet.JobStatus;
 
                     // Act.
-                    bool actual = Evaluate("failure()");
+                    bool actual = _expressionManager.Evaluate(_ec.Object, "failure()").Value;
 
                     // Assert.
                     Assert.Equal(variableSet.Expected, actual);
@@ -126,7 +126,37 @@ namespace GitHub.Runner.Common.Tests.Worker.Expressions
                     _jobContext.Status = variableSet.JobStatus;
 
                     // Act.
-                    bool actual = Evaluate("success()");
+                    bool actual = _expressionManager.Evaluate(_ec.Object, "success()").Value;
+
+                    // Assert.
+                    Assert.Equal(variableSet.Expected, actual);
+                }
+            }
+        }
+
+        [Fact]
+        [Trait("Level", "L0")]
+        [Trait("Category", "Worker")]
+        public void ContextNamedValue()
+        {
+            using (TestHostContext hc = CreateTestContext())
+            {
+                // Arrange.
+                var variableSets = new[]
+                {
+                    new { Condition = "github.ref == 'refs/heads/master'", VariableName = "ref", VariableValue = "refs/heads/master", Expected = true },
+                    new { Condition = "github['ref'] == 'refs/heads/master'", VariableName = "ref", VariableValue = "refs/heads/master", Expected = true },
+                    new { Condition = "github.nosuch || '' == ''", VariableName = "ref", VariableValue = "refs/heads/master", Expected = true },
+                    new { Condition = "github['ref'] == 'refs/heads/release'", VariableName = "ref", VariableValue = "refs/heads/master", Expected = false },
+                    new { Condition = "github.ref == 'refs/heads/release'", VariableName = "ref", VariableValue = "refs/heads/master", Expected = false },
+                };
+                foreach (var variableSet in variableSets)
+                {
+                    InitializeExecutionContext(hc);
+                    _ec.Object.ExpressionValues["github"] = new GitHubContext() { { variableSet.VariableName, new StringContextData(variableSet.VariableValue) } };
+
+                    // Act.
+                    bool actual = _expressionManager.Evaluate(_ec.Object, variableSet.Condition).Value;
 
                     // Assert.
                     Assert.Equal(variableSet.Expected, actual);
@@ -136,34 +166,21 @@ namespace GitHub.Runner.Common.Tests.Worker.Expressions
 
         private TestHostContext CreateTestContext([CallerMemberName] String testName = "")
         {
-            return new TestHostContext(this, testName);
+            var hc = new TestHostContext(this, testName);
+            _expressionManager = new ExpressionManager();
+            _expressionManager.Initialize(hc);
+            return hc;
         }
 
         private void InitializeExecutionContext(TestHostContext hc)
         {
+            _expressions = new DictionaryContextData();
             _jobContext = new JobContext();
 
-            var executionContext = new Mock<IExecutionContext>();
+            _ec = new Mock<IExecutionContext>();
-            executionContext.SetupAllProperties();
+            _ec.SetupAllProperties();
-            executionContext.Setup(x => x.JobContext).Returns(_jobContext);
+            _ec.Setup(x => x.ExpressionValues).Returns(_expressions);
-
+            _ec.Setup(x => x.JobContext).Returns(_jobContext);
-            _templateContext = new TemplateContext();
-            _templateContext.State[nameof(IExecutionContext)] = executionContext.Object;
-        }
-
-        private bool Evaluate(string expression)
-        {
-            var parser = new ExpressionParser();
-            var functions = new IFunctionInfo[]
-            {
-                new FunctionInfo<AlwaysFunction>(PipelineTemplateConstants.Always, 0, 0),
-                new FunctionInfo<CancelledFunction>(PipelineTemplateConstants.Cancelled, 0, 0),
-                new FunctionInfo<FailureFunction>(PipelineTemplateConstants.Failure, 0, 0),
-                new FunctionInfo<SuccessFunction>(PipelineTemplateConstants.Success, 0, 0),
-            };
-            var tree = parser.CreateTree(expression, null, null, functions);
-            var result = tree.Evaluate(null, null, _templateContext, null);
-            return result.IsTruthy;
         }
     }
 }

src/Test/L0/Worker/JobExtensionL0.cs

@@ -22,6 +22,7 @@ namespace GitHub.Runner.Common.Tests.Worker
         private Mock<IJobServerQueue> _jobServerQueue;
         private Mock<IConfigurationStore> _config;
         private Mock<IPagingLogger> _logger;
+        private Mock<IExpressionManager> _express;
         private Mock<IContainerOperationProvider> _containerProvider;
         private Mock<IDiagnosticLogManager> _diagnosticLogManager;
 
@@ -34,6 +35,7 @@ namespace GitHub.Runner.Common.Tests.Worker
             _jobServerQueue = new Mock<IJobServerQueue>();
             _config = new Mock<IConfigurationStore>();
             _logger = new Mock<IPagingLogger>();
+            _express = new Mock<IExpressionManager>();
             _containerProvider = new Mock<IContainerOperationProvider>();
             _diagnosticLogManager = new Mock<IDiagnosticLogManager>();
             _directoryManager = new Mock<IPipelineDirectoryManager>();
@@ -98,7 +100,7 @@ namespace GitHub.Runner.Common.Tests.Worker
             };
 
             Guid jobId = Guid.NewGuid();
-            _message = new Pipelines.AgentJobRequestMessage(plan, timeline, jobId, "test", "test", null, null, null, new Dictionary<string, VariableValue>(), new List<MaskHint>(), new Pipelines.JobResources(), new Pipelines.ContextData.DictionaryContextData(), new Pipelines.WorkspaceOptions(), steps, null, null, null, null);
+            _message = new Pipelines.AgentJobRequestMessage(plan, timeline, jobId, "test", "test", null, null, null, new Dictionary<string, VariableValue>(), new List<MaskHint>(), new Pipelines.JobResources(), new Pipelines.ContextData.DictionaryContextData(), new Pipelines.WorkspaceOptions(), steps, null, null);
             GitHubContext github = new GitHubContext();
             github["repository"] = new Pipelines.ContextData.StringContextData("actions/runner");
             _message.ContextData.Add("github", github);
@@ -106,6 +108,7 @@ namespace GitHub.Runner.Common.Tests.Worker
             hc.SetSingleton(_actionManager.Object);
             hc.SetSingleton(_config.Object);
             hc.SetSingleton(_jobServerQueue.Object);
+            hc.SetSingleton(_express.Object);
             hc.SetSingleton(_containerProvider.Object);
             hc.SetSingleton(_directoryManager.Object);
             hc.SetSingleton(_diagnosticLogManager.Object);

src/Test/L0/Worker/JobRunnerL0.cs

@@ -53,6 +53,9 @@ namespace GitHub.Runner.Common.Tests.Worker
             }
 
             _tokenSource = new CancellationTokenSource();
+            var expressionManager = new ExpressionManager();
+            expressionManager.Initialize(hc);
+            hc.SetSingleton<IExpressionManager>(expressionManager);
 
             _jobRunner = new JobRunner();
             _jobRunner.Initialize(hc);
@@ -60,7 +63,7 @@ namespace GitHub.Runner.Common.Tests.Worker
             TaskOrchestrationPlanReference plan = new TaskOrchestrationPlanReference();
             TimelineReference timeline = new Timeline(Guid.NewGuid());
             Guid jobId = Guid.NewGuid();
-            _message = new Pipelines.AgentJobRequestMessage(plan, timeline, jobId, testName, testName, null, null, null, new Dictionary<string, VariableValue>(), new List<MaskHint>(), new Pipelines.JobResources(), new Pipelines.ContextData.DictionaryContextData(), new Pipelines.WorkspaceOptions(), new List<Pipelines.ActionStep>(), null, null, null, null);
+            _message = new Pipelines.AgentJobRequestMessage(plan, timeline, jobId, testName, testName, null, null, null, new Dictionary<string, VariableValue>(), new List<MaskHint>(), new Pipelines.JobResources(), new Pipelines.ContextData.DictionaryContextData(), new Pipelines.WorkspaceOptions(), new List<Pipelines.ActionStep>(), null, null);
             _message.Variables[Constants.Variables.System.Culture] = "en-US";
             _message.Resources.Endpoints.Add(new ServiceEndpoint()
             {

src/Test/L0/Worker/StepsRunnerL0.cs

@@ -1,17 +1,16 @@
-using System;
+using GitHub.DistributedTask.WebApi;
+using GitHub.Runner.Worker;
+using Moq;
+using System;
 using System.Collections.Generic;
 using System.Globalization;
 using System.Linq;
 using System.Runtime.CompilerServices;
 using System.Threading.Tasks;
-using Moq;
 using Xunit;
 using GitHub.DistributedTask.Expressions2;
 using GitHub.DistributedTask.Pipelines.ContextData;
 using GitHub.DistributedTask.ObjectTemplating.Tokens;
-using GitHub.DistributedTask.WebApi;
-using GitHub.Runner.Common.Util;
-using GitHub.Runner.Worker;
 
 namespace GitHub.Runner.Common.Tests.Worker
 {
@@ -27,6 +26,9 @@ namespace GitHub.Runner.Common.Tests.Worker
         private TestHostContext CreateTestContext([CallerMemberName] String testName = "")
         {
             var hc = new TestHostContext(this, testName);
+            var expressionManager = new ExpressionManager();
+            expressionManager.Initialize(hc);
+            hc.SetSingleton<IExpressionManager>(expressionManager);
             Dictionary<string, VariableValue> variablesToCopy = new Dictionary<string, VariableValue>();
             _variables = new Variables(
                 hostContext: hc,
@@ -46,7 +48,6 @@ namespace GitHub.Runner.Common.Tests.Worker
             _contexts["runner"] = new DictionaryContextData();
             _contexts["job"] = _jobContext;
             _ec.Setup(x => x.ExpressionValues).Returns(_contexts);
-            _ec.Setup(x => x.ExpressionFunctions).Returns(new List<IFunctionInfo>());
             _ec.Setup(x => x.JobContext).Returns(_jobContext);
 
             _stepContext = new StepsContext();
@@ -54,9 +55,6 @@ namespace GitHub.Runner.Common.Tests.Worker
 
             _ec.Setup(x => x.PostJobSteps).Returns(new Stack<IStep>());
 
-            var trace = hc.GetTrace();
-            _ec.Setup(x => x.Write(It.IsAny<string>(), It.IsAny<string>())).Callback((string tag, string message) => { trace.Info($"[{tag}]{message}"); });
-
             _stepsRunner = new StepsRunner();
             _stepsRunner.Initialize(hc);
             return hc;
@@ -381,11 +379,16 @@ namespace GitHub.Runner.Common.Tests.Worker
         {
             using (TestHostContext hc = CreateTestContext())
             {
+                var expressionManager = new Mock<IExpressionManager>();
+                expressionManager.Object.Initialize(hc);
+                hc.SetSingleton<IExpressionManager>(expressionManager.Object);
+                expressionManager.Setup(x => x.Evaluate(It.IsAny<IExecutionContext>(), It.IsAny<string>(), It.IsAny<bool>())).Throws(new Exception());
+
                 // Arrange.
                 var variableSets = new[]
                 {
-                    new[] { CreateStep(hc, TaskResult.Succeeded, "fromJson('not json')") },
+                    new[] { CreateStep(hc, TaskResult.Succeeded, "success()") },
-                    new[] { CreateStep(hc, TaskResult.Succeeded, "fromJson('not json')") },
+                    new[] { CreateStep(hc, TaskResult.Succeeded, "success()") },
                 };
                 foreach (var variableSet in variableSets)
                 {
@@ -510,78 +513,7 @@ namespace GitHub.Runner.Common.Tests.Worker
             }
         }
 
-        [Fact]
+        private Mock<IActionRunner> CreateStep(TestHostContext hc, TaskResult result, string condition, Boolean continueOnError = false, MappingToken env = null, string name = "Test", bool setOutput = false)
-        [Trait("Level", "L0")]
-        [Trait("Category", "Worker")]
-        public async Task StepContextOutcome()
-        {
-            using (TestHostContext hc = CreateTestContext())
-            {
-                // Arrange.
-                var step1 = CreateStep(hc, TaskResult.Succeeded, "success()", contextName: "step1");
-                var step2 = CreateStep(hc, TaskResult.Failed, "steps.step1.outcome == 'success'", continueOnError: true, contextName: "step2");
-                var step3 = CreateStep(hc, TaskResult.Succeeded, "steps.step1.outcome == 'success' && steps.step2.outcome == 'failure'", contextName: "step3");
-
-                _ec.Object.Result = null;
-
-                _ec.Setup(x => x.JobSteps).Returns(new Queue<IStep>(new[] { step1.Object, step2.Object, step3.Object }));
-
-                // Act.
-                await _stepsRunner.RunAsync(jobContext: _ec.Object);
-
-                // Assert.
-                Assert.Equal(TaskResult.Succeeded, _ec.Object.Result ?? TaskResult.Succeeded);
-
-                step1.Verify(x => x.RunAsync(), Times.Once);
-                step2.Verify(x => x.RunAsync(), Times.Once);
-                step3.Verify(x => x.RunAsync(), Times.Once);
-
-                Assert.Equal(TaskResult.Succeeded.ToActionResult().ToString(), _stepContext.GetScope(null)["step1"].AssertDictionary("")["outcome"].AssertString(""));
-                Assert.Equal(TaskResult.Succeeded.ToActionResult().ToString(), _stepContext.GetScope(null)["step1"].AssertDictionary("")["conclusion"].AssertString(""));
-                Assert.Equal(TaskResult.Failed.ToActionResult().ToString(), _stepContext.GetScope(null)["step2"].AssertDictionary("")["outcome"].AssertString(""));
-                Assert.Equal(TaskResult.Succeeded.ToActionResult().ToString(), _stepContext.GetScope(null)["step2"].AssertDictionary("")["conclusion"].AssertString(""));
-                Assert.Equal(TaskResult.Succeeded.ToActionResult().ToString(), _stepContext.GetScope(null)["step3"].AssertDictionary("")["outcome"].AssertString(""));
-                Assert.Equal(TaskResult.Succeeded.ToActionResult().ToString(), _stepContext.GetScope(null)["step3"].AssertDictionary("")["conclusion"].AssertString(""));
-
-            }
-        }
-
-        [Fact]
-        [Trait("Level", "L0")]
-        [Trait("Category", "Worker")]
-        public async Task StepContextConclusion()
-        {
-            using (TestHostContext hc = CreateTestContext())
-            {
-                // Arrange.
-                var step1 = CreateStep(hc, TaskResult.Succeeded, "false", contextName: "step1");
-                var step2 = CreateStep(hc, TaskResult.Failed, "steps.step1.conclusion == 'skipped'", continueOnError: true, contextName: "step2");
-                var step3 = CreateStep(hc, TaskResult.Succeeded, "steps.step1.outcome == 'skipped' && steps.step2.outcome == 'failure' && steps.step2.conclusion == 'success'", contextName: "step3");
-
-                _ec.Object.Result = null;
-
-                _ec.Setup(x => x.JobSteps).Returns(new Queue<IStep>(new[] { step1.Object, step2.Object, step3.Object }));
-
-                // Act.
-                await _stepsRunner.RunAsync(jobContext: _ec.Object);
-
-                // Assert.
-                Assert.Equal(TaskResult.Succeeded, _ec.Object.Result ?? TaskResult.Succeeded);
-
-                step1.Verify(x => x.RunAsync(), Times.Never);
-                step2.Verify(x => x.RunAsync(), Times.Once);
-                step3.Verify(x => x.RunAsync(), Times.Once);
-
-                Assert.Equal(TaskResult.Skipped.ToActionResult().ToString(), _stepContext.GetScope(null)["step1"].AssertDictionary("")["outcome"].AssertString(""));
-                Assert.Equal(TaskResult.Skipped.ToActionResult().ToString(), _stepContext.GetScope(null)["step1"].AssertDictionary("")["conclusion"].AssertString(""));
-                Assert.Equal(TaskResult.Failed.ToActionResult().ToString(), _stepContext.GetScope(null)["step2"].AssertDictionary("")["outcome"].AssertString(""));
-                Assert.Equal(TaskResult.Succeeded.ToActionResult().ToString(), _stepContext.GetScope(null)["step2"].AssertDictionary("")["conclusion"].AssertString(""));
-                Assert.Equal(TaskResult.Succeeded.ToActionResult().ToString(), _stepContext.GetScope(null)["step3"].AssertDictionary("")["outcome"].AssertString(""));
-                Assert.Equal(TaskResult.Succeeded.ToActionResult().ToString(), _stepContext.GetScope(null)["step3"].AssertDictionary("")["conclusion"].AssertString(""));
-            }
-        }
-
-        private Mock<IActionRunner> CreateStep(TestHostContext hc, TaskResult result, string condition, Boolean continueOnError = false, MappingToken env = null, string name = "Test", bool setOutput = false, string contextName = null)
         {
             // Setup the step.
             var step = new Mock<IActionRunner>();
@@ -592,8 +524,7 @@ namespace GitHub.Runner.Common.Tests.Worker
                 {
                     Name = name,
                     Id = Guid.NewGuid(),
-                    Environment = env,
+                    Environment = env
-                    ContextName = contextName ?? "Test"
                 });
 
             // Setup the step execution context.
@@ -603,10 +534,8 @@ namespace GitHub.Runner.Common.Tests.Worker
             stepContext.Setup(x => x.Variables).Returns(_variables);
             stepContext.Setup(x => x.EnvironmentVariables).Returns(_env);
             stepContext.Setup(x => x.ExpressionValues).Returns(_contexts);
-            stepContext.Setup(x => x.ExpressionFunctions).Returns(new List<IFunctionInfo>());
             stepContext.Setup(x => x.JobContext).Returns(_jobContext);
             stepContext.Setup(x => x.StepsContext).Returns(_stepContext);
-            stepContext.Setup(x => x.ContextName).Returns(step.Object.Action.ContextName);
             stepContext.Setup(x => x.Complete(It.IsAny<TaskResult?>(), It.IsAny<string>(), It.IsAny<string>()))
                 .Callback((TaskResult? r, string currentOperation, string resultCode) =>
                 {
@@ -614,9 +543,6 @@ namespace GitHub.Runner.Common.Tests.Worker
                     {
                         stepContext.Object.Result = r;
                     }
-
-                    _stepContext.SetOutcome("", stepContext.Object.ContextName, (stepContext.Object.Outcome ?? stepContext.Object.Result ?? TaskResult.Succeeded).ToActionResult().ToString());
-                    _stepContext.SetConclusion("", stepContext.Object.ContextName, (stepContext.Object.Result ?? TaskResult.Succeeded).ToActionResult().ToString());
                 });
             var trace = hc.GetTrace();
             stepContext.Setup(x => x.Write(It.IsAny<string>(), It.IsAny<string>())).Callback((string tag, string message) => { trace.Info($"[{tag}]{message}"); });

src/Test/L0/Worker/WorkerL0.cs

@@ -67,7 +67,7 @@ namespace GitHub.Runner.Common.Tests.Worker
                     new Pipelines.ContextData.DictionaryContextData()
                 },
             };
-            var jobRequest = new Pipelines.AgentJobRequestMessage(plan, timeline, JobId, jobName, jobName, new StringToken(null, null, null, "ubuntu"), sidecarContainers, null, variables, new List<MaskHint>(), resources, context, null, actions, null, null, null, null);
+            var jobRequest = new Pipelines.AgentJobRequestMessage(plan, timeline, JobId, jobName, jobName, new StringToken(null, null, null, "ubuntu"), sidecarContainers, null, variables, new List<MaskHint>(), resources, context, null, actions, null, null);
             return jobRequest;
         }
 

src/runnerversion

@@ -1 +1 @@
-2.168.0
+2.165.2