Check if file exists before reading

.gitignore

@@ -2,7 +2,6 @@
 **/bin
 **/obj
 **/libs
-**/lib
 
 # editors
 **/*.xproj

docs/adrs/0263-proxy-support.md

@@ -1,61 +0,0 @@
-# ADR 263: Self Hosted Runner Proxies
-
-**Date**: 2019-11-13
-
-**Status**: Accepted
-
-## Context
-
-- Proxy support is required for some enterprises and organizations to start using their own self hosted runners
-- While there is not a standard convention, many applications support setting proxies via the environmental variables `http_proxy`, `https_proxy`, `no_proxy`, such as curl, wget, perl, python, docker, git, R, ect
-  - Some of these applications use `HTTPS_PROXY` versus `https_proxy`, but most understand or primarily support the lowercase variant
-
-## Decision
-
-We will update the Runner to use the conventional environment variables for proxies: `http_proxy`, `https_proxy` and `no_proxy` if they are set.
-These are described in detail below:
-- `https_proxy` a proxy URL for all https traffic. It may contain basic authentication credentials. For example:
-  - http://proxy.com
-  - http://127.0.0.1:8080
-  - http://user:password@proxy.com
-- `http_proxy` a proxy URL for all http traffic. It may contain basic authentication credentials. For example:
-  - http://proxy.com
-  - http://127.0.0.1:8080
-  - http://user:password@proxy.com
-- `no_proxy` a comma seperated list of hosts that should not use the proxy. An optional port may be specified
-  - `google.com`
-  - `yahoo.com:443`
-  - `google.com,bing.com`
-
-We won't use `http_proxy` for https traffic when `https_proxy` is not set, this behavior lines up with any libcurl based tools (curl, git) and wget.
-Otherwise action authors and workflow users need to adjust to differences between the runner proxy convention, and tools used by their actions and scripts.  
-
-Example:  
-  Customer set `http_proxy=http://127.0.0.1:8888` and configure the runner against `https://github.com/owner/repo`, with the `https_proxy` -> `http_proxy` fallback, the runner will connect to server without any problem. However, if user runs `git push` to `https://github.com/owner/repo`, `git` won't use the proxy since it require `https_proxy` to be set for any https traffic.
-
-> `golang`, `node.js` and other dev tools from the linux community use `http_proxy` for both http and https traffic base on my research.
-
-A majority of our users are using Linux where these variables are commonly required to be set by various programs. By reading these values, we simplify the process for self hosted runners to set up proxy, and expose it in a way users are already familiar with.
-
-A password provided for a proxy will be masked in the logs.
-
-We will support the lowercase and uppercase variants, with lowercase taking priority if both are set.
-
-### No Proxy Format
-
-While exact implementations are different per application on handle `no_proxy` env, most applications accept a comma separated list of hosts. Some accept wildcard characters (*). We are going to do exact case-insentive matches, and not support wildcards at this time.
-For example:
-- example.com will match example.com, foo.example.com, foo.bar.example.com
-- foo.example.com will match bar.foo.example.com and foo.example.com
-
-We will not support IP addresses for `no_proxy`, only hostnames.
-
-## Consequences
-
-1. Enterprises and organizations needing proxy support will be able to embrace self hosted runners
-2. Users will need to set these environmental variables before configuring the runner in order to use a proxy when configuring
-3. The runner will read from the environmental variables during config and runtime and use the provided proxy if it exists
-4. Users may need to pass these environmental variables into other applications if they do not natively take these variables
-5. Action authors may need to update their workflows to react to the these environment variables
-6. We will document the way of setting environmental variables for runners using the environmental variables and how the runner uses them
-7. Like all other secrets, users will be able to relatively easily figure out proxy password if they can modify a workflow file running on a self hosted machine

docs/adrs/0274-step-outcome-and-conclusion.md

@@ -1,62 +0,0 @@
-# ADR 0274: Step outcome and conclusion
-
-**Date**: 2020-01-13
-
-**Status**: Accepted
-
-## Context
-
-This ADR proposes adding `steps.<id>.outcome` and `steps.<id>.conclusion` to the steps context.
-
-This allows downstream a step to run based on whether a previous step succeeded or failed.
-
-Reminder, currently the steps contains `steps.<id>.outputs`.
-
-## Decision
-
-For steps that have completed, populate `steps.<id>.outcome` and `steps.<id>.conclusion` with one of the following values:
-
-- `success`
-- `failure`
-- `cancelled`
-- `skipped`
-
-When a continue-on-error step fails, the outcome will be `failure` even though the final conclusion is `success`.
-
-### Example
-
-```yaml
-steps:
-
-  - id: experimental
-    continue-on-error: true
-    run: ./build.sh experimental
-
-  - if: ${{ steps.experimental.outcome == 'success' }}
-    run: ./publish.sh experimental
-```
-
-### Terminology
-
-The runs API uses the term `conclusion`.
-
-Therefore we use a different term `outcome` for the value prior to continue-on-error.
-
-The following is a snippet from the runs API response payload:
-
-```json
-      "steps": [
-        {
-          "name": "Set up job",
-          "status": "completed",
-          "conclusion": "success",
-          "number": 1,
-          "started_at": "2020-01-09T11:06:16.000-05:00",
-          "completed_at": "2020-01-09T11:06:18.000-05:00"
-        },
-```
-
-## Consequences
-
-- Update runner
-- Update [docs](https://help.github.com/en/actions/automating-your-workflow-with-github-actions/contexts-and-expression-syntax-for-github-actions#steps-context)

docs/adrs/0276-problem-matchers.md

@@ -1,263 +0,0 @@
-# ADR 0276: Problem Matchers
-
-**Date** 2019-06-05
-
-**Status** Accepted
-
-## Context
-
-Compilation failures during a CI build should surface good error messages.
-
-For example, the actual compile errors from the typescript compiler should bubble as issues in the UI. And not simply "tsc exited with exit code 1".
-
-VSCode has an extensible model for solving this type of problem. VSCode allows users to configure which problems matchers to use, when scanning output. For example, a user can apply the `tsc` problem matcher to receive a rich error output experience in VSCode, when compiling their typescript project.
-
-The problem-matcher concept fits well with "setup" actions. For example, the `setup-nodejs` action will download node.js, add it to the PATH, and register the `tsc` problem matcher. For the duration of the job, the `tsc` problem matcher will be applied against the output.
-
-## Decision
-
-### Registration
-
-#### Using `##` command
-
-`##[add-matcher]path-to-problem-matcher-config.json`
-
-Using a `##` command allows for flexibility:
-- Ad hoc scripts can register problem matchers
-- Allows problem matchers to be conditionally registered
-
-Note, if a matcher with the same name is registered a second time, it will clobber the first instance.
-
-#### Unregister using `##` command
-
-A way out for rare cases where scoping is a problem.
-
-`##[remove-matcher]owner`
-
-For the this to be usable, the `owner` needs to be discoverable. Therefore, debug print the owner on registration.
-
-### Single line matcher
-
-Consider the output:
-
-```
-[...]
-
-Build FAILED.
-
-"C:\temp\problemmatcher\myproject\ConsoleApp1\ConsoleApp1.sln" (default target) (1) ->
-"C:\temp\problemmatcher\myproject\ConsoleApp1\ConsoleApp1\ConsoleApp1.csproj" (default target) (2) ->
-"C:\temp\problemmatcher\myproject\ConsoleApp1\ClassLibrary1\ClassLibrary1.csproj" (default target) (3) ->
-(CoreCompile target) -> 
-  Class1.cs(16,24): warning CS0612: 'ClassLibrary1.Helpers.MyHelper.Name' is obsolete [C:\temp\problemmatcher\myproject\ConsoleApp1\ClassLibrary1\ClassLibrary1.csproj]
-
-
-"C:\temp\problemmatcher\myproject\ConsoleApp1\ConsoleApp1.sln" (default target) (1) ->
-"C:\temp\problemmatcher\myproject\ConsoleApp1\ConsoleApp1\ConsoleApp1.csproj" (default target) (2) ->
-"C:\temp\problemmatcher\myproject\ConsoleApp1\ClassLibrary1\ClassLibrary1.csproj" (default target) (3) ->
-(CoreCompile target) -> 
-  Helpers\MyHelper.cs(16,30): error CS1002: ; expected [C:\temp\problemmatcher\myproject\ConsoleApp1\ClassLibrary1\ClassLibrary1.csproj]
-
-    1 Warning(s)
-    1 Error(s)
-```
-
-The below match configuration uses a regular expression to discover problem lines. And the match groups are mapped into issue-properties.
-
-```json
-"owner": "msbuild",
-"pattern": [
-  {
-    "regexp": "^\\s*([^:]+)\\((\\d+),(\\d+)\\): (error|warning) ([^:]+): (.*) \\[(.+)\\]$",
-    "file": 1,
-    "line": 2,
-    "column": 3,
-    "severity": 4,
-    "code": 5,
-    "message": 6,
-    "fromPath": 7
-  }
-]
-```
-
-The above output and match configuration produces the following matches:
-
-```
-line:     Class1.cs(16,24): warning CS0612: 'ClassLibrary1.Helpers.MyHelper.Name' is obsolete [C:\myrepo\myproject\ConsoleApp1\ClassLibrary1\ClassLibrary1.csproj]
-file:     Class1.cs
-line:     16
-column:   24
-severity: warning
-code:     CS0612
-message:  'ClassLibrary1.Helpers.MyHelper.Name' is obsolete
-fromPath: C:\myrepo\myproject\ConsoleApp1\ClassLibrary1\ClassLibrary1.csproj
-```
-
-```
-line:     Helpers\MyHelper.cs(16,30): error CS1002: ; expected [C:\myrepo\myproject\ConsoleApp1\ClassLibrary1\ClassLibrary1.csproj]
-file:     Helpers\MyHelper.cs
-line:     16
-column:   30
-severity: error
-code:     CS1002
-message:  ; expected
-fromPath: C:\myrepo\myproject\ConsoleApp1\ClassLibrary1\ClassLibrary1.csproj
-```
-
-Additionally the line will appear red in the web UI (prefix with `##[error]`).
-
-Note, an error does not imply task failure. Exit codes communicate failure.
-
-Note, strip color codes when evaluating regular expressions.
-
-### Multi-line matcher
-
-Consider the below output from ESLint in stylish mode. The file name is printed once, yet multiple error lines are printed.
-
-```
-test.js
-  1:0   error  Missing "use strict" statement                 strict
-  5:10  error  'addOne' is defined but never used             no-unused-vars
-✖ 2 problems (2 errors, 0 warnings)
-```
-
-The below match configuration uses multiple regular expressions, for the multiple lines.
-
-And the last pattern of a multiline matcher can specify the `loop` property. This allows multiple errors to be discovered.
-
-```json
-"owner": "eslint-stylish",
-"pattern": [
-  {
-    "regexp": "^([^\\s].*)$",
-    "file": 1
-  },
-  {
-    "regexp": "^\\s+(\\d+):(\\d+)\\s+(error|warning|info)\\s+(.*)\\s\\s+(.*)$",
-    "line": 1,
-    "column": 2,
-    "severity": 3,
-    "message": 4,
-    "code": 5,
-    "loop": true
-  }
-]
-```
-
-The above output and match configuration produces two matches:
-
-```
-line:     1:0   error  Missing "use strict" statement                 strict
-file:     test.js
-line:     1
-column:   0
-severity: error
-message:  Missing "use strict" statement
-code:     strict
-```
-
-```
-line:     5:10  error  'addOne' is defined but never used             no-unused-vars
-file:     test.js
-line:     5
-column:   10
-severity: error
-message:  'addOne' is defined but never used
-code:     no-unused-vars
-```
-
-Note, in the above example only the error line will appear red in the web UI. The \"file\" line will not appear red.
-
-### Other details
-
-#### Configuration `owner`
-
-Can be used to stomp over or remove.
-
-#### Rooting the file
-
-The goal of the file information is to provide a hyperlink in the UI.
-
-Solving this problem means:
-- Rooting the file when unrooted:
-  - Use the `fromPath` if specified (assume file path)
-  - Use the `github.workspace` (where the repo is cloned on disk)
-- Match against a repository to determine the relative path within the repo
-
-This is a place where we diverge from VSCode. VSCode task configuration are specific to the local workspace (workspace root is known or can be specified). We're solving a more generic problem, so we need more information - specifically the `fromPath` property - in order to accurately root the path.
-
-In order to avoid creating inaccurate hyperlinks on the error issues, the agent will verify the file exists and is in the main repository. Otherwise omit the file property from the error issue and debug trace what happened.
-
-#### Supported severity levels
-
-Ordinal ignore case:
-
-- `warning`
-- `error`
-
-Coalesce empty with \"error\". For any other values, omit logging an issue and debug trace what happened.
-
-#### Default severity level
-
-Problem matchers are unable to interpret severity strings other than `warning` and `error`. The `severity` match group expects `warning` or `error` (case insensitive).
-
-However some tools indicate error/warning in different ways. For example `flake8` uses codes like `E100`, `W200`, and `F300` (error, warning, fatal, respectively).
-
-Therefore, allow a property `severity`, sibling to `owner`, which identifies the default severity for the problem matcher. This allows two problem matchers are registered - one for warnings and one for errors.
-
-For example, given the following `flake8` output:
-
-```
-./bootcamp/settings.py:156:80: E501 line too long (94 > 79 characters)
-./bootcamp/settings.py:165:5: F403 'from local_settings import *' used; unable to detect undefined names
-```
-
-Two problem matchers can be used:
-
-```json
-{
-    "problemMatcher": [
-        {
-            "owner": "flake8",
-            "pattern": [
-                {
-                    "regexp": "^(.+):(\\d+):(\\d+): ([EF]\\d+) (.+)$",
-                    "file": 1,
-                    "line": 2,
-                    "column": 3,
-                    "code": 4,
-                    "message": 5
-                }
-            ]
-        },
-        {
-            "owner": "flake8-warnings",
-            "severity": "warning",
-            "pattern": [
-                {
-                    "regexp": "^(.+):(\\d+):(\\d+): (W\\d+) (.+)$",
-                    "file": 1,
-                    "line": 2,
-                    "column": 3,
-                    "code": 4,
-                    "message": 5
-                }
-            ]
-        }
-    ]
-}
-```
-
-#### Mitigate regular expression denial of service (ReDos)
-
-If a matcher exceeds a 1 second timeout when processing a line, retry up to two three times total.
-After three unsuccessful attempts, warn and eject the matcher. The matcher will not run again for the duration of the job.
-
-### Where we diverge from VSCode
-
-- We added the `fromPath` concept for rooting paths. This is done differently in VSCode, since a task is the scope (root path well known). For us, the job is the scope.
-- VSCode allows additional activation info background tasks that are always running (recompile on files changed). They allow regular expressions to define when the matcher scope begins and ends. This is an interesting concept that we could leverage to help solve our scoping problem.
-
-## Consequences
-
-- Setup actions should register problem matchers

docs/adrs/0277-run-action-shell-options.md

@@ -1,93 +0,0 @@
-# ADR 0277: Run action shell option
-
-**Date** 2019-07-09
-
-**Status** Accepted
-
-## Context
-run-actions run scripts using a platform specific shell:
-`bash -eo pipefail` on non-windows, and `cmd.exe /c /d /s` on windows
-
-The `shell` option overwrites this to allow different flags or completely different shells/interpreters
-
-A small example is:
-```yml
-jobs:
-  bash-job:
-    actions:
-    - run: echo "Hello"
-      shell: bash
-  python-job:
-    actions:
-    - run: print("Hello")
-      shell: python {0}
-```
-
-## Decision
-
-___
-
-### Shell option
-The keyword being used is `shell` 
-
-`shell` can be either:
-
-1. Builtins / Explicitly supported keywords. It is useful to support at least `cmd`, and `powershell` on Windows. Because `cmd my_cmd_script` and `powershell my_ps1_script` are not valid the same way many Linux/cross-platform interpreters are, e.g. `bash myscript` or `python myscript`. Those tools (and potentially others) also require the correct file extension to run, or must be run in a particular way to get the exit codes consistently, so we must have first class knowledge about them. We provide default templates for these keywords as follows:
-    - `cmd`: Default is: `%ComSpec% /D /E:ON /V:OFF /S /C "CALL "{0}""` where the script name is automatically appended with `.cmd` and substituted for `{0}`
-        - Note this is equivalent to the default Windows behavior if no shell option is given
-    - `pwsh`: Default is: `pwsh -command "& '{0}'"` where the script is automatically appended with `.ps1`
-    - `powershell`: Default is: `powershell -command "& '{0}'"` where the script is automatically appended with `.ps1`
-    - `bash`: Uses `bash --noprofile --norc -eo pipefail {0}`
-        - The default behavior on non-Windows if no shell is given is to attempt this first
-    - `sh`: Uses `sh -e {0}`
-        - This is the default behavior on non-Windows if no shell is given, AND `bash` (see above) was not located on the PATH
-    - `python`: `python {0}`
-    - **NOTE**: The exact command ran may vary by machine. We only provide default arguments and command format for the listed shell. While the above behavior is expected on hosted machines, private runners may vary. For example, `sh` (or other commands) may actually be a link to `/bin/dash`, `/bin/bash`, or other
-
-1. A template string: `command [...options] {0} [...more_options]`
-    - As above, the file name of the temporary script will be templated in. This gives users more control to have options at any location relative to the script path
-    - The first whitespace-delimited word of the string will be interpreted as the command
-    - e.g. `python {0} arg1 arg2` or similar can be used if passing args is needed. Some shells will require other options after the filename for various reasons
-
-Note that (1) simply provides defaults that are executed with the same mechanism as (2). That is:
-- A temporary script file is generated, and the path to that file is templated into the string at `{0}`
-- The first word of the formatted string is assumed to be a command, and we attempt to locate its full path
-- The fully qualified path to the command, plus the remaining arguments, is executed
-    - e.g. `shell: bash` expands to `/bin/bash --noprofile --norc -eo pipefail /runner/_layout/_work/_temp/f8d4fb2b-19d9-47e6-a786-4cc538d52761.sh` on my private runner
-
-At this time, **THE LIST OF WELL-KNOWN SHELL OPTIONS IS**:
-- cmd - Windows (hosted vs2017, vs2019) only
-- powershell - Windows (hosted vs2017, vs2019) only
-- sh - All hosted platforms
-- pwsh - All hosted platforms
-- bash - All hosted platforms
-- python - All hosted platforms. Can use setup-python to configure which python will be used
-___
-
-### Containers
-For container jobs, `shell` should just work the same as above, transparently. We will simply `exec` the command in the job container, passing the same arguments in
-
-___
-
-### Exit codes / Error action preference
-
-For builtin shells, we provide defaults that make the most sense for CI, running within Actions, and being executed by our runner
-
-bash/sh:
-- Fail-fast behavior using `set -e o pipefail` is the default for `bash` and `shell` builtins, and by default when no option is given on non-Windows platforms
-- Users can opt out of fail-fast and take full control easily by providing a template string to the shell options, eg: `bash {0}`.
-- sh-like shells exit with the exit code of the last command executed in a script, and is our default behavior. Thus the runner reports the status of the step as fail/succeed based on this exit code
-
-powershell/pwsh
-- Fail-fast behavior when possible. For `pwsh` and `powershell` builtins, we will prepend `$ErrorActionPreference = 'stop'` to script contents
-- We append `if ((Test-Path -LiteralPath variable:\LASTEXITCODE)) { exit $LASTEXITCODE }` to powershell scripts to get Action statuses to reflect the script's last exit code
-- Users can always opt out by not using the builtins, and providing a shell option like: `pwsh -File {0}`, or `powershell -Command "& '{0}'"`, depending on need
-
-cmd
-- There doesnt seem to be a way to fully opt in to fail-fast behavior other than writing your script to check each error code and respond accordingly, so we cant actually provide that behavior by default, it will be completely up to the user to write this behavior into their script
-- cmd.exe will exit (return the error code to the runner) with the errorlevel of the last program it executed. This is internally consistent with the previous default behavior (sh, pwsh) and is the cmd.exe default, so we keep that behavior
-
-## Consequences
-Valid `shell` options will depend on the hosted images. We will need to maintain tight image compat
-
-First class support for a shell will require a major version schema change to modify. We cannot remove or modify the behavior of a well-known supported option, However, adding first class support for new shells is backwards compatible. For instance, we can add a well-known `python` option, because non-well-known options would have always needed to include `{0}`, e.g. `python {0}`

docs/adrs/0278-env-context.md

@@ -1,60 +0,0 @@
-# ADR 0278: Env Context
-
-**Date**: 2019-09-30
-
-**Status**: Accepted
-
-## Context
-
-User wants to reference workflow variables defined in workflow yaml file for action's input, displayName and condition.  
-
-## Decision
-
-### Add `env` context in the runner
-
-Runner will create and populate the `env` context for every job execution using following logic:
-1. On job start, create `env` context with any environment variables in the job message, these are env defined in customer's YAML file's job/workflow level `env` section.
-2. Update `env` context when customer use `::set-env::` to set env at the runner level.
-3. Update `env` context with step's `env` block before each step runs.
-
-The `env` context is only available in the runner, customer can't use the `env` context in any server evaluation part, just like the `runner` context
-
-Example yaml:
-```yaml
-
-env:
-  env1: 10
-  env2: 20
-  env3: 30
-jobs:
-  build:
-    env:
-      env1: 100
-      env2: 200
-    runs-on: ubuntu-latest
-    steps:
-      - run: |
-          echo ${{ env.env1 }}  // 1000
-          echo $env1            // 1000
-          echo $env2            // 200
-          echo $env3            // 30
-        if: env.env2 == 200     // true
-        name: ${{ env.env1 }}_${{ env.env2 }}   //1000_200
-        env:
-          env1: 1000
-```
-
-### Don't populate the `env` context with environment variables from runner machine. 
-
-With job container and container action, the `env` context may not have the right value customer want and will cause confusion.  
-Ex:
-```yaml
-build:
-  runs-on: ubuntu-latest   <- $USER=runner in hosted machine
-  container: ubuntu:16.04   <- $USER=root in container
-  steps:
-  - run: echo ${{env.USER}}    <- what should customer expect this output?  runner/root
-  - uses: docker://ubuntu:18.04
-    with:
-      args: echo ${{env.USER}}  <- what should customer expect this output? runner/root
-```

docs/adrs/0279-hashFiles-expression-function.md

@@ -1,71 +0,0 @@
-# ADR 0279: HashFiles Expression Function
-
-**Date**: 2019-09-30
-
-**Status**: Accepted
-
-## Context
-First party action `actions/cache` needs a input which is an explicit `key` used for restoring and saving the cache. For packages caching, the most comment `key` might be the hash result of contents from all `package-lock.json` under `node_modules` folder.
-  
-There are serval different ways to get the hash `key` input for `actions/cache` action.
-
-1. Customer calculate the `key` themselves from a different action, customer won't like this since it needs extra step for using cache feature
-```yaml
-  steps:
-  - run: |
-      hash=some_linux_hash_method(file1, file2, file3)
-      echo ::set-output name=hash::$hash
-    id: createHash
-  - uses: actions/cache@v1
-    with:
-      key: ${{ steps.createHash.outputs.hash }}
-``` 
-
-2. Make the `key` input of `actions/cache` follow certain convention to calculate hash, this limited the `key` input to a certain format customer may not want.
-```yaml
-  steps:
-  - uses: actions/cache@v1
-    with:
-      key: ${{ runner.os }}|${{ github.workspace }}|**/package-lock.json
-```
-
-## Decision
-
-### Add hashFiles() function to expression engine for calculate files' hash
-
-`hashFiles()` will only allow on runner side since it needs to read files on disk, using `hashFiles()` on any server side evaluated expression will cause runtime errors.
-
-`hashFiles()` will only support hashing files under the `$GITHUB_WORKSPACE` since the expression evaluated on the runner, if customer use job container or container action, the runner won't have access to file system inside the container.
-
-`hashFiles()` will only take 1 parameters:
- - `hashFiles('**/package-lock.json')`  // Search files under $GITHUB_WORKSPACE and calculate a hash for them
-
-**Question: Do we need to support more than one match patterns?**  
-Ex: `hashFiles('**/package-lock.json', '!toolkit/core/package-lock.json', '!toolkit/io/package-lock.json')`  
-Answer: Only support single match pattern for GA, we can always add later.
-
-This will help customer has better experience with the `actions/cache` action's input.
-```yaml
-  steps:
-  - uses: actions/cache@v1
-    with:
-      key: ${{hashFiles('**/package-lock.json')}}-${{github.ref}}-${{runner.os}}
-```
-
-For search pattern, we will use basic globbing (`*` `?` and `[]`) and globstar (`**`).
-
-Additional pattern details:
-- Root relative paths with `github.workspace` (the main repo)
-- Make `*` match files that start with `.`
-- Case insensitive on Windows
-- Accept `\` or `/` path separators on Windows
-
-Hashing logic:
-1. Get all files under `$GITHUB_WORKSPACE`.
-2. Use search pattern filter all files to get files that matches the search pattern. (search pattern only apply to file path not folder path)
-3. Sort all matched files by full file path in alphabet order.
-4. Use SHA256 algorithm to hash each matched file and store hash result.
-5. Use SHA256 to hash all stored files' hash results to get the final 64 chars hash result.
-
-**Question: Should we include the folder structure info into the hash?**  
-Answer: No

docs/adrs/0280-command-input-echoing.md

@@ -1,30 +0,0 @@
-# ADR 0280: Echoing of Command Input
-
-**Date**: 2019-11-04  
-
-**Status**: Accepted
-
-## Context
-
-Command echoing as a default behavior tends to clutter the user logs, so we want to swap to a system where users have to opt in to see this information.
-
-Command outputs will still be echoed in the case there are any errors processing such commands. This is so the end user can have more context on why the command failed and help with troubleshooting.
-
-Echo output in the user logs can be explicitly controlled by the new commands `::echo::on` and `::echo::off`. By default, echoing is enabled if `ACTIONS_STEP_DEBUG` secret is enabled, otherwise echoing is disabled.
-
-## Decision
-- The only commands that currently echo output are
-  - `remove-matcher`
-  - `add-matcher`
-  - `add-path`
-- These will no longer echo the command, if processed successfully
-- All commands echo the input when any of these conditions is fulfilled:
-  1. When such commands fail with an error
-  2. When `::echo::on` is set
-  3. When the `ACTIONS_STEP_DEBUG` is set, and echoing hasn't been explicitly disabled with `::echo::off`
-- There are a few commands that won't be echoed, even when echo is enabled. These are (as of 2019/11/04):
-   - `add-mask`
-   - `debug`
-   - `warning`
-   - `error`
-- The three commands above will not echo, either because echoing the command would leak secrets (e.g. `add-mask`), or it would not add any additional troubleshooting information to the logs (e.g. `debug`). It's expected that future commands would follow these "echo-suppressing" guidelines as well. Echo-suppressed commands are still free to output other information to the logs, as deemed fit.

docs/adrs/0297-base64-masking-trailing-characters.md

@@ -1,48 +0,0 @@
-# ADR 0297: Base64 Masking Trailing Characters
-
-**Date** 2020-01-21
-
-**Status** Proposed
-
-## Context
-
-The Runner registers a number of Value Encoders, which mask various encodings of a provided secret. Currently, we register a 3 base64 Encoders:
-- The base64 encoded secret
-- The secret with the first character removed then base64 encoded
-- The secret with the first two characters removed then base64 encoded
-
-This gives us good coverage across the board for secrets and secrets with a prefix (i.e. `base64($user:$pass)`).
-
-However, we don't have great coverage for cases where the secret has a string appended to it before it is base64 encoded (i.e.: `base64($pass\n))`). 
-
-Most notably we've seen this as a result of user error where a user accidentially appends a newline or space character before encoding their secret in base64.
-
-## Decision
-
-### Trim end characters
-
-We are going to modify all existing base64 encoders to trim information before registering as a secret.
-We will trim:
-- `=` from the end of all base64 strings. This is a padding character that contains no information. 
-  - Based on the number of `=`'s at the end of a base64 string, a malicious user could predict the length of the original secret modulo 3. 
-    - If a user saw `***==`, they would know the secret could be 1,4,7,10... characters.
-- If a string contains `=` we will also trim the last non-padding character from the base64 secret.
-  - This character can change if a string is appended to the secret before the encoding.
-
-
-### Register a fourth encoder
-
-We will also add back in the original base64 encoded secret encoder for four total encoders:
-- The base64 encoded secret
-- The base64 encoded secret trimmed
-- The secret with the first character removed then base64 encoded and trimmed
-- The secret with the first two characters removed then base64 encoded and trimmed
-
-This allows us to fully cover the most common scenario where a user base64 encodes their secret and expects the entire thing to be masked.
-This will result in us only revealing length or bit information when a prefix or suffix is added to a secret before encoding. 
-
-## Consequences
-
-- In the case where a secret has a prefix or suffix added before base64 encoding, we may now reveal up to 20 bits of information and the length of the original string modulo 3, rather then the original 16 bits and no length information
-- Secrets with a suffix appended before encoding will now be masked across the board. Previously it was only masked if it was a multiple of 3 characters
-- Performance will suffer in a neglible way

docs/adrs/0354-runner-machine-info.md

@@ -1,35 +0,0 @@
-# ADR 354: Expose runner machine info
-
-**Date**: 2020-03-02
-
-**Status**: Pending
-
-## Context
-
-- Provide a mechanism in the runner to include extra information in `Set up job` step's log.
-  Ex: Include OS/Software info from Hosted image.
-
-## Decision
-
-The runner will look for a file `.setup_info` under the runner's root directory, The file can be a JSON with a simple schema.
-```json
-[
-  {
-    "group": "OS Detail",
-    "detail": "........"
-  },
-  {
-    "group": "Software Detail",
-    "detail": "........"
-  }
-]
-```
-The runner will use `##[group]` and `##[endgroup]` to fold all detail info into an expandable group.
-
-Both [virtual-environments](https://github.com/actions/virtual-environments) and self-hosted runners can use this mechanism to add extra logging info to the `Set up job` step's log.
-
-## Consequences
-
-1. Change the runner to best effort read/parse `.extra_setup_info` file under runner root directory.
-2. [virtual-environments](https://github.com/actions/virtual-environments) generate the file during image generation.
-3. Change MMS provisioner to properly copy the file to runner root directory at runtime.

docs/contribute.md

@@ -44,7 +44,7 @@ Sample developer flow:
 ```bash
 git clone https://github.com/actions/runner
 cd ./src
-./dev.(sh/cmd) layout # the runner that built from source is in {root}/_layout
+./dev.(sh/cmd) layout # the runner that build from source is in {root}/_layout
 <make code changes>
 ./dev.(sh/cmd) build # {root}/_layout will get updated
 ./dev.(sh/cmd) test # run all unit tests before git commit/push

docs/design/auth.md

@@ -1,61 +0,0 @@
-# Runner Authentication and Authorization
-
-## Goals
-  - Support runner installs in untrusted domains.
-  - The account that configures or runs the runner process is not relevant for accessing GitHub resources.
-  - Accessing GitHub resources is done with a per-job token which expires when job completes.
-  - The token is granted to trusted parts of the system including the runner, actions and script steps specified by the workflow author as trusted.
-  - All OAuth tokens that come from the Token Service that the runner uses to access Actions Service resources are the same. It's just the scope and expiration of the token that may vary.
-
-## Configuration
-
-Configuring a self-hosted runner is [covered here in the documentation](https://help.github.com/en/actions/hosting-your-own-runners/adding-self-hosted-runners).
-
-Configuration is done with the user being authenticated via a time-limited, GitHub runner registration token.
-
-*Your credentials are never used for registering the runner with the service.*
-
-![Self-hosted runner config](../res/self-hosted-config.png)
-
-During configuration, an RSA public/private key pair is created, the private key is stored in file on disk. On Windows, the content is protected with DPAPI (machine level encrypted - runner only valid on that machine) and on Linux/OSX with `chmod` permissions.
-
-Using your credentials, the runner is registered with the service by sending the public key to the service which adds that runner to the pool and stores the public key, the Token Service will generate a `clientId` associated with the public key.
-
-## Start and Listen
-
-After configuring the runner, the runner can be started interactively (`./run.cmd` or `./run.sh`) or as a service.
-
-![Self-hosted runner start](../res/self-hosted-start.png)
-
-On start, the runner listener process loads the RSA private key (on Windows decrypting with machine key DPAPI), and asks the Token Service for an OAuth token which is signed with the RSA private key.
-The server then responds with an OAuth token that grants permission to access the message queue (HTTP long poll), allowing the runner to acquire the messages it will eventually run.
-
-## Run a workflow
-
-When a workflow is run, its labels are evaluated, it is matched to a runner and a message is placed in a queue of messages for that runner.  
-The runner then starts listening for jobs via the message queue HTTP long poll.  
-The message is encrypted with the runner's public key, stored during runner configuration.  
-
-![Runner workflow run](../res/workflow-run.png)
-
-A workflow is queued as a result of a triggered [event](https://help.github.com/en/actions/reference/events-that-trigger-workflows). Workflows can be scheduled to [run at specific UTC times](https://help.github.com/en/actions/reference/events-that-trigger-workflows#scheduled-events-schedule) using POSIX `cron` syntax.
-An [OAuth token](http://self-issued.info/docs/draft-ietf-oauth-json-web-token.html) is generated, granting limited access to the host in Actions Service associated with the github.com repository/organization.
-The lifetime of the OAuth token is the lifetime of the run or at most the [job timeout (default: 6 hours)](https://help.github.com/en/actions/reference/workflow-syntax-for-github-actions#jobsjob_idtimeout-minutes), plus 10 additional minutes.
-
-## Accessing GitHub resources
-
-The job message sent to the runner contains the OAuth token to talk back to the Actions Service.
-The runner listener parent process will spawn a runner worker process for that job and send it the job message over IPC.
-The token is never persisted.
-
-Each action is run as a unique subprocess.
-The encrypted access token will be provided as an environment variable in each action subprocess.  
-The token is registered with the runner as a secret and scrubbed from the logs as they are written.
-
-Authentication in a workflow run to github.com can be accomplished by using the [`GITHUB_TOKEN`](https://help.github.com/en/actions/configuring-and-managing-workflows/authenticating-with-the-github_token#about-the-github_token-secret)) secret. This token expires after 60 minutes. Please note that this token is different from the OAuth token that the runner uses to talk to the Actions Service.
-
-## Hosted runner authentication
-
-Hosted runner authentication differs from self-hosted authentication in that runners do not undergo a registration process, but instead, the hosted runners get the OAuth token directly by reading the `.credentials` file. The scope of this particular token is limited for a given workflow job execution, and the token is revoked as soon as the job is finished.
-
-![Hosted runner config and start](../res/hosted-config-start.png)

docs/res/runner-auth-diags.txt

@@ -1,52 +0,0 @@
-# Markup used to generate the runner auth diagrams: https://websequencediagrams.com
-
-title Runner Configuration (self-hosted only)
-
-note left of Runner: GitHub repo URL as input
-Runner->github.com: Retrieve Actions Service access using runner registration token
-github.com->Runner: Access token for Actions Service
-note left of Runner: Generate RSA key pair
-note left of Runner: Store encrypted RSA private key on disk
-Runner->Actions Service: Register runner using Actions Service access token
-note right of Runner: Runner name, RSA public key sent
-note right of Actions Service: Public key stored
-Actions Service->Token Service: Register runner as an app along with the RSA public key
-note right of Token Service: Public key stored
-Token Service->Actions Service: Client Id for the runner application
-Actions Service->Runner: Client Id and Token Endpoint URL
-note left of Runner: Store runner configuration info into .runner file
-note left of Runner: Store Token registration info into .credentials file
-
-title Runner Start and Running (self-hosted only)
-
-Runner.Listener->Runner.Listener: Start
-note left of Runner.Listener: Load config info from .runner
-note left of Runner.Listener: Load token registration from .credentials
-Runner.Listener->Token Service: Exchange OAuth token (happens every 50 mins)
-note right of Runner.Listener: Construct JWT token, use Client Id signed by RSA private key
-note left of Actions Service: Find corresponding RSA public key, use Client Id\nVerify JWT token's signature
-Token Service->Runner.Listener: OAuth token with limited permission and valid for 50 mins
-Runner.Listener->Actions Service: Connect to Actions Service with OAuth token
-Actions Service->Runner.Listener: Workflow job
-
-title Running workflow
-
-Runner.Listener->Service (Message Queue): Get message
-note right of Runner.Listener: Authenticate with exchanged OAuth token
-Event->Actions Service: Queue workflow
-Actions Service->Actions Service: Generate OAuth token per job
-Actions Service->Actions Service: Build job message with the OAuth token
-Actions Service->Actions Service: Encrypt job message with the target runner's public key
-Actions Service->Service (Message Queue): Send encrypted job message to runner
-Service (Message Queue)->Runner.Listener: Send job
-note right of Runner.Listener: Decrypt message with runner's private key
-Runner.Listener->Runner.Worker: Create worker process per job and run the job
-
-title Runner Configuration, Start and Running (hosted only)
-
-Machine Management Service->Runner.Listener: Construct .runner configuration file, store token in .credentials
-Runner.Listener->Runner.Listener: Start
-note left of Runner.Listener: Load config info from .runner
-note left of Runner.Listener: Load OAuth token from .credentials
-Runner.Listener->Actions Service: Connect to Actions Service with OAuth token in .credentials
-Actions Service->Runner.Listener: Workflow job

releaseNote.md

@@ -1,27 +1,17 @@
 ## Features
-  - Update Runner Register GitHub API URL to Support Org-level Runner (#339 #345 #352) 
-  - Preserve workflow file/line/column for better error messages (#356)
-  - Switch to use token service instead of SPS for exchanging oauth token. (#325)
-  - Load and print machine setup info from .setup_info (#364)
-  - Expose job name as $GITHUB_JOB (#366)
-  - Add support for job outputs. (#365) 
-  - Set CI=true when launch process in actions runner. (#374)
-  - Set steps.<id>.outcome and steps.<id>.conclusion. (#372)
-  - Add support for workflow/job defaults. (#369)
-  - Expose GITHUB_REPOSITORY_OWNER and ${{github.repository_owner}}. (#378)
+  - Remove runner flow: Change from PAT to "deletion token" in prompt (#225) 
+  - Expose github.run_id and github.run_number to action runtime env. (#224)
 
 ## Bugs
-  - Use authenticate endpoint for testing runner connection. (#311) 
-  - Commands translate file path from container action (#331)
-  - Change problem matchers output to debug (#363)
-  - Switch hashFiles to extension function (#362)
-  - Add expanded volumes strings to container mounts (#384)
+  - Clean up error messages for container scenarios (#221)
+  - Pick shell from prependpath (#231)
 
 ## Misc
-  - Add runner auth documentation (#357) 
+  - Runner code cleanup  (#218 #227, #228, #229, #230) 
+  - Consume dotnet core 3.1 in runner. (#213)
 
 ## Windows x64
-We recommend configuring the runner in a root folder of the Windows drive (e.g. "C:\actions-runner"). This will help avoid issues related to service identity folder permissions and long file path restrictions on Windows
+We recommend configuring the runner under "<DRIVE>:\actions-runner". This will help avoid issues related to service identity folder permissions and long file path restrictions on Windows
 ``` 
 // Create a folder under the drive root
 mkdir \actions-runner ; cd \actions-runner
@@ -29,7 +19,7 @@ mkdir \actions-runner ; cd \actions-runner
 Invoke-WebRequest -Uri https://github.com/actions/runner/releases/download/v<RUNNER_VERSION>/actions-runner-win-x64-<RUNNER_VERSION>.zip -OutFile actions-runner-win-x64-<RUNNER_VERSION>.zip
 // Extract the installer
 Add-Type -AssemblyName System.IO.Compression.FileSystem ; 
-[System.IO.Compression.ZipFile]::ExtractToDirectory("$PWD\actions-runner-win-x64-<RUNNER_VERSION>.zip", "$PWD")
+[System.IO.Compression.ZipFile]::ExtractToDirectory("$HOME\Downloads\actions-runner-win-x64-<RUNNER_VERSION>.zip", "$PWD")
 ```
 
 ## OSX
@@ -38,7 +28,7 @@ Add-Type -AssemblyName System.IO.Compression.FileSystem ;
 // Create a folder
 mkdir actions-runner && cd actions-runner
 // Download the latest runner package
-curl -O -L https://github.com/actions/runner/releases/download/v<RUNNER_VERSION>/actions-runner-osx-x64-<RUNNER_VERSION>.tar.gz
+curl -O https://github.com/actions/runner/releases/download/v<RUNNER_VERSION>/actions-runner-osx-x64-<RUNNER_VERSION>.tar.gz
 // Extract the installer
 tar xzf ./actions-runner-osx-x64-<RUNNER_VERSION>.tar.gz
 ```
@@ -49,7 +39,7 @@ tar xzf ./actions-runner-osx-x64-<RUNNER_VERSION>.tar.gz
 // Create a folder
 mkdir actions-runner && cd actions-runner
 // Download the latest runner package
-curl -O -L https://github.com/actions/runner/releases/download/v<RUNNER_VERSION>/actions-runner-linux-x64-<RUNNER_VERSION>.tar.gz
+curl -O https://github.com/actions/runner/releases/download/v<RUNNER_VERSION>/actions-runner-linux-x64-<RUNNER_VERSION>.tar.gz
 // Extract the installer
 tar xzf ./actions-runner-linux-x64-<RUNNER_VERSION>.tar.gz
 ```
@@ -60,7 +50,7 @@ tar xzf ./actions-runner-linux-x64-<RUNNER_VERSION>.tar.gz
 // Create a folder
 mkdir actions-runner && cd actions-runner
 // Download the latest runner package
-curl -O -L https://github.com/actions/runner/releases/download/v<RUNNER_VERSION>/actions-runner-linux-arm64-<RUNNER_VERSION>.tar.gz
+curl -O https://github.com/actions/runner/releases/download/v<RUNNER_VERSION>/actions-runner-linux-arm64-<RUNNER_VERSION>.tar.gz
 // Extract the installer
 tar xzf ./actions-runner-linux-arm64-<RUNNER_VERSION>.tar.gz
 ```
@@ -71,7 +61,7 @@ tar xzf ./actions-runner-linux-arm64-<RUNNER_VERSION>.tar.gz
 // Create a folder
 mkdir actions-runner && cd actions-runner
 // Download the latest runner package
-curl -O -L https://github.com/actions/runner/releases/download/v<RUNNER_VERSION>/actions-runner-linux-arm-<RUNNER_VERSION>.tar.gz
+curl -O https://github.com/actions/runner/releases/download/v<RUNNER_VERSION>/actions-runner-linux-arm-<RUNNER_VERSION>.tar.gz
 // Extract the installer
 tar xzf ./actions-runner-linux-arm-<RUNNER_VERSION>.tar.gz
 ```

src/Misc/expressionFunc/hashFiles/.eslintignore

@@ -1,3 +0,0 @@
-dist/
-lib/
-node_modules/

src/Misc/expressionFunc/hashFiles/.eslintrc.json

@@ -1,59 +0,0 @@
-{
-    "plugins": ["jest", "@typescript-eslint"],
-    "extends": ["plugin:github/es6"],
-    "parser": "@typescript-eslint/parser",
-    "parserOptions": {
-      "ecmaVersion": 9,
-      "sourceType": "module",
-      "project": "./tsconfig.json"
-    },
-    "rules": {
-      "eslint-comments/no-use": "off",
-      "import/no-namespace": "off",
-      "no-console": "off",
-      "no-unused-vars": "off",
-      "@typescript-eslint/no-unused-vars": "error",
-      "@typescript-eslint/explicit-member-accessibility": ["error", {"accessibility": "no-public"}],
-      "@typescript-eslint/no-require-imports": "error",
-      "@typescript-eslint/array-type": "error",
-      "@typescript-eslint/await-thenable": "error",
-      "@typescript-eslint/ban-ts-ignore": "error",
-      "camelcase": "off",
-      "@typescript-eslint/camelcase": "error",
-      "@typescript-eslint/class-name-casing": "error",
-      "@typescript-eslint/explicit-function-return-type": ["error", {"allowExpressions": true}],
-      "@typescript-eslint/func-call-spacing": ["error", "never"],
-      "@typescript-eslint/generic-type-naming": ["error", "^[A-Z][A-Za-z]*$"],
-      "@typescript-eslint/no-array-constructor": "error",
-      "@typescript-eslint/no-empty-interface": "error",
-      "@typescript-eslint/no-explicit-any": "error",
-      "@typescript-eslint/no-extraneous-class": "error",
-      "@typescript-eslint/no-for-in-array": "error",
-      "@typescript-eslint/no-inferrable-types": "error",
-      "@typescript-eslint/no-misused-new": "error",
-      "@typescript-eslint/no-namespace": "error",
-      "@typescript-eslint/no-non-null-assertion": "warn",
-      "@typescript-eslint/no-object-literal-type-assertion": "error",
-      "@typescript-eslint/no-unnecessary-qualifier": "error",
-      "@typescript-eslint/no-unnecessary-type-assertion": "error",
-      "@typescript-eslint/no-useless-constructor": "error",
-      "@typescript-eslint/no-var-requires": "error",
-      "@typescript-eslint/prefer-for-of": "warn",
-      "@typescript-eslint/prefer-function-type": "warn",
-      "@typescript-eslint/prefer-includes": "error",
-      "@typescript-eslint/prefer-interface": "error",
-      "@typescript-eslint/prefer-string-starts-ends-with": "error",
-      "@typescript-eslint/promise-function-async": "error",
-      "@typescript-eslint/require-array-sort-compare": "error",
-      "@typescript-eslint/restrict-plus-operands": "error",
-      "semi": "off",
-      "@typescript-eslint/semi": ["error", "never"],
-      "@typescript-eslint/type-annotation-spacing": "error",
-      "@typescript-eslint/unbound-method": "error"
-    },
-    "env": {
-      "node": true,
-      "es6": true,
-      "jest/globals": true
-    }
-  }

src/Misc/expressionFunc/hashFiles/.prettierignore

@@ -1,3 +0,0 @@
-dist/
-lib/
-node_modules/

src/Misc/expressionFunc/hashFiles/.prettierrc.json

@@ -1,11 +0,0 @@
-{
-    "printWidth": 80,
-    "tabWidth": 2,
-    "useTabs": false,
-    "semi": false,
-    "singleQuote": true,
-    "trailingComma": "none",
-    "bracketSpacing": false,
-    "arrowParens": "avoid",
-    "parser": "typescript"
-  }

src/Misc/expressionFunc/hashFiles/README.md

@@ -1 +0,0 @@
-To update hashFiles under `Misc/layoutbin` run `npm install && npm run all`

src/Misc/expressionFunc/hashFiles/package.json

@@ -1,35 +0,0 @@
-{
-  "name": "hashFiles",
-  "version": "1.0.0",
-  "description": "GitHub Actions HashFiles() expression function",
-  "main": "lib/hashFiles.js",
-  "scripts": {
-    "build": "tsc",
-    "format": "prettier --write **/*.ts",
-    "format-check": "prettier --check **/*.ts",
-    "lint": "eslint src/**/*.ts",
-    "pack": "ncc build -o ../../layoutbin/hashFiles",
-    "all": "npm run build && npm run format && npm run lint && npm run pack"
-  },
-  "repository": {
-    "type": "git",
-    "url": "git+https://github.com/actions/runner.git"
-  },
-  "keywords": [
-    "actions"
-  ],
-  "author": "GitHub Actions",
-  "license": "MIT",
-  "dependencies": {
-    "@actions/glob": "^0.1.0"
-  },
-  "devDependencies": {
-    "@types/node": "^12.7.12",
-    "@typescript-eslint/parser": "^2.8.0",
-    "@zeit/ncc": "^0.20.5",
-    "eslint": "^5.16.0",
-    "eslint-plugin-github": "^2.0.0",
-    "prettier": "^1.19.1",
-    "typescript": "^3.6.4"
-  }
-}

src/Misc/expressionFunc/hashFiles/src/hashFiles.ts

@@ -1,55 +0,0 @@
-import * as glob from '@actions/glob'
-import * as crypto from 'crypto'
-import * as fs from 'fs'
-import * as stream from 'stream'
-import * as util from 'util'
-import * as path from 'path'
-
-async function run(): Promise<void> {
-  // arg0 -> node
-  // arg1 -> hashFiles.js
-  // env[followSymbolicLinks] = true/null
-  // env[patterns] -> glob patterns
-  let followSymbolicLinks = false
-  const matchPatterns = process.env.patterns || ''
-  if (process.env.followSymbolicLinks === 'true') {
-    console.log('Follow symbolic links')
-    followSymbolicLinks = true
-  }
-
-  console.log(`Match Pattern: ${matchPatterns}`)
-  let hasMatch = false
-  const githubWorkspace = process.cwd()
-  const result = crypto.createHash('sha256')
-  let count = 0
-  const globber = await glob.create(matchPatterns, {followSymbolicLinks})
-  for await (const file of globber.globGenerator()) {
-    console.log(file)
-    if (!file.startsWith(`${githubWorkspace}${path.sep}`)) {
-      console.log(`Ignore '${file}' since it is not under GITHUB_WORKSPACE.`)
-      continue
-    }
-    if (fs.statSync(file).isDirectory()) {
-      console.log(`Skip directory '${file}'.`)
-      continue
-    }
-    const hash = crypto.createHash('sha256')
-    const pipeline = util.promisify(stream.pipeline)
-    await pipeline(fs.createReadStream(file), hash)
-    result.write(hash.digest())
-    count++
-    if (!hasMatch) {
-      hasMatch = true
-    }
-  }
-  result.end()
-
-  if (hasMatch) {
-    console.log(`Find ${count} files to hash.`)
-    console.error(`__OUTPUT__${result.digest('hex')}__OUTPUT__`)
-  } else {
-    console.error(`__OUTPUT____OUTPUT__`)
-  }
-}
-
-run()

src/Misc/expressionFunc/hashFiles/tsconfig.json

@@ -1,12 +0,0 @@
-{
-  "compilerOptions": {
-    "target": "es6",                          /* Specify ECMAScript target version: 'ES3' (default), 'ES5', 'ES2015', 'ES2016', 'ES2017', 'ES2018', 'ES2019' or 'ESNEXT'. */
-    "module": "commonjs",                     /* Specify module code generation: 'none', 'commonjs', 'amd', 'system', 'umd', 'es2015', or 'ESNext'. */
-    "outDir": "./lib",                        /* Redirect output structure to the directory. */
-    "rootDir": "./src",                       /* Specify the root directory of input files. Use to control the output directory structure with --outDir. */
-    "strict": true,                           /* Enable all strict type-checking options. */
-    "noImplicitAny": true,                    /* Raise error on expressions and declarations with an implied 'any' type. */
-    "esModuleInterop": true                   /* Enables emit interoperability between CommonJS and ES Modules via creation of namespace objects for all imports. Implies 'allowSyntheticDefaultImports'. */
-  },
-  "exclude": ["node_modules", "**/*.test.ts"]
-}

src/NuGet.Config

@@ -3,6 +3,8 @@
   <packageSources>
     <!--To inherit the global NuGet package sources remove the <clear/> line below -->
     <clear />
+    <add key="dotnet-core" value="https://www.myget.org/F/dotnet-core/api/v3/index.json" />
+    <add key="dotnet-buildtools" value="https://www.myget.org/F/dotnet-buildtools/api/v3/index.json" />    
     <add key="api.nuget.org" value="https://api.nuget.org/v3/index.json" />
   </packageSources>
 </configuration>

src/Runner.Common/ActionCommand.cs

@@ -9,27 +9,26 @@ namespace GitHub.Runner.Common
     {
         private static readonly EscapeMapping[] _escapeMappings = new[]
         {
+            new EscapeMapping(token: "%", replacement: "%25"),
             new EscapeMapping(token: ";", replacement: "%3B"),
             new EscapeMapping(token: "\r", replacement: "%0D"),
             new EscapeMapping(token: "\n", replacement: "%0A"),
             new EscapeMapping(token: "]", replacement: "%5D"),
-            new EscapeMapping(token: "%", replacement: "%25"),
         };
 
         private static readonly EscapeMapping[] _escapeDataMappings = new[]
         {
             new EscapeMapping(token: "\r", replacement: "%0D"),
             new EscapeMapping(token: "\n", replacement: "%0A"),
-            new EscapeMapping(token: "%", replacement: "%25"),
         };
 
         private static readonly EscapeMapping[] _escapePropertyMappings = new[]
         {
+            new EscapeMapping(token: "%", replacement: "%25"),
             new EscapeMapping(token: "\r", replacement: "%0D"),
             new EscapeMapping(token: "\n", replacement: "%0A"),
             new EscapeMapping(token: ":", replacement: "%3A"),
             new EscapeMapping(token: ",", replacement: "%2C"),
-            new EscapeMapping(token: "%", replacement: "%25"),
         };
 
         private readonly Dictionary<string, string> _properties = new Dictionary<string, string>(StringComparer.OrdinalIgnoreCase);

src/Runner.Common/ConfigurationStore.cs

@@ -15,9 +15,6 @@ namespace GitHub.Runner.Common
     [DataContract]
     public sealed class RunnerSettings
     {
-        [DataMember(Name = "IsHostedServer", EmitDefaultValue = false)]
-        private bool? _isHostedServer;
-
         [DataMember(EmitDefaultValue = false)]
         public int AgentId { get; set; }
 
@@ -45,21 +42,6 @@ namespace GitHub.Runner.Common
         [DataMember(EmitDefaultValue = false)]
         public string MonitorSocketAddress { get; set; }
 
-        [IgnoreDataMember]
-        public bool IsHostedServer
-        {
-            get
-            {
-                // Old runners do not have this property. Hosted runners likely don't have this property either.
-                return _isHostedServer ?? true;
-            }
-
-            set
-            {
-                _isHostedServer = value;
-            }
-        }
-
         /// <summary>
         // Computed property for convenience. Can either return:
         // 1. If runner was configured at the repo level, returns something like: "myorg/myrepo"
@@ -87,15 +69,6 @@ namespace GitHub.Runner.Common
                 return repoOrOrgName;
             }
         }
-
-        [OnSerializing]
-        private void OnSerializing(StreamingContext context)
-        {
-            if (_isHostedServer.HasValue && _isHostedServer.Value)
-            {
-                _isHostedServer = null;
-            }
-        }
     }
 
     [ServiceLocator(Default = typeof(ConfigurationStore))]
@@ -105,10 +78,8 @@ namespace GitHub.Runner.Common
         bool IsServiceConfigured();
         bool HasCredentials();
         CredentialData GetCredentials();
-        CredentialData GetMigratedCredentials();
         RunnerSettings GetSettings();
         void SaveCredential(CredentialData credential);
-        void SaveMigratedCredential(CredentialData credential);
         void SaveSettings(RunnerSettings settings);
         void DeleteCredential();
         void DeleteSettings();
@@ -119,11 +90,9 @@ namespace GitHub.Runner.Common
         private string _binPath;
         private string _configFilePath;
         private string _credFilePath;
-        private string _migratedCredFilePath;
         private string _serviceConfigFilePath;
 
         private CredentialData _creds;
-        private CredentialData _migratedCreds;
         private RunnerSettings _settings;
 
         public override void Initialize(IHostContext hostContext)
@@ -145,9 +114,6 @@ namespace GitHub.Runner.Common
             _credFilePath = hostContext.GetConfigFile(WellKnownConfigFile.Credentials);
             Trace.Info("CredFilePath: {0}", _credFilePath);
 
-            _migratedCredFilePath = hostContext.GetConfigFile(WellKnownConfigFile.MigratedCredentials);
-            Trace.Info("MigratedCredFilePath: {0}", _migratedCredFilePath);
-
             _serviceConfigFilePath = hostContext.GetConfigFile(WellKnownConfigFile.Service);
             Trace.Info("ServiceConfigFilePath: {0}", _serviceConfigFilePath);
         }
@@ -157,7 +123,7 @@ namespace GitHub.Runner.Common
         public bool HasCredentials()
         {
             Trace.Info("HasCredentials()");
-            bool credsStored = (new FileInfo(_credFilePath)).Exists || (new FileInfo(_migratedCredFilePath)).Exists;
+            bool credsStored = (new FileInfo(_credFilePath)).Exists;
             Trace.Info("stored {0}", credsStored);
             return credsStored;
         }
@@ -188,16 +154,6 @@ namespace GitHub.Runner.Common
             return _creds;
         }
 
-        public CredentialData GetMigratedCredentials()
-        {
-            if (_migratedCreds == null && File.Exists(_migratedCredFilePath))
-            {
-                _migratedCreds = IOUtil.LoadObject<CredentialData>(_migratedCredFilePath);
-            }
-
-            return _migratedCreds;
-        }
-
         public RunnerSettings GetSettings()
         {
             if (_settings == null)
@@ -232,21 +188,6 @@ namespace GitHub.Runner.Common
             File.SetAttributes(_credFilePath, File.GetAttributes(_credFilePath) | FileAttributes.Hidden);
         }
 
-        public void SaveMigratedCredential(CredentialData credential)
-        {
-            Trace.Info("Saving {0} migrated credential @ {1}", credential.Scheme, _migratedCredFilePath);
-            if (File.Exists(_migratedCredFilePath))
-            {
-                // Delete existing credential file first, since the file is hidden and not able to overwrite.
-                Trace.Info("Delete exist runner migrated credential file.");
-                IOUtil.DeleteFile(_migratedCredFilePath);
-            }
-
-            IOUtil.SaveObject(credential, _migratedCredFilePath);
-            Trace.Info("Migrated Credentials Saved.");
-            File.SetAttributes(_migratedCredFilePath, File.GetAttributes(_migratedCredFilePath) | FileAttributes.Hidden);
-        }
-
         public void SaveSettings(RunnerSettings settings)
         {
             Trace.Info("Saving runner settings.");
@@ -265,7 +206,6 @@ namespace GitHub.Runner.Common
         public void DeleteCredential()
         {
             IOUtil.Delete(_credFilePath, default(CancellationToken));
-            IOUtil.Delete(_migratedCredFilePath, default(CancellationToken));
         }
 
         public void DeleteSettings()

src/Runner.Common/Constants.cs

@@ -19,13 +19,11 @@ namespace GitHub.Runner.Common
     {
         Runner,
         Credentials,
-        MigratedCredentials,
         RSACredentials,
         Service,
         CredentialStore,
         Certificates,
         Options,
-        SetupInfo,
     }
 
     public static class Constants
@@ -138,12 +136,6 @@ namespace GitHub.Runner.Common
             }
         }
 
-        public static class RunnerEvent
-        {
-            public static readonly string Register = "register";
-            public static readonly string Remove = "remove";
-        }
-
         public static class Pipeline
         {
             public static class Path
@@ -170,8 +162,7 @@ namespace GitHub.Runner.Common
         public static class Path
         {
             public static readonly string ActionsDirectory = "_actions";
-            public static readonly string ActionManifestYmlFile = "action.yml";
-            public static readonly string ActionManifestYamlFile = "action.yaml";
+            public static readonly string ActionManifestFile = "action.yml";
             public static readonly string BinDirectory = "bin";
             public static readonly string DiagDirectory = "_diag";
             public static readonly string ExternalsDirectory = "externals";

src/Runner.Common/HostContext.cs

@@ -281,12 +281,6 @@ namespace GitHub.Runner.Common
                         ".credentials");
                     break;
 
-                case WellKnownConfigFile.MigratedCredentials:
-                    path = Path.Combine(
-                        GetDirectory(WellKnownDirectory.Root),
-                        ".credentials_migrated");
-                    break;
-
                 case WellKnownConfigFile.RSACredentials:
                     path = Path.Combine(
                         GetDirectory(WellKnownDirectory.Root),
@@ -322,13 +316,6 @@ namespace GitHub.Runner.Common
                         GetDirectory(WellKnownDirectory.Root),
                         ".options");
                     break;
-
-                case WellKnownConfigFile.SetupInfo:
-                    path = Path.Combine(
-                        GetDirectory(WellKnownDirectory.Root),
-                        ".setup_info");
-                    break;
-
                 default:
                     throw new NotSupportedException($"Unexpected well known config file: '{configFile}'");
             }

src/Runner.Common/Logging.cs

@@ -24,6 +24,7 @@ namespace GitHub.Runner.Common
 
         private Guid _timelineId;
         private Guid _timelineRecordId;
+        private string _pageId;
         private FileStream _pageData;
         private StreamWriter _pageWriter;
         private int _byteCount;
@@ -39,6 +40,7 @@ namespace GitHub.Runner.Common
         {
             base.Initialize(hostContext);
             _totalLines = 0;
+            _pageId = Guid.NewGuid().ToString();
             _pagesFolder = Path.Combine(hostContext.GetDirectory(WellKnownDirectory.Diag), PagingFolder);
             _jobServerQueue = HostContext.GetService<IJobServerQueue>();
             Directory.CreateDirectory(_pagesFolder);
@@ -100,7 +102,7 @@ namespace GitHub.Runner.Common
         {
             EndPage();
             _byteCount = 0;
-            _dataFileName = Path.Combine(_pagesFolder, $"{_timelineId}_{_timelineRecordId}_{++_pageCount}.log");
+            _dataFileName = Path.Combine(_pagesFolder, $"{_pageId}_{++_pageCount}.log");
             _pageData = new FileStream(_dataFileName, FileMode.CreateNew);
             _pageWriter = new StreamWriter(_pageData, System.Text.Encoding.UTF8);
         }

src/Runner.Common/RunnerServer.cs

@@ -50,10 +50,6 @@ namespace GitHub.Runner.Common
 
         // agent update
         Task<TaskAgent> UpdateAgentUpdateStateAsync(int agentPoolId, int agentId, string currentState);
-
-        // runner authorization url
-        Task<string> GetRunnerAuthUrlAsync(int runnerPoolId, int runnerId);
-        Task ReportRunnerAuthUrlErrorAsync(int runnerPoolId, int runnerId, string error);
     }
 
     public sealed class RunnerServer : RunnerService, IRunnerServer
@@ -338,20 +334,5 @@ namespace GitHub.Runner.Common
             CheckConnection(RunnerConnectionType.Generic);
             return _genericTaskAgentClient.UpdateAgentUpdateStateAsync(agentPoolId, agentId, currentState);
         }
-
-        //-----------------------------------------------------------------
-        // Runner Auth Url
-        //-----------------------------------------------------------------
-        public Task<string> GetRunnerAuthUrlAsync(int runnerPoolId, int runnerId)
-        {
-            CheckConnection(RunnerConnectionType.MessageQueue);
-            return _messageTaskAgentClient.GetAgentAuthUrlAsync(runnerPoolId, runnerId);
-        }
-
-        public Task ReportRunnerAuthUrlErrorAsync(int runnerPoolId, int runnerId, string error)
-        {
-            CheckConnection(RunnerConnectionType.MessageQueue);
-            return _messageTaskAgentClient.ReportAgentAuthUrlMigrationErrorAsync(runnerPoolId, runnerId, error);
-        }
     }
 }

src/Runner.Listener/CommandSettings.cs

@@ -190,7 +190,7 @@ namespace GitHub.Runner.Listener
         {
             return GetArgOrPrompt(
                 name: Constants.Runner.CommandLine.Args.Token,
-                description: "Enter runner remove token:",
+                description: "Enter runner deletion token:",
                 defaultValue: string.Empty,
                 validator: Validators.NonEmptyValidator);
         }
@@ -291,7 +291,7 @@ namespace GitHub.Runner.Listener
             if (!string.IsNullOrEmpty(result))
             {
                 // After read the arg from input commandline args, remove it from Arg dictionary,
-                // This will help if bad arg value passed through CommandLine arg, when ConfigurationManager ask CommandSetting the second time,
+                // This will help if bad arg value passed through CommandLine arg, when ConfigurationManager ask CommandSetting the second time, 
                 // It will prompt for input instead of continue use the bad input.
                 _trace.Info($"Remove {name} from Arg dictionary.");
                 RemoveArg(name);

src/Runner.Listener/Configuration/ConfigurationManager.cs

@@ -1,18 +1,19 @@
 using GitHub.DistributedTask.WebApi;
-using GitHub.Runner.Common;
 using GitHub.Runner.Common.Util;
-using GitHub.Runner.Sdk;
 using GitHub.Services.Common;
 using GitHub.Services.OAuth;
 using GitHub.Services.WebApi;
 using System;
 using System.Collections.Generic;
+using System.IO;
 using System.Linq;
-using System.Net.Http;
-using System.Net.Http.Headers;
-using System.Runtime.InteropServices;
 using System.Security.Cryptography;
 using System.Threading.Tasks;
+using System.Runtime.InteropServices;
+using GitHub.Runner.Common;
+using GitHub.Runner.Sdk;
+using System.Net.Http;
+using System.Net.Http.Headers;
 
 namespace GitHub.Runner.Listener.Configuration
 {
@@ -86,6 +87,7 @@ namespace GitHub.Runner.Listener.Configuration
 
             RunnerSettings runnerSettings = new RunnerSettings();
 
+            bool isHostedServer = false;
             // Loop getting url and creds until you can connect
             ICredentialProvider credProvider = null;
             VssCredentials creds = null;
@@ -93,11 +95,9 @@ namespace GitHub.Runner.Listener.Configuration
             while (true)
             {
                 // Get the URL
-                var isGitHub = StringUtil.ConvertToBoolean(Environment.GetEnvironmentVariable("_IS_GITHUB"));
                 var inputUrl = command.GetUrl();
                 if (!inputUrl.Contains("github.com", StringComparison.OrdinalIgnoreCase) &&
-                    !inputUrl.Contains("github.localhost", StringComparison.OrdinalIgnoreCase) &&
-                    !isGitHub)
+                    !inputUrl.Contains("github.localhost", StringComparison.OrdinalIgnoreCase))
                 {
                     runnerSettings.ServerUrl = inputUrl;
                     // Get the credentials
@@ -109,7 +109,7 @@ namespace GitHub.Runner.Listener.Configuration
                 {
                     runnerSettings.GitHubUrl = inputUrl;
                     var githubToken = command.GetRunnerRegisterToken();
-                    GitHubAuthResult authResult = await GetTenantCredential(inputUrl, githubToken, Constants.RunnerEvent.Register);
+                    GitHubAuthResult authResult = await GetTenantCredential(inputUrl, githubToken);
                     runnerSettings.ServerUrl = authResult.TenantUrl;
                     creds = authResult.ToVssCredentials();
                     Trace.Info("cred retrieved via GitHub auth");
@@ -118,7 +118,7 @@ namespace GitHub.Runner.Listener.Configuration
                 try
                 {
                     // Determine the service deployment type based on connection data. (Hosted/OnPremises)
-                    runnerSettings.IsHostedServer = await IsHostedServer(runnerSettings.ServerUrl, creds);
+                    isHostedServer = await IsHostedServer(runnerSettings.ServerUrl, creds);
 
                     // Validate can connect.
                     await _runnerServer.ConnectAsync(new Uri(runnerSettings.ServerUrl), creds);
@@ -200,7 +200,7 @@ namespace GitHub.Runner.Listener.Configuration
                 }
                 else
                 {
-                    // Create a new agent.
+                    // Create a new agent. 
                     agent = CreateNewAgent(runnerSettings.AgentName, publicKey);
 
                     try
@@ -249,7 +249,7 @@ namespace GitHub.Runner.Listener.Configuration
             {
                 UriBuilder configServerUrl = new UriBuilder(runnerSettings.ServerUrl);
                 UriBuilder oauthEndpointUrlBuilder = new UriBuilder(agent.Authorization.AuthorizationUrl);
-                if (!runnerSettings.IsHostedServer && Uri.Compare(configServerUrl.Uri, oauthEndpointUrlBuilder.Uri, UriComponents.SchemeAndServer, UriFormat.Unescaped, StringComparison.OrdinalIgnoreCase) != 0)
+                if (!isHostedServer && Uri.Compare(configServerUrl.Uri, oauthEndpointUrlBuilder.Uri, UriComponents.SchemeAndServer, UriFormat.Unescaped, StringComparison.OrdinalIgnoreCase) != 0)
                 {
                     oauthEndpointUrlBuilder.Scheme = configServerUrl.Scheme;
                     oauthEndpointUrlBuilder.Host = configServerUrl.Host;
@@ -277,22 +277,19 @@ namespace GitHub.Runner.Listener.Configuration
                 throw new NotSupportedException("Message queue listen OAuth token.");
             }
 
-            // Testing agent connection, detect any potential connection issue, like local clock skew that cause OAuth token expired.
+            // Testing agent connection, detect any protential connection issue, like local clock skew that cause OAuth token expired.
             var credMgr = HostContext.GetService<ICredentialManager>();
             VssCredentials credential = credMgr.LoadCredentials();
             try
             {
                 await _runnerServer.ConnectAsync(new Uri(runnerSettings.ServerUrl), credential);
-                // ConnectAsync() hits _apis/connectionData which is an anonymous endpoint
-                // Need to hit an authenticate endpoint to trigger OAuth token exchange.
-                await _runnerServer.GetAgentPoolsAsync();
                 _term.WriteSuccessMessage("Runner connection is good");
             }
             catch (VssOAuthTokenRequestException ex) when (ex.Message.Contains("Current server time is"))
             {
                 // there are two exception messages server send that indicate clock skew.
                 // 1. The bearer token expired on {jwt.ValidTo}. Current server time is {DateTime.UtcNow}.
-                // 2. The bearer token is not valid until {jwt.ValidFrom}. Current server time is {DateTime.UtcNow}.
+                // 2. The bearer token is not valid until {jwt.ValidFrom}. Current server time is {DateTime.UtcNow}.                
                 Trace.Error("Catch exception during test agent connection.");
                 Trace.Error(ex);
                 throw new Exception("The local machine's clock may be out of sync with the server time by more than five minutes. Please sync your clock with your domain or internet time and try again.");
@@ -376,12 +373,13 @@ namespace GitHub.Runner.Listener.Configuration
                     else
                     {
                         var githubToken = command.GetRunnerDeletionToken();
-                        GitHubAuthResult authResult = await GetTenantCredential(settings.GitHubUrl, githubToken, Constants.RunnerEvent.Remove);
+                        GitHubAuthResult authResult = await GetTenantCredential(settings.GitHubUrl, githubToken);
                         creds = authResult.ToVssCredentials();
                         Trace.Info("cred retrieved via GitHub auth");
                     }
 
                     // Determine the service deployment type based on connection data. (Hosted/OnPremises)
+                    bool isHostedServer = await IsHostedServer(settings.ServerUrl, creds);
                     await _runnerServer.ConnectAsync(new Uri(settings.ServerUrl), creds);
 
                     var agents = await _runnerServer.GetAgentsAsync(settings.PoolId, settings.AgentName);
@@ -404,7 +402,7 @@ namespace GitHub.Runner.Listener.Configuration
                     _term.WriteLine("Cannot connect to server, because config files are missing. Skipping removing runner from the server.");
                 }
 
-                //delete credential config files
+                //delete credential config files               
                 currentAction = "Removing .credentials";
                 if (hasCredentials)
                 {
@@ -418,7 +416,7 @@ namespace GitHub.Runner.Listener.Configuration
                     _term.WriteLine("Does not exist. Skipping " + currentAction);
                 }
 
-                //delete settings config file
+                //delete settings config file                
                 currentAction = "Removing .runner";
                 if (isConfigured)
                 {
@@ -519,23 +517,17 @@ namespace GitHub.Runner.Listener.Configuration
             }
         }
 
-        private async Task<GitHubAuthResult> GetTenantCredential(string githubUrl, string githubToken, string runnerEvent)
+        private async Task<GitHubAuthResult> GetTenantCredential(string githubUrl, string githubToken)
         {
-            var gitHubUrlBuilder = new UriBuilder(githubUrl);
-            var githubApiUrl = $"{gitHubUrlBuilder.Scheme}://{gitHubUrlBuilder.Host}/api/v3/actions/runner-registration";
+            var gitHubUrl = new UriBuilder(githubUrl);
+            var githubApiUrl = $"https://api.{gitHubUrl.Host}/repos/{gitHubUrl.Path.Trim('/')}/actions-runners/registration";
             using (var httpClientHandler = HostContext.CreateHttpClientHandler())
             using (var httpClient = new HttpClient(httpClientHandler))
             {
                 httpClient.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("RemoteAuth", githubToken);
                 httpClient.DefaultRequestHeaders.UserAgent.Add(HostContext.UserAgent);
-
-                var bodyObject = new Dictionary<string, string>()
-                {
-                    {"url", githubUrl},
-                    {"runner_event", runnerEvent}
-                };
-
-                var response = await httpClient.PostAsync(githubApiUrl, new StringContent(StringUtil.ConvertToJson(bodyObject), null, "application/json"));
+                httpClient.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/vnd.github.shuri-preview+json"));
+                var response = await httpClient.PostAsync(githubApiUrl, new StringContent("", null, "application/json"));
 
                 if (response.IsSuccessStatusCode)
                 {

src/Runner.Listener/Configuration/CredentialManager.cs

@@ -13,7 +13,7 @@ namespace GitHub.Runner.Listener.Configuration
     public interface ICredentialManager : IRunnerService
     {
         ICredentialProvider GetCredentialProvider(string credType);
-        VssCredentials LoadCredentials(bool preferMigrated = true);
+        VssCredentials LoadCredentials();
     }
 
     public class CredentialManager : RunnerService, ICredentialManager
@@ -40,7 +40,7 @@ namespace GitHub.Runner.Listener.Configuration
             return creds;
         }
 
-        public VssCredentials LoadCredentials(bool preferMigrated = true)
+        public VssCredentials LoadCredentials()
         {
             IConfigurationStore store = HostContext.GetService<IConfigurationStore>();
 
@@ -50,16 +50,6 @@ namespace GitHub.Runner.Listener.Configuration
             }
 
             CredentialData credData = store.GetCredentials();
-
-            if (preferMigrated)
-            {
-                var migratedCred = store.GetMigratedCredentials();
-                if (migratedCred != null)
-                {
-                    credData = migratedCred;
-                }
-            }
-
             ICredentialProvider credProv = GetCredentialProvider(credData.Scheme);
             credProv.CredentialData = credData;
 

src/Runner.Listener/Configuration/OAuthCredential.cs

@@ -1,5 +1,6 @@
 using System;
 using GitHub.Runner.Common;
+using GitHub.Runner.Common.Util;
 using GitHub.Runner.Sdk;
 using GitHub.Services.Common;
 using GitHub.Services.OAuth;
@@ -28,7 +29,7 @@ namespace GitHub.Runner.Listener.Configuration
             var authorizationUrl = this.CredentialData.Data.GetValueOrDefault("authorizationUrl", null);
 
             // For back compat with .credential file that doesn't has 'oauthEndpointUrl' section
-            var oauthEndpointUrl = this.CredentialData.Data.GetValueOrDefault("oauthEndpointUrl", authorizationUrl);
+            var oathEndpointUrl = this.CredentialData.Data.GetValueOrDefault("oauthEndpointUrl", authorizationUrl);
 
             ArgUtil.NotNullOrEmpty(clientId, nameof(clientId));
             ArgUtil.NotNullOrEmpty(authorizationUrl, nameof(authorizationUrl));
@@ -38,7 +39,7 @@ namespace GitHub.Runner.Listener.Configuration
             var keyManager = context.GetService<IRSAKeyManager>();
             var signingCredentials = VssSigningCredentials.Create(() => keyManager.GetKey());
             var clientCredential = new VssOAuthJwtBearerClientCredential(clientId, authorizationUrl, signingCredentials);
-            var agentCredential = new VssOAuthCredential(new Uri(oauthEndpointUrl, UriKind.Absolute), VssOAuthGrant.ClientCredentials, clientCredential);
+            var agentCredential = new VssOAuthCredential(new Uri(oathEndpointUrl, UriKind.Absolute), VssOAuthGrant.ClientCredentials, clientCredential);
 
             // Construct a credentials cache with a single OAuth credential for communication. The windows credential
             // is explicitly set to null to ensure we never do that negotiation.

src/Runner.Listener/JobDispatcher.cs

@@ -18,7 +18,6 @@ namespace GitHub.Runner.Listener
     [ServiceLocator(Default = typeof(JobDispatcher))]
     public interface IJobDispatcher : IRunnerService
     {
-        bool Busy { get; }
         TaskCompletionSource<bool> RunOnceJobCompleted { get; }
         void Run(Pipelines.AgentJobRequestMessage message, bool runOnce = false);
         bool Cancel(JobCancelMessage message);
@@ -70,8 +69,6 @@ namespace GitHub.Runner.Listener
 
         public TaskCompletionSource<bool> RunOnceJobCompleted => _runOnceJobCompleted;
 
-        public bool Busy { get; private set; }
-
         public void Run(Pipelines.AgentJobRequestMessage jobRequestMessage, bool runOnce = false)
         {
             Trace.Info($"Job request {jobRequestMessage.RequestId} for plan {jobRequestMessage.Plan.PlanId} job {jobRequestMessage.JobId} received.");
@@ -250,7 +247,7 @@ namespace GitHub.Runner.Listener
                     Task completedTask = await Task.WhenAny(jobDispatch.WorkerDispatch, Task.Delay(TimeSpan.FromSeconds(45)));
                     if (completedTask != jobDispatch.WorkerDispatch)
                     {
-                        // at this point, the job execution might encounter some dead lock and even not able to be cancelled.
+                        // at this point, the job exectuion might encounter some dead lock and even not able to be canclled.
                         // no need to localize the exception string should never happen.
                         throw new InvalidOperationException($"Job dispatch process for {jobDispatch.JobId} has encountered unexpected error, the dispatch task is not able to be canceled within 45 seconds.");
                     }
@@ -299,143 +296,250 @@ namespace GitHub.Runner.Listener
 
         private async Task RunAsync(Pipelines.AgentJobRequestMessage message, WorkerDispatcher previousJobDispatch, CancellationToken jobRequestCancellationToken, CancellationToken workerCancelTimeoutKillToken)
         {
-            Busy = true;
-            try
+            if (previousJobDispatch != null)
             {
-                if (previousJobDispatch != null)
+                Trace.Verbose($"Make sure the previous job request {previousJobDispatch.JobId} has successfully finished on worker.");
+                await EnsureDispatchFinished(previousJobDispatch);
+            }
+            else
+            {
+                Trace.Verbose($"This is the first job request.");
+            }
+
+            var term = HostContext.GetService<ITerminal>();
+            term.WriteLine($"{DateTime.UtcNow:u}: Running job: {message.JobDisplayName}");
+
+            // first job request renew succeed.
+            TaskCompletionSource<int> firstJobRequestRenewed = new TaskCompletionSource<int>();
+            var notification = HostContext.GetService<IJobNotification>();
+
+            // lock renew cancellation token.
+            using (var lockRenewalTokenSource = new CancellationTokenSource())
+            using (var workerProcessCancelTokenSource = new CancellationTokenSource())
+            {
+                long requestId = message.RequestId;
+                Guid lockToken = Guid.Empty; // lockToken has never been used, keep this here of compat
+
+                // start renew job request
+                Trace.Info($"Start renew job request {requestId} for job {message.JobId}.");
+                Task renewJobRequest = RenewJobRequestAsync(_poolId, requestId, lockToken, firstJobRequestRenewed, lockRenewalTokenSource.Token);
+
+                // wait till first renew succeed or job request is canceled
+                // not even start worker if the first renew fail
+                await Task.WhenAny(firstJobRequestRenewed.Task, renewJobRequest, Task.Delay(-1, jobRequestCancellationToken));
+
+                if (renewJobRequest.IsCompleted)
                 {
-                    Trace.Verbose($"Make sure the previous job request {previousJobDispatch.JobId} has successfully finished on worker.");
-                    await EnsureDispatchFinished(previousJobDispatch);
-                }
-                else
-                {
-                    Trace.Verbose($"This is the first job request.");
+                    // renew job request task complete means we run out of retry for the first job request renew.
+                    Trace.Info($"Unable to renew job request for job {message.JobId} for the first time, stop dispatching job to worker.");
+                    return;
                 }
 
-                var term = HostContext.GetService<ITerminal>();
-                term.WriteLine($"{DateTime.UtcNow:u}: Running job: {message.JobDisplayName}");
-
-                // first job request renew succeed.
-                TaskCompletionSource<int> firstJobRequestRenewed = new TaskCompletionSource<int>();
-                var notification = HostContext.GetService<IJobNotification>();
-
-                // lock renew cancellation token.
-                using (var lockRenewalTokenSource = new CancellationTokenSource())
-                using (var workerProcessCancelTokenSource = new CancellationTokenSource())
+                if (jobRequestCancellationToken.IsCancellationRequested)
                 {
-                    long requestId = message.RequestId;
-                    Guid lockToken = Guid.Empty; // lockToken has never been used, keep this here of compat
+                    Trace.Info($"Stop renew job request for job {message.JobId}.");
+                    // stop renew lock
+                    lockRenewalTokenSource.Cancel();
+                    // renew job request should never blows up.
+                    await renewJobRequest;
 
-                    // start renew job request
-                    Trace.Info($"Start renew job request {requestId} for job {message.JobId}.");
-                    Task renewJobRequest = RenewJobRequestAsync(_poolId, requestId, lockToken, firstJobRequestRenewed, lockRenewalTokenSource.Token);
+                    // complete job request with result Cancelled
+                    await CompleteJobRequestAsync(_poolId, message, lockToken, TaskResult.Canceled);
+                    return;
+                }
 
-                    // wait till first renew succeed or job request is canceled
-                    // not even start worker if the first renew fail
-                    await Task.WhenAny(firstJobRequestRenewed.Task, renewJobRequest, Task.Delay(-1, jobRequestCancellationToken));
+                HostContext.WritePerfCounter($"JobRequestRenewed_{requestId.ToString()}");
 
-                    if (renewJobRequest.IsCompleted)
+                Task<int> workerProcessTask = null;
+                object _outputLock = new object();
+                List<string> workerOutput = new List<string>();
+                using (var processChannel = HostContext.CreateService<IProcessChannel>())
+                using (var processInvoker = HostContext.CreateService<IProcessInvoker>())
+                {
+                    // Start the process channel.
+                    // It's OK if StartServer bubbles an execption after the worker process has already started.
+                    // The worker will shutdown after 30 seconds if it hasn't received the job message.
+                    processChannel.StartServer(
+                        // Delegate to start the child process.
+                        startProcess: (string pipeHandleOut, string pipeHandleIn) =>
+                        {
+                            // Validate args.
+                            ArgUtil.NotNullOrEmpty(pipeHandleOut, nameof(pipeHandleOut));
+                            ArgUtil.NotNullOrEmpty(pipeHandleIn, nameof(pipeHandleIn));
+
+                            // Save STDOUT from worker, worker will use STDOUT report unhandle exception.
+                            processInvoker.OutputDataReceived += delegate (object sender, ProcessDataReceivedEventArgs stdout)
+                            {
+                                if (!string.IsNullOrEmpty(stdout.Data))
+                                {
+                                    lock (_outputLock)
+                                    {
+                                        workerOutput.Add(stdout.Data);
+                                    }
+                                }
+                            };
+
+                            // Save STDERR from worker, worker will use STDERR on crash.
+                            processInvoker.ErrorDataReceived += delegate (object sender, ProcessDataReceivedEventArgs stderr)
+                            {
+                                if (!string.IsNullOrEmpty(stderr.Data))
+                                {
+                                    lock (_outputLock)
+                                    {
+                                        workerOutput.Add(stderr.Data);
+                                    }
+                                }
+                            };
+
+                            // Start the child process.
+                            HostContext.WritePerfCounter("StartingWorkerProcess");
+                            var assemblyDirectory = HostContext.GetDirectory(WellKnownDirectory.Bin);
+                            string workerFileName = Path.Combine(assemblyDirectory, _workerProcessName);
+                            workerProcessTask = processInvoker.ExecuteAsync(
+                                workingDirectory: assemblyDirectory,
+                                fileName: workerFileName,
+                                arguments: "spawnclient " + pipeHandleOut + " " + pipeHandleIn,
+                                environment: null,
+                                requireExitCodeZero: false,
+                                outputEncoding: null,
+                                killProcessOnCancel: true,
+                                redirectStandardIn: null,
+                                inheritConsoleHandler: false,
+                                keepStandardInOpen: false,
+                                highPriorityProcess: true,
+                                cancellationToken: workerProcessCancelTokenSource.Token);
+                        });
+
+                    // Send the job request message.
+                    // Kill the worker process if sending the job message times out. The worker
+                    // process may have successfully received the job message.
+                    try
                     {
-                        // renew job request task complete means we run out of retry for the first job request renew.
-                        Trace.Info($"Unable to renew job request for job {message.JobId} for the first time, stop dispatching job to worker.");
-                        return;
+                        Trace.Info($"Send job request message to worker for job {message.JobId}.");
+                        HostContext.WritePerfCounter($"RunnerSendingJobToWorker_{message.JobId}");
+                        using (var csSendJobRequest = new CancellationTokenSource(_channelTimeout))
+                        {
+                            await processChannel.SendAsync(
+                                messageType: MessageType.NewJobRequest,
+                                body: JsonUtility.ToString(message),
+                                cancellationToken: csSendJobRequest.Token);
+                        }
                     }
-
-                    if (jobRequestCancellationToken.IsCancellationRequested)
+                    catch (OperationCanceledException)
                     {
+                        // message send been cancelled.
+                        // timeout 30 sec. kill worker.
+                        Trace.Info($"Job request message sending for job {message.JobId} been cancelled, kill running worker.");
+                        workerProcessCancelTokenSource.Cancel();
+                        try
+                        {
+                            await workerProcessTask;
+                        }
+                        catch (OperationCanceledException)
+                        {
+                            Trace.Info("worker process has been killed.");
+                        }
+
                         Trace.Info($"Stop renew job request for job {message.JobId}.");
                         // stop renew lock
                         lockRenewalTokenSource.Cancel();
                         // renew job request should never blows up.
                         await renewJobRequest;
 
-                        // complete job request with result Cancelled
-                        await CompleteJobRequestAsync(_poolId, message, lockToken, TaskResult.Canceled);
+                        // not finish the job request since the job haven't run on worker at all, we will not going to set a result to server.
                         return;
                     }
 
-                    HostContext.WritePerfCounter($"JobRequestRenewed_{requestId.ToString()}");
+                    // we get first jobrequest renew succeed and start the worker process with the job message.
+                    // send notification to machine provisioner.
+                    var systemConnection = message.Resources.Endpoints.SingleOrDefault(x => string.Equals(x.Name, WellKnownServiceEndpointNames.SystemVssConnection, StringComparison.OrdinalIgnoreCase));
+                    var accessToken = systemConnection?.Authorization?.Parameters["AccessToken"];
+                    notification.JobStarted(message.JobId, accessToken, systemConnection.Url);
 
-                    Task<int> workerProcessTask = null;
-                    object _outputLock = new object();
-                    List<string> workerOutput = new List<string>();
-                    using (var processChannel = HostContext.CreateService<IProcessChannel>())
-                    using (var processInvoker = HostContext.CreateService<IProcessInvoker>())
+                    HostContext.WritePerfCounter($"SentJobToWorker_{requestId.ToString()}");
+
+                    try
                     {
-                        // Start the process channel.
-                        // It's OK if StartServer bubbles an execption after the worker process has already started.
-                        // The worker will shutdown after 30 seconds if it hasn't received the job message.
-                        processChannel.StartServer(
-                            // Delegate to start the child process.
-                            startProcess: (string pipeHandleOut, string pipeHandleIn) =>
+                        TaskResult resultOnAbandonOrCancel = TaskResult.Succeeded;
+                        // wait for renewlock, worker process or cancellation token been fired.
+                        var completedTask = await Task.WhenAny(renewJobRequest, workerProcessTask, Task.Delay(-1, jobRequestCancellationToken));
+                        if (completedTask == workerProcessTask)
+                        {
+                            // worker finished successfully, complete job request with result, attach unhandled exception reported by worker, stop renew lock, job has finished.
+                            int returnCode = await workerProcessTask;
+                            Trace.Info($"Worker finished for job {message.JobId}. Code: " + returnCode);
+
+                            string detailInfo = null;
+                            if (!TaskResultUtil.IsValidReturnCode(returnCode))
                             {
-                                // Validate args.
-                                ArgUtil.NotNullOrEmpty(pipeHandleOut, nameof(pipeHandleOut));
-                                ArgUtil.NotNullOrEmpty(pipeHandleIn, nameof(pipeHandleIn));
+                                detailInfo = string.Join(Environment.NewLine, workerOutput);
+                                Trace.Info($"Return code {returnCode} indicate worker encounter an unhandled exception or app crash, attach worker stdout/stderr to JobRequest result.");
+                                await LogWorkerProcessUnhandledException(message, detailInfo);
+                            }
 
-                                // Save STDOUT from worker, worker will use STDOUT report unhandle exception.
-                                processInvoker.OutputDataReceived += delegate (object sender, ProcessDataReceivedEventArgs stdout)
-                                    {
-                                        if (!string.IsNullOrEmpty(stdout.Data))
-                                        {
-                                            lock (_outputLock)
-                                            {
-                                                workerOutput.Add(stdout.Data);
-                                            }
-                                        }
-                                    };
+                            TaskResult result = TaskResultUtil.TranslateFromReturnCode(returnCode);
+                            Trace.Info($"finish job request for job {message.JobId} with result: {result}");
+                            term.WriteLine($"{DateTime.UtcNow:u}: Job {message.JobDisplayName} completed with result: {result}");
 
-                                // Save STDERR from worker, worker will use STDERR on crash.
-                                processInvoker.ErrorDataReceived += delegate (object sender, ProcessDataReceivedEventArgs stderr)
-                                    {
-                                        if (!string.IsNullOrEmpty(stderr.Data))
-                                        {
-                                            lock (_outputLock)
-                                            {
-                                                workerOutput.Add(stderr.Data);
-                                            }
-                                        }
-                                    };
+                            Trace.Info($"Stop renew job request for job {message.JobId}.");
+                            // stop renew lock
+                            lockRenewalTokenSource.Cancel();
+                            // renew job request should never blows up.
+                            await renewJobRequest;
 
-                                // Start the child process.
-                                HostContext.WritePerfCounter("StartingWorkerProcess");
-                                var assemblyDirectory = HostContext.GetDirectory(WellKnownDirectory.Bin);
-                                string workerFileName = Path.Combine(assemblyDirectory, _workerProcessName);
-                                workerProcessTask = processInvoker.ExecuteAsync(
-                                    workingDirectory: assemblyDirectory,
-                                    fileName: workerFileName,
-                                    arguments: "spawnclient " + pipeHandleOut + " " + pipeHandleIn,
-                                    environment: null,
-                                    requireExitCodeZero: false,
-                                    outputEncoding: null,
-                                    killProcessOnCancel: true,
-                                    redirectStandardIn: null,
-                                    inheritConsoleHandler: false,
-                                    keepStandardInOpen: false,
-                                    highPriorityProcess: true,
-                                    cancellationToken: workerProcessCancelTokenSource.Token);
-                            });
+                            // complete job request
+                            await CompleteJobRequestAsync(_poolId, message, lockToken, result, detailInfo);
 
-                        // Send the job request message.
-                        // Kill the worker process if sending the job message times out. The worker
-                        // process may have successfully received the job message.
+                            // print out unhandled exception happened in worker after we complete job request.
+                            // when we run out of disk space, report back to server has higher priority.
+                            if (!string.IsNullOrEmpty(detailInfo))
+                            {
+                                Trace.Error("Unhandled exception happened in worker:");
+                                Trace.Error(detailInfo);
+                            }
+
+                            return;
+                        }
+                        else if (completedTask == renewJobRequest)
+                        {
+                            resultOnAbandonOrCancel = TaskResult.Abandoned;
+                        }
+                        else
+                        {
+                            resultOnAbandonOrCancel = TaskResult.Canceled;
+                        }
+
+                        // renew job request completed or job request cancellation token been fired for RunAsync(jobrequestmessage)
+                        // cancel worker gracefully first, then kill it after worker cancel timeout
                         try
                         {
-                            Trace.Info($"Send job request message to worker for job {message.JobId}.");
-                            HostContext.WritePerfCounter($"RunnerSendingJobToWorker_{message.JobId}");
-                            using (var csSendJobRequest = new CancellationTokenSource(_channelTimeout))
+                            Trace.Info($"Send job cancellation message to worker for job {message.JobId}.");
+                            using (var csSendCancel = new CancellationTokenSource(_channelTimeout))
                             {
+                                var messageType = MessageType.CancelRequest;
+                                if (HostContext.RunnerShutdownToken.IsCancellationRequested)
+                                {
+                                    switch (HostContext.RunnerShutdownReason)
+                                    {
+                                        case ShutdownReason.UserCancelled:
+                                            messageType = MessageType.RunnerShutdown;
+                                            break;
+                                        case ShutdownReason.OperatingSystemShutdown:
+                                            messageType = MessageType.OperatingSystemShutdown;
+                                            break;
+                                    }
+                                }
+
                                 await processChannel.SendAsync(
-                                    messageType: MessageType.NewJobRequest,
-                                    body: JsonUtility.ToString(message),
-                                    cancellationToken: csSendJobRequest.Token);
+                                    messageType: messageType,
+                                    body: string.Empty,
+                                    cancellationToken: csSendCancel.Token);
                             }
                         }
                         catch (OperationCanceledException)
                         {
                             // message send been cancelled.
-                            // timeout 30 sec. kill worker.
-                            Trace.Info($"Job request message sending for job {message.JobId} been cancelled, kill running worker.");
+                            Trace.Info($"Job cancel message sending for job {message.JobId} been cancelled, kill running worker.");
                             workerProcessCancelTokenSource.Cancel();
                             try
                             {
@@ -445,166 +549,47 @@ namespace GitHub.Runner.Listener
                             {
                                 Trace.Info("worker process has been killed.");
                             }
-
-                            Trace.Info($"Stop renew job request for job {message.JobId}.");
-                            // stop renew lock
-                            lockRenewalTokenSource.Cancel();
-                            // renew job request should never blows up.
-                            await renewJobRequest;
-
-                            // not finish the job request since the job haven't run on worker at all, we will not going to set a result to server.
-                            return;
                         }
 
-                        // we get first jobrequest renew succeed and start the worker process with the job message.
-                        // send notification to machine provisioner.
-                        var systemConnection = message.Resources.Endpoints.SingleOrDefault(x => string.Equals(x.Name, WellKnownServiceEndpointNames.SystemVssConnection, StringComparison.OrdinalIgnoreCase));
-                        var accessToken = systemConnection?.Authorization?.Parameters["AccessToken"];
-                        notification.JobStarted(message.JobId, accessToken, systemConnection.Url);
+                        // wait worker to exit 
+                        // if worker doesn't exit within timeout, then kill worker.
+                        completedTask = await Task.WhenAny(workerProcessTask, Task.Delay(-1, workerCancelTimeoutKillToken));
 
-                        HostContext.WritePerfCounter($"SentJobToWorker_{requestId.ToString()}");
-
-                        try
+                        // worker haven't exit within cancellation timeout.
+                        if (completedTask != workerProcessTask)
                         {
-                            TaskResult resultOnAbandonOrCancel = TaskResult.Succeeded;
-                            // wait for renewlock, worker process or cancellation token been fired.
-                            var completedTask = await Task.WhenAny(renewJobRequest, workerProcessTask, Task.Delay(-1, jobRequestCancellationToken));
-                            if (completedTask == workerProcessTask)
-                            {
-                                // worker finished successfully, complete job request with result, attach unhandled exception reported by worker, stop renew lock, job has finished.
-                                int returnCode = await workerProcessTask;
-                                Trace.Info($"Worker finished for job {message.JobId}. Code: " + returnCode);
-
-                                string detailInfo = null;
-                                if (!TaskResultUtil.IsValidReturnCode(returnCode))
-                                {
-                                    detailInfo = string.Join(Environment.NewLine, workerOutput);
-                                    Trace.Info($"Return code {returnCode} indicate worker encounter an unhandled exception or app crash, attach worker stdout/stderr to JobRequest result.");
-                                    await LogWorkerProcessUnhandledException(message, detailInfo);
-                                }
-
-                                TaskResult result = TaskResultUtil.TranslateFromReturnCode(returnCode);
-                                Trace.Info($"finish job request for job {message.JobId} with result: {result}");
-                                term.WriteLine($"{DateTime.UtcNow:u}: Job {message.JobDisplayName} completed with result: {result}");
-
-                                Trace.Info($"Stop renew job request for job {message.JobId}.");
-                                // stop renew lock
-                                lockRenewalTokenSource.Cancel();
-                                // renew job request should never blows up.
-                                await renewJobRequest;
-
-                                // complete job request
-                                await CompleteJobRequestAsync(_poolId, message, lockToken, result, detailInfo);
-
-                                // print out unhandled exception happened in worker after we complete job request.
-                                // when we run out of disk space, report back to server has higher priority.
-                                if (!string.IsNullOrEmpty(detailInfo))
-                                {
-                                    Trace.Error("Unhandled exception happened in worker:");
-                                    Trace.Error(detailInfo);
-                                }
-
-                                return;
-                            }
-                            else if (completedTask == renewJobRequest)
-                            {
-                                resultOnAbandonOrCancel = TaskResult.Abandoned;
-                            }
-                            else
-                            {
-                                resultOnAbandonOrCancel = TaskResult.Canceled;
-                            }
-
-                            // renew job request completed or job request cancellation token been fired for RunAsync(jobrequestmessage)
-                            // cancel worker gracefully first, then kill it after worker cancel timeout
+                            Trace.Info($"worker process for job {message.JobId} haven't exit within cancellation timout, kill running worker.");
+                            workerProcessCancelTokenSource.Cancel();
                             try
                             {
-                                Trace.Info($"Send job cancellation message to worker for job {message.JobId}.");
-                                using (var csSendCancel = new CancellationTokenSource(_channelTimeout))
-                                {
-                                    var messageType = MessageType.CancelRequest;
-                                    if (HostContext.RunnerShutdownToken.IsCancellationRequested)
-                                    {
-                                        switch (HostContext.RunnerShutdownReason)
-                                        {
-                                            case ShutdownReason.UserCancelled:
-                                                messageType = MessageType.RunnerShutdown;
-                                                break;
-                                            case ShutdownReason.OperatingSystemShutdown:
-                                                messageType = MessageType.OperatingSystemShutdown;
-                                                break;
-                                        }
-                                    }
-
-                                    await processChannel.SendAsync(
-                                        messageType: messageType,
-                                        body: string.Empty,
-                                        cancellationToken: csSendCancel.Token);
-                                }
+                                await workerProcessTask;
                             }
                             catch (OperationCanceledException)
                             {
-                                // message send been cancelled.
-                                Trace.Info($"Job cancel message sending for job {message.JobId} been cancelled, kill running worker.");
-                                workerProcessCancelTokenSource.Cancel();
-                                try
-                                {
-                                    await workerProcessTask;
-                                }
-                                catch (OperationCanceledException)
-                                {
-                                    Trace.Info("worker process has been killed.");
-                                }
+                                Trace.Info("worker process has been killed.");
                             }
-
-                            // wait worker to exit 
-                            // if worker doesn't exit within timeout, then kill worker.
-                            completedTask = await Task.WhenAny(workerProcessTask, Task.Delay(-1, workerCancelTimeoutKillToken));
-
-                            // worker haven't exit within cancellation timeout.
-                            if (completedTask != workerProcessTask)
-                            {
-                                Trace.Info($"worker process for job {message.JobId} haven't exit within cancellation timout, kill running worker.");
-                                workerProcessCancelTokenSource.Cancel();
-                                try
-                                {
-                                    await workerProcessTask;
-                                }
-                                catch (OperationCanceledException)
-                                {
-                                    Trace.Info("worker process has been killed.");
-                                }
-
-                                // When worker doesn't exit within cancel timeout, the runner will kill the worker process and worker won't finish upload job logs.
-                                // The runner will try to upload these logs at this time.
-                                await TryUploadUnfinishedLogs(message);
-                            }
-
-                            Trace.Info($"finish job request for job {message.JobId} with result: {resultOnAbandonOrCancel}");
-                            term.WriteLine($"{DateTime.UtcNow:u}: Job {message.JobDisplayName} completed with result: {resultOnAbandonOrCancel}");
-                            // complete job request with cancel result, stop renew lock, job has finished.
-
-                            Trace.Info($"Stop renew job request for job {message.JobId}.");
-                            // stop renew lock
-                            lockRenewalTokenSource.Cancel();
-                            // renew job request should never blows up.
-                            await renewJobRequest;
-
-                            // complete job request
-                            await CompleteJobRequestAsync(_poolId, message, lockToken, resultOnAbandonOrCancel);
-                        }
-                        finally
-                        {
-                            // This should be the last thing to run so we don't notify external parties until actually finished
-                            await notification.JobCompleted(message.JobId);
                         }
+
+                        Trace.Info($"finish job request for job {message.JobId} with result: {resultOnAbandonOrCancel}");
+                        term.WriteLine($"{DateTime.UtcNow:u}: Job {message.JobDisplayName} completed with result: {resultOnAbandonOrCancel}");
+                        // complete job request with cancel result, stop renew lock, job has finished.
+
+                        Trace.Info($"Stop renew job request for job {message.JobId}.");
+                        // stop renew lock
+                        lockRenewalTokenSource.Cancel();
+                        // renew job request should never blows up.
+                        await renewJobRequest;
+
+                        // complete job request
+                        await CompleteJobRequestAsync(_poolId, message, lockToken, resultOnAbandonOrCancel);
+                    }
+                    finally
+                    {
+                        // This should be the last thing to run so we don't notify external parties until actually finished
+                        await notification.JobCompleted(message.JobId);
                     }
                 }
             }
-            finally
-            {
-                Busy = false;
-            }
         }
 
         public async Task RenewJobRequestAsync(int poolId, long requestId, Guid lockToken, TaskCompletionSource<int> firstJobRequestRenewed, CancellationToken token)
@@ -727,121 +712,6 @@ namespace GitHub.Runner.Listener
             }
         }
 
-        // Best effort upload any logs for this job.
-        private async Task TryUploadUnfinishedLogs(Pipelines.AgentJobRequestMessage message)
-        {
-            Trace.Entering();
-
-            var logFolder = Path.Combine(HostContext.GetDirectory(WellKnownDirectory.Diag), PagingLogger.PagingFolder);
-            if (!Directory.Exists(logFolder))
-            {
-                return;
-            }
-
-            var logs = Directory.GetFiles(logFolder);
-            if (logs.Length == 0)
-            {
-                return;
-            }
-
-            try
-            {
-                var systemConnection = message.Resources.Endpoints.SingleOrDefault(x => string.Equals(x.Name, WellKnownServiceEndpointNames.SystemVssConnection));
-                ArgUtil.NotNull(systemConnection, nameof(systemConnection));
-
-                var jobServer = HostContext.GetService<IJobServer>();
-                VssCredentials jobServerCredential = VssUtil.GetVssCredential(systemConnection);
-                VssConnection jobConnection = VssUtil.CreateConnection(systemConnection.Url, jobServerCredential);
-
-                await jobServer.ConnectAsync(jobConnection);
-
-                var timeline = await jobServer.GetTimelineAsync(message.Plan.ScopeIdentifier, message.Plan.PlanType, message.Plan.PlanId, message.Timeline.Id, CancellationToken.None);
-
-                var updatedRecords = new List<TimelineRecord>();
-                var logPages = new Dictionary<Guid, Dictionary<int, string>>();
-                var logRecords = new Dictionary<Guid, TimelineRecord>();
-                foreach (var log in logs)
-                {
-                    var logName = Path.GetFileNameWithoutExtension(log);
-                    var logNameParts = logName.Split('_', StringSplitOptions.RemoveEmptyEntries);
-                    if (logNameParts.Length != 3)
-                    {
-                        Trace.Warning($"log file '{log}' doesn't follow naming convension 'GUID_GUID_INT'.");
-                        continue;
-                    }
-                    var logPageSeperator = logName.IndexOf('_');
-                    var logRecordId = Guid.Empty;
-                    var pageNumber = 0;
-
-                    if (!Guid.TryParse(logNameParts[0], out Guid timelineId) || timelineId != timeline.Id)
-                    {
-                        Trace.Warning($"log file '{log}' is not belongs to current job");
-                        continue;
-                    }
-
-                    if (!Guid.TryParse(logNameParts[1], out logRecordId))
-                    {
-                        Trace.Warning($"log file '{log}' doesn't follow naming convension 'GUID_GUID_INT'.");
-                        continue;
-                    }
-
-                    if (!int.TryParse(logNameParts[2], out pageNumber))
-                    {
-                        Trace.Warning($"log file '{log}' doesn't follow naming convension 'GUID_GUID_INT'.");
-                        continue;
-                    }
-
-                    var record = timeline.Records.FirstOrDefault(x => x.Id == logRecordId);
-                    if (record != null)
-                    {
-                        if (!logPages.ContainsKey(record.Id))
-                        {
-                            logPages[record.Id] = new Dictionary<int, string>();
-                            logRecords[record.Id] = record;
-                        }
-
-                        logPages[record.Id][pageNumber] = log;
-                    }
-                }
-
-                foreach (var pages in logPages)
-                {
-                    var record = logRecords[pages.Key];
-                    if (record.Log == null)
-                    {
-                        // Create the log
-                        record.Log = await jobServer.CreateLogAsync(message.Plan.ScopeIdentifier, message.Plan.PlanType, message.Plan.PlanId, new TaskLog(String.Format(@"logs\{0:D}", record.Id)), default(CancellationToken));
-
-                        // Need to post timeline record updates to reflect the log creation
-                        updatedRecords.Add(record.Clone());
-                    }
-
-                    for (var i = 1; i <= pages.Value.Count; i++)
-                    {
-                        var logFile = pages.Value[i];
-                        // Upload the contents
-                        using (FileStream fs = File.Open(logFile, FileMode.Open, FileAccess.Read, FileShare.ReadWrite))
-                        {
-                            var logUploaded = await jobServer.AppendLogContentAsync(message.Plan.ScopeIdentifier, message.Plan.PlanType, message.Plan.PlanId, record.Log.Id, fs, default(CancellationToken));
-                        }
-
-                        Trace.Info($"Uploaded unfinished log '{logFile}' for current job.");
-                        IOUtil.DeleteFile(logFile);
-                    }
-                }
-
-                if (updatedRecords.Count > 0)
-                {
-                    await jobServer.UpdateTimelineRecordsAsync(message.Plan.ScopeIdentifier, message.Plan.PlanType, message.Plan.PlanId, message.Timeline.Id, updatedRecords, CancellationToken.None);
-                }
-            }
-            catch (Exception ex)
-            {
-                // Ignore any error during log upload since it's best effort
-                Trace.Error(ex);
-            }
-        }
-
         // TODO: We need send detailInfo back to DT in order to add an issue for the job
         private async Task CompleteJobRequestAsync(int poolId, Pipelines.AgentJobRequestMessage message, Guid lockToken, TaskResult result, string detailInfo = null)
         {

src/Runner.Listener/MessageListener.cs

@@ -13,10 +13,7 @@ using System.Diagnostics;
 using System.Runtime.InteropServices;
 using GitHub.Runner.Common;
 using GitHub.Runner.Sdk;
-using GitHub.Services.WebApi;
-using System.Runtime.CompilerServices;
 
-[assembly: InternalsVisibleTo("Test")]
 namespace GitHub.Runner.Listener
 {
     [ServiceLocator(Default = typeof(MessageListener))]
@@ -35,30 +32,18 @@ namespace GitHub.Runner.Listener
         private ITerminal _term;
         private IRunnerServer _runnerServer;
         private TaskAgentSession _session;
-        private ICredentialManager _credMgr;
-        private IConfigurationStore _configStore;
         private TimeSpan _getNextMessageRetryInterval;
         private readonly TimeSpan _sessionCreationRetryInterval = TimeSpan.FromSeconds(30);
         private readonly TimeSpan _sessionConflictRetryLimit = TimeSpan.FromMinutes(4);
         private readonly TimeSpan _clockSkewRetryLimit = TimeSpan.FromMinutes(30);
         private readonly Dictionary<string, int> _sessionCreationExceptionTracker = new Dictionary<string, int>();
 
-        // Whether load credentials from .credentials_migrated file
-        internal bool _useMigratedCredentials;
-
-        // need to check auth url if there is only .credentials and auth schema is OAuth
-        internal bool _needToCheckAuthorizationUrlUpdate;
-        internal Task<VssCredentials> _authorizationUrlMigrationBackgroundTask;
-        internal Task _authorizationUrlRollbackReattemptDelayBackgroundTask;
-
         public override void Initialize(IHostContext hostContext)
         {
             base.Initialize(hostContext);
 
             _term = HostContext.GetService<ITerminal>();
             _runnerServer = HostContext.GetService<IRunnerServer>();
-            _credMgr = HostContext.GetService<ICredentialManager>();
-            _configStore = HostContext.GetService<IConfigurationStore>();
         }
 
         public async Task<Boolean> CreateSessionAsync(CancellationToken token)
@@ -73,8 +58,8 @@ namespace GitHub.Runner.Listener
 
             // Create connection.
             Trace.Info("Loading Credentials");
-            _useMigratedCredentials = !StringUtil.ConvertToBoolean(Environment.GetEnvironmentVariable("GITHUB_ACTIONS_RUNNER_SPSAUTHURL"));
-            VssCredentials creds = _credMgr.LoadCredentials(_useMigratedCredentials);
+            var credMgr = HostContext.GetService<ICredentialManager>();
+            VssCredentials creds = credMgr.LoadCredentials();
 
             var agent = new TaskAgentReference
             {
@@ -89,17 +74,6 @@ namespace GitHub.Runner.Listener
             string errorMessage = string.Empty;
             bool encounteringError = false;
 
-            var originalCreds = _configStore.GetCredentials();
-            var migratedCreds = _configStore.GetMigratedCredentials();
-            if (migratedCreds == null)
-            {
-                _useMigratedCredentials = false;
-                if (originalCreds.Scheme == Constants.Configuration.OAuth)
-                {
-                    _needToCheckAuthorizationUrlUpdate = true;
-                }
-            }
-
             while (true)
             {
                 token.ThrowIfCancellationRequested();
@@ -109,7 +83,7 @@ namespace GitHub.Runner.Listener
                     Trace.Info("Connecting to the Runner Server...");
                     await _runnerServer.ConnectAsync(new Uri(serverUrl), creds);
                     Trace.Info("VssConnection created");
-
+                    
                     _term.WriteLine();
                     _term.WriteSuccessMessage("Connected to GitHub");
                     _term.WriteLine();
@@ -127,12 +101,6 @@ namespace GitHub.Runner.Listener
                         encounteringError = false;
                     }
 
-                    if (_needToCheckAuthorizationUrlUpdate)
-                    {
-                        // start background task try to get new authorization url
-                        _authorizationUrlMigrationBackgroundTask = GetNewOAuthAuthorizationSetting(token);
-                    }
-
                     return true;
                 }
                 catch (OperationCanceledException) when (token.IsCancellationRequested)
@@ -152,21 +120,8 @@ namespace GitHub.Runner.Listener
 
                     if (!IsSessionCreationExceptionRetriable(ex))
                     {
-                        if (_useMigratedCredentials)
-                        {
-                            // migrated credentials might cause lose permission during permission check,
-                            // we will force to use original credential and try again
-                            _useMigratedCredentials = false;
-                            var reattemptBackoff = BackoffTimerHelper.GetRandomBackoff(TimeSpan.FromHours(24), TimeSpan.FromHours(36));
-                            _authorizationUrlRollbackReattemptDelayBackgroundTask = HostContext.Delay(reattemptBackoff, token); // retry migrated creds in 24-36 hours.
-                            creds = _credMgr.LoadCredentials(false);
-                            Trace.Error("Fallback to original credentials and try again.");
-                        }
-                        else
-                        {
-                            _term.WriteError($"Failed to create session. {ex.Message}");
-                            return false;
-                        }
+                        _term.WriteError($"Failed to create session. {ex.Message}");
+                        return false;
                     }
 
                     if (!encounteringError) //print the message only on the first error
@@ -227,51 +182,6 @@ namespace GitHub.Runner.Listener
                         encounteringError = false;
                         continuousError = 0;
                     }
-
-                    if (_needToCheckAuthorizationUrlUpdate &&
-                        _authorizationUrlMigrationBackgroundTask?.IsCompleted == true)
-                    {
-                        if (HostContext.GetService<IJobDispatcher>().Busy ||
-                            HostContext.GetService<ISelfUpdater>().Busy)
-                        {
-                            Trace.Info("Job or runner updates in progress, update credentials next time.");
-                        }
-                        else
-                        {
-                            try
-                            {
-                                var newCred = await _authorizationUrlMigrationBackgroundTask;
-                                await _runnerServer.ConnectAsync(new Uri(_settings.ServerUrl), newCred);
-                                Trace.Info("Updated connection to use migrated credential for next GetMessage call.");
-                                _useMigratedCredentials = true;
-                                _authorizationUrlMigrationBackgroundTask = null;
-                                _needToCheckAuthorizationUrlUpdate = false;
-                            }
-                            catch (Exception ex)
-                            {
-                                Trace.Error("Fail to refresh connection with new authorization url.");
-                                Trace.Error(ex);
-                            }
-                        }
-                    }
-
-                    if (_authorizationUrlRollbackReattemptDelayBackgroundTask?.IsCompleted == true)
-                    {
-                        try
-                        {
-                            // we rolled back to use original creds about 2 days before, now it's a good time to try migrated creds again.
-                            Trace.Info("Re-attempt to use migrated credential");
-                            var migratedCreds = _credMgr.LoadCredentials();
-                            await _runnerServer.ConnectAsync(new Uri(_settings.ServerUrl), migratedCreds);
-                            _useMigratedCredentials = true;
-                            _authorizationUrlRollbackReattemptDelayBackgroundTask = null;
-                        }
-                        catch (Exception ex)
-                        {
-                            Trace.Error("Fail to refresh connection with new authorization url on rollback reattempt.");
-                            Trace.Error(ex);
-                        }
-                    }
                 }
                 catch (OperationCanceledException) when (token.IsCancellationRequested)
                 {
@@ -295,21 +205,7 @@ namespace GitHub.Runner.Listener
                     }
                     else if (!IsGetNextMessageExceptionRetriable(ex))
                     {
-                        if (_useMigratedCredentials)
-                        {
-                            // migrated credentials might cause lose permission during permission check,
-                            // we will force to use original credential and try again
-                            _useMigratedCredentials = false;
-                            var reattemptBackoff = BackoffTimerHelper.GetRandomBackoff(TimeSpan.FromHours(24), TimeSpan.FromHours(36));
-                            _authorizationUrlRollbackReattemptDelayBackgroundTask = HostContext.Delay(reattemptBackoff, token); // retry migrated creds in 24-36 hours.
-                            var originalCreds = _credMgr.LoadCredentials(false);
-                            await _runnerServer.ConnectAsync(new Uri(_settings.ServerUrl), originalCreds);
-                            Trace.Error("Fallback to original credentials and try again.");
-                        }
-                        else
-                        {
-                            throw;
-                        }
+                        throw;
                     }
                     else
                     {
@@ -501,80 +397,5 @@ namespace GitHub.Runner.Listener
                 return true;
             }
         }
-
-        private async Task<VssCredentials> GetNewOAuthAuthorizationSetting(CancellationToken token)
-        {
-            Trace.Info("Start checking oauth authorization url update.");
-            while (true)
-            {
-                var backoff = BackoffTimerHelper.GetRandomBackoff(TimeSpan.FromMinutes(30), TimeSpan.FromMinutes(45));
-                await HostContext.Delay(backoff, token);
-
-                try
-                {
-                    var migratedAuthorizationUrl = await _runnerServer.GetRunnerAuthUrlAsync(_settings.PoolId, _settings.AgentId);
-                    if (!string.IsNullOrEmpty(migratedAuthorizationUrl))
-                    {
-                        var credData = _configStore.GetCredentials();
-                        var clientId = credData.Data.GetValueOrDefault("clientId", null);
-                        var currentAuthorizationUrl = credData.Data.GetValueOrDefault("authorizationUrl", null);
-                        Trace.Info($"Current authorization url: {currentAuthorizationUrl}, new authorization url: {migratedAuthorizationUrl}");
-
-                        if (string.Equals(currentAuthorizationUrl, migratedAuthorizationUrl, StringComparison.OrdinalIgnoreCase))
-                        {
-                            // We don't need to update credentials.
-                            Trace.Info("No needs to update authorization url");
-                            await Task.Delay(TimeSpan.FromMilliseconds(-1), token);
-                        }
-
-                        var keyManager = HostContext.GetService<IRSAKeyManager>();
-                        var signingCredentials = VssSigningCredentials.Create(() => keyManager.GetKey());
-
-                        var migratedClientCredential = new VssOAuthJwtBearerClientCredential(clientId, migratedAuthorizationUrl, signingCredentials);
-                        var migratedRunnerCredential = new VssOAuthCredential(new Uri(migratedAuthorizationUrl, UriKind.Absolute), VssOAuthGrant.ClientCredentials, migratedClientCredential);
-
-                        Trace.Info("Try connect service with Token Service OAuth endpoint.");
-                        var runnerServer = HostContext.CreateService<IRunnerServer>();
-                        await runnerServer.ConnectAsync(new Uri(_settings.ServerUrl), migratedRunnerCredential);
-                        await runnerServer.GetAgentPoolsAsync();
-                        Trace.Info($"Successfully connected service with new authorization url.");
-
-                        var migratedCredData = new CredentialData
-                        {
-                            Scheme = Constants.Configuration.OAuth,
-                            Data =
-                            {
-                                { "clientId", clientId },
-                                { "authorizationUrl", migratedAuthorizationUrl },
-                                { "oauthEndpointUrl", migratedAuthorizationUrl },
-                            },
-                        };
-
-                        _configStore.SaveMigratedCredential(migratedCredData);
-                        return migratedRunnerCredential;
-                    }
-                    else
-                    {
-                        Trace.Verbose("No authorization url updates");
-                    }
-                }
-                catch (Exception ex)
-                {
-                    Trace.Error("Fail to get/test new authorization url.");
-                    Trace.Error(ex);
-
-                    try
-                    {
-                        await _runnerServer.ReportRunnerAuthUrlErrorAsync(_settings.PoolId, _settings.AgentId, ex.ToString());
-                    }
-                    catch (Exception e)
-                    {
-                        // best effort
-                        Trace.Error("Fail to report the migration error");
-                        Trace.Error(e);
-                    }
-                }
-            }
-        }
     }
 }

src/Runner.Listener/Runner.cs

@@ -451,38 +451,16 @@ namespace GitHub.Runner.Listener
             ext = "sh";
 #endif
             _term.WriteLine($@"
-Commands:
- .{separator}config.{ext}         Configures the runner
- .{separator}config.{ext} remove  Unconfigures the runner
- .{separator}run.{ext}            Runs the runner interactively. Does not require any options.
+Commands:,
+ .{separator}config.{ext}          Configures the runner
+ .{separator}config.{ext} remove   Unconfigures the runner
+ .{separator}run.{ext}             Runs the runner interactively. Does not require any options.
 
 Options:
- --help     Prints the help for each command
  --version  Prints the runner version
  --commit   Prints the runner commit
-
-Config Options:
- --unattended     Disable interactive prompts for missing arguments. Defaults will be used for missing options
- --url string     Repository to add the runner to. Required if unattended
- --token string   Registration token. Required if unattended
- --name string    Name of the runner to configure (default {Environment.MachineName ?? "myrunner"})
- --work string    Relative runner work directory (default {Constants.Path.WorkDirectory})
- --replace        Replace any existing runner with the same name (default false)");
-#if OS_WINDOWS
-    _term.WriteLine($@" --runasservice   Run the runner as a service");
-    _term.WriteLine($@" --windowslogonaccount string   Account to run the service as. Requires runasservice");
-    _term.WriteLine($@" --windowslogonpassword string  Password for the service account. Requires runasservice");
-#endif
-    _term.WriteLine($@"
-Examples:
- Configure a runner non-interactively:
-  .{separator}config.{ext} --unattended --url <url> --token <token>
- Configure a runner non-interactively, replacing any existing runner with the same name:
-  .{separator}config.{ext} --unattended --url <url> --token <token> --replace [--name <name>]");
-#if OS_WINDOWS
-    _term.WriteLine($@" Configure a runner to run as a service:");
-    _term.WriteLine($@"  .{separator}config.{ext} --url <url> --token <token> --runasservice");
-#endif
+ --help     Prints the help for each command
+");
         }
     }
 }

src/Runner.Listener/SelfUpdater.cs

@@ -17,7 +17,6 @@ namespace GitHub.Runner.Listener
     [ServiceLocator(Default = typeof(SelfUpdater))]
     public interface ISelfUpdater : IRunnerService
     {
-        bool Busy { get; }
         Task<bool> SelfUpdate(AgentRefreshMessage updateMessage, IJobDispatcher jobDispatcher, bool restartInteractiveRunner, CancellationToken token);
     }
 
@@ -32,8 +31,6 @@ namespace GitHub.Runner.Listener
         private int _poolId;
         private int _agentId;
 
-        public bool Busy { get; private set; }
-
         public override void Initialize(IHostContext hostContext)
         {
             base.Initialize(hostContext);
@@ -48,60 +45,52 @@ namespace GitHub.Runner.Listener
 
         public async Task<bool> SelfUpdate(AgentRefreshMessage updateMessage, IJobDispatcher jobDispatcher, bool restartInteractiveRunner, CancellationToken token)
         {
-            Busy = true;
-            try
+            if (!await UpdateNeeded(updateMessage.TargetVersion, token))
             {
-                if (!await UpdateNeeded(updateMessage.TargetVersion, token))
-                {
-                    Trace.Info($"Can't find available update package.");
-                    return false;
-                }
+                Trace.Info($"Can't find available update package.");
+                return false;
+            }
 
-                Trace.Info($"An update is available.");
+            Trace.Info($"An update is available.");
 
-                // Print console line that warn user not shutdown runner.
-                await UpdateRunnerUpdateStateAsync("Runner update in progress, do not shutdown runner.");
-                await UpdateRunnerUpdateStateAsync($"Downloading {_targetPackage.Version} runner");
+            // Print console line that warn user not shutdown runner.
+            await UpdateRunnerUpdateStateAsync("Runner update in progress, do not shutdown runner.");
+            await UpdateRunnerUpdateStateAsync($"Downloading {_targetPackage.Version} runner");
 
-                await DownloadLatestRunner(token);
-                Trace.Info($"Download latest runner and unzip into runner root.");
+            await DownloadLatestRunner(token);
+            Trace.Info($"Download latest runner and unzip into runner root.");
 
-                // wait till all running job finish
-                await UpdateRunnerUpdateStateAsync("Waiting for current job finish running.");
+            // wait till all running job finish
+            await UpdateRunnerUpdateStateAsync("Waiting for current job finish running.");
 
-                await jobDispatcher.WaitAsync(token);
-                Trace.Info($"All running job has exited.");
+            await jobDispatcher.WaitAsync(token);
+            Trace.Info($"All running job has exited.");
 
-                // delete runner backup
-                DeletePreviousVersionRunnerBackup(token);
-                Trace.Info($"Delete old version runner backup.");
+            // delete runner backup
+            DeletePreviousVersionRunnerBackup(token);
+            Trace.Info($"Delete old version runner backup.");
 
-                // generate update script from template
-                await UpdateRunnerUpdateStateAsync("Generate and execute update script.");
+            // generate update script from template
+            await UpdateRunnerUpdateStateAsync("Generate and execute update script.");
 
-                string updateScript = GenerateUpdateScript(restartInteractiveRunner);
-                Trace.Info($"Generate update script into: {updateScript}");
+            string updateScript = GenerateUpdateScript(restartInteractiveRunner);
+            Trace.Info($"Generate update script into: {updateScript}");
 
-                // kick off update script
-                Process invokeScript = new Process();
+            // kick off update script
+            Process invokeScript = new Process();
 #if OS_WINDOWS
             invokeScript.StartInfo.FileName = WhichUtil.Which("cmd.exe", trace: Trace);
             invokeScript.StartInfo.Arguments = $"/c \"{updateScript}\"";
 #elif (OS_OSX || OS_LINUX)
-                invokeScript.StartInfo.FileName = WhichUtil.Which("bash", trace: Trace);
-                invokeScript.StartInfo.Arguments = $"\"{updateScript}\"";
+            invokeScript.StartInfo.FileName = WhichUtil.Which("bash", trace: Trace);
+            invokeScript.StartInfo.Arguments = $"\"{updateScript}\"";
 #endif
-                invokeScript.Start();
-                Trace.Info($"Update script start running");
+            invokeScript.Start();
+            Trace.Info($"Update script start running");
 
-                await UpdateRunnerUpdateStateAsync("Runner will exit shortly for update, should back online within 10 seconds.");
+            await UpdateRunnerUpdateStateAsync("Runner will exit shortly for update, should back online within 10 seconds.");
 
-                return true;
-            }
-            finally
-            {
-                Busy = false;
-            }
+            return true;
         }
 
         private async Task<bool> UpdateNeeded(string targetVersion, CancellationToken token)

src/Runner.Sdk/ProcessInvoker.cs

@@ -271,14 +271,6 @@ namespace GitHub.Runner.Sdk
             // Indicate GitHub Actions process.
             _proc.StartInfo.Environment["GITHUB_ACTIONS"] = "true";
 
-            // Set CI=true when no one else already set it.
-            // CI=true is common set in most CI provider in GitHub
-            if (!_proc.StartInfo.Environment.ContainsKey("CI") &&
-                Environment.GetEnvironmentVariable("CI") == null)
-            {
-                _proc.StartInfo.Environment["CI"] = "true";
-            }
-
             // Hook up the events.
             _proc.EnableRaisingEvents = true;
             _proc.Exited += ProcessExitedHandler;

src/Runner.Sdk/RunnerWebProxy.cs

@@ -21,7 +21,6 @@ namespace GitHub.Runner.Sdk
         private string _httpsProxyAddress;
         private string _httpsProxyUsername;
         private string _httpsProxyPassword;
-        private string _noProxyString;
 
         private readonly List<ByPassInfo> _noProxyList = new List<ByPassInfo>();
         private readonly HashSet<string> _noProxyUnique = new HashSet<string>(StringComparer.OrdinalIgnoreCase);
@@ -34,7 +33,6 @@ namespace GitHub.Runner.Sdk
         public string HttpsProxyAddress => _httpsProxyAddress;
         public string HttpsProxyUsername => _httpsProxyUsername;
         public string HttpsProxyPassword => _httpsProxyPassword;
-        public string NoProxyString => _noProxyString;
 
         public List<ByPassInfo> NoProxyList => _noProxyList;
 
@@ -73,10 +71,6 @@ namespace GitHub.Runner.Sdk
             {
                 _httpProxyAddress = proxyHttpUri.AbsoluteUri;
 
-                // Set both environment variables since there are tools support both casing (curl, wget) and tools support only one casing (docker)
-                Environment.SetEnvironmentVariable("HTTP_PROXY", _httpProxyAddress);
-                Environment.SetEnvironmentVariable("http_proxy", _httpProxyAddress);
-
                 // the proxy url looks like http://[user:pass@]127.0.0.1:8888
                 var userInfo = Uri.UnescapeDataString(proxyHttpUri.UserInfo).Split(':', 2, StringSplitOptions.RemoveEmptyEntries);
                 if (userInfo.Length == 2)
@@ -103,10 +97,6 @@ namespace GitHub.Runner.Sdk
             {
                 _httpsProxyAddress = proxyHttpsUri.AbsoluteUri;
 
-                // Set both environment variables since there are tools support both casing (curl, wget) and tools support only one casing (docker)
-                Environment.SetEnvironmentVariable("HTTPS_PROXY", _httpsProxyAddress);
-                Environment.SetEnvironmentVariable("https_proxy", _httpsProxyAddress);
-
                 // the proxy url looks like http://[user:pass@]127.0.0.1:8888
                 var userInfo = Uri.UnescapeDataString(proxyHttpsUri.UserInfo).Split(':', 2, StringSplitOptions.RemoveEmptyEntries);
                 if (userInfo.Length == 2)
@@ -131,12 +121,6 @@ namespace GitHub.Runner.Sdk
 
             if (!string.IsNullOrEmpty(noProxyList))
             {
-                _noProxyString = noProxyList;
-
-                // Set both environment variables since there are tools support both casing (curl, wget) and tools support only one casing (docker)
-                Environment.SetEnvironmentVariable("NO_PROXY", noProxyList);
-                Environment.SetEnvironmentVariable("no_proxy", noProxyList);
-
                 var noProxyListSplit = noProxyList.Split(',', StringSplitOptions.RemoveEmptyEntries);
                 foreach (string noProxy in noProxyListSplit)
                 {

src/Runner.Worker/ActionCommandManager.cs

@@ -1,7 +1,6 @@
 using GitHub.DistributedTask.Pipelines;
 using GitHub.DistributedTask.WebApi;
 using GitHub.Runner.Common.Util;
-using GitHub.Runner.Worker.Container;
 using System;
 using System.Collections.Generic;
 using System.IO;
@@ -16,14 +15,14 @@ namespace GitHub.Runner.Worker
     {
         void EnablePluginInternalCommand();
         void DisablePluginInternalCommand();
-        bool TryProcessCommand(IExecutionContext context, string input, ContainerInfo container);
+        bool TryProcessCommand(IExecutionContext context, string input);
     }
 
     public sealed class ActionCommandManager : RunnerService, IActionCommandManager
     {
         private const string _stopCommand = "stop-commands";
         private readonly Dictionary<string, IActionCommandExtension> _commandExtensions = new Dictionary<string, IActionCommandExtension>(StringComparer.OrdinalIgnoreCase);
-        private readonly HashSet<string> _registeredCommands = new HashSet<string>(StringComparer.OrdinalIgnoreCase);
+        private HashSet<string> _registeredCommands = new HashSet<string>(StringComparer.OrdinalIgnoreCase);
         private readonly object _commandSerializeLock = new object();
         private bool _stopProcessCommand = false;
         private string _stopToken = null;
@@ -59,7 +58,7 @@ namespace GitHub.Runner.Worker
             _registeredCommands.Remove("internal-set-repo-path");
         }
 
-        public bool TryProcessCommand(IExecutionContext context, string input, ContainerInfo container)
+        public bool TryProcessCommand(IExecutionContext context, string input)
         {
             if (string.IsNullOrEmpty(input))
             {
@@ -115,7 +114,7 @@ namespace GitHub.Runner.Worker
 
                         try
                         {
-                            extension.ProcessCommand(context, input, actionCommand, container);
+                            extension.ProcessCommand(context, input, actionCommand);
                         }
                         catch (Exception ex)
                         {
@@ -141,7 +140,7 @@ namespace GitHub.Runner.Worker
         string Command { get; }
         bool OmitEcho { get; }
 
-        void ProcessCommand(IExecutionContext context, string line, ActionCommand command, ContainerInfo container);
+        void ProcessCommand(IExecutionContext context, string line, ActionCommand command);
     }
 
     public sealed class InternalPluginSetRepoPathCommandExtension : RunnerService, IActionCommandExtension
@@ -151,7 +150,7 @@ namespace GitHub.Runner.Worker
 
         public Type ExtensionType => typeof(IActionCommandExtension);
 
-        public void ProcessCommand(IExecutionContext context, string line, ActionCommand command, ContainerInfo container)
+        public void ProcessCommand(IExecutionContext context, string line, ActionCommand command)
         {
             if (!command.Properties.TryGetValue(SetRepoPathCommandProperties.repoFullName, out string repoFullName) || string.IsNullOrEmpty(repoFullName))
             {
@@ -181,7 +180,7 @@ namespace GitHub.Runner.Worker
 
         public Type ExtensionType => typeof(IActionCommandExtension);
 
-        public void ProcessCommand(IExecutionContext context, string line, ActionCommand command, ContainerInfo container)
+        public void ProcessCommand(IExecutionContext context, string line, ActionCommand command)
         {
             if (!command.Properties.TryGetValue(SetEnvCommandProperties.Name, out string envName) || string.IsNullOrEmpty(envName))
             {
@@ -206,7 +205,7 @@ namespace GitHub.Runner.Worker
 
         public Type ExtensionType => typeof(IActionCommandExtension);
 
-        public void ProcessCommand(IExecutionContext context, string line, ActionCommand command, ContainerInfo container)
+        public void ProcessCommand(IExecutionContext context, string line, ActionCommand command)
         {
             if (!command.Properties.TryGetValue(SetOutputCommandProperties.Name, out string outputName) || string.IsNullOrEmpty(outputName))
             {
@@ -230,7 +229,7 @@ namespace GitHub.Runner.Worker
 
         public Type ExtensionType => typeof(IActionCommandExtension);
 
-        public void ProcessCommand(IExecutionContext context, string line, ActionCommand command, ContainerInfo container)
+        public void ProcessCommand(IExecutionContext context, string line, ActionCommand command)
         {
             if (!command.Properties.TryGetValue(SaveStateCommandProperties.Name, out string stateName) || string.IsNullOrEmpty(stateName))
             {
@@ -254,7 +253,7 @@ namespace GitHub.Runner.Worker
 
         public Type ExtensionType => typeof(IActionCommandExtension);
 
-        public void ProcessCommand(IExecutionContext context, string line, ActionCommand command, ContainerInfo container)
+        public void ProcessCommand(IExecutionContext context, string line, ActionCommand command)
         {
             if (string.IsNullOrWhiteSpace(command.Data))
             {
@@ -280,7 +279,7 @@ namespace GitHub.Runner.Worker
 
         public Type ExtensionType => typeof(IActionCommandExtension);
 
-        public void ProcessCommand(IExecutionContext context, string line, ActionCommand command, ContainerInfo container)
+        public void ProcessCommand(IExecutionContext context, string line, ActionCommand command)
         {
             ArgUtil.NotNullOrEmpty(command.Data, "path");
             context.PrependPath.RemoveAll(x => string.Equals(x, command.Data, StringComparison.CurrentCulture));
@@ -295,7 +294,7 @@ namespace GitHub.Runner.Worker
 
         public Type ExtensionType => typeof(IActionCommandExtension);
 
-        public void ProcessCommand(IExecutionContext context, string line, ActionCommand command, ContainerInfo container)
+        public void ProcessCommand(IExecutionContext context, string line, ActionCommand command)
         {
             var file = command.Data;
 
@@ -307,9 +306,9 @@ namespace GitHub.Runner.Worker
             }
 
             // Translate file path back from container path
-            if (container != null)
+            if (context.Container != null)
             {
-                file = container.TranslateToHostPath(file);
+                file = context.Container.TranslateToHostPath(file);
             }
 
             // Root the path
@@ -342,7 +341,7 @@ namespace GitHub.Runner.Worker
 
         public Type ExtensionType => typeof(IActionCommandExtension);
 
-        public void ProcessCommand(IExecutionContext context, string line, ActionCommand command, ContainerInfo container)
+        public void ProcessCommand(IExecutionContext context, string line, ActionCommand command)
         {
             command.Properties.TryGetValue(RemoveMatcherCommandProperties.Owner, out string owner);
             var file = command.Data;
@@ -370,9 +369,9 @@ namespace GitHub.Runner.Worker
             else
             {
                 // Translate file path back from container path
-                if (container != null)
+                if (context.Container != null)
                 {
-                    file = container.TranslateToHostPath(file);
+                    file = context.Container.TranslateToHostPath(file);
                 }
 
                 // Root the path
@@ -410,7 +409,7 @@ namespace GitHub.Runner.Worker
 
         public Type ExtensionType => typeof(IActionCommandExtension);
 
-        public void ProcessCommand(IExecutionContext context, string inputLine, ActionCommand command, ContainerInfo container)
+        public void ProcessCommand(IExecutionContext context, string inputLine, ActionCommand command)
         {
             context.Debug(command.Data);
         }
@@ -438,7 +437,7 @@ namespace GitHub.Runner.Worker
 
         public Type ExtensionType => typeof(IActionCommandExtension);
 
-        public void ProcessCommand(IExecutionContext context, string inputLine, ActionCommand command, ContainerInfo container)
+        public void ProcessCommand(IExecutionContext context, string inputLine, ActionCommand command)
         {
             command.Properties.TryGetValue(IssueCommandProperties.File, out string file);
             command.Properties.TryGetValue(IssueCommandProperties.Line, out string line);
@@ -455,10 +454,10 @@ namespace GitHub.Runner.Worker
             {
                 issue.Category = "Code";
 
-                if (container != null)
+                if (context.Container != null)
                 {
                     // Translate file path back from container path
-                    file = container.TranslateToHostPath(file);
+                    file = context.Container.TranslateToHostPath(file);
                     command.Properties[IssueCommandProperties.File] = file;
                 }
 
@@ -518,7 +517,7 @@ namespace GitHub.Runner.Worker
 
         public Type ExtensionType => typeof(IActionCommandExtension);
 
-        public void ProcessCommand(IExecutionContext context, string line, ActionCommand command, ContainerInfo container)
+        public void ProcessCommand(IExecutionContext context, string line, ActionCommand command)
         {
             var data = this is GroupCommandExtension ? command.Data : string.Empty;
             context.Output($"##[{Command}]{data}");
@@ -532,7 +531,7 @@ namespace GitHub.Runner.Worker
 
         public Type ExtensionType => typeof(IActionCommandExtension);
 
-        public void ProcessCommand(IExecutionContext context, string line, ActionCommand command, ContainerInfo container)
+        public void ProcessCommand(IExecutionContext context, string line, ActionCommand command)
         {
             ArgUtil.NotNullOrEmpty(command.Data, "value");
 

src/Runner.Worker/ActionManager.cs

@@ -33,7 +33,7 @@ namespace GitHub.Runner.Worker
     {
         private const int _defaultFileStreamBufferSize = 4096;
 
-        //81920 is the default used by System.IO.Stream.CopyTo and is under the large object heap threshold (85k).
+        //81920 is the default used by System.IO.Stream.CopyTo and is under the large object heap threshold (85k). 
         private const int _defaultCopyBufferSize = 81920;
 
         private readonly Dictionary<Guid, ContainerInfo> _cachedActionContainers = new Dictionary<Guid, ContainerInfo>();
@@ -58,14 +58,8 @@ namespace GitHub.Runner.Worker
                 executionContext.Warning("The 'PREVIEW_ACTION_TOKEN' secret is depreciated. Please remove it from the repository's secrets");
             }
 
-            // Clear the cache (for self-hosted runners)
-            // Note, temporarily avoid this step for the on-premises product, to avoid rate limiting.
-            var configurationStore = HostContext.GetService<IConfigurationStore>();
-            var isHostedServer = configurationStore.GetSettings().IsHostedServer;
-            if (isHostedServer)
-            {
-                IOUtil.DeleteDirectory(HostContext.GetDirectory(WellKnownDirectory.Actions), executionContext.CancellationToken);
-            }
+            // Clear the cache (local runner)
+            IOUtil.DeleteDirectory(HostContext.GetDirectory(WellKnownDirectory.Actions), executionContext.CancellationToken);
 
             foreach (var action in actions)
             {
@@ -204,21 +198,14 @@ namespace GitHub.Runner.Worker
                 Trace.Info($"Load action that reference repository from '{actionDirectory}'");
                 definition.Directory = actionDirectory;
 
-                string manifestFile = Path.Combine(actionDirectory, Constants.Path.ActionManifestYmlFile);
-                string manifestFileYaml = Path.Combine(actionDirectory, Constants.Path.ActionManifestYamlFile);
+                string manifestFile = Path.Combine(actionDirectory, "action.yml");
                 string dockerFile = Path.Combine(actionDirectory, "Dockerfile");
                 string dockerFileLowerCase = Path.Combine(actionDirectory, "dockerfile");
-                if (File.Exists(manifestFile) || File.Exists(manifestFileYaml))
+                if (File.Exists(manifestFile))
                 {
                     var manifestManager = HostContext.GetService<IActionManifestManager>();
-                    if (File.Exists(manifestFile))
-                    {
-                        definition.Data = manifestManager.Load(executionContext, manifestFile);
-                    }
-                    else
-                    {
-                        definition.Data = manifestManager.Load(executionContext, manifestFileYaml);
-                    }
+                    definition.Data = manifestManager.Load(executionContext, manifestFile);
+
                     Trace.Verbose($"Action friendly name: '{definition.Data.Name}'");
                     Trace.Verbose($"Action description: '{definition.Data.Description}'");
 
@@ -327,7 +314,7 @@ namespace GitHub.Runner.Worker
                 else
                 {
                     var fullPath = IOUtil.ResolvePath(actionDirectory, "."); // resolve full path without access filesystem.
-                    throw new NotSupportedException($"Can't find 'action.yml', 'action.yaml' or 'Dockerfile' under '{fullPath}'. Did you forget to run actions/checkout before running your local action?");
+                    throw new NotSupportedException($"Can't find 'action.yml' or 'Dockerfile' under '{fullPath}'. Did you forget to run actions/checkout before running your local action?");
                 }
             }
             else if (action.Reference.Type == Pipelines.ActionSourceType.Script)
@@ -454,8 +441,7 @@ namespace GitHub.Runner.Worker
             ArgUtil.NotNullOrEmpty(repositoryReference.Ref, nameof(repositoryReference.Ref));
 
             string destDirectory = Path.Combine(HostContext.GetDirectory(WellKnownDirectory.Actions), repositoryReference.Name.Replace(Path.AltDirectorySeparatorChar, Path.DirectorySeparatorChar), repositoryReference.Ref);
-            string watermarkFile = destDirectory + ".completed";
-            if (File.Exists(watermarkFile))
+            if (File.Exists(destDirectory + ".completed"))
             {
                 executionContext.Debug($"Action '{repositoryReference.Name}@{repositoryReference.Ref}' already downloaded at '{destDirectory}'.");
                 return;
@@ -491,7 +477,7 @@ namespace GitHub.Runner.Worker
 
                 int retryCount = 0;
 
-                // Allow up to 20 * 60s for any action to be downloaded from github graph.
+                // Allow up to 20 * 60s for any action to be downloaded from github graph. 
                 int timeoutSeconds = 20 * 60;
                 while (retryCount < 3)
                 {
@@ -505,33 +491,24 @@ namespace GitHub.Runner.Worker
                             using (var httpClientHandler = HostContext.CreateHttpClientHandler())
                             using (var httpClient = new HttpClient(httpClientHandler))
                             {
-                                var configurationStore = HostContext.GetService<IConfigurationStore>();
-                                var isHostedServer = configurationStore.GetSettings().IsHostedServer;
-                                if (isHostedServer)
+                                var authToken = Environment.GetEnvironmentVariable("_GITHUB_ACTION_TOKEN");
+                                if (string.IsNullOrEmpty(authToken))
                                 {
-                                    var authToken = Environment.GetEnvironmentVariable("_GITHUB_ACTION_TOKEN");
-                                    if (string.IsNullOrEmpty(authToken))
-                                    {
-                                        // TODO: Depreciate the PREVIEW_ACTION_TOKEN
-                                        authToken = executionContext.Variables.Get("PREVIEW_ACTION_TOKEN");
-                                    }
+                                    // TODO: Depreciate the PREVIEW_ACTION_TOKEN
+                                    authToken = executionContext.Variables.Get("PREVIEW_ACTION_TOKEN");
+                                }
 
-                                    if (!string.IsNullOrEmpty(authToken))
-                                    {
-                                        HostContext.SecretMasker.AddValue(authToken);
-                                        var base64EncodingToken = Convert.ToBase64String(Encoding.UTF8.GetBytes($"PAT:{authToken}"));
-                                        httpClient.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Basic", base64EncodingToken);
-                                    }
-                                    else
-                                    {
-                                        var accessToken = executionContext.GetGitHubContext("token");
-                                        var base64EncodingToken = Convert.ToBase64String(Encoding.UTF8.GetBytes($"x-access-token:{accessToken}"));
-                                        httpClient.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Basic", base64EncodingToken);
-                                    }
+                                if (!string.IsNullOrEmpty(authToken))
+                                {
+                                    HostContext.SecretMasker.AddValue(authToken);
+                                    var base64EncodingToken = Convert.ToBase64String(Encoding.UTF8.GetBytes($"PAT:{authToken}"));
+                                    httpClient.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Basic", base64EncodingToken);
                                 }
                                 else
                                 {
-                                    // Intentionally empty. Temporary for GHES alpha release, download from dotcom unauthenticated.
+                                    var accessToken = executionContext.GetGitHubContext("token");
+                                    var base64EncodingToken = Convert.ToBase64String(Encoding.UTF8.GetBytes($"x-access-token:{accessToken}"));
+                                    httpClient.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Basic", base64EncodingToken);
                                 }
 
                                 httpClient.DefaultRequestHeaders.UserAgent.Add(HostContext.UserAgent);
@@ -626,7 +603,7 @@ namespace GitHub.Runner.Worker
                 }
 
                 Trace.Verbose("Create watermark file indicate action download succeed.");
-                File.WriteAllText(watermarkFile, DateTime.UtcNow.ToString());
+                File.WriteAllText(destDirectory + ".completed", DateTime.UtcNow.ToString());
 
                 executionContext.Debug($"Archive '{archiveFile}' has been unzipped into '{destDirectory}'.");
                 Trace.Info("Finished getting action repository.");
@@ -678,21 +655,12 @@ namespace GitHub.Runner.Worker
             // find the docker file or action.yml file
             var dockerFile = Path.Combine(actionEntryDirectory, "Dockerfile");
             var dockerFileLowerCase = Path.Combine(actionEntryDirectory, "dockerfile");
-            var actionManifest = Path.Combine(actionEntryDirectory, Constants.Path.ActionManifestYmlFile);
-            var actionManifestYaml = Path.Combine(actionEntryDirectory, Constants.Path.ActionManifestYamlFile);
-            if (File.Exists(actionManifest) || File.Exists(actionManifestYaml))
+            var actionManifest = Path.Combine(actionEntryDirectory, "action.yml");
+            if (File.Exists(actionManifest))
             {
                 executionContext.Debug($"action.yml for action: '{actionManifest}'.");
                 var manifestManager = HostContext.GetService<IActionManifestManager>();
-                ActionDefinitionData actionDefinitionData = null;
-                if (File.Exists(actionManifest))
-                {
-                    actionDefinitionData = manifestManager.Load(executionContext, actionManifest);
-                }
-                else
-                {
-                    actionDefinitionData = manifestManager.Load(executionContext, actionManifestYaml);
-                }
+                var actionDefinitionData = manifestManager.Load(executionContext, actionManifest);
 
                 if (actionDefinitionData.Execution.ExecutionType == ActionExecutionType.Container)
                 {
@@ -752,7 +720,7 @@ namespace GitHub.Runner.Worker
             else
             {
                 var fullPath = IOUtil.ResolvePath(actionEntryDirectory, "."); // resolve full path without access filesystem.
-                throw new InvalidOperationException($"Can't find 'action.yml', 'action.yaml' or 'Dockerfile' under '{fullPath}'. Did you forget to run actions/checkout before running your local action?");
+                throw new InvalidOperationException($"Can't find 'action.yml' or 'Dockerfile' under '{fullPath}'. Did you forget to run actions/checkout before running your local action?");
             }
         }
     }

src/Runner.Worker/ActionManifestManager.cs

@@ -22,17 +22,16 @@ namespace GitHub.Runner.Worker
     {
         ActionDefinitionData Load(IExecutionContext executionContext, string manifestFile);
 
-        List<string> EvaluateContainerArguments(IExecutionContext executionContext, SequenceToken token, IDictionary<string, PipelineContextData> extraExpressionValues);
+        List<string> EvaluateContainerArguments(IExecutionContext executionContext, SequenceToken token, IDictionary<string, PipelineContextData> contextData);
 
-        Dictionary<string, string> EvaluateContainerEnvironment(IExecutionContext executionContext, MappingToken token, IDictionary<string, PipelineContextData> extraExpressionValues);
+        Dictionary<string, string> EvaluateContainerEnvironment(IExecutionContext executionContext, MappingToken token, IDictionary<string, PipelineContextData> contextData);
 
-        string EvaluateDefaultInput(IExecutionContext executionContext, string inputName, TemplateToken token);
+        string EvaluateDefaultInput(IExecutionContext executionContext, string inputName, TemplateToken token, IDictionary<string, PipelineContextData> contextData);
     }
 
     public sealed class ActionManifestManager : RunnerService, IActionManifestManager
     {
         private TemplateSchema _actionManifestSchema;
-        private IReadOnlyList<String> _fileTable;
 
         public override void Initialize(IHostContext hostContext)
         {
@@ -54,7 +53,7 @@ namespace GitHub.Runner.Worker
 
         public ActionDefinitionData Load(IExecutionContext executionContext, string manifestFile)
         {
-            var context = CreateContext(executionContext);
+            var context = CreateContext(executionContext, null);
             ActionDefinitionData actionDefinition = new ActionDefinitionData();
             try
             {
@@ -62,9 +61,6 @@ namespace GitHub.Runner.Worker
 
                 // Get the file ID
                 var fileId = context.GetFileId(manifestFile);
-                _fileTable = context.GetFileTable();
-
-                // Read the file
                 var fileContent = File.ReadAllText(manifestFile);
                 using (var stringReader = new StringReader(fileContent))
                 {
@@ -120,7 +116,7 @@ namespace GitHub.Runner.Worker
             if (actionDefinition.Execution == null)
             {
                 executionContext.Debug($"Loaded action.yml file: {StringUtil.ConvertToJson(actionDefinition)}");
-                throw new ArgumentException($"Top level 'runs:' section is required for {manifestFile}");
+                throw new ArgumentException($"Top level 'run:' section is required for {manifestFile}");
             }
             else
             {
@@ -133,13 +129,13 @@ namespace GitHub.Runner.Worker
         public List<string> EvaluateContainerArguments(
             IExecutionContext executionContext,
             SequenceToken token,
-            IDictionary<string, PipelineContextData> extraExpressionValues)
+            IDictionary<string, PipelineContextData> contextData)
         {
             var result = new List<string>();
 
             if (token != null)
             {
-                var context = CreateContext(executionContext, extraExpressionValues);
+                var context = CreateContext(executionContext, contextData);
                 try
                 {
                     var evaluateResult = TemplateEvaluator.Evaluate(context, "container-runs-args", token, 0, null, omitHeader: true);
@@ -172,13 +168,13 @@ namespace GitHub.Runner.Worker
         public Dictionary<string, string> EvaluateContainerEnvironment(
             IExecutionContext executionContext,
             MappingToken token,
-            IDictionary<string, PipelineContextData> extraExpressionValues)
+            IDictionary<string, PipelineContextData> contextData)
         {
             var result = new Dictionary<string, string>(StringComparer.OrdinalIgnoreCase);
 
             if (token != null)
             {
-                var context = CreateContext(executionContext, extraExpressionValues);
+                var context = CreateContext(executionContext, contextData);
                 try
                 {
                     var evaluateResult = TemplateEvaluator.Evaluate(context, "container-runs-env", token, 0, null, omitHeader: true);
@@ -216,12 +212,13 @@ namespace GitHub.Runner.Worker
         public string EvaluateDefaultInput(
             IExecutionContext executionContext,
             string inputName,
-            TemplateToken token)
+            TemplateToken token,
+            IDictionary<string, PipelineContextData> contextData)
         {
             string result = "";
             if (token != null)
             {
-                var context = CreateContext(executionContext);
+                var context = CreateContext(executionContext, contextData);
                 try
                 {
                     var evaluateResult = TemplateEvaluator.Evaluate(context, "input-default-context", token, 0, null, omitHeader: true);
@@ -246,7 +243,7 @@ namespace GitHub.Runner.Worker
 
         private TemplateContext CreateContext(
             IExecutionContext executionContext,
-            IDictionary<string, PipelineContextData> extraExpressionValues = null)
+            IDictionary<string, PipelineContextData> contextData)
         {
             var result = new TemplateContext
             {
@@ -260,36 +257,14 @@ namespace GitHub.Runner.Worker
                 TraceWriter = executionContext.ToTemplateTraceWriter(),
             };
 
-            // Expression values from execution context
-            foreach (var pair in executionContext.ExpressionValues)
+            if (contextData?.Count > 0)
             {
-                result.ExpressionValues[pair.Key] = pair.Value;
-            }
-
-            // Extra expression values
-            if (extraExpressionValues?.Count > 0)
-            {
-                foreach (var pair in extraExpressionValues)
+                foreach (var pair in contextData)
                 {
                     result.ExpressionValues[pair.Key] = pair.Value;
                 }
             }
 
-            // Expression functions from execution context
-            foreach (var item in executionContext.ExpressionFunctions)
-            {
-                result.ExpressionFunctions.Add(item);
-            }
-
-            // Add the file table
-            if (_fileTable?.Count > 0)
-            {
-                for (var i = 0 ; i < _fileTable.Count ; i++)
-                {
-                    result.GetFileId(_fileTable[i]);
-                }
-            }
-
             return result;
         }
 
@@ -366,7 +341,7 @@ namespace GitHub.Runner.Worker
                             EntryPoint = entrypointToken?.Value,
                             Environment = envToken,
                             Cleanup = postEntrypointToken?.Value,
-                            CleanupCondition = postIfToken?.Value ?? "always()"
+                            CleanupCondition = postIfToken?.Value
                         };
                     }
                 }
@@ -382,7 +357,7 @@ namespace GitHub.Runner.Worker
                         {
                             Script = mainToken.Value,
                             Cleanup = postToken?.Value,
-                            CleanupCondition = postIfToken?.Value ?? "always()"
+                            CleanupCondition = postIfToken?.Value
                         };
                     }
                 }
@@ -440,5 +415,566 @@ namespace GitHub.Runner.Worker
             }
         }
     }
+
+    /// <summary>
+    /// Converts a YAML file into a TemplateToken
+    /// </summary>
+    internal sealed class YamlObjectReader : IObjectReader
+    {
+        internal YamlObjectReader(
+            Int32? fileId,
+            TextReader input)
+        {
+            m_fileId = fileId;
+            m_parser = new Parser(input);
+        }
+
+        public Boolean AllowLiteral(out LiteralToken value)
+        {
+            if (EvaluateCurrent() is Scalar scalar)
+            {
+                // Tag specified
+                if (!string.IsNullOrEmpty(scalar.Tag))
+                {
+                    // String tag
+                    if (string.Equals(scalar.Tag, c_stringTag, StringComparison.Ordinal))
+                    {
+                        value = new StringToken(m_fileId, scalar.Start.Line, scalar.Start.Column, scalar.Value);
+                        MoveNext();
+                        return true;
+                    }
+
+                    // Not plain style
+                    if (scalar.Style != ScalarStyle.Plain)
+                    {
+                        throw new NotSupportedException($"The scalar style '{scalar.Style}' on line {scalar.Start.Line} and column {scalar.Start.Column} is not valid with the tag '{scalar.Tag}'");
+                    }
+
+                    // Boolean, Float, Integer, or Null
+                    switch (scalar.Tag)
+                    {
+                        case c_booleanTag:
+                            value = ParseBoolean(scalar);
+                            break;
+                        case c_floatTag:
+                            value = ParseFloat(scalar);
+                            break;
+                        case c_integerTag:
+                            value = ParseInteger(scalar);
+                            break;
+                        case c_nullTag:
+                            value = ParseNull(scalar);
+                            break;
+                        default:
+                            throw new NotSupportedException($"Unexpected tag '{scalar.Tag}'");
+                    }
+
+                    MoveNext();
+                    return true;
+                }
+
+                // Plain style, determine type using YAML 1.2 "core" schema https://yaml.org/spec/1.2/spec.html#id2804923
+                if (scalar.Style == ScalarStyle.Plain)
+                {
+                    if (MatchNull(scalar, out var nullToken))
+                    {
+                        value = nullToken;
+                    }
+                    else if (MatchBoolean(scalar, out var booleanToken))
+                    {
+                        value = booleanToken;
+                    }
+                    else if (MatchInteger(scalar, out var numberToken) ||
+                        MatchFloat(scalar, out numberToken))
+                    {
+                        value = numberToken;
+                    }
+                    else
+                    {
+                        value = new StringToken(m_fileId, scalar.Start.Line, scalar.Start.Column, scalar.Value);
+                    }
+
+                    MoveNext();
+                    return true;
+                }
+
+                // Otherwise assume string
+                value = new StringToken(m_fileId, scalar.Start.Line, scalar.Start.Column, scalar.Value);
+                MoveNext();
+                return true;
+            }
+
+            value = default;
+            return false;
+        }
+
+        public Boolean AllowSequenceStart(out SequenceToken value)
+        {
+            if (EvaluateCurrent() is SequenceStart sequenceStart)
+            {
+                value = new SequenceToken(m_fileId, sequenceStart.Start.Line, sequenceStart.Start.Column);
+                MoveNext();
+                return true;
+            }
+
+            value = default;
+            return false;
+        }
+
+        public Boolean AllowSequenceEnd()
+        {
+            if (EvaluateCurrent() is SequenceEnd)
+            {
+                MoveNext();
+                return true;
+            }
+
+            return false;
+        }
+
+        public Boolean AllowMappingStart(out MappingToken value)
+        {
+            if (EvaluateCurrent() is MappingStart mappingStart)
+            {
+                value = new MappingToken(m_fileId, mappingStart.Start.Line, mappingStart.Start.Column);
+                MoveNext();
+                return true;
+            }
+
+            value = default;
+            return false;
+        }
+
+        public Boolean AllowMappingEnd()
+        {
+            if (EvaluateCurrent() is MappingEnd)
+            {
+                MoveNext();
+                return true;
+            }
+
+            return false;
+        }
+
+        /// <summary>
+        /// Consumes the last parsing events, which are expected to be DocumentEnd and StreamEnd.
+        /// </summary>
+        public void ValidateEnd()
+        {
+            if (EvaluateCurrent() is DocumentEnd)
+            {
+                MoveNext();
+            }
+            else
+            {
+                throw new InvalidOperationException("Expected document end parse event");
+            }
+
+            if (EvaluateCurrent() is StreamEnd)
+            {
+                MoveNext();
+            }
+            else
+            {
+                throw new InvalidOperationException("Expected stream end parse event");
+            }
+
+            if (MoveNext())
+            {
+                throw new InvalidOperationException("Expected end of parse events");
+            }
+        }
+
+        /// <summary>
+        /// Consumes the first parsing events, which are expected to be StreamStart and DocumentStart.
+        /// </summary>
+        public void ValidateStart()
+        {
+            if (EvaluateCurrent() != null)
+            {
+                throw new InvalidOperationException("Unexpected parser state");
+            }
+
+            if (!MoveNext())
+            {
+                throw new InvalidOperationException("Expected a parse event");
+            }
+
+            if (EvaluateCurrent() is StreamStart)
+            {
+                MoveNext();
+            }
+            else
+            {
+                throw new InvalidOperationException("Expected stream start parse event");
+            }
+
+            if (EvaluateCurrent() is DocumentStart)
+            {
+                MoveNext();
+            }
+            else
+            {
+                throw new InvalidOperationException("Expected document start parse event");
+            }
+        }
+
+        private ParsingEvent EvaluateCurrent()
+        {
+            if (m_current == null)
+            {
+                m_current = m_parser.Current;
+                if (m_current != null)
+                {
+                    if (m_current is Scalar scalar)
+                    {
+                        // Verify not using achors
+                        if (scalar.Anchor != null)
+                        {
+                            throw new InvalidOperationException($"Anchors are not currently supported. Remove the anchor '{scalar.Anchor}'");
+                        }
+                    }
+                    else if (m_current is MappingStart mappingStart)
+                    {
+                        // Verify not using achors
+                        if (mappingStart.Anchor != null)
+                        {
+                            throw new InvalidOperationException($"Anchors are not currently supported. Remove the anchor '{mappingStart.Anchor}'");
+                        }
+                    }
+                    else if (m_current is SequenceStart sequenceStart)
+                    {
+                        // Verify not using achors
+                        if (sequenceStart.Anchor != null)
+                        {
+                            throw new InvalidOperationException($"Anchors are not currently supported. Remove the anchor '{sequenceStart.Anchor}'");
+                        }
+                    }
+                    else if (!(m_current is MappingEnd) &&
+                        !(m_current is SequenceEnd) &&
+                        !(m_current is DocumentStart) &&
+                        !(m_current is DocumentEnd) &&
+                        !(m_current is StreamStart) &&
+                        !(m_current is StreamEnd))
+                    {
+                        throw new InvalidOperationException($"Unexpected parsing event type: {m_current.GetType().Name}");
+                    }
+                }
+            }
+
+            return m_current;
+        }
+
+        private Boolean MoveNext()
+        {
+            m_current = null;
+            return m_parser.MoveNext();
+        }
+
+        private BooleanToken ParseBoolean(Scalar scalar)
+        {
+            if (MatchBoolean(scalar, out var token))
+            {
+                return token;
+            }
+
+            ThrowInvalidValue(scalar, c_booleanTag); // throws
+            return default;
+        }
+
+        private NumberToken ParseFloat(Scalar scalar)
+        {
+            if (MatchFloat(scalar, out var token))
+            {
+                return token;
+            }
+
+            ThrowInvalidValue(scalar, c_floatTag); // throws
+            return default;
+        }
+
+        private NumberToken ParseInteger(Scalar scalar)
+        {
+            if (MatchInteger(scalar, out var token))
+            {
+                return token;
+            }
+
+            ThrowInvalidValue(scalar, c_integerTag); // throws
+            return default;
+        }
+
+        private NullToken ParseNull(Scalar scalar)
+        {
+            if (MatchNull(scalar, out var token))
+            {
+                return token;
+            }
+
+            ThrowInvalidValue(scalar, c_nullTag); // throws
+            return default;
+        }
+
+        private Boolean MatchBoolean(
+            Scalar scalar,
+            out BooleanToken value)
+        {
+            // YAML 1.2 "core" schema https://yaml.org/spec/1.2/spec.html#id2804923
+            switch (scalar.Value ?? string.Empty)
+            {
+                case "true":
+                case "True":
+                case "TRUE":
+                    value = new BooleanToken(m_fileId, scalar.Start.Line, scalar.Start.Column, true);
+                    return true;
+                case "false":
+                case "False":
+                case "FALSE":
+                    value = new BooleanToken(m_fileId, scalar.Start.Line, scalar.Start.Column, false);
+                    return true;
+            }
+
+            value = default;
+            return false;
+        }
+
+        private Boolean MatchFloat(
+            Scalar scalar,
+            out NumberToken value)
+        {
+            // YAML 1.2 "core" schema https://yaml.org/spec/1.2/spec.html#id2804923
+            var str = scalar.Value;
+            if (!string.IsNullOrEmpty(str))
+            {
+                // Check for [-+]?(\.inf|\.Inf|\.INF)|\.nan|\.NaN|\.NAN
+                switch (str)
+                {
+                    case ".inf":
+                    case ".Inf":
+                    case ".INF":
+                    case "+.inf":
+                    case "+.Inf":
+                    case "+.INF":
+                        value = new NumberToken(m_fileId, scalar.Start.Line, scalar.Start.Column, Double.PositiveInfinity);
+                        return true;
+                    case "-.inf":
+                    case "-.Inf":
+                    case "-.INF":
+                        value = new NumberToken(m_fileId, scalar.Start.Line, scalar.Start.Column, Double.NegativeInfinity);
+                        return true;
+                    case ".nan":
+                    case ".NaN":
+                    case ".NAN":
+                        value = new NumberToken(m_fileId, scalar.Start.Line, scalar.Start.Column, Double.NaN);
+                        return true;
+                }
+
+                // Otherwise check [-+]?(\.[0-9]+|[0-9]+(\.[0-9]*)?)([eE][-+]?[0-9]+)?
+
+                // Skip leading sign
+                var index = str[0] == '-' || str[0] == '+' ? 1 : 0;
+
+                // Check for integer portion
+                var length = str.Length;
+                var hasInteger = false;
+                while (index < length && str[index] >= '0' && str[index] <= '9')
+                {
+                    hasInteger = true;
+                    index++;
+                }
+
+                // Check for decimal point
+                var hasDot = false;
+                if (index < length && str[index] == '.')
+                {
+                    hasDot = true;
+                    index++;
+                }
+
+                // Check for decimal portion
+                var hasDecimal = false;
+                while (index < length && str[index] >= '0' && str[index] <= '9')
+                {
+                    hasDecimal = true;
+                    index++;
+                }
+
+                // Check [-+]?(\.[0-9]+|[0-9]+(\.[0-9]*)?)
+                if ((hasDot && hasDecimal) || hasInteger)
+                {
+                    // Check for end
+                    if (index == length)
+                    {
+                        // Try parse
+                        if (Double.TryParse(str, NumberStyles.AllowLeadingSign | NumberStyles.AllowDecimalPoint, CultureInfo.InvariantCulture, out var doubleValue))
+                        {
+                            value = new NumberToken(m_fileId, scalar.Start.Line, scalar.Start.Column, doubleValue);
+                            return true;
+                        }
+                        // Otherwise exceeds range
+                        else
+                        {
+                            ThrowInvalidValue(scalar, c_floatTag); // throws
+                        }
+                    }
+                    // Check [eE][-+]?[0-9]
+                    else if (index < length && (str[index] == 'e' || str[index] == 'E'))
+                    {
+                        index++;
+
+                        // Skip sign
+                        if (index < length && (str[index] == '-' || str[index] == '+'))
+                        {
+                            index++;
+                        }
+
+                        // Check for exponent
+                        var hasExponent = false;
+                        while (index < length && str[index] >= '0' && str[index] <= '9')
+                        {
+                            hasExponent = true;
+                            index++;
+                        }
+
+                        // Check for end
+                        if (hasExponent && index == length)
+                        {
+                            // Try parse
+                            if (Double.TryParse(str, NumberStyles.AllowLeadingSign | NumberStyles.AllowDecimalPoint | NumberStyles.AllowExponent, CultureInfo.InvariantCulture, out var doubleValue))
+                            {
+                                value = new NumberToken(m_fileId, scalar.Start.Line, scalar.Start.Column, (Double)doubleValue);
+                                return true;
+                            }
+                            // Otherwise exceeds range
+                            else
+                            {
+                                ThrowInvalidValue(scalar, c_floatTag); // throws
+                            }
+                        }
+                    }
+                }
+            }
+
+            value = default;
+            return false;
+        }
+
+        private Boolean MatchInteger(
+            Scalar scalar,
+            out NumberToken value)
+        {
+            // YAML 1.2 "core" schema https://yaml.org/spec/1.2/spec.html#id2804923
+            var str = scalar.Value;
+            if (!string.IsNullOrEmpty(str))
+            {
+                // Check for [0-9]+
+                var firstChar = str[0];
+                if (firstChar >= '0' && firstChar <= '9' &&
+                    str.Skip(1).All(x => x >= '0' && x <= '9'))
+                {
+                    // Try parse
+                    if (Double.TryParse(str, NumberStyles.None, CultureInfo.InvariantCulture, out var doubleValue))
+                    {
+                        value = new NumberToken(m_fileId, scalar.Start.Line, scalar.Start.Column, doubleValue);
+                        return true;
+                    }
+
+                    // Otherwise exceeds range
+                    ThrowInvalidValue(scalar, c_integerTag); // throws
+                }
+                // Check for (-|+)[0-9]+
+                else if ((firstChar == '-' || firstChar == '+') &&
+                    str.Length > 1 &&
+                    str.Skip(1).All(x => x >= '0' && x <= '9'))
+                {
+                    // Try parse
+                    if (Double.TryParse(str, NumberStyles.AllowLeadingSign, CultureInfo.InvariantCulture, out var doubleValue))
+                    {
+                        value = new NumberToken(m_fileId, scalar.Start.Line, scalar.Start.Column, doubleValue);
+                        return true;
+                    }
+
+                    // Otherwise exceeds range
+                    ThrowInvalidValue(scalar, c_integerTag); // throws
+                }
+                // Check for 0x[0-9a-fA-F]+
+                else if (firstChar == '0' &&
+                    str.Length > 2 &&
+                    str[1] == 'x' &&
+                    str.Skip(2).All(x => (x >= '0' && x <= '9') || (x >= 'a' && x <= 'f') || (x >= 'A' && x <= 'F')))
+                {
+                    // Try parse
+                    if (Int32.TryParse(str.Substring(2), NumberStyles.AllowHexSpecifier, CultureInfo.InvariantCulture, out var integerValue))
+                    {
+                        value = new NumberToken(m_fileId, scalar.Start.Line, scalar.Start.Column, integerValue);
+                        return true;
+                    }
+
+                    // Otherwise exceeds range
+                    ThrowInvalidValue(scalar, c_integerTag); // throws
+                }
+                // Check for 0o[0-9]+
+                else if (firstChar == '0' &&
+                    str.Length > 2 &&
+                    str[1] == 'o' &&
+                    str.Skip(2).All(x => x >= '0' && x <= '7'))
+                {
+                    // Try parse
+                    var integerValue = default(Int32);
+                    try
+                    {
+                        integerValue = Convert.ToInt32(str.Substring(2), 8);
+                    }
+                    // Otherwise exceeds range
+                    catch (Exception)
+                    {
+                        ThrowInvalidValue(scalar, c_integerTag); // throws
+                    }
+
+                    value = new NumberToken(m_fileId, scalar.Start.Line, scalar.Start.Column, integerValue);
+                    return true;
+                }
+            }
+
+            value = default;
+            return false;
+        }
+
+        private Boolean MatchNull(
+            Scalar scalar,
+            out NullToken value)
+        {
+            // YAML 1.2 "core" schema https://yaml.org/spec/1.2/spec.html#id2804923
+            switch (scalar.Value ?? string.Empty)
+            {
+                case "":
+                case "null":
+                case "Null":
+                case "NULL":
+                case "~":
+                    value = new NullToken(m_fileId, scalar.Start.Line, scalar.Start.Column);
+                    return true;
+            }
+
+            value = default;
+            return false;
+        }
+
+        private void ThrowInvalidValue(
+            Scalar scalar,
+            String tag)
+        {
+            throw new NotSupportedException($"The value '{scalar.Value}' on line {scalar.Start.Line} and column {scalar.Start.Column} is invalid for the type '{scalar.Tag}'");
+        }
+
+        private const String c_booleanTag = "tag:yaml.org,2002:bool";
+        private const String c_floatTag = "tag:yaml.org,2002:float";
+        private const String c_integerTag = "tag:yaml.org,2002:int";
+        private const String c_nullTag = "tag:yaml.org,2002:null";
+        private const String c_stringTag = "tag:yaml.org,2002:string";
+        private readonly Int32? m_fileId;
+        private readonly Parser m_parser;
+        private ParsingEvent m_current;
+    }
 }
 

src/Runner.Worker/ActionRunner.cs

@@ -26,7 +26,7 @@ namespace GitHub.Runner.Worker
     public interface IActionRunner : IStep, IRunnerService
     {
         ActionRunStage Stage { get; set; }
-        bool TryEvaluateDisplayName(DictionaryContextData contextData, IExecutionContext context);
+        Boolean TryEvaluateDisplayName(DictionaryContextData contextData, IExecutionContext context);
         Pipelines.ActionStep Action { get; set; }
     }
 
@@ -141,8 +141,10 @@ namespace GitHub.Runner.Worker
 
             // Load the inputs.
             ExecutionContext.Debug("Loading inputs");
-            var templateEvaluator = ExecutionContext.ToPipelineTemplateEvaluator();
-            var inputs = templateEvaluator.EvaluateStepInputs(Action.Inputs, ExecutionContext.ExpressionValues, ExecutionContext.ExpressionFunctions);
+            var templateTrace = ExecutionContext.ToTemplateTraceWriter();
+            var schema = new PipelineTemplateSchemaFactory().CreateSchema();
+            var templateEvaluator = new PipelineTemplateEvaluator(templateTrace, schema);
+            var inputs = templateEvaluator.EvaluateStepInputs(Action.Inputs, ExecutionContext.ExpressionValues);
 
             foreach (KeyValuePair<string, string> input in inputs)
             {
@@ -162,7 +164,13 @@ namespace GitHub.Runner.Worker
                     string key = input.Key.AssertString("action input name").Value;
                     if (!inputs.ContainsKey(key))
                     {
-                        inputs[key] = manifestManager.EvaluateDefaultInput(ExecutionContext, key, input.Value);
+                        var evaluateContext = new Dictionary<string, PipelineContextData>(StringComparer.OrdinalIgnoreCase);
+                        foreach (var data in ExecutionContext.ExpressionValues)
+                        {
+                            evaluateContext[data.Key] = data.Value;
+                        }
+
+                        inputs[key] = manifestManager.EvaluateDefaultInput(ExecutionContext, key, input.Value, evaluateContext);
                     }
                 }
             }
@@ -171,15 +179,17 @@ namespace GitHub.Runner.Worker
             ExecutionContext.Debug("Loading env");
             var environment = new Dictionary<String, String>(VarUtil.EnvironmentVariableKeyComparer);
 
-#if OS_WINDOWS
-            var envContext = ExecutionContext.ExpressionValues["env"] as DictionaryContextData;
-#else
-            var envContext = ExecutionContext.ExpressionValues["env"] as CaseSensitiveDictionaryContextData;
-#endif
-            // Apply environment from env context, env context contains job level env and action's evn block
-            foreach (var env in envContext)
+            // Apply environment set using ##[set-env] first since these are job level env
+            foreach (var env in ExecutionContext.EnvironmentVariables)
             {
-                environment[env.Key] = env.Value.ToString();
+                environment[env.Key] = env.Value ?? string.Empty;
+            }
+
+            // Apply action's env block later.
+            var actionEnvironment = templateEvaluator.EvaluateStepEnvironment(Action.Environment, ExecutionContext.ExpressionValues, VarUtil.EnvironmentVariableKeyComparer);
+            foreach (var env in actionEnvironment)
+            {
+                environment[env.Key] = env.Value ?? string.Empty;
             }
 
             // Apply action's intra-action state at last
@@ -287,14 +297,11 @@ namespace GitHub.Runner.Worker
                 return displayName;
             }
             // Try evaluating fully
+            var schema = new PipelineTemplateSchemaFactory().CreateSchema();
+            var templateEvaluator = new PipelineTemplateEvaluator(context.ToTemplateTraceWriter(), schema);
             try
             {
-                if (tokenToParse.CheckHasRequiredContext(contextData, context.ExpressionFunctions))
-                {
-                    var templateEvaluator = context.ToPipelineTemplateEvaluator();
-                    displayName = templateEvaluator.EvaluateStepDisplayName(tokenToParse, contextData, context.ExpressionFunctions);
-                    didFullyEvaluate = true;
-                }
+                didFullyEvaluate = templateEvaluator.TryEvaluateStepDisplayName(tokenToParse, contextData, out displayName);
             }
             catch (TemplateValidationException e)
             {

src/Runner.Worker/Container/ContainerInfo.cs

@@ -2,9 +2,9 @@
 using System.Collections.Generic;
 using System.IO;
 using GitHub.Runner.Common.Util;
+using Pipelines = GitHub.DistributedTask.Pipelines;
 using GitHub.Runner.Common;
 using GitHub.Runner.Sdk;
-using Pipelines = GitHub.DistributedTask.Pipelines;
 
 namespace GitHub.Runner.Worker.Container
 {
@@ -19,6 +19,7 @@ namespace GitHub.Runner.Worker.Container
 
         public ContainerInfo()
         {
+
         }
 
         public ContainerInfo(IHostContext hostContext, Pipelines.JobContainer container, bool isJobContainer = true, string networkAlias = null)
@@ -61,11 +62,8 @@ namespace GitHub.Runner.Worker.Container
                 foreach (var volume in container.Volumes)
                 {
                     UserMountVolumes[volume] = volume;
-                    MountVolumes.Add(new MountVolume(volume));
                 }
             }
-
-            UpdateWebProxyEnv(hostContext.WebProxy);
         }
 
         public string ContainerId { get; set; }
@@ -225,26 +223,6 @@ namespace GitHub.Runner.Worker.Container
         {
             _pathMappings.Insert(0, new PathMapping(hostCommonPath, containerCommonPath));
         }
-
-        private void UpdateWebProxyEnv(RunnerWebProxy webProxy)
-        {
-            // Set common forms of proxy variables if configured in Runner and not set directly by container.env
-            if (!String.IsNullOrEmpty(webProxy.HttpProxyAddress))
-            {
-                ContainerEnvironmentVariables.TryAdd("HTTP_PROXY", webProxy.HttpProxyAddress);
-                ContainerEnvironmentVariables.TryAdd("http_proxy", webProxy.HttpProxyAddress);
-            }
-            if (!String.IsNullOrEmpty(webProxy.HttpsProxyAddress))
-            {
-                ContainerEnvironmentVariables.TryAdd("HTTPS_PROXY", webProxy.HttpsProxyAddress);
-                ContainerEnvironmentVariables.TryAdd("https_proxy", webProxy.HttpsProxyAddress);
-            }
-            if (!String.IsNullOrEmpty(webProxy.NoProxyString))
-            {
-                ContainerEnvironmentVariables.TryAdd("NO_PROXY", webProxy.NoProxyString);
-                ContainerEnvironmentVariables.TryAdd("no_proxy", webProxy.NoProxyString);
-            }
-        }
     }
 
     public class MountVolume

src/Runner.Worker/Container/DockerCommandManager.cs

@@ -130,13 +130,6 @@ namespace GitHub.Runner.Worker.Container
             // Watermark for GitHub Action environment
             dockerOptions.Add("-e GITHUB_ACTIONS=true");
 
-            // Set CI=true when no one else already set it.
-            // CI=true is common set in most CI provider in GitHub
-            if (!container.ContainerEnvironmentVariables.ContainsKey("CI"))
-            {
-                dockerOptions.Add("-e CI=true");
-            }
-
             foreach (var volume in container.MountVolumes)
             {
                 // replace `"` with `\"` and add `"{0}"` to all path.
@@ -196,13 +189,6 @@ namespace GitHub.Runner.Worker.Container
             // Watermark for GitHub Action environment
             dockerOptions.Add("-e GITHUB_ACTIONS=true");
 
-            // Set CI=true when no one else already set it.
-            // CI=true is common set in most CI provider in GitHub
-            if (!container.ContainerEnvironmentVariables.ContainsKey("CI"))
-            {
-                dockerOptions.Add("-e CI=true");
-            }
-
             if (!string.IsNullOrEmpty(container.ContainerEntryPoint))
             {
                 dockerOptions.Add($"--entrypoint \"{container.ContainerEntryPoint}\"");

src/Runner.Worker/ContainerOperationProvider.cs

@@ -61,11 +61,16 @@ namespace GitHub.Runner.Worker
             {
                 throw new NotSupportedException("Container feature is not supported when runner is already running inside container.");
             }
-#else
-            var initProcessCgroup = File.ReadLines("/proc/1/cgroup");
-            if (initProcessCgroup.Any(x => x.IndexOf(":/docker/", StringComparison.OrdinalIgnoreCase) >= 0))
+#else       
+            var path = "/proc/1/cgroup";
+            // OSX does not have this file, but you cannot run OSX as a base image for docker containers currently.
+            if (File.Exists(path))
             {
-                throw new NotSupportedException("Container feature is not supported when runner is already running inside container.");
+                var initProcessCgroup = File.ReadLines(path);
+                if (initProcessCgroup.Any(x => x.IndexOf(":/docker/", StringComparison.OrdinalIgnoreCase) >= 0))
+                {
+                    throw new NotSupportedException("Container feature is not supported when runner is already running inside container.");
+                }
             }
 #endif
 

src/Runner.Worker/ExecutionContext.cs

@@ -1,25 +1,23 @@
 using System;
-using System.Collections;
 using System.Collections.Generic;
 using System.Collections.Specialized;
 using System.Globalization;
 using System.IO;
 using System.Linq;
-using System.Text;
 using System.Threading;
 using System.Threading.Tasks;
 using System.Web;
-using GitHub.DistributedTask.Expressions2;
+using GitHub.Runner.Worker.Container;
+using GitHub.Services.WebApi;
 using GitHub.DistributedTask.Pipelines;
 using GitHub.DistributedTask.Pipelines.ContextData;
-using GitHub.DistributedTask.Pipelines.ObjectTemplating;
 using GitHub.DistributedTask.WebApi;
 using GitHub.Runner.Common.Util;
 using GitHub.Runner.Common;
 using GitHub.Runner.Sdk;
-using GitHub.Runner.Worker.Container;
-using GitHub.Services.WebApi;
 using Newtonsoft.Json;
+using System.Text;
+using System.Collections;
 using ObjectTemplating = GitHub.DistributedTask.ObjectTemplating;
 using Pipelines = GitHub.DistributedTask.Pipelines;
 
@@ -39,7 +37,6 @@ namespace GitHub.Runner.Worker
         string ContextName { get; }
         Task ForceCompleted { get; }
         TaskResult? Result { get; set; }
-        TaskResult? Outcome { get; set; }
         string ResultCode { get; set; }
         TaskResult? CommandResult { get; set; }
         CancellationToken CancellationToken { get; }
@@ -48,14 +45,11 @@ namespace GitHub.Runner.Worker
         PlanFeatures Features { get; }
         Variables Variables { get; }
         Dictionary<string, string> IntraActionState { get; }
-        IDictionary<String, IDictionary<String, String>> JobDefaults { get; }
-        Dictionary<string, VariableValue> JobOutputs { get; }
+        HashSet<string> OutputVariables { get; }
         IDictionary<String, String> EnvironmentVariables { get; }
         IDictionary<String, ContextScope> Scopes { get; }
-        IList<String> FileTable { get; }
         StepsContext StepsContext { get; }
         DictionaryContextData ExpressionValues { get; }
-        IList<IFunctionInfo> ExpressionFunctions { get; }
         List<string> PrependPath { get; }
         ContainerInfo Container { get; set; }
         List<ContainerInfo> ServiceContainers { get; }
@@ -113,6 +107,7 @@ namespace GitHub.Runner.Worker
         private readonly TimelineRecord _record = new TimelineRecord();
         private readonly Dictionary<Guid, TimelineRecord> _detailRecords = new Dictionary<Guid, TimelineRecord>();
         private readonly object _loggerLock = new object();
+        private readonly HashSet<string> _outputvariables = new HashSet<string>(StringComparer.OrdinalIgnoreCase);
         private readonly object _matchersLock = new object();
 
         private event OnMatcherChanged _onMatcherChanged;
@@ -142,14 +137,11 @@ namespace GitHub.Runner.Worker
         public List<ServiceEndpoint> Endpoints { get; private set; }
         public Variables Variables { get; private set; }
         public Dictionary<string, string> IntraActionState { get; private set; }
-        public IDictionary<String, IDictionary<String, String>> JobDefaults { get; private set; }
-        public Dictionary<string, VariableValue> JobOutputs { get; private set; }
+        public HashSet<string> OutputVariables => _outputvariables;
         public IDictionary<String, String> EnvironmentVariables { get; private set; }
         public IDictionary<String, ContextScope> Scopes { get; private set; }
-        public IList<String> FileTable { get; private set; }
         public StepsContext StepsContext { get; private set; }
         public DictionaryContextData ExpressionValues { get; } = new DictionaryContextData();
-        public IList<IFunctionInfo> ExpressionFunctions { get; } = new List<IFunctionInfo>();
         public bool WriteDebug { get; private set; }
         public List<string> PrependPath { get; private set; }
         public ContainerInfo Container { get; set; }
@@ -176,8 +168,6 @@ namespace GitHub.Runner.Worker
             }
         }
 
-        public TaskResult? Outcome { get; set; }
-
         public TaskResult? CommandResult { get; set; }
 
         private string ContextType => _record.RecordType;
@@ -274,18 +264,12 @@ namespace GitHub.Runner.Worker
                 child.IntraActionState = intraActionState;
             }
             child.EnvironmentVariables = EnvironmentVariables;
-            child.JobDefaults = JobDefaults;
             child.Scopes = Scopes;
-            child.FileTable = FileTable;
             child.StepsContext = StepsContext;
             foreach (var pair in ExpressionValues)
             {
                 child.ExpressionValues[pair.Key] = pair.Value;
             }
-            foreach (var item in ExpressionFunctions)
-            {
-                child.ExpressionFunctions.Add(item);
-            }
             child._cancellationTokenSource = new CancellationTokenSource();
             child.WriteDebug = WriteDebug;
             child._parentExecutionContext = this;
@@ -358,12 +342,6 @@ namespace GitHub.Runner.Worker
 
             _logger.End();
 
-            if (!string.IsNullOrEmpty(ContextName))
-            {
-                StepsContext.SetOutcome(ScopeName, ContextName, (Outcome ?? Result ?? TaskResult.Succeeded).ToActionResult().ToString());
-                StepsContext.SetConclusion(ScopeName, ContextName, (Result ?? TaskResult.Succeeded).ToActionResult().ToString());
-            }
-
             return Result.Value;
         }
 
@@ -574,12 +552,6 @@ namespace GitHub.Runner.Worker
             // Environment variables shared across all actions
             EnvironmentVariables = new Dictionary<string, string>(VarUtil.EnvironmentVariableKeyComparer);
 
-            // Job defaults shared across all actions
-            JobDefaults = new Dictionary<string, IDictionary<string, string>>(StringComparer.OrdinalIgnoreCase);
-
-            // Job Outputs
-            JobOutputs = new Dictionary<string, VariableValue>(StringComparer.OrdinalIgnoreCase);
-
             // Service container info
             ServiceContainers = new List<ContainerInfo>();
 
@@ -596,9 +568,6 @@ namespace GitHub.Runner.Worker
                 }
             }
 
-            // File table
-            FileTable = new List<String>(message.FileTable ?? new string[0]);
-
             // Expression values
             if (message.ContextData?.Count > 0)
             {
@@ -616,13 +585,8 @@ namespace GitHub.Runner.Worker
             var githubAccessToken = new StringContextData(Variables.Get("system.github.token"));
             var base64EncodedToken = Convert.ToBase64String(Encoding.UTF8.GetBytes($"x-access-token:{githubAccessToken}"));
             HostContext.SecretMasker.AddValue(base64EncodedToken);
-            var githubJob = Variables.Get("system.github.job");
             var githubContext = new GitHubContext();
             githubContext["token"] = githubAccessToken;
-            if (!string.IsNullOrEmpty(githubJob))
-            {
-                githubContext["job"] = new StringContextData(githubJob);
-            }
             var githubDictionary = ExpressionValues["github"].AssertDictionary("github");
             foreach (var pair in githubDictionary)
             {
@@ -758,7 +722,7 @@ namespace GitHub.Runner.Worker
                 var owners = config.Matchers.Select(x => $"'{x.Owner}'");
                 var joinedOwners = string.Join(", ", owners);
                 // todo: loc
-                this.Debug($"Added matchers: {joinedOwners}. Problem matchers scan action output for known warning or error strings and report these inline.");
+                this.Output($"Added matchers: {joinedOwners}. Problem matchers scan action output for known warning or error strings and report these inline.");
             }
         }
 
@@ -800,7 +764,7 @@ namespace GitHub.Runner.Worker
                 owners = removedMatchers.Select(x => $"'{x.Owner}'");
                 var joinedOwners = string.Join(", ", owners);
                 // todo: loc
-                this.Debug($"Removed matchers: {joinedOwners}");
+                this.Output($"Removed matchers: {joinedOwners}");
             }
         }
 
@@ -915,21 +879,6 @@ namespace GitHub.Runner.Worker
             }
         }
 
-        public static IEnumerable<KeyValuePair<string, object>> ToExpressionState(this IExecutionContext context)
-        {
-            return new[] { new KeyValuePair<string, object>(nameof(IExecutionContext), context) };
-        }
-
-        public static PipelineTemplateEvaluator ToPipelineTemplateEvaluator(this IExecutionContext context, ObjectTemplating.ITraceWriter traceWriter = null)
-        {
-            if (traceWriter == null)
-            {
-                traceWriter = context.ToTemplateTraceWriter();
-            }
-            var schema = PipelineTemplateSchemaFactory.GetSchema();
-            return new PipelineTemplateEvaluator(traceWriter, schema, context.FileTable);
-        }
-
         public static ObjectTemplating.ITraceWriter ToTemplateTraceWriter(this IExecutionContext context)
         {
             return new TemplateTraceWriter(context);
@@ -942,7 +891,6 @@ namespace GitHub.Runner.Worker
 
         internal TemplateTraceWriter(IExecutionContext executionContext)
         {
-            ArgUtil.NotNull(executionContext, nameof(executionContext));
             _executionContext = executionContext;
         }
 

src/Runner.Worker/ExpressionManager.cs

@@ -0,0 +1,162 @@
+using System;
+using System.Collections;
+using System.Collections.Generic;
+using System.Linq;
+using System.Text;
+using GitHub.DistributedTask.Expressions2;
+using GitHub.DistributedTask.Expressions2.Sdk;
+using GitHub.DistributedTask.WebApi;
+using GitHub.Runner.Common;
+using GitHub.Runner.Common.Util;
+using GitHub.Runner.Sdk;
+using ObjectTemplating = GitHub.DistributedTask.ObjectTemplating;
+using PipelineTemplateConstants = GitHub.DistributedTask.Pipelines.ObjectTemplating.PipelineTemplateConstants;
+
+namespace GitHub.Runner.Worker
+{
+    [ServiceLocator(Default = typeof(ExpressionManager))]
+    public interface IExpressionManager : IRunnerService
+    {
+        ConditionResult Evaluate(IExecutionContext context, string condition, bool hostTracingOnly = false);
+    }
+
+    public sealed class ExpressionManager : RunnerService, IExpressionManager
+    {
+        public ConditionResult Evaluate(IExecutionContext executionContext, string condition, bool hostTracingOnly = false)
+        {
+            ArgUtil.NotNull(executionContext, nameof(executionContext));
+
+            ConditionResult result = new ConditionResult();
+            var expressionTrace = new TraceWriter(Trace, hostTracingOnly ? null : executionContext);
+            var tree = Parse(executionContext, expressionTrace, condition);
+            var expressionResult = tree.Evaluate(expressionTrace, HostContext.SecretMasker, state: executionContext, options: null);
+            result.Value = expressionResult.IsTruthy;
+            result.Trace = expressionTrace.Trace;
+
+            return result;
+        }
+
+        private static IExpressionNode Parse(IExecutionContext executionContext, TraceWriter expressionTrace, string condition)
+        {
+            ArgUtil.NotNull(executionContext, nameof(executionContext));
+
+            if (string.IsNullOrWhiteSpace(condition))
+            {
+                condition = $"{PipelineTemplateConstants.Success}()";
+            }
+
+            var parser = new ExpressionParser();
+            var namedValues = executionContext.ExpressionValues.Keys.Select(x => new NamedValueInfo<ContextValueNode>(x)).ToArray();
+            var functions = new IFunctionInfo[]
+            {
+                new FunctionInfo<AlwaysNode>(name: Constants.Expressions.Always, minParameters: 0, maxParameters: 0),
+                new FunctionInfo<CancelledNode>(name: Constants.Expressions.Cancelled, minParameters: 0, maxParameters: 0),
+                new FunctionInfo<FailureNode>(name: Constants.Expressions.Failure, minParameters: 0, maxParameters: 0),
+                new FunctionInfo<SuccessNode>(name: Constants.Expressions.Success, minParameters: 0, maxParameters: 0),
+            };
+            return parser.CreateTree(condition, expressionTrace, namedValues, functions) ?? new SuccessNode();
+        }
+
+        private sealed class TraceWriter : DistributedTask.Expressions2.ITraceWriter
+        {
+            private readonly IExecutionContext _executionContext;
+            private readonly Tracing _trace;
+            private readonly StringBuilder _traceBuilder = new StringBuilder();
+
+            public string Trace => _traceBuilder.ToString();
+
+            public TraceWriter(Tracing trace, IExecutionContext executionContext)
+            {
+                ArgUtil.NotNull(trace, nameof(trace));
+                _trace = trace;
+                _executionContext = executionContext;
+            }
+
+            public void Info(string message)
+            {
+                _trace.Info(message);
+                _executionContext?.Debug(message);
+                _traceBuilder.AppendLine(message);
+            }
+
+            public void Verbose(string message)
+            {
+                _trace.Verbose(message);
+                _executionContext?.Debug(message);
+            }
+        }
+
+        private sealed class AlwaysNode : Function
+        {
+            protected override Object EvaluateCore(EvaluationContext context, out ResultMemory resultMemory)
+            {
+                resultMemory = null;
+                return true;
+            }
+        }
+
+        private sealed class CancelledNode : Function
+        {
+            protected sealed override object EvaluateCore(EvaluationContext evaluationContext, out ResultMemory resultMemory)
+            {
+                resultMemory = null;
+                var executionContext = evaluationContext.State as IExecutionContext;
+                ArgUtil.NotNull(executionContext, nameof(executionContext));
+                ActionResult jobStatus = executionContext.JobContext.Status ?? ActionResult.Success;
+                return jobStatus == ActionResult.Cancelled;
+            }
+        }
+
+        private sealed class FailureNode : Function
+        {
+            protected sealed override object EvaluateCore(EvaluationContext evaluationContext, out ResultMemory resultMemory)
+            {
+                resultMemory = null;
+                var executionContext = evaluationContext.State as IExecutionContext;
+                ArgUtil.NotNull(executionContext, nameof(executionContext));
+                ActionResult jobStatus = executionContext.JobContext.Status ?? ActionResult.Success;
+                return jobStatus == ActionResult.Failure;
+            }
+        }
+
+        private sealed class SuccessNode : Function
+        {
+            protected sealed override object EvaluateCore(EvaluationContext evaluationContext, out ResultMemory resultMemory)
+            {
+                resultMemory = null;
+                var executionContext = evaluationContext.State as IExecutionContext;
+                ArgUtil.NotNull(executionContext, nameof(executionContext));
+                ActionResult jobStatus = executionContext.JobContext.Status ?? ActionResult.Success;
+                return jobStatus == ActionResult.Success;
+            }
+        }
+
+        private sealed class ContextValueNode : NamedValue
+        {
+            protected override Object EvaluateCore(EvaluationContext evaluationContext, out ResultMemory resultMemory)
+            {
+                resultMemory = null;
+                var jobContext = evaluationContext.State as IExecutionContext;
+                ArgUtil.NotNull(jobContext, nameof(jobContext));
+                return jobContext.ExpressionValues[Name];
+            }
+        }
+    }
+
+    public class ConditionResult
+    {
+        public ConditionResult(bool value = false, string trace = null)
+        {
+            this.Value = value;
+            this.Trace = trace;
+        }
+
+        public bool Value { get; set; }
+        public string Trace { get; set; }
+
+        public static implicit operator ConditionResult(bool value)
+        {
+            return new ConditionResult(value);
+        }
+    }
+}

src/Runner.Worker/Expressions/AlwaysFunction.cs

@@ -1,25 +0,0 @@
-using System;
-using System.Collections;
-using System.Collections.Generic;
-using System.Linq;
-using System.Text;
-using GitHub.DistributedTask.Expressions2;
-using GitHub.DistributedTask.Expressions2.Sdk;
-using GitHub.DistributedTask.WebApi;
-using GitHub.Runner.Common;
-using GitHub.Runner.Common.Util;
-using GitHub.Runner.Sdk;
-using ObjectTemplating = GitHub.DistributedTask.ObjectTemplating;
-using PipelineTemplateConstants = GitHub.DistributedTask.Pipelines.ObjectTemplating.PipelineTemplateConstants;
-
-namespace GitHub.Runner.Worker.Expressions
-{
-    public sealed class AlwaysFunction : Function
-    {
-        protected override Object EvaluateCore(EvaluationContext context, out ResultMemory resultMemory)
-        {
-            resultMemory = null;
-            return true;
-        }
-    }
-}

src/Runner.Worker/Expressions/CancelledFunction.cs

@@ -1,31 +0,0 @@
-using System;
-using System.Collections;
-using System.Collections.Generic;
-using System.Linq;
-using System.Text;
-using GitHub.DistributedTask.Expressions2;
-using GitHub.DistributedTask.Expressions2.Sdk;
-using GitHub.DistributedTask.ObjectTemplating;
-using GitHub.DistributedTask.WebApi;
-using GitHub.Runner.Common;
-using GitHub.Runner.Common.Util;
-using GitHub.Runner.Sdk;
-using ObjectTemplating = GitHub.DistributedTask.ObjectTemplating;
-using PipelineTemplateConstants = GitHub.DistributedTask.Pipelines.ObjectTemplating.PipelineTemplateConstants;
-
-namespace GitHub.Runner.Worker.Expressions
-{
-    public sealed class CancelledFunction : Function
-    {
-        protected sealed override object EvaluateCore(EvaluationContext evaluationContext, out ResultMemory resultMemory)
-        {
-            resultMemory = null;
-            var templateContext = evaluationContext.State as TemplateContext;
-            ArgUtil.NotNull(templateContext, nameof(templateContext));
-            var executionContext = templateContext.State[nameof(IExecutionContext)] as IExecutionContext;
-            ArgUtil.NotNull(executionContext, nameof(executionContext));
-            ActionResult jobStatus = executionContext.JobContext.Status ?? ActionResult.Success;
-            return jobStatus == ActionResult.Cancelled;
-        }
-    }
-}

src/Runner.Worker/Expressions/FailureFunction.cs

@@ -1,31 +0,0 @@
-using System;
-using System.Collections;
-using System.Collections.Generic;
-using System.Linq;
-using System.Text;
-using GitHub.DistributedTask.Expressions2;
-using GitHub.DistributedTask.Expressions2.Sdk;
-using GitHub.DistributedTask.ObjectTemplating;
-using GitHub.DistributedTask.WebApi;
-using GitHub.Runner.Common;
-using GitHub.Runner.Common.Util;
-using GitHub.Runner.Sdk;
-using ObjectTemplating = GitHub.DistributedTask.ObjectTemplating;
-using PipelineTemplateConstants = GitHub.DistributedTask.Pipelines.ObjectTemplating.PipelineTemplateConstants;
-
-namespace GitHub.Runner.Worker.Expressions
-{
-    public sealed class FailureFunction : Function
-    {
-        protected sealed override object EvaluateCore(EvaluationContext evaluationContext, out ResultMemory resultMemory)
-        {
-            resultMemory = null;
-            var templateContext = evaluationContext.State as TemplateContext;
-            ArgUtil.NotNull(templateContext, nameof(templateContext));
-            var executionContext = templateContext.State[nameof(IExecutionContext)] as IExecutionContext;
-            ArgUtil.NotNull(executionContext, nameof(executionContext));
-            ActionResult jobStatus = executionContext.JobContext.Status ?? ActionResult.Success;
-            return jobStatus == ActionResult.Failure;
-        }
-    }
-}

src/Runner.Worker/Expressions/HashFilesFunction.cs

@@ -1,126 +0,0 @@
-using System;
-using System.IO;
-using GitHub.DistributedTask.Expressions2.Sdk;
-using GitHub.DistributedTask.Pipelines.ContextData;
-using GitHub.DistributedTask.Pipelines.ObjectTemplating;
-using GitHub.Runner.Sdk;
-using System.Reflection;
-using System.Threading;
-using System.Collections.Generic;
-
-namespace GitHub.Runner.Worker.Expressions
-{
-    public sealed class HashFilesFunction : Function
-    {
-        protected sealed override Object EvaluateCore(
-            EvaluationContext context,
-            out ResultMemory resultMemory)
-        {
-            resultMemory = null;
-            var templateContext = context.State as DistributedTask.ObjectTemplating.TemplateContext;
-            ArgUtil.NotNull(templateContext, nameof(templateContext));
-            templateContext.ExpressionValues.TryGetValue(PipelineTemplateConstants.GitHub, out var githubContextData);
-            ArgUtil.NotNull(githubContextData, nameof(githubContextData));
-            var githubContext = githubContextData as DictionaryContextData;
-            ArgUtil.NotNull(githubContext, nameof(githubContext));
-            githubContext.TryGetValue(PipelineTemplateConstants.Workspace, out var workspace);
-            var workspaceData = workspace as StringContextData;
-            ArgUtil.NotNull(workspaceData, nameof(workspaceData));
-
-            string githubWorkspace = workspaceData.Value;
-            bool followSymlink = false;
-            List<string> patterns = new List<string>();
-            var firstParameter = true;
-            foreach (var parameter in Parameters)
-            {
-                var parameterString = parameter.Evaluate(context).ConvertToString();
-                if (firstParameter)
-                {
-                    firstParameter = false;
-                    if (parameterString.StartsWith("--"))
-                    {
-                        if (string.Equals(parameterString, "--follow-symbolic-links", StringComparison.OrdinalIgnoreCase))
-                        {
-                            followSymlink = true;
-                            continue;
-                        }
-                        else
-                        {
-                            throw new ArgumentOutOfRangeException($"Invalid glob option {parameterString}, avaliable option: '--follow-symbolic-links'.");
-                        }
-                    }
-                }
-
-                patterns.Add(parameterString);
-            }
-
-            context.Trace.Info($"Search root directory: '{githubWorkspace}'");
-            context.Trace.Info($"Search pattern: '{string.Join(", ", patterns)}'");
-
-            string binDir = Path.GetDirectoryName(Assembly.GetEntryAssembly().Location);
-            string runnerRoot = new DirectoryInfo(binDir).Parent.FullName;
-
-            string node = Path.Combine(runnerRoot, "externals", "node12", "bin", $"node{IOUtil.ExeExtension}");
-            string hashFilesScript = Path.Combine(binDir, "hashFiles");
-            var hashResult = string.Empty;
-            var p = new ProcessInvoker(new HashFilesTrace(context.Trace));
-            p.ErrorDataReceived += ((_, data) =>
-            {
-                if (!string.IsNullOrEmpty(data.Data) && data.Data.StartsWith("__OUTPUT__") && data.Data.EndsWith("__OUTPUT__"))
-                {
-                    hashResult = data.Data.Substring(10, data.Data.Length - 20);
-                    context.Trace.Info($"Hash result: '{hashResult}'");
-                }
-                else
-                {
-                    context.Trace.Info(data.Data);
-                }
-            });
-
-            p.OutputDataReceived += ((_, data) =>
-            {
-                context.Trace.Info(data.Data);
-            });
-
-            var env = new Dictionary<string, string>();
-            if (followSymlink)
-            {
-                env["followSymbolicLinks"] = "true";
-            }
-            env["patterns"] = string.Join(Environment.NewLine, patterns);
-
-            int exitCode = p.ExecuteAsync(workingDirectory: githubWorkspace,
-                                          fileName: node,
-                                          arguments: $"\"{hashFilesScript.Replace("\"", "\\\"")}\"",
-                                          environment: env,
-                                          requireExitCodeZero: false,
-                                          cancellationToken: new CancellationTokenSource(TimeSpan.FromSeconds(120)).Token).GetAwaiter().GetResult();
-
-            if (exitCode != 0)
-            {
-                throw new InvalidOperationException($"hashFiles('{ExpressionUtility.StringEscape(string.Join(", ", patterns))}') failed. Fail to hash files under directory '{githubWorkspace}'");
-            }
-
-            return hashResult;
-        }
-
-        private sealed class HashFilesTrace : ITraceWriter
-        {
-            private GitHub.DistributedTask.Expressions2.ITraceWriter _trace;
-
-            public HashFilesTrace(GitHub.DistributedTask.Expressions2.ITraceWriter trace)
-            {
-                _trace = trace;
-            }
-            public void Info(string message)
-            {
-                _trace.Info(message);
-            }
-
-            public void Verbose(string message)
-            {
-                _trace.Info(message);
-            }
-        }
-    }
-}

src/Runner.Worker/Expressions/SuccessFunction.cs

@@ -1,31 +0,0 @@
-using System;
-using System.Collections;
-using System.Collections.Generic;
-using System.Linq;
-using System.Text;
-using GitHub.DistributedTask.Expressions2;
-using GitHub.DistributedTask.Expressions2.Sdk;
-using GitHub.DistributedTask.ObjectTemplating;
-using GitHub.DistributedTask.WebApi;
-using GitHub.Runner.Common;
-using GitHub.Runner.Common.Util;
-using GitHub.Runner.Sdk;
-using ObjectTemplating = GitHub.DistributedTask.ObjectTemplating;
-using PipelineTemplateConstants = GitHub.DistributedTask.Pipelines.ObjectTemplating.PipelineTemplateConstants;
-
-namespace GitHub.Runner.Worker.Expressions
-{
-    public sealed class SuccessFunction : Function
-    {
-        protected sealed override object EvaluateCore(EvaluationContext evaluationContext, out ResultMemory resultMemory)
-        {
-            resultMemory = null;
-            var templateContext = evaluationContext.State as TemplateContext;
-            ArgUtil.NotNull(templateContext, nameof(templateContext));
-            var executionContext = templateContext.State[nameof(IExecutionContext)] as IExecutionContext;
-            ArgUtil.NotNull(executionContext, nameof(executionContext));
-            ActionResult jobStatus = executionContext.JobContext.Status ?? ActionResult.Success;
-            return jobStatus == ActionResult.Success;
-        }
-    }
-}

src/Runner.Worker/GitHubContext.cs

@@ -10,19 +10,15 @@ namespace GitHub.Runner.Worker
         {
             "action",
             "actor",
-            "api_url", // temp for GHES alpha release
             "base_ref",
             "event_name",
             "event_path",
             "head_ref",
-            "job",
             "ref",
             "repository",
-            "repository_owner",
             "run_id",
             "run_number",
             "sha",
-            "url", // temp for GHES alpha release
             "workflow",
             "workspace",
         };

src/Runner.Worker/Handlers/ContainerActionHandler.cs

@@ -97,14 +97,14 @@ namespace GitHub.Runner.Worker.Handlers
                 }
             }
 
-            var extraExpressionValues = new Dictionary<string, PipelineContextData>(StringComparer.OrdinalIgnoreCase);
-            extraExpressionValues["inputs"] = inputsContext;
+            var evaluateContext = new Dictionary<string, PipelineContextData>(StringComparer.OrdinalIgnoreCase);
+            evaluateContext["inputs"] = inputsContext;
 
             var manifestManager = HostContext.GetService<IActionManifestManager>();
             if (Data.Arguments != null)
             {
                 container.ContainerEntryPointArgs = "";
-                var evaluatedArgs = manifestManager.EvaluateContainerArguments(ExecutionContext, Data.Arguments, extraExpressionValues);
+                var evaluatedArgs = manifestManager.EvaluateContainerArguments(ExecutionContext, Data.Arguments, evaluateContext);
                 foreach (var arg in evaluatedArgs)
                 {
                     if (!string.IsNullOrEmpty(arg))
@@ -124,7 +124,7 @@ namespace GitHub.Runner.Worker.Handlers
 
             if (Data.Environment != null)
             {
-                var evaluatedEnv = manifestManager.EvaluateContainerEnvironment(ExecutionContext, Data.Environment, extraExpressionValues);
+                var evaluatedEnv = manifestManager.EvaluateContainerEnvironment(ExecutionContext, Data.Environment, evaluateContext);
                 foreach (var env in evaluatedEnv)
                 {
                     if (!this.Environment.ContainsKey(env.Key))
@@ -189,13 +189,13 @@ namespace GitHub.Runner.Worker.Handlers
                 container.ContainerEnvironmentVariables[variable.Key] = container.TranslateToContainerPath(variable.Value);
             }
 
-            using (var stdoutManager = new OutputManager(ExecutionContext, ActionCommandManager, container))
-            using (var stderrManager = new OutputManager(ExecutionContext, ActionCommandManager, container))
+            using (var stdoutManager = new OutputManager(ExecutionContext, ActionCommandManager))
+            using (var stderrManager = new OutputManager(ExecutionContext, ActionCommandManager))
             {
                 var runExitCode = await dockerManger.DockerRun(ExecutionContext, container, stdoutManager.OnDataReceived, stderrManager.OnDataReceived);
-                ExecutionContext.Debug($"Docker Action run completed with exit code {runExitCode}");
                 if (runExitCode != 0)
                 {
+                    ExecutionContext.Error($"Docker run failed with exit code {runExitCode}");
                     ExecutionContext.Result = TaskResult.Failed;
                 }
             }

src/Runner.Worker/Handlers/NodeScriptActionHandler.cs

@@ -122,9 +122,9 @@ namespace GitHub.Runner.Worker.Handlers
                 else
                 {
                     var exitCode = await step;
-                    ExecutionContext.Debug($"Node Action run completed with exit code {exitCode}");
                     if (exitCode != 0)
                     {
+                        ExecutionContext.Error($"Node run failed with exit code {exitCode}");
                         ExecutionContext.Result = TaskResult.Failed;
                     }
                 }

src/Runner.Worker/Handlers/OutputManager.cs

@@ -6,7 +6,6 @@ using System.Linq;
 using System.Text.RegularExpressions;
 using GitHub.Runner.Common;
 using GitHub.Runner.Sdk;
-using GitHub.Runner.Worker.Container;
 using DTWebApi = GitHub.DistributedTask.WebApi;
 
 namespace GitHub.Runner.Worker.Handlers
@@ -18,7 +17,6 @@ namespace GitHub.Runner.Worker.Handlers
         private const string _timeoutKey = "GITHUB_ACTIONS_RUNNER_ISSUE_MATCHER_TIMEOUT";
         private static readonly Regex _colorCodeRegex = new Regex(@"\x0033\[[0-9;]*m?", RegexOptions.Compiled | RegexOptions.CultureInvariant);
         private readonly IActionCommandManager _commandManager;
-        private readonly ContainerInfo _container;
         private readonly IExecutionContext _executionContext;
         private readonly int _failsafe = 50;
         private readonly object _matchersLock = new object();
@@ -27,11 +25,10 @@ namespace GitHub.Runner.Worker.Handlers
         // Mapping that indicates whether a directory belongs to the workflow repository
         private readonly Dictionary<string, string> _directoryMap = new Dictionary<string, string>();
 
-        public OutputManager(IExecutionContext executionContext, IActionCommandManager commandManager, ContainerInfo container = null)
+        public OutputManager(IExecutionContext executionContext, IActionCommandManager commandManager)
         {
             _executionContext = executionContext;
             _commandManager = commandManager;
-            _container = container ?? executionContext.Container;
 
             // Recursion failsafe (test override)
             var failsafeString = Environment.GetEnvironmentVariable("RUNNER_TEST_GET_REPOSITORY_PATH_FAILSAFE");
@@ -84,7 +81,7 @@ namespace GitHub.Runner.Worker.Handlers
             {
                 // This does not need to be inside of a critical section.
                 // The logging queues and command handlers are thread-safe.
-                if (_commandManager.TryProcessCommand(_executionContext, line, _container))
+                if (_commandManager.TryProcessCommand(_executionContext, line))
                 {
                     return;
                 }
@@ -260,7 +257,6 @@ namespace GitHub.Runner.Worker.Handlers
                 if (!string.IsNullOrWhiteSpace(match.File))
                 {
                     var file = match.File;
-                    var translate = _container != null;
 
                     // Root using fromPath
                     if (!string.IsNullOrWhiteSpace(match.FromPath) && !Path.IsPathFullyQualified(file))
@@ -279,19 +275,11 @@ namespace GitHub.Runner.Worker.Handlers
                         ArgUtil.NotNullOrEmpty(workspace, "workspace");
 
                         file = Path.Combine(workspace, file);
-                        translate = false;
                     }
 
                     // Remove relative pathing and normalize slashes
                     file = Path.GetFullPath(file);
 
-                    // Translate to host
-                    if (translate)
-                    {
-                        file = _container.TranslateToHostPath(file);
-                        file = Path.GetFullPath(file);
-                    }
-
                     // Check whether the file exists
                     if (File.Exists(file))
                     {

src/Runner.Worker/Handlers/ScriptHandler.cs

@@ -58,21 +58,12 @@ namespace GitHub.Runner.Worker.Handlers
             string shellCommandPath = null;
             bool validateShellOnHost = !(StepHost is ContainerStepHost);
             string prependPath = string.Join(Path.PathSeparator.ToString(), ExecutionContext.PrependPath.Reverse<string>());
-            string shell = null;
-            if (!Inputs.TryGetValue("shell", out shell) || string.IsNullOrEmpty(shell))
-            {
-                // TODO: figure out how defaults interact with template later
-                // for now, we won't check job.defaults if we are inside a template.
-                if (string.IsNullOrEmpty(ExecutionContext.ScopeName) && ExecutionContext.JobDefaults.TryGetValue("run", out var runDefaults))
-                {
-                    runDefaults.TryGetValue("shell", out shell);
-                }
-            }
+            Inputs.TryGetValue("shell", out var shell);
             if (string.IsNullOrEmpty(shell))
             {
 #if OS_WINDOWS
                 shellCommand = "pwsh";
-                if (validateShellOnHost)
+                if(validateShellOnHost)
                 {
                     shellCommandPath = WhichUtil.Which(shellCommand, require: false, Trace, prependPath);
                     if (string.IsNullOrEmpty(shellCommandPath))
@@ -148,36 +139,11 @@ namespace GitHub.Runner.Worker.Handlers
             Inputs.TryGetValue("script", out var contents);
             contents = contents ?? string.Empty;
 
-            string workingDirectory = null;
-            if (!Inputs.TryGetValue("workingDirectory", out workingDirectory))
-            {
-                // TODO: figure out how defaults interact with template later
-                // for now, we won't check job.defaults if we are inside a template.
-                if (string.IsNullOrEmpty(ExecutionContext.ScopeName) && ExecutionContext.JobDefaults.TryGetValue("run", out var runDefaults))
-                {
-                    if (runDefaults.TryGetValue("working-directory", out workingDirectory))
-                    {
-                        ExecutionContext.Debug("Overwrite 'working-directory' base on job defaults.");
-                    }
-                }
-            }
+            Inputs.TryGetValue("workingDirectory", out var workingDirectory);
             var workspaceDir = githubContext["workspace"] as StringContextData;
             workingDirectory = Path.Combine(workspaceDir, workingDirectory ?? string.Empty);
 
-            string shell = null;
-            if (!Inputs.TryGetValue("shell", out shell) || string.IsNullOrEmpty(shell))
-            {
-                // TODO: figure out how defaults interact with template later
-                // for now, we won't check job.defaults if we are inside a template.
-                if (string.IsNullOrEmpty(ExecutionContext.ScopeName) && ExecutionContext.JobDefaults.TryGetValue("run", out var runDefaults))
-                {
-                    if (runDefaults.TryGetValue("shell", out shell))
-                    {
-                        ExecutionContext.Debug("Overwrite 'shell' base on job defaults.");
-                    }
-                }
-            }
-
+            Inputs.TryGetValue("shell", out var shell);
             var isContainerStepHost = StepHost is ContainerStepHost;
 
             string prependPath = string.Join(Path.PathSeparator.ToString(), ExecutionContext.PrependPath.Reverse<string>());

src/Runner.Worker/JobExtension.cs

@@ -1,14 +1,10 @@
 using System;
 using System.Collections.Generic;
 using System.Diagnostics;
-using System.Globalization;
 using System.IO;
 using System.Linq;
-using System.Runtime.Serialization;
 using System.Threading.Tasks;
 using GitHub.DistributedTask.Expressions2;
-using GitHub.DistributedTask.ObjectTemplating.Tokens;
-using GitHub.DistributedTask.Pipelines.ContextData;
 using GitHub.DistributedTask.Pipelines.ObjectTemplating;
 using GitHub.DistributedTask.WebApi;
 using GitHub.Runner.Common;
@@ -18,16 +14,6 @@ using Pipelines = GitHub.DistributedTask.Pipelines;
 
 namespace GitHub.Runner.Worker
 {
-    [DataContract]
-    public class SetupInfo
-    {
-        [DataMember]
-        public string Group { get; set; }
-
-        [DataMember]
-        public string Detail { get; set; }
-    }
-
     [ServiceLocator(Default = typeof(JobExtension))]
 
     public interface IJobExtension : IRunnerService
@@ -63,44 +49,6 @@ namespace GitHub.Runner.Worker
                     context.Start();
                     context.Debug($"Starting: Set up job");
                     context.Output($"Current runner version: '{BuildConstants.RunnerPackage.Version}'");
-
-                    var setupInfoFile = HostContext.GetConfigFile(WellKnownConfigFile.SetupInfo);
-                    if (File.Exists(setupInfoFile))
-                    {
-                        Trace.Info($"Load machine setup info from {setupInfoFile}");
-                        try
-                        {
-                            var setupInfo = IOUtil.LoadObject<List<SetupInfo>>(setupInfoFile);
-                            if (setupInfo?.Count > 0)
-                            {
-                                foreach (var info in setupInfo)
-                                {
-                                    if (!string.IsNullOrEmpty(info?.Detail))
-                                    {
-                                        var groupName = info.Group;
-                                        if (string.IsNullOrEmpty(groupName))
-                                        {
-                                            groupName = "Machine Setup Info";
-                                        }
-
-                                        context.Output($"##[group]{groupName}");
-                                        var multiLines = info.Detail.Replace("\r\n", "\n").TrimEnd('\n').Split('\n');
-                                        foreach (var line in multiLines)
-                                        {
-                                            context.Output(line);
-                                        }
-                                        context.Output("##[endgroup]");
-                                    }
-                                }
-                            }
-                        }
-                        catch (Exception ex)
-                        {
-                            context.Output($"Fail to load and print machine setup info: {ex.Message}");
-                            Trace.Error(ex);
-                        }
-                    }
-
                     var repoFullName = context.GetGitHubContext("repository");
                     ArgUtil.NotNull(repoFullName, nameof(repoFullName));
                     context.Debug($"Primary repository: {repoFullName}");
@@ -128,23 +76,14 @@ namespace GitHub.Runner.Worker
                     context.SetRunnerContext("workspace", Path.Combine(_workDirectory, trackingConfig.PipelineDirectory));
                     context.SetGitHubContext("workspace", Path.Combine(_workDirectory, trackingConfig.WorkspaceDirectory));
 
-                    // Temporary hack for GHES alpha
-                    var configurationStore = HostContext.GetService<IConfigurationStore>();
-                    var runnerSettings = configurationStore.GetSettings();
-                    if (!runnerSettings.IsHostedServer && !string.IsNullOrEmpty(runnerSettings.GitHubUrl))
-                    {
-                        var url = new Uri(runnerSettings.GitHubUrl);
-                        var portInfo = url.IsDefaultPort ? string.Empty : $":{url.Port.ToString(CultureInfo.InvariantCulture)}";
-                        context.SetGitHubContext("url", $"{url.Scheme}://{url.Host}{portInfo}");
-                        context.SetGitHubContext("api_url", $"{url.Scheme}://api.{url.Host}{portInfo}");
-                    }
-
                     // Evaluate the job-level environment variables
                     context.Debug("Evaluating job-level environment variables");
-                    var templateEvaluator = context.ToPipelineTemplateEvaluator();
+                    var templateTrace = context.ToTemplateTraceWriter();
+                    var schema = new PipelineTemplateSchemaFactory().CreateSchema();
+                    var templateEvaluator = new PipelineTemplateEvaluator(templateTrace, schema);
                     foreach (var token in message.EnvironmentVariables)
                     {
-                        var environmentVariables = templateEvaluator.EvaluateStepEnvironment(token, jobContext.ExpressionValues, jobContext.ExpressionFunctions, VarUtil.EnvironmentVariableKeyComparer);
+                        var environmentVariables = templateEvaluator.EvaluateStepEnvironment(token, jobContext.ExpressionValues, VarUtil.EnvironmentVariableKeyComparer);
                         foreach (var pair in environmentVariables)
                         {
                             context.EnvironmentVariables[pair.Key] = pair.Value ?? string.Empty;
@@ -154,7 +93,7 @@ namespace GitHub.Runner.Worker
 
                     // Evaluate the job container
                     context.Debug("Evaluating job container");
-                    var container = templateEvaluator.EvaluateJobContainer(message.JobContainer, jobContext.ExpressionValues, jobContext.ExpressionFunctions);
+                    var container = templateEvaluator.EvaluateJobContainer(message.JobContainer, jobContext.ExpressionValues);
                     if (container != null)
                     {
                         jobContext.Container = new Container.ContainerInfo(HostContext, container);
@@ -162,7 +101,7 @@ namespace GitHub.Runner.Worker
 
                     // Evaluate the job service containers
                     context.Debug("Evaluating job service containers");
-                    var serviceContainers = templateEvaluator.EvaluateJobServiceContainers(message.JobServiceContainers, jobContext.ExpressionValues, jobContext.ExpressionFunctions);
+                    var serviceContainers = templateEvaluator.EvaluateJobServiceContainers(message.JobServiceContainers, jobContext.ExpressionValues);
                     if (serviceContainers?.Count > 0)
                     {
                         foreach (var pair in serviceContainers)
@@ -173,26 +112,6 @@ namespace GitHub.Runner.Worker
                         }
                     }
 
-                    // Evaluate the job defaults
-                    context.Debug("Evaluating job defaults");
-                    foreach (var token in message.Defaults)
-                    {
-                        var defaults = token.AssertMapping("defaults");
-                        if (defaults.Any(x => string.Equals(x.Key.AssertString("defaults key").Value, "run", StringComparison.OrdinalIgnoreCase)))
-                        {
-                            context.JobDefaults["run"] = new Dictionary<string, string>(StringComparer.OrdinalIgnoreCase);
-                            var defaultsRun = defaults.First(x => string.Equals(x.Key.AssertString("defaults key").Value, "run", StringComparison.OrdinalIgnoreCase));
-                            var jobDefaults = templateEvaluator.EvaluateJobDefaultsRun(defaultsRun.Value, jobContext.ExpressionValues, jobContext.ExpressionFunctions);
-                            foreach (var pair in jobDefaults)
-                            {
-                                if (!string.IsNullOrEmpty(pair.Value))
-                                {
-                                    context.JobDefaults["run"][pair.Key] = pair.Value;
-                                }
-                            }
-                        }
-                    }
-
                     // Build up 2 lists of steps, pre-job, job
                     // Download actions not already in the cache
                     Trace.Info("Downloading actions");
@@ -325,58 +244,6 @@ namespace GitHub.Runner.Worker
                     context.Start();
                     context.Debug("Starting: Complete job");
 
-                    // Evaluate job outputs
-                    if (message.JobOutputs != null && message.JobOutputs.Type != TokenType.Null)
-                    {
-                        try
-                        {
-                            context.Output($"Evaluate and set job outputs");
-
-                            // Populate env context for each step
-                            Trace.Info("Initialize Env context for evaluating job outputs");
-#if OS_WINDOWS
-                            var envContext = new DictionaryContextData();
-#else
-                            var envContext = new CaseSensitiveDictionaryContextData();
-#endif
-                            context.ExpressionValues["env"] = envContext;
-                            foreach (var pair in context.EnvironmentVariables)
-                            {
-                                envContext[pair.Key] = new StringContextData(pair.Value ?? string.Empty);
-                            }
-
-                            Trace.Info("Initialize steps context for evaluating job outputs");
-                            context.ExpressionValues["steps"] = context.StepsContext.GetScope(context.ScopeName);
-
-                            var templateEvaluator = context.ToPipelineTemplateEvaluator();
-                            var outputs = templateEvaluator.EvaluateJobOutput(message.JobOutputs, context.ExpressionValues, context.ExpressionFunctions);
-                            foreach (var output in outputs)
-                            {
-                                if (string.IsNullOrEmpty(output.Value))
-                                {
-                                    context.Debug($"Skip output '{output.Key}' since it's empty");
-                                    continue;
-                                }
-
-                                if (!string.Equals(output.Value, HostContext.SecretMasker.MaskSecrets(output.Value)))
-                                {
-                                    context.Warning($"Skip output '{output.Key}' since it may contain secret.");
-                                    continue;
-                                }
-
-                                context.Output($"Set output '{output.Key}'");
-                                jobContext.JobOutputs[output.Key] = output.Value;
-                            }
-                        }
-                        catch (Exception ex)
-                        {
-                            context.Result = TaskResult.Failed;
-                            context.Error($"Fail to evaluate job outputs");
-                            context.Error(ex);
-                            jobContext.Result = TaskResultUtil.MergeTaskResults(jobContext.Result, TaskResult.Failed);
-                        }
-                    }
-
                     if (context.Variables.GetBoolean(Constants.Variables.Actions.RunnerDebug) ?? false)
                     {
                         Trace.Info("Support log upload starting.");

src/Runner.Worker/JobRunner.cs

@@ -107,11 +107,6 @@ namespace GitHub.Runner.Worker
                     return await CompleteJobAsync(jobServer, jobContext, message, TaskResult.Failed);
                 }
 
-                if (jobContext.WriteDebug)
-                {
-                    jobContext.SetRunnerContext("debug", "1");
-                }
-
                 jobContext.SetRunnerContext("os", VarUtil.OS);
 
                 string toolsDirectory = HostContext.GetDirectory(WellKnownDirectory.Tools);
@@ -231,7 +226,7 @@ namespace GitHub.Runner.Worker
             }
 
             Trace.Info("Raising job completed event.");
-            var jobCompletedEvent = new JobCompletedEvent(message.RequestId, message.JobId, result, jobContext.JobOutputs);
+            var jobCompletedEvent = new JobCompletedEvent(message.RequestId, message.JobId, result);
 
             var completeJobRetryLimit = 5;
             var exceptions = new List<Exception>();

src/Runner.Worker/StepsContext.cs

@@ -56,22 +56,13 @@ namespace GitHub.Runner.Worker
             }
         }
 
-        public void SetConclusion(
+        public void SetResult(
             string scopeName,
             string stepName,
-            string conclusion)
+            string result)
         {
             var step = GetStep(scopeName, stepName);
-            step["conclusion"] = new StringContextData(conclusion);
-        }
-
-        public void SetOutcome(
-            string scopeName,
-            string stepName,
-            string outcome)
-        {
-            var step = GetStep(scopeName, stepName);
-            step["outcome"] = new StringContextData(outcome);
+            step["result"] = new StringContextData(result);
         }
 
         private DictionaryContextData GetStep(string scopeName, string stepName)

src/Runner.Worker/StepsRunner.cs

@@ -1,6 +1,8 @@
+using GitHub.DistributedTask.WebApi;
+using Pipelines = GitHub.DistributedTask.Pipelines;
+using GitHub.Runner.Common.Util;
 using System;
 using System.Collections.Generic;
-using System.Text;
 using System.Threading;
 using System.Threading.Tasks;
 using GitHub.DistributedTask.Expressions2;
@@ -8,13 +10,8 @@ using GitHub.DistributedTask.ObjectTemplating.Tokens;
 using GitHub.DistributedTask.Pipelines;
 using GitHub.DistributedTask.Pipelines.ContextData;
 using GitHub.DistributedTask.Pipelines.ObjectTemplating;
-using GitHub.DistributedTask.WebApi;
 using GitHub.Runner.Common;
-using GitHub.Runner.Common.Util;
 using GitHub.Runner.Sdk;
-using GitHub.Runner.Worker.Expressions;
-using ObjectTemplating = GitHub.DistributedTask.ObjectTemplating;
-using Pipelines = GitHub.DistributedTask.Pipelines;
 
 namespace GitHub.Runner.Worker
 {
@@ -66,7 +63,11 @@ namespace GitHub.Runner.Worker
                 }
 
                 var step = jobContext.JobSteps.Dequeue();
-                var nextStep = jobContext.JobSteps.Count > 0 ? jobContext.JobSteps.Peek() : null;
+                IStep nextStep = null;
+                if (jobContext.JobSteps.Count > 0)
+                {
+                    nextStep = jobContext.JobSteps.Peek();
+                }
 
                 Trace.Info($"Processing step: DisplayName='{step.DisplayName}'");
                 ArgUtil.NotNull(step.ExecutionContext, nameof(step.ExecutionContext));
@@ -75,43 +76,16 @@ namespace GitHub.Runner.Worker
                 // Start
                 step.ExecutionContext.Start();
 
-                // Expression functions
-                step.ExecutionContext.ExpressionFunctions.Add(new FunctionInfo<AlwaysFunction>(PipelineTemplateConstants.Always, 0, 0));
-                step.ExecutionContext.ExpressionFunctions.Add(new FunctionInfo<CancelledFunction>(PipelineTemplateConstants.Cancelled, 0, 0));
-                step.ExecutionContext.ExpressionFunctions.Add(new FunctionInfo<FailureFunction>(PipelineTemplateConstants.Failure, 0, 0));
-                step.ExecutionContext.ExpressionFunctions.Add(new FunctionInfo<SuccessFunction>(PipelineTemplateConstants.Success, 0, 0));
-                step.ExecutionContext.ExpressionFunctions.Add(new FunctionInfo<HashFilesFunction>(PipelineTemplateConstants.HashFiles, 1, byte.MaxValue));
+                // Set GITHUB_ACTION
+                if (step is IActionRunner actionStep)
+                {
+                    step.ExecutionContext.SetGitHubContext("action", actionStep.Action.Name);
+                }
 
                 // Initialize scope
                 if (InitializeScope(step, scopeInputs))
                 {
-                    // Populate env context for each step
-                    Trace.Info("Initialize Env context for step");
-#if OS_WINDOWS
-                    var envContext = new DictionaryContextData();
-#else
-                    var envContext = new CaseSensitiveDictionaryContextData();
-#endif
-                    step.ExecutionContext.ExpressionValues["env"] = envContext;
-                    foreach (var pair in step.ExecutionContext.EnvironmentVariables)
-                    {
-                        envContext[pair.Key] = new StringContextData(pair.Value ?? string.Empty);
-                    }
-
-                    if (step is IActionRunner actionStep)
-                    {
-                        // Set GITHUB_ACTION
-                        step.ExecutionContext.SetGitHubContext("action", actionStep.Action.Name);
-
-                        // Evaluate and merge action's env block to env context
-                        var templateEvaluator = step.ExecutionContext.ToPipelineTemplateEvaluator();
-                        var actionEnvironment = templateEvaluator.EvaluateStepEnvironment(actionStep.Action.Environment, step.ExecutionContext.ExpressionValues, step.ExecutionContext.ExpressionFunctions, VarUtil.EnvironmentVariableKeyComparer);
-                        foreach (var env in actionEnvironment)
-                        {
-                            envContext[env.Key] = new StringContextData(env.Value ?? string.Empty);
-                        }
-                    }
-
+                    var expressionManager = HostContext.GetService<IExpressionManager>();
                     try
                     {
                         // Register job cancellation call back only if job cancellation token not been fire before each step run
@@ -125,29 +99,28 @@ namespace GitHub.Runner.Worker
                                 jobContext.JobContext.Status = jobContext.Result?.ToActionResult();
 
                                 step.ExecutionContext.Debug($"Re-evaluate condition on job cancellation for step: '{step.DisplayName}'.");
-                                var conditionReTestTraceWriter = new ConditionTraceWriter(Trace, null); // host tracing only
-                                var conditionReTestResult = false;
+                                ConditionResult conditionReTestResult;
                                 if (HostContext.RunnerShutdownToken.IsCancellationRequested)
                                 {
                                     step.ExecutionContext.Debug($"Skip Re-evaluate condition on runner shutdown.");
+                                    conditionReTestResult = false;
                                 }
                                 else
                                 {
                                     try
                                     {
-                                        var templateEvaluator = step.ExecutionContext.ToPipelineTemplateEvaluator(conditionReTestTraceWriter);
-                                        var condition = new BasicExpressionToken(null, null, null, step.Condition);
-                                        conditionReTestResult = templateEvaluator.EvaluateStepIf(condition, step.ExecutionContext.ExpressionValues, step.ExecutionContext.ExpressionFunctions, step.ExecutionContext.ToExpressionState());
+                                        conditionReTestResult = expressionManager.Evaluate(step.ExecutionContext, step.Condition, hostTracingOnly: true);
                                     }
                                     catch (Exception ex)
                                     {
                                         // Cancel the step since we get exception while re-evaluate step condition.
                                         Trace.Info("Caught exception from expression when re-test condition on job cancellation.");
                                         step.ExecutionContext.Error(ex);
+                                        conditionReTestResult = false;
                                     }
                                 }
 
-                                if (!conditionReTestResult)
+                                if (!conditionReTestResult.Value)
                                 {
                                     // Cancel the step.
                                     Trace.Info("Cancel current running step.");
@@ -167,35 +140,34 @@ namespace GitHub.Runner.Worker
 
                         // Evaluate condition.
                         step.ExecutionContext.Debug($"Evaluating condition for step: '{step.DisplayName}'");
-                        var conditionTraceWriter = new ConditionTraceWriter(Trace, step.ExecutionContext);
-                        var conditionResult = false;
-                        var conditionEvaluateError = default(Exception);
+                        Exception conditionEvaluateError = null;
+                        ConditionResult conditionResult;
                         if (HostContext.RunnerShutdownToken.IsCancellationRequested)
                         {
                             step.ExecutionContext.Debug($"Skip evaluate condition on runner shutdown.");
+                            conditionResult = false;
                         }
                         else
                         {
                             try
                             {
-                                var templateEvaluator = step.ExecutionContext.ToPipelineTemplateEvaluator(conditionTraceWriter);
-                                var condition = new BasicExpressionToken(null, null, null, step.Condition);
-                                conditionResult = templateEvaluator.EvaluateStepIf(condition, step.ExecutionContext.ExpressionValues, step.ExecutionContext.ExpressionFunctions, step.ExecutionContext.ToExpressionState());
+                                conditionResult = expressionManager.Evaluate(step.ExecutionContext, step.Condition);
                             }
                             catch (Exception ex)
                             {
                                 Trace.Info("Caught exception from expression.");
                                 Trace.Error(ex);
+                                conditionResult = false;
                                 conditionEvaluateError = ex;
                             }
                         }
 
                         // no evaluate error but condition is false
-                        if (!conditionResult && conditionEvaluateError == null)
+                        if (!conditionResult.Value && conditionEvaluateError == null)
                         {
                             // Condition == false
                             Trace.Info("Skipping step due to condition evaluation.");
-                            CompleteStep(step, nextStep, TaskResult.Skipped, resultCode: conditionTraceWriter.Trace);
+                            CompleteStep(step, nextStep, TaskResult.Skipped, resultCode: conditionResult.Trace);
                         }
                         else if (conditionEvaluateError != null)
                         {
@@ -252,10 +224,10 @@ namespace GitHub.Runner.Worker
 
             // Set the timeout
             var timeoutMinutes = 0;
-            var templateEvaluator = step.ExecutionContext.ToPipelineTemplateEvaluator();
+            var templateEvaluator = CreateTemplateEvaluator(step.ExecutionContext);
             try
             {
-                timeoutMinutes = templateEvaluator.EvaluateStepTimeout(step.Timeout, step.ExecutionContext.ExpressionValues, step.ExecutionContext.ExpressionFunctions);
+                timeoutMinutes = templateEvaluator.EvaluateStepTimeout(step.Timeout, step.ExecutionContext.ExpressionValues);
             }
             catch (Exception ex)
             {
@@ -346,7 +318,7 @@ namespace GitHub.Runner.Worker
                 var continueOnError = false;
                 try
                 {
-                    continueOnError = templateEvaluator.EvaluateStepContinueOnError(step.ContinueOnError, step.ExecutionContext.ExpressionValues, step.ExecutionContext.ExpressionFunctions);
+                    continueOnError = templateEvaluator.EvaluateStepContinueOnError(step.ContinueOnError, step.ExecutionContext.ExpressionValues);
                 }
                 catch (Exception ex)
                 {
@@ -358,7 +330,6 @@ namespace GitHub.Runner.Worker
 
                 if (continueOnError)
                 {
-                    step.ExecutionContext.Outcome = step.ExecutionContext.Result;
                     step.ExecutionContext.Result = TaskResult.Succeeded;
                     Trace.Info($"Updated step result (continue on error)");
                 }
@@ -395,11 +366,11 @@ namespace GitHub.Runner.Worker
                     executionContext.Debug($"Initializing scope '{scope.Name}'");
                     executionContext.ExpressionValues["steps"] = stepsContext.GetScope(scope.ParentName);
                     executionContext.ExpressionValues["inputs"] = !String.IsNullOrEmpty(scope.ParentName) ? scopeInputs[scope.ParentName] : null;
-                    var templateEvaluator = executionContext.ToPipelineTemplateEvaluator();
+                    var templateEvaluator = CreateTemplateEvaluator(executionContext);
                     var inputs = default(DictionaryContextData);
                     try
                     {
-                        inputs = templateEvaluator.EvaluateStepScopeInputs(scope.Inputs, executionContext.ExpressionValues, executionContext.ExpressionFunctions);
+                        inputs = templateEvaluator.EvaluateStepScopeInputs(scope.Inputs, executionContext.ExpressionValues);
                     }
                     catch (Exception ex)
                     {
@@ -451,11 +422,11 @@ namespace GitHub.Runner.Worker
                     executionContext.Debug($"Finalizing scope '{scope.Name}'");
                     executionContext.ExpressionValues["steps"] = stepsContext.GetScope(scope.Name);
                     executionContext.ExpressionValues["inputs"] = null;
-                    var templateEvaluator = executionContext.ToPipelineTemplateEvaluator();
+                    var templateEvaluator = CreateTemplateEvaluator(executionContext);
                     var outputs = default(DictionaryContextData);
                     try
                     {
-                        outputs = templateEvaluator.EvaluateStepScopeOutputs(scope.Outputs, executionContext.ExpressionValues, executionContext.ExpressionFunctions);
+                        outputs = templateEvaluator.EvaluateStepScopeOutputs(scope.Outputs, executionContext.ExpressionValues);
                     }
                     catch (Exception ex)
                     {
@@ -484,42 +455,11 @@ namespace GitHub.Runner.Worker
             executionContext.Complete(result, resultCode: resultCode);
         }
 
-        private sealed class ConditionTraceWriter : ObjectTemplating::ITraceWriter
+        private PipelineTemplateEvaluator CreateTemplateEvaluator(IExecutionContext executionContext)
         {
-            private readonly IExecutionContext _executionContext;
-            private readonly Tracing _trace;
-            private readonly StringBuilder _traceBuilder = new StringBuilder();
-
-            public string Trace => _traceBuilder.ToString();
-
-            public ConditionTraceWriter(Tracing trace, IExecutionContext executionContext)
-            {
-                ArgUtil.NotNull(trace, nameof(trace));
-                _trace = trace;
-                _executionContext = executionContext;
-            }
-
-            public void Error(string format, params Object[] args)
-            {
-                var message = StringUtil.Format(format, args);
-                _trace.Error(message);
-                _executionContext?.Debug(message);
-            }
-
-            public void Info(string format, params Object[] args)
-            {
-                var message = StringUtil.Format(format, args);
-                _trace.Info(message);
-                _executionContext?.Debug(message);
-                _traceBuilder.AppendLine(message);
-            }
-
-            public void Verbose(string format, params Object[] args)
-            {
-                var message = StringUtil.Format(format, args);
-                _trace.Verbose(message);
-                _executionContext?.Debug(message);
-            }
+            var templateTrace = executionContext.ToTemplateTraceWriter();
+            var schema = new PipelineTemplateSchemaFactory().CreateSchema();
+            return new PipelineTemplateEvaluator(templateTrace, schema);
         }
     }
 }

src/Runner.Worker/action_yaml.json

@@ -90,9 +90,10 @@
                 "github",
                 "strategy",
                 "matrix",
+                "steps",
                 "job",
                 "runner",
-                "hashFiles(1,255)"
+                "env"
             ],
             "string": {}
         },

src/Sdk/DTExpressions2/Expressions2/ExpressionConstants.cs

@@ -15,7 +15,7 @@ namespace GitHub.DistributedTask.Expressions2
             AddFunction<Join>("join", 1, 2);
             AddFunction<StartsWith>("startsWith", 2, 2);
             AddFunction<ToJson>("toJson", 1, 1);
-            AddFunction<FromJson>("fromJson", 1, 1);
+            AddFunction<HashFiles>("hashFiles", 1, 1);
         }
 
         private static void AddFunction<T>(String name, Int32 minParameters, Int32 maxParameters)

src/Sdk/DTExpressions2/Expressions2/Sdk/Functions/FromJson.cs

@@ -1,24 +0,0 @@
-using System;
-using System.IO;
-using GitHub.DistributedTask.Pipelines.ContextData;
-using Newtonsoft.Json;
-using Newtonsoft.Json.Linq;
-
-namespace GitHub.DistributedTask.Expressions2.Sdk.Functions
-{
-    internal sealed class FromJson : Function
-    {
-        protected sealed override Object EvaluateCore(
-            EvaluationContext context,
-            out ResultMemory resultMemory)
-        {
-            resultMemory = null;
-            var json = Parameters[0].Evaluate(context).ConvertToString();
-            using (var stringReader = new StringReader(json))
-            using (var jsonReader = new JsonTextReader(stringReader) { DateParseHandling = DateParseHandling.None, FloatParseHandling = FloatParseHandling.Double })
-            {
-                var token = JToken.ReadFrom(jsonReader);
-                return token.ToPipelineContextData();
-            }
-        }
-    }}

src/Sdk/DTExpressions2/Expressions2/Sdk/Functions/HashFiles.cs

@@ -0,0 +1,122 @@
+using System;
+using System.Collections.Generic;
+using System.Linq;
+using System.Text;
+using Minimatch;
+using System.IO;
+using System.Security.Cryptography;
+using GitHub.DistributedTask.Expressions2.Sdk;
+using GitHub.DistributedTask.Pipelines.ContextData;
+using GitHub.DistributedTask.Pipelines.ObjectTemplating;
+namespace GitHub.DistributedTask.Expressions2.Sdk.Functions
+{
+    internal sealed class HashFiles : Function
+    {
+        protected sealed override Object EvaluateCore(
+            EvaluationContext context,
+            out ResultMemory resultMemory)
+        {
+            resultMemory = null;
+
+            // hashFiles() only works on the runner and only works with files under GITHUB_WORKSPACE
+            // Since GITHUB_WORKSPACE is set by runner, I am using that as the fact of this code runs on server or runner.
+            if (context.State is ObjectTemplating.TemplateContext templateContext &&
+                templateContext.ExpressionValues.TryGetValue(PipelineTemplateConstants.GitHub, out var githubContextData) &&
+                githubContextData is DictionaryContextData githubContext &&
+                githubContext.TryGetValue(PipelineTemplateConstants.Workspace, out var workspace) == true &&
+                workspace is StringContextData workspaceData)
+            {
+                string searchRoot = workspaceData.Value;
+                string pattern = Parameters[0].Evaluate(context).ConvertToString();
+
+                // Convert slashes on Windows
+                if (s_isWindows)
+                {
+                    pattern = pattern.Replace('\\', '/');
+                }
+
+                // Root the pattern
+                if (!Path.IsPathRooted(pattern))
+                {
+                    var patternRoot = s_isWindows ? searchRoot.Replace('\\', '/').TrimEnd('/') : searchRoot.TrimEnd('/');
+                    pattern = string.Concat(patternRoot, "/", pattern);
+                }
+
+                // Get all files
+                context.Trace.Info($"Search root directory: '{searchRoot}'");
+                context.Trace.Info($"Search pattern: '{pattern}'");
+                var files = Directory.GetFiles(searchRoot, "*", SearchOption.AllDirectories)
+                    .Select(x => s_isWindows ? x.Replace('\\', '/') : x)
+                    .OrderBy(x => x, StringComparer.Ordinal)
+                    .ToList();
+                if (files.Count == 0)
+                {
+                    throw new ArgumentException($"hashFiles('{ExpressionUtility.StringEscape(pattern)}') failed. Directory '{searchRoot}' is empty");
+                }
+                else
+                {
+                    context.Trace.Info($"Found {files.Count} files");
+                }
+
+                // Match
+                var matcher = new Minimatcher(pattern, s_minimatchOptions);
+                files = matcher.Filter(files)
+                    .Select(x => s_isWindows ? x.Replace('/', '\\') : x)
+                    .ToList();
+                if (files.Count == 0)
+                {
+                    throw new ArgumentException($"hashFiles('{ExpressionUtility.StringEscape(pattern)}') failed. Search pattern '{pattern}' doesn't match any file under '{searchRoot}'");
+                }
+                else
+                {
+                    context.Trace.Info($"{files.Count} matches to hash");
+                }
+
+                // Hash each file
+                List<byte> filesSha256 = new List<byte>();
+                foreach (var file in files)
+                {
+                    context.Trace.Info($"Hash {file}");
+                    using (SHA256 sha256hash = SHA256.Create())
+                    {
+                        using (var fileStream = File.OpenRead(file))
+                        {
+                            filesSha256.AddRange(sha256hash.ComputeHash(fileStream));
+                        }
+                    }
+                }
+
+                // Hash the hashes
+                using (SHA256 sha256hash = SHA256.Create())
+                {
+                    var hashBytes = sha256hash.ComputeHash(filesSha256.ToArray());
+                    StringBuilder hashString = new StringBuilder();
+                    for (int i = 0; i < hashBytes.Length; i++)
+                    {
+                        hashString.Append(hashBytes[i].ToString("x2"));
+                    }
+                    var result = hashString.ToString();
+                    context.Trace.Info($"Final hash result: '{result}'");
+                    return result;
+                }
+            }
+            else
+            {
+                throw new InvalidOperationException("'hashfiles' expression function is only supported under runner context.");
+            }
+        }
+
+        private static readonly bool s_isWindows = Environment.OSVersion.Platform != PlatformID.Unix && Environment.OSVersion.Platform != PlatformID.MacOSX;
+
+        // Only support basic globbing (* ? and []) and globstar (**)
+        private static readonly Options s_minimatchOptions = new Options
+        {
+            Dot = true,
+            NoBrace = true,
+            NoCase = s_isWindows,
+            NoComment = true,
+            NoExt = true,
+            NoNegate = true,
+        };
+    }
+}

src/Sdk/DTGenerated/Generated/TaskAgentHttpClientBase.cs

@@ -779,65 +779,5 @@ namespace GitHub.DistributedTask.WebApi
                 userState: userState,
                 cancellationToken: cancellationToken);
         }
-
-        /// <summary>
-        /// [Preview API]
-        /// </summary>
-        /// <param name="poolId"></param>
-        /// <param name="agentId"></param>
-        /// <param name="userState"></param>
-        /// <param name="cancellationToken">The cancellation token to cancel operation.</param>
-        public Task<String> GetAgentAuthUrlAsync(
-            int poolId,
-            int agentId,
-            object userState = null,
-            CancellationToken cancellationToken = default)
-        {
-            HttpMethod httpMethod = new HttpMethod("GET");
-            Guid locationId = new Guid("a82a119c-1e46-44b6-8d75-c82a79cf975b");
-            object routeValues = new { poolId = poolId, agentId = agentId };
-
-            return SendAsync<String>(
-                httpMethod,
-                locationId,
-                routeValues: routeValues,
-                version: new ApiResourceVersion(6.0, 1),
-                userState: userState,
-                cancellationToken: cancellationToken);
-        }
-
-        /// <summary>
-        /// [Preview API]
-        /// </summary>
-        /// <param name="poolId"></param>
-        /// <param name="agentId"></param>
-        /// <param name="error"></param>
-        /// <param name="userState"></param>
-        /// <param name="cancellationToken">The cancellation token to cancel operation.</param>
-        [EditorBrowsable(EditorBrowsableState.Never)]
-        public virtual async Task ReportAgentAuthUrlMigrationErrorAsync(
-            int poolId,
-            int agentId,
-            string error,
-            object userState = null,
-            CancellationToken cancellationToken = default)
-        {
-            HttpMethod httpMethod = new HttpMethod("POST");
-            Guid locationId = new Guid("a82a119c-1e46-44b6-8d75-c82a79cf975b");
-            object routeValues = new { poolId = poolId, agentId = agentId };
-            HttpContent content = new ObjectContent<string>(error, new VssJsonMediaTypeFormatter(true));
-
-            using (HttpResponseMessage response = await SendAsync(
-                httpMethod,
-                locationId,
-                routeValues: routeValues,
-                version: new ApiResourceVersion(6.0, 1),
-                userState: userState,
-                cancellationToken: cancellationToken,
-                content: content).ConfigureAwait(false))
-            {
-                return;
-            }
-        }
     }
 }

src/Sdk/DTObjectTemplating/ObjectTemplating/Schema/Definition.cs

@@ -1,5 +1,4 @@
 using System;
-using System.Collections.Generic;
 using System.Linq;
 using GitHub.DistributedTask.ObjectTemplating.Tokens;
 
@@ -23,27 +22,10 @@ namespace GitHub.DistributedTask.ObjectTemplating.Schema
                 {
                     var context = definition[i].Value.AssertSequence($"{TemplateConstants.Context}");
                     definition.RemoveAt(i);
-                    var readerContext = new HashSet<String>(StringComparer.OrdinalIgnoreCase);
-                    var evaluatorContext = new HashSet<String>(StringComparer.OrdinalIgnoreCase);
-                    foreach (TemplateToken item in context)
-                    {
-                        var itemStr = item.AssertString($"{TemplateConstants.Context} item").Value;
-                        readerContext.Add(itemStr);
-
-                        // Remove min/max parameter info
-                        var paramIndex = itemStr.IndexOf('(');
-                        if (paramIndex > 0)
-                        {
-                            evaluatorContext.Add(String.Concat(itemStr.Substring(0, paramIndex + 1), ")"));
-                        }
-                        else
-                        {
-                            evaluatorContext.Add(itemStr);
-                        }
-                    }
-
-                    ReaderContext = readerContext.ToArray();
-                    EvaluatorContext = evaluatorContext.ToArray();
+                    Context = context
+                        .Select(x => x.AssertString($"{TemplateConstants.Context} item").Value)
+                        .Distinct()
+                        .ToArray();
                 }
                 else if (String.Equals(definitionKey.Value, TemplateConstants.Description, StringComparison.Ordinal))
                 {
@@ -58,17 +40,7 @@ namespace GitHub.DistributedTask.ObjectTemplating.Schema
 
         internal abstract DefinitionType DefinitionType { get; }
 
-        /// <summary>
-        /// Used by the template reader to determine allowed expression values and functions.
-        /// Also used by the template reader to validate function min/max parameters.
-        /// </summary>
-        internal String[] ReaderContext { get; private set; } = new String[0];
-
-        /// <summary>
-        /// Used by the template evaluator to determine allowed expression values and functions.
-        /// The min/max parameter info is omitted.
-        /// </summary>
-        internal String[] EvaluatorContext { get; private set; } = new String[0];
+        internal String[] Context { get; private set; } = new String[0];
 
         internal abstract void Validate(
             TemplateSchema schema,

src/Sdk/DTObjectTemplating/ObjectTemplating/Schema/MappingDefinition.cs

@@ -30,7 +30,8 @@ namespace GitHub.DistributedTask.ObjectTemplating.Schema
                                     foreach (var propertiesPair in properties)
                                     {
                                         var propertyName = propertiesPair.Key.AssertString($"{TemplateConstants.Definition} {TemplateConstants.Mapping} {TemplateConstants.Properties} key");
-                                        Properties.Add(propertyName.Value, new PropertyValue(propertiesPair.Value));
+                                        var propertyValue = propertiesPair.Value.AssertString($"{TemplateConstants.Definition} {TemplateConstants.Mapping} {TemplateConstants.Properties} value");
+                                        Properties.Add(propertyName.Value, new PropertyValue(propertyValue.Value));
                                     }
                                     break;
 
@@ -84,7 +85,7 @@ namespace GitHub.DistributedTask.ObjectTemplating.Schema
                 }
                 else
                 {
-                    throw new ArgumentException($"Property '{TemplateConstants.LooseKeyType}' is defined but '{TemplateConstants.LooseValueType}' is not defined on '{name}'");
+                    throw new ArgumentException($"Property '{TemplateConstants.LooseKeyType}' is defined but '{TemplateConstants.LooseValueType}' is not defined");
                 }
             }
             // Otherwise validate loose value type not be defined
@@ -94,21 +95,16 @@ namespace GitHub.DistributedTask.ObjectTemplating.Schema
             }
 
             // Lookup each property
-            foreach (var property in Properties)
+            foreach (var property in Properties.Values)
             {
-                if (String.IsNullOrEmpty(property.Value.Type))
-                {
-                    throw new ArgumentException($"Type not specified for the '{property.Key}' property on the '{name}' type");
-                }
-
-                schema.GetDefinition(property.Value.Type);
+                schema.GetDefinition(property.Type);
             }
 
             if (!String.IsNullOrEmpty(Inherits))
             {
                 var inherited = schema.GetDefinition(Inherits);
 
-                if (inherited.ReaderContext.Length > 0)
+                if (inherited.Context.Length > 0)
                 {
                     throw new NotSupportedException($"Property '{TemplateConstants.Context}' is not supported on inhertied definitions");
                 }

src/Sdk/DTObjectTemplating/ObjectTemplating/Schema/OneOfDefinition.cs

@@ -62,7 +62,7 @@ namespace GitHub.DistributedTask.ObjectTemplating.Schema
             {
                 var nestedDefinition = schema.GetDefinition(nestedType);
 
-                if (nestedDefinition.ReaderContext.Length > 0)
+                if (nestedDefinition.Context.Length > 0)
                 {
                     throw new ArgumentException($"'{name}' is a one-of definition and references another definition that defines context. This is currently not supported.");
                 }

src/Sdk/DTObjectTemplating/ObjectTemplating/Schema/PropertyValue.cs

@@ -1,40 +1,18 @@
 using System;
-using GitHub.DistributedTask.ObjectTemplating.Tokens;
 
 namespace GitHub.DistributedTask.ObjectTemplating.Schema
 {
     internal sealed class PropertyValue
     {
-        internal PropertyValue(TemplateToken token)
+        internal PropertyValue()
         {
-            if (token is StringToken stringToken)
-            {
-                Type = stringToken.Value;
-            }
-            else
-            {
-                var mapping = token.AssertMapping($"{TemplateConstants.MappingPropertyValue}");
-                foreach (var mappingPair in mapping)
-                {
-                    var mappingKey = mappingPair.Key.AssertString($"{TemplateConstants.MappingPropertyValue} key");
-                    switch (mappingKey.Value)
-                    {
-                        case TemplateConstants.Type:
-                            Type = mappingPair.Value.AssertString($"{TemplateConstants.MappingPropertyValue} {TemplateConstants.Type}").Value;
-                            break;
-                        case TemplateConstants.Required:
-                            Required = mappingPair.Value.AssertBoolean($"{TemplateConstants.MappingPropertyValue} {TemplateConstants.Required}").Value;
-                            break;
-                        default:
-                            mappingKey.AssertUnexpectedValue($"{TemplateConstants.MappingPropertyValue} key");
-                            break;
-                    }
-                }
-            }
+        }
+
+        internal PropertyValue(String type)
+        {
+            Type = type;
         }
 
         internal String Type { get; set; }
-
-        internal Boolean Required { get; set; }
     }
 }

src/Sdk/DTObjectTemplating/ObjectTemplating/Schema/TemplateSchema.cs

@@ -312,8 +312,8 @@ namespace GitHub.DistributedTask.ObjectTemplating.Schema
 
                     // template-schema
                     mappingDefinition = new MappingDefinition();
-                    mappingDefinition.Properties.Add(TemplateConstants.Version, new PropertyValue(new StringToken(null, null, null, TemplateConstants.NonEmptyString)));
-                    mappingDefinition.Properties.Add(TemplateConstants.Definitions, new PropertyValue(new StringToken(null, null, null, TemplateConstants.Definitions)));
+                    mappingDefinition.Properties.Add(TemplateConstants.Version, new PropertyValue(TemplateConstants.NonEmptyString));
+                    mappingDefinition.Properties.Add(TemplateConstants.Definitions, new PropertyValue(TemplateConstants.Definitions));
                     schema.Definitions.Add(TemplateConstants.TemplateSchema, mappingDefinition);
 
                     // definitions
@@ -335,9 +335,9 @@ namespace GitHub.DistributedTask.ObjectTemplating.Schema
 
                     // null-definition
                     mappingDefinition = new MappingDefinition();
-                    mappingDefinition.Properties.Add(TemplateConstants.Description, new PropertyValue(new StringToken(null, null, null, TemplateConstants.String)));
-                    mappingDefinition.Properties.Add(TemplateConstants.Context, new PropertyValue(new StringToken(null, null, null, TemplateConstants.SequenceOfNonEmptyString)));
-                    mappingDefinition.Properties.Add(TemplateConstants.Null, new PropertyValue(new StringToken(null, null, null, TemplateConstants.NullDefinitionProperties)));
+                    mappingDefinition.Properties.Add(TemplateConstants.Description, new PropertyValue(TemplateConstants.String));
+                    mappingDefinition.Properties.Add(TemplateConstants.Context, new PropertyValue(TemplateConstants.SequenceOfNonEmptyString));
+                    mappingDefinition.Properties.Add(TemplateConstants.Null, new PropertyValue(TemplateConstants.NullDefinitionProperties));
                     schema.Definitions.Add(TemplateConstants.NullDefinition, mappingDefinition);
 
                     // null-definition-properties
@@ -346,9 +346,9 @@ namespace GitHub.DistributedTask.ObjectTemplating.Schema
 
                     // boolean-definition
                     mappingDefinition = new MappingDefinition();
-                    mappingDefinition.Properties.Add(TemplateConstants.Description, new PropertyValue(new StringToken(null, null, null, TemplateConstants.String)));
-                    mappingDefinition.Properties.Add(TemplateConstants.Context, new PropertyValue(new StringToken(null, null, null, TemplateConstants.SequenceOfNonEmptyString)));
-                    mappingDefinition.Properties.Add(TemplateConstants.Boolean, new PropertyValue(new StringToken(null, null, null, TemplateConstants.BooleanDefinitionProperties)));
+                    mappingDefinition.Properties.Add(TemplateConstants.Description, new PropertyValue(TemplateConstants.String));
+                    mappingDefinition.Properties.Add(TemplateConstants.Context, new PropertyValue(TemplateConstants.SequenceOfNonEmptyString));
+                    mappingDefinition.Properties.Add(TemplateConstants.Boolean, new PropertyValue(TemplateConstants.BooleanDefinitionProperties));
                     schema.Definitions.Add(TemplateConstants.BooleanDefinition, mappingDefinition);
 
                     // boolean-definition-properties
@@ -357,9 +357,9 @@ namespace GitHub.DistributedTask.ObjectTemplating.Schema
 
                     // number-definition
                     mappingDefinition = new MappingDefinition();
-                    mappingDefinition.Properties.Add(TemplateConstants.Description, new PropertyValue(new StringToken(null, null, null, TemplateConstants.String)));
-                    mappingDefinition.Properties.Add(TemplateConstants.Context, new PropertyValue(new StringToken(null, null, null, TemplateConstants.SequenceOfNonEmptyString)));
-                    mappingDefinition.Properties.Add(TemplateConstants.Number, new PropertyValue(new StringToken(null, null, null, TemplateConstants.NumberDefinitionProperties)));
+                    mappingDefinition.Properties.Add(TemplateConstants.Description, new PropertyValue(TemplateConstants.String));
+                    mappingDefinition.Properties.Add(TemplateConstants.Context, new PropertyValue(TemplateConstants.SequenceOfNonEmptyString));
+                    mappingDefinition.Properties.Add(TemplateConstants.Number, new PropertyValue(TemplateConstants.NumberDefinitionProperties));
                     schema.Definitions.Add(TemplateConstants.NumberDefinition, mappingDefinition);
 
                     // number-definition-properties
@@ -368,68 +368,55 @@ namespace GitHub.DistributedTask.ObjectTemplating.Schema
 
                     // string-definition
                     mappingDefinition = new MappingDefinition();
-                    mappingDefinition.Properties.Add(TemplateConstants.Description, new PropertyValue(new StringToken(null, null, null, TemplateConstants.String)));
-                    mappingDefinition.Properties.Add(TemplateConstants.Context, new PropertyValue(new StringToken(null, null, null, TemplateConstants.SequenceOfNonEmptyString)));
-                    mappingDefinition.Properties.Add(TemplateConstants.String, new PropertyValue(new StringToken(null, null, null, TemplateConstants.StringDefinitionProperties)));
+                    mappingDefinition.Properties.Add(TemplateConstants.Description, new PropertyValue(TemplateConstants.String));
+                    mappingDefinition.Properties.Add(TemplateConstants.Context, new PropertyValue(TemplateConstants.SequenceOfNonEmptyString));
+                    mappingDefinition.Properties.Add(TemplateConstants.String, new PropertyValue(TemplateConstants.StringDefinitionProperties));
                     schema.Definitions.Add(TemplateConstants.StringDefinition, mappingDefinition);
 
                     // string-definition-properties
                     mappingDefinition = new MappingDefinition();
-                    mappingDefinition.Properties.Add(TemplateConstants.Constant, new PropertyValue(new StringToken(null, null, null, TemplateConstants.NonEmptyString)));
-                    mappingDefinition.Properties.Add(TemplateConstants.IgnoreCase, new PropertyValue(new StringToken(null, null, null,TemplateConstants.Boolean)));
-                    mappingDefinition.Properties.Add(TemplateConstants.RequireNonEmpty, new PropertyValue(new StringToken(null, null, null, TemplateConstants.Boolean)));
+                    mappingDefinition.Properties.Add(TemplateConstants.Constant, new PropertyValue(TemplateConstants.NonEmptyString));
+                    mappingDefinition.Properties.Add(TemplateConstants.IgnoreCase, new PropertyValue(TemplateConstants.Boolean));
+                    mappingDefinition.Properties.Add(TemplateConstants.RequireNonEmpty, new PropertyValue(TemplateConstants.Boolean));
                     schema.Definitions.Add(TemplateConstants.StringDefinitionProperties, mappingDefinition);
 
                     // sequence-definition
                     mappingDefinition = new MappingDefinition();
-                    mappingDefinition.Properties.Add(TemplateConstants.Description, new PropertyValue(new StringToken(null, null, null, TemplateConstants.String)));
-                    mappingDefinition.Properties.Add(TemplateConstants.Context, new PropertyValue(new StringToken(null, null, null, TemplateConstants.SequenceOfNonEmptyString)));
-                    mappingDefinition.Properties.Add(TemplateConstants.Sequence, new PropertyValue(new StringToken(null, null, null, TemplateConstants.SequenceDefinitionProperties)));
+                    mappingDefinition.Properties.Add(TemplateConstants.Description, new PropertyValue(TemplateConstants.String));
+                    mappingDefinition.Properties.Add(TemplateConstants.Context, new PropertyValue(TemplateConstants.SequenceOfNonEmptyString));
+                    mappingDefinition.Properties.Add(TemplateConstants.Sequence, new PropertyValue(TemplateConstants.SequenceDefinitionProperties));
                     schema.Definitions.Add(TemplateConstants.SequenceDefinition, mappingDefinition);
 
                     // sequence-definition-properties
                     mappingDefinition = new MappingDefinition();
-                    mappingDefinition.Properties.Add(TemplateConstants.ItemType, new PropertyValue(new StringToken(null, null, null, TemplateConstants.NonEmptyString)));
+                    mappingDefinition.Properties.Add(TemplateConstants.ItemType, new PropertyValue(TemplateConstants.NonEmptyString));
                     schema.Definitions.Add(TemplateConstants.SequenceDefinitionProperties, mappingDefinition);
 
                     // mapping-definition
                     mappingDefinition = new MappingDefinition();
-                    mappingDefinition.Properties.Add(TemplateConstants.Description, new PropertyValue(new StringToken(null, null, null, TemplateConstants.String)));
-                    mappingDefinition.Properties.Add(TemplateConstants.Context, new PropertyValue(new StringToken(null, null, null, TemplateConstants.SequenceOfNonEmptyString)));
-                    mappingDefinition.Properties.Add(TemplateConstants.Mapping, new PropertyValue(new StringToken(null, null, null, TemplateConstants.MappingDefinitionProperties)));
+                    mappingDefinition.Properties.Add(TemplateConstants.Description, new PropertyValue(TemplateConstants.String));
+                    mappingDefinition.Properties.Add(TemplateConstants.Context, new PropertyValue(TemplateConstants.SequenceOfNonEmptyString));
+                    mappingDefinition.Properties.Add(TemplateConstants.Mapping, new PropertyValue(TemplateConstants.MappingDefinitionProperties));
                     schema.Definitions.Add(TemplateConstants.MappingDefinition, mappingDefinition);
 
                     // mapping-definition-properties
                     mappingDefinition = new MappingDefinition();
-                    mappingDefinition.Properties.Add(TemplateConstants.Properties, new PropertyValue(new StringToken(null, null, null, TemplateConstants.Properties)));
-                    mappingDefinition.Properties.Add(TemplateConstants.LooseKeyType, new PropertyValue(new StringToken(null, null, null, TemplateConstants.NonEmptyString)));
-                    mappingDefinition.Properties.Add(TemplateConstants.LooseValueType, new PropertyValue(new StringToken(null, null, null, TemplateConstants.NonEmptyString)));
+                    mappingDefinition.Properties.Add(TemplateConstants.Properties, new PropertyValue(TemplateConstants.Properties));
+                    mappingDefinition.Properties.Add(TemplateConstants.LooseKeyType, new PropertyValue(TemplateConstants.NonEmptyString));
+                    mappingDefinition.Properties.Add(TemplateConstants.LooseValueType, new PropertyValue(TemplateConstants.NonEmptyString));
                     schema.Definitions.Add(TemplateConstants.MappingDefinitionProperties, mappingDefinition);
 
                     // properties
                     mappingDefinition = new MappingDefinition();
                     mappingDefinition.LooseKeyType = TemplateConstants.NonEmptyString;
-                    mappingDefinition.LooseValueType = TemplateConstants.PropertyValue;
+                    mappingDefinition.LooseValueType = TemplateConstants.NonEmptyString;
                     schema.Definitions.Add(TemplateConstants.Properties, mappingDefinition);
 
-                    // property-value
-                    oneOfDefinition = new OneOfDefinition();
-                    oneOfDefinition.OneOf.Add(TemplateConstants.NonEmptyString);
-                    oneOfDefinition.OneOf.Add(TemplateConstants.MappingPropertyValue);
-                    schema.Definitions.Add(TemplateConstants.PropertyValue, oneOfDefinition);
-
-                    // mapping-property-value
-                    mappingDefinition = new MappingDefinition();
-                    mappingDefinition.Properties.Add(TemplateConstants.Type, new PropertyValue(new StringToken(null, null, null, TemplateConstants.NonEmptyString)));
-                    mappingDefinition.Properties.Add(TemplateConstants.Required, new PropertyValue(new StringToken(null, null, null, TemplateConstants.Boolean)));
-                    schema.Definitions.Add(TemplateConstants.MappingPropertyValue, mappingDefinition);
-
-
                     // one-of-definition
                     mappingDefinition = new MappingDefinition();
-                    mappingDefinition.Properties.Add(TemplateConstants.Description, new PropertyValue(new StringToken(null, null, null, TemplateConstants.String)));
-                    mappingDefinition.Properties.Add(TemplateConstants.Context, new PropertyValue(new StringToken(null, null, null, TemplateConstants.SequenceOfNonEmptyString)));
-                    mappingDefinition.Properties.Add(TemplateConstants.OneOf, new PropertyValue(new StringToken(null, null, null, TemplateConstants.SequenceOfNonEmptyString)));
+                    mappingDefinition.Properties.Add(TemplateConstants.Description, new PropertyValue(TemplateConstants.String));
+                    mappingDefinition.Properties.Add(TemplateConstants.Context, new PropertyValue(TemplateConstants.SequenceOfNonEmptyString));
+                    mappingDefinition.Properties.Add(TemplateConstants.OneOf, new PropertyValue(TemplateConstants.SequenceOfNonEmptyString));
                     schema.Definitions.Add(TemplateConstants.OneOfDefinition, mappingDefinition);
 
                     // non-empty-string
@@ -490,4 +477,4 @@ namespace GitHub.DistributedTask.ObjectTemplating.Schema
         private static readonly Regex s_definitionNameRegex = new Regex("^[a-zA-Z_][a-zA-Z0-9_-]*$", RegexOptions.Compiled);
         private static TemplateSchema s_schema;
     }
-}
+}

src/Sdk/DTObjectTemplating/ObjectTemplating/TemplateConstants.cs

@@ -22,11 +22,9 @@ namespace GitHub.DistributedTask.ObjectTemplating
         internal const String ItemType = "item-type";
         internal const String LooseKeyType = "loose-key-type";
         internal const String LooseValueType = "loose-value-type";
-        internal const String MaxConstant = "MAX";
         internal const String Mapping = "mapping";
         internal const String MappingDefinition = "mapping-definition";
         internal const String MappingDefinitionProperties = "mapping-definition-properties";
-        internal const String MappingPropertyValue = "mapping-property-value";
         internal const String NonEmptyString = "non-empty-string";
         internal const String Null = "null";
         internal const String NullDefinition = "null-definition";
@@ -37,9 +35,7 @@ namespace GitHub.DistributedTask.ObjectTemplating
         internal const String OneOf = "one-of";
         internal const String OneOfDefinition = "one-of-definition";
         internal const String OpenExpression = "${{";
-        internal const String PropertyValue = "property-value";
         internal const String Properties = "properties";
-        internal const String Required = "required";
         internal const String RequireNonEmpty = "require-non-empty";
         internal const String Scalar = "scalar";
         internal const String ScalarDefinition = "scalar-definition";
@@ -47,7 +43,6 @@ namespace GitHub.DistributedTask.ObjectTemplating
         internal const String Sequence = "sequence";
         internal const String SequenceDefinition = "sequence-definition";
         internal const String SequenceDefinitionProperties = "sequence-definition-properties";
-        internal const String Type = "type";
         internal const String SequenceOfNonEmptyString = "sequence-of-non-empty-string";
         internal const String String = "string";
         internal const String StringDefinition = "string-definition";

src/Sdk/DTObjectTemplating/ObjectTemplating/TemplateContext.cs

@@ -184,7 +184,6 @@ namespace GitHub.DistributedTask.ObjectTemplating
                 id = FileIds.Count + 1;
                 FileIds.Add(file, id);
                 FileNames.Add(file);
-                Memory.AddBytes(file);
             }
 
             return id;
@@ -192,12 +191,7 @@ namespace GitHub.DistributedTask.ObjectTemplating
 
         internal String GetFileName(Int32 fileId)
         {
-            return FileNames.Count >= fileId ? FileNames[fileId - 1] : null;
-        }
-
-        internal IReadOnlyList<String> GetFileTable()
-        {
-            return FileNames.AsReadOnly();
+            return FileNames[fileId - 1];
         }
 
         private String GetErrorPrefix(
@@ -205,9 +199,9 @@ namespace GitHub.DistributedTask.ObjectTemplating
             Int32? line,
             Int32? column)
         {
-            var fileName = fileId.HasValue ? GetFileName(fileId.Value) : null;
-            if (!String.IsNullOrEmpty(fileName))
+            if (fileId != null)
             {
+                var fileName = GetFileName(fileId.Value);
                 if (line != null && column != null)
                 {
                     return $"{fileName} {TemplateStrings.LineColumn(line, column)}:";

src/Sdk/DTObjectTemplating/ObjectTemplating/TemplateEvaluator.cs

@@ -47,16 +47,7 @@ namespace GitHub.DistributedTask.ObjectTemplating
             var evaluator = new TemplateEvaluator(context, template, removeBytes);
             try
             {
-                var availableContext = new HashSet<String>(StringComparer.OrdinalIgnoreCase);
-                foreach (var key in context.ExpressionValues.Keys)
-                {
-                    availableContext.Add(key);
-                }
-                foreach (var function in context.ExpressionFunctions)
-                {
-                    availableContext.Add($"{function.Name}()");
-                }
-
+                var availableContext = new HashSet<String>(context.ExpressionValues.Keys);
                 var definitionInfo = new DefinitionInfo(context.Schema, type, availableContext);
                 result = evaluator.Evaluate(definitionInfo);
 
@@ -191,14 +182,12 @@ namespace GitHub.DistributedTask.ObjectTemplating
             }
 
             var keys = new HashSet<String>(StringComparer.OrdinalIgnoreCase);
-            var hasExpressionKey = false;
 
             while (m_unraveler.AllowScalar(definition.Expand, out ScalarToken nextKeyScalar))
             {
                 // Expression
                 if (nextKeyScalar is ExpressionToken)
                 {
-                    hasExpressionKey = true;
                     var anyDefinition = new DefinitionInfo(definition, TemplateConstants.Any);
                     mapping.Add(nextKeyScalar, Evaluate(anyDefinition));
                     continue;
@@ -279,19 +268,6 @@ namespace GitHub.DistributedTask.ObjectTemplating
                 String listToDeDuplicate = String.Join(", ", nonDuplicates);
                 m_context.Error(mapping, TemplateStrings.UnableToDetermineOneOf(listToDeDuplicate));
             }
-            else if (mappingDefinitions.Count == 1 && !hasExpressionKey)
-            {
-                foreach (var property in mappingDefinitions[0].Properties)
-                {
-                    if (property.Value.Required)
-                    {
-                        if (!keys.Contains(property.Key))
-                        {
-                            m_context.Error(mapping, $"Required property is missing: {property.Key}");
-                        }
-                    }
-                }
-            }
 
             m_unraveler.ReadMappingEnd();
         }
@@ -402,13 +378,14 @@ namespace GitHub.DistributedTask.ObjectTemplating
                 Definition = m_schema.GetDefinition(name);
 
                 // Determine whether to expand
-                m_allowedContext = Definition.EvaluatorContext;
-                if (Definition.EvaluatorContext.Length > 0)
+                if (Definition.Context.Length > 0)
                 {
+                    m_allowedContext = Definition.Context;
                     Expand = m_availableContext.IsSupersetOf(m_allowedContext);
                 }
                 else
                 {
+                    m_allowedContext = new String[0];
                     Expand = false;
                 }
             }
@@ -424,9 +401,9 @@ namespace GitHub.DistributedTask.ObjectTemplating
                 Definition = m_schema.GetDefinition(name);
 
                 // Determine whether to expand
-                if (Definition.EvaluatorContext.Length > 0)
+                if (Definition.Context.Length > 0)
                 {
-                    m_allowedContext = new HashSet<String>(parent.m_allowedContext.Concat(Definition.EvaluatorContext), StringComparer.OrdinalIgnoreCase).ToArray();
+                    m_allowedContext = new HashSet<String>(parent.m_allowedContext.Concat(Definition.Context)).ToArray();
                     Expand = m_availableContext.IsSupersetOf(m_allowedContext);
                 }
                 else

src/Sdk/DTObjectTemplating/ObjectTemplating/TemplateException.cs

@@ -49,14 +49,6 @@ namespace GitHub.DistributedTask.ObjectTemplating
             m_errors = new List<TemplateValidationError>(errors ?? Enumerable.Empty<TemplateValidationError>());
         }
 
-        public TemplateValidationException(
-            String message,
-            IEnumerable<TemplateValidationError> errors)
-            : this(message)
-        {
-            m_errors = new List<TemplateValidationError>(errors ?? Enumerable.Empty<TemplateValidationError>());
-        }
-
         public TemplateValidationException(String message)
             : base(message)
         {

src/Sdk/DTObjectTemplating/ObjectTemplating/TemplateReader.cs

@@ -178,15 +178,14 @@ namespace GitHub.DistributedTask.ObjectTemplating
             }
 
             var keys = new HashSet<String>(StringComparer.OrdinalIgnoreCase);
-            var hasExpressionKey = false;
 
             while (m_objectReader.AllowLiteral(out LiteralToken rawLiteral))
             {
                 var nextKeyScalar = ParseScalar(rawLiteral, definition.AllowedContext);
+
                 // Expression
                 if (nextKeyScalar is ExpressionToken)
                 {
-                    hasExpressionKey = true;
                     // Legal
                     if (definition.AllowedContext.Length > 0)
                     {
@@ -281,19 +280,7 @@ namespace GitHub.DistributedTask.ObjectTemplating
                 String listToDeDuplicate = String.Join(", ", nonDuplicates);
                 m_context.Error(mapping, TemplateStrings.UnableToDetermineOneOf(listToDeDuplicate));
             }
-            else if (mappingDefinitions.Count == 1 && !hasExpressionKey)
-            {
-                foreach (var property in mappingDefinitions[0].Properties)
-                {
-                    if (property.Value.Required)
-                    {
-                        if (!keys.Contains(property.Key))
-                        {
-                            m_context.Error(mapping, $"Required property is missing: {property.Key}");
-                        }
-                    }
-                }
-            }
+
             ExpectMappingEnd();
         }
 
@@ -780,8 +767,15 @@ namespace GitHub.DistributedTask.ObjectTemplating
                 // Lookup the definition
                 Definition = m_schema.GetDefinition(name);
 
-                // Record allowed context
-                AllowedContext = Definition.ReaderContext;
+                // Determine whether to expand
+                if (Definition.Context.Length > 0)
+                {
+                    AllowedContext = Definition.Context;
+                }
+                else
+                {
+                    AllowedContext = new String[0];
+                }
             }
 
             public DefinitionInfo(
@@ -793,10 +787,10 @@ namespace GitHub.DistributedTask.ObjectTemplating
                 // Lookup the definition
                 Definition = m_schema.GetDefinition(name);
 
-                // Record allowed context
-                if (Definition.ReaderContext.Length > 0)
+                // Determine whether to expand
+                if (Definition.Context.Length > 0)
                 {
-                    AllowedContext = new HashSet<String>(parent.AllowedContext.Concat(Definition.ReaderContext), StringComparer.OrdinalIgnoreCase).ToArray();
+                    AllowedContext = new HashSet<String>(parent.AllowedContext.Concat(Definition.Context)).ToArray();
                 }
                 else
                 {

src/Sdk/DTObjectTemplating/ObjectTemplating/TemplateValidationErrors.cs

@@ -2,7 +2,6 @@
 using System.Collections;
 using System.Collections.Generic;
 using System.ComponentModel;
-using System.Linq;
 using System.Runtime.Serialization;
 
 namespace GitHub.DistributedTask.ObjectTemplating
@@ -42,7 +41,7 @@ namespace GitHub.DistributedTask.ObjectTemplating
         {
             for (int i = 0; i < 50; i++)
             {
-                String message = !String.IsNullOrEmpty(messagePrefix) ? $"{messagePrefix} {ex.Message}" : ex.ToString();
+                String message = !String.IsNullOrEmpty(messagePrefix) ? $"{messagePrefix} {ex.Message}" : ex.Message;
                 Add(new TemplateValidationError(message));
                 if (ex.InnerException == null)
                 {
@@ -89,23 +88,6 @@ namespace GitHub.DistributedTask.ObjectTemplating
             }
         }
 
-        /// <summary>
-        /// Throws <c ref="TemplateValidationException" /> if any errors.
-        /// <param name="prefix">The error message prefix</param>
-        /// </summary>
-        public void Check(String prefix)
-        {
-            if (String.IsNullOrEmpty(prefix))
-            {
-                this.Check();
-            }
-            else if (m_errors.Count > 0)
-            {
-                var message = $"{prefix.Trim()} {String.Join(",", m_errors.Select(e => e.Message))}";
-                throw new TemplateValidationException(message, m_errors);
-            }
-        }
-
         public void Clear()
         {
             m_errors.Clear();

src/Sdk/DTObjectTemplating/ObjectTemplating/Tokens/ExpressionToken.cs

@@ -1,9 +1,7 @@
 using System;
-using System.Collections.Generic;
 using System.ComponentModel;
-using System.Globalization;
+using System.Linq;
 using System.Runtime.Serialization;
-using System.Text.RegularExpressions;
 using GitHub.DistributedTask.Expressions2;
 using GitHub.DistributedTask.Expressions2.Sdk;
 using GitHub.Services.WebApi.Internal;
@@ -37,29 +35,11 @@ namespace GitHub.DistributedTask.ObjectTemplating.Tokens
             String[] allowedContext,
             out Exception ex)
         {
-            // Create dummy named values and functions
-            var namedValues = new List<INamedValueInfo>();
-            var functions = new List<IFunctionInfo>();
+            // Create dummy allowed contexts
+            INamedValueInfo[] namedValues = null;
             if (allowedContext?.Length > 0)
             {
-                foreach (var contextItem in allowedContext)
-                {
-                    var match = s_function.Match(contextItem);
-                    if (match.Success)
-                    {
-                        var functionName = match.Groups[1].Value;
-                        var minParameters = Int32.Parse(match.Groups[2].Value, NumberStyles.None, CultureInfo.InvariantCulture);
-                        var maxParametersRaw = match.Groups[3].Value;
-                        var maxParameters = String.Equals(maxParametersRaw, TemplateConstants.MaxConstant, StringComparison.Ordinal)
-                            ? Int32.MaxValue
-                            : Int32.Parse(maxParametersRaw, NumberStyles.None, CultureInfo.InvariantCulture);
-                        functions.Add(new FunctionInfo<DummyFunction>(functionName, minParameters, maxParameters));
-                    }
-                    else
-                    {
-                        namedValues.Add(new NamedValueInfo<ContextValueNode>(contextItem));
-                    }
-                }
+                namedValues = allowedContext.Select(x => new NamedValueInfo<ContextValueNode>(x)).ToArray();
             }
 
             // Parse
@@ -67,7 +47,7 @@ namespace GitHub.DistributedTask.ObjectTemplating.Tokens
             ExpressionNode root = null;
             try
             {
-                root = new ExpressionParser().CreateTree(expression, null, namedValues, functions) as ExpressionNode;
+                root = new ExpressionParser().CreateTree(expression, null, namedValues, null) as ExpressionNode;
 
                 result = true;
                 ex = null;
@@ -80,18 +60,5 @@ namespace GitHub.DistributedTask.ObjectTemplating.Tokens
 
             return result;
         }
-
-        private sealed class DummyFunction : Function
-        {
-            protected override Object EvaluateCore(
-                EvaluationContext context,
-                out ResultMemory resultMemory)
-            {
-                resultMemory = null;
-                return null;
-            }
-        }
-
-        private static readonly Regex s_function = new Regex(@"^([a-zA-Z0-9_]+)\(([0-9]+),([0-9]+|MAX)\)$", RegexOptions.Compiled);
     }
 }

src/Sdk/DTObjectTemplating/ObjectTemplating/Tokens/TemplateToken.cs

@@ -30,14 +30,14 @@ namespace GitHub.DistributedTask.ObjectTemplating.Tokens
             Column = column;
         }
 
-        [DataMember(Name = "file", EmitDefaultValue = false)]
-        internal Int32? FileId { get; private set; }
+        [IgnoreDataMember]
+        internal Int32? FileId { get; set; }
 
         [DataMember(Name = "line", EmitDefaultValue = false)]
-        internal Int32? Line { get; private set; }
+        internal Int32? Line { get; }
 
         [DataMember(Name = "col", EmitDefaultValue = false)]
-        internal Int32? Column { get; private set; }
+        internal Int32? Column { get; }
 
         [DataMember(Name = "type", EmitDefaultValue = false)]
         internal Int32 Type { get; }

src/Sdk/DTObjectTemplating/ObjectTemplating/Tokens/TemplateTokenExtensions.cs

@@ -1,8 +1,5 @@
 using System;
 using System.Collections.Generic;
-using System.Linq;
-using GitHub.DistributedTask.Expressions2;
-using GitHub.DistributedTask.Expressions2.Sdk;
 
 namespace GitHub.DistributedTask.ObjectTemplating.Tokens
 {
@@ -109,43 +106,6 @@ namespace GitHub.DistributedTask.ObjectTemplating.Tokens
             throw new ArgumentException($"Error while reading '{objectDescription}'. Unexpected value '{literal.ToString()}'");
         }
 
-        /// <summary>
-        /// Traverses the token and checks whether all required expression values
-        /// and functions are provided.
-        /// </summary>
-        public static bool CheckHasRequiredContext(
-            this TemplateToken token,
-            IReadOnlyObject expressionValues,
-            IList<IFunctionInfo> expressionFunctions)
-        {
-            var expressionTokens = token.Traverse()
-                .OfType<BasicExpressionToken>()
-                .ToArray();
-            var parser = new ExpressionParser();
-            foreach (var expressionToken in expressionTokens)
-            {
-                var tree = parser.ValidateSyntax(expressionToken.Expression, null);
-                foreach (var node in tree.Traverse())
-                {
-                    if (node is NamedValue namedValue)
-                    {
-                        if (expressionValues?.Keys.Any(x => string.Equals(x, namedValue.Name, StringComparison.OrdinalIgnoreCase)) != true)
-                        {
-                            return false;
-                        }
-                    }
-                    else if (node is Function function &&
-                        !ExpressionConstants.WellKnownFunctions.ContainsKey(function.Name) &&
-                        expressionFunctions?.Any(x => string.Equals(x.Name, function.Name, StringComparison.OrdinalIgnoreCase)) != true)
-                    {
-                        return false;
-                    }
-                }
-            }
-
-            return true;
-        }
-
         /// <summary>
         /// Returns all tokens (depth first)
         /// </summary>

src/Sdk/DTObjectTemplating/ObjectTemplating/Tokens/TemplateTokenJsonConverter.cs

@@ -115,12 +115,13 @@ namespace GitHub.DistributedTask.ObjectTemplating.Tokens
             Object value,
             JsonSerializer serializer)
         {
+            base.WriteJson(writer, value, serializer);
             if (value is TemplateToken token)
             {
                 switch (token.Type)
                 {
                     case TokenType.Null:
-                        if (token.FileId == null && token.Line == null && token.Column == null)
+                        if (token.Line == null && token.Column == null)
                         {
                             writer.WriteNull();
                         }
@@ -129,17 +130,12 @@ namespace GitHub.DistributedTask.ObjectTemplating.Tokens
                             writer.WriteStartObject();
                             writer.WritePropertyName("type");
                             writer.WriteValue(token.Type);
-                            if (token.FileId != null)
-                            {
-                                writer.WritePropertyName("file");
-                                writer.WriteValue(token.FileId);
-                            }
                             if (token.Line != null)
                             {
                                 writer.WritePropertyName("line");
                                 writer.WriteValue(token.Line);
                             }
-                            if (token.Column != null)
+                            if (token.Line != null)
                             {
                                 writer.WritePropertyName("col");
                                 writer.WriteValue(token.Column);
@@ -150,7 +146,7 @@ namespace GitHub.DistributedTask.ObjectTemplating.Tokens
 
                     case TokenType.Boolean:
                         var booleanToken = token as BooleanToken;
-                        if (token.FileId == null && token.Line == null && token.Column == null)
+                        if (token.Line == null && token.Column == null)
                         {
                             writer.WriteValue(booleanToken.Value);
                         }
@@ -159,17 +155,12 @@ namespace GitHub.DistributedTask.ObjectTemplating.Tokens
                             writer.WriteStartObject();
                             writer.WritePropertyName("type");
                             writer.WriteValue(token.Type);
-                            if (token.FileId != null)
-                            {
-                                writer.WritePropertyName("file");
-                                writer.WriteValue(token.FileId);
-                            }
                             if (token.Line != null)
                             {
                                 writer.WritePropertyName("line");
                                 writer.WriteValue(token.Line);
                             }
-                            if (token.Column != null)
+                            if (token.Line != null)
                             {
                                 writer.WritePropertyName("col");
                                 writer.WriteValue(token.Column);
@@ -182,7 +173,7 @@ namespace GitHub.DistributedTask.ObjectTemplating.Tokens
 
                     case TokenType.Number:
                         var numberToken = token as NumberToken;
-                        if (token.FileId == null && token.Line == null && token.Column == null)
+                        if (token.Line == null && token.Column == null)
                         {
                             writer.WriteValue(numberToken.Value);
                         }
@@ -191,17 +182,12 @@ namespace GitHub.DistributedTask.ObjectTemplating.Tokens
                             writer.WriteStartObject();
                             writer.WritePropertyName("type");
                             writer.WriteValue(token.Type);
-                            if (token.FileId != null)
-                            {
-                                writer.WritePropertyName("file");
-                                writer.WriteValue(token.FileId);
-                            }
                             if (token.Line != null)
                             {
                                 writer.WritePropertyName("line");
                                 writer.WriteValue(token.Line);
                             }
-                            if (token.Column != null)
+                            if (token.Line != null)
                             {
                                 writer.WritePropertyName("col");
                                 writer.WriteValue(token.Column);
@@ -214,7 +200,7 @@ namespace GitHub.DistributedTask.ObjectTemplating.Tokens
 
                     case TokenType.String:
                         var stringToken = token as StringToken;
-                        if (token.FileId == null && token.Line == null && token.Column == null)
+                        if (token.Line == null && token.Column == null)
                         {
                             writer.WriteValue(stringToken.Value);
                         }
@@ -223,17 +209,12 @@ namespace GitHub.DistributedTask.ObjectTemplating.Tokens
                             writer.WriteStartObject();
                             writer.WritePropertyName("type");
                             writer.WriteValue(token.Type);
-                            if (token.FileId != null)
-                            {
-                                writer.WritePropertyName("file");
-                                writer.WriteValue(token.FileId);
-                            }
                             if (token.Line != null)
                             {
                                 writer.WritePropertyName("line");
                                 writer.WriteValue(token.Line);
                             }
-                            if (token.Column != null)
+                            if (token.Line != null)
                             {
                                 writer.WritePropertyName("col");
                                 writer.WriteValue(token.Column);
@@ -249,17 +230,12 @@ namespace GitHub.DistributedTask.ObjectTemplating.Tokens
                         writer.WriteStartObject();
                         writer.WritePropertyName("type");
                         writer.WriteValue(token.Type);
-                        if (token.FileId != null)
-                        {
-                            writer.WritePropertyName("file");
-                            writer.WriteValue(token.FileId);
-                        }
                         if (token.Line != null)
                         {
                             writer.WritePropertyName("line");
                             writer.WriteValue(token.Line);
                         }
-                        if (token.Column != null)
+                        if (token.Line != null)
                         {
                             writer.WritePropertyName("col");
                             writer.WriteValue(token.Column);
@@ -277,17 +253,12 @@ namespace GitHub.DistributedTask.ObjectTemplating.Tokens
                         writer.WriteStartObject();
                         writer.WritePropertyName("type");
                         writer.WriteValue(token.Type);
-                        if (token.FileId != null)
-                        {
-                            writer.WritePropertyName("file");
-                            writer.WriteValue(token.FileId);
-                        }
                         if (token.Line != null)
                         {
                             writer.WritePropertyName("line");
                             writer.WriteValue(token.Line);
                         }
-                        if (token.Column != null)
+                        if (token.Line != null)
                         {
                             writer.WritePropertyName("col");
                             writer.WriteValue(token.Column);
@@ -302,17 +273,12 @@ namespace GitHub.DistributedTask.ObjectTemplating.Tokens
                         writer.WriteStartObject();
                         writer.WritePropertyName("type");
                         writer.WriteValue(token.Type);
-                        if (token.FileId != null)
-                        {
-                            writer.WritePropertyName("file");
-                            writer.WriteValue(token.FileId);
-                        }
                         if (token.Line != null)
                         {
                             writer.WritePropertyName("line");
                             writer.WriteValue(token.Line);
                         }
-                        if (token.Column != null)
+                        if (token.Line != null)
                         {
                             writer.WritePropertyName("col");
                             writer.WriteValue(token.Column);
@@ -335,17 +301,12 @@ namespace GitHub.DistributedTask.ObjectTemplating.Tokens
                         writer.WriteStartObject();
                         writer.WritePropertyName("type");
                         writer.WriteValue(token.Type);
-                        if (token.FileId != null)
-                        {
-                            writer.WritePropertyName("file");
-                            writer.WriteValue(token.FileId);
-                        }
                         if (token.Line != null)
                         {
                             writer.WritePropertyName("line");
                             writer.WriteValue(token.Line);
                         }
-                        if (token.Column != null)
+                        if (token.Line != null)
                         {
                             writer.WritePropertyName("col");
                             writer.WriteValue(token.Column);

src/Sdk/DTPipelines/Pipelines/AgentJobRequestMessage.cs

@@ -39,10 +39,7 @@ namespace GitHub.DistributedTask.Pipelines
             DictionaryContextData contextData,
             WorkspaceOptions workspaceOptions,
             IEnumerable<JobStep> steps,
-            IEnumerable<ContextScope> scopes,
-            IList<String> fileTable,
-            TemplateToken jobOutputs,
-            IList<TemplateToken> defaults)
+            IEnumerable<ContextScope> scopes)
         {
             this.MessageType = JobRequestMessageTypes.PipelineAgentJobRequest;
             this.Plan = plan;
@@ -54,7 +51,6 @@ namespace GitHub.DistributedTask.Pipelines
             this.Timeline = timeline;
             this.Resources = jobResources;
             this.Workspace = workspaceOptions;
-            this.JobOutputs = jobOutputs;
 
             m_variables = new Dictionary<String, VariableValue>(variables, StringComparer.OrdinalIgnoreCase);
             m_maskHints = new List<MaskHint>(maskHints);
@@ -70,11 +66,6 @@ namespace GitHub.DistributedTask.Pipelines
                 m_environmentVariables = new List<TemplateToken>(environmentVariables);
             }
 
-            if (defaults?.Count > 0)
-            {
-                m_defaults = new List<TemplateToken>(defaults);
-            }
-
             this.ContextData = new Dictionary<String, PipelineContextData>(StringComparer.OrdinalIgnoreCase);
             if (contextData?.Count > 0)
             {
@@ -83,11 +74,6 @@ namespace GitHub.DistributedTask.Pipelines
                     this.ContextData[pair.Key] = pair.Value;
                 }
             }
-
-            if (fileTable?.Count > 0)
-            {
-                m_fileTable = new List<String>(fileTable);
-            }
         }
 
         [DataMember]
@@ -146,13 +132,6 @@ namespace GitHub.DistributedTask.Pipelines
             private set;
         }
 
-        [DataMember(EmitDefaultValue = false)]
-        public TemplateToken JobOutputs
-        {
-            get;
-            private set;
-        }
-
         [DataMember]
         public Int64 RequestId
         {
@@ -219,21 +198,6 @@ namespace GitHub.DistributedTask.Pipelines
             }
         }
 
-        /// <summary>
-        /// Gets the hierarchy of defaults to overlay, last wins.
-        /// </summary>
-        public IList<TemplateToken> Defaults
-        {
-            get
-            {
-                if (m_defaults == null)
-                {
-                    m_defaults = new List<TemplateToken>();
-                }
-                return m_defaults;
-            }
-        }
-
         /// <summary>
         /// Gets the collection of variables associated with the current context.
         /// </summary>
@@ -273,21 +237,6 @@ namespace GitHub.DistributedTask.Pipelines
             }
         }
 
-        /// <summary>
-        /// Gets the table of files used when parsing the pipeline (e.g. yaml files)
-        /// </summary>
-        public IList<String> FileTable
-        {
-            get
-            {
-                if (m_fileTable == null)
-                {
-                    m_fileTable = new List<String>();
-                }
-                return m_fileTable;
-            }
-        }
-
         // todo: remove after feature-flag DistributedTask.EvaluateContainerOnRunner is enabled everywhere
         public void SetJobSidecarContainers(IDictionary<String, String> value)
         {
@@ -396,16 +345,6 @@ namespace GitHub.DistributedTask.Pipelines
                 m_environmentVariables = null;
             }
 
-            if (m_defaults?.Count == 0)
-            {
-                m_defaults = null;
-            }
-
-            if (m_fileTable?.Count == 0)
-            {
-                m_fileTable = null;
-            }
-
             if (m_maskHints?.Count == 0)
             {
                 m_maskHints = null;
@@ -435,12 +374,6 @@ namespace GitHub.DistributedTask.Pipelines
         [DataMember(Name = "EnvironmentVariables", EmitDefaultValue = false)]
         private List<TemplateToken> m_environmentVariables;
 
-        [DataMember(Name = "Defaults", EmitDefaultValue = false)]
-        private List<TemplateToken> m_defaults;
-
-        [DataMember(Name = "FileTable", EmitDefaultValue = false)]
-        private List<String> m_fileTable;
-
         [DataMember(Name = "Mask", EmitDefaultValue = false)]
         private List<MaskHint> m_maskHints;
 

src/Sdk/DTPipelines/Pipelines/ContextData/PipelineContextDataExtensions.cs

@@ -35,19 +35,6 @@ namespace GitHub.DistributedTask.Pipelines.ContextData
             throw new ArgumentException($"Unexpected type '{value?.GetType().Name}' encountered while reading '{objectDescription}'. The type '{nameof(DictionaryContextData)}' was expected.");
         }
 
-        [EditorBrowsable(EditorBrowsableState.Never)]
-        public static CaseSensitiveDictionaryContextData AssertCaseSensitiveDictionary(
-                    this PipelineContextData value,
-                    String objectDescription)
-        {
-            if (value is CaseSensitiveDictionaryContextData dictionary)
-            {
-                return dictionary;
-            }
-
-            throw new ArgumentException($"Unexpected type '{value?.GetType().Name}' encountered while reading '{objectDescription}'. The type '{nameof(CaseSensitiveDictionaryContextData)}' was expected.");
-        }
-
         [EditorBrowsable(EditorBrowsableState.Never)]
         public static StringContextData AssertString(
             this PipelineContextData value,

src/Sdk/DTPipelines/Pipelines/ObjectTemplating/PipelineTemplateConstants.cs

@@ -14,7 +14,6 @@ namespace GitHub.DistributedTask.Pipelines.ObjectTemplating
         public const String Clean = "clean";
         public const String Container = "container";
         public const String ContinueOnError = "continue-on-error";
-        public const String Defaults = "defaults";
         public const String Env = "env";
         public const String Event = "event";
         public const String EventPattern = "github.event";
@@ -24,18 +23,13 @@ namespace GitHub.DistributedTask.Pipelines.ObjectTemplating
         public const String FetchDepth = "fetch-depth";
         public const String GeneratedId = "generated-id";
         public const String GitHub = "github";
-        public const String HashFiles = "hashFiles";
         public const String Id = "id";
         public const String If = "if";
         public const String Image = "image";
         public const String Include = "include";
         public const String Inputs = "inputs";
         public const String Job = "job";
-        public const String JobDefaultsRun = "job-defaults-run";
-        public const String JobIfResult = "job-if-result";
-        public const String JobOutputs = "job-outputs";
         public const String Jobs = "jobs";
-        public const String Labels = "labels";
         public const String Lfs = "lfs";
         public const String Matrix = "matrix";
         public const String MaxParallel = "max-parallel";
@@ -62,7 +56,6 @@ namespace GitHub.DistributedTask.Pipelines.ObjectTemplating
         public const String Shell = "shell";
         public const String Skipped = "skipped";
         public const String StepEnv = "step-env";
-        public const String StepIfResult = "step-if-result";
         public const String Steps = "steps";
         public const String StepsScopeInputs = "steps-scope-inputs";
         public const String StepsScopeOutputs = "steps-scope-outputs";

src/Sdk/DTPipelines/Pipelines/ObjectTemplating/PipelineTemplateConverter.cs

@@ -16,20 +16,6 @@ namespace GitHub.DistributedTask.Pipelines.ObjectTemplating
 {
     internal static class PipelineTemplateConverter
     {
-        internal static Boolean ConvertToIfResult(
-            TemplateContext context,
-            TemplateToken ifResult)
-        {
-            var expression = ifResult.Traverse().FirstOrDefault(x => x is ExpressionToken);
-            if (expression != null)
-            {
-                throw new ArgumentException($"Unexpected type '{expression.GetType().Name}' encountered while reading 'if'.");
-            }
-
-            var evaluationResult = EvaluationResult.CreateIntermediateResult(null, ifResult);
-            return evaluationResult.IsTruthy;
-        }
-
         internal static Boolean? ConvertToStepContinueOnError(
             TemplateContext context,
             TemplateToken token,
@@ -168,11 +154,6 @@ namespace GitHub.DistributedTask.Pipelines.ObjectTemplating
 
             if (value is StringToken containerLiteral)
             {
-                if (String.IsNullOrEmpty(containerLiteral.Value))
-                {
-                    return null;
-                }
-
                 result.Image = containerLiteral.Value;
             }
             else

src/Sdk/DTPipelines/Pipelines/ObjectTemplating/PipelineTemplateEvaluator.cs

@@ -4,7 +4,7 @@ using System.ComponentModel;
 using System.Linq;
 using System.Threading;
 using GitHub.DistributedTask.Expressions2;
-using GitHub.DistributedTask.Expressions2.Sdk.Functions;
+using GitHub.DistributedTask.Expressions2.Sdk;
 using GitHub.DistributedTask.ObjectTemplating;
 using GitHub.DistributedTask.ObjectTemplating.Schema;
 using GitHub.DistributedTask.ObjectTemplating.Tokens;
@@ -14,16 +14,12 @@ using ITraceWriter = GitHub.DistributedTask.ObjectTemplating.ITraceWriter;
 
 namespace GitHub.DistributedTask.Pipelines.ObjectTemplating
 {
-    /// <summary>
-    /// Evaluates parts of the workflow DOM. For example, a job strategy or step inputs.
-    /// </summary>
     [EditorBrowsable(EditorBrowsableState.Never)]
     public class PipelineTemplateEvaluator
     {
         public PipelineTemplateEvaluator(
             ITraceWriter trace,
-            TemplateSchema schema,
-            IList<String> fileTable)
+            TemplateSchema schema)
         {
             if (!String.Equals(schema.Version, PipelineTemplateConstants.Workflow_1_0, StringComparison.Ordinal))
             {
@@ -32,7 +28,6 @@ namespace GitHub.DistributedTask.Pipelines.ObjectTemplating
 
             m_trace = trace;
             m_schema = schema;
-            m_fileTable = fileTable;
         }
 
         public Int32 MaxDepth => 50;
@@ -53,14 +48,13 @@ namespace GitHub.DistributedTask.Pipelines.ObjectTemplating
 
         public DictionaryContextData EvaluateStepScopeInputs(
             TemplateToken token,
-            DictionaryContextData contextData,
-            IList<IFunctionInfo> expressionFunctions)
+            DictionaryContextData contextData)
         {
             var result = default(DictionaryContextData);
 
             if (token != null && token.Type != TokenType.Null)
             {
-                var context = CreateContext(contextData, expressionFunctions);
+                var context = CreateContext(contextData);
                 try
                 {
                     token = TemplateEvaluator.Evaluate(context, PipelineTemplateConstants.StepsScopeInputs, token, 0, null, omitHeader: true);
@@ -80,14 +74,13 @@ namespace GitHub.DistributedTask.Pipelines.ObjectTemplating
 
         public DictionaryContextData EvaluateStepScopeOutputs(
             TemplateToken token,
-            DictionaryContextData contextData,
-            IList<IFunctionInfo> expressionFunctions)
+            DictionaryContextData contextData)
         {
             var result = default(DictionaryContextData);
 
             if (token != null && token.Type != TokenType.Null)
             {
-                var context = CreateContext(contextData, expressionFunctions);
+                var context = CreateContext(contextData);
                 try
                 {
                     token = TemplateEvaluator.Evaluate(context, PipelineTemplateConstants.StepsScopeOutputs, token, 0, null, omitHeader: true);
@@ -107,14 +100,13 @@ namespace GitHub.DistributedTask.Pipelines.ObjectTemplating
 
         public Boolean EvaluateStepContinueOnError(
             TemplateToken token,
-            DictionaryContextData contextData,
-            IList<IFunctionInfo> expressionFunctions)
+            DictionaryContextData contextData)
         {
             var result = default(Boolean?);
 
             if (token != null && token.Type != TokenType.Null)
             {
-                var context = CreateContext(contextData, expressionFunctions);
+                var context = CreateContext(contextData);
                 try
                 {
                     token = TemplateEvaluator.Evaluate(context, PipelineTemplateConstants.BooleanStepsContext, token, 0, null, omitHeader: true);
@@ -132,44 +124,16 @@ namespace GitHub.DistributedTask.Pipelines.ObjectTemplating
             return result ?? false;
         }
 
-        public String EvaluateStepDisplayName(
-            TemplateToken token,
-            DictionaryContextData contextData,
-            IList<IFunctionInfo> expressionFunctions)
-        {
-            var result = default(String);
-
-            if (token != null && token.Type != TokenType.Null)
-            {
-                var context = CreateContext(contextData, expressionFunctions);
-                try
-                {
-                    token = TemplateEvaluator.Evaluate(context, PipelineTemplateConstants.StringStepsContext, token, 0, null, omitHeader: true);
-                    context.Errors.Check();
-                    result = PipelineTemplateConverter.ConvertToStepDisplayName(context, token);
-                }
-                catch (Exception ex) when (!(ex is TemplateValidationException))
-                {
-                    context.Errors.Add(ex);
-                }
-
-                context.Errors.Check();
-            }
-
-            return result;
-        }
-
         public Dictionary<String, String> EvaluateStepEnvironment(
             TemplateToken token,
             DictionaryContextData contextData,
-            IList<IFunctionInfo> expressionFunctions,
             StringComparer keyComparer)
         {
             var result = default(Dictionary<String, String>);
 
             if (token != null && token.Type != TokenType.Null)
             {
-                var context = CreateContext(contextData, expressionFunctions);
+                var context = CreateContext(contextData);
                 try
                 {
                     token = TemplateEvaluator.Evaluate(context, PipelineTemplateConstants.StepEnv, token, 0, null, omitHeader: true);
@@ -187,44 +151,15 @@ namespace GitHub.DistributedTask.Pipelines.ObjectTemplating
             return result ?? new Dictionary<String, String>(keyComparer);
         }
 
-        public Boolean EvaluateStepIf(
-            TemplateToken token,
-            DictionaryContextData contextData,
-            IList<IFunctionInfo> expressionFunctions,
-            IEnumerable<KeyValuePair<String, Object>> expressionState)
-        {
-            var result = default(Boolean?);
-
-            if (token != null && token.Type != TokenType.Null)
-            {
-                var context = CreateContext(contextData, expressionFunctions, expressionState);
-                try
-                {
-                    token = TemplateEvaluator.Evaluate(context, PipelineTemplateConstants.StepIfResult, token, 0, null, omitHeader: true);
-                    context.Errors.Check();
-                    result = PipelineTemplateConverter.ConvertToIfResult(context, token);
-                }
-                catch (Exception ex) when (!(ex is TemplateValidationException))
-                {
-                    context.Errors.Add(ex);
-                }
-
-                context.Errors.Check();
-            }
-
-            return result ?? throw new InvalidOperationException("Step if cannot be null");
-        }
-
         public Dictionary<String, String> EvaluateStepInputs(
             TemplateToken token,
-            DictionaryContextData contextData,
-            IList<IFunctionInfo> expressionFunctions)
+            DictionaryContextData contextData)
         {
             var result = default(Dictionary<String, String>);
 
             if (token != null && token.Type != TokenType.Null)
             {
-                var context = CreateContext(contextData, expressionFunctions);
+                var context = CreateContext(contextData);
                 try
                 {
                     token = TemplateEvaluator.Evaluate(context, PipelineTemplateConstants.StepWith, token, 0, null, omitHeader: true);
@@ -244,14 +179,13 @@ namespace GitHub.DistributedTask.Pipelines.ObjectTemplating
 
         public Int32 EvaluateStepTimeout(
             TemplateToken token,
-            DictionaryContextData contextData,
-            IList<IFunctionInfo> expressionFunctions)
+            DictionaryContextData contextData)
         {
             var result = default(Int32?);
 
             if (token != null && token.Type != TokenType.Null)
             {
-                var context = CreateContext(contextData, expressionFunctions);
+                var context = CreateContext(contextData);
                 try
                 {
                     token = TemplateEvaluator.Evaluate(context, PipelineTemplateConstants.NumberStepsContext, token, 0, null, omitHeader: true);
@@ -271,14 +205,13 @@ namespace GitHub.DistributedTask.Pipelines.ObjectTemplating
 
         public JobContainer EvaluateJobContainer(
             TemplateToken token,
-            DictionaryContextData contextData,
-            IList<IFunctionInfo> expressionFunctions)
+            DictionaryContextData contextData)
         {
             var result = default(JobContainer);
 
             if (token != null && token.Type != TokenType.Null)
             {
-                var context = CreateContext(contextData, expressionFunctions);
+                var context = CreateContext(contextData);
                 try
                 {
                     token = TemplateEvaluator.Evaluate(context, PipelineTemplateConstants.Container, token, 0, null, omitHeader: true);
@@ -296,90 +229,15 @@ namespace GitHub.DistributedTask.Pipelines.ObjectTemplating
             return result;
         }
 
-        public Dictionary<String, String> EvaluateJobOutput(
-            TemplateToken token,
-            DictionaryContextData contextData,
-            IList<IFunctionInfo> expressionFunctions)
-        {
-            var result = default(Dictionary<String, String>);
-
-            if (token != null && token.Type != TokenType.Null)
-            {
-                var context = CreateContext(contextData, expressionFunctions);
-                try
-                {
-                    token = TemplateEvaluator.Evaluate(context, PipelineTemplateConstants.JobOutputs, token, 0, null, omitHeader: true);
-                    context.Errors.Check();
-                    result = new Dictionary<String, String>(StringComparer.OrdinalIgnoreCase);
-                    var mapping = token.AssertMapping("outputs");
-                    foreach (var pair in mapping)
-                    {
-                        // Literal key
-                        var key = pair.Key.AssertString("output key");
-
-                        // Literal value
-                        var value = pair.Value.AssertString("output value");
-                        result[key.Value] = value.Value;
-                    }
-                }
-                catch (Exception ex) when (!(ex is TemplateValidationException))
-                {
-                    context.Errors.Add(ex);
-                }
-
-                context.Errors.Check();
-            }
-
-            return result;
-        }
-
-        public Dictionary<String, String> EvaluateJobDefaultsRun(
-            TemplateToken token,
-            DictionaryContextData contextData,
-            IList<IFunctionInfo> expressionFunctions)
-        {
-            var result = default(Dictionary<String, String>);
-
-            if (token != null && token.Type != TokenType.Null)
-            {
-                var context = CreateContext(contextData, expressionFunctions);
-                try
-                {
-                    token = TemplateEvaluator.Evaluate(context, PipelineTemplateConstants.JobDefaultsRun, token, 0, null, omitHeader: true);
-                    context.Errors.Check();
-                    result = new Dictionary<String, String>(StringComparer.OrdinalIgnoreCase);
-                    var mapping = token.AssertMapping("defaults run");
-                    foreach (var pair in mapping)
-                    {
-                        // Literal key
-                        var key = pair.Key.AssertString("defaults run key");
-
-                        // Literal value
-                        var value = pair.Value.AssertString("defaults run value");
-                        result[key.Value] = value.Value;
-                    }
-                }
-                catch (Exception ex) when (!(ex is TemplateValidationException))
-                {
-                    context.Errors.Add(ex);
-                }
-
-                context.Errors.Check();
-            }
-
-            return result;
-        }
-
         public IList<KeyValuePair<String, JobContainer>> EvaluateJobServiceContainers(
             TemplateToken token,
-            DictionaryContextData contextData,
-            IList<IFunctionInfo> expressionFunctions)
+            DictionaryContextData contextData)
         {
             var result = default(List<KeyValuePair<String, JobContainer>>);
 
             if (token != null && token.Type != TokenType.Null)
             {
-                var context = CreateContext(contextData, expressionFunctions);
+                var context = CreateContext(contextData);
                 try
                 {
                     token = TemplateEvaluator.Evaluate(context, PipelineTemplateConstants.Services, token, 0, null, omitHeader: true);
@@ -397,10 +255,62 @@ namespace GitHub.DistributedTask.Pipelines.ObjectTemplating
             return result;
         }
 
-        private TemplateContext CreateContext(
+        public Boolean TryEvaluateStepDisplayName(
+            TemplateToken token,
             DictionaryContextData contextData,
-            IList<IFunctionInfo> expressionFunctions,
-            IEnumerable<KeyValuePair<String, Object>> expressionState = null)
+            out String stepName)
+        {
+            stepName = default(String);
+            var context = CreateContext(contextData);
+
+            if (token != null && token.Type != TokenType.Null)
+            {
+                // We should only evaluate basic expressions if we are sure we have context on all the Named Values and functions
+                // Otherwise return and use a default name
+                if (token is BasicExpressionToken expressionToken)
+                {
+                    ExpressionNode root = null;
+                    try
+                    {
+                        root = new ExpressionParser().ValidateSyntax(expressionToken.Expression, null) as ExpressionNode;
+                    }
+                    catch (Exception exception)
+                    {
+                        context.Errors.Add(exception);
+                        context.Errors.Check();
+                    }
+                    foreach (var node in root.Traverse())
+                    {
+                        if (node is NamedValue namedValue && !contextData.ContainsKey(namedValue.Name))
+                        {
+                            return false;
+                        }
+                        else if (node is Function function &&
+                            !context.ExpressionFunctions.Any(item => String.Equals(item.Name, function.Name)) &&
+                            !ExpressionConstants.WellKnownFunctions.ContainsKey(function.Name))
+                        {
+                            return false;
+                        }
+                    }
+                }
+
+                try
+                {
+                    token = TemplateEvaluator.Evaluate(context, PipelineTemplateConstants.StringStepsContext, token, 0, null, omitHeader: true);
+                    context.Errors.Check();
+                    stepName = PipelineTemplateConverter.ConvertToStepDisplayName(context, token);
+                }
+                catch (Exception ex) when (!(ex is TemplateValidationException))
+                {
+                    context.Errors.Add(ex);
+                }
+
+                context.Errors.Check();
+            }
+            return true;
+        }
+
+        private TemplateContext CreateContext(DictionaryContextData contextData)
         {
             var result = new TemplateContext
             {
@@ -414,16 +324,6 @@ namespace GitHub.DistributedTask.Pipelines.ObjectTemplating
                 TraceWriter = m_trace,
             };
 
-            // Add the file table
-            if (m_fileTable?.Count > 0)
-            {
-                foreach (var file in m_fileTable)
-                {
-                    result.GetFileId(file);
-                }
-            }
-
-            // Add named values
             if (contextData != null)
             {
                 foreach (var pair in contextData)
@@ -432,60 +332,25 @@ namespace GitHub.DistributedTask.Pipelines.ObjectTemplating
                 }
             }
 
-            // Add functions
-            var functionNames = new HashSet<String>(StringComparer.OrdinalIgnoreCase);
-            if (expressionFunctions?.Count > 0)
-            {
-                foreach (var function in expressionFunctions)
-                {
-                    result.ExpressionFunctions.Add(function);
-                    functionNames.Add(function.Name);
-                }
-            }
-
-            // Add missing expression values and expression functions.
-            // This solves the following problems:
-            //   - Compat for new agent against old server (new contexts not sent down in job message)
-            //   - Evaluating early when all referenced contexts are available, even though all allowed
-            //     contexts may not yet be available. For example, evaluating step display name can often
-            //     be performed early.
-            foreach (var name in s_expressionValueNames)
+            // Compat for new agent against old server
+            foreach (var name in s_contextNames)
             {
                 if (!result.ExpressionValues.ContainsKey(name))
                 {
                     result.ExpressionValues[name] = null;
                 }
             }
-            foreach (var name in s_expressionFunctionNames)
-            {
-                if (!functionNames.Contains(name))
-                {
-                    result.ExpressionFunctions.Add(new FunctionInfo<NoOperation>(name, 0, Int32.MaxValue));
-                }
-            }
-
-            // Add state
-            if (expressionState != null)
-            {
-                foreach (var pair in expressionState)
-                {
-                    result.State[pair.Key] = pair.Value;
-                }
-            }
 
             return result;
         }
 
         private readonly ITraceWriter m_trace;
         private readonly TemplateSchema m_schema;
-        private readonly IList<String> m_fileTable;
-        private readonly String[] s_expressionValueNames = new[]
+        private readonly String[] s_contextNames = new[]
         {
             PipelineTemplateConstants.GitHub,
-            PipelineTemplateConstants.Needs,
             PipelineTemplateConstants.Strategy,
             PipelineTemplateConstants.Matrix,
-            PipelineTemplateConstants.Needs,
             PipelineTemplateConstants.Secrets,
             PipelineTemplateConstants.Steps,
             PipelineTemplateConstants.Inputs,
@@ -493,13 +358,5 @@ namespace GitHub.DistributedTask.Pipelines.ObjectTemplating
             PipelineTemplateConstants.Runner,
             PipelineTemplateConstants.Env,
         };
-        private readonly String[] s_expressionFunctionNames = new[]
-        {
-            PipelineTemplateConstants.Always,
-            PipelineTemplateConstants.Cancelled,
-            PipelineTemplateConstants.Failure,
-            PipelineTemplateConstants.HashFiles,
-            PipelineTemplateConstants.Success,
-        };
     }
 }

src/Sdk/DTPipelines/Pipelines/ObjectTemplating/PipelineTemplateSchemaFactory.cs

@@ -2,35 +2,25 @@
 using System.ComponentModel;
 using System.IO;
 using System.Reflection;
-using System.Threading;
-using System.Threading.Tasks;
 using GitHub.DistributedTask.ObjectTemplating.Schema;
 
 namespace GitHub.DistributedTask.Pipelines.ObjectTemplating
 {
     [EditorBrowsable(EditorBrowsableState.Never)]
-    public static class PipelineTemplateSchemaFactory
+    public sealed class PipelineTemplateSchemaFactory
     {
-        public static TemplateSchema GetSchema()
+        public TemplateSchema CreateSchema()
         {
-            if (s_schema == null)
+            var assembly = Assembly.GetExecutingAssembly();
+            var json = default(String);
+            using (var stream = assembly.GetManifestResourceStream("GitHub.DistributedTask.Pipelines.ObjectTemplating.workflow-v1.0.json"))
+            using (var streamReader = new StreamReader(stream))
             {
-                var assembly = Assembly.GetExecutingAssembly();
-                var json = default(String);
-                using (var stream = assembly.GetManifestResourceStream("GitHub.DistributedTask.Pipelines.ObjectTemplating.workflow-v1.0.json"))
-                using (var streamReader = new StreamReader(stream))
-                {
-                    json = streamReader.ReadToEnd();
-                }
-
-                var objectReader = new JsonObjectReader(null, json);
-                var schema = TemplateSchema.Load(objectReader);
-                Interlocked.CompareExchange(ref s_schema, schema, null);
+                json = streamReader.ReadToEnd();
             }
 
-            return s_schema;
+            var objectReader = new JsonObjectReader(null, json);
+            return TemplateSchema.Load(objectReader);
         }
-
-        private static TemplateSchema s_schema;
     }
 }

src/Sdk/DTPipelines/Pipelines/ObjectTemplating/YamlObjectReader.cs

@@ -1,572 +0,0 @@
-using System;
-using System.Globalization;
-using System.IO;
-using System.Linq;
-using GitHub.DistributedTask.ObjectTemplating;
-using GitHub.DistributedTask.ObjectTemplating.Tokens;
-using YamlDotNet.Core;
-using YamlDotNet.Core.Events;
-
-namespace GitHub.DistributedTask.Pipelines.ObjectTemplating
-{
-    /// <summary>
-    /// Converts a YAML file into a TemplateToken
-    /// </summary>
-    public sealed class YamlObjectReader : IObjectReader
-    {
-        internal YamlObjectReader(
-            Int32? fileId,
-            TextReader input)
-        {
-            m_fileId = fileId;
-            m_parser = new Parser(input);
-        }
-
-        public Boolean AllowLiteral(out LiteralToken value)
-        {
-            if (EvaluateCurrent() is Scalar scalar)
-            {
-                // Tag specified
-                if (!String.IsNullOrEmpty(scalar.Tag))
-                {
-                    // String tag
-                    if (String.Equals(scalar.Tag, c_stringTag, StringComparison.Ordinal))
-                    {
-                        value = new StringToken(m_fileId, scalar.Start.Line, scalar.Start.Column, scalar.Value);
-                        MoveNext();
-                        return true;
-                    }
-
-                    // Not plain style
-                    if (scalar.Style != ScalarStyle.Plain)
-                    {
-                        throw new NotSupportedException($"The scalar style '{scalar.Style}' on line {scalar.Start.Line} and column {scalar.Start.Column} is not valid with the tag '{scalar.Tag}'");
-                    }
-
-                    // Boolean, Float, Integer, or Null
-                    switch (scalar.Tag)
-                    {
-                        case c_booleanTag:
-                            value = ParseBoolean(scalar);
-                            break;
-                        case c_floatTag:
-                            value = ParseFloat(scalar);
-                            break;
-                        case c_integerTag:
-                            value = ParseInteger(scalar);
-                            break;
-                        case c_nullTag:
-                            value = ParseNull(scalar);
-                            break;
-                        default:
-                            throw new NotSupportedException($"Unexpected tag '{scalar.Tag}'");
-                    }
-
-                    MoveNext();
-                    return true;
-                }
-
-                // Plain style, determine type using YAML 1.2 "core" schema https://yaml.org/spec/1.2/spec.html#id2804923
-                if (scalar.Style == ScalarStyle.Plain)
-                {
-                    if (MatchNull(scalar, out var nullToken))
-                    {
-                        value = nullToken;
-                    }
-                    else if (MatchBoolean(scalar, out var booleanToken))
-                    {
-                        value = booleanToken;
-                    }
-                    else if (MatchInteger(scalar, out var numberToken) ||
-                        MatchFloat(scalar, out numberToken))
-                    {
-                        value = numberToken;
-                    }
-                    else
-                    {
-                        value = new StringToken(m_fileId, scalar.Start.Line, scalar.Start.Column, scalar.Value);
-                    }
-
-                    MoveNext();
-                    return true;
-                }
-
-                // Otherwise assume string
-                value = new StringToken(m_fileId, scalar.Start.Line, scalar.Start.Column, scalar.Value);
-                MoveNext();
-                return true;
-            }
-
-            value = default;
-            return false;
-        }
-
-        public Boolean AllowSequenceStart(out SequenceToken value)
-        {
-            if (EvaluateCurrent() is SequenceStart sequenceStart)
-            {
-                value = new SequenceToken(m_fileId, sequenceStart.Start.Line, sequenceStart.Start.Column);
-                MoveNext();
-                return true;
-            }
-
-            value = default;
-            return false;
-        }
-
-        public Boolean AllowSequenceEnd()
-        {
-            if (EvaluateCurrent() is SequenceEnd)
-            {
-                MoveNext();
-                return true;
-            }
-
-            return false;
-        }
-
-        public Boolean AllowMappingStart(out MappingToken value)
-        {
-            if (EvaluateCurrent() is MappingStart mappingStart)
-            {
-                value = new MappingToken(m_fileId, mappingStart.Start.Line, mappingStart.Start.Column);
-                MoveNext();
-                return true;
-            }
-
-            value = default;
-            return false;
-        }
-
-        public Boolean AllowMappingEnd()
-        {
-            if (EvaluateCurrent() is MappingEnd)
-            {
-                MoveNext();
-                return true;
-            }
-
-            return false;
-        }
-
-        /// <summary>
-        /// Consumes the last parsing events, which are expected to be DocumentEnd and StreamEnd.
-        /// </summary>
-        public void ValidateEnd()
-        {
-            if (EvaluateCurrent() is DocumentEnd)
-            {
-                MoveNext();
-            }
-            else
-            {
-                throw new InvalidOperationException("Expected document end parse event");
-            }
-
-            if (EvaluateCurrent() is StreamEnd)
-            {
-                MoveNext();
-            }
-            else
-            {
-                throw new InvalidOperationException("Expected stream end parse event");
-            }
-
-            if (MoveNext())
-            {
-                throw new InvalidOperationException("Expected end of parse events");
-            }
-        }
-
-        /// <summary>
-        /// Consumes the first parsing events, which are expected to be StreamStart and DocumentStart.
-        /// </summary>
-        public void ValidateStart()
-        {
-            if (EvaluateCurrent() != null)
-            {
-                throw new InvalidOperationException("Unexpected parser state");
-            }
-
-            if (!MoveNext())
-            {
-                throw new InvalidOperationException("Expected a parse event");
-            }
-
-            if (EvaluateCurrent() is StreamStart)
-            {
-                MoveNext();
-            }
-            else
-            {
-                throw new InvalidOperationException("Expected stream start parse event");
-            }
-
-            if (EvaluateCurrent() is DocumentStart)
-            {
-                MoveNext();
-            }
-            else
-            {
-                throw new InvalidOperationException("Expected document start parse event");
-            }
-        }
-
-        private ParsingEvent EvaluateCurrent()
-        {
-            if (m_current == null)
-            {
-                m_current = m_parser.Current;
-                if (m_current != null)
-                {
-                    if (m_current is Scalar scalar)
-                    {
-                        // Verify not using achors
-                        if (scalar.Anchor != null)
-                        {
-                            throw new InvalidOperationException($"Anchors are not currently supported. Remove the anchor '{scalar.Anchor}'");
-                        }
-                    }
-                    else if (m_current is MappingStart mappingStart)
-                    {
-                        // Verify not using achors
-                        if (mappingStart.Anchor != null)
-                        {
-                            throw new InvalidOperationException($"Anchors are not currently supported. Remove the anchor '{mappingStart.Anchor}'");
-                        }
-                    }
-                    else if (m_current is SequenceStart sequenceStart)
-                    {
-                        // Verify not using achors
-                        if (sequenceStart.Anchor != null)
-                        {
-                            throw new InvalidOperationException($"Anchors are not currently supported. Remove the anchor '{sequenceStart.Anchor}'");
-                        }
-                    }
-                    else if (!(m_current is MappingEnd) &&
-                        !(m_current is SequenceEnd) &&
-                        !(m_current is DocumentStart) &&
-                        !(m_current is DocumentEnd) &&
-                        !(m_current is StreamStart) &&
-                        !(m_current is StreamEnd))
-                    {
-                        throw new InvalidOperationException($"Unexpected parsing event type: {m_current.GetType().Name}");
-                    }
-                }
-            }
-
-            return m_current;
-        }
-
-        private Boolean MoveNext()
-        {
-            m_current = null;
-            return m_parser.MoveNext();
-        }
-
-        private BooleanToken ParseBoolean(Scalar scalar)
-        {
-            if (MatchBoolean(scalar, out var token))
-            {
-                return token;
-            }
-
-            ThrowInvalidValue(scalar, c_booleanTag); // throws
-            return default;
-        }
-
-        private NumberToken ParseFloat(Scalar scalar)
-        {
-            if (MatchFloat(scalar, out var token))
-            {
-                return token;
-            }
-
-            ThrowInvalidValue(scalar, c_floatTag); // throws
-            return default;
-        }
-
-        private NumberToken ParseInteger(Scalar scalar)
-        {
-            if (MatchInteger(scalar, out var token))
-            {
-                return token;
-            }
-
-            ThrowInvalidValue(scalar, c_integerTag); // throws
-            return default;
-        }
-
-        private NullToken ParseNull(Scalar scalar)
-        {
-            if (MatchNull(scalar, out var token))
-            {
-                return token;
-            }
-
-            ThrowInvalidValue(scalar, c_nullTag); // throws
-            return default;
-        }
-
-        private Boolean MatchBoolean(
-            Scalar scalar,
-            out BooleanToken value)
-        {
-            // YAML 1.2 "core" schema https://yaml.org/spec/1.2/spec.html#id2804923
-            switch (scalar.Value ?? String.Empty)
-            {
-                case "true":
-                case "True":
-                case "TRUE":
-                    value = new BooleanToken(m_fileId, scalar.Start.Line, scalar.Start.Column, true);
-                    return true;
-                case "false":
-                case "False":
-                case "FALSE":
-                    value = new BooleanToken(m_fileId, scalar.Start.Line, scalar.Start.Column, false);
-                    return true;
-            }
-
-            value = default;
-            return false;
-        }
-
-        private Boolean MatchFloat(
-            Scalar scalar,
-            out NumberToken value)
-        {
-            // YAML 1.2 "core" schema https://yaml.org/spec/1.2/spec.html#id2804923
-            var str = scalar.Value;
-            if (!String.IsNullOrEmpty(str))
-            {
-                // Check for [-+]?(\.inf|\.Inf|\.INF)|\.nan|\.NaN|\.NAN
-                switch (str)
-                {
-                    case ".inf":
-                    case ".Inf":
-                    case ".INF":
-                    case "+.inf":
-                    case "+.Inf":
-                    case "+.INF":
-                        value = new NumberToken(m_fileId, scalar.Start.Line, scalar.Start.Column, Double.PositiveInfinity);
-                        return true;
-                    case "-.inf":
-                    case "-.Inf":
-                    case "-.INF":
-                        value = new NumberToken(m_fileId, scalar.Start.Line, scalar.Start.Column, Double.NegativeInfinity);
-                        return true;
-                    case ".nan":
-                    case ".NaN":
-                    case ".NAN":
-                        value = new NumberToken(m_fileId, scalar.Start.Line, scalar.Start.Column, Double.NaN);
-                        return true;
-                }
-
-                // Otherwise check [-+]?(\.[0-9]+|[0-9]+(\.[0-9]*)?)([eE][-+]?[0-9]+)?
-
-                // Skip leading sign
-                var index = str[0] == '-' || str[0] == '+' ? 1 : 0;
-
-                // Check for integer portion
-                var length = str.Length;
-                var hasInteger = false;
-                while (index < length && str[index] >= '0' && str[index] <= '9')
-                {
-                    hasInteger = true;
-                    index++;
-                }
-
-                // Check for decimal point
-                var hasDot = false;
-                if (index < length && str[index] == '.')
-                {
-                    hasDot = true;
-                    index++;
-                }
-
-                // Check for decimal portion
-                var hasDecimal = false;
-                while (index < length && str[index] >= '0' && str[index] <= '9')
-                {
-                    hasDecimal = true;
-                    index++;
-                }
-
-                // Check [-+]?(\.[0-9]+|[0-9]+(\.[0-9]*)?)
-                if ((hasDot && hasDecimal) || hasInteger)
-                {
-                    // Check for end
-                    if (index == length)
-                    {
-                        // Try parse
-                        if (Double.TryParse(str, NumberStyles.AllowLeadingSign | NumberStyles.AllowDecimalPoint, CultureInfo.InvariantCulture, out var doubleValue))
-                        {
-                            value = new NumberToken(m_fileId, scalar.Start.Line, scalar.Start.Column, doubleValue);
-                            return true;
-                        }
-                        // Otherwise exceeds range
-                        else
-                        {
-                            ThrowInvalidValue(scalar, c_floatTag); // throws
-                        }
-                    }
-                    // Check [eE][-+]?[0-9]
-                    else if (index < length && (str[index] == 'e' || str[index] == 'E'))
-                    {
-                        index++;
-
-                        // Skip sign
-                        if (index < length && (str[index] == '-' || str[index] == '+'))
-                        {
-                            index++;
-                        }
-
-                        // Check for exponent
-                        var hasExponent = false;
-                        while (index < length && str[index] >= '0' && str[index] <= '9')
-                        {
-                            hasExponent = true;
-                            index++;
-                        }
-
-                        // Check for end
-                        if (hasExponent && index == length)
-                        {
-                            // Try parse
-                            if (Double.TryParse(str, NumberStyles.AllowLeadingSign | NumberStyles.AllowDecimalPoint | NumberStyles.AllowExponent, CultureInfo.InvariantCulture, out var doubleValue))
-                            {
-                                value = new NumberToken(m_fileId, scalar.Start.Line, scalar.Start.Column, (Double)doubleValue);
-                                return true;
-                            }
-                            // Otherwise exceeds range
-                            else
-                            {
-                                ThrowInvalidValue(scalar, c_floatTag); // throws
-                            }
-                        }
-                    }
-                }
-            }
-
-            value = default;
-            return false;
-        }
-
-        private Boolean MatchInteger(
-            Scalar scalar,
-            out NumberToken value)
-        {
-            // YAML 1.2 "core" schema https://yaml.org/spec/1.2/spec.html#id2804923
-            var str = scalar.Value;
-            if (!String.IsNullOrEmpty(str))
-            {
-                // Check for [0-9]+
-                var firstChar = str[0];
-                if (firstChar >= '0' && firstChar <= '9' &&
-                    str.Skip(1).All(x => x >= '0' && x <= '9'))
-                {
-                    // Try parse
-                    if (Double.TryParse(str, NumberStyles.None, CultureInfo.InvariantCulture, out var doubleValue))
-                    {
-                        value = new NumberToken(m_fileId, scalar.Start.Line, scalar.Start.Column, doubleValue);
-                        return true;
-                    }
-
-                    // Otherwise exceeds range
-                    ThrowInvalidValue(scalar, c_integerTag); // throws
-                }
-                // Check for (-|+)[0-9]+
-                else if ((firstChar == '-' || firstChar == '+') &&
-                    str.Length > 1 &&
-                    str.Skip(1).All(x => x >= '0' && x <= '9'))
-                {
-                    // Try parse
-                    if (Double.TryParse(str, NumberStyles.AllowLeadingSign, CultureInfo.InvariantCulture, out var doubleValue))
-                    {
-                        value = new NumberToken(m_fileId, scalar.Start.Line, scalar.Start.Column, doubleValue);
-                        return true;
-                    }
-
-                    // Otherwise exceeds range
-                    ThrowInvalidValue(scalar, c_integerTag); // throws
-                }
-                // Check for 0x[0-9a-fA-F]+
-                else if (firstChar == '0' &&
-                    str.Length > 2 &&
-                    str[1] == 'x' &&
-                    str.Skip(2).All(x => (x >= '0' && x <= '9') || (x >= 'a' && x <= 'f') || (x >= 'A' && x <= 'F')))
-                {
-                    // Try parse
-                    if (Int32.TryParse(str.Substring(2), NumberStyles.AllowHexSpecifier, CultureInfo.InvariantCulture, out var integerValue))
-                    {
-                        value = new NumberToken(m_fileId, scalar.Start.Line, scalar.Start.Column, integerValue);
-                        return true;
-                    }
-
-                    // Otherwise exceeds range
-                    ThrowInvalidValue(scalar, c_integerTag); // throws
-                }
-                // Check for 0o[0-9]+
-                else if (firstChar == '0' &&
-                    str.Length > 2 &&
-                    str[1] == 'o' &&
-                    str.Skip(2).All(x => x >= '0' && x <= '7'))
-                {
-                    // Try parse
-                    var integerValue = default(Int32);
-                    try
-                    {
-                        integerValue = Convert.ToInt32(str.Substring(2), 8);
-                    }
-                    // Otherwise exceeds range
-                    catch (Exception)
-                    {
-                        ThrowInvalidValue(scalar, c_integerTag); // throws
-                    }
-
-                    value = new NumberToken(m_fileId, scalar.Start.Line, scalar.Start.Column, integerValue);
-                    return true;
-                }
-            }
-
-            value = default;
-            return false;
-        }
-
-        private Boolean MatchNull(
-            Scalar scalar,
-            out NullToken value)
-        {
-            // YAML 1.2 "core" schema https://yaml.org/spec/1.2/spec.html#id2804923
-            switch (scalar.Value ?? String.Empty)
-            {
-                case "":
-                case "null":
-                case "Null":
-                case "NULL":
-                case "~":
-                    value = new NullToken(m_fileId, scalar.Start.Line, scalar.Start.Column);
-                    return true;
-            }
-
-            value = default;
-            return false;
-        }
-
-        private void ThrowInvalidValue(
-            Scalar scalar,
-            String tag)
-        {
-            throw new NotSupportedException($"The value '{scalar.Value}' on line {scalar.Start.Line} and column {scalar.Start.Column} is invalid for the type '{scalar.Tag}'");
-        }
-
-        private const String c_booleanTag = "tag:yaml.org,2002:bool";
-        private const String c_floatTag = "tag:yaml.org,2002:float";
-        private const String c_integerTag = "tag:yaml.org,2002:int";
-        private const String c_nullTag = "tag:yaml.org,2002:null";
-        private const String c_stringTag = "tag:yaml.org,2002:string";
-        private readonly Int32? m_fileId;
-        private readonly Parser m_parser;
-        private ParsingEvent m_current;
-    }
-}

src/Sdk/DTPipelines/workflow-v1.0.json

@@ -9,7 +9,6 @@
         "properties": {
           "on": "any",
           "name": "string",
-          "defaults": "workflow-defaults",
           "env": "workflow-env",
           "jobs": "jobs"
         }
@@ -38,7 +37,6 @@
     "steps-scope-input-value": {
       "context": [
         "github",
-        "needs",
         "strategy",
         "matrix",
         "secrets",
@@ -66,7 +64,6 @@
     "steps-scope-output-value": {
       "context": [
         "github",
-        "needs",
         "strategy",
         "matrix",
         "secrets",
@@ -91,7 +88,6 @@
       "description": "Default input values for a steps template",
       "context": [
         "github",
-        "needs",
         "strategy",
         "matrix"
       ],
@@ -114,7 +110,6 @@
       "description": "Output values for a steps template",
       "context": [
         "github",
-        "needs",
         "strategy",
         "matrix",
         "secrets",
@@ -126,23 +121,6 @@
       "string": {}
     },
 
-    "workflow-defaults": {
-      "mapping": {
-        "properties": {
-          "run": "workflow-defaults-run"
-        }
-      }
-    },
-
-    "workflow-defaults-run": {
-      "mapping": {
-        "properties": {
-          "shell": "non-empty-string",
-          "working-directory": "non-empty-string"
-        }
-      }
-    },
-
     "workflow-env": {
       "context": [
         "github",
@@ -165,21 +143,16 @@
       "mapping": {
         "properties": {
           "needs": "needs",
-          "if": "job-if",
+          "if": "string",
           "strategy": "strategy",
           "name": "string-strategy-context",
-          "runs-on": {
-            "type": "runs-on",
-            "required": true
-          },
+          "runs-on": "runs-on",
           "timeout-minutes": "number-strategy-context",
           "cancel-timeout-minutes": "number-strategy-context",
-          "continue-on-error": "boolean-strategy-context",
+          "continue-on-error": "boolean",
           "container": "container",
           "services": "services",
           "env": "job-env",
-          "outputs": "job-outputs",
-          "defaults": "job-defaults",
           "steps": "steps"
         }
       }
@@ -192,41 +165,9 @@
       ]
     },
 
-    "job-if": {
-      "context": [
-        "github",
-        "needs",
-        "always(0,0)",
-        "failure(0,MAX)",
-        "cancelled(0,0)",
-        "success(0,MAX)"
-      ],
-      "string": {}
-    },
-
-    "job-if-result": {
-      "context": [
-        "github",
-        "needs",
-        "always(0,0)",
-        "failure(0,MAX)",
-        "cancelled(0,0)",
-        "success(0,MAX)"
-      ],
-      "one-of": [
-        "null",
-        "boolean",
-        "number",
-        "string",
-        "sequence",
-        "mapping"
-      ]
-    },
-
     "strategy": {
       "context": [
-        "github",
-        "needs"
+        "github"
       ],
       "mapping": {
         "properties": {
@@ -291,24 +232,25 @@
     "runs-on": {
       "context": [
         "github",
-        "needs",
         "strategy",
         "matrix"
       ],
       "one-of": [
-        "non-empty-string",
-        "sequence-of-non-empty-string",
+        "runs-on-string",
         "runs-on-mapping"
       ]
     },
 
+    "runs-on-string": {
+      "string": {
+        "require-non-empty": true
+      }
+    },
+
     "runs-on-mapping": {
       "mapping": {
         "properties": {
-          "pool": {
-            "type": "non-empty-string",
-            "required": true
-          }
+          "pool": "non-empty-string"
         }
       }
     },
@@ -316,10 +258,9 @@
     "job-env": {
       "context": [
         "github",
-        "needs",
+        "secrets",
         "strategy",
-        "matrix",
-        "secrets"
+        "matrix"
       ],
       "mapping": {
         "loose-key-type": "non-empty-string",
@@ -327,37 +268,6 @@
       }
     },
 
-    "job-defaults": {
-      "mapping": {
-        "properties": {
-          "run": "job-defaults-run"
-        }
-      }
-    },
-
-    "job-defaults-run": {
-      "context": [
-        "github",
-        "strategy",
-        "matrix",
-        "needs",
-        "env"
-      ],
-      "mapping": {
-        "properties": {
-          "shell": "non-empty-string",
-          "working-directory": "non-empty-string"
-        }
-      }
-    },
-
-    "job-outputs": {
-      "mapping": {
-        "loose-key-type": "non-empty-string",
-        "loose-value-type": "string-runner-context"
-      }
-    },
-
     "steps": {
       "sequence": {
         "item-type": "steps-item"
@@ -391,12 +301,9 @@
         "properties": {
           "name": "string-steps-context",
           "id": "non-empty-string",
-          "if": "step-if",
+          "if": "string",
           "timeout-minutes": "number-steps-context",
-          "run": {
-            "type": "string-steps-context",
-            "required": true
-          },
+          "run": "string-steps-context",
           "continue-on-error": "boolean-steps-context",
           "env": "step-env",
           "working-directory": "string-steps-context",
@@ -410,12 +317,9 @@
         "properties": {
           "name": "string-steps-context-in-template",
           "id": "non-empty-string",
-          "if": "step-if-in-template",
+          "if": "string",
           "timeout-minutes": "number-steps-context-in-template",
-          "run": {
-            "type": "string-steps-context-in-template",
-            "required": true
-          },
+          "run": "string-steps-context-in-template",
           "continue-on-error": "boolean-steps-context-in-template",
           "env": "step-env-in-template",
           "working-directory": "string-steps-context-in-template",
@@ -429,13 +333,10 @@
         "properties": {
           "name": "string-steps-context",
           "id": "non-empty-string",
-          "if": "step-if",
+          "if": "string",
           "continue-on-error": "boolean-steps-context",
           "timeout-minutes": "number-steps-context",
-          "uses": {
-            "type": "non-empty-string",
-            "required": true
-          },
+          "uses": "non-empty-string",
           "with": "step-with",
           "env": "step-env"
         }
@@ -447,109 +348,16 @@
         "properties": {
           "name": "string-steps-context-in-template",
           "id": "non-empty-string",
-          "if": "step-if-in-template",
+          "if": "string",
           "continue-on-error": "boolean-steps-context-in-template",
           "timeout-minutes": "number-steps-context-in-template",
-          "uses": {
-            "type": "non-empty-string",
-            "required": true
-          },
+          "uses": "non-empty-string",
           "with": "step-with-in-template",
           "env": "step-env-in-template"
         }
       }
     },
 
-    "step-if": {
-      "context": [
-        "github",
-        "needs",
-        "strategy",
-        "matrix",
-        "steps",
-        "job",
-        "runner",
-        "env",
-        "always(0,0)",
-        "failure(0,0)",
-        "cancelled(0,0)",
-        "success(0,0)",
-        "hashFiles(1,255)"
-      ],
-      "string": {}
-    },
-
-    "step-if-in-template": {
-      "context": [
-        "github",
-        "needs",
-        "strategy",
-        "matrix",
-        "steps",
-        "inputs",
-        "job",
-        "runner",
-        "env",
-        "always(0,0)",
-        "failure(0,0)",
-        "cancelled(0,0)",
-        "success(0,0)",
-        "hashFiles(1,255)"
-      ],
-      "string": {}
-    },
-
-    "step-if-result": {
-      "context": [
-        "github",
-        "strategy",
-        "matrix",
-        "steps",
-        "job",
-        "runner",
-        "env",
-        "always(0,0)",
-        "failure(0,0)",
-        "cancelled(0,0)",
-        "success(0,0)",
-        "hashFiles(1,255)"
-      ],
-      "one-of": [
-        "null",
-        "boolean",
-        "number",
-        "string",
-        "sequence",
-        "mapping"
-      ]
-    },
-
-    "step-if-result-in-template": {
-      "context": [
-        "github",
-        "strategy",
-        "matrix",
-        "steps",
-        "inputs",
-        "job",
-        "runner",
-        "env",
-        "always(0,0)",
-        "failure(0,0)",
-        "cancelled(0,0)",
-        "success(0,0)",
-        "hashFiles(1,255)"
-      ],
-      "one-of": [
-        "null",
-        "boolean",
-        "number",
-        "string",
-        "sequence",
-        "mapping"
-      ]
-    },
-
     "steps-template-reference": {
       "mapping": {
         "properties": {
@@ -573,7 +381,6 @@
     "steps-template-reference-inputs": {
       "context": [
         "github",
-        "needs",
         "strategy",
         "matrix",
         "secrets",
@@ -591,7 +398,6 @@
     "steps-template-reference-inputs-in-template": {
       "context": [
         "github",
-        "needs",
         "strategy",
         "matrix",
         "secrets",
@@ -610,15 +416,13 @@
     "step-env": {
       "context": [
         "github",
-        "needs",
         "strategy",
         "matrix",
         "secrets",
         "steps",
         "job",
         "runner",
-        "env",
-        "hashFiles(1,255)"
+        "env"
       ],
       "mapping": {
         "loose-key-type": "non-empty-string",
@@ -629,7 +433,6 @@
     "step-env-in-template": {
       "context": [
         "github",
-        "needs",
         "strategy",
         "matrix",
         "secrets",
@@ -637,8 +440,7 @@
         "inputs",
         "job",
         "runner",
-        "env",
-        "hashFiles(1,255)"
+        "env"
       ],
       "mapping": {
         "loose-key-type": "non-empty-string",
@@ -649,35 +451,13 @@
     "step-with": {
       "context": [
         "github",
-        "needs",
         "strategy",
         "matrix",
         "secrets",
         "steps",
         "job",
         "runner",
-        "env",
-        "hashFiles(1,255)"
-      ],
-      "mapping": {
-        "loose-key-type": "non-empty-string",
-        "loose-value-type": "string"
-      }
-    },
-
-    "step-with-in-template": {
-      "context": [
-        "github",
-        "needs",
-        "strategy",
-        "matrix",
-        "secrets",
-        "steps",
-        "inputs",
-        "job",
-        "runner",
-        "env",
-        "hashFiles(1,255)"
+        "env"
       ],
       "mapping": {
         "loose-key-type": "non-empty-string",
@@ -688,12 +468,11 @@
     "container": {
       "context": [
         "github",
-        "needs",
         "strategy",
         "matrix"
       ],
       "one-of": [
-        "string",
+        "non-empty-string",
         "container-mapping"
       ]
     },
@@ -713,30 +492,34 @@
     "services": {
       "context": [
         "github",
-        "needs",
         "strategy",
         "matrix"
       ],
       "mapping": {
         "loose-key-type": "non-empty-string",
-        "loose-value-type": "services-container"
+        "loose-value-type": "container"
       }
     },
 
-    "services-container": {
-      "context": [
-        "github",
-        "needs",
-        "strategy",
-        "matrix"
-      ],
-      "one-of": [
-        "non-empty-string",
-        "container-mapping"
-      ]
+    "container-env": {
+      "mapping": {
+        "loose-key-type": "non-empty-string",
+        "loose-value-type": "string"
+      }
     },
 
-    "container-env": {
+    "step-with-in-template": {
+      "context": [
+        "github",
+        "strategy",
+        "matrix",
+        "secrets",
+        "steps",
+        "inputs",
+        "job",
+        "runner",
+        "env"
+      ],
       "mapping": {
         "loose-key-type": "non-empty-string",
         "loose-value-type": "string"
@@ -755,20 +538,9 @@
       }
     },
 
-    "boolean-strategy-context": {
-      "context": [
-        "github",
-        "needs",
-        "strategy",
-        "matrix"
-      ],
-      "boolean": {}
-    },
-
     "number-strategy-context": {
       "context": [
         "github",
-        "needs",
         "strategy",
         "matrix"
       ],
@@ -778,7 +550,6 @@
     "string-strategy-context": {
       "context": [
         "github",
-        "needs",
         "strategy",
         "matrix"
       ],
@@ -788,15 +559,13 @@
     "boolean-steps-context": {
       "context": [
         "github",
-        "needs",
         "strategy",
         "matrix",
         "secrets",
         "steps",
         "job",
         "runner",
-        "env",
-        "hashFiles(1,255)"
+        "env"
       ],
       "boolean": {}
     },
@@ -804,7 +573,6 @@
     "boolean-steps-context-in-template": {
       "context": [
         "github",
-        "needs",
         "strategy",
         "matrix",
         "secrets",
@@ -812,8 +580,7 @@
         "inputs",
         "job",
         "runner",
-        "env",
-        "hashFiles(1,255)"
+        "env"
       ],
       "boolean": {}
     },
@@ -821,15 +588,13 @@
     "number-steps-context": {
       "context": [
         "github",
-        "needs",
         "strategy",
         "matrix",
         "secrets",
         "steps",
         "job",
         "runner",
-        "env",
-        "hashFiles(1,255)"
+        "env"
       ],
       "number": {}
     },
@@ -837,7 +602,6 @@
     "number-steps-context-in-template": {
       "context": [
         "github",
-        "needs",
         "strategy",
         "matrix",
         "secrets",
@@ -845,16 +609,14 @@
         "inputs",
         "job",
         "runner",
-        "env",
-        "hashFiles(1,255)"
+        "env"
       ],
       "number": {}
     },
 
-    "string-runner-context": {
+    "string-steps-context": {
       "context": [
         "github",
-        "needs",
         "strategy",
         "matrix",
         "secrets",
@@ -866,26 +628,9 @@
       "string": {}
     },
 
-    "string-steps-context": {
-      "context": [
-        "github",
-        "needs",
-        "strategy",
-        "matrix",
-        "secrets",
-        "steps",
-        "job",
-        "runner",
-        "env",
-        "hashFiles(1,255)"
-      ],
-      "string": {}
-    },
-
     "string-steps-context-in-template": {
       "context": [
         "github",
-        "needs",
         "strategy",
         "matrix",
         "secrets",
@@ -893,8 +638,7 @@
         "inputs",
         "job",
         "runner",
-        "env",
-        "hashFiles(1,255)"
+        "env"
       ],
       "string": {}
     }

src/Sdk/DTWebApi/WebApi/JobEvent.cs

@@ -31,7 +31,7 @@ namespace GitHub.DistributedTask.WebApi
         }
 
         protected JobEvent(
-            String name,
+            String name, 
             Guid jobId)
         {
             this.Name = name;
@@ -123,12 +123,11 @@ namespace GitHub.DistributedTask.WebApi
             Int64 requestId,
             Guid jobId,
             TaskResult result,
-            Dictionary<String, VariableValue> outputs)
+            IDictionary<String, VariableValue> outputVariables)
             : base(JobEventTypes.JobCompleted, jobId)
         {
             this.RequestId = requestId;
             this.Result = result;
-            this.Outputs = outputs;
         }
 
         [DataMember(EmitDefaultValue = false)]
@@ -144,13 +143,6 @@ namespace GitHub.DistributedTask.WebApi
             get;
             set;
         }
-
-        [DataMember(EmitDefaultValue = false)]
-        public IDictionary<String, VariableValue> Outputs
-        {
-            get;
-            set;
-        }
     }
 
     [DataContract]
@@ -161,9 +153,9 @@ namespace GitHub.DistributedTask.WebApi
         }
 
         protected TaskEvent(
-            string name,
-            Guid jobId,
-            Guid taskId)
+            string name, 
+            Guid jobId, 
+            Guid taskId) 
             : base(name, jobId)
         {
             TaskId = taskId;
@@ -193,9 +185,9 @@ namespace GitHub.DistributedTask.WebApi
         }
 
         public override Object ReadJson(
-            JsonReader reader,
-            Type objectType,
-            Object existingValue,
+            JsonReader reader, 
+            Type objectType, 
+            Object existingValue, 
             JsonSerializer serializer)
         {
             var eventObject = JObject.Load(reader);

src/Sdk/DTWebApi/WebApi/TaskResourceIds.cs

@@ -260,8 +260,5 @@ namespace GitHub.DistributedTask.WebApi
         public static readonly Guid CheckpointResourcesLocationId = new Guid(CheckpointResourcesLocationIdString);
         public const String CheckpointResourcesResource = "references";
 
-        public static readonly Guid RunnerAuthUrl = new Guid("{A82A119C-1E46-44B6-8D75-C82A79CF975B}");
-        public const string RunnerAuthUrlResource = "authurl";
-
     }
 }

src/Test/L0/Listener/CommandSettingsL0.cs

@@ -498,7 +498,7 @@ namespace GitHub.Runner.Common.Tests
                 _promptManager
                     .Setup(x => x.ReadValue(
                         Constants.Runner.CommandLine.Args.Token, // argName
-                        "Enter runner remove token:", // description
+                        "Enter runner deletion token:", // description
                         true, // secret
                         string.Empty, // defaultValue
                         Validators.NonEmptyValidator, // validator

src/Test/L0/Listener/Configuration/ConfigurationManagerL0.cs

@@ -175,8 +175,8 @@ namespace GitHub.Runner.Common.Tests.Listener.Configuration
                 Assert.True(s.PoolId.Equals(_expectedPoolId));
                 Assert.True(s.WorkFolder.Equals(_expectedWorkFolder));
 
-                // validate GetAgentPoolsAsync gets called twice with automation pool type
-                _runnerServer.Verify(x => x.GetAgentPoolsAsync(It.IsAny<string>(), It.Is<TaskAgentPoolType>(p => p == TaskAgentPoolType.Automation)), Times.Exactly(2));
+                // validate GetAgentPoolsAsync gets called once with automation pool type
+                _runnerServer.Verify(x => x.GetAgentPoolsAsync(It.IsAny<string>(), It.Is<TaskAgentPoolType>(p => p == TaskAgentPoolType.Automation)), Times.Once);
 
                 _runnerServer.Verify(x => x.AddAgentAsync(It.IsAny<int>(), It.Is<TaskAgent>(a => a.Labels.Contains("self-hosted") && a.Labels.Contains(VarUtil.OS) && a.Labels.Contains(VarUtil.OSArchitecture))), Times.Once);
             }