Cleanup

docs/adrs/2494-runner-image-tags.md

@@ -1,65 +0,0 @@
-# ADR 2494: Runner Image Tags
-
-**Date**: 2023-03-17
-
-**Status**: Accepted<!-- |Accepted|Rejected|Superceded|Deprecated -->
-
-## Context
-
-Following the [adoption of actions-runner-controller by GitHub](https://github.com/actions/actions-runner-controller/discussions/2072) and the introduction of the new runner scale set autoscaling mode, we needed to provide a basic runner image that could be used off the shelf without much friction.
-
-The [current runner image](https://github.com/actions/runner/pkgs/container/actions-runner) is published to GHCR. Each release of this image is tagged with the runner version and the most recent release is also tagged with `latest`.
-
-While the use of `latest` is common practice, we recommend that users pin a specific version of the runner image for a predictable runtime and improved security posture. However, we still notice that a large number of end users are relying on the `latest` tag & raising issues when they encounter problems.
-
-Add to that, the community actions-runner-controller maintainers have issued a [deprecation notice](https://github.com/actions/actions-runner-controller/issues/2056) of the `latest` tag for the existing runner images (https://github.com/orgs/actions-runner-controller/packages).
-
-## Decision
-
-Proceed with Option 2, keeping the `latest` tag and adding the `NOTES.txt` file to our helm charts with the notice.
-
-### Option 1: Remove the `latest` tag
-
-By removing the `latest` tag, we have to proceed with either of these options:
-
-1. Remove the runner image reference in the `values.yaml` provided with the `gha-runner-scale-set` helm chart and mark these fields as required so that users have to explicitly specify a runner image and a specific tag. This will obviously introduce more  friction for users who want to start using actions-runner-controller for the first time.
-
-```yaml
-  spec:
-    containers:
-    - name: runner
-      image: ""
-      tag: ""
-      command: ["/home/runner/run.sh"]
-```
-
-1. Pin a specific runner image tag in the `values.yaml` provided with the `gha-runner-scale-set` helm chart. This will reduce friction for users who want to start using actions-runner-controller for the first time but will require us to update the `values.yaml` with every new runner release.
-
-```yaml
-  spec:
-    containers:
-    - name: runner
-      image: "ghcr.io/actions/actions-runner"
-      tag: "v2.300.0"
-      command: ["/home/runner/run.sh"]
-```
-
-### Option 2: Keep the `latest` tag
-
-Keeping the `latest` tag is also a reasonable option especially if we don't expect to make any breaking changes to the runner image. We could enhance this by adding a [NOTES.txt](https://helm.sh/docs/chart_template_guide/notes_files/) to the helm chart which will be displayed to the user after a successful helm install/upgrade. This will help users understand the implications of using the `latest` tag and how to pin a specific version of the runner image.
-
-The runner image release workflow will need to be updated so that the image is pushed to GHCR and tagged only when the runner rollout has reached all scale units.
-
-## Consequences
-
-Proceeding with **option 1** means:
-
-1. We will enhance the runtime predictability and security posture of our end users
-1. We will have to update the `values.yaml` with every new runner release (that can be automated)
-1. We will introduce friction for users who want to start using actions-runner-controller for the first time
-
-Proceeding with **option 2** means:
-
-1. We will have to continue to maintain the `latest` tag
-1. We will assume that end users will be able to handle the implications of using the `latest` tag
-1. Runner image release workflow needs to be updated 

src/Runner.Common/HostContext.cs

@@ -1,4 +1,4 @@
-using System;
+using System;
 using System.Collections.Concurrent;
 using System.Collections.Generic;
 using System.Diagnostics;
@@ -53,7 +53,7 @@ namespace GitHub.Runner.Common
         private static int[] _vssHttpCredentialEventIds = new int[] { 11, 13, 14, 15, 16, 17, 18, 20, 21, 22, 27, 29 };
         private readonly ConcurrentDictionary<Type, object> _serviceInstances = new();
         private readonly ConcurrentDictionary<Type, Type> _serviceTypes = new();
-        private readonly ISecretMasker _secretMasker;
+        private readonly ISecretMasker _secretMasker = new SecretMasker();
         private readonly List<ProductInfoHeaderValue> _userAgents = new() { new ProductInfoHeaderValue($"GitHubActionsRunner-{BuildConstants.RunnerPackage.PackageName}", BuildConstants.RunnerPackage.Version) };
         private CancellationTokenSource _runnerShutdownTokenSource = new();
         private object _perfLock = new();
@@ -82,20 +82,17 @@ namespace GitHub.Runner.Common
             _loadContext = AssemblyLoadContext.GetLoadContext(typeof(HostContext).GetTypeInfo().Assembly);
             _loadContext.Unloading += LoadContext_Unloading;
 
-            var masks = new List<ValueEncoder>()
-            {
-                ValueEncoders.EnumerateBase64Variations,
-                ValueEncoders.CommandLineArgumentEscape,
-                ValueEncoders.ExpressionStringEscape,
-                ValueEncoders.JsonStringEscape,
-                ValueEncoders.UriDataEscape,
-                ValueEncoders.XmlDataEscape,
-                ValueEncoders.TrimDoubleQuotes,
-                ValueEncoders.PowerShellPreAmpersandEscape,
-                ValueEncoders.PowerShellPostAmpersandEscape
-            };
-            _secretMasker = new SecretMasker(masks);
-
+            this.SecretMasker.AddValueEncoder(ValueEncoders.Base64StringEscape);
+            this.SecretMasker.AddValueEncoder(ValueEncoders.Base64StringEscapeShift1);
+            this.SecretMasker.AddValueEncoder(ValueEncoders.Base64StringEscapeShift2);
+            this.SecretMasker.AddValueEncoder(ValueEncoders.CommandLineArgumentEscape);
+            this.SecretMasker.AddValueEncoder(ValueEncoders.ExpressionStringEscape);
+            this.SecretMasker.AddValueEncoder(ValueEncoders.JsonStringEscape);
+            this.SecretMasker.AddValueEncoder(ValueEncoders.UriDataEscape);
+            this.SecretMasker.AddValueEncoder(ValueEncoders.XmlDataEscape);
+            this.SecretMasker.AddValueEncoder(ValueEncoders.TrimDoubleQuotes);
+            this.SecretMasker.AddValueEncoder(ValueEncoders.PowerShellPreAmpersandEscape);
+            this.SecretMasker.AddValueEncoder(ValueEncoders.PowerShellPostAmpersandEscape);
 
             // Create StdoutTraceListener if ENV is set
             StdoutTraceListener stdoutTraceListener = null;
@@ -223,7 +220,7 @@ namespace GitHub.Runner.Common
             var runnerFile = GetConfigFile(WellKnownConfigFile.Runner);
             if (File.Exists(runnerFile))
             {
-                var runnerSettings = IOUtil.LoadObject<RunnerSettings>(runnerFile, true);
+                var runnerSettings = IOUtil.LoadObject<RunnerSettings>(runnerFile);
                 _userAgents.Add(new ProductInfoHeaderValue("RunnerId", runnerSettings.AgentId.ToString(CultureInfo.InvariantCulture)));
                 _userAgents.Add(new ProductInfoHeaderValue("GroupId", runnerSettings.PoolId.ToString(CultureInfo.InvariantCulture)));
             }

src/Runner.Common/JobServer.cs

@@ -24,11 +24,15 @@ namespace GitHub.Runner.Common
         Task ConnectAsync(VssConnection jobConnection);
 
         void InitializeWebsocketClient(ServiceEndpoint serviceEndpoint);
+        void InitializeResultsClient(Uri uri, string token);
 
         // logging and console
         Task<TaskLog> AppendLogContentAsync(Guid scopeIdentifier, string hubName, Guid planId, int logId, Stream uploadStream, CancellationToken cancellationToken);
         Task AppendTimelineRecordFeedAsync(Guid scopeIdentifier, string hubName, Guid planId, Guid timelineId, Guid timelineRecordId, Guid stepId, IList<string> lines, long? startLine, CancellationToken cancellationToken);
         Task<TaskAttachment> CreateAttachmentAsync(Guid scopeIdentifier, string hubName, Guid planId, Guid timelineId, Guid timelineRecordId, String type, String name, Stream uploadStream, CancellationToken cancellationToken);
+        Task CreateStepSummaryAsync(string planId, string jobId, Guid stepId, string file, CancellationToken cancellationToken);
+        Task CreateResultsStepLogAsync(string planId, string jobId, Guid stepId, string file, bool finalize, bool firstBlock, long lineCount, CancellationToken cancellationToken);
+        Task CreateResultsJobLogAsync(string planId, string jobId, string file, bool finalize, bool firstBlock, long lineCount, CancellationToken cancellationToken);
         Task<TaskLog> CreateLogAsync(Guid scopeIdentifier, string hubName, Guid planId, TaskLog log, CancellationToken cancellationToken);
         Task<Timeline> CreateTimelineAsync(Guid scopeIdentifier, string hubName, Guid planId, Guid timelineId, CancellationToken cancellationToken);
         Task<List<TimelineRecord>> UpdateTimelineRecordsAsync(Guid scopeIdentifier, string hubName, Guid planId, Guid timelineId, IEnumerable<TimelineRecord> records, CancellationToken cancellationToken);
@@ -42,6 +46,7 @@ namespace GitHub.Runner.Common
         private bool _hasConnection;
         private VssConnection _connection;
         private TaskHttpClient _taskClient;
+        private ResultsHttpClient _resultsClient;
         private ClientWebSocket _websocketClient;
 
         private ServiceEndpoint _serviceEndpoint;
@@ -145,6 +150,12 @@ namespace GitHub.Runner.Common
             InitializeWebsocketClient(TimeSpan.Zero);
         }
 
+        public void InitializeResultsClient(Uri uri, string token)
+        {
+            var httpMessageHandler = HostContext.CreateHttpClientHandler();
+            this._resultsClient = new ResultsHttpClient(uri, httpMessageHandler, token, disposeHandler: true);
+        }
+
         public ValueTask DisposeAsync()
         {
             CloseWebSocket(WebSocketCloseStatus.NormalClosure, CancellationToken.None);
@@ -307,6 +318,32 @@ namespace GitHub.Runner.Common
             return _taskClient.CreateAttachmentAsync(scopeIdentifier, hubName, planId, timelineId, timelineRecordId, type, name, uploadStream, cancellationToken: cancellationToken);
         }
 
+        public Task CreateStepSummaryAsync(string planId, string jobId, Guid stepId, string file, CancellationToken cancellationToken)
+        {
+            if (_resultsClient != null)
+            {
+                return _resultsClient.UploadStepSummaryAsync(planId, jobId, stepId, file, cancellationToken: cancellationToken);
+            }
+            throw new InvalidOperationException("Results client is not initialized.");
+        }
+
+        public Task CreateResultsStepLogAsync(string planId, string jobId, Guid stepId, string file, bool finalize, bool firstBlock, long lineCount, CancellationToken cancellationToken)
+        {
+            if (_resultsClient != null)
+            {
+                return _resultsClient.UploadResultsStepLogAsync(planId, jobId, stepId, file, finalize, firstBlock, lineCount, cancellationToken: cancellationToken);
+            }
+            throw new InvalidOperationException("Results client is not initialized.");
+        }
+
+        public Task CreateResultsJobLogAsync(string planId, string jobId, string file, bool finalize, bool firstBlock, long lineCount, CancellationToken cancellationToken)
+        {
+            if (_resultsClient != null)
+            {
+                return _resultsClient.UploadResultsJobLogAsync(planId, jobId, file, finalize, firstBlock, lineCount, cancellationToken: cancellationToken);
+            }
+            throw new InvalidOperationException("Results client is not initialized.");
+        }
 
         public Task<TaskLog> CreateLogAsync(Guid scopeIdentifier, string hubName, Guid planId, TaskLog log, CancellationToken cancellationToken)
         {

src/Runner.Common/JobServerQueue.cs

@@ -65,7 +65,6 @@ namespace GitHub.Runner.Common
 
         // common
         private IJobServer _jobServer;
-        private IResultsServer _resultsServer;
         private Task[] _allDequeueTasks;
         private readonly TaskCompletionSource<int> _jobCompletionSource = new();
         private readonly TaskCompletionSource<int> _jobRecordUpdated = new();
@@ -92,7 +91,6 @@ namespace GitHub.Runner.Common
         {
             base.Initialize(hostContext);
             _jobServer = hostContext.GetService<IJobServer>();
-            _resultsServer = hostContext.GetService<IResultsServer>();
         }
 
         public void Start(Pipelines.AgentJobRequestMessage jobRequest)
@@ -113,7 +111,7 @@ namespace GitHub.Runner.Common
                 !string.IsNullOrEmpty(resultsReceiverEndpoint))
             {
                 Trace.Info("Initializing results client");
-                _resultsServer.InitializeResultsClient(new Uri(resultsReceiverEndpoint), accessToken);
+                _jobServer.InitializeResultsClient(new Uri(resultsReceiverEndpoint), accessToken);
                 _resultsClientInitiated = true;
             }
 
@@ -514,14 +512,19 @@ namespace GitHub.Runner.Common
                         }
                         catch (Exception ex)
                         {
+                            var issue = new Issue() { Type = IssueType.Warning, Message = $"Caught exception during file upload to results. {ex.Message}" };
+                            issue.Data[Constants.Runner.InternalTelemetryIssueDataKey] = Constants.Runner.ResultsUploadFailure;
+
+                            var telemetryRecord = new TimelineRecord()
+                            {
+                                Id = Constants.Runner.TelemetryRecordId,
+                            };
+                            telemetryRecord.Issues.Add(issue);
+                            QueueTimelineRecordUpdate(_jobTimelineId, telemetryRecord);
+
                             Trace.Info("Catch exception during file upload to results, keep going since the process is best effort.");
                             Trace.Error(ex);
                             errorCount++;
-
-                            // If we hit any exceptions uploading to Results, let's skip any additional uploads to Results
-                            _resultsClientInitiated = false;
-
-                            SendResultsTelemetry(ex);
                         }
                     }
 
@@ -539,19 +542,6 @@ namespace GitHub.Runner.Common
             }
         }
 
-        private void SendResultsTelemetry(Exception ex)
-        {
-            var issue = new Issue() { Type = IssueType.Warning, Message = $"Caught exception with results. {ex.Message}" };
-            issue.Data[Constants.Runner.InternalTelemetryIssueDataKey] = Constants.Runner.ResultsUploadFailure;
-
-            var telemetryRecord = new TimelineRecord()
-            {
-                Id = Constants.Runner.TelemetryRecordId,
-            };
-            telemetryRecord.Issues.Add(issue);
-            QueueTimelineRecordUpdate(_jobTimelineId, telemetryRecord);
-        }
-
         private async Task ProcessTimelinesUpdateQueueAsync(bool runOnce = false)
         {
             while (!_jobCompletionSource.Task.IsCompleted || runOnce)
@@ -622,22 +612,6 @@ namespace GitHub.Runner.Common
                         try
                         {
                             await _jobServer.UpdateTimelineRecordsAsync(_scopeIdentifier, _hubName, _planId, update.TimelineId, update.PendingRecords, default(CancellationToken));
-                            try
-                            {
-                                if (_resultsClientInitiated)
-                                {
-                                    await _resultsServer.UpdateResultsWorkflowStepsAsync(_scopeIdentifier, _hubName, _planId, update.TimelineId, update.PendingRecords, default(CancellationToken));
-                                }
-                            }
-                            catch (Exception e)
-                            {
-                                Trace.Info("Catch exception during update steps, skip update Results.");
-                                Trace.Error(e);
-                                _resultsClientInitiated = false;
-
-                                SendResultsTelemetry(e);
-                            }
-
                             if (_bufferedRetryRecords.Remove(update.TimelineId))
                             {
                                 Trace.Verbose("Cleanup buffered timeline record for timeline: {0}.", update.TimelineId);
@@ -845,7 +819,7 @@ namespace GitHub.Runner.Common
             Trace.Info($"Starting to upload summary file to results service {file.Name}, {file.Path}");
             ResultsFileUploadHandler summaryHandler = async (file) =>
             {
-                await _resultsServer.CreateResultsStepSummaryAsync(file.PlanId, file.JobId, file.RecordId, file.Path, CancellationToken.None);
+                await _jobServer.CreateStepSummaryAsync(file.PlanId, file.JobId, file.RecordId, file.Path, CancellationToken.None);
             };
 
             await UploadResultsFile(file, summaryHandler);
@@ -856,7 +830,7 @@ namespace GitHub.Runner.Common
             Trace.Info($"Starting upload of step log file to results service {file.Name}, {file.Path}");
             ResultsFileUploadHandler stepLogHandler = async (file) =>
             {
-                await _resultsServer.CreateResultsStepLogAsync(file.PlanId, file.JobId, file.RecordId, file.Path, file.Finalize, file.FirstBlock, file.TotalLines, CancellationToken.None);
+                await _jobServer.CreateResultsStepLogAsync(file.PlanId, file.JobId, file.RecordId, file.Path, file.Finalize, file.FirstBlock, file.TotalLines, CancellationToken.None);
             };
 
             await UploadResultsFile(file, stepLogHandler);
@@ -867,7 +841,7 @@ namespace GitHub.Runner.Common
             Trace.Info($"Starting upload of job log file to results service {file.Name}, {file.Path}");
             ResultsFileUploadHandler jobLogHandler = async (file) =>
             {
-                await _resultsServer.CreateResultsJobLogAsync(file.PlanId, file.JobId, file.Path, file.Finalize, file.FirstBlock, file.TotalLines, CancellationToken.None);
+                await _jobServer.CreateResultsJobLogAsync(file.PlanId, file.JobId, file.Path, file.Finalize, file.FirstBlock, file.TotalLines, CancellationToken.None);
             };
 
             await UploadResultsFile(file, jobLogHandler);
@@ -875,11 +849,6 @@ namespace GitHub.Runner.Common
 
         private async Task UploadResultsFile(ResultsUploadFileInfo file, ResultsFileUploadHandler uploadHandler)
         {
-            if (!_resultsClientInitiated)
-            {
-                return;
-            }
-
             bool uploadSucceed = false;
             try
             {
@@ -934,6 +903,8 @@ namespace GitHub.Runner.Common
         public long TotalLines { get; set; }
     }
 
+
+
     internal class ConsoleLineInfo
     {
         public ConsoleLineInfo(Guid recordId, string line, long? lineNumber)

src/Runner.Common/ResultsServer.cs

@@ -1,98 +0,0 @@
-using System;
-using System.Collections.Generic;
-using System.Linq;
-using System.Threading;
-using System.Threading.Tasks;
-using GitHub.DistributedTask.WebApi;
-using GitHub.Services.Results.Client;
-
-namespace GitHub.Runner.Common
-{
-    [ServiceLocator(Default = typeof(ResultServer))]
-    public interface IResultsServer : IRunnerService
-    {
-        void InitializeResultsClient(Uri uri, string token);
-
-        // logging and console
-        Task CreateResultsStepSummaryAsync(string planId, string jobId, Guid stepId, string file,
-            CancellationToken cancellationToken);
-
-        Task CreateResultsStepLogAsync(string planId, string jobId, Guid stepId, string file, bool finalize,
-            bool firstBlock, long lineCount, CancellationToken cancellationToken);
-
-        Task CreateResultsJobLogAsync(string planId, string jobId, string file, bool finalize, bool firstBlock,
-            long lineCount, CancellationToken cancellationToken);
-
-        Task UpdateResultsWorkflowStepsAsync(Guid scopeIdentifier, string hubName, Guid planId, Guid timelineId,
-            IEnumerable<TimelineRecord> records, CancellationToken cancellationToken);
-    }
-
-    public sealed class ResultServer : RunnerService, IResultsServer
-    {
-        private ResultsHttpClient _resultsClient;
-
-        public void InitializeResultsClient(Uri uri, string token)
-        {
-            var httpMessageHandler = HostContext.CreateHttpClientHandler();
-            this._resultsClient = new ResultsHttpClient(uri, httpMessageHandler, token, disposeHandler: true);
-        }
-
-        public Task CreateResultsStepSummaryAsync(string planId, string jobId, Guid stepId, string file,
-            CancellationToken cancellationToken)
-        {
-            if (_resultsClient != null)
-            {
-                return _resultsClient.UploadStepSummaryAsync(planId, jobId, stepId, file,
-                    cancellationToken: cancellationToken);
-            }
-
-            throw new InvalidOperationException("Results client is not initialized.");
-        }
-
-        public Task CreateResultsStepLogAsync(string planId, string jobId, Guid stepId, string file, bool finalize,
-            bool firstBlock, long lineCount, CancellationToken cancellationToken)
-        {
-            if (_resultsClient != null)
-            {
-                return _resultsClient.UploadResultsStepLogAsync(planId, jobId, stepId, file, finalize, firstBlock,
-                    lineCount, cancellationToken: cancellationToken);
-            }
-
-            throw new InvalidOperationException("Results client is not initialized.");
-        }
-
-        public Task CreateResultsJobLogAsync(string planId, string jobId, string file, bool finalize, bool firstBlock,
-            long lineCount, CancellationToken cancellationToken)
-        {
-            if (_resultsClient != null)
-            {
-                return _resultsClient.UploadResultsJobLogAsync(planId, jobId, file, finalize, firstBlock, lineCount,
-                    cancellationToken: cancellationToken);
-            }
-
-            throw new InvalidOperationException("Results client is not initialized.");
-        }
-
-        public Task UpdateResultsWorkflowStepsAsync(Guid scopeIdentifier, string hubName, Guid planId, Guid timelineId,
-            IEnumerable<TimelineRecord> records, CancellationToken cancellationToken)
-        {
-            if (_resultsClient != null)
-            {
-                try
-                {
-                    var timelineRecords = records.ToList();
-                    return _resultsClient.UpdateWorkflowStepsAsync(planId, new List<TimelineRecord>(timelineRecords),
-                        cancellationToken: cancellationToken);
-                }
-                catch (Exception ex)
-                {
-                    // Log error, but continue as this call is best-effort
-                    Trace.Info($"Failed to update steps status due to {ex.GetType().Name}");
-                    Trace.Error(ex);
-                }
-            }
-
-            throw new InvalidOperationException("Results client is not initialized.");
-        }
-    }
-}

src/Runner.Listener/Configuration/ConfigurationManager.cs

@@ -14,6 +14,7 @@ using GitHub.Runner.Sdk;
 using GitHub.Services.Common;
 using GitHub.Services.Common.Internal;
 using GitHub.Services.OAuth;
+using GitHub.Services.WebApi.Jwt;
 
 namespace GitHub.Runner.Listener.Configuration
 {
@@ -300,14 +301,8 @@ namespace GitHub.Runner.Listener.Configuration
                         if (runnerSettings.UseV2Flow)
                         {
                             var runner = await _dotcomServer.AddRunnerAsync(runnerSettings.PoolId, agent, runnerSettings.GitHubUrl, registerToken, publicKeyXML);
+                            runner.ApplyToTaskAgent(agent);
                             runnerSettings.ServerUrlV2 = runner.RunnerAuthorization.ServerUrl;
-
-                            agent.Id = runner.Id;
-                            agent.Authorization = new TaskAgentAuthorization()
-                            {
-                                AuthorizationUrl = runner.RunnerAuthorization.AuthorizationUrl,
-                                ClientId = new Guid(runner.RunnerAuthorization.ClientId)
-                            };
                         }
                         else
                         {

src/Runner.Listener/Runner.cs

@@ -343,7 +343,6 @@ namespace GitHub.Runner.Listener
         {
             if (settings.UseV2Flow)
             {
-                Trace.Info($"Using BrokerMessageListener");
                 var brokerListener = new BrokerMessageListener();
                 brokerListener.Initialize(HostContext);
                 return brokerListener;

src/Runner.Sdk/Util/IOUtil.cs

@@ -40,19 +40,10 @@ namespace GitHub.Runner.Sdk
             File.WriteAllText(path, StringUtil.ConvertToJson(obj), Encoding.UTF8);
         }
 
-        public static T LoadObject<T>(string path, bool required = false)
+        public static T LoadObject<T>(string path)
         {
             string json = File.ReadAllText(path, Encoding.UTF8);
-            if (required && string.IsNullOrEmpty(json))
-            {
-                throw new ArgumentNullException($"File {path} is empty");
-            }
-            T result = StringUtil.ConvertFromJson<T>(json);
-            if (required && result == null)
-            {
-                throw new ArgumentException("Converting json to object resulted in a null value");
-            }
-            return result;
+            return StringUtil.ConvertFromJson<T>(json);
         }
 
         public static string GetSha256Hash(string path)

src/Sdk/Common/Common/RawClientHttpRequestSettings.cs

@@ -184,33 +184,9 @@ namespace GitHub.Services.Common
             return settings;
         }
 
-        /// <summary>
-        /// Gets or sets the maximum size allowed for response content buffering.
-        /// </summary>
-        [DefaultValue(c_defaultContentBufferSize)]
-        public Int32 MaxContentBufferSize
-        {
-            get
-            {
-                return m_maxContentBufferSize;
-            }
-            set
-            {
-                ArgumentUtility.CheckForOutOfRange(value, nameof(value), 0, c_maxAllowedContentBufferSize);
-                m_maxContentBufferSize = value;
-            }
-        }
-
         private static Lazy<RawClientHttpRequestSettings> s_defaultSettings
             = new Lazy<RawClientHttpRequestSettings>(ConstructDefaultSettings);
 
-        private Int32 m_maxContentBufferSize;
-        // We will buffer a maximum of 1024MB in the message handler
-        private const Int32 c_maxAllowedContentBufferSize = 1024 * 1024 * 1024;
-
-        // We will buffer, by default, up to 512MB in the message handler
-        private const Int32 c_defaultContentBufferSize = 1024 * 1024 * 512;
-
         private const Int32 c_defaultMaxRetry = 3;
         private static readonly TimeSpan s_defaultTimeout = TimeSpan.FromSeconds(100); //default WebAPI timeout
         private ICollection<CultureInfo> m_acceptLanguages = new List<CultureInfo>();

src/Sdk/Common/Common/RawHttpMessageHandler.cs

@@ -9,7 +9,7 @@ using GitHub.Services.OAuth;
 
 namespace GitHub.Services.Common
 {
-    public class RawHttpMessageHandler : HttpMessageHandler
+    public class RawHttpMessageHandler: HttpMessageHandler
     {
         public RawHttpMessageHandler(
             FederatedCredential credentials)
@@ -120,7 +120,6 @@ namespace GitHub.Services.Common
             Boolean succeeded = false;
             HttpResponseMessageWrapper responseWrapper;
 
-            Boolean lastResponseDemandedProxyAuth = false;
             Int32 retries = m_maxAuthRetries;
             try
             {
@@ -139,13 +138,7 @@ namespace GitHub.Services.Common
 
                     // Let's start with sending a token
                     IssuedToken token = await m_tokenProvider.GetTokenAsync(null, tokenSource.Token).ConfigureAwait(false);
-                    ApplyToken(request, token, applyICredentialsToWebProxy: lastResponseDemandedProxyAuth);
-
-                    // The WinHttpHandler will chunk any content that does not have a computed length which is
-                    // not what we want. By loading into a buffer up-front we bypass this behavior and there is
-                    // no difference in the normal HttpClientHandler behavior here since this is what they were
-                    // already doing.
-                    await BufferRequestContentAsync(request, tokenSource.Token).ConfigureAwait(false);
+                    ApplyToken(request, token);
 
                     // ConfigureAwait(false) enables the continuation to be run outside any captured
                     // SyncronizationContext (such as ASP.NET's) which keeps things from deadlocking...
@@ -154,8 +147,7 @@ namespace GitHub.Services.Common
                     responseWrapper = new HttpResponseMessageWrapper(response);
 
                     var isUnAuthorized = responseWrapper.StatusCode == HttpStatusCode.Unauthorized;
-                    lastResponseDemandedProxyAuth = responseWrapper.StatusCode == HttpStatusCode.ProxyAuthenticationRequired;
-                    if (!isUnAuthorized && !lastResponseDemandedProxyAuth)
+                    if (!isUnAuthorized)
                     {
                         // Validate the token after it has been successfully authenticated with the server.
                         m_tokenProvider?.ValidateToken(token, responseWrapper);
@@ -219,42 +211,15 @@ namespace GitHub.Services.Common
             }
         }
 
-        private static async Task BufferRequestContentAsync(
-             HttpRequestMessage request,
-             CancellationToken cancellationToken)
-        {
-            if (request.Content != null &&
-                request.Headers.TransferEncodingChunked != true)
-            {
-                Int64? contentLength = request.Content.Headers.ContentLength;
-                if (contentLength == null)
-                {
-                    await request.Content.LoadIntoBufferAsync().EnforceCancellation(cancellationToken).ConfigureAwait(false);
-                }
-
-                // Explicitly turn off chunked encoding since we have computed the request content size
-                request.Headers.TransferEncodingChunked = false;
-            }
-        }
-
         private void ApplyToken(
             HttpRequestMessage request,
-            IssuedToken token,
-            bool applyICredentialsToWebProxy = false)
+            IssuedToken token)
         {
             switch (token)
             {
                 case null:
                     return;
                 case ICredentials credentialsToken:
-                    if (applyICredentialsToWebProxy)
-                    {
-                        HttpClientHandler httpClientHandler = m_transportHandler as HttpClientHandler;
-                        if (httpClientHandler != null && httpClientHandler.Proxy != null)
-                        {
-                            httpClientHandler.Proxy.Credentials = credentialsToken;
-                        }
-                    }
                     m_credentialWrapper.InnerCredentials = credentialsToken;
                     break;
                 default:

src/Sdk/Common/Common/VssHttpMessageHandler.cs

@@ -1,4 +1,4 @@
-using System;
+using System;
 using System.Collections.Generic;
 using System.Linq;
 using System.Net;

src/Sdk/Common/Common/VssHttpRequestSettings.cs

@@ -1,4 +1,4 @@
-using System;
+using System;
 using System.Collections.Generic;
 using System.ComponentModel;
 using System.Globalization;

src/Sdk/DTExpressions2/Expressions2/Sdk/ExpressionNode.cs

@@ -1,6 +1,6 @@
-using System;
+using System;
+using System.Collections.Generic;
 using System.ComponentModel;
-using System.Linq;
 using GitHub.DistributedTask.Logging;
 
 namespace GitHub.DistributedTask.Expressions2.Sdk
@@ -55,7 +55,7 @@ namespace GitHub.DistributedTask.Expressions2.Sdk
             try
             {
                 // Evaluate
-                secretMasker = secretMasker?.Clone() ?? new SecretMasker(Enumerable.Empty<ValueEncoder>());
+                secretMasker = secretMasker?.Clone() ?? new SecretMasker();
                 trace = new EvaluationTraceWriter(trace, secretMasker);
                 var context = new EvaluationContext(trace, secretMasker, state, options, this);
                 trace.Info($"Evaluating: {ConvertToExpression()}");

src/Sdk/DTLogging/Logging/ISecretMasker.cs

@@ -1,4 +1,4 @@
-using System;
+using System;
 using System.ComponentModel;
 
 namespace GitHub.DistributedTask.Logging
@@ -8,6 +8,7 @@ namespace GitHub.DistributedTask.Logging
     {
         void AddRegex(String pattern);
         void AddValue(String value);
+        void AddValueEncoder(ValueEncoder encoder);
         ISecretMasker Clone();
         String MaskSecrets(String input);
     }

src/Sdk/DTLogging/Logging/SecretMasker.cs

@@ -1,4 +1,4 @@
-using System;
+using System;
 using System.Collections.Generic;
 using System.ComponentModel;
 using System.Linq;
@@ -10,11 +10,11 @@ namespace GitHub.DistributedTask.Logging
     [EditorBrowsable(EditorBrowsableState.Never)]
     public sealed class SecretMasker : ISecretMasker, IDisposable
     {
-        public SecretMasker(IEnumerable<ValueEncoder> encoders)
+        public SecretMasker()
         {
             m_originalValueSecrets = new HashSet<ValueSecret>();
             m_regexSecrets = new HashSet<RegexSecret>();
-            m_valueEncoders = new HashSet<ValueEncoder>(encoders ?? Enumerable.Empty<ValueEncoder>());
+            m_valueEncoders = new HashSet<ValueEncoder>();
             m_valueSecrets = new HashSet<ValueSecret>();
         }
 
@@ -104,11 +104,15 @@ namespace GitHub.DistributedTask.Logging
                 }
             }
 
-            var secretVariations = valueEncoders.SelectMany(encoder => encoder(value))
-                                                .Where(variation => !string.IsNullOrEmpty(variation))
-                                                .Distinct()
-                                                .Select(variation => new ValueSecret(variation));
-            valueSecrets.AddRange(secretVariations);
+            // Compute the encoded values.
+            foreach (ValueEncoder valueEncoder in valueEncoders)
+            {
+                String encodedValue = valueEncoder(value);
+                if (!String.IsNullOrEmpty(encodedValue))
+                {
+                    valueSecrets.Add(new ValueSecret(encodedValue));
+                }
+            }
 
             // Write section.
             try
@@ -131,6 +135,69 @@ namespace GitHub.DistributedTask.Logging
             }
         }
 
+        /// <summary>
+        /// This implementation assumes no more than one thread is adding regexes, values, or encoders at any given time.
+        /// </summary>
+        public void AddValueEncoder(ValueEncoder encoder)
+        {
+            ValueSecret[] originalSecrets;
+
+            // Read section.
+            try
+            {
+                m_lock.EnterReadLock();
+
+                // Test whether already added.
+                if (m_valueEncoders.Contains(encoder))
+                {
+                    return;
+                }
+
+                // Read the original value secrets.
+                originalSecrets = m_originalValueSecrets.ToArray();
+            }
+            finally
+            {
+                if (m_lock.IsReadLockHeld)
+                {
+                    m_lock.ExitReadLock();
+                }
+            }
+
+            // Compute the encoded values.
+            var encodedSecrets = new List<ValueSecret>();
+            foreach (ValueSecret originalSecret in originalSecrets)
+            {
+                String encodedValue = encoder(originalSecret.m_value);
+                if (!String.IsNullOrEmpty(encodedValue))
+                {
+                    encodedSecrets.Add(new ValueSecret(encodedValue));
+                }
+            }
+
+            // Write section.
+            try
+            {
+                m_lock.EnterWriteLock();
+
+                // Add the encoder.
+                m_valueEncoders.Add(encoder);
+
+                // Add the values.
+                foreach (ValueSecret encodedSecret in encodedSecrets)
+                {
+                    m_valueSecrets.Add(encodedSecret);
+                }
+            }
+            finally
+            {
+                if (m_lock.IsWriteLockHeld)
+                {
+                    m_lock.ExitWriteLock();
+                }
+            }
+        }
+
         public ISecretMasker Clone() => new SecretMasker(this);
 
         public void Dispose()

src/Sdk/DTLogging/Logging/ValueEncoders.cs

@@ -1,97 +1,73 @@
-using System;
-using System.Collections.Generic;
+using System;
 using System.ComponentModel;
-using System.Linq;
 using System.Security;
 using System.Text;
+using System.Text.RegularExpressions;
 using Newtonsoft.Json;
 
 namespace GitHub.DistributedTask.Logging
 {
     [EditorBrowsable(EditorBrowsableState.Never)]
-    public delegate IEnumerable<string> ValueEncoder(string value);
+    public delegate String ValueEncoder(String value);
 
     [EditorBrowsable(EditorBrowsableState.Never)]
     public static class ValueEncoders
     {
-
-        public static IEnumerable<string> EnumerateBase64Variations(string value)
+        public static String Base64StringEscape(String value)
         {
-            if (!string.IsNullOrEmpty(value))
-            {
-                // A byte is 8 bits.  A Base64 "digit" can hold a maximum of 6 bits (2^64 - 1, or values 0 to 63).
-                // As a result, many Unicode characters (including single-byte letters) cannot be represented using a single Base64 digit.
-                // Furthermore, on average a Base64 string will be about 33% longer than the original text.
-                // This is because it generally requires 4 Base64 digits to represent 3 Unicode bytes.  (4 / 3 ~ 1.33)
-                //
-                // Because of this 4:3 ratio (or, more precisely, 8 bits : 6 bits ratio), there's a cyclical pattern
-                // to when a byte boundary aligns with a Base64 digit boundary.
-                // The pattern repeats every 24 bits (the lowest common multiple of 8 and 6).
-                //
-                //                    |-----------24 bits-------------|-----------24 bits------------|
-                // Base64 Digits:     |digit 0|digit 1|digit 2|digit 3|digit 4|digit 5|digit 6|digit7|
-                // Allocated Bits:    aaaaaa  aaBBBB  BBBBcc  cccccc  DDDDDD  DDeeee  eeeeFF  FFFFFF
-                // Unicode chars:     |0th char |1st char |2nd char   |3rd char |4th char |5th char  |
+            return Convert.ToBase64String(Encoding.UTF8.GetBytes(value));
+        }
 
-                // Depending on alignment, the Base64-encoded secret can take any of 3 basic forms.
-                // For example, the Base64 digits representing "abc" could appear as any of the following:
-                //    "YWJj"     when aligned
-                //    ".!FiYw==" when preceded by 3x + 1 bytes
-                //    "..!hYmM=" when preceded by 3x + 2 bytes
-                // (where . represents an unrelated Base64 digit, ! represents a Base64 digit that should be masked, and x represents any non-negative integer)
+        // Base64 is 6 bits -> char
+        // A byte is 8 bits
+        // When end user doing somthing like base64(user:password)
+        // The length of the leading content will cause different base64 encoding result on the password
+        // So we add base64(value shifted 1 and two bytes) as secret as well. 
+        //    B1         B2      B3                    B4      B5     B6     B7
+        // 000000|00 0000|0000 00|000000|            000000|00 0000|0000 00|000000| 
+        //  Char1  Char2    Char3   Char4  
+        // See the above, the first byte has a character beginning at index 0, the second byte has a character beginning at index 4, the third byte has a character beginning at index 2 and then the pattern repeats
+        // We register byte offsets for all these possible values
+        public static String Base64StringEscapeShift1(String value)
+        {
+            return Base64StringEscapeShift(value, 1);
+        }
 
-                var rawBytes = Encoding.UTF8.GetBytes(value);
-
-                for (var offset = 0; offset <= 2; offset++)
-                {
-                    var prunedBytes = rawBytes.Skip(offset).ToArray();
-                    if (prunedBytes.Length > 0)
-                    {
-                        // Don't include Base64 padding characters (=) in Base64 representations of the secret.
-                        // They don't represent anything interesting, so they don't need to be masked.
-                        // (Some clients omit the padding, so we want to be sure we recognize the secret regardless of whether the padding is present or not.)
-                        var buffer = new StringBuilder(Convert.ToBase64String(prunedBytes).TrimEnd(BASE64_PADDING_SUFFIX));
-                        yield return buffer.ToString();
-
-                        // Also, yield the RFC4648-equivalent RegEx.
-                        buffer.Replace('+', '-');
-                        buffer.Replace('/', '_');
-                        yield return buffer.ToString();
-                    }
-                }
-            }
+        public static String Base64StringEscapeShift2(String value)
+        {
+            return Base64StringEscapeShift(value, 2);
         }
 
         // Used when we pass environment variables to docker to escape " with \"
-        public static IEnumerable<string> CommandLineArgumentEscape(string value)
+        public static String CommandLineArgumentEscape(String value)
         {
-            yield return value.Replace("\"", "\\\"");
+            return value.Replace("\"", "\\\"");
         }
 
-        public static IEnumerable<string> ExpressionStringEscape(string value)
+        public static String ExpressionStringEscape(String value)
         {
-            yield return Expressions2.Sdk.ExpressionUtility.StringEscape(value);
+            return Expressions2.Sdk.ExpressionUtility.StringEscape(value);
         }
 
-        public static IEnumerable<string> JsonStringEscape(string value)
+        public static String JsonStringEscape(String value)
         {
             // Convert to a JSON string and then remove the leading/trailing double-quote.
             String jsonString = JsonConvert.ToString(value);
             String jsonEscapedValue = jsonString.Substring(startIndex: 1, length: jsonString.Length - 2);
-            yield return jsonEscapedValue;
+            return jsonEscapedValue;
         }
 
-        public static IEnumerable<string> UriDataEscape(string value)
+        public static String UriDataEscape(String value)
         {
-            yield return UriDataEscape(value, 65519);
+            return UriDataEscape(value, 65519);
         }
 
-        public static IEnumerable<string> XmlDataEscape(string value)
+        public static String XmlDataEscape(String value)
         {
-            yield return SecurityElement.Escape(value);
+            return SecurityElement.Escape(value);
         }
 
-        public static IEnumerable<string> TrimDoubleQuotes(string value)
+        public static String TrimDoubleQuotes(String value)
         {
             var trimmed = string.Empty;
             if (!string.IsNullOrEmpty(value) &&
@@ -102,17 +78,17 @@ namespace GitHub.DistributedTask.Logging
                 trimmed = value.Substring(1, value.Length - 2);
             }
 
-            yield return trimmed;
+            return trimmed;
         }
 
-        public static IEnumerable<string> PowerShellPreAmpersandEscape(string value)
+        public static String PowerShellPreAmpersandEscape(String value)
         {
             // if the secret is passed to PS as a command and it causes an error, sections of it can be surrounded by color codes
-            // or printed individually.
-
+            // or printed individually. 
+            
             // The secret secretpart1&secretpart2&secretpart3 would be split into 2 sections:
             // 'secretpart1&secretpart2&' and 'secretpart3'. This method masks for the first section.
-
+            
             // The secret secretpart1&+secretpart2&secretpart3 would be split into 2 sections:
             // 'secretpart1&+' and (no 's') 'ecretpart2&secretpart3'. This method masks for the first section.
 
@@ -136,10 +112,10 @@ namespace GitHub.DistributedTask.Logging
                 }
             }
 
-            yield return trimmed;
+            return trimmed;
         }
 
-        public static IEnumerable<string> PowerShellPostAmpersandEscape(string value)
+        public static String PowerShellPostAmpersandEscape(String value)
         {
             var trimmed = string.Empty;
             if (!string.IsNullOrEmpty(value) && value.Contains("&"))
@@ -161,10 +137,27 @@ namespace GitHub.DistributedTask.Logging
                 }
             }
 
-            yield return trimmed;
+            return trimmed;
         }
 
-        private static string UriDataEscape(string value, Int32 maxSegmentSize)
+        private static string Base64StringEscapeShift(String value, int shift)
+        {
+            var bytes = Encoding.UTF8.GetBytes(value);
+            if (bytes.Length > shift)
+            {
+                var shiftArray = new byte[bytes.Length - shift];
+                Array.Copy(bytes, shift, shiftArray, 0, bytes.Length - shift);
+                return Convert.ToBase64String(shiftArray);
+            }
+            else
+            {
+                return Convert.ToBase64String(bytes);
+            }
+        }
+
+        private static String UriDataEscape(
+            String value,
+            Int32 maxSegmentSize)
         {
             if (value.Length <= maxSegmentSize)
             {
@@ -190,7 +183,5 @@ namespace GitHub.DistributedTask.Logging
 
             return result.ToString();
         }
-
-        private const char BASE64_PADDING_SUFFIX = '=';
     }
 }

src/Sdk/DTWebApi/WebApi/Runner.cs

@@ -59,5 +59,16 @@ namespace GitHub.DistributedTask.WebApi
             get;
             internal set;
         }
+
+        public TaskAgent ApplyToTaskAgent(TaskAgent agent)
+        {
+            agent.Id = this.Id;
+            agent.Authorization = new TaskAgentAuthorization()
+            {
+                AuthorizationUrl = this.RunnerAuthorization.AuthorizationUrl,
+                ClientId = new Guid(this.RunnerAuthorization.ClientId)
+            };
+            return agent;
+        }
     }
 }

src/Sdk/WebApi/WebApi/Contracts.cs

@@ -1,4 +1,3 @@
-using System.Collections.Generic;
 using System.Runtime.Serialization;
 using Newtonsoft.Json;
 using Newtonsoft.Json.Serialization;
@@ -127,46 +126,6 @@ namespace GitHub.Services.Results.Contracts
         public bool Ok;
     }
 
-    [DataContract]
-    [JsonObject(NamingStrategyType = typeof(SnakeCaseNamingStrategy))]
-    public class StepsUpdateRequest
-    {
-        [DataMember]
-        public IEnumerable<Step> Steps;
-        [DataMember]
-        public long ChangeOrder;
-        [DataMember]
-        public string WorkflowJobRunBackendId;
-        [DataMember]
-        public string WorkflowRunBackendId;
-    }
-
-    [DataContract]
-    [JsonObject(NamingStrategyType = typeof(SnakeCaseNamingStrategy))]
-    public class Step
-    {
-        [DataMember]
-        public string ExternalId;
-        [DataMember]
-        public int Number;
-        [DataMember]
-        public string Name;
-        [DataMember]
-        public Status Status;
-        [DataMember]
-        public string StartedAt;
-        [DataMember]
-        public string CompletedAt;
-    }
-
-    public enum Status
-    {
-        StatusUnknown = 0,
-        StatusInProgress = 3,
-        StatusPending = 5,
-        StatusCompleted = 6
-    }
-
     public static class BlobStorageTypes
     {
         public static readonly string AzureBlobStorage = "BLOB_STORAGE_TYPE_AZURE";

src/Sdk/WebApi/WebApi/ResultsHttpClient.cs

@@ -1,16 +1,11 @@
 using System;
-using System.Collections.Generic;
-using System.Diagnostics;
 using System.IO;
-using System.Linq;
 using System.Net.Http;
 using System.Net.Http.Headers;
 using System.Threading;
 using System.Threading.Tasks;
-using System.Net.Http.Formatting;
-using GitHub.DistributedTask.WebApi;
-using GitHub.Services.Common;
 using GitHub.Services.Results.Contracts;
+using System.Net.Http.Formatting;
 using Sdk.WebApi.WebApi;
 
 namespace GitHub.Services.Results.Client
@@ -27,7 +22,6 @@ namespace GitHub.Services.Results.Client
             m_token = token;
             m_resultsServiceUrl = baseUrl;
             m_formatter = new JsonMediaTypeFormatter();
-            m_changeIdCounter = 1;
         }
 
         // Get Sas URL calls
@@ -92,7 +86,7 @@ namespace GitHub.Services.Results.Client
 
         // Create metadata calls
 
-        private async Task SendRequest<R>(Uri uri, CancellationToken cancellationToken, R request, string timestamp)
+        private async Task CreateMetadata<R>(Uri uri, CancellationToken cancellationToken, R request, string timestamp)
         {
             using (HttpRequestMessage requestMessage = new HttpRequestMessage(HttpMethod.Post, uri))
             {
@@ -127,7 +121,7 @@ namespace GitHub.Services.Results.Client
             };
 
             var createStepSummaryMetadataEndpoint = new Uri(m_resultsServiceUrl, Constants.CreateStepSummaryMetadata);
-            await SendRequest<StepSummaryMetadataCreate>(createStepSummaryMetadataEndpoint, cancellationToken, request, timestamp);
+            await CreateMetadata<StepSummaryMetadataCreate>(createStepSummaryMetadataEndpoint, cancellationToken, request, timestamp);
         }
 
         private async Task StepLogUploadCompleteAsync(string planId, string jobId, Guid stepId, long lineCount, CancellationToken cancellationToken)
@@ -143,7 +137,7 @@ namespace GitHub.Services.Results.Client
             };
 
             var createStepLogsMetadataEndpoint = new Uri(m_resultsServiceUrl, Constants.CreateStepLogsMetadata);
-            await SendRequest<StepLogsMetadataCreate>(createStepLogsMetadataEndpoint, cancellationToken, request, timestamp);
+            await CreateMetadata<StepLogsMetadataCreate>(createStepLogsMetadataEndpoint, cancellationToken, request, timestamp);
         }
 
         private async Task JobLogUploadCompleteAsync(string planId, string jobId, long lineCount, CancellationToken cancellationToken)
@@ -158,7 +152,7 @@ namespace GitHub.Services.Results.Client
             };
 
             var createJobLogsMetadataEndpoint = new Uri(m_resultsServiceUrl, Constants.CreateJobLogsMetadata);
-            await SendRequest<JobLogsMetadataCreate>(createJobLogsMetadataEndpoint, cancellationToken, request, timestamp);
+            await CreateMetadata<JobLogsMetadataCreate>(createJobLogsMetadataEndpoint, cancellationToken, request, timestamp);
         }
 
         private async Task<HttpResponseMessage> UploadBlockFileAsync(string url, string blobStorageType, FileStream file, CancellationToken cancellationToken)
@@ -258,7 +252,7 @@ namespace GitHub.Services.Results.Client
             await StepSummaryUploadCompleteAsync(planId, jobId, stepId, fileSize, cancellationToken);
         }
 
-        // Handle file upload for step log
+        // Handle file upload for step log 
         public async Task UploadResultsStepLogAsync(string planId, string jobId, Guid stepId, string file, bool finalize, bool firstBlock, long lineCount, CancellationToken cancellationToken)
         {
             // Get the upload url
@@ -268,7 +262,7 @@ namespace GitHub.Services.Results.Client
                 throw new Exception("Failed to get step log upload url");
             }
 
-            // Create the Append blob
+            // Create the Append blob 
             if (firstBlock)
             {
                 await CreateAppendFileAsync(uploadUrlResponse.LogsUrl, uploadUrlResponse.BlobStorageType, cancellationToken);
@@ -289,7 +283,7 @@ namespace GitHub.Services.Results.Client
             }
         }
 
-        // Handle file upload for job log
+        // Handle file upload for job log 
         public async Task UploadResultsJobLogAsync(string planId, string jobId, string file, bool finalize, bool firstBlock, long lineCount, CancellationToken cancellationToken)
         {
             // Get the upload url
@@ -299,7 +293,7 @@ namespace GitHub.Services.Results.Client
                 throw new Exception("Failed to get job log upload url");
             }
 
-            // Create the Append blob
+            // Create the Append blob 
             if (firstBlock)
             {
                 await CreateAppendFileAsync(uploadUrlResponse.LogsUrl, uploadUrlResponse.BlobStorageType, cancellationToken);
@@ -320,57 +314,9 @@ namespace GitHub.Services.Results.Client
             }
         }
 
-        private Step ConvertTimelineRecordToStep(TimelineRecord r)
-        {
-            return new Step()
-            {
-                ExternalId = r.Id.ToString(),
-                Number = r.Order.GetValueOrDefault(),
-                Name = r.Name,
-                Status = ConvertStateToStatus(r.State.GetValueOrDefault()),
-                StartedAt = r.StartTime?.ToString(Constants.TimestampFormat),
-                CompletedAt = r.FinishTime?.ToString(Constants.TimestampFormat)
-            };
-        }
-
-        private Status ConvertStateToStatus(TimelineRecordState s)
-        {
-            switch (s)
-            {
-                case TimelineRecordState.Completed:
-                    return Status.StatusCompleted;
-                case TimelineRecordState.Pending:
-                    return Status.StatusPending;
-                case TimelineRecordState.InProgress:
-                    return Status.StatusInProgress;
-                default:
-                    return Status.StatusUnknown;
-            }
-        }
-
-        public async Task UpdateWorkflowStepsAsync(Guid planId, IEnumerable<TimelineRecord> records, CancellationToken cancellationToken)
-        {
-            var timestamp = DateTime.UtcNow.ToString(Constants.TimestampFormat);
-            var stepRecords = records.Where(r => String.Equals(r.RecordType, "Task", StringComparison.Ordinal));
-            var stepUpdateRequests = stepRecords.GroupBy(r => r.ParentId).Select(sg => new StepsUpdateRequest()
-            {
-                WorkflowRunBackendId = planId.ToString(),
-                WorkflowJobRunBackendId = sg.Key.ToString(),
-                ChangeOrder = m_changeIdCounter++,
-                Steps = sg.Select(ConvertTimelineRecordToStep)
-            });
-
-            var stepUpdateEndpoint = new Uri(m_resultsServiceUrl, Constants.WorkflowStepsUpdate);
-            foreach (var request in stepUpdateRequests)
-            {
-                await SendRequest<StepsUpdateRequest>(stepUpdateEndpoint, cancellationToken, request, timestamp);
-            }
-        }
-
         private MediaTypeFormatter m_formatter;
         private Uri m_resultsServiceUrl;
         private string m_token;
-        private int m_changeIdCounter;
     }
 
     // Constants specific to results
@@ -385,8 +331,6 @@ namespace GitHub.Services.Results.Client
         public static readonly string CreateStepLogsMetadata = ResultsReceiverTwirpEndpoint + "CreateStepLogsMetadata";
         public static readonly string GetJobLogsSignedBlobURL = ResultsReceiverTwirpEndpoint + "GetJobLogsSignedBlobURL";
         public static readonly string CreateJobLogsMetadata = ResultsReceiverTwirpEndpoint + "CreateJobLogsMetadata";
-        public static readonly string ResultsProtoApiV1Endpoint = "twirp/github.actions.results.api.v1.WorkflowStepUpdateService/";
-        public static readonly string WorkflowStepsUpdate = ResultsProtoApiV1Endpoint + "WorkflowStepsUpdate";
 
         public static readonly string AzureBlobSealedHeader = "x-ms-blob-sealed";
         public static readonly string AzureBlobTypeHeader = "x-ms-blob-type";

src/Test/L0/HostContextL0.cs

@@ -1,3 +1,4 @@
+using GitHub.Runner.Common.Util;
 using System;
 using System.IO;
 using System.Reflection;
@@ -12,7 +13,6 @@ namespace GitHub.Runner.Common.Tests
     {
         private HostContext _hc;
         private CancellationTokenSource _tokenSource;
-        private const string EXPECTED_SECRET_MASK = "***";
 
         [Fact]
         [Trait("Level", "L0")]
@@ -95,11 +95,11 @@ namespace GitHub.Runner.Common.Tests
                 Assert.Equal("123***123", _hc.SecretMasker.MaskSecrets("123Pass%20word%20123%21123"));
                 Assert.Equal("123***123", _hc.SecretMasker.MaskSecrets("123Pass<word>123!123"));
                 Assert.Equal("123***123", _hc.SecretMasker.MaskSecrets("123Pass''word''123!123"));
-                Assert.Equal("OlBh***==", _hc.SecretMasker.MaskSecrets(Convert.ToBase64String(Encoding.UTF8.GetBytes($":Password123!"))));
-                Assert.Equal("YTpQ***=", _hc.SecretMasker.MaskSecrets(Convert.ToBase64String(Encoding.UTF8.GetBytes($"a:Password123!"))));
+                Assert.Equal("OlBh***", _hc.SecretMasker.MaskSecrets(Convert.ToBase64String(Encoding.UTF8.GetBytes($":Password123!"))));
+                Assert.Equal("YTpQ***", _hc.SecretMasker.MaskSecrets(Convert.ToBase64String(Encoding.UTF8.GetBytes($"a:Password123!"))));
                 Assert.Equal("YWI6***", _hc.SecretMasker.MaskSecrets(Convert.ToBase64String(Encoding.UTF8.GetBytes($"ab:Password123!"))));
-                Assert.Equal("YWJjOlBh***==", _hc.SecretMasker.MaskSecrets(Convert.ToBase64String(Encoding.UTF8.GetBytes($"abc:Password123!"))));
-                Assert.Equal("YWJjZDpQ***=", _hc.SecretMasker.MaskSecrets(Convert.ToBase64String(Encoding.UTF8.GetBytes($"abcd:Password123!"))));
+                Assert.Equal("YWJjOlBh***", _hc.SecretMasker.MaskSecrets(Convert.ToBase64String(Encoding.UTF8.GetBytes($"abc:Password123!"))));
+                Assert.Equal("YWJjZDpQ***", _hc.SecretMasker.MaskSecrets(Convert.ToBase64String(Encoding.UTF8.GetBytes($"abcd:Password123!"))));
                 Assert.Equal("YWJjZGU6***", _hc.SecretMasker.MaskSecrets(Convert.ToBase64String(Encoding.UTF8.GetBytes($"abcde:Password123!"))));
                 Assert.Equal("123***123", _hc.SecretMasker.MaskSecrets("123Password123!!123"));
                 Assert.Equal("123short123", _hc.SecretMasker.MaskSecrets("123short123"));
@@ -112,116 +112,6 @@ namespace GitHub.Runner.Common.Tests
             }
         }
 
-        [Fact]
-        [Trait("Level", "L0")]
-        [Trait("Category", "Common")]
-        public void Base64SecretMaskers()
-        {
-
-            // The following are good candidate strings for Base64 encoding because they include
-            // both standard and RFC 4648 Base64 digits in all offset variations.
-            //   TeLL? noboDy~ SEcreT?
-            //   tElL~ NEVER~ neveR?
-            //   TIGht? Tight~ guard~
-            //   pRIVAte~ guARd? TIghT~
-            //   KeeP~ TIgHT? tIgHT~
-            //   LoCk? TiGhT~ TIght~
-            //   DIvULGe~ nObODY~ noBOdy?
-            //   foreVER~ Tight~ GUaRd?
-
-            try
-            {
-                // Arrange.
-                Setup();
-
-                // Act.
-                _hc.SecretMasker.AddValue("TeLL? noboDy~ SEcreT?");
-
-                // The above string has the following Base64 variations based on the chop leading byte(s) method of Base64 aliasing:
-                var base64Variations = new[]
-                {
-                    "VGVMTD8gbm9ib0R5fiBTRWNyZVQ/",
-                    "ZUxMPyBub2JvRHl+IFNFY3JlVD8",
-                    "TEw/IG5vYm9EeX4gU0VjcmVUPw",
-
-                    // RFC 4648 (URL-safe Base64)
-                    "VGVMTD8gbm9ib0R5fiBTRWNyZVQ_",
-                    "ZUxMPyBub2JvRHl-IFNFY3JlVD8",
-                    "TEw_IG5vYm9EeX4gU0VjcmVUPw"
-                };
-
-                var bookends = new[]
-                {
-                    (string.Empty, string.Empty),
-                    (string.Empty, "="),
-                    (string.Empty, "=="),
-                    (string.Empty, "==="),
-                    ("a", "z"),
-                    ("A", "Z"),
-                    ("abc", "abc"),
-                    ("ABC", "ABC"),
-                    ("0", "0"),
-                    ("00", "00"),
-                    ("000", "000"),
-                    ("123", "789"),
-                    ("`", "`"),
-                    ("'", "'"),
-                    ("\"", "\""),
-                    ("[", "]"),
-                    ("(", ")"),
-                    ("$(", ")"),
-                    ("{", "}"),
-                    ("${", "}"),
-                    ("!", "!"),
-                    ("!!", "!!"),
-                    ("%", "%"),
-                    ("%%", "%%"),
-                    ("_", "_"),
-                    ("__", "__"),
-                    (":", ":"),
-                    ("::", "::"),
-                    (";", ";"),
-                    (";;", ";;"),
-                    (":", string.Empty),
-                    (";", string.Empty),
-                    (string.Empty, ":"),
-                    (string.Empty, ";"),
-                    ("VGVMTD8gbm9ib", "ZUxMPy"),
-                    ("VGVMTD8gbm9ib", "TEw/IG5vYm9EeX4"),
-                    ("ZUxMPy", "TEw/IG5vYm9EeX4"),
-                    ("VGVMTD8gbm9ib", string.Empty),
-                    ("TEw/IG5vYm9EeX4", string.Empty),
-                    ("ZUxMPy", string.Empty),
-                    (string.Empty, "VGVMTD8gbm9ib"),
-                    (string.Empty, "TEw/IG5vYm9EeX4"),
-                    (string.Empty, "ZUxMPy"),
-                };
-
-                foreach (var variation in base64Variations)
-                {
-                    foreach (var pair in bookends)
-                    {
-                        var (prefix, suffix) = pair;
-                        var expected = string.Format("{0}{1}{2}", prefix, EXPECTED_SECRET_MASK, suffix);
-                        var payload = string.Format("{0}{1}{2}", prefix, variation, suffix);
-                        Assert.Equal(expected, _hc.SecretMasker.MaskSecrets(payload));
-                    }
-
-                    // Verify no masking is performed on a partial match.
-                    for (int i = 1; i < variation.Length - 1; i++)
-                    {
-                        var fragment = variation[..i];
-                        Assert.Equal(fragment, _hc.SecretMasker.MaskSecrets(fragment));
-                    }
-                }
-            }
-            finally
-            {
-                // Cleanup.
-                Teardown();
-            }
-        }
-
         [Theory]
         [InlineData("secret&secret&secret", "secret&secret&\x0033[96msecret\x0033[0m", "***\x0033[96m***\x0033[0m")]
         [InlineData("secret&secret+secret", "secret&\x0033[96msecret+secret\x0033[0m", "***\x0033[96m***\x0033[0m")]

src/Test/L0/Listener/Configuration/ConfigurationManagerL0.cs

@@ -73,13 +73,6 @@ namespace GitHub.Runner.Common.Tests.Listener.Configuration
                 AuthorizationUrl = new Uri("http://localhost:8080/pipelines"),
             };
 
-            var expectedRunner = new GitHub.DistributedTask.WebApi.Runner() { Name = expectedAgent.Name, Id = 1 };
-            expectedRunner.RunnerAuthorization = new GitHub.DistributedTask.WebApi.Runner.Authorization
-            {
-                ClientId = expectedAgent.Authorization.ClientId.ToString(),
-                AuthorizationUrl = new Uri("http://localhost:8080/pipelines"),
-            };
-
             var connectionData = new ConnectionData()
             {
                 InstanceId = Guid.NewGuid(),
@@ -117,7 +110,7 @@ namespace GitHub.Runner.Common.Tests.Listener.Configuration
 
             _dotcomServer.Setup(x => x.GetRunnersAsync(It.IsAny<int>(), It.IsAny<string>(), It.IsAny<string>(), It.IsAny<string>())).Returns(Task.FromResult(expectedAgents));
             _dotcomServer.Setup(x => x.GetRunnerGroupsAsync(It.IsAny<string>(), It.IsAny<string>())).Returns(Task.FromResult(expectedPools));
-            _dotcomServer.Setup(x => x.AddRunnerAsync(It.IsAny<int>(), It.IsAny<TaskAgent>(), It.IsAny<string>(), It.IsAny<string>(), It.IsAny<string>())).Returns(Task.FromResult(expectedRunner));
+            _dotcomServer.Setup(x => x.AddRunnerAsync(It.IsAny<int>(), It.IsAny<TaskAgent>(), It.IsAny<string>(), It.IsAny<string>())).Returns(Task.FromResult(expectedAgent));
 
             rsa = new RSACryptoServiceProvider(2048);
 

src/Test/L0/TestHostContext.cs

@@ -1,4 +1,4 @@
-using GitHub.Runner.Common.Util;
+using GitHub.Runner.Common.Util;
 using System;
 using System.Collections.Concurrent;
 using System.Globalization;
@@ -56,12 +56,9 @@ namespace GitHub.Runner.Common.Tests
             }
 
             var traceListener = new HostTraceListener(TraceFileName);
-            var encoders = new List<ValueEncoder>()
-            {
-                ValueEncoders.JsonStringEscape,
-                ValueEncoders.UriDataEscape
-            };
-            _secretMasker = new SecretMasker(encoders);
+            _secretMasker = new SecretMasker();
+            _secretMasker.AddValueEncoder(ValueEncoders.JsonStringEscape);
+            _secretMasker.AddValueEncoder(ValueEncoders.UriDataEscape);
             _traceManager = new TraceManager(traceListener, null, _secretMasker);
             _trace = GetTrace(nameof(TestHostContext));
 

src/Test/L0/Util/IOUtilL0.cs

@@ -1,3 +1,4 @@
+using GitHub.Runner.Common.Util;
 using GitHub.Runner.Sdk;
 using System;
 using System.IO;
@@ -930,36 +931,6 @@ namespace GitHub.Runner.Common.Tests.Util
             }
         }
 
-        [Fact]
-        [Trait("Level", "L0")]
-        [Trait("Category", "Common")]
-        public void LoadObject_ThrowsOnRequiredLoadObject()
-        {
-            using (TestHostContext hc = new(this))
-            {
-                Tracing trace = hc.GetTrace();
-
-                // Arrange: Create a directory with a file.
-                string directory = Path.Combine(hc.GetDirectory(WellKnownDirectory.Bin), Path.GetRandomFileName());
-
-                string file = Path.Combine(directory, "empty file");
-                Directory.CreateDirectory(directory);
-
-                File.WriteAllText(path: file, contents: "");
-                Assert.Throws<ArgumentNullException>(() => IOUtil.LoadObject<RunnerSettings>(file, true));
-
-                file = Path.Combine(directory, "invalid type file");
-                File.WriteAllText(path: file, contents: " ");
-                Assert.Throws<ArgumentException>(() => IOUtil.LoadObject<RunnerSettings>(file, true));
-
-                // Cleanup.
-                if (Directory.Exists(directory))
-                {
-                    Directory.Delete(directory, recursive: true);
-                }
-            }
-        }
-
         private static async Task CreateDirectoryReparsePoint(IHostContext context, string link, string target)
         {
 #if OS_WINDOWS