Commit Graph

158 Commits

Author SHA1 Message Date
Tingluo Huang
f672567acc Collect telemetry to measure upload speed for different backend. (#2912) 2023-10-05 23:29:38 -04:00
Tingluo Huang
f57ecd8e3c Allow use action archive cache to speed up workflow jobs. (#2857)
* Refactor ActionManager.cs

* changes.

* cleanup
2023-09-28 23:43:55 +00:00
Yang Cao
463ec00cb4 Do not give up if Results is powering logs (#2893)
* Do not give us if Results is powering logs

* No need to send telemtry to Actions server in Results only case
2023-09-28 20:05:11 +00:00
Tatyana Kostromskaya
462337a4a4 Add warning about node16 deprecation (#2887)
* add warning about node16 deprecation

* fix comments
2023-09-28 08:52:35 +00:00
Tingluo Huang
8f1c070506 Use Directory.EnumerateFiles instead of Directory.GetFiles in WhichUtil. (#2882)
* Use Directory.EnumerateFiles instead of Directory.GetFiles in WhichUtil.

* .

* .
2023-09-26 20:49:15 +00:00
Pavel Iakovenko
2b66cbe699 Delegating handler for Http redirects (#2814)
* Delegating handler for Http redirects

* Renamed varible name
2023-08-31 17:35:50 -04:00
Luke Tomlinson
0e9e9f1e8d Support replacing runners in v2 flow (#2791)
* Support replacing runners in v2 flow

* Use correct endpoint for listing runners

* fix test

* really fix test

* lint
2023-08-31 09:55:05 -04:00
dependabot[bot]
be65955a9d Bump Newtonsoft.Json from 13.0.1 to 13.0.3 in /src (#2797)
Bumps [Newtonsoft.Json](https://github.com/JamesNK/Newtonsoft.Json) from 13.0.1 to 13.0.3.
- [Release notes](https://github.com/JamesNK/Newtonsoft.Json/releases)
- [Commits](https://github.com/JamesNK/Newtonsoft.Json/compare/13.0.1...13.0.3)

---
updated-dependencies:
- dependency-name: Newtonsoft.Json
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-08-25 13:10:40 -04:00
Tingluo Huang
e0acb14bfc Query runner by name on server side. (#2771) 2023-08-24 21:02:55 -04:00
Kensuke Nagae
1183100ab8 Fix typo (#2741) 2023-08-16 09:04:49 +02:00
Tingluo Huang
4f40f29cff Trace x-github-request-id when download action tarball. (#2755) 2023-08-15 19:00:54 -04:00
Tatyana Kostromskaya
8206cf4e73 Add node20 to runner (#2732)
* .

* fix hashes

* fix linux-arm

* linux-arm64

* osx-x64

* win-x64

* clean up node12 deprecation warning

* Add node20

* Remove node12 from the BuildInVersions list

* fix externals hash
2023-08-11 11:11:28 +02:00
Tatyana Kostromskaya
9eb4b96713 Remove node12 (#2717)
* .

* fix hashes

* clean up node12 deprecation warning

* Remove node12 from the BuildInVersions list
2023-08-10 15:16:30 +02:00
Tingluo Huang
8fa970a1e6 Change RunnerId/AgentId from int32 to uint64 (#2661)
* RunnerId int -> ulong
2023-08-04 16:22:52 -04:00
Tatyana Kostromskaya
e8ee6f7b1b Add warning to notify about forcing node16 (#2678)
* add warning

* fix tests

* aggregate to one warning

* fix linter

* fix comm
2023-07-07 11:35:54 +02:00
Dylan Geraci
1096b975e4 Send environment url to Run Service (#2650)
* add EnvironmentUrl to CompleteJobRequest

* Send environment url to Run Service

* Fix whitespace

* Fix test

* Fix more whitespace

* Apply suggestions from code review

Co-authored-by: Tingluo Huang <tingluohuang@github.com>

* Apply suggestion from code review

Co-authored-by: Tingluo Huang <tingluohuang@github.com>

---------

Co-authored-by: Tingluo Huang <tingluohuang@github.com>
2023-06-16 04:13:08 +00:00
JoannaaKL
282ba4cfc8 Just dont log these errors (#2656)
* Dont log template errors
2023-06-13 21:10:53 -04:00
JoannaaKL
efffbaeabc Add utf8 with bom (#2641)
* Change default file encoding
2023-06-02 21:47:59 +02:00
Gabriel
eeb0cf6f1e Add --no-default-labels config option to self-hosted runners (#2443)
* Add --no-default-labels option

Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>

* Add tests

Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>

* .

---------

Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
Co-authored-by: Tingluo Huang <tingluohuang@github.com>
2023-05-11 12:37:56 -04:00
John Wesley Walker III
f8a28c3c4e [1742] Ensure multiple composite annoations are correctly written. (#2311)
* [1742] Ensure multiple composite annoations are correctly written.

This implementation uses a collector pattern to allow embedded ExecutionContexts to stash Issue objects for later processing by a non-embedded ancestor ExecutionContext.

Also:
 - Provide explicit constructor implementations for ExecutionContext
 - Leverage explicit constructors to solidify immutability of several ExecutionContext class members.
 - Fixed erroneous call to ExecutionContext.Complete in CompositeActionHandler.cs
 - Use a consistent timestamp for FinishTime in ExecutionContext::Complete

* Ensure collected issues are processed only by a non-embedded ExecutionContext.
This was already implicit.  Now, just making it explicit.

* Provide a clear mechanism that allows callers to opt-in/opt-out of ExecutionContext::AddIssue's logging behavior.
* Addressed deserialization inconsistencies in TimelineRecord.cs
* Added TimelineRecord unit tests.
* Refined unit tests related to TimelineRecord::Variables case-insensitivity
* Add a unit test that verifies ExecutionContextLogOptions::LogMessageOverride has the desired effect.
* Responded to PR feedback.
* Don't allow embedded ExecutionContexts to add Issues to a TimelineRecord
2023-05-10 16:24:02 +02:00
Tingluo Huang
1bc14f0607 Trace WebSocket exception into verbose level to reduce noise in diag log. (#2591) 2023-05-08 18:54:34 -04:00
John Hernley
22d1938ac4 Resolve Actions Directly From Launch for Run Service Jobs (#2529)
Co-authored-by: Tingluo Huang <tingluohuang@github.com>
2023-05-03 16:04:21 -04:00
Yashwanth Anantharaju
896152d78e send annotations to run-service (#2574)
* send annotations to run-service

* skip message deletion

* actually don't skip deletion

* enum as numbers

* fix enum

* linting

* remove unncessary file

* feedback
2023-05-01 08:33:03 -04:00
Yang Cao
8d74a9ead6 Fix null guard bug (#2576) 2023-04-28 20:56:10 +00:00
Yashwanth Anantharaju
c8c47d4f27 handle conflict errors from run service (#2570)
* handle conflict errors from run service

* nit: formatting

* fix formatting
2023-04-27 13:59:12 -04:00
Tingluo Huang
6353ac84d7 Add orchestrationId to useragent for better correlation. (#2568)
* Add orchestrationId to useragent for better correlation.

* .
2023-04-26 19:15:19 +00:00
Yang Cao
ad9a4a45d1 Bypass job server when run service is enabled (#2516)
* Only upload to Results with new job message type

* No need to have separate websocketFeedServer

* Linting fix

* Update src/Runner.Common/JobServerQueue.cs

Co-authored-by: Tingluo Huang <tingluohuang@github.com>

* add connection timeout

* Consolidate initializing webclient to result client

* Retry websocket delivery for console logs

* Linter fix

* Do not give up for good, reconnect again in 10 minutes

* Has to reset delivered

* Only first time retry 3 times to connect to websocket

---------

Co-authored-by: Tingluo Huang <tingluohuang@github.com>
2023-04-26 12:51:56 -04:00
Tingluo Huang
e8b2380a20 Limit the time we wait for waiting websocket to connect. (#2554) 2023-04-19 10:20:00 -04:00
Yashwanth Anantharaju
c7629700ad Handle non success raw http responses (#2541)
* handle non success responses

* fix format

* nits

Co-authored-by: Tingluo Huang <tingluohuang@github.com>

---------

Co-authored-by: Tingluo Huang <tingluohuang@github.com>
2023-04-14 16:17:20 -04:00
Yang Cao
0484afeec7 Update Runner to send step updates to Results (#2510)
* Also send Steps update to Results service

* Refactor to separate results server from current job server

* If hit any error while uploading to Results, skip Results upload

* Add proxy authentication and buffer request for WinHttpHandler

* Remove unnecessary null guard

* Also send Results telemetry when step update fails

* IResultsServer is not disposable
2023-04-03 16:31:13 -04:00
Nikola Jokic
1ceb1a67f2 Guard against NullReference while creating HostContext (#2343)
* Guard against NullReference while creating HostContext

* Throw on IOUtil loading object if content is empty or object is null

* removed unused dependency

* Changed exceptions to ArgumentNullException and ArgumentException

* fixed my mistake on LoadObject

* fixed tests

* trigger workflows

* trigger workflows

* encode files using utf8
2023-03-30 15:39:09 +02:00
Luke Tomlinson
9f778b814d Register Runners against V2 servers (#2505)
* Parse runners and send publicKey

* wip

* Fix conflicts

* Cleanup

* Cleanup

* fix test

* fix test

* Add trace for broker message listener

* Feedback

* refactor

* remove dead code

* Remove old comment
2023-03-28 15:45:00 -04:00
Luke Tomlinson
92258f9ea1 Listen directly to Broker for messages (#2500) 2023-03-24 14:00:34 -04:00
Tatyana Kostromskaya
bb7b1e8259 Update runner to handle Dotcom/runner-admin based registration flow (#2487) 2023-03-21 16:05:32 +01:00
Yashwanth Anantharaju
e8975514fd call run service renewjob (#2461)
* call run service renewjob

* format

* formatting

* make it private and expose internals

* lint

* fix exception class

* lint

* fix test as well
2023-02-27 16:50:28 +00:00
Yang Cao
0befa62f64 Add job log upload support (#2447)
* Refactor and add job log upload support

* Rename method to be consistent
2023-02-21 09:55:47 -05:00
Yang Cao
aaf02ab34c Properly guard upload (#2439)
* Revert "Revert "Uploading step logs to Results as well  (#2422)" (#2437)"

This reverts commit 8c096baf49.

* Properly guard the upload to results feature

* Delete skipped file if deletesource is true
2023-02-17 10:31:41 -05:00
Yang Cao
8c096baf49 Revert "Uploading step logs to Results as well (#2422)" (#2437)
This reverts commit e979331be4.
2023-02-15 11:21:32 -05:00
Yang Cao
e979331be4 Uploading step logs to Results as well (#2422)
* Rename queue to results queue

* Add results contracts

* Add Results logging handling

* Adding calls to create and finalize append blob

* Modifications for azurite upload

* Only call upload complete on final section and remove size

* Make method specific to step log so we can support job log later

* Change contract for results

* Add totalline count to the result log upload file

* Actually pass lineCount to Results Service

* Fix typos

* Code cleanup

* Fixing typos

* Apply suggestions from code review

Co-authored-by: Konrad Pabjan <konradpabjan@github.com>

---------

Co-authored-by: Brittany Ellich <brittanyellich@github.com>
Co-authored-by: Konrad Pabjan <konradpabjan@github.com>
2023-02-13 13:18:56 -05:00
Yashwanth Anantharaju
67356a3305 Run service: send more stuff as part of job completed (#2423)
* send more stuff as part of job completed

* feedback

* set only once

* feedback

* feedback

* fix test

* feedback

* nit: spacing

* nit: line

Co-authored-by: Tingluo Huang <tingluohuang@github.com>

---------

Co-authored-by: Tingluo Huang <tingluohuang@github.com>
2023-02-07 20:10:53 +00:00
Yashwanth Anantharaju
e6e5f36dd0 start calling run service for job completion (#2412)
* start calling run service for job completion

* cleanup

* nit: lines

Co-authored-by: Tingluo Huang <tingluohuang@github.com>

* clean up

* give sanity back to thboop

Co-authored-by: Thomas Boop <52323235+thboop@users.noreply.github.com>

* add clean up back

* clean up

* clean up more

* oops

* copied from existing, but :thumb:

Co-authored-by: Thomas Boop <52323235+thboop@users.noreply.github.com>

---------

Co-authored-by: Tingluo Huang <tingluohuang@github.com>
Co-authored-by: Thomas Boop <52323235+thboop@users.noreply.github.com>
2023-02-01 21:18:31 +00:00
Yashwanth Anantharaju
24a27efd4f fix small bug (#2396) 2023-01-30 10:00:31 -05:00
John Sudol
d6f8633efc new option to remove local config files (#2367) 2023-01-18 11:28:43 -05:00
Tingluo Huang
4a6630531b Allow provide extra User-Agent for better correlation. (#2370) 2023-01-16 10:18:55 -05:00
Yang Cao
caec043085 Always upload to avoid issues (#2334)
* Remove unnecessary timelineId and timelineRecordId and use Guid stepId

* Log upload error to kusto

* Remove try-catch

* Using a well known telmetry record to avoid replacing issues

* fix Guid format
2022-12-28 11:56:53 -05:00
Tingluo Huang
a1244d2269 Add Header/Footer to multi-line message in StdoutTraceListener. (#2336) 2022-12-22 10:38:29 -05:00
Nikola Jokic
332b97f838 Treat jitconfig as secret. (#2335)
Co-authored-by: TingluoHuang <TingluoHuang@github.com>
2022-12-21 13:30:22 -05:00
Brittany Ellich
f41f5d259d Use results for uploading step summaries (#2301)
* Use results service for uploading step summaries

* Use results summary over generic results naming convention

* Apply suggestions from code review

Co-authored-by: Tingluo Huang <tingluohuang@github.com>

* Addressing feedback

* Fix merge issue

* Remove empty line

* Update Results json objects to use snake case

* Adding the reference

Co-authored-by: Yang Cao <yacaovsnc@github.com>
Co-authored-by: Tingluo Huang <tingluohuang@github.com>
2022-12-14 09:28:33 +01:00
Ava Stancu
369a4eccad Made worker logs available to stdout (#2307)
* Made worker logs available to stdout

* Log Worker Standard out line by line

Co-authored-by: Ferenc Hammerl <31069338+fhammerl@users.noreply.github.com>
2022-12-08 16:23:52 -05:00
Ava Stancu
088981a372 Listener stdout logging (#2291)
* Added env variable to control wether the terminal is silent

* Log to stdout if PrintLogToStdout is enabled

* Extracted console logging to stdouttracelistener

* Remove useless usings

* Rewrite TraceListener as superclass

* Only print to stdout if env is set

* Add comment for Console.Out

* Format Listener

* Revert var name in terminal

* Check env in hostcontext instead of Tracing constructor

* Remove superclass & dupe logging code

* Log hostType

* Readonly '_' prefix 'hostType'

* Fix test

* Revert Terminal change

Co-authored-by: Ferenc Hammerl <31069338+fhammerl@users.noreply.github.com>
2022-12-06 16:16:00 +01:00