Commit Graph

711 Commits

Author SHA1 Message Date
Yashwanth Anantharaju
896152d78e send annotations to run-service (#2574)
* send annotations to run-service

* skip message deletion

* actually don't skip deletion

* enum as numbers

* fix enum

* linting

* remove unncessary file

* feedback
2023-05-01 08:33:03 -04:00
Yang Cao
8d74a9ead6 Fix null guard bug (#2576) 2023-04-28 20:56:10 +00:00
Per Lundberg
77b8586a03 contribute.md: Fix link to style guidelines (#2560) 2023-04-27 16:19:56 -04:00
Yashwanth Anantharaju
c8c47d4f27 handle conflict errors from run service (#2570)
* handle conflict errors from run service

* nit: formatting

* fix formatting
2023-04-27 13:59:12 -04:00
Tingluo Huang
58f3ff55aa Prepare 2.304.0 runner release. (#2569) 2023-04-26 15:48:13 -04:00
Tingluo Huang
6353ac84d7 Add orchestrationId to useragent for better correlation. (#2568)
* Add orchestrationId to useragent for better correlation.

* .
2023-04-26 19:15:19 +00:00
Yang Cao
ad9a4a45d1 Bypass job server when run service is enabled (#2516)
* Only upload to Results with new job message type

* No need to have separate websocketFeedServer

* Linting fix

* Update src/Runner.Common/JobServerQueue.cs

Co-authored-by: Tingluo Huang <tingluohuang@github.com>

* add connection timeout

* Consolidate initializing webclient to result client

* Retry websocket delivery for console logs

* Linter fix

* Do not give up for good, reconnect again in 10 minutes

* Has to reset delivered

* Only first time retry 3 times to connect to websocket

---------

Co-authored-by: Tingluo Huang <tingluohuang@github.com>
2023-04-26 12:51:56 -04:00
John Wesley Walker III
a41397ae93 Rename AcquireJobRequest::StreamID to AcquireJobRequest::JobMessageID (#2547)
* Rename AcquireJobRequest::StreamID to AcquireJobRequest::JobMessageID to match corresponding server-side update.
* appeased the linter
* Added unit tests to prove AcquireJobRequest serialization/deserialization is as-expected.
* Distinguish unit test variations.
* Incorporated PR Feedback.
2023-04-26 18:31:41 +02:00
Erez Testiler
c4d41e95cb Add *.ghe.localhost domains to hosted server check (#2536) 2023-04-24 12:52:28 -04:00
Maggie Spletzer
af6ed41bcb Adding curl --retry-all-errors for external tool downloads (#2557)
* adding retry-all-errors based on curl version >= 7.71.0

* adding logging

* fixing conditional logic
2023-04-20 21:01:11 +00:00
Tingluo Huang
e8b2380a20 Limit the time we wait for waiting websocket to connect. (#2554) 2023-04-19 10:20:00 -04:00
Maggie Spletzer
38ab9dedf4 Adding curl retry for external tool downloads (#2552)
* adding retry for flaky downloads

* adding comment

* Update src/Misc/externals.sh

Co-authored-by: Tingluo Huang <tingluohuang@github.com>

---------

Co-authored-by: Tingluo Huang <tingluohuang@github.com>
2023-04-18 18:31:46 +00:00
Yashwanth Anantharaju
c7629700ad Handle non success raw http responses (#2541)
* handle non success responses

* fix format

* nits

Co-authored-by: Tingluo Huang <tingluohuang@github.com>

---------

Co-authored-by: Tingluo Huang <tingluohuang@github.com>
2023-04-14 16:17:20 -04:00
Yang Cao
b9a0b5dba9 If conclusion is not set, we cannot get default value (#2535) 2023-04-13 13:03:57 -04:00
Yang Cao
766cefe599 Also send in conclusion for steps (#2531) 2023-04-13 15:19:00 +02:00
Eli Entelis
2ecd7d2fc6 which handles broken symlink & unit test added (#2150) (#2196)
* which handles broken symlink & unit test added (#2150)

* Update src/Runner.Sdk/Util/WhichUtil.cs

Co-authored-by: Ferenc Hammerl <31069338+fhammerl@users.noreply.github.com>

* fix pr comments

* trace log added,  in case we find a symlink that is broken

* add check in case the target of the symlink is a relative path; added test that check symlink that targets a full path; added test that check symlink that targets a relative path;

* fix tests

* fix tests for linux

---------

Co-authored-by: Eli Entelis <eli.entelis@hexagon.com>
Co-authored-by: Eli Entelis <42981948+Eliminator1999@users.noreply.github.com>
Co-authored-by: Ferenc Hammerl <31069338+fhammerl@users.noreply.github.com>
2023-04-06 22:15:11 -04:00
Yang Cao
0484afeec7 Update Runner to send step updates to Results (#2510)
* Also send Steps update to Results service

* Refactor to separate results server from current job server

* If hit any error while uploading to Results, skip Results upload

* Add proxy authentication and buffer request for WinHttpHandler

* Remove unnecessary null guard

* Also send Results telemetry when step update fails

* IResultsServer is not disposable
2023-04-03 16:31:13 -04:00
Nikola Jokic
1ceb1a67f2 Guard against NullReference while creating HostContext (#2343)
* Guard against NullReference while creating HostContext

* Throw on IOUtil loading object if content is empty or object is null

* removed unused dependency

* Changed exceptions to ArgumentNullException and ArgumentException

* fixed my mistake on LoadObject

* fixed tests

* trigger workflows

* trigger workflows

* encode files using utf8
2023-03-30 15:39:09 +02:00
Luke Tomlinson
9f778b814d Register Runners against V2 servers (#2505)
* Parse runners and send publicKey

* wip

* Fix conflicts

* Cleanup

* Cleanup

* fix test

* fix test

* Add trace for broker message listener

* Feedback

* refactor

* remove dead code

* Remove old comment
2023-03-28 15:45:00 -04:00
Luke Tomlinson
92258f9ea1 Listen directly to Broker for messages (#2500) 2023-03-24 14:00:34 -04:00
Bassem Dghaidi
74eeb82684 ADR: Runner Image Tags (#2494)
* WIP

* WIP

* Add context

* Add 2494-runner-image-tags ADR draft

* Fix ADR title

* Add more information to option 2

* Add decision

* Fix status
2023-03-23 12:09:10 -04:00
Matisse Hack
0e7ca9aedb Fix JIT configurations on Windows (#2497)
* Fix JIT configurations on Windows

* Update src/Runner.Listener/Runner.cs
2023-03-21 15:04:50 -04:00
Tatyana Kostromskaya
bb7b1e8259 Update runner to handle Dotcom/runner-admin based registration flow (#2487) 2023-03-21 16:05:32 +01:00
Tingluo Huang
440c81b770 Bump container hooks version to 0.3.1 (#2496) 2023-03-20 14:53:47 +00:00
Bassem Dghaidi
9958fc0374 Update container hooks version to 0.3.0 (#2492) 2023-03-17 06:00:55 -04:00
Tingluo Huang
81b07eb1c4 Prepare runner release 2.303.0 (#2482) 2023-03-10 10:45:56 +00:00
Jongwoo Han
514ecec5a3 Replace deprecated command with environment file (#2429)
Signed-off-by: jongwooo <jongwooo.han@gmail.com>
2023-03-09 22:09:05 -05:00
Ajay
128b212b13 Support matrix context in output keys (#2477) 2023-03-09 21:30:28 -05:00
Nikola Jokic
2dfa28e6e0 Add update certificates to ./run.sh if RUNNER_UPDATE_CA_CERTS env is set (#2471)
* Included entrypoint that will update certs and run ./run.sh

* update ca if RUNNER_UPDATE_CA env is set

* changed env variable to RUNNER_UPDATE_TRUST_STORE

* moved entrypoint to be run.sh, removed Dockerfile entrypoint, added envvar that will update certs

* Update src/Misc/layoutroot/run.sh

Co-authored-by: Bassem Dghaidi <568794+Link-@users.noreply.github.com>

* Update src/Misc/layoutroot/run.sh

Co-authored-by: Bassem Dghaidi <568794+Link-@users.noreply.github.com>

* Update src/Misc/layoutroot/run.sh

Co-authored-by: Bassem Dghaidi <568794+Link-@users.noreply.github.com>

* Update src/Misc/layoutroot/run.sh

Co-authored-by: Tingluo Huang <tingluohuang@github.com>

* Update src/Misc/layoutroot/run.sh

Co-authored-by: Tingluo Huang <tingluohuang@github.com>

* removed doc comment on func

---------

Co-authored-by: Bassem Dghaidi <568794+Link-@users.noreply.github.com>
Co-authored-by: Tingluo Huang <tingluohuang@github.com>
2023-03-08 12:29:55 -05:00
Nikola Jokic
fd96246580 Exit on deprication error (#2299)
* terminate the runner on deprication message

* added TaskAgentVersion exception to catch deprication

* AccessDenied exception with inner exception InvalidTaskAgent

* Access denied exception in program and in message listener

* Fixed copy

* remove trace message from message listener
2023-03-07 14:09:10 +01:00
Ferenc Hammerl
8ef48200b4 Bypass all proxies for all hosts if no_proxy='*' is set (#2395)
* Bypass top level domain even if no_proxy specified it with leading '.'

E.g. no_proxy='.github.com' will now bypass github.com.

* Bypass proxy on all hosts when no_proxy is * (wildcard)

* Undo '.' stripping

* Simplify unit tests

* Respect wildcard even if it's one of many no_proxy items
2023-03-06 11:01:45 +01:00
Tingluo Huang
d61b27b839 Change runner image to make user/folder align with ubuntu-latest hosted runner. (#2469) 2023-03-02 13:42:23 -05:00
Nikola Jokic
542e8a3c98 Runner service exit after consecutive re-try exits (#2426)
* Runner service exit after consecutive re-try exits

* Rename failure counts and include reset count on runner listening for jobs

* Changed from graceful shutdown to stopping=true
2023-02-28 16:00:36 +01:00
Yashwanth Anantharaju
e8975514fd call run service renewjob (#2461)
* call run service renewjob

* format

* formatting

* make it private and expose internals

* lint

* fix exception class

* lint

* fix test as well
2023-02-27 16:50:28 +00:00
Yang Cao
0befa62f64 Add job log upload support (#2447)
* Refactor and add job log upload support

* Rename method to be consistent
2023-02-21 09:55:47 -05:00
Yang Cao
aaf02ab34c Properly guard upload (#2439)
* Revert "Revert "Uploading step logs to Results as well  (#2422)" (#2437)"

This reverts commit 8c096baf49.

* Properly guard the upload to results feature

* Delete skipped file if deletesource is true
2023-02-17 10:31:41 -05:00
JoannaaKL
02c9d1c704 Don't disable lint errors (#2436)
* Update lint.yml

Don't ignore the formatting errors

* Add formatting made by @cory-miller

* Use dotnet format

* Format only modified files

* Add instruction to contribute.md

* Use git status instead of git diff
2023-02-16 14:33:03 +01:00
Tingluo Huang
982784d704 Wait for docker base on env RUNNER_WAIT_FOR_DOCKER_IN_SECONDS. (#2440) 2023-02-15 16:05:36 -05:00
Yang Cao
8c096baf49 Revert "Uploading step logs to Results as well (#2422)" (#2437)
This reverts commit e979331be4.
2023-02-15 11:21:32 -05:00
JoannaaKL
8d6972e38b Don't add Needs constant twice 2023-02-15 10:50:22 +00:00
Tingluo Huang
1ab35b0938 Use v2 version based on https://github.blog/changelog/2023-01-18-code-scanning-codeql-action-v1-is-now-deprecated/ (#2434) 2023-02-14 16:01:52 +00:00
Tingluo Huang
f86e968d38 Prepare runner release 2.302.0 (#2433) 2023-02-14 09:50:09 -05:00
Yang Cao
e979331be4 Uploading step logs to Results as well (#2422)
* Rename queue to results queue

* Add results contracts

* Add Results logging handling

* Adding calls to create and finalize append blob

* Modifications for azurite upload

* Only call upload complete on final section and remove size

* Make method specific to step log so we can support job log later

* Change contract for results

* Add totalline count to the result log upload file

* Actually pass lineCount to Results Service

* Fix typos

* Code cleanup

* Fixing typos

* Apply suggestions from code review

Co-authored-by: Konrad Pabjan <konradpabjan@github.com>

---------

Co-authored-by: Brittany Ellich <brittanyellich@github.com>
Co-authored-by: Konrad Pabjan <konradpabjan@github.com>
2023-02-13 13:18:56 -05:00
Ferenc Hammerl
97195bad58 Replace '(' and ')' with '[' and '] from OS.Description so it doesn't fail User-Agent header validation (#2288)
* Sanitize OS Desc for UserAgents

* Only drop brackets if needed, refactoring

* Add missing ')'

* Readd missing brackets around '(header)'

* Add comments

* Use bracket solution from SDK

* Rename tests
2023-02-08 17:42:27 +01:00
Tingluo Huang
6d1d2460ac Add docker cli to the runner image. (#2425) 2023-02-08 09:21:02 -05:00
Yashwanth Anantharaju
67356a3305 Run service: send more stuff as part of job completed (#2423)
* send more stuff as part of job completed

* feedback

* set only once

* feedback

* feedback

* fix test

* feedback

* nit: spacing

* nit: line

Co-authored-by: Tingluo Huang <tingluohuang@github.com>

---------

Co-authored-by: Tingluo Huang <tingluohuang@github.com>
2023-02-07 20:10:53 +00:00
John Wesley Walker III
9a228e52e9 Defer evaluation of a step's DisplayName until its condition is evaluated. (#2313)
* Defer evaluation of a step's DisplayName until its condition is evaluated.
* Formalize TryUpdateDisplayName and EvaluateDisplayName as members of interface `IStep` (#2374)
2023-02-07 11:42:30 +01:00
Erez Testiler
3cd76671dd Add support for ghe.com domain (#2420) 2023-02-06 17:16:38 -05:00
Yashwanth Anantharaju
e6e5f36dd0 start calling run service for job completion (#2412)
* start calling run service for job completion

* cleanup

* nit: lines

Co-authored-by: Tingluo Huang <tingluohuang@github.com>

* clean up

* give sanity back to thboop

Co-authored-by: Thomas Boop <52323235+thboop@users.noreply.github.com>

* add clean up back

* clean up

* clean up more

* oops

* copied from existing, but :thumb:

Co-authored-by: Thomas Boop <52323235+thboop@users.noreply.github.com>

---------

Co-authored-by: Tingluo Huang <tingluohuang@github.com>
Co-authored-by: Thomas Boop <52323235+thboop@users.noreply.github.com>
2023-02-01 21:18:31 +00:00
Yashwanth Anantharaju
24a27efd4f fix small bug (#2396) 2023-01-30 10:00:31 -05:00