Commit Graph

29 Commits

Author SHA1 Message Date
Tingluo Huang
65201ff6be Include whether http proxy configured as part of UserAgent. (#3009) 2023-11-28 09:43:13 -05:00
Kensuke Nagae
1183100ab8 Fix typo (#2741) 2023-08-16 09:04:49 +02:00
JoannaaKL
efffbaeabc Add utf8 with bom (#2641)
* Change default file encoding
2023-06-02 21:47:59 +02:00
Nikola Jokic
1ceb1a67f2 Guard against NullReference while creating HostContext (#2343)
* Guard against NullReference while creating HostContext

* Throw on IOUtil loading object if content is empty or object is null

* removed unused dependency

* Changed exceptions to ArgumentNullException and ArgumentException

* fixed my mistake on LoadObject

* fixed tests

* trigger workflows

* trigger workflows

* encode files using utf8
2023-03-30 15:39:09 +02:00
Tingluo Huang
4a6630531b Allow provide extra User-Agent for better correlation. (#2370) 2023-01-16 10:18:55 -05:00
Ava Stancu
088981a372 Listener stdout logging (#2291)
* Added env variable to control wether the terminal is silent

* Log to stdout if PrintLogToStdout is enabled

* Extracted console logging to stdouttracelistener

* Remove useless usings

* Rewrite TraceListener as superclass

* Only print to stdout if env is set

* Add comment for Console.Out

* Format Listener

* Revert var name in terminal

* Check env in hostcontext instead of Tracing constructor

* Remove superclass & dupe logging code

* Log hostType

* Readonly '_' prefix 'hostType'

* Fix test

* Revert Terminal change

Co-authored-by: Ferenc Hammerl <31069338+fhammerl@users.noreply.github.com>
2022-12-06 16:16:00 +01:00
Cory Miller
b87b4aac5c Fix IDE0090 (#2211) 2022-10-18 10:54:08 -04:00
Ferenc Hammerl
591f8c3510 Runner container hooks Beta (#1853)
* Added ability to run Dockerfile.SUFFIX ContainerAction

* Extracted IsDockerFile method

* reformatted, moved from index to Last()

* extracted IsDockerfile to DockerUtil with L0

* added check for IsDockerfile to account for docker://

* updated test to clearly show path/dockerfile:tag

* fail if Data.Image is not Dockerfile or docker://[image]

* Setup noops for JobPrepare and JobCleanup hooks

* Add container jobstarted and jobcomplete hooks

* Run 'index.js' instead of specific command hooks

* Call jobprepare with command arg

* Use right command name (hardcoded)

Co-authored-by: Nikola Jokic <nikola-jokic@users.noreply.github.com>

* Invoke hooks with arguments

* Add PrepareJob hook to work with jobcontainers

Co-authored-by: Nikola Jokic <nikola-jokic@users.noreply.github.com>

* Rename methods

* Use new hookcontainer to run prep and clean hooks

* Get path from ENV

* Use enums

* Use IOUtils.cs

* Move container files to folder

* Move namespaces

* Store "state" between hooks

* Remove stdin stream in containerstephosts

* Update Constants.cs

* Throw if stdin fails

* Cleanup obvious nullrefs and unused vars

* Cleanup containerhook directory

* Call step exec hook

* Fix windows build

* Remove hook from hookContainer

* Rename file

* More renamings

* Add TODOs

* Fix env name

* Fix missing imports

* Fix imports

* Run script step on jobcontainer

* Enable feature if env is set

* Update ContainerHookManager.cs

* Update ContainerHookManager.cs

* Hooks allowed to work even when context isn't returned

* Custom hooks enabled flag and additional null checks

* New line at the end of the FeatureFlagManager.cs

* Code refactoring

* Supported just in time container building or pulling

* Try mock-build for osx

* Build all platforms

* Run mock on self-hosted

* Remove GITHUB prefix

* Use ContainerHooksPath instead of CustomHooksPath

* Null checks simplified

* Code refactoring

* Changing condition for image builing/pulling

* Code refactoring

* TODO comment removed

Co-authored-by: Ferenc Hammerl <31069338+fhammerl@users.noreply.github.com>

* Call container step if FF is on

* Rename run script function

* Use JToken instead of dynamic

* Add TODO

* Small refactoring + renames + TODOs

* Throw on DetermineNodeRuntimeVersion

* Fix formatting

* Add run-container-step

* Supported nodeJS in Alpine containers

* Renamed Alpine to IsAlpine in HookResponse

* Method for checking platform for alpine container

* Added container hooks feature flag check

* Update IsHookFeatureEnabled with new params

* Rename featureflag method

* Finish rename

* Set collection null values to empty arrays when JSON serialising them

* Disable FF until we merge

* Update src/Runner.Worker/Container/ContainerHooks/HookContainer.cs

* Fix method name

* Change hookargs to superclass from interface

* Using only Path.Combine in GenerateResponsePath

* fix merge error

* EntryPointArgs changed to list of args instead of one args string

* Changed List to IEnumerable for EntryPointArgs and MountVolumes

* Get ContainerRuntimePath for JobContainers from hooks

* Read ContainerEnv from response file

* Port mappings saved after creating services

* Support case when responseFile doesn't exist

* Check if response file exists

* Logging in ExecuteHookScript

* Save hook state after all 4 hooks

* Code refactoring

* Remove TODO

Co-authored-by: Ferenc Hammerl <31069338+fhammerl@users.noreply.github.com>

* Remove second TODO

Co-authored-by: Ferenc Hammerl <31069338+fhammerl@users.noreply.github.com>

* Removing container env changes

* Removing containerEnv and dockerManager

* Delete mock-build.yml

* Update IOUtil.cs

* Add comment about containerhooks

* Fix merge mistake

* Remove solved todo

* Determine which shell to use for hooks scenario

* Overload for method ExecuteHookScript with prependPath as arg

* Adding HostContext to the GetDefaultShellForScript call

* prependPath as a mandatory parameter

* Improve logging for hooks

* Small changes in logging

* Allow null for ContainerEntryPointArgs

* Changed log messages

* Skip setting EntryPoint and EntryPointArgs if hooks are enabled

* Throw if IsAlpine is null in PrepareJob

* Code refactoring - added GetAndValidateResponse method

* Code refactoring

* Changes in exception message

* Only save hookState if returned

* Use FF from server

* Empty line

* Code refactoring

Co-authored-by: Ferenc Hammerl <31069338+fhammerl@users.noreply.github.com>

* Send null instead of string empty

* Remove TODO

* Code refactoring and some small changes

* Allow Globals to be null to pass L0

* Fix setup in StepHostL0

* Throw exception earlier if response file doesn't exist and prepare_job hook is running

* Refactoring GetResponse method

* Changing exception message if response file is not found

Co-authored-by: Ferenc Hammerl <31069338+fhammerl@users.noreply.github.com>

* Chaning exception message if isAlpine is null for prepare_job hook

* Rename hook folder

* Fail if compatible hookfile not found

* Use .Value instead of casting bool? to bool

* Format spacing

* Formatting

* User user and system mvs

* Use variables instead of entire context in featuremanager

* Update stepTelemetry if step uses containerhooks

* Restore  import

* Remove unneccessary field from HookContainer

* Refactor response context and portmappings

* Force allow hooks if FF is on

* Code refactoring

* Revert deleting usings

* Better hookContainer defaults and use correct portmapping list

* Make GetDefaultShellForScript a  HostContext extension method

* Generic hookresponse

* Code refactoring, unnecessary properties removed - HookContainer moved to the HookInput.cs

* Remove empty line

* Code refactoring and better exception handling

* code refactor, removing unnecessary props

* Move hookstate to global ContainerHookState

* Trace exception before we throw it for not losing information

* Fix for null ref exception in GetResponse

* Adding additional check for null response in prepareJob hook

* Refactoring GetResponse with additional check

* Update error messages

* Ports in ResponseContainer changed from IList to IDictionary

* Fix port format

* Include dockerfile

* Send null Registry obj if there's nothing in it

* Minor formatting

* Check if hookIndexPath exists relocated to the ContainerHookManager

* Code refactoring - ValidateHookExecutable added to the ContainerHookManager

* check if ContainerHooksPath when AllowRunnerContainerHooks is on

* Submit JSON telemetry instead of boolean

* Prefix step hooks with "run"

* Rename FeatureManager

* Fix flipped condition

* Unify js shell path getter with ps1 and sh getter

* Validate on run, not on instantiation of manager

* Cleanup ExecuteAsync methods

* Handle exception in executeHookScript

* Better exception types

* Remove comment

* Simplify boolean

* Allow jobs without jobContainer to run

* Use JObject instead of JToken

* Use correct Response type

* Format class to move cleanupJobAsync to the end of public methods

* Rename HookIndexPath to HookScriptPath

* Refactor methods into expression bodies

* Fix args class hierarchy

* Fix argument order

* Formatting

* Fix nullref and don't swallow stacktrace

* Whilelist HookArgs

* Use FF in FeatureManager

* Update src/Runner.Worker/ContainerOperationProvider.cs

Co-authored-by: Tingluo Huang <tingluohuang@github.com>

* Update src/Runner.Worker/ActionRunner.cs

Co-authored-by: Thomas Boop <52323235+thboop@users.noreply.github.com>

* Update src/Runner.Worker/ActionRunner.cs

Co-authored-by: Thomas Boop <52323235+thboop@users.noreply.github.com>

* Only mount well known dirs to job containers

* Get trace from hostcontext

* Use hook execution for setting telemetry

Co-authored-by: Nikola Jokic <nikola.jokic@akvelon.com>
Co-authored-by: Nikola Jokic <nikola-jokic@users.noreply.github.com>
Co-authored-by: Nikola Jokic <97525037+nikola-jokic@users.noreply.github.com>
Co-authored-by: Stefan Ruvceski <stefan.ruvceski@akvelon.com>
Co-authored-by: ruvceskistefan <96768603+ruvceskistefan@users.noreply.github.com>
Co-authored-by: Thomas Boop <thboop@github.com>
Co-authored-by: stefanruvceski <ruvceskistefan@github.com>
Co-authored-by: Tingluo Huang <tingluohuang@github.com>
Co-authored-by: Thomas Boop <52323235+thboop@users.noreply.github.com>
2022-06-10 13:51:20 +00:00
Tingluo Huang
28f0027938 Add SHA to useragent. (#1694) 2022-02-17 20:01:48 +00:00
Tingluo Huang
ac31fd10b2 Introduce GITHUB_ACTIONS_RUNNER_TLS_NO_VERIFY=1 to skip SSL cert verification for the runner. (#1616) 2022-01-19 10:31:17 -05:00
Meng Ye
a519f96a41 fix Log size and retention settings not work (#1507)
env
- RUNNER_LOGRETENTION
- WORKER_LOGRETENTION
- RUNNER _LOGSIZE
- WORKER _LOGSIZE
2021-12-02 10:04:39 -05:00
eric sciple
3e34fb10c1 improve telemetry to better diagnose runner configuration issues (#1487) 2021-11-15 13:42:57 -06:00
Thomas Boop
04269f7b1b Handle keeping previous OSX versions more smoothly on Mac (#1381)
* Handle macOS upgrade smoothly

* cleanup

* misc cleanup

* final updates

* Update src/Misc/layoutbin/update.sh.template

Co-authored-by: Patrick Ellis <319655+pje@users.noreply.github.com>

* Update src/Misc/layoutbin/update.sh.template

Co-authored-by: Patrick Ellis <319655+pje@users.noreply.github.com>

* Upload telemetry and default to old method as needed

* minor fix

* add one more bit of logging

* some more telemetry

* quote variables to handle spaces

* tiny fix for ubuntu

* remove version and move telemetry to diag

* use full path

Co-authored-by: Patrick Ellis <319655+pje@users.noreply.github.com>
2021-09-29 15:49:31 -04:00
Ferenc Hammerl
07c00f6a8a PowerShell secret masking (#1258)
* Trim pwsh special chars when masking secrets

* Add pwsh valueEncoder

* Explain regex

* Update ValueEncoders.cs

* Add tests for pwsh color codes in secrets

* Formatting

* Group tests into theories

* Split secret on PS chars and mask for them

* Clean up comments

* Remove unused unittest

* Rename escape methods
2021-08-25 23:07:19 +02:00
eric sciple
8109c962f0 mask secrets with double-quotes when passed to docker command line (#1002) 2021-03-05 15:17:55 -06:00
PJ Quirk
f798f5606b Use the API_URL and munge action URLs for GHES (#437)
* First pass at logic for GHES, not all correct

* Need to mock out file downloading

* Allowed for mocking of HTTP responses

* Added test for builtin GHES action download

* More tests

* Don't retry on action 404

* Remove commented out code

* Add a using statement back, because Windows

* Make windows happy again

* Another windows fix

* Always delete the cache since it isn't fully implemented

* Use RunnerService base class

* Add examples, update URL path

* Remove forceDotCom

* Fix a bug

* Remove a test that's no longer relevant

* PR feedback

* Add missing return

* More trace info

* Use the new agreed-upon format

* Use the auth token since we're hitting GHES directly

* Fixing tests on windows

* Fixed one more test
2020-04-23 17:02:13 -04:00
Tingluo Huang
3f7a01af93 add secret masker for trimming double qoutes. (#440) 2020-04-21 22:07:55 -04:00
Tingluo Huang
baa6ded3bc Better Kusto Tracing for self-hosted runner. (#405) 2020-04-09 14:33:16 -04:00
Tingluo Huang
f9b5d626c5 load and print machine setup info from .setup_info (#364) 2020-03-11 10:36:56 -04:00
Tingluo Huang
6bec1e3bb8 Switch to use token service instead of SPS for exchanging oauth token. (#325)
* Gracefully switch the runner to use Token Service instead of SPS.

* PR feedback.

* feedback2

* report error.
2020-03-04 21:40:58 -05:00
Tingluo Huang
745b90a8b2 Revert "Update Base64 Encoders to deal with suffixes (#284)" (#330)
This reverts commit c45aebc9ab.
2020-02-12 14:26:30 -05:00
Thomas Boop
c45aebc9ab Update Base64 Encoders to deal with suffixes (#284)
* Update Base64 Encoders to deal with suffixes

* Set UriDataEscape to public for unit tests
2020-01-27 21:38:31 -05:00
Julio Barba
50d979f1b2 Bring back tools folder fallback code (#232) 2019-12-17 18:21:13 -05:00
Tingluo Huang
91b7e7a07a delete more unused code. (#230)
* delete more unused code.

* pr feedback.
2019-12-17 16:47:14 -05:00
Tingluo Huang
d0a4a41a63 delete un-used code. (#218) 2019-12-16 17:05:26 -05:00
Tingluo Huang
d81a7656a4 Add Proxy Support for self-hosted runner. (#206) 2019-12-09 15:15:54 -05:00
Julio Barba
cd2cec8282 Another runner code cleanup round (#197)
* Remove remaining non-SDK references of capabilities/demands
* Remove unused Runner.Common constants
* Remove more variables
* Clean up RU link, and named-pipe support
* Remove NotificationSocketAddress
* Re-add legacy OnPremises JobDispatcher code (commented out)
* More misc cleanup
2019-12-04 10:18:37 -05:00
Thomas Boop
c3b11b36e9 Remove unneeded base64 encoders (#128)
* Remove unneeded base64 encoders

* update comment
2019-10-14 10:47:41 -04:00
Tingluo Huang
c8afc84840 GitHub Actions Runner 2019-10-10 00:52:42 -04:00