Commit Graph

676 Commits

Author SHA1 Message Date
Lokesh Gopu
813af29886 Include current runner status while getting messages (#2026)
* get messages with runner status

* fixed l0 tests

* PR feedback
2022-07-28 16:42:02 -04:00
Tingluo Huang
72e2107b5e Change the auth challenge 401 to be verbose trace. (#2021) 2022-07-25 16:02:54 -04:00
Tingluo Huang
3567c042ea Bump newtonsoft.json to 13.0.1 (#2012) 2022-07-22 11:21:04 -04:00
Tingluo Huang
e646b6fec4 Move --jitconfig to valid ./run.sh args. (#2013) 2022-07-22 11:14:09 -04:00
aaros-pl
8d2be3d4fa missing .com in pipelines.actions url (#1973) 2022-07-21 23:12:36 -04:00
eric sciple
407a347f83 tweak verbiage (#1977) 2022-06-28 21:03:53 -04:00
ChristopherHX
7e74f8c9d5 fix: GITHUB_ENV in composite (#1794)
* fix: GITHUB_ENV in composite

* fix L0 Test
2022-06-28 20:50:50 -04:00
eric sciple
efdda93aeb Update git troubleshooting (#1971) 2022-06-24 12:12:07 -05:00
eric sciple
1d1998aabb Update description for command line arg "--pat" (#1970) 2022-06-24 11:44:16 -04:00
Thomas Boop
d2c6a4e4bc 294.0 release notes (#1963)
* 293.1 release notes

* let make it a minor version bump
2022-06-22 11:57:10 -04:00
Stefan Ruvceski
d11bd3d8be Created env var for forcing node12 actions to run on node16 (#1913)
* Created env var for forcing node12 actions to run on node16

* get value of hostContext environment variable

* changing location of forced node version check

* small code refactoring

Co-authored-by: Ferenc Hammerl <31069338+fhammerl@users.noreply.github.com>

* more of small code refactoring

Co-authored-by: Ferenc Hammerl <31069338+fhammerl@users.noreply.github.com>

* refactoring of conditions for getting internal node version

* changing expected value for node version env var

* Adding empty line between two methods

* Created method GetNodeVersion

* GetNodeVersion from function to inline call and PR fixes

Co-authored-by: Ferenc Hammerl <31069338+fhammerl@users.noreply.github.com>
Co-authored-by: Thomas Boop <52323235+thboop@users.noreply.github.com>
2022-06-22 11:13:28 -04:00
Tingluo Huang
761785620f Support pass runner JitConfig as arg. (#1925) 2022-06-22 10:59:29 -04:00
Ferenc Hammerl
416771d4b1 Fix PrependPath format to be array instead of a concatenated string (#1948)
* Fix prependPath format
2022-06-22 09:11:17 -04:00
Tatyana Kostromskaya
9499f477a2 Add retry logic around getting job messages from broker (#1939)
* Jsut simple solution without additional funcs

* Delete old comment

* resolve

* Refactor retry function, make it more common

* Make retry function generic, get rid of extra params

* delete extra using

* Add cancellation token and limit of attempts

* Add some additional logging

* Rework condition

* replace to do..while

* return `while (true)` to simplify code structure

* Add other cancelling token, add TODO comment
2022-06-21 16:12:07 +02:00
Ferenc Hammerl
6bc6d475f9 No longer trace 'ex' twice, only further up in the callstack (#1949) 2022-06-16 16:36:54 -04:00
Tuukka Lahti
ca2b1bc6d5 Update dependencies list to support Ubuntu 22.04 (#1946) 2022-06-15 12:37:02 -04:00
Ferenc Hammerl
591f8c3510 Runner container hooks Beta (#1853)
* Added ability to run Dockerfile.SUFFIX ContainerAction

* Extracted IsDockerFile method

* reformatted, moved from index to Last()

* extracted IsDockerfile to DockerUtil with L0

* added check for IsDockerfile to account for docker://

* updated test to clearly show path/dockerfile:tag

* fail if Data.Image is not Dockerfile or docker://[image]

* Setup noops for JobPrepare and JobCleanup hooks

* Add container jobstarted and jobcomplete hooks

* Run 'index.js' instead of specific command hooks

* Call jobprepare with command arg

* Use right command name (hardcoded)

Co-authored-by: Nikola Jokic <nikola-jokic@users.noreply.github.com>

* Invoke hooks with arguments

* Add PrepareJob hook to work with jobcontainers

Co-authored-by: Nikola Jokic <nikola-jokic@users.noreply.github.com>

* Rename methods

* Use new hookcontainer to run prep and clean hooks

* Get path from ENV

* Use enums

* Use IOUtils.cs

* Move container files to folder

* Move namespaces

* Store "state" between hooks

* Remove stdin stream in containerstephosts

* Update Constants.cs

* Throw if stdin fails

* Cleanup obvious nullrefs and unused vars

* Cleanup containerhook directory

* Call step exec hook

* Fix windows build

* Remove hook from hookContainer

* Rename file

* More renamings

* Add TODOs

* Fix env name

* Fix missing imports

* Fix imports

* Run script step on jobcontainer

* Enable feature if env is set

* Update ContainerHookManager.cs

* Update ContainerHookManager.cs

* Hooks allowed to work even when context isn't returned

* Custom hooks enabled flag and additional null checks

* New line at the end of the FeatureFlagManager.cs

* Code refactoring

* Supported just in time container building or pulling

* Try mock-build for osx

* Build all platforms

* Run mock on self-hosted

* Remove GITHUB prefix

* Use ContainerHooksPath instead of CustomHooksPath

* Null checks simplified

* Code refactoring

* Changing condition for image builing/pulling

* Code refactoring

* TODO comment removed

Co-authored-by: Ferenc Hammerl <31069338+fhammerl@users.noreply.github.com>

* Call container step if FF is on

* Rename run script function

* Use JToken instead of dynamic

* Add TODO

* Small refactoring + renames + TODOs

* Throw on DetermineNodeRuntimeVersion

* Fix formatting

* Add run-container-step

* Supported nodeJS in Alpine containers

* Renamed Alpine to IsAlpine in HookResponse

* Method for checking platform for alpine container

* Added container hooks feature flag check

* Update IsHookFeatureEnabled with new params

* Rename featureflag method

* Finish rename

* Set collection null values to empty arrays when JSON serialising them

* Disable FF until we merge

* Update src/Runner.Worker/Container/ContainerHooks/HookContainer.cs

* Fix method name

* Change hookargs to superclass from interface

* Using only Path.Combine in GenerateResponsePath

* fix merge error

* EntryPointArgs changed to list of args instead of one args string

* Changed List to IEnumerable for EntryPointArgs and MountVolumes

* Get ContainerRuntimePath for JobContainers from hooks

* Read ContainerEnv from response file

* Port mappings saved after creating services

* Support case when responseFile doesn't exist

* Check if response file exists

* Logging in ExecuteHookScript

* Save hook state after all 4 hooks

* Code refactoring

* Remove TODO

Co-authored-by: Ferenc Hammerl <31069338+fhammerl@users.noreply.github.com>

* Remove second TODO

Co-authored-by: Ferenc Hammerl <31069338+fhammerl@users.noreply.github.com>

* Removing container env changes

* Removing containerEnv and dockerManager

* Delete mock-build.yml

* Update IOUtil.cs

* Add comment about containerhooks

* Fix merge mistake

* Remove solved todo

* Determine which shell to use for hooks scenario

* Overload for method ExecuteHookScript with prependPath as arg

* Adding HostContext to the GetDefaultShellForScript call

* prependPath as a mandatory parameter

* Improve logging for hooks

* Small changes in logging

* Allow null for ContainerEntryPointArgs

* Changed log messages

* Skip setting EntryPoint and EntryPointArgs if hooks are enabled

* Throw if IsAlpine is null in PrepareJob

* Code refactoring - added GetAndValidateResponse method

* Code refactoring

* Changes in exception message

* Only save hookState if returned

* Use FF from server

* Empty line

* Code refactoring

Co-authored-by: Ferenc Hammerl <31069338+fhammerl@users.noreply.github.com>

* Send null instead of string empty

* Remove TODO

* Code refactoring and some small changes

* Allow Globals to be null to pass L0

* Fix setup in StepHostL0

* Throw exception earlier if response file doesn't exist and prepare_job hook is running

* Refactoring GetResponse method

* Changing exception message if response file is not found

Co-authored-by: Ferenc Hammerl <31069338+fhammerl@users.noreply.github.com>

* Chaning exception message if isAlpine is null for prepare_job hook

* Rename hook folder

* Fail if compatible hookfile not found

* Use .Value instead of casting bool? to bool

* Format spacing

* Formatting

* User user and system mvs

* Use variables instead of entire context in featuremanager

* Update stepTelemetry if step uses containerhooks

* Restore  import

* Remove unneccessary field from HookContainer

* Refactor response context and portmappings

* Force allow hooks if FF is on

* Code refactoring

* Revert deleting usings

* Better hookContainer defaults and use correct portmapping list

* Make GetDefaultShellForScript a  HostContext extension method

* Generic hookresponse

* Code refactoring, unnecessary properties removed - HookContainer moved to the HookInput.cs

* Remove empty line

* Code refactoring and better exception handling

* code refactor, removing unnecessary props

* Move hookstate to global ContainerHookState

* Trace exception before we throw it for not losing information

* Fix for null ref exception in GetResponse

* Adding additional check for null response in prepareJob hook

* Refactoring GetResponse with additional check

* Update error messages

* Ports in ResponseContainer changed from IList to IDictionary

* Fix port format

* Include dockerfile

* Send null Registry obj if there's nothing in it

* Minor formatting

* Check if hookIndexPath exists relocated to the ContainerHookManager

* Code refactoring - ValidateHookExecutable added to the ContainerHookManager

* check if ContainerHooksPath when AllowRunnerContainerHooks is on

* Submit JSON telemetry instead of boolean

* Prefix step hooks with "run"

* Rename FeatureManager

* Fix flipped condition

* Unify js shell path getter with ps1 and sh getter

* Validate on run, not on instantiation of manager

* Cleanup ExecuteAsync methods

* Handle exception in executeHookScript

* Better exception types

* Remove comment

* Simplify boolean

* Allow jobs without jobContainer to run

* Use JObject instead of JToken

* Use correct Response type

* Format class to move cleanupJobAsync to the end of public methods

* Rename HookIndexPath to HookScriptPath

* Refactor methods into expression bodies

* Fix args class hierarchy

* Fix argument order

* Formatting

* Fix nullref and don't swallow stacktrace

* Whilelist HookArgs

* Use FF in FeatureManager

* Update src/Runner.Worker/ContainerOperationProvider.cs

Co-authored-by: Tingluo Huang <tingluohuang@github.com>

* Update src/Runner.Worker/ActionRunner.cs

Co-authored-by: Thomas Boop <52323235+thboop@users.noreply.github.com>

* Update src/Runner.Worker/ActionRunner.cs

Co-authored-by: Thomas Boop <52323235+thboop@users.noreply.github.com>

* Only mount well known dirs to job containers

* Get trace from hostcontext

* Use hook execution for setting telemetry

Co-authored-by: Nikola Jokic <nikola.jokic@akvelon.com>
Co-authored-by: Nikola Jokic <nikola-jokic@users.noreply.github.com>
Co-authored-by: Nikola Jokic <97525037+nikola-jokic@users.noreply.github.com>
Co-authored-by: Stefan Ruvceski <stefan.ruvceski@akvelon.com>
Co-authored-by: ruvceskistefan <96768603+ruvceskistefan@users.noreply.github.com>
Co-authored-by: Thomas Boop <thboop@github.com>
Co-authored-by: stefanruvceski <ruvceskistefan@github.com>
Co-authored-by: Tingluo Huang <tingluohuang@github.com>
Co-authored-by: Thomas Boop <52323235+thboop@users.noreply.github.com>
2022-06-10 13:51:20 +00:00
Ferenc Hammerl
ac7b34a071 Release 2.293.0 (#1940)
* Update releaseNote.md

* 2.293.0 Release

* Update releaseNote.md

Co-authored-by: Thomas Boop <52323235+thboop@users.noreply.github.com>

Co-authored-by: Thomas Boop <52323235+thboop@users.noreply.github.com>
2022-06-10 15:42:51 +02:00
Thomas Boop
0d1e6fd57b Add ADR for Container Hooks (#1891)
* Add ADR for Container Hooks

* Rename 0000-container-hooks.md to 1891-container-hooks.md

* Update 1891-container-hooks.md

* Update docs/adrs/1891-container-hooks.md

Co-authored-by: Ferenc Hammerl <31069338+fhammerl@users.noreply.github.com>

* Update docs/adrs/1891-container-hooks.md

Co-authored-by: Ferenc Hammerl <31069338+fhammerl@users.noreply.github.com>

* Update docs/adrs/1891-container-hooks.md

Co-authored-by: Ferenc Hammerl <31069338+fhammerl@users.noreply.github.com>

* Update docs/adrs/1891-container-hooks.md

Co-authored-by: Ferenc Hammerl <31069338+fhammerl@users.noreply.github.com>

* Update docs/adrs/1891-container-hooks.md

Co-authored-by: Ferenc Hammerl <31069338+fhammerl@users.noreply.github.com>

* Update docs/adrs/1891-container-hooks.md

Co-authored-by: Ferenc Hammerl <31069338+fhammerl@users.noreply.github.com>

Co-authored-by: Ferenc Hammerl <31069338+fhammerl@users.noreply.github.com>
2022-06-10 09:35:30 -04:00
Ferenc Hammerl
9623a44c2f Allow admins to fail jobs without container (#1895)
* Allow admins to fail jobs without container

* Make method static

* Update src/Runner.Common/Constants.cs

Co-authored-by: Thomas Boop <52323235+thboop@users.noreply.github.com>

* Update src/Runner.Worker/JobExtension.cs

Co-authored-by: Thomas Boop <52323235+thboop@users.noreply.github.com>

* Update src/Runner.Worker/JobExtension.cs

Co-authored-by: Thomas Boop <52323235+thboop@users.noreply.github.com>

* Rename env

* Add test for throwing when no container but required

* Update src/Runner.Worker/JobExtension.cs

* Update src/Test/L0/Worker/JobExtensionL0.cs

* Update src/Runner.Common/Constants.cs

Co-authored-by: Thomas Boop <52323235+thboop@users.noreply.github.com>

Co-authored-by: Thomas Boop <52323235+thboop@users.noreply.github.com>
2022-06-09 17:17:11 -04:00
Rob Herley
b2e2aa68c8 remove job summary feature flag (#1936) 2022-06-09 15:49:04 -04:00
eric sciple
a9ce6b92c4 Allow redirect get message call to broker (#1935) 2022-06-09 18:36:55 +00:00
eric sciple
a1bf8401d7 Handle message from broker (#1934) 2022-06-09 14:07:44 -04:00
eric sciple
a7152f1370 server wrapper for pulling full job message (#1933) 2022-06-09 17:50:52 +00:00
eric sciple
af285115e7 http client updates for broker flow (#1931) 2022-06-09 12:46:08 -04:00
Ferenc Hammerl
0431b6fd40 Revert bash and shell -e filePath escape (#1932)
It generated invalid arguments for `Process()` when the `bash` command itself was an argument as well, for example:

```
            _proc.StartInfo.FileName = "/usr/bin/docker";
            _proc.StartInfo.Arguments = "exec -i --workdir /__w/container-hook-e2e/container-hook-e2e 47105c66144d8809d9fa2bce9a58ea0564cd14def0ae7952cd6231fba3576db1 sh -e '/__w/_temp/fd086560-cb92-4f3b-a99c-35a6b7b1bbdb.sh'";
```
2022-06-09 14:37:08 +02:00
Nikola Jokic
c3d5449146 Job hook provider now sets shell name for script handler (#1826)
* Job hook provider now sets shell name for script handler

* fixed script handler and job hook provider to work with the name without fail

* returned used import by osx

* fixed order of imports

* added quotes around resolved script path allowing space in script path

* added quotes around bash and sh _defaultArguments

* Changed double quotes to single quotes in sh -e

Co-authored-by: Ferenc Hammerl <31069338+fhammerl@users.noreply.github.com>

* Changed double quotes to single quotes in bash

Co-authored-by: Ferenc Hammerl <31069338+fhammerl@users.noreply.github.com>

Co-authored-by: Ferenc Hammerl <31069338+fhammerl@users.noreply.github.com>
2022-06-08 13:54:23 +02:00
Tingluo Huang
9c5300b5b2 Handle HostedRunnerShutdownMessage from service to shutdown hosted runner faster. (#1922) 2022-06-02 13:14:50 -04:00
ruvceskistefan
183b1f387c targetArchitecture removed from launch.json after macos arm64 release (#1908) 2022-05-23 21:46:54 -04:00
Thomas Boop
42ad85741e 292 release (#1906) 2022-05-23 09:53:08 -04:00
Ferenc Hammerl
88ee16fb02 Save original, pre-parsed string from workflow input for the user's custom volume mounts (#1889)
* Save pre parsed string from workflow input for volume mounts that have one

* Use property

* Use named params
2022-05-23 12:07:38 +02:00
Thomas Boop
5cca207314 Port the 291.1 hotfix to main (#1905)
* Revert "Added ability to run Dockerfile.SUFFIX ContainerAction (#1738)"

20b7e86e47

* port release notes
2022-05-22 16:24:18 -04:00
Tingluo Huang
0b73794267 Set timeout on sending live console log. (#1903) 2022-05-20 21:31:21 -04:00
Tingluo Huang
d7694774a4 Update release note, workflow, doc for osx-arm64. (#1904) 2022-05-20 12:28:52 -04:00
Tingluo Huang
0398f57125 Create runner layout for osx-arm64 (Apple M1) platform. (#1618)
* Create runner layout for osx-arm64 (Apple M1) platform.

* bypass m1 macos

* l0
2022-05-20 11:00:54 -04:00
Tingluo Huang
fade0f46e7 Bump dotnet SDK to 6.0.300 (#1900)
* Bump dotnet SDK to 6.0.300
2022-05-17 22:51:32 -04:00
Thomas Boop
02b52e8497 ADR: Runner Job Started/Completed Hooks (#1751)
* RunnerHookADR

* Rename 0000-runner-job-hooks.md to 1751-runner-job-hooks.md

* Update docs/adrs/1751-runner-job-hooks.md

Co-authored-by: Edward Thomson <ethomson@github.com>

* Update docs/adrs/1751-runner-job-hooks.md

Co-authored-by: Edward Thomson <ethomson@github.com>

* update step names

Co-authored-by: Edward Thomson <ethomson@github.com>
2022-05-12 15:18:11 -04:00
Ferenc Hammerl
628f462ab7 Use header of redirect instead of parsing content (#1874)
* Use header of redirect instead of parsing content

* Add exception so we don't hit 404s later

* Fix typo

* Update SelfUpdaterL0.cs
2022-05-09 14:04:18 +02:00
Ferenc Hammerl
7ba4f8587e 2.291.0 Release Notes (#1854)
* Update releaseNote.md

* Update runnerversion

* Update releaseNote.md

Co-authored-by: Thomas Boop <52323235+thboop@users.noreply.github.com>

Co-authored-by: Thomas Boop <52323235+thboop@users.noreply.github.com>
2022-04-29 12:53:36 +02:00
ruvceskistefan
88f7c56757 Issue 1528: use OS specific path separator (#1617)
* Issue 1528: use OS specific path separator

* Using Path.Combine instead of OS specific c_defaultPathSeparator
2022-04-27 22:16:03 -04:00
Nikola Jokic
20b7e86e47 Added ability to run Dockerfile.SUFFIX ContainerAction (#1738)
* Added ability to run Dockerfile.SUFFIX ContainerAction

* Extracted IsDockerFile method

* reformatted, moved from index to Last()

* extracted IsDockerfile to DockerUtil with L0

* added check for IsDockerfile to account for docker://

* updated test to clearly show path/dockerfile:tag

* fail if Data.Image is not Dockerfile or docker://[image]
2022-04-27 21:23:12 -04:00
Tingluo Huang
bd5f275830 Update runnerversion to match latest release. 2022-04-26 09:54:42 -04:00
Yang Cao
a7aadf5615 Update Actions Summary limit to 1MiB (#1839)
* Update Actions Summary limit to 1MiB

* Making limit a public const so other part of the codebase is aware of the limit too
2022-04-20 17:08:50 -04:00
Tingluo Huang
1c582abc8b Skip running L0 tests in release workflow to prevent package pollution (#1832) 2022-04-19 16:10:47 -04:00
Tingluo Huang
44d4d076fe Capture telemetry when git errors on unsafe repository. (#1823) 2022-04-13 12:48:52 -04:00
Ferenc Hammerl
b6195624ac 2.290.0 Release notes (#1820)
* 2.290.0 rel notes

* Update releaseNote.md
2022-04-12 10:34:15 -04:00
ruvceskistefan
ead3509d5a Added warning in case of invalid combination of command and flags and/or arguments (#1781)
* Added warning in case of invalid combination of command and flags and/or arguments

* Deleting unnecessary comments

* Added separate list for generic options

* Added PAT to the valid remove options

* Added command name to the error message
2022-04-11 09:23:58 -04:00
Nikola Jokic
fee24199cb Added input context to shell in composite run-step (#1767)
* Added input context to shell in composite run-step

* moved from string-shell-context to string-steps-context
2022-04-11 14:50:45 +02:00
Nikola Jokic
c8cb600ac7 Use StepHost when evaluating inputs to actions (#1762)
* composite action github.action_path set based on the StepHost

* in progress on updating github context for input template

* Fixed updating the context data for evaluation

* refactored logic so it is a little cleaner

* removed resolving the action_path in CompositeActionHandler

* removed added DeepClone

* added feature flag and modified the dict in place

* refactored step host to change context data. Added L0

* repaired spaces

* moved logic from step host to execution context, added recursive translation

* removed empty lines

* moved to extension methods

* Update src/Test/L0/Worker/StepHostL0.cs

Co-authored-by: Ferenc Hammerl <31069338+fhammerl@users.noreply.github.com>
2022-04-11 12:43:24 +00:00
Thomas Boop
f48f314a70 fix an issue where container hooks used the job default working directory (#1809) 2022-04-06 14:33:32 +02:00