Commit Graph

85 Commits

Author SHA1 Message Date
Nikola Jokic
04761e5353 Initialize container manager based on whether the ContainerHooksPath is set (#2317)
* Added tests around checking if correct manager's Initialize method has been called

* repaired missing initialization on container action handler
2022-12-16 15:40:49 +01:00
Nikola Jokic
9122fe7e10 added to lowercase on setting github.action_status (#1944) 2022-11-22 15:05:32 +01:00
Tingluo Huang
4adaf9c1e6 Use Global.Variables instead of JobContext and include action path/ref in the message. (#2214)
* Use Global.Variables instead of JobContext and include action path/ref in the message.

* encoding

* .
2022-11-02 09:40:19 -04:00
Cory Miller
b87b4aac5c Fix IDE0090 (#2211) 2022-10-18 10:54:08 -04:00
Thomas Boop
ae2f4a6f27 POC: Windows arm64 runner build (#2022)
Prerelease for windows-arm64 runner build
2022-09-26 09:20:43 -04:00
Nikola Jokic
01fd04464d Escaping key and quoting it to avoid key based command injection (#2062)
* escaping key and quoting it to avoid key based command injection

* extracted creation of flags to DockerUtil, with testing included
2022-08-23 10:42:29 -04:00
ChristopherHX
7e74f8c9d5 fix: GITHUB_ENV in composite (#1794)
* fix: GITHUB_ENV in composite

* fix L0 Test
2022-06-28 20:50:50 -04:00
Stefan Ruvceski
d11bd3d8be Created env var for forcing node12 actions to run on node16 (#1913)
* Created env var for forcing node12 actions to run on node16

* get value of hostContext environment variable

* changing location of forced node version check

* small code refactoring

Co-authored-by: Ferenc Hammerl <31069338+fhammerl@users.noreply.github.com>

* more of small code refactoring

Co-authored-by: Ferenc Hammerl <31069338+fhammerl@users.noreply.github.com>

* refactoring of conditions for getting internal node version

* changing expected value for node version env var

* Adding empty line between two methods

* Created method GetNodeVersion

* GetNodeVersion from function to inline call and PR fixes

Co-authored-by: Ferenc Hammerl <31069338+fhammerl@users.noreply.github.com>
Co-authored-by: Thomas Boop <52323235+thboop@users.noreply.github.com>
2022-06-22 11:13:28 -04:00
Ferenc Hammerl
591f8c3510 Runner container hooks Beta (#1853)
* Added ability to run Dockerfile.SUFFIX ContainerAction

* Extracted IsDockerFile method

* reformatted, moved from index to Last()

* extracted IsDockerfile to DockerUtil with L0

* added check for IsDockerfile to account for docker://

* updated test to clearly show path/dockerfile:tag

* fail if Data.Image is not Dockerfile or docker://[image]

* Setup noops for JobPrepare and JobCleanup hooks

* Add container jobstarted and jobcomplete hooks

* Run 'index.js' instead of specific command hooks

* Call jobprepare with command arg

* Use right command name (hardcoded)

Co-authored-by: Nikola Jokic <nikola-jokic@users.noreply.github.com>

* Invoke hooks with arguments

* Add PrepareJob hook to work with jobcontainers

Co-authored-by: Nikola Jokic <nikola-jokic@users.noreply.github.com>

* Rename methods

* Use new hookcontainer to run prep and clean hooks

* Get path from ENV

* Use enums

* Use IOUtils.cs

* Move container files to folder

* Move namespaces

* Store "state" between hooks

* Remove stdin stream in containerstephosts

* Update Constants.cs

* Throw if stdin fails

* Cleanup obvious nullrefs and unused vars

* Cleanup containerhook directory

* Call step exec hook

* Fix windows build

* Remove hook from hookContainer

* Rename file

* More renamings

* Add TODOs

* Fix env name

* Fix missing imports

* Fix imports

* Run script step on jobcontainer

* Enable feature if env is set

* Update ContainerHookManager.cs

* Update ContainerHookManager.cs

* Hooks allowed to work even when context isn't returned

* Custom hooks enabled flag and additional null checks

* New line at the end of the FeatureFlagManager.cs

* Code refactoring

* Supported just in time container building or pulling

* Try mock-build for osx

* Build all platforms

* Run mock on self-hosted

* Remove GITHUB prefix

* Use ContainerHooksPath instead of CustomHooksPath

* Null checks simplified

* Code refactoring

* Changing condition for image builing/pulling

* Code refactoring

* TODO comment removed

Co-authored-by: Ferenc Hammerl <31069338+fhammerl@users.noreply.github.com>

* Call container step if FF is on

* Rename run script function

* Use JToken instead of dynamic

* Add TODO

* Small refactoring + renames + TODOs

* Throw on DetermineNodeRuntimeVersion

* Fix formatting

* Add run-container-step

* Supported nodeJS in Alpine containers

* Renamed Alpine to IsAlpine in HookResponse

* Method for checking platform for alpine container

* Added container hooks feature flag check

* Update IsHookFeatureEnabled with new params

* Rename featureflag method

* Finish rename

* Set collection null values to empty arrays when JSON serialising them

* Disable FF until we merge

* Update src/Runner.Worker/Container/ContainerHooks/HookContainer.cs

* Fix method name

* Change hookargs to superclass from interface

* Using only Path.Combine in GenerateResponsePath

* fix merge error

* EntryPointArgs changed to list of args instead of one args string

* Changed List to IEnumerable for EntryPointArgs and MountVolumes

* Get ContainerRuntimePath for JobContainers from hooks

* Read ContainerEnv from response file

* Port mappings saved after creating services

* Support case when responseFile doesn't exist

* Check if response file exists

* Logging in ExecuteHookScript

* Save hook state after all 4 hooks

* Code refactoring

* Remove TODO

Co-authored-by: Ferenc Hammerl <31069338+fhammerl@users.noreply.github.com>

* Remove second TODO

Co-authored-by: Ferenc Hammerl <31069338+fhammerl@users.noreply.github.com>

* Removing container env changes

* Removing containerEnv and dockerManager

* Delete mock-build.yml

* Update IOUtil.cs

* Add comment about containerhooks

* Fix merge mistake

* Remove solved todo

* Determine which shell to use for hooks scenario

* Overload for method ExecuteHookScript with prependPath as arg

* Adding HostContext to the GetDefaultShellForScript call

* prependPath as a mandatory parameter

* Improve logging for hooks

* Small changes in logging

* Allow null for ContainerEntryPointArgs

* Changed log messages

* Skip setting EntryPoint and EntryPointArgs if hooks are enabled

* Throw if IsAlpine is null in PrepareJob

* Code refactoring - added GetAndValidateResponse method

* Code refactoring

* Changes in exception message

* Only save hookState if returned

* Use FF from server

* Empty line

* Code refactoring

Co-authored-by: Ferenc Hammerl <31069338+fhammerl@users.noreply.github.com>

* Send null instead of string empty

* Remove TODO

* Code refactoring and some small changes

* Allow Globals to be null to pass L0

* Fix setup in StepHostL0

* Throw exception earlier if response file doesn't exist and prepare_job hook is running

* Refactoring GetResponse method

* Changing exception message if response file is not found

Co-authored-by: Ferenc Hammerl <31069338+fhammerl@users.noreply.github.com>

* Chaning exception message if isAlpine is null for prepare_job hook

* Rename hook folder

* Fail if compatible hookfile not found

* Use .Value instead of casting bool? to bool

* Format spacing

* Formatting

* User user and system mvs

* Use variables instead of entire context in featuremanager

* Update stepTelemetry if step uses containerhooks

* Restore  import

* Remove unneccessary field from HookContainer

* Refactor response context and portmappings

* Force allow hooks if FF is on

* Code refactoring

* Revert deleting usings

* Better hookContainer defaults and use correct portmapping list

* Make GetDefaultShellForScript a  HostContext extension method

* Generic hookresponse

* Code refactoring, unnecessary properties removed - HookContainer moved to the HookInput.cs

* Remove empty line

* Code refactoring and better exception handling

* code refactor, removing unnecessary props

* Move hookstate to global ContainerHookState

* Trace exception before we throw it for not losing information

* Fix for null ref exception in GetResponse

* Adding additional check for null response in prepareJob hook

* Refactoring GetResponse with additional check

* Update error messages

* Ports in ResponseContainer changed from IList to IDictionary

* Fix port format

* Include dockerfile

* Send null Registry obj if there's nothing in it

* Minor formatting

* Check if hookIndexPath exists relocated to the ContainerHookManager

* Code refactoring - ValidateHookExecutable added to the ContainerHookManager

* check if ContainerHooksPath when AllowRunnerContainerHooks is on

* Submit JSON telemetry instead of boolean

* Prefix step hooks with "run"

* Rename FeatureManager

* Fix flipped condition

* Unify js shell path getter with ps1 and sh getter

* Validate on run, not on instantiation of manager

* Cleanup ExecuteAsync methods

* Handle exception in executeHookScript

* Better exception types

* Remove comment

* Simplify boolean

* Allow jobs without jobContainer to run

* Use JObject instead of JToken

* Use correct Response type

* Format class to move cleanupJobAsync to the end of public methods

* Rename HookIndexPath to HookScriptPath

* Refactor methods into expression bodies

* Fix args class hierarchy

* Fix argument order

* Formatting

* Fix nullref and don't swallow stacktrace

* Whilelist HookArgs

* Use FF in FeatureManager

* Update src/Runner.Worker/ContainerOperationProvider.cs

Co-authored-by: Tingluo Huang <tingluohuang@github.com>

* Update src/Runner.Worker/ActionRunner.cs

Co-authored-by: Thomas Boop <52323235+thboop@users.noreply.github.com>

* Update src/Runner.Worker/ActionRunner.cs

Co-authored-by: Thomas Boop <52323235+thboop@users.noreply.github.com>

* Only mount well known dirs to job containers

* Get trace from hostcontext

* Use hook execution for setting telemetry

Co-authored-by: Nikola Jokic <nikola.jokic@akvelon.com>
Co-authored-by: Nikola Jokic <nikola-jokic@users.noreply.github.com>
Co-authored-by: Nikola Jokic <97525037+nikola-jokic@users.noreply.github.com>
Co-authored-by: Stefan Ruvceski <stefan.ruvceski@akvelon.com>
Co-authored-by: ruvceskistefan <96768603+ruvceskistefan@users.noreply.github.com>
Co-authored-by: Thomas Boop <thboop@github.com>
Co-authored-by: stefanruvceski <ruvceskistefan@github.com>
Co-authored-by: Tingluo Huang <tingluohuang@github.com>
Co-authored-by: Thomas Boop <52323235+thboop@users.noreply.github.com>
2022-06-10 13:51:20 +00:00
Ferenc Hammerl
0431b6fd40 Revert bash and shell -e filePath escape (#1932)
It generated invalid arguments for `Process()` when the `bash` command itself was an argument as well, for example:

```
            _proc.StartInfo.FileName = "/usr/bin/docker";
            _proc.StartInfo.Arguments = "exec -i --workdir /__w/container-hook-e2e/container-hook-e2e 47105c66144d8809d9fa2bce9a58ea0564cd14def0ae7952cd6231fba3576db1 sh -e '/__w/_temp/fd086560-cb92-4f3b-a99c-35a6b7b1bbdb.sh'";
```
2022-06-09 14:37:08 +02:00
Nikola Jokic
c3d5449146 Job hook provider now sets shell name for script handler (#1826)
* Job hook provider now sets shell name for script handler

* fixed script handler and job hook provider to work with the name without fail

* returned used import by osx

* fixed order of imports

* added quotes around resolved script path allowing space in script path

* added quotes around bash and sh _defaultArguments

* Changed double quotes to single quotes in sh -e

Co-authored-by: Ferenc Hammerl <31069338+fhammerl@users.noreply.github.com>

* Changed double quotes to single quotes in bash

Co-authored-by: Ferenc Hammerl <31069338+fhammerl@users.noreply.github.com>

Co-authored-by: Ferenc Hammerl <31069338+fhammerl@users.noreply.github.com>
2022-06-08 13:54:23 +02:00
Thomas Boop
5cca207314 Port the 291.1 hotfix to main (#1905)
* Revert "Added ability to run Dockerfile.SUFFIX ContainerAction (#1738)"

20b7e86e47

* port release notes
2022-05-22 16:24:18 -04:00
Tingluo Huang
0398f57125 Create runner layout for osx-arm64 (Apple M1) platform. (#1618)
* Create runner layout for osx-arm64 (Apple M1) platform.

* bypass m1 macos

* l0
2022-05-20 11:00:54 -04:00
Nikola Jokic
20b7e86e47 Added ability to run Dockerfile.SUFFIX ContainerAction (#1738)
* Added ability to run Dockerfile.SUFFIX ContainerAction

* Extracted IsDockerFile method

* reformatted, moved from index to Last()

* extracted IsDockerfile to DockerUtil with L0

* added check for IsDockerfile to account for docker://

* updated test to clearly show path/dockerfile:tag

* fail if Data.Image is not Dockerfile or docker://[image]
2022-04-27 21:23:12 -04:00
Tingluo Huang
44d4d076fe Capture telemetry when git errors on unsafe repository. (#1823) 2022-04-13 12:48:52 -04:00
Nikola Jokic
c8cb600ac7 Use StepHost when evaluating inputs to actions (#1762)
* composite action github.action_path set based on the StepHost

* in progress on updating github context for input template

* Fixed updating the context data for evaluation

* refactored logic so it is a little cleaner

* removed resolving the action_path in CompositeActionHandler

* removed added DeepClone

* added feature flag and modified the dict in place

* refactored step host to change context data. Added L0

* repaired spaces

* moved logic from step host to execution context, added recursive translation

* removed empty lines

* moved to extension methods

* Update src/Test/L0/Worker/StepHostL0.cs

Co-authored-by: Ferenc Hammerl <31069338+fhammerl@users.noreply.github.com>
2022-04-11 12:43:24 +00:00
Thomas Boop
f48f314a70 fix an issue where container hooks used the job default working directory (#1809) 2022-04-06 14:33:32 +02:00
Nikola Jokic
d70f9f6174 Continue on error for the composite actions (#1763)
* Added continue on error to composite action

* changed from boolean-strategy-context -> boolean-steps-context for action_yaml

* refactored composite handler to always set outcome

* retrigger checks

* fixed typo in ??= operator

* boolean-steps-context accepts the same context as string-steps-context

* setting the outcome only on continue-on-error

* moved continue on error logic to the execution context

* Added L0 table tests for continue-on-error ExecutionContext

* Added missing mocks on StepsRunnerL0 for this update

* removed empty line and added one line separating the call

* Removed empty line

Co-authored-by: Ferenc Hammerl <31069338+fhammerl@users.noreply.github.com>

Co-authored-by: Ferenc Hammerl <31069338+fhammerl@users.noreply.github.com>
2022-04-01 09:18:53 -04:00
Ferenc Hammerl
408d6c579c Add annotations if Node 12 action is found and FF is on (#1735)
* Add annotations if node 12 action is found

* Better placeholder

* Only warn if FF is on

* Move annotation logic

* Pass in the LTS Url

* Raise annotation right before executing the action

* Match server side FF name

* Change name back to features

* Better warning text

* Update src/Runner.Common/Constants.cs

Co-authored-by: Thomas Boop <52323235+thboop@users.noreply.github.com>

Co-authored-by: Thomas Boop <52323235+thboop@users.noreply.github.com>
2022-03-22 10:43:25 +01:00
Ferenc Hammerl
7d5e9cd70f Runner Job Started/Completed Hooks (#1737)
* Prototype for pre job hook

* Remove debug log

* Enable hooks again

* Initialize with hostContext

* Add event_path, fix no-path bug

* Allow script post steps

* Call script handler with correct pre post stage

* Add job completed hook

* Make filecommand work and hardcode shell

* Conditionally print step details and no telemetry for hooks

* Figure out whih script to use

* Only check path for managed scripts

* Resture win dependency

* Nits

* Remove unused, add named params

* Telemetry + refactoring

* add message to job

* rename hooks remove stale comment

* cleanup

* Use .CreateService to create step

* Add L0s

* pr feedback

* update tests

* add disclaimer, clean up code

* spacing fix

* little more cleanup

* pr fix

* pr feedback

* Refactor to use JobExtension

* fix tests

* fix typo

* cleanup code

* more cleanup

* little more cleanup

* last bit of cleanup

* fix tests

* nit fix

* Update src/Runner.Worker/JobHookProvider.cs

Co-authored-by: Edward Thomson <ethomson@github.com>

* don't override runner telemtry

* pr feedback

* pr feedback

* pr feedback

Co-authored-by: Thomas Boop <thboop@github.com>
Co-authored-by: Thomas Boop <52323235+thboop@users.noreply.github.com>
Co-authored-by: Edward Thomson <ethomson@github.com>
2022-03-17 21:35:04 -04:00
Ferenc Hammerl
100c99f263 Force JS Actions Node version to 16 if FF is on unless user opted out (#1716)
* Set GH actions Node version to 16 if FF is on unless user opted out

* Add L0s (WIP)

* Wrap tests into theory

* Only check for node12 actions

* Refactor node version picking
2022-02-25 14:59:16 -05:00
Ferenc Hammerl
e8ccafea63 Add internal to node version function and use better env var name (#1715) 2022-02-25 14:59:02 -05:00
ruvceskistefan
934027da60 Issue 1261: inconsistency of outputs (both canceled and cancelled are used) (#1624)
* Issue 1261: inconsistency of outputs

* Changing cancelled to canceled in one error message
2022-02-23 09:48:24 -05:00
Tingluo Huang
882f36dcf8 Sending telemetry about actions usage. (#1688)
* Sending telemetry about actions usage.

* .

* L0 tests.

* .
2022-02-16 12:18:21 -05:00
Ferenc Hammerl
bd77ccf34e Prefer node16 over node12 when running internal scripts (#1621)
* Use 16 to run RunnerService.js

* Execute hashfiles using node16

* Run downloadCert.js using node16

* Run makeWebRequest.js using node16

* Run macos-run-invoker.js using node16

* Run hashFiles.js using node16

* Update tests to use node16

* Update documentation to recommend node16

* Duplicate macos service js fix for 16

* Add PR link

* Revert ADR node change

* Merge node12/16 retainment IFs

* Try both node12 and node16

* Close if

* Revert "Update tests to use node16"

This reverts commit bbca7b9f1c.

* Fix condition

* Unfurl if condition

* Allow user to force a node version

* Format update template

* Comment env var

* Rename vars

* Fix naming

* Fix rename

* Set node ver override if job message has it

* Format executionContext

* Can only receive 'forceNode12' or nothing from FF
No specific node version from server

Co-authored-by: Ferenc Hammerl <hammerl.ferenc@gmail.com>
2022-02-14 15:06:08 +01:00
Tingluo Huang
cb19da9638 Move JobTelemetry and StepsTelemetry into GlobalContext. (#1680)
* Move JobTelemetry and StepsTelemetry into GlobalContext.

* .

* .
2022-02-11 16:18:41 -05:00
Thomas Boop
47dfebdf48 Set outcome/conclusion for composite action steps in steps context (#1600)
* set outcome/conclusion for composite action steps in steps context

* fix typo

* pr fixes

* fix happy case!
2022-01-12 11:14:46 -05:00
Tingluo Huang
e3e977fd84 Support node.js 16 and bump node.js 12 version. (#1439)
* Support node.js 16 and bump node.js 12 version.

* L0
2021-11-18 15:25:33 -05:00
Ferenc Hammerl
67ba8a7d42 Support Conditional Steps in Composite Actions (#1438)
* conditional support for composite actions

* Fix Conditional function evaluation

* Push launch.json temporarily

* Revert "Push launch.json temporarily"

* rename context

* Cleanup comments

* fix success/failure functions to run based on pre/main steps

* idea of step_status

* change to use steps context, WIP

* add inputs to possible if condition expressions

* use action_status

* pr cleanup

* Added right stages

* Test on stage in conditional functions

* Fix naming and formatting

* Fix tests

* Add success and failure L0s

* Remove comment

* Remove whitespace

* Undo formatting

* Add L0 for step-if parsing

* Add ADR

Co-authored-by: Thomas Boop <thboop@github.com>
2021-10-29 15:45:42 +02:00
Thomas Boop
854d5e3bf3 Fix an issue where nested local composite actions did not correctly register post steps (#1433)
* Always register post steps for local actions

* Register post steps along with their conditions

* remove debug code

Co-authored-by: Ferenc Hammerl <fhammerl@github.com>
2021-10-27 15:31:58 +02:00
Thomas Boop
9a829995e0 Show More Step Information in composite Actions (#1279) 2021-09-01 16:04:27 -04:00
Tingluo Huang
62d568674c Add ACTIONS_ID_TOKEN_REQUEST_URL/Token to script as well. (#1287) 2021-08-26 13:29:02 -04:00
Tingluo Huang
05b84297b7 Add extra env for the Token log-in action is going to use to request ID_TOKEN. (#1270) 2021-08-23 14:50:35 -04:00
Thomas Boop
d2ca24fa43 For Main Steps, just run the step, don't check condition (#1273)
* For Main Steps, just run the step, don't check condition

* fix whitespace

* pr feedback
2021-08-18 16:40:25 -04:00
Thomas Boop
22a9d89772 Correctly set post step step context (#1243) 2021-08-04 11:39:22 -04:00
Thomas Boop
3851acd0cf fix continue on error (#1238) 2021-08-03 17:44:58 -04:00
Tingluo Huang
110eb3a5de Add generateIdTokenUrl to env vars for actions. (#1234) 2021-08-02 14:47:50 -07:00
Thomas Boop
85ce33b1d3 Composite Code Cleanup (#1232)
* composite polish

* Cleanup Condition Handling

* Refactor ConditionTraceWriter

* pr feedback

* cleanup
2021-08-02 14:57:25 -04:00
Thomas Boop
92ec3d0f29 Add better step telemetry and tracing for composite Actions (#1229)
* Add Step Telemetry

* better telemetry and tracing

* cleanup
2021-07-30 10:45:49 -04:00
Thomas Boop
4e95d0d6ad Support pre/post/container/composite actions within composite actions (#1222)
Support Composite Actions with uses: steps
2021-07-28 15:35:21 -04:00
Luke Tomlinson
fff31e11c5 Add notice annotation level and support more annotation fields (#1175)
* Add Notice Command

* Add Feature Flag For Enhanced Annotations

* Fix tests

* Add validation for columns and lines

* Fix order to match service

* Remove console.write

* Make Validation Better

* Cleanup

* Handle empty/whitespace strings

* Add more validation for line/column ranges

* Make Validation Debug, Not Throw

* Change casing to 🐫 from 🐍

* Give notice a well known tag

* Cleanup

* Sanitize invalid commands rather than fail
2021-07-13 11:38:16 -04:00
Thomas Boop
7c90b2a929 Composite Actions (#1170)
Composite Actions Refactoring
2021-07-01 13:34:28 -04:00
Rob Bos
ebadce7958 Typo: Changed dockerManger to dockerManager (#1133) 2021-06-04 10:51:30 -04:00
eric sciple
419ed24c1e rename nestedSteps to embeddedSteps (#1071) 2021-05-01 12:51:01 -05:00
eric sciple
7cc689b0d9 minor cleanup in composite (#1045) 2021-04-30 15:48:53 -05:00
Tingluo Huang
5941cceb7c Unset NODE_ICU_DATA before starting node in NodeHandler. (#1060)
* Unset NODE_ICU_DATA before starting node in NodeHandler.

* Update src/Runner.Worker/Handlers/NodeScriptActionHandler.cs

Co-authored-by: Ross Brodbeck <hross@users.noreply.github.com>

Co-authored-by: Ross Brodbeck <hross@users.noreply.github.com>
2021-04-23 11:10:22 -04:00
Thomas Boop
67ff8d3460 Release 2.277.1 runner (#977)
* Revert "Enable tty output from Docker Actions (#916)"

5972bd0060

* Release notes

* add pr
2021-02-09 14:45:33 -05:00
Joel Dickson
5972bd0060 Enable tty output from Docker Actions (#916)
* Update DockerCommandManager.cs

* Update StepHost.cs

Co-authored-by: Tingluo Huang <tingluohuang@github.com>
2021-01-21 22:35:57 -05:00
klassiker
007ac8138b Add proxy support for container actions (#840)
* Add proxy support for container actions in Runner.Worker/StepsRunner

* Move proxy modifications to ContainerActionHandler
2020-12-11 13:08:45 -05:00
eric sciple
3a76a2e291 read env file (#683) 2020-08-29 23:18:35 -04:00