* Added ability to run Dockerfile.SUFFIX ContainerAction
* Extracted IsDockerFile method
* reformatted, moved from index to Last()
* extracted IsDockerfile to DockerUtil with L0
* added check for IsDockerfile to account for docker://
* updated test to clearly show path/dockerfile:tag
* fail if Data.Image is not Dockerfile or docker://[image]
* Setup noops for JobPrepare and JobCleanup hooks
* Add container jobstarted and jobcomplete hooks
* Run 'index.js' instead of specific command hooks
* Call jobprepare with command arg
* Use right command name (hardcoded)
Co-authored-by: Nikola Jokic <nikola-jokic@users.noreply.github.com>
* Invoke hooks with arguments
* Add PrepareJob hook to work with jobcontainers
Co-authored-by: Nikola Jokic <nikola-jokic@users.noreply.github.com>
* Rename methods
* Use new hookcontainer to run prep and clean hooks
* Get path from ENV
* Use enums
* Use IOUtils.cs
* Move container files to folder
* Move namespaces
* Store "state" between hooks
* Remove stdin stream in containerstephosts
* Update Constants.cs
* Throw if stdin fails
* Cleanup obvious nullrefs and unused vars
* Cleanup containerhook directory
* Call step exec hook
* Fix windows build
* Remove hook from hookContainer
* Rename file
* More renamings
* Add TODOs
* Fix env name
* Fix missing imports
* Fix imports
* Run script step on jobcontainer
* Enable feature if env is set
* Update ContainerHookManager.cs
* Update ContainerHookManager.cs
* Hooks allowed to work even when context isn't returned
* Custom hooks enabled flag and additional null checks
* New line at the end of the FeatureFlagManager.cs
* Code refactoring
* Supported just in time container building or pulling
* Try mock-build for osx
* Build all platforms
* Run mock on self-hosted
* Remove GITHUB prefix
* Use ContainerHooksPath instead of CustomHooksPath
* Null checks simplified
* Code refactoring
* Changing condition for image builing/pulling
* Code refactoring
* TODO comment removed
Co-authored-by: Ferenc Hammerl <31069338+fhammerl@users.noreply.github.com>
* Call container step if FF is on
* Rename run script function
* Use JToken instead of dynamic
* Add TODO
* Small refactoring + renames + TODOs
* Throw on DetermineNodeRuntimeVersion
* Fix formatting
* Add run-container-step
* Supported nodeJS in Alpine containers
* Renamed Alpine to IsAlpine in HookResponse
* Method for checking platform for alpine container
* Added container hooks feature flag check
* Update IsHookFeatureEnabled with new params
* Rename featureflag method
* Finish rename
* Set collection null values to empty arrays when JSON serialising them
* Disable FF until we merge
* Update src/Runner.Worker/Container/ContainerHooks/HookContainer.cs
* Fix method name
* Change hookargs to superclass from interface
* Using only Path.Combine in GenerateResponsePath
* fix merge error
* EntryPointArgs changed to list of args instead of one args string
* Changed List to IEnumerable for EntryPointArgs and MountVolumes
* Get ContainerRuntimePath for JobContainers from hooks
* Read ContainerEnv from response file
* Port mappings saved after creating services
* Support case when responseFile doesn't exist
* Check if response file exists
* Logging in ExecuteHookScript
* Save hook state after all 4 hooks
* Code refactoring
* Remove TODO
Co-authored-by: Ferenc Hammerl <31069338+fhammerl@users.noreply.github.com>
* Remove second TODO
Co-authored-by: Ferenc Hammerl <31069338+fhammerl@users.noreply.github.com>
* Removing container env changes
* Removing containerEnv and dockerManager
* Delete mock-build.yml
* Update IOUtil.cs
* Add comment about containerhooks
* Fix merge mistake
* Remove solved todo
* Determine which shell to use for hooks scenario
* Overload for method ExecuteHookScript with prependPath as arg
* Adding HostContext to the GetDefaultShellForScript call
* prependPath as a mandatory parameter
* Improve logging for hooks
* Small changes in logging
* Allow null for ContainerEntryPointArgs
* Changed log messages
* Skip setting EntryPoint and EntryPointArgs if hooks are enabled
* Throw if IsAlpine is null in PrepareJob
* Code refactoring - added GetAndValidateResponse method
* Code refactoring
* Changes in exception message
* Only save hookState if returned
* Use FF from server
* Empty line
* Code refactoring
Co-authored-by: Ferenc Hammerl <31069338+fhammerl@users.noreply.github.com>
* Send null instead of string empty
* Remove TODO
* Code refactoring and some small changes
* Allow Globals to be null to pass L0
* Fix setup in StepHostL0
* Throw exception earlier if response file doesn't exist and prepare_job hook is running
* Refactoring GetResponse method
* Changing exception message if response file is not found
Co-authored-by: Ferenc Hammerl <31069338+fhammerl@users.noreply.github.com>
* Chaning exception message if isAlpine is null for prepare_job hook
* Rename hook folder
* Fail if compatible hookfile not found
* Use .Value instead of casting bool? to bool
* Format spacing
* Formatting
* User user and system mvs
* Use variables instead of entire context in featuremanager
* Update stepTelemetry if step uses containerhooks
* Restore import
* Remove unneccessary field from HookContainer
* Refactor response context and portmappings
* Force allow hooks if FF is on
* Code refactoring
* Revert deleting usings
* Better hookContainer defaults and use correct portmapping list
* Make GetDefaultShellForScript a HostContext extension method
* Generic hookresponse
* Code refactoring, unnecessary properties removed - HookContainer moved to the HookInput.cs
* Remove empty line
* Code refactoring and better exception handling
* code refactor, removing unnecessary props
* Move hookstate to global ContainerHookState
* Trace exception before we throw it for not losing information
* Fix for null ref exception in GetResponse
* Adding additional check for null response in prepareJob hook
* Refactoring GetResponse with additional check
* Update error messages
* Ports in ResponseContainer changed from IList to IDictionary
* Fix port format
* Include dockerfile
* Send null Registry obj if there's nothing in it
* Minor formatting
* Check if hookIndexPath exists relocated to the ContainerHookManager
* Code refactoring - ValidateHookExecutable added to the ContainerHookManager
* check if ContainerHooksPath when AllowRunnerContainerHooks is on
* Submit JSON telemetry instead of boolean
* Prefix step hooks with "run"
* Rename FeatureManager
* Fix flipped condition
* Unify js shell path getter with ps1 and sh getter
* Validate on run, not on instantiation of manager
* Cleanup ExecuteAsync methods
* Handle exception in executeHookScript
* Better exception types
* Remove comment
* Simplify boolean
* Allow jobs without jobContainer to run
* Use JObject instead of JToken
* Use correct Response type
* Format class to move cleanupJobAsync to the end of public methods
* Rename HookIndexPath to HookScriptPath
* Refactor methods into expression bodies
* Fix args class hierarchy
* Fix argument order
* Formatting
* Fix nullref and don't swallow stacktrace
* Whilelist HookArgs
* Use FF in FeatureManager
* Update src/Runner.Worker/ContainerOperationProvider.cs
Co-authored-by: Tingluo Huang <tingluohuang@github.com>
* Update src/Runner.Worker/ActionRunner.cs
Co-authored-by: Thomas Boop <52323235+thboop@users.noreply.github.com>
* Update src/Runner.Worker/ActionRunner.cs
Co-authored-by: Thomas Boop <52323235+thboop@users.noreply.github.com>
* Only mount well known dirs to job containers
* Get trace from hostcontext
* Use hook execution for setting telemetry
Co-authored-by: Nikola Jokic <nikola.jokic@akvelon.com>
Co-authored-by: Nikola Jokic <nikola-jokic@users.noreply.github.com>
Co-authored-by: Nikola Jokic <97525037+nikola-jokic@users.noreply.github.com>
Co-authored-by: Stefan Ruvceski <stefan.ruvceski@akvelon.com>
Co-authored-by: ruvceskistefan <96768603+ruvceskistefan@users.noreply.github.com>
Co-authored-by: Thomas Boop <thboop@github.com>
Co-authored-by: stefanruvceski <ruvceskistefan@github.com>
Co-authored-by: Tingluo Huang <tingluohuang@github.com>
Co-authored-by: Thomas Boop <52323235+thboop@users.noreply.github.com>
* Added continue on error to composite action
* changed from boolean-strategy-context -> boolean-steps-context for action_yaml
* refactored composite handler to always set outcome
* retrigger checks
* fixed typo in ??= operator
* boolean-steps-context accepts the same context as string-steps-context
* setting the outcome only on continue-on-error
* moved continue on error logic to the execution context
* Added L0 table tests for continue-on-error ExecutionContext
* Added missing mocks on StepsRunnerL0 for this update
* removed empty line and added one line separating the call
* Removed empty line
Co-authored-by: Ferenc Hammerl <31069338+fhammerl@users.noreply.github.com>
Co-authored-by: Ferenc Hammerl <31069338+fhammerl@users.noreply.github.com>
* conditional support for composite actions
* Fix Conditional function evaluation
* Push launch.json temporarily
* Revert "Push launch.json temporarily"
* rename context
* Cleanup comments
* fix success/failure functions to run based on pre/main steps
* idea of step_status
* change to use steps context, WIP
* add inputs to possible if condition expressions
* use action_status
* pr cleanup
* Added right stages
* Test on stage in conditional functions
* Fix naming and formatting
* Fix tests
* Add success and failure L0s
* Remove comment
* Remove whitespace
* Undo formatting
* Add L0 for step-if parsing
* Add ADR
Co-authored-by: Thomas Boop <thboop@github.com>
* Always register post steps for local actions
* Register post steps along with their conditions
* remove debug code
Co-authored-by: Ferenc Hammerl <fhammerl@github.com>
* Add environment variable for GITHUB_ACTION_PATH
* ah
* Remove debugging messages
* Set github action path at step level instead of global scope to avoid necessary removal
* Remove set context for github action
* Set github action path before and after composite action
* Copy GitHub Context, use this copied context for each composit step, and then set the action_path for each one (to avoid stamping over parent pointer GitHubContext
* Remove redundant code (display name is already evaluated in ActionRunner beforehand for each step)
* remove
* Remove nesting information for composite steps.
* put messages in debug logs if composite. if not, put these messages as outputs
* Fix group issue
* Fix end group issue
* Exploring child Linked Cancellation Tokens
* Preliminary Timeout-minutes fix
* Final Solution for resolving cancellation token's timeout vs. cancellation
* Clean up + Fix error handling
* Use linked tokens instead
* Clean up
* one liner
* Remove JobExecutionContext => Replace with public Root accessor
* Move CreateLinkedTokenSource in the CreateCompositeStep Function
This PR changes GITHUB_ACTION to use the step ContextName, instead of refname. The behavior is behind a feature flag. Refname is an otherwise deprecated property.
Primary motivation: For composite actions, we need a distinct GITHUB_ACTION for each nested step. This PR adds code to generate a default context name for nested steps.
For nested steps, GITHUB_ACTION will be set to "{ScopeName}.{ContextName}" to ensure no collisions.
A corresponding change will be made on the server so context name is never empty. Generated context names will start with "__".
A follow-up PR is required to avoid tracking "step" context values (outputs/conclusion/result) for generated context names. Waiting on telemetry from the server to confirm it's safe to assume leading "__" is a generate context name.
* Add basic framework for baby steps runner
* Basic logic for adding steps / invoking composite action steps
* Composite Steps Runner MVP
* Fix null object reference error
* intialize composiute
* Comment out code that is handled by stepsrunner
* Add composite clean up step
* Remove previous 'workarounds' from StepsRunner. Clean Up PR
* Remove todo
* Remove todo
* Fix using unitialized object yikes
* Remove time delay
* Format handler
* Move output handler into action handler
* Add try to evaluate display name
* Remove while loop yikes
* Abstract away the windows encoding check during step running
* Github context set to {ScopeName}.{ContextName} or {ContextName} if ScopeName is null
* Remove setting result to sucess since result defaults to sucess
* Fix windows error
* Fix windows
* revert:
* Windows fix
* Fix Windows Error in Abstraction
* Remove Composite Steps Runner => consolidate into Composite Steps Runner
* Remove unn. attribute in ExecutionContext
* Change protection levels, plus change function name to more clear meaning
* Remove location param
* location pt.2 fix
* Remove outputs step
* Remove temp directory
* new line
* Add arguitl not null
* better comment
* Change encoding name
* Check count > 0 for composite steps, import System.Threading
* Change function header encodingutil
* Add TODO
* Add await
* Handle Failed Step
* Move over SetAllCompositeOutputs to the handler
* Remove timeout-minutes setting in steps-level
* Use only ExecutionContext
* Move using to the top
* Remove redundant check
* Change function name
* Remove testing code
* Consolidate error code
* Consolidate code
* Change HandleOutput => ProcessCompositeActionOutputs
* Remove set the timeout comment
* Add Cancelling functionality + Remove unn. parameter
* Composite Action Run Steps
* Env Flow => Able to get env variables and overwrite current env variables => but it doesn't 'stick'
* clean up
* Clean up trace messages + add Trace debug in ActionManager
* Add debugging message
* Optimize runtime of code
* Change String to string
* Add comma to Composite
* Change JobSteps to a List, Change Register Step function name
* Add TODO, remove unn. content
* Remove unnecessary code
* Fix unit tests
* Fix env format
* Remove comment
* Remove TODO message for context
* Add verbose trace logs which are only viewable by devs
* Initial Start for FileTable stuff
* Progress towards passing FileTable or FileID or FileName
* Sort usings in Composite Action Handler
* Change 0 to location
* Update context variables in composite action yaml
* Add helpful error message for null steps
* Pass fileID to all children token of root action token
* Change confusing term context => templateContext, Eliminate _fileTable and only use ExecutionContext.FileTable + update this table when need be
* Remove unnessary FileID attribute from CompositeActionExecutionData
* Clean up file path for error message
* Remove todo
* Initial start/framework for output handling
* Outline different class vs Handler approach
* Remove InitializeScope
* Remove InitializeScope
* Fix Workflow Step Env overiding Parent Env
* First Approach for Attaching ID + Group ID to each Composite Action Step
* Add GroupID to the ActionDefinitionData
* starting foundation for handling clean up outputs step
* Pass outputs data to each composite action step to enable set-output functionality
* Create ScopeName for whole composite action.
This will enable us to add to the StepsContext[ScopeName] for the composite action which will allow us to use all these outputs in the cleanup step
* Hook up composite output step to handler => tmmrw implement composite output handler
* Add post composite action step to cleanup outputs => triggers composite output cleanup handler
* Fix Outputs Token handling start. Add individual step scope names.
* Set up Scope Name and Context Name naming system{
* Figured out how to pass Parent Execution Context to clean up step
* Figured out how to pass Parent Execution Context and scope names to
clean up step
* Add GetOutput function for StepsContext
* Generate child scope name correctly if parent scope name is null
* Simplify InitializeScope()
* Outputs are set correctly and able to get all final outputs in handler
* Parse through Action Outputs
* Fix null ScopeName + ContextName in CompositeOutputHandler
* Shift over handling of Action Outputs to output handler
* First attempt to fix null retrievals for output variables
* Basic Support for Outputs Done.
* Clean up pt.1
* Refactor outputs to avoid using Action Reference
* Clean up code
* Clean up part 2
* Add clarifying comments for the output handler
* Remove TODO
* Remove env in composite action scope
* Clean up
* Revert back
* revert back
* add back envToken
* Remove unnecessary code
* Add file length check
* Clean up
* Fix logging issue
* Figure out how to handle set-env edge cases
* formatting
* fix unit tests
* Fix windows unit test syntax error
* Fix period
* Sanity check for fileTable add + remove unn. code
* revert back
* Add back line break
* Fix null errors
* Address situation if FileTable is null + add sanity check for adding file to fileTable
* add line
* Revert
* Fix unit tests to instantiate a FileTable
* Fix logic for trimming manifestfile path
* Add null check
* Revert
* Revert
* revert
* spacing
* Add filetable to testing file, remove ? since we know filetable should never be non null
* Fix Throw logic
* Clarify template outputs token
* Add another type support for outputs to avoid container unit tests errors
* Add mapping for parity
* Build support for new outputs format
* Build support for new outputs format
* Refactor to avoid duplication of action yaml for workflow yaml
* revert
* revert
* revert
* spacing
* Composite Action Run Steps
* Env Flow => Able to get env variables and overwrite current env variables => but it doesn't 'stick'
* clean up
* Clean up trace messages + add Trace debug in ActionManager
* Add debugging message
* Optimize runtime of code
* Change String to string
* Add comma to Composite
* Change JobSteps to a List, Change Register Step function name
* Add TODO, remove unn. content
* Remove unnecessary code
* Fix unit tests
* Fix env format
* Remove comment
* Remove TODO message for context
* Add verbose trace logs which are only viewable by devs
* Initial Start for FileTable stuff
* Progress towards passing FileTable or FileID or FileName
* Sort usings in Composite Action Handler
* Change 0 to location
* Update context variables in composite action yaml
* Add helpful error message for null steps
* Pass fileID to all children token of root action token
* Change confusing term context => templateContext, Eliminate _fileTable and only use ExecutionContext.FileTable + update this table when need be
* Remove unnessary FileID attribute from CompositeActionExecutionData
* Clean up file path for error message
* Remove todo
* Initial start/framework for output handling
* Outline different class vs Handler approach
* Remove InitializeScope
* Remove InitializeScope
* Fix Workflow Step Env overiding Parent Env
* First Approach for Attaching ID + Group ID to each Composite Action Step
* Add GroupID to the ActionDefinitionData
* starting foundation for handling clean up outputs step
* Pass outputs data to each composite action step to enable set-output functionality
* Create ScopeName for whole composite action.
This will enable us to add to the StepsContext[ScopeName] for the composite action which will allow us to use all these outputs in the cleanup step
* Hook up composite output step to handler => tmmrw implement composite output handler
* Add post composite action step to cleanup outputs => triggers composite output cleanup handler
* Fix Outputs Token handling start. Add individual step scope names.
* Set up Scope Name and Context Name naming system{
* Figured out how to pass Parent Execution Context to clean up step
* Figured out how to pass Parent Execution Context and scope names to
clean up step
* Add GetOutput function for StepsContext
* Generate child scope name correctly if parent scope name is null
* Simplify InitializeScope()
* Outputs are set correctly and able to get all final outputs in handler
* Parse through Action Outputs
* Fix null ScopeName + ContextName in CompositeOutputHandler
* Shift over handling of Action Outputs to output handler
* First attempt to fix null retrievals for output variables
* Basic Support for Outputs Done.
* Clean up pt.1
* Refactor outputs to avoid using Action Reference
* Clean up code
* Clean up part 2
* Add clarifying comments for the output handler
* Remove TODO
* Remove env in composite action scope
* Clean up
* Revert back
* revert back
* add back envToken
* Remove unnecessary code
* Add file length check
* Clean up
* Figure out how to handle set-env edge cases
* formatting
* fix unit tests
* Fix windows unit test syntax error
* Fix period
* Sanity check for fileTable add + remove unn. code
* revert back
* Add back line break
* Fix null errors
* Address situation if FileTable is null + add sanity check for adding file to fileTable
* add line
* Revert
* Fix unit tests to instantiate a FileTable
* Fix logic for trimming manifestfile path
* Add null check
* Revert
* Revert
* revert
* spacing
* Add filetable to testing file, remove ? since we know filetable should never be non null
* Fix Throw logic
* Clarify template outputs token
* Add another type support for outputs to avoid container unit tests errors
* Add mapping for parity
* Build support for new outputs format
* Refactor to avoid duplication of action yaml for workflow yaml
* Move SDK work in ActionManifestManager, Condense Code
* Defer runs evaluation till after for loop to ensure order doesn't matter
* Fix logic error in setting scope and context names
* Add Regex + Add Child Context name null resolution
* move private function to bottom of class
* Composite Action Run Steps
* Env Flow => Able to get env variables and overwrite current env variables => but it doesn't 'stick'
* clean up
* Clean up trace messages + add Trace debug in ActionManager
* Add debugging message
* Optimize runtime of code
* Change String to string
* Add comma to Composite
* Change JobSteps to a List, Change Register Step function name
* Add TODO, remove unn. content
* Remove unnecessary code
* Fix unit tests
* Fix env format
* Remove comment
* Remove TODO message for context
* Add verbose trace logs which are only viewable by devs
* Sort usings in Composite Action Handler
* Change 0 to location
* Update context variables in composite action yaml
* Add helpful error message for null steps
* Fix Workflow Step Env overiding Parent Env
* Remove env in composite action scope
* Clean up
* Revert back
* revert back
* add back envToken
* Remove unnecessary code
* Figure out how to handle set-env edge cases
* formatting
* fix unit tests
* Fix windows unit test syntax error
* Composite Action Run Steps
* Clean up trace messages + add Trace debug in ActionManager
* Change String to string
* Add comma to Composite
* Change JobSteps to a List, Change Register Step function name
* Add TODO, remove unn. content
* Remove unnecessary code
* Fix unit tests
* Add verbose trace logs which are only viewable by devs
* Sort usings in Composite Action Handler
* Change 0 to location
* Update context variables in composite action yaml
* Add helpful error message for null steps
