* Print immutable action package details in set up job logs
* "Source commit SHA" instead of "Commit SHA" for immutable actions logs
---------
Co-authored-by: Thomas Boop <52323235+thboop@users.noreply.github.com>
* fix: add `git` and `curl` to default packages installed
Hi 👋🏼
These packages are used in a ton of actions on the marketplace. It would be nice if they were installed and ready for use instead of having to install them with `apt-get` on every single Github workflow.
* Update Dockerfile
* Update images/Dockerfile
Co-authored-by: Guillermo Caracuel <633810+gcaracuel@users.noreply.github.com>
* Update images/Dockerfile
Co-authored-by: Tingluo Huang <tingluohuang@github.com>
---------
Co-authored-by: Guillermo Caracuel <633810+gcaracuel@users.noreply.github.com>
Co-authored-by: Tingluo Huang <tingluohuang@github.com>
* Upgrade node20: 20.8.1 → 20.13.1
* Call out the release process for `alpine_nodejs` in a comment
* move the comment to the end of the line so it's more obvious which variable it's talking about
* backoff if we retried polling for more than 50 times in less than 30minutes
* run dotnet format
* move delay to after no message trace
* run dotnet format
* Add new SessionConflict return code
* formatting
* Change return type of CreateSessionAsync to new enum
* Update entry scripts to handle new exit code
* Move enum
* consume pipelines service url if present
updates how the `ACTIONS_RUNTIME_URL` variable is set to utilize a new value, `PipelinesServiceUrl` if present in the endpoint. if this value is not present then the existing system connection endpoint is used to retain backward compatibility.
* consume pipelines url
updates how the `ACTIONS_RUNTIME_URL` variable is set to utilize a new value, `PipelinesServiceUrl` if present in the endpoint. if this value is not present then the existing system connection endpoint is used to retain backward compatibility.
While there's an existing retry mechanism for the `docker pull` command
[^1], it's missing for `docker login`.
Similar to the `docker pull` scenario, the container registry could
potentially be briefly unavailable or inaccessible, leading to failed
`docker login` attempt and subsequent workflow run failures.
Since it's container based workflow, there is not way to retry on
customer side. The runner should retry itself.
It also aligns with community feedback [^2].
[^1]: 8e0cd36cd8/src/Runner.Worker/ContainerOperationProvider.cs (L201)
[^2]: https://github.com/orgs/community/discussions/73069
Co-authored-by: Thomas Boop <52323235+thboop@users.noreply.github.com>
This is required in order to ssh into a codespace for the repo:
> error getting ssh server details: failed to start SSH server:
Please check if an SSH server is installed in the container.
If the docker image being used for the codespace does not have an SSH server,
install it in your Dockerfile or, for codespaces that use Debian-based images,
you can add the following to your devcontainer.json:
>
> "features": {
> "ghcr.io/devcontainers/features/sshd:1": {
> "version": "latest"
> }
> }
* add content-type to block blob upload
* Add content-type for sdk path
* fix spacing
* merge headers and only when file extension is .txt
* add conditions
* tweak conditions and path matching
* pass in headers
* add content-type for appendblob
* Set an explicit timeout on ResultsHttpClient
* Hook up retry and standard message handler to ResultsHttpClient
* Remove explicit timeout constructor
* Fix linter
* Bump directly dotnet vulnerable packages
* Use just single file sdk upgrade for vulnerable dependencies
* Save with UTF8-BOM
* Trim down sdk to only Sdk.csproj upgrade
---------
Co-authored-by: Ferenc Hammerl <fhammerl@github.com>
* Add in workflow to automatically generate dotnet SDK patches
* Update workflow to move the creation of the PR until all the hashes have been compiled and added to the path
* Update upgrade to run not in parallel to prevent git command overlaps
* Update for tings feedback and do better user error handling
* JSON spec doesn't have comments
* Has to use the outputs to proxy variables between the runs
* Wrong output variable
* Be more explicit on the branch check
* Fix comments and json output
* Missed variable name in rename
* Fix race condition on jq write
* Update for comments and more optimized hash updates
* Need to forcibly exit for state to be detected
* Fixed the failure to use a variable instead
* hash files translation works with host context translated
* Change file encoding to utf8 for PipelineTemplateConstants.cs
* infer the feature flag on hash files based on the existence of the context
* encoded utf8 with bom
* rename host-workspace to host-work-directory and feature flag the new context var
* Added comment to explain ignore host work directory resolution
Co-authored-by: fhammerl <fhammerl@github.com>
* trigger pipeline
* Move to the github host-workspace context instead of the runner host-work-directory
* remove unused imports
---------
Co-authored-by: fhammerl <fhammerl@github.com>
* Add 'http://' to http(s)_proxy if there is no protocol
Before this commit, these URIs used to be ignored (nullproxy).
The new behaviour aligns with go style http(s)_proxy
* Update src/Runner.Sdk/RunnerWebProxy.cs
Co-authored-by: Nikola Jokic <jokicnikola07@gmail.com>
* Assert that original protocol is preserved
* Use startsWith for testing protocol
* Only modify proxy if useful
---------
Co-authored-by: Nikola Jokic <jokicnikola07@gmail.com>
* Fix `uses: docker://image:tag` steps when container hook is used
* Update src/Runner.Worker/ActionManager.cs
---------
Co-authored-by: Ferenc Hammerl <31069338+fhammerl@users.noreply.github.com>
* Set runner environment in runner context and env
Extract runner_environment from the global context and expose in the
`github.runner` context and env as `RUNNER_ENVIRONMENT`.
Signed-off-by: Philip Harrison <philip@mailharrison.com>
* encoding.
---------
Signed-off-by: Philip Harrison <philip@mailharrison.com>
Co-authored-by: Tingluo Huang <tingluohuang@github.com>
* [1742] Ensure multiple composite annoations are correctly written.
This implementation uses a collector pattern to allow embedded ExecutionContexts to stash Issue objects for later processing by a non-embedded ancestor ExecutionContext.
Also:
- Provide explicit constructor implementations for ExecutionContext
- Leverage explicit constructors to solidify immutability of several ExecutionContext class members.
- Fixed erroneous call to ExecutionContext.Complete in CompositeActionHandler.cs
- Use a consistent timestamp for FinishTime in ExecutionContext::Complete
* Ensure collected issues are processed only by a non-embedded ExecutionContext.
This was already implicit. Now, just making it explicit.
* Provide a clear mechanism that allows callers to opt-in/opt-out of ExecutionContext::AddIssue's logging behavior.
* Addressed deserialization inconsistencies in TimelineRecord.cs
* Added TimelineRecord unit tests.
* Refined unit tests related to TimelineRecord::Variables case-insensitivity
* Add a unit test that verifies ExecutionContextLogOptions::LogMessageOverride has the desired effect.
* Responded to PR feedback.
* Don't allow embedded ExecutionContexts to add Issues to a TimelineRecord
* Only upload to Results with new job message type
* No need to have separate websocketFeedServer
* Linting fix
* Update src/Runner.Common/JobServerQueue.cs
Co-authored-by: Tingluo Huang <tingluohuang@github.com>
* add connection timeout
* Consolidate initializing webclient to result client
* Retry websocket delivery for console logs
* Linter fix
* Do not give up for good, reconnect again in 10 minutes
* Has to reset delivered
* Only first time retry 3 times to connect to websocket
---------
Co-authored-by: Tingluo Huang <tingluohuang@github.com>
* Rename AcquireJobRequest::StreamID to AcquireJobRequest::JobMessageID to match corresponding server-side update.
* appeased the linter
* Added unit tests to prove AcquireJobRequest serialization/deserialization is as-expected.
* Distinguish unit test variations.
* Incorporated PR Feedback.
* which handles broken symlink & unit test added (#2150)
* Update src/Runner.Sdk/Util/WhichUtil.cs
Co-authored-by: Ferenc Hammerl <31069338+fhammerl@users.noreply.github.com>
* fix pr comments
* trace log added, in case we find a symlink that is broken
* add check in case the target of the symlink is a relative path; added test that check symlink that targets a full path; added test that check symlink that targets a relative path;
* fix tests
* fix tests for linux
---------
Co-authored-by: Eli Entelis <eli.entelis@hexagon.com>
Co-authored-by: Eli Entelis <42981948+Eliminator1999@users.noreply.github.com>
Co-authored-by: Ferenc Hammerl <31069338+fhammerl@users.noreply.github.com>
* Also send Steps update to Results service
* Refactor to separate results server from current job server
* If hit any error while uploading to Results, skip Results upload
* Add proxy authentication and buffer request for WinHttpHandler
* Remove unnecessary null guard
* Also send Results telemetry when step update fails
* IResultsServer is not disposable
* Guard against NullReference while creating HostContext
* Throw on IOUtil loading object if content is empty or object is null
* removed unused dependency
* Changed exceptions to ArgumentNullException and ArgumentException
* fixed my mistake on LoadObject
* fixed tests
* trigger workflows
* trigger workflows
* encode files using utf8
* terminate the runner on deprication message
* added TaskAgentVersion exception to catch deprication
* AccessDenied exception with inner exception InvalidTaskAgent
* Access denied exception in program and in message listener
* Fixed copy
* remove trace message from message listener
* Bypass top level domain even if no_proxy specified it with leading '.'
E.g. no_proxy='.github.com' will now bypass github.com.
* Bypass proxy on all hosts when no_proxy is * (wildcard)
* Undo '.' stripping
* Simplify unit tests
* Respect wildcard even if it's one of many no_proxy items
* Runner service exit after consecutive re-try exits
* Rename failure counts and include reset count on runner listening for jobs
* Changed from graceful shutdown to stopping=true
* Revert "Revert "Uploading step logs to Results as well (#2422)" (#2437)"
This reverts commit 8c096baf49.
* Properly guard the upload to results feature
* Delete skipped file if deletesource is true
* Update lint.yml
Don't ignore the formatting errors
* Add formatting made by @cory-miller
* Use dotnet format
* Format only modified files
* Add instruction to contribute.md
* Use git status instead of git diff
* Rename queue to results queue
* Add results contracts
* Add Results logging handling
* Adding calls to create and finalize append blob
* Modifications for azurite upload
* Only call upload complete on final section and remove size
* Make method specific to step log so we can support job log later
* Change contract for results
* Add totalline count to the result log upload file
* Actually pass lineCount to Results Service
* Fix typos
* Code cleanup
* Fixing typos
* Apply suggestions from code review
Co-authored-by: Konrad Pabjan <konradpabjan@github.com>
---------
Co-authored-by: Brittany Ellich <brittanyellich@github.com>
Co-authored-by: Konrad Pabjan <konradpabjan@github.com>
* Sanitize OS Desc for UserAgents
* Only drop brackets if needed, refactoring
* Add missing ')'
* Readd missing brackets around '(header)'
* Add comments
* Use bracket solution from SDK
* Rename tests
* send more stuff as part of job completed
* feedback
* set only once
* feedback
* feedback
* fix test
* feedback
* nit: spacing
* nit: line
Co-authored-by: Tingluo Huang <tingluohuang@github.com>
---------
Co-authored-by: Tingluo Huang <tingluohuang@github.com>
* Defer evaluation of a step's DisplayName until its condition is evaluated.
* Formalize TryUpdateDisplayName and EvaluateDisplayName as members of interface `IStep` (#2374)
* Remove unnecessary timelineId and timelineRecordId and use Guid stepId
* Log upload error to kusto
* Remove try-catch
* Using a well known telmetry record to avoid replacing issues
* fix Guid format
* Use results service for uploading step summaries
* Use results summary over generic results naming convention
* Apply suggestions from code review
Co-authored-by: Tingluo Huang <tingluohuang@github.com>
* Addressing feedback
* Fix merge issue
* Remove empty line
* Update Results json objects to use snake case
* Adding the reference
Co-authored-by: Yang Cao <yacaovsnc@github.com>
Co-authored-by: Tingluo Huang <tingluohuang@github.com>
* Added env variable to control wether the terminal is silent
* Log to stdout if PrintLogToStdout is enabled
* Extracted console logging to stdouttracelistener
* Remove useless usings
* Rewrite TraceListener as superclass
* Only print to stdout if env is set
* Add comment for Console.Out
* Format Listener
* Revert var name in terminal
* Check env in hostcontext instead of Tracing constructor
* Remove superclass & dupe logging code
* Log hostType
* Readonly '_' prefix 'hostType'
* Fix test
* Revert Terminal change
Co-authored-by: Ferenc Hammerl <31069338+fhammerl@users.noreply.github.com>
* Workflow
* add back github-token to id:image
* added handle to image name
* removed core require
* multi line added :latest
* added label
* added repository_owner in IMAGE_NAME
* with the release
* release
* markdown label description
* Remove markdown desciprion
* Remove double quotes in labels
* Reverted back releaseVersion
* run.sh installs SIGINT and SIGTERM traps to gracefully stop runner
* replaced trap to force wait for child processes and to send kill to run-helper instead of Runner.Listener
* run with signal handling if RUNNER_MANUALLY_TRAP_SIG is set
* Update src/Misc/layoutroot/run.sh
Co-authored-by: Tingluo Huang <tingluohuang@github.com>
Co-authored-by: Tingluo Huang <tingluohuang@github.com>
* Allow container.image to be null/empty
* Remove unnecessary workflow type
* Update src/Sdk/DTPipelines/Pipelines/ObjectTemplating/PipelineTemplateConverter.cs
Co-authored-by: Ferenc Hammerl <31069338+fhammerl@users.noreply.github.com>
* Rename ConvertToJobContainer to ConvertToContainer
It converts serviceContainers too
* Remove unused parameter
AllowExpressions was always false
* Allow service in yaml with empty string for value
* Don't attempt to start services without an image
* Improve error messages
* Revert "Remove unused parameter"
This reverts commit f4ab2d50d0.
* Revert "Rename ConvertToJobContainer to ConvertToContainer"
This reverts commit ffc62f8343.
* Fix revert merge conflict
* Removed info logs
* Added info log for services without container images
Co-authored-by: moonblade <moonblade168@gmail.com>
Co-authored-by: Ferenc Hammerl <31069338+fhammerl@users.noreply.github.com>
Co-authored-by: Ava Stancu <avastancu@github.com>
For runners that are already configured but need to add systemd services after the fact, a new command option is added to generate the file only. Once generated, ./svc.sh install and other commands will be available.
This also adds support for falling back to the tenant name in the Actions URL in cases when the GitHub URL is not provided, such as for our hosted larger runners.
* Avastancu/joannaakl/service container error log (#2110)
* adding support for a service container docker logs
* Adding Unit test to ContainerOperationProvider
* Adding another test to ContainerOperationProvider
* placed the docker logs output in dedicated ##group section
* Removed the exception thrown if the service container was not healthy
* Removed duplicated logging to the executionContext
* Updated the container logs sub-section message
* Print service containers only if they were healthy
Unhealthy service logs are printed in ContainerHealthCheckLogs called prior to this step.
* Removed recently added method to inspect docker logs
The method was doing the same thing as the existing DockerLogs method.
* Added execution context error
This will make a failed health check more visible in the UI without disrupting the execution of the program.
* Removing the section 'Waiting for all services to be ready'
Since nested subsections are not being displayed properly and we already need one subsection per service error.
* Update src/Runner.Worker/Container/DockerCommandManager.cs
Co-authored-by: Tingluo Huang <tingluohuang@github.com>
* Update src/Test/L0/TestHostContext.cs
Co-authored-by: Tingluo Huang <tingluohuang@github.com>
* Change the logic for printing Service Containers logs
Service container logs will be printed in the 'Start containers' section only if there is an error.
Healthy services will have their logs printed in the 'Stop Containers' section.
* Removed unused import
* Added back section group.
* Moved service containers error logs to separate group sections
* Removed the test testing the old logic flow.
* Remove unnecessary 'IsAnyUnhealthy' flag
* Remove printHello() function
* Add newline to TestHostContext
* Remove unnecessary field 'UnhealthyContainers'
* Rename boolean flag indicating service container failure
* Refactor healthcheck logic to separate method to enable unit testing.
* Remove the default value for bool variable
* Update src/Runner.Worker/ContainerOperationProvider.cs
Co-authored-by: Tingluo Huang <tingluohuang@github.com>
* Update src/Runner.Worker/ContainerOperationProvider.cs
Co-authored-by: Tingluo Huang <tingluohuang@github.com>
* Rename Healthcheck back to ContainerHealthcheck
* Make test sequential
* Unextract the container error logs method
* remove test asserting thrown exception
* Add configure await
* Update src/Test/L0/Worker/ContainerOperationProviderL0.cs
Co-authored-by: Tingluo Huang <tingluohuang@github.com>
* Update src/Test/L0/Worker/ContainerOperationProviderL0.cs
Co-authored-by: Tingluo Huang <tingluohuang@github.com>
* Update src/Test/L0/Worker/ContainerOperationProviderL0.cs
Co-authored-by: Tingluo Huang <tingluohuang@github.com>
* Update src/Test/L0/Worker/ContainerOperationProviderL0.cs
Co-authored-by: Tingluo Huang <tingluohuang@github.com>
* Update src/Test/L0/Worker/ContainerOperationProviderL0.cs
Co-authored-by: Tingluo Huang <tingluohuang@github.com>
* Add back test asserting exception
* Check service exit code if there is no healtcheck configured
* Remove unnecessary healthcheck for healthy service container
* Revert "Check service exit code if there is no healtcheck configured"
This reverts commit fec24e8341.
Co-authored-by: Ava S <avastancu@github.com>
Co-authored-by: Tingluo Huang <tingluohuang@github.com>
* Do not fail service containers without the healthcheck
Co-authored-by: JoannaaKL <joannaakl@github.com>
Co-authored-by: Tingluo Huang <tingluohuang@github.com>
* adding support for a service container docker logs
* Adding Unit test to ContainerOperationProvider
* Adding another test to ContainerOperationProvider
* placed the docker logs output in dedicated ##group section
* Removed the exception thrown if the service container was not healthy
* Removed duplicated logging to the executionContext
* Updated the container logs sub-section message
* Print service containers only if they were healthy
Unhealthy service logs are printed in ContainerHealthCheckLogs called prior to this step.
* Removed recently added method to inspect docker logs
The method was doing the same thing as the existing DockerLogs method.
* Added execution context error
This will make a failed health check more visible in the UI without disrupting the execution of the program.
* Removing the section 'Waiting for all services to be ready'
Since nested subsections are not being displayed properly and we already need one subsection per service error.
* Update src/Runner.Worker/Container/DockerCommandManager.cs
Co-authored-by: Tingluo Huang <tingluohuang@github.com>
* Update src/Test/L0/TestHostContext.cs
Co-authored-by: Tingluo Huang <tingluohuang@github.com>
* Change the logic for printing Service Containers logs
Service container logs will be printed in the 'Start containers' section only if there is an error.
Healthy services will have their logs printed in the 'Stop Containers' section.
* Removed unused import
* Added back section group.
* Moved service containers error logs to separate group sections
* Removed the test testing the old logic flow.
* Remove unnecessary 'IsAnyUnhealthy' flag
* Remove printHello() function
* Add newline to TestHostContext
* Remove unnecessary field 'UnhealthyContainers'
* Rename boolean flag indicating service container failure
* Refactor healthcheck logic to separate method to enable unit testing.
* Remove the default value for bool variable
* Update src/Runner.Worker/ContainerOperationProvider.cs
Co-authored-by: Tingluo Huang <tingluohuang@github.com>
* Update src/Runner.Worker/ContainerOperationProvider.cs
Co-authored-by: Tingluo Huang <tingluohuang@github.com>
* Rename Healthcheck back to ContainerHealthcheck
* Make test sequential
* Unextract the container error logs method
* remove test asserting thrown exception
* Add configure await
* Update src/Test/L0/Worker/ContainerOperationProviderL0.cs
Co-authored-by: Tingluo Huang <tingluohuang@github.com>
* Update src/Test/L0/Worker/ContainerOperationProviderL0.cs
Co-authored-by: Tingluo Huang <tingluohuang@github.com>
* Update src/Test/L0/Worker/ContainerOperationProviderL0.cs
Co-authored-by: Tingluo Huang <tingluohuang@github.com>
* Update src/Test/L0/Worker/ContainerOperationProviderL0.cs
Co-authored-by: Tingluo Huang <tingluohuang@github.com>
* Update src/Test/L0/Worker/ContainerOperationProviderL0.cs
Co-authored-by: Tingluo Huang <tingluohuang@github.com>
* Add back test asserting exception
* Check service exit code if there is no healtcheck configured
* Remove unnecessary healthcheck for healthy service container
* Revert "Check service exit code if there is no healtcheck configured"
This reverts commit fec24e8341.
Co-authored-by: Ava S <avastancu@github.com>
Co-authored-by: Tingluo Huang <tingluohuang@github.com>
* Adding a new vars context for non-secret variables
* Fix test case
* Trigger checks
* Remove variables from env context and environment varibale
* remove extra references
* Add prefix handling to configuration variables
* Fix test cases
* Consume variables using vars in context data
* removed action_yaml changes
* update.sh new logic with creating a file and waiting for it for 30 sec
* pr refactor
* removing file when it's detected bu run-helper
* Update src/Misc/layoutroot/run-helper.sh.template
Co-authored-by: Ferenc Hammerl <31069338+fhammerl@users.noreply.github.com>
* clean up update.finished in Runner.Listener right before it starts a process _update.sh
* pr fix IOUtil.DeleteFile
* self update widnows version - cmd
* same logic for returnCode 4 - ephemeral
* Init var so cmd doesn't run into syntax errors
* Use constants, setting a var messed up ERRORLEVEL
* Use var for updatefile
Co-authored-by: stefanruvceski <ruvceskistefan@github.com>
Co-authored-by: Thomas Boop <52323235+thboop@users.noreply.github.com>
Co-authored-by: Ferenc Hammerl <hammerl.ferenc@gmail.com>
* Created env var for forcing node12 actions to run on node16
* get value of hostContext environment variable
* changing location of forced node version check
* small code refactoring
Co-authored-by: Ferenc Hammerl <31069338+fhammerl@users.noreply.github.com>
* more of small code refactoring
Co-authored-by: Ferenc Hammerl <31069338+fhammerl@users.noreply.github.com>
* refactoring of conditions for getting internal node version
* changing expected value for node version env var
* Adding empty line between two methods
* Created method GetNodeVersion
* GetNodeVersion from function to inline call and PR fixes
Co-authored-by: Ferenc Hammerl <31069338+fhammerl@users.noreply.github.com>
Co-authored-by: Thomas Boop <52323235+thboop@users.noreply.github.com>
* Jsut simple solution without additional funcs
* Delete old comment
* resolve
* Refactor retry function, make it more common
* Make retry function generic, get rid of extra params
* delete extra using
* Add cancellation token and limit of attempts
* Add some additional logging
* Rework condition
* replace to do..while
* return `while (true)` to simplify code structure
* Add other cancelling token, add TODO comment
* Added ability to run Dockerfile.SUFFIX ContainerAction
* Extracted IsDockerFile method
* reformatted, moved from index to Last()
* extracted IsDockerfile to DockerUtil with L0
* added check for IsDockerfile to account for docker://
* updated test to clearly show path/dockerfile:tag
* fail if Data.Image is not Dockerfile or docker://[image]
* Setup noops for JobPrepare and JobCleanup hooks
* Add container jobstarted and jobcomplete hooks
* Run 'index.js' instead of specific command hooks
* Call jobprepare with command arg
* Use right command name (hardcoded)
Co-authored-by: Nikola Jokic <nikola-jokic@users.noreply.github.com>
* Invoke hooks with arguments
* Add PrepareJob hook to work with jobcontainers
Co-authored-by: Nikola Jokic <nikola-jokic@users.noreply.github.com>
* Rename methods
* Use new hookcontainer to run prep and clean hooks
* Get path from ENV
* Use enums
* Use IOUtils.cs
* Move container files to folder
* Move namespaces
* Store "state" between hooks
* Remove stdin stream in containerstephosts
* Update Constants.cs
* Throw if stdin fails
* Cleanup obvious nullrefs and unused vars
* Cleanup containerhook directory
* Call step exec hook
* Fix windows build
* Remove hook from hookContainer
* Rename file
* More renamings
* Add TODOs
* Fix env name
* Fix missing imports
* Fix imports
* Run script step on jobcontainer
* Enable feature if env is set
* Update ContainerHookManager.cs
* Update ContainerHookManager.cs
* Hooks allowed to work even when context isn't returned
* Custom hooks enabled flag and additional null checks
* New line at the end of the FeatureFlagManager.cs
* Code refactoring
* Supported just in time container building or pulling
* Try mock-build for osx
* Build all platforms
* Run mock on self-hosted
* Remove GITHUB prefix
* Use ContainerHooksPath instead of CustomHooksPath
* Null checks simplified
* Code refactoring
* Changing condition for image builing/pulling
* Code refactoring
* TODO comment removed
Co-authored-by: Ferenc Hammerl <31069338+fhammerl@users.noreply.github.com>
* Call container step if FF is on
* Rename run script function
* Use JToken instead of dynamic
* Add TODO
* Small refactoring + renames + TODOs
* Throw on DetermineNodeRuntimeVersion
* Fix formatting
* Add run-container-step
* Supported nodeJS in Alpine containers
* Renamed Alpine to IsAlpine in HookResponse
* Method for checking platform for alpine container
* Added container hooks feature flag check
* Update IsHookFeatureEnabled with new params
* Rename featureflag method
* Finish rename
* Set collection null values to empty arrays when JSON serialising them
* Disable FF until we merge
* Update src/Runner.Worker/Container/ContainerHooks/HookContainer.cs
* Fix method name
* Change hookargs to superclass from interface
* Using only Path.Combine in GenerateResponsePath
* fix merge error
* EntryPointArgs changed to list of args instead of one args string
* Changed List to IEnumerable for EntryPointArgs and MountVolumes
* Get ContainerRuntimePath for JobContainers from hooks
* Read ContainerEnv from response file
* Port mappings saved after creating services
* Support case when responseFile doesn't exist
* Check if response file exists
* Logging in ExecuteHookScript
* Save hook state after all 4 hooks
* Code refactoring
* Remove TODO
Co-authored-by: Ferenc Hammerl <31069338+fhammerl@users.noreply.github.com>
* Remove second TODO
Co-authored-by: Ferenc Hammerl <31069338+fhammerl@users.noreply.github.com>
* Removing container env changes
* Removing containerEnv and dockerManager
* Delete mock-build.yml
* Update IOUtil.cs
* Add comment about containerhooks
* Fix merge mistake
* Remove solved todo
* Determine which shell to use for hooks scenario
* Overload for method ExecuteHookScript with prependPath as arg
* Adding HostContext to the GetDefaultShellForScript call
* prependPath as a mandatory parameter
* Improve logging for hooks
* Small changes in logging
* Allow null for ContainerEntryPointArgs
* Changed log messages
* Skip setting EntryPoint and EntryPointArgs if hooks are enabled
* Throw if IsAlpine is null in PrepareJob
* Code refactoring - added GetAndValidateResponse method
* Code refactoring
* Changes in exception message
* Only save hookState if returned
* Use FF from server
* Empty line
* Code refactoring
Co-authored-by: Ferenc Hammerl <31069338+fhammerl@users.noreply.github.com>
* Send null instead of string empty
* Remove TODO
* Code refactoring and some small changes
* Allow Globals to be null to pass L0
* Fix setup in StepHostL0
* Throw exception earlier if response file doesn't exist and prepare_job hook is running
* Refactoring GetResponse method
* Changing exception message if response file is not found
Co-authored-by: Ferenc Hammerl <31069338+fhammerl@users.noreply.github.com>
* Chaning exception message if isAlpine is null for prepare_job hook
* Rename hook folder
* Fail if compatible hookfile not found
* Use .Value instead of casting bool? to bool
* Format spacing
* Formatting
* User user and system mvs
* Use variables instead of entire context in featuremanager
* Update stepTelemetry if step uses containerhooks
* Restore import
* Remove unneccessary field from HookContainer
* Refactor response context and portmappings
* Force allow hooks if FF is on
* Code refactoring
* Revert deleting usings
* Better hookContainer defaults and use correct portmapping list
* Make GetDefaultShellForScript a HostContext extension method
* Generic hookresponse
* Code refactoring, unnecessary properties removed - HookContainer moved to the HookInput.cs
* Remove empty line
* Code refactoring and better exception handling
* code refactor, removing unnecessary props
* Move hookstate to global ContainerHookState
* Trace exception before we throw it for not losing information
* Fix for null ref exception in GetResponse
* Adding additional check for null response in prepareJob hook
* Refactoring GetResponse with additional check
* Update error messages
* Ports in ResponseContainer changed from IList to IDictionary
* Fix port format
* Include dockerfile
* Send null Registry obj if there's nothing in it
* Minor formatting
* Check if hookIndexPath exists relocated to the ContainerHookManager
* Code refactoring - ValidateHookExecutable added to the ContainerHookManager
* check if ContainerHooksPath when AllowRunnerContainerHooks is on
* Submit JSON telemetry instead of boolean
* Prefix step hooks with "run"
* Rename FeatureManager
* Fix flipped condition
* Unify js shell path getter with ps1 and sh getter
* Validate on run, not on instantiation of manager
* Cleanup ExecuteAsync methods
* Handle exception in executeHookScript
* Better exception types
* Remove comment
* Simplify boolean
* Allow jobs without jobContainer to run
* Use JObject instead of JToken
* Use correct Response type
* Format class to move cleanupJobAsync to the end of public methods
* Rename HookIndexPath to HookScriptPath
* Refactor methods into expression bodies
* Fix args class hierarchy
* Fix argument order
* Formatting
* Fix nullref and don't swallow stacktrace
* Whilelist HookArgs
* Use FF in FeatureManager
* Update src/Runner.Worker/ContainerOperationProvider.cs
Co-authored-by: Tingluo Huang <tingluohuang@github.com>
* Update src/Runner.Worker/ActionRunner.cs
Co-authored-by: Thomas Boop <52323235+thboop@users.noreply.github.com>
* Update src/Runner.Worker/ActionRunner.cs
Co-authored-by: Thomas Boop <52323235+thboop@users.noreply.github.com>
* Only mount well known dirs to job containers
* Get trace from hostcontext
* Use hook execution for setting telemetry
Co-authored-by: Nikola Jokic <nikola.jokic@akvelon.com>
Co-authored-by: Nikola Jokic <nikola-jokic@users.noreply.github.com>
Co-authored-by: Nikola Jokic <97525037+nikola-jokic@users.noreply.github.com>
Co-authored-by: Stefan Ruvceski <stefan.ruvceski@akvelon.com>
Co-authored-by: ruvceskistefan <96768603+ruvceskistefan@users.noreply.github.com>
Co-authored-by: Thomas Boop <thboop@github.com>
Co-authored-by: stefanruvceski <ruvceskistefan@github.com>
Co-authored-by: Tingluo Huang <tingluohuang@github.com>
Co-authored-by: Thomas Boop <52323235+thboop@users.noreply.github.com>
It generated invalid arguments for `Process()` when the `bash` command itself was an argument as well, for example:
```
_proc.StartInfo.FileName = "/usr/bin/docker";
_proc.StartInfo.Arguments = "exec -i --workdir /__w/container-hook-e2e/container-hook-e2e 47105c66144d8809d9fa2bce9a58ea0564cd14def0ae7952cd6231fba3576db1 sh -e '/__w/_temp/fd086560-cb92-4f3b-a99c-35a6b7b1bbdb.sh'";
```
* Job hook provider now sets shell name for script handler
* fixed script handler and job hook provider to work with the name without fail
* returned used import by osx
* fixed order of imports
* added quotes around resolved script path allowing space in script path
* added quotes around bash and sh _defaultArguments
* Changed double quotes to single quotes in sh -e
Co-authored-by: Ferenc Hammerl <31069338+fhammerl@users.noreply.github.com>
* Changed double quotes to single quotes in bash
Co-authored-by: Ferenc Hammerl <31069338+fhammerl@users.noreply.github.com>
Co-authored-by: Ferenc Hammerl <31069338+fhammerl@users.noreply.github.com>
* Added ability to run Dockerfile.SUFFIX ContainerAction
* Extracted IsDockerFile method
* reformatted, moved from index to Last()
* extracted IsDockerfile to DockerUtil with L0
* added check for IsDockerfile to account for docker://
* updated test to clearly show path/dockerfile:tag
* fail if Data.Image is not Dockerfile or docker://[image]
* Added warning in case of invalid combination of command and flags and/or arguments
* Deleting unnecessary comments
* Added separate list for generic options
* Added PAT to the valid remove options
* Added command name to the error message
* composite action github.action_path set based on the StepHost
* in progress on updating github context for input template
* Fixed updating the context data for evaluation
* refactored logic so it is a little cleaner
* removed resolving the action_path in CompositeActionHandler
* removed added DeepClone
* added feature flag and modified the dict in place
* refactored step host to change context data. Added L0
* repaired spaces
* moved logic from step host to execution context, added recursive translation
* removed empty lines
* moved to extension methods
* Update src/Test/L0/Worker/StepHostL0.cs
Co-authored-by: Ferenc Hammerl <31069338+fhammerl@users.noreply.github.com>
* Added continue on error to composite action
* changed from boolean-strategy-context -> boolean-steps-context for action_yaml
* refactored composite handler to always set outcome
* retrigger checks
* fixed typo in ??= operator
* boolean-steps-context accepts the same context as string-steps-context
* setting the outcome only on continue-on-error
* moved continue on error logic to the execution context
* Added L0 table tests for continue-on-error ExecutionContext
* Added missing mocks on StepsRunnerL0 for this update
* removed empty line and added one line separating the call
* Removed empty line
Co-authored-by: Ferenc Hammerl <31069338+fhammerl@users.noreply.github.com>
Co-authored-by: Ferenc Hammerl <31069338+fhammerl@users.noreply.github.com>
* Fixing null reference exception when configuring runner with invalid repo URL or token
* Throw exception instead of ConvertFromJson
* Storing the response code
* Add annotations if node 12 action is found
* Better placeholder
* Only warn if FF is on
* Move annotation logic
* Pass in the LTS Url
* Raise annotation right before executing the action
* Match server side FF name
* Change name back to features
* Better warning text
* Update src/Runner.Common/Constants.cs
Co-authored-by: Thomas Boop <52323235+thboop@users.noreply.github.com>
Co-authored-by: Thomas Boop <52323235+thboop@users.noreply.github.com>
* feed via websocket
* feed via websocket
* feedback
* ensure right schema is used
* fix resiliency
* some fixes
* fix sending message
* chunk data
* let's abort, which will also dispose
* close gracefully
* use safe_sleep executable in bash scripts
* new line at the end of safe_sleep bash script
* Replacing relative paths with absolute paths and changing location of safe_sleep
Co-authored-by: Ferenc Hammerl <31069338+fhammerl@users.noreply.github.com>
* Set GH actions Node version to 16 if FF is on unless user opted out
* Add L0s (WIP)
* Wrap tests into theory
* Only check for node12 actions
* Refactor node version picking
* Lets allow up to 150 characters on linux/mac, just to avoid some issues with runner naming
* Add 4 randomized digits on mac/linux
* fix pragma issue
* fix test
* Address pr feedback
* reduce complexity
* lets make it cleaner!
* fix test
* fix logic
* Issue 1596: runner throws nullref exception when writting env var
* Adding tests for missing new line after EOF marker
* Changing newline to new line
* Issue 1662: Adding retry policy for methods GetTenantCredential and GetJITRunnerTokenAsync
* Adding HttpClient creation to the retry
* Random backoff time
* Use 16 to run RunnerService.js
* Execute hashfiles using node16
* Run downloadCert.js using node16
* Run makeWebRequest.js using node16
* Run macos-run-invoker.js using node16
* Run hashFiles.js using node16
* Update tests to use node16
* Update documentation to recommend node16
* Duplicate macos service js fix for 16
* Add PR link
* Revert ADR node change
* Merge node12/16 retainment IFs
* Try both node12 and node16
* Close if
* Revert "Update tests to use node16"
This reverts commit bbca7b9f1c.
* Fix condition
* Unfurl if condition
* Allow user to force a node version
* Format update template
* Comment env var
* Rename vars
* Fix naming
* Fix rename
* Set node ver override if job message has it
* Format executionContext
* Can only receive 'forceNode12' or nothing from FF
No specific node version from server
Co-authored-by: Ferenc Hammerl <hammerl.ferenc@gmail.com>
* Solving issue with debugging on macOS M1
* Fixing problem with debugging on macOS M1
* Adding targetArchitecture in launch.json configs
* Code refactor
* Kill node process to recover handle
So we can print to the console in Runner.Listener once again
* Revert testing changes
Co-authored-by: Ferenc Hammerl <hammerl.ferenc@gmail.com>
* First prototype of step summary environment variable
* Fix file contention issue
* Try to simplify cleaning up file references
* use step id as md file name, queue file attachment
* separate logic into attachment summary func
* Fix indentation
* Add (experimental) feature flag support
* reorganize summary upload determination logic
* file i/o exception handling + pr feedback
* Revert changes for now to reintroduce them later
* Add skeleton SetStepSummaryCommand
* Update step summary feature flag name
* Port ShouldUploadAttachment from previous iteration
* Port QueueStepSummaryUpload from previous iteration
* Improve exception handling when uploading attachment
* Add some minor logging improvements
* Refuse to upload files larger than 128k
* Implement secrets scrubbing
* Add TODO comment to remove debugging temp files
* Add first tests
* Add test for secret masking
* Add some naming/style fixes suggested in feedback
* inline check for feature flag
* Inline method for style consistency
* Make sure that scrubbed file doesn't exist before creating it
* Rename SetStepSummaryCommand to CreateStepSummaryCommand
* Fix error handling messages
* Fix file command name when registering extension
* Remove unnecessary file deletion
Co-authored-by: Rob Herley <robherley@github.com>
* Only execute post for actions that have one
* Working container runner update with run.sh
* Revert "Only execute post for actions that have one"
This reverts commit 9675941fdb.
* Relaunch the listener without quitting run.cmd
* Fix typo
* Extract most os run.sh logic so we can update it
* Add bash line endings
* Extract the logic from run.cmd
* Add EoF lines
* Add unexpected ERRORLEVEL messages to cmd
* Simplify contract between run and helper
* Remove unused exit
* WIP: run a copy of the helper so it's safe to update
* Throw NonRetryableException if not configured
* Log and format
* Fix typo
* Fix typo
* Use helper template system for bash as well
* Update run.sh
* Remove unnecessary comments
* Use ping instead of timeout
* Use localhost in ping-timeout (n times, w timeout)
Co-authored-by: Ferenc Hammerl <hammerl.ferenc@gmail.com>
* Revert "Make `run.sh|cmd` handle update without quitting so containers using them as entrypoints don't exit on update (#1494)"
d8251bf912
* update runnerversion as well
* Update 0276-problem-matchers.md
Update to reflect current behavior
* Update docs/adrs/0276-problem-matchers.md
Co-authored-by: Ferenc Hammerl <31069338+fhammerl@users.noreply.github.com>
* Only execute post for actions that have one
* Working container runner update with run.sh
* Revert "Only execute post for actions that have one"
This reverts commit 9675941fdb.
* Relaunch the listener without quitting run.cmd
* Fix typo
* Extract most os run.sh logic so we can update it
* Add bash line endings
* Extract the logic from run.cmd
* Add EoF lines
* Add unexpected ERRORLEVEL messages to cmd
* Simplify contract between run and helper
* Remove unused exit
* WIP: run a copy of the helper so it's safe to update
* Throw NonRetryableException if not configured
* Log and format
* Fix typo
* Fix typo
* Use helper template system for bash as well
* Update run.sh
* Remove unnecessary comments
* Use ping instead of timeout
* Use localhost in ping-timeout (n times, w timeout)
Co-authored-by: Ferenc Hammerl <hammerl.ferenc@gmail.com>
Many people open bug reports or feature requests in the `actions/runner`
repository that are more generally about GitHub Actions. Often changes
in GitHub Actions are cross-cutting across multiple teams or feature
areas, so it's best if we direct people to the more general areas
(Actions Community Support or GitHub Feedback) so that we can get the
most eyes on the problem and give the quickest response.
* conditional support for composite actions
* Fix Conditional function evaluation
* Push launch.json temporarily
* Revert "Push launch.json temporarily"
* rename context
* Cleanup comments
* fix success/failure functions to run based on pre/main steps
* idea of step_status
* change to use steps context, WIP
* add inputs to possible if condition expressions
* use action_status
* pr cleanup
* Added right stages
* Test on stage in conditional functions
* Fix naming and formatting
* Fix tests
* Add success and failure L0s
* Remove comment
* Remove whitespace
* Undo formatting
* Add L0 for step-if parsing
* Add ADR
Co-authored-by: Thomas Boop <thboop@github.com>
* Always register post steps for local actions
* Register post steps along with their conditions
* remove debug code
Co-authored-by: Ferenc Hammerl <fhammerl@github.com>
Without creating a session, the service is not able to access the keychains for the user specified under `UserName`. This causes any workflow that deals with code signing to fail as the only keychain loaded with be the system one. This should fix#350
* Handle macOS upgrade smoothly
* cleanup
* misc cleanup
* final updates
* Update src/Misc/layoutbin/update.sh.template
Co-authored-by: Patrick Ellis <319655+pje@users.noreply.github.com>
* Update src/Misc/layoutbin/update.sh.template
Co-authored-by: Patrick Ellis <319655+pje@users.noreply.github.com>
* Upload telemetry and default to old method as needed
* minor fix
* add one more bit of logging
* some more telemetry
* quote variables to handle spaces
* tiny fix for ubuntu
* remove version and move telemetry to diag
* use full path
Co-authored-by: Patrick Ellis <319655+pje@users.noreply.github.com>
* Prevent stopTokens that are workflow commands
Co-authored-by: Thomas Boop <52323235+thboop@users.noreply.github.com>
* Check context for env var too
* Accept true, 1 and $true instead of just "true"
* Setup ExpressionValues in tests
* Update src/Runner.Common/Constants.cs
Co-authored-by: Thomas Boop <52323235+thboop@users.noreply.github.com>
* Separate success and fail tests for invalid token
* Fix envcontext for tests
Co-authored-by: Thomas Boop <52323235+thboop@users.noreply.github.com>
* Ignore agentpool when unconfiguring the runner
Runner names and IDs are unique within a ServiceHost
They don't need to be included when unconfiguring the runner.
* Use -1 instead of 0 to highlight how it is ignored
* Use overloads and 0 instead of -1
Using 0 seems to be the convention
* Fix typo calling the wrong method
* Add generateIdTokenUrl as an env var
* Add generateIdTokenUrl to env vars
* Add basic telemetry class and submit it on jobcompleted
* Use constructor overload
* Rename telemetry to jobTelemetry
* Rename telemetry file
* Make JobTelemetryType a string
* Collect telemetry
* Remove debugger
* Update src/Runner.Worker/ActionCommandManager.cs
Co-authored-by: Thomas Boop <52323235+thboop@users.noreply.github.com>
* Use same JobTelemetry for all contexts
* Mask telemetry data
* Mask in JobRunner instead
* Empty line
* Change method signature
Returning with a List suggests we clone it and that the
original doesn't change..
* Update launch.json
Co-authored-by: Thomas Boop <52323235+thboop@users.noreply.github.com>
* Add generateIdTokenUrl as an env var
* Add generateIdTokenUrl to env vars
* Add name and runner_group to context
* No longer add runner-group
* Update runner name if needed
* Get interface instead of concrete class
* Check for nulls on ReservedAgent
* Avoid loading setting file unnecesseraly
* Only check agentName once
* Use Trace.Error when can't update settings
* Better equals and exception handling
* Update JobDispatcher.cs
* Add tests and null check
* Avoid white, it is unreadable on light themes (#1295)
* remove ', ConsoleColor.White' from banner
* remove ', ConsoleColor.White' from prompt
* cleanups
* Update error to say 'uninstall' not 'unconfigure'
* Say uninstall service in *nix config error msgs
Co-authored-by: Ferenc Hammerl <31069338+fhammerl@users.noreply.github.com>
* Trim pwsh special chars when masking secrets
* Add pwsh valueEncoder
* Explain regex
* Update ValueEncoders.cs
* Add tests for pwsh color codes in secrets
* Formatting
* Group tests into theories
* Split secret on PS chars and mask for them
* Clean up comments
* Remove unused unittest
* Rename escape methods
* Use flags in svc creation script
* Refactor regex and add comments
* Fix indentation and typo in user matching
* Consistency use flags in automation scripts
* Update documentation to reflect new usage
* Make example more readable
* Remove test echos from script
* Remove test echo
* Format scripts and remove test script
* Remove tar
* Use getopts and single letter flags
* Update docs to show flag usage
* Update usage of create svc
* Revert svc to not use flags
* Revert delete script
* Update docs
* Readd deleted comments
* Add Notice Command
* Add Feature Flag For Enhanced Annotations
* Fix tests
* Add validation for columns and lines
* Fix order to match service
* Remove console.write
* Make Validation Better
* Cleanup
* Handle empty/whitespace strings
* Add more validation for line/column ranges
* Make Validation Debug, Not Throw
* Change casing to 🐫 from 🐍
* Give notice a well known tag
* Cleanup
* Sanitize invalid commands rather than fail
* fix some typos in docs of JobDispatcher
* one more typo
* Update src/Runner.Listener/JobDispatcher.cs
Co-authored-by: Ferenc Hammerl <31069338+fhammerl@users.noreply.github.com>
* Stop ignoring .vscode (launch scripts)
* Check in launch scripts for config and run
This can cause an issue with existing launch configuration on the machines of contributors.
* Improve error msg when runner is not configured
* Unignore .vscode/launch and tasks only
* Remove stopAtEntry and add eof newline
* Remove Runner.Listener from error message
* Rename tasks and run configs
* Ignore BuildConstants.cs
* Use better error msg
* Explain development steps in depth
* Add launch config to directly debug worker
* Update docs with VS Code tips
* Remove auto-generated comments
* Fix link to quickstart in vscode.md
* Remove ':' from link to quickstart
* Revert "Ignore BuildConstants.cs"
This reverts commit 0f13922a87.
* Replace `.sh` with `.(sh/cmd)` in docs
* Wait for debugger to attach in Worker
Only if GITHUB_ACTIONS_RUNNER_ATTACH_DEBUGGER is set
* Only wait if env variable parses to boolean 'true'
* Add 30s timeout to the wait
* Clean up leftover line
* Decrease wait to 20s
* Use ConvertToBoolean isntead of TryParse
* Unset NODE_ICU_DATA before starting node in NodeHandler.
* Update src/Runner.Worker/Handlers/NodeScriptActionHandler.cs
Co-authored-by: Ross Brodbeck <hross@users.noreply.github.com>
Co-authored-by: Ross Brodbeck <hross@users.noreply.github.com>
* Delete script files before replacing during update
Signed-off-by: Tim Etchells <tetchel@gmail.com>
* Use IOUtil.DelteFile()
Co-authored-by: Tingluo Huang <tingluohuang@github.com>
From February 2021, in order to provide feedback on pull requests, Code Scanning workflows must be configured with both `push` and `pull_request` triggers. This is because Code Scanning compares the results from a pull request against the results for the base branch to tell you only what has changed between the two.
Early in the beta period we supported displaying results on pull requests for workflows with only `push` triggers, but have discontinued support as this proved to be less robust.
See https://docs.github.com/en/free-pro-team@latest/github/finding-security-vulnerabilities-and-errors-in-your-code/configuring-code-scanning#scanning-pull-requests for more information on how best to configure your Code Scanning workflows.
During job run we may fail to resolve actions download info, and this
stack is fully controlled by GitHub actions so it should be counted as
infrastructure failure instead of user failure.
* Use FIPS compliant crypto when required
* Comment cleanup
* Store OAuth signing scheme in credentialData instead of runner setting
Add encryption scheme for job message encyption key to session
Further cleanup of unused crypto code
* Update windows rsa key manager to use crossplat dotnet RSA api
* Undo unneeded ConfigurationManager change
* add `workflow_dispatch`
* Add an environment variable to indicate which repository the currently running Action came from.
* Expose the Action ref as well.
* Move setting `github.action_repository` and `github.action_ref` to `ActionRunner.cs`.
* Don't set `action_repository` and `action_ref` for local Actions.
Co-authored-by: Tingluo Huang <tingluohuang@github.com>
remove 3 "redundant" text and put one text for DRY.
and developers always forget `sudo` and annoying `Need to run with sudo privilege` message.
so, add first.
* Change `ping .. > nul` to `sleep`
The filename `nul` is a Windows-ism that causes the update script to
create such a file in the current working directory. The `ping`
utility is also an dependency not installed by
`installdependencies.sh`, so it seemed easier to change it to the
standard `sleep` command.
* Update dotnet-install script as requested by test
* Update dotnet-install.ps1
Co-authored-by: Tingluo Huang <tingluohuang@github.com>
* Unify apt/apt-get logic
The previous logic was buggy in that it tried to use `apt` in the `apt-get` branch after deciding that `apt` was unavailable...
* Prefer apt-get over apt
apt does not have a stable cli and using it from scripts yields annoying messages
* Improve English for missing apt-get & apt case
* Fix apt-get/apt fallback behavior for $ patterns
If there's a `$` in the apt install pattern, it will not fail if it selects a thing and decides it isn't interested in installing it.
* Fix spelling of libssl
This change updates the .NET Core SDK used by the Actions Runner to
version 3.1.302 to address the issues that are caused by the following issue:
https://github.com/dotnet/runtime/issues/13475
See #574 for more information.
Fixes#574
* Log in with container credentials if given
* Stub in registry aware auth for later
* Fix hang if password is empty
* Remove default param to fix build
* PR Feedback. Add some tests and fix parse
* Add environment variable for GITHUB_ACTION_PATH
* ah
* Remove debugging messages
* Set github action path at step level instead of global scope to avoid necessary removal
* Remove set context for github action
* Set github action path before and after composite action
* Copy GitHub Context, use this copied context for each composit step, and then set the action_path for each one (to avoid stamping over parent pointer GitHubContext
* Explicitly define what is allowed for an action
* Add step-env
* Remove secrets + defaults
* new line
* Add safety check to prevent from checking defaults in ScriptHandler for composite action
* Revert "Add safety check to prevent from checking defaults in ScriptHandler for composite action"
This reverts commit aeae15de7b.
* Need to explictly use ActionStep type since we need the .Inputs attribute which is only found in the ActionStep not IStep
* Fix ActionManifestManager
* Remove todos
* Revert "Revert "Add safety check to prevent from checking defaults in ScriptHandler for composite action""
This reverts commit a22fcbc036.
* revert
* Remove needs in env
* Make shell required + add inputs
* Remove passing context to all composite steps attribuyte
2020-07-28 10:15:46 -04:00
679 changed files with 77741 additions and 10081 deletions
about: If you have issues with GitHub Actions, please follow the "support for GitHub Actions" link, below.
title: ''
labels: bug
assignees: ''
---
<!--
👋 You're opening a bug report against the GitHub Actions**runner application**.
🛑 Please stop if you're not certain that the bug you're seeing is in the runner application - if you have general problems with actions, workflows, or runners, please see the [GitHub Community Support Forum](https://github.community/c/code-to-cloud/52) which is actively monitored. Using the forum ensures that we route your problem to the correct team. 😃
-->
**Describe the bug**
A clear and concise description of what the bug is.
about:If you have feedback or suggestions about GitHub Actions, please open a discussion (or add to an existing one) in the GitHub Actions Feedback. GitHub Actions Product Managers and Engineers monitor the feedback forum.
Thank you 🙇♀ for wanting to create a feature in this repository. Before you do, please ensure you are filing the issue in the right place. Issues should only be opened on if the issue **relates to code in this repository**.
* If you have found a security issue [please submit it here](https://hackerone.com/github)
* If you have questions or issues with the service, writing workflows or actions, then please [visit the GitHub Community Forum's Actions Board](https://github.community/t5/GitHub-Actions/bd-p/actions)
* If you are having an issue or question about GitHub Actions then please [contact customer support](https://help.github.com/en/actions/automating-your-workflow-with-github-actions/about-github-actions#contacting-support)
If you have a feature request that is relevant to this repository, the runner, then please include the information below:
**Describe the enhancement**
A clear and concise description of what the features or enhancement you need.
**Code Snippet**
If applicable, add a code snippet.
**Additional information**
Add any other context about the feature here.
NOTE: if the feature request has been agreed upon then the assignee will create an ADR. See docs/adrs/README.md
Whenever possible, changes should be accompanied by non-trivial tests that meaningfully exercise the core functionality of the new code being introduced.
All tests are in the `Test/` directory at the repo root. Fast unit tests are in the `Test/L0` directory and by convention have the suffix `L0.cs`. For example: unit tests for a hypothetical `src/Runner.Worker/Foo.cs` would go in `src/Test/L0/Worker/FooL0.cs`.
Run tests using this command:
```sh
cd src && ./dev.sh test
```
### Formatting
After editing .cs files, always format the code using this command:
```sh
cd src && ./dev.sh format
```
### Feature Flags
Wherever possible, all changes should be safeguarded by a feature flag; `Features` are declared in [Constants.cs](src/Runner.Common/Constants.cs).
close-issue-message:"This issue does not seem to be a problem with the runner application, it concerns the GitHub actions platform more generally. Could you please post your feedback on the [GitHub Community Support Forum](https://github.com/orgs/community/discussions/categories/actions) which is actively monitored. Using the forum ensures that we route your problem to the correct team. 😃"
close-issue-message:"Thank you for your interest in the runner application and taking the time to provide your valuable feedback. We kindly ask you to redirect this feedback to the [GitHub Community Support Forum](https://github.com/orgs/community/discussions/categories/actions) which our team actively monitors and would be a better place to start a discussion for new feature requests in GitHub Actions. For more information on this policy please [read our contribution guidelines](https://github.com/actions/runner#contribute). 😃"
if [[ "$docker_should_update" == "1" && "$buildx_should_update" == "0" ]]; then
echo "::warning ::Docker version (${{ steps.check_docker_version.outputs.LATEST_VERSION }}) needs update but Buildx is current. Only updating when both need updates."
fi
# Show annotation if only Buildx needs update
if [[ "$docker_should_update" == "0" && "$buildx_should_update" == "1" ]]; then
echo "::warning ::Buildx version (${{ steps.check_buildx_version.outputs.LATEST_VERSION }}) needs update but Docker is current. Only updating when both need updates."
fi
# Show annotation when both are current
if [[ "$docker_should_update" == "0" && "$buildx_should_update" == "0" ]]; then
echo "::warning ::Latest Docker version is ${{ steps.check_docker_version.outputs.LATEST_VERSION }} and Buildx version is ${{ steps.check_buildx_version.outputs.LATEST_VERSION }}. No updates needed."
sed -i "s/ARG BUILDX_VERSION=$current_version/ARG BUILDX_VERSION=$latest_version/g" ./images/Dockerfile
- name:Commit changes and create Pull Request
env:
GITHUB_TOKEN:${{ secrets.GITHUB_TOKEN }}
run:|
# Setup branch and commit information
branch_name="feature/docker-buildx-upgrade"
commit_message="Upgrade Docker to v${{ needs.check-versions.outputs.DOCKER_LATEST_VERSION }} and Buildx to v${{ needs.check-versions.outputs.BUILDX_LATEST_VERSION }}"
pr_title="Update Docker to v${{ needs.check-versions.outputs.DOCKER_LATEST_VERSION }} and Buildx to v${{ needs.check-versions.outputs.BUILDX_LATEST_VERSION }}"
run:echo "::error links::feature/dotnet-sdk-upgrade${{ steps.fetch_latest_version.outputs.DOTNET_LATEST_MAJOR_MINOR_PATCH_VERSION }} https://github.com/actions/runner/tree/feature/dotnet-sdk-upgrade${{ steps.fetch_latest_version.outputs.DOTNET_LATEST_MAJOR_MINOR_PATCH_VERSION }}::Branch feature/dotnetsdk-upgrade/${{ steps.fetch_latest_version.outputs.DOTNET_LATEST_MAJOR_MINOR_PATCH_VERSION }} already exists. Please take a look and delete that branch if you wish to recreate"
- name:Create a warning annotation if no need to update
run:echo "::warning ::Latest DotNet SDK patch is ${{ steps.fetch_latest_version.outputs.DOTNET_LATEST_MAJOR_MINOR_PATCH_VERSION }}, and we are on ${{ steps.fetch_current_version.outputs.DOTNET_CURRENT_MAJOR_MINOR_PATCH_VERSION }}. No need to update"
This update ensures we're using the latest stable Node.js versions for security and performance improvements.
**Note**: When updating Node versions, remember to also create a new release of alpine_nodejs at the updated version following the instructions at: https://github.com/actions/alpine_nodejs
---
Autogenerated by [Node Version Upgrade Workflow](https://github.com/actions/runner/blob/main/.github/workflows/node-upgrade.yml)
EOF
# Create PR
gh pr create -B main -H "$branch_name" \
--title "chore: update Node versions" \
--label "dependencies" \
--label "dependencies-weekly-check" \
--label "dependencies-not-dependabot" \
--label "node" \
--label "javascript" \
--body-file pr_body.txt
echo "::notice title=PR Created::Successfully created Node.js version update PR on branch $branch_name"
stale-issue-message:"This issue is stale because it has been open 365 days with no activity. Remove stale label or comment or this will be closed in 15 days."
close-issue-message:"This issue was closed because it has been stalled for 15 days with no activity."
The runner is the application that runs a job from a GitHub Actions workflow. The runner can run on the [hosted machine pools](https://github.com/actions/virtual-environments) or run on [self-hosted environments](https://help.github.com/en/actions/automating-your-workflow-with-github-actions/about-self-hosted-runners).
The runner is the application that runs a job from a GitHub Actions workflow. It is used by GitHub Actions in the [hosted virtual environments](https://github.com/actions/virtual-environments), or you can [self-host the runner](https://help.github.com/en/actions/automating-your-workflow-with-github-actions/about-self-hosted-runners) in your own environment.
We accept contributions in the form of issues and pull requests. [Read more here](docs/contribute.md) before contributing.
Thank you for your interest in this GitHub repo, however, right now we are not taking contributions.
We continue to focus our resources on strategic areas that help our customers be successful while making developers' lives easier. While GitHub Actions remains a key part of this vision, we are allocating resources towards other areas of Actions and are not taking contributions to this repository at this time. The GitHub public roadmap is the best place to follow along for any updates on features we’re working on and what stage they’re in.
We are taking the following steps to better direct requests related to GitHub Actions, including:
1. We will be directing questions and support requests to our [Community Discussions area](https://github.com/orgs/community/discussions/categories/actions)
2. High Priority bugs can be reported through Community Discussions or you can report these to our support team https://support.github.com/contact/bug-report.
3. Security Issues should be handled as per our [security.md](security.md)
We will still provide security updates for this project and fix major breaking changes during this time.
- Proxy support is required for some enterprises and organizations to start using their own selfhosted runners
- While there is not a standard convention, many applications support setting proxies via the environmental variables `http_proxy`, `https_proxy`, `no_proxy`, such as curl, wget, perl, python, docker, git, R, ect
- Proxy support is required for some enterprises and organizations to start using their own self-hosted runners
- While there is not a standard convention, many applications support setting proxies via the environment variables `http_proxy`, `https_proxy`, `no_proxy`, such as curl, wget, perl, python, docker, git, and R
- Some of these applications use `HTTPS_PROXY` versus `https_proxy`, but most understand or primarily support the lowercase variant
## Decision
We will update the Runner to use the conventional environment variables for proxies: `http_proxy`, `https_proxy` and `no_proxy` if they are set.
We will update the Runner to use the conventional environment variables for proxies: `http_proxy`, `https_proxy`, and `no_proxy` if they are set.
These are described in detail below:
-`https_proxy` a proxy URL for all https traffic. It may contain basic authentication credentials. For example:
- http://proxy.com
@@ -22,20 +22,20 @@ These are described in detail below:
- http://proxy.com
- http://127.0.0.1:8080
- http://user:password@proxy.com
-`no_proxy` a commaseperated list of hosts that should not use the proxy. An optional port may be specified
-`no_proxy` a comma-separated list of hosts that should not use the proxy. An optional port may be specified. For example:
-`google.com`
-`yahoo.com:443`
-`google.com,bing.com`
We won't use `http_proxy` for https traffic when `https_proxy` is not set, this behavior lines up with any libcurl based tools (curl, git) and wget.
Otherwise action authors and workflow users need to adjust to differences between the runner proxy convention, and tools used by their actions and scripts.
Otherwise, action authors and workflow users need to adjust to differences between the runner proxy convention, and tools used by their actions and scripts.
Example:
Customer set `http_proxy=http://127.0.0.1:8888` and configure the runner against `https://github.com/owner/repo`, with the `https_proxy` -> `http_proxy` fallback, the runner will connect to server without any problem. However, if user runs `git push` to `https://github.com/owner/repo`, `git` won't use the proxy since it require `https_proxy` to be set for any https traffic.
Customer sets`http_proxy=http://127.0.0.1:8888` and configures the runner against `https://github.com/owner/repo`, with the `https_proxy` -> `http_proxy` fallback, the runner will connect to the server without any problem. However, if a user runs `git push` to `https://github.com/owner/repo`, `git` won't use the proxy since it requires`https_proxy` to be set for any https traffic.
> `golang`, `node.js` and other dev tools from the linux community use `http_proxy` for both http and https traffic base on my research.
> `golang`, `node.js`, and other dev tools from the Linux community use `http_proxy` for both http and https traffic based on my research.
A majority of our users are using Linux where these variables are commonly required to be set by various programs. By reading these values, we simplify the process for selfhosted runners to set up proxy, and expose it in a way users are already familiar with.
A majority of our users are using Linux where these variables are commonly required to be set by various programs. By reading these values, we simplify the process for self-hosted runners to set up a proxy and expose it in a way users are already familiar with.
A password provided for a proxy will be masked in the logs.
@@ -43,19 +43,19 @@ We will support the lowercase and uppercase variants, with lowercase taking prio
### No Proxy Format
While exact implementations are different per application on handle`no_proxy` env, most applications accept a commaseparated list of hosts. Some accept wildcard characters (*). We are going to do exact case-insentive matches, and not support wildcards at this time.
While exact implementations are different per application on handling`no_proxy` env, most applications accept a comma-separated list of hosts. Some accept wildcard characters (`*`). We are going to do exact case-insensitive matches, and not support wildcards at this time.
For example:
- example.com will match example.com, foo.example.com, foo.bar.example.com
- foo.example.com will match bar.foo.example.com and foo.example.com
-`example.com` will match `example.com`, `foo.example.com`, and `foo.bar.example.com`
-`foo.example.com` will match `bar.foo.example.com` and `foo.example.com`
We will not support IP addresses for `no_proxy`, only hostnames.
## Consequences
1. Enterprises and organizations needing proxy support will be able to embrace selfhosted runners
2. Users will need to set these environmental variables before configuring the runner in order to use a proxy when configuring
3. The runner will read from the environmental variables during config and runtime and use the provided proxy if it exists
4. Users may need to pass these environmental variables into other applications if they do not natively take these variables
5. Action authors may need to update their workflows to react to the these environment variables
6. We will document the way of setting environmental variables for runners using the environmental variables and how the runner uses them
7. Like all other secrets, users will be able to relatively easily figure out proxy password if they can modify a workflow file running on a selfhosted machine
1. Enterprises and organizations needing proxy support will be able to embrace self-hosted runners
2. Users will need to set these environment variables before configuring the runner in order to use a proxy when configuring
3. The runner will read from the environment variables during config and runtime and use the provided proxy if it exists
4. Users may need to pass these environment variables into other applications if they do not natively take these variables
5. Action authors may need to update their workflows to react to these environment variables
6. We will document the way of setting environment variables for runners using the environment variables and how the runner uses them
7. Like all other secrets, users will be able to relatively easily figure out the proxy password if they can modify a workflow file running on a self-hosted machine
@@ -10,7 +10,7 @@ Compilation failures during a CI build should surface good error messages.
For example, the actual compile errors from the typescript compiler should bubble as issues in the UI. And not simply "tsc exited with exit code 1".
VSCode has an extensible model for solving this type of problem. VSCode allows users to configure which problems matchers to use, when scanning output. For example, a user can apply the `tsc` problem matcher to receive a rich error output experience in VSCode, when compiling their typescript project.
VSCode has an extensible model for solving this type of problem. VSCode allows users to configure which [problems matchers](https://code.visualstudio.com/docs/editor/tasks#_defining-a-problem-matcher) to use, when scanning output. For example, a user can apply the `tsc` problem matcher to receive a rich error output experience in VSCode, when compiling their typescript project.
The problem-matcher concept fits well with "setup" actions. For example, the `setup-nodejs` action will download node.js, add it to the PATH, and register the `tsc` problem matcher. For the duration of the job, the `tsc` problem matcher will be applied against the output.
@@ -18,23 +18,25 @@ The problem-matcher concept fits well with "setup" actions. For example, the `se
Additionally the line will appear red in the web UI (prefix with `##[error]`).
Additionally the line will appear red in the web UI (prefix with `::error`).
Note, an error does not imply task failure. Exit codes communicate failure.
@@ -184,7 +186,7 @@ Solving this problem means:
- Use the `github.workspace` (where the repo is cloned on disk)
- Match against a repository to determine the relative path within the repo
This is a place where we diverge from VSCode. VSCode task configuration are specific to the local workspace (workspace root is known or can be specified). We're solving a more generic problem, so we need more information - specifically the `fromPath` property - in order to accurately root the path.
This is a place where we diverge from VSCode. VSCode task configurations are specific to the local workspace (workspace root is known or can be specified). We're solving a more generic problem, so we need more information - specifically the `fromPath` property - in order to accurately root the path.
In order to avoid creating inaccurate hyperlinks on the error issues, the agent will verify the file exists and is in the main repository. Otherwise omit the file property from the error issue and debug trace what happened.
@@ -203,7 +205,7 @@ Problem matchers are unable to interpret severity strings other than `warning` a
However some tools indicate error/warning in different ways. For example `flake8` uses codes like `E100`, `W200`, and `F300` (error, warning, fatal, respectively).
Therefore, allow a property `severity`, sibling to `owner`, which identifies the default severity for the problem matcher. This allows two problem matchers are registered - one for warnings and one for errors.
Therefore, allow a property `severity`, sibling to `owner`, which identifies the default severity for the problem matcher. This allows two problem matchers to be registered - one for warnings and one for errors.
For example, given the following `flake8` output:
@@ -248,6 +250,42 @@ Two problem matchers can be used:
}
```
#### Default from path
The problem matcher can specify a `fromPath` property at the top level, which applies when a specific pattern doesn't provide a value for `fromPath`. This is useful for tools that don't include project file information in their output.
For example, given the following compiler output that doesn't include project file information:
```
ClassLibrary.cs(16,24): warning CS0612: 'ClassLibrary.Helpers.MyHelper.Name' is obsolete
```
A problem matcher with a default from path can be used:
run-actions run scripts using a platform specific shell:
`bash -eo pipefail` on non-windows, and `cmd.exe /c /d /s` on windows
The `shell` option overwrites this to allow different flags or completely different shells/interpreters
The `shell` option overrides this to allow different flags or completely different shells/interpreters
A small example is:
```yml
@@ -84,7 +84,7 @@ powershell/pwsh
- Users can always opt out by not using the builtins, and providing a shell option like: `pwsh -File {0}`, or `powershell -Command "& '{0}'"`, depending on need
cmd
- There doesnt seem to be a way to fully opt in to fail-fast behavior other than writing your script to check each error code and respond accordingly, so we cant actually provide that behavior by default, it will be completely up to the user to write this behavior into their script
- There doesn't seem to be a way to fully opt in to fail-fast behavior other than writing your script to check each error code and respond accordingly, so we can't actually provide that behavior by default, it will be completely up to the user to write this behavior into their script
- cmd.exe will exit (return the error code to the runner) with the errorlevel of the last program it executed. This is internally consistent with the previous default behavior (sh, pwsh) and is the cmd.exe default, so we keep that behavior
First party action `actions/cache` needs a input which is an explicit `key` used for restoring and saving the cache. For packages caching, the most comment`key` might be the hash result of contents from all `package-lock.json` under `node_modules` folder.
First party action `actions/cache` needs a input which is an explicit `key` used for restoring and saving the cache. For packages caching, the most common`key` might be the hash result of contents from all `package-lock.json` under `node_modules` folder.
There are serval different ways to get the hash `key` input for `actions/cache` action.
@@ -38,7 +38,7 @@ There are serval different ways to get the hash `key` input for `actions/cache`
`hashFiles()` will only support hashing files under the `$GITHUB_WORKSPACE` since the expression evaluated on the runner, if customer use job container or container action, the runner won't have access to file system inside the container.
`hashFiles()` will only take 1 parameters:
-`hashFiles('**/package-lock.json')` // Search files under $GITHUB_WORKSPACE and calculate a hash for them
-`hashFiles('**/package-lock.json')` // Search files under `$GITHUB_WORKSPACE` and calculate a hash for them
**Question: Do we need to support more than one match patterns?**
@@ -15,7 +15,7 @@ This gives us good coverage across the board for secrets and secrets with a pref
However, we don't have great coverage for cases where the secret has a string appended to it before it is base64 encoded (i.e.: `base64($pass\n))`).
Most notably we've seen this as a result of user error where a user accidentially appends a newline or space character before encoding their secret in base64.
Most notably we've seen this as a result of user error where a user accidentally appends a newline or space character before encoding their secret in base64.
## Decision
@@ -45,4 +45,4 @@ This will result in us only revealing length or bit information when a prefix or
- In the case where a secret has a prefix or suffix added before base64 encoding, we may now reveal up to 20 bits of information and the length of the original string modulo 3, rather then the original 16 bits and no length information
- Secrets with a suffix appended before encoding will now be masked across the board. Previously it was only masked if it was a multiple of 3 characters
@@ -24,7 +24,7 @@ The runner will look for a file `.setup_info` under the runner's root directory,
}
]
```
The runner will use `##[group]` and `##[endgroup]` to fold all detail info into an expandable group.
The runner will use `::group` and `::endgroup` to fold all detail info into an expandable group.
Both [virtual-environments](https://github.com/actions/virtual-environments) and self-hosted runners can use this mechanism to add extra logging info to the `Set up job` step's log.
In addition to action's regular execution, action author may wants their action has a chance to participate in:
- Job initialize
My Action will collect machine resource usage (CPU/RAM/Disk) during a workflow job execution, we need to start perf recorder at the begin of the job.
- Job cleanup
In addition to action's regular execution, action author may wants their action to have a chance to participate in:
- Job initialization
My Action will collect machine resource usage (CPU/RAM/Disk) during a workflow job execution, we need to start perf recorder at the beginning of the job.
- Job cleanup
My Action will dirty local workspace or machine environment during execution, we need to cleanup these changes at the end of the job.
Ex: `actions/checkout@v2` will write `github.token` into local `.git/config` during execution, it has post job cleanup defined to undo the changes.
@@ -46,12 +46,12 @@ Container Action Example:
post-if:'success()'// Optional
```
Both `pre` and `post` will has default `pre-if/post-if` sets to `always()`.
Both `pre` and `post` will have default `pre-if/post-if` set to `always()`.
Setting `pre` to `always()` will make sure no matter what condition evaluate result the `main` gets at runtime, the `pre` has always run already.
`pre` executes in order of how the steps are defined.
`pre` will always be added to job steps list during job setup.
> Action referenced from local repository (`./my-action`) won't get `pre` setup correctly since the repository haven't checkout during job initialize.
> We can't use GitHub api to download the repository since there is a about 3 mins delay between `git push` and the new commit available to download using GitHub api.
> Action referenced from local repository (`./my-action`) won't get `pre` setup correctly since the repository haven't checked-out during job initialization.
> We can't use GitHub api to download the repository since there is about a 3 minute delay between `git push` and the new commit available to download using GitHub api.
`post` will be pushed into a `poststeps` stack lazily when the action's `pre` or `main` execution passed `if` condition check and about to run, you can't have an action that only contains a `post`, we will pop and run each `post` after all `pre` and `main` finished.
> Currently `post` works for both repository action (`org/repo@v1`) and local action (`./my-action`)
It would not be possible to remove labels from an existing runner using `config.sh`, instead labels would have to be removed using the GitHub UI.
The labels argument will split on commas, trim and discard empty strings. That effectively means don't use commans in unattended config label names. Alternatively we could choose to escape commans but it's a nice to have.
The labels argument will split on commas, trim and discard empty strings. That effectively means don't use commas in unattended config label names. Alternatively, we could choose to escape commas but it's a nice to have.
## Replace
If an existing runner exists and the option to replace is chosen (interactively of via unattend as in this scenario), then the labels will be replaced / overwritten (not merged).
If an existing runner exists and the option to replace is chosen (interactively or via unattended as in this scenario), then the labels will be replaced/overwritten (not merged).
## Overriding built-in labels
Note that it is possible to register "built-in" hosted labels like `ubuntu-latest` and is not considered an error. This is an effective way for the org / runner admin to dictate by policy through registration that this set of runners will be used without having to edit all the workflow files now and in the future.
Note that it is possible to register "built-in" hosted labels like `ubuntu-latest` and is not considered an error. This is an effective way for the org/runner admin to dictate by policy through registration that this set of runners will be used without having to edit all the workflow files now and in the future.
We will also not make other restrictions such as limiting explicitly adding os / arch labels and validating. We will assume that explicit labels were added for a reason and not restricting offers the most flexibility and futureproofing / compat.
We will also not make other restrictions such as limiting explicitly adding os/arch labels and validating. We will assume that explicit labels were added for a reason and not restricting offers the most flexibility and future-proofing / compatibility.
Customers want to be able to compose actions from actions (ex: https://github.com/actions/runner/issues/438)
An important step towards meeting this goal is to build in functionality for actions where users can simply execute any number of steps.
An important step towards meeting this goal is to build functionality for actions where users can simply execute any number of steps.
### Guiding Principles
We don't want the workflow author to need to know how the internal workings of the action work. Users shouldn't know the internal workings of the composite action (for example, `default.shell` and `default.workingDir` should not be inherited from the workflow file to the action file). When deciding how to design certain parts of composite run steps, we want to think one logical step from the consumer.
We don't want the workflow author to need to know how the internal workings of the action work. Users shouldn't know the internal workings of the composite action (for example, `default.shell` and `default.workingDir` should not be inherited from the workflow file to the action file). When deciding how to design certain parts of composite run steps, we want to treat it as one logical step for the consumer.
A composite action is treated as **one** individual job step (this is known as encapsulation).
## Decision
**In this ADR, we only support running multiple run steps in an Action.** In doing so, we build in support for mapping and flowing the inputs, outputs, and env variables (ex: All nested steps should have access to its parents' input variables and nested steps can overwrite the input variables).
**In this ADR, we only support running multiple run steps in an Action.** In doing so, we build in support for mapping and flowing the inputs, outputs, and env variables (ex: All nested steps should have access to their parents' input variables and nested steps can overwrite the input variables).
### Composite Run Steps Features
This feature supports at the top action level:
- name
- description
- inputs
- runs
- outputs
This feature supports at the run step level:
- name
- id
- run
- env
- shell
- working-directory
This feature **does not support** at the run step level:
- timeout-minutes
- secrets
- conditionals (needs, if, etc.)
- continue-on-error
### Steps
@@ -49,7 +70,9 @@ runs:
using:"composite"
steps:
- run:pip install -r requirements.txt
shell:bash
- run:npm install
shell:bash
```
Example Output
@@ -63,6 +86,69 @@ echo hello world 4
We add a token called "composite" which allows our Runner code to process composite actions. By invoking "using: composite", our Runner code then processes the "steps" attribute, converts this template code to a list of steps, and finally runs each run step sequentially. If any step fails and there are no `if` conditions defined, the whole composite action job fails.
### Defaults
We will not support "defaults" in a composite action.
### Shell and Working-directory
For each run step in a composite action, the action author can set the `shell` and `working-directory` attributes for that step. The shell attribute is **required** for each run step because the action author does not know what the workflow author is using for the operating system so we need to explicitly prevent unknown behavior by making sure that each run step has an explicit shell **set by the action author.** On the other hand, `working-directory` is optional. Moreover, the composite action author can map in values from the `inputs` for its `shell` and `working-directory` attributes at the step level for an action.
For example,
`action.yml`
```yaml
inputs:
shell_1:
description:'Your name'
default:'pwsh'
steps:
- run:echo 1
shell:${{ inputs.shell_1 }}
```
Note, the workflow file and action file are treated as separate entities. **So, the workflow `defaults` will never change the `shell` and `working-directory` value in the run steps in a composite action.** Note, `defaults` in a workflow only apply to run steps not "uses" steps (steps that use an action).
Users will be able to run scripts located in their action folder by first prepending the relative path and script name with `$GITHUB_ACTION_PATH` or `github.action_path` which contains the path in which the composite action is downloaded to and where those "files" live. Note, you'll have to use `chmod` before running each script if you do not git check in your script files into your github repo with the executable bit turned on.
Each of the output variables from the composite action is viewable from the workflow file that uses the composite action. In other words, every child action output(s) is viewable only by its parent using dot notation (ex `steps.foo.outputs.random-number`).
Each of the output variables from the composite action is viewable from the workflow file that uses the composite action. In other words, every child's action output(s) are only viewable by its parent using dot notation (ex `steps.foo.outputs.random-number`).
Moreover, the output ids are only accessible within the scope where it was defined. Note that in the example above, in our `workflow.yml` file, it should not have access to output id (i.e. `random-id`). The reason why we are doing this is because we don't want to require the workflow author to know the internal workings of the composite action.
Moreover, the output ids are only accessible within the scope where it was defined. Note that in the example above, in our `workflow.yml` file, it should not have access to output id (i.e. `random-id`). The reason why we are doing this is that we don't want to require the workflow author to know the internal workings of the composite action.
### Context
@@ -143,19 +232,23 @@ In the Composite Action, you'll only be able to use `::set-env::` to set environ
### Secrets
**Note** : This feature will be focused on in a future ADR.
**We will not support "Secrets" in a composite action for now. This functionality will be focused on in a future ADR.**
We'll pass the secrets from the composite action's parents (ex: the workflow file) to the composite action. Secrets can be created in the composite action with the secrets context. In the actions yaml, we'll automatically mask the secret.
### IfCondition
### If-Condition
**`If` and `needs` conditions will not be supported in the composite run steps feature. It will be supported later on in a new feature. **
Old reasoning:
Example `workflow.yml`:
```yaml
steps:
- run:exit 1
- uses:user/composite@v1 # <--- this will run, as it's marked as always runing
- uses:user/composite@v1 # <--- this will run, as it's marked as always running
if:always()
```
@@ -166,19 +259,25 @@ runs:
using:"composite"
steps:
- run:echo "just succeeding"
shell:bash
- run:echo "I will run, as my current scope is succeeding"
shell:bash
if:success()
- run:exit 1
shell:bash
- run:echo "I will not run, as my current scope is now failing"
shell:bash
```
**We will not support "if-condition" in a composite action for now. This functionality will be focused on in a future ADR.**
See the paragraph below for a rudimentary approach (thank you to @cybojenix for the idea, example, and explanation for this approach):
The `if` statement in the parent (in the example above, this is the `workflow.yml`) shows whether or not we should run the composite action. So, our composite action will run since the `if` condition for running the composite action is `always()`.
**Note that the ifcondition on the parent does not propagate to the rest of its children though.**
**Note that the "if-condition" on the parent does not propagate to the rest of its children though.**
In the child action (in this example, this is the `action.yml`), it starts with a clean slate (in other words, no imposing ifconditions). Similar to the logic in the paragraph above, `echo "I will run, as my current scope is succeeding"` will run since the `if` condition checks if the previous steps **within this composite action** has not failed. `run: echo "I will not run, as my current scope is now failing"` will not run since the previous step resulted in an error and by default, the if expression is set to `success()` if the ifcondition is not set for a step.
In the child action (in this example, this is the `action.yml`), it starts with a clean slate (in other words, no imposing if-conditions). Similar to the logic in the paragraph above, `echo "I will run, as my current scope is succeeding"` will run since the `if` condition checks if the previous steps **within this composite action** have not failed. `run: echo "I will not run, as my current scope is now failing"` will not run since the previous step resulted in an error and by default, the if expression is set to `success()` if the if-condition is not set for a step.
What if a step has `cancelled()`? We do the opposite of our approach above if `cancelled()` is used for any of our composite run steps. We will cancel any step that has this condition if the workflow is cancelled at all.
@@ -203,20 +302,25 @@ runs:
- id:foo1
run:echo test 1
timeout-minutes:10
shell:bash
- id:foo2
run:echo test 2
shell:bash
- id:foo3
run:echo test 3
timeout-minutes:10
shell:bash
```
A composite action in its entirety is a job. You can set both timeout-minutes for the whole composite action or its steps as long as the the sum of the `timeout-minutes` for each composite action step that has the attribute `timeout-minutes` is less than or equals to `timeout-minutes` for the composite action. There is no default timeout-minutes for each composite action step.
**We will not support "timeout-minutes" in a composite action for now. This functionality will be focused on in a future ADR.**
If the time taken for any of the steps in combination or individually exceed the whole composite action `timeout-minutes` attribute, the whole job will fail (1). If an individual step exceeds its own `timeout-minutes` attribute but the total time that has been used including this step is below the overall composite action`timeout-minutes`, the individual step will fail but therest of the steps will run based on their own `timeout-minutes` attribute (they will still abide by condition (1) though).
A composite action in its entirety is a job. You can set both timeout-minutes for the whole composite action or its steps as long as the sum of the `timeout-minutes` for each composite action step that has the attribute `timeout-minutes` is less than or equals to`timeout-minutes` for the composite action. There is no default timeout-minutes for each composite action step.
For reference, in the example above, if the composite step `foo1` takes 11 minutes to run, that step will fail but the rest of the steps, `foo1` and `foo2`, will proceed as long as their total runtime with the previous failed `foo1` action is less than the composite action's`timeout-minutes` (50 minutes). If the composite step `foo2` takes 51 minutes to run, it will cause the whole composite action job to fail. I
If the time taken for any of the steps in combination or individually exceeds the whole composite action `timeout-minutes` attribute, the whole job will fail (1). If an individual step exceeds its own `timeout-minutes` attribute but the total time that has been used including this step is below the overall composite action `timeout-minutes`, the individual step will fail but the rest of the steps will run based on their own `timeout-minutes` attribute (they will still abide by condition (1) though).
The rationale behind this is that users can configure their steps with the `if` condition to conditionally set how steps rely on each other. Due to the additional capabilities that are offered with combining`timeout-minutes`and/or `if`, we wanted the `timeout-minutes` condition to be as dumb as possible and not effect other steps.
For reference, in the example above, if the composite step `foo1` takes 11 minutes to run, that step will fail but the rest of the steps, `foo1` and `foo2`, will proceed as long as their total runtime with the previous failed `foo1` action is less than the composite action's`timeout-minutes`(50 minutes). If the composite step `foo2` takes 51 minutes to run, it will cause the whole composite action job to fail.
The rationale behind this is that users can configure their steps with the `if` condition to conditionally set how steps rely on each other. Due to the additional capabilities that are offered with combining `timeout-minutes` and/or `if`, we wanted the `timeout-minutes` condition to be as dumb as possible and not affect other steps.
[Usage limits still apply](https://help.github.com/en/actions/reference/workflow-syntax-for-github-actions?query=if%28%29#usage-limits)
@@ -243,40 +347,21 @@ runs:
steps:
- run:exit 1
continue-on-error:true
shell:bash
- run:echo "Hello World 2" <----- This step will run
shell:bash
```
**We will not support "continue-on-error" in a composite action for now. This functionality will be focused on in a future ADR.**
If any of the steps fail in the composite action and the `continue-on-error` is set to `false` for the whole composite action step in the workflow file, then the steps below it will run. On the flip side, if `continue-on-error` is set to `true` for the whole composite action step in the workflow file, the next job step will run.
For the composite action steps, it follows the same logic as above. In this example, `"Hello World 2"` will be outputted because the previous step has `continue-on-error` set to `true` although that previous step errored.
### Defaults
We will not support "defaults" in a composite action.
### Shell and Working-directory
For each run step in a composite action, the action author can set the `shell` and `working-directory` attributes for that step. These attributes are optional for each run step - by default, the `shell` is set to whatever default value is associated with the runner os (ex: bash =\> Mac). Moreover, the composite action author can map in values from the `inputs` for it's `shell` and `working-directory` attributes at the step level for an action.
For example,
`action.yml`
```yaml
inputs:
shell_1:
description:'Your name'
default:'pwsh'
steps:
- run:echo 1
shell:${{ inputs.shell_1 }}
```
Note, the workflow file and action file are treated as separate entities. **So, the workflow `defaults` will never change the `shell` and `working-directory` value in the run steps in a composite action.** Note, `defaults` in a workflow only apply to run steps not "uses" steps (steps that use an action).
### Visualizing Composite Action in the GitHub Actions UI
We want all the composite action's steps to be condensed into the original composite action node.
Here is a visual represenation of the [first example](#Steps)
Here is a visual representation of the [first example](#Steps)
We released [composite run steps](https://github.com/actions/runner/pull/554) last year which started our journey of reusing steps across different workflow files. To continue that journey, we want to expand composite run steps into composite actions.
We want to support the `uses` steps from workflows in composite actions, including:
- Container actions
- Javascript actions
- Other Composite actions (up to a limit of course!)
- The pre and post steps these actions can generate
## Guiding Principles
- Composite Actions should function as a single step or action, no matter how many steps it is composed of or how many levels of recursion it has
- In the future we may add a configurable option to make this no longer the case
- A workflow author should not need to understand the inner workings of a composite action in order to use it
- Composite actions should leverage inputs to get values they need, they will not have full access to the `context` objects. The secrets context will **not** be available to composite actions, users will need to pass these values in as an input.
- Other Actions should **just work** inside a composite action, without any code changes
## Decisions
### Composite Recursion Limit
- We will start with supporting a recursion limit of `10` composite actions deep
- We are free to bump this limit in the future, the code will be written to just require updating a variable. If the graph evaluates beyond the recursion limit, the job will fail in the pre-job phase (The `Set up job` step).
- A composite actions interface is its inputs and outputs, nothing else is carried over when invoking recursively.
### Pre/Post Steps in nested Actions
- We do not plan on adding the ability to configure a customizable pre or post step for composite actions at this time. However, we will execute the pre and post steps of any actions referenced in a composite action.
- Composite actions will generate a single pre-step and post-step for the entire composite action, even if there are multiple pre-steps and post-steps in the referenced actions.
- These steps will execute following the same ordering rules we have today, first to run has their pre step run first and their post step run last.
- For example, if you had a composite action with two pre steps and two posts steps:
- While the composite action has an individual combined pre/post action, the `set-state` command will not be shared.
- If the `set-state` command is used during a composite step, only the action that originally called `set-state` will have access to the env variable during the post run step.
- This prevents multiple actions that set the same state from interfering with the execution of another action's post step.
### Resolve Action Endpoint changes
- The resolve actions endpoint will now validate policy to ensure that the given workflow run has access to download that action.
- Older GHES/GHAE customers with newer runners will be locked out of composite uses steps until they upgrade their instance.
### Local actions
- Local actions will expand the tree, perform policy checks, and download actions Just in Time when the step is running.
- Like current local actions, we will not support presteps. If an action is running local, by the time we know that, the time to run presteps have already passed.
### If, continue-on-error, timeout-minutes - Not being considered at this time
- `if`, `continue-on-error`, `timeout-minutes` could be supported in composite run/uses steps. These values were not originally supported in our composite run steps implementation.
- Browsing the community forums and runner repo, there hasn't been a lot of noise asking for these features, so we will hold off on them.
- These values passed as input into the composite action will **not** be carried over as input into the individual steps the composite action runs.
### Defaults - Not being considered at this time
- In actions, we have the idea of [defaults](https://docs.github.com/en/actions/reference/workflow-syntax-for-github-actions#defaultsrun) , which allow you to specify a shell and working directory in one location, rather then on each step.
- However, `shell` is currently required in composite run steps
- In regular run steps, it is optional, and defaults to a different value based on the OS.
- We want to prioritize the right experience for the consumer, and make the action author continue to explicitly set these values. We can consider improving this experience in the future.
## Consequences
- Workflows are now more reusable across multiple workflow files
- Composite actions implement most of the existing workflow run steps, with room to expand these in the future
# ADR 1438: Support Conditionals In Composite Actions
**Date**: 2021-10-13
**Status**: Accepted
## Context
We recently shipped composite actions, which allows you to reuse individual steps inside an action.
However, one of the [most requested features](https://github.com/actions/runner/issues/834) has been a way to support the `if` keyword.
### Goals
- We want to keep consistent with current behavior
- We want to support conditionals via the `if` keyword
- Our built in functions like `success` should be implementable without calling them, for example you can do `job.status == success` rather then `success()` currently.
### How does composite currently work?
Currently, we have limited conditional support in composite actions for `pre` and `post` steps.
These are based on the `job status`, and support keywords like `always()`, `failed()`, `success()` and `cancelled()`.
However, generic or main steps do **not** support conditionals.
By default, in a regular workflow, a step runs on the `success()` condition. Which looks at the **job****status**, sees if it is successful and runs.
By default, in a composite action, main steps run until a single step fails in that composite action, then the composite action is halted early. It does **not** care about the job status.
Pre, and post steps in composite actions use the job status to determine if they should run.
### How do we go forward?
Well, if we think about what composite actions are currently doing when invoking main steps, they are checking if the current composite action is successful.
Lets formalize that concept into a "real" idea.
- We will add an `action_status` field to the github context to mimic the [job's context status](https://docs.github.com/en/actions/learn-github-actions/contexts#job-context).
- We have an existing concept that does this `action_path` which is only set for composite actions on the github context.
- In a composite action during a main step, the `success()` function will check if `action_status == success`, rather then `job_status == success`. Failure will work the same way.
- Pre and post steps in composite actions will not change, they will continue to check the job status.
### Nested Scenario
For nested composite actions, we will follow the existing behavior, you only care about your current composite action, not any parents.
For example, lets imagine a scenario with a simple nested composite action
```
- Job
- Regular Step
- Composite Action
- runs: exit 1
- if: always()
uses: A child composite action
- if: success()
runs: echo "this should print"
- runs: echo "this should also print"
- if: success()
runs: echo "this will not print as the current composite action has failed already"
```
The child composite actions steps should run in this example, the child composite action has not yet failed, so it should run all steps until a step fails. This is consistent with how a composite action currently works in production if the main job fails but a composite action is invoked with `if:always()` or `if: failure()`
### Other options explored
We could add the `current_step_status` to the job context rather then `__status` to the steps context, however this comes with two major downsides:
- We need to support the field for every type of step, because its non trivial to remove a field from the job context once it has been added (its readonly)
- For all actions besides composite it would only every be `success`
- Its weird to have a `current_step` value on the job context
- We also explored a `__status` on the steps context.
- The `__` is required to prevent us from colliding with a step with id: status
- This felt wrong because the naming was not smooth, and did not fit into current conventions.
### Consequences
- github context has a new field for the status of the current composite action.
- We support conditional's in composite actions
- We keep the existing behavior for all users, but allow them to expand that functionality.
This ADR details the design changes for supporting custom configurable hooks for on various runner events. This has been a long requested user feature [here](https://github.com/actions/runner/issues/1543), [here](https://github.com/actions/runner/issues/699) and [here](https://github.com/actions/runner/issues/1116) for users to have more information on runner observability, and for the ability to run cleanup and teardown jobs.
This feature is mainly intended for self hosted runner administrators.
**What we hope to solve with this feature**
1. A runner admininstrator is able to add custom scripts to cleanup their runner environment at the start or end of a job
2. A runner admininstrator is able to add custom scripts to help setup their runner environment at the beginning of a job, for reasons like [caching](https://github.com/actions/runner/issues/1543#issuecomment-1050346279)
3. A runner administrator is able to grab custom telemetry of jobs running on their self hosted runner
**What we don't think this will solve**
- Policy features that require certain steps run at the beginning or end of all jobs
- This would be better solved to in a central place in settings, rather then decentralized on each runner.
- The Proposed `Notification Hooks for Runners` is limited to self hosted runners, we don't beileve Policy features should be
- Reuse scenarios between jobs are covered by [composite actions](https://docs.github.com/en/actions/creating-actions/creating-a-composite-action) and [resuable workflows](https://docs.github.com/en/actions/using-workflows/reusing-workflows)
- Security applications, security should be handled on the policy side on the server, not decentralized on each runner
## Hooks
- We will expose 2 variables that users can set to enable hooks
-`ACTIONS_RUNNER_HOOK_JOB_STARTED`
-`ACTIONS_RUNNER_HOOK_JOB_COMPLETED`
You can set these variables to the **absolute** path of a `.sh` or `.ps1` file.
We will execute `pwsh` (fallback to `powershell`) or `bash` (fallback to `sh`) as appropriate.
-`.sh` files will execute with the args `-e {pathtofile}`
-`.ps1` files will execute with the args `-command \". '{pathtofile}'\"`
We will **not** set the [standard flags we typically set](https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#jobsjob_idstepsshell) for `runs` commands. So, if you want to set `pipefail` on `bash` for example, you will need to do that in your script.
### UI
We want to ensure the experience for users invoking workflows is good, if hooks take too long, you may feel your job is delayed or broken. So, much like `Set Up Job`, we will generate two new steps automatically in your job, one for each configured hook:
-`Set up runner`
-`Complete runner`
These steps will contain all of the output from invoking your hook, so you will have visibility into the runtime. We will also provide information on the path to the hook, and what shell we are invoking it as, much like we do for `run: ` steps.
### Contexts
When running your hooks, some context on your job may be helpful.
- The scripts will have access to the standard [default environment variables](https://docs.github.com/en/actions/learn-github-actions/environment-variables#default-environment-variables)
- Some of these variables are step specific like `GITHUB_ACTION`, in which case they will not be set
- You can pull the full webhook event payload from `GITHUB_EVENT_PATH`
### Commands
Should we expose [Commands](https://docs.github.com/en/actions/using-workflows/workflow-commands-for-github-actions) and [Environment Files](https://docs.github.com/en/actions/using-workflows/workflow-commands-for-github-actions#environment-files)
**Yes**. Imagine a scenario where a runner administrator is deprecating a runner pool, and they need to [warn users](https://docs.github.com/en/actions/using-workflows/workflow-commands-for-github-actions#setting-a-warning-message) to swap to a different pool, we should support them in doing this. However, there are some limitations:
- [save-state](https://docs.github.com/en/actions/using-workflows/workflow-commands-for-github-actions#sending-values-to-the-pre-and-post-actions) will **not** be supported, these are not traditional steps with pre and post actions
- [set-output](https://docs.github.com/en/actions/using-workflows/workflow-commands-for-github-actions#using-workflow-commands-to-access-toolkit-functions) will **not** be supported, there is no `id` as this is not a traditional step
### Environment Files
We will also enable [Environment Files](https://docs.github.com/en/actions/using-workflows/workflow-commands-for-github-actions#environment-files) to support setup scenarios for the runner environment.
While a self hosted runner admin can [set env variables](https://docs.github.com/en/actions/hosting-your-own-runners/using-a-proxy-server-with-self-hosted-runners#using-a-env-file-to-set-the-proxy-configuration), these apply to all jobs. By enabling the ability to `add a path` and `set an env` we give runner admins the ability to do this dynamically based on the [workflows environment variables](https://docs.github.com/en/actions/learn-github-actions/environment-variables#default-environment-variables) to empower setup scenarios.
### Exit codes
These are **synchronous** hooks, so they will block job execution while they are being run. Exit code 0 will indicate a successful run of the hook and we will proceed with the job, any other exit code will fail the job with an appropriate annotation.
- There will be no support for `continue-on-error`
## Key Decisions
- We will expose 2 variables that users can set to enable hooks
-`ACTIONS_RUNNER_HOOK_JOB_STARTED`
-`ACTIONS_RUNNER_HOOK_JOB_COMPLETED`
- Users can set these variables to the path of a `.sh` or `.ps1` file, which we will execute when Jobs are started or completed.
- Output from these will be added to a new step at the start/end of a job named `Set up runner` or `Complete runner`.
- These steps will only be generated on runs with these hooks
- These hooks `always()` execute if the env variable is set
- These files will execute as the Runner user, outside of any container specification on the job
- These are **synchronous** hooks
- Runner admins can execute a background process for async hooks if they want
- We will fail the job and halt execution on any exit code that is not 0. The Runner admin is responsible for returning the correct exit code and ensuring resilency.
- This includes that the runner user needs access to the file in the env and the file must exist
- There will be no `continue-on-error` type option on launch
- There will be no `timeout` option on launch
## Consequences
- Runner admins have the ability to tie into the runner job execution to publish their own telemetry or perform their own cleanup or setup
- New steps will be added to the UI showcasing the output of these hooks
[Job Hooks](https://github.com/actions/runner/blob/main/docs/adrs/1751-runner-job-hooks.md) have given users the ability to customize how their self hosted runners run a job.
Users also want the ability to customize how they run containers during the scope of the job, rather then being locked into the docker implementation we have in the runner. They may want to use podman, kubernetes, or even change the docker commands we run.
We should give them that option, and publish examples how how they can create their own hooks.
# Guiding Principles
- **Extensibility** is the focus, we need to make sure we are flexible enough to cover current and future scenarios, even at the cost of making it harder to utilize these hooks
- Args should map **directly** to yaml values provided by the user.
- For example, the current runner overrides `HOME`, we can do that in the hook, but we shouldn't pass that hook as an ENV with the other env's the user has set, as that is not user input, it is how the runner invokes containers
## Interface
- You will set the variable `ACTIONS_RUNNER_CONTAINER_HOOKS=/Users/foo/runner/hooks.js` which is the entrypoint to your hook handler.
- There is no partial opt in, you must handle every hook
- We will pass a command and some args via `stdin`
- An exit code of 0 is a success, every other exit code is a failure
- We will support the same runner commands we support in [Job Hooks](https://github.com/actions/runner/blob/main/docs/adrs/1751-runner-job-hooks.md)
- On timeout, we will send a sigint to your process. If you fail to terminate within a reasonable amount of time, we will send a sigkill, and eventually kill the process tree.
`responseFile` is the file you need to write your output to, if the command has output
`args` are the specific arguments the command needs
`state` is a json blog you can pass around to maintain your state, this is covered in more details below.
### Writing responses to a file
All text written to stdout or stderr should appear in the job or step logs. With that in mind, we support a few ways to actually return data:
1. Wrapping the json in some unique tag and processing it like we do commands
2. Writing to a file
For 1, users typically view logging information as a safe action, so we worry someone accidentialy logging unsantized information and causing unexpected or un-secure behavior. We eventually plan to move off of stdout/stderr style commands in favor of a runner cli.
Investing in this area doesn't make a lot of sense at this time.
While writing to a file to communicate isn't the most ideal pattern, its an existing pattern in the runner and serves us well, so lets reuse it.
### Output
Your output must be correctly formatted json. An example output looks like:
`state` is a unique field any command can return. If it is not empty, we will store the state for you and pass it into all future commands. You can overwrite it by having the next hook invoked return a unique state.
Other fields are dependent upon the command being run.
### Versioning
We will not version these hooks at launch. If needed, we can always major version split these hooks in the future. We will ship in Beta to allow for breaking changes for a few months.
### The Job Context
The [job context](https://docs.github.com/en/actions/learn-github-actions/contexts#example-contents-of-the-job-context) currently has a variety of fields that correspond to containers. We should consider allowing hooks to populate new fields in the job context. That is out of scope for this original release however.
## Hooks
Hooks are to be implemented at a very high level, and map to actions the runner does, rather then specific docker actions like `docker build` or `docker create`. By mapping to runner actions, we create a very extensible framework that is flexible enough to solve any user concerns in the future. By providing first party implementations, we give users easy starting points to customize specific hooks (like `docker build`) without having to write full blown solutions.
The other would be to provide hooks that mirror every docker call we make, and expose more hooks to help support k8s users, with the expectation that users may have to no-op on multiple hooks if they don't correspond to our use case.
Why we don't want to go that way
- It feels clunky, users need to understand which hooks they need to implement and which they can ignore, which isn't a great UX
- It doesn't scale well, I don't want to build a solution where we may need to add more hooks, by mapping to runner actions, updating hooks is a painful experience for users
- Its overwhelming, its easier to tell users to build 4 hooks and track data themselves, rather then 16 hooks where the runner needs certain information and then needs to provide that information back into each hook. If we expose `Container Create`, you need to return the container you created, then we do `container run` which uses that container. If we just give you an image and say create and run this container, you don't need to store the container id in the runner, and it maps better to k8s scenarios where we don't really have container ids.
### Prepare_job hook
The `prepare_job` hook is called when a job is started. We pass in any job or service containers the job has. We expect that you:
- Prune anything from previous jobs if needed
- Create a network if needed
- Pull the job and service containers
- Start the job container
- Start the service containers
- Write to the response file some information we need
- Required: if the container is alpine, otherwise x64
- Optional: any context fields you want to set on the job context, otherwise they will be unavailable for users to use
- Return 0 when the health checks have succeeded and the job/service containers are started
This hook will **always** be called if you have container hooks enabled, even if no service or job containers exist in the job. This allows you to fail the job or implement a default job container if you want to and no job container has been provided.
jobContainer: **Optional** An Object containing information about the specified job container
"image": **Required** A string containing the docker image
"workingDirectory": **Required** A string containing the absolute path of the working directory
"createOptions": **Optional** The optional create options specified in the [YAML](https://docs.github.com/en/actions/using-jobs/running-jobs-in-a-container#example-running-a-job-within-a-container)
"environmentVariables": **Optional** A map of key value env's to set
"userMountVolumes: ** Optional** an array of user mount volumes set in the [YAML](https://docs.github.com/en/actions/using-jobs/running-jobs-in-a-container#example-running-a-job-within-a-container)
"sourceVolumePath": **Required** The source path to the volume to be mounted into the docker container
"targetVolumePath": **Required** The target path to the volume to be mounted into the docker container
"readOnly": false **Required** whether or not the mount should be read only
"mountVolumes": **Required** an array of mounts to mount into the container, same fields as above
"sourceVolumePath": **Required** The source path to the volume to be mounted into the docker container
"targetVolumePath": **Required** The target path to the volume to be mounted into the docker container
"readOnly": false **Required** whether or not the mount should be read only
"registry" **Optional** docker registry credentials to use when using a private container registry
"username": **Optional** the username
"password": **Optional** the password
"serverUrl": **Optional** the registry url
"portMappings": **Optional** an array of source:target ports to map into the container
"services": an array of service containers to spin up
"contextName": **Required** the name of the service in the Job context
"image": **Required** A string containing the docker image
"createOptions": **Optional** The optional create options specified in the [YAML](https://docs.github.com/en/actions/using-jobs/running-jobs-in-a-container#example-running-a-job-within-a-container)
"environmentVariables": **Optional** A map of key value env's to set
"mountVolumes": **Required** an array of mounts to mount into the container, same fields as above
"sourceVolumePath": **Required** The source path to the volume to be mounted into the docker container
"targetVolumePath": **Required** The target path to the volume to be mounted into the docker container
"readOnly": false **Required** whether or not the mount should be read only
"registry" **Optional** docker registry credentials to use when using a private container registry
"username": **Optional** the username
"password": **Optional** the password
"serverUrl": **Optional** the registry url
"portMappings": **Optional** an array of source:target ports to map into the container
"image": **Optional** A string containing the docker image. Otherwise a dockerfile must be provided
"dockerfile": **Optional** A string containing the path to the dockerfile, otherwise an image must be provided
"entryPointArgs": **Optional** A list containing the entry point args
"entryPoint": **Optional** The container entry point to use if the default image entrypoint should be overwritten
"workingDirectory": **Required** A string containing the absolute path of the working directory
"createOptions": **Optional** The optional create options specified in the [YAML](https://docs.github.com/en/actions/using-jobs/running-jobs-in-a-container#example-running-a-job-within-a-container)
"environmentVariables": **Optional** A map of key value env's to set
"prependPath": **Optional** an array of additional paths to prepend to the $PATH variable
"userMountVolumes: ** Optional** an array of user mount volumes set in the [YAML](https://docs.github.com/en/actions/using-jobs/running-jobs-in-a-container#example-running-a-job-within-a-container)
"sourceVolumePath": **Required** The source path to the volume to be mounted into the docker container
"targetVolumePath": **Required** The target path to the volume to be mounted into the docker container
"readOnly": false **Required** whether or not the mount should be read only
"mountVolumes": **Required** an array of mounts to mount into the container, same fields as above
"sourceVolumePath": **Required** The source path to the volume to be mounted into the docker container
"targetVolumePath": **Required** The target path to the volume to be mounted into the docker container
"readOnly": false **Required** whether or not the mount should be read only
"registry" **Optional** docker registry credentials to use when using a private container registry
"username": **Optional** the username
"password": **Optional** the password
"serverUrl": **Optional** the registry url
"portMappings": **Optional** an array of source:target ports to map into the container
```
</details>
No output is expected
Currently we build all container actions at the start of the job. By doing it during the hook, we move this to just in time building for hooks. We could expose a hook to build/pull a container action, and have those called at the start of a job, but doing so would require hook authors to track the build containers in the state, which could be painful.
### Run Script Step
The `run_script_step` expects you to:
- Invoke the provided script inside the job container and return the exit code
Following the [adoption of actions-runner-controller by GitHub](https://github.com/actions/actions-runner-controller/discussions/2072) and the introduction of the new runner scale set autoscaling mode, we needed to provide a basic runner image that could be used off the shelf without much friction.
The [current runner image](https://github.com/actions/runner/pkgs/container/actions-runner) is published to GHCR. Each release of this image is tagged with the runner version and the most recent release is also tagged with `latest`.
While the use of `latest` is common practice, we recommend that users pin a specific version of the runner image for a predictable runtime and improved security posture. However, we still notice that a large number of end users are relying on the `latest` tag & raising issues when they encounter problems.
Add to that, the community actions-runner-controller maintainers have issued a [deprecation notice](https://github.com/actions/actions-runner-controller/issues/2056) of the `latest` tag for the existing runner images (https://github.com/orgs/actions-runner-controller/packages).
## Decision
Proceed with Option 2, keeping the `latest` tag and adding the `NOTES.txt` file to our helm charts with the notice.
### Option 1: Remove the `latest` tag
By removing the `latest` tag, we have to proceed with either of these options:
1. Remove the runner image reference in the `values.yaml` provided with the `gha-runner-scale-set` helm chart and mark these fields as required so that users have to explicitly specify a runner image and a specific tag. This will obviously introduce more friction for users who want to start using actions-runner-controller for the first time.
```yaml
spec:
containers:
- name:runner
image:""
tag:""
command:["/home/runner/run.sh"]
```
1. Pin a specific runner image tag in the `values.yaml` provided with the `gha-runner-scale-set` helm chart. This will reduce friction for users who want to start using actions-runner-controller for the first time but will require us to update the `values.yaml` with every new runner release.
```yaml
spec:
containers:
- name:runner
image:"ghcr.io/actions/actions-runner"
tag:"v2.300.0"
command:["/home/runner/run.sh"]
```
### Option 2: Keep the `latest` tag
Keeping the `latest` tag is also a reasonable option especially if we don't expect to make any breaking changes to the runner image. We could enhance this by adding a [NOTES.txt](https://helm.sh/docs/chart_template_guide/notes_files/) to the helm chart which will be displayed to the user after a successful helm install/upgrade. This will help users understand the implications of using the `latest` tag and how to pin a specific version of the runner image.
The runner image release workflow will need to be updated so that the image is pushed to GHCR and tagged only when the runner rollout has reached all scale units.
## Consequences
Proceeding with **option 1** means:
1. We will enhance the runtime predictability and security posture of our end users
1. We will have to update the `values.yaml` with every new runner release (that can be automated)
1. We will introduce friction for users who want to start using actions-runner-controller for the first time
Proceeding with **option 2** means:
1. We will have to continue to maintain the `latest` tag
1. We will assume that end users will be able to handle the implications of using the `latest` tag
1. Runner image release workflow needs to be updated
Make sure the runner has access to actions service for GitHub.com or GitHub Enterprise Server
- For GitHub.com
- The runner needs to access `https://api.github.com` for downloading actions.
- The runner needs to access `https://codeload.github.com` for downloading actions tar.gz/zip.
- The runner needs to access `https://vstoken.actions.githubusercontent.com/_apis/.../` for requesting an access token.
- The runner needs to access `https://pipelines.actions.githubusercontent.com/_apis/.../` for receiving workflow jobs.
- The runner needs to access `https://results-receiver.actions.githubusercontent.com/.../` for reporting progress and uploading logs during a workflow job execution.
---
**NOTE:** for the full list of domains that are required to be in the firewall allow list refer to the [GitHub self-hosted runners requirements documentation](https://docs.github.com/en/actions/hosting-your-own-runners/managing-self-hosted-runners/about-self-hosted-runners#communication-between-self-hosted-runners-and-github).
These can by tested by running the following `curl` commands from your self-hosted runner machine:
- The runner needs to access `https://[hostname]/api/v3` for downloading actions.
- The runner needs to access `https://codeload.[hostname]/_ping` for downloading actions tar.gz/zip.
- The runner needs to access `https://[hostname]/_services/vstoken/_apis/.../` for requesting an access token.
- The runner needs to access `https://[hostname]/_services/pipelines/_apis/.../` for receiving workflow jobs.
These can by tested by running the following `curl` commands from your self-hosted runner machine, replacing `[hostname]` with the hostname of your appliance, for instance `github.example.com`:
A common cause of this these connectivity issues is if your to GitHub Enterprise Server appliance is using [the self-signed certificate that is enabled the first time](https://docs.github.com/en/enterprise-server/admin/configuration/configuring-network-settings/configuring-tls) your appliance is started. As self-signed certificates are not trusted by web browsers and Git clients, these clients (including the GitHub Actions runner) will report certificate warnings.
We recommend [upload a certificate signed by a trusted authority](https://docs.github.com/en/enterprise-server/admin/configuration/configuring-network-settings/configuring-tls) to GitHub Enterprise Server, or enabling the built-in ][Let's Encrypt support](https://docs.github.com/en/enterprise-server/admin/configuration/configuring-network-settings/configuring-tls).
## What is checked?
- DNS lookup for api.github.com or myGHES.com using dotnet
- Ping api.github.com or myGHES.com using dotnet
- Make HTTP GET to https://api.github.com or https://myGHES.com/api/v3 using dotnet, check response headers contains `X-GitHub-Request-Id`
---
- DNS lookup for codeload.github.com or codeload.myGHES.com using dotnet
- Ping codeload.github.com or codeload.myGHES.com using dotnet
- Make HTTP GET to https://codeload.github.com/_ping or https://codeload.myGHES.com/_ping using dotnet, check response headers contains `X-GitHub-Request-Id`
---
- DNS lookup for vstoken.actions.githubusercontent.com using dotnet
- Ping vstoken.actions.githubusercontent.com using dotnet
- Make HTTP GET to https://vstoken.actions.githubusercontent.com/_apis/health or https://myGHES.com/_services/vstoken/_apis/health using dotnet, check response headers contains `x-vss-e2eid`
---
- DNS lookup for pipelines.actions.githubusercontent.com using dotnet
- Ping pipelines.actions.githubusercontent.com using dotnet
- Make HTTP GET to https://pipelines.actions.githubusercontent.com/_apis/health or https://myGHES.com/_services/pipelines/_apis/health using dotnet, check response headers contains `x-vss-e2eid`
- Make HTTP POST to https://pipelines.actions.githubusercontent.com/_apis/health or https://myGHES.com/_services/pipelines/_apis/health using dotnet, check response headers contains `x-vss-e2eid`
---
- DNS lookup for results-receiver.actions.githubusercontent.com using dotnet
- Ping results-receiver.actions.githubusercontent.com using dotnet
- Make HTTP GET to https://results-receiver.actions.githubusercontent.com/health using dotnet, check response headers contains `X-GitHub-Request-Id`
## How to fix the issue?
### 1. Check the common network issue
> Please check the [network doc](./network.md)
### 2. SSL certificate related issue
If you are seeing `System.Net.Http.HttpRequestException: The SSL connection could not be established, see inner exception.` in the log, it means the runner can't connect to Actions service due to SSL handshake failure.
> Please check the [SSL cert doc](./sslcert.md)
## Still not working?
Contact [GitHub Support](https://support.github.com) if you have further questuons, or log an issue at https://github.com/actions/runner if you think it's a runner issue.
Make sure `git` can access GitHub.com or your GitHub Enterprise Server.
## What is checked?
The test is done by executing
```bash
# For GitHub.com
git ls-remote --exit-code https://github.com/actions/checkout HEAD
# For GitHub Enterprise Server
git ls-remote --exit-code https://ghes.me/actions/checkout HEAD
```
The test also set environment variable `GIT_TRACE=1` and `GIT_CURL_VERBOSE=1` before running `git ls-remote`, this will make `git` to produce debug log for better debug any potential issues.
## How to fix the issue?
### 1. Check global and system git config
If you are having issues connecting to the server, check your global and system git config for any unexpected authentication headers. You might be seeing an error like:
```
fatal: unable to access 'https://github.com/actions/checkout/': The requested URL returned error: 400
```
The following commands can be used to check for unexpected authentication headers:
Make sure the runner has access to https://api.github.com
The runner needs to access https://api.github.com to download any actions from the marketplace.
Even the runner is configured to GitHub Enterprise Server, the runner can still download actions from GitHub.com with [GitHub Connect](https://docs.github.com/en/enterprise-server@2.22/admin/github-actions/enabling-automatic-access-to-githubcom-actions-using-github-connect)
## What is checked?
- DNS lookup for api.github.com using dotnet
- Ping api.github.com using dotnet
- Make HTTP GET to https://api.github.com using dotnet, check response headers contains `X-GitHub-Request-Id`
## How to fix the issue?
### 1. Check the common network issue
> Please check the [network doc](./network.md)
## Still not working?
Contact GitHub customer service or log an issue at https://github.com/actions/runner if you think it's a runner issue.
### Common things that can cause the runner to not working properly
- A bug in the runner or the dotnet framework that causes the actions runner to be unable to make Http requests in a certain network environment.
- A Proxy or Firewall may block certain HTTP method, such as blocking all POST and PUT calls which the runner will use to upload logs.
- A Proxy or Firewall may only allows requests with certain user-agent to pass through and the actions runner user-agent is not in the allow list.
- A Proxy try to decrypt and exam HTTPS traffic for security purpose but cause the actions-runner to fail to finish SSL handshake due to the lack of trusting proxy's CA.
- The SSL handshake may fail if the client and server do not support the same TLS version, or the same cipher suites.
- A Proxy may try to modify the HTTPS request (like add or change some http headers) and causes the request become incompatible with the Actions Service (ASP.NetCore), Ex: [Nginx](https://github.com/dotnet/aspnetcore/issues/17081)
- Firewall rules that block action runner from accessing [certain hosts](https://docs.github.com/en/actions/hosting-your-own-runners/managing-self-hosted-runners/about-self-hosted-runners#communication-between-self-hosted-runners-and-github), ex: `*.github.com`, `*.actions.githubusercontent.com`, etc
### Identify and solve these problems
The key is to figure out where is the problem, the network environment, or the actions runner?
Use a 3rd party tool to make the same requests as the runner did would be a good start point.
- Use `nslookup` to check DNS
- Use `ping` to check Ping
- Use `traceroute`, `tracepath`, or `tracert` to check the network route between the runner and the Actions service
- Use `curl -v` to check the network stack, good for verifying default certificate/proxy settings.
- Use `Invoke-WebRequest` from `pwsh` (`PowerShell Core`) to check the dotnet network stack, good for verifying bugs in the dotnet framework.
If the 3rd party tool is also experiencing the same error as the runner does, then you might want to contact your network administrator for help.
Otherwise, contact GitHub customer support or log an issue at https://github.com/actions/runner
### Troubleshooting: Why can't I configure a runner?
If you are having trouble connecting, try these steps:
1. Validate you can reach our endpoints from your web browser. If not, double check your local network connection
2. Validate you can reach those endpoints in powershell core
- The runner runs on .net core, lets validate the local settings for that stack
- Open up `pwsh`
- Run the command using the urls above `Invoke-WebRequest {url}`
3. If not, get a packet trace using a tool like wireshark and start looking at the TLS handshake.
- If you see a Client Hello followed by a Server RST:
- You may need to configure your TLS settings to use the correct version
- You should support TLS version 1.2 or later
- You may need to configure your TLS settings to have up to date cipher suites, this may be solved by system updates and patches.
- Most notably, on windows server 2012 make sure [the tls cipher suite update](https://support.microsoft.com/en-us/topic/update-adds-new-tls-cipher-suites-and-changes-cipher-suite-priorities-in-windows-8-1-and-windows-server-2012-r2-8e395e43-c8ef-27d8-b60c-0fc57d526d94) is installed
- Your firewall, proxy or network configuration may be blocking the connection
- You will want to reach out to whoever is in charge of your network with these pcap files to further troubleshoot
Make sure the built-in node.js has access to GitHub.com or GitHub Enterprise Server.
The runner carries its own copies of node.js executables under `<runner_root>/externals/node20/` and `<runner_root>/externals/node24/`.
All javascript base Actions will get executed by the built-in `node` at either `<runner_root>/externals/node20/` or `<runner_root>/externals/node24/` depending on the version specified in the action's metadata.
> Not the `node` from `$PATH`
## What is checked?
- Make HTTPS GET to https://api.github.com or https://myGHES.com/api/v3 using node.js, make sure it gets 200 response code.
## How to fix the issue?
### 1. Check the common network issue
> Please check the [network doc](./network.md)
### 2. SSL certificate related issue
If you are seeing `Https request failed due to SSL cert issue` in the log, it means the `node.js` can't connect to the GitHub server due to SSL handshake failure.
> Please check the [SSL cert doc](./sslcert.md)
## Still not working?
Contact GitHub customer service or log an issue at https://github.com/actions/runner if you think it's a runner issue.
You might run into an SSL certificate error when your GitHub Enterprise Server is using a self-signed SSL server certificate or a web proxy within your network is decrypting HTTPS traffic for a security audit.
As long as your certificate is generated properly, most of the issues should be fixed after your trust the certificate properly on the runner machine.
> Different OS might have extra requirements on SSL certificate,
Depends on how your SSL server certificate gets configured, you might need to download the whole certificate chain from a machine that has trusted the SSL certificate's CA.
- Approach 1: Download certificate chain using a browser (Chrome, Firefox, IT), you can google for more example, [here is what I found](https://medium.com/@menakajain/export-download-ssl-certificate-from-server-site-url-bcfc41ea46a2)
- Approach 2: Download certificate chain using OpenSSL, you can google for more example, [here is what I found](https://superuser.com/a/176721)
- Approach 3: Ask your network administrator or the owner of the CA certificate to send you a copy of it
### Trust CA certificate for the Runner
The actions runner is a dotnet core application which will follow how dotnet load SSL CA certificates on each OS.
You can get full details documentation at [here](https://docs.microsoft.com/en-us/dotnet/standard/security/cross-platform-cryptography#x509store)
In short:
- Windows: Load from Windows certificate store.
- Linux: Load from OpenSSL CA cert bundle.
- macOS: Load from macOS KeyChain.
To let the runner trusts your CA certificate, you will need to:
1. Save your SSL certificate chain which includes the root CA and all intermediate CAs into a `.pem` file.
2. Use `OpenSSL` to convert `.pem` file to a proper format for different OS, here is some [doc with sample commands](https://www.sslshopper.com/ssl-converter.html)
3. Google search: "trust ca certificate on [linux distribution]"
4. If all approaches failed, set environment variable `SSL_CERT_FILE` to the CA bundle `.pem` file we get.
> To verify cert gets installed properly on Linux, you can try use `curl -v https://sitewithsslissue.com` and `pwsh -Command \"Invoke-WebRequest -Uri https://sitewithsslissue.com\"`
### Trust CA certificate for Git CLI
Git uses various CA bundle file depends on your operation system.
- Git packaged the CA bundle file within the Git installation on Windows
- Git use OpenSSL certificate CA bundle file on Linux and macOS
You can check where Git check CA file by running:
```bash
exportGIT_CURL_VERBOSE=1
git ls-remote https://github.com/actions/runner HEAD
```
You should see something like:
```
* Couldn't find host github.com in the .netrc file; using defaults
* Trying 140.82.114.4...
* TCP_NODELAY set
* Connected to github.com (140.82.114.4) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
* CAfile: /etc/ssl/cert.pem
CApath: none
* SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256
```
This tells me `/etc/ssl/cert.pem` is where it read trusted CA certificates.
To let Git trusts your CA certificate, you will need to:
1. Save your SSL certificate chain which includes the root CA and all intermediate CAs into a `.pem` file.
2. Set `http.sslCAInfo` Git config or `GIT_SSL_CAINFO` environment variable to the full path of the `.pem` file [Git Doc](https://git-scm.com/docs/git-config#Documentation/git-config.txt-httpsslCAInfo)
> I would recommend using `http.sslCAInfo` since it can be scope to certain hosts that need the extra trusted CA.
> This will make Git use the `/extra/ca/cert.pem` only when communicates with `https://myghes.com` and keep using the default CA bundle with others.
### Trust CA certificate for Node.js
Node.js has compiled a snapshot of the Mozilla CA store that is fixed at each version of Node.js' release time.
To let Node.js trusts your CA certificate, you will need to:
1. Save your SSL certificate chain which includes the root CA and all intermediate CAs into a `.pem` file.
2. Set environment variable `NODE_EXTRA_CA_CERTS` which point to the file. ex: `export NODE_EXTRA_CA_CERTS=/full/path/to/cacert.pem` or `set NODE_EXTRA_CA_CERTS=C:\full\path\to\cacert.pem`
We welcome contributions in the form of issues and pull requests. We view the contributions and the process as the same for github and external contributors.
We welcome contributions in the form of issues and pull requests. We view the contributions and the process as the same for github and external contributors. Please note the runner typically requires changes across the entire system and we aim for issues in the runner to be entirely self contained and fixable here. Therefore, we will primarily handle bug issues opened in this repo and we kindly request you to create all feature and enhancement requests on the [GitHub Feedback](https://github.com/community/community/discussions/categories/actions-and-packages) page.
> IMPORTANT: Building your own runner is critical for the dev inner loop process when contributing changes. However, only runners built and distributed by GitHub (releases) are supported in production. Be aware that workflows and orchestrations run service side with the runner being a remote process to run steps. For that reason, the service can pull the runner forward so customizations can be lost.
@@ -12,20 +12,49 @@ Issues in this repository should be for the runner application. Note that the V
## Enhancements and Feature Requests
We ask that before significant effort is put into code changes, that we have agreement on taking the change before time is invested in code changes.
We ask that before significant effort is put into code changes, that we have agreement on taking the change before time is invested in code changes.
1. Create a feature request. Once agreed we will take the enhancment
1. Create a feature request. Once agreed we will take the enhancement
2. Create an ADR to agree on the details of the change.
An ADR is an Architectural Decision Record. This allows consensus on the direction forward and also serves as a record of the change and motivation. [Read more here](adrs/README.md)
## Development Life Cycle
### Required Dev Dependencies
## Required Dev Dependencies
  Git for Windows and Linux [Install Here](https://git-scm.com/downloads) (needed for dev sh script)
### To Build, Test, Layout
 cURL [Install here](https://curl.se/download.html) (needed for external sh script)
 Visual Studio 2017 or newer [Install here](https://visualstudio.microsoft.com) (needed for dev sh script)
 Visual Studio 2022 17.3 Preview or later. [Install here](https://docs.microsoft.com/en-us/visualstudio/releases/2022/release-notes-preview)
## Quickstart: Run a job from a real repository
If you just want to get from building the sourcecode to using it to execute an action, you will need:
- The url of your repository
- A runner registration token. You can find it at `https://github.com/{your-repo}/settings/actions/runners/new`
```bash
git clone https://github.com/actions/runner
cd runner/src
./dev.(sh/cmd) layout # the runner that built from source is in {root}/_layout
cd ../_layout
./config.(sh/cmd) --url https://github.com/{your-repo} --token ABCABCABCABCABCABCABCABCABCAB # accept default name, labels and work folder
./run.(sh/cmd)
```
If you trigger a job now, you can see the runner execute it.
Tip: Make sure your job can run on this runner. The easiest way is to set `runs-on: self-hosted` in the workflow file.
## Development Life Cycle
If you're using VS Code, you can follow [these](contribute/vscode.md) steps instead.
### To Build, Test, Layout
Navigate to the `src` directory and run the following command:
@@ -39,7 +68,7 @@ Navigate to the `src` directory and run the following command:
*`build` (`b`): Build everything and update runner layout folder
*`test` (`t`): Build runner binaries and run unit tests
Sample developer flow:
### Sample developer flow:
```bash
git clone https://github.com/actions/runner
@@ -51,25 +80,89 @@ cd ./src
./dev.(sh/cmd)test# run all unit tests before git commit/push
```
View logs:
Let's break that down.
### Clone repository:
```bash
git clone https://github.com/actions/runner
cd runner
```
If you want to push your changes to a remote, it is recommended you fork the repository and use that fork as your origin instead of `https://github.com/actions/runner`.
### Build Layout:
This command will build all projects, then copies them and other dependencies into a folder called `_layout`. The binaries in this folder are then used for running, debugging the runner.
```bash
cd ./src # execute the script from this folder
./dev.(sh/cmd) layout # the runner that built from source is in {root}/_layout
```
If you make code changes after this point, use the argument `build` to build your code in the `src` folder to keep your `_layout` folder up to date.
```bash
cd ./src
./dev.(sh/cmd) build # {root}/_layout will get updated
```
### Test Layout:
This command runs the suite of unit tests in the project
```bash
cd ./src
./dev.(sh/cmd)test# run all unit tests before git commit/push
```
### Configure Runner:
If you want to manually test your runner and run actions from a real repository, you'll have to configure it before running it.
```bash
cd runner/_layout
./config.(sh/cmd)# configure your custom runner
```
You will need your the name of your repository and a runner registration token.
Check [Quickstart](##Quickstart:-Run-a-job-from-a-real-repository) if you don't know how to get this token.
These can also be passed down as arguments to `config.(sh/cmd)`:
This document outlines the automated dependency management process for the GitHub Actions Runner, designed to ensure we maintain up-to-date and secure dependencies while providing predictable release cycles.
## Release Schedule
- **Monthly Runner Releases**: New runner versions are released monthly
- **Weekly Dependency Checks**: Automated workflows check for dependency updates every Monday
- **Security Patches**: Critical security vulnerabilities are addressed immediately outside the regular schedule
## Automated Workflows
**Note**: These workflows are implemented across separate PRs for easier review and independent deployment. Each workflow includes comprehensive error handling and security-focused vulnerability detection.
@@ -58,4 +58,4 @@ Authentication in a workflow run to github.com can be accomplished by using the
Hosted runner authentication differs from self-hosted authentication in that runners do not undergo a registration process, but instead, the hosted runners get the OAuth token directly by reading the `.credentials` file. The scope of this particular token is limited for a given workflow job execution, and the token is revoked as soon as the job is finished.


Please see "[Supported architectures and operating systems for self-hosted runners](https://docs.github.com/en/actions/reference/runners/self-hosted-runners#linux)."
## Install .Net Core 3.x Linux Dependencies
@@ -23,27 +14,27 @@ You might see something like this which indicate a dependency's missing.
./config.sh
libunwind.so.8 => not found
libunwind-x86_64.so.8 => not found
Dependencies is missing for Dotnet Core 3.0
Execute ./bin/installdependencies.sh to install any missing Dotnet Core 3.0 dependencies.
Dependencies is missing for Dotnet Core 6.0
Execute ./bin/installdependencies.sh to install any missing Dotnet Core 6.0 dependencies.
```
You can easily correct the problem by executing `./bin/installdependencies.sh`.
The `installdependencies.sh` script should install all required dependencies on all supported Linux versions
The `installdependencies.sh` script should install all required dependencies on all supported Linux versions
> Note: The `installdependencies.sh` script will try to use the default package management mechanism on your Linux flavor (ex. `yum`/`apt-get`/`apt`).
### Full dependencies list
Debian based OS (Debian, Ubuntu, Linux Mint)
- liblttng-ust0
- libkrb5-3
- liblttng-ust1 or liblttng-ust0
- libkrb5-3
- zlib1g
- libssl1.1, libssl1.0.2 or libssl1.0.0
- libicu63, libicu60, libicu57 or libicu55
Fedora based OS (Fedora, Red Hat Enterprise Linux, CentOS, Oracle Linux 7)
Please see "[Supported architectures and operating systems for self-hosted runners](https://docs.github.com/en/actions/reference/runners/self-hosted-runners#macos)."
Please see "[Supported architectures and operating systems for self-hosted runners](https://docs.github.com/en/actions/reference/runners/self-hosted-runners#windows)."
_Note: Actions Runner follows a progressive release policy, so the latest release might not be available to your enterprise, organization, or repository yet.
To confirm which version of the Actions Runner you should expect, please view the download instructions for your enterprise, organization, or repository.
See https://docs.github.com/en/enterprise-cloud@latest/actions/hosting-your-own-runners/adding-self-hosted-runners_
## Windows x64
We recommend configuring the runner in a root folder of the Windows drive (e.g. "C:\actions-runner"). This will help avoid issues related to service identity folder permissions and long file path restrictions on Windows.
The following snipped needs to be run on `powershell`:
We recommend configuring the runner in a root folder of the Windows drive (e.g. "C:\actions-runner"). This will help avoid issues related to service identity folder permissions and long file path restrictions on Windows.
The following snipped needs to be run on `powershell`:
tar xzf ./actions-runner-linux-arm64-<RUNNER_VERSION>.tar.gz
```
## Linux arm (Pre-release)
## Linux arm
``` bash
```bash
# Create a folder
mkdir actions-runner &&cd actions-runner
# Download the latest runner package
@@ -73,4 +118,17 @@ tar xzf ./actions-runner-linux-arm-<RUNNER_VERSION>.tar.gz
```
## Using your self hosted runner
For additional details about configuring, running, or shutting down the runner please check out our [product docs.](https://help.github.com/en/actions/automating-your-workflow-with-github-actions/adding-self-hosted-runners)
## SHA-256 Checksums
The SHA-256 checksums for the packages included in this build are shown below:
- actions-runner-win-x64-<RUNNER_VERSION>.zip <!-- BEGIN SHA win-x64 --><WIN_X64_SHA><!-- END SHA win-x64 -->
- actions-runner-win-arm64-<RUNNER_VERSION>.zip <!-- BEGIN SHA win-arm64 --><WIN_ARM64_SHA><!-- END SHA win-arm64 -->
- actions-runner-osx-x64-<RUNNER_VERSION>.tar.gz <!-- BEGIN SHA osx-x64 --><OSX_X64_SHA><!-- END SHA osx-x64 -->
- actions-runner-osx-arm64-<RUNNER_VERSION>.tar.gz <!-- BEGIN SHA osx-arm64 --><OSX_ARM64_SHA><!-- END SHA osx-arm64 -->
- actions-runner-linux-x64-<RUNNER_VERSION>.tar.gz <!-- BEGIN SHA linux-x64 --><LINUX_X64_SHA><!-- END SHA linux-x64 -->
- actions-runner-linux-arm64-<RUNNER_VERSION>.tar.gz <!-- BEGIN SHA linux-arm64 --><LINUX_ARM64_SHA><!-- END SHA linux-arm64 -->
- actions-runner-linux-arm-<RUNNER_VERSION>.tar.gz <!-- BEGIN SHA linux-arm --><LINUX_ARM_SHA><!-- END SHA linux-arm -->
if [ $user_id -eq 0 -a -z "$RUNNER_ALLOW_RUNASROOT" ]; then
echo "Must not run interactively with sudo"
exit 1
fi
# Run
shopt -s nocasematch
SOURCE="${BASH_SOURCE[0]}"
while [ -h "$SOURCE" ]; do # resolve $SOURCE until the file is no longer a symlink
DIR="$( cd -P "$( dirname "$SOURCE" )" && pwd )"
SOURCE="$(readlink "$SOURCE")"
[[ $SOURCE != /* ]] && SOURCE="$DIR/$SOURCE" # if $SOURCE was a relative symlink, we need to resolve it relative to the path where the symlink file was located
done
DIR="$( cd -P "$( dirname "$SOURCE" )" && pwd )"
# Wait for docker to start
if [ ! -z "$RUNNER_WAIT_FOR_DOCKER_IN_SECONDS" ]; then
if [ "$RUNNER_WAIT_FOR_DOCKER_IN_SECONDS" -gt 0 ]; then
echo "Waiting for docker to be ready."
for i in $(seq "$RUNNER_WAIT_FOR_DOCKER_IN_SECONDS"); do
if docker ps > /dev/null 2>&1; then
echo "Docker is ready."
break
fi
"$DIR"/safe_sleep.sh 1
done
fi
fi
updateFile="update.finished"
"$DIR"/bin/Runner.Listener run $*
returnCode=$?
if [[ $returnCode == 0 ]]; then
echo "Runner listener exit with 0 return code, stop the service, no retry needed."
exit 0
elif [[ $returnCode == 1 ]]; then
echo "Runner listener exit with terminated error, stop the service, no retry needed."
exit 0
elif [[ $returnCode == 2 ]]; then
echo "Runner listener exit with retryable error, re-launch runner in 5 seconds."
"$DIR"/safe_sleep.sh 5
exit 2
elif [[ $returnCode == 3 ]]; then
# Wait for 30 seconds or for flag file to exists for the runner update process finish
echo "Runner listener exit because of updating, re-launch runner after successful update"
for i in {0..30}; do
if test -f "$updateFile"; then
echo "Update finished successfully."
rm "$updateFile"
break
fi
"$DIR"/safe_sleep.sh 1
done
exit 2
elif [[ $returnCode == 4 ]]; then
# Wait for 30 seconds or for flag file to exists for the ephemeral runner update process finish
echo "Runner listener exit because of updating, re-launch runner after successful update"
for i in {0..30}; do
if test -f "$updateFile"; then
echo "Update finished successfully."
rm "$updateFile"
break
fi
"$DIR"/safe_sleep.sh 1
done
exit 2
elif [[ $returnCode == 5 ]]; then
echo "Runner listener exit with Session Conflict error, stop the service, no retry needed."
exit 0
else
echo "Exiting with unknown error code: ${returnCode}"
while[ -h "$SOURCE"];do# resolve $SOURCE until the file is no longer a symlink
DIR="$(cd -P "$( dirname "$SOURCE")"&&pwd)"
SOURCE="$(readlink "$SOURCE")"
[[$SOURCE != /* ]]&&SOURCE="$DIR/$SOURCE"# if $SOURCE was a relative symlink, we need to resolve it relative to the path where the symlink file was located
DIR="$(cd -P "$( dirname "$SOURCE")"&&pwd)"
SOURCE="$(readlink "$SOURCE")"
[[$SOURCE != /* ]]&&SOURCE="$DIR/$SOURCE"# if $SOURCE was a relative symlink, we need to resolve it relative to the path where the symlink file was located
done
DIR="$(cd -P "$( dirname "$SOURCE")"&&pwd)"
# Do not "cd $DIR". For localRun, the current directory is expected to be the repo location on disk.
# Run
shopt -s nocasematch
if[["$1"=="localRun"]];then
"$DIR"/bin/Runner.Listener$*
else
"$DIR"/bin/Runner.Listener run $*
# Return code 4 means the run once runner received an update message.
# Sleep 5 seconds to wait for the update process finish and run the runner again.
returnCode=$?
if[[$returnCode==4]];then
if[ ! -x "$(command -v sleep)"];then
if[ ! -x "$(command -v ping)"];then
COUNT="0"
while[[$COUNT !=5000]];do
echo"SLEEP" >nul
COUNT=$[$COUNT+1]
done
else
ping -n 5 127.0.0.1 >nul
fi
run(){
# run the helper process which keep the listener alive
echo"Warning: failed to update certificate store: update-ca-certificates or update-ca-trust not found. This can happen if you're using a different runner base image."
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.