Compare commits

61 Commits

Author SHA1 Message Date
Nikola Jokic
9705deeb08 Release 0.6.0 (#148) 2024-03-14 14:36:02 +01:00
Katarzyna
99efdeca99 Mount /github/workflow to docker action pods (#137)
* Mount /github/workflow to docker action pods, the same way as for container job pods

* Adjust tests
2024-03-14 12:36:27 +01:00
dependabot[bot]
bb09a79b22 Bump jose from 4.11.4 to 4.15.5 in /packages/k8s (#142)
Bumps [jose](https://github.com/panva/jose) from 4.11.4 to 4.15.5.
- [Release notes](https://github.com/panva/jose/releases)
- [Changelog](https://github.com/panva/jose/blob/v4.15.5/CHANGELOG.md)
- [Commits](https://github.com/panva/jose/compare/v4.11.4...v4.15.5)

---
updated-dependencies:
- dependency-name: jose
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-03-14 12:33:36 +01:00
Katarzyna
746e644039 ADR-0134 superseding ADR-0096 (#136)
Related to https://github.com/actions/runner-container-hooks/issues/132
2024-03-14 12:33:06 +01:00
Katarzyna
7223e1dbb2 Use ACTIONS_RUNNER_CONTAINER_HOOK_TEMPLATE to extend service containers (#134)
https://github.com/actions/runner-container-hooks/issues/132

Co-authored-by: Katarzyna Radkowska <katarzyna.radkowska@sabre.com>
2024-02-20 16:19:29 +01:00
Katarzyna
af27abe1f7 Read logs also from failed child (container job/container action) pod (#135)
Co-authored-by: Katarzyna Radkowska <katarzyna.radkowska@sabre.com>
2024-02-20 12:01:11 +01:00
Nikola Jokic
638bd19c9d Release 0.5.1 (#131)
* Release 0.5.1

* Add one more PR that is part of the release
2024-02-05 15:00:35 +01:00
Nikola Jokic
50e14cf868 Switch exec pod promise to reject on websocket error (#127)
* Switch exec pod promise to reject on websocket error

* Fix incorrectly resolved merge conflict

* Apply suggestions from code review

Co-authored-by: Ferenc Hammerl <31069338+fhammerl@users.noreply.github.com>

---------

Co-authored-by: Ferenc Hammerl <31069338+fhammerl@users.noreply.github.com>
2024-02-05 14:40:15 +01:00
Nikola Jokic
921be5b85f Fix is alpine check using shlex (#130) 2024-02-05 09:50:51 +01:00
Nikola Jokic
0cce49705b Try to get response body message and log entire error response in debug mode (#123) 2023-12-15 13:01:04 +01:00
Nikola Jokic
46c92fe43e Release 0.5.0 (#119) 2023-11-22 13:15:52 +01:00
Nikola Jokic
56208347f1 Update 0096-hook-extensions.md (#118) 2023-11-20 15:10:21 +01:00
Nikola Jokic
c093f87779 Docker and K8s: Fix shell arguments when split by the runner (#115)
* Docker: Fix shell arguments when split by the runner

* Add shlex to k8s hook as well
2023-11-20 15:09:36 +01:00
Nikola Jokic
c47c74ad9e Update CODEOWNERS (#114) 2023-11-09 14:11:50 +01:00
Wout Van De Wiel
90a6236466 Add option to use the kubernetes scheduler for workflow pods (#111)
* Add option to use kube scheduler

This should only be used when rwx volumes are supported or when using a single node cluster.

* Add option to set timeout for prepare job

If the kube scheduler is used to hold jobs until sufficient resources are available,
then prepare job needs to wait for a longer period until the workflow pod is running.
This timeout will mostly need an increase in cases where many jobs are triggered
which together exceed the resources available in the cluster.
The workflows can then be gracefully handled later when sufficient resources become available again.

* Skip name override warning when names match or job extension

* Add guard for positive timeouts with a warning

* Write out ReadWriteMany in full
2023-10-31 12:51:09 +01:00
dependabot[bot]
496287d61d Bump @babel/traverse from 7.18.2 to 7.23.2 in /packages/k8s (#110)
Bumps [@babel/traverse](https://github.com/babel/babel/tree/HEAD/packages/babel-traverse) from 7.18.2 to 7.23.2.
- [Release notes](https://github.com/babel/babel/releases)
- [Changelog](https://github.com/babel/babel/blob/main/CHANGELOG.md)
- [Commits](https://github.com/babel/babel/commits/v7.23.2/packages/babel-traverse)

---
updated-dependencies:
- dependency-name: "@babel/traverse"
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-10-18 09:36:40 +02:00
dependabot[bot]
5264b6cd7d Bump @babel/traverse from 7.17.9 to 7.23.2 in /packages/docker (#109)
Bumps [@babel/traverse](https://github.com/babel/babel/tree/HEAD/packages/babel-traverse) from 7.17.9 to 7.23.2.
- [Release notes](https://github.com/babel/babel/releases)
- [Changelog](https://github.com/babel/babel/blob/main/CHANGELOG.md)
- [Commits](https://github.com/babel/babel/commits/v7.23.2/packages/babel-traverse)

---
updated-dependencies:
- dependency-name: "@babel/traverse"
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-10-18 09:36:30 +02:00
Nikola Jokic
b58b13134a import fs in CD workflow (#106) 2023-09-25 14:39:20 +02:00
Nikola Jokic
8ea7e21dec Fix CD release order (#105) 2023-09-25 14:34:49 +02:00
Nikola Jokic
64000d716a Release notes for 0.4.0 version (#104) 2023-09-25 13:53:01 +02:00
Nikola Jokic
4ff4b552a6 [ADR] Hook extensions (#96)
* [ADR] Hook extensions

* Add ADR number

* Add image field to specify that it is going to be ignored

* Update env name, explain that the file is going to be applied to both job and container step pods

* rewphrase job container to job pod

* update name for the job to $job
2023-09-25 13:52:48 +02:00
Nikola Jokic
4cdcf09c43 Implement yaml extensions overwriting the default pod/container spec (#75)
* Implement yaml extensions overwriting the default pod/container spec

* format files

* Extend specs for container job and include docker and k8s tests in k8s

* Create table tests for docker tests

* included warnings and extracted append logic as generic

* updated merge to allow for file read

* reverted back examples and k8s/tests

* reverted back docker tests

* Tests for extension prepare-job

* Fix lint and format and merge error

* Added basic test for container step

* revert hooklib since new definition for container options is received from a file

* revert docker options since create options are a string

* Fix revert

* Update package locks and deps

* included example of extension.yaml. Added side-car container that was missing

* Ignore spec modification for the service containers, change selector to

* fix lint error

* Add missing image override

* Add comment explaining merge object meta with job and pod

* fix test
2023-09-25 11:49:03 +02:00
Nikola Jokic
5107bb1d41 Escape backtick in writeEntryPointScript (#101) 2023-08-28 10:27:20 +02:00
Nikola Jokic
547ed30dc3 Include sha256 checksums in releaseNotes (#98)
* Include sha256 checksums in releaseNotes

* Add ul for sha
2023-08-28 10:15:08 +02:00
dependabot[bot]
17fb66892c Bump word-wrap from 1.2.3 to 1.2.5 in /packages/docker (#95)
Bumps [word-wrap](https://github.com/jonschlinkert/word-wrap) from 1.2.3 to 1.2.5.
- [Release notes](https://github.com/jonschlinkert/word-wrap/releases)
- [Commits](https://github.com/jonschlinkert/word-wrap/compare/1.2.3...1.2.5)

---
updated-dependencies:
- dependency-name: word-wrap
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-07-25 13:30:07 +02:00
dependabot[bot]
9319a8566a Bump word-wrap from 1.2.3 to 1.2.4 in /packages/hooklib (#88)
Bumps [word-wrap](https://github.com/jonschlinkert/word-wrap) from 1.2.3 to 1.2.4.
- [Release notes](https://github.com/jonschlinkert/word-wrap/releases)
- [Commits](https://github.com/jonschlinkert/word-wrap/compare/1.2.3...1.2.4)

---
updated-dependencies:
- dependency-name: word-wrap
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-07-25 13:27:12 +02:00
dependabot[bot]
669ec6f706 Bump word-wrap from 1.2.3 to 1.2.4 in /packages/k8s (#89)
Bumps [word-wrap](https://github.com/jonschlinkert/word-wrap) from 1.2.3 to 1.2.4.
- [Release notes](https://github.com/jonschlinkert/word-wrap/releases)
- [Commits](https://github.com/jonschlinkert/word-wrap/compare/1.2.3...1.2.4)

---
updated-dependencies:
- dependency-name: word-wrap
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-07-25 13:26:57 +02:00
dependabot[bot]
aa658859f8 Bump word-wrap from 1.2.3 to 1.2.4 (#90)
Bumps [word-wrap](https://github.com/jonschlinkert/word-wrap) from 1.2.3 to 1.2.4.
- [Release notes](https://github.com/jonschlinkert/word-wrap/releases)
- [Commits](https://github.com/jonschlinkert/word-wrap/compare/1.2.3...1.2.4)

---
updated-dependencies:
- dependency-name: word-wrap
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-07-25 13:26:27 +02:00
Nikola Jokic
8b83223a2b Add limitation and throw if an entrypoint is not specified for container step (#77) 2023-07-17 11:02:03 +02:00
Takamasa Saichi
586a052286 Do not overwrite entrypoint if it has already been set or if it is Service container (#83) 2023-07-17 10:33:34 +02:00
dependabot[bot]
730509f702 Bump tough-cookie from 4.0.0 to 4.1.3 in /packages/docker (#87)
Bumps [tough-cookie](https://github.com/salesforce/tough-cookie) from 4.0.0 to 4.1.3.
- [Release notes](https://github.com/salesforce/tough-cookie/releases)
- [Changelog](https://github.com/salesforce/tough-cookie/blob/master/CHANGELOG.md)
- [Commits](https://github.com/salesforce/tough-cookie/compare/v4.0.0...v4.1.3)

---
updated-dependencies:
- dependency-name: tough-cookie
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-07-11 10:41:36 +02:00
Marko Zagožen
3fc91e4132 Fix argument order for 'docker pull' (#85)
The optional --config option must come *before* the pull argument.
2023-06-30 15:03:01 +02:00
Nikola Jokic
ebbe2bdaff Update package.json version (#79) 2023-05-17 11:18:23 +02:00
Nikola Jokic
17837d25d2 Release notes for 0.3.2 (#78) 2023-05-17 11:03:05 +02:00
Arthur Baars
c37c5ca584 k8s: handle $ symbols in environment variable names and values (#74)
* Add test cases

* Handle $ symbols in environment variable names and values
2023-04-18 15:14:10 +02:00
Bassem Dghaidi
04b58be49a ADR: using ephemeral containers (#72)
* Add ephemeral containers ADR draft

* Add ADR PR number to filename and title

* Add motivation

* Add evaluation section with details

* Add storage configuration

* Add the remaining sections

* Fix formatting

* Add guidance

* Update ADR status to rejected
2023-04-05 06:48:48 -04:00
Nikola Jokic
89ff7d1155 Release 0.3.1 (#71)
* Update releaseNotes.md

* updated package.json and package lock
2023-03-20 15:26:23 +01:00
Tingluo Huang
6dbb0b61b7 Ensure response consist no matter having ports or not. (#70)
* Ensure responseconsist no matter having ports or not.

* Update packages/k8s/src/hooks/prepare-job.ts

Co-authored-by: Nikola Jokic <jokicnikola07@gmail.com>

---------

Co-authored-by: Nikola Jokic <jokicnikola07@gmail.com>
2023-03-20 10:11:19 +01:00
Bassem Dghaidi
c92bb5544e Fix 0.3.0 release notes (#69) 2023-03-17 05:30:10 -04:00
Nikola Jokic
26f4a32c30 0.3.0 release notes (#68) 2023-03-17 10:18:56 +01:00
dependabot[bot]
10c6c0aa70 Bump cacheable-request and @kubernetes/client-node in /packages/k8s (#66)
Removes [cacheable-request](https://github.com/jaredwray/cacheable-request). It's no longer used after updating ancestor dependency [@kubernetes/client-node](https://github.com/kubernetes-client/javascript). These dependencies need to be updated together.


Removes `cacheable-request`

Updates `@kubernetes/client-node` from 0.16.3 to 0.18.1
- [Release notes](https://github.com/kubernetes-client/javascript/releases)
- [Changelog](https://github.com/kubernetes-client/javascript/blob/master/CHANGELOG.md)
- [Commits](https://github.com/kubernetes-client/javascript/commits)

---
updated-dependencies:
- dependency-name: cacheable-request
  dependency-type: indirect
- dependency-name: "@kubernetes/client-node"
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-03-02 11:28:06 +01:00
Nikola Jokic
d735152125 Exit from run k8s not allowing promise rejection (#65)
* Exit from run k8s not allowing promise rejection

* Unused case removed k8s
2023-02-14 11:30:16 +01:00
Nikola Jokic
ae31f04223 removed equal sign from env buffer, added defensive guard against the key (#62)
* removed equal sign from env buffer, added defensive guard against the key

* Update packages/k8s/src/k8s/utils.ts

Co-authored-by: John Sudol <24583161+johnsudol@users.noreply.github.com>

* Update packages/k8s/src/k8s/utils.ts

Co-authored-by: Ferenc Hammerl <31069338+fhammerl@users.noreply.github.com>

* fix format

---------

Co-authored-by: John Sudol <24583161+johnsudol@users.noreply.github.com>
Co-authored-by: Ferenc Hammerl <31069338+fhammerl@users.noreply.github.com>
2023-02-09 17:11:16 +01:00
dependabot[bot]
7754cb80eb Bump http-cache-semantics from 4.1.0 to 4.1.1 in /packages/k8s (#63)
Bumps [http-cache-semantics](https://github.com/kornelski/http-cache-semantics) from 4.1.0 to 4.1.1.
- [Release notes](https://github.com/kornelski/http-cache-semantics/releases)
- [Commits](https://github.com/kornelski/http-cache-semantics/compare/v4.1.0...v4.1.1)

---
updated-dependencies:
- dependency-name: http-cache-semantics
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-02-09 14:54:32 +01:00
Nikola Jokic
ae432db512 docker and k8s: read from stdin inside try catch block (#49)
There might be situation where reading from standard input fails. In
that case, we should encapsulate that exception within the try catch
block to avoid unhandeled Promise rejection exception and provide more
information about the error
2023-01-23 12:46:47 +01:00
Nikola Jokic
4448b61e00 Fix service port mappings when input is undefined, null, or empty (#60)
* fix: service without ports defined

* fix port mappings when ports are undefined,null or empty

* fix

Co-authored-by: Ronald Claveau <ronald.claveau@pennylane.com>
2023-01-06 11:54:52 +01:00
dependabot[bot]
bf39b9bf16 Bump json5 from 1.0.1 to 1.0.2 in /packages/hooklib (#56)
Bumps [json5](https://github.com/json5/json5) from 1.0.1 to 1.0.2.
- [Release notes](https://github.com/json5/json5/releases)
- [Changelog](https://github.com/json5/json5/blob/main/CHANGELOG.md)
- [Commits](https://github.com/json5/json5/compare/v1.0.1...v1.0.2)

---
updated-dependencies:
- dependency-name: json5
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-01-06 11:05:26 +01:00
dependabot[bot]
5b597b0fe2 Bump json5 from 2.2.1 to 2.2.3 in /packages/k8s (#57)
Bumps [json5](https://github.com/json5/json5) from 2.2.1 to 2.2.3.
- [Release notes](https://github.com/json5/json5/releases)
- [Changelog](https://github.com/json5/json5/blob/main/CHANGELOG.md)
- [Commits](https://github.com/json5/json5/compare/v2.2.1...v2.2.3)

---
updated-dependencies:
- dependency-name: json5
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-01-06 11:05:05 +01:00
dependabot[bot]
0e1ba7bdc8 Bump json5 from 1.0.1 to 1.0.2 in /packages/docker (#58)
Bumps [json5](https://github.com/json5/json5) from 1.0.1 to 1.0.2.
- [Release notes](https://github.com/json5/json5/releases)
- [Changelog](https://github.com/json5/json5/blob/main/CHANGELOG.md)
- [Commits](https://github.com/json5/json5/compare/v1.0.1...v1.0.2)

---
updated-dependencies:
- dependency-name: json5
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-01-06 11:04:42 +01:00
Niels ten Boom
73914b840c fix: naming for services & service entrypoint (#53)
* rename to container

* fix container image name bug

* fix entrypoint bug

* bump patch version

* formatting

* fix versions in package-lock

* add test

* revert version bump

* added check + test for args as well

* formatting

* remove cscode launch.json

* expand example json

* wrong version, revert to correct one

* correct lock

* throw error on invalid image definition

* change falsy check

* Update packages/k8s/src/k8s/utils.ts

Co-authored-by: Ferenc Hammerl <31069338+fhammerl@users.noreply.github.com>

Co-authored-by: Ferenc Hammerl <31069338+fhammerl@users.noreply.github.com>
2023-01-06 10:22:41 +01:00
Nikola Jokic
b537fd4c92 Upgrade package json5 (#55) 2023-01-05 10:30:51 +01:00
Ferenc Hammerl
17d2b3b850 Release notes for v0.2.0 (#47)
* Update releaseNotes.md

* Bump version to 0.2.0
2022-12-15 15:29:15 +01:00
dependabot[bot]
ea011028f5 Bump @actions/core from 1.6.0 to 1.9.1 in /packages/hooklib (#29)
* Bump @actions/core from 1.6.0 to 1.9.1 in /packages/hooklib

Bumps [@actions/core](https://github.com/actions/toolkit/tree/HEAD/packages/core) from 1.6.0 to 1.9.1.
- [Release notes](https://github.com/actions/toolkit/releases)
- [Changelog](https://github.com/actions/toolkit/blob/main/packages/core/RELEASES.md)
- [Commits](https://github.com/actions/toolkit/commits/HEAD/packages/core)

---
updated-dependencies:
- dependency-name: "@actions/core"
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>

* Trigger Build

* Update package lock for docker and k8s

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Ferenc Hammerl <31069338+fhammerl@users.noreply.github.com>
2022-12-15 14:58:13 +01:00
Nikola Jokic
eaae191ebb k8s: don't overwriting service entrypoint (#45) 2022-12-15 14:13:57 +01:00
dependabot[bot]
418d484160 Bump jose from 2.0.5 to 2.0.6 in /packages/k8s (#31)
Bumps [jose](https://github.com/panva/jose) from 2.0.5 to 2.0.6.
- [Release notes](https://github.com/panva/jose/releases)
- [Changelog](https://github.com/panva/jose/blob/v2.0.6/CHANGELOG.md)
- [Commits](https://github.com/panva/jose/compare/v2.0.5...v2.0.6)

---
updated-dependencies:
- dependency-name: jose
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-12-15 14:04:57 +01:00
Nikola Jokic
ce3c55d086 exposing env variables from runner with DOCKER_ envs to respect docker options set on host (#40)
* exposing env variables from runner with DOCKER_ prefix to respect rootless docker

* Prioritize DOCKER cli over workflow envs

* formatted
2022-12-08 08:09:51 +01:00
Nikola Jokic
d988d965c5 fixing issue related to setting hostPort and containerPort when format is port/proto (#38)
* fixing issue related to setting hostPort and containerPort when format is port/proto

* added one more test case and refactored containerPorts to be without regexp

* added throw on ports outside of (0,65536) range with test

* repaired error message and added tests to multi splits. refactored port checking
2022-11-15 14:23:09 +01:00
Nikola Jokic
23cc6dda6f fixed substring issue with /github/workspace and /github/file_commands (#35)
* fixed substring issue with /github/workspace and /github/file_commands

* npm run format

* last 3 parts of the path are mounted to /github/workspace and /github/file_commands

* file commands now point to _temp/_runner_file_commands
2022-11-03 14:55:07 +01:00
dependabot[bot]
8986035ca8 Bump @actions/core from 1.8.2 to 1.9.1 in /packages/k8s (#28)
Bumps [@actions/core](https://github.com/actions/toolkit/tree/HEAD/packages/core) from 1.8.2 to 1.9.1.
- [Release notes](https://github.com/actions/toolkit/releases)
- [Changelog](https://github.com/actions/toolkit/blob/main/packages/core/RELEASES.md)
- [Commits](https://github.com/actions/toolkit/commits/HEAD/packages/core)

---
updated-dependencies:
- dependency-name: "@actions/core"
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-10-25 17:12:49 +02:00
dependabot[bot]
e975289683 Bump @actions/core from 1.6.0 to 1.9.1 in /packages/docker (#27)
Bumps [@actions/core](https://github.com/actions/toolkit/tree/HEAD/packages/core) from 1.6.0 to 1.9.1.
- [Release notes](https://github.com/actions/toolkit/releases)
- [Changelog](https://github.com/actions/toolkit/blob/main/packages/core/RELEASES.md)
- [Commits](https://github.com/actions/toolkit/commits/HEAD/packages/core)

---
updated-dependencies:
- dependency-name: "@actions/core"
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-10-25 17:12:17 +02:00
Nikola Jokic
a555151eef repaired env variable name in CONTRIBUTING (HOOK(S)) (#37) 2022-10-25 16:26:59 +02:00
40 changed files with 2699 additions and 1480 deletions

1
.gitattributes vendored Normal file
View File

@@ -0,0 +1 @@
*.png filter=lfs diff=lfs merge=lfs -text

View File

@@ -13,28 +13,47 @@ jobs:
- run: npm run build-all
name: Build packages
- uses: actions/github-script@v6
id: releaseNotes
id: releaseVersion
with:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
script: |
const fs = require('fs');
const hookVersion = require('./package.json').version
var releaseNotes = fs.readFileSync('${{ github.workspace }}/releaseNotes.md', 'utf8').replace(/<HOOK_VERSION>/g, hookVersion)
console.log(releaseNotes)
core.setOutput('version', hookVersion);
core.setOutput('note', releaseNotes);
- name: Zip up releases
run: |
zip -r -j actions-runner-hooks-docker-${{ steps.releaseNotes.outputs.version }}.zip packages/docker/dist
zip -r -j actions-runner-hooks-k8s-${{ steps.releaseNotes.outputs.version }}.zip packages/k8s/dist
zip -r -j actions-runner-hooks-docker-${{ steps.releaseVersion.outputs.version }}.zip packages/docker/dist
zip -r -j actions-runner-hooks-k8s-${{ steps.releaseVersion.outputs.version }}.zip packages/k8s/dist
- name: Calculate SHA
id: sha
shell: bash
run: |
sha_docker=$(sha256sum actions-runner-hooks-docker-${{ steps.releaseVersion.outputs.version }}.zip | awk '{print $1}')
echo "Docker SHA: $sha_docker"
echo "docker-sha=$sha_docker" >> $GITHUB_OUTPUT
sha_k8s=$(sha256sum actions-runner-hooks-k8s-${{ steps.releaseVersion.outputs.version }}.zip | awk '{print $1}')
echo "K8s SHA: $sha_k8s"
echo "k8s-sha=$sha_k8s" >> $GITHUB_OUTPUT
- name: replace SHA
id: releaseNotes
uses: actions/github-script@v6
with:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
script: |
const fs = require('fs');
var releaseNotes = fs.readFileSync('${{ github.workspace }}/releaseNotes.md', 'utf8').replace(/<HOOK_VERSION>/g, '${{ steps.releaseVersion.outputs.version }}')
releaseNotes = releaseNotes.replace(/<DOCKER_SHA>/g, '${{ steps.sha.outputs.docker-sha }}')
releaseNotes = releaseNotes.replace(/<K8S_SHA>/g, '${{ steps.sha.outputs.k8s-sha }}')
console.log(releaseNotes)
core.setOutput('note', releaseNotes);
- uses: actions/create-release@v1
id: createRelease
name: Create ${{ steps.releaseNotes.outputs.version }} Hook Release
name: Create ${{ steps.releaseVersion.outputs.version }} Hook Release
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:
tag_name: "v${{ steps.releaseNotes.outputs.version }}"
release_name: "v${{ steps.releaseNotes.outputs.version }}"
tag_name: "v${{ steps.releaseVersion.outputs.version }}"
release_name: "v${{ steps.releaseVersion.outputs.version }}"
body: |
${{ steps.releaseNotes.outputs.note }}
- name: Upload K8s hooks
@@ -43,8 +62,8 @@ jobs:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:
upload_url: ${{ steps.createRelease.outputs.upload_url }}
asset_path: ${{ github.workspace }}/actions-runner-hooks-k8s-${{ steps.releaseNotes.outputs.version }}.zip
asset_name: actions-runner-hooks-k8s-${{ steps.releaseNotes.outputs.version }}.zip
asset_path: ${{ github.workspace }}/actions-runner-hooks-k8s-${{ steps.releaseVersion.outputs.version }}.zip
asset_name: actions-runner-hooks-k8s-${{ steps.releaseVersion.outputs.version }}.zip
asset_content_type: application/octet-stream
- name: Upload docker hooks
uses: actions/upload-release-asset@v1
@@ -52,6 +71,6 @@ jobs:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:
upload_url: ${{ steps.createRelease.outputs.upload_url }}
asset_path: ${{ github.workspace }}/actions-runner-hooks-docker-${{ steps.releaseNotes.outputs.version }}.zip
asset_name: actions-runner-hooks-docker-${{ steps.releaseNotes.outputs.version }}.zip
asset_path: ${{ github.workspace }}/actions-runner-hooks-docker-${{ steps.releaseVersion.outputs.version }}.zip
asset_name: actions-runner-hooks-docker-${{ steps.releaseVersion.outputs.version }}.zip
asset_content_type: application/octet-stream

View File

@@ -1 +1 @@
* @actions/actions-runtime @actions/runner-akvelon
* @actions/actions-launch @actions/runner-akvelon

View File

@@ -13,7 +13,7 @@ You'll need a runner compatible with hooks, a repository with container workflow
- You'll need a runner compatible with hooks, a repository with container workflows to which you can register the runner and the hooks from this repository.
- See [the runner contributing.md](../../github/CONTRIBUTING.MD) for how to get started with runner development.
- Build your hook using `npm run build`
- Enable the hooks by setting `ACTIONS_RUNNER_CONTAINER_HOOK=./packages/{libraryname}/dist/index.js` file generated by [ncc](https://github.com/vercel/ncc)
- Enable the hooks by setting `ACTIONS_RUNNER_CONTAINER_HOOKS=./packages/{libraryname}/dist/index.js` file generated by [ncc](https://github.com/vercel/ncc)
- Configure your self hosted runner against the a repository you have admin access
- Run a workflow with a container job, for example
```

View File

@@ -0,0 +1,184 @@
# ADR 0072: Using Ephemeral Containers
**Date:** 27 March 2023
**Status**: Rejected <!--Accepted|Rejected|Superceded|Deprecated-->
## Context
We are evaluating using Kubernetes [ephemeral containers](https://kubernetes.io/docs/concepts/workloads/pods/ephemeral-containers/) as a drop-in replacement for creating pods for [jobs that run in containers](https://docs.github.com/en/actions/using-jobs/running-jobs-in-a-container) and [service containers](https://docs.github.com/en/actions/using-containerized-services/about-service-containers).
The main motivator behind using ephemeral containers is to eliminate the need for [Persistent Volumes](https://kubernetes.io/docs/concepts/storage/persistent-volumes/). Persistent Volume implementations vary depending on the provider and we want to avoid building a dependency on it in order to provide our end-users a consistent experience.
With ephemeral containers we could leverage [emptyDir volumes](https://kubernetes.io/docs/concepts/storage/volumes/#emptydir) which fits our use case better and its behaviour is consistent across providers.
However, it's important to acknowledge that ephemeral containers were not designed to handle workloads but rather provide a mechanism to inspect running containers for debugging and troubleshooting purposes.
## Evaluation
The criteria that we are using to evaluate whether ephemeral containers are fit for purpose are:
- Networking
- Storage
- Security
- Resource limits
- Logs
- Customizability
### Networking
Ephemeral containers share the networking namespace of the pod they are attached to. This means that ephemeral containers can access the same network interfaces as the pod and can communicate with other containers in the same pod. However, ephemeral containers cannot have ports configured and as such the fields ports, livenessProbe, and readinessProbe are not available [^1][^2]
In this scenario we have 3 containers in a pod:
- `runner`: the main container that runs the GitHub Actions job
- `debugger`: the first ephemeral container
- `debugger2`: the second ephemeral container
By sequentially opening ports on each of these containers and connecting to them we can demonstrate that the communication flow between the runner and the debuggers is feasible.
<details>
<summary>1. Runner -> Debugger communication</summary>
![runner->debugger](./images/runner-debugger.png)
</details>
<details>
<summary>2. Debugger -> Runner communication</summary>
![debugger->runner](./images/debugger-runner.png)
</details>
<details>
<summary>3. Debugger2 -> Debugger communication</summary>
![debugger2->debugger](./images/debugger2-debugger.png)
</details>
### Storage
An emptyDir volume can be successfully mounted (read/write) by the runner as well as the ephemeral containers. This means that ephemeral containers can share data with the runner and other ephemeral containers.
<details>
<summary>Configuration</summary>
```yaml
# Extracted from the values.yaml for the gha-runner-scale-set helm chart
spec:
containers:
- name: runner
image: ghcr.io/actions/actions-runner:latest
command: ["/home/runner/run.sh"]
volumeMounts:
- mountPath: /workspace
name: work-volume
volumes:
- name: work-volume
emptyDir:
sizeLimit: 1Gi
```
```bash
# The API call to the Kubernetes API used to create the ephemeral containers
POD_NAME="arc-runner-set-6sfwd-runner-k7qq6"
NAMESPACE="arc-runners"
curl -v "https://<IP>:<PORT>/api/v1/namespaces/$NAMESPACE/pods/$POD_NAME/ephemeralcontainers" \
-X PATCH \
-H 'Content-Type: application/strategic-merge-patch+json' \
--cacert <PATH_TO_CACERT> \
--cert <PATH_TO_CERT> \
--key <PATH_TO_CLIENT_KEY> \
-d '
{
"spec":
{
"ephemeralContainers":
[
{
"name": "debugger",
"command": ["sh"],
"image": "ghcr.io/actions/actions-runner:latest",
"targetContainerName": "runner",
"stdin": true,
"tty": true,
"volumeMounts": [{
"mountPath": "/workspace",
"name": "work-volume",
"readOnly": false
}]
},
{
"name": "debugger2",
"command": ["sh"],
"image": "ghcr.io/actions/actions-runner:latest",
"targetContainerName": "runner",
"stdin": true,
"tty": true,
"volumeMounts": [{
"mountPath": "/workspace",
"name": "work-volume",
"readOnly": false
}]
}
]
}
}'
```
</details>
<details>
<summary>emptyDir volume mount</summary>
![emptyDir volume mount](./images/emptyDir_volume.png)
</details>
### Security
According to the [ephemeral containers API specification](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.26/#ephemeralcontainer-v1-core) the configuration of the `securityContext` field is possible.
Ephemeral containers share the same network namespace as the pod they are attached to. This means that ephemeral containers can access the same network interfaces as the pod and can communicate with other containers in the same pod.
It is also possible for ephemeral containers to [share the process namespace](https://kubernetes.io/docs/tasks/configure-pod-container/share-process-namespace/) with the other containers in the pod. This is disabled by default.
The above could have unpredictable security implications.
### Resource limits
Resources are not allowed for ephemeral containers. Ephemeral containers use spare resources already allocated to the pod. [^1] This is a major drawback as it means that ephemeral containers cannot be configured to have resource limits.
There are no guaranteed resources for ad-hoc troubleshooting. If troubleshooting causes a pod to exceed its resource limit it may be evicted. [^3]
### Logs
Since ephemeral containers can share volumes with the runner container, it's possible to write logs to the same volume and have them available to the runner container.
### Customizability
Ephemeral containers can run any image and tag provided, they can be customized to run any arbitrary job. However, it's important to note that the following are not feasible:
- Lifecycle is not allowed for ephemeral containers
- Ephemeral containers will stop when their command exits, such as exiting a shell, and they will not be restarted. Unlike `kubectl exec`, processes in Ephemeral Containers will not receive an `EOF` if their connections are interrupted, so shells won't automatically exit on disconnect. There is no API support for killing or restarting an ephemeral container. The only way to exit the container is to send it an OS signal. [^4]
- Probes are not allowed for ephemeral containers.
- Ports are not allowed for ephemeral containers.
## Decision
While the evaluation shows that ephemeral containers can be used to run jobs in containers, it's important to acknowledge that ephemeral containers were not designed to handle workloads but rather provide a mechanism to inspect running containers for debugging and troubleshooting purposes.
Given the limitations of ephemeral containers, we decided not to use them outside of their intended purpose.
## Consequences
Proposal rejected, no further action required. This document will be used as a reference for future discussions.
[^1]: https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.26/#ephemeralcontainer-v1-core
[^2]: https://kubernetes.io/docs/concepts/workloads/pods/ephemeral-containers/
[^3]: https://github.com/kubernetes/enhancements/blob/master/keps/sig-node/277-ephemeral-containers/README.md#notesconstraintscaveats
[^4]: https://github.com/kubernetes/enhancements/blob/master/keps/sig-node/277-ephemeral-containers/README.md#ephemeral-container-lifecycle

View File

@@ -0,0 +1,34 @@
# ADR 0096: Hook extensions
**Date:** 3 August 2023
**Status**: Superceded [^1]
## Context
The current implementation of container hooks does not allow users to customize the pods created by the hook. While the implementation is designed to be used as is or as a starting point, building and maintaining a custom hook implementation just to specify additional fields is not a good user experience.
## Decision
We have decided to add hook extensions to the container hook implementation. This will allow users to customize the pods created by the hook by specifying additional fields. The hook extensions will be implemented in a way that is backwards-compatible with the existing hook implementation.
To allow customization, the runner executing the hook should have `ACTIONS_RUNNER_CONTAINER_HOOK_TEMPLATE` environment variable pointing to a yaml file on the runner system. The extension specified in that file will be applied both for job pods, and container steps.
If environment variable is set, but the file can't be read, the hook will fail, signaling incorrect configuration.
If the environment variable does not exist, the hook will apply the default spec.
In case the hook is able to read the extended spec, it will first create a default configuration, and then merged modified fields in the following way:
1. The `.metadata` fields that will be appended if they are not reserved are `labels` and `annotations`.
2. The pod spec fields except for `containers` and `volumes` are applied from the template, possibly overwriting the field.
3. The volumes are applied in form of appending additional volumes to the default volumes.
4. The containers are merged based on the name assigned to them:
1. If the name of the container *is not* "$job", the entire spec of the container will be added to the pod definition.
2. If the name of the container *is* "$job", the `name` and the `image` fields are going to be ignored and the spec will be applied so that `env`, `volumeMounts`, `ports` are appended to the default container spec created by the hook, while the rest of the fields are going to be applied to the newly created container spec.
## Consequences
The addition of hook extensions will provide a better user experience for users who need to customize the pods created by the container hook. However, it will require additional effort to provide the template to the runner pod, and configure it properly.
[^1]: Superseded by [ADR 0134](0134-hook-extensions.md)

View File

@@ -0,0 +1,41 @@
# ADR 0134: Hook extensions
**Date:** 20 February 2024
**Status**: Accepted [^1]
## Context
The current implementation of container hooks does not allow users to customize the pods created by the hook.
While the implementation is designed to be used as is or as a starting point, building and maintaining a custom hook implementation just to specify additional fields is not a good user experience.
## Decision
We have decided to add hook extensions to the container hook implementation.
This will allow users to customize the pods created by the hook by specifying additional fields.
The hook extensions will be implemented in a way that is backwards-compatible with the existing hook implementation.
To allow customization, the runner executing the hook should have `ACTIONS_RUNNER_CONTAINER_HOOK_TEMPLATE` environment variable pointing to a yaml file on the runner system.
The extension specified in that file will be applied both for job pods, and container steps.
If environment variable is set, but the file can't be read, the hook will fail, signaling incorrect configuration.
If the environment variable does not exist, the hook will apply the default spec.
In case the hook is able to read the extended spec, it will first create a default configuration, and then merged modified fields in the following way:
1. The `.metadata` fields that will be appended if they are not reserved are `labels` and `annotations`.
2. The pod spec fields except for `containers` and `volumes` are applied from the template, possibly overwriting the field.
3. The volumes are applied in form of appending additional volumes to the default volumes.
4. The containers are merged based on the name assigned to them:
1. If the name of the container *is* "$job", the `name` and the `image` fields are going to be ignored and the spec will be applied so that `env`, `volumeMounts`, `ports` are appended to the default container spec created by the hook, while the rest of the fields are going to be applied to the newly created container spec.
2. If the name of the container *starts with* "$", and matches the name of the [container service](https://docs.github.com/en/actions/using-containerized-services/about-service-containers), the `name` and the `image` fields are going to be ignored and the spec will be applied to that service container, so that `env`, `volumeMounts`, `ports` are appended to the default container spec for service created by the hook, while the rest of the fields are going to be applied to the created container spec.
If there is no container service with such name defined in the workflow, such spec extension will be ignored.
3. If the name of the container *does not start with* "$", the entire spec of the container will be added to the pod definition.
## Consequences
The addition of hook extensions will provide a better user experience for users who need to customize the pods created by the container hook.
However, it will require additional effort to provide the template to the runner pod, and configure it properly.
[^1]: Supersedes [ADR 0096](0096-hook-extensions.md)

BIN
docs/adrs/images/debugger-runner.png (Stored with Git LFS) Normal file

Binary file not shown.

BIN
docs/adrs/images/debugger2-debugger.png (Stored with Git LFS) Normal file

Binary file not shown.

BIN
docs/adrs/images/emptyDir_volume.png (Stored with Git LFS) Normal file

Binary file not shown.

BIN
docs/adrs/images/runner-debugger.png (Stored with Git LFS) Normal file

Binary file not shown.

41
examples/extension.yaml Normal file
View File

@@ -0,0 +1,41 @@
metadata:
annotations:
annotated-by: "extension"
labels:
labeled-by: "extension"
spec:
securityContext:
runAsUser: 1000
runAsGroup: 3000
restartPolicy: Never
containers:
- name: $job # overwrites job container
env:
- name: ENV1
value: "value1"
imagePullPolicy: Always
image: "busybox:1.28" # Ignored
command:
- sh
args:
- -c
- sleep 50
- name: $redis # overwrites redis service
env:
- name: ENV2
value: "value2"
image: "busybox:1.28" # Ignored
resources:
requests:
memory: "1Mi"
cpu: "1"
limits:
memory: "1Gi"
cpu: "2"
- name: side-car
image: "ubuntu:latest" # required
command:
- sh
args:
- -c
- sleep 60

View File

@@ -73,6 +73,8 @@
"contextName": "redis",
"image": "redis",
"createOptions": "--cpus 1",
"entrypoint": null,
"entryPointArgs": [],
"environmentVariables": {},
"userMountVolumes": [
{

40
package-lock.json generated
View File

@@ -1,12 +1,12 @@
{
"name": "hooks",
"version": "0.1.3",
"version": "0.6.0",
"lockfileVersion": 2,
"requires": true,
"packages": {
"": {
"name": "hooks",
"version": "0.1.3",
"version": "0.6.0",
"license": "MIT",
"devDependencies": {
"@types/jest": "^27.5.1",
@@ -1800,9 +1800,9 @@
"dev": true
},
"node_modules/json5": {
"version": "1.0.1",
"resolved": "https://registry.npmjs.org/json5/-/json5-1.0.1.tgz",
"integrity": "sha512-aKS4WQjPenRxiQsC93MNfjx+nbF4PAdYzmd/1JIj8HYzqfbu86beTuNgXDzPknWk0n0uARlyewZo4s++ES36Ow==",
"version": "1.0.2",
"resolved": "https://registry.npmjs.org/json5/-/json5-1.0.2.tgz",
"integrity": "sha512-g1MWMLBiz8FKi1e4w0UyVL3w+iJceWAFBAaBnnGKOpNa5f8TLktkbre1+s6oICydWAm+HRUGTmI+//xv2hvXYA==",
"dev": true,
"dependencies": {
"minimist": "^1.2.0"
@@ -2322,9 +2322,9 @@
}
},
"node_modules/semver": {
"version": "7.3.7",
"resolved": "https://registry.npmjs.org/semver/-/semver-7.3.7.tgz",
"integrity": "sha512-QlYTucUYOews+WeEujDoEGziz4K6c47V/Bd+LjSSYcA94p+DmINdf7ncaUinThfvZyu13lN9OY1XDxt8C0Tw0g==",
"version": "7.6.0",
"resolved": "https://registry.npmjs.org/semver/-/semver-7.6.0.tgz",
"integrity": "sha512-EnwXhrlwXMk9gKu5/flx5sv/an57AkRplG3hTK68W7FRDN+k+OWBj65M7719OkA82XLBxrcX0KSHj+X5COhOVg==",
"dev": true,
"dependencies": {
"lru-cache": "^6.0.0"
@@ -2625,9 +2625,9 @@
}
},
"node_modules/word-wrap": {
"version": "1.2.3",
"resolved": "https://registry.npmjs.org/word-wrap/-/word-wrap-1.2.3.tgz",
"integrity": "sha512-Hz/mrNwitNRh/HUAtM/VT/5VH+ygD6DV7mYKZAtHOrbs8U7lvPS6xf7EJKMF0uW1KJCl0H701g3ZGus+muE5vQ==",
"version": "1.2.4",
"resolved": "https://registry.npmjs.org/word-wrap/-/word-wrap-1.2.4.tgz",
"integrity": "sha512-2V81OA4ugVo5pRo46hAoD2ivUJx8jXmWXfUkY4KFNw0hEptvN0QfH3K4nHiwzGeKl5rFKedV48QVoqYavy4YpA==",
"dev": true,
"engines": {
"node": ">=0.10.0"
@@ -3926,9 +3926,9 @@
"dev": true
},
"json5": {
"version": "1.0.1",
"resolved": "https://registry.npmjs.org/json5/-/json5-1.0.1.tgz",
"integrity": "sha512-aKS4WQjPenRxiQsC93MNfjx+nbF4PAdYzmd/1JIj8HYzqfbu86beTuNgXDzPknWk0n0uARlyewZo4s++ES36Ow==",
"version": "1.0.2",
"resolved": "https://registry.npmjs.org/json5/-/json5-1.0.2.tgz",
"integrity": "sha512-g1MWMLBiz8FKi1e4w0UyVL3w+iJceWAFBAaBnnGKOpNa5f8TLktkbre1+s6oICydWAm+HRUGTmI+//xv2hvXYA==",
"dev": true,
"requires": {
"minimist": "^1.2.0"
@@ -4292,9 +4292,9 @@
}
},
"semver": {
"version": "7.3.7",
"resolved": "https://registry.npmjs.org/semver/-/semver-7.3.7.tgz",
"integrity": "sha512-QlYTucUYOews+WeEujDoEGziz4K6c47V/Bd+LjSSYcA94p+DmINdf7ncaUinThfvZyu13lN9OY1XDxt8C0Tw0g==",
"version": "7.6.0",
"resolved": "https://registry.npmjs.org/semver/-/semver-7.6.0.tgz",
"integrity": "sha512-EnwXhrlwXMk9gKu5/flx5sv/an57AkRplG3hTK68W7FRDN+k+OWBj65M7719OkA82XLBxrcX0KSHj+X5COhOVg==",
"dev": true,
"requires": {
"lru-cache": "^6.0.0"
@@ -4509,9 +4509,9 @@
}
},
"word-wrap": {
"version": "1.2.3",
"resolved": "https://registry.npmjs.org/word-wrap/-/word-wrap-1.2.3.tgz",
"integrity": "sha512-Hz/mrNwitNRh/HUAtM/VT/5VH+ygD6DV7mYKZAtHOrbs8U7lvPS6xf7EJKMF0uW1KJCl0H701g3ZGus+muE5vQ==",
"version": "1.2.4",
"resolved": "https://registry.npmjs.org/word-wrap/-/word-wrap-1.2.4.tgz",
"integrity": "sha512-2V81OA4ugVo5pRo46hAoD2ivUJx8jXmWXfUkY4KFNw0hEptvN0QfH3K4nHiwzGeKl5rFKedV48QVoqYavy4YpA==",
"dev": true
},
"wrappy": {

View File

@@ -1,6 +1,6 @@
{
"name": "hooks",
"version": "0.1.3",
"version": "0.6.0",
"description": "Three projects are included - k8s: a kubernetes hook implementation that spins up pods dynamically to run a job - docker: A hook implementation of the runner's docker implementation - A hook lib, which contains shared typescript definitions and utilities that the other packages consume",
"main": "",
"directories": {

File diff suppressed because it is too large Load Diff

View File

@@ -10,9 +10,10 @@
"author": "",
"license": "MIT",
"dependencies": {
"@actions/core": "^1.6.0",
"@actions/core": "^1.9.1",
"@actions/exec": "^1.1.1",
"hooklib": "file:../hooklib",
"shlex": "^2.1.2",
"uuid": "^8.3.2"
},
"devDependencies": {

View File

@@ -91,11 +91,12 @@ export async function containerPull(
image: string,
configLocation: string
): Promise<void> {
const dockerArgs: string[] = ['pull']
const dockerArgs: string[] = []
if (configLocation) {
dockerArgs.push('--config')
dockerArgs.push(configLocation)
}
dockerArgs.push('pull')
dockerArgs.push(image)
for (let i = 0; i < 3; i++) {
try {
@@ -427,6 +428,9 @@ export async function containerRun(
dockerArgs.push(args.image)
if (args.entryPointArgs) {
for (const entryPointArg of args.entryPointArgs) {
if (!entryPointArg) {
continue
}
dockerArgs.push(entryPointArg)
}
}
@@ -440,7 +444,7 @@ export async function isContainerAlpine(containerId: string): Promise<boolean> {
containerId,
'sh',
'-c',
"[ $(cat /etc/*release* | grep -i -e '^ID=*alpine*' -c) != 0 ] || exit 1"
`'[ $(cat /etc/*release* | grep -i -e "^ID=*alpine*" -c) != 0 ] || exit 1'`
]
try {
await runDockerCommand(dockerArgs)

View File

@@ -40,7 +40,7 @@ export async function prepareJob(
if (!container?.image) {
core.info('No job container provided, skipping')
} else {
setupContainer(container)
setupContainer(container, true)
const configLocation = await registryLogin(container.registry)
try {
@@ -174,10 +174,12 @@ function generateResponseFile(
writeToResponseFile(responseFile, JSON.stringify(response))
}
function setupContainer(container): void {
function setupContainer(container, jobContainer = false): void {
if (!container.entryPoint && jobContainer) {
container.entryPointArgs = [`-f`, `/dev/null`]
container.entryPoint = 'tail'
}
}
function generateNetworkName(): string {
return `github_network_${uuidv4()}`

View File

@@ -16,15 +16,14 @@ import {
import { checkEnvironment } from './utils'
async function run(): Promise<void> {
try {
checkEnvironment()
const input = await getInputFromStdin()
const args = input['args']
const command = input['command']
const responseFile = input['responseFile']
const state = input['state']
try {
checkEnvironment()
switch (command) {
case Command.PrepareJob:
await prepareJob(args as PrepareJobArgs, responseFile)

View File

@@ -5,6 +5,7 @@ import * as core from '@actions/core'
import { env } from 'process'
// Import this way otherwise typescript has errors
const exec = require('@actions/exec')
const shlex = require('shlex')
export interface RunDockerCommandOptions {
workingDir?: string
@@ -16,6 +17,8 @@ export async function runDockerCommand(
args: string[],
options?: RunDockerCommandOptions
): Promise<string> {
options = optionsWithDockerEnvs(options)
args = fixArgs(args)
const pipes = await exec.getExecOutput('docker', args, options)
if (pipes.exitCode !== 0) {
core.error(`Docker failed with exit code ${pipes.exitCode}`)
@@ -24,6 +27,45 @@ export async function runDockerCommand(
return Promise.resolve(pipes.stdout)
}
export function optionsWithDockerEnvs(
options?: RunDockerCommandOptions
): RunDockerCommandOptions | undefined {
// From https://docs.docker.com/engine/reference/commandline/cli/#environment-variables
const dockerCliEnvs = new Set([
'DOCKER_API_VERSION',
'DOCKER_CERT_PATH',
'DOCKER_CONFIG',
'DOCKER_CONTENT_TRUST_SERVER',
'DOCKER_CONTENT_TRUST',
'DOCKER_CONTEXT',
'DOCKER_DEFAULT_PLATFORM',
'DOCKER_HIDE_LEGACY_COMMANDS',
'DOCKER_HOST',
'DOCKER_STACK_ORCHESTRATOR',
'DOCKER_TLS_VERIFY',
'BUILDKIT_PROGRESS'
])
const dockerEnvs = {}
for (const key in process.env) {
if (dockerCliEnvs.has(key)) {
dockerEnvs[key] = process.env[key]
}
}
const newOptions = {
workingDir: options?.workingDir,
input: options?.input,
env: options?.env || {}
}
// Set docker envs or overwrite provided ones
for (const [key, value] of Object.entries(dockerEnvs)) {
newOptions.env[key] = value as string
}
return newOptions
}
export function sanitize(val: string): string {
if (!val || typeof val !== 'string') {
return ''
@@ -44,6 +86,10 @@ export function sanitize(val: string): string {
return newNameBuilder.join('')
}
export function fixArgs(args: string[]): string[] {
return shlex.split(args.join(' '))
}
export function checkEnvironment(): void {
if (!env.GITHUB_WORKSPACE) {
throw new Error('GITHUB_WORKSPACE is not set')

View File

@@ -40,21 +40,36 @@ describe('run script step', () => {
definitions.runScriptStep.args.entryPoint = '/bin/bash'
definitions.runScriptStep.args.entryPointArgs = [
'-c',
`if [[ ! $(env | grep "^PATH=") = "PATH=${definitions.runScriptStep.args.prependPath}:"* ]]; then exit 1; fi`
`'if [[ ! $(env | grep "^PATH=") = "PATH=${definitions.runScriptStep.args.prependPath}:"* ]]; then exit 1; fi'`
]
await expect(
runScriptStep(definitions.runScriptStep.args, prepareJobResponse.state)
).resolves.not.toThrow()
})
it("Should fix expansion and print correctly in container's stdout", async () => {
const spy = jest.spyOn(process.stdout, 'write').mockImplementation()
definitions.runScriptStep.args.entryPoint = 'echo'
definitions.runScriptStep.args.entryPointArgs = ['"Mona', 'the', `Octocat"`]
await expect(
runScriptStep(definitions.runScriptStep.args, prepareJobResponse.state)
).resolves.not.toThrow()
expect(spy).toHaveBeenCalledWith(
expect.stringContaining('Mona the Octocat')
)
spy.mockRestore()
})
it('Should have path variable changed in container with prepend path string array', async () => {
definitions.runScriptStep.args.prependPath = ['/some/other/path']
definitions.runScriptStep.args.entryPoint = '/bin/bash'
definitions.runScriptStep.args.entryPointArgs = [
'-c',
`if [[ ! $(env | grep "^PATH=") = "PATH=${definitions.runScriptStep.args.prependPath.join(
`'if [[ ! $(env | grep "^PATH=") = "PATH=${definitions.runScriptStep.args.prependPath.join(
':'
)}:"* ]]; then exit 1; fi`
)}:"* ]]; then exit 1; fi'`
]
await expect(
runScriptStep(definitions.runScriptStep.args, prepareJobResponse.state)

View File

@@ -1,4 +1,4 @@
import { sanitize } from '../src/utils'
import { optionsWithDockerEnvs, sanitize, fixArgs } from '../src/utils'
describe('Utilities', () => {
it('should return sanitized image name', () => {
@@ -9,4 +9,72 @@ describe('Utilities', () => {
const validStr = 'teststr8_one'
expect(sanitize(validStr)).toBe(validStr)
})
test.each([
[['"Hello', 'World"'], ['Hello World']],
[
[
'sh',
'-c',
`'[ $(cat /etc/*release* | grep -i -e "^ID=*alpine*" -c) != 0 ] || exit 1'`
],
[
'sh',
'-c',
`[ $(cat /etc/*release* | grep -i -e "^ID=*alpine*" -c) != 0 ] || exit 1`
]
],
[
[
'sh',
'-c',
`'[ $(cat /etc/*release* | grep -i -e '\\''^ID=*alpine*'\\'' -c) != 0 ] || exit 1'`
],
[
'sh',
'-c',
`[ $(cat /etc/*release* | grep -i -e '^ID=*alpine*' -c) != 0 ] || exit 1`
]
]
])('should fix split arguments(%p, %p)', (args, expected) => {
const got = fixArgs(args)
expect(got).toStrictEqual(expected)
})
describe('with docker options', () => {
it('should augment options with docker environment variables', () => {
process.env.DOCKER_HOST = 'unix:///run/user/1001/docker.sock'
process.env.DOCKER_NOTEXIST = 'notexist'
const optionDefinitions: any = [
undefined,
{},
{ env: {} },
{ env: { DOCKER_HOST: 'unix://var/run/docker.sock' } }
]
for (const opt of optionDefinitions) {
let options = optionsWithDockerEnvs(opt)
expect(options).toBeDefined()
expect(options?.env).toBeDefined()
expect(options?.env?.DOCKER_HOST).toBe(process.env.DOCKER_HOST)
expect(options?.env?.DOCKER_NOTEXIST).toBeUndefined()
}
})
it('should not overwrite other options', () => {
process.env.DOCKER_HOST = 'unix:///run/user/1001/docker.sock'
const opt = {
workingDir: 'test',
input: Buffer.from('test')
}
const options = optionsWithDockerEnvs(opt)
expect(options).toBeDefined()
expect(options?.workingDir).toBe(opt.workingDir)
expect(options?.input).toBe(opt.input)
expect(options?.env).toStrictEqual({
DOCKER_HOST: process.env.DOCKER_HOST
})
})
})
})

View File

@@ -9,7 +9,7 @@
"version": "0.1.0",
"license": "MIT",
"dependencies": {
"@actions/core": "^1.6.0"
"@actions/core": "^1.9.1"
},
"devDependencies": {
"@types/node": "^17.0.23",
@@ -22,19 +22,20 @@
}
},
"node_modules/@actions/core": {
"version": "1.6.0",
"resolved": "https://registry.npmjs.org/@actions/core/-/core-1.6.0.tgz",
"integrity": "sha512-NB1UAZomZlCV/LmJqkLhNTqtKfFXJZAUPcfl/zqG7EfsQdeUJtaWO98SGbuQ3pydJ3fHl2CvI/51OKYlCYYcaw==",
"version": "1.9.1",
"resolved": "https://registry.npmjs.org/@actions/core/-/core-1.9.1.tgz",
"integrity": "sha512-5ad+U2YGrmmiw6du20AQW5XuWo7UKN2052FjSV7MX+Wfjf8sCqcsZe62NfgHys4QI4/Y+vQvLKYL8jWtA1ZBTA==",
"dependencies": {
"@actions/http-client": "^1.0.11"
"@actions/http-client": "^2.0.1",
"uuid": "^8.3.2"
}
},
"node_modules/@actions/http-client": {
"version": "1.0.11",
"resolved": "https://registry.npmjs.org/@actions/http-client/-/http-client-1.0.11.tgz",
"integrity": "sha512-VRYHGQV1rqnROJqdMvGUbY/Kn8vriQe/F9HR2AlYHzmKuM/p3kjNuXhmdBfcVgsvRWTz5C5XW5xvndZrVBuAYg==",
"version": "2.0.1",
"resolved": "https://registry.npmjs.org/@actions/http-client/-/http-client-2.0.1.tgz",
"integrity": "sha512-PIXiMVtz6VvyaRsGY268qvj57hXQEpsYogYOu2nrQhlf+XCGmZstmuZBbAybUl1nQGnvS1k1eEsQ69ZoD7xlSw==",
"dependencies": {
"tunnel": "0.0.6"
"tunnel": "^0.0.6"
}
},
"node_modules/@eslint/eslintrc": {
@@ -1741,9 +1742,9 @@
"dev": true
},
"node_modules/json5": {
"version": "1.0.1",
"resolved": "https://registry.npmjs.org/json5/-/json5-1.0.1.tgz",
"integrity": "sha512-aKS4WQjPenRxiQsC93MNfjx+nbF4PAdYzmd/1JIj8HYzqfbu86beTuNgXDzPknWk0n0uARlyewZo4s++ES36Ow==",
"version": "1.0.2",
"resolved": "https://registry.npmjs.org/json5/-/json5-1.0.2.tgz",
"integrity": "sha512-g1MWMLBiz8FKi1e4w0UyVL3w+iJceWAFBAaBnnGKOpNa5f8TLktkbre1+s6oICydWAm+HRUGTmI+//xv2hvXYA==",
"dev": true,
"dependencies": {
"minimist": "^1.2.0"
@@ -2214,9 +2215,9 @@
}
},
"node_modules/semver": {
"version": "7.3.7",
"resolved": "https://registry.npmjs.org/semver/-/semver-7.3.7.tgz",
"integrity": "sha512-QlYTucUYOews+WeEujDoEGziz4K6c47V/Bd+LjSSYcA94p+DmINdf7ncaUinThfvZyu13lN9OY1XDxt8C0Tw0g==",
"version": "7.5.4",
"resolved": "https://registry.npmjs.org/semver/-/semver-7.5.4.tgz",
"integrity": "sha512-1bCSESV6Pv+i21Hvpxp3Dx+pSD8lIPt8uVjRrxAUt/nbswYc+tK6Y2btiULjd4+fnq15PX+nqQDC7Oft7WkwcA==",
"dev": true,
"dependencies": {
"lru-cache": "^6.0.0"
@@ -2485,6 +2486,14 @@
"punycode": "^2.1.0"
}
},
"node_modules/uuid": {
"version": "8.3.2",
"resolved": "https://registry.npmjs.org/uuid/-/uuid-8.3.2.tgz",
"integrity": "sha512-+NYs2QeMWy+GWFOEm9xnn6HCDp0l7QBD7ml8zLUmJ+93Q5NF0NocErnwkTkXVFNiX3/fpC6afS8Dhb/gz7R7eg==",
"bin": {
"uuid": "dist/bin/uuid"
}
},
"node_modules/v8-compile-cache": {
"version": "2.3.0",
"resolved": "https://registry.npmjs.org/v8-compile-cache/-/v8-compile-cache-2.3.0.tgz",
@@ -2523,9 +2532,9 @@
}
},
"node_modules/word-wrap": {
"version": "1.2.3",
"resolved": "https://registry.npmjs.org/word-wrap/-/word-wrap-1.2.3.tgz",
"integrity": "sha512-Hz/mrNwitNRh/HUAtM/VT/5VH+ygD6DV7mYKZAtHOrbs8U7lvPS6xf7EJKMF0uW1KJCl0H701g3ZGus+muE5vQ==",
"version": "1.2.4",
"resolved": "https://registry.npmjs.org/word-wrap/-/word-wrap-1.2.4.tgz",
"integrity": "sha512-2V81OA4ugVo5pRo46hAoD2ivUJx8jXmWXfUkY4KFNw0hEptvN0QfH3K4nHiwzGeKl5rFKedV48QVoqYavy4YpA==",
"dev": true,
"engines": {
"node": ">=0.10.0"
@@ -2546,19 +2555,20 @@
},
"dependencies": {
"@actions/core": {
"version": "1.6.0",
"resolved": "https://registry.npmjs.org/@actions/core/-/core-1.6.0.tgz",
"integrity": "sha512-NB1UAZomZlCV/LmJqkLhNTqtKfFXJZAUPcfl/zqG7EfsQdeUJtaWO98SGbuQ3pydJ3fHl2CvI/51OKYlCYYcaw==",
"version": "1.9.1",
"resolved": "https://registry.npmjs.org/@actions/core/-/core-1.9.1.tgz",
"integrity": "sha512-5ad+U2YGrmmiw6du20AQW5XuWo7UKN2052FjSV7MX+Wfjf8sCqcsZe62NfgHys4QI4/Y+vQvLKYL8jWtA1ZBTA==",
"requires": {
"@actions/http-client": "^1.0.11"
"@actions/http-client": "^2.0.1",
"uuid": "^8.3.2"
}
},
"@actions/http-client": {
"version": "1.0.11",
"resolved": "https://registry.npmjs.org/@actions/http-client/-/http-client-1.0.11.tgz",
"integrity": "sha512-VRYHGQV1rqnROJqdMvGUbY/Kn8vriQe/F9HR2AlYHzmKuM/p3kjNuXhmdBfcVgsvRWTz5C5XW5xvndZrVBuAYg==",
"version": "2.0.1",
"resolved": "https://registry.npmjs.org/@actions/http-client/-/http-client-2.0.1.tgz",
"integrity": "sha512-PIXiMVtz6VvyaRsGY268qvj57hXQEpsYogYOu2nrQhlf+XCGmZstmuZBbAybUl1nQGnvS1k1eEsQ69ZoD7xlSw==",
"requires": {
"tunnel": "0.0.6"
"tunnel": "^0.0.6"
}
},
"@eslint/eslintrc": {
@@ -3779,9 +3789,9 @@
"dev": true
},
"json5": {
"version": "1.0.1",
"resolved": "https://registry.npmjs.org/json5/-/json5-1.0.1.tgz",
"integrity": "sha512-aKS4WQjPenRxiQsC93MNfjx+nbF4PAdYzmd/1JIj8HYzqfbu86beTuNgXDzPknWk0n0uARlyewZo4s++ES36Ow==",
"version": "1.0.2",
"resolved": "https://registry.npmjs.org/json5/-/json5-1.0.2.tgz",
"integrity": "sha512-g1MWMLBiz8FKi1e4w0UyVL3w+iJceWAFBAaBnnGKOpNa5f8TLktkbre1+s6oICydWAm+HRUGTmI+//xv2hvXYA==",
"dev": true,
"requires": {
"minimist": "^1.2.0"
@@ -4109,9 +4119,9 @@
}
},
"semver": {
"version": "7.3.7",
"resolved": "https://registry.npmjs.org/semver/-/semver-7.3.7.tgz",
"integrity": "sha512-QlYTucUYOews+WeEujDoEGziz4K6c47V/Bd+LjSSYcA94p+DmINdf7ncaUinThfvZyu13lN9OY1XDxt8C0Tw0g==",
"version": "7.5.4",
"resolved": "https://registry.npmjs.org/semver/-/semver-7.5.4.tgz",
"integrity": "sha512-1bCSESV6Pv+i21Hvpxp3Dx+pSD8lIPt8uVjRrxAUt/nbswYc+tK6Y2btiULjd4+fnq15PX+nqQDC7Oft7WkwcA==",
"dev": true,
"requires": {
"lru-cache": "^6.0.0"
@@ -4300,6 +4310,11 @@
"punycode": "^2.1.0"
}
},
"uuid": {
"version": "8.3.2",
"resolved": "https://registry.npmjs.org/uuid/-/uuid-8.3.2.tgz",
"integrity": "sha512-+NYs2QeMWy+GWFOEm9xnn6HCDp0l7QBD7ml8zLUmJ+93Q5NF0NocErnwkTkXVFNiX3/fpC6afS8Dhb/gz7R7eg=="
},
"v8-compile-cache": {
"version": "2.3.0",
"resolved": "https://registry.npmjs.org/v8-compile-cache/-/v8-compile-cache-2.3.0.tgz",
@@ -4329,9 +4344,9 @@
}
},
"word-wrap": {
"version": "1.2.3",
"resolved": "https://registry.npmjs.org/word-wrap/-/word-wrap-1.2.3.tgz",
"integrity": "sha512-Hz/mrNwitNRh/HUAtM/VT/5VH+ygD6DV7mYKZAtHOrbs8U7lvPS6xf7EJKMF0uW1KJCl0H701g3ZGus+muE5vQ==",
"version": "1.2.4",
"resolved": "https://registry.npmjs.org/word-wrap/-/word-wrap-1.2.4.tgz",
"integrity": "sha512-2V81OA4ugVo5pRo46hAoD2ivUJx8jXmWXfUkY4KFNw0hEptvN0QfH3K4nHiwzGeKl5rFKedV48QVoqYavy4YpA==",
"dev": true
},
"wrappy": {

View File

@@ -23,6 +23,6 @@
"typescript": "^4.6.3"
},
"dependencies": {
"@actions/core": "^1.6.0"
"@actions/core": "^1.9.1"
}
}

File diff suppressed because it is too large Load Diff

View File

@@ -13,11 +13,13 @@
"author": "",
"license": "MIT",
"dependencies": {
"@actions/core": "^1.6.0",
"@actions/core": "^1.9.1",
"@actions/exec": "^1.1.1",
"@actions/io": "^1.1.2",
"@kubernetes/client-node": "^0.16.3",
"hooklib": "file:../hooklib"
"@kubernetes/client-node": "^0.18.1",
"hooklib": "file:../hooklib",
"js-yaml": "^4.1.0",
"shlex": "^2.1.2"
},
"devDependencies": {
"@types/jest": "^27.4.1",

View File

@@ -41,7 +41,9 @@ export function getSecretName(): string {
export const MAX_POD_NAME_LENGTH = 63
export const STEP_POD_NAME_SUFFIX_LENGTH = 8
export const CONTAINER_EXTENSION_PREFIX = '$'
export const JOB_CONTAINER_NAME = 'job'
export const JOB_CONTAINER_EXTENSION_NAME = '$job'
export class RunnerInstanceLabel {
private podName: string

View File

@@ -1,25 +1,35 @@
import * as core from '@actions/core'
import * as io from '@actions/io'
import * as k8s from '@kubernetes/client-node'
import { ContextPorts, prepareJobArgs, writeToResponseFile } from 'hooklib'
import {
JobContainerInfo,
ContextPorts,
PrepareJobArgs,
writeToResponseFile
} from 'hooklib'
import path from 'path'
import {
containerPorts,
createPod,
isPodContainerAlpine,
prunePods,
waitForPodPhases
waitForPodPhases,
getPrepareJobTimeoutSeconds
} from '../k8s'
import {
containerVolumes,
DEFAULT_CONTAINER_ENTRY_POINT,
DEFAULT_CONTAINER_ENTRY_POINT_ARGS,
PodPhase
generateContainerName,
mergeContainerWithOptions,
readExtensionFromFile,
PodPhase,
fixArgs
} from '../k8s/utils'
import { JOB_CONTAINER_NAME } from './constants'
import { CONTAINER_EXTENSION_PREFIX, JOB_CONTAINER_NAME } from './constants'
export async function prepareJob(
args: prepareJobArgs,
args: PrepareJobArgs,
responseFile
): Promise<void> {
if (!args.container) {
@@ -27,29 +37,51 @@ export async function prepareJob(
}
await prunePods()
const extension = readExtensionFromFile()
await copyExternalsToRoot()
let container: k8s.V1Container | undefined = undefined
if (args.container?.image) {
core.debug(`Using image '${args.container.image}' for job image`)
container = createPodSpec(args.container, JOB_CONTAINER_NAME, true)
container = createContainerSpec(
args.container,
JOB_CONTAINER_NAME,
true,
extension
)
}
let services: k8s.V1Container[] = []
if (args.services?.length) {
services = args.services.map(service => {
core.debug(`Adding service '${service.image}' to pod definition`)
return createPodSpec(service, service.image.split(':')[0])
return createContainerSpec(
service,
generateContainerName(service.image),
false,
extension
)
})
}
if (!container && !services?.length) {
throw new Error('No containers exist, skipping hook invocation')
}
let createdPod: k8s.V1Pod | undefined = undefined
try {
createdPod = await createPod(container, services, args.container.registry)
createdPod = await createPod(
container,
services,
args.container.registry,
extension
)
} catch (err) {
await prunePods()
throw new Error(`failed to create job pod: ${err}`)
core.debug(`createPod failed: ${JSON.stringify(err)}`)
const message = (err as any)?.response?.body?.message || err
throw new Error(`failed to create job pod: ${message}`)
}
if (!createdPod?.metadata?.name) {
@@ -63,11 +95,12 @@ export async function prepareJob(
await waitForPodPhases(
createdPod.metadata.name,
new Set([PodPhase.RUNNING]),
new Set([PodPhase.PENDING])
new Set([PodPhase.PENDING]),
getPrepareJobTimeoutSeconds()
)
} catch (err) {
await prunePods()
throw new Error(`Pod failed to come online with error: ${err}`)
throw new Error(`pod failed to come online with error: ${err}`)
}
core.debug('Job pod is ready for traffic')
@@ -79,7 +112,11 @@ export async function prepareJob(
JOB_CONTAINER_NAME
)
} catch (err) {
throw new Error(`Failed to determine if the pod is alpine: ${err}`)
core.debug(
`Failed to determine if the pod is alpine: ${JSON.stringify(err)}`
)
const message = (err as any)?.response?.body?.message || err
throw new Error(`failed to determine if the pod is alpine: ${message}`)
}
core.debug(`Setting isAlpine to ${isAlpine}`)
generateResponseFile(responseFile, createdPod, isAlpine)
@@ -124,14 +161,12 @@ function generateResponseFile(
)
if (serviceContainers?.length) {
response.context['services'] = serviceContainers.map(c => {
if (!c.ports) {
return
}
const ctxPorts: ContextPorts = {}
if (c.ports?.length) {
for (const port of c.ports) {
ctxPorts[port.containerPort] = port.hostPort
}
}
return {
image: c.image,
@@ -153,12 +188,13 @@ async function copyExternalsToRoot(): Promise<void> {
}
}
function createPodSpec(
container,
export function createContainerSpec(
container: JobContainerInfo,
name: string,
jobContainer = false
jobContainer = false,
extension?: k8s.V1PodTemplateSpec
): k8s.V1Container {
if (!container.entryPoint) {
if (!container.entryPoint && jobContainer) {
container.entryPoint = DEFAULT_CONTAINER_ENTRY_POINT
container.entryPointArgs = DEFAULT_CONTAINER_ENTRY_POINT_ARGS
}
@@ -166,14 +202,20 @@ function createPodSpec(
const podContainer = {
name,
image: container.image,
command: [container.entryPoint],
args: container.entryPointArgs,
ports: containerPorts(container)
} as k8s.V1Container
if (container.workingDirectory) {
podContainer.workingDir = container.workingDirectory
}
if (container.entryPoint) {
podContainer.command = [container.entryPoint]
}
if (container.entryPointArgs?.length > 0) {
podContainer.args = fixArgs(container.entryPointArgs)
}
podContainer.env = []
for (const [key, value] of Object.entries(
container['environmentVariables']
@@ -188,5 +230,17 @@ function createPodSpec(
jobContainer
)
if (!extension) {
return podContainer
}
const from = extension.spec?.containers?.find(
c => c.name === CONTAINER_EXTENSION_PREFIX + name
)
if (from) {
mergeContainerWithOptions(podContainer, from)
}
return podContainer
}

View File

@@ -12,12 +12,12 @@ import {
} from '../k8s'
import {
containerVolumes,
DEFAULT_CONTAINER_ENTRY_POINT,
DEFAULT_CONTAINER_ENTRY_POINT_ARGS,
fixArgs,
mergeContainerWithOptions,
PodPhase,
writeEntryPointScript
readExtensionFromFile
} from '../k8s/utils'
import { JOB_CONTAINER_NAME } from './constants'
import { JOB_CONTAINER_EXTENSION_NAME, JOB_CONTAINER_NAME } from './constants'
export async function runContainerStep(
stepContainer: RunContainerStepArgs
@@ -31,10 +31,20 @@ export async function runContainerStep(
secretName = await createSecretForEnvs(stepContainer.environmentVariables)
}
core.debug(`Created secret ${secretName} for container job envs`)
const container = createPodSpec(stepContainer, secretName)
const extension = readExtensionFromFile()
core.debug(`Created secret ${secretName} for container job envs`)
const container = createContainerSpec(stepContainer, secretName, extension)
let job: k8s.V1Job
try {
job = await createJob(container, extension)
} catch (err) {
core.debug(`createJob failed: ${JSON.stringify(err)}`)
const message = (err as any)?.response?.body?.message || err
throw new Error(`failed to run script step: ${message}`)
}
const job = await createJob(container)
if (!job.metadata?.name) {
throw new Error(
`Expected job ${JSON.stringify(
@@ -44,10 +54,23 @@ export async function runContainerStep(
}
core.debug(`Job created, waiting for pod to start: ${job.metadata?.name}`)
const podName = await getContainerJobPodName(job.metadata.name)
let podName: string
try {
podName = await getContainerJobPodName(job.metadata.name)
} catch (err) {
core.debug(`getContainerJobPodName failed: ${JSON.stringify(err)}`)
const message = (err as any)?.response?.body?.message || err
throw new Error(`failed to get container job pod name: ${message}`)
}
await waitForPodPhases(
podName,
new Set([PodPhase.COMPLETED, PodPhase.RUNNING, PodPhase.SUCCEEDED]),
new Set([
PodPhase.COMPLETED,
PodPhase.RUNNING,
PodPhase.SUCCEEDED,
PodPhase.FAILED
]),
new Set([PodPhase.PENDING, PodPhase.UNKNOWN])
)
core.debug('Container step is running or complete, pulling logs')
@@ -56,6 +79,7 @@ export async function runContainerStep(
core.debug('Waiting for container job to complete')
await waitForJobToComplete(job.metadata.name)
// pod has failed so pull the status code from the container
const status = await getPodStatus(podName)
if (status?.phase === 'Succeeded') {
@@ -75,24 +99,21 @@ export async function runContainerStep(
return Number(exitCode) || 1
}
function createPodSpec(
function createContainerSpec(
container: RunContainerStepArgs,
secretName?: string
secretName?: string,
extension?: k8s.V1PodTemplateSpec
): k8s.V1Container {
const podContainer = new k8s.V1Container()
podContainer.name = JOB_CONTAINER_NAME
podContainer.image = container.image
const { entryPoint, entryPointArgs } = container
container.entryPoint = 'sh'
const { containerPath } = writeEntryPointScript(
container.workingDirectory,
entryPoint || DEFAULT_CONTAINER_ENTRY_POINT,
entryPoint ? entryPointArgs || [] : DEFAULT_CONTAINER_ENTRY_POINT_ARGS
)
container.entryPointArgs = ['-e', containerPath]
podContainer.command = [container.entryPoint, ...container.entryPointArgs]
podContainer.workingDir = container.workingDirectory
podContainer.command = container.entryPoint
? [container.entryPoint]
: undefined
podContainer.args = container.entryPointArgs?.length
? fixArgs(container.entryPointArgs)
: undefined
if (secretName) {
podContainer.envFrom = [
@@ -106,5 +127,16 @@ function createPodSpec(
}
podContainer.volumeMounts = containerVolumes(undefined, false, true)
if (!extension) {
return podContainer
}
const from = extension.spec?.containers?.find(
c => c.name === JOB_CONTAINER_EXTENSION_NAME
)
if (from) {
mergeContainerWithOptions(podContainer, from)
}
return podContainer
}

View File

@@ -1,5 +1,6 @@
/* eslint-disable @typescript-eslint/no-unused-vars */
import * as fs from 'fs'
import * as core from '@actions/core'
import { RunScriptStepArgs } from 'hooklib'
import { execPodStep } from '../k8s'
import { writeEntryPointScript } from '../k8s/utils'
@@ -28,7 +29,9 @@ export async function runScriptStep(
JOB_CONTAINER_NAME
)
} catch (err) {
throw new Error(`failed to run script step: ${err}`)
core.debug(`execPodStep failed: ${JSON.stringify(err)}`)
const message = (err as any)?.response?.body?.message || err
throw new Error(`failed to run script step: ${message}`)
} finally {
fs.rmSync(runnerPath)
}

View File

@@ -9,15 +9,13 @@ import {
import { isAuthPermissionsOK, namespace, requiredPermissions } from './k8s'
async function run(): Promise<void> {
try {
const input = await getInputFromStdin()
const args = input['args']
const command = input['command']
const responseFile = input['responseFile']
const state = input['state']
let exitCode = 0
try {
if (!(await isAuthPermissionsOK())) {
throw new Error(
`The Service account needs the following permissions ${JSON.stringify(
@@ -25,28 +23,28 @@ async function run(): Promise<void> {
)} on the pod resource in the '${namespace()}' namespace. Please contact your self hosted runner administrator.`
)
}
let exitCode = 0
switch (command) {
case Command.PrepareJob:
await prepareJob(args as prepareJobArgs, responseFile)
break
return process.exit(0)
case Command.CleanupJob:
await cleanupJob()
break
return process.exit(0)
case Command.RunScriptStep:
await runScriptStep(args, state, null)
break
return process.exit(0)
case Command.RunContainerStep:
exitCode = await runContainerStep(args)
break
case Command.runContainerStep:
return process.exit(exitCode)
default:
throw new Error(`Command not recognized: ${command}`)
}
} catch (error) {
core.error(error as Error)
exitCode = 1
process.exit(1)
}
process.exitCode = exitCode
}
void run()

View File

@@ -10,7 +10,13 @@ import {
getVolumeClaimName,
RunnerInstanceLabel
} from '../hooks/constants'
import { PodPhase } from './utils'
import {
PodPhase,
mergePodSpecWithOptions,
mergeObjectMeta,
useKubeScheduler,
fixArgs
} from './utils'
const kc = new k8s.KubeConfig()
@@ -20,6 +26,8 @@ const k8sApi = kc.makeApiClient(k8s.CoreV1Api)
const k8sBatchV1Api = kc.makeApiClient(k8s.BatchV1Api)
const k8sAuthorizationV1Api = kc.makeApiClient(k8s.AuthorizationV1Api)
const DEFAULT_WAIT_FOR_POD_TIME_SECONDS = 10 * 60 // 10 min
export const POD_VOLUME_NAME = 'work'
export const requiredPermissions = [
@@ -58,7 +66,8 @@ export const requiredPermissions = [
export async function createPod(
jobContainer?: k8s.V1Container,
services?: k8s.V1Container[],
registry?: Registry
registry?: Registry,
extension?: k8s.V1PodTemplateSpec
): Promise<k8s.V1Pod> {
const containers: k8s.V1Container[] = []
if (jobContainer) {
@@ -80,11 +89,16 @@ export async function createPod(
appPod.metadata.labels = {
[instanceLabel.key]: instanceLabel.value
}
appPod.metadata.annotations = {}
appPod.spec = new k8s.V1PodSpec()
appPod.spec.containers = containers
appPod.spec.restartPolicy = 'Never'
if (!useKubeScheduler()) {
appPod.spec.nodeName = await getCurrentNodeName()
}
const claimName = getVolumeClaimName()
appPod.spec.volumes = [
{
@@ -103,12 +117,21 @@ export async function createPod(
appPod.spec.imagePullSecrets = [secretReference]
}
if (extension?.metadata) {
mergeObjectMeta(appPod, extension.metadata)
}
if (extension?.spec) {
mergePodSpecWithOptions(appPod.spec, extension.spec)
}
const { body } = await k8sApi.createNamespacedPod(namespace(), appPod)
return body
}
export async function createJob(
container: k8s.V1Container
container: k8s.V1Container,
extension?: k8s.V1PodTemplateSpec
): Promise<k8s.V1Job> {
const runnerInstanceLabel = new RunnerInstanceLabel()
@@ -118,6 +141,7 @@ export async function createJob(
job.metadata = new k8s.V1ObjectMeta()
job.metadata.name = getStepPodName()
job.metadata.labels = { [runnerInstanceLabel.key]: runnerInstanceLabel.value }
job.metadata.annotations = {}
job.spec = new k8s.V1JobSpec()
job.spec.ttlSecondsAfterFinished = 300
@@ -125,9 +149,15 @@ export async function createJob(
job.spec.template = new k8s.V1PodTemplateSpec()
job.spec.template.spec = new k8s.V1PodSpec()
job.spec.template.metadata = new k8s.V1ObjectMeta()
job.spec.template.metadata.labels = {}
job.spec.template.metadata.annotations = {}
job.spec.template.spec.containers = [container]
job.spec.template.spec.restartPolicy = 'Never'
if (!useKubeScheduler()) {
job.spec.template.spec.nodeName = await getCurrentNodeName()
}
const claimName = getVolumeClaimName()
job.spec.template.spec.volumes = [
@@ -137,6 +167,17 @@ export async function createJob(
}
]
if (extension) {
if (extension.metadata) {
// apply metadata both to the job and the pod created by the job
mergeObjectMeta(job, extension.metadata)
mergeObjectMeta(job.spec.template, extension.metadata)
}
if (extension.spec) {
mergePodSpecWithOptions(job.spec.template.spec, extension.spec)
}
}
const { body } = await k8sBatchV1Api.createNamespacedJob(namespace(), job)
return body
}
@@ -186,8 +227,11 @@ export async function execPodStep(
stdin?: stream.Readable
): Promise<void> {
const exec = new k8s.Exec(kc)
await new Promise(async function (resolve, reject) {
await exec.exec(
command = fixArgs(command)
// Exec returns a websocket. If websocket fails, we should reject the promise. Otherwise, websocket will call a callback. Since at that point, websocket is not failing, we can safely resolve or reject the promise.
await new Promise(function (resolve, reject) {
exec
.exec(
namespace(),
podName,
containerName,
@@ -211,6 +255,9 @@ export async function execPodStep(
}
}
)
// If exec.exec fails, explicitly reject the outer promise
// eslint-disable-next-line github/no-then
.catch(e => reject(e))
})
}
@@ -320,7 +367,7 @@ export async function waitForPodPhases(
podName: string,
awaitingPhases: Set<PodPhase>,
backOffPhases: Set<PodPhase>,
maxTimeSeconds = 10 * 60 // 10 min
maxTimeSeconds = DEFAULT_WAIT_FOR_POD_TIME_SECONDS
): Promise<void> {
const backOffManager = new BackOffManager(maxTimeSeconds)
let phase: PodPhase = PodPhase.UNKNOWN
@@ -343,6 +390,25 @@ export async function waitForPodPhases(
}
}
export function getPrepareJobTimeoutSeconds(): number {
const envTimeoutSeconds =
process.env['ACTIONS_RUNNER_PREPARE_JOB_TIMEOUT_SECONDS']
if (!envTimeoutSeconds) {
return DEFAULT_WAIT_FOR_POD_TIME_SECONDS
}
const timeoutSeconds = parseInt(envTimeoutSeconds, 10)
if (!timeoutSeconds || timeoutSeconds <= 0) {
core.warning(
`Prepare job timeout is invalid ("${timeoutSeconds}"): use an int > 0`
)
return DEFAULT_WAIT_FOR_POD_TIME_SECONDS
}
return timeoutSeconds
}
async function getPodPhase(podName: string): Promise<PodPhase> {
const podPhaseLookup = new Set<string>([
PodPhase.PENDING,
@@ -452,7 +518,7 @@ export async function isPodContainerAlpine(
[
'sh',
'-c',
"[ $(cat /etc/*release* | grep -i -e '^ID=*alpine*' -c) != 0 ] || exit 1"
`'[ $(cat /etc/*release* | grep -i -e "^ID=*alpine*" -c) != 0 ] || exit 1'`
],
podName,
containerName
@@ -473,6 +539,7 @@ async function getCurrentNodeName(): Promise<string> {
}
return nodeName
}
export function namespace(): string {
if (process.env['ACTIONS_RUNNER_KUBERNETES_NAMESPACE']) {
return process.env['ACTIONS_RUNNER_KUBERNETES_NAMESPACE']
@@ -516,28 +583,46 @@ class BackOffManager {
export function containerPorts(
container: ContainerInfo
): k8s.V1ContainerPort[] {
// 8080:8080/tcp
const portFormat = /(\d{1,5})(:(\d{1,5}))?(\/(tcp|udp))?/
const ports: k8s.V1ContainerPort[] = []
if (!container.portMappings?.length) {
return ports
}
for (const portDefinition of container.portMappings) {
const submatches = portFormat.exec(portDefinition)
if (!submatches) {
throw new Error(
`Port definition "${portDefinition}" is in incorrect format`
)
const portProtoSplit = portDefinition.split('/')
if (portProtoSplit.length > 2) {
throw new Error(`Unexpected port format: ${portDefinition}`)
}
const port = new k8s.V1ContainerPort()
port.hostPort = Number(submatches[1])
if (submatches[3]) {
port.containerPort = Number(submatches[3])
port.protocol =
portProtoSplit.length === 2 ? portProtoSplit[1].toUpperCase() : 'TCP'
const portSplit = portProtoSplit[0].split(':')
if (portSplit.length > 2) {
throw new Error('ports should have at most one ":" separator')
}
if (submatches[5]) {
port.protocol = submatches[5].toUpperCase()
const parsePort = (p: string): number => {
const num = Number(p)
if (!Number.isInteger(num) || num < 1 || num > 65535) {
throw new Error(`invalid container port: ${p}`)
}
return num
}
if (portSplit.length === 1) {
port.containerPort = parsePort(portSplit[0])
} else {
port.protocol = 'TCP'
port.hostPort = parsePort(portSplit[0])
port.containerPort = parsePort(portSplit[1])
}
ports.push(port)
}
return ports
}
export async function getPodByName(name): Promise<k8s.V1Pod> {
const { body } = await k8sApi.readNamespacedPod(name, namespace())
return body
}

View File

@@ -1,13 +1,20 @@
import * as k8s from '@kubernetes/client-node'
import * as fs from 'fs'
import * as yaml from 'js-yaml'
import * as core from '@actions/core'
import { Mount } from 'hooklib'
import * as path from 'path'
import { v1 as uuidv4 } from 'uuid'
import { POD_VOLUME_NAME } from './index'
import { CONTAINER_EXTENSION_PREFIX } from '../hooks/constants'
import * as shlex from 'shlex'
export const DEFAULT_CONTAINER_ENTRY_POINT_ARGS = [`-f`, `/dev/null`]
export const DEFAULT_CONTAINER_ENTRY_POINT = 'tail'
export const ENV_HOOK_TEMPLATE_PATH = 'ACTIONS_RUNNER_CONTAINER_HOOK_TEMPLATE'
export const ENV_USE_KUBE_SCHEDULER = 'ACTIONS_RUNNER_USE_KUBE_SCHEDULER'
export function containerVolumes(
userMountVolumes: Mount[] = [],
jobContainer = true,
@@ -22,16 +29,23 @@ export function containerVolumes(
const workspacePath = process.env.GITHUB_WORKSPACE as string
if (containerAction) {
const i = workspacePath.lastIndexOf('_work/')
const workspaceRelativePath = workspacePath.slice(i + '_work/'.length)
mounts.push(
{
name: POD_VOLUME_NAME,
mountPath: '/github/workspace',
subPath: workspacePath.substring(workspacePath.indexOf('work/') + 1)
subPath: workspaceRelativePath
},
{
name: POD_VOLUME_NAME,
mountPath: '/github/file_commands',
subPath: workspacePath.substring(workspacePath.indexOf('work/') + 1)
subPath: '_temp/_runner_file_commands'
},
{
name: POD_VOLUME_NAME,
mountPath: '/github/workflow',
subPath: '_temp/_github_workflow'
}
)
return mounts
@@ -109,11 +123,22 @@ export function writeEntryPointScript(
if (environmentVariables && Object.entries(environmentVariables).length) {
const envBuffer: string[] = []
for (const [key, value] of Object.entries(environmentVariables)) {
if (
key.includes(`=`) ||
key.includes(`'`) ||
key.includes(`"`) ||
key.includes(`$`)
) {
throw new Error(
`environment key ${key} is invalid - the key must not contain =, $, ', or "`
)
}
envBuffer.push(
`"${key}=${value
.replace(/\\/g, '\\\\')
.replace(/"/g, '\\"')
.replace(/=/g, '\\=')}"`
.replace(/\$/g, '\\$')
.replace(/`/g, '\\`')}"`
)
}
environmentPrefix = `env ${envBuffer.join(' ')} `
@@ -135,6 +160,119 @@ exec ${environmentPrefix} ${entryPoint} ${
}
}
export function generateContainerName(image: string): string {
const nameWithTag = image.split('/').pop()
const name = nameWithTag?.split(':').at(0)
if (!name) {
throw new Error(`Image definition '${image}' is invalid`)
}
return name
}
// Overwrite or append based on container options
//
// Keep in mind, envs and volumes could be passed as fields in container definition
// so default volume mounts and envs are appended first, and then create options are used
// to append more values
//
// Rest of the fields are just applied
// For example, container.createOptions.container.image is going to overwrite container.image field
export function mergeContainerWithOptions(
base: k8s.V1Container,
from: k8s.V1Container
): void {
for (const [key, value] of Object.entries(from)) {
if (key === 'name') {
if (value !== CONTAINER_EXTENSION_PREFIX + base.name) {
core.warning("Skipping name override: name can't be overwritten")
}
continue
} else if (key === 'image') {
core.warning("Skipping image override: image can't be overwritten")
continue
} else if (key === 'env') {
const envs = value as k8s.V1EnvVar[]
base.env = mergeLists(base.env, envs)
} else if (key === 'volumeMounts' && value) {
const volumeMounts = value as k8s.V1VolumeMount[]
base.volumeMounts = mergeLists(base.volumeMounts, volumeMounts)
} else if (key === 'ports' && value) {
const ports = value as k8s.V1ContainerPort[]
base.ports = mergeLists(base.ports, ports)
} else {
base[key] = value
}
}
}
export function mergePodSpecWithOptions(
base: k8s.V1PodSpec,
from: k8s.V1PodSpec
): void {
for (const [key, value] of Object.entries(from)) {
if (key === 'containers') {
base.containers.push(
...from.containers.filter(
e => !e.name?.startsWith(CONTAINER_EXTENSION_PREFIX)
)
)
} else if (key === 'volumes' && value) {
const volumes = value as k8s.V1Volume[]
base.volumes = mergeLists(base.volumes, volumes)
} else {
base[key] = value
}
}
}
export function mergeObjectMeta(
base: { metadata?: k8s.V1ObjectMeta },
from: k8s.V1ObjectMeta
): void {
if (!base.metadata?.labels || !base.metadata?.annotations) {
throw new Error(
"Can't merge metadata: base.metadata or base.annotations field is undefined"
)
}
if (from?.labels) {
for (const [key, value] of Object.entries(from.labels)) {
if (base.metadata?.labels?.[key]) {
core.warning(`Label ${key} is already defined and will be overwritten`)
}
base.metadata.labels[key] = value
}
}
if (from?.annotations) {
for (const [key, value] of Object.entries(from.annotations)) {
if (base.metadata?.annotations?.[key]) {
core.warning(
`Annotation ${key} is already defined and will be overwritten`
)
}
base.metadata.annotations[key] = value
}
}
}
export function readExtensionFromFile(): k8s.V1PodTemplateSpec | undefined {
const filePath = process.env[ENV_HOOK_TEMPLATE_PATH]
if (!filePath) {
return undefined
}
const doc = yaml.load(fs.readFileSync(filePath, 'utf8'))
if (!doc || typeof doc !== 'object') {
throw new Error(`Failed to parse ${filePath}`)
}
return doc as k8s.V1PodTemplateSpec
}
export function useKubeScheduler(): boolean {
return process.env[ENV_USE_KUBE_SCHEDULER] === 'true'
}
export enum PodPhase {
PENDING = 'Pending',
RUNNING = 'Running',
@@ -143,3 +281,16 @@ export enum PodPhase {
UNKNOWN = 'Unknown',
COMPLETED = 'Completed'
}
function mergeLists<T>(base?: T[], from?: T[]): T[] {
const b: T[] = base || []
if (!from?.length) {
return b
}
b.push(...from)
return b
}
export function fixArgs(args: string[]): string[] {
return shlex.split(args.join(' '))
}

View File

@@ -4,6 +4,7 @@ import {
getSecretName,
getStepPodName,
getVolumeClaimName,
JOB_CONTAINER_NAME,
MAX_POD_NAME_LENGTH,
RunnerInstanceLabel,
STEP_POD_NAME_SUFFIX_LENGTH
@@ -170,4 +171,12 @@ describe('constants', () => {
}
})
})
describe('const values', () => {
it('should have constants set', () => {
expect(JOB_CONTAINER_NAME).toBeTruthy()
expect(MAX_POD_NAME_LENGTH).toBeGreaterThan(0)
expect(STEP_POD_NAME_SUFFIX_LENGTH).toBeGreaterThan(0)
})
})
})

View File

@@ -1,6 +1,15 @@
import * as fs from 'fs'
import { POD_VOLUME_NAME } from '../src/k8s'
import { containerVolumes, writeEntryPointScript } from '../src/k8s/utils'
import * as fs from 'fs'
import { containerPorts, POD_VOLUME_NAME } from '../src/k8s'
import {
containerVolumes,
generateContainerName,
writeEntryPointScript,
mergePodSpecWithOptions,
mergeContainerWithOptions,
readExtensionFromFile,
ENV_HOOK_TEMPLATE_PATH
} from '../src/k8s/utils'
import * as k8s from '@kubernetes/client-node'
import { TestHelper } from './test-setup'
let testHelper: TestHelper
@@ -45,6 +54,81 @@ describe('k8s utils', () => {
).toThrow()
})
it('should throw if environment variable name contains double quote', () => {
expect(() =>
writeEntryPointScript(
'/test',
'sh',
['-e', 'script.sh'],
['/prepend/path'],
{
'SOME"_ENV': 'SOME_VALUE'
}
)
).toThrow()
})
it('should throw if environment variable name contains =', () => {
expect(() =>
writeEntryPointScript(
'/test',
'sh',
['-e', 'script.sh'],
['/prepend/path'],
{
'SOME=ENV': 'SOME_VALUE'
}
)
).toThrow()
})
it('should throw if environment variable name contains single quote', () => {
expect(() =>
writeEntryPointScript(
'/test',
'sh',
['-e', 'script.sh'],
['/prepend/path'],
{
"SOME'_ENV": 'SOME_VALUE'
}
)
).toThrow()
})
it('should throw if environment variable name contains dollar', () => {
expect(() =>
writeEntryPointScript(
'/test',
'sh',
['-e', 'script.sh'],
['/prepend/path'],
{
SOME_$_ENV: 'SOME_VALUE'
}
)
).toThrow()
})
it('should escape double quote, dollar and backslash in environment variable values', () => {
const { runnerPath } = writeEntryPointScript(
'/test',
'sh',
['-e', 'script.sh'],
['/prepend/path'],
{
DQUOTE: '"',
BACK_SLASH: '\\',
DOLLAR: '$'
}
)
expect(fs.existsSync(runnerPath)).toBe(true)
const script = fs.readFileSync(runnerPath, 'utf8')
expect(script).toContain('"DQUOTE=\\"')
expect(script).toContain('"BACK_SLASH=\\\\"')
expect(script).toContain('"DOLLAR=\\$"')
})
it('should return object with containerPath and runnerPath', () => {
const { containerPath, runnerPath } = writeEntryPointScript(
'/test',
@@ -101,28 +185,44 @@ describe('k8s utils', () => {
expect(volumes.find(e => e.mountPath === '/__w')).toBeTruthy()
})
it('should have container action volumes', () => {
it('should always have /github/workflow mount if working on container job or container action', () => {
let volumes = containerVolumes([], true, true)
expect(
volumes.find(e => e.mountPath === '/github/workspace')
).toBeTruthy()
expect(
volumes.find(e => e.mountPath === '/github/file_commands')
).toBeTruthy()
expect(volumes.find(e => e.mountPath === '/github/workflow')).toBeTruthy()
volumes = containerVolumes([], true, false)
expect(volumes.find(e => e.mountPath === '/github/workflow')).toBeTruthy()
volumes = containerVolumes([], false, true)
expect(volumes.find(e => e.mountPath === '/github/workflow')).toBeTruthy()
volumes = containerVolumes([], false, false)
expect(
volumes.find(e => e.mountPath === '/github/workspace')
).toBeTruthy()
expect(
volumes.find(e => e.mountPath === '/github/file_commands')
).toBeTruthy()
volumes.find(e => e.mountPath === '/github/workflow')
).toBeUndefined()
})
it('should have externals, github home and github workflow mounts if job container', () => {
it('should have container action volumes', () => {
let volumes = containerVolumes([], true, true)
let workspace = volumes.find(e => e.mountPath === '/github/workspace')
let fileCommands = volumes.find(
e => e.mountPath === '/github/file_commands'
)
expect(workspace).toBeTruthy()
expect(workspace?.subPath).toBe('repo/repo')
expect(fileCommands).toBeTruthy()
expect(fileCommands?.subPath).toBe('_temp/_runner_file_commands')
volumes = containerVolumes([], false, true)
workspace = volumes.find(e => e.mountPath === '/github/workspace')
fileCommands = volumes.find(e => e.mountPath === '/github/file_commands')
expect(workspace).toBeTruthy()
expect(workspace?.subPath).toBe('repo/repo')
expect(fileCommands).toBeTruthy()
expect(fileCommands?.subPath).toBe('_temp/_runner_file_commands')
})
it('should have externals, github home mounts if job container', () => {
const volumes = containerVolumes()
expect(volumes.find(e => e.mountPath === '/__e')).toBeTruthy()
expect(volumes.find(e => e.mountPath === '/github/home')).toBeTruthy()
expect(volumes.find(e => e.mountPath === '/github/workflow')).toBeTruthy()
})
it('should throw if user volume source volume path is not in workspace', () => {
@@ -149,5 +249,280 @@ describe('k8s utils', () => {
volumes = containerVolumes([], false, false)
expect(volumes.every(e => e.name === POD_VOLUME_NAME)).toBeTruthy()
})
it('should parse container ports', () => {
const tt = [
{
spec: '8080:80',
want: {
containerPort: 80,
hostPort: 8080,
protocol: 'TCP'
}
},
{
spec: '8080:80/udp',
want: {
containerPort: 80,
hostPort: 8080,
protocol: 'UDP'
}
},
{
spec: '8080/udp',
want: {
containerPort: 8080,
hostPort: undefined,
protocol: 'UDP'
}
},
{
spec: '8080',
want: {
containerPort: 8080,
hostPort: undefined,
protocol: 'TCP'
}
}
]
for (const tc of tt) {
const got = containerPorts({ portMappings: [tc.spec] })
for (const [key, value] of Object.entries(tc.want)) {
expect(got[0][key]).toBe(value)
}
}
})
it('should throw when ports are out of range (0, 65536)', () => {
expect(() => containerPorts({ portMappings: ['65536'] })).toThrow()
expect(() => containerPorts({ portMappings: ['0'] })).toThrow()
expect(() => containerPorts({ portMappings: ['65536/udp'] })).toThrow()
expect(() => containerPorts({ portMappings: ['0/udp'] })).toThrow()
expect(() => containerPorts({ portMappings: ['1:65536'] })).toThrow()
expect(() => containerPorts({ portMappings: ['65536:1'] })).toThrow()
expect(() => containerPorts({ portMappings: ['1:65536/tcp'] })).toThrow()
expect(() => containerPorts({ portMappings: ['65536:1/tcp'] })).toThrow()
expect(() => containerPorts({ portMappings: ['1:'] })).toThrow()
expect(() => containerPorts({ portMappings: [':1'] })).toThrow()
expect(() => containerPorts({ portMappings: ['1:/tcp'] })).toThrow()
expect(() => containerPorts({ portMappings: [':1/tcp'] })).toThrow()
})
it('should throw on multi ":" splits', () => {
expect(() => containerPorts({ portMappings: ['1:1:1'] })).toThrow()
})
it('should throw on multi "/" splits', () => {
expect(() => containerPorts({ portMappings: ['1:1/tcp/udp'] })).toThrow()
expect(() => containerPorts({ portMappings: ['1/tcp/udp'] })).toThrow()
})
})
describe('generate container name', () => {
it('should return the container name from image string', () => {
expect(
generateContainerName('public.ecr.aws/localstack/localstack')
).toEqual('localstack')
expect(
generateContainerName(
'public.ecr.aws/url/with/multiple/slashes/postgres:latest'
)
).toEqual('postgres')
expect(generateContainerName('postgres')).toEqual('postgres')
expect(generateContainerName('postgres:latest')).toEqual('postgres')
expect(generateContainerName('localstack/localstack')).toEqual(
'localstack'
)
expect(generateContainerName('localstack/localstack:latest')).toEqual(
'localstack'
)
})
it('should throw on invalid image string', () => {
expect(() =>
generateContainerName('localstack/localstack/:latest')
).toThrow()
expect(() => generateContainerName(':latest')).toThrow()
})
})
describe('read extension', () => {
beforeEach(async () => {
testHelper = new TestHelper()
await testHelper.initialize()
})
afterEach(async () => {
await testHelper.cleanup()
})
it('should throw if env variable is set but file does not exist', () => {
process.env[ENV_HOOK_TEMPLATE_PATH] =
'/path/that/does/not/exist/data.yaml'
expect(() => readExtensionFromFile()).toThrow()
})
it('should return undefined if env variable is not set', () => {
delete process.env[ENV_HOOK_TEMPLATE_PATH]
expect(readExtensionFromFile()).toBeUndefined()
})
it('should throw if file is empty', () => {
let filePath = testHelper.createFile('data.yaml')
process.env[ENV_HOOK_TEMPLATE_PATH] = filePath
expect(() => readExtensionFromFile()).toThrow()
})
it('should throw if file is not valid yaml', () => {
let filePath = testHelper.createFile('data.yaml')
fs.writeFileSync(filePath, 'invalid yaml')
process.env[ENV_HOOK_TEMPLATE_PATH] = filePath
expect(() => readExtensionFromFile()).toThrow()
})
it('should return object if file is valid', () => {
let filePath = testHelper.createFile('data.yaml')
fs.writeFileSync(
filePath,
`
metadata:
labels:
label-name: label-value
annotations:
annotation-name: annotation-value
spec:
containers:
- name: test
image: node:14.16
- name: job
image: ubuntu:latest`
)
process.env[ENV_HOOK_TEMPLATE_PATH] = filePath
const extension = readExtensionFromFile()
expect(extension).toBeDefined()
})
})
it('should merge container spec', () => {
const base = {
image: 'node:14.16',
name: 'test',
env: [
{
name: 'TEST',
value: 'TEST'
}
],
ports: [
{
containerPort: 8080,
hostPort: 8080,
protocol: 'TCP'
}
]
} as k8s.V1Container
const from = {
ports: [
{
containerPort: 9090,
hostPort: 9090,
protocol: 'TCP'
}
],
env: [
{
name: 'TEST_TWO',
value: 'TEST_TWO'
}
],
image: 'ubuntu:latest',
name: 'overwrite'
} as k8s.V1Container
const expectContainer = {
name: base.name,
image: base.image,
ports: [
...(base.ports as k8s.V1ContainerPort[]),
...(from.ports as k8s.V1ContainerPort[])
],
env: [...(base.env as k8s.V1EnvVar[]), ...(from.env as k8s.V1EnvVar[])]
}
const expectJobContainer = JSON.parse(JSON.stringify(expectContainer))
expectJobContainer.name = base.name
mergeContainerWithOptions(base, from)
expect(base).toStrictEqual(expectContainer)
})
it('should merge pod spec', () => {
const base = {
containers: [
{
image: 'node:14.16',
name: 'test',
env: [
{
name: 'TEST',
value: 'TEST'
}
],
ports: [
{
containerPort: 8080,
hostPort: 8080,
protocol: 'TCP'
}
]
}
],
restartPolicy: 'Never'
} as k8s.V1PodSpec
const from = {
securityContext: {
runAsUser: 1000,
fsGroup: 2000
},
restartPolicy: 'Always',
volumes: [
{
name: 'work',
emptyDir: {}
}
],
containers: [
{
image: 'ubuntu:latest',
name: 'side-car',
env: [
{
name: 'TEST',
value: 'TEST'
}
],
ports: [
{
containerPort: 8080,
hostPort: 8080,
protocol: 'TCP'
}
]
}
]
} as k8s.V1PodSpec
const expected = JSON.parse(JSON.stringify(base))
expected.securityContext = from.securityContext
expected.restartPolicy = from.restartPolicy
expected.volumes = from.volumes
expected.containers.push(from.containers[0])
mergePodSpecWithOptions(base, from)
expect(base).toStrictEqual(expected)
})
})

View File

@@ -1,8 +1,18 @@
import * as fs from 'fs'
import * as path from 'path'
import { cleanupJob } from '../src/hooks'
import { prepareJob } from '../src/hooks/prepare-job'
import { createContainerSpec, prepareJob } from '../src/hooks/prepare-job'
import { TestHelper } from './test-setup'
import {
ENV_HOOK_TEMPLATE_PATH,
ENV_USE_KUBE_SCHEDULER,
generateContainerName,
readExtensionFromFile
} from '../src/k8s/utils'
import { getPodByName } from '../src/k8s'
import { V1Container } from '@kubernetes/client-node'
import * as yaml from 'js-yaml'
import { JOB_CONTAINER_NAME } from '../src/hooks/constants'
jest.useRealTimers()
@@ -71,4 +81,94 @@ describe('Prepare job', () => {
prepareJob(prepareJobData.args, prepareJobOutputFilePath)
).rejects.toThrow()
})
it('should not set command + args for service container if not passed in args', async () => {
const services = prepareJobData.args.services.map(service => {
return createContainerSpec(service, generateContainerName(service.image))
}) as [V1Container]
expect(services[0].command).toBe(undefined)
expect(services[0].args).toBe(undefined)
})
it('should determine alpine correctly', async () => {
prepareJobData.args.container.image = 'alpine:latest'
await prepareJob(prepareJobData.args, prepareJobOutputFilePath)
const content = JSON.parse(
fs.readFileSync(prepareJobOutputFilePath).toString()
)
expect(content.isAlpine).toBe(true)
})
it('should run pod with extensions applied', async () => {
process.env[ENV_HOOK_TEMPLATE_PATH] = path.join(
__dirname,
'../../../examples/extension.yaml'
)
await expect(
prepareJob(prepareJobData.args, prepareJobOutputFilePath)
).resolves.not.toThrow()
delete process.env[ENV_HOOK_TEMPLATE_PATH]
const content = JSON.parse(
fs.readFileSync(prepareJobOutputFilePath).toString()
)
const got = await getPodByName(content.state.jobPod)
expect(got.metadata?.annotations?.['annotated-by']).toBe('extension')
expect(got.metadata?.labels?.['labeled-by']).toBe('extension')
expect(got.spec?.securityContext?.runAsUser).toBe(1000)
expect(got.spec?.securityContext?.runAsGroup).toBe(3000)
// job container
expect(got.spec?.containers[0].name).toBe(JOB_CONTAINER_NAME)
expect(got.spec?.containers[0].image).toBe('node:14.16')
expect(got.spec?.containers[0].command).toEqual(['sh'])
expect(got.spec?.containers[0].args).toEqual(['-c', 'sleep 50'])
// service container
expect(got.spec?.containers[1].image).toBe('redis')
expect(got.spec?.containers[1].command).toBeFalsy()
expect(got.spec?.containers[1].args).toBeFalsy()
expect(got.spec?.containers[1].env).toEqual([
{ name: 'ENV2', value: 'value2' }
])
expect(got.spec?.containers[1].resources).toEqual({
requests: { memory: '1Mi', cpu: '1' },
limits: { memory: '1Gi', cpu: '2' }
})
// side-car
expect(got.spec?.containers[2].name).toBe('side-car')
expect(got.spec?.containers[2].image).toBe('ubuntu:latest')
expect(got.spec?.containers[2].command).toEqual(['sh'])
expect(got.spec?.containers[2].args).toEqual(['-c', 'sleep 60'])
})
it('should not throw exception using kube scheduler', async () => {
// only for ReadWriteMany volumes or single node cluster
process.env[ENV_USE_KUBE_SCHEDULER] = 'true'
await expect(
prepareJob(prepareJobData.args, prepareJobOutputFilePath)
).resolves.not.toThrow()
delete process.env[ENV_USE_KUBE_SCHEDULER]
})
test.each([undefined, null, []])(
'should not throw exception when portMapping=%p',
async pm => {
prepareJobData.args.services.forEach(s => {
s.portMappings = pm
})
await prepareJob(prepareJobData.args, prepareJobOutputFilePath)
const content = JSON.parse(
fs.readFileSync(prepareJobOutputFilePath).toString()
)
expect(() => content.context.services[0].image).not.toThrow()
}
)
})

View File

@@ -1,5 +1,9 @@
import { runContainerStep } from '../src/hooks'
import { TestHelper } from './test-setup'
import { ENV_HOOK_TEMPLATE_PATH } from '../src/k8s/utils'
import * as fs from 'fs'
import * as yaml from 'js-yaml'
import { JOB_CONTAINER_EXTENSION_NAME } from '../src/hooks/constants'
jest.useRealTimers()
@@ -23,9 +27,45 @@ describe('Run container step', () => {
expect(exitCode).toBe(0)
})
it('should fail if the working directory does not exist', async () => {
runContainerStepData.args.workingDirectory = '/foo/bar'
await expect(runContainerStep(runContainerStepData.args)).rejects.toThrow()
it('should run pod with extensions applied', async () => {
const extension = {
metadata: {
annotations: {
foo: 'bar'
},
labels: {
bar: 'baz'
}
},
spec: {
containers: [
{
name: JOB_CONTAINER_EXTENSION_NAME,
command: ['sh'],
args: ['-c', 'echo test']
},
{
name: 'side-container',
image: 'ubuntu:latest',
command: ['sh'],
args: ['-c', 'echo test']
}
],
restartPolicy: 'Never',
securityContext: {
runAsUser: 1000,
runAsGroup: 3000
}
}
}
let filePath = testHelper.createFile()
fs.writeFileSync(filePath, yaml.dump(extension))
process.env[ENV_HOOK_TEMPLATE_PATH] = filePath
await expect(
runContainerStep(runContainerStepData.args)
).resolves.not.toThrow()
delete process.env[ENV_HOOK_TEMPLATE_PATH]
})
it('should shold have env variables available', async () => {

View File

@@ -89,6 +89,28 @@ describe('Run script step', () => {
).resolves.not.toThrow()
})
it('Dollar symbols in environment variables should not be expanded', async () => {
runScriptStepDefinition.args.environmentVariables = {
VARIABLE1: '$VAR',
VARIABLE2: '${VAR}',
VARIABLE3: '$(VAR)'
}
runScriptStepDefinition.args.entryPointArgs = [
'-c',
'\'if [[ -z "$VARIABLE1" ]]; then exit 1; fi\'',
'\'if [[ -z "$VARIABLE2" ]]; then exit 2; fi\'',
'\'if [[ -z "$VARIABLE3" ]]; then exit 3; fi\''
]
await expect(
runScriptStep(
runScriptStepDefinition.args,
prepareJobOutputData.state,
null
)
).resolves.not.toThrow()
})
it('Should have path variable changed in container with prepend path string array', async () => {
runScriptStepDefinition.args.prependPath = ['/some/other/path']
runScriptStepDefinition.args.entryPoint = '/bin/bash'

View File

@@ -1,6 +1,20 @@
## Features
- Extend the hook extension to allow service container modifications [#134]
## Bugs
- Fixed an issue where default private registry images did not pull correctly [#25]
- Mount /github/workflow to docker action pods [#137]
- Read logs from failed job container [#135]
## Misc
- Bump jose from 4.11.4 to 4.15.5 [#142]
- ADR-0134 - superseding ADR-0096 [#136]
## SHA-256 Checksums
The SHA-256 checksums for the packages included in this build are shown below:
- actions-runner-hooks-docker-<HOOK_VERSION>.zip <DOCKER_SHA>
- actions-runner-hooks-k8s-<HOOK_VERSION>.zip <K8S_SHA>