actions/runner
1
0
Fork 0
mirror of https://github.com/actions/runner.git synced 2025-12-11 04:46:58 +00:00
Code Issues Packages Projects Releases Wiki Activity

Adding --check to run a serials network test against GitHub or GHES. (#900)

Browse Source 
* add --check.
This commit is contained in:
Tingluo Huang
2021-01-14 13:26:07 -05:00
committed by GitHub
parent 3b34e203dc
commit 7ee333b5cd
22 changed files with 1404 additions and 16 deletions
Download Patch File Download Diff File Expand all files Collapse all files

44
docs/checks/actions.md Normal file
View File

@@ -0,0 +1,44 @@
# Actions Connection Check
## What is this check for?
Make sure the runner has access to actions service for GitHub.com or GitHub Enterprise Server
- For GitHub.com
- The runner needs to access https://api.github.com for downloading actions.
- The runner needs to access https://vstoken.actions.githubusercontent.com/_apis/.../ for requesting an access token.
- The runner needs to access https://pipelines.actions.githubusercontent.com/_apis/.../ for receiving workflow jobs.
- For GitHub Enterprise Server
- The runner needs to access https://myGHES.com/api/v3 for downloading actions.
- The runner needs to access https://myGHES.com/_services/vstoken/_apis/.../ for requesting an access token.
- The runner needs to access https://myGHES.com/_services/pipelines/_apis/.../ for receiving workflow jobs.
## What is checked?
- DNS lookup for api.github.com or myGHES.com using dotnet
- Ping api.github.com or myGHES.com using dotnet
- Make HTTP GET to https://api.github.com or https://myGHES.com/api/v3 using dotnet, check response headers contains `X-GitHub-Request-Id`
---
- DNS lookup for vstoken.actions.githubusercontent.com using dotnet
- Ping vstoken.actions.githubusercontent.com using dotnet
- Make HTTP GET to https://vstoken.actions.githubusercontent.com/_apis/health or https://myGHES.com/_services/vstoken/_apis/health using dotnet, check response headers contains `x-vss-e2eid`
---
- DNS lookup for pipelines.actions.githubusercontent.com using dotnet
- Ping pipelines.actions.githubusercontent.com using dotnet
- Make HTTP GET to https://pipelines.actions.githubusercontent.com/_apis/health or https://myGHES.com/_services/pipelines/_apis/health using dotnet, check response headers contains `x-vss-e2eid`
## How to fix the issue?
### 1. Check the common network issue
> Please check the [network doc](./network.md)
### 2. SSL certificate related issue
If you are seeing `System.Net.Http.HttpRequestException: The SSL connection could not be established, see inner exception.` in the log, it means the runner can't connect to Actions service due to SSL handshake failure.
> Please check the [SSL cert doc](./sslcert.md)
## Still not working?
Contact GitHub customer service or log an issue at https://github.com/actions/runner if you think it's a runner issue.

34
docs/checks/git.md Normal file
View File

@@ -0,0 +1,34 @@
# Git Connection Check
## What is this check for?
Make sure `git` can access GitHub.com or your GitHub Enterprise Server.
## What is checked?
The test is done by executing
```bash
# For GitHub.com
git ls-remote --exit-code https://github.com/actions/checkout HEAD
# For GitHub Enterprise Server
git ls-remote --exit-code https://ghes.me/actions/checkout HEAD
```
The test also set environment variable `GIT_TRACE=1` and `GIT_CURL_VERBOSE=1` before running `git ls-remote`, this will make `git` to produce debug log for better debug any potential issues.
## How to fix the issue?
### 1. Check the common network issue
> Please check the [network doc](./network.md)
### 2. SSL certificate related issue
If you are seeing `SSL Certificate problem:` in the log, it means the `git` can't connect to the GitHub server due to SSL handshake failure.
> Please check the [SSL cert doc](./sslcert.md)
## Still not working?
Contact GitHub customer service or log an issue at https://github.com/actions/runner if you think it's a runner issue.

26
docs/checks/internet.md Normal file
View File

@@ -0,0 +1,26 @@
# Internet Connection Check
## What is this check for?
Make sure the runner has access to https://api.github.com
The runner needs to access https://api.github.com to download any actions from the marketplace.
Even the runner is configured to GitHub Enterprise Server, the runner can still download actions from GitHub.com with [GitHub Connect](https://docs.github.com/en/enterprise-server@2.22/admin/github-actions/enabling-automatic-access-to-githubcom-actions-using-github-connect)
## What is checked?
- DNS lookup for api.github.com using dotnet
- Ping api.github.com using dotnet
- Make HTTP GET to https://api.github.com using dotnet, check response headers contains `X-GitHub-Request-Id`
## How to fix the issue?
### 1. Check the common network issue
> Please check the [network doc](./network.md)
## Still not working?
Contact GitHub customer service or log an issue at https://github.com/actions/runner if you think it's a runner issue.

29
docs/checks/network.md Normal file
View File

@@ -0,0 +1,29 @@
## Common Network Related Issues
### Common things that can cause the runner to not working properly
- Bug in the runner or the dotnet framework that causes actions runner can't make Http request in a certain network environment.
- Proxy/Firewall block certain HTTP method, like it block all POST and PUT calls which the runner will use to upload logs.
- Proxy/Firewall only allows requests with certain user-agent to pass through and the actions runner user-agent is not in the allow list.
- Proxy try to decrypt and exam HTTPS traffic for security purpose but cause the actions-runner to fail to finish SSL handshake due to the lack of trusting proxy's CA.
- Firewall rules that block action runner from accessing certain hosts, ex: `*.github.com`, `*.actions.githubusercontent.com`, etc.
### Identify and solve these problems
The key is to figure out where is the problem, the network environment, or the actions runner?
Use a 3rd party tool to make the same requests as the runner did would be a good start point.
- Use `nslookup` to check DNS
- Use `ping` to check Ping
- Use `curl -v` to check the network stack, good for verifying default certificate/proxy settings.
- Use `Invoke-WebRequest` from `pwsh` (`PowerShell Core`) to check the dotnet network stack, good for verifying bugs in the dotnet framework.
If the 3rd party tool is also experiencing the same error as the runner does, then you might want to contact your network administrator for help.
Otherwise, contact GitHub customer support or log an issue at https://github.com/actions/runner

30
docs/checks/nodejs.md Normal file
View File

@@ -0,0 +1,30 @@
# Node.js Connection Check
## What is this check for?
Make sure the built-in node.js has access to GitHub.com or GitHub Enterprise Server.
The runner carries it's own copy of node.js executable under `<runner_root>/externals/node12/`.
All javascript base Actions will get executed by the built-in `node` at `<runner_root>/externals/node12/`.
> Not the `node` from `$PATH`
## What is checked?
- Make HTTPS GET to https://api.github.com or https://myGHES.com/api/v3 using node.js, make sure it gets 200 response code.
## How to fix the issue?
### 1. Check the common network issue
> Please check the [network doc](./network.md)
### 2. SSL certificate related issue
If you are seeing `Https request failed due to SSL cert issue` in the log, it means the `node.js` can't connect to the GitHub server due to SSL handshake failure.
> Please check the [SSL cert doc](./sslcert.md)
## Still not working?
Contact GitHub customer service or log an issue at https://github.com/actions/runner if you think it's a runner issue.

89
docs/checks/sslcert.md Normal file
View File

@@ -0,0 +1,89 @@
## SSL Certificate Related Issues
You might run into an SSL certificate error when your GitHub Enterprise Server is using a self-signed SSL server certificate or a web proxy within your network is decrypting HTTPS traffic for a security audit.
As long as your certificate is generated properly, most of the issues should be fixed after your trust the certificate properly on the runner machine.
> Different OS might have extra requirements on SSL certificate,
> Ex: macOS requires `ExtendedKeyUsage` https://support.apple.com/en-us/HT210176
### Don't skip SSL cert validation
> !!! DO NOT SKIP SSL CERT VALIDATION !!!
> !!! IT IS A BAD SECURITY PRACTICE !!!
### Download SSL certificate chain
Depends on how your SSL server certificate gets configured, you might need to download the whole certificate chain from a machine that has trusted the SSL certificate's CA.
- Approach 1: Download certificate chain using a browser (Chrome, Firefox, IT), you can google for more example, [here is what I found](https://medium.com/@menakajain/export-download-ssl-certificate-from-server-site-url-bcfc41ea46a2)
- Approach 2: Download certificate chain using OpenSSL, you can google for more example, [here is what I found](https://superuser.com/a/176721)
- Approach 3: Ask your network administrator or the owner of the CA certificate to send you a copy of it
### Trust CA certificate for the Runner
The actions runner is a dotnet core application which will follow how dotnet load SSL CA certificates on each OS.
You can get full details documentation at [here](https://docs.microsoft.com/en-us/dotnet/standard/security/cross-platform-cryptography#x509store)
In short:
- Windows: Load from Windows certificate store.
- Linux: Load from OpenSSL CA cert bundle.
- macOS: Load from macOS KeyChain.
To let the runner trusts your CA certificate, you will need to:
1. Save your SSL certificate chain which includes the root CA and all intermediate CAs into a `.pem` file.
2. Use `OpenSSL` to convert `.pem` file to a proper format for different OS, here is some [doc with sample commands](https://www.sslshopper.com/ssl-converter.html)
3. Trust CA on different OS:
- Windows: https://docs.microsoft.com/en-us/skype-sdk/sdn/articles/installing-the-trusted-root-certificate
- macOS: ![trust ca cert](./../res/macOStrustCA.gif)
- Linux: Refer to the distribution documentation
1. RedHat: https://www.redhat.com/sysadmin/ca-certificates-cli
2. Ubuntu: http://manpages.ubuntu.com/manpages/focal/man8/update-ca-certificates.8.html
3. Google search: "trust ca certificate on [linux distribution]"
4. If all approaches failed, set environment variable `SSL_CERT_FILE` to the CA bundle `.pem` file we get.
> To verity cert gets installed properly on Linux, you can try use `curl -v https://sitewithsslissue.com` and `pwsh -Command \"Invoke-WebRequest -Uri https://sitewithsslissue.com\"`
### Trust CA certificate for Git CLI
Git uses various CA bundle file depends on your operation system.
- Git packaged the CA bundle file within the Git installation on Windows
- Git use OpenSSL certificate CA bundle file on Linux and macOS
You can check where Git check CA file by running:
```bash
export GIT_CURL_VERBOSE=1
git ls-remote https://github.com/actions/runner HEAD
```
You should see something like:
```
* Couldn't find host github.com in the .netrc file; using defaults
* Trying 140.82.114.4...
* TCP_NODELAY set
* Connected to github.com (140.82.114.4) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
* CAfile: /etc/ssl/cert.pem
CApath: none
* SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256
```
This tells me `/etc/ssl/cert.pem` is where it read trusted CA certificates.
To let Git trusts your CA certificate, you will need to:
1. Save your SSL certificate chain which includes the root CA and all intermediate CAs into a `.pem` file.
2. Set `http.sslCAInfo` Git config or `GIT_SSL_CAINFO` environment variable to the full path of the `.pem` file [Git Doc](https://git-scm.com/docs/git-config#Documentation/git-config.txt-httpsslCAInfo)
> I would recommend using `http.sslCAInfo` since it can be scope to certain hosts that need the extra trusted CA.
> Ex: `git config --global http.https://myghes.com/.sslCAInfo /extra/ca/cert.pem`
> This will make Git use the `/extra/ca/cert.pem` only when communicates with `https://myghes.com` and keep using the default CA bundle with others.
### Trust CA certificate for Node.js
Node.js has compiled a snapshot of the Mozilla CA store that is fixed at each version of Node.js' release time.
To let Node.js trusts your CA certificate, you will need to:
1. Save your SSL certificate chain which includes the root CA and all intermediate CAs into a `.pem` file.
2. Set environment variable `NODE_EXTRA_CA_CERTS` which point to the file. ex: `export NODE_EXTRA_CA_CERTS=/full/path/to/cacert.pem` or `set NODE_EXTRA_CA_CERTS=C:\full\path\to\cacert.pem`

BIN
docs/res/macOStrustCA.gif Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 14 MiB

115
src/Misc/layoutbin/checkScripts/downloadCert.js Normal file
View File

@@ -0,0 +1,115 @@
const https = require('https')
const fs = require('fs')
const http = require('http')
const hostname = process.env['HOSTNAME'] || ''
const port = process.env['PORT'] || ''
const path = process.env['PATH'] || ''
const pat = process.env['PAT'] || ''
const proxyHost = process.env['PROXYHOST'] || ''
const proxyPort = process.env['PROXYPORT'] || ''
const proxyUsername = process.env['PROXYUSERNAME'] || ''
const proxyPassword = process.env['PROXYPASSWORD'] || ''
process.env['NODE_TLS_REJECT_UNAUTHORIZED'] = '0'
if (proxyHost === '') {
const options = {
hostname: hostname,
port: port,
path: path,
method: 'GET',
headers: {
'User-Agent': 'GitHubActionsRunnerCheck/1.0',
'Authorization': `token ${pat}`
},
}
const req = https.request(options, res => {
console.log(`statusCode: ${res.statusCode}`)
console.log(`headers: ${JSON.stringify(res.headers)}`)
let cert = socket.getPeerCertificate(true)
let certPEM = ''
let fingerprints = {}
while (cert != null && fingerprints[cert.fingerprint] != '1') {
fingerprints[cert.fingerprint] = '1'
certPEM = certPEM + '-----BEGIN CERTIFICATE-----\n'
let certEncoded = cert.raw.toString('base64')
for (let i = 0; i < certEncoded.length; i++) {
certPEM = certPEM + certEncoded[i]
if (i != certEncoded.length - 1 && (i + 1) % 64 == 0) {
certPEM = certPEM + '\n'
}
}
certPEM = certPEM + '\n-----END CERTIFICATE-----\n'
cert = cert.issuerCertificate
}
console.log(certPEM)
fs.writeFileSync('./download_ca_cert.pem', certPEM)
res.on('data', d => {
process.stdout.write(d)
})
})
req.on('error', error => {
console.error(error)
})
req.end()
}
else {
const auth = 'Basic ' + Buffer.from(proxyUsername + ':' + proxyPassword).toString('base64')
const options = {
host: proxyHost,
port: proxyPort,
method: 'CONNECT',
path: `${hostname}:${port}`,
}
if (proxyUsername != '' || proxyPassword != '') {
options.headers = {
'Proxy-Authorization': auth,
}
}
http.request(options).on('connect', (res, socket) => {
if (res.statusCode != 200) {
throw new Error(`Proxy returns code: ${res.statusCode}`)
}
https.get({
host: hostname,
port: port,
socket: socket,
agent: false,
path: '/',
headers: {
'User-Agent': 'GitHubActionsRunnerCheck/1.0',
'Authorization': `token ${pat}`
}
}, (res) => {
let cert = res.socket.getPeerCertificate(true)
let certPEM = ''
let fingerprints = {}
while (cert != null && fingerprints[cert.fingerprint] != '1') {
fingerprints[cert.fingerprint] = '1'
certPEM = certPEM + '-----BEGIN CERTIFICATE-----\n'
let certEncoded = cert.raw.toString('base64')
for (let i = 0; i < certEncoded.length; i++) {
certPEM = certPEM + certEncoded[i]
if (i != certEncoded.length - 1 && (i + 1) % 64 == 0) {
certPEM = certPEM + '\n'
}
}
certPEM = certPEM + '\n-----END CERTIFICATE-----\n'
cert = cert.issuerCertificate
}
console.log(certPEM)
fs.writeFileSync('./download_ca_cert.pem', certPEM)
console.log(`statusCode: ${res.statusCode}`)
console.log(`headers: ${JSON.stringify(res.headers)}`)
res.on('data', d => {
process.stdout.write(d)
})
})
}).on('error', (err) => {
console.error('error', err)
}).end()
}

75
src/Misc/layoutbin/checkScripts/makeWebRequest.js Normal file
View File

@@ -0,0 +1,75 @@
const https = require('https')
const http = require('http')
const hostname = process.env['HOSTNAME'] || ''
const port = process.env['PORT'] || ''
const path = process.env['PATH'] || ''
const pat = process.env['PAT'] || ''
const proxyHost = process.env['PROXYHOST'] || ''
const proxyPort = process.env['PROXYPORT'] || ''
const proxyUsername = process.env['PROXYUSERNAME'] || ''
const proxyPassword = process.env['PROXYPASSWORD'] || ''
if (proxyHost === '') {
const options = {
hostname: hostname,
port: port,
path: path,
method: 'GET',
headers: {
'User-Agent': 'GitHubActionsRunnerCheck/1.0',
'Authorization': `token ${pat}`,
}
}
const req = https.request(options, res => {
console.log(`statusCode: ${res.statusCode}`)
console.log(`headers: ${JSON.stringify(res.headers)}`)
res.on('data', d => {
process.stdout.write(d)
})
})
req.on('error', error => {
console.error(error)
})
req.end()
}
else {
const proxyAuth = 'Basic ' + Buffer.from(proxyUsername + ':' + proxyPassword).toString('base64')
const options = {
hostname: proxyHost,
port: proxyPort,
method: 'CONNECT',
path: `${hostname}:${port}`
}
if (proxyUsername != '' || proxyPassword != '') {
options.headers = {
'Proxy-Authorization': proxyAuth,
}
}
http.request(options).on('connect', (res, socket) => {
if (res.statusCode != 200) {
throw new Error(`Proxy returns code: ${res.statusCode}`)
}
https.get({
host: hostname,
port: port,
socket: socket,
agent: false,
path: path,
headers: {
'User-Agent': 'GitHubActionsRunnerCheck/1.0',
'Authorization': `token ${pat}`,
}
}, (res) => {
console.log(`statusCode: ${res.statusCode}`)
console.log(`headers: ${JSON.stringify(res.headers)}`)
res.on('data', d => {
process.stdout.write(d)
})
})
}).on('error', (err) => {
console.error('error', err)
}).end()
}

1
src/Runner.Common/Constants.cs
View File

@@ -121,6 +121,7 @@ namespace GitHub.Runner.Common
//validFlags array as well present in the CommandSettings.cs //validFlags array as well present in the CommandSettings.cs
public static class Flags public static class Flags
{ {
public static readonly string Check = "check";
public static readonly string Commit = "commit"; public static readonly string Commit = "commit";
public static readonly string Help = "help"; public static readonly string Help = "help";
public static readonly string Replace = "replace"; public static readonly string Replace = "replace";

6
src/Runner.Common/ExtensionManager.cs
View File

@@ -60,6 +60,12 @@ namespace GitHub.Runner.Common
Add<T>(extensions, "GitHub.Runner.Worker.AddPathFileCommand, Runner.Worker"); Add<T>(extensions, "GitHub.Runner.Worker.AddPathFileCommand, Runner.Worker");
Add<T>(extensions, "GitHub.Runner.Worker.SetEnvFileCommand, Runner.Worker"); Add<T>(extensions, "GitHub.Runner.Worker.SetEnvFileCommand, Runner.Worker");
break; break;
case "GitHub.Runner.Listener.Check.ICheckExtension":
Add<T>(extensions, "GitHub.Runner.Listener.Check.InternetCheck, Runner.Listener");
Add<T>(extensions, "GitHub.Runner.Listener.Check.ActionsCheck, Runner.Listener");
Add<T>(extensions, "GitHub.Runner.Listener.Check.GitCheck, Runner.Listener");
Add<T>(extensions, "GitHub.Runner.Listener.Check.NodeJsCheck, Runner.Listener");
break;
default: default:
// This should never happen. // This should never happen.
throw new NotSupportedException($"Unexpected extension type: '{typeof(T).FullName}'"); throw new NotSupportedException($"Unexpected extension type: '{typeof(T).FullName}'");

90
src/Runner.Listener/Checks/ActionsCheck.cs Normal file
View File

@@ -0,0 +1,90 @@
using System;
using System.Collections.Generic;
using System.IO;
using System.Threading.Tasks;
using GitHub.Runner.Common;
using GitHub.Runner.Sdk;
namespace GitHub.Runner.Listener.Check
{
public sealed class ActionsCheck : RunnerService, ICheckExtension
{
private string _logFile = null;
public int Order => 2;
public string CheckName => "GitHub Actions Connection";
public string CheckDescription => "Make sure the actions runner have access to the GitHub Actions Service.";
public string CheckLog => _logFile;
public string HelpLink => "https://github.com/actions/runner/blob/main/docs/checks/actions.md";
public Type ExtensionType => typeof(ICheckExtension);
public override void Initialize(IHostContext hostContext)
{
base.Initialize(hostContext);
_logFile = Path.Combine(HostContext.GetDirectory(WellKnownDirectory.Diag), StringUtil.Format("{0}_{1:yyyyMMdd-HHmmss}-utc.log", nameof(ActionsCheck), DateTime.UtcNow));
}
// runner access to actions service
public async Task<bool> RunCheck(string url, string pat)
{
await File.AppendAllLinesAsync(_logFile, HostContext.WarnLog());
await File.AppendAllLinesAsync(_logFile, HostContext.CheckProxy());
var checkTasks = new List<Task<CheckResult>>();
string githubApiUrl = null;
string actionsTokenServiceUrl = null;
string actionsPipelinesServiceUrl = null;
var urlBuilder = new UriBuilder(url);
if (UrlUtil.IsHostedServer(urlBuilder))
{
urlBuilder.Host = $"api.{urlBuilder.Host}";
urlBuilder.Path = "";
githubApiUrl = urlBuilder.Uri.AbsoluteUri;
actionsTokenServiceUrl = "https://vstoken.actions.githubusercontent.com/_apis/health";
actionsPipelinesServiceUrl = "https://pipelines.actions.githubusercontent.com/_apis/health";
}
else
{
urlBuilder.Path = "api/v3";
githubApiUrl = urlBuilder.Uri.AbsoluteUri;
urlBuilder.Path = "_services/vstoken/_apis/health";
actionsTokenServiceUrl = urlBuilder.Uri.AbsoluteUri;
urlBuilder.Path = "_services/pipelines/_apis/health";
actionsPipelinesServiceUrl = urlBuilder.Uri.AbsoluteUri;
}
// check github api
checkTasks.Add(CheckUtil.CheckDns(githubApiUrl));
checkTasks.Add(CheckUtil.CheckPing(githubApiUrl));
checkTasks.Add(HostContext.CheckHttpsRequests(githubApiUrl, pat, expectedHeader: "X-GitHub-Request-Id"));
// check actions token service
checkTasks.Add(CheckUtil.CheckDns(actionsTokenServiceUrl));
checkTasks.Add(CheckUtil.CheckPing(actionsTokenServiceUrl));
checkTasks.Add(HostContext.CheckHttpsRequests(actionsTokenServiceUrl, pat, expectedHeader: "x-vss-e2eid"));
// check actions pipelines service
checkTasks.Add(CheckUtil.CheckDns(actionsPipelinesServiceUrl));
checkTasks.Add(CheckUtil.CheckPing(actionsPipelinesServiceUrl));
checkTasks.Add(HostContext.CheckHttpsRequests(actionsPipelinesServiceUrl, pat, expectedHeader: "x-vss-e2eid"));
var result = true;
while (checkTasks.Count > 0)
{
var finishedCheckTask = await Task.WhenAny<CheckResult>(checkTasks);
var finishedCheck = await finishedCheckTask;
result = result && finishedCheck.Pass;
await File.AppendAllLinesAsync(_logFile, finishedCheck.Logs);
checkTasks.Remove(finishedCheckTask);
}
await Task.WhenAll(checkTasks);
return result;
}
}
}

351
src/Runner.Listener/Checks/CheckUtil.cs Normal file
View File

@@ -0,0 +1,351 @@
using System;
using System.Collections.Generic;
using System.Diagnostics.Tracing;
using System.IO;
using System.Linq;
using System.Net;
using System.Net.Http;
using System.Net.Http.Headers;
using System.Net.NetworkInformation;
using System.Threading;
using System.Threading.Tasks;
using GitHub.Runner.Common;
using GitHub.Runner.Sdk;
using GitHub.Services.Common;
namespace GitHub.Runner.Listener.Check
{
public static class CheckUtil
{
public static List<string> WarnLog(this IHostContext hostContext)
{
var logs = new List<string>();
logs.Add($"{DateTime.UtcNow.ToString("O")} ***************************************************************************************************************");
logs.Add($"{DateTime.UtcNow.ToString("O")} **** ****");
logs.Add($"{DateTime.UtcNow.ToString("O")} **** !!! WARNING !!! ");
logs.Add($"{DateTime.UtcNow.ToString("O")} **** DO NOT share the log in public place! The log may contains secrets in plain text. ");
logs.Add($"{DateTime.UtcNow.ToString("O")} **** !!! WARNING !!! ");
logs.Add($"{DateTime.UtcNow.ToString("O")} **** ****");
logs.Add($"{DateTime.UtcNow.ToString("O")} ***************************************************************************************************************");
return logs;
}
public static List<string> CheckProxy(this IHostContext hostContext)
{
var logs = new List<string>();
if (!string.IsNullOrEmpty(hostContext.WebProxy.HttpProxyAddress) ||
!string.IsNullOrEmpty(hostContext.WebProxy.HttpsProxyAddress))
{
logs.Add($"{DateTime.UtcNow.ToString("O")} ***************************************************************************************************************");
logs.Add($"{DateTime.UtcNow.ToString("O")} **** ****");
logs.Add($"{DateTime.UtcNow.ToString("O")} **** Runner is behind web proxy {hostContext.WebProxy.HttpsProxyAddress ?? hostContext.WebProxy.HttpProxyAddress} ");
logs.Add($"{DateTime.UtcNow.ToString("O")} **** ****");
logs.Add($"{DateTime.UtcNow.ToString("O")} ***************************************************************************************************************");
}
return logs;
}
public static async Task<CheckResult> CheckDns(string targetUrl)
{
var result = new CheckResult();
var url = new Uri(targetUrl);
try
{
result.Logs.Add($"{DateTime.UtcNow.ToString("O")} ***************************************************************************************************************");
result.Logs.Add($"{DateTime.UtcNow.ToString("O")} **** ****");
result.Logs.Add($"{DateTime.UtcNow.ToString("O")} **** Try DNS lookup for {url.Host} ");
result.Logs.Add($"{DateTime.UtcNow.ToString("O")} **** ****");
result.Logs.Add($"{DateTime.UtcNow.ToString("O")} ***************************************************************************************************************");
IPHostEntry host = await Dns.GetHostEntryAsync(url.Host);
foreach (var address in host.AddressList)
{
result.Logs.Add($"{DateTime.UtcNow.ToString("O")} Resolved DNS for {url.Host} to '{address}'");
}
result.Pass = true;
}
catch (Exception ex)
{
result.Pass = false;
result.Logs.Add($"{DateTime.UtcNow.ToString("O")} ***************************************************************************************************************");
result.Logs.Add($"{DateTime.UtcNow.ToString("O")} **** ****");
result.Logs.Add($"{DateTime.UtcNow.ToString("O")} **** Resolved DNS for {url.Host} failed with error: {ex}");
result.Logs.Add($"{DateTime.UtcNow.ToString("O")} **** ****");
result.Logs.Add($"{DateTime.UtcNow.ToString("O")} ***************************************************************************************************************");
}
return result;
}
public static async Task<CheckResult> CheckPing(string targetUrl)
{
var result = new CheckResult();
var url = new Uri(targetUrl);
try
{
result.Logs.Add($"{DateTime.UtcNow.ToString("O")} ***************************************************************************************************************");
result.Logs.Add($"{DateTime.UtcNow.ToString("O")} **** ****");
result.Logs.Add($"{DateTime.UtcNow.ToString("O")} **** Try ping {url.Host} ");
result.Logs.Add($"{DateTime.UtcNow.ToString("O")} **** ****");
result.Logs.Add($"{DateTime.UtcNow.ToString("O")} ***************************************************************************************************************");
using (var ping = new Ping())
{
var reply = await ping.SendPingAsync(url.Host);
if (reply.Status == IPStatus.Success)
{
result.Pass = true;
result.Logs.Add($"{DateTime.UtcNow.ToString("O")} Ping {url.Host} ({reply.Address}) succeed within to '{reply.RoundtripTime} ms'");
}
else
{
result.Pass = false;
result.Logs.Add($"{DateTime.UtcNow.ToString("O")} Ping {url.Host} ({reply.Address}) failed with '{reply.Status}'");
}
}
}
catch (Exception ex)
{
result.Pass = false;
result.Logs.Add($"{DateTime.UtcNow.ToString("O")} ***************************************************************************************************************");
result.Logs.Add($"{DateTime.UtcNow.ToString("O")} **** ****");
result.Logs.Add($"{DateTime.UtcNow.ToString("O")} **** Ping api.github.com failed with error: {ex}");
result.Logs.Add($"{DateTime.UtcNow.ToString("O")} **** ****");
result.Logs.Add($"{DateTime.UtcNow.ToString("O")} ***************************************************************************************************************");
}
return result;
}
public static async Task<CheckResult> CheckHttpsRequests(this IHostContext hostContext, string url, string pat, string expectedHeader)
{
var result = new CheckResult();
try
{
result.Logs.Add($"{DateTime.UtcNow.ToString("O")} ***************************************************************************************************************");
result.Logs.Add($"{DateTime.UtcNow.ToString("O")} **** ****");
result.Logs.Add($"{DateTime.UtcNow.ToString("O")} **** Send HTTPS Request to {url} ");
result.Logs.Add($"{DateTime.UtcNow.ToString("O")} **** ****");
result.Logs.Add($"{DateTime.UtcNow.ToString("O")} ***************************************************************************************************************");
using (var _ = new HttpEventSourceListener(result.Logs))
using (var httpClientHandler = hostContext.CreateHttpClientHandler())
using (var httpClient = new HttpClient(httpClientHandler))
{
httpClient.DefaultRequestHeaders.UserAgent.AddRange(hostContext.UserAgents);
if (!string.IsNullOrEmpty(pat))
{
httpClient.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("token", pat);
}
var response = await httpClient.GetAsync(url);
result.Logs.Add($"{DateTime.UtcNow.ToString("O")} Http status code: {response.StatusCode}");
result.Logs.Add($"{DateTime.UtcNow.ToString("O")} Http response headers: {response.Headers}");
var responseContent = await response.Content.ReadAsStringAsync();
result.Logs.Add($"{DateTime.UtcNow.ToString("O")} Http response body: {responseContent}");
if (response.IsSuccessStatusCode)
{
if (response.Headers.Contains(expectedHeader))
{
result.Pass = true;
result.Logs.Add($"{DateTime.UtcNow.ToString("O")} ***************************************************************************************************************");
result.Logs.Add($"{DateTime.UtcNow.ToString("O")} Http request 'GET' to {url} succeed");
result.Logs.Add($"{DateTime.UtcNow.ToString("O")} ***************************************************************************************************************");
result.Logs.Add($"{DateTime.UtcNow.ToString("O")} ");
result.Logs.Add($"{DateTime.UtcNow.ToString("O")} ");
}
else
{
result.Pass = false;
result.Logs.Add($"{DateTime.UtcNow.ToString("O")} ***************************************************************************************************************");
result.Logs.Add($"{DateTime.UtcNow.ToString("O")} Http request 'GET' to {url} succeed but doesn't have expected HTTP Header.");
result.Logs.Add($"{DateTime.UtcNow.ToString("O")} ***************************************************************************************************************");
result.Logs.Add($"{DateTime.UtcNow.ToString("O")} ");
result.Logs.Add($"{DateTime.UtcNow.ToString("O")} ");
}
}
else
{
result.Pass = false;
result.Logs.Add($"{DateTime.UtcNow.ToString("O")} ***************************************************************************************************************");
result.Logs.Add($"{DateTime.UtcNow.ToString("O")} Http request 'GET' to {url} failed with {response.StatusCode}");
result.Logs.Add($"{DateTime.UtcNow.ToString("O")} ***************************************************************************************************************");
result.Logs.Add($"{DateTime.UtcNow.ToString("O")} ");
result.Logs.Add($"{DateTime.UtcNow.ToString("O")} ");
}
}
}
catch (Exception ex)
{
result.Pass = false;
result.Logs.Add($"{DateTime.UtcNow.ToString("O")} ***************************************************************************************************************");
result.Logs.Add($"{DateTime.UtcNow.ToString("O")} **** ****");
result.Logs.Add($"{DateTime.UtcNow.ToString("O")} **** Https request 'GET' to {url} failed with error: {ex}");
result.Logs.Add($"{DateTime.UtcNow.ToString("O")} **** ****");
result.Logs.Add($"{DateTime.UtcNow.ToString("O")} ***************************************************************************************************************");
}
return result;
}
public static async Task<CheckResult> DownloadExtraCA(this IHostContext hostContext, string url, string pat)
{
var result = new CheckResult();
try
{
result.Logs.Add($"{DateTime.UtcNow.ToString("O")} ***************************************************************************************************************");
result.Logs.Add($"{DateTime.UtcNow.ToString("O")} **** ****");
result.Logs.Add($"{DateTime.UtcNow.ToString("O")} **** Download SSL Certificate from {url} ");
result.Logs.Add($"{DateTime.UtcNow.ToString("O")} **** ****");
result.Logs.Add($"{DateTime.UtcNow.ToString("O")} ***************************************************************************************************************");
var uri = new Uri(url);
var env = new Dictionary<string, string>()
{
{ "HOSTNAME", uri.Host },
{ "PORT", uri.IsDefaultPort ? (uri.Scheme.ToLowerInvariant() == "https" ? "443" : "80") : uri.Port.ToString() },
{ "PATH", uri.AbsolutePath },
{ "PAT", pat }
};
var proxy = hostContext.WebProxy.GetProxy(uri);
if (proxy != null)
{
env["PROXYHOST"] = proxy.Host;
env["PROXYPORT"] = proxy.IsDefaultPort ? (proxy.Scheme.ToLowerInvariant() == "https" ? "443" : "80") : proxy.Port.ToString();
if (hostContext.WebProxy.HttpProxyUsername != null ||
hostContext.WebProxy.HttpsProxyUsername != null)
{
env["PROXYUSERNAME"] = hostContext.WebProxy.HttpProxyUsername ?? hostContext.WebProxy.HttpsProxyUsername;
env["PROXYPASSWORD"] = hostContext.WebProxy.HttpProxyPassword ?? hostContext.WebProxy.HttpsProxyPassword;
}
else
{
env["PROXYUSERNAME"] = "";
env["PROXYPASSWORD"] = "";
}
}
else
{
env["PROXYHOST"] = "";
env["PROXYPORT"] = "";
env["PROXYUSERNAME"] = "";
env["PROXYPASSWORD"] = "";
}
using (var processInvoker = hostContext.CreateService<IProcessInvoker>())
{
processInvoker.OutputDataReceived += new EventHandler<ProcessDataReceivedEventArgs>((sender, args) =>
{
if (!string.IsNullOrEmpty(args.Data))
{
result.Logs.Add($"{DateTime.UtcNow.ToString("O")} [STDOUT] {args.Data}");
}
});
processInvoker.ErrorDataReceived += new EventHandler<ProcessDataReceivedEventArgs>((sender, args) =>
{
if (!string.IsNullOrEmpty(args.Data))
{
result.Logs.Add($"{DateTime.UtcNow.ToString("O")} [STDERR] {args.Data}");
}
});
var downloadCertScript = Path.Combine(hostContext.GetDirectory(WellKnownDirectory.Bin), "checkScripts", "downloadCert");
var node12 = Path.Combine(hostContext.GetDirectory(WellKnownDirectory.Externals), "node12", "bin", $"node{IOUtil.ExeExtension}");
result.Logs.Add($"{DateTime.UtcNow.ToString("O")} Run '{node12} \"{downloadCertScript}\"' ");
result.Logs.Add($"{DateTime.UtcNow.ToString("O")} {StringUtil.ConvertToJson(env)}");
await processInvoker.ExecuteAsync(
hostContext.GetDirectory(WellKnownDirectory.Root),
node12,
$"\"{downloadCertScript}\"",
env,
true,
CancellationToken.None);
}
result.Pass = true;
}
catch (Exception ex)
{
result.Pass = false;
result.Logs.Add($"{DateTime.UtcNow.ToString("O")} ***************************************************************************************************************");
result.Logs.Add($"{DateTime.UtcNow.ToString("O")} **** ****");
result.Logs.Add($"{DateTime.UtcNow.ToString("O")} **** Download SSL Certificate from '{url}' failed with error: {ex}");
result.Logs.Add($"{DateTime.UtcNow.ToString("O")} **** ****");
result.Logs.Add($"{DateTime.UtcNow.ToString("O")} ***************************************************************************************************************");
}
return result;
}
}
// EventSource listener for dotnet debug trace for HTTP and SSL
public sealed class HttpEventSourceListener : EventListener
{
private readonly List<string> _logs;
private readonly object _lock = new object();
private readonly Dictionary<string, HashSet<string>> _ignoredEvent = new Dictionary<string, HashSet<string>>
{
{
"Private.InternalDiagnostics.System.Net.Http",
new HashSet<string>
{
"Info",
"Associate"
}
},
{
"Private.InternalDiagnostics.System.Net.Security",
new HashSet<string>
{
"Info",
"SslStreamCtor",
"SecureChannelCtor",
"NoDelegateNoClientCert",
"CertsAfterFiltering",
"UsingCachedCredential",
"SspiSelectedCipherSuite"
}
}
};
public HttpEventSourceListener(List<string> logs)
{
_logs = logs;
if (Environment.GetEnvironmentVariable("ACTIONS_RUNNER_TRACE_ALL_HTTP_EVENT") == "1")
{
_ignoredEvent.Clear();
}
}
protected override void OnEventSourceCreated(EventSource eventSource)
{
base.OnEventSourceCreated(eventSource);
if (eventSource.Name == "Private.InternalDiagnostics.System.Net.Http" ||
eventSource.Name == "Private.InternalDiagnostics.System.Net.Security")
{
EnableEvents(eventSource, EventLevel.Verbose, EventKeywords.All);
}
}
protected override void OnEventWritten(EventWrittenEventArgs eventData)
{
base.OnEventWritten(eventData);
lock (_lock)
{
if (_ignoredEvent.TryGetValue(eventData.EventSource.Name, out var ignored) &&
ignored.Contains(eventData.EventName))
{
return;
}
_logs.Add($"{DateTime.UtcNow.ToString("O")} [START {eventData.EventSource.Name} - {eventData.EventName}]");
_logs.AddRange(eventData.Payload.Select(x => string.Join(Environment.NewLine, x.ToString().Split(Environment.NewLine).Select(y => $"{DateTime.UtcNow.ToString("O")} {y}"))));
_logs.Add($"{DateTime.UtcNow.ToString("O")} [END {eventData.EventSource.Name} - {eventData.EventName}]");
}
}
}
}

171
src/Runner.Listener/Checks/GitCheck.cs Normal file
View File

@@ -0,0 +1,171 @@
using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Net;
using System.Threading;
using System.Threading.Tasks;
using GitHub.Runner.Common;
using GitHub.Runner.Sdk;
namespace GitHub.Runner.Listener.Check
{
public sealed class GitCheck : RunnerService, ICheckExtension
{
private string _logFile = null;
private string _gitPath = null;
public int Order => 3;
public string CheckName => "Git Certificate/Proxy Validation";
public string CheckDescription => "Make sure the git cli can access to GitHub.com or the GitHub Enterprise Server.";
public string CheckLog => _logFile;
public string HelpLink => "https://github.com/actions/runner/blob/main/docs/checks/git.md";
public Type ExtensionType => typeof(ICheckExtension);
public override void Initialize(IHostContext hostContext)
{
base.Initialize(hostContext);
_logFile = Path.Combine(HostContext.GetDirectory(WellKnownDirectory.Diag), StringUtil.Format("{0}_{1:yyyyMMdd-HHmmss}-utc.log", nameof(GitCheck), DateTime.UtcNow));
_gitPath = WhichUtil.Which("git");
}
// git access to ghes/gh
public async Task<bool> RunCheck(string url, string pat)
{
await File.AppendAllLinesAsync(_logFile, HostContext.WarnLog());
await File.AppendAllLinesAsync(_logFile, HostContext.CheckProxy());
if (string.IsNullOrEmpty(_gitPath))
{
await File.AppendAllLinesAsync(_logFile, new[] { $"{DateTime.UtcNow.ToString("O")} Can't verify git with GitHub.com or GitHub Enterprise Server since git is not installed." });
return false;
}
var checkGit = await CheckGit(url, pat);
var result = checkGit.Pass;
await File.AppendAllLinesAsync(_logFile, checkGit.Logs);
// try fix SSL error by providing extra CA certificate.
if (checkGit.SslError)
{
await File.AppendAllLinesAsync(_logFile, new[] { $"{DateTime.UtcNow.ToString("O")} Try fix SSL error by providing extra CA certificate." });
var downloadCert = await HostContext.DownloadExtraCA(url, pat);
await File.AppendAllLinesAsync(_logFile, downloadCert.Logs);
if (downloadCert.Pass)
{
var recheckGit = await CheckGit(url, pat, extraCA: true);
await File.AppendAllLinesAsync(_logFile, recheckGit.Logs);
if (recheckGit.Pass)
{
await File.AppendAllLinesAsync(_logFile, new[] { $"{DateTime.UtcNow.ToString("O")} Fixed SSL error by providing extra CA certs." });
}
}
}
return result;
}
private async Task<CheckResult> CheckGit(string url, string pat, bool extraCA = false)
{
var result = new CheckResult();
try
{
result.Logs.Add($"{DateTime.UtcNow.ToString("O")} ***************************************************************************************************************");
result.Logs.Add($"{DateTime.UtcNow.ToString("O")} **** ****");
result.Logs.Add($"{DateTime.UtcNow.ToString("O")} **** Validate server cert and proxy configuration with Git ");
result.Logs.Add($"{DateTime.UtcNow.ToString("O")} **** ****");
result.Logs.Add($"{DateTime.UtcNow.ToString("O")} ***************************************************************************************************************");
var repoUrlBuilder = new UriBuilder(url);
repoUrlBuilder.Path = "actions/checkout";
repoUrlBuilder.UserName = "gh";
repoUrlBuilder.Password = pat;
var gitProxy = "";
var proxy = HostContext.WebProxy.GetProxy(repoUrlBuilder.Uri);
if (proxy != null)
{
result.Logs.Add($"{DateTime.UtcNow.ToString("O")} Runner is behind http proxy '{proxy.AbsoluteUri}'");
if (HostContext.WebProxy.HttpProxyUsername != null ||
HostContext.WebProxy.HttpsProxyUsername != null)
{
var proxyUrlWithCred = UrlUtil.GetCredentialEmbeddedUrl(
proxy,
HostContext.WebProxy.HttpProxyUsername ?? HostContext.WebProxy.HttpsProxyUsername,
HostContext.WebProxy.HttpProxyPassword ?? HostContext.WebProxy.HttpsProxyPassword);
gitProxy = $"-c http.proxy={proxyUrlWithCred}";
}
else
{
gitProxy = $"-c http.proxy={proxy.AbsoluteUri}";
}
}
using (var processInvoker = HostContext.CreateService<IProcessInvoker>())
{
processInvoker.OutputDataReceived += new EventHandler<ProcessDataReceivedEventArgs>((sender, args) =>
{
if (!string.IsNullOrEmpty(args.Data))
{
result.Logs.Add($"{DateTime.UtcNow.ToString("O")} {args.Data}");
}
});
processInvoker.ErrorDataReceived += new EventHandler<ProcessDataReceivedEventArgs>((sender, args) =>
{
if (!string.IsNullOrEmpty(args.Data))
{
result.Logs.Add($"{DateTime.UtcNow.ToString("O")} {args.Data}");
}
});
var gitArgs = $"{gitProxy} ls-remote --exit-code {repoUrlBuilder.Uri.AbsoluteUri} HEAD";
result.Logs.Add($"{DateTime.UtcNow.ToString("O")} Run 'git {gitArgs}' ");
var env = new Dictionary<string, string>
{
{ "GIT_TRACE", "1" },
{ "GIT_CURL_VERBOSE", "1" }
};
if (extraCA)
{
env["GIT_SSL_CAINFO"] = Path.Combine(HostContext.GetDirectory(WellKnownDirectory.Root), "download_ca_cert.pem");
}
await processInvoker.ExecuteAsync(
HostContext.GetDirectory(WellKnownDirectory.Root),
_gitPath,
gitArgs,
env,
true,
CancellationToken.None);
}
result.Pass = true;
}
catch (Exception ex)
{
result.Pass = false;
result.Logs.Add($"{DateTime.UtcNow.ToString("O")} ***************************************************************************************************************");
result.Logs.Add($"{DateTime.UtcNow.ToString("O")} **** ****");
result.Logs.Add($"{DateTime.UtcNow.ToString("O")} **** git ls-remote failed with error: {ex}");
if (result.Logs.Any(x => x.Contains("SSL Certificate problem", StringComparison.OrdinalIgnoreCase)))
{
result.Logs.Add($"{DateTime.UtcNow.ToString("O")} **** git ls-remote failed due to SSL cert issue.");
result.SslError = true;
}
result.Logs.Add($"{DateTime.UtcNow.ToString("O")} **** ****");
result.Logs.Add($"{DateTime.UtcNow.ToString("O")} ***************************************************************************************************************");
}
return result;
}
}
}

30
src/Runner.Listener/Checks/ICheckExtension.cs Normal file
View File

@@ -0,0 +1,30 @@
using System.Collections.Generic;
using System.Threading.Tasks;
using GitHub.Runner.Common;
namespace GitHub.Runner.Listener.Check
{
public interface ICheckExtension : IExtension
{
int Order { get; }
string CheckName { get; }
string CheckDescription { get; }
string CheckLog { get; }
string HelpLink { get; }
Task<bool> RunCheck(string url, string pat);
}
public class CheckResult
{
public CheckResult()
{
Logs = new List<string>();
}
public bool Pass { get; set; }
public bool SslError { get; set; }
public List<string> Logs { get; set; }
}
}

59
src/Runner.Listener/Checks/InternetCheck.cs Normal file
View File

@@ -0,0 +1,59 @@
using System;
using System.Collections.Generic;
using System.IO;
using System.Threading.Tasks;
using GitHub.Runner.Common;
using GitHub.Runner.Sdk;
namespace GitHub.Runner.Listener.Check
{
public sealed class InternetCheck : RunnerService, ICheckExtension
{
private string _logFile = null;
public int Order => 1;
public string CheckName => "Internet Connection";
public string CheckDescription => "Make sure the actions runner have access to public internet.";
public string CheckLog => _logFile;
public string HelpLink => "https://github.com/actions/runner/blob/main/docs/checks/internet.md";
public Type ExtensionType => typeof(ICheckExtension);
public override void Initialize(IHostContext hostContext)
{
base.Initialize(hostContext);
_logFile = Path.Combine(HostContext.GetDirectory(WellKnownDirectory.Diag), StringUtil.Format("{0}_{1:yyyyMMdd-HHmmss}-utc.log", nameof(InternetCheck), DateTime.UtcNow));
}
// check runner access to api.github.com
public async Task<bool> RunCheck(string url, string pat)
{
await File.AppendAllLinesAsync(_logFile, HostContext.WarnLog());
await File.AppendAllLinesAsync(_logFile, HostContext.CheckProxy());
var checkTasks = new List<Task<CheckResult>>();
checkTasks.Add(CheckUtil.CheckDns("https://api.github.com"));
checkTasks.Add(CheckUtil.CheckPing("https://api.github.com"));
// We don't need to pass a PAT since it might be a token for GHES.
checkTasks.Add(HostContext.CheckHttpsRequests("https://api.github.com", pat: null, expectedHeader: "X-GitHub-Request-Id"));
var result = true;
while (checkTasks.Count > 0)
{
var finishedCheckTask = await Task.WhenAny<CheckResult>(checkTasks);
var finishedCheck = await finishedCheckTask;
result = result && finishedCheck.Pass;
await File.AppendAllLinesAsync(_logFile, finishedCheck.Logs);
checkTasks.Remove(finishedCheckTask);
}
await Task.WhenAll(checkTasks);
return result;
}
}
}

181
src/Runner.Listener/Checks/NodeJsCheck.cs Normal file
View File

@@ -0,0 +1,181 @@
using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Net;
using System.Threading;
using System.Threading.Tasks;
using GitHub.Runner.Common;
using GitHub.Runner.Sdk;
namespace GitHub.Runner.Listener.Check
{
public sealed class NodeJsCheck : RunnerService, ICheckExtension
{
private string _logFile = null;
public int Order => 4;
public string CheckName => "Node.js Certificate/Proxy Validation";
public string CheckDescription => "Make sure the node.js have access to GitHub.com or the GitHub Enterprise Server.";
public string CheckLog => _logFile;
public string HelpLink => "https://github.com/actions/runner/blob/main/docs/checks/nodejs.md";
public Type ExtensionType => typeof(ICheckExtension);
public override void Initialize(IHostContext hostContext)
{
base.Initialize(hostContext);
_logFile = Path.Combine(HostContext.GetDirectory(WellKnownDirectory.Diag), StringUtil.Format("{0}_{1:yyyyMMdd-HHmmss}-utc.log", nameof(NodeJsCheck), DateTime.UtcNow));
}
// node access to ghes/gh
public async Task<bool> RunCheck(string url, string pat)
{
await File.AppendAllLinesAsync(_logFile, HostContext.WarnLog());
await File.AppendAllLinesAsync(_logFile, HostContext.CheckProxy());
// Request to github.com or ghes server
var urlBuilder = new UriBuilder(url);
if (UrlUtil.IsHostedServer(urlBuilder))
{
urlBuilder.Host = $"api.{urlBuilder.Host}";
urlBuilder.Path = "";
}
else
{
urlBuilder.Path = "api/v3";
}
var checkNode = await CheckNodeJs(urlBuilder.Uri.AbsoluteUri, pat);
var result = checkNode.Pass;
await File.AppendAllLinesAsync(_logFile, checkNode.Logs);
// try fix SSL error by providing extra CA certificate.
if (checkNode.SslError)
{
var downloadCert = await HostContext.DownloadExtraCA(urlBuilder.Uri.AbsoluteUri, pat);
await File.AppendAllLinesAsync(_logFile, downloadCert.Logs);
if (downloadCert.Pass)
{
var recheckNode = await CheckNodeJs(urlBuilder.Uri.AbsoluteUri, pat, extraCA: true);
await File.AppendAllLinesAsync(_logFile, recheckNode.Logs);
if (recheckNode.Pass)
{
await File.AppendAllLinesAsync(_logFile, new[] { $"{DateTime.UtcNow.ToString("O")} Fixed SSL error by providing extra CA certs." });
}
}
}
return result;
}
private async Task<CheckResult> CheckNodeJs(string url, string pat, bool extraCA = false)
{
var result = new CheckResult();
try
{
result.Logs.Add($"{DateTime.UtcNow.ToString("O")} ***************************************************************************************************************");
result.Logs.Add($"{DateTime.UtcNow.ToString("O")} **** ****");
result.Logs.Add($"{DateTime.UtcNow.ToString("O")} **** Make Http request to {url} using node.js ");
result.Logs.Add($"{DateTime.UtcNow.ToString("O")} **** ****");
result.Logs.Add($"{DateTime.UtcNow.ToString("O")} ***************************************************************************************************************");
// Request to github.com or ghes server
Uri requestUrl = new Uri(url);
var env = new Dictionary<string, string>()
{
{ "HOSTNAME", requestUrl.Host },
{ "PORT", requestUrl.IsDefaultPort ? (requestUrl.Scheme.ToLowerInvariant() == "https" ? "443" : "80") : requestUrl.Port.ToString() },
{ "PATH", requestUrl.AbsolutePath },
{ "PAT", pat }
};
var proxy = HostContext.WebProxy.GetProxy(requestUrl);
if (proxy != null)
{
env["PROXYHOST"] = proxy.Host;
env["PROXYPORT"] = proxy.IsDefaultPort ? (proxy.Scheme.ToLowerInvariant() == "https" ? "443" : "80") : proxy.Port.ToString();
if (HostContext.WebProxy.HttpProxyUsername != null ||
HostContext.WebProxy.HttpsProxyUsername != null)
{
env["PROXYUSERNAME"] = HostContext.WebProxy.HttpProxyUsername ?? HostContext.WebProxy.HttpsProxyUsername;
env["PROXYPASSWORD"] = HostContext.WebProxy.HttpProxyPassword ?? HostContext.WebProxy.HttpsProxyPassword;
}
else
{
env["PROXYUSERNAME"] = "";
env["PROXYPASSWORD"] = "";
}
}
else
{
env["PROXYHOST"] = "";
env["PROXYPORT"] = "";
env["PROXYUSERNAME"] = "";
env["PROXYPASSWORD"] = "";
}
if (extraCA)
{
env["NODE_EXTRA_CA_CERTS"] = Path.Combine(HostContext.GetDirectory(WellKnownDirectory.Root), "download_ca_cert.pem");
}
using (var processInvoker = HostContext.CreateService<IProcessInvoker>())
{
processInvoker.OutputDataReceived += new EventHandler<ProcessDataReceivedEventArgs>((sender, args) =>
{
if (!string.IsNullOrEmpty(args.Data))
{
result.Logs.Add($"{DateTime.UtcNow.ToString("O")} [STDOUT] {args.Data}");
}
});
processInvoker.ErrorDataReceived += new EventHandler<ProcessDataReceivedEventArgs>((sender, args) =>
{
if (!string.IsNullOrEmpty(args.Data))
{
result.Logs.Add($"{DateTime.UtcNow.ToString("O")} [STDERR] {args.Data}");
}
});
var makeWebRequestScript = Path.Combine(HostContext.GetDirectory(WellKnownDirectory.Bin), "checkScripts", "makeWebRequest.js");
var node12 = Path.Combine(HostContext.GetDirectory(WellKnownDirectory.Externals), "node12", "bin", $"node{IOUtil.ExeExtension}");
result.Logs.Add($"{DateTime.UtcNow.ToString("O")} Run '{node12} \"{makeWebRequestScript}\"' ");
result.Logs.Add($"{DateTime.UtcNow.ToString("O")} {StringUtil.ConvertToJson(env)}");
await processInvoker.ExecuteAsync(
HostContext.GetDirectory(WellKnownDirectory.Root),
node12,
$"\"{makeWebRequestScript}\"",
env,
true,
CancellationToken.None);
}
result.Pass = true;
}
catch (Exception ex)
{
result.Pass = false;
result.Logs.Add($"{DateTime.UtcNow.ToString("O")} ***************************************************************************************************************");
result.Logs.Add($"{DateTime.UtcNow.ToString("O")} **** ****");
result.Logs.Add($"{DateTime.UtcNow.ToString("O")} **** Make https request to {url} using node.js failed with error: {ex}");
if (result.Logs.Any(x => x.Contains("UNABLE_TO_VERIFY_LEAF_SIGNATURE") ||
x.Contains("UNABLE_TO_GET_ISSUER_CERT_LOCALLY") ||
x.Contains("SELF_SIGNED_CERT_IN_CHAIN")))
{
result.Logs.Add($"{DateTime.UtcNow.ToString("O")} **** Https request failed due to SSL cert issue.");
result.SslError = true;
}
result.Logs.Add($"{DateTime.UtcNow.ToString("O")} **** ****");
result.Logs.Add($"{DateTime.UtcNow.ToString("O")} ***************************************************************************************************************");
}
return result;
}
}
}

17
src/Runner.Listener/CommandSettings.cs
View File

@@ -27,6 +27,7 @@ namespace GitHub.Runner.Listener
private readonly string[] validFlags = private readonly string[] validFlags =
{ {
Constants.Runner.CommandLine.Flags.Check,
Constants.Runner.CommandLine.Flags.Commit, Constants.Runner.CommandLine.Flags.Commit,
Constants.Runner.CommandLine.Flags.Help, Constants.Runner.CommandLine.Flags.Help,
Constants.Runner.CommandLine.Flags.Replace, Constants.Runner.CommandLine.Flags.Replace,
@@ -60,6 +61,7 @@ namespace GitHub.Runner.Listener
public bool Warmup => TestCommand(Constants.Runner.CommandLine.Commands.Warmup); public bool Warmup => TestCommand(Constants.Runner.CommandLine.Commands.Warmup);
// Flags. // Flags.
public bool Check => TestFlag(Constants.Runner.CommandLine.Flags.Check);
public bool Commit => TestFlag(Constants.Runner.CommandLine.Flags.Commit); public bool Commit => TestFlag(Constants.Runner.CommandLine.Flags.Commit);
public bool Help => TestFlag(Constants.Runner.CommandLine.Flags.Help); public bool Help => TestFlag(Constants.Runner.CommandLine.Flags.Help);
public bool Unattended => TestFlag(Constants.Runner.CommandLine.Flags.Unattended); public bool Unattended => TestFlag(Constants.Runner.CommandLine.Flags.Unattended);
@@ -188,9 +190,20 @@ namespace GitHub.Runner.Listener
validator: Validators.NonEmptyValidator); validator: Validators.NonEmptyValidator);
} }
public string GetGitHubPersonalAccessToken() public string GetGitHubPersonalAccessToken(bool required = false)
{ {
return GetArg(name: Constants.Runner.CommandLine.Args.PAT); if (required)
{
return GetArgOrPrompt(
name: Constants.Runner.CommandLine.Args.PAT,
description: "What is your GitHub personal access token?",
defaultValue: string.Empty,
validator: Validators.NonEmptyValidator);
}
else
{
return GetArg(name: Constants.Runner.CommandLine.Args.PAT);
}
} }
public string GetRunnerRegisterToken() public string GetRunnerRegisterToken()

15
src/Runner.Listener/Configuration/ConfigurationManager.cs
View File

@@ -117,7 +117,7 @@ namespace GitHub.Runner.Listener.Configuration
try try
{ {
// Determine the service deployment type based on connection data. (Hosted/OnPremises) // Determine the service deployment type based on connection data. (Hosted/OnPremises)
runnerSettings.IsHostedServer = runnerSettings.GitHubUrl == null || IsHostedServer(new UriBuilder(runnerSettings.GitHubUrl)); runnerSettings.IsHostedServer = runnerSettings.GitHubUrl == null || UrlUtil.IsHostedServer(new UriBuilder(runnerSettings.GitHubUrl));
// Warn if the Actions server url and GHES server url has different Host // Warn if the Actions server url and GHES server url has different Host
if (!runnerSettings.IsHostedServer) if (!runnerSettings.IsHostedServer)
@@ -508,13 +508,6 @@ namespace GitHub.Runner.Listener.Configuration
return agent; return agent;
} }
private bool IsHostedServer(UriBuilder gitHubUrl)
{
return string.Equals(gitHubUrl.Host, "github.com", StringComparison.OrdinalIgnoreCase) ||
string.Equals(gitHubUrl.Host, "www.github.com", StringComparison.OrdinalIgnoreCase) ||
string.Equals(gitHubUrl.Host, "github.localhost", StringComparison.OrdinalIgnoreCase);
}
private async Task<string> GetRunnerTokenAsync(CommandSettings command, string githubUrl, string tokenType) private async Task<string> GetRunnerTokenAsync(CommandSettings command, string githubUrl, string tokenType)
{ {
var githubPAT = command.GetGitHubPersonalAccessToken(); var githubPAT = command.GetGitHubPersonalAccessToken();
@@ -551,7 +544,7 @@ namespace GitHub.Runner.Listener.Configuration
if (path.Length == 1) if (path.Length == 1)
{ {
// org runner // org runner
if (IsHostedServer(gitHubUrlBuilder)) if (UrlUtil.IsHostedServer(gitHubUrlBuilder))
{ {
githubApiUrl = $"{gitHubUrlBuilder.Scheme}://api.{gitHubUrlBuilder.Host}/orgs/{path[0]}/actions/runners/{tokenType}-token"; githubApiUrl = $"{gitHubUrlBuilder.Scheme}://api.{gitHubUrlBuilder.Host}/orgs/{path[0]}/actions/runners/{tokenType}-token";
} }
@@ -569,7 +562,7 @@ namespace GitHub.Runner.Listener.Configuration
repoScope = ""; repoScope = "";
} }
if (IsHostedServer(gitHubUrlBuilder)) if (UrlUtil.IsHostedServer(gitHubUrlBuilder))
{ {
githubApiUrl = $"{gitHubUrlBuilder.Scheme}://api.{gitHubUrlBuilder.Host}/{repoScope}{path[0]}/{path[1]}/actions/runners/{tokenType}-token"; githubApiUrl = $"{gitHubUrlBuilder.Scheme}://api.{gitHubUrlBuilder.Host}/{repoScope}{path[0]}/{path[1]}/actions/runners/{tokenType}-token";
} }
@@ -615,7 +608,7 @@ namespace GitHub.Runner.Listener.Configuration
{ {
var githubApiUrl = ""; var githubApiUrl = "";
var gitHubUrlBuilder = new UriBuilder(githubUrl); var gitHubUrlBuilder = new UriBuilder(githubUrl);
if (IsHostedServer(gitHubUrlBuilder)) if (UrlUtil.IsHostedServer(gitHubUrlBuilder))
{ {
githubApiUrl = $"{gitHubUrlBuilder.Scheme}://api.{gitHubUrlBuilder.Host}/actions/runner-registration"; githubApiUrl = $"{gitHubUrlBuilder.Scheme}://api.{gitHubUrlBuilder.Host}/actions/runner-registration";
} }

43
src/Runner.Listener/Runner.cs
View File

@@ -1,6 +1,5 @@
using GitHub.DistributedTask.WebApi; using GitHub.DistributedTask.WebApi;
using GitHub.Runner.Listener.Configuration; using GitHub.Runner.Listener.Configuration;
using GitHub.Runner.Common.Util;
using System; using System;
using System.Threading; using System.Threading;
using System.Threading.Tasks; using System.Threading.Tasks;
@@ -11,6 +10,8 @@ using System.Reflection;
using System.Runtime.CompilerServices; using System.Runtime.CompilerServices;
using GitHub.Runner.Common; using GitHub.Runner.Common;
using GitHub.Runner.Sdk; using GitHub.Runner.Sdk;
using System.Linq;
using GitHub.Runner.Listener.Check;
namespace GitHub.Runner.Listener namespace GitHub.Runner.Listener
{ {
@@ -72,6 +73,46 @@ namespace GitHub.Runner.Listener
return Constants.Runner.ReturnCode.Success; return Constants.Runner.ReturnCode.Success;
} }
if (command.Check)
{
var url = command.GetUrl();
var pat = command.GetGitHubPersonalAccessToken(required: true);
var checkExtensions = HostContext.GetService<IExtensionManager>().GetExtensions<ICheckExtension>();
var sortedChecks = checkExtensions.OrderBy(x => x.Order);
foreach (var check in sortedChecks)
{
_term.WriteLine($"**********************************************************************************************************************");
_term.WriteLine($"** Check: {check.CheckName}");
_term.WriteLine($"** Description: {check.CheckDescription}");
_term.WriteLine($"**********************************************************************************************************************");
var result = await check.RunCheck(url, pat);
if (!result)
{
_term.WriteLine($"** **");
_term.WriteLine($"** F A I L **");
_term.WriteLine($"** **");
_term.WriteLine($"**********************************************************************************************************************");
_term.WriteLine($"** Log: {check.CheckLog}");
_term.WriteLine($"** Help Doc: {check.HelpLink}");
_term.WriteLine($"**********************************************************************************************************************");
}
else
{
_term.WriteLine($"** **");
_term.WriteLine($"** P A S S **");
_term.WriteLine($"** **");
_term.WriteLine($"**********************************************************************************************************************");
_term.WriteLine($"** Log: {check.CheckLog}");
_term.WriteLine($"**********************************************************************************************************************");
}
_term.WriteLine();
_term.WriteLine();
}
return Constants.Runner.ReturnCode.Success;
}
// Configure runner prompt for args if not supplied // Configure runner prompt for args if not supplied
// Unattended configure mode will not prompt for args if not supplied and error on any missing or invalid value. // Unattended configure mode will not prompt for args if not supplied and error on any missing or invalid value.
if (command.Configure) if (command.Configure)

7
src/Runner.Sdk/Util/UrlUtil.cs
View File

@@ -4,6 +4,13 @@ namespace GitHub.Runner.Sdk
{ {
public static class UrlUtil public static class UrlUtil
{ {
public static bool IsHostedServer(UriBuilder gitHubUrl)
{
return string.Equals(gitHubUrl.Host, "github.com", StringComparison.OrdinalIgnoreCase) ||
string.Equals(gitHubUrl.Host, "www.github.com", StringComparison.OrdinalIgnoreCase) ||
string.Equals(gitHubUrl.Host, "github.localhost", StringComparison.OrdinalIgnoreCase);
}
public static Uri GetCredentialEmbeddedUrl(Uri baseUrl, string username, string password) public static Uri GetCredentialEmbeddedUrl(Uri baseUrl, string username, string password)
{ {
ArgUtil.NotNull(baseUrl, nameof(baseUrl)); ArgUtil.NotNull(baseUrl, nameof(baseUrl));

7
src/Test/L0/ServiceInterfacesL0.cs
View File

@@ -1,4 +1,5 @@
using GitHub.Runner.Listener; using GitHub.Runner.Listener;
using GitHub.Runner.Listener.Check;
using GitHub.Runner.Listener.Configuration; using GitHub.Runner.Listener.Configuration;
using GitHub.Runner.Worker; using GitHub.Runner.Worker;
using GitHub.Runner.Worker.Handlers; using GitHub.Runner.Worker.Handlers;
@@ -21,7 +22,8 @@ namespace GitHub.Runner.Common.Tests
// Otherwise, the interface needs to whitelisted. // Otherwise, the interface needs to whitelisted.
var whitelist = new[] var whitelist = new[]
{ {
typeof(ICredentialProvider) typeof(ICredentialProvider),
typeof(ICheckExtension),
}; };
Validate( Validate(
assembly: typeof(IMessageListener).GetTypeInfo().Assembly, assembly: typeof(IMessageListener).GetTypeInfo().Assembly,
@@ -85,7 +87,8 @@ namespace GitHub.Runner.Common.Tests
continue; continue;
} }
if (interfaceTypeInfo.FullName.Contains("IConverter")){ if (interfaceTypeInfo.FullName.Contains("IConverter"))
{
continue; continue;
} }