[Windows] Implement new directories hierarchy (#8616)

.github/workflows/windows2019.yml

@@ -5,7 +5,7 @@ on:
   pull_request_target:
     types: labeled
     paths:
-    - 'images/win/**'
+    - 'images/windows/**'
 
 defaults:
   run:

.github/workflows/windows2022.yml

@@ -5,7 +5,7 @@ on:
   pull_request_target:
     types: labeled
     paths:
-    - 'images/win/**'
+    - 'images/windows/**'
 
 defaults:
   run:

CONTRIBUTING.md

@@ -34,18 +34,18 @@ Here are a few things you can do that will increase the likelihood of your pull
 
 - For every new tool add validation scripts and update software report script to make sure that it is included to documentation
 - If the tool is available in other platforms (macOS, Windows, Linux), make sure you include it in as many as possible.
-- If installing a few versions of the tool, consider putting the list of versions in the corresponding `toolset.json` file. It will help other customers to configure their builds flexibly. See [toolset-windows-2019.json](images/win/toolsets/toolset-2019.json) as example.
+- If installing a few versions of the tool, consider putting the list of versions in the corresponding `toolset.json` file. It will help other customers to configure their builds flexibly. See [toolset-windows-2019.json](images/windows/toolsets/toolset-2019.json) as example.
 - Use consistent naming across all files
 - Validation scripts should be simple and shouldn't change image content
 
 ### Windows
 
-- Add a script that will install the tool and put the script in the `scripts/Installers` folder.  
-There are a bunch of helper functions that could simplify your code: `Choco-Install`, `Install-Binary`, `Install-VsixExtension`, `Start-DownloadWithRetry`, `Test-IsWin19`, `Test-IsWin22` (find the full list of helpers in [ImageHelpers.psm1](images/win/scripts/ImageHelpers/ImageHelpers.psm1)).
-- Add a script that will validate the tool installation and put the script in the `scripts/Tests` folder.  
+- Add a script that will install the tool and put the script in the `scripts/build` folder.  
+There are a bunch of helper functions that could simplify your code: `Choco-Install`, `Install-Binary`, `Install-VsixExtension`, `Start-DownloadWithRetry`, `Test-IsWin19`, `Test-IsWin22` (find the full list of helpers in [ImageHelpers.psm1](images/windows/scripts/helpers/ImageHelpers.psm1)).
+- Add a script that will validate the tool installation and put the script in the `scripts/tests` folder.  
 We use [Pester v5](https://github.com/pester/pester) for validation scripts. If the tests for the tool are complex enough, create a separate `*.Tests.ps1`. Otherwise, use `Tools.Tests.ps1` for simple tests.  
 Add `Invoke-PesterTests -TestFile <testFileName> [-TestName <describeName>]` at the end of the installation script to make sure that your tests will be run.
-- Add changes to the software report generator `images/win/scripts/SoftwareReport/SoftwareReport.Generator.ps1`. The software report generator is used to generate an image's README file, e.g. [Windows2019-Readme.md](images/win/Windows2019-Readme.md) and uses [MarkdownPS](https://github.com/Sarafian/MarkdownPS).
+- Add changes to the software report generator `images/windows/scripts/docs-gen/SoftwareReport.Generator.ps1`. The software report generator is used to generate an image's README file, e.g. [Windows2019-Readme.md](images/windows/Windows2019-Readme.md) and uses [MarkdownPS](https://github.com/Sarafian/MarkdownPS).
 
 ### Ubuntu
 

README.md

@@ -36,12 +36,11 @@ To build a VM machine from this repo's source, see the [instructions](docs/creat
 
 [ubuntu-22.04]: https://github.com/actions/runner-images/blob/main/images/linux/Ubuntu2204-Readme.md
 [ubuntu-20.04]: https://github.com/actions/runner-images/blob/main/images/linux/Ubuntu2004-Readme.md
-[windows-2022]: https://github.com/actions/runner-images/blob/main/images/win/Windows2022-Readme.md
-[windows-2019]: https://github.com/actions/runner-images/blob/main/images/win/Windows2019-Readme.md
+[windows-2022]: https://github.com/actions/runner-images/blob/main/images/windows/Windows2022-Readme.md
+[windows-2019]: https://github.com/actions/runner-images/blob/main/images/windows/Windows2019-Readme.md
 [macOS-11]: https://github.com/actions/runner-images/blob/main/images/macos/macos-11-Readme.md
 [macOS-12]: https://github.com/actions/runner-images/blob/main/images/macos/macos-12-Readme.md
 [macOS-13]: https://github.com/actions/runner-images/blob/main/images/macos/macos-13-Readme.md
-[macOS-10.15]: https://github.com/actions/runner-images/blob/main/images/macos/macos-10.15-Readme.md
 [self-hosted runners]: https://help.github.com/en/actions/hosting-your-own-runners
 
 ## Announcements

docs/create-image-and-azure-resources.md

@@ -216,7 +216,7 @@ Where:
 - `InstallPassword` - password for the user used to install software (Windows only)
 - `Location` - location where resources will be created (e.g. "East US")
 - `ImageName` and `ImageResourceGroupName` - name of the resource group where managed image will be stored
-- `TemplatePath` - path to the Packer template file (e.g. "images/win/windows2022.json")
+- `TemplatePath` - path to the Packer template file (e.g. "images/windows/templates/windows-2022.json")
 
 ### Required variables
 
@@ -269,9 +269,9 @@ Generated tool versions and details can be found in related projects:
 
 > :warning: These scripts are intended to run on a VM deployed in Azure
 
-The user, created during the image generation, does not exist in the result image hence some configuration files related to the user's home directory need to be changed as well as the file permissions for some directories. Scripts for that are located in the `post-generation` folder in the repository:
+The user, created during the image generation, does not exist in the result image hence some configuration files related to the user's home directory need to be changed as well as the file permissions for some directories. Scripts for that are located in the `post-gen` folder in the repository:
 
-- Windows: <https://github.com/actions/runner-images/tree/main/images/win/post-generation>
+- Windows: <https://github.com/actions/runner-images/tree/main/images/windows/assets/post-gen>
 - Linux: <https://github.com/actions/runner-images/tree/main/images/linux/post-generation>
 
 **Note:** The default user for Linux should have `sudo privileges`.

helpers/GenerateResourcesAndImage.ps1

@@ -18,10 +18,10 @@ Function Get-PackerTemplatePath {
 
     switch ($ImageType) {
         ([ImageType]::Windows2019) {
-            $relativeTemplatePath = Join-Path "win" "windows2019.json"
+            $relativeTemplatePath = Join-Path "windows" "templates" "windows-2019.json"
         }
         ([ImageType]::Windows2022) {
-            $relativeTemplatePath = Join-Path "win" "windows2022.json"
+            $relativeTemplatePath = Join-Path "windows" "templates" "windows-2022.json"
         }
         ([ImageType]::Ubuntu2004) {
             $relativeTemplatePath = Join-Path "linux" "ubuntu2004.json"

images.CI/linux-and-win/azure-pipelines/image-generation.yml

@@ -12,6 +12,9 @@ parameters:
   - name: image_type
     type: string
 
+  - name: image_template_name
+    type: string
+
   - name: image_readme_name
     type: string
 
@@ -62,11 +65,17 @@ jobs:
       targetType: 'inline'
       script: |
         $ImageType = "${{ parameters.image_type }}"
-        $TemplateDirectoryName = if ($ImageType.StartsWith("ubuntu")) { "linux" } else { "win" }
+        $TemplateDirectoryName = if ($ImageType.StartsWith("ubuntu")) { "linux" } else { "windows/templates" }
         $TemplateDirectoryPath = Join-Path "images" $TemplateDirectoryName | Resolve-Path
-        $TemplatePath = Join-Path $TemplateDirectoryPath "$ImageType.pkr.hcl"
-        if ( -not (Test-Path $TemplatePath) ) {
-            $TemplatePath = Join-Path $TemplateDirectoryPath "$ImageType.json"
+        
+        $TemplateFileName = "${{ parameters.image_template_name }}"
+        if ($TemplateFileName) {
+          $TemplatePath = Join-Path $TemplateDirectoryPath $TemplateFileName
+        } else {
+          $TemplatePath = Join-Path $TemplateDirectoryPath "$ImageType.pkr.hcl"
+          if ( -not (Test-Path $TemplatePath) ) {
+              $TemplatePath = Join-Path $TemplateDirectoryPath "$ImageType.json"
+          }
         }
         Write-Host "##vso[task.setvariable variable=TemplateDirectoryPath;]$TemplateDirectoryPath"
         Write-Host "##vso[task.setvariable variable=TemplatePath;]$TemplatePath"
@@ -104,8 +113,12 @@ jobs:
     inputs:
       targetType: 'inline'
       script: |
-        $readmePath = Join-Path "$(TemplateDirectoryPath)" "${{ parameters.image_readme_name }}"
-        $softwareReportPath = Join-Path "$(TemplateDirectoryPath)" "software-report.json"
+        $ImageType = "${{ parameters.image_type }}"
+        $rootDirectoryName = if ($ImageType.StartsWith("ubuntu")) { "linux" } else { "windows" }
+        $rootDirectoryPath = Join-Path "images" $rootDirectoryName | Resolve-Path
+        
+        $readmePath = Join-Path $rootDirectoryPath "${{ parameters.image_readme_name }}"
+        $softwareReportPath = Join-Path $rootDirectoryPath "software-report.json"
 
         Copy-Item -Path $readmePath -Destination "$(Build.ArtifactStagingDirectory)/"
         if (Test-Path $softwareReportPath) {

images.CI/linux-and-win/azure-pipelines/windows2019.yml

@@ -17,4 +17,5 @@ jobs:
 - template: image-generation.yml
   parameters:
     image_type: windows2019
-    image_readme_name: Windows2019-Readme.md
+    image_readme_name: Windows2019-Readme.md
+    image_template_name: windows-2019.json

images.CI/linux-and-win/azure-pipelines/windows2022.yml

@@ -17,4 +17,5 @@ jobs:
 - template: image-generation.yml
   parameters:
     image_type: windows2022
-    image_readme_name: Windows2022-Readme.md
+    image_readme_name: Windows2022-Readme.md
+    image_template_name: windows-2022.json

images/win/scripts/README.md

@@ -1 +0,0 @@
-Common scripts for all Windows images regardless of Visual Studio or OS version

images/windows/assets/post-gen/VSConfiguration.ps1

@@ -1,14 +1,14 @@
-$vsInstallRoot = (Get-VisualStudioInstance).InstallationPath
-$devEnvPath = "$vsInstallRoot\Common7\IDE\devenv.exe"
-
-# Initialize Visual Studio Experimental Instance
-# The Out-Null cmdlet is required to ensure PowerShell waits until the '/ResetSettings' command fully completes.
-& "$devEnvPath" /RootSuffix Exp /ResetSettings General.vssettings /Command File.Exit | Out-Null
-
-cmd.exe /c "`"$devEnvPath`" /updateconfiguration"
-
-#
-# https://github.com/actions/runner-images/issues/5301
-#
-$warmup_vdproj = $(Join-Path $PSScriptRoot 'warmup.vdproj')
-& "$devEnvPath" $warmup_vdproj /build Release | Out-Null
+$vsInstallRoot = (Get-VisualStudioInstance).InstallationPath
+$devEnvPath = "$vsInstallRoot\Common7\IDE\devenv.exe"
+
+# Initialize Visual Studio Experimental Instance
+# The Out-Null cmdlet is required to ensure PowerShell waits until the '/ResetSettings' command fully completes.
+& "$devEnvPath" /RootSuffix Exp /ResetSettings General.vssettings /Command File.Exit | Out-Null
+
+cmd.exe /c "`"$devEnvPath`" /updateconfiguration"
+
+#
+# https://github.com/actions/runner-images/issues/5301
+#
+$warmup_vdproj = $(Join-Path $PSScriptRoot 'warmup.vdproj')
+& "$devEnvPath" $warmup_vdproj /build Release | Out-Null

images/windows/scripts/build/Configure-DynamicPort.ps1

@@ -1,14 +1,14 @@
-# https://support.microsoft.com/en-us/help/929851/the-default-dynamic-port-range-for-tcp-ip-has-changed-in-windows-vista
-# The new default start port is 49152, and the new default end port is 65535.
-# Default port configuration was changed during image generation by Visual Studio Enterprise Installer to:
-#   Protocol tcp Dynamic Port Range
-#   ---------------------------------
-#   Start Port      : 1024
-#   Number of Ports : 64511
-Write-Host "Set the dynamic port range to start at port 49152 and to end at the 65536 (16384 ports)"
-$null = netsh int ipv4 set dynamicport tcp start=49152 num=16384
-$null = netsh int ipv4 set dynamicport udp start=49152 num=16384
-$null = netsh int ipv6 set dynamicport tcp start=49152 num=16384
-$null = netsh int ipv6 set dynamicport udp start=49152 num=16384
-
+# https://support.microsoft.com/en-us/help/929851/the-default-dynamic-port-range-for-tcp-ip-has-changed-in-windows-vista
+# The new default start port is 49152, and the new default end port is 65535.
+# Default port configuration was changed during image generation by Visual Studio Enterprise Installer to:
+#   Protocol tcp Dynamic Port Range
+#   ---------------------------------
+#   Start Port      : 1024
+#   Number of Ports : 64511
+Write-Host "Set the dynamic port range to start at port 49152 and to end at the 65536 (16384 ports)"
+$null = netsh int ipv4 set dynamicport tcp start=49152 num=16384
+$null = netsh int ipv4 set dynamicport udp start=49152 num=16384
+$null = netsh int ipv6 set dynamicport tcp start=49152 num=16384
+$null = netsh int ipv6 set dynamicport udp start=49152 num=16384
+
 Invoke-PesterTests -TestFile "WindowsFeatures" -TestName "DynamicPorts"

images/windows/scripts/build/Disable-JITDebugger.ps1

@@ -1,16 +1,16 @@
-Write-Host "Disable Just-In-Time Debugger"
-
-# Turn off Application Error Debugger
-New-ItemProperty -Path "HKLM:\SOFTWARE\Microsoft\Windows NT\CurrentVersion\AeDebug" -Name Debugger -Value "-" -Type String -Force
-New-ItemProperty -Path "HKLM:\SOFTWARE\WOW6432Node\Microsoft\Windows NT\CurrentVersion\AeDebug" -Name Debugger -Value "-" -Type String -Force
-
-# Turn off the Debug dialog
-New-ItemProperty -Path "HKLM:\SOFTWARE\Microsoft\.NETFramework" -Name DbgManagedDebugger -Value "-" -Type String -Force
-New-ItemProperty -Path "HKLM:\SOFTWARE\WOW6432Node\Microsoft\.NETFramework" -Name DbgManagedDebugger -Value "-" -Type String -Force
-
-# Disable the WER UI
-New-ItemProperty -Path "HKLM:\SOFTWARE\Microsoft\Windows\Windows Error Reporting" -Name DontShowUI -Value 1 -Type DWORD -Force
-# Send all reports to the user's queue
-New-ItemProperty -Path "HKLM:\SOFTWARE\Microsoft\Windows\Windows Error Reporting" -Name ForceQueue -Value 1 -Type DWORD -Force
-# Default consent choice 1 - Always ask (default)
+Write-Host "Disable Just-In-Time Debugger"
+
+# Turn off Application Error Debugger
+New-ItemProperty -Path "HKLM:\SOFTWARE\Microsoft\Windows NT\CurrentVersion\AeDebug" -Name Debugger -Value "-" -Type String -Force
+New-ItemProperty -Path "HKLM:\SOFTWARE\WOW6432Node\Microsoft\Windows NT\CurrentVersion\AeDebug" -Name Debugger -Value "-" -Type String -Force
+
+# Turn off the Debug dialog
+New-ItemProperty -Path "HKLM:\SOFTWARE\Microsoft\.NETFramework" -Name DbgManagedDebugger -Value "-" -Type String -Force
+New-ItemProperty -Path "HKLM:\SOFTWARE\WOW6432Node\Microsoft\.NETFramework" -Name DbgManagedDebugger -Value "-" -Type String -Force
+
+# Disable the WER UI
+New-ItemProperty -Path "HKLM:\SOFTWARE\Microsoft\Windows\Windows Error Reporting" -Name DontShowUI -Value 1 -Type DWORD -Force
+# Send all reports to the user's queue
+New-ItemProperty -Path "HKLM:\SOFTWARE\Microsoft\Windows\Windows Error Reporting" -Name ForceQueue -Value 1 -Type DWORD -Force
+# Default consent choice 1 - Always ask (default)
 New-ItemProperty -Path "HKLM:\SOFTWARE\Microsoft\Windows\Windows Error Reporting\Consent" -Name DefaultConsent -Value 1 -Type DWORD -Force

images/windows/scripts/build/Install-AliyunCli.ps1

@@ -1,29 +1,29 @@
-################################################################################
-##  File:  Install-AliyunCli.ps1
-##  Desc:  Install Alibaba Cloud CLI
-##  Supply chain security: Alibaba Cloud CLI - checksum validation
-################################################################################
-
-Write-Host "Download Latest aliyun-cli archive"
-$repoUrl = "https://api.github.com/repos/aliyun/aliyun-cli/releases/latest"
-$installerFileName = "aliyun-cli-windows"
-$assets = (Invoke-RestMethod -Uri $repoUrl).assets
-$downloadUrl = ($assets.browser_download_url -ilike "*aliyun-cli-windows-*-amd64.zip*") | Select-Object -First 1
-$packagePath = Start-DownloadWithRetry -Url $downloadUrl -Name "$installerFileName.zip"
-
-#region Supply chain security - Alibaba Cloud CLI
-$fileHash = (Get-FileHash -Path $packagePath -Algorithm SHA256).Hash
-$hashUrl = ($assets.browser_download_url -ilike "*SHASUMS256.txt*") | Select-Object -First 1
-$externalHash = (Invoke-RestMethod -Uri $hashURL).ToString().Split("`n").Where({ $_ -ilike "*$installerFileName*" }).Split(' ')[0]
-Use-ChecksumComparison $fileHash $externalHash
-#endregion
-
-Write-Host "Expand aliyun-cli archive"
-$aliyunPath = "C:\aliyun-cli"
-New-Item -Path $aliyunPath -ItemType Directory -Force
-Extract-7Zip -Path $packagePath -DestinationPath $aliyunPath
-
-# Add aliyun-cli to path
-Add-MachinePathItem $aliyunPath
-
-Invoke-PesterTests -TestFile "CLI.Tools" -TestName "Aliyun CLI"
+################################################################################
+##  File:  Install-AliyunCli.ps1
+##  Desc:  Install Alibaba Cloud CLI
+##  Supply chain security: Alibaba Cloud CLI - checksum validation
+################################################################################
+
+Write-Host "Download Latest aliyun-cli archive"
+$repoUrl = "https://api.github.com/repos/aliyun/aliyun-cli/releases/latest"
+$installerFileName = "aliyun-cli-windows"
+$assets = (Invoke-RestMethod -Uri $repoUrl).assets
+$downloadUrl = ($assets.browser_download_url -ilike "*aliyun-cli-windows-*-amd64.zip*") | Select-Object -First 1
+$packagePath = Start-DownloadWithRetry -Url $downloadUrl -Name "$installerFileName.zip"
+
+#region Supply chain security - Alibaba Cloud CLI
+$fileHash = (Get-FileHash -Path $packagePath -Algorithm SHA256).Hash
+$hashUrl = ($assets.browser_download_url -ilike "*SHASUMS256.txt*") | Select-Object -First 1
+$externalHash = (Invoke-RestMethod -Uri $hashURL).ToString().Split("`n").Where({ $_ -ilike "*$installerFileName*" }).Split(' ')[0]
+Use-ChecksumComparison $fileHash $externalHash
+#endregion
+
+Write-Host "Expand aliyun-cli archive"
+$aliyunPath = "C:\aliyun-cli"
+New-Item -Path $aliyunPath -ItemType Directory -Force
+Extract-7Zip -Path $packagePath -DestinationPath $aliyunPath
+
+# Add aliyun-cli to path
+Add-MachinePathItem $aliyunPath
+
+Invoke-PesterTests -TestFile "CLI.Tools" -TestName "Aliyun CLI"

images/windows/scripts/build/Install-Bazel.ps1

@@ -1,10 +1,10 @@
-################################################################################
-##  File:  Install-Bazel.ps1
-##  Desc:  Install Bazel and Bazelisk (A user-friendly launcher for Bazel)
-################################################################################
-
-Choco-Install -PackageName bazel
-
-npm install -g @bazel/bazelisk
-
+################################################################################
+##  File:  Install-Bazel.ps1
+##  Desc:  Install Bazel and Bazelisk (A user-friendly launcher for Bazel)
+################################################################################
+
+Choco-Install -PackageName bazel
+
+npm install -g @bazel/bazelisk
+
 Invoke-PesterTests -TestFile "Tools" -TestName "Bazel"

images/windows/scripts/build/Install-Pipx.ps1

@@ -1,14 +1,14 @@
-#   Set custom directorys for pipx
-$env:PIPX_BIN_DIR = "${env:ProgramFiles(x86)}\pipx_bin"
-$env:PIPX_HOME = "${env:ProgramFiles(x86)}\pipx"
-
-#   Install pipx
-pip install pipx
-
-#   Set environment variables
-Add-MachinePathItem "${env:PIPX_BIN_DIR}"
-Set-SystemVariable -SystemVariable PIPX_BIN_DIR -Value $env:PIPX_BIN_DIR
-Set-SystemVariable -SystemVariable PIPX_HOME -Value $env:PIPX_HOME
-
-#   Test pipx
+#   Set custom directorys for pipx
+$env:PIPX_BIN_DIR = "${env:ProgramFiles(x86)}\pipx_bin"
+$env:PIPX_HOME = "${env:ProgramFiles(x86)}\pipx"
+
+#   Install pipx
+pip install pipx
+
+#   Set environment variables
+Add-MachinePathItem "${env:PIPX_BIN_DIR}"
+Set-SystemVariable -SystemVariable PIPX_BIN_DIR -Value $env:PIPX_BIN_DIR
+Set-SystemVariable -SystemVariable PIPX_HOME -Value $env:PIPX_HOME
+
+#   Test pipx
 Invoke-PesterTests -TestFile "Tools" -TestName "Pipx"

images/windows/scripts/build/Install-PipxPackages.ps1

@@ -1,12 +1,12 @@
-$pipxToolset = (Get-ToolsetContent).pipx
-foreach($tool in $pipxToolset) {
-    if ($tool.python) {
-        $pythonPath = (Get-Item  -Path "${env:AGENT_TOOLSDIRECTORY}\Python\${tool.python}.*\x64\python-${tool.python}*").FullName
-        Write-Host "Install ${tool.package} into python ${tool.python}"
-        pipx install $tool.package --python $pythonPath
-    } else {
-        Write-Host "Install ${tool.package} into default python"
-        pipx install $tool.package
-    }
-}
+$pipxToolset = (Get-ToolsetContent).pipx
+foreach($tool in $pipxToolset) {
+    if ($tool.python) {
+        $pythonPath = (Get-Item  -Path "${env:AGENT_TOOLSDIRECTORY}\Python\${tool.python}.*\x64\python-${tool.python}*").FullName
+        Write-Host "Install ${tool.package} into python ${tool.python}"
+        pipx install $tool.package --python $pythonPath
+    } else {
+        Write-Host "Install ${tool.package} into default python"
+        pipx install $tool.package
+    }
+}
 Invoke-PesterTests -TestFile "PipxPackages"

images/windows/scripts/build/Install-PowerShellModules.ps1

@@ -1,26 +1,26 @@
-# Set TLS1.2
-[Net.ServicePointManager]::SecurityProtocol = [Net.ServicePointManager]::SecurityProtocol -bor "Tls12"
-
-# Install PowerShell modules
-$modules = (Get-ToolsetContent).powershellModules
-
-foreach($module in $modules)
-{
-    $moduleName = $module.name
-    Write-Host "Installing ${moduleName} module"
-
-    if ($module.versions)
-    {
-        foreach ($version in $module.versions)
-        {
-            Write-Host " - $version"
-            Install-Module -Name $moduleName -RequiredVersion $version -Scope AllUsers -SkipPublisherCheck -Force
-        }
-        continue
-    }
-
-    Install-Module -Name $moduleName -Scope AllUsers -SkipPublisherCheck -Force
-}
-
-Import-Module Pester
-Invoke-PesterTests -TestFile "PowerShellModules" -TestName "PowerShellModules"
+# Set TLS1.2
+[Net.ServicePointManager]::SecurityProtocol = [Net.ServicePointManager]::SecurityProtocol -bor "Tls12"
+
+# Install PowerShell modules
+$modules = (Get-ToolsetContent).powershellModules
+
+foreach($module in $modules)
+{
+    $moduleName = $module.name
+    Write-Host "Installing ${moduleName} module"
+
+    if ($module.versions)
+    {
+        foreach ($version in $module.versions)
+        {
+            Write-Host " - $version"
+            Install-Module -Name $moduleName -RequiredVersion $version -Scope AllUsers -SkipPublisherCheck -Force
+        }
+        continue
+    }
+
+    Install-Module -Name $moduleName -Scope AllUsers -SkipPublisherCheck -Force
+}
+
+Import-Module Pester
+Invoke-PesterTests -TestFile "PowerShellModules" -TestName "PowerShellModules"

images/windows/scripts/build/Install-PyPy.ps1

@@ -1,124 +1,124 @@
-################################################################################
-##  File:  Install-PyPy.ps1
-##  Team:  CI-Build
-##  Desc:  Install PyPy
-##  Supply chain security: checksum validation
-################################################################################
-function Install-PyPy
-{
-    param(
-        [String]$PackagePath,
-        [String]$Architecture
-    )
-
-    # Create PyPy toolcache folder
-    $pypyToolcachePath = Join-Path -Path $env:AGENT_TOOLSDIRECTORY -ChildPath "PyPy"
-    if (-not (Test-Path $pypyToolcachePath)) {
-        Write-Host "Create PyPy toolcache folder"
-        New-Item -ItemType Directory -Path $pypyToolcachePath | Out-Null
-    }
-
-    # Expand archive with binaries
-    $packageName = [IO.Path]::GetFileNameWithoutExtension((Split-Path -Path $packagePath -Leaf))
-    $tempFolder = Join-Path -Path $pypyToolcachePath -ChildPath $packageName
-    Extract-7Zip -Path $packagePath -DestinationPath $pypyToolcachePath
-
-    # Get Python version from binaries
-    $pypyApp = Get-ChildItem -Path "$tempFolder\pypy*.exe" | Where-Object Name -match "pypy(\d+)?.exe" | Select-Object -First 1
-    $pythonVersion = & $pypyApp -c "import sys;print('{}.{}.{}'.format(sys.version_info[0],sys.version_info[1],sys.version_info[2]))"
-
-    $pypyFullVersion = & $pypyApp -c "import sys;print('{}.{}.{}'.format(*sys.pypy_version_info[0:3]))"
-    Write-Host "Put '$pypyFullVersion' to PYPY_VERSION file"
-    New-Item -Path "$tempFolder\PYPY_VERSION" -Value $pypyFullVersion | Out-Null
-
-    if ($pythonVersion)
-    {
-        Write-Host "Installing PyPy $pythonVersion"
-        $pypyVersionPath = Join-Path -Path $pypyToolcachePath -ChildPath $pythonVersion
-        $pypyArchPath = Join-Path -Path $pypyVersionPath -ChildPath $architecture
-
-        Write-Host "Create PyPy '${pythonVersion}' folder in '${pypyVersionPath}'"
-        New-Item -ItemType Directory -Path $pypyVersionPath -Force | Out-Null
-
-        Write-Host "Move PyPy '${pythonVersion}' files to '${pypyArchPath}'"
-        Invoke-SBWithRetry -Command {
-            Move-Item -Path $tempFolder -Destination $pypyArchPath -ErrorAction Stop | Out-Null
-        }
-
-        Write-Host "Install PyPy '${pythonVersion}' in '${pypyArchPath}'"
-        if (Test-Path "$pypyArchPath\python.exe") {
-            cmd.exe /c "cd /d $pypyArchPath && python.exe -m ensurepip && python.exe -m pip install --upgrade pip"
-        } else {
-            $pypyName = $pypyApp.Name
-            cmd.exe /c "cd /d $pypyArchPath && mklink python.exe $pypyName && python.exe -m ensurepip && python.exe -m pip install --upgrade pip"
-        }
-
-        # Create pip.exe if missing
-        $pipPath = Join-Path -Path $pypyArchPath -ChildPath "Scripts/pip.exe"
-        if (-not (Test-Path $pipPath))
-        {
-            $pip3Path = Join-Path -Path $pypyArchPath -ChildPath "Scripts/pip3.exe"
-            Copy-Item -Path $pip3Path -Destination $pipPath 
-        }
-
-        if ($LASTEXITCODE -ne 0)
-        {
-            Throw "Error happened during PyPy installation"
-            exit 1
-        }
-
-        Write-Host "Create complete file"
-        New-Item -ItemType File -Path $pypyVersionPath -Name "$architecture.complete" | Out-Null
-    }
-    else
-    {
-        Write-Host "PyPy application is not found. Failed to expand '$packagePath' archive"
-        exit 1
-    }
-}
-
-# Get PyPy content from toolset
-$toolsetVersions = Get-ToolsetContent | Select-Object -ExpandProperty toolcache | Where-Object Name -eq "PyPy"
-
-# Get PyPy releases
-$pypyVersions = Invoke-RestMethod https://downloads.python.org/pypy/versions.json
-
-# required for html parsing
-$checksums = (Invoke-RestMethod -Uri 'https://www.pypy.org/checksums.html' | ConvertFrom-HTML).SelectNodes('//*[@id="content"]/article/div/pre')
-
-Write-Host "Starting installation PyPy..."
-foreach($toolsetVersion in $toolsetVersions.versions)
-{
-    # Query latest PyPy version
-    $latestMajorPyPyVersion = $pypyVersions |
-        Where-Object {$_.python_version.StartsWith("$toolsetVersion") -and $_.stable -eq $true} |
-        Select-Object -ExpandProperty files -First 1 |
-        Where-Object platform -like "win*"
-    
-    if ($latestMajorPyPyVersion)
-    {
-        $filename = $latestMajorPyPyVersion.filename
-        Write-Host "Found PyPy '$filename' package"
-        $tempPyPyPackagePath = Start-DownloadWithRetry -Url $latestMajorPyPyVersion.download_url -Name $filename
-
-        #region Supply chain security
-        $localFileHash = (Get-FileHash -Path $tempPyPyPackagePath -Algorithm SHA256).Hash
-        $distributorFileHash = $null
-
-        ForEach($node in $checksums) {
-            if($node.InnerText -ilike "*${filename}*") {
-                $distributorFileHash = $node.InnerText.ToString().Split("`n").Where({ $_ -ilike "*${filename}*" }).Split(' ')[0]
-            }
-        }
-        
-        Use-ChecksumComparison -LocalFileHash $localFileHash -DistributorFileHash $distributorFileHash
-        #endregion
-
-        Install-PyPy -PackagePath $tempPyPyPackagePath -Architecture $toolsetVersions.arch
-    }
-    else
-    {
-        Write-Host "Failed to query PyPy version '$toolsetVersion'"
-        exit 1
-    }
-}
+################################################################################
+##  File:  Install-PyPy.ps1
+##  Team:  CI-Build
+##  Desc:  Install PyPy
+##  Supply chain security: checksum validation
+################################################################################
+function Install-PyPy
+{
+    param(
+        [String]$PackagePath,
+        [String]$Architecture
+    )
+
+    # Create PyPy toolcache folder
+    $pypyToolcachePath = Join-Path -Path $env:AGENT_TOOLSDIRECTORY -ChildPath "PyPy"
+    if (-not (Test-Path $pypyToolcachePath)) {
+        Write-Host "Create PyPy toolcache folder"
+        New-Item -ItemType Directory -Path $pypyToolcachePath | Out-Null
+    }
+
+    # Expand archive with binaries
+    $packageName = [IO.Path]::GetFileNameWithoutExtension((Split-Path -Path $packagePath -Leaf))
+    $tempFolder = Join-Path -Path $pypyToolcachePath -ChildPath $packageName
+    Extract-7Zip -Path $packagePath -DestinationPath $pypyToolcachePath
+
+    # Get Python version from binaries
+    $pypyApp = Get-ChildItem -Path "$tempFolder\pypy*.exe" | Where-Object Name -match "pypy(\d+)?.exe" | Select-Object -First 1
+    $pythonVersion = & $pypyApp -c "import sys;print('{}.{}.{}'.format(sys.version_info[0],sys.version_info[1],sys.version_info[2]))"
+
+    $pypyFullVersion = & $pypyApp -c "import sys;print('{}.{}.{}'.format(*sys.pypy_version_info[0:3]))"
+    Write-Host "Put '$pypyFullVersion' to PYPY_VERSION file"
+    New-Item -Path "$tempFolder\PYPY_VERSION" -Value $pypyFullVersion | Out-Null
+
+    if ($pythonVersion)
+    {
+        Write-Host "Installing PyPy $pythonVersion"
+        $pypyVersionPath = Join-Path -Path $pypyToolcachePath -ChildPath $pythonVersion
+        $pypyArchPath = Join-Path -Path $pypyVersionPath -ChildPath $architecture
+
+        Write-Host "Create PyPy '${pythonVersion}' folder in '${pypyVersionPath}'"
+        New-Item -ItemType Directory -Path $pypyVersionPath -Force | Out-Null
+
+        Write-Host "Move PyPy '${pythonVersion}' files to '${pypyArchPath}'"
+        Invoke-SBWithRetry -Command {
+            Move-Item -Path $tempFolder -Destination $pypyArchPath -ErrorAction Stop | Out-Null
+        }
+
+        Write-Host "Install PyPy '${pythonVersion}' in '${pypyArchPath}'"
+        if (Test-Path "$pypyArchPath\python.exe") {
+            cmd.exe /c "cd /d $pypyArchPath && python.exe -m ensurepip && python.exe -m pip install --upgrade pip"
+        } else {
+            $pypyName = $pypyApp.Name
+            cmd.exe /c "cd /d $pypyArchPath && mklink python.exe $pypyName && python.exe -m ensurepip && python.exe -m pip install --upgrade pip"
+        }
+
+        # Create pip.exe if missing
+        $pipPath = Join-Path -Path $pypyArchPath -ChildPath "Scripts/pip.exe"
+        if (-not (Test-Path $pipPath))
+        {
+            $pip3Path = Join-Path -Path $pypyArchPath -ChildPath "Scripts/pip3.exe"
+            Copy-Item -Path $pip3Path -Destination $pipPath 
+        }
+
+        if ($LASTEXITCODE -ne 0)
+        {
+            Throw "Error happened during PyPy installation"
+            exit 1
+        }
+
+        Write-Host "Create complete file"
+        New-Item -ItemType File -Path $pypyVersionPath -Name "$architecture.complete" | Out-Null
+    }
+    else
+    {
+        Write-Host "PyPy application is not found. Failed to expand '$packagePath' archive"
+        exit 1
+    }
+}
+
+# Get PyPy content from toolset
+$toolsetVersions = Get-ToolsetContent | Select-Object -ExpandProperty toolcache | Where-Object Name -eq "PyPy"
+
+# Get PyPy releases
+$pypyVersions = Invoke-RestMethod https://downloads.python.org/pypy/versions.json
+
+# required for html parsing
+$checksums = (Invoke-RestMethod -Uri 'https://www.pypy.org/checksums.html' | ConvertFrom-HTML).SelectNodes('//*[@id="content"]/article/div/pre')
+
+Write-Host "Starting installation PyPy..."
+foreach($toolsetVersion in $toolsetVersions.versions)
+{
+    # Query latest PyPy version
+    $latestMajorPyPyVersion = $pypyVersions |
+        Where-Object {$_.python_version.StartsWith("$toolsetVersion") -and $_.stable -eq $true} |
+        Select-Object -ExpandProperty files -First 1 |
+        Where-Object platform -like "win*"
+    
+    if ($latestMajorPyPyVersion)
+    {
+        $filename = $latestMajorPyPyVersion.filename
+        Write-Host "Found PyPy '$filename' package"
+        $tempPyPyPackagePath = Start-DownloadWithRetry -Url $latestMajorPyPyVersion.download_url -Name $filename
+
+        #region Supply chain security
+        $localFileHash = (Get-FileHash -Path $tempPyPyPackagePath -Algorithm SHA256).Hash
+        $distributorFileHash = $null
+
+        ForEach($node in $checksums) {
+            if($node.InnerText -ilike "*${filename}*") {
+                $distributorFileHash = $node.InnerText.ToString().Split("`n").Where({ $_ -ilike "*${filename}*" }).Split(' ')[0]
+            }
+        }
+        
+        Use-ChecksumComparison -LocalFileHash $localFileHash -DistributorFileHash $distributorFileHash
+        #endregion
+
+        Install-PyPy -PackagePath $tempPyPyPackagePath -Architecture $toolsetVersions.arch
+    }
+    else
+    {
+        Write-Host "Failed to query PyPy version '$toolsetVersion'"
+        exit 1
+    }
+}

images/windows/scripts/build/Install-Toolset.ps1

@@ -1,58 +1,58 @@
-################################################################################
-##  File:  Install-Toolset.ps1
-##  Team:  CI-Build
-##  Desc:  Install toolset
-################################################################################
-
-[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12 -bor [Net.SecurityProtocolType]::Tls13
-
-Function Install-Asset {
-    param(
-        [Parameter(Mandatory=$true)]
-        [object] $ReleaseAsset
-    )
-
-    $releaseAssetName = [System.IO.Path]::GetFileNameWithoutExtension($ReleaseAsset.filename)
-    $assetFolderPath = Join-Path $env:TEMP $releaseAssetName
-    $assetArchivePath = Start-DownloadWithRetry -Url $ReleaseAsset.download_url -Name $ReleaseAsset.filename
-
-    Write-Host "Extract $($ReleaseAsset.filename) content..."
-    if ($assetArchivePath.EndsWith(".tar.gz")) {
-        $assetTarPath = $assetArchivePath.TrimEnd(".tar.gz")
-        Extract-7Zip -Path $assetArchivePath -DestinationPath $assetTarPath
-        Extract-7Zip -Path $assetTarPath -DestinationPath $assetFolderPath
-    } else {
-        Extract-7Zip -Path $assetArchivePath -DestinationPath $assetFolderPath
-    }
-
-    Write-Host "Invoke installation script..."
-    Push-Location -Path $assetFolderPath
-    Invoke-Expression .\setup.ps1
-    Pop-Location
-}
-
-# Get toolcache content from toolset
-$ToolsToInstall = @("Python", "Node", "Go")
-
-$tools = Get-ToolsetContent | Select-Object -ExpandProperty toolcache | Where-Object { $ToolsToInstall -contains $_.Name }
-
-foreach ($tool in $tools) {
-    # Get versions manifest for current tool
-    $assets = Invoke-SBWithRetry -Command { Invoke-RestMethod $tool.url }
-
-    # Get github release asset for each version
-    foreach ($toolVersion in $tool.versions) {
-        $asset = $assets | Where-Object version -like $toolVersion `
-                         | Select-Object -ExpandProperty files `
-                         | Where-Object { ($_.platform -eq $tool.platform) -and ($_.arch -eq $tool.arch) -and ($_.toolset -eq $tool.toolset) } `
-                         | Select-Object -First 1
-
-        Write-Host "Installing $($tool.name) $toolVersion $($tool.arch)..."
-        if ($null -ne $asset) {
-            Install-Asset -ReleaseAsset $asset
-        } else {
-            Write-Host "Asset was not found in versions manifest"
-            exit 1
-        }
-    }
+################################################################################
+##  File:  Install-Toolset.ps1
+##  Team:  CI-Build
+##  Desc:  Install toolset
+################################################################################
+
+[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12 -bor [Net.SecurityProtocolType]::Tls13
+
+Function Install-Asset {
+    param(
+        [Parameter(Mandatory=$true)]
+        [object] $ReleaseAsset
+    )
+
+    $releaseAssetName = [System.IO.Path]::GetFileNameWithoutExtension($ReleaseAsset.filename)
+    $assetFolderPath = Join-Path $env:TEMP $releaseAssetName
+    $assetArchivePath = Start-DownloadWithRetry -Url $ReleaseAsset.download_url -Name $ReleaseAsset.filename
+
+    Write-Host "Extract $($ReleaseAsset.filename) content..."
+    if ($assetArchivePath.EndsWith(".tar.gz")) {
+        $assetTarPath = $assetArchivePath.TrimEnd(".tar.gz")
+        Extract-7Zip -Path $assetArchivePath -DestinationPath $assetTarPath
+        Extract-7Zip -Path $assetTarPath -DestinationPath $assetFolderPath
+    } else {
+        Extract-7Zip -Path $assetArchivePath -DestinationPath $assetFolderPath
+    }
+
+    Write-Host "Invoke installation script..."
+    Push-Location -Path $assetFolderPath
+    Invoke-Expression .\setup.ps1
+    Pop-Location
+}
+
+# Get toolcache content from toolset
+$ToolsToInstall = @("Python", "Node", "Go")
+
+$tools = Get-ToolsetContent | Select-Object -ExpandProperty toolcache | Where-Object { $ToolsToInstall -contains $_.Name }
+
+foreach ($tool in $tools) {
+    # Get versions manifest for current tool
+    $assets = Invoke-SBWithRetry -Command { Invoke-RestMethod $tool.url }
+
+    # Get github release asset for each version
+    foreach ($toolVersion in $tool.versions) {
+        $asset = $assets | Where-Object version -like $toolVersion `
+                         | Select-Object -ExpandProperty files `
+                         | Where-Object { ($_.platform -eq $tool.platform) -and ($_.arch -eq $tool.arch) -and ($_.toolset -eq $tool.toolset) } `
+                         | Select-Object -First 1
+
+        Write-Host "Installing $($tool.name) $toolVersion $($tool.arch)..."
+        if ($null -ne $asset) {
+            Install-Asset -ReleaseAsset $asset
+        } else {
+            Write-Host "Asset was not found in versions manifest"
+            exit 1
+        }
+    }
 }

images/windows/scripts/build/Install-Vsix.ps1

@@ -1,24 +1,24 @@
-###################################################################################
-##  File:  Install-Vsix.ps1
-##  Desc:  Install the Visual Studio Extensions from toolset.json
-###################################################################################
-
-$toolset = Get-ToolsetContent
-$vsixPackagesList = $toolset.visualStudio.vsix
-if (-not $vsixPackagesList) {
-    Write-Host "No extensions to install"
-    exit 0
-}
-
-$vsixPackagesList | ForEach-Object {
-    # Retrieve cdn endpoint to avoid HTTP error 429 https://github.com/actions/runner-images/issues/3074
-    $vsixPackage = Get-VsixExtenstionFromMarketplace -ExtensionMarketPlaceName $_
-    if ($vsixPackage.FileName.EndsWith(".vsix")) {
-        Install-VsixExtension -Url $vsixPackage.DownloadUri -Name $vsixPackage.FileName
-    } else {
-        $argumentList = ('/install', '/quiet', '/norestart')
-        Install-Binary -Url $vsixPackage.DownloadUri -Name $vsixPackage.FileName -ArgumentList $argumentList
-    }
-}
-
+###################################################################################
+##  File:  Install-Vsix.ps1
+##  Desc:  Install the Visual Studio Extensions from toolset.json
+###################################################################################
+
+$toolset = Get-ToolsetContent
+$vsixPackagesList = $toolset.visualStudio.vsix
+if (-not $vsixPackagesList) {
+    Write-Host "No extensions to install"
+    exit 0
+}
+
+$vsixPackagesList | ForEach-Object {
+    # Retrieve cdn endpoint to avoid HTTP error 429 https://github.com/actions/runner-images/issues/3074
+    $vsixPackage = Get-VsixExtenstionFromMarketplace -ExtensionMarketPlaceName $_
+    if ($vsixPackage.FileName.EndsWith(".vsix")) {
+        Install-VsixExtension -Url $vsixPackage.DownloadUri -Name $vsixPackage.FileName
+    } else {
+        $argumentList = ('/install', '/quiet', '/norestart')
+        Install-Binary -Url $vsixPackage.DownloadUri -Name $vsixPackage.FileName -ArgumentList $argumentList
+    }
+}
+
 Invoke-PesterTests -TestFile "Vsix"