Skip to content

Create OneBranch build and release pipeline #1605

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 65 commits into from
Mar 22, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
65 commits
Select commit Hold shift + click to select a range
d45834c
Adding OneBranch pipeline YAML config file for PSResourceGet-Official
Mar 15, 2024
fdbff8a
Add yaml and tsaoptions.json
alerickson Mar 15, 2024
9d5563c
Add space in yaml
alerickson Mar 15, 2024
4d3a88b
Change cred in test
alerickson Mar 15, 2024
c3282f9
Add tsaoptions json
alerickson Mar 15, 2024
da1cbd7
Change OSS_Microsoft_PSResourceGet to PSResourceGet
alerickson Mar 15, 2024
13b4429
Add global.json
alerickson Mar 15, 2024
ef432fd
Add capture environment task
alerickson Mar 15, 2024
b521cbe
update Capture Env task
alerickson Mar 15, 2024
7a4043d
Update Capture env task
alerickson Mar 15, 2024
242be8e
Add logging for PSResourceGet version and powershell version
alerickson Mar 15, 2024
4e6a555
Add Gmo PSResourceGet -ListAvailable
alerickson Mar 15, 2024
162124d
Install PSResourceGet
alerickson Mar 15, 2024
f7719b8
Install PSResourceGet again when creating nupkg
alerickson Mar 15, 2024
78c54dc
Add release stage
alerickson Mar 15, 2024
aed53a9
Change indentation
alerickson Mar 15, 2024
9e7977c
Update release stage
alerickson Mar 15, 2024
644cae7
Remove extra pwsh task
alerickson Mar 15, 2024
d4b5c63
Change job name
alerickson Mar 15, 2024
6ac4055
change job name again
alerickson Mar 15, 2024
2ebe805
Change stage and job names
alerickson Mar 15, 2024
59c1e2d
Change depends on
alerickson Mar 15, 2024
38cf94f
Add logging
alerickson Mar 16, 2024
27a3819
add more logging
alerickson Mar 16, 2024
a67171f
Add copy task
alerickson Mar 16, 2024
14f060a
Change publish artifact
alerickson Mar 17, 2024
8ba9544
and publish pipeline artifact tast
alerickson Mar 17, 2024
3f30af4
move indentation
alerickson Mar 17, 2024
2f68e9c
Fix indentations
alerickson Mar 17, 2024
fbb8ed4
REmove publish task
alerickson Mar 17, 2024
eb3c8a6
Add changes to build job
alerickson Mar 17, 2024
d76fa80
uncomment some names
alerickson Mar 17, 2024
2b302f0
fix indentations
alerickson Mar 17, 2024
3f0b7aa
Revert yaml
alerickson Mar 17, 2024
e3e493c
more updates to build, package and sign
alerickson Mar 17, 2024
7be1d90
Remove quotations
alerickson Mar 17, 2024
030ccb2
Change SourceLocation to Uri
alerickson Mar 17, 2024
e225002
update variable
alerickson Mar 17, 2024
adf4f10
find nupkg for release
alerickson Mar 17, 2024
e351016
Update pkg path to publish
alerickson Mar 17, 2024
e3da6b2
Update Nuget push task
alerickson Mar 17, 2024
8b37f89
update pkg to push
alerickson Mar 17, 2024
1215fd3
add missing )
alerickson Mar 17, 2024
529239e
Add test version of module
alerickson Mar 18, 2024
9757cc0
fix validation stage
alerickson Mar 18, 2024
b1d71de
Clean up
alerickson Mar 18, 2024
2cbae2c
Update version
alerickson Mar 18, 2024
1196496
Uncomment 2 tasks
alerickson Mar 18, 2024
70ada34
uncomment another task
alerickson Mar 18, 2024
91267ff
Uncomment more
alerickson Mar 18, 2024
cccc089
Update .config/tsaoptions.json
alerickson Mar 21, 2024
a1cb415
Update global.json
alerickson Mar 21, 2024
9601e12
Update .pipelines/PSResourceGet-Official.yml
alerickson Mar 21, 2024
66eadab
Add code review suggestions
alerickson Mar 21, 2024
c86b835
Merge branch 'createOneBranchPipeline' of https://github.com/alericks…
alerickson Mar 21, 2024
0d4b4e1
Add 3rd party signing validation/copy
alerickson Mar 21, 2024
e25467a
Update 3rd party signing path
alerickson Mar 21, 2024
268e2d5
Update 3rd party signing files to sign
alerickson Mar 21, 2024
74f9c53
Add debugging for signing validation
alerickson Mar 21, 2024
f5c189e
Update cert check
alerickson Mar 21, 2024
778cebc
Move signed file back into depsPath
alerickson Mar 21, 2024
852ebe0
Clean up depsPath
alerickson Mar 21, 2024
62b25d7
Remove whitespace
alerickson Mar 21, 2024
feb3ee6
Update dotnet sdk
alerickson Mar 21, 2024
3d47762
Merge branch 'OBP_Official_americks_3PXrcq' of https://dev.azure.com/…
alerickson Mar 21, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions .config/tsaoptions.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
{
"instanceUrl": "https://msazure.visualstudio.com",
"projectName": "One",
"areaPath": "One\\MGMT\\Compute\\Powershell\\Powershell\\PowerShell Core",
"notificationAliases": [
"adityap@microsoft.com",
"americks@microsoft.com",
"annavied@microsoft.com"
]
}
334 changes: 334 additions & 0 deletions .pipelines/PSResourceGet-Official.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,334 @@
#################################################################################
# OneBranch Pipelines #
# This pipeline was created by EasyStart from a sample located at: #
# https://aka.ms/obpipelines/easystart/samples #
# Documentation: https://aka.ms/obpipelines #
# Yaml Schema: https://aka.ms/obpipelines/yaml/schema #
# Retail Tasks: https://aka.ms/obpipelines/tasks #
# Support: https://aka.ms/onebranchsup #
#################################################################################
name: PSResourceGet-Release-$(Build.BuildId)
trigger: none # https://aka.ms/obpipelines/triggers
pr:
branches:
include:
- main
- release*
parameters: # parameters are shown up in ADO UI in a build queue time
- name: 'debug'
displayName: 'Enable debug output'
type: boolean
default: false

variables:
- name: DOTNET_CLI_TELEMETRY_OPTOUT
value: 1
- name: POWERSHELL_TELEMETRY_OPTOUT
value: 1
- name: WindowsContainerImage
value: onebranch.azurecr.io/windows/ltsc2022/vse2022:latest # Docker image which is used to build the project https://aka.ms/obpipelines/containers

resources:
repositories:
- repository: onebranchTemplates
type: git
name: OneBranch.Pipelines/GovernedTemplates
ref: refs/heads/main

extends:
template: v2/OneBranch.Official.CrossPlat.yml@onebranchTemplates # https://aka.ms/obpipelines/templates
parameters:
featureFlags:
WindowsHostVersion: '1ESWindows2022'
customTags: 'ES365AIMigrationTooling'
globalSdl:
disableLegacyManifest: true
sbom:
enabled: true
packageName: Microsoft.PowerShell.PSResourceGet
codeql:
compiled:
enabled: true
asyncSdl: # https://aka.ms/obpipelines/asyncsdl
enabled: true
forStages: [stagebuild]
credscan:
enabled: true
scanFolder: $(Build.SourcesDirectory)\PSResourceGet
binskim:
enabled: true
apiscan:
enabled: false

stages:
- stage: stagebuild
displayName: Build and Package Microsoft.PowerShell.PSResourceGet
jobs:
- job: jobbuild
displayName: Build Microsoft.PowerShell.PSResourceGet Files
variables: # More settings at https://aka.ms/obpipelines/yaml/jobs
- name: ob_outputDirectory
value: '$(Build.ArtifactStagingDirectory)/ONEBRANCH_ARTIFACT'
- name: repoRoot
value: $(Build.SourcesDirectory)\PSResourceGet
- name: ob_sdl_tsa_configFile
value: $(Build.SourcesDirectory)\PSResourceGet\.config\tsaoptions.json
- name: signSrcPath
value: $(repoRoot)/out
- name: depsPath
value: $(signSrcPath)\Microsoft.PowerShell.PSResourceGet\Dependencies
- name: ob_sdl_sbom_enabled
value: true
- name: ob_signing_setup_enabled
value: true
#CodeQL tasks added manually to workaround signing failures
- name: ob_sdl_codeql_compiled_enabled
value: false
pool:
type: windows
steps:
- checkout: self
env:
ob_restore_phase: true # Set ob_restore_phase to run this step before '🔒 Setup Signing' step.

- pwsh: |
if (-not (Test-Path $(repoRoot)/.config/tsaoptions.json)) {
Get-ChildItem $(Build.SourcesDirectory) -recurse -ErrorAction SilentlyContinue
throw "tsaoptions.json does not exist under $(repoRoot)/.config"
}
displayName: Test if tsaoptions.json exists
env:
ob_restore_phase: true # Set ob_restore_phase to run this step before '🔒 Setup Signing' step.

- pwsh: |
Get-ChildItem env:
displayName: Capture Environment
env:
ob_restore_phase: true # Set ob_restore_phase to run this step before '🔒 Setup Signing' step.

- task: UseDotNet@2
displayName: 'Install .NET dependencies'
env:
ob_restore_phase: true # Set ob_restore_phase to run this step before '🔒 Setup Signing' step.
inputs:
packageType: 'sdk'
useGlobalJson: true
# this is to ensure that we are installing the dotnet at the same location as container by default install the dotnet sdks
installationPath: 'C:\Program Files\dotnet\'
workingDirectory: $(repoRoot)

- task: CodeQL3000Init@0 # Add CodeQL Init task right before your 'Build' step.
env:
ob_restore_phase: true # Set ob_restore_phase to run this step before '🔒 Setup Signing' step.
inputs:
Enabled: true
AnalyzeInPipeline: true
Language: csharp

- pwsh: |
$module = 'Microsoft.PowerShell.PSResourceGet'
Write-Verbose "installing $module..." -verbose
$ProgressPreference = 'SilentlyContinue'
Install-Module $module -AllowClobber -Force
displayName: Install PSResourceGet 0.9.0 or above for build.psm1
env:
ob_restore_phase: true # Set ob_restore_phase to run this step before '🔒 Setup Signing' step.

# this is installing .NET
- pwsh: |
Set-Location "$(repoRoot)"
try { ./build.ps1 -Build -Clean -BuildConfiguration Release -BuildFramework 'net472'} catch { throw $_ }
displayName: Execute build
env:
ob_restore_phase: true # Set ob_restore_phase to run this step before '🔒 Setup Signing' step.

- task: CodeQL3000Finalize@0 # Add CodeQL Finalize task right after your 'Build' step.
condition: always()
env:
ob_restore_phase: true # Set ob_restore_phase to run this step before '🔒 Setup Signing' step.

- task: onebranch.pipeline.signing@1
displayName: Sign 1st party files
inputs:
command: 'sign'
signing_profile: external_distribution
files_to_sign: '**\*.psd1;**\*.psm1;**\*.ps1xml;**\Microsoft*.dll'
search_root: $(signSrcPath)

- pwsh: |
$unsignedDepsPath = Join-Path -Path $(signSrcPath) -ChildPath "Microsoft.PowerShell.PSResourceGet" -AdditionalChildPath "UnsignedDependencies"
New-Item -Path $unsignedDepsPath -ItemType Directory -Force

Get-ChildItem -Path $(depsPath) -Filter '*.dll' | Foreach-Object {
$sig = Get-AuthenticodeSignature -FilePath $_.FullName
if ($sig.Status -ne 'Valid' -or $sig.SignerCertificate.Subject -notlike '*Microsoft*' -or $sig.SignerCertificate.Issuer -notlike '*Microsoft Code Signing PCA*') {
# Copy for third party signing
Copy-Item -Path $_.FullName -Dest $unsignedDepsPath -Force -Verbose
}
}
displayName: Find all 3rd party files that need to be signed

- task: onebranch.pipeline.signing@1
displayName: Sign 3rd Party files
inputs:
command: 'sign'
signing_profile: 135020002
files_to_sign: '*.dll'
search_root: $(signSrcPath)/Microsoft.PowerShell.PSResourceGet/UnsignedDependencies

- pwsh: |
$newlySignedDepsPath = Join-Path -Path $(signSrcPath) -ChildPath "Microsoft.PowerShell.PSResourceGet" -AdditionalChildPath "UnsignedDependencies"
Get-ChildItem -Path $newlySignedDepsPath -Filter '*.dll' | Foreach-Object {
$sig = Get-AuthenticodeSignature -FilePath $_.FullName
if ($sig.Status -ne 'Valid' -or $sig.SignerCertificate.Subject -notlike '*Microsoft*' -or $sig.SignerCertificate.Issuer -notlike '*Microsoft Windows Production PCA*') {
Write-Error "File $($_.FileName) is not signed by Microsoft"
}
else {
Copy-Item -Path $_.FullName -Dest $(depsPath) -Force -Verbose
}
}
Remove-Item -Path $newlySignedDepsPath -Recurse -Force
displayName: Validate 3rd party files were signed

- task: CopyFiles@2
displayName: "Copy signed files to ob_outputDirectory - '$(ob_outputDirectory)'"
inputs:
SourceFolder: "$(signSrcPath)"
Contents: '**'
TargetFolder: $(ob_outputDirectory)

- pwsh: |
Write-Host "Displaying contents of signSrcPath:"
Get-ChildItem $(signSrcPath) -Recurse
Write-Host "Displaying contents of ob_outputDirectory:"
Get-ChildItem $(ob_outputDirectory) -Recurse
displayName: Get contents of dirs with signed files

- job: nupkg
dependsOn: jobbuild
displayName: Package Microsoft.PowerShell.PSResourceGet
variables:
- name: ob_outputDirectory
value: '$(Build.ArtifactStagingDirectory)/ONEBRANCH_ARTIFACT'
- name: repoRoot
value: $(Build.SourcesDirectory)\PSResourceGet
- name: ob_sdl_tsa_configFile
value: $(Build.SourcesDirectory)\PSResourceGet\.config\tsaoptions.json
# Disable because SBOM was already built in the previous job
- name: ob_sdl_sbom_enabled
value: false
- name: signOutPath
value: $(repoRoot)/signed
- name: ob_signing_setup_enabled
value: true
# This job is not compiling code, so disable codeQL
- name: ob_sdl_codeql_compiled_enabled
value: false

pool:
type: windows
steps:
- checkout: self

- pwsh: |
if (-not (Test-Path $(repoRoot)/.config/tsaoptions.json)) {
Get-ChildItem $(Build.SourcesDirectory) -recurse -ErrorAction SilentlyContinue
throw "tsaoptions.json does not exist under $(repoRoot)/.config"
}
displayName: Test if tsaoptions.json exists

- task: DownloadPipelineArtifact@2
displayName: 'Download build files'
inputs:
targetPath: $(signOutPath)
artifact: drop_stagebuild_jobbuild

- pwsh: |
Set-Location "$(signOutPath)"
Write-Host "Contents of signOutPath:"
Get-ChildItem $(signOutPath) -Recurse
displayName: Capture artifacts directory structure

- pwsh: |
$module = 'Microsoft.PowerShell.PSResourceGet'
Write-Verbose "installing $module..." -verbose
$ProgressPreference = 'SilentlyContinue'
Install-Module $module -AllowClobber -Force
displayName: Install PSResourceGet 0.9.0 or above for build.psm1

- pwsh: |
Set-Location "$(signOutPath)\Microsoft.PowerShell.PSResourceGet"
New-Item -ItemType Directory -Path "$(signOutPath)\PublishedNupkg" -Force
Register-PSResourceRepository -Name 'localRepo' -Uri "$(signOutPath)\PublishedNupkg"
Publish-PSResource -Path "$(signOutPath)\Microsoft.PowerShell.PSResourceGet" -Repository 'localRepo' -Verbose
displayName: Create nupkg for publishing

- task: onebranch.pipeline.signing@1
displayName: Sign nupkg
inputs:
command: 'sign'
signing_profile: external_distribution
files_to_sign: '**\*.nupkg'
search_root: "$(signOutPath)\PublishedNupkg"

- pwsh: |
Set-Location "$(signOutPath)\PublishedNupkg"
Write-Host "Contents of signOutPath:"
Get-ChildItem "$(signOutPath)" -Recurse
displayName: Find Nupkg

- task: CopyFiles@2
displayName: "Copy nupkg to ob_outputDirectory - '$(ob_outputDirectory)'"
inputs:
Contents: $(signOutPath)\PublishedNupkg\Microsoft.PowerShell.PSResourceGet.*.nupkg
TargetFolder: $(ob_outputDirectory)

- pwsh: |
Write-Host "Contents of ob_outputDirectory:"
Get-ChildItem "$(ob_outputDirectory)" -Recurse
displayName: Find Signed Nupkg

- stage: release
displayName: Release PSResourceGet
dependsOn: stagebuild
variables:
version: $[ stageDependencies.build.main.outputs['package.version'] ]
drop: $(Pipeline.Workspace)/drop_build_main
jobs:
- job: validation
displayName: Manual validation
pool:
type: agentless
timeoutInMinutes: 1440
steps:
- task: ManualValidation@0
displayName: Wait 24 hours for validation
inputs:
instructions: Please validate the release
timeoutInMinutes: 1440
- job: PSGalleryPublish
displayName: Publish to PSGallery
dependsOn: validation
pool:
type: windows
variables:
ob_outputDirectory: '$(Build.ArtifactStagingDirectory)/ONEBRANCH_ARTIFACT'
steps:
- download: current
displayName: Download artifact

- pwsh: |
Get-ChildItem $(Pipeline.Workspace) -Recurse
displayName: Capture environment

- pwsh: |
Get-ChildItem "$(Pipeline.Workspace)/drop_stagebuild_nupkg" -Recurse
displayName: Find signed Nupkg

- task: NuGetCommand@2
displayName: Push PowerShellGet module artifacts to PSGallery feed
inputs:
command: push
packagesToPush: '$(Pipeline.Workspace)\drop_stagebuild_nupkg\PSResourceGet\signed\PublishedNupkg\Microsoft.PowerShell.PSResourceGet.*.nupkg'
nuGetFeedType: external
publishFeedCredentials: PSGet-PSGalleryPush
5 changes: 5 additions & 0 deletions global.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
{
"sdk": {
"version": "8.0.202"
}
}
4 changes: 2 additions & 2 deletions test/PublishPSResourceTests/PublishPSResource.Tests.ps1
Original file line number Diff line number Diff line change
Expand Up @@ -433,8 +433,8 @@ Describe "Test Publish-PSResource" -tags 'CI' {
It "Publish a module to PSGallery using incorrect API key, should throw" {
$version = "1.0.0"
New-ModuleManifest -Path (Join-Path -Path $script:PublishModuleBase -ChildPath "$script:PublishModuleName.psd1") -ModuleVersion $version -Description "$script:PublishModuleName module"

Publish-PSResource -Path $script:PublishModuleBase -Repository PSGallery -APIKey "123456789" -ErrorAction SilentlyContinue
$APIKey = New-Guid
Publish-PSResource -Path $script:PublishModuleBase -Repository PSGallery -APIKey $APIKey -ErrorAction SilentlyContinue

$Error[0].FullyQualifiedErrorId | Should -be "403Error,Microsoft.PowerShell.PSResourceGet.Cmdlets.PublishPSResource"
}
Expand Down