Skip to content

Commit d3e0fc8

Browse files
authored
[CI] Avoid use of scipt: | blocks (#4634)
Context: https://github.com/MicrosoftDocs/azure-devops-docs/issues/3265 Context: https://devdiv.visualstudio.com/DevDiv/_build/results?buildId=3679916&view=logs&j=8556562a-ae5f-5bd1-7c4d-bf1af4b6f1e1&t=5076e147-fc66-561e-6c69-3aa777afefc5 We sometimes observe experiencing "weird issue" on AzDO CI in which the **Test** > **APK Instrumentation** step fails: …/external/Java.Interop/src/Java.Interop/Java.Interop.targets(37,5): error MSB3073: The command """ -source 1.6 -target 1.6 -bootclasspath "" -d "obj/Release/ji-classes" java/com/xamarin/java_interop/internal/JavaProxyObject.java java/com/xamarin/java_interop/internal/JavaProxyThrowable.java java/com/xamarin/java_interop/GCUserPeerable.java java/com/xamarin/java_interop/ManagedPeer.java" exited with code 127. The cause of the issue is that `make -C external/Java.Interop prepare` is never executed, which is *bizarre*, because `xaprepare -s=Required` is responsible for calling it; see 130905e. Checking the build logs, and we see that the **install test dependencies** job *appears* to run it: ##[debug]script=mono build-tools/xaprepare/xaprepare/bin/Release/xaprepare.exe --s=UpdateMono --auto-provision=yes --auto-provision-uses-sudo=yes --no-emoji --run-mode=CI mono build-tools/xaprepare/xaprepare/bin/Release/xaprepare.exe --s=Required --auto-provision=yes --auto-provision-uses-sudo=yes --no-emoji --run-mode=CI mono build-tools/xaprepare/xaprepare/bin/Release/xaprepare.exe --s=AndroidToolchain --no-emoji --run-mode=CI …but it doesn't actually *successfully* run *all* those commands! Instead, only `xaprepare -s=AndroidToolchain` finishes: % grep 'Running scenario' prepare-* prepare-20200427T203230.log:[00:00:21.6506301] Running scenario: Install Android SDK, NDK and Corretto JDK. Compare to: % grep 'Initializing scenario' prepare-* prepare-20200427T203008.log:[00:00:00.2793986] Initializing scenario UpdateMono prepare-20200427T203116.log:[00:00:00.4118305] Initializing scenario Required prepare-20200427T203230.log:[00:00:00.2998047] Initializing scenario AndroidToolchain It appears that at present Azure DevOps `script` blocks don't run via [`bash -e`][0] -- which would cause a failure if any command exits with a failre -- which means that the errors emitted from `xaprepare -s=UpdateMono` and `xaprepare -s=Required` are *ignored*. ***We do not want to ignore errors.*** As such, @jonpryor now regards *all* `script: |` YAML as potentially wrong or buggy. Replace all `script: |` blocks with `script: >` blocks. `script: >` blocks replace newlines with a space, turning the block into a *single command*. This in turn requires inserting `&&` between intermediate commands, so that subsequent commands only execute if the previous commands exited successfully. Thus, turn this: - script: | command1 command2 into: - script: > command1 && command2 The DevOps `script` handler doesn't ignore *all* errors, and the above transformation means that the `script` block itself will exit with a failure if any command within the pipeline fails. [0]: https://stackoverflow.com/a/9952249
1 parent bea063b commit d3e0fc8

File tree

4 files changed

+31
-31
lines changed

4 files changed

+31
-31
lines changed

build-tools/automation/azure-pipelines-oss.yaml

Lines changed: 11 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -32,12 +32,12 @@ jobs:
3232
inputs:
3333
versionSpec: 5.x
3434

35-
- script: |
36-
sudo apt-get install -y gnupg ca-certificates
37-
sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys 3FA7E0328081BFF6A14DA29AA6A19B38D3D831EF
38-
echo "deb https://download.mono-project.com/repo/ubuntu preview-bionic main" | sudo tee /etc/apt/sources.list.d/mono-official-preview.list
39-
sudo apt-get update
40-
sudo apt-get install -y mono-devel
35+
- script: >
36+
sudo apt-get install -y gnupg ca-certificates &&
37+
sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys 3FA7E0328081BFF6A14DA29AA6A19B38D3D831EF &&
38+
(echo "deb https://download.mono-project.com/repo/ubuntu preview-bionic main" | sudo tee /etc/apt/sources.list.d/mono-official-preview.list) &&
39+
sudo apt-get update &&
40+
sudo apt-get install -y mono-devel &&
4141
sudo apt-get install -y ca-certificates-mono
4242
displayName: install mono preview
4343
@@ -47,11 +47,11 @@ jobs:
4747
- script: make package-deb V=1 CONFIGURATION=$(OSSBuildConfiguration)
4848
displayName: make package-deb
4949

50-
- script: |
51-
mkdir -p $(System.DefaultWorkingDirectory)/bin/Build$(OSSBuildConfiguration)/linux-artifacts
52-
cp $(System.DefaultWorkingDirectory)/*xamarin.android*.tar.bz2 $(System.DefaultWorkingDirectory)/bin/Build$(OSSBuildConfiguration)/linux-artifacts
53-
cp $(System.DefaultWorkingDirectory)/*.changes $(System.DefaultWorkingDirectory)/bin/Build$(OSSBuildConfiguration)/linux-artifacts
54-
cp $(System.DefaultWorkingDirectory)/*.dsc $(System.DefaultWorkingDirectory)/bin/Build$(OSSBuildConfiguration)/linux-artifacts
50+
- script: >
51+
mkdir -p $(System.DefaultWorkingDirectory)/bin/Build$(OSSBuildConfiguration)/linux-artifacts &&
52+
cp $(System.DefaultWorkingDirectory)/*xamarin.android*.tar.bz2 $(System.DefaultWorkingDirectory)/bin/Build$(OSSBuildConfiguration)/linux-artifacts &&
53+
cp $(System.DefaultWorkingDirectory)/*.changes $(System.DefaultWorkingDirectory)/bin/Build$(OSSBuildConfiguration)/linux-artifacts &&
54+
cp $(System.DefaultWorkingDirectory)/*.dsc $(System.DefaultWorkingDirectory)/bin/Build$(OSSBuildConfiguration)/linux-artifacts &&
5555
cp $(System.DefaultWorkingDirectory)/*.deb $(System.DefaultWorkingDirectory)/bin/Build$(OSSBuildConfiguration)/linux-artifacts
5656
displayName: copy linux artifacts
5757

build-tools/automation/azure-pipelines.yaml

Lines changed: 11 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -141,8 +141,8 @@ stages:
141141
workingDirectory: $(System.DefaultWorkingDirectory)/xamarin-android
142142
displayName: make all-tests
143143

144-
- script: |
145-
cp -r bin/$(XA.Build.Configuration)/bcl-tests bin/Test$(XA.Build.Configuration)/bcl-tests
144+
- script: >
145+
cp -r bin/$(XA.Build.Configuration)/bcl-tests bin/Test$(XA.Build.Configuration)/bcl-tests &&
146146
cp bin/Build$(XA.Build.Configuration)/ProfileAssemblies.projitems bin/Test$(XA.Build.Configuration)/bcl-tests/
147147
workingDirectory: $(System.DefaultWorkingDirectory)/xamarin-android
148148
displayName: copy bcl-tests assemblies
@@ -166,15 +166,15 @@ stages:
166166
workingDirectory: $(System.DefaultWorkingDirectory)/xamarin-android
167167
displayName: make create-installers
168168

169-
- script: |
170-
mkdir -p bin/Build$(XA.Build.Configuration)/$(InstallerArtifactName)
171-
cp bin/Build$(XA.Build.Configuration)/*.vsix bin/Build$(XA.Build.Configuration)/$(InstallerArtifactName)
169+
- script: >
170+
mkdir -p bin/Build$(XA.Build.Configuration)/$(InstallerArtifactName) &&
171+
cp bin/Build$(XA.Build.Configuration)/*.vsix bin/Build$(XA.Build.Configuration)/$(InstallerArtifactName) &&
172172
cp bin/Build$(XA.Build.Configuration)/*.pkg bin/Build$(XA.Build.Configuration)/$(InstallerArtifactName)
173173
workingDirectory: $(System.DefaultWorkingDirectory)/xamarin-android
174174
displayName: copy unsigned installers
175175
176-
- script: |
177-
VERSION=`LANG=C; export LANG && git log --no-color --first-parent -n1 --pretty=format:%ct`
176+
- script: >
177+
VERSION=`LANG=C; export LANG && git log --no-color --first-parent -n1 --pretty=format:%ct` &&
178178
echo "d1ec039f-f3db-468b-a508-896d7c382999 $VERSION" > bin/Build$(XA.Build.Configuration)/$(InstallerArtifactName)/updateinfo
179179
workingDirectory: $(System.DefaultWorkingDirectory)/xamarin-android
180180
displayName: create updateinfo file
@@ -402,10 +402,10 @@ stages:
402402
parameters:
403403
UnsignedPkgPath: $(XA.Unsigned.Pkg)
404404

405-
- script: |
406-
cd $(System.DefaultWorkingDirectory)/release-scripts
407-
git checkout $(ReleaseScriptsBranch)
408-
sudo xcode-select -s /Applications/$(NotarizationXcode)
405+
- script: >
406+
cd $(System.DefaultWorkingDirectory)/release-scripts &&
407+
git checkout $(ReleaseScriptsBranch) &&
408+
sudo xcode-select -s /Applications/$(NotarizationXcode) &&
409409
ruby notarize.rb $(XA.Unsigned.Pkg) $(XamarinIdentifier) $(XamarinUserId) $(XamarinPassword) $(TeamID)
410410
displayName: Notarize PKG
411411

build-tools/automation/yaml-templates/apk-instrumentation.yaml

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -24,10 +24,10 @@ steps:
2424
condition: ${{ parameters.condition }}
2525
continueOnError: true
2626

27-
- script: |
28-
SOURCE=$(System.DefaultWorkingDirectory)/bin/Test${{ parameters.configuration }}/${{ parameters.artifactName }}
29-
DEST=$(Build.ArtifactStagingDirectory)/${{ parameters.artifactFolder }}/
30-
mkdir -p "$DEST"
27+
- script: >
28+
SOURCE="$(System.DefaultWorkingDirectory)/bin/Test${{ parameters.configuration }}/${{ parameters.artifactName }}" &&
29+
DEST="$(Build.ArtifactStagingDirectory)/${{ parameters.artifactFolder }}/" &&
30+
mkdir -p "$DEST" &&
3131
cp "$SOURCE" "$DEST"
3232
displayName: copy apk/aab
3333
condition: ${{ parameters.condition }}

build-tools/automation/yaml-templates/setup-test-environment.yaml

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -39,15 +39,15 @@ steps:
3939
configuration: ${{ parameters.configuration }}
4040
msbuildArguments: /restore
4141

42-
- script: |
43-
mono build-tools/xaprepare/xaprepare/bin/${{ parameters.configuration }}/xaprepare.exe --s=UpdateMono --auto-provision=yes --auto-provision-uses-sudo=yes --no-emoji --run-mode=CI
44-
mono build-tools/xaprepare/xaprepare/bin/${{ parameters.configuration }}/xaprepare.exe --s=Required --auto-provision=yes --auto-provision-uses-sudo=yes --no-emoji --run-mode=CI
42+
- script: >
43+
mono build-tools/xaprepare/xaprepare/bin/${{ parameters.configuration }}/xaprepare.exe --s=UpdateMono --auto-provision=yes --auto-provision-uses-sudo=yes --no-emoji --run-mode=CI &&
44+
mono build-tools/xaprepare/xaprepare/bin/${{ parameters.configuration }}/xaprepare.exe --s=Required --auto-provision=yes --auto-provision-uses-sudo=yes --no-emoji --run-mode=CI &&
4545
mono build-tools/xaprepare/xaprepare/bin/${{ parameters.configuration }}/xaprepare.exe --s=AndroidToolchain --no-emoji --run-mode=CI
4646
displayName: install test dependencies
4747
condition: and(succeeded(), eq(variables['agent.os'], 'Darwin'))
4848

49-
- script: |
50-
$(System.DefaultWorkingDirectory)\build-tools\xaprepare\xaprepare\bin\${{ parameters.configuration }}\xaprepare.exe --s=Required --auto-provision=yes --no-emoji --run-mode=CI
49+
- script: >
50+
$(System.DefaultWorkingDirectory)\build-tools\xaprepare\xaprepare\bin\${{ parameters.configuration }}\xaprepare.exe --s=Required --auto-provision=yes --no-emoji --run-mode=CI &&
5151
$(System.DefaultWorkingDirectory)\build-tools\xaprepare\xaprepare\bin\${{ parameters.configuration }}\xaprepare.exe --s=AndroidToolchain --no-emoji --run-mode=CI
5252
displayName: install test dependencies
5353
condition: and(succeeded(), eq(variables['agent.os'], 'Windows_NT'))

0 commit comments

Comments
 (0)