Re-order build pipeline to perform all "build tasks" successfully before launching aqa-tests

[andrew-m-leonard]

As the build pipeline overall result is a propagation of all sub-jobs, including aqa-tests, all it takes for the overall Failure to be reported is a failed test-job. That result maybe resolved by a simple re-Grinder... however that Failure result might mask an important failure like "Signing", as was the case for jdk8u422-b05 Mac x64, which was published without noticing the failure.

We should "gate" the running of AQA tests in the successful completion of all the "Build stages", thus an important build failure will get immediately noticed during AQA triage, since no tests will have been run !

[llxia]

I believe it would be more effective not to use AQA tests as a blocker for highlighting installer failures. If the issue is related to pipeline propagation, addressing the root cause would be a better approach.

Furthermore, I kindly suggest that any future changes related to AQA tests involve the AQAvit leads in issue discussion and PR review.

FYI @smlambert

[sxa]

This was discussed with Shelley before implementing (in fact we spoke about it in a meeting earlier today as I wasn't sure if the PR had been merged yet)

[smlambert]

Signing failures were not spotted as part of July release and a respin was required, so this change was identified as part of the retrospective activities. If it is not something wanted by all, we can look at using a parameter to gate or not gate the launch of the tests.

[llxia]

We run nightly and weekly builds. It appears that the installer isn't working for the nightly build (assuming it works for the release). These installer failures are currently blocking our nightly testing.

FYI @AdamBrousseau @pshipton

[sxa]

We run nightly and weekly builds. It appears that the installer isn't working for the nightly build (assuming it works for the release). These installer failures are currently blocking our nightly testing.

FYI @AdamBrousseau @pshipton

Do you believe that is related to the ordering in the pipeline? And which installer are you seeing issues with? Adoptium only produces the Mac and windows ones by default for our regular beta (non-release) builds.

[AdamBrousseau]

Semeru produces nightly rpms. They're only failing for 2, it seems. We either didn't notice before or didn't care enough to fix it. But this change brings it to light, which is good and bad. We've opened an issue internally to fix it but in the meantime we don't get any testing on any Linux platforms.

[sxa]

Semeru produces nightly rpms. They're only failing for 2, it seems. We either didn't notice before or didn't care enough to fix it. But this change brings it to light, which is good and bad. We've opened an issue internally to fix it but in the meantime we don't get any testing on any Linux platforms.

OK that makes sense -so you've got the Linux installers added into the build pipelines somewhere that causes it to trip up over this change presumably because it now stops on the installer failure (either just it failing, or not being able to check it's signing?)

[AdamBrousseau]

Correct, stops because the installer step failed. If it was configurable, we could choose to non block until it's resolved. But I could probably just as easily revert the change on my end until it's resolved.