get latest nightly build files from given base url

[OscarQQ]

related: backlog issue 542
Signed-off-by: Yixin Qian Yixin.Qian@ibm.com

[OscarQQ]

run aqa --sdk_resource nightly --build_list functional --target floatSanityTests --platform x86-64_linux --customized_sdk_url https://openj9-artifactory.osuosl.org/artifactory/ci-openj9/Build_JDK11_x86-64_linux_Nightly/

[github-actions]

@OscarQQ

Traceback (most recent call last):
 File "TKG/scripts/testBot/runAqaArgParse.py", line 121, in <module>
 main()
 File "TKG/scripts/testBot/runAqaArgParse.py", line 64, in main
 if config['custom_openj9_repo']:
KeyError: 'custom_openj9_repo'

No builds were started.

[OscarQQ]

run aqa --sdk_resource nightly --build_list functional --target testSCCMLTests2 --platform x86-64_windows --customized_sdk_url https://openj9-artifactory.osuosl.org/artifactory/ci-openj9/Build_JDK11_x86-64_linux_Nightly/

[github-actions]

@OscarQQ

Traceback (most recent call last):
 File "TKG/scripts/testBot/runAqaArgParse.py", line 121, in <module>
 main()
 File "TKG/scripts/testBot/runAqaArgParse.py", line 64, in main
 if config['custom_openj9_repo']:
KeyError: 'custom_openj9_repo'

No builds were started.

[OscarQQ]

https://ci.adoptopenjdk.net/view/Test_grinder/job/Grinder/1000/consoleFull
getting tar.gz given the base url https://openj9-artifactory.osuosl.org/artifactory/ci-openj9/Build_JDK11_x86-64_linux_Nightly/

[llxia]

Regarding run aqa failure (#2695 (comment)), issue opened at #2697

[llxia]

Thanks @OscarQQ . With this PR, we will allow users to provide a base artifactory URL (URL before the build number). Our test script will try to figure out the latest build number and it will try to download all tar (including SDK, test image, debug image, etc).

Use case in Grinder:
SDK_RESOURCE=nightly
CUSTOMIZED_SDK_URL=https://openj9-artifactory.osuosl.org/artifactory/ci-openj9/Build_JDK11_x86-64_linux_Nightly

[llxia]

@OscarQQ users may or may not have / at the end of the URL, please make sure we cover both cases. That is, https://ci.adoptopenjdk.net/view/Test_grinder/job/Grinder/1001/console should work. Thanks

[OscarQQ]

Grinder/16835. aix
Grinder/16836 x86-64_windows
Grinder/16837. zos
Grinder/16838. x86-64_linux
Grinder/16839. ppc64le
Grinder/16840. s390x
Grinder/16841. x86-64_mac

[llxia]

LGTM. Thanks a lot @OscarQQ
During the testing, we noticed that the latest build may not contain SDK (due to SDK build failure). We will add further enhancement in future PR - try old builds if the latest build does not contain SDK.

[smlambert]

thanks for working on this enhancement @OscarQQ ! LGTM