Skip to content

[Yamato] Succeed on no tests found by testfilter #2954

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Dec 22, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .yamato/all-hdrp.yml
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,9 @@ Nightly_HDRP_trunk:
rerun: on-new-revision
- path: .yamato/hdrp-linux-vulkan.yml#HDRP_Linux_Vulkan_editmode_mono_Linear_trunk
rerun: on-new-revision
variables:
UTR_VERSION: "current"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The "current" variable everywhere is just because we forgot to add it in all the jobs when making the PR about this feature, right? It is technically not related to the test filter?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep correct, forgot to mention that, sorry!

TEST_FILTER: .*
PR_HDRP_CUSTOM-REVISION:
name: HDRP PR Job - CUSTOM-REVISION
dependencies:
Expand Down Expand Up @@ -118,3 +121,5 @@ Nightly_HDRP_CUSTOM-REVISION:
rerun: always
variables:
CUSTOM_REVISION: custom_revision_not_set
UTR_VERSION: "current"
TEST_FILTER: .*
5 changes: 5 additions & 0 deletions .yamato/all-hdrp_dxr.yml
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,9 @@ Nightly_HDRP_DXR_trunk:
rerun: on-new-revision
- path: .yamato/hdrp_dxr-win-dx12.yml#HDRP_DXR_Win_DX12_playmode_XR_mono_Linear_trunk
rerun: on-new-revision
variables:
UTR_VERSION: "current"
TEST_FILTER: .*
PR_HDRP_DXR_CUSTOM-REVISION:
name: HDRP_DXR PR Job - CUSTOM-REVISION
dependencies:
Expand All @@ -40,3 +43,5 @@ Nightly_HDRP_DXR_CUSTOM-REVISION:
rerun: always
variables:
CUSTOM_REVISION: custom_revision_not_set
UTR_VERSION: "current"
TEST_FILTER: .*
5 changes: 5 additions & 0 deletions .yamato/all-hdrp_hybrid.yml
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,9 @@ Nightly_HDRP_Hybrid_trunk:
rerun: on-new-revision
- path: .yamato/all-hdrp_hybrid.yml#PR_HDRP_Hybrid_trunk
rerun: on-new-revision
variables:
UTR_VERSION: "current"
TEST_FILTER: .*
PR_HDRP_Hybrid_CUSTOM-REVISION:
name: HDRP_Hybrid PR Job - CUSTOM-REVISION
dependencies:
Expand All @@ -40,3 +43,5 @@ Nightly_HDRP_Hybrid_CUSTOM-REVISION:
rerun: always
variables:
CUSTOM_REVISION: custom_revision_not_set
UTR_VERSION: "current"
TEST_FILTER: .*
5 changes: 5 additions & 0 deletions .yamato/all-shadergraph.yml
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,9 @@ Nightly_ShaderGraph_trunk:
rerun: on-new-revision
- path: .yamato/shadergraph-android-vulkan.yml#ShaderGraph_Android_Vulkan_Standalone_il2cpp_Linear_trunk
rerun: on-new-revision
variables:
UTR_VERSION: "current"
TEST_FILTER: .*
PR_ShaderGraph_CUSTOM-REVISION:
name: ShaderGraph PR Job - CUSTOM-REVISION
dependencies:
Expand Down Expand Up @@ -82,3 +85,5 @@ Nightly_ShaderGraph_CUSTOM-REVISION:
rerun: always
variables:
CUSTOM_REVISION: custom_revision_not_set
UTR_VERSION: "current"
TEST_FILTER: .*
5 changes: 5 additions & 0 deletions .yamato/all-shadergraph_stereo.yml
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,9 @@ Nightly_ShaderGraph_Stereo_trunk:
dependencies:
- path: .yamato/all-shadergraph_stereo.yml#PR_ShaderGraph_Stereo_trunk
rerun: on-new-revision
variables:
UTR_VERSION: "current"
TEST_FILTER: .*
PR_ShaderGraph_Stereo_CUSTOM-REVISION:
name: ShaderGraph_Stereo PR Job - CUSTOM-REVISION
dependencies:
Expand All @@ -40,3 +43,5 @@ Nightly_ShaderGraph_Stereo_CUSTOM-REVISION:
rerun: always
variables:
CUSTOM_REVISION: custom_revision_not_set
UTR_VERSION: "current"
TEST_FILTER: .*
5 changes: 5 additions & 0 deletions .yamato/all-smoke_test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,9 @@ Nightly_Smoke_Test_trunk:
dependencies:
- path: .yamato/all-smoke_test.yml#PR_Smoke_Test_trunk
rerun: on-new-revision
variables:
UTR_VERSION: "current"
TEST_FILTER: .*
PR_Smoke_Test_CUSTOM-REVISION:
name: Smoke_Test PR Job - CUSTOM-REVISION
dependencies:
Expand All @@ -44,3 +47,5 @@ Nightly_Smoke_Test_CUSTOM-REVISION:
rerun: always
variables:
CUSTOM_REVISION: custom_revision_not_set
UTR_VERSION: "current"
TEST_FILTER: .*
5 changes: 5 additions & 0 deletions .yamato/all-universal.yml
Original file line number Diff line number Diff line change
Expand Up @@ -78,6 +78,9 @@ Nightly_Universal_trunk:
rerun: on-new-revision
- path: .yamato/universal-linux-vulkan.yml#Universal_Linux_Vulkan_playmode_mono_Linear_trunk
rerun: on-new-revision
variables:
UTR_VERSION: "current"
TEST_FILTER: .*
PR_Universal_CUSTOM-REVISION:
name: Universal PR Job - CUSTOM-REVISION
dependencies:
Expand Down Expand Up @@ -150,3 +153,5 @@ Nightly_Universal_CUSTOM-REVISION:
rerun: always
variables:
CUSTOM_REVISION: custom_revision_not_set
UTR_VERSION: "current"
TEST_FILTER: .*
5 changes: 5 additions & 0 deletions .yamato/all-universal_hybrid.yml
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,9 @@ Nightly_Universal_Hybrid_trunk:
dependencies:
- path: .yamato/all-universal_hybrid.yml#PR_Universal_Hybrid_trunk
rerun: on-new-revision
variables:
UTR_VERSION: "current"
TEST_FILTER: .*
PR_Universal_Hybrid_CUSTOM-REVISION:
name: Universal_Hybrid PR Job - CUSTOM-REVISION
dependencies:
Expand All @@ -40,3 +43,5 @@ Nightly_Universal_Hybrid_CUSTOM-REVISION:
rerun: always
variables:
CUSTOM_REVISION: custom_revision_not_set
UTR_VERSION: "current"
TEST_FILTER: .*
5 changes: 5 additions & 0 deletions .yamato/all-universal_stereo.yml
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,9 @@ Nightly_Universal_Stereo_trunk:
dependencies:
- path: .yamato/all-universal_stereo.yml#PR_Universal_Stereo_trunk
rerun: on-new-revision
variables:
UTR_VERSION: "current"
TEST_FILTER: .*
PR_Universal_Stereo_CUSTOM-REVISION:
name: Universal_Stereo PR Job - CUSTOM-REVISION
dependencies:
Expand All @@ -40,3 +43,5 @@ Nightly_Universal_Stereo_CUSTOM-REVISION:
rerun: always
variables:
CUSTOM_REVISION: custom_revision_not_set
UTR_VERSION: "current"
TEST_FILTER: .*
5 changes: 5 additions & 0 deletions .yamato/all-urp_performance_boatattack.yml
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,9 @@ Nightly_URP_Performance_BoatAttack_trunk:
dependencies:
- path: .yamato/all-urp_performance_boatattack.yml#PR_URP_Performance_BoatAttack_trunk
rerun: on-new-revision
variables:
UTR_VERSION: "current"
TEST_FILTER: .*
PR_URP_Performance_BoatAttack_CUSTOM-REVISION:
name: URP_Performance_BoatAttack PR Job - CUSTOM-REVISION
dependencies:
Expand Down Expand Up @@ -104,3 +107,5 @@ Nightly_URP_Performance_BoatAttack_CUSTOM-REVISION:
rerun: always
variables:
CUSTOM_REVISION: custom_revision_not_set
UTR_VERSION: "current"
TEST_FILTER: .*
5 changes: 5 additions & 0 deletions .yamato/all-urpupdate.yml
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,9 @@ Nightly_URPUpdate_trunk:
dependencies:
- path: .yamato/all-urpupdate.yml#PR_URPUpdate_trunk
rerun: on-new-revision
variables:
UTR_VERSION: "current"
TEST_FILTER: .*
PR_URPUpdate_CUSTOM-REVISION:
name: URPUpdate PR Job - CUSTOM-REVISION
dependencies:
Expand All @@ -44,3 +47,5 @@ Nightly_URPUpdate_CUSTOM-REVISION:
rerun: always
variables:
CUSTOM_REVISION: custom_revision_not_set
UTR_VERSION: "current"
TEST_FILTER: .*
5 changes: 5 additions & 0 deletions .yamato/all-urpupdate_boatattack.yml
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,9 @@ Nightly_URPUpdate_BoatAttack_trunk:
dependencies:
- path: .yamato/all-urpupdate_boatattack.yml#PR_URPUpdate_BoatAttack_trunk
rerun: on-new-revision
variables:
UTR_VERSION: "current"
TEST_FILTER: .*
PR_URPUpdate_BoatAttack_CUSTOM-REVISION:
name: URPUpdate_BoatAttack PR Job - CUSTOM-REVISION
dependencies:
Expand All @@ -40,3 +43,5 @@ Nightly_URPUpdate_BoatAttack_CUSTOM-REVISION:
rerun: always
variables:
CUSTOM_REVISION: custom_revision_not_set
UTR_VERSION: "current"
TEST_FILTER: .*
5 changes: 5 additions & 0 deletions .yamato/all-urpupdate_top_asset_store.yml
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,9 @@ Nightly_URPUpdate_Top_Asset_Store_trunk:
dependencies:
- path: .yamato/all-urpupdate_top_asset_store.yml#PR_URPUpdate_Top_Asset_Store_trunk
rerun: on-new-revision
variables:
UTR_VERSION: "current"
TEST_FILTER: .*
PR_URPUpdate_Top_Asset_Store_CUSTOM-REVISION:
name: URPUpdate_Top_Asset_Store PR Job - CUSTOM-REVISION
dependencies:
Expand All @@ -40,3 +43,5 @@ Nightly_URPUpdate_Top_Asset_Store_CUSTOM-REVISION:
rerun: always
variables:
CUSTOM_REVISION: custom_revision_not_set
UTR_VERSION: "current"
TEST_FILTER: .*
5 changes: 5 additions & 0 deletions .yamato/all-vfx_hdrp.yml
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,9 @@ Nightly_VFX_HDRP_trunk:
rerun: on-new-revision
- path: .yamato/vfx_hdrp-win-dx12.yml#VFX_HDRP_Win_DX12_playmode_mono_Linear_trunk
rerun: on-new-revision
variables:
UTR_VERSION: "current"
TEST_FILTER: .*
PR_VFX_HDRP_CUSTOM-REVISION:
name: VFX_HDRP PR Job - CUSTOM-REVISION
dependencies:
Expand All @@ -58,3 +61,5 @@ Nightly_VFX_HDRP_CUSTOM-REVISION:
rerun: always
variables:
CUSTOM_REVISION: custom_revision_not_set
UTR_VERSION: "current"
TEST_FILTER: .*
5 changes: 5 additions & 0 deletions .yamato/all-vfx_urp.yml
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,9 @@ Nightly_VFX_URP_trunk:
rerun: on-new-revision
- path: .yamato/vfx_urp-osx-metal.yml#VFX_URP_OSX_Metal_playmode_mono_Linear_trunk
rerun: on-new-revision
variables:
UTR_VERSION: "current"
TEST_FILTER: .*
PR_VFX_URP_CUSTOM-REVISION:
name: VFX_URP PR Job - CUSTOM-REVISION
dependencies:
Expand All @@ -46,3 +49,5 @@ Nightly_VFX_URP_CUSTOM-REVISION:
rerun: always
variables:
CUSTOM_REVISION: custom_revision_not_set
UTR_VERSION: "current"
TEST_FILTER: .*
3 changes: 3 additions & 0 deletions .yamato/config/__shared.metafile
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,7 @@ test_platforms:
- [all]: --suite=playmode
- [all]: --timeout=1200
- [all]: --reruncount=2
- [all]: --zero-tests-are-ok=1

- [Win_DX11, Win_DX12, Win_Vulkan, Win_]: --platform=StandaloneWindows64
- [Linux_OpenGlCore, Linux_Vulkan]: --platform=StandaloneLinux64
Expand Down Expand Up @@ -112,6 +113,7 @@ test_platforms:
- type: playmode
utr_flags:
- [all]: --testfilter=%TEST_FILTER%
- [all]: --zero-tests-are-ok=1
- [all]: --suite=playmode
- [all]: --scripting-backend=<SCRIPTING_BACKEND>
- [all]: --reruncount=2
Expand All @@ -136,6 +138,7 @@ test_platforms:
- type: editmode
utr_flags:
- [all]: --testfilter=%TEST_FILTER%
- [all]: --zero-tests-are-ok=1
- [all]: --suite=editor
- [all]: --reruncount=2
- [all]: --scripting-backend=<SCRIPTING_BACKEND>
Expand Down
12 changes: 6 additions & 6 deletions .yamato/hdrp-linux-vulkan.yml
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ HDRP_Linux_Vulkan_playmode_mono_Linear_trunk:
- curl -s https://artifactory.internal.unity3d.com/core-automation/tools/utr-standalone/utr --output TestProjects/HDRP_Tests/utr
- chmod +x TestProjects/HDRP_Tests/utr
- cd TestProjects/HDRP_Tests && sudo unity-downloader-cli -u {{editor_versions.trunk_latest_internal.linux.revision}} -c editor -c il2cpp --wait --published-only
- cd TestProjects/HDRP_Tests && DISPLAY=:0.0 ./utr --artifacts_path=test-results --editor-location=.Editor --extra-editor-arg="-colorspace=Linear" --extra-editor-arg="-force-vulkan" --reruncount=2 --scripting-backend=Mono2x --suite=playmode --testfilter=$TEST_FILTER --testproject=.
- cd TestProjects/HDRP_Tests && DISPLAY=:0.0 ./utr --artifacts_path=test-results --editor-location=.Editor --extra-editor-arg="-colorspace=Linear" --extra-editor-arg="-force-vulkan" --reruncount=2 --scripting-backend=Mono2x --suite=playmode --testfilter=$TEST_FILTER --testproject=. --zero-tests-are-ok=1
artifacts:
logs:
paths:
Expand All @@ -45,7 +45,7 @@ HDRP_Linux_Vulkan_editmode_mono_Linear_trunk:
- curl -s https://artifactory.internal.unity3d.com/core-automation/tools/utr-standalone/utr --output TestProjects/HDRP_Tests/utr
- chmod +x TestProjects/HDRP_Tests/utr
- cd TestProjects/HDRP_Tests && sudo unity-downloader-cli -u {{editor_versions.trunk_latest_internal.linux.revision}} -c editor -c il2cpp --wait --published-only
- cd TestProjects/HDRP_Tests && DISPLAY=:0.0 ./utr --artifacts_path=test-results --editor-location=.Editor --extra-editor-arg="-colorspace=Linear" --extra-editor-arg="-force-vulkan" --extra-editor-arg="-playergraphicsapi=Vulkan" --platform=editmode --reruncount=2 --scripting-backend=Mono2x --suite=editor --testfilter=$TEST_FILTER --testproject=.
- cd TestProjects/HDRP_Tests && DISPLAY=:0.0 ./utr --artifacts_path=test-results --editor-location=.Editor --extra-editor-arg="-colorspace=Linear" --extra-editor-arg="-force-vulkan" --extra-editor-arg="-playergraphicsapi=Vulkan" --platform=editmode --reruncount=2 --scripting-backend=Mono2x --suite=editor --testfilter=$TEST_FILTER --testproject=. --zero-tests-are-ok=1
artifacts:
logs:
paths:
Expand All @@ -64,7 +64,7 @@ HDRP_Linux_Vulkan_Standalone_mono_Linear_trunk:
commands:
- curl -s https://artifactory.internal.unity3d.com/core-automation/tools/utr-standalone/utr --output TestProjects/HDRP_RuntimeTests/utr
- chmod +x TestProjects/HDRP_RuntimeTests/utr
- cd TestProjects/HDRP_RuntimeTests && ./utr --artifacts_path=test-results --platform=StandaloneLinux64 --player-connection-ip=auto --player-load-path=../../players --reruncount=2 --suite=playmode --timeout=1200
- cd TestProjects/HDRP_RuntimeTests && ./utr --artifacts_path=test-results --platform=StandaloneLinux64 --player-connection-ip=auto --player-load-path=../../players --reruncount=2 --suite=playmode --timeout=1200 --zero-tests-are-ok=1
artifacts:
logs:
paths:
Expand Down Expand Up @@ -115,7 +115,7 @@ HDRP_Linux_Vulkan_playmode_mono_Linear_CUSTOM-REVISION:
- curl -s https://artifactory.internal.unity3d.com/core-automation/tools/utr-standalone/utr --output TestProjects/HDRP_Tests/utr
- chmod +x TestProjects/HDRP_Tests/utr
- cd TestProjects/HDRP_Tests && sudo unity-downloader-cli --source-file ../../unity_revision.txt -c editor -c il2cpp --wait --published-only
- cd TestProjects/HDRP_Tests && DISPLAY=:0.0 ./utr --artifacts_path=test-results --editor-location=.Editor --extra-editor-arg="-colorspace=Linear" --extra-editor-arg="-force-vulkan" --reruncount=2 --scripting-backend=Mono2x --suite=playmode --testfilter=$TEST_FILTER --testproject=.
- cd TestProjects/HDRP_Tests && DISPLAY=:0.0 ./utr --artifacts_path=test-results --editor-location=.Editor --extra-editor-arg="-colorspace=Linear" --extra-editor-arg="-force-vulkan" --reruncount=2 --scripting-backend=Mono2x --suite=playmode --testfilter=$TEST_FILTER --testproject=. --zero-tests-are-ok=1
artifacts:
logs:
paths:
Expand All @@ -141,7 +141,7 @@ HDRP_Linux_Vulkan_editmode_mono_Linear_CUSTOM-REVISION:
- curl -s https://artifactory.internal.unity3d.com/core-automation/tools/utr-standalone/utr --output TestProjects/HDRP_Tests/utr
- chmod +x TestProjects/HDRP_Tests/utr
- cd TestProjects/HDRP_Tests && sudo unity-downloader-cli --source-file ../../unity_revision.txt -c editor -c il2cpp --wait --published-only
- cd TestProjects/HDRP_Tests && DISPLAY=:0.0 ./utr --artifacts_path=test-results --editor-location=.Editor --extra-editor-arg="-colorspace=Linear" --extra-editor-arg="-force-vulkan" --extra-editor-arg="-playergraphicsapi=Vulkan" --platform=editmode --reruncount=2 --scripting-backend=Mono2x --suite=editor --testfilter=$TEST_FILTER --testproject=.
- cd TestProjects/HDRP_Tests && DISPLAY=:0.0 ./utr --artifacts_path=test-results --editor-location=.Editor --extra-editor-arg="-colorspace=Linear" --extra-editor-arg="-force-vulkan" --extra-editor-arg="-playergraphicsapi=Vulkan" --platform=editmode --reruncount=2 --scripting-backend=Mono2x --suite=editor --testfilter=$TEST_FILTER --testproject=. --zero-tests-are-ok=1
artifacts:
logs:
paths:
Expand All @@ -164,7 +164,7 @@ HDRP_Linux_Vulkan_Standalone_mono_Linear_CUSTOM-REVISION:
commands:
- curl -s https://artifactory.internal.unity3d.com/core-automation/tools/utr-standalone/utr --output TestProjects/HDRP_RuntimeTests/utr
- chmod +x TestProjects/HDRP_RuntimeTests/utr
- cd TestProjects/HDRP_RuntimeTests && ./utr --artifacts_path=test-results --platform=StandaloneLinux64 --player-connection-ip=auto --player-load-path=../../players --reruncount=2 --suite=playmode --timeout=1200
- cd TestProjects/HDRP_RuntimeTests && ./utr --artifacts_path=test-results --platform=StandaloneLinux64 --player-connection-ip=auto --player-load-path=../../players --reruncount=2 --suite=playmode --timeout=1200 --zero-tests-are-ok=1
artifacts:
logs:
paths:
Expand Down
Loading