Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Report functional test pass only when all subtests finish #7032

Conversation

willdavsmith
Copy link
Contributor

Description

  • Change functional test logic to only report complete when all subtests (shared, samples, msgrp, etc.) are complete.

Type of change

  • This pull request fixes a bug in Radius and has an approved issue (issue link required).

Fixes: #6939

@radius-functional-tests
Copy link

radius-functional-tests bot commented Jan 24, 2024

Radius functional test overview

🔍 Go to test action run

Name Value
Repository willdavsmith/radius
Commit ref 6b95370
Unique ID 87477f9a20
Image tag pr-87477f9a20
Click here to see the list of tools in the current test run
  • gotestsum 1.10.0
  • KinD: v0.20.0
  • Dapr: 1.12.0
  • Azure KeyVault CSI driver: 1.4.2
  • Azure Workload identity webhook: 1.1.0
  • Bicep recipe location ghcr.io/radius-project/dev/test/functional/shared/recipes/<name>:pr-87477f9a20
  • Terraform recipe location http://tf-module-server.radius-test-tf-module-server.svc.cluster.local/<name>.zip (in cluster)
  • applications-rp test image location: ghcr.io/radius-project/dev/applications-rp:pr-87477f9a20
  • controller test image location: ghcr.io/radius-project/dev/controller:pr-87477f9a20
  • ucp test image location: ghcr.io/radius-project/dev/ucpd:pr-87477f9a20
  • deployment-engine test image location: ghcr.io/radius-project/deployment-engine:latest

Test Status

⌛ Building Radius and pushing container images for functional tests...
✅ Container images build succeeded
⌛ Publishing Bicep Recipes for functional tests...
✅ Recipe publishing succeeded
⌛ Starting shared functional tests...
⌛ Starting msgrp functional tests...
⌛ Starting daprrp functional tests...
⌛ Starting datastoresrp functional tests...
⌛ Starting samples functional tests...
⌛ Starting kubernetes functional tests...
✅ msgrp functional tests succeeded
✅ samples functional tests succeeded
✅ kubernetes functional tests succeeded
✅ ucp functional tests succeeded
✅ daprrp functional tests succeeded
✅ datastoresrp functional tests succeeded
✅ shared functional tests succeeded

@@ -614,7 +640,7 @@ jobs:
repo: ${{ github.repository }}
sha: ${{ env.CHECKOUT_REF }}
status: completed
conclusion: ${{ job.status }}
conclusion: ${{ needs.tests.outputs.test_status }}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe a dumb question about this about this ... does this work?

Since tests is a matrix are we going to end up with the same issue because each entry in the matrix will set test_status?

Would using needs.tests.status do the same thing? https://github.com/orgs/community/discussions/26822#discussioncomment-3253537

@willdavsmith willdavsmith force-pushed the willdavsmith/functional-test-pass branch from 3586d33 to 6b95370 Compare March 4, 2024 18:40
Signed-off-by: willdavsmith <willdavsmith@gmail.com>
@willdavsmith willdavsmith force-pushed the willdavsmith/functional-test-pass branch 2 times, most recently from 360dbd1 to c25ae26 Compare March 4, 2024 18:46
Signed-off-by: willdavsmith <willdavsmith@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Functional test checks don't work correctly
2 participants