Skip to content
This repository has been archived by the owner on Jul 25, 2022. It is now read-only.

add -v -progress to show detailed information for each test case #321

Merged

Conversation

neo-liang-sap
Copy link
Contributor

What this PR does / why we need it:
from discussion in #32 (comment) someone is complaining current test framework hard to show which test case is failing, i'm referring to onsi/ginkgo#155 and added -v -progress flag to ginko test framework.
now each test case showing detailed information like:

------------------------------
Shell command with non-existing node name
  should return error
  /Users/i352986/git_repo/gardenctl/pkg/cmd/shell_test.go:133
[BeforeEach] Shell command
  /Users/i352986/git_repo/gardenctl/pkg/cmd/shell_test.go:84
[It] should return error
  /Users/i352986/git_repo/gardenctl/pkg/cmd/shell_test.go:133
[AfterEach] Shell command
  /Users/i352986/git_repo/gardenctl/pkg/cmd/shell_test.go:90
•Error: nodes "minikube" not found

Which issue(s) this PR fixes:
Fixes #

Special notes for your reviewer:

Release note:

add -v -progress to show detailed information for each test case

@neo-liang-sap neo-liang-sap requested a review from a team as a code owner September 22, 2020 06:04
@gardener-robot-ci-1 gardener-robot-ci-1 added reviewed/ok-to-test Has approval for testing (check PR in detail before setting this label because PR is run on CI/CD) needs/ok-to-test Needs approval for testing (check PR in detail before setting this label because PR is run on CI/CD) and removed reviewed/ok-to-test Has approval for testing (check PR in detail before setting this label because PR is run on CI/CD) labels Sep 22, 2020
@tedteng
Copy link
Contributor

tedteng commented Sep 22, 2020

emm.. My idea is more about identify failure test case easily. Is it possible to just print the test name as boundary in side each test case beginning and ending. which looks like below instead of breaking the current display structure

Running Suite: Cmd Suite
========================
Random Seed: 1600755393
Will run 53 of 53 specs

=== target test start===
••Error: project targeted
•Error: nodes "minikube" not found
•Error: project targeted
=== target test end===
=== get test start===
•Error: command must be in the format: gardenctl shell (node|pod)
•Error: shoot "test-shoot" is hibernated
•Error: target stack is empty
=== get test end===
......

if there a failure test which will be look like this

Running Suite: Cmd Suite
========================
Random Seed: 1600656329
Will run 53 of 53 specs
=== ls test ===
•Error: no garden cluster targeted
••Error: no shoot targeted
open : no such file or directory
=== ls test ===
Ginkgo ran 1 suite in 4.074176668s
Test Suite Failed

Its up to you the key ward === target test start=== or === target test === or others. just use as mark to identify the test case easily when failure test occured

@neo-liang-sap
Copy link
Contributor Author

neo-liang-sap commented Sep 22, 2020

i believe you didn't try this PR before commenting out, the real thing looks like this
image
i think it's very clear on what you are talking about print the test name as boundary
for what you desire === target test start=== i didn't see feasible support within ginko framework, off course you can achieve this by adding fmt.Println() in each test case but i don't consider that's an nice implementation

from your description still you can't figure which test case is failing, say we have three test cases in target test but you can't find which test cases is failing, you can't see test case name but you just see return message

=== target test start===
••Error: project targeted
•Error: nodes "minikube" not found
•Error: project targeted
=== target test end===

my implementation has test case name / each phase / file line and is based on ginko native framework, i'm not saying mine implementation is best but at least for me it's much more clear compared with what we have before

@neo-liang-sap neo-liang-sap merged commit ab788fd into gardener-attic:master Sep 23, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
kind/test Test needs/ok-to-test Needs approval for testing (check PR in detail before setting this label because PR is run on CI/CD)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants