Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

test: test explorer UI does not recognize FuzzTest #2023

Closed
hyangah opened this issue Jan 21, 2022 · 8 comments
Closed

test: test explorer UI does not recognize FuzzTest #2023

hyangah opened this issue Jan 21, 2022 · 8 comments
Assignees
@hyangah
Labels
FrozenDueToAge go-test issues related to go test support (test output, test explorer, ...)
Milestone
v0.31.1

Comments

@hyangah
Copy link
Contributor

hyangah commented Jan 21, 2022

run test codelens appears but it's not recognized in the Testing UI.

Screen Shot 2022-01-21 at 2 23 36 PM

We also need to check how Testing UI handles the fuzz test case that looks like dynamically found subtest.

Running tool: /Users/hakim/sdk/gotip/bin/go test -timeout 30s -run ^FuzzReverse$ work

--- FAIL: FuzzReverse (0.00s)
    --- FAIL: FuzzReverse/69b6bf02dbaa368582724dd438c1ca203a082a6f7648e47766b7251d68dbde57 (0.00s)
        /Users/hakim/ww/reverse_test.go:36: Reverse produced invalid UTF-8 string "\x97\xcf"
FAIL
FAIL	work	0.184s
FAIL
@hyangah hyangah added the go-test issues related to go test support (test output, test explorer, ...) label Jan 21, 2022
@hyangah hyangah added this to the v0.32.0 milestone Jan 21, 2022
@hyangah hyangah self-assigned this Jan 24, 2022
@gopherbot
Copy link
Collaborator

Change https://golang.org/cl/380501 mentions this issue: src/goTest: remove '#' in subtest name parsing cutset

@gopherbot
Copy link
Collaborator

Change https://golang.org/cl/380500 mentions this issue: src/goTest: detect Fuzz test and adjust subtest name parsing

@gopherbot
Copy link
Collaborator

Change https://golang.org/cl/380500 mentions this issue: src/goTest: detect Fuzz tests and list them in test explorer UI

gopherbot pushed a commit that referenced this issue Jan 26, 2022
@hyangah
src/goTest: detect Fuzz tests and list them in test explorer UI
c42b3cd 
Detected fuzz functions are classified as a 'fuzz' type, so
we can later add special handling for fuzzing functionality.
Now they are like other test functions.

For #2023

Change-Id: I7f3998ec6acf8a766c05380077d41c459d685079
Reviewed-on: https://go-review.googlesource.com/c/vscode-go/+/380500
Trust: Hyang-Ah Hana Kim <hyangah@gmail.com>
Run-TryBot: Hyang-Ah Hana Kim <hyangah@gmail.com>
TryBot-Result: kokoro <noreply+kokoro@google.com>
Reviewed-by: Suzy Mueller <suzmue@golang.org>
@hyangah hyangah added the go1.18 label Jan 26, 2022
@hyangah hyangah modified the milestones: v0.32.0, v0.31.1 Feb 4, 2022
@gopherbot
Copy link
Collaborator

Change https://golang.org/cl/383218 mentions this issue: [release] src/goTest: detect Fuzz tests and list them in test explorer UI

@gopherbot
Copy link
Collaborator

Change https://go.dev/cl/380500 mentions this issue: src/goTest: detect Fuzz tests and list them in test explorer UI

@gopherbot
Copy link
Collaborator

Change https://go.dev/cl/383218 mentions this issue: [release] src/goTest: detect Fuzz tests and list them in test explorer UI

@gopherbot
Copy link
Collaborator

Change https://go.dev/cl/380501 mentions this issue: src/goTest: remove '#' in subtest name parsing cutset

@gopherbot
Copy link
Collaborator

Change https://go.dev/cl/383935 mentions this issue: [release] src/goTest: remove '#' in subtest name parsing cutset

gopherbot pushed a commit that referenced this issue Feb 8, 2022
@hyangah
[release] src/goTest: detect Fuzz tests and list them in test explore…
0e33c84 
…r UI

Detected fuzz functions are classified as a 'fuzz' type, so
we can later add special handling for fuzzing functionality.
Now they are like other test functions.

For #2023

Change-Id: I7f3998ec6acf8a766c05380077d41c459d685079
Reviewed-on: https://go-review.googlesource.com/c/vscode-go/+/380500
Trust: Hyang-Ah Hana Kim <hyangah@gmail.com>
Run-TryBot: Hyang-Ah Hana Kim <hyangah@gmail.com>
TryBot-Result: kokoro <noreply+kokoro@google.com>
Reviewed-by: Suzy Mueller <suzmue@golang.org>
(cherry picked from commit c42b3cd)
Reviewed-on: https://go-review.googlesource.com/c/vscode-go/+/383218
Reviewed-by: Robert Findley <rfindley@google.com>
gopherbot pushed a commit that referenced this issue Feb 8, 2022
@hyangah
[release] src/goTest: remove '#' in subtest name parsing cutset
c845547 
Subtests are detected dynamically - while running tests.
GoTestRunner heuristically builds subtest trees by parsing
the test names observed while running tests.

Before this CL, it used both '/' and '#' as the separator for
parsing.  For example, it placed a test item for Foo as a
child of TestXxx when it observed TestXxx/Foo, and a test item
01 as a child of TestXxx if it saw TestXxx#01.

However, go test uses '#' to resolve conflicts in subtest names,
so it's not right to assume TestXXX#01 is a subtest of TestXXX.

Moreover, treating TestXXX/subtest#01 or TestXXX/subtest#02 as
a subtest of TestXXX/subtest confuses the text UI API because
`go test -json` will generate event sequence like

TestXXX run
 TestXXX/subtest run
 TestXXX/subtest#01 run
 TestXXX/subtest#02 run
 TestXXX/subtest pass
 TestXXX/subtest#01 pass
 TestXXX/subtest#02 pass
TestXXX pass

That causes the test UI to show only the last item for TestXXX/subtest#02
to appear in TestXXX/subtest's child.

This bug also makes Fuzz test's crasher and seed case listing confusing.

This CL removes '#' from the cutset.

Fixes #2023
For #1922

Change-Id: I33ba5c17e9095686a87c719d44fe7330269d9cc3
Reviewed-on: https://go-review.googlesource.com/c/vscode-go/+/380501
Trust: Hyang-Ah Hana Kim <hyangah@gmail.com>
Run-TryBot: Hyang-Ah Hana Kim <hyangah@gmail.com>
TryBot-Result: kokoro <noreply+kokoro@google.com>
Reviewed-by: Suzy Mueller <suzmue@golang.org>
(cherry picked from commit 2735f15)
Reviewed-on: https://go-review.googlesource.com/c/vscode-go/+/383935
Reviewed-by: Robert Findley <rfindley@google.com>
@golang golang locked and limited conversation to collaborators Feb 8, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees

@hyangah hyangah

Labels
FrozenDueToAge go-test issues related to go test support (test output, test explorer, ...)
Projects
None yet
Milestone
v0.31.1
Development

No branches or pull requests
2 participants
@hyangah @gopherbot