cmd/go: docs are unclear that tests are run in parallel across packages

[joeshaw]

If you provide go test with more than one package, tests across packages are run in parallel by default (assuming you have more than one CPU). If you have tests across packages that contend for a singular resource (for example, listening on a particular TCP port) you can end up getting unexpected failures.

The docs aren't very clear in this respect and I think some improvements can be made to them. This has bit me in the past and just came up again in the Gophers Slack community. In these cases we developers had to go to the go command's source code to understand what was happening.

In my view, some of the issues:

• go test --help is equivalent to go help testflag rather than go help test. I tend to add --help by default and so I was seeing the wrong (IMO) help output.
• go help testflag mentions parallelism with its -parallel flag, but this only affects tests within a single package. The presence of this flag implies that tests that don't set t.Parallel will never be run in parallel, regardless of where they live.
• go help testflag does not mention go help build at all, and users might not be aware that all the flags there also apply to go test, in particular the -p flag. (go help test does mention build flags, however, which is good.)
• go help build mentions the -p flag, but it only talks about builds. It does not imply that the tests will also be run in parallel. I think most people assume that build and test are separate steps. I think it could also be cleaner that it's talking about packages specifically there.

The core issue is that there are multiple levels of parallelism at play, and only one is mentioned in the test help.

[marcosnils]

+1 we where speaking about this with @joeshaw and @ardan-bkennedy and we realized that test docs should be improved as all of us had different perspectives about how they work (specifically parallelism)

[ardan-bkennedy]

+1 more and better documentation would certainly help.

[aleksclark]

I'd just like to +1 and make something explicitly, painfully clear, since I didn't grok this the first time reading the OP, and had a long day as a result: -parallel controls parallelism if you're using t.Parallel. -p controls parallelism for go test and go build, so if you have tests that need to run serially across packages, you must set -p 1 or they will be run in parallel, e.g. you have tests that operate on a common db and they can step on each other.

[gopherbot]

CL https://golang.org/cl/17946 mentions this issue.