Skip to content

Commit 1dd567d

Browse files
committed
14313-benchmark-format: minor refreshes
This updates some links and some stale information in this design doc. This is slightly odd because we use this design doc also as a specification, which creates some tension between whether it should continue to capture the state of the world at the time or be kept fully up-to-date. I tried to take a light touch in this CL. Change-Id: Ifa3cf7d70e0f6a65cd9184f76f449d5618211f63 Reviewed-on: https://go-review.googlesource.com/c/proposal/+/357529 Reviewed-by: Michael Knyszek <mknyszek@google.com>
1 parent 0ecc9c7 commit 1dd567d

File tree

1 file changed

+12
-10
lines changed

1 file changed

+12
-10
lines changed

design/14313-benchmark-format.md

Lines changed: 12 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -56,31 +56,33 @@ For example, running compress/flate's benchmarks produces this output:
5656
BenchmarkEncodeDigitsCompress1e5-8 1 12301511 ns/op 8.13 MB/s
5757
BenchmarkEncodeDigitsCompress1e6-8 1 137962041 ns/op 7.25 MB/s
5858

59-
The testing package always reports ns/op, and each benchmark can request the addition of MB/s (throughput) and also B/op and allocs/op (allocation rates).
59+
The testing package always reports ns/op, and directly supports the
60+
addition of MB/s (throughput) and also B/op and allocs/op (allocation
61+
rates).
62+
Benchmarks can report additional metrics with any custom unit using
63+
[`B.ReportMetric`](https://pkg.go.dev/testing#B.ReportMetric).
6064

6165
### Benchmark processors
6266

6367
Multiple tools have been written that process this format,
64-
most notably [benchcmp](https://godoc.org/golang.org/x/tools/cmd/benchcmp)
65-
and its more statistically valid successor [benchstat](https://godoc.org/rsc.io/benchstat).
66-
There is also [benchmany](https://godoc.org/github.com/aclements/go-misc/benchmany)'s plot subcommand
68+
most notably [benchcmp](https://pkg.go.dev/golang.org/x/tools/cmd/benchcmp)
69+
and its more statistically valid successor [benchstat](https://pkg.go.dev/golang.org/x/perf/cmd/benchstat).
70+
There is also [benchmany](https://pkg.go.dev/github.com/aclements/go-misc/benchmany)'s plot subcommand
6771
and likely more unpublished programs.
6872

6973
### Benchmark runners
7074

7175
Multiple tools have also been written that generate this format.
7276
In addition to the standard Go testing package,
73-
[compilebench](https://godoc.org/rsc.io/compilebench)
77+
[compilebench](https://pkg.go.dev/golang.org/x/tools/cmd/compilebench)
7478
generates this data format based on runs of the Go compiler,
7579
and Austin's unpublished shellbench generates this data format
7680
after running an arbitrary shell command.
7781

78-
The [golang.org/x/benchmarks/bench](https://golang.org/x/benchmarks/bench) benchmarks
79-
are notable for _not_ generating this format,
80-
which has made all analysis of those results
82+
The [golang.org/x/benchmarks](https://golang.org/x/benchmarks) benchmarks
83+
are notable for _not_ originally generating this format,
84+
which made all analysis of those results
8185
more complex than we believe it should be.
82-
We intend to update those benchmarks to generate the standard format,
83-
once a standard format is defined.
8486
Part of the motivation for the proposal is to avoid
8587
the need to process custom output formats in future benchmarks.
8688

0 commit comments

Comments
 (0)