Skip to content

Commit 17650e1

Browse files
chore(NODE-6921): perf tests default to cwd and errexit (#791)
Co-authored-by: Bailey Pearson <bailey.pearson@mongodb.com>
1 parent 7f2a6d3 commit 17650e1

File tree

6 files changed

+25
-16
lines changed

6 files changed

+25
-16
lines changed

.evergreen/run-granular-benchmarks.sh

+6-4
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,10 @@
11
#!/usr/bin/env bash
22

33
source $DRIVERS_TOOLS/.evergreen/init-node-and-npm-env.sh
4-
set -o xtrace
5-
WARMUP=$WARMUP
6-
ITERATIONS=$ITERATIONS
74

8-
WARMUP=$WARMUP ITERATIONS=$ITERATIONS LIBRARY=$(pwd) npm run check:granular-bench
5+
set -euxo pipefail
6+
7+
export WARMUP=$WARMUP
8+
export ITERATIONS=$ITERATIONS
9+
10+
npm run check:granular-bench

.evergreen/run-spec-benchmarks.sh

+2
Original file line numberDiff line numberDiff line change
@@ -2,4 +2,6 @@
22

33
source $DRIVERS_TOOLS/.evergreen/init-node-and-npm-env.sh
44

5+
set -euxo pipefail
6+
57
npm run check:spec-bench

CONTRIBUTING.md

+12-10
Original file line numberDiff line numberDiff line change
@@ -83,15 +83,16 @@ WARMUP=<warmup iterations> ITERATIONS=<measured iterations> npm run check:granul
8383
This will build the granular tests and run them with `test/bench/etc/run_granular_benchmarks.js`. The `WARMUP` and `ITERATIONS` environment variables can be optionally provided to configure how these granular benchmarks
8484
are run. `WARMUP` changes the number of iterations run before results are collected to give v8's
8585
optimizing compiler time to reach steady state. `ITERATIONS` changes the number of iterations that
86-
are measured and used to calculate summary statistics. Note also that the test can be configured to
87-
make use of the local copy of bson when testing performance changes locally by setting the `LIBRARY`
88-
variable to the root directory of the bson library to be tested.
86+
are measured and used to calculate summary statistics.
8987

9088
```bash
91-
WARMUP=100 ITERATIONS=1000 LIBRARY=$(pwd) npm run check:granular-bench
89+
WARMUP=100 ITERATIONS=1000 npm run check:granular-bench
9290
```
93-
When the `LIBRARY` environment variable is unset, the benchmark clones and runs against the main
94-
branch of this repository.
91+
92+
When the `LIBRARY` environment variable is unset, the benchmark clones and runs against the current working directory.
93+
94+
> [!NOTE]
95+
> The test can be configured to make use of a different copy of bson when testing performance changes locally by setting the `LIBRARY` variable to the root directory of a copy of the bson library.
9596
9697
When the script is complete, results will be output to `test/bench/etc/resultsCollectedMeans.json`. These results will
9798
be in a format compatible with evergreen's perf.send command. To convert these results to CSV, run
@@ -109,14 +110,15 @@ npm run check:spec-bench
109110

110111
This will run the spec benchmarks in `test/bench/spec/bsonBench.ts` which also makes use of the
111112
`bson-bench` library. Results will be written to `bsonBench`. The warmup and iterations are not
112-
configurable as these are determined by the common driver benchmarking specification, but similar
113-
to the granular benchmarks, the spec benchmarks can be run against the local copy of bson by setting
114-
the `LIBRARY` environment variable appropriately.
113+
configurable as these are determined by the common driver benchmarking specification.
115114

116115
```bash
117-
LIBRARY=$(pwd) npm run check:spec-bench
116+
npm run check:spec-bench
118117
```
119118

119+
> [!NOTE]
120+
> The test can be configured to make use of a different copy of bson when testing performance changes locally by setting the `LIBRARY` variable to the root directory of a copy of the bson library.
121+
120122
### Commit messages
121123

122124
Please follow the [Conventional Commits specification][conventional-commit-style].

test/bench/etc/run_granular_benchmarks.js

+1-1
Original file line numberDiff line numberDiff line change
@@ -66,7 +66,7 @@ const DOCUMENT_ROOT = path.resolve(`${__dirname}/../documents`);
6666
}
6767
}
6868

69-
console.log('No duplcate testName:Option pairs found. Now merging files...');
69+
console.log('No duplicate testName:Option pairs found. Now merging files...');
7070

7171
const meansFile = `${__dirname}/resultsCollectedMeans.json`;
7272
// Iterate over all result files and merge into one file

test/bench/granular/common.ts

+2-1
Original file line numberDiff line numberDiff line change
@@ -61,6 +61,7 @@ export function getMixedTestTags(documentPath: string) {
6161
export async function runSuiteAndWriteResults(suite: Suite) {
6262
const targetDirectory = path.resolve(`${__dirname}/../../etc`);
6363
await suite.run();
64+
if (suite.errors.length) throw suite.errors[0].error;
6465
await suite.writeResults(`${targetDirectory}/${suite.name.toLowerCase()}Results.json`);
6566
}
6667

@@ -71,7 +72,7 @@ export function readEnvVars(): { warmup: number; iterations: number; library: st
7172
const rv = {
7273
warmup: Number.isSafeInteger(envWarmup) && envWarmup > 0 ? envWarmup : 100_000,
7374
iterations: Number.isSafeInteger(envIterations) && envIterations > 0 ? envIterations : 10_000,
74-
library: libraryPath ? `bson:${libraryPath}` : 'bson#main'
75+
library: libraryPath ? `bson:${libraryPath}` : `bson:${process.cwd()}`
7576
};
7677

7778
console.log(

test/bench/spec/bsonBench.ts

+2
Original file line numberDiff line numberDiff line change
@@ -78,6 +78,8 @@ suite.task({
7878
});
7979

8080
suite.run().then(async () => {
81+
if (suite.errors.length) throw suite.errors[0].error;
82+
8183
const cpuBaseline = await readFile(
8284
join(__dirname, '..', '..', 'etc', 'cpuBaseline.json'),
8385
'utf8'

0 commit comments

Comments
 (0)