Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

benchmark: add benchmark groups #39285

Open
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

Mesteery
Copy link
Contributor

@Mesteery Mesteery commented Jul 6, 2021

Add the byGroup option to have benchmark parameter grouping.
The environment variable NODE_RUN_BENCHMARK_GROUPS allows to benchmark
only certain groups: NODE_RUN_BENCHMARK_GROUPS=groupA,groupB.

Fixes: #26425

Add the `byGroup` option to have benchmark parameter grouping.
The environment variable `NODE_RUN_BENCHMARK_GROUPS` allows to benchmark
only certain groups: `NODE_RUN_BENCHMARK_GROUPS=groupA,groupB`.

Fixes: nodejs#26425
@github-actions github-actions bot added benchmark Issues and PRs related to the benchmark subsystem. doc Issues and PRs related to the documentations. labels Jul 6, 2021
benchmark/common.js Outdated Show resolved Hide resolved
Mesteery and others added 2 commits July 6, 2021 13:56
Co-authored-by: mscdex <mscdex@users.noreply.github.com>
@mscdex
Copy link
Contributor

mscdex commented Jul 6, 2021

I think we should be adding something to the output (preferably in a "backwards-compatible" fashion) to indicate which group a particular configuration result is for, in order to remedy the issue noted in a previous PR that attempted to add similar functionality.

@Mesteery
Copy link
Contributor Author

Mesteery commented Jul 6, 2021

I think we should be adding something to the output (preferably in a "backwards-compatible" fashion) to indicate which group a particular configuration result is for, in order to remedy the issue noted in a previous PR that attempted to add similar functionality.

That's what this line does:

config.group = key;

buffers\buffer-from-by-group.js group="groupA" n=2048 len=10 source="array": 2,591.525584798374
buffers\buffer-from-by-group.js group="groupA" n=2048 len=2048 source="array": 114.49576417665251
buffers\buffer-from-by-group.js group="groupA" n=2048 len=10 source="arraybuffer": 6,035.5383579386635
buffers\buffer-from-by-group.js group="groupA" n=2048 len=2048 source="arraybuffer": 7,932.364200868532
buffers\buffer-from-by-group.js group="groupA" n=2048 len=10 source="arraybuffer-middle": 7,275.558953342637
buffers\buffer-from-by-group.js group="groupA" n=2048 len=2048 source="arraybuffer-middle": 7,601.415615193181
buffers\buffer-from-by-group.js group="groupA" n=2048 len=10 source="buffer": 4,794.999714360369
buffers\buffer-from-by-group.js group="groupA" n=2048 len=2048 source="buffer": 285.891492173511
buffers\buffer-from-by-group.js group="groupA" n=2048 len=10 source="uint8array": 4,773.4621608726675
buffers\buffer-from-by-group.js group="groupA" n=2048 len=2048 source="uint8array": 288.5734902132302
buffers\buffer-from-by-group.js group="groupA" n=2048 len=10 source="string": 1,127.4664705344785
buffers\buffer-from-by-group.js group="groupA" n=2048 len=2048 source="string": 150.8948858783378
buffers\buffer-from-by-group.js group="groupA" n=2048 len=10 source="string-utf8": 969.7732383214418
buffers\buffer-from-by-group.js group="groupA" n=2048 len=2048 source="string-utf8": 152.72480591794306
buffers\buffer-from-by-group.js group="groupA" n=2048 len=10 source="string-base64": 811.0613882780957
buffers\buffer-from-by-group.js group="groupA" n=2048 len=2048 source="string-base64": 93.07144996598143
buffers\buffer-from-by-group.js group="groupA" n=2048 len=10 source="object": 4,568.509008640237
buffers\buffer-from-by-group.js group="groupA" n=2048 len=2048 source="object": 2,715.449648894217
buffers\buffer-from-by-group.js group="groupB" n=2048 len=2048 source="buffer": 221.14045045726675
buffers\buffer-from-by-group.js group="groupB" n=2048 len=2048 source="string": 140.29462281811206

@targos
Copy link
Member

targos commented Jul 7, 2021

@nodejs/benchmarking

option for specifying command line flags is supported.
* `options` {Object} The benchmark options:
* `flags` option for specifying command line flags
* `byGroup` option for processing `configs` by groups:
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚙️ (optional) Perhaps it's better to use the plural for consistency:

Suggested change
* `byGroup` option for processing `configs` by groups:
* `byGroups` option for processing `configs` by groups:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
benchmark Issues and PRs related to the benchmark subsystem. doc Issues and PRs related to the documentations.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Benchmark parameter group
5 participants