Skip to content

Conversation

@0ax1
Copy link
Contributor

@0ax1 0ax1 commented Nov 24, 2025

Inputs should be passed with_inputs to black box input args and benchmarked with bench_refs to exclude drop.

@codspeed-hq
Copy link

codspeed-hq bot commented Nov 24, 2025

CodSpeed Performance Report

Merging #5488 will degrade performances by 23.04%

Comparing ad/bench-refs-blackbox-inputs (e0822e4) with develop (a917a39)

Summary

⚡ 9 improvements
❌ 6 regressions
✅ 1507 untouched
⏩ 84 skipped1

⚠️ Please fix the performance issues or acknowledge them on CodSpeed.

Benchmarks breakdown

Benchmark BASE HEAD Change
bitpack_canonical_unpack[(1000, 1.0)] 13.7 µs 11.9 µs +15.55%
batch[100352] 773 µs 684.1 µs +13%
batch[262144] 1.8 ms 2 ms -11.47%
batch[8192] 64.7 µs 57.2 µs +13.1%
batch[86016] 586.8 µs 662.5 µs -11.43%
in_place_pipeline[1024] 3.8 µs 4.9 µs -23.04%
in_place_pipeline[262144] 397.4 µs 331 µs +20.05%
in_place_pipeline[86016] 111 µs 133 µs -16.54%
pipeline[65536] 89.3 µs 106 µs -15.73%
pipeline_extra_copy[100352] 158.7 µs 132.8 µs +19.48%
pipeline_extra_copy[16384] 27.9 µs 32 µs -12.76%
pipeline_extra_copy[65536] 106.1 µs 89.3 µs +18.93%
pipeline_extra_copy[73728] 118.5 µs 99.5 µs +19.12%
verify_all_methods[1024] 373.3 µs 131.1 µs ×2.8
verify_all_methods[16384] 35.7 ms 20.8 ms +71.2%

Footnotes

  1. 84 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

@codecov
Copy link

codecov bot commented Nov 24, 2025

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 85.68%. Comparing base (a917a39) to head (e0822e4).
⚠️ Report is 1 commits behind head on develop.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@0ax1 0ax1 added the changelog/fix A bug fix label Nov 24, 2025
@0ax1 0ax1 requested a review from joseph-isaacs November 24, 2025 14:00
Signed-off-by: Alexander Droste <alexander.droste@protonmail.com>
@0ax1 0ax1 force-pushed the ad/bench-refs-blackbox-inputs branch from 38c37b0 to c710e76 Compare November 24, 2025 14:01
@0ax1 0ax1 enabled auto-merge (squash) November 24, 2025 14:05
Comment on lines +799 to +800
.with_inputs(|| setup(SIZE))
.bench_refs(|(input_data, buffers)| {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

move this back out?

@0ax1 0ax1 merged commit f84c6b3 into develop Nov 24, 2025
45 of 46 checks passed
@0ax1 0ax1 deleted the ad/bench-refs-blackbox-inputs branch November 24, 2025 14:18
connortsui20 pushed a commit that referenced this pull request Nov 24, 2025
Inputs should be passed `with_inputs` to black box input args and
benchmarked with `bench_refs` to exclude `drop`.

Signed-off-by: Alexander Droste <alexander.droste@protonmail.com>
Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
a10y pushed a commit that referenced this pull request Nov 25, 2025
Inputs should be passed `with_inputs` to black box input args and
benchmarked with `bench_refs` to exclude `drop`.

Signed-off-by: Alexander Droste <alexander.droste@protonmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

changelog/fix A bug fix

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants