Skip to content

Conversation

@BD103
Copy link
Member

@BD103 BD103 commented Dec 26, 2024

Objective

Many of our benchmarks use criterion::black_box(), which is used to prevent the compiler from optimizing away computation that we're trying to time. This can be slow, though, because criterion::black_box() forces a point read each time it is called through ptr::road_volatile().

In Rust 1.66, the standard library introduced core::hint::black_box() (and std::hint::black_box()). This is an intended replacement for criterion's version that uses compiler intrinsics instead of volatile pointer reads, and thus has no runtime overhead. This increases benchmark accuracy, which is always nice 👍

Note that benchmarks may appear to improve in performance after this change, but that's just because we are eliminating the pointer read overhead.

Solution

  • Deny criterion::black_box in clippy.toml.
  • Fix all imports.

Testing

  • cargo clippy -p benches --benches

…tead

`core::hint::black_box()` has no performance overhead, and is the preferred method of blocking optimizations in benchmarks. It was likely introduced after our benchmarks were created, which is why we didn't use it before.
@BD103 BD103 added D-Trivial Nice and easy! A great choice to get started with Bevy C-Code-Quality A section of code that is hard to understand or change C-Benchmarks Stress tests and benchmarks used to measure how fast things are A-Cross-Cutting Impacts the entire engine X-Uncontroversial This work is generally agreed upon S-Needs-Review Needs reviewer attention (from anyone!) to move forward labels Dec 26, 2024
@BD103 BD103 added S-Ready-For-Final-Review This PR has been approved by the community. It's ready for a maintainer to consider merging it and removed S-Needs-Review Needs reviewer attention (from anyone!) to move forward labels Dec 26, 2024
@alice-i-cecile alice-i-cecile added this pull request to the merge queue Dec 29, 2024
Merged via the queue into bevyengine:main with commit 17ad855 Dec 29, 2024
28 checks passed
ecoskey pushed a commit to ecoskey/bevy that referenced this pull request Jan 6, 2025
# Objective

Many of our benchmarks use
[`criterion::black_box()`](https://docs.rs/criterion/latest/criterion/fn.black_box.html),
which is used to prevent the compiler from optimizing away computation
that we're trying to time. This can be slow, though, because
`criterion::black_box()` forces a point read each time it is called
through
[`ptr::road_volatile()`](https://doc.rust-lang.org/stable/std/ptr/fn.read_volatile.html).

In Rust 1.66, the standard library introduced
[`core::hint::black_box()`](https://doc.rust-lang.org/nightly/std/hint/fn.black_box.html)
(and `std::hint::black_box()`). This is an intended replacement for
`criterion`'s version that uses compiler intrinsics instead of volatile
pointer reads, and thus has no runtime overhead. This increases
benchmark accuracy, which is always nice 👍

Note that benchmarks may _appear_ to improve in performance after this
change, but that's just because we are eliminating the pointer read
overhead.

## Solution

- Deny `criterion::black_box` in `clippy.toml`.
- Fix all imports.

## Testing

- `cargo clippy -p benches --benches`
mrchantey pushed a commit to mrchantey/bevy that referenced this pull request Feb 4, 2025
# Objective

Many of our benchmarks use
[`criterion::black_box()`](https://docs.rs/criterion/latest/criterion/fn.black_box.html),
which is used to prevent the compiler from optimizing away computation
that we're trying to time. This can be slow, though, because
`criterion::black_box()` forces a point read each time it is called
through
[`ptr::road_volatile()`](https://doc.rust-lang.org/stable/std/ptr/fn.read_volatile.html).

In Rust 1.66, the standard library introduced
[`core::hint::black_box()`](https://doc.rust-lang.org/nightly/std/hint/fn.black_box.html)
(and `std::hint::black_box()`). This is an intended replacement for
`criterion`'s version that uses compiler intrinsics instead of volatile
pointer reads, and thus has no runtime overhead. This increases
benchmark accuracy, which is always nice 👍

Note that benchmarks may _appear_ to improve in performance after this
change, but that's just because we are eliminating the pointer read
overhead.

## Solution

- Deny `criterion::black_box` in `clippy.toml`.
- Fix all imports.

## Testing

- `cargo clippy -p benches --benches`
@BD103 BD103 deleted the migrate-black-box branch February 11, 2025 19:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

A-Cross-Cutting Impacts the entire engine C-Benchmarks Stress tests and benchmarks used to measure how fast things are C-Code-Quality A section of code that is hard to understand or change D-Trivial Nice and easy! A great choice to get started with Bevy S-Ready-For-Final-Review This PR has been approved by the community. It's ready for a maintainer to consider merging it X-Uncontroversial This work is generally agreed upon

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants