Add example to test the smoothness/accuracy of the timestep #4865

nfagerlund · 2022-05-29T00:06:04Z

Objective

Bevy's timestep is sometimes inaccurate, which can manifest as hiccups and stutters in what should
be smooth motion. This is most clearly visible when using pixel-perfect movement with large
visible pixels.

It would be good to have some kind of shared baseline for any conversations about hiccups, stutters, and other time-based jank!

Solution

This PR adds a new example (in the stress tests directory) that scrolls a background behind a character at 120
pixels per second; by default, it makes sure to always move in whole-pixel increments. You can modify
some behavior while it's running by pressing keys:

P: cycle between different window::PresentModes. (NOTE: Window.set_present_mode() doesn't appear to actually work at the moment! Not sure what's up there and would appreciate any hints.)
W: cycle between different window::WindowModes. (NOTE: Window.set_resolution() doesn't appear to do anything? I might be using it wrong?)
T: cycle between updating scroll position based on Time.delta_seconds(), and updating it by
just assuming a 16.667 millisecond timestep (only useful in PresentMode::Fifo on a 60fps display).
M: cycle between whole-pixel movement and simple sub-pixel movement.

Changelog

Added a time_smoothness example (under stress_tests) for qualitatively evaluating the smoothness and accuracy of delta time.

Videos

A couple of videos of weird stuttering in action on Windows.

This computer is a desktop with a Haswell Core i3-4160 (entry-level processor from 2014), 8gb of ram, and a GTX 1060 (3gb) video card, running Windows 10 Home 21H2. It's quite modest! But it runs a wide range of commercial games with solid and respectable performance, and scrolling a background probably shouldn't look as bad as it does here.

Windowed mode:

2022-05-24.00-03-43.bevy.windowed.mp4

Borderless fullscreen:

2022-05-24.00-05-52.bevy.fullscreen.mp4

examples/stress_tests/time_smoothness.rs

alice-i-cecile

I'm impressed; this should be very useful! A few administrative notes:

Please add a module style doc comment to the top of your example.
Please add the example to examples/README.md

examples/stress_tests/time_smoothness.rs

alice-i-cecile

Like I suggested on Discord, I'd like to see some numbers reported periodically in the console. To start, I think "mean, standard deviation, min and max frame time" for the last 10 seconds should be enough to give testers a better sense of what's going on.

You should be able to get this information from the FrameTimeDiagnosticsPlugin (I think in a resource); ping if you can't figure it out.

nfagerlund · 2022-06-01T17:20:00Z

@alice-i-cecile Thanks for the review! I'll add the comments/links now, and I'll look into doing math on the diags resource later this week.

alice-i-cecile · 2022-06-01T20:19:46Z

@nfagerlund you can press "Resolve Comment" on the feedback that you've addressed, which keeps the thread tidier for reviewers :)

nfagerlund · 2022-06-08T06:30:36Z

@alice-i-cecile Ok! I've added some frame time analysis to the example. I had to roll my own diagnostic, since it turns out the built-in one only keeps a non-configurable 20 frames, but now it prints this every 10 seconds:

2022-06-08T06:21:55.395210Z  INFO time_smoothness: -------------------------
2022-06-08T06:21:55.395538Z  INFO time_smoothness: Average frame time: 0.016681
2022-06-08T06:21:55.395631Z  INFO time_smoothness: Standard deviation: 0.000569
2022-06-08T06:21:55.395689Z  INFO time_smoothness: Shortest frame:     0.006377
2022-06-08T06:21:55.395752Z  INFO time_smoothness: 95th percentile:    0.016797
2022-06-08T06:21:55.395840Z  INFO time_smoothness: 99th percentile:    0.017011
2022-06-08T06:21:55.395928Z  INFO time_smoothness: 99.5th percentile:  0.017293
2022-06-08T06:21:55.396014Z  INFO time_smoothness: Longest frame:      0.026963
2022-06-08T06:21:55.396077Z  INFO time_smoothness: -------------------------

I still need to go deal with Clippy, but figured I'd paste that before the output scrolls out of my terminal.

nfagerlund · 2022-06-08T06:52:37Z

OK, all hail clippy, TIL about .copied().

alice-i-cecile · 2022-06-08T07:05:54Z

Awesome, that's super useful both as a snippet and as a testing tool.

nfagerlund · 2022-06-08T19:39:10Z

Hmm, I can't tell why CI is failing.

| ^^^^^^^^^^^^^^^^^^^^^^^^use of undeclared type `OrthographicCameraBundle`
# ...
| ^^^^^^^^^^^^^^use of undeclared type `UiCameraBundle`

I think those are in the prelude?? It compiles and runs?? What's going on here.

hymm · 2022-06-08T19:43:16Z

They were renamed/refactored in this PR #4745.

nfagerlund · 2022-06-08T19:53:25Z

Oh, I also want to surface a question from the original post that maybe got lost — I have code in there that calls Window.set_present_mode() and Window.set_resolution(), and neither one of those seems to actually work. Anyone know what's up with those?

nfagerlund · 2022-06-08T19:54:25Z

@hymm oh! Hmm, so the CI job must be merging or rebasing, instead of running on the branch. I'll rebase later and sort out the rename. thank you!

alice-i-cecile · 2022-06-08T19:57:58Z

Oh, I also want to surface a question from the original post that maybe got lost — I have code in there that calls Window.set_present_mode() and Window.set_resolution(), and neither one of those seems to actually work. Anyone know what's up with those?

IIRC you had to set this before calling .add_default_plugins, due to a quirk in the way winit owns the control loop :(

nfagerlund · 2022-06-08T22:56:03Z

Oh... huh. I mean, I'm adding the window descriptor resource before that, so the initial settings take effect just fine. But so, can they just not be changed at runtime? Why would the functions on Window exist, then? I don't think you can even get hold of a Window struct until the app's running, yeah?

Maybe I should remove those config bits and just tell users to edit and recompile. :/

aevyrie · 2022-06-08T23:38:44Z

Could you change the stats to ms instead of s? Easier to read at those scales.

alice-i-cecile · 2022-06-08T23:48:17Z

But so, can they just not be changed at runtime? Why would the functions on Window exist, then? I don't think you can even get hold of a Window struct until the app's running, yeah?

This is about my feeling. Can you open an issue?

Bevy's timestep is sometimes inaccurate, and sometimes it misses its frame timings. Both of these can manifest as hiccups and stutters in what should be smooth motion. This is most clearly visible when using pixel-perfect movement with large visible pixels. This new example (in the stress tests directory) scrolls a background behind a character at 120 pixels per second; by default, it makes sure to always move in whole-pixel increments. You can modify some behavior while it's running by pressing keys: - P: cycle between different `window::PresentMode`s. - W: cycle between different `window::WindowMode`s. - T: cycle between updating scroll position based on `Time.delta_seconds()`, and updating it by just assuming a 16.667 millisecond timestep (only useful in `PresentMode::Fifo` on a 60fps display). - M: cycle between whole-pixel movement and simple sub-pixel movement.

nfagerlund · 2022-06-27T01:02:17Z

OK, I'm back!!

I changed the frame time analysis to report in milliseconds.
I updated stuff that had fallen out of date with main, including the new camera types and the blurry-by-default texture smoothing. And I think I got the new examples metadata into shape correctly; we'll see in a sec.
I filed an issue for those inert Window methods, Window.set_resolution() and set_present_mode() don't have expected side-effects #5111 -- it looks like they're intended to work, so I left the example using them as advertised.

alice-i-cecile

Looks good! I'd be more comfortable merging this if you can drop a comment on the broken window commands linking to the issue in question.

I think it's probably worth leaving them in, if only to make it easy to test fixes to that. Other opinions would be welcome though.

Outside of that question, I'm very happy with this now.

hymm

The back of the Y in bevy is getting cut off for me on every other scroll by.

hymm · 2022-06-27T18:31:04Z

examples/stress_tests/time_smoothness.rs

+}
+
+fn std_deviation(diagnostic: &Diagnostic) -> Option<f64> {
+    if let Some(average) = diagnostic.average() {


the average is being calculated both in this function and in the above function. Could we remove one of these?

hymm · 2022-06-27T19:38:00Z

examples/stress_tests/time_smoothness.rs

+const BG_WIDTH: f32 = 755.0;
+const BG_HEIGHT: f32 = 363.0;
+const BG_TILES: usize = 3;
+const BG_SPEED: f32 = 120.0;


Maybe add a recommendation here to change this number for different monitors. This is always going to hit the "big jump" warning on a 30Hz monitor and will probably always look jumpy on a 144Hz monitor.

bas-ie · 2024-10-02T22:37:03Z

Closing as part of backlog cleanup: still seems like an interesting example, but would need significant revision and may not be as relevant as Bevy matures.

alice-i-cecile added C-Examples An addition or correction to our examples C-Performance A change motivated by improving speed, memory usage or compile times A-Core labels May 29, 2022

alice-i-cecile reviewed May 30, 2022

View reviewed changes

examples/stress_tests/time_smoothness.rs Outdated Show resolved Hide resolved

alice-i-cecile requested changes May 30, 2022

View reviewed changes

alice-i-cecile reviewed May 30, 2022

View reviewed changes

examples/stress_tests/time_smoothness.rs Show resolved Hide resolved

alice-i-cecile requested changes May 30, 2022

View reviewed changes

nfagerlund force-pushed the nf/may22-time-smoothness-stresstest branch from f2b244c to 7e80881 Compare June 8, 2022 06:20

nfagerlund force-pushed the nf/may22-time-smoothness-stresstest branch from 7e80881 to c996988 Compare June 8, 2022 06:51

alice-i-cecile approved these changes Jun 8, 2022

View reviewed changes

alice-i-cecile requested review from aevyrie, inodentry and hymm June 8, 2022 07:05

nfagerlund force-pushed the nf/may22-time-smoothness-stresstest branch from c996988 to 80580eb Compare June 27, 2022 00:30

nfagerlund mentioned this pull request Jun 27, 2022

Window.set_resolution() and set_present_mode() don't have expected side-effects #5111

Open

nfagerlund force-pushed the nf/may22-time-smoothness-stresstest branch from 80580eb to af82dff Compare June 27, 2022 00:57

nfagerlund added 2 commits June 26, 2022 18:03

Appease clippy and remove identical match arm

de9b9a0

Run build-example-pages script

5850902

alice-i-cecile requested changes Jun 27, 2022

View reviewed changes

hymm reviewed Jun 27, 2022

View reviewed changes

hymm self-requested a review January 28, 2023 19:06

ItsDoot added the S-Adopt-Me The original PR author has no intent to complete this work. Pick me up! label Aug 19, 2024

bas-ie closed this Oct 2, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add example to test the smoothness/accuracy of the timestep #4865

Add example to test the smoothness/accuracy of the timestep #4865

nfagerlund commented May 29, 2022 •

edited

Loading

alice-i-cecile left a comment

alice-i-cecile left a comment

nfagerlund commented Jun 1, 2022

alice-i-cecile commented Jun 1, 2022

nfagerlund commented Jun 8, 2022

nfagerlund commented Jun 8, 2022

alice-i-cecile commented Jun 8, 2022

nfagerlund commented Jun 8, 2022

hymm commented Jun 8, 2022

nfagerlund commented Jun 8, 2022

nfagerlund commented Jun 8, 2022

alice-i-cecile commented Jun 8, 2022

nfagerlund commented Jun 8, 2022

aevyrie commented Jun 8, 2022

alice-i-cecile commented Jun 8, 2022

nfagerlund commented Jun 27, 2022

alice-i-cecile left a comment

hymm left a comment

hymm Jun 27, 2022

hymm Jun 27, 2022

bas-ie commented Oct 2, 2024

Add example to test the smoothness/accuracy of the timestep #4865

Add example to test the smoothness/accuracy of the timestep #4865

Conversation

nfagerlund commented May 29, 2022 • edited Loading

Objective

Solution

Changelog

Videos

alice-i-cecile left a comment

Choose a reason for hiding this comment

alice-i-cecile left a comment

Choose a reason for hiding this comment

nfagerlund commented Jun 1, 2022

alice-i-cecile commented Jun 1, 2022

nfagerlund commented Jun 8, 2022

nfagerlund commented Jun 8, 2022

alice-i-cecile commented Jun 8, 2022

nfagerlund commented Jun 8, 2022

hymm commented Jun 8, 2022

nfagerlund commented Jun 8, 2022

nfagerlund commented Jun 8, 2022

alice-i-cecile commented Jun 8, 2022

nfagerlund commented Jun 8, 2022

aevyrie commented Jun 8, 2022

alice-i-cecile commented Jun 8, 2022

nfagerlund commented Jun 27, 2022

alice-i-cecile left a comment

Choose a reason for hiding this comment

hymm left a comment

Choose a reason for hiding this comment

hymm Jun 27, 2022

Choose a reason for hiding this comment

hymm Jun 27, 2022

Choose a reason for hiding this comment

bas-ie commented Oct 2, 2024

nfagerlund commented May 29, 2022 •

edited

Loading