-
-
Notifications
You must be signed in to change notification settings - Fork 4.1k
Environment Map Filtering GPU pipeline #19076
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Some thoughts on the user API:
|
Ran it through NSight.
Generally those buffer to image copies are way too slow, need to figure out what those are and remove them. Radiance pass looks good, and irradiance pass isn't that slow but has very suspiciously low active threads per warp. |
Based on the shader profiler, the reason irradiance map is performing poorly is the giant switch statement absolutely kills perf as every thread gets masked out. Should 100% index into an array for that one. |
Oh, I see that you're also doing a separate dispatch per mip for the radiance pass. Does each level depend on the previous? |
This fixes #19125 |
After a discussion this morning, we agreed to add the spatio temporal blue noise texture to Bevy PBR plugin. This is from Electronic Arts' fastnoise repo. https://github.com/electronicarts/fastnoise/ I combined the texture using the following command: wget https://github.com/electronicarts/fastnoise/blob/main/noise.zip
unzip noise.zip && cd noise
ktx create \
--format R8G8B8_UNORM \
--assign-tf linear \
--layers 32 \
vector3/temporal/exp/vector3_uniform_box3x3_exp0101_product_*.png \
stbn.ktx2 ![]() I did a quick sanity check and decided that the best option going with box3x3. This is the texture most path tracers and ReSTIR implementations ship with today. even for the filtering, I read exactly one texel per pixel, so aliasing that the Gaussian blur fights is not an issue. Box3x3 gives the cleanest high-frequency spectrum and fastest convergence.
Since this step is complete, no remaining actions needed for this PR. ready for a final review and merge. |
CI is failing; please take a look! |
Fixed the runtime issue on linux with the vulkan backend. I got a working version of web build locally, switching the bindings around a bit. Since the web support was not in the initial scope of the PR but now including it so the CI can pass on merge. I will create a separate branch and review the changes merging into this branch since everything here has already been reviewed and approved. |
Got feedback from Jasmine that I should try to keep the original setup with the full bind group for the native build, and rely on feature flags - I will attempt to make these changes in the PR to this branch linked above without making a mess. |
@alice-i-cecile this is ready for a merge again. The CI should pass for web and linux. |
this PR broke WebGL2: #20276 |
this PR also added an unconditional 1.5MB embedded file to everyone enabling I think it shouldn't have been merged without this being more discussed, and with a way to avoid it. I'll revert it unless we have a credible plan to fix both issues (webgl2 and size) |
# Objective - #19076 (comment) broke webgl2 and bloated binary size by a megabyte - Fixes #20276 ## Solution - Don't embed stbn.ktx2, just load it for now - Gate gen env maps by limit checks - Extract a plugin to do this cleanly - Only load stbn.ktx2 when its needed ## Testing - reflection_probes example - someone please test webgl2 --------- Co-authored-by: Máté Homolya <mate.homolya@gmail.com>
# Objective - bevyengine#19076 (comment) broke webgl2 and bloated binary size by a megabyte - Fixes bevyengine#20276 ## Solution - Don't embed stbn.ktx2, just load it for now - Gate gen env maps by limit checks - Extract a plugin to do this cleanly - Only load stbn.ktx2 when its needed ## Testing - reflection_probes example - someone please test webgl2 --------- Co-authored-by: Máté Homolya <mate.homolya@gmail.com>
# Objective - bevyengine#19076 (comment) broke webgl2 and bloated binary size by a megabyte - Fixes bevyengine#20276 ## Solution - Don't embed stbn.ktx2, just load it for now - Gate gen env maps by limit checks - Extract a plugin to do this cleanly - Only load stbn.ktx2 when its needed ## Testing - reflection_probes example - someone please test webgl2 --------- Co-authored-by: Máté Homolya <mate.homolya@gmail.com>
Objective
This PR implements a robust GPU-based pipeline for dynamically generating environment maps in Bevy. It builds upon PR #19037, allowing these changes to be evaluated independently from the atmosphere implementation.
While existing offline tools can process environment maps, generate mip levels, and calculate specular lighting with importance sampling, they're limited to static file-based workflows. This PR introduces a real-time GPU pipeline that dynamically generates complete environment maps from a single cubemap texture on each frame.
Closes #9380
Solution
Implemented a Single Pass Downsampling (SPD) pipeline that processes textures without pre-existing mip levels or pre-filtered lighting data.
Single Pass Downsampling (SPD) pipeline:
copy_mip0
) before the down-sampling pass;Pre-filtering pipeline:
The pre-filtering pipeline is largely based on these articles:
Previous work: #9414
Testing
The
reflection_probes.rs
example has been updated:Render Graph
Composed of two nodes and a graph edge:
Pass breakdown:
Showcase
User facing API:
Computed Environment Maps
To use fully dynamic environment maps, create a new placeholder image handle with
Image::new_fill
, extract it to the render world. Then dispatch a compute shader, bind the image as a 2d array storage texture. Anything can be rendered to the custom dynamic environment map.This is already demonstrated in PR #19037 with the
atmosphere.rs
example.We can extend this idea further and run the entire PBR pipeline from the perspective of the light probe, and it is possible to have some form of global illumination or baked lighting information this way, especially if we make use of irradiance volumes for the realtime aspect. This method could very well be extended to bake indirect lighting in the scene.
#13840 should make this possible!
Notes for reviewers
This PR no longer bundles any large test textures.