Implement parallax-corrected cubemaps (PCCMs) for reflection probes. #22288
Conversation
Bevy doesn't currently ever apply parallax correction to cubemaps, so reflections are rendered as though the environment were infinitely far away. This is often acceptable for outdoor scenes in which the environment is very distant, but for indoor scenes and dense environments this is undesirable. The standard solution for this problem is *parallax correction*, in which each reflection probe is augmented with a bounding box, and a raytrace is performed against the bounding box in order to determine the proper direction for sampling the cubemap. This commit implements parallax correction in Bevy in an opt-in manner. Add the `ParallaxCorrect` component to a `LightProbe` with an `EnvironmentMapLight` in order to opt into it. The bounding box used for parallax correction is assumed to be identical to the bounding box of the influence of the reflection probe itself. This is a reasonable default and matches what Blender does; it's what you want when you have, for example, a cubemap that captures the interior of a rectangular room. However, a future follow-up PR may wish to extend this so that the bounding box used for parallax correction might not coincide with the bounding box used for the influence of the reflection probe. This would require increasing the GPU size of the light probe data, so in order to keep this patch small and self-contained, I opted to defer this potential future enhancement to a follow-up. The patch is generally straightforward, adding only the minimal enhancements to the `LightProbe` trait and the GPU representation of light probes needed to propagate the parallax correction flag through the rendering pipeline. Additionally, this commit fixes a bug whereby the transform of each cubemap reflection probe wasn't being taken into account in the shader. I believe that this was being masked because most cubemaps are rendered in world space and therefore most cubemap reflection probes have an identity rotation. A new example, `pccm`, has been added, demonstrating the effect of parallax correction. It shows a scene consisting of an outer textured cube with an inner rotating reflective cube. The outer textured cube contains a reflection probe containing a snapshot of the scene (prerendered in Blender). Parallax correction can be toggled on and off in the example in order to demonstrate its effect.
|
After this gets initial review, I'll add the new baked cubemap reflection assets to https://github.com/bevyengine/bevy_asset_files so that we don't have to add them to the main repo. Screenshot of the new And with parallax correction off: |
|
Can we make ParallaxCorrect not-opt-in? I think it's generally helpful to have less knobs to tune, and I don't see anyone wanting to turn this off. |
|
@JMS55 OK, I made parallax correction opt-out instead of opt-in. |
pcwalton
added
S-Needs-Review
|
Closes #16709 |
atlv24
left a comment
atlv24
left a comment
There was a problem hiding this comment.
I played around with this - seems like something isnt quite right. Digging into the math now.
| intensity: ENVIRONMENT_MAP_INTENSITY, | ||
| ..default() | ||
| }, | ||
| Transform::from_scale(Vec3::splat(5.0)), |
There was a problem hiding this comment.
The glb cube spans -5.0..5.0 and as such has size 10x10x10, but the environment map light is a unit cube at the origin (-0.5..0.5) so this needs to be 10.0 actually
| Transform::from_scale(Vec3::splat(5.0)), | |
| Transform::from_scale(Vec3::splat(10.0)), |
There was a problem hiding this comment.
Doing this seems to cause the glb to have a form of z-fighting, where it can't decide if its inside or outside the environment light so some pixels get lit by it and some dont. Might need to exclude it from env map lights.
|
I think the glb is just wrong. |
There was a problem hiding this comment.
+23 MB +8 MB for this example is not justifiable, for reference all the other bevy assets combined are 54 MB. Can we put this on the web assets repo or use something smaller?
There was a problem hiding this comment.
There was a problem hiding this comment.
Yeah, my plan was to put this on bevy asset files once it’s reviewed.
atlv24
left a comment
atlv24
left a comment
There was a problem hiding this comment.
The implementation is correct and high quality. The example is comprehensive, has a few flaws though. The only blocker is the asset sizes imo.
tychedelia
left a comment
tychedelia
left a comment
There was a problem hiding this comment.
Looks great thanks. Approve being move of assets to web.
4 participants
Bevy doesn't currently ever apply parallax correction to cubemaps, so reflections are rendered as though the environment were infinitely far away. This is often acceptable for outdoor scenes in which the environment is very distant, but for indoor scenes and dense environments this is undesirable. The standard solution for this problem is parallax correction, in which each reflection probe is augmented with a bounding box, and a raytrace is performed against the bounding box in order to determine the proper direction for sampling the cubemap.
This commit implements parallax correction in Bevy for light probes in an opt-out manner. (You may add the
NoParallaxCorrectioncomponent to aLightProbewith anEnvironmentMapLightin order to opt out of it.) The bounding box used for parallax correction is assumed to be identical to the bounding box of the influence of the reflection probe itself. This is a reasonable default and matches what Blender does; it's what you want when you have, for example, a cubemap that captures the interior of a rectangular room. However, a future follow-up PR may wish to extend this so that the bounding box used for parallax correction might not coincide with the bounding box used for the influence of the reflection probe. This would require increasing the GPU size of the light probe data, so in order to keep this patch small and self-contained, I opted to defer this potential future enhancement to a follow-up.The patch is generally straightforward, adding only the minimal enhancements to the
LightProbetrait and the GPU representation of light probes needed to propagate the parallax correction flag through the rendering pipeline.Additionally, this commit fixes a bug whereby the transform of each cubemap reflection probe wasn't being taken into account in the shader. I believe that this was being masked because most cubemaps are rendered in world space and therefore most cubemap reflection probes have an identity rotation.
A new example,
pccm, has been added, demonstrating the effect of parallax correction. It shows a scene consisting of an outer textured cube with an inner rotating reflective cube. The outer textured cube contains a reflection probe containing a snapshot of the scene (prerendered in Blender). Parallax correction can be toggled on and off in the example in order to demonstrate its effect.