Skip to content

[XR][URP] Fix issue with vignette location in XR #5471

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 12 commits into from
Oct 12, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions com.unity.render-pipelines.universal/CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -179,6 +179,7 @@ and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.
- Support undo of URP Global Settings asset assignation (case 1342987).
- Removed unsupported fields from Presets of Light and Camera [case 1335979].
- Fixed graphical artefact when terrain height map is used with rendering layer mask for lighting.
- Fixed URP's vignette effect to respect XR's view center, since with Asymmetric FOV, the center of the view is not always the center of the texture [case 1358336](https://issuetracker.unity3d.com/issues/xr-sdk-urp-vignette-post-processing-effect-is-overlapping-between-eyes)
- Fixed an issue where screen space shadows has flickering with deferred mode [case 1354681](https://issuetracker.unity3d.com/issues/screen-space-shadows-flicker-in-scene-view-when-using-deferred-rendering)

### Changed
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -510,7 +510,7 @@ void Swap(ref ScriptableRenderer r)
// Setup other effects constants
SetupLensDistortion(m_Materials.uber, isSceneViewCamera);
SetupChromaticAberration(m_Materials.uber);
SetupVignette(m_Materials.uber);
SetupVignette(m_Materials.uber, cameraData.xr);
SetupColorGrading(cmd, ref renderingData, m_Materials.uber);

// Only apply dithering & grain if there isn't a final pass.
Expand Down Expand Up @@ -1263,12 +1263,25 @@ void SetupChromaticAberration(Material material)

#region Vignette

void SetupVignette(Material material)
void SetupVignette(Material material, XRPass xrPass)
{
var color = m_Vignette.color.value;
var center = m_Vignette.center.value;
var aspectRatio = m_Descriptor.width / (float)m_Descriptor.height;


#if ENABLE_VR && ENABLE_XR_MODULE
if (xrPass != null && xrPass.enabled)
{
if (xrPass.singlePassEnabled)
material.SetVector(ShaderConstants._Vignette_ParamsXR, xrPass.ApplyXRViewCenterOffset(center));
else
// In multi-pass mode we need to modify the eye center with the values from .xy of the corrected
// center since the version of the shader that is not single-pass will use the value in _Vignette_Params2
center = xrPass.ApplyXRViewCenterOffset(center);
}
#endif

var v1 = new Vector4(
color.r, color.g, color.b,
m_Vignette.rounded.value ? aspectRatio : 1f
Expand Down Expand Up @@ -1531,6 +1544,7 @@ static class ShaderConstants
public static readonly int _Chroma_Params = Shader.PropertyToID("_Chroma_Params");
public static readonly int _Vignette_Params1 = Shader.PropertyToID("_Vignette_Params1");
public static readonly int _Vignette_Params2 = Shader.PropertyToID("_Vignette_Params2");
public static readonly int _Vignette_ParamsXR = Shader.PropertyToID("_Vignette_ParamsXR");
public static readonly int _Lut_Params = Shader.PropertyToID("_Lut_Params");
public static readonly int _UserLut_Params = Shader.PropertyToID("_UserLut_Params");
public static readonly int _InternalLut = Shader.PropertyToID("_InternalLut");
Expand Down
35 changes: 35 additions & 0 deletions com.unity.render-pipelines.universal/Runtime/XR/XRPass.cs
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,7 @@ internal struct XRView
internal readonly Rect viewport;
internal readonly Mesh occlusionMesh;
internal readonly int textureArraySlice;
internal readonly Vector2 eyeCenterUV;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Quick note : this will create a merge conflict with #3557
Not a huge deal but let's discuss the order of PRs.


internal XRView(Matrix4x4 proj, Matrix4x4 view, Rect vp, int dstSlice)
{
Expand All @@ -46,6 +47,8 @@ internal XRView(Matrix4x4 proj, Matrix4x4 view, Rect vp, int dstSlice)
viewport = vp;
occlusionMesh = null;
textureArraySlice = dstSlice;

eyeCenterUV = ComputeEyeCenterUV(proj);
}

internal XRView(XRDisplaySubsystem.XRRenderPass renderPass, XRDisplaySubsystem.XRRenderParameter renderParameter)
Expand All @@ -61,6 +64,18 @@ internal XRView(XRDisplaySubsystem.XRRenderPass renderPass, XRDisplaySubsystem.X
viewport.width *= renderPass.renderTargetDesc.width;
viewport.y *= renderPass.renderTargetDesc.height;
viewport.height *= renderPass.renderTargetDesc.height;

eyeCenterUV = ComputeEyeCenterUV(projMatrix);
}

private static Vector2 ComputeEyeCenterUV(Matrix4x4 proj)
{
var projectionParameters = proj.decomposeProjection;
float left = Math.Abs(projectionParameters.left);
float right = Math.Abs(projectionParameters.right);
float top = Math.Abs(projectionParameters.top);
float bottom = Math.Abs(projectionParameters.bottom);
return new Vector2(left / (right + left), top / (top + bottom));
}
}

Expand Down Expand Up @@ -436,6 +451,26 @@ internal void RenderOcclusionMesh(CommandBuffer cmd)
}
}

// Take a point that is center-relative (0.5, 0.5) and modify it to be placed relative to the view's center instead,
// respecting the asymmetric FOV (if it is used)
internal Vector4 ApplyXRViewCenterOffset(Vector2 center)
{
Vector4 result = Vector4.zero;
float centerDeltaX = 0.5f - center.x;
float centerDeltaY = 0.5f - center.y;

result.x = views[0].eyeCenterUV.x - centerDeltaX;
result.y = views[0].eyeCenterUV.y - centerDeltaY;
if (singlePassEnabled)
{
// With single-pass XR, we need to add the data for the 2nd view
result.z = views[1].eyeCenterUV.x - centerDeltaX;
result.w = views[1].eyeCenterUV.y - centerDeltaY;
}

return result;
}

// Store array to avoid allocating every frame
private Matrix4x4[] stereoProjectionMatrix = new Matrix4x4[2];
private Matrix4x4[] stereoViewMatrix = new Matrix4x4[2];
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,9 @@ Shader "Hidden/Universal Render Pipeline/UberPost"
float _Chroma_Params;
half4 _Vignette_Params1;
float4 _Vignette_Params2;
#ifdef USING_STEREO_MATRICES
float4 _Vignette_ParamsXR;
#endif
float2 _Grain_Params;
float4 _Grain_TilingParams;
float4 _Bloom_Texture_TexelSize;
Expand All @@ -68,7 +71,12 @@ Shader "Hidden/Universal Render Pipeline/UberPost"
#define LensDirtIntensity _LensDirt_Intensity.x

#define VignetteColor _Vignette_Params1.xyz
#ifdef USING_STEREO_MATRICES
#define VignetteCenterEye0 _Vignette_ParamsXR.xy
#define VignetteCenterEye1 _Vignette_ParamsXR.zw
#else
#define VignetteCenter _Vignette_Params2.xy
#endif
#define VignetteIntensity _Vignette_Params2.z
#define VignetteSmoothness _Vignette_Params2.w
#define VignetteRoundness _Vignette_Params1.w
Expand Down Expand Up @@ -191,6 +199,12 @@ Shader "Hidden/Universal Render Pipeline/UberPost"
UNITY_BRANCH
if (VignetteIntensity > 0)
{
#ifdef USING_STEREO_MATRICES
// With XR, the views can use asymmetric FOV which will have the center of each
// view be at a different location.
const float2 VignetteCenter = unity_StereoEyeIndex == 0 ? VignetteCenterEye0 : VignetteCenterEye1;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In general, we'd like to be able to support more than 2 views. Can we evaluate moving this code to an array with indexing instead ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since this use case is very specific to eyes, I don't know if we would even need that for "more than two" displays.
And, as far as I know, we don't have 3-eyes human, yet ;)

Also, for more than 2 display scenarios, like domes, super panoramic views or stuff like that we would need to change the whole vignette post-process to have a center in world space to be able to sync all views to work together, a screen-space effect might not work well in that case without being fully custom.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Update: I had a chat with Fabien and I will implement "more than 2 views" for XR as there is some use-cases for it with some specific headsets

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Update 2 : Returning to 2 views to keep the code simple and we'll add more when we need to.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One of our partners is moving away from asymmetric frustums. We should keep this fix as simple as possible. What about extracting the center from the current stereo projection matrix in either vertex or pixel shader directly? Then there's no need to pass in per eye centers or changing script code.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't expect this change to prevent symmetric FOV from being used, it will only allow asymmetric FOV to work correctly. This change will work with both symmetric and asymmetric FOV, the center calculated for symmetric FOV will be (0.5, 0.5) for both eyes since in that case leftFov == rightFov && topFov == bottomFov.

Extracting the data from the shader (Vertex or Fragment) will make this change more complicated than it currently is since this is a post process and uses a generic Vertex Shader which does not have information about the projection. Changing that will introduce code duplication and as a side effect, more possible point of failures.

I believe that the eye center should be calculated on the CPU since it is something that will likely be used by other post process or shaders that rely on knowing where the center of the view is (any type of radial post-process, some screen space effects or more), it is also something that I expect some games will want to know too (I used these values for a few things in StarWars Squadrons VR).

In fact, I was very surprised that the XR view center coordinates was not already available in our XR C# code !

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, the points sound reasonable.

#endif

color = ApplyVignette(color, uvDistorted, VignetteCenter, VignetteIntensity, VignetteRoundness, VignetteSmoothness, VignetteColor);
}

Expand Down