Skip to content

Universal, Fix Swap Buffer Issue #6522

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Dec 9, 2021
Merged

Conversation

etienne-p
Copy link
Contributor

While writing custom Scriptable Render Features,
I stumbled upon a Dimensions of color surface does not match dimensions of depth surface error.

After some digging, I noticed that in the UniversalRenderer, line 1196 in SwapColorBuffer, we have:
ConfigureCameraTarget(m_ColorBufferSystem.GetBackBuffer(cmd), m_ColorBufferSystem.GetBufferA());

In some cases, this would result in m_CameraColorTarget and m_CameraDepthTarget RTs having different dimensions.

The cause is that when invoking GetBufferA on RenderTargetBufferSystem,
if msaa is not used, the RTHandle we get may not have been reallocated with the proper dimensions.

Indeed, in the RenderTargetBufferSystem.ReAllocate,
we do not reallocate the msaa rtHandles if msaa is not used (!(desc.msaaSamples > 1)).
So previously allocated msaa targets (with different dimensions) survive and later get used,
assigned as depth target through GetBufferA().

While my fix seems to work,
I'm not very familiar with this area and will let you determine if it is indeed the right way to fix it.
I'm not sure I understand the introduction/use/implementation of GetBufferA() to begin with.
(It is only used in one place afaik.)

@github-actions
Copy link

github-actions bot commented Dec 8, 2021

Hi! This comment will help you figure out which jobs to run before merging your PR. The suggestions are dynamic based on what files you have changed.
Link to Yamato: https://unity-ci.cds.internal.unity3d.com/project/902/
Search for your PR branch using the search bar at the top, then add the following segment(s) to the end of the URL (you may need multiple tabs depending on how many packages you change)

URP
/jobDefinition/.yamato%252Fall-urp.yml%2523PR_URP_trunk
With changes to URP packages, you should also run
/jobDefinition/.yamato%2Fall-lightmapping.yml%23PR_Lightmapping_trunk

Depending on the scope of your PR, you may need to run more jobs than what has been suggested. Please speak to your lead or a Graphics SDET (#devs-graphics-automation) if you are unsure.

@etienne-p etienne-p changed the title Fix Swap Buffer Issue Universal, Fix Swap Buffer Issue Dec 8, 2021
@sandy-carter-unity
Copy link
Contributor

sandy-carter-unity commented Dec 9, 2021

Thanks for pointing this out

Looks like an omission, probably happened during a merge from the RTHandle branch and some work @robin-demoor was doing.

@etienne-p etienne-p marked this pull request as ready for review December 9, 2021 17:39
@etienne-p etienne-p requested review from a team as code owners December 9, 2021 17:39
@etienne-p etienne-p merged commit a31eb2a into master Dec 9, 2021
@etienne-p etienne-p deleted the universal/fix/swapbuffers branch December 9, 2021 17:39
@robin-demoor
Copy link
Contributor

At first sight this looks good to me, I don't have access to my workstation right now so will have a closer look tomorrow to be sure. A note on the GetBufferA. Before RTHandles the depth and color were stored together in one RenderTargetHandle. Because there was no need for a swapbuffer system for depth. The depth was always stored in A and never in B so we don't initialise two depth buffers. Because of that there was a need to specifically be able to access buffer A to get the depth. As far as I am aware RTHandles splits depth and color? So this GetBufferA can probably be removed and be replaced by wherever the intermediate depth is currently stored.

@etienne-p
Copy link
Contributor Author

@robin-demoor thanks, I hope I didn't land too fast (in any case it's an improvement afaik). If we can ditch GetBufferA it'd likely be better. It kind of stands out as odd in the RenderTargetBufferSystem interface and the unique usage confirms this. Feel free to submit another PR if you'd rather iterate on the fix, I'd be interested so add me as a (non blocking) reviewer please. (I'm on the Virtual Production team, not graphics, I can only allocate so much time to SRP fixes). We could use a comment explaining what you just stated in SwapColorBuffer :)

unity-emilk pushed a commit that referenced this pull request Jun 10, 2022
JIRA Epic: https://jira.unity3d.com/browse/CD-74

Re-submission of #6597 which got reverted due to test failures.

This PR introduces **Screen Coordinates Override** (**SCO** for the remainder of this document) for both URP and HDRP. SCO
provides a transformation of screen space coordinates when evaluating screen space effects.

We have found it useful in virtual production scenarios.

The first use case for this is **Cluster Display** (formerly known as [Cluster Rendering](https://docs.unity3d.com/560/Documentation/Manual/ClusterRendering.html)), that is, distributed rendering. It allows for post effects such as vignette to be properly rendered at the scale of a grid of displays.

Another use case is the implementation of Lens Distortion when replicating the distortion incurred by a physical lens. Since pixels near the edges may be pushed inwards of the viewport by the distortion, we need to render the scene with overscan. (By overscan, we mean rendering to an expanded pixel surface, later cropped out to its original size before being
presented.) To preserve the distortion's aspect in the cropped image, we need to apply a transform to screen space coordinates when the lens distortion is being computed. 

The first implementation of this technique was part of the first Cluster Display effort, and was then called **Cluster
Display Space** or **Global Screen Space**. A high level description of the technique can be found in
[this document](https://docs.google.com/document/d/1I_mR42Tt119nVhhHwHG9F_n7DwpPin7Tds4yhx59oko/edit?usp=sharing) and [this one](https://docs.google.com/document/d/1qZqUTA5VAeV35PkldjOXXjpKD-JxWfgs2KJC6rh-4mo/edit?usp=sharing).

The switch to the **Screen Coordinates Override** terminology is justified by the fact that the technique is relevant outside of
Cluster Display. As illustrated above by our Lens Distortion example.

In the changes we regularly see the introduction of 2 fields:

- `screenCoordScaleBias` holds the scale and bias used to transform screen space coordinates.
- `screenSizeOverride` holds the screen size (and its reciprocal) to be used in conjunction with transformed screen
  space coordinates.

---
### Changes Overview

* #### SRP Core
    - `ScreenCoordOverrideUtils` -> basic utilities to manage the shader keyword. Main point is to avoid the duplication of
      the shader keyword.
    - `ScreenCoordOverride.hlsl` -> shader code and macros to perform the screen space coordinates conversion.

* #### HDRP
    - `FrameSettings` -> add `ScreenCoordOverride` and `AsymmetricProjection` settings. `AsymmetricProjection` allows the
      activation of the asymmetric projection code path even when XR is not enabled. This is useful for Cluster Display.
    - `HDAdditionalCameraData` -> add the `ScreenSizeOverride` and `ScreenCoordScaleBias` fields.
    - `ShaderVariablesGlobal` -> add the `ScreenSizeOverride` and `ScreenCoordScaleBias` fields.
    - `HDCamera` -> update global shader variables.
    - `HDRenderPipeline` -> activates the shader keyword.
    - `HDRenderPipeline.PostProcess` -> activates the shader keyword based on frame settings.
    - `UberPost.compute` -> add SCO support.

* #### URP
    - `ScriptableRenderer` -> fix a swap buffer issue (first
      fixed [here](#6522))
    - `PostProcessUtils` -> init random generation using frame count so that it's consistent across machines in a Cluster
      Display scenario.
    - `UniversalAdditionalCameraData` -> add the `ScreenSizeOverride` and `ScreenCoordScaleBias` fields and feature
      activation.
    - `UniversalRenderPipeline` -> populate additional camera data.
    - `UniversalRenderPipelineCore` -> add fields and associated shader properties.
    - `UniversalRenderer` -> related to swap buffer fix.
    - `Input.hlsl` -> add the `ScreenSizeOverride` and `ScreenCoordScaleBias` fields.
    - `FinalPost.shader` -> add SCO support.
    - `UberPost.shader` -> add SCO support.
    - `CapturePass` -> fixed color buffer access (before, post effects would not be captured)
    - `UniversalRenderPipelineGlobalSettings` -> add shader stripping option (`SerializedUniversalRenderPipelineGlobalSettings`, `UniversalRenderPipelineGlobalSettingsUI.Drawers` and `UniversalRenderPipelineGlobalSettingsUI.Skin` were modified to reflect this change)
    - `ShaderPreprocessor` -> implement shader stripping option
unity-emilk pushed a commit that referenced this pull request Oct 14, 2022
Error-mdl pushed a commit to StressLevelZero/Custom-URP that referenced this pull request Jan 7, 2023
JIRA Epic: https://jira.unity3d.com/browse/CD-74

Re-submission of Unity-Technologies/Graphics#6597 which got reverted due to test failures.

This PR introduces **Screen Coordinates Override** (**SCO** for the remainder of this document) for both URP and HDRP. SCO
provides a transformation of screen space coordinates when evaluating screen space effects.

We have found it useful in virtual production scenarios.

The first use case for this is **Cluster Display** (formerly known as [Cluster Rendering](https://docs.unity3d.com/560/Documentation/Manual/ClusterRendering.html)), that is, distributed rendering. It allows for post effects such as vignette to be properly rendered at the scale of a grid of displays.

Another use case is the implementation of Lens Distortion when replicating the distortion incurred by a physical lens. Since pixels near the edges may be pushed inwards of the viewport by the distortion, we need to render the scene with overscan. (By overscan, we mean rendering to an expanded pixel surface, later cropped out to its original size before being
presented.) To preserve the distortion's aspect in the cropped image, we need to apply a transform to screen space coordinates when the lens distortion is being computed. 

The first implementation of this technique was part of the first Cluster Display effort, and was then called **Cluster
Display Space** or **Global Screen Space**. A high level description of the technique can be found in
[this document](https://docs.google.com/document/d/1I_mR42Tt119nVhhHwHG9F_n7DwpPin7Tds4yhx59oko/edit?usp=sharing) and [this one](https://docs.google.com/document/d/1qZqUTA5VAeV35PkldjOXXjpKD-JxWfgs2KJC6rh-4mo/edit?usp=sharing).

The switch to the **Screen Coordinates Override** terminology is justified by the fact that the technique is relevant outside of
Cluster Display. As illustrated above by our Lens Distortion example.

In the changes we regularly see the introduction of 2 fields:

- `screenCoordScaleBias` holds the scale and bias used to transform screen space coordinates.
- `screenSizeOverride` holds the screen size (and its reciprocal) to be used in conjunction with transformed screen
  space coordinates.

---
### Changes Overview

* #### SRP Core
    - `ScreenCoordOverrideUtils` -> basic utilities to manage the shader keyword. Main point is to avoid the duplication of
      the shader keyword.
    - `ScreenCoordOverride.hlsl` -> shader code and macros to perform the screen space coordinates conversion.

* #### HDRP
    - `FrameSettings` -> add `ScreenCoordOverride` and `AsymmetricProjection` settings. `AsymmetricProjection` allows the
      activation of the asymmetric projection code path even when XR is not enabled. This is useful for Cluster Display.
    - `HDAdditionalCameraData` -> add the `ScreenSizeOverride` and `ScreenCoordScaleBias` fields.
    - `ShaderVariablesGlobal` -> add the `ScreenSizeOverride` and `ScreenCoordScaleBias` fields.
    - `HDCamera` -> update global shader variables.
    - `HDRenderPipeline` -> activates the shader keyword.
    - `HDRenderPipeline.PostProcess` -> activates the shader keyword based on frame settings.
    - `UberPost.compute` -> add SCO support.

* #### URP
    - `ScriptableRenderer` -> fix a swap buffer issue (first
      fixed [here](Unity-Technologies/Graphics#6522))
    - `PostProcessUtils` -> init random generation using frame count so that it's consistent across machines in a Cluster
      Display scenario.
    - `UniversalAdditionalCameraData` -> add the `ScreenSizeOverride` and `ScreenCoordScaleBias` fields and feature
      activation.
    - `UniversalRenderPipeline` -> populate additional camera data.
    - `UniversalRenderPipelineCore` -> add fields and associated shader properties.
    - `UniversalRenderer` -> related to swap buffer fix.
    - `Input.hlsl` -> add the `ScreenSizeOverride` and `ScreenCoordScaleBias` fields.
    - `FinalPost.shader` -> add SCO support.
    - `UberPost.shader` -> add SCO support.
    - `CapturePass` -> fixed color buffer access (before, post effects would not be captured)
    - `UniversalRenderPipelineGlobalSettings` -> add shader stripping option (`SerializedUniversalRenderPipelineGlobalSettings`, `UniversalRenderPipelineGlobalSettingsUI.Drawers` and `UniversalRenderPipelineGlobalSettingsUI.Skin` were modified to reflect this change)
    - `ShaderPreprocessor` -> implement shader stripping option
Error-mdl pushed a commit to StressLevelZero/Custom-RenderPipelineCore that referenced this pull request Jan 10, 2023
JIRA Epic: https://jira.unity3d.com/browse/CD-74

Re-submission of Unity-Technologies/Graphics#6597 which got reverted due to test failures.

This PR introduces **Screen Coordinates Override** (**SCO** for the remainder of this document) for both URP and HDRP. SCO
provides a transformation of screen space coordinates when evaluating screen space effects.

We have found it useful in virtual production scenarios.

The first use case for this is **Cluster Display** (formerly known as [Cluster Rendering](https://docs.unity3d.com/560/Documentation/Manual/ClusterRendering.html)), that is, distributed rendering. It allows for post effects such as vignette to be properly rendered at the scale of a grid of displays.

Another use case is the implementation of Lens Distortion when replicating the distortion incurred by a physical lens. Since pixels near the edges may be pushed inwards of the viewport by the distortion, we need to render the scene with overscan. (By overscan, we mean rendering to an expanded pixel surface, later cropped out to its original size before being
presented.) To preserve the distortion's aspect in the cropped image, we need to apply a transform to screen space coordinates when the lens distortion is being computed. 

The first implementation of this technique was part of the first Cluster Display effort, and was then called **Cluster
Display Space** or **Global Screen Space**. A high level description of the technique can be found in
[this document](https://docs.google.com/document/d/1I_mR42Tt119nVhhHwHG9F_n7DwpPin7Tds4yhx59oko/edit?usp=sharing) and [this one](https://docs.google.com/document/d/1qZqUTA5VAeV35PkldjOXXjpKD-JxWfgs2KJC6rh-4mo/edit?usp=sharing).

The switch to the **Screen Coordinates Override** terminology is justified by the fact that the technique is relevant outside of
Cluster Display. As illustrated above by our Lens Distortion example.

In the changes we regularly see the introduction of 2 fields:

- `screenCoordScaleBias` holds the scale and bias used to transform screen space coordinates.
- `screenSizeOverride` holds the screen size (and its reciprocal) to be used in conjunction with transformed screen
  space coordinates.

---
### Changes Overview

* #### SRP Core
    - `ScreenCoordOverrideUtils` -> basic utilities to manage the shader keyword. Main point is to avoid the duplication of
      the shader keyword.
    - `ScreenCoordOverride.hlsl` -> shader code and macros to perform the screen space coordinates conversion.

* #### HDRP
    - `FrameSettings` -> add `ScreenCoordOverride` and `AsymmetricProjection` settings. `AsymmetricProjection` allows the
      activation of the asymmetric projection code path even when XR is not enabled. This is useful for Cluster Display.
    - `HDAdditionalCameraData` -> add the `ScreenSizeOverride` and `ScreenCoordScaleBias` fields.
    - `ShaderVariablesGlobal` -> add the `ScreenSizeOverride` and `ScreenCoordScaleBias` fields.
    - `HDCamera` -> update global shader variables.
    - `HDRenderPipeline` -> activates the shader keyword.
    - `HDRenderPipeline.PostProcess` -> activates the shader keyword based on frame settings.
    - `UberPost.compute` -> add SCO support.

* #### URP
    - `ScriptableRenderer` -> fix a swap buffer issue (first
      fixed [here](Unity-Technologies/Graphics#6522))
    - `PostProcessUtils` -> init random generation using frame count so that it's consistent across machines in a Cluster
      Display scenario.
    - `UniversalAdditionalCameraData` -> add the `ScreenSizeOverride` and `ScreenCoordScaleBias` fields and feature
      activation.
    - `UniversalRenderPipeline` -> populate additional camera data.
    - `UniversalRenderPipelineCore` -> add fields and associated shader properties.
    - `UniversalRenderer` -> related to swap buffer fix.
    - `Input.hlsl` -> add the `ScreenSizeOverride` and `ScreenCoordScaleBias` fields.
    - `FinalPost.shader` -> add SCO support.
    - `UberPost.shader` -> add SCO support.
    - `CapturePass` -> fixed color buffer access (before, post effects would not be captured)
    - `UniversalRenderPipelineGlobalSettings` -> add shader stripping option (`SerializedUniversalRenderPipelineGlobalSettings`, `UniversalRenderPipelineGlobalSettingsUI.Drawers` and `UniversalRenderPipelineGlobalSettingsUI.Skin` were modified to reflect this change)
    - `ShaderPreprocessor` -> implement shader stripping option
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants