[Backport 10.x.x][HDRP] Enable renderer sorting for mesh decals #5674 (#5703)

* Enable renderer sorting for mesh decals #5674

* reset to default shaderproperty for draworder

* clamping

Co-authored-by: Adrien de Tocqueville <adrien.tocqueville@unity3d.com>

Author: sebastienlagarde

com.unity.render-pipelines.high-definition/CHANGELOG.md

@@ -39,6 +39,7 @@ and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.
 - Fixed shadow sampling artifact when using the spot light shadow option 'custom spot angle'
 - Fixed issue with fading in SSR applying fade factor twice, resulting in darkening of the image in the transition areas.
 - Fixed error when disabling opaque objects on a camera with MSAA.
+- Fixed sorting for mesh decals.
 
 ## [10.6.0] - 2021-04-29
 

com.unity.render-pipelines.high-definition/Documentation~/Decal-Shader.md

@@ -48,7 +48,7 @@ These properties allow you to change the rendering behavior of the decal.
 
 | **Property**               | **Description**                                              |
 | -------------------------- | ------------------------------------------------------------ |
-| **Draw Order**             | Controls the order in which HDRP draws decals in the Scene. HDRP draws decals with lower values first, so it draws decals with a higher draw order value on top of those with lower values. This feature works for decals projected on opaque and transparent surfaces.<br />**Note**: This property only applies to decals the [Decal Projector](Decal-Projector.md) creates and has no effect on Mesh decals. Additionally, if you have multiple Decal Materials with the same **Draw Order**, the order HDRP renders them in depends on the order you create the Materials. HDRP renders Decal Materials you create first before those you create later with the same **Draw Order**. |
+| **Draw Order**             | Controls the order in which HDRP draws decals in the Scene. HDRP draws decals with lower values first, so it draws decals with a higher draw order value on top of those with lower values. This feature works for decals projected on opaque and transparent surfaces. Additionally, if you have multiple Decal Materials with the same **Draw Order**, the order HDRP renders them in depends on the order you create the Materials. HDRP renders Decal Materials you create first before those you create later with the same **Draw Order**.<br/>To control the order of Mesh decals using the same Material, you should set the priority value on the MeshRenderer of the Mesh decals. |
 | **Mesh Decal Bias Type**   | Determines the type of bias that HDRP applies to the decal’s Mesh to stop it from overlapping with other Meshes. The options are:<br/>&#8226; **Depth Bias**: Applies a bias to the final depth value, <br/>&#8226; **View Bias**: Applies a world-space bias (in meters) alongside the view vector. |
 | **- Mesh Decal View Bias** | A world-space bias (in meters) that HDRP applies to the decal’s Mesh to stop it from overlapping with other Meshes along the view vector. A positive value draws the decal in front of any overlapping Mesh, while a negative value offsets the decal and draws it behind. This property only affects decal Materials directly attached to GameObjects with a Mesh Renderer, so Decal Projectors do not use this property. This property is only visible if **Mesh Decal Bias Type** is set to **View Bias**. |
 | **Mesh Decal Depth Bias**  | A depth bias that HDRP applies to the decal’s Mesh to stop it from overlapping with other Meshes. A negative value draws the decal in front of any overlapping Mesh, while a positive value offsets the decal and draw it behind. This property only affects decal Materials directly attached to GameObjects with a Mesh Renderer, so Decal Projectors do not use this property. This property is only visible if **Mesh Decal Bias Type** is set to **Depth Bias**. |

com.unity.render-pipelines.high-definition/Editor/Material/Decal/DecalUI.cs

@@ -99,6 +99,15 @@ static public void SetupCommonDecalMaterialKeywordsAndPass(Material material)
             // Set stencil state
             material.SetInt(kDecalStencilWriteMask, (int)StencilUsage.Decals);
             material.SetInt(kDecalStencilRef, (int)StencilUsage.Decals);
+
+            // Set render queue
+            var renderQueue = -1;
+            if (material.HasProperty(HDShaderIDs._DrawOrder))
+                renderQueue = (int)RenderQueue.Geometry + material.GetInt(HDShaderIDs._DrawOrder);
+            material.renderQueue = renderQueue;
+
+            // always instanced
+            material.enableInstancing = true;
         }
 
         protected const string kBaseColorMap = "_BaseColorMap";

com.unity.render-pipelines.high-definition/Editor/Material/UIBlocks/DecalSortingInputsUIBlock.cs

@@ -73,7 +73,11 @@ public override void OnGUI()
 
         void DrawSortingInputsGUI()
         {
+            EditorGUI.BeginChangeCheck();
             materialEditor.ShaderProperty(drawOrder, Styles.drawOrderText);
+            if (EditorGUI.EndChangeCheck())
+                drawOrder.floatValue = Math.Max(-HDRenderQueue.meshDecalPriorityRange, Math.Min((int)drawOrder.floatValue, HDRenderQueue.meshDecalPriorityRange));
+
             materialEditor.ShaderProperty(decalMeshBiasType, Styles.meshDecalBiasType);
             if ((int)decalMeshBiasType.floatValue == (int)DecalMeshDepthBiasType.DepthBias)
             {

com.unity.render-pipelines.high-definition/Runtime/RenderPipeline/HDRenderPipeline.cs

@@ -4009,7 +4009,7 @@ RendererListDesc PrepareMeshDecalsRendererList(CullingResults cullingResults, HD
         {
             var desc = new RendererListDesc(m_MeshDecalsPassNames, cullingResults, hdCamera.camera)
             {
-                sortingCriteria = SortingCriteria.CommonOpaque,
+                sortingCriteria = SortingCriteria.CommonOpaque | SortingCriteria.RendererPriority,
                 rendererConfiguration = PerObjectData.None,
                 renderQueueRange = HDRenderQueue.k_RenderQueue_AllOpaque
             };

com.unity.render-pipelines.high-definition/Runtime/RenderPipeline/HDRenderQueue.cs

@@ -109,6 +109,8 @@ internal static RenderQueueType MigrateRenderQueueToHDRP10(RenderQueueType rende
 
         public static int ClampsTransparentRangePriority(int value) => Math.Max(-k_TransparentPriorityQueueRange, Math.Min(value, k_TransparentPriorityQueueRange));
 
+        public const int meshDecalPriorityRange = 50; // This range is arbitrary and match sorting priority
+
         public static RenderQueueType GetTypeByRenderQueueValue(int renderQueue)
         {
             if (renderQueue == (int)Priority.Background)