Transparency pipeline state bugs in chrono_vsg ShaderUtils.cpp

[Tudor-MS]

Target repo: projectchrono/chrono
Component: chrono_vsg — VSG/Vulkan visualization module
File: src/chrono_vsg/shapes/ShaderUtils.cpp, function createPbrStateGroup(), lines 433–453
Affected versions: Confirmed on 9.0.1; likely present since VSG backend introduction
Severity: Visual — all three bugs are rendering-only, no simulation impact

---

Summary

The SetPipelineStates visitor inside createPbrStateGroup() has three transparency-related pipeline configuration bugs that affect any application rendering semi-transparent Chrono visual materials through the VSG backend.

Observed symptoms

1. One-sided transparency — semi-transparent objects (e.g. tires with opacity < 1.0) appear fully opaque from one viewing direction because backface culling is still active on the transparent pipeline.
2. Geometry hidden behind transparent surfaces — objects positioned behind a transparent surface disappear entirely, because the transparent surface writes to the depth buffer and occludes them.
3. Opaque objects rendered double-sided — all opaque geometry is rendered without backface culling, wasting fill rate and occasionally producing z-fighting artifacts on thin-walled meshes.

Root cause analysis

The SetPipelineStates visitor (lines 433–453) contains a logic inversion and a missing state override:

struct SetPipelineStates : public vsg::Visitor {
    bool wireframe;
    bool blending;
    SetPipelineStates(bool inWire, bool inBlend) : wireframe(inWire), blending(inBlend) {}

    void apply(vsg::Object& object) { object.traverse(*this); }
    void apply(vsg::RasterizationState& rs) {
        if (!blending) {                          // <-- BUG 1: condition is inverted
            // combination of color blending and two sided lighting leads to strange effects
            rs.cullMode = VK_CULL_MODE_NONE;      //     disables culling for OPAQUE objects
        }
        if (wireframe)
            rs.polygonMode = VK_POLYGON_MODE_LINE;
        else
            rs.polygonMode = VK_POLYGON_MODE_FILL;
    }
    void apply(vsg::InputAssemblyState& ias) {
        // if (wireframe) ias.topology = VK_POLYGON_MODE_LINE;
    }
    void apply(vsg::ColorBlendState& cbs) { cbs.configureAttachments(blending); }
    //                                        ^^^ BUG 2: no DepthStencilState override
} sps(wireframe, use_blending);

Bug 1 — Inverted cull-mode condition

The comment on line 441 reads "combination of color blending and two sided lighting leads to strange effects", indicating the author's intent was to disable culling when blending is active (transparent objects need to render both faces). However, the condition if (!blending) does the opposite — it disables culling for opaque objects and leaves backface culling enabled for transparent ones.

Fix: Change if (!blending) to if (blending).

Bug 2 — Missing depth-write disable for transparent pipelines

The visitor does not override apply(vsg::DepthStencilState&). Transparent objects therefore inherit the default depthWriteEnable = VK_TRUE, causing them to write into the depth buffer. Any geometry behind a transparent surface fails the depth test and is not drawn — defeating the purpose of transparency.

Fix: Add a DepthStencilState override that sets depthWriteEnable = VK_FALSE when blending == true.

Bug 3 (enhancement) — No depth-sorted rendering for transparent objects

Transparent objects are not wrapped in vsg::DepthSorted bins, so they render in arbitrary submission order. This causes order-dependent blending artifacts when multiple transparent objects overlap. This is a longer-term enhancement rather than a bug fix.

Proposed fix

struct SetPipelineStates : public vsg::Visitor {
    bool wireframe;
    bool blending;
    SetPipelineStates(bool inWire, bool inBlend) : wireframe(inWire), blending(inBlend) {}

    void apply(vsg::Object& object) { object.traverse(*this); }

    void apply(vsg::RasterizationState& rs) {
        if (blending) {
            // Transparent objects must render both faces
            rs.cullMode = VK_CULL_MODE_NONE;
        }
        if (wireframe)
            rs.polygonMode = VK_POLYGON_MODE_LINE;
        else
            rs.polygonMode = VK_POLYGON_MODE_FILL;
    }

    void apply(vsg::DepthStencilState& dss) {
        if (blending) {
            // Transparent surfaces must not write to depth buffer,
            // otherwise geometry behind them is occluded
            dss.depthWriteEnable = VK_FALSE;
        }
    }

    void apply(vsg::InputAssemblyState& ias) {
        // if (wireframe) ias.topology = VK_POLYGON_MODE_LINE;
    }

    void apply(vsg::ColorBlendState& cbs) { cbs.configureAttachments(blending); }
} sps(wireframe, use_blending);

What changes

Pipeline state	Before (buggy)	After (fixed)
Opaque RasterizationState::cullMode	VK_CULL_MODE_NONE	VK_CULL_MODE_BACK_BIT (VSG default — no longer overridden)
Transparent RasterizationState::cullMode	VK_CULL_MODE_BACK_BIT (default, never set)	VK_CULL_MODE_NONE
Transparent DepthStencilState::depthWriteEnable	VK_TRUE (default, never set)	VK_FALSE

What does NOT change

• Wireframe mode logic (unrelated)
• ColorBlendState::configureAttachments() call (correct as-is)
• Opaque depth behavior (correct as-is)
• Shader code (not involved)

Reproduction

Any Chrono VSG demo that creates a body with a semi-transparent ChVisualMaterial (opacity < 1.0) will exhibit these symptoms. For example:

auto mat = chrono_types::make_shared<ChVisualMaterial>();
mat->SetDiffuseColor({0.2f, 0.2f, 0.2f});
mat->SetOpacity(0.35f);  // 35% opaque
// ... assign to a body's visual shape

Then rotate the camera around the object:

• The object will appear opaque from one side (backface culled)
• Any geometry behind it will be invisible (depth buffer occlusion)
• Nearby opaque objects will show occasional z-fighting (double-sided rendering)

Environment

• Chrono: 9.0.1 (built from source)
• VSG: 1.1.x
• Vulkan: 1.3 (NVIDIA)
• Platform: Windows 11, MSVC 2022

Workaround

Until the fix is merged, downstream applications can apply a post-BindAll() scene graph traversal that locates BindGraphicsPipeline nodes, inspects ColorBlendState attachments for blendEnable == VK_TRUE, and patches the DepthStencilState and RasterizationState accordingly. This workaround is functional but adds an unnecessary traversal that would be eliminated by fixing the source.