Stencil Op is hardcoded to DONT_CARE

[ishitatsuyuki]

rendy/graph/src/node/render/pass.rs

Line 542 in 2d5a4a1

stencil_ops: gfx_hal::pass::AttachmentOps::DONT_CARE,

Stencil isn't enabled by default in Amethyst, but well, this causes trouble (actually nondeterminism) when it's actually used. Only AMD drivers seem to emit checkboard patterns.

[zakarumych]

Correct me if I'm wrong. Someone tries to use stencil and stencil buffer contains checkboard?

[ishitatsuyuki]

Some drivers will intentionally give you some random pattern when you read uninitialized (or undefined) memory, for the purpose of catching bugs.

[zakarumych]

Well, you probably shouldn't read from uninitialized memory :)
If DONT_CARE is hardcoded then using stencil buffer across passes is not supported.
When pass implementation was written originally stencil buffer was put out of scope for the time.

[ishitatsuyuki]

You keep missing the point, and therefore your statements like “you probably shouldn't read from uninitialized memory” are completely out of context.

Also I think you don’t understand how loadOp is supposed to work. loadOp is performed for the first surpass that uses the resource, and afterwards other subpasses can access the buffer normally. Therefore it’s not any unsupported thing.

Back to the topic, the rendy clear settings is supposed to be applied with VK_ATTACHMENT_LOAD_OP_CLEAR and thus only works when it's set. That's what this issue is about.

[zakarumych]

I guess that I have some knowledge about how it works ;)
But I probably miss your point. As I still don't get what your issue is.
My points are:

1. Stencil buffer usage was put out of scope during RenderPassNode original implementation.
2. Yes. Stencil buffer is important and it should be supported by RenderPassNode. This is tech debt.
3. loadOp = DONT_CARE means that stencil buffer value won't be loaded or cleared at the beginning of pass execution so any reads that happens before writes in earlier subpass yield undefined content.

So, if I'm not miss anything, you're trying to set clear value for stencil buffer and expect it from read operations, right?
Then clear value should consist of two Options for both depth and stencil and then choose between Load and Clear depending on it.

[ishitatsuyuki]

So, if I'm not miss anything, you're trying to set clear value for stencil buffer and expect it from read operations, right?

This is correct.

Then clear value should consist of two Options for both depth and stencil and then choose between Load and Clear depending on it.

Yeah, this is probably one of a way to properly expose it.

I don't know well about implementations, but I have read NVIDIA's tips saying that you should avoid clearing depth or stencil individually (because that triggers the slow path). I'm not sure if it applies to loadOp too.

[zakarumych]

Interesting. What about AMD?
I guess slow path is triggered when you Load one and Clear another.

[ishitatsuyuki]

AMD does not support the packed format (D24S8) at all so I suspect they have separate storages for the two.