Unexpectedly low performance of temporal antialiasing due to high GPU cost

[Calinou]

Godot version

4.0.alpha (b9375ea)

System information

Fedora 36, GeForce GTX 1080 (NVIDIA 510.68.02)

Issue description

In Godot, enabling TAA on a GTX 1080 and a 2560×1440 viewport takes up more than 1.2 ms of GPU time, even in an empty scene with just a Camera3D. The performance impact seems fairly constant regardless of the scene's contents:

Empty scene

TAA disabled	TAA enabled

1 BoxMesh

TAA disabled	TAA enabled

1 BoxMesh + 1 DirectionalLight3D with shadows + Default Environment

TAA disabled	TAA enabled

According to the View Frame Time panel, nearly all of the rendering cost is on the GPU. CPU frame time barely changes when TAA is enabled, at least in simple scenes. Therefore, motion vector generation isn't the bottleneck. The actual TAA shader is more likely to be a bottleneck.

Replacing the main() function's contents with just return; in the taa_resolve.glsl shader results in a black image, but still increases GPU time by 0.7 mspf compared to TAA disabled.

Renderdoc confirms that TAA is indeed taking around 1.2 ms of GPU time. (I don't have access to actual Vulkan profiling tools on this GPU, as it's too old to use Vulkan profiling on NSight.)

At lower resolutions, the performance impact of TAA is much less noticeable:

575×310

TAA disabled	TAA enabled

905×550

TAA disabled	TAA enabled

This doesn't compare favorably to the TAA implementation in other open source rendering engines. For example, in Tesseract, in a semi-complex scene with many objects and lights, the frame time difference between TAA disabled and enabled is only ~0.2 ms ((1.0/333 - 1.0/357) * 1000):

TAA disabled in Tesseract	TAA enabled in Tesseract

There are many technical differences between Godot 4 and Tesseract's rendering engines:

• Tesseract uses traditional deferred rendering, not clustered forward rendering. It has a non-PBR renderer and favors performance over flexibility.
• Tesseract uses a seemingly simpler (yet effective) form of TAA called TQAA (temporal quincunx antialiasing).
• Tesseract uses OpenGL 4.3 on supported hardware (with 3.3 as a minimum), not Vulkan. This is less efficient than Godot on paper, even though OpenGL can be pretty efficient if used well.

Still, I feel TAA should not be this expensive on the GPU in Godot.

Steps to reproduce

• Enable Perspective > View Frame Time in the top-left corner of the 3D viewport.
• Enable Use Taa in the Project Settings.

Minimal reproduction project

test_taa_performance.zip

[Zireael07]

Tesseract uses a seemingly simpler (yet effective) form of TAA called TQAA (temporal quincunx antialiasing).

Interesting. Sounds like something Godot should try, especially for GLES3 backend imho

[Calinou]

Interesting. Sounds like something Godot should try, especially for GLES3 backend imho

The OpenGL backend will probably never get TAA, as it's intended for old/low-end hardware where TAA is too expensive. A TAA implementation also adds a lot of complexity to a renderer, and I think the OpenGL renderer is best kept simple so we can focus on making it stable.

I think Godot using a forward renderer will penalize it for TAA (compared to a deferred renderer), but there are probably some optimizations we can figure out.

[mrjustaguy]

To my Knowledge, Forward vs Deferred rendering should have No impact on the TAA Costs.

[Calinou]

JFonS said that the high TAA cost is expected due to how it works currently. It's a separate pass that requires a full-screen copy, which is expensive in itself. I suppose avoiding this copy could halve the GPU cost of TAA, if not more.

To make TAA cheaper, it should avoid performing this copy. For instance, this can be done by moving TAA to the tonemapping shader, but doing so will break FXAA and glow (so they won't be usable at the same time as TAA). Another solution needs to be found – let us know if you can think of one 🙂

[Calinou]

We discussed this issue in today's rendering meeting and concluded on possible optimizations:

• Do not run the motion vector generation for objects that haven't moved since the last frame.
• Generate motion vectors in screen-space for what's in view.

[mrjustaguy]

I've been trying to wrap my head around the TAA to tonemap migration for a while, and I don't get it.. Why would that break FXAA and glow? I mean the TAA step could be done before the two, and you'd apply those to the TAA result... Right?

I mean based on the post above I'm guessing currently TAA works something like scenario 1:

1. Creates copy of Full Screen
2. Does TAA
3. Passes result to Tonemapper
4. Tonemap applies Glow/FXAA if enabled

or less likely, scenario 2 like:

1. Tonemapper applies Glow/FXAA if enabled
2. TAA Creates copy of Full Screen based on Tonemapper output
3. Does TAA

Whereas TAA in Tonemapper would work something like this:

1. Does TAA
2. Modifies output with Glow/FXAA after the TAA step is done

The main differences are that FXAA/Glow would possibly change from being applied before TAA to after TAA, which for Glow wouldn't change things much, but for FXAA could result in visible differences, as it'd look kind of like applying a filter to TAA, whereas if FXAA image is getting TAA it'd smear it a little