Performance investigations on RDNA2 cards.

[Azralee]

I ran the Raytracer through AMDs GPU profiler to check out how it runs on my RX6800. (I will upload the results in a different issue.)
It reported that the RT shader is limited by it's LDS usage of 4 KB to half occupancy, and that it uses 80 vector registers.
Decreasing LDS usage to 3072B could increase the occupancy up to 12 parallel wavefronts (warps on nVidia Hardware) the maximum for 80 Vector registers. This should improve performance as less time is spent idle.

Reducing LDS usage to 2048B would allow further optimizations to VGPR (Vector General Purpose Register) usage. Reducing VGPRS to 64 would allow full occupancy and presumably maximum performance.

Edit: I believe LDS is AMDs name for Workgroup Memory.

[Azralee]

This is way more complicated than the original comment stated. I need to study this more.

[GPSnoopy]

Yes, I think that it might be specific to AMD as well as they use the normal shader resources for BVH tranversals. Register usage might be different on other cards.

I've tried to use NVIDIA Nsight Graphics 2021.1.0 for profiling the Vulkan Ray Tracing shaders in Scene 1 last night. It hinted that the main bottleneck was L1TEX ( I assume L1 cache) and MIO (Memory IO) but the feature is Beta and light on information, it doesn't seem to correlate yet with the source code (even SPIR-V bytecode) in all but the simplest case, so it's hard to identify the actual location of bottlenecks in the shaders.

[Azralee]

Yeah, I tried to use AMD's static Analyzer (RGA) yesterday, but they have not released an update with the latest SPIR-V tools.

AMD's Vulkan traversal code seems under-optimized at the moment, especially in comparison with the DX12 one.
I have attached some traces. Metro Exodus seems pretty much like the most optimized RT on AMD that I found until now. And even Control (which runs awfull) seems to use half the LDS of the Vulkan examples.

Trace from this Project:

Trace from Quake2 RTX:

Trace from Metro Exodus (DX12):

Trace from Control (Corridor of DOOM, as Digital Foundry puts it) (RT passes: BVH, reflections, diffuse Reflections, GI, Contact shadows):

[Azralee]

I think I figured the performance difference out.

If you take a look at the heatmap, you can see that the heatmap is relatively coarse, specifically it's batches of 8x8 pixels, or 64 pixels. This means that the wavefront runs until the last ray has terminated. Combined with the currently relatively low occupancy, this could explain the odd results.

[Azralee]

It could probably be faster to trace a single sample per shader and dispatch a bunch of samples per pixel at the same time, no idea if it's possible though.

[Azralee]

This hypothesis seems convincing. I ran the raytracer at 5120x2880 and at 1280x720:


The 64-pixel blocks are very visible in 720p and cover a lot ot the central statue, while in 2880p the hot blocks less widely distributed. This looks much closer to the nVidia heatmap in README.md and corresponds more to the naively expected behaviour.

To test if it improves performance, I ran the benchmark at identical Ray counts, (2880p has 16x the pixels of 720p), at different resolutions (60s every scene):

Resolution\Scene	Rays per frame	Scene 1	Scene 2	Scene 3	Scene 4	Scene 5
720p 16 Samples	14,745,600	70.02	69.37	30.60	55.71	20.23
2880p 1 Sample	14,745,600	71.67	70.76	35.51	54.79	20.33
Performance of higher res	100%	102,36%	102%	116,05%	98,35%	100,5%

The performance is highly scene dependent, with most scenes being more or less within the margin of error. Scene 3 with the Lucy statues is improved by quite a lot, just as expected from the heatmaps.

[Azralee]

The bad performance in the Cornell box scenes make a lot of sense, since there are only very few triangles (36 if I count correctly) making up the scene.
Ampere improved twofold on Ray-Triangle-Intersections, which is basically all that happens in the Cornell Box. On real models, like Lucy, the BVH-hirarchy should be much deeper. If we assume a BVH4 (like the RDNA2 ISA-Guide implies), then the 448K triangles of the Lucy Statues should be in a BVH of more than 10 Layers deep each. The Cornell Box on the other hand should be in a BVH Box about 5 layers total.
The Cornell box is a micro benchmark of Ray-Triangle performance, so it's no wonder that Ampere runs so fast here.

[Azralee]

I did some more testing and found a way to force wave32 execution on the RT shaders on RDNA2. This improved performance by ~6%. Moving line 101 directly behind the load in line 85, reduced the amount of used vector registers and increased performance further:
FPS table:

execution mode	Scene 1	Scene 2	Scene 3	Scene 4	Scene 5
wave32 optimized	48.78	48.14	23.12	36.26	13.83
wave32	43.67	43.09	20.13	33.09	12.04
wave64 (base)	40.99	40.66	18.94	31.59	11.29

Improvements over base:

execution mode	Scene 1	Scene 2	Scene 3	Scene 4	Scene 5
wave32 optimized	19.00%	18.40%	22.07%	14.78%	22.50%
wave32	6.54%	5.98%	6.28%	4.75%	6.64%
wave64 (base)	0.00%	0.00%	0.00%	0.00%	0.00%

Unfortunately there is currently no way to indicate a preference for wave32, outside the compiler heuristics.

I will open a pull request later.