Add DLSS 2.0 support

Describe the project you are working on

I am working on a 3D Horror Game World with 2K textures

Describe the problem or limitation you are having in your project

As the GPU Market tends to shift to AI up- and downscaling, games will tend to have GLSS in the future more likely. So first of all why shouldn't Godot have it, but everyone else? 2. It gives pretty much every game a major performance boost. 3. Even AMD will soon announce their FidelityFX driver, which will have their so called Super Resolution.

@ https://techengage.com/best-gaming-graphics-cards-gpus/

Describe the feature / enhancement and how it helps to overcome the problem or limitation

DLSS helps games with bad performance on higher settings in an immense way. It would overcome some of the limitations Developers get with considering the Hardware of Players.

Describe how your proposal will work, with code, pseudo-code, mock-ups, and/or diagrams

I personally don't know. NVIDIA for example has a DLSS beta program or you need to consultate a NVIDIA Developer to talk it out.

If this enhancement will not be used often, can it be worked around with a few lines of script?

I don't think it can be worked around as it is from the companys themselves

Is there a reason why this should be core and not an add-on in the asset library?

You could see DLSS as an Rendering enhancement and I don't think changing the rendering as an add-on is such a good idea. I don't know how big the program itself is, so if it is too big for Godot (in terms of Volume Space) then it could be added the same way as the Android Build Template.

Thanks for reading :)

[volzhs]

accroding to wikipedia

DLSS 2.0 works as follows:[14]

• The neural network is trained by Nvidia using "ideal" images of video games of ultra-high resolution on supercomputers and low resolution images of the same games. The result is stored on the video card driver. It is said that Nvidia uses DGX-1 servers to perform the training of the network.[15]
• The Neural Network stored on the driver compares the actual low resolution image with the reference and produce a full high resolution result. The inputs used by the trained Neural Network are the low resolution aliased images rendered by the game engine, and the low resolution, motion vectors from the same images, also generated by the game engine. The motion vectors tell the network which direction objects in the scene are moving from frame to frame, in order to estimate what the next frame will look like.[16]

[mrjustaguy]

DLSS 2.0 Cannot be added to the game engine (or any game engine really).. It has to be implemented on a Game by Game basis, and is not general enough to work on a Game engine basis. DLSS 3.0 or further maybe when the AI gets more general, or efficient enough that the AI training for the game can take place on local hardware instead of needing very powerful Servers to do so.

DLSS 2.0 Cannot be added to the game engine (or any game engine really).. It has to be implemented on a Game by Game basis, and is not general enough to work on a Game engine basis. DLSS 3.0 or further maybe when the AI gets more general, or efficient enough that the AI training for the game can take place on local hardware instead of needing very powerful Servers to do so.

I am sorry to say but DLSS 2.0 is not per game basis. It's per Engine. See: https://developer.nvidia.com/dlss

[mrjustaguy]

Ok my bad, I was under the impression that DLSS 2.0 was still not general enough and that it too needed a per-game training.

[Calinou]

DLSS is a proprietary library; we can't integrate it officially in Godot. Since this feature must be implemented as a third-party effort, I'll close this as godot-proposals is only meant to discuss features to be added to core Godot.

Even AMD will soon announce their FidelityFX driver, which will have their so called Super Resolution.

When (and only when) AMD makes this solution public, it should be discussed in a separate proposal. We'll need to see if it's actually open source-friendly, because GPUOpen solutions have historically not always been open source (see RadeonRays 2.0 vs 4.0).

[mahdisml]

DLSS is a proprietary library; we can't integrate it officially in Godot. Since this feature must be implemented as a third-party effort, I'll close this as godot-proposals is only meant to discuss features to be added to core Godot.

Even AMD will soon announce their FidelityFX driver, which will have their so called Super Resolution.

When (and only when) AMD makes this solution public, it should be discussed in a separate proposal. We'll need to see if it's actually open source-friendly, because GPUOpen solutions have historically not always been open source (see RadeonRays 2.0 vs 4.0).

#2809

[tonymacx86PRO]

Add DLSS now! Its already open source https://github.com/NVIDIA/DLSS

[Calinou]

Add DLSS now! Its already open source NVIDIA/DLSS

DLSS is still under a proprietary EULA: https://github.com/NVIDIA/DLSS/blob/main/LICENSE.txt
Its full source code is also not publicly available – you have to dynamically link against a proprietary DLL to implement DLSS in an engine. (The DLSS 3 SDK is also not publicly released yet as of writing.)

Only NIS (NVIDIA's spatial upscaler, equivalent to FSR 1.0) is open source. Since FSR 1.0 is already present in 4.0.beta, I don't see much point in adding NIS. In general, FSR 1.0 is considered to look slightly better in practice while also being faster. NIS goes for a more conservative approach but ends up being more expensive, making it a less "bang for buck" spatial upscaler.

There are already plans to integrate FSR 2.0 in future 4.x releases.

Edit: FSR2 will be available in Godot 4.2: godotengine/godot#81197

[MikeZTM]

Add DLSS now! Its already open source NVIDIA/DLSS

DLSS is still under a proprietary EULA: https://github.com/NVIDIA/DLSS/blob/main/LICENSE.txt Its full source code is also not publicly available – you have to dynamically link against a proprietary DLL to implement DLSS in an engine. (The DLSS 3 SDK is also not publicly released yet as of writing.)

Only NIS (NVIDIA's spatial upscaler, equivalent to FSR 1.0) is open source. Since FSR 1.0 is already present in 4.0.beta, I don't see much point in adding NIS. In general, FSR 1.0 is considered to look slightly better in practice while also being faster. NIS goes for a more conservative approach but ends up being more expensive, making it a less "bang for buck" spatial upscaler.

There are already plans to integrate FSR 2.0 in future 4.x releases.

https://github.com/NVIDIAGameWorks/Streamline

They have an open-source wrapper for both XeSS and DLSS and it's MIT licensed so should be compatible.
I think implementing this is benefiting a lot of users as it's quite inefficient to run FSR2 on NVIDIA and Intel GPUs. Those card are much more capable of much better image quality and potentially lower latency. Starfield already generating a lot of noise on that front and it's pretty clear future game developer will facing same request from their PC audience.

Obviously, developer will still need to put proprietary blobs to actually enable those features, but this makes it possible to have almost ready-to-use support and keep blobs out of the engine.