Implement an advanced parametrizable audio engine

[IntangibleMatter]

Describe the project you are working on

Various games with complex audio needs that also need to be built for web.

Describe the problem or limitation you are having in your project

Godot has no built-in way to make parametrizable audio. You either have to program a system yourself on top of Godot's (admittedly powerful) existing audio engine, or use middleware like FMOD or Wwise. These programs tend to be rather heavy, and frequently are less portable than Godot, meaning you may not be able to build for web or other things.

Describe the feature / enhancement and how it helps to overcome the problem or limitation

Add a dedicated tab/dock for designing sounds to be used in Godot. I believe a dedicated interface would be the optimal solution for something like this, as the default resource editor would be clunky when editing audio. This would be a system which exists to create advanced, parametrizable audio.

It would at least need to do the following:

1. Allow easy creation of sounds that can be modified with parameters.
2. Make it easy for audio designers to add/modify tracks to sounds.
3. Have a system which allows modifying the timing of sounds, and defining how they overlap.
4. Allow for defining loops and ways to jump around on a track.

Describe how your proposal will work, with code, pseudo-code, mock-ups, and/or diagrams

There are two primary ways I see an audio editor like this being implemented:

1. With a dedicated track interface
2. With a node graph, akin to a VisualShader

I think the ideal way to implement it would be the former, and probably with a dock. I personally think a whole tab would be amazing, but the fact is that there are enough people who don't need a dynamic audio engine that it may not deserve a place with the script, scene, and asset library. (Though I could argue that it would probably get more use than that last one :p)

Here's a very rough mockup of what it could look like.

The way that makes the most sense to me is to have these be a new resource type, something along the lines of AudioStreamDynamic. This would mean it would be easy to use them interchangeably with existing AudioStream types.

AudioStreamDynamics would have a Dictionary with their parameters, and whatever additional info may be helpful (like if the parameter has a seek speed, etc.) To allow for global parameters, something new could be added to AudioServer, along the lines of get_global_parameter(), set_global_parameter(), and add_global_parameter(). I think optimally one would be able to add global parameters via the audio editor. Although it's also possible that it would make more sense to keep parameters that would need to be accessed by multiple AudioStreamDynamics within a certain scope.

Again, this isn't necessarily the solution to this, it would just make a lot of sense for there to be a dedicated Audio editor, allowing for more complex audio to be created more easily. This is mostly about creating an editor to access these tools, as Godot's existing Audio Engine has a lot of power. It's just... stupid hard to access.

If this enhancement will not be used often, can it be worked around with a few lines of script?

I feel like enough game developers would use dynamic audio if given the chance more readily that it's worth the trade-off of doing this as a few thousand lines of C++ versus creating it as GDscript spaghetti. It's a rather complicated UI element that would need integration through multiple different parts of the engine, and as such is rather unwieldy to implement at a higher level

Is there a reason why this should be core and not an add-on in the asset library?

Godot lacks several key pieces of infrastructure needed for this, such as a better track layout system. Additionally, it would need to be performant, and a large part of this proposal is ensuring portability, as such it would make sense for it to be core.

[Calinou]

Thanks for the proposal! Consolidating in #3394.