[Question] Accessing Audio Output Data for Visualization?

[JamesMoreau]

Hi, I’m working on a music player using rodio and would like to implement a real-time audio visualizer, akin to what is seen in WinAmp. To achieve this, I need access to the audio data being played.

Is there a way to access the audio samples rodio sends to the output device?

If not, what’s the easiest approach to achieve this while still using rodio for playback?

Thanks.

[yara-blue]

Yes but you might need to do a little more work then usual since you can not use Sink. Instead you want to use rodio's audio pipeline directly and add a custom component for visualization.

Here are some helpful examples for using rodio's pipeline directly:

• how to use periodic access to do something (control playback/volume etc) once in a while, you will need this to re-create Sink: https://github.com/RustAudio/rodio/blob/master/examples/automatic_gain_control.rs
• use the mixer directly without Sink: https://github.com/RustAudio/rodio/blob/master/examples/custom_config.rs

In addition to that you will use inner_mut() within the periodic_callback() closure to get to to different components in the pipeline. See this part of the Sink source as an example for that.

Regarding the audio visualization, you probably need to do something with every sample for that. You need to add a custom Source for that. It will wrap any other source (see amplify as example). In the iterator implementation for the source you can the freely do something with the samples as long as its fast. You might want to send the samples over to another thread and make the visualization there.

Good luck!

[yara-blue]

I am going to assume it worked out. If it didn't do reopen the issue and show us a minimal example (maybe in a separate repo?). Then we can work on that together.

[JamesMoreau]

Hey

So, I still want to use the Sink api for playing, pausing, volume control, seeking, etc. Are you suggesting I clone the source code to add my own functionality?

The other part about wrapping the original audio with a new source to do work on each sample makes sense to me. However, is there a reason I cannot just send the data over to another thread immediately inside .next() instead of using the .periodic_access() method? Is it because there would be too much of a performance cost to processing data potentially thousands of times per second?

Also, I have created a minimal example here:

https://github.com/JamesMoreau/audio-visualizer

Cheers.

[yara-blue]

Is it because there would be too much of a performance cost to processing data potentially thousands of times per second?

it could be less efficient, it depends on the compiler and if its smart enough to optimize periodic access. It also depends on what you are visualizing, if its just volume a once every x millisecond sample is enough. If however you want to do an FFT to show frequencies you need to 'sample' the source at the nyquest fequency.

Therefore my advise now is to begin with sending everything from Source::next and later optimizing that. One optimization you could do is buffer what you want to send that way you are not calling mpsc::send so often.

So, I still want to use the Sink api for playing, pausing, volume control, seeking, etc. Are you suggesting I clone the source code to add my own functionality?

I was, however on second thought it should be enough to just wrap every source with your Visualizer before adding it to the Sink.