torchaudio support

[williamlzw]

Are there any plans to support torchaudio, such as StreamReader and StreamWriter classes

[yueyinqiu]

Personally I think it won't be supported in the near future. NAudio might be a possible workaround?

[GeorgeS2019]

https://github.com/dotnet/TorchSharp/tree/main/src/TorchAudio

I think we need to figure out how to connect NAudio to TorchSharp TorchAudio Audio I/O. There are .NET user controls for variety of spectrograms

• Audio I/O

TorchAudio feasibility

#598

• Audio I/O
• Audio Resampling
• Audio Data Augmentation
• Audio Feature Extractions
• Audio Feature Augmentation
• Audio Datasets
• Speech Recognition with Wav2Vec2
• Speech Command Classification with torchaudio
• Text-to-speech with torchaudio
• Text-to-speech with torchaudio: tacotron2

[GeorgeS2019]

https://github.com/dotnet/TorchSharp/blob/main/test/TorchSharpTest/TestTorchAudio.cs

[yueyinqiu]

By the way, NAudio uses windows media player to support some audio formats, such as flac, which is not cross platform. (And even on Windows, some problems may happen, like the mis-calculation of audio length.) I think we need to deal with this carefully.

And @GeorgeS2019, it's really nice to find that you are still here. But it seems that @NiklasGustafsson isn't that active recently. Do you know what's going on?

[NiklasGustafsson]

But it seems that @NiklasGustafsson isn't that active recently. Do you know what's going on?

I am looking at the list every day...

There hasn't been a lot of issue activity at all since the release of v0.103.0, I've had some family things to deal with (a child's wedding, specifically), and I've had other projects to work on. TorchSharp is not my full-time job at Microsoft. That's what's going on. :-)

[yueyinqiu]

Wow! Congratulations!

[GeorgeS2019]

@NiklasGustafsson
@yueyinqiu

Many people are now going to be very curious about Torchsharp for Reinforcement Learning

dotnet/machinelearning#181 (comment)

https://youtu.be/Aoyj7qzyung

[williamlzw]

I use NAudio to load wav and resample.

using NAudio.Wave.SampleProviders;
using NAudio.Wave;

namespace HelpersTool
{
    public static class Helpers
    {
        public static byte[] ConvertFloatToByte(float[] floatBuffer)
        {
            var byteBuffer = new byte[floatBuffer.Length * 4];
            Buffer.BlockCopy(floatBuffer, 0, byteBuffer, 0, byteBuffer.Length);
            return byteBuffer;
        }

        public static (float[] Samples, int SampleRate, int channels) ReadWav(string filePath)
        {
            using var reader = new WaveFileReader(filePath);
            var bytes = new byte[reader.Length];
            reader.Read(bytes, 0, bytes.Length);
            float[] samples;
            int sampleRate = reader.WaveFormat.SampleRate;
            var channels = reader.WaveFormat.Channels;
            if (reader.WaveFormat.BitsPerSample == 16)
            {
                samples = new float[bytes.Length / 2];
                for (int i = 0; i < samples.Length; i++)
                {
                    short int16 = BitConverter.ToInt16(bytes, i * 2);
                    samples[i] = int16 / 32768f;
                }
            }
            else if (reader.WaveFormat.BitsPerSample == 32 && reader.WaveFormat.Encoding == WaveFormatEncoding.IeeeFloat)
            {
                samples = new float[bytes.Length / 4];
                Buffer.BlockCopy(bytes, 0, samples, 0, bytes.Length);
            }
            else
            {
                throw new NotSupportedException("Unsupported WAV format.");
            }
            return (samples, sampleRate, channels);
        }

        public static float[] ResampleAudio(float[] originalSamples, int originalSampleRate, int targetSampleRate, int channels = 1)
        {
            var originalFormat = WaveFormat.CreateIeeeFloatWaveFormat(originalSampleRate, channels);
            var provider = new RawSourceWaveStream(
                new MemoryStream(SenseVoiceHelpers.ConvertFloatToByte(originalSamples)),
                originalFormat
            ).ToSampleProvider();
            var resampler = new WdlResamplingSampleProvider(provider, targetSampleRate);
            var targetLength = (int)(originalSamples.Length * (double)targetSampleRate / originalSampleRate);
            var resampled = new float[targetLength];
            int samplesRead = resampler.Read(resampled, 0, resampled.Length);
            return resampled.Take(samplesRead).ToArray();
        }

    }
}

var (samples, sampleRate, channels) = Helpers.ReadWav(audioPath);
if (sampleRate != 16000)
{
    samples = Helpers.ResampleAudio(
        originalSamples: samples,
        originalSampleRate: sampleRate,
        targetSampleRate: 16000,
        channels: channels
    );
}

[yueyinqiu]

#1453