Skip to content

1.1 #7

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 32 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
32 commits
Select commit Hold shift + click to select a range
381051e
delete v1.0 files
swharden May 27, 2020
2741fd1
new sample audio
swharden May 27, 2020
8c398eb
Update README.md
swharden May 27, 2020
d60aa59
create new projects
swharden May 27, 2020
1dfa5d2
Create IColormap.cs
swharden May 28, 2020
3ec6a0b
Create Viridis.cs
swharden May 28, 2020
435f096
file reading class
swharden May 28, 2020
758d2ce
functional spectrogram
swharden May 28, 2020
9563537
Update README.md
swharden May 28, 2020
6501110
demo spectrogram
swharden May 28, 2020
fa3140d
refactor to improve intensity calculation
swharden May 28, 2020
f2cc0f0
add song to front page
swharden May 28, 2020
1bf53c9
support frequency limiting
swharden May 28, 2020
f71535a
calculate pixel values in FFT loop
swharden May 28, 2020
35901bb
refactor buffer limiting system
swharden May 28, 2020
3ba5fb2
refactor to Spectrogram and SpectrogramLive
swharden May 29, 2020
7be8a96
start investigator app
swharden May 29, 2020
4ac06f0
FFT investigator fully functional
swharden May 29, 2020
4480bb4
refactor microphone demo
swharden May 29, 2020
75bf197
improve colormap support
swharden May 30, 2020
102dc6c
improve quickstart
swharden May 30, 2020
3b42d3e
refactor
swharden Jun 2, 2020
5a42076
argonot
swharden Jun 2, 2020
973927f
refactor
swharden Jun 2, 2020
c438ce5
use updated API
swharden Jun 2, 2020
957728b
only save if checked
swharden Jun 2, 2020
c26eef7
axis ticks
swharden Jun 3, 2020
c8d8bfc
update argonot
swharden Jun 4, 2020
3b4cdec
refactor tick system
swharden Jun 4, 2020
960876d
refactor wspr labling
swharden Jun 4, 2020
f596f4e
update argonot
swharden Jun 6, 2020
6072610
experimentation
swharden Jun 6, 2020
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
109 changes: 48 additions & 61 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,82 +1,74 @@
# Spectrogram
**Spectrogram** is a .NET library which makes it easy to create spectrograms from pre-recorded signals or live audio from the sound card. This library supports .NET Framework (4.5) and .NET Core (3.0) and can be installed using [NuGet](https://www.nuget.org/packages/Spectrogram/).
**Spectrogram** is a .NET library for creating spectrograms from pre-recorded signals or live audio from the sound card. Spectrogram uses FFT algorithms and window functions provided by the [FftSharp](https://github.com/swharden/FftSharp) project, and it targets .NET Standard 2.0 so it can be used in .NET Framework and .NET Core projects.

<div align="center">

![](dev/spectrogram.png)

_"I'm sorry Dave... I'm afraid I can't do that"_

</div>

![](data/mozart.jpg)

## Quickstart

### Song to Spectrogram
The code below converts a WAV file to a spectrograph and saves it as an image. This code analyzed [Mozart's Piano Sonata No. 11 in A major](https://www.youtube.com/watch?v=aeEmGvm7kDk) to produce the picture above.
### Install

```cs
// load audio and process FFT
var spec = new Spectrogram.Spectrogram(sampleRate: 8000, fftSize: 2048, step: 700);
float[] values = Spectrogram.Tools.ReadWav("mozart.wav");
spec.AddExtend(values);

// convert FFT to an image and save it
Bitmap bmp = spec.GetBitmap(intensity: 2, freqHigh: 2500);
spec.SaveBitmap(bmp, "mozart.jpg");
```
Spectrogram can be installed with NuGet:\
https://www.nuget.org/packages/Spectrogram

### Human Voice
This code analyzes audio from HAL's famous quote, "I'm sorry Dave, I'm afraid I can't do that". The output is can be rendered using different colormaps.
### Create a Spectrogram

This code in [Program.cs](src/Spectrogram.Quickstart/Program.cs) was used to create the above image:
```cs
// load audio and process FFT
var spec = new Spectrogram.Spectrogram(sampleRate: 15000, fftSize: 4096, step: 400);
float[] values = Spectrogram.Tools.ReadMp3("cant-do-that.mp3");
spec.AddExtend(values);

// convert FFT to an image and save it
Bitmap bmp = spec.GetBitmap(intensity: .2, freqHigh: 1000,
colormap: Spectrogram.Colormap.grayscaleInverted);
spec.SaveBitmap(bmp, "cant-do-that-grayscale-inverted.jpg");
```
colormap | sample output
---|---
**Grayscale Inverted** is used in many scientific publications when analyzing things like human voices and bird sounds|![](/data/cant-do-that-grayscale-inverted.jpg)
**Grayscale** provides highest contrast output but does not benefit from color vision|![](/data/cant-do-that-grayscale.jpg)
**Viridis** is the default colormap. It was specifically designed to represent 2D data in a way [ideally suited for human vision](https://www.youtube.com/watch?v=xAoljeRJ3lU).|![](/data/cant-do-that.jpg)
**vdGreen** is the default colormap used for [QRSS-VD](https://github.com/swharden/QRSS-VD), a very old software project of mine. |![](/data/cant-do-that-green.jpg)
(double[] audio, int sampleRate) = Read.MP3("cant-do-that.mp3");

### QRSS Analysis
var spec = new Spectrogram(
signal: audio,
sampleRate: sampleRate,
fftSize: 4096,
stepSize: 500,
freqMax: 2500,
);

Experimenters with ultra-narrowband radio transmissions often use continuous wave frequency-shifting radio transmitters to send data at very low rates over very long distances using very little power. See [_What is QRSS?_](https://www.qsl.net/m0ayf/What-is-QRSS.html) for more information.
spec.SaveJPG("output.jpg");
```

The following code produces a QRSS spectrogram from an MP3 file. This program took less than 5 seconds to analyze 30 minutes of audio, producing the image below.
If you're using Spectrogram in a graphical application you may find it helpful to retrieve the output as a Bitmap suitable for applying to a Picturebox or similar control:

```cs
// load audio and process FFT
var spec = new Spectrogram.Spectrogram(sampleRate: 8000, fftSize: 16384, step: 8000);
float[] values = Spectrogram.Tools.ReadMp3("qrss-w4hbk.mp3");
spec.AddExtend(values);

// convert FFT to an image and save it
Bitmap bmp = spec.GetBitmap(intensity: 1.5, freqLow: 1100, freqHigh: 1500,
showTicks: true, tickSpacingHz: 50, tickSpacingSec: 60);
spec.SaveBitmap(bmp, "qrss.png");
Bitmap bmp = spec.GetBitmap();
pictureBox1.Image = bmp;
```

After calculating the Spectrogram (the slow step) FFT magnitudes are stored in memory so you can rapidly recalculate pixel intensities based on new parameters:

![](data/qrss.png)
```cs
spec.Recalculate(
multiplier: 2.8,
dB: true,
cmap: new Inferno()
);
spec.SaveJPG("output2.jpg");
```

## Demo Applications
This project comes with a few interactive applications which serve as useful references for some of the ways this Spectrogram library can be used.
Notice the use of custom colormaps in this example.

### Download Demo EXE Files
If you want to see what this library can do without downloading source code, click-to-run (EXE) demos are available in **[SpectrogramDemo.zip](https://github.com/swharden/Spectrogram/raw/master/dev/compiled-demos/SpectrogramDemo.zip)**
Viridis | Inferno
---|---
![](dev/spectrogram.jpg) | ![](dev/spectrogram2.jpg)

### Audio Monitor
## Song-to-Spectrogram

A demo program is included which monitors the sound card and continuously creates spectrograms from microphone input. It runs fast enough that the entire bitmap can be recreated on each render. This means brightness and color adjustments can be applied to the whole image, not just new parts.
This example demonstrates how to convert a MP3 file to a spectrogram image. A sample MP3 audio file in the [data folder](data) contains the audio track from Ken Barker's excellent piano performance of George Frideric Handel's Suite No. 5 in E major for harpsichord ([_The Harmonious Blacksmith_](https://en.wikipedia.org/wiki/The_Harmonious_Blacksmith)). This audio file is included [with permission](dev/Handel%20-%20Air%20and%20Variations.txt), and the [original video can be viewed on YouTube](https://www.youtube.com/watch?v=Mza-xqk770k).

![](data/screenshot4.gif)
![](dev/spectrogram-song.jpg)

### Waterfall with Graphs
This demo program was created to demonstrate Spectrogram and ScottPlot working together.
If you [listen to the audio track](https://www.youtube.com/watch?v=Mza-xqk770k) while closely inspecting the spectrogram you can identify individual piano notes and chords, and may be surprised by the interesting patterns that emerge around trills and glissandos.

![](data/screenshot7.gif)
```cs
// TODO: update this example
```

## Resources

Expand All @@ -85,9 +77,4 @@ This demo program was created to demonstrate Spectrogram and ScottPlot working t
* SpectrumLab ([website](http://www.qsl.net/dl4yhf/spectra1.html)) - closed-source spectrum analyzer for Windows
* QrssPIG ([GitLab](https://gitlab.com/hb9fxx/qrsspig)) - open-source spectrograph for Raspberry Pi (C++)
* Lopora ([GitHub](https://github.com/swharden/Lopora)) - open-source spectrograph (Python 3)
* QRSS VD ([GitHub](https://github.com/swharden/QRSS-VD)) - open source spectrograph (Python 2)

### QRSS Information
* [What is QRSS?](https://www.qsl.net/m0ayf/What-is-QRSS.html)
* [QRSS and you](http://www.ka7oei.com/qrss1.html)
* [QRSS (slow CW)](https://sites.google.com/site/qrssinfo/QRSS-Slow-CW)
* QRSS VD ([GitHub](https://github.com/swharden/QRSS-VD)) - open source spectrograph (Python 2)
Binary file added data/Handel - Air and Variations.mp3
Binary file not shown.
9 changes: 9 additions & 0 deletions data/Handel - Air and Variations.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
Sep 3, 2019, 9:47 AM

Hi Scott.

I confirm that I am the sole copyright owner of the recording of the Handel piece "Air and Variations (The Harmonious Blacksmith)" in the following video published on YouTube: https://www.youtube.com/watch?v=Mza-xqk770k

I hereby grant Scott W. Harden permission to use the above recording in whole or in part as a sample audio file for the Spectrogram audio analysis library (https://github.com/swharden/Spectrogram).

Ken Barker <kbub6f@gmail.com>
Binary file removed data/cant-do-that-grayscale-inverted.jpg
Binary file not shown.
Binary file removed data/cant-do-that-grayscale.jpg
Binary file not shown.
Binary file removed data/cant-do-that-green.jpg
Binary file not shown.
Binary file removed data/cant-do-that.jpg
Binary file not shown.
Binary file removed data/mozart.jpg
Binary file not shown.
Binary file removed data/mozart.wav
Binary file not shown.
Binary file removed data/mozartSmall.jpg
Binary file not shown.
Binary file removed data/qrss.png
Binary file not shown.
Binary file removed data/screenshot2.gif
Binary file not shown.
Binary file removed data/screenshot3.gif
Binary file not shown.
Binary file removed data/screenshot4.gif
Binary file not shown.
Binary file removed data/screenshot6.gif
Binary file not shown.
Binary file removed data/screenshot7.gif
Binary file not shown.
23 changes: 23 additions & 0 deletions dev/LUT generation/mp.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
import matplotlib.pyplot as plt
import pyperclip

def cmapToCode(cmapName="inferno"):
cmap = plt.get_cmap(cmapName)

out = f"""\n // {cmapName}\n"""
out += """ public readonly byte[,] RGB =\n {\n"""
for i in range(256):
r, g, b, a = cmap(i/255.0)
out += " {%d, %d, %d},\n" % (r*256, g*256, b*256)
out += " };\n"

print(out)
pyperclip.copy(out)
print(f"Colormap {cmapName} copied to clipboard.")


if __name__ == "__main__":
cmapToCode("cividis")
# maps = sorted(m for m in plt.cm.datad if not m.endswith("_r"))
# for cmap in maps:
# cmapToCode(cmap)
Binary file removed dev/compiled-demos/SpectrogramDemo.zip
Binary file not shown.
Binary file added dev/spectrogram-song.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added dev/spectrogram.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added dev/spectrogram.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added dev/spectrogram2.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
2 changes: 1 addition & 1 deletion src/AudioMonitor/App.config → src/ArgoNot/App.config
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
<?xml version="1.0" encoding="utf-8" ?>
<configuration>
<startup>
<supportedRuntime version="v4.0" sku=".NETFramework,Version=v4.5" />
<supportedRuntime version="v4.0" sku=".NETFramework,Version=v4.7.2" />
</startup>
</configuration>
Original file line number Diff line number Diff line change
Expand Up @@ -4,12 +4,13 @@
<PropertyGroup>
<Configuration Condition=" '$(Configuration)' == '' ">Debug</Configuration>
<Platform Condition=" '$(Platform)' == '' ">AnyCPU</Platform>
<ProjectGuid>{806D56D1-077A-4E8C-A742-2AEED037AF17}</ProjectGuid>
<ProjectGuid>{F3BD387C-657F-4185-8420-9D491BDDFD24}</ProjectGuid>
<OutputType>WinExe</OutputType>
<RootNamespace>AudioMonitor</RootNamespace>
<AssemblyName>AudioMonitor</AssemblyName>
<TargetFrameworkVersion>v4.5</TargetFrameworkVersion>
<RootNamespace>ArgoNot</RootNamespace>
<AssemblyName>ArgoNot</AssemblyName>
<TargetFrameworkVersion>v4.7.2</TargetFrameworkVersion>
<FileAlignment>512</FileAlignment>
<AutoGenerateBindingRedirects>true</AutoGenerateBindingRedirects>
<Deterministic>true</Deterministic>
</PropertyGroup>
<PropertyGroup Condition=" '$(Configuration)|$(Platform)' == 'Debug|AnyCPU' ">
Expand All @@ -32,11 +33,20 @@
<WarningLevel>4</WarningLevel>
</PropertyGroup>
<ItemGroup>
<Reference Include="NAudio, Version=1.9.0.0, Culture=neutral, processorArchitecture=MSIL">
<HintPath>..\packages\NAudio.1.9.0\lib\net35\NAudio.dll</HintPath>
<Reference Include="FftSharp, Version=1.0.3.0, Culture=neutral, processorArchitecture=MSIL">
<HintPath>..\packages\FftSharp.1.0.3\lib\netstandard2.0\FftSharp.dll</HintPath>
</Reference>
<Reference Include="MP3Sharp, Version=1.0.5.0, Culture=neutral, processorArchitecture=MSIL">
<HintPath>..\packages\MP3Sharp.1.0.5\lib\netstandard2.0\MP3Sharp.dll</HintPath>
</Reference>
<Reference Include="NAudio, Version=1.10.0.0, Culture=neutral, processorArchitecture=MSIL">
<HintPath>..\packages\NAudio.1.10.0\lib\net35\NAudio.dll</HintPath>
</Reference>
<Reference Include="System" />
<Reference Include="System.Core" />
<Reference Include="System.Drawing.Common, Version=4.0.0.1, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51, processorArchitecture=MSIL">
<HintPath>..\packages\System.Drawing.Common.4.7.0\lib\net461\System.Drawing.Common.dll</HintPath>
</Reference>
<Reference Include="System.Xml.Linq" />
<Reference Include="System.Data.DataSetExtensions" />
<Reference Include="Microsoft.CSharp" />
Expand All @@ -48,18 +58,40 @@
<Reference Include="System.Xml" />
</ItemGroup>
<ItemGroup>
<Compile Include="Form2.cs">
<SubType>Form</SubType>
</Compile>
<Compile Include="Form2.Designer.cs">
<DependentUpon>Form2.cs</DependentUpon>
</Compile>
<Compile Include="Form3.cs">
<SubType>Form</SubType>
</Compile>
<Compile Include="Form3.Designer.cs">
<DependentUpon>Form3.cs</DependentUpon>
</Compile>
<Compile Include="Grab.cs" />
<Compile Include="Ticks.cs" />
<Compile Include="Form1.cs">
<SubType>Form</SubType>
</Compile>
<Compile Include="Form1.Designer.cs">
<DependentUpon>Form1.cs</DependentUpon>
</Compile>
<Compile Include="Listener.cs" />
<Compile Include="Program.cs" />
<Compile Include="Properties\AssemblyInfo.cs" />
<Compile Include="WsprBands.cs" />
<Compile Include="WsprSpot.cs" />
<Compile Include="WsprLogWatcher.cs" />
<EmbeddedResource Include="Form1.resx">
<DependentUpon>Form1.cs</DependentUpon>
</EmbeddedResource>
<EmbeddedResource Include="Form2.resx">
<DependentUpon>Form2.cs</DependentUpon>
</EmbeddedResource>
<EmbeddedResource Include="Form3.resx">
<DependentUpon>Form3.cs</DependentUpon>
</EmbeddedResource>
<EmbeddedResource Include="Properties\Resources.resx">
<Generator>ResXFileCodeGenerator</Generator>
<LastGenOutput>Resources.Designer.cs</LastGenOutput>
Expand All @@ -85,7 +117,7 @@
</ItemGroup>
<ItemGroup>
<ProjectReference Include="..\Spectrogram\Spectrogram.csproj">
<Project>{8717f4ce-4497-4eaa-b95d-0f7a04fb397d}</Project>
<Project>{19372fc8-8c95-48cf-a70a-ed2eebaaa173}</Project>
<Name>Spectrogram</Name>
</ProjectReference>
</ItemGroup>
Expand Down
Loading