Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] MP3 headers are malformed #410

Open
D0han opened this issue Sep 5, 2023 · 13 comments
Open

[BUG] MP3 headers are malformed #410

D0han opened this issue Sep 5, 2023 · 13 comments

Comments

@D0han
Copy link

D0han commented Sep 5, 2023

Describe your environment

  • RTLSDR-Airband version you are using (stable release number or branch/commit): v4.1.1
  • make options used to build the program: cmake ../ && make
  • Hardware platform (eg. x86_64, Raspberry Pi v4): Raspberry Pi 3 Model B Rev 1.2
  • Operating system name and version: Debian GNU/Linux 11 (bullseye)

What happened?
Created MP3 files are malformed. In most cases audio is still usable, not sure if that is always the case.

What you expected to happen?
Fully standard and proper MP3 file

Steps to Reproduce
Use Airband to save radio traffic to mp3

Additional context
Code

import librosa
audio_data, sample_rate = librosa.load("airband-created-file.mp3", mono=True)

output example

Note: Illegal Audio-MPEG-Header 0x4cb44fd5 at offset 119664.
Note: Trying to resync...
Note: Skipped 8 bytes in input.
Note: Illegal Audio-MPEG-Header 0x2c66b66d at offset 182168.
Note: Trying to resync...
Note: Skipped 23 bytes in input.
Note: Illegal Audio-MPEG-Header 0x55555555 at offset 186151.
Note: Trying to resync...
Note: Skipped 24 bytes in input.

Exact output varies.
This is not a new issue, saw the same in old versions from 6-12 months ago.

Your rtl_airband.conf file

devices:
({
  type = "rtlsdr";
  index = 0;
  serial = "00000001";
  gain = 35;
  mode = "multichannel";
  multiple_demod_threads = true;
  centerfreq = 149.5;
  correction = 36;
  channels:
  (
    {
      freq = 148.825000;
      modulation = "nfm";
      outputs: (
        {
        type = "file";
        directory = "/home/airband/saved";
        filename_template = "ASD";
        split_on_transmission = true;
        include_freq = true;
        }
      );
    }
  );
 }
);
@charlie-foxtrot
Copy link
Owner

Yes, I have noticed this as well but haven't looked into it because the audio seems fine / hasn't been a priority to look into.

Is this causing issues? If so, I wonder if the LAME parameter changes in #411 have any impact

FWIW I ran into this using both ffmpeg as well as trying to open files in Audacity (I used the work around here)

@D0han
Copy link
Author

D0han commented Sep 15, 2023

Not sure if that causes data loss or not, i do not have parallel samples from another source to compare.
If you can do first review pass on #411 to confirm the code is doing what we expect, i can try deploying it to see if that fixes the mp3 structure.

@dseven
Copy link

dseven commented Oct 21, 2023

I've been grappling with an issue that may or may not be related to this. RTLSDR-Airband is capturing ATC traffic to mp3 files that rdio-scanner picks up (via dirwatch). The problem is that some of the recordings stop playback via rdio-scanner part-way through, and some don't play at all. Perhaps interestingly, the ones that stop in the middle always seems to do so at the point where there's an "over" in the conversation - i.e. ATC stops talking, then the aircraft starts, or vice-versa. If I download the mp3 from rdio-scanner, and play it with any other player, all of the audio is there. If I import the mp3 into audacity, re-export it as mp3, and drop the result in the dirwatch directory, rdio-scanner plays it OK. I tried applying the proposed "MP3 Quality" changes from #414, but that didn't fix it. I thought it might be something to do with VBR and periods of silence, but apparently not. My solution, for now, is to have rdio-scanner use ffmpeg to transcode the mp3 to h264/m4a, which seems to work.

@D0han
Copy link
Author

D0han commented Nov 3, 2023

I checked out recordings, and a lot of them cuts off words. Having said that, i do not have any concrete arguments to tie it up with malformed MP3 headers.

@coatebra
Copy link

coatebra commented Dec 5, 2023

Most of recorded mp3 files are not playable directly from a web server using html5 audio component. When I tried to convert mp3 to ogg using ffmpeg, I got a lot of errors:

Header missing=N/A speed= 0x
Error while decoding stream #0:0: Invalid data found when processing input

@zigsphere
Copy link

Following - I really think this is causing many of the issue Im having. Since the headers are missing, icecast and perhaps the client is unaware what media format the audio is. With that being the case, there is a delay in the feed.

Can this be fixed?

@krbaker
Copy link

krbaker commented Jun 21, 2024

I'm also seeing this and curious to see where it goes. I'm doing speech-to-text on Marine VHF and I upload the results to a Slack channel along with the audio. 90% of the time Slack doesn't recognize this as an audio file.

ffplay gets results like:
[mp3 @ 0x75bbc4000cc0] Format mp3 detected only with low score of 1, misdetection possible!
[mp3 @ 0x75bbc4000cc0] Skipping 19 bytes of junk at 0.
[mp3 @ 0x75bbc4000cc0] Estimating duration from bitrate, this may be inaccurate
Input #0, mp3, from 'vhf-78a/recording_20240614_125529.mp3':
Duration: 00:00:10.56, start: 0.000000, bitrate: 8 kb/s
Stream #0:0: Audio: mp3, 8000 Hz, mono, fltp, 8 kb/s
[mp3float @ 0x75bbc4006a80] Header missing 0KB sq= 0B f=0/0
[mp3float @ 0x75bbc4006a80] Header missing 0KB sq= 0B f=0/0
[mp3float @ 0x75bbc4006a80] Header missing 0KB sq= 0B f=0/0
4.68 M-A: 0.000 fd= 0 aq= 0KB vq= 0KB sq= 0B f=0/0

I'm in file split mode but it still seems to match to moments of discontinuity (changing speakers where squelch doesn't actually have time to close)

--- a bit of research

If I'm not mistaken the code creates a brand new instance of lame for encoding silenceand the inserted tones. If I read correctly lame_encode_flush_nogap (used to close the files) can make a set of validly concatenate-able files but only if you continue with that instance
-- some PCM data can/is often still in the lame buffers after the flush nogap call https://android.googlesource.com/platform/external/lame/+/master/include/lame.h
-- The 'why can't I copy-paste mp3s' section of https://lame.sourceforge.io/tech-FAQ.txt has a lot of detail that seems relevant.

@rough316
Copy link

Ugh, this just bit me too. I was trying to get mp3 for each channel in scan mode so I can process those and get a transcription... Malformed header on every mp3.

@D0han
Copy link
Author

D0han commented Jul 13, 2024

#411 was abandoned to the point of branch deletion.
@charlie-foxtrot seems to not use github at all since February, which is abnormal in comparison to previous months. I think that this project lost its maintainer.

There are few forks visible in https://github.com/charlie-foxtrot/RTLSDR-Airband/network but i did not check if anyone did pick this up already in full capacity.

@charlie-foxtrot
Copy link
Owner

I have unfortunately had to take a step back. I do intend to return but things are taking longer than I expected.

@D0han
Copy link
Author

D0han commented Jul 14, 2024

Glad you're still here

@krbaker
Copy link

krbaker commented Aug 5, 2024

@charlie-foxtrot I hope all is well, great to know you'll be back :-)

@ljkenny
Copy link

ljkenny commented Aug 20, 2024

A little more info after tripping over this myself.

Google Drive can only directly play a sub-set (the non-LAME3 ones) of the saved files.

Here is a file and hexdump of each file:

$ for f in `find . | grep mp3`; do file ${f}; hexdump -C ${f} | head -n1; echo; done
./mcr_20240819_16.mp3: data
00000000  4c 41 4d 45 33 2e 31 30  30 55 55 55 55 55 55 55  |LAME3.100UUUUUUU|

./mcr_20240819_17.mp3: data
00000000  4c 41 4d 45 33 2e 31 30  30 aa aa aa aa aa aa aa  |LAME3.100.......|

./mcr_20240819_18.mp3: MPEG ADTS, layer III,  v2.5,   8 kbps, 8 kHz, Monaural
00000000  ff e3 18 c4 00 00 00 03  48 01 40 00 00 ff e3 bb  |........H.@.....|

./mcr_20240819_19.mp3: MPEG ADTS, layer III,  v2.5,   8 kbps, 8 kHz, Monaural
00000000  ff e3 18 c4 00 00 00 03  fc 01 40 00 00 66 86 62  |..........@..f.b|

./mcr_20240819_20.mp3: data
00000000  4c 41 4d 45 33 2e 31 30  30 55 55 55 55 55 55 55  |LAME3.100UUUUUUU|

./mcr_20240819_21.mp3: data
00000000  4c 41 4d 45 33 2e 31 30  30 aa aa aa aa aa aa aa  |LAME3.100.......|

./mcr_20240819_22.mp3: data
00000000  4c 41 4d 45 33 2e 31 30  30 55 55 55 55 55 55 55  |LAME3.100UUUUUUU|

./mcr_20240819_23.mp3: MPEG ADTS, layer III,  v2.5,   8 kbps, 8 kHz, Monaural
00000000  ff e3 18 c4 00 00 00 03  48 01 40 00 00 03 fe 3f  |........H.@....?|

./mcr_20240820_00.mp3: data
00000000  4c 41 4d 45 33 2e 31 30  30 aa aa aa aa aa aa aa  |LAME3.100.......|

./mcr_20240820_01.mp3: MPEG ADTS, layer III,  v2.5,   8 kbps, 8 kHz, Monaural
00000000  ff e3 18 c4 00 00 00 03  48 01 40 00 00 ff ff ff  |........H.@.....|

./mcr_20240820_02.mp3: data
00000000  4c 41 4d 45 33 2e 31 30  30 aa aa aa aa aa aa aa  |LAME3.100.......|

./mcr_20240820_03.mp3: data
00000000  4c 41 4d 45 33 2e 31 30  30 aa aa aa aa aa aa aa  |LAME3.100.......|

./mcr_20240820_04.mp3: data
00000000  4c 41 4d 45 33 2e 31 30  30 55 55 55 55 55 55 55  |LAME3.100UUUUUUU|

./mcr_20240820_05.mp3: data
00000000  4c 41 4d 45 33 2e 31 30  30 55 55 55 55 55 55 55  |LAME3.100UUUUUUU|

./mcr_20240820_06.mp3: data
00000000  4c 41 4d 45 33 2e 31 30  30 aa aa aa aa aa aa aa  |LAME3.100.......|

./mcr_20240820_07.mp3: data
00000000  4c 41 4d 45 33 2e 31 30  30 55 55 55 55 55 55 55  |LAME3.100UUUUUUU|

./mcr_20240820_08.mp3: data
00000000  4c 41 4d 45 33 2e 31 30  30 55 55 55 55 55 55 55  |LAME3.100UUUUUUU|

The ones identified as MPEG play fine naively in Google Drive.

However the LAME3 ones are not identified as playable and file only IDs them as data.

Downloading the LAME3 ones and playing them on an Android device works fine, however.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

8 participants