FLAC: Floating Point Exception (Division by Zero) in flac__encode_file when processing crafted WAVE files with bitsPerSample < 8

[Criticayon]

Description

The crash occurs in: flac__encode_file() at encode.c:967

A Floating Point Exception (FPE / SIGFPE) is triggered by a division-by-zero when FLAC's WAV parser processes a crafted WAVE file with bitsPerSample < 8 through WAVE_FORMAT_EXTENSIBLE. The computed bytes_per_wide_sample becomes zero, which is then used as a divisor to calculate the total number of input samples, causing a crash.

ASAN reports: FPE (Floating Point Exception) — division by zero.

---

Environment

• FLAC version: 1.5.0-31-gb430c3a5 (commit b430c3a)
• Compiler: afl-clang-fast / afl-clang-fast++
• Instrumentation: AFL++ + AddressSanitizer
• OS: Linux (x86_64)

---

Build Configuration

AFL_USE_ASAN=1 CC=afl-clang-fast CXX=afl-clang-fast++ cmake .. -DCMAKE_C_COMPILER=afl-clang-fast -DCMAKE_CXX_COMPILER=afl-clang-fast++ && make -j$(nproc)

---

Reproduction Steps

Run:

ASAN_OPTIONS=detect_leaks=0:abort_on_error=0 ./flac --replay-gain --cuesheet cue_sheet.cue -T ARTIST=Test --picture pic_cover.jpg poc 

Where malicious.wav is a crafted WAVE file with:

• wFormatTag = 65534 (WAVE_FORMAT_EXTENSIBLE)
• bitsPerSample = 0 (any value < 8)
• wValidBitsPerSample = 0

---

PoC

cue_sheet.cue:

REM GENRE "Test"
REM DATE "2024"
PERFORMER "Test Artist"
TITLE "Test Title"
FILE "test.wav" WAVE
  TRACK 01 AUDIO
    TITLE "Track 1"
    PERFORMER "Test Artist"
    INDEX 01 00:00:00
  TRACK 02 AUDIO
    TITLE "Track 2"
    PERFORMER "Test Artist"
    INDEX 01 01:00:00

See attached file: poc.wav

---

ASAN Report

Full ASAN output:

flac git-b430c3a5 20260508
Copyright (C) 2000-2009 Josh Coalson, 2011-2025 Xiph.Org Foundation
flac comes with ABSOLUTELY NO WARRANTY.

id:000000,sig:08,src:000381: WARNING: skipping unknown chunk 'fact'
  (use --keep-foreign-metadata to keep)
AddressSanitizer:DEADLYSIGNAL
=================================================================
==3933569==ERROR: AddressSanitizer: FPE on unknown address 0x5863919f3540
    (pc 0x5863919f3540 bp 0x7ffff76b5ce0 sp 0x7ffff76b5ae0 T0)
    #0 0x5863919f3540 in flac__encode_file /workspace/flac/src/flac/encode.c:967:65
    #1 0x586391a1d6f5 in encode_file /workspace/flac/src/flac/main.c:1763:12
    #2 0x586391a170ad in do_it /workspace/flac/src/flac/main.c:570:8
    #3 0x586391a170ad in main /workspace/flac/src/flac/main.c:382:13
    #4 0x73af30d291c9 (/lib/x86_64-linux-gnu/libc.so.6+0x2a1c9)
    #5 0x73af30d2928a in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x2a28a)
    #6 0x5863918f7854 in _start (/workspace/flac/build_afl/src/flac/flac+0x47854)

Register values:
rax = 0x0000000000000016  rbx = 0x00007ffff76b5ae0  rcx = 0x0000000000000000
rdx = 0x0000000000000000  rdi = 0x000058639288a19d  rsi = 0x000073af2f7006c8
rbp = 0x00007ffff76b5ce0  rsp = 0x00007ffff76b5ae0
 r8 = 0x0000000000000007   r9 = 0x000000000000ac44  r10 = 0x0000000000000001
r11 = 0x0000000000000001  r12 = 0x0000586392889db0  r13 = 0x000073af2f700e90
r14 = 0x000050b000000111  r15 = 0x00000e7665ed8000
AddressSanitizer can not provide additional info.
SUMMARY: AddressSanitizer: FPE /workspace/flac/src/flac/encode.c:967:65 in flac__encode_file
==3933569==ABORTING

---

Affected Code

src/flac/encode.c:439 (root cause) and encode.c:967 (crash site)

Root cause — bytes_per_wide_sample calculation (encode.c:439):

e->info.bytes_per_wide_sample = channels * (bps / 8);  // <--- division truncates to 0 when bps < 8

Crash site — division by zero (encode.c:967):

total_samples_in_input = encoder_session.fmt.iff.data_bytes
    / encoder_session.info.bytes_per_wide_sample;  // <--- FPE HERE (divide by zero)

---

Root Cause Analysis

The WAV fmt chunk parser reads bitsPerSample (bps) directly from the WAV file header at encode.c:374. At line 439, bytes_per_wide_sample is calculated using integer division: bps / 8.

When bps < 8 (including bps=0), this division yields 0, making bytes_per_wide_sample = 0.

Later, at line 967 in flac__encode_file(), the code computes:

total_samples_in_input = data_bytes / bytes_per_wide_sample;

Division by zero triggers a SIGFPE (floating point exception), crashing the process.

Why does bps < 8 pass validation?

• For WAVE_FORMAT_PCM (wFormatTag=1): The code validates bps at lines 385-399, restricting it to 8, 16, 24, or 32. Values < 8 are rejected.
• For WAVE_FORMAT_EXTENSIBLE (wFormatTag=65534): The code takes the else branch at line 409. It checks wValidBitsPerSample <= bps and the format GUID, but it never validates that bps >= 8. A crafted file with bps=0 (or any value 1-7) passes all checks.
• The resulting division by zero at line 967 is unconditionally reachable once the malformed WAV passes the fmt chunk parser.

Classification: CWE-369 (Divide By Zero). No memory corruption involved, but the crash constitutes a reliable denial-of-service vector against any application using FLAC to process untrusted WAV files with --replay-gain.

---

Impact

• Denial of Service (DoS): The flac encoder crashes immediately when processing a crafted WAV file through the --replay-gain code path. Any service or application that uses flac to encode untrusted WAV input with replay-gain processing is affected.

[madah81pnz1]

Same as #873