Crash: Floating-point to integer conversion overflow in ass_render_event() on extreme override-tag values (CWE-681) -> UBSAN SIGILL

[1820893135-pixel]

libass version

libass 0.17.5 (commit 3087d2b, 2026-08-10, master).

Is it a regression?

Unknown — I did not test older releases. The trigger uses only extreme numeric values in override tags that libass parses as doubles; the crash occurs when those doubles are later converted back to integers. I suspect it is not a recent regression, but I have not bisected.

ASS Sample

This minimal, fully functional ASS file reproduces the crash deterministically (5 of 13 fuzz crashes share this root cause). With -fno-sanitize-recover=all UBSAN traps as ud1 and the process dies with SIGILL.

[Script Info]
ScriptType: v4.00+
PlayResX: 854
PlayResY: 480

[V4+ Styles]
Format: Name, Fontname, Fontsize, PrimaryColour, SecondaryColour, OutlineColour, BackColour, Bold, Italic, Underline, StrikeOut, ScaleX, ScaleY, Spacing, Angle, BorderStyle, Outline, Shadow, Alignment, MarginL, MarginR, MarginV, Encoding
Style: Default, Arial, 20, &H00FFFFFF, &H000000FF, &H00000000, &H00000000, 0, 0, 0, 0, 100, 100, 0, 0, 1, 2, 1, 2, 10, 10, 10, 1

[Events]
Format: Layer, Start, End, Style, Name, MarginL, MarginR, MarginV, Effect, Text
Dialogue: 0,0:00:01.00,0:00:03.00,Default,,0,0,0,,{\frx(1000000)}{\fry(1000000)}{\frz(1000000)}rot 1e6
Dialogue: 0,0:00:01.00,0:00:03.00,Default,,0,0,0,,{\frx(2147483647)}{\fry(-2147483648)}{\frz(999999999)}rot extremes
Dialogue: 0,0:00:01.00,0:00:03.00,Default,,0,0,0,,{\org(5000,5000)}{\frz(1000000000)}{\frx(1000000000)}org big rot
Dialogue: 0,0:00:01.00,0:00:03.00,Default,,0,0,0,,{\fax(1000000000)}{\fay(1000000000)}{\fr(1000000000)}shear 1e9
Dialogue: 0,0:00:01.00,0:00:03.00,Default,,0,0,0,,{\frx(-1000000000)}{\fry(1000000000)}neg rot

Key ingredients: extreme \frx/\fry/\frz/\fax/\fay/\fr override values (up to 2147483647 / 1000000000). These are parsed as doubles and later converted back to integers in ass_render_event (via ass_lrint/lround), overflowing the integer conversion.

Special Fonts

None — the crash happens during per-event transform handling before font-specific rasterization; default font substitution is sufficient.

System Information

• OS: Debian (x86-64 Linux)
• CPU: 64-bit x86-64

Log and Stacktrace

UBSAN traps as a ud1 instruction (SIGILL); ASAN reports DEADLYSIGNAL ILL. Representative stack:

#0 __clang_trap_msg$UndefinedBehaviorSanitizer$Floating-point to integer conversion overflowed /src/libass/ass_render.c
#1 ass_render_event /src/libass/ass_render.c
#2 ass_render_frame /src/libass/ass_render.c:3392:17
#3 consume_track /src/harness.c:128:23
#4 LLVMFuzzerTestOneInput /src/harness.c:168:13

Affected area, libass/ass_render.c, ass_render_event: ass_lrint/lround on extreme float override values (e.g. \frx, \frz, \pos, \fscx) overflows the integer conversion.

Reproduction:

# build the harness (ass_read_memory + ass_render_frame) with clang ASan+UBSan
clang -O1 -g -fsanitize=fuzzer,address,undefined -fno-sanitize-recover=all \
      harness.c -o libass_read_memory_fuzzer $(pkg-config --cflags --libs libass)

# save the ASS sample above as sample_b.ass, then:
./libass_read_memory_fuzzer sample_b.ass
# -> exit 132 (SIGILL) from UBSAN trap

Or with ffmpeg:

ffmpeg -f lavfi -i 'color=black:s=854x480' -vf 'ass=sample_b.ass' -frames:v 1 out.png

Deterministic (5/5 inputs for this root cause reproduce). Crash is target code, not harness misuse.

Special Fonts

I uploaded or linked to the required font

System Information

No response

Log and Stacktrace

No response

Additional info

No response