Skip to content

UBSan: shift exponent 32 in JPC_CEILDIVPOW2() from COD numdlvls=32 (jpc_dec.c) #436

Open

Description

@MMu-Aqua

Affected: JasPer 4.2.9 (5ce57f67e0011e0d2483992a5d77a092b1b64eb7)
Sanitizer: UndefinedBehaviorSanitizer (shift exponent 32 is too large for 32-bit type 'int'), not ASan
Entry point: imginfo -f poc.jpc

Summary

COD/COC parsing allows numdlvls <= 32 (JPEG2000 max decomposition levels). That becomes numrlvls = numdlvls + 1 (up to 33, which is JPC_MAXRLVLS). In jpc_dec_tileinit() the first resolution level then does:

rlvl->xstart = JPC_CEILDIVPOW2(tcomp->xstart, tcomp->numrlvls - 1 - rlvlno);

At rlvlno == 0 the shift count is 32. The macro is:

#define JPC_CEILDIVPOW2(x, y)  (((x) + (1 << (y)) - 1) >> (y))  /* jpc_math.h:87 */

1 << 32 is undefined for 32-bit int. UBSan:

jpc_dec.c:778:19: runtime error: shift exponent 32 is too large for 32-bit type 'int'

Code

jpc_cs.c accepts 32 decomposition levels:

	if (compparms->numdlvls > 32) {
		goto error;
	}
	compparms->numrlvls = compparms->numdlvls + 1;
	if (compparms->numrlvls > JPC_MAXRLVLS) {  /* JPC_MAXRLVLS is 33 */
		goto error;
	}

jpc_dec.c jpc_dec_tileinit():

			rlvl->xstart = JPC_CEILDIVPOW2(tcomp->xstart,
			  tcomp->numrlvls - 1 - rlvlno);  /* :778 */
			rlvl->ystart = JPC_CEILDIVPOW2(tcomp->ystart,
			  tcomp->numrlvls - 1 - rlvlno);
			rlvl->xend = JPC_CEILDIVPOW2(tcomp->xend,
			  tcomp->numrlvls - 1 - rlvlno);
			rlvl->yend = JPC_CEILDIVPOW2(tcomp->yend,
			  tcomp->numrlvls - 1 - rlvlno);

PoC

poc_jpc_ceildivpow2.zip

# poc_jpc_ceildivpow2.py
open("poc_jpc_ceildivpow2.jpc", "wb").write(bytes.fromhex(
    "ff4fff5100360101000000010000000800000000000000000101000001010117"
    "00000000000000000004010d0d0d0d0d0d0d0d0d0d0d01180081ff52000c0001"
    "010a32201c030000ff5c010341ffd8424143464f0023001a0000000800000008"
    "0000000042424442464141424146444441454246454244414146014501011b01"
    "4441464246415551434759454645454446464243444443414443454145414143"
    "464243434152000bf2000001000003030001ff5b00044000ff90000a00000000"
    "00464601011b01414646bb4144ff0101014546444500007fff42414345414144"
    "3443464544464144444444414143414545b92141454142444642424346414646"
    "4642464145434441424441434464434245424545434145444444414441454243"
    "4543454243424341464246444345464444574442010001014341454244454545"
    "41414443444442454543434246ff90000a0000000000160001ff93ffd9010101"
    "00791c0f"
))

Reproduce

cmake -S . -B build \
  -DCMAKE_C_FLAGS="-fsanitize=undefined,address -fno-omit-frame-pointer" \
  -DCMAKE_EXE_LINKER_FLAGS="-fsanitize=undefined,address"
cmake --build build -j
export UBSAN_OPTIONS='halt_on_error=1:abort_on_error=1:print_stacktrace=1'
./build/src/app/imginfo -f poc_jpc_ceildivpow2.jpc

Stack

jpc_dec.c:778:19: runtime error: shift exponent 32 is too large for 32-bit type 'int'
#0 jpc_dec_tileinit    jpc_dec.c:778:19
#1 jpc_dec_process_sod jpc_dec.c:605:7
#2 jpc_dec_decode      jpc_dec.c:434:10
#3 jpc_decode          jpc_dec.c:270:6
#4 jas_image_decode    jas_image.c:477:16
#5 main                imginfo.c:334:16
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior .../jpc_dec.c:778:19

Suggested fix

JPC_CEILDIVPOW2 should not do 1 << y with a signed 32-bit int when y can be 32. Either reject numdlvls == 32 (inconsistent with the current <= 32 / JPC_MAXRLVLS == 33 checks), or implement the ceiling-shift with a 64-bit / unsigned type and an explicit y >= 32 path.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions