Skip to content

[Bug] hwdec_cuda: mapper_init returns 0 on ext_init failure #17976

@hklcf

Description

@hklcf

File

video/out/hwdec/hwdec_cuda.c:195-205

Description

In mapper_init, when ext_init fails, the function returns 0 (success) instead of an error code:

ret = CHECK_CU(cu->cuCtxPushCurrent(p->display_ctx));  // ret = 0 (success)
if (ret < 0)
    return ret;

for (int n = 0; n < desc.num_planes; n++) {
    if (!p_owner->ext_init(mapper, desc.planes[n], n))
        goto error;  // ret is still 0!
}

error:
    eret = CHECK_CU(cu->cuCtxPopCurrent(&dummy));
    if (eret < 0)
        return eret;

    return ret;  // returns 0 even though ext_init failed

Why It's a Bug

When ext_init fails, ret is still 0 (from the successful cuCtxPushCurrent). The goto error does NOT set ret = -1. The framework treats the mapper as successfully initialized, but no CUDA resources (textures/arrays) are actually allocated. A subsequent mapper_map will dereference zero-initialized arrays and crash.

Suggested Fix

Add ret = -1; before goto error:

if (!p_owner->ext_init(mapper, desc.planes[n], n)) {
    ret = -1;
    goto error;
}

Severity

Critical

Metadata

Metadata

Assignees

No one assigned

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions