File
video/out/hwdec/hwdec_cuda.c:195-205
Description
In mapper_init, when ext_init fails, the function returns 0 (success) instead of an error code:
ret = CHECK_CU(cu->cuCtxPushCurrent(p->display_ctx)); // ret = 0 (success)
if (ret < 0)
return ret;
for (int n = 0; n < desc.num_planes; n++) {
if (!p_owner->ext_init(mapper, desc.planes[n], n))
goto error; // ret is still 0!
}
error:
eret = CHECK_CU(cu->cuCtxPopCurrent(&dummy));
if (eret < 0)
return eret;
return ret; // returns 0 even though ext_init failed
Why It's a Bug
When ext_init fails, ret is still 0 (from the successful cuCtxPushCurrent). The goto error does NOT set ret = -1. The framework treats the mapper as successfully initialized, but no CUDA resources (textures/arrays) are actually allocated. A subsequent mapper_map will dereference zero-initialized arrays and crash.
Suggested Fix
Add ret = -1; before goto error:
if (!p_owner->ext_init(mapper, desc.planes[n], n)) {
ret = -1;
goto error;
}
Severity
Critical
File
video/out/hwdec/hwdec_cuda.c:195-205
Description
In
mapper_init, whenext_initfails, the function returns 0 (success) instead of an error code:Why It's a Bug
When
ext_initfails,retis still 0 (from the successfulcuCtxPushCurrent). Thegoto errordoes NOT setret = -1. The framework treats the mapper as successfully initialized, but no CUDA resources (textures/arrays) are actually allocated. A subsequentmapper_mapwill dereference zero-initialized arrays and crash.Suggested Fix
Add
ret = -1;beforegoto error:Severity
Critical