Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 15 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -20369,6 +20369,21 @@ uses the kernel-template readback pair: `integer_psnr_cuda`, `integer_ssim_cuda`
were already correct; `speed_chroma_cuda.c` was the only remaining file.


- **`speed_chroma_cuda` and `speed_temporal_cuda` leaked their CUDA module and stream on init failure paths.**
`cuModuleLoadData()` and `cuStreamCreate()` both jump to failure labels on error,
but those labels (`fail_pop`, `fail_after_pop`, and `free_cpu`/`free_all`) only
called `free_cuda_buffers` / `free_cuda_buffers_st`, which only release device
and host memory and never touch `s->module` or `s->stream`.
Any failure after module loading leaked the module, and any failure after stream
creation leaked both. In `speed_chroma_cuda.c`, PR #1007 had originally added
inline cleanup which was inadvertently removed during the PR #1029 merge;
factoring the teardown into `release_cuda_module_and_stream` (and
`release_cuda_module_and_stream_st` in `speed_temporal_cuda.c`) prevents future
drift and unifies cleanup with `close_fex_cuda` / `close_fex_st`.
`integer_ms_ssim_cuda.c` and `integer_psnr_hvs_cuda.c` failure exits are
hardened to route through `close_fex_cuda` as well.


- **fix(cuda)**: add `static` to 12 TU-internal helper functions across
`integer_vif_cuda.c`, `integer_adm_cuda.c`, and `integer_motion_cuda.c` —
eliminates ODR name-clash risk in static-lib builds; restores parity with
Expand Down
13 changes: 13 additions & 0 deletions changelog.d/fixed/cuda-speed-init-leak-regression.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
- **`speed_chroma_cuda` and `speed_temporal_cuda` leaked their CUDA module and stream on init failure paths.**
`cuModuleLoadData()` and `cuStreamCreate()` both jump to failure labels on error,
but those labels (`fail_pop`, `fail_after_pop`, and `free_cpu`/`free_all`) only
called `free_cuda_buffers` / `free_cuda_buffers_st`, which only release device
and host memory and never touch `s->module` or `s->stream`.
Any failure after module loading leaked the module, and any failure after stream
creation leaked both. In `speed_chroma_cuda.c`, PR #1007 had originally added
inline cleanup which was inadvertently removed during the PR #1029 merge;
factoring the teardown into `release_cuda_module_and_stream` (and
`release_cuda_module_and_stream_st` in `speed_temporal_cuda.c`) prevents future
drift and unifies cleanup with `close_fex_cuda` / `close_fex_st`.
`integer_ms_ssim_cuda.c` and `integer_psnr_hvs_cuda.c` failure exits are
hardened to route through `close_fex_cuda` as well.
6 changes: 4 additions & 2 deletions core/src/feature/cuda/integer_ms_ssim_cuda.c
Original file line number Diff line number Diff line change
Expand Up @@ -311,7 +311,7 @@ static int init_fex_cuda(VmafFeatureExtractor *fex, enum VmafPixelFormat pix_fmt
if (ctx_pushed)
(void)cu_f->cuCtxPopCurrent(NULL);
fail_after_pop:
(void)vmaf_cuda_kernel_lifecycle_close(&s->lc, fex->cu_state);
(void)close_fex_cuda(fex);
return _cuda_err;
}

Expand Down Expand Up @@ -628,8 +628,10 @@ static int close_fex_cuda(VmafFeatureExtractor *fex)
}
ret |= vmaf_dictionary_free(&s->feature_name_dict);
const CudaFunctions *cu_f = fex->cu_state->f;
if (cu_f && s->module)
if (cu_f && s->module) {
(void)cu_f->cuModuleUnload(s->module);
s->module = NULL;
}
return ret;
}

Expand Down
6 changes: 4 additions & 2 deletions core/src/feature/cuda/integer_psnr_hvs_cuda.c
Original file line number Diff line number Diff line change
Expand Up @@ -240,7 +240,7 @@ static int init_fex_cuda(VmafFeatureExtractor *fex, enum VmafPixelFormat pix_fmt
if (ctx_pushed)
(void)cu_f->cuCtxPopCurrent(NULL);
fail_after_pop:
(void)vmaf_cuda_kernel_lifecycle_close(&s->lc, fex->cu_state);
(void)close_fex_cuda(fex);
return _cuda_err;
}

Expand Down Expand Up @@ -521,8 +521,10 @@ static int close_fex_cuda(VmafFeatureExtractor *fex)
(void)vmaf_cuda_buffer_host_free(fex->cu_state, s->h_uint_dist[p]);
}
ret |= vmaf_dictionary_free(&s->feature_name_dict);
if (cu_f && s->module)
if (cu_f && s->module) {
(void)cu_f->cuModuleUnload(s->module);
s->module = NULL;
}
return ret;
}

Expand Down
9 changes: 4 additions & 5 deletions core/src/feature/cuda/speed_chroma_cuda.c
Original file line number Diff line number Diff line change
Expand Up @@ -835,6 +835,7 @@ static int init_fex_cuda(VmafFeatureExtractor *fex, enum VmafPixelFormat pix_fmt
return 0;

free_cpu:
release_cuda_module_and_stream(s, cu_f);
free_cuda_buffers(s, cu_f);
return err;

Expand Down Expand Up @@ -914,14 +915,12 @@ static int extract_fex_cuda(VmafFeatureExtractor *fex, VmafPicture *ref_pic,
static int close_fex_cuda(VmafFeatureExtractor *fex)
{
SpeedChromaCudaState *s = fex->priv;
if (fex->cu_state && fex->cu_state->f)
if (fex->cu_state && fex->cu_state->f) {
free_cuda_buffers(s, fex->cu_state->f);
release_cuda_module_and_stream(s, fex->cu_state->f);
}
if (s->feature_name_dict)
vmaf_dictionary_free(&s->feature_name_dict);
if (s->stream && fex->cu_state && fex->cu_state->f)
(void)fex->cu_state->f->cuStreamDestroy(s->stream);
if (fex->cu_state && fex->cu_state->f && s->module)
(void)fex->cu_state->f->cuModuleUnload(s->module);
return 0;
}

Expand Down
21 changes: 17 additions & 4 deletions core/src/feature/cuda/speed_temporal_cuda.c
Original file line number Diff line number Diff line change
Expand Up @@ -442,6 +442,20 @@ static int run_score_st(SpeedTemporalCudaState *s, CudaFunctions *cu_f, float *s
/* Lifecycle */
/* ------------------------------------------------------------------ */

static void release_cuda_module_and_stream_st(SpeedTemporalCudaState *s, CudaFunctions *cu_f)
{
if (!s || !cu_f)
return;
if (s->stream) {
(void)cu_f->cuStreamDestroy(s->stream);
s->stream = 0;
}
if (s->module) {
(void)cu_f->cuModuleUnload(s->module);
s->module = NULL;
}
}

static int init_fex_st(VmafFeatureExtractor *fex, enum VmafPixelFormat pix_fmt, unsigned bpc,
unsigned w, unsigned h)
{
Expand Down Expand Up @@ -550,10 +564,12 @@ static int init_fex_st(VmafFeatureExtractor *fex, enum VmafPixelFormat pix_fmt,
return 0;

free_all:
release_cuda_module_and_stream_st(s, cu_f);
free_cuda_buffers_st(s, cu_f);
return err;
fail_pop:
(void)cu_f->cuCtxPopCurrent(NULL);
release_cuda_module_and_stream_st(s, cu_f);
free_cuda_buffers_st(s, cu_f);
return _cuda_err;
fail:
Expand Down Expand Up @@ -715,13 +731,10 @@ static int close_fex_st(VmafFeatureExtractor *fex)
SpeedTemporalCudaState *s = fex->priv;
if (fex->cu_state && fex->cu_state->f) {
free_cuda_buffers_st(s, fex->cu_state->f);
if (s->stream)
(void)fex->cu_state->f->cuStreamDestroy(s->stream);
release_cuda_module_and_stream_st(s, fex->cu_state->f);
}
if (s->feature_name_dict)
vmaf_dictionary_free(&s->feature_name_dict);
if (fex->cu_state && fex->cu_state->f && s->module)
(void)fex->cu_state->f->cuModuleUnload(s->module);
return 0;
}

Expand Down
12 changes: 12 additions & 0 deletions docs/backends/cuda/overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -159,6 +159,18 @@ Adding a new CUDA extractor: see [`/add-feature-extractor`](../../../.claude/ski
that amortises `cuStreamSynchronize` over 8 frames rather than 1, superseding
the engine-level optimization for this extractor.

### Error-path resource release (PR #1279)

Every CUDA feature extractor releases what it has already acquired when `init`
or `submit` fails part-way: pinned host buffers, device buffers, the module and
the stream, in the reverse order of acquisition and each one NULL-guarded. The
`speed_chroma` / `speed_temporal` twins share one `release_cuda_module_and_stream()`
helper so the two failure labels cannot drift apart again (PR #1007 added the
calls inline and PR #1029 removed them the same day); `integer_ms_ssim` and
`integer_psnr_hvs` release their pinned buffers on the same labels. There is no
user-visible behaviour change: a failed init still returns the same error code,
it just no longer leaks (docs/state.md `T-CUDA-INIT-SUBMIT-LEAKS-2026-06-19`).

## Profiling

See [nvtx/profiling.md](../nvtx/profiling.md) for Nsight Systems recipes that
Expand Down
11 changes: 11 additions & 0 deletions docs/rebase-notes.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,17 @@
<!-- markdownlint-disable MD001 MD003 MD004 MD007 MD013 MD018 MD022 MD024 MD025 MD026 MD028 MD029 MD031 MD032 MD033 MD036 MD037 MD038 MD040 MD041 MD046 MD049 MD050 MD051 MD052 MD053 MD055 MD056 MD058 MD059 -->
# Rebase notes

## fix/gpu-init-leaks-and-hip-mirror — fix CUDA init error path leaks and close verified GPU state issues (2026-09-04)

no rebase impact: fork-local CUDA and documentation files.

- `core/src/feature/cuda/speed_chroma_cuda.c`, `core/src/feature/cuda/speed_temporal_cuda.c`,
`core/src/feature/cuda/integer_ms_ssim_cuda.c`, `core/src/feature/cuda/integer_psnr_hvs_cuda.c`:
wholly fork-added CUDA feature extractors (upstream Netflix/vmaf does not have GPU SpEED, and
upstream CUDA extractors lack these fork-specific teardown paths). No upstream rebase conflicts.
- `docs/state.md`: closed out resolved audit tasks `T-HIP-MOTION-V2-MIRROR-OFF-BY-ONE-2026-06-13`,
`T-SYCL-INIT-LEAKS-EXC-2026-06-19`, `T-SPEED-GPU-REGISTRY-ORPHAN-2026-06-19`, and
`T-CUDA-INIT-SUBMIT-LEAKS-2026-06-19`.
<!-- markdownlint-disable MD001 MD003 MD004 MD007 MD013 MD018 MD022 MD024 MD025 MD026 MD028 MD029 MD031 MD032 MD033 MD036 MD037 MD038 MD040 MD041 MD046 MD049 MD050 MD051 MD052 MD053 MD055 MD056 MD058 MD059 -->
## fix/vmaftune-state-bugs — libx264 two-pass CRF conflict fix (2026-09-03)

Expand Down
Loading
Loading