Repository navigation
fix(cuda): scope the drain batch to its engine and fence the indexed reads (Netflix/vmaf#1305) - #1321
Merged
Merged
Conversation
lusoris
force-pushed
the
fix/cuda-drain-batch-per-state-lifetime
branch
2 times, most recently
from
September 5, 2026 23:12
71c011d to
50a368b
Compare
…reads (Netflix/vmaf#1305) Two defects of the same shape, one row (docs/state.md T-UPSTREAM-1305-CUDA-DRAIN-BATCH-THREAD-GLOBAL-2026-09-03). Fork half: the ADR-0242 fence batch in core/src/cuda/drain_batch.c was `static _Thread_local` and keyed by nothing else, so two VmafContexts sharing an OS thread shared the batch. A context that closed left its CUevents and its `bool *` drained flags registered, and the next context's flush waited on destroyed events and wrote through freed pointers. The batch now carries the owning VmafCudaState: open() claims it and drops a previous owner's entries, flush() returns 0 without touching CUDA for a foreign owner and clears the entries it consumed, and thread_destroy() wipes entries, the open flag and the owner before the caller frees the engine state. Upstream half: vmaf_score_at_index() and vmaf_feature_score_at_index() read the feature collector with no CUDA sync, no drain flush and no thread-pool wait, so a caller draining index N-2 while N was in flight read unwritten slots. They now call fence_for_read() — worker-thread wait, drain flush, pending collect for indices <= the requested one — but only after the lock-free read reports the slot unwritten, so reads that hit written slots keep their cost. Verified on the RTX 4090: CUDA vs CPU on the Netflix src01 pair is unchanged (adm/vif/aim identical, motion2/motion3/vmaf 1e-6, the documented GPU tolerance); core/test/test_cuda_drain_batch.c (4 cases, needs nvcc but no device) pins the ownership rules.
lusoris
force-pushed
the
fix/cuda-drain-batch-per-state-lifetime
branch
from
September 6, 2026 00:01
50a368b to
d5ba773
Compare
lusoris
marked this pull request as ready for review
September 6, 2026 00:16
13 of 18 tasks
This was referenced Sep 6, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Closes
T-UPSTREAM-1305-CUDA-DRAIN-BATCH-THREAD-GLOBAL-2026-09-03, which is two defects of the same shape.Fork half. The ADR-0242 fence batch in
core/src/cuda/drain_batch.cwasstatic _Thread_localand keyed by nothing else, so twoVmafContexts sharing an OS thread shared the batch. A context that closed left itsCUevents and itsbool *drained flags registered, and the next context's flush waited on destroyed events and wrote through freed pointers. The batch now carries the owningVmafCudaState:open()claims it and drops a previous owner's entries,flush()returns 0 without touching CUDA for a foreign owner and clears the entries it consumed, andthread_destroy()wipes entries, the open flag and the owner before the caller frees the engine state.Upstream half (Netflix/vmaf#1305).
vmaf_score_at_index()andvmaf_feature_score_at_index()read the feature collector with no CUDA sync, no drain flush and no thread-pool wait, so a caller draining index N-2 while N was in flight read unwritten slots. They now call afence_for_read()helper — worker-thread wait, drain flush, pending collect for indices at or below the requested one — but only after the lock-free read reports the slot unwritten, so reads that hit written slots keep their cost and the streaming path keeps its throughput.Verified on the RTX 4090: CUDA vs CPU on the Netflix src01 pair is unchanged (adm2/adm3/adm_scale0-3/aim/vif_scale0-3 identical, motion2 / motion3 / vmaf 1e-6, the documented GPU tolerance); the Netflix golden gate is 271 passed / 12 skipped / 0 failed; the new unit test pins the ownership rules and needs nvcc but no device.
Type
fix— CUDA engine lifetime + public read-API correctnessChecklist
make format && make lintis green locally (pre-commit on every touched file;assertion-density.shPASS).core/test/test_cuda_drain_batch.c(4 cases,fastsuite, no device needed).docs/state.mdand the CUDAAGENTS.mdinvariant.vmaf_cuda_drain_batch_pending()is in the privatedrain_batch.h), no ADR (ADR-0242 already governs the batch).Bug-status hygiene (ADR-0165)
docs/state.md—T-UPSTREAM-1305-CUDA-DRAIN-BATCH-THREAD-GLOBAL-2026-09-03moved to Recently closed with both halves and the evidence.Netflix golden-data gate (ADR-0024)
assertAlmostEqual(...)score in the Netflix golden Python tests.Deep-dive deliverables (ADR-0108)
AGENTS.mdinvariant note —core/src/cuda/AGENTS.md§ Lifecycle invariants: the drain batch belongs to one engine at a time; never restore the owner-lessopen(void)signature.changelog.d/fixed/cuda-drain-batch-per-state-lifetime.md.docs/rebase-notes.mdentry (core/src/libvmaf.cis upstream-mirror; the two hunks there are fork-only).Reproducer
🤖 Generated with Claude Code