Repository navigation
Zoom thumbnail packaging serializes Program frame fetch under runtime mutex #832
Description
Activity
The #823 mixed-source tests still exceed CPU-tail/deadline gates. Add independent default-off CPU attribution for Zoom configuration/roster/decoded-frame fetch and video publication: lock wait versus body, with thumbnail encoding measured as a subset of publication-body time. Fixed numeric records carry same-domain start/acquire/end timestamps and use the existing bounded asynchronous logger after the local guard releases. No ownership, GPU, playout, thumbnail, quality or buffering behavior changes.
The first real consumers are those runtime paths and the maintained mixed-source QA harness. It pins the explicit flags, retains logger refusal counters, and offers normal logging without changing the historical verbose default. The trace-overhead checker now rejects unknown/mismatched logging choices. Preserve thumbnail timestamp evaluation order and stale-generation behavior.
Validation on f5987f2 (native b08b1f3018fcbb92f407b7abcecf001c787f449f): Release build; all 1,520 native tests pass (external NDI/network opt-ins were not executed); 59 runtime/state tests pass with the flag enabled; 24 portable monitor/timing tests and diff checks pass. Final exact-head CI remains in progress.
The final clean normal-logging pair delivered 14,410 measured Program packets with zero native-buffer, recording or raw-trace loss. It observed 290 slow publication records and 85 slow fetch waits; 76 waits overlapped a publication body for at least 95% of their duration. Those selective CPU records include warmup/shutdown. Both processes reported 7 dropped logger records (0 truncated/sink failures); absent scopes cannot exclude a stall. Raw traces finalized with zero losses/export failures. Seven/five post-measurement-window Program misses are retained in the complete traces, separately from the measured verdict.
CPU-tail qualification still fails: inline p99 bounds 9.792–10.065 ms / p99.9 10.903–14.223 ms; isolated p99 8.198–8.465 ms / p99.9 9.211–13.245 ms. The earlier normal-logging pair retained 34/27 render deadline misses despite zero output loss. A clean average or zero measured losses does not close the parent gates.
This supports the shared-runtime-mutex/optional-thumbnail bottleneck filed as #832, with an off-lock packaging repair proposal and owner ranking pending. It does not attribute the separate retained 77 ms encoder stall/five delivery losses, establish kernel wait owner/GPU duration/driver cause, or close #823/#517. Installed/receiver/display, actual content latency, A/V, trace cost and fleet gates remain open. No beta, defaults, settings or installation changed.
During owner-approved #517/#823 qualification, normal-logging mixed-source trials reproduce recurring ingress stalls even with CPU source preparation on. Source aabbdb1, native tree 23b3293b4d826e1c091018699498a441aab96a7e, Release core SHA 714C70B7A39C4A8329839086769904B0F9D1594D8446F84258366AA87BE6298B. Eight synthetic Zoom I420 1080p30 inputs, two BGRA 1080p60/1440p60 mappings, 1080p60 Program with two-frame buffer, local MP4; normal logging off for ordinary verbose diagnostics and independent explicit CPU scopes/trace on.
Observed fixed numeric records: publication-body 12.074–12.985 ms under ZoomEngineRuntime::mutex_, six thumbnail base64 encodes contributing 6.474–7.061 ms, and Program configured() waits 10.457–12.321 ms for that same mutex. This is reproducible CPU serialization, including any OS preemption in the measured intervals. It does not prove which mutex owner caused each individual wait, or explain the distinct retained 77 ms encoder export stall/five delivery losses from #823.
Source confirms base64Encode and construction/deep copies of the large thumbnail JSON event still occur inside publishVideoFrameLocked, called within the three-phase publication mutex. Program configured(), compositor roster poll and latestDecodedVideoFrames acquire that mutex. Conversion and I420 ownership were already moved off the lock, but optional thumbnail packaging remains coupled to Program and audio fetch.
Proposed focused repair: construct and encode the owned thumbnail event in the existing unlocked snapshot phase; publish only validated generation/stream facts, small timestamps and an immutable moved event under the mutex. Preserve stale-generation rejection, source epochs, video playout, thumbnails and CPU/ISO paths. Use deterministic tests that block optional packaging off-lock while Program fetch completes and replace/resize a stream before publication. Compare matched real-pixel mixed-source normal-logging trials before/after; preserve failures and all parent gates.
Done when the off-lock packaging and stale-generation/pixel tests pass, exact-head CI is green, and matched evidence shows the recurring thumbnail-driven lock delay removed without delivery, identity, A/V or latency regression. Parent #517/#823 qualification remains separate. Unranked intake; no repair or default/release promotion in this finding. #831 is diagnosis only.