Skip to content

F1: one source bus — every adapter must publish real VideoFrame pixels + PCM #535

Description

@iamfatness

Problem

The UI treats Zoom, UVC, SRT, media, and capture cards as first-class sources. The core still merges private pipes on a tick. Most non-synthetic kinds do not guarantee populated VideoFrame.pixels / PCM on one contract.

Documented as F1 — Real frame-pixel transport in docs/native-production-completion-plan.md. There was no issue until 2026-09-17.

What exists today

Done when

  1. One adapter contract: source_id, format, dimensions, timestamp, populated pixels, PCM, health, clock offset.
  2. Zoom SHM reader, UVC, test pattern, and media decode all land on that contract.
  3. Compositor / ISO / encode consume only that bus — no per-kind special case for empty frames.
  4. Snapshot reports framesIngested / droppedFrames per source from real counters.
  5. Stub build stays green.

Not this issue

MXL receive, Kubernetes workers, DeckLink vendor SDK (child issue), SRT decode (child issue). Those adapters plug in after this contract exists.

Rank

Post-beta / Later until the owner promotes it. Do not start MXL or a fifth private pipe before this lands.

Activity

  1. added 14 commits that reference this issue on Sep 17, 2026
  2. 23 remaining items

  3. added a commit that references this issue on Sep 21, 2026
  4. iamfatness commented on Sep 21, 2026

    @iamfatness
    OwnerAuthor

    Owner priority update is on main through #580: #535 first, #555 second, #513 idle validation deferred after an overnight run without recurrence.

    Draft #577 implements the Zoom-audio delivery slice. Pre-polled PCM passes through the participant source bus, audio and video polls have separate cadences, audio-only guests keep identity across camera changes, and snapshots expose actual PCM ingestion. Engine polling remains outside coreMutex; no added queue or retimestamping. The final native suite passed 1,155/0, including five audio-bus tests and D3D resolution-ramp pixel coverage relevant to #555.

    Acceptance is NOT complete. The clap harness first failed with silence on both baseline and candidate; its missing subscription is repaired in #577 (#578). The corrected 50ms content gate fails on both: candidate median audio lag 74.0ms (repeat 73.5ms), released baseline 95.1ms. #579 records the evidence; a separate matched-content encoder probe aligned to one sample, narrowing investigation upstream. No production timing compensation or reduced priming was installed.

    PR remains draft pending sync acceptance and final-head CI. Media/capture audio and legacy producer-interface retirement still remain after this Zoom slice. The installed beta is unchanged, rejoined to the meeting, and passed six feed/meter checks; it does not contain this new audio-bus code.

  5. iamfatness commented on Sep 21, 2026

    @iamfatness
    OwnerAuthor

    Updated PR #577 now passes the recorded content-sync gate that blocked the Zoom PCM slice. Full Release native suite: 1,159 passed / zero failed. Four 24-second clap recordings (eight events each) stayed inside 50 ms, including 30/60 fps input and 2/3-frame Program buffer configurations. Detailed ranges and timing tradeoff are in #579 and the rewritten PR description.

    PR head f61eb53 merges main's priority-document history, resolving its conflict; its tree is identical to tested/package commit 0c91afd. CI is running on the new head. Keep draft pending real-source acceptance; #535's broader adapter lifecycle and interface retirement remain outstanding.

    Local test ZIP is prepared at C:/Users/walla/cvp-535-audio/artifacts/releases/beta-2026-09-21-0c91afd/CoreVideoPro-win-x64-beta-2026-09-21-0c91afd.zip with TEST-CANDIDATE.txt alongside. 849 payload hashes and bundled WinUI runtime probe passed; packaged native handshake passed. Legacy CPU-preview smoke failed on both baseline and candidate, so is not claimed as a pass. No app replacement or public release occurred.

  6. added a commit that references this issue on Sep 21, 2026
  7. iamfatness commented on Sep 21, 2026

    @iamfatness
    OwnerAuthor

    PR #577 is merged into main as db9e703. Exact-main CI and CodeQL passed.

    Published and installed beta: https://github.com/iamfatness/CoreVideoPro/releases/tag/beta-2026-09-21-db9e703

    All 849 installed payload hashes and all six downloaded release assets verified. Installed shell/native core start successfully and produce Program frames. The automated clap gate passed 32 paired events within 41 ms; real-meeting lip-sync acceptance remains pending. This release does not close broader #535 migration or #555/#581 visual acceptance.

  8. iamfatness commented on Sep 22, 2026

    @iamfatness
    OwnerAuthor

    Owner update 2026-09-22: live lip sync looked good and first Takes showed no oddness in the last-night test. #579 and #555 are now closed against the shipped fixes and this live acceptance. The next priority is the remaining #535 work: media/capture PCM on the source bus, adapter lifecycle, and retirement of old interfaces; SRT/NDI hardening (#538) follows. Code audit confirms Zoom PCM already uses SourceBus::ingestAudio, while capture endpoint/transport PCM and MediaTransports::popAudio still append directly in gatherAudioOutputWork. Preserve existing PCM timestamps, routing identities, audio priming and media pause/restart behavior during migration. #535 remains open.

  9. iamfatness commented on Sep 22, 2026

    @iamfatness
    OwnerAuthor

    Media/capture PCM migration is ready for review in #583. All 1,167 native tests pass. The real decoder/Program recording check passed the first-frame Take oracle, published 130 media PCM packets / 124,800 samples on the bus, and preserved the 1 kHz fixture tone against released db9e703 (RMS ratio 0.99989).

    Capture membership now survives configured PCM gaps and video loss without retaining a stale picture; retirement and exactly-once PCM consumption have regression coverage. This does not close #535: remaining adapter lifecycle/interface retirement and physical-device acceptance still need completion. No beta release/install was changed. Owner order remains #535, then #538; #582 remains paused pending Zoom feedback.

  10. added a commit that references this issue on Sep 22, 2026
  11. iamfatness commented on Sep 22, 2026

    @iamfatness
    OwnerAuthor

    PR #583 is merged into main at 6782b09 and published/installed as beta-2026-09-22-6782b09: https://github.com/iamfatness/CoreVideoPro/releases/tag/beta-2026-09-22-6782b09 . Exact-main CI and CodeQL passed (macOS stub required one retry, tracked separately in #584). Installed payload hashes, WinUI runtime probe, media first-frame Take/recording and recorded tone checks passed. Remaining #535 lifecycle/interface work stays open; physical capture acceptance is still required. #584 is an unranked CI follow-up and does not change the owner order.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    backlogRanked in docs/BACKLOG.md

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions