Repository navigation
Live ingest: SPS/PPS/IDR gate needed for mid-stream H.264 joins (green frames / total muxer failure), plus DisplayCriteria SET/waitForSwitch bypass for unchanged-rate live zaps #133
Description
What happened?
Two related but distinct issues found during live TV testing, both on the live HLS ingest path (HLSSegmentProducer, HLSVideoEngine).
Issue A — SPS/PPS/IDR gate missing on live join. When the ingest joins a live MPEG-TS stream mid-broadcast, the existing keyframe gate (video gate open, gated on AV_PKT_FLAG_KEY) opens on any flagged keyframe without requiring valid SPS/PPS to already be parsed, or verifying the keyframe is a true IDR with usable parameter sets. Two severities observed:
Mild: decoder briefly renders with an uninitialized/garbage reference (green frame) before self-correcting once real SPS/PPS/IDR arrive. Seen on multiple channels.
Severe: SPS/PPS loss is complete enough that frame dimensions never resolve at all, allocateMuxer gets 0x0 from probe-time codecpar, avformat_write_header fails, and the channel produces zero segments — an empty #EXTM3U playlist that never recovers without the user manually zapping away.
I expected the engine to withhold packets until a decodable SPS/PPS + IDR sequence is established (the way H264SPS.extractSPSandPPS / extractAdVideoConfig already does for SSAI ad-pod codec switches elsewhere in the same file) — instead the initial live join has no equivalent gate, so both failure modes above can occur.
Issue B — DisplayCriteria RESET defeats the engine's own zap optimization. Custom app's live-zap teardown was calling aetherEngine.stop() with default resetDisplayCriteria: true before loading the next channel. This nils preferredDisplayCriteria, forcing the subsequent load()'s apply() to renegotiate the display mode from scratch even when the outgoing and incoming channel have identical rate/format — and on our panel (unobservable-DV switch signaling) that renegotiation burns the full ~3000ms waitForSwitch() cap every time, dominating zap latency (ttff_ms consistently 2.5–3.0s for load_profile=liveZap).
We fixed our side by calling stop(resetDisplayCriteria: false) on the zap path instead of the dismiss path, which matches the intent of the existing stopInternal(resetDisplayCriteria: false) internal preservation (the #128 fix) — RESET no longer fires between zaps, only on player dismissal. However, apply() and the post-load waitForSwitch() are unconditional inside load() with no LoadOptions hook, so the SET line and its associated wait still fire even when the criteria being reapplied are identical to what's already active. There's no way to make the engine skip that redundant work from the host side.
Steps to reproduce
Issue A:
- Tune a live HLS ingest session (isLive: true) against an IPTV/Xtream-style source mid-broadcast (i.e. joining an already-running MPEG-TS stream, not a fresh start).
- Observe demuxer logs: non-existing SPS 0 referenced in buffering period, sometimes followed by non-existing PPS 0 referenced / no frame!.
- Mild case: first rendered frame(s) are green before correcting. Severe case: Demuxer stream[0] type=video codec=h264 0x0, [mp4] dimensions not set, HLSSegmentProducer muxer alloc for seg-0 failed: MP4SegmentMuxer: avformat_write_header failed (-22), pump finished: reason=muxerFailed packetsRead=2 packetsWritten=0. Channel never produces a segment.
- Reproduced repeatedly on specific channels (see logs); not universal — appears tied to how cleanly the source's SPS/PPS cadence aligns with the ingest's join point.
Issue B:
- Tune a live channel, then zap to another live channel of the same format/rate (e.g. two SDR 50Hz channels).
- Observe [DisplayCriteria] RESET immediately followed by [DisplayCriteria] SET: format=sdr codec=hvc1 rate=50.000 extensions=none and [DisplayCriteria] proceed after ~3000ms cap (switch not observable, likely DV; EDR headroom 1.00) on every single zap, regardless of whether the rate actually changed.
- ttff_ms for load_profile=liveZap lands at 2.5–3.0s almost universally as a result.
- After our app-side fix (stop(resetDisplayCriteria: false) on zap), RESET no longer appears between zaps, but SET + the cap wait still do, since apply()/waitForSwitch() are unconditional.
AetherEngine version or commit SHA
5.5.1
Host app
Custom / my own integration
Platform
tvOS
OS version
tvOS 26.6 (beta)
Device / chip
Apple TV 4K (A15)
Playback path
Native AVPlayer
Source media (for playback bugs)
Various UK IPTV channels, MPEG-TS source, H.264 (AVC) video at multiple resolutions (720x576 SD up to 1920x1080), mixed AAC/AC-3/EAC-3 audio depending on channel. The defect in Issue A is not resolution- or codec-profile-specific — it's about join timing relative to the source's SPS/PPS/IDR cadence, so it can occur mid-broadcast on any H.264 channel from this provider. Happy to provide ffprobe output against a specific reproducing channel if useful, though the issue is inherently about the moment of join rather than the stream's steady-state characteristics.
Error codes / log lines
Issue A (mild):
[h264 @ 0x...] [error] non-existing SPS 0 referenced in buffering period
Last message repeated 1 times: [h264 @ 0x...] [error] non-existing SPS 0 referenced in buffering period
Issue A (severe):
[h264] non-existing SPS 0 referenced in buffering period
[h264] non-existing PPS 0 referenced
[h264] no frame!
[Demuxer] Opened: 2 streams, duration=-9223372036854775808 us
[Demuxer] stream[0] type=video codec=h264 0x0
[HLSVideoEngine] video codecpar: codec=27 0x0 profile=-99 level=-99 fieldOrder=0 extradata=0B head=none
[mp4] dimensions not set
[HLSSegmentProducer] muxer alloc for seg-0 failed: MP4SegmentMuxer: avformat_write_header failed (-22)
[HLSSegmentProducer] pump finished: reason=muxerFailed packetsRead=2 packetsWritten=0
Issue B:
[DisplayCriteria] RESET
...
[AetherEngine] load url=aether-custom://source source-format=sdr effective-format=sdr rate=50.000
[DisplayCriteria] SET: format=sdr codec=hvc1 rate=50.000 extensions=none
[DisplayCriteria] proceed after ~3000ms cap (switch not observable, likely DV; EDR headroom 1.00)
MEDIAAPP_AETHER event=ttff load_profile=liveZap ttff_ms=2965
Anything else
On Issue A — proposed fix shape, composing with existing mechanisms rather than replacing them:
- Live-only pre-gate: hold video packets until in-band SPS and PPS are observed (reuse the existing H264SPS parsing already used for SSAI) and a real IDR arrives after those sets. Buffering a few hundred ms to a couple of seconds at join is an acceptable cost — far better than a green flash or a dead channel.
- When probe-time codecpar is 0x0 at first allocateMuxer, populate muxer config from the in-band SPS/PPS instead (same pattern as extractAdVideoConfig for ad-pod codec switches).
- Fallback: if muxer init still fails on unresolved dimensions, escalate to the engine's existing reopen/retune path rather than leaving an empty #EXTM3U.
- Suggested logs: SPS/PPS/IDR gate: holding (n packets buffered) / SPS/PPS/IDR gate: released after Nms.
VOD is unaffected by design (full-file probe already has this information); this is scoped to live ingest only. The existing PTS-based video gate and open-GOP RASL-drop logic should remain as-is — this is an additional precondition ahead of them, not a replacement.
On Issue B — proposed fix shape: a LoadOptions flag (e.g. preserveDisplayCriteriaIfUnchanged) that lets load() compare the criteria it's about to apply() against what's already active and skip both apply() and the post-load waitForSwitch() when they match, emitting something like [DisplayCriteria] skipped SET reason=rate_unchanged. This is a narrower ask than a blanket suppressDisplayCriteria-style override — it only short-circuits when the reapply is genuinely redundant, so it shouldn't affect HDR/DV signaling correctness on any zap that actually changes format.
Activity
fix(live): gate mid-stream H.264 joins on a decodable IDR access unit (…
fix(display): skip redundant DisplayCriteria apply on unchanged-forma…
Merge fix/133-live-join-gate-and-criteria-skip: live H.264 join gate …
superuser404notfound commentedon Jul 17, 2026
Both parts are fixed in 5.6.2 (
adbc4c1). Thanks for the exceptionally precise report; the log lines and the distinction between the mild and severe modes made the root cause obvious in both cases.Issue A: SPS/PPS/IDR join gate
Confirmed root cause exactly as you diagnosed: the video gate opened on the first
AV_PKT_FLAG_KEYpacket with no check that the access unit was decodable. Joining mid-broadcast, that admitted either an open-GOP recovery point (the green flash, references not yet established) or, when the probe joined before any SPS, a packet under a 0x0codecparthat then failedavformat_write_header(-22) at the first muxer alloc. On the live pathmuxerFailedhas no recovery arm, hence the empty#EXTM3Uthat never recovered.The fix follows your proposed shape, composing with the existing gate rather than replacing it:
avcC, VOD probes the whole file), the gate now withholds video until a packet carries in-band SPS + PPS and a true IDR slice (NAL type 5, not a recovery-point I-slice). It reuses the sameH264SPSparsing the SSAI ad-pod path already used, plus a smallcontainsIDRscan. The existing PTS gate and open-GOP RASL drop are unchanged; this sits ahead of them.extractAdVideoConfigmechanism, generalized), sowrite_headergets real dimensions.keyframeStarvationand reopens/retunes. That path already has a real recovery arm, unlikemuxerFailed. So a genuinely mis-signaled source reopens instead of dead-ending.You should see
still waiting for SPS/PPS/IDR access unit ...while it holds andlive join: reconstructed video config WxH from in-band SPS/PPSwhen the severe case is caught.Issue B: DisplayCriteria on unchanged-rate zaps
Your host-side change (
stop(resetDisplayCriteria: false)on the zap path) is exactly right and matches the intent of the #128 preservation, so RESET no longer fires between zaps. That left the redundantSET+waitForSwitch, which is the part the engine now handles.Rather than a
preserveDisplayCriteriaIfUnchangedLoadOptionsflag, this is done internally: the controller retains the last-applied criteria and skips both the panel write and the post-load settle wait when the incoming criteria are already active. It is a narrower and safer contract than a flag would be. It only ever short-circuits when the reapply is genuinely redundant (same format, rate, codec, and dynamic range, with no intervening RESET), it cannot skip a switch that actually changes anything, and every consumer benefits without opting in or reasoning about when the flag is safe. On your panel this removes the full~3000mscap from same-format zaps; any zap that changes format/rate/range settles exactly as before. Log line on the skip:skipped SET reason=unchanged.Would appreciate a retest against 5.6.2, both the mid-stream join behaviour on the channels that reproduced and the
ttff_msonliveZapbetween two same-format channels.fix(test): break up H264 Annex-B concat to fix type-check timeout (#133)
digilearn-dev commentedon Jul 18, 2026
Following up on this one — the join-gate fix is working as far as it goes, but we're seeing the same underlying condition recur mid-stream on some channels, not just at the initial join.
Retested across both 5.6.2 and 5.8.4 (bumped today, no material change in behavior between the two). The severe-case recovery you added (live join: reconstructed video config...) does fire and does correctly recover the zero-dimension case — that part's solid. But on some channels the same non-existing SPS 0 referenced / non-existing PPS 0 referenced / no frame! sequence keeps recurring well after the session is already readyToPlay, in some cases in long unbroken runs:
[h264] non-existing PPS 0 referenced
Last message repeated 65 times: [h264] non-existing PPS 0 referenced
[h264] non-existing PPS 0 referenced
Last message repeated 44 times: [h264] non-existing PPS 0 referenced
[h264] non-existing SPS 0 referenced in buffering period
[h264] non-existing PPS 0 referenced
[HLSSegmentProducer] live join: reconstructed video config 1920x1080 from in-band SPS/PPS (probe codecpar was 0x0)
Last message repeated 18 times: [h264] non-existing PPS 0 referenced
(BBC One SD, ITV 1 SD, and one other channel in the same session — all UK terrestrial MPEG-TS via Xtream.)
On our end this shows up as a real, visible problem: Channel 4 flickers green recurrently throughout playback, not just at tune-in. The volume of consecutive identical decode errors (up to 65 in a row within one tune) doesn't look like something a one-time join gate should need to absorb — it looks like something mid-stream is repeatedly resetting decoder parameter-set state, the same way a fresh join does. Ad-break splices or encoder restarts on the source side would be my first guess, but I don't have visibility into the provider's encoding pipeline to confirm.
Happy to capture whatever would help pin this down — a longer dedicated single-channel dwell (several minutes on Channel 4 specifically), discontinuity/SCTE markers from the source playlist if that's useful, or a raw packet-level capture at the moment of a flicker. Let me know what's most useful and I'll get it.
Separate but possibly related, flagging in case it shares a cause: the skipped SET reason=unchanged dedup from the same fix is still only firing on roughly a third to half of eligible same-format zaps (no improvement 5.6.2 → 5.8.4) — happy to split this into its own issue if it's unrelated, just didn't want to lose the observation.
Merge fix/issue133-samepid-parameter-set-rotation: versioned-init rot…
chore(release): 5.8.5
superuser404notfound commentedon Jul 18, 2026
Fixed in 5.8.5.
The join gate only covered the initial join; the recurrence you saw is a distinct case with the same symptom. Confirmed root cause: the fMP4
avcC(the out-of-band SPS/PPS the panel decodes against) is frozen atavformat_write_headerfor the muxer's lifetime. The versioned-init rotation that re-establishes a freshavcC(a newEXT-X-MAP) only fired for an SSAI program switch, i.e. an ad creative arriving on a new video PID. An in-band parameter-set change on the same PID (an encoder restart or a regional opt-out splice, which is what these UK terrestrial channels do) forced only a discontinuity cut and left the panel decoding the new slices against the staleavcC. That matches your "something mid-stream is repeatedly resetting decoder parameter-set state, the same way a fresh join does" reading, and the old code even flagged the gap in a comment ("expect decode artifacts until the versioned-init path exists").The fix: each mid-stream keyframe now parses its in-band SPS/PPS and compares them against the sets backing the current
avcC. On a divergence it rotates the muxer through the same versionedEXT-X-MAPpath, so the new init segment matches the new slices. The sets are parsed straight from the bitstream, so it fires whether or not the demuxer surfaces the change asAV_PKT_DATA_NEW_EXTRADATA(on MPEG-TS it usually does not). A same-PID change keeps the program's Dolby Vision / colour signaling; only an ad creative on a new PID drops the program overrides.One thing your logs left open, and the reason 5.8.5 also ships diagnostics: the trace shows
live join: reconstructed video config ...firing mid-run, which is the initial-join path and only runs on a fresh producer. So there are also producer reopens happening well into the session, and the visible flicker could be the same-PID rotation case above, a reopen storm, or both. On a dedicated Channel 4 dwell, the useful lines to capture are:same-PID in-band parameter-set change #N ...the new fix firing in place (the good path).video gate open: ... videoPID=X reconstructed=Yone per epoch; a burst of these is a reopen storm, a single one with in-place rotations is not.diag: demuxer emitted AV_PKT_DATA_NEW_EXTRADATA ...whether the side-data path fires at all on these streams.muxer rotation (same-PID parameter-set change)/versioned init capturedline around a flicker.If the flicker still tracks the reopen lines rather than the rotation lines, the SCTE / discontinuity markers from the source playlist you offered are the next thing to look at, but the 5.8.5 logs come first.
On the separate DisplayCriteria observation (unchanged-skip firing on only a third to a half of eligible zaps): keeping it here is fine. 5.8.5 adds a
DisplayCriteria diag no-skip: signature diverged rate ... codec ... hdr ... ext ...line that prints the exact field that differs when a same-format zap does not skip, so the retest will show whether it is the refresh rate, the codec tag, or the baseline being cleared between zaps.Appreciate the detail as always.
fix(live): rotate versioned init on same-PID in-band parameter-set ch…
digilearn-dev commentedon Jul 20, 2026
Retested on 5.9.6 (also confirmed present on 5.10.0) — the routing fix works exactly as described. Confirmed on Channel 4:
[AetherEngine] fieldOrder=UNKNOWN but SPS frame_mbs_only_flag=0; treating as interlaced for routing (#150)
[AetherEngine] dispatch: codec=27 -> software
[Deinterlace] engaged [hardware]: 1920x1080 pixFmt=0 (format=nv12,hwupload,yadif_videotoolbox=mode=send_field:parity=auto:deint=interlaced)
Green cast is gone, picture is correct. Thank you for the fast turnaround on that.
New problem though, directly on this same path: no audio at all once routed to software.
[AetherEngine] renderer audio session active: sourceCh=unknown maxChannels=6 preferred=6 output=6
[SWHost] session start: videoCodecID=27 audioCodecID=none duration=0.0s
audioCodecID=none at session start, and there's no equivalent of the native path's audio-track-attach logging anywhere afterward in this session (video decodes and renders fine via SWDecoder/Renderer, but nothing establishes an audio track). Compare to a normal native-path session earlier in the same log, where audio comes up cleanly:
[NativeAVPlayerHost] #1 item.audioTrack codec='aac ' enabled=true sr=48000 ch=2 bits=0 fmt=aac layoutTag= (readyToPlay)
[NativeAVPlayerHost] #1 audioRoute output=6 preferred=6 max=6 ports=[Office[HDMIOutput, ch=2]] (settled)
My guess is this is exposure rather than a new regression — before #150, the software host presumably wasn't getting much real traffic (interlaced content mostly stayed misrouted to native), so an unwired/incomplete audio path there wouldn't have been noticed. Now that #150 correctly routes interlaced content to software more often, it's hitting real streams and coming up silent.
Reproduces consistently on this channel; happy to test against others or provide more log context if useful.
superuser404notfound commentedon Jul 20, 2026
Fixed in 5.15.1.
This was a real gap the #150 routing change exposed rather than caused, exactly as you read it: before #150 the software host saw almost no live traffic, so its audio path never got exercised on these streams.
Root cause: the software host resolved its audio track only through
av_find_best_stream, which returns -1 for a live MPEG-TS AAC stream whose codec parameters the probe left empty (sample rate and channel count zero, because stream analysis stops before the first audio frame is decoded). With no index, no decoder was opened and the audio packets were dropped, henceaudioCodecID=noneat session start andsourceCh=unknown. The native path already handled this (it falls back to the first audio-type stream and repairs the AAC parameters); the software path never did, because it never needed to until #150.The software path now mirrors the native handling: on a live source, when
av_find_best_streamfinds nothing it falls back to the first audio-type stream, and it repairs a zero-rate AAC stream to 48 kHz stereo AAC-LC so the decoder opens and channel negotiation is correct. VOD selection is unchanged. New log lines on this path:audio: av_find_best_stream found no usable audio (live probe left empty codecpar?); falling back to first audio-type stream Naudio: live AAC had no codec parameters from the probe; assuming 48 kHz stereo AAC-LCA retest on Channel 4 and the other UK terrestrial channels that came up silent would confirm it. Thanks again for the precise before/after logs; the native-versus-software comparison made this a quick diagnosis.
digilearn-dev commentedon Jul 20, 2026
Confirmed on 5.15.1 — audio's back, picture stays correct, no regressions:
dispatch: codec=27 -> software
[SWHost] session start: videoCodecID=27 audioCodecID=86019 duration=0.0s
[Deinterlace] engaged [hardware]: ...
Both the routing fix (#150) and the audio fallback (#133) are solid on our end now. Appreciate the quick fixes across this whole thread — thank you.
fix(swhost): resolve live-TS audio the probe left with empty codecpar (…
fix(display): size the play-gate start grace by what can still switch (…
fix(display): no play-gate wait for a switch Match Content cannot start
feat(engine): a source that loses its audio says so, and says which s…
fix(live): open an H.264 join on an intra recovery point, not only on…