Skip to content

Conversation

@mickel8
Copy link
Contributor

@mickel8 mickel8 commented Dec 11, 2023

This PR unfortunately doesn't work and I ran out of ideas on how to push it further. Current status:

  • chromium successfully decodes and displays the first video frame
  • I can't see any logs indicating we are doing something wrong. I ran dev build of chromium. The only thing I can see is
[2098991:3:1211/162436.394069:WARNING:dav1d_decoder.cc(135)] Dav1dDecoder::Decode decoding failed with error code -11
[2098991:3:1211/162436.394213:WARNING:generic_decoder.cc(302)] Failed to decode frame with timestamp 2618223154, error code: -1

According to dav1d IIRC this might not be the error but I think it shouldn't be logged
image

  • I assumed that sending every OBU in a separate RTP packet is ok
  • I assumed that every frame is a separate Temporal Unit (in the test file this should be true if only the code checking OBU type is correct) and therefore I set marker of the last RTP packet for every IVF frame
  • I tried batching OBUs e.g. by 2 or setting W flag to 1 (instead of 0 and using LEB size) - didn't help
  • I tried prepending every OBU with sequence header - this gives a massive number of decoder errors (-11)
  • Pion send_from_file example also doesn't work but here I am getting
[2077476:12:1211/123520.805336:WARNING:video_rtp_depacketizer_av1.cc(314)] Mismatch in obu_size. signaled: 0, actual: 591
[2077476:12:1211/123520.838810:WARNING:video_rtp_depacketizer_av1.cc(314)] Mismatch in obu_size. signaled: 0, actual: 3
[2077476:12:1211/123520.871969:WARNING:video_rtp_depacketizer_av1.cc(314)] Mismatch in obu_size. signaled: 0, actual: 33
[2077476:12:1211/123520.904092:WARNING:video_rtp_depacketizer_av1.cc(314)] Mismatch in obu_size. signaled: 0, actual: 3
[2077476:12:1211/123520.936935:WARNING:video_rtp_depacketizer_av1.cc(314)] Mismatch in obu_size. signaled: 0, actual: 62
[2077476:12:1211/123520.970613:WARNING:video_rtp_depacketizer_av1.cc(314)] Mismatch in obu_size. signaled: 0, actual: 3
[2077476:12:1211/123520.996106:WARNING:video_receive_stream2.cc(825)] No decodable frame in 230195 us, requesting keyframe.
[2077476:12:1211/123521.003523:WARNING:video_rtp_depacketizer_av1.cc(314)] Mismatch in obu_size. signaled: 0, actual: 33
[2077476:12:1211/123521.037086:WARNING:video_rtp_depacketizer_av1.cc(314)] Mismatch in obu_size. signaled: 0, actual: 3
[2077476:12:1211/123521.069772:WARNING:video_rtp_depacketizer_av1.cc(314)] Mismatch in obu_size. signaled: 0, actual: 180
[2077476:12:1211/123521.102389:WARNING:video_rtp_depacketizer_av1.cc(314)] Mismatch in obu_size. signaled: 0, actual: 3
[2077476:12:1211/123521.135413:WARNING:video_rtp_depacketizer_av1.cc(314)] Mismatch in obu_size. signaled: 0, actual: 28
[2077476:12:1211/123521.168366:WARNING:video_rtp_depacketizer_av1.cc(314)] Mismatch in obu_size. signaled: 0, actual: 3
[2077476:12:1211/123521.201628:WARNING:video_rtp_depacketizer_av1.cc(314)] Mismatch in obu_size. signaled: 0, actual: 95
[2077476:12:1211/123521.214954:WARNING:video_receive_stream2.cc(825)] No decodable frame in 218582 us, requesting keyframe.
  • if I don't drop Temporal Units, I can see
[2080197:11:1211/130520.654168:WARNING:rtp_seq_num_only_ref_finder.cc(65)] Generic frame with packet range [51605, 51605] has no GoP, dropping frame.
  • maybe this has something to do with this paragraph of AV1 RTP format specification (see here)
If a sequence header OBU is present in an RTP packet and operating_points_cnt_minus_1 > 0 then for any number i where 0 <= i < operating_points_cnt_minus_1 the following MUST be true: (operating_point_idc[i] & operating_point_idc[i+1]) == operating_point_idc[i+1].

A sender MAY produce a sequence header with operating_points_cnt_minus_1 = 0 and operating_point_idc[0] = 0xFFF and seq_level_idx[0] = 0. In such case, seq_level_idx[0] does not reflect the level of the operating point.

Note: The intent is to disable OBU dropping in the AV1 decoder. To ensure a decoder’s capabilities are not exceeded, OBU filtering should instead be implemented at the system level (e.g., in a MANE).

but because it sounds pretty complex I didn't dive into it

@mickel8 mickel8 changed the title wip AV1 payloader Dec 11, 2023
@mickel8 mickel8 mentioned this pull request Dec 14, 2023
63 tasks
@sgfn sgfn mentioned this pull request Jul 2, 2025
@mickel8
Copy link
Contributor Author

mickel8 commented Jul 3, 2025

Moved to #222

@mickel8 mickel8 closed this Jul 3, 2025
@sgfn sgfn deleted the av1-payloader branch July 3, 2025 10:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants