Fix side data type tables lagging FFmpeg

CodecContext.coded_side_data and decoded_side_data raise if a type has
no name in PyAV, and both tables had drifted: sidedata.Type was missing
five members and PktSideDataT three, including the AV_PKT_DATA_HEVC_CONF
that mov and matroska attach to HEVC streams, and the AV_PKT_DATA_EXIF.

Complete both tables, skip names a newer FFmpeg may add rather than
losing the whole dict to one of them.

Author: WyattBlue

CHANGELOG.rst

@@ -47,10 +47,11 @@ Features:
 - ``ContainerFormat.fixed_framesize`` reports whether a format wants fixed size audio frames.
 - :class:`.CodecContext` exposes more of ``AVCodecContext``: ``pkt_timebase``, ``frame_num``, ``active_thread_type``, ``bits_per_raw_sample``, ``compression_level``, ``rc_buffer_size``, ``min_bit_rate``, a setter for ``max_bit_rate``, the audio ``initial_padding``, ``trailing_padding``, and ``seek_preroll``, and ``stats_in``/``stats_out`` for two-pass encoding. ``VideoCodecContext`` gains ``chroma_sample_location``, ``refs``, and ``mb_decision``; ``AudioCodecContext`` gains ``block_align``.
 - ``CodecContext.coded_side_data`` and ``CodecContext.decoded_side_data`` expose the context's global side data as dicts of ``bytes``, keyed by packet side data name and :class:`~av.sidedata.sidedata.Type` respectively. Stream wide HDR metadata, such as mastering display and content light level, arrives in ``decoded_side_data`` once a frame has been decoded.
-- Enums gained the members FFmpeg has since added: ``Properties.FIELDS``, ``Properties.ENHANCEMENT``, ``PixFmtLoss.EXCESS_RESOLUTION``, ``PixFmtLoss.EXCESS_DEPTH``, ``Flags2.icc_profiles``, ``format.Flags.experimental``, ``Interpolation.STRICT``, ``Interpolation.UNSTABLE``, ``ColorTrc.V_LOG``, ``ColorPrimaries.V_GAMUT``, and the ``LCEVC``, ``VIEW_ID``, ``THREE_D_REFERENCE_DISPLAYS``, and ``EXIF`` members of ``sidedata.Type``.
+- Enums gained the members FFmpeg has since added: ``Properties.FIELDS``, ``Properties.ENHANCEMENT``, ``PixFmtLoss.EXCESS_RESOLUTION``, ``PixFmtLoss.EXCESS_DEPTH``, ``Flags2.icc_profiles``, ``format.Flags.experimental``, ``Interpolation.STRICT``, ``Interpolation.UNSTABLE``, ``ColorTrc.V_LOG``, ``ColorPrimaries.V_GAMUT``, the ``LCEVC``, ``VIEW_ID``, ``THREE_D_REFERENCE_DISPLAYS``, ``EXIF``, ``DYNAMIC_HDR_SMPTE_2094_APP5``, ``IAMF_MIX_GAIN_PARAM``, ``IAMF_DEMIXING_INFO_PARAM``, ``IAMF_RECON_GAIN_INFO_PARAM``, and ``RAW_COLOR_PARAMS`` members of ``sidedata.Type``, and the ``exif``, ``dynamic_hdr_smpte_2094_app5``, and ``hevc_conf`` packet side data names.
 
 Fixes:
 
+- ``Frame.side_data`` and ``PacketSideData.data_type`` no longer raise on side data types FFmpeg has added since PyAV last listed them. ``sidedata.Type`` was missing five members and the packet side data names three, so reading, say, the ``AV_PKT_DATA_HEVC_CONF`` an HEVC stream in MP4 or Matroska carries raised ``IndexError``.
 - ``CodecContext.bit_rate_tolerance`` returns its value instead of always ``None``; the getter was missing its ``return``.
 - A rejected ``add_stream()`` or ``add_mux_stream()`` no longer breaks the container.
 - ``InputContainer.size`` returns ``None`` when the size cannot be determined rather than the negative ``AVERROR`` it was passing through, which read as a plausible byte count. A non-seekable input, such as a pipe, reported ``-78``.

av/codec/context.py

@@ -1083,8 +1083,7 @@ def delay(self):
 
     @property
     def pkt_timebase(self):
-        """Timebase of the packets fed to this context, as a
-        :class:`~fractions.Fraction`.
+        """Timebase of the packets fed to this context.
 
         Decoders use it to set :attr:`.Frame.time_base`. Containers set it for
         you; set it yourself when driving a bare CodecContext.
@@ -1123,16 +1122,24 @@ def bits_per_raw_sample(self, value: cython.int):
 
     @property
     def initial_padding(self):
-        """Audio only. Samples the decoder should skip at the start of the
-        stream, i.e. the encoder delay. Needed for gapless playback.
+        """Audio only. Priming samples the encoder inserted at the start of the
+        stream, which must be discarded to recover the original audio. Needed
+        for gapless playback.
+
+        Set by libavcodec when encoding, and taken from the stream parameters
+        when decoding.
 
         Wraps :ffmpeg:`AVCodecContext.initial_padding`.
         """
         return self.ptr.initial_padding
 
     @property
     def trailing_padding(self):
-        """Audio only. Samples to discard at the end of the stream.
+        """Audio only. Padding samples appended by the encoder, which must be
+        discarded from the end of the stream to recover the original audio.
+
+        libavcodec neither sets nor acts on this; it only travels between the
+        context and the container's stream parameters.
 
         Wraps :ffmpeg:`AVCodecContext.trailing_padding`.
         """
@@ -1144,8 +1151,8 @@ def trailing_padding(self, value: cython.int):
 
     @property
     def seek_preroll(self):
-        """Number of samples to decode before the target seek point for the
-        output to be correct, in ``1 / AV_TIME_BASE`` units.
+        """Audio only. Number of samples to skip after a discontinuity, such as
+        a seek, before the decoded output is correct.
 
         Wraps :ffmpeg:`AVCodecContext.seek_preroll`.
         """
@@ -1184,8 +1191,13 @@ def stats_in(self, value):
             self.ptr.stats_in = cython.NULL
             return
 
+        if type(value) is str:
+            value = value.encode("utf-8")
+        elif not isinstance(value, (bytes, bytearray)):
+            raise TypeError("stats_in must be str, bytes, or None")
+
         # libavcodec never frees stats_in, so we keep the bytes alive ourselves.
-        self._stats_in = value.encode("utf-8") if type(value) is str else bytes(value)
+        self._stats_in = bytes(value)
         self.ptr.stats_in = self._stats_in
 
     @property
@@ -1196,14 +1208,16 @@ def coded_side_data(self):
         Wraps :ffmpeg:`AVCodecContext.coded_side_data`.
         """
         i: cython.int
-        return {
-            packet_sidedata_type_to_literal(
-                self.ptr.coded_side_data[i].type
-            ): _to_bytes(
+        out = {}
+        for i in range(self.ptr.nb_coded_side_data):
+            try:
+                key = packet_sidedata_type_to_literal(self.ptr.coded_side_data[i].type)
+            except IndexError:
+                continue
+            out[key] = _to_bytes(
                 self.ptr.coded_side_data[i].data, self.ptr.coded_side_data[i].size
             )
-            for i in range(self.ptr.nb_coded_side_data)
-        }
+        return out
 
     @property
     def decoded_side_data(self):
@@ -1218,9 +1232,13 @@ def decoded_side_data(self):
         from av.sidedata.sidedata import Type
 
         i: cython.int
-        return {
-            Type(self.ptr.decoded_side_data[i].type): _to_bytes(
+        out = {}
+        for i in range(self.ptr.nb_decoded_side_data):
+            try:
+                key = Type(self.ptr.decoded_side_data[i].type)
+            except ValueError:
+                continue
+            out[key] = _to_bytes(
                 self.ptr.decoded_side_data[i].data, self.ptr.decoded_side_data[i].size
             )
-            for i in range(self.ptr.nb_decoded_side_data)
-        }
+        return out

av/packet.py

@@ -56,6 +56,9 @@
     "lcevc",
     "3d_reference_displays",
     "rtcp_sr",
+    "exif",
+    "dynamic_hdr_smpte_2094_app5",
+    "hevc_conf",
 ]
 
 

av/packet.pyi

@@ -56,6 +56,9 @@ PktSideDataT = Literal[
     "lcevc",
     "3d_reference_displays",
     "rtcp_sr",
+    "exif",
+    "dynamic_hdr_smpte_2094_app5",
+    "hevc_conf",
 ]
 
 class PacketSideData(Buffer):

av/sidedata/sidedata.py

@@ -50,6 +50,11 @@ class Type(Enum):
     VIEW_ID = lib.AV_FRAME_DATA_VIEW_ID
     THREE_D_REFERENCE_DISPLAYS = lib.AV_FRAME_DATA_3D_REFERENCE_DISPLAYS
     EXIF = lib.AV_FRAME_DATA_EXIF
+    DYNAMIC_HDR_SMPTE_2094_APP5 = lib.AV_FRAME_DATA_DYNAMIC_HDR_SMPTE_2094_APP5
+    IAMF_MIX_GAIN_PARAM = lib.AV_FRAME_DATA_IAMF_MIX_GAIN_PARAM
+    IAMF_DEMIXING_INFO_PARAM = lib.AV_FRAME_DATA_IAMF_DEMIXING_INFO_PARAM
+    IAMF_RECON_GAIN_INFO_PARAM = lib.AV_FRAME_DATA_IAMF_RECON_GAIN_INFO_PARAM
+    RAW_COLOR_PARAMS = lib.AV_FRAME_DATA_RAW_COLOR_PARAMS
 
 
 @cython.cfunc

av/sidedata/sidedata.pyi

@@ -38,6 +38,11 @@ class Type(Enum):
     VIEW_ID = cast(ClassVar[Type], ...)
     THREE_D_REFERENCE_DISPLAYS = cast(ClassVar[Type], ...)
     EXIF = cast(ClassVar[Type], ...)
+    DYNAMIC_HDR_SMPTE_2094_APP5 = cast(ClassVar[Type], ...)
+    IAMF_MIX_GAIN_PARAM = cast(ClassVar[Type], ...)
+    IAMF_DEMIXING_INFO_PARAM = cast(ClassVar[Type], ...)
+    IAMF_RECON_GAIN_INFO_PARAM = cast(ClassVar[Type], ...)
+    RAW_COLOR_PARAMS = cast(ClassVar[Type], ...)
 
 class SideData(Buffer):
     type: Type

include/avcodec.pxd

@@ -352,6 +352,11 @@ cdef extern from "libavcodec/avcodec.h" nogil:
         AV_FRAME_DATA_VIEW_ID
         AV_FRAME_DATA_3D_REFERENCE_DISPLAYS
         AV_FRAME_DATA_EXIF
+        AV_FRAME_DATA_DYNAMIC_HDR_SMPTE_2094_APP5
+        AV_FRAME_DATA_IAMF_MIX_GAIN_PARAM
+        AV_FRAME_DATA_IAMF_DEMIXING_INFO_PARAM
+        AV_FRAME_DATA_IAMF_RECON_GAIN_INFO_PARAM
+        AV_FRAME_DATA_RAW_COLOR_PARAMS
 
     cdef struct AVFrameSideData:
         AVFrameSideDataType type
@@ -540,4 +545,3 @@ cdef extern from "libavcodec/packet.h" nogil:
         AVPacket *pkt, AVPacketSideDataType type, uint8_t *data, size_t size
     )
     const char *av_packet_side_data_name(AVPacketSideDataType type)
-    const char *av_frame_side_data_name(AVFrameSideDataType type)

tests/test_codec_context.py

@@ -649,6 +649,11 @@ def test_encoder_scalars_roundtrip(self) -> None:
         assert ctx.pkt_timebase == Fraction(1, 1000)
         assert ctx.frame_num == 0
 
+    def test_stats_in_rejects_junk(self) -> None:
+        ctx = av.CodecContext.create("libx264", "w")
+        with pytest.raises(TypeError):
+            ctx.stats_in = 5  # type: ignore[assignment]
+
     def test_stats_in_out(self) -> None:
         ctx = av.CodecContext.create("libx264", "w")
         assert ctx.stats_in is None