Segmented VTT - absolute or relative timestamps?

[sanbornhilland]

WebVTT segments are not syncing properly with live streams because they are not accounting for the segmentStartTime. The first cue in each segment starts at 00:00:00.000 so the segmentStartTime needs to be used to offset each segment properly.

[joeyparrish]

I would not expect the first cue in each segment to start at 0. What packager are you using?

[sanbornhilland]

#481 has more discussion. It seems that it's unclear whether VTT timestamps are supposed to be absolute of relative.

[joeyparrish]

We treat them as absolute because that's how timestamps work in all other kinds of media segments.

[joeyparrish]

Are you able to share any details on the encoder vendor or packager software you're using?

[sanbornhilland]

I'm working on getting you details and a test stream I can share.

[sanbornhilland]

@joeyparrish I emailed you a test stream.

[sanbornhilland]

Is there any update on this? The stream I emailed @joeyparrish is will expire on Sept. 7 so if there is an opportunity to discuss this issue before then, it would be useful.

[joeyparrish]

Sorry for the delay in my response.

I ran into a small, unrelated issue with your manifest that caused us to ignore the WebVTT text:

  <AdaptationSet mimeType="text/vtt" ...>
    <Representation codecs="vtt" ... />
  </AdaptationSet>

Our parser is registered for text/vtt, but not text/vtt; codecs="vtt". This is easy to fix by registering the additional string. I'll take care of this shortly.

As for the timestamps, we treat everything generically. Since timestamps in video segments are relative to the period, so should text timestamps be.

The only exception to this so far has been WebVTT embedded in MP4. The atom containing the cue does not actually contain a timestamp. The format specifies that the cue time should be the segment time. We treat this differently than the others because this is part of the spec.

Now, looking at your text content, I see this:

WEBVTT
X-TIMESTAMP-MAP=MPEGTS:2214643072,LOCAL:00:00:00.000

00:00:00.000 --> 00:00:00.698 position:10%,start align:left line:19%
<c.darkgray>SO WE HAVE A CHANGE OF THE</c>

I researched X-TIMESTAMP-MAP, and it seems to be an HLS extension and not part of the WebVTT spec. This page from BrightCove states:

Adding the X-TIMESTAMP-MAP header may cause inconsistencies on other platforms, such as Windows Mobile applications. Make sure to test across all platforms after adding the offset header. The addition of this header tag is an HLS spec requirement, and not a WebVTT spec parameter.

I think it makes more sense to add support for X-TIMESTAMP-MAP than to add a configuration option to decide if timestamps are relative or not. Thoughts?

[joeyparrish]

Actually, no, that won't work after all. As I'm looking more closely at the numbers, the map in the WebVTT files don't match up to the DASH presentation timeline at all.

For example, the segment at time 144323839, timescale 1000 contains a map that says it should be offset to 2560783072. Comparing numbers across segments, I see that 1k in the DASH timescale seems to equal 90k in the MPEG2 timescale used by whatever generated the VTT.

I'm increasingly convinced now that these timestamps are something the encoder needs to fix. They are taking something meant for HLS and just serving it as DASH, which doesn't make sense. Have you reached out to them for support?