EMSG box support

[joeyparrish]

We could support EMSG boxes through a network response filter on segments. The filter would parse the MP4 segments looking for EMSG boxes, then emit JavaScript events through Player if found.

Please comment and give specifics if this is of interest to you.

[lwchan2k1]

I am very interested in this feature which can enable advertising support with multi period support together.

[dougdoe]

I have now also subscribed to this issue. Lewis (above) indicates why we need this, but thought I could add a little context. Our third-party advertising vendor requires that the player to support the following features:

1. DASH multi-period manifest (w/ support for switching between DRM and non-DRM streams across periods) – I think this requirement is fulfilled given Various issues with multi-period live streams #310 and Update end time of last segment in a period when new periods are added #315.
2. EMSG box support as per DASH specification, which is this enhancement issue.

Since this enhancement is not planned for Shaka v2.0, we may need to look at implementing this ourselves. If we move ahead with this, we will contact you to discuss a potential solution.

Let us know if you have any feedback or guidance that may help us here. Thanks (as always)!

[joeyparrish]

We would be happy to discuss design and ultimately to review a pull request for this. Thanks for expressing your interest!

What we had in mind is a network filter that would listen for the responses to segment requests and parse those segments before they are processed by StreamingEngine and MediaSourceEngine.

The only tutorial we have right now that discusses network response filters is docs/tutorials/license-wrapping.md, so you can look at that for examples of how response filters work. For EMSG, we'd be filtering on RequestType.SEGMENT instead of RequestType.LICENSE.

I imagine the parsing would require a generic MP4 box parser to locate the EMSG box, if present. There may be some other way to approach it as well, but a generic MP4 box parser would be useful in other places, too, such as VTT (#277) and TTML (#278) text embedded in MP4.

The data extracted from EMSG would be dispatched as an event from Player. All events are constructed with shaka.util.FakeEvent and their types and structures documented in jsdoc annotations in player.js.

The whole feature would ideally be modular and easily omitted from the build. See docs/tutorials/plugins.md for an overview of plugins and how they exist in the build system.

[joeyparrish]

We're changing direction on this slightly. Instead of dispatching emsg events from Player, we will use the presence of an emsg box to trigger an MPD update. For efficiency, we will only filter segments for emsg boxes if the manifest indicates that they are present. This appears to be the only use of the emsg box from what I can tell.

[dougdoe]

Hi Joey, reading over your latest update, I'm not sure this is sufficient. We are planning on using EMSG box for the following:

1. ID3 tags – ID3 tags will be inserted into EMSG box and when detected by the player, we will need to take action
2. Dynamic Ad Insertion

In both cases, I believe we will need EMSG events to be dispatched from the player. Does this make sense?

[joeyparrish]

@dougdoe Can you provide some details on what those EMSG boxes will look like? For example, what scheme URIs will be used for those two cases?

[dougdoe]

Hi @joeyparrish I am about to send you an email with two documents that contain information on EMSG box and ID3 tags.

[baconz]

@dougdoe be careful of #409 -- I think one of your assumptions might be broken.

@joeyparrish it seems like any implementation of EMSG support should be inline with support for Event and EventStream, since the IOP allows you to use either one for the type of data that @dougdoe is interested in. It would be great if Shaka provided an API that allowed you to add listeners for events in the manifest that fire when the player hits the relevant position.

[joeyparrish]

@baconz, can you point me to a reference for Event and EventStream? I admit, I'm not familiar with those yet.

[baconz]

Sure! Check out 5.3.3.7 and 5.3.2.4 of the latest IOP (http://dashif.org/wp-content/uploads/2016/06/DASH-IF-IOP-v3.3.pdf). Here's the example that they give:

<EventStream schemeIdUri="urn:scte:scte35:2014:xml+bin">
 <Event timescale="90000" presentationTime="54054000" duration="5400000" id="1">
 <scte35:Signal>
 <scte35:Binary>
 /DAIAAAAAAAAAAAQAAZ/I0VniQAQAgBDVUVJQAAAAH+cAAAAAA==
 </scte35:Binary>
 </scte35:Signal>
 </Event>
</EventStream>

There's also section 12.1.5 of the SCTE-67 spec, which talks about how SCTE-35 data should be integrated into DASH (http://www.scte.org/documents/pdf/standards/SCTE%2067%202014.pdf).

There isn't a ton of info fleshed out in the IOP, but the features that they hint at would be extremely useful to us.

[joeyparrish]

@baconz, we would consider EventStream and VAST support as a separate enhancement from the basic EMSG support we're working on here.