Support http-mp4 file stream, fmp4 stream.

[winlinvip]

https://github.com/winlinvip/simple-rtmp-server/issues/174

TRANS_BY_GPT3

[winlinvip]

MP4 itself is a box within a box, not a streaming mode. Later, someone improved the box structure of MP4, similar to turning the box into an FLV tag or stacking the boxes side by side. If the client supports parsing and playing the boxes simultaneously, it can achieve similar functionality to HTTP-FLV.

It seems that Silverlight, the deceased Silverlight, and the MP4 stream directly with IIS are examples of this approach.

TRANS_BY_GPT3

[winlinvip]

WeChat Mini Program supports playing HLS live streaming and on-demand streaming, as well as HTTP-MP4 on-demand files. Let's see if it also supports HTTP MP4 live streaming and how much the latency can be reduced.

Microsoft's Live Smooth Streaming uses fragmented MP4: https://technet.microsoft.com/en-us/library/7d5f47fa-886b-4321-8367-60b1a126d4dd

This article specifically discusses fragmented MP4: http://blog.csdn.net/yu_yuan_1314/article/details/9289827

mp4box can generate this type of file.

MP4Box -dash 500000 -frag 50000 -rap file_name.mp4

GPAC (mp4box) official website: https://gpac.wp.mines-telecom.fr/mp4box/

GPAC on GitHub: https://github.com/gpac/gpac

There is an article discussing the replacement of HTTP with RTMP: http://blog.streamingmedia.com/2016/09/flash-rtmp-streaming-dying-out.html. It does not propose a fundamental solution to support RTMP-level latency.

Here is someone who tried fmp4 and said that H5 does not support it: https://my.oschina.net/langhuihui/blog/519901.

You can convert the SRS instance avatar.flv to mp4.

ffmpeg -i doc/source.200kbps.768x320.flv -c copy -y avatar.mp4 

Use mp4box to view information.

winlin:mp4 winlin$ mp4box -info avatar.mp4 
* Movie Info *
	Timescale 1000 - 2 tracks
	Computed Duration 00:03:30.652 - Indicated Duration 00:03:30.652
	Fragmented File: no
	File Brand isom - version 512
	Created: UNKNOWN DATE	Modified: UNKNOWN DATE
File has no MPEG4 IOD/OD

iTunes Info:
	Encoder Software: Lavf55.19.104
1 UDTA types: meta (1) 

Track # 1 Info - TrackID 1 - TimeScale 16000
Media Duration 00:03:30.560 - Indicated Duration 00:03:30.560
Track has 2 edit lists: track duration is 00:03:30.606
Media Info: Language "Undetermined (und)" - Type "vide:avc1" - 5264 samples
Visual Track layout: x=0 y=0 width=768 height=320
MPEG-4 Config: Visual Stream - ObjectTypeIndication 0x21
AVC/H264 Video - Visual Size 768 x 320
	AVC Info: 1 SPS - 1 PPS - Profile High @ Level 3.2
	NAL Unit length bits: 32
	Pixel Aspect Ratio 1:1 - Indicated track size 768 x 320
	Chroma format 0 - Luma bit depth 8 - chroma bit depth 8
	SPS#1 hash: 3031FBB2E6D922C701D45CCE28FFC6743FCA7D2A
	PPS#1 hash: 6BBE60BFBE383CE9144FA78B782534A704912778
Self-synchronized
	RFC6381 Codec Parameters: avc1.640020
	Average GOP length: 48 samples

Track # 2 Info - TrackID 2 - TimeScale 44100
Media Duration 00:03:30.651 - Indicated Duration 00:03:30.651
Track has 1 edit lists: track duration is 00:03:30.652
Media Info: Language "Undetermined (und)" - Type "soun:mp4a" - 9072 samples
MPEG-4 Config: Audio Stream - ObjectTypeIndication 0x40
MPEG-4 Audio AAC LC - 2 Channel(s) - SampleRate 44100
Synchronized on stream 1
	RFC6381 Codec Parameters: mp4a.40.2
Alternate Group ID 1
	All samples are sync

mp4box parameters:

winlin:mp4 winlin$ mp4box -h all
General Options:
 -frag time_in_ms     fragments file (track fragments of time_in_ms)
                       * Note: Always disables interleaving
 -out filename        specifies output file name
                       * Note: By default input (MP4,3GP) file is overwritten

DASH Options:
 -mpd m3u8            converts HLS manifest (local or remote http) to MPD 
                       Note: not compatible with other DASH options (except -out and -tmp) and does not convert associated segments
 -dash dur            enables DASH-ing of the file(s) with a segment duration of DUR ms
                       Note: the duration of a fragment (subsegment) is set
	                            using the -frag switch.
                       Note: for onDemand profile, sets duration of a subsegment

Convert regular mp4 to fmp4:

mp4box -frag 3000 avatar.mp4 -out avatar_f.mp4

You can use mp4info to see a small mp4 fragment every 3 seconds:

However, with WeChat mini program, it can only play the first 3 seconds, which means it does not have complete support for fragment mp4.

The loading speed and the speed of starting playback are both fast, but unfortunately it does not continue to parse the remaining moof box...

The two MP4 links are as follows. Regular MP4:
http://ossrs.net/weixin/avatar.mp4
Fragmented MP4, with a duration of 3 seconds per fragment:
http://ossrs.net/weixin/avatar_f.mp4

For other live streaming methods on WeChat, please refer to:
http://blog.csdn.net/win_lin/article/details/54410876

TRANS_BY_GPT3

[winlinvip]

In addition, the browsers on PC for fmp4 are as follows:

• Chrome cannot play fmp4.
• Safari can play fmp4, but it is very slow.
• Firefox has relatively good support, continuously downloading the following fragments, and the duration keeps increasing.

Consider putting this unreliable protocol into consideration for SRS4+.

TRANS_BY_GPT3

[jswh]

You can try to support MPEG-TS and then use jsmpeg to achieve playback on regular HTML5 pages and within WeChat. If you use Canvas, it will also work fine in mini programs, although it may not be as efficient as WebGL.

---

It seems that it would be better to directly raise an issue for this.

TRANS_BY_GPT3

[wony-zheng]

flv.js is used in the frontend to convert flv to fmp4, why is it no longer supported?

TRANS_BY_GPT3

[winlinvip]

@wony-zheng MSE is supported, but playing MPEG4 streams directly in the browser is a different framework from MSE.

TRANS_BY_GPT3

[winlinvip]

The conclusion is that MP4 streaming is unreliable.

TRANS_BY_GPT3