3.0 version of HLS generates a large number of TS slice files.

[baiyu0820]

Description
Version 3.0 generates a large number of ts slice files in HLS. Version 2.6 does not have this issue.

1. Operating System: CentOS 7

Environment
Streaming command: C:\MinGW\msys\1.0\bin>ffmpeg.exe -re -i c:/1.mp4 -vcodec libx264 -acodec aac -f
flv rtmp://192.168.0.39:1935/live/b1

Logs: srs.log
Configuration file: srs.conf.txt
Video file (28mb): https://pan.baidu.com/s/1hPu46MmYOllJe1C0AXiM8g (Password: 5vdm)

Image:

TRANS_BY_GPT3

[winlinvip]

Your configuration is set to not clean up HLS, so of course there will be a lot of TS files.

        hls_dispose     0;
        hls_cleanup     off;

TRANS_BY_GPT3

[winlinvip]

Your stream will generate a large number of small TS files, with a duration exceeding 100 seconds, which is also a problem.

Mac:live chengli.ycl$ ffmpeg -i livestream-3899.ts
Input #0, mpeg, from 'livestream-3899.ts':
  Duration: 00:00:59.63, start: 157.570611, bitrate: 0 kb/s
    Stream #0:0[0x1c0]: Audio: mp2, 0 channels, s16p
    Stream #0:1[0x1e0]: Video: none, none, 90k tbr, 90k tbn

TRANS_BY_GPT3

[winlinvip]

The fourth one is abnormal.

-rw-r--r--  1 chengli.ycl  staff  185932 Dec  5 18:03 livestream-4.ts

ffmpeg -i livestream-4.ts
ffmpeg version 3.2.4 Copyright (c) 2000-2017 the FFmpeg developers
[h264 @ 0x7fc072004200] non-existing PPS 0 referenced
    Last message repeated 1 times
[h264 @ 0x7fc072004200] decode_slice_header error
[h264 @ 0x7fc072004200] no frame!
[h264 @ 0x7fc072004200] non-existing PPS 0 referenced
    Last message repeated 1 times
[h264 @ 0x7fc072004200] decode_slice_header error
[h264 @ 0x7fc072004200] no frame!
[h264 @ 0x7fc072004200] non-existing PPS 0 referenced
    Last message repeated 1 times
[h264 @ 0x7fc072004200] decode_slice_header error
[h264 @ 0x7fc072004200] no frame!
[h264 @ 0x7fc072004200] non-existing PPS 0 referenced
    Last message repeated 1 times
[h264 @ 0x7fc072004200] decode_slice_header error
[h264 @ 0x7fc072004200] no frame!
Input #0, mpegts, from 'livestream-4.ts':
  Duration: 00:00:16.99, start: 33.994011, bitrate: 87 kb/s
  Program 1 
    Stream #0:0[0x101]: Audio: aac (LC) ([15][0][0][0] / 0x000F), 32000 Hz, stereo, fltp, 132 kb/s
    Stream #0:1[0x100]: Video: h264 (High) ([27][0][0][0] / 0x001B), yuvj420p(pc, bt709, progressive), 1920x1080, 90k tbr, 90k tbn, 180k tbc

If audio is disabled and only video is pushed, there is no problem.

    hls {
        enabled         on;
        hls_acodec an;
    }

ffmpeg -re -i 1.mp4 -vcodec copy -an -f flv -y rtmp://127.0.0.1/live/livestream

Mac:live chengli.ycl$ ffmpeg -i livestream-4.ts 
ffmpeg version 3.2.4 Copyright (c) 2000-2017 the FFmpeg developers
Input #0, mpegts, from 'livestream-4.ts':
  Duration: 00:00:10.97, start: 44.016000, bitrate: 221 kb/s
  Program 1 
    Stream #0:0[0x100]: Video: h264 (High) ([27][0][0][0] / 0x001B), yuvj420p(pc, bt709, progressive), 1920x1080, 90k tbr, 90k tbn, 180k tbc

So this is definitely related to audio. The audio packet is over 500 bytes, which doesn't seem like a normal AAC packet. SRS uses the size of AAC packets to calculate timestamps, which makes it more accurate and avoids the problem of inaccurate timestamps causing audio distortion. Reference: #547 (comment)

If it is changed to pure audio HLS, it will be easier to troubleshoot.

    hls {
        enabled         on;
        hls_vcodec vn;
    }

ffmpeg -re -i 1.mp4 -acodec copy -vn -f flv -y rtmp://127.0.0.1/live/livestream

Debugging the received data, the first one is the audio sequence header encoding header, and the second packet is audio data with 501 bytes.

$4 = timestamp = 0, stream_id = 1, size = 501

The third audio packet is 48 milliseconds and has 522 bytes.

timestamp = 48, stream_id = 1, size = 522

Generally, AAC has a sampling rate of 44100Hz, and one frame contains 1024 samples, so one frame is usually 23 milliseconds.

1024/44100.0=0.02321995s=23.21995ms

However, this stream is 48 milliseconds, so it should contain 2048 samples, which is larger than the usual. Due to the previous logic being fixed at 1024 samples or 960 samples to calculate the timestamp, it resulted in incorrect calculations.

    // Use the diff to guess whether the samples is 1024 or 960.
    int nb_samples_per_frame = 1024;
    if (diff > 100 && diff < 950) {
        nb_samples_per_frame = 960;
    }
    
    // Recalc the DTS by the samples of AAC.
    int64_t dts = 90000 * aac_samples / srs_flv_srates[format->acodec->sound_rate];
    aac_samples += nb_samples_per_frame;

After modifying the guess algorithm for the number of samples, the pure audio slicing is working correctly. Please refer to 31f341e for more information.

-rw-r--r--@ 1 chengli.ycl  staff  456840 Dec  5 20:09 livestream-1.ts
-rw-r--r--  1 chengli.ycl  staff  456840 Dec  5 20:10 livestream-2.ts
-rw-r--r--  1 chengli.ycl  staff  456652 Dec  5 20:10 livestream-3.ts
-rw-r--r--  1 chengli.ycl  staff  457028 Dec  5 20:11 livestream-4.ts
-rw-r--r--  1 chengli.ycl  staff  456652 Dec  5 20:11 livestream-5.ts

After the modification, there are still abnormalities when audio and video are together, and the slices are getting smaller and smaller.

-rw-r--r--  1 chengli.ycl  staff   112K Dec  5 20:14 livestream-4.ts
-rw-r--r--  1 chengli.ycl  staff   223K Dec  5 20:14 livestream-5.ts
-rw-r--r--  1 chengli.ycl  staff    90K Dec  5 20:14 livestream-6.ts
-rw-r--r--  1 chengli.ycl  staff   266K Dec  5 20:14 livestream-7.ts
-rw-r--r--  1 chengli.ycl  staff    45K Dec  5 20:14 livestream-8.ts
-rw-r--r--  1 chengli.ycl  staff   1.1K Dec  5 20:15 livestream-499.ts

Debugging to retrieve two audio packets.

timestamp = 102304, stream_id = 1, size = 501
diff=32, nb_samples_per_frame=1024, aac_samples=3416704, dts=77499.5ms

timestamp = 102336, stream_id = 1, size = 504
diff=32, nb_samples_per_frame=1024, aac_samples=3418752, dts=77522ms

The timestamp of the initial AAC packet was 48ms, but later it changed to 32ms, which caused incorrect calculation of the audio's DTS. It seems that estimating the timestamp based on the AAC sample size is not accurate. It would be more reasonable to calculate it directly based on the timestamp. A configuration option needs to be added for this. Please refer to: b4870a6.

    hls {
        enabled         on;
        hls_dts_directly on;
    }

-rw-r--r--  1 chengli.ycl  staff   488K Dec  5 20:44 livestream-7.ts
-rw-r--r--  1 chengli.ycl  staff   491K Dec  5 20:45 livestream-8.ts
-rw-r--r--  1 chengli.ycl  staff   489K Dec  5 20:45 livestream-9.ts

Please note that you need to update to SRS 3.0.68 in order to use this configuration.

Improvement measures:

1. The number of samples can be determined by the sampling rate and the difference between frames, instead of fixedly setting the sample size to 1024.
2. Provide a configuration option to disable the functionality of calculating timestamps based on the AAC sample count, and calculate them directly based on the package timestamps.

TRANS_BY_GPT3

[winlinvip]

When the video is not encoded in H.264, it will cause SRS to crash.

    hls {
        enabled         on;
        hls_acodec an;
    }

ffmpeg -re -i 1.mp4 -acodec copy -f flv -y rtmp://127.0.0.1/live/livestream

In fact, this is actually encoding the video as H.263, which causes the codec to be unable to parse it.

srs_error_t SrsFormat::on_video(int64_t timestamp, char* data, int size)
{
    ......
    if (codec_id != SrsVideoCodecIdAVC) {
        return err;
    }

As a result, format->vcodec and format-video are both NULL, triggering an assert error.

srs_error_t SrsHls::on_video(SrsSharedPtrMessage* shared_video, SrsFormat* format)
{
    ......
    srs_assert(format->video);

Fixed in 52c6c7c

TRANS_BY_GPT3