Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

WebRTC to RTMP audio and video synchronization issue #2507

Closed
bywwcnll opened this issue Aug 4, 2021 · 1 comment · Fixed by #2470
Closed

WebRTC to RTMP audio and video synchronization issue #2507

bywwcnll opened this issue Aug 4, 2021 · 1 comment · Fixed by #2470
Assignees
@duiniuluantanqin
Labels
Bug It might be a bug. TransByAI Translated by AI/GPT. WebRTC WebRTC, RTC2RTMP or RTMP2RTC.
Milestone
4.0

Comments

@bywwcnll
Copy link

bywwcnll commented Aug 4, 2021 •
edited by winlinvip
Loading

Audio and video are not synchronized

When enabling the RTC to RTMP function, the WebRTC stream is normal, but there will be audio and video synchronization issues in both the preview screen and the recorded video.

  1. SRS version: 4.0 release (compiled using the latest committed code)
  2. The log of SRS is as follows:
[2021-08-04 19:37:47.212][Trace][11647][19f84k5p] XCORE-SRS/4.0.146(Leo)
[2021-08-04 19:37:47.222][Trace][11647][19f84k5p] config parse complete
[2021-08-04 19:37:47.222][Trace][11647][19f84k5p] write log to console
[2021-08-04 19:37:47.222][Trace][11647][19f84k5p][MAIN] SRS/4.0.146(Leo), MIT
[2021-08-04 19:37:47.222][Trace][11647][19f84k5p] authors: https://github.com/ossrs/srs/blob/4.0release/trunk/AUTHORS.txt
[2021-08-04 19:37:47.222][Trace][11647][19f84k5p] contributors: winlinvip<winlin@vip.126.com> winlinvip<winterserver@126.com> winlinvip<chengli.ycl@alibaba-inc.com> wenjiegit<zhaowenjie@tal.com> wenjiegit<740936897@qq.com> wenjiegit<wenjie.zhao@chinacache.com> xiangcheng.liu<liuxc0116@foxmail.com> naijia.liu<youngcow@youngcow.net> alcoholyi<alcoholyi@qq.com> byteman<wangchen2011@gmail.com> chad.wang<chad.wang.cn@gmail.com> suhetao<suhetao@gmail.com> Johnny<fengjihu@163.com> karthikeyan<keyanmca@gmail.com> StevenLiu<lq@chinaffmpeg.org> zhengfl<zhengfl_1989@126.com> tufang14<breadbean1449@gmail.com> allspace<allspace@gmail.com> niesongsong<nie950@gmail.com> rudeb0t<nimrod@themanxgroup.tw> CallMeNP<np.liamg@gmail.com> synote<synote@qq.com> lovecat<littlefawn@163.com> panda1986<542638787@qq.com> YueHonghui<hongf.yue@hotmail.com> ThomasDreibholz<dreibh@simula.no> JuntaoLiu<juntliu@gmail.com> RocFang<fangpeng1986@gmail.com> MakarovYaroslav<yaroslav.makarov.97@mail.ru> MirkoVelic<mvelic@inoxx.net> HuiZhang(huzhang2)<huzhang2@cisco.com> OtterWa<simpleotter23@gmail.com> walkermi<172192667@qq.com> haofz<fuzhuang.hao@vhall.com> ME_Kun_Han<hanvskun@hotmail.com> ljx0305<ljx0305@gmail.com> cenxinwei<censhanhe@163.com> StarBrilliant<m13253@hotmail.com> xubin<xubin@chnvideo.com> intliang<yintiliang@gmail.com> flowerwrong<sysuyangkang@gmail.com> YLX<568414379@qq.com> J<guotaojiang@qq.com> Harlan<hailiang@gvrcraft.com> hankun<hankun@bravovcloud.com> JonathanBarratt<jonathan.barratt@gmail.com> KeeganH<keeganwharris@gmail.com> StevenLiu<lingjiujianke@gmail.com> liuxc0116<liuxc0116@gmail.com> ChengdongZhang<lmajzcd@sina.com> lovacat<lovecat@china.sina.com> qiang.li<qiang.li@verycdn.com.cn> HungMingWu<u9089000@gmail.com> Himer<xishizhaohua@qq.com> xialixin<xialixin@kanzhun.com> xialixin<68469352@qq.com> xialixin<xlx0625@163.com> xialixin<xialx@yuntongxun.com> alphonsetai<tyh_123@163.com> Michael.Ma<wnpllr@gmail.com> lam2003<linmin3@yy.com> runner365<shiwei05@kuaishou.com> runner365<shi.weibd@hotmail.com> XiaofengWang<wasphin@gmail.com> xiaozhihong<hondaxiao@tencent.com> xiaozhihong<xiaozhihong8@gmail.com> xiaozhihong<xiaozhihong@huya.com> yanghuiwen<cainiaodj@qq.com> Bepartofyou<309554135@qq.com> Bepartofyou<pengqiang.wpq@alibaba-inc.com> l<22312935+lam2003@github> xfalcon<x-falcon@github> chen-guanghua<jinxue.cgh@alibaba-inc.com> chen-guanghua<chengh_math@126.com> lipeng19811218<mozhan.lp@alibaba-inc.com> lipeng19811218<lipeng19811218@gmail.com> yajun18<yajun18@staff.sina.com.cn> liulichuan<liulichuan@kuaishou.com> yapingcat<caoyapingneu@163.com> chenchengbin<chenchengbin@yy.com> duiniuluantanqin<nmgchenhaibo@foxmail.com> jasongwq<jasongwq@gmail.com> yinjiaoyuan<yinjiaoyuan@163.com> PieerePi<pihuibin@hotmail.com> JesseXi<jesse.jinjin@wo.cn> PieerePi<40780488+PieerePi@github> ghostsf<ghost_sf@163.com> xbpeng121<53243357+xbpeng121@github> johzzy<hellojinqiang@gmail.com> stone<bluestn@163.com> cfw11<34058899+cfw11@github> Hung-YiChen<gaod.chen@gmail.com> long<liyalong12345@126.com> 
[2021-08-04 19:37:47.222][Trace][11647][19f84k5p] cwd=/Users/ares/Desktop/srs/trunk, work_dir=./, build: 2021-08-04 15:32:29, configure: --osx, uname: Darwin AresMBP.local 20.2.0 Darwin Kernel Version 20.2.0: Wed Dec  2 20:39:59 PST 2020; root:xnu-7195.60.75~1/RELEASE_X86_64 x86_64, osx: 1
[2021-08-04 19:37:47.222][Trace][11647][19f84k5p] configure detail: --prefix=/usr/local/srs --hls=on --hds=off --dvr=on --ssl=on --https=on --ssl-1-0=off --ssl-local=off --sys-ssl=off --transcode=on --ingest=on --stat=on --http-callback=on --http-server=on --stream-caster=on --http-api=on --utest=off --cherrypy=off --srt=off --rtc=on --simulator=off --cxx11=off --cxx14=off --ffmpeg-fit=on --nasm=on --srtp-nasm=on --clean=on --gperf=off --gmc=off --gmd=off --gmp=off --gcp=off --gprof=off --static=off --shared-st=off --shared-srt=off --shared-ffmpeg=off --log-verbose=off --log-info=off --log-trace=on --gcov=off --debug=off --debug-stats=off --cross-build=off --cc=gcc --cxx=g++ --ar=ar --ld=gcc --randlib=randlib
[2021-08-04 19:37:47.222][Trace][11647][19f84k5p] srs checking config...
[2021-08-04 19:37:47.223][Trace][11647][19f84k5p] ips, iface[0] en1 ipv4 0x8863 192.168.31.102, iface[1] en1 ipv6 0x8863 fe80::100b:168e:b602:9fbc%en1, iface[2] llw0 ipv6 0x8863 fe80::ec83:aaff:feeb:9e2c%llw0
[2021-08-04 19:37:47.223][Trace][11647][19f84k5p] devices, intranet en1 192.168.31.102, intranet en1 fe80::100b:168e:b602:9fbc%en1, intranet llw0 fe80::ec83:aaff:feeb:9e2c%llw0
[2021-08-04 19:37:47.223][Warn][11647][19f84k5p][0] stats network use index=0, ip=192.168.31.102, ifname=en1
[2021-08-04 19:37:47.223][Warn][11647][19f84k5p][0] stats disk not configed, disk iops disabled.
[2021-08-04 19:37:47.223][Trace][11647][19f84k5p] write log to console
[2021-08-04 19:37:47.223][Trace][11647][19f84k5p] features, rch:on, dash:on, hls:on, hds:off, srt:off, hc:on, ha:on, hs:on, hp:on, dvr:on, trans:on, inge:on, stat:on, sc:on
[2021-08-04 19:37:47.223][Trace][11647][19f84k5p] SRS on  amd64 x86_64, conf:objs/myconf/test.conf, limit:1000, writev:1024, encoding:little-endian, HZ:100
[2021-08-04 19:37:47.223][Trace][11647][19f84k5p] mw sleep:350ms. mr enabled:on, default:0, sleep:350ms
[2021-08-04 19:37:47.223][Trace][11647][19f84k5p] gc:on, pq:30000ms, cscc:[0,16), csa:on, tn:on(may hurts performance), ss:auto(guess by merged write)
[2021-08-04 19:37:47.223][Trace][11647][19f84k5p] system default latency(ms): mw(0-350) + mr(0-350) + play-queue(0-30000)
[2021-08-04 19:37:47.223][Warn][11647][19f84k5p][0] SRS/4.0.146 is not stable
[2021-08-04 19:37:47.230][Trace][11647][19f84k5p] fingerprint=AF:12:C7:3F:A4:E2:D1:58:7C:A3:86:7E:E5:A7:6C:EC:BE:81:DE:1C:B5:08:FC:D1:A9:E4:79:A8:8B:20:31:84
[2021-08-04 19:37:47.230][Trace][11647][19f84k5p] CircuitBreaker: enabled=1, high=2x90, critical=1x95, dying=5x99
[2021-08-04 19:37:47.230][Trace][11647][19f84k5p] http flv live stream, vhost=__defaultVhost__, mount=[vhost]/[app]/[stream].flv
[2021-08-04 19:37:47.230][Trace][11647][19f84k5p] http: root mount to ./objs/nginx/html
[2021-08-04 19:37:47.230][Trace][11647][19f84k5p] server main cid=19f84k5p, pid=11647, ppid=5542, asprocess=0
[2021-08-04 19:37:47.230][Trace][11647][19f84k5p] write pid=11647 to ./objs/srs.1935.pid success!
[2021-08-04 19:37:47.231][Trace][11647][19f84k5p] RTMP listen at tcp://0.0.0.0:1935, fd=7
[2021-08-04 19:37:47.232][Trace][11647][19f84k5p] HTTP-API listen at tcp://0.0.0.0:1985, fd=8
[2021-08-04 19:37:47.232][Trace][11647][19f84k5p] HTTP-Server listen at tcp://0.0.0.0:8020, fd=9
[2021-08-04 19:37:47.233][Trace][11647][19f84k5p] signal installed, reload=1, reopen=30, fast_quit=15, grace_quit=3
[2021-08-04 19:37:47.233][Trace][11647][19f84k5p] http: api mount /console to ./objs/nginx/html/console
[2021-08-04 19:37:47.234][Trace][11647][19f84k5p] rtc listen at udp://0.0.0.0:8000, fd=10
[2021-08-04 19:37:47.234][Trace][11647][f2o079bf] Hybrid cpu=0.00%,0MB
[2021-08-04 19:37:47.234][Warn][11647][35rn2c93][2] use private address as ip: 192.168.31.102, ifname=en1
[2021-08-04 19:37:47.250][Trace][11647][bl306s7l] TCP: connection manager run, conns=0
[2021-08-04 19:37:47.250][Warn][11647][o598q86j][55] set SO_SNDBUF failed, expect=10485760, r0=-1
[2021-08-04 19:37:47.250][Warn][11647][o598q86j][55] set SO_RCVBUF failed, expect=10485760, r0=-1
[2021-08-04 19:37:47.250][Trace][11647][o598q86j] UDP #10 LISTEN at 0.0.0.0:8000, SO_SNDBUF(default=9216, expect=10485760, actual=9216, r0=-1), SO_RCVBUF(default=786896, expect=10485760, actual=786896, r0=-1)
[2021-08-04 19:37:47.250][Trace][11647][ftlp42d2] RTC: connection manager run, conns=0
[2021-08-04 19:37:47.702][Trace][11647][35rn2c93] Startup query id=s17mub1n41294m4m, eip=192.168.31.102, match=v4.0.139, stable=3.0.164, wait=3342s, cost=467ms
[2021-08-04 19:37:51.913][Trace][11647][69a9vo43] HTTP #0 127.0.0.1:51980 OPTIONS http://localhost:1985/rtc/v1/publish/, content-length=-1
[2021-08-04 19:37:51.914][Trace][11647][69a9vo43] HTTP #1 127.0.0.1:51980 POST http://localhost:1985/rtc/v1/publish/, content-length=6112
[2021-08-04 19:37:51.914][Trace][11647][69a9vo43] RTC publish webrtc://localhost/live/livestream, api=http://localhost:1985/rtc/v1/publish/, tid=25c4059, clientip=127.0.0.1, app=live, stream=livestream, offer=5637B, eip=, codec=
[2021-08-04 19:37:51.914][Trace][11647][69a9vo43] ignore attribute=, value=
[2021-08-04 19:37:51.915][Trace][11647][69a9vo43] new source, stream_url=/live/livestream
[2021-08-04 19:37:51.916][Trace][11647][69a9vo43] RTC publisher nack=1, nnc=1, pt-drop=0, twcc=1/3
[2021-08-04 19:37:51.916][Trace][11647][69a9vo43] new source, stream_url=/live/livestream
[2021-08-04 19:37:51.918][Trace][11647][69a9vo43] RTC: Init tracks {track: 8ac166ee-ef00-4330-96e9-d02d73ce121f, is_active: 0=>1},{track: d9b81c90-709e-4b59-8db6-43e1c2d0b898, is_active: 0=>1}, ok
[2021-08-04 19:37:51.918][Trace][11647][69a9vo43] RTC: Use candidates 222.95.55.53
[2021-08-04 19:37:51.920][Trace][11647][69a9vo43] RTC init session, user=4l2p52xi:Hjdi, url=/live/livestream, encrypt=1/1, DTLS(role=passive, version=auto), timeout=30000ms, nack=1
[2021-08-04 19:37:51.920][Trace][11647][69a9vo43] RTC username=4l2p52xi:Hjdi, offer=5637B, answer=1454B
[2021-08-04 19:37:51.920][Trace][11647][69a9vo43] RTC remote offer: v=0\r\no=- 3770685905438148112 2 IN IP4 127.0.0.1\r\ns=-\r\nt=0 0\r\na=group:BUNDLE 0 1\r\na=extmap-allow-mixed\r\na=msid-semantic: WMS\r\nm=audio 9 UDP/TLS/RTP/SAVPF 111 103 104 9 0 8 106 105 13 110 112 113 126\r\nc=IN IP4 0.0.0.0\r\na=rtcp:9 IN IP4 0.0.0.0\r\na=ice-ufrag:Hjdi\r\na=ice-pwd:cQPKJaiLGK/1C1gKWpEd/iVr\r\na=ice-options:trickle\r\na=fingerprint:sha-256 1E:B8:EB:48:AD:A9:07:3D:C3:79:59:0C:56:3E:72:68:B1:0F:45:47:F7:BF:60:D3:DE:D5:CA:61:F8:84:43:D7\r\na=setup:actpass\r\na=mid:0\r\na=extmap:1 urn:ietf:params:rtp-hdrext:ssrc-audio-level\r\na=extmap:2 http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time\r\na=extmap:3 http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01\r\na=extmap:4 urn:ietf:params:rtp-hdrext:sdes:mid\r\na=extmap:5 urn:ietf:params:rtp-hdrext:sdes:rtp-stream-id\r\na=extmap:6 urn:ietf:params:rtp-hdrext:sdes:repaired-rtp-stream-id\r\na=sendonly\r\na=msid:- d9b81c90-709e-4b59-8db6-43e1c2d0b898\r\na=rtcp-mux\r\na=rtpmap:111 opus/48000/2\r\na=rtcp-fb:111 transport-cc\r\na=fmtp:111 minptime=10;useinbandfec=1\r\na=rtpmap:103 ISAC/16000\r\na=rtpmap:104 ISAC/32000\r\na=rtpmap:9 G722/8000\r\na=rtpmap:0 PCMU/8000\r\na=rtpmap:8 PCMA/8000\r\na=rtpmap:106 CN/32000\r\na=rtpmap:105 CN/16000\r\na=rtpmap:13 CN/8000\r\na=rtpmap:110 telephone-event/48000\r\na=rtpmap:112 telephone-event/32000\r\na=rtpmap:113 telephone-event/16000\r\na=rtpmap:126 telephone-event/8000\r\na=ssrc:3884480917 cname:NsTEZhd3amIdg0GZ\r\na=ssrc:3884480917 msid:- d9b81c90-709e-4b59-8db6-43e1c2d0b898\r\na=ssrc:3884480917 mslabel:-\r\na=ssrc:3884480917 label:d9b81c90-709e-4b59-8db6-43e1c2d0b898\r\nm=video 9 UDP/TLS/RTP/SAVPF 96 97 98 99 100 101 102 121 127 120 125 107 108 109 35 36 124 119 123 118 114 115 116\r\nc=IN IP4 0.0.0.0\r\na=rtcp:9 IN IP4 0.0.0.0\r\na=ice-ufrag:Hjdi\r\na=ice-pwd:cQPKJaiLGK/1C1gKWpEd/iVr\r\na=ice-options:trickle\r\na=fingerprint:sha-256 1E:B8:EB:48:AD:A9:07:3D:C3:79:59:0C:56:3E:72:68:B1:0F:45:47:F7:BF:60:D3:DE:D5:CA:61:F8:84:43:D7\r\na=setup:actpass\r\na=mid:1\r\na=extmap:14 urn:ietf:params:rtp-hdrext:toffset\r\na=extmap:2 http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time\r\na=extmap:13 urn:3gpp:video-orientation\r\na=extmap:3 http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01\r\na=extmap:12 http://www.webrtc.org/experiments/rtp-hdrext/playout-delay\r\na=extmap:11 http://www.webrtc.org/experiments/rtp-hdrext/video-content-type\r\na=extmap:7 http://www.webrtc.org/experiments/rtp-hdrext/video-timing\r\na=extmap:8 http://www.webrtc.org/experiments/rtp-hdrext/color-space\r\na=extmap:4 urn:ietf:params:rtp-hdrext:sdes:mid\r\na=extmap:5 urn:ietf:params:rtp-hdrext:sdes:rtp-stream-id\r\na=extmap:6 urn:ietf:params:rtp-hdrext:sdes:repaired-rtp-stream-id\r\na=sendonly\r\na=msid:- 8ac166ee-ef00-4330-96e9-d02d73ce121f\r\na=rtcp-mux\r\na=rtcp-rsize\r\na=rtpmap:96 VP8/90000\r\na=rtcp-fb:96 goog-remb\r\na=rtcp-fb:96 transport-cc\r\na=rtcp-fb:96 ccm fir\r\na=rtcp-fb:96 nack\r\na=rtcp-fb:96 nack pli\r\na=rtpmap:97 rtx/90000\r\na=fmtp:97 apt=96\r\na=rtpmap:98 VP9/90000\r\na=rtcp-fb:98 goog-remb\r\na=rtcp-fb:98 transport-cc\r\na=rtcp-fb:98 ccm fir\r\na=rtcp-fb:98 nack\r\na=rtcp-fb:98 nack pli\r\na=fmtp:98 profile-id=0\r\na=rtpmap:99 rtx/90000\r\na=fmtp:99 apt=98\r\na=rtpmap:100 VP9/90000\r\na=rtcp-fb:100 goog-remb\r\na=rtcp-fb:100 transport-cc\r\na=rtcp-fb:100 ccm fir\r\na=rtcp-fb:100 nack\r\na=rtcp-fb:100 nack pli\r\na=fmtp:100 profile-id=2\r\na=rtpmap:101 rtx/90000\r\na=fmtp:101 apt=100\r\na=rtpmap:102 H264/90000\r\na=rtcp-fb:102 goog-remb\r\na=rtcp-fb:102 transport-cc\r\na=rtcp-fb:102 ccm fir\r\na=rtcp-fb:102 nack\r\na=rtcp-fb:102 nack pli\r\na=fmtp:102 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42001f\r\na=rtpmap:121 rtx/90000\r\na=fmtp:121 apt=102\r\na=rtpmap:127 H264/90000\r\na=rtcp-fb:127 goog-remb\r\na=rtcp-fb:127 transport-cc\r\na=rtcp-fb:127 ccm fir\r\na=rtcp-fb:127 nack\r\na=rtcp-fb:127 nack pli\r\na=fmtp:127 level-asymmetry-allowed=1;packetization-mode=0;profile-level-id=42001f\r\na=rtpmap:120 rtx/90000\r\na=fmtp:120 apt=127\r\na=rtpmap:125 H264/90000\r\na=rtcp-fb:125 goog-remb\r\na=rtcp-fb:125 transport-cc\r\na=rtcp-fb:125 ccm fir\r\na=rtcp-fb:125 nack\r\na=rtcp-fb:125 nack pli\r\na=fmtp:125 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42e01f\r\na=rtpmap:107 rtx/90000\r\na=fmtp:107 apt=125\r\na=rtpmap:108 H264/90000\r\na=rtcp-fb:108 goog-remb\r\na=rtcp-fb:108 transport-cc\r\na=rtcp-fb:108 ccm fir\r\na=rtcp-fb:108 nack\r\na=rtcp-fb:108 nack pli\r\na=fmtp:108 level-asymmetry-allowed=1;packetization-mode=0;profile-level-id=42e01f\r\na=rtpmap:109 rtx/90000\r\na=fmtp:109 apt=108\r\na=rtpmap:35 AV1X/90000\r\na=rtcp-fb:35 goog-remb\r\na=rtcp-fb:35 transport-cc\r\na=rtcp-fb:35 ccm fir\r\na=rtcp-fb:35 nack\r\na=rtcp-fb:35 nack pli\r\na=rtpmap:36 rtx/90000\r\na=fmtp:36 apt=35\r\na=rtpmap:124 H264/90000\r\na=rtcp-fb:124 goog-remb\r\na=rtcp-fb:124 transport-cc\r\na=rtcp-fb:124 ccm fir\r\na=rtcp-fb:124 nack\r\na=rtcp-fb:124 nack pli\r\na=fmtp:124 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=4d0032\r\na=rtpmap:119 rtx/90000\r\na=fmtp:119 apt=124\r\na=rtpmap:123 H264/90000\r\na=rtcp-fb:123 goog-remb\r\na=rtcp-fb:123 transport-cc\r\na=rtcp-fb:123 ccm fir\r\na=rtcp-fb:123 nack\r\na=rtcp-fb:123 nack pli\r\na=fmtp:123 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=640032\r\na=rtpmap:118 rtx/90000\r\na=fmtp:118 apt=123\r\na=rtpmap:114 red/90000\r\na=rtpmap:115 rtx/90000\r\na=fmtp:115 apt=114\r\na=rtpmap:116 ulpfec/90000\r\na=ssrc-group:FID 466381623 1665184804\r\na=ssrc:466381623 cname:NsTEZhd3amIdg0GZ\r\na=ssrc:466381623 msid:- 8ac166ee-ef00-4330-96e9-d02d73ce121f\r\na=ssrc:466381623 mslabel:-\r\na=ssrc:466381623 label:8ac166ee-ef00-4330-96e9-d02d73ce121f\r\na=ssrc:1665184804 cname:NsTEZhd3amIdg0GZ\r\na=ssrc:1665184804 msid:- 8ac166ee-ef00-4330-96e9-d02d73ce121f\r\na=ssrc:1665184804 mslabel:-\r\na=ssrc:1665184804 label:8ac166ee-ef00-4330-96e9-d02d73ce121f\r\n
[2021-08-04 19:37:51.920][Trace][11647][69a9vo43] RTC local answer: v=0\r\no=SRS/4.0.146(Leo) 140280753848832 2 IN IP4 0.0.0.0\r\ns=SRSPublishSession\r\nt=0 0\r\na=ice-lite\r\na=group:BUNDLE 0 1\r\na=msid-semantic: WMS live/livestream\r\nm=audio 9 UDP/TLS/RTP/SAVPF 111\r\nc=IN IP4 0.0.0.0\r\na=ice-ufrag:4l2p52xi\r\na=ice-pwd:9098140r300885i5638780744e3b442e\r\na=fingerprint:sha-256 AF:12:C7:3F:A4:E2:D1:58:7C:A3:86:7E:E5:A7:6C:EC:BE:81:DE:1C:B5:08:FC:D1:A9:E4:79:A8:8B:20:31:84\r\na=setup:passive\r\na=mid:0\r\na=extmap:3 http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01\r\na=recvonly\r\na=rtcp-mux\r\na=rtcp-rsize\r\na=rtpmap:111 opus/48000/2\r\na=rtcp-fb:111 transport-cc\r\na=fmtp:111 minptime=10;useinbandfec=1\r\na=candidate:0 1 udp 2130706431 222.95.55.53 8000 typ host generation 0\r\nm=video 9 UDP/TLS/RTP/SAVPF 125 114\r\nc=IN IP4 0.0.0.0\r\na=ice-ufrag:4l2p52xi\r\na=ice-pwd:9098140r300885i5638780744e3b442e\r\na=fingerprint:sha-256 AF:12:C7:3F:A4:E2:D1:58:7C:A3:86:7E:E5:A7:6C:EC:BE:81:DE:1C:B5:08:FC:D1:A9:E4:79:A8:8B:20:31:84\r\na=setup:passive\r\na=mid:1\r\na=extmap:3 http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01\r\na=recvonly\r\na=rtcp-mux\r\na=rtcp-rsize\r\na=rtpmap:125 H264/90000\r\na=rtcp-fb:125 transport-cc\r\na=rtcp-fb:125 nack\r\na=rtcp-fb:125 nack pli\r\na=fmtp:125 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42e01f\r\na=rtpmap:114 red/90000\r\na=candidate:0 1 udp 2130706431 222.95.55.53 8000 typ host generation 0\r\n
[2021-08-04 19:37:51.921][Trace][11647][69a9vo43] TCP: before dispose resource(HttpConn)(0x7f95a6412ec0), conns=1, zombies=0, ign=0, inz=0, ind=0
[2021-08-04 19:37:51.921][Warn][11647][69a9vo43][54] client disconnect peer. ret=1007
[2021-08-04 19:37:51.921][Trace][11647][bl306s7l] TCP: clear zombies=1 resources, conns=1, removing=0, unsubs=0
[2021-08-04 19:37:51.921][Trace][11647][69a9vo43] TCP: disposing #0 resource(HttpConn)(0x7f95a6412ec0), conns=1, disposing=1, zombies=0
[2021-08-04 19:37:51.928][Trace][11647][69a9vo43] RTC: session address init 222.95.55.53:59494
[2021-08-04 19:37:51.928][Trace][11647][69a9vo43] RTC: session STUN done, waiting DTLS handshake.
[2021-08-04 19:37:51.930][Trace][11647][69a9vo43] DTLS: State Passive RECV, done=0, arq=0/0, r0=1, r1=0, len=159, cnt=22, size=146, hs=1
[2021-08-04 19:37:51.930][Trace][11647][69a9vo43] DTLS: State Passive SEND, done=0, arq=0/0, r0=-1, r1=2, len=680, cnt=22, size=82, hs=2
[2021-08-04 19:37:51.934][Trace][11647][69a9vo43] DTLS: State Passive RECV, done=0, arq=0/0, r0=1, r1=0, len=578, cnt=22, size=299, hs=11
[2021-08-04 19:37:51.934][Trace][11647][69a9vo43] DTLS: State Passive SEND, done=1, arq=0/0, r0=1, r1=0, len=554, cnt=22, size=466, hs=4
[2021-08-04 19:37:51.934][Trace][11647][69a9vo43] RTC: DTLS handshake done.
[2021-08-04 19:37:51.935][Trace][11647][69a9vo43] RTC: session pub=1, sub=0, to=30000ms connection established
[2021-08-04 19:37:51.935][Trace][11647][69a9vo43] RTC: Publisher url=/live/livestream established
[2021-08-04 19:37:51.936][Trace][11647][69a9vo43] dvr stream livestream to file ./objs/nginx/html/obs/1628077071935.mp4
[2021-08-04 19:37:51.936][Trace][11647][69a9vo43] ignore disabled exec for vhost=__defaultVhost__
[2021-08-04 19:37:51.936][Trace][11647][69a9vo43] http: mount flv stream for sid=/live/livestream, mount=/live/livestream.flv
[2021-08-04 19:37:51.952][Trace][11647][69a9vo43] 7B audio sh, codec(10, profile=LC, 2channels, 0kbps, 48000HZ), flv(16bits, 2channels, 44100HZ)
[2021-08-04 19:37:51.989][Warn][11647][69a9vo43][35] VIDEO: stream not monotonically increase, please open mix_correct.
[2021-08-04 19:37:51.989][Trace][11647][69a9vo43] 34B video sh,  codec(7, profile=Baseline, level=2.1, 320x240, 0kbps, 0.0fps, 0.0s)
[2021-08-04 19:37:51.989][Trace][11647][69a9vo43] set ts=3361681977, header=10524, lost=10525
[2021-08-04 19:37:51.989][Trace][11647][69a9vo43] NACK: update seq=10524, nack range [10524, 10529]
[2021-08-04 19:37:52.000][Trace][11647][628813k4] HTTP #0 127.0.0.1:51982 GET http://localhost:8020/favicon.ico, content-length=-1
[2021-08-04 19:37:52.000][Trace][11647][628813k4] http match file=./objs/nginx/html/favicon.ico, pattern=/, upath=/favicon.ico
[2021-08-04 19:37:52.002][Trace][11647][628813k4] TCP: before dispose resource(HttpStream)(0x7f95a8004ea0), conns=1, zombies=0, ign=0, inz=0, ind=0
[2021-08-04 19:37:52.002][Warn][11647][628813k4][54] client disconnect peer. ret=1007
[2021-08-04 19:37:52.002][Trace][11647][bl306s7l] TCP: clear zombies=1 resources, conns=1, removing=0, unsubs=0
[2021-08-04 19:37:52.002][Trace][11647][628813k4] TCP: disposing #0 resource(HttpStream)(0x7f95a8004ea0), conns=1, disposing=1, zombies=0
[2021-08-04 19:37:52.020][Warn][11647][o598q86j][35] handle udp pkt, count=1/1, err: code=5015 : size=53, data=[90 7d 29 1c c8 5f 32 39] : marker=0, pt=125, seq=10524, ts=3361681977, ssrc=466381623, pad=0, payload=33B : rtp unprotect r0=9
thread [11647][o598q86j]: cycle() [src/app/srs_app_listener.cpp:630][errno=35]
thread [11647][69a9vo43]: on_rtp() [src/app/srs_app_rtc_conn.cpp:1271][errno=35]
thread [11647][69a9vo43]: unprotect_rtp() [src/app/srs_app_rtc_dtls.cpp:1145][errno=35]
[2021-08-04 19:37:52.031][Trace][11647][69a9vo43] NACK: update seq=10531, nack range [10530, 10531]
[2021-08-04 19:37:52.105][Trace][11647][69a9vo43] NACK: update seq=10543, nack range [10542, 10543]
[2021-08-04 19:37:52.210][Trace][11647][f2o079bf] Hybrid cpu=0.00%,0MB
[2021-08-04 19:37:52.210][Trace][11647][f2o079bf] RTC: Server conns=1
[2021-08-04 19:37:52.352][Trace][11647][f2o079bf][MAYBE] RTC: NACK timeout=1, request PLI, track=8ac166ee-ef00-4330-96e9-d02d73ce121f, ssrc=466381623
[2021-08-04 19:37:57.211][Trace][11647][f2o079bf] Hybrid cpu=0.00%,0MB
[2021-08-04 19:37:57.211][Trace][11647][f2o079bf] RTC: Server conns=1
[2021-08-04 19:37:57.253][Trace][11647][o598q86j] <- RTC RECV #10, udp 526, pps 52/52, schedule 526
[2021-08-04 19:37:57.925][Trace][11647][f2o079bf] RTC: Need PLI ssrc=466381623, play=[f2o079bf], publish=[69a9vo43], count=1/1
[2021-08-04 19:37:57.925][Trace][11647][69a9vo43] RTC: Request PLI ssrc=466381623, play=[f2o079bf], count=1/1, bytes=12B
[2021-08-04 19:37:57.951][Trace][11647][69a9vo43] 34B video sh,  codec(7, profile=Baseline, level=2.1, 320x240, 0kbps, 0.0fps, 0.0s)
[2021-08-04 19:37:57.951][Trace][11647][69a9vo43] set ts=3362216307, header=10826, lost=10827
[2021-08-04 19:38:00.912][Trace][11647][12w4348i] HTTP #0 127.0.0.1:52020 GET http://localhost:8020/live/livestream.flv, content-length=-1
[2021-08-04 19:38:00.912][Trace][11647][12w4348i] dispatch cached gop success. count=0, duration=19
[2021-08-04 19:38:00.912][Trace][11647][12w4348i] create consumer, active=1, queue_size=0.00, jitter=30000000
[2021-08-04 19:38:00.912][Trace][11647][12w4348i] set fd=16, SO_SNDBUF=146988=>87500, buffer=350ms
[2021-08-04 19:38:00.912][Trace][11647][12w4348i] FLV /live/livestream.flv, encoder=FLV, nodelay=0, mw_sleep=350ms, cache=0, msgs=128
[2021-08-04 19:38:00.912][Trace][11647][12w4348i] FLV: write header audio=1, video=1
[2021-08-04 19:38:02.211][Trace][11647][f2o079bf] Hybrid cpu=0.00%,0MB, cid=6,4, timer=59,7,32, clock=0,23,22,1,0,0,0,0,0, free=1, objs=(pkt:100,raw:58,fua:41,msg:189,oth:1,buf:67)
[2021-08-04 19:38:02.211][Trace][11647][f2o079bf] RTC: Server conns=1
[2021-08-04 19:38:03.985][Trace][11647][69a9vo43] 34B video sh,  codec(7, profile=Baseline, level=2.1, 320x240, 0kbps, 0.0fps, 0.0s)
[2021-08-04 19:38:03.985][Trace][11647][69a9vo43] set ts=3362762067, header=11089, lost=11090
[2021-08-04 19:38:07.212][Trace][11647][f2o079bf] Hybrid cpu=0.00%,0MB, cid=6,4, timer=59,7,32, clock=0,23,22,1,0,0,0,0,0, free=1, objs=(pkt:100,raw:58,fua:41,msg:189,oth:1,buf:67)
[2021-08-04 19:38:07.212][Trace][11647][f2o079bf] RTC: Server conns=1, rpkts=(99,rtp:97,stun:1,rtcp:1), spkts=(14,rtp:0,stun:1,rtcp:24), rtcp=(pli:1,twcc:9,rr:1), snk=(95,a:47,v:47,h:1), fid=(id:0,fid:0,ffid:99,addr:1,faddr:99)
[2021-08-04 19:38:07.258][Trace][11647][o598q86j] <- RTC RECV #10, udp 1014, pps 76/101, schedule 1014
[2021-08-04 19:38:10.024][Trace][11647][f2o079bf] RTC: Need PLI ssrc=466381623, play=[f2o079bf], publish=[69a9vo43], count=3/3
[2021-08-04 19:38:10.024][Trace][11647][69a9vo43] RTC: Request PLI ssrc=466381623, play=[f2o079bf], count=3/3, bytes=12B
[2021-08-04 19:38:10.054][Trace][11647][69a9vo43] 34B video sh,  codec(7, profile=Baseline, level=2.1, 320x240, 0kbps, 0.0fps, 0.0s)
[2021-08-04 19:38:10.054][Trace][11647][69a9vo43] set ts=3363305847, header=11432, lost=11433
[2021-08-04 19:38:11.084][Trace][11647][12w4348i] -> HTS http: got 26 msgs, age=10173046, min=8, mw=350
[2021-08-04 19:38:12.212][Trace][11647][f2o079bf] Hybrid cpu=0.00%,0MB, cid=1,1, timer=61,10,47, clock=0,32,14,0,0,0,0,0,0, objs=(pkt:162,raw:75,fua:85,msg:394,oth:1,buf:106)
[2021-08-04 19:38:12.212][Trace][11647][f2o079bf] RTC: Server conns=1, rpkts=(99,rtp:97,stun:1,rtcp:1), spkts=(14,rtp:0,stun:1,rtcp:24), rtcp=(pli:1,twcc:9,rr:1), snk=(95,a:47,v:47,h:1), fid=(id:0,fid:0,ffid:99,addr:1,faddr:99)
[2021-08-04 19:38:16.091][Trace][11647][69a9vo43] 34B video sh,  codec(7, profile=Baseline, level=2.1, 320x240, 0kbps, 0.0fps, 0.0s)
[2021-08-04 19:38:16.091][Trace][11647][69a9vo43] set ts=3363851517, header=11843, lost=11844
[2021-08-04 19:38:17.212][Trace][11647][f2o079bf] Hybrid cpu=0.00%,0MB, cid=1,1, timer=61,10,47, clock=0,32,14,0,0,0,0,0,0, objs=(pkt:162,raw:75,fua:85,msg:394,oth:1,buf:106)
[2021-08-04 19:38:17.212][Trace][11647][f2o079bf] RTC: Server conns=1, rpkts=(118,rtp:116,stun:1,rtcp:1), spkts=(14,rtp:0,stun:1,rtcp:23), rtcp=(pli:1,twcc:9,rr:1), snk=(95,a:47,v:47,h:0), fid=(id:0,fid:0,ffid:118,addr:1,faddr:118)
[2021-08-04 19:38:17.284][Trace][11647][o598q86j] <- RTC RECV #10, udp 1183, pps 90/118, schedule 1183
[2021-08-04 19:38:21.252][Trace][11647][12w4348i] -> HTS http: got 27 msgs, age=20345209, min=8, mw=350
[2021-08-04 19:38:22.106][Trace][11647][f2o079bf] RTC: Need PLI ssrc=466381623, play=[f2o079bf], publish=[69a9vo43], count=5/5
[2021-08-04 19:38:22.106][Trace][11647][69a9vo43] RTC: Request PLI ssrc=466381623, play=[f2o079bf], count=5/5, bytes=12B
[2021-08-04 19:38:22.151][Trace][11647][69a9vo43] 34B video sh,  codec(7, profile=Baseline, level=2.1, 320x240, 0kbps, 0.0fps, 0.0s)
[2021-08-04 19:38:22.151][Trace][11647][69a9vo43] set ts=3364392597, header=12281, lost=12282
[2021-08-04 19:38:22.213][Trace][11647][f2o079bf] Hybrid cpu=0.00%,0MB, cid=1,1, timer=61,10,47, clock=0,32,14,0,0,0,0,0,0, objs=(pkt:162,raw:75,fua:85,msg:394,oth:1,buf:106)
[2021-08-04 19:38:22.213][Trace][11647][f2o079bf] RTC: Server conns=1, rpkts=(118,rtp:116,stun:1,rtcp:1), spkts=(14,rtp:0,stun:1,rtcp:23), rtcp=(pli:1,twcc:9,rr:1), snk=(95,a:47,v:47,h:0), fid=(id:0,fid:0,ffid:118,addr:1,faddr:118)
  1. The configuration of SRS is as follows:
listen              1935;
pid                 ./objs/srs.1935.pid;
max_connections     1000;
daemon              off;
srs_log_tank        console;

http_server {
    enabled         on;
    listen          8020;
    dir             ./objs/nginx/html;
}

http_api {
    enabled         on;
    listen          1985;
}
stats {
    network         0;
}
rtc_server {
    enabled         on;
    listen          8000;
    candidate       $CANDIDATE;
}

vhost __defaultVhost__ {
    rtc {
        enabled         on;
        rtc_to_rtmp     on;
    }
    dvr {
        enabled         on;
        dvr_path        ./objs/nginx/html/obs/[timestamp].mp4;
    }
    http_remux {
        enabled     on;
        mount       [vhost]/[app]/[stream].flv;
    }
}

Replay

How to replay bug?

Steps to reproduce the bug

Steps to reproduce the bug:

  1. Start SRS: ./objs/srs -c conf/test.conf
  2. Push RTC stream: http://localhost:8020/players/rtc_publisher.html
  3. Preview: http://localhost:8020/players/srs_player.html?port=8020
  4. Stop pushing the stream and check the recorded video: ./objs/nginx/html/obs/[timestamp].mp4
  5. In the above steps, the preview video is approximately 1-2 seconds ahead of the audio, and the recorded video has audio approximately 3-4 seconds ahead of the video.

Expected behavior:

The audio and video should be synchronized in both the preview and recorded videos.

RTC to RTMP with normal audio and video synchronization:

The audio and video synchronization function is working properly when converting RTC to RTMP.

TRANS_BY_GPT3
@bywwcnll
Copy link
Author

bywwcnll commented Aug 4, 2021 •
edited by winlinvip
Loading

After testing, all the above functions are working fine when using OBS for direct streaming (without using RTC functionality). Please pay attention to it, experts.

TRANS_BY_GPT3

@duiniuluantanqin duiniuluantanqin linked a pull request Aug 16, 2021 that will close this issue
RTC: Fix rtc to rtmp sync timestamp using sender report. #2470
Merged
@winlinvip winlinvip added Bug It might be a bug. WebRTC WebRTC, RTC2RTMP or RTMP2RTC. labels Aug 27, 2021
@winlinvip winlinvip added this to the 4.0 milestone Sep 4, 2021
@winlinvip winlinvip changed the title WebRTC 转 RTMP 音画不同步问题 WebRTC to RTMP audio and video synchronization issue Jul 29, 2023
@winlinvip winlinvip added the TransByAI Translated by AI/GPT. label Jul 29, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@duiniuluantanqin duiniuluantanqin

Labels
Bug It might be a bug. TransByAI Translated by AI/GPT. WebRTC WebRTC, RTC2RTMP or RTMP2RTC.
Projects
None yet
Milestone
4.0
Development

Successfully merging a pull request may close this issue.

RTC: Fix rtc to rtmp sync timestamp using sender report. xiaozhihong/srs
3 participants
@winlinvip @duiniuluantanqin @bywwcnll