Empty RemoteParticipant Issue

[williamleeag6688]

LiveKit SDK Version

V0.3.2

IDF Version

v5.4.0(ESPIDF IN ESPADF)

Issue Description

Failed:

I (62335) livekit_signaling: Connecting to server: wss://secrect/rtc?sdk=esp32&version=0.3.2&os=idf&os_version=v5.4-dirty&device_model=9&auto_subscribe=false&protocol=1
I (62348) websocket_client: Started
I (62351) livekit_example: Room state changed: Connecting
I (62690) esp-x509-crt-bundle: Certificate validated
I (64593) livekit_example: Agent has joined the room
I (64593) DTLS: Init SRTP OK
I (65099) AGENT: Start agent as Controlling
I (65100) PEER_DEF: Start DTLS role as 1
I (65101) livekit_engine: signal_res(Connecting): UPDATE participants=1
I (65103) livekit_engine: participant_update: ts=63798407 us, participants=1
I (65111) livekit_engine: participant[0]: sid=PA_HFJo8RnbaQuA, kind=0, state=1, tracks_count=0
I (65113) AGENT: Send STUN binding request
I (65124) AGENT: Send allocate now
I (65193) AGENT: Send allocate now
I (65293) AGENT: 0 Get candidate success user:secrect
W (65294) AGENT: Skip TCP stun server turns:secrect?transport=tcp
W (65302) AGENT: Skip TCP stun server turns:secrect?transport=tcp
I (65311) livekit_peer.pub: Generated offer:
v=0
o=- 7801614086411001801 2 IN IP4 0.0.0.0
s=-
t=0 0
a=msid-semantic: esp-webrtc
a=group:BUNDLE 0 1
m=audio 9 UDP/TLS/RTP/SAVP 111
a=rtpmap:111 opus/48000/2
a=ssrc:6 cname:webrtc-opus
a=ssrc:6 msid:esp_stream a0
a=sendrecv
a=mid:0
a=msid:esp_stream a0
c=IN IP4 0.0.0.0
a=rtcp-mux
a=fingerprint:sha-256 secrect
a=setup:passive
a=ice-ufrag:WwUl
a=ice-pwd:secrect
a=candidate:0 1 UDP 2130388991 192.168.0.101 64295 typ host
a=candidate:1 1 UDP 1694181375 218.103.102.60 64295 typ srflx raddr 0.0.0.0 rport 0
a=candidate:2 1 UDP 7320831 143.223.95.144 28596 typ relay raddr 0.0.0.0 rport 0
m=application 50712 UDP/DTLS/SCTP webrtc-datachannel
a=mid:1
a=sctp-port:5000
c=IN IP4 0.0.0.0
a=max-message-size:262144
a=fingerprint:sha-256 secrect
a=setup:passive
a=ice-ufrag:WwUl
a=ice-pwd:secrect
a=candidate:0 1 UDP 2130388991 192.168.0.101 64295 typ host
a=candidate:1 1 UDP 1694181375 218.103.102.60 64295 typ srflx raddr 0.0.0.0 rport 0
a=candidate:2 1 UDP 7320831 143.223.95.144 28596 typ relay raddr 0.0.0.0 rport 0

I (65570) PEER_DEF: Get peer role 0
I (65570) PEER_DEF: Get peer role 0
SDP not contain candidate continue
I (65589) AGENT: 0 Add remote type:1 143.223.95.144:50009
0 Sorted pair 0 type: 1 local:192.168.0.101:64295 Remote:143.223.95.144:50009
I (65593) livekit_peer.pub: State changed: 0 -> 1
0 Sorted pair 0 type: 1 local:192.168.0.101:64295 Remote:143.223.95.144:50009
I (65606) AGENT: 0 Send binding request (cand:0) local:192.168.0.101:64295 remote:143.223.95.144:50009 id:329b54fb381fc0385378d5a8
I (65707) AGENT: 0 Select pair143.223.95.144:50009
I (65708) AGENT: 0 Send binding request (cand:1) local:192.168.0.101:64295 remote:143.223.95.144:50009 id:76e5004e797d87a62756c714
I (65713) AGENT: 0 PeerBinding recv local:192.168.0.101:64295 remote:143.223.95.144:50009
I (65720) AGENT: 0 Send binding response local:192.168.0.101:64295 remote:143.223.95.144:50009
0 Sorted pair 0 type: 1 local:192.168.0.101:64295 Remote:143.223.95.144:50009
0 Sorted pair 0 type: 1 local:192.168.0.101:64295 Remote:143.223.95.144:50009
I (65770) AGENT: 0 PeerBinding recv local:192.168.0.101:64295 remote:143.223.95.144:50009
I (65770) AGENT: 0 Send binding response local:192.168.0.101:64295 remote:143.223.95.144:50009
I (65793) AGENT: 0 Candidate responsed
I (65793) AGENT: 0 Connection OK 143.223.95.144:50009
I (65803) DTLS: Start to do server handshake
I (66121) DTLS: SRTP connected OK
I (66123) DTLS: Server handshake success
I (66124) PEER_DEF: DTLS handshake success
I (66124) PEER_DEF: Sctp role as 0
I (66127) SCTP: Send DCEP _reliable event:3 type:0 si:1
I (66131) SCTP: Send DCEP _lossy event:3 type:81 si:3
I (66187) livekit_engine: signal_res(Connecting): UPDATE participants=1
I (66188) livekit_engine: participant_update: ts=64883282 us, participants=1
I (66188) SCTP: 0 Receive chunk 1 SCTP_INIT
I (66191) livekit_engine: participant[0]: sid=PA_HFJo8RnbaQuA, kind=0, state=2, tracks_count=1
I (66195) SCTP: 0 state 2
I (66205) livekit_engine: signal_res(Connecting): UPDATE participants=1
I (66207) SCTP: Support ext 82
I (66215) livekit_engine: participant_update: ts=64909603 us, participants=1
I (66217) SCTP: Support ext c0
I (66225) livekit_engine: participant[0]: sid=PA_HFJo8RnbaQuA, kind=0, state=2, tracks_count=1
I (66227) SCTP: Send INIT_ACK chunk
I (66287) SCTP: 0 Receive chunk 10 SCTP_COOKIE_ECHO
I (66288) SCTP: Send ECHO_ACK chunk
I (66288) SCTP: 0 state 5
I (66360) livekit_peer.pub: Channel open: label=_reliable, stream_id=1
I (66361) livekit_peer.pub: Channel open: label=_lossy, stream_id=3
I (66363) livekit_peer.pub: State changed: 1 -> 2
I (66369) AUD_PIPE_NEGO: Negotiate return 0 src_format:541934416 sample_rate:16000

I (66377) AUD_PIPE_NEGO: Path mask 1 select sink:0 format 1398100047
I (66384) AUD_SRC: Get rate:16000, ch:1, bits:16
W (66389) ESP_GMF_PIPELINE: There is no thread for add jobs, pipe:0x3c3b1704, tsk:0x0, [el:aud_src-0x3c3b1454]
I (66400) ESP_GMF_TASK: Waiting to run... [tsk:aenc_0-0x3fce4424, wk:0x0, run:0]
I (66408) livekit_example: Room state changed: Connected
I (66690) RINGMUSIC: [music] codec music played, 351087 bytes are written.
I (66690) RINGMUSIC: [music] about to close render_device ts=65385082 us
I (66699) RINGMUSIC: [music] closed render_device ts=65393892 us
E (66900) i2s_common: i2s_channel_disable(1187): the channel has not been enabled yet
I (66900) I2S_IF: channel mode 0 bits:16/16 channel:2 mask:3
I (66904) I2S_IF: STD Mode 1 bits:16/16 channel:2 sample_rate:16000 mask:3
I (66926) Adev_Codec: Open codec device OK
I (67208) I2S_IF: Mode 1 need extend bits 32 to 64
I (67208) I2S_IF: channel mode 2 bits:16/16 channel:4 mask:3
I (67209) I2S_IF: TDM Mode 0 bits:16/16 channel:4 sample_rate:16000 mask:3
I (67217) I2S_IF: channel mode 0 bits:16/32 channel:2 mask:3
I (67223) I2S_IF: STD Mode 1 bits:16/32 channel:2 sample_rate:16000 mask:3
I (67239) Adev_Codec: Open codec device OK
E (67239) MODEL_LOADER: Can not find model in partition table
W (67242) AUD_AEC_SRC: No model to load
W (67246) AFE_CONFIG: wakenet model not found. please load wakenet model...
I (67295) AFE: AFE Version: (1MIC_V250121)
I (67295) AFE: Input PCM Config: total 2 channels(1 microphone, 1 playback), sample rate:16000
I (67296) AFE: AFE Pipeline: [input] -> |AEC(SR_LOW_COST)| -> |VAD(WebRTC)| -> [output]
I (67305) AUD_SRC: Start to fetch audio src data now
I (67305) ESP_GMF_TASK: One times job is complete, del[wk:0x3c3aab78, ctx:0x3c3b1454, label:aud_src_open]
I (67320) ESP_GMF_PORT: ACQ IN, new self payload:0x3c3aab78, port:0x3c3ac1a8, el:0x3c3b1454-aud_src
I (67330) ESP_GMF_TASK: One times job is complete, del[wk:0x3c422934, ctx:0x3c3b1e1c, label:aud_ch_cvt_open]
I (67341) ESP_GMF_AENC: Open, type:OPUS, acquire in frame: 1280, out frame: 100
I (67348) ESP_GMF_TASK: One times job is complete, del[wk:0x3c42297c, ctx:0x3c3b1ec8, label:aud_enc_open]

Success:

I (20538) livekit_signaling: Connecting to server: wss://secrect?sdk=esp32&version=0.3.2&os=idf&os_version=v5.4-dirty&device_model=9&auto_subscribe=false&protocol=1
I (20551) websocket_client: Started
I (20554) livekit_example: Room state changed: Connecting
I (21000) esp-x509-crt-bundle: Certificate validated
I (21798) DTLS: Init SRTP OK
I (22925) AGENT: Start agent as Controlling
I (22925) PEER_DEF: Start DTLS role as 1
I (22927) livekit_engine: signal_res(Connecting): UPDATE participants=1
I (22929) livekit_engine: participant_update: ts=21624052 us, participants=1
I (22937) livekit_engine: participant[0]: sid=PA_8kkiDVrFFas2, kind=0, state=1, tracks_count=0
I (22946) livekit_engine: signal_res(Connecting): UPDATE participants=1
I (22953) livekit_engine: participant_update: ts=21648350 us, participants=1
I (22961) livekit_engine: participant[0]: sid=PA_8rABFMshu86z, kind=4, state=2, tracks_count=1
I (22970) livekit_engine: participant[0].track[0]: type=0 sid=TR_AMyxSbzyrZSaw9 muted=0
I (22979) livekit_engine: Subscribing to audio track: sid=TR_AMyxSbzyrZSaw9
I (22987) livekit_engine: subscribe_tracks: ts=21681686 us, renderer_handle=0x3fcd7a50
I (22997) livekit_example: Agent has joined the room
I (23007) AGENT: Send STUN binding request
I (23007) AGENT: Send allocate now
I (23096) AGENT: Send allocate now
I (23175) AGENT: 0 Get candidate success user:secrect psw:Q7b6KdB7PqgjI1ejyO71otyhl6qeooTDL2UsNSIsqVq
W (23176) AGENT: Skip TCP stun server turns:secrect?transport=tcp
W (23184) AGENT: Skip TCP stun server turns:secrect:443?transport=tcp
I (23193) livekit_peer.pub: Generated offer:
v=0
o=- 7801614086411001801 2 IN IP4 0.0.0.0
s=-
t=0 0
a=msid-semantic: esp-webrtc
a=group:BUNDLE 0 1
m=audio 9 UDP/TLS/RTP/SAVP 111
a=rtpmap:111 opus/48000/2
a=ssrc:6 cname:webrtc-opus
a=ssrc:6 msid:esp_stream a0
a=sendrecv
a=mid:0
a=msid:esp_stream a0
c=IN IP4 0.0.0.0
a=rtcp-mux
a=fingerprint:sha-256 secrect
a=setup:passive
a=ice-ufrag:WwUl
a=ice-pwd:secrect
a=candidate:0 1 UDP 2130039039 192.168.0.101 62928 typ host
a=candidate:1 1 UDP 1693831423 218.103.102.60 62928 typ srflx raddr 0.0.0.0 rport 0
a=candidate:2 1 UDP 5675519 143.223.95.129 22169 typ relay raddr 0.0.0.0 rport 0
m=application 50712 UDP/DTLS/SCTP webrtc-datachannel
a=mid:1
a=sctp-port:5000
c=IN IP4 0.0.0.0
a=max-message-size:262144
a=fingerprint:sha-256 secrect
a=setup:passive
a=ice-ufrag:WwUl
a=ice-pwd:Pd9ZwfQgpeN3WiStdYMx5Ru0
a=candidate:0 1 UDP 2130039039 192.168.0.101 62928 typ host
a=candidate:1 1 UDP 1693831423 218.103.102.60 62928 typ srflx raddr 0.0.0.0 rport 0
a=candidate:2 1 UDP 5675519 143.223.95.129 22169 typ relay raddr 0.0.0.0 rport 0

I (23197) PEER_DEF: Get peer role 0
I (23313) PEER_DEF: A mid: 0
I (23316) PEER_DEF: A SRC: 3623209765
I (23319) PEER_DEF: A SRC: 3623209765
I (23323) PEER_DEF: A SRC: 3623209765
I (23326) PEER_DEF: A SRC: 3623209765
SDP not contain candidate continue
I (23333) AGENT: 1 Add remote type:1 143.223.95.129:50004
I (23347) AGENT: Start agent as Controlled
I (23347) PEER_DEF: Start DTLS role as 1
I (23348) AGENT: Send STUN binding request
I (23349) AGENT: Send allocate now
I (23397) AGENT: Send allocate now
I (23442) AGENT: 1 Get candidate success user:secrect psw:secrect
W (23443) AGENT: Skip TCP stun server turns:secrect?transport=tcp
W (23451) AGENT: Skip TCP stun server turns:secrect?transport=tcp
1 Sorted pair 0 type: 1 local:192.168.0.101:62929 Remote:143.223.95.129:50004
I (23467) livekit_engine: on_peer_sub_audio_info: ts=22161858 us codec=3 sr=16000 ch=2 renderer=0x3fcd7a50
I (23479) livekit_peer.sub: Generated answer:
v=0
o=- 7801614086411001801 2 IN IP4 0.0.0.0
s=-
t=0 0
a=msid-semantic: esp-webrtc
a=group:BUNDLE 0
m=audio 9 UDP/TLS/RTP/SAVP 111
a=rtpmap:111 opus/48000/2
a=ssrc:6 cname:webrtc-opus
a=ssrc:6 msid:esp_stream a0
a=sendrecv
a=mid:0
a=msid:esp_stream a0
c=IN IP4 0.0.0.0
a=rtcp-mux
a=fingerprint:sha-256 secrect
a=setup:passive
a=ice-ufrag:WwUl
a=ice-pwd:secrect
a=candidate:0 1 UDP 2130039295 192.168.0.101 62929 typ host
a=candidate:1 1 UDP 1693831679 218.103.102.60 62929 typ srflx raddr 0.0.0.0 rport 0
a=candidate:2 1 UDP 5542655 143.223.95.129 21650 typ relay raddr 0.0.0.0 rport 0

I (23482) PEER_DEF: Get peer role 0
I (23546) livekit_peer.sub: State changed: 0 -> 1
I (23549) PEER_DEF: Get peer role 0
I (23565) AGENT: 1 Send binding request (cand:0) local:192.168.0.101:62929 remote:143.223.95.129:50004 id:329b54fb381fc0385378d5a8
SDP not contain candidate continue
I (23634) AGENT: 0 Add remote type:1 143.223.95.129:50007
0 Sorted pair 0 type: 1 local:192.168.0.101:62928 Remote:143.223.95.129:50007
I (23638) livekit_peer.pub: State changed: 0 -> 1
0 Sorted pair 0 type: 1 local:192.168.0.101:62928 Remote:143.223.95.129:50007
I (23651) AGENT: 0 Send binding request (cand:0) local:192.168.0.101:62928 remote:143.223.95.129:50007 id:329b54fb381fc0385378d5a8
I (23671) AGENT: 0 PeerBinding recv local:192.168.0.101:62928 remote:143.223.95.129:50007
I (23671) AGENT: 0 Send binding response local:192.168.0.101:62928 remote:143.223.95.129:50007
0 Sorted pair 0 type: 1 local:192.168.0.101:62928 Remote:143.223.95.129:50007
I (23681) RINGMUSIC: [music] codec music played, 351087 bytes are written.
0 Sorted pair 0 type: 1 local:192.168.0.101:62928 Remote:143.223.95.129:50007
I (23701) RINGMUSIC: [music] about to close render_device ts=22395912 us
I (23708) AGENT: 0 Select pair143.223.95.129:50007
I (23711) AGENT: 0 Send binding request (cand:1) local:192.168.0.101:62928 remote:143.223.95.129:50007 id:76e5004e797d87a62756c714
I (23712) RINGMUSIC: [music] closed render_device ts=22407089 us
I (23724) AGENT: 0 PeerBinding recv local:192.168.0.101:62928 remote:143.223.95.129:50007
I (23737) AGENT: 0 Send binding response local:192.168.0.101:62928 remote:143.223.95.129:50007
I (23768) AGENT: 0 Candidate responsed
I (23768) AGENT: 0 Connection OK 143.223.95.129:50007
I (23778) DTLS: Start to do server handshake
I (23832) AGENT: 1 PeerBinding recv local:192.168.0.101:62929 remote:143.223.95.129:50004
E (23929) i2s_common: i2s_channel_disable(1187): the channel has not been enabled yet
I (23929) I2S_IF: channel mode 0 bits:16/16 channel:2 mask:3
I (23933) I2S_IF: STD Mode 1 bits:16/16 channel:2 sample_rate:16000 mask:3
I (23955) Adev_Codec: Open codec device OK
I (24110) DTLS: SRTP connected OK
I (24112) DTLS: Server handshake success
I (24112) PEER_DEF: DTLS handshake success
I (24112) PEER_DEF: Sctp role as 0
I (24115) SCTP: Send DCEP _reliable event:3 type:0 si:1
I (24120) SCTP: Send DCEP _lossy event:3 type:81 si:3
I (24159) SCTP: 0 Receive chunk 1 SCTP_INIT
I (24159) SCTP: 0 state 2
I (24159) SCTP: Support ext 82
I (24159) SCTP: Support ext c0
I (24159) SCTP: Send INIT_ACK chunk
I (24163) livekit_engine: signal_res(Connecting): UPDATE participants=1
I (24170) livekit_engine: participant_update: ts=22865127 us, participants=1
I (24178) livekit_engine: participant[0]: sid=PA_8kkiDVrFFas2, kind=0, state=2, tracks_count=1
I (24187) livekit_engine: signal_res(Connecting): UPDATE participants=1
I (24194) livekit_engine: participant_update: ts=22889415 us, participants=1
I (24202) livekit_engine: participant[0]: sid=PA_8kkiDVrFFas2, kind=0, state=2, tracks_count=1
I (24209) SCTP: 0 Receive chunk 10 SCTP_COOKIE_ECHO
I (24216) SCTP: Send ECHO_ACK chunk
I (24219) SCTP: 0 state 5
I (24267) livekit_peer.pub: Channel open: label=_reliable, stream_id=1
I (24268) livekit_peer.pub: Channel open: label=_lossy, stream_id=3
I (24270) livekit_peer.pub: State changed: 1 -> 2
I (24276) AUD_PIPE_NEGO: Negotiate return 0 src_format:541934416 sample_rate:16000

I (24284) AUD_PIPE_NEGO: Path mask 1 select sink:0 format 1398100047
I (24291) AUD_SRC: Get rate:16000, ch:1, bits:16
W (24296) ESP_GMF_PIPELINE: There is no thread for add jobs, pipe:0x3c3b2084, tsk:0x0, [el:aud_src-0x3c3ac5d4]
I (24308) ESP_GMF_TASK: Waiting to run... [tsk:aenc_0-0x3fce5440, wk:0x0, run:0]
I (24315) livekit_example: Room state changed: Connected
I (24429) AGENT: 1 PeerBinding recv local:192.168.0.101:62929 remote:143.223.95.129:50004
I (24429) AGENT: 1 Select pair143.223.95.129:50004
I (24431) AGENT: 1 Connection OK 143.223.95.129:50004
I (24445) DTLS: Start to do server handshake
I (24797) DTLS: SRTP connected OK
I (24799) DTLS: Server handshake success
I (24799) PEER_DEF: DTLS handshake success
I (24800) PEER_DEF: Sctp role as 0
I (24815) I2S_IF: Mode 1 need extend bits 32 to 64
I (24815) I2S_IF: channel mode 2 bits:16/16 channel:4 mask:3
I (24816) I2S_IF: TDM Mode 0 bits:16/16 channel:4 sample_rate:16000 mask:3
I (24824) I2S_IF: channel mode 0 bits:16/32 channel:2 mask:3
I (24830) I2S_IF: STD Mode 1 bits:16/32 channel:2 sample_rate:16000 mask:3
I (24846) Adev_Codec: Open codec device OK
E (24846) MODEL_LOADER: Can not find model in partition table
W (24849) AUD_AEC_SRC: No model to load
W (24853) AFE_CONFIG: wakenet model not found. please load wakenet model...
I (24902) AFE: AFE Version: (1MIC_V250121)
I (24902) AFE: Input PCM Config: total 2 channels(1 microphone, 1 playback), sample rate:16000
I (24903) AFE: AFE Pipeline: [input] -> |AEC(SR_LOW_COST)| -> |VAD(WebRTC)| -> [output]
I (24912) AUD_SRC: Start to fetch audio src data now
I (24912) ESP_GMF_TASK: One times job is complete, del[wk:0x3c3aa6b4, ctx:0x3c3ac5d4, label:aud_src_open]
I (24927) ESP_GMF_PORT: ACQ IN, new self payload:0x3c3aa6b4, port:0x3c3b2100, el:0x3c3ac5d4-aud_src
I (24937) ESP_GMF_TASK: One times job is complete, del[wk:0x3c422618, ctx:0x3c3b1650, label:aud_ch_cvt_open]
I (24948) ESP_GMF_AENC: Open, type:OPUS, acquire in frame: 1280, out frame: 100
I (24955) ESP_GMF_TASK: One times job is complete, del[wk:0x3c422afc, ctx:0x3c3b13a4, label:aud_enc_open]
I (25237) livekit_engine: on_peer_sub_audio_frame: ts=23932354 us size=3 pts=8081345
Get samplerate 16000 chanel 2
I (25240) AV_RENDER: Get need resample 1 in:16000 out:16000
I (25244) I2S_RENDER: open channel:2 sample_rate:16000 bits:16
I (25250) Adev_Codec: Input already open
I (25255) AV_RENDER: ARender Auto pause audio first frame at 8081345
I (25257) livekit_engine: on_peer_sub_audio_frame: ts=23952392 us size=1 pts=8081405
I (25636) livekit_engine: on_peer_sub_audio_frame: ts=24331490 us size=190 pts=8082545
D 9
D 9

There is a case that when the latency of WIFI connection is high, the remote participants will not join the room and no track was added, I am not sure it is because the server didn't send this message to the device or the device didn't receive due to some packet missed, but whatever a confirmation after the device receive this message is required, then the handshake continue, otherwise there is always this kind of problem that even connected but no audio. If there any solution(reconnect mechanism)? This case is in urgent, please help.

Hardware Configuration

ESP32S3 ES8311.

[ladvoc]

Hi @williamleeag6688, sorry to hear you're running into this issue. Could you please share if you are connecting to an agent or another client?

[williamleeag]

Hi @williamleeag6688, sorry to hear you're running into this issue. Could you please share if you are connecting to an agent or another client?

Hi ladvoc,

Should be an agent, it is an Livekit voice to voice agent. Could you help with this? Or what exact you want me to share? Since this part of code is almost all the same as the livekit component in this github.

Best,
William

[ladvoc]

Got it, I just wanted to confirm your use case is talking with an agent. I will take a look at this on my end and let you know what I find.

[williamleeag6688]

Got it, I just wanted to confirm your use case is talking with an agent. I will take a look at this on my end and let you know what I find.

Thank you so much, we used to use another sdk to do the connection but meet some connection problem during actual scene, we want to migrate to this one but the same that there are some network issue seems will cause failures.

[williamleeag6688]

Got it, I just wanted to confirm your use case is talking with an agent. I will take a look at this on my end and let you know what I find.

Hi, do we have any update progress? Thanks.

[zkoch]

Got it, I just wanted to confirm your use case is talking with an agent. I will take a look at this on my end and let you know what I find.

I'm working with William. This is not using LiveKit Agents, it's just connecting to a remote livekit room

[ladvoc]

Thank you, that additional context is helpful. Looking into this, I unfortunately haven't been able to reproduce on my end yet. However, there is a mechanism in place to trigger a reconnection in the event the client stops receiving pings from the SFU—as should be the case with a poor Wi-Fi connection. To ensure this mechanism is working properly, please add a log in this function and let me know if this gets triggered.

[williamleeag6688]

Thank you, that additional context is helpful. Looking into this, I unfortunately haven't been able to reproduce on my end yet. However, there is a mechanism in place to trigger a reconnection in the event the client stops receiving pings from the SFU—as should be the case with a poor Wi-Fi connection. To ensure this mechanism is working properly, please add a log in this function and let me know if this gets triggered.

Hi ladvoc,

I try and when it triggered the above issue I mentioned, the log you suggest to add didn't gets triggered, but same as before, I can only trigger the above issue when I am using the poor WIFI(1.8Mps, 11-1850ms ping latency download, 9Mbps, 30-393ms latency updload,speedtest result), if it possible for you to test it with some network environment tools as well? Thank you so much

[ladvoc]

I am going to inspect the SFU logs for your cloud project. In the meantime, if it’s not too much trouble, please try reproducing the issue using this SDK branch and share the log output here.

[williamleeag6688]

I am going to inspect the SFU logs for your cloud project. In the meantime, if it’s not too much trouble, please try reproducing the issue using this SDK branch and share the log output here.

I (15719) livekit_signaling: Connecting to server: wss://secret.cloud/rtc?sdk=esp32&version=0.3.2&os=idf&os_version=v5.4-dirty&device_model=9&auto_subscribe=false&protocol=1
I (15732) websocket_client: Started
I (15736) livekit_example: Room state changed: Connecting
I (15981) esp-x509-crt-bundle: Certificate validated
I (16748) livekit_signaling: [!] Join response
I (16748) livekit_signaling: [!]   Other participant: PA_ujUqTQZwmgh8
I (16749) livekit_signaling: [!]     Track: sid=TR_AMow2596gdfhwe, type=0
I (16757) livekit_example: Agent has joined the room
I (16758) livekit_signaling: [!] Participant update
I (16762) DTLS: Init SRTP OK
I (16768) livekit_signaling: [!] Participant: PA_6eRHdECfjP7r
I (17112) AGENT: Start agent as Controlling
I (17113) PEER_DEF: Start DTLS role as 1
I (17114) livekit_engine: [!] Participant update in connecting state
I (17123) AGENT: Send STUN binding request
I (17124) AGENT: Send allocate now
I (17176) AGENT: Send allocate now
I (17224) AGENT: 0 Get candidate success user:ydDUqZ2QFRGSSVmNfKIo4bposBqoi7GZGis1yJJoCC psw:fFjRetKKwIaXl06W2NJ9GwmNy7swoY9czgr14lTcXi3
W (17225) AGENT: Skip TCP stun server turns:osingapore1b.turn.livekit.cloud:443?transport=tcp
W (17233) AGENT: Skip TCP stun server turns:fixie-test-73wt3hzv.turn.livekit.cloud:443?transport=tcp
I (17243) livekit_peer.pub: Generated offer:
v=0
o=- 7801614086411001801 2 IN IP4 0.0.0.0
s=-
t=0 0
a=msid-semantic: esp-webrtc
a=group:BUNDLE 0 1
m=audio 9 UDP/TLS/RTP/SAVP 111
a=rtpmap:111 opus/48000/2
a=ssrc:6 cname:webrtc-opus
a=ssrc:6 msid:esp_stream a0
a=sendrecv
a=mid:0
a=msid:esp_stream a0
c=IN IP4 0.0.0.0
a=rtcp-mux
a=fingerprint:sha-256 B8:1D:F7:82:70:D5:D4:E5:EC:84:C8:D7:48:B2:88:88:E9:3A:A7:AE:8E:AF:EA:D1:3A:0E:12:04:25:E6:CF:E0
a=setup:passive
a=ice-ufrag:WwUl
a=ice-pwd:Pd9ZwfQgpeN3WiStdYMx5Ru0
a=candidate:0 1 UDP 2129503487 192.168.0.101 60836 typ host
a=candidate:1 1 UDP 1693295871 218.103.102.60 60836 typ srflx raddr 0.0.0.0 rport 0
a=candidate:2 1 UDP 6117119 143.223.95.128 23894 typ relay raddr 0.0.0.0 rport 0
m=application 50712 UDP/DTLS/SCTP webrtc-datachannel
a=mid:1
a=sctp-port:5000
c=IN IP4 0.0.0.0
a=max-message-size:262144
a=fingerprint:sha-256 B8:1D:F7:82:70:D5:D4:E5:EC:84:C8:D7:48:B2:88:88:E9:3A:A7:AE:8E:AF:EA:D1:3A:0E:12:04:25:E6:CF:E0
a=setup:passive
a=ice-ufrag:WwUl
a=ice-pwd:Pd9ZwfQgpeN3WiStdYMx5Ru0
a=candidate:0 1 UDP 2129503487 192.168.0.101 60836 typ host
a=candidate:1 1 UDP 1693295871 218.103.102.60 60836 typ srflx raddr 0.0.0.0 rport 0
a=candidate:2 1 UDP 6117119 143.223.95.128 23894 typ relay raddr 0.0.0.0 rport 0

I (17492) PEER_DEF: Get peer role 0
I (17492) PEER_DEF: Get peer role 0
SDP not contain candidate continue
I (17521) livekit_peer.pub: State changed: 0 -> 1
I (17522) AGENT: 0 Add remote type:1 143.223.95.128:50012
0 Sorted pair 0 type: 1 local:192.168.0.101:60836 Remote:143.223.95.128:50012
0 Sorted pair 0 type: 1 local:192.168.0.101:60836 Remote:143.223.95.128:50012
I (17538) AGENT: 0 Send binding request (cand:0) local:192.168.0.101:60836 remote:143.223.95.128:50012 id:329b54fb381fc0385378d5a8
I (17607) AGENT: 0 Select pair143.223.95.128:50012
I (17607) AGENT: 0 Send binding request (cand:1) local:192.168.0.101:60836 remote:143.223.95.128:50012 id:76e5004e797d87a62756c714
I (17613) AGENT: 0 PeerBinding recv local:192.168.0.101:60836 remote:143.223.95.128:50012
I (17620) AGENT: 0 Send binding response local:192.168.0.101:60836 remote:143.223.95.128:50012
0 Sorted pair 0 type: 1 local:192.168.0.101:60836 Remote:143.223.95.128:50012
I (17664) AGENT: 0 Candidate responsed
I (17664) AGENT: 0 Connection OK 143.223.95.128:50012
I (17674) DTLS: Start to do server handshake
I (17991) DTLS: SRTP connected OK
I (17993) DTLS: Server handshake success
I (17993) PEER_DEF: DTLS handshake success
I (17993) PEER_DEF: Sctp role as 0
I (17996) SCTP: Send DCEP _reliable event:3 type:0 si:1
I (18001) SCTP: Send DCEP _lossy event:3 type:81 si:3
I (18055) livekit_signaling: [!] Participant update
I (18055) livekit_signaling: [!] Participant: PA_6eRHdECfjP7r
I (18056) livekit_signaling: [!]   Track: sid=TR_AMhpZ2p68RbkVJ, type=0
I (18057) SCTP: 0 Receive chunk 1 SCTP_INIT
I (18064) livekit_engine: [!] Participant update in connecting state
I (18067) SCTP: 0 state 2
I (18068) livekit_signaling: [!] Participant update
I (18077) SCTP: Support ext 82
I (18082) livekit_signaling: [!] Participant: PA_6eRHdECfjP7r
I (18085) SCTP: Support ext c0
I (18092) livekit_signaling: [!]   Track: sid=TR_AMhpZ2p68RbkVJ, type=0
I (18094) SCTP: Send INIT_ACK chunk
I (18102) livekit_engine: [!] Participant update in connecting state
I (18156) SCTP: 0 Receive chunk 10 SCTP_COOKIE_ECHO
I (18156) SCTP: Send ECHO_ACK chunk
I (18156) SCTP: 0 state 5
I (18207) livekit_peer.pub: Channel open: label=_reliable, stream_id=1
I (18208) livekit_peer.pub: Channel open: label=_lossy, stream_id=3
I (18210) livekit_peer.pub: State changed: 1 -> 2
I (18216) AUD_PIPE_NEGO: Negotiate return 0 src_format:541934416 sample_rate:16000

I (18224) AUD_PIPE_NEGO: Path mask 1 select sink:0 format 1398100047
I (18231) AUD_SRC: Get rate:16000, ch:1, bits:16
W (18236) ESP_GMF_PIPELINE: There is no thread for add jobs, pipe:0x3c399f28, tsk:0x0, [el:aud_src-0x3c411cbc]
I (18247) ESP_GMF_TASK: Waiting to run... [tsk:aenc_0-0x3fce1568, wk:0x0, run:0]
I (18255) livekit_example: Room state changed: Connected
I (19483) RINGMUSIC: [music] codec music played, 351087 bytes are written.
E (19689) i2s_common: i2s_channel_disable(1187): the channel has not been enabled yet
I (19689) I2S_IF: channel mode 0 bits:16/16 channel:2 mask:3
I (19693) I2S_IF: STD Mode 1 bits:16/16 channel:2 sample_rate:16000 mask:3
I (19715) Adev_Codec: Open codec device OK
I (20055) I2S_IF: Mode 1 need extend bits 32 to 64
I (20055) I2S_IF: channel mode 2 bits:16/16 channel:4 mask:3
I (20056) I2S_IF: TDM Mode 0 bits:16/16 channel:4 sample_rate:16000 mask:3
I (20064) I2S_IF: channel mode 0 bits:16/32 channel:2 mask:3
I (20070) I2S_IF: STD Mode 1 bits:16/32 channel:2 sample_rate:16000 mask:3
I (20086) Adev_Codec: Open codec device OK
E (20086) MODEL_LOADER: Can not find model in partition table
W (20089) AUD_AEC_SRC: No model to load
W (20093) AFE_CONFIG: wakenet model not found. please load wakenet model...
I (20142) AFE: AFE Version: (1MIC_V250121)
I (20142) AFE: Input PCM Config: total 2 channels(1 microphone, 1 playback), sample rate:16000
I (20143) AFE: AFE Pipeline: [input] -> |AEC(SR_LOW_COST)| -> |VAD(WebRTC)| -> [output]
I (20152) AUD_SRC: Start to fetch audio src data now
I (20152) ESP_GMF_TASK: One times job is complete, del[wk:0x3c3a25e4, ctx:0x3c411cbc, label:aud_src_open]
I (20167) ESP_GMF_PORT: ACQ IN, new self payload:0x3c3a25e4, port:0x3c399f6c, el:0x3c411cbc-aud_src
I (20177) ESP_GMF_TASK: One times job is complete, del[wk:0x3c4146f8, ctx:0x3c411a30, label:aud_ch_cvt_open]
I (20188) ESP_GMF_AENC: Open, type:OPUS, acquire in frame: 1280, out frame: 100
I (20195) ESP_GMF_TASK: One times job is complete, del[wk:0x3c414740, ctx:0x3c411adc, label:aud_enc_open]

Hello, it seems it did have a remote participant, but the track subscription is not surely triggered every time(handle_participant_update)
And I added a subscribe track here:(in the engine.c file)

static bool handle_join(engine_t *eng, const livekit_pb_join_response_t *join)
{
    // 1. Store connection settings
    eng->session.is_subscriber_primary = join->subscriber_primary;

    // 2. Store local Participant SID
    strlcpy(
        eng->session.local_participant_sid,
        join->participant.sid,
        sizeof(eng->session.local_participant_sid)
    );

    // 3. Dispatch initial room info
    if (eng->options.on_room_info && join->has_room) {
        eng->options.on_room_info(&join->room, eng->options.ctx);
    }

    // 4. Dispatch initial participant info
    if (eng->options.on_participant_info) {
        eng->options.on_participant_info(&join->participant, true, eng->options.ctx);
        for (pb_size_t i = 0; i < join->other_participants_count; i++) {
            eng->options.on_participant_info(&join->other_participants[i], false, eng->options.ctx);
        }
    }

    // 5. Establish peer connections
    if (!establish_peer_connections(eng, join)) {
        ESP_LOGE(TAG, "Failed to establish peer connections");
        return false;
    }

    //6. NEW
    for (pb_size_t i = 0; i < join->other_participants_count; i++) {
        const livekit_pb_participant_info_t *p = &join->other_participants[i];
        if (p->tracks_count > 0) {
            ESP_LOGI(TAG, "Subscribing (from JOIN) to participant: %s, tracks=%d", p->sid, p->tracks_count);
            subscribe_tracks(eng, (livekit_pb_track_info_t *)p->tracks, p->tracks_count);
            // Only subscribe to the first available audio track overall.
            if (eng->session.sub_audio_track_sid[0] != '\0') {
                break;
            }
        }
    }
    return true;
}

I (19093) livekit_signaling: Connecting to server: wss://secrect.cloud/rtc?sdk=esp32&version=0.3.2&os=idf&os_version=v5.4-dirty&device_model=9&auto_subscribe=false&protocol=1
I (19106) websocket_client: Started
I (19110) livekit_example: Room state changed: Connecting
I (19337) esp-x509-crt-bundle: Certificate validated
I (21147) livekit_signaling: [!] Join response
I (21147) livekit_signaling: [!]   Other participant: PA_5u9hHarqEDAV
I (21148) livekit_signaling: [!]     Track: sid=TR_AM3fvozMq888F7, type=0
I (21155) livekit_example: Agent has joined the room
I (21156) livekit_signaling: [!] Participant update
I (21161) DTLS: Init SRTP OK
I (21167) livekit_signaling: [!] Participant: PA_BQMuLQbSCPGn
I (21501) AGENT: Start agent as Controlling
I (21502) PEER_DEF: Start DTLS role as 1
I (21503) livekit_engine: Subscribing (from JOIN) to participant: PA_5u9hHarqEDAV, tracks=1
I (21507) livekit_engine: Subscribing to audio track: sid=TR_AM3fvozMq888F7
I (21509) AGENT: Send STUN binding request
I (21515) livekit_signaling: [!] Sending update subscription: sid=TR_AM3fvozMq888F7, subscribe=1
I (21519) AGENT: Send allocate now
I (21531) livekit_engine: [!] Participant update in connecting state
I (21579) AGENT: Send allocate now
I (21620) AGENT: 0 Get candidate success user:udEUDNlYRxUdNFlQfKIo4bposBqoi7GZGis1yJJoCC psw:JfYS091eM2gfFNT8CNfJI4foBePrtuw5VQ5Rx8xoni8G
W (21621) AGENT: Skip TCP stun server turns:osingapore1b.turn.livekit.cloud:443?transport=tcp
W (21630) AGENT: Skip TCP stun server turns:fixie-test-73wt3hzv.turn.livekit.cloud:443?transport=tcp
I (21639) livekit_peer.pub: Generated offer:
v=0
o=- 7801614086411001801 2 IN IP4 0.0.0.0
s=-
t=0 0
a=msid-semantic: esp-webrtc
a=group:BUNDLE 0 1
m=audio 9 UDP/TLS/RTP/SAVP 111
a=rtpmap:111 opus/48000/2
a=ssrc:6 cname:webrtc-opus
a=ssrc:6 msid:esp_stream a0
a=sendrecv
a=mid:0
a=msid:esp_stream a0
c=IN IP4 0.0.0.0
a=rtcp-mux
a=fingerprint:sha-256 92:F5:8A:68:B8:79:29:99:D3:DA:3A:55:34:F0:67:33:4B:7F:89:D6:88:6E:90:28:F4:2A:A5:EC:85:6B:4A:DB
a=setup:passive
a=ice-ufrag:WwUl
a=ice-pwd:Pd9ZwfQgpeN3WiStdYMx5Ru0
a=candidate:0 1 UDP 2130313471 192.168.0.101 64000 typ host
a=candidate:1 1 UDP 1694105855 218.103.102.60 64000 typ srflx raddr 0.0.0.0 rport 0
a=candidate:2 1 UDP 6192895 143.223.95.170 24190 typ relay raddr 0.0.0.0 rport 0
m=application 50712 UDP/DTLS/SCTP webrtc-datachannel
a=mid:1
a=sctp-port:5000
c=IN IP4 0.0.0.0
a=max-message-size:262144
a=fingerprint:sha-256 92:F5:8A:68:B8:79:29:99:D3:DA:3A:55:34:F0:67:33:4B:7F:89:D6:88:6E:90:28:F4:2A:A5:EC:85:6B:4A:DB
a=setup:passive
a=ice-ufrag:WwUl
a=ice-pwd:Pd9ZwfQgpeN3WiStdYMx5Ru0
a=candidate:0 1 UDP 2130313471 192.168.0.101 64000 typ host
a=candidate:1 1 UDP 1694105855 218.103.102.60 64000 typ srflx raddr 0.0.0.0 rport 0
a=candidate:2 1 UDP 6192895 143.223.95.170 24190 typ relay raddr 0.0.0.0 rport 0

I (21671) PEER_DEF: Get peer role 0
I (21759) PEER_DEF: A mid: 0
I (21762) PEER_DEF: A SRC: 1141301876
I (21765) PEER_DEF: A SRC: 1141301876
I (21768) PEER_DEF: A SRC: 1141301876
I (21772) PEER_DEF: A SRC: 1141301876
SDP not contain candidate continue
I (21778) AGENT: 1 Add remote type:1 143.223.95.170:50005
I (21784) AGENT: Start agent as Controlled
I (21787) PEER_DEF: Start DTLS role as 1
I (21791) AGENT: Send STUN binding request
I (21795) AGENT: Send allocate now
I (21847) AGENT: Send allocate now
I (21892) AGENT: 1 Get candidate success user:udEUDNlYRxUdNFlQfKIo4bposBqoi7GZGis1yJJoCC psw:JfYS091eM2gfFNT8CNfJI4foBePrtuw5VQ5Rx8xoni8G
W (21893) AGENT: Skip TCP stun server turns:osingapore1b.turn.livekit.cloud:443?transport=tcp
W (21901) AGENT: Skip TCP stun server turns:fixie-test-73wt3hzv.turn.livekit.cloud:443?transport=tcp
1 Sorted pair 0 type: 1 local:192.168.0.101:64001 Remote:143.223.95.170:50005
I (21918) PEER_DEF: Get peer role 0
I (21918) livekit_peer.sub: Generated answer:
v=0
o=- 7801614086411001801 2 IN IP4 0.0.0.0
s=-
t=0 0
a=msid-semantic: esp-webrtc
a=group:BUNDLE 0
m=audio 9 UDP/TLS/RTP/SAVP 111
a=rtpmap:111 opus/48000/2
a=ssrc:6 cname:webrtc-opus
a=ssrc:6 msid:esp_stream a0
a=sendrecv
a=mid:0
a=msid:esp_stream a0
c=IN IP4 0.0.0.0
a=rtcp-mux
a=fingerprint:sha-256 92:F5:8A:68:B8:79:29:99:D3:DA:3A:55:34:F0:67:33:4B:7F:89:D6:88:6E:90:28:F4:2A:A5:EC:85:6B:4A:DB
a=setup:passive
a=ice-ufrag:WwUl
a=ice-pwd:Pd9ZwfQgpeN3WiStdYMx5Ru0
a=candidate:0 1 UDP 2130313727 192.168.0.101 64001 typ host
a=candidate:1 1 UDP 1694106111 218.103.102.60 64001 typ srflx raddr 0.0.0.0 rport 0
a=candidate:2 1 UDP 5184767 143.223.95.170 20252 typ relay raddr 0.0.0.0 rport 0

I (21920) PEER_DEF: Get peer role 0
I (21989) livekit_peer.sub: State changed: 0 -> 1
I (22007) AGENT: 1 Send binding request (cand:0) local:192.168.0.101:64001 remote:143.223.95.170:50005 id:329b54fb381fc0385378d5a8
SDP not contain candidate continue
I (22080) AGENT: 0 Add remote type:1 143.223.95.170:50006
0 Sorted pair 0 type: 1 local:192.168.0.101:64000 Remote:143.223.95.170:50006
I (22084) livekit_peer.pub: State changed: 0 -> 1
0 Sorted pair 0 type: 1 local:192.168.0.101:64000 Remote:143.223.95.170:50006
I (22097) AGENT: 0 Send binding request (cand:0) local:192.168.0.101:64000 remote:143.223.95.170:50006 id:329b54fb381fc0385378d5a8
I (22184) AGENT: 0 Select pair143.223.95.170:50006
I (22184) AGENT: 0 Send binding request (cand:1) local:192.168.0.101:64000 remote:143.223.95.170:50006 id:76e5004e797d87a62756c714
I (22189) AGENT: 0 PeerBinding recv local:192.168.0.101:64000 remote:143.223.95.170:50006
I (22197) AGENT: 0 Send binding response local:192.168.0.101:64000 remote:143.223.95.170:50006
0 Sorted pair 0 type: 1 local:192.168.0.101:64000 Remote:143.223.95.170:50006
0 Sorted pair 0 type: 1 local:192.168.0.101:64000 Remote:143.223.95.170:50006
I (22219) livekit_signaling: [!] Sending answer: v=0
o=- 7801614086411001801 2 IN IP4 0.0.0.0
s=-
t=0 0
a=msid-semantic: esp-webrtc
a=group:BUNDLE 0
m=audio 9 UDP/TLS/RTP/SAVP 111
a=rtpmap:111 opus/48000/2
a=ssrc:6 cname:webrtc-opus
a=ssrc:6 msid:esp_stream a0
a=sendrecv
a=mid:0
a=msid:esp_stream a0
c=IN IP4 0.0.0.0
a=rtcp-mux
a=fingerprint:sha-256 92:F5:8A:68:B8:79:29:99:D3:DA:3A:55:34:F0:67:33:4B:7F:89:D6:88:6E:90:28:F4:2A:A5:EC:85:6B:4A:DB
a=setup:passive
a=ice-ufrag:WwUl
a=ice-pwd:Pd9ZwfQgpeN3WiStdYMx5Ru0
a=candidate:0 1 UDP 2130313727 192.168.0.101 64001 typ host
a=candidate:1 1 UDP 1694106111 218.103.102.60 64001 typ srflx raddr 0.0.0.0 rport 0
a=candidate:2 1 UDP 5184767 143.223.95.170 20252 typ relay raddr 0.0.0.0 rport 0

I (22233) AGENT: 0 Candidate responsed
I (22291) AGENT: 0 Connection OK 143.223.95.170:50006
I (22306) DTLS: Start to do server handshake
I (22425) AGENT: 1 PeerBinding recv local:192.168.0.101:64001 remote:143.223.95.170:50005
I (22426) AGENT: 1 Send binding response local:192.168.0.101:64001 remote:143.223.95.170:50005
I (22599) DTLS: SRTP connected OK
I (22601) DTLS: Server handshake success
I (22602) PEER_DEF: DTLS handshake success
I (22602) PEER_DEF: Sctp role as 0
I (22605) SCTP: Send DCEP _reliable event:3 type:0 si:1
I (22609) SCTP: Send DCEP _lossy event:3 type:81 si:3
I (22643) livekit_signaling: [!] Participant update
I (22643) livekit_signaling: [!] Participant: PA_BQMuLQbSCPGn
I (22644) livekit_signaling: [!]   Track: sid=TR_AMn6WGGPW6pKGu, type=0
I (22648) SCTP: 0 Receive chunk 1 SCTP_INIT
I (22651) livekit_engine: [!] Participant update in connecting state
I (22655) SCTP: 0 state 2
I (22652) livekit_signaling: [!] Participant update
I (22665) SCTP: Support ext 82
I (22670) livekit_signaling: [!] Participant: PA_BQMuLQbSCPGn
I (22673) SCTP: Support ext c0
I (22679) livekit_signaling: [!]   Track: sid=TR_AMn6WGGPW6pKGu, type=0
I (22682) SCTP: Send INIT_ACK chunk
I (22689) livekit_engine: [!] Participant update in connecting state
I (22733) SCTP: 0 Receive chunk 10 SCTP_COOKIE_ECHO
I (22734) SCTP: Send ECHO_ACK chunk
I (22734) SCTP: 0 state 5
I (22773) livekit_peer.pub: Channel open: label=_reliable, stream_id=1
I (22774) livekit_peer.pub: Channel open: label=_lossy, stream_id=3
I (22776) livekit_peer.pub: State changed: 1 -> 2
I (22782) AUD_PIPE_NEGO: Negotiate return 0 src_format:541934416 sample_rate:16000

I (22790) AUD_PIPE_NEGO: Path mask 1 select sink:0 format 1398100047
I (22797) AUD_SRC: Get rate:16000, ch:1, bits:16
W (22802) ESP_GMF_PIPELINE: There is no thread for add jobs, pipe:0x3c4144ec, tsk:0x0, [el:aud_src-0x3c3a26e8]
I (22814) ESP_GMF_TASK: Waiting to run... [tsk:aenc_0-0x3fce1a8c, wk:0x0, run:0]
I (22821) livekit_example: Room state changed: Connected
I (22879) RINGMUSIC: [music] codec music played, 351087 bytes are written.
I (23020) AGENT: 1 PeerBinding recv local:192.168.0.101:64001 remote:143.223.95.170:50005
I (23021) AGENT: 1 Select pair143.223.95.170:50005
I (23022) AGENT: 1 Connection OK 143.223.95.170:50005
I (23036) DTLS: Start to do server handshake
E (23084) i2s_common: i2s_channel_disable(1187): the channel has not been enabled yet
I (23084) I2S_IF: channel mode 0 bits:16/16 channel:2 mask:3
I (23088) I2S_IF: STD Mode 1 bits:16/16 channel:2 sample_rate:16000 mask:3
I (23110) Adev_Codec: Open codec device OK
I (23359) DTLS: SRTP connected OK
I (23361) DTLS: Server handshake success
I (23361) PEER_DEF: DTLS handshake success
I (23361) PEER_DEF: Sctp role as 0
I (23421) I2S_IF: Mode 1 need extend bits 32 to 64
I (23421) I2S_IF: channel mode 2 bits:16/16 channel:4 mask:3
I (23422) I2S_IF: TDM Mode 0 bits:16/16 channel:4 sample_rate:16000 mask:3
I (23429) I2S_IF: channel mode 0 bits:16/32 channel:2 mask:3
I (23436) I2S_IF: STD Mode 1 bits:16/32 channel:2 sample_rate:16000 mask:3
I (23452) Adev_Codec: Open codec device OK
E (23452) MODEL_LOADER: Can not find model in partition table
W (23455) AUD_AEC_SRC: No model to load
W (23459) AFE_CONFIG: wakenet model not found. please load wakenet model...
I (23508) AFE: AFE Version: (1MIC_V250121)
I (23508) AFE: Input PCM Config: total 2 channels(1 microphone, 1 playback), sample rate:16000
I (23509) AFE: AFE Pipeline: [input] -> |AEC(SR_LOW_COST)| -> |VAD(WebRTC)| -> [output]
I (23518) AUD_SRC: Start to fetch audio src data now
I (23518) ESP_GMF_TASK: One times job is complete, del[wk:0x3c39b56c, ctx:0x3c3a26e8, label:aud_src_open]
I (23533) ESP_GMF_PORT: ACQ IN, new self payload:0x3c39b56c, port:0x3c412294, el:0x3c3a26e8-aud_src
I (23543) ESP_GMF_TASK: One times job is complete, del[wk:0x3c41270c, ctx:0x3c3a27c4, label:aud_ch_cvt_open]
I (23554) ESP_GMF_AENC: Open, type:OPUS, acquire in frame: 1280, out frame: 100
I (23561) ESP_GMF_TASK: One times job is complete, del[wk:0x3c41dbe8, ctx:0x3c3a2870, label:aud_enc_open]
Get samplerate 16000 chanel 2
I (23646) AV_RENDER: Get need resample 1 in:16000 out:16000
I (23647) I2S_RENDER: open channel:2 sample_rate:16000 bits:16
I (23651) Adev_Codec: Input already open
I (23656) AV_RENDER: ARender Auto pause audio first frame at 40614107

Do you have any suggestion?

[ladvoc]

Hi @williamleeag6688, thank you for checking that. This is a bug in the SDK; as you suggested above, what is happening here is tracks that have already been published by remote participants prior to joining the room never get subscribed to. Under normal network conditions, you likely don't run into this since the client joins before the remote track has been published. I have implemented a fix in #52, which is very similar to what you show above. Please let me know if this addresses your issue, and if so, I will get a release out ASAP.

[williamleeag6688]

Hi @williamleeag6688, thank you for checking that. This is a bug in the SDK; as you suggested above, what is happening here is tracks that have already been published by remote participants prior to joining the room never get subscribed to. Under normal network conditions, you likely don't run into this since the client joins before the remote track has been published. I have implemented a fix in #52, which is very similar to what you show above. Please let me know if this addresses your issue, and if so, I will get a release out ASAP.

Hi ladvoc, thanks for your update, we will test more first then update you.