Connection reset by peer (os error 104) #1340
Comments
|
What version is this? I don't see any crashes in that log. |
|
My bad, left the version out: 0.4.2. Well, it might not be an actual librespot crash, but the music stops. Spotify goes from playing -> idle. I thought it was a crash because every time I start librespot in my container, it starts off with the apresolve warnings, then it says how it authenticated as my user, etc. - Meaning something broke, because it started the whole apresolve and authentication loop again |
|
I seem to observe the same in the past week. I am not 100% sure but it almost seems like Spotify has become much more stringent with keeping the phone/pc/tablet client connected. Ever since I tweaked my wifi network to allow for lower latency roaming it has not dropped out at all (the librespot sever is connected via Ethernet btw.). I never had these issues before though. |
|
Same here, over ethernet, although I did have this issue before, where it would just stop playing. I didn't exactly inspect the logs before, but it happened at least 10x less frequently, once every couple of days, as far as I remember. |
FWIW I've been observing the same with the current Probably a duplicate of #1151, which for some people seems to be getting more severe now. |
|
The effect of tweaking my network was short lived. I have already experienced quite a few ‘subscription terminated’ errors today. |
|
Seeing the same Snapcast v0.28.0 |
|
oh no :c Just today I tried disabling the apresolve bypass (0.0.0.0 in etc/hosts), thinking it might fix the issue, but judging from your comment I think we're back at square one. EDIT: To confirm, this issue is unrelated to the apresolve bypass, I get the same error whether enabling the apresolve bypass, or not. |
|
i am affected too, with 0.4.2 and dev versions. could it be caused by the login-method change (password not working)? |
|
It seems to be getting worse for me.. this evening it is giving me the error almost every 30min… is there any way to debug this? |
|
I've also noticing frequent disconnects at the moment, almost every other minute. I'm using current dev branch (118a9e1, which has new reconnect handling, but was also happening before): |
|
Someone who is experiencing this is going to have to try and debug it. Maybe compare the look of our traffic with the official client in wireshark? Is there some obvious thing there? Otherwise go the full hog with spotify-analyze and a mitm proxy setup? |
When you say that, do you really mean it's always every two minutes? That's a big hint.... |
I've performed a tcpdump (of spotifyd) and observed the following after ~30min of playback:
I guess this might be the two minute interval you're looking for? Seems like a keepalive sequence to me but then again I don't know the Spotify protocol at all |
|
That apparently corresponds to my log: |
|
Their protocol has a Ping-Pong keep alive system. They also send a Pong-ACK too. But, do I understand correctly, they RST the connection despite the 4th Pong being returned? We could try some builds with all of this logged. we could try disabling our pong entirely to see if the same thing happens sooner. I wonder why only some people see this. Have you tried forcing the use of an access point using port 443 or 80? |
|
"session lost connection to server" is slightly different. It's the opposite to "Connection reset by peer". It's librespot saying it didn't get a Ping from Spotify within the last 125 seconds. |
For me, these disconnects happen with or without the apresolve bypass (forcing 80/443) |
To be clear, I'm not suggesting using the fallback access point. I'm suggesting trying one of the normal access points on either port 443 or 80, rather than 4070. And only because firewalls like traffic on HTTP ports. |
I've just had a quick look at Spotify's Windows client in Wireshark. It doesn't ping-pong exactly like we do (did it change or did we always do this wrong?) Maybe this matters, maybe not - I'm still not having any issues with how we do it. Feel free to see what your Spotify client is doing, especially if you're experiencing issues. Ideally do this on Linux/mac using spotify-analyze to get a better idea. |
FYI I couldn't get it to work with latest version of wireshark but maybe someone else will have more luck. |
Here's a patch for Wireshark 4.2: |
We probably Pong right away, no? The above could be explained as follows:
|
Yes, we do. Yes, it could, I thought something along those lines too but it seems elaborate. My only real thought about this was maybe there was some value we should be sending back in the Pong. (We send zero). Maybe sometimes it's expecting more? I am guessing. I'm hoping someone with the issue will take a look at theirs. I still couldn't get spotify-analyze to work properly to check the value. Value out of range errors now. It used to just work... |
|
What I'm seeing is (Spotify Desktop on Linux) i.e. the sequence is always the same, even after start-up. I could open a PR with a quick & dirty fix, but I can't easily test it myself: The disconnects don't happen so frequently for me right now, so it's hard to say whether there's any difference (it might run for half an hour without connection resets). EDIT: The |
That other project I'm working on actually does something as elaborate as that.
Does the Ping message have some payload, like a unique ID that we should echo with the Pong? |
I vote for ope a PR and let someone try it.
It contains the server timestamp, which we do some strange arithmetic on (it looks backwards to me) and I think we send back in some other spirc response later. It's possible it's this value coming back that Spotify doesn't like. |
Have you applied the entire patch above? That works for me (if it doesn't, I'll check whether I pasted some half-baked version).
👍 |
|
Yes I did apply the whole thing. It complains the dump is corrupt but I've tried it a few times. |
|
Oh, that's why I also updated |
|
Excellent, I missed that. Do you want to PR those spotify-analyze fixes too? |
|
@rwjack @artenverho @herrernst @ius @tla44 could you test whether #1357 fixes the issue for you? |
I was able to keep librespot running for 7h+ with that patch applied without a single reconnect. I'll follow up with some A/B testing to be sure (unless enough others confirm it works for them). |
|
I would gladly try it out, but I cannot seem to build the damn thing: #1346 (comment) |
Works flawlessly for me now since some hours (wasn't possible before), thank you very much! |
Got this built and running for my LibreElec/kodi setup on a rpi4 and it has been working flawlessly for several hours today. Thank you for the fix @wisp3rwind <3 Couldn't get the LibreElec buildchain to build it successfully, however. It worked fine with commit Was able to get it built w |
Thanks in particular also to @kingosticks for the initial diagnosis! |
|
I can confirm that the changes from #1357 works now for hours without any issue. I have seen several interruptions of Podcasts/Songs using librespot v0.4.2 but no interruption so far using this PR. Thanks. |
|
I was getting quite frequent connection resets. After applying the proposed change in #1359 none have happened this afternoon with 5 hours of straight playback. Thanks for fixing the issue! prior to the change, this is the frequency at which they'd been happening: |
|
This fixed it for me as well! Thanks! Perhaps a rookie question: my setup originally used Spotifyd (mainly for the dbus feature). Compiling Spotifyd requires librespot version 0.4.2. Is it possible to apply the fix on this version too and use it to recompile spotifyd? |
Compiled #1359 and been running without any problems for about an hour now. 0.4.2 was "connection reset"-ing for the past couple of weeks every 5 minutes or so. |
|
Another data point in case it helps: I've been running a build with #1359 for most of today, playing background music for many hours straight. I no longer saw "Spirc shut down unexpectedly" at all during this time. Previously I ran into this problem at least once every 5 to 30 minutes. So the PR appears to have resolved the issue for me. |
|
Thanks for this PR. Tested with #1359 I no longer have But, now, even when I don't listen music, I get a I'm running 4 instances of librespot with snapcast, on a VM machine (Proxmox VE 8.1) Here is a log when error happens in the night, when nobody is listening music: |
|
I think we'd need a debug log for that. And not here, a separate issue. |
|
OK, but do you know how I can access librespot log when launched from snapcast ? (debian OS) |
Snapcast has a Failing that, launch it standalone for this debugging purpose. |
|
anyone have a working binary for Alpine they can share? been failing miserably at compiling this myself and am facing the above issue. |
Describe the bug
Librespot keeps crashing on random intervals. I implemented the
0.0.0.0 apresolve.spotify.combypass in /etc/hosts, but now this issue just replaced the other one: #1322To reproduce
librespotwithLog
Host (what you are running
librespoton):The text was updated successfully, but these errors were encountered: