very temp 429 fix

[Roki100]

very simple and temp fix for latest update that causes big boi 429, code can be edited
im also not sure if i did it right(implemented)/if it is working but i dont get 429 after adding this for now

[fent]

can others comment on this? does this solve the 429 status error for you?

[userlip]

Did not fix it for me

[RocoGreen]

Didn't fix for me too

[Roki100]

for me it fixed getting 429 from getting audio data
@RocoGreen @userlip did you both got ratelimited next time when using these headers?

[omarroth]

Currently testing this as a fix for iv-org/invidious#811 (iv-org/invidious@dad885c). Additionally added:

User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/77.0.3865.90 Safari/537.36

Will report back with results.

[RocoGreen]

@Roki100 nope, i didnt get rate limited

[Roki100]

@RocoGreen then it works

[userlip]

I just tried executing the BasicInfo function on a server (that is 429 blocked). With all 3 Options (normal ytdl-core, using this pull request, using this pull request + @omarroth user agent)

In all 3 cases no output was generated and no error was logged (which I find strange) however the same script gives me a perfectly good output on my local machine and a new server, so I dont think its the script.

If anyone wants me to test something on that server that is blocked, let me know

Edit: the script: https://hastebin.com/qezanixiwa.js
Again: absolutely nothing is being printed to the terminal so I do not know if the 429 response code is actually the issue here!

[Roki100]

@userlip this isnt 429 removal pr(you cant remove existing 429), it should prevent getting more 429 in the future

[omarroth]

After testing the fix for a couple days I have not encountered the issue again. I've been testing on about 15 droplets from DigitalOcean with each droplet handling about 25 req/s.

[WaqasIbrahim]

After testing the fix for a couple days I have not encountered the issue again. I've been testing on about 15 droplets from DigitalOcean with each droplet handling about 25 req/s.

@omarroth Were those droplets rate limited in the past? Are they fresh?

[omarroth]

Looks like I spoke a bit too soon. Quite literally 10 minutes after posting all nodes have been banned.

@WaqasIbrahim They are fresh.

[Roki100]

@omarroth are you using other features than downloading like info etc.?

[omarroth]

Yes, in addition to video metadata there are also requests for comments, search, channels, playlists, etc. All are made currently with the above headers.

[Roki100]

this would need use of &pbj=1 which would require rewrite of big part of ytdl
so the reason of your ratelimit is getting infos etc.

[h-exx]

fixed it for a second and then it broke again, you're getting close

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.