Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update v0.107.30 causes connection and speed drops. #5874

Open
3 tasks done
Ice2192 opened this issue Jun 8, 2023 · 104 comments
Open
3 tasks done

Update v0.107.30 causes connection and speed drops. #5874

Ice2192 opened this issue Jun 8, 2023 · 104 comments
Assignees
Labels
bug P2: High waiting for data Waiting for users to provide more data.
Milestone

Comments

@Ice2192
Copy link

Ice2192 commented Jun 8, 2023

Prerequisites

  • I have checked the Wiki and Discussions and found no answer

  • I have searched other issues and found no duplicates

  • I want to report a bug and not ask a question

Operating system type

Windows

CPU architecture

64-bit ARM

Installation

GitHub releases or script from README

Setup

On a router, DHCP is handled by the router

AdGuard Home version

v0.107.30

Description

What did you do?

Go on the internet.

Expected result

View pages.

Actual result

Error: "This Site Can’t Be Reached"

Screenshots (if applicable)

Additional information

I just updated ASAP when I got the notification today and then the internet just drops. I thought it was the router but I changed the DNS on my PC and it was fine. I recently upgraded from a miro center sd card to a sandisk a few months ago as the microcenter one hangs when updating the blocklists for some reason. Had no problems since the san disk upgrade. Now with this update, the DNS just gets lost when I point anything to my pi running adguard home.

@dfsworld
Copy link

dfsworld commented Jun 8, 2023

Same here, it starts well and suddenly all devices cannot reach internet. I had to restore previous version backup (v0.107.28) and all was fine since then.

@werdlerk
Copy link

werdlerk commented Jun 8, 2023

Same here. Running in Docker container on Synology NAS.
I've pinned the Docker image version to v0.107.29 to get it working again.

I've noticed in the Query logs that the DNS queries of my computer were shown but the Elapsed time was very high. They were in the range of 4300 ms or 7500 ms. This was probably why the DNS requests were timing out and not working anymore. It seems only to be having issues with forwarding DNS requests to the upstream DNS servers as DNS rewrites were working fine.

@andreschoppe
Copy link

Same problem. My watchtower did an update last night, basically no internet connection since then. Reverted back to 0.107.29, problems gone.

@Insight16
Copy link

same problem
@ameshkov @ainar-g need help

@EugeneOne1
Copy link
Member

Hello @Ice2192, @werdlerk, @dfsworld, @andreschoppe, @Insight16! Thanks for reporting. Could you please collect a verbose log capturing the incoming requests? This would help us troubleshoot this issue. You may send it to devteam@adguard.com.

@ainar-g ainar-g added the needs investigation Needs to be reproduced reliably. label Jun 8, 2023
adguard pushed a commit that referenced this issue Jun 8, 2023
Updates #5874.

Squashed commit of the following:

commit 297839f
Author: Ainar Garipov <A.Garipov@AdGuard.COM>
Date:   Thu Jun 8 12:38:37 2023 +0300

    all: upd dnsproxy
@ainar-g
Copy link
Contributor

ainar-g commented Jun 8, 2023

This is likely caused by the same bug as #5873, so the fix for that should also improve the upstream situation generally.

adguard pushed a commit to AdguardTeam/dnsproxy that referenced this issue Jun 8, 2023
Merge in GO/dnsproxy from 5874-fix-fallback to master

Updates AdguardTeam/AdGuardHome#5874.

Squashed commit of the following:

commit 6687dac
Author: Eugene Burkov <E.Burkov@AdGuard.COM>
Date:   Thu Jun 8 16:10:06 2023 +0300

    upstream: close plain connections

commit 4162e21
Author: Eugene Burkov <E.Burkov@AdGuard.COM>
Date:   Thu Jun 8 15:32:49 2023 +0300

    upstream: imp fallback test

commit 2614653
Author: Eugene Burkov <E.Burkov@AdGuard.COM>
Date:   Thu Jun 8 15:16:40 2023 +0300

    upstream: imp logging, test

commit 7fa6a9d
Author: Eugene Burkov <E.Burkov@AdGuard.COM>
Date:   Thu Jun 8 14:53:51 2023 +0300

    upstream: fix fallback to tcp
@EugeneOne1 EugeneOne1 added bug P2: High and removed needs investigation Needs to be reproduced reliably. labels Jun 8, 2023
@EugeneOne1
Copy link
Member

@Ice2192, @werdlerk, @dfsworld, @andreschoppe, @Insight16, we've just pushed the edge build that should fix this. Could you please check if it now works as intended? If it does, we're going to prepare a proper hot-fix release.

@Ice2192
Copy link
Author

Ice2192 commented Jun 8, 2023

Same problem. My watchtower did an update last night, basically no internet connection since then. Reverted back to 0.107.29, problems gone.

How do you downgrade the version via SSH?

@andreschoppe
Copy link

@Ice2192, @werdlerk, @dfsworld, @andreschoppe, @Insight16, we've just pushed the edge build that should fix this. Could you please check if it now works as intended? If it does, we're going to prepare a proper hot-fix release.

Can confirm edge build works as intended. No more problems here. Thanks guys.

@andreschoppe
Copy link

Same problem. My watchtower did an update last night, basically no internet connection since then. Reverted back to 0.107.29, problems gone.

How do you downgrade the version via SSH?

You can always use any version with docker that is still available in the hub by using the tags.
E.g. docker run --name adguardhome --restart unless-stopped -v /pathtoyouroptdir:/opt/adguardhome/work -v /pathtoyourconfdir:/opt/adguardhome/conf -p 8080:80/tcp -p 3000:3000/tcp -p 53:53/tcp -p 53:53/udp -d adguard/adguardhome:v0.107.29
You can find all available tags on the docker hub page for adguard

@ainar-g ainar-g added this to the v0.107.31 milestone Jun 8, 2023
@ainar-g
Copy link
Contributor

ainar-g commented Jun 8, 2023

@Ice2192, the newly released v0.107.31 should contain a fix. Please update to it and tell us if the slowdown went away.

@Ice2192
Copy link
Author

Ice2192 commented Jun 8, 2023

@Ice2192, the newly released v0.107.31 should contain a fix. Please update to it and tell us if the slowdown went away.

Thanks! It's back to its normal speed!

@Ice2192 Ice2192 closed this as completed Jun 8, 2023
@werdlerk
Copy link

werdlerk commented Jun 8, 2023

I'm going to test it later but many thanks for the quick response and resolution! 👍

@Ice2192 Ice2192 reopened this Jun 8, 2023
@Ice2192
Copy link
Author

Ice2192 commented Jun 8, 2023

No dice. It doesn't work anymore. Worked for about an hour and it's back to slowdown again.

@Xerdunits
Copy link

No dice. It doesn't work anymore. Worked for about an hour and it's back to slowdown again.

Same for me, it worked at first, but I still get a boat load of errors in the interface and DNS timeouts

@degofedal
Copy link

No dice. It doesn't work anymore. Worked for about an hour and it's back to slowdown again.

Same for me, it worked at first, but I still get a boat load of errors in the interface and DNS timeouts

Same for me. Worked fine for the first couple of hour, the started slowing down and then failing entirely.

@andreschoppe
Copy link

Edge version still doing fine for me ...

@Mur1thi
Copy link

Mur1thi commented Jun 9, 2023

Edge version still has issues.

@EugeneOne1
Copy link
Member

@Ice2192, @Xerdunits, @degofedal, @andreschoppe, @Mur1thi, thanks for your concern. Could you please collect a verbose log again, so that we could revise the actual source of the current problem? You may, again, send it to devteam@adguard.com.

@Ice2192
Copy link
Author

Ice2192 commented Jun 18, 2023

But I had 90%-95% before the change. I'm not sure if Unbound is actually working in terms of helping blocking ads. It's fast as when I turned off the ad blocker but maybe because it isn't blocking ads at all. Is there a line of code to revert back to 29?

@ppfeufer
Copy link

You're misunderstanding what unbound is. unbound is just a DNS resolver, which AGH is using if AGH is configured to do so. unbound does not block anything, that's AGH's job. AGH does so via block lists you can configure. These block lists are subject to change, meaning, whoever maintains them also changes them — adding or removing items. AGH will update these lists periodically, which results in AGH having fewer items to block today than it had yesterday, tomorrow it might be again a different story.

All unbound does is resolve names that AGH throws at it and return their IP addresses to AGH.

@Ice2192
Copy link
Author

Ice2192 commented Jun 18, 2023

Thanks for the clarification. The version 29 is still on the GitHub I see. How do I tell my raspberry pi to download that rather than the latest update?

@ppfeufer
Copy link

ppfeufer commented Jun 18, 2023

That I don't know.

But, if your speed issue is resolved by using unbound, I highly recommend checking your block lists for updates. (Or try mine and whitelist what you need and is blocked by it).

Another point of error might be that unbound is listening on port 53, which it should not. Check that as well just for good measures, because if that is the case, AGH does not block anything because it's out of the loop.

  • unbound should listen to port 5335
  • The only upstream DNS configured in AGH should be: 127.0.0.1:5335
  • Your router/dhcp should propagate AGH as DNS in your network (primary and secondary DNS!)

@ppfeufer
Copy link

ppfeufer commented Jun 18, 2023

The /etc/resolv.conf file on your AGH Raspi should look like this:

nameserver fd00::2
nameserver 10.10.10.2

Just replace the IPs with the ones your AGH Raspi has.

@ppfeufer
Copy link

Also, check the DNS configuration of your devices and restart them (with the AGH Raspi being the first). They still might work with a cached DNS configuration you eventually have entered for testing purposes.

@Xerdunits
Copy link

I'm honestly not sure if this isn't just a coincidence with the update and not an issue with AGH in itself. And I'm also not sure why you would setup AGH and not using any of the features it offers, but that just me ;)

@Mur1thi
Copy link

Mur1thi commented Jun 19, 2023

@boarder010 @Mur1thi Why does the localhost show up in the log though? Does AHG only provide filtering and the actual DNS server is your router?What DNS server are you guys using, how does your setup look? Do you use multiple DNS server, or just one?

Mind sharing your DNS settings?

My router points to one ADGH IP address. I'm running ADGH on a raspberry pi. After a few min of having ADGH as the DNS server, I'm completely cut off from the internet. I only have 2 cloudflare IPs as upstream. After that, my query log looks like this. see image
image

@boarder010
Copy link

@Ice2192 @Mur1thi - I think I may have figured out the issue...something you may want to check. I had a clean reinstall, everything processes quickly but it would just hang on actually loading the page. Looking at all the settings again, I noticed "use private reverse dns resolver" was checked even though I didn't have anything there entered. Unchecked, applied, and everything has been running as normal for the past 30 minutes at least
Screenshot 2023-06-20 at 8 07 47 AM

@Ice2192
Copy link
Author

Ice2192 commented Jun 20, 2023

@boarder010 Keep us updated. I have clean installed and followed ppfeufer's instructions (3 times) to the T and I'm just not getting the same results in terms of adblocking. It's either speed but no blocking (which is the same as not having AGH on) or speed of 56k modem with adblocking. I somehow can't get both. If I do get both, it'll last for about an hour or so or I need to constantly use the internet because any downtime of not using it will go back to slow speeds. It's like every time I have to use the internet again I have to make AGH stretch for a few minutes before it can exercise.

@boarder010
Copy link

@Ice2192 - I'm an hour in now and all good still just like v29. My Adblock score was 99% and my ADGH avg processing time is around 23ms. I just have router dns pointing to ADGH and ADGH is only using one cloudfare upstream.

@chaoscalm
Copy link

Hello boarder010, I am testing what you have described, and since 1 hour I am not having problems anymore.

Thank you for sharing your fix.

Best Regards, chaoscalm

@Ice2192
Copy link
Author

Ice2192 commented Jun 20, 2023

Check Up
Average processing time: 85 ms
Scores:
https://d3ward.github.io/toolz/adblock.html - 93%
https://adblock-tester.com/ - 80/100

@Ice2192
Copy link
Author

Ice2192 commented Jun 20, 2023

If I toggle the counting of "Banner advertising" to off in the adblock tester website I have a full 100/100. Anyone knows how to block those?

@Mur1thi
Copy link

Mur1thi commented Jun 20, 2023

@Ice2192 @Mur1thi - I think I may have figured out the issue...something you may want to check. I had a clean reinstall, everything processes quickly but it would just hang on actually loading the page. Looking at all the settings again, I noticed "use private reverse dns resolver" was checked even though I didn't have anything there entered. Unchecked, applied, and everything has been running as normal for the past 30 minutes at least
Screenshot 2023-06-20 at 8 07 47 AM

This fix is working well for me, thanks a lot

@webocalypse
Copy link

Hi everyone,

i ran into the same issue as some of you did.
For context, i run adguardhome via docker/docker-compose and the only upstream is unbound. Unbound has a hostname "unbound" und my configured upstream is udp://unbound.
With version 0.107.30 dns queries to unbound seemed to fail. Also version .31 and .32 did not fix it for me.
A friend of mine had the issue as well and he found that when validating upstreams, udp://unbound could not be resolved.
So i checked the changelog and found that since version .30 upstream hostnames are being resolved via bootstrap dns servers. In consequence the internal docker dns was not used resulting in not resolving unbound correctly.

So is added

udp://127.0.0.11:53

as bootstrap dns and everything is currently working fine. DNS is being resolved via unbound again.
It's running for like 10 minutes.

I hope this possible solution might help some of you.

@ainar-g
Copy link
Contributor

ainar-g commented Jun 21, 2023

@EugeneOne1, I feel like this could be because of query recursion not being detected. Please investigate if any changes could have broken that.

@Ice2192
Copy link
Author

Ice2192 commented Jun 21, 2023

@boarder010 Yup that did it. A day later and I'm at 30ms WITH actual ad blocking. Thanks!

@Ice2192 Ice2192 closed this as completed Jun 21, 2023
@ainar-g
Copy link
Contributor

ainar-g commented Jun 21, 2023

@Ice2192, we'd like to keep the issue open until we find the actual reason behind the bug, if you don't mind. Having a workaround is good, but the bug is still there.

@ainar-g ainar-g reopened this Jun 21, 2023
@Ice2192
Copy link
Author

Ice2192 commented Jun 21, 2023

@ainar-g No worries!

@Mur1thi
Copy link

Mur1thi commented Jun 22, 2023

My AGH has stopped working again. Unable to resolve DNS

@Freebase394
Copy link

Freebase394 commented Jul 4, 2023

Hello everyone,
My dear mate @Ice2192,
I can see that you are having a nice results with bolocking the most of your adds.. Could you share your special configs | lists for that please ?

It would be great to share such knowledge as yours!
On https://d3ward.github.io/toolz/adblock.html sometimes i get
image
And on https://adblock-tester.com/
Others times i get 50% and others i get 77% when my AdGuardHome version v0.107.33 is running fine due to new issue #5964
image

@Xerdunits
Copy link

Xerdunits commented Jul 4, 2023

@Freebase394
Since I'm still subscribed to this thread, I might as well answer as well ;)

It all depends on what block list you use, there are a lot of lists out there and you should always keep in mind, that the more you block, the more sites might stop working. Also AGH updates those list automatically, so what worked one day, might not work the next.
One of the biggest lists that combines the most common ones is: https://oisd.nl
This should block 99% that can be blocked via DNS. When you test your adblocker in your browser, you should use an adblocker there as well. Some things can't be blocked by AGH, so you need an additional blocker.
2023-07-04 14 48 32 d3ward github io 59ca28fcbf6d
2023-07-04 14 48 50 adblock-tester com eba9d0eaf8f5

@Ice2192
Copy link
Author

Ice2192 commented Jul 4, 2023

@Freebase394 The highest I could get is 93% and 80/100 with AGH alone. Come it with uBlock origin and I get a full score on both. There is a FAQ on d3ward site that has a blocklist that might help achieve 100%.

@Elwyr
Copy link

Elwyr commented Aug 27, 2023

@Ice2192 @Mur1thi - I think I may have figured out the issue...something you may want to check. I had a clean reinstall, everything processes quickly but it would just hang on actually loading the page. Looking at all the settings again, I noticed "use private reverse dns resolver" was checked even though I didn't have anything there entered. Unchecked, applied, and everything has been running as normal for the past 30 minutes at least Screenshot 2023-06-20 at 8 07 47 AM

As others have said, this fix worked for me. Replying just to add, this only affected the instance on my NAS/LAN. I have two other fully identical instances on VPSs that were not affected by the bug despite being on the same version with identical settings.

@bobloadmire
Copy link

not 100% sure if my issue is related, but I was getting massive query times, 2-300ms using 1.1.1.1. it would get to the point where pages would load super slow, i would disconnect mid game, and windows reports internet disconnection randomly. I removed all references to 1.1.1.1 and use only google and opendns and everything has been great now, sub 20ms query times. Not sure if its a AGH bug or something with cloudflare.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug P2: High waiting for data Waiting for users to provide more data.
Projects
None yet
Development

No branches or pull requests