Logging of the number of blocked sites causing constant HDD noises in permanent PBM

[Sworddragon]

Prerequisites

• I verified that this is not a filter list issue. Report any issues with filter lists or broken website functionality in the uAssets issue tracker.
• This is NOT a YouTube, Facebook or Twitch report. These sites MUST be reported by clicking their respective links.
• This is not a support issue or a question. For support, questions, or help, visit /r/uBlockOrigin.
• I performed a cursory search of the issue tracker to avoid opening a duplicate issue.
• The issue is not present after disabling uBO in the browser.
• I checked the documentation to understand that the issue I am reporting is not normal behavior.

I tried to reproduce the issue when...

• uBO is the only extension.
• uBO uses default lists and settings.
• using a new, unmodified browser profile.

Description

It is probably the same issue Adblock Plus was affected years ago ( https://gitlab.com/adblockinc/ext/adblockplus/adblockplusui/-/issues/899 ) and today I even suspect my HDD might be SMR capable which might explain this even more.

Even when automatic filter list updates are disabled to rule this out as a cause and visiting a website that causes uBlock Origin to block anything (so the counter for the number of blocked sites goes up) I see in Windows's resource monitor (can be opened from the task manager) I/O activities in Firefox's profile folder under \storage\default\moz-extension*\idb*.sqlite .

The I/O activities continue even when doing nothing on the website and in my test it was sufficient to go to https://www.youtube.com , clicking reject all in the eventual appearing cookie/privacy banner and then just watching Windows's resource monitor for a few minutes.

I'm not sure I fully agree with Adblock Plus on their chosen solution to just not log the number of blocked entries anymore as I don't think this is too problematic information here (as long as no URL's are logged as well, which I assume uBlock Origin took already care of). If possible it might be sufficient for (permanent) PBM sessions to just keep the new numbers in memory and save them before uBlock Origin unloads (e.g. the user closes Firefox).

A specific URL where the issue occurs.

https://www.youtube.com

Steps to Reproduce

1. Enable permanent private browsing mode in Firefox.
2. Disable automatic filter list updates in uBlock Origin.
3. Go to https://www.youtube.com and reject all in the cookie/privacy banner if it appears.
4. Open Windows's resource monitor (e.g. via the task manager).
5. In Windows's resource monitor filter for all occurrences of firefox.exe.
6. Wait a bit (5-10 minutes) and look at the I/O activity.

Expected behavior

I/O activities from logging should not happen (or only once at the end) when (permanent) PBM is used.

Actual behavior

Disk activities can be seen in Firefox's profile on \storage\default\moz-extension*\idb*.sqlite (matching the generated ID of uBlock Origin in the path) that occur initially when opening the website and then indefinitely every 1-3 minutes resulting in significant increased noises on HDD's.

uBO version

1.55.0

Browser name and version

Firefox 122.0

Operating System and version

Windows 10 22H2 64 bit

[gorhill]

save them before uBlock Origin unloads

Saving is asynchronous and the documentation warns against making asynchronous calls when the extension shutdown, as there is no guarantee they will be fulfilled:

since the page is unloading, any asynchronous operations started while handling this event are not guaranteed to complete.

So what you are asking is not possible.

uBO saves the blocked/allowed request counts, but only when they change, and at most once every 23 seconds: https://github.com/gorhill/uBlock/blob/1.55.0/src/js/storage.js#L103-L108. I could possibly make use of storage.session() to extend the period to ~4 minutes (as it used to be before changes related to #2969), but I can't avoid saving that data on a regular basis.

[Sworddragon]

I fear 4 minutes would be not enough to really improve this considering all the circumstances so unless there are other reasons to do this it might be not worth for just this ticket.

Would the ABP solution then be a possibility (not storing the blocked/allowed request count at all when in PBM)? The counters are nice to have but back then when they changed that it was quite a big improvement here when the HDD was finally silent. Or alternatively an advanced setting could do this as well if others would like to keep the counters in PBM.

[gorhill]

not storing the blocked/allowed request count at all when in PBM

I disagree that PBM should affect whether request stats should be counted or not.

However, I do think that an advanced setting to not persist the request stats across sessions regardless on PBM is something that can be useful for the issue here and other requests that I have seen in the past.

[Sworddragon]

I think that would be fine then and I don't have to go nearly crazy again from the noises.

(Or I wait some years until a SSD is built in - but HDD's are still quite common and also useful in some cases)

[gwarser]

@Sworddragon toggle requestStatsDisabled in dev build to see if there is any improvement https://github.com/gorhill/uBlock/releases

[Sworddragon]

I didn't expect to see this implemented that fast, so thanks for doing all the work associated with it!

From the commit description it seems any write/read I/O for those statistics will be disabled and instead we will get memory-only counters since the last start of the extension which can still be quite useful.

Things are currently quite hectic here but at the latest when a final version is being released I will do another detailed test to monitor for any disk noises and logged I/O activities.

[gorhill]

What I fixed here is to provide a way to not keep track of global blocked/allowed requests, nothing more nothing less -- and if that works as described, the issue should be closed.

Your personal issue of "HDD noises" is not a uBO issue, so this should not prevent closing the issue here. uBO will of course continue to persist data as needed through the various storage-related APIs, this is not something that will change if the goal on your side is no "HDD noises" -- this is something you will have to figure yourself.

[garry-ut99]

From #2838 (comment) :
Reality is that I have no motivation to investigate such issues for outdated browsers. (...)
People using older browsers are welcome to put in the work to contribute to the project to help investigate and provide solution to issues specific to their old browser versions. (...)

I agree, that rule should apply not only to old software, but also to old / obsolete hardware.

Your personal issue of "HDD noises" is not a uBO issue,
if the goal on your side is no "HDD noises" -- this is something you will have to figure yourself.

I agree, as nowadays, no one, except dinosaurs, uses HDD for operating system and / or internet browsing, this is 2024, not 2014, OP can get a very fast SSD for a few bucks, if he can't for some reason, it's his problem, not uBO's.

Also I wonder whether OP has tried adjusting AAM (Automatic-Acoustic-Management) of his HDD, and set it to "Quiet mode" as most HDDs are capable of it: https://recoverhdd.com/blog/make-your-hdd-faster-or-quieter-with-automatic-acoustic-management-aam.html and whether it does help or not in this case.

[Sworddragon]

What I fixed here is to provide a way to not keep track of global blocked/allowed requests, nothing more nothing less -- and if that works as described, the issue should be closed.

I was thinking about to close the issue with my last post but I wasn't sure if this project prefers closing them on their own (for example bulk-closing on new releases). But yes, the issue can be closed then.

Your personal issue of "HDD noises" is not a uBO issue, so this should not prevent closing the issue here. uBO will of course continue to persist data as needed through the various storage-related APIs, this is not something that will change if the goal on your side is no "HDD noises" -- this is something you will have to figure yourself.

HDD noises are fine as long as they aren't permanent even on idle and the commit should fix that. In the past there were only 2 sources of constant HDD noises here (as anything else on the system including Windows itself usually does anything via bursts-writes): Firefox due to an issue where it is not fully known yet if this is a bug and Adblock Plus due to the same issue as described here for uBlock Origin. Both ad-/content-blockers addressed this issue.

Also the HDD noises were tied to uBlock Origins I/O activities from the blocked/allowed counters as observed via Windows's resource monitor and as being a downside this is not only limited to HDD noises - for example it might also prevent a more effective power saving on a system independent of the used storage type. I wouldn't call this a personal issue but it also isn't technically a bug either if one is fine with that - it might just not so really have been optimized (pre-patch).

From #2838 (comment) :
Reality is that I have no motivation to investigate such issues for outdated browsers. (...)
People using older browsers are welcome to put in the work to contribute to the project to help investigate and provide solution to issues specific to their old browser versions. (...)

I agree, that rule should apply not only to old software, but also to old / obsolete hardware.

Your personal issue of "HDD noises" is not a uBO issue,
if the goal on your side is no "HDD noises" -- this is something you will have to figure yourself.

I agree, as nowadays, no one, except dinosaurs, uses HDD for operating system and / or internet browsing, this is 2024, not 2014, OP can get a very fast SSD for a few bucks, if he can't for some reason, it's his problem, not uBO's.

The HDD is from 2019/2020 (due to replacement) which really isn't that ancient. Also back then 2TB SSD's were around 5 times as expensive as a same-sized HDD (and they are still around 2 times as expensive as of today) and they work just fine even with Windows. But yes, I have already plans for quite a while that my next storage device will be a SSD - once the HDD malfunctions.

[garry-ut99]

Sworddragon : The HDD is from 2019/2020 (due to replacement) which really isn't that ancient.

It's not about a year of production of a HDD, it's about how old/ancient is the HDD technology and it's speed, it's too ancient for running OS or internet browsing on it comfortably, they are produced still in 2024 and will be even in 2030, but their purpose is to serve as a storage for big or many files: backups, videos, games, music, setups, iSOs, etc.

Sworddragon : Also back then 2TB SSD's were around 5 times as expensive as a same-sized HDD

I didn't say it was wrong that you bought HDD many years ago.

Sworddragon : (and they are still around 2 times as expensive as of today)

I don't understand this argument, you don't need a 2 TB SSD disk for OS you can buy a cheap 256 GB or 512GB SSD for OS and still keep your 2TB HDD for files, unless you use a notebook rather than a desktop PC, and you have only a single slot for disk, but even then you can keep your HDD as an external portable HDD connected via USB to your notebook or can buy a combined SDD + HDD disk called SSHD (Solid-State Hybrid Drive).

Sworddragon : and they work just fine even with Windows.

No, they don't, they are very slow, when you get SSD you will feel the difference, it's like switching from a tractor to a racing car.

Sworddragon : But yes, I have already plans for quite a while that my next storage device will be a SSD - once the HDD malfunctions.

When is it going to happen, in 2030 or 2035, I wish you luck...

Also thanks for not answering the question about AAM (Automatic-Acoustic-Management).

[Sworddragon]

@Sworddragon toggle requestStatsDisabled in dev build to see if there is any improvement https://github.com/gorhill/uBlock/releases

On testing this with uBlock Origin 1.57.0 on idling on YouTube I indeed don't see any I/O activities in Windows's resource monitor anymore related to uBlock Origin after I set requestStatsDisabled to true (while I still see them when set to false and in both cases the blocked counter increases over time).

So this works as expected but there is one slight visual flaw: The blocked counter still claims to list all since the last installation while it lists only all since the last launch. However, the wiki implies using the same wording for both cases might be the intended behavior.