Cosmetic filters randomly ignored #3403
Comments
|
Provide a cosmetic filter you created on a specific webpage and which is not working. The most likely explanation at this point is that you create cosmetic filters with parts that change after a page load, see Element picker does not work, removed element reappears when you reload the page |
|
For example: After refresh "Webinar"s are still there. |
|
I cannot reproduce. I added While on the page |
|
If the issue is in the number of filter rows that preceed this one, it might be only reproduceable with high number of other filters. Troubleshooting information: uBlock Origin: 1.59.0
Firefox: 130
filterset (summary):
network: 239262
cosmetic: 278133
scriptlet: 54505
html: 2268
listset (total-discarded, last-updated):
removed:
easyprivacy: null
plowe-0: null
added:
https://hostfiles.frogeye.fr/firstparty-only-trackers-hosts.txt: 11908-0, 5d.6h.29m
https://malware-filter.gitlab.io/pup-filter/pup-filter.txt: 189-0, 11h.24m
https://raw.githubusercontent.com/AdguardTeam/FiltersRegistry/master/filters/filter_17_TrackParam/filter.txt: 1733-8, 5d.6h.29m
https://raw.githubusercontent.com/AdguardTeam/FiltersRegistry/master/filters/filter_3_Spyware/filter.txt: 85375-691, 2d.7h.11m
https://raw.githubusercontent.com/DandelionSprout/adfilt/master/LegitimateURLShortener.txt: 2678-156, 11h.24m
https://secure.fanboy.co.nz/fanboy-antifonts.txt: 58-0, 6d.2h.30m
https://www.i-dont-care-about-cookies.eu/abp/: 25840-4, 1d.6h.34m
adguard-generic: 84586-5173, 1d.11h.42m
adguard-mobile: 9835-75, 1d.11h.42m
adguard-spyware-url: 1730-1721, 1d.11h.43m
block-lan: 62-0, 5d.1h.26m
adguard-social: 23179-47, 1d.11h.42m
[17 lists not shown]: [too many]
default:
user-filters: 1358-59, never
ublock-filters: 40633-180, 1h.13m Δ
ublock-badware: 11582-6, 1h.13m Δ
ublock-privacy: 1277-410, 1h.13m Δ
ublock-unbreak: 2529-54, 1h.13m Δ
easylist: 85764-2236, 1h.13m Δ
urlhaus-1: 25350-0, 11h.24m
ublock-quick-fixes: 162-12, 1h.13m Δ
filterset (user): [array of 1359 redacted]
trustedset:
added: [array of 8 redacted]
switchRuleset:
added: [array of 109 redacted]
hostRuleset:
added: [array of 775 redacted]
removed:
behind-the-scene * * noop
behind-the-scene * image noop
behind-the-scene * 3p noop
behind-the-scene * inline-script noop
behind-the-scene * 1p-script noop
behind-the-scene * 3p-script noop
behind-the-scene * 3p-frame noop
userSettings:
advancedUserEnabled: true
hiddenSettings: [none]
supportStats:
allReadyAfter: 848 ms (selfie)
maxAssetCacheWait: 376 ms
cacheBackend: indexedDB
popupPanel:
blocked: 6
no-remote-fonts: true
network:
dellcdn.com: 4
dscx.akamaiedge.net: 2 |
There is no need to presume this is an issue. Filter lists contains thousand of cosmetic filters without issue, there are no internal limit for the number of cosmetic filters. However you have so many lists enabled and other customization that it would help if you could test in a new profile with only default lists. It's possible a bad cosmetic filter in another list not caught by uBO could be causing the issue. Also just to be sure, please provide the exact filter as it appears in your filter list, |
|
The exact filter is "www.dell.com##li.dec-card--clickable.dec-card:has-text(ebinar)" (added through Element Picker). After trying different Firefox profile with different filters chosen and adding all my custom My Filters from the original profile, issue is reproduced. New Troubleshooting info:
|
|
Can you add If it works, can you try
Your other profile says you had 1359 custom filters. The new profile says 28 custom filters. Can you share those 28 custom filters? Also, to be sure, can you show me a screenshot of the page with uBO's popup panel opened over it? Are you using any other extensions which also change the DOM? |
|
I copied the Filters with Ctrl+A,C,V and only a few got copied. I tried fixing that mistake with exporting and importing them (into the testing profile), but only a few get imported again. Weird. |
|
Btw the dots can be selected but not copied; they don't seem to be a text. I've noticed them before, presuming they're a sign of copying/importing new filters and do nothing, but could they be behind this issue? Should I delete all of their occurences and retest? |
|
These dots might be zero-width whitespace, see https://en.wikipedia.org/wiki/Zero-width_space. They should not be used in filter lists. |
|
I never added them manually, I think it was uBlock what added it. I deleted them which helped so that the Filter export now contains all entries until now. (Strangely enough the filesize of both was almost similar (around 114 kB), even when one had 69 rows and the other 2600 rows) However the main issue we're working on still persists. |
|
Try in a new browser profile, uBO only default settings/lists, no other extension, and only one single custom filter, |
Interesting, I have no idea, I can only say that I noticed zero-width spaces a few times in the past in My filters, but I don't remember where did they come from, I always removed them manually then anyway, I haven't seen them for a long time already.
This means that the both files contained all the filters, just that when importing from the one which contained zero-width spaces, filters behind the zero-width spaces failed to be imported. I'm not sure whether it's a Codemirror's or uBO's quirk, maybe it could be addressed as a bug and fixed on uBO's side, like in a similiar issue:
Some other guy recently has suffered a similiar issue caused by some of his many filters from "My filters", you can use bisection method like advised to him: uBlockOrigin/uAssets#24776 (reply in thread) to find out which filters cause the issue for you. |
People who cut & paste content from webpages can ends with these. Nowhere in uBO are there zero-width spaces added to filter text. I suppose it can happen when people cut&paste to manually craft |
In this case the filter worked. (I didn't even change the filter lists, only deleted all other custom filters)
Actually, no. Even when opening the .txt file in text editor, it only had 69 rows.
Hmm, you mean comment out 1300 lines manually one by one, then 650 and so on? That'd be awful lot of work. Could the problem be pinpointed from Logger or Troubleshooting information?
I've only started crafting ':has-text' and similar filters in recent months, but the whitespaces appeared between 2023-04-09 and 2023-04-13, maybe this was how uBlock marked that new filters were imported...? (Like when adding desktop filters into Android installation and vice versa) |
|
There's no line in uBO's code that adds zero-width spaces. |
You keep making presumptions, this is not the way to go. It's detective work, makes no assumption and narrow down as much as possible by using bisection. I've done that in the past with tens of thousands of filters to find the one culprit filter causing an issue, it's the only way to get to the bottom of it. If you do not want to do it, I can do it if you share your custom filters in a text file which I can copy/paste to "My filters" and see if I can reproduce on my side. If I can, then I will do the bisect. |
|
Here is a log of the website where it does not get blocked: Logger output
Hmm, if it's not clear from the logger, OK. But I'll share the filters only privately if I may. Where should I send it? |
You can send it to rhill at raymondhill dot net Once you paste all your filters in a text file, save the text file then delete the whole content of "My filters", copy paste again the content of the text file back to "My filters", and see if the issue still exist. If not, then there is no point to send me the file since this is essentially what I need to do on my side to first reproduce the issue. |
This is strange, maybe the rest are empty zero-length spaces or other invisible characters which might be invisible by default in some text editors, unless option is set to see them by default.
Nope, you seem to have missed an easy way to select many filters (lines) at once, which is already mentioned in the uBlock documentation: you can just select half of the filters and press Tab key like already explained:
|
It does persist.
A-HA!! That streamlined the process well enough. Looks like it was generic enough to affect a great number of websites. What still bugs me is why it affected them intermittently. Thanks a lot, garry-ut99 (and gorhill too). Sending a virtual hug to both of you 🤗 |
That filter is rejected on my side, and
Since the filter is rejected at compile time, it can't possibly be causing your issue. |
|
On Chromium yes - it's rejected, but in Firefox (which OP uses) it is not rejected: Another thig is that even after changing Doesn't occur with http://xpather.com/ throws an error:
|
|
Firefox returns a valid XPathExpression object, and surely it should have thrown. I am guessing the original intent was to fulfill two conditions, in which case this works: This also works: Problem with the faulty filter is that In any case, that Firefox does not report an error prevents uBO from detecting and discarding the faulty filter. I get an exception at evaluation time: |
On my side in Firefox it was not marked red. And commenting it out solved the issue on dell.com and at least one more website where the issue was occurring (even after repeated refreshes).
So it's not only my poorly constructed filter but I've co-found an actual bug?
Not sure, the filter is from July but yes, it was most likely meant to filter stuff out if both conditions are met on a particular website. Modifying the filter to |
|
I think the issue is asking for I will have to dig to figure the best fix for this. |
Related issue: uBlockOrigin/uBlock-issues#3403 To ensure XPath expressions not meant to return a nodeset are discarded at compile time.
|
I am going to assume there are no issue in Firefox. The fact that an xpath expression can return a boolean value is supported by the fact that one can ask for a Then the question becomes why does Chromium flag the expression as invalid, more accurately as "syntax error". That I don't know. In any case, in the next dev build uBO will be able to catch as invalid xpath expressions which are valid but not meant to return a nodeset. |
|
Note that this works in Chromium (until I publish the fix): But then it will fail later like in Firefox at evaluation time. The asterisk should be used when you mean to match any element. If you want relative matching to the current node: |
|
Is there a difference between |
|
|
I presume that when website loads uBO only looks up filters that apply to THE site and ALL sites and only then applies/computes them hence the performance penalty. Oops, I have 128 instances of
|
How would uBO know that a generic filter is not meant to apply to a site without actually applying the filter to the site to find out if there is a hit? |
https://github.com/gorhill/uBlock/wiki/Static-filter-syntax#specific-generic
https://github.com/gorhill/uBlock/wiki/Procedural-cosmetic-filters#important
Maybe consider reporting such sites (or at least some of them) to https://github.com/easylist/easylist or https://github.com/uBlockOrigin/uAssets to let devs try to fix them, the whole community will benefit as well, you will benefit too if these sites will be fixed using domains |
However, it should be 111 KiB vs 114 KiB - maybe your system automatically compresses files and the file is even larger than 114 KiB. As for reporting these 128 sites, it is likely that many applications at the 127 stage check whether they can abandon XPath in favor of network filters, HTML Filtering, or scriptlets -the popularization of React templates without stable attributes like aria-label with English may be the culprit. |
Nice! It seems to work that way. I found a discussion from few years ago claiming it's not possible and suggesting possible workarounds. Apparently it got implemented. 🥲 |
|
Which discussion? |
|
That discussion talked about a total different syntax of combing multiple filters with multiple domains in 1 pre-defined line. It's not related to what you are thinking about. What you are talking about ( |
gorhill
added
fixed
Prerequisites
I tried to reproduce the issue when...
Description
Lately (last 2-4 weeks) I've been noticing some inconsistencies in the way cosmetic filters are applied (or rather not applied). They are sometimes randomly ignored and elements that have been already blocked are sometimes visible.
This happens on various websites, so it's definitely issue of uBlock, rather than website changing layouts. It most often happens right after creating new cosmetic filter, that after refresh the filter gets ignored, less often it happens with filters that are older.
I should mention that I'm a heavy user, using uBlock to save some screen space from visual clutter on websites I visit often so I currently have over 2600 rows in my filters (and counting). Could that be that uBlock doesn't work well with filters of this length?
A specific URL where the issue occurs.
Steps to Reproduce
Expected behavior
Elements being blocked consistently.
Actual behavior
Elements being blocked inconsistently.
uBO version
1.59.0
Browser name and version
Firefox - 130.0.1 (64-bit) - flatpak
Operating System and version
Linux 6.10.10
The text was updated successfully, but these errors were encountered: