$important option doesn't work if domains are not prefixed by ||

[Yuki2718]

Prerequisites

• I verified that this is not a filter issue
  • Filter issues MUST be reported at filter issue tracker
• This is not a support issue or a question
  • Support issues and questions are handled at /r/uBlockOrigin
• I performed a cursory search of the issue tracker to avoid opening a duplicate issue
  • Your issue may already be reported.
• I tried to reproduce the issue when...
  • uBlock Origin is the only extension
  • uBlock Origin with default lists/settings
  • using a new, unmodified browser profile
• I am running the latest version of uBlock Origin
• I checked the documentation to understand that the issue I report is not a normal behavior

Description

$important option does not override exception filters unless its prefixed by ||example.com, in other words a rule like
/path/ads$important,domain=example.com|example.org
doesn't override the exception and this behavior is not explained in wiki.

A specific URL where the issue occurs

https://9to5mac.com/

Steps to Reproduce

1. Go to the site with EasyList enabled and
   /wp-content/*/prebid$script,important,domain=9to5google.com|9to5mac.com|9to5toys.com|electrek.co|dronedj.com
   added to My Filters and also logger opened.
2. See the request to the resource was allowed.
3. If you have a rule like
   ||9to5mac.com/wp-content/*/prebid$script,important
   it will be blocked.

Expected behavior:

$important should override any exception.

Actual behavior:

Stated

Your environment

• uBlock Origin version: 1.25.2
• Browser Name and version: Firefox 74.0 / Firefox Android 68.6
• Operating System and version: Windows 10 / Android 10

[Yuki2718]

The title might be bad, as a rule like
/wp-content/*/prebid$script,important
also doesn't work.

[uBlock-user]

/prebid$script,important is not working either.

[gorhill]

The URL of the request which you expect uBO will block is:

https://9to5mac.com/wp-content/themes/9to5-2015/assets/js/vendor/prebid1.20.0.js

In uBO, /prebid won't match /prebid1, you need to make the rule /prebid*, so this works:

/wp-content/*/prebid*$script,important

This has been discussed in the past, see gorhill/uBlock#1065 (comment).

Now the fact that it works with || is completely accidental and due to implementation details (the use of the new biditrie code). To make that working case behave like the non-working case would be to add a pointless overhead. The thing to keep in mind is still true: if the token to match is not a bounded word as the case above, add an asterisk to make it explicit.

[gorhill]

due to implementation details (the use of the new biditrie code)

I was guessing as I wrote in there. The real reason is that wp and content -- the other tokens beside prebid -- are categorized as "bad tokens" by uBO, see https://github.com/gorhill/uBlock/blob/381498daa2a9ce089a69d044760190b1dd14b5ac/src/js/static-net-filtering.js#L2062.

So the best token uBO saw in the filter /wp-content/*/prebid was prebid, which it turns out was not to be used as a token (a trailing asterisk would have told uBO to not use prebid as token).

The filter ||9to5mac.com/wp-content/*/prebid worked because uBO used 9to5mac as token, which is not categorized as a "bad token".

[gorhill]

So to conclude, the reported issue here is essentially a duplicate of gorhill/uBlock#1065.

[uBlock-user]

are categorized as "bad tokens" by uBO

Is that list present since the beginning and why is this needed ?

[gorhill]

Is that list present since the beginning and why is this needed ?

A list of bad tokens has been present since the first commit, which was simply guessed and manually entered.

In this commit, it was expanded from a programmatically generated histogram (search for tokenHistogram in commit message).

ABP adopted the idea in https://gitlab.com/eyeo/adblockplus/adblockpluscore/-/issues/30.

"Bad tokens" are simply all the most frequent tokens from non-blocked URLs, so tokens which are not useful to use to store blocking filters.

[Yuki2718]

Ah, I wish I knew it earlier. So the rule would have worked IF the request was like /prebid. or /prebid-. As some users suggested in the past issues it may be worth being added to the syntax page of wiki. Sorry for the duplicate, I thought it to be a matter of $important option.