-
Notifications
You must be signed in to change notification settings - Fork 22
Chrome Manifest V3 discussion megathread #238
Comments
I see 3 options:
I'll explain in detail later. I would guess Edge will closely follow Chromium, so that's probably not going to work. |
Option 1: Use the new API If the new API is really weak, then I don't think Firefox can survive for long. If websites can more easily get around the weak APIs of Chrome, then website owners will have a motivation to push everyone to use Chrome. And soon Chromium-based browsers will probably become the only usable browsers. If the new API is OK, then converting to the new API is probably going to be the easiest solution. The currently proposed APIs are on the weak side. If the 30k limit is lifted and rules can be dynamically modified, then I would say it's OK. In the other thread, gorhill listed his concerns:
The rules count limit is a legitimate concern (one of my two primary concerns mentioned above), as EasyList alone has more than 80k rules. As for dynamic filtering, I never use it. To be honest I don't even know how it works. So as far as I'm concerned, that's not an issue. When it comes to the For the For extended syntax, these are a few notable ones:
For cosmetic filtering (normal and extended), pretty much everything is implemented in content scripts, which are not part of the manifest v3 proposal, so I would assume they are not going to change. The new APIs will actually make script snippets injection better, as the proposal contains new APIs that allow extensions to dynamically modify content scripts. Same thing for redirection, (the current draft of) The The only thing missing is RegExp rules (and HTML filtering if you want to count that). I don't think it'll be a problem because neither uAssets (nor me) really use RegExp rules. On the other hand, overly broad RegExp rules from EasyList are constantly causing problems. |
Option 2: Fork Chromium I would assume code for manifest v2 will eventually be removed, so the forking process will eventually become not a simple flag flip. Maintaining a browser fork is really difficult in general, and vulnerabilities can be opened if not careful. Another way is to use Electron to build a new browser. Unlike WebExtension, Electron provides really powerful APIs, which are unlikely to go away. However, the amount of work to maintain a browser is likely more than that of a fork, but since Electron apps use JavaScript and not native code, vulnerabilities are less likely to occur. Also, implementing WideVine (DRM) will be really difficult in either case. |
Option 3: Make a native app There's a shell there's a way. I'm sad to see that there's no open source native app that does more than DNS blocking. A native app is the best way to get around the constantly changing browser APIs, and as a bonus, it'll probably work for all browsers and even non-browser apps. I'm not sure how hard this will be though, last time I tried it didn't go so well. |
I probably will go with option 1 or 3. As for the timeline, I don't think it'll happen within 12 months. Right now, the APIs are not released yet, and the phase out period will likely take more than a year. I think it's a good time to get started on option 3, if things go wrong, option 1 can be used as fallback. |
First thanks for your very detailed answer. 👍 The problem I see with option 3 if my understanding is fine is that a native app would not be able to remove some objects like cosmetic filters are doing ? For me a native app sound like when I use AdAway on android, I'm pretty sure I'm missing something here about how would work a native app 🤔 ❓ I would miss "Dynamic filtering" since I use it occasionally to un-break some filters that are good but that on specific website cause problems. I don't think Firefox will disappear, it can lose users but Google will always keep Firefox alive as a caution to prevent a split of their business by US Gov (like Microsoft have done with Apple). IMO if Google weaken their API for blocking ads it could revigorante Firefox with power-users and gain 2-3% of users. When I migrated from Firefox to Chromium I dragged with me ~150 users and those users today recommend Chrome/chromium to their friends and I'm pretty sure I'm not alone. But Mozilla management is crazy those day they removed so many things/features that I migrated to Chromium to stop from suffering from the many memory leaks in Firefox. So in one way it would not surprise me that they do some crazy thing in the future like following the very same Chrome API unless the management at Mozilla change. For the fork I was thinking it was more easy than you said it is to keep the old API, like some module you glue to the main code when you build chromium and that don't require lot of maintenance. I've seen Slimjet a Chromium fork and with an integrated ad-blocker so I was thinking it was "easy" to integrate in a fork. Time will tell. |
Again, there's a shell there's a way. Native apps are (theoretically) able to do cosmetic filtering on their own as they can patch everything the browser receives, however, I do agree that for cosmetic filtering, it's easier to implement using extensions. There's nothing stopping native apps from loading extensions into browsers, so if needed, the native app can have a companion extension.
AdAway is a DNS manager, it is really limited in what it can do. I'm talking about native apps that can repack HTTPS requests.
Possibly. But Google can also point to those 20+ Chromium forks and say their code is under a permissive license, and everyone is free to fork and compete.
I never tried so I don't know for sure. Maybe you're right. |
Actually instead of trying to reinventing the wheel and fail miserably, I'll probably just take a wheel off the shelf. I'll probably still need a JavaScript runtime though, as RegExp in filters are JS RegExp. For a prototype it'll be fine, if it works out I'll worry about optimization later. |
Thanks for your complete and detailed answer it's enlighting and it remove interrogations I had. 👍 So it would be a little bit how AdGuard work, I never used AdGuard but my understanding is that there is a component that work at the OS level and another part as an extension for better efficiency.
I don't have experience but it is something I 'm thinking about for 2-3 months to be able to modify the Chromium GUI and keep it tight, maybe I will give it a try this year, this issue will ad some motivation make this project higher in my priority. Regards |
I don't really know about Electron, but I remember this talk about building a web browser in Electron. The security engineer actually recommends not try to do it. Their company tried it and it didn't go so well. |
Are you saying that websites will simply block Firefox users entirely if Firefox insists on keeping the existing webRequest API?
For this to work it would need to support HTTPS interception/rewriting which while it can be done opens up a huge can of worms security wise, especially if it is not done properly. It would also require having a companion GUI app for creating and installing the custom SSL certificate used for said interception as most people aren't going to be able to do it on their own using the standard command line tools. Even worse, on Android by default apps will not trust user installed root certificates (Chrome does but many other browsers and apps do not) and will need to be patched (unless the device is rooted, then the certificate can be installed to the system certificate store either directly or with a Magisk module). |
@jspenguin2017 I have my answer for a fork on chromium to bypass a weak API if it was needed it'S possible and Brave seem to already use this strategy: A developer on the Brave team was explaining it on Reddit: _Bat-chriscat It's worth noting that our Brave Shields (ad blocker) is not an extension; it is natively implemented. So extension API changes leave our shields unaffected. Edit: We can always remove any code or update we don't like from the Chromium base we use. So even if this didn't just affect extensions but something deeper, we could just exclude it._ So maybe if they already do that you could fork Brave to integrate natively Nano inside and bypassing the API to intercept the calls you need ? 🤔 This way it would be future proof and the control would be very granular to block everything. User would just install the browser and use it. Regards |
@jspenguin2017 You might want to checkout Proxydomo hosted here on github. https://github.com/amate/Proxydomo It's a local proxy with https filtering capability. I've been using it to strip out <script /> among other filters of blacklisted pages before it even reach the browser. It's a reverse engineered program of the popular Proximitron in the early 2000s. There's also a recent separate effort called Proximitron Reborn, but I think the author of that hasn't release the source she promised yet. On going discussion of it is on the ancient prxbx forum here: https://prxbx.com/forums/showthread.php?tid=2331 |
Just to let know user that even the security guys at Mozilla seem to think it's a good Idea to weaken the API for extensions: https://old.reddit.com/r/firefox/comments/aithmh/raymond_hill_creator_of_ublock_origin_ubo_and/eerce78/ Firefox seem to be just an inferior "Chrome clone" right now with no real differentiation and no advantage sadly. Keeping a strong and better API than Chromium could have been their Hail Mary but hey it's Mozilla as usual AKA stupid. Regards |
Yes, I'm aware that a browser based on Electron is hard to secure, but so is extending a browser using native code. As I stated before, I have no plan in pursuing this option. Thanks for the information though. |
That's a possibility.
I'm aware of it, that's why I want to use a well established project to handle the HTTPS repacking part. If done right, HTTPS repacking can be stricter security wise, as you can choose to drop support for weak encryption before major browser vendors do. Security is hard though, so we definitely need to be careful about this.
We'll worry about that part later. That's going to be easy compared to the rest.
I thought that's only for apps that use certificate pinning. Either way, let's get it working for desktop Linux and Windows first, we'll worry about Android later. |
Something that passed under the radar this autumn Mozilla want to follow V3 manifest too: https://blog.mozilla.org/addons/2018/10/26/firefox-chrome-and-the-future-of-trustworthy-extensions/ So the comment of Mozilla on Reddit is logical: https://old.reddit.com/r/firefox/comments/aithmh/raymond_hill_creator_of_ublock_origin_ubo_and/eerce78 |
I knew it's possible, and I knew Brave is doing it. That does not change the fact that coding in native code comes with risks, especially when the native code interfaces with untrusted inputs (web traffic).
It doesn't have to be. As you mentioned, the hard part is already taken care of by major tech companies and their paid developers. All we need is to insert our code at the right place and keep the fork up to date with upstream. There are 2 problems though:
I won't say it's impossible, but it's hard enough that we want to explore other possibilities first. It's not necessarily a massive effort, but it'll probably be a massive paperwork. |
Noted. Thanks for the information. |
@jspenguin2017 commented on 23 janv. 2019 à 22:22 UTC−5:
I'm not sure on how hard it is for Widevine since https://chromium.woolyss.com/ have all the version build in almost all flavors quickly since many years with Widevine, if it was complicated they would not distribute it so widely. But I understand that other API have quotas limit or a restricted but I never suffered from it but like I said I don't use the sync. I browsed quickly https://chromium.woolyss.com/#google-api-keys and found it's relatively easy to install API key if you really need it, it seem it's a thing you do once and forget about it. 😏 I would really like a Chromium fork for power-users on desktop with a more compact GUI with less padding and above all with Nano integrated. I know it's not the subject but take a look at CentBrowser they added lot of features for power-users. but he is closed source that's the downside but I use it with other forks. Tomorrow I will ask to the CentBrowser team what is their take about V3 Manifest and restriction, they are a small team (~3 people) but they are responsive and open and I will report. Regards |
Nope, it applies to Nougat and above. Even worse, there is no reason to think that Chromium and Firefox won't do something similar as they have already started heading that way with DNS over HTTPS (which also conveniently gives google access to all your DNS requests even if you run your own DNS server). So you might end up having to make either a fork or possibly a program loader (since DLL injection is blocked that leaves a loader program that patches the browser in memory) but of course with a loader they could fight back by using various tricks to make pattern based smartpatching fail possibly even including a Denuvo style custom virtual machine. . |
even with all the harmful decisions firefox is still the better choice. i think it is more likely that firefox at least keeps the api somewhat better than chrome. and even if not, there are also firefox forks like for example https://github.com/intika/Librefox/issues which aims to create an always up to date version with an automated build system |
Even with the potential issues mentioned earlier I would think the native, standalone app would be the best choice as it will be far easier to maintain a fork that merely makes SSL interception easier than it would be to reimplement the existing webRequest API especially if the browser code ends up being refactored to make it harder to do so as short of going proprietary or heavy code obfuscation/encryption it will always be easier to comment out a few functions regarding SSL certificate checks or make them always return "true" even if the check failed than it will be to keep the webRequest API working. |
I'm not sure about it at the end you will need an extension to be able to select element on the fly and create filters, even AdGuard that use a stand alone app need to have an extension and the dev already stated the such change in the API will hurt AdGuard. |
"All we need" eh? :-) Kidding aside, I understand what you mean. Perhaps time-consuming is a more accurate term for a Chromium fork with the level of custom code we'd want for robust content filtering. Just building a Chromium pull alone can take an hour or more depending on your CPU. And in the long run, there's the possibility of Google changing something internally that would impact an integrated custom filter engine. That could be a nightmare to deal with. So I'm still of the opinion that modifying the Brave engine, which they generously share as MPL, is the best approach. It's already integrated into Chromium, and if Google breaks that interface in some way, the Brave devs will have to deal with it. As for the other proposed idea, intercepting security certs sounds too much like SuperFish. I don't think I'd be comfortable with a benevolent FOSS one. |
It could be done without an extension, it would just be uglier and depending on how it is done would likely require customization for each browser and major browser revision. Most browsers have an inspect element right click function which will bring up the source for the element in the developer tools which could then be OCRed via screen grab and compared to the page source captured by the standalone app. Or possibly by injecting custom javascript (AJAX?) code that communicates with a locally running server to get around the lack of sufficient APIs whenever the option to choose an element to block is selected. Or worst case the user would have to find the element using the inspect element and create the filter on their own (meaning only advanced users who know html would be able to block elements on their own). Though that could probably be worked around by using a fork that has dll injection detection/prevention disabled combined with an injected dll (or equivalent for other operating systems) meaning you would have to install Nanofox or Nanonium to be able to create your own element filters through a point and click interface. |
At his point it would be lot more effort and time that forking Chromium.... 😵 Also most users are not interested in opening Dev tool to hide an element, it's not convenient at all. I do that myself but only for really complex ads, 99% of the time I use the selector mode of Nano. The goal of @jspenguin2017 seem to be to lower the load as much as humanly possible so i'm really not sure. Also @joey04 have a really good argument it's really not everybody that would be willing to install a root certificate for an Adblocker I don't even trust AV to install a certificate. Regards |
I've made the switch to Brave on all my devices (macOS/iOS/Win 10) recently and while there's some things that are still inferior/missing in comparison to Chrome, for me personally, it made more sense than starting to use Firefox (where I was missing extensions and didn't like the dev tools as much). Plus, I support Brave's content monetization ideas; they are not a perfect solution, but any alternatives to privacy-invasive ads/trackers are welcome. So, I hope that you will continue developing Nano for Chromium-based browsers like Brave that will bypass the V3 changes, regardless of what happens with Chrome. |
@mserajnik commented on 20 juin 2019 à 14:42 UTC−4:
From @jspenguin2017 mouth:
Brave is not the only one who stated that they will keep alive webRequest API Vivaldi too: https://www.theinquirer.net/inquirer/news/3077032/vivaldi-promises-to-restore-third-party-ad-blocking-to-chromium But we need more than webRequest API we need other API/methods that will be deprecated too, but Brave said that he will keep them too along with the webRequest API. At the end of the summer I will switch to Brave or Opera too for my desktop use, worst come to worst it will be Firefox. |
Tampermonkey needs to execute remote code, which Google will no longer allow. Note that Mozilla never allowed executing remote code, but they made exceptions for those popular userscript hosts. I'm not aware of any other APIs that we need being removed at this point. |
@jspenguin2017 commented on 20 juin 2019 à 20:58 UTC−4:
The founder of Brave told that he will keep the permission to use remote code so extensions like Userstyles Managers & Userscripts managers will be able to exist, I'm pretty sure it will be the same for Vivaldi but I'm not 100% sure for Vivaldi. The other things that many devs complaint is the removal of "background page" but I'm not sure if it will be replaced or not with others alternatives or not I remember that Manifest V1 was completely removed from Chrome code only 12-18 months ago so we should have at least 12-24 months after the approval of V3 before they remove it completely from Chrome. |
Nano does not strictly need a background "page", it is a good-to-have, but not required. The new background service worker should be enough. The only time Nano uses the background page renderer is to validate CSS selectors, but there are many alternatives:
One crazy idea is to ship a small part of Chromium renderer as WebAssembly, and load / unload it as needed. |
@jspenguin2017 I still think that the standalone app is probably the way to go especially if uMatrix (NanoMatrix?) and Tampermonkey/Violentmonkey like functionality can be added later as well. After all, even without any special effort by google to complicate things it will be increasingly difficult for the various forks to maintain said compatibility as well as having to keep the extension updated on half a dozen to a dozen different addon/extension "web stores" assuming said fork even has their own. |
@nl255 commented on 21 juin 2019 à 11:43 UTC−4:
I don,t agree here, it would require a colossal sum of time and it would be really hard to maintain, All in one extension are the worst to maintain that's why UBO & Umatrix are 2 different extensions adding Tampermonkey/Stylus to the mix would be foolish IMO. Moreover with a standalone approach you will lose lot of granularity and ability. There is no real problem since 2 major forks (at least) are committed to keep the old API and extensions are far more "easy" to keep updated compared to a Frankenstein All-in-one standalone app 😉 |
@mikhoul Not compared to maintaining multiple standalone proxies for each extension and trying to get them to play nice when daisy chaining them together. As for those forks that is nice in the short term but it will become increasingly difficult to maintain those old APIs as time goes on. Not to mention that with Brave and other forks besides Opera how exactly are you going to get the extension since you won't be able to get it from google's "web store" anymore (unless they disable extension signing completely like Kiwi does)? |
Depends on how the proxy engine is designed, it can be designed to be extensible. It can even be designed to emulate the current
I don't believe a MITM proxy has less granularity. |
If you're talking about legacy manifest v2 extensions, then yes. But for manifest v3 extensions that use blocking
Yes, that is certainly a problem. Especially when some stores don't support publishing from command line. |
I know it's been a while, but I have yet to make up my mind. I have been observing the situation develop though, and here's my update on the "3 options" I listed at the very beginning.
Here are the important things to watch next.
|
@nl255 commented on 21 juin 2019 à 13:29 UTC−4:
Not really since Brave already maintain its own code for their own addblocking and for the webrequest API like @jspenguin2017 already stated the code will be in Chrome/Chromium and even it it was removed it's easy for Brave/Opera to maintain different API they both have already of custom API/mechanism.
There is many way for me I just side-load in dev mode for the casual user Opera have already it's own store and it will probably be the same for Brave. With Microsoft browser it's the same they have thir own store for extension and you can choose from the Chrome store of the Microsoft store. Brave is already ready to have their own store for the extension that would like to use their specific API/features: https://github.com/brave-experiments/brave-extension-store Also there is nothing complicated to keep an extension updated for the user the update is automatic and for the Dev I'm pretty sure Brave and Opera will make the publishing process easy for the uploading/updating for the extensions that will take advantage of their particularity. Don't forget that Firefox could also be an alternative again for power users IF they don't follow the path of Google for V3 API... However I will not hold my breath for Mozilla doing the "good" thing... I have been deceived to many times by their decisions. BTW: Chrome team today stated again they want global limit for rules not per extensions which will limit a lot many extensions. |
@jspenguin2017 As I understand it, the blocking webRequest API is only available in the paid Enterprise version and thus the code for it won't be present in the Chromium source code (i.e. the patch to enable/implement it will be closed source). Otherwise it would probably be fairly easy to patch Chrome or write a Chrome loader to enable it for all users*. Of course said loader would also need to disable signature checks or at least enable installing extensions permanently whenever in dev mode. As for the native app, if you do decide to go that way you might want to contact the EFF as their privacy badger addon has the same issues so perhaps they can help finish it so they can use the same code as well. *While Chrome does have some dll injection protection it probably doesn't have anything even on the level of Shiva to prevent in memory or even on disk patching much less Denuvo level. You could probably even use smart patching (i.e. pattern based rather than offset based patching) to make it work with multiple versions. |
If they roll their own APIs, I would count that as a fail. That would quickly become too fragmented.
I only heard about complains about the Opera store. Don't keep your hope up.
Since the per-extension limit would be lifted, I'm not sure if this is positive or negative. Note that the rules limit for Safari is 50k, and some people / newspapers consider Safari to be an adblocker-friendly browser. |
While it is possible for Google to strip the code from Chromium and only ship it in Chrome, I think you can get the enterprise version for free. Even if they do strip the code, I don't think they'll do anything to make it painful to re-add it as they would be stepping on their own toes. |
One more desktop browser will officially support the old webRequest API: https://www.zdnet.com/article/opera-brave-vivaldi-to-ignore-chromes-anti-ad-blocker-changes-despite-shared-codebase/ 😄 |
Hi @jspenguin2017 , Not precisely related to Manifest V3 but it's in the same vein with Google attempt to restrict ad-blockers ... From this Issue in UBO https://old.reddit.com/r/uBlockOrigin/comments/dgncga/dev_build_1225rc1_rejected_from_chrome_web_store/ I understand that UBO will possibly be no longer available from the CWS and I fear the same thing will happen to Nano. What is your take on this @jspenguin2017 ? 🤔 Regards ! |
@mikhoul Sometimes false positive happen, it usually gets resolved after a resubmission or an email. I don't think it's a problem. |
|
Gorhill have a rigid personality and did not change anything in the code when he resubmitted so it's expected to receive the same results. Just changing some code "cosmetically" could have resulted in the the extension to have passed instead of being rejected. Gorhill has many quality (very good developer, focused, hard worker) but he is not flexible and he is a little bit in his bubble, he did not even try to contact Google by other mean than answering to the email, he just marked the issue as "wontfix". Anyway right now Google is looking in the issue: https://old.reddit.com/r/chrome/comments/dgoymg/warning_ubo_ublock_origin_will_possibly_be/f3e4fi6/ |
1.22.5rc2 has been approved in the CWS.
After they rejected |
Good news so it was just some kind of mistake of a false positive like @jspenguin2017 was saying. |
It's likely caught in some automatic filter, usually (so far, every time for me) sending an email to request a human review corrects the problem. |
I never thought it was some nefarious scheme to get rid of |
I feel that WebStore is easier to use (for developers) than Microsoft Store and AMO. Although it looks ugly, it is blazing fast. Microsoft Store is definitely the worst in term of performance, it takes forever to load and does not offer a functioning automation API. As for clearing false positives, WebStore have been the easiest among the three in my experience, simply replying the automatic email is enough to get a human review. Unlike the other two which require submitting a new version with an explanation letter. |
Point taken, you are clearly in a better position to make an argument about the quality of CWS than I am. |
Every store have some weird policies. Microsoft Store doesn't allow screenshots to have description on the top 75% of the image, but it's perfectly fine on the bottom. AMO doesn't allow I do agree that WebStore emails have poorer quality than the other two stores' and I do wish WebStore emails would contains proper description and actionable directives like those from the other two stores, but I don't think this is something to freak out about. |
Hi,
I really like Nano more than UBO but I was asking to myself what will be your direction if the proposal is adopted and that the actual API is neutered ?
My understanding is that Chrome (for me ) would become inferior to Firefox for adblockers (unless Firefox implement the same restriction which I doubt).
If the change happen they can happen in the next 12 months which is relatively fast so the solutions/alternatives I've see are:
Maybe there is other solutions that I don't foresee. 🤔
What is your take on this @jspenguin2017 ?
The text was updated successfully, but these errors were encountered: