Investigate Faster Injection

[quoid]

Related: #183 #178

At the time of posting this issue script injection occurs in the content script. The reason this is done is because reliably setting up injection through the background page, while theoretically should be faster, does not seem feasible.

The reason background injection does not seem feasible is that there seems to be no reliable method to detect when a user will visit a page. browser.webNavigation.onCommitted would seem like a good event listener to attach to but it's not. There's 2 reasons for this:

1. The details object that is passed to the callback can not, currently, get the actual tabId where the navigation took place; the tabId is always 0.
2. The event gets fired even if a user does not "commit" to the navigation. When a user starts entering an address in the the address bar of Safari and suggestions pre-fill the address bar, that triggers the event. There's no way to discern, through onCommitted, "real" navigation events and suggestions from the address bar. However, even one could make the distinction, the tabId would still be needed.

The other browser.webNavigation events either fire later than the content script or too early.

I looked into using the browser.tabs.onUpdated event and while this API works as it should, it does not seem viable. The injection time seems slower than content script injection and this event won't handle subframe injection. So additional programming would be required to also check if there are sub frames in the current tab. It's pointless if even without sub frame checking the injection time is slower.

Another method could be attaching to a browser.webRequest event except it seems like Safari does not currently support that API (even though MDN says it does) or there is a somewhat recent bug that is preventing access to that API. I am not sure if this API will be reliable and/or have the same issues as above, but it would worth investigating when whatever bugs, whether that is upstream or my implementation, are fixed.

For the time being I believe the extension is using the fastest possible injection method available. If anyone has opinions or suggestion please share.

[specter78]

@quoid I tried developing a redirector extension, which works as I intend it to using background.js which gets injected on every page and even runs before page load. However, it may/may not provide the same flexibility as user scripts.

So an option may be to allow user to edit this background.js based on his/her requirements. For other use cases, user scripts may be preferred.

I don’t think event binding is required, at least for me it is working without it. My function is simply if..elseif...else like below.

if(window.location.hostname == "www.reddit.com"){
    window.location.replace("https://libredd.it" + window.location.href.split("www.reddit.com")[1])
} else if ...

[quoid]

@specter78

Thanks for the reply.

To be clear, you are using background.js as a content script, not from the background page. If you were calling window.location from the background page, window would represent the background page's window object, not the one the user is visiting in a browser tab. Correct me if I am wrong.

This extension currently uses that methodology for injection. The reason your injection could be happening quicker is that there is barely any processing that is happening. As I am sure you know, this extension needs to gather all the userscripts data, read the files, process for injection, etc...

That is why event binding is seemingly so important. If we can attach to an event, from the background page, right before or right at the navigation event, all that processing can be done (hopefully) by the time the first load event is fired.

an option may be to allow user to edit this background.js

Even with that, the data needs to be stored and read and then we can be running into the situation that exists currently.

It seems the biggest blocker for this is either unsupported api or potential bugs in the existing api.

[specter78]

@quoid

To be clear, you are using background.js as a content script, not from the background page. If you were calling window.location from the background page, window would represent the background page's window object, not the one the user is visiting in a browser tab. Correct me if I am wrong.

background.js (this is the same to what Xcode names as script.js by default, if that helps) is a content script that is being injected into every page that is visited. So the window.location would refer to every webpage being visited and not just in background. (sorry for the name confusion!)

As I am sure you know, this extension needs to gather all the userscripts data, read the files, process for injection, etc...

That is precisely why you need to separate this file from other user scripts.

The content of this file is supposed to be plain old javascript (none of the userscript). So you can inject them into each page, without any processing and let the browser take over.

That is why event binding is seemingly so important. If we can attach to an event, from the background page, right before or right at the navigation event, all that processing can be done (hopefully) by the time the first load event is fired.

The processing vs first load event is a big if. Another alternative here can be allowing plain js files along with userscripts (you may need to refactor code), and only process userscripts. That should theoretically save some time.

[quoid]

@specter78 Thanks again for the reply.

That is precisely why you need to separate this file from other user scripts

The content of this file is supposed to be plain old javascript (none of the userscript). So you can inject them into each page, without any processing and let the browser take over.

There's not a way to omit processing when we working with user generated content. At the very least it still needs to be saved to the user's filesystem, fetched before a page load, read and finally injected. That involves multiple messages being sent from the content script, to the background page, to the native application, back to the background page and then finally the content script (as it is done currently).

The reason your extension is likely so quick is because none of this is required and you are hardcoding directly into the content script. And I suspect if that content script remained as slim as a few redirects, even with the aforementioned steps it would still be quick. In fact a very early version of this app/extension basically did what you are talking about and it injected relatively quickly from the content script. However once that javascript file grew in size and scope, the injection times suffered, which I reckon is something that would happen in the case as well.

Another thing to point out is when injecting in the page scope through a content script, the page's overhead needs to be taken into account. For example, a smaller, much more lightweight page will see quicker injection time than a massively bloated website, like Reddit.

Another alternative here can be allowing plain js files along with userscripts (you may need to refactor code), and only process userscripts. That should theoretically save some time.

I think that could save some time but I am not sure it would be significant as I think that is related to what will be injected. Right now plain js files are loaded, but their metadata is checked to see if it's minimally valid before they are injected. We'd still need to parse the file and check if they have metadata to distinguish between what is a userscript and what is not a userscript. Further, as mentioned above, the material to be injected will affect the the speed of injection. A handful of code would be a lot quicker than 1000 lines.

I am not entirely sure what is suggested here will guarantee quicker injection for the reasons mentioned above and it would require an entire offshoot of functionality to be developed as a "band aid fix" to a larger dilemma.

The main goal for me is to find a way to speed up userscript injection. Doing that will solve for the slight delay in injection times that can affect the functionality of certain scripts.

[specter78]

@quoid Thank you for the explanation. I think I understand the issues.

From what you’ve said, it seems that a lot of processing is done just before page load event (correct me if I’m wrong). So can the processing be done beforehand, like in case of dynamic content blockers like AdGuard or 1Blocker ?

Whenever any user script is modified, process it (as much as possible) and load it in extension app which should save disk read overhead (apart from saving to disk, obviously!). So now whenever page load event is fired, you don’t need to go to disk and process, just do minimal checks and injects accordingly.

Another option is to bind browser events, as you suggested. But for that to function, the script needs to get injected within a certain time frame. So I suggested the above.

[cobaltt7]

Maybe some of the processing can be done in the background page?

…

________________________________ From: specter78 ***@***.***> Sent: Friday, December 17, 2021 11:34:14 PM To: quoid/userscripts ***@***.***> Cc: Subscribed ***@***.***> Subject: Re: [quoid/userscripts] Investigate Faster Injection (Issue #184) @quoid<https://github.com/quoid> Thank you for the explanation. I think I understand the issues. From what you’ve said, it seems that a lot of processing is done just before page load event (correct me if I’m wrong). So can the processing be done beforehand, like in case of dynamic content blockers like AdGuard or 1Blocker ? Whenever any user script is modified, process it (as much as possible) and load it in extension app which should save disk read overhead (apart from saving to disk, obviously!). So now whenever page load event is fired, you don’t need to go to disk and process, just do minimal checks and injects accordingly. — Reply to this email directly, view it on GitHub<#184 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AOT5DEQXIONM5NU6U7NJ7WLURQMNNANCNFSM5KHUXU6A>. You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[quoid]

@specter78 Yeah, I think the key is doing the processing beforehand. That's why the webNavigation or webRequest events seem to have so much potential. We can get navigation intent before or right at the time so we can seemingly pre-process so everything is ready at the time of the first load event. I just wish there weren't the idiosyncrasies with Safari when compared to the other browsers, but I will continue to investigate.

Maybe some of the processing can be done in the background page?

@RedGuy12 I am not sure that would speed anything up by doing that alone, but it is a good first step because eventually that logic should live there instead of the content script. So this should be a goal for the next major release for sure.

[quoid]

Just a note to add on this topic in general, I am not sure it matters browser.webRequest will be a viable option. According the google documentation, it is incompatible with non-persistent background pages, which is a requirement for iOS WebExtensions.

The only occasion to keep a background script persistently active is if the extension uses chrome.webRequest API to block or modify network requests. The webRequest API is incompatible with non-persistent background pages.