Support for js-ipfs as a replacement for HTTP gateway

[daviddias]

Right now this extension relays on the HTTP Gateway to serve the pages through IPFS. Ideally, we should be able to capture the requests and reply back without having to make a request to a gateway, this way we could serve pages from js-ipfs too.

Thoughts @lidel ?

[victorb]

I've been trying to do this, by intercepting the requests with the API that the web extensions provide and there are ways of canceling or redirecting requests, but I haven't found a way of injecting content into the request's body or overwriting it completely

[daviddias]

Would https://flyweb.github.io/ be a way to achieve this? Seems that the project has matured a lot since I last checked.

[lidel]

Can't do much with current WebExtension APIs related to WebRequest. One can cancel request, redirect it and modify headers, but we can't create artificial response with arbitrary body.

I glanced at FlyWeb and it does not seem to be feasible: my understanding is that "FlyWeb node" can't be run from within WebExtension, and if we expect user to run additional process in her system that could be "discovered" by Firefox.. it is just as good as running go-ipfs in the first place. Or am I missing something?

The only(??) "stable" thing that could work right now is HTTP gateway, we could use it the same way as go-ipfs one.

I am not sure if it is possible for browserified js-ipfs to provide local IPFS2HTTP Gateway (exposing tcp port with HTTP service just like go-ipfs does), but there is an open issue for this very topic at ipfs/js-ipfs#693 (and PR at ipfs/js-ipfs#698).

Any ideas / hacks would be appreciated. I am shooting blanks at this one.

[daviddias]

I am not sure if it is possible for browserified js-ipfs to provide local IPFS2HTTP Gateway (exposing tcp port with HTTP service just like go-ipfs does),

It wouldn't be browserified/webpacked as that step would remove any native Node.js modules dependencies. To confirm, what you are saying is that we can open a TCP server or an HTTP server from a WebExtension? If yes, could you provide me with an example? I'm pretty sure we can get the full gateway working for js-ipfs as well pretty quickly.

[victorb]

I don't we will be able to open TCP sockets in web extensions, there is no API for that. However, we might be able to create a Websocket API that you can use from a webpage, and that way load the content. That would be pretty nifty actually.

…

On Tue, May 9, 2017 at 10:07 PM, David Dias ***@***.***> wrote: I am not sure if it is possible for browserified js-ipfs to provide local IPFS2HTTP Gateway (exposing tcp port with HTTP service just like go-ipfs does), It wouldn't be browserified/webpacked as that step would remove any native Node.js modules dependencies. To confirm, what you are saying is that we can open a TCP server or an HTTP server from a WebExtension? If yes, could you provide me with an example? I'm pretty sure we can get the full gateway working for js-ipfs as well pretty quickly. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#248 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAcD9I3SokDUyd-9VKkbq5MiDr0L5zswks5r4MeQgaJpZM4NVxl9> .

[lidel]

@victorbjelkholm even if we hack around and pass resource data over Websocket API, I am not sure how we could construct artificial HTTP response with it.

Example: addon intercepts request to resource at https://ipfs.io/ipfs/<mhash>, we read data at <mhash> from js-ipfs instance via Websocket API. And.. what then?

@diasdavid there is an open Bug 1247628 - Enable "TCP and UDP Socket API" for WebExtensions, which would be game-changer for us. Without it, we can't really have real "http gateway" provided by js-ipfs.

[victorb]

Yeah, not fully developed thought yet but if we inject a tiny script to read from websocket endpoint on `onHeadersReceived` and use that to read the content, it might be possible. But as I said, not fully sure this is even possible.

…

On Tue, May 9, 2017 at 10:23 PM, Marcin Rataj ***@***.***> wrote: @victorbjelkholm <https://github.com/VictorBjelkholm> even if we hack around and pass resource data over Websocket API, I am not sure how we could construct artificial HTTP response with it. Example: addon intercepts request to resource at https://ipfs.io/ipfs/ <mhash>, we read data at <mhash> from js-ipfs instance via Websocket API. And.. what then? PS. FYI there is an open Bug 1247628 - Enable "TCP and UDP Socket API" for WebExtensions <https://bugzilla.mozilla.org/show_bug.cgi?id=1247628>, which would be game-changer for us. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#248 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAcD9G0StJDqgekBZehgzbAOw66EGCDKks5r4MtNgaJpZM4NVxl9> .

[lidel]

onHeadersReceived is too late, it is fired when the HTTP response headers sent by public gateway already have been received[1]:

onBeforeRequest is what addon uses right now for redirect, but its API is very limited and you can either cancel or redirect request[2].

There is no obvious way to "manually build response" with WX APIs, but I wonder if thinking outside the WX box and using something like Service Worker API could be our silver bullet[3]:

Your service worker can respond to requests using the FetchEvent event. You can modify the response to these requests in any way you want, using the FetchEvent.respondWith method.

---

[1]: https://developer.mozilla.org/en-US/Add-ons/WebExtensions/API/webRequest
[2]: https://developer.mozilla.org/en-US/Add-ons/WebExtensions/API/webRequest/onBeforeRequest
[3]: https://developer.mozilla.org/en-US/docs/Web/API/Service_Worker_API

[daviddias]

There is no obvious way to "manually build response" with WX APIs, but I wonder if thinking outside the WX box and using something like Service Worker API could be our silver bullet[3]:

That would require Service Workers to be spawned from a WebExtension and we would have to spawn one for every domain so that the requests can be captured.

That gives me an idea of having a service-worker.ipfs.io that will install a service worker on your browser with js-ipfs on it, and doing the requests through js-ipfs instead of loading hashes from http. This is actually pretty cool. It would only work for ipfs.io urls, it is an interesting start :)

[lidel]

That would require Service Workers to be spawned from a WebExtension and we would have to spawn one for every domain so that the requests can be captured.

Idea: if we keep already existing redirect in onBeforeRequest, then all IPFS requests have the same hostname and we only need Service Worker for 127.0.0.1:8080 (or service-worker.ipfs.io).

[daviddias]

Yeah, that can work two

• We should not use 127.0.0.1:8080 and use redirect requests to service-worker.ipfs.io, this way we keep the browser happy with https
• We lead the user to open the service-worker.ipfs.io page once the extension is installed, so it gets installed on their browser
• At that moment, we don't need the extension to do all the redirects because the service-worker.ipfs.io can do it. We will still want the extension to be a nice UI though that can configure the node that is running on that Service Worker.

[daviddias]

So, just to make sure we are all on the same page (this is indeed very interesting). This is what I'm proposing:

I'm convinced that this actually works!

Then, we can juice this up by running another js-ipfs node in the extension so that we can also have WebRTC enabled, since WebExtensions are closer to get RTC DataChannel than Service Workers.

This means offline use cases will be impossible eh?

What about @kyledrake's search bar hack?

[daviddias]

@lgierth it doesn't mean that at all. You just need to load it once and you are golden.
I believe your confusion is because you are seeing a domain there, but that is how service workers work, they get downloaded/installed and then they are capable of capturing requests to that domain, it is kind of an app cache with the ability to touch network requests. Read more at https://github.com/w3c/ServiceWorker/blob/master/explainer.md

What search bar hack? Could you provide url?