Sharing IPFS node across browsing contexts (tabs) on same origin

[Gozala]

cc ipfs/in-web-browsers#158

When site with origin e.g. foo.com uses JS-IPFS, new node is bootstrapped for every browser context (tab, iframe) this implies:

• All the network connections need to be reestablished.
• General memory / CPU overhead is multiplied by number of browsing contexts.
• Create a possibility for race condition (two nodes read / write data concurrently).

There is an opportunity to improve this by offloading most of JS-IPFS work into SharedWorker in browsers where API is available and fallback to a dedicated Worker elsewhere (basically Safari).

We can explore ServiceWorker as a better fallback mechanism in the future. Unilke dedicated worker they can be shared, but they also come with enough challenges to justify keeping it out of scope initially.

However use of SharedWorker implies some trade-offs:

Worker script distribution

Worker needs to be instantiated with a separate JS script. Which leaves us with the following options:

1. Create a separate "worker" dist as opt-in for shared node. That still has two more options:
   1. Spit out two scripts on for main context and other for worker context.
   2. Use something like a worker-loader to bundle worker script as a blob embedded in the the main script.
2. Self load main script into worker and context detect to choose between main and worker operational modes.

Lack of WebRTC in workers

This is a major problem as WebRTC is largely replacing WebSocket transport and requires more research. Pragmatic approach could be to tunnel WebRTC traffic into a worker, however that would still imply reestablishing existing connections which would be a major drawback.

[Gozala]

Another thing I'm recognizing now is that two different browsing contexts (tabs) could start IPFS node with different configurations. Which needs to be handled in some way.

[Gozala]

I have started hacking on the ipfs-message-port-client that in the spirit is very similar to ipfs-http-client except it wraps around MessagePort.

However I'm realizing that API surface is pretty large, which reminded me of js-ipfs-lite that is a lot slimmer but also well type typed (TS ❤️). This combined with the configuration point above got me wondering if this maybe a good opportunity to consider stripping some of the convenience stuff which could be pulled in separately (maybe even opportunity to converge code bases ?)

[Gozala]

I had some discussion with @hugomrdias last night, about aegir not yet supporting multiple webpack configs where he pointed out shiny package.json exports which could be a better solution going forward.

After spending all day yesterday running into walls, I was desperate for some progress so I have come up with a following plan and would really love your feedback @achingbrain

• Create ipfs-message-port-client sub-package (with it's own .aegir.js) that is the spirit of ipfs-http-client except it wraps around MessagePort. Idea is that it will be loaded in the main thread and will RPC with IPFS node through the MessagePort.
• Create ipfs-message-port-host sub-package (with it's own .aegir.js) that wraps around MessagePort and starts IPFS node and executes incoming commands.
• I think you wanted to keep it all in the same package and just spit out different dists, but I hope it's reasonable to stat those as separate sub-packages first and then figure out how include it into js-ipfs afterwards. Especially since it would involve changes to aegir and I would much rather defer it until we know exactly what we want.
• I would like to reduce scope as much a possible for MVP and iterate on features incrementally. Hence the Sharing IPFS node across browsing contexts (tabs) on same origin #3022 (comment) @hugomrdias also mentioned efforts to strip everything to essentials and Dependency injection patterns for reducing bundle size ipld/js-block#10. That coupled with the fact that nodes are highly configurable challenges general idea of "node sharing" I would really really like to define what the bare minimum API is and make it an MVP.
• As I started work on ipfs-message-port-client (Gozala@f2994c7) I've attempted to use TS JSDoc but it seems that this does not yet support class properties Support JSDoc @property microsoft/TypeScript#15715 nor type checker seems to be much of help. This got me wondering if using ts here would be acceptable ? I know it's not the first time I'm bringing up type checker, but it does really help with large code bases.
• APIs like current ipfs.add are somewhat tricky due to highly polymorphic API and streaming.
  • e.g. normalise-input takes almost anything and turning it into AsyncIterable<{ path, content: AsyncIterable<Buffer> }> which adds ton of complexity especially when things need to be moved across the wire. It also makes it easy to pass sub-optimal input. I think it would be nice if bare minimum API was optimized for better performance. That is to suggest used ArrayBuffer as they can be transferred without encoding / decoding or copying across the threads. Support for more exotic inputs could be than added via helper libs that users could choose to use.
  • It is not exactly yet clear to me how important support for streaming is. If absolutely necessary we could map AsyncIterators to MessagePorts and transfer those or alternatively do multiplexing over the same MessagePort but if it's mostly for convenience I think it would be more justified to do it as another helper lib. e.g. from what I can tell ipfs.files.write could effectively be used for writing out of stream input which again could reduce complexity of the RPC protocol and could be implemented only on the client end.

[lidel]

• I like the idea of ipfs-message-port-* packages.

  • this work could help us improve modularity of js-ipfs itself, dogfooding the "core library" in packages/ipfs and use parts of it to assemble runtime-optimized distribution in packages/ipfs-web etc
    • would be good to get early feedback from @achingbrain about this direction ;^)
  • (loose thought) if those are generic enough we could reuse them for window.ipfs interface in IPFS Companion (I had some prior discussions about something like that with @alanshaw, and we always wanted similar sandboxing and limited API surface for window.ipfs anyway)

• Agree the API surface is too big and IMO for MVP.

  • I suggest we aim at the most basic file and DAG operations, punting things like Peer Identity, MFS/key "namespacing" etc for later phase.
    • For extremely lean MVP supported APIs would be: ipfs.add, ipfs.get, ipfs.cat, ipfs.dag.* (unsure about unixfs-specific ipfs.object.*)
  • This would also enable us to test those browser optimizations for reading big files during add

• TS JSDoc is something @hugomrdias experimented with before, not sure what is the status tho. Perhaps drop it for now if it gets too distracting.

• Note that ipfs files write produces different DAGs than ipfs add (Upload via WebUI (ipfs.files.write --create) produces different hash than CLI (ipfs.add) ipfs-webui#676)

[Gozala]

punting things like Peer Identity, MFS/key "namespacing" etc for later phase.

I think those should not matter for same-origin node sharing unless I'm overlooking something.

For extremely lean MVP supported APIs would be: ipfs.add, ipfs.get, ipfs.cat

Unfortunately those are the most complicated and problematic ones, getting ipfs.files.* would be a lot easier. I wrote down notes explaining why https://gozala.io/pl-notes/JSCoffee-2020-05-13 and hope to discuss in more details during the call.

ipfs.dag.* (unsure about unixfs-specific ipfs.object.*)

Complexity is mostly in having jQuery style APIs where functions take pretty much anything. ipfs.dag.* and ipfs.object.* don't seem to have pretty straight forward API so it should be a big deal.

This would also enable us to test those browser optimizations for reading big files during add

Could please explain what you mean here, I'm not sure I follow.

TS JSDoc is something @hugomrdias experimented with before, not sure what is the status tho. Perhaps drop it for now if it gets too distracting.

He actually filled me in on this today, very exciting stuff! We also took same approach so things should merge nicely. I don't think it's distraction it make navigating large codebase like js-ipfs a lot more manageable.

Note that ipfs files write produces different DAGs than ipfs add (ipfs/ipfs-webui#676)

I don't think that matters because mostly client forwards ArrayBuffers / Blobs to the worker thread which than runs corresponding API calls.

[carsonfarmer]

This is exciting! I'm looking forward to contributing here more as time permits. In the mean time, the minimal API alluded to above is very close/similar to ipfs-lite. There is a (currently on the back burner) TS implementing here: https://github.com/textileio/js-ipfs-lite. Just for reference!

[Gozala]

I have outlined some high level questions I would love to get answered by surveying our community / users. I think it would help to ensure that effort is focused on user needs (in browser context) as opposed to API compatibility with IPFS in runtimes which have different constraints.

1. What are the essential configuration options for JS-IPFS in browsers ?
   It is important to distinguish it from incidental, where it is means to an end. In other words, it is worth pondering if existing use cases could be addressed through different means.

2. What subset of JS-IPFS API is used in browsers ?
   It is worth considering this, so we can better prioritize efforts and optimize experience of what is important as opposed to aiming for API compatibility.

3. Why users are choosing to use JS-IPFS in main thread over worker thread ?
   Given that effort effectively moves JS-IPFS to a worker thread, we need to understand why users chose to use JS-IPFS in main thread in first place. (Validate working assumption that doing it on main thread was just easier). It is also worth considering how important high level JS-IPFS API in main thread is (It may be that users would be perfectly happy with using JS-IPFS in worker thread)

4. How does user priorities fall in regards to convenient API vs bare-bone API ?
   JS-IPFS prioritizes convenience. At the same time >60% of code written for this effort deals with accepting variadic inputs, effectively encoding / decoding streams etc… It is worth checking our assumptions about users needs / wants to avoid bloat that may be irrelevant in practice

5. What are other important things (not on the list) that JS-IPFS does or you wish it did for your browser use case ?
   E.g Textile chose to invest into js-ipfs-lite & which probably motivated by mismatch in what JS-IPFS provides vs what team needed. By asking we hope to identify other high priority needs / wants for JS-IPFS in browser that we may have overlooked otherwise.

If you spot something overlooked or missed please let me know.

[Gozala]

@carsonfarmer thanks for joining the thread and prompt to look closer to the js-ipfs-lite API. I have few questions remarks based on what I see there, which I would like to understand:

1. Library uses modular architecture, which is great way to manage complexity. However I would like to understand how important that is for textile. Reason is that plug-able components is at incompatible with idea of sharing node across multiple browsing contexts (tabs, applications) because they can end up with different setups which would prevent sharing.

   Argument could be made towards say sharing networking stack and allowing plugging stores. However that also has complications. For one tabs (on different origins) will not be able to access data that in the disk without tunneling it over the network. From UX perspective it also means there is no canonical place for seeing all the data.

   That being said, I do recognize value say in e.g in-memory store which is not persisted, which could be achieved differently e.g. DAG staging staging area (Something along the lines of Discuss: improving ipfs.add  #3033 (comment)) which could address use case without complicating node sharing.

2. I see node Buffer is used as canonical representation for data. Is that motivated by a convenience or some other reasons ?

   Current approach attempts to avoid bringing node baggage into browser, that is ArrayBuffer|ArrayBufferView captures all binary types (including node Buffer it being subclass of Uint8Array) and given that all native browser APIs do return those it appears that there is no good reason to demand Buffer in any of the user facing APIs (removing it from internals is more complicated).

[carsonfarmer]

I would say that, in the interest of sharing the node across multiple browsing contexts, a less modular design might be ideal. Additionally, while there are other considerations for the modular design (allows much smaller packages and dynamic imports if done properly), the bundle size savings are so large that the benefits from reuse would certainly be larger than the benefits of smaller one-off bundles. So to answer the question: not that important for use (though I would still advocate for a modular design in terms of its development).

As far as Buffer goes, that is purely convenience. All the existing tooling expects/supports it. But we (Textile) internally have been trying to move most of our code and APIs to Uint8Array. So happy to support that push here as well.

[oed]

Thanks for the discussion @Gozala. Here are details as we discussed :)

1. What are the essential configuration options for JS-IPFS in browsers ?

The only configuration option that we rely on is passing a rendezvous server as a swarm address. Previously this was the only way to connect to a rendezvous server, but a way to do this has been added: #2508 (comment)
Other than that we disable preload and bootstrap, in order to create the instance faster. This should however be less of a problem with the shared instance.

2. What subset of JS-IPFS API is used in browsers ?'

We use the following:

• ipfs.add
• ipfs.cat
• ipfs.dag
• ipfs.pubsub
• ipfs.libp2p

I believe that's all, but I might be missing something. The main reason we're using ipfs.libp2p is because of pubsub room. Important to note here is that this is only supported in js-ipfs and not in ipfs-http-client. This should probably be added to the readme of pubsub room as well in order to not confuse people. cc @achingbrain

Another thing to note is that we don't use garbage collection right now. This was only recently introduced into js-ipfs. Instead we assume that all data that we add to ipfs will just remain in storage, and sync it from the network if it's not there.

3. Why users are choosing to use JS-IPFS in main thread over worker thread ?

We are currently in a situation where our software package (3box) is run on multiple tabs across multiple domains. There are two main things that could be solved using worker threads:

• Multiple across tabs causes connectivity issues.
  If you have two instances of ipfs on the same doamin but in multiple tabs they will get the same PeerId. This causes issues when connecting to remote nodes (especially with pubsub) where only one of the nodes becomes fully connected. We haven't figured out exactly what causes this, instead our workaround involves creating a random PeerId every time a new instance is created.
• Sharing data across origins.
  We want to be able to share data across origins. Currently we rely on the data being available in the network, but this is less then ideal. Instead it would be nice if nodes on multiple origins could share block storage. We are currently working on a solution for this by putting the blockstore api into an iframe.

4. How does user priorities fall in regards to convenient API vs bare-bone API ?

Don't have strong opinions here. On the security on things like the pinning api on multiple origins I tend to lean towards the side of usability now rather than origin specific pins. This gives us the chance to experiment more.

5. What are other important things (not on the list) that JS-IPFS does or you wish it did for your browser use case ?

The question of codecs is important. Initially a requirement from our side would be that we can provide a custom codec to the worker. We are working on codecs for signed and encrypted data, which we will start using as soon as it's ready.