Offline support

[sorin-davidoi]

We could leverage the Service Worker API to make the book available offline.

Brief outline:

• Off by default
• Add option to make available offline
• Cache all JavaScript and HTML files in the Service Worker
• Handle offline state in the UI
• Handle update logic

The main question is - would this be worth it? If I understand correctly, The Rust Book is bundled with Rust (or packaged in some distributions), so most of the people interested in it you have it locally. But then again, it might be nice to provide this regardless.

[Michael-F-Bryan]

How would an offline version work? My understanding is that it'd have to cache all the pages in the book and if the user goes offline we serve pages up from the cache. That'd involve the user effectively trying to download everything the moment they visit a book and the service worker starts up in the background, wouldn't it? That's probably not a good idea for something as big as The Book or Rust By Example...

My opinion is that if it's easy enough to implement and maintain, then I'm all for it. I've (briefly) played around with Service Workers and found they can have issues with updating and caching. That could just be me being a noob though.

[sorin-davidoi]

We could make it configurable, such that the user makes an active choice regarding which chapter should be available offline. My idea would be to don't cache anything by default and show a kind of unobtrusive action the first time the user loads the page, prompting to make it available offline (if its small enough, otherwise just the first few chapters).

[Michael-F-Bryan]

We could make it configurable, such that the user makes an active choice regarding which chapter should be available offline.

I don't know if this would work in practice. People reading a book probably won't want to spend a while configuring how they read the book, and I'm quite conscious of how many buttons and options we add to the UI.

[sorin-davidoi]

Yes, I guess you're right. Could we then try to cache as much as possible?

[Michael-F-Bryan]

I just checked and the user guide itself is already 1.5M (644K when compressed), so storing the entire book may be a bit heavy. If you're only counting the HTML though, it drops to 284k (uncompressed).

What are your thoughts?

[sorin-davidoi]

Well, it has to be HTML + JS, but I think that it is perfectly reasonable - this PWA caches entire videos in the Service Worker, so I don't think we should hit major roadblocks.

We could even compute, at compile time, how much we need to store and show that to the user. I imagine a small banner at the bottom, something like: "Want to make available offline (2.45 MB)?"

[Michael-F-Bryan]

yeah I guess that's feasible, although we'd need to make sure the notification collapses down or hides when the user isn't interacting with it.

How does cache invalidation work? At the moment your typical dev experience is to run mdbook serve on your computer and then it'll livereload as you're editing. If the service worker gets in the way of that livereload, or if it tries to download the entire book a dozen times a minute (because you typed a couple words, hit save, then typed some more, all in quick succession), then that's gonna be a problem...

We can also use book.toml to tell the HTML renderer whether to include the service worker.

[sorin-davidoi]

Service Workers are usually disabled in the development environment to avoid the issues you've mentioned. Not sure how that would work in this case.

This is a great idea. The service work acts as a cacher and updater from a central place like a s3 bucket, etc.

Does stdweb provide Service Worker scaffolding ?
https://github.com/koute/stdweb

[jasonwilliams]

@Michael-F-Bryan having something like The Book available offline is exactly the sort of thing service workers were built for. We should certainly do this.
we could either do it automatically (users can easily clear their cache) or we can add a “store for offline” button. I don’t think 1.5M is heavy but that’s subjective.

@sorin-davidoi what’s the update on your PR?

Linking: rust-lang/rust#20866 (comment)

[Michael-F-Bryan]

@jasonwilliams I agree that it'd be super useful and would be another step towards making mdbook more convenient to work with offline.

We just need to keep in mind that a book isn't always backed by a server, so something like mdbook build --open should continue to work and not care about service workers. From memory there are some general issues with running JavaScript from a file://... URL and using it to fetch documents from the file system.

I support the idea, we just need to be mindful of how it's implemented so it'll integrate well with the current workflow. Off the top of my head, this would require we

• make sure books are still usable without a server
• not use exorbitant amounts of bandwidth/memory
• ensure the service worker cache gets cleared automatically during development (clearing the browser cache and restarting/updating workers manually is a pain)

[sorin-davidoi]

@jasonwilliams No updates, there were some issues that needed to be handled before implementing this (see #571).

[jasonwilliams]

@sorin-davidoi what are the issues?
Is there anything i can pick up from this to carry on the work you were doing? It looks like this has just come to a stop