Origin Private File System (OPFS)

[tomayac]

Description

The origin private file system is a storage endpoint that, as the name suggests, is private to the origin of the page. While browsers typically implement this by persisting the contents of this origin private file system to disk somewhere, it is not intended that the contents be easily user accessible. Similarly, there is no expectation that files or directories with names matching the names of children of the origin private file system exist. While the browser might make it seem that there are files, internally—since this is an origin private file system—the browser might store these "files" in a database or any other data structure.

Rationale

Getting access to the local file system for building reliable local storage has been identified as one of the core developer needs all the way back in 2019 in the MDN Developer Needs Assessment. The OPFS is what powers, among other things, Photoshop on the Web. Developers from the database community highly anticipate the underlying SyncAccessHandles API as a way to Bring-Your-Own-Database (BYOD) to the web. From the developer of DuckDB:

“Now that we have a WebAssembly version of DuckDB running, we realized that this API proposal here holds a tremendeous(!) potential in the form of locally persistent fast embedded databases in browsers.”

Since the release of Photoshop, the feature usage has increased significantly. As database developers such as SQLite and DuckDB migrate to the SyncAccessHandles API, we expect the BYOD use case to increase dramatically in popularity and likely replace some IndexedDB usage.

The feature is currently supported by Chrome and Safari.

Specification

File System—Living Standard

Tests

wpt / fs

[gsnedders]

Tests

wpt / fs

Currently these all fail in Safari, almost all with:

UnknownError: The operation failed for an unknown transient reason (e.g. out of memory).

AFAIK, this is a limitation with the anonymous storage used by Safari under automation (i.e., under WebDriver), so these results aren't representative of Safari's implementation.

[jesup]

Most of the code for OPFS is in Firefox behind a pref; WritableFileStreams, postMessage and quota support is landing RSN, after which it will be enabled on Nightly. With WritableFileStream & postMessage we pass almost all the WPTs (some of the WPTs test implementation artifacts of Chrome; some others are for issues currently being hammered out in the spec).

[foolip]

@gsnedders are those failures something you expect could be fixed by either changing how the tests work on some change to Safari under automation?

[foolip]

In the MDN short survey on APIs & JavaScript, "File System (navigator.storage.getDirectory())" was selected by ~27% of survey takers, putting it in the top third of options. (There is some uncertainty as with any survey data.)

[gsnedders]

@gsnedders are those failures something you expect could be fixed by either changing how the tests work on some change to Safari under automation?

Pretty sure the only path here is for Safari to change behaviour; this is tied to how ephemeral sessions work (so Automation, Private Browsing, etc.)

[nairnandu]

Thank you for proposing the Origin Private File System for inclusion in Interop 2023.

We wanted to let you know that this proposal was not selected to be part of Interop this year. As discussed in the issue comments, we found that our testing infrastructure is not capable of testing this technology in all three browser engines. Resubmitting a proposal when WPT testing infrastructure is improved would be welcome.

For an overview of our process, see the proposal selection summary. Thank you again for contributing to Interop 2023!

Posted on behalf of the Interop team.

[Prid13]

it is not intended that the contents be easily user accessible

Why is that the case, if I may ask? :) It's always a bonus to decouple the actual data from any specific browser software, giving users the option to view their stored data even outside of the browser context. This is something I'm struggling with right now since Chrome stores extension data as .ldb files, and with WebSQL deprecated, which was the only option that had files on disk that could be read with any sqlite reader, it's almost impossible to easily and meaningfully extract any (extension) user data and view it outside of the browser :(

The use case here is a separate desktop app gaining access to that (offline) user data, so true offline apps and extensions can be developed that aren't dependent on cloud storage or sync. A simple example would be a notes app for quickly jotting down notes in Chrome, which also has a native desktop app that has access to those notes and can even add to them, all without requiring an internet-dependent middleware.

Sorry for this somewhat random post, but I saw that WebSQL was deprecated, and heard that IndexedDB and localStorage/chrome.storage weren't reliable/good enough options for storing user data locally without fearing things like limited space, eviction or corruption. If SQLite Wasm + Origin Private File System is the way to go forward, then there should definitely be a way to easily access that data without needing the whole browser, imo :)

[tomayac]

Please note that this isn't quite the forum for requests like this. Three quick links:

• Background on the OPFS: https://developer.chrome.com/articles/file-system-access/#accessing-the-origin-private-file-system
• Consider using SQLite as your storage backend: https://developer.chrome.com/blog/sqlite-wasm-in-the-browser-backed-by-the-origin-private-file-system/
• Debug the OPFS: https://goo.gle/opfs-explorer

[projektorius96]

Dear @tomayac, I'd like to ask the following.

Consider I have a local developer server established on localhost:5500 that I had been served files from directory A during session one (let's call it – SESS_ONE), but 5 minutes later I served different files from directory B on the same origin of localhost:5500, yet as a new session (let's call it – SESS_TWO).

My question is as follows: Does OPFS override (should I say overwrite?) the data saved during SESS_ONE the files served from directory A in favour of directory B persistence during new re-established SESS_TWO ? As far as I do understand OPFS provides a single bucket per origin, unless those buckets are IDed internally, otherwise OPFS is what says "per origin" and that specific origin should be reserved for particular directory N? Could you elaborate a bit further on, please . I tried to dig into specification, but could not find much about it. Plus we all know that depending on user-agent vendors the implementation can slightly differ. To underline, I am particularly interested in cases of Chrome / Microsoft Edge .

Cheers ♥

[tomayac]

@projektorius96 The name OPFS is a bit misleading, as it suggests there's just one per origin, which no longer holds true with Storage Buckets (even if this part is not implemented yet). The more correct way of referring to the OPFS would be Bucket File System. In the current implementation, though, even if the session changes and you serve the content from a different directory, as long as the origin stays the same, your files should stay the same.