Discussing storage access for Quicktext

[jobisoft]

Let us use this issue for a general discussion on how to solve the storage access. I know some of you want to keep storing on the local file system, others need to sync their templates.

Here is a proposal. Please play with it in your heads, to find edges cases where it might fail.

The Thunderbird Team is thinking about creating a general storage virtual file system API (VFS API), similar to the cloudFile API, where other developers can create storage providers for seafile, nextcloud, Amazon, box and what not. These providers can expose one or more folders from their supported backend as a virtual file system, and add-ons can read and write inside that VFS.

One of the providers for the VFS API could also be a local-file-system provider.

The quicktext config UI will present a list with all installed providers for the user to select one:

• add-on webextension storage (default)
• file-system-access (folder xy)
• nextcloud (user xy, folder xy)
• ...

The templates and scripts will be loaded and saved according to the selected storage.

If templates use attachments or images, which are included from the local file system, they have the following source options in the future

• file : current default, will be deprecated and removed, templates have to be adjusted
• url : a remote url
• vfs-xy: the file path is interpreted as a path in the specified virtual file system

Quicktext should work without additional providers using only the WebExtension storage, but users can opt to use any storage they want.

Will this solve all the issues?

[SamuelPlentz]

I think thats a good idea. I don't see any problems beside increased complexity.

[jobisoft]

Remember: The issue with accessing the user's file system is: It is not future-proof. It is currently only possible via legacy Experiment add-ons, and we do not know how long those will be supported by Mozilla. So far, Mozilla has denied requests to create an official API to directly access the user's local file system.

Update 1: New Name for the API

I think the API should be called VirtualFileSystem API or just VFS API instead of storage API. This fits, even if the planned File System provider for the VFS API can map onto the real file system of the user.

Update 2: FilePicker

The API will have a picker method, which will open a custom file picker dialog. The user will be able to switch between the installed providers and browse through the available folders and select files for read/write access. The dialog will also have an import/export feature to copy files between the user's local file system and the selected VFS folder.

Update 3: Build-in provider(s)

Users should be able to use Quicktext without any additional provider installed. How can they select attachments and pictures for the IMAGE and ATTACHMENT tags? In order to re-use the same UI and file pickers, Quicktext could ship a VFS provider directly with the add-on, which will be a simple IndexedDB based implementation for the VFS API. It could be called Internal Quicktext Data.

Users would then have the choice to pick attachments and images to be used in templates from the installed providers, for example:

• Internal Quicktext Data (default)
• File System (Folder XY)
• NextCloud (Description XY)

Update 4: A way forward for a premium services

In addition, we could create a premium Quicktext Cloud VFS provider, which is connected to a pre-defined cloud service, is Quicktext-branded, and would allow users who do not want to set up a Nextcloud instance on their own to easily sync their Quicktext data (which is feature request #54).

Update 5: Collection of use-cases

I am aware of the following use cases:

• Users do not need sync, they store templates in the local storage and use the built-in Local Quicktext Data VFS provider for attachments and images. Optionally, they can use the File System provider for the VFS API for attachments and images, if the content is changing often (this also works with the default Local Quicktext Data VFS, but the files have to be updated in the VFS before attaching the "new" version, which is not needed if the real file system is mapped directly to the VFS)

• Fixed templates are pushed to employees via managed storage. Employees can create custom templates which are saved in local storage.

• Templates are synced to the user's disc, for example via the nextcloud client, and Quicktext should operate on these files. Collision handling is done by the nextcloud client. The UI needs a storage selector for the primary template and script storage. Default is WebExtension local storage, but users can install and select the File System provider for the VFS API and select the folder synced by the nextcloud client.

• Quicktext should operate on a cloud service directly. Users can install a Nextcloud provider for the VFS API and select it as the primary storage for templates and scripts. Quicktext directly interacts with the remote storage, collision handling is then done by the provider. This is future-proof, compared to the local sync option.

• The defaultImport field can be used to allow users to pick default imports as needed. Instead of using managed storage to push all allowed templates, employees can use the nextcloud VFS provider to select any of the granted templates as default imports. The defaultImport field will no longer be a text field, but a list, which will be managed by the file picker from the VFS API (this should cover Import on startup not working, Thunderbird 128.13.0esr, QT 6.4.3 #548)

[SamuelPlentz]

Ambitious and promising plans!

It would be helpful/important to provide a unified "file-access" / "cloud-x-access" / "cloud-y-access" / "cloud-z-access" / "..." interface, so that any Thunderbird plugin (not just Quicktext) can make use of it.

[jobisoft]

The next version will have this UI:

This brings back direct file system access for the general template storage and the imports. All "picker" buttons (+ Folder, + File, Attachment, Images) are again using the Legacy Experiment FilePicker and will be replaced with the VFS FilePicker, once the API is done. The goal is indeed that any add-on can use the VFS API.

[jobisoft]

The new UI and the storage switcher is available in
https://github.com/jobisoft/quicktext/releases/tag/v6.4.5

Feedback is appreciated.

[RayCulp]

I just now discovered this new version... after setting up a local HTTP server to fool my old 6.4 version into thinking I was loading local template files from the Internet. Serves me right for installing from the official Thunderbird extensions page lol. XD

I like this UI and the underlying functionality very much. The ability to load multiple template files from different locations at startup brings back the essential modularity.

[RayCulp]

Speaking of HTTP and syncing template files from a "remote" server, I discovered something interesting while I was trying to fool version 4.6 into working with local template files.

Thunderbird caches a template file that you sync via HTTP just as it would any other remote content.

If you make a change to the remote template file, Thunderbird / Quicktext may not show those changes if you just restart. You need to clear the cache first, then restart. I fixed this by activating the option "Clear cache on shutdown".

Obviously this is unnecessary if you're loading local templates using a newer version of Quicktext, like 4.6.7. But if anyone is actually pulling template files from a server on the Internet, this might be helpful information.