Support web workers with alternative to localStorage (IndexedDB ?)

[FunMiles]

I have a Progressive Web App (PWA) that maintains a local copy of a user's data in an IndexedDB database. This app uses Firebase and I am trying to switch to Parse as it is not only open source, but would allow easier queries directly from the database than what Firebase supports.
The synchronization between the local copy and the cloud database is done in a web worker as to not impact the UI performance. When implementing the Parse code in the web worker, I discovered an annoying stumbling block: Parse-SDK-JS keeps a record of the logged in user, session and other data using localStorage. However localStorage is not available in web workers. The documentation mentions some mechanisms used with react-native that relies on Parse.setAsyncStorage(StorageController);. However trying to implement my own StorageController, not from react-native, I discovered that this is not the right controller that I needed to do the first Parse operation. I had to use (and didn't find the doc about it, I had to read the code to find out the details of the call sequence that still used localStorage) Parse.CoreManager.setStorageController(StorageController);

It would be good to have an easier mechanism to handle storing the data in a way that can cover all cases.

The solution I'd like
At minimum, the documentation needs to explain how to cover the web-worker case. However, though I can't speak for the react-native situation, IndexedDB is supported by both the browser side and the web-worker side. So substituting indexedDB to localStorage would be a solution covering both sides without any more consideration for a first-time user.

** Alternatives I've considered**
There are two alternatives I can see that could address the issue:

• Provide an alternative StorageController in the library for those who want to use Parse from web workers and provide the documentation on how to activate it correctly.
• There is a setLocalDatastoreController method, but it is rather obscure if it could help here. The comments for it mention the use of setAsyncStorage instead of this method. Shouldn't all this be unified so that a single customization point of local storage could cover all the uses of a local storage mechanism?

Additional context
The documentation of the various storage objects, their use, and inter-relations should be improved.

[dplewis]

I agree that the CoreManager should be documented properly. A lot of the SDK is modular to allow for different adapters.

[FunMiles]

As a quick note, and possibly as a basis for a PR, here is my adapter using IndexedDB. The code for the database creation is quickly extracted from my more complex code. It may have errors if I did that too quickly, but the gist is there.

const DATABASE_NAME = 'ParseData';
const DATABASE_VERSION = 1;

export const dbPromise = openDB(DATABASE_NAME, DATABASE_VERSION, {
  upgrade(db, oldVersion, newVersion, transaction) {
    console.log("Upgrading to %d from ", newVersion, oldVersion);
    switch (oldVersion) {
      case 0:
        // a placeholder case so that the switch block will
        // execute when the database is first created
        // (oldVersion is 0)
        if (!db.objectStoreNames.contains('parseStorage'))
          db.createObjectStore('parseStorage');
        break;
      default:
        console.error('unknown db version');
    }
  }
});


export const IDBStorageController = {
    async: 1,

    async getItemAsync(path:string) {
        return (await dbPromise).get('parseStorage', path);
    },
    async setItemAsync(path: string, value: string) {
        return (await dbPromise).put('parseStorage', value, path);
    },
    async removeItemAsync(path:string) {
        return (await dbPromise).delete('parseStorage', path);
    },
    async getAllKeysAsync() {
        return (await dbPromise).getAllKeys('parseStorage');
    },
    async clear() {
        return (await dbPromise).clear('parseStorage');
    }
};

With this imported, the following is my initialization code which allows everything to run smoothly in a web-worker:

Parse.initialize(Env.APPLICATION_ID, Env.JAVASCRIPT_KEY);
Parse.serverURL = Env.SERVER_URL;
Parse.CoreManager.setStorageController(IDBStorageController);

PS: As the IDBStorageController is async, there may be a few adjustments to make. For example, to get the current user, the async version Parse.User.currentAsync() must be used instead of the synchronous version.

[dplewis]

@davimacedo What do you think? I'm not too familiar with IndexedDB or the database creation.

[FunMiles]

Compared to localStorage, IndexedDB has the advantage that it is available in both the browser side and the worker side. Its disadvantage is that it is asynchronous. Personally I do not see that as a big issue because I don't think that, apart from toy applications, any full application can be written only with synchronous calls. IndexedDB behaves like a key-value storage with the kind of code I gave above.
I wouldn't mind doing a PR. But, though I've contributed to several C++ open source projects, this would be the first time I'd do something in JS. I must say I am a bit intimidated with that. So maybe I'd rather someone else takes the above code, and put it in the right place and add documentation.

For more in depth solutions of customization, I've only started using parse a week ago, after looking for an alternative to firebase, so I don't think I'm qualified to get into that yet.

[dplewis]

I don't see an initial problem with adding a generic implementation into the project. I wanted to have a discussion on the best way to implement this.

1. What will users do if the browser doesn't support IndexedDB, or would we make a point of this being web worker only.
2. Would a generic way to open the database and create an objectStore be enough or will users want to customize it in the future.

[FunMiles]

Here are my thoughts:

1. Support can be an issue. Searching on IndexedDB support across browsers seems to show it is well supported in modern browsers, though there are comments about bugs in MS's Edge browser (no surprise there 🙄 ). I was thinking that using the same storage across both the browser side and the worker side may allow to have one side handle the login and being able to talk with the parse server from both sides without re-logging in... It is possible to handle the login on one side and pass the token to the other as well, so it is not an absolute necessity, but it'd be nice to have in order to minimize the head-scratching of howtos for new users.
2. I am not sure what customization points are desirable... From my experience, it seems that the most important part would be to have something that has a good default so that it simply works out of the box on all tested browsers. One possible simple customization might be to cache the data for the last few keys retrieved/stored in a memory object. There are comments that in Edge a few years back, IndexedDB was blocking fairly long time even though it is supposed to be async.

I just installed Edge on my mac to test. What I have done works for me on:
Chrome 87; Mac OS
Safari iOS 14.0.1; Safari Mac OS 14.0.1
Safari iOS 12.4.8;
Edge 87.0.664.57 on Mac OS.
Firefox 83.0 on Mac OS.

If any browser support issue appears, I think it might be in the more corner places. I've read that (past?) version of MS browsers do not support the getAllKeys. So some kind of test/fallback to retrieve the keys might be good. (It does not look like this function was being used in my tests, but it is there, so I presume there are some use cases).

[dplewis]

it'd be nice to have in order to minimize the head-scratching of how to for new users.

I agree, feel free to open a Pull Request with tests and update the Readme. We'll gladly review it.

[FunMiles]

@dplewis I would love to, but as I mentioned a bit earlier, I am quite intimidated with this. Particularly the tests. I'm used to tests in C++ code but the tests in this project, though I can guess what they do, I don't know how to write and run them. I'm only running my code with the npm installed library.
Is there a guide to properly adding such a feature here?

[dplewis]

You can check out the Contributing Guide to get started with the tests. There are a lot of test examples you can use from the existing StorageControllers. Let us know if there is anything you need help with.