Persisting our Redux state to browser storage (IndexedDB) allows us to avoid completely rebuilding the Redux tree from scratch on each page load and to display cached data in the UI (instead of placeholders) while fetching the latest updates from the REST API is still in progress.
Note that the entire Redux state is not persisted to the browser. In order to persist state in browser storage the reducer must be wrapped with withSchemaValidation as instructed below.
This feature was originally implemented in #2754.
At a high level, implementing this is straightforward. We subscribe to any Redux store changes, and on change we update our browser storage with the new state of the Redux tree. On page load, if we detect stored state in browser storage during our initial render, we create our Redux store with that persisted initial state. However, significant issues exist that require special solutions:
- Subtrees may contain class instances
- Data shapes change over time
- Some reducers are loaded dynamically
The implementation details for theses solutions are discussed in detail below.
Note that we opt-in to persistence simply by wrapping the reducer with withSchemaValidation.
withSchemaValidation returns a wrapped reducer that validates on deserialize() and returns
initial state on deserialize() when the state doesn't match the schema. Implementation of custom serialize() and deserialize() to handle subtrees with class instances is discussed below.
In Calypso, we combine all of our reducers using combineReducers from state/utils at every level of the tree instead
of the default implementation of combineReducers from redux.
The custom combineReducers handles persistence for the reducers it's combining.
To opt-out of persistence we simply combine reducers without any attached schema.
return combineReducers( {
age,
height,
} );To persist, we add the schema by wrapping the reducer with the withSchemaValidation util:
return combineReducers( {
age: withSchemaValidation( ageSchema, age ),
height,
} );Some subtrees may choose to never persist data. One such example of this is our online connection state. If connection values are persisted we will not be able to reliably tell when the application is offline or online. Please remember to reason about if items should be persisted.
However we quickly run into the following problems:
Subtrees may contain class instances. In some cases this is expected, because certain state subtrees have chosen to use Immutable.js. Other subtrees use specialized classes like QueryManager whose instances are stored in Redux state. However, IndexedDB storage requires that objects be serialized and thus attempting to store a class instance in IndexedDB will throw an error. We must create a custom solution to serialize these classes before saving to IndexedDB.
#### Solution: custom serialize and deserialize methods
To work around this we can assign two special methods to the reducer function: serialize and deserialize. They are called either to prepare state to be serialized to browser storage, or to deserialize persisted state to an acceptable initialState for the Redux store.
serialize( reducer, reduxStore.getState() );and
deserialize( reducer, browserState );Because browser storage is only capable of storing simple JavaScript objects, the purpose of the serialize method
on the reducer is to return a plain object representation.
In turn, when the store instance is initialized with the browser storage copy of state, you can convert
your subtree state back to its expected format by the deserialize method.
In a subtree that uses Immutable.js, we serialize to a plain object and deserialize into an Immutable.js instance:
const items = withPersistence(
( state = defaultState, action ) => {
switch ( action.type ) {
case ACCOUNT_RECOVERY_SETTINGS_UPDATE:
return; // ...
default:
return state;
}
},
{
serialize: ( state ) => state.toJS(),
deserialize: ( persisted ) => Immutable.fromJS( persisted ),
}
);If your reducer state can be serialized by the browser without additional work (e.g. a plain object, string or boolean),
the serialize and deserialize methods are not needed. However, please note that the subtree can still see errors
from changing data shapes, as described below.
Problem: Data shapes change over time ( #3101 )
As time passes, the shape of our data will change very drastically in our Redux store and in each subtree. If we now persist state, we run into the issue of our persisted data shape no longer matching what the Redux store expects.
As a developer, this case is extremely easy to hit. If Redux persistence is enabled and we are running trunk, first allow state to be persisted to the browser and then switch to another git branch that contains minor refactors for an existing sub-tree. What happens when a selector reaches for a data property that doesn't exist or has been renamed? Errors!
A normal user can hit this case too by visiting our website and returning two weeks later.
How can we tell that our persisted data is good to use as initial state?
Before we can detect data shape changes, we need to be able to describe what our data looks like. To accomplish this, we use JSON Schema. JSON Schema is a well-known human and machine readable format that defines the structure of JSON data. It is also easily adapted for use with plain JavaScript objects.
A schema file schema.js is added at the same level of each reducer. Our schema should aim to describe our data needs,
specifically: what the general shape looks like, which properties must be required, and what additional optional
properties they might contain. Ideally, we should try to balance readability and strictness.
A simple example schema.js:
export const itemsSchema = {
type: 'object',
patternProperties: {
'^\\d+$': {
type: 'object',
required: [ 'ID', 'name' ],
properties: {
ID: { type: 'number' },
name: { type: 'string' },
description: { type: 'string' },
},
},
additionalProperties: false,
},
};A JSON Schema must be provided if the subtree chooses to persist state. If we find that our persisted data doesn't match our described data shape, we should throw it out and rebuild that section of the tree with our default state.
You can use withSchemaValidation to wrap a plain reducer, passing the schema as the first param, and all
that will be handled for you.
export const items = withSchemaValidation( itemsSchema, ( state = defaultState, action ) => {
switch ( action.type ) {
case THEMES_RECEIVE:
return; // ...
default:
return state;
}
} );If you are not satisfied with the default handling, it is possible to further wrap the inner reducer with
withPersistence and implement your own serialize and deserialize methods to customize data persistence.
Always use a schema with your custom methods to avoid data shape errors.
Dynamically loaded JS modules can add new reducers to the existing state tree. The state tree shape is therefore not the same at all times. The initial reducer can be small and new reducers can be added as the user navigates to new parts of the app and new code modules are loaded at runtime.
If we persist the state tree as one monolithic object, we run into trouble. To deserialize and check a stored
state subtree against a JSON schema, the corresponding reducer needs to be loaded and available.
It's therefore not possible to load such a state subtree during Calypso boot.
A reducer for a state subtree can have a storageKey property that is added using the withStorageKey helper:
const readerReducer = withStorageKey(
'reader',
combineReducers( {
feeds,
follows,
streams,
teams,
} )
);When this storageKey property is encountered when dispatching the SERIALIZE action, the result of the serialization
will be an instance of the SerializationResult class that contains two serialized objects. One for root key, with the state that doesn't have a storageKey set,
and another one for reader key. Both objects will be stored as two distinct rows in the IndexedDB table.
When booting Calypso, we initially load only the root stored state. The reader key is loaded and deserialized only
when the reader reducer is being added dynamically.