Saving and loading GPT-tfjs models with IndexedDB fails with error

[JulienVig]

tfjs fails to load a GPT-tfjs model previously saved to the browser's IndexedDB memory:

TypeError: Cannot read properties of undefined (reading 'push')
    at new CausalSelfAttention (layers.js:66:22)
    at fromConfig (@tensorflow_tfjs.js?v=27f417ed:12618:12)
    at deserializeKerasObject (@tensorflow_tfjs.js?v=27f417ed:19329:25)
    at deserialize (@tensorflow_tfjs.js?v=27f417ed:22696:10)
    at processLayer (@tensorflow_tfjs.js?v=27f417ed:24102:21)
    at fromConfig (@tensorflow_tfjs.js?v=27f417ed:24116:7)
    at deserializeKerasObject (@tensorflow_tfjs.js?v=27f417ed:19329:25)
    at deserialize (@tensorflow_tfjs.js?v=27f417ed:22696:10)
    at processLayer (@tensorflow_tfjs.js?v=27f417ed:24102:21)
    at fromConfig (@tensorflow_tfjs.js?v=27f417ed:24116:7)

The code triggering the error is the updateWorkingModel method of discojs-web memory:

await model.extract().save(indexedDBURL, { includeOptimizer: false })
const loadedModel = await tf.loadLayersModel(indexedDBURL); // raises an error

which is called at the end of each training epoch.

Ideally, we would also like to save the optimizer but saving fails when including it. (which is not surprising given that the optimizer is re-implemented manually)

Overall, the GPT-tfjs models have not be integrated to the browser memory framework, which currently assumes that all models are TFJS models.