Tab storage requires serializable items

[ReubenVandezande]

Description

If I read the code correctly here , storage.tab should return an ObservableDict except that on handshake the tab storage is initialized as a persistent dict. Back in 2.9.0 I found it correctly uses an ObservableDict.

I was previously relying on this since I store non-serializable objects in the tab storage for caching. This contradicts serializable requirements.

Can we revert this behavior to match the documented behavior of storage? For now, I have to pin my dependencies to 2.9.0

[falkoschindler]

Hi @ReubenVandezande,

PersistentDict derives from ObservableDict:
https://github.com/zauberzeug/nicegui/blob/main/nicegui/persistence/persistent_dict.py#L6

So storage.tab should still be an ObservableDict. And indeed, it is:

@ui.page('/')
async def index():
    await ui.context.client.connected()
    ui.label('Yes' if isinstance(app.storage.tab, observables.ObservableDict) else 'No')

[ReubenVandezande]

Huh. Well, I guess I assumed that change came from storage as that's the only place where I cache something. Let me try and reproduce.

[ReubenVandezande]

Try this one. I expect this to work, but I get a TypeError:

from nicegui import app, ui


class CustomClass:

    def __init__(self):
        self.key = "value"


@ui.page('/')
async def index():
    ui.button('Update storage', on_click=set_non_serializable)


async def set_non_serializable():
    await ui.context.client.connected()
    tab_storage = app.storage.tab
    tab_storage['item'] = CustomClass()


ui.run(show=False, reload=False)

TypeError: Object of type CustomClass is not JSON serializable

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File ".venv/lib/python3.11/site-packages/nicegui/background_tasks.py", line 52, in _handle_task_result
    task.result()
  File ".venv/lib/python3.11/site-packages/nicegui/persistence/file_persistent_dict.py", line 40, in backup
    await f.write(json.dumps(self, indent=self.indent))
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".venv/lib/python3.11/site-packages/nicegui/json/orjson_wrapper.py", line 40, in dumps
    return orjson.dumps(obj, option=opts, default=_orjson_converter).decode('utf-8')
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: Type is not JSON serializable: CustomClass

[falkoschindler]

Ah, I see. Using the non-serializable builtin len we can boil it down to this:

@ui.page('/')
def index():
    ui.button('Update storage', on_click=lambda: app.storage.tab.update(x=len))

I think this comes from PR #4074 where persistent tab storage was introduced. Unfortunately, this breaks existing code like yours. @rodja Did we think about non-serializable tab storage? What was our reasoning behind this change?

[rodja]

Oh no. I totally forgot about that requirement. I stated it explicitly when creating app.storage.tab: #2837 (comment). But I guess we never wrote a test for that 😞.

For Redis storage this requirement can not be uphold: a reload in the same tab can be routed to a different instance. To make app.storage.tab work, it must be serializable.

A way to support both use-cases would be to check whether we are using Redis storage and then create a PersistentDict, otherwise an ObservableDict. Of course this would need to be mentioned in the docs.