State sharing + POC on simulation config

[jankrepl]

Background

The idea here is to introduce some shared state between the backend (LLM) and the frontend that they can both GET and PATCH.

What exactly is a state? It is just a (potentially very nested) JSON

{
    "simulation_config": {...}
}

For now the only key is simulation_config (POC for this PR) but in the future there will be other keys. The final goal of this PR is to allow for bidirectional state updates/fetches

• backend-> frontend When the "obi-one-generatesimulationsconfig" tool executes we want the frontend form to automatically update with the suggested changes (no copy pasting).
  • Note that we already have a similar mechanism in our frontend (also the official one) since vercel streams tool outputs so the frontend can literally just check what tool was called and conditionally perform actions (e.g. display plot)
• frontend->backend When someone manually modifies the frontend form we want the LLM to have access to the latest state.
  • We have a mechanism that does that here: Context analyzer tool #433 . We force the frontend to send frontend_url with every request to qa/chat_streamed

Inspiration

There seem to be some projects/frameworks that introduce the concept already. See an example below

• github repo: https://github.com/ag-ui-protocol/ag-ui
• tutorial video: https://youtu.be/0YnMZQ3UiSo?t=874

Now, in this PR I would maybe first think of doing this without a framework - but feel free to disagree. Also, the reason why I am saying this is that our backend is in Python and our frontend is in Next.js.

TODO

• Read through docs of the ag-ui tool. Especially state management: https://docs.ag-ui.com/concepts/state. And if you think we should use the framework (or any other framework) instead of doing this from scratch speak up.
• (Optional) Adjust our frontend to demo this feature. Use zustand to store the whole state and maybe create a /state page that just shows the value. The goal is to show that calling the tool will lead to changes in the state ( backend -> frontend direction)
• Implement a POC from scratch. I could think of 2 general approaches and they differ in where the "ground truth" state is stored.

Approach 1 (Frontend has the ground truth state)

TLDR it is basically what we do with the frontend_url but we also force the frontend to send us the whole state in the request body.

Approach 2 (Backend has the ground truth state)

Note that the get_state and set_state function will require the Session from SQLAlchemy so we we will have to insert it to our context variables. This would be the first time we allow any tool to connect to our DB.

Comments

• Using the diff (json patch) instead of the full snapshot should not overload the LLM (see explanation below). However, if we want to ask the LLM things like "How does the state look like" maybe we should implement a separate tool get-state that will just dump the whole state inside the tool output.
  • Why not full JSON (snapshot)? The docs of ag-ui talk about the JSON snapshot vs delta. They recommend the delta using the JSON patch standard which IMO is clever and makes it less likely for the agent to mess up (way fewer tokens in the tool output)
  • Python helper package: https://github.com/stefankoegl/python-json-patch/tree/master
  • JS helper package: https://www.npmjs.com/package/fast-json-patch

[evakennyobi]

Update: nearing completion from the ML side

[evakennyobi]

To get input from the frontend team, meeting to discuss next week

[evakennyobi]

New tickets will be created to connect chat & to implement the proposal, I close this