[Bug]: Restoring Shiny inputs does not happen within ui.Chat

[schloerke]

Related: #1958

Component

UI (ui.*)

Severity

P2 - Medium (workaround exists)

Shiny Version

1.4.0

Minimal Reproducible Example

from chatlas import ChatOllama

from shiny import reactive
from shiny.bookmark import RestoreState
from shiny.express import input, session, ui

chat_model = ChatOllama(model="llama3.2")


ui.page_opts(
    title="Hello input bindings in Chat",
    fillable=True,
    fillable_mobile=True,
)

welcome = ui.TagList(
    """**Hello! How would you like me to respond?**""",
    ui.input_select("tone", "", choices=["Happy", "Sad"]),
)

chat = ui.Chat(id="chat", messages = [welcome])
chat.ui()

# # Workaround
# @session.bookmark.on_restore
# def _go_through_messages_and_update_inputs(state: RestoreState):
#     ui.update_select("tone", selected=state.input["tone"])
#     # ui.update_select("tone1", selected=state.input["tone1"])
#     # ui.update_select("tone2", selected=state.input["tone2"])
#     # ui.update_select("tone3", selected=state.input["tone3"])

chat_model = ChatOllama(model="llama3.2")

chat.enable_bookmarking(chat_model, bookmark_store="url")

@reactive.effect
@reactive.event(input.tone)
def _():
    chat_model.system_prompt = f"""
    You are a terse and {input.tone()} assistant.
    """


@chat.on_user_submit
async def _(user_input: str):
    stream = await chat_model.stream_async(user_input)
    await chat.append_message_stream(stream)

Behavior

The message being added to ui.Chat() is serialized on it's way to the browser. This message may contain already processed Shiny ui elements.

This serialized value is what is stored when bookmarking. We currently believe this is the best approach as we can not automatically recreate the UI messages without a user provided function.

However, restoring static HTML will not dynamically restore input values within the bookmark state. This leads to unexpected behavior and confusion as the rest of the app has bookmark support.

TLDR: If the UI does ANY transformation of a Turn content, when we restore the Turns state the transformed information will be lost.

• Ex: A Turn containing a shiny input is updated.
• Ex: All text is multiplied within the browser according to a side shiny input
• Ex: Generators causing side effect with other Generators (ex: capitalization of an inner generator).

Work arounds

User:

• The user provides all messages containing Shiny inputs within chat.ui(messages=). As the UI is sent to the user, it will have the restore context to restore all inputs during creation.
• The user provides @session.bookmark.on_restore method to manually restore all known input values (within chat messages).

Solution

We update ui.Chat().ui() to save all input values if bookmarking is enabled to a local JS variable. Add a listener to inspect all DOM elements under #CHAT and if any existing elements are shiny inputs or any elements are discovered via shiny event for input bound, call el.data("shinyInputBinding").setValue(el, value) on the shiny input.

Once the value is utilized, similar to how Shiny input values are reset on flush, remove the input value from the set to avoid restoring the same value multiple times.

Ex: $("#tone").data("shinyInputBinding").setValue($("#tone").get(0), "Sad")

Add this via an inline HtmlDependency script tag.

---

Implementation Facts:

• We have a input["tone"] value
• chat.ui() uses the restored inputs to construct the init UI
• We do not have a "i see an ID for FOO with value BAR, please update yourself given this value" for UI components