Edit: I realized I have the ChatAccordion and ChatStatusCard reversed below in the design, i.e. ChatCard should wrap ChatAccordion

The design should also mention how to integrate this functionality seamlessly in callback. My initial thought is yielding/returning a dict with intermediate=True {"The text to stream", "intermediate": True} creates a ChatMessage with these chat status cards. Maybe context manager too:

def callback(contents, user, instance):
    with ChatCard(title="Thinking...") as cc:
        yield cc
        cc.append(...)
        cc.append(...)
        cc.title = "Completed result"

--

To improve user experience while LLMs are performing intermediate steps, I'd like to propose two new chat components, namely, ChatAccordion and ChatStatusCard.

Unlike streaming text, character by character, these two components will improve perception of runtime by outputting each intermediate steps as ChatStatusCards.

Here's an example:

The LLM could output a step by step plan to achieve the outcome:

• as each step is determined, a new ChatStatusCard is appended to ChatAccordion
• as each step is completed, the results are nested under the ChatStatusCard which users may open at any time to view
• The status emojis will change from yellow to green if succeeded or red if failed, and perhaps another emoji if needs user input to continue

Once completed, it'll show the final result at the bottom.

The intermediate steps can also be hidden.

Here's the abstract version:

I believe the API would look like:

class ChatAccordion(Accordion):

    objects = param.List()

    status = param.Selector()

    result = param.Parameter()

    def send()  #  to match feed
        ...

    def clear()
        ...

class ChatStatusCard(Card):

    objects = param.List()

    status = param.Selector(...)

    default_emojis = param.Dict()  # maybe?

At the current moment, I'm having a hard time to figuring out:

• how to stream text inside the ChatStatusCard
  • do we use ChatMessage as an object inside without an avatar and all the other things
  • do we add yet another component ChatText for streaming text?)
• the names of these components; I don't really like them

Any feedback appreciated