-
Notifications
You must be signed in to change notification settings - Fork 1.5k
Description
Describe the bug
I'm in the process of updating some old code to the latest version of Chainlit. In the current version, the pulsating dot that indicates that content is being generated does not show up until at least on token has been streamed. This is a regression introduced in in 1.1.400, version 1.1.306 works as expected.
To Reproduce
In 1.1.306, below code would generate an empty response message and show a pulsating dot immediately:
import chainlit as cl
@cl.on_message
async def main(message: cl.Message):
await cl.sleep(3) # Simulate LLM call taking some time
await cl.Message(f"You said: {message.content}").send()
Starting with 1.1.400 it doesn't anymore. Now this seems somewhat reasonable because it is faking a response where there is none yet. So I have tried this instead:
import chainlit as cl
@cl.on_message
async def main(message: cl.Message):
response = cl.Message("")
await response.send()
await cl.sleep(3) # Simulate LLM call taking some time
await response.stream_token(f"You said: {message.content}")
await response.update()
I would expect at least this code to generate the pulsating dot in the response immediately, but it does not in versions later than 1.1.400. In 1.1.306 it does as expected.
Streaming an "invisible" token triggers the pulsating dot, but is really only a workaround that imho should not be necessary.
import chainlit as cl
@cl.on_message
async def main(message: cl.Message):
response = cl.Message("")
await response.send()
await response.stream_token("\u00A0") # Zero-width non-breaking space to force the pulsating dot
await cl.sleep(3) # Simulate LLM call taking some time
await response.stream_token(f"You said: {message.content}")
await response.update()
Expected behavior
A pulsating dot is shown to indicate to the user that a response is being generated and to avoid users thinking the application is broken because there is no feedback.
Screenshots
Chainlit 2.6.7, first code snipped:
Chainlit 2.6.7, second code snipped:
Chainlit 2.6.7, third code snipped. Note the gray dot next to the avatar, it would be pulsating if this was a video instead of a screenshot)
Chainlit 1.1.306, first code snippet. Note that this works exactly like the third code snipped in the latest version, but with less amount of code.
Desktop (please complete the following information):
Any browser, N/A
Smartphone (please complete the following information):
N/A
Additional context
N/A