add llm flush sentinel for llm_node

[longcw]

No description provided.

[github-actions]

✅ Changeset File Detected

The following changeset entries were found:

• patch - livekit-agents

Change description:
add llm flush sentinel for llm_node (#2023)

[davidzhao]

can we do this without modifying the entire chain? it seems this can be limited to the llm_node itself.. where once we detect it, close the next tts stream/transcription stream and then start another one.

that seems cleaner than having to propagate this down. user code in tts_node shouldn't have to care about it

[longcw]

can we do this without modifying the entire chain? it seems this can be limited to the llm_node itself.. where once we detect it, close the next tts stream/transcription stream and then start another one.

that seems cleaner than having to propagate this down. user code in tts_node shouldn't have to care about it

it's feasible to propagate to perform_tts_inference in agent activity, and handle the flush sentinel there by close and create tts_stream so the tts_node only see a text stream.