feature request - use reasoning_content to show loading state

[mostlygeek]

For streaming style chat requests inject loading messages / state into the chat stream so there’s no “dead air” when loading.

It may cause issues that llama-swap will have to be more content aware as it will have to suppress things like headers into that the upstream may be sending.

See: #326 (reply in thread)

[mostlygeek]

@ServeurpersoCom

Thanks for this awesome idea. Implemented now.

llama-swap-issue-366.mp4

[ServeurpersoCom]

Tested sendLoadingState swap speed is now identical to my swap.js POC!
The snarky loading messages are gold ("The model forgot where it put its keys..." 😂), and performance-wise it's perfect. Let's see if stability holds up over time, will monitor for crashes.

[mostlygeek]

@ServeurpersoCom thanks for testing for stability. I created a custom http ResponseWriter and usually an edge failure is clients disconnecting in the middle of streams.