Error handling for too-large-requests

[mrsimpson]

Motivation

A lot of frustration, particularly when using local LLMs with bolt.diy is that the requests become too large for the LLM to handle. The LLM then forgets the instructions from the system prompt (e. g. how it should format artifacts). Consequently, the workbench cannot properly react on the response, which majorly results in files not being created, but also mor subtly issues like semantically bad responses.

Scope

Make it transparent to the user that the LLM could not properly respond to a request
if this is the case, do not execute any actions (as this is likely rubbish)
Ideally, we could provide guidance for the user what he could do

Options

• Inject a "protocol guard" into the very beginning of the system prompt (e. g. wrap your complete reply with <boltComplete>)
• Utilize the models metadata

Related

• Files context reduction #1041