maxTokenAllowed

[vavanv]

Describe the bug

AI_APICallError: This model's maximum context length is 65536 tokens. However, you requested 66008 tokens (58008 in the messages, 8000 in the completion). Please reduce the length of the messages or completion.

I am getting this message even though there is a configuration in deepseek.ts
staticModels: ModelInfo[] = [
{ name: 'deepseek-coder', label: 'Deepseek-Coder', provider: 'Deepseek', maxTokenAllowed: 8000 },
{ name: 'deepseek-chat', label: 'Deepseek-Chat', provider: 'Deepseek', maxTokenAllowed: 8000 },
];

Link to the Bolt URL that caused the error

local

Steps to reproduce

after 2-3 cnah messages in bolt.diy

Expected behavior

no messages

Screen Recording / Screenshot

n/a

Platform

• OS: [e.g. macOS, Windows, Linux]
• Browser: [e.g. Chrome, Safari, Firefox]
• Version: [e.g. 91.1]

Provider Used

No response

Model Used

No response

Additional context

No response

[Bortus-AI]

Same issue. Code needs to be updated to drop some the context tokens to free up enough for a response.
We also need an option in settings to change the max tokens allowed per model. I am getting very small responses from Grok and other models that have high max token limits. Its truncating them to 600-1000 tokens when it should be able to provide me the full 6000 token response.

[leex279]

Context Optimizations coming with #1091 and more planned. So all already in planning / known problems

=> Closed. If you want to discuss/investigate more, please use the community, as its made for it and much more active people

https://thinktank.ottomator.ai/c/bolt-diy/bolt-diy-new-features-discussion/21