Closed
Description
Describe the bug
If generateObjectV2
is used, the passed modelClass is not processed appropriately and would produce a bug.
trimTokens receive the parameters models
which is expected to be strings like gpt-4o
. However, in generateObjectV2, modelClass, which is supposed to be small
, "medium" or "large" is provided.
context = await trimTokens(context, max_context_length, modelClass);
export function trimTokens(context, maxTokens, model) {
// Count tokens and truncate context if necessary
const encoding = encoding_for_model(model as TiktokenModel);
let tokens = encoding.encode(context);
const textDecoder = new TextDecoder();
if (tokens.length > maxTokens) {
tokens = tokens.reverse().slice(maxTokens).reverse();
context = textDecoder.decode(encoding.decode(tokens));
}
return context;
}
Screenshots

Additional context
Activity