Togetherai LLM error - Cannot read properties of undefined (reading 'choices')

[raducostea]

Checked other resources

• I added a very descriptive title to this issue.
• I searched the LangChain.js documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.
• I am sure that this is a bug in LangChain.js rather than my code.
• The bug is not resolved by updating to the latest stable version of LangChain (or the specific integration package).

Example Code

import {TogetherAI} from "@langchain/community/llms/togetherai";

async function test() {
    const llm = new TogetherAI({
        model: 'meta-llama/Llama-3.2-11B-Vision-Instruct-Turbo',
    });
    const response = await llm.invoke('Hello')
}
test();

Error Message and Stack Trace (if applicable)

node_modules/@langchain/community/dist/llms/togetherai.cjs:163
        const outputText = response.output.choices[0].text;
                                           ^

TypeError: Cannot read properties of undefined (reading 'choices')
    at TogetherAI._call (node_modules/@langchain/community/dist/llms/togetherai.cjs:163:44)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async Promise.all (index 0)
    at async TogetherAI._generate (node_modules/@langchain/core/dist/language_models/llms.cjs:358:29)
    at async TogetherAI._generateUncached (node_modules/@langchain/core/dist/language_models/llms.cjs:172:26)
    at async TogetherAI.invoke (node_modules/@langchain/core/dist/language_models/llms.cjs:35:24)
    at test (bla.ts:7:22)

Node.js v20.11.1

Description

I am using TogetherAI LLM

System Info

node v20.11.1
platform mac
"langchain": "^0.3.2"
"@langchain/community": "^0.3.5",

[dosubot]

Hey @raducostea! I'm here to help you with your TogetherAI LLM issue. I'm a bot designed to assist with bug solving, answering questions, and contributing. Let's tackle this problem together!

To resolve the "Cannot read properties of undefined (reading 'choices')" error when using TogetherAI LLM with LangChain.js, ensure that the response from the TogetherAI API contains the choices array in the output object. This error occurs when the code attempts to access response.output.choices[0].text but choices is undefined.

Here are some steps to troubleshoot and resolve the issue:

1. Check API Response: Ensure that the API response from TogetherAI includes the choices array. You can log the response to verify its structure:

   const response: TogetherAIInferenceResult = await llm.completionWithRetry('Hello');
   console.log(response);

2. Handle Missing Choices: Add a check to handle cases where choices might be undefined or empty:

   const outputText = response.output.choices?.[0]?.text ?? "No output generated";
   return outputText;

3. Verify API Call: Ensure that the API call is correctly formed and that the TogetherAI service is returning a valid response. Check for any errors in the API call that might lead to an incomplete response.

By implementing these checks, you can prevent the error and handle cases where the choices array is not present in the response ^[1].

To continue talking to Dosu, mention @dosu.