google-genai[patch]: Extend GoogleGenerativeAIEmbeddings to embed more than 100 documents

[imjwang]

Currently GoogleGenerativeAIEmbeddings does not work if there are more than 100 documents, it silently fails. It seems like the underlying GoogleGenerativeAI client's batchEmbedContents is maxed at 100. Running the client directly results in a 400.

  const gclient = new GoogleGenerativeAI(
    process.env.GOOGLE_API_KEY!
  ).getGenerativeModel({
    model: 'embedding-001'
  })

  await gclient.batchEmbedContents({
    requests: text.map(t => ({
      content: { role: 'user', parts: [{ text: t }] },
      taskType: TaskType.SEMANTIC_SIMILARITY
    }))
  })

 ⨯ GoogleGenerativeAIError: [400 Bad Request] * BatchEmbedContentsRequest.requests: at most 100 requests can be in one batch

This pr chunks the documents to size 100 and uses Promise.allSettled

It returns an array of embeddings as expected by a vectorstore.

Failed embeddings return empty array - current behavior
Failed requests add empty arrays equal to size of chunk.

Replicate:

  const embeddings = new GoogleGenerativeAIEmbeddings({
    modelName: 'embedding-001',
    taskType: TaskType.RETRIEVAL_DOCUMENT
  })
  // call on small array
  const justRight = Array(100).fill('a')
  const res = await embeddings.embedDocuments(justRight)
  console.log(res.length)
  // call on big array
  const tooBig = Array(101).fill('a') 
  try {
    const res2 = await embeddings.embedDocuments(tooBig)
    console.log(res2.length)
  } catch (e) {
    console.log('caught') // doesn't log
  }

First call succeeds. Second call fails and gives uncaught error.

100
 ⨯ GoogleGenerativeAIError: [400 Bad Request] * BatchEmbedContentsRequest.requests: at most 100 requests can be in one batch

    at makeRequest (/Users/jeff/Documents/repos/langchainjs/.next/server/chunks/node_modules__pnpm_f385b9._.js:12364:19)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async batchEmbedContents (/Users/jeff/Documents/repos/langchainjs/.next/server/chunks/node_modules__pnpm_f385b9._.js:12926:22)
    at async GoogleGenerativeAIEmbeddings._embedDocumentsContent (/Users/jeff/Documents/repos/langchainjs/.next/server/chunks/node_modules__pnpm_f385b9._.js:15519:21)
    at async RetryOperation._fn (/Users/jeff/Documents/repos/langchainjs/.next/server/chunks/node_modules__pnpm_f385b9._.js:10900:25) {
  attemptNumber: 7,
  retriesLeft: 0
}

Fixes #4278

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
langchainjs-api-refs	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Feb 6, 2024 4:41am
langchainjs-docs	✅ Ready (Inspect)	Visit Preview		Feb 6, 2024 4:41am

[jacoblee93]

Thank you!

[jacoblee93]

Thank you!