cosmosdbnosql: Add Semantic Cache Integration #7033
Conversation
|
The latest updates on your projects. Learn more about Vercel for Git ↗︎
1 Skipped Deployment
|
dosubot
bot
added
size:L
aditishree1
changed the title
| @@ -78,7 +78,7 @@ export interface AzureCosmosDBNoSQLConfig | |||
| readonly metadataKey?: string; | |||
| } | |||
|
|
|||
| const USER_AGENT_PREFIX = "langchainjs-azure-cosmosdb-nosql"; | |||
| const USER_AGENT_SUFFIX = "LangChain-CDBNoSQL-VectorStore-JavaScript"; | |||
There was a problem hiding this comment.
Could you explain why you made this change?
The naming is following a common pattern we have for all our JS integrations, which makes it easier to use filters: <framework>-<integration_name>
If we need a distinction, it would be best to use <framework>-<integration>-<type>, ie langchainjs-azure-cosmosdb-nosql-vectorstore
There was a problem hiding this comment.
@sinedied we are following a user agent pattern across all AI integrations, and we came up with "LangChain-CDBNoSQL-VectorStore-JavaScript"
| AzureCosmosDBNoSQLVectorStore, | ||
| } from "./azure_cosmosdb_nosql.js"; | ||
|
|
||
| const USER_AGENT_SUFFIX = "LangChain-CDBNoSQL-SemanticCache-JavaScript"; |
There was a problem hiding this comment.
If possible, it would be preferable to use langchainjs-azure-cosmosdb-nosql-semanticcache (see previous comment)
There was a problem hiding this comment.
@sinedied we are following a user agent pattern across all AI integrations, and we came up with "LangChain-CDBNoSQL-VectorStore-JavaScript"
| @@ -68,7 +68,7 @@ export interface AzureCosmosDBNoSQLInitOptions { | |||
| */ | |||
| export interface AzureCosmosDBNoSQLConfig | |||
| extends AzureCosmosDBNoSQLInitOptions { | |||
| readonly client?: CosmosClient; | |||
| client?: CosmosClient; | |||
There was a problem hiding this comment.
The client should not be overridable by the user aside from the constructor, please keep it read-only. When creating the new client in the semantic cache, you can use the constructor to use it.
| private getLlmCache(llmKey: string) { | ||
| const key = getCacheKey(llmKey); | ||
| if (!this.cacheDict[key]) { | ||
| this.cacheDict[key] = new AzureCosmosDBNoSQLVectorStore( |
There was a problem hiding this comment.
In the current state of this implementation, this use the same default container name as for the VectorStore, which can be problematic:
For example, if a user uses default values and have a vector store for RAG and semantic cache, the results will get mixed up.
I suggest 2 changes:
- Add a metadata to indicate that the document is used for semantic caching (so users wants to use a single table, they can use filters to distinguish cache vs documents)
- Use a different container name for the semantic caching by default, for example
vectorSearchContainerto avoid conflicts
There was a problem hiding this comment.
I would prefer adding a different name for the semantic caching container. We are doing the same in LangChain python semantic cache as well. Let's keep the vector search and semantic cache container different.
This PR adds following changes: