diff --git a/README.md b/README.md index 5367f9b..f63863e 100644 --- a/README.md +++ b/README.md @@ -23,7 +23,7 @@ -This sample shows how to build a serverless AI chat experience with Retrieval-Augmented Generation using [LangChain.js](https://js.langchain.com/) and Azure. The application is hosted on [Azure Static Web Apps](https://learn.microsoft.com/azure/static-web-apps/overview) and [Azure Functions](https://learn.microsoft.com/azure/azure-functions/functions-overview?pivots=programming-language-javascript), with [Azure AI Search](https://learn.microsoft.com/azure/search/search-what-is-azure-search) as the vector database. You can use it as a starting point for building more complex AI applications. +This sample shows how to build a serverless AI chat experience with Retrieval-Augmented Generation using [LangChain.js](https://js.langchain.com/) and Azure. The application is hosted on [Azure Static Web Apps](https://learn.microsoft.com/azure/static-web-apps/overview) and [Azure Functions](https://learn.microsoft.com/azure/azure-functions/functions-overview?pivots=programming-language-javascript), with [Azure Cosmos DB for NoSQL](https://learn.microsoft.com/azure/cosmos-db/nosql/vector-) as the vector database. You can use it as a starting point for building more complex AI applications. > [!TIP] > You can test this application locally without any cost using [Ollama](https://ollama.com/). Follow the instructions in the [Local Development](#local-development) section to get started. @@ -44,7 +44,7 @@ This application is made from multiple components: - A serverless API built with [Azure Functions](https://learn.microsoft.com/azure/azure-functions/functions-overview?pivots=programming-language-javascript) and using [LangChain.js](https://js.langchain.com/) to ingest the documents and generate responses to the user chat queries. The code is located in the `packages/api` folder. -- A database to store the text extracted from the documents and the vectors generated by LangChain.js, using [Azure AI Search](https://learn.microsoft.com/azure/search/search-what-is-azure-search). +- A database to store the text extracted from the documents and the vectors generated by LangChain.js, using [Azure Cosmos DB for NoSQL](https://learn.microsoft.com/azure/cosmos-db/nosql/). - A file storage to store the source documents, using [Azure Blob Storage](https://learn.microsoft.com/azure/storage/blobs/storage-blobs-introduction). @@ -53,7 +53,7 @@ We use the [HTTP protocol for AI chat apps](https://aka.ms/chatprotocol) to comm ## Features - **Serverless Architecture**: Utilizes Azure Functions and Azure Static Web Apps for a fully serverless deployment. -- **Retrieval-Augmented Generation (RAG)**: Combines the power of Azure AI Search and LangChain.js to provide relevant and accurate responses. +- **Retrieval-Augmented Generation (RAG)**: Combines the power of Azure Cosmos DB and LangChain.js to provide relevant and accurate responses. - **Scalable and Cost-Effective**: Leverages Azure's serverless offerings to provide a scalable and cost-effective solution. - **Local Development**: Supports local development using Ollama for testing without any cloud costs. @@ -212,7 +212,7 @@ Here are some resources to learn more about the technologies used in this sample - [LangChain.js documentation](https://js.langchain.com) - [Generative AI For Beginners](https://github.com/microsoft/generative-ai-for-beginners) - [Azure OpenAI Service](https://learn.microsoft.com/azure/ai-services/openai/overview) -- [Azure AI Search](https://learn.microsoft.com/azure/search/search-what-is-azure-search) +- [Azure Cosmos DB for NoSQL](https://learn.microsoft.com/azure/cosmos-db/nosql/) - [Ask YouTube: LangChain.js + Azure Quickstart sample](https://github.com/Azure-Samples/langchainjs-quickstart-demo) - [Chat + Enterprise data with Azure OpenAI and Azure AI Search](https://github.com/Azure-Samples/azure-search-openai-javascript) - [Revolutionize your Enterprise Data with Chat: Next-gen Apps w/ Azure OpenAI and AI Search](https://aka.ms/entgptsearchblog) diff --git a/docs/cost.md b/docs/cost.md index 27093e0..666737c 100644 --- a/docs/cost.md +++ b/docs/cost.md @@ -1,12 +1,12 @@ ## Cost estimation Pricing varies per region and usage, so it isn't possible to predict exact costs for your usage. -However, you can use the [Azure pricing calculator](https://azure.com/e/c504007c9f024699a37f5d947dbb1e79) for the resources below to get an estimate. +However, you can use the [Azure pricing calculator](https://azure.com/e/a586bf32fdfa4bb9b368a6b6543c4b50) for the resources below to get an estimate. - Azure Functions: Consumption plan, Free for the first 1M executions. Pricing per execution and memory used. [Pricing](https://azure.microsoft.com/pricing/details/functions/) - Azure Static Web Apps: Free tier, 100GB bandwidth. Pricing per GB served. [Pricing](https://azure.microsoft.com/pricing/details/app-service/static/) - Azure OpenAI: Standard tier, GPT and Ada models. Pricing per 1K tokens used, and at least 1K tokens are used per question. [Pricing](https://azure.microsoft.com/pricing/details/cognitive-services/openai-service/) -- Azure AI Search: Basic tier, 1 replica. Pricing per hour. [Pricing](https://azure.microsoft.com/pricing/details/search/) +- Azure Cosmos DB: Serverless tier. Pricing per request unit (RU). [Pricing](https://azure.microsoft.com/pricing/details/cosmos-db/autoscale-provisioned/) - Azure Blob Storage: Standard tier with LRS. Pricing per GB stored and data transfer. [Pricing](https://azure.microsoft.com/pricing/details/storage/blobs/) ⚠️ To avoid unnecessary costs, remember to take down your app if it's no longer in use, diff --git a/docs/faq.md b/docs/faq.md index 9fb419f..d6e38b2 100644 --- a/docs/faq.md +++ b/docs/faq.md @@ -7,7 +7,7 @@ Retrieval-Augmented Generation (RAG) is a method used in artificial intelligence At its core, RAG involves two main components: -- **Retriever**: Think "_like a search engine_", finding relevant information from a knowledgebase, usually a vector database. In this sample, we're using Azure AI Search as our vector database. +- **Retriever**: Think "_like a search engine_", finding relevant information from a knowledgebase, usually a vector database. In this sample, we're using Azure Cosmos DB for NoSQL as our vector database. - **Generator**: Acts like a writer, taking the prompt and information retrieved to create a response. We're using here a Large Language Model (LLM) for this task. @@ -114,7 +114,7 @@ The `azd up` command comes from the [Azure Developer CLI](https://learn.microsof The `azd up` command uses the `azure.yaml` file combined with the infrastructure-as-code `.bicep` files in the `infra/` folder. The `azure.yaml` file for this project declares several "hooks" for the prepackage step and postprovision steps. The `up` command first runs the `prepackage` hook which installs Node dependencies and builds the TypeScript files. It then packages all the code (both frontend and backend services) into a zip file which it will deploy later. -Next, it provisions the resources based on `main.bicep` and `main.parameters.json`. At that point, since there is no default value for the OpenAI resource location, it asks you to pick a location from a short list of available regions. Then it will send requests to Azure to provision all the required resources. With everything provisioned, it runs the `postprovision` hook to process the local data and add it to an Azure AI Search index. +Next, it provisions the resources based on `main.bicep` and `main.parameters.json`. At that point, since there is no default value for the OpenAI resource location, it asks you to pick a location from a short list of available regions. Then it will send requests to Azure to provision all the required resources. With everything provisioned, it runs the `postprovision` hook to process the local data and add it to an Azure Cosmos DB index. Finally, it looks at `azure.yaml` to determine the Azure host (Functions and Static Web Apps, in this case) and uploads the zip to Azure. The `azd up` command is now complete, but it may take some time for the app to be fully available and working after the initial deploy. diff --git a/docs/readme.md b/docs/readme.md index 64efd0d..e8e1bfe 100644 --- a/docs/readme.md +++ b/docs/readme.md @@ -17,7 +17,7 @@ description: Build your own serverless AI Chat with Retrieval-Augmented-Generati -This sample shows how to build a serverless AI chat experience with Retrieval-Augmented Generation using [LangChain.js](https://js.langchain.com/) and Azure. The application is hosted on [Azure Static Web Apps](https://learn.microsoft.com/azure/static-web-apps/overview) and [Azure Functions](https://learn.microsoft.com/azure/azure-functions/functions-overview?pivots=programming-language-javascript), with [Azure AI Search](https://learn.microsoft.com/azure/search/search-what-is-azure-search) as the vector database. You can use it as a starting point for building more complex AI applications. +This sample shows how to build a serverless AI chat experience with Retrieval-Augmented Generation using [LangChain.js](https://js.langchain.com/) and Azure. The application is hosted on [Azure Static Web Apps](https://learn.microsoft.com/azure/static-web-apps/overview) and [Azure Functions](https://learn.microsoft.com/azure/azure-functions/functions-overview?pivots=programming-language-javascript), with [Azure Cosmos DB for NoSQL](https://learn.microsoft.com/azure/cosmos-db/nosql/vector-search) as the vector database. You can use it as a starting point for building more complex AI applications. ![Animation showing the chat app in action](./images/demo.gif) @@ -35,7 +35,7 @@ This application is made from multiple components: - A serverless API built with [Azure Functions](https://learn.microsoft.com/azure/azure-functions/functions-overview?pivots=programming-language-javascript) and using [LangChain.js](https://js.langchain.com/) to ingest the documents and generate responses to the user chat queries. The code is located in the `packages/api` folder. -- A database to store the text extracted from the documents and the vectors generated by LangChain.js, using [Azure AI Search](https://learn.microsoft.com/azure/search/search-what-is-azure-search). +- A database to store the text extracted from the documents and the vectors generated by LangChain.js, using [Azure Cosmos DB for NoSQL](https://learn.microsoft.com/azure/cosmos-db/nosql/). - A file storage to store the source documents, using [Azure Blob Storage](https://learn.microsoft.com/azure/storage/blobs/storage-blobs-introduction). @@ -106,7 +106,7 @@ Here are some resources to learn more about the technologies used in this sample - [LangChain.js documentation](https://js.langchain.com) - [Generative AI For Beginners](https://github.com/microsoft/generative-ai-for-beginners) - [Azure OpenAI Service](https://learn.microsoft.com/azure/ai-services/openai/overview) -- [Azure AI Search](https://learn.microsoft.com/azure/search/search-what-is-azure-search) +- [Azure Cosmos DB for NoSQL](https://learn.microsoft.com/azure/cosmos-db/nosql/) - [Ask YouTube: LangChain.js + Azure Quickstart sample](https://github.com/Azure-Samples/langchainjs-quickstart-demo) - [Chat + Enterprise data with Azure OpenAI and Azure AI Search](https://github.com/Azure-Samples/azure-search-openai-javascript) - [Revolutionize your Enterprise Data with Chat: Next-gen Apps w/ Azure OpenAI and AI Search](https://aka.ms/entgptsearchblog) diff --git a/docs/tutorial/03-understanding-rag.md b/docs/tutorial/03-understanding-rag.md index 717d8bb..bbf21c5 100644 --- a/docs/tutorial/03-understanding-rag.md +++ b/docs/tutorial/03-understanding-rag.md @@ -73,7 +73,7 @@ RAG has significant implications in many fields: - **Educational Content:** Educators can update their materials with the latest research and studies to ensure relevance and accuracy. -> **Note:** To learn more about the RAG architecture, please refer to the official documentation of the Azure AI Search Documentation service, which can be accessed [here](https://learn.microsoft.com/azure/search/retrieval-augmented-generation-overview). +> **Note:** To learn more about the RAG architecture, please refer to the official documentation of the Azure Cosmos DB Documentation service, which can be accessed [here](https://learn.microsoft.com/azure/cosmos-db/gen-ai/rag). ## Next Steps diff --git a/infra/core/database/cosmos/cosmos-account.bicep b/infra/core/database/cosmos/cosmos-account.bicep new file mode 100644 index 0000000..96e97ee --- /dev/null +++ b/infra/core/database/cosmos/cosmos-account.bicep @@ -0,0 +1,40 @@ +metadata description = 'Creates an Azure Cosmos DB account.' +param name string +param location string = resourceGroup().location +param tags object = {} + +@allowed([ 'GlobalDocumentDB', 'MongoDB', 'Parse' ]) +param kind string + +param disableLocalAuth bool = false + +resource cosmos 'Microsoft.DocumentDB/databaseAccounts@2023-11-15' = { + name: name + kind: kind + location: location + tags: tags + properties: { + consistencyPolicy: { defaultConsistencyLevel: 'Session' } + locations: [ + { + locationName: location + failoverPriority: 0 + isZoneRedundant: false + } + ] + databaseAccountOfferType: 'Standard' + enableAutomaticFailover: false + enableMultipleWriteLocations: false + apiProperties: (kind == 'MongoDB') ? { serverVersion: '4.2' } : {} + capabilities: [ + { name: 'EnableServerless' } + { name: 'EnableNoSQLVectorSearch' } + ] + minimalTlsVersion: 'Tls12' + disableLocalAuth: disableLocalAuth + } +} + +output endpoint string = cosmos.properties.documentEndpoint +output id string = cosmos.id +output name string = cosmos.name diff --git a/infra/core/database/cosmos/sql/cosmos-sql-account.bicep b/infra/core/database/cosmos/sql/cosmos-sql-account.bicep new file mode 100644 index 0000000..a394425 --- /dev/null +++ b/infra/core/database/cosmos/sql/cosmos-sql-account.bicep @@ -0,0 +1,21 @@ +metadata description = 'Creates an Azure Cosmos DB for NoSQL account.' +param name string +param location string = resourceGroup().location +param tags object = {} + +param disableLocalAuth bool = false + +module cosmos '../../cosmos/cosmos-account.bicep' = { + name: 'cosmos-account' + params: { + name: name + location: location + tags: tags + kind: 'GlobalDocumentDB' + disableLocalAuth: disableLocalAuth + } +} + +output endpoint string = cosmos.outputs.endpoint +output id string = cosmos.outputs.id +output name string = cosmos.outputs.name diff --git a/infra/core/database/cosmos/sql/cosmos-sql-db.bicep b/infra/core/database/cosmos/sql/cosmos-sql-db.bicep new file mode 100644 index 0000000..7711723 --- /dev/null +++ b/infra/core/database/cosmos/sql/cosmos-sql-db.bicep @@ -0,0 +1,73 @@ +metadata description = 'Creates an Azure Cosmos DB for NoSQL account with a database.' +param accountName string +param databaseName string +param location string = resourceGroup().location +param tags object = {} + +param containers array = [] +param principalIds array = [] +param disableLocalAuth bool = false + +module cosmos 'cosmos-sql-account.bicep' = { + name: 'cosmos-sql-account' + params: { + name: accountName + location: location + tags: tags + disableLocalAuth: disableLocalAuth + } +} + +resource database 'Microsoft.DocumentDB/databaseAccounts/sqlDatabases@2022-05-15' = { + name: '${accountName}/${databaseName}' + properties: { + resource: { id: databaseName } + } + + resource list 'containers' = [for container in containers: { + name: container.name + properties: { + resource: { + id: container.id + partitionKey: { paths: [ container.partitionKey ] } + } + options: {} + } + }] + + dependsOn: [ + cosmos + ] +} + +module roleDefinition 'cosmos-sql-role-def.bicep' = { + name: 'cosmos-sql-role-definition' + params: { + accountName: accountName + } + dependsOn: [ + cosmos + database + ] +} + +// We need batchSize(1) here because sql role assignments have to be done sequentially +@batchSize(1) +module userRole 'cosmos-sql-role-assign.bicep' = [for principalId in principalIds: if (!empty(principalId)) { + name: 'cosmos-sql-user-role-${uniqueString(principalId)}' + params: { + accountName: accountName + roleDefinitionId: roleDefinition.outputs.id + principalId: principalId + } + dependsOn: [ + cosmos + database + ] +}] + +output accountId string = cosmos.outputs.id +output accountName string = cosmos.outputs.name +output databaseName string = databaseName +output endpoint string = cosmos.outputs.endpoint +output roleDefinitionId string = roleDefinition.outputs.id diff --git a/infra/core/database/cosmos/sql/cosmos-sql-role-assign.bicep b/infra/core/database/cosmos/sql/cosmos-sql-role-assign.bicep new file mode 100644 index 0000000..3949efe --- /dev/null +++ b/infra/core/database/cosmos/sql/cosmos-sql-role-assign.bicep @@ -0,0 +1,19 @@ +metadata description = 'Creates a SQL role assignment under an Azure Cosmos DB account.' +param accountName string + +param roleDefinitionId string +param principalId string = '' + +resource role 'Microsoft.DocumentDB/databaseAccounts/sqlRoleAssignments@2022-05-15' = { + parent: cosmos + name: guid(roleDefinitionId, principalId, cosmos.id) + properties: { + principalId: principalId + roleDefinitionId: roleDefinitionId + scope: cosmos.id + } +} + +resource cosmos 'Microsoft.DocumentDB/databaseAccounts@2022-08-15' existing = { + name: accountName +} diff --git a/infra/core/database/cosmos/sql/cosmos-sql-role-def.bicep b/infra/core/database/cosmos/sql/cosmos-sql-role-def.bicep new file mode 100644 index 0000000..778d6dc --- /dev/null +++ b/infra/core/database/cosmos/sql/cosmos-sql-role-def.bicep @@ -0,0 +1,30 @@ +metadata description = 'Creates a SQL role definition under an Azure Cosmos DB account.' +param accountName string + +resource roleDefinition 'Microsoft.DocumentDB/databaseAccounts/sqlRoleDefinitions@2022-08-15' = { + parent: cosmos + name: guid(cosmos.id, accountName, 'sql-role') + properties: { + assignableScopes: [ + cosmos.id + ] + permissions: [ + { + dataActions: [ + 'Microsoft.DocumentDB/databaseAccounts/readMetadata' + 'Microsoft.DocumentDB/databaseAccounts/sqlDatabases/containers/items/*' + 'Microsoft.DocumentDB/databaseAccounts/sqlDatabases/containers/*' + ] + notDataActions: [] + } + ] + roleName: 'Reader Writer' + type: 'CustomRole' + } +} + +resource cosmos 'Microsoft.DocumentDB/databaseAccounts@2022-08-15' existing = { + name: accountName +} + +output id string = roleDefinition.id diff --git a/infra/core/search/search-services.bicep b/infra/core/search/search-services.bicep deleted file mode 100644 index 33fd83e..0000000 --- a/infra/core/search/search-services.bicep +++ /dev/null @@ -1,68 +0,0 @@ -metadata description = 'Creates an Azure AI Search instance.' -param name string -param location string = resourceGroup().location -param tags object = {} - -param sku object = { - name: 'standard' -} - -param authOptions object = {} -param disableLocalAuth bool = false -param disabledDataExfiltrationOptions array = [] -param encryptionWithCmk object = { - enforcement: 'Unspecified' -} -@allowed([ - 'default' - 'highDensity' -]) -param hostingMode string = 'default' -param networkRuleSet object = { - bypass: 'None' - ipRules: [] -} -param partitionCount int = 1 -@allowed([ - 'enabled' - 'disabled' -]) -param publicNetworkAccess string = 'enabled' -param replicaCount int = 1 -@allowed([ - 'disabled' - 'free' - 'standard' -]) -param semanticSearch string = 'disabled' - -var searchIdentityProvider = (sku.name == 'free') ? null : { - type: 'SystemAssigned' -} - -resource search 'Microsoft.Search/searchServices@2021-04-01-preview' = { - name: name - location: location - tags: tags - // The free tier does not support managed identity - identity: searchIdentityProvider - properties: { - authOptions: disableLocalAuth ? null : authOptions - disableLocalAuth: disableLocalAuth - disabledDataExfiltrationOptions: disabledDataExfiltrationOptions - encryptionWithCmk: encryptionWithCmk - hostingMode: hostingMode - networkRuleSet: networkRuleSet - partitionCount: partitionCount - publicNetworkAccess: publicNetworkAccess - replicaCount: replicaCount - semanticSearch: semanticSearch - } - sku: sku -} - -output id string = search.id -output endpoint string = 'https://${name}.search.windows.net/' -output name string = search.name -output principalId string = !empty(searchIdentityProvider) ? search.identity.principalId : '' - diff --git a/infra/main.bicep b/infra/main.bicep index 4b6a288..7f58390 100644 --- a/infra/main.bicep +++ b/infra/main.bicep @@ -14,11 +14,7 @@ param webappName string = 'webapp' param apiServiceName string = 'api' param appServicePlanName string = '' param storageAccountName string = '' -param searchServiceName string = '' - -// The free tier does not support managed identity (required) or semantic search (optional) -@allowed(['basic', 'standard', 'standard2', 'standard3', 'storage_optimized_l1', 'storage_optimized_l2']) -param searchServiceSkuName string +param cosmosDbServiceName string = '' @description('Location for the OpenAI resource group') @allowed(['australiaeast', 'canadaeast', 'eastus', 'eastus2', 'francecentral', 'japaneast', 'northcentralus', 'swedencentral', 'switzerlandnorth', 'uksouth', 'westeurope']) @@ -68,7 +64,6 @@ var resourceToken = toLower(uniqueString(subscription().id, environmentName, loc var tags = { 'azd-env-name': environmentName } var finalOpenAiUrl = empty(openAiUrl) ? 'https://${openAi.outputs.name}.openai.azure.com' : openAiUrl var storageUrl = 'https://${storage.outputs.name}.blob.${environment().suffixes.storage}' -var searchUrl = 'https://${search.outputs.name}.search.windows.net' var apiResourceName = '${abbrs.webSitesFunctions}api-${resourceToken}' // Organize resources in a resource group @@ -117,7 +112,7 @@ module api './app/api.bicep' = { AZURE_OPENAI_API_VERSION: openAiApiVersion AZURE_OPENAI_API_DEPLOYMENT_NAME: chatDeploymentName AZURE_OPENAI_API_EMBEDDINGS_DEPLOYMENT_NAME: embeddingsDeploymentName - AZURE_AISEARCH_ENDPOINT: searchUrl + AZURE_COSMOSDB_NOSQL_ENDPOINT: cosmosDb.outputs.endpoint AZURE_STORAGE_URL: storageUrl AZURE_STORAGE_CONTAINER_NAME: blobContainerName } @@ -242,21 +237,26 @@ module openAi 'core/ai/cognitiveservices.bicep' = if (empty(openAiUrl)) { } } -module search 'core/search/search-services.bicep' = { - name: 'search' +module cosmosDb './core/database/cosmos/sql/cosmos-sql-db.bicep' = { + name: 'cosmosDb' scope: resourceGroup params: { - name: !empty(searchServiceName) ? searchServiceName : '${abbrs.searchSearchServices}${resourceToken}' + accountName: !empty(cosmosDbServiceName) ? cosmosDbServiceName : '${abbrs.documentDBDatabaseAccounts}${resourceToken}' location: location tags: tags + containers: [ + { + name: 'vectorSearchContainer' + id: 'vectorSearchContainer' + partitionKey: '/id' + } + ] + databaseName: 'vectorSearchDB' disableLocalAuth: true - authOptions: null - sku: { - name: searchServiceSkuName - } } } + // Managed identity roles assignation // --------------------------------------------------------------------------- @@ -283,25 +283,14 @@ module storageRoleUser 'core/security/role.bicep' = if (!isContinuousDeployment) } } -module searchIndexContribRoleUser 'core/security/role.bicep' = if (!isContinuousDeployment) { +module dbContribRoleUser './core/database/cosmos/sql/cosmos-sql-role-assign.bicep' = if (!isContinuousDeployment) { scope: resourceGroup - name: 'search-index-contrib-role-user' + name: 'db-contrib-role-user' params: { + accountName: cosmosDb.outputs.accountName principalId: principalId - // Search Index Data Contributor - roleDefinitionId: '8ebe5a00-799e-43f5-93ac-243d3dce84a7' - principalType: 'User' - } -} - -module searchContribRoleIndexerUser 'core/security/role.bicep' = if (!isContinuousDeployment) { - scope: resourceGroup - name: 'search-contrib-role-user' - params: { - principalId: principalId - // Search Service Contributor - roleDefinitionId: '7ca78c08-252a-4471-8644-bb5ff32d4ba0' - principalType: 'User' + // Cosmos DB Data Contributor + roleDefinitionId: cosmosDb.outputs.roleDefinitionId } } @@ -328,25 +317,14 @@ module storageRoleApi 'core/security/role.bicep' = { } } -module searchIndexContribRoleApi 'core/security/role.bicep' = { +module dbContribRoleApi './core/database/cosmos/sql/cosmos-sql-role-assign.bicep' = { scope: resourceGroup - name: 'search-index-contrib-role-api' + name: 'db-contrib-role-api' params: { + accountName: cosmosDb.outputs.accountName principalId: api.outputs.identityPrincipalId - // Search Index Data Contributor - roleDefinitionId: '8ebe5a00-799e-43f5-93ac-243d3dce84a7' - principalType: 'ServicePrincipal' - } -} - -module searchContribRoleIndexerApi 'core/security/role.bicep' = { - scope: resourceGroup - name: 'search-contrib-role-api' - params: { - principalId: api.outputs.identityPrincipalId - // Search Service Contributor - roleDefinitionId: '7ca78c08-252a-4471-8644-bb5ff32d4ba0' - principalType: 'ServicePrincipal' + // Cosmos DB Data Contributor + roleDefinitionId: cosmosDb.outputs.roleDefinitionId } } @@ -361,7 +339,7 @@ output AZURE_OPENAI_API_DEPLOYMENT_NAME string = chatDeploymentName output AZURE_OPENAI_API_EMBEDDINGS_DEPLOYMENT_NAME string = embeddingsDeploymentName output AZURE_STORAGE_URL string = storageUrl output AZURE_STORAGE_CONTAINER_NAME string = blobContainerName -output AZURE_AISEARCH_ENDPOINT string = searchUrl +output AZURE_COSMOSDB_NOSQL_ENDPOINT string = cosmosDb.outputs.endpoint output API_URL string = useVnet ? '' : api.outputs.uri output WEBAPP_URL string = webapp.outputs.uri diff --git a/infra/main.parameters.json b/infra/main.parameters.json index 64bf87c..a69ab3c 100644 --- a/infra/main.parameters.json +++ b/infra/main.parameters.json @@ -35,9 +35,6 @@ "webappLocation": { "value": "${AZURE_WEBAPP_LOCATION=eastus2}" }, - "searchServiceSkuName": { - "value": "${AZURE_SEARCH_SERVICE_SKU=basic}" - }, "useVnet": { "value": "${USE_VNET=false}" }, diff --git a/package-lock.json b/package-lock.json index 1a18041..fd5845f 100644 --- a/package-lock.json +++ b/package-lock.json @@ -223,6 +223,30 @@ "node": ">=18.0.0" } }, + "node_modules/@azure/cosmos": { + "version": "4.0.1-beta.3", + "resolved": "https://registry.npmjs.org/@azure/cosmos/-/cosmos-4.0.1-beta.3.tgz", + "integrity": "sha512-CpRGt+S5jnvtGUi4TmlS79YvxpbNc8/5/QHgIvvQ9D2ZFUqO0MjbMCU3lVZV2NAJT02BsbLfRAFe+FPn8nMRQw==", + "license": "MIT", + "dependencies": { + "@azure/abort-controller": "^1.0.0", + "@azure/core-auth": "^1.3.0", + "@azure/core-rest-pipeline": "^1.2.0", + "@azure/core-tracing": "^1.0.0", + "debug": "^4.1.1", + "fast-json-stable-stringify": "^2.1.0", + "jsbi": "^3.1.3", + "node-abort-controller": "^3.0.0", + "priorityqueuejs": "^2.0.0", + "semaphore": "^1.0.5", + "tslib": "^2.2.0", + "universal-user-agent": "^6.0.0", + "uuid": "^8.3.0" + }, + "engines": { + "node": ">=18.0.0" + } + }, "node_modules/@azure/functions": { "version": "4.5.0", "resolved": "https://registry.npmjs.org/@azure/functions/-/functions-4.5.0.tgz", @@ -1107,6 +1131,21 @@ "@jridgewell/sourcemap-codec": "^1.4.14" } }, + "node_modules/@langchain/azure-cosmosdb": { + "version": "0.1.0", + "resolved": "https://registry.npmjs.org/@langchain/azure-cosmosdb/-/azure-cosmosdb-0.1.0.tgz", + "integrity": "sha512-xJQDTBOlp9ZR6bLqOTOPcEhncxFH8HKOFkrI7l6j+wS4JBa50gCJhiZ3b7CtcHJx2jBoZfeCF2mnnVgrccxjhA==", + "license": "MIT", + "dependencies": { + "@azure/cosmos": "4.0.1-beta.3", + "@azure/identity": "^4.2.0", + "@langchain/core": "~0.2", + "mongodb": "^6.8.0" + }, + "engines": { + "node": ">=18" + } + }, "node_modules/@langchain/community": { "version": "0.2.22", "resolved": "https://registry.npmjs.org/@langchain/community/-/community-0.2.22.tgz", @@ -1737,6 +1776,15 @@ "@typespec/ts-http-runtime": "^1.0.0-alpha.20240228.1" } }, + "node_modules/@mongodb-js/saslprep": { + "version": "1.1.8", + "resolved": "https://registry.npmjs.org/@mongodb-js/saslprep/-/saslprep-1.1.8.tgz", + "integrity": "sha512-qKwC/M/nNNaKUBMQ0nuzm47b7ZYWQHN3pcXq4IIcoSBc2hOIrflAxJduIvvqmhoz3gR2TacTAs8vlsCVPkiEdQ==", + "license": "MIT", + "dependencies": { + "sparse-bitfield": "^3.0.3" + } + }, "node_modules/@nodelib/fs.scandir": { "version": "2.1.5", "resolved": "https://registry.npmjs.org/@nodelib/fs.scandir/-/fs.scandir-2.1.5.tgz", @@ -2154,6 +2202,21 @@ "resolved": "https://registry.npmjs.org/@types/uuid/-/uuid-9.0.8.tgz", "integrity": "sha512-jg+97EGIcY9AGHJJRaaPVgetKDsrTgbRjQ5Msgjh/DQKEFl0DtyRr/VCOyD1T2R1MNeWPK/u7JoGhlDZnKBAfA==" }, + "node_modules/@types/webidl-conversions": { + "version": "7.0.3", + "resolved": "https://registry.npmjs.org/@types/webidl-conversions/-/webidl-conversions-7.0.3.tgz", + "integrity": "sha512-CiJJvcRtIgzadHCYXw7dqEnMNRjhGZlYK05Mj9OyktqV8uVT8fD2BFOB7S1uwBE3Kj2Z+4UyPmFw/Ixgw/LAlA==", + "license": "MIT" + }, + "node_modules/@types/whatwg-url": { + "version": "11.0.5", + "resolved": "https://registry.npmjs.org/@types/whatwg-url/-/whatwg-url-11.0.5.tgz", + "integrity": "sha512-coYR071JRaHa+xoEvvYqvnIHaVqaYrLPbsufM9BF63HkwI5Lgmy2QR8Q5K/lYDYo5AK82wOvSOS0UsLTpTG7uQ==", + "license": "MIT", + "dependencies": { + "@types/webidl-conversions": "*" + } + }, "node_modules/@types/yauzl": { "version": "2.10.3", "resolved": "https://registry.npmjs.org/@types/yauzl/-/yauzl-2.10.3.tgz", @@ -3446,6 +3509,15 @@ "node": "^6 || ^7 || ^8 || ^9 || ^10 || ^11 || ^12 || >=13.7" } }, + "node_modules/bson": { + "version": "6.8.0", + "resolved": "https://registry.npmjs.org/bson/-/bson-6.8.0.tgz", + "integrity": "sha512-iOJg8pr7wq2tg/zSlCCHMi3hMm5JTOxLTagf3zxhcenHsFp+c6uOs6K7W5UE7A4QIJGtqh/ZovFNMP4mOPJynQ==", + "license": "Apache-2.0", + "engines": { + "node": ">=16.20.1" + } + }, "node_modules/buffer": { "version": "5.7.1", "resolved": "https://registry.npmjs.org/buffer/-/buffer-5.7.1.tgz", @@ -5644,8 +5716,7 @@ "node_modules/fast-json-stable-stringify": { "version": "2.1.0", "resolved": "https://registry.npmjs.org/fast-json-stable-stringify/-/fast-json-stable-stringify-2.1.0.tgz", - "integrity": "sha512-lhd/wF+Lk98HZoTCtlVraHtfh5XYijIjalXck7saUtuanSDyLMxnHhSXEDJqHxD7msR8D0uCmqlkwjCV8xvwHw==", - "dev": true + "integrity": "sha512-lhd/wF+Lk98HZoTCtlVraHtfh5XYijIjalXck7saUtuanSDyLMxnHhSXEDJqHxD7msR8D0uCmqlkwjCV8xvwHw==" }, "node_modules/fast-levenshtein": { "version": "2.0.6", @@ -7226,6 +7297,12 @@ "js-yaml": "bin/js-yaml.js" } }, + "node_modules/jsbi": { + "version": "3.2.5", + "resolved": "https://registry.npmjs.org/jsbi/-/jsbi-3.2.5.tgz", + "integrity": "sha512-aBE4n43IPvjaddScbvWRA2YlTzKEynHzu7MqOyTipdHucf/VxS63ViCjxYRg86M8Rxwbt/GfzHl1kKERkt45fQ==", + "license": "Apache-2.0" + }, "node_modules/jsbn": { "version": "1.1.0", "resolved": "https://registry.npmjs.org/jsbn/-/jsbn-1.1.0.tgz", @@ -8132,6 +8209,12 @@ "integrity": "sha512-+y4mDxU4rvXXu5UDSGCGNiesFmwCHuefGMoPCO1WYucNYj7DsLqrFaa2fXVI0H+NNiPTwwzKwspn9yTZqUGqng==", "dev": true }, + "node_modules/memory-pager": { + "version": "1.5.0", + "resolved": "https://registry.npmjs.org/memory-pager/-/memory-pager-1.5.0.tgz", + "integrity": "sha512-ZS4Bp4r/Zoeq6+NLJpP+0Zzm0pR8whtGPf1XExKLJBAczGMnSi3It14OiNCStjQjM6NU1okjQGSxgEZN8eBYKg==", + "license": "MIT" + }, "node_modules/meow": { "version": "13.2.0", "resolved": "https://registry.npmjs.org/meow/-/meow-13.2.0.tgz", @@ -8325,6 +8408,96 @@ "num-sort": "^2.0.0" } }, + "node_modules/mongodb": { + "version": "6.8.0", + "resolved": "https://registry.npmjs.org/mongodb/-/mongodb-6.8.0.tgz", + "integrity": "sha512-HGQ9NWDle5WvwMnrvUxsFYPd3JEbqD3RgABHBQRuoCEND0qzhsd0iH5ypHsf1eJ+sXmvmyKpP+FLOKY8Il7jMw==", + "license": "Apache-2.0", + "dependencies": { + "@mongodb-js/saslprep": "^1.1.5", + "bson": "^6.7.0", + "mongodb-connection-string-url": "^3.0.0" + }, + "engines": { + "node": ">=16.20.1" + }, + "peerDependencies": { + "@aws-sdk/credential-providers": "^3.188.0", + "@mongodb-js/zstd": "^1.1.0", + "gcp-metadata": "^5.2.0", + "kerberos": "^2.0.1", + "mongodb-client-encryption": ">=6.0.0 <7", + "snappy": "^7.2.2", + "socks": "^2.7.1" + }, + "peerDependenciesMeta": { + "@aws-sdk/credential-providers": { + "optional": true + }, + "@mongodb-js/zstd": { + "optional": true + }, + "gcp-metadata": { + "optional": true + }, + "kerberos": { + "optional": true + }, + "mongodb-client-encryption": { + "optional": true + }, + "snappy": { + "optional": true + }, + "socks": { + "optional": true + } + } + }, + "node_modules/mongodb-connection-string-url": { + "version": "3.0.1", + "resolved": "https://registry.npmjs.org/mongodb-connection-string-url/-/mongodb-connection-string-url-3.0.1.tgz", + "integrity": "sha512-XqMGwRX0Lgn05TDB4PyG2h2kKO/FfWJyCzYQbIhXUxz7ETt0I/FqHjUeqj37irJ+Dl1ZtU82uYyj14u2XsZKfg==", + "license": "Apache-2.0", + "dependencies": { + "@types/whatwg-url": "^11.0.2", + "whatwg-url": "^13.0.0" + } + }, + "node_modules/mongodb-connection-string-url/node_modules/tr46": { + "version": "4.1.1", + "resolved": "https://registry.npmjs.org/tr46/-/tr46-4.1.1.tgz", + "integrity": "sha512-2lv/66T7e5yNyhAAC4NaKe5nVavzuGJQVVtRYLyQ2OI8tsJ61PMLlelehb0wi2Hx6+hT/OJUWZcw8MjlSRnxvw==", + "license": "MIT", + "dependencies": { + "punycode": "^2.3.0" + }, + "engines": { + "node": ">=14" + } + }, + "node_modules/mongodb-connection-string-url/node_modules/webidl-conversions": { + "version": "7.0.0", + "resolved": "https://registry.npmjs.org/webidl-conversions/-/webidl-conversions-7.0.0.tgz", + "integrity": "sha512-VwddBukDzu71offAQR975unBIGqfKZpM+8ZX6ySk8nYhVoo5CYaZyzt3YBvYtRtO+aoGlqxPg/B87NGVZ/fu6g==", + "license": "BSD-2-Clause", + "engines": { + "node": ">=12" + } + }, + "node_modules/mongodb-connection-string-url/node_modules/whatwg-url": { + "version": "13.0.0", + "resolved": "https://registry.npmjs.org/whatwg-url/-/whatwg-url-13.0.0.tgz", + "integrity": "sha512-9WWbymnqj57+XEuqADHrCJ2eSXzn8WXIW/YSGaZtb2WKAInQ6CHfaUUcTyyver0p8BDg5StLQq8h1vtZuwmOig==", + "license": "MIT", + "dependencies": { + "tr46": "^4.1.1", + "webidl-conversions": "^7.0.0" + }, + "engines": { + "node": ">=16" + } + }, "node_modules/ms": { "version": "2.1.2", "resolved": "https://registry.npmjs.org/ms/-/ms-2.1.2.tgz", @@ -8395,6 +8568,12 @@ "node": ">=10" } }, + "node_modules/node-abort-controller": { + "version": "3.1.1", + "resolved": "https://registry.npmjs.org/node-abort-controller/-/node-abort-controller-3.1.1.tgz", + "integrity": "sha512-AGK2yQKIjRuqnc6VkX2Xj5d+QW8xZ87pa1UK6yA6ouUyuxfHuMP6umE5QK7UmTeOAymo+Zx1Fxiuw9rVx8taHQ==", + "license": "MIT" + }, "node_modules/node-addon-api": { "version": "6.1.0", "resolved": "https://registry.npmjs.org/node-addon-api/-/node-addon-api-6.1.0.tgz", @@ -9307,6 +9486,12 @@ "url": "https://github.com/sponsors/sindresorhus" } }, + "node_modules/priorityqueuejs": { + "version": "2.0.0", + "resolved": "https://registry.npmjs.org/priorityqueuejs/-/priorityqueuejs-2.0.0.tgz", + "integrity": "sha512-19BMarhgpq3x4ccvVi8k2QpJZcymo/iFUcrhPd4V96kYGovOdTsWwy7fxChYi4QY+m2EnGBWSX9Buakz+tWNQQ==", + "license": "MIT" + }, "node_modules/progress": { "version": "2.0.3", "resolved": "https://registry.npmjs.org/progress/-/progress-2.0.3.tgz", @@ -9376,7 +9561,6 @@ "version": "2.3.1", "resolved": "https://registry.npmjs.org/punycode/-/punycode-2.3.1.tgz", "integrity": "sha512-vYt7UD1U9Wg6138shLtLOvdAu+8DsC/ilFtEVHcH+wydcSpNE20AfSOduf6MkRFahL5FY7X1oU7nKVZFtfq8Fg==", - "dev": true, "engines": { "node": ">=6" } @@ -9908,6 +10092,14 @@ "url": "https://opencollective.com/webpack" } }, + "node_modules/semaphore": { + "version": "1.1.0", + "resolved": "https://registry.npmjs.org/semaphore/-/semaphore-1.1.0.tgz", + "integrity": "sha512-O4OZEaNtkMd/K0i6js9SL+gqy0ZCBMgUvlSqHKi4IBdjhe7wB8pwztUk1BbZ1fmrvpwFrPbHzqd2w5pTcJH6LA==", + "engines": { + "node": ">=0.8.0" + } + }, "node_modules/semver": { "version": "7.6.3", "resolved": "https://registry.npmjs.org/semver/-/semver-7.6.3.tgz", @@ -10185,6 +10377,15 @@ "source-map": "^0.6.0" } }, + "node_modules/sparse-bitfield": { + "version": "3.0.3", + "resolved": "https://registry.npmjs.org/sparse-bitfield/-/sparse-bitfield-3.0.3.tgz", + "integrity": "sha512-kvzhi7vqKTfkh0PZU+2D2PIllw2ymqJKujUcyPMd9Y75Nv4nPbGJZXNhxsgdQab2BmlDct1YnfQCguEvHr7VsQ==", + "license": "MIT", + "dependencies": { + "memory-pager": "^1.0.2" + } + }, "node_modules/spawn-command": { "version": "0.0.2", "resolved": "https://registry.npmjs.org/spawn-command/-/spawn-command-0.0.2.tgz", @@ -10916,6 +11117,12 @@ "url": "https://github.com/sponsors/sindresorhus" } }, + "node_modules/universal-user-agent": { + "version": "6.0.1", + "resolved": "https://registry.npmjs.org/universal-user-agent/-/universal-user-agent-6.0.1.tgz", + "integrity": "sha512-yCzhz6FN2wU1NiiQRogkTQszlQSlpWaw8SvVegAc+bDxbzHgh1vX8uIe8OYyMH6DwH+sdTJsgMl36+mSMdRJIQ==", + "license": "ISC" + }, "node_modules/universalify": { "version": "2.0.1", "resolved": "https://registry.npmjs.org/universalify/-/universalify-2.0.1.tgz", @@ -11725,6 +11932,7 @@ "@azure/identity": "^4.2.0", "@azure/search-documents": "^12.0.0", "@azure/storage-blob": "^12.17.0", + "@langchain/azure-cosmosdb": "^0.1.0", "@langchain/community": "^0.2.22", "@langchain/core": "~0.2.0", "@langchain/ollama": "^0.0.2", diff --git a/packages/api/package.json b/packages/api/package.json index 729148f..6b151b8 100644 --- a/packages/api/package.json +++ b/packages/api/package.json @@ -18,6 +18,7 @@ "@azure/identity": "^4.2.0", "@azure/search-documents": "^12.0.0", "@azure/storage-blob": "^12.17.0", + "@langchain/azure-cosmosdb": "^0.1.0", "@langchain/community": "^0.2.22", "@langchain/core": "~0.2.0", "@langchain/ollama": "^0.0.2", diff --git a/packages/api/src/functions/chat-post.ts b/packages/api/src/functions/chat-post.ts index 558287a..6f7beea 100644 --- a/packages/api/src/functions/chat-post.ts +++ b/packages/api/src/functions/chat-post.ts @@ -11,7 +11,7 @@ import { ChatOllama } from '@langchain/ollama'; import { FaissStore } from '@langchain/community/vectorstores/faiss'; import { ChatPromptTemplate, PromptTemplate } from '@langchain/core/prompts'; import { createStuffDocumentsChain } from 'langchain/chains/combine_documents'; -import { AzureAISearchVectorStore } from '@langchain/community/vectorstores/azure_aisearch'; +import { AzureCosmosDBNoSQLVectorStore } from '@langchain/azure-cosmosdb'; import { createRetrievalChain } from 'langchain/chains/retrieval'; import 'dotenv/config'; import { badRequest, data, serviceUnavailable } from '../http-response.js'; @@ -62,7 +62,7 @@ export async function postChat(request: HttpRequest, context: InvocationContext) temperature: 0.7, azureADTokenProvider, }); - store = new AzureAISearchVectorStore(embeddings, { credentials }); + store = new AzureCosmosDBNoSQLVectorStore(embeddings, { credentials }); } else { // If no environment variables are set, it means we are running locally context.log('No Azure OpenAI endpoint set, using Ollama models and local DB'); diff --git a/packages/api/src/functions/documents-post.ts b/packages/api/src/functions/documents-post.ts index 81da63b..135b7f5 100644 --- a/packages/api/src/functions/documents-post.ts +++ b/packages/api/src/functions/documents-post.ts @@ -3,7 +3,7 @@ import { type HttpRequest, type HttpResponseInit, type InvocationContext, app } import { AzureOpenAIEmbeddings } from '@langchain/openai'; import { PDFLoader } from '@langchain/community/document_loaders/fs/pdf'; import { RecursiveCharacterTextSplitter } from 'langchain/text_splitter'; -import { AzureAISearchVectorStore } from '@langchain/community/vectorstores/azure_aisearch'; +import { AzureCosmosDBNoSQLVectorStore } from '@langchain/azure-cosmosdb'; import { OllamaEmbeddings } from '@langchain/community/embeddings/ollama'; import { FaissStore } from '@langchain/community/vectorstores/faiss'; import 'dotenv/config'; @@ -25,7 +25,8 @@ export async function postDocuments(request: HttpRequest, context: InvocationCon return badRequest('"file" field not found in form data.'); } - const file = parsedForm.get('file') as File; + // Type mismatch between Node.js FormData and Azure Functions FormData + const file = parsedForm.get('file') as any as File; const filename = file.name; // Extract text from the PDF @@ -49,7 +50,7 @@ export async function postDocuments(request: HttpRequest, context: InvocationCon // Initialize embeddings model and vector database const embeddings = new AzureOpenAIEmbeddings({ azureADTokenProvider }); - await AzureAISearchVectorStore.fromDocuments(documents, embeddings, { credentials }); + await AzureCosmosDBNoSQLVectorStore.fromDocuments(documents, embeddings, { credentials }); } else { // If no environment variables are set, it means we are running locally context.log('No Azure OpenAI endpoint set, using Ollama models and local DB');