python/samples/getting_started/06-memory-and-embeddings.ipynb

{
 "cells": [
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "68e1c158",
   "metadata": {},
   "source": [
    "# Building Semantic Memory with Embeddings\n",
    "\n",
    "So far, we've mostly been treating the kernel as a stateless orchestration engine.\n",
    "We send text into a model API and receive text out.\n",
    "\n",
    "In a [previous notebook](04-kernel-arguments-chat.ipynb), we used `kernel arguments` to pass in additional\n",
    "text into prompts to enrich them with more data. This allowed us to create a basic chat experience.\n",
    "\n",
    "However, if you solely relied on kernel arguments, you would quickly realize that eventually your prompt\n",
    "would grow so large that you would run into the model's token limit. What we need is a way to persist state\n",
    "and build both short-term and long-term memory to empower even more intelligent applications.\n",
    "\n",
    "To do this, we dive into the key concept of `Semantic Memory` in the Semantic Kernel.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a77bdf89",
   "metadata": {},
   "outputs": [],
   "source": [
    "!python -m pip install semantic-kernel==1.0.3\n",
    "!python -m pip install azure-core==1.30.1\n",
    "!python -m pip install azure-search-documents==11.4.0"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1b95af24",
   "metadata": {},
   "outputs": [],
   "source": [
    "from services import Service\n",
    "\n",
    "# Select a service to use for this notebook (available services: OpenAI, AzureOpenAI, HuggingFace)\n",
    "selectedService = Service.OpenAI"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "d8ddffc1",
   "metadata": {},
   "source": [
    "In order to use memory, we need to instantiate the Kernel with a Memory Storage\n",
    "and an Embedding service. In this example, we make use of the `VolatileMemoryStore` which can be thought of as a temporary in-memory storage. This memory is not written to disk and is only available during the app session.\n",
    "\n",
    "When developing your app you will have the option to plug in persistent storage like Azure AI Search, Azure Cosmos Db, PostgreSQL, SQLite, etc. Semantic Memory allows also to index external data sources, without duplicating all the information as you will see further down in this notebook.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8f8dcbc6",
   "metadata": {},
   "outputs": [],
   "source": [
    "from semantic_kernel.connectors.ai.open_ai.services.azure_chat_completion import AzureChatCompletion\n",
    "from semantic_kernel.connectors.ai.open_ai.services.azure_text_embedding import AzureTextEmbedding\n",
    "from semantic_kernel.connectors.ai.open_ai.services.open_ai_chat_completion import OpenAIChatCompletion\n",
    "from semantic_kernel.connectors.ai.open_ai.services.open_ai_text_embedding import OpenAITextEmbedding\n",
    "from semantic_kernel.core_plugins.text_memory_plugin import TextMemoryPlugin\n",
    "from semantic_kernel.kernel import Kernel\n",
    "from semantic_kernel.memory.semantic_text_memory import SemanticTextMemory\n",
    "from semantic_kernel.memory.volatile_memory_store import VolatileMemoryStore\n",
    "\n",
    "kernel = Kernel()\n",
    "\n",
    "chat_service_id = \"chat\"\n",
    "\n",
    "# Configure AI service used by the kernel\n",
    "if selectedService == Service.AzureOpenAI:\n",
    "    azure_chat_service = AzureChatCompletion(service_id=chat_service_id)\n",
    "    # next line assumes embeddings deployment name is \"text-embedding\", adjust the deployment name to the value of your chat model if needed\n",
    "    embedding_gen = AzureTextEmbedding(deployment_name=\"text-embedding\")\n",
    "    kernel.add_service(azure_chat_service)\n",
    "    kernel.add_service(embedding_gen)\n",
    "elif selectedService == Service.OpenAI:\n",
    "    oai_chat_service = OpenAIChatCompletion(service_id=chat_service_id, ai_model_id=\"gpt-3.5-turbo\")\n",
    "    embedding_gen = OpenAITextEmbedding(ai_model_id=\"text-embedding-ada-002\")\n",
    "    kernel.add_service(oai_chat_service)\n",
    "    kernel.add_service(embedding_gen)\n",
    "\n",
    "memory = SemanticTextMemory(storage=VolatileMemoryStore(), embeddings_generator=embedding_gen)\n",
    "kernel.add_plugin(TextMemoryPlugin(memory), \"TextMemoryPlugin\")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "e7fefb6a",
   "metadata": {},
   "source": [
    "At its core, Semantic Memory is a set of data structures that allow you to store the meaning of text that come from different data sources, and optionally to store the source text too. These texts can be from the web, e-mail providers, chats, a database, or from your local directory, and are hooked up to the Semantic Kernel through data source connectors.\n",
    "\n",
    "The texts are embedded or compressed into a vector of floats representing mathematically the texts' contents and meaning. You can read more about embeddings [here](https://aka.ms/sk/embeddings).\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "2a7e7ca4",
   "metadata": {},
   "source": [
    "### Manually adding memories\n",
    "\n",
    "Let's create some initial memories \"About Me\". We can add memories to our `VolatileMemoryStore` by using `SaveInformationAsync`\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d096504c",
   "metadata": {},
   "outputs": [],
   "source": [
    "collection_id = \"generic\"\n",
    "\n",
    "\n",
    "async def populate_memory(memory: SemanticTextMemory) -> None:\n",
    "    # Add some documents to the semantic memory\n",
    "    await memory.save_information(collection=collection_id, id=\"info1\", text=\"Your budget for 2024 is $100,000\")\n",
    "    await memory.save_information(collection=collection_id, id=\"info2\", text=\"Your savings from 2023 are $50,000\")\n",
    "    await memory.save_information(collection=collection_id, id=\"info3\", text=\"Your investments are $80,000\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5338d3ac",
   "metadata": {},
   "outputs": [],
   "source": [
    "await populate_memory(memory)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "2calf857",
   "metadata": {},
   "source": [
    "Let's try searching the memory:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "628c843e",
   "metadata": {},
   "outputs": [],
   "source": [
    "async def search_memory_examples(memory: SemanticTextMemory) -> None:\n",
    "    questions = [\n",
    "        \"What is my budget for 2024?\",\n",
    "        \"What are my savings from 2023?\",\n",
    "        \"What are my investments?\",\n",
    "    ]\n",
    "\n",
    "    for question in questions:\n",
    "        print(f\"Question: {question}\")\n",
    "        result = await memory.search(collection_id, question)\n",
    "        print(f\"Answer: {result[0].text}\\n\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "24764c48",
   "metadata": {},
   "outputs": [],
   "source": [
    "await search_memory_examples(memory)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "e70c2b22",
   "metadata": {},
   "source": [
    "Let's now revisit the our chat sample from the [previous notebook](04-kernel-arguments-chat.ipynb).\n",
    "If you remember, we used kernel arguments to fill the prompt with a `history` that continuously got populated as we chatted with the bot. Let's add also memory to it!\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "1ed54a32",
   "metadata": {},
   "source": [
    "This is done by using the `TextMemoryPlugin` which exposes the `recall` native function.\n",
    "\n",
    "`recall` takes an input ask and performs a similarity search on the contents that have\n",
    "been embedded in the Memory Store and returns the most relevant memory.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "fb8549b2",
   "metadata": {},
   "outputs": [],
   "source": [
    "from semantic_kernel.functions import KernelFunction\n",
    "from semantic_kernel.prompt_template import PromptTemplateConfig\n",
    "\n",
    "\n",
    "async def setup_chat_with_memory(\n",
    "    kernel: Kernel,\n",
    "    service_id: str,\n",
    ") -> KernelFunction:\n",
    "    prompt = \"\"\"\n",
    "    ChatBot can have a conversation with you about any topic.\n",
    "    It can give explicit instructions or say 'I don't know' if\n",
    "    it does not have an answer.\n",
    "\n",
    "    Information about me, from previous conversations:\n",
    "    - {{recall 'budget by year'}} What is my budget for 2024?\n",
    "    - {{recall 'savings from previous year'}} What are my savings from 2023?\n",
    "    - {{recall 'investments'}} What are my investments?\n",
    "\n",
    "    {{$request}}\n",
    "    \"\"\".strip()\n",
    "\n",
    "    prompt_template_config = PromptTemplateConfig(\n",
    "        template=prompt,\n",
    "        execution_settings={\n",
    "            service_id: kernel.get_service(service_id).get_prompt_execution_settings_class()(service_id=service_id)\n",
    "        },\n",
    "    )\n",
    "\n",
    "    return kernel.add_function(\n",
    "        function_name=\"chat_with_memory\",\n",
    "        plugin_name=\"chat\",\n",
    "        prompt_template_config=prompt_template_config,\n",
    "    )"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "1ac62457",
   "metadata": {},
   "source": [
    "The `RelevanceParam` is used in memory search and is a measure of the relevance score from 0.0 to 1.0, where 1.0 means a perfect match. We encourage users to experiment with different values.\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "645b55a1",
   "metadata": {},
   "source": [
    "Now that we've included our memories, let's chat!\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e3875a34",
   "metadata": {},
   "outputs": [],
   "source": [
    "print(\"Populating memory...\")\n",
    "await populate_memory(memory)\n",
    "\n",
    "print(\"Asking questions... (manually)\")\n",
    "await search_memory_examples(memory)\n",
    "\n",
    "print(\"Setting up a chat (with memory!)\")\n",
    "chat_func = await setup_chat_with_memory(kernel, chat_service_id)\n",
    "\n",
    "print(\"Begin chatting (type 'exit' to exit):\\n\")\n",
    "print(\n",
    "    \"Welcome to the chat bot!\\\n",
    "    \\n  Type 'exit' to exit.\\\n",
    "    \\n  Try asking a question about your finances (i.e. \\\"talk to me about my finances\\\").\"\n",
    ")\n",
    "\n",
    "\n",
    "async def chat(user_input: str):\n",
    "    print(f\"User: {user_input}\")\n",
    "    answer = await kernel.invoke(chat_func, request=user_input)\n",
    "    print(f\"ChatBot:> {answer}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6b55f64f",
   "metadata": {},
   "outputs": [],
   "source": [
    "await chat(\"What is my budget for 2024?\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "243f9eb2",
   "metadata": {},
   "outputs": [],
   "source": [
    "await chat(\"talk to me about my finances\")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "0a51542b",
   "metadata": {},
   "source": [
    "### Adding documents to your memory\n",
    "\n",
    "Many times in your applications you'll want to bring in external documents into your memory. Let's see how we can do this using our VolatileMemoryStore.\n",
    "\n",
    "Let's first get some data using some of the links in the Semantic Kernel repo.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c3d5a1b9",
   "metadata": {},
   "outputs": [],
   "source": [
    "github_files = {}\n",
    "github_files[\"https://github.com/microsoft/semantic-kernel/blob/main/README.md\"] = (\n",
    "    \"README: Installation, getting started, and how to contribute\"\n",
    ")\n",
    "github_files[\n",
    "    \"https://github.com/microsoft/semantic-kernel/blob/main/dotnet/notebooks/02-running-prompts-from-file.ipynb\"\n",
    "] = \"Jupyter notebook describing how to pass prompts from a file to a semantic plugin or function\"\n",
    "github_files[\"https://github.com/microsoft/semantic-kernel/blob/main/dotnet/notebooks/00-getting-started.ipynb\"] = (\n",
    "    \"Jupyter notebook describing how to get started with the Semantic Kernel\"\n",
    ")\n",
    "github_files[\"https://github.com/microsoft/semantic-kernel/tree/main/samples/plugins/ChatPlugin/ChatGPT\"] = (\n",
    "    \"Sample demonstrating how to create a chat plugin interfacing with ChatGPT\"\n",
    ")\n",
    "github_files[\n",
    "    \"https://github.com/microsoft/semantic-kernel/blob/main/dotnet/src/SemanticKernel/Memory/Volatile/VolatileMemoryStore.cs\"\n",
    "] = \"C# class that defines a volatile embedding store\""
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "75f3ea5e",
   "metadata": {},
   "source": [
    "Now let's add these files to our VolatileMemoryStore using `SaveReferenceAsync`. We'll separate these memories from the chat memories by putting them in a different collection.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "170e7142",
   "metadata": {},
   "outputs": [],
   "source": [
    "memory_collection_name = \"SKGitHub\"\n",
    "print(\"Adding some GitHub file URLs and their descriptions to a volatile Semantic Memory.\")\n",
    "\n",
    "for index, (entry, value) in enumerate(github_files.items()):\n",
    "    await memory.save_reference(\n",
    "        collection=memory_collection_name,\n",
    "        description=value,\n",
    "        text=value,\n",
    "        external_id=entry,\n",
    "        external_source_name=\"GitHub\",\n",
    "    )\n",
    "    print(\"  URL {} saved\".format(index))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "143911c3",
   "metadata": {},
   "outputs": [],
   "source": [
    "ask = \"I love Jupyter notebooks, how should I get started?\"\n",
    "print(\"===========================\\n\" + \"Query: \" + ask + \"\\n\")\n",
    "\n",
    "memories = await memory.search(memory_collection_name, ask, limit=5, min_relevance_score=0.77)\n",
    "\n",
    "for index, memory in enumerate(memories):\n",
    "    print(f\"Result {index}:\")\n",
    "    print(\"  URL:     : \" + memory.id)\n",
    "    print(\"  Title    : \" + memory.description)\n",
    "    print(\"  Relevance: \" + str(memory.relevance))\n",
    "    print()"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "59294dac",
   "metadata": {},
   "source": [
    "Now you might be wondering what happens if you have so much data that it doesn't fit into your RAM? That's where you want to make use of an external Vector Database made specifically for storing and retrieving embeddings. Fortunately, semantic kernel makes this easy thanks to an extensive list of available connectors. In the following section, we will connect to an existing Azure AI Search service that we will use as an external Vector Database to store and retrieve embeddings.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "77fdfa86",
   "metadata": {},
   "outputs": [],
   "source": [
    "from semantic_kernel.connectors.memory.azure_cognitive_search import AzureCognitiveSearchMemoryStore\n",
    "\n",
    "acs_memory_store = AzureCognitiveSearchMemoryStore(vector_size=1536)\n",
    "\n",
    "memory = SemanticTextMemory(storage=acs_memory_store, embeddings_generator=embedding_gen)\n",
    "kernel.add_plugin(TextMemoryPlugin(memory), \"TextMemoryPluginACS\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "94f9e83b",
   "metadata": {},
   "source": [
    "The implementation of Semantic Kernel allows to easily swap memory store for another. Here, we will re-use the functions we initially created for `VolatileMemoryStore` with our new external Vector Store leveraging Azure AI Search\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "fc3da7e1",
   "metadata": {},
   "outputs": [],
   "source": [
    "await populate_memory(memory)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b0bbe830",
   "metadata": {},
   "source": [
    "Let's now try to query from Azure AI Search!\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1a09d0ca",
   "metadata": {},
   "outputs": [],
   "source": [
    "await search_memory_examples(memory)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3d33dcdc",
   "metadata": {},
   "source": [
    "We have laid the foundation which will allow us to store an arbitrary amount of data in an external Vector Store above and beyond what could fit in memory at the expense of a little more latency.\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.14"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}