03-Quering-AOpenAI.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "d59d527f-1100-45ff-b051-5f7c9029d94d",
   "metadata": {},
   "source": [
    "# Queries with and without Azure OpenAI"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "eb9a9444-dc90-4fc3-aea7-8ee918301aba",
   "metadata": {},
   "source": [
    "So far, you have your Search Engine loaded **from two different data sources in two diferent indexes**, on this notebook we are going to try some example queries and then use Azure OpenAI service to see if we can get a good answer for the user query.\n",
    "\n",
    "The idea is that a user can ask a question about Computer Science (first datasource/index) or about Covid (second datasource/index), and the engine will respond accordingly.\n",
    "This **Multi-Index** demo, mimics the scenario where a company loads multiple type of documents of different types and about completly different topics and the search engine must respond with the most relevant results."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "71f6c7e3-9037-4b1e-ae17-1deaa27b9c08",
   "metadata": {},
   "source": [
    "## Set up variables"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "8e50b404-a061-49e7-a3c7-c6eabc98ff0f",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import os\n",
    "import urllib\n",
    "import requests\n",
    "import random\n",
    "import json\n",
    "from collections import OrderedDict\n",
    "from IPython.display import display, HTML, Markdown\n",
    "from typing import List\n",
    "from operator import itemgetter\n",
    "\n",
    "# LangChain Imports needed\n",
    "from langchain_openai import AzureChatOpenAI\n",
    "from langchain_openai import AzureOpenAIEmbeddings\n",
    "from langchain_core.prompts import ChatPromptTemplate\n",
    "from langchain_core.output_parsers import StrOutputParser\n",
    "from langchain_core.retrievers import BaseRetriever\n",
    "from langchain_core.callbacks import CallbackManagerForRetrieverRun\n",
    "from langchain_core.documents import Document\n",
    "from langchain_core.runnables import ConfigurableField\n",
    "\n",
    "\n",
    "# Our own libraries needed\n",
    "from common.prompts import DOCSEARCH_PROMPT\n",
    "from common.utils import get_search_results\n",
    "\n",
    "from dotenv import load_dotenv\n",
    "load_dotenv(\"credentials.env\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "2f2c22f8-79ab-405c-95e8-77a1978e53bc",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Setup the Payloads header\n",
    "headers = {'Content-Type': 'application/json','api-key': os.environ['AZURE_SEARCH_KEY']}\n",
    "params = {'api-version': os.environ['AZURE_SEARCH_API_VERSION']}"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9297d29b-1f61-4dce-858e-bf4272172dba",
   "metadata": {},
   "source": [
    "## Multi-Index Search queries"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "5a46e2d3-298a-4708-83de-9e108b1a117a",
   "metadata": {
    "scrolled": true,
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Text-based Indexes that we are going to query (from Notebook 01 and 02)\n",
    "index1_name = \"cogsrch-index-files\"\n",
    "index2_name = \"cogsrch-index-csv\"\n",
    "indexes = [index2_name, index1_name]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1c62ebb2-d7be-4bfb-b1ba-4db86c11839a",
   "metadata": {},
   "source": [
    "Try questions that you think might be answered or addressed in computer science papers in 2020-2021 or that can be addressed by medical publications about COVID in 2020-2021. Try comparing the results with the open version of ChatGPT.<br>\n",
    "The idea is that the answers using Azure OpenAI only looks at the information contained on these publications.\n",
    "\n",
    "**Example Questions you can ask**:\n",
    "- What is CLP?\n",
    "- How Markov chains work?\n",
    "- What are some examples of reinforcement learning?\n",
    "- What are the main risk factors for Covid-19?\n",
    "- What medicine reduces inflamation in the lungs?\n",
    "- Why Covid doesn't affect kids that much compared to adults?\n",
    "- Does chloroquine really works against covid?\n",
    "- Who won the 1994 soccer world cup? # This question should yield no answer if the system is correctly grounded"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "b9b53c14-19bd-451f-aa43-7ad27ccfeead",
   "metadata": {},
   "outputs": [],
   "source": [
    "QUESTION = \"What medicine reduces inflamation in the lungs?\""
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f6d925eb-7f9c-429e-a62a-4c37d7702caf",
   "metadata": {},
   "source": [
    "### Search on both indexes individually and aggragate results\n",
    "\n",
    "#### **Note**: \n",
    "In order to standarize the indexes, **there must be 6 mandatory fields present on each index**: `id, title, name, location, chunk, chunkVector`. This is so that each document can be treated the same along the code. Also, **all indexes must have a semantic configuration**.\n",
    "\n",
    "We are going to use Hybrid Queries: Text + Vector Search combined for optimal results!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "faf2e30f-e71f-4533-ab52-27d048b80a89",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200\n",
      "Index: cogsrch-index-csv Results Found: 70150, Results Returned: 10\n",
      "200\n",
      "Index: cogsrch-index-files Results Found: 118874, Results Returned: 10\n"
     ]
    }
   ],
   "source": [
    "agg_search_results = dict()\n",
    "k = 10\n",
    "\n",
    "for index in indexes:\n",
    "    search_payload = {\n",
    "        \"search\": QUESTION, # Text query\n",
    "        \"select\": \"id, title, name, location, chunk\",\n",
    "        \"queryType\": \"semantic\",\n",
    "        \"vectorQueries\": [{\"text\": QUESTION, \"fields\": \"chunkVector\", \"kind\": \"text\", \"k\": k}], # Vector query\n",
    "        \"semanticConfiguration\": \"my-semantic-config\",\n",
    "        \"captions\": \"extractive\",\n",
    "        \"answers\": \"extractive\",\n",
    "        \"count\":\"true\",\n",
    "        \"top\": k\n",
    "    }\n",
    "\n",
    "    r = requests.post(os.environ['AZURE_SEARCH_ENDPOINT'] + \"/indexes/\" + index + \"/docs/search\",\n",
    "                     data=json.dumps(search_payload), headers=headers, params=params)\n",
    "    print(r.status_code)\n",
    "\n",
    "    search_results = r.json()\n",
    "    agg_search_results[index]=search_results\n",
    "    print(\"Index:\", index, \"Results Found: {}, Results Returned: {}\".format(search_results['@odata.count'], len(search_results['value'])))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "255c40f5-d836-480c-8c68-06a2282c8146",
   "metadata": {},
   "outputs": [],
   "source": [
    "# agg_search_results"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b7fd0fe5-4ee0-42e2-a920-72b93a407389",
   "metadata": {
    "tags": []
   },
   "source": [
    "### Display the top results (from both searches) based on the score"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "9e938337-602d-4b61-8141-b8c92a5d91da",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<h4>Top Answers</h4>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<h5>Answer - score: 0.92</h5>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "Importantly, we show that administering recombinant IL-22 in vivo reduces inflammation and fluid leak into the lung. Taken together, our results demonstrate the IL-22/IL-22BP axis is a potential targetable pathway for reducing influenza-induced pneumonia..\u0000"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<h4>Top Results</h4>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<h5><a href=\"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6917921/?sv=2022-11-02&ss=b&srt=sco&sp=rl&se=2026-01-03T02:11:44Z&st=2024-01-02T18:11:44Z&spr=https&sig=ngrEqvqBVaxyuSYqgPVeF%2B9c0fXLs94v3ASgwg7LDBs%3D\">Targeting the IL-22/IL-22BP axis enhances tight junctions and reduces inflammation during influenza infection</a> - score: 3.29</h5>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "Importantly, we show that administering recombinant IL-22 in vivo reduces inflammation and fluid leak into the lung. Taken together, our results demonstrate the IL-22/IL-22BP axis is a potential targetable pathway for reducing influenza-induced pneumonia..\u0000"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<h5><a href=\"https://doi.org/10.1055/s-0033-1351119; https://www.ncbi.nlm.nih.gov/pubmed/23934717/?sv=2022-11-02&ss=b&srt=sco&sp=rl&se=2026-01-03T02:11:44Z&st=2024-01-02T18:11:44Z&spr=https&sig=ngrEqvqBVaxyuSYqgPVeF%2B9c0fXLs94v3ASgwg7LDBs%3D\">Treatment of acute lung injury: current and emerging pharmacological therapies.</a> - score: 3.03</h5>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "Two main pharmacological treatment strategies have arisen-the attempted inhibition of excessive inflammation or the manipulation of the resulting physiological derangement causing respiratory failure. Additionally, such interventions may allow the delivery of less injurious mechanical ventilation."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<h5><a href=\"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7120595/?sv=2022-11-02&ss=b&srt=sco&sp=rl&se=2026-01-03T02:11:44Z&st=2024-01-02T18:11:44Z&spr=https&sig=ngrEqvqBVaxyuSYqgPVeF%2B9c0fXLs94v3ASgwg7LDBs%3D\">Therapeutic Applications</a> - score: 2.74</h5>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "For instance, glucocorticoids are considered the most effective anti-inflammatory therapies for chronic inflammatory and immune diseases, such as asthma. But they are relatively ineffective in asthmatic smokers, and patients with chronic obstructive pulmonary disease (COPD) or cystic fibrosis (CF)."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<h5><a href=\"https://doi.org/10.1002/jcph.1644; https://www.ncbi.nlm.nih.gov/pubmed/32441805/?sv=2022-11-02&ss=b&srt=sco&sp=rl&se=2026-01-03T02:11:44Z&st=2024-01-02T18:11:44Z&spr=https&sig=ngrEqvqBVaxyuSYqgPVeF%2B9c0fXLs94v3ASgwg7LDBs%3D\">Severe Acute Lung Injury Related to COVID-19 Infection: A Review and the Possible Role for Escin.</a> - score: 2.58</h5>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "In this article reviewing literature data related to the use of escin, an agent having potent anti-inflammatory and anti-viral effects in lung injury, we suggest that it could represent a therapeutic opportunity as add-on therapy in ALI related to COVID-19 infection..\u0000"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<h5><a href=\"https://doi.org/10.1021/mp070048k; https://www.ncbi.nlm.nih.gov/pubmed/18491918/?sv=2022-11-02&ss=b&srt=sco&sp=rl&se=2026-01-03T02:11:44Z&st=2024-01-02T18:11:44Z&spr=https&sig=ngrEqvqBVaxyuSYqgPVeF%2B9c0fXLs94v3ASgwg7LDBs%3D\">Inhalable siRNA: potential as a therapeutic agent in the lungs.</a> - score: 2.46</h5>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "In this timely review we look at the potential applications of RNAi in the lungs for the treatment of a range of diseases including inflammatory and immune conditions, cystic fibrosis, infectious disease and cancer. In 2006 Alnylam initiated the first phase 1 clinical study of an inhaled siRNA for the treatment of respiratory syncytial virus."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<h5><a href=\"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2892358/?sv=2022-11-02&ss=b&srt=sco&sp=rl&se=2026-01-03T02:11:44Z&st=2024-01-02T18:11:44Z&spr=https&sig=ngrEqvqBVaxyuSYqgPVeF%2B9c0fXLs94v3ASgwg7LDBs%3D\">Using Complementary and Alternative Medicines to Target the Host Response during Severe Influenza</a> - score: 2.46</h5>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "Biochanin A, a ligand of the peroxisome proliferator activated receptors (PPAR) alpha and gamma and the active isoflavone in Trifolium pratense (red clover), has anti-inflammatory properties, and thus could be used as an influenza treatment."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<h5><a href=\"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5647329/?sv=2022-11-02&ss=b&srt=sco&sp=rl&se=2026-01-03T02:11:44Z&st=2024-01-02T18:11:44Z&spr=https&sig=ngrEqvqBVaxyuSYqgPVeF%2B9c0fXLs94v3ASgwg7LDBs%3D\">ROCK regulates the intermittent mode of interstitial T cell migration in inflamed lungs</a> - score: 2.2</h5>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "ROCK regulates the intermittent mode of interstitial T cell migration in inflamed lungs. Effector T cell migration through tissues can enable control of infection or mediate inflammatory damage."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<h5><a href=\"https://doi.org/10.1002/ddr.21676; https://www.ncbi.nlm.nih.gov/pubmed/32337769/?sv=2022-11-02&ss=b&srt=sco&sp=rl&se=2026-01-03T02:11:44Z&st=2024-01-02T18:11:44Z&spr=https&sig=ngrEqvqBVaxyuSYqgPVeF%2B9c0fXLs94v3ASgwg7LDBs%3D\">Noscapine, a possible drug candidate for attenuation of cytokine release associated with SARS-CoV-2.</a> - score: 2.2</h5>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "Inactivation of ACE2 during virus fusion into the host cell may be one of the underlying reasons for intense immunological reaction seen in the lung tissue. This overreaction is probably mediated through the bradykinin receptor activation."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<h5><a href=\"https://www.ncbi.nlm.nih.gov/pubmed/15339574/?sv=2022-11-02&ss=b&srt=sco&sp=rl&se=2026-01-03T02:11:44Z&st=2024-01-02T18:11:44Z&spr=https&sig=ngrEqvqBVaxyuSYqgPVeF%2B9c0fXLs94v3ASgwg7LDBs%3D\">[Development of the integrated TCM hepatology with western medicine].</a> - score: 1.93</h5>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "The treatment for severe acute respiratory syndrome (SARS) that happened epidemically this spring confirmed again that the integrated traditional Chinese and western medicine had its clinical advantage and vitality, and indicated that this integration could benefit the pathologic understanding and therapeutic strategies for the diseases."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<h5><a href=\"None?sv=2022-11-02&ss=b&srt=sco&sp=rl&se=2026-01-03T02:11:44Z&st=2024-01-02T18:11:44Z&spr=https&sig=ngrEqvqBVaxyuSYqgPVeF%2B9c0fXLs94v3ASgwg7LDBs%3D\">COVID-19 in the heart and the lungs: could we \"Notch\" the inflammatory storm?</a> - score: 1.84</h5>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "The Notch signalling is a major regulator of cardiovascular function and it is also implicated in several biological processes mediating viral infections. In this report we discuss the possibility to target Notch signalling to prevent SARS-CoV-2 infection and interfere with the progression of COVID-19- associated heart and lungs disease..\u0000"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<h5><a href=\"https://datasetsgptsmartsearch.blob.core.windows.net/arxivcs/pdf/0601/0601030v1.pdf?sv=2022-11-02&ss=b&srt=sco&sp=rl&se=2026-01-03T02:11:44Z&st=2024-01-02T18:11:44Z&spr=https&sig=ngrEqvqBVaxyuSYqgPVeF%2B9c0fXLs94v3ASgwg7LDBs%3D\">arXiv:cs/0601030v1  [cs.DL]  9 Jan 2006</a> - score: 1.22</h5>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "isi category isi category name ds critical care medicine ff emergency medicine fy dentistry, oral surgery & medicine oi integrative & complementary medicine op medicine, legal py medicine, general & internal qa medicine, research & experimental vy radiology, nuclear medicine & medical imaging yu tropical medicine  table 4: isi subject categories …"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<h5><a href=\"https://datasetsgptsmartsearch.blob.core.windows.net/arxivcs/pdf/0207/0207088v1.pdf?sv=2022-11-02&ss=b&srt=sco&sp=rl&se=2026-01-03T02:11:44Z&st=2024-01-02T18:11:44Z&spr=https&sig=ngrEqvqBVaxyuSYqgPVeF%2B9c0fXLs94v3ASgwg7LDBs%3D\">arXiv:cs/0207088v1  [cs.LO]  25 Jul 2002</a> - score: 1.01</h5>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "Three experts in medicine provided information related to the diagnosis of two diseases: disease-1 and disease-2. The information concerning John and Mary can be paraphrased as follows:  — Expert I (a clinician):  Symptom-1 and symptom-2 together imply disease-1. Symptom-1 and symptom-3 together imply disease-2."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "display(HTML('<h4>Top Answers</h4>'))\n",
    "\n",
    "for index,search_results in agg_search_results.items():\n",
    "    for result in search_results['@search.answers']:\n",
    "        if result['score'] > 0.5: # Show answers that are at least 50% of the max possible score=1\n",
    "            display(HTML('<h5>' + 'Answer - score: ' + str(round(result['score'],2)) + '</h5>'))\n",
    "            display(HTML(result['text']))\n",
    "            \n",
    "print(\"\\n\\n\")\n",
    "display(HTML('<h4>Top Results</h4>'))\n",
    "\n",
    "content = dict()\n",
    "ordered_content = OrderedDict()\n",
    "\n",
    "\n",
    "for index,search_results in agg_search_results.items():\n",
    "    for result in search_results['value']:\n",
    "        if result['@search.rerankerScore'] > 1:# Show answers that are at least 25% of the max possible score=4\n",
    "            content[result['id']]={\n",
    "                                    \"title\": result['title'],\n",
    "                                    \"chunk\": result['chunk'], \n",
    "                                    \"name\": result['name'], \n",
    "                                    \"location\": result['location'] ,\n",
    "                                    \"caption\": result['@search.captions'][0]['text'],\n",
    "                                    \"score\": result['@search.rerankerScore'],\n",
    "                                    \"index\": index\n",
    "                                    }\n",
    "    \n",
    "#After results have been filtered we will Sort and add them as an Ordered list\\n\",\n",
    "for id in sorted(content, key= lambda x: content[x][\"score\"], reverse=True):\n",
    "    ordered_content[id] = content[id]\n",
    "    url = str(ordered_content[id]['location']) + os.environ['BLOB_SAS_TOKEN']\n",
    "    title = str(ordered_content[id]['title']) if (ordered_content[id]['title']) else ordered_content[id]['name']\n",
    "    score = str(round(ordered_content[id]['score'],2))\n",
    "    display(HTML('<h5><a href=\"'+ url + '\">' + title + '</a> - score: '+ score + '</h5>'))\n",
    "    display(HTML(ordered_content[id]['caption']))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "52a6d3e6-afb2-4fa7-96d3-69bc2373ded5",
   "metadata": {},
   "source": [
    "### Comments on Query results"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "84e02227-6a92-4944-86f8-6c1e38d90fe4",
   "metadata": {},
   "source": [
    "As seen above the semantic re-ranking feature of Azure AI Search service is good. It gives answers (sometimes) and also the top results with the corresponding file and the paragraph where the answers is possible located.\n",
    "\n",
    "Let's see if we can make this better with Azure OpenAI"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8df3e6d4-9a09-4b0f-b328-238738ccfaec",
   "metadata": {},
   "source": [
    "# Using Azure OpenAI\n",
    "\n",
    "To use OpenAI to get a better answer to our question, the thought process is simple: let's **give the answer and the content of the documents from the search result to the GPT model as context and let it provide a better response**. This is what RAG (Retreival Augmented Generation) is about.\n",
    "\n",
    "Now, before we do this, we need to understand a few things first:\n",
    "\n",
    "1) Chainning and Prompt Engineering\n",
    "2) Embeddings\n",
    "\n",
    "We will use a library call **LangChain** that wraps a lot of boiler plate code.\n",
    "Langchain is one library that does a lot of the prompt engineering for us under the hood, for more information see [here](https://python.langchain.com/en/latest/index.html)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "eea62a7d-7e0e-4a93-a89c-20c96560c665",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Set the ENV variables that Langchain needs to connect to Azure OpenAI\n",
    "os.environ[\"OPENAI_API_VERSION\"] = os.environ[\"AZURE_OPENAI_API_VERSION\"]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "325d9138-2250-4f6b-bc88-50d7957f8d33",
   "metadata": {},
   "source": [
    "**Important Note**: Starting now, we will utilize OpenAI models. Please ensure that you have deployed the following models within the Azure OpenAI portal:\n",
    "\n",
    "- text-embedding-ada-002 (or newer)\n",
    "- gpt-35-turbo (1106 or newer)\n",
    "- gpt-4-turbo (1106 or newer)\n",
    "\n",
    "Reference for Azure OpenAI models (regions, limits, dimensions, etc): [HERE](https://learn.microsoft.com/en-us/azure/ai-services/openai/concepts/models)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0e7c720e-ece1-45ad-9d01-2dfd15c182bb",
   "metadata": {},
   "source": [
    "## A gentle intro to chaining LLMs and prompt engineering"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2bcd7028-5a6c-4296-8c85-4f420d408d69",
   "metadata": {},
   "source": [
    "Chains refer to sequences of calls - whether to an LLM, a tool, or a data preprocessing step.\n",
    "\n",
    "Azure OpenAI is a type of LLM (provider) that you can use but there are others like Cohere, Huggingface, etc.\n",
    "\n",
    "Chains can be simple (i.e. Generic) or specialized (i.e. Utility).\n",
    "\n",
    "* Generic — A single LLM is the simplest chain. It takes an input prompt and the name of the LLM and then uses the LLM for text generation (i.e. output for the prompt).\n",
    "\n",
    "Here’s an example:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "13df9247-e784-4e04-9475-55e672efea47",
   "metadata": {},
   "outputs": [],
   "source": [
    "COMPLETION_TOKENS = 1500\n",
    "llm = AzureChatOpenAI(deployment_name=os.environ[\"GPT35_DEPLOYMENT_NAME\"], \n",
    "                      temperature=0, \n",
    "                      max_tokens=COMPLETION_TOKENS)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "a3b55adb-6f98-4f15-b67a-9fbba5820560",
   "metadata": {},
   "outputs": [],
   "source": [
    "output_parser = StrOutputParser()\n",
    "prompt = ChatPromptTemplate.from_messages([\n",
    "    (\"system\", \"You are an assistant that give thorough responses to users.\"),\n",
    "    (\"user\", \"{input}. Give your response in {language}\")\n",
    "])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6417d052-0035-4635-93e8-2bd3ec50d796",
   "metadata": {},
   "source": [
    "The | symbol is similar to a unix pipe operator, which chains together the different components feeds the output from one component as input into the next component."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "77a37e60-a1ef-4750-a1ec-9e4fe5ba07fa",
   "metadata": {},
   "outputs": [],
   "source": [
    "chain = prompt | llm | output_parser"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "6be6b4df-ee2c-4a0c-8ad3-a672d70f4f8d",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/markdown": [
       "El medicamento que reduce la inflamación en los pulmones se llama corticoide inhalado. Este tipo de medicamento se utiliza comúnmente para tratar enfermedades respiratorias como el asma y la enfermedad pulmonar obstructiva crónica (EPOC). Los corticoides inhalados ayudan a reducir la inflamación en los pulmones y a aliviar los síntomas respiratorios. Es importante seguir las indicaciones de un médico para el uso adecuado de este medicamento."
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 40 ms, sys: 3.71 ms, total: 43.7 ms\n",
      "Wall time: 894 ms\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "display(Markdown(chain.invoke({\"input\": QUESTION, \"language\": \"Spanish\"})))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cd8539d0-a538-4368-82c3-5f91d8370f1e",
   "metadata": {},
   "source": [
    "**Note**: this is the first time you use OpenAI in this Accelerator, so if you get a Resource not found error, is most likely because the name of your OpenAI model deployment is different than the environmental variable set above `os.environ[\"GPT35_DEPLOYMENT_NAME\"]`"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "50ed014c-0c6b-448c-b995-fe7970b92ad5",
   "metadata": {},
   "source": [
    "Great!!, now you know how to create a simple prompt and use a chain in order to answer a general question using ChatGPT knowledge!. \n",
    "\n",
    "It is important to note that we rarely use generic chains as standalone chains. More often they are used as building blocks for Utility chains (as we will see next). Also important to notice is that we are NOT using our documents or the result of the Azure Search yet, just the knowledge of ChatGPT on the data it was trained on."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "12c48038-b1af-4228-8ffb-720e554fd3b2",
   "metadata": {
    "tags": []
   },
   "source": [
    "**The second type of Chains are Utility:**\n",
    "\n",
    "* Utility — These are specialized chains, comprised of many building blocks to help solve a specific task. For example, LangChain supports some end-to-end chains (such as `create_retrieval_chain` for QnA Doc retrieval, Summarization, etc).\n",
    "\n",
    "We will build our own specific chain in this workshop for digging deeper and solve our use case of enhancing the results of Azure AI Search."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b0454ddb-44d8-4fa9-929a-5e5563dd28f8",
   "metadata": {},
   "source": [
    "\n",
    "But before dealing with the utility chain needed, let's first review the concept of Embeddings and Vector Search and RAG. \n",
    "\n",
    "## Embeddings and Vector Search\n",
    "\n",
    "From the Azure OpenAI documentation ([HERE](https://learn.microsoft.com/en-us/azure/ai-services/openai/how-to/embeddings?tabs=python)), An embedding is a special format of data representation that can be easily utilized by machine learning models and algorithms. The embedding is an information dense representation of the semantic meaning of a piece of text. Each embedding is a vector of floating point numbers, such that the distance between two embeddings in the vector space is correlated with semantic similarity between two inputs in the original format. For example, if two texts are similar, then their vector representations should also be similar. \n",
    "\n",
    "### Why Do We Need Vectors?\n",
    "\n",
    "Vectors are essential for several reasons:\n",
    "\n",
    "- **Semantic Richness**: They convert the semantic meaning of text into mathematical vectors, capturing nuances that simple keyword searches miss. This makes them incredibly powerful for understanding and processing language.\n",
    "- **Human-like Searching**: Searching using vector distances mimics the human approach to finding information based on context and meaning, rather than relying solely on exact word matches.\n",
    "- **Efficiency in Scale**: Vector representations allow for efficient handling and searching of large datasets. By reducing complex text to numerical vectors, algorithms can quickly sift through vast amounts of information.\n",
    "\n",
    "### Understanding LLM Tokens' Context Limitation\n",
    "\n",
    "Large Language Models (LLMs) like GPT come with a token limit for each input, which poses a challenge when dealing with lengthy documents or extensive data sets. This limitation restricts the model's ability to understand and generate responses based on the full context of the information provided. It becomes crucial, therefore, to devise strategies that can effectively manage and circumvent this limitation to leverage the full power of LLMs.\n",
    "\n",
    "To address this challenge, the solution incorporates several key steps:\n",
    "\n",
    "1. **Segmenting Documents**: Breaking down large documents into smaller, manageable segments.\n",
    "2. **Vectorization of Chunks**: Converting these segments into vectors, making them compatible with vector-based search techniques.\n",
    "3. **Hybrid Search**: Employing both vector and text search methods to pinpoint the most relevant segments in relation to the query.\n",
    "4. **Optimal Context Provision**: Presenting the LLM with the most pertinent segments, ensuring a balance between detail and brevity to stay within token limits.\n",
    "\n",
    "\n",
    "Our ultimate goal is to rely solely on vector indexes and hybrid searchs (vector + text). While it is possible to manually code parsers with OCR for various file types and develop a scheduler to synchronize data with the index, there is a more efficient alternative: **Azure AI Search has automated chunking strategies and vectorization**.\n",
    "\n",
    "It's important to note that **document segmentation and vectorization have already been completed in AI Azure Search**, as seen in the `ordered_content` dictionary. This pre-processing step simplifies subsequent operations, ensuring rapid response times and adherence to the token limits of the chosen OpenAI model.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "80e79235-3d8b-4713-9336-5004cc4a1556",
   "metadata": {},
   "source": [
    "So really, our only job now is to make sure that the results from the Azure AI Search queries fit on the LLM context size, and then let it do its magic."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "12682a1b-df92-49ce-a638-7277103f6cb3",
   "metadata": {},
   "outputs": [],
   "source": [
    "index_name = \"cogsrch-index-files\"\n",
    "index2_name = \"cogsrch-index-csv\"\n",
    "indexes = [index1_name, index2_name]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "78a6d6a7-18ef-45b2-a216-3c1f50006593",
   "metadata": {},
   "source": [
    "In order to not duplicate code, we have put many of the code used above into functions. These functions are in the `common/utils.py` and `common/prompts.py` files. This way we can use these functios in the app that we will build later.\n",
    "\n",
    "`get_search_results()` do the multi-index search and returns the combined ordered list of documents/chunks."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "3bccca45-d1dd-476f-b109-a528b857b6b3",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of results: 20\n"
     ]
    }
   ],
   "source": [
    "k = 20  # play with this parameter and see the quality of the final answer\n",
    "ordered_results = get_search_results(QUESTION, indexes, k=k, reranker_threshold=1)\n",
    "print(\"Number of results:\",len(ordered_results))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "7714f38a-daaa-4fc5-a95a-dd025d153216",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Uncomment the below line if you want to inspect the ordered results\n",
    "# ordered_results"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "235d4238-df6e-40eb-ab38-4fe6db614acd",
   "metadata": {},
   "source": [
    "Now let's create a Prompt Template that will ground the response only in the chunks retrieve by our hybrid AI Search."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "f86ed786-aca0-4e25-947b-d9cf3a82665c",
   "metadata": {},
   "outputs": [],
   "source": [
    "template = \"\"\"Answer the question thoroughly, based **ONLY** on the following context:\n",
    "{context}\n",
    "\n",
    "Question: {question}\n",
    "\"\"\"\n",
    "prompt = ChatPromptTemplate.from_template(template)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "25cba3d1-b5ab-4e28-96b3-ef923d99dc9f",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/markdown": [
       "Chloroquine is a medicine that has been shown to reduce inflammation in the lungs. It has been found to inhibit the lipopolysaccharide-induced release of inflammatory cytokines such as TNF-α, IL-6, CCL2, and CCL3. This suggests that chloroquine might mitigate the cytokine storm associated with severe pneumonia caused by coronaviruses. Therefore, chloroquine is a potential medicine for reducing inflammation in the lungs."
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 19.4 ms, sys: 569 µs, total: 19.9 ms\n",
      "Wall time: 1.28 s\n"
     ]
    }
   ],
   "source": [
    "%%time \n",
    "# Creation of our custom chain\n",
    "chain = prompt | llm | output_parser\n",
    "\n",
    "try:\n",
    "    display(Markdown(chain.invoke({\"question\": QUESTION, \"context\": ordered_results})))\n",
    "except Exception as e:\n",
    "    print(e)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "406c2b91-6752-4ea2-b95c-d1a52dbdd62b",
   "metadata": {},
   "source": [
    "### From GPT-3.5 to GPT-4\n",
    "\n",
    "Now let's see how the response changes if we change to GPT-4\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "f01705f1-7194-452b-a170-c09ba7752b1d",
   "metadata": {},
   "outputs": [],
   "source": [
    "llm_2 = AzureChatOpenAI(deployment_name=os.environ[\"GPT4_DEPLOYMENT_NAME\"], temperature=0.5, max_tokens=COMPLETION_TOKENS)\n",
    "chain = prompt | llm_2 | output_parser"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "4c83bb17-36d3-4eb6-ae6f-9c68f3033d2b",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/markdown": [
       "Based on the provided context, several medicines and therapeutic strategies are mentioned that may reduce inflammation in the lungs:\n",
       "\n",
       "1. **Chloroquine**: It has been shown to inhibit the release of inflammatory cytokines by human lung explants, suggesting it might mitigate the cytokine storm associated with severe pneumonia caused by coronaviruses.\n",
       "\n",
       "2. **Recombinant IL-22**: Administering recombinant IL-22 in vivo has been shown to reduce inflammation and fluid leak into the lung, indicating its potential to reduce influenza-induced pneumonia.\n",
       "\n",
       "3. **Radiation Therapy (Low doses)**: Low doses of radiation therapy have been reported to increase the immunosuppressive profile of lung macrophages via IL-10 production and IFNγ/IL-6 suppression, which could be a therapeutic strategy to counteract lung inflammation.\n",
       "\n",
       "4. **Pharmacological Treatments**: Strategies include the inhibition of excessive inflammation or manipulation of the physiological derangement causing respiratory failure, potentially allowing for less injurious mechanical ventilation.\n",
       "\n",
       "5. **IL6 Blockade with Tocilizumab**: In selected patients with severe COVID-19, IL6 blockade with tocilizumab could curb the cytokine storm, prevent ICU admission, and the requirement for mechanical ventilation.\n",
       "\n",
       "6. **Glucocorticoids**: Considered effective anti-inflammatory therapies for chronic inflammatory and immune diseases such as asthma, although they may be relatively ineffective in certain patient groups like asthmatic smokers and those with COPD or cystic fibrosis.\n",
       "\n",
       "7. **Escin**: Suggested as a therapeutic opportunity as add-on therapy in acute lung injury (ALI) related to COVID-19 infection due to its anti-inflammatory and anti-viral effects.\n",
       "\n",
       "8. **Anti-inflammatory Therapy in Neonates**: Early post-natal anti-inflammatory therapy could help in preventing the development of chronic lung disease (CLD) in preterm newborns.\n",
       "\n",
       "9. **Inhaled siRNA**: Potential applications of RNA interference (RNAi) in the lungs for the treatment of diseases including inflammatory conditions.\n",
       "\n",
       "10. **Biochanin A**: An anti-inflammatory agent that could be used as an influenza treatment.\n",
       "\n",
       "11. **Tanshinone IIA (TIIA)**: Displayed significant anti-pulmonary fibrosis effects in animal models and may exert protective effects through the ACE-2/ANG-(1-7) axis.\n",
       "\n",
       "12. **Rho-associated protein kinase (ROCK) Inhibitors**: Required for both high-speed migration and straight motion of T cells, which may regulate tissue-sampling during acute lung injury.\n",
       "\n",
       "13. **Noscapine**: Suggested as a possible drug candidate for attenuation of cytokine release associated with SARS-CoV-2.\n",
       "\n",
       "14. **Rapid-acting bronchodilators, systemic corticosteroids, and antibiotics**: Keys to managing exacerbations of chronic obstructive pulmonary disease.\n",
       "\n",
       "15. **Macrolide and Ketolide Antibiotics**: May exert anti-inflammatory effects on the chronically inflamed airways in asthma, in addition to their anti-infective action.\n",
       "\n",
       "16. **Dexamethasone (Corticosteroid)**: Used in a porcine model for SARS, it was found to suppress innate and Th1 cytokines in the lungs, although with potential immunosuppressive effects.\n",
       "\n",
       "17. **α-1 Adrenergic Receptor Antagonists**: Medications that target catecholamine-associated inflammation may prevent cytokine storm syndrome.\n",
       "\n",
       "These medicines and treatments target various mechanisms of inflammation and are at different stages of research and clinical use. It is important to note that their effectiveness and safety may vary based on the specific disease, patient population, and clinical context."
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 19.3 ms, sys: 364 µs, total: 19.7 ms\n",
      "Wall time: 35 s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "try:\n",
    "    display(Markdown(chain.invoke({\"question\": QUESTION, \"context\": ordered_results})))\n",
    "except Exception as e:\n",
    "    print(e)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "92a28869-13c1-463e-8285-9870e6ac1946",
   "metadata": {},
   "source": [
    "#### As we can see, the model selection MATTERS!\n",
    "\n",
    "We will dive deeper into this later, but for now, **look at the diference between GPT3.5 and GPT4, in quality and in response time**."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "417925af-486a-40bc-a290-28c35968c581",
   "metadata": {},
   "source": [
    "# Improving the Prompt and adding citations\n",
    "\n",
    "We could see above that the answer given by GPT3.5 was very simple compared to GPT4, even when the prompt says \"thorough responses to users\". We also could see that there is no citations or references. **How do we know if the answer is grounded on the context or not?**\n",
    "\n",
    "Let's see if these two issues can be improved by Prompt Engineering.<br>\n",
    "On `common/prompts.py` we created a prompt called `DOCSEARCH_PROMPT` check it out!\n",
    "\n",
    "Let's also create a custom Retriever class so we can plug it in easily within the chain building. \n",
    "Note: we can also use the Azure AI Search retriever class [HERE](https://python.langchain.com/docs/integrations/vectorstores/azuresearch), however we want to create a custom Retriever for the following reasons:\n",
    "\n",
    "1) We want to do multi-index searches in one call\n",
    "2) Easier to teach complex concepts of LangChain in this notebook\n",
    "3) We want to use the REST API vs the Python Azure Search SDK"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "bdf31f99-0dfb-423a-81f5-03018e61d9a9",
   "metadata": {},
   "outputs": [],
   "source": [
    "class CustomRetriever(BaseRetriever):\n",
    "    \n",
    "    topK : int\n",
    "    reranker_threshold : int\n",
    "    indexes: List\n",
    "    sas_token: str = None\n",
    "    \n",
    "    def _get_relevant_documents(self, query: str) -> List[Document]:\n",
    "        \n",
    "        ordered_results = get_search_results(query, self.indexes, k=self.topK, \n",
    "                                             reranker_threshold=self.reranker_threshold, \n",
    "                                             sas_token=self.sas_token)\n",
    "        top_docs = []\n",
    "        for key,value in ordered_results.items():\n",
    "            location = value[\"location\"] if value[\"location\"] is not None else \"\"\n",
    "            top_docs.append(Document(page_content=value[\"chunk\"], metadata={\"source\": location, \"score\":value[\"score\"]}))\n",
    "\n",
    "        return top_docs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "19b39c79-c827-4437-b58b-6a6fae53b968",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create the retriever\n",
    "retriever = CustomRetriever(indexes=indexes, topK=k, reranker_threshold=1, sas_token=os.environ['BLOB_SAS_TOKEN'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "c7aa4f58-4791-40a0-80c5-6582e0574579",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "20"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Test retreiver\n",
    "results = retriever.get_relevant_documents(QUESTION)\n",
    "len(results)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "11b6546f-b5c5-4168-97fc-2636c50e41c2",
   "metadata": {},
   "outputs": [],
   "source": [
    "# We can create now a dynamically configurable llm object that can change the model at runtime\n",
    "dynamic_llm = AzureChatOpenAI(deployment_name=os.environ[\"GPT35_DEPLOYMENT_NAME\"], \n",
    "                              temperature=0.5, max_tokens=COMPLETION_TOKENS).configurable_alternatives(\n",
    "    # This gives this field an id\n",
    "    # When configuring the end runnable, we can then use this id to configure this field\n",
    "    ConfigurableField(id=\"model\"),\n",
    "    # This sets a default_key.\n",
    "    # If we specify this key, the default LLM  (initialized above) will be used\n",
    "    default_key=\"gpt35\",\n",
    "    # This adds a new option, with name `gpt4`\n",
    "    gpt4=AzureChatOpenAI(deployment_name=os.environ[\"GPT4_DEPLOYMENT_NAME\"], \n",
    "                         temperature=0.5, max_tokens=COMPLETION_TOKENS),\n",
    "    # You can add more configuration options here\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "id": "d7da2f31-cf5d-4f3a-aad5-67b50b56968e",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Declaration of the chain with the dynamic llm and the new prompt\n",
    "configurable_chain = (\n",
    "    {\n",
    "        \"context\": itemgetter(\"question\") | retriever, # Passes the question to the retriever and the results are assign to context\n",
    "        \"question\": itemgetter(\"question\")\n",
    "    }\n",
    "    | DOCSEARCH_PROMPT  # Passes the input variables above to the prompt template\n",
    "    | dynamic_llm   # Passes the finished prompt to the LLM\n",
    "    | StrOutputParser()  # converts the output (Runnable object) to the desired output (string)\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "id": "b67200e5-d3ae-4c86-9f69-bc7b964ab532",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/markdown": [
       "Chloroquine is a medicine that reduces inflammation in the lungs. It has been shown to inhibit the lipopolysaccharide-induced release of TNF-α, IL-6, CCL2, and CCL3, which are pro-inflammatory cytokines involved in lung inflammation<sup><a href=\"https://doi.org/10.1093/cid/ciaa546\" target=\"_blank\">1</a></sup>. Additionally, chest irradiation at low doses has been found to protect the lung from inflammation by increasing the percentage of NAMs (nerve- and airway-associated macrophages) producing IL-10, which is an anti-inflammatory cytokine, thus leading to lung protection from inflammation<sup><a href=\"https://doi.org/10.1101/2020.05.11.077651\" target=\"_blank\">4</a></sup>. Furthermore, RNA interference (RNAi) has been proposed as a potential treatment for lung inflammation. It is a gene regulatory mechanism that offers a unique opportunity to treat a range of respiratory conditions, including inflammatory and immune conditions<sup><a href=\"https://doi.org/10.1021/mp070048k\" target=\"_blank\">8</a></sup>. Lastly, noscapine, a medication used for the treatment of cough, has been shown to inhibit bradykinin-enhanced cough response, which may help decrease the severe symptoms due to lung damage and attenuate mortality rates<sup><a href=\"https://doi.org/10.1002/ddr.21676\" target=\"_blank\">11</a></sup>.\n",
       "\n",
       "In summary, chloroquine, chest irradiation at low doses, RNA interference, and noscapine are all potential treatments that reduce inflammation in the lungs.\n",
       "\n"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 101 ms, sys: 3.33 ms, total: 104 ms\n",
      "Wall time: 7.32 s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "\n",
    "try:\n",
    "    display(Markdown(configurable_chain.with_config(configurable={\"model\": \"gpt35\"}).invoke({\"question\": QUESTION})))\n",
    "except Exception as e:\n",
    "    print(e)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8661b48d-1e57-4a70-9b0a-cc59f9093267",
   "metadata": {},
   "source": [
    "As seen above, we were able to improve the quality and breath of the answer and add citations with only prompt engineering!\n",
    "\n",
    "Let's try again GPT-4"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "id": "efcfac6b-bac2-40c6-9ded-e4ee38e3093f",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/markdown": [
       "Several medications and approaches have been identified to reduce inflammation in the lungs:\n",
       "\n",
       "1. **Chloroquine**: This drug has been shown to inhibit the release of inflammatory cytokines (TNF-α, IL-6, CCL2, and CCL3) in human lung parenchymal explants, suggesting it might mitigate the cytokine storm associated with severe pneumonia caused by coronaviruses<sup><a href=\"https://doi.org/10.1093/cid/ciaa546\" target=\"_blank\">[1]</a></sup>.\n",
       "\n",
       "2. **Interleukin-22 (IL-22)**: Administration of recombinant IL-22 in vivo reduces inflammation and fluid leak into the lung, indicating its potential role in reducing influenza-induced pneumonia<sup><a href=\"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6917921/\" target=\"_blank\">[2]</a></sup>.\n",
       "\n",
       "3. **Low doses of radiation therapy (RT)**: Low doses of RT have been used to increase IL-10 secretion and decrease IFNγ production, leading to lung protection from inflammation in experimental models<sup><a href=\"https://doi.org/10.1101/2020.05.11.077651\" target=\"_blank\">[3]</a></sup>.\n",
       "\n",
       "4. **Cell-based therapy**: This emerging approach seeks to convert the inflammatory process from an injurious one to a reparative one in the treatment of acute lung injury<sup><a href=\"https://doi.org/10.1055/s-0033-1351119\" target=\"_blank\">[4]</a></sup>.\n",
       "\n",
       "5. **IL6 blockade with tocilizumab**: In selected patients with severe COVID-19, IL6 blockade has been reported to curb the cytokine storm, prevent ICU admission, and the requirement for mechanical ventilation<sup><a href=\"https://doi.org/10.1101/2020.04.20.20061861\" target=\"_blank\">[5]</a></sup>.\n",
       "\n",
       "6. **Glucocorticoids**: These are considered effective anti-inflammatory therapies for chronic inflammatory and immune diseases, although their efficacy may be reduced in certain conditions like in asthmatic smokers, and patients with COPD or cystic fibrosis (CF)<sup><a href=\"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7120595/\" target=\"_blank\">[6]</a></sup>.\n",
       "\n",
       "7. **Escin**: This agent has been suggested for its anti-inflammatory and anti-viral effects in lung injury, potentially as add-on therapy in acute lung injury (ALI) related to COVID-19 infection<sup><a href=\"https://doi.org/10.1002/jcph.1644\" target=\"_blank\">[7]</a></sup>.\n",
       "\n",
       "8. **RNA interference (RNAi)**: Targeted local delivery of RNAi to the lungs via inhalation offers a unique opportunity to treat a range of respiratory conditions<sup><a href=\"https://doi.org/10.1021/mp070048k\" target=\"_blank\">[8]</a></sup>.\n",
       "\n",
       "9. **Traditional Chinese herbal medicines**: Some Chinese herbs have been shown to reduce inflammation in the context of influenza and other diseases<sup><a href=\"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2892358/\" target=\"_blank\">[9]</a></sup>.\n",
       "\n",
       "10. **Tanshinone IIA (TIIA)**: This traditional Chinese herbal abstraction has shown anti-pulmonary fibrosis effects in animal models<sup><a href=\"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4003542/\" target=\"_blank\">[10]</a></sup>.\n",
       "\n",
       "11. **Noscapine**: A cough medicine that has been suggested to inhibit bradykinin enhanced cough response, which might help attenuate mortality rates by decreasing severe symptoms due to lung damage<sup><a href=\"https://doi.org/10.1002/ddr.21676\" target=\"_blank\">[11]</a></sup>.\n",
       "\n",
       "12. **Macrolide and ketolide antibiotics**: These antibiotics may exert anti-inflammatory effects on the chronically inflamed airways in addition to their anti-infective action<sup><a href=\"https://www.ncbi.nlm.nih.gov/pubmed/17353114/\" target=\"_blank\">[12]</a></sup>.\n",
       "\n",
       "It is important to note that the effectiveness and safety of these treatments can vary, and their use should be guided by healthcare professionals based on the specific condition and individual patient circumstances."
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 155 ms, sys: 15.2 ms, total: 170 ms\n",
      "Wall time: 1min 23s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "try:\n",
    "    display(Markdown(configurable_chain.with_config(configurable={\"model\": \"gpt4\"}).invoke({\"question\": QUESTION})))\n",
    "except Exception as e:\n",
    "    print(e)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c93b000a-8d0a-4574-be7c-48d26dfb4c70",
   "metadata": {},
   "source": [
    "#### As you can see the answer from GPT4 is richer, and includes all the relevant chunks. GPT3.5 tends to focus in the first and last chunks only"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6690453b-a9b1-4907-bd43-8c6b3ecba26e",
   "metadata": {},
   "source": [
    "## Adding Streaming to improve user experience and performance\n",
    "\n",
    "It is obvious by now that **GPT4 answers are better quality than GPT3.5**. None are incorrect, but GPT4 is better at understanding the context, following the prompt instructions and on giving a comprehensive answer.\n",
    "\n",
    "One way to make GPT4 look faster is to stream the answer, so the user can see the response as it is typed. To do this, we just simply need to call the method `stream` instead of `invoke`. More on Streaming and Callbacks in later notebooks, but for now, this is one simple way to do it:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "id": "6d250c88-5984-438f-8390-1d93756048ab",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Several medicines and therapeutic approaches have been identified to reduce inflammation in the lungs, according to the provided documents:\n",
      "\n",
      "1. **Chloroquine**: It has been found to inhibit the lipopolysaccharide-induced release of pro-inflammatory cytokines such as TNF-α, IL-6, CCL2, and CCL3 in human lung parenchymal explants, suggesting that it might mitigate the cytokine storm associated with severe pneumonia caused by coronaviruses<sup><a href=\"https://doi.org/10.1093/cid/ciaa546\" target=\"_blank\">[1]</a></sup>.\n",
      "\n",
      "2. **Interleukin-22 (IL-22)**: This cytokine has been shown to protect the lung during infection by promoting tight junction formation and reducing pulmonary inflammation during H1N1 infection<sup><a href=\"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6917921/\" target=\"_blank\">[2]</a></sup>.\n",
      "\n",
      "3. **Low doses of radiation therapy (RT)**: Irradiation of lung macrophages with low doses of RT has been shown to increase the secretion of the anti-inflammatory cytokine IL-10 and decrease the production of pro-inflammatory cytokines such as IL-6 and IFNγ, suggesting a possible mechanism to mitigate lung inflammation in conditions like COVID-19-induced ARDS<sup><a href=\"https://doi.org/10.1101/2020.05.11.077651\" target=\"_blank\">[3]</a></sup>.\n",
      "\n",
      "4. **Cell-based therapy**: Rather than inhibiting inflammation, cell-based therapy seeks to convert the inflammatory process from an injurious one to a reparative one<sup><a href=\"https://doi.org/10.1055/s-0033-1351119\" target=\"_blank\">[4]</a></sup>.\n",
      "\n",
      "5. **IL6 blockade with tocilizumab**: In a study, tocilizumab was demonstrated to potentially curb the \"cytokine storm\", prevent ICU admission, and the requirement for mechanical ventilation in highly selected patients with severe, rapidly deteriorating COVID-19-related pneumonia<sup><a href=\"https://doi.org/10.1101/2020.04.20.20061861\" target=\"_blank\">[5]</a></sup>.\n",
      "\n",
      "6. **Purinergic receptor agonists and antagonists**: These are being explored as new therapeutic approaches for major respiratory diseases<sup><a href=\"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7120595/\" target=\"_blank\">[6]</a></sup>.\n",
      "\n",
      "7. **Escin**: This agent has been suggested to have potent anti-inflammatory and anti-viral effects in lung injury and could be a therapeutic opportunity as add-on therapy in ALI related to COVID-19 infection<sup><a href=\"https://doi.org/10.1002/jcph.1644\" target=\"_blank\">[7]</a></sup>.\n",
      "\n",
      "8. **Anti-inflammatory cytokine IL-10**: Early post-natal anti-inflammatory therapy with IL-10 could help in preventing the development of chronic lung disease (CLD) in preterm newborns<sup><a href=\"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7102381/\" target=\"_blank\">[8]</a></sup>.\n",
      "\n",
      "9. **RNA interference (RNAi)**: Targeted delivery of RNAi to the lungs via inhalation offers a unique opportunity to treat a range of respiratory conditions, including inflammatory and immune conditions<sup><a href=\"https://doi.org/10.1021/mp070048k\" target=\"_blank\">[9]</a></sup>.\n",
      "\n",
      "10. **Herbal medicines**: Some herbal medicines like Angelica sinensis, Salvia miltiorrhiza, and biochanin A have been shown to reduce inflammation and could be considered for influenza treatment<sup><a href=\"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2892358/\" target=\"_blank\">[10]</a></sup>.\n",
      "\n",
      "11. **Tanshinone IIA (TIIA)**: This traditional Chinese herbal abstraction has displayed significant anti-pulmonary fibrosis effects in animal models<sup><a href=\"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4003542/\" target=\"_blank\">[11]</a></sup>.\n",
      "\n",
      "12. **Glucocorticoids**: These are considered effective anti-inflammatory therapies for chronic inflammatory and immune diseases like asthma, although they may be ineffective in some patients such as asthmatic smokers and those with COPD or cystic fibrosis<sup><a href=\"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7120595/\" target=\"_blank\">[6]</a></sup>.\n",
      "\n",
      "13. **Macrolide antibiotics**: These may exert anti-inflammatory effects on the chronically inflamed airways in asthma in addition to their anti-infective action<sup><a href=\"https://www.ncbi.nlm.nih.gov/pubmed/17353114/\" target=\"_blank\">[12]</a></sup>.\n",
      "\n",
      "14. **Dexamethasone (DEX)**: This corticosteroid was used in a study with pigs infected with porcine respiratory coronavirus (PRCV), and it was found to decrease innate and Th1 cytokines in bronchoalveolar lavage (BAL) and serum<sup><a href=\"https://doi.org/10.1128/jvi.02190-07\" target=\"_blank\">[13]</a></sup>.\n",
      "\n",
      "15. **Medications targeting catecholamine-associated inflammation**: These may prevent cytokine storm syndrome associated with COVID-19 and other diseases<sup>[14]</sup>. However, the source for this information was not provided in the extracted parts.\n",
      "\n",
      "Please note that while some of these therapies have shown promise, their safety and efficacy may vary, and they may not be approved or recommended for all patients or conditions. Always consult with healthcare professionals before starting any new treatment."
     ]
    }
   ],
   "source": [
    "for chunk in configurable_chain.with_config(configurable={\"model\": \"gpt4\"}).stream({\"question\": QUESTION}):\n",
    "    print(chunk, end=\"\", flush=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f347373a-a5be-473d-b64e-0f6b6dbcd0e0",
   "metadata": {},
   "source": [
    "# Summary\n",
    "##### By using OpenAI, the answers to user questions are way better than taking just the results from Azure AI Search. So the summary is:\n",
    "- Utilizing Azure AI Search, we conduct a multi-index hybrid search that identifies the top chunks of documents from each index.\n",
    "- Subsequently, Azure OpenAI utilizes these extracted chunks as context, comprehends the content, and employs it to deliver optimal answers.\n",
    "- Best of two worlds!\n",
    "\n",
    "##### Important observations on this notebook:\n",
    "\n",
    "1) Answers with GPT-3.5 are less quality but way faster\n",
    "2) Answers with GPT-3.5 sometimes failed on provinding citations in the right format\n",
    "3) Answers with GPT-4 are great quality but way slower\n",
    "4) Answers with GPT-4 always provide good and diverse citations in the right format\n",
    "5) Streaming the answers improves the user experience big time!"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fdc6e2fe-1c34-4952-99ad-14940f022379",
   "metadata": {},
   "source": [
    "# NEXT\n",
    "In the next notebook, we are going to see how we can treat complex and large documents separately, also using Vector Search"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3.10 - SDK v2",
   "language": "python",
   "name": "python310-sdkv2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.11"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}