03-Quering-AOpenAI.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "d59d527f-1100-45ff-b051-5f7c9029d94d",
   "metadata": {},
   "source": [
    "# Queries with and without Azure OpenAI"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "eb9a9444-dc90-4fc3-aea7-8ee918301aba",
   "metadata": {},
   "source": [
    "So far, you have your Search Engine loaded **from two different data sources in two diferent indexes**, on this notebook we are going to try some example queries and then use Azure OpenAI service to see if we can get a good answer for the user query.\n",
    "\n",
    "The idea is that a user can ask a question about Computer Science (first datasource/index) or about Covid (second datasource/index), and the engine will respond accordingly.\n",
    "This **Multi-Index** demo, mimics the scenario where a company loads multiple type of documents of different types and about completly different topics and the search engine must respond with the most relevant results."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "71f6c7e3-9037-4b1e-ae17-1deaa27b9c08",
   "metadata": {},
   "source": [
    "## Set up variables"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "8e50b404-a061-49e7-a3c7-c6eabc98ff0f",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import os\n",
    "import urllib\n",
    "import requests\n",
    "import random\n",
    "import json\n",
    "from collections import OrderedDict\n",
    "from IPython.display import display, HTML, Markdown\n",
    "from typing import List\n",
    "from operator import itemgetter\n",
    "\n",
    "# LangChain Imports needed\n",
    "from langchain_openai import AzureChatOpenAI\n",
    "from langchain_openai import AzureOpenAIEmbeddings\n",
    "from langchain_core.prompts import ChatPromptTemplate\n",
    "from langchain_core.output_parsers import StrOutputParser\n",
    "from langchain_core.retrievers import BaseRetriever\n",
    "from langchain_core.callbacks import CallbackManagerForRetrieverRun\n",
    "from langchain_core.documents import Document\n",
    "from langchain_core.runnables import ConfigurableField\n",
    "\n",
    "\n",
    "# Our own libraries needed\n",
    "from common.prompts import DOCSEARCH_PROMPT\n",
    "from common.utils import get_search_results\n",
    "\n",
    "from dotenv import load_dotenv\n",
    "load_dotenv(\"credentials.env\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "2f2c22f8-79ab-405c-95e8-77a1978e53bc",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Setup the Payloads header\n",
    "headers = {'Content-Type': 'application/json','api-key': os.environ['AZURE_SEARCH_KEY']}\n",
    "params = {'api-version': os.environ['AZURE_SEARCH_API_VERSION']}"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9297d29b-1f61-4dce-858e-bf4272172dba",
   "metadata": {},
   "source": [
    "## Multi-Index Search queries"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "5a46e2d3-298a-4708-83de-9e108b1a117a",
   "metadata": {
    "scrolled": true,
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Text-based Indexes that we are going to query (from Notebook 01 and 02)\n",
    "index1_name = \"cogsrch-index-files\"\n",
    "index2_name = \"cogsrch-index-csv\"\n",
    "indexes = [index2_name, index1_name]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1c62ebb2-d7be-4bfb-b1ba-4db86c11839a",
   "metadata": {},
   "source": [
    "Try questions that you think might be answered or addressed in computer science papers in 2020-2021 or that can be addressed by medical publications about COVID in 2020-2021. Try comparing the results with the open version of ChatGPT.<br>\n",
    "The idea is that the answers using Azure OpenAI only looks at the information contained on these publications.\n",
    "\n",
    "**Example Questions you can ask**:\n",
    "- What is CLP?\n",
    "- How Markov chains work?\n",
    "- What are some examples of reinforcement learning?\n",
    "- What are the main risk factors for Covid-19?\n",
    "- What medicine reduces inflamation in the lungs?\n",
    "- Why Covid doesn't affect kids that much compared to adults?\n",
    "- Does chloroquine really works against covid?\n",
    "- Who won the 1994 soccer world cup? # This question should yield no answer if the system is correctly grounded"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "b9b53c14-19bd-451f-aa43-7ad27ccfeead",
   "metadata": {},
   "outputs": [],
   "source": [
    "QUESTION = \"What medicine reduces inflamation in the lungs?\""
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f6d925eb-7f9c-429e-a62a-4c37d7702caf",
   "metadata": {},
   "source": [
    "### Search on both indexes individually and aggragate results\n",
    "\n",
    "#### **Note**: \n",
    "In order to standarize the indexes, **there must be 6 mandatory fields present on each index**: `id, title, name, location, chunk, chunkVector`. This is so that each document can be treated the same along the code. Also, **all indexes must have a semantic configuration**.\n",
    "\n",
    "We are going to use Hybrid Queries: Text + Vector Search combined for optimal results!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "faf2e30f-e71f-4533-ab52-27d048b80a89",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200\n",
      "Index: cogsrch-index-csv Results Found: 70150, Results Returned: 10\n",
      "200\n",
      "Index: cogsrch-index-files Results Found: 118874, Results Returned: 10\n"
     ]
    }
   ],
   "source": [
    "agg_search_results = dict()\n",
    "k = 10\n",
    "\n",
    "for index in indexes:\n",
    "    search_payload = {\n",
    "        \"search\": QUESTION, # Text query\n",
    "        \"select\": \"id, title, name, location, chunk\",\n",
    "        \"queryType\": \"semantic\",\n",
    "        \"vectorQueries\": [{\"text\": QUESTION, \"fields\": \"chunkVector\", \"kind\": \"text\", \"k\": k}], # Vector query\n",
    "        \"semanticConfiguration\": \"my-semantic-config\",\n",
    "        \"captions\": \"extractive\",\n",
    "        \"answers\": \"extractive\",\n",
    "        \"count\":\"true\",\n",
    "        \"top\": k\n",
    "    }\n",
    "\n",
    "    r = requests.post(os.environ['AZURE_SEARCH_ENDPOINT'] + \"/indexes/\" + index + \"/docs/search\",\n",
    "                     data=json.dumps(search_payload), headers=headers, params=params)\n",
    "    print(r.status_code)\n",
    "\n",
    "    search_results = r.json()\n",
    "    agg_search_results[index]=search_results\n",
    "    print(\"Index:\", index, \"Results Found: {}, Results Returned: {}\".format(search_results['@odata.count'], len(search_results['value'])))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "255c40f5-d836-480c-8c68-06a2282c8146",
   "metadata": {},
   "outputs": [],
   "source": [
    "# agg_search_results"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b7fd0fe5-4ee0-42e2-a920-72b93a407389",
   "metadata": {
    "tags": []
   },
   "source": [
    "### Display the top results (from both searches) based on the score"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "9e938337-602d-4b61-8141-b8c92a5d91da",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<h4>Top Answers</h4>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<h5>Answer - score: 0.92</h5>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "Importantly, we show that administering recombinant IL-22 in vivo reduces inflammation and fluid leak into the lung. Taken together, our results demonstrate the IL-22/IL-22BP axis is a potential targetable pathway for reducing influenza-induced pneumonia..\u0000"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<h4>Top Results</h4>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<h5><a href=\"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6917921/?sv=2022-11-02&ss=b&srt=sco&sp=rl&se=2026-01-03T02:11:44Z&st=2024-01-02T18:11:44Z&spr=https&sig=ngrEqvqBVaxyuSYqgPVeF%2B9c0fXLs94v3ASgwg7LDBs%3D\">Targeting the IL-22/IL-22BP axis enhances tight junctions and reduces inflammation during influenza infection</a> - score: 3.29</h5>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "Importantly, we show that administering recombinant IL-22 in vivo reduces inflammation and fluid leak into the lung. Taken together, our results demonstrate the IL-22/IL-22BP axis is a potential targetable pathway for reducing influenza-induced pneumonia..\u0000"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<h5><a href=\"https://doi.org/10.1055/s-0033-1351119; https://www.ncbi.nlm.nih.gov/pubmed/23934717/?sv=2022-11-02&ss=b&srt=sco&sp=rl&se=2026-01-03T02:11:44Z&st=2024-01-02T18:11:44Z&spr=https&sig=ngrEqvqBVaxyuSYqgPVeF%2B9c0fXLs94v3ASgwg7LDBs%3D\">Treatment of acute lung injury: current and emerging pharmacological therapies.</a> - score: 3.03</h5>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "Two main pharmacological treatment strategies have arisen-the attempted inhibition of excessive inflammation or the manipulation of the resulting physiological derangement causing respiratory failure. Additionally, such interventions may allow the delivery of less injurious mechanical ventilation."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<h5><a href=\"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7120595/?sv=2022-11-02&ss=b&srt=sco&sp=rl&se=2026-01-03T02:11:44Z&st=2024-01-02T18:11:44Z&spr=https&sig=ngrEqvqBVaxyuSYqgPVeF%2B9c0fXLs94v3ASgwg7LDBs%3D\">Therapeutic Applications</a> - score: 2.74</h5>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "For instance, glucocorticoids are considered the most effective anti-inflammatory therapies for chronic inflammatory and immune diseases, such as asthma. But they are relatively ineffective in asthmatic smokers, and patients with chronic obstructive pulmonary disease (COPD) or cystic fibrosis (CF)."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<h5><a href=\"https://doi.org/10.1002/jcph.1644; https://www.ncbi.nlm.nih.gov/pubmed/32441805/?sv=2022-11-02&ss=b&srt=sco&sp=rl&se=2026-01-03T02:11:44Z&st=2024-01-02T18:11:44Z&spr=https&sig=ngrEqvqBVaxyuSYqgPVeF%2B9c0fXLs94v3ASgwg7LDBs%3D\">Severe Acute Lung Injury Related to COVID-19 Infection: A Review and the Possible Role for Escin.</a> - score: 2.58</h5>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "In this article reviewing literature data related to the use of escin, an agent having potent anti-inflammatory and anti-viral effects in lung injury, we suggest that it could represent a therapeutic opportunity as add-on therapy in ALI related to COVID-19 infection..\u0000"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<h5><a href=\"https://doi.org/10.1021/mp070048k; https://www.ncbi.nlm.nih.gov/pubmed/18491918/?sv=2022-11-02&ss=b&srt=sco&sp=rl&se=2026-01-03T02:11:44Z&st=2024-01-02T18:11:44Z&spr=https&sig=ngrEqvqBVaxyuSYqgPVeF%2B9c0fXLs94v3ASgwg7LDBs%3D\">Inhalable siRNA: potential as a therapeutic agent in the lungs.</a> - score: 2.46</h5>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "In this timely review we look at the potential applications of RNAi in the lungs for the treatment of a range of diseases including inflammatory and immune conditions, cystic fibrosis, infectious disease and cancer. In 2006 Alnylam initiated the first phase 1 clinical study of an inhaled siRNA for the treatment of respiratory syncytial virus."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<h5><a href=\"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2892358/?sv=2022-11-02&ss=b&srt=sco&sp=rl&se=2026-01-03T02:11:44Z&st=2024-01-02T18:11:44Z&spr=https&sig=ngrEqvqBVaxyuSYqgPVeF%2B9c0fXLs94v3ASgwg7LDBs%3D\">Using Complementary and Alternative Medicines to Target the Host Response during Severe Influenza</a> - score: 2.46</h5>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "Biochanin A, a ligand of the peroxisome proliferator activated receptors (PPAR) alpha and gamma and the active isoflavone in Trifolium pratense (red clover), has anti-inflammatory properties, and thus could be used as an influenza treatment."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<h5><a href=\"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5647329/?sv=2022-11-02&ss=b&srt=sco&sp=rl&se=2026-01-03T02:11:44Z&st=2024-01-02T18:11:44Z&spr=https&sig=ngrEqvqBVaxyuSYqgPVeF%2B9c0fXLs94v3ASgwg7LDBs%3D\">ROCK regulates the intermittent mode of interstitial T cell migration in inflamed lungs</a> - score: 2.2</h5>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "ROCK regulates the intermittent mode of interstitial T cell migration in inflamed lungs. Effector T cell migration through tissues can enable control of infection or mediate inflammatory damage."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<h5><a href=\"https://doi.org/10.1002/ddr.21676; https://www.ncbi.nlm.nih.gov/pubmed/32337769/?sv=2022-11-02&ss=b&srt=sco&sp=rl&se=2026-01-03T02:11:44Z&st=2024-01-02T18:11:44Z&spr=https&sig=ngrEqvqBVaxyuSYqgPVeF%2B9c0fXLs94v3ASgwg7LDBs%3D\">Noscapine, a possible drug candidate for attenuation of cytokine release associated with SARS-CoV-2.</a> - score: 2.2</h5>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "Inactivation of ACE2 during virus fusion into the host cell may be one of the underlying reasons for intense immunological reaction seen in the lung tissue. This overreaction is probably mediated through the bradykinin receptor activation."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<h5><a href=\"https://www.ncbi.nlm.nih.gov/pubmed/15339574/?sv=2022-11-02&ss=b&srt=sco&sp=rl&se=2026-01-03T02:11:44Z&st=2024-01-02T18:11:44Z&spr=https&sig=ngrEqvqBVaxyuSYqgPVeF%2B9c0fXLs94v3ASgwg7LDBs%3D\">[Development of the integrated TCM hepatology with western medicine].</a> - score: 1.93</h5>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "The treatment for severe acute respiratory syndrome (SARS) that happened epidemically this spring confirmed again that the integrated traditional Chinese and western medicine had its clinical advantage and vitality, and indicated that this integration could benefit the pathologic understanding and therapeutic strategies for the diseases."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<h5><a href=\"None?sv=2022-11-02&ss=b&srt=sco&sp=rl&se=2026-01-03T02:11:44Z&st=2024-01-02T18:11:44Z&spr=https&sig=ngrEqvqBVaxyuSYqgPVeF%2B9c0fXLs94v3ASgwg7LDBs%3D\">COVID-19 in the heart and the lungs: could we \"Notch\" the inflammatory storm?</a> - score: 1.84</h5>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "The Notch signalling is a major regulator of cardiovascular function and it is also implicated in several biological processes mediating viral infections. In this report we discuss the possibility to target Notch signalling to prevent SARS-CoV-2 infection and interfere with the progression of COVID-19- associated heart and lungs disease..\u0000"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<h5><a href=\"https://datasetsgptsmartsearch.blob.core.windows.net/arxivcs/pdf/0601/0601030v1.pdf?sv=2022-11-02&ss=b&srt=sco&sp=rl&se=2026-01-03T02:11:44Z&st=2024-01-02T18:11:44Z&spr=https&sig=ngrEqvqBVaxyuSYqgPVeF%2B9c0fXLs94v3ASgwg7LDBs%3D\">arXiv:cs/0601030v1  [cs.DL]  9 Jan 2006</a> - score: 1.22</h5>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "isi category isi category name ds critical care medicine ff emergency medicine fy dentistry, oral surgery & medicine oi integrative & complementary medicine op medicine, legal py medicine, general & internal qa medicine, research & experimental vy radiology, nuclear medicine & medical imaging yu tropical medicine  table 4: isi subject categories …"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<h5><a href=\"https://datasetsgptsmartsearch.blob.core.windows.net/arxivcs/pdf/0207/0207088v1.pdf?sv=2022-11-02&ss=b&srt=sco&sp=rl&se=2026-01-03T02:11:44Z&st=2024-01-02T18:11:44Z&spr=https&sig=ngrEqvqBVaxyuSYqgPVeF%2B9c0fXLs94v3ASgwg7LDBs%3D\">arXiv:cs/0207088v1  [cs.LO]  25 Jul 2002</a> - score: 1.01</h5>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "Three experts in medicine provided information related to the diagnosis of two diseases: disease-1 and disease-2. The information concerning John and Mary can be paraphrased as follows:  — Expert I (a clinician):  Symptom-1 and symptom-2 together imply disease-1. Symptom-1 and symptom-3 together imply disease-2."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "display(HTML('<h4>Top Answers</h4>'))\n",
    "\n",
    "for index,search_results in agg_search_results.items():\n",
    "\n",
    "    for result in search_results['@search.answers']:\n",
    "        if result['score'] > 0.5: # Show answers that are at least 50% of the max possible score=1\n",
    "            display(HTML('<h5>' + 'Answer - score: ' + str(round(result['score'],2)) + '</h5>'))\n",
    "            display(HTML(result['text']))\n",
    "\n",
    "            \n",
    "print(\"\\n\\n\")\n",
    "display(HTML('<h4>Top Results</h4>'))\n",
    "\n",
    "content = dict()\n",
    "ordered_content = OrderedDict()\n",
    "\n",
    "\n",
    "for index,search_results in agg_search_results.items():\n",
    "    for result in search_results['value']:\n",
    "        if result['@search.rerankerScore'] > 1:# Show answers that are at least 25% of the max possible score=4\n",
    "            content[result['id']]={\n",
    "                                    \"title\": result['title'],\n",
    "                                    \"chunk\": result['chunk'], \n",
    "                                    \"name\": result['name'], \n",
    "                                    \"location\": result['location'] ,\n",
    "                                    \"caption\": result['@search.captions'][0]['text'],\n",
    "                                    \"score\": result['@search.rerankerScore'],\n",
    "                                    \"index\": index\n",
    "                                    }\n",
    "    \n",
    "#After results have been filtered we will Sort and add them as an Ordered list\\n\",\n",
    "for id in sorted(content, key= lambda x: content[x][\"score\"], reverse=True):\n",
    "    ordered_content[id] = content[id]\n",
    "    url = str(ordered_content[id]['location']) + os.environ['BLOB_SAS_TOKEN']\n",
    "    title = str(ordered_content[id]['title']) if (ordered_content[id]['title']) else ordered_content[id]['name']\n",
    "    score = str(round(ordered_content[id]['score'],2))\n",
    "    display(HTML('<h5><a href=\"'+ url + '\">' + title + '</a> - score: '+ score + '</h5>'))\n",
    "    display(HTML(ordered_content[id]['caption']))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "52a6d3e6-afb2-4fa7-96d3-69bc2373ded5",
   "metadata": {},
   "source": [
    "### Comments on Query results"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "84e02227-6a92-4944-86f8-6c1e38d90fe4",
   "metadata": {},
   "source": [
    "As seen above the semantic re-ranking feature of Azure AI Search service is good. It gives answers (sometimes) and also the top results with the corresponding file and the paragraph where the answers is possible located.\n",
    "\n",
    "Let's see if we can make this better with Azure OpenAI"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8df3e6d4-9a09-4b0f-b328-238738ccfaec",
   "metadata": {},
   "source": [
    "# Using Azure OpenAI\n",
    "\n",
    "To use OpenAI to get a better answer to our question, the thought process is simple: let's **give the answer and the content of the documents from the search result to the GPT model as context and let it provide a better response**. This is what RAG (Retreival Augmented Generation) is about.\n",
    "\n",
    "Now, before we do this, we need to understand a few things first:\n",
    "\n",
    "1) Chainning and Prompt Engineering\n",
    "2) Embeddings\n",
    "\n",
    "We will use a library call **LangChain** that wraps a lot of boiler plate code.\n",
    "Langchain is one library that does a lot of the prompt engineering for us under the hood, for more information see [here](https://python.langchain.com/en/latest/index.html)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "eea62a7d-7e0e-4a93-a89c-20c96560c665",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Set the ENV variables that Langchain needs to connect to Azure OpenAI\n",
    "os.environ[\"OPENAI_API_VERSION\"] = os.environ[\"AZURE_OPENAI_API_VERSION\"]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "325d9138-2250-4f6b-bc88-50d7957f8d33",
   "metadata": {},
   "source": [
    "**Important Note**: Starting now, we will utilize OpenAI models. Please ensure that you have deployed the following models within the Azure OpenAI portal:\n",
    "\n",
    "- text-embedding-ada-002 (or newer)\n",
    "- gpt-35-turbo (1106 or newer)\n",
    "- gpt-4-turbo (1106 or newer)\n",
    "\n",
    "Reference for Azure OpenAI models (regions, limits, dimensions, etc): [HERE](https://learn.microsoft.com/en-us/azure/ai-services/openai/concepts/models)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0e7c720e-ece1-45ad-9d01-2dfd15c182bb",
   "metadata": {},
   "source": [
    "## A gentle intro to chaining LLMs and prompt engineering"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2bcd7028-5a6c-4296-8c85-4f420d408d69",
   "metadata": {},
   "source": [
    "Chains refer to sequences of calls - whether to an LLM, a tool, or a data preprocessing step.\n",
    "\n",
    "Azure OpenAI is a type of LLM (provider) that you can use but there are others like Cohere, Huggingface, etc.\n",
    "\n",
    "Chains can be simple (i.e. Generic) or specialized (i.e. Utility).\n",
    "\n",
    "* Generic — A single LLM is the simplest chain. It takes an input prompt and the name of the LLM and then uses the LLM for text generation (i.e. output for the prompt).\n",
    "\n",
    "Here’s an example:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "13df9247-e784-4e04-9475-55e672efea47",
   "metadata": {},
   "outputs": [],
   "source": [
    "COMPLETION_TOKENS = 1500\n",
    "llm = AzureChatOpenAI(deployment_name=os.environ[\"GPT35_DEPLOYMENT_NAME\"], \n",
    "                      temperature=0, \n",
    "                      max_tokens=COMPLETION_TOKENS)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "a3b55adb-6f98-4f15-b67a-9fbba5820560",
   "metadata": {},
   "outputs": [],
   "source": [
    "output_parser = StrOutputParser()\n",
    "prompt = ChatPromptTemplate.from_messages([\n",
    "    (\"system\", \"You are an assistant that give thorough responses to users.\"),\n",
    "    (\"user\", \"{input}. Give your response in {language}\")\n",
    "])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6417d052-0035-4635-93e8-2bd3ec50d796",
   "metadata": {},
   "source": [
    "The | symbol is similar to a unix pipe operator, which chains together the different components feeds the output from one component as input into the next component."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "77a37e60-a1ef-4750-a1ec-9e4fe5ba07fa",
   "metadata": {},
   "outputs": [],
   "source": [
    "chain = prompt | llm | output_parser"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "6be6b4df-ee2c-4a0c-8ad3-a672d70f4f8d",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/markdown": [
       "El medicamento que reduce la inflamación en los pulmones se llama corticosteroide. Los corticosteroides son medicamentos que se utilizan para reducir la inflamación en los pulmones y pueden ser recetados para tratar enfermedades como el asma y la enfermedad pulmonar obstructiva crónica (EPOC). Es importante consultar a un médico para obtener la dosis y el tratamiento adecuados."
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 28.2 ms, sys: 4.52 ms, total: 32.7 ms\n",
      "Wall time: 860 ms\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "display(Markdown(chain.invoke({\"input\": QUESTION, \"language\": \"Spanish\"})))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cd8539d0-a538-4368-82c3-5f91d8370f1e",
   "metadata": {},
   "source": [
    "**Note**: this is the first time you use OpenAI in this Accelerator, so if you get a Resource not found error, is most likely because the name of your OpenAI model deployment is different than the environmental variable set above `os.environ[\"GPT35_DEPLOYMENT_NAME\"]`"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "50ed014c-0c6b-448c-b995-fe7970b92ad5",
   "metadata": {},
   "source": [
    "Great!!, now you know how to create a simple prompt and use a chain in order to answer a general question using ChatGPT knowledge!. \n",
    "\n",
    "It is important to note that we rarely use generic chains as standalone chains. More often they are used as building blocks for Utility chains (as we will see next). Also important to notice is that we are NOT using our documents or the result of the Azure Search yet, just the knowledge of ChatGPT on the data it was trained on."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "12c48038-b1af-4228-8ffb-720e554fd3b2",
   "metadata": {
    "tags": []
   },
   "source": [
    "**The second type of Chains are Utility:**\n",
    "\n",
    "* Utility — These are specialized chains, comprised of many building blocks to help solve a specific task. For example, LangChain supports some end-to-end chains (such as `create_retrieval_chain` for QnA Doc retrieval, Summarization, etc).\n",
    "\n",
    "We will build our own specific chain in this workshop for digging deeper and solve our use case of enhancing the results of Azure AI Search."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b0454ddb-44d8-4fa9-929a-5e5563dd28f8",
   "metadata": {},
   "source": [
    "\n",
    "But before dealing with the utility chain needed, let's first review the concept of Embeddings and Vector Search and RAG. \n",
    "\n",
    "## Embeddings and Vector Search\n",
    "\n",
    "From the Azure OpenAI documentation ([HERE](https://learn.microsoft.com/en-us/azure/ai-services/openai/how-to/embeddings?tabs=python)), An embedding is a special format of data representation that can be easily utilized by machine learning models and algorithms. The embedding is an information dense representation of the semantic meaning of a piece of text. Each embedding is a vector of floating point numbers, such that the distance between two embeddings in the vector space is correlated with semantic similarity between two inputs in the original format. For example, if two texts are similar, then their vector representations should also be similar. \n",
    "\n",
    "### Why Do We Need Vectors?\n",
    "\n",
    "Vectors are essential for several reasons:\n",
    "\n",
    "- **Semantic Richness**: They convert the semantic meaning of text into mathematical vectors, capturing nuances that simple keyword searches miss. This makes them incredibly powerful for understanding and processing language.\n",
    "- **Human-like Searching**: Searching using vector distances mimics the human approach to finding information based on context and meaning, rather than relying solely on exact word matches.\n",
    "- **Efficiency in Scale**: Vector representations allow for efficient handling and searching of large datasets. By reducing complex text to numerical vectors, algorithms can quickly sift through vast amounts of information.\n",
    "\n",
    "### Understanding LLM Tokens' Context Limitation\n",
    "\n",
    "Large Language Models (LLMs) like GPT come with a token limit for each input, which poses a challenge when dealing with lengthy documents or extensive data sets. This limitation restricts the model's ability to understand and generate responses based on the full context of the information provided. It becomes crucial, therefore, to devise strategies that can effectively manage and circumvent this limitation to leverage the full power of LLMs.\n",
    "\n",
    "To address this challenge, the solution incorporates several key steps:\n",
    "\n",
    "1. **Segmenting Documents**: Breaking down large documents into smaller, manageable segments.\n",
    "2. **Vectorization of Chunks**: Converting these segments into vectors, making them compatible with vector-based search techniques.\n",
    "3. **Hybrid Search**: Employing both vector and text search methods to pinpoint the most relevant segments in relation to the query.\n",
    "4. **Optimal Context Provision**: Presenting the LLM with the most pertinent segments, ensuring a balance between detail and brevity to stay within token limits.\n",
    "\n",
    "\n",
    "Our ultimate goal is to rely solely on vector indexes and hybrid searchs (vector + text). While it is possible to manually code parsers with OCR for various file types and develop a scheduler to synchronize data with the index, there is a more efficient alternative: **Azure AI Search has automated chunking strategies and vectorization**.\n",
    "\n",
    "It's important to note that **document segmentation and vectorization have already been completed in AI Azure Search**, as seen in the `ordered_content` dictionary. This pre-processing step simplifies subsequent operations, ensuring rapid response times and adherence to the token limits of the chosen OpenAI model.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "80e79235-3d8b-4713-9336-5004cc4a1556",
   "metadata": {},
   "source": [
    "So really, our only job now is to make sure that the results from the Azure AI Search queries fit on the LLM context size, and then let it do its magic."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "12682a1b-df92-49ce-a638-7277103f6cb3",
   "metadata": {},
   "outputs": [],
   "source": [
    "index_name = \"cogsrch-index-files\"\n",
    "index2_name = \"cogsrch-index-csv\"\n",
    "indexes = [index1_name, index2_name]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "78a6d6a7-18ef-45b2-a216-3c1f50006593",
   "metadata": {},
   "source": [
    "In order to not duplicate code, we have put many of the code used above into functions. These functions are in the `common/utils.py` and `common/prompts.py` files. This way we can use these functios in the app that we will build later.\n",
    "\n",
    "`get_search_results()` do the multi-index search and returns the combined ordered list of documents/chunks."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "3bccca45-d1dd-476f-b109-a528b857b6b3",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of results: 20\n"
     ]
    }
   ],
   "source": [
    "k = 20  # play with this parameter and see the quality of the final answer\n",
    "ordered_results = get_search_results(QUESTION, indexes, k=k, reranker_threshold=1)\n",
    "print(\"Number of results:\",len(ordered_results))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "7714f38a-daaa-4fc5-a95a-dd025d153216",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Uncomment the below line if you want to inspect the ordered results\n",
    "# ordered_results"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "235d4238-df6e-40eb-ab38-4fe6db614acd",
   "metadata": {},
   "source": [
    "Now let's create a Prompt Template that will ground the response only in the chunks retrieve by our hybrid AI Search."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "f86ed786-aca0-4e25-947b-d9cf3a82665c",
   "metadata": {},
   "outputs": [],
   "source": [
    "template = \"\"\"Answer the question thoroughly, based **ONLY** on the following context:\n",
    "{context}\n",
    "\n",
    "Question: {question}\n",
    "\"\"\"\n",
    "prompt = ChatPromptTemplate.from_template(template)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "25cba3d1-b5ab-4e28-96b3-ef923d99dc9f",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/markdown": [
       "Chloroquine is a medicine that has been shown to reduce inflammation in the lungs. It has been found to inhibit the lipopolysaccharide-induced release of inflammatory cytokines such as TNF-α, IL-6, CCL2, and CCL3. This suggests that chloroquine might mitigate the cytokine storm associated with severe pneumonia caused by coronaviruses. Therefore, chloroquine is a potential medicine for reducing inflammation in the lungs."
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 17 ms, sys: 247 µs, total: 17.3 ms\n",
      "Wall time: 1.32 s\n"
     ]
    }
   ],
   "source": [
    "%%time \n",
    "# Creation of our custom chain\n",
    "chain = prompt | llm | output_parser\n",
    "\n",
    "try:\n",
    "    display(Markdown(chain.invoke({\"question\": QUESTION, \"context\": ordered_results})))\n",
    "except Exception as e:\n",
    "    print(e)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "406c2b91-6752-4ea2-b95c-d1a52dbdd62b",
   "metadata": {},
   "source": [
    "### From GPT-3.5 to GPT-4\n",
    "\n",
    "Now let's see how the response changes if we change to GPT-4\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "f01705f1-7194-452b-a170-c09ba7752b1d",
   "metadata": {},
   "outputs": [],
   "source": [
    "llm_2 = AzureChatOpenAI(deployment_name=os.environ[\"GPT4_DEPLOYMENT_NAME\"], temperature=0.5, max_tokens=COMPLETION_TOKENS)\n",
    "chain = prompt | llm_2 | output_parser"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "4c83bb17-36d3-4eb6-ae6f-9c68f3033d2b",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/markdown": [
       "Based on the provided context, several medicines and therapeutic strategies have been mentioned that have the potential to reduce inflammation in the lungs:\n",
       "\n",
       "1. **Chloroquine** - In human lung parenchymal explants, chloroquine at a concentration of 100 μM inhibited the release of inflammatory cytokines such as TNF-α, IL-6, CCL2, and CCL3, suggesting that chloroquine might mitigate the cytokine storm associated with severe pneumonia caused by coronaviruses.\n",
       "\n",
       "2. **IL-22/IL-22BP axis targeting** - Enhancing the environment of Interleukin-22 (IL-22) by using recombinant IL-22 or by knocking out IL-22-binding protein (IL-22BP) in mice reduced pulmonary inflammation and promoted tight junction formation, which could reduce influenza-induced pneumonia.\n",
       "\n",
       "3. **Low doses of radiation therapy (RT)** - Low doses of RT (0.5-1 Gy) increased IL-10 secretion and decreased IFNγ and IL-6 production in human lung macrophages, suggesting a potential therapeutic strategy to counteract lung inflammation.\n",
       "\n",
       "4. **Pharmacological treatments for acute lung injury** - The text mentions two main strategies: inhibition of excessive inflammation and manipulation of physiological derangements causing respiratory failure. Cell-based therapy is also an emerging approach.\n",
       "\n",
       "5. **IL-6 blockade with tocilizumab** - In selected patients with severe COVID-19-related pneumonia, tocilizumab was shown to curb the \"cytokine storm,\" prevent ICU admission, and reduce the requirement for mechanical ventilation.\n",
       "\n",
       "6. **Glucocorticoids** - Although glucocorticoids are considered effective anti-inflammatory therapies for diseases like asthma, their effectiveness can vary depending on conditions like smoking status and the specific disease (e.g., COPD or cystic fibrosis).\n",
       "\n",
       "7. **Escin** - Escin, which has anti-inflammatory and anti-viral effects, is suggested as a potential add-on therapy in acute lung injury (ALI) related to COVID-19 infection.\n",
       "\n",
       "8. **Anti-inflammatory therapy in neonates** - Early post-natal anti-inflammatory therapy could help prevent the development of chronic lung disease (CLD) in preterm newborns, as they may be unable to activate anti-inflammatory cytokine IL-10.\n",
       "\n",
       "9. **Inhaled siRNA** - Targeted delivery of RNA interference (RNAi) to the lungs via inhalation was considered for treating respiratory conditions due to its anti-inflammatory properties.\n",
       "\n",
       "10. **Complementary and alternative medicines** - Biochanin A and traditional Chinese herbs like Angelica sinensis and Salvia miltiorrhiza have been mentioned for their anti-inflammatory properties and potential use in influenza treatment.\n",
       "\n",
       "11. **Tanshinone IIA (TIIA)** - TIIA has been shown to attenuate bleomycin-induced pulmonary fibrosis and inflammation in rats, involving the angiotensin-converting enzyme 2 (ACE-2)/angiotensin-(1-7) axis.\n",
       "\n",
       "12. **Rho-associated protein kinase (ROCK) inhibitors** - ROCK is required for high-speed migration and straight motion of T cells, which can control infection or mediate inflammatory damage in lung tissues.\n",
       "\n",
       "13. **Noscapine** - As a cough medicine, noscapine could potentially attenuate the intense immunological reaction in the lung tissue mediated through bradykinin receptor activation.\n",
       "\n",
       "14. **Systemic corticosteroids and antibiotics** - These are key to managing exacerbations of chronic obstructive pulmonary disease (COPD) and could also help in preventing exacerbations.\n",
       "\n",
       "15. **Macrolide and ketolide antibiotics** - These antibiotics may have a part to play in the management of asthma due to their anti-inflammatory effects on the airways.\n",
       "\n",
       "16. **Dexamethasone (DEX)** - Used in a porcine model of respiratory coronavirus infection, DEX decreased innate and Th1 cytokines in bronchoalveolar lavage and serum but may have potential immunosuppressive effects.\n",
       "\n",
       "17. **α-1 adrenergic receptor antagonists** - Medications targeting catecholamine-associated inflammation may prevent cytokine storm syndrome associated with diseases like COVID-19.\n",
       "\n",
       "It is important to note that the context provided does not offer exhaustive information on the safety, efficacy, dosing, or regulatory approval status of these treatments. Clinical decisions should always be based on the most current and comprehensive medical knowledge and guidelines."
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 20.8 ms, sys: 687 µs, total: 21.5 ms\n",
      "Wall time: 1min 3s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "try:\n",
    "    display(Markdown(chain.invoke({\"question\": QUESTION, \"context\": ordered_results})))\n",
    "except Exception as e:\n",
    "    print(e)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "92a28869-13c1-463e-8285-9870e6ac1946",
   "metadata": {},
   "source": [
    "#### As we can see, the model selection MATTERS!\n",
    "\n",
    "We will dive deeper into this later, but for now, **look at the diference between GPT3.5 and GPT4, in quality and in response time**."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "417925af-486a-40bc-a290-28c35968c581",
   "metadata": {},
   "source": [
    "# Improving the Prompt and adding citations\n",
    "\n",
    "We could see above that the answer given by GPT3.5 was very simple compared to GPT4, even when the prompt says \"thorough responses to users\". We also could see that there is no citations or references. **How do we know if the answer is grounded on the context or not?**\n",
    "\n",
    "Let's see if these two issues can be improved by Prompt Engineering.<br>\n",
    "On `common/prompts.py` we created a prompt called `DOCSEARCH_PROMPT` check it out!\n",
    "\n",
    "**Let's also create a custom Retriever class** so we can plug it in easily within the chain building. \n",
    "Note: we can also use the Azure AI Search retriever class [HERE](https://python.langchain.com/docs/integrations/vectorstores/azuresearch), however we want to create a custom Retriever for the following reasons:\n",
    "\n",
    "1) We want to do multi-index searches in one call\n",
    "2) Easier to teach complex concepts of LangChain in this notebook\n",
    "3) We want to use the REST API vs the Python Azure Search SDK"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "bdf31f99-0dfb-423a-81f5-03018e61d9a9",
   "metadata": {},
   "outputs": [],
   "source": [
    "class CustomRetriever(BaseRetriever):\n",
    "    \n",
    "    topK : int\n",
    "    reranker_threshold : int\n",
    "    indexes: List\n",
    "    sas_token: str = None\n",
    "    \n",
    "    def _get_relevant_documents(self, query: str) -> List[Document]:\n",
    "        \n",
    "        ordered_results = get_search_results(query, self.indexes, k=self.topK, \n",
    "                                             reranker_threshold=self.reranker_threshold, \n",
    "                                             sas_token=self.sas_token)\n",
    "        top_docs = []\n",
    "        for key,value in ordered_results.items():\n",
    "            location = value[\"location\"] if value[\"location\"] is not None else \"\"\n",
    "            top_docs.append(Document(page_content=value[\"chunk\"], metadata={\"source\": location, \"score\":value[\"score\"]}))\n",
    "\n",
    "        return top_docs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "19b39c79-c827-4437-b58b-6a6fae53b968",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create the retriever\n",
    "retriever = CustomRetriever(indexes=indexes, topK=k, reranker_threshold=1, sas_token=os.environ['BLOB_SAS_TOKEN'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "c7aa4f58-4791-40a0-80c5-6582e0574579",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "20"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Test retreiver\n",
    "results = retriever.get_relevant_documents(QUESTION)\n",
    "len(results)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "id": "11b6546f-b5c5-4168-97fc-2636c50e41c2",
   "metadata": {},
   "outputs": [],
   "source": [
    "# We can create now a dynamically configurable llm object that can change the model at runtime\n",
    "dynamic_llm = AzureChatOpenAI(deployment_name=os.environ[\"GPT35_DEPLOYMENT_NAME\"], \n",
    "                              temperature=0.5, max_tokens=COMPLETION_TOKENS).configurable_alternatives(\n",
    "    # This gives this field an id\n",
    "    # When configuring the end runnable, we can then use this id to configure this field\n",
    "    ConfigurableField(id=\"model\"),\n",
    "    # This sets a default_key.\n",
    "    # If we specify this key, the default LLM  (initialized above) will be used\n",
    "    default_key=\"gpt35\",\n",
    "    # This adds a new option, with name `gpt4`\n",
    "    gpt4=AzureChatOpenAI(deployment_name=os.environ[\"GPT4_DEPLOYMENT_NAME\"], \n",
    "                         temperature=0.5, max_tokens=COMPLETION_TOKENS),\n",
    "    # You can add more configuration options here\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "id": "d7da2f31-cf5d-4f3a-aad5-67b50b56968e",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Declaration of the chain with the dynamic llm and the new prompt\n",
    "configurable_chain = (\n",
    "    {\n",
    "        \"context\": itemgetter(\"question\") | retriever, # Passes the question to the retriever and the results are assign to context\n",
    "        \"question\": itemgetter(\"question\")\n",
    "    }\n",
    "    | DOCSEARCH_PROMPT  # Passes the input variables above to the prompt template\n",
    "    | dynamic_llm   # Passes the finished prompt to the LLM\n",
    "    | StrOutputParser()  # converts the output (Runnable object) to the desired output (string)\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "id": "b67200e5-d3ae-4c86-9f69-bc7b964ab532",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/markdown": [
       "On human lung parenchymal explants, chloroquine concentration clinically achievable in the lung (100 μM) inhibited the lipopolysaccharide-induced release of TNF-α (by 76%), IL-6 (by 68%), CCL2 (by 72%) and CCL3 (by 67%). Beside its antiviral activity, chloroquine might also mitigate the cytokine storm associated with severe pneumonia caused by coronaviruses [[0]](https://doi.org/10.1093/cid/ciaa546; https://www.ncbi.nlm.nih.gov/pubmed/32382733/?sv=2022-11-02&ss=b&srt=sco&sp=rl&se=2026-01-03T02:11:44Z&st=2024-01-02T18:11:44Z&spr=https&sig=ngrEqvqBVaxyuSYqgPVeF%2B9c0fXLs94v3ASgwg7LDBs%3D). Chest irradiation at equivalent doses below 1 Gy has been used successfully in the past to treat pneumonia. However, after the onset of effective antimicrobial agents, the use of low doses in the treatment of patients has been discontinued and the involved mechanism remained unknown. The objective of this study was to determine the mechanism by which low doses of radiation therapy (RT) protect the lung from inflammation. Our results show that irradiation of Poly(I:C)-stimulated human lung macrophages in vitro using low doses of RT (0.5-1 Gy) increased IL-10 secretion and decreased IFNγ production in the culture supernatant. The percentage of human lung macrophages producing IL-6 was also decreased in vitro with low doses of RT. Furthermore, using experimental models (LPS- and Poly(I :C)-induced lung inflammation), we showed that chest irradiation using low doses of RT significantly increased the percentage of NAMs producing IL-10, leading to lung protection from inflammation [[3]](https://doi.org/10.1101/2020.05.11.077651?sv=2022-11-02&ss=b&srt=sco&sp=rl&se=2026-01-03T02:11:44Z&st=2024-01-02T18:11:44Z&spr=https&sig=ngrEqvqBVaxyuSYqgPVeF%2B9c0fXLs94v3ASgwg7LDBs%3D). Noscapine, a medication used for the treatment of cough, has been shown to inhibit bradykinin enhanced cough response in man. As it is already marketed in a number of countries as a cough medicine, even for children, a suitable formulation with all the required licenses is available that can be rapidly utilized in preliminary trials [[11]](https://doi.org/10.1002/ddr.21676; https://www.ncbi.nlm.nih.gov/pubmed/32337769/?sv=2022-11-02&ss=b&srt=sco&sp=rl&se=2026-01-03T02:11:44Z&st=2024-01-02T18:11:44Z&spr=https&sig=ngrEqvqBVaxyuSYqgPVeF%2B9c0fXLs94v3ASgwg7LDBs%3D).\n",
       "\n",
       "These medications have shown to reduce inflammation in the lungs:\n",
       "1. Chloroquine\n",
       "2. Low doses of radiation therapy (RT)\n",
       "3. Noscapine\n",
       "\n",
       "These medications have been shown to mitigate lung inflammation and could be potential targets for treating lung inflammation."
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 111 ms, sys: 0 ns, total: 111 ms\n",
      "Wall time: 11.3 s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "\n",
    "try:\n",
    "    display(Markdown(configurable_chain.with_config(configurable={\"model\": \"gpt35\"}).invoke({\"question\": QUESTION})))\n",
    "except Exception as e:\n",
    "    print(e)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8661b48d-1e57-4a70-9b0a-cc59f9093267",
   "metadata": {},
   "source": [
    "As seen above, we were able to improve the quality and breath of the answer and add citations with only prompt engineering!\n",
    "\n",
    "#### Let's try again, but with GPT-4"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "id": "efcfac6b-bac2-40c6-9ded-e4ee38e3093f",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/markdown": [
       "Several medicines have been identified that may reduce inflammation in the lungs:\n",
       "\n",
       "1. **Chloroquine:** It has been shown to inhibit the release of pro-inflammatory cytokines such as TNF-α, IL-6, CCL2, and CCL3 in human lung parenchymal explants, suggesting that it could mitigate the cytokine storm associated with severe pneumonia caused by coronaviruses [[1]](https://doi.org/10.1093/cid/ciaa546).\n",
       "\n",
       "2. **Interleukin-22 (IL-22):** This cytokine, regulated by the soluble receptor IL-22-binding protein (IL-22BP), is critical in protecting the lung during infection. IL-22 administration in vivo has been shown to reduce inflammation and fluid leak into the lung, indicating its potential as a targetable pathway for reducing influenza-induced pneumonia [[2]](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6917921).\n",
       "\n",
       "3. **Low doses of radiation therapy (RT):** Low doses of RT have been used to increase IL-10 secretion and decrease IFNγ production by lung macrophages, thereby regulating lung inflammation and favoring anti-inflammatory cytokine secretion. This suggests that low doses of RT could be used to mitigate lung inflammatory processes, such as in COVID-19-induced ARDS [[3]](https://doi.org/10.1101/2020.05.11.077651).\n",
       "\n",
       "4. **Glucocorticoids:** While considered the most effective anti-inflammatory therapies for chronic inflammatory and immune diseases like asthma, they may be relatively ineffective in certain patient groups, such as asthmatic smokers and patients with COPD or cystic fibrosis. However, they are still explored for new therapeutic approaches for major respiratory diseases [[6]](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7120595).\n",
       "\n",
       "5. **IL6 blockade with tocilizumab:** In selected patients with severe, rapidly deteriorating COVID-19-related pneumonia, IL6 blockade may curb the cytokine storm, prevent ICU admission, and reduce the requirement for mechanical ventilation [[5]](https://doi.org/10.1101/2020.04.20.20061861).\n",
       "\n",
       "6. **Escin:** This agent has potent anti-inflammatory and anti-viral effects and has been suggested as a therapeutic opportunity as add-on therapy in acute lung injury (ALI) related to COVID-19 infection [[7]](https://doi.org/10.1002/jcph.1644).\n",
       "\n",
       "7. **Anti-inflammatory cytokine IL-10:** Infants with lung inflammation may be unable to activate IL-10, and early post-natal anti-inflammatory therapy could help in preventing the development of chronic lung disease (CLD) [[8]](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7102381).\n",
       "\n",
       "8. **RNA interference (RNAi):** RNAi, a gene regulatory mechanism, is being explored as a potential therapeutic agent for a range of respiratory conditions including inflammatory and immune conditions, cystic fibrosis, infectious disease, and cancer [[9]](https://doi.org/10.1021/mp070048k).\n",
       "\n",
       "9. **Herbal medicine:** Some herbal medications have been reported to have anti-inflammatory properties, although there is a need for awareness about potential side effects and drug-induced lung injury (DLI) [[16]](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7123307).\n",
       "\n",
       "10. **Medications targeting catecholamine-associated inflammation:** These medications may prevent cytokine storm syndrome associated with COVID-19 and other diseases, though the document does not specify which medications these are [[17]](source).\n",
       "\n",
       "It is important to note that the safety and efficacy of these treatments can vary, and they may not be suitable for all patients or conditions. Clinical trials and medical advice are crucial for determining the best course of action for individual patients."
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 120 ms, sys: 9.65 ms, total: 129 ms\n",
      "Wall time: 50.9 s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "try:\n",
    "    display(Markdown(configurable_chain.with_config(configurable={\"model\": \"gpt4\"}).invoke({\"question\": QUESTION})))\n",
    "except Exception as e:\n",
    "    print(e)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c93b000a-8d0a-4574-be7c-48d26dfb4c70",
   "metadata": {},
   "source": [
    "#### As you can see the answer from GPT4 is richer, and includes all the relevant chunks. GPT3.5 tends to focus in the first and last chunks only"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6690453b-a9b1-4907-bd43-8c6b3ecba26e",
   "metadata": {},
   "source": [
    "## Adding Streaming to improve user experience and performance\n",
    "\n",
    "It is obvious by now that **GPT4 answers are better quality than GPT3.5**. None are incorrect, but GPT4 is better at understanding the context, following the prompt instructions and on giving a comprehensive answer.\n",
    "\n",
    "One way to make GPT4 look faster is to stream the answer, so the user can see the response as it is typed. To do this, we just simply need to call the method `stream` instead of `invoke`. More on Streaming and Callbacks in later notebooks, but for now, this is one simple way to do it:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "id": "6d250c88-5984-438f-8390-1d93756048ab",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Several medications and treatments have been identified to reduce inflammation in the lungs:\n",
      "\n",
      "1. Chloroquine: In human lung parenchymal explants, a clinically achievable concentration of chloroquine (100 μM) inhibited the release of pro-inflammatory cytokines like TNF-α, IL-6, CCL2, and CCL3, which suggests that it may mitigate the cytokine storm associated with severe pneumonia caused by coronaviruses[[1]](https://doi.org/10.1093/cid/ciaa546).\n",
      "\n",
      "2. Interleukin-22 (IL-22): The IL-22/IL-22BP axis is critical in protecting the lung during infections such as influenza. In IL-22BP-knockout mice, a pro-IL-22 environment reduced pulmonary inflammation during H1N1 infection and promoted tight junction formation, which protects the lung. Administering recombinant IL-22 in vivo reduced inflammation and fluid leak into the lung[[2]](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6917921).\n",
      "\n",
      "3. Low doses of radiation therapy (RT): Low doses of RT (0.5-1 Gy) have been shown to protect the lung from inflammation by increasing IL-10 secretion and decreasing IFNγ production in Poly(I:C)-stimulated human lung macrophages in vitro. It also increased the percentage of nerve- and airway-associated macrophages (NAMs) producing IL-10 in experimental models of lung inflammation[[3]](https://doi.org/10.1101/2020.05.11.077651).\n",
      "\n",
      "4. Tocilizumab: An IL6 blockade with tocilizumab in selected patients with severe, rapidly deteriorating COVID-19-related pneumonia showed promise in curbing the cytokine storm, preventing ICU admission, and the requirement for mechanical ventilation[[5]](https://doi.org/10.1101/2020.04.20.20061861).\n",
      "\n",
      "5. Glucocorticoids: While glucocorticoids are effective anti-inflammatory therapies for chronic inflammatory and immune diseases like asthma, they are relatively ineffective in asthmatic smokers, and patients with chronic obstructive pulmonary disease (COPD) or cystic fibrosis (CF)[[6]](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7120595).\n",
      "\n",
      "6. Escin: Escin has potent anti-inflammatory and anti-viral effects in lung injury and has been suggested as a therapeutic opportunity as add-on therapy in acute lung injury (ALI) related to COVID-19 infection[[7]](https://doi.org/10.1002/jcph.1644).\n",
      "\n",
      "7. Anti-inflammatory cytokine IL-10: Preterm newborns with lung inflammation may benefit from early post-natal anti-inflammatory therapy, as they may be unable to activate the anti-inflammatory cytokine IL-10[[8]](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7102381).\n",
      "\n",
      "8. Noscapine: This medication used for the treatment of cough has been shown to inhibit bradykinin enhanced cough response in humans and may help attenuate mortality rates by decreasing severe symptoms due to lung damage[[9]](https://doi.org/10.1002/ddr.21676).\n",
      "\n",
      "9. Macrolide and ketolide antibiotics: These antibiotics may have a role in the management of asthma by exerting anti-inflammatory effects on the chronically inflamed airways in addition to their anti-infective action[[10]](https://www.ncbi.nlm.nih.gov/pubmed/17353114).\n",
      "\n",
      "10. Herbal medications: Herbal medicine can also have anti-inflammatory effects, but clinicians should be aware of drug-induced lung injury (DLI) symptoms caused by herbal medication[[11]](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7123307).\n",
      "\n",
      "Please note that the effectiveness and safety of these treatments can vary depending on the specific condition and patient, and some of them are still under investigation or have been used historically. Always consult with a healthcare professional before starting any new treatment for lung inflammation."
     ]
    }
   ],
   "source": [
    "for chunk in configurable_chain.with_config(configurable={\"model\": \"gpt4\"}).stream({\"question\": QUESTION}):\n",
    "    print(chunk, end=\"\", flush=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f347373a-a5be-473d-b64e-0f6b6dbcd0e0",
   "metadata": {},
   "source": [
    "# Summary\n",
    "##### By using OpenAI, the answers to user questions are way better than taking just the results from Azure AI Search. So the summary is:\n",
    "- Utilizing Azure AI Search, we conduct a multi-index hybrid search that identifies the top chunks of documents from each index.\n",
    "- Subsequently, Azure OpenAI utilizes these extracted chunks as context, comprehends the content, and employs it to deliver optimal answers.\n",
    "- Best of two worlds!\n",
    "\n",
    "##### Important observations on this notebook:\n",
    "\n",
    "1) Answers with GPT-3.5 are less quality but way faster\n",
    "2) Answers with GPT-3.5 sometimes failed on provinding citations in the right format\n",
    "3) Answers with GPT-4 are great quality but way slower\n",
    "4) Answers with GPT-4 always provide good and diverse citations in the right format\n",
    "5) Streaming the answers improves the user experience big time!"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fdc6e2fe-1c34-4952-99ad-14940f022379",
   "metadata": {},
   "source": [
    "# NEXT\n",
    "In the next notebook, we are going to see how we can treat complex and large documents separately, also using Vector Search"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3.10 - SDK v2",
   "language": "python",
   "name": "python310-sdkv2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.11"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}