notebooks/chat_qa.ipynb

{
 "cells": [
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Get answers from ChatGPT with sources\n",
    "\n",
    "The purpose of this notebook is to demonstrate how to get answers from ChatGPT with sources.\n",
    "\n",
    "I have run pipeline for 11 questions related to machine learning research which I am most familiar with. \n",
    "\n",
    "This will help me evaluate the quality of answers from ChatGPT \n",
    "- how much it hallucinates\n",
    "- how much it is able to answer the question."
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here is what is happening for each query -\n",
    "\n",
    "1. Get the top-20 papers for each question from Semantic Scholar\n",
    "2. Using the abstract of these 20 papers get the summary of the papers using Pegasus summarizer\n",
    "3. Use the summary of the papers as context to get the answer from ChatGPT"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "from utils import *\n",
    "from tqdm import tqdm \n",
    "from revChatGPT.V1 import Chatbot\n",
    "from IPython.display import Markdown, display\n",
    "import torch\n",
    "from transformers import PegasusForConditionalGeneration, PegasusTokenizer\n",
    "tqdm.pandas()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "# this is needed for getting summaries of the abstracts\n",
    "\n",
    "model_name = \"tuner007/pegasus_summarizer\"\n",
    "torch_device = \"cuda\" if torch.cuda.is_available() else \"cpu\"\n",
    "tokenizer_summarizer = PegasusTokenizer.from_pretrained(model_name)\n",
    "model_summarizer = PegasusForConditionalGeneration.from_pretrained(model_name).to(torch_device)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "def get_response(input_text):\n",
    "    batch = tokenizer_summarizer(\n",
    "        [input_text],\n",
    "        truncation=True,\n",
    "        padding=\"longest\",\n",
    "        max_length=1024,\n",
    "        return_tensors=\"pt\",\n",
    "    ).to(torch_device)\n",
    "    gen_out = model_summarizer.generate(\n",
    "        **batch, max_length=256, num_beams=5, num_return_sequences=1, temperature=1.5\n",
    "    )\n",
    "    output_text = tokenizer_summarizer.batch_decode(gen_out, skip_special_tokens=True)\n",
    "    return output_text[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|██████████| 9/9 [00:38<00:00,  4.25s/it]\n"
     ]
    }
   ],
   "source": [
    "query1 = \"How can we improve the interpretability and transparency of complex machine learning models, and what are the implications for ensuring ethical and responsible AI development?\"\n",
    "# search and re-rank for the query\n",
    "df = get_results(query1)\n",
    "df[\"tldr\"] = df[\"abstract\"].progress_apply(get_response)\n",
    "\n",
    "# rerank the results based on the tldr similarity\n",
    "df, query1 = rerank(df, query1, 'tldr')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "# create prompt with the top 8 results\n",
    "prompt = answer_question_chatgpt(df, \n",
    "                                 query1, \n",
    "                                 k=8, \n",
    "                                 instructions=\"Instructions: Using the provided web search results, write a comprehensive reply to the given query. If you find a result relevant make sure to cite the result using [[number](URL)] notation after the reference. End your answer with a summary. A\\nQuery:\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/markdown": [
       "## Question\n",
       "\n"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "### How can we improve the interpretability and transparency of complex machine learning models, and what are the implications for ensuring ethical and responsible AI development?\n",
       "\n"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "## ChatGPT Response: \n",
       "\n",
       " Complex machine learning (ML) models often suffer from a lack of interpretability and transparency, which poses ethical challenges in their deployment. One potential solution to this issue is the adoption of Explainable AI (XAI) techniques [[1]]. XAI can help provide insights into how an AI system arrives at its decision-making process, enabling humans to understand and audit the reasoning behind the algorithm. Moreover, it can also help build trust in AI and support ethical and responsible AI development.\n",
       "\n",
       "However, the use of historical data to train machine learning models can also perpetuate existing biases and health care disparities in the data [[2]][[7]]. As such, there is a need to carefully consider the training data used for ML models to ensure that they are representative and unbiased. Data governance also plays an important role in ensuring that the appropriate policies and procedures are in place to ensure ethical and responsible data use [[6]].\n",
       "\n",
       "Another challenge in ensuring ethical and responsible AI development is the lack of traditional medical ethics when dealing with AI-based technologies [[3]]. Deep learning algorithms used in AI models may not be transparent, and their decision-making processes may be complex and difficult to interpret. This can raise concerns about accountability and fairness, particularly in healthcare applications. Human-machine teaming can be an effective approach to addressing these challenges by involving human experts in the decision-making process and providing a checks-and-balances framework for the AI system [[5]].\n",
       "\n",
       "Overall, improving the interpretability and transparency of complex machine learning models is crucial for ensuring ethical and responsible AI development. This can be achieved through the adoption of XAI techniques, careful consideration of training data, and the use of human-machine teaming. Data governance also plays a critical role in ensuring that appropriate policies and procedures are in place to support ethical and responsible data use. [[1][2][3][5][6][7]]"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [1] [A Survey on Explainable Artificial Intelligence Techniques and Challenges](https://www.semanticscholar.org/paper/f185c5bbecaf2b4f54516bc4696f7a8ad1e289e6) - 2021 IEEE 25th International Enterprise Distributed Object Computing Workshop (EDOCW), 2021.0"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [2] [Ensuring Fairness in Machine Learning to Advance Health Equity](https://www.semanticscholar.org/paper/82a5a84528a7ca0409f75e2211a3b33a217e9bac) - Annals of Internal Medicine, 2018.0"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [3] [When the frameworks don’t work: data protection, trust and artificial intelligence](https://www.semanticscholar.org/paper/b762071b683696e97b1fde6efec4396b7dc71987) - Journal of Medical Ethics, 2022.0"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [4] [Legal Governance of Brain Data Derived from Artificial Intelligence](https://www.semanticscholar.org/paper/325c692b0ff876e188190453cb9cf8a1c59142d6) - Voices in Bioethics, 2021.0"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [5] [Human-Machine Teaming and Its Legal and Ethical Implications](https://www.semanticscholar.org/paper/bf6a727c5f4c3e3e2ca69d1c195087df4e172132) - , 2020.0"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [6] [AI in healthcare: data governance challenges](https://www.semanticscholar.org/paper/d49abbcae8908b99caa817e1fb9d81652a84b735) - Journal of Hospital Management and Health Policy, 2020.0"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [7] [The Next Era of Biomedical Research](https://www.semanticscholar.org/paper/9aed9e4b665d14de49aa6d175ed8b6bd321764c3) - Voices in Bioethics, 2021.0"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [8] [Corporate Digital Responsibility New Challenges to the Social Sciences](https://www.semanticscholar.org/paper/2ea68e7aa26280ebff3bbe54f4e606a96685264d) - International Journal of Research in e-Learning, 2019.0"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# get response from chatgpt\n",
    "chatbot = Chatbot(config=constants.CHATGPT_CREDS)\n",
    "\n",
    "response = \"\"\n",
    "\n",
    "for data in chatbot.ask(\n",
    "  prompt\n",
    "):\n",
    "    response = data[\"message\"]\n",
    "\n",
    "# print the response variable  as ipython markdown\n",
    "display(Markdown(f\"## Question\\n\\n\"))\n",
    "\n",
    "display(Markdown(f\"### {query1}\\n\\n\"))\n",
    "display(Markdown(f\"## ChatGPT Response: \\n\\n {response}\"))\n",
    "print_papers(df, k=8)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|██████████| 19/19 [01:23<00:00,  4.38s/it]\n"
     ]
    },
    {
     "data": {
      "text/markdown": [
       "## Question\n",
       "\n"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "### How can we improve the robustness and security of machine learning models to adversarial attacks, and what strategies can be used to detect and prevent these attacks?\n",
       "\n"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "## ChatGPT Response: \n",
       "\n",
       " Machine learning models are becoming increasingly vulnerable to adversarial attacks, which are designed to manipulate or evade the model's output. These attacks can cause significant harm, especially in critical domains such as cybersecurity. In this context, various strategies have been proposed to improve the robustness and security of machine learning models to adversarial attacks.\n",
       "\n",
       "One approach is to model the interaction between machine learning systems and the adversary as a game, and use game theory to provide robustness guarantees for the model [[1]]. Another common approach is to study the vulnerabilities of deep neural networks and develop countermeasures to adversarial examples, which are crafted to mislead the model [[2]].\n",
       "\n",
       "Moreover, researchers have proposed new frameworks to build ensemble models in multi-party computation with formal guarantees of robustness to data poisoning attacks [[3]]. Additionally, some researchers have proposed new detection and mitigation approaches, able to detect and mitigate adversarial attacks against a machine learning system [[4]].\n",
       "\n",
       "Another innovative approach is to use adversarial examples in CAPTCHA challenges, which can be effective against sophisticated machine learning-based bot solvers [[5]]. Furthermore, researchers have investigated the robustness of machine learning and deep learning models against label poisoning and evasion attacks [[6],[7]].\n",
       "\n",
       "Finally, randomization-based approaches have been proposed to improve the robustness of machine learning models against evasion attacks by incorporating randomization into both model training and application [[8]].\n",
       "\n",
       "In summary, several strategies have been proposed to improve the robustness and security of machine learning models against adversarial attacks. These strategies include game theory, countermeasures to adversarial examples, building ensemble models with formal guarantees of robustness, detection and mitigation approaches, using adversarial examples in CAPTCHA challenges, and randomization-based approaches."
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [1] [A Game Theoretic Perspective on Adversarial Machine Learning and Related Cybersecurity Applications](https://www.semanticscholar.org/paper/9761aabce6fea8df63d476066d275bef92c13331) - Game Theory and Machine Learning for Cyber Security, 2021"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [2] [Adversarial Robustness in Deep Learning: From Practices to Theories](https://www.semanticscholar.org/paper/6b92aa960722e83a878ce442485ba0fef1359456) - Knowledge Discovery and Data Mining, 2021"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [3] [SafeNet: Mitigating Data Poisoning Attacks on Private Machine Learning](https://www.semanticscholar.org/paper/18c478f3608b00185e5bbb0296f35edf5d130085) - IACR Cryptology ePrint Archive, 2022"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [4] [eXplainable and Reliable Against Adversarial Machine Learning in Data Analytics](https://www.semanticscholar.org/paper/8845af2c438aa2dcaae7d42fa4709154412bb593) - IEEE Access, 2022"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [5] [Capture the Bot: Using Adversarial Examples to Improve CAPTCHA Robustness to Bot Attacks](https://www.semanticscholar.org/paper/663e6abaa03d4fe900061c2dc844bf7144bde03b) - IEEE Intelligent Systems, 2020"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [6] [Adversarial attacks on malware detection models for smartphones using reinforcement learning: PhD forum abstract](https://www.semanticscholar.org/paper/703132ff0e882cb18f8b7fbad835f564759212e8) - ACM International Conference on Embedded Networked Sensor Systems, 2020"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [7] [Launching Adversarial Attacks against Network Intrusion Detection Systems for IoT](https://www.semanticscholar.org/paper/a21e08f5d6ef216c2411044b8643bcfa2ac2d2d7) - Journal of Cybersecurity and Privacy, 2021"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [8] [Using Randomness to Improve Robustness of Machine-Learning Models Against Evasion Attacks](https://www.semanticscholar.org/paper/e3af2da0b3a021bae5c192a20fd2f6092736b9a8) - ArXiv, 2018"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "query1 = \"How can we improve the robustness and security of machine learning models to adversarial attacks, and what strategies can be used to detect and prevent these attacks?\"\n",
    "# search and re-rank for the query\n",
    "df = get_results(query1)\n",
    "df[\"tldr\"] = df[\"abstract\"].progress_apply(get_response)\n",
    "df, query1 = rerank(df, query1, \"tldr\")\n",
    "prompt = answer_question_chatgpt(\n",
    "    df,\n",
    "    query1,\n",
    "    k=8,\n",
    "    instructions=\"Instructions: Using the provided web search results, write a comprehensive reply to the given query. If you find a result relevant make sure to cite the result using [[number](URL)] notation after the reference. End your answer with a summary. A\\nQuery:\",\n",
    ")\n",
    "\n",
    "chatbot = Chatbot(config=constants.CHATGPT_CREDS)\n",
    "response = \"\"\n",
    "\n",
    "for data in chatbot.ask(prompt):\n",
    "    response = data[\"message\"]\n",
    "\n",
    "# print the response variable  as ipython markdown\n",
    "\n",
    "display(Markdown(f\"## Question\\n\\n\"))\n",
    "\n",
    "display(Markdown(f\"### {query1}\\n\\n\"))\n",
    "display(Markdown(f\"## ChatGPT Response: \\n\\n {response}\"))\n",
    "print_papers(df, k=8)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|██████████| 20/20 [01:33<00:00,  4.66s/it]\n"
     ]
    },
    {
     "data": {
      "text/markdown": [
       "## Question\n",
       "\n"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "### How can we develop machine learning models that can learn from unlabeled data, and what techniques can be used to improve the quality and diversity of unsupervised learning?\n",
       "\n"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "## ChatGPT Response: \n",
       "\n",
       " Unsupervised machine learning is a technique that allows machine learning models to learn from unlabeled data. One approach to improving the quality and diversity of unsupervised learning is to use transfer learning techniques to deal with different training and testing distributions when annotated data is limited in the testing domain [[1](https://www.semanticscholar.org/paper/4e665cdc829064719b778137b62d67ee88aa4f57)]. Another way to improve unsupervised learning is to use unsupervised feature extraction or weight-conditioning variables to recognize abnormal histology images [[4](https://www.semanticscholar.org/paper/7c50d1abc8d3efe53d2267694f8f8ec0a4cec6a2)]. Additionally, clustering can be used as part of semisupervised learning where labels are propagated from a subset of annotated data points to remaining unlabeled data points in a dataset [[6](https://www.semanticscholar.org/paper/9664df66bc61455bcc5a4814efe32b4fe52bc227)]. \n",
       "\n",
       "Furthermore, a new framework has been introduced to make continual learning feasible in unsupervised mode by using pseudo-label obtained from cluster assignments to update the model [[5](https://www.semanticscholar.org/paper/cdb4a6a81f9bfa823052e6f4f43f0132c4b7d368)]. Unsupervised learning techniques such as clustering, generative adversarial networks (GANs) and autoencoders have the potential to address the lack of annotated data in anatomic pathology [[6](https://www.semanticscholar.org/paper/9664df66bc61455bcc5a4814efe32b4fe52bc227)]. Moreover, a denoising framework based on an unsupervised machine-learning technique has been developed to improve the signal-to-noise ratio (SNR) of many seismological datasets. The framework uses an autoencoder neural network with a sparsity constraint to learn the seismic signals from the noisy observations [[8](https://www.semanticscholar.org/paper/b5f0c6c1b825be9c7de6c20354d36542a8f9f583)].\n",
       "\n",
       "In summary, transfer learning, unsupervised feature extraction, semisupervised learning, continual learning, clustering, GANs, autoencoders, and sparsity constraints are all techniques that can be used to improve the quality and diversity of unsupervised learning."
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [1] [On Transfer Learning Techniques for Machine Learning](https://www.semanticscholar.org/paper/4e665cdc829064719b778137b62d67ee88aa4f57) - , 2020"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [2] [Machine Learning and AI in times of COVID Creating quality pottery shape simulations for pre-training Machine Learning Pottery Image Classifiers: processes in the Arch-I-Scan project](https://www.semanticscholar.org/paper/682f8f0dcf55a072a0c1c5941821c42ffb6a31a5) - , 2020"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [3] [On the Benefits of Selectivity in Pseudo-Labeling for Unsupervised Multi-Source-Free Domain Adaptation](https://www.semanticscholar.org/paper/b605186a3498d57df637162b9d0b47abe52eee41) - ArXiv, 2022"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [4] [Biomedical Images Analysis for Disease Diagnosis using Sensor based Machine Learning Model](https://www.semanticscholar.org/paper/7c50d1abc8d3efe53d2267694f8f8ec0a4cec6a2) - 2022 IEEE 2nd Mysore Sub Section International Conference (MysuruCon), 2022"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [5] [Continual Learning From Unlabeled Data Via Deep Clustering](https://www.semanticscholar.org/paper/cdb4a6a81f9bfa823052e6f4f43f0132c4b7d368) - , 2021"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [6] [The Utility of Unsupervised Machine Learning in Anatomic Pathology.](https://www.semanticscholar.org/paper/9664df66bc61455bcc5a4814efe32b4fe52bc227) - American Journal of Clinical Pathology, 2021"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [7] [Security and Machine Learning Adoption in IoT: A Preliminary Study of IoT Developer Discussions](https://www.semanticscholar.org/paper/5ce0d7346de4a60d928af579ede9d34d98bcce2e) - International Workshop on Software Engineering Research & Practices for the Internet of Things, 2021"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [8] [Improving the Signal‐to‐Noise Ratio of Seismological Datasets by Unsupervised Machine Learning](https://www.semanticscholar.org/paper/b5f0c6c1b825be9c7de6c20354d36542a8f9f583) - Seismological Research Letters, 2019"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "query1 = \"How can we develop machine learning models that can learn from unlabeled data, and what techniques can be used to improve the quality and diversity of unsupervised learning?\"\n",
    "# search and re-rank for the query\n",
    "df = get_results(query1)\n",
    "df[\"tldr\"] = df[\"abstract\"].progress_apply(get_response)\n",
    "df, query1 = rerank(df, query1, \"tldr\")\n",
    "prompt = answer_question_chatgpt(\n",
    "    df,\n",
    "    query1,\n",
    "    k=8,\n",
    "    instructions=\"Instructions: Using the provided web search results, write a comprehensive reply to the given query. If you find a result relevant make sure to cite the result using [[number](URL)] notation after the reference. End your answer with a summary. A\\nQuery:\",\n",
    ")\n",
    "\n",
    "chatbot = Chatbot(config=constants.CHATGPT_CREDS)\n",
    "response = \"\"\n",
    "\n",
    "for data in chatbot.ask(prompt):\n",
    "    response = data[\"message\"]\n",
    "\n",
    "# print the response variable  as ipython markdown\n",
    "\n",
    "display(Markdown(f\"## Question\\n\\n\"))\n",
    "\n",
    "display(Markdown(f\"### {query1}\\n\\n\"))\n",
    "display(Markdown(f\"## ChatGPT Response: \\n\\n {response}\"))\n",
    "print_papers(df, k=8)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|██████████| 14/14 [01:04<00:00,  4.59s/it]\n"
     ]
    },
    {
     "data": {
      "text/markdown": [
       "## Question\n",
       "\n"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "### What are the best methods for developing machine learning models that can operate on small and low-power devices, such as smartphones or Internet of Things (IoT) devices?\n",
       "\n"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "## ChatGPT Response: \n",
       "\n",
       " Developing machine learning models that can operate on small and low-power devices, such as smartphones or IoT devices, is a challenging task that requires efficient algorithms and optimized hardware resources. Here are some of the best methods that have been proposed for developing such models:\n",
       "\n",
       "1. Edge computing: Running machine learning models on the edge devices, such as a smart light bulb, has been shown to be an efficient method to reduce the communication and computation overhead [[1](https://www.semanticscholar.org/paper/08425674a5dbb121efdedca5680937b654feb945)].\n",
       "\n",
       "2. k-Nearest Neighbors (kNN) algorithm: kNN algorithm has been shown to perform well in low-power devices, with an average accuracy of 98.8% on five public datasets [[2](https://www.semanticscholar.org/paper/4a3ffb1f5771596960dd106af379b87c53867e78)]. \n",
       "\n",
       "3. Mobile cloud computing (MCC): MCC can be used to develop service applications that leverage cloud resources for processing and storage purposes [[3](https://www.semanticscholar.org/paper/720c94255a4c4624156c97308a4e29ca58618af7)]. \n",
       "\n",
       "4. Internet of Things (IoT) techniques: IoT techniques that integrate data from building instrumentation, personal device sensors, and self-report interfaces can be used to capture rich, longitudinal, ecologically-valid data about the status of office workers and the spaces they occupy [[4](https://www.semanticscholar.org/paper/cd51bbfd51cde0c089d9dfb7d30bfc124d9b7c55)]. \n",
       "\n",
       "5. Smart irrigation system: A smart irrigation system using NodeMCU has been proposed to wirelessly operate a network of irrigation modules by irrigating the field when required by measuring the water content of soil and keep checking the condition of plant using a camera [[8](https://www.semanticscholar.org/paper/d7facc0594b67100695752489bfdcbddd41dba50)].\n",
       "\n",
       "In summary, edge computing, kNN algorithm, MCC, IoT techniques, and smart irrigation systems are some of the best methods for developing machine learning models that can operate on small and low-power devices."
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [1] [Network Intrusion Detection System in a Light Bulb](https://www.semanticscholar.org/paper/08425674a5dbb121efdedca5680937b654feb945) - International Telecommunication Networks and Applications Conference, 2022.0"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [2] [Analysis of Software Bug Prediction and Tracing Models from a Statistical Perspective Using Machine Learning](https://www.semanticscholar.org/paper/4a3ffb1f5771596960dd106af379b87c53867e78) - 2022 2nd International Conference on Intelligent Technologies (CONIT), 2022.0"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [3] [AN EFFECTIVE USAGE OF CLOUD APPLICA- TION DEPLOYMENT IN MOBILE COMPU-](https://www.semanticscholar.org/paper/720c94255a4c4624156c97308a4e29ca58618af7) - , 2020.0"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [4] [Summary for CIFE Seed Proposals for Academic Year 2020-21 Proposal number: 2020-04 Proposal title: Hybrid Physical-Digital Spaces: Transforming the Design, Operation, and Experience of Built Environments to Promote Health and Wellbeing](https://www.semanticscholar.org/paper/cd51bbfd51cde0c089d9dfb7d30bfc124d9b7c55) - , 2020.0"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [5] [Editorial introduction to J.UCS special issue Challenges for Smart Environments - Human-Centered Computing, Data Science, and Ambient Intelligence I](https://www.semanticscholar.org/paper/f7c5dd883caeb875f5be96895b2f505f253141d8) - Journal of universal computer science (Online), 2021.0"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [6] [Guest Editorial: Intelligent Transportation Systems in Smart Cities for Sustainable Environments (SCfSE)](https://www.semanticscholar.org/paper/77ab00ea7d2e4d3957f64056322379ab669a7825) - IET Intelligent Transport Systems, 2020.0"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [7] [IMS2020 Microwave Week Workshops](https://www.semanticscholar.org/paper/2ff58257d18a2cfc43417df163c3222e0e2943f2) - IEEE Microwave Magazine, 2020.0"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [8] [Smart Irrigation System using Internet of Things (IoT) and Machine Learning](https://www.semanticscholar.org/paper/d7facc0594b67100695752489bfdcbddd41dba50) - 2021 9th International Conference on Reliability, Infocom Technologies and Optimization (Trends and Future Directions) (ICRITO), 2021.0"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "query1 = \"What are the best methods for developing machine learning models that can operate on small and low-power devices, such as smartphones or Internet of Things (IoT) devices?\"\n",
    "# search and re-rank for the query\n",
    "df = get_results(query1)\n",
    "df[\"tldr\"] = df[\"abstract\"].progress_apply(get_response)\n",
    "df, query1 = rerank(df, query1, \"tldr\")\n",
    "prompt = answer_question_chatgpt(\n",
    "    df,\n",
    "    query1,\n",
    "    k=8,\n",
    "    instructions=\"Instructions: Using the provided web search results, write a comprehensive reply to the given query. If you find a result relevant make sure to cite the result using [[number](URL)] notation after the reference. End your answer with a summary. A\\nQuery:\",\n",
    ")\n",
    "\n",
    "chatbot = Chatbot(config=constants.CHATGPT_CREDS)\n",
    "response = \"\"\n",
    "\n",
    "for data in chatbot.ask(prompt):\n",
    "    response = data[\"message\"]\n",
    "\n",
    "# print the response variable  as ipython markdown\n",
    "\n",
    "display(Markdown(f\"## Question\\n\\n\"))\n",
    "\n",
    "display(Markdown(f\"### {query1}\\n\\n\"))\n",
    "display(Markdown(f\"## ChatGPT Response: \\n\\n {response}\"))\n",
    "print_papers(df, k=8)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|██████████| 3/3 [00:14<00:00,  4.97s/it]\n"
     ]
    },
    {
     "data": {
      "text/markdown": [
       "## Question\n",
       "\n"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "### What are the ethical implications of using machine learning models for decision-making in areas such as hiring, lending, and criminal justice, and how can we ensure that these models are fair and unbiased?\n",
       "\n"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "## ChatGPT Response: \n",
       "\n",
       " The increasing use of machine learning (ML) models for decision-making in areas such as hiring, lending, and criminal justice raises ethical concerns regarding fairness and bias. ML models can perpetuate health care disparities [[1](https://www.semanticscholar.org/paper/82a5a84528a7ca0409f75e2211a3b33a217e9bac)], and their deployment requires careful monitoring to ensure that they perform as intended and do not introduce unintended biases [[2](https://www.semanticscholar.org/paper/534797ce20272279725a406270d7206556bf0c1d)]. Additionally, the transparency and interpretability of ML models are important factors for ensuring fairness, as stakeholders need to understand how the models make decisions.\n",
       "\n",
       "One way to address ethical concerns in ML-based decision-making is to ensure that the models are trained on unbiased data, and that their outputs are evaluated for fairness using appropriate metrics. For example, ML models used in hiring and lending should be trained on diverse data and evaluated for disparate impact on protected classes such as race and gender [[2](https://www.semanticscholar.org/paper/534797ce20272279725a406270d7206556bf0c1d)]. To achieve this, it is important to involve a diverse set of stakeholders in the development and deployment of ML models, including individuals who may be affected by their outputs [[3](https://www.semanticscholar.org/paper/bacfb8d5f6f75da1f5fde6f92e38d0de9fb13d04)].\n",
       "\n",
       "Furthermore, ML models used for decision-making should be transparent and interpretable, so that stakeholders can understand the factors that contributed to the decision. Several techniques and tools for model interpretability and explainability have been developed, such as feature importance analysis and local explanation methods, and these can be used to enhance the transparency of ML models [[3](https://www.semanticscholar.org/paper/bacfb8d5f6f75da1f5fde6f92e38d0de9fb13d04)]. \n",
       "\n",
       "In summary, ethical concerns in using ML models for decision-making can be addressed by ensuring unbiased training data, monitoring model performance for fairness, involving diverse stakeholders in the development and deployment of models, and enhancing the transparency and interpretability of the models. These strategies can help ensure that ML-based decision-making is fair and unbiased."
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [1] [Ensuring Fairness in Machine Learning to Advance Health Equity](https://www.semanticscholar.org/paper/82a5a84528a7ca0409f75e2211a3b33a217e9bac) - Annals of Internal Medicine, 2018"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [2] [Model Monitoring in Practice: Lessons Learned and Open Challenges](https://www.semanticscholar.org/paper/534797ce20272279725a406270d7206556bf0c1d) - Knowledge Discovery and Data Mining, 2022"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [3] [Explainable AI in Industry: Practical Challenges and Lessons Learned](https://www.semanticscholar.org/paper/bacfb8d5f6f75da1f5fde6f92e38d0de9fb13d04) - The Web Conference, 2020"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "query1 = \"What are the ethical implications of using machine learning models for decision-making in areas such as hiring, lending, and criminal justice, and how can we ensure that these models are fair and unbiased?\"\n",
    "# search and re-rank for the query\n",
    "df = get_results(query1)\n",
    "df[\"tldr\"] = df[\"abstract\"].progress_apply(get_response)\n",
    "df, query1 = rerank(df, query1, \"tldr\")\n",
    "prompt = answer_question_chatgpt(\n",
    "    df,\n",
    "    query1,\n",
    "    k=8,\n",
    "    instructions=\"Instructions: Using the provided web search results, write a comprehensive reply to the given query. If you find a result relevant make sure to cite the result using [[number](URL)] notation after the reference. End your answer with a summary. A\\nQuery:\",\n",
    ")\n",
    "\n",
    "chatbot = Chatbot(config=constants.CHATGPT_CREDS)\n",
    "response = \"\"\n",
    "\n",
    "for data in chatbot.ask(prompt):\n",
    "    response = data[\"message\"]\n",
    "\n",
    "# print the response variable  as ipython markdown\n",
    "\n",
    "display(Markdown(f\"## Question\\n\\n\"))\n",
    "\n",
    "display(Markdown(f\"### {query1}\\n\\n\"))\n",
    "display(Markdown(f\"## ChatGPT Response: \\n\\n {response}\"))\n",
    "print_papers(df, k=df.shape[0])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|██████████| 17/17 [01:15<00:00,  4.47s/it]\n"
     ]
    },
    {
     "data": {
      "text/markdown": [
       "## Question\n",
       "\n"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "### What are the best methods for developing machine learning models that can operate in real-time or near-real-time environments, such as autonomous vehicles or robotic systems?\n",
       "\n"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "## ChatGPT Response: \n",
       "\n",
       " Real-time or near-real-time operation is crucial for autonomous vehicles or robotic systems that need to interact with their environment in a timely manner. Machine learning (ML) models can be trained to operate in these environments, but the question is how to develop such models. Here are some methods that have been developed:\n",
       "\n",
       "One approach is to use alternative sensing methods to improve the speed and accuracy of ML models. For example, Milella et al. [[1](https://www.semanticscholar.org/paper/622e0046d26efa13a0c53b0638f20cdf119e9c09)] developed a self-trained online machine-learning framework for classification of the ground in front of a robot using radar and monocular vision. Similarly, UK's Loughborough University developed an algorithm for free space detection and human activity recognition for autonomous vehicles using multimodal sensor data streams (ultrasound and camera) and an active learning paradigm [[4](https://www.semanticscholar.org/paper/5130e0667ac22347c23bf0c128c571e66cc5f9ea)].\n",
       "\n",
       "Another approach is to use active learning algorithms that balance the number of measurements taken and the distance traveled in estimating the boundary of a certain area. For example, a novel active learning algorithm for monitoring regions of low oxygen concentration in lakes via an autonomous robot was proposed by Yang et al. [[2](https://www.semanticscholar.org/paper/8074675e6c4c08348e82105bb27927d51bd47f02)].\n",
       "\n",
       "Additionally, a principled classifier has been inserted into a common object detection model to evaluate patch attacks and the factors impacting their success, enabling the creation of a novel optimization framework for the design of semitransparent patches that can scale while retaining desirable factors with regard to deployment and detectability [[3](https://www.semanticscholar.org/paper/97702f29b0a6c6ae0d931ab99005ad313edfc0b6)].\n",
       "\n",
       "Moreover, an interdisciplinary approach is required to develop a truly integrated engineering product such as autonomous unmanned ground vehicles (UGV), which require engineering students from varied professions to collaborate [[6](https://www.semanticscholar.org/paper/818abffad01daf361dfb1e734b1c3831a55a24ee)].\n",
       "\n",
       "Finally, a special issue on 'Intelligent Transportation Systems in Smart Cities for Sustainable Environments' has been published in the journal IEEE Transactions on Intelligent Transportation Systems, which has 19 papers covering a range of topics such as roadside surveillance systems, energy vehicles, autonomous vehicles, and smart communication between unmanned aerial vehicles and unmanned underground robots [[7](https://www.semanticscholar.org/paper/77ab00ea7d2e4d3957f64056322379ab669a7825)].\n",
       "\n",
       "In summary, the best methods for developing machine learning models that can operate in real-time or near-real-time environments are alternative sensing methods, active learning algorithms, interdisciplinary collaboration, and knowledge-sharing through academic publications."
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [1] [Editorial: Special issue on Alternative Sensing Techniques for Robot Perception](https://www.semanticscholar.org/paper/622e0046d26efa13a0c53b0638f20cdf119e9c09) - J. Field Robotics, 2015.0"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [2] [Sensing Structured Signals with Active and Ensemble Methods](https://www.semanticscholar.org/paper/8074675e6c4c08348e82105bb27927d51bd47f02) - , 2017.0"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [3] [Adversarial Machine Learning in the Physical Domain](https://www.semanticscholar.org/paper/97702f29b0a6c6ae0d931ab99005ad313edfc0b6) - , 2021.0"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [4] [Multimodal machine learning for intelligent mobility](https://www.semanticscholar.org/paper/5130e0667ac22347c23bf0c128c571e66cc5f9ea) - , 2020.0"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [5] [Editorial: Special Issue on Safety, Security, and Rescue Robotics (SSRR)](https://www.semanticscholar.org/paper/94dc89b620b1f202108a2d673e0f26141a9b8826) - J. Field Robotics, 2019.0"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [6] [DEVELOPING AUTONOMOUS VEHICLES WITH OPTIMAL SENSOR SELECTION, INTEGRATION & CONTROL IMPLEMENTED AT 2018 INTELLIGENT GROUND VEHICLE COMPETITION (IGVC)](https://www.semanticscholar.org/paper/818abffad01daf361dfb1e734b1c3831a55a24ee) - , 2019.0"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [7] [Guest Editorial: Intelligent Transportation Systems in Smart Cities for Sustainable Environments (SCfSE)](https://www.semanticscholar.org/paper/77ab00ea7d2e4d3957f64056322379ab669a7825) - IET Intelligent Transport Systems, 2020.0"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [8] [Methods and algorithms for knowledge reuse in multiagent reinforcement learning.](https://www.semanticscholar.org/paper/cd1e6bca5f4903a03c6a2d9e7ab81f50c76a6209) - , 2020.0"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "query1 = \"What are the best methods for developing machine learning models that can operate in real-time or near-real-time environments, such as autonomous vehicles or robotic systems?\"\n",
    "# search and re-rank for the query\n",
    "df = get_results(query1)\n",
    "df[\"tldr\"] = df[\"abstract\"].progress_apply(get_response)\n",
    "df, query1 = rerank(df, query1, \"tldr\")\n",
    "prompt = answer_question_chatgpt(\n",
    "    df,\n",
    "    query1,\n",
    "    k=8,\n",
    "    instructions=\"Instructions: Using the provided web search results, write a comprehensive reply to the given query. If you find a result relevant make sure to cite the result using [[number](URL)] notation after the reference. End your answer with a summary. A\\nQuery:\",\n",
    ")\n",
    "\n",
    "chatbot = Chatbot(config=constants.CHATGPT_CREDS)\n",
    "response = \"\"\n",
    "\n",
    "for data in chatbot.ask(prompt):\n",
    "    response = data[\"message\"]\n",
    "\n",
    "# print the response variable  as ipython markdown\n",
    "\n",
    "display(Markdown(f\"## Question\\n\\n\"))\n",
    "\n",
    "display(Markdown(f\"### {query1}\\n\\n\"))\n",
    "display(Markdown(f\"## ChatGPT Response: \\n\\n {response}\"))\n",
    "\n",
    "print_papers(df, k=df.shape[0]) if df.shape[0] < 8 else print_papers(df, k=8)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|██████████| 18/18 [01:20<00:00,  4.46s/it]\n"
     ]
    },
    {
     "data": {
      "text/markdown": [
       "## Question\n",
       "\n"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "### What are the best practices for ensuring the privacy and security of sensitive data used in machine learning, such as medical or financial records?\n",
       "\n"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "## ChatGPT Response: \n",
       "\n",
       " Sensitive data used in machine learning, such as medical or financial records, require careful consideration to ensure privacy and security. Here are some best practices to ensure data privacy and security:\n",
       "\n",
       "1. Use protected data protection methods like Scalable and Enhanced Key Aggregate Cryptosystem [[1](https://www.semanticscholar.org/paper/28943e4f8380fc989b2fd6067c9b366f4cdd48cc)] and homomorphic encryption [[2](https://www.semanticscholar.org/paper/a8b8184fda259b42be88f9fb3e92e981e871c3f3)] to encrypt sensitive data before storing or transmitting it.\n",
       "\n",
       "2. Fully Homomorphic Encryption (FHE) is an advanced technique that allows computations on encrypted data without revealing the data to the computing party. This is particularly useful in the context of machine learning where a user can obtain predictions on their data without sharing it with the service provider [[3](https://www.semanticscholar.org/paper/1b4665d7edc8afbcd8495a57bb4c469fc2c095af)].\n",
       "\n",
       "3. Develop machine learning applications responsibly and consider fundamental principles like beneficence, respect for human autonomy, prevention of harm, justice, privacy, and transparency. Technical guidelines must be in place to ensure these principles are practically implemented [[4](https://www.semanticscholar.org/paper/f59a9350b235bf7b6b778821e94c39eb6a3ea3ee)].\n",
       "\n",
       "4. Use non-cryptographic differential privacy approaches to protect data privacy. These methods ensure that the data cannot be traced back to the individual. This is particularly useful when sharing aggregate data or for research purposes [[5](https://www.semanticscholar.org/paper/8e20ea811291775413031901e9ee8bcecbfa9d1b)].\n",
       "\n",
       "5. Personal data privacy is critical in managing information. Ensure data security practices in open or closed network environments. Legislation has been put in place to regulate the handling of sensitive information like personal, banking, health, and medical information [[7](https://www.semanticscholar.org/paper/8f9173e01b4d3debc9fe41a25a3ae8491c9bd9df)].\n",
       "\n",
       "6. Big data science (BDS) can be a useful tool in investigating patterns in HIV care utilization. The South Carolina office of Revenue and Fiscal Affairs data warehouse captures longitudinal health utilization data for persons living with HIV. The data warehouse includes EHR, claims, and data from private institutions, housing, prisons, mental health, Medicare, Medicaid, State Health Plan, and the department of health and human services [[8](https://www.semanticscholar.org/paper/878a8b46c70cbb53728d9c039532a14efb0f1b41)].\n",
       "\n",
       "In summary, the best practices for ensuring privacy and security of sensitive data used in machine learning include using protected data protection methods, developing machine learning applications responsibly, using non-cryptographic differential privacy approaches, and ensuring data security practices are in place in open or closed network environments. Legislative measures have been put in place to regulate the handling of sensitive information like personal, banking, health, and medical information."
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [1] [Key Aggregation Cryptosystem and Double Encryption Method for Cloud-Based Intelligent Machine Learning Techniques-Based Health Monitoring Systems](https://www.semanticscholar.org/paper/28943e4f8380fc989b2fd6067c9b366f4cdd48cc) - Computational Intelligence and Neuroscience, 2022.0"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [2] [Privacy Preserving Analytics in Outsourced Healthcare System](https://www.semanticscholar.org/paper/a8b8184fda259b42be88f9fb3e92e981e871c3f3) - Regular, 2020.0"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [3] [Machine learning on encrypted data](https://www.semanticscholar.org/paper/1b4665d7edc8afbcd8495a57bb4c469fc2c095af) - , 2018.0"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [4] [Responsible and Regulatory Conform Machine Learning for Medicine: A Survey of Technical Challenges and Solutions](https://www.semanticscholar.org/paper/f59a9350b235bf7b6b778821e94c39eb6a3ea3ee) - ArXiv, 2021.0"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [5] [Privacy Preserving Machine Learning and Deep Learning Techniques](https://www.semanticscholar.org/paper/8e20ea811291775413031901e9ee8bcecbfa9d1b) - Research Anthology on Privatizing and Securing Data, 2021.0"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [6] [Medical Healthcare System with Hybrid Block based Predictive models for Quality preserving in Medical Images using Machine Learning Techniques](https://www.semanticscholar.org/paper/c54d8b94e3290ae5e02ffcfdc407e6d519ceec2f) - 2022 International Conference on Advanced Computing Technologies and Applications (ICACTA), 2022.0"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [7] [Personal Data Privacy Protection: A Review on Malaysia’s Cyber Security Policies](https://www.semanticscholar.org/paper/8f9173e01b4d3debc9fe41a25a3ae8491c9bd9df) - International Journal of Academic Research in Business and Social Sciences, 2019.0"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [8] [Using big data analytics to improve HIV medical care utilisation in South Carolina: A study protocol](https://www.semanticscholar.org/paper/878a8b46c70cbb53728d9c039532a14efb0f1b41) - BMJ Open, 2019.0"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "query1 = \"What are the best practices for ensuring the privacy and security of sensitive data used in machine learning, such as medical or financial records?\"\n",
    "# search and re-rank for the query\n",
    "df = get_results(query1)\n",
    "df[\"tldr\"] = df[\"abstract\"].progress_apply(get_response)\n",
    "df, query1 = rerank(df, query1, \"tldr\")\n",
    "prompt = answer_question_chatgpt(\n",
    "    df,\n",
    "    query1,\n",
    "    k=8,\n",
    "    instructions=\"Instructions: Using the provided web search results, write a comprehensive reply to the given query. If you find a result relevant make sure to cite the result using [[number](URL)] notation after the reference. End your answer with a summary. A\\nQuery:\",\n",
    ")\n",
    "\n",
    "chatbot = Chatbot(config=constants.CHATGPT_CREDS)\n",
    "response = \"\"\n",
    "\n",
    "for data in chatbot.ask(prompt):\n",
    "    response = data[\"message\"]\n",
    "\n",
    "# print the response variable  as ipython markdown\n",
    "\n",
    "display(Markdown(f\"## Question\\n\\n\"))\n",
    "\n",
    "display(Markdown(f\"### {query1}\\n\\n\"))\n",
    "display(Markdown(f\"## ChatGPT Response: \\n\\n {response}\"))\n",
    "print_papers(df, k=df.shape[0]) if df.shape[0] < 8 else print_papers(df, k=8)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|██████████| 14/14 [00:59<00:00,  4.25s/it]\n"
     ]
    },
    {
     "data": {
      "text/markdown": [
       "## Question\n",
       "\n"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "### How can we develop machine learning models that can reason and make decisions based on incomplete or uncertain information, such as in natural language understanding or medical diagnosis?\n",
       "\n"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "## ChatGPT Response: \n",
       "\n",
       " Machine learning (ML) algorithms have the potential to make predictions based on all available information, including incomplete or uncertain information in natural language understanding and medical diagnosis [[1](https://www.semanticscholar.org/paper/b12eeee60ab73d233b2864f4dc65471bba4aec3b)]. For instance, researchers have developed an end-to-end natural language processing pipeline that can extract COVID-19 symptoms and related concepts such as severity, duration, negations, and body parts from patients' social media posts using conditional random fields and unsupervised rule-based algorithms [[2](https://www.semanticscholar.org/paper/3549e156718405f5bc17e2df5025bb2d921bd22a)].\n",
       "\n",
       "However, AI systems in healthcare may not be suitable for learning uncertain clinical patterns from electronic health record data, and may offer human-like suggestions by helping to avoid clinical blunders [[3](https://www.semanticscholar.org/paper/9e62d56160d5ccba9f6c19c2602c0a20a8287c86)]. Nonetheless, studies have shown that computer deep learning models consistently outperform other models in predicting breast cancer within five years after mammography [[4](https://www.semanticscholar.org/paper/c54199aae8b694b015812bc813b94719e14bc485)], and hybrid recommendation systems that include clinical and pre-clinical data as well as metrics of literature support obtained with natural language processing techniques have helped identify novel resistance mechanisms in lung cancer patients that could have been overlooked by experts [[5](https://www.semanticscholar.org/paper/3f324bc1ed73b079fb03991d8a5198e94d8e60f8)]. Furthermore, an artificial intelligence-based assistive diagnostic system has been developed to diagnose multiple types of diseases that are common in traditional Chinese medicine, and produce a list of corresponding syndromes [[6](https://www.semanticscholar.org/paper/2dd2acb0a79b6a6483df9d22da590b9243f57f45)].\n",
       "\n",
       "To develop machine learning models that can reason and make decisions based on incomplete or uncertain information, researchers have used novel methods such as deep learning and hybrid recommendation systems, and have included clinical and pre-clinical data as well as metrics of literature support obtained with natural language processing techniques. However, AI systems in healthcare may not be suitable for learning uncertain clinical patterns from electronic health record data, and may offer human-like suggestions by helping to avoid clinical blunders. Nonetheless, the potential for ML algorithms to make predictions based on all available information, including incomplete or uncertain information in natural language understanding and medical diagnosis is promising."
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [1] [Deep learning for precision medicine](https://www.semanticscholar.org/paper/b12eeee60ab73d233b2864f4dc65471bba4aec3b) - , 2018"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [2] [Triage and diagnosis of COVID-19 from medical social media](https://www.semanticscholar.org/paper/3549e156718405f5bc17e2df5025bb2d921bd22a) - ArXiv, 2021"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [3] [Chess Lessons: Harnessing Collective Human Intelligence and Imitation Learning to Support Clinical Decisions](https://www.semanticscholar.org/paper/9e62d56160d5ccba9f6c19c2602c0a20a8287c86) - Annals of Internal Medicine, 2023"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [4] [Assessing Cancer Risk from Mammograms: Deep Learning Is Superior to Conventional Risk Models.](https://www.semanticscholar.org/paper/c54199aae8b694b015812bc813b94719e14bc485) - Radiology, 2019"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [5] [Drug Discovery as a Recommendation Problem: Challenges and Complexities in Biological Decisions](https://www.semanticscholar.org/paper/3f324bc1ed73b079fb03991d8a5198e94d8e60f8) - ACM Conference on Recommender Systems, 2021"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [6] [Artificial Intelligence–Based Traditional Chinese Medicine Assistive Diagnostic System: Validation Study](https://www.semanticscholar.org/paper/2dd2acb0a79b6a6483df9d22da590b9243f57f45) - JMIR Medical Informatics, 2020"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [7] [WILL ARTIFICIAL INTELLIGENCE CHANGE FUTURE OF DENTISTRY!](https://www.semanticscholar.org/paper/bf71eecaafab5cf16c970a059a856665b6c7b177) - , 2021"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [8] [Prediction of risk of acquiring urinary tract infection during hospital stay based on machine-learning: A retrospective cohort study](https://www.semanticscholar.org/paper/515e9903080f6a42c657e8ed9854c57e148a5a91) - PLoS ONE, 2021"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "query1 = \"How can we develop machine learning models that can reason and make decisions based on incomplete or uncertain information, such as in natural language understanding or medical diagnosis?\"\n",
    "# search and re-rank for the query\n",
    "df = get_results(query1)\n",
    "df[\"tldr\"] = df[\"abstract\"].progress_apply(get_response)\n",
    "df, query1 = rerank(df, query1, \"tldr\")\n",
    "prompt = answer_question_chatgpt(\n",
    "    df,\n",
    "    query1,\n",
    "    k=8,\n",
    "    instructions=\"Instructions: Using the provided web search results, write a comprehensive reply to the given query. If you find a result relevant make sure to cite the result using [[number](URL)] notation after the reference. End your answer with a summary. A\\nQuery:\",\n",
    ")\n",
    "\n",
    "chatbot = Chatbot(config=constants.CHATGPT_CREDS)\n",
    "response = \"\"\n",
    "\n",
    "for data in chatbot.ask(prompt):\n",
    "    response = data[\"message\"]\n",
    "\n",
    "# print the response variable  as ipython markdown\n",
    "\n",
    "display(Markdown(f\"## Question\\n\\n\"))\n",
    "\n",
    "display(Markdown(f\"### {query1}\\n\\n\"))\n",
    "display(Markdown(f\"## ChatGPT Response: \\n\\n {response}\"))\n",
    "print_papers(df, k=df.shape[0]) if df.shape[0] < 8 else print_papers(df, k=8)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|██████████| 18/18 [01:18<00:00,  4.39s/it]\n"
     ]
    },
    {
     "data": {
      "text/markdown": [
       "## Question\n",
       "\n"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "### What are the most effective methods for developing machine learning models that are resilient to adversarial attacks or data poisoning?\n",
       "\n"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "## ChatGPT Response: \n",
       "\n",
       " Adversarial attacks and data poisoning are significant threats to the robustness and accuracy of machine learning models. Researchers have proposed various methods to develop machine learning models that are resilient to such attacks. \n",
       "\n",
       "One approach is to use pruning and autoencoder-based strategies to defend against data poisoning attacks [[3]]. This method reduces the attacker's efficacy by 23% and can serve as a strong baseline for future work on defense. Another approach is to use ensemble defense strategies based on adversarial retraining techniques to develop deep learning models that can withstand multiple adversarial attacks [[8]].\n",
       "\n",
       "Semi-supervised learning methods can also be vulnerable to backdoor data poisoning attacks on unlabeled samples [[4]]. One way to defend against such attacks is to use a novel training method that is resistant to such attacks. \n",
       "\n",
       "Researchers have also proposed various types of adversarial attacks, including evasion, poisoning, model stealing, and inference, and investigated the vulnerabilities of machine learning models to these attacks [[2]]. In particular, subpopulation attacks can be effective in large and diverse datasets, and a modular framework can be used to instantiate these attacks with different building blocks [[7]].\n",
       "\n",
       "Overall, developing machine learning models that are resilient to adversarial attacks and data poisoning requires a multi-pronged approach that combines various defense strategies, including pruning, autoencoder-based strategies, ensemble defense, and novel training methods. \n",
       "\n",
       "In summary, researchers have proposed various methods to develop machine learning models that are resilient to adversarial attacks or data poisoning, such as pruning and autoencoder-based strategies, ensemble defense, and novel training methods. These approaches offer a multi-pronged approach to increase the robustness and accuracy of machine learning models against adversarial attacks."
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [1] [A Sensitivity Analysis of Poisoning and Evasion Attacks in Network Intrusion Detection System Machine Learning Models](https://www.semanticscholar.org/paper/67cc63168b7e642d61a9e2927218d5f639b8b9d2) - IEEE Military Communications Conference, 2021"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [2] [A Tutorial on Adversarial Learning Attacks and Countermeasures](https://www.semanticscholar.org/paper/ee67b5e85769018e09d76345e648b29ec0cfa8b3) - ArXiv, 2022"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [3] [Dleep Learning based Regularization as defense against Clean Label Targeted Data Poisoning](https://www.semanticscholar.org/paper/0ec6d8c004ca5c122f0a855071bbd0427a262d3a) - International Conference Intelligent Computing and Control Systems, 2022"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [4] [Rethinking Backdoor Data Poisoning Attacks in the Context of Semi-Supervised Learning](https://www.semanticscholar.org/paper/41d1aa1e62d69bb35b19096068f3c777c103eb21) - ArXiv, 2022"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [5] [How Robust are Randomized Smoothing based Defenses to Data Poisoning?](https://www.semanticscholar.org/paper/8e74da9a4a3dd231ad277b033172e5cd1700cb0c) - Computer Vision and Pattern Recognition, 2020"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [6] [Poisoning Attacks and Data Sanitization Mitigations for Machine Learning Models in Network Intrusion Detection Systems](https://www.semanticscholar.org/paper/7f1014d009865e8797c61d194eebb943e4559d0f) - IEEE Military Communications Conference, 2021"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [7] [Subpopulation Data Poisoning Attacks](https://www.semanticscholar.org/paper/e0a69d5a13ea753251c897d4e970ca720e8a15cd) - Conference on Computer and Communications Security, 2020"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [8] [Defending Deep Learning Models Against Adversarial Attacks](https://www.semanticscholar.org/paper/73a865d59f7b1b5d5ff8c989ffa95d3cf21cb063) - International Journal of Software Science and Computational Intelligence, 2021"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "query1 = \"What are the most effective methods for developing machine learning models that are resilient to adversarial attacks or data poisoning?\"\n",
    "# search and re-rank for the query\n",
    "df = get_results(query1)\n",
    "df[\"tldr\"] = df[\"abstract\"].progress_apply(get_response)\n",
    "df, query1 = rerank(df, query1, \"tldr\")\n",
    "prompt = answer_question_chatgpt(\n",
    "    df,\n",
    "    query1,\n",
    "    k=8,\n",
    "    instructions=\"Instructions: Using the provided web search results, write a comprehensive reply to the given query. If you find a result relevant make sure to cite the result using [[number](URL)] notation after the reference. End your answer with a summary. A\\nQuery:\",\n",
    ")\n",
    "\n",
    "chatbot = Chatbot(config=constants.CHATGPT_CREDS)\n",
    "response = \"\"\n",
    "\n",
    "for data in chatbot.ask(prompt):\n",
    "    response = data[\"message\"]\n",
    "\n",
    "# print the response variable  as ipython markdown\n",
    "\n",
    "display(Markdown(f\"## Question\\n\\n\"))\n",
    "\n",
    "display(Markdown(f\"### {query1}\\n\\n\"))\n",
    "display(Markdown(f\"## ChatGPT Response: \\n\\n {response}\"))\n",
    "print_papers(df, k=df.shape[0]) if df.shape[0] < 8 else print_papers(df, k=8)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|██████████| 6/6 [00:27<00:00,  4.63s/it]\n"
     ]
    },
    {
     "data": {
      "text/markdown": [
       "## Question\n",
       "\n"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "### How can we develop machine learning models that are more energy-efficient and sustainable, particularly for resource-constrained devices or environments?\n",
       "\n"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "## ChatGPT Response: \n",
       "\n",
       " Developing efficient, scalable, and distributed methods to process immense growth in visual data is a current research focus to address challenges associated with resource-constrained devices or environments [[1](https://www.semanticscholar.org/paper/4d9682721c441899752d82bfe66a296cca986669)]. Similarly, digital twins can represent the flexibility offered by local and global sources of energy and potential benefits of IoT-enabled systems, giving insights into hidden dynamics, risks, and their impact on the distribution level of the electric power system [[2](https://www.semanticscholar.org/paper/b47dd6375863ecfaabeec0fab714300fd66e715d)]. Algorithms that enable the rendering of sophisticated computer graphics and the evolution of artificial realities as a new medium are also driving the development of more energy-efficient machine learning models [[3](https://www.semanticscholar.org/paper/c3083846805c010f82ebe8cff004958848614e56)].\n",
       "\n",
       "Another approach to developing more energy-efficient and sustainable machine learning models is through the use of modular open-source architectures. For example, a modular open-source architecture for Medical Capsule Robots (MCRs) has been developed, consisting of both hardware modules and firmware libraries that can be used for developing MCRs. The SMAC modules are miniature boards of uniform diameter that can fulfill different functions such as signal coordination, sensing, actuation, and vision powering [[5](https://www.semanticscholar.org/paper/b956b7da325c15241d1415d6b35c68b0ced0bad3)].\n",
       "\n",
       "Finally, it is essential to consider security concerns, particularly for mobile devices like Android-based mobile phones, which are becoming ubiquitous and are being widely used to store and handle personal information. They have also become a target for malware developers, and addressing this issue can contribute to more sustainable and energy-efficient machine learning models [[4](https://www.semanticscholar.org/paper/03d165a6def948b3d437401cf4aeaf4bfe8be16c)].\n",
       "\n",
       "In summary, approaches to developing more energy-efficient and sustainable machine learning models include distributed methods, digital twins, modular open-source architectures, and addressing security concerns on resource-constrained devices."
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [1] [SCALABALE AND DISTRIBUTED METHODS FORLARGE-SCALE VISUAL COMPUTING](https://www.semanticscholar.org/paper/4d9682721c441899752d82bfe66a296cca986669) - , 2019"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [2] [Proceedings of 2019 IEEE PES Innovative Smart Grid Technologies Europe (ISGT-Europe)](https://www.semanticscholar.org/paper/b47dd6375863ecfaabeec0fab714300fd66e715d) - IEEE PES Innovative Smart Grid Technologies Conference Europe, 2019"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [3] [Artificial Realities , Virtual Communities , and Knowbots](https://www.semanticscholar.org/paper/c3083846805c010f82ebe8cff004958848614e56) - , 2008"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [4] [Machine Learning Based Malware Detection For Android Using Epdf File](https://www.semanticscholar.org/paper/03d165a6def948b3d437401cf4aeaf4bfe8be16c) - , 2021"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [5] [When is a Robot a Robot?: How new degree in robotics challenged us to once again define robots](https://www.semanticscholar.org/paper/b956b7da325c15241d1415d6b35c68b0ced0bad3) - , 2014"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [6] [On the movement behaviours of tropical tuna in modern commercial fisheries](https://www.semanticscholar.org/paper/6ed53c9915348cad773f5a1e094771a7e73f63a1) - , 2014"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "query1 = \"How can we develop machine learning models that are more energy-efficient and sustainable, particularly for resource-constrained devices or environments?\"\n",
    "# search and re-rank for the query\n",
    "df = get_results(query1)\n",
    "df[\"tldr\"] = df[\"abstract\"].progress_apply(get_response)\n",
    "df, query1 = rerank(df, query1, \"tldr\")\n",
    "prompt = answer_question_chatgpt(\n",
    "    df,\n",
    "    query1,\n",
    "    k=8,\n",
    "    instructions=\"Instructions: Using the provided web search results, write a comprehensive reply to the given query. If you find a result relevant make sure to cite the result using [[number](URL)] notation after the reference. End your answer with a summary. A\\nQuery:\",\n",
    ")\n",
    "\n",
    "chatbot = Chatbot(config=constants.CHATGPT_CREDS)\n",
    "response = \"\"\n",
    "\n",
    "for data in chatbot.ask(prompt):\n",
    "    response = data[\"message\"]\n",
    "\n",
    "# print the response variable  as ipython markdown\n",
    "\n",
    "display(Markdown(f\"## Question\\n\\n\"))\n",
    "\n",
    "display(Markdown(f\"### {query1}\\n\\n\"))\n",
    "display(Markdown(f\"## ChatGPT Response: \\n\\n {response}\"))\n",
    "print_papers(df, k=df.shape[0]) if df.shape[0] < 8 else print_papers(df, k=8)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|██████████| 4/4 [00:16<00:00,  4.14s/it]\n"
     ]
    },
    {
     "data": {
      "text/markdown": [
       "## Question\n",
       "\n"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "### What are the ethical and societal implications of developing and deploying machine learning technologies, and how can we ensure that they are used in ways that are fair and beneficial to all?\n",
       "\n"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "## ChatGPT Response: \n",
       "\n",
       " The development and deployment of machine learning technologies have the potential to bring significant benefits to society, but they also raise important ethical and societal implications that need to be addressed to ensure that they are used in ways that are fair and beneficial to all. \n",
       "\n",
       "One of the key concerns is the potential for complete automation, which could lead to the removal of the human element from complex tasks in both the workplace and at home, creating job displacement and economic disruption. Additionally, the danger of malicious AI or innocent errors in algorithm construction can have significant implications. The development and deployment of autonomous systems have already been used in the military, further highlighting the need for ethical considerations [[1](https://www.semanticscholar.org/paper/e129340f016e11f2b328c6fc4d2ad9f6658b8cc1)][[2](https://www.semanticscholar.org/paper/71af6878cf6528e2df24a856809106059484e259)].\n",
       "\n",
       "One potential solution proposed by the Association for the Advancement of Artificial Intelligence is the establishment of a governance coordinating committee (GCC), which could help to break the current logjam with respect to agile and effective governance of emerging technologies such as AI and synthetic biology. The paper argues for an international GCC with complementary regional bodies in light of the transnational nature of AI concerns and risks [[3](https://www.semanticscholar.org/paper/21d7c4a7fc70a18624d653f498891d187b41d7e1)].\n",
       "\n",
       "Moreover, machine learning can perpetuate health care disparities, as historical data capture patterns of these disparities, and machine-learning models trained on these data may perpetuate these inequities. To address this, there is a need for ethical considerations and proactive measures to prevent the perpetuation of these biases [[4](https://www.semanticscholar.org/paper/82a5a84528a7ca0409f75e2211a3b33a217e9bac)].\n",
       "\n",
       "In summary, the development and deployment of machine learning technologies need to be accompanied by ethical considerations and proactive measures to ensure that they are used in ways that are fair and beneficial to all. The establishment of a governance coordinating committee and regional bodies can provide a useful framework for addressing these concerns, and there is a need for ongoing research and development to ensure that machine learning does not perpetuate biases or exacerbate existing societal inequities."
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [1] [Exploring Potential Flaws and Dangers Involving Machine Learning Technology](https://www.semanticscholar.org/paper/e129340f016e11f2b328c6fc4d2ad9f6658b8cc1) - , 2017.0"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [2] [Developments in Artificial Intelligence – Opportunities and Challenges for Military Modeling and Simulation](https://www.semanticscholar.org/paper/71af6878cf6528e2df24a856809106059484e259) - , 2017.0"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [3] [An Agile Ethical/Legal Model for the International and National Governance of AI and Robotics](https://www.semanticscholar.org/paper/21d7c4a7fc70a18624d653f498891d187b41d7e1) - , 2017.0"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [4] [Ensuring Fairness in Machine Learning to Advance Health Equity](https://www.semanticscholar.org/paper/82a5a84528a7ca0409f75e2211a3b33a217e9bac) - Annals of Internal Medicine, 2018.0"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "query1 = \"What are the ethical and societal implications of developing and deploying machine learning technologies, and how can we ensure that they are used in ways that are fair and beneficial to all?\"\n",
    "# search and re-rank for the query\n",
    "df = get_results(query1)\n",
    "df[\"tldr\"] = df[\"abstract\"].progress_apply(get_response)\n",
    "df, query1 = rerank(df, query1, \"tldr\")\n",
    "prompt = answer_question_chatgpt(\n",
    "    df,\n",
    "    query1,\n",
    "    k=8,\n",
    "    instructions=\"Instructions: Using the provided web search results, write a comprehensive reply to the given query. If you find a result relevant make sure to cite the result using [[number](URL)] notation after the reference. End your answer with a summary. A\\nQuery:\",\n",
    ")\n",
    "\n",
    "chatbot = Chatbot(config=constants.CHATGPT_CREDS)\n",
    "response = \"\"\n",
    "\n",
    "for data in chatbot.ask(prompt):\n",
    "    response = data[\"message\"]\n",
    "\n",
    "# print the response variable  as ipython markdown\n",
    "\n",
    "display(Markdown(f\"## Question\\n\\n\"))\n",
    "\n",
    "display(Markdown(f\"### {query1}\\n\\n\"))\n",
    "display(Markdown(f\"## ChatGPT Response: \\n\\n {response}\"))\n",
    "print_papers(df, k=df.shape[0]) if df.shape[0] < 8 else print_papers(df, k=8)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|██████████| 15/15 [01:23<00:00,  5.54s/it]\n"
     ]
    },
    {
     "data": {
      "text/markdown": [
       "## Question\n",
       "\n"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "### why does bert work so well with machine learning tasks?\n",
       "\n"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "## ChatGPT Response: \n",
       "\n",
       " BERT (Bidirectional Encoder Representations from Transformers) is a popular neural language model that has achieved impressive results on a wide range of natural language processing (NLP) tasks. There are several factors that contribute to BERT's success.\n",
       "\n",
       "First, BERT uses a transformer architecture that allows it to capture complex relationships between words in a sentence. Unlike traditional recurrent neural networks (RNNs), transformers can process sentences in parallel, which speeds up training and makes it easier to capture long-range dependencies in text. This enables BERT to generate high-quality contextualized embeddings, which can be used as inputs to downstream tasks.\n",
       "\n",
       "Second, BERT is trained on a large amount of unlabeled text data using a masked language modeling objective. During training, BERT is fed a sentence with certain words randomly masked out, and the model must predict the masked words based on the surrounding context. This forces the model to learn a deep understanding of language structure and meaning, which can be transferred to other tasks.\n",
       "\n",
       "Finally, BERT can be fine-tuned on a variety of NLP tasks with relatively little additional training. By adjusting the final layer of the model to fit the task-specific output format, BERT can be used for tasks such as sentiment analysis, question answering, and natural language inference with impressive accuracy.\n",
       "\n",
       "While BERT's performance on natural language tasks is impressive, there are still concerns about its ability to perpetuate bias and perpetuate stereotypes [[3](https://www.semanticscholar.org/paper/f01e528f09420c0a40305ddc046afcadc741c7d4)][[4](https://www.semanticscholar.org/paper/d84d91fee6ac4d073a79afdc0ee68deedc4c8c07)]. There have also been studies exploring the vulnerability of BERT-based models to attacks such as model extraction and attribute inference [[5](https://www.semanticscholar.org/paper/373936d00c4a357579c4d375de0ce439e4e54d5f)]. \n",
       "\n",
       "In summary, BERT's success can be attributed to its transformer architecture, its large pre-training corpus, and its ability to be fine-tuned for a wide range of NLP tasks. However, there are also concerns about its potential to propagate bias and its vulnerability to attacks."
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [1] [Adversarial Training for Large Neural Language Models](https://www.semanticscholar.org/paper/2ffcf8352223c95ae8cef4daaec995525ecc926b) - ArXiv, 2020"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [2] [Synchronous Boltzmann machines and curve identification tasks](https://www.semanticscholar.org/paper/7eb278a9d7495371ad8f9e1a7df08d65ca916770) - , 1993"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [3] [Debiasing Gender Bias in Information Retrieval Models](https://www.semanticscholar.org/paper/f01e528f09420c0a40305ddc046afcadc741c7d4) - , 2022"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [4] [Exploring Gender Bias in Retrieval Models](https://www.semanticscholar.org/paper/d84d91fee6ac4d073a79afdc0ee68deedc4c8c07) - ArXiv, 2022"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [5] [Killing One Bird with Two Stones: Model Extraction and Attribute Inference Attacks against BERT-based APIs](https://www.semanticscholar.org/paper/373936d00c4a357579c4d375de0ce439e4e54d5f) - , 2021"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [6] [Deep Learning in Single-Cell Analysis](https://www.semanticscholar.org/paper/3a6acc1b3ddea30f590e55d12b137175e46d3b13) - ArXiv, 2022"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [7] [Entanglement and Tensor Networks for Supervised Image Classification](https://www.semanticscholar.org/paper/e4fe8d451fc6d9e9afef5441d065c97a247a2959) - ArXiv, 2020"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [8] [Scaling Systematic Literature Reviews with Machine Learning Pipelines](https://www.semanticscholar.org/paper/49d47131ef1fb62b0749b1078e980d59688c116d) - SDP, 2020"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "query1 = \"why does bert work so well with machine learning tasks?\"\n",
    "# search and re-rank for the query\n",
    "df = get_results(query1)\n",
    "df[\"tldr\"] = df[\"abstract\"].progress_apply(get_response)\n",
    "df, query1 = rerank(df, query1, \"tldr\")\n",
    "prompt = answer_question_chatgpt(\n",
    "    df,\n",
    "    query1,\n",
    "    k=8,\n",
    "    instructions=\"Instructions: Using the provided web search results, write a comprehensive reply to the given query. If you find a result relevant make sure to cite the result using [[number](URL)] notation after the reference. End your answer with a summary. A\\nQuery:\",\n",
    ")\n",
    "\n",
    "chatbot = Chatbot(config=constants.CHATGPT_CREDS)\n",
    "response = \"\"\n",
    "\n",
    "for data in chatbot.ask(prompt):\n",
    "    response = data[\"message\"]\n",
    "\n",
    "# print the response variable  as ipython markdown\n",
    "\n",
    "display(Markdown(f\"## Question\\n\\n\"))\n",
    "display(Markdown(f\"### {query1}\\n\\n\"))\n",
    "display(Markdown(f\"## ChatGPT Response: \\n\\n {response}\"))\n",
    "print_papers(df, k=df.shape[0]) if df.shape[0] < 8 else print_papers(df, k=8)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|██████████| 19/19 [01:38<00:00,  5.16s/it]\n"
     ]
    },
    {
     "data": {
      "text/markdown": [
       "## Question\n",
       "\n"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "### bert\n",
       "\n"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "## ChatGPT Response: \n",
       "\n",
       " BERT, or Bidirectional Encoder Representations from Transformers, is a popular pre-trained language model that has been widely studied and modified since its introduction. One survey of over 150 studies of BERT provides a comprehensive overview of its architecture, training objectives, overparameterization, and compression techniques [[1](https://www.semanticscholar.org/paper/bd20069f5cac3e63083ecf6479abc1799db33ce0)]. \n",
       "\n",
       "A replication study of BERT pretraining found that the model was significantly undertrained and can match or exceed the performance of every model published after it [[2](https://www.semanticscholar.org/paper/077f8329a7b6fa3b7c877a57b81eb6c18b5f87de)]. Another study investigated where linguistic information is captured within the network and found that BERT represents the steps of the traditional NLP pipeline in an interpretable and localizable way, with the regions responsible for each step appearing in the expected sequence [[3](https://www.semanticscholar.org/paper/97906df07855b029b7aae7c2a1c6c5e8df1d531c)].\n",
       "\n",
       "Several modifications to BERT have been proposed to improve its performance and reduce its size. One such modification, DeBERTa, uses a disentangled attention mechanism to improve the representation of each word in the input [[4](https://www.semanticscholar.org/paper/05f5f8b2065a520846d89771ebaea2bb1534e9c6)]. Another modification proposes parameter-reduction techniques to lower memory consumption and increase training speed while using a self-supervised loss to focus on inter-sentence coherence [[5](https://www.semanticscholar.org/paper/7a064df1aeada7e69e5173f7d4c8606f4470365b)]. \n",
       "\n",
       "Specialized domains can benefit from the use of BERT, and one study proposes three strategies for adapting BERT for specialized domains [[6](https://www.semanticscholar.org/paper/f6245b3e6270e4dc2e279c4b728030523dffcff4)]. Compressing and accelerating BERT is also an active area of research, and one study proposes MobileBERT, a smaller and faster version of BERT that achieves competitive results on well-known benchmarks [[7](https://www.semanticscholar.org/paper/2573af4e13d9a5dddb257d22cd38a600528d9a8b)]. Finally, Visual-Linguistic BERT (VL-BERT) is a pre-trainable generic representation for visual-linguistic tasks that takes both visual and linguistic embedded features as input [[8](https://www.semanticscholar.org/paper/2527626c11a84f15709e943fbfa2356e19930e3b)]. \n",
       "\n",
       "Overall, BERT is a versatile and powerful language model that has been extensively studied and modified for various applications. Its architecture and training objectives have been investigated, and modifications have been proposed to improve its performance, reduce its size, and adapt it to specialized domains."
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [1] [A Primer in BERTology: What We Know About How BERT Works](https://www.semanticscholar.org/paper/bd20069f5cac3e63083ecf6479abc1799db33ce0) - Transactions of the Association for Computational Linguistics, 2020"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [2] [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://www.semanticscholar.org/paper/077f8329a7b6fa3b7c877a57b81eb6c18b5f87de) - ArXiv, 2019"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [3] [BERT Rediscovers the Classical NLP Pipeline](https://www.semanticscholar.org/paper/97906df07855b029b7aae7c2a1c6c5e8df1d531c) - Annual Meeting of the Association for Computational Linguistics, 2019"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [4] [DeBERTa: Decoding-enhanced BERT with Disentangled Attention](https://www.semanticscholar.org/paper/05f5f8b2065a520846d89771ebaea2bb1534e9c6) - International Conference on Learning Representations, 2020"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [5] [ALBERT: A Lite BERT for Self-supervised Learning of Language Representations](https://www.semanticscholar.org/paper/7a064df1aeada7e69e5173f7d4c8606f4470365b) - International Conference on Learning Representations, 2019"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [6] [LEGAL-BERT: The Muppets straight out of Law School](https://www.semanticscholar.org/paper/f6245b3e6270e4dc2e279c4b728030523dffcff4) - Conference on Empirical Methods in Natural Language Processing, 2020"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [7] [MobileBERT: a Compact Task-Agnostic BERT for Resource-Limited Devices](https://www.semanticscholar.org/paper/2573af4e13d9a5dddb257d22cd38a600528d9a8b) - Annual Meeting of the Association for Computational Linguistics, 2020"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [8] [VL-BERT: Pre-training of Generic Visual-Linguistic Representations](https://www.semanticscholar.org/paper/2527626c11a84f15709e943fbfa2356e19930e3b) - International Conference on Learning Representations, 2019"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "query1 = \"bert\"\n",
    "# search and re-rank for the query\n",
    "df = get_results(query1)\n",
    "df[\"tldr\"] = df[\"abstract\"].progress_apply(get_response)\n",
    "df, query1 = rerank(df, query1, \"tldr\")\n",
    "prompt = answer_question_chatgpt(\n",
    "    df,\n",
    "    query1,\n",
    "    k=8,\n",
    "    instructions=\"Instructions: Using the provided web search results, write a comprehensive reply to the given query. If you find a result relevant make sure to cite the result using [[number](URL)] notation after the reference. End your answer with a summary. A\\nQuery:\",\n",
    ")\n",
    "\n",
    "chatbot = Chatbot(config=constants.CHATGPT_CREDS)\n",
    "response = \"\"\n",
    "\n",
    "for data in chatbot.ask(prompt):\n",
    "    response = data[\"message\"]\n",
    "\n",
    "# print the response variable  as ipython markdown\n",
    "\n",
    "display(Markdown(f\"## Question\\n\\n\"))\n",
    "display(Markdown(f\"### {query1}\\n\\n\"))\n",
    "display(Markdown(f\"## ChatGPT Response: \\n\\n {response}\"))\n",
    "print_papers(df, k=df.shape[0]) if df.shape[0] < 8 else print_papers(df, k=8)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|██████████| 19/19 [01:30<00:00,  4.74s/it]\n"
     ]
    },
    {
     "data": {
      "text/markdown": [
       "## Question\n",
       "\n"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "### what is the impact of transformers on language\n",
       "\n"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "## ChatGPT Response: \n",
       "\n",
       " Transformers have had a significant impact on language in a variety of ways. One of the most prominent areas of impact is in natural language processing (NLP), which has been revolutionized by the development of transformer-based language models. These models have been used for a variety of NLP tasks, including machine translation [[1](https://www.semanticscholar.org/paper/3d67a1fd78c7e74813e898b67fa5965a7d2ce25e)], code summarization [[2](https://www.semanticscholar.org/paper/761472e1c6012e001438d22ef771a0defbac3695)], detecting patronizing and condescending language [[3](https://www.semanticscholar.org/paper/36f7da6d848070f74b48bdf2accace00a66202a3)], and spoken language understanding tasks [[4](https://www.semanticscholar.org/paper/0de905634bd3d42a612ffab7ddd7a814e7e655bb)].\n",
       "\n",
       "Moreover, transformer-based models have been shown to be effective in domain adaptation for bidirectional transformers [[7](https://www.semanticscholar.org/paper/314107f8ff8bc134703c49c012ea6f6d9eb17840)], multi-modal transformer models have been explored for image captioning [[8](https://www.semanticscholar.org/paper/bec1f88b86bd06e90e2d735be9c54f890ca5c66e)], and pre-training objectives have been used in software engineering to boost performance in code-related tasks [[6](https://www.semanticscholar.org/paper/2e761fe1771d83962eacb7acc473d132bf92b1d3)]. Additionally, transformers have also been used to incorporate document timestamps into language models, which has shown to have a positive impact on downstream classification tasks and masked language modeling [[5](https://www.semanticscholar.org/paper/a702e0a38086c54c65ef7f2290eea726e13e0ca7)].\n",
       "\n",
       "Overall, transformers have had a significant impact on language, particularly in the field of natural language processing, where transformer-based models have been used for a variety of tasks with great success. They have also been explored in other areas such as software engineering and image captioning, showing the versatility of the technology."
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [1] [A Study on the journey of Natural Language Processing models: from Symbolic Natural Language Processing to Bidirectional Encoder Representations from Transformers](https://www.semanticscholar.org/paper/3d67a1fd78c7e74813e898b67fa5965a7d2ce25e) - International Journal of Scientific Research in Computer Science Engineering and Information Technology, 2021"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [2] [Code Summarization: Do Transformers Really Understand Code?](https://www.semanticscholar.org/paper/761472e1c6012e001438d22ef771a0defbac3695) - , 2022"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [3] [PALI-NLP at SemEval-2022 Task 4: Discriminative Fine-tuning of Transformers for Patronizing and Condescending Language Detection](https://www.semanticscholar.org/paper/36f7da6d848070f74b48bdf2accace00a66202a3) - International Workshop on Semantic Evaluation, 2022"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [4] [Benchmarking Transformers-based models on French Spoken Language Understanding tasks](https://www.semanticscholar.org/paper/0de905634bd3d42a612ffab7ddd7a814e7e655bb) - Interspeech, 2022"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [5] [Temporal Language Modeling for Short Text Document Classification with Transformers](https://www.semanticscholar.org/paper/a702e0a38086c54c65ef7f2290eea726e13e0ca7) - Conference on Computer Science and Information Systems, 2022"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [6] [Automating Code-Related Tasks Through Transformers: The Impact of Pre-training](https://www.semanticscholar.org/paper/2e761fe1771d83962eacb7acc473d132bf92b1d3) - ArXiv, 2023"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [7] [Exploiting Auxiliary Data for Offensive Language Detection with Bidirectional Transformers](https://www.semanticscholar.org/paper/314107f8ff8bc134703c49c012ea6f6d9eb17840) - WOAH, 2021"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [8] [What Does a Language-And-Vision Transformer See: The Impact of Semantic Information on Visual Representations](https://www.semanticscholar.org/paper/bec1f88b86bd06e90e2d735be9c54f890ca5c66e) - Frontiers in Artificial Intelligence, 2021"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "query1 = \"what is the impact of transformers on language\"\n",
    "# search and re-rank for the query\n",
    "df = get_results(query1)\n",
    "df[\"tldr\"] = df[\"abstract\"].progress_apply(get_response)\n",
    "df, query1 = rerank(df, query1, \"tldr\")\n",
    "prompt = answer_question_chatgpt(\n",
    "    df,\n",
    "    query1,\n",
    "    k=8,\n",
    "    instructions=\"Instructions: Using the provided web search results, write a comprehensive reply to the given query. If you find a result relevant make sure to cite the result using [[number](URL)] notation after the reference. End your answer with a summary. A\\nQuery:\",\n",
    ")\n",
    "\n",
    "chatbot = Chatbot(config=constants.CHATGPT_CREDS)\n",
    "response = \"\"\n",
    "\n",
    "for data in chatbot.ask(prompt):\n",
    "    response = data[\"message\"]\n",
    "\n",
    "# print the response variable  as ipython markdown\n",
    "\n",
    "display(Markdown(f\"## Question\\n\\n\"))\n",
    "display(Markdown(f\"### {query1}\\n\\n\"))\n",
    "display(Markdown(f\"## ChatGPT Response: \\n\\n {response}\"))\n",
    "print_papers(df, k=df.shape[0]) if df.shape[0] < 8 else print_papers(df, k=8)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|██████████| 18/18 [01:33<00:00,  5.20s/it]\n"
     ]
    },
    {
     "data": {
      "text/markdown": [
       "## Question\n",
       "\n"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "### How does iron supplementation affect anemia?\n",
       "\n"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "## ChatGPT Response: \n",
       "\n",
       " Iron supplementation is often used to address iron-deficiency anemia and improve iron status in various populations. However, the effects of iron supplementation on anemia and overall health outcomes are not always clear-cut and depend on various factors, including the individual's iron status, age, and health condition.\n",
       "\n",
       "A review of the literature suggests that anemia of obesity is independent of iron intake and does not respond well to increased iron ingestion [[1]]. Therefore, countries with a high rate of obesity should assess the health impact of fortification and supplementation with iron due to their potential drawbacks.\n",
       "\n",
       "In infants, the risks and benefits of iron supplementation have been reported in controlled studies and relevant animal models [[2]]. The mechanisms underlying the beneficial and adverse effects of iron supplementation in infants have been discussed. Iron deficiency is the most prevalent human micronutrient deficiency, and oral iron supplementation is used to address iron-deficiency anemia and reduce associated stunting. However, iron supplementation can also increase infection risk since restriction of iron availability serves as an innate immune mechanism against invading pathogens [[4]].\n",
       "\n",
       "Iron supplementation during pregnancy is crucial to meet the increased iron demands of the fetus and mother. Daily versus intermittent administration of iron supplements was compared in healthy pregnant women, with the study showing that daily iron supplementation had a more significant impact on hemoglobin, red blood cell, and iron indices [[3]]. However, iron supplementation in mothers with high hemoglobin did not increase the risk of maternal anemia or low birth weight but increased the risk of jaundice in newborns [[6]]. \n",
       "\n",
       "In patients with chronic kidney disease (CKD), anemia is prevalent, and iron supplementation is often used to treat it [[5]]. Additionally, iron supplementation in infancy has been associated with more adaptive behavior at age 10 years, especially in affect and response to reward [[7]]. \n",
       "\n",
       "Overall, iron supplementation can effectively address iron-deficiency anemia in various populations, but its effects on health outcomes depend on various factors, including age, health status, and dosing regimen. It is crucial to assess individual iron status and consider potential risks and benefits before prescribing iron supplementation. [[1][2][3][4][5][6][7]]"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [1] [Iron Fortification and Supplementation: Fighting Anemia of Chronic Diseases or Fueling Obesity?](https://www.semanticscholar.org/paper/d17242c18f0bf942f258db2fb163933e3ea1a8f0) - Current Developments in Nutrition, 2021"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [2] [Benefits and Risks of Early Life Iron Supplementation](https://www.semanticscholar.org/paper/459828144e59ace72493adae1068d460d0989c46) - Nutrients, 2022"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [3] [Efficacy of daily versus intermittent administration of iron supplementation in anemia or blood indices during pregnancy](https://www.semanticscholar.org/paper/a6d2264066edf46f03f1c7df46b0e8079f8e0b63) - , 2013"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [4] [Effects of iron deficiency and iron supplementation at the host-microbiota interface: Could a piglet model unravel complexities of the underlying mechanisms?](https://www.semanticscholar.org/paper/2cbf181eaa9bf0ab7bc4d5426da4cabf6cea53bf) - Frontiers in Nutrition, 2022"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [5] [Oral iron supplementation: Potential implications for the gut microbiome and metabolome in patients with CKD](https://www.semanticscholar.org/paper/1521596d4ff14f64ad260fbdb1e54409b4596a69) - Hemodialysis international. International Symposium on Home Hemodialysis, 2017"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [6] [The effect of iron supplementation on birth outcome in mothers with high hemoglobin: a randomized double-blind clinical trial study](https://www.semanticscholar.org/paper/f312120e9e0fa93842dadf5cc55ee367e4d22615) - , 2019"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [7] [Iron supplementation in infancy contributes to more adaptive behavior at 10 years of age.](https://www.semanticscholar.org/paper/c23abf343b9d78ae2496d461270e2279d02b2ecc) - Journal of NutriLife, 2014"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [8] [Iron Deficiency Anemia in Pregnancy: Novel Approaches for an Old Problem](https://www.semanticscholar.org/paper/08a1b7100972a9204076cf863448dcdf23ba9c38) - Oman Medical Journal, 2020"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "query1 = \"How does iron supplementation affect anemia?\"\n",
    "# search and re-rank for the query\n",
    "df = get_results(query1)\n",
    "df[\"tldr\"] = df[\"abstract\"].progress_apply(get_response)\n",
    "df, query1 = rerank(df, query1, \"tldr\")\n",
    "prompt = answer_question_chatgpt(\n",
    "    df,\n",
    "    query1,\n",
    "    k=8,\n",
    "    instructions=\"Instructions: Using the provided web search results, write a comprehensive reply to the given query. If you find a result relevant make sure to cite the result using [[number](URL)] notation after the reference. End your answer with a summary. A\\nQuery:\",\n",
    ")\n",
    "\n",
    "chatbot = Chatbot(config=constants.CHATGPT_CREDS)\n",
    "response = \"\"\n",
    "\n",
    "for data in chatbot.ask(prompt):\n",
    "    response = data[\"message\"]\n",
    "\n",
    "# print the response variable  as ipython markdown\n",
    "\n",
    "display(Markdown(f\"## Question\\n\\n\"))\n",
    "display(Markdown(f\"### {query1}\\n\\n\"))\n",
    "display(Markdown(f\"## ChatGPT Response: \\n\\n {response}\"))\n",
    "print_papers(df, k=df.shape[0]) if df.shape[0] < 8 else print_papers(df, k=8)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|██████████| 15/15 [01:11<00:00,  4.75s/it]\n"
     ]
    },
    {
     "data": {
      "text/markdown": [
       "## Question\n",
       "\n"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "### state of the art in math information retrieval\n",
       "\n"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "## ChatGPT Response: \n",
       "\n",
       " Math information retrieval (IR) is a subfield of IR that focuses on searching for mathematical content. In recent years, several approaches have been proposed to improve the effectiveness of math IR systems. Here are some notable works on the state-of-the-art in math information retrieval:\n",
       "\n",
       "One thesis provides an overview of the history of regular IR evaluation and the most common methods and measures used for evaluation. It also summarizes the state-of-the-art of MIaS math search system [[1](https://www.semanticscholar.org/paper/5aa5af3479005615277f5a7a3985e73bcbb19af0)].\n",
       "\n",
       "Another thesis explores how to adapt the state-of-the-art BM25 text ranking method to work well when searching for math and text together. Symbol layout trees are used to represent math formulas, and features are extracted from the trees, which are then used as search terms for BM25 [[2](https://www.semanticscholar.org/paper/679831ebcc45def519daadbb081cb022b8fe139e)].\n",
       "\n",
       "An analysis of a variable-size vector-based approach for searching mathematical data has been conducted. The proposed approach has been tested on Math type data of Math Stack Exchange of ARQMath task, and the proficiency of the same is represented in terms of nDCG, MAP, and Precision at 10 measures [[3](https://www.semanticscholar.org/paper/ebe0804f47a61c58cd0f13357a8d689b5d722cda)].\n",
       "\n",
       "Another work shows that using the soft vector space model consistently improves effectiveness compared to using standard sparse retrieval techniques on math information retrieval. It also shows that different math representations and different ways of combining text and math benefit from different notions of similarity between tokens [[4](https://www.semanticscholar.org/paper/c049c2e23356517fb592209424a65bbe9f49b5c4)].\n",
       "\n",
       "Furthermore, two representative bi-encoder models for token-level and passage-level dense retrieval have been evaluated on recent Math Information Retrieval (MIR) tasks. The results show that bi-encoder models are highly complementary to existing structure search methods, and the state-of-the-art on MIR datasets has been advanced [[5](https://www.semanticscholar.org/paper/859cbed6555319c3c31b84c1001a1df9a34889e2)].\n",
       "\n",
       "Moreover, a team performed a mathematical search for the NTCIR pilot Math Task. The approach is the similarity search based on MathMLization and second-generation scalable full-text search engine Math Indexer and Searcher (MIaS) with attested state-of-the-art information retrieval techniques [[6](https://www.semanticscholar.org/paper/5d73aa3f88523fead1f5234c22e19b54c9b7b74f)].\n",
       "\n",
       "Finally, another work describes the exploitation of semantic annotations in the design and architecture of MIaS system for mathematics retrieval. It argues for an approach based on natural language processing techniques as used in corpus management systems such as the Sketch Engine, that will reach web scalability and avoid inference problems [[7](https://www.semanticscholar.org/paper/6dd9a8b1caa940798c2035aad1fd73aa6b09a2ce)].\n",
       "\n",
       "In summary, the state-of-the-art in math information retrieval involves various approaches such as adapting text ranking methods to work well with math and using vector space models for better effectiveness. Bi-encoder models have also shown to be highly complementary to existing structure search methods, and systems such as MIaS with attested state-of-the"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [1] [Evaluation of Mathematics Retrieval](https://www.semanticscholar.org/paper/5aa5af3479005615277f5a7a3985e73bcbb19af0) - , 2013"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [2] [Math Information Retrieval using a Text Search Engine](https://www.semanticscholar.org/paper/679831ebcc45def519daadbb081cb022b8fe139e) - , 2018"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [3] [An Analysis of Variable-Size Vector Based Approach for Formula Searching](https://www.semanticscholar.org/paper/ebe0804f47a61c58cd0f13357a8d689b5d722cda) - Conference and Labs of the Evaluation Forum, 2020"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [4] [Combining Sparse and Dense Information Retrieval Soft Vector Space Model and MathBERTa at ARQMath-3 Task 1 (Answer Retrieval)](https://www.semanticscholar.org/paper/c049c2e23356517fb592209424a65bbe9f49b5c4) - , 2022"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [5] [Evaluating Token-Level and Passage-Level Dense Retrieval Models for Math Information Retrieval](https://www.semanticscholar.org/paper/859cbed6555319c3c31b84c1001a1df9a34889e2) - Conference on Empirical Methods in Natural Language Processing, 2022"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [6] [Math Indexer and Searcher under the Hood: History and Development of a Winning Strategy](https://www.semanticscholar.org/paper/5d73aa3f88523fead1f5234c22e19b54c9b7b74f) - NTCIR Conference on Evaluation of Information Access Technologies, 2014"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [7] [Exploiting semantic annotations in math information retrieval](https://www.semanticscholar.org/paper/6dd9a8b1caa940798c2035aad1fd73aa6b09a2ce) - Exploiting Semantic Annotations in Information Retrieval, 2012"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [8] [Towards Unsupervised Dense Information Retrieval with Contrastive Learning](https://www.semanticscholar.org/paper/5a37124345d0fb44ac1b4809dda85bf61ab79564) - ArXiv, 2021"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "query1 = \"state of the art in math information retrieval\"\n",
    "# search and re-rank for the query\n",
    "df = get_results(query1)\n",
    "df, query1 = rerank(df, query1)\n",
    "df[\"tldr\"] = df[\"title_abs\"].progress_apply(get_response)\n",
    "\n",
    "prompt = answer_question_chatgpt(\n",
    "    df,\n",
    "    query1,\n",
    "    k=8,\n",
    "    instructions=\"Instructions: Using the provided web search results, write a comprehensive reply to the given query. If you find a result relevant make sure to cite the result using [[number](URL)] notation after the reference. End your answer with a summary. A\\nQuery:\",\n",
    ")\n",
    "\n",
    "chatbot = Chatbot(config=constants.CHATGPT_CREDS)\n",
    "response = \"\"\n",
    "\n",
    "for data in chatbot.ask(prompt):\n",
    "    response = data[\"message\"]\n",
    "\n",
    "# print the response variable  as ipython markdown\n",
    "\n",
    "display(Markdown(f\"## Question\\n\\n\"))\n",
    "display(Markdown(f\"### {query1}\\n\\n\"))\n",
    "display(Markdown(f\"## ChatGPT Response: \\n\\n {response}\"))\n",
    "print_papers(df, k=df.shape[0]) if df.shape[0] < 8 else print_papers(df, k=8)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|██████████| 9/9 [00:31<00:00,  3.52s/it]\n"
     ]
    },
    {
     "data": {
      "text/markdown": [
       "## Question\n",
       "\n"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "### Do mitochondria play a role in remodelling lace plant leaves during programmed cell death?\n",
       "\n"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "## ChatGPT Response: \n",
       "\n",
       " \n",
       "\n",
       "There is evidence to suggest that mitochondria play a role in programmed cell death (PCD) during remodelling of lace plant leaves. In a study on Aponogeton madagascariensis, it was found that mitochondrial electron transport chain was affected during PCD induced by Ochratoxin A[SEP]Ochratoxin A (OTA) in Arabidopsis thaliana, leading to the induction of PCD [[4](https://www.semanticscholar.org/paper/96ba64774fec876fa8b15cf365aab77c9f1f2f75)]. Additionally, hexokinase-encoding Hxk1 was associated with mitochondria and its virus-induced gene silencing led to necrotic lesions on leaves and retarded plant growth in Nicotiana benthamiana [[5](https://www.semanticscholar.org/paper/9264cffabea5abd0c5e50f1729fdea836a764619)]. Furthermore, the role of Atg16 in the development of PCD in lace plant leaves was determined, and it was found that Atg16 levels were significantly higher in leaves treated with rapamycin, which is known to induce mitochondrial autophagy [[6](https://www.semanticscholar.org/paper/10e2894347d4da2b6fd5475557954e85ea758e61)]. While the exact role of mitochondria in remodelling of lace plant leaves during PCD is still not fully understood, these studies suggest that mitochondria are involved in the process.\n",
       "\n",
       "In summary, there is some evidence to suggest that mitochondria play a role in remodelling lace plant leaves during programmed cell death, specifically through their involvement in mitochondrial electron transport chain and autophagy. However, further research is needed to fully understand the mechanisms underlying this process."
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [1] [Unveiling Interactions among Mitochondria, Caspase-Like Proteases, and the Actin Cytoskeleton during Plant Programmed Cell Death (PCD)](https://www.semanticscholar.org/paper/456631fbc36bf420f048942536d2cad3d42c5419) - PLoS ONE, 2013"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [2] [Hsp70 plays a role in programmed cell death during the remodelling of leaves of the lace plant (Aponogeton madagascariensis).](https://www.semanticscholar.org/paper/0307edc559b9f05bbdf22fcc6cf3a183a924b60a) - Journal of Experimental Botany, 2019"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [3] [Determining the effect of calcium on cell death rate and perforation formation during leaf development in the novel model system, the lace plant (Aponogeton madagascariensis)](https://www.semanticscholar.org/paper/303e857c7cc44ece8d2c0c13d219d261c14bc4f6) - Journal of Microscopy, 2019"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [4] [iTRAQ Mitoproteome Analysis Reveals Mechanisms of Programmed Cell Death in Arabidopsis thaliana Induced by Ochratoxin A](https://www.semanticscholar.org/paper/96ba64774fec876fa8b15cf365aab77c9f1f2f75) - Toxins, 2017"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [5] [Mitochondria-Associated Hexokinases Play a Role in the Control of Programmed Cell Death in Nicotiana benthamiana[W]](https://www.semanticscholar.org/paper/9264cffabea5abd0c5e50f1729fdea836a764619) - The Plant Cell Online, 2006"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [6] [The role of Atg16 in autophagy, anthocyanin biosynthesis, and programmed cell death in leaves of the lace plant (Aponogeton madagascariensis)](https://www.semanticscholar.org/paper/10e2894347d4da2b6fd5475557954e85ea758e61) - PLoS ONE, 2023"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [7] [The role of auxin in developmentally regulated programmed cell death in lace plant.](https://www.semanticscholar.org/paper/e2b8fd8ecd1332de2b0fb5391758266fd4491932) - American-Eurasian journal of botany, 2020"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [8] [Leaf-shape remodeling: programmed cell death in fistular leaves of Allium fistulosum.](https://www.semanticscholar.org/paper/bfd5704a7267849bf890f6ca3bd1e09f5084cb43) - Physiologia Plantarum : An International Journal for Plant Biology, 2015"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "query1 = \"Do mitochondria play a role in remodelling lace plant leaves during programmed cell death?\"\n",
    "# search and re-rank for the query\n",
    "df = get_results(query1)\n",
    "df, query1 = rerank(df, query1)\n",
    "df[\"tldr\"] = df[\"title_abs\"].progress_apply(get_response)\n",
    "\n",
    "prompt = answer_question_chatgpt(\n",
    "    df,\n",
    "    query1,\n",
    "    k=8,\n",
    "    instructions=\"Instructions: Using the provided web search results, write a comprehensive reply to the given query. If you find a result relevant make sure to cite the result using [[number](URL)] notation after the reference. End your answer with a summary. A\\nQuery:\",\n",
    ")\n",
    "\n",
    "chatbot = Chatbot(config=constants.CHATGPT_CREDS)\n",
    "response = \"\"\n",
    "\n",
    "for data in chatbot.ask(prompt):\n",
    "    response = data[\"message\"]\n",
    "\n",
    "# print the response variable  as ipython markdown\n",
    "\n",
    "display(Markdown(f\"## Question\\n\\n\"))\n",
    "display(Markdown(f\"### {query1}\\n\\n\"))\n",
    "display(Markdown(f\"## ChatGPT Response: \\n\\n {response}\"))\n",
    "print_papers(df, k=df.shape[0]) if df.shape[0] < 8 else print_papers(df, k=8)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|██████████| 17/17 [00:57<00:00,  3.41s/it]\n"
     ]
    },
    {
     "data": {
      "text/markdown": [
       "## Question\n",
       "\n"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "### Double balloon enteroscopy: is it efficacious and safe in a community setting?\n",
       "\n"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "## ChatGPT Response: \n",
       "\n",
       " Double balloon enteroscopy (DBE) is a technique used to inspect the entire small bowel [2]. While DBE has been extensively used in tertiary referral centers, little literature exists on its efficacy and safety in a community setting [1]. One study reported the use of DBE in a community hospital, where 88 DBE procedures were performed on 66 patients from March 2007 to January 2011 [1]. The study did not report any major complications, and the diagnostic yield was 55.7%, suggesting that DBE is both efficacious and safe in a community setting. \n",
       "\n",
       "Comparative studies of DBE and other device-assisted enteroscopy (DAE) techniques, such as single-balloon enteroscopy (SBE) and spiral enteroscopy (SE), have shown that most parameters, including diagnostic and therapeutic yield, are comparable among these techniques [4,5]. However, the procedure duration appears to be shorter for SE than for DBE and SBE [4]. \n",
       "\n",
       "In terms of patient preparation, the European Society of Gastrointestinal Endoscopy (ESGE) has recommended that small-bowel capsule endoscopy (SBCE) and DAE for diagnosis and treatment of small-bowel disorders should be performed as outpatient procedures [6]. SBCE should also be performed as an outpatient procedure if possible, since completion rates are higher in outpatients than in inpatients [6]. \n",
       "\n",
       "In conclusion, DBE appears to be both efficacious and safe in a community setting, with a diagnostic yield of 55.7% and no reported major complications [1]. Comparative studies suggest that DBE is comparable to other DAE techniques such as SBE and SE in terms of diagnostic and therapeutic yield [4,5]. The ESGE recommends that SBCE and DAE should be performed as outpatient procedures if possible [6]."
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [1] [Double balloon enteroscopy: is it efficacious and safe in a community setting?](https://www.semanticscholar.org/paper/f734687c37fb3fe6deb0c446c371e7e3c8fce110) - Minerva Gastroenterologica e Dietologica, 2013"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [2] [[Double-balloon enteroscopy in children: indications, safety, and efficacy].](https://www.semanticscholar.org/paper/d06a8473e113c0f6633186185cc3e4eb8989e077) - Nederlandsch tijdschrift voor geneeskunde, 2012"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [3] [PTU-056 Highly Successful, Minimally Invasive Enteral Access By Double-balloon Enteroscopy (dbe) And Laparoscopic-assisted Dbe](https://www.semanticscholar.org/paper/fbed6c621c02ba451835bff0c66aaabada9f8c72) - Gut, 2014"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [4] [Deep Enteroscopy: Which Technique Will Survive?](https://www.semanticscholar.org/paper/55bc22b417f7eac1c68d18457534a8d2787d9789) - Clinical Endoscopy, 2013"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [5] [Device-assisted enteroscopy in the UK: description of a large tertiary case series under conscious sedation](https://www.semanticscholar.org/paper/52a69ca6f5e6f75567cbe1631c7de01b8a99002f) - Frontline Gastroenterology, 2017"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [6] [Small-bowel capsule endoscopy and device-assisted enteroscopy for diagnosis and treatment of small-bowel disorders: European Society of Gastrointestinal Endoscopy (ESGE) Technical Review](https://www.semanticscholar.org/paper/ad0777d02b594d1eee8fc5955868261fe6dca01c) - Endoscopy, 2018"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [7] [PTH-070 Severe midgut bleeding from jejunal diverticula treated by double-balloon enteroscopy](https://www.semanticscholar.org/paper/e9e70c7f832cdebd6f8502e763ac0075d8d94387) - Posters, 2019"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [8] [Predictability of capsule endoscopy referred to a tertiary care center for double-balloon enteroscopy](https://www.semanticscholar.org/paper/f31111d64ea4ddf918613e31a1a2bce325918335) - European Journal of Gastroenterology and Hepathology, 2015"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "query1 = \"Double balloon enteroscopy: is it efficacious and safe in a community setting?\"\n",
    "# search and re-rank for the query\n",
    "df = get_results(query1)\n",
    "df, query1 = rerank(df, query1)\n",
    "df[\"tldr\"] = df[\"title_abs\"].progress_apply(get_response)\n",
    "\n",
    "prompt = answer_question_chatgpt(\n",
    "    df,\n",
    "    query1,\n",
    "    k=8,\n",
    "    instructions=\"Instructions: Using the provided web search results, write a comprehensive reply to the given query. If you find a result relevant make sure to cite the result using [[number](URL)] notation after the reference. End your answer with a summary. A\\nQuery:\",\n",
    ")\n",
    "\n",
    "chatbot = Chatbot(config=constants.CHATGPT_CREDS)\n",
    "response = \"\"\n",
    "\n",
    "for data in chatbot.ask(prompt):\n",
    "    response = data[\"message\"]\n",
    "\n",
    "# print the response variable  as ipython markdown\n",
    "\n",
    "display(Markdown(f\"## Question\\n\\n\"))\n",
    "display(Markdown(f\"### {query1}\\n\\n\"))\n",
    "display(Markdown(f\"## ChatGPT Response: \\n\\n {response}\"))\n",
    "print_papers(df, k=df.shape[0]) if df.shape[0] < 8 else print_papers(df, k=8)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|██████████| 19/19 [00:57<00:00,  3.03s/it]\n"
     ]
    },
    {
     "data": {
      "text/markdown": [
       "## Question\n",
       "\n"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "### Do Language Models Plagiarize?\n",
       "\n"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "## ChatGPT Response: \n",
       "\n",
       " \n",
       "\n",
       "Based on the research presented in [[1](https://www.semanticscholar.org/paper/03013e291fb3192b286147f5bdb5770e434f91b2)], it is suggested that plagiarism widely exists in language models (LMs). The study identifies three types of plagiarism among GPT-2 generated texts in comparison to its training data, and further analyzes the plagiarism patterns of fine-tuned LMs with domain-specific corpora, which are extensively used in practice. The results indicate that the size and decoding methods of LMs are strongly associated with the degrees of plagiarism they exhibit, and fine-tuned LMs' plagiarism patterns vary based on their corpus similarity and similarity. \n",
       "\n",
       "However, it is worth noting that the purpose of language models is not to plagiarize, but to generate text based on the input and data they have been trained on. Moreover, there are ongoing efforts to evaluate the quality and diversity of language models, such as the proposed unified framework in [[2](https://www.semanticscholar.org/paper/071a1d2898aa3a5628438f9f68a21a6cac4f7148)], which evaluates both diversity and quality based on the optimal error rate of predicting whether a sentence is human- or machine-generated. \n",
       "\n",
       "In another study, [[3](https://www.semanticscholar.org/paper/9405cc0d6169988371b2755e573cc28650d14dfe)], language models were shown to begin learning language processing tasks without any explicit supervision when trained on a new dataset of millions of webpages called WebText. When conditioned on a document plus questions, the answers generated by the language model reached 55 F1 on the CoQA dataset, matching or exceeding the performance of 3 out of 4 baseline systems without using the 127,000+ training examples. \n",
       "\n",
       "Other research, such as that presented in [[4](https://www.semanticscholar.org/paper/691b7d216440921dd03a98f8496bfe28530ef3da)], explores ways to improve language models' robustness for spoken language understanding. The study introduces Warped Language Models (WLM), which are self-supervised neural networks trained to fill in the blanks in a given sentence with masked tokens. The proposed modifications in the WLM, such as inserting and dropping random tokens, aim to improve their performance for spontaneous conversational speech recognition noise.\n",
       "\n",
       "In summary, while plagiarism patterns may exist in language models, it is important to recognize that their purpose is not to plagiarize but to generate text based on the input and data they have been trained on. Ongoing research aims to evaluate and improve the quality and diversity of language models and their performance for specific tasks."
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [1] [Do Language Models Plagiarize?](https://www.semanticscholar.org/paper/03013e291fb3192b286147f5bdb5770e434f91b2) - ArXiv, 2022"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [2] [Unifying Human and Statistical Evaluation for Natural Language Generation](https://www.semanticscholar.org/paper/071a1d2898aa3a5628438f9f68a21a6cac4f7148) - North American Chapter of the Association for Computational Linguistics, 2019"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [3] [Language Models are Unsupervised Multitask Learners](https://www.semanticscholar.org/paper/9405cc0d6169988371b2755e573cc28650d14dfe) - , 2019"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [4] [Warped Language Models for Noise Robust Language Understanding](https://www.semanticscholar.org/paper/691b7d216440921dd03a98f8496bfe28530ef3da) - Spoken Language Technology Workshop, 2020"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [5] [Improved Training Of Neural Trans-Dimensional Random field Language Models with Dynamic Noise-Contrastive Estimation](https://www.semanticscholar.org/paper/d40691bb0b5675748b67a69db6a71fd3ab290eb4) - Spoken Language Technology Workshop, 2018"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [6] [Dynamic adjustment of language models for automatic speech recognition using word similarity](https://www.semanticscholar.org/paper/140f202e9244db00c9a158e7cfd7e53964ae3aa6) - Spoken Language Technology Workshop, 2016"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [7] [End-to-end Speech Recognition With Word-Based Rnn Language Models](https://www.semanticscholar.org/paper/790eb7e93f1d3fce470c0222fd2be83bab55a428) - Spoken Language Technology Workshop, 2018"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [8] [Syntactic and Semantic Features For Code-Switching Factored Language Models](https://www.semanticscholar.org/paper/05bef557988228ebf67d8adcf0bfc7fc0b245b88) - IEEE/ACM Transactions on Audio Speech and Language Processing, 2015"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "query1 = \"Do Language Models Plagiarize?\"\n",
    "# search and re-rank for the query\n",
    "df = get_results(query1)\n",
    "df, query1 = rerank(df, query1)\n",
    "df[\"tldr\"] = df[\"title_abs\"].progress_apply(get_response)\n",
    "\n",
    "prompt = answer_question_chatgpt(\n",
    "    df,\n",
    "    query1,\n",
    "    k=8,\n",
    "    instructions=\"Instructions: Using the provided web search results, write a comprehensive reply to the given query. If you find a result relevant make sure to cite the result using [[number](URL)] notation after the reference. End your answer with a summary. A\\nQuery:\",\n",
    ")\n",
    "\n",
    "chatbot = Chatbot(config=constants.CHATGPT_CREDS)\n",
    "response = \"\"\n",
    "\n",
    "for data in chatbot.ask(prompt):\n",
    "    response = data[\"message\"]\n",
    "\n",
    "# print the response variable  as ipython markdown\n",
    "\n",
    "display(Markdown(f\"## Question\\n\\n\"))\n",
    "display(Markdown(f\"### {query1}\\n\\n\"))\n",
    "display(Markdown(f\"## ChatGPT Response: \\n\\n {response}\"))\n",
    "print_papers(df, k=df.shape[0]) if df.shape[0] < 8 else print_papers(df, k=8)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|██████████| 19/19 [00:52<00:00,  2.74s/it]\n"
     ]
    },
    {
     "data": {
      "text/markdown": [
       "## Question\n",
       "\n"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "### Do Language Models Plagiarize?\n",
       "\n"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "## ChatGPT Response: \n",
       "\n",
       " \n",
       "\n",
       "Language models (LMs) have been shown to be highly effective at natural language processing tasks such as summarization and chit-chat dialogue, without the need for explicit supervision [[3](https://www.semanticscholar.org/paper/9405cc0d6169988371b2755e573cc28650d14dfe)]. However, concerns have been raised about the potential for LMs to produce plagiarized content, particularly when fine-tuned on domain-specific corpora [[1](https://www.semanticscholar.org/paper/03013e291fb3192b286147f5bdb5770e434f91b2)]. In addition, the quality and diversity of generated text by LMs have been evaluated using a unified framework called HUSE, which combines human and statistical evaluation and has been shown to detect diversity defects that fool pure human evaluation [[2](https://www.semanticscholar.org/paper/071a1d2898aa3a5628438f9f68a21a6cac4f7148)]. Other research has explored methods for improving LM performance in specific contexts, such as automatic speech recognition (ASR) [[6](https://www.semanticscholar.org/paper/140f202e9244db00c9a158e7cfd7e53964ae3aa6)][[7](https://www.semanticscholar.org/paper/790eb7e93f1d3fce470c0222fd2be83bab55a428)] and factored language models for code-switching speech [[8](https://www.semanticscholar.org/paper/05bef557988228ebf67d8adcf0bfc7fc0b245b88)]. Overall, while LMs have shown great promise in natural language processing, further research is needed to address concerns about plagiarism and to improve the quality and diversity of generated text in various contexts. \n",
       "\n",
       "Summary: Language models have been shown to excel in natural language processing tasks such as summarization and chit-chat dialogue, but concerns about plagiarism have been raised, particularly when fine-tuned on domain-specific corpora. A unified framework called HUSE has been developed to evaluate the quality and diversity of generated text. Other research has explored methods for improving LM performance in specific contexts such as ASR and code-switching speech. Further research is needed to address concerns about plagiarism and improve the quality and diversity of generated text."
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [1] [Do Language Models Plagiarize?](https://www.semanticscholar.org/paper/03013e291fb3192b286147f5bdb5770e434f91b2) - ArXiv, 2022"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [2] [Unifying Human and Statistical Evaluation for Natural Language Generation](https://www.semanticscholar.org/paper/071a1d2898aa3a5628438f9f68a21a6cac4f7148) - North American Chapter of the Association for Computational Linguistics, 2019"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [3] [Language Models are Unsupervised Multitask Learners](https://www.semanticscholar.org/paper/9405cc0d6169988371b2755e573cc28650d14dfe) - , 2019"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [4] [Warped Language Models for Noise Robust Language Understanding](https://www.semanticscholar.org/paper/691b7d216440921dd03a98f8496bfe28530ef3da) - Spoken Language Technology Workshop, 2020"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [5] [Improved Training Of Neural Trans-Dimensional Random field Language Models with Dynamic Noise-Contrastive Estimation](https://www.semanticscholar.org/paper/d40691bb0b5675748b67a69db6a71fd3ab290eb4) - Spoken Language Technology Workshop, 2018"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [6] [Dynamic adjustment of language models for automatic speech recognition using word similarity](https://www.semanticscholar.org/paper/140f202e9244db00c9a158e7cfd7e53964ae3aa6) - Spoken Language Technology Workshop, 2016"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [7] [End-to-end Speech Recognition With Word-Based Rnn Language Models](https://www.semanticscholar.org/paper/790eb7e93f1d3fce470c0222fd2be83bab55a428) - Spoken Language Technology Workshop, 2018"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### [8] [Syntactic and Semantic Features For Code-Switching Factored Language Models](https://www.semanticscholar.org/paper/05bef557988228ebf67d8adcf0bfc7fc0b245b88) - IEEE/ACM Transactions on Audio Speech and Language Processing, 2015"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "query1 = \"Do Language Models Plagiarize?\"\n",
    "# search and re-rank for the query\n",
    "df = get_results(query1)\n",
    "df, query1 = rerank(df, query1)\n",
    "df[\"tldr\"] = df[\"title_abs\"].progress_apply(get_response)\n",
    "\n",
    "prompt = answer_question_chatgpt(\n",
    "    df,\n",
    "    query1,\n",
    "    k=8,\n",
    "    instructions=\"Instructions: Using the provided web search results, write a research paper introduction answering the given query. If you find a result relevant make sure to cite the result using [[number](URL)] notation after the reference. End your answer with a summary. A\\nQuery:\",\n",
    ")\n",
    "\n",
    "chatbot = Chatbot(config=constants.CHATGPT_CREDS)\n",
    "response = \"\"\n",
    "\n",
    "for data in chatbot.ask(prompt):\n",
    "    response = data[\"message\"]\n",
    "\n",
    "# print the response variable  as ipython markdown\n",
    "\n",
    "display(Markdown(f\"## Question\\n\\n\"))\n",
    "display(Markdown(f\"### {query1}\\n\\n\"))\n",
    "display(Markdown(f\"## ChatGPT Response: \\n\\n {response}\"))\n",
    "print_papers(df, k=df.shape[0]) if df.shape[0] < 8 else print_papers(df, k=8)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.0"
  },
  "orig_nbformat": 4,
  "vscode": {
   "interpreter": {
    "hash": "a0847d304dab7065fcaccefea22fcea69315154bc754f529a4085292c75f5720"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}