Merge branch 'dev' into feat/cog-954-rag-choice

topoteretes · Jan 15, 2025 · 3a8f62e · 3a8f62e
2 parents 5aa0f05 + 6653d73
commit 3a8f62e
Show file tree

Hide file tree

Showing 2 changed files with 37 additions and 40 deletions.
diff --git a/evals/eval_on_hotpot.py b/evals/eval_on_hotpot.py
@@ -14,6 +14,18 @@
 logger = logging.getLogger(__name__)
 
 
+async def answer_qa_instance(instance, context_provider):
+    context = await context_provider(instance)
+
+    search_results_str = "\n".join([context_item["text"] for context_item in search_results])
+
+    return search_results_str
+
+
+async def get_context_without_cognee(instance):
+    return instance["context"]
+
+
 async def answer_qa_instance(instance, context_provider):
     context = await context_provider(instance)
 
@@ -73,7 +85,6 @@ async def eval_on_QA_dataset(
 
     if eval_metric_name.startswith("promptfoo"):
         return await eval_metric.measure(instances, context_provider)
-
     else:
         return await deepeval_on_instances(instances, context_provider, eval_metric)
 

diff --git a/notebooks/llama_index_cognee_integration.ipynb b/notebooks/llama_index_cognee_integration.ipynb
@@ -1,5 +1,10 @@
 {
  "cells": [
+  {
+   "metadata": {},
+   "cell_type": "markdown",
+   "source": "[![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1EpokQ8Y_5jIJ7HdixZms81Oqgh2sp7-E?usp=sharing)"
+  },
   {
    "metadata": {},
    "cell_type": "markdown",
@@ -45,16 +50,14 @@
     "### 1. Setting Up the Environment\n",
     "\n",
     "Start by importing the required libraries and defining the environment:"
-   ],
-   "id": "d0d7a82d729bbef6"
+   ]
   },
   {
    "metadata": {},
    "cell_type": "code",
    "outputs": [],
    "execution_count": null,
-   "source": "!pip install llama-index-graph-rag-cognee==0.1.1",
-   "id": "598b52e384086512"
+   "source": "!pip install llama-index-graph-rag-cognee==0.1.2"
   },
   {
    "metadata": {},
@@ -69,8 +72,7 @@
     "\n",
     "if \"OPENAI_API_KEY\" not in os.environ:\n",
     "    os.environ[\"OPENAI_API_KEY\"] = \"\""
-   ],
-   "id": "892a1b1198ec662f"
+   ]
   },
   {
    "metadata": {},
@@ -81,8 +83,7 @@
     "### 2. Preparing the Dataset\n",
     "\n",
     "We’ll use a brief profile of an individual as our sample dataset:"
-   ],
-   "id": "a1f16f5ca5249ebb"
+   ]
   },
   {
    "metadata": {},
@@ -98,8 +99,7 @@
     "            text=\"David Thompson, Creative Graphic Designer with over 8 years of experience in visual design and branding.\"\n",
     "        ),\n",
     "    ]"
-   ],
-   "id": "198022c34636a3a0"
+   ]
   },
   {
    "metadata": {},
@@ -108,8 +108,7 @@
     "### 3. Initializing CogneeGraphRAG\n",
     "\n",
     "Instantiate the Cognee framework with configurations for LLM, graph, and database providers:"
-   ],
-   "id": "781ae78e52ff49a"
+   ]
   },
   {
    "metadata": {},
@@ -126,8 +125,7 @@
     "    relational_db_provider=\"sqlite\",\n",
     "    relational_db_name=\"cognee_db\",\n",
     ")"
-   ],
-   "id": "17e466821ab88d50"
+   ]
   },
   {
    "metadata": {},
@@ -136,16 +134,14 @@
     "### 4. Adding Data to Cognee\n",
     "\n",
     "Load the dataset into the cognee framework:"
-   ],
-   "id": "2a55d5be9de0ce81"
+   ]
   },
   {
    "metadata": {},
    "cell_type": "code",
    "outputs": [],
    "execution_count": null,
-   "source": "await cogneeRAG.add(documents, \"test\")",
-   "id": "238b716429aba541"
+   "source": "await cogneeRAG.add(documents, \"test\")"
   },
   {
    "metadata": {},
@@ -156,16 +152,14 @@
     "### 5. Processing Data into a Knowledge Graph\n",
     "\n",
     "Transform the data into a structured knowledge graph:"
-   ],
-   "id": "23e5316aa7e5dbc7"
+   ]
   },
   {
    "metadata": {},
    "cell_type": "code",
    "outputs": [],
    "execution_count": null,
-   "source": "await cogneeRAG.process_data(\"test\")",
-   "id": "c3b3063d428b07a2"
+   "source": "await cogneeRAG.process_data(\"test\")"
   },
   {
    "metadata": {},
@@ -176,8 +170,7 @@
     "### 6. Performing Searches\n",
     "\n",
     "### Answer prompt based on knowledge graph approach:"
-   ],
-   "id": "e32327de54e98dc8"
+   ]
   },
   {
    "metadata": {},
@@ -190,14 +183,12 @@
     "print(\"\\n\\nAnswer based on knowledge graph:\\n\")\n",
     "for result in search_results:\n",
     "    print(f\"{result}\\n\")"
-   ],
-   "id": "fddbf5916d1e50e5"
+   ]
   },
   {
    "metadata": {},
    "cell_type": "markdown",
-   "source": "### Answer prompt based on RAG approach:",
-   "id": "9246aed7f69ceb7e"
+   "source": "### Answer prompt based on RAG approach:"
   },
   {
    "metadata": {},
@@ -210,14 +201,12 @@
     "print(\"\\n\\nAnswer based on RAG:\\n\")\n",
     "for result in search_results:\n",
     "    print(f\"{result}\\n\")"
-   ],
-   "id": "fe77c7a7c57fe4e4"
+   ]
   },
   {
    "metadata": {},
    "cell_type": "markdown",
-   "source": "In conclusion, the results demonstrate a significant advantage of the knowledge graph-based approach (Graphrag) over the RAG approach. Graphrag successfully identified all the mentioned individuals across multiple documents, showcasing its ability to aggregate and infer information from a global context. In contrast, the RAG approach was limited to identifying individuals within a single document due to its chunking-based processing constraints. This highlights Graphrag's superior capability in comprehensively resolving queries that span across a broader corpus of interconnected data.",
-   "id": "89cc99628392eb99"
+   "source": "In conclusion, the results demonstrate a significant advantage of the knowledge graph-based approach (Graphrag) over the RAG approach. Graphrag successfully identified all the mentioned individuals across multiple documents, showcasing its ability to aggregate and infer information from a global context. In contrast, the RAG approach was limited to identifying individuals within a single document due to its chunking-based processing constraints. This highlights Graphrag's superior capability in comprehensively resolving queries that span across a broader corpus of interconnected data."
   },
   {
    "metadata": {},
@@ -226,8 +215,7 @@
     "### 7. Finding Related Nodes\n",
     "\n",
     "Explore relationships in the knowledge graph:"
-   ],
-   "id": "44c9b67c09763610"
+   ]
   },
   {
    "metadata": {},
@@ -240,8 +228,7 @@
     "print(\"\\n\\nRelated nodes are:\\n\")\n",
     "for node in related_nodes:\n",
     "    print(f\"{node}\\n\")"
-   ],
-   "id": "efbc1511586f46fe"
+   ]
   },
   {
    "metadata": {},
@@ -274,9 +261,8 @@
     "\n",
     "Try running it yourself\n",
     "\n",
-    "Join cognee community"
-   ],
-   "id": "d0f82c2c6eb7793"
+    "[join the cognee community](https://discord.gg/tV7pr5XSj7)"
+   ]
   }
  ],
  "metadata": {},