notebooks/first_steps.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# First Steps with HybridAGI\n",
    "\n",
    "In this notebook, we will guide you through the basic concepts and features of HybridAGI, a neuro-symbolic agent system that allows you to program its behavior using a graph-based prompt programming approach.\n",
    "\n",
    "In this notebook, we will cover the following topics:\n",
    "\n",
    "- Introduction to HybridAGI Graph Prompt Programming\n",
    "- Installation and setup of HybridAGI\n",
    "- Overview of the Domain Specific Language (DSL) of HybridAGI and its basic syntax\n",
    "- Examples of graph-based prompt programming using the DSL\n",
    "\n",
    "By the end of this notebook, you will have an understanding of the basic concepts of HybridAGI and be able to start experimenting with it on your own.\n",
    "\n",
    "## Setting up the Knowledge Base locally\n",
    "\n",
    "HybridAGI works with a low-latency hybrid vector/graph database called [FalkorDB](https://www.falkordb.com/). This knowledge base should be started when you work with HybridAGI.\n",
    "\n",
    "Start the database using docker with the following command in you terminal:\n",
    "\n",
    "```bash\n",
    "docker run -p 6379:6379 -p 3000:3000 -it --rm falkordb/falkordb:edge\n",
    "```\n",
    "\n",
    "## Setting up Ollama to test your pipelines locally\n",
    "\n",
    "To test your pipelines locally, we need to setup Ollama to run the LLMs in a separate server. \n",
    "\n",
    "### Linux:\n",
    "Open a terminal and use the following commands:\n",
    "\n",
    "```bash\n",
    "# Install it on linux\n",
    "curl -fsSL https://ollama.com/install.sh | sh\n",
    "# Then launch the Ollama server with\n",
    "ollama pull mistral\n",
    "ollama serve\n",
    "```\n",
    "\n",
    "### Windows:\n",
    "Downloads the Ollama executable [here](https://ollama.com/download/OllamaSetup.exe)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...\n",
      "To disable this warning, you can either:\n",
      "\t- Avoid using `tokenizers` before the fork if possible\n",
      "\t- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Processing /home/yoan/Workspace/HybridAGI\n",
      "  Installing build dependencies ... \u001b[?25ldone\n",
      "\u001b[?25h  Getting requirements to build wheel ... \u001b[?25ldone\n",
      "\u001b[?25h  Preparing metadata (pyproject.toml) ... \u001b[?25ldone\n",
      "\u001b[?25hRequirement already satisfied: colorama>=0.4.6 in /home/yoan/Workspace/HybridAGI/venv/lib/python3.10/site-packages (from hybridagi==0.2.3) (0.4.6)\n",
      "Requirement already satisfied: dspy-ai<3.0.0,>=2.4.9 in /home/yoan/Workspace/HybridAGI/venv/lib/python3.10/site-packages (from hybridagi==0.2.3) (2.4.10)\n",
      "Requirement already satisfied: duckduckgo-search>=5.2.1 in /home/yoan/Workspace/HybridAGI/venv/lib/python3.10/site-packages (from hybridagi==0.2.3) (6.1.7)\n",
      "Requirement already satisfied: faiss-cpu>=1.8.0.post1 in /home/yoan/Workspace/HybridAGI/venv/lib/python3.10/site-packages (from hybridagi==0.2.3) (1.8.0.post1)\n",
      "Requirement already satisfied: falkordb>=1.0.3 in /home/yoan/Workspace/HybridAGI/venv/lib/python3.10/site-packages (from hybridagi==0.2.3) (1.0.5)\n",
      "Requirement already satisfied: ipykernel<7.0.0,>=6.29.4 in /home/yoan/Workspace/HybridAGI/venv/lib/python3.10/site-packages (from hybridagi==0.2.3) (6.29.4)\n",
      "Requirement already satisfied: jupyter-client<9.0.0,>=8.3.0 in /home/yoan/Workspace/HybridAGI/venv/lib/python3.10/site-packages (from hybridagi==0.2.3) (8.6.2)\n",
      "Requirement already satisfied: jupyter-core<6.0.0,>=5.5.0 in /home/yoan/Workspace/HybridAGI/venv/lib/python3.10/site-packages (from hybridagi==0.2.3) (5.7.2)\n",
      "Requirement already satisfied: matplotlib>=3.9.0 in /home/yoan/Workspace/HybridAGI/venv/lib/python3.10/site-packages (from hybridagi==0.2.3) (3.9.0)\n",
      "Requirement already satisfied: mistralai>=0.1.8 in /home/yoan/Workspace/HybridAGI/venv/lib/python3.10/site-packages (from hybridagi==0.2.3) (0.4.1)\n",
      "Requirement already satisfied: pillow>=10.3.0 in /home/yoan/Workspace/HybridAGI/venv/lib/python3.10/site-packages (from hybridagi==0.2.3) (10.3.0)\n",
      "Requirement already satisfied: python-dotenv>=1.0.1 in /home/yoan/Workspace/HybridAGI/venv/lib/python3.10/site-packages (from hybridagi==0.2.3) (1.0.1)\n",
      "Requirement already satisfied: sentence-transformers>=2.6.0 in /home/yoan/Workspace/HybridAGI/venv/lib/python3.10/site-packages (from hybridagi==0.2.3) (3.0.1)\n",
      "Requirement already satisfied: tree-sitter>=0.22.3 in /home/yoan/Workspace/HybridAGI/venv/lib/python3.10/site-packages (from hybridagi==0.2.3) (0.22.3)\n",
      "Requirement already satisfied: tree-sitter-python==0.21.0 in /home/yoan/Workspace/HybridAGI/venv/lib/python3.10/site-packages (from hybridagi==0.2.3) (0.21.0)\n",
      "Requirement already satisfied: backoff~=2.2.1 in /home/yoan/Workspace/HybridAGI/venv/lib/python3.10/site-packages (from dspy-ai<3.0.0,>=2.4.9->hybridagi==0.2.3) (2.2.1)\n",
      "Requirement already satisfied: joblib~=1.3.2 in /home/yoan/Workspace/HybridAGI/venv/lib/python3.10/site-packages (from dspy-ai<3.0.0,>=2.4.9->hybridagi==0.2.3) (1.3.2)\n",
      "Requirement already satisfied: openai<2.0.0,>=0.28.1 in /home/yoan/Workspace/HybridAGI/venv/lib/python3.10/site-packages (from dspy-ai<3.0.0,>=2.4.9->hybridagi==0.2.3) (1.35.7)\n",
      "Requirement already satisfied: pandas in /home/yoan/Workspace/HybridAGI/venv/lib/python3.10/site-packages (from dspy-ai<3.0.0,>=2.4.9->hybridagi==0.2.3) (2.2.2)\n",
      "Requirement already satisfied: regex in /home/yoan/Workspace/HybridAGI/venv/lib/python3.10/site-packages (from dspy-ai<3.0.0,>=2.4.9->hybridagi==0.2.3) (2024.5.15)\n",
      "Requirement already satisfied: ujson in /home/yoan/Workspace/HybridAGI/venv/lib/python3.10/site-packages (from dspy-ai<3.0.0,>=2.4.9->hybridagi==0.2.3) (5.10.0)\n",
      "Requirement already satisfied: tqdm in /home/yoan/Workspace/HybridAGI/venv/lib/python3.10/site-packages (from dspy-ai<3.0.0,>=2.4.9->hybridagi==0.2.3) (4.66.4)\n",
      "Requirement already satisfied: datasets<3.0.0,>=2.14.6 in /home/yoan/Workspace/HybridAGI/venv/lib/python3.10/site-packages (from dspy-ai<3.0.0,>=2.4.9->hybridagi==0.2.3) (2.20.0)\n",
      "Requirement already satisfied: requests in /home/yoan/Workspace/HybridAGI/venv/lib/python3.10/site-packages (from dspy-ai<3.0.0,>=2.4.9->hybridagi==0.2.3) (2.32.3)\n",
      "Requirement already satisfied: optuna in /home/yoan/Workspace/HybridAGI/venv/lib/python3.10/site-packages (from dspy-ai<3.0.0,>=2.4.9->hybridagi==0.2.3) (3.6.1)\n",
      "Requirement already satisfied: pydantic~=2.0 in /home/yoan/Workspace/HybridAGI/venv/lib/python3.10/site-packages (from dspy-ai<3.0.0,>=2.4.9->hybridagi==0.2.3) (2.7.4)\n",
      "Requirement already satisfied: structlog in /home/yoan/Workspace/HybridAGI/venv/lib/python3.10/site-packages (from dspy-ai<3.0.0,>=2.4.9->hybridagi==0.2.3) (24.2.0)\n",
      "Requirement already satisfied: jinja2 in /home/yoan/Workspace/HybridAGI/venv/lib/python3.10/site-packages (from dspy-ai<3.0.0,>=2.4.9->hybridagi==0.2.3) (3.1.4)\n",
      "Requirement already satisfied: click>=8.1.7 in /home/yoan/Workspace/HybridAGI/venv/lib/python3.10/site-packages (from duckduckgo-search>=5.2.1->hybridagi==0.2.3) (8.1.7)\n",
      "Requirement already satisfied: pyreqwest-impersonate>=0.4.8 in /home/yoan/Workspace/HybridAGI/venv/lib/python3.10/site-packages (from duckduckgo-search>=5.2.1->hybridagi==0.2.3) (0.4.8)\n",
      "Requirement already satisfied: orjson>=3.10.5 in /home/yoan/Workspace/HybridAGI/venv/lib/python3.10/site-packages (from duckduckgo-search>=5.2.1->hybridagi==0.2.3) (3.10.5)\n",
      "Requirement already satisfied: numpy<2.0,>=1.0 in /home/yoan/Workspace/HybridAGI/venv/lib/python3.10/site-packages (from faiss-cpu>=1.8.0.post1->hybridagi==0.2.3) (1.26.4)\n",
      "Requirement already satisfied: packaging in /home/yoan/Workspace/HybridAGI/venv/lib/python3.10/site-packages (from faiss-cpu>=1.8.0.post1->hybridagi==0.2.3) (24.1)\n",
      "Requirement already satisfied: redis<6.0.0,>=5.0.1 in /home/yoan/Workspace/HybridAGI/venv/lib/python3.10/site-packages (from falkordb>=1.0.3->hybridagi==0.2.3) (5.0.7)\n",
      "Requirement already satisfied: comm>=0.1.1 in /home/yoan/Workspace/HybridAGI/venv/lib/python3.10/site-packages (from ipykernel<7.0.0,>=6.29.4->hybridagi==0.2.3) (0.2.2)\n",
      "Requirement already satisfied: debugpy>=1.6.5 in /home/yoan/Workspace/HybridAGI/venv/lib/python3.10/site-packages (from ipykernel<7.0.0,>=6.29.4->hybridagi==0.2.3) (1.8.2)\n",
      "Requirement already satisfied: ipython>=7.23.1 in /home/yoan/Workspace/HybridAGI/venv/lib/python3.10/site-packages (from ipykernel<7.0.0,>=6.29.4->hybridagi==0.2.3) (8.26.0)\n",
      "Requirement already satisfied: matplotlib-inline>=0.1 in /home/yoan/Workspace/HybridAGI/venv/lib/python3.10/site-packages (from ipykernel<7.0.0,>=6.29.4->hybridagi==0.2.3) (0.1.7)\n",
      "Requirement already satisfied: nest-asyncio in /home/yoan/Workspace/HybridAGI/venv/lib/python3.10/site-packages (from ipykernel<7.0.0,>=6.29.4->hybridagi==0.2.3) (1.6.0)\n",
      "Requirement already satisfied: psutil in /home/yoan/Workspace/HybridAGI/venv/lib/python3.10/site-packages (from ipykernel<7.0.0,>=6.29.4->hybridagi==0.2.3) (6.0.0)\n",
      "Requirement already satisfied: pyzmq>=24 in /home/yoan/Workspace/HybridAGI/venv/lib/python3.10/site-packages (from ipykernel<7.0.0,>=6.29.4->hybridagi==0.2.3) (26.0.3)\n",
      "Requirement already satisfied: tornado>=6.1 in /home/yoan/Workspace/HybridAGI/venv/lib/python3.10/site-packages (from ipykernel<7.0.0,>=6.29.4->hybridagi==0.2.3) (6.4.1)\n",
      "Requirement already satisfied: traitlets>=5.4.0 in /home/yoan/Workspace/HybridAGI/venv/lib/python3.10/site-packages (from ipykernel<7.0.0,>=6.29.4->hybridagi==0.2.3) (5.14.3)\n",
      "Requirement already satisfied: python-dateutil>=2.8.2 in /home/yoan/Workspace/HybridAGI/venv/lib/python3.10/site-packages (from jupyter-client<9.0.0,>=8.3.0->hybridagi==0.2.3) (2.9.0.post0)\n",
      "Requirement already satisfied: platformdirs>=2.5 in /home/yoan/Workspace/HybridAGI/venv/lib/python3.10/site-packages (from jupyter-core<6.0.0,>=5.5.0->hybridagi==0.2.3) (4.2.2)\n",
      "Requirement already satisfied: contourpy>=1.0.1 in /home/yoan/Workspace/HybridAGI/venv/lib/python3.10/site-packages (from matplotlib>=3.9.0->hybridagi==0.2.3) (1.2.1)\n",
      "Requirement already satisfied: cycler>=0.10 in /home/yoan/Workspace/HybridAGI/venv/lib/python3.10/site-packages (from matplotlib>=3.9.0->hybridagi==0.2.3) (0.12.1)\n",
      "Requirement already satisfied: fonttools>=4.22.0 in /home/yoan/Workspace/HybridAGI/venv/lib/python3.10/site-packages (from matplotlib>=3.9.0->hybridagi==0.2.3) (4.53.0)\n",
      "Requirement already satisfied: kiwisolver>=1.3.1 in /home/yoan/Workspace/HybridAGI/venv/lib/python3.10/site-packages (from matplotlib>=3.9.0->hybridagi==0.2.3) (1.4.5)\n",
      "Requirement already satisfied: pyparsing>=2.3.1 in /home/yoan/Workspace/HybridAGI/venv/lib/python3.10/site-packages (from matplotlib>=3.9.0->hybridagi==0.2.3) (3.1.2)\n",
      "Requirement already satisfied: httpx<1,>=0.25 in /home/yoan/Workspace/HybridAGI/venv/lib/python3.10/site-packages (from mistralai>=0.1.8->hybridagi==0.2.3) (0.27.0)\n",
      "Requirement already satisfied: transformers<5.0.0,>=4.34.0 in /home/yoan/Workspace/HybridAGI/venv/lib/python3.10/site-packages (from sentence-transformers>=2.6.0->hybridagi==0.2.3) (4.42.2)\n",
      "Requirement already satisfied: torch>=1.11.0 in /home/yoan/Workspace/HybridAGI/venv/lib/python3.10/site-packages (from sentence-transformers>=2.6.0->hybridagi==0.2.3) (2.3.1)\n",
      "Requirement already satisfied: scikit-learn in /home/yoan/Workspace/HybridAGI/venv/lib/python3.10/site-packages (from sentence-transformers>=2.6.0->hybridagi==0.2.3) (1.5.0)\n",
      "Requirement already satisfied: scipy in /home/yoan/Workspace/HybridAGI/venv/lib/python3.10/site-packages (from sentence-transformers>=2.6.0->hybridagi==0.2.3) (1.14.0)\n",
      "Requirement already satisfied: huggingface-hub>=0.15.1 in /home/yoan/Workspace/HybridAGI/venv/lib/python3.10/site-packages (from sentence-transformers>=2.6.0->hybridagi==0.2.3) (0.23.4)\n",
      "Requirement already satisfied: filelock in /home/yoan/Workspace/HybridAGI/venv/lib/python3.10/site-packages (from datasets<3.0.0,>=2.14.6->dspy-ai<3.0.0,>=2.4.9->hybridagi==0.2.3) (3.15.4)\n",
      "Requirement already satisfied: pyarrow>=15.0.0 in /home/yoan/Workspace/HybridAGI/venv/lib/python3.10/site-packages (from datasets<3.0.0,>=2.14.6->dspy-ai<3.0.0,>=2.4.9->hybridagi==0.2.3) (16.1.0)\n",
      "Requirement already satisfied: pyarrow-hotfix in /home/yoan/Workspace/HybridAGI/venv/lib/python3.10/site-packages (from datasets<3.0.0,>=2.14.6->dspy-ai<3.0.0,>=2.4.9->hybridagi==0.2.3) (0.6)\n",
      "Requirement already satisfied: dill<0.3.9,>=0.3.0 in /home/yoan/Workspace/HybridAGI/venv/lib/python3.10/site-packages (from datasets<3.0.0,>=2.14.6->dspy-ai<3.0.0,>=2.4.9->hybridagi==0.2.3) (0.3.8)\n",
      "Requirement already satisfied: xxhash in /home/yoan/Workspace/HybridAGI/venv/lib/python3.10/site-packages (from datasets<3.0.0,>=2.14.6->dspy-ai<3.0.0,>=2.4.9->hybridagi==0.2.3) (3.4.1)\n",
      "Requirement already satisfied: multiprocess in /home/yoan/Workspace/HybridAGI/venv/lib/python3.10/site-packages (from datasets<3.0.0,>=2.14.6->dspy-ai<3.0.0,>=2.4.9->hybridagi==0.2.3) (0.70.16)\n",
      "Requirement already satisfied: fsspec<=2024.5.0,>=2023.1.0 in /home/yoan/Workspace/HybridAGI/venv/lib/python3.10/site-packages (from fsspec[http]<=2024.5.0,>=2023.1.0->datasets<3.0.0,>=2.14.6->dspy-ai<3.0.0,>=2.4.9->hybridagi==0.2.3) (2024.5.0)\n",
      "Requirement already satisfied: aiohttp in /home/yoan/Workspace/HybridAGI/venv/lib/python3.10/site-packages (from datasets<3.0.0,>=2.14.6->dspy-ai<3.0.0,>=2.4.9->hybridagi==0.2.3) (3.9.5)\n",
      "Requirement already satisfied: pyyaml>=5.1 in /home/yoan/Workspace/HybridAGI/venv/lib/python3.10/site-packages (from datasets<3.0.0,>=2.14.6->dspy-ai<3.0.0,>=2.4.9->hybridagi==0.2.3) (6.0.1)\n",
      "Requirement already satisfied: anyio in /home/yoan/Workspace/HybridAGI/venv/lib/python3.10/site-packages (from httpx<1,>=0.25->mistralai>=0.1.8->hybridagi==0.2.3) (4.4.0)\n",
      "Requirement already satisfied: certifi in /home/yoan/Workspace/HybridAGI/venv/lib/python3.10/site-packages (from httpx<1,>=0.25->mistralai>=0.1.8->hybridagi==0.2.3) (2024.6.2)\n",
      "Requirement already satisfied: httpcore==1.* in /home/yoan/Workspace/HybridAGI/venv/lib/python3.10/site-packages (from httpx<1,>=0.25->mistralai>=0.1.8->hybridagi==0.2.3) (1.0.5)\n",
      "Requirement already satisfied: idna in /home/yoan/Workspace/HybridAGI/venv/lib/python3.10/site-packages (from httpx<1,>=0.25->mistralai>=0.1.8->hybridagi==0.2.3) (3.7)\n",
      "Requirement already satisfied: sniffio in /home/yoan/Workspace/HybridAGI/venv/lib/python3.10/site-packages (from httpx<1,>=0.25->mistralai>=0.1.8->hybridagi==0.2.3) (1.3.1)\n",
      "Requirement already satisfied: h11<0.15,>=0.13 in /home/yoan/Workspace/HybridAGI/venv/lib/python3.10/site-packages (from httpcore==1.*->httpx<1,>=0.25->mistralai>=0.1.8->hybridagi==0.2.3) (0.14.0)\n",
      "Requirement already satisfied: typing-extensions>=3.7.4.3 in /home/yoan/Workspace/HybridAGI/venv/lib/python3.10/site-packages (from huggingface-hub>=0.15.1->sentence-transformers>=2.6.0->hybridagi==0.2.3) (4.12.2)\n",
      "Requirement already satisfied: decorator in /home/yoan/Workspace/HybridAGI/venv/lib/python3.10/site-packages (from ipython>=7.23.1->ipykernel<7.0.0,>=6.29.4->hybridagi==0.2.3) (5.1.1)\n",
      "Requirement already satisfied: jedi>=0.16 in /home/yoan/Workspace/HybridAGI/venv/lib/python3.10/site-packages (from ipython>=7.23.1->ipykernel<7.0.0,>=6.29.4->hybridagi==0.2.3) (0.19.1)\n",
      "Requirement already satisfied: prompt-toolkit<3.1.0,>=3.0.41 in /home/yoan/Workspace/HybridAGI/venv/lib/python3.10/site-packages (from ipython>=7.23.1->ipykernel<7.0.0,>=6.29.4->hybridagi==0.2.3) (3.0.47)\n",
      "Requirement already satisfied: pygments>=2.4.0 in /home/yoan/Workspace/HybridAGI/venv/lib/python3.10/site-packages (from ipython>=7.23.1->ipykernel<7.0.0,>=6.29.4->hybridagi==0.2.3) (2.18.0)\n",
      "Requirement already satisfied: stack-data in /home/yoan/Workspace/HybridAGI/venv/lib/python3.10/site-packages (from ipython>=7.23.1->ipykernel<7.0.0,>=6.29.4->hybridagi==0.2.3) (0.6.3)\n",
      "Requirement already satisfied: exceptiongroup in /home/yoan/Workspace/HybridAGI/venv/lib/python3.10/site-packages (from ipython>=7.23.1->ipykernel<7.0.0,>=6.29.4->hybridagi==0.2.3) (1.2.1)\n",
      "Requirement already satisfied: pexpect>4.3 in /home/yoan/Workspace/HybridAGI/venv/lib/python3.10/site-packages (from ipython>=7.23.1->ipykernel<7.0.0,>=6.29.4->hybridagi==0.2.3) (4.9.0)\n",
      "Requirement already satisfied: distro<2,>=1.7.0 in /home/yoan/Workspace/HybridAGI/venv/lib/python3.10/site-packages (from openai<2.0.0,>=0.28.1->dspy-ai<3.0.0,>=2.4.9->hybridagi==0.2.3) (1.9.0)\n",
      "Requirement already satisfied: annotated-types>=0.4.0 in /home/yoan/Workspace/HybridAGI/venv/lib/python3.10/site-packages (from pydantic~=2.0->dspy-ai<3.0.0,>=2.4.9->hybridagi==0.2.3) (0.7.0)\n",
      "Requirement already satisfied: pydantic-core==2.18.4 in /home/yoan/Workspace/HybridAGI/venv/lib/python3.10/site-packages (from pydantic~=2.0->dspy-ai<3.0.0,>=2.4.9->hybridagi==0.2.3) (2.18.4)\n",
      "Requirement already satisfied: six>=1.5 in /home/yoan/Workspace/HybridAGI/venv/lib/python3.10/site-packages (from python-dateutil>=2.8.2->jupyter-client<9.0.0,>=8.3.0->hybridagi==0.2.3) (1.16.0)\n",
      "Requirement already satisfied: async-timeout>=4.0.3 in /home/yoan/Workspace/HybridAGI/venv/lib/python3.10/site-packages (from redis<6.0.0,>=5.0.1->falkordb>=1.0.3->hybridagi==0.2.3) (4.0.3)\n",
      "Requirement already satisfied: charset-normalizer<4,>=2 in /home/yoan/Workspace/HybridAGI/venv/lib/python3.10/site-packages (from requests->dspy-ai<3.0.0,>=2.4.9->hybridagi==0.2.3) (3.3.2)\n",
      "Requirement already satisfied: urllib3<3,>=1.21.1 in /home/yoan/Workspace/HybridAGI/venv/lib/python3.10/site-packages (from requests->dspy-ai<3.0.0,>=2.4.9->hybridagi==0.2.3) (2.2.2)\n",
      "Requirement already satisfied: sympy in /home/yoan/Workspace/HybridAGI/venv/lib/python3.10/site-packages (from torch>=1.11.0->sentence-transformers>=2.6.0->hybridagi==0.2.3) (1.12.1)\n",
      "Requirement already satisfied: networkx in /home/yoan/Workspace/HybridAGI/venv/lib/python3.10/site-packages (from torch>=1.11.0->sentence-transformers>=2.6.0->hybridagi==0.2.3) (3.3)\n",
      "Requirement already satisfied: nvidia-cuda-nvrtc-cu12==12.1.105 in /home/yoan/Workspace/HybridAGI/venv/lib/python3.10/site-packages (from torch>=1.11.0->sentence-transformers>=2.6.0->hybridagi==0.2.3) (12.1.105)\n",
      "Requirement already satisfied: nvidia-cuda-runtime-cu12==12.1.105 in /home/yoan/Workspace/HybridAGI/venv/lib/python3.10/site-packages (from torch>=1.11.0->sentence-transformers>=2.6.0->hybridagi==0.2.3) (12.1.105)\n",
      "Requirement already satisfied: nvidia-cuda-cupti-cu12==12.1.105 in /home/yoan/Workspace/HybridAGI/venv/lib/python3.10/site-packages (from torch>=1.11.0->sentence-transformers>=2.6.0->hybridagi==0.2.3) (12.1.105)\n",
      "Requirement already satisfied: nvidia-cudnn-cu12==8.9.2.26 in /home/yoan/Workspace/HybridAGI/venv/lib/python3.10/site-packages (from torch>=1.11.0->sentence-transformers>=2.6.0->hybridagi==0.2.3) (8.9.2.26)\n",
      "Requirement already satisfied: nvidia-cublas-cu12==12.1.3.1 in /home/yoan/Workspace/HybridAGI/venv/lib/python3.10/site-packages (from torch>=1.11.0->sentence-transformers>=2.6.0->hybridagi==0.2.3) (12.1.3.1)\n",
      "Requirement already satisfied: nvidia-cufft-cu12==11.0.2.54 in /home/yoan/Workspace/HybridAGI/venv/lib/python3.10/site-packages (from torch>=1.11.0->sentence-transformers>=2.6.0->hybridagi==0.2.3) (11.0.2.54)\n",
      "Requirement already satisfied: nvidia-curand-cu12==10.3.2.106 in /home/yoan/Workspace/HybridAGI/venv/lib/python3.10/site-packages (from torch>=1.11.0->sentence-transformers>=2.6.0->hybridagi==0.2.3) (10.3.2.106)\n",
      "Requirement already satisfied: nvidia-cusolver-cu12==11.4.5.107 in /home/yoan/Workspace/HybridAGI/venv/lib/python3.10/site-packages (from torch>=1.11.0->sentence-transformers>=2.6.0->hybridagi==0.2.3) (11.4.5.107)\n",
      "Requirement already satisfied: nvidia-cusparse-cu12==12.1.0.106 in /home/yoan/Workspace/HybridAGI/venv/lib/python3.10/site-packages (from torch>=1.11.0->sentence-transformers>=2.6.0->hybridagi==0.2.3) (12.1.0.106)\n",
      "Requirement already satisfied: nvidia-nccl-cu12==2.20.5 in /home/yoan/Workspace/HybridAGI/venv/lib/python3.10/site-packages (from torch>=1.11.0->sentence-transformers>=2.6.0->hybridagi==0.2.3) (2.20.5)\n",
      "Requirement already satisfied: nvidia-nvtx-cu12==12.1.105 in /home/yoan/Workspace/HybridAGI/venv/lib/python3.10/site-packages (from torch>=1.11.0->sentence-transformers>=2.6.0->hybridagi==0.2.3) (12.1.105)\n",
      "Requirement already satisfied: triton==2.3.1 in /home/yoan/Workspace/HybridAGI/venv/lib/python3.10/site-packages (from torch>=1.11.0->sentence-transformers>=2.6.0->hybridagi==0.2.3) (2.3.1)\n",
      "Requirement already satisfied: nvidia-nvjitlink-cu12 in /home/yoan/Workspace/HybridAGI/venv/lib/python3.10/site-packages (from nvidia-cusolver-cu12==11.4.5.107->torch>=1.11.0->sentence-transformers>=2.6.0->hybridagi==0.2.3) (12.5.40)\n",
      "Requirement already satisfied: safetensors>=0.4.1 in /home/yoan/Workspace/HybridAGI/venv/lib/python3.10/site-packages (from transformers<5.0.0,>=4.34.0->sentence-transformers>=2.6.0->hybridagi==0.2.3) (0.4.3)\n",
      "Requirement already satisfied: tokenizers<0.20,>=0.19 in /home/yoan/Workspace/HybridAGI/venv/lib/python3.10/site-packages (from transformers<5.0.0,>=4.34.0->sentence-transformers>=2.6.0->hybridagi==0.2.3) (0.19.1)\n",
      "Requirement already satisfied: MarkupSafe>=2.0 in /home/yoan/Workspace/HybridAGI/venv/lib/python3.10/site-packages (from jinja2->dspy-ai<3.0.0,>=2.4.9->hybridagi==0.2.3) (2.1.5)\n",
      "Requirement already satisfied: alembic>=1.5.0 in /home/yoan/Workspace/HybridAGI/venv/lib/python3.10/site-packages (from optuna->dspy-ai<3.0.0,>=2.4.9->hybridagi==0.2.3) (1.13.2)\n",
      "Requirement already satisfied: colorlog in /home/yoan/Workspace/HybridAGI/venv/lib/python3.10/site-packages (from optuna->dspy-ai<3.0.0,>=2.4.9->hybridagi==0.2.3) (6.8.2)\n",
      "Requirement already satisfied: sqlalchemy>=1.3.0 in /home/yoan/Workspace/HybridAGI/venv/lib/python3.10/site-packages (from optuna->dspy-ai<3.0.0,>=2.4.9->hybridagi==0.2.3) (2.0.31)\n",
      "Requirement already satisfied: pytz>=2020.1 in /home/yoan/Workspace/HybridAGI/venv/lib/python3.10/site-packages (from pandas->dspy-ai<3.0.0,>=2.4.9->hybridagi==0.2.3) (2024.1)\n",
      "Requirement already satisfied: tzdata>=2022.7 in /home/yoan/Workspace/HybridAGI/venv/lib/python3.10/site-packages (from pandas->dspy-ai<3.0.0,>=2.4.9->hybridagi==0.2.3) (2024.1)\n",
      "Requirement already satisfied: threadpoolctl>=3.1.0 in /home/yoan/Workspace/HybridAGI/venv/lib/python3.10/site-packages (from scikit-learn->sentence-transformers>=2.6.0->hybridagi==0.2.3) (3.5.0)\n",
      "Requirement already satisfied: Mako in /home/yoan/Workspace/HybridAGI/venv/lib/python3.10/site-packages (from alembic>=1.5.0->optuna->dspy-ai<3.0.0,>=2.4.9->hybridagi==0.2.3) (1.3.5)\n",
      "Requirement already satisfied: aiosignal>=1.1.2 in /home/yoan/Workspace/HybridAGI/venv/lib/python3.10/site-packages (from aiohttp->datasets<3.0.0,>=2.14.6->dspy-ai<3.0.0,>=2.4.9->hybridagi==0.2.3) (1.3.1)\n",
      "Requirement already satisfied: attrs>=17.3.0 in /home/yoan/Workspace/HybridAGI/venv/lib/python3.10/site-packages (from aiohttp->datasets<3.0.0,>=2.14.6->dspy-ai<3.0.0,>=2.4.9->hybridagi==0.2.3) (23.2.0)\n",
      "Requirement already satisfied: frozenlist>=1.1.1 in /home/yoan/Workspace/HybridAGI/venv/lib/python3.10/site-packages (from aiohttp->datasets<3.0.0,>=2.14.6->dspy-ai<3.0.0,>=2.4.9->hybridagi==0.2.3) (1.4.1)\n",
      "Requirement already satisfied: multidict<7.0,>=4.5 in /home/yoan/Workspace/HybridAGI/venv/lib/python3.10/site-packages (from aiohttp->datasets<3.0.0,>=2.14.6->dspy-ai<3.0.0,>=2.4.9->hybridagi==0.2.3) (6.0.5)\n",
      "Requirement already satisfied: yarl<2.0,>=1.0 in /home/yoan/Workspace/HybridAGI/venv/lib/python3.10/site-packages (from aiohttp->datasets<3.0.0,>=2.14.6->dspy-ai<3.0.0,>=2.4.9->hybridagi==0.2.3) (1.9.4)\n",
      "Requirement already satisfied: parso<0.9.0,>=0.8.3 in /home/yoan/Workspace/HybridAGI/venv/lib/python3.10/site-packages (from jedi>=0.16->ipython>=7.23.1->ipykernel<7.0.0,>=6.29.4->hybridagi==0.2.3) (0.8.4)\n",
      "Requirement already satisfied: ptyprocess>=0.5 in /home/yoan/Workspace/HybridAGI/venv/lib/python3.10/site-packages (from pexpect>4.3->ipython>=7.23.1->ipykernel<7.0.0,>=6.29.4->hybridagi==0.2.3) (0.7.0)\n",
      "Requirement already satisfied: wcwidth in /home/yoan/Workspace/HybridAGI/venv/lib/python3.10/site-packages (from prompt-toolkit<3.1.0,>=3.0.41->ipython>=7.23.1->ipykernel<7.0.0,>=6.29.4->hybridagi==0.2.3) (0.2.13)\n",
      "Requirement already satisfied: greenlet!=0.4.17 in /home/yoan/Workspace/HybridAGI/venv/lib/python3.10/site-packages (from sqlalchemy>=1.3.0->optuna->dspy-ai<3.0.0,>=2.4.9->hybridagi==0.2.3) (3.0.3)\n",
      "Requirement already satisfied: executing>=1.2.0 in /home/yoan/Workspace/HybridAGI/venv/lib/python3.10/site-packages (from stack-data->ipython>=7.23.1->ipykernel<7.0.0,>=6.29.4->hybridagi==0.2.3) (2.0.1)\n",
      "Requirement already satisfied: asttokens>=2.1.0 in /home/yoan/Workspace/HybridAGI/venv/lib/python3.10/site-packages (from stack-data->ipython>=7.23.1->ipykernel<7.0.0,>=6.29.4->hybridagi==0.2.3) (2.4.1)\n",
      "Requirement already satisfied: pure-eval in /home/yoan/Workspace/HybridAGI/venv/lib/python3.10/site-packages (from stack-data->ipython>=7.23.1->ipykernel<7.0.0,>=6.29.4->hybridagi==0.2.3) (0.2.2)\n",
      "Requirement already satisfied: mpmath<1.4.0,>=1.1.0 in /home/yoan/Workspace/HybridAGI/venv/lib/python3.10/site-packages (from sympy->torch>=1.11.0->sentence-transformers>=2.6.0->hybridagi==0.2.3) (1.3.0)\n",
      "Building wheels for collected packages: hybridagi\n",
      "  Building wheel for hybridagi (pyproject.toml) ... \u001b[?25ldone\n",
      "\u001b[?25h  Created wheel for hybridagi: filename=hybridagi-0.2.3-py3-none-any.whl size=111987 sha256=c0fa1392aa6317c3ab923ab83e593f79955a8c2ff363a6238428211c87b87a53\n",
      "  Stored in directory: /tmp/pip-ephem-wheel-cache-2hhkaiuf/wheels/89/e0/7c/3879c93b02bba3dbfbb0662e1d980e95abc68bfe985dbda52d\n",
      "Successfully built hybridagi\n",
      "Installing collected packages: hybridagi\n",
      "  Attempting uninstall: hybridagi\n",
      "    Found existing installation: hybridagi 0.2.3\n",
      "    Uninstalling hybridagi-0.2.3:\n",
      "      Successfully uninstalled hybridagi-0.2.3\n",
      "Successfully installed hybridagi-0.2.3\n",
      "\n",
      "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m A new release of pip is available: \u001b[0m\u001b[31;49m24.0\u001b[0m\u001b[39;49m -> \u001b[0m\u001b[32;49m24.1\u001b[0m\n",
      "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m To update, run: \u001b[0m\u001b[32;49mpip install --upgrade pip\u001b[0m\n"
     ]
    }
   ],
   "source": [
    "# Install HybridAGI package in the current Jupyter kernel\n",
    "import sys\n",
    "#!{sys.executable} -m pip install git+https://github.com/SynaLinks/HybridAGI.git\n",
    "!{sys.executable} -m pip install .."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Graph-based prompt programs introduction\n",
    "\n",
    "HybridAGI is an agent system that introduces a `GraphProgramInterpreter`. Similar to how the Python interpreter executes a program line by line, HybridAGI's interpreter executes a graph program node by node. To store these graph programs, we use a hybrid vector/graph database. \n",
    "\n",
    "Additionally, a dependency tree is automatically generated when loading the programs. This dependency tree enables the system to protect the main prompting mechanism and its dependencies, as well as allowing similarity-based search over the Agent's graph programs.\n",
    "\n",
    "In the following example, we have a simple graph program consisting of three nodes: `start`, `end`, and `answer`. The `start` and `end` nodes are control nodes that denote the beginning and end of the program, respectively. The `answer` node is an action node that uses the `Speak` tool to answer the objective's question.\n",
    "\n",
    "The edges between the nodes are defined using the `-[:NEXT]->` syntax, which indicates that the interpreter should move to the next node in the program. In this case, the interpreter will start at the `start` node, move to the `answer` node, and then finally move to the `end` node."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The city of Toulouse is known for its significant contributions to the spatial and robotics industry in France.\n"
     ]
    }
   ],
   "source": [
    "\n",
    "import dspy\n",
    "from hybridagi import HybridAGI\n",
    "\n",
    "# Setup the LM\n",
    "lm = dspy.OllamaLocal(model='mistral', max_tokens=1024, stop=[\"\\n\\n\\n\"])\n",
    "\n",
    "dspy.configure(lm=lm)\n",
    "\n",
    "agent = HybridAGI(\n",
    "    agent_name = \"first_steps\",\n",
    ")\n",
    "\n",
    "agent.add_programs_from_folders([\"programs/first_steps\"])\n",
    "\n",
    "# Now let's try our pipeline\n",
    "prediction = agent.execute(\"What is the city of the spacial and robotics industry in France?\")\n",
    "\n",
    "print(prediction.final_answer)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " --- Step 0 ---\n",
      "Call Program: main\n",
      "Program Purpose: What is the city of the spacial and robotics industry in France?\n",
      " --- Step 1 ---\n",
      "Action Purpose: Answer the objective's question\n",
      "Action: {\n",
      "  \"message\": \"The city of Toulouse is known for its significant contributions to the spatial and robotics industry in France.\"\n",
      "}\n",
      " --- Step 2 ---\n",
      "End Program: main\n"
     ]
    }
   ],
   "source": [
    "# We can also check the program trace\n",
    "\n",
    "print(prediction.program_trace)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[\n",
      "  {\n",
      "    \"role\": \"User\",\n",
      "    \"message\": \"What is the city of the spacial and robotics industry in France?\"\n",
      "  },\n",
      "  {\n",
      "    \"role\": \"AI\",\n",
      "    \"message\": \"The city of Toulouse is known for its significant contributions to the spatial and robotics industry in France.\"\n",
      "  }\n",
      "]\n"
     ]
    }
   ],
   "source": [
    "# Or the chat history\n",
    "\n",
    "print(prediction.chat_history)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}