examples/notebooks/Hyena-DNA-Inference.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Downstream prediction \n",
    "\n",
    "Run this notebook on google colab to use a free GPU! \n",
    "\n",
    "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/helicalAI/helical/blob/main/examples/notebooks/Hyena-DNA-Inference.ipynb)\n",
    "\n",
    "In this notebook, a HyenaDNA model is used for various classifications tasks with a given sequence of nucleotides.\n",
    "\n",
    "A HyenaDNA model (2 layers and width 256) is used to create embeddings of nucleotides.\n",
    "\n",
    "A neural network is then trained, using the embeddings as inputs, to make a prediction.\n",
    "\n",
    "This notebook can be used for any of the 18 [nucleotide transformer downstream tasks](https://huggingface.co/datasets/InstaDeepAI/nucleotide_transformer_downstream_tasks)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "# !pip install helical"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/benoit/miniconda3/envs/helical-package/lib/python3.11/site-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html\n",
      "  from .autonotebook import tqdm as notebook_tqdm\n"
     ]
    }
   ],
   "source": [
    "from sklearn import svm\n",
    "from sklearn.metrics import accuracy_score\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.preprocessing import LabelEncoder\n",
    "import numpy as np\n",
    "from datasets import DatasetDict\n",
    "import torch\n",
    "from torch import nn\n",
    "from typing import Tuple\n",
    "import logging, warnings\n",
    "from torch.nn.functional import one_hot\n",
    "from torch.utils.data import DataLoader, TensorDataset\n",
    "import torch.optim as optim\n",
    "\n",
    "logging.getLogger().setLevel(logging.ERROR)\n",
    "\n",
    "warnings.filterwarnings(\"ignore\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Use the Helical package to get the Hyena model\n",
    "We use a small HyenaDNA model with 2 layers and width 256."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:helical.utils.downloader:File: '/home/benoit/.cache/helical/models/hyena_dna/hyenadna-tiny-1k-seqlen-d256.ckpt' exists already. File is not overwritten and nothing is downloaded.\n",
      "INFO:helical.utils.downloader:File saved to: '/home/benoit/.cache/helical/models/hyena_dna/hyenadna-tiny-1k-seqlen-d256.ckpt'\n",
      "INFO:helical.models.hyena_dna.pretrained_model:Loaded pretrained weights ok!\n",
      "INFO:helical.models.hyena_dna.model:Model finished initializing.\n"
     ]
    }
   ],
   "source": [
    "from helical.models.hyena_dna.model import HyenaDNA\n",
    "from helical.models.hyena_dna.hyena_dna_config import HyenaDNAConfig  \n",
    "device = \"cuda\" if torch.cuda.is_available() else \"cpu\"\n",
    "configurer = HyenaDNAConfig(model_name=\"hyenadna-tiny-1k-seqlen-d256\", device=device, batch_size=5)\n",
    "hyena_model = HyenaDNA(configurer=configurer)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Download the dataset\n",
    "Several datasets are available from the [Nucleotide Transformer](https://arxiv.org/abs/2306.15794). Using the `get_dataset_config_names()` function, we get a list of the available the datasets for the downstream tasks."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['H4ac',\n",
       " 'H3K36me3',\n",
       " 'splice_sites_donors',\n",
       " 'splice_sites_acceptors',\n",
       " 'H3',\n",
       " 'H4',\n",
       " 'H3K4me3',\n",
       " 'splice_sites_all',\n",
       " 'H3K4me1',\n",
       " 'H3K14ac',\n",
       " 'enhancers_types',\n",
       " 'promoter_no_tata',\n",
       " 'H3K79me3',\n",
       " 'H3K4me2',\n",
       " 'promoter_tata',\n",
       " 'enhancers',\n",
       " 'H3K9ac',\n",
       " 'promoter_all']"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from datasets import get_dataset_config_names\n",
    "\n",
    "configs = get_dataset_config_names(\"InstaDeepAI/nucleotide_transformer_downstream_tasks\", trust_remote_code=True)\n",
    "configs"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can select any of the 18 downstream tasks. Let us take the `promoter_tata` as an example."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "from datasets import load_dataset\n",
    "label = \"promoter_tata\"\n",
    "dataset = load_dataset(\"InstaDeepAI/nucleotide_transformer_downstream_tasks_revised\", label, trust_remote_code=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To familiarize ourselves with the data, we can print the first seqence and see if it is a splice site acceptor or not:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Nucleotide sequence: ATGTGGAACT ...\n",
      "Label name: promoter_tata and value: 1\n",
      "Number of classes: 2\n"
     ]
    }
   ],
   "source": [
    "print(\"Nucleotide sequence:\", dataset[\"train\"][\"sequence\"][0][:10], \"...\")\n",
    "print(\"Label name:\", dataset[\"train\"].config_name, \"and value:\", dataset[\"train\"][\"label\"][0])\n",
    "num_classes = len(set(dataset[\"train\"][\"label\"]))\n",
    "print(\"Number of classes:\", num_classes)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Define a function that gets the embeddings for each nucleotide sequence in the training dataset.\n",
    "\n",
    "According to the HyenaDNA [paper](https://arxiv.org/pdf/2306.15794): \"[they] average across the tokens to obtain a single classification token\".\n",
    "\n",
    "In our code below, the Hyena model returns a (302, 256) matrix. We average column wise resulting in a vector of shape (256, ) for each observation.\n",
    "\n",
    "During the training process, we also found that it is beneficial to normalize the data row-wise."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "def get_model_inputs(dataset: DatasetDict) -> Tuple[np.ndarray, np.ndarray]:\n",
    "    \"\"\"\n",
    "    Parameters\n",
    "    ----------\n",
    "    dataset : DatasetDict\n",
    "        The dataset containing the sequences and labels.\n",
    "\n",
    "    Returns\n",
    "    -------\n",
    "    Tuple[np.ndarray, np.ndarray]\n",
    "        A tuple containing the input features and labels.\n",
    "    \"\"\"\n",
    "    processed_data = hyena_model.process_data(dataset[\"sequence\"])\n",
    "\n",
    "    embeddings = hyena_model.get_embeddings(processed_data)\n",
    "    embeddings = embeddings.mean(axis=1)\n",
    "    means = np.mean(embeddings, axis=1, keepdims=True)\n",
    "    stds = np.std(embeddings, axis=1, keepdims=True)\n",
    "\n",
    "    x = (embeddings - means) / stds\n",
    "\n",
    "    return x, dataset[\"label\"]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It may be beneficial to do this step once and save the output in a `.npy` file."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:helical.models.hyena_dna.model:Processing data\n",
      "Processing sequences: 100%|██████████| 5062/5062 [00:00<00:00, 10420.00it/s]\n",
      "INFO:helical.models.hyena_dna.model:Data processing finished.\n",
      "INFO:helical.models.hyena_dna.model:Inference started\n",
      "Getting embeddings: 100%|██████████| 1013/1013 [00:01<00:00, 510.49it/s]\n"
     ]
    }
   ],
   "source": [
    "x, y = get_model_inputs(dataset[\"train\"])\n",
    "#np.save(f\"data/train/x_{label}_norm_256\", x)\n",
    "#np.save(f\"data/train/y_{label}_norm_256\", y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Load the data and one-hot-encode the labels.\n",
    "\n",
    "We split the training set into actual training data and a test set. \n",
    "\n",
    "This is optional and the entire dataset can be used for training. We did this to avoid data leakage by not touching the test set during the training process."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[0., 1.],\n",
       "        [0., 1.],\n",
       "        [0., 1.],\n",
       "        ...,\n",
       "        [1., 0.],\n",
       "        [1., 0.],\n",
       "        [1., 0.]])"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#x = np.load(f\"data/train/x_{label}_norm_256.npy\")\n",
    "#y = np.load(f\"data/train/y_{label}_norm_256.npy\")\n",
    "\n",
    "# # One-hot encode the labels\n",
    "encoder = LabelEncoder()\n",
    "y_encoded = encoder.fit_transform(y)\n",
    "y_encoded = one_hot(torch.tensor(y_encoded),num_classes).float()\n",
    "y_encoded\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "X_train, X_test, y_train, y_test = train_test_split(x, y_encoded, test_size=0.1, random_state=42)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Sequential(\n",
      "  (0): Linear(in_features=256, out_features=256, bias=True)\n",
      "  (1): ReLU()\n",
      "  (2): Dropout(p=0.4, inplace=False)\n",
      "  (3): Linear(in_features=256, out_features=128, bias=True)\n",
      "  (4): ReLU()\n",
      "  (5): Dropout(p=0.4, inplace=False)\n",
      "  (6): Linear(in_features=128, out_features=128, bias=True)\n",
      "  (7): ReLU()\n",
      "  (8): Dropout(p=0.4, inplace=False)\n",
      "  (9): Linear(in_features=128, out_features=64, bias=True)\n",
      "  (10): ReLU()\n",
      "  (11): Dropout(p=0.4, inplace=False)\n",
      "  (12): Linear(in_features=64, out_features=2, bias=True)\n",
      "  (13): Softmax(dim=1)\n",
      ")\n"
     ]
    }
   ],
   "source": [
    "input_shape = configurer.config['d_model']\n",
    "\n",
    "# Define the model architecture\n",
    "head_model = nn.Sequential(\n",
    "    nn.Linear(input_shape, 256),\n",
    "    nn.ReLU(),\n",
    "    nn.Dropout(0.4),\n",
    "    nn.Linear(256, 128),\n",
    "    nn.ReLU(),\n",
    "    nn.Dropout(0.4),\n",
    "    nn.Linear(128, 128),\n",
    "    nn.ReLU(),\n",
    "    nn.Dropout(0.4),\n",
    "    nn.Linear(128, 64),\n",
    "    nn.ReLU(),\n",
    "    nn.Dropout(0.4),\n",
    "    nn.Linear(64, num_classes),\n",
    "    nn.Softmax(dim=1)\n",
    "    )\n",
    "\n",
    "print(head_model)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "def train_model(model: nn.Sequential,\n",
    "                X_train: torch.Tensor,  \n",
    "                y_train: torch.Tensor,  \n",
    "                X_val: torch.Tensor, \n",
    "                y_val: torch.Tensor, \n",
    "                optimizer = optim.Adam, \n",
    "                loss_fn = nn.CrossEntropyLoss(),\n",
    "                num_epochs = 25, \n",
    "                batch = 64):    \n",
    "\n",
    "    # Create DataLoader for batching\n",
    "    train_dataset = TensorDataset(X_train, y_train)\n",
    "    train_loader = DataLoader(train_dataset, batch_size=batch, shuffle=True)\n",
    "\n",
    "    # Validation dataset\n",
    "    val_dataset = TensorDataset(X_val, y_val)\n",
    "    val_loader = DataLoader(val_dataset, batch_size=batch, shuffle=False)\n",
    "\n",
    "    # Ensure model is in training mode\n",
    "    model.train()\n",
    "\n",
    "    for epoch in range(num_epochs):\n",
    "        for batch_X, batch_y in train_loader:\n",
    "            # Zero the parameter gradients\n",
    "            optimizer.zero_grad()\n",
    "            \n",
    "            # Forward pass\n",
    "            outputs = model(batch_X)\n",
    "            \n",
    "            # Compute loss\n",
    "            loss = loss_fn(outputs, batch_y)\n",
    "            \n",
    "            # Backward pass and optimize\n",
    "            loss.backward()\n",
    "            optimizer.step()\n",
    "            optimizer.zero_grad()\n",
    "        \n",
    "        # Validation phase (optional)\n",
    "        model.eval()\n",
    "        with torch.no_grad():\n",
    "            val_losses = []\n",
    "            for val_X, val_y in val_loader:\n",
    "                val_outputs = model(val_X)\n",
    "                val_loss = loss_fn(val_outputs, val_y)\n",
    "                val_losses.append(val_loss.item())\n",
    "            \n",
    "            print(f\"Epoch {epoch+1}, Validation Loss: {sum(val_losses)/len(val_losses)}\")\n",
    "        \n",
    "        # Set back to training mode for next epoch\n",
    "        model.train()\n",
    "        \n",
    "    model.eval()   \n",
    "    return model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Define and train the model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1, Validation Loss: 0.4806538000702858\n",
      "Epoch 2, Validation Loss: 0.48406418040394783\n",
      "Epoch 3, Validation Loss: 0.4835537150502205\n",
      "Epoch 4, Validation Loss: 0.48033208027482033\n",
      "Epoch 5, Validation Loss: 0.49775027111172676\n",
      "Epoch 6, Validation Loss: 0.47879376634955406\n",
      "Epoch 7, Validation Loss: 0.47708773985505104\n",
      "Epoch 8, Validation Loss: 0.4774513877928257\n",
      "Epoch 9, Validation Loss: 0.4782363697886467\n",
      "Epoch 10, Validation Loss: 0.4791817106306553\n",
      "Epoch 11, Validation Loss: 0.47751539573073387\n",
      "Epoch 12, Validation Loss: 0.4845072031021118\n",
      "Epoch 13, Validation Loss: 0.4801477864384651\n",
      "Epoch 14, Validation Loss: 0.4804844334721565\n",
      "Epoch 15, Validation Loss: 0.47806529700756073\n",
      "Epoch 16, Validation Loss: 0.4759875535964966\n",
      "Epoch 17, Validation Loss: 0.48205339536070824\n",
      "Epoch 18, Validation Loss: 0.4805366061627865\n",
      "Epoch 19, Validation Loss: 0.4773087054491043\n",
      "Epoch 20, Validation Loss: 0.47644931450486183\n",
      "Epoch 21, Validation Loss: 0.476366750895977\n",
      "Epoch 22, Validation Loss: 0.4779576063156128\n",
      "Epoch 23, Validation Loss: 0.4803308770060539\n",
      "Epoch 24, Validation Loss: 0.4766850620508194\n",
      "Epoch 25, Validation Loss: 0.4757780134677887\n"
     ]
    }
   ],
   "source": [
    "trained_model = train_model(head_model, \n",
    "                            torch.from_numpy(X_train), \n",
    "                            y_train, \n",
    "                            torch.from_numpy(X_test), \n",
    "                            y_test,\n",
    "                            optim.Adam(head_model.parameters(), lr=0.001),\n",
    "                            nn.CrossEntropyLoss())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:helical.models.hyena_dna.model:Processing data\n",
      "Processing sequences: 100%|██████████| 212/212 [00:00<00:00, 9593.29it/s]\n",
      "INFO:helical.models.hyena_dna.model:Data processing finished.\n",
      "INFO:helical.models.hyena_dna.model:Inference started\n",
      "Getting embeddings: 100%|██████████| 43/43 [00:00<00:00, 1005.46it/s]\n"
     ]
    }
   ],
   "source": [
    "X_unseen, y_unseen = get_model_inputs(dataset[\"test\"])\n",
    "#np.save(f\"data/test/x_{label}_norm_256.npy\")\n",
    "#np.save(f\"data/test/y_{label}_norm_256.npy\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Evaluate the model on the test data\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Correct predictions: 86.32%\n"
     ]
    }
   ],
   "source": [
    "#X_unseen = np.load(f\"data/test/x_{label}_norm_256.npy\")\n",
    "#y_unseen = np.load(f\"data/test/y_{label}_norm_256.npy\")\n",
    "\n",
    "predictions_nn = trained_model(torch.from_numpy(X_unseen))\n",
    "\n",
    "y_pred = np.array(torch.argmax(predictions_nn, dim=1))\n",
    "print(\"Correct predictions: {:.2f}%\".format(sum(np.equal(y_pred, y_unseen))*100/len(y_unseen)))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The [Hyena](https://arxiv.org/pdf/2306.15794) and the [Nucleotide transformer](https://www.biorxiv.org/content/10.1101/2023.01.11.523679v1.full.pdf) papers, report accuracies around 95% for this task. Our results underperform in comparison. This is probably due to the much larger models being used for the NT, while the Hyena model was re-trained from scratch for this task. In future work, we want to achieve these accuracies too with either approaches."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## OPTIONAL\n",
    "For reference, we also trained an SVM model and obtained similar results (to our small NN)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<style>#sk-container-id-1 {color: black;background-color: white;}#sk-container-id-1 pre{padding: 0;}#sk-container-id-1 div.sk-toggleable {background-color: white;}#sk-container-id-1 label.sk-toggleable__label {cursor: pointer;display: block;width: 100%;margin-bottom: 0;padding: 0.3em;box-sizing: border-box;text-align: center;}#sk-container-id-1 label.sk-toggleable__label-arrow:before {content: \"▸\";float: left;margin-right: 0.25em;color: #696969;}#sk-container-id-1 label.sk-toggleable__label-arrow:hover:before {color: black;}#sk-container-id-1 div.sk-estimator:hover label.sk-toggleable__label-arrow:before {color: black;}#sk-container-id-1 div.sk-toggleable__content {max-height: 0;max-width: 0;overflow: hidden;text-align: left;background-color: #f0f8ff;}#sk-container-id-1 div.sk-toggleable__content pre {margin: 0.2em;color: black;border-radius: 0.25em;background-color: #f0f8ff;}#sk-container-id-1 input.sk-toggleable__control:checked~div.sk-toggleable__content {max-height: 200px;max-width: 100%;overflow: auto;}#sk-container-id-1 input.sk-toggleable__control:checked~label.sk-toggleable__label-arrow:before {content: \"▾\";}#sk-container-id-1 div.sk-estimator input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-1 div.sk-label input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-1 input.sk-hidden--visually {border: 0;clip: rect(1px 1px 1px 1px);clip: rect(1px, 1px, 1px, 1px);height: 1px;margin: -1px;overflow: hidden;padding: 0;position: absolute;width: 1px;}#sk-container-id-1 div.sk-estimator {font-family: monospace;background-color: #f0f8ff;border: 1px dotted black;border-radius: 0.25em;box-sizing: border-box;margin-bottom: 0.5em;}#sk-container-id-1 div.sk-estimator:hover {background-color: #d4ebff;}#sk-container-id-1 div.sk-parallel-item::after {content: \"\";width: 100%;border-bottom: 1px solid gray;flex-grow: 1;}#sk-container-id-1 div.sk-label:hover label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-1 div.sk-serial::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: 0;}#sk-container-id-1 div.sk-serial {display: flex;flex-direction: column;align-items: center;background-color: white;padding-right: 0.2em;padding-left: 0.2em;position: relative;}#sk-container-id-1 div.sk-item {position: relative;z-index: 1;}#sk-container-id-1 div.sk-parallel {display: flex;align-items: stretch;justify-content: center;background-color: white;position: relative;}#sk-container-id-1 div.sk-item::before, #sk-container-id-1 div.sk-parallel-item::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: -1;}#sk-container-id-1 div.sk-parallel-item {display: flex;flex-direction: column;z-index: 1;position: relative;background-color: white;}#sk-container-id-1 div.sk-parallel-item:first-child::after {align-self: flex-end;width: 50%;}#sk-container-id-1 div.sk-parallel-item:last-child::after {align-self: flex-start;width: 50%;}#sk-container-id-1 div.sk-parallel-item:only-child::after {width: 0;}#sk-container-id-1 div.sk-dashed-wrapped {border: 1px dashed gray;margin: 0 0.4em 0.5em 0.4em;box-sizing: border-box;padding-bottom: 0.4em;background-color: white;}#sk-container-id-1 div.sk-label label {font-family: monospace;font-weight: bold;display: inline-block;line-height: 1.2em;}#sk-container-id-1 div.sk-label-container {text-align: center;}#sk-container-id-1 div.sk-container {/* jupyter's `normalize.less` sets `[hidden] { display: none; }` but bootstrap.min.css set `[hidden] { display: none !important; }` so we also need the `!important` here to be able to override the default hidden behavior on the sphinx rendered scikit-learn.org. See: https://github.com/scikit-learn/scikit-learn/issues/21755 */display: inline-block !important;position: relative;}#sk-container-id-1 div.sk-text-repr-fallback {display: none;}</style><div id=\"sk-container-id-1\" class=\"sk-top-container\"><div class=\"sk-text-repr-fallback\"><pre>SVC(C=1)</pre><b>In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. <br />On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.</b></div><div class=\"sk-container\" hidden><div class=\"sk-item\"><div class=\"sk-estimator sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-1\" type=\"checkbox\" checked><label for=\"sk-estimator-id-1\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">SVC</label><div class=\"sk-toggleable__content\"><pre>SVC(C=1)</pre></div></div></div></div></div>"
      ],
      "text/plain": [
       "SVC(C=1)"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Train the SVM model\n",
    "svm_model = svm.SVC(kernel='rbf', degree=3, C=1, decision_function_shape='ovr')  # One-vs-rest strategy\n",
    "X_train, X_test, y_train, y_test = train_test_split(x, y, test_size=0.1, random_state=42)\n",
    "svm_model.fit(X_train, y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Test accuracy: 87.3%\n"
     ]
    }
   ],
   "source": [
    "# Evaluate the model\n",
    "unseen_predictions_svm = svm_model.predict(X_unseen)\n",
    "\n",
    "accuracy = accuracy_score(y_unseen, unseen_predictions_svm)\n",
    "print(\"Test accuracy: {:.1f}%\".format(accuracy*100))\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAxEAAAJfCAYAAAAAQlvOAAAAP3RFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjkuMS5wb3N0MSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8kixA/AAAACXBIWXMAAA9hAAAPYQGoP6dpAAA7BElEQVR4nO3de5xXdZ0/8NcAMiLCICq3FEVN0VXJSyGbNxIVdC3S1rwVpqkp6gqZyuZdc8xL3iJpK0FTu66SYekiCmqiJUaZq64oiSYgyAIyxnCZ7+8Pf85+J9AvR2Fm0Oezx3k8mHPO95z3zOMRzpvX51JVKpVKAQAAWENtWroAAABg/aKJAAAACtFEAAAAhWgiAACAQjQRAABAIZoIAACgEE0EAABQiCYCAAAoRBMBAAAUookAAAAK0UQAAMB6oLa2Np/85CfTqVOndOvWLUOHDs3zzz/feH3BggU544wzssMOO6RDhw7p3bt3zjzzzCxatKjJc6qqqlY5fvrTnxaqRRMBAADrgSlTpmT48OF5/PHHM3HixCxfvjwHHXRQ6urqkiSvvfZaXnvttVxzzTX5y1/+knHjxuW+++7LiSeeuMqzxo4dm9mzZzceQ4cOLVRLValUKq2NbwoAAGg+8+bNS7du3TJlypTsu+++q73nF7/4RY477rjU1dWlXbt2Sd5OIu6+++7CjUM5SQQAALSQ+vr6LF68uMlRX1+/Rp99Z5hS165d3/Oezp07NzYQ7xg+fHg222yzfOpTn8ott9ySorlCu8q3rH+Wz3+ppUsAWKs69NqnpUsAWKtWLPtbS5fwrprzd8na796WSy65pMm5iy66KBdffPF7fq6hoSFnnXVWPv3pT2fnnXde7T3z58/PZZddlpNPPrnJ+UsvvTSf+cxnstFGG+W//uu/ctppp2XJkiU588wz17juD+VwJk0E8GGjiQA+bDQRb2vo9LFVkofq6upUV1e/5+dOPfXU/Pa3v82jjz6aLbbYYpXrixcvzoEHHpiuXbvmnnvuyQYbbPCuz7rwwgszduzYvPLKK2tct+FMAABQrmFlsx3V1dXp3Llzk6NSA3H66adnwoQJeeihh1bbQLz55psZPHhwOnXqlLvvvvs9G4gk6d+/f1599dU1HkaVaCIAAGC9UCqVcvrpp+fuu+/Ogw8+mD59+qxyz+LFi3PQQQelffv2ueeee7LhhhtWfO706dOzySabVGxeyn0o50QAAMD7Vmpo6QpWa/jw4bnzzjvzq1/9Kp06dcqcOXOSJDU1NenQoUNjA/HWW2/l9ttvb5yonSSbb7552rZtm1//+teZO3du9tprr2y44YaZOHFirrjiipx99tmFajEnAmA9YE4E8GHTqudEzH2+8k1ryQbdd1jje6uqqlZ7fuzYsTn++OMzefLkDBw4cLX3zJw5M1tvvXXuu+++jBo1KjNmzEipVMp2222XU089NSeddFLatFnzQUqaCID1gCYC+LBp1U3E7Geb7V0b9Nyx2d61NpkTAQAAFGJOBAAAlCm10jkRrYkkAgAAKEQSAQAA5RokEZVIIgAAgEIkEQAAUM6ciIokEQAAQCGSCAAAKNewsqUraPUkEQAAQCGaCAAAoBDDmQAAoJyJ1RVJIgAAgEIkEQAAUM5mcxVJIgAAgEIkEQAAUKZkTkRFkggAAKAQSQQAAJQzJ6IiSQQAAFCIJAIAAMqZE1GRJAIAAChEEgEAAOUaVrZ0Ba2eJAIAAChEEgEAAOXMiahIEgEAABQiiQAAgHL2iahIEgEAABQiiQAAgHLmRFQkiQAAAArRRAAAAIUYzgQAAOVMrK5IEgEAABQiiQAAgDKl0sqWLqHVk0QAAACFSCIAAKCcJV4rkkQAAACFSCIAAKCc1ZkqkkQAAACFSCIAAKCcOREVSSIAAIBCJBEAAFCuwT4RlUgiAACAQiQRAABQzpyIiiQRAABAIZIIAAAoZ5+IiiQRAABAIZIIAAAoZ05ERZIIAACgEEkEAACUMyeiIkkEAABQiCYCAAAoxHAmAAAoZzhTRZIIAACgEEkEAACUKZVWtnQJrZ4kAgAAKEQSAQAA5cyJqEgSAQAAFCKJAACAciVJRCWSCAAAoBBNBAAAlGtoaL6jgNra2nzyk59Mp06d0q1btwwdOjTPP/98k3uWLl2a4cOHZ9NNN83GG2+cI444InPnzm1yz6xZs3LooYdmo402Srdu3fKNb3wjK1asKFSLJgIAANYDU6ZMyfDhw/P4449n4sSJWb58eQ466KDU1dU13jNixIj8+te/zi9+8YtMmTIlr732Wg4//PDG6ytXrsyhhx6aZcuW5bHHHsutt96acePG5cILLyxUS1WpVCqtte+slVg+/6WWLgFgrerQa5+WLgFgrVqx7G8tXcK7+vsDY5rtXW32+Urq6+ubnKuurk51dXXFz86bNy/dunXLlClTsu+++2bRokXZfPPNc+edd+YLX/hCkuS5557LjjvumKlTp2avvfbKb3/72/zLv/xLXnvttXTv3j1JMmbMmJx77rmZN29e2rdvv2Z1F/w+AQCAtaS2tjY1NTVNjtra2jX67KJFi5IkXbt2TZJMmzYty5cvz6BBgxrv6du3b3r37p2pU6cmSaZOnZpddtmlsYFIkoMPPjiLFy/OM888s8Z1W50JAADKNeM+EaNGjcrIkSObnFuTFKKhoSFnnXVWPv3pT2fnnXdOksyZMyft27dPly5dmtzbvXv3zJkzp/Ge8gbinevvXFtTmggAAGghazp06R8NHz48f/nLX/Loo4+ug6oqM5wJAADKlRqa73gfTj/99EyYMCEPPfRQtthii8bzPXr0yLJly7Jw4cIm98+dOzc9evRovOcfV2t65+t37lkTmggAAFgPlEqlnH766bn77rvz4IMPpk+fPk2u77HHHtlggw0yadKkxnPPP/98Zs2alQEDBiRJBgwYkKeffjqvv/564z0TJ05M586ds9NOO61xLYYzAQBAuWacE1HE8OHDc+edd+ZXv/pVOnXq1DiHoaamJh06dEhNTU1OPPHEjBw5Ml27dk3nzp1zxhlnZMCAAdlrr72SJAcddFB22mmnfOlLX8pVV12VOXPm5Pzzz8/w4cMLDavSRAAAwHrg5ptvTpLsv//+Tc6PHTs2xx9/fJLkuuuuS5s2bXLEEUekvr4+Bx98cL73ve813tu2bdtMmDAhp556agYMGJCOHTtm2LBhufTSSwvVYp8IgPWAfSKAD5tWvU/Eb29stnd1GHJms71rbZJEAABAuVY6nKk1MbEaAAAoRBIBAADl3ufSqx8lkggAAKAQSQQAAJQzJ6IiSQQAAFCIJAIAAMqZE1GRJAIAAChEEgEAAOXMiahIEgEAABQiiQAAgHLmRFQkiQAAAAqRRAAAQDlzIiqSRAAAAIVIIgAAoJwkoiJJBAAAUIgkAgAAypVKLV1BqyeJAAAACpFEAABAOXMiKpJEAAAAhWgiAACAQgxnAgCAcoYzVSSJAAAACpFEAABAuZIkohJJBAAAUIgkAgAAypkTUZEkAgAAKEQSAQAA5Uqllq6g1ZNEAAAAhUgiAACgnDkRFUkiAACAQiQRAABQThJRkSQCAAAoRBIBAADl7FhdkSQCAAAoRBIBAABlSg32iahEEgEAABQiiQAAgHJWZ6pIEgEAABSiiQAAAAoxnAkAAMpZ4rUiSQQAAFCIJAIAAMpZ4rUiSQQAAFCIJAIAAMpZ4rUiSQQAAFCIJAIAAMpJIiqSRAAAAIVIIgAAoFzJ6kyVSCIAAIBCJBEAAFDOnIiKJBEAAEAhkggAAChnx+qKNBFQ5ge3/SwPTPldZr78ajasbp9P7LJTRpx6QvpstUWSZNHiNzP6hz/OY79/KrPnzssmm9TkM/sMyBknfTmdNu7Y+JydPz1klWdfdcm5OWTQ/s31rQCs1rnnnJ6hQ4ek7w7b5e9/X5qpjz+ZUf9+Rf7nf15svGebbbbKVd++IJ/+50+lurp97v+vyfm3s87P66/Pb8HKgdZEEwFlnpz+dI4+/LDsvOP2WbFyZW74/ricPOKb+dUd389GHTbM6/PfyOvzF+Ts07+abbbundlzX8+lV3838+a/keu+dX6TZ13+7yOz9157NH7daeONm/vbAVjFvvvslZtvvjVPTpuedu3a5fJLz8tv770zu/TbP2+99fdstFGH/PbeO/Pnp/87Bx58ZJLkkou/kV/dPS7/vPdhKVm1ho+CkjkRlVSVPoR/Gyyf/1JLl8CHxIL/XZh9/+XojBt9Vfb8xC6rvef+Bx/JeZdelT88MD7t2rVN8nYScUPtBTlg339uznL5EOvQa5+WLoEPqc0265o5rz2dgZ85PI88+kQOHLRvJvz69mzWbae8+eaSJEnnzp0y//X/zpBDjsmkBx9p4Yr5sFix7G8tXcK7euvqE5rtXRt945Zme9faZGI1vIcldW8lSWo6d3rXe95cUpeNO27U2EC841vXfi97H/LFHPXVf8tdE+73r3dAq1RT0znJ2/9okiTV1dUplUqpr1/WeM/SpfVpaGjIpz/9yZYoEZpfQ6n5jvVUizYR8+fPz1VXXZXPf/7zGTBgQAYMGJDPf/7zufrqqzNv3rw1ekZ9fX0WL17c5Kivr1/HlfNR0NDQkCtv+H5223WnfHybrVd7z/8uXJTvj/tJvvDZpnMgTv/ql3LNZaPyg+u/lQP3/3Quv3Z07vjlPc1QNcCaq6qqyneuuSS/+93v88wzzydJHn9iWurq3krtFd9Mhw4bZqONOuSqb1+Qdu3apUeP7i1cMXy0PfzwwznssMPSq1evVFVVZfz48U2uV1VVrfa4+uqrG+/ZeuutV7l+5ZVXFq6lxZqIP/zhD9l+++1z4403pqamJvvuu2/23Xff1NTU5MYbb0zfvn3z5JNPVnxObW1tampqmhzfvmFMM3wHfNhdfu3ozHjpr7n6kvNWe31JXV1O+8ZF2bZP75x24nFNrn3tK8dk913/KTtuv11OPO7InHDMFzL2zl82R9kAa+ymG6/IP/3TDjnmuNMaz82fvyBHHX1K/uXQQVn0vy9kwfzn0qVLTaY99ec0WDufj4hSQ0OzHUXU1dWlX79+GT169Gqvz549u8lxyy23pKqqKkcccUST+y699NIm951xxhmFf0YtNrH6jDPOyL/+679mzJgxqaqqanKtVCrla1/7Ws4444xMnTr1PZ8zatSojBw5ssm5Nm+23jF2rB++de33MuWx3+fW0VenR7fNV7leV/dWThl5QTpu1CE3XHFBNmj33v9X2uWf+mbMuJ9k2bJlad++/boqG2CN3XD95Tn0kEEZeMDh+dvfZje5NvGBh7PDjp/OpptukhUrVmbRosV5ddYf8/OZL7dQtUCSDBkyJEOGrLoC5Dt69OjR5Otf/epXGThwYLbZZpsm5zt16rTKvUW1WBLxpz/9KSNGjFilgUjejmJGjBiR6dOnV3xOdXV1Onfu3OSorq5eBxXzUVAqlfKta7+XSQ8/lltuvDJb9Fr1/2BL6upy8ohvZoMN2uWmb1+U6urKTcFzL7yYzp021kAArcIN11+eoZ8bnAMPPjJ//esr73rfG2/8bxYtWpyB+3863bptll9PmNiMVcJHw7oamj937tzce++9OfHEE1e5duWVV2bTTTfNbrvtlquvvjorVqwo/PwWSyJ69OiR3//+9+nbt+9qr//+979P9+7GXtK8Lr92dH4zcXJuvPLCdNyoQ+a/sSBJsvHGHbNhdfXbDcRZ38zf6+tzw4XfSF3dW6n7/5OvN+lSk7Zt22byo49n/oKF6bdz31S3b5/H/vBUfnjbzzLs6CPe69UAzeKmG6/I0UcNzeFHnJA331yS7t3fTlsXLXozS5cuTZIM+/KRee65GZk3/43stdceue7aS3PDDT9ospcEfKg144Tn2traXHLJJU3OXXTRRbn44os/0HNvvfXWdOrUKYcffniT82eeeWZ23333dO3aNY899lhGjRqV2bNn5zvf+U6h57fYEq+jR4/O17/+9Zxyyik54IADGhuGuXPnZtKkSfnBD36Qa665JqeddlqFJ63KEq+8X6vbJC55e8+HoYcemN8/9eeccMa5q73n/l+Oy8d6ds+jjz+Z68eMzaxXZ6eUUnp/rFe++PlD84XPDk6bNhZE4/2xxCtry7stq3nCiSNy249/niS54luj8uUvHZmuXbvkry+/mv/4jx/n+hv+oznL5COgNS/xWvetLzfbu9qd/YNVkofq6uqKI2uqqqpy9913Z+jQoau93rdv3xx44IG56aab3vM5t9xyS0455ZQsWbKk0GieFt0n4mc/+1muu+66TJs2LStXrkyStG3bNnvssUdGjhyZI4888n09VxMBfNhoIoAPm1bdRFx+XOWb1pKO59/+vj73Xk3EI488kn333TfTp09Pv3793vM5zzzzTHbeeec899xz2WGHHdb4/S26Y/UXv/jFfPGLX8zy5cszf/78JMlmm22WDTbYoCXLAgCA9daPfvSj7LHHHhUbiCSZPn162rRpk27duhV6R4s2Ee/YYIMN0rNnz5YuAwAAWu0mcEuWLMmMGTMav545c2amT5+erl27pnfv3kmSxYsX5xe/+EWuvfbaVT4/derUPPHEExk4cGA6deqUqVOnZsSIETnuuOOyySabFKqlVTQRAADAe3vyySczcODAxq/f2eZg2LBhGTduXJLkpz/9aUqlUo4++uhVPl9dXZ2f/vSnufjii1NfX58+ffpkxIgRq2yXsCZadE7EumJOBPBhY04E8GHTqudEXLzqL+DrSseLf9Js71qbLBUDAAAUYjgTAACUa6VzIloTSQQAAFCIJAIAAMqVGlq6glZPEgEAABQiiQAAgHLmRFQkiQAAAAqRRAAAQJlSgzkRlUgiAACAQiQRAABQzpyIiiQRAABAIZoIAACgEMOZAACgnOFMFUkiAACAQiQRAABQrmSJ10okEQAAQCGSCAAAKGdOREWSCAAAoBBJBAAAlClJIiqSRAAAAIVIIgAAoJwkoiJJBAAAUIgkAgAAyjXYJ6ISSQQAAFCIJAIAAMqZE1GRJAIAAChEEgEAAOUkERVJIgAAgEIkEQAAUKZUkkRUIokAAAAKkUQAAEA5cyIqkkQAAACFaCIAAIBCDGcCAIByhjNVJIkAAAAKkUQAAECZkiSiIkkEAABQiCQCAADKSSIqkkQAAACFSCIAAKBcQ0sX0PpJIgAAgEIkEQAAUMbqTJVJIgAAgEIkEQAAUE4SUZEkAgAAKEQSAQAA5azOVJEkAgAAKEQSAQAAZazOVJkkAgAAKEQSAQAA5cyJqEgSAQAAFKKJAAAACjGcCQAAyphYXZkkAgAAKEQSAQAA5UysrkgSAQAAFCKJAACAMiVJREWSCAAAoBBNBAAAlGtoxqOAhx9+OIcddlh69eqVqqqqjB8/vsn1448/PlVVVU2OwYMHN7lnwYIFOfbYY9O5c+d06dIlJ554YpYsWVKskGgiAABgvVBXV5d+/fpl9OjR73rP4MGDM3v27MbjJz/5SZPrxx57bJ555plMnDgxEyZMyMMPP5yTTz65cC3mRAAAQJnWOidiyJAhGTJkyHveU11dnR49eqz22rPPPpv77rsvf/jDH7LnnnsmSW666aYccsghueaaa9KrV681rkUSAQAALaS+vj6LFy9uctTX17/v502ePDndunXLDjvskFNPPTVvvPFG47WpU6emS5cujQ1EkgwaNCht2rTJE088Ueg9mggAACjXjHMiamtrU1NT0+Sora19X2UPHjw4t912WyZNmpRvf/vbmTJlSoYMGZKVK1cmSebMmZNu3bo1+Uy7du3StWvXzJkzp9C7DGcCAIAWMmrUqIwcObLJuerq6vf1rKOOOqrxz7vsskt23XXXbLvttpk8eXIOOOCAD1TnP9JEAABAmeacE1FdXf2+m4ZKttlmm2y22WaZMWNGDjjggPTo0SOvv/56k3tWrFiRBQsWvOs8indjOBMAAHwIvfrqq3njjTfSs2fPJMmAAQOycOHCTJs2rfGeBx98MA0NDenfv3+hZ0siAACgTGtdnWnJkiWZMWNG49czZ87M9OnT07Vr13Tt2jWXXHJJjjjiiPTo0SMvvvhizjnnnGy33XY5+OCDkyQ77rhjBg8enJNOOiljxozJ8uXLc/rpp+eoo44qtDJTIokAAID1wpNPPpnddtstu+22W5Jk5MiR2W233XLhhRembdu2+fOf/5zPfvaz2X777XPiiSdmjz32yCOPPNJkuNQdd9yRvn375oADDsghhxySvffeO//xH/9RuJaqUqlUWmvfWSuxfP5LLV0CwFrVodc+LV0CwFq1YtnfWrqEdzV34H7N9q7uD01ptnetTZIIAACgEHMiAACgXKmqpSto9SQRAABAIZoIAACgEMOZAACgTGtd4rU1kUQAAACFSCIAAKBMqcHE6kokEQAAQCGSCAAAKGNORGWSCAAAoBBJBAAAlCnZbK4iSQQAAFCIJAIAAMqYE1GZJAIAAChEEgEAAGXsE1GZJAIAAChEEgEAAGVKpZauoPWTRAAAAIVIIgAAoIw5EZVJIgAAgEIkEQAAUEYSUZkkAgAAKEQTAQAAFGI4EwAAlLHEa2WSCAAAoBBJBAAAlDGxujJJBAAAUIgkAgAAypRKkohKJBEAAEAhkggAAChTamjpClo/SQQAAFCIJAIAAMo0mBNRkSQCAAAoRBIBAABlrM5UmSQCAAAoRBIBAABl7FhdmSQCAAAoRBIBAABlSqWWrqD1k0QAAACFSCIAAKCMORGVSSIAAIBC3ncSsWzZsrz++utpaGhocr53794fuCgAAGgpdqyurHAT8cILL+SEE07IY4891uR8qVRKVVVVVq5cudaKAwAAWp/CTcTxxx+fdu3aZcKECenZs2eqqnRqAADwUVK4iZg+fXqmTZuWvn37rot6AACgRZUMZ6qo8MTqnXbaKfPnz18XtQAAAOuBNWoiFi9e3Hh8+9vfzjnnnJPJkyfnjTfeaHJt8eLF67peAABYp0ql5jvWV2s0nKlLly5N5j6USqUccMABTe4xsRoAAD4a1qiJeOihh9Z1HQAA0CpY4rWyNWoi9ttvv8Y/z5o1K1tuueUqqzKVSqW88sora7c6AACg1Sk8sbpPnz6ZN2/eKucXLFiQPn36rJWiAACgpZRKVc12rK8KNxHvzH34R0uWLMmGG264VooCAABarzXeJ2LkyJFJkqqqqlxwwQXZaKONGq+tXLkyTzzxRD7xiU+s9QIBAKA5rc+rJjWXNW4i/vjHPyZ5O4l4+umn0759+8Zr7du3T79+/XL22Wev/QoBAIBWZY2biHdWaPrKV76SG264IZ07d15nRQEAQEuxOlNla9xEvGPs2LHrog4AAGA9UbiJ+MxnPvOe1x988MH3Xcza0n+XL7d0CQBr1Zu/OrelSwD4yFifV01qLoWbiH79+jX5evny5Zk+fXr+8pe/ZNiwYWutMAAAoHUq3ERcd911qz1/8cUXZ8mSJR+4IAAAaEmtdU7Eww8/nKuvvjrTpk3L7Nmzc/fdd2fo0KFJ3v6H/fPPPz+/+c1v8tJLL6WmpiaDBg3KlVdemV69ejU+Y+utt87LL7/c5Lm1tbU577zzCtVSeJ+Id3PcccfllltuWVuPAwAAytTV1aVfv34ZPXr0KtfeeuutPPXUU7ngggvy1FNP5a677srzzz+fz372s6vce+mll2b27NmNxxlnnFG4lsJJxLuZOnWqzeYAAFjvtdZtIoYMGZIhQ4as9lpNTU0mTpzY5Nx3v/vdfOpTn8qsWbPSu3fvxvOdOnVKjx49PlAthZuIww8/vMnXpVIps2fPzpNPPpkLLrjgAxUDAAAfJfX19amvr29yrrq6OtXV1R/42YsWLUpVVVW6dOnS5PyVV16Zyy67LL17984xxxyTESNGpF27Ym1B4eFMNTU1TY6uXbtm//33z29+85tcdNFFRR8HAAAfWbW1tav8fl1bW/uBn7t06dKce+65Ofroo5vs73bmmWfmpz/9aR566KGccsopueKKK3LOOecUfn6hlmPlypX5yle+kl122SWbbLJJ4ZcBAEBr15wTq0eNGpWRI0c2OfdBU4jly5fnyCOPTKlUys0339zkWvm7dt1117Rv3z6nnHJKamtrC723UBLRtm3bHHTQQVm4cGGRjwEAAKtRXV2dzp07Nzk+SBPxTgPx8ssvZ+LEiU1SiNXp379/VqxYkb/+9a+F3lN4ONPOO++cl156qejHAABgvVAqVTXbsTa900C88MILeeCBB7LppptW/Mz06dPTpk2bdOvWrdC7Ck+svvzyy3P22Wfnsssuyx577JGOHTs2uV6p2wEAAIpbsmRJZsyY0fj1zJkzM3369HTt2jU9e/bMF77whTz11FOZMGFCVq5cmTlz5iRJunbtmvbt22fq1Kl54oknMnDgwHTq1ClTp07NiBEjctxxxxWeqlBVKpXWaBWrSy+9NF//+tfTqVOn//tw1f91T6VSKVVVVVm5cmWhAtaF3Xvu3dIlAKxVv7vlyJYuAWCt6jDkzJYu4V090uMLzfaufeb8co3vnTx5cgYOHLjK+WHDhuXiiy9Onz59Vvu5hx56KPvvv3+eeuqpnHbaaXnuuedSX1+fPn365Etf+lJGjhxZeAjVGjcRbdu2zezZs/Pss8++53377bdfoQLWBU0E8GGjiQA+bDQRbyvSRLQmazyc6Z1eozU0CQAAsK6U0nyrM62vCk2sLh++BAAAfDQVmli9/fbbV2wkFixY8IEKAgCAltSwRoP9P9oKNRGXXHJJampq1lUtAADAeqBQE3HUUUcVXkMWAADWJw3mRFS0xnMizIcAAACS97E6EwAAfJhZnamyNW4iGhoa1mUdAADAeqLQnAgAAPiw80/nlRXaJwIAAEASAQAAZcyJqEwSAQAAFCKJAACAMuZEVCaJAAAACtFEAAAAhRjOBAAAZQxnqkwSAQAAFCKJAACAMpZ4rUwSAQAAFCKJAACAMg2CiIokEQAAQCGSCAAAKNNgTkRFkggAAKAQSQQAAJQptXQB6wFJBAAAUIgkAgAAytixujJJBAAAUIgkAgAAyjRUWZ2pEkkEAABQiCQCAADKWJ2pMkkEAABQiCQCAADKWJ2pMkkEAABQiCYCAAAoxHAmAAAo02CF14okEQAAQCGSCAAAKNMQUUQlkggAAKAQSQQAAJSx2VxlkggAAKAQSQQAAJSxOlNlkggAAKAQSQQAAJRpaOkC1gOSCAAAoBBJBAAAlLE6U2WSCAAAoBBJBAAAlLE6U2WSCAAAoBBJBAAAlLE6U2WSCAAAoBBJBAAAlJFEVCaJAAAACpFEAABAmZLVmSqSRAAAAIVoIgAAgEIMZwIAgDImVlcmiQAAAAqRRAAAQBlJRGWSCAAAWA88/PDDOeyww9KrV69UVVVl/PjxTa6XSqVceOGF6dmzZzp06JBBgwblhRdeaHLPggULcuyxx6Zz587p0qVLTjzxxCxZsqRwLZoIAAAoU2rGo4i6urr069cvo0ePXu31q666KjfeeGPGjBmTJ554Ih07dszBBx+cpUuXNt5z7LHH5plnnsnEiRMzYcKEPPzwwzn55JMLVmI4EwAAtJj6+vrU19c3OVddXZ3q6upV7h0yZEiGDBmy2ueUSqVcf/31Of/88/O5z30uSXLbbbele/fuGT9+fI466qg8++yzue+++/KHP/whe+65Z5LkpptuyiGHHJJrrrkmvXr1WuO6JREAAFCmoar5jtra2tTU1DQ5amtrC9c8c+bMzJkzJ4MGDWo8V1NTk/79+2fq1KlJkqlTp6ZLly6NDUSSDBo0KG3atMkTTzxR6H2SCAAAaCGjRo3KyJEjm5xbXQpRyZw5c5Ik3bt3b3K+e/fujdfmzJmTbt26Nbnerl27dO3atfGeNaWJAACAMs25OtO7DV1q7QxnAgCA9VyPHj2SJHPnzm1yfu7cuY3XevTokddff73J9RUrVmTBggWN96wpTQQAAJRpaMZjbenTp0969OiRSZMmNZ5bvHhxnnjiiQwYMCBJMmDAgCxcuDDTpk1rvOfBBx9MQ0ND+vfvX+h9hjMBAMB6YMmSJZkxY0bj1zNnzsz06dPTtWvX9O7dO2eddVYuv/zyfPzjH0+fPn1ywQUXpFevXhk6dGiSZMcdd8zgwYNz0kknZcyYMVm+fHlOP/30HHXUUYVWZko0EQAA0ETR/Ruay5NPPpmBAwc2fv3OhOxhw4Zl3LhxOeecc1JXV5eTTz45CxcuzN5775377rsvG264YeNn7rjjjpx++uk54IAD0qZNmxxxxBG58cYbC9dSVSqVWuvP6X3bvefeLV0CwFr1u1uObOkSANaqDkPObOkS3tU1vY9rtnedPev2ZnvX2iSJAACAMg1VLV1B62diNQAAUIgkAgAAyjTnPhHrK0kEAABQiCYCAAAoxHAmAAAo86FbunQdkEQAAACFSCIAAKBMgyyiIkkEAABQiCQCAADKWOK1MkkEAABQiCQCAADKmBFRmSQCAAAoRBIBAABlzImoTBIBAAAUIokAAIAyDVUtXUHrJ4kAAAAKkUQAAEAZO1ZXJokAAAAKkUQAAEAZOURlkggAAKAQSQQAAJSxT0RlkggAAKAQSQQAAJSxOlNlkggAAKAQTQQAAFCI4UwAAFDGYKbKJBEAAEAhkggAAChjidfKJBEAAEAhkggAAChjidfKJBEAAEAhkggAACgjh6hMEgEAABQiiQAAgDJWZ6pMEgEAABQiiQAAgDIlsyIqkkQAAACFSCIAAKCMORGVSSIAAIBCJBEAAFDGjtWVSSIAAIBCJBEAAFBGDlGZJAIAAChEEwEAABRiOBMAAJQxsboySQQAAFCIJAIAAMrYbK4yTQRUcMrXT8gpZ5/Q5NzMGS/niH2OTZL8x3/elD3/ebcm13952/hcce41zVYjwHuZ9uJrufXBP+bZV17PvMVv5TsnDMlndt2m8fobb76V6++Zmsefn5U3/74su2/bK+cesU+22rxL4z3zF9flunsey+PPv5K6+uXZuluXfPXAPTOo37Yt8B0BLU0TAWtgxnMv5dQjz2r8euXKlU2u33X7Pbn5qh82fr3070ubqzSAiv5evzzb99o0Q/vvmJG3/LbJtVKplBE//E3atW2T6756SDaubp8fT56er33vV7nrvGPSoXqDJMn5d0zKm3+vz/VfPTSbdNwwv33qhZwz7v7c+fV/Td8tNm+JbwvWmZI5ERWZEwFrYOWKlXlj3oLGY+GCRU2uL/370ibX65a81UKVAqxq7522yumH7tUkfXjHrHmL8ueX5+bf/3W/7Ny7e7buvkm++a/7Z+nylfntUy803venmbNz9D67ZpetumeLzWpy0kF7plOH9vnvV+Y157cCtBKaCFgDvbfZIvf/cXzuefznuXz0henxse5Nrg85/MBMemZCfv7QbTn930/Jhh2qW6hSgGKWrXg7Wa3e4P8GJ7RpU5X27drmjy/NbjzXr0/P3P/HF7KobmkaGkq576kXUr9iZfbcrlez1wzrWkMzHuurVj2c6ZVXXslFF12UW2655V3vqa+vT319fZNzDaWGtKnSH7F2PP3H/85F/3ZFXn5xVjbrvmlOHvmV/Gj86Pzr/l/KW3V/z313T8zsV+dk3pz5+fhO2+bMb56arbftnbNP/GZLlw5Q0dbdu6TnJhvnxglTc8GR+6dD+w1y++Q/Ze7CJZm/uK7xvquGHZxzb70/+33zR2nXpk02bN8u3zlhSHqXzZsAPjpadROxYMGC3Hrrre/ZRNTW1uaSSy5pcq5Hxy3Ts1PvdV0eHxGPPfh4459fePbFPP3Uf+feP/wyB372M/nVT+7NXbff03h9xnMvZf7cN/L9X96YLbbqlVdffq0lSgZYYxu0bZtrTxiSi3/yYPb99x+lbZuq9N9+y3x6x94pHxb+vd8+kTf/Xp/vn/bZdOnYIQ89/VLOGXd/xp55eD7ea9OW+wZgHTAnorIWbSLuueee97z+0ksvVXzGqFGjMnLkyCbn9t1+8AeqC97LksVLMuulV7Jlny1We/3pp/47SbJlny00EcB6Yactu+Xn5xyVN/9en+UrG9J14w457ju/yE69uyVJXpm/KD995On88tyjsl3PtxuGHT62Wf740uz87NGnc/6R+7dg9UBLaNEmYujQoamqqkqp9O7dXlVV1Xs+o7q6OtXVTcefG8rEutRhow7ZYquP5d5f3r/a6zvs/PEkyfy5bzRnWQAfWKf/P5/r5XkL89+vzMtph/RPkixdtiJJ0uYf/pvcpqoqDe/x33BYX63PcxWaS4v+tt2zZ8/cddddaWhoWO3x1FNPtWR5kCQ568Lh2X3AJ9Jzix7Zdc+dc+0tV6ShYWXuG/9AttiqV746Ylh23HWH9NyiR/Y96NO59MbzM23qH/PCsy+2dOkASZK36pfluVfn5blX315J6W8LFue5V+dl9v++mST5r+kz8ocX/pZX5y/KQ0+/lK99754M3KVP/rnv20ODt+7eJVtuVpPLfz45T788N6/MX5TbHvpjHv+fVzJwlz4t9n0BLadFk4g99tgj06ZNy+c+97nVXq+UUkBz6N5z89R+7+LUbNI5//vGwkz//Z8z7NBTsvCNhamubp/+++yZY756ZDpstGHmvvZ6Hrx3cn54/a0tXTZAo2dmzctJo8c3fn3t+N8lSQ77ZN9cduwBmb+oLteO/13eePOtbN55o/zLJ/vm5IP2bLx/g7Zt891T/iU3/npq/u0H9+atZcvTe7OaXHbMoOyz09bN/N3AutdaE7att946L7/88irnTzvttIwePTr7779/pkyZ0uTaKaeckjFjxqz1WqpKLfhb+iOPPJK6uroMHrz6OQx1dXV58skns99++xV67u49914b5QG0Gr+75ciWLgFgreow5MyWLuFdfWmrw5vtXT9++a41vnfevHlNNrz9y1/+kgMPPDAPPfRQ9t9//+y///7Zfvvtc+mllzbes9FGG6Vz585rteakhZOIffbZ5z2vd+zYsXADAQAAH0TrzCGSzTdvujv8lVdemW233bbJ78sbbbRRevTosc5rMQMZAABaSH19fRYvXtzk+Mc90FZn2bJluf3223PCCSc0WYjojjvuyGabbZadd945o0aNyltvvbVO6tZEAABAmYaUmu2ora1NTU1Nk6O2trZijePHj8/ChQtz/PHHN5475phjcvvtt+ehhx7KqFGj8uMf/zjHHXfcOvkZteiciHXFnAjgw8acCODDpjXPiThmq88327vG/s9PV0keVreFwT86+OCD0759+/z6179+13sefPDBHHDAAZkxY0a23XbbtVLvO1r1jtUAANDcmnPH6jVpGP7Ryy+/nAceeCB33fXek7L79397r5d10UQYzgQAAOuRsWPHplu3bjn00EPf877p06cneXtvtrVNEgEAAOuJhoaGjB07NsOGDUu7dv/3q/yLL76YO++8M4ccckg23XTT/PnPf86IESOy7777Ztddd13rdWgiAACgTENLF/AeHnjggcyaNSsnnHBCk/Pt27fPAw88kOuvvz51dXXZcsstc8QRR+T8889fJ3VoIgAAYD1x0EEHZXXrIm255Zar7Fa9LmkiAACgTEOr3W6u9TCxGgAAKEQSAQAAZZpzidf1lSQCAAAoRBIBAABlWvPqTK2FJAIAAChEEgEAAGVWt4QqTUkiAACAQiQRAABQxj4RlUkiAACAQiQRAABQxupMlUkiAACAQiQRAABQxo7VlUkiAACAQiQRAABQxupMlUkiAACAQjQRAABAIYYzAQBAmVLJcKZKJBEAAEAhkggAAChjs7nKJBEAAEAhkggAAChjs7nKJBEAAEAhkggAAChjs7nKJBEAAEAhkggAAChjn4jKJBEAAEAhkggAAChjTkRlkggAAKAQSQQAAJSxT0RlkggAAKAQSQQAAJRpsDpTRZIIAACgEEkEAACUkUNUJokAAAAK0UQAAACFGM4EAABlbDZXmSQCAAAoRBIBAABlJBGVSSIAAIBCJBEAAFCmZLO5iiQRAABAIZIIAAAoY05EZZIIAACgEEkEAACUKUkiKpJEAAAAhUgiAACgjNWZKpNEAAAAhUgiAACgjNWZKpNEAAAAhUgiAACgjDkRlUkiAACAQiQRAABQxpyIyiQRAABAIZIIAAAoY8fqyiQRAABAIZoIAACgEMOZAACgTIMlXiuSRAAAwHrg4osvTlVVVZOjb9++jdeXLl2a4cOHZ9NNN83GG2+cI444InPnzl0ntWgiAACgTKkZ/1fUP/3TP2X27NmNx6OPPtp4bcSIEfn1r3+dX/ziF5kyZUpee+21HH744WvzR9PIcCYAAFhPtGvXLj169Fjl/KJFi/KjH/0od955Zz7zmc8kScaOHZsdd9wxjz/+ePbaa6+1WockAgAAyjSUSs121NfXZ/HixU2O+vr6d63thRdeSK9evbLNNtvk2GOPzaxZs5Ik06ZNy/LlyzNo0KDGe/v27ZvevXtn6tSpa/1npIkAAIAWUltbm5qamiZHbW3tau/t379/xo0bl/vuuy8333xzZs6cmX322Sdvvvlm5syZk/bt26dLly5NPtO9e/fMmTNnrddtOBMAAJRpzs3mRo0alZEjRzY5V11dvdp7hwwZ0vjnXXfdNf37989WW22Vn//85+nQocM6rfMfSSIAAKCFVFdXp3Pnzk2Od2si/lGXLl2y/fbbZ8aMGenRo0eWLVuWhQsXNrln7ty5q51D8UFpIgAAoExzzon4IJYsWZIXX3wxPXv2zB577JENNtggkyZNarz+/PPPZ9asWRkwYMAH/ZGswnAmAABYD5x99tk57LDDstVWW+W1117LRRddlLZt2+boo49OTU1NTjzxxIwcOTJdu3ZN586dc8YZZ2TAgAFrfWWmRBMBAABNNOeciCJeffXVHH300XnjjTey+eabZ++9987jjz+ezTffPEly3XXXpU2bNjniiCNSX1+fgw8+ON/73vfWSS1VpdKHb1/v3Xvu3dIlAKxVv7vlyJYuAWCt6jDkzJYu4V19fPM9mu1dL8yb1mzvWpskEQAAUOaDzlX4KDCxGgAAKEQSAQAAZVrrnIjWRBIBAAAUIokAAIAypVJDS5fQ6kkiAACAQjQRAABAIYYzAQBAmQYTqyuSRAAAAIVIIgAAoEzJZnMVSSIAAIBCJBEAAFDGnIjKJBEAAEAhkggAAChjTkRlkggAAKAQSQQAAJRpkERUJIkAAAAKkUQAAECZktWZKpJEAAAAhUgiAACgjNWZKpNEAAAAhUgiAACgjB2rK5NEAAAAhUgiAACgjDkRlUkiAACAQiQRAABQxo7VlUkiAACAQjQRAABAIYYzAQBAGROrK5NEAAAAhUgiAACgjM3mKpNEAAAAhUgiAACgjDkRlUkiAACAQiQRAABQxmZzlUkiAACAQiQRAABQpmR1pookEQAAQCGSCAAAKGNORGWSCAAAoBBJBAAAlLFPRGWSCAAAoBBJBAAAlLE6U2WSCAAAoBBJBAAAlDEnojJJBAAAUIgmAgAAKMRwJgAAKGM4U2WSCAAAoBBJBAAAlJFDVCaJAAAACqkqGfQF70t9fX1qa2szatSoVFdXt3Q5AB+Yv9eANaWJgPdp8eLFqampyaJFi9K5c+eWLgfgA/P3GrCmDGcCAAAK0UQAAACFaCIAAIBCNBHwPlVXV+eiiy4y+RD40PD3GrCmTKwGAAAKkUQAAACFaCIAAIBCNBEAAEAhmggAAKAQTQS8T6NHj87WW2+dDTfcMP3798/vf//7li4J4H15+OGHc9hhh6VXr16pqqrK+PHjW7okoJXTRMD78LOf/SwjR47MRRddlKeeeir9+vXLwQcfnNdff72lSwMorK6uLv369cvo0aNbuhRgPWGJV3gf+vfvn09+8pP57ne/myRpaGjIlltumTPOOCPnnXdeC1cH8P5VVVXl7rvvztChQ1u6FKAVk0RAQcuWLcu0adMyaNCgxnNt2rTJoEGDMnXq1BasDACgeWgioKD58+dn5cqV6d69e5Pz3bt3z5w5c1qoKgCA5qOJAAAACtFEQEGbbbZZ2rZtm7lz5zY5P3fu3PTo0aOFqgIAaD6aCCioffv22WOPPTJp0qTGcw0NDZk0aVIGDBjQgpUBADSPdi1dAKyPRo4cmWHDhmXPPffMpz71qVx//fWpq6vLV77ylZYuDaCwJUuWZMaMGY1fz5w5M9OnT0/Xrl3Tu3fvFqwMaK0s8Qrv03e/+91cffXVmTNnTj7xiU/kxhtvTP/+/Vu6LIDCJk+enIEDB65yftiwYRk3blzzFwS0epoIAACgEHMiAACAQjQRAABAIZoIAACgEE0EAABQiCYCAAAoRBMBAAAUookAAAAK0UQAAACFaCIAWpnjjz8+Q4cObfx6//33z1lnndXsdUyePDlVVVVZuHBhs78bgNZNEwGwho4//vhUVVWlqqoq7du3z3bbbZdLL700K1asWKfvveuuu3LZZZet0b1+8QegObRr6QIA1ieDBw/O2LFjU19fn9/85jcZPnx4Nthgg4waNarJfcuWLUv79u3Xyju7du26Vp4DAGuLJAKggOrq6vTo0SNbbbVVTj311AwaNCj33HNP4xCkb33rW+nVq1d22GGHJMkrr7ySI488Ml26dEnXrl3zuc99Ln/9618bn7dy5cqMHDkyXbp0yaabbppzzjknpVKpyTv/cThTfX19zj333Gy55Zaprq7Odtttlx/96Ef561//moEDByZJNtlkk1RVVeX4449PkjQ0NKS2tjZ9+vRJhw4d0q9fv/zyl79s8p7f/OY32X777dOhQ4cMHDiwSZ0AUE4TAfABdOjQIcuWLUuSTJo0Kc8//3wmTpyYCRMmZPny5Tn44IPTqVOnPPLII/nd736XjTfeOIMHD278zLXXXptx48bllltuyaOPPpoFCxbk7rvvfs93fvnLX85PfvKT3HjjjXn22Wfz/e9/PxtvvHG23HLL/Od//meS5Pnnn8/s2bNzww03JElqa2tz2223ZcyYMXnmmWcyYsSIHHfccZkyZUqSt5udww8/PIcddlimT5+er371qznvvPPW1Y8NgPWc4UwA70OpVMqkSZNy//3354wzzsi8efPSsWPH/PCHP2wcxnT77benoaEhP/zhD1NVVZUkGTt2bLp06ZLJkyfnoIMOyvXXX59Ro0bl8MMPT5KMGTMm999//7u+93/+53/y85//PBMnTsygQYOSJNtss03j9XeGPnXr1i1dunRJ8nZyccUVV+SBBx7IgAEDGj/z6KOP5vvf/37222+/3Hzzzdl2221z7bXXJkl22GGHPP300/n2t7+9Fn9qAHxYaCIACpgwYUI23njjLF++PA0NDTnmmGNy8cUXZ/jw4dlll12azIP405/+lBkzZqRTp05NnrF06dK8+OKLWbRoUWbPnp3+/fs3XmvXrl323HPPVYY0vWP69Olp27Zt9ttvvzWuecaMGXnrrbdy4IEHNjm/bNmy7LbbbkmSZ599tkkdSRobDgD4R5oIgAIGDhyYm2++Oe3bt0+vXr3Srt3//TXasWPHJvcuWbIke+yxR+64445VnrP55pu/r/d36NCh8GeWLFmSJLn33nvzsY99rMm16urq91UHAB9tmgiAAjp27Jjttttuje7dfffd87Of/SzdunVL586dV3tPz54988QTT2TfffdNkqxYsSLTpk3L7rvvvtr7d9lllzQ0NGTKlCmNw5nKvZOErFy5svHcTjvtlOrq6syaNetdE4wdd9wx99xzT5Nzjz/+eOVvEoCPJBOrAdaRY489Nptttlk+97nP5ZFHHsnMmTMzefLknHnmmXn11VeTJP/2b/+WK6+8MuPHj89zzz2X00477T33eNh6660zbNiwnHDCCRk/fnzjM3/+858nSbbaaqtUVVVlwoQJmTdvXpYsWZJOnTrl7LPPzogRI3LrrbfmxRdfzFNPPZWbbropt956a5Lka1/7Wl544YV84xvfyPPPP58777wz48aNW9c/IgDWU5oIgHVko402ysMPP5zevXvn8MMPz4477pgTTzwxS5cubUwmvv71r+dLX/pShg0blgEDBqRTp075/Oc//57Pvfnmm/OFL3whp512Wvr27ZuTTjopdXV1SZKPfexjueSSS3Leeeele/fuOf3005Mkl112WS644ILU1tZmxx13zODBg3PvvfemT58+SZLevXvnP//zPzN+/Pj069cvY8aMyRVXXLEOfzoArM+qSu82ew8AAGA1JBEAAEAhmggAAKAQTQQAAFCIJgIAAChEEwEAABSiiQAAAArRRAAAAIVoIgAAgEI0EQAAQCGaCAAAoBBNBAAAUMj/A4sRVGslmTu1AAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 1000x700 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import seaborn as sns\n",
    "import matplotlib.pyplot as plt\n",
    "from sklearn.metrics import confusion_matrix\n",
    "\n",
    "# NN\n",
    "# y_true = np.argmax(y_test, axis=1)\n",
    "# y_pred_classes = np.argmax(predictions_nn, axis=1)\n",
    "\n",
    "# SVM\n",
    "y_true = y_test\n",
    "y_pred_classes = svm_model.predict(X_test)\n",
    "\n",
    "# Compute the confusion matrix\n",
    "cm = confusion_matrix(y_true, y_pred_classes)\n",
    "\n",
    "# Create a heatmap\n",
    "plt.figure(figsize=(10,7))\n",
    "sns.heatmap(cm, annot=True, fmt='d')\n",
    "plt.xlabel('Predicted')\n",
    "plt.ylabel('Truth')\n",
    "plt.show()"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "helical-package",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}