diff --git a/frontend/index.html b/frontend/index.html
index a088c7a..3fe7076 100644
--- a/frontend/index.html
+++ b/frontend/index.html
@@ -2,6 +2,15 @@
+
+
+
diff --git a/frontend/src/components/about.vue b/frontend/src/components/about.vue
index cd4d6db..b386cf2 100644
--- a/frontend/src/components/about.vue
+++ b/frontend/src/components/about.vue
@@ -14,12 +14,10 @@
The Moore Institute for Plastic Pollution Research
- and Walter Yu from CALTRANS. There
- have also been
- many contributors
- to the code base.
+ and Walter Yu from CALTRANS.
+
+ Steven Hollingsworth
+ is the lead developer and contributor to the code base.
To get started, visit the Upload Tab or
@@ -68,6 +66,8 @@
>open a Github Issue
in our repository.
+ If you would like to provide more general feedback, please fill out
+ our feedback form here .
To get started, visit the Upload Tab or
@@ -68,6 +66,8 @@
open a Github Issue
in our repository.
+ If you would like to provide more general feedback, please fill out
+ our feedback form here .
diff --git a/notebooks/Evaluation_Accuracy.ipynb b/notebooks/Evaluation_Accuracy.ipynb
new file mode 100644
index 0000000..4d78b3b
--- /dev/null
+++ b/notebooks/Evaluation_Accuracy.ipynb
@@ -0,0 +1,1354 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "source": [
+ "# This notebook: Build evaluation method\n",
+ "* Aim at >.90 accuracy\n",
+ "\n",
+ "Currently it is tested with yolov5 prediction results. But it is compatible for all prediction outputs as long as they are in the form of `.pandas().xywh`. (see section `1.3` for examples)\n",
+ "\n",
+ "Using `Google Colab` to view this notebook is highly recommended."
+ ],
+ "metadata": {
+ "id": "qN8V989HzHDT"
+ }
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "### Questions:\n",
+ "* Want the **big TACO**? i.e. the **unofficial TACO** that contains 5,000+ images. It's much larger a dataset, but the label quality of the big TACO might be poor. \n",
+ "\n",
+ "* **Reduce target classes**? There are 60 categories and 28 super-categories. Currently we predict 60 classes, which might be too many considering that we only have less than 1500 training images. Should we use the 28 super-categories as classes to be predicted? Or one step further, 5~10 classes of plastic, metal, glass, etc. based on the material of trash objects.\n",
+ "\n",
+ "* Better **Train/Test split**? Currently I do a non-stratified 1300/100/100 split for train/val/test. This is not the best choice. Classes(categories)'s distribution in training and testing set will be different which might be a problem. Inputs are greatly welcomed!"
+ ],
+ "metadata": {
+ "id": "mS4ry1DRFSes"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "mount_drive = True # set to true only if you have weights and images in your own google drive\n",
+ "reduced = True #True if using reduced classes (28 categories)\n",
+ "\n",
+ "\n",
+ "model_choice = 2 ## keys: 1: 'Yolo on original TACO', 2: 'Yolo on reduced TACO', 3: 'Yolo on reduced big TACO'"
+ ],
+ "metadata": {
+ "id": "R_Dc3-3xaTwr"
+ },
+ "execution_count": 1,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 2,
+ "metadata": {
+ "id": "0D7J191IYwp8",
+ "colab": {
+ "base_uri": "https://localhost:8080/"
+ },
+ "outputId": "a3ddbdc1-7c41-43ec-ca8e-16cd829d8579"
+ },
+ "outputs": [
+ {
+ "output_type": "stream",
+ "name": "stdout",
+ "text": [
+ "Sun Oct 23 14:17:50 2022 \n",
+ "+-----------------------------------------------------------------------------+\n",
+ "| NVIDIA-SMI 460.32.03 Driver Version: 460.32.03 CUDA Version: 11.2 |\n",
+ "|-------------------------------+----------------------+----------------------+\n",
+ "| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |\n",
+ "| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |\n",
+ "| | | MIG M. |\n",
+ "|===============================+======================+======================|\n",
+ "| 0 A100-SXM4-40GB Off | 00000000:00:04.0 Off | 0 |\n",
+ "| N/A 29C P0 43W / 400W | 0MiB / 40536MiB | 0% Default |\n",
+ "| | | Disabled |\n",
+ "+-------------------------------+----------------------+----------------------+\n",
+ " \n",
+ "+-----------------------------------------------------------------------------+\n",
+ "| Processes: |\n",
+ "| GPU GI CI PID Type Process name GPU Memory |\n",
+ "| ID ID Usage |\n",
+ "|=============================================================================|\n",
+ "| No running processes found |\n",
+ "+-----------------------------------------------------------------------------+\n"
+ ]
+ }
+ ],
+ "source": [
+ "!nvidia-smi"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "# 0. Prep works, install yolov5, download and partition datasets"
+ ],
+ "metadata": {
+ "id": "_9jgRJYlL_-H"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "%cd /content/"
+ ],
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/"
+ },
+ "id": "ucAYkRiCIxFW",
+ "outputId": "27839e27-1d53-4653-d151-3832b18e10be"
+ },
+ "execution_count": 3,
+ "outputs": [
+ {
+ "output_type": "stream",
+ "name": "stdout",
+ "text": [
+ "/content\n"
+ ]
+ }
+ ]
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "%%bash\n",
+ "#rm -rf /content/*\n",
+ "find . \\! -name 'rotated2.zip' -delete"
+ ],
+ "metadata": {
+ "id": "cJDeErmxIuY8"
+ },
+ "execution_count": 4,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 5,
+ "metadata": {
+ "id": "2TTOdqkAk_Ad"
+ },
+ "outputs": [],
+ "source": [
+ "%%capture\n",
+ "!git clone https://github.com/ultralytics/yolov5 \n",
+ "%cd yolov5\n",
+ "!pip install -r requirements.txt #wandb\n",
+ "%cd .."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 6,
+ "metadata": {
+ "id": "FBHHieZsCbGS"
+ },
+ "outputs": [],
+ "source": [
+ "%%capture\n",
+ "!wget https://raw.githubusercontent.com/scikit-multilearn/scikit-multilearn/master/skmultilearn/model_selection/iterative_stratification.py\n",
+ "from iterative_stratification import *\n",
+ "from collections import Counter\n",
+ "\n",
+ "from PIL import Image, ExifTags\n",
+ "from pycocotools.coco import COCO\n",
+ "from matplotlib.patches import Polygon, Rectangle\n",
+ "from matplotlib.collections import PatchCollection\n",
+ "import colorsys\n",
+ "import random\n",
+ "import pylab\n",
+ "\n",
+ "import json\n",
+ "import numpy as np\n",
+ "import pandas as pd\n",
+ "import matplotlib.pyplot as plt\n",
+ "import seaborn as sns; sns.set()\n",
+ "from tqdm import tqdm\n",
+ "\n",
+ "import shutil\n",
+ "import os\n",
+ "import re\n",
+ "\n",
+ "import torch\n",
+ "from torch.utils.data import Dataset, DataLoader\n",
+ "from torchvision import transforms, utils"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "if not mount_drive:\n",
+ " # gdown a gdrive file too frequently triggers google's control and makes the file un-gdown-able\n",
+ " # You can download them manually by adding this prefix: https://drive.google.com/file/d/ to the below codes after !gdown\n",
+ "\n",
+ " !gdown 106n90SCtJz3GsNqJbP0ZGUZaZkDKOSLL # download best trained yolov5x6 weights at https://drive.google.com/file/d/106n90SCtJz3GsNqJbP0ZGUZaZkDKOSLL\n",
+ " !gdown 1X3O2v3GIPveq3ylWF6o1qHI5uzbN1vWA # download organized TACO images (TACO itself, 1500 images) at https://drive.google.com/file/d/1X3O2v3GIPveq3ylWF6o1qHI5uzbN1vWA\n",
+ "\n",
+ "if mount_drive:\n",
+ " from google.colab import drive\n",
+ " drive.mount('/gdrive')\n",
+ " if model_choice ==1:\n",
+ " %cp /gdrive/MyDrive/trash_ai_trained_weights/yolov5x6_best_weights_og.pt /content/yolov5x6_best_weights.pt #get trained weights\n",
+ " elif model_choice ==2:\n",
+ " %cp /gdrive/MyDrive/yolo_og_reduced/exp/weights/best.pt /content/yolov5x6_best_weights.pt\n",
+ " elif model_choice ==3:\n",
+ " %cp /gdrive/MyDrive/best_yolov5_bigTACO/exp/weights/best.pt /content/yolov5x6_best_weights.pt \n",
+ "\n",
+ " \n",
+ " if not os.path.isfile('/content/rotated2.zip'):\n",
+ " if not reduced:\n",
+ " %cp /gdrive/MyDrive/rotated2_og.zip /content/rotated2.zip #get images\n",
+ " else:\n",
+ " %cp /gdrive/MyDrive/rotated2_reduced.zip /content/rotated2.zip #get images\n",
+ "\n",
+ " \n"
+ ],
+ "metadata": {
+ "id": "M0nSxY0wdrr2",
+ "colab": {
+ "base_uri": "https://localhost:8080/"
+ },
+ "outputId": "c78d1453-2583-4fd6-871e-4afe77b4f08f"
+ },
+ "execution_count": 7,
+ "outputs": [
+ {
+ "output_type": "stream",
+ "name": "stdout",
+ "text": [
+ "Mounted at /gdrive\n"
+ ]
+ }
+ ]
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "!unzip -qq /content/rotated2.zip \n",
+ "%mv /content/content/* /content/"
+ ],
+ "metadata": {
+ "id": "Pfvddv2pS_ZX"
+ },
+ "execution_count": 8,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "%%capture\n",
+ "!wget https://raw.githubusercontent.com/pedropro/TACO/master/data/annotations.json\n",
+ "!wget https://raw.githubusercontent.com/pedropro/TACO/master/data/annotations_unofficial.json"
+ ],
+ "metadata": {
+ "id": "nbwKD15fKpfa"
+ },
+ "execution_count": 9,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "%pwd"
+ ],
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/",
+ "height": 36
+ },
+ "id": "V4v7BSn2dT2R",
+ "outputId": "0b5b01cb-0c34-4319-d8ca-1a6af31d1a2a"
+ },
+ "execution_count": 10,
+ "outputs": [
+ {
+ "output_type": "execute_result",
+ "data": {
+ "text/plain": [
+ "'/content'"
+ ],
+ "application/vnd.google.colaboratory.intrinsic+json": {
+ "type": "string"
+ }
+ },
+ "metadata": {},
+ "execution_count": 10
+ }
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 11,
+ "metadata": {
+ "id": "CPFCzX31IGBq",
+ "colab": {
+ "base_uri": "https://localhost:8080/"
+ },
+ "outputId": "9fb1e013-357d-484f-9758-b20bee9d208b"
+ },
+ "outputs": [
+ {
+ "output_type": "stream",
+ "name": "stdout",
+ "text": [
+ "Number of all images:\n",
+ "1500\n"
+ ]
+ },
+ {
+ "output_type": "stream",
+ "name": "stderr",
+ "text": [
+ " 10%|▉ | 149/1500 [00:01<00:07, 182.50it/s]"
+ ]
+ },
+ {
+ "output_type": "stream",
+ "name": "stdout",
+ "text": [
+ "image id number 101 skipped due to no label found\n"
+ ]
+ },
+ {
+ "output_type": "stream",
+ "name": "stderr",
+ "text": [
+ "100%|██████████| 1500/1500 [00:06<00:00, 239.88it/s]\n"
+ ]
+ }
+ ],
+ "source": [
+ "nr_imgs=None\n",
+ "for root, dirnames, filenames in os.walk('./yoloTACO/labels/'):\n",
+ " nr_imgs = len(filenames)\n",
+ " break\n",
+ "print('Number of all images:\\n'+str(nr_imgs))\n",
+ "\n",
+ "if reduced: \n",
+ " nr_class = 28\n",
+ "else: nr_class = 60\n",
+ "\n",
+ "xy = {}\n",
+ "for i in tqdm([i for i in range(nr_imgs)]):\n",
+ " try: \n",
+ " X = pd.read_csv('./yoloTACO/labels/'+str(i)+'.txt',header = None,delimiter = ' ')\n",
+ " x = pd.DataFrame([Counter(X[0].tolist())],columns=[*range(nr_class)]).fillna(0).astype(int).iloc[[0]].squeeze().tolist()\n",
+ " y=i\n",
+ " xy[y]=x\n",
+ " except: \n",
+ " print('image id number '+str(i)+' skipped due to no label found')\n",
+ " pass\n",
+ "X = pd.DataFrame.from_dict(xy, orient='index')\n",
+ "\n",
+ "y = xy.keys()\n",
+ "one_hot_y = pd.get_dummies(y)\n",
+ "\n",
+ "## train test split\n",
+ "'''\n",
+ "train: images/train\n",
+ "val: images/val\n",
+ "test: images/test\n",
+ "'''\n",
+ "\n",
+ "np.random.seed(5) # sk-multilearn is based on sk, sk uses np random state. \n",
+ " # so, setting np random seed will clamp the results of iterative_train_test_split\n",
+ "\n",
+ "X_train, y_train, X_temp, y_temp = iterative_train_test_split(X.values, one_hot_y.values, test_size = 0.5)\n",
+ "X_train1, y_train1, X_val, y_val = iterative_train_test_split(X_temp, y_temp, test_size = 0.5)\n",
+ "X_val, y_val,X_test,y_test = iterative_train_test_split(X_val, y_val, test_size = 0.5)\n",
+ "\n",
+ "# ISSUE: in this environment, any test_size!= 0.5 results in nothing in testing set. \n",
+ "# therefore train/val/test split is roughly .75, .125, .125\n",
+ "\n",
+ "y_train = np.vstack((y_train,y_train1))\n",
+ "train_ids,val_ids,test_ids = pd.DataFrame(y_train,columns = y).idxmax(axis=1).tolist(),\\\n",
+ " pd.DataFrame(y_val,columns = y).idxmax(axis=1).tolist(),\\\n",
+ " pd.DataFrame(y_test,columns = y).idxmax(axis=1).tolist()\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 12,
+ "metadata": {
+ "id": "kwCWClsrSD4z"
+ },
+ "outputs": [],
+ "source": [
+ "%%capture\n",
+ "\n",
+ "def move_helper(ids, desti):\n",
+ " for id in ids:\n",
+ " img_name = os.path.join( './yoloTACO/images', str(id)+'.jpg' )\n",
+ " lbl_name = os.path.join( './yoloTACO/labels', str(id)+'.txt' )\n",
+ " print(img_name)\n",
+ " if os.path.isfile(img_name):\n",
+ " shutil.copy( img_name, './yoloTACO/images/'+desti)\n",
+ " shutil.copy( lbl_name, './yoloTACO/labels/'+desti)\n",
+ " else :\n",
+ " print('file does not exist', img_name)\n",
+ "\n",
+ "!mkdir yoloTACO/images/train\n",
+ "!mkdir yoloTACO/images/val\n",
+ "!mkdir yoloTACO/images/test\n",
+ "!mkdir yoloTACO/labels/train\n",
+ "!mkdir yoloTACO/labels/val\n",
+ "!mkdir yoloTACO/labels/test\n",
+ "move_helper(test_ids,'test')\n",
+ "move_helper(train_ids,'train')\n",
+ "move_helper(val_ids,'val')"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "%%bash\n",
+ "mkdir ./datasets\n",
+ "mv yoloTACO datasets/"
+ ],
+ "metadata": {
+ "id": "gHyjxWAW7DF0"
+ },
+ "execution_count": 13,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [],
+ "metadata": {
+ "id": "S96FWSElHY9d"
+ },
+ "execution_count": 13,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 14,
+ "metadata": {
+ "id": "evH7jgs5Dnj7",
+ "cellView": "form"
+ },
+ "outputs": [],
+ "source": [
+ "#@title yml\n",
+ "\n",
+ "if reduced == True:\n",
+ "\n",
+ " with open('/content/yolov5/data/yoloTACO.yaml', mode='w') as fp:\n",
+ " lines = '''path: ../datasets/yoloTACO # dataset root dir\n",
+ "train: images/train # train images \n",
+ "val: images/val # val images \n",
+ "test: images/test # test images (optional)\n",
+ "\n",
+ "# Classes\n",
+ "names:\n",
+ " 0: Aluminium foil\n",
+ " 1: Battery\n",
+ " 2: Blister pack\n",
+ " 3: Bottle\n",
+ " 4: Bottle cap\n",
+ " 5: Broken glass\n",
+ " 6: Can\n",
+ " 7: Carton\n",
+ " 8: Cup\n",
+ " 9: Food waste\n",
+ " 10: Glass jar\n",
+ " 11: Lid\n",
+ " 12: Other plastic\n",
+ " 13: Paper\n",
+ " 14: Paper bag\n",
+ " 15: Plastic bag & wrapper\n",
+ " 16: Plastic container\n",
+ " 17: Plastic glooves\n",
+ " 18: Plastic utensils\n",
+ " 19: Pop tab\n",
+ " 20: Rope & strings\n",
+ " 21: Scrap metal\n",
+ " 22: Shoe\n",
+ " 23: Squeezable tube\n",
+ " 24: Straw\n",
+ " 25: Styrofoam piece\n",
+ " 26: Unlabeled litter\n",
+ " 27: Cigarette'''\n",
+ " fp.writelines(lines)\n",
+ "\n",
+ "else: \n",
+ " with open('/content/yolov5/data/yoloTACO.yaml', mode='w') as fp:\n",
+ " lines = '''path: ../datasets/yoloTACO # dataset root dir\n",
+ "train: images/train # train images (relative to 'path') 128 images\n",
+ "val: images/val # val images (relative to 'path') 128 images\n",
+ "test: images/test # test images (optional)\n",
+ "\n",
+ "# Classes\n",
+ "names:\n",
+ " 0: Aluminium foil\n",
+ " 1: Battery\n",
+ " 2: Aluminium blister pack\n",
+ " 3: Carded blister pack\n",
+ " 4: Other plastic bottle\n",
+ " 5: Clear plastic bottle\n",
+ " 6: Glass bottle\n",
+ " 7: Plastic bottle cap\n",
+ " 8: Metal bottle cap\n",
+ " 9: Broken glass\n",
+ " 10: Food Can\n",
+ " 11: Aerosol\n",
+ " 12: Drink can\n",
+ " 13: Toilet tube\n",
+ " 14: Other carton\n",
+ " 15: Egg carton\n",
+ " 16: Drink carton\n",
+ " 17: Corrugated carton\n",
+ " 18: Meal carton\n",
+ " 19: Pizza box\n",
+ " 20: Paper cup\n",
+ " 21: Disposable plastic cup\n",
+ " 22: Foam cup\n",
+ " 23: Glass cup\n",
+ " 24: Other plastic cup\n",
+ " 25: Food waste\n",
+ " 26: Glass jar\n",
+ " 27: Plastic lid\n",
+ " 28: Metal lid\n",
+ " 29: Other plastic\n",
+ " 30: Magazine paper\n",
+ " 31: Tissues\n",
+ " 32: Wrapping paper\n",
+ " 33: Normal paper\n",
+ " 34: Paper bag\n",
+ " 35: Plastified paper bag\n",
+ " 36: Plastic film\n",
+ " 37: Six pack rings\n",
+ " 38: Garbage bag\n",
+ " 39: Other plastic wrapper\n",
+ " 40: Single-use carrier bag\n",
+ " 41: Polypropylene bag\n",
+ " 42: Crisp packet\n",
+ " 43: Spread tub\n",
+ " 44: Tupperware\n",
+ " 45: Disposable food container\n",
+ " 46: Foam food container\n",
+ " 47: Other plastic container\n",
+ " 48: Plastic glooves\n",
+ " 49: Plastic utensils\n",
+ " 50: Pop tab\n",
+ " 51: Rope & strings\n",
+ " 52: Scrap metal\n",
+ " 53: Shoe\n",
+ " 54: Squeezable tube\n",
+ " 55: Plastic straw\n",
+ " 56: Paper straw\n",
+ " 57: Styrofoam piece\n",
+ " 58: Unlabeled litter\n",
+ " 59: Cigarette'''\n",
+ " fp.writelines(lines)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 15,
+ "metadata": {
+ "id": "CdMrwEgnB9C4",
+ "colab": {
+ "base_uri": "https://localhost:8080/"
+ },
+ "outputId": "7889b2bc-7032-4700-e827-99db1d1e3750"
+ },
+ "outputs": [
+ {
+ "output_type": "stream",
+ "name": "stdout",
+ "text": [
+ "/content/yolov5\n",
+ "benchmarks.py\t detect.py models\t setup.cfg val.py\n",
+ "classify\t export.py README.md\t train.py\n",
+ "CONTRIBUTING.md hubconf.py requirements.txt tutorial.ipynb\n",
+ "data\t\t LICENSE segment\t utils\n"
+ ]
+ }
+ ],
+ "source": [
+ "%cd ./yolov5\n",
+ "!ls"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "source": [],
+ "metadata": {
+ "id": "67Xa-feZH9q5"
+ },
+ "execution_count": 15,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "# 1. Evaluate with our best trained weights so far"
+ ],
+ "metadata": {
+ "id": "C37qgWyEMLpj"
+ }
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "## 1.1 eval with yolo default scripts"
+ ],
+ "metadata": {
+ "id": "fHmO_QRlN4bQ"
+ }
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 16,
+ "metadata": {
+ "id": "DoLh0BGlXQMC"
+ },
+ "outputs": [],
+ "source": [
+ "#!python val.py --data yoloTACO.yaml --task test --weights /content/yolov5x6_best_weights.pt\n",
+ "#!python detect.py --weights /content/yolov5x6_best_weights.pt --source /content/datasets/yoloTACO/images/test"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Note that the default `MAP` is not the \"wanted\" metrics for our project, as our sponsor specifically requested a metrics under the name \"accuracy\" and a target score of >.90."
+ ],
+ "metadata": {
+ "id": "4of5Qufu1Pd_"
+ }
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "## 1.2 detect with torch framework manually\n",
+ "\n",
+ "This is a necessary step to use our accuracy evaluator."
+ ],
+ "metadata": {
+ "id": "5Ay3MOsDN90j"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "model = torch.hub.load('ultralytics/yolov5', 'custom', path='/content/yolov5x6_best_weights.pt',force_reload=True) # load our local model"
+ ],
+ "metadata": {
+ "id": "30QWyk7yiMsk",
+ "colab": {
+ "base_uri": "https://localhost:8080/"
+ },
+ "outputId": "bc4c3b4d-638a-4663-c0b6-40e8ea059fb8"
+ },
+ "execution_count": 17,
+ "outputs": [
+ {
+ "output_type": "stream",
+ "name": "stderr",
+ "text": [
+ "/usr/local/lib/python3.7/dist-packages/torch/hub.py:267: UserWarning: You are about to download and run code from an untrusted repository. In a future release, this won't be allowed. To add the repository to your trusted list, change the command to {calling_fn}(..., trust_repo=False) and a command prompt will appear asking for an explicit confirmation of trust, or load(..., trust_repo=True), which will assume that the prompt is to be answered with 'yes'. You can also use load(..., trust_repo='check') which will only prompt for confirmation if the repo is not already trusted. This will eventually be the default behaviour\n",
+ " \"You are about to download and run code from an untrusted repository. In a future release, this won't \"\n",
+ "Downloading: \"https://github.com/ultralytics/yolov5/zipball/master\" to /root/.cache/torch/hub/master.zip\n",
+ "YOLOv5 🚀 2022-10-23 Python-3.7.15 torch-1.12.1+cu113 CUDA:0 (A100-SXM4-40GB, 40536MiB)\n",
+ "\n",
+ "Fusing layers... \n",
+ "Model summary: 416 layers, 140230396 parameters, 0 gradients, 208.5 GFLOPs\n",
+ "Adding AutoShape... \n"
+ ]
+ }
+ ]
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Load test imgs\n",
+ "test_dir = '/content/datasets/yoloTACO/images/test/'\n",
+ "test_list = test_ids # [i[2] for i in os.walk(test_dir)][0] # or alternatively read from files # test_list = [re.findall(r'\\d+',i)[0] for i in test_list]\n",
+ "\n",
+ "test_read_img_list = [Image.open(test_dir+str(i)+'.jpg') for i in test_list] # alternatively use cv2: cv2.imread('target_path')[..., ::-1] # OpenCV image (BGR to RGB)"
+ ],
+ "metadata": {
+ "id": "GUjCnveZk2sf"
+ },
+ "execution_count": 18,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Inference\n",
+ "results = model(test_read_img_list) # batch of images\n",
+ "pred_pd = results.pandas().xywh\n",
+ "\n",
+ "for j,i in enumerate(pred_pd):\n",
+ " i=i.assign(image_id=[test_list[j]]*i.shape[0])\n",
+ " pred_pd[j]=i"
+ ],
+ "metadata": {
+ "id": "RKr_CupeiMvD"
+ },
+ "execution_count": 19,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# clear GPU mem\n",
+ "def free_memory(to_delete: list, debug=False):\n",
+ " import gc\n",
+ " import inspect\n",
+ " calling_namespace = inspect.currentframe().f_back\n",
+ " if debug:\n",
+ " print('Before:')\n",
+ " torch.get_less_used_gpu(debug=True)\n",
+ "\n",
+ " for _var in to_delete:\n",
+ " calling_namespace.f_locals.pop(_var, None)\n",
+ " gc.collect()\n",
+ " torch.cuda.empty_cache()\n",
+ " if debug:\n",
+ " print('After:')\n",
+ " torch.get_less_used_gpu(debug=True)\n",
+ "\n",
+ "free_memory([model])"
+ ],
+ "metadata": {
+ "id": "HvwZN3vzHB5u"
+ },
+ "execution_count": 20,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "%%capture\n",
+ "!wget -O data/annotations.json https://raw.githubusercontent.com/pedropro/TACO/master/data/annotations.json\n",
+ "anno_path = './data/annotations.json'\n",
+ "annos = COCO(annotation_file=anno_path)\n",
+ "with open(anno_path, 'r') as f:\n",
+ " annos_json = json.loads(f.read())\n",
+ "no_to_clname = {i:j for i,j in enumerate([i['name'] for i in annos_json['categories']])}\n"
+ ],
+ "metadata": {
+ "id": "QRj9_Dk_iMxG"
+ },
+ "execution_count": 21,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "truth_pd = []\n",
+ "for i in test_list:\n",
+ " img_info = annos.loadImgs(i)[0] \n",
+ " img_height = img_info['height']\n",
+ " img_width = img_info['width']\n",
+ "\n",
+ " cache = pd.read_csv('/content/datasets/yoloTACO/labels/'+str(i)+'.txt',header=None,\n",
+ " names = ['class','xcenter','ycenter','width','height'],delimiter=' ')\n",
+ " cache[\"xcenter\"] = img_width * cache[\"xcenter\"]\n",
+ " cache[\"ycenter\"] = img_height * cache[\"ycenter\"]\n",
+ " cache[\"width\"] = img_width * cache[\"width\"]\n",
+ " cache[\"height\"] = img_height * cache[\"height\"]\n",
+ "\n",
+ " cache = cache.assign(confidence = [1]*cache.shape[0])\n",
+ " cache = cache.reindex(columns=['xcenter','ycenter','width','height','confidence','class'])\n",
+ " cache = cache.assign(image_id = [i]*cache.shape[0])\n",
+ "\n",
+ " # cache = cache.assign(img_width = [width]*cache.shape[0])\n",
+ " # cache = cache.assign(img_height = [height]*cache.shape[0])\n",
+ "\n",
+ " truth_pd.append(cache)"
+ ],
+ "metadata": {
+ "id": "o6dfoDEzdV-1"
+ },
+ "execution_count": 22,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "## 1.3 example prediction and truth"
+ ],
+ "metadata": {
+ "id": "QNrQCtxcBiw0"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "pred_pd[:2] \n",
+ "# predictions for first two images\n",
+ "# there will be a list of two dataframes"
+ ],
+ "metadata": {
+ "id": "WuvboCQRLUvU",
+ "colab": {
+ "base_uri": "https://localhost:8080/"
+ },
+ "outputId": "b6b7cefd-d665-456a-e4f8-42c525d94f12"
+ },
+ "execution_count": 23,
+ "outputs": [
+ {
+ "output_type": "execute_result",
+ "data": {
+ "text/plain": [
+ "[Empty DataFrame\n",
+ " Columns: [xcenter, ycenter, width, height, confidence, class, name, image_id]\n",
+ " Index: [],\n",
+ " xcenter ycenter width height confidence class \\\n",
+ " 0 869.249634 673.73645 92.600647 255.507812 0.751637 16 \n",
+ " \n",
+ " name image_id \n",
+ " 0 Plastic container 8 ]"
+ ]
+ },
+ "metadata": {},
+ "execution_count": 23
+ }
+ ]
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "pred_pd[3]"
+ ],
+ "metadata": {
+ "id": "WFbRwGhdR0o5",
+ "colab": {
+ "base_uri": "https://localhost:8080/",
+ "height": 49
+ },
+ "outputId": "f7d84397-a973-4b0c-e730-69a6fb6e4b93"
+ },
+ "execution_count": 24,
+ "outputs": [
+ {
+ "output_type": "execute_result",
+ "data": {
+ "text/plain": [
+ "Empty DataFrame\n",
+ "Columns: [xcenter, ycenter, width, height, confidence, class, name, image_id]\n",
+ "Index: []"
+ ],
+ "text/html": [
+ "\n",
+ "