notebooks/206-vision-paddlegan-anime/206-vision-paddlegan-anime.ipynb

{
 "cells": [
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "5c044bd4",
   "metadata": {},
   "source": [
    "# Photos to Anime with PaddleGAN and OpenVINO\n",
    "\n",
    "This tutorial demonstrates converting a [PaddlePaddle/PaddleGAN](https://github.com/PaddlePaddle/PaddleGAN) AnimeGAN model to OpenVINO IR format, and shows inference results on the PaddleGAN and OpenVINO IR models.\n",
    "\n",
    "For more information about the model, see [PaddleGAN's AnimeGAN documentation](https://github.com/PaddlePaddle/PaddleGAN/blob/develop/docs/en_US/tutorials/animegan.md)\n",
    "\n",
    "![anime](https://user-images.githubusercontent.com/15709723/123559130-04550100-d74f-11eb-819c-a02284654428.jpg)\n",
    "\n",
    "#### Table of content:\n",
    "- [Preparation](#Preparation-Uparrow)\n",
    "    - [Install requirements](#Install-requirements-Uparrow)\n",
    "    - [Imports](#Imports-Uparrow)\n",
    "    - [Settings](#Settings-Uparrow)\n",
    "    - [Functions](#Functions-Uparrow)\n",
    "- [Inference on PaddleGAN Model](#Inference-on-PaddleGAN-Model-Uparrow)\n",
    "    - [Show Inference Results on PaddleGAN model](#Show-Inference-Results-on-PaddleGAN-model-Uparrow)\n",
    "- [Model Conversion to ONNX and OpenVINO IR](#Model-Conversion-to-ONNX-and-OpenVINO-IR-Uparrow)\n",
    "    - [Convert to ONNX](#Convert-to-ONNX-Uparrow)\n",
    "    - [Convert to OpenVINO IR](#Convert-to-OpenVINO-IR-Uparrow)\n",
    "- [Show Inference Results on OpenVINO IR and PaddleGAN Models](#Show-Inference-Results-on-OpenVINO-IR-and-PaddleGAN-Models-Uparrow)\n",
    "    - [Create Postprocessing Functions](#Create-Postprocessing-Functions-Uparrow)\n",
    "    - [Do Inference on OpenVINO IR Model](#Do-Inference-on-OpenVINO-IR-Model-Uparrow)\n",
    "        - [Select inference device](#Select-inference-device-Uparrow)\n",
    "- [Performance Comparison](#Performance-Comparison-Uparrow)\n",
    "- [References](#References-Uparrow)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9a3d38f1",
   "metadata": {},
   "source": [
    "## Preparation [$\\Uparrow$](#Table-of-content:)\n",
    "\n",
    "### Install requirements [$\\Uparrow$](#Table-of-content:)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b6bcf442-f6c9-467f-bfcf-cd3111772cbf",
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "%pip install -q \"openvino>=2023.1.0\"\n",
    "\n",
    "%pip install -q \"paddlepaddle==2.5.0\" \"paddle2onnx>=0.6\"\n",
    "%pip install -q \"git+https://github.com/PaddlePaddle/PaddleGAN.git\" --no-deps\n",
    "\n",
    "%pip install -q opencv-python matplotlib scikit-learn scikit-image\n",
    "%pip install -q \"imageio==2.9.0\" \"imageio-ffmpeg\" \"numba>=0.53.1\" easydict munch natsort"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "96e1f503",
   "metadata": {},
   "source": [
    "### Imports [$\\Uparrow$](#Table-of-content:)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "fe02fa14",
   "metadata": {},
   "outputs": [],
   "source": [
    "import sys\n",
    "import time\n",
    "import os\n",
    "from pathlib import Path\n",
    "import urllib.request\n",
    "\n",
    "import cv2\n",
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "import openvino as ov\n",
    "from IPython.display import HTML, display\n",
    "\n",
    "# PaddlePaddle requires a C++ compiler. If importing the paddle packages fails,\n",
    "# install C++.\n",
    "try:\n",
    "    import paddle\n",
    "    from paddle.static import InputSpec\n",
    "    from ppgan.apps import AnimeGANPredictor\n",
    "except NameError:\n",
    "    if sys.platform == \"win32\":\n",
    "        install_message = (\n",
    "            \"To use this notebook, please install the free Microsoft \"\n",
    "            \"Visual C++ redistributable from <a href='https://aka.ms/vs/16/release/vc_redist.x64.exe'>\"\n",
    "            \"https://aka.ms/vs/16/release/vc_redist.x64.exe</a>\"\n",
    "        )\n",
    "    else:\n",
    "        install_message = (\n",
    "            \"To use this notebook, please install a C++ compiler. On macOS, \"\n",
    "            \"`xcode-select --install` installs many developer tools, including C++. On Linux, \"\n",
    "            \"install gcc with your distribution's package manager.\"\n",
    "        )\n",
    "    display(\n",
    "        HTML(\n",
    "            f\"\"\"<div class=\"alert alert-danger\" ><i>\n",
    "    <b>Error: </b>PaddlePaddle requires installation of C++. {install_message}\"\"\"\n",
    "        )\n",
    "    )\n",
    "    raise"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "16b2e5ee",
   "metadata": {},
   "source": [
    "### Settings [$\\Uparrow$](#Table-of-content:)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "ccdb81b9",
   "metadata": {},
   "outputs": [],
   "source": [
    "MODEL_DIR = \"model\"\n",
    "MODEL_NAME = \"paddlegan_anime\"\n",
    "\n",
    "os.makedirs(MODEL_DIR, exist_ok=True)\n",
    "\n",
    "# Create filenames of the models that will be converted in this notebook.\n",
    "model_path = Path(f\"{MODEL_DIR}/{MODEL_NAME}\")\n",
    "ir_path = model_path.with_suffix(\".xml\")\n",
    "onnx_path = model_path.with_suffix(\".onnx\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0d1fa797",
   "metadata": {},
   "source": [
    "### Functions [$\\Uparrow$](#Table-of-content:)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "ceef6133",
   "metadata": {},
   "outputs": [],
   "source": [
    "def resize_to_max_width(image, max_width):\n",
    "    \"\"\"\n",
    "    Resize `image` to `max_width`, preserving the aspect ratio of the image.\n",
    "    \"\"\"\n",
    "    if image.shape[1] > max_width:\n",
    "        hw_ratio = image.shape[0] / image.shape[1]\n",
    "        new_height = int(max_width * hw_ratio)\n",
    "        image = cv2.resize(image, (max_width, new_height))\n",
    "    return image"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b42a1025",
   "metadata": {},
   "source": [
    "## Inference on PaddleGAN Model [$\\Uparrow$](#Table-of-content:)\n",
    "\n",
    "The PaddleGAN [documentation](https://github.com/PaddlePaddle/PaddleGAN/blob/develop/docs/en_US/tutorials/animegan.md) explains how to run the model with `.run()` method. Find out what that function does with Jupyter's `??` shortcut to show the docstring and source of the function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "2a83fd1a",
   "metadata": {},
   "outputs": [],
   "source": [
    "# This cell will initialize the AnimeGANPredictor() and download the weights from PaddlePaddle.\n",
    "# This may take a while. The weights are stored in a cache and are downloaded once.\n",
    "predictor = AnimeGANPredictor()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "19656fda",
   "metadata": {},
   "outputs": [],
   "source": [
    "# In a Jupyter Notebook, ?? shows the source and docstring\n",
    "predictor.run??"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cca83782",
   "metadata": {},
   "source": [
    "The `AnimeGANPredictor.run()` method works as follow:\n",
    "\n",
    "1. Loads an image with OpenCV and converts it to RGB.\n",
    "2. Transforms the image.\n",
    "3. Propagates the transformed image through the generator model and postprocesses the results to return an array with a [0,255] range.\n",
    "4. Transposes the result from (C,H,W) to (H,W,C) shape.\n",
    "5. Resizes the result image to the original image size.\n",
    "6. (optional) Adjusts the brightness of the result image.\n",
    "7. Saves the image.\n",
    "\n",
    "You can execute these steps manually and confirm that the result looks correct. To speed up inference time, resize large images before propagating them through the network. The inference step in the next cell will still take some time to execute. If you want to skip this step, set `PADDLEGAN_INFERENCE = False` in the first line of the next cell."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "4286e254",
   "metadata": {},
   "outputs": [],
   "source": [
    "PADDLEGAN_INFERENCE = True\n",
    "OUTPUT_DIR = \"output\"\n",
    "\n",
    "os.makedirs(OUTPUT_DIR, exist_ok=True)\n",
    "# Step 1. Load the image and convert to RGB.\n",
    "image_path = Path(\"./data/coco_bricks.png\")\n",
    "# fetch the image from the web\n",
    "image_path.parent.mkdir(parents=True, exist_ok=True)\n",
    "urllib.request.urlretrieve(\n",
    "    \"https://storage.openvinotoolkit.org/repositories/openvino_notebooks/data/data/image/coco_bricks.png\",\n",
    "    image_path\n",
    ")\n",
    "\n",
    "image = cv2.cvtColor(cv2.imread(str(image_path), flags=cv2.IMREAD_COLOR), cv2.COLOR_BGR2RGB)\n",
    "\n",
    "## Inference takes a long time on large images. Resize to a max width of 600.\n",
    "image = resize_to_max_width(image, 600)\n",
    "\n",
    "# Step 2. Transform the image.\n",
    "transformed_image = predictor.transform(image)\n",
    "input_tensor = paddle.to_tensor(transformed_image[None, ::])\n",
    "\n",
    "if PADDLEGAN_INFERENCE:\n",
    "    # Step 3. Do inference. \n",
    "    predictor.generator.eval()\n",
    "    with paddle.no_grad():\n",
    "        result = predictor.generator(input_tensor)\n",
    "\n",
    "    # Step 4. Convert the inference result to an image, following the same steps as\n",
    "    # PaddleGAN's predictor.run() function.\n",
    "    result_image_pg = (result * 0.5 + 0.5)[0].numpy() * 255\n",
    "    result_image_pg = result_image_pg.transpose((1, 2, 0))\n",
    "\n",
    "    # Step 5. Resize the result image.\n",
    "    result_image_pg = cv2.resize(result_image_pg, image.shape[:2][::-1])\n",
    "\n",
    "    # Step 6. Adjust the brightness.\n",
    "    result_image_pg = predictor.adjust_brightness(result_image_pg, image)\n",
    "\n",
    "    # Step 7. Save the result image.\n",
    "    anime_image_path_pg = Path(f\"{OUTPUT_DIR}/{image_path.stem}_anime_pg\").with_suffix(\".jpg\")\n",
    "    if cv2.imwrite(str(anime_image_path_pg), result_image_pg[:, :, (2, 1, 0)]):\n",
    "        print(f\"The anime image was saved to {anime_image_path_pg}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1fe63995",
   "metadata": {},
   "source": [
    "### Show Inference Results on PaddleGAN model [$\\Uparrow$](#Table-of-content:)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "0e8346ad",
   "metadata": {},
   "outputs": [],
   "source": [
    "if PADDLEGAN_INFERENCE:\n",
    "    fig, ax = plt.subplots(1, 2, figsize=(25, 15))\n",
    "    ax[0].imshow(image)\n",
    "    ax[1].imshow(result_image_pg)\n",
    "else:\n",
    "    print(\"PADDLEGAN_INFERENCE is not enabled. Set PADDLEGAN_INFERENCE = True in the previous cell and run that cell to show inference results.\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "062c50c2",
   "metadata": {},
   "source": [
    "## Model Conversion to ONNX and OpenVINO IR [$\\Uparrow$](#Table-of-content:)\n",
    "\n",
    "Convert the PaddleGAN model to OpenVINO IR by first converting PaddleGAN to ONNX with `paddle2onnx` and then converting the ONNX model to OpenVINO IR with model conversion API.\n",
    "\n",
    "### Convert to ONNX [$\\Uparrow$](#Table-of-content:)\n",
    "\n",
    "Exporting to ONNX requires specifying an input shape with PaddlePaddle `InputSpec` and calling `paddle.onnx.export`. Then, check the input shape of the transformed image and use that as the input shape for the ONNX model. Exporting to ONNX should not take long. If the export succeeds, the output of the next cell will include `ONNX model saved in paddlegan_anime.onnx`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "e463f3d5",
   "metadata": {},
   "outputs": [],
   "source": [
    "target_height, target_width = transformed_image.shape[1:]\n",
    "target_height, target_width"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "bd613c87",
   "metadata": {},
   "outputs": [],
   "source": [
    "predictor.generator.eval()\n",
    "x_spec = InputSpec([None, 3, target_height, target_width], \"float32\", \"x\")\n",
    "paddle.onnx.export(predictor.generator, str(model_path), input_spec=[x_spec], opset_version=11)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d6e3b045",
   "metadata": {},
   "source": [
    "### Convert to OpenVINO IR [$\\Uparrow$](#Table-of-content:)\n",
    "\n",
    "The OpenVINO IR format enables storing the preprocessing normalization in the model file. It is then no longer necessary to normalize input images manually. See the transforms that the `.run()` method used:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "68b63d9c",
   "metadata": {},
   "outputs": [],
   "source": [
    "predictor.__init__??"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "b0ee771d",
   "metadata": {},
   "outputs": [],
   "source": [
    "t = predictor.transform.transforms[0]\n",
    "t.params"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "566ae694",
   "metadata": {},
   "outputs": [],
   "source": [
    "## Uncomment the line below to see the documentation and code of the ResizeToScale transformation\n",
    "# t??"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7fa54081",
   "metadata": {},
   "source": [
    "There are three transformations: resize, transpose, and normalize, where normalize uses a mean and scale of `[127.5, 127.5, 127.5]`. \n",
    "\n",
    "The `ResizeToScale` class is called with `(256,256)` as the argument for size. Further analysis shows that this is\n",
    "the minimum size to resize to. The `ResizeToScale` class transform resizes images to the size specified in the\n",
    "`ResizeToScale` parameters, with width and height as multiples of 32. We will preprocess the images the same way before feeding them to the converted model.\n",
    "\n",
    "Now we use model conversion API and convert the model to OpenVINO IR."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "63ecd375",
   "metadata": {},
   "source": [
    "**Convert ONNX Model to OpenVINO IR with [Model Conversion Python API](https://docs.openvino.ai/2023.0/openvino_docs_model_processing_introduction.html)**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "21e58ce1",
   "metadata": {},
   "outputs": [],
   "source": [
    "print(\"Exporting ONNX model to OpenVINO IR... This may take a few minutes.\")\n",
    "\n",
    "model = ov.convert_model(\n",
    "    onnx_path,\n",
    "    input=[1, 3, target_height, target_width],\n",
    ")\n",
    "\n",
    "# Serialize model in IR format\n",
    "ov.save_model(model, str(ir_path))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7d43b2e1",
   "metadata": {},
   "source": [
    "## Show Inference Results on OpenVINO IR and PaddleGAN Models [$\\Uparrow$](#Table-of-content:)\n",
    "\n",
    "If the conversion is successful, the output of model conversion API in the cell above will show *SUCCESS*, and the OpenVINO IR model will be generated.\n",
    "\n",
    "Now, use the model for inference with the `adjust_brightness()` method from the PaddleGAN model. However, in order to use the OpenVINO IR model without installing PaddleGAN, it is useful to check what these functions do and extract them."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "409c755c",
   "metadata": {},
   "source": [
    "### Create Postprocessing Functions [$\\Uparrow$](#Table-of-content:)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "78fb8d56",
   "metadata": {},
   "outputs": [],
   "source": [
    "predictor.adjust_brightness??"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "42120455",
   "metadata": {},
   "outputs": [],
   "source": [
    "predictor.calc_avg_brightness??"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "79bb7ffb",
   "metadata": {},
   "source": [
    "The average brightness is computed by a [standard formula](https://www.w3.org/TR/AERT/#color-contrast). To adjust the brightness, the difference in brightness between the source and destination (anime) image is computed and the brightness of the destination image is adjusted based on that. Then, the image is converted to an 8-bit image.\n",
    "\n",
    "Copy these functions to the next cell, use them for inference on the OpenVINO IR model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "2e44615f",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Copyright (c) 2020 PaddlePaddle Authors. Licensed under the Apache License, Version 2.0\n",
    "\n",
    "\n",
    "def calc_avg_brightness(img):\n",
    "    R = img[..., 0].mean()\n",
    "    G = img[..., 1].mean()\n",
    "    B = img[..., 2].mean()\n",
    "\n",
    "    brightness = 0.299 * R + 0.587 * G + 0.114 * B\n",
    "    return brightness, B, G, R\n",
    "\n",
    "\n",
    "def adjust_brightness(dst, src):\n",
    "    brightness1, B1, G1, R1 = AnimeGANPredictor.calc_avg_brightness(src)\n",
    "    brightness2, B2, G2, R2 = AnimeGANPredictor.calc_avg_brightness(dst)\n",
    "    brightness_difference = brightness1 / brightness2\n",
    "    dstf = dst * brightness_difference\n",
    "    dstf = np.clip(dstf, 0, 255)\n",
    "    dstf = np.uint8(dstf)\n",
    "    return dstf"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c643f9eb",
   "metadata": {
    "tags": []
   },
   "source": [
    "### Do Inference on OpenVINO IR Model [$\\Uparrow$](#Table-of-content:)\n",
    "\n",
    "Load the OpenVINO IR model and do inference, following the same steps as for the PaddleGAN model. For more information about inference on OpenVINO IR models, see the [OpenVINO Runtime API notebook](../002-openvino-api/002-openvino-api.ipynb).\n",
    "\n",
    "The OpenVINO IR model is generated with an input shape that is computed based on the input image. If you do inference on images with different input shapes, results may differ from the PaddleGAN results. \n",
    "\n",
    "#### Select inference device [$\\Uparrow$](#Table-of-content:)\n",
    "\n",
    "select device from dropdown list for running inference using OpenVINO"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "7e28cf0b-8204-422a-a7af-54824a1c98fd",
   "metadata": {
    "tags": [
     "hide-input"
    ]
   },
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "cf5dee181aa54d4eb86112a9a7f1eb1e",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Dropdown(description='Device:', index=2, options=('CPU', 'GPU', 'AUTO'), value='AUTO')"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import ipywidgets as widgets\n",
    "\n",
    "core = ov.Core()\n",
    "device = widgets.Dropdown(\n",
    "    options=core.available_devices + [\"AUTO\"],\n",
    "    value='AUTO',\n",
    "    description='Device:',\n",
    "    disabled=False,\n",
    ")\n",
    "\n",
    "device"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "67b8d317",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Load and prepare the IR model.\n",
    "core = ov.Core()\n",
    "\n",
    "model = core.read_model(model=ir_path)\n",
    "compiled_model = core.compile_model(model=model, device_name=device.value)\n",
    "input_key = compiled_model.input(0)\n",
    "output_key = compiled_model.output(0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "bfb9d826",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Step 1. Load an image and convert it to RGB.\n",
    "image_path = Path(\"./data/coco_bricks.png\")\n",
    "image = cv2.cvtColor(cv2.imread(str(image_path), flags=cv2.IMREAD_COLOR), cv2.COLOR_BGR2RGB)\n",
    "\n",
    "# Step 2. Do preprocess transformations.\n",
    "# Resize the image\n",
    "resized_image = cv2.resize(image, (target_width, target_height))\n",
    "input_image = resized_image.transpose(2, 0, 1)[None, :, :, :]\n",
    "# Normalize the image\n",
    "input_mean = np.array([127.5,127.5,127.5]).reshape(1, 3, 1, 1)\n",
    "input_scale = np.array([127.5,127.5,127.5]).reshape(1, 3, 1, 1)\n",
    "input_image = (input_image - input_mean) / input_scale\n",
    "\n",
    "# Step 3. Do inference.\n",
    "result_ir = compiled_model([input_image])[output_key]\n",
    "\n",
    "# Step 4. Convert the inference result to an image, following the same steps as\n",
    "# PaddleGAN's predictor.run() function.\n",
    "result_image_ir = (result_ir * 0.5 + 0.5)[0] * 255\n",
    "result_image_ir = result_image_ir.transpose((1, 2, 0))\n",
    "\n",
    "# Step 5. Resize the result image.\n",
    "result_image_ir = cv2.resize(result_image_ir, image.shape[:2][::-1])\n",
    "\n",
    "# Step 6. Adjust the brightness.\n",
    "result_image_ir = adjust_brightness(result_image_ir, image)\n",
    "\n",
    "# Step 7. Save the result image.\n",
    "anime_fn_ir = Path(f\"{OUTPUT_DIR}/{image_path.stem}_anime_ir\").with_suffix(\".jpg\")\n",
    "if cv2.imwrite(str(anime_fn_ir), result_image_ir[:, :, (2, 1, 0)]):\n",
    "    print(f\"The anime image was saved to {anime_fn_ir}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0620d3f3",
   "metadata": {},
   "source": [
    "**Show Inference Results**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "9bdbd71f",
   "metadata": {},
   "outputs": [],
   "source": [
    "fig, ax = plt.subplots(1, 2, figsize=(25, 15))\n",
    "ax[0].imshow(image)\n",
    "ax[1].imshow(result_image_ir)\n",
    "ax[0].set_title(\"Image\")\n",
    "ax[1].set_title(\"OpenVINO IR result\");"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e4a230f7",
   "metadata": {},
   "source": [
    "## Performance Comparison [$\\Uparrow$](#Table-of-content:)\n",
    "\n",
    "Measure the time it takes to do inference on an image. This gives an indication of performance. It is not a perfect measure. Since the PaddleGAN model requires quite a bit of memory for inference, only measure inference on one image. For more accurate benchmarking, use [Benchmark Tool](../104-model-tools/104-model-tools.ipynb)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "f78c6aaf",
   "metadata": {},
   "outputs": [],
   "source": [
    "NUM_IMAGES = 1\n",
    "start = time.perf_counter()\n",
    "for _ in range(NUM_IMAGES):\n",
    "    compiled_model([input_image])\n",
    "end = time.perf_counter()\n",
    "time_ir = end - start\n",
    "print(\n",
    "    f\"OpenVINO IR model in OpenVINO Runtime/CPU: {time_ir/NUM_IMAGES:.3f} \"\n",
    "    f\"seconds per image, FPS: {NUM_IMAGES/time_ir:.2f}\"\n",
    ")\n",
    "\n",
    "## `PADDLEGAN_INFERENCE` is defined in the \"Inference on PaddleGAN model\" section above.\n",
    "## Uncomment the next line to enable a performance comparison with the PaddleGAN model\n",
    "## if you disabled it earlier. \n",
    "\n",
    "# PADDLEGAN_INFERENCE = True\n",
    "\n",
    "if PADDLEGAN_INFERENCE:\n",
    "    with paddle.no_grad():\n",
    "        start = time.perf_counter()\n",
    "        for _ in range(NUM_IMAGES):\n",
    "            predictor.generator(input_tensor)\n",
    "        end = time.perf_counter()\n",
    "        time_paddle = end - start\n",
    "    print(\n",
    "        f\"PaddleGAN model on CPU: {time_paddle/NUM_IMAGES:.3f} seconds per image, \"\n",
    "        f\"FPS: {NUM_IMAGES/time_paddle:.2f}\"\n",
    "    )"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "bc996414",
   "metadata": {},
   "source": [
    "## References [$\\Uparrow$](#Table-of-content:)\n",
    "\n",
    "* [PaddleGAN](https://github.com/PaddlePaddle/PaddleGAN)\n",
    "* [Paddle2ONNX](https://github.com/PaddlePaddle/paddle2onnx)\n",
    "* [OpenVINO ONNX support](https://docs.openvino.ai/2021.4/openvino_docs_IE_DG_ONNX_Support.html)\n",
    "* [Model Conversion API](https://docs.openvino.ai/2023.0/openvino_docs_model_processing_introduction.html)\n",
    "\n",
    "The PaddleGAN code that is shown in this notebook is written by PaddlePaddle Authors and licensed under the Apache 2.0 license. \n",
    "The license for this code is displayed below.\n",
    "\n",
    "    #  Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserve.\n",
    "    #\n",
    "    #Licensed under the Apache License, Version 2.0 (the \"License\");\n",
    "    #you may not use this file except in compliance with the License.\n",
    "    #You may obtain a copy of the License at\n",
    "    #\n",
    "    #    http://www.apache.org/licenses/LICENSE-2.0\n",
    "    #\n",
    "    #Unless required by applicable law or agreed to in writing, software\n",
    "    #distributed under the License is distributed on an \"AS IS\" BASIS,\n",
    "    #WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
    "    #See the License for the specific language governing permissions and\n",
    "    #limitations under the License."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.12"
  },
  "vscode": {
   "interpreter": {
    "hash": "cec18e25feb9469b5ff1085a8097bdcd86db6a4ac301d6aeff87d0f3e7ce4ca5"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}