4-Optimization.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div>\n",
    "<img src=\"https://discuss.pytorch.org/uploads/default/original/2X/3/35226d9fbc661ced1c5d17e374638389178c3176.png\" width=\"400\" style=\"margin: 50px auto; display: block; position: relative; left: -30px;\" />\n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<!--NAVIGATION-->\n",
    "# < [Autograd](3-Autograd.ipynb) | Optimization | [Modules](5-Modules.ipynb) >"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Optimization\n",
    "\n",
    "In this short notebook, we will see how to use the gradient obtained with Autograd to perform optimization of an objective function.  \n",
    "Then we will also present some off-the-shelf Pytorch optimizers, loss functions, and learning rate schedulers.  \n",
    "\n",
    "### Table of Contents\n",
    "\n",
    "#### 1. [Optimizing by hand](#Optimizing-by-hand)  \n",
    "#### 2. [Optimizing with an Optimizer](#Optimizing-with-an-optimizer)\n",
    "#### 3. [2D Plots](#2D-Plot---Optimization-process)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "___"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import sys\n",
    "import torch\n",
    "\n",
    "\n",
    "if 'google.colab' in sys.modules: # Execute if you're using Google Colab\n",
    "    !wget -q https://raw.githubusercontent.com/theevann/amld-pytorch-workshop/master/live_plot.py -O live_plot.py\n",
    "    !pip install -q ipympl\n",
    "\n",
    "%matplotlib ipympl\n",
    "torch.set_printoptions(precision=3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "toc-hr-collapsed": false
   },
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Optimizing by hand"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We will start with a simple example : finding the zero of the cube function.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def f(x):\n",
    "    return x ** 3"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We will search the zero of the function $f$ \"by hand\" by minimizing $\\|f(x) \\|_1$ using the **gradient descent algorithm**.\n",
    "\n",
    "We will call $\\|f(x) \\|_1$ the **loss** as this is the value we wish to minimise.\n",
    "\n",
    "As a reminder, the update step of the gradient descent algorithm is:\n",
    "$$x_{t+1} = x_{t} - \\lambda \\nabla_x f (x_t)$$\n",
    "\n",
    "Note:\n",
    "- You can use `.abs()` to compute the loss\n",
    "- The gradient information $\\nabla_x f (x)$ will be stored in `x.grad` once we run the `backward` function.\n",
    "- The gradient is accumulated by default, so we need to clear `x.grad` after each iteration.\n",
    "- We need to use `with torch.no_grad():` context for the update step since we want to change `x` in place but don't want autograd to track this change."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### **Your turn !**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "x0 = 7\n",
    "lr = 0.01\n",
    "iterations = 10\n",
    "\n",
    "x = torch.Tensor([x0]).requires_grad_()\n",
    "y = f(x)\n",
    "\n",
    "for i in range(iterations):\n",
    "    \n",
    "    # YOUR TURN\n",
    "    # Compute y, the loss and the gradient of the loss wrt. x\n",
    "    \n",
    "    with torch.no_grad():\n",
    "        # Update x\n",
    "        # Clear the gradient of x\n",
    "    \n",
    "    print(y.item())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Why do we use `with torch.no_grad()` ?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Because `x` \"requires grad\", any operation we apply to `x` is recorded for automatic differentiation. As we don't want to track the update step of the parameters, we need to \"tell\" autograd not to track this change. This is done by using the `torch.no_grad()` context."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Optimizing with an optimizer"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Different optimizers\n",
    "PyTorch provides most common optimization algorithms encapsulated into \"optimizer classes\".  \n",
    "An optimizer is an object that automatically loops through all the numerous parameters of your model and performs the (potentially complex) update step for you.\n",
    "\n",
    "You first need to import `torch.optim`. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import torch.optim as optim"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Below are the most commonly used optimizers. Each of them has its specific parameters that you can check on the [Pytorch Doc](https://pytorch.org/docs/master/optim.html#algorithms)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "parameters = [x]  # This should be the list of model parameters\n",
    "\n",
    "optimizer = optim.SGD(parameters, lr=0.01, momentum=0.9)\n",
    "optimizer = optim.Adam(parameters, lr=0.01)\n",
    "optimizer = optim.Adadelta(parameters, lr=0.01)\n",
    "optimizer = optim.Adagrad(parameters, lr=0.01)\n",
    "optimizer = optim.RMSprop(parameters, lr=0.01)\n",
    "optimizer = optim.LBFGS(parameters, lr=0.01)\n",
    "\n",
    "# and there is more ..."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Different loss functions"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "PyTorch comes with a lot of predefined loss functions :\n",
    "- `L1Loss`\n",
    "- `MSELoss`\n",
    "- `CrossEntropyLoss`\n",
    "- `NLLLoss`\n",
    "- `PoissonNLLLoss`\n",
    "- `KLDivLoss`\n",
    "- `BCELoss`\n",
    "- `...`\n",
    "\n",
    "Check out the [PyTorch Documentation](https://pytorch.org/docs/master/nn.html#loss-functions)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "loss_function = torch.nn.L1Loss()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "x = torch.Tensor([1,1,1])\n",
    "y = torch.Tensor([1,2,3])\n",
    "\n",
    "loss_function(x, y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This ${L}_1$ `loss_function` computes the average absolute difference between $x$ and $y$ as $\\dfrac{1}{size(x)} \\| x - y \\|_{1}$."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Using an optimizer"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, let's use an optimizer to do the optimization !\n",
    "\n",
    "You will need 2 new functions:\n",
    "- `optimizer.zero_grad()` : This function sets the gradient of the parameters (`x` here) to 0 (otherwise it will get accumulated)\n",
    "- `optimizer.step()` :  This function applies an update step\n",
    "\n",
    "We will also use a loss function which you need to call with:\n",
    "- as first argument your function output `y`\n",
    "- as second argument your target value `y_target`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### **Your turn !**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "x0 = 8\n",
    "lr = 0.01\n",
    "iterations = 10\n",
    "\n",
    "x = torch.Tensor([x0]).requires_grad_()\n",
    "y = f(x)\n",
    "y_target = torch.Tensor([0])\n",
    "\n",
    "# Define your optimizer\n",
    "optimizer =  # < YOUR CODE HERE >\n",
    "loss_function = # < YOUR CODE HERE >\n",
    "\n",
    "for i in range(iterations):\n",
    "    \n",
    "    # < YOUR CODE HERE >\n",
    "    \n",
    "    print(y.data)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Using a learning rate scheduler"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In addition to an optimizer, a learning rate scheduler can be used to adjust the learning rate during training by reducing it according to a pre-defined schedule.  \n",
    "Below are some of the schedulers available in PyTorch."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "optim.lr_scheduler.LambdaLR\n",
    "optim.lr_scheduler.ExponentialLR\n",
    "optim.lr_scheduler.MultiStepLR\n",
    "optim.lr_scheduler.StepLR\n",
    "\n",
    "# and some more ..."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's try `optim.lr_scheduler.ExponentialLR`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def f(x):\n",
    "    return x.abs() * 5"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "x0 = 8\n",
    "lr = 0.5\n",
    "iterations = 150\n",
    "\n",
    "x = torch.Tensor([x0]).requires_grad_()\n",
    "y_target = torch.Tensor([0])\n",
    "\n",
    "optimizer = optim.SGD([x], lr=lr)\n",
    "loss_function = torch.nn.L1Loss()\n",
    "scheduler = optim.lr_scheduler.ExponentialLR(optimizer, 0.8)\n",
    "\n",
    "for i in range(iterations):\n",
    "    optimizer.zero_grad()\n",
    "    y = f(x)\n",
    "    loss = loss_function(y, y_target)\n",
    "    loss.backward()\n",
    "    optimizer.step()\n",
    "    scheduler.step()\n",
    "    print(y.data, \" | lr : \", optimizer.param_groups[0]['lr'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2D Plot - Optimization process"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Below are some live plots to see what actually happens when you optimize a function.  \n",
    "You can play with learning rates, optimizers and also define new functions to optimize !\n",
    "\n",
    "_Note: These are not stricly speaking live plots as it is not possible to do so in colab. We actually create a video of the optimization process instead_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from live_plot import anim_2d"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def function_2d(x):\n",
    "    return x ** 2 / 20 + x.sin().tanh()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "x0 = 8\n",
    "lr = 2\n",
    "iterations = 15\n",
    "points= []\n",
    "\n",
    "x_range = torch.arange(-10, 10, 0.1)\n",
    "x = torch.Tensor([x0]).requires_grad_()\n",
    "optimizer = torch.optim.Adam([x], lr=lr)\n",
    "\n",
    "for i in range(iterations):\n",
    "    optimizer.zero_grad()\n",
    "    f = function_2d(x)\n",
    "    f.backward()\n",
    "    points += [(x.item(), f.item())]\n",
    "    optimizer.step()\n",
    "    \n",
    "anim_2d(x_range, function_2d, points, 400)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "___"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<!--NAVIGATION-->\n",
    "# < [Autograd](3-Autograd.ipynb) | Optimization | [Modules](5-Modules.ipynb) >"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}