image_project.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# DATA20001 Deep Learning - Group Project\n",
    "## Image project\n",
    "\n",
    "**Due Wednesday December 13, before 23:59.**\n",
    "\n",
    "The task is to learn to assign the correct labels to a set of images.  The images are originally from a photo-sharing site and released under Creative Commons-licenses allowing sharing.  The training set contains 20 000 images. We have resized them and cropped them to 128x128 to make the task a bit more manageable.\n",
    "\n",
    "We're only giving you the code for downloading the data. The rest you'll have to do yourselves.\n",
    "\n",
    "Some comments and hints particular to the image project:\n",
    "\n",
    "- One image may belong to many classes in this problem, i.e., it's a multi-label classification problem. In fact there are images that don't belong to any of our classes, and you should also be able to handle these correctly. Pay careful attention to how you design the outputs of the network (e.g., what activation to use) and what loss function should be used.\n",
    "\n",
    "- As the dataset is pretty imbalanced, don't focus too strictly on the outputs being probabilistic. (Meaning that the right threshold for selecting the label might not be 0.5.)\n",
    "\n",
    "- Image files can be loaded as numpy matrices for example using `imageio.imread`. Most images are color, but a few grayscale. You need to handle the grayscale ones somehow as they would have a different number of color channels (depth) than the color ones.\n",
    "\n",
    "- Loading all the images into one big matrix as we have done in the exercises is not feasible (e.g. the virtual servers in CSC have only 3 GB of RAM). You need to load the images in smaller chunks for the training. This shouldn't be a problem we are doing mini-batch training anyway, and thus we don't need to keep all the images in memory. You can simply pass you current chunk of images to `model.fit()` as it remembers the weights from the previous run.\n",
    "\n",
    "- You need to think carefully about how you load the annotations and match them up with the corresponding images, especially as you are loading them in smaller chunks."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Download the data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from keras.utils.data_utils import get_file\n",
    "\n",
    "database_path = 'train/'\n",
    "\n",
    "dl_file='dl2017-image-proj.zip'\n",
    "dl_url='https://www.cs.helsinki.fi/u/mvsjober/misc/'\n",
    "get_file(dl_file, dl_url+dl_file, cache_dir='./', cache_subdir=database_path, extract=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The above command downloaded and extracted the data files into the `train` subdirectory.\n",
    "\n",
    "The images can be found in `train/images`, and are named as `im1.jpg`, `im2.jpg` and so on until `im20000.jpg`.\n",
    "\n",
    "The class labels, or annotations, can be found in `train/annotations` as `CLASSNAME.txt`, where CLASSNAME is one of the fourteen classes: *baby, bird, car, clouds, dog, female, flower, male, night, people, portrait, river, sea,* and *tree*.\n",
    "\n",
    "Each annotation file is a simple text file that lists the images that depict that class, one per line. The images are listed with their number, not the full filename. For example `5969` refers to the image `im5969.jpg`."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Your stuff goes here ..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Here goes your code ..."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Save your model\n",
    "\n",
    "It might be useful to save your model if you want to continue your work later, or use it for inference later."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "model.save('model.h5')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The model file should now be visible in the \"Home\" screen of the jupyter notebooks interface.  There you should be able to select it and press \"download\"."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Download test set\n",
    "\n",
    "The testset (images only) can now be downloaded with the following commands:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from keras.utils.data_utils import get_file\n",
    "\n",
    "testset_path = 'test/'\n",
    "\n",
    "dl_file='dl2017-image-test.zip'\n",
    "dl_url='https://www.cs.helsinki.fi/u/mvsjober/misc/'\n",
    "get_file(dl_file, dl_url+dl_file, cache_dir='./', cache_subdir=testset_path, extract=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Predict for test set\n",
    "\n",
    "You should return your predictions for the test set in a plain text file.  The text file contains one row for each test set image.  Each row contains a binary prediction for each label (separated by a single space), 1 if it's present in the image, and 0 if not. The order of the labels is as follows (alphabetic order of the label names):\n",
    "\n",
    "    baby bird car clouds dog female flower male night people portrait river sea tree\n",
    "\n",
    "An example row could like like this if your system predicts the presense of a bird and clouds:\n",
    "\n",
    "    0 1 0 1 0 0 0 0 0 0 0 0 0 0\n",
    "    \n",
    "The order of the rows should be according to the numeric order of the image numbers.  In the test set, this means that the first row refers to image `im20001.jpg`, the second to `im20002.jpg`, and so on."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If you have the prediction output matrix prepared in `y` (e.g., by calling `y=model.predict(x_test)`) you can use the following function to save it to a text file."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "np.savetxt('results.txt', y, fmt='%d')"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}