python_fm_3.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Working with Python: functions and modules\n",
    "\n",
    "## Session 3: Modules\n",
    "\n",
    "- [Importing modules and libraries](#Importing-modules-and-libraries)\n",
    "- [Python file library](#Python-file-library)\n",
    "- [Exercise 3.1](#Exercise-3.1)\n",
    "- [Using the `csv` module](#Using-the-csv-module)\n",
    "- [Exercise 3.2](#Exercise-3.2)\n",
    "- [Create your own module](#Create-your-own-module)\n",
    "- [Exercise 3.3](#Exercise-3.3)\n",
    "- [Exercise 3.4](#Exercise-3.4)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Importing modules and libraries"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Like other laguages, Python has the ability to import external modules (or libraries) into the current program. These modules may be part of the standard library that is automatically included with the Python installation, they may be extra libraries which you install separately or they may be other Python programs you have written yourself. Whatever the source of the module, they are imported into a program via an **`import`** command.\n",
    "\n",
    "For example, if we wish to access the mathematical constants `pi` and `e` we can use the import keyword to get [the module named `math`](https://docs.python.org/3/library/math.html) and access its contents with the dot notation:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import math\n",
    "print(math.pi, math.e)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Also we can use the `as` keyword to give the module a different name in our code, which can be useful for brevity and avoiding name conflicts:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import math as m\n",
    "print(m.pi, m.e)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Alternatively we can import the separate components using the `from … import` keyword combination:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from math import pi, e\n",
    "print(pi, e)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can import multiple components from a single module, either on one line like as seen above or on separate lines:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from math import pi\n",
    "from math import e"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Listing module contents\n",
    "\n",
    "Using the [method `dir()`](https://docs.python.org/3/library/functions.html?highlight=dir#dir) and passing the module name:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import math\n",
    "dir(math)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "or directly using an instance, like with this String:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "dir(\"mystring\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "or using the object type"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "dir(str)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Getting help from the official Python documentation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The most useful information is online on https://www.python.org/ website and should  be used as a reference guide.\n",
    "\n",
    "- [Python3 documentation](https://docs.python.org/3/) is the starting page with links to tutorials and libraries' documentation for Python 3\n",
    "    - [The Python Tutorial](https://docs.python.org/3/tutorial/index.html)\n",
    "        - [Modules](https://docs.python.org/3/tutorial/modules.html)\n",
    "        - [Brief Tour of the Standard Library: Mathematics](https://docs.python.org/3/tutorial/stdlib.html#mathematics)\n",
    "    - [The Python Standard Library Reference](https://docs.python.org/3/library/index.html) is the reference documentation of all libraries included in Python like:\n",
    "        - [`math` — Mathematical functions](https://docs.python.org/3/library/math.html)\n",
    "        - [`os.path` — Common pathname manipulations](https://docs.python.org/3/library/os.path.html)\n",
    "        - [`os` — Miscellaneous operating system interfaces](https://docs.python.org/3/library/os.html)\n",
    "        - [`csv` — CSV File Reading and Writing](https://docs.python.org/3/library/csv.html)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Python file library"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### [`os.path` — Common pathname manipulations](https://docs.python.org/3/library/os.path.html)\n",
    "\n",
    "- `exists(path)` : returns whether path exists\n",
    "- `isfile(path)` : returns whether path is a “regular” file (as opposed to a directory)\n",
    "- `isdir(path)` : returns whether path is a directory\n",
    "- `islink(path)` : returns whether path is a symbolic link\n",
    "- `join(*paths)` : joins the paths together into one long path\n",
    "- `dirname(path)` : returns directory containing the path\n",
    "- `basename(path)` : returns the path minus the dirname(path) in front\n",
    "- `split(path)` : returns (dirname(path), basename(path))\n",
    "\n",
    "### [`os` — Miscellaneous operating system interfaces](https://docs.python.org/3/library/os.html)\n",
    "\n",
    "- `chdir(path)` : change the current working directory to be path\n",
    "- `getcwd()` : return the current working directory\n",
    "- `listdir(path)` : returns a list of files/directories in the directory path\n",
    "- `mkdir(path)` : create the directory path\n",
    "- `rmdir(path)` : remove the directory path\n",
    "- `remove(path)` : remove the file path\n",
    "- `rename(src, dst)` : move the file/directory from src to dst"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Building the path to your file from a list of directory and filename makes your script able to run on any platforms."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os.path\n",
    "os.path.join(\"data\", \"mydata.txt\")\n",
    "# data/mydata.txt - Unix\n",
    "# data\\mydata.txt - Windows"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Check if a file exists before opening it:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os.path\n",
    "data_file = os.path.join(\"data\", \"mydata.txt\")\n",
    "if os.path.exists(data_file):\n",
    "    print(\"file\", data_file, \"exists\")\n",
    "    with open(data_file) as f:\n",
    "        print(f.read())\n",
    "else:\n",
    "    print(\"file\", data_file, \"not found!\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exercise 3.1\n",
    "\n",
    "Write a script that reads a tab delimited file which has 4 columns: gene, chromosome, start and end coordinates. Check if the file exists, then compute the length of each gene and store its name and corresponding length into a dictionary. Write the results into a new tab separated file. You can find a data file in `data/genes.txt` directory of the course materials."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Using the `csv` module\n",
    "\n",
    "The so-called CSV (Comma Separated Values) format is the most common import and export format for spreadsheets and databases. The csv module implements methods to read and write tabular data in CSV format.\n",
    "\n",
    "The csv module’s `reader()` and `writer()` methods read and write CSV files. You can also read and write data into dictionary form using the `DictReader()` and `DictWriter()` methods.\n",
    "\n",
    "For more information about this built-in Python library about [CSV File Reading and Writing documentation](https://docs.python.org/3/library/csv.html).\n",
    "\n",
    "Let's now read our `data/mydata.txt` space separated file using the `csv` module."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import csv\n",
    "with open(\"data/mydata.txt\") as f:\n",
    "    reader = csv.reader(f, delimiter = \" \") # default delimiter is \",\"\n",
    "    for row in reader:\n",
    "        print(row)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Change the `csv.reader()` by the `csv.DictReader()` and it builds up a dictionary automatically based on the column headers."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "with open(\"data/mydata.txt\") as f:\n",
    "    reader = csv.DictReader(f, delimiter = \" \")\n",
    "    for row in reader:\n",
    "        print(row)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Write a tab delimited file using the csv module\n",
    "import csv\n",
    "\n",
    "mydata = [\n",
    "    ['1', 'Human', '1.076'], \n",
    "    ['2', 'Mouse', '1.202'], \n",
    "    ['3', 'Frog', '2.2362'], \n",
    "    ['4', 'Fly', '0.9853']\n",
    "]\n",
    "\n",
    "with open(\"data.txt\", \"w\") as f:\n",
    "    writer = csv.writer(f, delimiter='\\t' )\n",
    "    writer.writerow( [ \"Index\", \"Organism\", \"Score\" ] ) # write header\n",
    "    for record in mydata:\n",
    "        writer.writerow( record )\n",
    "\n",
    "# Open the output file and print out its content\n",
    "with open(\"data.txt\") as f:\n",
    "    print(f.read())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Write a delimited file using the csv module from a list of dictionaries \n",
    "import csv\n",
    "\n",
    "mydata = [\n",
    "    {'Index': '1', 'Score': '1.076', 'Organism': 'Human'}, \n",
    "    {'Index': '2', 'Score': '1.202', 'Organism': 'Mouse'}, \n",
    "    {'Index': '3', 'Score': '2.2362', 'Organism': 'Frog'}, \n",
    "    {'Index': '4', 'Score': '0.9853', 'Organism': 'Fly'}\n",
    "]\n",
    "\n",
    "with open(\"dict_data.txt\", \"w\") as f:\n",
    "    writer = csv.DictWriter(f, mydata[0].keys(), delimiter='\\t')\n",
    "    writer.writeheader() # write header\n",
    "\n",
    "    for record in mydata:\n",
    "        writer.writerow( record )\n",
    "\n",
    "# Open the output file and print out its content\n",
    "with open(\"dict_data.txt\") as f:\n",
    "    print(f.read())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exercise 3.2\n",
    "\n",
    "Now change the script you wrote for [Exercise 3.1](#Exercise-3.1) to make use of the `csv` module."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Create your own module\n",
    "\n",
    "So far we have been writing Python code in files as executable scripts without knowing that they are also modules from which we are able to call the different functions defined in them.\n",
    "\n",
    "A module is a file containing Python definitions and statements. The file name is the module name with the suffix .py appended. Create a file called `my_first_module.py` in the current directory with the following contents:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def say_hello(user):\n",
    "    print('hello', user, '!')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now enter the Python interpreter from the directory you've created `my_first_module.py` file and import the `say_hello` function from this module with the following command:\n",
    "\n",
    "```bash\n",
    "python3\n",
    "Python 3.5.2 (default, Jun 30 2016, 18:10:25) \n",
    "[GCC 4.2.1 Compatible Apple LLVM 7.0.2 (clang-700.1.81)] on darwin\n",
    "Type \"help\", \"copyright\", \"credits\" or \"license\" for more information.\n",
    ">>> from my_first_module import say_hello\n",
    ">>> say_hello('Anne')\n",
    "hello Anne !\n",
    ">>> \n",
    "```\n",
    "\n",
    "There is one module already stored in the course directory called `my_first_module.py`, if you wish to import it into this notebook, below is what you need to do. If you wish to edit this file and change the code or add another function, you will have to restart the notebook to have these changes taken into account using the restart the kernel button in the menu bar."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from my_first_module import say_hello\n",
    "say_hello('Anne')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A module can contain executable statements as well as function definitions. These statements are intended to initialize the module. They are executed only the first time the module name is encountered in an import statement. \n",
    "They are also run if the file is executed as a script.\n",
    "\n",
    "Do comment out these executable statements if you do not wish to have them executed when importing your module.\n",
    "\n",
    "For more information about modules, https://docs.python.org/3/tutorial/modules.html."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exercise 3.3\n",
    "\n",
    "### Calculate the GC content of a DNA sequence\n",
    "\n",
    "Write a function that calculates the GC content of a DNA sequence.\n",
    "\n",
    "### Extract the list of all overlapping sub-sequences\n",
    "\n",
    "Write a function that extracts a list of overlapping sub-sequences for a given window size from a given sequence. Do not forget to test it on a given DNA sequence."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exercise 3.4\n",
    "### Calculate GC content along the DNA sequence\n",
    "Combine the two methods written above to calculate the GC content of each overlapping sliding window along a DNA sequence from start to end. \n",
    "\n",
    "Import the two methods you wrote above at exercise 3.3, to solve this exercise.\n",
    "\n",
    "The new function should take two arguments, the DNA sequence and the size of the sliding window, and re-use the previous methods written to calculate the GC content of a DNA sequence and to extract the list of all overlapping sub-sequences. It returns a list of GC% along the DNA sequence."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Next session\n",
    "\n",
    "Go to our next notebook: [python_functions_and_modules_4](python_fm_4.ipynb)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}