PY0101EN-4-1-ReadFile.ipynb

{
  "metadata": {
    "kernelspec": {
      "name": "python",
      "display_name": "Pyolite",
      "language": "python"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "python",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.8"
    }
  },
  "nbformat_minor": 4,
  "nbformat": 4,
  "cells": [
    {
      "cell_type": "markdown",
      "source": "<center>\n    <img src=\"https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/IBMDeveloperSkillsNetwork-PY0101EN-SkillsNetwork/IDSNlogo.png\" width=\"300\" alt=\"cognitiveclass.ai logo\"  />\n</center>\n\n# Reading Files Python\n\nEstimated time needed: **40** minutes\n\n## Objectives\n\nAfter completing this lab you will be able to:\n\n*   Read text files using Python libraries\n",
      "metadata": {}
    },
    {
      "cell_type": "markdown",
      "source": "<h2>Table of Contents</h2>\n<div class=\"alert alert-block alert-info\" style=\"margin-top: 20px\">\n    <ul>\n        <li><a href=\"https://download/?utm_medium=Exinfluencer&utm_source=Exinfluencer&utm_content=000026UJ&utm_term=10006555&utm_id=NA-SkillsNetwork-Channel-SkillsNetworkCoursesIBMDeveloperSkillsNetworkPY0101ENSkillsNetwork19487395-2021-01-01\">Download Data</a></li>\n        <li><a href=\"https://read/?utm_medium=Exinfluencer&utm_source=Exinfluencer&utm_content=000026UJ&utm_term=10006555&utm_id=NA-SkillsNetwork-Channel-SkillsNetworkCoursesIBMDeveloperSkillsNetworkPY0101ENSkillsNetwork19487395-2021-01-01\">Reading Text Files</a></li>\n        <li><a href=\"https://better/?utm_medium=Exinfluencer&utm_source=Exinfluencer&utm_content=000026UJ&utm_term=10006555&utm_id=NA-SkillsNetwork-Channel-SkillsNetworkCoursesIBMDeveloperSkillsNetworkPY0101ENSkillsNetwork19487395-2021-01-01\">A Better Way to Open a File</a></li>\n    </ul>\n\n</div>\n\n<hr>\n",
      "metadata": {}
    },
    {
      "cell_type": "markdown",
      "source": "<h2 id=\"download\">Download Data</h2>\n",
      "metadata": {}
    },
    {
      "cell_type": "code",
      "source": "## Uncomment these if working locally, else let the following code cell run.\n\n# import urllib.request\n# url = 'https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/IBMDeveloperSkillsNetwork-PY0101EN-SkillsNetwork/labs/Module%204/data/example1.txt'\n# filename = 'Example1.txt'\n# urllib.request.urlretrieve(url, filename)\n\n## Download Example file\n# !wget -O /resources/data/Example1.txt https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/IBMDeveloperSkillsNetwork-PY0101EN-SkillsNetwork/labs/Module%204/data/example1.txt",
      "metadata": {
        "trusted": true
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": "from pyodide.http import pyfetch\nimport pandas as pd\n\nfilename = \"https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/IBMDeveloperSkillsNetwork-PY0101EN-SkillsNetwork/labs/Module%204/data/example1.txt\"\n\nasync def download(url, filename):\n    response = await pyfetch(url)\n    if response.status == 200:\n        with open(filename, \"wb\") as f:\n            f.write(await response.bytes())\n\n\nawait download(filename, \"Example1.txt\")",
      "metadata": {
        "trusted": true
      },
      "execution_count": 2,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": "<hr>\n",
      "metadata": {}
    },
    {
      "cell_type": "markdown",
      "source": "<h2 id=\"read\">Reading Text Files</h2>\n",
      "metadata": {}
    },
    {
      "cell_type": "markdown",
      "source": "One way to read or write a file in Python is to use the built-in <code>open</code> function. The <code>open</code> function provides a **File object** that contains the methods and attributes you need in order to read, save, and manipulate the file. In this notebook, we will only cover **.txt** files. The first parameter you need is the file path and the file name. An example is shown as follow:\n",
      "metadata": {}
    },
    {
      "cell_type": "markdown",
      "source": "<img src=\"https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/IBMDeveloperSkillsNetwork-PY0101EN-SkillsNetwork/labs/Module%204/images/ReadOpen.png\" width=\"500\" />\n",
      "metadata": {}
    },
    {
      "cell_type": "markdown",
      "source": "The mode argument is optional and the default value is **r**. In this notebook we only cover two modes:\n\n<ul>\n    <li>**r**: Read mode for reading files </li>\n    <li>**w**: Write mode for writing files</li>\n</ul>\n",
      "metadata": {}
    },
    {
      "cell_type": "markdown",
      "source": "For the next example, we will use the text file **Example1.txt**. The file is shown as follows:\n",
      "metadata": {}
    },
    {
      "cell_type": "markdown",
      "source": "<img src=\"https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/IBMDeveloperSkillsNetwork-PY0101EN-SkillsNetwork/labs/Module%204/images/ReadFile.png\" width=\"100\" />\n",
      "metadata": {}
    },
    {
      "cell_type": "markdown",
      "source": "We read the file:\n",
      "metadata": {}
    },
    {
      "cell_type": "code",
      "source": "# Read the Example1.txt\nexample1 = \"Example1.txt\"\nfile1 = open(example1, \"r\")\n",
      "metadata": {
        "trusted": true
      },
      "execution_count": 3,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": "We can view the attributes of the file.\n",
      "metadata": {}
    },
    {
      "cell_type": "markdown",
      "source": "The name of the file:\n",
      "metadata": {}
    },
    {
      "cell_type": "code",
      "source": "# Print the path of file\n\nfile1.name",
      "metadata": {
        "trusted": true
      },
      "execution_count": 4,
      "outputs": [
        {
          "execution_count": 4,
          "output_type": "execute_result",
          "data": {
            "text/plain": "'Example1.txt'"
          },
          "metadata": {}
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": "The mode the file object is in:\n",
      "metadata": {}
    },
    {
      "cell_type": "code",
      "source": "# Print the mode of file, either 'r' or 'w'\n\nfile1.mode",
      "metadata": {
        "trusted": true
      },
      "execution_count": 5,
      "outputs": [
        {
          "execution_count": 5,
          "output_type": "execute_result",
          "data": {
            "text/plain": "'r'"
          },
          "metadata": {}
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": "We can read the file and assign it to a variable :\n",
      "metadata": {}
    },
    {
      "cell_type": "code",
      "source": "# Read the file\n\nFileContent = file1.read()\nFileContent",
      "metadata": {
        "trusted": true
      },
      "execution_count": 6,
      "outputs": [
        {
          "execution_count": 6,
          "output_type": "execute_result",
          "data": {
            "text/plain": "'This is line 1 \\nThis is line 2\\nThis is line 3'"
          },
          "metadata": {}
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": "The **/n** means that there is a new line.\n",
      "metadata": {}
    },
    {
      "cell_type": "markdown",
      "source": "We can print the file:\n",
      "metadata": {}
    },
    {
      "cell_type": "code",
      "source": "# Print the file with '\\n' as a new line\n\nprint(FileContent)",
      "metadata": {
        "trusted": true
      },
      "execution_count": 7,
      "outputs": [
        {
          "name": "stdout",
          "text": "This is line 1 \nThis is line 2\nThis is line 3\n",
          "output_type": "stream"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": "The file is of type string:\n",
      "metadata": {}
    },
    {
      "cell_type": "code",
      "source": "# Type of file content\n\ntype(FileContent)",
      "metadata": {
        "trusted": true
      },
      "execution_count": 8,
      "outputs": [
        {
          "execution_count": 8,
          "output_type": "execute_result",
          "data": {
            "text/plain": "str"
          },
          "metadata": {}
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": "It is very important that the file is closed in the end. This frees up resources and ensures consistency across different python versions.\n",
      "metadata": {}
    },
    {
      "cell_type": "code",
      "source": "# Close file after finish\n\nfile1.close()",
      "metadata": {
        "trusted": true
      },
      "execution_count": 9,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": "<hr>\n",
      "metadata": {}
    },
    {
      "cell_type": "markdown",
      "source": "<h2 id=\"better\">A Better Way to Open a File</h2>\n",
      "metadata": {}
    },
    {
      "cell_type": "markdown",
      "source": "Using the <code>with</code> statement is better practice, it automatically closes the file even if the code encounters an exception. The code will run everything in the indent block then close the file object.\n",
      "metadata": {}
    },
    {
      "cell_type": "code",
      "source": "# Open file using with\n\nwith open(example1, \"r\") as file1:\n    FileContent = file1.read()\n    print(FileContent)",
      "metadata": {
        "trusted": true
      },
      "execution_count": 10,
      "outputs": [
        {
          "name": "stdout",
          "text": "This is line 1 \nThis is line 2\nThis is line 3\n",
          "output_type": "stream"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": "The file object is closed, you can verify it by running the following cell:\n",
      "metadata": {}
    },
    {
      "cell_type": "code",
      "source": "# Verify if the file is closed\n\nfile1.closed",
      "metadata": {
        "trusted": true
      },
      "execution_count": 11,
      "outputs": [
        {
          "execution_count": 11,
          "output_type": "execute_result",
          "data": {
            "text/plain": "True"
          },
          "metadata": {}
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": "We can see the info in the file:\n",
      "metadata": {}
    },
    {
      "cell_type": "code",
      "source": "# See the content of file\n\nprint(FileContent)",
      "metadata": {
        "trusted": true
      },
      "execution_count": 12,
      "outputs": [
        {
          "name": "stdout",
          "text": "This is line 1 \nThis is line 2\nThis is line 3\n",
          "output_type": "stream"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": "The syntax is a little confusing as the file object is after the <code>as</code> statement. We also don’t explicitly close the file. Therefore we summarize the steps in a figure:\n",
      "metadata": {}
    },
    {
      "cell_type": "markdown",
      "source": "<img src=\"https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/IBMDeveloperSkillsNetwork-PY0101EN-SkillsNetwork/labs/Module%204/images/ReadWith.png\" width=\"500\" />\n",
      "metadata": {}
    },
    {
      "cell_type": "markdown",
      "source": "We don’t have to read the entire file, for example, we can read the first 4 characters by entering three as a parameter to the method **.read()**:\n",
      "metadata": {}
    },
    {
      "cell_type": "code",
      "source": "# Read first four characters\n\nwith open(example1, \"r\") as file1:\n    print(file1.read(4))",
      "metadata": {
        "trusted": true
      },
      "execution_count": 14,
      "outputs": [
        {
          "name": "stdout",
          "text": "This\n",
          "output_type": "stream"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": "Once the method <code>.read(4)</code> is called the first 4 characters are called. If we call the method again, the next 4 characters are called. The output for the following cell will demonstrate the process for different inputs to the method <code>read()</code>:\n",
      "metadata": {}
    },
    {
      "cell_type": "code",
      "source": "# Read certain amount of characters\n\nwith open(example1, \"r\") as file1:\n    print(file1.read(4))\n    print(file1.read(4))\n    print(file1.read(7))\n    print(file1.read(15))",
      "metadata": {
        "trusted": true
      },
      "execution_count": 15,
      "outputs": [
        {
          "name": "stdout",
          "text": "This\n is \nline 1 \n\nThis is line 2\n",
          "output_type": "stream"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": "The process is illustrated in the below figure, and each color represents the part of the file read after the method <code>read()</code> is called:\n",
      "metadata": {}
    },
    {
      "cell_type": "markdown",
      "source": "<img src=\"https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/IBMDeveloperSkillsNetwork-PY0101EN-SkillsNetwork/labs/Module%204/images/read.png\" width=\"500\" />\n",
      "metadata": {}
    },
    {
      "cell_type": "markdown",
      "source": "Here is an example using the same file, but instead we read 16, 5, and then 9 characters at a time:\n",
      "metadata": {}
    },
    {
      "cell_type": "code",
      "source": "# Read certain amount of characters\n\nwith open(example1, \"r\") as file1:\n    print(file1.read(16))\n    print(file1.read(5))\n    print(file1.read(9))",
      "metadata": {
        "trusted": true
      },
      "execution_count": 16,
      "outputs": [
        {
          "name": "stdout",
          "text": "This is line 1 \n\nThis \nis line 2\n",
          "output_type": "stream"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": "We can also read one line of the file at a time using the method <code>readline()</code>:\n",
      "metadata": {}
    },
    {
      "cell_type": "code",
      "source": "# Read one line\n\nwith open(example1, \"r\") as file1:\n    print(\"first line: \" + file1.readline())",
      "metadata": {
        "trusted": true
      },
      "execution_count": 17,
      "outputs": [
        {
          "name": "stdout",
          "text": "first line: This is line 1 \n\n",
          "output_type": "stream"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": "We can also pass an argument to <code> readline() </code> to specify the number of charecters we want to read. However, unlike <code> read()</code>, <code> readline()</code> can only read one line at most.\n",
      "metadata": {}
    },
    {
      "cell_type": "code",
      "source": "with open(example1, \"r\") as file1:\n    print(file1.readline(20)) # does not read past the end of line\n    print(file1.read(20)) # Returns the next 20 chars\n",
      "metadata": {
        "trusted": true
      },
      "execution_count": 18,
      "outputs": [
        {
          "name": "stdout",
          "text": "This is line 1 \n\nThis is line 2\nThis \n",
          "output_type": "stream"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": "We can use a loop to iterate through each line:\n",
      "metadata": {}
    },
    {
      "cell_type": "code",
      "source": "# Iterate through the lines\n\nwith open(example1,\"r\") as file1:\n        i = 0;\n        for line in file1:\n            print(\"Iteration\", str(i), \": \", line)\n            i = i + 1",
      "metadata": {
        "trusted": true
      },
      "execution_count": 19,
      "outputs": [
        {
          "name": "stdout",
          "text": "Iteration 0 :  This is line 1 \n\nIteration 1 :  This is line 2\n\nIteration 2 :  This is line 3\n",
          "output_type": "stream"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": "We can use the method <code>readlines()</code> to save the text file to a list:\n",
      "metadata": {}
    },
    {
      "cell_type": "code",
      "source": "# Read all lines and save as a list\n\nwith open(example1, \"r\") as file1:\n    FileasList = file1.readlines()",
      "metadata": {
        "trusted": true
      },
      "execution_count": 20,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": "Each element of the list corresponds to a line of text:\n",
      "metadata": {}
    },
    {
      "cell_type": "code",
      "source": "# Print the first line\n\nFileasList[0]",
      "metadata": {
        "trusted": true
      },
      "execution_count": 21,
      "outputs": [
        {
          "execution_count": 21,
          "output_type": "execute_result",
          "data": {
            "text/plain": "'This is line 1 \\n'"
          },
          "metadata": {}
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": "# Print the second line\n\nFileasList\\[1]\n",
      "metadata": {}
    },
    {
      "cell_type": "code",
      "source": "# Print the third line\n\nFileasList[2]",
      "metadata": {
        "trusted": true
      },
      "execution_count": 22,
      "outputs": [
        {
          "execution_count": 22,
          "output_type": "execute_result",
          "data": {
            "text/plain": "'This is line 3'"
          },
          "metadata": {}
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": "<hr>\n<h2>The last exercise!</h2>\n<p>Congratulations, you have completed your first lesson and hands-on lab in Python.\n<hr>\n",
      "metadata": {}
    },
    {
      "cell_type": "markdown",
      "source": "## Author\n\n<a href=\"https://www.linkedin.com/in/joseph-s-50398b136/?utm_medium=Exinfluencer&utm_source=Exinfluencer&utm_content=000026UJ&utm_term=10006555&utm_id=NA-SkillsNetwork-Channel-SkillsNetworkCoursesIBMDeveloperSkillsNetworkPY0101ENSkillsNetwork19487395-2021-01-01\" target=\"_blank\">Joseph Santarcangelo</a>\n\n## Other contributors\n\n<a href=\"https://www.linkedin.com/in/jiahui-mavis-zhou-a4537814a?utm_medium=Exinfluencer&utm_source=Exinfluencer&utm_content=000026UJ&utm_term=10006555&utm_id=NA-SkillsNetwork-Channel-SkillsNetworkCoursesIBMDeveloperSkillsNetworkPY0101ENSkillsNetwork19487395-2021-01-01\">Mavis Zhou</a>\n\n## Change Log\n\n| Date (YYYY-MM-DD) | Version | Changed By    | Change Description                                        |\n| ----------------- | ------- | ------------- | --------------------------------------------------------- |\n| 2022-01-10        | 2.1     | Malika        | Removed the readme for GitShare                           |\n| 2020-09-30        | 1.3     | Malika        | Deleted exericse \"Weather Data\"                           |\n| 2020-09-30        | 1.2     | Malika Singla | Weather Data dataset link added                           |\n| 2020-09-30        | 1.1     | Arjun Swani   | Added exericse \"Weather Data\"                             |\n| 2020-09-30        | 1.0     | Arjun Swani   | Added blurbs about closing files and read() vs readline() |\n| 2020-08-26        | 0.2     | Lavanya       | Moved lab to course repo in GitLab                        |\n|                   |         |               |                                                           |\n|                   |         |               |                                                           |\n\n<hr/>\n\n## <h3 align=\"center\"> © IBM Corporation 2020. All rights reserved. <h3/>\n",
      "metadata": {}
    }
  ]
}