Write-and-Save-Files-in-Python.ipynb

{
  "metadata": {
    "interpreter": {
      "hash": "aee8b7b246df8f9039afb4144a1f6fd8d2ca17a180786b69acc140d282b71a49"
    },
    "kernelspec": {
      "display_name": "Python 3.9.4 64-bit",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.9.4"
    }
  },
  "nbformat_minor": 4,
  "nbformat": 4,
  "cells": [
    {
      "cell_type": "markdown",
      "source": "<p style=\"text-align:center\">\n    <a href=\"https://skills.network/?utm_medium=Exinfluencer&utm_source=Exinfluencer&utm_content=000026UJ&utm_term=10006555&utm_id=NA-SkillsNetwork-Channel-SkillsNetworkCoursesIBMDeveloperSkillsNetworkPY0101ENSkillsNetwork19487395-2022-01-01\" target=\"_blank\">\n    <img src=\"https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/assets/logos/SN_web_lightmode.png\" width=\"200\" alt=\"Skills Network Logo\"  />\n    </a>\n</p>\n\n# Write and Save Files in Python\n\nEstimated time needed: **25** minutes\n\n## Objectives\n\nAfter completing this lab you will be able to:\n\n*   Write to files using Python libraries\n",
      "metadata": {}
    },
    {
      "cell_type": "markdown",
      "source": "<h2>Table of Contents</h2>\n<div class=\"alert alert-block alert-info\" style=\"margin-top: 20px\">\n    <ul>\n        <li><a href=\"https://write/?utm_medium=Exinfluencer&utm_source=Exinfluencer&utm_content=000026UJ&utm_term=10006555&utm_id=NA-SkillsNetwork-Channel-SkillsNetworkCoursesIBMDeveloperSkillsNetworkPY0101ENSkillsNetwork19487395-2022-01-01\">Writing Files</a></li>\n        <li><a href=\"https://append/?utm_medium=Exinfluencer&utm_source=Exinfluencer&utm_content=000026UJ&utm_term=10006555&utm_id=NA-SkillsNetwork-Channel-SkillsNetworkCoursesIBMDeveloperSkillsNetworkPY0101ENSkillsNetwork19487395-2022-01-01\">Appending Files</a></li>\n        <li><a href=\"https://add/?utm_medium=Exinfluencer&utm_source=Exinfluencer&utm_content=000026UJ&utm_term=10006555&utm_id=NA-SkillsNetwork-Channel-SkillsNetworkCoursesIBMDeveloperSkillsNetworkPY0101ENSkillsNetwork19487395-2022-01-01\">Additional File modes</a></li>\n        <li><a href=\"https://copy/?utm_medium=Exinfluencer&utm_source=Exinfluencer&utm_content=000026UJ&utm_term=10006555&utm_id=NA-SkillsNetwork-Channel-SkillsNetworkCoursesIBMDeveloperSkillsNetworkPY0101ENSkillsNetwork19487395-2022-01-01\">Copy a File</a></li>\n    </ul>\n\n</div>\n\n<hr>\n",
      "metadata": {}
    },
    {
      "cell_type": "markdown",
      "source": "<h2 id=\"write\">Writing Files</h2>\n",
      "metadata": {}
    },
    {
      "cell_type": "markdown",
      "source": "We can open a file object using the method <code>write()</code> to save the text file to a list. To write to a file, the mode argument must be set to **w**. Let’s write a file **Example2.txt** with the line: **“This is line A”**\n",
      "metadata": {}
    },
    {
      "cell_type": "code",
      "source": "# Write line to file\nexmp2 = 'Example2.txt'\nwith open(exmp2, 'w') as writefile:\n    writefile.write(\"This is line A\")",
      "metadata": {},
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": "We can read the file to see if it worked:\n",
      "metadata": {}
    },
    {
      "cell_type": "code",
      "source": "# Read file\n\nwith open(exmp2, 'r') as testwritefile:\n    print(testwritefile.read())",
      "metadata": {},
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": "We can write multiple lines:\n",
      "metadata": {}
    },
    {
      "cell_type": "code",
      "source": "# Write lines to file\n\nwith open(exmp2, 'w') as writefile:\n    writefile.write(\"This is line A\\n\")\n    writefile.write(\"This is line B\\n\")",
      "metadata": {},
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": "The method <code>.write()</code> works similar to the method <code>.readline()</code>, except instead of reading a new line it writes a new line. The process is illustrated in the figure. The different colour coding of the grid represents a new line added to the file after each method call.\n",
      "metadata": {}
    },
    {
      "cell_type": "markdown",
      "source": "<img src=\"https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/IBMDeveloperSkillsNetwork-PY0101EN-SkillsNetwork/labs/Module%204/images/WriteLine.png\" width=\"500\" />\n",
      "metadata": {}
    },
    {
      "cell_type": "markdown",
      "source": "You can check the file to see if your results are correct\n",
      "metadata": {}
    },
    {
      "cell_type": "code",
      "source": "# Check whether write to file\n\nwith open(exmp2, 'r') as testwritefile:\n    print(testwritefile.read())",
      "metadata": {},
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": "We write a list to a **.txt** file  as follows:\n",
      "metadata": {}
    },
    {
      "cell_type": "code",
      "source": "# Sample list of text\n\nLines = [\"This is line A\\n\", \"This is line B\\n\", \"This is line C\\n\"]\nLines",
      "metadata": {},
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": "# Write the strings in the list to text file\n\nwith open('Example2.txt', 'w') as writefile:\n    for line in Lines:\n        print(line)\n        writefile.write(line)",
      "metadata": {},
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": "We can verify the file is written by reading it and printing out the values:\n",
      "metadata": {}
    },
    {
      "cell_type": "code",
      "source": "# Verify if writing to file is successfully executed\n\nwith open('Example2.txt', 'r') as testwritefile:\n    print(testwritefile.read())",
      "metadata": {},
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": "However, note that setting the mode to **w** overwrites all the existing data in the file.\n",
      "metadata": {}
    },
    {
      "cell_type": "code",
      "source": "with open('Example2.txt', 'w') as writefile:\n    writefile.write(\"Overwrite\\n\")\nwith open('Example2.txt', 'r') as testwritefile:\n    print(testwritefile.read())",
      "metadata": {},
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": "<hr>\n<h2 id=\"Append\">Appending Files</h2>\n",
      "metadata": {}
    },
    {
      "cell_type": "markdown",
      "source": "We can write to files without losing any of the existing data as follows by setting the mode argument to append: **a**.  you can append a new line as follows:\n",
      "metadata": {}
    },
    {
      "cell_type": "code",
      "source": "# Write a new line to text file\n\nwith open('Example2.txt', 'a') as testwritefile:\n    testwritefile.write(\"This is line C\\n\")\n    testwritefile.write(\"This is line D\\n\")\n    testwritefile.write(\"This is line E\\n\")",
      "metadata": {},
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": "You can verify the file has changed by running the following cell:\n",
      "metadata": {}
    },
    {
      "cell_type": "code",
      "source": "# Verify if the new line is in the text file\n\nwith open('Example2.txt', 'r') as testwritefile:\n    print(testwritefile.read())",
      "metadata": {},
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": "<hr>\n<h2 id=\"add\">Additional modes</h2> \n",
      "metadata": {}
    },
    {
      "cell_type": "markdown",
      "source": "It's fairly ineffecient to open the file in **a** or **w** and then reopening it in **r** to read any lines. Luckily we can access the file in the following modes:\n\n*   **r+** : Reading and writing. Cannot truncate the file.\n*   **w+** : Writing and reading. Truncates the file.\n*   **a+** : Appending and Reading. Creates a new file, if none exists.\n\nYou dont have to dwell on the specifics of each mode for this lab.\n",
      "metadata": {}
    },
    {
      "cell_type": "markdown",
      "source": "Let's try out the **a+** mode:\n",
      "metadata": {}
    },
    {
      "cell_type": "code",
      "source": "with open('Example2.txt', 'a+') as testwritefile:\n    testwritefile.write(\"This is line E\\n\")\n    print(testwritefile.read())",
      "metadata": {},
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": "There were no errors but <code>read()</code> also did not output anything. This is because of our location in the file.\n",
      "metadata": {}
    },
    {
      "cell_type": "markdown",
      "source": "Most of the file methods we've looked at work in a certain location in the file. <code>.write() </code> writes at a certain location in the file. <code>.read()</code> reads at a certain location in the file and so on. You can think of this as moving your pointer around in the notepad to make changes at specific location.\n",
      "metadata": {}
    },
    {
      "cell_type": "markdown",
      "source": "Opening the file in **w** is akin to opening the .txt file, moving your cursor to the beginning of the text file, writing new text and deleting everything that follows.\nWhereas opening the file in **a** is similiar to opening the .txt file, moving your cursor to the very end and then adding the new pieces of text. <br>\nIt is often very useful to know where the 'cursor' is in a file and be able to control it. The following methods allow us to do precisely this -\n\n*   <code>.tell()</code> - returns the current position in bytes\n*   <code>.seek(offset,from)</code> - changes the position by 'offset' bytes with respect to 'from'. From can take the value of 0,1,2 corresponding to beginning, relative to current position and end\n",
      "metadata": {}
    },
    {
      "cell_type": "markdown",
      "source": "Now lets revisit **a+**\n",
      "metadata": {}
    },
    {
      "cell_type": "code",
      "source": "with open('Example2.txt', 'a+') as testwritefile:\n    print(\"Initial Location: {}\".format(testwritefile.tell()))\n    \n    data = testwritefile.read()\n    if (not data):  #empty strings return false in python\n            print('Read nothing') \n    else: \n            print(testwritefile.read())\n            \n    testwritefile.seek(0,0) # move 0 bytes from beginning.\n    \n    print(\"\\nNew Location : {}\".format(testwritefile.tell()))\n    data = testwritefile.read()\n    if (not data): \n            print('Read nothing') \n    else: \n            print(data)\n    \n    print(\"Location after read: {}\".format(testwritefile.tell()) )",
      "metadata": {},
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": "Finally, a note on the difference between **w+** and **r+**. Both of these modes allow access to read and write methods, however, opening a file in **w+** overwrites it and deletes all pre-existing data. <br>\nTo work with a file on existing data, use **r+** and **a+**. While using **r+**, it can be useful to add a <code>.truncate()</code> method at the end of your data. This will reduce the file to your data and delete everything that follows. <br>\nIn the following code block, Run the code as it is first and then run it with the <code>.truncate()</code>.\n",
      "metadata": {}
    },
    {
      "cell_type": "code",
      "source": "with open('Example2.txt', 'r+') as testwritefile:\n    data = testwritefile.readlines()\n    testwritefile.seek(0,0) #write at beginning of file\n   \n    testwritefile.write(\"Line 1\" + \"\\n\")\n    testwritefile.write(\"Line 2\" + \"\\n\")\n    testwritefile.write(\"Line 3\" + \"\\n\")\n    testwritefile.write(\"finished\\n\")\n    #Uncomment the line below\n    #testwritefile.truncate()\n    testwritefile.seek(0,0)\n    print(testwritefile.read())\n    ",
      "metadata": {},
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": "<hr>\n",
      "metadata": {}
    },
    {
      "cell_type": "markdown",
      "source": "<h2 id=\"copy\">Copy a File</h2> \n",
      "metadata": {}
    },
    {
      "cell_type": "markdown",
      "source": "Let's copy the file **Example2.txt** to the file **Example3.txt**:\n",
      "metadata": {}
    },
    {
      "cell_type": "code",
      "source": "# Copy file to another\n\nwith open('Example2.txt','r') as readfile:\n    with open('Example3.txt','w') as writefile:\n          for line in readfile:\n                writefile.write(line)",
      "metadata": {},
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": "We can read the file to see if everything works:\n",
      "metadata": {}
    },
    {
      "cell_type": "code",
      "source": "# Verify if the copy is successfully executed\n\nwith open('Example3.txt','r') as testwritefile:\n    print(testwritefile.read())",
      "metadata": {},
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": "After reading files, we can also write data into files and save them in different file formats like **.txt, .csv, .xls (for excel files) etc**. You will come across these in further examples\n",
      "metadata": {}
    },
    {
      "cell_type": "markdown",
      "source": "**NOTE:** If you wish to open and view the `example3.txt` file, download this lab [here](https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/IBMDeveloperSkillsNetwork-PY0101EN-SkillsNetwork/labs/Module%204/PY0101EN-4-2-WriteFile.ipynb) and run it locally on your machine. Then go to the working directory to ensure the `example3.txt` file exists and contains the summary data that we wrote.\n",
      "metadata": {}
    },
    {
      "cell_type": "markdown",
      "source": "<hr>\n",
      "metadata": {}
    },
    {
      "cell_type": "markdown",
      "source": "<h2> Exercise </h2>\n",
      "metadata": {}
    },
    {
      "cell_type": "markdown",
      "source": "Your local university's Raptors fan club maintains a register of its active members on a .txt document. Every month they update the file by removing the members who are not active. You have been tasked with automating this with your Python skills. <br>\nGiven the file `currentMem`, Remove each member with a 'no' in their Active column. Keep track of each of the removed members and append them to the `exMem` file. Make sure that the format of the original files in preserved.   (*Hint: Do this by reading/writing whole lines and ensuring the header remains* ) <br>\nRun the code block below prior to starting the exercise. The skeleton code has been provided for you. Edit only the `cleanFiles` function.\n",
      "metadata": {}
    },
    {
      "cell_type": "code",
      "source": "#Run this prior to starting the exercise\nfrom random import randint as rnd\n\nmemReg = 'members.txt'\nexReg = 'inactive.txt'\nfee =('yes','no')\n\ndef genFiles(current,old):\n    with open(current,'w+') as writefile: \n        writefile.write('Membership No  Date Joined  Active  \\n')\n        data = \"{:^13}  {:<11}  {:<6}\\n\"\n\n        for rowno in range(20):\n            date = str(rnd(2015,2020))+ '-' + str(rnd(1,12))+'-'+str(rnd(1,25))\n            writefile.write(data.format(rnd(10000,99999),date,fee[rnd(0,1)]))\n\n\n    with open(old,'w+') as writefile: \n        writefile.write('Membership No  Date Joined  Active  \\n')\n        data = \"{:^13}  {:<11}  {:<6}\\n\"\n        for rowno in range(3):\n            date = str(rnd(2015,2020))+ '-' + str(rnd(1,12))+'-'+str(rnd(1,25))\n            writefile.write(data.format(rnd(10000,99999),date,fee[1]))\n\n\ngenFiles(memReg,exReg)\n",
      "metadata": {},
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": "Now that you've run the prerequisite code cell above, which prepared the files for this exercise, you are ready to move on to the implementation.\n\n#### **Exercise:** Implement the cleanFiles function in the code cell below.\n",
      "metadata": {}
    },
    {
      "cell_type": "code",
      "source": "  \n'''\nThe two arguments for this function are the files:\n    - currentMem: File containing list of current members\n    - exMem: File containing list of old members\n    \n    This function should remove all rows from currentMem containing 'no' \n    in the 'Active' column and appends them to exMem.\n    '''\ndef cleanFiles(currentMem, exMem):\n    # TODO: Open the currentMem file as in r+ mode\n        #TODO: Open the exMem file in a+ mode\n\n        #TODO: Read each member in the currentMem (1 member per row) file into a list.\n        # Hint: Recall that the first line in the file is the header.\n\n        #TODO: iterate through the members and create a new list of the innactive members\n\n        # Go to the beginning of the currentMem file\n        # TODO: Iterate through the members list. \n        # If a member is inactive, add them to exMem, otherwise write them into currentMem\n\n        \n    \n    pass # Remove this line when done implementation\n\n\n# The code below is to help you view the files.\n# Do not modify this code for this exercise.\nmemReg = 'members.txt'\nexReg = 'inactive.txt'\ncleanFiles(memReg,exReg)\n\n\nheaders = \"Membership No  Date Joined  Active  \\n\"\nwith open(memReg,'r') as readFile:\n    print(\"Active Members: \\n\\n\")\n    print(readFile.read())\n    \nwith open(exReg,'r') as readFile:\n    print(\"Inactive Members: \\n\\n\")\n    print(readFile.read())\n                \n    ",
      "metadata": {},
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": "The code cell below is to verify your solution. Please do not modify the code and run it to test your implementation of `cleanFiles`.\n",
      "metadata": {}
    },
    {
      "cell_type": "code",
      "source": "def testMsg(passed):\n    if passed:\n       return 'Test Passed'\n    else :\n       return 'Test Failed'\n\ntestWrite = \"testWrite.txt\"\ntestAppend = \"testAppend.txt\" \npassed = True\n\ngenFiles(testWrite,testAppend)\n\nwith open(testWrite,'r') as file:\n    ogWrite = file.readlines()\n\nwith open(testAppend,'r') as file:\n    ogAppend = file.readlines()\n\ntry:\n    cleanFiles(testWrite,testAppend)\nexcept:\n    print('Error')\n\nwith open(testWrite,'r') as file:\n    clWrite = file.readlines()\n\nwith open(testAppend,'r') as file:\n    clAppend = file.readlines()\n        \n# checking if total no of rows is same, including headers\n\nif (len(ogWrite) + len(ogAppend) != len(clWrite) + len(clAppend)):\n    print(\"The number of rows do not add up. Make sure your final files have the same header and format.\")\n    passed = False\n    \nfor line in clWrite:\n    if  'no' in line:\n        passed = False\n        print(\"Inactive members in file\")\n        break\n    else:\n        if line not in ogWrite:\n            print(\"Data in file does not match original file\")\n            passed = False\nprint (\"{}\".format(testMsg(passed)))\n    \n\n",
      "metadata": {},
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": "<details><summary>Click here for the solution</summary>\n\n```python\ndef cleanFiles(currentMem,exMem):\n    with open(currentMem,'r+') as writeFile: \n        with open(exMem,'a+') as appendFile:\n            #get the data\n            writeFile.seek(0)\n            members = writeFile.readlines()\n            #remove header\n            header = members[0]\n            members.pop(0)\n                \n            inactive = [member for member in members if ('no' in member)]\n            '''\n            The above is the same as \n\n            for member in members:\n            if 'no' in member:\n                inactive.append(member)\n            '''\n            #go to the beginning of the write file\n            writeFile.seek(0) \n            writeFile.write(header)\n            for member in members:\n                if (member in inactive):\n                    appendFile.write(member)\n                else:\n                    writeFile.write(member)      \n            writeFile.truncate()\n                \nmemReg = 'members.txt'\nexReg = 'inactive.txt'\ncleanFiles(memReg,exReg)\n\n# code to help you see the files\n\nheaders = \"Membership No  Date Joined  Active  \\n\"\n\nwith open(memReg,'r') as readFile:\n    print(\"Active Members: \\n\\n\")\n    print(readFile.read())\n    \nwith open(exReg,'r') as readFile:\n    print(\"Inactive Members: \\n\\n\")\n    print(readFile.read())\n    \n```\n\n</details>\n",
      "metadata": {}
    },
    {
      "cell_type": "markdown",
      "source": "<hr>\n<h2>The last exercise!</h2>\n<p>Congratulations, you have completed this lesson and hands-on lab in Python. \n<hr>\n",
      "metadata": {}
    },
    {
      "cell_type": "markdown",
      "source": "## Author\n\n<a href=\"https://www.linkedin.com/in/joseph-s-50398b136/?utm_medium=Exinfluencer&utm_source=Exinfluencer&utm_content=000026UJ&utm_term=10006555&utm_id=NA-SkillsNetwork-Channel-SkillsNetworkCoursesIBMDeveloperSkillsNetworkPY0101ENSkillsNetwork19487395-2022-01-01\" target=\"_blank\">Joseph Santarcangelo</a>\n\n### Other Contributors\n\n<a href=\"https://www.linkedin.com/in/jiahui-mavis-zhou-a4537814a?utm_medium=Exinfluencer&utm_source=Exinfluencer&utm_content=000026UJ&utm_term=10006555&utm_id=NA-SkillsNetwork-Channel-SkillsNetworkCoursesIBMDeveloperSkillsNetworkPY0101ENSkillsNetwork19487395-2022-01-01\">Mavis Zhou</a>\n\n## Change Log\n\n| Date (YYYY-MM-DD) | Version | Changed By  | Change Description                  |\n| ----------------- | ------- | ----------- | ----------------------------------- |\n| 2022-01-10        | 2.1     | Malika      | Removed the readme for GitShare     |\n| 2020-10-16        | 1.3     | Arjun Swani | Added exercise                      |\n| 2020-10-16        | 1.2     | Arjun Swani | Added section additional file modes |\n| 2020-10-16        | 1.1     | Arjun Swani | Made append a different section     |\n| 2020-08-28        | 0.2     | Lavanya     | Moved lab to course repo in GitLab  |\n\n<hr>\n\n## <h3 align=\"center\"> © IBM Corporation 2020. All rights reserved. <h3/>\n",
      "metadata": {}
    }
  ]
}