Selecting-data-in-a-Dataframe-Pandas.ipynb

{
  "metadata": {
    "anaconda-cloud": {},
    "kernelspec": {
      "display_name": "Python 3 (ipykernel)",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.9.7"
    }
  },
  "nbformat_minor": 4,
  "nbformat": 4,
  "cells": [
    {
      "cell_type": "markdown",
      "source": "<p style=\"text-align:center\">\n    <a href=\"https://skills.network/?utm_medium=Exinfluencer&utm_source=Exinfluencer&utm_content=000026UJ&utm_term=10006555&utm_id=NA-SkillsNetwork-Channel-SkillsNetworkCoursesIBMDeveloperSkillsNetworkPY0101ENSkillsNetwork19487395-2022-01-01\" target=\"_blank\">\n    <img src=\"https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/assets/logos/SN_web_lightmode.png\" width=\"200\" alt=\"Skills Network Logo\"  />\n    </a>\n</p>\n\n# Practice Lab: Selecting data in a Dataframe\n\nEstimated time needed: **15** minutes\n\n## Objectives\n\nAfter completing this lab you will be able to:\n\n*   Use Pandas Library to create DataFrame and Series\n*   Locate data in the DataFrame using <code>loc()</code> and <code>iloc()</code> functions\n*   Use slicing\n",
      "metadata": {}
    },
    {
      "cell_type": "markdown",
      "source": "### Exercise 1: Pandas: DataFrame and Series\n\n**Pandas** is a popular library for data analysis built on top of the Python programming language. Pandas generally provide two data structures for manipulating data, They are:\n\n*   DataFrame\n*   Series\n\nA **DataFrame** is a two-dimensional data structure, i.e., data is aligned in a tabular fashion in rows and columns.\n\n*   A Pandas DataFrame will be created by loading the datasets from existing storage.\n*   Storage can be SQL Database, CSV file, an Excel file, etc.\n*   It can also be created from the lists, dictionary, and from a list of dictionaries.\n\n**Series** represents a one-dimensional array of indexed data.\nIt has two main components :\n\n1.  An array of actual data.\n2.  An associated array of indexes or data labels.\n\nThe index is used to access individual data values. You can also get a column of a dataframe as a **Series**. You can think of a Pandas series as a 1-D dataframe.\n",
      "metadata": {}
    },
    {
      "cell_type": "code",
      "source": "# let us import the Pandas Library\nimport pandas as pd",
      "metadata": {},
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": "Once you’ve imported pandas, you can then use the functions built in it to create and analyze data.\n",
      "metadata": {}
    },
    {
      "cell_type": "markdown",
      "source": "**In this practice lab, we will learn how to create a DataFrame out of a dictionary.**\n",
      "metadata": {}
    },
    {
      "cell_type": "markdown",
      "source": "Let us consider a dictionary 'x' with keys and values as shown below.\n\nWe then create a dataframe from the dictionary using the function pd.DataFrame(dict)\n",
      "metadata": {}
    },
    {
      "cell_type": "code",
      "source": "#Define a dictionary 'x'\n\nx = {'Name': ['Rose','John', 'Jane', 'Mary'], 'ID': [1, 2, 3, 4], 'Department': ['Architect Group', 'Software Group', 'Design Team', 'Infrastructure'], \n      'Salary':[100000, 80000, 50000, 60000]}\n\n#casting the dictionary to a DataFrame\ndf = pd.DataFrame(x)\n\n#display the result df\ndf",
      "metadata": {},
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": "We can see the direct correspondence between the table. The keys correspond to the column labels and the values or lists corresponding to the rows.\n",
      "metadata": {}
    },
    {
      "cell_type": "markdown",
      "source": "#### Column Selection:\n\nTo select a column in Pandas DataFrame, we can either access the columns by calling them by their columns name.\n\nLet's Retrieve the data present in the <code>ID</code> column.\n",
      "metadata": {}
    },
    {
      "cell_type": "code",
      "source": "#Retrieving the \"ID\" column and assigning it to a variable x\nx = df[['ID']]\nx",
      "metadata": {
        "scrolled": true
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": "Let's use the <code>type()</code> function and check the type of the variable.\n",
      "metadata": {}
    },
    {
      "cell_type": "code",
      "source": "#check the type of x\ntype(x)",
      "metadata": {},
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": "The output shows us that the type of the variable is a DataFrame object.\n",
      "metadata": {}
    },
    {
      "cell_type": "markdown",
      "source": "#### Access to multiple columns\n\nLet us retrieve the data for <code>Department</code>, <code>Salary</code> and <code>ID</code> columns\n",
      "metadata": {}
    },
    {
      "cell_type": "code",
      "source": "#Retrieving the Department, Salary and ID columns and assigning it to a variable z\n\nz = df[['Department','Salary','ID']]\nz",
      "metadata": {},
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": "### Try it yourself\n",
      "metadata": {}
    },
    {
      "cell_type": "markdown",
      "source": "##### Problem 1: Create a dataframe to display the result as below:\n",
      "metadata": {}
    },
    {
      "cell_type": "markdown",
      "source": "<center>\n    <img src=\"https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/IBMDeveloperSkillsNetwork-PY0101EN-SkillsNetwork/labs/Module%204/images/Student_data.png\" width=\"300\" alt=\"Student Data\"  />\n</center>\n",
      "metadata": {}
    },
    {
      "cell_type": "code",
      "source": "#write your code here\n",
      "metadata": {},
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": "<details><summary>Click here for the solution</summary>\n\n```python\na = {'Student':['David', 'Samuel', 'Terry', 'Evan'],\n     'Age':['27', '24', '22', '32'],\n     'Country':['UK', 'Canada', 'China', 'USA'],\n     'Course':['Python','Data Structures','Machine Learning','Web Development'],\n     'Marks':['85','72','89','76']}\ndf1 = pd.DataFrame(a)\ndf1\n    \n```\n\n</details>\n",
      "metadata": {}
    },
    {
      "cell_type": "markdown",
      "source": "##### Problem 2: Retrieve the Marks column and assign it to a variable b\n",
      "metadata": {}
    },
    {
      "cell_type": "code",
      "source": "#write your code here\n",
      "metadata": {},
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": "<details><summary>Click here for the solution</summary>\n\n```python\nb = df1[['Marks']]\nb\n    \n```\n\n</details>\n",
      "metadata": {}
    },
    {
      "cell_type": "markdown",
      "source": "##### Problem 3: Retrieve the Country and Course columns and assign it to a variable c\n",
      "metadata": {}
    },
    {
      "cell_type": "code",
      "source": "#write your code here\n",
      "metadata": {},
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": "<details><summary>Click here for the solution</summary>\n\n```python\nc = df1[['Country','Course']]\nc\n    \n```\n\n</details>\n",
      "metadata": {}
    },
    {
      "cell_type": "markdown",
      "source": "#### To view the column as a series, just use one bracket:\n",
      "metadata": {}
    },
    {
      "cell_type": "code",
      "source": "# Get the Student column as a series Object\n\nx = df1['Student']\nx",
      "metadata": {},
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": "#check the type of x\ntype(x)",
      "metadata": {},
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": "The output shows us that the type of the variable is a Series object.\n",
      "metadata": {}
    },
    {
      "cell_type": "markdown",
      "source": "### Exercise 2: <code>loc()</code> and <code>iloc()</code> functions\n\n<code>loc()</code> is a label-based data selecting method which means that we have to pass the name of the row or column that we want to select. This method includes the last element of the range passed in it.\n\nSimple syntax for your understanding:\n\n*   loc\\[row_label, column_label]\n\n<code>iloc()</code> is an indexed-based selecting method which means that we have to pass integer index in the method to select a specific row/column. This method does not include the last element of the range passed in it.\n\nSimple syntax for your understanding:\n\n*   iloc\\[row_index, column_index]\n\n<h4 id=\"data\">Let us see some examples on the same.</h4>\n",
      "metadata": {}
    },
    {
      "cell_type": "code",
      "source": "# Access the value on the first row and the first column\n\ndf.iloc[0, 0]",
      "metadata": {},
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": "# Access the value on the first row and the third column\n\ndf.iloc[0,2]",
      "metadata": {},
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": "# Access the column using the name\n\ndf.loc[0, 'Salary']",
      "metadata": {},
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": "Let us create a new dataframe called 'df2' and assign 'df' to it. Now, let us set the \"Name\" column as an index column using the method set_index().\n",
      "metadata": {}
    },
    {
      "cell_type": "code",
      "source": "df2=df\ndf2=df2.set_index(\"Name\")",
      "metadata": {},
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": "#To display the first 5 rows of new dataframe\ndf2.head()",
      "metadata": {},
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": "#Now, let us access the column using the name\ndf2.loc['Jane', 'Salary']",
      "metadata": {},
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": "### Try it yourself\n",
      "metadata": {}
    },
    {
      "cell_type": "markdown",
      "source": "Use the <code>loc()</code> function,to get the Department of Jane in the newly created dataframe df2.\n",
      "metadata": {}
    },
    {
      "cell_type": "code",
      "source": "#write your code here\n",
      "metadata": {},
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": "<details><summary>Click here for the solution</summary>\n\n```python\ndf2.loc['Jane', 'Department']\n    \n```\n\n</details>\n",
      "metadata": {}
    },
    {
      "cell_type": "markdown",
      "source": "Use the <code>iloc()</code> function,to get the Salary of Mary in the newly created dataframe df2.\n",
      "metadata": {}
    },
    {
      "cell_type": "code",
      "source": "#write your code here\n",
      "metadata": {},
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": "<details><summary>Click here for the solution</summary>\n\n```python\ndf2.iloc[3,2]\n    \n```\n\n</details>\n",
      "metadata": {}
    },
    {
      "cell_type": "markdown",
      "source": "### Exercise 3: Slicing\n\nSlicing uses the \\[] operator to select a set of rows and/or columns from a DataFrame.\n\nTo slice out a set of rows, you use this syntax: data\\[start:stop],\n\nhere the start represents the index from where to consider, and stop represents the index one step BEYOND the row you want to select. You can perform slicing using both the index and the name of the column.\n\n> NOTE: When slicing in pandas, the start bound is included in the output.\n\nSo if you want to select rows 0, 1, and 2 your code would look like this: df.iloc\\[0:3].\n\nIt means you are telling Python to start at index 0 and select rows 0, 1, 2 up to but not including 3.\n\n> NOTE: Labels must be found in the DataFrame or you will get a KeyError.\n\nIndexing by labels(i.e. using <code>loc()</code>) differs from indexing by integers (i.e. using <code>iloc()</code>). With <code>loc()</code>, both the start bound and the stop bound are inclusive. When using <code>loc()</code>, integers can be used, but the integers refer to the index label and not the position.\n\nFor example, using <code>loc()</code> and select 1:4 will get a different result than using <code>iloc()</code> to select rows 1:4.\n\n<h4 id=\"data\">We can also select a specific data value using a row and column location within the DataFrame and iloc indexing.\n",
      "metadata": {}
    },
    {
      "cell_type": "code",
      "source": "# let us do the slicing using old dataframe df\n\ndf.iloc[0:2, 0:3]",
      "metadata": {},
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": "#let us do the slicing using loc() function on old dataframe df where index column is having labels as 0,1,2\ndf.loc[0:2,'ID':'Department']",
      "metadata": {},
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": "#let us do the slicing using loc() function on new dataframe df2 where index column is Name having labels: Rose, John and Jane\ndf2.loc['Rose':'Jane', 'ID':'Department']",
      "metadata": {},
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": "<h2 id=\"quiz\">Try it yourself</h2>\n",
      "metadata": {}
    },
    {
      "cell_type": "markdown",
      "source": "using <code>loc()</code> function, do slicing on old dataframe df to retrieve the Name, ID and department of index column having labels as 2,3\n",
      "metadata": {}
    },
    {
      "cell_type": "code",
      "source": "# Write your code below and press Shift+Enter to execute\n",
      "metadata": {},
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": "<details><summary>Click here for the solution</summary>\n\n```python\ndf.loc[2:3,'Name':'Department']\n    \n```\n\n</details>\n",
      "metadata": {}
    },
    {
      "cell_type": "markdown",
      "source": "<h3 id=\"quiz\">Congratulations, you have completed this lesson and the practice lab on Pandas</h3>\n",
      "metadata": {}
    },
    {
      "cell_type": "markdown",
      "source": "## Author(s):\n\n[Appalabhaktula Hema](https://www.linkedin.com/in/hema-a-39002b128/?utm_medium=Exinfluencer&utm_source=Exinfluencer&utm_content=000026UJ&utm_term=10006555&utm_id=NA-SkillsNetwork-Channel-SkillsNetworkCoursesIBMDeveloperSkillsNetworkPY0101ENSkillsNetwork19487395-2022-01-01)\n\n## Change Log\n\n| Date (YYYY-MM-DD) | Version | Changed By          | Change Description      |\n| ----------------- | ------- | ------------------- | ----------------------- |\n| 2022-03-31        | 0.1     | Appalabhaktula Hema | Created initial version |\n\n<hr/>\n\n## <h3 align=\"center\"> © IBM Corporation 2022. All rights reserved. <h3/>\n",
      "metadata": {}
    }
  ]
}