folium/folium_tutorial.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step-by-step guide to generating an interactive climate map in Folium (& Geopandas)\n",
    "\n",
    "- **With some specific boilerplate code already filled in.**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"../img/folium_logo.jpg\" align=\"center\" alt=\"bokeh_map\" width=\"100/\"/>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- **CREDITS**:\n",
    "    \n",
    "    This tutorial was prepared by [Paul Wlodkowski](https://github.com/pawlodkowski) for the *Plotting on Maps* lesson @ Spiced Academy.\n",
    "    \n",
    "    \n",
    "- **DATA**:\n",
    "\n",
    "     - The data for this particular lesson is **synthetic** / **fake!** It was originally scraped from [Berkeley Earth](http://berkeleyearth.lbl.gov/country-list/), but then regenerated based a simple polynomial trend and monthly seasonality modeled from the original data source & injected with random noise. Hence, the data in this repo is somewhat realistic but not authentic.\n",
    "\n",
    "\n",
    "- **INSTALLATION**:\n",
    "\n",
    "    - **Make sure you already have folium and geopandas installed**! (e.g. `pip` or `conda`), e.g.:\n",
    "\n",
    "        - `pip install folium`\n",
    "\n",
    "        - `pip install geopandas==0.8.0`\n",
    "    "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "*Note*: this notebook was last run and tested on geopandas version 0.8.0 (released July 15, 2020). The latest version of folium should work just fine."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### STEP 1: Read in historical temperature data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "DATA = '../data/temp_data.csv'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Read in the data with pandas\n",
    "import pandas as pd\n",
    "df = pd.read_csv(DATA)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### STEP 2: Read in the geographic data (geometric shapes of all countries in the world) \n",
    "- Hint: Use GeoPandas\n",
    "    - What is a **Shape file (.shp)?**\n",
    "        - https://en.wikipedia.org/wiki/Shapefile#Shapefile_shape_format_(.shp)\n",
    "- Publicly available GIS data downloaded from *Natural Earth*: https://www.naturalearthdata.com/downloads/110m-cultural-vectors/"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "SHAPEFILE = '../data/ne_110m_admin_0_countries.shp'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Read in the shapefile with geopandas\n",
    "import geopandas as gpd\n",
    "gdf = gpd.read_file(____)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "*What happens if I have trouble installing geopandas*?\n",
    "\n",
    "- Unfortunately, this happened a couple times in the past with students, as the conda / pip installation on some Windows machines has been somewhat unreliable.\n",
    "    - In this case, there have been some helpful posts from StackOverflow (for example, [this one](https://stackoverflow.com/questions/54734667/error-installing-geopandas-a-gdal-api-version-must-be-specified-in-anaconda)), which finally got the installation working.\n",
    "- However, if you still have trouble with the installation despite lots of trial and error, you may consider creating a virtual environment or building a custom Docker image that builds off another image with Jupyter (e.g. https://hub.docker.com/r/jupyter/datascience-notebook)\n",
    "    - If you choose to go this route, please reach out to a teacher for help. \n",
    "- If you would rather not try those options (for very understandable reasons), then don't worry; a teacher can provide you with a `.geojson` file you can use.\n",
    "    - After all, the only reason we use GeoPandas in the first place is to read in the data and generate these GeoJSON files, so you shouldn't let this hold you back -- it's not that important in the end! \n",
    "    - Also, in general, with enough time spent googling around, you can find GeoJSON files from pretty much everywhere! For example:\n",
    "        - https://github.com/johan/world.geo.json/tree/master/countries\n",
    "        - https://github.com/isellsoap/deutschlandGeoJSON/blob/master/2_bundeslaender/4_niedrig.geo.json\n",
    "        - https://github.com/funkeinteraktiv/Berlin-Geodaten/blob/master/berlin_bezirke.geojson\n",
    "        - _And many more! They're super easy to find online! The world of open source is truly wonderful..._"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### STEP 3: Group / aggregate the temperature anomaly data by country, year\n",
    "- For simplicity, we're only interested in yearly averages"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df.groupby(...)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### STEP 4: Merge Data Sets.\n",
    "- We want to have our temperature data and geometric data in one place.\n",
    "- **Make sure you're still left with a GeoDataFrame at the end!**\n",
    "    - otherwise, if the resulting object is a regular Pandas dataframe, the ``.to_json()`` export will produce a normal ``.json`` file rather than a specialized ``.geojson`` file.\n",
    "    - A geojson file is very similar to a json file, with the exception that it's a bit more strict and specialized and works better with plotting libraries that usually expect the data to be in that format."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "gdf.merge(...)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "---\n",
    "### Time for Visualization with Folium\n",
    "\n",
    "<img src=\"../img/folium_logo.jpg\" align=\"left\" alt=\"bokeh_logo\" width=\"100/\"/>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### STEP 5: Plot data on a map for a single year (we can make it interactive later)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "   ### 5a. Generate a blank canvas / figure."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import folium"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "basemap = folium.Map(location=[52.54, 13.36],\n",
    "                zoom_start=2,\n",
    "                tiles='CartoDB positron') #try out the other options!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "basemap #display the map directly in Jupyter!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "\n",
    "### 5a. Generate a GeoJSON string for a single year.\n",
    "- The Folium library (as well as many other JavaScript-based mapping libraries) requires the data to be in GeoJSON format.\n",
    "    - **HINT**: How can you convert a GeoDataFrame into a GeoJSON? Think of how you might do this in regular pandas.\n",
    "- Let's use the year 2000 as an example."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "gdf_2000 = gdf[gdf['___'] == ___]\n",
    "json_2000 = gdf_2000.to____()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### STEP 6: Generate an interactive choropleth map of the data for a single year\n",
    "- Use the convenient ``folium.Choropleth()`` class to generate interactive tiles, which we can overlay on top of our base map!\n",
    "- The trick to get it working is to make sure that the Choropleth layer understands which column from the dataframes is supposed to be mapped to which key in the JSON string in order for the data to render properly."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "tiles = folium.Choropleth(\n",
    "            geo_data=____,                          # geojson string that includes the geo data (for the year 2000)\n",
    "            name=\"____\",\n",
    "            data=____,                              # dataframe that includes the data (for the year 2000)\n",
    "            columns=[\"country\", \"monthly_anomaly\"], # names of the columns to include from the dataframe\n",
    "            key_on=\"properties._____\",              # name of JSON key within the \"properties\" value that contains country names\n",
    "            fill_color=\"YlOrRd\",                    # play around with the rest of the aesthetic options\n",
    "            nan_fill_color=\"#ededed\",         \n",
    "            fill_opacity=0.7,\n",
    "            line_opacity=0.2,\n",
    "            legend_name=\"Monthly Temperature Anomaly\",\n",
    "            highlight=True,\n",
    "        )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "tiles.add_to(basemap) #add the layer to the blank canvas / figure"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# basemap"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### STEP 7: Add additional features to the map!\n",
    "Some ideas:\n",
    "- **Tooltip highlighting** (included here)\n",
    "- Adding other layers / markers\n",
    "- Making the tiles \"clickable\"\n",
    "- Make the legend a fixed size\n",
    "- The documentation isn't *great*, but with enough persistence / looking at examples online (some pretty good tutorials out there) / reading forums, you can add nice little widgets and extra interactivity to your map!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "style_html = \"font-size: 10px; font-weight: normal\" #add a little bit of HTML, if you know some.\n",
    "\n",
    "tooltip = folium.features.GeoJsonTooltip(fields = ['country', 'monthly_anomaly'],\n",
    "                                         style = style_html) \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "tiles.geojson.add_child(tooltip)\n",
    "#overwrite some of the original properties of the geojson attribute of the tile"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "basemap #the map should now display information whenever you hover over a country tile!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### STEP 8: Export the figure to an HTML file, so you can open it in your web browser!\n",
    "- The reason the figure is interactive (e.g. you can zoom around, it has hover effects, etc.) is because there's a of front-end (i.e. client-side) JavaScript code that Plotly creates for you automatically.\n",
    "- Use the ``.save()`` method to export the file and open it up in a web browser.\n",
    "    - Bonus: Open the ``.html`` file in a text editor and see if you can understand any of it :)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "basemap.save(\"2000_map.html\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#bash command to open the file (should use your web browser by default)\n",
    "!open 2000_map.html"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### And that's it!\n",
    "- Unfortunately, there doesn't seem to be an easy way to add interactive sliders to the map (e.g. to change frames / years) like how you can do it in Plotly or Bokeh (at least, not as far as I could find).\n",
    "    - Folium is pretty easy to use to make interactive maps quickly, but it seems to be lacking for more advanced features. The library is meant to be a bit more minimal / lightweight (which is also great, depending on your use case!)\n",
    "    - You can, of course, write your own for-loop to generate multiple files for different years, but if you really want to add animation, you either have to write your own custom JavaScript (probably not a reasonable option) or switch over to a more powerful visualization library like Plotly or Bokeh.\n",
    "    - However, there seem to be enough pretty good Folium tutorials / examples online where people have been able to make some pretty amazing things."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}