ML-Explainability/Tabular/fairness_xai_COMPAS.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ug4vqKC2DfZg"
      },
      "source": [
        "# Exploring models trained on the COMPAS Recidivism Racial Bias dataset\n",
        "\n",
        "<a href=\"https://colab.research.google.com/drive/1BRb3i82Z4m7L1zCfTJw_QBGswxtOkwhY\" target=\"_blank\">\n",
        "  <img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\">\n",
        "</a>\n",
        "\n",
        "Return to the [castle](https://github.com/Nkluge-correa/TeenyTinyCastle).\n",
        "\n",
        "Fairness and interpretability are two important considerations in machine learning, and they are closely related. Fairness refers to the absence of bias or discrimination in the ML models and their outputs, while interpretability refers to the ability to understand and explain how the models make their predictions.\n",
        "\n",
        "In some cases, improving a model's interpretability can help identify and correct sources of bias or discrimination, improving the model's fairness. For example, suppose a model is biased against a particular group of people. In that case, it may be necessary to examine the features the model uses to make its predictions and determine whether any of these features unfairly influence the model's output. Hence, by improving the interpretability of the model, it may be possible to identify these problematic features and make changes to reduce bias and discrimination. Conversely, fairness can also be used as a metric to evaluate the interpretability of a model, given that a more interpretable model may be easier to assess for bias and discrimination, as it is easier to understand how the model is making its predictions.\n",
        "\n",
        "Therefore, fairness and interpretability are closely linked, and it is important to consider both aspects when developing ML models. We will work with these two principles in this notebook using the `COMPAS` dataset.\n",
        "\n",
        "<img src=\"https://static.propublica.org/projects/algorithmic-bias/assets/img/generated/opener-b-crop-1200*675-00796e.jpg\" width=\"600\"/>\n",
        "\n",
        "Source: [ProPublica](https://www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing).\n",
        "\n",
        "**Correctional Offender Management Profiling for Alternative Sanctions** (`COMPAS`) is a case management and decision support tool developed and owned by Northpointe (now [Equivant](https://www.equivant.com/)) used by USA courts to assess the likelihood of a defendant becoming a recidivist. In short, the `COMPAS` software uses an algorithm to assess potential recidivism risk. Northpointe created risk scales for general and **violent recidivism**, and for **pretrial misconduct**.\n",
        "\n",
        "> **Note**: all datasets and models related to the course and repo are in the Hub. 🤗\n",
        "\n",
        "A general critique of the use of proprietary software such as COMPAS is that since the algorithms it uses are **trade secrets**, they cannot be examined by the public and affected parties, which may violate due process. Another general criticism of machine-learning-based algorithms is that since they are data-dependent, the software will likely yield biased results if the data used in training is biased.\n",
        "\n",
        "To test this critique, we will create a classifier from scratch and train it on the COMPAS dataset.\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 27,
      "metadata": {
        "id": "IZBb7lKQfPXv"
      },
      "outputs": [],
      "source": [
        "!pip install datasets -q\n",
        "\n",
        "from datasets import load_dataset\n",
        "\n",
        "# Load the datasets from the hub\n",
        "dataset = load_dataset('AiresPucrs/COMPAS', split=\"train\")\n",
        "\n",
        "# Turn the datasets into a pandas.DataFrame\n",
        "df = dataset.to_pandas()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "v6JKNoHge83f"
      },
      "source": [
        "After downloading our dataset, we need to eliminate the labels (scores and categories) that the original algorithm produced. We will only use a subset of the features from the original dataset in this tutorial. Also, for simplicity, we are merging the `Low` and `Medium` labels to turn this classification task into a binary problem (Fairness analyses are more straightforward in these cases).\n",
        "\n",
        "> **Note:** `High` risk samples represent only 25% of our dataset, and these are precisely the cases we want to distinguish better."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 28,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 617
        },
        "id": "xhLRIeN7g2Nz",
        "outputId": "48e8316d-03f9-4bd3-ed83-ac18d6c4177d"
      },
      "outputs": [
        {
          "data": {
            "text/html": [
              "\n",
              "  <div id=\"df-8c5e3458-7766-4963-a332-cc1cb660459d\" class=\"colab-df-container\">\n",
              "    <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>sex</th>\n",
              "      <th>age_cat</th>\n",
              "      <th>race</th>\n",
              "      <th>juv_fel_count</th>\n",
              "      <th>juv_misd_count</th>\n",
              "      <th>juv_other_count</th>\n",
              "      <th>priors_count</th>\n",
              "      <th>days_b_screening_arrest</th>\n",
              "      <th>c_days_from_compas</th>\n",
              "      <th>c_charge_degree</th>\n",
              "      <th>is_recid</th>\n",
              "      <th>is_violent_recid</th>\n",
              "      <th>label</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>Male</td>\n",
              "      <td>Greater than 45</td>\n",
              "      <td>Other</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>-1.0</td>\n",
              "      <td>1.0</td>\n",
              "      <td>(F3)</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>Male</td>\n",
              "      <td>Greater than 45</td>\n",
              "      <td>Other</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>-1.0</td>\n",
              "      <td>1.0</td>\n",
              "      <td>(F3)</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>Male</td>\n",
              "      <td>25 - 45</td>\n",
              "      <td>African-American</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>-1.0</td>\n",
              "      <td>1.0</td>\n",
              "      <td>(F3)</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>Male</td>\n",
              "      <td>Less than 25</td>\n",
              "      <td>African-American</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>4</td>\n",
              "      <td>-1.0</td>\n",
              "      <td>1.0</td>\n",
              "      <td>(F3)</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>Male</td>\n",
              "      <td>Less than 25</td>\n",
              "      <td>African-American</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>4</td>\n",
              "      <td>-1.0</td>\n",
              "      <td>1.0</td>\n",
              "      <td>(F3)</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>...</th>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>17014</th>\n",
              "      <td>Female</td>\n",
              "      <td>25 - 45</td>\n",
              "      <td>African-American</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>5</td>\n",
              "      <td>-1.0</td>\n",
              "      <td>1.0</td>\n",
              "      <td>(M1)</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>17015</th>\n",
              "      <td>Male</td>\n",
              "      <td>Greater than 45</td>\n",
              "      <td>Other</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>-1.0</td>\n",
              "      <td>1.0</td>\n",
              "      <td>(F2)</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>17016</th>\n",
              "      <td>Female</td>\n",
              "      <td>25 - 45</td>\n",
              "      <td>African-American</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>3</td>\n",
              "      <td>-1.0</td>\n",
              "      <td>1.0</td>\n",
              "      <td>(M1)</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>17017</th>\n",
              "      <td>Female</td>\n",
              "      <td>Less than 25</td>\n",
              "      <td>Hispanic</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>2</td>\n",
              "      <td>-2.0</td>\n",
              "      <td>2.0</td>\n",
              "      <td>(F3)</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>17018</th>\n",
              "      <td>Female</td>\n",
              "      <td>Less than 25</td>\n",
              "      <td>Hispanic</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>2</td>\n",
              "      <td>-2.0</td>\n",
              "      <td>2.0</td>\n",
              "      <td>(F3)</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>17019 rows × 13 columns</p>\n",
              "</div>\n",
              "    <div class=\"colab-df-buttons\">\n",
              "\n",
              "  <div class=\"colab-df-container\">\n",
              "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-8c5e3458-7766-4963-a332-cc1cb660459d')\"\n",
              "            title=\"Convert this dataframe to an interactive table.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
              "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
              "  </svg>\n",
              "    </button>\n",
              "\n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    .colab-df-buttons div {\n",
              "      margin-bottom: 4px;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "    <script>\n",
              "      const buttonEl =\n",
              "        document.querySelector('#df-8c5e3458-7766-4963-a332-cc1cb660459d button.colab-df-convert');\n",
              "      buttonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      async function convertToInteractive(key) {\n",
              "        const element = document.querySelector('#df-8c5e3458-7766-4963-a332-cc1cb660459d');\n",
              "        const dataTable =\n",
              "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                    [key], {});\n",
              "        if (!dataTable) return;\n",
              "\n",
              "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "          + ' to learn more about interactive tables.';\n",
              "        element.innerHTML = '';\n",
              "        dataTable['output_type'] = 'display_data';\n",
              "        await google.colab.output.renderOutput(dataTable, element);\n",
              "        const docLink = document.createElement('div');\n",
              "        docLink.innerHTML = docLinkHtml;\n",
              "        element.appendChild(docLink);\n",
              "      }\n",
              "    </script>\n",
              "  </div>\n",
              "\n",
              "\n",
              "<div id=\"df-fa577e30-8a92-4606-9113-ee9bc9769e94\">\n",
              "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-fa577e30-8a92-4606-9113-ee9bc9769e94')\"\n",
              "            title=\"Suggest charts\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "     width=\"24px\">\n",
              "    <g>\n",
              "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
              "    </g>\n",
              "</svg>\n",
              "  </button>\n",
              "\n",
              "<style>\n",
              "  .colab-df-quickchart {\n",
              "      --bg-color: #E8F0FE;\n",
              "      --fill-color: #1967D2;\n",
              "      --hover-bg-color: #E2EBFA;\n",
              "      --hover-fill-color: #174EA6;\n",
              "      --disabled-fill-color: #AAA;\n",
              "      --disabled-bg-color: #DDD;\n",
              "  }\n",
              "\n",
              "  [theme=dark] .colab-df-quickchart {\n",
              "      --bg-color: #3B4455;\n",
              "      --fill-color: #D2E3FC;\n",
              "      --hover-bg-color: #434B5C;\n",
              "      --hover-fill-color: #FFFFFF;\n",
              "      --disabled-bg-color: #3B4455;\n",
              "      --disabled-fill-color: #666;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart {\n",
              "    background-color: var(--bg-color);\n",
              "    border: none;\n",
              "    border-radius: 50%;\n",
              "    cursor: pointer;\n",
              "    display: none;\n",
              "    fill: var(--fill-color);\n",
              "    height: 32px;\n",
              "    padding: 0;\n",
              "    width: 32px;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart:hover {\n",
              "    background-color: var(--hover-bg-color);\n",
              "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "    fill: var(--button-hover-fill-color);\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart-complete:disabled,\n",
              "  .colab-df-quickchart-complete:disabled:hover {\n",
              "    background-color: var(--disabled-bg-color);\n",
              "    fill: var(--disabled-fill-color);\n",
              "    box-shadow: none;\n",
              "  }\n",
              "\n",
              "  .colab-df-spinner {\n",
              "    border: 2px solid var(--fill-color);\n",
              "    border-color: transparent;\n",
              "    border-bottom-color: var(--fill-color);\n",
              "    animation:\n",
              "      spin 1s steps(1) infinite;\n",
              "  }\n",
              "\n",
              "  @keyframes spin {\n",
              "    0% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "      border-left-color: var(--fill-color);\n",
              "    }\n",
              "    20% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    30% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    40% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    60% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    80% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "    90% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "  }\n",
              "</style>\n",
              "\n",
              "  <script>\n",
              "    async function quickchart(key) {\n",
              "      const quickchartButtonEl =\n",
              "        document.querySelector('#' + key + ' button');\n",
              "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
              "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
              "      try {\n",
              "        const charts = await google.colab.kernel.invokeFunction(\n",
              "            'suggestCharts', [key], {});\n",
              "      } catch (error) {\n",
              "        console.error('Error during call to suggestCharts:', error);\n",
              "      }\n",
              "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
              "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
              "    }\n",
              "    (() => {\n",
              "      let quickchartButtonEl =\n",
              "        document.querySelector('#df-fa577e30-8a92-4606-9113-ee9bc9769e94 button');\n",
              "      quickchartButtonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "    })();\n",
              "  </script>\n",
              "</div>\n",
              "    </div>\n",
              "  </div>\n"
            ],
            "text/plain": [
              "          sex          age_cat              race  juv_fel_count  \\\n",
              "0        Male  Greater than 45             Other              0   \n",
              "1        Male  Greater than 45             Other              0   \n",
              "2        Male          25 - 45  African-American              0   \n",
              "3        Male     Less than 25  African-American              0   \n",
              "4        Male     Less than 25  African-American              0   \n",
              "...       ...              ...               ...            ...   \n",
              "17014  Female          25 - 45  African-American              0   \n",
              "17015    Male  Greater than 45             Other              0   \n",
              "17016  Female          25 - 45  African-American              0   \n",
              "17017  Female     Less than 25          Hispanic              0   \n",
              "17018  Female     Less than 25          Hispanic              0   \n",
              "\n",
              "       juv_misd_count  juv_other_count  priors_count  days_b_screening_arrest  \\\n",
              "0                   0                0             0                     -1.0   \n",
              "1                   0                0             0                     -1.0   \n",
              "2                   0                0             0                     -1.0   \n",
              "3                   0                1             4                     -1.0   \n",
              "4                   0                1             4                     -1.0   \n",
              "...               ...              ...           ...                      ...   \n",
              "17014               0                0             5                     -1.0   \n",
              "17015               0                0             0                     -1.0   \n",
              "17016               0                0             3                     -1.0   \n",
              "17017               0                0             2                     -2.0   \n",
              "17018               0                0             2                     -2.0   \n",
              "\n",
              "       c_days_from_compas c_charge_degree  is_recid  is_violent_recid  label  \n",
              "0                     1.0            (F3)         0                 0      1  \n",
              "1                     1.0            (F3)         0                 0      1  \n",
              "2                     1.0            (F3)         1                 1      1  \n",
              "3                     1.0            (F3)         1                 0      1  \n",
              "4                     1.0            (F3)         1                 0      1  \n",
              "...                   ...             ...       ...               ...    ...  \n",
              "17014                 1.0            (M1)         0                 0      1  \n",
              "17015                 1.0            (F2)         0                 0      1  \n",
              "17016                 1.0            (M1)         0                 0      1  \n",
              "17017                 2.0            (F3)         1                 0      1  \n",
              "17018                 2.0            (F3)         1                 0      1  \n",
              "\n",
              "[17019 rows x 13 columns]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        }
      ],
      "source": [
        "import pandas as pd\n",
        "\n",
        "# Create a Label column\n",
        "df['label'] = df['score_text'].apply(lambda x: 0 if x == 'High' else 1)\n",
        "\n",
        "# Select the features we will use\n",
        "features = ['sex', 'age_cat', 'race',\n",
        "        'juv_fel_count', 'juv_misd_count',\n",
        "        'juv_other_count', 'priors_count',\n",
        "        'days_b_screening_arrest', 'c_days_from_compas',\n",
        "        'c_charge_degree', 'is_recid', 'is_violent_recid',\n",
        "        'label']\n",
        "\n",
        "df = df[features].dropna()\n",
        "df.reset_index(inplace=True, drop=True)\n",
        "\n",
        "with pd.option_context('display.max_columns', None):\n",
        "    display(df)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "v5JzmO-rDfZl"
      },
      "source": [
        "Now, let us see how our target is related to the sensitive attributes of our dataset (`Age`, `Race`, and `Sex`)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 29,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 1000
        },
        "id": "J_2R_uB0DfZl",
        "outputId": "5dbd246a-3a43-4d9a-eead-1ca3c31f3b74"
      },
      "outputs": [
        {
          "data": {
            "text/html": [
              "<html>\n",
              "<head><meta charset=\"utf-8\" /></head>\n",
              "<body>\n",
              "    <div>            <script src=\"https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/MathJax.js?config=TeX-AMS-MML_SVG\"></script><script type=\"text/javascript\">if (window.MathJax && window.MathJax.Hub && window.MathJax.Hub.Config) {window.MathJax.Hub.Config({SVG: {font: \"STIX-Web\"}});}</script>                <script type=\"text/javascript\">window.PlotlyConfig = {MathJaxConfig: 'local'};</script>\n",
              "        <script charset=\"utf-8\" src=\"https://cdn.plot.ly/plotly-2.24.1.min.js\"></script>                <div id=\"ff87d449-6ae0-473a-9883-7ec6c232747b\" class=\"plotly-graph-div\" style=\"height:525px; width:100%;\"></div>            <script type=\"text/javascript\">                                    window.PLOTLYENV=window.PLOTLYENV || {};                                    if (document.getElementById(\"ff87d449-6ae0-473a-9883-7ec6c232747b\")) {                    Plotly.newPlot(                        \"ff87d449-6ae0-473a-9883-7ec6c232747b\",                        [{\"name\":\"High Risk\",\"x\":[\"Male\",\"Female\"],\"y\":[3753,599],\"type\":\"bar\"},{\"name\":\"Low Risk\",\"x\":[\"Male\",\"Female\"],\"y\":[10157,2510],\"type\":\"bar\"}],                        {\"template\":{\"data\":{\"barpolar\":[{\"marker\":{\"line\":{\"color\":\"rgb(17,17,17)\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"barpolar\"}],\"bar\":[{\"error_x\":{\"color\":\"#f2f5fa\"},\"error_y\":{\"color\":\"#f2f5fa\"},\"marker\":{\"line\":{\"color\":\"rgb(17,17,17)\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"bar\"}],\"carpet\":[{\"aaxis\":{\"endlinecolor\":\"#A2B1C6\",\"gridcolor\":\"#506784\",\"linecolor\":\"#506784\",\"minorgridcolor\":\"#506784\",\"startlinecolor\":\"#A2B1C6\"},\"baxis\":{\"endlinecolor\":\"#A2B1C6\",\"gridcolor\":\"#506784\",\"linecolor\":\"#506784\",\"minorgridcolor\":\"#506784\",\"startlinecolor\":\"#A2B1C6\"},\"type\":\"carpet\"}],\"choropleth\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"type\":\"choropleth\"}],\"contourcarpet\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"type\":\"contourcarpet\"}],\"contour\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"contour\"}],\"heatmapgl\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"heatmapgl\"}],\"heatmap\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"heatmap\"}],\"histogram2dcontour\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"histogram2dcontour\"}],\"histogram2d\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"histogram2d\"}],\"histogram\":[{\"marker\":{\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"histogram\"}],\"mesh3d\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"type\":\"mesh3d\"}],\"parcoords\":[{\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"parcoords\"}],\"pie\":[{\"automargin\":true,\"type\":\"pie\"}],\"scatter3d\":[{\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scatter3d\"}],\"scattercarpet\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scattercarpet\"}],\"scattergeo\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scattergeo\"}],\"scattergl\":[{\"marker\":{\"line\":{\"color\":\"#283442\"}},\"type\":\"scattergl\"}],\"scattermapbox\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scattermapbox\"}],\"scatterpolargl\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scatterpolargl\"}],\"scatterpolar\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scatterpolar\"}],\"scatter\":[{\"marker\":{\"line\":{\"color\":\"#283442\"}},\"type\":\"scatter\"}],\"scatterternary\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scatterternary\"}],\"surface\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"surface\"}],\"table\":[{\"cells\":{\"fill\":{\"color\":\"#506784\"},\"line\":{\"color\":\"rgb(17,17,17)\"}},\"header\":{\"fill\":{\"color\":\"#2a3f5f\"},\"line\":{\"color\":\"rgb(17,17,17)\"}},\"type\":\"table\"}]},\"layout\":{\"annotationdefaults\":{\"arrowcolor\":\"#f2f5fa\",\"arrowhead\":0,\"arrowwidth\":1},\"autotypenumbers\":\"strict\",\"coloraxis\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"colorscale\":{\"diverging\":[[0,\"#8e0152\"],[0.1,\"#c51b7d\"],[0.2,\"#de77ae\"],[0.3,\"#f1b6da\"],[0.4,\"#fde0ef\"],[0.5,\"#f7f7f7\"],[0.6,\"#e6f5d0\"],[0.7,\"#b8e186\"],[0.8,\"#7fbc41\"],[0.9,\"#4d9221\"],[1,\"#276419\"]],\"sequential\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"sequentialminus\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]},\"colorway\":[\"#636efa\",\"#EF553B\",\"#00cc96\",\"#ab63fa\",\"#FFA15A\",\"#19d3f3\",\"#FF6692\",\"#B6E880\",\"#FF97FF\",\"#FECB52\"],\"font\":{\"color\":\"#f2f5fa\"},\"geo\":{\"bgcolor\":\"rgb(17,17,17)\",\"lakecolor\":\"rgb(17,17,17)\",\"landcolor\":\"rgb(17,17,17)\",\"showlakes\":true,\"showland\":true,\"subunitcolor\":\"#506784\"},\"hoverlabel\":{\"align\":\"left\"},\"hovermode\":\"closest\",\"mapbox\":{\"style\":\"dark\"},\"paper_bgcolor\":\"rgb(17,17,17)\",\"plot_bgcolor\":\"rgb(17,17,17)\",\"polar\":{\"angularaxis\":{\"gridcolor\":\"#506784\",\"linecolor\":\"#506784\",\"ticks\":\"\"},\"bgcolor\":\"rgb(17,17,17)\",\"radialaxis\":{\"gridcolor\":\"#506784\",\"linecolor\":\"#506784\",\"ticks\":\"\"}},\"scene\":{\"xaxis\":{\"backgroundcolor\":\"rgb(17,17,17)\",\"gridcolor\":\"#506784\",\"gridwidth\":2,\"linecolor\":\"#506784\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"#C8D4E3\"},\"yaxis\":{\"backgroundcolor\":\"rgb(17,17,17)\",\"gridcolor\":\"#506784\",\"gridwidth\":2,\"linecolor\":\"#506784\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"#C8D4E3\"},\"zaxis\":{\"backgroundcolor\":\"rgb(17,17,17)\",\"gridcolor\":\"#506784\",\"gridwidth\":2,\"linecolor\":\"#506784\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"#C8D4E3\"}},\"shapedefaults\":{\"line\":{\"color\":\"#f2f5fa\"}},\"sliderdefaults\":{\"bgcolor\":\"#C8D4E3\",\"bordercolor\":\"rgb(17,17,17)\",\"borderwidth\":1,\"tickwidth\":0},\"ternary\":{\"aaxis\":{\"gridcolor\":\"#506784\",\"linecolor\":\"#506784\",\"ticks\":\"\"},\"baxis\":{\"gridcolor\":\"#506784\",\"linecolor\":\"#506784\",\"ticks\":\"\"},\"bgcolor\":\"rgb(17,17,17)\",\"caxis\":{\"gridcolor\":\"#506784\",\"linecolor\":\"#506784\",\"ticks\":\"\"}},\"title\":{\"x\":0.05},\"updatemenudefaults\":{\"bgcolor\":\"#506784\",\"borderwidth\":0},\"xaxis\":{\"automargin\":true,\"gridcolor\":\"#283442\",\"linecolor\":\"#506784\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"#283442\",\"zerolinewidth\":2},\"yaxis\":{\"automargin\":true,\"gridcolor\":\"#283442\",\"linecolor\":\"#506784\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"#283442\",\"zerolinewidth\":2}}},\"barmode\":\"group\",\"xaxis\":{\"title\":{\"text\":\"\\u003cb\\u003eSex\\u003c\\u002fb\\u003e\"}},\"yaxis\":{\"title\":{\"text\":\"\\u003cb\\u003eRisk by Sex\\u003c\\u002fb\\u003e\"}},\"title\":{\"text\":\"Distribution of \\u003ci\\u003eRisk Scores\\u003c\\u002fi\\u003e by \\\"Sex\\\"\"},\"paper_bgcolor\":\"rgba(0, 0, 0, 0)\",\"plot_bgcolor\":\"rgba(0, 0, 0, 0)\"},                        {\"responsive\": true}                    ).then(function(){\n",
              "                            \n",
              "var gd = document.getElementById('ff87d449-6ae0-473a-9883-7ec6c232747b');\n",
              "var x = new MutationObserver(function (mutations, observer) {{\n",
              "        var display = window.getComputedStyle(gd).display;\n",
              "        if (!display || display === 'none') {{\n",
              "            console.log([gd, 'removed!']);\n",
              "            Plotly.purge(gd);\n",
              "            observer.disconnect();\n",
              "        }}\n",
              "}});\n",
              "\n",
              "// Listen for the removal of the full notebook cells\n",
              "var notebookContainer = gd.closest('#notebook-container');\n",
              "if (notebookContainer) {{\n",
              "    x.observe(notebookContainer, {childList: true});\n",
              "}}\n",
              "\n",
              "// Listen for the clearing of the current output cell\n",
              "var outputEl = gd.closest('.output');\n",
              "if (outputEl) {{\n",
              "    x.observe(outputEl, {childList: true});\n",
              "}}\n",
              "\n",
              "                        })                };                            </script>        </div>\n",
              "</body>\n",
              "</html>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/html": [
              "<html>\n",
              "<head><meta charset=\"utf-8\" /></head>\n",
              "<body>\n",
              "    <div>            <script src=\"https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/MathJax.js?config=TeX-AMS-MML_SVG\"></script><script type=\"text/javascript\">if (window.MathJax && window.MathJax.Hub && window.MathJax.Hub.Config) {window.MathJax.Hub.Config({SVG: {font: \"STIX-Web\"}});}</script>                <script type=\"text/javascript\">window.PlotlyConfig = {MathJaxConfig: 'local'};</script>\n",
              "        <script charset=\"utf-8\" src=\"https://cdn.plot.ly/plotly-2.24.1.min.js\"></script>                <div id=\"4ce1caf7-d29f-4820-ab2b-e5e0d04a60e2\" class=\"plotly-graph-div\" style=\"height:525px; width:100%;\"></div>            <script type=\"text/javascript\">                                    window.PLOTLYENV=window.PLOTLYENV || {};                                    if (document.getElementById(\"4ce1caf7-d29f-4820-ab2b-e5e0d04a60e2\")) {                    Plotly.newPlot(                        \"4ce1caf7-d29f-4820-ab2b-e5e0d04a60e2\",                        [{\"name\":\"High Risk\",\"x\":[\"Greater than 45\",\"25 - 45\",\"Less than 25\"],\"y\":[383,2510,1459],\"type\":\"bar\"},{\"name\":\"Low Risk\",\"x\":[\"Greater than 45\",\"25 - 45\",\"Less than 25\"],\"y\":[2923,7183,2561],\"type\":\"bar\"}],                        {\"template\":{\"data\":{\"barpolar\":[{\"marker\":{\"line\":{\"color\":\"rgb(17,17,17)\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"barpolar\"}],\"bar\":[{\"error_x\":{\"color\":\"#f2f5fa\"},\"error_y\":{\"color\":\"#f2f5fa\"},\"marker\":{\"line\":{\"color\":\"rgb(17,17,17)\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"bar\"}],\"carpet\":[{\"aaxis\":{\"endlinecolor\":\"#A2B1C6\",\"gridcolor\":\"#506784\",\"linecolor\":\"#506784\",\"minorgridcolor\":\"#506784\",\"startlinecolor\":\"#A2B1C6\"},\"baxis\":{\"endlinecolor\":\"#A2B1C6\",\"gridcolor\":\"#506784\",\"linecolor\":\"#506784\",\"minorgridcolor\":\"#506784\",\"startlinecolor\":\"#A2B1C6\"},\"type\":\"carpet\"}],\"choropleth\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"type\":\"choropleth\"}],\"contourcarpet\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"type\":\"contourcarpet\"}],\"contour\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"contour\"}],\"heatmapgl\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"heatmapgl\"}],\"heatmap\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"heatmap\"}],\"histogram2dcontour\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"histogram2dcontour\"}],\"histogram2d\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"histogram2d\"}],\"histogram\":[{\"marker\":{\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"histogram\"}],\"mesh3d\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"type\":\"mesh3d\"}],\"parcoords\":[{\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"parcoords\"}],\"pie\":[{\"automargin\":true,\"type\":\"pie\"}],\"scatter3d\":[{\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scatter3d\"}],\"scattercarpet\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scattercarpet\"}],\"scattergeo\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scattergeo\"}],\"scattergl\":[{\"marker\":{\"line\":{\"color\":\"#283442\"}},\"type\":\"scattergl\"}],\"scattermapbox\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scattermapbox\"}],\"scatterpolargl\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scatterpolargl\"}],\"scatterpolar\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scatterpolar\"}],\"scatter\":[{\"marker\":{\"line\":{\"color\":\"#283442\"}},\"type\":\"scatter\"}],\"scatterternary\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scatterternary\"}],\"surface\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"surface\"}],\"table\":[{\"cells\":{\"fill\":{\"color\":\"#506784\"},\"line\":{\"color\":\"rgb(17,17,17)\"}},\"header\":{\"fill\":{\"color\":\"#2a3f5f\"},\"line\":{\"color\":\"rgb(17,17,17)\"}},\"type\":\"table\"}]},\"layout\":{\"annotationdefaults\":{\"arrowcolor\":\"#f2f5fa\",\"arrowhead\":0,\"arrowwidth\":1},\"autotypenumbers\":\"strict\",\"coloraxis\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"colorscale\":{\"diverging\":[[0,\"#8e0152\"],[0.1,\"#c51b7d\"],[0.2,\"#de77ae\"],[0.3,\"#f1b6da\"],[0.4,\"#fde0ef\"],[0.5,\"#f7f7f7\"],[0.6,\"#e6f5d0\"],[0.7,\"#b8e186\"],[0.8,\"#7fbc41\"],[0.9,\"#4d9221\"],[1,\"#276419\"]],\"sequential\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"sequentialminus\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]},\"colorway\":[\"#636efa\",\"#EF553B\",\"#00cc96\",\"#ab63fa\",\"#FFA15A\",\"#19d3f3\",\"#FF6692\",\"#B6E880\",\"#FF97FF\",\"#FECB52\"],\"font\":{\"color\":\"#f2f5fa\"},\"geo\":{\"bgcolor\":\"rgb(17,17,17)\",\"lakecolor\":\"rgb(17,17,17)\",\"landcolor\":\"rgb(17,17,17)\",\"showlakes\":true,\"showland\":true,\"subunitcolor\":\"#506784\"},\"hoverlabel\":{\"align\":\"left\"},\"hovermode\":\"closest\",\"mapbox\":{\"style\":\"dark\"},\"paper_bgcolor\":\"rgb(17,17,17)\",\"plot_bgcolor\":\"rgb(17,17,17)\",\"polar\":{\"angularaxis\":{\"gridcolor\":\"#506784\",\"linecolor\":\"#506784\",\"ticks\":\"\"},\"bgcolor\":\"rgb(17,17,17)\",\"radialaxis\":{\"gridcolor\":\"#506784\",\"linecolor\":\"#506784\",\"ticks\":\"\"}},\"scene\":{\"xaxis\":{\"backgroundcolor\":\"rgb(17,17,17)\",\"gridcolor\":\"#506784\",\"gridwidth\":2,\"linecolor\":\"#506784\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"#C8D4E3\"},\"yaxis\":{\"backgroundcolor\":\"rgb(17,17,17)\",\"gridcolor\":\"#506784\",\"gridwidth\":2,\"linecolor\":\"#506784\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"#C8D4E3\"},\"zaxis\":{\"backgroundcolor\":\"rgb(17,17,17)\",\"gridcolor\":\"#506784\",\"gridwidth\":2,\"linecolor\":\"#506784\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"#C8D4E3\"}},\"shapedefaults\":{\"line\":{\"color\":\"#f2f5fa\"}},\"sliderdefaults\":{\"bgcolor\":\"#C8D4E3\",\"bordercolor\":\"rgb(17,17,17)\",\"borderwidth\":1,\"tickwidth\":0},\"ternary\":{\"aaxis\":{\"gridcolor\":\"#506784\",\"linecolor\":\"#506784\",\"ticks\":\"\"},\"baxis\":{\"gridcolor\":\"#506784\",\"linecolor\":\"#506784\",\"ticks\":\"\"},\"bgcolor\":\"rgb(17,17,17)\",\"caxis\":{\"gridcolor\":\"#506784\",\"linecolor\":\"#506784\",\"ticks\":\"\"}},\"title\":{\"x\":0.05},\"updatemenudefaults\":{\"bgcolor\":\"#506784\",\"borderwidth\":0},\"xaxis\":{\"automargin\":true,\"gridcolor\":\"#283442\",\"linecolor\":\"#506784\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"#283442\",\"zerolinewidth\":2},\"yaxis\":{\"automargin\":true,\"gridcolor\":\"#283442\",\"linecolor\":\"#506784\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"#283442\",\"zerolinewidth\":2}}},\"barmode\":\"group\",\"xaxis\":{\"title\":{\"text\":\"\\u003cb\\u003eAge\\u003c\\u002fb\\u003e\"}},\"yaxis\":{\"title\":{\"text\":\"\\u003cb\\u003eRisk by Age\\u003c\\u002fb\\u003e\"}},\"title\":{\"text\":\"Distribution of \\u003ci\\u003eRisk Scores\\u003c\\u002fi\\u003e by \\\"Age\\\"\"},\"paper_bgcolor\":\"rgba(0, 0, 0, 0)\",\"plot_bgcolor\":\"rgba(0, 0, 0, 0)\"},                        {\"responsive\": true}                    ).then(function(){\n",
              "                            \n",
              "var gd = document.getElementById('4ce1caf7-d29f-4820-ab2b-e5e0d04a60e2');\n",
              "var x = new MutationObserver(function (mutations, observer) {{\n",
              "        var display = window.getComputedStyle(gd).display;\n",
              "        if (!display || display === 'none') {{\n",
              "            console.log([gd, 'removed!']);\n",
              "            Plotly.purge(gd);\n",
              "            observer.disconnect();\n",
              "        }}\n",
              "}});\n",
              "\n",
              "// Listen for the removal of the full notebook cells\n",
              "var notebookContainer = gd.closest('#notebook-container');\n",
              "if (notebookContainer) {{\n",
              "    x.observe(notebookContainer, {childList: true});\n",
              "}}\n",
              "\n",
              "// Listen for the clearing of the current output cell\n",
              "var outputEl = gd.closest('.output');\n",
              "if (outputEl) {{\n",
              "    x.observe(outputEl, {childList: true});\n",
              "}}\n",
              "\n",
              "                        })                };                            </script>        </div>\n",
              "</body>\n",
              "</html>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/html": [
              "<html>\n",
              "<head><meta charset=\"utf-8\" /></head>\n",
              "<body>\n",
              "    <div>            <script src=\"https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/MathJax.js?config=TeX-AMS-MML_SVG\"></script><script type=\"text/javascript\">if (window.MathJax && window.MathJax.Hub && window.MathJax.Hub.Config) {window.MathJax.Hub.Config({SVG: {font: \"STIX-Web\"}});}</script>                <script type=\"text/javascript\">window.PlotlyConfig = {MathJaxConfig: 'local'};</script>\n",
              "        <script charset=\"utf-8\" src=\"https://cdn.plot.ly/plotly-2.24.1.min.js\"></script>                <div id=\"23748bb9-6abe-464f-84c9-3afa250b6d02\" class=\"plotly-graph-div\" style=\"height:525px; width:100%;\"></div>            <script type=\"text/javascript\">                                    window.PLOTLYENV=window.PLOTLYENV || {};                                    if (document.getElementById(\"23748bb9-6abe-464f-84c9-3afa250b6d02\")) {                    Plotly.newPlot(                        \"23748bb9-6abe-464f-84c9-3afa250b6d02\",                        [{\"name\":\"High Risk\",\"x\":[\"Other\",\"African-American\",\"Caucasian\",\"Hispanic\",\"Asian\",\"Native American\"],\"y\":[52,3197,904,176,9,14],\"type\":\"bar\"},{\"name\":\"Low Risk\",\"x\":[\"Other\",\"African-American\",\"Caucasian\",\"Hispanic\",\"Asian\",\"Native American\"],\"y\":[753,5964,4760,1102,55,33],\"type\":\"bar\"}],                        {\"template\":{\"data\":{\"barpolar\":[{\"marker\":{\"line\":{\"color\":\"rgb(17,17,17)\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"barpolar\"}],\"bar\":[{\"error_x\":{\"color\":\"#f2f5fa\"},\"error_y\":{\"color\":\"#f2f5fa\"},\"marker\":{\"line\":{\"color\":\"rgb(17,17,17)\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"bar\"}],\"carpet\":[{\"aaxis\":{\"endlinecolor\":\"#A2B1C6\",\"gridcolor\":\"#506784\",\"linecolor\":\"#506784\",\"minorgridcolor\":\"#506784\",\"startlinecolor\":\"#A2B1C6\"},\"baxis\":{\"endlinecolor\":\"#A2B1C6\",\"gridcolor\":\"#506784\",\"linecolor\":\"#506784\",\"minorgridcolor\":\"#506784\",\"startlinecolor\":\"#A2B1C6\"},\"type\":\"carpet\"}],\"choropleth\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"type\":\"choropleth\"}],\"contourcarpet\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"type\":\"contourcarpet\"}],\"contour\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"contour\"}],\"heatmapgl\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"heatmapgl\"}],\"heatmap\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"heatmap\"}],\"histogram2dcontour\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"histogram2dcontour\"}],\"histogram2d\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"histogram2d\"}],\"histogram\":[{\"marker\":{\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"histogram\"}],\"mesh3d\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"type\":\"mesh3d\"}],\"parcoords\":[{\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"parcoords\"}],\"pie\":[{\"automargin\":true,\"type\":\"pie\"}],\"scatter3d\":[{\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scatter3d\"}],\"scattercarpet\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scattercarpet\"}],\"scattergeo\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scattergeo\"}],\"scattergl\":[{\"marker\":{\"line\":{\"color\":\"#283442\"}},\"type\":\"scattergl\"}],\"scattermapbox\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scattermapbox\"}],\"scatterpolargl\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scatterpolargl\"}],\"scatterpolar\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scatterpolar\"}],\"scatter\":[{\"marker\":{\"line\":{\"color\":\"#283442\"}},\"type\":\"scatter\"}],\"scatterternary\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scatterternary\"}],\"surface\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"surface\"}],\"table\":[{\"cells\":{\"fill\":{\"color\":\"#506784\"},\"line\":{\"color\":\"rgb(17,17,17)\"}},\"header\":{\"fill\":{\"color\":\"#2a3f5f\"},\"line\":{\"color\":\"rgb(17,17,17)\"}},\"type\":\"table\"}]},\"layout\":{\"annotationdefaults\":{\"arrowcolor\":\"#f2f5fa\",\"arrowhead\":0,\"arrowwidth\":1},\"autotypenumbers\":\"strict\",\"coloraxis\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"colorscale\":{\"diverging\":[[0,\"#8e0152\"],[0.1,\"#c51b7d\"],[0.2,\"#de77ae\"],[0.3,\"#f1b6da\"],[0.4,\"#fde0ef\"],[0.5,\"#f7f7f7\"],[0.6,\"#e6f5d0\"],[0.7,\"#b8e186\"],[0.8,\"#7fbc41\"],[0.9,\"#4d9221\"],[1,\"#276419\"]],\"sequential\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"sequentialminus\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]},\"colorway\":[\"#636efa\",\"#EF553B\",\"#00cc96\",\"#ab63fa\",\"#FFA15A\",\"#19d3f3\",\"#FF6692\",\"#B6E880\",\"#FF97FF\",\"#FECB52\"],\"font\":{\"color\":\"#f2f5fa\"},\"geo\":{\"bgcolor\":\"rgb(17,17,17)\",\"lakecolor\":\"rgb(17,17,17)\",\"landcolor\":\"rgb(17,17,17)\",\"showlakes\":true,\"showland\":true,\"subunitcolor\":\"#506784\"},\"hoverlabel\":{\"align\":\"left\"},\"hovermode\":\"closest\",\"mapbox\":{\"style\":\"dark\"},\"paper_bgcolor\":\"rgb(17,17,17)\",\"plot_bgcolor\":\"rgb(17,17,17)\",\"polar\":{\"angularaxis\":{\"gridcolor\":\"#506784\",\"linecolor\":\"#506784\",\"ticks\":\"\"},\"bgcolor\":\"rgb(17,17,17)\",\"radialaxis\":{\"gridcolor\":\"#506784\",\"linecolor\":\"#506784\",\"ticks\":\"\"}},\"scene\":{\"xaxis\":{\"backgroundcolor\":\"rgb(17,17,17)\",\"gridcolor\":\"#506784\",\"gridwidth\":2,\"linecolor\":\"#506784\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"#C8D4E3\"},\"yaxis\":{\"backgroundcolor\":\"rgb(17,17,17)\",\"gridcolor\":\"#506784\",\"gridwidth\":2,\"linecolor\":\"#506784\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"#C8D4E3\"},\"zaxis\":{\"backgroundcolor\":\"rgb(17,17,17)\",\"gridcolor\":\"#506784\",\"gridwidth\":2,\"linecolor\":\"#506784\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"#C8D4E3\"}},\"shapedefaults\":{\"line\":{\"color\":\"#f2f5fa\"}},\"sliderdefaults\":{\"bgcolor\":\"#C8D4E3\",\"bordercolor\":\"rgb(17,17,17)\",\"borderwidth\":1,\"tickwidth\":0},\"ternary\":{\"aaxis\":{\"gridcolor\":\"#506784\",\"linecolor\":\"#506784\",\"ticks\":\"\"},\"baxis\":{\"gridcolor\":\"#506784\",\"linecolor\":\"#506784\",\"ticks\":\"\"},\"bgcolor\":\"rgb(17,17,17)\",\"caxis\":{\"gridcolor\":\"#506784\",\"linecolor\":\"#506784\",\"ticks\":\"\"}},\"title\":{\"x\":0.05},\"updatemenudefaults\":{\"bgcolor\":\"#506784\",\"borderwidth\":0},\"xaxis\":{\"automargin\":true,\"gridcolor\":\"#283442\",\"linecolor\":\"#506784\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"#283442\",\"zerolinewidth\":2},\"yaxis\":{\"automargin\":true,\"gridcolor\":\"#283442\",\"linecolor\":\"#506784\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"#283442\",\"zerolinewidth\":2}}},\"barmode\":\"group\",\"xaxis\":{\"title\":{\"text\":\"\\u003cb\\u003eRace\\u003c\\u002fb\\u003e\"}},\"yaxis\":{\"title\":{\"text\":\"\\u003cb\\u003eRisk by Race\\u003c\\u002fb\\u003e\"}},\"title\":{\"text\":\"Distribution of \\u003ci\\u003eRisk Scores\\u003c\\u002fi\\u003e by \\\"Race\\\"\"},\"paper_bgcolor\":\"rgba(0, 0, 0, 0)\",\"plot_bgcolor\":\"rgba(0, 0, 0, 0)\"},                        {\"responsive\": true}                    ).then(function(){\n",
              "                            \n",
              "var gd = document.getElementById('23748bb9-6abe-464f-84c9-3afa250b6d02');\n",
              "var x = new MutationObserver(function (mutations, observer) {{\n",
              "        var display = window.getComputedStyle(gd).display;\n",
              "        if (!display || display === 'none') {{\n",
              "            console.log([gd, 'removed!']);\n",
              "            Plotly.purge(gd);\n",
              "            observer.disconnect();\n",
              "        }}\n",
              "}});\n",
              "\n",
              "// Listen for the removal of the full notebook cells\n",
              "var notebookContainer = gd.closest('#notebook-container');\n",
              "if (notebookContainer) {{\n",
              "    x.observe(notebookContainer, {childList: true});\n",
              "}}\n",
              "\n",
              "// Listen for the clearing of the current output cell\n",
              "var outputEl = gd.closest('.output');\n",
              "if (outputEl) {{\n",
              "    x.observe(outputEl, {childList: true});\n",
              "}}\n",
              "\n",
              "                        })                };                            </script>        </div>\n",
              "</body>\n",
              "</html>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        }
      ],
      "source": [
        "high_risk = []\n",
        "low_risk = []\n",
        "\n",
        "for element in list(df['sex'].unique()):\n",
        "    a = df[df['sex'] == element]['label'].value_counts()[0]\n",
        "    b = df[df['sex'] == element]['label'].value_counts()[1]\n",
        "    high_risk.append(a)\n",
        "    low_risk.append(b)\n",
        "\n",
        "import plotly.graph_objects as go\n",
        "\n",
        "fig = go.Figure(data=[\n",
        "    go.Bar(name='High Risk', x=list(df['sex'].unique()), y=high_risk),\n",
        "    go.Bar(name='Low Risk', x=list(df['sex'].unique()), y=low_risk)\n",
        "])\n",
        "\n",
        "fig.update_layout(\n",
        "    barmode='group',\n",
        "    template='plotly_dark',\n",
        "    xaxis_title=\"<b>Sex</b>\",\n",
        "    yaxis_title=\"<b>Risk by Sex</b>\",\n",
        "    title='Distribution of <i>Risk Scores</i> by \"Sex\"',\n",
        "    paper_bgcolor='rgba(0, 0, 0, 0)',\n",
        "    plot_bgcolor='rgba(0, 0, 0, 0)',\n",
        "    )\n",
        "fig.show()\n",
        "\n",
        "high_risk = []\n",
        "low_risk = []\n",
        "\n",
        "for element in list(df['age_cat'].unique()):\n",
        "    a = df[df['age_cat'] == element]['label'].value_counts()[0]\n",
        "    b = df[df['age_cat'] == element]['label'].value_counts()[1]\n",
        "    high_risk.append(a)\n",
        "    low_risk.append(b)\n",
        "\n",
        "import plotly.graph_objects as go\n",
        "\n",
        "fig = go.Figure(data=[\n",
        "    go.Bar(name='High Risk', x=list(df['age_cat'].unique()), y=high_risk),\n",
        "    go.Bar(name='Low Risk', x=list(df['age_cat'].unique()), y=low_risk)\n",
        "])\n",
        "\n",
        "fig.update_layout(\n",
        "    barmode='group',\n",
        "    template='plotly_dark',\n",
        "    xaxis_title=\"<b>Age</b>\",\n",
        "    yaxis_title=\"<b>Risk by Age</b>\",\n",
        "    title='Distribution of <i>Risk Scores</i> by \"Age\"',\n",
        "    paper_bgcolor='rgba(0, 0, 0, 0)',\n",
        "    plot_bgcolor='rgba(0, 0, 0, 0)',\n",
        "    )\n",
        "fig.show()\n",
        "\n",
        "high_risk = []\n",
        "low_risk = []\n",
        "\n",
        "for element in list(df['race'].unique()):\n",
        "    a = df[df['race'] == element]['label'].value_counts()[0]\n",
        "    b = df[df['race'] == element]['label'].value_counts()[1]\n",
        "    high_risk.append(a)\n",
        "    low_risk.append(b)\n",
        "\n",
        "import plotly.graph_objects as go\n",
        "\n",
        "fig = go.Figure(data=[\n",
        "    go.Bar(name='High Risk', x=list(df['race'].unique()), y=high_risk),\n",
        "    go.Bar(name='Low Risk', x=list(df['race'].unique()), y=low_risk)\n",
        "])\n",
        "\n",
        "fig.update_layout(\n",
        "    barmode='group',\n",
        "    template='plotly_dark',\n",
        "    xaxis_title=\"<b>Race</b>\",\n",
        "    yaxis_title=\"<b>Risk by Race</b>\",\n",
        "    title='Distribution of <i>Risk Scores</i> by \"Race\"',\n",
        "    paper_bgcolor='rgba(0, 0, 0, 0)',\n",
        "    plot_bgcolor='rgba(0, 0, 0, 0)',\n",
        "    )\n",
        "fig.show()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "OP5OKlovDfZm"
      },
      "source": [
        "Samples with the features `African-American`, `Male`, and `25-45` represent this dataset's bulk of high-risk samples. Let us see if our sensitive attributes correlate with our label. If they are, this is already a sign that our future model could inherit these biases against a specific unprivileged class.\n",
        "\n",
        "> **Note: To be able to calculate correlations, let us transform all categorical values into numbers.**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 30,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 542
        },
        "id": "NENjEqKnDfZm",
        "outputId": "4734dec2-a162-4d85-e044-64c2dce46fa2"
      },
      "outputs": [
        {
          "data": {
            "text/html": [
              "<html>\n",
              "<head><meta charset=\"utf-8\" /></head>\n",
              "<body>\n",
              "    <div>            <script src=\"https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/MathJax.js?config=TeX-AMS-MML_SVG\"></script><script type=\"text/javascript\">if (window.MathJax && window.MathJax.Hub && window.MathJax.Hub.Config) {window.MathJax.Hub.Config({SVG: {font: \"STIX-Web\"}});}</script>                <script type=\"text/javascript\">window.PlotlyConfig = {MathJaxConfig: 'local'};</script>\n",
              "        <script charset=\"utf-8\" src=\"https://cdn.plot.ly/plotly-2.24.1.min.js\"></script>                <div id=\"a6284208-0101-4ed3-a0d9-21ce3a1258b1\" class=\"plotly-graph-div\" style=\"height:525px; width:100%;\"></div>            <script type=\"text/javascript\">                                    window.PLOTLYENV=window.PLOTLYENV || {};                                    if (document.getElementById(\"a6284208-0101-4ed3-a0d9-21ce3a1258b1\")) {                    Plotly.newPlot(                        \"a6284208-0101-4ed3-a0d9-21ce3a1258b1\",                        [{\"coloraxis\":\"coloraxis\",\"name\":\"0\",\"texttemplate\":\"%{z}\",\"x\":[\"sex\",\"age_cat\",\"race\",\"juv_fel_count\",\"juv_misd_count\",\"juv_other_count\",\"priors_count\",\"days_b_screening_arrest\",\"c_days_from_compas\",\"c_charge_degree\",\"is_recid\",\"is_violent_recid\",\"label\"],\"y\":[\"sex\",\"age_cat\",\"race\",\"juv_fel_count\",\"juv_misd_count\",\"juv_other_count\",\"priors_count\",\"days_b_screening_arrest\",\"c_days_from_compas\",\"c_charge_degree\",\"is_recid\",\"is_violent_recid\",\"label\"],\"z\":[[1.0,0.03113840205343482,-0.04603843899097753,0.06151452219179999,0.042750569709476736,0.07005305986196386,0.11228652688450137,0.006041572856964976,-0.009520497963085614,-0.036287327454834324,0.09803299094117854,0.05076038968736838,-0.06832334096894974],[0.03113840205343482,1.0,-0.02435714744260014,0.016928467233164383,0.004819250723213585,0.12482742046531076,-0.20079720491299427,-0.01284774868335103,-0.02656056321210857,-0.058424694847410266,0.04455549733945509,0.017214039179416718,-0.06458596328942577],[-0.04603843899097753,-0.02435714744260014,1.0,-0.07346535833159296,-0.08022841951540675,-0.06747974830780502,-0.19378050529483967,-0.009349644363426795,-0.034858641923782484,0.08699652134960802,-0.12393091299656377,-0.04277920416449448,0.22376624695129044],[0.06151452219179999,0.016928467233164383,-0.07346535833159296,1.0,0.08608664344136145,0.06882148388029831,0.1673976698284928,0.02112294829691979,0.005079124718125456,-0.03699552252811497,0.0754060140549542,0.03264070824854933,-0.18639830787904793],[0.042750569709476736,0.004819250723213585,-0.08022841951540675,0.08608664344136145,1.0,0.2844016968781474,0.2217409637948484,0.012076952608852113,0.0278098566415182,-0.025015731737241115,0.09801660237374711,0.07643463102021157,-0.19351204865381996],[0.07005305986196386,0.12482742046531076,-0.06747974830780502,0.06882148388029831,0.2844016968781474,1.0,0.09052869598675486,0.0036492268764133987,-0.012656986623963567,-0.030347919283496023,0.1036620819113717,0.05606520930143292,-0.16955485466904563],[0.11228652688450137,-0.20079720491299427,-0.19378050529483967,0.1673976698284928,0.2217409637948484,0.09052869598675486,1.0,0.0020940119412184025,0.015523039901195227,-0.09557888980240194,0.2440376464357242,0.05944568686452492,-0.33585304040651814],[0.006041572856964976,-0.01284774868335103,-0.009349644363426795,0.02112294829691979,0.012076952608852113,0.0036492268764133987,0.0020940119412184025,1.0,0.09397713111932289,0.01129545949252831,0.05402127670171265,0.01222869663440031,-0.033454152641045676],[-0.009520497963085614,-0.02656056321210857,-0.034858641923782484,0.005079124718125456,0.0278098566415182,-0.012656986623963567,0.015523039901195227,0.09397713111932289,1.0,0.09199312066829546,-0.05804300425695346,-0.013049913590232618,-0.029828902390914448],[-0.036287327454834324,-0.058424694847410266,0.08699652134960802,-0.03699552252811497,-0.025015731737241115,-0.030347919283496023,-0.09557888980240194,0.01129545949252831,0.09199312066829546,1.0,-0.04180388966414827,0.034436965610763356,0.10005562598705782],[0.09803299094117854,0.04455549733945509,-0.12393091299656377,0.0754060140549542,0.09801660237374711,0.1036620819113717,0.2440376464357242,0.05402127670171265,-0.05804300425695346,-0.04180388966414827,1.0,0.29627272217614825,-0.208062812672839],[0.05076038968736838,0.017214039179416718,-0.04277920416449448,0.03264070824854933,0.07643463102021157,0.05606520930143292,0.05944568686452492,0.01222869663440031,-0.013049913590232618,0.034436965610763356,0.29627272217614825,1.0,-0.09573994335605437],[-0.06832334096894974,-0.06458596328942577,0.22376624695129044,-0.18639830787904793,-0.19351204865381996,-0.16955485466904563,-0.33585304040651814,-0.033454152641045676,-0.029828902390914448,0.10005562598705782,-0.208062812672839,-0.09573994335605437,1.0]],\"type\":\"heatmap\",\"xaxis\":\"x\",\"yaxis\":\"y\",\"hovertemplate\":\"Features: %{x}\\u003cbr\\u003eFeatures: %{y}\\u003cbr\\u003ecolor: %{z}\\u003cextra\\u003e\\u003c\\u002fextra\\u003e\"}],                        {\"template\":{\"data\":{\"barpolar\":[{\"marker\":{\"line\":{\"color\":\"rgb(17,17,17)\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"barpolar\"}],\"bar\":[{\"error_x\":{\"color\":\"#f2f5fa\"},\"error_y\":{\"color\":\"#f2f5fa\"},\"marker\":{\"line\":{\"color\":\"rgb(17,17,17)\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"bar\"}],\"carpet\":[{\"aaxis\":{\"endlinecolor\":\"#A2B1C6\",\"gridcolor\":\"#506784\",\"linecolor\":\"#506784\",\"minorgridcolor\":\"#506784\",\"startlinecolor\":\"#A2B1C6\"},\"baxis\":{\"endlinecolor\":\"#A2B1C6\",\"gridcolor\":\"#506784\",\"linecolor\":\"#506784\",\"minorgridcolor\":\"#506784\",\"startlinecolor\":\"#A2B1C6\"},\"type\":\"carpet\"}],\"choropleth\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"type\":\"choropleth\"}],\"contourcarpet\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"type\":\"contourcarpet\"}],\"contour\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"contour\"}],\"heatmapgl\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"heatmapgl\"}],\"heatmap\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"heatmap\"}],\"histogram2dcontour\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"histogram2dcontour\"}],\"histogram2d\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"histogram2d\"}],\"histogram\":[{\"marker\":{\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"histogram\"}],\"mesh3d\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"type\":\"mesh3d\"}],\"parcoords\":[{\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"parcoords\"}],\"pie\":[{\"automargin\":true,\"type\":\"pie\"}],\"scatter3d\":[{\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scatter3d\"}],\"scattercarpet\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scattercarpet\"}],\"scattergeo\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scattergeo\"}],\"scattergl\":[{\"marker\":{\"line\":{\"color\":\"#283442\"}},\"type\":\"scattergl\"}],\"scattermapbox\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scattermapbox\"}],\"scatterpolargl\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scatterpolargl\"}],\"scatterpolar\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scatterpolar\"}],\"scatter\":[{\"marker\":{\"line\":{\"color\":\"#283442\"}},\"type\":\"scatter\"}],\"scatterternary\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scatterternary\"}],\"surface\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"surface\"}],\"table\":[{\"cells\":{\"fill\":{\"color\":\"#506784\"},\"line\":{\"color\":\"rgb(17,17,17)\"}},\"header\":{\"fill\":{\"color\":\"#2a3f5f\"},\"line\":{\"color\":\"rgb(17,17,17)\"}},\"type\":\"table\"}]},\"layout\":{\"annotationdefaults\":{\"arrowcolor\":\"#f2f5fa\",\"arrowhead\":0,\"arrowwidth\":1},\"autotypenumbers\":\"strict\",\"coloraxis\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"colorscale\":{\"diverging\":[[0,\"#8e0152\"],[0.1,\"#c51b7d\"],[0.2,\"#de77ae\"],[0.3,\"#f1b6da\"],[0.4,\"#fde0ef\"],[0.5,\"#f7f7f7\"],[0.6,\"#e6f5d0\"],[0.7,\"#b8e186\"],[0.8,\"#7fbc41\"],[0.9,\"#4d9221\"],[1,\"#276419\"]],\"sequential\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"sequentialminus\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]},\"colorway\":[\"#636efa\",\"#EF553B\",\"#00cc96\",\"#ab63fa\",\"#FFA15A\",\"#19d3f3\",\"#FF6692\",\"#B6E880\",\"#FF97FF\",\"#FECB52\"],\"font\":{\"color\":\"#f2f5fa\"},\"geo\":{\"bgcolor\":\"rgb(17,17,17)\",\"lakecolor\":\"rgb(17,17,17)\",\"landcolor\":\"rgb(17,17,17)\",\"showlakes\":true,\"showland\":true,\"subunitcolor\":\"#506784\"},\"hoverlabel\":{\"align\":\"left\"},\"hovermode\":\"closest\",\"mapbox\":{\"style\":\"dark\"},\"paper_bgcolor\":\"rgb(17,17,17)\",\"plot_bgcolor\":\"rgb(17,17,17)\",\"polar\":{\"angularaxis\":{\"gridcolor\":\"#506784\",\"linecolor\":\"#506784\",\"ticks\":\"\"},\"bgcolor\":\"rgb(17,17,17)\",\"radialaxis\":{\"gridcolor\":\"#506784\",\"linecolor\":\"#506784\",\"ticks\":\"\"}},\"scene\":{\"xaxis\":{\"backgroundcolor\":\"rgb(17,17,17)\",\"gridcolor\":\"#506784\",\"gridwidth\":2,\"linecolor\":\"#506784\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"#C8D4E3\"},\"yaxis\":{\"backgroundcolor\":\"rgb(17,17,17)\",\"gridcolor\":\"#506784\",\"gridwidth\":2,\"linecolor\":\"#506784\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"#C8D4E3\"},\"zaxis\":{\"backgroundcolor\":\"rgb(17,17,17)\",\"gridcolor\":\"#506784\",\"gridwidth\":2,\"linecolor\":\"#506784\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"#C8D4E3\"}},\"shapedefaults\":{\"line\":{\"color\":\"#f2f5fa\"}},\"sliderdefaults\":{\"bgcolor\":\"#C8D4E3\",\"bordercolor\":\"rgb(17,17,17)\",\"borderwidth\":1,\"tickwidth\":0},\"ternary\":{\"aaxis\":{\"gridcolor\":\"#506784\",\"linecolor\":\"#506784\",\"ticks\":\"\"},\"baxis\":{\"gridcolor\":\"#506784\",\"linecolor\":\"#506784\",\"ticks\":\"\"},\"bgcolor\":\"rgb(17,17,17)\",\"caxis\":{\"gridcolor\":\"#506784\",\"linecolor\":\"#506784\",\"ticks\":\"\"}},\"title\":{\"x\":0.05},\"updatemenudefaults\":{\"bgcolor\":\"#506784\",\"borderwidth\":0},\"xaxis\":{\"automargin\":true,\"gridcolor\":\"#283442\",\"linecolor\":\"#506784\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"#283442\",\"zerolinewidth\":2},\"yaxis\":{\"automargin\":true,\"gridcolor\":\"#283442\",\"linecolor\":\"#506784\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"#283442\",\"zerolinewidth\":2}}},\"xaxis\":{\"anchor\":\"y\",\"domain\":[0.0,1.0],\"scaleanchor\":\"y\",\"constrain\":\"domain\",\"title\":{\"text\":\"Features\"},\"side\":\"top\"},\"yaxis\":{\"anchor\":\"x\",\"domain\":[0.0,1.0],\"autorange\":\"reversed\",\"constrain\":\"domain\",\"title\":{\"text\":\"Features\"}},\"coloraxis\":{\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"showscale\":false},\"margin\":{\"t\":60},\"title\":{\"text\":\"Correlation Matrix\"},\"paper_bgcolor\":\"rgba(0, 0, 0, 0)\",\"plot_bgcolor\":\"rgba(0, 0, 0, 0)\"},                        {\"responsive\": true}                    ).then(function(){\n",
              "                            \n",
              "var gd = document.getElementById('a6284208-0101-4ed3-a0d9-21ce3a1258b1');\n",
              "var x = new MutationObserver(function (mutations, observer) {{\n",
              "        var display = window.getComputedStyle(gd).display;\n",
              "        if (!display || display === 'none') {{\n",
              "            console.log([gd, 'removed!']);\n",
              "            Plotly.purge(gd);\n",
              "            observer.disconnect();\n",
              "        }}\n",
              "}});\n",
              "\n",
              "// Listen for the removal of the full notebook cells\n",
              "var notebookContainer = gd.closest('#notebook-container');\n",
              "if (notebookContainer) {{\n",
              "    x.observe(notebookContainer, {childList: true});\n",
              "}}\n",
              "\n",
              "// Listen for the clearing of the current output cell\n",
              "var outputEl = gd.closest('.output');\n",
              "if (outputEl) {{\n",
              "    x.observe(outputEl, {childList: true});\n",
              "}}\n",
              "\n",
              "                        })                };                            </script>        </div>\n",
              "</body>\n",
              "</html>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        }
      ],
      "source": [
        "from sklearn.preprocessing import LabelEncoder\n",
        "import plotly.express as px\n",
        "\n",
        "corr_df = df.copy()\n",
        "\n",
        "le = LabelEncoder()\n",
        "\n",
        "for column in list(set(df.columns) - set(df._get_numeric_data().columns)):\n",
        "    corr_df[column] = le.fit_transform(corr_df[column])\n",
        "\n",
        "fig = px.imshow(corr_df.corr(numeric_only=True).values,\n",
        "                labels=dict(x=\"Features\", y=\"Features\"),\n",
        "                x=list(corr_df.columns),\n",
        "                y=list(corr_df.columns),\n",
        "                text_auto=True\n",
        "                )\n",
        "\n",
        "fig.update_xaxes(side='top')\n",
        "\n",
        "fig.update_layout(template='plotly_dark',\n",
        "                  title='Correlation Matrix',\n",
        "                  coloraxis_showscale=False,\n",
        "                  paper_bgcolor='rgba(0, 0, 0, 0)',\n",
        "                  plot_bgcolor='rgba(0, 0, 0, 0)')\n",
        "fig.show()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "AqaYBiGGDfZn"
      },
      "source": [
        "According to the correlation scores, `race` has an alarming 0.22 correlation with our label. Let us see how this will impact our future model.\n",
        "\n",
        "We will create two classifiers to deal with the classification problem: a `RandomForestClassifier` and a `LogisticRegressor`. We will make two classifiers to (1) compare their performance and (2) because analysis of coefficients is not possible with forest-type classifiers (decision trees)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 31,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "dVGRCAkqDfZo",
        "outputId": "e1092874-ee76-446f-fd7d-b201671dd959"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Accuracy (Random Forest): 76.94 %\n"
          ]
        }
      ],
      "source": [
        "from sklearn.preprocessing import StandardScaler, OneHotEncoder\n",
        "from sklearn.model_selection import train_test_split\n",
        "from sklearn.compose import make_column_transformer\n",
        "from sklearn.pipeline import make_pipeline\n",
        "\n",
        "# Create a seed for reproducibility\n",
        "seed = 42\n",
        "\n",
        "# Define features and labels\n",
        "X, y = df[df.columns.values.tolist()[0:12]], df[df.columns.values.tolist()[-1]]\n",
        "\n",
        "# Split the dataset\n",
        "X_train, X_test, y_train, y_test = train_test_split(\n",
        "    X,\n",
        "    y,\n",
        "    test_size=0.2,\n",
        "    random_state=seed\n",
        ")\n",
        "\n",
        "# Preprocess features\n",
        "preprocess = make_column_transformer(\n",
        "    (StandardScaler(), ['juv_fel_count', 'juv_misd_count',\n",
        "    'juv_other_count', 'priors_count', 'days_b_screening_arrest',\n",
        "    'c_days_from_compas']),\n",
        "    (OneHotEncoder(), ['sex', 'age_cat', 'race', 'c_charge_degree',\n",
        "    'is_recid', 'is_violent_recid']))\n",
        "\n",
        "from sklearn.ensemble import RandomForestClassifier\n",
        "\n",
        "# Create an instance of a `RandomForestClassifier`\n",
        "model_rf = make_pipeline(\n",
        "    preprocess,\n",
        "    RandomForestClassifier(max_depth=3, n_estimators=500))\n",
        "\n",
        "# Fit the model\n",
        "model_rf.fit(X_train, y_train.values.ravel())\n",
        "\n",
        "# Evaluate the model\n",
        "score = model_rf.score(X_test, y_test.values.ravel())\n",
        "\n",
        "print(f'Accuracy (Random Forest): ' + '{:.2f}'.format(score * 100) + ' %')"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 32,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 559
        },
        "id": "SbIG-KaBDfZo",
        "outputId": "e6c3cbc1-ef1d-4d7f-cd68-11cffb23d6d7"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Accuracy (Logistic Regression): 80.20 %\n"
          ]
        },
        {
          "data": {
            "text/html": [
              "<html>\n",
              "<head><meta charset=\"utf-8\" /></head>\n",
              "<body>\n",
              "    <div>            <script src=\"https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/MathJax.js?config=TeX-AMS-MML_SVG\"></script><script type=\"text/javascript\">if (window.MathJax && window.MathJax.Hub && window.MathJax.Hub.Config) {window.MathJax.Hub.Config({SVG: {font: \"STIX-Web\"}});}</script>                <script type=\"text/javascript\">window.PlotlyConfig = {MathJaxConfig: 'local'};</script>\n",
              "        <script charset=\"utf-8\" src=\"https://cdn.plot.ly/plotly-2.24.1.min.js\"></script>                <div id=\"967130bb-7f16-47fc-9d3a-cabbd29c9f5b\" class=\"plotly-graph-div\" style=\"height:525px; width:100%;\"></div>            <script type=\"text/javascript\">                                    window.PLOTLYENV=window.PLOTLYENV || {};                                    if (document.getElementById(\"967130bb-7f16-47fc-9d3a-cabbd29c9f5b\")) {                    Plotly.newPlot(                        \"967130bb-7f16-47fc-9d3a-cabbd29c9f5b\",                        [{\"coloraxis\":\"coloraxis\",\"name\":\"0\",\"texttemplate\":\"%{z}\",\"x\":[\"High\",\"Low\"],\"y\":[\"High\",\"Low\"],\"z\":[[299,554],[120,2431]],\"type\":\"heatmap\",\"xaxis\":\"x\",\"yaxis\":\"y\",\"hovertemplate\":\"Predicted: %{x}\\u003cbr\\u003eTrue label: %{y}\\u003cbr\\u003ecolor: %{z}\\u003cextra\\u003e\\u003c\\u002fextra\\u003e\"}],                        {\"template\":{\"data\":{\"barpolar\":[{\"marker\":{\"line\":{\"color\":\"rgb(17,17,17)\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"barpolar\"}],\"bar\":[{\"error_x\":{\"color\":\"#f2f5fa\"},\"error_y\":{\"color\":\"#f2f5fa\"},\"marker\":{\"line\":{\"color\":\"rgb(17,17,17)\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"bar\"}],\"carpet\":[{\"aaxis\":{\"endlinecolor\":\"#A2B1C6\",\"gridcolor\":\"#506784\",\"linecolor\":\"#506784\",\"minorgridcolor\":\"#506784\",\"startlinecolor\":\"#A2B1C6\"},\"baxis\":{\"endlinecolor\":\"#A2B1C6\",\"gridcolor\":\"#506784\",\"linecolor\":\"#506784\",\"minorgridcolor\":\"#506784\",\"startlinecolor\":\"#A2B1C6\"},\"type\":\"carpet\"}],\"choropleth\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"type\":\"choropleth\"}],\"contourcarpet\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"type\":\"contourcarpet\"}],\"contour\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"contour\"}],\"heatmapgl\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"heatmapgl\"}],\"heatmap\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"heatmap\"}],\"histogram2dcontour\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"histogram2dcontour\"}],\"histogram2d\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"histogram2d\"}],\"histogram\":[{\"marker\":{\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"histogram\"}],\"mesh3d\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"type\":\"mesh3d\"}],\"parcoords\":[{\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"parcoords\"}],\"pie\":[{\"automargin\":true,\"type\":\"pie\"}],\"scatter3d\":[{\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scatter3d\"}],\"scattercarpet\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scattercarpet\"}],\"scattergeo\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scattergeo\"}],\"scattergl\":[{\"marker\":{\"line\":{\"color\":\"#283442\"}},\"type\":\"scattergl\"}],\"scattermapbox\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scattermapbox\"}],\"scatterpolargl\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scatterpolargl\"}],\"scatterpolar\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scatterpolar\"}],\"scatter\":[{\"marker\":{\"line\":{\"color\":\"#283442\"}},\"type\":\"scatter\"}],\"scatterternary\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scatterternary\"}],\"surface\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"surface\"}],\"table\":[{\"cells\":{\"fill\":{\"color\":\"#506784\"},\"line\":{\"color\":\"rgb(17,17,17)\"}},\"header\":{\"fill\":{\"color\":\"#2a3f5f\"},\"line\":{\"color\":\"rgb(17,17,17)\"}},\"type\":\"table\"}]},\"layout\":{\"annotationdefaults\":{\"arrowcolor\":\"#f2f5fa\",\"arrowhead\":0,\"arrowwidth\":1},\"autotypenumbers\":\"strict\",\"coloraxis\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"colorscale\":{\"diverging\":[[0,\"#8e0152\"],[0.1,\"#c51b7d\"],[0.2,\"#de77ae\"],[0.3,\"#f1b6da\"],[0.4,\"#fde0ef\"],[0.5,\"#f7f7f7\"],[0.6,\"#e6f5d0\"],[0.7,\"#b8e186\"],[0.8,\"#7fbc41\"],[0.9,\"#4d9221\"],[1,\"#276419\"]],\"sequential\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"sequentialminus\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]},\"colorway\":[\"#636efa\",\"#EF553B\",\"#00cc96\",\"#ab63fa\",\"#FFA15A\",\"#19d3f3\",\"#FF6692\",\"#B6E880\",\"#FF97FF\",\"#FECB52\"],\"font\":{\"color\":\"#f2f5fa\"},\"geo\":{\"bgcolor\":\"rgb(17,17,17)\",\"lakecolor\":\"rgb(17,17,17)\",\"landcolor\":\"rgb(17,17,17)\",\"showlakes\":true,\"showland\":true,\"subunitcolor\":\"#506784\"},\"hoverlabel\":{\"align\":\"left\"},\"hovermode\":\"closest\",\"mapbox\":{\"style\":\"dark\"},\"paper_bgcolor\":\"rgb(17,17,17)\",\"plot_bgcolor\":\"rgb(17,17,17)\",\"polar\":{\"angularaxis\":{\"gridcolor\":\"#506784\",\"linecolor\":\"#506784\",\"ticks\":\"\"},\"bgcolor\":\"rgb(17,17,17)\",\"radialaxis\":{\"gridcolor\":\"#506784\",\"linecolor\":\"#506784\",\"ticks\":\"\"}},\"scene\":{\"xaxis\":{\"backgroundcolor\":\"rgb(17,17,17)\",\"gridcolor\":\"#506784\",\"gridwidth\":2,\"linecolor\":\"#506784\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"#C8D4E3\"},\"yaxis\":{\"backgroundcolor\":\"rgb(17,17,17)\",\"gridcolor\":\"#506784\",\"gridwidth\":2,\"linecolor\":\"#506784\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"#C8D4E3\"},\"zaxis\":{\"backgroundcolor\":\"rgb(17,17,17)\",\"gridcolor\":\"#506784\",\"gridwidth\":2,\"linecolor\":\"#506784\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"#C8D4E3\"}},\"shapedefaults\":{\"line\":{\"color\":\"#f2f5fa\"}},\"sliderdefaults\":{\"bgcolor\":\"#C8D4E3\",\"bordercolor\":\"rgb(17,17,17)\",\"borderwidth\":1,\"tickwidth\":0},\"ternary\":{\"aaxis\":{\"gridcolor\":\"#506784\",\"linecolor\":\"#506784\",\"ticks\":\"\"},\"baxis\":{\"gridcolor\":\"#506784\",\"linecolor\":\"#506784\",\"ticks\":\"\"},\"bgcolor\":\"rgb(17,17,17)\",\"caxis\":{\"gridcolor\":\"#506784\",\"linecolor\":\"#506784\",\"ticks\":\"\"}},\"title\":{\"x\":0.05},\"updatemenudefaults\":{\"bgcolor\":\"#506784\",\"borderwidth\":0},\"xaxis\":{\"automargin\":true,\"gridcolor\":\"#283442\",\"linecolor\":\"#506784\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"#283442\",\"zerolinewidth\":2},\"yaxis\":{\"automargin\":true,\"gridcolor\":\"#283442\",\"linecolor\":\"#506784\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"#283442\",\"zerolinewidth\":2}}},\"xaxis\":{\"anchor\":\"y\",\"domain\":[0.0,1.0],\"scaleanchor\":\"y\",\"constrain\":\"domain\",\"title\":{\"text\":\"Predicted\"},\"side\":\"top\"},\"yaxis\":{\"anchor\":\"x\",\"domain\":[0.0,1.0],\"autorange\":\"reversed\",\"constrain\":\"domain\",\"title\":{\"text\":\"True label\"}},\"coloraxis\":{\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"showscale\":false},\"margin\":{\"t\":60},\"title\":{\"text\":\"Confusion Matrix (Logistic Regression Model)\"},\"paper_bgcolor\":\"rgba(0, 0, 0, 0)\",\"plot_bgcolor\":\"rgba(0, 0, 0, 0)\"},                        {\"responsive\": true}                    ).then(function(){\n",
              "                            \n",
              "var gd = document.getElementById('967130bb-7f16-47fc-9d3a-cabbd29c9f5b');\n",
              "var x = new MutationObserver(function (mutations, observer) {{\n",
              "        var display = window.getComputedStyle(gd).display;\n",
              "        if (!display || display === 'none') {{\n",
              "            console.log([gd, 'removed!']);\n",
              "            Plotly.purge(gd);\n",
              "            observer.disconnect();\n",
              "        }}\n",
              "}});\n",
              "\n",
              "// Listen for the removal of the full notebook cells\n",
              "var notebookContainer = gd.closest('#notebook-container');\n",
              "if (notebookContainer) {{\n",
              "    x.observe(notebookContainer, {childList: true});\n",
              "}}\n",
              "\n",
              "// Listen for the clearing of the current output cell\n",
              "var outputEl = gd.closest('.output');\n",
              "if (outputEl) {{\n",
              "    x.observe(outputEl, {childList: true});\n",
              "}}\n",
              "\n",
              "                        })                };                            </script>        </div>\n",
              "</body>\n",
              "</html>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        }
      ],
      "source": [
        "from sklearn.linear_model import LogisticRegression\n",
        "from sklearn.metrics import confusion_matrix\n",
        "import plotly.express as px\n",
        "\n",
        "# Create an instance of a `LogisticRegression`\n",
        "model_lr = make_pipeline(\n",
        "    preprocess,\n",
        "    LogisticRegression(penalty='l2', max_iter= 500))\n",
        "\n",
        "# Fit the model\n",
        "model_lr.fit(X_train, y_train.values.ravel())\n",
        "\n",
        "# Evaluate the model\n",
        "score = model_lr.score(X_test, y_test.values.ravel())\n",
        "\n",
        "print(f'Accuracy (Logistic Regression): ' +\n",
        "      '{:.2f}'.format(score * 100) + ' %')\n",
        "\n",
        "# Plot results as a confusion matrix\n",
        "preds = model_lr.predict(X_test)\n",
        "matrix = confusion_matrix(y_test.values.ravel(), preds)\n",
        "\n",
        "fig = px.imshow(matrix,\n",
        "                labels=dict(x=\"Predicted\", y=\"True label\"),\n",
        "                x=['High', 'Low'],\n",
        "                y=['High', 'Low'],\n",
        "                text_auto=True\n",
        "                )\n",
        "\n",
        "fig.update_xaxes(side='top')\n",
        "\n",
        "fig.update_layout(template='plotly_dark',\n",
        "                  title='Confusion Matrix (Logistic Regression Model)',\n",
        "                  coloraxis_showscale=False,\n",
        "                  paper_bgcolor='rgba(0, 0, 0, 0)',\n",
        "                  plot_bgcolor='rgba(0, 0, 0, 0)')\n",
        "\n",
        "fig.show()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "g3Yr0qmRDfZo"
      },
      "source": [
        "Accuracy varies considerably between classifiers. If we look at the confusion matrix of the second classifier, we see that the class the algorithm has the most trouble getting right is \"`High`\" (label = 0). Meanwhile, the algorithm's most significant error is in the False Negatives (individuals classified as \"`Low`\" who should be given \"`High`\" risk score).\n",
        "\n",
        "It is worth remembering that given the original distribution of the data (only 25% of samples are labeled \"`High`\"). Hence, to achieve 75% accuracy on this problem, it would be sufficient for the classifier to label all entries as \"`Low`.\"\n",
        "\n",
        "Let us now analyze the coefficients learned by the model during its training."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 35,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 1000
        },
        "id": "5Ai4Nn_TDfZo",
        "outputId": "a7c4da33-262a-43b3-e9ee-4d0e97b4c9a2"
      },
      "outputs": [
        {
          "data": {
            "text/markdown": [
              "|                                         |   Coefficients |\n",
              "|:----------------------------------------|---------------:|\n",
              "| standardscaler__juv_fel_count           |    -0.257249   |\n",
              "| standardscaler__juv_misd_count          |    -0.155662   |\n",
              "| standardscaler__juv_other_count         |    -0.121177   |\n",
              "| standardscaler__priors_count            |    -0.785661   |\n",
              "| standardscaler__days_b_screening_arrest |    -0.0396254  |\n",
              "| standardscaler__c_days_from_compas      |    -0.111371   |\n",
              "| onehotencoder__sex_Female               |     0.046411   |\n",
              "| onehotencoder__sex_Male                 |    -0.0527134  |\n",
              "| onehotencoder__age_cat_25 - 45          |    -0.0985223  |\n",
              "| onehotencoder__age_cat_Greater than 45  |     1.11599    |\n",
              "| onehotencoder__age_cat_Less than 25     |    -1.02377    |\n",
              "| onehotencoder__race_African-American    |    -0.631542   |\n",
              "| onehotencoder__race_Asian               |    -0.188078   |\n",
              "| onehotencoder__race_Caucasian           |    -0.0938192  |\n",
              "| onehotencoder__race_Hispanic            |     0.135407   |\n",
              "| onehotencoder__race_Native American     |    -0.349588   |\n",
              "| onehotencoder__race_Other               |     1.12132    |\n",
              "| onehotencoder__c_charge_degree_(CO3)    |     0.275411   |\n",
              "| onehotencoder__c_charge_degree_(F1)     |    -0.377484   |\n",
              "| onehotencoder__c_charge_degree_(F2)     |    -0.255121   |\n",
              "| onehotencoder__c_charge_degree_(F3)     |    -0.388378   |\n",
              "| onehotencoder__c_charge_degree_(F5)     |     1.09637    |\n",
              "| onehotencoder__c_charge_degree_(F6)     |     0.407385   |\n",
              "| onehotencoder__c_charge_degree_(F7)     |     0.00279768 |\n",
              "| onehotencoder__c_charge_degree_(M1)     |     0.091829   |\n",
              "| onehotencoder__c_charge_degree_(M2)     |    -0.184265   |\n",
              "| onehotencoder__c_charge_degree_(MO3)    |    -0.72925    |\n",
              "| onehotencoder__c_charge_degree_(NI0)    |    -0.351153   |\n",
              "| onehotencoder__c_charge_degree_(TCX)    |     0.231925   |\n",
              "| onehotencoder__c_charge_degree_(X)      |     0.173631   |\n",
              "| onehotencoder__is_recid_-1              |     0.0177461  |\n",
              "| onehotencoder__is_recid_0               |     0.179593   |\n",
              "| onehotencoder__is_recid_1               |    -0.203641   |\n",
              "| onehotencoder__is_violent_recid_0       |     0.14288    |\n",
              "| onehotencoder__is_violent_recid_1       |    -0.149182   |"
            ],
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/html": [
              "<html>\n",
              "<head><meta charset=\"utf-8\" /></head>\n",
              "<body>\n",
              "    <div>            <script src=\"https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/MathJax.js?config=TeX-AMS-MML_SVG\"></script><script type=\"text/javascript\">if (window.MathJax && window.MathJax.Hub && window.MathJax.Hub.Config) {window.MathJax.Hub.Config({SVG: {font: \"STIX-Web\"}});}</script>                <script type=\"text/javascript\">window.PlotlyConfig = {MathJaxConfig: 'local'};</script>\n",
              "        <script charset=\"utf-8\" src=\"https://cdn.plot.ly/plotly-2.24.1.min.js\"></script>                <div id=\"5d0a942a-fe4c-4be6-913f-a0d2fa4b7197\" class=\"plotly-graph-div\" style=\"height:525px; width:100%;\"></div>            <script type=\"text/javascript\">                                    window.PLOTLYENV=window.PLOTLYENV || {};                                    if (document.getElementById(\"5d0a942a-fe4c-4be6-913f-a0d2fa4b7197\")) {                    Plotly.newPlot(                        \"5d0a942a-fe4c-4be6-913f-a0d2fa4b7197\",                        [{\"orientation\":\"h\",\"x\":[-0.2572487105253267,-0.15566196631815787,-0.12117688569734114,-0.7856606637155377,-0.039625392405417216,-0.11137119888640214,0.0464110259372118,-0.052713440693610954,-0.09852226502253587,1.1159853448645338,-1.0237654945984562,-0.6315419272677828,-0.1880775095011791,-0.09381923051593317,0.1354067579828832,-0.3495879307789132,1.1213174253245548,0.2754110228727997,-0.377483724093868,-0.25512057624524737,-0.38837779490191116,1.096368583316851,0.4073851962421299,0.002797684298419269,0.09182898427837922,-0.18426544244991505,-0.7292495403876537,-0.3511532920826123,0.23192509812967402,0.1736313862665463,0.01774613718470398,0.17959289445105556,-0.20364144639213108,0.14288003025613372,-0.14918244501253092],\"y\":[\"standardscaler__juv_fel_count\",\"standardscaler__juv_misd_count\",\"standardscaler__juv_other_count\",\"standardscaler__priors_count\",\"standardscaler__days_b_screening_arrest\",\"standardscaler__c_days_from_compas\",\"onehotencoder__sex_Female\",\"onehotencoder__sex_Male\",\"onehotencoder__age_cat_25 - 45\",\"onehotencoder__age_cat_Greater than 45\",\"onehotencoder__age_cat_Less than 25\",\"onehotencoder__race_African-American\",\"onehotencoder__race_Asian\",\"onehotencoder__race_Caucasian\",\"onehotencoder__race_Hispanic\",\"onehotencoder__race_Native American\",\"onehotencoder__race_Other\",\"onehotencoder__c_charge_degree_(CO3)\",\"onehotencoder__c_charge_degree_(F1)\",\"onehotencoder__c_charge_degree_(F2)\",\"onehotencoder__c_charge_degree_(F3)\",\"onehotencoder__c_charge_degree_(F5)\",\"onehotencoder__c_charge_degree_(F6)\",\"onehotencoder__c_charge_degree_(F7)\",\"onehotencoder__c_charge_degree_(M1)\",\"onehotencoder__c_charge_degree_(M2)\",\"onehotencoder__c_charge_degree_(MO3)\",\"onehotencoder__c_charge_degree_(NI0)\",\"onehotencoder__c_charge_degree_(TCX)\",\"onehotencoder__c_charge_degree_(X)\",\"onehotencoder__is_recid_-1\",\"onehotencoder__is_recid_0\",\"onehotencoder__is_recid_1\",\"onehotencoder__is_violent_recid_0\",\"onehotencoder__is_violent_recid_1\"],\"type\":\"bar\"}],                        {\"template\":{\"data\":{\"barpolar\":[{\"marker\":{\"line\":{\"color\":\"rgb(17,17,17)\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"barpolar\"}],\"bar\":[{\"error_x\":{\"color\":\"#f2f5fa\"},\"error_y\":{\"color\":\"#f2f5fa\"},\"marker\":{\"line\":{\"color\":\"rgb(17,17,17)\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"bar\"}],\"carpet\":[{\"aaxis\":{\"endlinecolor\":\"#A2B1C6\",\"gridcolor\":\"#506784\",\"linecolor\":\"#506784\",\"minorgridcolor\":\"#506784\",\"startlinecolor\":\"#A2B1C6\"},\"baxis\":{\"endlinecolor\":\"#A2B1C6\",\"gridcolor\":\"#506784\",\"linecolor\":\"#506784\",\"minorgridcolor\":\"#506784\",\"startlinecolor\":\"#A2B1C6\"},\"type\":\"carpet\"}],\"choropleth\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"type\":\"choropleth\"}],\"contourcarpet\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"type\":\"contourcarpet\"}],\"contour\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"contour\"}],\"heatmapgl\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"heatmapgl\"}],\"heatmap\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"heatmap\"}],\"histogram2dcontour\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"histogram2dcontour\"}],\"histogram2d\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"histogram2d\"}],\"histogram\":[{\"marker\":{\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"histogram\"}],\"mesh3d\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"type\":\"mesh3d\"}],\"parcoords\":[{\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"parcoords\"}],\"pie\":[{\"automargin\":true,\"type\":\"pie\"}],\"scatter3d\":[{\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scatter3d\"}],\"scattercarpet\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scattercarpet\"}],\"scattergeo\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scattergeo\"}],\"scattergl\":[{\"marker\":{\"line\":{\"color\":\"#283442\"}},\"type\":\"scattergl\"}],\"scattermapbox\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scattermapbox\"}],\"scatterpolargl\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scatterpolargl\"}],\"scatterpolar\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scatterpolar\"}],\"scatter\":[{\"marker\":{\"line\":{\"color\":\"#283442\"}},\"type\":\"scatter\"}],\"scatterternary\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scatterternary\"}],\"surface\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"surface\"}],\"table\":[{\"cells\":{\"fill\":{\"color\":\"#506784\"},\"line\":{\"color\":\"rgb(17,17,17)\"}},\"header\":{\"fill\":{\"color\":\"#2a3f5f\"},\"line\":{\"color\":\"rgb(17,17,17)\"}},\"type\":\"table\"}]},\"layout\":{\"annotationdefaults\":{\"arrowcolor\":\"#f2f5fa\",\"arrowhead\":0,\"arrowwidth\":1},\"autotypenumbers\":\"strict\",\"coloraxis\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"colorscale\":{\"diverging\":[[0,\"#8e0152\"],[0.1,\"#c51b7d\"],[0.2,\"#de77ae\"],[0.3,\"#f1b6da\"],[0.4,\"#fde0ef\"],[0.5,\"#f7f7f7\"],[0.6,\"#e6f5d0\"],[0.7,\"#b8e186\"],[0.8,\"#7fbc41\"],[0.9,\"#4d9221\"],[1,\"#276419\"]],\"sequential\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"sequentialminus\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]},\"colorway\":[\"#636efa\",\"#EF553B\",\"#00cc96\",\"#ab63fa\",\"#FFA15A\",\"#19d3f3\",\"#FF6692\",\"#B6E880\",\"#FF97FF\",\"#FECB52\"],\"font\":{\"color\":\"#f2f5fa\"},\"geo\":{\"bgcolor\":\"rgb(17,17,17)\",\"lakecolor\":\"rgb(17,17,17)\",\"landcolor\":\"rgb(17,17,17)\",\"showlakes\":true,\"showland\":true,\"subunitcolor\":\"#506784\"},\"hoverlabel\":{\"align\":\"left\"},\"hovermode\":\"closest\",\"mapbox\":{\"style\":\"dark\"},\"paper_bgcolor\":\"rgb(17,17,17)\",\"plot_bgcolor\":\"rgb(17,17,17)\",\"polar\":{\"angularaxis\":{\"gridcolor\":\"#506784\",\"linecolor\":\"#506784\",\"ticks\":\"\"},\"bgcolor\":\"rgb(17,17,17)\",\"radialaxis\":{\"gridcolor\":\"#506784\",\"linecolor\":\"#506784\",\"ticks\":\"\"}},\"scene\":{\"xaxis\":{\"backgroundcolor\":\"rgb(17,17,17)\",\"gridcolor\":\"#506784\",\"gridwidth\":2,\"linecolor\":\"#506784\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"#C8D4E3\"},\"yaxis\":{\"backgroundcolor\":\"rgb(17,17,17)\",\"gridcolor\":\"#506784\",\"gridwidth\":2,\"linecolor\":\"#506784\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"#C8D4E3\"},\"zaxis\":{\"backgroundcolor\":\"rgb(17,17,17)\",\"gridcolor\":\"#506784\",\"gridwidth\":2,\"linecolor\":\"#506784\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"#C8D4E3\"}},\"shapedefaults\":{\"line\":{\"color\":\"#f2f5fa\"}},\"sliderdefaults\":{\"bgcolor\":\"#C8D4E3\",\"bordercolor\":\"rgb(17,17,17)\",\"borderwidth\":1,\"tickwidth\":0},\"ternary\":{\"aaxis\":{\"gridcolor\":\"#506784\",\"linecolor\":\"#506784\",\"ticks\":\"\"},\"baxis\":{\"gridcolor\":\"#506784\",\"linecolor\":\"#506784\",\"ticks\":\"\"},\"bgcolor\":\"rgb(17,17,17)\",\"caxis\":{\"gridcolor\":\"#506784\",\"linecolor\":\"#506784\",\"ticks\":\"\"}},\"title\":{\"x\":0.05},\"updatemenudefaults\":{\"bgcolor\":\"#506784\",\"borderwidth\":0},\"xaxis\":{\"automargin\":true,\"gridcolor\":\"#283442\",\"linecolor\":\"#506784\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"#283442\",\"zerolinewidth\":2},\"yaxis\":{\"automargin\":true,\"gridcolor\":\"#283442\",\"linecolor\":\"#506784\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"#283442\",\"zerolinewidth\":2}}},\"xaxis\":{\"range\":[-1.1261420440583019,1.2334491678570103],\"tickmode\":\"linear\",\"tick0\":0,\"dtick\":0.5},\"title\":{\"text\":\"LogisticRegression Coefficients\"},\"paper_bgcolor\":\"rgba(0, 0, 0, 0)\",\"plot_bgcolor\":\"rgba(0, 0, 0, 0)\"},                        {\"responsive\": true}                    ).then(function(){\n",
              "                            \n",
              "var gd = document.getElementById('5d0a942a-fe4c-4be6-913f-a0d2fa4b7197');\n",
              "var x = new MutationObserver(function (mutations, observer) {{\n",
              "        var display = window.getComputedStyle(gd).display;\n",
              "        if (!display || display === 'none') {{\n",
              "            console.log([gd, 'removed!']);\n",
              "            Plotly.purge(gd);\n",
              "            observer.disconnect();\n",
              "        }}\n",
              "}});\n",
              "\n",
              "// Listen for the removal of the full notebook cells\n",
              "var notebookContainer = gd.closest('#notebook-container');\n",
              "if (notebookContainer) {{\n",
              "    x.observe(notebookContainer, {childList: true});\n",
              "}}\n",
              "\n",
              "// Listen for the clearing of the current output cell\n",
              "var outputEl = gd.closest('.output');\n",
              "if (outputEl) {{\n",
              "    x.observe(outputEl, {childList: true});\n",
              "}}\n",
              "\n",
              "                        })                };                            </script>        </div>\n",
              "</body>\n",
              "</html>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        }
      ],
      "source": [
        "import plotly.graph_objects as go\n",
        "\n",
        "# Get the model's coefficients and turn them into a data frame\n",
        "coefs = pd.DataFrame(\n",
        "    model_lr[-1].coef_,\n",
        "    columns=model_lr[:-1].get_feature_names_out(),\n",
        "    index=['Coefficients']).transpose()\n",
        "\n",
        "display(coefs)\n",
        "\n",
        "# Create a Go.figure\n",
        "fig = go.Figure(go.Bar(\n",
        "    x=coefs['Coefficients'],\n",
        "    y=model_lr[:-1].get_feature_names_out(),\n",
        "    orientation='h'))\n",
        "\n",
        "# Updated the range of the x axis\n",
        "fig.update_xaxes(range=[model_lr[-1].coef_.min(\n",
        ") + (model_lr[-1].coef_.min() * 0.1), model_lr[-1].coef_.max() + (model_lr[-1].coef_.max() * 0.1)])\n",
        "\n",
        "# Updated the layout of the figure\n",
        "fig.update_layout(\n",
        "    xaxis=dict(\n",
        "        tickmode='linear',\n",
        "        tick0=0,\n",
        "        dtick=0.5\n",
        "    ),\n",
        "    template='plotly_dark',\n",
        "    title_text='LogisticRegression Coefficients',\n",
        "    paper_bgcolor='rgba(0, 0, 0, 0)',\n",
        "    plot_bgcolor='rgba(0, 0, 0, 0)'\n",
        "\n",
        ")\n",
        "\n",
        "fig.show()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "LH5baE_aDfZp"
      },
      "source": [
        "At first glance, several things contribute equally to this classifier.\n",
        "\n",
        "However, we need to be cautious when interpreting coefficients from linear models. Since each feature represents a measured quantity on its scale, it doesn't make sense to compare them:\n",
        "\n",
        "- Age can range from, e.g., 16 to 100, but binary features only from 0 to 1. **This does not mean that an age of 100 has 100 times more weight than a feature with gender.**\n",
        "\n",
        "To get a correct view, we first need to normalize these values by their `standard deviation` (which brings all the values to a standard scale)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 36,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 1000
        },
        "id": "6hhC9D1TDfZp",
        "outputId": "dde8568f-c7b9-410c-d3eb-57ca5a9f698a"
      },
      "outputs": [
        {
          "data": {
            "text/html": [
              "\n",
              "  <div id=\"df-8a14d872-395f-408b-a1eb-dde8f2b7d275\" class=\"colab-df-container\">\n",
              "    <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Coefficients (Normalized by STD)</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>standardscaler__juv_fel_count</th>\n",
              "      <td>-0.257258</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>standardscaler__juv_misd_count</th>\n",
              "      <td>-0.155668</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>standardscaler__juv_other_count</th>\n",
              "      <td>-0.121181</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>standardscaler__priors_count</th>\n",
              "      <td>-0.785690</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>standardscaler__days_b_screening_arrest</th>\n",
              "      <td>-0.039627</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>standardscaler__c_days_from_compas</th>\n",
              "      <td>-0.111375</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>onehotencoder__sex_Female</th>\n",
              "      <td>0.017934</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>onehotencoder__sex_Male</th>\n",
              "      <td>-0.020369</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>onehotencoder__age_cat_25 - 45</th>\n",
              "      <td>-0.048760</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>onehotencoder__age_cat_Greater than 45</th>\n",
              "      <td>0.442049</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>onehotencoder__age_cat_Less than 25</th>\n",
              "      <td>-0.433359</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>onehotencoder__race_African-American</th>\n",
              "      <td>-0.314890</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>onehotencoder__race_Asian</th>\n",
              "      <td>-0.010795</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>onehotencoder__race_Caucasian</th>\n",
              "      <td>-0.044262</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>onehotencoder__race_Hispanic</th>\n",
              "      <td>0.035712</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>onehotencoder__race_Native American</th>\n",
              "      <td>-0.018200</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>onehotencoder__race_Other</th>\n",
              "      <td>0.236987</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>onehotencoder__c_charge_degree_(CO3)</th>\n",
              "      <td>0.005277</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>onehotencoder__c_charge_degree_(F1)</th>\n",
              "      <td>-0.047490</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>onehotencoder__c_charge_degree_(F2)</th>\n",
              "      <td>-0.071836</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>onehotencoder__c_charge_degree_(F3)</th>\n",
              "      <td>-0.190513</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>onehotencoder__c_charge_degree_(F5)</th>\n",
              "      <td>0.032536</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>onehotencoder__c_charge_degree_(F6)</th>\n",
              "      <td>0.011575</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>onehotencoder__c_charge_degree_(F7)</th>\n",
              "      <td>0.000285</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>onehotencoder__c_charge_degree_(M1)</th>\n",
              "      <td>0.037292</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>onehotencoder__c_charge_degree_(M2)</th>\n",
              "      <td>-0.047015</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>onehotencoder__c_charge_degree_(MO3)</th>\n",
              "      <td>-0.066452</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>onehotencoder__c_charge_degree_(NI0)</th>\n",
              "      <td>-0.009978</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>onehotencoder__c_charge_degree_(TCX)</th>\n",
              "      <td>0.001988</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>onehotencoder__c_charge_degree_(X)</th>\n",
              "      <td>0.001488</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>onehotencoder__is_recid_-1</th>\n",
              "      <td>0.000152</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>onehotencoder__is_recid_0</th>\n",
              "      <td>0.089780</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>onehotencoder__is_recid_1</th>\n",
              "      <td>-0.101802</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>onehotencoder__is_violent_recid_0</th>\n",
              "      <td>0.038153</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>onehotencoder__is_violent_recid_1</th>\n",
              "      <td>-0.039836</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "    <div class=\"colab-df-buttons\">\n",
              "\n",
              "  <div class=\"colab-df-container\">\n",
              "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-8a14d872-395f-408b-a1eb-dde8f2b7d275')\"\n",
              "            title=\"Convert this dataframe to an interactive table.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
              "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
              "  </svg>\n",
              "    </button>\n",
              "\n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    .colab-df-buttons div {\n",
              "      margin-bottom: 4px;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "    <script>\n",
              "      const buttonEl =\n",
              "        document.querySelector('#df-8a14d872-395f-408b-a1eb-dde8f2b7d275 button.colab-df-convert');\n",
              "      buttonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      async function convertToInteractive(key) {\n",
              "        const element = document.querySelector('#df-8a14d872-395f-408b-a1eb-dde8f2b7d275');\n",
              "        const dataTable =\n",
              "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                    [key], {});\n",
              "        if (!dataTable) return;\n",
              "\n",
              "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "          + ' to learn more about interactive tables.';\n",
              "        element.innerHTML = '';\n",
              "        dataTable['output_type'] = 'display_data';\n",
              "        await google.colab.output.renderOutput(dataTable, element);\n",
              "        const docLink = document.createElement('div');\n",
              "        docLink.innerHTML = docLinkHtml;\n",
              "        element.appendChild(docLink);\n",
              "      }\n",
              "    </script>\n",
              "  </div>\n",
              "\n",
              "\n",
              "<div id=\"df-bdaf5c2d-7044-4532-92b8-69e9f08f35c4\">\n",
              "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-bdaf5c2d-7044-4532-92b8-69e9f08f35c4')\"\n",
              "            title=\"Suggest charts\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "     width=\"24px\">\n",
              "    <g>\n",
              "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
              "    </g>\n",
              "</svg>\n",
              "  </button>\n",
              "\n",
              "<style>\n",
              "  .colab-df-quickchart {\n",
              "      --bg-color: #E8F0FE;\n",
              "      --fill-color: #1967D2;\n",
              "      --hover-bg-color: #E2EBFA;\n",
              "      --hover-fill-color: #174EA6;\n",
              "      --disabled-fill-color: #AAA;\n",
              "      --disabled-bg-color: #DDD;\n",
              "  }\n",
              "\n",
              "  [theme=dark] .colab-df-quickchart {\n",
              "      --bg-color: #3B4455;\n",
              "      --fill-color: #D2E3FC;\n",
              "      --hover-bg-color: #434B5C;\n",
              "      --hover-fill-color: #FFFFFF;\n",
              "      --disabled-bg-color: #3B4455;\n",
              "      --disabled-fill-color: #666;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart {\n",
              "    background-color: var(--bg-color);\n",
              "    border: none;\n",
              "    border-radius: 50%;\n",
              "    cursor: pointer;\n",
              "    display: none;\n",
              "    fill: var(--fill-color);\n",
              "    height: 32px;\n",
              "    padding: 0;\n",
              "    width: 32px;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart:hover {\n",
              "    background-color: var(--hover-bg-color);\n",
              "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "    fill: var(--button-hover-fill-color);\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart-complete:disabled,\n",
              "  .colab-df-quickchart-complete:disabled:hover {\n",
              "    background-color: var(--disabled-bg-color);\n",
              "    fill: var(--disabled-fill-color);\n",
              "    box-shadow: none;\n",
              "  }\n",
              "\n",
              "  .colab-df-spinner {\n",
              "    border: 2px solid var(--fill-color);\n",
              "    border-color: transparent;\n",
              "    border-bottom-color: var(--fill-color);\n",
              "    animation:\n",
              "      spin 1s steps(1) infinite;\n",
              "  }\n",
              "\n",
              "  @keyframes spin {\n",
              "    0% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "      border-left-color: var(--fill-color);\n",
              "    }\n",
              "    20% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    30% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    40% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    60% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    80% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "    90% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "  }\n",
              "</style>\n",
              "\n",
              "  <script>\n",
              "    async function quickchart(key) {\n",
              "      const quickchartButtonEl =\n",
              "        document.querySelector('#' + key + ' button');\n",
              "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
              "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
              "      try {\n",
              "        const charts = await google.colab.kernel.invokeFunction(\n",
              "            'suggestCharts', [key], {});\n",
              "      } catch (error) {\n",
              "        console.error('Error during call to suggestCharts:', error);\n",
              "      }\n",
              "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
              "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
              "    }\n",
              "    (() => {\n",
              "      let quickchartButtonEl =\n",
              "        document.querySelector('#df-bdaf5c2d-7044-4532-92b8-69e9f08f35c4 button');\n",
              "      quickchartButtonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "    })();\n",
              "  </script>\n",
              "</div>\n",
              "    </div>\n",
              "  </div>\n"
            ],
            "text/plain": [
              "                                         Coefficients (Normalized by STD)\n",
              "standardscaler__juv_fel_count                                   -0.257258\n",
              "standardscaler__juv_misd_count                                  -0.155668\n",
              "standardscaler__juv_other_count                                 -0.121181\n",
              "standardscaler__priors_count                                    -0.785690\n",
              "standardscaler__days_b_screening_arrest                         -0.039627\n",
              "standardscaler__c_days_from_compas                              -0.111375\n",
              "onehotencoder__sex_Female                                        0.017934\n",
              "onehotencoder__sex_Male                                         -0.020369\n",
              "onehotencoder__age_cat_25 - 45                                  -0.048760\n",
              "onehotencoder__age_cat_Greater than 45                           0.442049\n",
              "onehotencoder__age_cat_Less than 25                             -0.433359\n",
              "onehotencoder__race_African-American                            -0.314890\n",
              "onehotencoder__race_Asian                                       -0.010795\n",
              "onehotencoder__race_Caucasian                                   -0.044262\n",
              "onehotencoder__race_Hispanic                                     0.035712\n",
              "onehotencoder__race_Native American                             -0.018200\n",
              "onehotencoder__race_Other                                        0.236987\n",
              "onehotencoder__c_charge_degree_(CO3)                             0.005277\n",
              "onehotencoder__c_charge_degree_(F1)                             -0.047490\n",
              "onehotencoder__c_charge_degree_(F2)                             -0.071836\n",
              "onehotencoder__c_charge_degree_(F3)                             -0.190513\n",
              "onehotencoder__c_charge_degree_(F5)                              0.032536\n",
              "onehotencoder__c_charge_degree_(F6)                              0.011575\n",
              "onehotencoder__c_charge_degree_(F7)                              0.000285\n",
              "onehotencoder__c_charge_degree_(M1)                              0.037292\n",
              "onehotencoder__c_charge_degree_(M2)                             -0.047015\n",
              "onehotencoder__c_charge_degree_(MO3)                            -0.066452\n",
              "onehotencoder__c_charge_degree_(NI0)                            -0.009978\n",
              "onehotencoder__c_charge_degree_(TCX)                             0.001988\n",
              "onehotencoder__c_charge_degree_(X)                               0.001488\n",
              "onehotencoder__is_recid_-1                                       0.000152\n",
              "onehotencoder__is_recid_0                                        0.089780\n",
              "onehotencoder__is_recid_1                                       -0.101802\n",
              "onehotencoder__is_violent_recid_0                                0.038153\n",
              "onehotencoder__is_violent_recid_1                               -0.039836"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/html": [
              "<html>\n",
              "<head><meta charset=\"utf-8\" /></head>\n",
              "<body>\n",
              "    <div>            <script src=\"https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/MathJax.js?config=TeX-AMS-MML_SVG\"></script><script type=\"text/javascript\">if (window.MathJax && window.MathJax.Hub && window.MathJax.Hub.Config) {window.MathJax.Hub.Config({SVG: {font: \"STIX-Web\"}});}</script>                <script type=\"text/javascript\">window.PlotlyConfig = {MathJaxConfig: 'local'};</script>\n",
              "        <script charset=\"utf-8\" src=\"https://cdn.plot.ly/plotly-2.24.1.min.js\"></script>                <div id=\"41b2f086-1a37-4c13-bdbf-3222f4e3dd3f\" class=\"plotly-graph-div\" style=\"height:525px; width:100%;\"></div>            <script type=\"text/javascript\">                                    window.PLOTLYENV=window.PLOTLYENV || {};                                    if (document.getElementById(\"41b2f086-1a37-4c13-bdbf-3222f4e3dd3f\")) {                    Plotly.newPlot(                        \"41b2f086-1a37-4c13-bdbf-3222f4e3dd3f\",                        [{\"orientation\":\"h\",\"x\":[-0.2572581582991891,-0.15566768319445917,-0.12118133606683405,-0.7856895180653541,-0.03962684769646957,-0.1113752891300411,0.017933535794746363,-0.020368831682842647,-0.04875975421420449,0.4420489052536677,-0.4333594726463795,-0.31488989593771466,-0.010795216336339607,-0.044261618171208224,0.03571237508381274,-0.018200088458659922,0.23698691463892527,0.005277080108147035,-0.04749043655491713,-0.07183572782588876,-0.1905131144794544,0.032535902527720956,0.011575325214701147,0.0002852207156765732,0.03729240858822335,-0.04701536404476852,-0.0664522763404137,-0.009977568143279589,0.001987645483464482,0.001488056460829502,0.00015208802198887547,0.08978034386476359,-0.10180215142444549,0.038152621786693736,-0.039835527691127345],\"y\":[\"standardscaler__juv_fel_count\",\"standardscaler__juv_misd_count\",\"standardscaler__juv_other_count\",\"standardscaler__priors_count\",\"standardscaler__days_b_screening_arrest\",\"standardscaler__c_days_from_compas\",\"onehotencoder__sex_Female\",\"onehotencoder__sex_Male\",\"onehotencoder__age_cat_25 - 45\",\"onehotencoder__age_cat_Greater than 45\",\"onehotencoder__age_cat_Less than 25\",\"onehotencoder__race_African-American\",\"onehotencoder__race_Asian\",\"onehotencoder__race_Caucasian\",\"onehotencoder__race_Hispanic\",\"onehotencoder__race_Native American\",\"onehotencoder__race_Other\",\"onehotencoder__c_charge_degree_(CO3)\",\"onehotencoder__c_charge_degree_(F1)\",\"onehotencoder__c_charge_degree_(F2)\",\"onehotencoder__c_charge_degree_(F3)\",\"onehotencoder__c_charge_degree_(F5)\",\"onehotencoder__c_charge_degree_(F6)\",\"onehotencoder__c_charge_degree_(F7)\",\"onehotencoder__c_charge_degree_(M1)\",\"onehotencoder__c_charge_degree_(M2)\",\"onehotencoder__c_charge_degree_(MO3)\",\"onehotencoder__c_charge_degree_(NI0)\",\"onehotencoder__c_charge_degree_(TCX)\",\"onehotencoder__c_charge_degree_(X)\",\"onehotencoder__is_recid_-1\",\"onehotencoder__is_recid_0\",\"onehotencoder__is_recid_1\",\"onehotencoder__is_violent_recid_0\",\"onehotencoder__is_violent_recid_1\"],\"type\":\"bar\"}],                        {\"template\":{\"data\":{\"barpolar\":[{\"marker\":{\"line\":{\"color\":\"rgb(17,17,17)\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"barpolar\"}],\"bar\":[{\"error_x\":{\"color\":\"#f2f5fa\"},\"error_y\":{\"color\":\"#f2f5fa\"},\"marker\":{\"line\":{\"color\":\"rgb(17,17,17)\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"bar\"}],\"carpet\":[{\"aaxis\":{\"endlinecolor\":\"#A2B1C6\",\"gridcolor\":\"#506784\",\"linecolor\":\"#506784\",\"minorgridcolor\":\"#506784\",\"startlinecolor\":\"#A2B1C6\"},\"baxis\":{\"endlinecolor\":\"#A2B1C6\",\"gridcolor\":\"#506784\",\"linecolor\":\"#506784\",\"minorgridcolor\":\"#506784\",\"startlinecolor\":\"#A2B1C6\"},\"type\":\"carpet\"}],\"choropleth\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"type\":\"choropleth\"}],\"contourcarpet\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"type\":\"contourcarpet\"}],\"contour\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"contour\"}],\"heatmapgl\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"heatmapgl\"}],\"heatmap\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"heatmap\"}],\"histogram2dcontour\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"histogram2dcontour\"}],\"histogram2d\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"histogram2d\"}],\"histogram\":[{\"marker\":{\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"histogram\"}],\"mesh3d\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"type\":\"mesh3d\"}],\"parcoords\":[{\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"parcoords\"}],\"pie\":[{\"automargin\":true,\"type\":\"pie\"}],\"scatter3d\":[{\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scatter3d\"}],\"scattercarpet\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scattercarpet\"}],\"scattergeo\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scattergeo\"}],\"scattergl\":[{\"marker\":{\"line\":{\"color\":\"#283442\"}},\"type\":\"scattergl\"}],\"scattermapbox\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scattermapbox\"}],\"scatterpolargl\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scatterpolargl\"}],\"scatterpolar\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scatterpolar\"}],\"scatter\":[{\"marker\":{\"line\":{\"color\":\"#283442\"}},\"type\":\"scatter\"}],\"scatterternary\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scatterternary\"}],\"surface\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"surface\"}],\"table\":[{\"cells\":{\"fill\":{\"color\":\"#506784\"},\"line\":{\"color\":\"rgb(17,17,17)\"}},\"header\":{\"fill\":{\"color\":\"#2a3f5f\"},\"line\":{\"color\":\"rgb(17,17,17)\"}},\"type\":\"table\"}]},\"layout\":{\"annotationdefaults\":{\"arrowcolor\":\"#f2f5fa\",\"arrowhead\":0,\"arrowwidth\":1},\"autotypenumbers\":\"strict\",\"coloraxis\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"colorscale\":{\"diverging\":[[0,\"#8e0152\"],[0.1,\"#c51b7d\"],[0.2,\"#de77ae\"],[0.3,\"#f1b6da\"],[0.4,\"#fde0ef\"],[0.5,\"#f7f7f7\"],[0.6,\"#e6f5d0\"],[0.7,\"#b8e186\"],[0.8,\"#7fbc41\"],[0.9,\"#4d9221\"],[1,\"#276419\"]],\"sequential\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"sequentialminus\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]},\"colorway\":[\"#636efa\",\"#EF553B\",\"#00cc96\",\"#ab63fa\",\"#FFA15A\",\"#19d3f3\",\"#FF6692\",\"#B6E880\",\"#FF97FF\",\"#FECB52\"],\"font\":{\"color\":\"#f2f5fa\"},\"geo\":{\"bgcolor\":\"rgb(17,17,17)\",\"lakecolor\":\"rgb(17,17,17)\",\"landcolor\":\"rgb(17,17,17)\",\"showlakes\":true,\"showland\":true,\"subunitcolor\":\"#506784\"},\"hoverlabel\":{\"align\":\"left\"},\"hovermode\":\"closest\",\"mapbox\":{\"style\":\"dark\"},\"paper_bgcolor\":\"rgb(17,17,17)\",\"plot_bgcolor\":\"rgb(17,17,17)\",\"polar\":{\"angularaxis\":{\"gridcolor\":\"#506784\",\"linecolor\":\"#506784\",\"ticks\":\"\"},\"bgcolor\":\"rgb(17,17,17)\",\"radialaxis\":{\"gridcolor\":\"#506784\",\"linecolor\":\"#506784\",\"ticks\":\"\"}},\"scene\":{\"xaxis\":{\"backgroundcolor\":\"rgb(17,17,17)\",\"gridcolor\":\"#506784\",\"gridwidth\":2,\"linecolor\":\"#506784\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"#C8D4E3\"},\"yaxis\":{\"backgroundcolor\":\"rgb(17,17,17)\",\"gridcolor\":\"#506784\",\"gridwidth\":2,\"linecolor\":\"#506784\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"#C8D4E3\"},\"zaxis\":{\"backgroundcolor\":\"rgb(17,17,17)\",\"gridcolor\":\"#506784\",\"gridwidth\":2,\"linecolor\":\"#506784\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"#C8D4E3\"}},\"shapedefaults\":{\"line\":{\"color\":\"#f2f5fa\"}},\"sliderdefaults\":{\"bgcolor\":\"#C8D4E3\",\"bordercolor\":\"rgb(17,17,17)\",\"borderwidth\":1,\"tickwidth\":0},\"ternary\":{\"aaxis\":{\"gridcolor\":\"#506784\",\"linecolor\":\"#506784\",\"ticks\":\"\"},\"baxis\":{\"gridcolor\":\"#506784\",\"linecolor\":\"#506784\",\"ticks\":\"\"},\"bgcolor\":\"rgb(17,17,17)\",\"caxis\":{\"gridcolor\":\"#506784\",\"linecolor\":\"#506784\",\"ticks\":\"\"}},\"title\":{\"x\":0.05},\"updatemenudefaults\":{\"bgcolor\":\"#506784\",\"borderwidth\":0},\"xaxis\":{\"automargin\":true,\"gridcolor\":\"#283442\",\"linecolor\":\"#506784\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"#283442\",\"zerolinewidth\":2},\"yaxis\":{\"automargin\":true,\"gridcolor\":\"#283442\",\"linecolor\":\"#506784\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"#283442\",\"zerolinewidth\":2}}},\"xaxis\":{\"range\":[-1.1261420440583019,1.2334491678570103],\"tickmode\":\"linear\",\"tick0\":0,\"dtick\":0.5},\"title\":{\"text\":\"LogisticRegression Coefficients (Normalized by STD)\"},\"paper_bgcolor\":\"rgba(0, 0, 0, 0)\",\"plot_bgcolor\":\"rgba(0, 0, 0, 0)\"},                        {\"responsive\": true}                    ).then(function(){\n",
              "                            \n",
              "var gd = document.getElementById('41b2f086-1a37-4c13-bdbf-3222f4e3dd3f');\n",
              "var x = new MutationObserver(function (mutations, observer) {{\n",
              "        var display = window.getComputedStyle(gd).display;\n",
              "        if (!display || display === 'none') {{\n",
              "            console.log([gd, 'removed!']);\n",
              "            Plotly.purge(gd);\n",
              "            observer.disconnect();\n",
              "        }}\n",
              "}});\n",
              "\n",
              "// Listen for the removal of the full notebook cells\n",
              "var notebookContainer = gd.closest('#notebook-container');\n",
              "if (notebookContainer) {{\n",
              "    x.observe(notebookContainer, {childList: true});\n",
              "}}\n",
              "\n",
              "// Listen for the clearing of the current output cell\n",
              "var outputEl = gd.closest('.output');\n",
              "if (outputEl) {{\n",
              "    x.observe(outputEl, {childList: true});\n",
              "}}\n",
              "\n",
              "                        })                };                            </script>        </div>\n",
              "</body>\n",
              "</html>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        }
      ],
      "source": [
        "X_train_preprocessed = pd.DataFrame(\n",
        "    model_lr[:-1].transform(X_train), columns=model_lr[:-1].get_feature_names_out(),\n",
        ")\n",
        "\n",
        "# Get the model's coefficients (standerdized by the std) and turn them into a data frame\n",
        "coefs = pd.DataFrame(\n",
        "    model_lr[-1].coef_ * X_train_preprocessed.std(axis=0).values,\n",
        "    columns=model_lr[:-1].get_feature_names_out(),\n",
        "    index=['Coefficients (Normalized by STD)']).transpose()\n",
        "\n",
        "display(coefs)\n",
        "\n",
        "fig = go.Figure(go.Bar(\n",
        "    x=coefs['Coefficients (Normalized by STD)'],\n",
        "    y=model_lr[:-1].get_feature_names_out(),\n",
        "    orientation='h'))\n",
        "\n",
        "fig.update_xaxes(range=[model_lr[-1].coef_.min(\n",
        ") + (model_lr[-1].coef_.min() * 0.1), model_lr[-1].coef_.max() + (model_lr[-1].coef_.max() * 0.1)])\n",
        "\n",
        "fig.update_layout(\n",
        "    xaxis=dict(\n",
        "        tickmode='linear',\n",
        "        tick0=0,\n",
        "        dtick=0.5\n",
        "    ),\n",
        "    template='plotly_dark',\n",
        "    title_text='LogisticRegression Coefficients (Normalized by STD)',\n",
        "    paper_bgcolor='rgba(0, 0, 0, 0)',\n",
        "    plot_bgcolor='rgba(0, 0, 0, 0)'\n",
        "\n",
        ")\n",
        "\n",
        "fig.show()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "-8hatF-oDfZp"
      },
      "source": [
        "Now, we have a more reliable view of what our model has learned. Its most important features are `race`, `age` and `prior`. This is something that shallow models allow us to do: opening the box to see what is inside. And maybe that is why, [for some researchers](https://www.nature.com/articles/s42256-019-0048-x), they should be the default in high-stakes decisions.\n",
        "\n",
        "For example, a fully interpretable model (based on an analysis of our `LogisticRegression` model) could be used in this problem. A model that can be written in 4 lines of pseudocode:\n",
        "\n",
        "````\n",
        "\n",
        "If sample has < 25 years & is African-American & has prior counts > 0:\n",
        "    Risk = High\n",
        "Else:\n",
        "    Risk = Low\n",
        "    \n",
        "````\n",
        "\n",
        "Let us call this model the `evil_model` and see how it performs compared to our `RandomForestClassifier`."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 38,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 559
        },
        "id": "t1DZL0T1DfZp",
        "outputId": "dc311b83-88f1-4a3f-dd9c-a051db5db0f0"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Accuracy (Evil-Model): 74.71%\n"
          ]
        },
        {
          "data": {
            "text/html": [
              "<html>\n",
              "<head><meta charset=\"utf-8\" /></head>\n",
              "<body>\n",
              "    <div>            <script src=\"https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/MathJax.js?config=TeX-AMS-MML_SVG\"></script><script type=\"text/javascript\">if (window.MathJax && window.MathJax.Hub && window.MathJax.Hub.Config) {window.MathJax.Hub.Config({SVG: {font: \"STIX-Web\"}});}</script>                <script type=\"text/javascript\">window.PlotlyConfig = {MathJaxConfig: 'local'};</script>\n",
              "        <script charset=\"utf-8\" src=\"https://cdn.plot.ly/plotly-2.24.1.min.js\"></script>                <div id=\"75b1a9f8-b099-48dc-a033-2d1381c9d43a\" class=\"plotly-graph-div\" style=\"height:525px; width:100%;\"></div>            <script type=\"text/javascript\">                                    window.PLOTLYENV=window.PLOTLYENV || {};                                    if (document.getElementById(\"75b1a9f8-b099-48dc-a033-2d1381c9d43a\")) {                    Plotly.newPlot(                        \"75b1a9f8-b099-48dc-a033-2d1381c9d43a\",                        [{\"coloraxis\":\"coloraxis\",\"name\":\"0\",\"texttemplate\":\"%{z}\",\"x\":[\"High\",\"Low\"],\"y\":[\"High\",\"Low\"],\"z\":[[178,675],[186,2365]],\"type\":\"heatmap\",\"xaxis\":\"x\",\"yaxis\":\"y\",\"hovertemplate\":\"Predicted: %{x}\\u003cbr\\u003eTrue label: %{y}\\u003cbr\\u003ecolor: %{z}\\u003cextra\\u003e\\u003c\\u002fextra\\u003e\"}],                        {\"template\":{\"data\":{\"barpolar\":[{\"marker\":{\"line\":{\"color\":\"rgb(17,17,17)\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"barpolar\"}],\"bar\":[{\"error_x\":{\"color\":\"#f2f5fa\"},\"error_y\":{\"color\":\"#f2f5fa\"},\"marker\":{\"line\":{\"color\":\"rgb(17,17,17)\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"bar\"}],\"carpet\":[{\"aaxis\":{\"endlinecolor\":\"#A2B1C6\",\"gridcolor\":\"#506784\",\"linecolor\":\"#506784\",\"minorgridcolor\":\"#506784\",\"startlinecolor\":\"#A2B1C6\"},\"baxis\":{\"endlinecolor\":\"#A2B1C6\",\"gridcolor\":\"#506784\",\"linecolor\":\"#506784\",\"minorgridcolor\":\"#506784\",\"startlinecolor\":\"#A2B1C6\"},\"type\":\"carpet\"}],\"choropleth\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"type\":\"choropleth\"}],\"contourcarpet\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"type\":\"contourcarpet\"}],\"contour\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"contour\"}],\"heatmapgl\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"heatmapgl\"}],\"heatmap\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"heatmap\"}],\"histogram2dcontour\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"histogram2dcontour\"}],\"histogram2d\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"histogram2d\"}],\"histogram\":[{\"marker\":{\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"histogram\"}],\"mesh3d\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"type\":\"mesh3d\"}],\"parcoords\":[{\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"parcoords\"}],\"pie\":[{\"automargin\":true,\"type\":\"pie\"}],\"scatter3d\":[{\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scatter3d\"}],\"scattercarpet\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scattercarpet\"}],\"scattergeo\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scattergeo\"}],\"scattergl\":[{\"marker\":{\"line\":{\"color\":\"#283442\"}},\"type\":\"scattergl\"}],\"scattermapbox\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scattermapbox\"}],\"scatterpolargl\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scatterpolargl\"}],\"scatterpolar\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scatterpolar\"}],\"scatter\":[{\"marker\":{\"line\":{\"color\":\"#283442\"}},\"type\":\"scatter\"}],\"scatterternary\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scatterternary\"}],\"surface\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"surface\"}],\"table\":[{\"cells\":{\"fill\":{\"color\":\"#506784\"},\"line\":{\"color\":\"rgb(17,17,17)\"}},\"header\":{\"fill\":{\"color\":\"#2a3f5f\"},\"line\":{\"color\":\"rgb(17,17,17)\"}},\"type\":\"table\"}]},\"layout\":{\"annotationdefaults\":{\"arrowcolor\":\"#f2f5fa\",\"arrowhead\":0,\"arrowwidth\":1},\"autotypenumbers\":\"strict\",\"coloraxis\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"colorscale\":{\"diverging\":[[0,\"#8e0152\"],[0.1,\"#c51b7d\"],[0.2,\"#de77ae\"],[0.3,\"#f1b6da\"],[0.4,\"#fde0ef\"],[0.5,\"#f7f7f7\"],[0.6,\"#e6f5d0\"],[0.7,\"#b8e186\"],[0.8,\"#7fbc41\"],[0.9,\"#4d9221\"],[1,\"#276419\"]],\"sequential\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"sequentialminus\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]},\"colorway\":[\"#636efa\",\"#EF553B\",\"#00cc96\",\"#ab63fa\",\"#FFA15A\",\"#19d3f3\",\"#FF6692\",\"#B6E880\",\"#FF97FF\",\"#FECB52\"],\"font\":{\"color\":\"#f2f5fa\"},\"geo\":{\"bgcolor\":\"rgb(17,17,17)\",\"lakecolor\":\"rgb(17,17,17)\",\"landcolor\":\"rgb(17,17,17)\",\"showlakes\":true,\"showland\":true,\"subunitcolor\":\"#506784\"},\"hoverlabel\":{\"align\":\"left\"},\"hovermode\":\"closest\",\"mapbox\":{\"style\":\"dark\"},\"paper_bgcolor\":\"rgb(17,17,17)\",\"plot_bgcolor\":\"rgb(17,17,17)\",\"polar\":{\"angularaxis\":{\"gridcolor\":\"#506784\",\"linecolor\":\"#506784\",\"ticks\":\"\"},\"bgcolor\":\"rgb(17,17,17)\",\"radialaxis\":{\"gridcolor\":\"#506784\",\"linecolor\":\"#506784\",\"ticks\":\"\"}},\"scene\":{\"xaxis\":{\"backgroundcolor\":\"rgb(17,17,17)\",\"gridcolor\":\"#506784\",\"gridwidth\":2,\"linecolor\":\"#506784\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"#C8D4E3\"},\"yaxis\":{\"backgroundcolor\":\"rgb(17,17,17)\",\"gridcolor\":\"#506784\",\"gridwidth\":2,\"linecolor\":\"#506784\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"#C8D4E3\"},\"zaxis\":{\"backgroundcolor\":\"rgb(17,17,17)\",\"gridcolor\":\"#506784\",\"gridwidth\":2,\"linecolor\":\"#506784\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"#C8D4E3\"}},\"shapedefaults\":{\"line\":{\"color\":\"#f2f5fa\"}},\"sliderdefaults\":{\"bgcolor\":\"#C8D4E3\",\"bordercolor\":\"rgb(17,17,17)\",\"borderwidth\":1,\"tickwidth\":0},\"ternary\":{\"aaxis\":{\"gridcolor\":\"#506784\",\"linecolor\":\"#506784\",\"ticks\":\"\"},\"baxis\":{\"gridcolor\":\"#506784\",\"linecolor\":\"#506784\",\"ticks\":\"\"},\"bgcolor\":\"rgb(17,17,17)\",\"caxis\":{\"gridcolor\":\"#506784\",\"linecolor\":\"#506784\",\"ticks\":\"\"}},\"title\":{\"x\":0.05},\"updatemenudefaults\":{\"bgcolor\":\"#506784\",\"borderwidth\":0},\"xaxis\":{\"automargin\":true,\"gridcolor\":\"#283442\",\"linecolor\":\"#506784\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"#283442\",\"zerolinewidth\":2},\"yaxis\":{\"automargin\":true,\"gridcolor\":\"#283442\",\"linecolor\":\"#506784\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"#283442\",\"zerolinewidth\":2}}},\"xaxis\":{\"anchor\":\"y\",\"domain\":[0.0,1.0],\"scaleanchor\":\"y\",\"constrain\":\"domain\",\"title\":{\"text\":\"Predicted\"},\"side\":\"top\"},\"yaxis\":{\"anchor\":\"x\",\"domain\":[0.0,1.0],\"autorange\":\"reversed\",\"constrain\":\"domain\",\"title\":{\"text\":\"True label\"}},\"coloraxis\":{\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"showscale\":false},\"margin\":{\"t\":60},\"title\":{\"text\":\"Confusion Matrix (Evil_Model)\"},\"paper_bgcolor\":\"rgba(0, 0, 0, 0)\",\"plot_bgcolor\":\"rgba(0, 0, 0, 0)\"},                        {\"responsive\": true}                    ).then(function(){\n",
              "                            \n",
              "var gd = document.getElementById('75b1a9f8-b099-48dc-a033-2d1381c9d43a');\n",
              "var x = new MutationObserver(function (mutations, observer) {{\n",
              "        var display = window.getComputedStyle(gd).display;\n",
              "        if (!display || display === 'none') {{\n",
              "            console.log([gd, 'removed!']);\n",
              "            Plotly.purge(gd);\n",
              "            observer.disconnect();\n",
              "        }}\n",
              "}});\n",
              "\n",
              "// Listen for the removal of the full notebook cells\n",
              "var notebookContainer = gd.closest('#notebook-container');\n",
              "if (notebookContainer) {{\n",
              "    x.observe(notebookContainer, {childList: true});\n",
              "}}\n",
              "\n",
              "// Listen for the clearing of the current output cell\n",
              "var outputEl = gd.closest('.output');\n",
              "if (outputEl) {{\n",
              "    x.observe(outputEl, {childList: true});\n",
              "}}\n",
              "\n",
              "                        })                };                            </script>        </div>\n",
              "</body>\n",
              "</html>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        }
      ],
      "source": [
        "# Define the evil model by applying a very biased and racist rule ...\n",
        "evil_model = lambda df: [0 if (sample['age_cat'] == 'Less than 25' and sample['race'] \\\n",
        "                               == 'African-American' and sample['priors_count'] > 0) else 1 for _, sample in df.iterrows()]\n",
        "\n",
        "# Make predictions on the test set\n",
        "predictions = evil_model(X_test)\n",
        "\n",
        "# Plot the confusion matrix and retrive its values\n",
        "matrix = confusion_matrix(y_test, predictions)\n",
        "TN, FP, FN, TP = matrix.ravel()\n",
        "\n",
        "print(f'Accuracy (Evil-Model): {((TP + TN)/(TP + FP + TN + FN)) * 100:.2f}%')\n",
        "\n",
        "\n",
        "fig = px.imshow(matrix,\n",
        "                labels=dict(x=\"Predicted\", y=\"True label\"),\n",
        "                x=['High', 'Low'],\n",
        "                y=['High', 'Low'],\n",
        "                text_auto=True\n",
        "                )\n",
        "\n",
        "fig.update_xaxes(side='top')\n",
        "\n",
        "fig.update_layout(template='plotly_dark',\n",
        "                  title='Confusion Matrix (Evil_Model)',\n",
        "                  coloraxis_showscale=False,\n",
        "                  paper_bgcolor='rgba(0, 0, 0, 0)',\n",
        "                  plot_bgcolor='rgba(0, 0, 0, 0)')\n",
        "\n",
        "fig.show()\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5sTfzPakDfZp"
      },
      "source": [
        "In the end:\n",
        "\n",
        "- Accuracy (Random Forest): 76%;\n",
        "- Accuracy (Evil-Model): 74%.\n",
        "\n",
        "The `evil_model` even seems wrong in the same way as the `LogisticRegressor` (i.e., most of the mistakes are False Negatives). However, the big difference between this model and the model produced by Northpointe is that the Northpointe model is a black box. We simply do not know how it works. In the case of the `evil_model`, with accuracy similar to the Random Forest model, we know exactly what it does. And so, we can say: **This is unacceptable**.\n",
        "\n",
        "Let us now further explore our `LogisticRegressor` by using fairness metrics."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 40,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 551
        },
        "id": "GnAob7uaDfZp",
        "outputId": "4129906d-93cc-4668-ee44-0e6395d44f1e"
      },
      "outputs": [
        {
          "data": {
            "text/markdown": [
              "| Fairness Metrics                                      | Scores                               |\n",
              "|:------------------------------------------------------|:-------------------------------------|\n",
              "| Chance of receiving the positive class - privileged   | 0.96                                 |\n",
              "| Chance of receiving the positive class - unprivileged | 0.8                                  |\n",
              "| Statistical Parity Ratio (SPR)                        | 0.83                                 |\n",
              "| True Positive Rate - privileged                       | 0.99                                 |\n",
              "| True Positive Rate - unprivileged                     | 0.91                                 |\n",
              "| Equal Opportunity Ratio (EOR)                         | 0.93                                 |\n",
              "| Positive Predictive Value - privileged                | 0.86                                 |\n",
              "| Positive Predictive Value - unprivileged              | 0.75                                 |\n",
              "| Predictive Parity Ratio (PPR)                         | 0.87                                 |\n",
              "| False Positive Rate - privileged                      | 0.83                                 |\n",
              "| False Positive Rate - unprivileged                    | 0.59                                 |\n",
              "| Predictive Equality Ratio (PER)                       | 0.7                                  |\n",
              "| Accuracy - privileged                                 | 0.86                                 |\n",
              "| Accuracy - unprivileged                               | 0.74                                 |\n",
              "| Accuracy Equality Ratio (AER)                         | 0.87                                 |\n",
              "| Equalized Odds                                        | TPR: 0.99 vs 0.91. FPR: 0.83 vs 0.59 |"
            ],
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        }
      ],
      "source": [
        "from IPython.display import Markdown\n",
        "\n",
        "def calc_fair(model, DataFrame, protected_atributte, group_priv, group_unpriv, label):\n",
        "    \"\"\"\n",
        "    The function calc_fair computes several fairness metrics for a given machine\n",
        "    learning model on a test set DataFrame. The fairness metrics calculated include\n",
        "    statistical parity ratio, true positive rate, positive predictive value,\n",
        "    false positive rate, accuracy, equal opportunity ratio, predictive parity\n",
        "    ratio, predictive equality ratio, and accuracy equality ratio. The function\n",
        "    takes in the following arguments:\n",
        "\n",
        "    Args:\n",
        "    --------\n",
        "        - model: The trained machine learning model to evaluate fairness on.\n",
        "        - DataFrame: The test set data used to evaluate the model.\n",
        "        - protected_attribute: The name of the protected attribute in the DataFrame.\n",
        "        - group_priv: The value of the protected attribute for the privileged group.\n",
        "        - group_unpriv: The value of the protected attribute for the unprivileged group.\n",
        "        - label: The name of the column in the DataFrame that contains the ground truth labels.\n",
        "\n",
        "    Returns:\n",
        "    --------\n",
        "    The function returns a dictionary containing the fairness metric names and their\n",
        "    corresponding scores. The scores are rounded to two decimal places. Additionally,\n",
        "    the function returns the equalized odds as a string.\n",
        "    \"\"\"\n",
        "    test_set = DataFrame.copy()\n",
        "\n",
        "    test_set_priv_labels, test_set_priv = list(test_set[test_set[protected_atributte] == group_priv][label]), test_set[test_set[protected_atributte] == group_priv].drop(label, axis = 1)\n",
        "    test_set_unpriv_labels, test_set_unpriv = list(test_set[test_set[protected_atributte] == group_unpriv][label]), test_set[test_set[protected_atributte] == group_unpriv].drop(label, axis = 1)\n",
        "\n",
        "    preds_priv = model.predict(test_set_priv)\n",
        "    preds_unpriv = model.predict(test_set_unpriv)\n",
        "\n",
        "    TN_PV, FP_PV, FN_PV, TP_PV = confusion_matrix(test_set_priv_labels, preds_priv).ravel()\n",
        "    TN_UPV, FP_UPV, FN_UPV, TP_UPV = confusion_matrix(test_set_unpriv_labels, preds_unpriv).ravel()\n",
        "\n",
        "    statistical_parity_priv = (TP_PV + FP_PV)/(TP_PV + FP_PV + TN_PV + FN_PV)  # STATISTICAL PARITY RATIO\n",
        "    statistical_parity_unpriv = (TP_UPV + FP_UPV)/(TP_UPV + FP_UPV + TN_UPV + FN_UPV)  # STATISTICAL PARITY RATIO\n",
        "    equal_oportunity_priv = TP_PV / (TP_PV+FN_PV)  # TRUE POSITIVE RATIO\n",
        "    equal_oportunity_unpriv = TP_UPV / (TP_UPV+FN_UPV)  # TRUE POSITIVE RATIO\n",
        "    predictive_parity_priv = TP_PV/(TP_PV + FP_PV)  # POSITIVE PREDICTIVE VALUE\n",
        "    predictive_parity_unpriv = TP_UPV/(TP_UPV + FP_UPV)  # POSITIVE PREDICTIVE VALUE\n",
        "    predictive_equality_priv = FP_PV / (FP_PV+TN_PV)  # FALSE POSITIVE RATE\n",
        "    predictive_equality_unpriv = FP_UPV / (FP_UPV+TN_UPV)  # FALSE POSITIVE RATE\n",
        "    accuracy_equality_priv = (TP_PV + TN_PV)/(TP_PV + FP_PV + TN_PV + FN_PV)  # ACCURACY EQUALITY RATIO\n",
        "    accuracy_equality_unpriv = (TP_UPV + TN_UPV)/(TP_UPV + FP_UPV + TN_UPV + FN_UPV)  # ACCURACY EQUALITY RATIO\n",
        "\n",
        "    if statistical_parity_priv >= statistical_parity_unpriv:\n",
        "        statistical_parity_ratio = statistical_parity_unpriv/statistical_parity_priv\n",
        "    elif statistical_parity_priv < statistical_parity_unpriv:\n",
        "        statistical_parity_ratio = statistical_parity_priv/statistical_parity_unpriv\n",
        "\n",
        "    if equal_oportunity_priv >= equal_oportunity_unpriv:\n",
        "        equal_oportunity_ratio = equal_oportunity_unpriv/equal_oportunity_priv\n",
        "    elif equal_oportunity_priv < equal_oportunity_unpriv:\n",
        "        equal_oportunity_ratio = equal_oportunity_priv/equal_oportunity_unpriv\n",
        "\n",
        "    if predictive_parity_priv >= predictive_parity_unpriv:\n",
        "        predictive_parity_ratio = predictive_parity_unpriv/predictive_parity_priv\n",
        "    elif predictive_parity_priv < predictive_parity_unpriv:\n",
        "        predictive_parity_ratio = predictive_parity_priv/predictive_parity_unpriv\n",
        "\n",
        "    if predictive_equality_priv >= predictive_equality_unpriv:\n",
        "        predictive_equality_ratio = predictive_equality_unpriv/predictive_equality_priv\n",
        "    elif predictive_equality_priv < predictive_equality_unpriv:\n",
        "        predictive_equality_ratio = predictive_equality_priv/predictive_equality_unpriv\n",
        "\n",
        "    if accuracy_equality_priv >= accuracy_equality_unpriv:\n",
        "        accuracy_equality_ratio = accuracy_equality_unpriv/accuracy_equality_priv\n",
        "    elif accuracy_equality_priv < accuracy_equality_unpriv:\n",
        "        accuracy_equality_ratio = accuracy_equality_priv/accuracy_equality_unpriv\n",
        "\n",
        "    equalized_odds = f'TPR: {round(equal_oportunity_priv, 2)} vs {round(equal_oportunity_unpriv, 2)} <br> FPR: {round(predictive_equality_priv,2)} vs {round(predictive_equality_unpriv,2)}'\n",
        "\n",
        "    data = {'Fairness Metrics': ['Chance of receiving the positive class - privileged',\n",
        "                                'Chance of receiving the positive class - unprivileged',\n",
        "                                'Statistical Parity Ratio (SPR)',\n",
        "                                'True Positive Rate - privileged',\n",
        "                                'True Positive Rate - unprivileged',\n",
        "                                'Equal Opportunity Ratio (EOR)',\n",
        "                                'Positive Predictive Value - privileged',\n",
        "                                'Positive Predictive Value - unprivileged',\n",
        "                                'Predictive Parity Ratio (PPR)',\n",
        "                                'False Positive Rate - privileged',\n",
        "                                'False Positive Rate - unprivileged',\n",
        "                                'Predictive Equality Ratio (PER)',\n",
        "                                'Accuracy - privileged',\n",
        "                                'Accuracy - unprivileged',\n",
        "                                'Accuracy Equality Ratio (AER)',\n",
        "                                'Equalized Odds'],\n",
        "            'Scores': [round(statistical_parity_priv, 2),\n",
        "                        round(statistical_parity_unpriv, 2),\n",
        "                        round(statistical_parity_ratio,2),\n",
        "                        round(equal_oportunity_priv, 2),\n",
        "                        round(equal_oportunity_unpriv, 2),\n",
        "                        round(equal_oportunity_ratio, 2),\n",
        "                        round(predictive_parity_priv,2),\n",
        "                        round(predictive_parity_unpriv,2),\n",
        "                        round(predictive_parity_ratio,2),\n",
        "                        round(predictive_equality_priv,2),\n",
        "                        round(predictive_equality_unpriv,2),\n",
        "                        round(predictive_equality_ratio,2),\n",
        "                        round(accuracy_equality_priv,2),\n",
        "                        round(accuracy_equality_unpriv,2),\n",
        "                        round(accuracy_equality_ratio,2),\n",
        "                        f'TPR: {round(equal_oportunity_priv, 2)} vs {round(equal_oportunity_unpriv, 2)}. FPR: {round(predictive_equality_priv,2)} vs {round(predictive_equality_unpriv,2)}']\n",
        "            }\n",
        "    df = pd.DataFrame(data).set_index('Fairness Metrics')\n",
        "\n",
        "    return pd.DataFrame(data).set_index('Fairness Metrics')\n",
        "\n",
        "X_test['labels'] = y_test\n",
        "\n",
        "fairness_df = calc_fair(model_lr, X_test, 'race', 'Caucasian', 'African-American', 'labels')\n",
        "display(Markdown(fairness_df.to_markdown()))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "qpu6A7_TDfZq"
      },
      "source": [
        "Let us review these results:\n",
        "\n",
        "1.  **Chance of receiving the positive class:**\n",
        "    \n",
        "    -   Privileged group: 0.96\n",
        "    -   Unprivileged group: 0.8\n",
        "    -   **Analysis:** There is a significant disparity in the chance of receiving the positive class between the privileged and unprivileged groups, indicating potential bias.\n",
        "\n",
        "2.  **Statistical Parity Ratio (SPR):**\n",
        "    \n",
        "    -   SPR: 0.83\n",
        "    -   **Analysis:** An SPR less than 1 suggests that a positive outcome is less likely for the unprivileged group, indicating a disparity in fairness.\n",
        "\n",
        "3.  **True Positive Rate (TPR):**\n",
        "    \n",
        "    -   Privileged group: 0.99\n",
        "    -   Unprivileged group: 0.91\n",
        "    -   **Analysis:** While the TPR for both groups is relatively high, there is still a notable difference, indicating a potential disparate impact on true positives.\n",
        "\n",
        "4.  **Equal Opportunity Ratio (EOR):**\n",
        "    \n",
        "    -   EOR: 0.93\n",
        "    -   **Analysis:** An EOR close to 1 suggests a relatively balanced opportunity for positive outcomes.\n",
        "\n",
        "5.  **Positive Predictive Value (PPV):**\n",
        "    \n",
        "    -   Privileged group: 0.86\n",
        "    -   Unprivileged group: 0.75\n",
        "    -   **Analysis:** The disparity in PPV values suggests differences in the accuracy of positive predictions between the two groups.\n",
        "\n",
        "6.  **Predictive Parity Ratio (PPR):**\n",
        "    \n",
        "    -   PPR: 0.87\n",
        "    -   **Analysis:** PPR less than 1 indicates potential disparities in the ratio of true positives to the total predicted positives between groups.\n",
        "\n",
        "7.  **False Positive Rate (FPR):**\n",
        "    \n",
        "    -   Privileged group: 0.83\n",
        "    -   Unprivileged group: 0.59\n",
        "    -   **Analysis:** The higher FPR for the privileged group indicates a potential disparity in false positives between groups.\n",
        "\n",
        "8.  **Predictive Equality Ratio (PER):**\n",
        "    \n",
        "    -   PER: 0.7\n",
        "    -   **Analysis:** PER less than 1 suggests potential disparities in the ratio of false positives to the total predicted negatives between groups.\n",
        "\n",
        "9.  **Accuracy:**\n",
        "    \n",
        "    -   Privileged group: 0.86\n",
        "    -   Unprivileged group: 0.74\n",
        "    -   **Analysis:** There is a noticeable difference in overall accuracy between the two groups, indicating potential bias.\n",
        "\n",
        "10.  **Accuracy Equality Ratio (AER):**\n",
        "    \n",
        "    -   AER: 0.87\n",
        "    -   **Analysis:** AER less than 1 suggests disparities in overall accuracy between the two groups.\n",
        "\n",
        "11.  **Equalized Odds:**\n",
        "    \n",
        "    -   TPR: 0.99 vs. 0.91\n",
        "    -   FPR: 0.83 vs. 0.59\n",
        "    -   **Analysis:** The TPR and FPR values indicate disparities in both true positive and false positive rates between the privileged and unprivileged groups.\n",
        "\n",
        "> **Note:** To learn more about ML Fairness, and how these scores are computed visit our directory on [ML Fairness](https://github.com/Nkluge-correa/TeenyTinyCastle/tree/master/ML-Fairness).\n",
        "\n",
        "Now, let's create an `Explainer` around our model using [Dalex](https://dalex.drwhy.ai/python/).\n",
        "\n",
        "> **Note:** The `dalex` library is a versatile tool designed to analyze and interpret the behavior of machine learning models. Its main feature is the Explainer object, which acts as a wrapper around predictive models, enabling users to gain insights into model intricacies. By providing model-level and prediction-level explanations, dalex aids in understanding the inner workings of complex models. Additionally, the library offers fairness methods and interactive exploration dashboards, empowering users to explore, compare, and enhance the transparency of their machine learning models."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 42,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "U7Yi-bjoDfZq",
        "outputId": "41a5a649-72ba-48a6-e3bc-63dbd85113e2"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "\u001b[?25l     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m0.0/1.0 MB\u001b[0m \u001b[31m?\u001b[0m eta \u001b[36m-:--:--\u001b[0m\r\u001b[2K     \u001b[91m━━━━\u001b[0m\u001b[91m╸\u001b[0m\u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m0.1/1.0 MB\u001b[0m \u001b[31m3.4 MB/s\u001b[0m eta \u001b[36m0:00:01\u001b[0m\r\u001b[2K     \u001b[91m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[91m╸\u001b[0m\u001b[90m━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m0.5/1.0 MB\u001b[0m \u001b[31m7.7 MB/s\u001b[0m eta \u001b[36m0:00:01\u001b[0m\r\u001b[2K     \u001b[91m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[91m╸\u001b[0m \u001b[32m1.0/1.0 MB\u001b[0m \u001b[31m10.2 MB/s\u001b[0m eta \u001b[36m0:00:01\u001b[0m\r\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.0/1.0 MB\u001b[0m \u001b[31m8.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25h  Preparing metadata (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
            "  Building wheel for dalex (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
            "Preparation of a new explainer is initiated\n",
            "\n",
            "  -> data              : 17019 rows 12 cols\n",
            "  -> target variable   : Parameter 'y' was a pandas.Series. Converted to a numpy.ndarray.\n",
            "  -> target variable   : 17019 values\n",
            "  -> model_class       : sklearn.linear_model._logistic.LogisticRegression (default)\n",
            "  -> label             : Logistic Regression explainer\n",
            "  -> predict function  : <function yhat_proba_default at 0x7a215d04d990> will be used (default)\n",
            "  -> predict function  : Accepts only pandas.DataFrame, numpy.ndarray causes problems.\n",
            "  -> predicted values  : min = 1.85e-05, mean = 0.744, max = 0.995\n",
            "  -> model type        : binary classification will be used\n",
            "  -> residual function : difference between y and yhat (default)\n",
            "  -> residuals         : min = -0.981, mean = 0.000664, max = 0.997\n",
            "  -> model_info        : package sklearn\n",
            "\n",
            "A new explainer has been created!\n"
          ]
        }
      ],
      "source": [
        "!pip install dalex -q\n",
        "\n",
        "import dalex as dx\n",
        "\n",
        "# Wrap the model with an `Explainer`\n",
        "model_lr_exp = dx.Explainer(model_lr,\n",
        "                            X, y, label='Logistic Regression explainer',\n",
        "                            model_type='binary classification')"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "46mj2yxLDfZq"
      },
      "source": [
        "Now, we will set up a break down plot of two different samples to finish our interpretability analysis. One of the samples has the sensitive attribute \"`African-American`\", and the other has the attribute \"`Caucasian`\".\n",
        "\n",
        "> **Note:** To learn more about Break-Down plots, visit this [tutorial](https://github.com/Nkluge-correa/TeenyTinyCastle/blob/master/ML-Explainability/Tabular/interpreter_for_tabular.ipynb)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 51,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 503
        },
        "id": "sVmdH7V1DfZq",
        "outputId": "932e1d28-d82b-46f3-f1e8-d005c210e245"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Probabilities for sample_1 (African-American):  [[0.96093348 0.03906652]]\n"
          ]
        },
        {
          "data": {
            "text/plain": [
              "sex                                  Female\n",
              "age_cat                        Less than 25\n",
              "race                       African-American\n",
              "juv_fel_count                             0\n",
              "juv_misd_count                            0\n",
              "juv_other_count                           0\n",
              "priors_count                              2\n",
              "days_b_screening_arrest                -1.0\n",
              "c_days_from_compas                      1.0\n",
              "c_charge_degree                        (F2)\n",
              "is_recid                                  0\n",
              "is_violent_recid                          0\n",
              "Name: 102, dtype: object"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Probabilities for sample_1 (Caucasian):  [[0.04934203 0.95065797]]\n"
          ]
        },
        {
          "data": {
            "text/plain": [
              "sex                           Female\n",
              "age_cat                      25 - 45\n",
              "race                       Caucasian\n",
              "juv_fel_count                      0\n",
              "juv_misd_count                     0\n",
              "juv_other_count                    0\n",
              "priors_count                       0\n",
              "days_b_screening_arrest         -1.0\n",
              "c_days_from_compas               1.0\n",
              "c_charge_degree                 (M1)\n",
              "is_recid                           0\n",
              "is_violent_recid                   0\n",
              "Name: 14, dtype: object"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        }
      ],
      "source": [
        "# Print the probability distribution for a sample [[High, Low]]\n",
        "print('Probabilities for sample_1 (African-American): ', model_lr.predict_proba(pd.DataFrame(df[df['label'] == 0]\\\n",
        "        .drop('label', axis = 1).iloc[57]).transpose()))\n",
        "\n",
        "# Display the features of the sample\n",
        "display(df[df['label'] == 0].drop('label', axis = 1).iloc[27])\n",
        "\n",
        "print('Probabilities for sample_1 (Caucasian): ', model_lr.predict_proba(pd.DataFrame(df[df['label'] == 1]\\\n",
        "        .drop('label', axis = 1).iloc[14]).transpose()))\n",
        "\n",
        "display( df[df['label'] == 1].drop('label', axis = 1).iloc[14])\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "LHJiWFNPxPby"
      },
      "source": [
        "In the cell below, using the samples from the African-American and Caucasian groups, the logistic regression explainer (model_lr_exp) is then employed to calculate prediction breakdowns for each sample using the `break_down` method. Subsequently, the prediction breakdowns are visualized using Plotly."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 52,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 843
        },
        "id": "BIo2nQ-VDfZq",
        "outputId": "5a12f5be-88bb-4151-e6c4-ee8e08968349"
      },
      "outputs": [
        {
          "data": {
            "text/html": [
              "<html>\n",
              "<head><meta charset=\"utf-8\" /></head>\n",
              "<body>\n",
              "    <div>            <script src=\"https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/MathJax.js?config=TeX-AMS-MML_SVG\"></script><script type=\"text/javascript\">if (window.MathJax && window.MathJax.Hub && window.MathJax.Hub.Config) {window.MathJax.Hub.Config({SVG: {font: \"STIX-Web\"}});}</script>                <script type=\"text/javascript\">window.PlotlyConfig = {MathJaxConfig: 'local'};</script>\n",
              "        <script charset=\"utf-8\" src=\"https://cdn.plot.ly/plotly-2.24.1.min.js\"></script>                <div id=\"5d2becc6-bbe3-4579-852c-cddad6539131\" class=\"plotly-graph-div\" style=\"height:413.0px; width:100%;\"></div>            <script type=\"text/javascript\">                                    window.PLOTLYENV=window.PLOTLYENV || {};                                    if (document.getElementById(\"5d2becc6-bbe3-4579-852c-cddad6539131\")) {                    Plotly.newPlot(                        \"5d2becc6-bbe3-4579-852c-cddad6539131\",                        [{\"base\":0.7436215707722658,\"connector\":{\"line\":{\"color\":\"#371ea3\",\"dash\":\"solid\",\"width\":1},\"mode\":\"spanning\"},\"decreasing\":{\"marker\":{\"color\":\"#f05a71\"}},\"hoverinfo\":\"text+delta\",\"hoverlabel\":{\"bgcolor\":\"rgba(0,0,0,0.8)\"},\"hovertext\":[\"Average response: 0.744\\u003cbr\\u003ePrediction: 0.039\",\"age_cat = 25 - 45\\u003cbr\\u003eincreases average response by\",\"c_charge_degree = (F2)\\u003cbr\\u003eincreases average response by\",\"c_days_from_compas = 1.0\\u003cbr\\u003eincreases average response by\",\"days_b_screening_arrest = -1.0\\u003cbr\\u003edecreases average response by\",\"sex = Male\\u003cbr\\u003edecreases average response by\",\"is_recid = 1.0\\u003cbr\\u003edecreases average response by\",\"juv_other_count = 1.0\\u003cbr\\u003edecreases average response by\",\"race = African-American\\u003cbr\\u003edecreases average response by\",\"is_violent_recid = 1.0\\u003cbr\\u003edecreases average response by\",\"juv_fel_count = 1.0\\u003cbr\\u003edecreases average response by\",\"+ all other factors\\u003cbr\\u003edecreases average response by\",\"Average response: 0.744\\u003cbr\\u003ePrediction: 0.039\"],\"increasing\":{\"marker\":{\"color\":\"#8bdcbe\"}},\"measure\":[\"relative\",\"relative\",\"relative\",\"relative\",\"relative\",\"relative\",\"relative\",\"relative\",\"relative\",\"relative\",\"relative\",\"relative\",\"total\"],\"orientation\":\"h\",\"showlegend\":false,\"text\":[\"0.744\",\"+0.014\",\"+0.003\",\"+0.003\",\"0.0\",\"-0.002\",\"-0.025\",\"-0.033\",\"-0.05\",\"-0.054\",\"-0.106\",\"-0.455\",\"0.039\"],\"textposition\":\"outside\",\"totals\":{\"marker\":{\"color\":\"#371ea3\"}},\"x\":[0.0,0.014,0.003,0.003,0.0,-0.002,-0.025,-0.033,-0.05,-0.054,-0.106,-0.455,-0.705],\"y\":[\"intercept\",\"age_cat = 25 - 45\",\"c_charge_degree = (F2)\",\"c_days_from_compas = 1.0\",\"days_b_screening_arrest = -1.0\",\"sex = Male\",\"is_recid = 1.0\",\"juv_other_count = 1.0\",\"race = African-American\",\"is_violent_recid = 1.0\",\"juv_fel_count = 1.0\",\"+ all other factors\",\"prediction\"],\"type\":\"waterfall\",\"xaxis\":\"x\",\"yaxis\":\"y\"}],                        {\"template\":{\"data\":{\"barpolar\":[{\"marker\":{\"line\":{\"color\":\"rgb(17,17,17)\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"barpolar\"}],\"bar\":[{\"error_x\":{\"color\":\"#f2f5fa\"},\"error_y\":{\"color\":\"#f2f5fa\"},\"marker\":{\"line\":{\"color\":\"rgb(17,17,17)\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"bar\"}],\"carpet\":[{\"aaxis\":{\"endlinecolor\":\"#A2B1C6\",\"gridcolor\":\"#506784\",\"linecolor\":\"#506784\",\"minorgridcolor\":\"#506784\",\"startlinecolor\":\"#A2B1C6\"},\"baxis\":{\"endlinecolor\":\"#A2B1C6\",\"gridcolor\":\"#506784\",\"linecolor\":\"#506784\",\"minorgridcolor\":\"#506784\",\"startlinecolor\":\"#A2B1C6\"},\"type\":\"carpet\"}],\"choropleth\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"type\":\"choropleth\"}],\"contourcarpet\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"type\":\"contourcarpet\"}],\"contour\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"contour\"}],\"heatmapgl\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"heatmapgl\"}],\"heatmap\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"heatmap\"}],\"histogram2dcontour\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"histogram2dcontour\"}],\"histogram2d\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"histogram2d\"}],\"histogram\":[{\"marker\":{\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"histogram\"}],\"mesh3d\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"type\":\"mesh3d\"}],\"parcoords\":[{\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"parcoords\"}],\"pie\":[{\"automargin\":true,\"type\":\"pie\"}],\"scatter3d\":[{\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scatter3d\"}],\"scattercarpet\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scattercarpet\"}],\"scattergeo\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scattergeo\"}],\"scattergl\":[{\"marker\":{\"line\":{\"color\":\"#283442\"}},\"type\":\"scattergl\"}],\"scattermapbox\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scattermapbox\"}],\"scatterpolargl\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scatterpolargl\"}],\"scatterpolar\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scatterpolar\"}],\"scatter\":[{\"marker\":{\"line\":{\"color\":\"#283442\"}},\"type\":\"scatter\"}],\"scatterternary\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scatterternary\"}],\"surface\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"surface\"}],\"table\":[{\"cells\":{\"fill\":{\"color\":\"#506784\"},\"line\":{\"color\":\"rgb(17,17,17)\"}},\"header\":{\"fill\":{\"color\":\"#2a3f5f\"},\"line\":{\"color\":\"rgb(17,17,17)\"}},\"type\":\"table\"}]},\"layout\":{\"annotationdefaults\":{\"arrowcolor\":\"#f2f5fa\",\"arrowhead\":0,\"arrowwidth\":1},\"autotypenumbers\":\"strict\",\"coloraxis\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"colorscale\":{\"diverging\":[[0,\"#8e0152\"],[0.1,\"#c51b7d\"],[0.2,\"#de77ae\"],[0.3,\"#f1b6da\"],[0.4,\"#fde0ef\"],[0.5,\"#f7f7f7\"],[0.6,\"#e6f5d0\"],[0.7,\"#b8e186\"],[0.8,\"#7fbc41\"],[0.9,\"#4d9221\"],[1,\"#276419\"]],\"sequential\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"sequentialminus\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]},\"colorway\":[\"#636efa\",\"#EF553B\",\"#00cc96\",\"#ab63fa\",\"#FFA15A\",\"#19d3f3\",\"#FF6692\",\"#B6E880\",\"#FF97FF\",\"#FECB52\"],\"font\":{\"color\":\"#f2f5fa\"},\"geo\":{\"bgcolor\":\"rgb(17,17,17)\",\"lakecolor\":\"rgb(17,17,17)\",\"landcolor\":\"rgb(17,17,17)\",\"showlakes\":true,\"showland\":true,\"subunitcolor\":\"#506784\"},\"hoverlabel\":{\"align\":\"left\"},\"hovermode\":\"closest\",\"mapbox\":{\"style\":\"dark\"},\"paper_bgcolor\":\"rgb(17,17,17)\",\"plot_bgcolor\":\"rgb(17,17,17)\",\"polar\":{\"angularaxis\":{\"gridcolor\":\"#506784\",\"linecolor\":\"#506784\",\"ticks\":\"\"},\"bgcolor\":\"rgb(17,17,17)\",\"radialaxis\":{\"gridcolor\":\"#506784\",\"linecolor\":\"#506784\",\"ticks\":\"\"}},\"scene\":{\"xaxis\":{\"backgroundcolor\":\"rgb(17,17,17)\",\"gridcolor\":\"#506784\",\"gridwidth\":2,\"linecolor\":\"#506784\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"#C8D4E3\"},\"yaxis\":{\"backgroundcolor\":\"rgb(17,17,17)\",\"gridcolor\":\"#506784\",\"gridwidth\":2,\"linecolor\":\"#506784\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"#C8D4E3\"},\"zaxis\":{\"backgroundcolor\":\"rgb(17,17,17)\",\"gridcolor\":\"#506784\",\"gridwidth\":2,\"linecolor\":\"#506784\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"#C8D4E3\"}},\"shapedefaults\":{\"line\":{\"color\":\"#f2f5fa\"}},\"sliderdefaults\":{\"bgcolor\":\"#C8D4E3\",\"bordercolor\":\"rgb(17,17,17)\",\"borderwidth\":1,\"tickwidth\":0},\"ternary\":{\"aaxis\":{\"gridcolor\":\"#506784\",\"linecolor\":\"#506784\",\"ticks\":\"\"},\"baxis\":{\"gridcolor\":\"#506784\",\"linecolor\":\"#506784\",\"ticks\":\"\"},\"bgcolor\":\"rgb(17,17,17)\",\"caxis\":{\"gridcolor\":\"#506784\",\"linecolor\":\"#506784\",\"ticks\":\"\"}},\"title\":{\"x\":0.05},\"updatemenudefaults\":{\"bgcolor\":\"#506784\",\"borderwidth\":0},\"xaxis\":{\"automargin\":true,\"gridcolor\":\"#283442\",\"linecolor\":\"#506784\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"#283442\",\"zerolinewidth\":2},\"yaxis\":{\"automargin\":true,\"gridcolor\":\"#283442\",\"linecolor\":\"#506784\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"#283442\",\"zerolinewidth\":2}}},\"xaxis\":{\"anchor\":\"y\",\"domain\":[0.0,1.0],\"type\":\"linear\",\"gridwidth\":2,\"zeroline\":false,\"automargin\":true,\"ticks\":\"outside\",\"tickcolor\":\"white\",\"ticklen\":3,\"fixedrange\":true,\"range\":[-0.06975,0.87275]},\"yaxis\":{\"anchor\":\"x\",\"domain\":[0.0,1.0],\"type\":\"category\",\"autorange\":\"reversed\",\"gridwidth\":2,\"automargin\":true,\"ticks\":\"outside\",\"tickcolor\":\"white\",\"ticklen\":10,\"fixedrange\":true},\"annotations\":[{\"font\":{\"size\":16},\"showarrow\":false,\"text\":\"Logistic Regression explainer\",\"x\":0.5,\"xanchor\":\"center\",\"xref\":\"paper\",\"y\":1.0,\"yanchor\":\"bottom\",\"yref\":\"paper\"},{\"font\":{\"size\":16},\"showarrow\":false,\"text\":\"contribution\",\"x\":0.5,\"xanchor\":\"center\",\"xref\":\"paper\",\"y\":0,\"yanchor\":\"top\",\"yref\":\"paper\",\"yshift\":-30}],\"shapes\":[{\"line\":{\"color\":\"#371ea3\",\"dash\":\"dot\",\"width\":1.5},\"type\":\"line\",\"x0\":0.7436215707722658,\"x1\":0.7436215707722658,\"xref\":\"x\",\"y0\":-1,\"y1\":13,\"yref\":\"y\"}],\"title\":{\"text\":\"sample_african_american (Logistic Regression Explainer)\"},\"font\":{\"color\":\"white\"},\"margin\":{\"t\":78,\"b\":71,\"r\":30},\"height\":413.0,\"paper_bgcolor\":\"rgba(0, 0, 0, 0)\",\"plot_bgcolor\":\"rgba(0, 0, 0, 0)\"},                        {\"responsive\": true}                    ).then(function(){\n",
              "                            \n",
              "var gd = document.getElementById('5d2becc6-bbe3-4579-852c-cddad6539131');\n",
              "var x = new MutationObserver(function (mutations, observer) {{\n",
              "        var display = window.getComputedStyle(gd).display;\n",
              "        if (!display || display === 'none') {{\n",
              "            console.log([gd, 'removed!']);\n",
              "            Plotly.purge(gd);\n",
              "            observer.disconnect();\n",
              "        }}\n",
              "}});\n",
              "\n",
              "// Listen for the removal of the full notebook cells\n",
              "var notebookContainer = gd.closest('#notebook-container');\n",
              "if (notebookContainer) {{\n",
              "    x.observe(notebookContainer, {childList: true});\n",
              "}}\n",
              "\n",
              "// Listen for the clearing of the current output cell\n",
              "var outputEl = gd.closest('.output');\n",
              "if (outputEl) {{\n",
              "    x.observe(outputEl, {childList: true});\n",
              "}}\n",
              "\n",
              "                        })                };                            </script>        </div>\n",
              "</body>\n",
              "</html>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/html": [
              "<html>\n",
              "<head><meta charset=\"utf-8\" /></head>\n",
              "<body>\n",
              "    <div>            <script src=\"https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/MathJax.js?config=TeX-AMS-MML_SVG\"></script><script type=\"text/javascript\">if (window.MathJax && window.MathJax.Hub && window.MathJax.Hub.Config) {window.MathJax.Hub.Config({SVG: {font: \"STIX-Web\"}});}</script>                <script type=\"text/javascript\">window.PlotlyConfig = {MathJaxConfig: 'local'};</script>\n",
              "        <script charset=\"utf-8\" src=\"https://cdn.plot.ly/plotly-2.24.1.min.js\"></script>                <div id=\"e4db1e7a-f4e5-4eb3-8928-8e43f4e68eb2\" class=\"plotly-graph-div\" style=\"height:413.0px; width:100%;\"></div>            <script type=\"text/javascript\">                                    window.PLOTLYENV=window.PLOTLYENV || {};                                    if (document.getElementById(\"e4db1e7a-f4e5-4eb3-8928-8e43f4e68eb2\")) {                    Plotly.newPlot(                        \"e4db1e7a-f4e5-4eb3-8928-8e43f4e68eb2\",                        [{\"base\":0.7436215707722658,\"connector\":{\"line\":{\"color\":\"#371ea3\",\"dash\":\"solid\",\"width\":1},\"mode\":\"spanning\"},\"decreasing\":{\"marker\":{\"color\":\"#f05a71\"}},\"hoverinfo\":\"text+delta\",\"hoverlabel\":{\"bgcolor\":\"rgba(0,0,0,0.8)\"},\"hovertext\":[\"Average response: 0.744\\u003cbr\\u003ePrediction: 0.951\",\"priors_count = 0.0\\u003cbr\\u003eincreases average response by\",\"c_charge_degree = (M1)\\u003cbr\\u003eincreases average response by\",\"race = Caucasian\\u003cbr\\u003eincreases average response by\",\"is_recid = 0.0\\u003cbr\\u003eincreases average response by\",\"age_cat = 25 - 45\\u003cbr\\u003eincreases average response by\",\"sex = Female\\u003cbr\\u003eincreases average response by\",\"juv_fel_count = 0.0\\u003cbr\\u003eincreases average response by\",\"juv_other_count = 0.0\\u003cbr\\u003eincreases average response by\",\"juv_misd_count = 0.0\\u003cbr\\u003eincreases average response by\",\"is_violent_recid = 0.0\\u003cbr\\u003eincreases average response by\",\"+ all other factors\\u003cbr\\u003eincreases average response by\",\"Average response: 0.744\\u003cbr\\u003ePrediction: 0.951\"],\"increasing\":{\"marker\":{\"color\":\"#8bdcbe\"}},\"measure\":[\"relative\",\"relative\",\"relative\",\"relative\",\"relative\",\"relative\",\"relative\",\"relative\",\"relative\",\"relative\",\"relative\",\"relative\",\"total\"],\"orientation\":\"h\",\"showlegend\":false,\"text\":[\"0.744\",\"+0.096\",\"+0.039\",\"+0.027\",\"+0.016\",\"+0.012\",\"+0.005\",\"+0.004\",\"+0.002\",\"+0.002\",\"+0.001\",\"+0.001\",\"0.951\"],\"textposition\":\"outside\",\"totals\":{\"marker\":{\"color\":\"#371ea3\"}},\"x\":[0.0,0.096,0.039,0.027,0.016,0.012,0.005,0.004,0.002,0.002,0.001,0.001,0.207],\"y\":[\"intercept\",\"priors_count = 0.0\",\"c_charge_degree = (M1)\",\"race = Caucasian\",\"is_recid = 0.0\",\"age_cat = 25 - 45\",\"sex = Female\",\"juv_fel_count = 0.0\",\"juv_other_count = 0.0\",\"juv_misd_count = 0.0\",\"is_violent_recid = 0.0\",\"+ all other factors\",\"prediction\"],\"type\":\"waterfall\",\"xaxis\":\"x\",\"yaxis\":\"y\"}],                        {\"template\":{\"data\":{\"barpolar\":[{\"marker\":{\"line\":{\"color\":\"rgb(17,17,17)\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"barpolar\"}],\"bar\":[{\"error_x\":{\"color\":\"#f2f5fa\"},\"error_y\":{\"color\":\"#f2f5fa\"},\"marker\":{\"line\":{\"color\":\"rgb(17,17,17)\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"bar\"}],\"carpet\":[{\"aaxis\":{\"endlinecolor\":\"#A2B1C6\",\"gridcolor\":\"#506784\",\"linecolor\":\"#506784\",\"minorgridcolor\":\"#506784\",\"startlinecolor\":\"#A2B1C6\"},\"baxis\":{\"endlinecolor\":\"#A2B1C6\",\"gridcolor\":\"#506784\",\"linecolor\":\"#506784\",\"minorgridcolor\":\"#506784\",\"startlinecolor\":\"#A2B1C6\"},\"type\":\"carpet\"}],\"choropleth\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"type\":\"choropleth\"}],\"contourcarpet\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"type\":\"contourcarpet\"}],\"contour\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"contour\"}],\"heatmapgl\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"heatmapgl\"}],\"heatmap\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"heatmap\"}],\"histogram2dcontour\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"histogram2dcontour\"}],\"histogram2d\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"histogram2d\"}],\"histogram\":[{\"marker\":{\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"histogram\"}],\"mesh3d\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"type\":\"mesh3d\"}],\"parcoords\":[{\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"parcoords\"}],\"pie\":[{\"automargin\":true,\"type\":\"pie\"}],\"scatter3d\":[{\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scatter3d\"}],\"scattercarpet\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scattercarpet\"}],\"scattergeo\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scattergeo\"}],\"scattergl\":[{\"marker\":{\"line\":{\"color\":\"#283442\"}},\"type\":\"scattergl\"}],\"scattermapbox\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scattermapbox\"}],\"scatterpolargl\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scatterpolargl\"}],\"scatterpolar\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scatterpolar\"}],\"scatter\":[{\"marker\":{\"line\":{\"color\":\"#283442\"}},\"type\":\"scatter\"}],\"scatterternary\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scatterternary\"}],\"surface\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"surface\"}],\"table\":[{\"cells\":{\"fill\":{\"color\":\"#506784\"},\"line\":{\"color\":\"rgb(17,17,17)\"}},\"header\":{\"fill\":{\"color\":\"#2a3f5f\"},\"line\":{\"color\":\"rgb(17,17,17)\"}},\"type\":\"table\"}]},\"layout\":{\"annotationdefaults\":{\"arrowcolor\":\"#f2f5fa\",\"arrowhead\":0,\"arrowwidth\":1},\"autotypenumbers\":\"strict\",\"coloraxis\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"colorscale\":{\"diverging\":[[0,\"#8e0152\"],[0.1,\"#c51b7d\"],[0.2,\"#de77ae\"],[0.3,\"#f1b6da\"],[0.4,\"#fde0ef\"],[0.5,\"#f7f7f7\"],[0.6,\"#e6f5d0\"],[0.7,\"#b8e186\"],[0.8,\"#7fbc41\"],[0.9,\"#4d9221\"],[1,\"#276419\"]],\"sequential\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"sequentialminus\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]},\"colorway\":[\"#636efa\",\"#EF553B\",\"#00cc96\",\"#ab63fa\",\"#FFA15A\",\"#19d3f3\",\"#FF6692\",\"#B6E880\",\"#FF97FF\",\"#FECB52\"],\"font\":{\"color\":\"#f2f5fa\"},\"geo\":{\"bgcolor\":\"rgb(17,17,17)\",\"lakecolor\":\"rgb(17,17,17)\",\"landcolor\":\"rgb(17,17,17)\",\"showlakes\":true,\"showland\":true,\"subunitcolor\":\"#506784\"},\"hoverlabel\":{\"align\":\"left\"},\"hovermode\":\"closest\",\"mapbox\":{\"style\":\"dark\"},\"paper_bgcolor\":\"rgb(17,17,17)\",\"plot_bgcolor\":\"rgb(17,17,17)\",\"polar\":{\"angularaxis\":{\"gridcolor\":\"#506784\",\"linecolor\":\"#506784\",\"ticks\":\"\"},\"bgcolor\":\"rgb(17,17,17)\",\"radialaxis\":{\"gridcolor\":\"#506784\",\"linecolor\":\"#506784\",\"ticks\":\"\"}},\"scene\":{\"xaxis\":{\"backgroundcolor\":\"rgb(17,17,17)\",\"gridcolor\":\"#506784\",\"gridwidth\":2,\"linecolor\":\"#506784\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"#C8D4E3\"},\"yaxis\":{\"backgroundcolor\":\"rgb(17,17,17)\",\"gridcolor\":\"#506784\",\"gridwidth\":2,\"linecolor\":\"#506784\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"#C8D4E3\"},\"zaxis\":{\"backgroundcolor\":\"rgb(17,17,17)\",\"gridcolor\":\"#506784\",\"gridwidth\":2,\"linecolor\":\"#506784\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"#C8D4E3\"}},\"shapedefaults\":{\"line\":{\"color\":\"#f2f5fa\"}},\"sliderdefaults\":{\"bgcolor\":\"#C8D4E3\",\"bordercolor\":\"rgb(17,17,17)\",\"borderwidth\":1,\"tickwidth\":0},\"ternary\":{\"aaxis\":{\"gridcolor\":\"#506784\",\"linecolor\":\"#506784\",\"ticks\":\"\"},\"baxis\":{\"gridcolor\":\"#506784\",\"linecolor\":\"#506784\",\"ticks\":\"\"},\"bgcolor\":\"rgb(17,17,17)\",\"caxis\":{\"gridcolor\":\"#506784\",\"linecolor\":\"#506784\",\"ticks\":\"\"}},\"title\":{\"x\":0.05},\"updatemenudefaults\":{\"bgcolor\":\"#506784\",\"borderwidth\":0},\"xaxis\":{\"automargin\":true,\"gridcolor\":\"#283442\",\"linecolor\":\"#506784\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"#283442\",\"zerolinewidth\":2},\"yaxis\":{\"automargin\":true,\"gridcolor\":\"#283442\",\"linecolor\":\"#506784\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"#283442\",\"zerolinewidth\":2}}},\"xaxis\":{\"anchor\":\"y\",\"domain\":[0.0,1.0],\"type\":\"linear\",\"gridwidth\":2,\"zeroline\":false,\"automargin\":true,\"ticks\":\"outside\",\"tickcolor\":\"white\",\"ticklen\":3,\"fixedrange\":true,\"range\":[0.71295,0.98205]},\"yaxis\":{\"anchor\":\"x\",\"domain\":[0.0,1.0],\"type\":\"category\",\"autorange\":\"reversed\",\"gridwidth\":2,\"automargin\":true,\"ticks\":\"outside\",\"tickcolor\":\"white\",\"ticklen\":10,\"fixedrange\":true},\"annotations\":[{\"font\":{\"size\":16},\"showarrow\":false,\"text\":\"Logistic Regression explainer\",\"x\":0.5,\"xanchor\":\"center\",\"xref\":\"paper\",\"y\":1.0,\"yanchor\":\"bottom\",\"yref\":\"paper\"},{\"font\":{\"size\":16},\"showarrow\":false,\"text\":\"contribution\",\"x\":0.5,\"xanchor\":\"center\",\"xref\":\"paper\",\"y\":0,\"yanchor\":\"top\",\"yref\":\"paper\",\"yshift\":-30}],\"shapes\":[{\"line\":{\"color\":\"#371ea3\",\"dash\":\"dot\",\"width\":1.5},\"type\":\"line\",\"x0\":0.7436215707722658,\"x1\":0.7436215707722658,\"xref\":\"x\",\"y0\":-1,\"y1\":13,\"yref\":\"y\"}],\"title\":{\"text\":\"sample_caucasian (Logistic Regression Explainer)\"},\"font\":{\"color\":\"white\"},\"margin\":{\"t\":78,\"b\":71,\"r\":30},\"height\":413.0,\"paper_bgcolor\":\"rgba(0, 0, 0, 0)\",\"plot_bgcolor\":\"rgba(0, 0, 0, 0)\"},                        {\"responsive\": true}                    ).then(function(){\n",
              "                            \n",
              "var gd = document.getElementById('e4db1e7a-f4e5-4eb3-8928-8e43f4e68eb2');\n",
              "var x = new MutationObserver(function (mutations, observer) {{\n",
              "        var display = window.getComputedStyle(gd).display;\n",
              "        if (!display || display === 'none') {{\n",
              "            console.log([gd, 'removed!']);\n",
              "            Plotly.purge(gd);\n",
              "            observer.disconnect();\n",
              "        }}\n",
              "}});\n",
              "\n",
              "// Listen for the removal of the full notebook cells\n",
              "var notebookContainer = gd.closest('#notebook-container');\n",
              "if (notebookContainer) {{\n",
              "    x.observe(notebookContainer, {childList: true});\n",
              "}}\n",
              "\n",
              "// Listen for the clearing of the current output cell\n",
              "var outputEl = gd.closest('.output');\n",
              "if (outputEl) {{\n",
              "    x.observe(outputEl, {childList: true});\n",
              "}}\n",
              "\n",
              "                        })                };                            </script>        </div>\n",
              "</body>\n",
              "</html>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        }
      ],
      "source": [
        "# Format the sample\n",
        "sample_african_american = df[df['label'] == 0].drop('label', axis = 1).iloc[57]\n",
        "\n",
        "# Pass the sample by the `Explainer`\n",
        "bd_sample_1 = model_lr_exp.predict_parts(sample_african_american,\n",
        "                                         type='break_down')\n",
        "\n",
        "# Plot the results!\n",
        "fig = bd_sample_1.plot(show=False)\n",
        "\n",
        "# Some extra styling\n",
        "fig.update_layout(\n",
        "    template='plotly_dark',\n",
        "    title='sample_african_american (Logistic Regression Explainer)',\n",
        "    paper_bgcolor='rgba(0, 0, 0, 0)',\n",
        "    plot_bgcolor='rgba(0, 0, 0, 0)',\n",
        "    font_color='white')\n",
        "\n",
        "fig.show()\n",
        "\n",
        "sample_caucasian = df[df['label'] == 1].drop('label', axis = 1).iloc[14]\n",
        "\n",
        "bd_sample_2 = model_lr_exp.predict_parts(sample_caucasian,\n",
        "                                         type='break_down')\n",
        "\n",
        "fig = bd_sample_2.plot(show=False)\n",
        "\n",
        "fig.update_layout(\n",
        "    template='plotly_dark',\n",
        "    title='sample_caucasian (Logistic Regression Explainer)',\n",
        "    paper_bgcolor='rgba(0, 0, 0, 0)',\n",
        "    plot_bgcolor='rgba(0, 0, 0, 0)',\n",
        "    font_color='white')\n",
        "\n",
        "fig.show()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "3BFFOmvnDfZq"
      },
      "source": [
        "According to these graphs, `caucasian` is the third most influential feature for attributing a beneficial label. Meanwhile, African-American is the fourth most influential feature for attributing the high-risk prediction. A fair algorithm should not even consider such attributes, or they should not influence the decision-making process. But this is not the case in our example. In summary, this visual analysis of prediction breakdowns reveals that the model in this example is influenced by sensitive attributes such as race, underscoring the importance of addressing fairness considerations in machine learning models.\n",
        "\n",
        "----\n",
        "\n",
        "Return to the [castle](https://github.com/Nkluge-correa/TeenyTinyCastle)."
      ]
    }
  ],
  "metadata": {
    "colab": {
      "provenance": []
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.9.13"
    },
    "orig_nbformat": 4,
    "vscode": {
      "interpreter": {
        "hash": "aca09746cf57686f00a55ae76e987247ecfb5dd0b3b2e2474d8dbbf0c5e3377e"
      }
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}