OpenML-Python-API.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "8TQSYrLRbPuc"
   },
   "source": [
    "# OpenML Datasets"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "id": "ZYILdzb6ZK6J"
   },
   "outputs": [],
   "source": [
    "!pip install -U pip numpy pandas jupyter\n",
    "!pip install -U openml\n",
    "!pip install seaborn"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\nanos\\miniconda3\\lib\\site-packages\\numpy\\_distributor_init.py:30: UserWarning: loaded more than 1 DLL from .libs:\n",
      "C:\\Users\\nanos\\miniconda3\\lib\\site-packages\\numpy\\.libs\\libopenblas.GK7GX5KEQ4F6UYO3P26ULGBQYHGQO7J4.gfortran-win_amd64.dll\n",
      "C:\\Users\\nanos\\miniconda3\\lib\\site-packages\\numpy\\.libs\\libopenblas.XWYDX2IKJW2NMTWSFYNGFUWKQU3LYTCZ.gfortran-win_amd64.dll\n",
      "  warnings.warn(\"loaded more than 1 DLL from .libs:\"\n"
     ]
    }
   ],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "import seaborn as sn\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "from sklearn.model_selection import train_test_split, cross_val_score\n",
    "from sklearn.naive_bayes import GaussianNB \n",
    "from sklearn.metrics import classification_report, confusion_matrix, accuracy_score, f1_score\n",
    "from sklearn.neighbors import KNeighborsClassifier \n",
    "from sklearn.linear_model import LogisticRegression\n",
    "from sklearn.neural_network import MLPClassifier\n",
    "from sklearn.svm import SVC\n",
    "from sklearn.dummy import DummyClassifier\n",
    "from sklearn.decomposition import PCA\n",
    "\n",
    "from sklearn.utils._testing import ignore_warnings\n",
    "from sklearn.exceptions import ConvergenceWarning\n",
    "\n",
    "import time\n",
    "import warnings\n",
    "warnings.simplefilter(action='ignore', category=FutureWarning)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "AfIJDdxkQBse",
    "nbpresent": {
     "id": "22990c96-6359-4864-bfc4-eb4c3c5a1ec1"
    }
   },
   "source": [
    "## Authentication\n",
    "\n",
    "You first need to find your API key.\n",
    "\n",
    "* Create an OpenML account (free) on http://www.openml.org. \n",
    "* Log in, click your avatar/picture, open 'API authentication'.\n",
    "* Your API key is a secret 32-character string"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "vQDvlm3IQBsf"
   },
   "source": [
    "You can copy this API key into your code (but only if you never share it):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "id": "5ydtzFKwQBsf"
   },
   "outputs": [],
   "source": [
    "# Uncomment and set your OpenML key. \n",
    "import openml as oml\n",
    "# Replace the foloowing string with yout API key\n",
    "oml.config.apikey = '5ffe699b268a9320e6d7daae89758124'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "id": "tRL-2QpuaF66"
   },
   "outputs": [],
   "source": [
    "# Making sure we are using the right settings\n",
    "oml.config.server = 'https://www.openml.org/api/v1/xml' \n",
    "oml.config.apikey = '5ffe699b268a9320e6d7daae89758124'\n",
    "# Suppress sklearn warnings\n",
    "import warnings\n",
    "warnings.simplefilter(action=\"ignore\", category=DeprecationWarning)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "TZElMK9bbZlK"
   },
   "source": [
    "## Import Dataset"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "I2bqD5jiQBsp",
    "nbpresent": {
     "id": "7b58c1f7-3484-4e26-b6b5-67ed6f99b9e9"
    }
   },
   "source": [
    "`datasets.get_dataset(data_id)` returns an `OpenMLData` object with the dataset and meta-data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "id": "HsCH78NlQBsq",
    "nbpresent": {
     "id": "d377efff-2484-4ac3-8706-6434644949fd"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "This is dataset 'dna', the target feature is 'class'\n",
      "URL: https://www.openml.org/data/v1/download/4965245/dna.arff\n",
      "**Author**: Ross King, based on data from Genbank 64.1  \n",
      "\n",
      "**Source**: [MLbench](https://www.rdocumentation.org/packages/mlbench/versions/2.1-1/topics/DNA). Originally from the StatLog project.  \n",
      "\n",
      "**Please Cite**:   \n",
      "\n",
      "\n",
      "\n",
      "**Primate Splice-Junction Gene Sequences (DNA)**  \n",
      "\n",
      "Originally from the StatLog project. The raw data is still available on [UCI](https://archive.ics.uci.edu/ml/datasets/Molecular+Biology+(Splice-junction+Gene+Sequences)).\n",
      "\n",
      "\n",
      "\n",
      "The data consists of 3,186 data points (splice junctions). The data points are described by 180 indicator binary variables and the problem is to recognize the 3 classes (ei, ie, neither), i.e., the boundaries between exons (the parts of the DNA sequence retained after splicing) and introns (the parts of the DNA sequence that are spliced out). The StatLog DNA dataset is a processed version of the [Irvine database]((https://archive.ics.uci.edu/ml/datasets/Molecular+Biology+(Splice-junction+Gene+Sequences))). The main difference is that the symbolic variables representing the nucleotides (only A,G,T,C) were replaced by 3 binary indicator variables. Thus the original 60 symbolic attributes were changed into 180 binary attributes. The names of the examples were removed. The examples with ambiguities were removed (there was very few of them, 4). The StatLog version of this dataset was produced by Ross King at Strathclyde University. For original details see the Irvine database documentation.\n",
      "\n",
      "\n",
      "\n",
      "The nucleotides A,C,G,T were given indicator values as follows:\n",
      "\n",
      "A -> 1 0 0\n",
      "\n",
      "C -> 0 1 0\n",
      "\n",
      "G -> 0 0 1\n",
      "\n",
      "T -> 0 0 0\n",
      "\n",
      "\n",
      "\n",
      "Hint: Much better performance is generally observed if attributes closest to the junction are used. In the StatLog version, this means using attributes A61 to A120 only.\n"
     ]
    }
   ],
   "source": [
    "# This is done based on the dataset ID.\n",
    "# Replace with yours\n",
    "\n",
    "dataset = oml.datasets.get_dataset(40670)\n",
    "\n",
    "# Print a summary\n",
    "print(\n",
    "    f\"This is dataset '{dataset.name}', the target feature is \"\n",
    "    f\"'{dataset.default_target_attribute}'\"\n",
    ")\n",
    "print(f\"URL: {dataset.url}\")\n",
    "print(dataset.description[::])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "ag4_fZfUQBsq",
    "nbpresent": {
     "id": "a80f9990-e073-48e6-9df3-4e27f5db74f7"
    }
   },
   "source": [
    "#### Get the data itself\n",
    "`OpenMLData.getdata()` returns the actual data as numpy arrays. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "id": "Ojwjgv26QBsr",
    "nbpresent": {
     "id": "ab60383f-fc6d-4ca0-80f7-55ece02a0ac4"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "    A0   A1   A2   A3   A4   A5   A6   A7   A8   A9  ...  A171  A172  A173  \\\n",
      "0  0.0  1.0  0.0  0.0  0.0  0.0  1.0  0.0  0.0  0.0  ...   1.0   0.0   0.0   \n",
      "1  0.0  0.0  1.0  0.0  0.0  1.0  0.0  0.0  0.0  0.0  ...   0.0   0.0   0.0   \n",
      "2  0.0  0.0  1.0  0.0  0.0  1.0  0.0  1.0  0.0  0.0  ...   0.0   1.0   0.0   \n",
      "3  0.0  0.0  0.0  0.0  0.0  0.0  0.0  1.0  0.0  0.0  ...   0.0   0.0   1.0   \n",
      "4  0.0  1.0  0.0  0.0  0.0  0.0  0.0  1.0  0.0  0.0  ...   0.0   1.0   0.0   \n",
      "5  0.0  1.0  0.0  0.0  0.0  0.0  0.0  0.0  1.0  1.0  ...   0.0   0.0   1.0   \n",
      "6  0.0  0.0  1.0  1.0  0.0  0.0  0.0  0.0  1.0  0.0  ...   1.0   0.0   0.0   \n",
      "7  1.0  0.0  0.0  1.0  0.0  0.0  0.0  0.0  1.0  0.0  ...   1.0   0.0   0.0   \n",
      "8  0.0  0.0  0.0  1.0  0.0  0.0  0.0  0.0  1.0  0.0  ...   1.0   0.0   0.0   \n",
      "9  0.0  0.0  0.0  0.0  0.0  1.0  1.0  0.0  0.0  0.0  ...   1.0   0.0   0.0   \n",
      "\n",
      "   A174  A175  A176  A177  A178  A179  class  \n",
      "0   0.0   0.0   1.0   1.0   0.0   0.0      2  \n",
      "1   1.0   0.0   0.0   0.0   1.0   0.0      2  \n",
      "2   0.0   0.0   1.0   0.0   0.0   1.0      2  \n",
      "3   0.0   0.0   1.0   0.0   0.0   1.0      0  \n",
      "4   0.0   1.0   0.0   1.0   0.0   0.0      1  \n",
      "5   0.0   1.0   0.0   0.0   1.0   0.0      1  \n",
      "6   0.0   0.0   1.0   0.0   0.0   1.0      0  \n",
      "7   1.0   0.0   0.0   1.0   0.0   0.0      2  \n",
      "8   1.0   0.0   0.0   0.0   0.0   1.0      2  \n",
      "9   0.0   1.0   0.0   1.0   0.0   0.0      2  \n",
      "\n",
      "[10 rows x 181 columns]\n"
     ]
    }
   ],
   "source": [
    "import pandas as pd\n",
    "\n",
    "X_train, y_train, categorical_indicator, attribute_names = dataset.get_data(\n",
    "    dataset_format=\"array\", target=dataset.default_target_attribute\n",
    ")\n",
    "eeg = pd.DataFrame(X_train, columns=attribute_names)\n",
    "eeg[\"class\"] = y_train\n",
    "print(eeg[:10])\n",
    "#print(eeg[::])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "lJEP7dPwQBss"
   },
   "source": [
    "#### Get the meta-data\n",
    "Every dataset comes with rich meta-data:\n",
    "* name, version, date, creator, licence, description, ...\n",
    "* `dataset.qualities` returns 100+ statistical data properties\n",
    "* `dataset.features` returns all variables and their data types\n",
    "* tags added by the OpenML community"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "iyMvG_4_QBtF"
   },
   "source": [
    "## Further information\n",
    "That's it. You are now an expert in using OpenML. In case you have further questions:\n",
    "\n",
    "[OpenML Documentation](https://docs.openml.org)  \n",
    "[Python API Documentation and examples](https://openml.github.io/openml-python)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The dataset consists of 3186 dna splice junctions. Each splice junction is described by 180 binary variables corresponding to 60 nucleotides (A, G, T, C). \n",
    "- Exons are nucleotide sequences in DNA and RNA that are conserved in mature RNA generation after the splicing process.\n",
    "\n",
    "- Introns are nucleotide sequences in DNA and RNA that do not directly encode proteins and are removed during the precursor messenger RNA (pre-mRNA) mRNA stage by RNA splicing..\n",
    "\n",
    "The problem is the recognition of the three classes\n",
    "- None \n",
    "- Exons\n",
    "- Introns\n",
    "\n",
    "Ο συμβολισμός των νουκλεοτιδίων είναι (A,G,T,C):\n",
    "\n",
    "    A -> 1 0 0\n",
    "\n",
    "    C -> 0 1 0\n",
    "\n",
    "    G -> 0 0 1\n",
    "\n",
    "    T -> 0 0 0"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>A0</th>\n",
       "      <th>A1</th>\n",
       "      <th>A2</th>\n",
       "      <th>A3</th>\n",
       "      <th>A4</th>\n",
       "      <th>A5</th>\n",
       "      <th>A6</th>\n",
       "      <th>A7</th>\n",
       "      <th>A8</th>\n",
       "      <th>A9</th>\n",
       "      <th>...</th>\n",
       "      <th>A171</th>\n",
       "      <th>A172</th>\n",
       "      <th>A173</th>\n",
       "      <th>A174</th>\n",
       "      <th>A175</th>\n",
       "      <th>A176</th>\n",
       "      <th>A177</th>\n",
       "      <th>A178</th>\n",
       "      <th>A179</th>\n",
       "      <th>class</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 181 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "    A0   A1   A2   A3   A4   A5   A6   A7   A8   A9  ...  A171  A172  A173  \\\n",
       "0  0.0  1.0  0.0  0.0  0.0  0.0  1.0  0.0  0.0  0.0  ...   1.0   0.0   0.0   \n",
       "1  0.0  0.0  1.0  0.0  0.0  1.0  0.0  0.0  0.0  0.0  ...   0.0   0.0   0.0   \n",
       "2  0.0  0.0  1.0  0.0  0.0  1.0  0.0  1.0  0.0  0.0  ...   0.0   1.0   0.0   \n",
       "3  0.0  0.0  0.0  0.0  0.0  0.0  0.0  1.0  0.0  0.0  ...   0.0   0.0   1.0   \n",
       "4  0.0  1.0  0.0  0.0  0.0  0.0  0.0  1.0  0.0  0.0  ...   0.0   1.0   0.0   \n",
       "\n",
       "   A174  A175  A176  A177  A178  A179  class  \n",
       "0   0.0   0.0   1.0   1.0   0.0   0.0      2  \n",
       "1   1.0   0.0   0.0   0.0   1.0   0.0      2  \n",
       "2   0.0   0.0   1.0   0.0   0.0   1.0      2  \n",
       "3   0.0   0.0   1.0   0.0   0.0   1.0      0  \n",
       "4   0.0   1.0   0.0   1.0   0.0   0.0      1  \n",
       "\n",
       "[5 rows x 181 columns]"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "eeg.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of data samples is :  3186\n",
      "Number of characteristics of each sample is  181\n",
      "There exist 3 classes in column 180 of the dataset \n",
      "The labels of the classes are : [0 1 2]\n"
     ]
    }
   ],
   "source": [
    "print(\"Number of data samples is : \", len(eeg))\n",
    "print(\"Number of characteristics of each sample is \", len(eeg.iloc[0]))\n",
    "print(\"There exist {} classes in column {} of the dataset \".format(len (np.unique(eeg.iloc[:, len(eeg.iloc[0]) - 1])), len(eeg.iloc[0]) -1))\n",
    "print(\"The labels of the classes are : {}\" .format(np.unique(eeg.iloc[:, len(eeg.iloc[0]) - 1])))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Checking for missing data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Total missing</th>\n",
       "      <th>Percent missing</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3181</th>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3182</th>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3183</th>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3184</th>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3185</th>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>3186 rows × 2 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "      Total missing  Percent missing\n",
       "0                 0              0.0\n",
       "1                 0              0.0\n",
       "2                 0              0.0\n",
       "3                 0              0.0\n",
       "4                 0              0.0\n",
       "...             ...              ...\n",
       "3181              0              0.0\n",
       "3182              0              0.0\n",
       "3183              0              0.0\n",
       "3184              0              0.0\n",
       "3185              0              0.0\n",
       "\n",
       "[3186 rows x 2 columns]"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Total missing values\n",
    "total = eeg.isnull().sum(axis=1)\n",
    "\n",
    "# Percent Missing\n",
    "percent = (eeg.isnull().sum(axis=1)/eeg.isnull().count(axis=1)*100)\n",
    "\n",
    "missing_data  = pd.concat([total, percent], axis=1, keys=[\"Total missing\", \"Percent missing\"])\n",
    "\n",
    "missing_data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Class 0 appers : 767 times, with frequency 24.074074074074073 %\n",
      "Class 1 appers : 765 times, with frequency 24.01129943502825 %\n",
      "Class 2 appers : 1654 times, with frequency 51.914626490897675 %\n"
     ]
    }
   ],
   "source": [
    "for i in range(len (np.unique(eeg.iloc[:, len(eeg.iloc[0]) - 1]))):\n",
    "    print(\"Class {} appers : {} times, with frequency {} %\".format (i, np.count_nonzero(eeg.iloc[:, len(eeg.iloc[0]) - 1] == i), np.count_nonzero(eeg.iloc[:, len(eeg.iloc[0]) - 1] == i)/len(eeg) * 100))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "With the above frequency results for each class we can conclude that the dataset is not balanced"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## __Preprocessing__\n",
    "- Split the data set into a training set (train set) and a set (test set) with 30% of the samples in the test set"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "X_train, X_test, y_train, y_test  = train_test_split(X_train, y_train, test_size=0.3, random_state=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# __Performance out-of-the-box__"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## __dummy()__"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Dummy Accuracy: 50.10460251046025% \n",
      "\n"
     ]
    }
   ],
   "source": [
    "dummy = DummyClassifier().fit(X_train, y_train) \n",
    "\n",
    "pred_dummy = dummy.predict(X_test) \n",
    "  \n",
    "accuracy_dummy = dummy.score(X_test, y_test) * 100 \n",
    "print(\"Dummy Accuracy: {}% \\n\".format(accuracy_dummy))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## __GaussianNB()__"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "GaussianNB Accuracy: 94.35146443514645% \n",
      "\n"
     ]
    }
   ],
   "source": [
    "gnb = GaussianNB().fit(X_train, y_train) \n",
    "\n",
    "pred_gnb = gnb.predict(X_test) \n",
    "  \n",
    "accuracy_gnb = gnb.score(X_test, y_test) *100 \n",
    "print(\"GaussianNB Accuracy: {}% \\n\".format(accuracy_gnb))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## __Confusion Matrix and Classification Report of GaussianNB()__"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[0.97560976 0.01626016 0.00813008]\n",
      " [0.03030303 0.94372294 0.02597403]\n",
      " [0.03966597 0.03340292 0.92693111]]\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYcAAAEsCAYAAAAoxX9TAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8rg+JYAAAACXBIWXMAAAsTAAALEwEAmpwYAAAa30lEQVR4nO3deXQUVdrH8V83CQHSnRCEQV/UKKAFzowICo4QFxZlG8FdEEVFQRlnFJHNYZF9Cw6IOiIKAuOGgGwGlH1RFnEfWWoQEJcgKpKNBEi6+/0jmRb6kggZkiqS74dTJ6f6VtW5N0X1k+feW1WeUCgkAACO5XW6AgAA9yE4AAAMBAcAgIHgAAAwEBwAAAaCAwDAEFVSB879eTdzZF0sttY1TlcBRQgyxdzV8o5+7zmdxyvO92V09dqntQ6RSiw4AABOUjDgdA0MBAcAcFoo6HQNDAQHAHBakOAAAIgQInMAABjIHAAABjIHAICB2UoAAAOZAwDAwJgDACASs5UAACYyBwCAgcwBAGBgthIAwEDmAAAwMOYAADC4MHPgTXAAAAOZAwA4jW4lAECkUIjZSgCASC4ccyA4AIDT6FYCABjIHAAABu6QBgAYyBwAAAbGHAAABjIHAICBzAEAYCA4AAAicYc0AMBE5gAAMDAgDQAwkDkAAAwuzBx42Q8AwEDmAABOo1sJAGBwYbcSwQEAnEbmAAAwEBwAAAa6lQAABhdmDuV+KmswGNSw8c+qS4/Hdd9f++mb71KPK1/07krd3LWnuvbso3mL35Mk5eblqd/QceryUG917dlHu/d+60TVyzSPx6PnnhujdWsXavmyOapT54Ljytu3b6UNH7yjdWsXqlu3u44ra9y4oZYvmxNer1HjLM2bO00rV8zVmtXzVbt2Ymk0oczxeDx6/rmxen/dIq1cbp6TP7e/Xhs3pOj9dYv0QME5KWyfBg1+rw/WL9ba1fP10tSn5fF4wsepXr2atm99XzExMaXVNOeFgqe+lLByHxxWrtuoo0eP6rWpE/X4w/cr+dmXwmUH09L17NRZmvHceM14frxSlq3W9/v2a/3GLQoEAnrtxX/o4fvv0uQXZzrYgrKpY4c2qhQTo2uu7aiBg8Zo/LjB4bKoqCglJw9Vu/Zd1LLVbXrwgbtUs2YNSdITT/TUi1OSVanSr18sY0YP1BtvzFfLVrfpqaHJsqw6pd6esqBjxzaqVClGSdd00N8HjlHy+CHhsqioKE1Ifkpt292l5i1v1YMPdlHNmjUK3WfwoMc1ctREXdv8ZsXEVFT7dq0kSTdcf62WLnlDNWtWd6SNjgkGT30pYeU+OHz6xVY1+9PlkqQGf6ivrTt2hsu+S/1B9S6qrfg4v7xer/5Q/yJ9vnW7Es+rpby8gILBoA4dylZUVAWnql9mNW3WWMuWrZEkffjhJ2rUqEG4rH69i7Rr19dKS0tXbm6uPtiwRUnNmkiSdu/aqzvu7H7csa5q2li1zj1HS5e+oc6dbtbatRtLrR1lSVLTJnpv2WpJ0uYPP9HljS4Nl9Wvf/w52fDBFiUlXVnoPp999qUSqlWVJPn9PuXm5krKz+Rbt+mkX35JK72GuQGZg/tkHcqWPzY2vO6t4FVeXv7jc88/9//01Z69+vmXg8o5fFibPvpcOTlHVKVyZaX+sF833tVDT417Rl1u7+hU9cusOL9f6RmZ4fVAIKAKFfKDsD/Op4z0jHBZVuYhxcXHSZLmL1gS/qL5rwsSz9XBg+lq27azvv32e/Xt80gptKDsyf+9H3tOguFzEuf3HXe+MrOyFB/nL3SfnV/t0aR/DNeX/16rmr+rrjUFAXvFyvX65ZeDpdQiF3Fh5lDogLRlWfdLqqX8ALLbtu1XS7w2DvDFVtGh7JzweigYDGcC8XF+9X+0hx4fOFI1a1TXJVYdJVSN079mz1fTJpfr8Z73a9/+n/TAowM0f9YLiomp6FQzypyMzEz5fccEba9XgUB+0M7MyJLP7wuX+fyxSk/LMI7xXwcOHNQ77yyTJKWkLNew4f1LqNZlW+Tv/dhzkpGZddz58vt8SkvPKHSfiU8P13UtbtG2bf9Rz4fvVfL4IXr0sYGl1xi3OcMGpH22bY+0bXu4pGqlVaHS1vCPl2j9xi2SpM+/3K6L6lwYLsvLC+jzrTs08/lkjRncR3v2fqeGf7xEcX5f+EKIj/MrLy9PARee3DPZxg0fqU2bFpKkJk0a6cutO8Jl23fsVN26Fyohoaqio6N1ddKV2rT540KPtWHDFrVt01KSlHT1n7Rt239KtvJl1Acbt6htwTm5skkjffnl9nDZ9u07Vbdu7fA5Sbr6Sm3a9HGh+/xyME0ZGVmSpNR9+5WQEF/KrXGZUOjUlxJW1FTWvZZlPSUpT9L2IrY7o7W8tqk2bPlUXR7qLYVCGjGwt1KWrVZ2To5u79hO0dHRuqPb3xRTsaLu7XyLEqrGq+udN2vwmInq2rOPcvPy9NhD96lK5UpON6VMWbBwqVq2ulpr1yyQx+NR9x691enOmxTri9W0aa+pX79hSnnnVXm9Xs2YOVupqT8Ueqx+/YdrypRk9ehxj9IzMtW1619LsSVlx4IFS9Wq5TVav3ahPB6PHuj+uDp1ukm+2Fi9PO019e03TEtSXss/JzPeVGrqDyfcR5IeeqiPXn/1n8rLy9PRo7l6qGdfh1vnMBf+cekJlVAEyv15d8mHNhRbbK1rnK4CihAshb8MUXx5R7/3/PZWJy/ntcGnfMIvG/5WgqSqJyhKs2077b8rlmV1lJQoKUvSvySNkFRJ0ljbtgv9q6rcD0gDgOOKN1upl6Q9J1h6RRy9kW3bk5U/hhySVFNSgqRDRVWJO6QBwGnF61aaJGnGCT5Pi1jPK/jplXShpPckfS2puaRFhR2c4AAAZ6CCrqO0k9h0u2VZjyk/q8iQ1EzSZZL+WdROBAcAcFoJjjHZtj034qO/ncx+BAcAcJoLZysRHADAaQQHAICB9zkAACKFgu67r4XgAABOo1sJAGCgWwkAYKBbCQBgoFsJAGAgOAAADC58Ci/BAQCcRuYAADAwIA0AMDCVFQBgIHMAAEQKuXDMgdeEAgAMZA4A4DS6lQAABgakAQAGMgcAgMGFA9IEBwBwGpkDAMDAmAMAwEDmAACI5Mab4AgOAOA0MgcAgIHgAAAwMCANADCQOQAAIoUIDgAAA8EBAGBgKisAwEDmAAAwuDA48CY4AICBzAEAHBYKuS9zIDgAgNNc2K1EcAAAp5Wn4BB3XvOSOjROgyx7gdNVQBHi6t3idBVQirgJDgBgIjgAAAzuuweO4AAATqNbCQBgIjgAAAx0KwEAItGtBAAwkTkAACKROQAATGQOAIBIIYIDAMBAcAAARHJj5sDLfgAABjIHAHBaMTIHy7KqSqp6gqI027bTjtmuo6RESVmSXpE0XFJA0vO2bf9U2PHJHADAYaHgqS+Seknac4KlV8ThG9m2PVlSLUmXSqpW8Hl2UXUicwAAhxVzzGGSpBkn+DwtYj2v4KdXUrSkLyR9KqmNpHmFHZzgAAAOK05wKOg6SjuJTbdblvWY8rOKf0u6Q9KFkiYXtRPBAQCcFvKU2KFt254b8VG/k9mP4AAADnPjVFaCAwA4LBQsucyhuAgOAOAwMgcAgCFUgmMOxUVwAACHkTkAAAyMOQAADCH3veuH4AAATiNzAAAYCA4AAAPdSgAAgxszBx7ZDQAwkDkAgMO4CQ4AYOAmOACAIUjmAACIRLcSAMDgxtlKBAcAcBj3OQAADGQOAAADA9IAAAMD0gAAA2MOAACDG7uVyv2zlTwejyZPHqU1a+brvffeVO3aiceVt2vXUu+/v0hr1szX/fd3kiR5vV5NmZKsVavmafnyt3Thhec7UfVyIxgMavgz09Sl11O6v+9IffP9D8eVL16xXrc8PED39h6ut99dc1zZgbR0teryN+3+JrUUa1z2eTwePfvsaK1ZM1/Lls0+wXXTSu+/v1hr1sxXt26dJUlRUVGaPn2SVq6cq/XrF6l9++slSTVqnKU5c17WihVztHr128axyoNQyHPKS0kr95lDhw6tValSjK677mY1adJQY8cO0h13dJeU/595/PghSkq6UYcO5Wj16nlasmSlmjRpKElq0eJWXX31nzRu3ODwPjj9Vm34WEdyc/XapGH6fPtOJU99Tc8Oe0KSdDA9U8/OnKM5z4+W31dF3QeM0ZWX/V61zq6h3Lw8DX9muirFVHS4BWVP5HUzbtxg3X77g5Lyr5vk5CFq1uxGHTqUrdWr31ZKygq1bn2dDhw4qG7deqlataravHmpUlKWa/Tov+vNNxdo3rx3dO21V+nii+to9+69DrewdLmxW6ncZw5NmzbW8uVrJUkffvipLr/80nBZvXp1tWvX10pLy1Bubq42bNiiZs0aa/HiZXrkkQGSpPPPr6Uff/zZkbqXF59stZV0RQNJUoP6F2nbzj3hsu/2/SirdqLi43zyer36/cW19cWOryRJT7/0uu5o31I1zkpwpN5lWdOmjbVs2RpJ+ddNo0Ynum7Sj7lummjevBQNGzYhvF1eXkCSdNVVV6hWrbO1ZMnr6tTpJq1bt7FU2+IGwZDnlJeSVu6Dg9/vU3p6Zng9EAioQoUKkqS4OJ8yMn4ty8w8pLi4uPB2L730tP7xj2GaP39J6Va6nDmUnSNfbOXwutfrVV4g/4vl/Fpna9fe7/XzwXTlHD6izZ9tVc7hI1qwbK0S4v1qdsWlhR0W/4O4uOOvm2Dw2OvGf1xZVlaW4uP9OnQoW1lZh+TzxeqNN6Zo6NBkSVJi4rlKS0tXu3Z36dtvU9WnT8/SbYwLnHHdSpZlHdeZbtv2NyVbndKXmZklvz82vO71ehUo+OLJyMiSz+cLl/n9sUpPzwivd+/+hAYNGqt16xaoYcNWys7OKb2KlyOxVSrrUPbh8HowFFRUwRdRvD9W/R6+W71HTFLN6tVUv+4Fqhrn08x5S+TxeLTp062yd+3VwOQX9OywJ1S9WlWHWlG2ZGRkye//9drweI69bjKPu6Z8Pp/S0vKvm3PPPUezZ7+kqVNnafbshZKkAwcO6p13lkuSUlJWaNiwvqXVDNc4EwekR0i6VVJzSdeVeG0csHHjR2rdurkkqUmThvrySztctmPHV6pb9wIlJMQrOjpazZpdqc2bP1bnzjerT5+/SJKys3MUDIYUCLjwmbtlRMNLLtb6LZ9Jkj7fvlMXXXBeuCwvENAX27/SjAmDNbpvT+35NlUNf29p5tNDNGPCYL2SPEhWnUSN6tuTwHAabdz4kdq0+fW62bp1R7gs/7q5MHzdJCXlXze/+111vfPOqxo0aIxmznwrvP2GDR+pTZsWkqSkpCu1bdt/SrcxOKHfGpB+SNI5tm3v+Y3tzlgLF76rFi2StHr12/J4POrRo4/uvLOjYmOraPr0N9S//wgtXvwveTxezZr1llJT92vhwnc1deoELV/+lqKjo9W37zAdOXLE6aaUWS2bXaGNn/xbd/caqpBCGtH7IaWs+kDZh4/o9nYtFB1dQXf+dZAqVozWvbe2U0K83+kql3kLF76rli2vNq4bny9W06a9rn79Rmjx4lfl9Xo1c+Zspabu14QJQ1W1aryefPJRPfnko5KkDh26asCAEXrhhfHq3v1uZWRk6t57H3W4daXPhePR8oRKaJi8cuVEN7YXBdK3z3O6CihCXL1bnK4CinD48DentR9owzm3nvL3ZdN980q0L6rcT2UFAKfx+AwAgMGNI5YEBwBwWEhkDgCACEEXjtASHADAYUEyBwBAJLqVAAAGBqQBAAYyBwCAgcwBAGAgOAAADHQrAQAMQffFBoIDADiN+xwAAAYX3iDNa0IBACYyBwBwGLOVAACGoIcxBwBABDeOORAcAMBhdCsBAAzc5wAAMBTnPgfLsqpKqnqCojTbttOO2a6jpERJWbZtT7csK1HSSNu27ynq+ExlBQCHhYqxSOolac8Jll4Rh29k2/ZkSbUsy4qRdKOkXb9VJzIHAHBYMbuVJkmacYLP0yLW8wp+eiU1l+SX9CfLshJt295b2MEJDgDgsOIMSBd0HaWdxKbbLct6TPlZxSrbtt+1LCumqMAgERwAwHElOZXVtu25J/hs6G/tR3AAAIcxWwkAYOA+BwCAgeAAADCE6FYCAEQicwAAGAgOAACDG5/KyuMzAAAGMgcAcBj3OQAADIw5AAAMBAcAgMGNA9IEBwBwGGMOAAAD3UoAAAPdSgAAQ9CF4aHEgkO0t0JJHRqnQcIltzldBRQh/YvXna4CShHdSgAAg/vyBoIDADiOzAEAYGAqKwDAUK4GpAEAJ8d9oYHgAACOY8wBAGBwY7cSL/sBABjIHADAYe7LGwgOAOA4xhwAAAY3jjkQHADAYe4LDQQHAHAc3UoAAEPIhbkDwQEAHEbmAAAwMCANADC4LzQQHADAcWQOAAADYw4AAAOzlQAABjIHAICBzAEAYCBzAAAYgiH3ZQ687AcAYCBzAACHuS9vIDgAgOO4CQ4AYGC2EgDAwGwlAICBbiUAgIFuJQCAgW4lAIAh5MKb4AgOAOAwxhwAAIbidCtZllVVUtUTFKXZtp12zHYdJSVKypK0UNLQgqLhtm3/VNjxeXwGADgsVIx/knpJ2nOCpVfE4RvZtj1ZUi1JhyUNlGRL8hdVJzIHAHBYMbuVJkmacYLP0yLW8wp+em3bPlSQSfxi2/buog5OcAAAhxVnQLqg6yjtJDbdblnWY5L2WJbVQFJvScsty0q0bXtvYTsRHADAYSU5ldW27bkRH117MvsRHADAYW68Ca7cD0h7PB5NfGakVqyaq5Slr6t27cTjytu0baE16xZoxaq5uve+O48rq17jLG2z39dFF9cuzSqXCx6PR89MHqVVq9/W0nffNM5L23YttW79Qq1a/bbuu7+TJMnr9eqFKeO1YuVcvbdsti688HxJUr16dbV8xRytWDlXEyeNkNdb7v/bn1bBYFAjnp+lu/uMUrcnx+ub1P3HlS9etUG3/u0p3dt/rN5etl6SFAgENeSZ6erab4zuGzBW3+770Ymqu0ZQoVNeSlq5v0r+fOMNqlSpolq1uE1Dh4zXqDF/D5dFRUVp7LhBuqlDV7Vt3Vn3d+us39WsHi57ZvJIHc454lTVy7QbO9ygSjExatH8Fg0ZPE5jxg4Kl0VFRWncuMHqcOM9an3DnerWrbNq1qyhdu1bSZJatbxNI0dM1NhxgyVJQ4f109CnktWq5W2qUrmy2v/5ekfaVFat2vSpjhzN1asTBuqxe2/VhOlvhcsOpmfquVfna/rovnplTD+lrN2k7/f/rLUffiZJmjX+ST3S5SYlT5vtUO1RmHLfrXRV0yu0Yvk6SdKWLZ+pYaM/hsusenW1e/depaVlSJI2bvxITZs21oL5SzVqzJOaPu119X6ipyP1LuuaXtVYy5evlSRt2fKpGh1zXupFnpcN+edl/vwlWrpkpSTp/PNr6ccf86dw39X5YQWDQUVHR6vm2TX04/5Cp3ajGD7dtlPNLv+DJKlBvTratvPrcNl3+3+SVft8xft9kqQ/XHSBvrB3qe01V+qaJg0kSak/HtBZVeNKvd5u4sY7pMt95uD3+5SRkRleDwSCqlChQrgsPf3XsqzMLMXF+XXX3bfq559+0coV60u9vuWFPy7yvAQKPS+ZWVmKi/eHt5s69WlNeHqoFsxfKim/2+O882rpo4+X6ayzErRzZ5Ez+HCKsrIPy1elSnjd6/UqLxCQJCX+X03t+iZVBw6mK+fwEW3+fLtyDh+VJEVVqKCBE6dp7Iuv6/qmlztSd7c4o7qVLMt6xLKs4ZZlJViW1aXEa+KQzMws+Xyx4XWv16NAwX/szMws+Qv+4pEkX8GX0j333K7mLZKUsvR1/fHSSzT1pafD3U04PTIzIs+LN+K8/Frm9/mUXpBFSFKPHk/osgYt9NzzY1SlSmVJ0rfffq8GlzbXyy+/prFjB5dSK8oHX5VKys45HF4PhkKKKgjkcb5Y9X3wTvUe8089NfkV1a+TqIS4X6+pUY8/oMUvjtaw52Yq+3D57aIt5k1wJaqozMEvabSk/pIuKfGaOGTTxo91Q+vrJEmNG1+mbVvtcJm94yvVqXOBEhLiFR0drabNGuvDzZ+obetOatems9q3vUv//mKbenR/Qj/u/9mhFpRNGzd+pNatm0uSGjduqK3HnJcdEeelWVITbf7wE3XufLP69PmLJCk7O0fBYEiBQFBvzXlJdepcIEnKyjykYMiNz8A8c11Wv67Wf/SFJOnzHbt0UWKtcFleIKAvduzWK2P7a9TjD2rPd/t0Wf26Wrxqg16ekyJJqhRTUR6PVxXK8USBYCh0yktJK2rM4aCkeEkjJU0p8Zo4ZPGi99S8RZKWr5wjj8ejng/30+13dFBsbBXNeOVNPTlgpOYvnCmP16NXZ83Vvn37f/ug+J8tWvSeWrS8WitXzZPH49HDD/XVHXd0UKwvVq9Mf0MDBozUwkWz5PV6NWvWW9qXul8LF76rKS8m671lsxUdHa1+/YbryJEjenrCC3px6gQdPZqrnJwcPfKX/k43r0xpeVUjbfpsm+7pO1qhUEgjHuumlDWblHP4iG5rc62ioyuo0+PDVTE6Wl1vbq2EeL9aNr1cQyZN130DxiovL6D+3TsppmK0001xjPtGHCRPSQ2ExMXWdmN7USDAX8+u9stnrzpdBRQh5uIkz+k8XrNaLU75+/KD71ed1jpEKvezlQDAaTyyGwBgcONUVoIDADiMzAEAYHDjs5UIDgDgMLqVAAAGupUAAAYyBwCAgcwBAGBgQBoAYCiNZyWdqvL7pCsAQKHIHADAYXQrAQAMbuxWIjgAgMPIHAAABjIHAICBzAEAYCBzAAAYyBwAAIaQC1/bS3AAAIfxbCUAgIGnsgIADGQOAAADmQMAwMBUVgCAgamsAAAD3UoAAAMD0gAAgxszB94EBwAwkDkAgMOYrQQAMLixW4ngAAAOY0AaAGAgcwAAGBhzAAAYuEMaAGAgcwAAGBhzAAAY6FYCABjIHAAABoIDAMDgvtAgedwYsQAAzuKprAAAA8EBAGAgOAAADAQHAICB4AAAMBAcAAAGggMAwEBwAAAYCA4AAAOPzzgJlmV1lJQoKcu27elO1wfHsyyrsqQXJQ2xbftrh6uDCJZlPSypmqQM27afc7o+ODlkDienkW3bkyXVcroiOKFzJH3udCVQqIWSJko62+mK4OQRHE5OXsFPfl8uZNv2bkkHnK4HCpUlaajyAwTOEHzZnZztlmU9JmmP0xUBzkBTJAUltXO6Ijh5PJUVAGAgcwAAGAgOAAADwQEAYCA4AAAMBAcAgIHgAAAwEBwAAIb/B70in206IueSAAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 504x360 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "cm = confusion_matrix(y_test, pred_gnb) \n",
    "cm = cm.astype('float') / cm.sum(axis=1)[:, np.newaxis]\n",
    "print(cm)\n",
    "\n",
    "df_cm = pd.DataFrame(cm, range(3), range(3))\n",
    "plt.figure(figsize=(7,5))\n",
    "sn.set(font_scale=0.5) # for label size\n",
    "sn.heatmap(df_cm, annot=True, annot_kws={\"size\": 10}) # font size\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "              precision    recall  f1-score   support\n",
      "\n",
      "     class 0       0.90      0.98      0.94       246\n",
      "     class 1       0.92      0.94      0.93       231\n",
      "     class 2       0.98      0.93      0.95       479\n",
      "\n",
      "    accuracy                           0.94       956\n",
      "   macro avg       0.93      0.95      0.94       956\n",
      "weighted avg       0.95      0.94      0.94       956\n",
      "\n",
      "\n",
      "F1-score is : 0.9403168769471653\n",
      "Cross validation score is : 93.54260089686099\n"
     ]
    }
   ],
   "source": [
    "target_names = ['class 0','class 1','class 2']\n",
    "print(classification_report(y_test, pred_gnb, target_names=target_names))\n",
    "print()\n",
    "\n",
    "f1_gnb = f1_score(y_test, pred_gnb, average='macro')\n",
    "cross_val_gnb = np.mean(cross_val_score(gnb, X_train, y_train, cv=10))*100\n",
    "print('F1-score is :', f1_gnb)\n",
    "print('Cross validation score is :', cross_val_gnb)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## __KNeighborsClassifier()__"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "scrolled": true,
    "slideshow": {
     "slide_type": "-"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAD/CAYAAAD/qh1PAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8rg+JYAAAACXBIWXMAAAsTAAALEwEAmpwYAAAot0lEQVR4nO3deXyU5b338c/MJDPJJJM9hKwQQtgJAawLUoWCFe1i7UHbakGpXWypnkNd+pzl6XOeY+tpa/s67YOe7mUR0NNSxV0hihFF0LAIJIQlELKSfV8nmXn+iKSiZEhImEnu+/t+ycsk3Nzz+zHhm2uu+5rrtni9Xi8iImJY1kAXICIil5eCXkTE4BT0IiIGp6AXETE4Bb2IiMEp6EVEDE5BLyJicEGBLuBCGhra8HgGt7w/NjacurrWy1zR6GPWvsG8vatvcxlK31arhejosAF/f1QGvcfjHXTQnzvejMzaN5i3d/VtLiPVt6ZuREQMTkEvImJwCnoREYNT0IuIGJyCXkTE4BT0IiIGZ5igL6tu5X/99l2a2roDXYqIyKhimKDv6umlurGDUxVNgS5FRGRUMUzQJ8X2vSusvKYtwJWIiIwuhgn6UEcQcZEhlNcq6EVEPsowQQ+QHBdGeY359sQQEfHFWEEfH05lXTs9vZ5AlyIiMmoYLOjD6PV4qWroCHQpIiKjhrGCPu7cBVlN34iInGOooE+MdWK1WLTyRkTkIwwV9MFBNhJiQrXyRkTkIwwV9KCVNyIiH2e8oI8Pp7qhg253b6BLEREZFYwX9HFheIHKuvZAlyIiMioM+56xOTk5VFRU4HQ6WbJkCY8//jgAq1evZuPGjdhsNu68805iYmKGXexgJMf3rbwpq2llwniXXx5TRGQ0G3bQFxQUcP/99/PEE0/gcDhYs2YN27Zto7q6mqamJmJiYggJCRnSOWNjw4d0fHz83wM9JiaM4CArDW3u875uREbvzxez9q6+zWWk+h520NtsNgC8Xi9Op5OcnBwiIyNxu91MnTqVGTNmsGvXLm688cZBn7OurnXQdz+Pj3dRU9Ny3tcSY5ycKGn4xNeN5EJ9m4VZe1ff5jKUvq1Wi88B8rDn6DMyMtiwYQMpKSkUFhayfv16SktLiYmJoaSkhO3bt5OdnT3chxmS5Pgwymu18kZEBEZgRL9s2bLzPt+0aVP/xw8//PBwT39JkuPDeTe/ivbOHpwhw25RRGRMM9yqG/j7VggVeuOUiIhBg/7cyhtN34iIGDPoYyNCcNht2vNGRASDBr3FYiFFWyGIiAAGDXo4t/JGI3oREeMGfVw4Le1umtu6A12KiEhAGTfo43UTEhERMHTQ971LrEzTNyJicoYN+ghnMOGhwVp5IyKmZ9igt1gspGgrBBER4wY99F2QLa9pw+sd3AZpIiJGZOygjw+js7uX+uauQJciIhIwhg96QNM3ImJqxg76uHNLLHVBVkTMy9BB7wwJJtrloExBLyImZuigB92ERETE8EGfEhdORW37oG9NKCJiNIYP+uT4MHp6PVQ3dgS6FBGRgDBF0AOUVWv6RkTMyfhBHxdGkM3KyfKmQJciIhIQhg/64CAbk5MjOFbSGOhSREQCwvBBDzA1LZqSqhbaO92BLkVExO+GHfQ5OTls3LiRrVu30tDQwCOPPMIjjzxCfX095eXlPPTQQyNR57BMS4vCCxwv1fSNiJjPsIO+oKCAlStXUlVVhcPhYM2aNaSnp9PY2MjOnTtJS0sbiTqHZVJSBEE2K8dKGwJdioiI3wUN9wQ2mw0Ar9eL0+kkJyeHyMhIysrKaGtr4+DBg5SXl5OcnDzoc8bGhg+phvh410WPmTYxmqKK5kEdO1YYqZehMmvv6ttcRqrvYQd9RkYGGzZsICUlhcLCQtavX8+CBQuYN28e1113Hd3d3UMKeYC6utZBv8EpPt5FTU3LRY+bNN7FC7uLOVPagDNk2G0H3GD7NiKz9q6+zWUofVutFp8D5GEn3rJly877fNOmTed9ft999w33IUbEtLRonn+nmBNljcyZHBfockRE/MYUq27g3Dy9RcssRcR0TBP09mAbk5IiKSzRBVkRMRfTBD30LbM8U9VCe2dPoEsREfEbUwX91LRovF44Wd4Y6FJERPzGVEGf8eE8faHm6UXEREwV9PZgG5MSIzimeXoRMRFTBT30Td+cOdtKR5fm6UXEHEwY9FF4vF5OlGnfGxExB9MFfUZyJDarRdM3ImIapgt6R7CNSUkRuiArIqZhuqCHc/P0LZqnFxFTMGnQ983T6/aCImIGpgz6yUl98/TaDkFEzMCUQe+w20hP0n1kRcQcTBn00LfvTXFlC53dmqcXEWMzbdBPTY3um6fXenoRMTjTBv3k5HPz9I2BLkVE5LIybdA77DZS4sM5c7Y50KWIiFxWpg16gNSEcEqqW/F6B3d/WhGRscjUQZ82LpyWdjeNrd2BLkVE5LIxd9AnuAAorTbfHeZFxDxMHfSp48IBKKlqDXAlIiKXj6mDPtQRxLioUEqqFfQiYlxBwz1BTk4OFRUVOJ1OlixZwuOPPw7Ad7/7XX7zm98AsHr1amJiYob7UJdFakI4pVWauhER4xp20BcUFHD//ffzxBNP4HA4WLNmDdu2baOhoaH/47a2tiEFfWxs+JBqiI93DbXsftPTY9l3rIYwVwjOkOBLPk8gDKfvsc6svatvcxmpvocd9DabDQCv14vT6SQnJ4fIyEgyMzP7P05NTR3SOevqWvF4BrfkMT7eRU3NpY/IY8LtABw8epbMlKhLPo+/DbfvscysvatvcxlK31arxecA2ecc/V//+teLPkBGRgYbNmwgJSWFwsJC1q9fT2lpKe+9917/x+Xl5YMqNhDSdEFWRAzO54g+Ly+PU6dOYbFYePjhhy94zLJly877fNOmTRf8eLSKdjkIDw3WEksRMSyfI/qbbrqJoKAgFixY4K96/M5isZA6LlwjehExLJ9Bf/z4cR544AFOnjzpr3oCYkKCi7KaNno9nkCXIiIy4nwGfWNjI9u3b6eqqspf9QREakI4Pb0ezta1B7oUEZER53OOfunSpdjtdq6//np/1RMQ/Rdkq1tJjh/a0k4RkdHO54h+/fr1HDx4kK1bt/qrnoAYH+skyGalRG+cEhED8jmiX7VqFQkJCf6qJWBsVisp8WG6ICsihuRzRL9u3TreeOMNdu7c6a96AiYtIZxS7U0vIgY04Ii+u7ubH/7whwDU1dX5raBASR3n4q0PKmlo6SImIiTQ5YiIjJgBR/R/+tOf2LlzJ8nJyezevdufNQVEWsLfL8iKiBiJz6mbjo4Of9URcCnx4VhAO1mKiOEMOHWza9cubDYb9fX1HDp0iHvvvdefdfldqCOIcdHam15EjGfAoN+yZYs/6xgVUhNclJzViF5EjMXUd5j6uLRx4VQ3dtDR1RPoUkRERozPdfT79u2jsLAQgDvvvNMvBQXSuQuypdWtTEmNCmwxIiIjxGfQv/fee9xyyy2mWVueOq7vbi4lVS0KehExDJ9TNydOnGDnzp28+eabfionsKLC7bicwbogKyKG4nNE/+1vf5uWlhaioqL8VE5gWSwW0saFU6qtEETEQHyO6J977jk6Ozt58cUX/VVPwKUluCivbaWnV3vTi4gx+Az6yMhIPv3pTxMWFuavegKub296r/amFxHD8Bn02dnZ/PKXv2T27Nn+qifg0s5dkNU9ZEXEIAYM+pycHHJzc+np6WHXrl3+rCmgxsc4sQdZtWWxiBjGgBdjFyxYQFRUFDNmzODdd9/1Z00BZbVaSI4P101IRMQwBgz6rVu3cvjwYSZMmEBsbOyAJ8jJyaGiogKn08mSJUt4/PHHAVi9ejVbtmyhs7OTb33rW0RGRo589ZfJ1NQoduSV0tHVQ6jD58IkEZFRb8Cpm5UrV3LjjTcC+Lw5eEFBAStXrqSqqgqHw8GaNWtIT0+nrKyMWbNmccstt7B3796Rr/wymjsljl6Pl8OnjL8Pv4gYn8/hamFhIdHR0T63K7bZbAB4vV6cTic5OTlERkZitVqxWCwAeDxDW6oYGzu0G3THx7uGdPzFxMSGE7Utn/wzjXzuuskjeu6RNNJ9jyVm7V19m8tI9e0z6N1uN263m7Nnzw54TEZGBhs2bCAlJYXCwkLWr1/PggULmDdvHi+//DIhISHcfffdQyqqrq4Vj2dw2y7Ex7uoqRn5+fSsjBjeKzhLRWUTwUGjb++3y9X3WGDW3tW3uQylb6vV4nOA7DPoFy5cyPTp03n99dcHPGbZsmXnfb5p06b+jx9++OFBFTkazc2M560PKiksaWD2pIGvUYiIjHY+h6r5+fmEh4dTUVHhr3pGjRkTo3HYbRw4XhPoUkREhsVn0Le0tLB+/XpaW823pjw4yMbsSbEcOFGLxyS7d4qIMQ04dVNTU8OyZctobm4mIiLCnzWNGvMy48grrOZURTOTk8fO8lARkY8acERfWVnJG2+8QWlpKfn5+f6sadTIyojFZrVo+kZExjSfb5iqrKxk8uTJ/cskzcYZEsy0CdHsP17D8kUZpv17EJGxbcCgv++++2hoaDB9uM3LjOPJ7cepqGsnOc48u3iKiHH4nLopKCggPz/ftFM3ANmZ8QCavhGRMWvAoM/KyqK5uZm6urr+G4SbUbTLQXpiBAdOKOhFZGzyubyys7OT5uZmYmJi/FXPqDRvShynK1uob+4MdCkiIkPmM+inTp3K8uXLmTVrlr/qGZXmTflw+uZEbYArEREZOp9Bf+bMGXbs2MFbb73lr3pGpcTYMMbHODV9IyJj0kWnbpqamkw/dQN9WxcfK2mkrdMd6FJERIZEUzeDNC8znl6Pl0NF2qNeRMYWn7tXHj16lNdee4329nYWLFjgr5pGpfSkCCLD7Rw4XsM1M8cHuhwRkUHzGfQhISFkZ2fT29vrr3pGLavFwtzMeHYfqaSptYvIcEegSxIRGRSfUzc2mw273U5RUZG/6hnVbrgiBY8HNu84HuhSREQGbcCg37x5M1arlba2NtLT0/1Z06iVGBvGLQsnkneshn3HtAJHRMaGAaduFi1aRE9PDz09PXR26o1C59x4ZRrvH61m0/ZjTJsQRVhIcKBLEhHxacARfXJyMi+99BIZGRls377dnzWNakE2K6tunk5Lu5u/vHEy0OWIiFyUzzn67u5uduzYQUdHh7/qGRMmjHdx41Wp7DpUSUFxfaDLERHxyWfQf/Ob3yQpKYlVq1b5q54x45Zr00mIDmXDq4V0dWtVkoiMXj6Dft26dezYsYONGzf6q54xwx5s4+6bplHT2Mmzu04FuhwRkQH5DPrW1laCgnwutScnJ4eNGzeydetWAPLy8nj00Udxu9384he/4Cc/+Qk1NcZcoTI1LZpFc5PZkVfKqYrmQJcjInJBAwZ9T08Pq1atYtWqVaxYsWLAExQUFLBy5UqqqqoAsNvtuFwuLBYLdXV1NDc3ExoaOvKVjxK3LcogKtzBuleO0uXWFI6IjD4DDtd3797NoUOH+j///ve/f8HjbDYbAF6vF+i7YUlubi5lZWUsXLiQ5ORk9u7dy5IlSwZdVGxs+KCPBYiPdw3p+JF23+3ZPPLnvfzn5v38cMUVpI2P8MvjBrrvQDJr7+rbXEaq7wGDfseOHbhcrv4AH0hGRgYbNmwgJSWF7u5u7HY7ANHR0ezfv5+jR49yxx13DKmourpWPB7fj3tOfLyLmpqWIZ1/pE2MD2PN7XP4wwsFrPlVLl+/YSoLsxIv62OOhr4Dxay9q29zGUrfVqvF5wDZ4h0gybdv305ZWRnjxo1jyZIlfp1+GWtBf05jaxe/fz6fwpJGFswaz9c/O4UQu+9rHJdqNPXtb2btXX2by0gG/YBz9J/97GdZtWoVXq+Xe+65Z+hVmlBUuIMHvzqXWxam8+6RszyyIY/S6tZAlyUiJjdg0K9du5af/vSnuFwu/vCHP/izpjHNarVwy8J0HvzaXNo7e/jxxjwOntQtCEUkcAYMeovFgsvl4siRI6xbt86fNRnC9AnR/Ps3riQpNozfP59PRW1boEsSEZMacAJ5oFU2MniRYXbu+4fZ/Mf691n7zGH+98orcIZcnjl7EZGB+HzDlAxfTEQI37t1NrWNHfz+hXw8F1nFJCIy0hT0fjAlNYqvLsnkUFEd23adDnQ5ImIyCno/+cy8ZBZmJfLi7mL2HasOdDkiYiIKej+xWCys+OwU0hMj+ONLRymv0bJLEfEPBb0fBQfZ+P6XZ+MItrH2mcO0dboDXZKImICC3s+iXQ5W3zqLuqZOfvSn93ju7dPUN+tWjSJy+WitXwBkpkTxg9vn8PLeEp57+zTPv3OaORlxXJ+dxOxJsVitlkCXKCIGoqAPkOkTY5g+MYbqxg52fVDB24cqOXiylpgIB7csTOfTWUmBLlFEDEJTNwE2LiqUf7g+g8e+t4DVt84iwmnnydeO0dqh+XsRGRkK+lEiyGZl/tRx3H3TNHp6vewtqAp0SSJiEAr6USYtwUXquHDeOVwZ6FJExCAU9KPQwtmJFJ9toUxr7UVkBCjoR6GrZiZgs1o0qheREaGgH4UinHbmTI7j3SNn6en1BLocERnjFPSj1LWzx9Pc7ubIqfpAlyIiY5yCfpSaPSmWCGcwb2v6RkSGSUE/SgXZrFw9czwfnKylub070OWIyBimoB/FFs5OpNfjZW++1tSLyKVT0I9iKePCmTDepdU3IjIsww76nJwcNm7cyNatWwHIy8vj0Ucfxev18qtf/Yq1a9dSX68Lipdq4exESqpbKalqCXQpIjJGDTvoCwoKWLlyJVVVfdMLdrsdl8vFsWPHaGpqAiAkJGS4D2NaV81IIMhm0UVZEblkw9690mazAeD98KbXWVlZ5Obm4na7mTp1KjNmzGDXrl3ceOONgz5nbGz4kGqIj3cN6fixJB64alYi7x2t5nu3zSU46O8/m43c98WYtXf1bS4j1fewgz4jI4MNGzaQkpJCd3c3drsdgGnTpvHKK69QVlbGihUrhnTOurpWPB7voI6Nj3dRU2PsaY1PTYnjnQ8qeH1PMfOnxgPm6HsgZu1dfZvLUPq2Wi0+B8jDDvply5Z94mv33XcfAA8//PBwTy/AzPQYIsPtvHO4sj/oRUQGS6tuxgCb1cqCWeP5oKiWv+UW0eXuDXRJIjKG6A5TY8Tnr5lIc2s3L717hj35VXxv+RzSx4UFuiwRGQM0oh8jQh1B3PP5GfzwjrmE2G088ue9/L+th6ht7Ah0aSIyyinox5ipadH8n1WfYtXnZ1Bwpp5/++NeXtxdTEdXT6BLE5FRSlM3Y1CQzcqXF2cyMy2Kp14/wTNvneKlPWe4ZkYC12cnM2G8OZeiiciFKejHsJiIEFbfOpui8ibePFDOO0fO8ubBCtITI1iUncSV0xNw2G2BLlNEAkxBbwAZyZFkJEfy1aWZ7D5yltyDFax7pZCn3zjBhAQXMREhH/5yEBsRQozLwfhYJzarZu5EzEBBbyBhIcHccEUqS+encKKsiXcOV1JZ105hSQMNLV14P/IetAkJLh74ajbhocGBK1hE/EJBb0AWi4UpqVFMSY3q/1qvx0NTazd1zZ2UVbfy1Osn+fmWAzz4tWwinPbAFSsil51eu5uEzWolJiKEzJQoFs9L4R+XZ1HV0M5jWw7Q1KYbm4gYmYLepGamx/BPy7Ooaerg51v209jaFeiSROQyUdCb2PSJMay5bQ71zV38bMsBGloU9iJGpKA3ualp0ay5fQ6NrV38bPN+6ps7A12SiIwwBb0wJTWKB76STUtHN/++7n2eyjmhO1qJGIhW3QgAk5Mj+eEd83hhdzFv7C9jR14pKfHhXDt7PFfPHE9kmFbmiIxVCnrpl5bgYvWts2ntcLO3oIrdRyr5nzdO8tedRUxNiyIyzI7DbsMRbCPEbsNhtxEWEsyV08cRYte3kshopX+d8gnhocEsmZ/CkvkplNe2sftIJfmn66lr6qTT3UtXd+95e+Lv3F/OP92WRWS4I4BVi8hAFPTiU3JcGLctmsxti87/usfrpdvdy9EzDfzu+Xx+8uQ+1tw+h8RY7ZEvMtroYqxcEqvFQog9iLmZ8fzwjnl0u3t59Ml9nCxrCnRpIvIxCnoZtvTECP5lxXzCQ4N57OkD7DtWE+iSROQjFPQyIsZFO/nnFfNJHRfOfz97mNf3lV32x+x29+L96E5tInJBmqOXERPhtPPQ1+byu+fy2bzjOMVnm1l+fcZluUhbUFzPr7cewuUMZlZ6DLPSY5k+MZqwEO3GKfJxww76nJwcKioqcDqdLF++nLy8PLZv386//Mu/UF5ezq9+9Ssee+yxkahVxgBHsI3VX57Ftl2neXVvCfuO1fD5BRO54YpUgoNG5gXksTP1rP3bYeKjQkmMcfJ+YQ1vfVCJxQKTkiKYlR7Lp7MSiYkIGZHHExnrhh30BQUF3H///TzxxBMA2O12XC4X3d3d7Ny5k7S0tGEXKWOLzWrlH67PYOHsRP7njZNsfbOI3IPl3L44k3lT4rBYLJd87rLqVn7+1AEiw+w8+NVsosId9Ho8nKpo5sipeo6cruf5t0+z+0gl/7ryCm3BLMIIBL3N1nerunNzpVlZWeTm5rJnzx7a2to4ePAg5eXlJCcnD/qcsbHhQ6ohPt6c90gd7X3Hx7uYNTWB/ceq+eNzR3ji2cNkTY7jm7fMIj0pcsjnq6xt47/++gH2YBuPrl5IQoyz//fGJ0SyYG4qAIVn6vnX/36H3z1fwI/vXYA92Di3Uxztz/lIOJclHx0QjLW+axs7OFHawIz02GFNXY5U3xbvMK9mvfrqq1RVVREZGcnNN9+M3W5n7dq13HfffQDnfTxYdXWteDyDKys+3kVNjfn2ZRlrffd6PLx5oIJtu07R3tnDtbMTufW6SUS7BvePoKGli//ctI/O7l5+9v2FhNp8vyp4v7Ca32w7wtUzEvjWF2b4fBVR39xJ8dkW5mYO79XG5TbWnvOh6ujq4cXdxWx/v5TJyZHc/pnJpCdGXLTvs/XtnCxroqmti8bWbppau2hs6/u/1Wrli9dO5OoZCZf9ufV4vRwtbmDngXIOnqjF4/VisUBmciTZmfHMnRJHQrTz4if60FCeb6vV4nOAPOygvxwU9Bc3Vvtu7XDz4of76VgtFm74VCo3Xz2BUMfALy5b2rv56eb9NLR08dDX5nJlVvKgen/p3WL+lnuKWxamc8vC9Asesyf/LE9uP05HVw+fmZfMHTdMwTpKw36sPucX4/F62ZN/lr/uLKKprZv5U+I5UdZIc7ubK6eP45tfyiLI6zn/z3i8HCqq4/V9peQXN/R/PdQRRGSYnahwO5HhDirr2iipaiUzJZI7b5hCWoLvEXJPrweb1TKkHwqtHW7eOVzJmwfKqWroIDw0mOvmJDErPYbCkgYOnKiltLoV6HsDYnZmHNmT40hPjMBqHfhxFPQfYdRv/osZ633XNHbwzFun2FtQhcsZzBevTef67CS83r5gb27vprnNTXNbN6/vL6Oito0f3D6HqWnRg+7d6/Xy55eP8s7hs3zrCzO4Zub4/t9r63Tz5GvHeO9oNZOTI0lLCOeN/eVcNSOBez43nSDb0C8ce7xeXnr3DAnRoVw5PWHIf/5ixvpzfiGnK5vZknOcovJm0hMjuPOGKUxKiqCjq4dX95bw2vsleDxeFs1N5gsLJmKxWNh1qIKd+8upbeok2uVgUXYSn5qeQLTLgeNj03Qej5e3D1ey9c0i2jrdLMpO5tbrJvXfK9nr9VJR187hojoOn6rjeGkjoY4gJo53MTHRxYSECNITXUS7HFgsFlo73FTUtvX/qqxr43hZE+4eD5OTI1k8L5krpo77xMKDmsYODp6o5cCJGo6XNuHxegkPDWb2pFiyMmKZNSnmEyvGFPQfYcRv/sEwSt+nK5v5yxsnOVbaSJDNSk+v5xPH2IOs3PulWWRPjgOG1ntPr4dfPn2QooomHvzqXKakRnG0uJ4/vnSU5rZuvrgwnZuvTsNmtfLynjNsfbOI2ZNi+d6tsz4RGr54vV427zjOG/vLsVkt/NvKK5gwfmTnlY3ynEPfdNnz75xm1weVuMLsLL8+gwWzx3/i1VRjaxev5ZWxfe8ZHME2ej1e3D0epqZGsWR+CtmZcYP6odzW6WbbrtO8sb8MpyOIZVelUdfUyeFTddQ1991wJzk+jJkTY2jv6qG4soWK2jY8H8ajy9kXwi3t7v5z2oOtJMaGMSkpguvnJF301cI5rR1ujpyu41BRHUdO1dPa4cZqsTA5OYKv3ziVlPi+wFbQf4SRvvmHwkh9e71ePiiq42hxA+GhQUSE2Ylw2nGF2YkIs/ftmvmR0B1q760dbn6yMY+2zh4+NW0cOw+UMz7Gybe+MIP0xIjzjs09WM7G146RkRTJP96WNah1+V6vl6dfP8mOvFIWz0vmwPEaQh1B/OjuTw3ph8XF+PM5b2nvprXDTbfbg7vXg7vHg7unF3ePl6Q45yXvaXS2vp2X95zh3SNnAVh6RQpfWJCOM2Tgqbv4eBcHj57l5XeLcQTbWDwvhdRxQ1uwcU5ZdSubdxznWGkjDruNGROimZ0RS9ak2E8sx+1291Ja3Urx2RbOnG0BS9/US2JsGElxTmIiQoY9zefxeDlV2cyholqOlzTyhWvTmZke09+3gv5DRgq8oTBr33BpvVfVt/PjD8N+8bxkbl88ecAQzius5vcv5DM+JowffGUOUT5WTXi9Xra+WcQre0tYOj+Fry3N5OiZBn7x9EEWzU1m5Y1Th1SnL/56znceKGfz9uP9o9kLSYhxMndyHNmZcWQkR2Cz+h5VnznbwkvvFrPvWA1BQVauy0rixqtSiYsMvWg9I9231+ulqqGDuMiQS5qi85eRDHq9M1ZMISHGyT9/fT6tHW6mpEb5PPaKaeMIDQni8b8d5tEn9/GlT6czf+q4C/5g2LbrNK/sLWHR3GS+tjQTi8XCjIkxLLsyjVffK2F2egxzp8Rfpq5Glsfr5W+5Rbyyp4SsjFiumTkee5CV4A9/BQVZCbJaKapo4uCJWnbklfLqeyWEhwaTlRFL6rhwvN6+FVYeb99o1ePxcrqymSOn6wl12Lj5mgnccEUqEQG8kY3FYmF8zOBXvxiBRvRjlFn7Bv/1fqqimd+/kE91QwchdhtXTh/HtbMTmZwcicVi4YV3TvPsrtMszErk7pumnfcyvqfXw4835lHf3MV/3HOlz1cFg3U5+3b3ePjTSwW8d7SaxXOTueOGzIuO0ju6ejhyup6DJ2o4VFRHW2fPBY+LDLezdH4Ki+em+JyiGYhZv9c1dfMR+iYwH3/27vV6OV7ayNuHK8krrKHL3UtCjJP0RBd78qu4ZuZ47vnc9Asuk6usa+P/rnufzJRI1nwle9jzuZer79YON4//7RDHy5q4bVEGy65KG/Ka816Ph87uXqwWCzarBeu5XyOwVNWs3+uauhHxE4vFwtS0aKamRXPnDT3kFdbw9uFK9uRXceX0cXzjc9MGXAudGBvGV5dksvG1Y+x4v5Qbrxx924HUNHbwX3/5gNqmDu69ZeYlLwu1Wa2EhYze+W6zU9CLDFKIPYiFWYkszEqkua2bcGfwRUes12cncfhUHX/LLWL6hOhBL8Hzhw9O1rLu5aP0erz9S0/FmBT0IpdgsBcTLRYLd980jR/9+T1+vHEfsREOYiJCiHE5iI4IISbCQVSYgyDb36c8bFYrNpuFELuN8THOEX/r/tn6dp5+/QSHiupIjHXy/S/P1i0gDU5BL3KZuZx2HvhKNu8crqS+uYv65k4KzjTQ2NrFxa6QjeS2DB1dPbz4bjHb3yslOMjK7Ysns/SKlFG9xFBGhoJexA9S4sP5ymcyz/tar8dDY0vfdg+9vV56PR56P1yS2OvxcuRUPa/vL6Ozu5dVN0+75Mf2er3sya/iL2+epKm1m2tnj79sN4SR0UlBLxIgNquV2MgQYiMvfIOUrIxYXGHBbNt1mq7uXv71nquG/BglVS1s2nGck2VNTBzv4vu3ziYjeehbRMvYpqAXGaUsFgtfvDadUHsQT71+gkf+tJdvf34GDvvFt1Vo63Tz7Fun2HmgnLCQYO6+aRoLsxJH7c6ccnkp6EVGuRs+lUqIw8aGVwr55V8O8k/Ls3AOsAePx+vl7UN/363xM3NT+NJ16bqXrskp6EXGgE9nJZEQ5+KxTXn8/KkD3Hz1BLxe8OLlw//o7fWy80AZpytbBr3/upiDgl5kjLh2ThJdnVk88cxhfvtc/gWPiQyz863Pz+DqmZf/jkoydijoRcaQ2ZNieex7C2hud2MBzmW5xWLBAkS7HIa6R66MDAW9yBjjctpxOQO3+6OMPXqnhIiIwSnoRUQMTkEvImJwCnoREYMb9sXYnJwcKioqcDqdLF++nLy8PLZv3853v/tdHn/8cQBWr15NTEzMsIsVEZGhG/aIvqCggJUrV1JVVQWA3W7H5XLhcDhYs2YN6enptLW1DbtQERG5NMMe0dtsfWt2z92RMCsri9zcXJxOJzk5OURGRpKamjqkc/q6JdaFxMeb891/Zu0bzNu7+jaXkep72CP6jIwMNmzYQEpKCt3d3f1fP3bsGOvXr6e0tJTy8vLhPoyIiFyiUXlzcBERGTladSMiYnAKehERg1PQi4gYnIJeRMTgFPQiIganoBcRMTgFvYiIwSnoRUQMbkzfYerjG6qZwVNPPUVTUxOhoaFYLBZT9f7rX/+amTNnmuo537VrFydOnMBms5nq+X7llVeora2lqamJiIgIU/Td2dnJj370I2bOnNn/XEdFRY3I9/uYHtF/fEM1M1iyZAl333039fX1puo9NzeXuLg41q9fb6q+9+7di9vt5tlnnzVV3263m5qaGtatW2eavmtqapg2bdp5uTZSGTemg/7jG6qZgdPpZO3atQQHBwPm6X3v3r0UFRVx9dVXA+bpu6Ghge985zscO3YMME/fpaWl/OAHP6C1tRUwR9+pqalERUWRlpYG9PU8Uhk3pve6efXVV6mqqiIyMpIvfelLgS7HLx544AGSkpLIyMigqanJVL3v3buXhoYGUz3nr732GmVlZXR0dOByuUzT91NPPUVnZycej4egoCDT9P3MM8/Q0NDQ33NISMiIfL+P6aAXEZGLG9NTNyIicnEKehERg1PQi4gYnIJeRMTgFPQy5qxdu5Ynn3wSgJ/97GeD+jODPe6jmpqaePDBB6mtrQXg1ltvpby8nLa2NtauXXvescXFxZw4ceITdba1tQFQVlbGpk2bhlzDpdQt8nFj+p2xYl51dXUUFhYCfcsu29vbcTqdtLe3s3XrVq644gpKSkqIjY1l6dKlnDx5kg0bNhAaGkpYWBjl5eV0dHRw9dVX88ILL7By5UqmTJnCyy+/zNmzZ6mpqWH+/PlYLBaio6MByM7O5ne/+x0PPfQQAC+99NJ552lvb+fNN9/Ebrdz4MABMjIy2LhxIyUlJaxevZrdu3dTV1fHokWL2LNnDyEhIXg8Htra2rDZbISFheH1egkLC+t/F6TX6+W3v/0tS5cuZfLkyYH5y5YxTyN6GZPuuecetmzZQmdnJxaLBQCPxwNASkoKt99+O9HR0Vx//fUUFxeTlZXFXXfdRXFxMbm5ubhcLjo6OmhpaWHx4sVMmTIFgPz8fL7xjW+QlpZGREQEc+bM6X/TSkhICHfccQd//vOfAT5xHuh7FXDXXXcRExMDwIoVK0hKSqKjo4OlS5dy7733kpubS3NzM3fddRd1dXV4vV5WrFjB3Llzqa+v7+8D4OjRo9TW1irkZVgU9DImWa1WvvOd73Dw4EFSUlLYuXMnubm5/b8H9P8AACgqKuLZZ58lMzOTa665hra2NiIiIggPD+8/HmDatGls2LCByspKEhMTP/G406ZN6x/hf/w80PfO5c2bN/dP95yrwev18v7777N582YWLVpEeHg4W7ZsISEhof+406dPEx0dTWVlZf/jzZgxg8WLF7Nt27aR+qsTE9IbpkRG0NNPP01vby92u53bbrst0OWIAAp6ERHD09SNiIjBKehFRAxOQS8iYnAKehERg1PQi4gYnIJeRMTg/j9u36xp9LnEbgAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Optimal k is:  61\n",
      "KNN Accuracy: 85.98326359832636%\n"
     ]
    }
   ],
   "source": [
    "# training a KNN classifier \n",
    "myList = list(range(1,100))\n",
    "neighbors = list(filter(lambda x: x%2!=0, myList))\n",
    "cv_scores = []\n",
    "for k in neighbors:\n",
    "    knn_model = KNeighborsClassifier(n_neighbors=k)\n",
    "    scores = cross_val_score(knn_model, X_train, y_train, cv=10, scoring='f1_macro')\n",
    "    cv_scores.append(scores.mean())\n",
    "    \n",
    "mean_error = [1-x for x in cv_scores]\n",
    "\n",
    "plt.plot(neighbors, mean_error)\n",
    "plt.xlabel('Number of Neighbors k')\n",
    "plt.ylabel('Misclassification Error')\n",
    "plt.show()\n",
    "\n",
    "optimal_k = neighbors[mean_error.index(min(mean_error))]\n",
    "print('Optimal k is: ', optimal_k)\n",
    "\n",
    "knn_model = KNeighborsClassifier(n_neighbors = optimal_k, n_jobs=-1)\n",
    "knn_model.fit(X_train, y_train)\n",
    "pred_knn = knn_model.predict(X_test)\n",
    "accuracy_knn = accuracy_score(y_test, pred_knn)*100\n",
    "print(\"KNN Accuracy: {}%\".format(accuracy_knn))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## __Confusion Matrix and Classification Report of KNeighborsClassifier()__"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[0.78455285 0.12601626 0.08943089]\n",
      " [0.01298701 0.96103896 0.02597403]\n",
      " [0.03131524 0.11899791 0.84968685]]\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYcAAAEsCAYAAAAoxX9TAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8rg+JYAAAACXBIWXMAAAsTAAALEwEAmpwYAAAb00lEQVR4nO3de5yM5f/H8fcc9mB2dhyiA0ux5BBCteRYVHQipb6/VEsqqRxSVN/EKlLSyamzaop0IpVDabWU5FA5lCU5Zm1Rys6ed83M74/9Pobdq92Q3Xuyr6fH/HHf19y36zZm3vO5rvu+xxYMBoMCAOAwdqs7AAAIP4QDAMBAOAAADIQDAMBAOAAADIQDAMDgLK8d584cVV67xnHQdsTnVncBZcgozLa6CyjDrv0bjuv+Cn/fftTbRNRscFz7UFK5hQMA4AgF/Fb3wEA4AIDVggGre2AgHADAagHCAQBQQpDKAQBgoHIAABioHAAABs5WAgAYqBwAAAbmHAAAJXG2EgDAROUAADBQOQAADJytBAAwUDkAAAzMOQAADGFYOfBLcAAAA5UDAFiNYSUAQEnBIGcrAQBKCsM5B8IBAKzGsBIAwEDlAAAwcIU0AMBA5QAAMDDnAAAwUDkAAAxUDgAAA+EAACiJK6QBACYqBwCAgQlpAICBygEAYAjDyoEf+wEAGKgcAMBqDCsBAAxhOKxEOACA1agcAAAGwgEAYGBYCQBgoHIIP4FgUBMWrtWWvQcU4bQr6YpzVa+GW5L0e1ae7p+7MvTcH3/N0LBuzXVVq/oa/eEapWdky2GzacwV56h+TY9Vh3DCs9lsGvX4SDU+q6EKCgo19p7HtHtnWrHnRFeJ0ovvTFHSPRO0c+su2e12JT31X50RX08Bv1+j735Uabv2WHQEJx6bzabxk0apWfPGys8v0P13j9WuHbtD7d26d9GwkbfLf9Cvd2bN09tvzpHT6dTT08crrl5t+f0BPTB8rLb9tFPNWzbVo089pIL8QqX+sFlj/ztRwWDQwqOzQBhWDpX+OoeUzenKP+jXGwO6aljXFnr6s/WhtpruaM1IvEAzEi/Q0K4t1PS0arq6dQMt3/qr/IGg3ri5qwZ2bqZpKRstPIITX9dLOysqOlI3XTFQk8c/pxFjhxRrb3Z2E70273nVPaNOaF2XSzpKkvr1vF3TJ72skQ8PrdA+n+i6X95VUdFR6t3jJk0cN1kPjRsRanM6nRozfqRuvOZ2XXflzerb7xrVOvkkXXhxRzmcDl19aaImP/mCRo4qek0ee2aMHnnwCV17RX9l+rJ0VZ/LrDos6wQCR/8oZ5U+HNbu/l0d4k+VJLWMO0kbf/nTeE4wGNTET9Zp1KVt5LDbdHoNt/yBgALBoLLzC+W02yq625VK64Sz9dXnRRXchu82qtnZTYu1R0ZG6u6bH9COrbtC61I++UKPjHhcklQ77jTt/+2PiutwJXBe29ZatuQrSdLabzaoZatmobaGZ9bXzh275cvIVGHhQa1ZuVbntWujHdt2yel0yGazKTbWrcLCQknSqaedom/XFH0p+2bVOp3btnXFH5DVgoGjf5SzSj+slJ1fKHf0oX8Gh82mg4GAnPZDublsyy9qUMujM2rGSpJckU6lZ+Toquc+1YGcfE35vw4V3u/KxB0bo6zMrNBywO+Xw+GQ3190m+N1azb85XZ+v1/jp4xW10u76N5bH6yQvlYW7li3Mn2HXhO/PxB6TYraMkNt2VnZ8nhilZ2Vo7i6dfT5yg9V46TqGnD9YEnS7l1patv+HK1a8a0u6tFFrpgqFX48livHSiA5OVnp6elyuVzq1auXJk+erPz8fA0cOFC1atUqdbtSw2HOnDnau3evAoGA6tatq169epVLx60WExWh7PyDoeVAUMWCQZIWfP+z+rZtGFqeueontW9wioZ2a6FfM3J028wv9P7tFyvK6aiwflcmWZnZcrljQst2uz0UDH/noaHjdFKt6Zq16BX17txXuTl55dXNSiUrM0sxbldo+fDXJCszS+7DXq8Yd4x8GZm69Y6btCzlKz0xbopOq32KZs97Rd07XaMRQ8YoacL9GjTkZq1fu1H5+QUVfjyWO4Zw8Pl88vl8xnqPxyOP59AcaGpqqoYOHarp06fLZrNp//79CgQCqlKl7BAudVgpJydHd955pwYPHqyMjIyj7vi/Rau6J2n51l8lSRvS9qvRyebE8qZf/1SruJNCy57oCLmjIyRJVatE6qA/oECgkk2gVaB1azaoU7fzJUkt25ylnzZv+9ttrujTQ7cMSZQk5eXmKRgIyu8Pv0m/f6tvVq/ThRd3kiS1Prelfkz9KdS2dcsOndGgnqpW8ygiwqm255+jb79Zr4wDvlC1ceCAT84Ip+wOh7pe3Ekjh47RzdcPVvUaVbV86deWHJOlgsGjfni9XnXr1s14eL3eYrt2OBz/+yuCSktLU8eOHXX99ddr1apVZXap1Mqhdu3amjZtmhwOh+Lj44/D0Yenrk3qaOX2fUp87XMpKD3c81wt/P5n5RQeVJ82DfRHdr5ckU7ZbIfmFW5sd6aSPvpGN7+eokJ/QEMubK4qkZV+hK7cLFm4TO06J+iNj1+SzSaNvvtRXdb7ElWJqaI5Mz8sZZuleuTZh/TaB8/JGeHUxNHPqqAyfiMtJ5/MX6KOXdpp7qI3ZLPZNGLIaPW65jK5Yqpo9htzNG70k3rz/Rdkt9n17lsfaO8v+/TKC29q0pRH9N781xURGaFJ46cqNydXO7b/rNffnq683Dx9vXyNUpKXW314Fe8YKod+/fqpd+/exvrDqwZJio+Pl9frVVxcnNxut7777jtt2rRJffv2LXP/tmA5nTOWO3NUeewWx0nbEZ9b3QWUIaMw2+ouoAy79v/1PNexyp01+qi3qXLDuOPah5L4ugsAVgvD6xwIBwCwWhheIV3pr3MAAJioHADAamF4uxDCAQCsFobDSoQDAFiNcAAAGDhbCQBQUjAM77BAOACA1RhWAgAYGFYCABgYVgIAGBhWAgAYCAcAgIErpAEABioHAICBCWkAgIFTWQEABioHAEBJwTCcc+DHfgAABioHALAaw0oAAAMT0gAAA5UDAMAQhhPShAMAWI3KAQBgYM4BAGCgcgAAlBSOF8ERDgBgNSoHAICBcAAAGJiQBgAYqBwAACUFCQcAgIFwAAAYOJUVAGCgcgAAGMIwHPglOACAgcoBACwWDIZf5UA4AIDVwnBYiXAAAKtVpnCIHfB6ee0ax0Fu+pdWdwFlcMd1sboLqEBcBAcAMBEOAABD+F0DRzgAgNUYVgIAmAgHAICBYSUAQEkMKwEATFQOAICSqBwAACYqBwBASUHCAQBgIBwAACUdS+Xg8/nk8/mM9R6PRx6PJ7ScnJys9PR0uVwuXXPNNZo8ebIcDoduuOEG1ahRo9T982M/APAv5PV61a1bN+Ph9XqLPS81NVWJiYnau3evfvzxR2VkZEiSoqOjy9w/lQMAWO0YKod+/fqpd+/exvrDqwZJcjgckop+UKiwsFCNGzdWs2bN9OWXX6p79+6l7p9wAACLHcuwUsnho9LEx8fL6/UqLi5OjRs31qJFi5SWlqabbrqpzO0IBwCwWHmerdSjR49iy/fdd98RbUc4AIDFOJUVAGAK2qzugYFwAACLUTkAAAzBAJUDAKAEKgcAgCHInAMAoCQqBwCAgTkHAIAhGH6/9UM4AIDVqBwAAAbCAQBgYFgJAGAIx8qBH/sBABioHADAYlwEBwAwcBEcAMAQoHIAAJTEsBIAwBCOZysRDgBgMa5zAAAYqBwAAAYmpAEABiakAQAG5hwAAIZwHFaqtPdWstlsmj7tcS3/4iMt+ew9xcefUaz9issv1tcrFmj5Fx/plgF9i7UlnNdaSz57L7TctGkjLUv5QF8snaepUybIbq+0/6zlIhAI6OEnpuqGgcPVf/B9+jktvVj7R58sUe/EO5R4xwjN+fjT0PqX33hHNwwcrusGDCm2Hv+czWbTtKkTtGzpPC1e/K7iG5xRrP3yyy7SV8vna9nSeRow4HpJktPp1KuvPqslS+Zo+Zcf64rLL5Yk1ap1kt5/b4aSk99XSspcNWhwekUfjuWCQdtRP8pbpa0cevXqoejoKHXs3FNtE9po0hNjdPU1AyQV/Sd+clKS2rW/XNnZOfpi2TzNX/CZ9u79TSPuvUM33HCNcrJzQ/saP+4BPTT6cX25fJVmvPKMrrzyEn344SdWHdoJZ8kXX6ugoECzXnpG63/YpElTX9bUiUmSpD8PZGjqS2/o/denKdYdo1uHPah257bSnl/2at0PqXrzhaeUl5ev12bPsfgoTiw9e3ZXVHS0ulxwlRISWmvixNHqc+0tkoreP5MmJal9hyuUnZ2jpSkfaMGCZHXvfqH+2P+nBgy4WzVqVNOqVZ9o/oLPNGHCKM1++wPNmTNfXbqcr8Znxmv79l0WH2HFYlgpjHRsn6BPF6dIklat/k7ntGkZamvatJG2bdupAwcyJEkrvlqjjh3bas6c+dq2fZeuve42eV+bEnr+tdfdpkAgoIiICJ16Si3t2/t7xR7MCW7tho3q0O4cSdLZzZtq4+afQm1p6b+qSaMGquqJlSQ1b9pI6zdu0o8/7VCjBvU17L/jlJWdo3vvusWSvp+oOrRP0OLFSyVJq1evVZvD3j9NmjQs/v5ZsUYdOiRozpz5mjt3Qeh5/oMHJUntzz9XP3y/SYsWvqWdu9J0771JFXcgYYJhpTAS63HLl5EZWvb7A3I4HJIkT6xbGb5DbZlZWaEPnw8+WKjCwsJi+woEAqpXr442rEvRSTVr6Mct2yrgCCqPrOwcxcbEhJbtDrsOHvRLkurF1dbWHbv0+x9/KjcvTyu/Wa/c3HwdyPBp4+Ytenr8gxozcogeePgJBcPx69m/VNH7xxda9gf8h94/nlhlZJR4/1SNVXZ2jrKysuV2x2j27BeVNPZJSdLpp8fpzwMZuvSyvtq9e49GjLizYg8mDITjsFKZ4ZCenl7scSLJ9GXJHesOLdvtdvn9RR84vswsxboPfRjFut06cNgb4a/8/PMeNT2ro1566U09OanyffMpT+4Yl7JzDg3jBQMBOZ1FH0RVPbG6f+hADR81XqMnPKNmjeNVvZpH1arGqkPbcxQREaH6p8cpMipSf/zvmyz+OeP9Yzvs/ePLVGxsiffPgaL3T1zcaVr86bt66625euedeZKk/fv/1Pz5iyVJCxYkF6viK4tA0HbUj/JWZjhMnjxZixcv1qpVq7R69epy70xF+urrNbq0R1dJUtuENvrhh02htk2bflLDhg1UvXo1RUREqGOntlq58ttS9/XB3NfUsGF9SVJmVrYCgTC8/+6/WOsWzfTl12skSet/2KRG8fVDbQcP+rV+42Z5p0/SY6NHaMeuNLVu0UytW56l5Su/VTAY1L7f9is3N0/V/lf94Z9b8fUa9fjf+ychobU2btwcatu8easaNqx/6P3TMUGrVn2nk0+uqQXzZ2nUQxPk9b5zaF8rDu2rU8e2Sk3dUrEHg79kC5ZRa+fn52vfvn2qW7fuUe/YGVnnH3WsvBWdbfGYWrZoKpvNpltuG67WrVvIHROjV2bM0hWXX6xRo+6W3W7X66+/redf8Ia2Pf30OL0183l16HSlJOn8dudq4uMPqaCgUDk5uRo4aIR+/XWfVYd2RHLTv7S6C0csEAho3JPTtWXbDikY1LhR92jTj1uVk5ura3tdpudenaXPv1ihqMhI9bv+al1yYSdJ0lPTZ2j1d+sVDAY17Pb+6tD2HIuP5Mi547pY3YUy2Ww2TZ3yqJo3L3r/DBx4r1q1biG326UZM97S5ZddpAcfvFt2u01e77t64UWvnnpyrPr0ubLYsGvPnok6+eST9MLzT8jlcsnny1RivyGh+YpwlZ+3+7jub2Xtq496m3bpc49rH0oqMxz+iXAPh8ru3xQOlVG4h0Nld7zDYcVp1xz1Nu1/Kd8z8Crt2UoAEC64fQYAwBCOs5SEAwBYLCgqBwBACYEwvASHcAAAiwWoHAAAJTGsBAAwMCENADBQOQAADFQOAAAD4QAAMDCsBAAwBMIvGwgHALAa1zkAAAxheIF05f2ZUABA6agcAMBinK0EADAEbMw5AABKCMc5B8IBACzGsBIAwHAs1zn4fD75fD5jvcfjkcfjCS0nJycrPT1dLpdLffr00Z49e/Tss89q0qRJZe6fcAAAix3LdQ5er1fTpk0z1g8ePFhDhgwJLaempmro0KGaPn26CgoKlJKSonr16v3t/gkHALDYscw59OvXT7179zbWH141SJLD4Sj6O4JBrVy5UtnZ2Vq3bp327NmjOnXqlLp/wgEALHYsw0olh49KEx8fL6/Xq7i4OLVr106dO3dWQUFBmcEgEQ4AYLnynJDu0aOHse7wYafSEA4AYDFOZQUAGLgrKwDAwHUOAAAD4QAAMAQZVgIAlETlAAAwEA4AAEM4nsrKL8EBAAxUDgBgMa5zAAAYmHMAABgIBwCAIRwnpAkHALAYcw4AAAPDSgAAA8NKAABDIAzjodzCwWl3lNeucRy0Out6q7uAMhxYlGR1F1CBGFYCABjCr24gHADAclQOAAADp7ICAAyVakIaAHBkwi8aCAcAsBxzDgAAQzgOK/FjPwAAA5UDAFgs/OoGwgEALMecAwDAEI5zDoQDAFgs/KKBcAAAyzGsBAAwBMOwdiAcAMBiVA4AAAMT0gAAQ/hFA+EAAJajcgAAGJhzAAAYOFsJAGCgcgAAGKgcAAAGKgcAgCEQDL/KgR/7AQAYqBwAwGLhVzcQDgBgOS6CAwAYOFsJAGDgbCUAgIFhJQCAgWElAICBYSUAgCEYhhfBEQ4AYDHmHAAAhvIcVkpOTlZ6erpcLpe6deumadOmSZLuuusu1ahRo9TtuH0GAFgseAx/fD6f0tLSjIfP5yu279TUVCUmJmrv3r2KiorS8OHDVb9+fWVnZ5fZJyoHALDYsQwreb3eUBVwuMGDB2vIkCGhZYfDIaloXsPlcik5OVlVq1ZV3bp1y9w/4QAAFjuWCel+/fqpd+/exnqPx1NsOT4+Xl6vV3Fxcdq8ebNef/11tW/fXnv27FGdOnVK3T/hAAAWO5Y5B4/HYwTBX+nRo0ex5ZkzZx7R/gkHALAYF8GFEZvNpilTHlWLFk1VUFCgQYPu0/btu0Ltl112kR58cJgOHjyoN954V6++Olt2u13PPz9RZ54ZL7/fr4EDRxTb5oknxmjLlu165ZUjS2YcGZvNptET71PjsxqpIL9ASfdM0M8704o9J7pKlF55d6pGD39UO7buktPp0LhnR6tO3dMUGRWhF595TSmffmnREZzYAoGgJsz+TFvS9inC6VTSTd1V7+TqofYFq1L1ZvIa2e12XdW+ua7r0lqS9J9HvXJHR0mS6tSsqkf6XWpJ/8MBp7KGkZ49uys6OkoXXNBbCQmtNXHiaF177a2SJKfTqUmTxqhDhyuVnZ2jlJS5WrAgWQkJRf+pL7zwanXu3C60Tc2aNTRjxjNq1KiBnn76RSsP64TU7dIuioqK1A2X36qW5zTXyIeHaUi/kaH2s85uojFPPKBTa58cWndFn0uV8WeG/jt4rKpW92hO8puEQzlJWf+T8gsP6o37b9SG7el6+v2levbOQ2Phz8xZqjlJN8sVFamrH35VPc5tqqjIoo+eGff+n1Xdxt+otOHQvv15Wrx4qSRp9eq1atOmZaitSZOG2rZtpw4cyJAkrVixRh06JGju3AVauHCJJKlevTjt2/e7JMntjtH48c+oe/cLK/YgKok2bc/W8pSVkqQN3/6gs85uUqw9MjJSw26+T49NGxtat/ijJVr88eeh5YN+f4X0tTJauzVNHc6qL0lq2aC2Nu76tVh7o7haysrNl8NuL5p4tUlb0vYpr6BQgya/K38gqCG9Oqllg9pWdD8scIV0GPF43MrIyAwtBwJ+ORwO+f1+eTyxxdqysrJUtWqsJMnv9+uVV55Wz57d1bfvIEnSzp27tXPnbsKhnMTExijTlxVaDvgDoddKktau2WBsk5OTK0lyxbj07IzHNfXxFyqms5VQdl6B3FWiQssOu00H/QE5HUWXUTWsXVPXT3hTVaIi1LVVI3lc0YqOjFDiRefp6o4t9fO+P3XX1Pc17+FbQ9tUNuE4rFTqKzFr1ixNnjxZGRkZ+uijjyqyTxXC58tSbKw7tGyz2UMfNj5fpmJjY0JtbrdbBw4curDk1lvvUYsWF+i55ybK5apScZ2upLIzsxXjdoWWbfZDr1VZTq19sl6b+5w+em+RFsxdXJ5drNRioiOVnVcQWg4Eg6EP+S1p+/Tl99u14NGBWvjoQP2ZmaPF3/6o00+ursvbNpPNZtPpp9RQ1Zgq+j0jq7S/4oR3LBfBlbdSwyE7O1uDBg3Syy+/rG3btpV7Ryra119/ox49ir7pJyS01saNm0NtmzdvVcOG9VW9elVFRESoY8e2WrXqW/Xte7VGjrxLUtE300AgIL8/HO+neGJZu3qDOndrL0lqeU5z/bRp699uc1KtGnrpnSl6evw0fTD74/LuYqXWKr6Olv+wXZK0YXu6GtWpFWpzV4lSVKRT0RFOOex2VY91yZeTp3krvtdT7y+VJO07kKXsvALVrOr+q91XCoFg8Kgf5a3UYSWPx6PMzEzdcccdSkpKKveOVLQPP/xE3bp1UkrKXNlsNg0cOEL/+U8vud0xmjHjLd133zh9/PFM2e12eb3vKD19r+bNW6SXXnpKycnvyel0asSIh5Wfn2/1oZzwkhcu1fldEjRz/suy2Wx6aNg4XX71JXLFuPTem/P+cpvbhvVX1WoeDRo+QIOGD5AkDeo7XPl5vF7HW9dWZ2rlpl1KfGKWFAzq4X6XauHqVOXkF6pPp7PVp9PZ6j/pLUU4HYqrVU29zm8uSRrtXaT+k96SzSaNTexRaYeUJIXhoJJkC5bTTEh0dL3y2C2Ok/iqp1ndBZThm7dvs7oLKEOVC289rvvrUKfrUW/z1Z7P//5J/0ClnZAGgHARjhPShAMAWIxTWQEABioHAICBeysBAAwMKwEADAwrAQAMVA4AAAOVAwDAwIQ0AMBQEfdKOlqV92YmAIBSUTkAgMUYVgIAGMJxWIlwAACLUTkAAAxUDgAAA5UDAMBA5QAAMFA5AAAMwWDA6i4YCAcAsBj3VgIAGLgrKwDAQOUAADBQOQAADJzKCgAwcCorAMDAsBIAwMCENADAEI6VA78EBwAwUDkAgMU4WwkAYAjHYSXCAQAsxoQ0AMBA5QAAMDDnAAAwcIU0AMBA5QAAMDDnAAAwMKwEADBQOQAADIQDAMAQftEg2YLhGFkAAEtxV1YAgIFwAAAYCAcAgIFwAAAYCAcAgIFwAAAYCAcAgIFwAAAYCAcAgIHbZxyB5ORkpaeny+VyqU+fPlZ3ByXk5eVpzJgxGjp0qOLi4qzuDkqYPXu2MjIy5Ha7deONN1rdHRwhKocjkJqaqsTERO3du9fqruAv/Pbbb2rSpInV3UApunXrpv79++u3336zuis4CoTDEXA4HJLC886JkOrWratq1apZ3Q2UwuVyaerUqerfv7/VXcFRIByOQHx8vLxeL0MWwDFISkqS3W7XsmXLrO4KjgJ3ZQUAGKgcAAAGwgEAYCAcAAAGwgEAYCAcAAAGwgEAYCAcAACG/we6tDoeL7rUWgAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 504x360 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "cm = confusion_matrix(y_test, pred_knn) \n",
    "cm = cm.astype('float') / cm.sum(axis=1)[:, np.newaxis]\n",
    "print(cm)\n",
    "\n",
    "df_cm = pd.DataFrame(cm, range(3), range(3))\n",
    "plt.figure(figsize=(7,5))\n",
    "sn.set(font_scale=0.5) # for label size\n",
    "sn.heatmap(df_cm, annot=True, annot_kws={\"size\": 10}) # font size\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "              precision    recall  f1-score   support\n",
      "\n",
      "     class 0       0.91      0.78      0.84       246\n",
      "     class 1       0.72      0.96      0.82       231\n",
      "     class 2       0.94      0.85      0.89       479\n",
      "\n",
      "    accuracy                           0.86       956\n",
      "   macro avg       0.86      0.87      0.85       956\n",
      "weighted avg       0.88      0.86      0.86       956\n",
      "\n",
      "\n",
      "F1-score is : 0.8519773874190891\n",
      "Cross validation score is : 88.47533632286995\n"
     ]
    }
   ],
   "source": [
    "target_names = ['class 0','class 1','class 2']\n",
    "print(classification_report(y_test, pred_knn, target_names=target_names))\n",
    "print()\n",
    "\n",
    "f1_knn = f1_score(y_test, pred_knn, average='macro')\n",
    "cross_val_knn = np.mean(cross_val_score(knn_model, X_train, y_train, cv=10))*100\n",
    "print('F1-score is :', f1_knn)\n",
    "print('Cross validation score is :', cross_val_knn)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## __LogisticRegression()__"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "LogisticRegression Accuracy: 94.6652719665272%\n"
     ]
    }
   ],
   "source": [
    "with ignore_warnings(category=ConvergenceWarning):\n",
    "\n",
    "    logreg = LogisticRegression()\n",
    "    logreg.fit(X_train, y_train) \n",
    "    pred_logreg = logreg.predict(X_test) \n",
    "    accuracy_logreg = logreg.score(X_test, y_test) * 100\n",
    "    print(\"LogisticRegression Accuracy: {}%\".format(accuracy_logreg))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## __Confusion Matrix and Classification Report of LogisticRegression()__"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[0.96341463 0.02845528 0.00813008]\n",
      " [0.03896104 0.92640693 0.03463203]\n",
      " [0.01670146 0.03549061 0.94780793]]\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYcAAAEsCAYAAAAoxX9TAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8rg+JYAAAACXBIWXMAAAsTAAALEwEAmpwYAAAdMElEQVR4nO3de5xN9f7H8ffeM2OY2XvPkPKLoRgOSbl0UKhkXAbVHCl0o8vJXR2Fcs+hyKUIXagYUhwVkURTknIt0WFQbnWYGoTZczOXvffvj3E25tvMQWbWbub19Fh/rP3d6+v7taz5zOf7/a61bD6fzycAAM5it7oBAIDAQ3AAABgIDgAAA8EBAGAgOAAADAQHAIAhuKgqzjm2v6iqxiXgqnqb1U1AIXI8uVY3AYXIzT58Seu7mJ+XIRVrXNI25FdkwQEAcJ68HqtbYCA4AIDVfF6rW2AgOACA1bwEBwBAPj4yBwCAgcwBAGAgcwAAGFitBAAwkDkAAAzMOQAA8mO1EgDAROYAADCQOQAADKxWAgAYyBwAAAbmHAAAhgDMHHgTHADAQOYAAFZjWAkAkJ/Px2olAEB+ATjnQHAAAKsxrAQAMJA5AAAM3CENADCQOQAADMw5AAAMZA4AAAOZAwDAQHAAAOTHHdIAABOZAwDAwIQ0AMBA5gAAMARg5sDLfgAABjIHALAaw0oAAEMADisRHADAamQOAAADwQEAYGBYCQBgCMDModQvZfV6vRozcbru7zlQD/Ufop8PJZ1TvuyTz9Spex917zNI7y9f5f989rxFur/nQHV5ZMA5n+PSsNlsevnl5/TFF0u0atVC1ahx1TnlHTrE6KuvlumLL5bo4Ye7SZKCg4P15psvKSFhsdat+1AdO7aWJF1/fV2tXbtEn332nl57bZJsNlux96cksNlsmjljgr76cpk++3SxoqOvPqf89o5ttGH9Cn315TI9+sh9hR5Tv/61+nrdcq1ds0SzZ00555xUrFhBu3Z+pdDQ0OLqmvV83gvfilipDw6ffblB2dnZWjDrJQ3s/bAmTZ/tLztxMkXTZ83T3BkTNXfmRK1YvUaHf0nW5q3fa9uORM1/bYrmzpioX48ctbAHJdOdd7ZT2bKhatmyk0aOfEETJozwlwUHB2vixFG6/fYH1KZNFz366H2qVOly3XtvJx0/fkKtW9+juLgeeumlf0qShg9/Qs8//7JiYu5WaGgZtW/fyqpu/anFxcWqbNlQtbjlTg0bPl6TJo7ylwUHB2vypNFq3+E+3RbTWX//+/2qVOnyAo8ZOWKgxj33km69rZNCQ8uoY4e8QN62za1a+fG7qlSpoiV9tIzXe+FbESv1w0rffb9TzW+8QZJUv9412rn7R3/ZoaRfVadWDUW4nJKketfU0vadu7TnxwOqVaO6nhg6VmnpGXqq36OWtL0ka9assT79dK0kafPm73TDDdf7y+rUqal9+w7q5Em3JGn9+i1q3ryxPvhghZYs+dj/vdzcvCddbtu2U+XLR0qSHI5w5eTkFlMvSpYWzZpo1eo1kqRNm7fqhkZnzsk119Q6fU5SJEnrv96iFi2a6qamN/zuMdu27VD5CpGSJKfToZycHEl5mXy72G7avHFlcXUrMATgnEOpzxzS0jPkDA/379uD7P4fKtWiKmvvgZ907PgJZZ46pY3fbFdmZpZOpri1c/cPenHcMI0aPEDPjJkon89nVRdKJKfToZSUVP++x+NRUFCQJMnlcsjtPlOWmpoul8ul9PQMpaWly+EI1zvvvKYxYyZLkvbtO6gpU57Vtm2fqVKlivryy43F25kSwulyyH3OOfGeOSdOh1LOPidpaYpwOQs85se9BzT1xX9qx7/XqtIVFfXF2g2SpITP1un48RPF1KMA8mfKHN5//30lJyfL6/WqatWqiouLK/LGWMERHqb0jEz/vs/rVXBw3n/4CJdTTz/eUwOHj1Olyyuqbu1olY90KTLCqepXRSkkJETVr4pSmdAyOn4yRZed/u0Uf1xqapqczrOCtt0ujycvaLvdaXI4HP4ypzNcKSl5WURU1JVauHCWZs2ar0WLPpQkTZo0Wq1b361du35Ur17dNWHCCA0cOLIYe1MypLrT5HCe+Xc/55ykpsnpOHO+nA6HTqa4CzzmpSn/VMtWdykx8Qf16d1DkyaO0uNPDC++zgSai/hh73a75Xa7jc9dLpdcLpd/PyEhQUlJSQoLC1NcXJymTZumrKws9ezZU5dffnmB9ReYOWRkZKhv377q37+/UlJSLrjhfxYNr6urdRu2SJK279ilWtHV/WW5uR5t37lb8TMnafzIQTrw0yE1vK6uGl5/rb7a+K18Pp+OHP1NmZmnFHl66AmXxoYN36hdu9skSU2aNNSOHXv8Zbt371XNmlerfPkIhYSEqHnzptq06VtdcUVFLV/+tkaMmKB58/7l//6JEyeVmpomSfrll2SVLx9RvJ0pIb7esEXtY/Pma5o2aaQdO3b5y3bt+lE1a9ZQ+fKRCgkJUYubm2rjxm8LPOb4iZNyu/POSRLnRPL5LniLj49XTEyMscXHx59TdWJiorp3767k5GTZbDb99ttvcrvdKleuXKFNKjBzqFy5smbMmKGgoCBFR0dfmn+AABRzazOt3/Kd7u/1pOTzaezwJ7Vi9RplZGbqnrgOCgkJUZdHBii0TBn1uPculY+MUMvmTfXtth3q9vcn5PP5NOKpfv70GpfGhx9+olatWmjNmg9ks9nUs+cgde0ap/DwML311rt6+umxWr58vmw2u+bN+5eSkpI1efJoRUa6NHToAA0dOkCSFBfXQ337Pq1582YoNzdX2dk56tv3GYt79+e0dOlKtY65RevWfiibzaZHHxuobt3+Jkd4uN54c4EGDxmjj1cskN1u19y5C5WU9OvvHiNJvXoN0jtvv+I/J736DLa4dxa7iMyhR48e6tSpk/H52VmDJP/PJp/Pp0OHDqlFixaqUqWKNm3apJiYmALrt/mKaLA859j+oqgWl4ir6m1WNwGFyPEwaR7IcrMPX9L6Mhdc+DBnufvHntf3PvnkEyUnJysiIkItWrTQq6++qnLlyum+++5T5cqVCzyu1K9WAgDLFeFqpdjY2HP2R448v0BEcAAAq3GHNADgz4DMAQCsFoD3SREcAMBqATisRHAAAKsRHAAAhgB8thLBAQAs5vMy5wAAyI9hJQCAgWElAICBYSUAgIFhJQCAgeAAADBwhzQAwEDmAAAwMCENADCwlBUAYCBzAADk5wvAOQde9gMAMJA5AIDVGFYCABiYkAYAGMgcAACGAJyQJjgAgNXIHAAABuYcAAAGMgcAQH6BeBMcwQEArEbmAAAwEBwAAAYmpAEABjIHAEB+PoIDAMBAcAAAGFjKCgAwkDkAAAwBGBx4ExwAwEDmAAAW8/kCL3MgOACA1QJwWIngAABWK03B4bKrWhdV1bgEUnYutroJKESFel2tbgKKETfBAQBMBAcAgCHw7oEjOACA1RhWAgCYCA4AAAPDSgCA/BhWAgCYyBwAAPmROQAATGQOAID8fAQHAICB4AAAyO9iMge32y2322187nK55HK5/PsJCQlKSkpSWFiYOnfurGnTpikoKEj333+/KlSoUGD9vOwHAP6E4uPjFRMTY2zx8fHnfC8xMVHdu3dXcnKy9uzZo5SUFElS2bJlC62fzAEArHYRmUOPHj3UqVMn4/OzswZJCgoKkpT3QqGcnBzVrl1bdevW1bp169SuXbsC6yc4AIDFLmZYKf/wUUGio6MVHx+vqKgo1a5dWytXrtShQ4f04IMPFnocwQEALFaUq5ViY2PP2R8yZMh5HUdwAACLsZQVAGDy2axugYHgAAAWI3MAABh8XjIHAEA+ZA4AAIOPOQcAQH5kDgAAA3MOAACDL/De9UNwAACrkTkAAAwEBwCAgWElAIAhEDMHXvYDADCQOQCAxbgJDgBg4CY4AIDBS+YAAMiPYSUAgCEQVysRHADAYtznAAAwkDkAAAxMSAMADExIAwAMzDkAAAwMKwUQm82mF6eO1XXX1VFWVrYG9Buq/ft/8pfHtm+lZ4Y+rtzcXM2ft1jxcxfJbrdr+szxqlWrujwer/r2HqIDB35W/QbXauq0ccrKyta/v0/UkMH/lC8QfxX4k/J6vRo3M1579v+sMiHBGvOPv6ta5Ur+8uWffaU5730sZ3g5xbW5WXe1aymPx6tnp72pg4d/kd1u17iBj6nqWcfgj7HZbJo6bazqXXeNsrOy1a/vM+dcP+07xOiZoQOUm+vR/HmLNXfOQtntds14Zbxq1aohr8er3r0G+6+fxe+9oX17D0qS3pj9tt5/f4VFPbNGIA4rldoH791+R1uVLVtGrVvdrWdHTdRz44f5y4KDgzXhhRH6253d1b7dvXr4kXt1RaWKat8hRpLUtnUXPTfuJT0/Ybgkadr05/X0kLGKbdtVbnequnS905I+lVSfb/hWWdnZWvDSaP3j4a6aNPsdf9mJlFRNn/ee5kwcpjkTh2vFmvU6nHxUX2zaKkmaP2WU+j/YWRPPOgZ/3B13tlVoaKhibuusUSNf8F8L0pnrJ+6O7opt200PP9JNV1SqqA4d866fNjH3aNzYFzX+hRGSpAYN6mn6y2+qfey9ah97b6kLDFLesNKFbkWt1GYONzX7qxI+/VKStGXLNjVsdJ2/rHadmtq//yedPOmWJG3Y8I2aNWuspUtW6pOVn0uSqlWroiNHjkmSqlT5P20+/cNo48Zv1aFjGy1a+GFxdqdE27rzB7W44XpJUv1rairxxwP+skO/HlHtGlcpwumQJF1bq4a+37VX7VvepFubNpQkJSUf02WRruJveAl2001/VcKnayXlXT+NCrt+1n+j5s2aaMmSj7Xy47zrp2q1Kjp6+vpp2LCeatWqodtvb6O9+w7o6cFjlZaWXsw9slYgDiuV2szB6XTI7U7173s8XgUFBfnLUlLOlKWlpsnlcp7+nkevzZqkiZNH68OlKyVJBw/8rOYtmkiS2rePUXh4ueLqRqmQnpEpR3iYf99utyvX45EkVav8f9r30yEdO5GizFNZ2rQ9UZlZWZKk4KAgDZ/8usa/Ok9tT58fXBpOl1Mp51w/Hv/143I65D77+klLlyvizPXz+qzJmjzlWS1Zknf9fPPNdg0fPl7t2nbVwQP/0dBhTxRjTwKDz2e74K2oFRockpKSztlKktTUNDkc4f59u90mz+kfOKmpaXKe/k1Ukhz5gkXvnoPVqEGMXp4xXmFh5dSn9xA9NaiPFr//po4e/U2//Xai+DpSCoSHlVN65in/vtfrVfDpH0QRznAN6Xm/nhz3ska9NFvXRF+lyNOBXJKeG9RLH70xSc++/KYyTp0y6sbFSXWnyuk4c43Y7Xb/9eNOTZPDeebacjjClXI6i5CkXj0HqUH9VpoxM+/6Wb5slbZ9t0OStHzZKtWvX7eYehE4vD7bBW9FrdDgMG3aNK1evVqbNm3S5s2bi7wxxWnjhm/Vtl1LSVLjxg2UuHOPv2zP7r2Kjr5a5ctHKCQkRM2aN9bmTVvV7d6/6clBfSRJmRmn5PV65fF41C62lfr2flr3dH5UFSpEas3nX1nRpRKrYd2/aN2WbZKk7bv2qlb1qv6yXI9H3+/ep7mThuv5Qb114NAvalj3L1r+2Vd6Y9EySVLZ0FDZbTYF2UttonzJ5b9+dhZy/TRv0USbNm9Vt3s76anfuX6WLpunG/5aX5LUsmVzfXc6UMBaNl8hy2qysrJ05MgRVa1ataCvFMgVXuMPNayo/Xe1Ur16tWWz2dSn9xA1aFBP4eFhmjtnoX+1ks1u09vz3tPsWfMVFlZOr7w2UZUqXa6QkGC9OOU1fbwiQbHtW2nEqCeVmZGpL7/cqLFjpljdvf/p2L8XWt2E8/bf1Uo/HPhZPp809snHtGvvQWVkntI9HVrp1QUf6PP1W1WmTIh63NVebW9uooxTpzTyxdk6diJFubkePdrldrW66Qaru3LeKtTranUTCvXf1UrX1quTd/30Gqz6DerJ4QjXnLfe9a9Wstvtmj9vsWa9nnf9vPb6JFWqdLmCT18/Kz76VPUbXKspL45RTnaOkpOPakD/YUpNTbO6i4VKyzjwv790ATZWvuuCj7kx6YNL2ob8Cg0Of0SgB4fS7s8UHEqjQA8Opd2lDg7rr+x8wcc0++X9S9qG/ErtaiUACBSBeJ8DwQEALBaAbwklOACA1XwicwAA5OMNwKftEBwAwGJeMgcAQH4MKwEADExIAwAMZA4AAAOZAwDAQHAAABgYVgIAGLyBFxsIDgBgNe5zAAAYAvAG6dL7mlAAQMHIHADAYqxWAgAYvDbmHAAA+QTinAPBAQAsxrASAMBwMfc5uN1uud1u43OXyyWXy+XfT0hIUFJSksLCwnT33Xfr8OHDmjp1qiZNmlRo/QQHALDYxdznEB8frxkzZhif9+/fXwMGDPDvJyYm6vHHH9fMmTOVnZ2tNWvWqFq1av+zfoIDAFjsYuYcevTooU6dOhmfn501SFJQUFDe3+HzaePGjUpPT9e2bdt0+PBhValSpcD6CQ4AYLGLGVbKP3xUkOjoaMXHxysqKko33nijbrnlFmVnZxcaGCSCAwBYrignpGNjY43Pzh52KgjBAQAsxlJWAICBp7ICAAzc5wAAMBAcAAAGH8NKAID8yBwAAAaCAwDAEIhLWXkTHADAQOYAABbjPgcAgIE5BwCAgeAAADAE4oQ0wQEALMacAwDAwLASAMDAsBIAwOANwPBQZMHhVG52UVWNS6BCva5WNwGFOL73I6ubgGLEsBIAwBB4eQPBAQAsR+YAADCwlBUAYChVE9IAgPMTeKGB4AAAlmPOAQBgCMRhJV72AwAwkDkAgMUCL28gOACA5ZhzAAAYAnHOgeAAABYLvNBAcAAAyzGsBAAw+AIwdyA4AIDFyBwAAAYmpAEAhsALDQQHALAcmQMAwMCcAwDAwGolAICBzAEAYCBzAAAYyBwAAAavL/AyB172AwAwkDkAgMUCL28gOACA5bgJDgBgYLUSAMDAaiUAgIFhJQCAgWElAICBYSUAgMEXgDfBERwAwGJFOeeQkJCgpKQkhYWFKSYmRjNmzJAk9evXTxUqVCjwOO6QBgCLeS9ic7vdOnTokLG53e5z6k5MTFT37t2VnJys0NBQDRw4UNWrV1d6enqhbSJzAACLXcyEdHx8vD8LOFv//v01YMAA/35QUFDe3+HzKSwsTAkJCYqIiFDVqlULrZ/gAAAWu5hhpR49eqhTp07G5y6X65z96OhoxcfHKyoqSrt379bcuXPVrFkzHT58WFWqVCmwfoIDAFjsYiakXS6XEQh+T2xs7Dn7b7/99nnVT3AAAIuxlBUAYOAmuABis9k0ffrzuv66usrKylbvPoO1b99Bf3nHjq01fNg/lJvr0dz4RXrrrXf8ZY0bN9Tzzw1Tm7b3SJLenj9TlSpdIUm66qoobd68VQ882K9Y+1PS2Gw2TZ02VvWuu0bZWdnq1/cZ7d//k7+8fYcYPTN0gHJzPZo/b7Hmzlkou92uGa+MV61aNeT1eNW712AdOPCz6je4Vovfe0P79h6UJL0x+229//4Ki3pW8ni9Xo2bOkt79h1UmZAQjRncV9WqXOkvX776C81Z9KGc4WGKa3eb7urYWpJ0z2NPyREeJkmqcuUVGvf0gN+tvzTg8RkBJO7OWJUNDdUtt8apSZNGmvjCSHW++1FJUnBwsCZNelbNmnVUenqG1n6xRCtWfKrk5KN66qk+uv++zkpPz/DX9d9AEBkZoU9X/0uDBo+xpE8lyR13tlVoaKhibuusxo0b6PkJw9WtS09JeednwgsjdOvNcUpPz1TC54v18ccJatKkoSSpTcw9uvnmphr/wgh169JTDRrU0/SX39T0l9+wsksl1udfbVZWdo4WzJyg7Yl7NOmVuZr+3FBJ0okUt6a/9Y4Wz5oipyNcjw16Vk1vuF4VK0RKkuZMHWthy1GYUnufQ7PmjbV69ReSpM2bt6pRo/r+smvq1NK+fQd18mSKcnJy9PX6LWrRvIkkaf++n9Sl62O/W+eoUU9p5itz9OuvR4q8/SXdTTf9VQmfrpUkbdmyTY0aXecvq12npvbv/0knT7qVk5OjDeu/UfNmTfTR8k81oN8wSVLValV09MgxSVLDhvUUG3ubVq1epJmvTpDDEV78HSrBtv57l1qcDsz169ZW4g/7/GWHkpJVO7q6IlxO2e12XVu7pr5P/EF79h7Uqaws9Rw8Ro8+OUrbE/dY1fyA4PP5LngraqU2OLicTqW4U/37Ho/Hvx7Y6XLInXLmRpK01HS5IvJWBSxZ+rFycnKM+i6//DK1uq255s37VxG3vHRwugo+Py6nQ+6UM2VpaelyRTj933t91mRNnvKslixZKUn65pvtGj58vNq17aqDB/6jocOeKMaelHzpGRn+4SFJstvtyvV4JEnVoq7UvoP/0bHjJ5V5Kkubtv5bmZmnVLZsqHp0idPrE0dp5MDeeua5qf5jSiOvfBe8FbUCg8OCBQs0bdo0paSkaNmyZUXekOLmTk2V86zfIO12uzyn/3OmutPkcDr8ZQ5nuFJOuo06znbXXR21cOFSeb2BuO7gzyfVnSqn48w5OPv8uFPT5HCeOXcOx7nnp1fPQWpQv5VmzByvsLByWr5slbZ9t0OStHzZKtWvX7eYelE6hIeFKT0j07/v9XoVfDqQRzgdGtLvYT05eqJGTZyha2rVUGSES1dHVdbtbW6RzWbT1VUrK9Ll1LHfTljVBcv5LuJPUSswOKSnp6t3796aPXu29u3bV9DX/rQ2rP9GsbGtJElNmjTSjp27/WW7dv+omjWrq3z5SIWEhOjmFk21cdO3hdYX0+pmfbJqTZG2uTTZuOFbtW3XUpLUuHED7dx5Zthhz+69io6+WuXLRygkJETNWzTRps1b1e3eTnpqUB9JUmbGKXm9Xnk8Hi1dNk83/DVv2LBly+b67nSgwKXRsF4drdu0VZK0PXGPatW4yl+W6/Ho+8QfNHfaOD0/9HEd+M9hNbyujpas/EyTX5krSTpy7LjS0jNV8bLyVjQ/IHh9vgveilqBE9Iul0upqanq06ePRo8eXeQNKW5LP1ypmNY3a+0XS2Wz2fRYzyfVrevfFO4I15tvLtCQIWO04qO3ZbfbNTd+kZKSfi20vr/8pYYOHPi5mFpf8i1btkqtYloo4fP3ZLPZ1KfXYN3T5U45HOGa89a7GvrMc1q6LF52u13z5y3WL0nJWvbhJ3rt9UlatXqRgkOC9fSQscrKytY/nhihKS+OUU52jpKTj2pA/2FWd69Eibm5qTZ8u10P9B8qn8+nsU/314qEL5WReUr33NFWISHB6tprsMqUCVGPLneqfIRLd3WI0fAJM9R9wDDZbDaNHdLPn22URoG3Vkmy+YpoZqNMaFRRVItLpExQiNVNQCGO7/3I6iagEGUqX3tJ62tepdUFH/P14c8vaRvyK7VLWQEgUHCfAwDAwMt+AAAGMgcAgIFnKwEADAwrAQAMDCsBAAxkDgAAA5kDAMDAhDQAwFAcz0q6UKX2kd0AgIKROQCAxRhWAgAYAnFYieAAABYjcwAAGMgcAAAGMgcAgIHMAQBgIHMAABh8Pq/VTTAQHADAYjxbCQBg4KmsAAADmQMAwEDmAAAwsJQVAGBgKSsAwMCwEgDAwIQ0AMAQiJkDb4IDABjIHADAYqxWAgAYAnFYieAAABZjQhoAYCBzAAAYmHMAABi4QxoAYCBzAAAYmHMAABgYVgIAGMgcAAAGggMAwBB4oUGy+QIxZAEALMVTWQEABoIDAMBAcAAAGAgOAAADwQEAYCA4AAAMBAcAgIHgAAAwEBwAAAYen3EeEhISlJSUpLCwMN19991WNwf5nDp1SqNGjdLjjz+uqKgoq5uDfN59912lpKTI4XDogQcesLo5OE9kDuchMTFR3bt3V3JystVNwe84evSo6tSpY3UzUICYmBg99NBDOnr0qNVNwQUgOJyHoKAgSYH55ERIVatWVWRkpNXNQAHCwsI0ffp0PfTQQ1Y3BReA4HAeoqOjFR8fz5AFcBFGjx4tu92utWvXWt0UXACeygoAMJA5AAAMBAcAgIHgAAAwEBwAAAaCAwDAQHAAABgIDgAAw/8D6xJHeYDeHIkAAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 504x360 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "cm = confusion_matrix(y_test, pred_logreg) \n",
    "cm = cm.astype('float') / cm.sum(axis=1)[:, np.newaxis]\n",
    "print(cm)\n",
    "\n",
    "df_cm = pd.DataFrame(cm, range(3), range(3))\n",
    "plt.figure(figsize=(7,5))\n",
    "sn.set(font_scale=0.5) # for label size\n",
    "sn.heatmap(df_cm, annot=True, annot_kws={\"size\": 10}) # font size\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "              precision    recall  f1-score   support\n",
      "\n",
      "     class 0       0.93      0.96      0.95       246\n",
      "     class 1       0.90      0.93      0.91       231\n",
      "     class 2       0.98      0.95      0.96       479\n",
      "\n",
      "    accuracy                           0.95       956\n",
      "   macro avg       0.94      0.95      0.94       956\n",
      "weighted avg       0.95      0.95      0.95       956\n",
      "\n",
      "\n",
      "F1-score is : 0.9411547896029623\n",
      "Cross validation score is : 93.76681614349775\n"
     ]
    }
   ],
   "source": [
    "with ignore_warnings(category=ConvergenceWarning):\n",
    "    target_names = ['class 0','class 1','class 2']\n",
    "    print(classification_report(y_test, pred_logreg, target_names=target_names))\n",
    "    print()\n",
    "\n",
    "    f1_logreg = f1_score(y_test, pred_logreg, average='macro')\n",
    "    cross_val_logreg = np.mean(cross_val_score(logreg, X_train, y_train, cv=10))*100\n",
    "    print('F1-score is :', f1_logreg)\n",
    "    print('Cross validation score is :', cross_val_logreg)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## __MLPClassifier()__"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "MLP Accuracy: 94.76987447698745%\n"
     ]
    }
   ],
   "source": [
    "MLP_model = MLPClassifier()\n",
    "MLP_model.fit(X_train, y_train)\n",
    "\n",
    "pred_mlp = MLP_model.predict(X_test) \n",
    "\n",
    "accuracy_mlp = MLP_model.score(X_test, y_test) * 100\n",
    "print(\"MLP Accuracy: {}%\".format(accuracy_mlp))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## __Confusion Matrix and Classification Report of MLPClassifier()__"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[0.95121951 0.04065041 0.00813008]\n",
      " [0.03463203 0.93939394 0.02597403]\n",
      " [0.02087683 0.02922756 0.94989562]]\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYcAAAEsCAYAAAAoxX9TAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8rg+JYAAAACXBIWXMAAAsTAAALEwEAmpwYAAAc/UlEQVR4nO3deXxU1f3G8WcmicFkZkIiaIWAQqAgxbIoUYEqEjQRFERQUFtCsaJUlqKIZROpIrKIxoRapViGKqjIIhDWxIBQAuJGfxLAAopCLLIlkxWSzPz+iK+R5DQUKMkdk8/b1/xx5+RezvEyefiec+8dm8/n8wkAgDPYre4AACDwEA4AAAPhAAAwEA4AAAPhAAAwEA4AAENwdR245NiB6jo0LoLLruphdRdwFoUlp6zuAs6i9PThi3q8C/l9GdKg+UXtQ2XVFg4AgHPkLbO6BwbCAQCs5vNa3QMD4QAAVvMSDgCASnxUDgAAA5UDAMBA5QAAMHC1EgDAQOUAADCw5gAAqIyrlQAAJioHAICBygEAYOBqJQCAgcoBAGBgzQEAYAjAyoFvggMAGKgcAMBqTCsBACrz+bhaCQBQWQCuORAOAGA1ppUAAAYqBwCAgTukAQAGKgcAgIE1BwCAgcoBAGCgcgAAGAgHAEBl3CENADBROQAADCxIAwAMVA4AAEMAVg582Q8AwEDlAABWY1oJAGAIwGklwgEArEblAAAwEA4AAAPTSgAAQwBWDnX+Ulav16spM5L14NDRGjx8rL45lF2hfcXadPUdNEyDho3RkpXr/O/3H/yYBg8fq8HDx2ri1Nk13e1az2az6aWk55T2wXtKXbNQzZtfVaE94Y7u2vjhcqV98J4SBw+o0Nag4WXK2rtFLX/evML706ZP1JCHHqj2vtdWNptNc1Je0JYPVyh9w2LFxFxdof3OXrcpc2uqtny4Qg8NeeCs+7Rr9wv9Y/NKbcpYprmvvyibzeY/ToMGUdq9a4tCQ0NramjW83nP/1XN6nzlkP5hpk6fPq23Xn9JO7/YrZnJc5U8fbIk6WROrpJfX6D35qfI6QjX70aN143Xt1eDqEhJ0vyUGVZ2vVa7867bVa/eJerRvb86dWqvqdPG6/4Bj0iSgoOD9cL0iep2890qKCjShvTFWrMmXd8fOabg4GAlvfKciotO+Y91WYMovT53llq0aKakLw9YNaSfvD59ElSvXqi63txbN8R21MwZT+uefkMklZ+TWTMn68bOvVRQUKgPNy3XqtQNuumm6//jPpMmjtZzU1/SmrUfaIE7Wb169tCq1A26/bZbNHXqeF1xRQOLR1vDLqBy8Hg88ng8xvsul0sul8u/nZaWpuzsbIWFhalPnz5KSkrSqVOnNHToUDVs2LDK49f5yuGzf+5SlxuvkyS1a3uNdu35l7/tUPa/1bplc0W4nLLb7Wp7TUvt3LVbe/cdUHHxKT38h/EaMuKP2vnFbqu6X2vd1Pl6pW34UJK0Y8fn6tDxWn9bq9YtdODAQeXkeFRSUqLMzI/VuXMnSdLUaeP0xryF+u67I/6fd4SHadrUJL29aHmNjqG26do5VuvWZ0iStn/0qa7r+Et/2zXXtNT+/V8rJydXJSUl2vqPHera9YYq9/n88y8UGVVfkuR0OlRSUiKpvJKPTxioEydyam5ggeACKge32624uDjj5Xa7Kxw6KytLgwYN0pEjR2Sz2XT8+HF5PB5deumlZ+1Sna8c8gsK5QwP92/bg+wqLS1TcHCQmkY30r6vDurYiZMKD7tU2z7eqauaRKtevVANfqCf+t2VoIPfHtajT0zSqkV/VXBwkIUjqV2cToc8njz/dlmZV0FBQSorK5PT6VBu7o9t+Xn5crmceuDX/XTs6Amlp23W408M87cfPHhIBw8e0m23d6vJIdQ6TpdDntz/fE5cTodyzzhfefn5inA5q9znX/u+UnLSVI0fN0qeXI82bsqUJKWlb665AQWSC6gcEhMT1bdvX+P9M6sGSQoKKv+95PP5dOjQIXXt2lWNGzfW9u3bFRcXV+XxqwyHJUuW6MiRI/J6vWrSpIn69Olz3p3/KXCEh6mgsMi/7fN6/b/kI1xOPTVyqEZPeE5XNGygNq1iFFnfpaubNFbT6Eay2Wy6umm06ke4dPT4CV15RdUlGs5PXl6+HI4zQttuU1lZmb/N6XT42xw/hMWjwxLl8/nU7dYuuvaXbfT63Bc14L6H9f2RYzXe/9ooz5Mvxxn/3+12u/+cePLy5TzjfDkdDuXkeqrc56UX/6Ru3e9RVtaXGvZoombOeFojR02oucEEmgsIh8rTR1WJiYmR2+1WdHS0HA6HPv30U+3evVsPPHD29bcqp5UKCwv1+9//XsOHD1dubu55d/ynosO1bbQ5c4ckaecXu9Uyppm/rbS0TDt37ZF7zkxNmzRGXx08pA7XttHS1PWamTxXkvT90eMqKChUw8uiLOl/bbUt8xPdHt9NktSpU3tl7drrb9u7Z59iYq5WZGSEQkJC1LlLJ320/VPdET9QPRPuV687HtD//TNLQx9+gmC4iP6RuUN3JHSXJN0Q21FfnDGdunv3v9SiRXNFRtZXSEiIuv7qBm3b9kmV+5w4mSOPJ1+SlP3dEUVGRtTwaAKMz3f+r3OUkJCgxMRE3X333WrQoIEmTZqkMWPGqFGjRmfdr8rKoVGjRkpJSVFQUJBiYmLOfZA/MXG3dNbWHZ/pwUcel3w+PTvhcaWuz1BhUZHu7dNTISEhum/ICIVecokS779HkfUj1O/OeE2YOlu/GfaEbLLpT+NHM6V0ka1csU63du+qDemLZbPZNOzRsbr3vt4KDw/T/L+9rXF/fE7L3nfLZrfpzQXvVVhjQPVYvnyNesTdrM2b3pfNZtNDD4/WwIF3yxEerr/Oe0tPjp2i1alvyW63a/78t5Wd/e//uI8kPfLIGC18888qLS3V6dMlemTYkxaPzmIBeCmrzec7jwg6DyXHuCokkF12VQ+ru4CzKCw59d9/CJYpPX34oh6v6K1J573PpQ8+e1H7UFmdX5AGAMtxhzQAwBCA00p1/j4HAICJygEArFY9S7//E8IBAKwWgNNKhAMAWI1wAAAYuFoJAFCZz8uaAwCgMqaVAAAGppUAAAamlQAABqaVAAAGwgEAYOAOaQCAgcoBAGBgQRoAYOBSVgCAgcoBAFCZLwDXHPiyHwCAgcoBAKzGtBIAwMCCNADAQOUAADAE4II04QAAVqNyAAAYWHMAABioHAAAlQXiTXCEAwBYjcoBAGAgHAAABhakAQAGKgcAQGU+wgEAYCAcAAAGLmUFABioHAAAhgAMB74JDgBgoHIAAIv5fIFXORAOAGC1AJxWIhwAwGp1KRwim8ZV16FxEZzcv9rqLuAsnM3ire4CahA3wQEATIQDAMAQePfAEQ4AYDWmlQAAJsIBAGBgWgkAUBnTSgAAE5UDAKAyKgcAgKkaK4e0tDRlZ2crLCxM/fr1U1JSkoKCgvTggw8qKiqqyv0IBwCwmO8CwsHj8cjj8Rjvu1wuuVwu/3ZWVpZGjhypOXPmaO/evcrNzVVUVJTq1at31uMTDgBgtQsIB7fbrZSUFOP94cOHa8SIEf7toKAgSeVPfi0pKVGrVq3Upk0bbd68WfHxVT+mhXAAAItdSOWQmJiovn37Gu+fWTVIUkxMjNxut6Kjo9WqVSutWbNGhw4d0m9+85uzHp9wAICfoMrTR1VJSEiosD127NhzOj7hAABW41JWAEBlFzKtVN0IBwCwGOEAADAQDgAAk89mdQ8MhAMAWIzKAQBg8HmpHAAAlVA5AAAMPtYcAACVUTkAAAysOQAADL7A+64fwgEArEblAAAwEA4AAAPTSgAAQyBWDnarOwAACDxUDgBgMW6CAwAYuAkOAGDwUjkAACpjWgkAYAjEq5UIBwCwGPc5AAAMVA4AAAML0gAAAwvSAAADaw4AAEMgTivV2Wcr2Ww2Jb3ynNIzlmjN2kVq3vyqCu139IzTps3LlZ6xRIN/O1CSZLfb9ee/TNeG9MVat/4dNWvWVJLUrv0v9OW+TK1Zu0hr1i5Sv369anw8tZnX69WUF1/Vg79/SoNHTdA3h76r0L5ifYb6DhmlQSPGaUnqhgptx0/mKO7eh3Tg4KGa7HKtZ7PZlJz8vDZuXKb1698xPj89e/bQli0rtXHjMg0Zcr8kKTg4WG+88bLS09/T5s0r1KvXbZKkhg0v0+LFf1Va2mJlZCw1jlUX+Hy2835VtzpbOdzV+3aFhoYq7tZ+6tSpvZ5/YYIG3jdUUvlf4hemT9Qtv+qjgoIipX2wWKtXpyk2toMk6ba4e/WrX92gadMnauB9Q9W+fVslvzJPya/81coh1VrpW7br9OnTeuvP07Vz117NfPVvSp46XpJ0Msej5HkL9d7c2XI6wvW7Jybrxo6/VOMrr1BJaammvPiq6oWGWjyC2qd373jVqxeqbt36Kja2g6ZPn6R77/2dpPLPz8yZT6tLl7tUUFCojIylSk1NU3x8Nx0/flJDhvxBUVH1tX37GqWmbtDzz4/X228v15Ilq3TLLTfp5z+P0YEDBy0eYc0KxGmlOls53HTT9UrbsEmStGPH5+rY8Vp/W6vWLXTgwEHl5HhUUlKizK0fq0vnWK1auUEjHiv/pdSkaWMd/f6YJKlDh7ZKSLhV69a/ozmvviCHI7zmB1SLffZ/u9UltqMkqd0vWmnX3n3+tkPf/VutWzRThMspu92utq1baGfWl5KkWa/O14DeCWp4WaQl/a7NOnfupPXrN0qSPvroM3Xs+Et/W+vWLbR//9fKyclVSUmJtm7doS5dYrVkSaqmTJnl/7nS0jJJ5Z/Fxo1/ptWrF2rgwLv14YeZNTqWQOD12c77Vd3qbDg4XU7levL822VlZQoKCpIkuZwOeXJ/bMvPL5Arwun/udden6VZLz6jZcvWSJI+/ninJkyYpvjbB+jrr77VuPGjanAktV9+QaGcjjD/tt1u9/9iaRrdSPu++kbHTuSoqPiUtn3yTxUVF2v5mnRFRbjU5YdqDxeXy+VQ7hmfEa/3jM+Py1mhLT8/XxERThUUFCo/v0AOR7gWLfqLnnlmpiTpqquilZOTq549H9C332ZrzJhhNTuYABCI00pnDYfs7OwKr9okz5Mnp8Ph37bb7SorK/+F48nLl8P547/+HY5w5eZ4/NuPDB2j9u26K2XONIWFXaqVK9bp88++kCStXLFO7dq1qaFR1A2O8DAVFBb5t31en4KDy38RRTgdemr4EI1+eromTU9Wm583V2SES8vWpCvzk50aPGqC9u77SuOnJenY8ZNWDaHW8Xjy5XT++Pmx2c74/Hjy5Kzw+XEo54fPT3T0lVq37h0tXLhU77zzviTp+PGTWrWqfK0oNTWtQhVSV/zkKoekpCStX79e27dv10cffVTtnalJ2zI/0e3x3SRJnTq1165de/1te/fsU0zM1YqMjFBISIi6dI3V9o8+1cD7++qJH/5VU1RYLK/Xq7KyMi1fsUDXXd9OktStWxd99kNQ4OLo0La1Nm/7RJK0c9detTxjwbK0tEw7s76U+5WpmjZ+lL765rA6tL1G7lee1/ykqZqfNFWtWjTT8+NGqQHTSxdNZubHSki4VZIUG9tBu3bt8bft2bNPLVo0839+una9Qdu3f6LLL2+gVave1MSJ0+R2v+v/+a1bP1ZCQndJUteuNyjrh2lBWMvm81W9FHLq1Cl9//33atKkyXkf2BHW7H/qWHWz2Wx6OelZ/aJta9lsNg175Em1a99WDke4/vbGIt3RM05/HDdCdrtdf1+wWK+/9neFhV2qv7w2U1dc0VDBIcGa/eJflLpqg9q1/4VenD1FJadLdOTIUY0YPl55eflWD/GsTu5fbXUXzpnX69WzL72mLw98LfmkZ58aod3/2q/ComLde1e8/jz/bX2wZbtCL7lEiff10e3dOlfYf/CoCXr68WFqflW0NQO4AM5m8VZ34axsNpteeWWq2v7w+Rk6dIw6dCj//Mybt1A9e/bQ+PGjZLfb5Xa/o9deW6BZs55R//536ssv9/uP07v3IF1+eQO9+uoMhYVdKo8nT4mJI5WTk2vh6P674uJvLurxtjW657z3uTF76UXtQ2VnDYf/RaCHQ133UwqHuijQw6Guu9jhsPXKfue9T+fvllzUPlRWZy9lBYBAweMzAACGAPyWUMIBAKzmE5UDAKASbwDeIU04AIDFvFQOAIDKmFYCABhYkAYAGKgcAAAGKgcAgIFwAAAYmFYCABi8gZcNhAMAWI37HAAAhgC8Qbrufk0oAKBqVA4AYDGuVgIAGLw21hwAAJVcyJqDx+ORx+Mx3ne5XHK5XP7ttLQ0ZWdnKywsTP3799fhw4f18ssva+bMmWc9PuEAABa7kGklt9utlJQU4/3hw4drxIgR/u2srCyNHDlSc+bM0enTp5WRkaGmTZv+1+MTDgBgsQu5zyExMVF9+/Y13j+zapCkoKAgSZLP59O2bdtUUFCgzz//XIcPH1bjxo2rPD7hAAAWu5D7HCpPH1UlJiZGbrdb0dHRuvHGG3XzzTfr9OnTZw0GiXAAAMtV530OCQkJxntnTjtVhXAAAIvx+AwAgIH7HAAAhkB8fAbhAAAWY1oJAGBgWgkAYCAcAAAGH9NKAIDKqBwAAAbCAQBgCMRLWfkmOACAgcoBACzGfQ4AAANrDgAAA+EAADAE4oI04QAAFmPNAQBgYFoJAGBgWgkAYPAGYDxUWziUeEur69C4CFzNze+VReDIO7TR6i6gBjGtBAAwBF7dQDgAgOWoHAAABi5lBQAY6tSCNADg3AReNBAOAGA51hwAAIZAnFbiy34AAAYqBwCwWODVDYQDAFiONQcAgCEQ1xwIBwCwWOBFA+EAAJZjWgkAYPAFYO1AOACAxagcAAAGFqQBAIbAiwbCAQAsR+UAADCw5gAAMHC1EgDAQOUAADBQOQAADFQOAACD1xd4lQNf9gMAMFA5AIDFAq9uIBwAwHLcBAcAMHC1EgDAwNVKAAAD00oAAAPTSgAAA9NKAACDLwBvgiMcAMBiF7Lm4PF45PF4jPddLpdcLpd/Oy0tTdnZ2QoLC1NcXJxSUlIkSY899piioqKqPD53SAOAxbwX8HK73YqLizNebre7wrGzsrI0aNAgHTlyRKGhoRo9erSaNWumgoKCs/aJygEALHYhC9KJiYnq27ev8f6ZVYMkBQUFlf8ZPp/CwsKUlpamiIgINWnS5KzHJxwAwGIXMq1UefqoKjExMXK73YqOjtaePXs0f/58de7cWYcPH1bjxo2r3I9wAACLVeeCdEJCQoXtN99885z2IxwAwGJcygoAMATiTXB19molm82mlOTntWnjcq1f/65iml9dob1Xzx76x5ZV2rRxuYYMuV+SFBwcrDfeeFnp6Uu0ZfNK3dnrtgr7zJwxWQ//7tc1NYRazWaz6ZVXpmrjxmVat+5tNW9+VYX2nj3jtGXLCm3cuEy//e1ASeXnZ968l5SWtlibN7+vXr16SJLat2+rzZvfV1raYs2ePUU2m63Gx1Obeb1eTZmRrAeHjtbg4WP1zaHsCu0r1qar76BhGjRsjJasXOd/v//gxzR4+FgNHj5WE6fOruluBxSvfOf9qm51tnLo3TteofXq6ZZudys2toOmT5+k/vc+JKn8l8zMmZPVucudKigo1MaMZUpNTVN8/K06cfykhgz5g6Ki6mv79rValbpBDRpE6Y15L6tly+aaPXu/xSOrHXr3jle9eqHq1q2vYmM76IUXJuq++x6WVH5+Zsx4Wl273qWCgiJlZCzR6tXpuv32bjpx4qQeemi0oqLqa9u21UpNTVNKyjSNGfOMtm37RJMnj9GAAXfr7beXWTzC2iP9w0ydPn1ab73+knZ+sVszk+cqefpkSdLJnFwlv75A781PkdMRrt+NGq8br2+vBlGRkqT5KTOs7DrOos6GQ5fOsVq/fqMk6aOPPlPHjr/0t7Vu3UL793+tnJxcSdLWrTvUpUuslixZpaVLU/0/V1ZaKklyOML17HOzFR9/a80NoJbr3LmTNmzYJKn8/Fx33X86P+U3AJWfn05aujRVy5at9v9caWmZJKlx459p27ZPJEmZmR/rzjtvIxwuos/+uUtdbrxOktSu7TXatedf/rZD2f9W65bNFeFySpLaXtNSO3ftVvSVP1Nx8Sk9/IfxKivzatQjiWrX9hpL+h8IAvEO6To7reR0OeTJ/fHuwjJvmf96YJfLqdzcPH9bXn6+IiKcKigoVH5+gRyOcC1a9JomPzNLkvT1199qx47Pa7T/tZ3T6ahwDsrKzjw/Dnk8Z5yfvAK5XK4K52fhwr9oypQfz0/XrjdIknr27KHw8LAaHEntl19QKGd4uH/bHmT3B3PT6Eba99VBHTtxUkXFxdr28U4VFZ1SvXqhGvxAP73+0lQ9/eRwPTVlhn+fuigQp5WqDIe33npLSUlJys3N1YoVK6q9IzUtz5Mvh9Ph37bb7CorK//L6fHkyen88S+70+Hw/ys1OvpKrV/3rhYuXKp33lleo32uS/Ly8iucA7v9zPOTL4fjx3PndIYrN/fH87N27ds/nJ/3JUlDh47Rk0/+XkuX/k1Hjx7T8eMnanAktZ8jPEwFhUX+bZ/Xq+Dg8iCPcDn11MihGj3hOU16/iW1aRWjyPouXd2kse6M7y6bzaarm0arfoRLR+vwefFdwH/VrcpwKCgo0KOPPqq5c+dq//7aN4++NXOHEhK6S5JiYzto1649/rY9e/apRYtmioysr5CQEHXtGqvt2z/V5Zc3UOqqtzRh4vNyu9+xqut1Qmbmx/5putjYDvrii73+tvLzc7UiIyMUEhKiLl1u0Pbtn+jyyxto5co3NXHiC1qw4F3/z99xR3c9+uiTuuee3+qyyyKVnr6lxsdTm3W4to02Z+6QJO38YrdaxjTzt5WWlmnnrj1yz5mpaZPG6KuDh9Th2jZamrpeM5PnSpK+P3pcBQWFanhZ1c/5qe28Pt95v6pblWsOLpdLeXl5GjZsmCZPnlztHalp77+/Vj3ifqWNGctks9k0dOgTGjDgbjkcYZo3b6HGjv2TVq18U3a7TW73u8rO/rdenPWM6teP0LhxozRu3ChJUu/eg1RcXGzxaGqf999fq+7duyojY+kP52eMBgzoo/DwML3xxiI99dSzWrny77LZ7Fqw4F1lZx/RrFmTVb++S+PGjdC4cSMkSX36JGrfvq+0bNl8FRUVa9OmTK1bl2Hx6GqXuFs6a+uOz/TgI49LPp+enfC4UtdnqLCoSPf26amQkBDdN2SEQi+5RIn336PI+hHqd2e8Jkydrd8Me0I22fSn8aP91UZdFHgrDpLNV00rIaH1zv7cDljLbquzy00/CZ5vCbBAFtKg+UU9XpfG3c97n38c/uCi9qGyOnu1EgAECr4mFABgCMRLWQkHALAYlQMAwBCIz1YiHADAYkwrAQAMTCsBAAxUDgAAA5UDAMDAgjQAwFATz0o6XzxDAQBgoHIAAIsxrQQAMATitBLhAAAWo3IAABioHAAABioHAICBygEAYKByAAAYfD6v1V0wEA4AYDGerQQAMPBUVgCAgcoBAGCgcgAAGLiUFQBg4FJWAICBaSUAgIEFaQCAIRArB74JDgBgoHIAAItxtRIAwBCI00qEAwBYjAVpAICBygEAYGDNAQBg4A5pAICBygEAYGDNAQBgYFoJAGCgcgAAGAgHAIAh8KJBsvkCMbIAAJbiqawAAAPhAAAwEA4AAAPhAAAwEA4AAAPhAAAwEA4AAAPhAAAwEA4AAAOPzzgHaWlpys7OVlhYmPr37291d1BJcXGxnn76aY0cOVLR0dFWdweVLFq0SLm5uXI4HPr1r39tdXdwjqgczkFWVpYGDRqkI0eOWN0V/AdHjx5V69atre4GqhAXF6fBgwfr6NGjVncF54FwOAdBQUGSAvPJiZCaNGmi+vXrW90NVCEsLEzJyckaPHiw1V3BeSAczkFMTIzcbjdTFsAFmDx5sux2uzZt2mR1V3AeeCorAMBA5QAAMBAOAAAD4QAAMBAOAAAD4QAAMBAOAAAD4QAAMPw/Jtr8iSBnkAoAAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 504x360 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "cm = confusion_matrix(y_test, pred_mlp) \n",
    "cm = cm.astype('float') / cm.sum(axis=1)[:, np.newaxis]\n",
    "print(cm)\n",
    "\n",
    "df_cm = pd.DataFrame(cm, range(3), range(3))\n",
    "plt.figure(figsize=(7,5))\n",
    "sn.set(font_scale=0.5) # for label size\n",
    "sn.heatmap(df_cm, annot=True, annot_kws={\"size\": 10}) # font size\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "              precision    recall  f1-score   support\n",
      "\n",
      "     class 0       0.93      0.95      0.94       246\n",
      "     class 1       0.90      0.94      0.92       231\n",
      "     class 2       0.98      0.95      0.97       479\n",
      "\n",
      "    accuracy                           0.95       956\n",
      "   macro avg       0.94      0.95      0.94       956\n",
      "weighted avg       0.95      0.95      0.95       956\n",
      "\n",
      "\n",
      "F1-score is : 0.9417600951866049\n",
      "Cross validation score is : 94.48430493273543\n"
     ]
    }
   ],
   "source": [
    "with ignore_warnings(category=ConvergenceWarning):\n",
    "    target_names = ['class 0','class 1','class 2']\n",
    "    print(classification_report(y_test, pred_mlp, target_names=target_names))\n",
    "    print()\n",
    "\n",
    "    f1_mlp = f1_score(y_test, pred_mlp, average='macro')\n",
    "    cross_val_mlp = np.mean(cross_val_score(MLP_model, X_train, y_train, cv=10))*100\n",
    "    print('F1-score is :', f1_mlp)\n",
    "    print('Cross validation score is :', cross_val_mlp)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## __SVMClassifiers()__"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy of SVM classifier, considering different kernels\n",
      "\n",
      "Kernel = linear: 0.9351464435146444\n",
      "Kernel = rbf: 0.9612970711297071\n",
      "Kernel = poly: 0.9414225941422594\n",
      "Kernel = sigmoid: 0.9372384937238494\n"
     ]
    }
   ],
   "source": [
    "svm_lin = SVC(kernel=\"linear\", probability=True)  \n",
    "svm_lin.fit(X_train, y_train)\n",
    "\n",
    "svm_rbf = SVC(kernel=\"rbf\", probability=True)  \n",
    "svm_rbf.fit(X_train, y_train)\n",
    "\n",
    "svm_poly = SVC(kernel=\"poly\", probability=True)  \n",
    "svm_poly.fit(X_train, y_train)\n",
    "\n",
    "svm_sigm = SVC(kernel=\"sigmoid\", probability=True) \n",
    "svm_sigm.fit(X_train, y_train)\n",
    "\n",
    "print(\"Accuracy of SVM classifier, considering different kernels\")\n",
    "print()\n",
    "\n",
    "print(\"Kernel = linear: \" + str(svm_lin.score(X_test, y_test)))\n",
    "print(\"Kernel = rbf: \" + str(svm_rbf.score(X_test, y_test)))\n",
    "print(\"Kernel = poly: \" + str(svm_poly.score(X_test, y_test)))\n",
    "print(\"Kernel = sigmoid: \" + str(svm_sigm.score(X_test, y_test)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## __RBF SVMClassifier()__"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "SVM Accuracy: 96.1297071129707%\n"
     ]
    }
   ],
   "source": [
    "svm_model = SVC(probability=True)\n",
    "\n",
    "svm_model.fit(X_train, y_train)\n",
    "pred_svm = svm_model.predict(X_test)\n",
    "accuracy_svm = accuracy_score(y_test, pred_svm)*100\n",
    "print(\"SVM Accuracy: {}%\".format(accuracy_svm))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## __Confusion Matrix and Classification Report of RBF SVMClassifier()__"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[0.96341463 0.03252033 0.00406504]\n",
      " [0.02597403 0.95670996 0.01731602]\n",
      " [0.01878914 0.01878914 0.96242171]]\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYcAAAEsCAYAAAAoxX9TAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8rg+JYAAAACXBIWXMAAAsTAAALEwEAmpwYAAAa8ElEQVR4nO3deXxU5b3H8e9MEgLJzCRBkFsJIAQuS0UFa1CSuhAVBAtGXEAsIFqUsngRxA1E3BCCWmSpwtUycFHBS0UsIDEICLIKgpewqEjVkBJQyUw2lmTO/QM7kjwkJZTkDMnn7Wv+OOfJeXgeh5Mfv99zFodlWZYAADiF0+4BAABCD8EBAGAgOAAADAQHAICB4AAAMBAcAACG8Krq+MQP31RV1zgH4pqm2D0EVOBo8XG7h4AKFB8/cE77O5vflxENWpzTMZRVZcEBAHCGAiV2j8BAcAAAu1kBu0dgIDgAgN0CBAcAQBkWmQMAwEDmAAAwkDkAAAxcrQQAMJA5AAAMrDkAAMriaiUAgInMAQBgIHMAABi4WgkAYCBzAAAYWHMAABhCMHPgTXAAAAOZAwDYjbISAKAsy+JqJQBAWSG45kBwAAC7UVYCABjIHAAABu6QBgAYyBwAAAbWHAAABjIHAICBzAEAYCA4AADK4g5pAICJzAEAYGBBGgBgIHMAABhCMHPgZT8AAAOZAwDYjbISAMAQgmUlggMA2I3MAQBgIDgAAAyUlQAAhhDMHGr9payBQEATJk9Tv8EjNXDYGH2XlV2qfcmHK5Xaf4j6DxmtRR+sCO6fPXeB+g0eqTsHDS+1H+eGw+HQ1Fef08pVi7T8w7fVokWzUu03d0/RmrWLtXLVIg28t48kyel0auZrk/TRyne1In2BmjdvKklq06al0jMW6qOV7+qVPz0jp7PW/7U/Kw6HQzOmv6h1nyzRyo/eVULCxaXab+lxozasX6p1nyzRfYPuPqNj+vS5Ves+WVJqX4MG9bU7c50iIyOrcjqhxQpU/lPFav1ZsvKTDTp+/Ljmz3pFIx+8V2nTZgfbjuT6NG3WXM2ZPllzZkzW0vRVOvCPHG3e9oW279ylea+9pDnTJ+vgocM2zqBm+l3PmxQZGamU63vrqXGT9MKLTwbbwsPD9eKkser1u/7qdlMf3Tuojy5s1EDde6RIkm5MuUPPPfuyJk4aK0kaP+ERTRg/RTem3KF69eqpxy032DKn812vXt1Ut26kkq/pqSeenKi0yU8F28LDwzUlbbxu7n63rk/prfvv76dGjRpWeMxll/1agwb2lcPhCO676cZrtXzZ22rUqEG1zs12gUDlP1Ws1peVPv8iU0lXXSFJuuyStsrc81WwLSv7oNq0aqEYj1uSdEnbVtqRuVt7v9qvVi2a66HHn1V+QaFGDb3PlrHXZFdf/RtlfLRGkrRly3Z17Ng+2Na6TUt98823ys31S5I2rP9MSZ0T9d57y7R82ceSpCZNG+vwoR8kSf36DlEgEFBERIQa/UdDHcr5oZpnUzMkd07UivRVkqRNm7fpio6XBtvatm2lffv+rtxcnyRp/adblJzcSVd3uuK0x9SvH6cXnntcD48er9f/nBbsJxAIqGu3Ptq8cXl1TSs0sOYQevILCuWOjg5uO8OcKi4uUXh4mJrGX6Sv93+rH346ouioetr42Q41axKvXJ9f2QdzNDNtgrKyczT80af1wduzS/0LCP8et8ctnz8vuF1SUqKwsDCVlJTI43bJ7/ulLT+/QJ4Yd/DnXp81Rb/reZPu6TdU0slfOE2aNNYHS+fJ78/TV199U72TqSHcntL/30tKAqW+k1O/r7z8fMV43Kc9pk6dOpo9a4pGPfK0ioqOlvozMlaurfqJhKIqzAQyMjKUnZ2tqKgo9erVS1OnTtWxY8c0ePBgNWzYsNzjyg0OixYtUk5Ozs8nVhP16tWrSgZuN1d0lAoKi4LbViCg8PAwSVKMx61HRwzWyCefU6OGDdSudYLiYj2KjXGrebN4RUREqHmzeNWJrKOfcn26IC7WplnUPHn+PLldruC20+lUScnJZ9778/Llcv8S0F2uaPl+ziIk6YHBozVuXAOtXrNYv+l4owoLi/T99wd0+aVdNGDgXZr44lg9MHh09U2mhsjz58vlLv87cbt++U7cLpdyff7THnPZpe3UsmVzzZg2UXXr1lXbtq300pQJGjV6fPVNJtScRXDw+/3y+/3Gfo/HI4/HE9zetWuXRowYoRkzZsjhcOjHH39UIBBQvXr1Kuy/3DWHwsJC/fGPf9SwYcPk8/kqPfDzRYf27bR2wxZJ0o6du9UqoXmwrbi4RDsy98g7I00Tx43W/m+z1KF9O3W49Ndat3GrLMvSocM/qqjoqGJ/Lj3h3Ni4Yatu6nqdJOnKKy9XZubeYNvePV8rIeFixcXFKCIiQknJidq0eZv69E3VqNFDJElFhUcVCARUUlKiBe/ODi6E5uflKxCCKfz54NMNW3Rzty6SpE6JHbVz5+5g2+7dX6llyxaKi4tVRESEkn/bSRs3bj3tMVs+267LLu+ilBvv0N33DNHu3V/V7sAgSZZV6Y/X61VKSorx8Xq9pboOCwv7+Y+wlJWVpeTkZPXt21ebNm2qcEjlZg4XXXSRpk+frrCwMCUkJJyD2YemlGs7a/2Wz9XvgYcly9KzTz6spemrVFhUpDt6dVdERITuHDRckXXqaEDf2xQXG6Prkjpp6/ad6nP/Q7IsS2NHDQ1+ATg3lixZoS4pycr4+H/lcDg05IFHdMedPeVyResvb76txx97XouXeOV0OjVv7rv6R3aOlrz/oV57PU0r0hcoPCJcj455VseOHdfLU/6s12al6cTxEyosKtLQPz5m9/TOS4sXL9cNKddo7Zr35XA4dN8fRqpPn1vlio7Wf78xX4+MmaBlS+fL6XRqzpx3lJ198LTH4DTOInMYMGCAUlNTjf2nZg2SlJCQIK/Xq/j4eLlcLm3btk27d+/W3XffXWH/DsuyrEqP6gyc+IG6biiLa5pi9xBQgaPFx+0eAipQfPzAOe2vaP64Sh9Tr9+z53QMZdX6BWkAsF0IljoJDgBgN+6QBgCcD8gcAMBuVbP0+28hOACA3UKwrERwAAC7ERwAAAauVgIAlGUFWHMAAJRFWQkAYKCsBAAwUFYCABgoKwEADAQHAICBO6QBAAYyBwCAgQVpAICBS1kBAAYyBwBAWVYIrjnwsh8AgIHMAQDsRlkJAGBgQRoAYCBzAAAYQnBBmuAAAHYjcwAAGFhzAAAYyBwAAGWF4k1wBAcAsBuZAwDAQHAAABhYkAYAGMgcAABlWQQHAICB4AAAMHApKwDAQOYAADCEYHDgTXAAAAOZAwDYzLJCL3MgOACA3UKwrERwAAC71abg4GlyfVV1jXPA/22G3UNABVxNu9g9BFQjboIDAJgIDgAAQ+jdA0dwAAC7UVYCAJgIDgAAA2UlAEBZlJUAACYyBwBAWWQOAAATmQMAoCyL4AAAMJxFcPD7/fL7/cZ+j8cjj8cT3M7IyFB2draioqLUu3dvTZ06VWFhYerXr5/q169fbv+8zwEAbGYFKv/xer1KSUkxPl6vt1Tfu3btUv/+/ZWTk6O9e/fK5/NJkurWrVvhmMgcAOA8NGDAAKWmphr7T80aJCksLEzSyXdGnDhxQq1bt1a7du20du1ade3atdz+CQ4AYLezKCuVLR+VJyEhQV6vV/Hx8WrdurWWL1+urKws/f73v6/wOIIDANisKheku3XrVmp7zJgxZ3QcwQEAbMbVSgAAA8EBAGCyHHaPwEBwAACbkTkAAAxWgMwBAFAGmQMAwGCx5gAAKIvMAQBgYM0BAGCwQu9dPwQHALAbmQMAwEBwAAAYKCsBAAyhmDnwJjgAgIHMAQBsxk1wAAADN8EBAAwBMgcAQFmUlQAAhlC8WongAAA24z4HAICBzAEAYGBBGgBgYEEaAGBgzQEAYAjFslKtfbaSw+HQq68+r9Wr39OKFe+oRYtmpdq7d0/RunVLtHr1e7r33j6SpPDwcL3xxivKyHhXa9e+rx49bpAkNWx4gRYunK2PPlqojz9epObNm1b7fGqyQCCgCVOmq9+DozRw+GP6Liu7VPuSDz9W6oBh6j90jBb9LT24f/a8her34Cjded9Dpfbj3+dwODR92gtas3qx0tMXKqHFxaXae3S/QZ+u+5vWrF6sQYP6lmq78srLlZ6+MLg9b+4MpacvVHr6Qu3du17z5s6ojimEFMtyVPpT1Wpt5tCzZ1fVrRup665LVWJiB7344ljdeecfJJ0MApMnP6Xk5N+poKBIq1Yt0rJlK3XTTdfpp5+O6L77Rqp+/Vht3LhMS5dm6PnnH9eCBYu1aNFSXXPN1WrduqX27//O5hnWHCvXbtTxYyc0/7WXtCNzj9JmvKFpE8dJko7k+jTtv+fpf998VW5XtO4fOVZXXXGZDhzM0faduzVvZpqOHj2mv7zzV5tnUbP07NlVkXXr6trrblViYgdNmjROt99xn6ST509a2nh1TrpFBQWFWr3qPS1dmqGcnMMa9fCDuvvu3iooKAz29fv+QyVJsbExSl+xQI+MmWDLnOwUimWlWps5dO58pT76aI0kafPmz3XFFZcG29q0aal9+/6u3Fy/Tpw4ofXrtygp6Ur99a9LNWHCS8GfKy4ukSRdffVv1Ljxr7R06Xz16XOrPvlkQ/VOpob7/ItMJXXqKEm67NdtlLnnq2BbVvZBtWnVQjEet5xOpy5p00o7Mvfo003b1KrFxXroyec19LFndG3nRLuGXyMldU5UevpqSSfPn44dT3f++E45f07+/9/3zbe6664/nLbPp8Y9rJl//osOHjxU5eMPNQHLUelPVau1wcHtdsnnywtul5SUKCwsTJLk8bjk9//SlpdXII/Ho4KCQuXnF8jlitZbb72mCROmSJKaNYvXkSM+9ejRT99/f0CjRg2p3snUcPkFRXK7ooPbTmdYMDA3bdJYX+//Tj/8dERFR49q49YdKjp6TLk+vzL3fq2Xn3lMT40eqseemSIrFP95dp5ye1zy+/zB7ZLAqeePu9S5lZefr5gYtyRp8eLlOlFcbPTXsOEFuv76JM2d+24Vjzw0nXdlpezs0rXdiy66qEoHU53y8vLldp/6C8epkpKTv3D8/ny5XK5gm9sdLd/PJ0J8/K/0zjuzNGvWPC1Y8L4k6ccfj2jp0o8kScuWrdTTTz9SXdOoFVzR9VRQWBTctqyAwsNP/iKKcbv06PD7NXLsRDVqeIHa/WeC4mI8io3xqHmzeEVERKh503jVqROhn3J9uiAu1qZZ1Cx5/ny53L+cI07HqedPXqlzy+1yKTfXb/RxqttSe+idBe8rEAjBx5NWg/NuQXrq1KlKT0/Xpk2btHnz5uoaU7XYsOEzde16vSQpMbGDdu7cG2zbs+drtWx5seLiYhQREaGkpE7atGmrLrywgT744H80duyLmjt34Wn7Sk5O1O7dX1bvZGq4Du3bae2GzyRJOzL3qNUpi5/FxSXakblX3ukvauLYh7X/uyx1aN9WHdq307pN22RZlg798KOKjh5TrMdt0wxqnvUbtqhbty6STp4/mZl7gm0nz5/miouLVUREhJKTE7Vp07YK++vSJVkrVqyq0jGjchxWBbn2sWPHdOjQITVp0qTSHder1+xf/5CNHA6Hpk59Tu3bt5XD4dDgwaPVocMlio6O0ptvvq3u3VP0xBMPyeFwau7chXr99bmaMmW8eve+RV9+uS/YT69eA3ThhQ00c+YkRUdHyefL08CBw//lv5Ts5v82w+4hnLFAIKBnX56pL/f9XbIsPfv4f2n3l/tUWHRUd/Tsppl/eUsfr92oyDoRGnBXqm66PlmS9NLMN7X58/+TFQjoocH9ldTpCnsnUgmupl3sHkKFHA6Hpr36vC655J/nzyhd3qG9XK4ovfHGW+rR/QY98cR/yel0yOtdqNde9waPbdYsXvPmztA11/YK7vt8W4auu/62YIYe6o4d/f6c9rfxotsqfcxV2VV7kUWFweHfEerBobY7n4JDbRTqwaG2O9fBYf2velf6mM7/WHROx1BWrb2UFQBCBY/PAAAYQnEZnuAAADazROYAACgjEIK34BAcAMBmATIHAEBZlJUAAAYWpAEABjIHAICBzAEAYCA4AAAMlJUAAIZA6MUGggMA2I37HAAAhhC8Qbr2viYUAFA+MgcAsBlXKwEADAEHaw4AgDJCcc2B4AAANjubspLf75ffb75z2+PxyOPxBLczMjKUnZ2tqKgo3X777Tpw4ID+9Kc/KS0trcL+CQ4AYLOzuc/B6/Vq+vTpxv5hw4Zp+PDhwe1du3ZpxIgRmjFjho4fP65Vq1apadOm/7J/ggMA2Oxs7nMYMGCAUlNTjf2nZg2SFBYWJkmyLEsbN25UQUGBtm/frgMHDqhx48bl9k9wAACbnc2aQ9nyUXkSEhLk9XoVHx+vq666Stdcc42OHz9eYWCQCA4AYLuqfHxGt27djH2nlp3KQ3AAAJtxnwMAwMClrAAAA09lBQAYKCsBAAwEBwCAwaKsBAAoi8wBAGAgOAAADKF4KStvggMAGMgcAMBm3OcAADCw5gAAMBAcAACGUFyQJjgAgM1YcwAAGCgrAQAMlJUAAIZACIaHKgsOASsUEyX8k6tpF7uHgArkZ62xewioRqH425LMAQBsFnp5A8EBAGxH5gAAMHApKwDAUKsWpAEAZyb0QgPBAQBsx5oDAMAQimUlXvYDADCQOQCAzUIvbyA4AIDtWHMAABhCcc2B4AAANgu90EBwAADbUVYCABisEMwdCA4AYDMyBwCAgQVpAIAh9EIDwQEAbEfmAAAwsOYAADBwtRIAwEDmAAAwkDkAAAxkDgAAQ8AKvcyBl/0AAAxkDgBgs9DLGwgOAGA7boIDABi4WgkAYOBqJQCAgbISAMBAWQkAYKCsBAAwWFV4E1xGRoays7MVFRWllJQUTZ8+XZI0dOhQ1a9fv9zjuAkOAGwWkFXpj9/vV1ZWlvHx+/2l+t61a5f69++vnJwcRUZGauTIkWrevLkKCgoqHBOZAwDY7GzKSl6vN5gFnGrYsGEaPnx4cDssLEzSyewkKipKGRkZiomJUZMmTSrsn+AAADY7mwXpAQMGKDU11djv8XhKbSckJMjr9So+Pl579uzRnDlz1LlzZx04cECNGzcut3+HVUXFrsi6FUclAOXLz1pj9xBQgYgGLc5pf92bdq/0Mcu+W3ZOx1AWmQMA2KwqF6TPFsEBAGzGpawAAEMo3gRXay9ldTgcmj7tBa1ZvVjp6QuV0OLiUu09ut+gT9f9TWtWL9agQX1LtV155eVKT18Y3L788ku0bu0HWrlykV55+Rk5HI7qmEKNxvdz/ggEApoweZr6DR6pgcPG6Lus7FLtSz5cqdT+Q9R/yGgt+mBFcP/suQvUb/BI3TloeKn9tdHZXMpa1WptcOjZs6si69bVtdfdqrFjJ2rSpHHBtvDwcKWljVePW/rphhvv0H2D+qlRo4aSpFEPP6jX/pymupGRwZ+fOWOSRj/ytFJSesvnz1OfPrdW93RqHL6f88fKTzbo+PHjmj/rFY188F6lTZsdbDuS69O0WXM1Z/pkzZkxWUvTV+nAP3K0edsX2r5zl+a99pLmTJ+sg4cO2zgDnE6tDQ5JnROVnr5akrR58+fq2PHSYFubNi21b9/flZvr04kTJ7R+/RYlJSVKkvZ9863uuusPpfpq3Pg/tHHjVknShvVblNT5yuqZRA3G93P++PyLTCVddYUk6bJL2ipzz1fBtqzsg2rTqoViPG45nU5d0raVdmTu1qebtqpVi+Z66PFnNXTM07q2c6Jdww8JlmVV+lPVam1wcHtc8vt+uZOwJFASvFnE43HL58sLtuXl5ysmxi1JWrx4uU4UF5fqa//+7/Tb314lSerR40ZFRUdV9fBrPL6f80d+QaHc0dHBbWeYU8XFJZKkpvEX6ev93+qHn46o6OhRbfxsh4qKjinX51fmni/18nNP6KlHhuuxCZND8oqd6nJelZXmz5+vqVOnyufzacmSJVU+kOqW58+Xy+0KbjsdTpWUnPwL7ffnye3+5S+72+VSbq7f6OOfBg8epTGPDNXi9+bo0OEf9OMPR6pu4LUE38/5wxUdpYLCouC2FQgoPPxkII/xuPXoiMEa+eRzGvfCK2rXOkFxsR7FxriV1OkKRUREqHmzeNWJrKOfcn12TcF21ln8V9XKDQ4FBQV68MEHNXv2bO3bt6/KB1Ld1m/Yom7dukiSEhM7KDNzT7Btz56v1bJlc8XFxSoiIkLJyYnatGlbuX3dfHOKBj8wWremDtQF9eO0cuUnVT7+mo7v5/zRoX07rd2wRZK0Y+dutUpoHmwrLi7Rjsw98s5I08Rxo7X/2yx1aN9OHS79tdZt3CrLsnTo8I8qKjqqWI/brinYLmBZlf5UtXIvZfV4PMrLy9OQIUM0fvz4Kh9IdXv//Q91Q8pvtXrVe3I4HBo8eJTuuutWuVxReuONtzRmzDP62wf/I6fTIa93obKzD5bb19df79f7i70qLCzSmk826MMVq6pxJjUT38/5I+Xazlq/5XP1e+BhybL07JMPa2n6KhUWFemOXt0VERGhOwcNV2SdOhrQ9zbFxcbouqRO2rp9p/rc/5Asy9LYUUODZcPaKBQLajw+AwhBPD4jtJ3rx2ckNe5S6WM+PfDxOR1DWdwEBwA24zWhAABDKF6pRXAAAJuROQAADKH4bCWCAwDYjLISAMBAWQkAYCBzAAAYyBwAAAYWpAEAhup4VlJl1dpHdgMAykfmAAA2o6wEADCEYlmJ4AAANiNzAAAYyBwAAAYyBwCAgcwBAGAgcwAAGCwrYPcQDAQHALAZz1YCABh4KisAwEDmAAAwkDkAAAxcygoAMHApKwDAQFkJAGBgQRoAYAjFzIE3wQEADGQOAGAzrlYCABhCsaxEcAAAm7EgDQAwkDkAAAysOQAADNwhDQAwkDkAAAysOQAADJSVAAAGMgcAgIHgAAAwhF5okBxWKIYsAICteCorAMBAcAAAGAgOAAADwQEAYCA4AAAMBAcAgIHgAAAwEBwAAAaCAwDAwOMzzkBGRoays7MVFRWl22+/3e7hoIyjR4/qqaee0ogRIxQfH2/3cFDG22+/LZ/PJ5fLpXvuucfu4eAMkTmcgV27dql///7Kycmxeyg4jcOHD6tNmzZ2DwPlSElJ0cCBA3X48GG7h4JKIDicgbCwMEmh+eRESE2aNFFsbKzdw0A5oqKiNG3aNA0cONDuoaASCA5nICEhQV6vl5IFcBbGjx8vp9OpNWvW2D0UVAJPZQUAGMgcAAAGggMAwEBwAAAYCA4AAAPBAQBgIDgAAAwEBwCA4f8BWtsZgMaV524AAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 504x360 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "cm = confusion_matrix(y_test, pred_svm) \n",
    "cm = cm.astype('float') / cm.sum(axis=1)[:, np.newaxis]\n",
    "print(cm)\n",
    "\n",
    "df_cm = pd.DataFrame(cm, range(3), range(3))\n",
    "plt.figure(figsize=(7,5))\n",
    "sn.set(font_scale=0.5) # for label size\n",
    "sn.heatmap(df_cm, annot=True, annot_kws={\"size\": 10}) # font size\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "              precision    recall  f1-score   support\n",
      "\n",
      "     class 0       0.94      0.96      0.95       246\n",
      "     class 1       0.93      0.96      0.94       231\n",
      "     class 2       0.99      0.96      0.98       479\n",
      "\n",
      "    accuracy                           0.96       956\n",
      "   macro avg       0.95      0.96      0.96       956\n",
      "weighted avg       0.96      0.96      0.96       956\n",
      "\n",
      "\n",
      "F1-score is : 0.956633102733924\n",
      "Cross validation score is : 95.91928251121077\n"
     ]
    }
   ],
   "source": [
    "target_names = ['class 0','class 1','class 2']\n",
    "print(classification_report(y_test, pred_svm, target_names=target_names))\n",
    "print()\n",
    "\n",
    "f1_svm = f1_score(y_test, pred_svm, average='macro')\n",
    "cross_val_svm = np.mean(cross_val_score(svm_model, X_train, y_train, cv=10))*100\n",
    "print('F1-score is :', f1_svm)\n",
    "print('Cross validation score is :', cross_val_svm)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## __Linear SVMClassifier()__"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "SVM Linear Accuracy: 93.51464435146444%\n"
     ]
    }
   ],
   "source": [
    "#SVM Linear\n",
    "svm_model_linear = SVC(kernel = 'linear', C = 1).fit(X_train, y_train) \n",
    "\n",
    "pred_svm_lin = svm_model_linear.predict(X_test) \n",
    "  \n",
    "# model accuracy for X_test   \n",
    "accuracy_svm_lin = svm_model_linear.score(X_test, y_test) * 100\n",
    "print(\"SVM Linear Accuracy: {}%\".format(accuracy_svm_lin))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## __Confusion Matrix and Classification Report of Linear SVMClassifier()__"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[0.94715447 0.02845528 0.02439024]\n",
      " [0.05194805 0.90909091 0.03896104]\n",
      " [0.02713987 0.03131524 0.94154489]]\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYcAAAEsCAYAAAAoxX9TAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8rg+JYAAAACXBIWXMAAAsTAAALEwEAmpwYAAAbgElEQVR4nO3deXxTVd7H8W/SlqVNU9phUSigFGRxY3AEhcIzWh2KoIh1QVGKDMOiLLI6AoKIiFLBQcqMAiJhRBgFRQTZimyy44KPFFQQVKgsIjR0pU3y/FGM0DNF2of2xvbz9pU/bm5zOed1TL/9nXNuYvP5fD4BAHAOu9UNAAAEHsIBAGAgHAAABsIBAGAgHAAABsIBAGAILq0L5/30bWldGpdAaO22VjcBF8D+8sCWf+bwJb1eSX5fhlRvcEnbUFiphQMA4CJ5PVa3wEA4AIDVfF6rW2AgHADAal7CAQBQiI/KAQBgoHIAABioHAAABnYrAQAMVA4AAANrDgCAwtitBAAwUTkAAAxUDgAAA7uVAAAGKgcAgIE1BwCAIQArB74JDgBgoHIAAKsxrQQAKMznY7cSAKCwAFxzIBwAwGpMKwEADFQOAAADd0gDAAxUDgAAA2sOAAADlQMAwEDlAAAwEA4AgMK4QxoAYKJyAAAYWJAGABioHAAAhgCsHPiyHwCAgcoBAKzGtBIAwBCA00qEAwBYjcoBAGAgHAAABqaVAACGAKwcKvxWVq/Xq3GTpqlb78Hq0X+Evj+Udt75JSvWqEv3fureb5gWfbDS//y9PR5Xj/4j1KP/CI2eMKWsm13u2Ww2TU9+QRs3LFHK6ncUE3PFeec7drxdWzYv08YNS/TXng9JkoKDgzXnjVe09qN3tXnTUnXqdLsk6frrr9bHGz/QurXvaeaMybLZbGXdnXLnl/H5eMMSrfkv49Pp7Ph8/F/GZ91H72rLOePzi65d79bHG5aUVRcCi89b/Ecpq/CVw5oNW3TmzBnNm/Gydn25R0nTZmrai2MlSSdPpWvajLlaOCdZ4Y4w9Ro0Ujf9qbmqR0VKkuYkT7Ky6eVa587xqlKlstq2u0utWrbQpEljlJDQU1LBL5mXksbq5tYdlZmZpQ3rF2vpstWKb3+LTpw4qR6PDlRUVKR2bF+ppUtXa/TowXpuwstaseIjzXVN0x133KZly1Zb3MPft1/GJ/bs+CRNGqN7Co3PTb8xPjvPjo9UEOA9ezxYcYO7BJWD2+2W2+02nnc6nXI6nf7jlJQUpaWlKTQ0VJ07d9bUqVOVm5ur3r17q0aNGkVev8KHw2df7Fabm26QJF1/TVPt3vuN/9yhtCNq0qiBIpzhkqRrmjbSrt17FH35ZcrJydXfnhgpj8erQX0Sdf01TS1pf3nVpnVLrVy1VpK0bfunuqHFdf5zTZs20v79B3XqVLokadOmHYqNbaWFi5Zq0bvL/D+Xn58vSfr88y8VFVVNkuQIdygvL6+MelF+xRZjfDb/xvhERUXq+eee0pBhY/Xav5LKsBcBpASVgMvlUnJysvF8//79NWDAAP9xamqqBg4cqOnTp8tms+nEiRPyer2qWrXqBa9f4cMhIzNL4WFh/mN7kF35+R4FBwepXnRt7TvwnX76+aTCQqtq685dql83WlWqVFaPhxKUcGe8vvvhsPoOfVpL589ScHCQhT0pX5xOh9LTT/uPPR6vgoKC5PF45Ax3KN3967nTGRmKcIYrMzNLkuRwhOk/C2Zo7DMFld2+fQf0ytQJeuqpQXKnu7V+/Zay7Uw5FO50yP3/GJ+3F8zQmGcmyW63a+aMlzR0+DPKzs4p834EjBJUDomJierSpYvx/LlVgyQFBRX8XvL5fDp06JBiY2NVp04dbdu2TXFxcUVev8hwWLRokY4ePSqv16u6deuqc+fOxW7874EjLFSZWdn+Y5/X6/8lH+EM15MDe2vwqOdUq0Z1NWsco8hqTl1Rt47qRdeWzWbTFfWiVS3CqeMnftbltYou0VA8bneGwsMd/mO73S6Pp+Az792nMxTu+DXQwx0OnUovKK+jo2tr4Tuz9OqrLi1YsFiSNGXys7rl1nuUmvq1+vVNVNKkMRo4aFTZdaYcOu3OkOMSjM+Nf2quhg2v1PRpE1WlShU1bdpIk18ap6HDxpZth6xWgnAoPH1UlJiYGLlcLkVHR8vhcOjTTz/Vnj179NBDD13wdUWGQ1ZWlh577DFJ0ty5c4vZ7N+PP17bTOs2bVN8XDvt+nKPGsVc6T+Xn+/Rrt175ZqeJI/Ho16DRmpQnx56d9kqfbP/oJ4e1l/Hjp9QZmaWavwhysJelD+bt+xQp463a+HCD9SqZQt9+eUe/7k9e75Rw4YNFBlZTRkZmWrbtpWmvPyqatasrg8/fEuDBo3W2rUf+3/+55On5HZnSJLSfjyqm1vfWOb9KW82FWN8Ytu20uSz47P87Ph8dHZ8duz8XNc3v1WSVL9+tN56818VLxgkyecrtUvHx8efd/z0009f1OuKDIfatWsrOTlZQUFBiomJ+f+1LoDF/U9rbd7xmbr1GSL5fBo/aoiWrVqrrOxs3df5DoWEhOj+ngNUuVIlJT54jyKrRSihU3uNmjBFj/QbKptsenbkYKaULrHFi5frtrh22rD+fdlsNvX622B17Xq3HGFhmvX6PA0fMU4fLpsnu92uOXMWKC3tiKZMHqfIahEaNXKQRo0cJEnqdOcj6ttnmOa9+U/l5+frzJk89e033OLe/f79Mj4bz47PX0s4Ph3vfEQ5ORV4OukXAbiV1ebzlU5k5f30bWlcFpdIaO22VjcBF1B6f0fiUsg/c/iSXi973sX9NX+uqt3GX9I2FFbhF6QBwHLcIQ0AMATgtFKFv0MaAGCicgAAq5XibqWSIhwAwGoBOK1EOACA1QgHAICB3UoAgMJ8XtYcAACFMa0EADAwrQQAMDCtBAAwMK0EADAQDgAAA3dIAwAMVA4AAAML0gAAA1tZAQAGKgcAQGG+AFxz4Mt+AAAGKgcAsBrTSgAAAwvSAAADlQMAwBCAC9KEAwBYjcoBAGBgzQEAYKByAAAUFog3wREOAGA1KgcAgIFwAAAYWJAGABioHAAAhfkIBwCAgXAAABjYygoAMFA5AAAMARgOfBMcAMBA5QAAFvP5Aq9yIBwAwGoBOK1EOACA1SpSOETWiyutS+MSyNizyOom4AIirr7f6iagDHETHADARDgAAAyBdw8c4QAAVmNaCQBgIhwAAAamlQAAhTGtBAAwUTkAAAqjcgAAmKgcAACF+UoxHFJSUpSWlqbQ0FAlJCRo6tSpCgoKUrdu3RQVFVXk6wgHALBaCcLB7XbL7XYbzzudTjmdTv9xamqqBg4cqOnTp+urr75Senq6oqKiVKVKlQten3AAAIuVpHJwuVxKTk42nu/fv78GDBjgPw4KCir4N3w+5eXlqXHjxmrWrJk2btyo9u3bF3l9wgEAfocSExPVpUsX4/lzqwZJiomJkcvlUnR0tBo3bqzly5fr0KFDeuSRRy54fcIBAKxWgsqh8PRRUeLj4887HjFixEVdn3AAAIuV5oJ0SREOAGAxwgEAYCAcAAAmn83qFhgIBwCwGJUDAMDg81I5AAAKoXIAABh8rDkAAAqjcgAAGFhzAAAYfIH3XT+EAwBYjcoBAGAgHAAABqaVAACGQKwc7FY3AAAQeKgcAMBi3AQHADBwExwAwOClcgAAFMa0EgDAEIi7lQgHALAY9zkAAAxUDgAAAwvSAAADC9IAAANrDgAAQyBOK1XYz1ay2Wya+spzWrN2kZavmK8GDeqfd77DHXFav3Gx1qxdpB6PdvU/v2nLUi1fMV/LV8zXv16bJEm69rqmWrX6bS1fMV+L33epZs3qZdqX8s7r9Wr8tDl6eMiz6vnkRH2fdvS88x+s2aSEx0YpcfgEvbty/Xnnvti7Xz2fnFiWza0QbDabpk17XuvWvadVq/5jvH/uuOM2ffzxB1q37j317PmgJMlut+u115K0du27Skl5x/+a5s2v0caNS7RmzUJNmTJONlvg/aIsbT6frdiP0lZhw+HOu/6iypUrK+6WBI15+kU9/8Io/7ng4GC98OJodb6zu+L/0lWP9uyqmrWqq3LlSpKkDvEPqkP8g+rXZ4QkKSlprIYOHasO8Q9qyZIVGjykjyV9Kq8+2vKpcvPy9OaUMRr06H16adZ8/7mT6aeVPHeRZr/4lN548SktW7tFh48elyTNfmeZnpk6W7ln8qxqerl1113tVaVKZf35z100evQLevHFp/3ngoODlZQ0Rp06Pazbb79fPXs+pFq1aqhjx9skSbfcco+efXay/zXTp7+g4cPHKS7uXrndp9W1691WdMlSPl/xH6WtwobDzTf/SSmrC/7K3LHjc7Voca3/XOMmDfXtt9/p1Cm38vLytGXzTrVp3VLXXtdMVatW1ftL5mrZh/N0443NJUmJiQP0v1/skVTwxsjJzS3z/pRnn+3+Wm1uKBif65s0VOo3B/znDh05psYx9RQR7pDdbtc1V12pL/bulyTVvbymXh49wJI2l3etW9+oVavWSZK2b/9MLVpc5z/XpElD7d9/UKdOpSsvL0+bN+9QmzYt9cEHq/TYY3+XJNWrF61jx36SJNWpc5m2bv1EkrR58061bn1j2XYmAHh9tmI/SluFDYdwZ7jS3af9xx6PR0FBQZIkZ7hD7vRfz2VkZMoZEa7srGy9MnWmOt/VXYMGjtbrb/xDQUFBOnqk4C/VVq1aqE+f7po+bXbZdqacy8jKliO0qv/Ybrcr3+ORJNWvfZn2f3dYJ06mKzsnV9s+T1V2TkE43x57o4KDgyxpc3nndDqUfs57xOs95/3jDD/vXEZGhiIiwiUVvM9mzZqiKVPG6b33lkmSDhz4Xm3btpIkdex4m8LCQsuqGwEjEKeVLrggnZaWdt5x7dq1S7UxZem0+7TCHQ7/sd1ul+fsLxz36Qw5wsP85xyOMKWfcuubbw5o//6DkqR9+w7o559P6bLLaurw4R+VkNBRw0f0V8I9PfXTTz+XaV/KO0doVWVl5/iPvV6fgv1BHqbhvR/SkAnTVKt6lJo2rK9IZ7hVTa0w3O4MhYf/+v6x2c55/7hPK/y8949Dp065/ce9eg1RrVo1tHHj+2rePE69ew/T5MnPaMiQvvrkky+Um3um7DoSIH53C9JTp07VqlWrtG3bNm3fvr2s2lQmtm75RH9p/2dJ0o03Ntfu3V/5z321d59iYq5QZGSEQkJC1Ca2pbZt/1TdE+/TxLNrE5ddXlPOcIeOHDmmB7rerT59E9UhvqsOHvzBiu6Ua82bNdLGnV9Iknbt3adGV0T7z+V7PPpi7369MWmkJgztrQM//KjmzRpZ1dQKY8uWnYqPv0WS1LLlH7V7917/ub1796lhwyv975/Y2Fbatu0TPfTQPRo+/HFJUlZWtrxerzwerzp0uFV9+gxXly6PKiqqmtas2WhJn3C+C1YOzz77rI4dO6a6deuWVXvKzJIlK3VrXKxSPloom82mfn2G677775LDEaY3Zs/XU3+foMVLXLLb7fr33Hf0Y9pRuea8rddmJGlVytuST+rXd4R8Pp+SXhqrQ4fS9Nb8VyVJH3+8TROe+4e1HSxH4lrfoK2f7dYjQ8fL5/Np/OBeWrZ2i7JzcnRvh1sUEhykrgPHqlKlEHW/p4MiI6gcStv7769QXFxbrV37rmw2m3r3HqYHHugshyNMr7/+lkaMGK8PPnhTdrtdLtd/lJZ2VIsXL9eMGZOVkvKOgoODNWzYOOXm5mrfvoNavHiOsrKytWHDFq1cudbq7pW5ALzNQTafr3TWvR2hV5bGZXGJnPjf+b/9Q7BMxNX3W90EXEBOzveX9HqbL08o9mta/7jokrahMG6CAwCL8fEZAABDAH5LKOEAAFbzicoBAFCINwBXpAkHALCYl8oBAFAY00oAAAML0gAAA5UDAMBA5QAAMBAOAAAD00oAAIM38LKBcAAAq3GfAwDAEIA3SFfcrwkFABSNygEALMZuJQCAwWtjzQEAUEggrjkQDgBgsUCcVmJBGgAs5rUV/3GxUlJSNHfuXC1cuFCSdPjwYQ0fPvw3X0flAAAWK8l9Dm63W26323je6XTK6XT6j1NTUzVw4EBNnz5dZ86c0dq1a1WvXr3fvD7hAAAWK8mag8vlUnJysvF8//79NWDAAP9xUFBQwb/h82nr1q3KzMzU559/rsOHD6tOnTpFXp9wAACLleTjMxITE9WlSxfj+XOrBkmKiYmRy+VSdHS0brrpJrVr105nzpy5YDBIhAMAWK4kC9KFp4+KEh8fbzx3bmVRFMIBACzGVlYAgIFPZQUAGALxPgfCAQAsRjgAAAw+ppUAAIVROQAADIQDAMAQiFtZ+eA9AICBygEALMZ9DgAAA2sOAAAD4QAAMATigjThAAAWY80BAGBgWgkAYGBaCQBg8AZgPJRaOOTknymtS+MScDRNsLoJuICsQ+usbgLKENNKAABD4NUNhAMAWI7KAQBgYCsrAMBQoRakAQAXJ/CigXAAAMux5gAAMATitBJf9gMAMFA5AIDFAq9uIBwAwHKsOQAADIG45kA4AIDFAi8aCAcAsBzTSgAAgy8AawfCAQAsRuUAADCwIA0AMAReNBAOAGA5KgcAgIE1BwCAgd1KAAADlQMAwEDlAAAwUDkAAAxeX+BVDnzZDwDAQOUAABYLvLqBcAAAy3ETHADAwG4lAICB3UoAAAPTSgAAA9NKAAAD00oAAIMvAG+CIxwAwGKsOQAADEwrAQAMLEgDAAylOa2UkpKitLQ0hYaGKi4uTsnJyZKkxx9/XFFRUUW+jg/eAwCL+Xy+Yj/cbrcOHTpkPNxu93nXTk1NVffu3XX06FFVrlxZgwcP1pVXXqnMzMwLtonKAQAsVpI1B5fL5a8CztW/f38NGDDAfxwUFCSpIIBCQ0OVkpKiiIgI1a1b94LXJxwAwGIlWXNITExUly5djOedTud5xzExMXK5XIqOjtbevXs1Z84ctW7dWocPH1adOnWKvL7NV0obbIMrFf2PBgKbzabkaRN1/XXNlJubq959h2v//oP+85063q5Ro56QJ9+jN+Ys0Ouz31JwcLBmzZyiK+pHq3LlSpowcaqWLl2teW/+U5fVqiFJql+/rrZt/1TdHn7Mop5dHJvVDfgNv4zPdWfHp0+h8enY8XaNHvWE8vM9mnN2fOx2u157NUlXXRUjj8ejXn8bom+//c7/mpeSntHXX+/XjJn/tqBHxZN1aJ3VTbhoXq9X46f8U1/vO6CQkBA9++RA1Yuu7T+/ZMVHemP+uwp3hKpzh9uU0Okv/nMnTp7S/b2e0Mwp49Wg/oX/kg0kITUbXdLr3Va3fbFfk/LDykvahsIqbOXQuXO8qlSprNh2d6lVyxZKmjRG9yT0lCQFBwfrpaSxuql1R2VmZmnD+sVaumy14tvfohMnTqrHowMVFRWpndtXaunS1f4gqFYtQimr39HQYc9Y2LPy4ZfxaXt2fCZNGqOEQuNzc6HxadWqhSTpf/58t9q1u1lJSWOVkNBT1atH6Y3ZU9WoUQNNmbLfym6VS2s2btWZ3DzNe3Wydu3eq6Tpr2vaxKclSSdPpWvarH9r4exXFO4IU6/Bo3XTDderzuW1lJefr3FJyapSqZLFPcB/U2HDIbZ1S61ctVaStG37p7qhxXX+c02bNtL+/Qd16lS6JGnzph2KjW2lhYuWatG7y/w/l5+ff941x44ZqunTZ+vIkWNl0IPyrU0xxmfT2fFZtGipli1LkSTVrx+tY0ePS5IcjjCNHz9F7eNvKeNeVAyffbFbbc4G8/VXN9Huvd/4zx1KO6ImjRoowhkuSbqmSSPt2r1XdS6vpZemv64HOnfQzDffsaTdgSQQ75CusLuVwp0OudNP+489Hq9/4cYZ7lC6+9dzpzMyFOEMV2ZmljIyMuVwhOntBTM05plJ/p+pUeMPuvXWWLnmvl12nSjHnE6H0os5PgU/59Hs1/+hf7w83h/kBw/+oO07PivD1lcsGZnZCneE+Y/t9iDl53skSfXq1tG+A9/rp59PKjsnR1s/2aXsnFwt/jBFUdUi1KbVDVY1O6B45Sv2o7QVGQ7z5s3T1KlTlZ6eriVLlpR6Q8raaXeGHOEO/7HdbpfHU/A/tPt0xnn/s4c7HDqVXrA9LDq6tlJWv6M35y3UggWL/T+TcE8nLViwWF5vIN7r+PvjdmcovATjI0k9//qEml3dVq/+K0mhoVXLrtEVlCOsqjKzsv3HPp9XwcEFQR4R7tCTA3pp8OiJenriVDW7KkaREU699+FqbdnxuXoM+Lu+2ndAIydM0U8nTlrVBcv5SvBfaSsyHDIzM9W3b1/NnDlT+/eXv3naTVt2qEP8rZKkVi1b6Msv9/jP7dnzjRo2bKDIyGoKCQlRbNtW2rr1E9WsWV3LP3xLI0c+rzmu/5x3vbi4WK1Y8VGZ9qE821yM8Wl7dny6dUvQiBH9JUlZWdnyer3yeAjr0vbHa5tp45adkqRdu/eqUYMr/Ofy8z3atfsruZJf0MTRQ3Tg+0P647VN5Up+UXOSX9CcaS+occMr9fyoIar+h0iLemA9r89X7EdpK3LNwel06vTp0+rXr5/Gjh1b6g0pa4sXL9dtce20cf37stls+uvfBqtr17vlCAvTrNfnafiIcfpw2TzZ7XbNmbNAaWlHNGXyOEVWi9CokYM0auQgSVLHOx9RTk6OrroqRt8e+N7iXpUfv4zPhrPj0+sixue99z7UrFkv66M1ixQSEqKhw8YqNzfX6q6Ue3HtbtbmnZ+pW79hks+n8U89oWWr1ykrO0f33RWvkJBg3d/rCVWuFKLEB7ooslqE1U0OOIG34lCBt7JWdIG+lbWi+z1tZa2ILvVW1jZ1bi32azYdLt2Zigq7WwkAAgUf2Q0AMATiVlbCAQAsRuUAADDwfQ4AAAPTSgAAA9NKAAADlQMAwEDlAAAwsCANADCUxWclFVeF/chuAEDRqBwAwGJMKwEADIE4rUQ4AIDFqBwAAAYqBwCAgcoBAGCgcgAAGKgcAAAGn89rdRMMhAMAWIzPVgIAGPhUVgCAgcoBAGCgcgAAGNjKCgAwsJUVAGBgWgkAYGBBGgBgCMTKgW+CAwAYqBwAwGLsVgIAGAJxWolwAACLsSANADBQOQAADKw5AAAM3CENADBQOQAADKw5AAAMTCsBAAxUDgAAA+EAADAEXjRINl8gRhYAwFJ8KisAwEA4AAAMhAMAwEA4AAAMhAMAwEA4AAAMhAMAwEA4AAAMhAMAwMDHZ1yElJQUpaWlKTQ0VPfee6/VzUEhOTk5GjNmjAYOHKjo6Girm4NC5s+fr/T0dDkcDj388MNWNwcXicrhIqSmpqp79+46evSo1U3Bf3H8+HE1adLE6magCHFxcerRo4eOHz9udVNQDITDRQgKCpIUmJ+cCKlu3bqqVq2a1c1AEUJDQzVt2jT16NHD6qagGAiHixATEyOXy8WUBVACY8eOld1u1/r1661uCoqBT2UFABioHAAABsIBAGAgHAAABsIBAGAgHAAABsIBAGAgHAAAhv8DONmuobl9890AAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 504x360 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "cm = confusion_matrix(y_test, pred_svm_lin) \n",
    "cm = cm.astype('float') / cm.sum(axis=1)[:, np.newaxis]\n",
    "print(cm)\n",
    "\n",
    "df_cm = pd.DataFrame(cm, range(3), range(3))\n",
    "plt.figure(figsize=(7,5))\n",
    "sn.set(font_scale=0.5) # for label size\n",
    "sn.heatmap(df_cm, annot=True, annot_kws={\"size\": 10}) # font size\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "              precision    recall  f1-score   support\n",
      "\n",
      "     class 0       0.90      0.95      0.92       246\n",
      "     class 1       0.91      0.91      0.91       231\n",
      "     class 2       0.97      0.94      0.95       479\n",
      "\n",
      "    accuracy                           0.94       956\n",
      "   macro avg       0.93      0.93      0.93       956\n",
      "weighted avg       0.94      0.94      0.94       956\n",
      "\n",
      "\n",
      "F1-score is : 0.9287426529687149\n",
      "Cross validation score is : 90.98654708520179\n"
     ]
    }
   ],
   "source": [
    "target_names = ['class 0','class 1','class 2']\n",
    "print(classification_report(y_test, pred_svm_lin, target_names=target_names))\n",
    "print()\n",
    "\n",
    "f1_svm_linear = f1_score(y_test, pred_svm_lin, average='macro')\n",
    "cross_val_svm_linear = np.mean(cross_val_score(svm_model_linear, X_train, y_train, cv=10))*100\n",
    "print('F1-score is :', f1_svm_linear)\n",
    "print('Cross validation score is :', cross_val_svm_linear)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "# __Poly SVMClassifier()__"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "SVM Poly Accuracy: 94.14225941422593%\n"
     ]
    }
   ],
   "source": [
    "svm_model_poly = SVC(kernel='poly').fit(X_train, y_train) \n",
    "\n",
    "pred_svm_poly = svm_model_poly.predict(X_test) \n",
    "  \n",
    "accuracy_svm_poly = svm_model_poly.score(X_test, y_test) * 100\n",
    "print(\"SVM Poly Accuracy: {}%\".format(accuracy_svm_poly))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## __Confusion Matrix and Classification Report of Poly SVMClassifier()__"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[0.91869919 0.03658537 0.04471545]\n",
      " [0.02164502 0.94372294 0.03463203]\n",
      " [0.01878914 0.02922756 0.9519833 ]]\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYcAAAEsCAYAAAAoxX9TAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8rg+JYAAAACXBIWXMAAAsTAAALEwEAmpwYAAAdKklEQVR4nO3deXxU1d3H8e9MwmIyMwEqUiFgJaiAoARroCRSIVgiKMjmAhqoIo8goCBSDQLiBhIF2YoiKkOLShFkFYlhlx2x9JGtZVEbUsKiZLJBlpnnj9iB5DQ8hCa5Y/J585o/7py5h3M4TH75nXPPvTafz+cTAAAXsVvdAABA4CE4AAAMBAcAgIHgAAAwEBwAAAaCAwDAEFxeFece3VleVaMMuJr3troJuIRge5DVTcAlZGYfK9P68k4fLfU51a5uXKZtKK7cggMA4DJ5C6xugYHgAABW83mtboGB4AAAVvMSHAAAxfjIHAAABjIHAICBzAEAYOBqJQCAgcwBAGBgzQEAUBxXKwEATGQOAAADmQMAwMDVSgAAA5kDAMDAmgMAwBCAmQNPggMAGMgcAMBqTCsBAIrz+bhaCQBQXACuORAcAMBqTCsBAAxkDgAAAzukAQAGMgcAgIE1BwCAgcwBAGAgcwAAGAgOAIDi2CENADCROQAADCxIAwAMZA4AAEMAZg487AcAYCBzAACrMa0EADAE4LQSwQEArEbmAAAwEBwAAAamlQAABjKHwOP1evXKLLcOHf1e1asFa8LTA9Wofj1/+Yq1X+qDTz6TM/Qqdb/rDvXsfKfy8vM1bupcpaadUm5evgY91F0d2ra2sBeVj81m0/Tpr6ply2bKzc3VE0+M1tGj3/nLu3TppISEp5Sfn6/58/+i99//SHa7XbNnv64bb4xQQUGBBg0apaNHv9P8+TP1y1/WlSRdd124duz4WvHxQ63qWqVgs9n01rSX1aJlM+Wez9WTQ54rMj53d4nVc88PU35+gf40f5HmffCxv6xu3V9o85bl6nbPI/r734/q1lY3a9Enc3Xk8LeSpLnv/lmLF6+q6C5Zi8wh8Kzb9pXO5+ZqwdTx2nvgsBLf/VAzxo+QJP2YnqEZ8z/RopmvyBkaoscTJqlNq5u1c+8B1XI6NPHZJ3TWk6E+Q8cSHMpYt26dVbNmDd15Zw9FRUXq9dfHqk+fgZKk4OBgJSaOU3T0vcrKytb69Uu0alWyoqIiJUkdOvRU+/Zt/ef8OxDUqhWmNWs+1ujRL1nWr8ri3m6/U40aNRTboZduv72VXps0Rg/eP0hS4fhMev0F/faO7srKylHyukX67LNknUw7reDgYE2f8apycs7762rVqoVmTH9PM6bPtao71ivHzCE5OVmpqakKCQlR9+7dNW3aNJ0/f16DBg1S3bp1Szyvym+C27Pv74q57RZJ0q3Nmmj/P475y1JOnNRNja9TmNMhu92um29orL8dOKzOd0RpaHwv/+eCgqr8P2OZa9fudiUlbZAk7dz5tVq3vsVf1rRpEx058q3Onk1XXl6etm7dpejoKK1YkaQhQ56TJDVqFK6TJ08XqXPs2JGaPXueTpw4WWH9qKx+85tfK/mLjZKkXbv+qtatW/rLbmraREePfqezZz3Ky8vTtq27Fd0uSpL02sQEzZ37oU78K83/+cjIFoqL66A1SQs1a/YkORyhFduZQODzlvrl8XiUkpJivDweT5Gq9+/fr/j4eKWlpclms+nMmTPyeDy66qqrLtmkKv9TLSs7R47QEP+x3W5XfkHh7XMb1f+ljnyXotM/pivn3Hnt2LtfOefPK+SqmgoNuUpZ2Tka+eoMDYvvbVXzKy2Xy6H09Az/sddboKCgoJ/KnEXKMjMzFRbmlCQVFBRo7twpmjJlgj799MLURN26v1CHDtGaP39RBfWgcnO6nEr3XBiDgoKLxsfpkKfI+GTJFeZUv4d76fTpH7Q2eVORunbv3qsxYyaq8+8e0LfH/qnnE56qmE4EEq+31C+3263Y2Fjj5Xa7i1T973Hx+XxKSUlRTEyMHnroIe3YseOSTSpxWmnx4sVKS0uT1+tVw4YN1b179zL4Fwg8oSFXKSvnnP/Y6/Uq+Kd/zDBnqEYP6qeRr0xXvatrq1nEdarlKvwhdOLUGT318jQ92DVWXTu0s6TtlZnHkymn0+E/ttnsKvgpaHs8GXI6L/x26XA4dPbshd+WBg4cqXr16mrz5mVq1SpW2dk56tGjixYuXCpvAC78/RxleDLkdFwYH7v9ovHJyJSjyPiEKv2sR4OHDJDP51OHDtFqeUtzzZk7Rff3GagVy9f4g/2K5Wv0xpsvVmhfAsIV/L/s37+/evToYbzvcrmKHEdERMjtdis8PFwOh0N79uzRgQMH1Ldv30vWX2JwyM7O1pAhQyRJ8+fPL3XDfy4im9+oDTv2KK59G+09cFg3XN/QX5ZfUKC/HTyieYljVFDg1cCESXpqwP06/WO6Bo2ZrITB8WobebOFra+8tm3bra5dO2nx4pWKiorUvn0H/WUHDx5WkybXq3btMGVmZismpo3eeusd9e3bUw0aXKvExFnKzs6R1+tVQUHhl65jxxhNmjTDqu5UOtu3faW7u8RqyZJVuv32Vtq375C/7NDBw4qI+JV/fKJjojRt2rtaunS1/zOrP/9ITw0fo5Npp7V+46ca9cyL+mr3Xt15Z7S+/vobK7pkLZ+v1Ke4XC4jEPwncXFxRY7Hjh17WfWXGBzq16+vmTNnKigoSBEREZdV2c9RbLvbtO3rb/TwyAny+aSXRz6uVeu3KjvnnPp06ahq1YL0wLBxql69mvr3vFu1w5ya9Paf5MnM0jsfLdU7Hy2VJM1++VnVrFHd2s5UIsuWfa7Y2Du0fv0S2Ww2DRo0Sg880F0OR6jee+9DjR79slas+LPsdrvc7oVKTU3T0qWrNWfOm0pOXqTg4GCNGjVB588XLnzeeGOEjh373uJeVR7Ll69Rx9gYJa/7RDabTYP/51n1ub+bHI5QffD+R3r+uVe1dLlbdrtdf5q/SP9KTSuxrqefekFvTpmgvNw8paWd0rChCRXYkwARgBmtzee7gpB1GXKP7iyPalFGXM1ZJwlkwfYgq5uAS8jMPvb/f6gUchZc3m/zF7uq38tl2obiqvylrABgOfY5AAAMATitVOUvZQUAmMgcAMBq5bP0+18hOACA1QJwWongAABWIzgAAAxcrQQAKM7nZc0BAFAc00oAAAPTSgAAA9NKAAAD00oAAAPBAQBgYIc0AMBA5gAAMLAgDQAwcCkrAMBA5gAAKM4XgGsOPOwHAGAgcwAAqzGtBAAwsCANADCQOQAADAG4IE1wAACrkTkAAAysOQAADGQOAIDiAnETHMEBAKxG5gAAMBAcAAAGFqQBAAYyBwBAcT6CAwDAQHAAABi4lBUAYCBzAAAYAjA48CQ4AICBzAEALObzBV7mQHAAAKsF4LQSwQEArFaVgkNI0x7lVTXKQM4/11ndBFyC87q7rG4CKhCb4AAAJoIDAMAQeHvgCA4AYDWmlQAAJoIDAMDAtBIAoDimlQAAJjIHAEBxZA4AABOZAwCgOB/BAQBguILg4PF45PF4jPddLpdcLpf/ODk5WampqQoJCVGvXr00bdo0BQUFqV+/fqpTp06J9fM8BwCwmM9b+pfb7VZsbKzxcrvdRerev3+/4uPjlZaWpkOHDik9PV2SVLNmzUu2icwBAH6G+vfvrx49zBucXpw1SFJQUJCkwmdG5OXl6aabblLz5s21efNmde7cucT6CQ4AYLUrmFYqPn1UkoiICLndboWHh+umm27S6tWrlZKSokceeeSS5xEcAMBi5bkgHRcXV+R49OjRl3UewQEALMbVSgAAA8EBAGDy2axugYHgAAAWI3MAABh8XjIHAEAxZA4AAIOPNQcAQHFkDgAAA2sOAACDL/Ce9UNwAACrkTkAAAwEBwCAgWklAIAhEDMHngQHADCQOQCAxdgEBwAwsAkOAGDwkjkAAIpjWgkAYAjEq5UIDgBgMfY5AAAMZA4AAAML0gAAAwvSAAADaw4AAEMgTitV2Xsr2Ww2zZo5SV9uWq61XyxSRMSvipTf0/Uubdu6Sl9uWq7HHu0rSQoODta8D6Zrw7ol2rZlpe655y5J0q233qwN65Zo7ReL9NnKBbrmmqsrujuVmtfr1YQ3/qh+g5/VgOEJ+j4ltUj58jXr1WPAMMUPfU6LVyYVKTvz41nF9npUR79LqcgmV3o2m00zZrymDRs+VVLSQjVufF2R8i5dOunLL1dow4ZP9eijD0mS7Ha73nknUevXL1Fy8iL/Oa1atdCRIzuVlLRQSUkL1bv3vRXeH6v5fLZSv8pblc0cunePU82aNRTTvpvaRLVW4uRx6tnrUUmFQeCNxPFq266rsrKytWnjUq1c9YXiOnfQmTM/asDvh6tOndravXONVq78QlPfnKCnRozV3r379PjAhzV61JMaNXqCxT2sPNZu3q7c3FwtmJ2ovfsOKnHW+5ox8QVJ0o9nPZox98/65L235HSEauCIsWp7261qcG095eXna8Ibf1TNGtUt7kHl061bZ9WsWUN33tlDUVGRev31serTZ6Ckwu9PYuI4RUffq6ysbK1fv0SrViUrKipSktShQ0+1b9/Wf05kZAtNn/6upk1718ouWYpppQAS0y5Ka5LWS5J27Nyj21rf4i9r1uwGHTnyrc6eTZckbd2ySzExbfTJ4pVavGSV/3P5+fmSpL4PD9GJEyclScHBQTp3/nxFdaNK+Pp/Dyi6TWtJ0q03N9W+Q4f9ZSmpJ9S0yfUKczklSS2a3qC9+w+pwbX19MasD/RA9zi9++dPLGl3Zdau3e1KStogSdq582u1vuj707Rpk6Lfn627FB0dpSVLVumzz9ZKkho1CtfJk6clSZGRLXXjjY11772/0+HDxzRq1ARlZmZVbIcsxrRSAHG6HPKkZ/iPCwq8CgoKkiS5nA6ley6UZWRmKszlVFZWtjIzs+RwhOovH8/RuBcnS5I/MPym7a81ZMjv9da0ORXYk8ovMytbztBQ/7Hdbld+foEkqVHD+jr87fc6/cOPyjl3Xtv3/E05Oee0dPVa1anlUnRUa6uaXam5XA6lX/T98XoLLnx/XM4iZZmZmQoLKwzeBQUFmjt3iqZMmaBPPy38RWv37r/q+edfU6dOfXTs2PcaM+bpiutIgPjZTSulphad261fv365NqYiZXgy5XA6/Md2u10FBYU/cDwZmXI6LvwwcjocOpvukSSFh9fXJ4vm6u233fr446X+z/Tp003PPzdM3brH6/TpHyqmE1WEIzREWdk5/mOfz6fg4MIfRGFOh/4wdKBGjJ2kenWvVvMbG6t2mEvuhUtls9m07au9OnT4mBJenaqZE1/Q1b+obVU3KhWPJ1POi74/NttF3x9PhpzOC98fh8Ohs2c9/uOBA0eqXr262rx5mVq1itWyZWuU/tP3a9myNZo69aUK6kXg+NllDtOmTVNSUpJ27NihnTt3VlSbKsSWbbt0d1xHSVKbqNb65psD/rIDB/6hJk0aq3btWqpWrZpi7mij7du/0jXXXK3Vn32ohITXNM+90P/5vn176snBAxT7028+KFuRLZpp8/bdkqS9+w7qhosWP/PzC7R3/yG5Z0zUxDFP69h3KYps2VzumZM0b8ZEzZv+mm5qcr1eGzOCwFCGtm3brbi4DpKkqKhI7dt30F928OBhNWlyvWrXDiv8/sS00Y4dX6lv35569tknJUnZ2Tnyer0qKPBqxYo/6de/vlWS1KFDtPbs+d+K7xAMNp+v5KWQ8+fP6+TJk2rYsGGpKw6u3uC/alh5s9lsmjljom5p2Uw2m02PPT5CkZEt5QgN1dz3FuierndpzJinZbfbNW/ex5r9tltT3pyg+/t006GL5rzv7R6vb4/u0vf/TFX6T3OsmzZv14SX3rSqa5cl55/rrG7CZfN6vXp5ytv6+5FvJfn08nNP6cDfjyg7J0d9usXpjx98pHVfbleN6tXV/4H79Ls7o4ucP2B4gsY9M0SNrwu3pP1XwnndXVY34ZJsNpumT39VLVo0lc1m06BBoxQZ2UIOR6jee+9DdenSSQkJT8lut8vtXqh33pmvkJCrNGfOm/rlL+sWXvTxxh+1cuUXatWqhaZOfUl5eXk6ceKUnnzyOWVkZFrdxUs6d65sfwncXr9nqc9pm7qkTNtQ3CWDw38j0INDVfdzCg5VUaAHh6qurIPD1mt7lfqcdv9aXKZtKK7KXq0EAIGC22cAAAwB+JRQggMAWM0nMgcAQDFedkgDAIrzkjkAAIpjWgkAYGBBGgBgIHMAABjIHAAABoIDAMDAtBIAwOANvNhAcAAAq7HPAQBgCMAN0lX3MaEAgJKROQCAxbhaCQBg8NpYcwAAFBOIaw4EBwCwGNNKAADDlexz8Hg88ng8xvsul0sul8t/nJycrNTUVIWEhKh37946fvy43nrrLSUmJl6yfoIDAFjsSvY5uN1uzZw503h/6NChGjZsmP94//79Gj58uGbNmqXc3FytX79ejRo1+n/rJzgAgMWuZM2hf//+6tGjh/H+xVmDJAUFBRX+HT6ftm/frqysLP31r3/V8ePH1aBBgxLrJzgAgMWuZFqp+PRRSSIiIuR2uxUeHq62bduqffv2ys3NvWRgkAgOAGC58lyQjouLM967eNqpJAQHALAYl7ICAAzclRUAYGCfAwDAQHAAABh8TCsBAIojcwAAGAgOAABDIF7KypPgAAAGMgcAsBj7HAAABtYcAAAGggMAwBCIC9IEBwCwGGsOAAAD00oAAAPTSgAAgzcAwwPBoYoKbRRrdRNwCVnHN1ndBFQgppUAAIbAyxsIDgBgOTIHAICBS1kBAAYWpAEAhsALDQQHALAcaw4AAEMgTivxsB8AgIHMAQAsFnh5A8EBACzHmgMAwBCIaw4EBwCwWOCFBoIDAFiOaSUAgMEXgLkDwQEALEbmAAAwsCANADAEXmggOACA5cgcAAAG1hwAAAauVgIAGMgcAAAGMgcAgIHMAQBg8PoCL3PgYT8AAAOZAwBYLPDyBoIDAFiOTXAAAANXKwEADFytBAAwMK0EADAwrQQAMDCtBAAw+AJwExzBAQAsxpoDAMDAtBIAwFCeC9LJyclKTU1VSEiIYmNjNXPmTEnSk08+qTp16pR4HvdWAgCLeeUr9cvj8SglJcV4eTyeInXv379f8fHxSktLU40aNTRixAhdf/31ysrKumSbyBwAwGJXsiDtdrv9WcDFhg4dqmHDhvmPg4KC/H9HSEiIkpOTFRYWpoYNG16yfoIDAFjsStYc+vfvrx49ehjvu1yuIscRERFyu90KDw/XwYMHNW/ePLVr107Hjx9XgwYNSqzf5iuna6iCq5f8l8J6dpvN6ibgErKOb7K6CbiEalc3LtP6ftcwrtTnJP3z8zJtQ3FVds3BZrNp1sxJ+nLTcq39YpEiIn5VpPyerndp29ZV+nLTcj32aN8iZVG3R2rtF4v8x5GtWmjblpXasG6J3pr6smz84P2v2Ww2zZw5UZs2LtMXSeb4dO3aSVu3rNSmjcv06E/jExwcrA/en6Z1axdry5crdc89d0mSWrVqoS1frtS6tYs1lfEpc16vVxMmz1C/QSM0YOhofZ+SWqR8+edr1SN+sOIHj9LiFWv87/ce8KQGDB2tAUNH64VXp1R0swPKlaw5lLcqGxy6d49TzZo1FNO+mxLGTFTi5HH+suDgYL2ROF53d+mrDrG9NHBgP9WrV1eSNOqZwXrnnUTVrFnT//nZsydr5DMv6s6OPeXxZOihh8xUD6XTvVucataoofa/7a4xL0zU5NfH+suCg4OVmPiiunTtp9hOvTXwsb6qV6+u+vbtqTM//KiOsb10b7eH9dbUVyRJs//4up4ZNV4dY3vJk+7RQw8yPmVp7aZtys3N1YI5UzXiid8rcca7/rIfz6Zrxpz5mjdzsubNmqxVSet1/F9pOn8+V5IK3585Wa+MGWlV81GCKhscYtpFaU3SeknSjp17dFvrW/xlzZrdoCNHvtXZs+nKy8vT1i27FBPTRpJ05Oh36nP/40XqCm9wrbZt3y1J2rp1l6LbRVVQLyqvdtG3KylpgyRp5849at36Vn9Zs6ZFx2fL1l2KiY7S4sUr9eKLif7P5RfkS5IaNLhW27d/JUnaum232kXfXnEdqQK+/ts+Rbe9TZJ0a4tm2nfwH/6ylNQTanpDY4W5nLLb7WrR7Abt3XdAhw4f1blz5/X40wl6dNhz2vvNAauaHxB8Pl+pX+WtygYHp8shT3qG/7igwOtf1Xc5HUr3XCjLyMxUmMspSfr008+Ul5dXpK5jx75X+zvaSpK6dr1LoaEh5d38Ss/ldBYZg4KCAv/4FI7dhcv1MjOy5ApzKSsrW5mZWXI4QvXxR3P04vjCQHHs2Pe6wz8+nRQawviUpcysbDlDQ/3H9iC78vMLJEmNwuvr8LHvdPqHH5Vz7py2796rnJzzqlmzhgb07aU5U1/VuGeH6g8TJvvPqYp+VtNKCxYs0LRp05Senq7ly5eXe0MqWoYnUw6nw39st9tVUFD4n9OTkSmn48J/dqfDobPpHqOOf3vs8ZH6w+ihWr50vk6dOq0zp38ov4ZXEZ6MjCJjcPH4FB87hzNU6WcLxyc8/Fp9kfQXLfhwsT5euFSS9PigkRo9eqiWLnXr1MkzOn2G8SlLjtAQZWXn+I99Xq+CgwsDeZjLqT8MH6QRY17R2NemqvlNEapdy6VfNWygezp3lM1m068ahatWmEunqvC4+K7gT3krMThkZWXpiSee0LvvvqsjR46Ue0Mq2pZtu3R3XEdJUpuo1vrmorT2wIF/qEmTxqpdu5aqVaummDva+Kcl/pMud8dq4KBn1O2+eNWpU1vJa7nS5L+1betuxf00PlFRrfXNvoP+sgMH/6EmTa73j88dMW20fcdXuuaaq/XZqg+VMOY1ud0L/Z+/++5YDRr0jO67r7/q/KK21q7dXOH9qcwiWzbX5m27JEl7vzmgGyKu95fl5xdo776Dcs9K1MSxo3TsuxRFtmyuJauS/GsTJ0+dUVZWtur+ouTdupWd1+cr9au8lbjPweVyKSMjQ4MHD9b48ePLvSEVbenS1eoU216bNy6TzWbTY4+P0IMP3idHaKjmvrdAz46eoM9WLZDdbte8eR8rNfVEiXUdPnxMK5b/STnZOdqwcatWf76uAntSOS1dtlqxne7Qxg1LZbPZ9PigkXrwgfsU6gjVe+8t0OjRE7Rq5Z8Lx8e9UKmpJ/TmmxNUq1aYEp5/WgnPPy1JurfbIzp8+JiWL5uv7Owcbdy4VZ8zPmUq9rfttHXX1+r3PyMln08vjxmpVUnrlZ2Toz7du6hatWq6/9FhqlG9uvo/1FO1a4Wp1z2dNebVKXpk8DOyyaaXEkb4s42qKPBuu8c+hyqLfQ6BjX0Oga2s9zlEN+hY6nO2HC/fX3LYIQ0AFuOW3QAAAw/7AQAYyBwAAIaKuDS1tAgOAGAxppUAAAamlQAABjIHAICBzAEAYGBBGgBgqIh7JZVWlb1lNwCgZGQOAGAxppUAAIZAnFYiOACAxcgcAAAGMgcAgIHMAQBgIHMAABjIHAAABp/Pa3UTDAQHALAY91YCABi4KysAwEDmAAAwkDkAAAxcygoAMHApKwDAwLQSAMDAgjQAwBCImQNPggMAGMgcAMBiXK0EADAE4rQSwQEALMaCNADAQOYAADCw5gAAMLBDGgBgIHMAABhYcwAAGJhWAgAYyBwAAAaCAwDAEHihQbL5AjFkAQAsxV1ZAQAGggMAwEBwAAAYCA4AAAPBAQBgIDgAAAwEBwCAgeAAADAQHAAABm6fcRmSk5OVmpqqkJAQ9e7d2+rmoJhz585p3LhxGj58uMLDw61uDor56KOPlJ6eLofDoYcfftjq5uAykTlchv379ys+Pl5paWlWNwX/walTp9S0aVOrm4ESxMbGasCAATp16pTVTUEpEBwuQ1BQkKTAvHMipIYNG6pWrVpWNwMlCAkJ0YwZMzRgwACrm4JSIDhchoiICLndbqYsgCswfvx42e12bdy40eqmoBS4KysAwEDmAAAwEBwAAAaCAwDAQHAAABgIDgAAA8EBAGAgOAAADP8HUfkhT5kzaVcAAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 504x360 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "cm = confusion_matrix(y_test, pred_svm_poly) \n",
    "cm = cm.astype('float') / cm.sum(axis=1)[:, np.newaxis]\n",
    "print(cm)\n",
    "\n",
    "df_cm = pd.DataFrame(cm, range(3), range(3))\n",
    "plt.figure(figsize=(7,5))\n",
    "sn.set(font_scale=0.5) # for label size\n",
    "sn.heatmap(df_cm, annot=True, annot_kws={\"size\": 10}) # font size\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "              precision    recall  f1-score   support\n",
      "\n",
      "     class 0       0.94      0.92      0.93       246\n",
      "     class 1       0.90      0.94      0.92       231\n",
      "     class 2       0.96      0.95      0.96       479\n",
      "\n",
      "    accuracy                           0.94       956\n",
      "   macro avg       0.94      0.94      0.94       956\n",
      "weighted avg       0.94      0.94      0.94       956\n",
      "\n",
      "\n",
      "F1-score is : 0.9365816028633308\n",
      "Cross validation score is : 95.1121076233184\n"
     ]
    }
   ],
   "source": [
    "target_names = ['class 0','class 1','class 2']\n",
    "print(classification_report(y_test, pred_svm_poly, target_names=target_names))\n",
    "print()\n",
    "\n",
    "f1_svm_poly = f1_score(y_test, pred_svm_poly, average='macro')\n",
    "cross_val_svm_poly = np.mean(cross_val_score(svm_model_poly, X_train, y_train, cv=10))*100\n",
    "print('F1-score is :', f1_svm_poly)\n",
    "print('Cross validation score is :', cross_val_svm_poly)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## __Markdown table__"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "| Classifier             | Accuracy %  | F1-score | Cross validation |\n",
    "|------------------------|-------------|----------|------------------|\n",
    "| dummy                  | 50.1046     | 0.2225     | 52.6905          |\n",
    "| Gaussian Naive Bayes   | 94.3514     | 0.9403     | 93.5426          |\n",
    "| K Neirest Neighbors    | 85.9832     | 0.8519     | 88.4753          |\n",
    "| Logistic Regression    | 94.6652     | 0.9411     | 93.7668          |\n",
    "| Multi-Layer Perceptron | 95.3974     | 0.9492     | 94.2152          |\n",
    "| SVM (kernel = rbf)  | 96.1297     | 0.9566     | 95.9192          |\n",
    "| SVM (kernel = linear)  | 93.5146     | 0.9287     | 90. 9865         |\n",
    "| SVM (kernel = poly)    | 94.1422     | 0.9365     | 95.1121          |"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [],
   "source": [
    "acc = [accuracy_dummy, accuracy_gnb, accuracy_knn, accuracy_logreg, accuracy_mlp, accuracy_svm, accuracy_svm_lin, accuracy_svm_poly]\n",
    "f1 =[f1_dummy, f1_gnb, f1_knn, f1_logreg, f1_mlp, f1_svm, f1_svm_linear, f1_svm_poly]\n",
    "classifiers = ['dummy', 'GNB', 'KNN', 'LOGREG', 'MLP', 'SVM(RBF)', 'SVM(LIN)', 'SVM(POLY)']     "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## __Accuracy Bar plot__"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAtAAAAFpCAYAAAC1eh+8AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8rg+JYAAAACXBIWXMAAAsTAAALEwEAmpwYAABLVklEQVR4nO3deVxU9f7H8dewKSKKIO6AihumgIoWpqKWay5oppmpN80ytdS0XFLDNNfMFPXeRM3cSjJDDEvTisTMpGu5b7gvIK4sItvM749+TXmzK9MVZoD38/HwkXNmzjmfOd+oN9/H53yPwWQymRARERERkTyxs3YBIiIiIiKFiQK0iIiIiIgFFKBFRERERCygAC0iIiIiYgEFaBERERERCyhAi4iIiIhYQAFaRIql5ORk6tevT+fOna1dis05dOgQnTt3pkGDBsyePdvi/Tdu3Ej9+vXNr+vWrcumTZv+Vi3/y76W+M+a//Ma9O/fnzfeeCPf6xCRwsHB2gWIiFjDpk2b8PLyIiEhgfj4eIKCgqxdks14//33cXBwYMuWLbi6ulq7nALRuXNnWrVqZX79n9fAYDDg4KD/ZYrIrzQDLSLFUlRUFJ07d6Z+/fqsX7/e2uXYlNTUVPz8/PD29qZcuXLWLqdAlCxZkvLly5tf/+c1cHNzo3Tp0lasUERsiQK0iBQ7+/fv58SJEzRv3pz27duzdetWbt26ZX4/LS2NqVOn0rx5cxo1asTgwYM5deqU+f3Y2FieeuopAgICaNu2LcuWLTO/d6+Wgz9uGz9+PKNGjaJ///40adKEdevWkZmZycyZM2nTpg0NGjTgkUceYcKECWRkZNxVc//+/QkMDKRFixbMmTOHnJwcVq5cSbNmzcjKyjJ/Nj09ncDAQLZv337P73/8+HGGDBlC06ZNadasGa+//jrXr18HoG3btnz//fdERUVRt25dLly48Kf981KvJf7b9bTkvLdv32bChAk0b96chg0b0rt3b3bv3m3e/9tvvyU0NBR/f39atGjBtGnTyMzMBO5u4bjXNfjPFo74+Hiefvpp/P39eeyxx5g3b575WPDrmC9YsIBWrVrRqlUrkpOT2bhxI506daJBgwa0adOGhQsXYjQa/9Y1ExHrUoAWkWLns88+o3z58jRp0oROnTqRmZlJVFSU+f1Ro0axe/du5s2bx6effkqpUqV4/vnnyc7OZt++fQwdOpRHH32UqKgoJkyYwOLFi4mMjMzz+b/44gvatWtHZGQk7dq1Y/bs2XzzzTfMnTuXL7/8kilTphATE2OeGT9//jwDBgzAx8eHDRs2MHfuXKKjowkPD6dbt27cvn2b2NhY8/G3bduGs7MzISEhfzr3hQsX6Nu3L2XLlmXt2rUsWbKEo0ePMmjQIHJzc9mwYQNBQUF06tSJuLg4Kleu/Kdj3K9eS1hyPe933oULF3Ly5EmWL1/Oli1b8PPzY/jw4dy+fZvr168zYsQInn76ab744gvmzp3Lli1biIiI+NN57ncNjhw5wuDBg2nXrh2bN29m+vTpfPPNN4SFhd31uU8++YT333+fRYsWce3aNaZMmcLo0aPZtm0bEydOZPny5URHR1t8zUTE+tTQJSLFSlZWFlu2bKFz587Y2dlRvXp1HnroISIjIxk4cCCnTp1i586drFq1iocffhiAt956i/fff5+bN2+yevVqgoKCGDVqFAA1atTgzTffxN7ePs81eHp6MmDAAPPrgIAAnnjiCZo0aQJAtWrVWLduHcePHwcgMjKS8uXLM3XqVOzt7alVqxbTpk3j0qVLuLu707JlS6Kjo2nXrh3wa393ly5dcHR0/NO5161bR5kyZZg5c6b5/fnz59O5c2d27txJ69atcXR0pGTJknh6et6z/vvVawlLruf9znv27FlcXFyoVq0arq6ujBs3jg4dOmBvb09iYiLZ2dlUqlSJqlWrUrVqVZYtW0apUqX+dB53d/f/eg2WL19OSEgIgwcPBsDHx4epU6fyzDPPMHr0aCpUqABAjx498PPzA+Crr77CYDBQpUoV858PPviASpUqWXzNRMT6FKBFpFjZsWMHN2/epGPHjuZtnTp14p133iE+Pp6rV68C4O/vb36/XLlyjB8/Hvi1/eGPN5sBhIaGWlRDtWrV7nrdvXt34uLimDNnDmfOnOHkyZOcO3fO/Lnjx4/z0EMP3RUq27RpY/57z549efXVV0lJSSEjI4M9e/bw+uuv3/PcJ06coGHDhneFa19fX8qVK8fx48dp3br1feu/X72WsOR63u+8gwcPZtiwYQQHB9OoUSNatmxJ9+7dKVGiBH5+fnTq1IkXX3yRSpUq8eijj9KuXbu7rmNeHTlyhLNnz9KoUSPzNpPJBEBCQoI5QHt5eZnfb9myJQEBATz55JP4+PjQokULOnfuTJUqVSw+v4hYn1o4RKRY+eyzzwB47rnnqF+/PvXr1+fdd98Ffp3pvd9KC5auxJCTk/OnbSVLlrzr9aRJkxg7diwmk4n27duzePFimjZtmudztm7dGhcXF7Zu3crmzZupXbv2XUuy/VGJEiXuud1oNN5zxvpe7levJSy5nvc7b1BQELGxscyfP5+aNWuydu1aunfvzsmTJzEYDLz33nvExMQwYMAALl++zPDhw//UdpEXjo6OhIaGEhUVZf6zadMmtm3bRkBAgPlzf7zWJUuWZM2aNWzYsIHu3btz+PBhnn322Xu2kIiI7dMMtIgUG8nJycTFxfHMM8/Qt2/fu96bPXs2X375Jf369QPg4MGD5nCWlpZGu3btCA8Px9fXl4MHD9617/z58zlx4gRLlizB0dGRtLQ083tnz579rzWlpaXx6aefsmDBAtq3bw/8GrrPnz9vnp309fXliy++wGg0Ymf367zH+vXrWb9+PRs3bsTR0ZEuXbqwfft2bt26RY8ePf7yfLVq1WLTpk1kZ2ebA/PJkye5desWvr6+972GeanXEve7npacd9GiRTRq1Ih27drRrl07MjMzadmyJd988w0ZGRl8/vnnTJgwgVq1ajF48GAiIiJYtGiRxSG6Vq1aJCQk4OPjY962b98+IiIimDp16j3bQnbt2sXPP//M8OHDadiwoTm8f/bZZwwZMsSi84uI9WkGWkSKjU2bNmEymXj++eepU6fOXX+ef/55MjMz+eWXX3jssceYOnUq8fHxJCQkMGHCBFxdXfH392fQoEHs3buXJUuWcPbsWbZu3cqqVato27YtAIGBgURGRnL06FEOHTrEm2++iZOT01/WVKJECUqVKsWOHTs4d+4chw8fZsyYMVy+fNm8ska/fv24evUq06ZNIyEhgV27dhEeHn7XTYI9e/bk+++/5+DBg3Tt2vUvz/fss8+SmprKhAkTOHHiBPHx8YwdO5Z69eoRHBx832uYl3otcb/racl5L168yNSpU9mzZw8XL14kOjqa1NRUAgICcHV1Ze3atbz77rucO3eOI0eO8M0339zVqpNXQ4YMYf/+/cycOZOEhAR+/PFHxo0bR2pq6l/2jTs6OrJ48WJWrVrF+fPn2bdvH3v27LlrxlpECg8FaBEpNqKiomjdujVVq1b903vBwcHUq1ePyMhIZs2aRcOGDRk2bBi9e/cmOzubZcuW4eTkxEMPPUR4eDhffvklTzzxBHPnzmX06NH06tULgLCwMFxdXXnqqad45ZVX6N2793+9UczR0ZH33nuPQ4cO0aVLF4YNG0bZsmUZNGiQeWa2YsWKREREcOTIEUJDQ5k4cSK9evVixIgR5uPUr1+f6tWr06JFi7vWM/5P5cuXZ8WKFSQlJfHkk08yfPhw/Pz8+OCDD/LUwpGXei1xv+tpyXknTZrEI488wpgxY+jQoQMrV65k5syZNGvWjOrVq7N48WJ27dpFt27dGDBgAJUqVTK371iibt26vP/++/z73/8mNDSUUaNG0bRpUxYtWvSX+zRr1owZM2YQGRnJE088wfDhw2natKmebihSSBlMv935ICIihVZOTg6tW7dmypQp5hYHERHJH+qBFhEpxLKysvj666/ZuXMnTk5Of2tVCRERsYxmoEVECjGj0UjLli1xcnJi7ty5BAUFWbskEZEiTwFaRERERMQCuolQRERERMQCCtAiIiIiIhZQgBYRERERsYBW4chHN26kYzSqxbw48PAozbVraff/oBQJGu/iReNdfGisi5d7jbednYFy5Vzuu68CdD4yGk0K0MWIxrp40XgXLxrv4kNjXbz83fFWgM5HHh6lrV2CFCBPT1drlyAFqDiO953MbFJT7li7DBERq1OAzkevzIzi6o10a5chIvJArJvTj1QUoEVEdBOhiIiIiIgFFKBFRERERCygAC0iIiIiYgEFaBERERERCyhAi4iIiIhYQAFaRERERMQCCtAiIiIiIhZQgBYRERERsYACtIiIiIiIBRSgRUREREQsoAD9F4xGo7VLEBEREREbpAD9F+zsfr80JpPJipWIiIiIiC1RgP6DPwbl1NRUpk+fTm5uLgaDwYpViYiIiIgtUYDm9+BsMBjIycnBZDKRm5uLyWTi1q1bVq5ORERERGxJsQ7QfwzOAD/99BOzZs1i165dODk5cerUKVxcXKxZooiIiIjYmGIZoE0mEyaTyRyct2zZwoIFCyhXrhxNmzbln//8J+fPn6dq1ars3LnTytWKiIiIiC1xsHYBBc1oNJpvELx06RL//ve/2bVrF5mZmXz++ecMGjSIkiVLsmrVKhISEnjiiScA7grcIiIiIlJ8FbsAbWdnR0ZGBsuWLeP777/H09OTN954AxcXF9555x127dpFhw4dCAwMZNSoUWzbto3g4GCFZxEREREBikELx3+u53zo0CFGjBiBi4sL7733Hjk5ORw8eJDSpUsTEBDArl27OHz4MGXLlmXy5MmUL1/eSpWLiIiIiC0qsjPQv90g+Fu7xu7du8nJySE4OJicnBycnZ2pWLEibdq0YdeuXTRo0IB27dqRk5ODt7c3qampzJ8/n6ZNm1rza4iIiIiIjSmSATo3Nxd7e3sAbt++zYoVK4iPj+fWrVsMGDCA4cOHs2rVKnr06EFoaCjffPMNP//8Mx06dOCpp54CIDMzk9dffx0vLy9rfhURERERsTFFMkDb29uTmZnJggULSEtLIzs7m5UrV3Ly5EleeOEFvvrqKyIjI1m1ahUvvPACkyZNokqVKub9TSYTJUqUUHgWERERkT8pkj3QJ06c4LnnnqNChQp069aNmJgYzp07R61atQgICGDmzJm88sorlCxZEsAcnv9zXWgRERERkf9UqAP07du3AcjJyQHgwoULREdHU6JECc6dO0dgYCBBQUE899xzvPPOOwBMmTKF5ORkqlatyoABA+46noKziIiIiNxPoQ3QW7Zs4cUXX7xr2/79+4mPjyclJYURI0awYsUKAF555RX27NnDjz/+SLly5ViwYIG5R/q3WWcRERERkbwodAE6NzcXgM6dO3Pr1i3CwsL46KOPAHjkkUfw9fUlNjaWNm3akJWVRWRkJPb29qxatYpmzZr96TiadRYRERERSxS6AP3bzPGePXuoVasWW7du5euvv+bIkSO4u7tTs2ZNTp06xZEjR+jbty83b94EoG7dusDvM86/HUdERERExBKFbhWOnJwcpk2bxs2bN3n66afJycnhxx9/5KuvvsLPzw9nZ2eysrK4dOkSTz31FCEhIXftrxlnEREREflfFLoA7eDggJOTE8899xyBgYF4eXkxYMAADhw4wCuvvMLly5cZO3YsDz/8sHkfo9FofqBKQVo4IbTAzykikl/uZGZbuwQREZtQ6AJ0bm4uBoOBpKQkACpVqoSHhweVKlXiH//4B76+vubPmkwmDAaDVcIzwLVraRiNukmxOPD0dCU5OdXaZUgB0XiLiBRvhS5A29vb07lzZzZt2sSPP/7I5cuX6dy5M127dqV8+fLA708iVLuGiIiIiDxohS5AAwQGBuLt7c2WLVt44oknaNy4MfB7q4ZuEBQRERGR/FIoAzSAu7s7zz77rPm1yWSyWquGiIiIiBQfhT5x6vHbIiIiIlKQCn2AVnAWERERkYJkMOlZ1iIiIiJSAHKyMrlxK8vaZQD3XlHJzs6Ah0fp++5baHugC4MD/xpHVso1a5chIiIiYhOavL4MsI0A/b8o9C0cIiIiIiIFSQFaRERERMQCCtAiIiIiIhZQgBYRERERsYACtIiIiIiIBRSgRUREREQsoAAtIiIiImIBBWgREREREQsoQIuIiIiIWEABWkRERETEAgrQIiIiIiIWUIC+hxMnTmA0GgEwmUxWrkZEREREbImDtQuwJd9//z1Lly6lVKlSODs7M2rUKLy8vKxdloiIiIjYEAVoID09nXHjxpGSksLYsWNp2LAhCxcu5NChQwrQIiIiInKXYt3CkZWVxfHjx8nNzcXV1ZVWrVrh7+9PamoqO3fuJCUlhaNHjwKYWzpEREREpHgr1gE6NjaWSZMmYTQa6dWrF0eOHOHdd99l3Lhx+Pr6cvPmTXr27InJZMLOrlhfKhERERH5f8WuhePcuXN4e3sD0K5dOz7//HNiYmLo2bMnNWrUYMOGDXz44Yf4+PgA8MMPP3Do0CEaNGhgzbJFRERExEYUq2nVr7/+mmeeeYZp06axe/duAIYPH050dDRXr16lY8eOtGjRguzsbFJTU3njjTcoWbKkOXCLiIiIiBT5Gejc3FyWLVtG+fLlOXfuHO3bt+fcuXNMnz6dxo0bM2jQIOrVq8fnn3/OwIEDadKkCYMHD8bLy4uuXbvSp08fa38FEREREbEhRTpAx8TEsH79eurWrUutWrW4ffs2FSpUwNfXlxo1apCUlMTSpUtJSEhgx44dNGnShJYtW5KSkkK3bt0oV66ctb+CiIiIiNiYIhugU1JSiI2NZezYsfj7+wNw6dIlFi5cSKlSpXB0dKR379706NGDyMhIVq9eTVRUFDNmzGDgwIFWrl5EREREbFWRDdBfffUVJpMJf39/srKymD9/PsePHyc9PR1HR0fKlSvH7t27CQ4Opnfv3vTq1UsrbYiIiIjIfRXZxFijRg1cXV1JS0vDycmJVq1asXz5cqZPn05CQgJXrlzh2LFj3LlzB0DhWURERETypMjOQHt4eFCqVCni4+Np3bo1wcHBAPz8888MHDiQkJAQqlWrRsmSJa1cqYiIiIgUJkU2QHt7e+Pr68umTZu4fv06tWvXZtWqVdy8eZNXX32VevXqWbtEERERESmEimyANhgM9OjRAw8PDw4cOMCuXbsICQmhW7du1i5NRERERAqxIhugf9OqVStatWqF0WhUn7OIiIiI/M8MJpPJZO0iRERERKToy8nK5MatLGuXAYCnpyvJyal3bbOzM+DhUfq++xb5GWhrunYtDaNRv58UB/f6IZSiS+NdvGi8iw+NteSVehpERERERCygAC0iIiIiYgEFaBERERERCyhAi4iIiIhYQAFaRERERMQCCtAiIiIiIhbQOtAiIiIiNiozK4uUW5nWLqNI0jrQNmrsJ1O5mnbd2mWIiIhIIbXyuQWAArStUQuHiIiIiIgFFKBFRERERCygAC0iIiIiYgEFaBERERERCyhAi4iIiIhYQAFaRERERMQCCtAiIiIiIhZQgBYRERERsYACtIiIiIiIBRSgRUREREQsoAD9F/bt28f163oMt4iIiIjcTQH6D0wmEwCJiYmEh4dz4MABcnNzrVyViIiIiNgSBej/d/HiRQwGAwCVKlWiTZs27Nq1i6SkJCtXJiIiIiK2RAEaSE5OZtCgQfTu3ZuPPvoIgD59+nDz5k1++uknMjIyrFyhiIiIiNiKYh2gf/rpJyZOnIinpye9evWiVKlSfPLJJ4wZM4YvvviCwMBADh8+zIULF6xdqoiIiIjYCAdrF2BNgYGBDB8+nP3799OpUyeMRiPu7u40btyYtWvXcuLECRISEnB0dGT48OGUKFHC2iWLiIiIiJUVqxnoq1evsmLFChISEsjJycHe3p6XX36Zt99+m2rVqlGpUiWOHj1K6dKlmTJlCiNHjqRt27bs2rWLzMxMa5cvIiIiIjag2MxAr127lk8++YTk5GSio6OpW7cur776Kv369WPLli1s2rSJxx9/nDNnzhATE8OgQYMICgoiKCjI2qWLiIiIiA0pFjPQd+7c4eeff2bevHmEhoYSGhpKiRIlmDVrFrNnzyYkJIRly5ZRqlQp/P39OXXqFOfPn7d22SIiIiJig4psgL5+/TqLFi3CZDJx69YtkpOT8fX1JTAwkHPnztGrVy/efvttypUrx/79+zlx4gTTpk2jTZs2jBw5Ei8vL2t/BRERERGxQUW2hSM7O5uzZ8+yadMmPD09zdvbtWvH4cOH2bVrFzVq1OCFF17g1q1bdOnShezsbIC7Pi8iIiIi8kdFNkBXqFCBrl27smnTJnJzczl16hRvvPEGgYGBhISEsHHjRuLj4wkJCaFs2bJ07NjR2iWLiIiISCFQZAO0wWCgcePGxMXFERcXx4YNGzhx4gQRERGULFmSb7/9lgYNGli7TBEREREpZIpsgAYoXbo0Xbt25ciRI1y/fp1HH32UoKAgTp06xXPPPcfDDz9s7RJFREREpJApsjcR/qZevXo89thjLF++HIASJUrg5+en8CwiIiIif0uRnoEGcHR0pHXr1nh6emIymTAYDNYuSUREREQKsSIfoAGqV69O9erVrV2GiIiIiBQBBpPJZLJ2ESIiIiLyZ5lZWaTcyrR2GUWSp6crycmpd22zszPg4VH6vvsWixloa7l2LQ2jUb+fFAf3+iGUokvjXbxovIsPjbXkVZG/iVBERERE5EFSgBYRERERsYACtIiIiIiIBRSgRUREREQsoAAtIiIiImIBBWgREREREQtoHWgRERERKVDZdzK5mZpl1Rq0DrSN+ubVsWRcvWbtMkRERERsSudVH4CVA/T/Qi0cIiIiIiIWyFOA/vrrr8nNzc3vWkREREREbF6eWjjGjBlDqVKl6NKlCz169KBevXr5XZeIiIiIiE3K0wz0rl27GDt2LMePH6dnz56EhoaycuVKrl+/nt/1iYiIiIjYlDwF6FKlStGjRw8++OADvvnmG7p27cqXX35J69atGTZsGNu3b1eLh4iIiIgUCxbfROji4oKbmxtubm4AnD9/nrCwMNq3b8++ffsedH0iIiIiIjYlTz3QOTk5fPvtt0RHRxMbG4uLiwtdunRh5MiR+Pn5kZOTw5tvvsmYMWP4+uuv87tmERERERGryVOAfvTRR0lPT6dVq1bMmzeP1q1b4+Dw+64ODg60bNmS2NjYfCtURERERMQW5ClAv/TSS3Tr1g13d/e//Ezbtm3p0KHDAytMRERERMQW5akHun///qxatYp169aZt/Xs2ZNFixbx25PAnZycMBgM+VOlFelJ5yIiIiLyR3kK0O+++y4bNmygatWq5m19+vRh/fr1LFq0KN+Ks5b4+Hg++OADfvnlF7Kzs61djoiIiIjYkDwF6M2bNzNv3jxCQkLM2/r06cOsWbPYuHFjvhVX0C5evMjEiRNZtmwZKSkp7N69m4yMDGuXJSIiIiI2JE890KmpqZQvX/5P2ytXrlxkHqaSk5PDBx98gJ+fH/379wfAaDQSHR2Nv78/NWvWxGg0Ymdn8cp/IiIiIlKE5CkNNmzYkA8//PBP/cBr166lfv36+VJYQTly5AgAiYmJ7N+/n2eeeQaA2NhYnnnmGT777DPCwsIAFJ5FREREJG8z0GPHjmXgwIH88MMPPPTQQwAcPnyY5ORkli1blq8F5pf9+/ezdOlSHB0dmT17NllZWQQEBJCWlkbZsmXx8PDgzTffxM/Pj8cee4zdu3cTHBxs7bJFRERExMryFKD9/f2Jjo4mMjKSEydO4ODgQMeOHXnmmWeoWLFiftf4QCUnJ7N48WISExMJDQ2lY8eOwK9PWDx79ixnz56lYcOGNGjQAICEhASCg4Np2LChNcsWERERERuRpwAN4OXlxZgxY/KzlgLxz3/+k507d7Jjxw7ztk8++YQnnniCli1bsnnzZuLj4+nfvz9r165ly5YtdO/endKlS1uxahERERGxFXkK0JmZmaxfv57jx4+Tm5tr3p6VlcXBgwfZunVrvhX4IHz11VdcunSJnj17MmjQIM6dO8fZs2e5cOECERER1KpVi+7du9OvXz8OHjzIhg0bmDJlCiVLlmTJkiX3vIFSRERERIqnPAXoqVOnEhMTg7+/Pz/99BNBQUGcP3+exMREnnvuufyu8W87duwYERERpKSk4OHhweLFi3nhhRdo06YNTz75JCEhIYwcOZJGjRoBcOXKFfz9/fH39+f27duUKlXKyt9ARERERGxNnpaV+Oabb5g1axarV6/Gy8uLN998k+3bt9O+fXtu376d3zVazGg0Mm/ePEaPHk3Hjh1ZunQpr732GidOnAAgNDQUPz8/unbtSqNGjbh27RrTp09nypQp3Lx5E0DhWURERETuKc/rQAcEBABQq1YtDh48iK+vLy+++CIjRozI1wL/DqPRiIODA02aNOHxxx8H4Ouvv6Zy5co4Ozvj7OzMwIED+eCDDzhz5gyxsbEEBwczbtw4HB0drVy9iIiIiNiyPAXoChUqkJSURJUqVahevTrHjh0DwNXV1WYepBIbG8uXX35J+fLl6d+/P6GhoSxfvpw5c+aQkJBAcnIyqampLF++nNDQUIKDg9myZQu7d+/m7bffpkqVKtb+CiIiIiJSCOQpQLdr147x48cza9YsmjdvzmuvvUbjxo3ZsWMHXl5e+V3jf3XlyhVmz57N7du3GThwIAcPHsRgMODu7k6zZs2YNWsWb7/9NiEhISQkJLBu3TrGjh1L69atmTFjBiVLlrRq/SIiIiJSuOQpQI8ZM4acnBwuXLhA165dadu2La+88gouLi689957+VzifxcXF4e3tzcjR44EIDAwkMjISACCgoLo2bMnycnJAPj6+jJ58mQuXbqkGWcRERER+VvyFKA3btzIsGHD8PDwAGD69OlMmDCBEiVK4OCQ56WkH7iMjAzi4uLMj98+ePAgS5cuBaBSpUpUqFCB4OBgli1bRtOmTfHx8QFQeBYRERGRvy1Pq3DMmzePlJSUu7a5uLgUeHjeu3cvEyZMICIign379uHs7EzZsmX55ZdfAKhcuTLjx49n4cKFZGZmkpSURN26denatasehCIiIiIiD0SeErCfnx/ff/89NWrUyO967ikjI4O5c+dy4cIF+vfvT0JCAp999hnbtm2jR48eREREcOPGDfMMeVJSEjdu3KBevXq4u7sTGhpqlbpFREREpOgxmEwm0/0+NHr0aPMKF15eXn+68W7FihX5ViDAgQMHiIyMZNq0aeZtycnJhIWF0blzZxITE/n+++95+umnOXXqFHFxcbRu3ZrBgwfna10iIiIiYrnsO5ncTM2yag2enq4kJ6fetc3OzoCHx/27FvI0A12yZEmrzOLm5uZib2/PyZMnqVixIvDr48OdnJzw9PQkJCSEW7duMXjwYDw9Pbl8+TJZWVmEh4fj5uZW4PX+p2vX0jAa7/v7iRQB9/ohlKJL4128aLyLD4215FWeAvTMmTPzuw6z69evs27dOl588UXzQ02Sk5O5dOkSAHZ2dhiNRuzs7Pjll1+oX78+AN26dSuwGkVERESk+MpTgN68efN/fb9r164PpBiA7Oxszp8/T1RUFE899RQA/fv3p0OHDhw9epR69eoBcOvWLYxGI02aNHlg5xYRERERuZ88BejXXnvtnttLlChBpUqVHmiArlChAp07dyYqKopWrVpRsWJFnJ2deemllwgPD6dy5cpUqFCBb7/9lpYtW5oDtYiIiIhIQchTgD569Ohdr3Nzczlz5gxhYWH06dPngRZkMBho0qQJ+/btY/369bzyyisA9O3bl+rVq5OSksK5c+d49913qVSp0gM9t4iIiIjI/eRpHej/ZG9vj6+vL+PHj2fBggUPuiZKly5N27ZtOX/+PBcvXuTChQsMGjSI7777jnbt2jFkyBCFZxERERGxiv/pSSj29vZcuXLlQdVyl3r16lG/fn169+6Nl5cXffr0oUePHvlyrvySl2VQpOjw9HS1dglSgDTexYvGu/jQWBecrMwcbqVkWLuMv+Vv30SYlpZGZGQk/v7+D7woACcnJx5//HHs7Ozo27cvTk5O+XKe/LR47hZu3bxt7TJEREREbM7Et3tZu4S/7W/fROjg4ECjRo0ICwt70DWZeXl5MXDgwHw7voiIiIiIpf7WTYQiIiIiIsVVnm8i/OSTT4iJiTG/HjFiBJ999lm+FCUiIiIiYqvyFKCXL1/OjBkzyMnJMW/z9fXlrbfeYu3atflWnIiIiIiIrclTgF63bh1z586le/fu5m2jR49m1qxZfPjhh/lWnIiIiIiIrclTgL527Rq1a9f+03Y/Pz8SExMfeFEiIiIiIrYqTwG6Tp06REdH/2l7TEwMNWvWfOBFiYiIiIjYqjytwjF8+HBeeukl9u7da173+eDBg+zdu5fw8PB8LVBERERExJbkaQY6JCSEtWvX4unpSWxsLLt27cLDw4NPPvmEtm3b5neNIiIiIiI2I8+P8vb392fixIl4eHgA8O9///uefdEiIiIiIkVZnmagT58+Tfv27Vm2bJl524gRI+jatSvnz5/Pt+KszWg0AmAymaxciYiIiIjYijwF6OnTp/PQQw/x4osvmrdt27aN2rVrM2PGjHwrzlq+/fZbxowZw5IlS7hz5w4Gg0EhWkRERESAPAboffv28eqrr+Lm5mbeVrp0aUaNGkV8fHx+1VbgEhISmDBhAhs3bqRPnz4kJCQwZcoUAAwGg5WrExERERFbkKceaGdnZ65cuUL16tXv2n7jxg3s7PL8NHCbdv78ed588028vLxYuHAhAI6OjsTGxmIymRSgRURERATI4wx0+/btCQsLIz4+nszMTDIzM4mPj2fq1Kk89thj+V1jvvqtNcPNzY2OHTtSt25dsrOz2bRpE/PmzaNRo0YKzyIiIiJilqcZ6LFjxzJy5EieffZZc5g0mUw8/vjjTJw4MV8LzC/Hjh2jbt265v5mV1dX/P39iYqKolu3bjRt2pSpU6fi6+tr7VJFRERExIbkKUC7uLiwbNkyTp06xYkTJ3BwcMDT05NffvmFvn37snnz5vyu84G5cuUKW7duZf78+bz11lt06dKF3NxcHBwcqF+/PqdPn+batWuMHj2acuXKkZOTg729vWahRURERATIYwvHb2rWrImnpydfffUVAwYM4O2338be3j6/assXkZGRRERE0KxZM9asWUNOTg4ODg4YjUYcHBzw9/enbt26bNq0CQAHBweFZxERERExy1OATk1NZfXq1XTt2pV+/fqxadMmgoKCWLlyJVFRUflc4v8uJyfH/Pf27dtTp04dvLy8uHDhAh9++CEAubm5ANSoUcP83tmzZ61Sr4iIiIjYrv8aoH/66SfGjRtHy5Ytefvtt3F0dOTVV1/Fzs6O8ePH88gjjxRUnX9LcnIyvXr1YsWKFdy8eROAmzdvUrVqVYKDg6lfvz4fffQRqampODo6snfvXmbOnEmpUqUYNmwYPj4+1v0CIiIiImJz/rIHukuXLiQkJODn58fQoUPp1KmTOVC+9957BVXf/8Te3p4SJUrw0UcfkZiYyJQpUwgKCiIiIoKsrCyCg4M5ffo006ZNo2zZshw4cIABAwYU+pVFRERERCT//GWAPnXqFD4+PrRp04agoKBCORtbrlw5hg4dyubNmzl79iwrV66kXr169OnThxIlSlC1alVq167Nv//9b3r16sXHH39s7ZJFRERExMb9ZYD+7rvv2LRpE1FRUSxZsgQPDw86duxIhw4dCs1NdQaDgYCAAI4fP86FCxcIDAxk+vTpeHt788ILL1CuXDkMBgNNmjTB3d3d2uWKiIiISCHwlz3Q5cuXZ/DgwWzevJn169fTrl07Nm/ezIABA8jNzeXjjz/m8uXLBVnr31KmTBmaNm3KrVu3KF++PC+99BInTpxgx44dVKxYkXbt2ik8i4iIiEieGUy/PYovD7Kzs9m+fTtRUVHExcUB0KZNGxYtWpRvBT4IWVlZrF69mpMnTzJz5kwSExOpVKlSvp938dwt3Lp5O9/PIyIiIlLYTHy7F8nJqVY7v6en65/Ob2dnwMOj9H33zdODVH7j6OhIp06d6NSpE1evXiUqKsq8XrItc3Jy4rHHHqNSpUqYTKYCCc8iIiIiUjRZFKD/qHz58jz//PM8//zzD7KefFO9enWqV69u7TJEREREpJCz6EmEIiIiIiLFnUU90CIiIiIiD0JWZg63UjKsdv4C64EWy1y7lobRqN9PioN7/RBK0aXxLl403sWHxlrySi0cIiIiIiIWUIAWEREREbGAArSIiIiIiAUUoEVERERELKAALSIiIiJiAS1jJyIiIiI2Jyszk1spWfl2fC1jZ6OWzZ5Iys1r1i5DREREpNB5deb7QP4F6P+FWjhERERERCygAC0iIiIiYgEFaBERERERCyhAi4iIiIhYQAFaRERERMQCCtAiIiIiIhZQgBYRERERsYACtIiIiIiIBRSgRUREREQsoAAtIiIiImIBBeh7OHz4sLVLEBEREREb5WDtAmzJ+fPnAdi+fTs//vgjjz/+ONWqVbNyVSIiIiJiSzQDDZw6dYpRo0axbt06KlasyODBg3F2diY8PJwDBw4AYDQarVyliIiIiNiCYj0DnZWVxZw5czhz5gwdOnTgqaeeAsDJyYkePXqQk5PDjBkz+Oijj7Cz0+8aIiIiIlLMZ6BjY2P56aefGDlypDk879q1C/g1RPfr1w8HBweioqIAMJlM1ipVRERERGxEsZuBjouL486dOzRo0IBHH32UhIQEjh49Snp6OqtXr6Zs2bI0btwYR0dHHBwcGDx4MHFxcXTv3h2DwWDt8kVERETEyopNgD537hyrVq3iyJEjNG/enHfeeYfo6GgCAwN5//33uXHjBm+++SaNGjW6a7+SJUtSrVo1DAYDJpNJIVpERESkmCsWAfr48eOEhoby+uuvM2nSJACSkpI4ffo0AQEBNG/enIyMDHN4zs7OxtHREYAKFSoQFRWF0WhUH7SIiIiIFO0e6B9++IHU1FR8fX3p3r07ZcuWBWDt2rVs3bqVzz77jEOHDhEcHExmZiYbN24EMIdnADc3N7p164bJZFIPtIiIiIgUzRno/fv3s3jxYm7dukWdOnXo2bMnAwcO5LXXXmPTpk1Uq1aNiIgI9u7dy/Tp0/nwww+pXLkyGRkZ5OTk4ODw+2Vxd3enefPmVvw2IiIiImJLilSATk1NJSwsjMzMTHr27EmHDh0YOXIkJ06c4KmnnqJFixbs37+f6dOnA1C5cmX27t2Lvb09Tz75JM7Ozlb+BiIiIiJi64pMgD558iTJycns37+f5cuX4+3tDYC3tzceHh4A/OMf/2DixIns3buXU6dOsXHjRkJDQyldurT5OLpRUERERET+myIRoPfs2cOlS5fo0qULPXr0ICYmhnr16hEeHk5CQgJly5bFy8uL2rVr0759e/r378/AgQNZvHgx5cuXv+tYCs8iIiIi8t8U2gB97NgxEhMTCQkJAX69YbBHjx6EhIQwZcoUYmNjCQ8P5+LFi+zdu5fnnnuOVq1a8corrxATE4Ovry/w6yO6DQaDgrOIiIiI5EmhDNDXr19n6tSpXLhwge3bt1O1alWqVKlCcnIydevWpXv37pw+fZqqVatStWpVmjVrxqOPPsqdO3eoVKkSgHlVDS1NJyIiIiKWKJTp0d3dnYCAADIzM1m2bBlubm6cPn2aa9eu4eDgQKtWrcjJyWH9+vXmfQIDA3nkkUfMrw0Gg8KziIiIiFisUCTInTt3MmrUKPbv32/eFhwczKhRo9iwYQO3b9+mTp06bNmyhbS0NHx8fGjatCnp6elau1lEREREHiiDyUYTpslkIikpif3795Obm8vo0aMJDg5m3Lhx1K5dm/Xr11OhQgWSk5PZs2cPc+bMISwsjICAANq3b0+5cuWs/RVERERE5G/KyszkVkpWvh3f09OV5OTUu7bZ2Rnw8Cj9F3v8zuYC9B+XkUtISDDf9Pfyyy9z5coVQkNDyczMJDQ0lMmTJ/Pee+8RGhrK2LFjadiwIZ999hknTpxgyJAh5hsFreXatTSMRpu6vJJP7vVDKEWXxrt40XgXHxrr4uV/CdA218LxW3g2mUzY29sTHByM0Whk0KBBJCYm0rZtW3bs2MEXX3xB1apVsbe3Z8iQIRw7dgx3d3cGDx5MmzZtuHHjhto3REREROSBs5kAbTQamTp1KosXLyYjIwODwYCrqysHDhwgMTGRRo0a8dBDD7F582YWLFhAQkICK1euJC0tjW7duvHCCy+Yj9W+fXuCgoK0NJ2IiIiIPHBWD9CXLl1i0qRJzJ07lwMHDnDq1CkiIiI4e/YsHh4eBAQEEBcXB0BYWBhLly4lLS2NSZMmsWbNGkqXLo3RaAQw/1PBWURERETyi1UDdGxsLG+88QYNGjRg8ODBjBw5kpdffhl7e3smT55MYmIi7u7u1KhRA4AKFSrQr18/Dh8+DEBQUBCAeTk6LUsnIiIiIvnNqg9SOXjwIF27dqVnz57ArzPIKSkpDB8+HIPBQHR0NDExMeTm5tK0aVMARo4cac2SRURERKSYs2qArl69OmFhYSQkJPDDDz+QlZWFh4cHrq6uhIeHc/z4cTZv3ozRaCQnJwcHh9/L/eNqHSIiIiIiBcXqy9jt3r2b9PR07O3tadOmDQC9evUiPDycypUrc/78eby8vKxZooiIiIhYKCcrhxu3Mqxdxl/6X5axs+oMNEDjxo1JSkrC29sbgBUrVlCnTh3c3NwAzOHZaDQWuh7nw6t/IDv1jrXLEBERESlwAcNaW7uEfGP1AJ2RkcHKlStJSkoiOTkZPz8/Ro4cibOz812fK2zhWURERESKJqsHaDc3N0aNGsWhQ4dwcXHB398fKJwzziIiIiJS9Fk9QAOUKVOG4OBg82uFZxERERGxVTaZUhWeRURERMRWKamKiIiIiFhAAVpERERExAIK0CIiIiIiFlCAFhERERGxgAK0iIiIiIgFFKBFRERERCygAH0Pd+78/vhtk8lkxUpERERExNYoQP+ByWTi6tWrzJ07l6ioKNLS0jAYDNYuS0RERERsiAL0/zt69CiLFy/GxcWF119/nXPnzvHOO++Qmppq7dJERERExIYoQP+/1NRU0tLS2LZtGyVKlGDEiBFkZmbyySefkJSUZO3yRERERMRGFNsAfeXKFSIiIkhMTASgcePG1KtXj6NHj3Ly5Ens7Ozo06cPV65c4ejRo1auVkRERERsRbEL0EajkevXr3PlyhXmzZtH9+7dWb16NUeOHKF58+aUKVOGuLg4AAIDA3FxceHChQuAbigUERERkWIYoOfNm8enn35K9erVGT9+PNWqVaNs2bKMGzeOVatWYTQauXz5MocPHwagZcuWbN26FUA3FIqIiIhI8QnQubm5ALRq1YozZ85w4sQJOnbsiIuLCw0bNmT9+vW4u7uze/du1q5dy9atW8nKyiIwMJCHH36Y27dvW/kbiIiIiIgtKNIBOj09ndjYWOD32eOHH36YChUq8P333+Pk5ES3bt2YPXs2pUuXZtCgQbz//vsMHTqU5s2b4+TkxPnz57l9+zalSpWy5lcRERERERtRZAP0iRMn2L9/P6tWreLcuXPY2dmRnZ0NQI8ePTh79iwHDhygffv2ODo6snr1agBcXFwYMWIEDz/8MABeXl689tprVvseIiIiImJbimSA3rNnD4cOHaJJkyY0atSIDRs2AODo6Eh2djbe3t4EBgbyww8/cOvWLfr06cOtW7fuOoZuGBQRERGReykyAfrYsWPmdg2A3bt34+TkRJs2bbh48SI//vgj8GuIBujYsSMXL17k1KlTNG/enBEjRtx1PN0wKCIiIiL3UiQC9PXr15k6dSqTJ08mKyuLqlWrUqVKFa5evUrt2rVp0qQJn3/+OfBrX/S4cePYu3cvr7/+OiEhIdjZ/XoZjEajNb+GiIiIiBQCRSJAu7u7ExAQQGZmJsuWLcPNzY3Tp09z9epVnJycaN68OSaTiaFDhzJ06FA8PT157LHHqFatGvB7u8ZvQVpERERE5K8UysS4c+dORo0axf79+83bgoODGTVqFBs2bOD27dvUqVOHLVu2kJaWho+PD02aNMHJyYkZM2YwduxYHBwczMFZ7RoiIiIikleFJkCbTCYSExPZtm0baWlpfPnll8yfP5+jR4+Sm5vLhQsX8PT0ZMiQIcyYMYPnn3+eq1evEhMTw40bNwgNDWXhwoV4eXlhNBoxmUwKziIiIiJiMZsP0H+cJU5PT2fBggV06tSJdu3acfv2bfbt28fq1avp3LkzmzZtonfv3iQkJLB7927Gjh1LWloac+bMISEhwXw8Ozs7hWcRERER+VscrF3A/fwWdE0mE/b29gQHB2M0Ghk0aBCjRo2ibdu2jB07lhIlSlC1alXs7e0ZMmQIx44dIyQkhMGDB7N161Zu3LhR4LPO9fs/UmDnEhEREbElOVk51i4h39hsgDYajUybNo3y5cszaNAgnJ2dcXV15cCBAyQmJtKoUSMeeughNm/ezIIFC1iyZAlr1qxhxIgRdOvW7a5jtW/f3iozzteupWE0aj3p4sDT05Xk5FRrlyEFRONdvGi8iw+NteSVzbVwXLp0iUmTJjF37lwOHDjAqVOniIiI4OzZs3h4eBAQEEBcXBwAYWFhLF26lLS0NCZNmsSaNWsoXbq0eTm63/6pdg0REREReVBsKkDHxsbyxhtv0KBBAwYPHszIkSN5+eWXsbe3Z/LkySQmJuLu7k6NGjUAqFChAv369ePw4cMABAUFAb8vR6dl6URERETkQbOpFo6DBw/StWtXevbsCfw6g5ySksLw4cMxGAxER0cTExNDbm4uTZs2BWDkyJHWLFlEREREihmbCtDVq1cnLCyMhIQEfvjhB7KysvDw8MDV1ZXw8HCOHz/O5s2bMRqN5OTk4ODwe/lalk5ERERECoJNBegnnngCd3d30tPTCQoKok2bNgD06tWLy5cvU6dOHZYsWYKXl9ef9lV4FhEREZGCYFMBGqBx48YkJSXh7e0NwIoVK6hTpw5ubm4A5vBsNBptvsfZw6O0tUuQAuTp6WrtEqQAabyLF4138aGxzj9ZWdncunXH2mU8EDYXoDMyMli5ciVJSUkkJyfj5+fHyJEjcXZ2vutzth6eASIiIkhJSbF2GSIiIiJWN2bMGEABOl+4ubkxatQoDh06hIuLC/7+/kDhmHEWERERkaLP5gI0QJkyZQgODja/VngWEREREVtRKFKpwrOIiIiI2AolUxERERERCyhAi4iIiIhYQAFaRERERMQCCtAiIiIiIhZQgBYRERERsYACtIiIiIiIBRSgRUREREQsoAAtIiIiImIBBeh7SEpKIiMjAwCTyWTlakRERETEltjko7ytJSkpiatXr7Jv3z4uXrzIyy+/TKlSpaxdloiIiIjYEM1A/7+EhARefvllqlSpwrPPPouzszPvv/8+O3fuBDQTLSIiIiK/UoD+fy4uLvj4+ODk5ATAwIEDqVmzJh9++CG3b9/GYDBYuUIRERERsQXFNkAfOXKEt99+m+3btwNw48YNEhMTcXFxAaBs2bJ06tQJHx8fPv74Y2uWKiIiIiI2pNgGaEdHR2rUqMGKFSv4xz/+wfHjx2nWrBkAOTk5ADg5OdG2bVtSUlLMNxWKiIiISPFWLAK00WgkJyeH1atXm7f5+vryzDPPsHr1arp37050dDRxcXEAODg4mHueHRwcyM7OxtnZWX3QIiIiIlI8ArSdnR0ODg4sX76c6Oho4NdQbTKZsLe3p0ePHsyfP58qVaoQExMDQG5uLgCNGzfmypUr3Lx5U33QIiIiIlK0A/Tu3btZt24du3btAuCdd95h6dKlGI1G7O3tMRqNGI1GAMqUKUOHDh14//33yc3NxcHh1xX+rl69SpcuXbScnYiIiIgARTRAX7t2jfHjx7NixQocHBxYv349Y8aMoU6dOjRs2JAFCxYAYG9vj52dHTdv3iQrK4uOHTsyZ84c7O3tze0alStXJiQkxLw6h4iIiIgUb0UyQG/fvp1atWoRERFB7969WbhwIXXr1iUsLIwJEybw+eefm28UfOeddwgLCyMxMRGAevXqAahdQ0RERETuqcgE6EuXLpGdnQ1AfHw8tWvXBn5fUeOFF14gKSmJnJwcnnjiCbp168aAAQPIyMhg6tSpeHt7W612ERERESk8Cv2jvPfv38+CBQtwdnbGw8ODN998E4BatWoBv95AmJOTQ2pqKmXKlCE7O5uXX36Zq1ev8uyzz1K/fn3g15sK7eyKzO8TIiIiIpJPCm1iTE1NZcyYMSxdupQ+ffqwaNEitm/fzunTp/Hw8GDlypXA7ytw3LlzBzc3N5ydnXF0dGTGjBnUr18fk8mk8CwiIiIieVYoZ6BPnjxJcnIy+/fvZ/ny5eb2i/bt2+Pk5MTQoUMZM2YMb7/9Nm3btuXo0aNs2bKFp59+mjJlypiP81twVr+ziIiIiORVoQvQe/bs4dKlS3Tp0oUePXoQExND/fr1Wb16NTVq1MDDw4NSpUoxbdo0fvrpJ3744Qdu3rzJkiVL8PT0vOtYmnUWEREREUsVigB97NgxEhMTCQkJAeCHH36gR48ehISEMGXKFL755htmz55NjRo1zPtUqVKFKlWqkJOTY17T2Wg0YjAYNOMsIiIiIn+bzQfo69evM3XqVC5cuMD27dupWrUqVapUITk5mbp16xIaGsqpU6fM4fmPgRm4KzwX9IzzkCFDCvR8IiIiIrYqKyvb2iU8MDYfoN3d3QkICCAhIYFly5YxYMAATp8+zbVr1/D09KRly5YcP36cyMhIevfufVd4/iNrtGtcu5aG0Wgq8PNKwfP0dCU5OdXaZUgB0XgXLxrv4kNjLXllcwF6586dfPrppwwaNAh/f38AgoOD8fb2JiIigl69elGnTh22bNlCtWrV8PHxoWnTply9ehWTyWRT7Rl2drZTi+Q/jXfxovEuXjTexYfGunj5z/HO6/gbTL89s9qKTCYTSUlJ7N+/n9zcXEaPHk1wcDDjxo2jdu3arF+/ngoVKpCcnMyePXuYM2cOYWFhBAQE0L59e8qVK2ftryAiIiIixYRVl6H4LbsbDAbS09NZsGABnTp1ol27dty+fZt9+/axevVqOnfuzKZNm+jduzcJCQns3r2bsWPHkpaWxuzZs0lISLDm1xARERGRYsSqAfq3dguTyYS9vT3BwcEYjUYGDRpEYmIibdu2ZceOHXzxxRdUrVoVe3t7hgwZwrFjx3B3d2fw4MG0adOGGzduYAMT6SIiIiJSDFilhcNoNDJt2jTKly/PoEGDcHZ25tq1awwbNoz58+dTpUoVhg0bRuPGjenZsydLlixhzZo1xMfHU7p06buOZWt9zyIiIiJStBVogL506RJLlizB1dWVvXv34uPjg4+PD927d8fHx4cZM2ZQq1YtevfuzZUrV+jSpQsbNmzA29ub+Ph4goKCzMvR6fHbIiIiImINBZZAY2NjeeONN2jQoAGDBw9m5MiRvPzyy9jb2zN58mQSExNxd3c3r+dcoUIF+vXrx+HDhwEICgr6teD/D80KzyIiIiJiDQW2jN3Bgwfp2rUrPXv2BH5t40hJSWH48OEYDAaio6OJiYkhNzeXpk2bAjBy5MiCKk9EREREJE8KLEBXr16dsLAwEhIS+OGHH8jKysLDwwNXV1fCw8M5fvw4mzdvxmg0/ulpgupzFhERERFbUaA90Lt37yY9PR17e3vatGkDQK9evQgPD6dy5cqcP38eLy+vgipHRERERMRiBfokwsaNG5OUlIS3tzcAK1asoE6dOri5uQGYw7NuEBQRERERW1WgATojI4OVK1eSlJREcnIyfn5+jBw5Emdn57s+V5jDs9FoJCwsjGPHjuHk5MT06dPx8fGxdlnyAGVnZzNx4kQuXrxIVlYWL730ErVq1WL8+PEYDAZq167Nm2++Waj/PZa7Xbt2jZ49e7JixQocHBw01kXY+++/z9dff012djZ9+/alWbNmGu8iKjs7m/Hjx3Px4kXs7OyYNm2afr6LqF9++YV33nmH1atXc/bs2XuOcWRkJB9//DEODg689NJL5k6Jv1Kg/1a4ubkxatQonn32WSZNmsTUqVOpWLEiRqOxIMvIV9u3bycrK4v169czZswYZs2aZe2S5AGLjo7Gzc2NdevWERERwbRp05g5cyajRo1i3bp1mEwmduzYYe0y5QHJzs5mypQplCxZEkBjXYTt2bOHffv28dFHH7F69WoSExM13kVYbGwsOTk5fPzxxwwfPpz33ntP410ERUREMGnSJDIzM4F7/zc8OTmZ1atX8/HHH7N8+XLeffddsrKy/utxC/zXqjJlyhAcHIy/vz9Q9No1fvrpJ1q2bAlAYGAgBw8etHJF8qB17NjxrhVi7O3tOXToEM2aNQOgVatWfP/999YqTx6w2bNn8/TTT1OhQgUAjXURFhcXR506dRg+fDhDhw6ldevWGu8irEaNGuTm5mI0GklLS8PBwUHjXQR5e3sTHh5ufn2vMd6/fz+NGjXCyckJV1dXvL29OXr06H89rtWTa1EKzwBpaWl3PS3R3t6enJwcK1YkD5qLiwulS5cmLS2NV155hVGjRt21UoyLiwupqalWrlIehI0bN+Lu7m7+pRjQWBdhN27c4ODBgyxYsICpU6cyduxYjXcRVqpUKS5evEinTp2YPHky/fv313gXQR06dPjLld1+G+O0tDRcXV3Nn3FxcSEtLe2/HrdAe6CLg9KlS5Oenm5+bTQa7xo4KRouX77M8OHDeeaZZ+jatStz5841v5eenk6ZMmWsWJ08KJ9++ikGg4Hdu3dz5MgRxo0bx/Xr183va6yLFjc3N2rWrImTkxM1a9akRIkSJCYmmt/XeBctK1eupEWLFowZM4bLly8zcOBAsrOzze9rvIumP07c/jbG/5nd0tPT7wrU9zxOvlVYTDVu3JjvvvsOgJ9//pk6depYuSJ50K5evcqgQYN47bXX6NWrFwD169dnz549AHz33XfmJ2dK4bZ27VrWrFnD6tWr8fPzY/bs2bRq1UpjXUQ1adKEnTt3YjKZSEpKIiMjg+DgYI13EVWmTBlzSCpbtiw5OTn6b3kxcK8x9vf356effiIzM5PU1FQSEhLum98KdB3o4uC3VTiOHz+OyWRixowZ+Pr6WrsseYCmT5/OF198Qc2aNc3b3njjDaZPn052djY1a9Zk+vTp2NvbW7FKedD69+9PWFgYdnZ2TJ48WWNdRM2ZM4c9e/ZgMpkYPXo01apV03gXUenp6UycOJHk5GSys7MZMGAADRo00HgXQRcuXODVV18lMjKS06dP33OMIyMjWb9+PSaTiRdffJEOHTr812MqQIuIiIiIWEAtHCIiIiIiFlCAFhERERGxgAK0iIiIiIgFFKBFRERERCygAC0iIiIiYgEFaBGRQigrK4vly5cTGhpKo0aNaN68OUOHDuXAgQPAr8s21a1bl/j4+HyvJTw8nHbt2plf79y5k7Zt29KwYUNWrVpF27ZtWbJkSb7XISJSULSMnYhIIZORkcGAAQO4ceMGr7zyCgEBAaSnp7Nq1Sq2bNnC0qVLqVatGo899hhr167N94dBpKenk5mZibu7OwBPPvkkbm5uTJ06FTc3N7KysihZsiSlSpXK1zpERAqKnjEtIlLIvPfee5w5c4bPP/+cihUrmrfPmjWLa9euMW3aNP71r38VWD0uLi64uLiYX6emphISEkK1atUKrAYRkYKkFg4RkUIkKyuLjRs30qtXr7vC82+mTJnCvHnzMBgMd22/efMmEyZMoEWLFjz00EO0aNGC2bNnYzQagV8fUT9ixAgefvhhAgMD+cc//sGRI0fM+2/cuJFOnTrRoEED2rRpw8KFC837/rGFo27dupw9e5bFixdTt25dgD+1cGzfvp1u3brRsGFDOnbsyPLly83H+q315F//+hfBwcF06tSJrKwsli5dymOPPUaDBg3o0KEDa9eufYBXVUTEMpqBFhEpRM6fP09KSgoBAQH3fN/Lywv4NYj+0bhx47hx4wb//Oc/cXNz47vvvmPatGk0adKExx9/nKlTp5KTk8O6deswGAzMmzePl19+me3bt3P06FGmTJnCu+++S4MGDTh06BBjx47F29ub0NDQu84TFxdHnz596NChA4MGDfpTfbGxsYwdO5ZJkybRrFkzTpw4wVtvvUVGRgYjRowwfy4mJoY1a9Zw584d4uLiWL58Oe+99x7e3t58//33TJ48mTp16tC0adP/8YqKiFhOAVpEpBBJSUkBoEyZMhbt17JlSx5++GFq164NQL9+/Vi2bBnHjh3j8ccf5+zZs9StW5dq1apRokQJ3nrrLU6ePInRaOT8+fMYDAaqVKli/vPBBx9QqVKlP53H09MTe3t7SpUqhaen55/e/9e//kXfvn3p1asXAN7e3qSnpzN58mSGDRtm/ly/fv3w9fUFYO/evTg6OlKlShWqVq3KU089RbVq1ahZs6ZF10BE5EFRgBYRKUTKlSsH/NqSYYm+ffuyY8cOPvnkE86cOcOxY8dITEw0t04MGzaMcePGsW3bNpo2bUqrVq0IDQ3Fzs6Oli1bEhAQwJNPPomPjw8tWrSgc+fOVKlSxeL6jxw5woEDB/j444/N24xGI3fu3OHixYvm1pPfZtIBunbtyoYNG2jfvj116tShRYsWdOvWDQ8PD4vPLyLyIKgHWkSkEPH29sbDw4Nffvnlnu/v2bOHoUOHkpycbN5mMpl44YUXmDVrFs7OznTv3p01a9ZQtWpV82c6duzIzp07mT59Op6enixZsoTQ0FCuXr1KyZIlWbNmDRs2bKB79+4cPnyYZ599loiICIvrd3R0ZOjQoURFRZn/REdHs23btrt6ukuUKGH+u4eHB9HR0axZs4a2bduye/dunnzySTZv3mzx+UVEHgQFaBGRQsTOzo4ePXrw6aefkpSUdNd7JpOJpUuXcvr0acqXL2/efvLkSeLi4ggPD2f06NE88cQTlCtXjuTkZEwmEzk5OcyePZuLFy/StWtXZs6cSUxMDBcvXuTHH39k165dLF68mIYNGzJ8+HA+/vhjnn76aT777DOL669VqxZnzpzBx8fH/Of48ePMnz//L/fZsmULH330EU2bNmX06NFERUXx6KOPEh0dbfH5RUQeBLVwiIgUMsOGDWPXrl0888wzjB49moCAAK5evcqKFSvYu3cvK1asuGsVjjJlyuDg4MAXX3xB2bJlSU5OZv78+WRlZZGVlYWDgwOHDh0iPj6eSZMm4e7uzubNm3F0dOShhx4iKSmJxYsX4+rqSps2bbh69Sp79uwhMDDQ4tpfeuklXnzxRerUqUP79u05c+YMU6ZMISQkBCcnp3vuk5WVxezZs3F1daVJkyacO3eOw4cP07dv3797CUVE/icK0CIihYyLiwtr1qwhIiKCRYsWcfnyZVxdXQkICGD9+vX4+fndtQpHxYoVmTFjBuHh4Xz44YdUrFiRTp06UbFiRfOTC+fNm8eMGTN48cUXSU9Pp3bt2ixevNg8SzxjxgyWLVvGO++8Q+nSpXn88cd5/fXXLa69VatWzJkzh6VLl7Jw4ULc3d0JDQ1l9OjRf7lPaGgo165dIzw8nMuXL+Ph4UHPnj0ZOnSo5RdPROQB0JMIRUREREQsoB5oERERERELKECLiIiIiFhAAVpERERExAIK0CIiIiIiFlCAFhERERGxgAK0iIiIiIgFFKBFRERERCygAC0iIiIiYgEFaBERERERC/wfgxuB3vRZghQAAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 720x360 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "fig, ax = plt.subplots(figsize=(10,5))\n",
    "ax.set_xlabel('Classifiers',fontsize = 15) #xlabel\n",
    "ax.set_ylabel('Accuracy', fontsize = 15) #ylabel\n",
    "\n",
    "plt.yticks(rotation=30, size = 10)\n",
    "plt.xticks(rotation=0, size = 10)\n",
    "\n",
    "ax = sn.barplot(x=acc, y=classifiers)\n",
    "ax.set_title(\"Accuracy of all classifiers\", fontsize= 15)\n",
    "\n",
    "\n",
    "plt.tight_layout()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## __F1-score Bar plot__"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAtEAAAFpCAYAAABauHSCAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8rg+JYAAAACXBIWXMAAAsTAAALEwEAmpwYAABIaElEQVR4nO3deVxU9f7H8dewI+AC4g5oKGqp4IJe3Mh9KRXNtE0tTTN3y5tZapp7aaak3cQtlzbN3CvLbl4xNPVamru4ayAoyqYMw8zvj35OcStjDJkB3s/Hw0dyZs45nzlfqDffPud7DBaLxYKIiIiIiOSZk70LEBEREREpbBSiRURERERspBAtIiIiImIjhWgRERERERspRIuIiIiI2EghWkRERETERgrRIlJs9OnTh5o1a/7hn1WrVv3u/e+99x5PP/10wRdqJ1OnTqV+/fo0bNiQ5ORkm/dv3bo1CxcuBCA6Opp27drdVR1/Z19b/bZmyH0Ntm7dSs2aNUlISCiQWkSkcHGxdwEiIgXp4Ycf5uWXX/7ddm9v71xff/zxx7z99ts0adKkoEqzq5MnT7Jy5UomT55M8+bNKVu2rL1LKhBr167Fw8MD+P01KFeuHLGxsfj5+dm5ShFxRArRIlKseHh44O/v/6evX7t2jddee42dO3cSFBRUgJXZV2pqKgDNmjWjSpUqdq6m4Pj6+lr//kfX4E7fKyJSvKmdQ0TkN+Lj4wFYv349YWFhedonOTmZYcOG0aRJE8LCwnj66ac5evSo9fX09HQmT55M06ZNqV+/PgMGDOD06dPW17dv306PHj0IDQ3lwQcfJDo6GpPJBMCePXuoW7cuCxcupHHjxvTp0weAEydOMGDAAEJDQ2nZsiUTJ060hsA/YjKZiImJoX379tStW5cuXbqwdetWANatW8cTTzwBQNu2bf9wpv52LU899RT169enTp06dOvWjf/85z95ukb/66+uiS3n/fbbb4mKiqJevXo0b96cKVOmkJWVBUBmZibjxo2jadOm1K1bl169ehEXF2fd93Y7xx9dgz179uRq5zAajcycOZPmzZvToEEDnnrqKX744QfrsaKjo+nTpw8jRoygQYMGzJ079y+/N0Sk8FKIFhH5jfDwcKKjo6latWqe95k8eTImk4kPPviAdevW4eXlxfDhw62vjxo1iri4OObMmcOnn35KiRIlePbZZ8nOzmbbtm0MHz6cTp06sX79el566SVWrlzJjBkzrPsbjUb27NnDmjVrGD9+PImJifTp04eQkBA+++wz5s+fz6lTpxg2bNif1jhz5kyWLFnCCy+8wMaNG3nooYd44YUX+PLLL+ncubO1L3jNmjW8+uqrv9v/559/ZuDAgTRs2JCNGzeydu1aKlasyNixYzEajXm+Vnm5Jrac99q1awwbNozHHnuMzz//nDfffJOtW7cSExMDYL02S5YsYevWrdSuXZuhQ4eSmZmZ6zx5uQYvvfQSe/fu5e233+bTTz/lH//4B3379uXMmTPW93z//fcEBATw2Wef0bNnz7/83hCRwkvtHCJSrKxfv946A3tb586dmTZt2l0f89y5c9SsWZMqVarg7u7O66+/zqlTpzCbzZw9e5adO3eyYsUKa3/166+/znvvvcf169dZtGgRnTp1YuDAgQBUq1aN69evM23aNEaNGmU9x7PPPmttL5k7dy5VqlRh7Nix1tfnzp1Ly5YtOXDgAPXr189VX3p6Oh9++CETJ06kY8eOAAwePJhjx46xaNEiOnToQKlSpYBf2ht8fHx+9xmzs7MZOXIk/fv3x2AwAPD000/Tr18/rl69SsWKFfN8vU6fPn3Ha2LLeVNSUsjOzqZChQpUrlyZypUrs3jxYkqUKGEdGy8vL6pUqYKPjw9jx46lQ4cOODs75zqPh4fHHa/BuXPn+Pzzz9m8eTM1atQAYNiwYezfv59ly5bx+uuvA2AwGBg+fLi1z/pO3xtOTprHEinMFKJFpFhp27YtL7zwQq5tXl5eed7/fwPqgQMHGDJkCGPHjmXbtm2Eh4fTsmVLoqKicHJy4sSJEwDUq1fPuk+ZMmWsLRMnT56ke/fuuY4ZHh6OyWTK1d4QEBBg/fvRo0c5evTo72qBX9pR/nf76dOnMZlMNGjQ4Hfn+eabb/L0uQMDA4mKiuL999/n+PHjnDt3ztqWkJOTk6dj3PZX18SW89auXZtOnTrx3HPPUaFCBZo1a0a7du1o1aoVAAMGDGDIkCFERERQv359WrRoQbdu3XB3d7ep5iNHjgDQq1evXNuNRmOumXh/f39rgAbu+L0hIoWbQrSIFCve3t5/64bB9evX/25bx44dadq0KTt27OC7775j4cKFLF++nE8++QQXlzv/a/a3geu226H0t/v+9n2urq40a9aM8ePH/27f394od5ubm9sfnjsnJ+cv67vt5MmTPPHEE4SGhhIREUHnzp0xmUwMHjw4T/v/Vl7PmZfzGgwG3n77bYYNG8aOHTuIjY1l6NCh9OrVi0mTJtGoUSPr9tjYWFavXs3SpUtZtWoV1atXz3Mdrq6uAHz00Ue/G7PfXt//fe1O3xvFZQUUkaJKvwqLiNggKCgo1x+TycSsWbO4dOkSXbp0YcaMGWzZsoVLly7x/fffExwcDMBPP/1kPUZ6ejoRERHs27eP4OBg9u/fn+sc+/fvx9XVlcDAwD+soXr16sTHx1OpUiVrHU5OTkyfPp2ff/75d++vWrUqrq6uf3ievAbJdevWUbFiRRYvXsyAAQNo0aIFiYmJAFgsljwd47a/uia2nPfQoUPMmDGD6tWrM2DAAJYtW8bo0aP57LPPAHjnnXf473//S7t27Zg8eTLbtm3DbDbz73//26aab7dwXL16Ndf4L1++nO3bt//hPn/1vSEihZtCtIjI3+Di4sLhw4eZOHEiP/74IxcuXODjjz/G1dWVBx54gGrVqtGmTRsmT57Mvn37iI+PZ9y4cfj4+FCvXj2ef/55Pv/8c2JiYjh79iyff/458+fP59FHH/3D3mSAp556itTUVF5++WWOHz/OoUOHeOGFFzh79uwf3hDp4eHBM888w9tvv80XX3zB2bNnWbRoEdu2beOZZ57J0+f09fXl0qVL7Nq1i0uXLrFhwwbmzp0LYPONhX91TWw5r4+PD6tXr+att97i/PnzHD16lH//+9/W41y6dInJkyezZ88eLl26xMaNG0lLSyM0NNSmmoOCgujcuTMTJkxgx44dnD9/nrlz5/LRRx9Zfyn4X3/1vSEihZvaOURE/qY5c+Ywffp0nnvuOTIyMqhRowYLFiywto3MnDmTGTNmMGTIEHJycggPD2fx4sW4ubnRokULZs2axXvvvce8efMoV64cffv25bnnnvvT8/n7+7Ns2TJmz55Nr1698PDwoEmTJsybN+9PWzdGjhxpna1OSUkhODiYt956i06dOuXpM/bt25f4+HhGjx5NTk4OwcHBTJ48mXHjxnHo0KE/DZJ/5k7XxJbzRkVFsWDBAubPn8+KFStwdXWlRYsWjBs3DoDx48cza9YsXnzxRa5fv05QUBAzZsygcePGNtULvzzNcM6cObzyyiukpaURHBxMdHQ0ERERf7rPX31viEjhZbDY+v/hRERERESKObVziIiIiIjYSCFaRERERMRGCtEiIiIiIjZSiBYRERERsZFCtIiIiIiIjRSiRURERERspHWi76GUlAzMZq0gWBz4+Xlz9Wq6vcuQAqLxLn405sWLxrt4uT3eTk4GypTxyvN+CtH3kNlsUYguRjTWxYvGu/jRmBcvGu/i5W7GWyH6HvLz87Z3CVKA/P3/+BHNd3IrK5u01Fv3oBoRERG5lxSi76ERM9aTnJJh7zLEgX3wxpOkoRAtIiJS2OjGQhERERERGylEi4iIiIjYSCFaRERERMRGCtEiIiIiIjZSiBYRERERsZFCtIiIiIiIjRSiRURERERspBAtIiIiImIjhWgRERERERspRIuIiIiI2Egh+k+YzWZ7lyAiIiIiDkoh+k84Of16aSwWix0rERERERFHoxD9G78Ny2lpaUydOpWcnBwMBoMdqxIRERERR6MQza/h2WAwYDKZsFgs5OTkYLFYuHHjhp2rExERERFHU6xD9G/DM8D+/fuZOXMmu3btws3NjdOnT+Pl5WXPEkVERETEARXLEG2xWLBYLNbwvHXrVubNm0eZMmUIDw/n3Xff5cKFC1SuXJmdO3fauVoRERERcTQu9i6goJnNZutNg5cvX+a///0vu3btIisri82bN9O/f388PDxYsWIF8fHxPPTQQwC5QreIiIiIFG/FLkQ7OTlx8+ZNFi9ezHfffYe/vz+vvvoqXl5ezJ49m127dtGhQwfCwsIYNWoU27ZtIyIiQgFaRERERKyKfDvH/673fPjwYYYNG4aXlxdvv/02JpOJn376CW9vb0JDQ9m1axdHjhyhVKlSTJgwgbJly9qpchERERFxVEV2Jvr2TYO3Wzfi4uIwmUxERERgMpnw9PSkfPnytGrVil27dlGnTh3atWuHyWQiMDCQtLQ05s6dS3h4uD0/hoiIiIg4oCIZonNycnB2dgYgMzOTpUuXsm/fPm7cuEHfvn0ZOnQoK1asoHv37kRFRfHvf/+bH374gQ4dOvDoo48CkJWVxUsvvURAQIA9P4qIiIiIOKAiGaKdnZ3Jyspi3rx5pKenk52dzfLlyzl16hSDBg3iq6++4pNPPmHFihUMGjSI8ePHU6lSJev+FosFd3d3BWgRERER+UNFsif65MmTPPPMM5QrV46uXbuyZcsWzp8/T/Xq1QkNDWXGjBmMGDECDw8PAGuA/t91o0VERERE/kihDtGZmZkAmEwmAC5evMjGjRtxd3fn/PnzhIWF0ahRI5555hlmz54NwMSJE0lKSqJy5cr07ds31/EUnkVEREQkLwptiN66dSvPPfdcrm0HDx5k3759pKamMmzYMJYuXQrAiBEj2LNnD99//z1lypRh3rx51p7p27PPIiIiIiJ5VehCdE5ODgCdO3fmxo0bTJo0iQ8//BCAf/zjHwQHB7Njxw5atWqF0Wjkk08+wdnZmRUrVtC4cePfHUezzyIiIiJiq0IXom/PIO/Zs4fq1avz5Zdf8s0333D06FF8fX257777OH36NEePHuXxxx/n+vXrANSsWRP4deb59nFERERERGxV6FbnMJlMTJkyhevXr/PYY49hMpn4/vvv+eqrr6hduzaenp4YjUYuX77Mo48+SmRkZK79NfMsIiIiIn9XoQvRLi4uuLm58cwzzxAWFkZAQAB9+/bl0KFDjBgxgp9//pkxY8bQpEkT6z5ms9n60JWCNH9cVIGfUwqXW1nZ9i5BRERE7kKhC9E5OTkYDAYSExMBqFChAn5+flSoUIGnn36a4OBg63stFgsGg8EuARrg6tV0zGbduFgc+Pv7kJSUZu8yREREpIAUuhDt7OxM586d2bBhA99//z0///wznTt3pkuXLpQtWxb49YmFat0QERERkXuh0IVogLCwMAIDA9m6dSsPPfQQDRo0AH5t29BNgyIiIiJyLxXKEA3g6+vLU089Zf3aYrHYrW1DRERERIqXQp869ahuERERESlohT5EKzyLiIiISEEzWPTcaxEREREpACZjFik3jPYuI5fbK2w5ORnw8/PO836Ftie6MDj0r7EYU6/auwwRERERh9DwpcWAY4Xou1Xo2zlERERERAqaQrSIiIiIiI0UokVEREREbKQQLSIiIiJiI4VoEREREREbKUSLiIiIiNhIIVpERERExEYK0SIiIiIiNlKIFhERERGxkUK0iIiIiIiNFKJFRERERGykEP0HTp48idlsBsBisdi5GhERERFxNC72LsCRfPfddyxatIgSJUrg6enJqFGjCAgIsHdZIiIiIuJgFKKBjIwMxo4dS2pqKmPGjKFu3brMnz+fw4cPK0SLiIiIyO8U63YOo9HIiRMnyMnJwcfHh5YtW1KvXj3S0tLYuXMnqampHDt2DMDa3iEiIiIiUqxD9I4dOxg/fjxms5mePXty9OhR3nrrLcaOHUtwcDDXr1+nR48eWCwWnJyK9aUSERERkd8odu0c58+fJzAwEIB27dqxefNmtmzZQo8ePahWrRpr167l/fffJygoCIDdu3dz+PBh6tSpY8+yRURERMSBFKvp1W+++YYnnniCKVOmEBcXB8DQoUPZuHEjycnJdOzYkebNm5OdnU1aWhqvvvoqHh4e1tAtIiIiIgLFYCY6JyeHxYsXU7ZsWc6fP0/79u05f/48U6dOpUGDBvTv359atWqxefNm+vXrR8OGDRkwYAABAQF06dKF3r172/sjiIiIiIiDKdIhesuWLXz88cfUrFmT6tWrk5mZSbly5QgODqZatWokJiayaNEi4uPj2b59Ow0bNqRFixakpqbStWtXypQpY++PICIiIiIOqMiG6NTUVHbs2MGYMWOoV68eAJcvX2b+/PmUKFECV1dXevXqRffu3fnkk09YuXIl69evZ/r06fTr18/O1YuIiIiIIyuyIfqrr77CYrFQr149jEYjc+fO5cSJE2RkZODq6kqZMmWIi4sjIiKCXr160bNnT63AISIiIiJ5UmRTY7Vq1fDx8SE9PR03NzdatmzJkiVLmDp1KvHx8Vy5coXjx49z69YtAAVoEREREcmzIjsT7efnR4kSJdi3bx8PPvggERERAPzwww/069ePyMhIqlSpgoeHh50rFREREZHCpsiG6MDAQIKDg9mwYQPXrl2jRo0arFixguvXr/PCCy9Qq1Yte5coIiIiIoVUkQ3RBoOB7t274+fnx6FDh9i1axeRkZF07drV3qWJiIiISCFXZEP0bS1btqRly5aYzWb1PYuIiIhIvjBYLBaLvYsQERERkaLPZMwi5YbR3mXk4u/vQ1JSGk5OBvz8vPO8X5Gfibanq1fTMZv1O0pxcPsHUIoHjXfxozEvXjTekhfqbxARERERsZFCtIiIiIiIjRSiRURERERspBAtIiIiImIjhWgRERERERspRIuIiIiI2EjrRIuIiIg4kCyjkdQbWfYuo9jQOtEOaMyaySSnX7N3GSIiIlKILH9mHqAQ7ejUziEiIiIiYiOFaBERERERGylEi4iIiIjYSCFaRERERMRGCtEiIiIiIjZSiBYRERERsZFCtIiIiIiIjRSiRURERERspBAtIiIiImIjhWgRERERERspRP+JAwcOcO2aHtktIiIiIr+nEP0bFosFgISEBKKjozl06BA5OTl2rkpEREREHI1C9P+7dOkSBoMBgAoVKtCqVSt27dpFYmKinSsTEREREUejEA0kJSXRv39/evXqxYcffghA7969uX79Ovv37+fmzZt2rlBEREREHEmxDtH79+/nlVdewd/fn549e1KiRAnWrFnDiy++yOeff05YWBhHjhzh4sWL9i5VRERERByIi70LsKewsDCGDh3KwYMH6dSpE2azGV9fXxo0aMDq1as5efIk8fHxuLq6MnToUNzd3e1dsoiIiIg4gGI1E52cnMzSpUuJj4/HZDLh7OzM8OHDmTZtGlWqVKFChQocO3YMb29vJk6cyMiRI2ndujW7du0iKyvL3uWLiIiIiIMoNjPRq1evZs2aNSQlJbFx40Zq1qzJCy+8wJNPPsnWrVvZsGEDbdu25ezZs2zZsoX+/fvTqFEjGjVqZO/SRURERMTBFIuZ6Fu3bvHDDz8wZ84coqKiiIqKwt3dnZkzZzJr1iwiIyNZvHgxJUqUoF69epw+fZoLFy7Yu2wRERERcVBFNkRfu3aNd955B4vFwo0bN0hKSiI4OJiwsDDOnz9Pz549mTZtGmXKlOHgwYOcPHmSKVOm0KpVK0aOHElAQIC9P4KIiIiIOKgi286RnZ3NuXPn2LBhA/7+/tbt7dq148iRI+zatYtq1aoxaNAgbty4wcMPP0x2djZArveLiIiIiPyvIhuiy5UrR5cuXdiwYQM5OTmcPn2aV199lbCwMCIjI1m3bh379u0jMjKSUqVK0bFjR3uXLCIiIiKFRJEN0QaDgQYNGhAbG0tsbCxr167l5MmTxMTE4OHhwbfffkudOnXsXaaIiIiIFEJFNkQDeHt706VLF44ePcq1a9do1qwZjRo14vTp0zzzzDM0adLE3iWKiIiISCFUZG8svK1WrVq0adOGJUuWAODu7k7t2rUVoEVERETkrhXpmWgAV1dXHnzwQfz9/bFYLBgMBnuXJCIiIiKFXJEP0QBVq1alatWq9i5DRERERIoIg8Visdi7CBERERH5RZbRSOqNLHuXUWz4+/uQlJSGk5MBPz/vPO9XLGai7eXq1XTMZv2OUhzc/gGU4kHjXfxozIsXjbfkRZG/sVBEREREJL8pRIuIiIiI2EghWkRERETERgrRIiIiIiI2UogWEREREbGRQrSIiIiIiI20TrSIiIiIFIjsW1lcTzPau4xctE60A/r3C2O4mXzV3mWIiIiIOITOK5aBg4Xou6V2DhERERERGylEi4iIiIjYSCFaRERERMRGCtEiIiIiIjZSiBYRERERsZHNIfrHH39k3bp1pKenc+rUKUwm072oS0RERETEYeV5ibu0tDSGDx/O7t27MRgMhIeHM3v2bM6dO8eyZcuoUKHCvaxTRERERMRh5Hkm+s0338RkMrFjxw48PDwAePXVV/Hx8WHWrFn3rEAREREREUeT5xC9Y8cOXnrpJcqXL2/dFhAQwIQJE4iLi7snxYmIiIiIOKI8h+gbN25QqlSp3213d3cnKysrX4tyJHoquoiIiIj8rzyH6IYNG7JmzZpc23Jycli0aBFhYWH5XZdd7du3j2XLlvHjjz+SnZ1t73JERERExMHk+cbCsWPH0qdPH/bs2UN2djZTp04lPj6e1NRUli1bdi9rLDCXLl1iwYIFXLt2jdq1axMXF0fVqlVxc3Ozd2kiIiIi4kDyHKJDQkLYuHEjH374Ib6+vri6uvLwww/z1FNPUbZs2XtZY4EwmUwsW7aM2rVr06dPHwDMZjMbN26kXr163HfffZjNZpyctLS2iIiISHGX5xDdr18/xo8fz6hRo+5hOQXv6NGj1K5dm4SEBA4ePMi4ceOAX26kfPfdd3F3d2fdunWsWLFCAVpEREREABt6oo8dO2Zd2q4oOHjwIMOGDWPRokUYjUaMRiOhoaGkp6cD4Ofnx2uvvcb777/PpUuXtAKJiIiIiFjleSb66aefZuLEiQwcOJAqVarg7u6e6/XfLn3nyJKSkliwYAEJCQlERUXRsWNHALy8vDh37hznzp2jbt261KlTB4D4+HgiIiKoW7euPcsWEREREQeS5xD97rvvYjQaiYuLw2AwWLdbLBYMBgNHjx69JwXmt3fffZedO3eyfft267Y1a9bw0EMP0aJFCzZt2sS+ffvo06cPq1evZuvWrXTr1g1vb287Vi0iIiIijiTPIXrx4sX3so576quvvuLy5cv06NGD/v37c/78ec6dO8fFixeJiYmhevXqdOvWjSeffJKffvqJtWvXMnHiRDw8PFi4cGGRuHFSRERERPJPnkN048aNAUhPT+f06dO4uroSEBDg0DO0x48fJyYmhtTUVPz8/FiwYAGDBg2iVatWPPLII0RGRjJy5Ejq168PwJUrV6hXrx716tUjMzOTEiVK2PkTiIiIiIgjynOIzsnJYcaMGXz00Ufk5ORgsVhwc3OjV69evPLKKw61coXZbGbu3Lls376dF154gbZt23Lt2jX++c9/AhAVFcUXX3xBly5dqF+/PlevXuXdd9/l4sWLzJw5k9KlSytAi4iIiMifsqknetOmTbz66quEh4eTk5PDvn37iI6OpmzZsgwePPhe1mkTs9mMi4sLDRs2pG3btgB88803VKxYEU9PTzw9PenXrx/Lli3j7Nmz7Nixg4iICMaOHYurq6udqxcRERERR5fnEP3pp58yadIkOnXqZN1Ws2ZNfH19mTNnjt1D9I4dO/jiiy8oW7Ysffr0ISoqiiVLlvDGG28QHx9PUlISaWlpLFmyhKioKCIiIti6dStxcXFMmzaNSpUq2bV+ERERESk88hyiU1JSuP/++3+3/f777ycxMTFfi7LFlStXmDVrFpmZmfTr14+ffvoJg8GAr68vjRs3ZubMmUybNo3IyEji4+P54IMPGDNmDA8++CDTp08vUmtfi4iIiEjByHOIDg4OZvv27fTv3z/X9q+++oqqVavmd115FhsbS2BgICNHjgQgLCyMTz75BIBGjRrRo0cPkpKSgF8+w4QJE7h8+bJmnkVERETkruU5RA8ZMoQRI0Zw9OhR62oW+/fv54svvmDWrFn3rMA7uXnzJrGxsTzxxBMA/PTTTyxatAiAChUqUK5cOSIiIli8eDHh4eEEBQUBKECLiIiIyN+S5xDdpk0b5s6dS0xMDF999RXu7u5Ur16d9957j+bNm9/LGq327t3LunXruO+++2jUqBH169enVKlS/PjjjzRq1IiKFSvy8ssvU6lSJV577TUSExNp3LgxXbp0ceil+ERERESkcMlziIZfgnSDBg2sDx/573//WyCPw7558yZvvvkmFy9epE+fPsTHx/PZZ5+xbds2unfvTkxMDCkpKfj5+QGQmJhISkoKtWrVwtfXl6ioqHteo4iIiIgUHwaLxWLJyxvPnDnDs88+S/v27Rk7diwATZs2pWTJksTExBAQEHDPijx06BCffPIJU6ZMsW5LSkpi0qRJdO7cmYSEBL777jsee+wxTp8+TWxsLA8++CADBgy4ZzWJiIiIiG2yb2VxPc1o7zJy8ff3ISkpDScnA35+ee9cyPNM9NSpU3nggQd47rnnrNu2bdvGuHHjmD59Ou+++65tFedBTk4Ozs7OnDp1ivLlywNgNBpxc3PD39+fyMhIbty4wYABA/D39+fnn3/GaDQSHR1N6dKl870eW129mo7ZnKffUaSQu/0DKMWDxrv40ZgXLxpvyYs8h+gDBw6wbt26XOHU29ubUaNG8dhjj+VbQdeuXeODDz7gueeesz74JCkpicuXLwPg5OSE2WzGycmJH3/80brsXteuXfOtBhERERGRO8nzs7o9PT25cuXK77anpKTk6yO/s7OzuXDhAuvXr7du69OnD7GxsRw7dgwXFxecnJy4ceMGZrOZhg0b5tu5RURERETyIs8z0e3bt2fSpEm8/vrr1psJDx06xOTJk62P1s4P5cqVo3Pnzqxfv56WLVtSvnx5PD09ef7554mOjqZixYqUK1eOb7/9lhYtWlCrVq18O7eIiIiISF7k+cbCjIwMRo4cSWxsLAaDAQCLxUKbNm2YNWtWvi4hl56ezuLFi3FycmLEiBHW7XFxcaSmpnL+/Hm6dOlChQoV8u2c94J6oosP9c8VLxrv4kdjXrxovIuXe35joZeXF4sXL+bMmTOcOHECFxcXgoOD78nTCr29vWndujUrV67k0qVLWCwWJk6cSM2aNfnnP/+Zr+0jIiIiIiK2yvNM9P+6du0a33//PQ888MA9Wd7OaDSyevVqFi9eTEBAAL1796Z79+75fh4RERERKTjGLBM3Um/auwyrez4TfezYMUaMGMG0adOoVasWvXr14uLFi7i6uvLuu+/m+1ML3dzcaNu2LU5OTjz++OO4ubnl6/ELwoI3t3Ljeqa9yxARERFxGK9M62nvEvJFnvsiZs2aRUhICMHBwaxfv57MzEy+++47nn/+ed5+++17UlxAQAD9+vUrlAFaRERERIquPIfoH374gTFjxuDr68t//vMfHnzwQXx9fenatSsnT568lzWKiIiIiDiUPIdoNzc3LBYLRqORvXv30qxZM+CX3mgvL697VqCIiIiIiKPJc09048aNeeONNyhZsiQAkZGRHDt2jGnTphEREXHPChQRERERcTR5nomeNGkSLi4uHDt2zLou9IYNG/Dw8OCVV165lzWKiIiIiDiUPM9E+/n5ER0dnWvbmDFjcHZ2zveiREREREQc2V09taRBgwZcuHBBAVpEREREiqW7CtF3+XwWEREREZEi4a5CtMFgyO86REREREQKDc1Ei4iIiIjYKM83Fv7WgQMH8rsOh2Q2m3FycsJisWj2XURERESs7ipEF3XffvstmzZtolq1ajz77LN4eHgoSIuIiIiI1V21cxRV8fHxjBs3jnXr1tG7d2/i4+OZOHEioD5wEREREfnVHWein3nmmTyHx6VLl+ZLQfZy4cIFXnvtNQICApg/fz4Arq6u7NixQ7PQIiIiIpLLHUP0P/7xD+bNm0fVqlUJDQ0tqJoK1O2AXLp0aTp27IjJZCI7O5utW7eyZs0aBg4cqAAtIiIiIrncMUQ/99xzlC5dmpkzZ/Kvf/2LwMDAgqrrnjt+/Dg1a9bEYDBgsVjw8fGhXr16rF+/nq5duxIeHs7kyZMJDg62d6kiIiIi4mD+sie6d+/eNG/enNmzZxdEPffclStXWLlyJY8//jibN28GICcnB4D777+f0NBQQkJCGD16NMHBwZhMJi3pJyIiIiK55OnGwtdff52+ffve61oKxCeffEJMTAyNGzdm1apVmEwmXFxcMJvNuLi4UK9ePWrWrMmGDRsAcHFxUTuHiIiIiORyxxC9evVqsrKyKFOmDI0aNSqomvKdyWSy/r19+/aEhIQQEBDAxYsXef/994FfZ6OrVatmfe3cuXN2qVdEREREHNsdQ/TUqVNJT0/PtW3ChAlcu3btnhaVX5KSkujZsydLly7l+vXrAFy/fp3KlSsTERHB/fffz4cffkhaWhqurq7s3buXGTNmUKJECYYMGUJQUJB9P4CIiIiIOKQ7hug/6gXesmULGRkZ96yg/OTs7Iy7uzsffvihddm6Ro0acfnyZYxGIxERERgMBqZMmcK0adOYM2cOoaGhtGnTBl9fXztXLyIiIiKOyuYnFhamm+zKlCnD4MGD2bRpE+fOnWP58uXUqlWL3r174+7uTuXKlalRowb//e9/6dmzJx999JG9SxYRERGRQqBIP/bbYDAQGhrKiRMnuHjxImFhYUydOpXAwEAGDRpEmTJlMBgMNGzYUDPPIiIiIpJnf7k6R2FfmaJkyZKEh4dz48YNypYty/PPP8/JkyfZvn075cuXp127dgrQIiIiImKTv5yJnjFjBh4eHtavs7Ozeeutt/D29s71vilTpuR/dfmkVq1aPPDAAyxYsIAZM2bwwAMPUKFCBXuXJSIiIiKF1B1DdHh4OAkJCbm21a9fn+TkZJKTk63bHH222s3NjTZt2lChQgUsFosCtIiIiIj8LXcM0StXriyoOu65qlWrUrVqVXuXISIiIiJFQJ6eWCgiIiIiIr8yWArTmnUiIiIiUqgZs0zcSL1p7zKs/P19SEpKw8nJgJ+f91/v8P+K9BJ39nb1ajpms35HKQ5u/wBK8aDxLn405sWLxlvyQu0cIiIiIiI2UogWEREREbGRQrSIiIiIiI0UokVEREREbKQQLSIiIiJiIy1xJyIiIiIOxZiVxY1UY4GcS0vcOaDFs14h9fpVe5chIiIiUqi8MOM9oGBC9N1SO4eIiIiIiI0UokVEREREbKQQLSIiIiJiI4VoEREREREbKUSLiIiIiNhIIVpERERExEYK0SIiIiIiNlKIFhERERGxkUK0iIiIiIiNFKJFRERERGykEP0Hjhw5Yu8SRERERMSBudi7AEdy4cIFAL7++mu+//572rZtS5UqVexclYiIiIg4Gs1EA6dPn2bUqFF88MEHlC9fngEDBuDp6Ul0dDSHDh0CwGw227lKEREREXEUxXom2mg08sYbb3D27Fk6dOjAo48+CoCbmxvdu3fHZDIxffp0PvzwQ5yc9PuGiIiIiPyiWCfDHTt2sH//fkaOHGkN0Lt27QJ+CdJPPvkkLi4urF+/HgCLxWKvUkVERETEgRS7mejY2Fhu3bpFnTp1aNasGfHx8Rw7doyMjAxWrlxJqVKlaNCgAa6urri4uDBgwABiY2Pp1q0bBoPB3uWLiIiIiAMoNiH6/PnzrFixgqNHj9K0aVNmz57Nxo0bCQsL47333iMlJYXXXnuN+vXr59rPw8ODKlWqYDAYsFgsCtIiIiIiUjxC9IkTJ4iKiuKll15i/PjxACQmJnLmzBlCQ0Np2rQpN2/etAbo7OxsXF1dAShXrhzr16/HbDarL1pEREREgCLeE717927S0tIIDg6mW7dulCpVCoDVq1fz5Zdf8tlnn3H48GEiIiLIyspi3bp1ANYADVC6dGm6du2KxWJRT7SIiIiIAEV0JvrgwYMsWLCAGzduEBISQo8ePejXrx///Oc/2bBhA1WqVCEmJoa9e/cydepU3n//fSpWrMjNmzcxmUy4uPx6WXx9fWnatKkdP42IiIiIOJoiFaLT0tKYNGkSWVlZ9OjRgw4dOjBy5EhOnjzJo48+SvPmzTl48CBTp04FoGLFiuzduxdnZ2ceeeQRPD097fwJRERERKQwKDIh+tSpUyQlJXHw4EGWLFlCYGAgAIGBgfj5+QHw9NNP88orr7B3715Onz7NunXriIqKwtvb23oc3TwoIiIiIn+lSIToPXv2cPnyZR5++GG6d+/Oli1bqFWrFtHR0cTHx1OqVCkCAgKoUaMG7du3p0+fPvTr148FCxZQtmzZXMdSgBYRERGRv1JoQ/Tx48dJSEggMjIS+OUmwu7duxMZGcnEiRPZsWMH0dHRXLp0ib179/LMM8/QsmVLRowYwZYtWwgODgZ+eZy3wWBQeBYRERGRPCuUIfratWtMnjyZixcv8vXXX1O5cmUqVapEUlISNWvWpFu3bpw5c4bKlStTuXJlGjduTLNmzbh16xYVKlQAsK62oWXrRERERMRWhTJB+vr6EhoaSlZWFosXL6Z06dKcOXOGq1ev4uLiQsuWLTGZTHz88cfWfcLCwvjHP/5h/dpgMChAi4iIiMhdKRQpcufOnYwaNYqDBw9at0VERDBq1CjWrl1LZmYmISEhbN26lfT0dIKCgggPDycjI0NrO4uIiIhIvjNYHDRlWiwWEhMTOXjwIDk5OYwePZqIiAjGjh1LjRo1+PjjjylXrhxJSUns2bOHN954g0mTJhEaGkr79u0pU6aMvT+CiIiIiNwFY1YWN1KNBXIuf38fkpLScHIy4Ofn/dc7/D+HC9G/XWIuPj7eeiPg8OHDuXLlClFRUWRlZREVFcWECRN4++23iYqKYsyYMdStW5fPPvuMkydPMnDgQOvNg/Zy9Wo6ZrNDXV65R27/AErxoPEufjTmxYvGu3i52xDtcO0ctwO0xWLB2dmZiIgIzGYz/fv3JyEhgdatW7N9+3Y+//xzKleujLOzMwMHDuT48eP4+voyYMAAWrVqRUpKilo5REREROSecJgQbTabmTx5MgsWLODmzZsYDAZ8fHw4dOgQCQkJ1K9fnwceeIBNmzYxb9484uPjWb58Oenp6XTt2pVBgwZZj9W+fXsaNWqkZetERERE5J6we4i+fPky48eP58033+TQoUOcPn2amJgYzp07h5+fH6GhocTGxgIwadIkFi1aRHp6OuPHj2fVqlV4e3tjNpsBrP9UeBYRERGRe8muIXrHjh28+uqr1KlThwEDBjBy5EiGDx+Os7MzEyZMICEhAV9fX6pVqwZAuXLlePLJJzly5AgAjRo1ArAuVacl60RERESkINj1YSs//fQTXbp0oUePHsAvM8mpqakMHToUg8HAxo0b2bJlCzk5OYSHhwMwcuRIe5YsIiIiImLfEF21alUmTZpEfHw8u3fvxmg04ufnh4+PD9HR0Zw4cYJNmzZhNpsxmUy4uPxa7m9X8RARERERKUh2X+IuLi6OjIwMnJ2dadWqFQA9e/YkOjqaihUrcuHCBQICAuxZooiIiIjkkcloIuXGTXuXkWd3u8SdXWeiARo0aEBiYiKBgYEALF26lJCQEEqXLg1gDdBms7nQ9TwfWbmb7LRb9i5DREREpMCEDnnQ3iUUCLuH6Js3b7J8+XISExNJSkqidu3ajBw5Ek9Pz1zvK2wBWkRERESKLruH6NKlSzNq1CgOHz6Ml5cX9erVAwrnzLOIiIiIFA92D9EAJUuWJCIiwvq1ArSIiIiIODKHTKoK0CIiIiLiyJRWRURERERspBAtIiIiImIjhWgRERERERspRIuIiIiI2EghWkRERETERgrRIiIiIiI2Uoj+A7du/fqobovFYsdKRERERMQRKUT/hsViITk5mTfffJP169eTnp6OwWCwd1kiIiIi4mAUov/fsWPHWLBgAV5eXrz00kucP3+e2bNnk5aWZu/SRERERMTBKET/v7S0NNLT09m2bRvu7u4MGzaMrKws1qxZQ2Jior3LExEREREHUmxD9JUrV4iJiSEhIQGABg0aUKtWLY4dO8apU6dwcnKid+/eXLlyhWPHjtm5WhERERFxJMUuRJvNZq5du8aVK1eYM2cO3bp1Y+XKlRw9epSmTZtSsmRJYmNjAQgLC8PLy4uLFy8CuslQRERERH5R7EL0nDlz+PTTT6latSovv/wyVapUoVSpUowdO5YVK1ZgNpv5+eefOXLkCAAtWrTgyy+/BNBNhiIiIiICFKMQnZOTA0DLli05e/YsJ0+epGPHjnh5eVG3bl0+/vhjfH19iYuLY/Xq1Xz55ZcYjUbCwsJo0qQJmZmZdv4EIiIiIuIoinSIzsjIYMeOHcCvs8hNmjShXLlyfPfdd7i5udG1a1dmzZqFt7c3/fv357333mPw4ME0bdoUNzc3Lly4QGZmJiVKlLDnRxERERERB1JkQ/TJkyc5ePAgK1as4Pz58zg5OZGdnQ1A9+7dOXfuHIcOHaJ9+/a4urqycuVKALy8vBg2bBhNmjQBICAggH/+8592+xwiIiIi4niKZIjes2cPhw8fpmHDhtSvX5+1a9cC4OrqSnZ2NoGBgYSFhbF7925u3LhB7969uXHjRq5j6CZCEREREfkzRSZEHz9+3Nq6ARAXF4ebmxutWrXi0qVLfP/998AvQRqgY8eOXLp0idOnT9O0aVOGDRuW63i6iVBERERE/kyRCNHXrl1j8uTJTJgwAaPRSOXKlalUqRLJycnUqFGDhg0bsnnzZuCXPumxY8eyd+9eXnrpJSIjI3Fy+uUymM1me34MERERESkkikSI9vX1JTQ0lKysLBYvXkzp0qU5c+YMycnJuLm50bRpUywWC4MHD2bw4MH4+/vTpk0bqlSpAvzaunE7TIuIiIiI3EmhTI07d+5k1KhRHDx40LotIiKCUaNGsXbtWjIzMwkJCWHr1q2kp6cTFBREw4YNcXNzY/r06YwZMwYXFxdreFbrhoiIiIjYotCEaIvFQkJCAtu2bSM9PZ0vvviCuXPncuzYMXJycrh48SL+/v4MHDiQ6dOn8+yzz5KcnMyWLVtISUkhKiqK+fPnExAQgNlsxmKxKDyLiIiIyF1x+BD929nijIwM5s2bR6dOnWjXrh2ZmZkcOHCAlStX0rlzZzZs2ECvXr2Ij48nLi6OMWPGkJ6ezhtvvEF8fLz1eE5OTgrQIiIiInLXXOxdwF+5HXYtFgvOzs5ERERgNpvp378/o0aNonXr1owZMwZ3d3cqV66Ms7MzAwcO5Pjx40RGRjJgwAC+/PJLUlJSCnz2+f4+/yiwc4mIiIg4ApPRZO8SCoTDhmiz2cyUKVMoW7Ys/fv3x9PTEx8fHw4dOkRCQgL169fngQceYNOmTcybN4+FCxeyatUqhg0bRteuXXMdq3379naZeb56NR2zWetNFwf+/j4kJaXZuwwpIBrv4kdjXrxovCUvHK6d4/Lly4wfP54333yTQ4cOcfr0aWJiYjh37hx+fn6EhoYSGxsLwKRJk1i0aBHp6emMHz+eVatW4e3tbV2q7vY/1bohIiIiIvnJoUL0jh07ePXVV6lTpw4DBgxg5MiRDB8+HGdnZyZMmEBCQgK+vr5Uq1YNgHLlyvHkk09y5MgRABo1agT8ulSdlqwTERERkXvBodo5fvrpJ7p06UKPHj2AX2aSU1NTGTp0KAaDgY0bN7JlyxZycnIIDw8HYOTIkfYsWURERESKIYcK0VWrVmXSpEnEx8eze/dujEYjfn5++Pj4EB0dzYkTJ9i0aRNmsxmTyYSLy6/la8k6ERERESkoDhWiH3roIXx9fcnIyKBRo0a0atUKgJ49e/Lzzz8TEhLCwoULCQgI+N2+CtAiIiIiUlAcKkQDNGjQgMTERAIDAwFYunQpISEhlC5dGsAaoM1ms8P3PPv5edu7BClA/v4+9i5BCpDGu/jRmBcvGu/8ZTRmc+PGLXuXka8cLkTfvHmT5cuXk5iYSFJSErVr12bkyJF4enrmep+jB2iAmJgYUlNT7V2GiIiIiF29+OKLgEL0PVW6dGlGjRrF4cOH8fLyol69ekDhmHkWERERkeLB4UI0QMmSJYmIiLB+rQAtIiIiIo6kUCRTBWgRERERcSRKpyIiIiIiNlKIFhERERGxkUK0iIiIiIiNFKJFRERERGykEC0iIiIiYiOFaBERERERGylEi4iIiIjYSCFaRERERMRGCtF/IDExkZs3bwJgsVjsXI2IiIiIOBqHfOy3vSQmJpKcnMyBAwe4dOkSw4cPp0SJEvYuS0REREQcjGai/198fDzDhw+nUqVKPPXUU3h6evLee++xc+dOQDPSIiIiIvIrhej/5+XlRVBQEG5ubgD069eP++67j/fff5/MzEwMBoOdKxQRERERR1FsQ/TRo0eZNm0aX3/9NQApKSkkJCTg5eUFQKlSpejUqRNBQUF89NFH9ixVRERERBxMsQ3Rrq6uVKtWjaVLl/L0009z4sQJGjduDIDJZALAzc2N1q1bk5qaar3RUERERESkWIRos9mMyWRi5cqV1m3BwcE88cQTrFy5km7durFx40ZiY2MBcHFxsfZAu7i4kJ2djaenp/qiRURERAQoJiHayckJFxcXlixZwsaNG4FfgrXFYsHZ2Znu3bszd+5cKlWqxJYtWwDIyckBoEGDBly5coXr16+rL1pEREREgCIeouPi4vjggw/YtWsXALNnz2bRokWYzWacnZ0xm82YzWYASpYsSYcOHXjvvffIycnBxeWX1f+Sk5N5+OGHtdSdiIiIiFgVyRB99epVXn75ZZYuXYqLiwsff/wxL774IiEhIdStW5d58+YB4OzsjJOTE9evX8doNNKxY0feeOMNnJ2dra0bFStWJDIy0rpqh4iIiIhIkQzRX3/9NdWrVycmJoZevXoxf/58atasyaRJkxg3bhybN2+23jw4e/ZsJk2aREJCAgC1atUCUOuGiIiIiPypIhOiL1++THZ2NgD79u2jRo0awK8rbQwaNIjExERMJhMPPfQQXbt2pW/fvty8eZPJkycTGBhot9pFREREpHAp9I/9PnjwIPPmzcPT0xM/Pz9ee+01AKpXrw78clOhyWQiLS2NkiVLkp2dzfDhw0lOTuapp57i/vvvB3650dDJqcj8TiEiIiIi91ChTY1paWm8+OKLLFq0iN69e/POO+/w9ddfc+bMGfz8/Fi+fDnw68oct27donTp0nh6euLq6sr06dO5//77sVgsCtAiIiIiYpNCORN96tQpkpKSOHjwIEuWLLG2YrRv3x43NzcGDx7Miy++yLRp02jdujXHjh1j69atPPbYY5QsWdJ6nNvhWf3PIiIiImKLQhei9+zZw+XLl3n44Yfp3r07W7Zs4f7772flypVUq1YNPz8/SpQowZQpU9i/fz+7d+/m+vXrLFy4EH9//1zH0uyziIiIiNyNQhGijx8/TkJCApGRkQDs3r2b7t27ExkZycSJE/n3v//NrFmzqFatmnWfSpUqUalSJUwmk3XNZ7PZjMFg0MyziIiIiPwtDh+ir127xuTJk7l48SJff/01lStXplKlSiQlJVGzZk2ioqI4ffq0NUD/NjQDuQJ0Qc88Dxw4sEDPJyIiIuKIjMZse5eQ7xw+RPv6+hIaGkp8fDyLFy+mb9++nDlzhqtXr+Lv70+LFi04ceIEn3zyCb169coVoH/LHq0bV6+mYzZbCvy8UvD8/X1ISkqzdxlSQDTexY/GvHjReEteOFyI3rlzJ59++in9+/enXr16AERERBAYGEhMTAw9e/YkJCSErVu3UqVKFYKCgggPDyc5ORmLxeJQrRpOTo5Ti9x7Gu/iReNd/GjMixeNd/Hi5GSwecwNltvPt7Yji8VCYmIiBw8eJCcnh9GjRxMREcHYsWOpUaMGH3/8MeXKlSMpKYk9e/bwxhtvMGnSJEJDQ2nfvj1lypSx90cQERERkWLErstT3M7vBoOBjIwM5s2bR6dOnWjXrh2ZmZkcOHCAlStX0rlzZzZs2ECvXr2Ij48nLi6OMWPGkJ6ezqxZs4iPj7fnxxARERGRYsauIfp264XFYsHZ2ZmIiAjMZjP9+/cnISGB1q1bs337dj7//HMqV66Ms7MzAwcO5Pjx4/j6+jJgwABatWpFSkoKDjChLiIiIiLFhF3aOcxmM1OmTKFs2bL0798fT09Prl69ypAhQ5g7dy6VKlViyJAhNGjQgB49erBw4UJWrVrFvn378Pb2znUsR+uDFhEREZGir0BD9OXLl1m4cCE+Pj7s3buXoKAggoKC6NatG0FBQUyfPp3q1avTq1cvrly5wsMPP8zatWsJDAxk3759NGrUyLpUnR7VLSIiIiL2UmApdMeOHbz66qvUqVOHAQMGMHLkSIYPH46zszMTJkwgISEBX19f63rP5cqV48knn+TIkSMANGrU6JeC/z84K0CLiIiIiL0U2BJ3P/30E126dKFHjx7ALy0dqampDB06FIPBwMaNG9myZQs5OTmEh4cDMHLkyIIqT0REREQkzwosRFetWpVJkyYRHx/P7t27MRqN+Pn54ePjQ3R0NCdOnGDTpk2YzebfPXVQfc8iIiIi4kgKtCc6Li6OjIwMnJ2dadWqFQA9e/YkOjqaihUrcuHCBQICAgqqHBERERGRu1KgTyxs0KABiYmJBAYGArB06VJCQkIoXbo0gDVA66ZBEREREXFkBRqib968yfLly0lMTCQpKYnatWszcuRIPD09c72vsARos9nMpEmTOH78OG5ubkydOpWgoCDr69988w0LFizAxcWFRx55hF69etmxWskPfzXmmzdv5v3338fZ2ZmQkBAmTZpUaL6f5ff+arxvmzBhAqVKlWLMmDF2qFLyy1+N98GDB5k5cyYWiwV/f3/efPNN3N3d7Vix/B1/Nd4bN25k2bJlODk58cgjj/DEE0/YsVrJLz/++COzZ89m5cqVubbfVWazFLAbN25YvvvuO8uPP/5o3ZaTk1PQZeSLL7/80jJ27FiLxWKxHDhwwDJ48GDra0aj0dK2bVvL9evXLVlZWZYePXpYrly5Yq9SJZ/cacxv3rxpadOmjSUzM9NisVgso0ePtnz99dd2qVPyx53G+7YPP/zQ0qtXL8ubb75Z0OVJPrvTeJvNZkvXrl0tZ8+etVgsFssnn3xiiY+Pt0udkj/+6ue7WbNmlpSUFEtWVpb1v+dSuC1atMjy8MMPWx599NFc2+82sxX4FFnJkiWJiIigXr16QOFu3di/fz8tWrQAICwsjJ9++sn6Wnx8PIGBgZQqVQo3NzcaNmzIvn377FWq5JM7jbmbmxsfffSR9f+smEwmzVIVcncab4ADBw7w448/0rt3b3uUJ/nsTuN95swZSpcuzfvvv89TTz3F9evXue++++xVquSDv/r5rlmzJmlpaRiNRi1wUEQEBgYSHR39u+13m9nsnl4La4AGSE9Pz/UERWdnZ0wmk/U1Hx8f62teXl6kp6cXeI2Sv+405k5OTpQtWxaAlStXkpmZSbNmzexSp+SPO433lStXeOedd5g4caK9ypN8dqfxTklJ4cCBAzzxxBMsW7aM3bt3ExcXZ69SJR/cabwBatSowSOPPMJDDz3Egw8+SMmSJe1RpuSjDh065Fr97ba7zWyFN8E6AG9vbzIyMqxfm81m6+D872sZGRm5BkgKpzuN+e2vZ82axa5du4iOjtbMRSF3p/H+4osvSElJYdCgQSxatIjNmzezbt06e5Uq+eBO4126dGmCgoKoXr06rq6utGjR4nczl1K43Gm8jx07xrfffsv27dv55ptvuHbtGp9//rm9SpV77G4zm0L039CgQQP+85//APDDDz8QEhJifS04OJhz585x/fp1jEYj+/bto379+vYqVfLJncYcYOLEiWRlZbFw4cLf3TArhc+dxrtv376sW7eOlStXMmjQIB5++GHrw6SkcLrTeAcEBJCRkcG5c+cA2LdvHzVq1LBLnZI/7jTePj4+eHh44O7ujrOzM76+vqSmptqrVLnH7jazFejqHEVNu3bt2LVrF4899hgWi4Xp06ezadMmMjMz6d27Ny+//DIDBgzAYrHwyCOPUL58eXuXLH/Tnca8Tp06rF27lkaNGtGvXz/gl6DVrl07O1ctd+uvfsalaPmr8Z42bRovvvgiFouF+vXr8+CDD9q7ZPkb/mq8e/fuzRNPPIGrqyuBgYF0797d3iVLPvu7ma1AH7YiIiIiIlIUqJ1DRERERMRGCtEiIiIiIjZSiBYRERERsZFCtIiIiIiIjRSiRURERERspBAtIlLIGI1GlixZQlRUFPXr16dp06YMHjyYQ4cOAXDx4kVq1qyZp8fW/l3R0dG5lnHcuXMnrVu3pm7duqxYsYLWrVuzcOHCe16HiEhB0xJ3IiKFyM2bN+nbty8pKSmMGDGC0NBQMjIyWLFiBVu3bmXRokVUqVKFNm3asHr1aho1anRP68nIyCArKwtfX18AHnnkEUqXLs3kyZMpXbo0RqMRDw8PSpQocU/rEBEpaHrYiohIIfL2229z9uxZNm/enOthADNnzuTq1atMmTKFf/3rXwVWj5eXF15eXtav09LSiIyMpEqVKgVWg4iIPaidQ0SkkDAajaxbt46ePXv+4dO0Jk6cyJw5czAYDLm2X79+nXHjxtG8eXMeeOABmjdvzqxZszCbzQAkJyczbNgwmjRpQlhYGE8//TRHjx617r9u3To6depEnTp1aNWqFfPnz7fu+9t2jpo1a3Lu3DkWLFhAzZo1AX7XzvH111/TtWtX6tatS8eOHVmyZIn1WLfbUP71r38RERFBp06dMBqNLFq0iDZt2lCnTh06dOjA6tWr8/GqiojcHc1Ei4gUEhcuXCA1NZXQ0NA/fD0gIAD4JYz+1tixY0lJSeHdd9+ldOnS/Oc//2HKlCk0bNiQtm3bMnnyZEwmEx988AEGg4E5c+YwfPhwvv76a44dO8bEiRN56623qFOnDocPH2bMmDEEBgYSFRWV6zyxsbH07t2bDh060L9//9/Vt2PHDsaMGcP48eNp3LgxJ0+e5PXXX+fmzZsMGzbM+r4tW7awatUqbt26RWxsLEuWLOHtt98mMDCQ7777jgkTJhASEkJ4ePjfvKIiIndPIVpEpJBITU0FoGTJkjbt16JFC5o0aUKNGjUAePLJJ1m8eDHHjx+nbdu2nDt3jpo1a1KlShXc3d15/fXXOXXqFGazmQsXLmAwGKhUqZL1z7Jly6hQocLvzuPv74+zszMlSpTA39//d6//61//4vHHH6dnz54ABAYGkpGRwYQJExgyZIj1fU8++STBwcEA7N27F1dXVypVqkTlypV59NFHqVKlCvfdd59N10BEJL8pRIuIFBJlypQBfmnPsMXjjz/O9u3bWbNmDWfPnuX48eMkJCRY2yiGDBnC2LFj2bZtG+Hh4bRs2ZKoqCicnJxo0aIFoaGhPPLIIwQFBdG8eXM6d+5MpUqVbK7/6NGjHDp0iI8++si6zWw2c+vWLS5dumRtQ7k9ow7QpUsX1q5dS/v27QkJCaF58+Z07doVPz8/m88vIpKf1BMtIlJIBAYG4ufnx48//viHr+/Zs4fBgweTlJRk3WaxWBg0aBAzZ87E09OTbt26sWrVKipXrmx9T8eOHdm5cydTp07F39+fhQsXEhUVRXJyMh4eHqxatYq1a9fSrVs3jhw5wlNPPUVMTIzN9bu6ujJ48GDWr19v/bNx40a2bduWq8fb3d3d+nc/Pz82btzIqlWraN26NXFxcTzyyCNs2rTJ5vOLiOQnhWgRkULCycmJ7t278+mnn5KYmJjrNYvFwqJFizhz5gxly5a1bj916hSxsbFER0czevRoHnroIcqUKUNSUhIWiwWTycSsWbO4dOkSXbp0YcaMGWzZsoVLly7x/fffs2vXLhYsWEDdunUZOnQoH330EY899hifffaZzfVXr16ds2fPEhQUZP1z4sQJ5s6d+6f7bN26lQ8//JDw8HBGjx7N+vXradasGRs3brT5/CIi+UntHCIihciQIUPYtWsXTzzxBKNHjyY0NJTk5GSWLl3K3r17Wbp0aa7VOUqWLImLiwuff/45pUqVIikpiblz52I0GjEajbi4uHD48GH27dvH+PHj8fX1ZdOmTbi6uvLAAw+QmJjIggUL8PHxoVWrViQnJ7Nnzx7CwsJsrv3555/nueeeIyQkhPbt23P27FkmTpxIZGQkbm5uf7iP0Whk1qxZ+Pj40LBhQ86fP8+RI0d4/PHH7/YSiojkC4VoEZFCxMvLi1WrVhETE8M777zDzz//jI+PD6GhoXz88cfUrl071+oc5cuXZ/r06URHR/P+++9Tvnx5OnXqRPny5a1POJwzZw7Tp0/nueeeIyMjgxo1arBgwQLrbPH06dNZvHgxs2fPxtvbm7Zt2/LSSy/ZXHvLli154403WLRoEfPnz8fX15eoqChGjx79p/tERUVx9epVoqOj+fnnn/Hz86NHjx4MHjzY9osnIpKP9MRCEREREREbqSdaRERERMRGCtEiIiIiIjZSiBYRERERsZFCtIiIiIiIjRSiRURERERspBAtIiIiImIjhWgRERERERspRIuIiIiI2EghWkRERETERv8HRHRcx65c3DEAAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 720x360 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "fig, ax = plt.subplots(figsize=(10,5))\n",
    "ax.set_xlabel('Classifiers',fontsize = 15) #xlabel\n",
    "ax.set_ylabel('F1-score', fontsize = 15) #ylabel\n",
    "\n",
    "plt.yticks(rotation=30, size = 10)\n",
    "plt.xticks(rotation=0, size = 10)\n",
    "\n",
    "ax = sn.barplot(x=f1, y=classifiers)\n",
    "ax.set_title(\"F1-score of all classifiers\", fontsize= 15)\n",
    "\n",
    "\n",
    "plt.tight_layout()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As we can see the dummy classifier has approximately 50% score, due to the imbalance of tha data. The best classifier is the SVM with rbf kernel. The KNN's performance isnt so good (except from the dummy one), due to the form of the data "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- ## Preprocessing "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- Imbalanced dataset Over Sampler"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Requirement already satisfied: imbalanced-learn in c:\\users\\nanos\\miniconda3\\lib\\site-packages (0.8.1)\n",
      "Requirement already satisfied: scikit-learn>=0.24 in c:\\users\\nanos\\miniconda3\\lib\\site-packages (from imbalanced-learn) (1.0)\n",
      "Requirement already satisfied: joblib>=0.11 in c:\\users\\nanos\\miniconda3\\lib\\site-packages (from imbalanced-learn) (1.0.1)\n",
      "Requirement already satisfied: numpy>=1.13.3 in c:\\users\\nanos\\miniconda3\\lib\\site-packages (from imbalanced-learn) (1.21.5)\n",
      "Requirement already satisfied: scipy>=0.19.1 in c:\\users\\nanos\\miniconda3\\lib\\site-packages (from imbalanced-learn) (1.7.1)\n",
      "Requirement already satisfied: threadpoolctl>=2.0.0 in c:\\users\\nanos\\miniconda3\\lib\\site-packages (from scikit-learn>=0.24->imbalanced-learn) (2.2.0)\n"
     ]
    }
   ],
   "source": [
    "!pip install -U imbalanced-learn"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [],
   "source": [
    "from imblearn.over_sampling import RandomOverSampler\n",
    "from sklearn.preprocessing import MultiLabelBinarizer ## MPOREI NA TO AFAIRESW AUTO!!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of data samples is :  3525\n",
      "Number of characteristics of each sample is  180\n",
      "There exist 3 classes in column 3524 of the dataset \n",
      "The labels of the classes are : [0 1 2]\n"
     ]
    }
   ],
   "source": [
    "ros = RandomOverSampler()\n",
    "X_train_ros, y_train_ros = ros.fit_resample(X_train, y_train)\n",
    "print(\"Number of data samples is : \", len(X_train_ros))\n",
    "print(\"Number of characteristics of each sample is \", len(X_train_ros[0]))\n",
    "print(\"There exist {} classes in column {} of the dataset \".format(len (np.unique(y_train_ros)), len(y_train_ros) -1))\n",
    "print(\"The labels of the classes are : {}\" .format(np.unique(y_train_ros[:])))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Class 0 appers : 1175 times, with frequency 33.33333333333333 %\n",
      "Class 1 appers : 1175 times, with frequency 33.33333333333333 %\n",
      "Class 2 appers : 1175 times, with frequency 33.33333333333333 %\n"
     ]
    }
   ],
   "source": [
    "for i in range(len (np.unique(y_train_ros))):\n",
    "    print(\"Class {} appers : {} times, with frequency {} %\".format (i, np.count_nonzero(y_train_ros[:] == i), np.count_nonzero(y_train_ros[:] == i)/len(y_train_ros) * 100))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- dummy"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Dummy Accuracy: 25.732217573221757% \n",
      "\n",
      "              precision    recall  f1-score   support\n",
      "\n",
      "     class 0       0.26      1.00      0.41       246\n",
      "     class 1       0.00      0.00      0.00       231\n",
      "     class 2       0.00      0.00      0.00       479\n",
      "\n",
      "    accuracy                           0.26       956\n",
      "   macro avg       0.09      0.33      0.14       956\n",
      "weighted avg       0.07      0.26      0.11       956\n",
      "\n",
      "\n",
      "F1-score is : 0.13643926788685523\n",
      "Cross validation score is : 33.19155614215813\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\nanos\\miniconda3\\lib\\site-packages\\sklearn\\metrics\\_classification.py:1308: UndefinedMetricWarning: Precision and F-score are ill-defined and being set to 0.0 in labels with no predicted samples. Use `zero_division` parameter to control this behavior.\n",
      "  _warn_prf(average, modifier, msg_start, len(result))\n",
      "C:\\Users\\nanos\\miniconda3\\lib\\site-packages\\sklearn\\metrics\\_classification.py:1308: UndefinedMetricWarning: Precision and F-score are ill-defined and being set to 0.0 in labels with no predicted samples. Use `zero_division` parameter to control this behavior.\n",
      "  _warn_prf(average, modifier, msg_start, len(result))\n",
      "C:\\Users\\nanos\\miniconda3\\lib\\site-packages\\sklearn\\metrics\\_classification.py:1308: UndefinedMetricWarning: Precision and F-score are ill-defined and being set to 0.0 in labels with no predicted samples. Use `zero_division` parameter to control this behavior.\n",
      "  _warn_prf(average, modifier, msg_start, len(result))\n"
     ]
    }
   ],
   "source": [
    "dummy = DummyClassifier().fit(X_train_ros, y_train_ros) \n",
    "\n",
    "pred_dummy = dummy.predict(X_test) \n",
    "  \n",
    "accuracy_dummy = dummy.score(X_test, y_test) * 100 \n",
    "print(\"Dummy Accuracy: {}% \\n\".format(accuracy_dummy))\n",
    "\n",
    "target_names = ['class 0','class 1','class 2']\n",
    "print(classification_report(y_test, pred_dummy, target_names=target_names))\n",
    "print()\n",
    "\n",
    "f1_dummy = f1_score(y_test, pred_dummy, average='macro')\n",
    "cross_val_dummy = np.mean(cross_val_score(dummy, X_train_ros, y_train_ros, cv=10))*100\n",
    "print('F1-score is :', f1_dummy)\n",
    "print('Cross validation score is :', cross_val_dummy)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- GaussianNB"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "GaussianNB Accuracy: 94.0376569037657% \n",
      "\n",
      "[[0.97154472 0.0203252  0.00813008]\n",
      " [0.02597403 0.94805195 0.02597403]\n",
      " [0.03966597 0.03966597 0.92066806]]\n",
      "              precision    recall  f1-score   support\n",
      "\n",
      "     class 0       0.91      0.97      0.94       246\n",
      "     class 1       0.90      0.95      0.92       231\n",
      "     class 2       0.98      0.92      0.95       479\n",
      "\n",
      "    accuracy                           0.94       956\n",
      "   macro avg       0.93      0.95      0.94       956\n",
      "weighted avg       0.94      0.94      0.94       956\n",
      "\n",
      "\n",
      "F1-score is : 0.9372455231183117\n",
      "Cross validation score is : 94.9786730620654\n"
     ]
    }
   ],
   "source": [
    "gnb = GaussianNB().fit(X_train_ros, y_train_ros) \n",
    "\n",
    "pred_gnb = gnb.predict(X_test) \n",
    "  \n",
    "accuracy_gnb = gnb.score(X_test, y_test) *100 \n",
    "print(\"GaussianNB Accuracy: {}% \\n\".format(accuracy_gnb))\n",
    "\n",
    "cm = confusion_matrix(y_test, pred_gnb) \n",
    "cm = cm.astype('float') / cm.sum(axis=1)[:, np.newaxis]\n",
    "print(cm)\n",
    "\n",
    "target_names = ['class 0','class 1','class 2']\n",
    "print(classification_report(y_test, pred_gnb, target_names=target_names))\n",
    "print()\n",
    "\n",
    "f1_gnb = f1_score(y_test, pred_gnb, average='macro')\n",
    "cross_val_gnb = np.mean(cross_val_score(gnb, X_train_ros, y_train_ros, cv=10))*100\n",
    "print('F1-score is :', f1_gnb)\n",
    "print('Cross validation score is :', cross_val_gnb)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- KNeighborsClassifier"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAD/CAYAAAD/qh1PAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8rg+JYAAAACXBIWXMAAAsTAAALEwEAmpwYAAApMklEQVR4nO3deVxU97038M8sDDAw7APIroOCoLhgEiXuGkOaNKtJE1NNbNLG3tTc2lT79N7n9nle7W2e2yX3NjU+SdrbKsYli42m2VyIlpoad3FDQEVFFnHYmX0794+RURRGkJmBOefzfr18BfBw+P3AfPjN9/zO98gEQRBARESiJR/qARARkX8x6ImIRI5BT0Qkcgx6IiKRY9ATEYkcg56ISOQY9EREIqcc6gH0pq3NCJerf9v74+Mj0dJi8POIhh+pzhuQ7tw5b2kZyLzlchliYyP6/PthGfQul9DvoO8+XoqkOm9AunPnvKXFV/Nm6YaISOQY9EREIsegJyISOQY9EZHIMeiJiESOQU9EJHIMeh84W9eOH67+Cg3NxqEeChHRLRj0gyQIAj7YfQ6dRhu+Otk41MMhIroFg36Qys8143xDJ9ShShyoaIKLD+wiomGGQT8ILpeAj8pqkBSnxqL7RqOty4qzl9uHelhERD0w6Adhf8UV1Dcb8diMkSgck4jQEAX2VzQN9bCIiHpg0N8hh9OFbXsvIDNJgym5iQhVKTBpTAIOV16Fw+ka6uEREXkw6HthdzhhtTm9HlNW3oDmDguemDUKcpkMADA1LwlGiwOnaloDMUwion5h0N/E6XLh15uPYdXb+1B5qa3XYyw2Bz75xwXkZsQgf2Sc5+N5WXGIDA/B/oorgRouEdFtMehvsuPgZZyv74RcLsNv3yvHjoO1EG7aSbPrcB06TXY8MUsH2bXVPAAoFXLcNTYR5WebYbY6Aj10IqJeMehvUN9sxLa9NSjM0eK1707FpNEJeH/3Obzz19Ow2NzBbTDbsf3AJUwanQBdavQt55ialwSbw4Xys82BHj4RUa8Y9Nc4XS78+bMKhKmU+PaCHISHKvFPj43Dwtk6HKq8il+uP4KmVhM+338JFqsTj88c1et5dKnRiI8Kw9cs3xDRMDEsnzA1FLYfqMWFxi4seyQf0REqAIBMJsM3pmYiM0mDd/56Gj8vOQSHU8C0cclI1Ub2eh65TIap+Un4Yn8tOo02RF07FxHRUOGKHu6SzcdfXUBhjhZ35Sbe8vf5I+Pws+enIDFGDQB4dPpIr+e7Jy8JLkHAocqrfhkvEdFADHpFX1paioaGBqjVasybNw9vvvkmAODll1+G2WzG7373O/zmN78Z9ED95caSzeIFOT0urt4oIToc/7qkEAazHTGRoV7PmaaNRJo2AvsrrmBeYZo/hk1E1G+DDvqKigq88sorWLNmDUJDQ7FixQps27YN7e3t2LdvHzIyMgZ8zvj43ssifdFqNQP+Gt0+/LIaFxq7sGrxFOiy4u/4PDebd3cmSj6rgFMuR3J8309nH4zBzDvYSXXunPfw53S6oG83w2C2IzNZgxClwuvxV1qM2HeiEeXVV7Fw3mgUZGs9f+ereQ866BUK9yQEQYBarUZpaSmio6NRV1cHo9GI8vJy1NfXIzU1td/nbGkx9Pvp51qtBnp91x2NvV5vwKYdlSjM0SIn5c7P05v8DPeOnC++qsFDRVk+O2+3wcw72El17pz38NPWZcXhyqu42mbG1XYzrraZ0NxhgfNafikVMmQlRyE7NRrZadHITo1GVIQKDc1GHKm6iiPVetQ2GQAAmUka2C12z1wHMm+5XOZ1gTzooNfpdCgpKUFaWhoqKyuxbt06FBUVYfLkyZg5cyZsNtuAQj6Q3t1ZfduSzZ1KiA7H6LRo7K9owoPTMn1+fiIaWh0GK/7fhiNo7rAgTKVAYmw40pM0KMxJRGJsOMJDlbjQ0Ilz9R0oPXIZ2w/WAgA06hB0mewAgOzUaDw1JxuFOVpoY8L9NtZBB31xcXGP9zds2NDj/eXLlw/2S/jNpStdmDFhhN92xkzNT8a7O6pw+aoBGUnB89KTiLyz2pz43ZYT6DTZ8C/fLoQuNarXxVz35g67w4lLVww4W9+OuqtGjEqJwuQxWsRqvF/v8xXJ7rpxOF2w2p2IDA/x29eYkqOFKkSO//60Al0mW78+p15vQFVt760XiGjwnC4Xapu64HTdWfNBp8uFtz8+hdqmLix7eByy06Jv+4o9RKlAdlo0HrgnE9/9Zh7mFaYFLOQBCe+jN11rURAR5r+g16hVWP5EAX6/5QR++145Vj4zyesvliNVevzxk9NwOAX885MFGD/KdxeHicTK6XLB5QJClN7XrTa7E1+dbMT2A7Vo7rAgJSECC2frMEEX3+/SqiAI2LTrLI6fb8G3F4zBxNEJvpiC30k36C3uoFeH+vdbkJ8Vh+WPj8fv/3ISv33vGFY+M+mWXy6CIGDnocv4YPc5jEyJgt3hwlvbTuFfFhcirY8bs4ikzO5woeJiK45U6XHsrB5Wuws56dEYNyoe40bFIyVe7Qlvk8WBPcfqsOvQZXSa7NClRGH+lHTsOVaP3285gZz0GDw1NxsjR0Td9utuP1iLPcfq8cA9GZg7OXi2Tks26I0W98UQdZj/vwXjRsXjB4+Pw5sfncTr75Xjx09PhPpa2DtdLmzadRZ7jtVjSo4WLz6UB4PZjl+sP4w3PjyO/71kCqJvs2+fSOwEQYDZ6sSZS+5wP36+GWarE+GhSkzMjkdkuAqnL7bi/d3n8P7uc4iLCsW4kfFQhypRdrweZqsT40bG4cFpmRiTHgOZTIa5k1Ox93gDPv7qAn5Rchh3j03EE7N0fV4UPXimCR/uOe8+brYuwN+BwZFs0Hev6P1ZurlRgS4B//ToeKzZehKvv38cr35rImQy4K2PT+FUTSsemJqBJ2bpIJfJEBeiwD8vLMB/bDyK3//lJFYtmoTQEO97cYnEoMtkQ1l5A662m9FltKHTZIPR4kB7lxU2h7umHhkegik5iSjMSUReViyUiuslm5YOC05daMGpmlYcqmyCxepEYW4iHpyaiczknhsilAo55kxOw9T8ZGw/UIsdh2pxpEqPVG0EYiJDERWhQkykCtERoZDLZdhcWo0xadF44cGxnmdQBAuZcHMP3mEgEPvoD1Q04Z2/nsa/v3gPUhL8c0NTb45V6/H/t51CVrIGVrsLDc1GLL5/DGZNvHUL6rFqPd786CQKc7RY9ui4Hv+4BrO3+FxdB5RK9/7eYDSc91X7k5jnbbE5sOuQewuixepEdKQKUREqRKlV0MapEaqQQxMRgqwkDcZkxEAhv/0+EofTBYut/xsu2rqs2HmoFo0tJnQYbGg3WtFptKE7IZPj1PiXxYV+3cBxo2G1jz5YmQJYurnRpDFaLHskH29tO41QlRwrnprQ4+ElNx/71NxsvL/7HD4qq8FCH7xcbDdY8foH5XBeu+Cbn9X71yYKBIfThbLyBnyy7yI6jTZMGp2Ax2eO6tE08E5/wSkVckSG939jYawmFN+aO7rHx1wuAV0mGzqMNiTFqYP2lbVkg97oKd0E/ltQmJOIf10ShojwECTe5iaJBXelo6nNjM/3X0JSbDhmTEgZ1Nf+6O81cDhcSIwNx+otJ/DDJycgNzN2UOckGiiz1YHyc83YtrcG+nYLxqTH4AePj0d2L894GEpyuQzRkaFBf51MskFvsjgQopTftg+Fv/TnCj/gbpW8aP5o6NvNWL+jCqPTY5Acp76jr3npShf+caIR99+dgeJ7MvDrzcfwxpYTWPHUBIxJj7mjcw6V8/UdKNleiXmFab2Wveg6m90Jh9Pl2QAwGK2dFqhCFAMqXzicLtTpDbjQ0Imaxk5cbOxCQ7MRAoD0xEj88MkJGD8qjneP+5F0g95qD3jZ5k4pFXK8+FAe/tfbX2Pb3hose2TcgM8hCAI2l1YjUh2Ch4qyoA5TYuXTE/GrTcfwXx+6Lw4Pt9VUX746Xo//3HwMgiCgZHsVbHYX7rsrfaiHNex0mmzYfaQOu4/Ww2Jz4NEZo1B8dwbk8jsL1Hq9Af++/gjsDhdyMmIwJUeLyWO0t6x2XYKA2qYunL7QitMXWnGuvhMOp/tCqkYdgpEjonBXbiJ0qdEYmxUbdBc2g1FwJJ0fGC0Ov++h96XoCBXuuysdn+67iAfu6RpwV7sjVXpU13VgSXGO5xdcdGQoVj4zCb/adBT/9UE5fvz0pH6/0hgKgiDg8/2X8JeyGmSnReP7j4zDptJqbP7yLGwOJx6cltXn55osDny67yK6TDbcd1e6qFtSXG03Y+fBWnx1ohE2hwsTsxMgl8uw5W/ncaxaj+88OBYjBthR1Wx1YM3WUwhVKTC3MBVHq5vx7s5qbNhZjey0aBTmJCJcpcDpi62ouNgGg9l9DSw9MRJzJ6dClxqNkckaxEeHceU+BIIn6XzMZHEEbGulrxTfnYE9R+vwl7+fx5Tx/a/V2x1OfLDnHNK0kZhZ0PPzYjWhWPXMJPzHxqN4/drduzdvQ/Mlh9OFDoMNMRpVv3ZO3Ph563dU4asTjZg1KQ2L5ukQolRg2SP5+NOnZ/CXshrY7C48OmNkjyARrj0AZvOXZ9FpsEGlUuAfp65gYnYCvnlvVsB+sbkEARBwx6vpvrj3lzvQ0mlFS4cFX5++gsNVV6GQyzAtPxn3352BlIQICIKAA2easHFnNf7v2kN4YpYO86ek9Ws1LQgC1n5RiattZqx8ZiJyMmKxcJYO9c1GHK3S43CVHu99eRaAe0FSoItH/sg45GXFeZ7WRkNLskFvtNz+ASLDjTpMiQenZeGDPedw8nwzkqP6N/6dhy6jucOClU9P7DVo4qLCsOrayv4/PyjHz79zt18uPpksDry24Qgamo2Qydy/ZOKiwhDf/Sc6DNqYMGhjwhEfFebZH22y2LFm6ymcudSGh+/NwouPFaC52d3aVSF3l7VClHJ8su8i7A4Xnpyjg0wmw9V2MzbsrMKpmlZkJmnwyhMFSIoNR+nhOuw8dBm/KDmM8aPi8c17s/xSthIEARcau3CgogkHK5tgs7swffwIzJ2ciqQ7vM5isTnw2deXcOlKF1q7rGjptMBqc3r+PjxUgeJ7MjC/ML1HLxWZTIapecnIzYhFyReVeO/LszhadRXfeXAsEmO9j2XX4TocrryKJ+fokJMR6zmf+wE7kXh4+khcbTPB7nAhJSGCK/ZhSLL76Fe9tQ+j06Lx3W/mD/hzh5LN7sRP/7AfSXFqrHx64m3/p2o3WPHTP+xHXmYslj9R4PXY+mYjfr7uEHLSY/DDpyb4tHbqcgn43ZbjOHOxDY/NHAWrzYnWTgtarv1p7bR6engDgEwGxEe5Q7+l04KWDguefyAX944f0evP3CUI2LSrGruP1mPO5FTERobik30XoZDL8NjMUZg3Oa3HLzmz1YHdR+uw4+BlGMx2FOji8dLD+Qj3QTmvvtnoDveKJlxtN0OpkGH8qHiEKOU4UqWH0yVg/Kh4zCtMw7hRcf3+PrdbHPhVySFcbTMjI1mD+KgwxF37ZRkXFYr4qDCkaiMQpvI+B0EQ8I+TV7D5y2o4XQIemT4S901J73HjUbfqy+34zeZjKNDF4wePjx+SEBfz/QPecB+9D5gsDp/sQgg0VYgCD9+bhZLtVSg/14xJo7Vej/+ozL2d8qm52bc9d2pCBJ6em413d1aj9HAdFvjwAucHe87hVE0rnivO6XWXjEsQ0N5lhb7dDH275dp/3X9USgV+/PREz2qyN3KZDM/eNwYqpcLT97swR4tF88f02iUwPNT96mheYRp2H63H1r/XeHYg3eleaX27GX/67AyqL7dDJgPGZsbiwWmZKMzRev6ttRus+Ht5A/aU1+N3Hx5HYmw45k5KxdT85D7bZbsEAdsP1GLr32sQFaHCqkWTvH4vbkcmk2F6wQjkZcViw85qfLjnPPaduoLFC3J67L7qMNrw1senEB8dhhcezONKPYhJMuhd1+qawXQx9kbTC0ag9EgdPiqrwQRdQp9134tXOvGPk+7tlEm3eXnebfakVJysacWWv51DbkaMTy5a7j3egJ2HLmO+l62Qcpns2so0DDkDf/okAHeAPTlHhxEJasREhvar+2eYSolvTM1EfFQY/vDX01jz0Uksf6Lgtp0Qb7a/4gre3VEFAPjW3GxMzUvqtfwVExmKh6ePxDemZeJIlR5fHq3De7vP4YM95zFuVByKxiVjYnYCVNd+2bR2WvDfn1agsrYd905IwdNzdD67thQXFYZXFhbg2Fk9Nu2qxn9sPIp7xyfjyTnZiAhT4p2PT8FsceBHT00Mmh1q1DtJ/vTMVgcEDM3NUr6gkMvxbPFY/Prdw9hfcQVF40bcckxblxXv7ri+nbK/ZDIZnv9GLv7Pnw7iD59U4GfPTfGEzp2ovtyO9TuqkJ8Vi2/Nu/2risGSyWSYUTDwm8ruyUuCze7E2i8q8c5fT+P7j+b362KxxebAxl3V+MfJK9ClRuGlb+YjoR9PClIq5LgnLwn35CWhXm/A16eb8PXpK3j749MID1WgMCcRmUkabNtbA4dTwNJv5OKxuWM81yZ8adJoLfIy4/DJvovYcbAW5WebkZ0ajcradrzw4FikJ7KDarALzqQbJE+L4iAs3XS7tyDlWhBcwF25SZ4VaIfBis/2X8LfjjVAEAS88NDYAa/GotQqvPhQHl5/vxzv7zmHxQty7miMze1mrNl6Egkx4Vj26LgB7bIZCjMmpMBid2Jz6Vn8+bMzeOGhPK/18wuNnfjDX0/jarsZ3yzKwsPTs+5ojqnaSCycHYnHZ41C1aU27Dt9BYcqr+KrE43IStbgpYfzkRSn9mvpJFSlwMLZOkwbl4wNO6pw/HwLZk9Mwb3jb11EUPCRdNAH64oecF98eWLWKPznB8dRVl6Pu/OSsH1/LXYfrYPDKaBofDIeLsrq1+qyN/kj47DgrnTsPHQZ40fGD/gBC2arA7//ywl3T52FBUGzlfW+Kemw2JzY+vcahKqUWLxgTI+AtTucqNMbceJ8Cz7dd9FdM39mcDXzbnKZDGOz4jA2Kw7fXuDE5SYDskZoer1I6i+pCRFYtWgSLjR2ITOZK3mxCN6kG4RA9qL3p/yRccjNiMHWvRfc+8gdTkzNS8bD07P6XZP35olZOlReasOfPz+Dn79wd7+2o3Y/EGLHwVo0NJuw4qkJd9yyYag8NC0TVpsTn++/BBmA5Hg1aq904VKTAQ3NRveeeACFY7R47oFcv3QzDA1xP3puKMhkMoxKGb43ztHABXfS3SExlG4A9/+QC2dn49ebjmLi6AQ8fO9In7ZcDlHK8b2H8/HzdYfwzsenMX9KOhKiw5AQE9ZjhW61OXGypgVHqvU4fq4ZFpsT4aEKLCnO6bMz53Amk7lfLVltTnx5tA4AEKUOQWZyFCaOjkdGogaZyZo+H1BBNNxIM+itwV+66TYqJQpvvTrLb/XblIQIPHvfGKzbXomqy+2ej4eHKpEQHYaIMCVqGjphc7gQGR6Cu3J7fyBEsJHJZHjmvtGYXjDC8wAKbi+kYBX8SXcHxFK66ebvAJoxIQWTc7RobregucOM5g6L+0+7Ge1GG6YXjEBhTiLGpEcP+wuuAyGXyfzaDoIoUMSRdANksjggl8mC9iECQyEiLAQRySEMPqIgJJ7l1wAYLQ6ow5R8KU5EkiDJoDdZ7KKozxMR9YdEgz44+9wQEd0JSQZ9d+mGiEgKJBn0LN0QkZRIM+itLN0QkXRILugFQbj2GEGu6IlIGgaddqWlpWhoaIBarca8efPw5ptvAgC+//3v46233gIAvPzyy4iLGx63wlvtTjhdQtD2oiciGqhBr+grKiqwZMkSNDU1ITQ0FCtWrMDIkSPR1tbmedtoNPpirD5xvc8Ng56IpGHQaadQuO8uFQQBarUapaWliI6OxujRoz1vp6cP7JF03p592Buttv93axod7s6DIxKjBvR5w1Gwj38wpDp3zltafDXvQQe9TqdDSUkJ0tLSUFlZiXXr1qGoqAgHDx70vF1fX4/U1N4fIdcbfz4cvK6hHQDgsNmD+oHDUn1gMiDduXPe0jKsHg5eXFzc4/0NGzb0+vZwwdINEUmN5HbdGEXSi56IqL8kF/Ri6kVPRNQf0gt6ix0yuB+cQUQkBZILeqPFgbBQJeRsUUxEEiG5oGefGyKSGgkGPTtXEpG0SC7ojVYHIrjjhogkRHJBb7I42OeGiCRFckFvtNhZuiEiSZFc0JstLN0QkbRIKujtDhdsDhdX9EQkKZIKepPFDoB9bohIWiQV9EY2NCMiCZJU0F/vc8MaPRFJh7SCnqUbIpIgSQW9p3TDffREJCGSCvruh46wdENEUuI16D/88MNAjSMgWLohIinymniHDx9GTU0NZDIZVq1aFagx+Y3R4kBoiAJKhaReyBCRxHlNvAceeABKpRJFRUWBGo9fsXMlEUmR16Cvrq7Gq6++inPnzgVqPH7FPjdEJEVeg769vR07d+5EU1NToMbjV2arAxHccUNEEuM19ebPnw+VSoVZs2YFajx+ZbQ4EB8VNtTDICIKKK8r+nXr1qG8vBxbtmwJ1Hj8ysTSDRFJkNfUW7p0KZKSkgI1Fr8z8mIsEUmQ1xX92rVrsXv3buzZsydQ4/Ebp8sFi83Jm6WISHL6XN7abDb85Cc/AQC0tLQEbED+YrY6AfBmKSKSnj5X9H/605+wZ88epKamYt++fYEck18Yu++K5a4bIpIYr6Ubs9kcqHH4HfvcEJFU9bm83bt3LxQKBVpbW3HixAksW7YskOPyORMfOkJEEtVn6m3atCmQ4/C77tJNBIOeiCRGMt29rq/oWbohImnxurw9cuQIKisrAQDPPvtsr8eUlpaioaEBarUa8+bNw5tvvgkAePnll7Fp0yZYLBZ897vfRXR0tI+HPjBGtigmIonymnoHDx7EI488AkEQ+jymoqICr7zyCtasWYPQ0FCsWLEC27ZtQ11dHcaNG4fU1FQcOHAACxYs6Peg4uMj+z8DAFqt5vYHyeVQKuRIHRENmUw2oPMPV/2at0hJde6ct7T4at5eg/7s2bOem6X6WtErFAoAgCAIUKvVKC0tRXR0NORyuSdQXS7XgAbV0mKAy9X3L5cbabUa6PVdtz2uuc0EdZgSzc2GAY1luOrvvMVIqnPnvKVlIPOWy2VeF8heg/573/seurq6EBMT0+cxOp0OJSUlSEtLQ2VlJdatW4eioiJMnjwZn3/+OcLCwvD888/3a7D+ZLQ4uIeeiCTJa/J9/PHHKCoqwqeffooVK1b0ekxxcXGP9zds2OB5ezg9lcpksXPHDRFJktddN9HR0ZgxYwYiIiICNR6/cT9dijtuiEh6vAb9xIkT8frrr2P8+PGBGo/fmCwOruiJSJL6DPrS0lKUlZXB4XBg7969gRyTXxgtdoQz6IlIgvpMvqKiIsTExCAvLw9ff/11IMfkcy5BgMnKFT0RSVOfybdlyxacPHkSmZmZiI+PD+SYfM5qc0IQAHUoa/REJD19lm6WLFmC+++/HwCC/uHg7HNDRFLmNfkqKysRGxsb9O2K2bmSiKTM664bu90Ou92OK1euBGo8fmFkQzMikjCvS9zp06dj7Nix+PLLLwM1Hr+4/tARruiJSHq8ruhPnz6NyMhINDQ0BGo8fmFi50oikjCvQd/V1YV169bBYAjuRmBGPkaQiCSszyWuXq9HcXExOjs7ERUVFcgx+ZzJaodMBoSqFEM9FCKigOsz6BsbG/H1118jKSkJADB69OiADcrXTNc6V8pF0oeeiGggvN4w1djYiOzs7KB/UIe7zw3LNkQkTX0G/fLly9HW1hb0IQ9c60XPC7FEJFFeSzc1NTWe94O7dGNn0BORZPW566agoACdnZ1oaWnxPCA8WJms7EVPRNLldZlrsVhgNBoRFxcXqPH4hZG96IlIwrymX05ODkaNGoXLly8Hajw+JwgCSzdEJGle0+/SpUs4f/48rl69iqKiokCNyadsDhccToEPBiciyRJ96cbEu2KJSOJEX7rx9KIPZ9ATkTR5DfozZ85gx44dMJlMQVu6MZr50BEikjav6RcWFoaJEyfC6XQGajw+ZzC7SzeRXNETkUR57V6pUCigUqlw/vz5QI3H564/RpBBT0TS1GfQb9y4EXK5HEajESNHjgzkmHzKU7oJZ+mGiKSpz/SbPXs2HA4HHA4HLBZLIMfkUwaLHUqFDKEhbFFMRNLU54o+NTUVn332GXQ6HXbu3BnIMfmU0ezuXCmG5mxERHfCa43eZrNh165dMJvNgRqPzxnNdl6IJSJJ8xr0L774IlJSUrB06dJAjcfnjBY7t1YSkaR5TcC1a9fC6XTCarXiJz/5SaDG5FMGswPamLChHgYR0ZDxuqI3GAxQKr2vhktLS7F+/Xps2bIFAHD48GG89tprsNvt+O1vf4tf/vKX0Ov1vhvxABktdt4VS0SS1meKOxwOLF26FBqNBh0dHX2eoKKiAq+88grWrFkDAFCpVNBoNJDJZGhpaYHL5UJ4eLjvR95PRrMdkdxDT0QS1mfQ79u3DydOnPC8/4Mf/KDX4xQK97ZFQRAAuB9YUlZWhrq6OkyfPh2pqak4cOAA5s2b1+9BxcdH9vtYANBqNb1+3Gp3wuZwITEhos9jgpkY59RfUp075y0tvpp3n0G/a9cuaDQaT4D3RafToaSkBGlpabDZbFCpVACA2NhYHD16FGfOnMGiRYsGNKiWFgNcLu9ft5tWq4Fe39Xr37V1Wd1vuFx9HhOsvM1b7KQ6d85bWgYyb7lc5nWB3GfQz5gxA3V1dUhMTPS6Gi8uLr7lY8uXLwcA/Nu//Vu/Bukv3XfFsnRDRFLW58XYBQsWYOnSpRAEAS+88EIgx+QzBnauJCLqe0W/evVqGAwGTJs2DX/84x8DOSafYS96IiIvK3qZTAaNRoNTp05h7dq1gRyTzxgtbFFMRNTnir6vXTbB5HrphkFPRNLl9YapYGc026FUyKEKEfU0iYi8EnUCuu+KVbJzJRFJmqiD3mB2sD5PRJIn6qA3mu2szxOR5Ik66A1sUUxEJO6g50NHiIjEHvQWB2+WIiLJE23QW+1O2B0ulm6ISPJEG/SehmZc0RORxIk36K+1P+CuGyKSOtEGvaf9AVf0RCRxog16lm6IiNzEG/QW9qInIgJEHPQGruiJiACIOOiNFgdClHKoQhRDPRQioiEl3qA3s/0BEREg4qA3sP0BEREAEQe90eLgHnoiIog56M127qEnIoKIg95gsSMynDV6IiJRBr0gCDCaWbohIgJEGvQ2uwsOp4ulGyIiiDTou++K5a4bIiKRBr2noRn30RMRiTPojZ6g54qeiEicQX+tFz1LN0REIg16g4W96ImIug26iF1aWoqGhgao1WosXLgQhw8fxs6dO/HTn/4Ub7zxBhQKBZ599lnExcX5Yrz9YmSNnojIY9Ar+oqKCixZsgRNTU0AAJVKBY1Gg6qqKnR0dAAAwsLCBvtlBsRodkDFzpVERAB8sKJXKNxhKggCAKCgoABlZWWw2+3IyclBXl4e9u7di/vvv7/f54yPjxzQGLRaTY/3HQKgiVDd8nGxEfv8vJHq3DlvafHVvAcd9DqdDiUlJUhLS4PNZoNKpQIA5Obm4osvvkBdXR0WL148oHO2tBjgcgn9Olar1UCv7+r5+e0mhKuUt3xcTHqbt1RIde6ct7QMZN5yuczrAnnQQV9cXHzLx5YvXw4AWLVq1WBPf0eMZva5ISLqJtJdN+xzQ0TUTZRBzxbFRETXiS7oBUGA0WJHBEs3REQARBj0VrsTDqeASJZuiIgAiDDojWZ3+wOWboiI3MQX9BY2NCMiupHogr67RTG3VxIRuYku6Ls7V7J0Q0TkJr6gZy96IqIeRBf0LN0QEfUkuqA3WuxQhcgRomTnSiIiQIRBbzDbWbYhIrqB6ILeaHbwEYJERDcQX9Bb7HyyFBHRDUQX9AY2NCMi6kF0QW+0sHRDRHQjUQW9IAjuFsW8GEtE5CGqoLfYnHC6BLYoJiK6gaiCvruhGVsUExFdJ66gZ4tiIqJbiCroDd0regY9EZGHqIL+ekMz1uiJiLqJM+i5oici8hBV0Bu6e9HzYiwRkYeogt5otiM0RIEQpaimRUQ0KKJKRKPZzj30REQ3EVfQWxzcQ09EdBNRBb3BwoZmREQ3E1XQu/vcsHRDRHQj0QU9b5YiIupJNEEvCAKMFgdLN0RENxFN0Hs6V/JiLBFRD4MO+tLSUqxfvx5btmwBABw+fBivvfYaAKC+vh4rV64c7Jfol+t3xbJGT0R0o0EHfUVFBZYsWYKmpiYAgEqlgkajgc1mw549e5CRkTHoQfaH8dpdsdxeSUTU06CXvwqFAoC7Rg4ABQUFKCsrw/79+2E0GlFeXo76+nqkpqb2+5zx8ZEDGoNWq0FdqxkAkDoiGlqtZkCfH6ykMs/eSHXunLe0+Gregw56nU6HkpISpKWlwWazQaVSAQBmzpyJmTNnwmazDSjkAaClxQCXS+jXsVqtBnp9F8wmK5QKGZSCC3p914DnEWy65y1FUp075y0tA5m3XC7zukCWCd1L8WHkToJeEARYbE6Eh0qjRi/Vf/yAdOfOeUuLL4NeNLtuZDKZZEKeiGggRBP0RETUOwY9EZHIMeiJiESOQU9EJHIMeiIikWPQExGJ3LDcjyiXy/x6vFhIdd6AdOfOeUtLf+d9u+OG5Q1TRETkOyzdEBGJHIOeiEjkGPRERCLHoCciEjkGPRGRyDHoiYhEjkFPRCRyDHoiIpFj0BMRidywbIHQX6WlpWhoaIBarcbChQuHejgBsXnzZnR0dCA8PBwymUxSc3/jjTeQn58vqZ/53r17cfbsWSgUCkn9vL/44gs0Nzejo6MDUVFRkpi3xWLBz372M+Tn53t+1jExMT759x7UK/qKigosWbIETU1NQz2UgJk3bx6ef/55tLa2SmruZWVlSEhIwLp16yQ17wMHDsBut2Pr1q2Smrfdboder8fatWslM2+9Xo/c3NweuearjAvqoFcoFAAAKbXrUavVWL16NUJCQgBIZ+4HDhzA+fPnMXXqVADSmXdbWxteeuklVFVVAZDOvC9fvowf/ehHMBgMAKQx7/T0dMTExCAjIwOAe86+yrigbmq2fft2NDU1ITo6Go8++uhQDycgXn31VaSkpECn06Gjo0NScz9w4ADa2tok9TPfsWMH6urqYDabodFoJDPvzZs3w2KxwOVyQalUSmbeH330Edra2jxzDgsL88m/96AOeiIiur2gLt0QEdHtMeiJiESOQU9EJHIMeiIikWPQU9BZvXo13n33XQDAr371q359Tn+Pu1FHRwd+/OMfo7m5GQDw2GOPob6+HkajEatXr+5x7MWLF3H27Nlbxmk0GgEAdXV12LBhw4DHcCfjJrpZUN8ZS9LV0tKCyspKAO5tlyaTCWq1GiaTCVu2bMGUKVNQW1uL+Ph4zJ8/H+fOnUNJSQnCw8MRERGB+vp6mM1mTJ06FZ988gmWLFmCMWPG4PPPP8eVK1eg1+tRWFgImUyG2NhYAMDEiRPxzjvvYOXKlQCAzz77rMd5TCYT/va3v0GlUuHYsWPQ6XRYv349amtr8fLLL2Pfvn1oaWnB7NmzsX//foSFhcHlcsFoNEKhUCAiIgKCICAiIsJzF6QgCHj77bcxf/58ZGdnD803m4IeV/QUlF544QVs2rQJFosFMpkMAOByuQAAaWlpeOqppxAbG4tZs2bh4sWLKCgowHPPPYeLFy+irKwMGo0GZrMZXV1dmDNnDsaMGQMAOH36NL7zne8gIyMDUVFRmDBhguemlbCwMCxatAh//vOfAeCW8wDuVwHPPfcc4uLiAACLFy9GSkoKzGYz5s+fj2XLlqGsrAydnZ147rnn0NLSAkEQsHjxYkyaNAmtra2eeQDAmTNn0NzczJCnQWHQU1CSy+V46aWXUF5ejrS0NOzZswdlZWWevwPg+QUAAOfPn8fWrVsxevRoTJs2DUajEVFRUYiMjPQcDwC5ubkoKSlBY2MjRowYccvXzc3N9azwbz4P4L5zeePGjZ5yT/cYBEHAoUOHsHHjRsyePRuRkZHYtGkTkpKSPMdduHABsbGxaGxs9Hy9vLw8zJkzB9u2bfPVt44kiDdMEfnQe++9B6fTCZVKhSeffHKoh0MEgEFPRCR6LN0QEYkcg56ISOQY9EREIsegJyISOQY9EZHIMeiJiETufwCy2RpuyQmykwAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Optimal k is:  1\n",
      "KNN Accuracy: 75.0%\n",
      "              precision    recall  f1-score   support\n",
      "\n",
      "     class 0       0.71      0.80      0.75       246\n",
      "     class 1       0.64      0.89      0.74       231\n",
      "     class 2       0.88      0.66      0.75       479\n",
      "\n",
      "    accuracy                           0.75       956\n",
      "   macro avg       0.74      0.78      0.75       956\n",
      "weighted avg       0.78      0.75      0.75       956\n",
      "\n",
      "\n",
      "F1-score is : 0.7492637382375148\n",
      "Cross validation score is : 86.81037535410766\n"
     ]
    }
   ],
   "source": [
    "# training a KNN classifier \n",
    "myList = list(range(1,100))\n",
    "neighbors = list(filter(lambda x: x%2!=0, myList))\n",
    "cv_scores = []\n",
    "for k in neighbors:\n",
    "    knn_model = KNeighborsClassifier(n_neighbors=k)\n",
    "    scores = cross_val_score(knn_model, X_train_ros, y_train_ros, cv=10, scoring='f1_macro')\n",
    "    cv_scores.append(scores.mean())\n",
    "    \n",
    "mean_error = [1-x for x in cv_scores]\n",
    "\n",
    "plt.plot(neighbors, mean_error)\n",
    "plt.xlabel('Number of Neighbors k')\n",
    "plt.ylabel('Misclassification Error')\n",
    "plt.show()\n",
    "\n",
    "optimal_k = neighbors[mean_error.index(min(mean_error))]\n",
    "print('Optimal k is: ', optimal_k)\n",
    "\n",
    "knn_model = KNeighborsClassifier(n_neighbors = optimal_k, n_jobs=-1)\n",
    "knn_model.fit(X_train_ros, y_train_ros)\n",
    "pred_knn = knn_model.predict(X_test)\n",
    "accuracy_knn = accuracy_score(y_test, pred_knn)*100\n",
    "print(\"KNN Accuracy: {}%\".format(accuracy_knn))\n",
    "\n",
    "target_names = ['class 0','class 1','class 2']\n",
    "print(classification_report(y_test, pred_knn, target_names=target_names))\n",
    "print()\n",
    "\n",
    "f1_knn = f1_score(y_test, pred_knn, average='macro')\n",
    "cross_val_knn = np.mean(cross_val_score(knn_model, X_train_ros, y_train_ros, cv=10))*100\n",
    "print('F1-score is :', f1_knn)\n",
    "print('Cross validation score is :', cross_val_knn)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- LogisticRegression"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "LogisticRegression Accuracy: 93.82845188284519%\n",
      "              precision    recall  f1-score   support\n",
      "\n",
      "     class 0       0.91      0.96      0.93       246\n",
      "     class 1       0.88      0.93      0.90       231\n",
      "     class 2       0.98      0.94      0.96       479\n",
      "\n",
      "    accuracy                           0.94       956\n",
      "   macro avg       0.93      0.94      0.93       956\n",
      "weighted avg       0.94      0.94      0.94       956\n",
      "\n",
      "\n",
      "F1-score is : 0.9321737249057053\n",
      "Cross validation score is : 96.79508434200359\n"
     ]
    }
   ],
   "source": [
    "with ignore_warnings(category=ConvergenceWarning):\n",
    "\n",
    "    logreg = LogisticRegression()\n",
    "    logreg.fit(X_train_ros, y_train_ros) \n",
    "    pred_logreg = logreg.predict(X_test) \n",
    "    accuracy_logreg = logreg.score(X_test, y_test) * 100\n",
    "    print(\"LogisticRegression Accuracy: {}%\".format(accuracy_logreg))\n",
    "\n",
    "    target_names = ['class 0','class 1','class 2']\n",
    "    print(classification_report(y_test, pred_logreg, target_names=target_names))\n",
    "    print()\n",
    "\n",
    "    f1_logreg = f1_score(y_test, pred_logreg, average='macro')\n",
    "    cross_val_logreg = np.mean(cross_val_score(logreg, X_train_ros, y_train_ros, cv=10))*100\n",
    "    print('F1-score is :', f1_logreg)\n",
    "    print('Cross validation score is :', cross_val_logreg)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- MLPClassifier"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "MLP Accuracy: 94.0376569037657%\n",
      "              precision    recall  f1-score   support\n",
      "\n",
      "     class 0       0.92      0.95      0.93       246\n",
      "     class 1       0.89      0.94      0.92       231\n",
      "     class 2       0.98      0.94      0.96       479\n",
      "\n",
      "    accuracy                           0.94       956\n",
      "   macro avg       0.93      0.94      0.93       956\n",
      "weighted avg       0.94      0.94      0.94       956\n",
      "\n",
      "\n",
      "F1-score is : 0.9349892970350178\n",
      "Cross validation score is : 97.58925122328095\n"
     ]
    }
   ],
   "source": [
    "MLP_model = MLPClassifier()\n",
    "MLP_model.fit(X_train_ros, y_train_ros)\n",
    "\n",
    "pred_mlp = MLP_model.predict(X_test) \n",
    "\n",
    "accuracy_mlp = MLP_model.score(X_test, y_test) * 100\n",
    "print(\"MLP Accuracy: {}%\".format(accuracy_mlp))\n",
    "\n",
    "with ignore_warnings(category=ConvergenceWarning):\n",
    "    target_names = ['class 0','class 1','class 2']\n",
    "    print(classification_report(y_test, pred_mlp, target_names=target_names))\n",
    "    print()\n",
    "\n",
    "    f1_mlp = f1_score(y_test, pred_mlp, average='macro')\n",
    "    cross_val_mlp = np.mean(cross_val_score(MLP_model, X_train_ros, y_train_ros, cv=10))*100\n",
    "    print('F1-score is :', f1_mlp)\n",
    "    print('Cross validation score is :', cross_val_mlp)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- SVMClassifier rbf_kernel"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "SVM Accuracy: 95.92050209205021%\n",
      "              precision    recall  f1-score   support\n",
      "\n",
      "     class 0       0.94      0.96      0.95       246\n",
      "     class 1       0.92      0.96      0.94       231\n",
      "     class 2       0.99      0.96      0.97       479\n",
      "\n",
      "    accuracy                           0.96       956\n",
      "   macro avg       0.95      0.96      0.95       956\n",
      "weighted avg       0.96      0.96      0.96       956\n",
      "\n",
      "\n",
      "F1-score is : 0.9542643225866563\n",
      "Cross validation score is : 98.12821916044297\n"
     ]
    }
   ],
   "source": [
    "svm_model = SVC(probability=True)\n",
    "\n",
    "svm_model.fit(X_train_ros, y_train_ros)\n",
    "pred_svm = svm_model.predict(X_test)\n",
    "accuracy_svm = accuracy_score(y_test, pred_svm)*100\n",
    "print(\"SVM Accuracy: {}%\".format(accuracy_svm))\n",
    "\n",
    "target_names = ['class 0','class 1','class 2']\n",
    "print(classification_report(y_test, pred_svm, target_names=target_names))\n",
    "print()\n",
    "\n",
    "f1_svm = f1_score(y_test, pred_svm, average='macro')\n",
    "cross_val_svm = np.mean(cross_val_score(svm_model, X_train_ros, y_train_ros, cv=10))*100\n",
    "print('F1-score is :', f1_svm)\n",
    "print('Cross validation score is :', cross_val_svm)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- SVMClassifier linear_kernel"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "SVM Linear Accuracy: 92.67782426778243%\n",
      "              precision    recall  f1-score   support\n",
      "\n",
      "     class 0       0.88      0.93      0.91       246\n",
      "     class 1       0.90      0.90      0.90       231\n",
      "     class 2       0.96      0.94      0.95       479\n",
      "\n",
      "    accuracy                           0.93       956\n",
      "   macro avg       0.92      0.92      0.92       956\n",
      "weighted avg       0.93      0.93      0.93       956\n",
      "\n",
      "\n",
      "F1-score is : 0.919187460213101\n",
      "Cross validation score is : 96.51187870203451\n"
     ]
    }
   ],
   "source": [
    "#SVM Linear\n",
    "svm_model_linear = SVC(kernel = 'linear', C = 1).fit(X_train_ros, y_train_ros) \n",
    "pred_svm_lin = svm_model_linear.predict(X_test)  \n",
    "accuracy_svm_lin = svm_model_linear.score(X_test, y_test) * 100\n",
    "print(\"SVM Linear Accuracy: {}%\".format(accuracy_svm_lin))\n",
    "\n",
    "target_names = ['class 0','class 1','class 2']\n",
    "print(classification_report(y_test, pred_svm_lin, target_names=target_names))\n",
    "print()\n",
    "\n",
    "f1_svm_linear = f1_score(y_test, pred_svm_lin, average='macro')\n",
    "cross_val_svm_linear = np.mean(cross_val_score(svm_model_linear, X_train_ros, y_train_ros, cv=10))*100\n",
    "print('F1-score is :', f1_svm_linear)\n",
    "print('Cross validation score is :', cross_val_svm_linear)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- SVMClassifier polynomial_kernel"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "SVM Poly Accuracy: 94.24686192468619%\n",
      "              precision    recall  f1-score   support\n",
      "\n",
      "     class 0       0.94      0.92      0.93       246\n",
      "     class 1       0.91      0.94      0.93       231\n",
      "     class 2       0.96      0.95      0.96       479\n",
      "\n",
      "    accuracy                           0.94       956\n",
      "   macro avg       0.94      0.94      0.94       956\n",
      "weighted avg       0.94      0.94      0.94       956\n",
      "\n",
      "\n",
      "F1-score is : 0.937599745440674\n",
      "Cross validation score is : 98.07172289466907\n"
     ]
    }
   ],
   "source": [
    "svm_model_poly = SVC(kernel='poly').fit(X_train_ros, y_train_ros) \n",
    "\n",
    "pred_svm_poly = svm_model_poly.predict(X_test) \n",
    "  \n",
    "accuracy_svm_poly = svm_model_poly.score(X_test, y_test) * 100\n",
    "print(\"SVM Poly Accuracy: {}%\".format(accuracy_svm_poly))\n",
    "\n",
    "target_names = ['class 0','class 1','class 2']\n",
    "print(classification_report(y_test, pred_svm_poly, target_names=target_names))\n",
    "print()\n",
    "\n",
    "f1_svm_poly = f1_score(y_test, pred_svm_poly, average='macro')\n",
    "cross_val_svm_poly = np.mean(cross_val_score(svm_model_poly, X_train_ros, y_train_ros, cv=10))*100\n",
    "print('F1-score is :', f1_svm_poly)\n",
    "print('Cross validation score is :', cross_val_svm_poly)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- Markdown table"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "| Classifier             | Accuracy %  | F1-score | Cross validation % |\n",
    "|------------------------|-------------|----------|------------------|\n",
    "| dummy                  | 25.7322     | 0.1364     | 33.1915          |\n",
    "| Gaussian Naive Bayes   | 94.0376     | 0.9372     | 94.9786          |\n",
    "| K Neirest Neighbors    | 75.0000     | 0.7492     | 86.8103          |\n",
    "| Logistic Regression    | 93.8284     | 0.9321     | 96.7950          |\n",
    "| Multi-Layer Perceptron | 94.0376     | 0.9349     | 97.5892          |\n",
    "| SVM (kernel = rbf)     | 95.9205     | 0.9542     | 98.1282          |\n",
    "| SVM (kernel = linear)  | 92.6778     | 0.9191     | 96.5118         |\n",
    "| SVM (kernel = poly)    | 94.2468     | 0.9375     | 98.0717          |"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAtAAAAFpCAYAAAC1eh+8AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8rg+JYAAAACXBIWXMAAAsTAAALEwEAmpwYAABLZElEQVR4nO3deViU5eLG8e+wKSKKIO7gghumgIoWpqKWay5oppmpJ80ytdS0XFLDNNfMFPWcRM3cSjJDDEvTisTMpGO5b7gvIK4sItvM749+TVFWTEeYAe7PdXmd5h1m3nvmyc7Ncz3v8xpMJpMJERERERHJEztrBxARERERKUxUoEVERERELKACLSIiIiJiARVoERERERELqECLiIiIiFhABVpERERExAIq0CJS7CQlJdGgQQO6dOli7Sg25/Dhw3Tp0oWGDRsyZ84ci1+/adMmGjRoYH5cr149Nm/e/I+y/C+vtcTvM//+OxgwYACvvfZavucQkcLDwdoBREQK2ubNm/Hy8iI+Pp64uDgCAwOtHclmvPvuuzg4OLB161ZcXV2tHadAdOnShdatW5sf//47MBgMODjo/y5F5FeagRaRYicyMpIuXbrQoEEDNmzYYO04NiUlJQVfX1+8vb0pV66cteMUiJIlS1K+fHnz499/B25ubpQuXdqKCUXE1qhAi0ixcuDAAU6ePEmLFi3o0KED27Zt4/bt2+bnU1NTmTZtGi1atKBx48YMGTKE06dPm5+PiYnhiSeewN/fn3bt2rF8+XLzc/dacvDbYxMmTGD06NEMGDCApk2bsn79ejIyMpg1axZt27alYcOGPPTQQ0ycOJH09PRcmQcMGEBAQAAtW7Zk7ty5ZGdns2rVKpo3b05mZqb5Z9PS0ggICGDHjh33/PwnTpxg6NChNGvWjObNm/Pqq69y48YNANq1a8e3335LZGQk9erV4+LFi394fV7yWuKvvk9Lznvnzh0mTpxIixYtaNSoEX369GHPnj3m13/99deEhITg5+dHy5YtmT59OhkZGUDuJRz3+g5+v4QjLi6OJ598Ej8/Px555BHmz59vfi/4ecwXLlxI69atad26NUlJSWzatInOnTvTsGFD2rZty6JFizAajf/oOxMR61OBFpFi5ZNPPqF8+fI0bdqUzp07k5GRQWRkpPn50aNHs2fPHubPn8/HH39MqVKlePbZZ8nKymL//v0MGzaMhx9+mMjISCZOnMiSJUuIiIjI8/k/++wz2rdvT0REBO3bt2fOnDl89dVXzJs3j88//5ypU6cSHR1tnhm/cOECAwcOpHr16mzcuJF58+YRFRVFWFgY3bt3586dO8TExJjff/v27Tg7OxMcHPyHc1+8eJF+/fpRtmxZ1q1bx9KlSzl27BiDBw8mJyeHjRs3EhgYSOfOnYmNjaVy5cp/eI+/y2sJS77PvzvvokWLOHXqFCtWrGDr1q34+voyYsQI7ty5w40bNxg5ciRPPvkkn332GfPmzWPr1q2Eh4f/4Tx/9x0cPXqUIUOG0L59e7Zs2cKMGTP46quvCA0NzfVzH330Ee+++y6LFy/m+vXrTJ06lTFjxrB9+3YmTZrEihUriIqKsvg7ExHboEVdIlJsZGZmsnXrVrp06YKdnR01atTggQceICIigkGDBnH69Gl27drF6tWrefDBBwF44403ePfdd7l16xZr1qwhMDCQ0aNHA1CzZk1ef/117O3t85zB09OTgQMHmh/7+/vz2GOP0bRpUwCqVavG+vXrOXHiBAARERGUL1+eadOmYW9vT+3atZk+fTqXL1/G3d2dVq1aERUVRfv27YGf13d37doVR0fHP5x7/fr1lClThlmzZpmfX7BgAV26dGHXrl20adMGR0dHSpYsiaen5z3z/11eS1jyff7dec+dO4eLiwvVqlXD1dWV8ePH07FjR+zt7UlISCArK4tKlSpRtWpVqlatyvLlyylVqtQfzuPu7v6X38GKFSsIDg5myJAhAFSvXp1p06bx1FNPMWbMGCpUqABAz5498fX1BeCLL77AYDBQpUoV85/33nuPSpUqWfydiYhtUIEWkWJj586d3Lp1i06dOpmPde7cmbfeeou4uDiuXbsGgJ+fn/n5cuXKMWHCBODn5Q+/vdgMICQkxKIM1apVy/W4R48exMbGMnfuXM6ePcupU6c4f/68+edOnDjBAw88kKtUtm3b1vzPvXr14uWXXyY5OZn09HT27t3Lq6++es9znzx5kkaNGuUq1z4+PpQrV44TJ07Qpk2bv83/d3ktYcn3+XfnHTJkCMOHDycoKIjGjRvTqlUrevToQYkSJfD19aVz5848//zzVKpUiYcffpj27dvn+h7z6ujRo5w7d47GjRubj5lMJgDi4+PNBdrLy8v8fKtWrfD39+fxxx+nevXqtGzZki5dulClShWLzy8itkFLOESk2Pjkk08AeOaZZ2jQoAENGjTg7bffBn6e6f27nRYs3YkhOzv7D8dKliyZ6/HkyZMZN24cJpOJDh06sGTJEpo1a5bnc7Zp0wYXFxe2bdvGli1bqFOnTq4t2X6rRIkS9zxuNBrvOWN9L3+X1xKWfJ9/d97AwEBiYmJYsGABtWrVYt26dfTo0YNTp05hMBh45513iI6OZuDAgVy5coURI0b8YdlFXjg6OhISEkJkZKT5z+bNm9m+fTv+/v7mn/vtd12yZEnWrl3Lxo0b6dGjB0eOHOHpp5++5xISESkcNAMtIsVCUlISsbGxPPXUU/Tr1y/Xc3PmzOHzzz+nf//+ABw6dMhczlJTU2nfvj1hYWH4+Phw6NChXK9dsGABJ0+eZOnSpTg6OpKammp+7ty5c3+ZKTU1lY8//piFCxfSoUMH4OfSfeHCBfPspI+PD5999hlGoxE7u5/nPDZs2MCGDRvYtGkTjo6OdO3alR07dnD79m169uz5p+erXbs2mzdvJisry1yYT506xe3bt/Hx8fnb7zAveS3xd9+nJeddvHgxjRs3pn379rRv356MjAxatWrFV199RXp6Op9++ikTJ06kdu3aDBkyhPDwcBYvXmxxia5duzbx8fFUr17dfGz//v2Eh4czbdq0ey4L2b17Nz/++CMjRoygUaNG5vL+ySefMHToUIvOLyK2QTPQIlIsbN68GZPJxLPPPkvdunVz/Xn22WfJyMjgp59+4pFHHmHatGnExcURHx/PxIkTcXV1xc/Pj8GDB7Nv3z6WLl3KuXPn2LZtG6tXr6Zdu3YABAQEEBERwbFjxzh8+DCvv/46Tk5Of5qpRIkSlCpVip07d3L+/HmOHDnC2LFjuXLlinlnjf79+3Pt2jWmT59OfHw8u3fvJiwsLNdFgr169eLbb7/l0KFDdOvW7U/P9/TTT5OSksLEiRM5efIkcXFxjBs3jvr16xMUFPS332Fe8lri775PS8576dIlpk2bxt69e7l06RJRUVGkpKTg7++Pq6sr69at4+233+b8+fMcPXqUr776KtdSnbwaOnQoBw4cYNasWcTHx/P9998zfvx4UlJS/nTduKOjI0uWLGH16tVcuHCB/fv3s3fv3lwz1iJSuKhAi0ixEBkZSZs2bahateofngsKCqJ+/fpEREQwe/ZsGjVqxPDhw+nTpw9ZWVksX74cJycnHnjgAcLCwvj888957LHHmDdvHmPGjKF3794AhIaG4urqyhNPPMFLL71Enz59/vJCMUdHR9555x0OHz5M165dGT58OGXLlmXw4MHmmdmKFSsSHh7O0aNHCQkJYdKkSfTu3ZuRI0ea36dBgwbUqFGDli1b5trP+PfKly/PypUrSUxM5PHHH2fEiBH4+vry3nvv5WkJR17yWuLvvk9Lzjt58mQeeughxo4dS8eOHVm1ahWzZs2iefPm1KhRgyVLlrB79266d+/OwIEDqVSpknn5jiXq1avHu+++y3//+19CQkIYPXo0zZo1Y/HixX/6mubNmzNz5kwiIiJ47LHHGDFiBM2aNdPdDUUKMYPpl6sfRESkUMrOzqZNmzZMnTrVvMRBRETyj9ZAi4gUUpmZmXz55Zfs2rULJyenf7SrhIiIWE4z0CIihZTRaKRVq1Y4OTkxb948AgMDrR1JRKRYUIEWEREREbGALiIUEREREbGACrSIiIiIiAVUoEVERERELKBdOPLRzZtpGI1aYl4ceHiU5vr11L//QSkSNN7Fi8a7+NBYFy+/H287OwPlyrnk6bUq0PnIaDSpQBcjGuviReNdvGi8iw+NdfHyT8dbBTofeXiUtnYEKUCenq7WjgDA3YwsUpLvWjuGiIhIkaUCnY9emhXJtZtp1o4hxcz6uf1JQQVaREQkv+giQhERERERC6hAi4iIiIhYQAVaRERERMQCKtAiIiIiIhZQgRYRERERsYAKtIiIiIiIBVSgRUREREQsoAItIiIiImIBFWgREREREQuoQIuIiIiIWEAF+k8YjUZrRxARERERG6QC/Sfs7H79akwmkxWTiIiIiIgtUYH+jd8W5ZSUFGbMmEFOTg4Gg8GKqURERETElqhA82txNhgMZGdnYzKZyMnJwWQycfv2bSunExERERFbUqwL9G+LM8APP/zA7Nmz2b17N05OTpw+fRoXFxdrRhQRERERG1MsC7TJZMJkMpmL89atW1m4cCHlypWjWbNm/Pvf/+bChQtUrVqVXbt2WTmtiIiIiNgSB2sHKGhGo9F8geDly5f573//y+7du8nIyODTTz9l8ODBlCxZktWrVxMfH89jjz0GkKtwi4iIiEjxVewKtJ2dHenp6Sxfvpxvv/0WT09PXnvtNVxcXHjrrbfYvXs3HTt2JCAggNGjR7N9+3aCgoJUnkVEREQEKAZLOH6/n/Phw4cZOXIkLi4uvPPOO2RnZ3Po0CFKly6Nv78/u3fv5siRI5QtW5YpU6ZQvnx5KyUXEREREVtUZGegf7lA8JflGnv27CE7O5ugoCCys7NxdnamYsWKtG3blt27d9OwYUPat29PdnY23t7epKSksGDBApo1a2bNjyEiIiIiNqZIFuicnBzs7e0BuHPnDitXriQuLo7bt28zcOBARowYwerVq+nZsychISF89dVX/Pjjj3Ts2JEnnngCgIyMDF599VW8vLys+VFERERExMYUyQJtb29PRkYGCxcuJDU1laysLFatWsWpU6d47rnn+OKLL4iIiGD16tU899xzTJ48mSpVqphfbzKZKFGihMqziIiIiPxBkVwDffLkSZ555hkqVKhA9+7diY6O5vz589SuXRt/f39mzZrFSy+9RMmSJQHM5fn3+0KLiIiIiPxeoS7Qd+7cASA7OxuAixcvEhUVRYkSJTh//jwBAQEEBgbyzDPP8NZbbwEwdepUkpKSqFq1KgMHDsz1firOIiIiIvJ3Cm2B3rp1K88//3yuYwcOHCAuLo7k5GRGjhzJypUrAXjppZfYu3cv33//PeXKlWPhwoXmNdK/zDqLiIiIiORFoSvQOTk5AHTp0oXbt28TGhrKBx98AMBDDz2Ej48PMTExtG3blszMTCIiIrC3t2f16tU0b978D++jWWcRERERsUShK9C/zBzv3buX2rVrs23bNr788kuOHj2Ku7s7tWrV4vTp0xw9epR+/fpx69YtAOrVqwf8OuP8y/uIiIiIiFii0O3CkZ2dzfTp07l16xZPPvkk2dnZfP/993zxxRf4+vri7OxMZmYmly9f5oknniA4ODjX6zXjLCIiIiL/i0JXoB0cHHBycuKZZ54hICAALy8vBg4cyMGDB3nppZe4cuUK48aN48EHHzS/xmg0mm+oUpAWTQwp8HOK3M3IsnYEERGRIq3QFeicnBwMBgOJiYkAVKpUCQ8PDypVqsS//vUvfHx8zD9rMpkwGAxWKc8A16+nYjTqIsXiwNPTlaSkFGvHEBERkQJQ6Aq0vb09Xbp0YfPmzXz//fdcuXKFLl260K1bN8qXLw/8eidCLdcQERERkfut0BVogICAALy9vdm6dSuPPfYYTZo0AX5dqqELBEVEREQkvxTKAg3g7u7O008/bX5sMpmstlRDRERERIqPQt84dfttERERESlIhb5AqziLiIiISEEymHQvaxERERHJR9mZGdy8nWntGLn8fgctOzsDHh6l8/TaQrsGujA4+J/xZCZft3YMEREREatq+upywLYK9P+i0C/hEBEREREpSCrQIiIiIiIWUIEWEREREbGACrSIiIiIiAVUoEVERERELKACLSIiIiJiARVoERERERELqECLiIiIiFhABVpERERExAIq0CIiIiIiFlCBFhERERGxgAr0PZw8eRKj0QiAyWSychoRERERsSUO1g5gS7799luWLVtGqVKlcHZ2ZvTo0Xh5eVk7loiIiIjYEBVoIC0tjfHjx5OcnMy4ceNo1KgRixYt4vDhwyrQIiIiIpJLsV7CkZmZyYkTJ8jJycHV1ZXWrVvj5+dHSkoKu3btIjk5mWPHjgGYl3SIiIiISPFWrAt0TEwMkydPxmg00rt3b44ePcrbb7/N+PHj8fHx4datW/Tq1QuTyYSdXbH+qkRERETk/xW7JRznz5/H29sbgPbt2/Ppp58SHR1Nr169qFmzJhs3buT999+nevXqAHz33XccPnyYhg0bWjO2iIiIiNiIYjWt+uWXX/LUU08xffp09uzZA8CIESOIiori2rVrdOrUiZYtW5KVlUVKSgqvvfYaJUuWNBduEREREZEiPwOdk5PD8uXLKV++POfPn6dDhw6cP3+eGTNm0KRJEwYPHkz9+vX59NNPGTRoEE2bNmXIkCF4eXnRrVs3+vbta+2PICIiIiI2pEgX6OjoaDZs2EC9evWoXbs2d+7coUKFCvj4+FCzZk0SExNZtmwZ8fHx7Ny5k6ZNm9KqVSuSk5Pp3r075cqVs/ZHEBEREREbU2QLdHJyMjExMYwbNw4/Pz8ALl++zKJFiyhVqhSOjo706dOHnj17EhERwZo1a4iMjGTmzJkMGjTIyulFRERExFYV2QL9xRdfYDKZ8PPzIzMzkwULFnDixAnS0tJwdHSkXLly7Nmzh6CgIPr06UPv3r2104aIiIiI/K0i2xhr1qyJq6srqampODk50bp1a1asWMGMGTOIj4/n6tWrHD9+nLt37wKoPIuIiIhInhTZGWgPDw9KlSpFXFwcbdq0ISgoCIAff/yRQYMGERwcTLVq1ShZsqSVk4qIiIhIYVJkC7S3tzc+Pj5s3ryZGzduUKdOHVavXs2tW7d4+eWXqV+/vrUjioiIiEghVGQLtMFgoGfPnnh4eHDw4EF2795NcHAw3bt3t3Y0ERERESnEimyB/kXr1q1p3bo1RqNR65xFRERE5H9mMJlMJmuHEBEREZGiKzszg5u3M60dIxdPT1eSklLMj+3sDHh4lM7Ta4v8DLQ1Xb+eitGo30+Kg9//JZSiTeNdvGi8iw+NteSV1jSIiIiIiFhABVpERERExAIq0CIiIiIiFlCBFhERERGxgAq0iIiIiIgFVKBFRERERCygbezyUV73EpSiwdPT1doRpABpvIsXjXfBysjMJPl2hrVjiPwpFeh8NO6jaVxLvWHtGCIiIoXKqmcWAirQYru0hENERERExAIq0CIiIiIiFlCBFhERERGxgAq0iIiIiIgFVKBFRERERCygAi0iIiIiYgEVaBERERERC6hAi4iIiIhYQAVaRERERMQCKtAiIiIiIhZQgf4T+/fv58YN3YZbRERERHJTgf4Nk8kEQEJCAmFhYRw8eJCcnBwrpxIRERERW6IC/f8uXbqEwWAAoFKlSrRt25bdu3eTmJho5WQiIiIiYktUoIGkpCQGDx5Mnz59+OCDDwDo27cvt27d4ocffiA9Pd3KCUVERETEVhTrAv3DDz8wadIkPD096d27N6VKleKjjz5i7NixfPbZZwQEBHDkyBEuXrxo7agiIiIiYiMcrB3AmgICAhgxYgQHDhygc+fOGI1G3N3dadKkCevWrePkyZPEx8fj6OjIiBEjKFGihLUji4iIiIiVFasZ6GvXrrFy5Uri4+PJzs7G3t6eF198kTfffJNq1apRqVIljh07RunSpZk6dSqjRo2iXbt27N69m4yMDGvHFxEREREbUGxmoNetW8dHH31EUlISUVFR1KtXj5dffpn+/fuzdetWNm/ezKOPPsrZs2eJjo5m8ODBBAYGEhgYaO3oIiIiImJDisUM9N27d/nxxx+ZP38+ISEhhISEUKJECWbPns2cOXMIDg5m+fLllCpVCj8/P06fPs2FCxesHVtEREREbFCRLdA3btxg8eLFmEwmbt++TVJSEj4+PgQEBHD+/Hl69+7Nm2++Sbly5Thw4AAnT55k+vTptG3bllGjRuHl5WXtjyAiIiIiNqjILuHIysri3LlzbN68GU9PT/Px9u3bc+TIEXbv3k3NmjV57rnnuH37Nl27diUrKwsg18+LiIiIiPxWkS3QFSpUoFu3bmzevJmcnBxOnz7Na6+9RkBAAMHBwWzatIm4uDiCg4MpW7YsnTp1snZkERERESkEimyBNhgMNGnShNjYWGJjY9m4cSMnT54kPDyckiVL8vXXX9OwYUNrxxQRERGRQqbIFmiA0qVL061bN44ePcqNGzd4+OGHCQwM5PTp0zzzzDM8+OCD1o4oIiIiIoVMkb2I8Bf169fnkUceYcWKFQCUKFECX19flWcRERER+UeK9Aw0gKOjI23atMHT0xOTyYTBYLB2JBEREREpxIp8gQaoUaMGNWrUsHYMERERESkCDCaTyWTtECIiIiK/yMjMJPl2RoGf19PTlaSklAI/r1jH78fbzs6Ah0fpPL22WMxAW8v166kYjfr9pDjQf3SLF4138aLxFpHfK/IXEYqIiIiI3E8q0CIiIiIiFlCBFhERERGxgAq0iIiIiIgFVKBFRERERCygAi0iIiIiYgHtAy0iIiIi+Sbrbga3UjKtHeMPtA+0jfrq5XGkX7tu7RgiIiIiVtNl9XtggwX6f6ElHCIiIiIiFshTgf7yyy/JycnJ7ywiIiIiIjYvT0s4xo4dS6lSpejatSs9e/akfv36+Z1LRERERMQm5WkGevfu3YwbN44TJ07Qq1cvQkJCWLVqFTdu3MjvfCIiIiIiNiVPBbpUqVL07NmT9957j6+++opu3brx+eef06ZNG4YPH86OHTu0xENEREREigWLLyJ0cXHBzc0NNzc3AC5cuEBoaCgdOnRg//799zufiIiIiIhNydMa6OzsbL7++muioqKIiYnBxcWFrl27MmrUKHx9fcnOzub1119n7NixfPnll/mdWURERETEavJUoB9++GHS0tJo3bo18+fPp02bNjg4/PpSBwcHWrVqRUxMTL4FFRERERGxBXkq0C+88ALdu3fH3d39T3+mXbt2dOzY8b4FExERERGxRXlaAz1gwABWr17N+vXrzcd69erF4sWL+eVO4E5OThgMhvxJaUW607mIiIiI/FaeCvTbb7/Nxo0bqVq1qvlY37592bBhA4sXL863cNYSFxfHe++9x08//URWVpa144iIiIiIDclTgd6yZQvz588nODjYfKxv377Mnj2bTZs25Vu4gnbp0iUmTZrE8uXLSU5OZs+ePaSnp1s7loiIiIjYkDytgU5JSaF8+fJ/OF65cuUiczOV7Oxs3nvvPXx9fRkwYAAARqORqKgo/Pz8qFWrFkajETs7i3f+ExEREZEiJE9tsFGjRrz//vt/WA+8bt06GjRokC/BCsrRo0cBSEhI4MCBAzz11FMAxMTE8NRTT/HJJ58QGhoKoPIsIiIiInmbgR43bhyDBg3iu+++44EHHgDgyJEjJCUlsXz58nwNmF8OHDjAsmXLcHR0ZM6cOWRmZuLv709qaiply5bFw8OD119/HV9fXx555BH27NlDUFCQtWOLiIiIiJXlqUD7+fkRFRVFREQEJ0+exMHBgU6dOvHUU09RsWLF/M54XyUlJbFkyRISEhIICQmhU6dOwM93WDx37hznzp2jUaNGNGzYEID4+HiCgoJo1KiRNWOLiIiIiI3IU4EG8PLyYuzYsfmZpUD8+9//ZteuXezcudN87KOPPuKxxx6jVatWbNmyhbi4OAYMGMC6devYunUrPXr0oHTp0lZMLSIiIiK2Ik8FOiMjgw0bNnDixAlycnLMxzMzMzl06BDbtm3Lt4D3wxdffMHly5fp1asXgwcP5vz585w7d46LFy8SHh5O7dq16dGjB/379+fQoUNs3LiRqVOnUrJkSZYuXXrPCyhFREREpHjKU4GeNm0a0dHR+Pn58cMPPxAYGMiFCxdISEjgmWeeye+M/9jx48cJDw8nOTkZDw8PlixZwnPPPUfbtm15/PHHCQ4OZtSoUTRu3BiAq1ev4ufnh5+fH3fu3KFUqVJW/gQiIiIiYmvytK3EV199xezZs1mzZg1eXl68/vrr7Nixgw4dOnDnzp38zmgxo9HI/PnzGTNmDJ06dWLZsmW88sornDx5EoCQkBB8fX3p1q0bjRs35vr168yYMYOpU6dy69YtAJVnEREREbmnPO8D7e/vD0Dt2rU5dOgQPj4+PP/884wcOTJfA/4TRqMRBwcHmjZtyqOPPgrAl19+SeXKlXF2dsbZ2ZlBgwbx3nvvcfbsWWJiYggKCmL8+PE4OjpaOb2IiIiI2LI8FegKFSqQmJhIlSpVqFGjBsePHwfA1dXVZm6kEhMTw+eff0758uUZMGAAISEhrFixgrlz5xIfH09SUhIpKSmsWLGCkJAQgoKC2Lp1K3v27OHNN9+kSpUq1v4IIiIiIlII5KlAt2/fngkTJjB79mxatGjBK6+8QpMmTdi5cydeXl75nfEvXb16lTlz5nDnzh0GDRrEoUOHMBgMuLu707x5c2bPns2bb75JcHAw8fHxrF+/nnHjxtGmTRtmzpxJyZIlrZpfRERERAqXPBXosWPHkp2dzcWLF+nWrRvt2rXjpZdewsXFhXfeeSefI/612NhYvL29GTVqFAABAQFEREQAEBgYSK9evUhKSgLAx8eHKVOmcPnyZc04i4iIiMg/kqcCvWnTJoYPH46HhwcAM2bMYOLEiZQoUQIHhzxvJX3fpaenExsba7799qFDh1i2bBkAlSpVokKFCgQFBbF8+XKaNWtG9erVAVSeRUREROQfy9MuHPPnzyc5OTnXMRcXlwIvz/v27WPixImEh4ezf/9+nJ2dKVu2LD/99BMAlStXZsKECSxatIiMjAwSExOpV68e3bp1041QREREROS+yFMD9vX15dtvv6VmzZr5neee0tPTmTdvHhcvXmTAgAHEx8fzySefsH37dnr27El4eDg3b940z5AnJiZy8+ZN6tevj7u7OyEhIVbJLSIiIiJFj8FkMpn+7ofGjBlj3uHCy8vrDxferVy5Mt8CAhw8eJCIiAimT59uPpaUlERoaChdunQhISGBb7/9lieffJLTp08TGxtLmzZtGDJkSL7mEhEREZG/lnU3g1spmdaO8Qeenq4kJaWYH9vZGfDwyNuKhTzNQJcsWdIqs7g5OTnY29tz6tQpKlasCPx8+3AnJyc8PT0JDg7m9u3bDBkyBE9PT65cuUJmZiZhYWG4ubkVeN7fu349FaPxb38/kSLg938JpWjTeBcvGu/iQ2MteZWnAj1r1qz8zmF248YN1q9fz/PPP2++qUlSUhKXL18GwM7ODqPRiJ2dHT/99BMNGjQAoHv37gWWUURERESKrzwV6C1btvzl8926dbsvYQCysrK4cOECkZGRPPHEEwAMGDCAjh07cuzYMerXrw/A7du3MRqNNG3a9L6dW0RERETk7+SpQL/yyiv3PF6iRAkqVap0Xwt0hQoV6NKlC5GRkbRu3ZqKFSvi7OzMCy+8QFhYGJUrV6ZChQp8/fXXtGrVylyoRUREREQKQp4K9LFjx3I9zsnJ4ezZs4SGhtK3b9/7GshgMNC0aVP279/Phg0beOmllwDo168fNWrUIDk5mfPnz/P2229TqVKl+3puEREREZG/k6d9oH/P3t4eHx8fJkyYwMKFC+93JkqXLk27du24cOECly5d4uLFiwwePJhvvvmG9u3bM3ToUJVnEREREbGK/+lOKPb29ly9evV+Zcmlfv36NGjQgD59+uDl5UXfvn3p2bNnvpwrv+R1KxQpGjw9Xa0dQQqQxrt40XgXHxrr/JGZkc3t5HRrx7hv/vFFhKmpqURERODn53ffQwE4OTnx6KOPYmdnR79+/XBycsqX8+SnJfO2cvvWHWvHEBEREbGqSW/2tnaE++ofX0To4OBA48aNCQ0Nvd+ZzLy8vBg0aFC+vb+IiIiIiKX+0UWEIiIiIiLFVZ4vIvzoo4+Ijo42Px45ciSffPJJvoQSEREREbFVeSrQK1asYObMmWRnZ5uP+fj48MYbb7Bu3bp8CyciIiIiYmvyVKDXr1/PvHnz6NGjh/nYmDFjmD17Nu+//36+hRMRERERsTV5KtDXr1+nTp06fzju6+tLQkLCfQ8lIiIiImKr8lSg69atS1RU1B+OR0dHU6tWrfseSkRERETEVuVpF44RI0bwwgsvsG/fPvO+z4cOHWLfvn2EhYXla0AREREREVuSpxno4OBg1q1bh6enJzExMezevRsPDw8++ugj2rVrl98ZRURERERsRp5v5e3n58ekSZPw8PAA4L///e8910WLiIiIiBRleZqBPnPmDB06dGD58uXmYyNHjqRbt25cuHAh38JZm9FoBMBkMlk5iYiIiIjYijwV6BkzZvDAAw/w/PPPm49t376dOnXqMHPmzHwLZy1ff/01Y8eOZenSpdy9exeDwaASLSIiIiJAHgv0/v37efnll3FzczMfK126NKNHjyYuLi6/shW4+Ph4Jk6cyKZNm+jbty/x8fFMnToVAIPBYOV0IiIiImIL8rQG2tnZmatXr1KjRo1cx2/evImdXZ7vBm7TLly4wOuvv46XlxeLFi0CwNHRkZiYGEwmkwq0iIiIiAB5nIHu0KEDoaGhxMXFkZGRQUZGBnFxcUybNo1HHnkkvzPmq1+WZri5udGpUyfq1atHVlYWmzdvZv78+TRu3FjlWURERETM8jQDPW7cOEaNGsXTTz9tLpMmk4lHH32USZMm5WvA/HL8+HHq1atnXt/s6uqKn58fkZGRdO/enWbNmjFt2jR8fHysHVVEREREbEieCrSLiwvLly/n9OnTnDx5EgcHBzw9Pfnpp5/o168fW7Zsye+c983Vq1fZtm0bCxYs4I033qBr167k5OTg4OBAgwYNOHPmDNevX2fMmDGUK1eO7Oxs7O3tNQstIiIiIkAel3D8olatWnh6evLFF18wcOBA3nzzTezt7fMrW76IiIggPDyc5s2bs3btWrKzs3FwcMBoNOLg4ICfnx/16tVj8+bNADg4OKg8i4iIiIhZngp0SkoKa9asoVu3bvTv35/NmzcTGBjIqlWriIyMzOeI/7vs7GzzP3fo0IG6devi5eXFxYsXef/99wHIyckBoGbNmubnzp07Z5W8IiIiImK7/rJA//DDD4wfP55WrVrx5ptv4ujoyMsvv4ydnR0TJkzgoYceKqic/0hSUhK9e/dm5cqV3Lp1C4Bbt25RtWpVgoKCaNCgAR988AEpKSk4Ojqyb98+Zs2aRalSpRg+fDjVq1e37gcQEREREZvzp2ugu3btSnx8PL6+vgwbNozOnTubC+U777xTUPn+J/b29pQoUYIPPviAhIQEpk6dSmBgIOHh4WRmZhIUFMSZM2eYPn06ZcuW5eDBgwwcOLDQ7ywiIiIiIvnnTwv06dOnqV69Om3btiUwMLBQzsaWK1eOYcOGsWXLFs6dO8eqVauoX78+ffv2pUSJElStWpU6derw3//+l969e/Phhx9aO7KIiIiI2Lg/LdDffPMNmzdvJjIykqVLl+Lh4UGnTp3o2LFjobmozmAw4O/vz4kTJ7h48SIBAQHMmDEDb29vnnvuOcqVK4fBYKBp06a4u7tbO66IiIiIFAJ/uga6fPnyDBkyhC1btrBhwwbat2/Pli1bGDhwIDk5OXz44YdcuXKlILP+I2XKlKFZs2bcvn2b8uXL88ILL3Dy5El27txJxYoVad++vcqziIiIiOSZwfTLrfjyICsrix07dhAZGUlsbCwAbdu2ZfHixfkW8H7IzMxkzZo1nDp1ilmzZpGQkEClSpXy/bxL5m3l9q07+X4eEREREVs26c3eJCWlWDtGLp6errky2dkZ8PAonafX5ulGKr9wdHSkc+fOdO7cmWvXrhEZGWneL9mWOTk58cgjj1CpUiVMJlOBlGcRERERKZosKtC/Vb58eZ599lmeffbZ+5kn39SoUYMaNWpYO4aIiIiIFHIW3YlQRERERKS4s2gNtIiIiIiIpTIzsrmdnG7tGLkU2Bposcz166kYjfr9pDj4/V9CKdo03sWLxrv40FhLXmkJh4iIiIiIBVSgRUREREQsoAItIiIiImIBFWgREREREQuoQIuIiIiIWEDb2ImIiIiITcnMyOB2cma+nkPb2Nmo5XMmkXzrurVjiIiIiBQqL896F8jfAv2/0BIOERERERELqECLiIiIiFhABVpERERExAIq0CIiIiIiFlCBFhERERGxgAq0iIiIiIgFVKBFRERERCygAi0iIiIiYgEVaBERERERC6hAi4iIiIhYQAX6Ho4cOWLtCCIiIiJioxysHcCWXLhwAYAdO3bw/fff8+ijj1KtWjUrpxIRERERW6IZaOD06dOMHj2a9evXU7FiRYYMGYKzszNhYWEcPHgQAKPRaOWUIiIiImILivUMdGZmJnPnzuXs2bN07NiRJ554AgAnJyd69uxJdnY2M2fO5IMPPsDOTr9riIiIiEgxn4GOiYnhhx9+YNSoUebyvHv3buDnEt2/f38cHByIjIwEwGQyWSuqiIiIiNiIYjcDHRsby927d2nYsCEPP/ww8fHxHDt2jLS0NNasWUPZsmVp0qQJjo6OODg4MGTIEGJjY+nRowcGg8Ha8UVERETEyopNgT5//jyrV6/m6NGjtGjRgrfeeouoqCgCAgJ49913uXnzJq+//jqNGzfO9bqSJUtSrVo1DAYDJpNJJVpERESkmCsWBfrEiROEhITw6quvMnnyZAASExM5c+YM/v7+tGjRgvT0dHN5zsrKwtHREYAKFSoQGRmJ0WjUOmgRERERKdproL/77jtSUlLw8fGhR48elC1bFoB169axbds2PvnkEw4fPkxQUBAZGRls2rQJwFyeAdzc3OjevTsmk0lroEVERESkaM5AHzhwgCVLlnD79m3q1q1Lr169GDRoEK+88gqbN2+mWrVqhIeHs2/fPmbMmMH7779P5cqVSU9PJzs7GweHX78Wd3d3WrRoYcVPIyIiIiK2pEgV6JSUFEJDQ8nIyKBXr1507NiRUaNGcfLkSZ544glatmzJgQMHmDFjBgCVK1dm37592Nvb8/jjj+Ps7GzlTyAiIiIitq7IFOhTp06RlJTEgQMHWLFiBd7e3gB4e3vj4eEBwL/+9S8mTZrEvn37OH36NJs2bSIkJITSpUub30cXCoqIiIjIXykSBXrv3r1cvnyZrl270rNnT6Kjo6lfvz5hYWHEx8dTtmxZvLy8qFOnDh06dGDAgAEMGjSIJUuWUL58+VzvpfIsIiIiIn+l0Bbo48ePk5CQQHBwMPDzBYM9e/YkODiYqVOnEhMTQ1hYGJcuXWLfvn0888wztG7dmpdeeono6Gh8fHyAn2/RbTAYVJxFREREJE8KZYG+ceMG06ZN4+LFi+zYsYOqVatSpUoVkpKSqFevHj169ODMmTNUrVqVqlWr0rx5cx5++GHu3r1LpUqVAMy7amhrOhERERGxRKFsj+7u7vj7+5ORkcHy5ctxc3PjzJkzXL9+HQcHB1q3bk12djYbNmwwvyYgIICHHnrI/NhgMKg8i4iIiIjFCkWD3LVrF6NHj+bAgQPmY0FBQYwePZqNGzdy584d6taty9atW0lNTaV69eo0a9aMtLQ07d0sIiIiIveVwWSjDdNkMpGYmMiBAwfIyclhzJgxBAUFMX78eOrUqcOGDRuoUKECSUlJ7N27l7lz5xIaGoq/vz8dOnSgXLly1v4IIiIiIvIPZGZkcDs5M1/P4enpSlJSivmxnZ0BD4/Sf/GKX9lcgf7tNnLx8fHmi/5efPFFrl69SkhICBkZGYSEhDBlyhTeeecdQkJCGDduHI0aNeKTTz7h5MmTDB061HyhoLVcv56K0WhTX6/kk9//JZSiTeNdvGi8iw+NdfHyvxRom1vC8Ut5NplM2NvbExQUhNFoZPDgwSQkJNCuXTt27tzJZ599RtWqVbG3t2fo0KEcP34cd3d3hgwZQtu2bbl586aWb4iIiIjIfWczBdpoNDJt2jSWLFlCeno6BoMBV1dXDh48SEJCAo0bN+aBBx5gy5YtLFy4kPj4eFatWkVqairdu3fnueeeM79Xhw4dCAwM1NZ0IiIiInLfWb1AX758mcmTJzNv3jwOHjzI6dOnCQ8P59y5c3h4eODv709sbCwAoaGhLFu2jNTUVCZPnszatWspXbo0RqMRwPy/Ks4iIiIikl+sWqBjYmJ47bXXaNiwIUOGDGHUqFG8+OKL2NvbM2XKFBISEnB3d6dmzZoAVKhQgf79+3PkyBEAAgMDAczb0WlbOhERERHJb1a9kcqhQ4fo1q0bvXr1An6eQU5OTmbEiBEYDAaioqKIjo4mJyeHZs2aATBq1ChrRhYRERGRYs6qBbpGjRqEhoYSHx/Pd999R2ZmJh4eHri6uhIWFsaJEyfYsmULRqOR7OxsHBx+jfvb3TpERERERAqK1bex27NnD2lpadjb29O2bVsAevfuTVhYGJUrV+bChQt4eXlZM6KIiIiI/IXszGxu3k63dgyL/C/b2Fl1BhqgSZMmJCYm4u3tDcDKlSupW7cubm5uAObybDQaC90a5yNrviMr5a61Y4iIiIjkK//hbawdoUBZvUCnp6ezatUqEhMTSUpKwtfXl1GjRuHs7Jzr5wpbeRYRERGRosnqBdrNzY3Ro0dz+PBhXFxc8PPzAwrnjLOIiIiIFH1WL9AAZcqUISgoyPxY5VlEREREbJVNtlSVZxERERGxVWqqIiIiIiIWUIEWEREREbGACrSIiIiIiAVUoEVERERELKACLSIiIiJiARVoERERERELqEDfw927v95+22QyWTGJiIiIiNgaFejfMJlMXLt2jXnz5hEZGUlqaioGg8HasURERETEhqhA/79jx46xZMkSXFxcePXVVzl//jxvvfUWKSkp1o4mIiIiIjZEBfr/paSkkJqayvbt2ylRogQjR44kIyODjz76iMTERGvHExEREREbUWwL9NWrVwkPDychIQGAJk2aUL9+fY4dO8apU6ews7Ojb9++XL16lWPHjlk5rYiIiIjYimJXoI1GIzdu3ODq1avMnz+fHj16sGbNGo4ePUqLFi0oU6YMsbGxAAQEBODi4sLFixcBXVAoIiIiIsWwQM+fP5+PP/6YGjVqMGHCBKpVq0bZsmUZP348q1evxmg0cuXKFY4cOQJAq1at2LZtG4AuKBQRERGR4lOgc3JyAGjdujVnz57l5MmTdOrUCRcXFxo1asSGDRtwd3dnz549rFu3jm3btpGZmUlAQAAPPvggd+7csfInEBERERFbUKQLdFpaGjExMcCvs8cPPvggFSpU4Ntvv8XJyYnu3bszZ84cSpcuzeDBg3n33XcZNmwYLVq0wMnJiQsXLnDnzh1KlSplzY8iIiIiIjaiyBbokydPcuDAAVavXs358+exs7MjKysLgJ49e3Lu3DkOHjxIhw4dcHR0ZM2aNQC4uLgwcuRIHnzwQQC8vLx45ZVXrPY5RERERMS2FMkCvXfvXg4fPkzTpk1p3LgxGzduBMDR0ZGsrCy8vb0JCAjgu+++4/bt2/Tt25fbt2/neg9dMCgiIiIi91JkCvTx48fNyzUA9uzZg5OTE23btuXSpUt8//33wM8lGqBTp05cunSJ06dP06JFC0aOHJnr/XTBoIiIiIjcS5Eo0Ddu3GDatGlMmTKFzMxMqlatSpUqVbh27Rp16tShadOmfPrpp8DP66LHjx/Pvn37ePXVVwkODsbO7uevwWg0WvNjiIiIiEghUCQKtLu7O/7+/mRkZLB8+XLc3Nw4c+YM165dw8nJiRYtWmAymRg2bBjDhg3D09OTRx55hGrVqgG/Ltf4pUiLiIiIiPyZQtkYd+3axejRozlw4ID5WFBQEKNHj2bjxo3cuXOHunXrsnXrVlJTU6levTpNmzbFycmJmTNnMm7cOBwcHMzFWcs1RERERCSvCk2BNplMJCQksH37dlJTU/n8889ZsGABx44dIycnh4sXL+Lp6cnQoUOZOXMmzz77LNeuXSM6OpqbN28SEhLCokWL8PLywmg0YjKZVJxFRERExGI2X6B/O0uclpbGwoUL6dy5M+3bt+fOnTvs37+fNWvW0KVLFzZv3kyfPn2Ij49nz549jBs3jtTUVObOnUt8fLz5/ezs7FSeRUREROQfcbB2gL/zS9E1mUzY29sTFBSE0Whk8ODBjB49mnbt2jFu3DhKlChB1apVsbe3Z+jQoRw/fpzg4GCGDBnCtm3buHnzZoHPOjcY8FCBnUtERETEWrIzs60doUDZbIE2Go1Mnz6d8uXLM3jwYJydnXF1deXgwYMkJCTQuHFjHnjgAbZs2cLChQtZunQpa9euZeTIkXTv3j3Xe3Xo0MEqM87Xr6diNGo/6eLA09OVpKQUa8eQAqLxLl403sWHxlryyuaWcFy+fJnJkyczb948Dh48yOnTpwkPD+fcuXN4eHjg7+9PbGwsAKGhoSxbtozU1FQmT57M2rVrKV26tHk7ul/+V8s1REREROR+sakCHRMTw2uvvUbDhg0ZMmQIo0aN4sUXX8Te3p4pU6aQkJCAu7s7NWvWBKBChQr079+fI0eOABAYGAj8uh2dtqUTERERkfvNppZwHDp0iG7dutGrVy/g5xnk5ORkRowYgcFgICoqiujoaHJycmjWrBkAo0aNsmZkERERESlmbKpA16hRg9DQUOLj4/nuu+/IzMzEw8MDV1dXwsLCOHHiBFu2bMFoNJKdnY2Dw6/xtS2diIiIiBQEmyrQjz32GO7u7qSlpREYGEjbtm0B6N27N1euXKFu3bosXboULy+vP7xW5VlERERECoJNFWiAJk2akJiYiLe3NwArV66kbt26uLm5AZjLs9FotPk1zh4epa0dQQqQp6ertSNIAdJ4Fy8a7+JDY52/MjOzuH37rrVj/M9srkCnp6ezatUqEhMTSUpKwtfXl1GjRuHs7Jzr52y9PAOEh4eTnJxs7RgiIiIiNmHs2LGACvR95+bmxujRozl8+DAuLi74+fkBhWPGWURERESKPpsr0ABlypQhKCjI/FjlWURERERsRaFopSrPIiIiImIr1ExFRERERCygAi0iIiIiYgEVaBERERERC6hAi4iIiIhYQAVaRERERMQCKtAiIiIiIhZQgRYRERERsYAKtIiIiIiIBVSg7yExMZH09HQATCaTldOIiIiIiC2xyVt5W0tiYiLXrl1j//79XLp0iRdffJFSpUpZO5aIiIiI2BDNQP+/+Ph4XnzxRapUqcLTTz+Ns7Mz7777Lrt27QI0Ey0iIiIiP1OB/n8uLi5Ur14dJycnAAYNGkStWrV4//33uXPnDgaDwcoJRURERMQWFNsCffToUd5880127NgBwM2bN0lISMDFxQWAsmXL0rlzZ6pXr86HH35ozagiIiIiYkOKbYF2dHSkZs2arFy5kn/961+cOHGC5s2bA5CdnQ2Ak5MT7dq1Izk52XxRoYiIiIgUb8WiQBuNRrKzs1mzZo35mI+PD0899RRr1qyhR48eREVFERsbC4CDg4N5zbODgwNZWVk4OztrHbSIiIiIFI8CbWdnh4ODAytWrCAqKgr4uVSbTCbs7e3p2bMnCxYsoEqVKkRHRwOQk5MDQJMmTbh69Sq3bt3SOmgRERERKdoFes+ePaxfv57du3cD8NZbb7Fs2TKMRiP29vYYjUaMRiMAZcqUoWPHjrz77rvk5OTg4PDzDn/Xrl2ja9eu2s5ORERERIAiWqCvX7/OhAkTWLlyJQ4ODmzYsIGxY8dSt25dGjVqxMKFCwGwt7fHzs6OW7dukZmZSadOnZg7dy729vbm5RqVK1cmODjYvDuHiIiIiBRvRbJA79ixg9q1axMeHk6fPn1YtGgR9erVIzQ0lIkTJ/Lpp5+aLxR86623CA0NJSEhAYD69esDaLmGiIiIiNxTkSnQly9fJisrC4C4uDjq1KkD/LqjxnPPPUdiYiLZ2dk89thjdO/enYEDB5Kens60adPw9va2WnYRERERKTwK/a28Dxw4wMKFC3F2dsbDw4PXX38dgNq1awM/X0CYnZ1NSkoKZcqUISsrixdffJFr167x9NNP06BBA+Dniwrt7IrM7xMiIiIikk8KbWNMSUlh7NixLFu2jL59+7J48WJ27NjBmTNn8PDwYNWqVcCvO3DcvXsXNzc3nJ2dcXR0ZObMmTRo0ACTyaTyLCIiIiJ5VihnoE+dOkVSUhIHDhxgxYoV5uUXHTp0wMnJiWHDhjF27FjefPNN2rVrx7Fjx9i6dStPPvkkZcqUMb/PL8VZ651FREREJK8KXYHeu3cvly9fpmvXrvTs2ZPo6GgaNGjAmjVrqFmzJh4eHpQqVYrp06fzww8/8N1333Hr1i2WLl2Kp6dnrvfSrLOIiIiIWKpQFOjjx4+TkJBAcHAwAN999x09e/YkODiYqVOn8tVXXzFnzhxq1qxpfk2VKlWoUqUK2dnZ5j2djUYjBoNBM84iIiIi8o/ZfIG+ceMG06ZN4+LFi+zYsYOqVatSpUoVkpKSqFevHiEhIZw+fdpcnn9bmIFc5bmgZ5yHDh1aoOcTERERsWWZmVnWjnBf2HyBdnd3x9/fn/j4eJYvX87AgQM5c+YM169fx9PTk1atWnHixAkiIiLo06dPrvL8W9ZYrnH9eipGo6nAzysFz9PTlaSkFGvHkAKi8S5eNN7Fh8Za8srmCvSuXbv4+OOPGTx4MH5+fgAEBQXh7e1NeHg4vXv3pm7dumzdupVq1apRvXp1mjVrxrVr1zCZTDa1PMPOznaySP7TeBcvGu/iReNdfGisi5ffjrclY28w/XLPaisymUwkJiZy4MABcnJyGDNmDEFBQYwfP546deqwYcMGKlSoQFJSEnv37mXu3LmEhobi7+9Phw4dKFeunLU/goiIiIgUE1bdhuKX7m4wGEhLS2PhwoV07tyZ9u3bc+fOHfbv38+aNWvo0qULmzdvpk+fPsTHx7Nnzx7GjRtHamoqc+bMIT4+3pofQ0RERESKEasW6F+WW5hMJuzt7QkKCsJoNDJ48GASEhJo164dO3fu5LPPPqNq1arY29szdOhQjh8/jru7O0OGDKFt27bcvHkTG5hIFxEREZFiwCpLOIxGI9OnT6d8+fIMHjwYZ2dnrl+/zvDhw1mwYAFVqlRh+PDhNGnShF69erF06VLWrl1LXFwcpUuXzvVetrbuWURERESKtgIt0JcvX2bp0qW4urqyb98+qlevTvXq1enRowfVq1dn5syZ1K5dmz59+nD16lW6du3Kxo0b8fb2Ji4ujsDAQPN2dLr9toiIiIhYQ4E10JiYGF577TUaNmzIkCFDGDVqFC+++CL29vZMmTKFhIQE3N3dzfs5V6hQgf79+3PkyBEAAgMDfw78/6VZ5VlERERErKHAtrE7dOgQ3bp1o1evXsDPyziSk5MZMWIEBoOBqKgooqOjycnJoVmzZgCMGjWqoOKJiIiIiORJgRXoGjVqEBoaSnx8PN999x2ZmZl4eHjg6upKWFgYJ06cYMuWLRiNxj/cTVDrnEVERETEVhToGug9e/aQlpaGvb09bdu2BaB3796EhYVRuXJlLly4gJeXV0HFERERERGxWIHeibBJkyYkJibi7e0NwMqVK6lbty5ubm4A5vKsCwRFRERExFYVaIFOT09n1apVJCYmkpSUhK+vL6NGjcLZ2TnXzxXm8mw0GgkNDeX48eM4OTkxY8YMqlevbu1Ych9lZWUxadIkLl26RGZmJi+88AK1a9dmwoQJGAwG6tSpw+uvv16o/z2W3K5fv06vXr1YuXIlDg4OGusi7N133+XLL78kKyuLfv360bx5c413EZWVlcWECRO4dOkSdnZ2TJ8+XX+/i6iffvqJt956izVr1nDu3Ll7jnFERAQffvghDg4OvPDCC+aVEn+mQP+tcHNzY/To0Tz99NNMnjyZadOmUbFiRYxGY0HGyFc7duwgMzOTDRs2MHbsWGbPnm3tSHKfRUVF4ebmxvr16wkPD2f69OnMmjWL0aNHs379ekwmEzt37rR2TLlPsrKymDp1KiVLlgTQWBdhe/fuZf/+/XzwwQesWbOGhIQEjXcRFhMTQ3Z2Nh9++CEjRozgnXfe0XgXQeHh4UyePJmMjAzg3v8NT0pKYs2aNXz44YesWLGCt99+m8zMzL983wL/tapMmTIEBQXh5+cHFL3lGj/88AOtWrUCICAggEOHDlk5kdxvnTp1yrVDjL29PYcPH6Z58+YAtG7dmm+//dZa8eQ+mzNnDk8++SQVKlQA0FgXYbGxsdStW5cRI0YwbNgw2rRpo/EuwmrWrElOTg5Go5HU1FQcHBw03kWQt7c3YWFh5sf3GuMDBw7QuHFjnJyccHV1xdvbm2PHjv3l+1q9uRal8gyQmpqa626J9vb2ZGdnWzGR3G8uLi6ULl2a1NRUXnrpJUaPHp1rpxgXFxdSUlKsnFLuh02bNuHu7m7+pRjQWBdhN2/e5NChQyxcuJBp06Yxbtw4jXcRVqpUKS5dukTnzp2ZMmUKAwYM0HgXQR07dvzTnd1+GePU1FRcXV3NP+Pi4kJqaupfvm+BroEuDkqXLk1aWpr5sdFozDVwUjRcuXKFESNG8NRTT9GtWzfmzZtnfi4tLY0yZcpYMZ3cLx9//DEGg4E9e/Zw9OhRxo8fz40bN8zPa6yLFjc3N2rVqoWTkxO1atWiRIkSJCQkmJ/XeBctq1atomXLlowdO5YrV64waNAgsrKyzM9rvIum307c/jLGv+9uaWlpuQr1Pd8n3xIWU02aNOGbb74B4Mcff6Ru3bpWTiT327Vr1xg8eDCvvPIKvXv3BqBBgwbs3bsXgG+++cZ850wp3NatW8fatWtZs2YNvr6+zJkzh9atW2usi6imTZuya9cuTCYTiYmJpKenExQUpPEuosqUKWMuSWXLliU7O1v/LS8G7jXGfn5+/PDDD2RkZJCSkkJ8fPzf9rcC3Qe6OPhlF44TJ05gMpmYOXMmPj4+1o4l99GMGTP47LPPqFWrlvnYa6+9xowZM8jKyqJWrVrMmDEDe3t7K6aU+23AgAGEhoZiZ2fHlClTNNZF1Ny5c9m7dy8mk4kxY8ZQrVo1jXcRlZaWxqRJk0hKSiIrK4uBAwfSsGFDjXcRdPHiRV5++WUiIiI4c+bMPcc4IiKCDRs2YDKZeP755+nYseNfvqcKtIiIiIiIBbSEQ0RERETEAirQIiIiIiIWUIEWEREREbGACrSIiIiIiAVUoEVERERELKACLSJSyGRmZrJixQpCQkJo3LgxLVq0YNiwYRw8eBD4ecumevXqERcXl+9ZwsLCaN++vfnxrl27aNeuHY0aNWL16tW0a9eOpUuX5nsOEZGCpG3sREQKkfT0dAYOHMjNmzd56aWX8Pf3Jy0tjdWrV7N161aWLVtGtWrVeOSRR1i3bl2+3wgiLS2NjIwM3N3dAXj88cdxc3Nj2rRpuLm5kZmZScmSJSlVqlS+5hARKUi6x7SISCHyzjvvcPbsWT799FMqVqxoPj579myuX7/O9OnT+c9//lNgeVxcXHBxcTE/TklJITg4mGrVqhVYBhGRgqYlHCIihURmZiabNm2id+/eucrzL6ZOncr8+fMxGAy5jt+6dYuJEyfSsmVLHnjgAVq2bMmcOXMwGo3Az7enHzlyJA8++CABAQH861//4ujRo+bXb9q0ic6dO9OwYUPatm3LokWLzK/97RKOevXqce7cOZYsWUK9evUA/rCEY8eOHXTv3p1GjRrRqVMnVqxYYX6vX5ae/Oc//yEoKIjOnTuTmZnJsmXLeOSRR2jYsCEdO3Zk3bp19/FbFRGxnGagRUQKiQsXLpCcnIy/v/89n/fy8gJ+LqK/NX78eG7evMm///1v3Nzc+Oabb5g+fTpNmzbl0UcfZdq0aWRnZ7N+/XoMBgPz58/nxRdfZMeOHRw7doypU6fy9ttv07BhQw4fPsy4cePw9vYmJCQk13liY2Pp27cvHTt2ZPDgwX/IFxMTw7hx45g8eTLNmzfn5MmTvPHGG6SnpzNy5Ejzz0VHR7N27Vru3r1LbGwsK1as4J133sHb25tvv/2WKVOmULduXZo1a/Y/fqMiIv+MCrSISCGRnJwMQJkyZSx6XatWrXjwwQepU6cOAP3792f58uUcP36cRx99lHPnzlGvXj2qVatGiRIleOONNzh16hRGo5ELFy5gMBioUqWK+c97771HpUqV/nAeT09P7O3tKVWqFJ6enn94/j//+Q/9+vWjd+/eAHh7e5OWlsaUKVMYPny4+ef69++Pj48PAPv27cPR0ZEqVapQtWpVnnjiCapVq0atWrUs+g5ERO4nFWgRkUKiXLlywM9LMizRr18/du7cyUcffcTZs2c5fvw4CQkJ5qUTw4cPZ/z48Wzfvp1mzZrRunVrQkJCsLOzo1WrVvj7+/P4449TvXp1WrZsSZcuXahSpYrF+Y8ePcrBgwf58MMPzceMRiN3797l0qVL5qUnv8ykA3Tr1o2NGzfSoUMH6tatS8uWLenevTseHh4Wn19E5H7RGmgRkULC29sbDw8Pfvrpp3s+v3fvXoYNG0ZSUpL5mMlk4rnnnmP27Nk4OzvTo0cP1q5dS9WqVc0/06lTJ3bt2sWMGTPw9PRk6dKlhISEcO3aNUqWLMnatWvZuHEjPXr04MiRIzz99NOEh4dbnN/R0ZFhw4YRGRlp/hMVFcX27dtzrekuUaKE+Z89PDyIiopi7dq1tGvXjj179vD444+zZcsWi88vInK/qECLiBQSdnZ29OzZk48//pjExMRcz5lMJpYtW8aZM2coX768+fipU6eIjY0lLCyMMWPG8Nhjj1GuXDmSkpIwmUxkZ2czZ84cLl26RLdu3Zg1axbR0dFcunSJ77//nt27d7NkyRIaNWrEiBEj+PDDD3nyySf55JNPLM5fu3Ztzp49S/Xq1c1/Tpw4wYIFC/70NVu3buWDDz6gWbNmjBkzhsjISB5++GGioqIsPr+IyP2iJRwiIoXI8OHD2b17N0899RRjxozB39+fa9eusXLlSvbt28fKlStz7cJRpkwZHBwc+OyzzyhbtixJSUksWLCAzMxMMjMzcXBw4PDhw8TFxTF58mTc3d3ZsmULjo6OPPDAAyQmJrJkyRJcXV1p27Yt165dY+/evQQEBFic/YUXXuD555+nbt26dOjQgbNnzzJ16lSCg4NxcnK652syMzOZM2cOrq6uNG3alPPnz3PkyBH69ev3T79CEZH/mQq0iEgh4uLiwtq1awkPD2fx4sVcuXIFV1dX/P392bBhA76+vrl24ahYsSIzZ84kLCyM999/n4oVK9K5c2cqVqxovnPh/PnzmTlzJs8//zxpaWnUqVOHJUuWmGeJZ86cyfLly3nrrbcoXbo0jz76KK+++qrF2Vu3bs3cuXNZtmwZixYtwt3dnZCQEMaMGfOnrwkJCeH69euEhYVx5coVPDw86NWrF8OGDbP8yxMRuU90J0IREREREQtoDbSIiIiIiAVUoEVERERELKACLSIiIiJiARVoERERERELqECLiIiIiFhABVpERERExAIq0CIiIiIiFlCBFhERERGxgAq0iIiIiIgF/g9JF3hOqS0D2AAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 720x360 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAtEAAAFpCAYAAABauHSCAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8rg+JYAAAACXBIWXMAAAsTAAALEwEAmpwYAABIPElEQVR4nO3deVxU9f7H8dewI+AC4g5ouJYKLujFjdyXUtFM29TSNHO3vJqlprmXZkraTdxyadPM3TLt5hVTU6+luYu7BoKiLCrDMPP7o59T3EoZA2aA9/Px4JGcmXPOZ84X6823z/keg8VisSAiIiIiItnmZO8CRERERETyG4VoEREREREbKUSLiIiIiNhIIVpERERExEYK0SIiIiIiNlKIFhERERGxkUK0iBQaPXv2pFq1an/6tWLFij+8/8MPP+T555/P+0LtZPLkydSpU4d69eqRmJho8/4tWrRg/vz5AERFRdG6desHquPv7Gur39cMWa/B5s2bqVatGnFxcXlSi4jkLy72LkBEJC89/vjjvPbaa3/Y7u3tneX7zz77jPfee4+GDRvmVWl2derUKZYvX87EiRNp0qQJJUuWtHdJeWL16tV4eHgAf7wGpUqVIiYmBj8/PztXKSKOSCFaRAoVDw8P/P39//L169ev8+abb7Jz506CgoLysDL7Sk5OBqBx48ZUqFDBztXkHV9fX+uf/+wa3OtnRUQKN7VziIj8TmxsLABr164lNDQ0W/skJiYyePBgGjZsSGhoKM8//zzHjh2zvp6amsrEiRNp1KgRderUoW/fvpw5c8b6+vbt2+natSshISE8+uijREVFYTKZANi7dy+1atVi/vz5NGjQgJ49ewJw8uRJ+vbtS0hICM2aNWP8+PHWEPhnTCYT0dHRtGnThlq1atGxY0c2b94MwJo1a3jmmWcAaNWq1Z/O1N+t5bnnnqNOnTrUrFmTzp0785///Cdb1+h/3e+a2HLe7777jsjISGrXrk2TJk2YNGkS6enpANy6dYsxY8bQqFEjatWqRffu3dm9e7d137vtHH92Dfbu3ZulncNoNDJ9+nSaNGlC3bp1ee655/jxxx+tx4qKiqJnz54MHTqUunXrMnv27Pv+bIhI/qUQLSLyO2FhYURFRVGxYsVs7zNx4kRMJhMff/wxa9aswcvLiyFDhlhfHz58OLt372bWrFl88cUXFClShBdffJGMjAy2bt3KkCFDaN++PWvXrmXUqFEsX76cadOmWfc3Go3s3buXVatWMXbsWOLj4+nZsydVq1blyy+/ZO7cuZw+fZrBgwf/ZY3Tp09n0aJFvPLKK6xfv57HHnuMV155ha+//poOHTpY+4JXrVrFG2+88Yf9f/nlF/r160e9evVYv349q1evpmzZsowePRqj0Zjta5Wda2LLea9fv87gwYN56qmn2LJlC++88w6bN28mOjoawHptFi1axObNm6lRowaDBg3i1q1bWc6TnWswatQo9u3bx3vvvccXX3zBP/7xD3r16sXZs2et7/nhhx8ICAjgyy+/pFu3bvf92RCR/EvtHCJSqKxdu9Y6A3tXhw4dmDJlygMf8/z581SrVo0KFSrg7u7OW2+9xenTpzGbzZw7d46dO3eybNkya3/1W2+9xYcffsiNGzdYsGAB7du3p1+/fgBUqlSJGzduMGXKFIYPH249x4svvmhtL5k9ezYVKlRg9OjR1tdnz55Ns2bNOHjwIHXq1MlSX2pqKp988gnjx4+nXbt2AAwYMIDjx4+zYMEC2rZtS7FixYBf2xt8fHz+8BkzMjIYNmwYffr0wWAwAPD888/Tu3dvrl27RtmyZbN9vc6cOXPPa2LLeZOSksjIyKBMmTKUL1+e8uXLs3DhQooUKWIdGy8vLypUqICPjw+jR4+mbdu2ODs7ZzmPh4fHPa/B+fPn2bJlCxs3bqRKlSoADB48mAMHDrBkyRLeeustAAwGA0OGDLH2Wd/rZ8PJSfNYIvmZQrSIFCqtWrXilVdeybLNy8sr2/v/b0A9ePAgAwcOZPTo0WzdupWwsDCaNWtGZGQkTk5OnDx5EoDatWtb9ylRooS1ZeLUqVN06dIlyzHDwsIwmUxZ2hsCAgKsfz527BjHjh37Qy3wazvK/24/c+YMJpOJunXr/uE83377bbY+d2BgIJGRkXz00UecOHGC8+fPW9sSMjMzs3WMu+53TWw5b40aNWjfvj0vvfQSZcqUoXHjxrRu3ZrmzZsD0LdvXwYOHEh4eDh16tShadOmdO7cGXd3d5tqPnr0KADdu3fPst1oNGaZiff397cGaOCePxsikr8pRItIoeLt7f23bhhcu3btH7a1a9eORo0asWPHDr7//nvmz5/P0qVL+fzzz3Fxufe/Zn8fuO66G0p/v+/v3+fq6krjxo0ZO3bsH/b9/Y1yd7m5uf3puTMzM+9b312nTp3imWeeISQkhPDwcDp06IDJZGLAgAHZ2v/3snvO7JzXYDDw3nvvMXjwYHbs2EFMTAyDBg2ie/fuTJgwgfr161u3x8TEsHLlShYvXsyKFSuoXLlytutwdXUF4NNPP/3DmP3++v7va/f62SgsK6CIFFT6VVhExAZBQUFZvkwmEzNmzODy5ct07NiRadOmsWnTJi5fvswPP/xAcHAwAD///LP1GKmpqYSHh7N//36Cg4M5cOBAlnMcOHAAV1dXAgMD/7SGypUrExsbS7ly5ax1ODk5MXXqVH755Zc/vL9ixYq4urr+6XmyGyTXrFlD2bJlWbhwIX379qVp06bEx8cDYLFYsnWMu+53TWw57+HDh5k2bRqVK1emb9++LFmyhBEjRvDll18C8P777/Pf//6X1q1bM3HiRLZu3YrZbObf//63TTXfbeG4du1alvFfunQp27dv/9N97vezISL5m0K0iMjf4OLiwpEjRxg/fjw//fQTFy9e5LPPPsPV1ZVHHnmESpUq0bJlSyZOnMj+/fuJjY1lzJgx+Pj4ULt2bV5++WW2bNlCdHQ0586dY8uWLcydO5cnn3zyT3uTAZ577jmSk5N57bXXOHHiBIcPH+aVV17h3Llzf3pDpIeHBy+88ALvvfceX331FefOnWPBggVs3bqVF154IVuf09fXl8uXL7Nr1y4uX77MunXrmD17NoDNNxbe75rYcl4fHx9WrlzJu+++y4ULFzh27Bj//ve/rce5fPkyEydOZO/evVy+fJn169eTkpJCSEiITTUHBQXRoUMHxo0bx44dO7hw4QKzZ8/m008/tf5S8L/u97MhIvmb2jlERP6mWbNmMXXqVF566SXS0tKoUqUK8+bNs7aNTJ8+nWnTpjFw4EAyMzMJCwtj4cKFuLm50bRpU2bMmMGHH37InDlzKFWqFL169eKll176y/P5+/uzZMkSZs6cSffu3fHw8KBhw4bMmTPnL1s3hg0bZp2tTkpKIjg4mHfffZf27dtn6zP26tWL2NhYRowYQWZmJsHBwUycOJExY8Zw+PDhvwySf+Ve18SW80ZGRjJv3jzmzp3LsmXLcHV1pWnTpowZMwaAsWPHMmPGDF599VVu3LhBUFAQ06ZNo0GDBjbVC78+zXDWrFm8/vrrpKSkEBwcTFRUFOHh4X+5z/1+NkQk/zJYbP3/cCIiIiIihZzaOUREREREbKQQLSIiIiJiI4VoEREREREbKUSLiIiIiNhIIVpERERExEYK0SIiIiIiNtI60bkoKSkNs1krCBYGfn7eXLuWau8yJI9ovAsfjXnhovEuXPz8vB9oP4XoXGQ2WxSiCxGNdeGi8S58NOaFi8Zb7kchOhc96G828ufupGeQknzH3mWIiIiIKETnpqHT1pKYlGbvMgqMj99+lhQUokVERMT+dGOhiIiIiIiNFKJFRERERGykEC0iIiIiYiOFaBERERERGylEi4iIiIjYSCFaRERERMRGCtEiIiIiIjZSiBYRERERsZFCtIiIiIiIjRSiRURERERspBD9F8xms71LEBEREREHpRD9F5ycfrs0FovFjpWIiIiIiKNRiP6d34fllJQUJk+eTGZmJgaDwY5ViYiIiIijUYjmt/BsMBgwmUxYLBYyMzOxWCzcvHnTztWJiIiIiKMp1CH69+EZ4MCBA0yfPp1du3bh5ubGmTNn8PLysmeJIiIiIuKACmWItlgsWCwWa3jevHkzc+bMoUSJEoSFhfHBBx9w8eJFypcvz86dO+1crYiIiIg4Ghd7F5DXzGaz9abBK1eu8N///pddu3aRnp7Oxo0b6dOnDx4eHixbtozY2Fgee+wxgCyhW0REREQKt0IXop2cnLh9+zYLFy7k+++/x9/fnzfeeAMvLy9mzpzJrl27aNu2LaGhoQwfPpytW7cSHh6uAC0iIiIiVgW+neN/13s+cuQIgwcPxsvLi/feew+TycTPP/+Mt7c3ISEh7Nq1i6NHj1KsWDHGjRtHyZIl7VS5iIiIiDiqAjsTffemwbutG7t378ZkMhEeHo7JZMLT05PSpUvTvHlzdu3aRc2aNWndujUmk4nAwEBSUlKYPXs2YWFh9vwYIiIiIuKACmSIzszMxNnZGYBbt26xePFi9u/fz82bN+nVqxeDBg1i2bJldOnShcjISP7973/z448/0rZtW5588kkA0tPTGTVqFAEBAfb8KCIiIiLigApkiHZ2diY9PZ05c+aQmppKRkYGS5cu5fTp0/Tv359vvvmGzz//nGXLltG/f3/Gjh1LuXLlrPtbLBbc3d0VoEVERETkTxXInuhTp07xwgsvUKpUKTp16sSmTZu4cOEClStXJiQkhGnTpjF06FA8PDwArAH6f9eNFhERERH5M/k6RN+6dQsAk8kEwKVLl1i/fj3u7u5cuHCB0NBQ6tevzwsvvMDMmTMBGD9+PAkJCZQvX55evXplOZ7Cs4iIiIhkR74N0Zs3b+all17Ksu3QoUPs37+f5ORkBg8ezOLFiwEYOnQoe/fu5YcffqBEiRLMmTPH2jN9d/ZZRERERCS78l2IzszMBKBDhw7cvHmTCRMm8MknnwDwj3/8g+DgYHbs2EHz5s0xGo18/vnnODs7s2zZMho0aPCH42j2WURERERsle9C9N0Z5L1791K5cmW+/vprvv32W44dO4avry8PPfQQZ86c4dixYzz99NPcuHEDgGrVqgG/zTzfPY6IiIiIiK3y3eocJpOJSZMmcePGDZ566ilMJhM//PAD33zzDTVq1MDT0xOj0ciVK1d48skniYiIyLK/Zp5FRERE5O/KdyHaxcUFNzc3XnjhBUJDQwkICKBXr14cPnyYoUOH8ssvvzBy5EgaNmxo3cdsNlsfupKX5o6JzPNzFmR30jPsXYKIiIgIkA9DdGZmJgaDgfj4eADKlCmDn58fZcqU4fnnnyc4ONj6XovFgsFgsEuABrh2LRWzWTcuioiIiBQ0+S5EOzs706FDB9atW8cPP/zAL7/8QocOHejYsSMlS5YEfntioVo3RERERCQ35LsQDRAaGkpgYCCbN2/mscceo27dusBvbRu6aVBEREREclO+DNEAvr6+PPfcc9bvLRaL3do2RERERKRwyfepU4/qFhEREZG8lu9DtMKziIiIiOQ1g0XPvRYRERGRXGQyppN002jvMv6Uv7/PA+2Xb3ui84PD/xqNMfmavcsQERERsat6oxYCjhmiH1S+b+cQEREREclrCtEiIiIiIjZSiBYRERERsZFCtIiIiIiIjRSiRURERERspBAtIiIiImIjhWgRERERERspRIuIiIiI2EghWkRERETERgrRIiIiIiI2UogWEREREbGRQvSfOHXqFGazGQCLxWLnakRERETE0bjYuwBH8v3337NgwQKKFCmCp6cnw4cPJyAgwN5liYiIiIiDUYgG0tLSGD16NMnJyYwcOZJatWoxd+5cjhw5ohAtIiIiIn9QqNs5jEYjJ0+eJDMzEx8fH5o1a0bt2rVJSUlh586dJCcnc/z4cQBre4eIiIiISKEO0Tt27GDs2LGYzWa6devGsWPHePfddxk9ejTBwcHcuHGDrl27YrFYcHIq1JdKRERERH6n0LVzXLhwgcDAQABat27Nxo0b2bRpE127dqVSpUqsXr2ajz76iKCgIAD27NnDkSNHqFmzpj3LFhEREREHUqimV7/99lueeeYZJk2axO7duwEYNGgQ69evJzExkXbt2tGkSRMyMjJISUnhjTfewMPDwxq6RURERESgEMxEZ2ZmsnDhQkqWLMmFCxdo06YNFy5cYPLkydStW5c+ffpQvXp1Nm7cSO/evalXrx59+/YlICCAjh070qNHD3t/BBERERFxMAU6RG/atInPPvuMatWqUblyZW7dukWpUqUIDg6mUqVKxMfHs2DBAmJjY9m+fTv16tWjadOmJCcn06lTJ0qUKGHvjyAiIiIiDqjAhujk5GR27NjByJEjqV27NgBXrlxh7ty5FClSBFdXV7p3706XLl34/PPPWb58OWvXrmXq1Kn07t3bztWLiIiIiCMrsCH6m2++wWKxULt2bYxGI7Nnz+bkyZOkpaXh6upKiRIl2L17N+Hh4XTv3p1u3bppBQ4RERERyZYCmxorVaqEj48PqampuLm50axZMxYtWsTkyZOJjY3l6tWrnDhxgjt37gAoQIuIiIhIthXYmWg/Pz+KFCnC/v37efTRRwkPDwfgxx9/pHfv3kRERFChQgU8PDzsXKmIiIiI5DcFNkQHBgYSHBzMunXruH79OlWqVGHZsmXcuHGDV155herVq9u7RBERERHJpwpsiDYYDHTp0gU/Pz8OHz7Mrl27iIiIoFOnTvYuTURERETyuQIbou9q1qwZzZo1w2w2q+9ZRERERHKEwWKxWOxdhIiIiIgUXCZjOkk3jfYu40/5+/s80H4Ffibanq5dS8Vs1u8ohYG/vw8JCSn2LkPyiMa78NGYFy4ab8kO9TeIiIiIiNhIIVpERERExEYK0SIiIiIiNlKIFhERERGxkUK0iIiIiIiNFKJFRERERGykdaJFRETEoaQbjSTfTLfb+bXEXeGidaId0MhVE0lMvW7vMkRERPKVpS/MAewXokWyQ+0cIiIiIiI2UogWEREREbGRQrSIiIiIiI0UokVEREREbKQQLSIiIiJiI4VoEREREREbKUSLiIiIiNhIIVpERERExEYK0SIiIiIiNlKIFhERERGxkUL0Xzh48CDXr+uR3SIiIiLyRwrRv2OxWACIi4sjKiqKw4cPk5mZaeeqRERERMTRKET/v8uXL2MwGAAoU6YMzZs3Z9euXcTHx9u5MhERERFxNArRQEJCAn369KF79+588sknAPTo0YMbN25w4MABbt++becKRURERMSRFOoQfeDAAV5//XX8/f3p1q0bRYoUYdWqVbz66qts2bKF0NBQjh49yqVLl+xdqoiIiIg4EBd7F2BPoaGhDBo0iEOHDtG+fXvMZjO+vr7UrVuXlStXcurUKWJjY3F1dWXQoEG4u7vbu2QRERERcQCFaiY6MTGRxYsXExsbi8lkwtnZmSFDhjBlyhQqVKhAmTJlOH78ON7e3owfP55hw4bRokULdu3aRXp6ur3LFxEREREHUWhmoleuXMmqVatISEhg/fr1VKtWjVdeeYVnn32WzZs3s27dOlq1asW5c+fYtGkTffr0oX79+tSvX9/epYuIiIiIgykUM9F37tzhxx9/ZNasWURGRhIZGYm7uzvTp09nxowZREREsHDhQooUKULt2rU5c+YMFy9etHfZIiIiIuKgCmyIvn79Ou+//z4Wi4WbN2+SkJBAcHAwoaGhXLhwgW7dujFlyhRKlCjBoUOHOHXqFJMmTaJ58+YMGzaMgIAAe38EEREREXFQBbadIyMjg/Pnz7Nu3Tr8/f2t21u3bs3Ro0fZtWsXlSpVon///ty8eZPHH3+cjIwMgCzvFxERERH5XwU2RJcqVYqOHTuybt06MjMzOXPmDG+88QahoaFERESwZs0a9u/fT0REBMWKFaNdu3b2LllERERE8okCG6INBgN169YlJiaGmJgYVq9ezalTp4iOjsbDw4PvvvuOmjVr2rtMEREREcmHCmyIBvD29qZjx44cO3aM69ev07hxY+rXr8+ZM2d44YUXaNiwob1LFBEREZF8qMDeWHhX9erVadmyJYsWLQLA3d2dGjVqKECLiIiIyAMr0DPRAK6urjz66KP4+/tjsVgwGAz2LklERERE8rkCH6IBKlasSMWKFe1dhoiIiIgUEAaLxWKxdxEiIiIid6UbjSTfTLfb+f39fUhISLHb+SVv+fv7PNB+hWIm2l6uXUvFbNbvKIWB/oVbuGi8Cx+NuYj8rwJ/Y6GIiIiISE5TiBYRERERsZFCtIiIiIiIjRSiRURERERspBAtIiIiImIjhWgRERERERtpnWgRERERyXEZd9K5kWK0dxn3pXWiHdC/XxnJ7cRr9i5DREREJM91WLYE8kGIflBq5xARERERsZFCtIiIiIiIjRSiRURERERspBAtIiIiImIjhWgRERERERvZHKJ/+ukn1qxZQ2pqKqdPn8ZkMuVGXSIiIiIiDivbS9ylpKQwZMgQ9uzZg8FgICwsjJkzZ3L+/HmWLFlCmTJlcrNOERERERGHke2Z6HfeeQeTycSOHTvw8PAA4I033sDHx4cZM2bkWoEiIiIiIo4m2yF6x44djBo1itKlS1u3BQQEMG7cOHbv3p0rxYmIiIiIOKJsh+ibN29SrFixP2x3d3cnPT09R4tyJHoquoiIiIj8r2yH6Hr16rFq1aos2zIzM1mwYAGhoaE5XZdd7d+/nyVLlvDTTz+RkZFh73JERERExMFk+8bC0aNH07NnT/bu3UtGRgaTJ08mNjaW5ORklixZkps15pnLly8zb948rl+/To0aNdi9ezcVK1bEzc3N3qWJiIiIiAPJdoiuWrUq69ev55NPPsHX1xdXV1cef/xxnnvuOUqWLJmbNeYJk8nEkiVLqFGjBj179gTAbDazfv16ateuzUMPPYTZbMbJSUtri4iIiBR22Q7RvXv3ZuzYsQwfPjwXy8l7x44do0aNGsTFxXHo0CHGjBkD/Hoj5QcffIC7uztr1qxh2bJlCtAiIiIiAtjQE338+HHr0nYFwaFDhxg8eDALFizAaDRiNBoJCQkhNTUVAD8/P958800++ugjLl++rBVIRERERMQq2zPRzz//POPHj6dfv35UqFABd3f3LK//fuk7R5aQkMC8efOIi4sjMjKSdu3aAeDl5cX58+c5f/48tWrVombNmgDExsYSHh5OrVq17Fm2iIiIiDiQbIfoDz74AKPRyO7duzEYDNbtFosFg8HAsWPHcqXAnPbBBx+wc+dOtm/fbt22atUqHnvsMZo2bcqGDRvYv38/PXv2ZOXKlWzevJnOnTvj7e1tx6pFRERExJFkO0QvXLgwN+vIVd988w1Xrlyha9eu9OnThwsXLnD+/HkuXbpEdHQ0lStXpnPnzjz77LP8/PPPrF69mvHjx+Ph4cH8+fMLxI2TIiIiIpJzsh2iGzRoAEBqaipnzpzB1dWVgIAAh56hPXHiBNHR0SQnJ+Pn58e8efPo378/zZs354knniAiIoJhw4ZRp04dAK5evUrt2rWpXbs2t27dokiRInb+BCIiIiLiiLIdojMzM5k2bRqffvopmZmZWCwW3Nzc6N69O6+//rpDrVxhNpuZPXs227dv55VXXqFVq1Zcv36df/7znwBERkby1Vdf0bFjR+rUqcO1a9f44IMPuHTpEtOnT6d48eIK0CIiIiLyl2zqid6wYQNvvPEGYWFhZGZmsn//fqKioihZsiQDBgzIzTptYjabcXFxoV69erRq1QqAb7/9lrJly+Lp6Ymnpye9e/dmyZIlnDt3jh07dhAeHs7o0aNxdXW1c/UiIiIi4ugMFovFkp03Nm/enFGjRtG+ffss27ds2cKsWbPYtm1brhSYXTt27OCrr76iZMmS9OzZk9u3b7No0SK8vb2JjY0lISGBlJQUOnfuTGRkJCVKlGDcuHGkpaXx5ptvUq5cuRyv6d+vjOR24rUcP66IiIiIo+uwbAkJCSn2LuO+/P19Hmi/bM9EJyUl8fDDD/9h+8MPP0x8fPwDnTwnXL16lRkzZnDr1i169+7Nzz//jMFgwNfXlwYNGjB9+nSmTJlCREQEsbGxfPzxx4wcOZJHH32UqVOnFqi1r0VEREQkb2Q7RAcHB7N9+3b69OmTZfs333xDxYoVc7qubIuJiSEwMJBhw4YBEBoayueffw5A/fr16dq1KwkJCcCvn2HcuHFcuXIlV2aeRURERKRwyHaIHjhwIEOHDuXYsWPW1SwOHDjAV199xYwZM3KtwHu5ffs2MTExPPPMMwD8/PPPLFiwAIAyZcpQqlQpwsPDWbhwIWFhYQQFBQEoQIuIiIjI35LtEN2yZUtmz55NdHQ033zzDe7u7lSuXJkPP/yQJk2a5GaNVvv27WPNmjU89NBD1K9fnzp16lCsWDF++ukn6tevT9myZXnttdcoV64cb775JvHx8TRo0ICOHTs69FJ8IiIiIpK/ZDtEw69Bum7dutaHj/z3v//Nk8dh3759m3feeYdLly7Rs2dPYmNj+fLLL9m6dStdunQhOjqapKQk/Pz8AIiPjycpKYnq1avj6+tLZGRkrtcoIiIiIoVHtlfnOHv2LC+++CJt2rRh9OjRADRq1IiiRYsSHR1NQEBArhV5+PBhPv/8cyZNmmTdlpCQwIQJE+jQoQNxcXF8//33PPXUU5w5c4aYmBgeffRR+vbtm2s1iYiIiMhfy7iTzo0Uo73LuK9cX51j8uTJPPLII7z00kvWbVu3bmXMmDFMnTqVDz744IEKuJfMzEycnZ05ffo0pUuXBsBoNOLm5oa/vz8RERHcvHmTvn374u/vzy+//ILRaCQqKorixYvneD22unYtFbM5W7+jSD7n7++TL5bxkZyh8S58NOaFi8ZbsiPbIfrgwYOsWbMmSzj19vZm+PDhPPXUUzlW0PXr1/n444956aWXrA8+SUhI4MqVKwA4OTlhNptxcnLip59+si6716lTpxyrQURERETkXrL9rG5PT0+uXr36h+1JSUk5+sjvjIwMLl68yNq1a63bevbsSUxMDMePH8fFxQUnJydu3ryJ2WymXr16OXZuEREREZHsyPZMdJs2bZgwYQJvvfWW9WbCw4cPM3HiROujtXNCqVKl6NChA2vXrqVZs2aULl0aT09PXn75ZaKioihbtiylSpXiu+++o2nTplSvXj3Hzi0iIiIikh3ZvrEwLS2NYcOGERMTg8FgAMBisdCyZUtmzJiRo0vIpaamsnDhQpycnBg6dKh1++7du0lOTubChQt07NiRMmXK5Ng5c4N6ogsP9c8VLhrvwkdjXrhovAuXXL+x0MvLi4ULF3L27FlOnjyJi4sLwcHBufK0Qm9vb1q0aMHy5cu5fPkyFouF8ePHU61aNf75z3/maPuIiIiIiIitsj0T/b+uX7/ODz/8wCOPPJIry9sZjUZWrlzJwoULCQgIoEePHnTp0iXHzyMiIiIiuceYbuJm8m17l/GXcn0m+vjx4wwdOpQpU6ZQvXp1unfvzqVLl3B1deWDDz7I8acWurm50apVK5ycnHj66adxc3PL0ePnhXnvbObmjVv2LkNERETEbl6f0s3eJeSKbPdFzJgxg6pVqxIcHMzatWu5desW33//PS+//DLvvfderhQXEBBA796982WAFhEREZGCK9sh+scff2TkyJH4+vryn//8h0cffRRfX186derEqVOncrNGERERERGHku0Q7ebmhsViwWg0sm/fPho3bgz82hvt5eWVawWKiIiIiDiabPdEN2jQgLfffpuiRYsCEBERwfHjx5kyZQrh4eG5VqCIiIiIiKPJ9kz0hAkTcHFx4fjx49Z1odetW4eHhwevv/56btYoIiIiIuJQsj0T7efnR1RUVJZtI0eOxNnZOceLEhERERFxZA/01JK6dety8eJFBWgRERERKZQeKEQ/4PNZREREREQKhAcK0QaDIafrEBERERHJNzQTLSIiIiJio2zfWPh7Bw8ezOk6HJLZbMbJyQmLxaLZdxERERGxeqAQXdB99913bNiwgUqVKvHiiy/i4eGhIC0iIiIiVg/UzlFQxcbGMmbMGNasWUOPHj2IjY1l/PjxgPrARUREROQ395yJfuGFF7IdHhcvXpwjBdnLxYsXefPNNwkICGDu3LkAuLq6smPHDs1Ci4iIiEgW9wzR//jHP5gzZw4VK1YkJCQkr2rKU3cDcvHixWnXrh0mk4mMjAw2b97MqlWr6NevnwK0iIiIiGRxzxD90ksvUbx4caZPn86//vUvAgMD86quXHfixAmqVauGwWDAYrHg4+ND7dq1Wbt2LZ06dSIsLIyJEycSHBxs71JFRERExMHctye6R48eNGnShJkzZ+ZFPbnu6tWrLF++nKeffpqNGzcCkJmZCcDDDz9MSEgIVatWZcSIEQQHB2MymbSkn4iIiIhkka0bC9966y169eqV27Xkic8//5zo6GgaNGjAihUrMJlMuLi4YDabcXFxoXbt2lSrVo1169YB4OLionYOEREREcniniF65cqVpKenU6JECerXr59XNeU4k8lk/XObNm2oWrUqAQEBXLp0iY8++gj4bTa6UqVK1tfOnz9vl3pFRERExLHdM0RPnjyZ1NTULNvGjRvH9evXc7WonJKQkEC3bt1YvHgxN27cAODGjRuUL1+e8PBwHn74YT755BNSUlJwdXVl3759TJs2jSJFijBw4ECCgoLs+wFERERExCHdM0T/WS/wpk2bSEtLy7WCcpKzszPu7u588skn1mXr6tevz5UrVzAajYSHh2MwGJg0aRJTpkxh1qxZhISE0LJlS3x9fe1cvYiIiIg4KpufWJifbrIrUaIEAwYMYMOGDZw/f56lS5dSvXp1evTogbu7O+XLl6dKlSr897//pVu3bnz66af2LllERERE8oEC/dhvg8FASEgIJ0+e5NKlS4SGhjJ58mQCAwPp378/JUqUwGAwUK9ePc08i4iIiEi23Xd1jvy+MkXRokUJCwvj5s2blCxZkpdffplTp06xfft2SpcuTevWrRWgRURERMQm952JnjZtGh4eHtbvMzIyePfdd/H29s7yvkmTJuV8dTmkevXqPPLII8ybN49p06bxyCOPUKZMGXuXJSIiIiL51D1DdFhYGHFxcVm21alTh8TERBITE63bHH222s3NjZYtW1KmTBksFosCtIiIiIj8LfcM0cuXL8+rOnJdxYoVqVixor3LEBEREZECIFtPLBQRERERkd8YLPlpzToRERERyVeM6SZuJt+2dxl/yd/f54H2K9BL3NnbtWupmM36HaUw8Pf3ISEhxd5lSB7ReBc+GvPCReMt2aF2DhERERERGylEi4iIiIjYSCFaRERERMRGCtEiIiIiIjZSiBYRERERsZGWuBMRERERh2FMT+dmsjHPzqcl7hzQwhmvk3zjmr3LEBEREck3Xpn2IZB3IfpBqZ1DRERERMRGCtEiIiIiIjZSiBYRERERsZFCtIiIiIiIjRSiRURERERspBAtIiIiImIjhWgRERERERspRIuIiIiI2EghWkRERETERgrRIiIiIiI2Uoj+E0ePHrV3CSIiIiLiwFzsXYAjuXjxIgDbtm3jhx9+oFWrVlSoUMHOVYmIiIiIo9FMNHDmzBmGDx/Oxx9/TOnSpenbty+enp5ERUVx+PBhAMxms52rFBERERFHUahnoo1GI2+//Tbnzp2jbdu2PPnkkwC4ubnRpUsXTCYTU6dO5ZNPPsHJSb9viIiIiMivCnUy3LFjBwcOHGDYsGHWAL1r1y7g1yD97LPP4uLiwtq1awGwWCz2KlVEREREHEihm4mOiYnhzp071KxZk8aNGxMbG8vx48dJS0tj+fLlFCtWjLp16+Lq6oqLiwt9+/YlJiaGzp07YzAY7F2+iIiIiDiAQhOiL1y4wLJlyzh27BiNGjVi5syZrF+/ntDQUD788EOSkpJ48803qVOnTpb9PDw8qFChAgaDAYvFoiAtIiIiIoUjRJ88eZLIyEhGjRrF2LFjAYiPj+fs2bOEhITQqFEjbt++bQ3QGRkZuLq6AlCqVCnWrl2L2WxWX7SIiIiIAAW8J3rPnj2kpKQQHBxM586dKVasGAArV67k66+/5ssvv+TIkSOEh4eTnp7OmjVrAKwBGqB48eJ06tQJi8WinmgRERERAQroTPShQ4eYN28eN2/epGrVqnTt2pXevXvzz3/+k3Xr1lGhQgWio6PZt28fkydP5qOPPqJs2bLcvn0bk8mEi8tvl8XX15dGjRrZ8dOIiIiIiKMpUCE6JSWFCRMmkJ6eTteuXWnbti3Dhg3j1KlTPPnkkzRp0oRDhw4xefJkAMqWLcu+fftwdnbmiSeewNPT086fQERERETygwITok+fPk1CQgKHDh1i0aJFBAYGAhAYGIifnx8Azz//PK+//jr79u3jzJkzrFmzhsjISLy9va3H0c2DIiIiInI/BSJE7927lytXrvD444/TpUsXNm3aRPXq1YmKiiI2NpZixYoREBBAlSpVaNOmDT179qR3797MmzePkiVLZjmWArSIiIiI3E++DdEnTpwgLi6OiIgI4NebCLt06UJERATjx49nx44dREVFcfnyZfbt28cLL7xAs2bNGDp0KJs2bSI4OBj49XHeBoNB4VlEREREsi1fhujr168zceJELl26xLZt2yhfvjzlypUjISGBatWq0blzZ86ePUv58uUpX748DRo0oHHjxty5c4cyZcoAWFfb0LJ1IiIiImKrfJkgfX19CQkJIT09nYULF1K8eHHOnj3LtWvXcHFxoVmzZphMJj777DPrPqGhofzjH/+wfm8wGBSgRUREROSB5IsUuXPnToYPH86hQ4es28LDwxk+fDirV6/m1q1bVK1alc2bN5OamkpQUBBhYWGkpaVpbWcRERERyXEGi4OmTIvFQnx8PIcOHSIzM5MRI0YQHh7O6NGjqVKlCp999hmlSpUiISGBvXv38vbbbzNhwgRCQkJo06YNJUqUsPdHEBEREREbGdPTuZlszLPz+fv7PNB+Dheif7/EXGxsrPVGwCFDhnD16lUiIyNJT08nMjKScePG8d577xEZGcnIkSOpVasWX375JadOnaJfv37Wmwft5dq1VMxmh7q8kkv8/X1ISEixdxmSRzTehY/GvHDReBcuDxqiHa6d426AtlgsODs7Ex4ejtlspk+fPsTFxdGiRQu2b9/Oli1bKF++PM7OzvTr148TJ07g6+tL3759ad68OUlJSWrlEBEREZFc4TAh2mw2M3HiRObNm8ft27cxGAz4+Phw+PBh4uLiqFOnDo888ggbNmxgzpw5xMbGsnTpUlJTU+nUqRP9+/e3HqtNmzbUr19fy9aJiIiISK6we4i+cuUKY8eO5Z133uHw4cOcOXOG6Ohozp8/j5+fHyEhIcTExAAwYcIEFixYQGpqKmPHjmXFihV4e3tjNpsBrP9UeBYRERGR3GTXEL1jxw7eeOMNatasSd++fRk2bBhDhgzB2dmZcePGERcXh6+vL5UqVQKgVKlSPPvssxw9ehSA+vXrA1iXqtOSdSIiIiKSF+z6sJWff/6Zjh070rVrV+DXmeTk5GQGDRqEwWBg/fr1bNq0iczMTMLCwgAYNmyYPUsWEREREbFviK5YsSITJkwgNjaWPXv2YDQa8fPzw8fHh6ioKE6ePMmGDRswm82YTCZcXH4r9/ereIiIiIiI5CW7L3G3e/du0tLScHZ2pnnz5gB069aNqKgoypYty8WLFwkICLBniSIiIiLyP0xGE0k3b9u7jL/tQZe4s+tMNEDdunWJj48nMDAQgMWLF1O1alWKFy8OYA3QZrM53/U8H12+h4yUO/YuQ0RERCTHhQx81N4l2JXdQ/Tt27dZunQp8fHxJCQkUKNGDYYNG4anp2eW9+W3AC0iIiIiBZfdQ3Tx4sUZPnw4R44cwcvLi9q1awP5c+ZZRERERAoHu4dogKJFixIeHm79XgFaRERERByZQyZVBWgRERERcWRKqyIiIiIiNlKIFhERERGxkUK0iIiIiIiNFKJFRERERGykEC0iIiIiYiOFaBERERERGylE/4k7d357VLfFYrFjJSIiIiLiiBSif8disZCYmMg777zD2rVrSU1NxWAw2LssEREREXEwCtH/7/jx48ybNw8vLy9GjRrFhQsXmDlzJikpKfYuTUREREQcjEL0/0tJSSE1NZWtW7fi7u7O4MGDSU9PZ9WqVcTHx9u7PBERERFxIIU2RF+9epXo6Gji4uIAqFu3LtWrV+f48eOcPn0aJycnevTowdWrVzl+/LidqxURERERR1LoQrTZbOb69etcvXqVWbNm0blzZ5YvX86xY8do1KgRRYsWJSYmBoDQ0FC8vLy4dOkSoJsMRURERORXhS5Ez5o1iy+++IKKFSvy2muvUaFCBYoVK8bo0aNZtmwZZrOZX375haNHjwLQtGlTvv76awDdZCgiIiIiQCEK0ZmZmQA0a9aMc+fOcerUKdq1a4eXlxe1atXis88+w9fXl927d7Ny5Uq+/vprjEYjoaGhNGzYkFu3btn5E4iIiIiIoyjQITotLY0dO3YAv80iN2zYkFKlSvH999/j5uZGp06dmDFjBt7e3vTp04cPP/yQAQMG0KhRI9zc3Lh48SK3bt2iSJEi9vwoIiIiIuJACmyIPnXqFIcOHWLZsmVcuHABJycnMjIyAOjSpQvnz5/n8OHDtGnTBldXV5YvXw6Al5cXgwcPpmHDhgAEBATwz3/+026fQ0REREQcT4EM0Xv37uXIkSPUq1ePOnXqsHr1agBcXV3JyMggMDCQ0NBQ9uzZw82bN+nRowc3b97McgzdRCgiIiIif6XAhOgTJ05YWzcAdu/ejZubG82bN+fy5cv88MMPwK9BGqBdu3ZcvnyZM2fO0KhRIwYPHpzleLqJUERERET+SoEI0devX2fixImMGzcOo9FI+fLlKVeuHImJiVSpUoV69eqxceNG4Nc+6dGjR7Nv3z5GjRpFREQETk6/Xgaz2WzPjyEiIiIi+USBCNG+vr6EhISQnp7OwoULKV68OGfPniUxMRE3NzcaNWqExWJhwIABDBgwAH9/f1q2bEmFChWA31o37oZpEREREZF7yZepcefOnQwfPpxDhw5Zt4WHhzN8+HBWr17NrVu3qFq1Kps3byY1NZWgoCDq1auHm5sbU6dOZeTIkbi4uFjDs1o3RERERMQW+SZEWywW4uLi2Lp1K6mpqXz11VfMnj2b48ePk5mZyaVLl/D396dfv35MnTqVF198kcTERDZt2kRSUhKRkZHMnTuXgIAAzGYzFotF4VlEREREHojDh+jfzxanpaUxZ84c2rdvT+vWrbl16xYHDx5k+fLldOjQgXXr1tG9e3diY2PZvXs3I0eOJDU1lbfffpvY2Fjr8ZycnBSgRUREROSBudi7gPu5G3YtFgvOzs6Eh4djNpvp06cPw4cPp0WLFowcORJ3d3fKly+Ps7Mz/fr148SJE0RERNC3b1++/vprkpKS8nz2+eGe/8izc4mIiIjkJZPRZO8S7MpgcdAFkc1mM5MmTaJkyZL06dMHT09Prl27xsCBA5k9ezblypVj4MCB1K1bl65duzJ//nxWrFjB/v378fb2znIse7VuXLuWitnskJdXcpi/vw8JCSn2LkPyiMa78NGYFy4a78LF39/ngfZzuHaOK1euMHbsWN555x0OHz7MmTNniI6O5vz58/j5+RESEkJMTAwAEyZMYMGCBaSmpjJ27FhWrFiBt7e3dam6u/9U64aIiIiI5CSHCtE7duzgjTfeoGbNmvTt25dhw4YxZMgQnJ2dGTduHHFxcfj6+lKpUiUASpUqxbPPPsvRo0cBqF+/PvDbUnVask5EREREcoND9UT//PPPdOzYka5duwK/ziQnJyczaNAgDAYD69evZ9OmTWRmZhIWFgbAsGHD7FmyiIiIiBRCDhWiK1asyIQJE4iNjWXPnj0YjUb8/Pzw8fEhKiqKkydPsmHDBsxmMyaTCReX38rXknUiIiIiklccKkQ/9thj+Pr6kpaWRv369WnevDkA3bp145dffqFq1arMnz+fgICAP+yrAC0iIiIiecXhVudIT08nPj6ewMBAABYvXszp06cZN24cnp6e1veZzWb1PIuIiIjkA0ZjBjdv3rF3GX/qQVfncKiZaIDbt2+zdOlS4uPjSUhIoEaNGgwbNixLgIb8cdNgdHQ0ycnJ9i5DRERExK5effVVwDFD9INyuBBdvHhxhg8fzpEjR/Dy8qJ27dqAZp5FRERExHE4XIgGKFq0KOHh4dbvFaBFRERExJHki2SqAC0iIiIijkTpVERERETERgrRIiIiIiI2UogWEREREbGRQrSIiIiIiI0UokVEREREbKQQLSIiIiJiI4VoEREREREbKUSLiIiIiNhIIfpPxMfHc/v2bQAsFoudqxERERERR+OQj/22l/j4eBITEzl48CCXL19myJAhFClSxN5liYiIiIiD0Uz0/4uNjWXIkCGUK1eO5557Dk9PTz788EN27twJaEZaRERERH6jEP3/vLy8CAoKws3NDYDevXvz0EMP8dFHH3Hr1i0MBoOdKxQRERERR1FoQ/SxY8eYMmUK27ZtAyApKYm4uDi8vLwAKFasGO3btycoKIhPP/3UnqWKiIiIiIMptCHa1dWVSpUqsXjxYp5//nlOnjxJgwYNADCZTAC4ubnRokULkpOTrTcaioiIiIgUihBtNpsxmUwsX77cui04OJhnnnmG5cuX07lzZ9avX09MTAwALi4u1h5oFxcXMjIy8PT0VF+0iIiIiACFJEQ7OTnh4uLCokWLWL9+PfBrsLZYLDg7O9OlSxdmz55NuXLl2LRpEwCZmZkA1K1bl6tXr3Ljxg31RYuIiIgIUMBD9O7du/n444/ZtWsXADNnzmTBggWYzWacnZ0xm82YzWYAihYtStu2bfnwww/JzMzExeXX1f8SExN5/PHHtdSdiIiIiFgVyBB97do1XnvtNRYvXoyLiwufffYZr776KlWrVqVWrVrMmTMHAGdnZ5ycnLhx4wZGo5F27drx9ttv4+zsbG3dKFu2LBEREdZVO0RERERECmSI3rZtG5UrVyY6Opru3bszd+5cqlWrxoQJExgzZgwbN2603jw4c+ZMJkyYQFxcHADVq1cHUOuGiIiIiPylAhOir1y5QkZGBgD79++nSpUqwG8rbfTv35/4+HhMJhOPPfYYnTp1olevXty+fZuJEycSGBhot9pFREREJH/J94/9PnToEHPmzMHT0xM/Pz/efPNNACpXrgz8elOhyWQiJSWFokWLkpGRwZAhQ0hMTOS5557j4YcfBn690dDJqcD8TiEiIiIiuSjfpsaUlBReffVVFixYQI8ePXj//ffZtm0bZ8+exc/Pj6VLlwK/rcxx584dihcvjqenJ66urkydOpWHH34Yi8WiAC0iIiIiNsmXM9GnT58mISGBQ4cOsWjRImsrRps2bXBzc2PAgAG8+uqrTJkyhRYtWnD8+HE2b97MU089RdGiRa3HuRue1f8sIiIiIrbIdyF67969XLlyhccff5wuXbqwadMmHn74YZYvX06lSpXw8/OjSJEiTJo0iQMHDrBnzx5u3LjB/Pnz8ff3z3IszT6LiIiIyIPIFyH6xIkTxMXFERERAcCePXvo0qULERERjB8/nn//+9/MmDGDSpUqWfcpV64c5cqVw2QyWdd8NpvNGAwGzTyLiIiIyN/i8CH6+vXrTJw4kUuXLrFt2zbKly9PuXLlSEhIoFq1akRGRnLmzBlrgP59aAayBOi8nnnu169fnp5PRERExBEZjRn2LiHHOXyI9vX1JSQkhNjYWBYuXEivXr04e/Ys165dw9/fn6ZNm3Ly5Ek+//xzunfvniVA/549WjeuXUvFbLbk+Xkl7/n7+5CQkGLvMiSPaLwLH4154aLxluxwuBC9c+dOvvjiC/r06UPt2rUBCA8PJzAwkOjoaLp160bVqlXZvHkzFSpUICgoiLCwMBITE7FYLA7VquHk5Di1SO7TeBcuGu/CR2NeuGi85X4MlrvPt7Yji8VCfHw8hw4dIjMzkxEjRhAeHs7o0aOpUqUKn332GaVKlSIhIYG9e/fy9ttvM2HCBEJCQmjTpg0lSpSw90cQERERkULErstT3M3vBoOBtLQ05syZQ/v27WndujW3bt3i4MGDLF++nA4dOrBu3Tq6d+9ObGwsu3fvZuTIkaSmpjJjxgxiY2Pt+TFEREREpJCxa4i+23phsVhwdnYmPDwcs9lMnz59iIuLo0WLFmzfvp0tW7ZQvnx5nJ2d6devHydOnMDX15e+ffvSvHlzkpKScIAJdREREREpJOzSzmE2m5k0aRIlS5akT58+eHp6cu3aNQYOHMjs2bMpV64cAwcOpG7dunTt2pX58+ezYsUK9u/fj7e3d5ZjOVoftIiIiIgUfHkaoq9cucL8+fPx8fFh3759BAUFERQUROfOnQkKCmLq1KlUrlyZ7t27c/XqVR5//HFWr15NYGAg+/fvp379+tal6vSobhERERGxlzxLoTt27OCNN96gZs2a9O3bl2HDhjFkyBCcnZ0ZN24ccXFx+Pr6Wtd7LlWqFM8++yxHjx4FoH79+r8W/P/BWQFaREREROwlz5a4+/nnn+nYsSNdu3YFfm3pSE5OZtCgQRgMBtavX8+mTZvIzMwkLCwMgGHDhuVVeSIiIiIi2ZZnIbpixYpMmDCB2NhY9uzZg9FoxM/PDx8fH6Kiojh58iQbNmzAbDb/4amD6nsWEREREUeSpz3Ru3fvJi0tDWdnZ5o3bw5At27diIqKomzZsly8eJGAgIC8KkdERERE5IHk6RML69atS3x8PIGBgQAsXryYqlWrUrx4cQBrgNZNgyIiIiLiyPI0RN++fZulS5cSHx9PQkICNWrUYNiwYXh6emZ5X34J0GazmQkTJnDixAnc3NyYPHkyQUFB1te//fZb5s2bh4uLC0888QTdu3e3Y7WSE+435hs3buSjjz7C2dmZqlWrMmHChHzz8yx/dL/xvmvcuHEUK1aMkSNH2qFKySn3G+9Dhw4xffp0LBYL/v7+vPPOO7i7u9uxYvk77jfe69evZ8mSJTg5OfHEE0/wzDPP2LFaySk//fQTM2fOZPny5Vm2P1Bms+SxmzdvWr7//nvLTz/9ZN2WmZmZ12XkiK+//toyevRoi8VisRw8eNAyYMAA62tGo9HSqlUry40bNyzp6emWrl27Wq5evWqvUiWH3GvMb9++bWnZsqXl1q1bFovFYhkxYoRl27ZtdqlTcsa9xvuuTz75xNK9e3fLO++8k9flSQ6713ibzWZLp06dLOfOnbNYLBbL559/bomNjbVLnZIz7vf3u3HjxpakpCRLenq69b/nkr8tWLDA8vjjj1uefPLJLNsfNLPl+RRZ0aJFCQ8Pp3bt2kD+bt04cOAATZs2BSA0NJSff/7Z+lpsbCyBgYEUK1YMNzc36tWrx/79++1VquSQe425m5sbn376qfX/rJhMJs1S5XP3Gm+AgwcP8tNPP9GjRw97lCc57F7jffbsWYoXL85HH33Ec889x40bN3jooYfsVarkgPv9/a5WrRopKSkYjUYtcFBABAYGEhUV9YftD5rZ7J5e82uABkhNTc3yBEVnZ2dMJpP1NR8fH+trXl5epKam5nmNkrPuNeZOTk6ULFkSgOXLl3Pr1i0aN25slzolZ9xrvK9evcr777/P+PHj7VWe5LB7jXdSUhIHDx7kmWeeYcmSJezZs4fdu3fbq1TJAfcab4AqVarwxBNP8Nhjj/Hoo49StGhRe5QpOaht27ZZVn+760EzW/5NsA7A29ubtLQ06/dms9k6OP/7WlpaWpYBkvzpXmN+9/sZM2awa9cuoqKiNHORz91rvL/66iuSkpLo378/CxYsYOPGjaxZs8ZepUoOuNd4Fy9enKCgICpXroyrqytNmzb9w8yl5C/3Gu/jx4/z3XffsX37dr799luuX7/Oli1b7FWq5LIHzWwK0X9D3bp1+c9//gPAjz/+SNWqVa2vBQcHc/78eW7cuIHRaGT//v3UqVPHXqVKDrnXmAOMHz+e9PR05s+f/4cbZiX/udd49+rVizVr1rB8+XL69+/P448/bn2YlORP9xrvgIAA0tLSOH/+PAD79++nSpUqdqlTcsa9xtvHxwcPDw/c3d1xdnbG19eX5ORke5UquexBM1uers5R0LRu3Zpdu3bx1FNPYbFYmDp1Khs2bODWrVv06NGD1157jb59+2KxWHjiiScoXbq0vUuWv+leY16zZk1Wr15N/fr16d27N/Br0GrdurWdq5YHdb+/41Kw3G+8p0yZwquvvorFYqFOnTo8+uij9i5Z/ob7jXePHj145plncHV1JTAwkC5duti7ZMlhfzez5enDVkRERERECgK1c4iIiIiI2EghWkRERETERgrRIiIiIiI2UogWEREREbGRQrSIiIiIiI0UokVE8hGj0ciiRYuIjIykTp06NGrUiAEDBnD48GEALl26RLVq1bL1yNq/KyoqKssSjjt37qRFixbUqlWLZcuW0aJFC+bPn5/rdYiI2IOWuBMRySdu375Nr169SEpKYujQoYSEhJCWlsayZcvYvHkzCxYsoEKFCrRs2ZKVK1dSv379XK0nLS2N9PR0fH19AXjiiScoXrw4EydOpHjx4hiNRjw8PChSpEiu1iEiYg962IqISD7x3nvvce7cOTZu3JjlQQDTp0/n2rVrTJo0iX/96195Vo+XlxdeXl7W71NSUoiIiKBChQp5VoOIiL2onUNEJB8wGo2sWbOGbt26/emTtMaPH8+sWbMwGAxZtt+4cYMxY8bQpEkTHnnkEZo0acKMGTMwm80AJCYmMnjwYBo2bEhoaCjPP/88x44ds+6/Zs0a2rdvT82aNWnevDlz58617vv7do5q1apx/vx55s2bR7Vq1QD+0M6xbds2OnXqRK1atWjXrh2LFi2yHutuG8q//vUvwsPDad++PUajkQULFtCyZUtq1qxJ27ZtWblyZQ5eVRGRB6eZaBGRfODixYskJycTEhLyp68HBAQAv4bR3xs9ejRJSUl88MEHFC9enP/85z9MmjSJevXq0apVKyZOnIjJZOLjjz/GYDAwa9YshgwZwrZt2zh+/Djjx4/n3XffpWbNmhw5coSRI0cSGBhIZGRklvPExMTQo0cP2rZtS58+ff5Q344dOxg5ciRjx46lQYMGnDp1irfeeovbt28zePBg6/s2bdrEihUruHPnDjExMSxatIj33nuPwMBAvv/+e8aNG0fVqlUJCwv7m1dUROTvUYgWEckHkpOTAShatKhN+zVt2pSGDRtSpUoVAJ599lkWLlzIiRMnaNWqFefPn6datWpUqFABd3d33nrrLU6fPo3ZbObixYsYDAbKlStn/VqyZAllypT5w3n8/f1xdnamSJEi+Pv7/+H1f/3rXzz99NN069YNgMDAQNLS0hg3bhwDBw60vu/ZZ58lODgYgH379uHq6kq5cuUoX748Tz75JBUqVOChhx6y6RqIiOQGhWgRkXygRIkSwK/tGbZ4+umn2b59O6tWreLcuXOcOHGCuLg4axvFwIEDGT16NFu3biUsLIxmzZoRGRmJk5MTTZs2JSQkhCeeeIKgoCCaNGlChw4dKFeunM31Hzt2jMOHD/Ppp59at5nNZu7cucPly5etbSh3Z9QBOnbsyOrVq2nTpg1Vq1alSZMmdOrUCT8/P5vPLyKS09QTLSKSDwQGBuLn58dPP/30p6/v3buXAQMGkJCQYN1msVjo378/06dPx9PTk86dO7NixQrKly9vfU+7du3YuXMnkydPxt/fn/nz5xMZGUliYiIeHh6sWLGC1atX07lzZ44ePcpzzz1HdHS0zfW7uroyYMAA1q5da/1av349W7duzdLj7e7ubv2zn58f69evZ8WKFbRo0YLdu3fzxBNPsGHDBpvPLyKS0xSiRUTyAScnJ7p06cIXX3xBfHx8ltcsFgsLFizg7NmzlCxZ0rr99OnTxMTEEBUVxYgRI3jssccoUaIECQkJWCwWTCYTM2bM4PLly3Ts2JFp06axadMmLl++zA8//MCuXbuYN28etWrVYtCgQXz66ac89dRTfPnllzbXX7lyZc6dO0dQUJD16+TJk8yePfsv99m8eTOffPIJYWFhjBgxgrVr19K4cWPWr19v8/lFRHKa2jlERPKJgQMHsmvXLp555hlGjBhBSEgIiYmJLF68mH379rF48eIsq3MULVoUFxcXtmzZQrFixUhISGD27NkYjUaMRiMuLi4cOXKE/fv3M3bsWHx9fdmwYQOurq488sgjxMfHM2/ePHx8fGjevDmJiYns3buX0NBQm2t/+eWXeemll6hatSpt2rTh3LlzjB8/noiICNzc3P50H6PRyIwZM/Dx8aFevXpcuHCBo0eP8vTTTz/oJRQRyTEK0SIi+YSXlxcrVqwgOjqa999/n19++QUfHx9CQkL47LPPqFGjRpbVOUqXLs3UqVOJiorio48+onTp0rRv357SpUtbn3A4a9Yspk6dyksvvURaWhpVqlRh3rx51tniqVOnsnDhQmbOnIm3tzetWrVi1KhRNtferFkz3n77bRYsWMDcuXPx9fUlMjKSESNG/OU+kZGRXLt2jaioKH755Rf8/Pzo2rUrAwYMsP3iiYjkMD2xUERERETERuqJFhERERGxkUK0iIiIiIiNFKJFRERERGykEC0iIiIiYiOFaBERERERGylEi4iIiIjYSCFaRERERMRGCtEiIiIiIjZSiBYRERERsdH/AZrFSmm3q8NfAAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 720x360 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "acc = [accuracy_dummy, accuracy_gnb, accuracy_knn, accuracy_logreg, accuracy_mlp, accuracy_svm, accuracy_svm_lin, accuracy_svm_poly]\n",
    "f1 =[f1_dummy, f1_gnb, f1_knn, f1_logreg, f1_mlp, f1_svm, f1_svm_linear, f1_svm_poly]\n",
    "classifiers = ['dummy', 'GNB', 'KNN', 'LOGREG', 'MLP', 'SVM(RBF)', 'SVM(LIN)', 'SVM(POLY)']  \n",
    "\n",
    "#accuracy bar plot\n",
    "fig, ax = plt.subplots(figsize=(10,5))\n",
    "ax.set_xlabel('Classifiers',fontsize = 15) #xlabel\n",
    "ax.set_ylabel('Accuracy', fontsize = 15) #ylabel\n",
    "\n",
    "plt.yticks(rotation=30, size = 10)\n",
    "plt.xticks(rotation=0, size = 10)\n",
    "\n",
    "ax = sn.barplot(x=acc, y=classifiers)\n",
    "ax.set_title(\"Accuracy of all classifiers\", fontsize= 15)\n",
    "\n",
    "\n",
    "plt.tight_layout()\n",
    "plt.show()\n",
    "\n",
    "#f1-score bar plot\n",
    "fig, ax = plt.subplots(figsize=(10,5))\n",
    "ax.set_xlabel('Classifiers',fontsize = 15) #xlabel\n",
    "ax.set_ylabel('F1-score', fontsize = 15) #ylabel\n",
    "\n",
    "plt.yticks(rotation=30, size = 10)\n",
    "plt.xticks(rotation=0, size = 10)\n",
    "\n",
    "ax = sn.barplot(x=f1, y=classifiers)\n",
    "ax.set_title(\"F1-score of all classifiers\", fontsize= 15)\n",
    "\n",
    "\n",
    "plt.tight_layout()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Except of the dummy classifier we see an increase at the scores due to oversampling"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- Reqularization of features "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Standard Scaler"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\nanos\\miniconda3\\lib\\site-packages\\sklearn\\preprocessing\\_data.py:235: UserWarning: Numerical issues were encountered when centering the data and might not be solved. Dataset may contain too large values. You may need to prescale your features.\n",
      "  warnings.warn(\n",
      "C:\\Users\\nanos\\miniconda3\\lib\\site-packages\\sklearn\\preprocessing\\_data.py:254: UserWarning: Numerical issues were encountered when scaling the data and might not be solved. The standard deviation of the data is probably very close to 0. \n",
      "  warnings.warn(\n"
     ]
    }
   ],
   "source": [
    "from sklearn import preprocessing\n",
    "scaler = preprocessing.StandardScaler().fit(X_train_ros)\n",
    "X_train_scaled = preprocessing.scale(X_train_ros)\n",
    "X_test_scaled = scaler.transform(X_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- dummy"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Dummy Accuracy: 25.732217573221757% \n",
      "\n",
      "              precision    recall  f1-score   support\n",
      "\n",
      "     class 0       0.26      1.00      0.41       246\n",
      "     class 1       0.00      0.00      0.00       231\n",
      "     class 2       0.00      0.00      0.00       479\n",
      "\n",
      "    accuracy                           0.26       956\n",
      "   macro avg       0.09      0.33      0.14       956\n",
      "weighted avg       0.07      0.26      0.11       956\n",
      "\n",
      "\n",
      "F1-score is : 0.13643926788685523\n",
      "Cross validation score is : 33.19155614215813\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\nanos\\miniconda3\\lib\\site-packages\\sklearn\\metrics\\_classification.py:1308: UndefinedMetricWarning: Precision and F-score are ill-defined and being set to 0.0 in labels with no predicted samples. Use `zero_division` parameter to control this behavior.\n",
      "  _warn_prf(average, modifier, msg_start, len(result))\n",
      "C:\\Users\\nanos\\miniconda3\\lib\\site-packages\\sklearn\\metrics\\_classification.py:1308: UndefinedMetricWarning: Precision and F-score are ill-defined and being set to 0.0 in labels with no predicted samples. Use `zero_division` parameter to control this behavior.\n",
      "  _warn_prf(average, modifier, msg_start, len(result))\n",
      "C:\\Users\\nanos\\miniconda3\\lib\\site-packages\\sklearn\\metrics\\_classification.py:1308: UndefinedMetricWarning: Precision and F-score are ill-defined and being set to 0.0 in labels with no predicted samples. Use `zero_division` parameter to control this behavior.\n",
      "  _warn_prf(average, modifier, msg_start, len(result))\n"
     ]
    }
   ],
   "source": [
    "dummy = DummyClassifier().fit(X_train_scaled, y_train_ros) \n",
    "\n",
    "pred_dummy = dummy.predict(X_test_scaled) \n",
    "  \n",
    "accuracy_dummy = dummy.score(X_test_scaled, y_test) * 100 \n",
    "print(\"Dummy Accuracy: {}% \\n\".format(accuracy_dummy))\n",
    "\n",
    "target_names = ['class 0','class 1','class 2']\n",
    "print(classification_report(y_test, pred_dummy, target_names=target_names))\n",
    "print()\n",
    "\n",
    "f1_dummy = f1_score(y_test, pred_dummy, average='macro')\n",
    "cross_val_dummy = np.mean(cross_val_score(dummy, X_train_scaled, y_train_ros, cv=10))*100\n",
    "print('F1-score is :', f1_dummy)\n",
    "print('Cross validation score is :', cross_val_dummy)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- GaussianNB"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "GaussianNB Accuracy: 94.0376569037657% \n",
      "\n",
      "[[0.97154472 0.0203252  0.00813008]\n",
      " [0.02597403 0.94805195 0.02597403]\n",
      " [0.03966597 0.03966597 0.92066806]]\n",
      "              precision    recall  f1-score   support\n",
      "\n",
      "     class 0       0.91      0.97      0.94       246\n",
      "     class 1       0.90      0.95      0.92       231\n",
      "     class 2       0.98      0.92      0.95       479\n",
      "\n",
      "    accuracy                           0.94       956\n",
      "   macro avg       0.93      0.95      0.94       956\n",
      "weighted avg       0.94      0.94      0.94       956\n",
      "\n",
      "\n",
      "F1-score is : 0.9372455231183117\n",
      "Cross validation score is : 95.0070821529745\n"
     ]
    }
   ],
   "source": [
    "gnb = GaussianNB().fit(X_train_scaled, y_train_ros) \n",
    "\n",
    "pred_gnb = gnb.predict(X_test_scaled) \n",
    "  \n",
    "accuracy_gnb = gnb.score(X_test_scaled, y_test) *100 \n",
    "print(\"GaussianNB Accuracy: {}% \\n\".format(accuracy_gnb))\n",
    "\n",
    "cm = confusion_matrix(y_test, pred_gnb) \n",
    "cm = cm.astype('float') / cm.sum(axis=1)[:, np.newaxis]\n",
    "print(cm)\n",
    "\n",
    "target_names = ['class 0','class 1','class 2']\n",
    "print(classification_report(y_test, pred_gnb, target_names=target_names))\n",
    "print()\n",
    "\n",
    "f1_gnb = f1_score(y_test, pred_gnb, average='macro')\n",
    "cross_val_gnb = np.mean(cross_val_score(gnb, X_train_scaled, y_train_ros, cv=10))*100\n",
    "print('F1-score is :', f1_gnb)\n",
    "print('Cross validation score is :', cross_val_gnb)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- KNeighborsClassifier"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAD/CAYAAAD/qh1PAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8rg+JYAAAACXBIWXMAAAsTAAALEwEAmpwYAAAnVElEQVR4nO3de3RTZb438G+SJm3TJuktbekNSylQrgVEoYLAKQNVzow64230iDKOtxdhXkbUNWu955w/3nPeNXNmzTnjAcbRmZHWAfSMKKioiPVSy6BF5E4vQC/0Xtq0Tdrck73fP0ojSBMamqbt3t/PWi6bdnfn+TXh26fPfp5nK0RRFEFERJKlHOsGEBHR6GLQExFJHIOeiEjiGPRERBLHoCcikjgGPRGRxDHoiYgkLmKsGzCUnh4rBGF40/sTE2NhMvWPcovGH7nWDci3dtYtL8HUrVQqEB8f4/fr4zLoBUEcdtAPHi9Hcq0bkG/trFteQlU3h26IiCSOQU9EJHEMeiIiiWPQExFJHIOeiEjiGPRERBLHoCciGkM2hxvvH27A8384jIrKjlF5jnE5j56ISOrMVhcOftOIz4+1wOHyYm5OInIzDKPyXAx6IqIw6jLbcaCiEeWn2uDxCLh5RjLWLpmMrBTdqD0ng56IKAwEUcTBI014u6wWALBkdiruXDwZqQnaUX9uBj0R0SgzW134y/5KnKnvxoJpRjy0KhcJ+qiwPT+DnohoFJ2t78af9lfC7vTgkTXTsSI/DQqFIqxtYNATEY0Cj1fA3i/r8FFFI9KSYrDlwXxkGGPHpC0MeiKi6zD3O9HSZYXV4YHV7obV4YbV7kG/ww2X24sIlRIRKgVUSiVUKgUiVErUNPagvq0PK/LT8EBhLiLVqjFrP4OeiCa0jh4bLrb3ITk+GinxWkRHhi7WHC4PPvz6Ij4+0gS3R7jqa5oIJWKi1dBEKOEVRHi8AjxeEV5BhNcrICoyAv/r7tm4eUZyyNpzoxj0RDRhfVtzCX/eXwWn2+v7nF6rRnKCFinx0bgpVY/b56VBHRHc2lBBFHH4dDve/rIW5n4XFs9MwbJ5adBp1YiJUiMmKgKaMeyhB4tBT0QTjiCKeP/vDXj3UD2mpOnxYGEuzP1OdPTY0dFtQ0ePHWfqu/H30+344ngL1t+Zhylp+mGdu6axB29+egEXO/owJU2PZ++Zg5z00VnIFC4jDvrS0lK0trZCq9VizZo12LlzJ3p6erBlyxa8+uqrcDgceOKJJ2AwTOwfFBGND06XF3/+oBLf1nSiYHYqHi2aDnXE0L3rU7VdKDlQg3//61GsWZSFu5dlD9kT9woCztb3oOxEC46f70K8LhJP/nAmbp2ZEvYZMqNhxEFfWVmJTZs2Yfv27dDpdFi7di22b9+OqqoqzJ49G+np6aioqMDq1atD0V4ikrEusx1b3z6N5s5+PPAPU7F6UWbAIJ6bk4T/+3gc3vriAg4cacTx851Yf2cepmXGQRRFNHb04/CZdlRUdcBidSEmKgJ3L83GmluzxvTiaaiNOOhVqoEfhiiK6O7uhl6vx/Lly6FSqXwvgCAIgU5xjcTE4KYgGY2jt3R4PJNr3YB8a5dj3aIoIiJSjaqGbmzfcxJer4B//fliLJyRMuxzbHlkEX6wuBNb/3YCv951DMvy09HQZkFTRx8iVEosmpmClQszcXNeStDj+aMpVK+3QhTFEd199sCBA+jo6IDBYMDSpUvxyiuvQK1W45lnnsHLL7+MqKgoPPbYY9Drhzc+BgAmU/+wb4prNOrQ2dl3o82fsORaNyDf2qVWt8crwOHywuX2wuHywnn5/w6nBx09drSarGgzWdHebYfV7gYApCRoseknczApMeaGntPp8uLtL2vx+bEWZKfpUTArFYvykhETpQ5laSERzOutVCoCdpBHHPSjgUF/fXKtG5Bv7ROpblEUYTI7UNdmQWuXFRabG31WF8w2F/qsLlhsbtidnoDn0GvVmJQYgykZcYiLUSMtMQZTMwwhGVIRRBHKcT72Hsqg56wbIhoxm8OD+nYL6lotqG+1oK7VDIttoBeuABCrVUOv1UCnVWNyqg56rQaxWjWiNRGI1KgQpVFBo1YhSq1CpEYFY1w0YqMHetmj8QtuvId8qDHoiSgobo8XjZf6Ud9qQX1bH+rbLGjvtvm+PilRizlTEjElTY/sND0yjLGIUI2fcW85YtAT0XWJoojKiz344HADzjeb4b08tGqI0WBKmh5LZqdiyiQ9sifpoB2H491yx6AnooDONfVi75d1qGnqRbwuEqsXZQ701ifpEa+LlMQ8c6lj0BPRkOpaLdhbXoez9d3Qx2jw0KpcLM9P87s4icYvBj2RDLk9Xnx2rMV3M+qICCXUqoGdF9UqJexOD6obexEbrcb9K6di5YJ0SS0gkhsGPZGMeAUBfz/djncP1aOnz4mcND20UerLOy8KcLgEuD0iABH33D4FqxZmhHQ3SBobfAWJRpnHK6CisgMWmwu3z0sb8eKcwaUvwYyNC6KIb2s6sffLOrR32zAlTY+f/+NM5E2OH1FbaGJg0FNYeS9vh6FSSn+6nSCI+LqyHe8dasClXjsAYP/hi1izKBM/WJR5Qz3lo9WX8PrHNdBGRmBRXjIWzUhGZnLskKEviiI6zQ5UX+zB58dbcLG9D2lJMXj2x3MwPzeJF1FlhEFPN6yh3YLPjrVg1cIMZKVcf08Oi82F/3zzBHqtLhTdkoUV89MQpZHeW1AQRXxTdQnvHqpHe7cNmcmx2PiTOUjUR+HdQ/XYd6genxxtQtGtWShcmDGsn4HT7cW2t07g468v4qZUHWKiIvDR14344KuLmJSoxaIZyViUl4IotQrVjT2ovtiD6sYemCxOAECSIQqPr83DklmpUCoZ8HLDLRAmqLGsu9viwDtf1uHwmXYAQHSkCpt+MhfTs/wPA1isLvz2zeO41GPHlEl61DT1IiYqAqsXZaJwYSa0UcMP/PH4mnu8Alq7rKhvs6D0aDNauqxIN8bg7qXZmD/NeNVKzIZ2C/aV1+NUrQmx0WoU3ZqFW2YkIykueshzN13qxx/fPYP2bhvuuHUy7l6WjQiVEhabC8dqOnGkqgM1jb248l9MbLQa07PikDc5HjOy4jEpUTthe/Dj8fUOB+51cwW+CcLH4fLgQEUjDlQ0QhCB1YsysWRWCv6w7wy6zA48c9ds5OcmXfN95n4nfvvmCXT12vGLe+ci76YE1LaYsf9wA07WmhAdGYHChRlYvSjTt+w9kHDWLooiPF4RTvfA5lvOy/85nF60mqy42N6Hxo5+tHT1w+MdeM+mJmhx19JsLMpLDrjUvrbFjH3ldTjb0AMAyDDGID83CflTjbhpkg4KAJ8da8H/fHYBMVER2PJPC5EeP/QvA3O/E8fOdcIjiJiRFY90Y4xklvnz3/j1MeglKpx1C6KIv59uwztf1sHc78Itecm4d3mOrwfaZ3Ph92+dxMX2fqy/cwZumzPJ9729/U789o3jMFkc2HzfvGt6/Rfb+7D/qwZ8W9OJ6EgVfnx7DlbOTw84vDDatVusLpy40IVj5zpRdbHnmnuFXik2Wo3JKbHIStEhK0WHyak6JMdHBxWy7d02nDjfhZMXunCuuReiOLDiNNEQhbpWC+bmJOJna/OQMzmR73UZYdBfgW+CoV1oNuMvH1YhNjri8lasKcPqLQ9lX3kd3vt7A3LS9HigMBdTh7itmt3pwbZ3TqPqYg8eLMzF6kWZ6Olz4j/eOI7ePic23z8P0zLj/D5HS2c/3vz0PM429GByqg7r1kxH9qSht7Y2GnVoae1F46V+TE6JDckCnq5eO46d68Sxc50432KGKA6Ma8+bmoS4WA00EQObbWnUSkSqVYhUq5CaoA35ytB+uxun60w4cb4LDe0WrFqYiVU3Z0ChUPC9LjMM+ivwTXCtQ6fa8PrH1YjXRUKjVqGl0wqVUoF5U5OwZFYq5uYkDvvmCueaevGb3cewZFYqHl+bFzDU3B4Br753Ft+e68QPbs7EydouWKwubL5/HnIz4q77XKIo4kjVJbz56XlYrC6sXJCOH9+e4xu/d7m9OF1nwun6HlScbYfT7cW0DAN+cd+8G57rbXd6sOuTc77rDRnGWCyYloQF04x+Z7OMFb7X5YVBfwW+Cb4jCCLe+uICPj7ShLzJ8Xjm7tmIiYpA06WB26V9Xfnd7dLWLrkJRbdmBXwOm8ONf33tCFRKJf51/aJhhakgiHj942p8ebIN0ZEq/PL+/KBvrGxzeLC3vA6fHWuGTqtB0S1ZaGi34OQFE5xuL/QxGszPTUJyfDTeKatDZnIsfvlAftB/sdS2mvHqe2fRZXag6NYsLJ+XhuR4bVDnCCe+1+WFQX8FvgkG2BwevPLeWZyuM6FwQQYeKJx6zdawXkFAZUMPSo8243SdCXctzcZdS7OHPL8oinj1/Up8U3UJv3pkAXLShh/Woiji0Ok23JSqR2ZycLeFvFJDuwWvH6hBQ3sfYqPVWDjdiJtnJGPZgkx0d1sBACcudOEPe88gJSEaWx7IhyE28rrnFQQRH3x9Ee+W1w/cBPpHM4f1F8dY43tdXhj0V+CbAOjotuG/3z6FSz12PPyDaVgxPz3g9wqiiB0fVuHvp9tx99Js/GiIsD98pg1/3l+Fe26fgh8W3DQaJQyLIIho67YhNSHat8jq+695ZUM3tr59GnGxGmx5cD4SDVF+z9dtceDV9ytxrqkXt+QlY92a6RNmW12+1+WFd5ginzaTFf/vr99CoVBgy4P5AeeyD1IqFFh/Rx4gAvsO1QMK4Ee3fRf2l3rt2HnwHKZlxmHt4smj2fzrUioVSE8KfH/QmTcl4LkH8vFfb53Er3d9iy0/nY+Uy0Mwoiiit9+Fpkv9uNhuwcdHmuAVRTy+Ng8Fs1PH1Rg80Whh0E9gXkHAn/dXAQD+z6M3I9nPgpuhKJUKrL8zDyKAfeX1UAD44W3Z8HgF/Om9s1AoFHjiH2dOmFWUUzMMeOGn8/G7/zmBX+88hltnpqC5sx9Nl/rRd/mWdoPHPb42z/eLgEgORhz0paWlaG1thVarRWFhIbZt2wYA2LBhA1577TU4nU48+eSTMBqNI24sXe1ARSPq2yx46kezggr5QUqlAj+7Mw+iCOwtrwcAeAURta0WPH3XrIBDIOPR5FQdXnx4Af7rbyfw2bEWpBtjMG9qEjKTY5GVHIvM5NgJM0xDFEojDvrKykps2rQJ27dvR2RkJDZv3ox9+/bBbDbDZDJBEARERwcfQhRYQ9vAMvqbpxtxS17yDZ9HqVTg8bV5AL4L+9tmp+KWvJSQtDPc0pNi8B9PF0CEKIuN04iGY8RBr1INLFYRRRFarRalpaUwGAxQKBRYunQp0tPTUVFRgcLCwmGfM9BFhaEYjdffUEtKPF4B//bSl4jVqvG/H1o4rJkm1/PiY7fgD3tOoralF5t+umDc93zl9poPYt3yEqq6Rxz0OTk5KCkpQUZGBqqrq1FcXIyCggIsWLAAx44dQ1VVFR566KGgzslZN4G9d6gedS1mbLhnNlx2FzrtrpCc98GVOQAAa58D1j5HSM45GuT4mgOsW244vfIKcnsTXGzvw7+9fhRL56Xj0TXTxro5Y0Jur/kg1i0vnF4pAcfPdeJCqxmiMDCvXRDFgY8hwqDVYN7UJGSlXL0E3+0R8JcPKhEbrcZTP54Dh9U5hhUQ0UTBoB8D31Rfwsv7zkClVEClVEChVECpUECpGLg9nNXhxr5D9UjURyJ/qhH505IwPTMO7x+uR3OnFZvunQudVsOgJ6JhYdCHWW2rGX/eX4mpGQY8/2D+kDsvWmwunLzQhRPnu1B+qhWfHmtGdGQEnC4vbpuTivyp1+75TkTkD4M+jLp67di65xTiYjV49sdz/G6vq9dqsGxuGpbNTYPT7UVlQzeOn+9CT58TPy3MDXOriWiiY9CHic3hwUt7TsHtFfHiffOg12qG9X2RahXm5xoxP5cLzojoxnBFSQjUtVqwedshFH9UjZYu6zVf9woCXr58z89n75mNSYmB924hIgol9uhD4O2yWjhdXnx1th1fnmzF7OwErF6UiVnZCQCAXZ+cx9n6bjx2xwzk3ZQwxq0lIrlh0I/Quabegdvn/cNULJmdii+Ot+CzYy34z7+dxKRELXLSDTh0qg13LM7C7fPSxrq5RCRDDPoRevdQPfQxGiyfn45ItQo/vC0bRbdOxpGqDnzyTRMOnWrDwmlG/GR5zlg3lYhkStZB/+f9lbBYXZg/zYj5uUmIC3LPmJrGHt/NsCPV382gUUcocducSSiYnYqWTitSE7VQct9zIhojsg16ryDg67MdUKkUOFPfjb9+XIOcNL0v9IdzwfTdQ/UwxGiwIn/oIRmFQoGMEdxKj4goFGQb9L19LgiiiH9aNQ256QYcO9+F4+c6seeLWuz5ohazshPw7D1zEKkZeq57TWMPqht78dPCXGjUQx9DRDQeyDbou8x2AIDREI10YyzSjbH4YcFN6LY48HVlB94uq8VLe07iF/fNu2pYZtC7h+phiNVguZ/ePBHReCHbefRd5oFteJO+dxelBH0U7lw8GT//x5moaezF1rdPweX2XnVM9cWB3vydiyezN09E455sg950OegT9EPfLm/JrFT8bG0eqhp6sG3vabg934X9YG/e39g8EdF4Itug7zI7YIjVQB3h/0dw25xJeOyOGThT143te8/A7RFQdbEHNU29WLt4st+9aoiIxhPZjtGbLI5rhm2GsmxeGryCiNc/rsHL+87A6nAjjmPzRDSByDbou8x2TEkzDOvYFfPT4RVE7PrkHADg4R9MY2+eiCYMWQa9IIjotjhxS971e/SDChdmQKkATlww4fZ5k0axdUREoSXLoO/td8IriEj0cyHWn5ULMrByQcYotYqIaHTI8mKsv6mVRERSNOIefWlpKVpbW6HValFYWIht27YBADZs2IDXX38dKpUKDz/8MBISxs/2vINTKxMZ9EQkAyMO+srKSmzatAnbt29HZGQkNm/ejH379uHSpUswm81ISEhAVFRwgZqYGNz+MEajLqjj7Z5WAMD0HOOQq14nimDrlhK51s665SVUdY846FWqgaAURRFarRalpaUwGAxwu92YPn06Zs6cifLycqxZs2bY5zSZ+iEI4rCONRp16OzsC6rNF1vNMMRoYOm1BfV948mN1C0Vcq2ddctLMHUrlYqAHeQRj9Hn5OSgpKQEGRkZqK6uRnFxMZqampCQkIDGxkYcPHgQ+fn5I32akOoyOzhsQ0SyMeIefVFR0VWPd+7c6fv4hRdeGOnpR4XJ4sBNqfL8U5CI5Ed2s24EUYSJPXoikhHZBb253wWvICLJED3WTSEiCgvZBf3gPvScQ09EciG7oPfNoQ9yVSwR0UQVMOjfeuutcLUjbLq4WIqIZCbgrJujR4+irq4OCoVi3M6gCVaX2QG9Vj2hF0oREQUjYI/+jjvuQEREBAoKCsLVnlFnMtuRyAuxRCQjAYP+3LlzeO6553DhwoVwtWfUdZmHd8MRIiKpCBj0vb29OHjwIDo6OsLVnlEliCJMFifH54lIVgKO0a9atQoajQbLly8PV3tGlcXqgscrsEdPRLISsEdfXFyMEydOYM+ePeFqz6jiPvREJEcBe/Tr169HSkpKuNoy6gYXS/FiLBHJScAe/Y4dO/DZZ5/h888/D1d7RtXgYqkkLpYiIhnx26N3uVx48cUXAQAmkylsDRpNJrMDsdFqRGo4h56I5MNvj/4vf/kLPv/8c6Snp+Pw4cPhbNOo4dRKIpKjgEM3drs9XO0ICwY9EcmR36Gb8vJyqFQqdHd349SpU3j66afD2a6QE0URJosD+VOTxropRERh5Tfod+/eHc52jDqLzQ23R+BiKSKSHdlsU/zd1EoGPRHJS8B59N9++y2qq6sBAA8//HBYGjRaTFwsRUQyFTDojxw5grvuuguiKPo9prS0FK2trdBqtSgsLMS2bdsAABs2bIDdbsfvf/97/Pa3vw1tq29AF284QkQyFTDoz58/71ss5a9HX1lZiU2bNmH79u2IjIzE5s2bsW/fPvT29uLw4cPIysoKulGJibFBHW806q57jNXlhU6rQVZGfNDtGa+GU7dUybV21i0voao7YNA/+eST6OvrQ1xcnN9jVKqBxUeiKEKr1aK0tBQGgwHNzc2wWq04ceIEWlpakJ6ePuxGmUz9EAT/f0VcyWjUobOz77rHNbf3IUEfOaxjJ4Lh1i1Fcq2ddctLMHUrlYqAHeSAF2PfffddOBwO7N+/3+8xOTk5KCkpQUZGBqqrq1FcXIympibk5OTgqaeeQn5+flAhP1q6zHZufUBEshSwR28wGLBs2TJUVVX5PaaoqOiqxzt37rzq8caNG0fQvNAQRREmswNzpiSOdVOIiMIuYI8+Pz8fv/vd7zBnzpxwtWdU9NnccHm4Dz0RyZPfoC8tLUVZWRk8Hg/Ky8vD2aaQM1kGp1Zye2Iikh+/QzcFBQWIi4vDzJkz8dVXX4WzTSHnm1rJHj0RyZDfoN+zZw9Onz6NyZMnIzFxYo9t+1bF8mIsEcmQ36GbdevWYc2aNQAw4W8O3mV2ICYqAtqogNeeiYgkKWDyVVdXIz4+fsJvV2wyOzhsQ0SyFXDWjdvthtvtRnt7e7jaMypMZgcvxBKRbAXs0S9duhR5eXn49NNPw9WekBNFEV1mB2belDDWTSEiGhMBe/Rnz55FbGwsWltbw9WekOu3u+F0ezl0Q0SyFTDo+/r6UFxcjP7+/nC1J+T6bG4AQFysZoxbQkQ0NvwO3XR2dqKoqAgWiwV6vT6cbQopm8MDANBGcsYNEcmT3/Rra2vDV199hZSUFABAbm5u2BoVSjbnQI8+mlMriUimAi6Yamtrw9SpU6FQKMLZppAa7NHHRKnHuCVERGPDb9Bv3LgRPT09EzrkAcDKoRsikrmAQzd1dXW+xxN36OZy0HPohohkyu+sm7lz58JiscBkMvluED4R2R0eaNRKRKgCTjAiIpKsgN1ch8MBq9WKhISJu9jI6nBz2IaIZC1gAk6fPh1TpkxBU1NTuNoTcjanB1peiCUiGQsY9BcvXkRtbS0uXbqEgoKCcLUppGwOD8fniUjWAg5cOxwOmM3mCT10Y3N4OHRDRLI24qGb0tJStLa2QqvVorCwENu2bQMAPPPMM3j55ZcBABs2bBizXxY2pxuTkrRj8txERONBwB59VVUVXn75Zfztb3/ze0xlZSXWrVuHjo4OREZGYvPmzcjOzkZPT4/vY6vVGvKGDxd79EQkdwETMCoqCvn5+fB6vX6PUalUAAa2A9ZqtSgtLYXBYEBubq7v48zMzKAalZgYG9TxRqNuyM+Logi70wNjQozfYyYyKdY0XHKtnXXLS6jqDhj0KpUKGo0GZ86c8XtMTk4OSkpKkJGRgerqahQXF6OgoABHjhzxfdzS0oL09PRhN8pk6ocgiMM61mjUobOzb8iv2Z0eCCIgegW/x0xUgeqWOrnWzrrlJZi6lUpFwA6y36DftWsXlEolrFYrsrOz/Z6gqKjoqsc7d+4c8uOx4Nu5krNuiEjG/CbgihUr4PF44PF44HA4wtmmkBnc/iCGQU9EMub3Ymx6ejo++OAD5OTk4ODBg+FsU8jYHANbFPNiLBHJWcBZNy6XC5988gnsdnu42hNS3w3dcGUsEclXwKD/+c9/jrS0NKxfvz5c7Qkp7lxJRHSdWTc7duyA1+uF0+nEiy++GK42hYyVF2OJiAL36Pv7+xERMXFDcnCMPlozcWsgIhopvwno8Xiwfv166HQ6mM3mcLYpZGxOD6IjVVAqJ/ZdsoiIRsJv0B8+fBinTp3yPX722WfD0qBQGtj+gBdiiUje/Ab9J598Ap1OB1Ec3grV8YhbFBMRBQj6ZcuWobm5GcnJySgsLAxnm0LG5uSGZkREfi/Grl69GuvXr4coinj88cfD2aaQsTnc7NETkez5TcGtW7eiv78fS5YswZ/+9KdwtilkBm4jyKAnInnz26NXKBTQ6XQ4c+YMduzYEc42hQwvxhIRBejRT8RZNlfyCgIcLi83NCMi2Qu4YGoiszsHbpYSzaAnIpmTbNBz50oiogHSDXpuaEZEBEDCQT+4oVkMtygmIpmTbNDbB3eu5NANEcmcZIOeQzdERANGnIKlpaVobW2FVqtFYWEhtm3bBgDYsGEDdu/eDYfDgSeeeAIGg2HEjQ2GdfBiLIOeiGRuxD36yspKrFu3Dh0dHYiMjMTmzZuRnZ2N5uZmzJ49G3fddRcqKipC0dag2BweKBUKRKpVYX9uIqLxZMTdXZVqIEhFUYRWq0VpaSkMBgOUSiUUioF94AVBCOqciYmxQR1vNOqu+ZyoUCAmWo3kZH1Q55pIhqpbLuRaO+uWl1DVPeKgz8nJQUlJCTIyMlBdXY3i4mIUFBRgwYIF+PDDDxEVFYXHHnssqHOaTP0QhOFtj2w06tDZ2XftOXrtiI5UDfk1KfBXtxzItXbWLS/B1K1UKgJ2kEcc9EVFRVc93rlzp+/jF154YaSnv2ED+9xwfJ6ISMKzbrhFMRERIOWgZ4+eiAiA1IOeq2KJiCQc9LzpCBERAIkGvdvjhdsjcOiGiAgSDXqbb0MzBj0RkTSD/vI+N7zpCBGRVIPet3MlL8YSEUky6K0cuiEi8pFk0Nuc3LmSiGiQJIOeNx0hIvqOJIN+cOiGPXoiIokGvc3pgTpCCXUE96InIpJm0HOfGyIiH4kGPXeuJCIaJM2g5z43REQ+0gx6h4eLpYiILpNu0LNHT0QEQKpBz6EbIiIfyQW9KIqcdUNEdAXJBb3D5YUgiuzRExFdNuI0LC0tRWtrK7RaLe69914cPXoUBw8exPPPP4+XXnoJTqcTTz75JIxGYyjae112J7c/ICK60ojTsLKyEps2bcL27dsBABqNBjqdDgqFAiaTCYIgIDo6OqhzJibGBnW80ajzfWz1iACAScn6qz4vRVKvLxC51s665SVUdY846FWqgW0GRHEgYOfOnYuysjI0Nzdj6dKlSE9PR0VFBQoLC4d9TpOpH4IgDutYo1GHzs4+3+OWNjMAwO1yX/V5qfl+3XIi19pZt7wEU7dSqQjYQR7xGH1OTg5KSkqQkZEBl8vl+3x8fDyOHTuG0tJS5OXljfRphs3quLxFMYduiIgAhKBHX1RUdM3nNm7cCAD453/+55GePmi8XywR0dUkN+tm8H6x2iiujCUiAqQY9Jd79NGR3KKYiAiQaNBHaVRQKSVXGhHRDZFcGtqc3KKYiOhK0gt6bn9ARHQVBj0RkcRJL+idHs64ISK6gvSCnrcRJCK6ivSC3smhGyKiK0kq6AVBhN3pZY+eiOgKkgp6roolIrqWNIOeQzdERD6SCno7NzQjIrqGpILeNrhFMYOeiMhHUkFv9W1oxqAnIhokqaD/7mIsg56IaJC0gt43Rs9ZN0REg6QV9E43FAogUsO96ImIBkkr6C9vaKZUKMa6KURE44a0gt7p4fg8EdH3jDgVS0tL0draCq1Wi3vvvRdHjx7FwYMH8atf/QovvfQSVCoVHn74YSQkJISivQEN9Og5Pk9EdKUR9+grKyuxbt06dHR0AAA0Gg10Oh1qampgNpsBAFFRUSN9mmGxOdijJyL6vhGnoko1cOFTFEUAwNy5c1FWVga3243p06dj5syZKC8vx5o1a4Z9zsTE2KDaYDTqAABOj4DkRK3vsdTJpc6hyLV21i0voap7xEGfk5ODkpISZGRkwOVyQaPRAABmzJiBjz76CM3NzXjkkUeCOqfJ1A9BEId1rNGoQ2dnHwDAYnVChVjfYym7sm65kWvtrFtegqlbqVQE7CCPOOiLioqu+dzGjRsBAC+88MJITx8UO4duiIiuIZlZN26PAJdH4M6VRETfI5mg5170RERDk07Qc+dKIqIhSSfoedMRIqIhSSfouaEZEdGQJBP0URoVVEoFEg3hWZxFRDRRSGacY2q6Af/9i2W86QgR0fdIpkevUCgY8kREQ5BM0BMR0dAY9EREEsegJyKSOAY9EZHEMeiJiCSOQU9EJHHjcj6iUhnczb2DPV4q5Fo3IN/aWbe8DLfu6x2nEAdvDUVERJLEoRsiIolj0BMRSRyDnohI4hj0REQSx6AnIpI4Bj0RkcQx6ImIJI5BT0QkcQx6IiKJG5dbIAxXaWkpWltbodVqce+99451c8LijTfegNlsRnR0NBQKhaxqf+mllzBr1ixZvebl5eU4f/48VCqVrF7vjz76CF1dXTCbzdDr9bKo2+Fw4F/+5V8wa9Ys32sdFxcXkvf7hO7RV1ZWYt26dejo6BjrpoRNYWEhHnvsMXR3d8uq9rKyMiQlJaG4uFhWdVdUVMDtdmPv3r2yqtvtdqOzsxM7duyQTd2dnZ2YMWPGVbkWqoyb0EGvUqkAAHLarker1WLr1q1Qq9UA5FN7RUUFamtrsXjxYgDyqbunpwdPPfUUampqAMin7qamJvzyl79Ef38/AHnUnZmZibi4OGRlZQEYqDlUGTehNzU7cOAAOjo6YDAYcPfdd491c8LiueeeQ1paGnJycmA2m2VVe0VFBXp6emT1mn/88cdobm6G3W6HTqeTTd1vvPEGHA4HBEFARESEbOp+55130NPT46s5KioqJO/3CR30RER0fRN66IaIiK6PQU9EJHEMeiIiiWPQExFJHIOeJpytW7fir3/9KwDgN7/5zbC+Z7jHXclsNmPLli3o6uoCANxzzz1oaWmB1WrF1q1brzq2oaEB58+fv6adVqsVANDc3IydO3cG3YYbaTfR903olbEkXyaTCdXV1QAGpl3abDZotVrYbDbs2bMHN998MxobG5GYmIhVq1bhwoULKCkpQXR0NGJiYtDS0gK73Y7Fixfj/fffx7p16zBt2jR8+OGHaG9vR2dnJxYuXAiFQoH4+HgAQH5+Pl555RU8//zzAIAPPvjgqvPYbDZ88cUX0Gg0OH78OHJycvD666+jsbERGzZswOHDh2EymbBixQp8/fXXiIqKgiAIsFqtUKlUiImJgSiKiImJ8a2CFEURf/zjH7Fq1SpMnTp1bH7YNOGxR08T0uOPP47du3fD4XBAoVAAAARBAABkZGTg/vvvR3x8PJYvX46GhgbMnTsXjz76KBoaGlBWVgadTge73Y6+vj6sXLkS06ZNAwCcPXsWP/vZz5CVlQW9Xo958+b5Fq1ERUXhoYcewmuvvQYA15wHGPgr4NFHH0VCQgIA4JFHHkFaWhrsdjtWrVqFp59+GmVlZbBYLHj00UdhMpkgiiIeeeQRzJ8/H93d3b46AKCqqgpdXV0MeRoRBj1NSEqlEk899RROnDiBjIwMfP755ygrK/N9DYDvFwAA1NbWYu/evcjNzcWSJUtgtVqh1+sRGxvrOx4AZsyYgZKSErS1tWHSpEnXPO+MGTN8PfzvnwcYWLm8a9cu33DPYBtEUcQ333yDXbt2YcWKFYiNjcXu3buRkpLiO66+vh7x8fFoa2vzPd/MmTOxcuVK7Nu3L1Q/OpIhLpgiCqE333wTXq8XGo0G991331g3hwgAg56ISPI4dENEJHEMeiIiiWPQExFJHIOeiEjiGPRERBLHoCcikrj/D4RLJIWPKJYwAAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Optimal k is:  1\n",
      "KNN Accuracy: 73.9539748953975%\n",
      "              precision    recall  f1-score   support\n",
      "\n",
      "     class 0       0.71      0.78      0.74       246\n",
      "     class 1       0.61      0.90      0.73       231\n",
      "     class 2       0.90      0.64      0.75       479\n",
      "\n",
      "    accuracy                           0.74       956\n",
      "   macro avg       0.74      0.77      0.74       956\n",
      "weighted avg       0.78      0.74      0.74       956\n",
      "\n",
      "\n",
      "F1-score is : 0.7382792302046749\n",
      "Cross validation score is : 85.61920551120268\n"
     ]
    }
   ],
   "source": [
    "# training a KNN classifier \n",
    "myList = list(range(1,100))\n",
    "neighbors = list(filter(lambda x: x%2!=0, myList))\n",
    "cv_scores = []\n",
    "for k in neighbors:\n",
    "    knn_model = KNeighborsClassifier(n_neighbors=k)\n",
    "    scores = cross_val_score(knn_model, X_train_scaled, y_train_ros, cv=10, scoring='f1_macro')\n",
    "    cv_scores.append(scores.mean())\n",
    "    \n",
    "mean_error = [1-x for x in cv_scores]\n",
    "\n",
    "plt.plot(neighbors, mean_error)\n",
    "plt.xlabel('Number of Neighbors k')\n",
    "plt.ylabel('Misclassification Error')\n",
    "plt.show()\n",
    "\n",
    "optimal_k = neighbors[mean_error.index(min(mean_error))]\n",
    "print('Optimal k is: ', optimal_k)\n",
    "\n",
    "knn_model = KNeighborsClassifier(n_neighbors = optimal_k, n_jobs=-1)\n",
    "knn_model.fit(X_train_scaled, y_train_ros)\n",
    "pred_knn = knn_model.predict(X_test_scaled)\n",
    "accuracy_knn = accuracy_score(y_test, pred_knn)*100\n",
    "print(\"KNN Accuracy: {}%\".format(accuracy_knn))\n",
    "\n",
    "target_names = ['class 0','class 1','class 2']\n",
    "print(classification_report(y_test, pred_knn, target_names=target_names))\n",
    "print()\n",
    "\n",
    "f1_knn = f1_score(y_test, pred_knn, average='macro')\n",
    "cross_val_knn = np.mean(cross_val_score(knn_model, X_train_scaled, y_train_ros, cv=10))*100\n",
    "print('F1-score is :', f1_knn)\n",
    "print('Cross validation score is :', cross_val_knn)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- LogisticRegression"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "LogisticRegression Accuracy: 92.78242677824268%\n",
      "              precision    recall  f1-score   support\n",
      "\n",
      "     class 0       0.90      0.94      0.92       246\n",
      "     class 1       0.88      0.91      0.90       231\n",
      "     class 2       0.97      0.93      0.95       479\n",
      "\n",
      "    accuracy                           0.93       956\n",
      "   macro avg       0.92      0.93      0.92       956\n",
      "weighted avg       0.93      0.93      0.93       956\n",
      "\n",
      "\n",
      "F1-score is : 0.9212562832032959\n",
      "Cross validation score is : 96.93729075457121\n"
     ]
    }
   ],
   "source": [
    "with ignore_warnings(category=ConvergenceWarning):\n",
    "\n",
    "    logreg = LogisticRegression()\n",
    "    logreg.fit(X_train_scaled, y_train_ros) \n",
    "    pred_logreg = logreg.predict(X_test_scaled) \n",
    "    accuracy_logreg = logreg.score(X_test_scaled, y_test) * 100\n",
    "    print(\"LogisticRegression Accuracy: {}%\".format(accuracy_logreg))\n",
    "\n",
    "    target_names = ['class 0','class 1','class 2']\n",
    "    print(classification_report(y_test, pred_logreg, target_names=target_names))\n",
    "    print()\n",
    "\n",
    "    f1_logreg = f1_score(y_test, pred_logreg, average='macro')\n",
    "    cross_val_logreg = np.mean(cross_val_score(logreg, X_train_scaled, y_train_ros, cv=10))*100\n",
    "    print('F1-score is :', f1_logreg)\n",
    "    print('Cross validation score is :', cross_val_logreg)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- MLPClassifier"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "MLP Accuracy: 92.67782426778243%\n",
      "              precision    recall  f1-score   support\n",
      "\n",
      "     class 0       0.90      0.94      0.92       246\n",
      "     class 1       0.89      0.94      0.91       231\n",
      "     class 2       0.97      0.91      0.94       479\n",
      "\n",
      "    accuracy                           0.93       956\n",
      "   macro avg       0.92      0.93      0.92       956\n",
      "weighted avg       0.93      0.93      0.93       956\n",
      "\n",
      "\n",
      "F1-score is : 0.9231627661546798\n",
      "Cross validation score is : 96.88055305176408\n"
     ]
    }
   ],
   "source": [
    "MLP_model = MLPClassifier(solver='adam', alpha=1e-5, hidden_layer_sizes=(15, 10, 5), random_state=1)\n",
    "MLP_model.fit(X_train_scaled, y_train_ros)\n",
    "\n",
    "pred_mlp = MLP_model.predict(X_test_scaled) \n",
    "\n",
    "accuracy_mlp = MLP_model.score(X_test_scaled, y_test) * 100\n",
    "print(\"MLP Accuracy: {}%\".format(accuracy_mlp))\n",
    "\n",
    "with ignore_warnings(category=ConvergenceWarning):\n",
    "    target_names = ['class 0','class 1','class 2']\n",
    "    print(classification_report(y_test, pred_mlp, target_names=target_names))\n",
    "    print()\n",
    "\n",
    "    f1_mlp = f1_score(y_test, pred_mlp, average='macro')\n",
    "    cross_val_mlp = np.mean(cross_val_score(MLP_model, X_train_scaled, y_train_ros, cv=10))*100\n",
    "    print('F1-score is :', f1_mlp)\n",
    "    print('Cross validation score is :', cross_val_mlp)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- SVMClassifier rbf_kernel"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "SVM Accuracy: 96.02510460251045%\n",
      "              precision    recall  f1-score   support\n",
      "\n",
      "     class 0       0.94      0.96      0.95       246\n",
      "     class 1       0.93      0.96      0.94       231\n",
      "     class 2       0.99      0.96      0.97       479\n",
      "\n",
      "    accuracy                           0.96       956\n",
      "   macro avg       0.95      0.96      0.96       956\n",
      "weighted avg       0.96      0.96      0.96       956\n",
      "\n",
      "\n",
      "F1-score is : 0.9556355914701241\n",
      "Cross validation score is : 98.09981006953386\n"
     ]
    }
   ],
   "source": [
    "svm_model = SVC(probability=True)\n",
    "\n",
    "svm_model.fit(X_train_scaled, y_train_ros)\n",
    "pred_svm = svm_model.predict(X_test_scaled)\n",
    "accuracy_svm = accuracy_score(y_test, pred_svm)*100\n",
    "print(\"SVM Accuracy: {}%\".format(accuracy_svm))\n",
    "\n",
    "target_names = ['class 0','class 1','class 2']\n",
    "print(classification_report(y_test, pred_svm, target_names=target_names))\n",
    "print()\n",
    "\n",
    "f1_svm = f1_score(y_test, pred_svm, average='macro')\n",
    "cross_val_svm = np.mean(cross_val_score(svm_model, X_train_scaled, y_train_ros, cv=10))*100\n",
    "print('F1-score is :', f1_svm)\n",
    "print('Cross validation score is :', cross_val_svm)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- SVMClassifier linear_kernel"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "SVM Linear Accuracy: 92.46861924686193%\n",
      "              precision    recall  f1-score   support\n",
      "\n",
      "     class 0       0.88      0.95      0.91       246\n",
      "     class 1       0.90      0.89      0.90       231\n",
      "     class 2       0.97      0.93      0.95       479\n",
      "\n",
      "    accuracy                           0.92       956\n",
      "   macro avg       0.91      0.92      0.92       956\n",
      "weighted avg       0.93      0.92      0.93       956\n",
      "\n",
      "\n",
      "F1-score is : 0.9175389781837805\n",
      "Cross validation score is : 96.45514099922741\n"
     ]
    }
   ],
   "source": [
    "#SVM Linear\n",
    "svm_model_linear = SVC(kernel = 'linear', C = 1).fit(X_train_scaled, y_train_ros) \n",
    "pred_svm_lin = svm_model_linear.predict(X_test_scaled)  \n",
    "accuracy_svm_lin = svm_model_linear.score(X_test_scaled, y_test) * 100\n",
    "print(\"SVM Linear Accuracy: {}%\".format(accuracy_svm_lin))\n",
    "\n",
    "target_names = ['class 0','class 1','class 2']\n",
    "print(classification_report(y_test, pred_svm_lin, target_names=target_names))\n",
    "print()\n",
    "\n",
    "f1_svm_linear = f1_score(y_test, pred_svm_lin, average='macro')\n",
    "cross_val_svm_linear = np.mean(cross_val_score(svm_model_linear, X_train_scaled, y_train_ros, cv=10))*100\n",
    "print('F1-score is :', f1_svm_linear)\n",
    "print('Cross validation score is :', cross_val_svm_linear)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- SVMClassifier polynomial_kernel"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "SVM Poly Accuracy: 89.01673640167364%\n",
      "              precision    recall  f1-score   support\n",
      "\n",
      "     class 0       0.99      0.70      0.82       246\n",
      "     class 1       0.93      0.89      0.91       231\n",
      "     class 2       0.84      0.99      0.91       479\n",
      "\n",
      "    accuracy                           0.89       956\n",
      "   macro avg       0.92      0.86      0.88       956\n",
      "weighted avg       0.90      0.89      0.89       956\n",
      "\n",
      "\n",
      "F1-score is : 0.8804161353447211\n",
      "Cross validation score is : 97.3929629152717\n"
     ]
    }
   ],
   "source": [
    "svm_model_poly = SVC(kernel='poly').fit(X_train_scaled, y_train_ros) \n",
    "\n",
    "pred_svm_poly = svm_model_poly.predict(X_test_scaled) \n",
    "  \n",
    "accuracy_svm_poly = svm_model_poly.score(X_test_scaled, y_test) * 100\n",
    "print(\"SVM Poly Accuracy: {}%\".format(accuracy_svm_poly))\n",
    "\n",
    "target_names = ['class 0','class 1','class 2']\n",
    "print(classification_report(y_test, pred_svm_poly, target_names=target_names))\n",
    "print()\n",
    "\n",
    "f1_svm_poly = f1_score(y_test, pred_svm_poly, average='macro')\n",
    "cross_val_svm_poly = np.mean(cross_val_score(svm_model_poly, X_train_scaled, y_train_ros, cv=10))*100\n",
    "print('F1-score is :', f1_svm_poly)\n",
    "print('Cross validation score is :', cross_val_svm_poly)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "| Classifier             | Accuracy %  | F1-score | Cross validation % |\n",
    "|------------------------|-------------|----------|------------------|\n",
    "| dummy                  | 25.7322     | 0.1364     | 33.1915          |\n",
    "| Gaussian Naive Bayes   | 94.0376     | 0.9372     | 95.0070          |\n",
    "| K Neirest Neighbors    | 73.9539     | 0.7382     | 85.6192          |\n",
    "| Logistic Regression    | 92.7824     | 0.9212    | 96.9372          |\n",
    "| Multi-Layer Perceptron | 92.6778     | 0.9231     | 96.8805          |\n",
    "| SVM (kernel = rbf)     |  96.0251     | 0.9556     | 98.0998          |\n",
    "| SVM (kernel = linear)  | 92.4686     |0.9175     | 96.4551          |\n",
    "| SVM (kernel = poly)    | 89.0167     | 0.8804     |  97.3929         |"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The scaling hasnt help any model improve. We observe a slight reduction in the scores, especially in the SVM poly kernel model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "GridsearchCV"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## __KNeighborsClassifier()__"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.model_selection import GridSearchCV\n",
    "from imblearn.pipeline import Pipeline\n",
    "from imblearn.pipeline import Pipeline\n",
    "from sklearn import neighbors\n",
    "from sklearn.feature_selection import VarianceThreshold\n",
    "from sklearn.preprocessing import StandardScaler \n",
    "from sklearn.decomposition import PCA\n",
    "import time\n",
    "n_components = [1, 10, 20, 40, 100, 180]\n",
    "selector = VarianceThreshold()\n",
    "scaler = StandardScaler()\n",
    "ros = RandomOverSampler()\n",
    "pca = PCA()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Total time fit and predict: 1947.5439858436584 seconds\n",
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       0.88      0.86      0.87       246\n",
      "           1       0.81      0.92      0.86       231\n",
      "           2       0.93      0.88      0.90       479\n",
      "\n",
      "    accuracy                           0.88       956\n",
      "   macro avg       0.87      0.89      0.88       956\n",
      "weighted avg       0.89      0.88      0.89       956\n",
      "\n",
      "0.8936072447751158\n",
      "{'kNN__metric': 'minkowski', 'kNN__n_neighbors': 49, 'kNN__weights': 'distance', 'pca__n_components': 10}\n",
      "dict_keys(['cv', 'error_score', 'estimator__memory', 'estimator__steps', 'estimator__verbose', 'estimator__scaler', 'estimator__sampler', 'estimator__pca', 'estimator__kNN', 'estimator__scaler__copy', 'estimator__scaler__with_mean', 'estimator__scaler__with_std', 'estimator__sampler__random_state', 'estimator__sampler__sampling_strategy', 'estimator__sampler__shrinkage', 'estimator__pca__copy', 'estimator__pca__iterated_power', 'estimator__pca__n_components', 'estimator__pca__random_state', 'estimator__pca__svd_solver', 'estimator__pca__tol', 'estimator__pca__whiten', 'estimator__kNN__algorithm', 'estimator__kNN__leaf_size', 'estimator__kNN__metric', 'estimator__kNN__metric_params', 'estimator__kNN__n_jobs', 'estimator__kNN__n_neighbors', 'estimator__kNN__p', 'estimator__kNN__weights', 'estimator', 'n_jobs', 'param_grid', 'pre_dispatch', 'refit', 'return_train_score', 'scoring', 'verbose'])\n"
     ]
    }
   ],
   "source": [
    "n_components = [1, 10, 20, 40, 100, 180]\n",
    "k = list(range(1, 180))\n",
    "clf = neighbors.KNeighborsClassifier(n_jobs=-1) \n",
    "metrics = ['euclidean', 'minkowski', 'manhattan']\n",
    "weight_options = ['uniform', 'distance']\n",
    "param_grid = dict(pca__n_components=n_components, kNN__n_neighbors=k, kNN__metric = metrics, kNN__weights = weight_options)\n",
    "pipe = Pipeline(steps=[('scaler', scaler), ('sampler', ros), ('pca', pca), ('kNN', clf) ])\n",
    "estimator = GridSearchCV(pipe, param_grid , cv=10, scoring='f1_macro', n_jobs=-1) \n",
    "start_time = time.time()\n",
    "estimator.fit(X_train, y_train)\n",
    "preds = estimator.predict(X_test)\n",
    "print(\"Total time fit and predict: %s seconds\" % (time.time() - start_time))\n",
    "print(classification_report(y_test, preds))\n",
    "#print(estimator.cv_results_)\n",
    "print(estimator.best_score_)\n",
    "print(estimator.best_params_)\n",
    "\n",
    "print(estimator.get_params().keys())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## __LogisticRegression()__"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Total time fit and predict: 199.4397783279419 seconds\n",
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       0.90      0.94      0.92       246\n",
      "           1       0.88      0.90      0.89       231\n",
      "           2       0.96      0.93      0.95       479\n",
      "\n",
      "    accuracy                           0.93       956\n",
      "   macro avg       0.92      0.92      0.92       956\n",
      "weighted avg       0.93      0.93      0.93       956\n",
      "\n",
      "0.9695277845283045\n",
      "{'LogisticRegression__C': 1, 'LogisticRegression__penalty': 'l2', 'LogisticRegression__solver': 'sag', 'pca__n_components': 180}\n"
     ]
    }
   ],
   "source": [
    "with ignore_warnings(category=ConvergenceWarning):\n",
    "    solvers = [\"newton-cg\", \"sag\", \"saga\", \"lbfgs\"]\n",
    "    penalty = ['l2']\n",
    "    C = [0.001,.009,0.01,.09,1,5,10,25]\n",
    "    #param_grid = dict(pca__n_components = n_components, LogisticRegression__penalty = penalty, LogisticRegression__solver = solvers, LogisticRegression__C = C)\n",
    "    param_grid = dict(pca__n_components=n_components, LogisticRegression__penalty = penalty, LogisticRegression__solver = solvers, LogisticRegression__C = C)\n",
    "    clf = LogisticRegression(n_jobs=-1)\n",
    "    #pipe = Pipeline(steps=[('scaler', scaler), ('sampler', ros), ('pca', pca), ('LogisticRegression', clf) ], memory = 'tmp')\n",
    "    pipe = Pipeline([('scaler', scaler), ('pca', pca), ('LogisticRegression' , clf)])\n",
    "    estimator = GridSearchCV(pipe, param_grid, cv=10, scoring='f1_macro', n_jobs=-1)\n",
    "    start_time = time.time()\n",
    "    estimator.fit(X_train_ros, y_train_ros)\n",
    "    preds = estimator.predict(X_test)\n",
    "    print(\"Total time fit and predict: %s seconds\" % (time.time() - start_time))\n",
    "    print(classification_report(y_test, preds))\n",
    "    print(estimator.best_score_)\n",
    "    print(estimator.best_params_)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## __GaussianNB()__"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Total time fit and predict: 0.6963467597961426 seconds\n",
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       0.91      0.97      0.94       246\n",
      "           1       0.90      0.95      0.92       231\n",
      "           2       0.98      0.92      0.95       479\n",
      "\n",
      "    accuracy                           0.94       956\n",
      "   macro avg       0.93      0.95      0.94       956\n",
      "weighted avg       0.94      0.94      0.94       956\n",
      "\n",
      "0.9517893759152202\n",
      "{'GaussianNB__var_smoothing': 1e-07}\n"
     ]
    }
   ],
   "source": [
    "smooth = [1e-20, 1e-14, 1e-13, 1e-08, 1e-07]\n",
    "param_grid = dict(GaussianNB__var_smoothing= smooth)\n",
    "clf = GaussianNB()\n",
    "pipe = Pipeline(steps=[('scaler', scaler), ('GaussianNB', clf) ])\n",
    "estimator = GridSearchCV(pipe, param_grid, cv=10, scoring='f1_macro', n_jobs=-1)\n",
    "start_time = time.time()\n",
    "estimator.fit(X_train_ros, y_train_ros)\n",
    "preds = estimator.predict(X_test)\n",
    "print(\"Total time fit and predict: %s seconds\" % (time.time() - start_time))\n",
    "print(classification_report(y_test, preds))\n",
    "print(estimator.best_score_)\n",
    "print(estimator.best_params_)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## __Multi-Layer Perceptron__()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 71,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Total time fit and predict: 1328.870459318161 seconds\n",
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       0.95      0.98      0.96       246\n",
      "           1       0.92      0.95      0.94       231\n",
      "           2       0.98      0.95      0.97       479\n",
      "\n",
      "    accuracy                           0.96       956\n",
      "   macro avg       0.95      0.96      0.96       956\n",
      "weighted avg       0.96      0.96      0.96       956\n",
      "\n",
      "0.9792031777766604\n",
      "{'MLP__alpha': 0.005, 'MLP__hidden_layer_sizes': (100,), 'MLP__learning_rate': 'adaptive', 'MLP__solver': 'adam', 'pca__n_components': 180}\n"
     ]
    }
   ],
   "source": [
    "solver = ['lbfgs', 'sgd', 'adam']\n",
    "hidden_layers = [(10,), (20,), (100,)]\n",
    "learning_rate = ['constant', 'adaptive']\n",
    "alpha = [0.0001, 0.005, 0.05]\n",
    "param_grid = dict(pca__n_components=n_components, MLP__learning_rate = learning_rate, MLP__hidden_layer_sizes = hidden_layers, MLP__solver = solver, MLP__alpha = alpha)\n",
    "clf = MLPClassifier()\n",
    "pipe = Pipeline(steps=[('pca', pca), ('MLP', clf) ], memory = 'tmp')\n",
    "estimator = GridSearchCV(pipe, param_grid, cv=10, scoring='f1_macro', n_jobs=-1)\n",
    "start_time = time.time()\n",
    "estimator.fit(X_train_ros, y_train_ros)\n",
    "preds = estimator.predict(X_test)\n",
    "print(\"Total time fit and predict: %s seconds\" % (time.time() - start_time))\n",
    "print(classification_report(y_test, preds))\n",
    "print(estimator.best_score_)\n",
    "print(estimator.best_params_)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## __SVMClassifier()__"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 73,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Total time fit and predict: 401.87223267555237 seconds\n",
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       0.96      0.84      0.89       246\n",
      "           1       0.93      0.92      0.93       231\n",
      "           2       0.91      0.98      0.94       479\n",
      "\n",
      "    accuracy                           0.93       956\n",
      "   macro avg       0.93      0.91      0.92       956\n",
      "weighted avg       0.93      0.93      0.93       956\n",
      "\n",
      "0.9813335153232622\n",
      "{'SVC__C': 100, 'SVC__gamma': 0.1, 'SVC__kernel': 'rbf', 'pca__n_components': 100}\n"
     ]
    }
   ],
   "source": [
    "C = [0.1, 1, 10, 100, 1000]\n",
    "gamma = [1, 0.1, 0.01, 0.001, 0.0001]\n",
    "#kernel = ['rbf', 'linear', 'poly', 'sigmoid', 'precomputed']  \n",
    "kernel = ['rbf']\n",
    "param_grid = dict( pca__n_components=n_components, SVC__kernel = kernel, SVC__C = C, SVC__gamma = gamma)\n",
    "clf = SVC()\n",
    "pipe = Pipeline(steps=[('pca', pca), ('SVC', clf) ])\n",
    "estimator = GridSearchCV(pipe, param_grid, cv=10, scoring='f1_macro', n_jobs=-1)\n",
    "start_time = time.time()\n",
    "estimator.fit(X_train_ros, y_train_ros)\n",
    "preds = estimator.predict(X_test)\n",
    "print(\"Total time fit and predict: %s seconds\" % (time.time() - start_time))\n",
    "print(classification_report(y_test, preds))\n",
    "print(estimator.best_score_)\n",
    "print(estimator.best_params_)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "SVM with other kernels than rbf never ended and timed out the kernel "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "| Classifier             | Accuracy   | F1-score | Improvement % | Time (sec)|\n",
    "|------------------------|------------|----------|---------------|-----------|\n",
    "| Gaussian Naive Bayes   | 0.9517893759152202     | 0.94     | 1.00877 |  0.6963 |     \n",
    "| K Neirest Neighbors    | 0.8936072447751158     | 0.88     | 1.03928 |  1947.5439|     \n",
    "| Logistic Regression    | 0.9695277845283045     | 0.93     | 1.02416 | 199.4397|       \n",
    "| Multi-Layer Perceptron | 0.9792031777766604     | 0.96     | 1.02644 | 1328.8704|      \n",
    "| SVM (kernel = rbf)     | 0.9813335153232622     | 0.93     | 1.0208 | 401.8722|          "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We observe that the best score is achieved in the SVM classifier. Grid search has helped a lot the K-NN classifier to improve, and as he has the biggest increment of all after the grid search. We we definately choose SVM with rbf kernel as he has the best score which is quite high, even in the out of the box performance."
   ]
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "celltoolbar": "Slideshow",
  "colab": {
   "name": "OpenML Python API.ipynb",
   "provenance": []
  },
  "colabVersion": "0.1",
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}