20309780_Mohan_Sameer_ex2.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/mohansameer1983/DeepLearning/blob/main/20309780_Mohan_Sameer_ex2.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "WwG5uxEpKaNL"
      },
      "source": [
        "# MMAI 894 - Exercise 2\n",
        "## Convolutional artificial neural network : Image classification\n",
        "The goal of this excercise is to build a convolutional neural network using the tensorflow/keras library. We will be using the MNIST dataset.\n",
        "Submission instructions:\n",
        "\n",
        "- You cannot edit this notebook directly. Save a copy to your drive, and make sure to identify yourself in the title using name and student number\n",
        "- Do not insert new cells before the final one (titled \"Further exploration\") \n",
        "- Verify that your notebook can _restart and run all_. \n",
        "- Select File -> Download as .py (important! not as ipynb)\n",
        "- Rename the file: `studentID_lastname_firstname_ex2.py`\n",
        "- The mark will be assessed on the implementation of the functions with #TODO\n",
        "- **Do not change anything outside the functions**  unless in the further exploration section\n",
        "- As you are encouraged to explore the network configuration, 20% of the mark is based on final accuracy. \n",
        "- Note: You do not have to answer the questions in thie notebook as part of your submission. They are meant to guide you.\n",
        "\n",
        "- You should not need to use any additional libraries other than the ones listed below. You may want to import additional modules from those libraries, however."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 8,
      "metadata": {
        "id": "o-DUt8ROKlvw"
      },
      "outputs": [],
      "source": [
        "# Import modules\n",
        "# Add modules as needed\n",
        "from sklearn.datasets import fetch_openml\n",
        "import numpy as np\n",
        "\n",
        "# For windows laptops add following 2 lines:\n",
        "# import matplotlib\n",
        "# matplotlib.use('agg')\n",
        "\n",
        "import matplotlib.pyplot as plt\n",
        "from sklearn.model_selection import train_test_split\n",
        "\n",
        "from sklearn.preprocessing import LabelEncoder\n",
        "from numpy import argmax\n",
        "import tensorflow.keras as keras\n",
        "from tensorflow.keras.utils import to_categorical\n",
        "import tensorflow.keras as keras\n",
        "from tensorflow.keras.layers import Conv2D\n",
        "from tensorflow.keras.layers import MaxPooling2D\n",
        "from tensorflow.keras.layers import Flatten\n",
        "from tensorflow.keras.optimizers import SGD,Adam\n",
        "from tensorflow.keras.models import Sequential\n",
        "from tensorflow.keras.layers import Dense, Dropout"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "arrbzOo4LR9q"
      },
      "source": [
        "### Data preparation\n",
        "\n",
        "#### Import data"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 9,
      "metadata": {
        "id": "s4wT7dbPLTNZ"
      },
      "outputs": [],
      "source": [
        "def load_data():\n",
        "    # Import MNIST dataset from openml\n",
        "    dataset = fetch_openml('mnist_784', version=1, data_home=None)\n",
        "\n",
        "    # Data preparation\n",
        "    raw_X = dataset['data']\n",
        "    raw_Y = dataset['target']\n",
        "    return raw_X, raw_Y\n",
        "\n",
        "raw_X, raw_Y = load_data()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "WRp1jQTrLc39"
      },
      "source": [
        "## Consider the following\n",
        "- Same as excercise 1\n",
        "- what shape should x be for a convolutional network?"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 10,
      "metadata": {
        "id": "8mG0hdDRLZgD"
      },
      "outputs": [],
      "source": [
        "def clean_data(raw_X, raw_Y):\n",
        "    # TODO: clean and QA raw_X and raw_Y\n",
        "    # DO NOT CHANGE THE INPUTS OR OUTPUTS TO THIS FUNCTION\n",
        "\n",
        "    #Normalize pixel values\n",
        "    norm_X = raw_X.values / 255\n",
        "\n",
        "    cleaned_X = norm_X.reshape(-1,28,28,1)\n",
        "    cleaned_Y = to_categorical(raw_Y)\n",
        "    \n",
        "    return cleaned_X, cleaned_Y\n",
        "\n",
        "cleaned_X, cleaned_Y = clean_data(raw_X, raw_Y)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "K5OYgzj7NU_y"
      },
      "source": [
        "#### Data split\n",
        "\n",
        "- Split your data into a train set (50%), validation set (20%) and a test set (30%). You can use scikit-learn's train_test_split function."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 11,
      "metadata": {
        "id": "7h7HIVDNNYap"
      },
      "outputs": [],
      "source": [
        "def split_data(cleaned_X, cleaned_Y):\n",
        "    # TODO: split the data\n",
        "    # DO NOT CHANGE THE INPUTS OR OUTPUTS TO THIS FUNCTION\n",
        "    X_train, X_test, Y_train, Y_test = train_test_split(cleaned_X, cleaned_Y, test_size=0.3, random_state=1)\n",
        "\n",
        "    X_train, X_val, Y_train, Y_val = train_test_split(X_train, Y_train, test_size=0.2, random_state=1)\n",
        "    \n",
        "    return X_val.reshape(-1,28,28,1), X_test.reshape(-1,28,28,1), X_train.reshape(-1,28,28,1), Y_val, Y_test, Y_train\n",
        "\n",
        "X_val, X_test, X_train, Y_val, Y_test, Y_train = split_data(cleaned_X, cleaned_Y)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "mY_z08TdNada"
      },
      "source": [
        "### Model\n",
        "\n",
        "#### Neural network structure\n",
        "\n",
        "This time, the exact model architecture is left to you to explore.  \n",
        "Keep the number of parameters below 2,000,000"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 12,
      "metadata": {
        "id": "dKAx26EDN3Yk"
      },
      "outputs": [],
      "source": [
        "from tensorflow.python.keras.backend import dropout\n",
        "def build_model():\n",
        "    # TODO: build the model, \n",
        "    # DO NOT CHANGE THE INPUTS OR OUTPUTS TO THIS FUNCTION\n",
        "    \n",
        "    model = Sequential()\n",
        "    model.add(Conv2D(128, (3, 3), activation='relu', kernel_initializer='he_uniform', input_shape=(28, 28, 1)))\n",
        "    model.add(Dropout(0.01))\n",
        "    model.add(MaxPooling2D((2, 2)))\n",
        "    model.add(Conv2D(64, (1,1), activation='relu')) \n",
        "    model.add(MaxPooling2D((2, 2)))\n",
        "    model.add(Flatten())\n",
        "    model.add(Dense(100, activation='relu', kernel_initializer='he_uniform'))\n",
        "    model.add(Dense(10, activation='softmax'))\n",
        "    \n",
        "    return model\n",
        "\n",
        "def compile_model(model):\n",
        "    # TODO: compile the model\n",
        "    # DO NOT CHANGE THE INPUTS OR OUTPUTS TO THIS FUNCTION\n",
        "    opt = SGD(learning_rate=0.01, momentum=0.8)\n",
        "    model.compile(optimizer=opt, loss='categorical_crossentropy', metrics=['accuracy'])\n",
        "   \n",
        "    return model\n",
        "\n",
        "def train_model(model, X_train, Y_train, X_val, Y_val):\n",
        "    # TODO: train the model\n",
        "    # DO NOT CHANGE THE INPUTS OR OUTPUTS TO THIS FUNCTION\n",
        "\n",
        "    history = model.fit(X_train, Y_train, epochs=15, batch_size=32, validation_data=(X_val, Y_val), verbose=2)\n",
        "\n",
        "    return model, history\n",
        "\n",
        "\n",
        "def eval_model(model, X_test, Y_test):\n",
        "    # TODO: evaluate the model\n",
        "    # DO NOT CHANGE THE INPUTS OR OUTPUTS TO THIS FUNCTION\n",
        "    test_loss, test_accuracy = model.evaluate(X_test, Y_test, verbose=2)\n",
        "\n",
        "    return test_loss, test_accuracy\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 13,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Bg5E9ChPOt_l",
        "outputId": "80f93a54-f2f7-49de-a3b0-4b086edaa44e"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 1/15\n",
            "1225/1225 - 89s - loss: 0.2214 - accuracy: 0.9323 - val_loss: 0.1058 - val_accuracy: 0.9678 - 89s/epoch - 72ms/step\n",
            "Epoch 2/15\n",
            "1225/1225 - 85s - loss: 0.0679 - accuracy: 0.9792 - val_loss: 0.0721 - val_accuracy: 0.9771 - 85s/epoch - 69ms/step\n",
            "Epoch 3/15\n",
            "1225/1225 - 84s - loss: 0.0493 - accuracy: 0.9848 - val_loss: 0.0780 - val_accuracy: 0.9763 - 84s/epoch - 69ms/step\n",
            "Epoch 4/15\n",
            "1225/1225 - 84s - loss: 0.0379 - accuracy: 0.9881 - val_loss: 0.0528 - val_accuracy: 0.9823 - 84s/epoch - 69ms/step\n",
            "Epoch 5/15\n",
            "1225/1225 - 84s - loss: 0.0293 - accuracy: 0.9909 - val_loss: 0.0501 - val_accuracy: 0.9837 - 84s/epoch - 68ms/step\n",
            "Epoch 6/15\n",
            "1225/1225 - 83s - loss: 0.0228 - accuracy: 0.9927 - val_loss: 0.0470 - val_accuracy: 0.9859 - 83s/epoch - 68ms/step\n",
            "Epoch 7/15\n",
            "1225/1225 - 84s - loss: 0.0192 - accuracy: 0.9942 - val_loss: 0.0552 - val_accuracy: 0.9839 - 84s/epoch - 68ms/step\n",
            "Epoch 8/15\n",
            "1225/1225 - 84s - loss: 0.0153 - accuracy: 0.9956 - val_loss: 0.0480 - val_accuracy: 0.9856 - 84s/epoch - 68ms/step\n",
            "Epoch 9/15\n",
            "1225/1225 - 84s - loss: 0.0120 - accuracy: 0.9965 - val_loss: 0.0499 - val_accuracy: 0.9856 - 84s/epoch - 68ms/step\n",
            "Epoch 10/15\n",
            "1225/1225 - 84s - loss: 0.0098 - accuracy: 0.9972 - val_loss: 0.0455 - val_accuracy: 0.9853 - 84s/epoch - 69ms/step\n",
            "Epoch 11/15\n",
            "1225/1225 - 84s - loss: 0.0075 - accuracy: 0.9981 - val_loss: 0.0485 - val_accuracy: 0.9861 - 84s/epoch - 68ms/step\n",
            "Epoch 12/15\n",
            "1225/1225 - 84s - loss: 0.0060 - accuracy: 0.9984 - val_loss: 0.0484 - val_accuracy: 0.9860 - 84s/epoch - 68ms/step\n",
            "Epoch 13/15\n",
            "1225/1225 - 84s - loss: 0.0040 - accuracy: 0.9991 - val_loss: 0.0463 - val_accuracy: 0.9868 - 84s/epoch - 68ms/step\n",
            "Epoch 14/15\n",
            "1225/1225 - 84s - loss: 0.0033 - accuracy: 0.9994 - val_loss: 0.0486 - val_accuracy: 0.9863 - 84s/epoch - 69ms/step\n",
            "Epoch 15/15\n",
            "1225/1225 - 84s - loss: 0.0021 - accuracy: 0.9998 - val_loss: 0.0513 - val_accuracy: 0.9863 - 84s/epoch - 68ms/step\n",
            "657/657 - 7s - loss: 0.0500 - accuracy: 0.9878 - 7s/epoch - 11ms/step\n"
          ]
        }
      ],
      "source": [
        "## You may use this space (and add additional cells for exploration)\n",
        "\n",
        "model = build_model()\n",
        "model = compile_model(model)\n",
        "model, history = train_model(model, X_train, Y_train, X_val, Y_val)\n",
        "test_loss, test_accuracy = eval_model(model, X_test, Y_test)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 14,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "B1pEjMtHebHR",
        "outputId": "6b82fca0-6b24-461a-9619-b1e0874d5169"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Test Accuracy > 98.776\n",
            "Test Loss > 0.050\n",
            "Model: \"sequential_1\"\n",
            "_________________________________________________________________\n",
            " Layer (type)                Output Shape              Param #   \n",
            "=================================================================\n",
            " conv2d_3 (Conv2D)           (None, 26, 26, 128)       1280      \n",
            "                                                                 \n",
            " dropout_1 (Dropout)         (None, 26, 26, 128)       0         \n",
            "                                                                 \n",
            " max_pooling2d_2 (MaxPooling  (None, 13, 13, 128)      0         \n",
            " 2D)                                                             \n",
            "                                                                 \n",
            " conv2d_4 (Conv2D)           (None, 13, 13, 64)        8256      \n",
            "                                                                 \n",
            " max_pooling2d_3 (MaxPooling  (None, 6, 6, 64)         0         \n",
            " 2D)                                                             \n",
            "                                                                 \n",
            " flatten_1 (Flatten)         (None, 2304)              0         \n",
            "                                                                 \n",
            " dense_2 (Dense)             (None, 100)               230500    \n",
            "                                                                 \n",
            " dense_3 (Dense)             (None, 10)                1010      \n",
            "                                                                 \n",
            "=================================================================\n",
            "Total params: 241,046\n",
            "Trainable params: 241,046\n",
            "Non-trainable params: 0\n",
            "_________________________________________________________________\n",
            "None\n"
          ]
        }
      ],
      "source": [
        "print('Test Accuracy > %.3f' % (test_accuracy * 100.0))\n",
        "\n",
        "print('Test Loss > %.3f' % (test_loss))\n",
        "print(model.summary())"
      ]
    }
  ],
  "metadata": {
    "colab": {
      "collapsed_sections": [],
      "name": "20309780_Mohan_Sameer_ex2.ipynb",
      "provenance": [],
      "include_colab_link": true
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}