Working_With_Text_Data.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "positive-windows",
   "metadata": {},
   "source": [
    "<h1>Practice</h1>\n",
    "\n",
    "<a href=\"https://scikit-learn.org/0.19/tutorial/text_analytics/working_with_text_data.html\">Link</a>\n",
    "\n",
    "<h2>Scikit-learn tutorial on Working With Text Data</h2>\n",
    "\n",
    "\n",
    "The goal of this guide is to explore some of the main scikit-learn tools on a single practical task: analysing a collection of text documents (newsgroups posts) on twenty different topics.\n",
    "\n",
    "In this section we will see how to:\n",
    "\n",
    "* load the file contents and the categories\n",
    "* extract feature vectors suitable for machine learning\n",
    "* train a linear model to perform categorization\n",
    "* use a grid search strategy to find a good configuration of both the feature extraction components and the classifier"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "wound-uganda",
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.datasets import fetch_20newsgroups"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "overhead-cornwall",
   "metadata": {},
   "outputs": [],
   "source": [
    "categories = ['alt.atheism', 'soc.religion.christian', 'comp.graphics', 'sci.med']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "herbal-indicator",
   "metadata": {},
   "outputs": [],
   "source": [
    "twenty_train = fetch_20newsgroups(subset='train', categories=categories, shuffle=True, random_state=42)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "valuable-november",
   "metadata": {},
   "source": [
    "The returned dataset is a <mark>scikit-learn</mark> “bunch”: a simple holder object with fields that can be both accessed as python <mark>dict</mark> keys or <mark>object</mark> attributes for convenience, for instance the <mark>target_names</mark> holds the list of the requested category names:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "still-cooling",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['alt.atheism', 'comp.graphics', 'sci.med', 'soc.religion.christian']"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "twenty_train.target_names"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "rapid-minister",
   "metadata": {},
   "source": [
    "The files themselves are loaded in memory in the `data` attribute. For refernce the filenames are also available:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "liable-cause",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(2257, 2257)"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(twenty_train.data), len(twenty_train.filenames)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "caroline-somalia",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "From: sd345@city.ac.uk (Michael Collier)\n",
      "Subject: Converting images to HP LaserJet III?\n",
      "Nntp-Posting-Host: hampton\n"
     ]
    }
   ],
   "source": [
    "#Print the first lines of the first loaded file:\n",
    "print(\"\\n\".join(twenty_train.data[0].split(\"\\n\")[:3]))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "premier-allen",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "comp.graphics\n"
     ]
    }
   ],
   "source": [
    "print(twenty_train.target_names[twenty_train.target[0]])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ahead-object",
   "metadata": {},
   "source": [
    "Supervised learning algorithms will require a category label for each document in the training set.\n",
    "In this case the category is the name of the newsgroup which \n",
    "also happens to be the name of the folder holding the individual documents.\n",
    "<br>\n",
    "For speed and space efficiency reasons `scikit-learn` loads the target attribute as an array of integers that corresonds to the index of the category name in the <mark>traget_names</mark>list. The category integer id of each sample is stored in the `target` attribute:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "regulation-franklin",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([1, 1, 3, 3, 3, 3, 3, 2, 2, 2], dtype=int64)"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "twenty_train.target[:10]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "viral-society",
   "metadata": {},
   "source": [
    "It is possible to get back the category names as follows:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "powered-sentence",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "comp.graphics\n",
      "comp.graphics\n",
      "soc.religion.christian\n",
      "soc.religion.christian\n",
      "soc.religion.christian\n",
      "soc.religion.christian\n",
      "soc.religion.christian\n",
      "sci.med\n",
      "sci.med\n",
      "sci.med\n"
     ]
    }
   ],
   "source": [
    "for t in twenty_train.target[:10]:\n",
    "    print (twenty_train.target_names[t])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "appointed-mortgage",
   "metadata": {},
   "source": [
    "<h2>Extracting features from text files</h2><br>\n",
    "\n",
    "In order to perform machine learining on text documents, we first need to turn the text into numerical feature vectors.<br>\n",
    "\n",
    "<mark><h3>Bags of words</h3></mark>\n",
    "\n",
    "The most intuitive way to do so is the bags of words representation:\n",
    "\n",
    "<ol type=\"1\">\n",
    "    <li>assign a fixed id to each word occurring in any document  of the training set (for instance by building a dictionary from words to integer indices)</li>\n",
    "    <li>for each document <mark>#i</mark>, count the number of occurrences of each word 'w' and store it in <mark>X[i, j]</mark> as the value of feature '#j' where <mark>j</mark> is the index of word 'w' in the dictionary.</li>\n",
    "</ol>\n",
    "\n",
    "The bags of words representation implies that 'n_features' is the number of distinct words in the corpus: this is typically larger than 100,000.\n",
    "\n",
    "If 'n_samples == 10000, storing 'x' as a numpy array of type float32 would require 10000 x 100000 x 4 bytes = 4GB in RAM which is barely manageable on today's computers.\n",
    "<br>\n",
    "\n",
    "Fortunately, <strong>most values in X will be zeros</strong> since for a given document less than a couple thousands of distinct words will be used. For this reason we say that bags of words are typically <strong>high-dimensional sparse datasets</strong>. We can save a lot of memory by only storing the non-zero parts of the feature vectors in memory.\n",
    "<br>\n",
    "<mark>scipy.sparse</mark> matrices are data structures that do exactly this, and 'scikit-learn' has built-in support for these structures.\n",
    "\n",
    "    "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "illegal-consent",
   "metadata": {},
   "source": [
    "<h2> Tokenizing text with  scikit-learn </h2>\n",
    "<br>\n",
    "Text preporcessing, tokenizing and filtering of stopwords are included in a high level component that is able to build a dictionary of features and transform documents to features vectors."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "comfortable-ebony",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(2257, 35788)"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.feature_extraction.text import CountVectorizer\n",
    "count_vect = CountVectorizer()\n",
    "x_train_counts = count_vect.fit_transform(twenty_train.data)\n",
    "x_train_counts.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "thrown-monroe",
   "metadata": {},
   "source": [
    "<strong>CountVectorizer</strong> supports counts of N-grams of words or consecutive characters. Once fitted, the vectorizer has built a dictionary of feature indices:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "round-priority",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "4690"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "count_vect.vocabulary_.get(u'algorithm')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "available-platform",
   "metadata": {},
   "source": [
    "The index value of a word in the vocabulary is linked to its frequency in the whole training corpus."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "higher-murray",
   "metadata": {},
   "source": [
    "<h2>From occurrences to frequencies</h2>\n",
    "Occurrence count is a good start but there is an issue: longer documents will have higher average count values than shorter doucments, even though they might talk about the same topics.\n",
    "\n",
    "More descripition on `tf` \"Terms Frequencies\" and `tf-idf` \"Term Frequency times inverse Document Frequency\" back in  <a href=\"https://scikit-learn.org/0.19/tutorial/text_analytics/working_with_text_data.html\">Link</a> <br>\n",
    "Both `tf` and `tf-idf` can be computed as follows:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "bright-quarterly",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(2257, 35788)"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.feature_extraction.text import TfidfTransformer\n",
    "tf_transformer = TfidfTransformer(use_idf=False).fit(x_train_counts)\n",
    "x_train_tf = tf_transformer.transform(x_train_counts)\n",
    "x_train_tf.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "geological-saturday",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(2257, 35788)"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tfidf_transformer = TfidfTransformer()\n",
    "x_train_tfidf = tfidf_transformer.fit_transform(x_train_counts)\n",
    "x_train_tfidf.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "utility-relevance",
   "metadata": {},
   "source": [
    "<h2>Training a  classifier</h2>\n",
    "\n",
    "Now that we have our features, we can train a classifier to try to predict the category of a post. Let's start with a <a href=\"https://scikit-learn.org/0.19/modules/naive_bayes.html#naive-bayes\">Naive Byase Classifier</a>, which provides a nice baseline fo this task. <mark>scikit-learn</mark> includes several variants of this calssifier; the one most suitable for word coutns is the multinomial variant:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "representative-details",
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.naive_bayes import MultinomialNB\n",
    "\n",
    "clf = MultinomialNB().fit(x_train_tfidf, twenty_train.target)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "parliamentary-theta",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "'God is love' => soc.religion.christian\n",
      "'OpenGL on the GPU is fast' => comp.graphics\n"
     ]
    }
   ],
   "source": [
    "docs_new = ['God is love', 'OpenGL on the GPU is fast']\n",
    "x_new_counts = count_vect.transform(docs_new)\n",
    "x_new_tfidf = tfidf_transformer.transform(x_new_counts)\n",
    "\n",
    "predicted = clf.predict(x_new_tfidf)\n",
    "\n",
    "for doc, category in zip( docs_new, predicted):\n",
    "    print('%r => %s' % (doc, twenty_train.target_names[category]))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "naval-machine",
   "metadata": {},
   "source": [
    "<h2>Building a pipeline</h2>\n",
    "In order to make the vectorizer => transformer => classifier easier to work with, <mark>scikit-learn</mark> provides a 'Pipeline' class that behaves like a compound classifier"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "advised-circle",
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.pipeline import Pipeline\n",
    "text_clf = Pipeline([('vect', CountVectorizer()),\n",
    "                    ('tfidf', TfidfTransformer()),\n",
    "                    ('clf', MultinomialNB()),\n",
    "                    ])\n",
    "# The names 'vect', 'tfidf' and clf (classifier) are arbitrary. We shall see their use in the section on grid search, below.\n",
    "# We can now train the model with a single command:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "requested-clerk",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Pipeline(steps=[('vect', CountVectorizer()), ('tfidf', TfidfTransformer()),\n",
       "                ('clf', MultinomialNB())])"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "text_clf.fit(twenty_train.data, twenty_train.target)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "accomplished-pottery",
   "metadata": {},
   "source": [
    "<h2> Evaulation of the performance on the test set</h2>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "virtual-criminal",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.8348868175765646"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import numpy as np\n",
    "twenty_test = fetch_20newsgroups(subset='test', categories=categories, shuffle=True, random_state=42)\n",
    "docs_test = twenty_test.data\n",
    "predicted = text_clf.predict(docs_test)\n",
    "np.mean(predicted == twenty_test.target)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "surrounded-explosion",
   "metadata": {},
   "source": [
    "We achived 83.4% accuracy. Let's see if we can do better with a linear <a href=\"https://scikit-learn.org/0.19/modules/svm.html#svm\">support vector machine(SVM)</a>, which is widely regarded as one of the best text classification algorithms (although it's also ba bit slower than naive Bayes). We can change the learner by just plugging a different classifier object into our pipeline:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "racial-coast",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Pipeline with Niave Bayes classier (OLD) copied from the above:\n",
    "# from sklearn.pipeline import Pipeline\n",
    "# from sklearn.naive_bayes import MultinomialNB\n",
    "# text_clf = Pipeline([('vect', CountVectorizer()),\n",
    "#                     ('tfidf', TfidfTransformer()),\n",
    "#                     ('clf', MultinomialNB()),\n",
    "#                     ])\n",
    "\n",
    "# Pipeline with Support Vector Machine Classifier:\n",
    "# from sklearn.pipeline import Pipeline\n",
    "from sklearn.linear_model import SGDClassifier\n",
    "text_clf = Pipeline([('vect', CountVectorizer()),\n",
    "                    ('tfidf', TfidfTransformer()),\n",
    "                    ('clf', SGDClassifier(loss='hinge', penalty='l2',\n",
    "                                          alpha=1e-3, random_state=42,\n",
    "                                          max_iter=5, tol=None)),\n",
    "                    ])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "champion-sarah",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Pipeline(steps=[('vect', CountVectorizer()), ('tfidf', TfidfTransformer()),\n",
       "                ('clf',\n",
       "                 SGDClassifier(alpha=0.001, max_iter=5, random_state=42,\n",
       "                               tol=None))])"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "text_clf.fit(twenty_train.data, twenty_train.target)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "presidential-portsmouth",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.9101198402130493"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "predicted = text_clf.predict(docs_test)\n",
    "np.mean(predicted == twenty_test.target)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "adult-wyoming",
   "metadata": {},
   "source": [
    "`scikit-learn` further provides utilities for more detailed performance analysis of the results:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "korean-sherman",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "                        precision    recall  f1-score   support\n",
      "\n",
      "           alt.atheism       0.95      0.80      0.87       319\n",
      "         comp.graphics       0.87      0.98      0.92       389\n",
      "               sci.med       0.94      0.89      0.91       396\n",
      "soc.religion.christian       0.90      0.95      0.93       398\n",
      "\n",
      "              accuracy                           0.91      1502\n",
      "             macro avg       0.91      0.91      0.91      1502\n",
      "          weighted avg       0.91      0.91      0.91      1502\n",
      "\n"
     ]
    }
   ],
   "source": [
    "from sklearn import metrics\n",
    "print(metrics.classification_report(twenty_test.target, predicted, target_names=twenty_test.target_names))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "tutorial-powder",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[256,  11,  16,  36],\n",
       "       [  4, 380,   3,   2],\n",
       "       [  5,  35, 353,   3],\n",
       "       [  5,  11,   4, 378]], dtype=int64)"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "metrics.confusion_matrix(twenty_test.target,predicted)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "final-atmosphere",
   "metadata": {},
   "source": [
    "As expected, the confusion matrix shows that posts from the newsgroups on atheism and christian are more often confused for one another than with computer graphics."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ceramic-bloom",
   "metadata": {},
   "source": [
    "<h2>Parameter tuning using grid search</h2>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "martial-topic",
   "metadata": {},
   "source": [
    "We've already encountered some parameters such as `use_idf` in the `TfidfTransformer`. Classifiers tend to have many parameters as well; e.g. `MultinomialNB` includes a smoothing parameter `alpha` and `SGDClassifier` has a penalty parameter `alpha` and configurable loss and penalty terms in the objective function (see the module documentation, to get a description of these).\n",
    "\n",
    "Instead of tweaking the parameters of the various components of the chain, it is possible to run an exhaustive search of the best parameters on a grid of possible values. We try out all classifiers on either words or bigrams, with or without idf, and with a penalty parameter of either 0.01 or 0.001 fo the linear SVM:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "id": "declared-radar",
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.model_selection import GridSearchCV\n",
    "parameters = {'vect__ngram_range': [(1,1), (1,2)],\n",
    "             'tfidf__use_idf': (True, False),\n",
    "             'clf__alpha': (1e-2, 1e-3)}"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "functioning-aluminum",
   "metadata": {},
   "source": [
    "Obviously such an exhaustive search can be expensive. If we have multiple CPU cores at our disposal, we can tell the grid searcher to try these eight parameter combinations in parallel with the `n_jobs` parameter. If we give parameter a value of `-1` grid search will detect how many cores are installed and uses them all:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "id": "strategic-disorder",
   "metadata": {},
   "outputs": [],
   "source": [
    "gs_clf = GridSearchCV(text_clf, parameters, n_jobs=-1)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "standard-haiti",
   "metadata": {},
   "source": [
    "The grid search instance behaves like a normal `scikit-learn` model. Let's perform the search on a smaller subset of the training data to speed up the computation:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "id": "indirect-klein",
   "metadata": {},
   "outputs": [],
   "source": [
    "gs_clf = gs_clf.fit(twenty_train.data[:400], twenty_train.target[:400])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "eastern-logan",
   "metadata": {},
   "source": [
    "The result of calling `fit` on a `GridSearchCV` object is a classifier that we cna use to `predict`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "id": "outer-dealing",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'soc.religion.christian'"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "twenty_train.target_names[gs_clf.predict(['God is love'])[0]]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "normal-cancer",
   "metadata": {},
   "source": [
    "The objects `best_score_` and `best_params_` attributes sotre the bes mean score and the parameters setting corresponding to that score:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "id": "large-channel",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.9175000000000001"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "gs_clf.best_score_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "id": "essential-protest",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "clf__alpha: 0.001\n",
      "tfidf__use_idf: True\n",
      "vect__ngram_range: (1, 1)\n"
     ]
    }
   ],
   "source": [
    "for param_name in sorted(parameters.keys()):\n",
    "    print(\"%s: %r\" % (param_name, gs_clf.best_params_[param_name]))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "opponent-leave",
   "metadata": {},
   "source": [
    "A more detailed summary of the search is available at `gs_clf.cv_results_`. <br>\n",
    "The `cv_results_` parameter can be easily imported into pandas as a `DataFrame` for further inspection."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "blessed-handy",
   "metadata": {},
   "source": [
    "<h2>Exercises</h2>\n",
    "All exercises can be found <a href=\"https://scikit-learn.org/0.19/tutorial/text_analytics/working_with_text_data.html\">here</a>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "sustainable-monkey",
   "metadata": {},
   "source": [
    "<h3>Exercise 1: Language identification</h3>\n",
    "<ol type=\"1\">\n",
    "<li>Write a text classification pipeline using a custom preprocessor and CharNGramAnalyzer using data from Wikipedia articles as training set.</li>\n",
    "\n",
    "<li>Evaluate the performance on some held out test set.</li>\n",
    "</ol>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "declared-intelligence",
   "metadata": {},
   "source": [
    "<h3>Exercise 2: Sentiment Analysis on movie reviews</h3>\n",
    "<ol type=\"1\">\n",
    "<li>Write a text classification pipeline to classify movie reviews as either positive or negative.</li>\n",
    "<li>Find a good set of parameters using grid search.</li>\n",
    "<li>Evaluate the performance on a held out test set.</li>\n",
    "</ol>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "extra-attachment",
   "metadata": {},
   "source": [
    "<h3>Exercise 3: CLI text classification utility</h3>\n",
    "sing the results of the previous exercises and the cPickle module of the standard library, write a command line utility that detects the language of some text provided on stdin and estimate the polarity (positive or negative) if the text is written in English.\n",
    "\n",
    "Bonus point if the utility is able to give a confidence level for its predictions."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "shaped-groove",
   "metadata": {},
   "source": [
    "<h3>Where to from here</h3>\n",
    "Here are a few suggestions to help further your scikit-learn intuition upon the completion of this tutorial:\n",
    "<ul>\n",
    "<li>Try playing around with the analyzer and token normalisation under CountVectorizer</li>\n",
    "<li>If you don’t have labels, try using Clustering on your problem.</li>\n",
    "<li>If you have multiple labels per document, e.g categories, have a look at the Multiclass and multilabel section</li>\n",
    "<li>Try using Truncated SVD for latent semantic analysis.</li>\n",
    "<li>Have a look at using Out-of-core Classification to learn from data that would not fit into the computer main memory.</li>\n",
    "<li>Have a look at the Hashing Vectorizer as a memory efficient alternative to CountVectorizer.</li>\n",
    "</ul>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "blond-coordinate",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}