3ATrain_flair_predictor_traditional.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.6.9"
    },
    "colab": {
      "name": "Train flair predictor.ipynb",
      "provenance": [],
      "collapsed_sections": [],
      "toc_visible": true
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "nEpgyN-XwWjE",
        "colab_type": "text"
      },
      "source": [
        "# Traditional Machine Learning Methods\n",
        "\n",
        "There are multiple traditional methods that perform classification really well on Tokenized text.\n",
        "We will utitlize:\n",
        "1. various Probabilistic Classifiers such as Multinomial Naive bayes and Complement Naive Bayes.\n",
        "2. Linear SVMs\n",
        "3. Decision Tree Method such as Random Forest Classifier\n",
        "4. Other old algorithms such as nearest centroid and gradient boost classifier"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "bKGctk56wWjH",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "import pandas as pd\n",
        "import numpy as np\n",
        "from sklearn.feature_extraction.text import TfidfTransformer\n",
        "from sklearn.feature_extraction.text import CountVectorizer\n",
        "from sklearn.model_selection import train_test_split\n",
        "from sklearn.naive_bayes import MultinomialNB\n",
        "from sklearn.naive_bayes import ComplementNB\n",
        "from sklearn.linear_model import SGDClassifier\n",
        "from sklearn.pipeline import Pipeline\n",
        "from sklearn.metrics import classification_report\n",
        "from sklearn.model_selection import GridSearchCV\n",
        "from sklearn.linear_model import LogisticRegression\n",
        "from sklearn.ensemble import RandomForestClassifier"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "YtmrpdThwdq2",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 124
        },
        "outputId": "db44a301-d86d-46e0-83c2-6241df7f85e9"
      },
      "source": [
        "from google.colab import drive\n",
        "drive.mount('/content/drive')"
      ],
      "execution_count": 4,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Go to this URL in a browser: https://accounts.google.com/o/oauth2/auth?client_id=947318989803-6bn6qk8qdgf4n4g3pfee6491hc0brc4i.apps.googleusercontent.com&redirect_uri=urn%3aietf%3awg%3aoauth%3a2.0%3aoob&response_type=code&scope=email%20https%3a%2f%2fwww.googleapis.com%2fauth%2fdocs.test%20https%3a%2f%2fwww.googleapis.com%2fauth%2fdrive%20https%3a%2f%2fwww.googleapis.com%2fauth%2fdrive.photos.readonly%20https%3a%2f%2fwww.googleapis.com%2fauth%2fpeopleapi.readonly\n",
            "\n",
            "Enter your authorization code:\n",
            "··········\n",
            "Mounted at /content/drive\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "aTOJGeA9wWjW",
        "colab_type": "text"
      },
      "source": [
        "We will first prepare the dataset and extract the required flair and full_text columns"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "j3Ba0RCDwWjc",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "df = pd.read_csv('drive/My Drive/dataset_final.csv', engine='python')"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Iw7KYp2hwWjm",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 573
        },
        "outputId": "0df52a21-278c-4afc-ea78-08db9d22b034"
      },
      "source": [
        "df.head(5)"
      ],
      "execution_count": 6,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Unnamed: 0</th>\n",
              "      <th>title</th>\n",
              "      <th>flair</th>\n",
              "      <th>score</th>\n",
              "      <th>num_comments</th>\n",
              "      <th>author</th>\n",
              "      <th>created_utc</th>\n",
              "      <th>self_post</th>\n",
              "      <th>over_18</th>\n",
              "      <th>full_text</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>0</td>\n",
              "      <td>Even the poorest are supporting Modi in this.</td>\n",
              "      <td>Politics</td>\n",
              "      <td>64.0</td>\n",
              "      <td>78.0</td>\n",
              "      <td>hungarywolf</td>\n",
              "      <td>1.586106e+09</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>Even the poorest are supporting Modi in this.</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>1</td>\n",
              "      <td>Someone tried to sell Statue of Unity on Olx. ...</td>\n",
              "      <td>Non-Political</td>\n",
              "      <td>50.0</td>\n",
              "      <td>143.0</td>\n",
              "      <td>Athar147</td>\n",
              "      <td>1.586106e+09</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>Someone tried to sell Statue of Unity on Olx. ...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>2</td>\n",
              "      <td>Captured India with my phone.</td>\n",
              "      <td>Photography</td>\n",
              "      <td>49.0</td>\n",
              "      <td>202.0</td>\n",
              "      <td>random_saiyajin</td>\n",
              "      <td>1.586102e+09</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>Captured India with my phone.</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>3</td>\n",
              "      <td>You guys are too impure to understand Modi Ji'...</td>\n",
              "      <td>Coronavirus</td>\n",
              "      <td>39.0</td>\n",
              "      <td>81.0</td>\n",
              "      <td>AdmiralSP</td>\n",
              "      <td>1.586101e+09</td>\n",
              "      <td>True</td>\n",
              "      <td>False</td>\n",
              "      <td>You guys are too impure to understand Modi Ji'...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>4</td>\n",
              "      <td>Posting again because stupidity was on show to...</td>\n",
              "      <td>Coronavirus</td>\n",
              "      <td>34.0</td>\n",
              "      <td>21.0</td>\n",
              "      <td>msbuttergourd</td>\n",
              "      <td>1.586108e+09</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>Posting again because stupidity was on show to...</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "  Unnamed: 0  ...                                          full_text\n",
              "0          0  ...    Even the poorest are supporting Modi in this.  \n",
              "1          1  ...  Someone tried to sell Statue of Unity on Olx. ...\n",
              "2          2  ...                    Captured India with my phone.  \n",
              "3          3  ...  You guys are too impure to understand Modi Ji'...\n",
              "4          4  ...  Posting again because stupidity was on show to...\n",
              "\n",
              "[5 rows x 10 columns]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 6
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "nXUVQnGEwWjx",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "df.drop(['Unnamed: 0'], axis=1, inplace=True)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "I7rLuPnBwWj5",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 295
        },
        "outputId": "d3084913-2c70-474b-ddba-90d665db9a3f"
      },
      "source": [
        "df.info()"
      ],
      "execution_count": 8,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "<class 'pandas.core.frame.DataFrame'>\n",
            "RangeIndex: 106037 entries, 0 to 106036\n",
            "Data columns (total 9 columns):\n",
            " #   Column        Non-Null Count   Dtype  \n",
            "---  ------        --------------   -----  \n",
            " 0   title         106036 non-null  object \n",
            " 1   flair         106036 non-null  object \n",
            " 2   score         106036 non-null  float64\n",
            " 3   num_comments  106036 non-null  float64\n",
            " 4   author        106036 non-null  object \n",
            " 5   created_utc   106036 non-null  float64\n",
            " 6   self_post     106036 non-null  object \n",
            " 7   over_18       106036 non-null  object \n",
            " 8   full_text     106022 non-null  object \n",
            "dtypes: float64(3), object(6)\n",
            "memory usage: 7.3+ MB\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "EH7lauSJwWkC",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 208
        },
        "outputId": "c4305d58-4388-46f5-ff03-195ec5f7e4b7"
      },
      "source": [
        "df['flair'].value_counts()"
      ],
      "execution_count": 9,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "Non-Political         34037\n",
              "Politics              30192\n",
              "Policy/Economy        12880\n",
              "AskIndia              12640\n",
              "Science/Technology     4616\n",
              "Business/Finance       4513\n",
              "[R]eddiquette          3351\n",
              "Sports                 1660\n",
              "Photography            1128\n",
              "Coronavirus            1019\n",
              "Name: flair, dtype: int64"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 9
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "JJa9OtdHwWkK",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "sr = pd.isnull(df['title'])\n",
        "sr.loc[sr==True]\n",
        "df.drop(83369, axis=0, inplace=True)\n",
        "df = df.reset_index(drop=True)\n",
        "sr = pd.isnull(df['full_text'])\n",
        "drop_arr = sr.loc[sr==True].index.tolist()\n",
        "df.drop(drop_arr, axis=0, inplace=True)\n",
        "df = df.reset_index(drop=True)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "FHcrDGOnwWkS",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 295
        },
        "outputId": "32a1fdc8-ce9b-411d-fa71-4b974276abbe"
      },
      "source": [
        "df.info()"
      ],
      "execution_count": 11,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "<class 'pandas.core.frame.DataFrame'>\n",
            "RangeIndex: 106022 entries, 0 to 106021\n",
            "Data columns (total 9 columns):\n",
            " #   Column        Non-Null Count   Dtype  \n",
            "---  ------        --------------   -----  \n",
            " 0   title         106022 non-null  object \n",
            " 1   flair         106022 non-null  object \n",
            " 2   score         106022 non-null  float64\n",
            " 3   num_comments  106022 non-null  float64\n",
            " 4   author        106022 non-null  object \n",
            " 5   created_utc   106022 non-null  float64\n",
            " 6   self_post     106022 non-null  object \n",
            " 7   over_18       106022 non-null  object \n",
            " 8   full_text     106022 non-null  object \n",
            "dtypes: float64(3), object(6)\n",
            "memory usage: 7.3+ MB\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "BtcbkDwGwWkY",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "target_flairs = df['flair'].index.tolist()\n",
        "X_text = list(df['full_text'])\n",
        "Y = list(df['flair'])\n",
        "X_train, X_test, y_train, y_test = train_test_split(X_text, Y, test_size=0.3)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "gWitW_xXwWke",
        "colab_type": "text"
      },
      "source": [
        "The first algorithm we will try is [Multinomial Naive Bayes](https://scikit-learn.org/stable/modules/generated/sklearn.naive_bayes.MultinomialNB.html) which is a recommended method for a task such as text classification\n",
        "\n",
        "1. We will first build a pipeline by which we will first tokenize the text and convert to vector and pass it through a TFIDF transform and then pass the output to the MultinomialNB object.\n",
        "\n",
        "2. We must try out various parameters to get the best out of our algorithms. To implement this in an easy and efficient way, we will use the GridSearch feature offered by Scikit-Learn where it makes a grid of the different parameters we set and executes each of them and returns to us the best parameters.\n",
        "\n",
        "3. We will predict using the trained model and get the final classification report"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "reP5VJNFwWkg",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "naive_bayes = Pipeline([('vect', CountVectorizer(stop_words='english')),\n",
        "               ('tfidf', TfidfTransformer()),\n",
        "               ('clf', MultinomialNB(fit_prior=False)),\n",
        "              ], verbose=True)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ARyT03GVwWkn",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "parameters = {'vect__ngram_range': [(1, 1), (1, 2)],\n",
        "               'tfidf__use_idf': (True, False),\n",
        "               'clf__alpha': (1, 1e-1, 1e-2)}"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "AxqhH5DwwWkt",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "gs_naive_bayes = GridSearchCV(naive_bayes, parameters, verbose=3)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "A2vt6sYjwWk0",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 1000
        },
        "outputId": "dc0cb4af-75dd-45da-9d69-f141f754e512"
      },
      "source": [
        "gs_naive_bayes = gs_naive_bayes.fit(X_train, y_train)"
      ],
      "execution_count": 19,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Fitting 5 folds for each of 12 candidates, totalling 60 fits\n",
            "[CV] clf__alpha=1, tfidf__use_idf=True, vect__ngram_range=(1, 1) .....\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "[Parallel(n_jobs=1)]: Using backend SequentialBackend with 1 concurrent workers.\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[Pipeline] .............. (step 1 of 3) Processing vect, total=   1.6s\n",
            "[Pipeline] ............. (step 2 of 3) Processing tfidf, total=   0.1s\n",
            "[Pipeline] ............... (step 3 of 3) Processing clf, total=   0.1s\n",
            "[CV]  clf__alpha=1, tfidf__use_idf=True, vect__ngram_range=(1, 1), score=0.596, total=   2.3s\n",
            "[CV] clf__alpha=1, tfidf__use_idf=True, vect__ngram_range=(1, 1) .....\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    2.3s remaining:    0.0s\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[Pipeline] .............. (step 1 of 3) Processing vect, total=   1.6s\n",
            "[Pipeline] ............. (step 2 of 3) Processing tfidf, total=   0.1s\n",
            "[Pipeline] ............... (step 3 of 3) Processing clf, total=   0.1s\n",
            "[CV]  clf__alpha=1, tfidf__use_idf=True, vect__ngram_range=(1, 1), score=0.591, total=   2.3s\n",
            "[CV] clf__alpha=1, tfidf__use_idf=True, vect__ngram_range=(1, 1) .....\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    4.6s remaining:    0.0s\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[Pipeline] .............. (step 1 of 3) Processing vect, total=   1.6s\n",
            "[Pipeline] ............. (step 2 of 3) Processing tfidf, total=   0.1s\n",
            "[Pipeline] ............... (step 3 of 3) Processing clf, total=   0.1s\n",
            "[CV]  clf__alpha=1, tfidf__use_idf=True, vect__ngram_range=(1, 1), score=0.582, total=   2.3s\n",
            "[CV] clf__alpha=1, tfidf__use_idf=True, vect__ngram_range=(1, 1) .....\n",
            "[Pipeline] .............. (step 1 of 3) Processing vect, total=   1.6s\n",
            "[Pipeline] ............. (step 2 of 3) Processing tfidf, total=   0.1s\n",
            "[Pipeline] ............... (step 3 of 3) Processing clf, total=   0.1s\n",
            "[CV]  clf__alpha=1, tfidf__use_idf=True, vect__ngram_range=(1, 1), score=0.589, total=   2.3s\n",
            "[CV] clf__alpha=1, tfidf__use_idf=True, vect__ngram_range=(1, 1) .....\n",
            "[Pipeline] .............. (step 1 of 3) Processing vect, total=   1.6s\n",
            "[Pipeline] ............. (step 2 of 3) Processing tfidf, total=   0.1s\n",
            "[Pipeline] ............... (step 3 of 3) Processing clf, total=   0.1s\n",
            "[CV]  clf__alpha=1, tfidf__use_idf=True, vect__ngram_range=(1, 1), score=0.592, total=   2.3s\n",
            "[CV] clf__alpha=1, tfidf__use_idf=True, vect__ngram_range=(1, 2) .....\n",
            "[Pipeline] .............. (step 1 of 3) Processing vect, total=   6.2s\n",
            "[Pipeline] ............. (step 2 of 3) Processing tfidf, total=   0.7s\n",
            "[Pipeline] ............... (step 3 of 3) Processing clf, total=   0.5s\n",
            "[CV]  clf__alpha=1, tfidf__use_idf=True, vect__ngram_range=(1, 2), score=0.580, total=   8.2s\n",
            "[CV] clf__alpha=1, tfidf__use_idf=True, vect__ngram_range=(1, 2) .....\n",
            "[Pipeline] .............. (step 1 of 3) Processing vect, total=   6.2s\n",
            "[Pipeline] ............. (step 2 of 3) Processing tfidf, total=   0.6s\n",
            "[Pipeline] ............... (step 3 of 3) Processing clf, total=   0.4s\n",
            "[CV]  clf__alpha=1, tfidf__use_idf=True, vect__ngram_range=(1, 2), score=0.572, total=   8.0s\n",
            "[CV] clf__alpha=1, tfidf__use_idf=True, vect__ngram_range=(1, 2) .....\n",
            "[Pipeline] .............. (step 1 of 3) Processing vect, total=   6.1s\n",
            "[Pipeline] ............. (step 2 of 3) Processing tfidf, total=   0.5s\n",
            "[Pipeline] ............... (step 3 of 3) Processing clf, total=   0.4s\n",
            "[CV]  clf__alpha=1, tfidf__use_idf=True, vect__ngram_range=(1, 2), score=0.571, total=   7.8s\n",
            "[CV] clf__alpha=1, tfidf__use_idf=True, vect__ngram_range=(1, 2) .....\n",
            "[Pipeline] .............. (step 1 of 3) Processing vect, total=   6.1s\n",
            "[Pipeline] ............. (step 2 of 3) Processing tfidf, total=   0.6s\n",
            "[Pipeline] ............... (step 3 of 3) Processing clf, total=   0.4s\n",
            "[CV]  clf__alpha=1, tfidf__use_idf=True, vect__ngram_range=(1, 2), score=0.578, total=   7.9s\n",
            "[CV] clf__alpha=1, tfidf__use_idf=True, vect__ngram_range=(1, 2) .....\n",
            "[Pipeline] .............. (step 1 of 3) Processing vect, total=   6.1s\n",
            "[Pipeline] ............. (step 2 of 3) Processing tfidf, total=   0.7s\n",
            "[Pipeline] ............... (step 3 of 3) Processing clf, total=   0.4s\n",
            "[CV]  clf__alpha=1, tfidf__use_idf=True, vect__ngram_range=(1, 2), score=0.573, total=   8.0s\n",
            "[CV] clf__alpha=1, tfidf__use_idf=False, vect__ngram_range=(1, 1) ....\n",
            "[Pipeline] .............. (step 1 of 3) Processing vect, total=   1.7s\n",
            "[Pipeline] ............. (step 2 of 3) Processing tfidf, total=   0.0s\n",
            "[Pipeline] ............... (step 3 of 3) Processing clf, total=   0.1s\n",
            "[CV]  clf__alpha=1, tfidf__use_idf=False, vect__ngram_range=(1, 1), score=0.597, total=   2.3s\n",
            "[CV] clf__alpha=1, tfidf__use_idf=False, vect__ngram_range=(1, 1) ....\n",
            "[Pipeline] .............. (step 1 of 3) Processing vect, total=   1.6s\n",
            "[Pipeline] ............. (step 2 of 3) Processing tfidf, total=   0.0s\n",
            "[Pipeline] ............... (step 3 of 3) Processing clf, total=   0.1s\n",
            "[CV]  clf__alpha=1, tfidf__use_idf=False, vect__ngram_range=(1, 1), score=0.586, total=   2.3s\n",
            "[CV] clf__alpha=1, tfidf__use_idf=False, vect__ngram_range=(1, 1) ....\n",
            "[Pipeline] .............. (step 1 of 3) Processing vect, total=   1.6s\n",
            "[Pipeline] ............. (step 2 of 3) Processing tfidf, total=   0.0s\n",
            "[Pipeline] ............... (step 3 of 3) Processing clf, total=   0.1s\n",
            "[CV]  clf__alpha=1, tfidf__use_idf=False, vect__ngram_range=(1, 1), score=0.580, total=   2.2s\n",
            "[CV] clf__alpha=1, tfidf__use_idf=False, vect__ngram_range=(1, 1) ....\n",
            "[Pipeline] .............. (step 1 of 3) Processing vect, total=   1.6s\n",
            "[Pipeline] ............. (step 2 of 3) Processing tfidf, total=   0.0s\n",
            "[Pipeline] ............... (step 3 of 3) Processing clf, total=   0.1s\n",
            "[CV]  clf__alpha=1, tfidf__use_idf=False, vect__ngram_range=(1, 1), score=0.589, total=   2.2s\n",
            "[CV] clf__alpha=1, tfidf__use_idf=False, vect__ngram_range=(1, 1) ....\n",
            "[Pipeline] .............. (step 1 of 3) Processing vect, total=   1.6s\n",
            "[Pipeline] ............. (step 2 of 3) Processing tfidf, total=   0.0s\n",
            "[Pipeline] ............... (step 3 of 3) Processing clf, total=   0.1s\n",
            "[CV]  clf__alpha=1, tfidf__use_idf=False, vect__ngram_range=(1, 1), score=0.588, total=   2.2s\n",
            "[CV] clf__alpha=1, tfidf__use_idf=False, vect__ngram_range=(1, 2) ....\n",
            "[Pipeline] .............. (step 1 of 3) Processing vect, total=   6.2s\n",
            "[Pipeline] ............. (step 2 of 3) Processing tfidf, total=   0.1s\n",
            "[Pipeline] ............... (step 3 of 3) Processing clf, total=   0.4s\n",
            "[CV]  clf__alpha=1, tfidf__use_idf=False, vect__ngram_range=(1, 2), score=0.572, total=   7.5s\n",
            "[CV] clf__alpha=1, tfidf__use_idf=False, vect__ngram_range=(1, 2) ....\n",
            "[Pipeline] .............. (step 1 of 3) Processing vect, total=   6.2s\n",
            "[Pipeline] ............. (step 2 of 3) Processing tfidf, total=   0.1s\n",
            "[Pipeline] ............... (step 3 of 3) Processing clf, total=   0.4s\n",
            "[CV]  clf__alpha=1, tfidf__use_idf=False, vect__ngram_range=(1, 2), score=0.559, total=   7.4s\n",
            "[CV] clf__alpha=1, tfidf__use_idf=False, vect__ngram_range=(1, 2) ....\n",
            "[Pipeline] .............. (step 1 of 3) Processing vect, total=   6.2s\n",
            "[Pipeline] ............. (step 2 of 3) Processing tfidf, total=   0.1s\n",
            "[Pipeline] ............... (step 3 of 3) Processing clf, total=   0.4s\n",
            "[CV]  clf__alpha=1, tfidf__use_idf=False, vect__ngram_range=(1, 2), score=0.559, total=   7.4s\n",
            "[CV] clf__alpha=1, tfidf__use_idf=False, vect__ngram_range=(1, 2) ....\n",
            "[Pipeline] .............. (step 1 of 3) Processing vect, total=   6.1s\n",
            "[Pipeline] ............. (step 2 of 3) Processing tfidf, total=   0.1s\n",
            "[Pipeline] ............... (step 3 of 3) Processing clf, total=   0.4s\n",
            "[CV]  clf__alpha=1, tfidf__use_idf=False, vect__ngram_range=(1, 2), score=0.567, total=   7.4s\n",
            "[CV] clf__alpha=1, tfidf__use_idf=False, vect__ngram_range=(1, 2) ....\n",
            "[Pipeline] .............. (step 1 of 3) Processing vect, total=   6.1s\n",
            "[Pipeline] ............. (step 2 of 3) Processing tfidf, total=   0.1s\n",
            "[Pipeline] ............... (step 3 of 3) Processing clf, total=   0.4s\n",
            "[CV]  clf__alpha=1, tfidf__use_idf=False, vect__ngram_range=(1, 2), score=0.565, total=   7.3s\n",
            "[CV] clf__alpha=0.1, tfidf__use_idf=True, vect__ngram_range=(1, 1) ...\n",
            "[Pipeline] .............. (step 1 of 3) Processing vect, total=   1.7s\n",
            "[Pipeline] ............. (step 2 of 3) Processing tfidf, total=   0.1s\n",
            "[Pipeline] ............... (step 3 of 3) Processing clf, total=   0.1s\n",
            "[CV]  clf__alpha=0.1, tfidf__use_idf=True, vect__ngram_range=(1, 1), score=0.558, total=   2.3s\n",
            "[CV] clf__alpha=0.1, tfidf__use_idf=True, vect__ngram_range=(1, 1) ...\n",
            "[Pipeline] .............. (step 1 of 3) Processing vect, total=   1.6s\n",
            "[Pipeline] ............. (step 2 of 3) Processing tfidf, total=   0.1s\n",
            "[Pipeline] ............... (step 3 of 3) Processing clf, total=   0.1s\n",
            "[CV]  clf__alpha=0.1, tfidf__use_idf=True, vect__ngram_range=(1, 1), score=0.559, total=   2.3s\n",
            "[CV] clf__alpha=0.1, tfidf__use_idf=True, vect__ngram_range=(1, 1) ...\n",
            "[Pipeline] .............. (step 1 of 3) Processing vect, total=   1.6s\n",
            "[Pipeline] ............. (step 2 of 3) Processing tfidf, total=   0.1s\n",
            "[Pipeline] ............... (step 3 of 3) Processing clf, total=   0.1s\n",
            "[CV]  clf__alpha=0.1, tfidf__use_idf=True, vect__ngram_range=(1, 1), score=0.550, total=   2.3s\n",
            "[CV] clf__alpha=0.1, tfidf__use_idf=True, vect__ngram_range=(1, 1) ...\n",
            "[Pipeline] .............. (step 1 of 3) Processing vect, total=   1.6s\n",
            "[Pipeline] ............. (step 2 of 3) Processing tfidf, total=   0.1s\n",
            "[Pipeline] ............... (step 3 of 3) Processing clf, total=   0.1s\n",
            "[CV]  clf__alpha=0.1, tfidf__use_idf=True, vect__ngram_range=(1, 1), score=0.555, total=   2.3s\n",
            "[CV] clf__alpha=0.1, tfidf__use_idf=True, vect__ngram_range=(1, 1) ...\n",
            "[Pipeline] .............. (step 1 of 3) Processing vect, total=   1.6s\n",
            "[Pipeline] ............. (step 2 of 3) Processing tfidf, total=   0.1s\n",
            "[Pipeline] ............... (step 3 of 3) Processing clf, total=   0.1s\n",
            "[CV]  clf__alpha=0.1, tfidf__use_idf=True, vect__ngram_range=(1, 1), score=0.555, total=   2.3s\n",
            "[CV] clf__alpha=0.1, tfidf__use_idf=True, vect__ngram_range=(1, 2) ...\n",
            "[Pipeline] .............. (step 1 of 3) Processing vect, total=   6.2s\n",
            "[Pipeline] ............. (step 2 of 3) Processing tfidf, total=   0.6s\n",
            "[Pipeline] ............... (step 3 of 3) Processing clf, total=   0.5s\n",
            "[CV]  clf__alpha=0.1, tfidf__use_idf=True, vect__ngram_range=(1, 2), score=0.596, total=   8.1s\n",
            "[CV] clf__alpha=0.1, tfidf__use_idf=True, vect__ngram_range=(1, 2) ...\n",
            "[Pipeline] .............. (step 1 of 3) Processing vect, total=   6.2s\n",
            "[Pipeline] ............. (step 2 of 3) Processing tfidf, total=   0.6s\n",
            "[Pipeline] ............... (step 3 of 3) Processing clf, total=   0.5s\n",
            "[CV]  clf__alpha=0.1, tfidf__use_idf=True, vect__ngram_range=(1, 2), score=0.592, total=   8.1s\n",
            "[CV] clf__alpha=0.1, tfidf__use_idf=True, vect__ngram_range=(1, 2) ...\n",
            "[Pipeline] .............. (step 1 of 3) Processing vect, total=   6.1s\n",
            "[Pipeline] ............. (step 2 of 3) Processing tfidf, total=   0.6s\n",
            "[Pipeline] ............... (step 3 of 3) Processing clf, total=   0.5s\n",
            "[CV]  clf__alpha=0.1, tfidf__use_idf=True, vect__ngram_range=(1, 2), score=0.586, total=   8.1s\n",
            "[CV] clf__alpha=0.1, tfidf__use_idf=True, vect__ngram_range=(1, 2) ...\n",
            "[Pipeline] .............. (step 1 of 3) Processing vect, total=   6.1s\n",
            "[Pipeline] ............. (step 2 of 3) Processing tfidf, total=   0.7s\n",
            "[Pipeline] ............... (step 3 of 3) Processing clf, total=   0.5s\n",
            "[CV]  clf__alpha=0.1, tfidf__use_idf=True, vect__ngram_range=(1, 2), score=0.590, total=   8.1s\n",
            "[CV] clf__alpha=0.1, tfidf__use_idf=True, vect__ngram_range=(1, 2) ...\n",
            "[Pipeline] .............. (step 1 of 3) Processing vect, total=   6.2s\n",
            "[Pipeline] ............. (step 2 of 3) Processing tfidf, total=   0.6s\n",
            "[Pipeline] ............... (step 3 of 3) Processing clf, total=   0.5s\n",
            "[CV]  clf__alpha=0.1, tfidf__use_idf=True, vect__ngram_range=(1, 2), score=0.592, total=   8.1s\n",
            "[CV] clf__alpha=0.1, tfidf__use_idf=False, vect__ngram_range=(1, 1) ..\n",
            "[Pipeline] .............. (step 1 of 3) Processing vect, total=   1.6s\n",
            "[Pipeline] ............. (step 2 of 3) Processing tfidf, total=   0.0s\n",
            "[Pipeline] ............... (step 3 of 3) Processing clf, total=   0.1s\n",
            "[CV]  clf__alpha=0.1, tfidf__use_idf=False, vect__ngram_range=(1, 1), score=0.576, total=   2.2s\n",
            "[CV] clf__alpha=0.1, tfidf__use_idf=False, vect__ngram_range=(1, 1) ..\n",
            "[Pipeline] .............. (step 1 of 3) Processing vect, total=   1.6s\n",
            "[Pipeline] ............. (step 2 of 3) Processing tfidf, total=   0.0s\n",
            "[Pipeline] ............... (step 3 of 3) Processing clf, total=   0.1s\n",
            "[CV]  clf__alpha=0.1, tfidf__use_idf=False, vect__ngram_range=(1, 1), score=0.574, total=   2.3s\n",
            "[CV] clf__alpha=0.1, tfidf__use_idf=False, vect__ngram_range=(1, 1) ..\n",
            "[Pipeline] .............. (step 1 of 3) Processing vect, total=   1.6s\n",
            "[Pipeline] ............. (step 2 of 3) Processing tfidf, total=   0.0s\n",
            "[Pipeline] ............... (step 3 of 3) Processing clf, total=   0.1s\n",
            "[CV]  clf__alpha=0.1, tfidf__use_idf=False, vect__ngram_range=(1, 1), score=0.568, total=   2.2s\n",
            "[CV] clf__alpha=0.1, tfidf__use_idf=False, vect__ngram_range=(1, 1) ..\n",
            "[Pipeline] .............. (step 1 of 3) Processing vect, total=   1.6s\n",
            "[Pipeline] ............. (step 2 of 3) Processing tfidf, total=   0.0s\n",
            "[Pipeline] ............... (step 3 of 3) Processing clf, total=   0.1s\n",
            "[CV]  clf__alpha=0.1, tfidf__use_idf=False, vect__ngram_range=(1, 1), score=0.572, total=   2.2s\n",
            "[CV] clf__alpha=0.1, tfidf__use_idf=False, vect__ngram_range=(1, 1) ..\n",
            "[Pipeline] .............. (step 1 of 3) Processing vect, total=   1.6s\n",
            "[Pipeline] ............. (step 2 of 3) Processing tfidf, total=   0.0s\n",
            "[Pipeline] ............... (step 3 of 3) Processing clf, total=   0.1s\n",
            "[CV]  clf__alpha=0.1, tfidf__use_idf=False, vect__ngram_range=(1, 1), score=0.574, total=   2.2s\n",
            "[CV] clf__alpha=0.1, tfidf__use_idf=False, vect__ngram_range=(1, 2) ..\n",
            "[Pipeline] .............. (step 1 of 3) Processing vect, total=   6.1s\n",
            "[Pipeline] ............. (step 2 of 3) Processing tfidf, total=   0.1s\n",
            "[Pipeline] ............... (step 3 of 3) Processing clf, total=   0.5s\n",
            "[CV]  clf__alpha=0.1, tfidf__use_idf=False, vect__ngram_range=(1, 2), score=0.604, total=   7.5s\n",
            "[CV] clf__alpha=0.1, tfidf__use_idf=False, vect__ngram_range=(1, 2) ..\n",
            "[Pipeline] .............. (step 1 of 3) Processing vect, total=   6.1s\n",
            "[Pipeline] ............. (step 2 of 3) Processing tfidf, total=   0.1s\n",
            "[Pipeline] ............... (step 3 of 3) Processing clf, total=   0.5s\n",
            "[CV]  clf__alpha=0.1, tfidf__use_idf=False, vect__ngram_range=(1, 2), score=0.594, total=   7.5s\n",
            "[CV] clf__alpha=0.1, tfidf__use_idf=False, vect__ngram_range=(1, 2) ..\n",
            "[Pipeline] .............. (step 1 of 3) Processing vect, total=   6.0s\n",
            "[Pipeline] ............. (step 2 of 3) Processing tfidf, total=   0.1s\n",
            "[Pipeline] ............... (step 3 of 3) Processing clf, total=   0.5s\n",
            "[CV]  clf__alpha=0.1, tfidf__use_idf=False, vect__ngram_range=(1, 2), score=0.592, total=   7.4s\n",
            "[CV] clf__alpha=0.1, tfidf__use_idf=False, vect__ngram_range=(1, 2) ..\n",
            "[Pipeline] .............. (step 1 of 3) Processing vect, total=   6.1s\n",
            "[Pipeline] ............. (step 2 of 3) Processing tfidf, total=   0.1s\n",
            "[Pipeline] ............... (step 3 of 3) Processing clf, total=   0.5s\n",
            "[CV]  clf__alpha=0.1, tfidf__use_idf=False, vect__ngram_range=(1, 2), score=0.595, total=   7.4s\n",
            "[CV] clf__alpha=0.1, tfidf__use_idf=False, vect__ngram_range=(1, 2) ..\n",
            "[Pipeline] .............. (step 1 of 3) Processing vect, total=   6.1s\n",
            "[Pipeline] ............. (step 2 of 3) Processing tfidf, total=   0.1s\n",
            "[Pipeline] ............... (step 3 of 3) Processing clf, total=   0.5s\n",
            "[CV]  clf__alpha=0.1, tfidf__use_idf=False, vect__ngram_range=(1, 2), score=0.595, total=   7.5s\n",
            "[CV] clf__alpha=0.01, tfidf__use_idf=True, vect__ngram_range=(1, 1) ..\n",
            "[Pipeline] .............. (step 1 of 3) Processing vect, total=   1.6s\n",
            "[Pipeline] ............. (step 2 of 3) Processing tfidf, total=   0.1s\n",
            "[Pipeline] ............... (step 3 of 3) Processing clf, total=   0.1s\n",
            "[CV]  clf__alpha=0.01, tfidf__use_idf=True, vect__ngram_range=(1, 1), score=0.518, total=   2.3s\n",
            "[CV] clf__alpha=0.01, tfidf__use_idf=True, vect__ngram_range=(1, 1) ..\n",
            "[Pipeline] .............. (step 1 of 3) Processing vect, total=   1.6s\n",
            "[Pipeline] ............. (step 2 of 3) Processing tfidf, total=   0.1s\n",
            "[Pipeline] ............... (step 3 of 3) Processing clf, total=   0.1s\n",
            "[CV]  clf__alpha=0.01, tfidf__use_idf=True, vect__ngram_range=(1, 1), score=0.519, total=   2.3s\n",
            "[CV] clf__alpha=0.01, tfidf__use_idf=True, vect__ngram_range=(1, 1) ..\n",
            "[Pipeline] .............. (step 1 of 3) Processing vect, total=   1.6s\n",
            "[Pipeline] ............. (step 2 of 3) Processing tfidf, total=   0.1s\n",
            "[Pipeline] ............... (step 3 of 3) Processing clf, total=   0.1s\n",
            "[CV]  clf__alpha=0.01, tfidf__use_idf=True, vect__ngram_range=(1, 1), score=0.513, total=   2.3s\n",
            "[CV] clf__alpha=0.01, tfidf__use_idf=True, vect__ngram_range=(1, 1) ..\n",
            "[Pipeline] .............. (step 1 of 3) Processing vect, total=   1.6s\n",
            "[Pipeline] ............. (step 2 of 3) Processing tfidf, total=   0.1s\n",
            "[Pipeline] ............... (step 3 of 3) Processing clf, total=   0.1s\n",
            "[CV]  clf__alpha=0.01, tfidf__use_idf=True, vect__ngram_range=(1, 1), score=0.516, total=   2.3s\n",
            "[CV] clf__alpha=0.01, tfidf__use_idf=True, vect__ngram_range=(1, 1) ..\n",
            "[Pipeline] .............. (step 1 of 3) Processing vect, total=   1.6s\n",
            "[Pipeline] ............. (step 2 of 3) Processing tfidf, total=   0.1s\n",
            "[Pipeline] ............... (step 3 of 3) Processing clf, total=   0.1s\n",
            "[CV]  clf__alpha=0.01, tfidf__use_idf=True, vect__ngram_range=(1, 1), score=0.515, total=   2.3s\n",
            "[CV] clf__alpha=0.01, tfidf__use_idf=True, vect__ngram_range=(1, 2) ..\n",
            "[Pipeline] .............. (step 1 of 3) Processing vect, total=   6.1s\n",
            "[Pipeline] ............. (step 2 of 3) Processing tfidf, total=   0.6s\n",
            "[Pipeline] ............... (step 3 of 3) Processing clf, total=   0.5s\n",
            "[CV]  clf__alpha=0.01, tfidf__use_idf=True, vect__ngram_range=(1, 2), score=0.561, total=   8.1s\n",
            "[CV] clf__alpha=0.01, tfidf__use_idf=True, vect__ngram_range=(1, 2) ..\n",
            "[Pipeline] .............. (step 1 of 3) Processing vect, total=   6.1s\n",
            "[Pipeline] ............. (step 2 of 3) Processing tfidf, total=   0.6s\n",
            "[Pipeline] ............... (step 3 of 3) Processing clf, total=   0.5s\n",
            "[CV]  clf__alpha=0.01, tfidf__use_idf=True, vect__ngram_range=(1, 2), score=0.554, total=   8.1s\n",
            "[CV] clf__alpha=0.01, tfidf__use_idf=True, vect__ngram_range=(1, 2) ..\n",
            "[Pipeline] .............. (step 1 of 3) Processing vect, total=   6.2s\n",
            "[Pipeline] ............. (step 2 of 3) Processing tfidf, total=   0.7s\n",
            "[Pipeline] ............... (step 3 of 3) Processing clf, total=   0.5s\n",
            "[CV]  clf__alpha=0.01, tfidf__use_idf=True, vect__ngram_range=(1, 2), score=0.554, total=   8.3s\n",
            "[CV] clf__alpha=0.01, tfidf__use_idf=True, vect__ngram_range=(1, 2) ..\n",
            "[Pipeline] .............. (step 1 of 3) Processing vect, total=   6.3s\n",
            "[Pipeline] ............. (step 2 of 3) Processing tfidf, total=   0.6s\n",
            "[Pipeline] ............... (step 3 of 3) Processing clf, total=   0.5s\n",
            "[CV]  clf__alpha=0.01, tfidf__use_idf=True, vect__ngram_range=(1, 2), score=0.558, total=   8.3s\n",
            "[CV] clf__alpha=0.01, tfidf__use_idf=True, vect__ngram_range=(1, 2) ..\n",
            "[Pipeline] .............. (step 1 of 3) Processing vect, total=   6.2s\n",
            "[Pipeline] ............. (step 2 of 3) Processing tfidf, total=   0.6s\n",
            "[Pipeline] ............... (step 3 of 3) Processing clf, total=   0.5s\n",
            "[CV]  clf__alpha=0.01, tfidf__use_idf=True, vect__ngram_range=(1, 2), score=0.559, total=   8.1s\n",
            "[CV] clf__alpha=0.01, tfidf__use_idf=False, vect__ngram_range=(1, 1) .\n",
            "[Pipeline] .............. (step 1 of 3) Processing vect, total=   1.6s\n",
            "[Pipeline] ............. (step 2 of 3) Processing tfidf, total=   0.0s\n",
            "[Pipeline] ............... (step 3 of 3) Processing clf, total=   0.1s\n",
            "[CV]  clf__alpha=0.01, tfidf__use_idf=False, vect__ngram_range=(1, 1), score=0.535, total=   2.3s\n",
            "[CV] clf__alpha=0.01, tfidf__use_idf=False, vect__ngram_range=(1, 1) .\n",
            "[Pipeline] .............. (step 1 of 3) Processing vect, total=   1.6s\n",
            "[Pipeline] ............. (step 2 of 3) Processing tfidf, total=   0.0s\n",
            "[Pipeline] ............... (step 3 of 3) Processing clf, total=   0.1s\n",
            "[CV]  clf__alpha=0.01, tfidf__use_idf=False, vect__ngram_range=(1, 1), score=0.537, total=   2.2s\n",
            "[CV] clf__alpha=0.01, tfidf__use_idf=False, vect__ngram_range=(1, 1) .\n",
            "[Pipeline] .............. (step 1 of 3) Processing vect, total=   1.6s\n",
            "[Pipeline] ............. (step 2 of 3) Processing tfidf, total=   0.0s\n",
            "[Pipeline] ............... (step 3 of 3) Processing clf, total=   0.1s\n",
            "[CV]  clf__alpha=0.01, tfidf__use_idf=False, vect__ngram_range=(1, 1), score=0.530, total=   2.2s\n",
            "[CV] clf__alpha=0.01, tfidf__use_idf=False, vect__ngram_range=(1, 1) .\n",
            "[Pipeline] .............. (step 1 of 3) Processing vect, total=   1.6s\n",
            "[Pipeline] ............. (step 2 of 3) Processing tfidf, total=   0.0s\n",
            "[Pipeline] ............... (step 3 of 3) Processing clf, total=   0.1s\n",
            "[CV]  clf__alpha=0.01, tfidf__use_idf=False, vect__ngram_range=(1, 1), score=0.533, total=   2.3s\n",
            "[CV] clf__alpha=0.01, tfidf__use_idf=False, vect__ngram_range=(1, 1) .\n",
            "[Pipeline] .............. (step 1 of 3) Processing vect, total=   1.6s\n",
            "[Pipeline] ............. (step 2 of 3) Processing tfidf, total=   0.0s\n",
            "[Pipeline] ............... (step 3 of 3) Processing clf, total=   0.1s\n",
            "[CV]  clf__alpha=0.01, tfidf__use_idf=False, vect__ngram_range=(1, 1), score=0.533, total=   2.2s\n",
            "[CV] clf__alpha=0.01, tfidf__use_idf=False, vect__ngram_range=(1, 2) .\n",
            "[Pipeline] .............. (step 1 of 3) Processing vect, total=   6.2s\n",
            "[Pipeline] ............. (step 2 of 3) Processing tfidf, total=   0.1s\n",
            "[Pipeline] ............... (step 3 of 3) Processing clf, total=   0.5s\n",
            "[CV]  clf__alpha=0.01, tfidf__use_idf=False, vect__ngram_range=(1, 2), score=0.580, total=   7.6s\n",
            "[CV] clf__alpha=0.01, tfidf__use_idf=False, vect__ngram_range=(1, 2) .\n",
            "[Pipeline] .............. (step 1 of 3) Processing vect, total=   6.2s\n",
            "[Pipeline] ............. (step 2 of 3) Processing tfidf, total=   0.1s\n",
            "[Pipeline] ............... (step 3 of 3) Processing clf, total=   0.5s\n",
            "[CV]  clf__alpha=0.01, tfidf__use_idf=False, vect__ngram_range=(1, 2), score=0.573, total=   7.5s\n",
            "[CV] clf__alpha=0.01, tfidf__use_idf=False, vect__ngram_range=(1, 2) .\n",
            "[Pipeline] .............. (step 1 of 3) Processing vect, total=   6.2s\n",
            "[Pipeline] ............. (step 2 of 3) Processing tfidf, total=   0.1s\n",
            "[Pipeline] ............... (step 3 of 3) Processing clf, total=   0.5s\n",
            "[CV]  clf__alpha=0.01, tfidf__use_idf=False, vect__ngram_range=(1, 2), score=0.571, total=   7.6s\n",
            "[CV] clf__alpha=0.01, tfidf__use_idf=False, vect__ngram_range=(1, 2) .\n",
            "[Pipeline] .............. (step 1 of 3) Processing vect, total=   6.2s\n",
            "[Pipeline] ............. (step 2 of 3) Processing tfidf, total=   0.1s\n",
            "[Pipeline] ............... (step 3 of 3) Processing clf, total=   0.5s\n",
            "[CV]  clf__alpha=0.01, tfidf__use_idf=False, vect__ngram_range=(1, 2), score=0.576, total=   7.6s\n",
            "[CV] clf__alpha=0.01, tfidf__use_idf=False, vect__ngram_range=(1, 2) .\n",
            "[Pipeline] .............. (step 1 of 3) Processing vect, total=   6.2s\n",
            "[Pipeline] ............. (step 2 of 3) Processing tfidf, total=   0.1s\n",
            "[Pipeline] ............... (step 3 of 3) Processing clf, total=   0.5s\n",
            "[CV]  clf__alpha=0.01, tfidf__use_idf=False, vect__ngram_range=(1, 2), score=0.577, total=   7.5s\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "[Parallel(n_jobs=1)]: Done  60 out of  60 | elapsed:  5.1min finished\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[Pipeline] .............. (step 1 of 3) Processing vect, total=   7.7s\n",
            "[Pipeline] ............. (step 2 of 3) Processing tfidf, total=   0.1s\n",
            "[Pipeline] ............... (step 3 of 3) Processing clf, total=   0.6s\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "mAtJ5MVgwWk6",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 312
        },
        "outputId": "c1c453a2-4196-4c2e-9638-453023cabb84"
      },
      "source": [
        "y_pred = gs_naive_bayes.predict(X_test)\n",
        "print(classification_report(y_test, y_pred))"
      ],
      "execution_count": 20,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "                    precision    recall  f1-score   support\n",
            "\n",
            "          AskIndia       0.54      0.65      0.59      3776\n",
            "  Business/Finance       0.62      0.15      0.24      1368\n",
            "       Coronavirus       0.72      0.11      0.19       324\n",
            "     Non-Political       0.55      0.69      0.61     10260\n",
            "       Photography       0.80      0.23      0.36       333\n",
            "    Policy/Economy       0.56      0.50      0.53      3899\n",
            "          Politics       0.69      0.77      0.73      8942\n",
            "Science/Technology       0.66      0.18      0.28      1419\n",
            "            Sports       0.86      0.41      0.56       512\n",
            "     [R]eddiquette       0.42      0.02      0.03       974\n",
            "\n",
            "          accuracy                           0.60     31807\n",
            "         macro avg       0.64      0.37      0.41     31807\n",
            "      weighted avg       0.60      0.60      0.57     31807\n",
            "\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "bnDBtdGpwWlA",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "7a3e01dd-7110-494e-9f85-a250580ecd6d"
      },
      "source": [
        "gs_naive_bayes.best_params_"
      ],
      "execution_count": 21,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "{'clf__alpha': 0.1, 'tfidf__use_idf': False, 'vect__ngram_range': (1, 2)}"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 21
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Vj_0nnsa6FC3",
        "colab_type": "text"
      },
      "source": [
        "Next, we will try the [Complement Naive Bayes](https://scikit-learn.org/stable/modules/generated/sklearn.naive_bayes.ComplementNB.html#sklearn.naive_bayes.ComplementNB) algorithm. This is particularly used to deal with imbalanced datasets such as ours"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "m65frNSnwWl6",
        "colab_type": "code",
        "colab": {},
        "outputId": "cb1718a9-e9a9-4bfb-effc-8914841fbb56"
      },
      "source": [
        "c_naive_bayes = Pipeline([('vect', CountVectorizer(stop_words='english')),\n",
        "               ('tfidf', TfidfTransformer()),\n",
        "               ('clf', ComplementNB(fit_prior=False)),\n",
        "              ], verbose=True)\n",
        "\n",
        "parameters = {'vect__ngram_range': [(1, 1), (1, 2)],\n",
        "               'tfidf__use_idf': (True, False),\n",
        "               'clf__alpha': (1, 1e-1, 1e-2)}\n",
        "\n",
        "c_gs_naive_bayes = GridSearchCV(c_naive_bayes, parameters, n_jobs=-1, verbose=3)\n",
        "c_gs_naive_bayes = c_gs_naive_bayes.fit(X_train, y_train)\n",
        "\n",
        "y_pred = c_gs_naive_bayes.predict(X_test)\n",
        "print(classification_report(y_test, y_pred))"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Fitting 5 folds for each of 12 candidates, totalling 60 fits\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "[Parallel(n_jobs=-1)]: Using backend LokyBackend with 8 concurrent workers.\n",
            "[Parallel(n_jobs=-1)]: Done  16 tasks      | elapsed:   23.7s\n",
            "[Parallel(n_jobs=-1)]: Done  60 out of  60 | elapsed:  1.4min finished\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[Pipeline] .............. (step 1 of 3) Processing vect, total=   7.2s\n",
            "[Pipeline] ............. (step 2 of 3) Processing tfidf, total=   0.5s\n",
            "[Pipeline] ............... (step 3 of 3) Processing clf, total=   0.7s\n",
            "                    precision    recall  f1-score   support\n",
            "\n",
            "          AskIndia       0.58      0.60      0.59      3811\n",
            "  Business/Finance       0.59      0.16      0.26      1341\n",
            "       Coronavirus       0.71      0.25      0.37       320\n",
            "     Non-Political       0.56      0.69      0.62     10232\n",
            "       Photography       0.78      0.22      0.35       338\n",
            "    Policy/Economy       0.59      0.45      0.51      3838\n",
            "          Politics       0.67      0.81      0.73      9037\n",
            "Science/Technology       0.65      0.22      0.33      1363\n",
            "            Sports       0.78      0.57      0.66       521\n",
            "     [R]eddiquette       0.56      0.02      0.04      1006\n",
            "\n",
            "          accuracy                           0.61     31807\n",
            "         macro avg       0.65      0.40      0.45     31807\n",
            "      weighted avg       0.61      0.61      0.58     31807\n",
            "\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "W61v_kIwwWmA",
        "colab_type": "code",
        "colab": {},
        "outputId": "a1ac0c22-04f0-47d1-9895-2f13aaf2b16f"
      },
      "source": [
        "c_gs_naive_bayes.best_params_"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "{'clf__alpha': 1, 'tfidf__use_idf': True, 'vect__ngram_range': (1, 2)}"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 17
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ddj7uN9N7MNV",
        "colab_type": "text"
      },
      "source": [
        "Here, we will train our model on a Linear SVM which we converge using Stochastic Gradient Descent"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "954jeQKcwWlS",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "sgd = Pipeline([('vect', CountVectorizer(stop_words='english')),\n",
        "                ('tfidf', TfidfTransformer()),\n",
        "                ('clf', SGDClassifier(loss='hinge', penalty='l2', random_state=42, tol=None)),\n",
        "               ], verbose=3)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "q5_b7t4jwWlX",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 1000
        },
        "outputId": "955cf027-6666-4fd3-f492-226290ed1a0d"
      },
      "source": [
        "gs_sgd = GridSearchCV(sgd, parameters, n_jobs=1, verbose=3, cv=2)\n",
        "gs_sgd = gs_sgd.fit(X_train, y_train)"
      ],
      "execution_count": 24,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Fitting 2 folds for each of 12 candidates, totalling 24 fits\n",
            "[CV] clf__alpha=1, tfidf__use_idf=True, vect__ngram_range=(1, 1) .....\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "[Parallel(n_jobs=1)]: Using backend SequentialBackend with 1 concurrent workers.\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[Pipeline] .............. (step 1 of 3) Processing vect, total=   1.0s\n",
            "[Pipeline] ............. (step 2 of 3) Processing tfidf, total=   0.0s\n",
            "[Pipeline] ............... (step 3 of 3) Processing clf, total=  56.5s\n",
            "[CV]  clf__alpha=1, tfidf__use_idf=True, vect__ngram_range=(1, 1), score=0.293, total=  58.6s\n",
            "[CV] clf__alpha=1, tfidf__use_idf=True, vect__ngram_range=(1, 1) .....\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:   58.6s remaining:    0.0s\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[Pipeline] .............. (step 1 of 3) Processing vect, total=   1.0s\n",
            "[Pipeline] ............. (step 2 of 3) Processing tfidf, total=   0.0s\n",
            "[Pipeline] ............... (step 3 of 3) Processing clf, total=  54.7s\n",
            "[CV]  clf__alpha=1, tfidf__use_idf=True, vect__ngram_range=(1, 1), score=0.320, total=  56.9s\n",
            "[CV] clf__alpha=1, tfidf__use_idf=True, vect__ngram_range=(1, 2) .....\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:  1.9min remaining:    0.0s\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[Pipeline] .............. (step 1 of 3) Processing vect, total=   3.8s\n",
            "[Pipeline] ............. (step 2 of 3) Processing tfidf, total=   0.3s\n",
            "[Pipeline] ............... (step 3 of 3) Processing clf, total= 2.0min\n",
            "[CV]  clf__alpha=1, tfidf__use_idf=True, vect__ngram_range=(1, 2), score=0.286, total= 2.1min\n",
            "[CV] clf__alpha=1, tfidf__use_idf=True, vect__ngram_range=(1, 2) .....\n",
            "[Pipeline] .............. (step 1 of 3) Processing vect, total=   3.8s\n",
            "[Pipeline] ............. (step 2 of 3) Processing tfidf, total=   0.3s\n",
            "[Pipeline] ............... (step 3 of 3) Processing clf, total= 1.9min\n",
            "[CV]  clf__alpha=1, tfidf__use_idf=True, vect__ngram_range=(1, 2), score=0.320, total= 2.0min\n",
            "[CV] clf__alpha=1, tfidf__use_idf=False, vect__ngram_range=(1, 1) ....\n",
            "[Pipeline] .............. (step 1 of 3) Processing vect, total=   1.0s\n",
            "[Pipeline] ............. (step 2 of 3) Processing tfidf, total=   0.0s\n",
            "[Pipeline] ............... (step 3 of 3) Processing clf, total=  53.2s\n",
            "[CV]  clf__alpha=1, tfidf__use_idf=False, vect__ngram_range=(1, 1), score=0.332, total=  55.3s\n",
            "[CV] clf__alpha=1, tfidf__use_idf=False, vect__ngram_range=(1, 1) ....\n",
            "[Pipeline] .............. (step 1 of 3) Processing vect, total=   1.0s\n",
            "[Pipeline] ............. (step 2 of 3) Processing tfidf, total=   0.0s\n",
            "[Pipeline] ............... (step 3 of 3) Processing clf, total=  56.9s\n",
            "[CV]  clf__alpha=1, tfidf__use_idf=False, vect__ngram_range=(1, 1), score=0.321, total=  59.0s\n",
            "[CV] clf__alpha=1, tfidf__use_idf=False, vect__ngram_range=(1, 2) ....\n",
            "[Pipeline] .............. (step 1 of 3) Processing vect, total=   3.7s\n",
            "[Pipeline] ............. (step 2 of 3) Processing tfidf, total=   0.1s\n",
            "[Pipeline] ............... (step 3 of 3) Processing clf, total= 2.1min\n",
            "[CV]  clf__alpha=1, tfidf__use_idf=False, vect__ngram_range=(1, 2), score=0.315, total= 2.2min\n",
            "[CV] clf__alpha=1, tfidf__use_idf=False, vect__ngram_range=(1, 2) ....\n",
            "[Pipeline] .............. (step 1 of 3) Processing vect, total=   3.9s\n",
            "[Pipeline] ............. (step 2 of 3) Processing tfidf, total=   0.1s\n",
            "[Pipeline] ............... (step 3 of 3) Processing clf, total= 2.1min\n",
            "[CV]  clf__alpha=1, tfidf__use_idf=False, vect__ngram_range=(1, 2), score=0.320, total= 2.2min\n",
            "[CV] clf__alpha=0.1, tfidf__use_idf=True, vect__ngram_range=(1, 1) ...\n",
            "[Pipeline] .............. (step 1 of 3) Processing vect, total=   1.1s\n",
            "[Pipeline] ............. (step 2 of 3) Processing tfidf, total=   0.0s\n",
            "[Pipeline] ............... (step 3 of 3) Processing clf, total=  48.2s\n",
            "[CV]  clf__alpha=0.1, tfidf__use_idf=True, vect__ngram_range=(1, 1), score=0.327, total=  50.4s\n",
            "[CV] clf__alpha=0.1, tfidf__use_idf=True, vect__ngram_range=(1, 1) ...\n",
            "[Pipeline] .............. (step 1 of 3) Processing vect, total=   1.0s\n",
            "[Pipeline] ............. (step 2 of 3) Processing tfidf, total=   0.0s\n",
            "[Pipeline] ............... (step 3 of 3) Processing clf, total=  47.7s\n",
            "[CV]  clf__alpha=0.1, tfidf__use_idf=True, vect__ngram_range=(1, 1), score=0.320, total=  49.8s\n",
            "[CV] clf__alpha=0.1, tfidf__use_idf=True, vect__ngram_range=(1, 2) ...\n",
            "[Pipeline] .............. (step 1 of 3) Processing vect, total=   3.8s\n",
            "[Pipeline] ............. (step 2 of 3) Processing tfidf, total=   0.3s\n",
            "[Pipeline] ............... (step 3 of 3) Processing clf, total= 1.7min\n",
            "[CV]  clf__alpha=0.1, tfidf__use_idf=True, vect__ngram_range=(1, 2), score=0.320, total= 1.8min\n",
            "[CV] clf__alpha=0.1, tfidf__use_idf=True, vect__ngram_range=(1, 2) ...\n",
            "[Pipeline] .............. (step 1 of 3) Processing vect, total=   3.9s\n",
            "[Pipeline] ............. (step 2 of 3) Processing tfidf, total=   0.3s\n",
            "[Pipeline] ............... (step 3 of 3) Processing clf, total= 1.7min\n",
            "[CV]  clf__alpha=0.1, tfidf__use_idf=True, vect__ngram_range=(1, 2), score=0.320, total= 1.8min\n",
            "[CV] clf__alpha=0.1, tfidf__use_idf=False, vect__ngram_range=(1, 1) ..\n",
            "[Pipeline] .............. (step 1 of 3) Processing vect, total=   1.1s\n",
            "[Pipeline] ............. (step 2 of 3) Processing tfidf, total=   0.0s\n",
            "[Pipeline] ............... (step 3 of 3) Processing clf, total=  49.0s\n",
            "[CV]  clf__alpha=0.1, tfidf__use_idf=False, vect__ngram_range=(1, 1), score=0.374, total=  51.1s\n",
            "[CV] clf__alpha=0.1, tfidf__use_idf=False, vect__ngram_range=(1, 1) ..\n",
            "[Pipeline] .............. (step 1 of 3) Processing vect, total=   1.0s\n",
            "[Pipeline] ............. (step 2 of 3) Processing tfidf, total=   0.0s\n",
            "[Pipeline] ............... (step 3 of 3) Processing clf, total=  49.3s\n",
            "[CV]  clf__alpha=0.1, tfidf__use_idf=False, vect__ngram_range=(1, 1), score=0.324, total=  51.4s\n",
            "[CV] clf__alpha=0.1, tfidf__use_idf=False, vect__ngram_range=(1, 2) ..\n",
            "[Pipeline] .............. (step 1 of 3) Processing vect, total=   3.8s\n",
            "[Pipeline] ............. (step 2 of 3) Processing tfidf, total=   0.1s\n",
            "[Pipeline] ............... (step 3 of 3) Processing clf, total= 1.8min\n",
            "[CV]  clf__alpha=0.1, tfidf__use_idf=False, vect__ngram_range=(1, 2), score=0.347, total= 1.9min\n",
            "[CV] clf__alpha=0.1, tfidf__use_idf=False, vect__ngram_range=(1, 2) ..\n",
            "[Pipeline] .............. (step 1 of 3) Processing vect, total=   3.8s\n",
            "[Pipeline] ............. (step 2 of 3) Processing tfidf, total=   0.1s\n",
            "[Pipeline] ............... (step 3 of 3) Processing clf, total= 1.8min\n",
            "[CV]  clf__alpha=0.1, tfidf__use_idf=False, vect__ngram_range=(1, 2), score=0.321, total= 1.9min\n",
            "[CV] clf__alpha=0.01, tfidf__use_idf=True, vect__ngram_range=(1, 1) ..\n",
            "[Pipeline] .............. (step 1 of 3) Processing vect, total=   1.0s\n",
            "[Pipeline] ............. (step 2 of 3) Processing tfidf, total=   0.0s\n",
            "[Pipeline] ............... (step 3 of 3) Processing clf, total=  46.7s\n",
            "[CV]  clf__alpha=0.01, tfidf__use_idf=True, vect__ngram_range=(1, 1), score=0.587, total=  48.8s\n",
            "[CV] clf__alpha=0.01, tfidf__use_idf=True, vect__ngram_range=(1, 1) ..\n",
            "[Pipeline] .............. (step 1 of 3) Processing vect, total=   1.0s\n",
            "[Pipeline] ............. (step 2 of 3) Processing tfidf, total=   0.0s\n",
            "[Pipeline] ............... (step 3 of 3) Processing clf, total=  47.0s\n",
            "[CV]  clf__alpha=0.01, tfidf__use_idf=True, vect__ngram_range=(1, 1), score=0.589, total=  49.2s\n",
            "[CV] clf__alpha=0.01, tfidf__use_idf=True, vect__ngram_range=(1, 2) ..\n",
            "[Pipeline] .............. (step 1 of 3) Processing vect, total=   3.7s\n",
            "[Pipeline] ............. (step 2 of 3) Processing tfidf, total=   0.3s\n",
            "[Pipeline] ............... (step 3 of 3) Processing clf, total= 1.6min\n",
            "[CV]  clf__alpha=0.01, tfidf__use_idf=True, vect__ngram_range=(1, 2), score=0.598, total= 1.7min\n",
            "[CV] clf__alpha=0.01, tfidf__use_idf=True, vect__ngram_range=(1, 2) ..\n",
            "[Pipeline] .............. (step 1 of 3) Processing vect, total=   3.9s\n",
            "[Pipeline] ............. (step 2 of 3) Processing tfidf, total=   0.3s\n",
            "[Pipeline] ............... (step 3 of 3) Processing clf, total= 1.7min\n",
            "[CV]  clf__alpha=0.01, tfidf__use_idf=True, vect__ngram_range=(1, 2), score=0.601, total= 1.8min\n",
            "[CV] clf__alpha=0.01, tfidf__use_idf=False, vect__ngram_range=(1, 1) .\n",
            "[Pipeline] .............. (step 1 of 3) Processing vect, total=   1.1s\n",
            "[Pipeline] ............. (step 2 of 3) Processing tfidf, total=   0.0s\n",
            "[Pipeline] ............... (step 3 of 3) Processing clf, total=  49.1s\n",
            "[CV]  clf__alpha=0.01, tfidf__use_idf=False, vect__ngram_range=(1, 1), score=0.579, total=  51.2s\n",
            "[CV] clf__alpha=0.01, tfidf__use_idf=False, vect__ngram_range=(1, 1) .\n",
            "[Pipeline] .............. (step 1 of 3) Processing vect, total=   1.0s\n",
            "[Pipeline] ............. (step 2 of 3) Processing tfidf, total=   0.0s\n",
            "[Pipeline] ............... (step 3 of 3) Processing clf, total=  47.7s\n",
            "[CV]  clf__alpha=0.01, tfidf__use_idf=False, vect__ngram_range=(1, 1), score=0.582, total=  49.8s\n",
            "[CV] clf__alpha=0.01, tfidf__use_idf=False, vect__ngram_range=(1, 2) .\n",
            "[Pipeline] .............. (step 1 of 3) Processing vect, total=   3.8s\n",
            "[Pipeline] ............. (step 2 of 3) Processing tfidf, total=   0.1s\n",
            "[Pipeline] ............... (step 3 of 3) Processing clf, total= 1.8min\n",
            "[CV]  clf__alpha=0.01, tfidf__use_idf=False, vect__ngram_range=(1, 2), score=0.587, total= 1.9min\n",
            "[CV] clf__alpha=0.01, tfidf__use_idf=False, vect__ngram_range=(1, 2) .\n",
            "[Pipeline] .............. (step 1 of 3) Processing vect, total=   3.9s\n",
            "[Pipeline] ............. (step 2 of 3) Processing tfidf, total=   0.1s\n",
            "[Pipeline] ............... (step 3 of 3) Processing clf, total= 1.8min\n",
            "[CV]  clf__alpha=0.01, tfidf__use_idf=False, vect__ngram_range=(1, 2), score=0.589, total= 1.9min\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "[Parallel(n_jobs=1)]: Done  24 out of  24 | elapsed: 33.9min finished\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[Pipeline] .............. (step 1 of 3) Processing vect, total=   7.6s\n",
            "[Pipeline] ............. (step 2 of 3) Processing tfidf, total=   0.8s\n",
            "[Pipeline] ............... (step 3 of 3) Processing clf, total= 4.3min\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "rsZsTxHowWlc",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 330
        },
        "outputId": "781b4f20-69fc-4969-c752-6ca622a07bf3"
      },
      "source": [
        "y_pred = gs_sgd.predict(X_test)\n",
        "print(classification_report(y_test, y_pred))\n",
        "print(gs_sgd.best_params_)"
      ],
      "execution_count": 25,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "                    precision    recall  f1-score   support\n",
            "\n",
            "          AskIndia       0.58      0.67      0.62      3776\n",
            "  Business/Finance       0.54      0.20      0.30      1368\n",
            "       Coronavirus       0.68      0.40      0.50       324\n",
            "     Non-Political       0.61      0.63      0.62     10260\n",
            "       Photography       0.74      0.18      0.29       333\n",
            "    Policy/Economy       0.58      0.46      0.51      3899\n",
            "          Politics       0.62      0.83      0.71      8942\n",
            "Science/Technology       0.57      0.26      0.36      1419\n",
            "            Sports       0.75      0.57      0.65       512\n",
            "     [R]eddiquette       0.71      0.02      0.03       974\n",
            "\n",
            "          accuracy                           0.61     31807\n",
            "         macro avg       0.64      0.42      0.46     31807\n",
            "      weighted avg       0.61      0.61      0.58     31807\n",
            "\n",
            "{'clf__alpha': 0.01, 'tfidf__use_idf': True, 'vect__ngram_range': (1, 2)}\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "TYrvWgmAwWll",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "logreg = Pipeline([('vect', CountVectorizer(stop_words='english', ngram_range=(1,2))),\n",
        "                ('tfidf', TfidfTransformer()),\n",
        "                ('clf', LogisticRegression(solver=\"saga\", penalty='l2',verbose=1)),\n",
        "               ], verbose=5)\n",
        "\n",
        "parameters = {'clf__C': np.logspace(0,4,10)}"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "D0SyAFAmwWlr",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 1000
        },
        "outputId": "ff0d08f6-ccf7-442f-9fca-71e2e570c6b5"
      },
      "source": [
        "gs_logreg = GridSearchCV(logreg, parameters, verbose=3, cv=2, n_jobs=1)\n",
        "gs_logreg = gs_logreg.fit(X_train, y_train)\n",
        "y_pred = gs_logreg.predict(X_test)\n",
        "print(classification_report(y_test, y_pred))\n",
        "print(gs_logreg.best_params_)"
      ],
      "execution_count": 37,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Fitting 2 folds for each of 10 candidates, totalling 20 fits\n",
            "[CV] clf__C=1.0 ......................................................\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "[Parallel(n_jobs=1)]: Using backend SequentialBackend with 1 concurrent workers.\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[Pipeline] .............. (step 1 of 3) Processing vect, total=   3.9s\n",
            "[Pipeline] ............. (step 2 of 3) Processing tfidf, total=   0.3s\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "[Parallel(n_jobs=1)]: Using backend SequentialBackend with 1 concurrent workers.\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "convergence after 22 epochs took 5 seconds\n",
            "[Pipeline] ............... (step 3 of 3) Processing clf, total=   5.2s\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    5.2s finished\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[CV] .......................... clf__C=1.0, score=0.599, total=  11.3s\n",
            "[CV] clf__C=1.0 ......................................................\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:   11.3s remaining:    0.0s\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[Pipeline] .............. (step 1 of 3) Processing vect, total=   3.9s\n",
            "[Pipeline] ............. (step 2 of 3) Processing tfidf, total=   0.4s\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "[Parallel(n_jobs=1)]: Using backend SequentialBackend with 1 concurrent workers.\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "convergence after 23 epochs took 5 seconds\n",
            "[Pipeline] ............... (step 3 of 3) Processing clf, total=   5.5s\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    5.5s finished\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[CV] .......................... clf__C=1.0, score=0.602, total=  11.7s\n",
            "[CV] clf__C=2.7825594022071245 .......................................\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:   23.1s remaining:    0.0s\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[Pipeline] .............. (step 1 of 3) Processing vect, total=   3.8s\n",
            "[Pipeline] ............. (step 2 of 3) Processing tfidf, total=   0.3s\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "[Parallel(n_jobs=1)]: Using backend SequentialBackend with 1 concurrent workers.\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "convergence after 27 epochs took 6 seconds\n",
            "[Pipeline] ............... (step 3 of 3) Processing clf, total=   6.2s\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    6.2s finished\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[CV] ........... clf__C=2.7825594022071245, score=0.607, total=  12.2s\n",
            "[CV] clf__C=2.7825594022071245 .......................................\n",
            "[Pipeline] .............. (step 1 of 3) Processing vect, total=   3.9s\n",
            "[Pipeline] ............. (step 2 of 3) Processing tfidf, total=   0.3s\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "[Parallel(n_jobs=1)]: Using backend SequentialBackend with 1 concurrent workers.\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "convergence after 29 epochs took 6 seconds\n",
            "[Pipeline] ............... (step 3 of 3) Processing clf, total=   6.8s\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    6.7s finished\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[CV] ........... clf__C=2.7825594022071245, score=0.609, total=  12.9s\n",
            "[CV] clf__C=7.742636826811269 ........................................\n",
            "[Pipeline] .............. (step 1 of 3) Processing vect, total=   3.8s\n",
            "[Pipeline] ............. (step 2 of 3) Processing tfidf, total=   0.3s\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "[Parallel(n_jobs=1)]: Using backend SequentialBackend with 1 concurrent workers.\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "convergence after 44 epochs took 10 seconds\n",
            "[Pipeline] ............... (step 3 of 3) Processing clf, total=  10.1s\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:   10.1s finished\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[CV] ............ clf__C=7.742636826811269, score=0.608, total=  16.1s\n",
            "[CV] clf__C=7.742636826811269 ........................................\n",
            "[Pipeline] .............. (step 1 of 3) Processing vect, total=   3.8s\n",
            "[Pipeline] ............. (step 2 of 3) Processing tfidf, total=   0.3s\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "[Parallel(n_jobs=1)]: Using backend SequentialBackend with 1 concurrent workers.\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "convergence after 48 epochs took 11 seconds\n",
            "[Pipeline] ............... (step 3 of 3) Processing clf, total=  11.1s\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:   11.1s finished\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[CV] ............ clf__C=7.742636826811269, score=0.608, total=  17.1s\n",
            "[CV] clf__C=21.544346900318832 .......................................\n",
            "[Pipeline] .............. (step 1 of 3) Processing vect, total=   3.8s\n",
            "[Pipeline] ............. (step 2 of 3) Processing tfidf, total=   0.3s\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "[Parallel(n_jobs=1)]: Using backend SequentialBackend with 1 concurrent workers.\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "convergence after 87 epochs took 20 seconds\n",
            "[Pipeline] ............... (step 3 of 3) Processing clf, total=  19.8s\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:   19.8s finished\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[CV] ........... clf__C=21.544346900318832, score=0.606, total=  25.9s\n",
            "[CV] clf__C=21.544346900318832 .......................................\n",
            "[Pipeline] .............. (step 1 of 3) Processing vect, total=   3.9s\n",
            "[Pipeline] ............. (step 2 of 3) Processing tfidf, total=   0.3s\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "[Parallel(n_jobs=1)]: Using backend SequentialBackend with 1 concurrent workers.\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "convergence after 95 epochs took 22 seconds\n",
            "[Pipeline] ............... (step 3 of 3) Processing clf, total=  22.0s\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:   22.0s finished\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[CV] ........... clf__C=21.544346900318832, score=0.603, total=  28.1s\n",
            "[CV] clf__C=59.94842503189409 ........................................\n",
            "[Pipeline] .............. (step 1 of 3) Processing vect, total=   3.9s\n",
            "[Pipeline] ............. (step 2 of 3) Processing tfidf, total=   0.3s\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "[Parallel(n_jobs=1)]: Using backend SequentialBackend with 1 concurrent workers.\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "max_iter reached after 23 seconds\n",
            "[Pipeline] ............... (step 3 of 3) Processing clf, total=  23.3s\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.6/dist-packages/sklearn/linear_model/_sag.py:330: ConvergenceWarning: The max_iter was reached which means the coef_ did not converge\n",
            "  \"the coef_ did not converge\", ConvergenceWarning)\n",
            "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:   23.3s finished\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[CV] ............ clf__C=59.94842503189409, score=0.603, total=  29.5s\n",
            "[CV] clf__C=59.94842503189409 ........................................\n",
            "[Pipeline] .............. (step 1 of 3) Processing vect, total=   3.9s\n",
            "[Pipeline] ............. (step 2 of 3) Processing tfidf, total=   0.4s\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "[Parallel(n_jobs=1)]: Using backend SequentialBackend with 1 concurrent workers.\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "max_iter reached after 23 seconds\n",
            "[Pipeline] ............... (step 3 of 3) Processing clf, total=  23.5s\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.6/dist-packages/sklearn/linear_model/_sag.py:330: ConvergenceWarning: The max_iter was reached which means the coef_ did not converge\n",
            "  \"the coef_ did not converge\", ConvergenceWarning)\n",
            "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:   23.4s finished\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[CV] ............ clf__C=59.94842503189409, score=0.601, total=  29.6s\n",
            "[CV] clf__C=166.81005372000593 .......................................\n",
            "[Pipeline] .............. (step 1 of 3) Processing vect, total=   4.0s\n",
            "[Pipeline] ............. (step 2 of 3) Processing tfidf, total=   0.4s\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "[Parallel(n_jobs=1)]: Using backend SequentialBackend with 1 concurrent workers.\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "max_iter reached after 24 seconds\n",
            "[Pipeline] ............... (step 3 of 3) Processing clf, total=  24.6s\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.6/dist-packages/sklearn/linear_model/_sag.py:330: ConvergenceWarning: The max_iter was reached which means the coef_ did not converge\n",
            "  \"the coef_ did not converge\", ConvergenceWarning)\n",
            "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:   24.6s finished\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[CV] ........... clf__C=166.81005372000593, score=0.601, total=  30.8s\n",
            "[CV] clf__C=166.81005372000593 .......................................\n",
            "[Pipeline] .............. (step 1 of 3) Processing vect, total=   4.0s\n",
            "[Pipeline] ............. (step 2 of 3) Processing tfidf, total=   0.3s\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "[Parallel(n_jobs=1)]: Using backend SequentialBackend with 1 concurrent workers.\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "max_iter reached after 23 seconds\n",
            "[Pipeline] ............... (step 3 of 3) Processing clf, total=  23.1s\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.6/dist-packages/sklearn/linear_model/_sag.py:330: ConvergenceWarning: The max_iter was reached which means the coef_ did not converge\n",
            "  \"the coef_ did not converge\", ConvergenceWarning)\n",
            "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:   23.1s finished\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[CV] ........... clf__C=166.81005372000593, score=0.598, total=  29.3s\n",
            "[CV] clf__C=464.15888336127773 .......................................\n",
            "[Pipeline] .............. (step 1 of 3) Processing vect, total=   3.9s\n",
            "[Pipeline] ............. (step 2 of 3) Processing tfidf, total=   0.3s\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "[Parallel(n_jobs=1)]: Using backend SequentialBackend with 1 concurrent workers.\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "max_iter reached after 23 seconds\n",
            "[Pipeline] ............... (step 3 of 3) Processing clf, total=  23.2s\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.6/dist-packages/sklearn/linear_model/_sag.py:330: ConvergenceWarning: The max_iter was reached which means the coef_ did not converge\n",
            "  \"the coef_ did not converge\", ConvergenceWarning)\n",
            "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:   23.2s finished\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[CV] ........... clf__C=464.15888336127773, score=0.600, total=  29.3s\n",
            "[CV] clf__C=464.15888336127773 .......................................\n",
            "[Pipeline] .............. (step 1 of 3) Processing vect, total=   3.9s\n",
            "[Pipeline] ............. (step 2 of 3) Processing tfidf, total=   0.3s\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "[Parallel(n_jobs=1)]: Using backend SequentialBackend with 1 concurrent workers.\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "max_iter reached after 23 seconds\n",
            "[Pipeline] ............... (step 3 of 3) Processing clf, total=  23.0s\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.6/dist-packages/sklearn/linear_model/_sag.py:330: ConvergenceWarning: The max_iter was reached which means the coef_ did not converge\n",
            "  \"the coef_ did not converge\", ConvergenceWarning)\n",
            "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:   22.9s finished\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[CV] ........... clf__C=464.15888336127773, score=0.596, total=  29.0s\n",
            "[CV] clf__C=1291.5496650148827 .......................................\n",
            "[Pipeline] .............. (step 1 of 3) Processing vect, total=   3.8s\n",
            "[Pipeline] ............. (step 2 of 3) Processing tfidf, total=   0.3s\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "[Parallel(n_jobs=1)]: Using backend SequentialBackend with 1 concurrent workers.\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "max_iter reached after 23 seconds\n",
            "[Pipeline] ............... (step 3 of 3) Processing clf, total=  23.0s\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.6/dist-packages/sklearn/linear_model/_sag.py:330: ConvergenceWarning: The max_iter was reached which means the coef_ did not converge\n",
            "  \"the coef_ did not converge\", ConvergenceWarning)\n",
            "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:   22.9s finished\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[CV] ........... clf__C=1291.5496650148827, score=0.599, total=  28.9s\n",
            "[CV] clf__C=1291.5496650148827 .......................................\n",
            "[Pipeline] .............. (step 1 of 3) Processing vect, total=   4.7s\n",
            "[Pipeline] ............. (step 2 of 3) Processing tfidf, total=   0.3s\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "[Parallel(n_jobs=1)]: Using backend SequentialBackend with 1 concurrent workers.\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "max_iter reached after 23 seconds\n",
            "[Pipeline] ............... (step 3 of 3) Processing clf, total=  22.9s\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.6/dist-packages/sklearn/linear_model/_sag.py:330: ConvergenceWarning: The max_iter was reached which means the coef_ did not converge\n",
            "  \"the coef_ did not converge\", ConvergenceWarning)\n",
            "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:   22.8s finished\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[CV] ........... clf__C=1291.5496650148827, score=0.597, total=  29.7s\n",
            "[CV] clf__C=3593.813663804626 ........................................\n",
            "[Pipeline] .............. (step 1 of 3) Processing vect, total=   3.8s\n",
            "[Pipeline] ............. (step 2 of 3) Processing tfidf, total=   0.3s\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "[Parallel(n_jobs=1)]: Using backend SequentialBackend with 1 concurrent workers.\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "max_iter reached after 22 seconds\n",
            "[Pipeline] ............... (step 3 of 3) Processing clf, total=  22.7s\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.6/dist-packages/sklearn/linear_model/_sag.py:330: ConvergenceWarning: The max_iter was reached which means the coef_ did not converge\n",
            "  \"the coef_ did not converge\", ConvergenceWarning)\n",
            "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:   22.7s finished\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[CV] ............ clf__C=3593.813663804626, score=0.598, total=  28.7s\n",
            "[CV] clf__C=3593.813663804626 ........................................\n",
            "[Pipeline] .............. (step 1 of 3) Processing vect, total=   3.8s\n",
            "[Pipeline] ............. (step 2 of 3) Processing tfidf, total=   0.3s\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "[Parallel(n_jobs=1)]: Using backend SequentialBackend with 1 concurrent workers.\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "max_iter reached after 23 seconds\n",
            "[Pipeline] ............... (step 3 of 3) Processing clf, total=  22.8s\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.6/dist-packages/sklearn/linear_model/_sag.py:330: ConvergenceWarning: The max_iter was reached which means the coef_ did not converge\n",
            "  \"the coef_ did not converge\", ConvergenceWarning)\n",
            "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:   22.8s finished\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[CV] ............ clf__C=3593.813663804626, score=0.596, total=  28.8s\n",
            "[CV] clf__C=10000.0 ..................................................\n",
            "[Pipeline] .............. (step 1 of 3) Processing vect, total=   3.8s\n",
            "[Pipeline] ............. (step 2 of 3) Processing tfidf, total=   0.3s\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "[Parallel(n_jobs=1)]: Using backend SequentialBackend with 1 concurrent workers.\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "max_iter reached after 23 seconds\n",
            "[Pipeline] ............... (step 3 of 3) Processing clf, total=  22.9s\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.6/dist-packages/sklearn/linear_model/_sag.py:330: ConvergenceWarning: The max_iter was reached which means the coef_ did not converge\n",
            "  \"the coef_ did not converge\", ConvergenceWarning)\n",
            "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:   22.9s finished\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[CV] ...................... clf__C=10000.0, score=0.599, total=  28.9s\n",
            "[CV] clf__C=10000.0 ..................................................\n",
            "[Pipeline] .............. (step 1 of 3) Processing vect, total=   3.9s\n",
            "[Pipeline] ............. (step 2 of 3) Processing tfidf, total=   0.3s\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "[Parallel(n_jobs=1)]: Using backend SequentialBackend with 1 concurrent workers.\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "max_iter reached after 23 seconds\n",
            "[Pipeline] ............... (step 3 of 3) Processing clf, total=  23.2s\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.6/dist-packages/sklearn/linear_model/_sag.py:330: ConvergenceWarning: The max_iter was reached which means the coef_ did not converge\n",
            "  \"the coef_ did not converge\", ConvergenceWarning)\n",
            "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:   23.2s finished\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[CV] ...................... clf__C=10000.0, score=0.595, total=  29.3s\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "[Parallel(n_jobs=1)]: Done  20 out of  20 | elapsed:  8.1min finished\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[Pipeline] .............. (step 1 of 3) Processing vect, total=   8.0s\n",
            "[Pipeline] ............. (step 2 of 3) Processing tfidf, total=   0.7s\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "[Parallel(n_jobs=1)]: Using backend SequentialBackend with 1 concurrent workers.\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "convergence after 26 epochs took 13 seconds\n",
            "[Pipeline] ............... (step 3 of 3) Processing clf, total=  13.2s\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:   13.2s finished\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "                    precision    recall  f1-score   support\n",
            "\n",
            "          AskIndia       0.61      0.63      0.62      3776\n",
            "  Business/Finance       0.47      0.27      0.34      1368\n",
            "       Coronavirus       0.70      0.43      0.53       324\n",
            "     Non-Political       0.58      0.70      0.63     10260\n",
            "       Photography       0.78      0.30      0.43       333\n",
            "    Policy/Economy       0.57      0.55      0.56      3899\n",
            "          Politics       0.71      0.74      0.72      8942\n",
            "Science/Technology       0.53      0.33      0.40      1419\n",
            "            Sports       0.80      0.57      0.67       512\n",
            "     [R]eddiquette       0.51      0.03      0.06       974\n",
            "\n",
            "          accuracy                           0.62     31807\n",
            "         macro avg       0.63      0.45      0.50     31807\n",
            "      weighted avg       0.62      0.62      0.61     31807\n",
            "\n",
            "{'clf__C': 2.7825594022071245}\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "GgeyFLlF9zm8",
        "colab_type": "text"
      },
      "source": [
        "We will now try the Random Forest Classifier method"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "zYiX3uHVwWlx",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "rfc = Pipeline([('vect', CountVectorizer(stop_words='english')),\n",
        "                ('tfidf', TfidfTransformer()),\n",
        "                ('clf', RandomForestClassifier(n_estimators=100, max_depth=160, verbose=5)),\n",
        "               ], verbose=2)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "5AzoUeL9wWl2",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 1000
        },
        "outputId": "a56e3ad3-a44a-4152-97da-746e177b1939"
      },
      "source": [
        "rfc.fit(X_train, y_train)\n",
        "y_pred = rfc.predict(X_test)\n",
        "print(classification_report(y_test, y_pred))"
      ],
      "execution_count": 58,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "[Pipeline] .............. (step 1 of 3) Processing vect, total=   2.1s\n",
            "[Pipeline] ............. (step 2 of 3) Processing tfidf, total=   0.1s\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "[Parallel(n_jobs=1)]: Using backend SequentialBackend with 1 concurrent workers.\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "building tree 1 of 100\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    2.6s remaining:    0.0s\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "building tree 2 of 100\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    4.5s remaining:    0.0s\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "building tree 3 of 100\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "[Parallel(n_jobs=1)]: Done   3 out of   3 | elapsed:    6.6s remaining:    0.0s\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "building tree 4 of 100\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "[Parallel(n_jobs=1)]: Done   4 out of   4 | elapsed:    8.8s remaining:    0.0s\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "building tree 5 of 100\n",
            "building tree 6 of 100\n",
            "building tree 7 of 100\n",
            "building tree 8 of 100\n",
            "building tree 9 of 100\n",
            "building tree 10 of 100\n",
            "building tree 11 of 100\n",
            "building tree 12 of 100\n",
            "building tree 13 of 100\n",
            "building tree 14 of 100\n",
            "building tree 15 of 100\n",
            "building tree 16 of 100\n",
            "building tree 17 of 100\n",
            "building tree 18 of 100\n",
            "building tree 19 of 100\n",
            "building tree 20 of 100\n",
            "building tree 21 of 100\n",
            "building tree 22 of 100\n",
            "building tree 23 of 100\n",
            "building tree 24 of 100\n",
            "building tree 25 of 100\n",
            "building tree 26 of 100\n",
            "building tree 27 of 100\n",
            "building tree 28 of 100\n",
            "building tree 29 of 100\n",
            "building tree 30 of 100\n",
            "building tree 31 of 100\n",
            "building tree 32 of 100\n",
            "building tree 33 of 100\n",
            "building tree 34 of 100\n",
            "building tree 35 of 100\n",
            "building tree 36 of 100\n",
            "building tree 37 of 100\n",
            "building tree 38 of 100\n",
            "building tree 39 of 100\n",
            "building tree 40 of 100\n",
            "building tree 41 of 100\n",
            "building tree 42 of 100\n",
            "building tree 43 of 100\n",
            "building tree 44 of 100\n",
            "building tree 45 of 100\n",
            "building tree 46 of 100\n",
            "building tree 47 of 100\n",
            "building tree 48 of 100\n",
            "building tree 49 of 100\n",
            "building tree 50 of 100\n",
            "building tree 51 of 100\n",
            "building tree 52 of 100\n",
            "building tree 53 of 100\n",
            "building tree 54 of 100\n",
            "building tree 55 of 100\n",
            "building tree 56 of 100\n",
            "building tree 57 of 100\n",
            "building tree 58 of 100\n",
            "building tree 59 of 100\n",
            "building tree 60 of 100\n",
            "building tree 61 of 100\n",
            "building tree 62 of 100\n",
            "building tree 63 of 100\n",
            "building tree 64 of 100\n",
            "building tree 65 of 100\n",
            "building tree 66 of 100\n",
            "building tree 67 of 100\n",
            "building tree 68 of 100\n",
            "building tree 69 of 100\n",
            "building tree 70 of 100\n",
            "building tree 71 of 100\n",
            "building tree 72 of 100\n",
            "building tree 73 of 100\n",
            "building tree 74 of 100\n",
            "building tree 75 of 100\n",
            "building tree 76 of 100\n",
            "building tree 77 of 100\n",
            "building tree 78 of 100\n",
            "building tree 79 of 100\n",
            "building tree 80 of 100\n",
            "building tree 81 of 100\n",
            "building tree 82 of 100\n",
            "building tree 83 of 100\n",
            "building tree 84 of 100\n",
            "building tree 85 of 100\n",
            "building tree 86 of 100\n",
            "building tree 87 of 100\n",
            "building tree 88 of 100\n",
            "building tree 89 of 100\n",
            "building tree 90 of 100\n",
            "building tree 91 of 100\n",
            "building tree 92 of 100\n",
            "building tree 93 of 100\n",
            "building tree 94 of 100\n",
            "building tree 95 of 100\n",
            "building tree 96 of 100\n",
            "building tree 97 of 100\n",
            "building tree 98 of 100\n",
            "building tree 99 of 100\n",
            "building tree 100 of 100\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "[Parallel(n_jobs=1)]: Done 100 out of 100 | elapsed:  3.6min finished\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[Pipeline] ............... (step 3 of 3) Processing clf, total= 3.6min\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "[Parallel(n_jobs=1)]: Using backend SequentialBackend with 1 concurrent workers.\n",
            "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
            "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s\n",
            "[Parallel(n_jobs=1)]: Done   3 out of   3 | elapsed:    0.1s remaining:    0.0s\n",
            "[Parallel(n_jobs=1)]: Done   4 out of   4 | elapsed:    0.1s remaining:    0.0s\n",
            "[Parallel(n_jobs=1)]: Done 100 out of 100 | elapsed:    2.3s finished\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "                    precision    recall  f1-score   support\n",
            "\n",
            "          AskIndia       0.60      0.56      0.58      3776\n",
            "  Business/Finance       0.78      0.04      0.08      1368\n",
            "       Coronavirus       0.77      0.22      0.34       324\n",
            "     Non-Political       0.48      0.81      0.60     10260\n",
            "       Photography       0.95      0.06      0.11       333\n",
            "    Policy/Economy       0.65      0.35      0.46      3899\n",
            "          Politics       0.71      0.66      0.69      8942\n",
            "Science/Technology       0.71      0.08      0.14      1419\n",
            "            Sports       0.87      0.27      0.41       512\n",
            "     [R]eddiquette       0.93      0.01      0.03       974\n",
            "\n",
            "          accuracy                           0.57     31807\n",
            "         macro avg       0.75      0.31      0.34     31807\n",
            "      weighted avg       0.63      0.57      0.53     31807\n",
            "\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "mA4ze8zADX38",
        "colab_type": "text"
      },
      "source": [
        "The NearestCentroid method is the last algorithm we will be implementing from the traditional ML methods"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Ay6Lpdl6wWmF",
        "colab_type": "code",
        "colab": {},
        "outputId": "78034121-84f0-45b7-ca55-289dc87df339"
      },
      "source": [
        "from sklearn.neighbors.nearest_centroid import NearestCentroid\n",
        "nrc = Pipeline([('vect', CountVectorizer()),\n",
        "                     ('tfidf', TfidfTransformer()),\n",
        "                     ('clf', NearestCentroid()),\n",
        "                     ])\n",
        "nrc.fit(X_train, y_train)\n",
        "y_pred = nrc.predict(X_test)\n",
        "print(classification_report(y_test, y_pred))"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.6/dist-packages/sklearn/utils/deprecation.py:144: FutureWarning: The sklearn.neighbors.nearest_centroid module is  deprecated in version 0.22 and will be removed in version 0.24. The corresponding classes / functions should instead be imported from sklearn.neighbors. Anything that cannot be imported from sklearn.neighbors is now part of the private API.\n",
            "  warnings.warn(message, FutureWarning)\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "                    precision    recall  f1-score   support\n",
            "\n",
            "          AskIndia       0.51      0.69      0.58      3811\n",
            "  Business/Finance       0.22      0.54      0.31      1341\n",
            "       Coronavirus       0.57      0.48      0.52       320\n",
            "     Non-Political       0.61      0.36      0.46     10232\n",
            "       Photography       0.19      0.68      0.30       338\n",
            "    Policy/Economy       0.52      0.44      0.47      3838\n",
            "          Politics       0.75      0.54      0.63      9037\n",
            "Science/Technology       0.25      0.48      0.33      1363\n",
            "            Sports       0.43      0.64      0.51       521\n",
            "     [R]eddiquette       0.06      0.15      0.09      1006\n",
            "\n",
            "          accuracy                           0.48     31807\n",
            "         macro avg       0.41      0.50      0.42     31807\n",
            "      weighted avg       0.57      0.48      0.50     31807\n",
            "\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "bwsnk0LkwWmN",
        "colab_type": "text"
      },
      "source": [
        "We see that the performance of Multinomial Naive Bayes, Complement Naive Bayes, Linear SVM and logistic regression is very comparable with each other with logistic regression just edging out the other algorithms with an accuracy of 62%.\n",
        "\n",
        "Although we found comparable accuracies, this was only possible when we explored various parameters and reached the optimum parameters for each algorithm as we can see from the grid search values that for some parameters, the performance was really bad.\n",
        "\n",
        "Random Forest and NearestCentroid did not perform as well as the other methods.\n",
        "\n",
        "There remains the question whether a balanced dataset would have performed better on the test set we have. In the extras folder of the repository is the code for which I have implemented these algorithms after augmenting the dataset by performing oversampling using SMOTE.\n",
        "\n",
        "From here, we proceed to Deep Learning algorithms to perform classification. These methods are implemented in the notebooks 3A, 3B, 3C"
      ]
    }
  ]
}