Feat: Spam Detection

[aimura09]

Attempts to close https://github.com/comses/planning/issues/113

Squashed commits and solved merge conflicts.

Summary

Management commands for Machine Learning spam detection.

Features

Before running the commands, make sure spam_dataset.csv is located in the curator folder.
spam_dataset.csv consist of user_id and is_spam columns. is_spam column contains 1(Spam) or 0(Ham).

• UserMetadataSpamClassifier() ... Uses XGboost as a classifier. Takes "user_id", "labelled_by_curator", "first_name", "last_name", "is_active", "email", "affiliations", "bio", "research_interests" fields of the MemberProfiles as input.
• TextSpamClassifier() ... Uses MultinomialNB as a classifier. Takes the "bio" and "research_interest" fields of the MemberProfiles as input. MemberProfiles that have NaN in these fields are disregarded.

1. To get a list of spam users (detected and labeled), run the following command.
   ./manage.py curator_spam_detection --exe
   This will train the machine learning models if no instance file (pickle file) is found. Otherwise, it loads pre-trained model files and performs predictions on the users that haven't been labeled by a curator.

2. options
   -- get_model_metrics: To print the accuracy, precision, recall, and f1 scores of spam detection models
   --load_labels: To load the labels in spam_dataset.csv into DB. Running --exe the first time will load labels without specifying this option. It might be useful when we want to update labels separately.
   --train_user: To train the UserMetadataSpamClassifier() individually
   --predict_user: To get the predictions of UserMetadataSpamClassifier() individually
   --train_text: To train the TextSpamClassifier() individually
   --predict_text: To get the predictions of TextSpamClassifier() individually

Tests

Wrote tests using Django tests

[alee]

closing for commit surgery on a second branch so we can integrate properly with Noel's tag changes