Note: The purpose of the project to explore the libraries and learn how to use them. Not to build a SOTA model.

Requirements:

This project uses Python 3.8

Create a virtual env with the following command:

conda create --name project-setup python=3.8
conda activate project-setup

Install the requirements:

pip install -r requirements.txt

Running

Training

After installing the requirements, in order to train the model simply run:

python train.py

Monitoring

Once the training is completed in the end of the logs you will see something like:

wandb: Synced 5 W&B file(s), 4 media file(s), 3 artifact file(s) and 0 other file(s)
wandb:
wandb: Synced proud-mountain-77: https://wandb.ai/raviraja/MLOps%20Basics/runs/3vp1twdc

Follow the link to see the wandb dashboard which contains all the plots.

Versioning data

Refer to the blog: DVC Configuration

Exporting model to ONNX

Once the model is trained, convert the model using the following command:

python convert_model_to_onnx.py

Inference

Inference using standard pytorch

python inference.py

Inference using ONNX Runtime

python inference_onnx.py

Google Service account

Create service account using the steps mentioned here: Create service account

Configuring dvc

dvc init
dvc remote add -d storage gdrive://19JK5AFbqOBlrFVwDHjTrf9uvQFtS0954
dvc remote modify storage gdrive_use_service_account true
dvc remote modify storage gdrive_service_account_json_file_path creds.json

creds.json is the file created during service account creation

Docker

Install the docker using the instructions here

Build the image using the command

docker build -t inference:latest .

Then run the container using the command

docker run -p 8000:8000 --name inference_container inference:latest

(or)

Build and run the container using the command

docker-compose up

Running notebooks

I am using Jupyter lab to run the notebooks.

Since I am using a virtualenv, when I run the command jupyter lab it might or might not use the virtualenv.

To make sure to use the virutalenv, run the following commands before running jupyter lab

conda install ipykernel
python -m ipykernel install --user --name project-setup
pip install ipywidgets