Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added documentation #35

Open
wants to merge 1 commit into
base: base
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 31 additions & 0 deletions APIs.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
# API
The APIs that are exposed for the platform are :

## GET /api/datasets
Returns all available datasets.
Looks in the dataset folder to see al the available folder and then returns it in form of a list.

## GET /api/datasets/{str:id}
Returns feature related to a specific dataset.
Runs the SchemaGenerator to create schema.pbtxt and then reads to return all the available feature in form of dictionary.

## GET /api/models
Returns all the available models.
Reads the MlModel database to achieve this.

## POST /api/models/create
Create a new model

## GET /api/models/available
Different type of model available for training.
Reads the available_model.py file to achieve this.

## GET /api/models/{str:id}
Information about a specific model

## GET /api/models/{str:id}/train
Train a specific model
Runs the training pipeline to achieve this.

## POST /api/models/{str:id}/predict
Use a trained model to make prediction
29 changes: 29 additions & 0 deletions HACKING.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
There are some major places that can be used to contribute to the project
# Frontend
The main goal of Machine Learning Platform is to provide a user friendly way for a user with no prior knowledge of machine learning to come train a model and then deploy it for day to day uses.<br>
Currently the user needs to use the APIs to access the features but wouldn't it be great if there was a frontend to make user interaction easier. Developing a frontend is one of the most important contribution you can make to the project.<br>
To start contributing to the frontend part. Create a directory called <code>frontend/</code> in the base of the project and get started.<br>
The frontend must have the following feature : <br>
<ul>
<li> UI to train model </li>
<li> Dashboard </li>
<ul>
You can make use of the APIs provided in API.md to connect your frontend.

# Backend
The app has a simple backend written in django and mainly uses DRF to provide REST Api. The backend of the project is present in <code>backend/</code> folder. If you are familiar with Django and think that the code can be better or you can add any feature. Make sure to make a PR with the proposed feature.<br>
<ul>
<li>
One of the places you can contribute is taking user inpute to overwrite the default hyperparameters for the model. Ensure that this is an optional feature for advanced users.
</li>
<li>
You can connect the MlModel database with the User database and add authentication.
</li>
</ul>
# ML Pipelines
AI is a fairly new field and new advancements are coming day by day. The frameworks are improving and new models are coming into picture. This part of the project needs constant contributions to keep it up to date with the modern standards. You can contribute to this part in the <code>backend/utils</code> directory. The pipelines are written in TensorflowExtended.
Here are some places where you can contribute to
## Available Models
Currently the platform supports only two kind of models. One for regression and one for classification. Both of them uses a neural network with variable width and depth as the model. You can add more model to it by adding a model with all it's hyperparameters in available_models.py <code>/backend/utils/ModelPreparation/AvailableModels/available_models.py</code>. You can describe the model in a seperate file and then import it to the available_model. TFX expects the model to be described in a certain way. You can take motivation from ffd.py <code>/backend/utils/ModelPreparation/AvailableModels/ffd.py</code> to create your new model.
## Dataset Diversity
Currentl the model only supports tabular data and it would be great to extend it for Image and text dataset. You will need to make few changes to make this happen. You might need to update your model for this you can refer to contributing to Available Models section of this file. Next you will need to make some transformation for your data for that you will need to change or play around with the module_file_generator.py file <code>/backend/utils/ModelPreparation/Trainers/module_file_generator.py</code> which is used to generate the code for generating model and performing transformation.
27 changes: 27 additions & 0 deletions SETUP.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
You can follow the following steps to setup the project locally.
<ol>
<li>
Fork the Repo.
</li>
<li>
Clone the repo locally using <code>git clone https://github.com/{your-username}/machine-learning-platform.git</code>
</li>
<li>
Go into the directory using <code>cd machine-learning-platform</code>
</li>
<li>
Create a new virtual enviornment <code>python3 -m venv env</code>. If you don't have virtual enviornment install. Install using <code>pip install virtualenv</code>
</li>
<li>
Install the dependencies using <code>pip3 install -r requirments.txt</code>
</li>
<li>
Ready your database for migration <code>python3 manage.py makemigrations</code>
</li>
<li>
migrate <code>python3 manage.py migrate --run-syncdb</code>
</li>
<li>
Start your backend development server <code>python3 manage.py runserver</code>
</li>
</ol>
6 changes: 6 additions & 0 deletions requirements.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
tfx==1.8.0
tensorflow==2.8.2
tensorflow-data-validation==1.8.0
apache-beam==2.39.0
Django==3.2.9
djangorestframework==3.12.4