Skip to content

Latest commit

 

History

History
59 lines (42 loc) · 1.8 KB

README.md

File metadata and controls

59 lines (42 loc) · 1.8 KB
Raw

MLDS Final

This is the Machine Learning and having it deep and structured course's final project experiment source code. The goal is to test the RNN parameters initializtation setting. You can see the complete experiment settings and content from here.

Environment

python3
TensorFlow 1.0

Experiments

  • Identity Recurrent Neural Network exp1 - adding problem:
    • adding/
  • LSTM forget gate bias:
    • adding-lstm_bias/

Data

  • Identity Recurrent Neural Network exp1 - adding problem:
    Use TensorFlow package to import the MNIST dataset.
  • LSTM forget gate bias:
    Source code will autogenerate a random training and testing data.

Usage

  1. Go to the experiment folder that you want to test.
  2. Start training !

Train

IRNN exp1 - adding problem sequence length 50

$ cd adding/ 
$ python3 main.py --epochs 300 --seq_length 50 --lstm_lr 0.01 --rnn_tanh_lr 0.01 --rnn_relu_lr 0.01 --irnn_lr 0.01 --ornn_lr 0.01

IRNN exp1 - adding problem with sequence length 150

$ cd adding/ 
$ python3 main.py --epochs 300 --seq_length 150 --lstm_lr 1e-4 --rnn_tanh_lr 1e-4 --rnn_relu_lr 1e-4 --irnn_lr 1e-4 --ornn_lr 0.01

LSTM forget bais

$ cd adding-lstm_bias/ 
$ python3 main.py --lstm_lr 1e-2

Experiment Results

  • Identity Recurrent Neural Network exp1 sequence length 50:
  • Identity Recurrent Neural Network exp1 sequence length 150:
  • LSTM forget gate bias: