Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor TRA #531

Merged
merged 8 commits into from
Jul 30, 2021
+1,596 −41
Merged

refactor TRA #531

Changes from 1 commit
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
94eb945
refactor TRA
Jul 21, 2021
740b83e
format by black
Jul 21, 2021
210d1ec
fix horizon
Jul 22, 2021
c2437a2
fix TRA when use single head
Jul 22, 2021
5fc909e
clean up
Jul 22, 2021
1320b4d
improve pretrain
Jul 28, 2021
08471a1
Merge remote-tracking branch 'origin/main' into tra
Jul 28, 2021
f41445c
update README
Jul 28, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
improve pretrain
Dong Zhou authored and Dong Zhou committed Jul 28, 2021
commit 1320b4d8f1081f545d0adf73545029c87c60a4ff
3 changes: 2 additions & 1 deletion examples/benchmarks/TRA/workflow_config_tra_Alpha158.yaml
View file
Original file line number Diff line number Diff line change
@@ -73,6 +73,7 @@ task:
kwargs:
tra_config: *tra_config
model_config: *model_config
model_type: RNN
lr: 1e-3
n_epochs: 100
max_steps_per_epoch: 100
@@ -86,7 +87,7 @@ task:
memory_mode: *memory_mode
eval_train: False
eval_test: True
pretrain: False
pretrain: True
init_state:
freeze_model: False
freeze_predictors: False
6 changes: 4 additions & 2 deletions examples/benchmarks/TRA/workflow_config_tra_Alpha158_full.yaml
View file
Original file line number Diff line number Diff line change
@@ -33,12 +33,13 @@ tra_config: &tra_config
num_states: *num_states
hidden_size: 16
tau: 1.0
src_info: TPE
src_info: LR_TPE

model_config: &model_config
input_size: 158
hidden_size: 256
num_layers: 2
rnn_arch: LSTM
use_attn: True
dropout: 0.2

@@ -66,6 +67,7 @@ task:
kwargs:
tra_config: *tra_config
model_config: *model_config
model_type: RNN
lr: 1e-3
n_epochs: 100
max_steps_per_epoch: 100
@@ -79,7 +81,7 @@ task:
memory_mode: *memory_mode
eval_train: False
eval_test: True
pretrain: False
pretrain: True
init_state:
freeze_model: False
freeze_predictors: False
3 changes: 2 additions & 1 deletion examples/benchmarks/TRA/workflow_config_tra_Alpha360.yaml
View file
Original file line number Diff line number Diff line change
@@ -67,6 +67,7 @@ task:
kwargs:
tra_config: *tra_config
model_config: *model_config
model_type: RNN
lr: 1e-3
n_epochs: 100
max_steps_per_epoch: 100
@@ -80,7 +81,7 @@ task:
memory_mode: *memory_mode
eval_train: False
eval_test: True
pretrain: False
pretrain: True
init_state:
freeze_model: False
freeze_predictors: False
2 changes: 1 addition & 1 deletion qlib/contrib/model/pytorch_tra.py
View file
Original file line number Diff line number Diff line change
@@ -392,7 +392,7 @@ def fit(self, dataset, evals_result=dict()):
self.logger.info("reset TRA")
self.tra.reset_parameters() # reset both router and predictors

self.optimizer = optim.Adam(self.tra.parameters(), lr=self.lr) # optimize TRA only
self.optimizer = optim.Adam(list(self.model.parameters()) + list(self.tra.parameters()), lr=self.lr)

self.logger.info("training...")
best_score, _ = self._fit(train_set, valid_set, test_set, evals_result, start_epoch=epoch, is_pretrain=False)