Feature/unsupervised training #29

cregouby · 2021-02-08T08:00:57Z

Hello,
Unsupervised training is working now, and can be part of training scenario ( see test-hardhat-scenarios.R)
except for CUDA where I can't get torch_bernoulli() work on GPU (see mlverse/torch#480)
Fix #18

…pervised_training

dead-end as function design is not ready

fix typo in epoch_shift

add random_obfuscator() fix unsupervised_loss()

fix y has one column in tabnet_train_unsupervised()

lower data size in secondary tests

fix typo in subseting dataset in test

…/unsupervised_training

add train_batch_un() and valid_batch_un() fix epoch_shift value missing

…rk-ai)

Merge branch 'main' of https://github.com/mlverse/tabnet into feature/unsupervised_training # Conflicts: # R/model.R

dfalbel · 2021-02-08T12:56:18Z

Hey @cregouby ! This looks awesome!! It's almost there!
I have pushed some changes.

I think the problem was that the decoder was returning only the combined steps output instead of the intermediate ones.

We now get to a print("Done") but fails in serialization.
Let me know if this is helpful!

cregouby · 2021-02-08T14:32:52Z

Thanks a lot for this push @dfalbel ! I'll definitively continue with the model serialization.

…ll fails to pipe unsupervised to supervised.

Still need to convert network from one to the other

still failing saveRDS serialisation and val-split

remove duplicate function

move some tests to -scenarios

…/unsupervised_training

relaxed tolerance between two training epoch

fix typo add few more explainations

…g the network()

dfalbel

Awesome @cregouby ! Merging now!

cregouby · 2021-03-02T18:59:49Z

Thanks a lot, @dfalbel !

Christophe-Regouby added 21 commits January 22, 2021 09:57

start with the vignette (beta)

420be1c

Merge remote-tracking branch 'remotes/mlverse/main' into feature/unsu…

b58be73

…pervised_training

tabnet_pretrain S3 boilerplate

90e4cd3

improve feature documentation but

c4aeeea

dead-end as function design is not ready

Merge branch 'main' into feature/unsupervised_training

05185b0

make the vignette focus on unsupervised training only.

01739bc

add unsupervised tests

e7898a3

frame the files

89d2e96

unsupervised front-end commands

aa15810

doc improvement

8e18b54

fix typo in epoch_shift

merge 2 tabnet_bridges into one via switch

78bf79f

add pretraqining_ratio in config()

f421ce8

add random_obfuscator() fix unsupervised_loss()

extract back tabnet_encoder() out of tabnet_no_embedding()

01cd4f7

fix y has one column in tabnet_train_unsupervised()

add tabnet_decoder()

e8e0661

add tabnet_pretraining() network initialization

b0bbe0c

lower data size in secondary tests

fix typo in tabnet_bridge()

e61ea1e

fix typo in subseting dataset in test

Merge branch 'main' of https://github.com/mlverse/tabnet into feature…

1b1ebd3

…/unsupervised_training

Better names for tests

abdc991

add train_batch_un() and valid_batch_un() fix epoch_shift value missing

make prior as argument and train on full batch (differs from dreamqua…

77b4bc8

…rk-ai)

generalise prior in encoder() call

8563ad6

strict copy of dreamquark-ai/tabnet code in tabnet_decoder() module

68ab447

cregouby marked this pull request as draft February 8, 2021 08:02

Christophe-Regouby and others added 2 commits February 8, 2021 09:08

merge master.

604b38d

Merge branch 'main' of https://github.com/mlverse/tabnet into feature/unsupervised_training # Conflicts: # R/model.R

make encoder return the intermediate step results

2bbb7f8

Christophe-Regouby added 4 commits February 8, 2021 16:55

fix unsupervised model serialization, improve unsupervised tests, sti…

b89418f

…ll fails to pipe unsupervised to supervised.

separate tabnet_fit from tabnet_pretrain S3 class.

9635e1a

Still need to convert network from one to the other

add print for class tabnet.pretrain

f129836

add a scenario for incompatible models

937d7f7

Christophe-Regouby added 7 commits February 11, 2021 22:27

add model_pretrain_to_fit()

7f21d76

still failing saveRDS serialisation and val-split

manage model relaod after read from file

134e872

remove duplicate function

fix validation

4cab727

fix unsupervised model reloading need switch to $eval()

131a064

move some tests to -scenarios

improve vignette with loss plots

69a714e

Merge branch 'main' of https://github.com/mlverse/tabnet into feature…

7baaf8b

…/unsupervised_training

fix reload a RDS pretrain_model

9506307

relaxed tolerance between two training epoch

cregouby marked this pull request as ready for review February 16, 2021 19:13

Christophe-Regouby and others added 8 commits February 16, 2021 20:40

fix roc_auc > 0.5 in vignette

cc0365e

fix typo add few more explainations

get rid of stringr:: dependancy for easier deployment

50765b9

accept snapshot update

0ebdf87

fix print() after model load from disk

080931b

coro::loop requires the move batch to device to be done before callin…

273f74c

…g the network()

workaround torch_bernoulli() is not available in CUDA

783fa9a

disambiguate names between function and torch_cuda dataset

cb55519

avoid crash on the gpu by checking model dimensions before loading

38c42ed

dfalbel approved these changes Mar 2, 2021

View reviewed changes

dfalbel merged commit 28a494a into mlverse:main Mar 2, 2021

cregouby deleted the feature/unsupervised_training branch March 2, 2021 18:59

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Feature/unsupervised training #29

Feature/unsupervised training #29

cregouby commented Feb 8, 2021 •

edited

Loading

dfalbel commented Feb 8, 2021

cregouby commented Feb 8, 2021 •

edited

Loading

dfalbel left a comment

cregouby commented Mar 2, 2021

Feature/unsupervised training #29

Feature/unsupervised training #29

Conversation

cregouby commented Feb 8, 2021 • edited Loading

dfalbel commented Feb 8, 2021

cregouby commented Feb 8, 2021 • edited Loading

dfalbel left a comment

Choose a reason for hiding this comment

cregouby commented Mar 2, 2021

cregouby commented Feb 8, 2021 •

edited

Loading

cregouby commented Feb 8, 2021 •

edited

Loading