Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Callback PR Rev 3 #615

Merged
merged 45 commits into from
May 28, 2020
Merged
Changes from 1 commit
Commits
Show all changes
45 commits
Select commit Hold shift + click to select a range
b16d356
Rebase off of master; add new working prototype of loss callback
blisc May 6, 2020
8024454
first working hack of computing uncomputed tensors
blisc May 7, 2020
879fcfc
style
blisc May 7, 2020
ddbf472
add a reference to Actions into TrainingState, remove deprecated func…
blisc May 12, 2020
912d83d
add decorators; add all events
blisc May 12, 2020
2e4eb18
style
blisc May 12, 2020
35d6b7d
more style
blisc May 12, 2020
4f6e1f7
initial refactor
blisc May 16, 2020
3c7b89e
adding checkpoint callback
blisc May 16, 2020
cf41850
enable fetching via NmTensor and string; add WandBCallback, Tensorboa…
blisc May 18, 2020
b1df99d
style
blisc May 18, 2020
d62f021
merge with master
blisc May 18, 2020
fa6553f
DDP bug fix
blisc May 18, 2020
ba84c80
clean up of checkpoint
blisc May 20, 2020
fc3ce62
update an4
blisc May 20, 2020
3a53e5e
merge with master
blisc May 20, 2020
e5b8258
style
blisc May 20, 2020
5361003
undo comenting
blisc May 20, 2020
9fc00d7
unpate
blisc May 20, 2020
d806e7e
wip
blisc May 21, 2020
f1c8aa8
more logging
blisc May 21, 2020
7608d45
remove debugging statements
blisc May 21, 2020
9f13e9d
style and merge
blisc May 21, 2020
5fe64fb
update new warning format with rank
blisc May 21, 2020
01dd179
add explicit rank marker
blisc May 21, 2020
f85f968
Merge branch 'warningformat_bug_fix' into U_callbacks_4
blisc May 21, 2020
46800cf
Merge remote-tracking branch 'nvidia/master' into U_callbacks_4
blisc May 21, 2020
c6ece47
docstrings and more
blisc May 21, 2020
3c3bee9
style
blisc May 21, 2020
dba4536
callback docstrings
blisc May 21, 2020
d95b2d4
style
blisc May 21, 2020
7bb53cd
add deprecation warnings
blisc May 22, 2020
d615efa
changelog
blisc May 22, 2020
21f4cf1
rename oldwandbcallback
blisc May 22, 2020
1c99f54
test
blisc May 22, 2020
b976ec0
style
blisc May 22, 2020
6ec04aa
first commit of changes
blisc May 27, 2020
7009bee
some fixes
blisc May 27, 2020
9f4566b
style
blisc May 27, 2020
307f550
move nmtensor_registry
blisc May 28, 2020
31fc556
update tests
blisc May 28, 2020
b9e4441
clean code for comments
blisc May 28, 2020
c036084
add back str_to_opt_level
blisc May 28, 2020
1e429af
split callbacks into two files; update error messages
blisc May 28, 2020
fdae1f3
add deprecated callbacks files
blisc May 28, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 4 additions & 2 deletions examples/asr/jasper_an4_debug.py
Original file line number Diff line number Diff line change
@@ -65,7 +65,7 @@ def create_dags(model_config_file, vocab, args, nf):
encoded, encoded_len = jasper_encoder(audio_signal=processed, length=processed_len)
log_probs = jasper_decoder(encoder_output=encoded)
predictions = greedy_decoder(log_probs=log_probs)
loss = ctc_loss(log_probs=log_probs, targets=transcript, input_length=encoded_len, target_length=transcript_len,)
loss = ctc_loss(log_probs=log_probs, targets=transcript, input_length=encoded_len, target_length=transcript_len)

# # Create an evaluation graph.
# audio_e, audio_len_e, transcript_e, transcript_len_e = data_layer_eval()
@@ -89,7 +89,9 @@ def create_dags(model_config_file, vocab, args, nf):
# loss.rename("test")
# train_callback = nemo.core.SimpleLossLogger(tensors_to_log=["test"])

train_callback = nemo.core.SimpleLossLogger()
# train_callback = nemo.core.SimpleLossLogger()
predictions.rename("test")
train_callback = nemo.core.SimpleLossLogger(tensors_to_log=["loss", "test"])
tkornuta-nvidia marked this conversation as resolved.
Show resolved Hide resolved

# checkpointer_callback = nemo.core.CheckpointCallback(folder=nf.checkpoint_dir, step_freq=args.checkpoint_save_freq)

Loading