Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Improvement] save best ckpt during training #464

Closed
wants to merge 9 commits into from

Conversation

irvingzhang0512
Copy link
Contributor

@irvingzhang0512 irvingzhang0512 commented Dec 19, 2020

Fix #460

Add arg save_best_ckpt for eval hook,. If set to True, then save best ckpt in work_dir with name best_{best_socre}_{epoch_id}.pth

  • modify eval hook
  • add unittest
  • add changelog
  • fix codecov

@codecov
Copy link

codecov bot commented Dec 19, 2020

Codecov Report

Merging #464 (4500f7f) into master (4df447b) will increase coverage by 0.05%.
The diff coverage is 76.00%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #464      +/-   ##
==========================================
+ Coverage   84.69%   84.74%   +0.05%     
==========================================
  Files         118      118              
  Lines        8344     8352       +8     
  Branches     1366     1367       +1     
==========================================
+ Hits         7067     7078      +11     
+ Misses        930      927       -3     
  Partials      347      347              
Flag Coverage Δ
unittests 84.73% <76.00%> (+0.05%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
mmaction/models/builder.py 55.26% <0.00%> (ø)
mmaction/models/heads/roi_head.py 28.88% <0.00%> (ø)
...maction/models/roi_extractors/single_straight3d.py 81.81% <50.00%> (-3.55%) ⬇️
mmaction/core/evaluation/eval_hooks.py 88.09% <89.47%> (+4.62%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 4df447b...4500f7f. Read the comment docs.

@dreamerlin
Copy link
Collaborator

Thanks for your contribution
Lately, we will refactor the EvalHook in the PR #395, which may then be applied into MMCV for other OpenMMLab repo usage:

  1. Add by_epoch param to determine perform evaluation by epoch or by iteration.
  2. Add automatically assign a compare key when users do not set one.
  3. Save checkpoint by creating a symlink as well as record best info in runner hook meta
  4. Refactor the code structure

you can modify based on #395

@irvingzhang0512
Copy link
Contributor Author

Thanks for your contribution
Lately, we will refactor the EvalHook in the PR #395, which may then be applied into MMCV for other OpenMMLab repo usage:

  1. Add by_epoch param to determine perform evaluation by epoch or by iteration.
  2. Add automatically assign a compare key when users do not set one.
  3. Save checkpoint by creating a symlink as well as record best info in runner hook meta
  4. Refactor the code structure

you can modify based on #395

Sure, I will close this pr and create a new one tonight.

@dreamerlin
Copy link
Collaborator

no hurry, you can make a new pr when #395 is merged

@irvingzhang0512
Copy link
Contributor Author

no hurry, you can make a new pr when #395 is merged

ok

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Training not storing best model
2 participants