Fix warmup lr

[twmht]

Thanks for your contribution and we appreciate it a lot. The following instructions would make your pull request more healthy and more easily get feedback. If you do not understand some items, don't worry, just make the pull request and seek help from maintainers.

Motivation

Fix bug when warmup by epoch.

Modification

When using the following lr_config, this is how ranger works.

lr_config={
type='CosineAnnealing',
warmup='constant',
warmup_iters=75,
warmup_by_epoch=True
min_lr=0,
by_epoch=True,
warmup_ration=1
}

total_epochs=100

in the current implementations, it does not work as expected. it would decay the learning rate by cosine in second epoch.

this PR would fix the bug.

[codecov]

Codecov Report

Merging #1079 (c0190a8) into master (d212bd5) will increase coverage by 0.00%.
The diff coverage is 57.14%.

❗ Current head c0190a8 differs from pull request most recent head 65d0500. Consider uploading reports for the commit 65d0500 to get more accurate results

@@           Coverage Diff           @@
##           master    #1079   +/-   ##
=======================================
  Coverage   65.72%   65.72%           
=======================================
  Files         157      157           
  Lines       10092    10098    +6     
  Branches     1828     1830    +2     
=======================================
+ Hits         6633     6637    +4     
- Misses       3113     3115    +2     
  Partials      346      346           

Flag	Coverage Δ
unittests	65.72% <57.14%> (+<0.01%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
mmcv/runner/hooks/lr_updater.py	70.03% <57.14%> (-0.07%)	⬇️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d212bd5...65d0500. Read the comment docs.

[wangg12]

The lr schedule for ranger could be achieved by #1066

[wangg12]

I think the lr functions in mmcv usually treat the beginning iteration as 1 (the first iteration) regardless of warmup. If you change it to the end of warmup iter, it might break the behavior of many other models.

[twmht]

@wangg12

What I think is that this is a bug-fix-PR, not a feature enhancement PR.

[wangg12]

Not sure if it is a "bug fix", because it breaks the old behavior.

[twmht]

@wangg12

Can you explain an example which is broken by this PR?