Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Extend the Learning Rate Monitor callback to log the initial lr for optimizers #7468

Closed
ananthsub opened this issue May 10, 2021 · 2 comments · Fixed by #9786
Closed

Extend the Learning Rate Monitor callback to log the initial lr for optimizers #7468

ananthsub opened this issue May 10, 2021 · 2 comments · Fixed by #9786
Labels
callback feature Is an improvement or enhancement good first issue Good for newcomers help wanted Open to be worked on
Milestone
pl:1.5

Comments

@ananthsub
Copy link
Contributor

🚀 Feature

Log the initial learning rate of each optimizer

Motivation

Currently the LR monitor callback only takes effect if the user specifies LR schedulers for training. However, one could want to record what the initial learning rate used for their optimizer's parameter groups even without any LR schedulers.

Pitch

We could log this once on train start following the logic here: https://github.com/PyTorchLightning/pytorch-lightning/blob/f6fe715e73cc87bfc106329f9103564097515379/pytorch_lightning/callbacks/lr_monitor.py#L138-L156

cc @simran2905

Alternatives

Additional context
@ananthsub ananthsub added feature Is an improvement or enhancement help wanted Open to be worked on callback labels May 10, 2021
@ananthsub ananthsub added this to the v1.4 milestone May 11, 2021
@edenlightning edenlightning modified the milestones: v1.4, v1.5 Jul 1, 2021
@ananthsub ananthsub added the good first issue Good for newcomers label Sep 10, 2021
@VirajBagal
Copy link
Contributor

@ananthsub @edenlightning I would like to work on this. Can you please assign this task to me?

1 similar comment
@VirajBagal
Copy link
Contributor

@ananthsub @edenlightning I would like to work on this. Can you please assign this task to me?

@VirajBagal VirajBagal mentioned this issue Oct 1, 2021
Merged
12 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
callback feature Is an improvement or enhancement good first issue Good for newcomers help wanted Open to be worked on
Projects
None yet
Milestone
pl:1.5
Development

Successfully merging a pull request may close this issue.

Log LR using LearningRateMonitor even when LR Scheduler is not defined. VirajBagal/pytorch-lightning
3 participants
@ananthsub @VirajBagal @edenlightning