Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Update gradient descent for per-parameter learning rates #567

Closed
BradyPlanden opened this issue Nov 25, 2024 · 0 comments · Fixed by #569
Closed

Update gradient descent for per-parameter learning rates #567

BradyPlanden opened this issue Nov 25, 2024 · 0 comments · Fixed by #569
Assignees
@BradyPlanden
Labels
enhancement New feature or request

Comments

@BradyPlanden
Copy link
Member

Feature description

Add a per-parameter learning rate implementation of gradient descent.

Motivation

The current gradient descent implementation uses the minimum sigma0 applied to all parameters. This is problematic if the gradient magnitudes are dissimilar, or for learning rate calibration to fine tune convergence.

Possible implementation

No response

Additional context

No response
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@BradyPlanden BradyPlanden

Labels
enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Parameter specific learning rates for Gradient Descent pybop-team/PyBOP
1 participant
@BradyPlanden