Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve adaptive stepsize for PANOC and ZeroFPR #50

Merged
merged 1 commit into from
Sep 27, 2021

Conversation

lostella
Copy link
Member

@lostella lostella commented Sep 27, 2021

PANOC and ZeroFPR require a step size gamma <= alpha/L for alpha < 1 and the smoothness constant L of the smooth term f. The backtracking procedure for gamma was not enforcing this: this fix updates the way the quadratic model for f is computed, and makes sure that alpha/gamma is used as diagonal Hessian in such quadratic model (and not 1/gamma).

In fact in some edge cases, the former behavior may have led the line search for tau to fail, later in the algorithms iteration. This fix makes sure that a more conservative gamma is selected in those cases, reducing the chance of line search failure.

@codecov
Copy link

codecov bot commented Sep 27, 2021

Codecov Report

Merging #50 (5c794b5) into master (cf6f7e9) will decrease coverage by 0.39%.
The diff coverage is 100.00%.

❗ Current head 5c794b5 differs from pull request most recent head 6838308. Consider uploading reports for the commit 6838308 to get more accurate results
Impacted file tree graph

@@            Coverage Diff             @@
##           master      #50      +/-   ##
==========================================
- Coverage   87.91%   87.51%   -0.40%     
==========================================
  Files          19       19              
  Lines         786      785       -1     
==========================================
- Hits          691      687       -4     
- Misses         95       98       +3     
Impacted Files Coverage Δ
src/algorithms/forwardbackward.jl 95.52% <100.00%> (ø)
src/algorithms/panoc.jl 94.94% <100.00%> (ø)
src/algorithms/zerofpr.jl 94.04% <100.00%> (ø)
src/utilities/fbetools.jl 100.00% <100.00%> (ø)
src/algorithms/drls.jl 92.98% <0.00%> (-3.51%) ⬇️
src/utilities/iterationtools.jl 82.43% <0.00%> (-1.36%) ⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update cf6f7e9...6838308. Read the comment docs.

@lostella lostella merged commit eb978e0 into JuliaFirstOrder:master Sep 27, 2021
@lostella lostella deleted the fix-gamma-backtracking branch September 27, 2021 21:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant