-
-
Notifications
You must be signed in to change notification settings - Fork 42
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Added a Trust Region solver. #116
Conversation
Codecov Report
@@ Coverage Diff @@
## master #116 +/- ##
==========================================
+ Coverage 83.33% 86.58% +3.25%
==========================================
Files 5 6 +1
Lines 228 343 +115
==========================================
+ Hits 190 297 +107
- Misses 38 46 +8
📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more |
Note to @oscardssmith if he wants to take a look and help this get over the finish line. |
I handled that. SciML/SimpleNonlinearSolve.jl#28 |
differentiation. Defaults to true, which thus uses the `NonlinearSolveTag`. If `Val{false}`, | ||
then ForwardDiff's default function naming tag is used, which results in larger stack | ||
traces. | ||
- `concrete_jac`: whether to build a concrete Jacobian. If a Krylov-subspace method is used, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this should just be jac
to be consistent with the rest of SciML
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
no, it's always concrete_jac
How does this benchmark compared to NLSolve? |
I'll have a look at benchmarking as soon as I finish this up :) |
What do you mean? Example? |
Sorry, just deleted the message. |
This input gives error
ONLY for the last iteration (p = 100.0), when
|
how far off is it? |
|
Solved the problem with |
That's the correct thing to do. |
solver = init(probN, TrustRegion(), abstol = 1e-9) | ||
sol = solve!(solver) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can you add a test that solve!
on the mutable code path does not allocate? I think if that passes then it's done.
cache.loss_new = get_loss(fu_new) | ||
|
||
# Compute the ratio of the actual reduction to the predicted reduction. | ||
cache.r = -(loss - cache.loss_new) / (step_size' * g + step_size' * H * step_size / 2) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This still needs to be made non-allocating.
Tried to fix the allocation problem (get them < 400), but why did you make |
solver = init(probN, TrustRegion(), abstol = 1e-9) | ||
sol = solve!(solver) | ||
end | ||
|
||
function benchmark_mutable(f, u0) | ||
probN = NonlinearProblem(f, u0) | ||
probN = NonlinearProblem{false}(f, u0) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this one should be true
test/basictests.jl
Outdated
@@ -161,6 +161,10 @@ sol = benchmark_scalar(sf, csu0) | |||
@test sol.retcode === ReturnCode.Success | |||
@test sol.u * sol.u - 2 < 1e-9 | |||
|
|||
@test (@ballocated benchmark_immutable(ff, cu0)) < 400 | |||
@test (@ballocated benchmark_mutable(ff, cu0)) < 400 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
the allocation test we should check on this one shouldn't include the problem building.
That looks like it's a typo. |
|
yeah my guess is that it's mostly line 299. Let's get this into a form that can be merged and follow up with fixing that. |
Sure! |
(I don't really understand why some checks, e.g "IntegrationTest / OrdinaryDiffEq.jl" fail and what to do about it) |
It seems like it's a v1.6 only error? |
yeah, sorry for not understanding... but I don't know what to do to fix it :( |
Precompilation of |
I want to say a huge thank you for all the help @ChrisRackauckas! If you would like me to continue with something specific, please tell me, otherwise I assume a good starting point is to implement the new solvers you just added to the issues. PS. Thank you for the invite to SciML, means a lot to me :D |
Add Line Search to (L)Broyden
Implementation of a
TrustRegion
solver.The solver is similar to the
TrustRegion
solver inSimpleNonlinearSolve.jl
, but this new solver is implemented withNewtonRaphson
as a template.Questions:
TrustRegion
solvers (inSimpleNonlinearSolve.jl
andNonlinearSolve.jl
) have the same name, should I change the name of theTurstRegion
solver inSimpleNonlinearSolve.jl
toSimpleTrustRegion
?TurstRegion
solver does not give any error (asNewtonRaphson
does). But the solver finds a solution with a "high" error. I suppose this is because it's getting stuck in a local minimum. But if you think that's not the case, please tell me.As always, just tell me if there is anything I can improve/change.