Ideas to make ProximalProjection more robust

[f0uriest]

Right now we still have cases where the proximal optimizer keeps trying smaller and smaller steps until eventually it quits saying Warning: A bad approximation caused failure to predict improvement.

This seems to be due to inconsistent linearizations or a possibly instability, where small changes in the boundary (the x variable the optimizer sees) lead to large changes in the volume (the f values the optimizer sees).

Some possible ideas to make this better:

• Have some "patience" criteria, where if the perturb/solve stuff doesn't find an acceptable step in N steps, we go back and do a full continuation solve.
• Have some check on the step size in both boundary and volume and reject steps if the ratio is too large.
• Use the 2nd order perturbations to do the above, where we reject a step if the 2nd order correction is larger than some fraction of the first order step (similar to geodesic acceleration).
• If several attempts don't find an acceptable step, return a smaller function value to force it to accept a "bad" step in the hopes that linearizing around the new point will be better.

[dpanici]

Case for a precise-QA-like equilibrium in free boundary, each step in the proximal just gives a bad approximation, files are here: https://drive.google.com/drive/folders/19-F7cVb1yxks42eCI1Nx7DlxjCkyxaaf?usp=drive_link

[dpanici]

@dpanici check thru notebooks and link here along with current commit that it occurs at

[dpanici]

• https://desc-docs--1189.org.readthedocs.build/en/1189/notebooks/tutorials/free_boundary_equilibrium.html#Solovev-Tokamak SOLOVEV free bdry (on PR Add easier to read print option #1189 )
• https://desc-docs--1189.org.readthedocs.build/en/1189/notebooks/tutorials/advanced_optimization.html#Multigrid-method-with-proximal-optimizer QH proximal optimization (again on Add easier to read print option #1189 )

[dpanici]

Something I have seen in a case where I get a bad approx error is when the eq subproblem default xtol=1e-6 is actually a larger xtol than the outer optimization loop's xtol (in my case I had it at like 1e-8), obviously this is stupid on my part as I don't let the eq change by the same size steps I allow the outer optimizer to take, but imo this can be easy to forget. Maybe we could change the default eq subproblem ftol and xtol to match the outer optimizer's, if it the optimizer has tighter tolerances?

(I have not tested that tightening the xtol fixes the bad approx yet, doing it now, but I suspect it should)

I also have not tested with #1073 though which maybe would help in this case?