TODO list

[antoine-levitt]

• Move the ManifoldObjective stuff to NLSolversBase
• Use in DifferentialEquations
• Implicit manifold defined by a residual, to be solved using NLSolversBase (or another solver)
• Redo the crappy interface for Product and Power manifolds

Ported over from JuliaNLSolvers/Optim.jl#448 :

• Second order algorithms
  Never used it, so I'm not the best person to do this
• Compare performance with other libraries
  Competitors include ROPTLIB, ManOpt, https://github.com/NickMcNutt/ManifoldOptim.jl (abandoned)
• Vector transport
  The proper way to implement algorithms like CG and BFGS is to use vector transport to transport the information at one point to another. Right now this is done with projections, which might not be the most efficient
• More manifolds and variants of existing manifolds (e.g. different retractions)
  See e.g. the list in http://www.math.fsu.edu/~whuang2/Indices/index_ROPTLIB.html
  Also {x, Ax = b}, or intersection manifold (just do the projection on both manifolds iteratively and hope it converges)
• Optimize number of retractions and projections
  I have been pretty liberal with the use of retractions and projections in the optimizers, maybe some of them are unnecessary
• A better way to do product manifolds?
  Right now, the two components are stored in a flat 1D array, which might be suboptimal
• Arbitrary inner product
  The Sphere and Stiefel manifolds could take a more general inner product