Mixed-mode AD?

[cgeoga]

I have a little package that I use for convenient and non-allocating reverse-mode gradients and forward-over-reverse Hessians (the FoRWrapper struct+methods....my attempts at a smarter wrapper for forward-over-forward that are in the source don't seem super helpful). I don't know how generally useful it would be and its current design is I think pretty not thread-safe. But I occasionally have problems that I solve with Ipopt where the objective is simple enough that I can compile the tapes and I see a nice speedup using an FoRWrapper instead of straight ForwardDiff.

It doesn't look like ReverseDiff shows up anywhere as a dependency and I respect that you keep the deps light. But if you would be interested in incorporating something like this somewhere, I'd be happy to try my best at a PR wherever you think is most appropriate. I'm also perfectly happy to change names or do whatever re-org you think is valuable.

No sweat if it's not a good fit---just thought I'd offer! Thanks for the work you've done in making and documenting this package.

[mohamed82008]

Thanks @cgeoga for your offer. Your package looks useful but perhaps better suited to AbstractDifferentiation. I am working on a refactor of AbstractDifferentiation.jl in my free time so it's taking a while but once I am done, you should be able to define a backend for FoR or FoF and specialise it for functions by pre-allocating memory.

[cgeoga]

Makes sense! I'll keep an eye on that package. I don't necessarily think the code device I use in that package is the best, but I haven't seen any other wrappers or convenience structures for compiling tapes for Dual inputs as well as, like, Float64s. So I'll be curious to see what ends up there!

[mohamed82008]

The code looks good and useful. I just think the best place for it is AbstractDifferentiation.