Fold into CuArrays

[MikeInnes]

CuArrays provides some machinery so that base functions (like sin or lgamma) can be replaced with their cudanative equivalents. With that this package could be folded into CuArrays and always "just work" for users, rather than having to load a new package for it. If you're up for it, happy to provide any clarification.

[xukai92]

Hi @MikeInnes. Yes I'm up for it. Can you give me some hints on how to approach

[MikeInnes]

The easiest thing is probably to use the @cufunc macro like so. This is really just defining a method for the cufunc function, like here where we define methods like cufunc(::typeof(Base.sin)) = CUDAnative.sin (the macro also does a replacement inside the definition you give it).

cufunc is called whenever broadcasting is used, so that will work automatically once you add that.

[xukai92]

@MikeInnes I'm about to fold this package into CuArrays but still have a few questions that needs your help:

1. As this also contains diff rules for digamma (which is missed in DiffRules.jl), I also need to submit a PR to DiffRules.jl. is it correct?
2. Where should I put the codes like https://github.com/xukai92/CuGammaFuns.jl/blob/master/src/CuGammaFuns.jl#L13? Do I put it in CuArrays and do cufunc(::typeof(SpecialFunctions.lgamma)) = CuArrays.lgamma?
3. Following 2, those codes are adapted from SpecialFunctions.jl, with only a few function calls inside pointed to CUDAnative's namespace, is there a way to not repeat codes there?

[MikeInnes]

1. A PR to DiffRules would probably be ideal. Short term you can also just define the rules in CuArrays (you'll need a rule for the GPU version as well anyway).
2. Yup, that sounds right
3. Unfortunately not, at the moment. The idea is to eventually have something Cassette-like to fix this stuff but for now we have to repeat everything (@cufunc automates this to some extent).