Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Zygote Flux and custom adjoints on GPU #1828

Closed
TsurHerman opened this issue Jan 11, 2022 · 2 comments
Closed

Zygote Flux and custom adjoints on GPU #1828

TsurHerman opened this issue Jan 11, 2022 · 2 comments

Comments

@TsurHerman
Copy link

I am incorporating some simple non-standard elements in a larger DNN.

To achieve this I had to manually define some reverse rules through ChainRulesCore syntax
ChainRulesCore.rrule(::typeof(f),args...)
Which works great on the cpu, but somehow is not portable to the gpu.

I narrowed it to a MWE that fails the GPU compilation of the pullback function

using Zygote
using CUDA

f(A) = map(A) do a
    a
end

A = rand(Float32,128,128)
gA = gpu(A)
f(A) # works
f(gA) # works

Δ,pb = pullback(f,A) #works

Δ,pb = pullback(f,gA) #fails

Any insights?

@DhairyaLGandhi
Copy link
Member

map on CUDA arrays is broken at the moment, you can use broadcast instead.

@TsurHerman
Copy link
Author

Yes this solves the problem
You can close the issue or leave it as reference until this gap is mitigated

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants