Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

onehotbatch with CuArray #1959

Merged
merged 1 commit into from
May 7, 2022
Merged

onehotbatch with CuArray #1959

merged 1 commit into from
May 7, 2022

Conversation

CarloLucibello
Copy link
Member

@CarloLucibello CarloLucibello commented May 6, 2022

Not a great solution but this used to error before (when scalar indexing is deactivated) due to the following operation:

function _onehotbatch(data, labels)
      indices = UInt32[something(_findval(i, labels), 0) for i in data]
      ...

so I guess going to cpu and back to gpu is strictly better.

Copy link
Member

@ToucheSir ToucheSir left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can't think of a great way of doing this either. If anyone does, we can test it out in OneHotArrays.jl first.

@CarloLucibello CarloLucibello merged commit 25457f5 into master May 7, 2022
@YichengDWu
Copy link

What about this?

 onehotbatch(data::CuArray,labels)= cu(labels).==reshape(data, 1,size(data)...)

soure: https://discourse.julialang.org/t/all-the-ways-to-do-one-hot-encoding/64807/9

@ToucheSir
Copy link
Member

That defeats the purpose of having a one-hot array type in the first place. If you can find a way to preserve the outer wrapper and use less memory than a dense matrix, we would be very interested.

@YichengDWu
Copy link

I will leave it to the smart ones

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants