Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] A data buffer is cast into a type with a different layout #5916

Closed
vuule opened this issue Aug 11, 2020 · 7 comments
Closed

[BUG] A data buffer is cast into a type with a different layout #5916

vuule opened this issue Aug 11, 2020 · 7 comments
Assignees
Labels
bug Something isn't working cuIO cuIO issue libcudf Affects libcudf (C++/CUDA) code.

Comments

@vuule
Copy link
Contributor

vuule commented Aug 11, 2020

A device array containing objects of type thrust::pair<const char*, size_type> is cast to nvstrdesc_s using reinterpret_cast. These two types are not guaranteed to have the same memory layout.
The code works, but it's not guaranteed to work correctly on all compilers/platforms.

Observed the issue in ORC and Parquet readers.

@vuule vuule added bug Something isn't working Needs Triage Need team to review and classify cuIO cuIO issue labels Aug 11, 2020
@harrism
Copy link
Member

harrism commented Aug 11, 2020

@vuule can you add a link to the problem code?

@vuule
Copy link
Contributor Author

vuule commented Aug 11, 2020

in column_buffer:

using str_pair = thrust::pair<const char*, size_type>;

rmm::device_vector<str_pair> _strings;

auto data() { return _strings.size() ? _strings.data().get() : _data.data(); }

In the Parquet reader:
chunks[c].column_data_base = out_buffers[chunk_col_map[c]].data();

and then in page decode there are a few lines like this one:
const char *ptr = reinterpret_cast<const nvstrdesc_s *>(s->col.column_data_base)[row].ptr;

@devavret
Copy link
Contributor

Can this be remedied by the suggestion in #5682 . Essentially doing away with nvstrdesc

@vuule
Copy link
Contributor Author

vuule commented Aug 11, 2020

Thanks @devavret . Assigned to you since the other issue is already on you.

@kkraus14 kkraus14 added libcudf Affects libcudf (C++/CUDA) code. and removed Needs Triage Need team to review and classify labels Aug 18, 2020
@github-actions
Copy link

This issue has been marked rotten due to no recent activity in the past 90d. Please close this issue if no further response or action is needed. Otherwise, please respond with a comment indicating any updates or changes to the original issue and/or confirm this issue still needs to be addressed.

@vuule
Copy link
Contributor Author

vuule commented Feb 17, 2021

Issue still relevant.

@vyasr
Copy link
Contributor

vyasr commented May 29, 2024

This was resolved in #7841

@vyasr vyasr closed this as completed May 29, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working cuIO cuIO issue libcudf Affects libcudf (C++/CUDA) code.
Projects
None yet
Development

No branches or pull requests

5 participants