Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Avro reader has very limited test coverage #6802

Closed
1 of 2 tasks
vuule opened this issue Nov 19, 2020 · 7 comments
Closed
1 of 2 tasks

Avro reader has very limited test coverage #6802

vuule opened this issue Nov 19, 2020 · 7 comments
Labels
cuIO cuIO issue libcudf Affects libcudf (C++/CUDA) code.

Comments

@vuule
Copy link
Contributor

vuule commented Nov 19, 2020

Avro reader currently has a single test, with a few more basic tests pending to be merged.

Some cases not covered:

  • Large input files;
  • Columns of all supported types;
@vuule
Copy link
Contributor Author

vuule commented Dec 2, 2020

Marking as P0 for 0.18 as it's preventing other Avro refactoring work.

@cwharris
Copy link
Contributor

cwharris commented Dec 2, 2020

Maybe we can use Apache's python-based reader/writer to generate test data. Generating files via python from the c++ side seems backwards, though. Do we have a methodology to store/load pre-generated test data/files in tests?

@vuule
Copy link
Contributor Author

vuule commented Dec 2, 2020

Maybe we can use Apache's python-based reader/writer to generate test data. Generating files via python from the c++ side seems backwards, though. Do we have a methodology to store/load pre-generated test data/files in tests?

Maybe I'm misunderstanding the question, but you can generate tmp avro files using fastavro in Python tests. There are tests that already do this.

@cwharris
Copy link
Contributor

cwharris commented Dec 3, 2020

It would be nice to have c++ tests, though, since the reader is implemented at that level.

@github-actions
Copy link

This issue has been labeled inactive-30d due to no recent activity in the past 30 days. Please close this issue if no further response or action is needed. Otherwise, please respond with a comment indicating any updates or changes to the original issue and/or confirm this issue still needs to be addressed. This issue will be labeled inactive-90d if there is no activity in the next 60 days.

@vuule
Copy link
Contributor Author

vuule commented Mar 31, 2021

missing large tests still relevant

@GregoryKimball GregoryKimball added the libcudf Affects libcudf (C++/CUDA) code. label Apr 2, 2023
@vuule
Copy link
Contributor Author

vuule commented Jun 29, 2023

Closing, since #12912 added larger tests

@vuule vuule closed this as completed Jun 29, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cuIO cuIO issue libcudf Affects libcudf (C++/CUDA) code.
Projects
None yet
Development

No branches or pull requests

3 participants