-
Notifications
You must be signed in to change notification settings - Fork 915
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[FEA] Fix case insensitive match on native parquet column pruning #10747
Comments
Is this feature requested in the cuIO reader? Is case insensitivity part of the parquet spec? I believe we can do this one layer above libcudf. |
This issue has been labeled |
@davidwendt, does |
For reference: https://docs.rapids.ai/api/libcudf/stable/group__strings__case.html#ga8ec672aad6467cc71f37b1a3ac8179eb |
This issue has been labeled |
@revans2 Is this still needed? Also is this a parquet project or a strings project? |
This is not needed. It was a nice to have even when it was filed. feel free to close it. |
After NVIDIA/spark-rapids-jni#199 and NVIDIA/spark-rapids#5310 we will have an option to use native code to do column pruning and parsing of the footer for parquet. One of the issues is that C++ does not have built in APIs to convert a unicode string to lower case. It can do it a single character at a time, and that works most of the time, but in some cases it can have problems. This is to find a better way to make the strings lowercase.
The text was updated successfully, but these errors were encountered: