Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BUG] RowConversionTest fails after UNKNOWN_NULL_COUNT changes #1169

Closed
mythrocks opened this issue May 24, 2023 · 2 comments · Fixed by #1170
Closed

[BUG] RowConversionTest fails after UNKNOWN_NULL_COUNT changes #1169

mythrocks opened this issue May 24, 2023 · 2 comments · Fixed by #1170
Assignees
@mythrocks
Labels
bug Something isn't working

Comments

@mythrocks
Copy link
Collaborator

Repro:

build/build-in-docker  package -DCPP_PARALLEL_LEVEL=30  -DBUILD_TESTS=ON -Dtest=RowConversionTest -DCMAKE_CUDA_ARCHITECTURES=NATIVE

Error:

org.opentest4j.AssertionFailedError: Null Count For Column 0 ==> expected: <1> but was: <0>        at org.junit.jupiter.api.AssertionUtils.fail(AssertionUtils.java:55)
        at org.junit.jupiter.api.AssertionUtils.failNotEqual(AssertionUtils.java:62)
        at org.junit.jupiter.api.AssertEquals.assertEquals(AssertEquals.java:166)        at org.junit.jupiter.api.Assertions.assertEquals(Assertions.java:660)
        at ai.rapids.cudf.AssertUtils.assertPartialColumnsAreEqual(AssertUtils.java:114)        at ai.rapids.cudf.AssertUtils.assertPartialColumnsAreEqual(AssertUtils.java:95)
        at ai.rapids.cudf.AssertUtils.assertPartialTablesAreEqual(AssertUtils.java:250)
        at ai.rapids.cudf.AssertUtils.assertTablesAreEqual(AssertUtils.java:260)        at com.nvidia.spark.rapids.jni.RowConversionTest.fixedWidthRowsRoundTrip(RowConversionTest.java:86)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:725)
        at org.junit.jupiter.engine.execution.MethodInvocation.proceed(MethodInvocation.java:60)
@mythrocks mythrocks added bug Something isn't working ? - Needs Triage and removed ? - Needs Triage labels May 24, 2023
@mythrocks mythrocks self-assigned this May 24, 2023
@mythrocks mythrocks changed the title [BUG] RowConversionTest fails after UNKNOWN_NULL_COUNT changes in #1155 [BUG] RowConversionTest fails after UNKNOWN_NULL_COUNT changes May 24, 2023
@mythrocks
Copy link
Collaborator Author

Looks like this isn't related to the changes in #1148 or #1155. This test is failing after merging cudf/pull/13372.

On round-tripping a test vector, the null mask is coming out empty. This then sets the null-count to null.

@mythrocks mythrocks mentioned this issue May 25, 2023
Merged
@mythrocks
Copy link
Collaborator Author

I'm working on a fix now. Will post a PR shortly.

mythrocks added a commit to mythrocks/spark-rapids-jni that referenced this issue May 25, 2023
@mythrocks
Fix null counts for output of row/column conversion:
3955a2e 
Fixes NVIDIA#1169.

This commit fixes the null counts in row/column conversion, for all output
columns.
The prior fix in NVIDIA#1155 only fixed the count for STRING outputs.

There are 2 additional/tangential fixes in this change:
  1. `cudf::detail::null_count()` is used in place of `cudf::null_count()`,
     thus running on the current stream.
  2. The Java `RowConversion.convertFromRowsFixedWidthOptimized()` now uses
     the `convertFromRowsFixedWidthOptimized()` native function instead of
     `convertFromRows()`. This should prove faster.
@mythrocks mythrocks mentioned this issue May 25, 2023
Merged
mythrocks added a commit to mythrocks/spark-rapids-jni that referenced this issue May 25, 2023
@mythrocks
Fix null counts for output of row/column conversion:
1c0dbc0 
Fixes NVIDIA#1169.

This commit fixes the null counts in row/column conversion, for all output
columns.
The prior fix in NVIDIA#1155 only fixed the count for STRING outputs.

There are 2 additional/tangential fixes in this change:
  1. `cudf::detail::null_count()` is used in place of `cudf::null_count()`,
     thus running on the current stream.
  2. The Java `RowConversion.convertFromRowsFixedWidthOptimized()` now uses
     the `convertFromRowsFixedWidthOptimized()` native function instead of
     `convertFromRows()`. This should prove faster.

Signed-off-by: MithunR <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@mythrocks mythrocks

Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Fix null counts for output of row/column conversion: mythrocks/spark-rapids-jni
1 participant
@mythrocks