Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix java float/double parsing tests [skip ci] #7473

Merged
merged 30 commits into from
Mar 2, 2021
Merged
Changes from all commits
Commits
Show all changes
30 commits
Select commit Hold shift + click to select a range
7a93e07
Add a benchmark for string <=> floats conversion.
ttnghia Feb 18, 2021
3ea06fb
Fix error and improve the function converting fromn string to float t…
ttnghia Feb 18, 2021
4bfd906
Fix the equivalent check function for floating point numbers that inc…
ttnghia Feb 18, 2021
0cfdbcd
Add a test for converting string to double number
ttnghia Feb 18, 2021
7778f85
Merge remote-tracking branch 'origin/branch-0.19' into branch-0.19-is…
ttnghia Feb 18, 2021
a5ede5d
Update copyright header
ttnghia Feb 18, 2021
2584840
Fix const qualifier position and change CMakeLists.txt
ttnghia Feb 18, 2021
ca58e26
Merge branch 'branch-0.19-issue-5225' of github.com:ttnghia/cudf into…
revans2 Feb 18, 2021
c23fdfc
Fix java tests for parsing double values
revans2 Feb 18, 2021
7fadcd0
Some improvement to atof
ttnghia Feb 19, 2021
f3ead3d
Use the results generated from std::stof to test our atof implementation
ttnghia Feb 19, 2021
30e98b3
Add a simple test case
ttnghia Feb 19, 2021
354e65e
Fix convert_floats_benchmark
ttnghia Feb 19, 2021
983d69e
Merge branch 'branch-0.19' into fix_double_parse_tests
revans2 Feb 19, 2021
4a5ca65
Merge branch 'branch-0.19-issue-5225' of github.com:ttnghia/cudf into…
revans2 Feb 19, 2021
0130e51
Merge remote-tracking branch 'origin/branch-0.19' into branch-0.19-is…
ttnghia Feb 19, 2021
dd7139e
Merge branch 'branch-0.19-issue-5225' of github.com:ttnghia/cudf into…
revans2 Feb 19, 2021
d411f7f
Merge remote-tracking branch 'origin/branch-0.19' into branch-0.19-is…
ttnghia Feb 20, 2021
8f5a580
Improve accuracy by casting from string to double then to float
ttnghia Feb 20, 2021
0c6fbb0
Merge branch 'branch-0.19-issue-5225' of github.com:ttnghia/cudf into…
revans2 Feb 22, 2021
a112a1b
Change the template function `stof` to a regular function `stod`.
ttnghia Feb 22, 2021
f8077a7
Re-instating important comment
ttnghia Feb 22, 2021
1238dba
Minor change in handling nan in typed_element_not_equivalent struct
ttnghia Feb 22, 2021
4aaa83f
Simplify data generation and change ranges for the benchmarks
ttnghia Feb 22, 2021
6b1af3b
Merge remote-tracking branch 'origin/branch-0.19' into branch-0.19-is…
ttnghia Feb 23, 2021
3a7c52e
Update header format for cpp/benchmarks/string/convert_floats_benchma…
ttnghia Feb 23, 2021
bb9595e
Merge branch 'branch-0.19' into fix_double_parse_tests
revans2 Mar 1, 2021
587127c
Merge branch 'branch-0.19-issue-5225' of github.com:ttnghia/cudf into…
revans2 Mar 1, 2021
d32c9b6
Updated tests
revans2 Mar 1, 2021
c1bb3f7
Merge branch 'branch-0.19' into fix_double_parse_tests
revans2 Mar 2, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 23 additions & 8 deletions java/src/test/java/ai/rapids/cudf/ColumnVectorTest.java
Original file line number Diff line number Diff line change
Expand Up @@ -3229,30 +3229,45 @@ void testIsFloat() {
String[] floatStrings = {"A", "nan", "Inf", "-Inf", "Infinity", "infinity", "-0.0", "0.0",
"3.4028235E38", "3.4028236E38", "-3.4028235E38", "-3.4028236E38", "1.2e-24", "NULL", "null",
null, "423"};
String[] doubleStrings = {"A", "nan", "Inf", "-Inf", "Infinity", "infinity", "-0.0", "0.0",
"1.7976931348623159E308", "1.7976931348623160E308", "-1.7976931348623159E308",
"-1.7976931348623160E308", "1.2e-234", "NULL", "null", null, "423"};
try (ColumnVector floatStringCV = ColumnVector.fromStrings(floatStrings);
ColumnVector doubleStringCV = ColumnVector.fromStrings(doubleStrings);
ColumnVector isFloat = floatStringCV.isFloat();
ColumnVector isDouble = doubleStringCV.isFloat();
ColumnVector doubles = doubleStringCV.asDoubles();
ColumnVector floats = floatStringCV.asFloats();
ColumnVector expectedFloats = ColumnVector.fromBoxedFloats(0f, 0f, Float.POSITIVE_INFINITY,
Float.NEGATIVE_INFINITY, 0f, 0f, -0f, 0f, Float.MAX_VALUE, Float.POSITIVE_INFINITY,
-Float.MAX_VALUE, Float.NEGATIVE_INFINITY, 1.2e-24f, 0f, 0f, null, 423f);
ColumnVector expected = ColumnVector.fromBoxedBooleans(false, false, true, true, false,
false, true, true, true, true, true, true, true, false, false, null, true)) {
assertColumnsAreEqual(expected, isFloat);
assertColumnsAreEqual(expectedFloats, floats);
}
}

@Test
void testIsDouble() {
String[] doubleStrings = {"A", "nan", "Inf", "-Inf", "Infinity", "infinity", "-0.0", "0.0",
"1.7976931348623157E308",
// Current CUDF Code does not detect overflow for this. "1.7976931348623158E308",
// So we make it a little larger for this test
"1.7976931348623159E308",
"-1.7976931348623157E308",
// Current CUDF Code does not detect overflow for this. "-1.7976931348623158E308",
// So we make it a little larger for this test
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

smaller?

"-1.7976931348623159E308",
"1.2e-234", "NULL", "null", null, "423"};
try (ColumnVector doubleStringCV = ColumnVector.fromStrings(doubleStrings);
ColumnVector isDouble = doubleStringCV.isFloat();
ColumnVector doubles = doubleStringCV.asDoubles();
ColumnVector expectedDoubles = ColumnVector.fromBoxedDoubles(0d, 0d,
Double.POSITIVE_INFINITY, Double.NEGATIVE_INFINITY, 0d, 0d, -0d, 0d, Double.MAX_VALUE,
Double.POSITIVE_INFINITY, -Double.MAX_VALUE, Double.NEGATIVE_INFINITY, 1.2e-234d, 0d,
0d, null, 423d);
ColumnVector expected = ColumnVector.fromBoxedBooleans(false, false, true, true, false,
false, true, true, true, true, true, true, true, false, false, null, true)) {
assertColumnsAreEqual(expected, isFloat);
assertColumnsAreEqual(expected, isDouble);
assertColumnsAreEqual(expectedFloats, floats);
assertColumnsAreEqual(expectedDoubles, doubles);
}
}

@Test
void testCreateDurationDays() {
Integer[] days = {100, 10, 23, 1, -1, 0, Integer.MAX_VALUE, null, Integer.MIN_VALUE};
Expand Down