-
Notifications
You must be signed in to change notification settings - Fork 486
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
ORC-1695: Upgrade gson to 2.10.1 #1897
Conversation
"loadFactor": 0.024791667237877846, | ||
"expectedFpp": 5.756256582500896E-12 | ||
"loadFactor": 0.024791667, | ||
"expectedFpp": 5.7562566E-12 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The com.google.gson.stream.JsonWriter#value(float)
method is not provided in gson 1.9.0, but is provided in gson 1.9.1.
This causes the test to fail after upgrading gson, because in version 2.9.0, the generated json float will be implicitly converted to double.
json format
orc/java/tools/src/java/org/apache/orc/tools/JsonFileDump.java
Lines 426 to 432 in bd2af54
float loadFactor = (float) popCount / (float) bitCount; | |
float expectedFpp = (float) Math.pow(loadFactor, k); | |
writer.name("numHashFunctions").value(k); | |
writer.name("bitCount").value(bitCount); | |
writer.name("popCount").value(popCount); | |
writer.name("loadFactor").value(loadFactor); | |
writer.name("expectedFpp").value(expectedFpp); |
not json format
orc/java/tools/src/java/org/apache/orc/tools/FileDump.java
Lines 719 to 726 in bd2af54
float loadFactor = (float) popCount / (float) bitCount; | |
float expectedFpp = (float) Math.pow(loadFactor, k); | |
DecimalFormat df = new DecimalFormat("###.####"); | |
sb.append(" numHashFunctions: ").append(k); | |
sb.append(" bitCount: ").append(bitCount); | |
sb.append(" popCount: ").append(popCount); | |
sb.append(" loadFactor: ").append(df.format(loadFactor)); | |
sb.append(" expectedFpp: ").append(expectedFpp); |
For other test files in non-json format, we can see that the loadFactor is float.
orc/java/tools/src/test/resources/orc-file-dump-bloomfilter2.out
Lines 68 to 69 in bd2af54
Entry 0: numHashFunctions: 7 bitCount: 9600 popCount: 4931 loadFactor: 0.5136 expectedFpp: 0.009432924 | |
Entry 1: numHashFunctions: 7 bitCount: 9600 popCount: 4956 loadFactor: 0.5163 expectedFpp: 0.009772834 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
+1, LGTM. Although it changes the value, it looks reasonable to me.
### What changes were proposed in this pull request? This PR aims to upgrade `Gson` to `2.10.1`. ### Why are the changes needed? Jan 7, 2023 https://github.com/google/gson/releases/tag/gson-parent-2.10.1 Oct 25, 2022 https://github.com/google/gson/releases/tag/gson-parent-2.10 Aug 1, 2022 https://github.com/google/gson/releases/tag/gson-parent-2.9.1 ### How was this patch tested? GA ### Was this patch authored or co-authored using generative AI tooling? No Closes #1897 from cxzl25/ORC-1695. Authored-by: sychen <[email protected]> Signed-off-by: Dongjoon Hyun <[email protected]> (cherry picked from commit dc89f18) Signed-off-by: Dongjoon Hyun <[email protected]>
Merged to main/2.0 |
What changes were proposed in this pull request?
This PR aims to upgrade
Gson
to2.10.1
.Why are the changes needed?
Jan 7, 2023
https://github.com/google/gson/releases/tag/gson-parent-2.10.1
Oct 25, 2022
https://github.com/google/gson/releases/tag/gson-parent-2.10
Aug 1, 2022
https://github.com/google/gson/releases/tag/gson-parent-2.9.1
How was this patch tested?
GA
Was this patch authored or co-authored using generative AI tooling?
No