-
Notifications
You must be signed in to change notification settings - Fork 2.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[HUDI-113]: Use Pair over # delimited string #672
[HUDI-113]: Use Pair over # delimited string #672
Conversation
b79c121
to
81c2ad7
Compare
See some errors due to casting mismatch.. High level question... I see that sorting a List already provides records ordering by the first and second level (in that order). So, can we just go back to |
Essentially, I saying the following should be sufficient information?
This will also reduce shuffled data.. a nice bonus.. |
hoodie-client/src/main/java/com/uber/hoodie/index/bloom/HoodieBloomIndex.java
Outdated
Show resolved
Hide resolved
81c2ad7
to
507c2fe
Compare
hoodie-client/src/main/java/com/uber/hoodie/index/bloom/BucketizedBloomCheckPartitioner.java
Show resolved
Hide resolved
hoodie-client/src/main/java/com/uber/hoodie/index/bloom/HoodieBloomIndex.java
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Minor style nits. lgtm otherwise. Can merge into the next release if you can confirm these
hoodie-client/src/main/java/com/uber/hoodie/index/bloom/BucketizedBloomCheckPartitioner.java
Show resolved
Hide resolved
hoodie-client/src/main/java/com/uber/hoodie/index/bloom/HoodieBloomIndex.java
Outdated
Show resolved
Hide resolved
hoodie-client/src/main/java/com/uber/hoodie/index/bloom/HoodieBloomIndex.java
Outdated
Show resolved
Hide resolved
}); | ||
return recordComparisons; | ||
}).flatMapToPair(List::iterator); | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nice clean up
507c2fe
to
c60a001
Compare
e554ea9
to
e8ddc55
Compare
Apologies for the number of rebases. At this point, I think this is good. Settled on using |
Co-authored-by: rmahindra123 <[email protected]>
@vinothchandar
As outlined in HUDI-113, I've updated the bloom index to use a Pair rather than "#" as the separator.