Storages: Add whether has null in the result of MinMaxIndex #9288
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
ti-chi-bot
bot
added
do-not-merge/needs-linked-issue
release-note-none
JinheLin
changed the title
ti-chi-bot
bot
removed
do-not-merge/work-in-progress
There was a problem hiding this comment.
How about
class RSResult
{
private:
enum class HitState : UInt8
{
Unknown = 0,
None = 1,
Some = 2,
All = 3,
};
bool has_null;
public:
HitState hit_state;
RSResult()
: has_null(false)
, hit_state(HitState::Unknown)
{}
RSResult(HitState hit_state_, bool has_null_)
: has_null(has_null_)
, hit_state(hit_state_)
{}
// NOLINTBEGIN (readability-identifier-naming）
static RSResult None() { return RSResult(HitState::None, false); }
static RSResult Some() { return RSResult(HitState::Some, false); }
static RSResult All() { return RSResult(HitState::All, false); }
static RSResult NoneWithNull() { return RSResult(HitState::None, true); }
static RSResult SomeWithNull() { return RSResult(HitState::Some, true); }
static RSResult AllWithNull() { return RSResult(HitState::All, true); }
// NOLINTEND
RSResult operator||(const RSResult & other) const
{
if (hit_state == HitState::All || other.hit_state == HitState::All)
return RSResult(HitState::All, false);
auto result
= (hit_state == HitState::Some || other.hit_state == HitState::Some) ? HitState::Some : HitState::None;
return RSResult(result, has_null || other.has_null);
}
RSResult operator&&(const RSResult & other) const
{
if (hit_state == HitState::None || other.hit_state == HitState::None)
return RSResult(HitState::None, false);
auto result = (hit_state == HitState::All && other.hit_state == HitState::All) ? HitState::All : HitState::Some;
return RSResult(result, has_null || other.has_null);
}
RSResult operator!() const
{
switch (hit_state)
{
case HitState::Some:
return RSResult(HitState::Some, has_null);
case HitState::None:
return RSResult(HitState::All, has_null);
case HitState::All:
return RSResult(HitState::None, has_null);
default:
throw Exception("Unknow RSResult: {}", static_cast<UInt8>(hit_state));
}
}
bool needToRead() const { return hit_state != HitState::None; }
bool needToFilter() const { return hit_state == HitState::All && !has_null; }
};
@Lloyd-Pottiger I have implemented a similar version before. It affetcs too many code including the tests. It is better to write another PR to refine |
|
@JinheLin JinheLin#3 seems the size of the changes is acceptable |
JinheLin
force-pushed
the
rs_result_add_null
branch
from
bed7cc9 to
3352754
Compare
JinheLin
commented
Aug 8, 2024
| @@ -169,6 +169,14 @@ static_assert( | |||
|
|
|||
| static constexpr bool DM_RUN_CHECK = true; | |||
|
|
|||
| struct Attr | |||
There was a problem hiding this comment.
Move struct Attr from RSResult.h to DeltaMergeDefines.h for simplifing the headers that RSResult.h included.
JinheLin
commented
Aug 8, 2024
| @@ -634,8 +634,8 @@ bool shouldCompactStableWithTooMuchDataOutOfSegmentRange( | |||
| "GC - shouldCompactStableWithTooMuchDataOutOfSegmentRange checked false " | |||
| "because segment DTFile is shared with a neighbor segment, " | |||
| "first_pack_inc={} last_pack_inc={} prev_seg_files=[{}] next_seg_files=[{}] my_files=[{}] segment={}", | |||
| magic_enum::enum_name(at_least_result.first_pack_intersection), | |||
| magic_enum::enum_name(at_least_result.last_pack_intersection), | |||
| at_least_result.first_pack_intersection, | |||
There was a problem hiding this comment.
RSResult is a class now, and formatter has beed implemented in RSResult.h.
|
@Lloyd-Pottiger @JaySon-Huang PTAL. |
ti-chi-bot
bot
added
needs-1-more-lgtm
|
/retest |
JaySon-Huang
approved these changes
Aug 13, 2024
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: JaySon-Huang, Lloyd-Pottiger The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
ti-chi-bot
bot
added
lgtm
and removed
needs-1-more-lgtm
[LGTM Timeline notifier]Timeline:
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What problem does this PR solve?
Issue Number: ref #9103
Problem Summary:
In the current implementation, the filtering results returned by
MinMaxIndexdo not take into account the presence of null values. That is to say, ifRSResult:: Allis returned, it is still necessary to filter null values instead of skipping this filtering calculation directly.What is changed and how it works?
This PR adds information about whether there is a null value in a pack in the filtering results returned by the
MinMaxIndex.Check List
Tests
Side effects
Documentation
Release note