RowKey-Agnostic ColumnSource Optimizations #3329
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Port DH-12201: Create BooleanSingleValueSource Port DH-12310: SingleValue Column Sources Must Handle NULL_ROW_KEY
nbauernfeind
added
query engine
NoDocumentationNeeded
ReleaseNotesNeeded
rcaudy
reviewed
Jan 19, 2023
engine/table/src/main/java/io/deephaven/engine/table/impl/SortOperation.java
Outdated
Show resolved
Hide resolved
...engine/table/impl/by/IncrementalChunkedOperatorAggregationStateManagerOpenAddressedBase.java
Show resolved
Hide resolved
engine/table/src/main/java/io/deephaven/engine/table/impl/sources/BooleanSingleValueSource.java
Show resolved
Hide resolved
.../main/java/io/deephaven/engine/table/impl/sources/immutable/ImmutableConstantCharSource.java
Outdated
Show resolved
Hide resolved
.../main/java/io/deephaven/engine/table/impl/sources/immutable/ImmutableConstantCharSource.java
Show resolved
Hide resolved
engine/table/src/main/java/io/deephaven/engine/table/impl/sources/RedirectedColumnSource.java
Outdated
Show resolved
Hide resolved
...e/table/src/main/java/io/deephaven/engine/table/impl/sources/CharacterSingleValueSource.java
Outdated
Show resolved
Hide resolved
| @Override | ||
| public void fillPrevChunk(@NotNull FillContext context, | ||
| @NotNull WritableChunk<? super Values> destination, @NotNull RowSequence rowSequence) { | ||
| // We can only hold one value, fill the chunk with the value obtained from an arbitrarily valid rowKey |
There was a problem hiding this comment.
Same question, except prev, although at least prev has logic you might want to avoid replicating.
There was a problem hiding this comment.
changed up the comment - but yeah, it feels easier to maintain to not duplicate
...e/table/src/main/java/io/deephaven/engine/table/impl/sources/CharacterSingleValueSource.java
Outdated
Show resolved
Hide resolved
...e/table/src/main/java/io/deephaven/engine/table/impl/sources/CrossJoinRightColumnSource.java
Outdated
Show resolved
Hide resolved
…Operation.java Co-authored-by: Ryan Caudy <[email protected]>
cpwright
approved these changes
Jan 19, 2023
rcaudy
approved these changes
Jan 19, 2023
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There are some cases where we do not actually need to wrap column sources with redirections. These three sets of
ColumnSources are agnostic to the row key:SingleValue,ImmutableConstant, andNullValue. Since all rows are identical for columns of these types we can avoid wrapping them in scenarios where we do not rely onNULL_ROW_KEYmappings.We do need
NULL_ROW_KEYmappings for naturalJoin, leftOuterJoin, and asOfJoin. It may make sense for me to back-out themaybeRedirectcalls in some of the hash table, and/or aggregation, implementations when we know that we've created column sources that will always be redirected.This change set includes some of the performance improvements suggested in #2425.
Includes the following DHE ports:
Nightlies ran here. They failed normal Check CI due to spotless. I've squashed the spotless fix into the commit already; and believe that we can use the CI run from regular PR actions to confirm they pass.