Skip to content

Commit

Permalink
Optimize partitionNotNullPositions
Browse files Browse the repository at this point in the history
Split one complicated loop inside partitionNotNullPositions
into two simple ones.
before
Benchmark                                   (channelCount)  (enableCompression)  (nullRate)  (partitionCount)  (positionCount)  (type)  Mode  Cnt    Score    Error  Units
BenchmarkPartitionedOutputOperator.addPage               1                false           0                16             8192  BIGINT  avgt   10  642.430 ± 24.573  ms/op

after
Benchmark                                   (channelCount)  (enableCompression)  (nullRate)  (partitionCount)  (positionCount)  (type)  Mode  Cnt    Score   Error  Units
BenchmarkPartitionedOutputOperator.addPage               1                false           0                16             8192  BIGINT  avgt   10  484.876 ± 4.875  ms/op
  • Loading branch information
lukasz-stec authored and sopel39 committed May 26, 2022
1 parent 65dd2af commit 8e6ff07
Showing 1 changed file with 8 additions and 2 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -434,9 +434,15 @@ private IntArrayList[] partitionNullablePositions(Page page, int position, IntAr

private IntArrayList[] partitionNotNullPositions(Page page, int startingPosition, IntArrayList[] partitionPositions, IntUnaryOperator partitionFunction)
{
for (int position = startingPosition; position < page.getPositionCount(); position++) {
int positionCount = page.getPositionCount();
int[] partitionPerPosition = new int[positionCount];
for (int position = startingPosition; position < positionCount; position++) {
int partition = partitionFunction.applyAsInt(position);
partitionPositions[partition].add(position);
partitionPerPosition[position] = partition;
}

for (int position = startingPosition; position < positionCount; position++) {
partitionPositions[partitionPerPosition[position]].add(position);
}

return partitionPositions;
Expand Down

0 comments on commit 8e6ff07

Please sign in to comment.