Skip to content

Commit

Permalink
Increase writerScalingMinDataProcessed threshold for scaling test
Browse files Browse the repository at this point in the history
Since we track page size for scaling writer
instead of actual compressed output file size,
we need to have a larger threshold for
writerScalingMinDataProcessed. This way we can
ensure that the writer scaling is not triggered.
  • Loading branch information
gaurav8297 authored and raunaqmorarka committed Aug 8, 2023
1 parent a4100f1 commit ac6b3cb
Showing 1 changed file with 3 additions and 1 deletion.
Original file line number Diff line number Diff line change
Expand Up @@ -4185,7 +4185,9 @@ public void testWriterTasksCountLimitPartitionedScaleWritersDisabled()
public void testWriterTasksCountLimitPartitionedScaleWritersEnabled()
{
testLimitWriterTasks(2, 4, true, true, true, DataSize.of(1, MEGABYTE));
testLimitWriterTasks(2, 2, true, true, true, DataSize.of(32, MEGABYTE));
// Since we track page size for scaling writer instead of actual compressed output file size, we need to have a
// larger threshold for writerScalingMinDataProcessed. This way we can ensure that the writer scaling is not triggered.
testLimitWriterTasks(2, 2, true, true, true, DataSize.of(128, MEGABYTE));
}

private void testLimitWriterTasks(int maxWriterTasks, int expectedFilesCount, boolean scaleWritersEnabled, boolean redistributeWrites, boolean partitioned, DataSize writerScalingMinDataProcessed)
Expand Down

0 comments on commit ac6b3cb

Please sign in to comment.