Skip to content
Celeborn CI

[WIP][CELEBORN-1319] Optimize skew partition logic for Reduce Mode to avoid sorting shuffle files #12609

Sign in to view logs

[WIP][CELEBORN-1319] Optimize skew partition logic for Reduce Mode to avoid sorting shuffle files

[WIP][CELEBORN-1319] Optimize skew partition logic for Reduce Mode to avoid sorting shuffle files #12609

Run time

Learn about OS pricing on GitHub Actions
Job Run time
flink (8, 1.14)
9m 42s
mr (8)
11m 49s
flink (8, 1.15)
10m 48s
mr (11)
12m 18s
flink (8, 1.16)
9m 39s
flink (8, 1.17)
10m 26s
service (8)
12m 14s
flink (8, 1.18)
10m 24s
service (11)
12m 12s
flink (8, 1.19)
9m 49s
service (17)
12m 30s
flink (8, 1.20)
10m 50s
flink (11, 1.14)
9m 58s
spark2 (8, 2.4)
35m 40s
flink (11, 1.15)
10m 1s
spark3 (8, 3.0, org.apache.spark.shuffle.sort.io.LocalDiskShuffleDataIO)
39m 21s
flink (11, 1.16)
10m 21s
spark3 (8, 3.1, org.apache.spark.shuffle.sort.io.LocalDiskShuffleDataIO)
38m 56s
flink (11, 1.17)
10m 11s
spark3 (8, 3.2, org.apache.spark.shuffle.sort.io.LocalDiskShuffleDataIO)
36m 49s
flink (11, 1.18)
9m 55s
spark3 (8, 3.3, org.apache.spark.shuffle.sort.io.LocalDiskShuffleDataIO)
43m 38s
flink (11, 1.19)
9m 58s
spark3 (8, 3.4, org.apache.spark.shuffle.sort.io.LocalDiskShuffleDataIO)
41m 35s
flink (11, 1.20)
10m 47s
spark3 (8, 3.5, org.apache.spark.shuffle.sort.io.LocalDiskShuffleDataIO)
43m 29s
spark3 (8, 3.5, org.apache.spark.shuffle.celeborn.CelebornShuffleDataIO)
46m 43s
spark3 (11, 3.0, org.apache.spark.shuffle.sort.io.LocalDiskShuffleDataIO)
38m 1s
spark3 (11, 3.1, org.apache.spark.shuffle.sort.io.LocalDiskShuffleDataIO)
38m 12s
spark3 (11, 3.2, org.apache.spark.shuffle.sort.io.LocalDiskShuffleDataIO)
36m 56s
spark3 (11, 3.3, org.apache.spark.shuffle.sort.io.LocalDiskShuffleDataIO)
38m 11s
spark3 (11, 3.4, org.apache.spark.shuffle.sort.io.LocalDiskShuffleDataIO)
38m 49s
spark3 (11, 3.5, org.apache.spark.shuffle.sort.io.LocalDiskShuffleDataIO)
37m 46s
spark3 (11, 3.5, org.apache.spark.shuffle.celeborn.CelebornShuffleDataIO)
38m 23s
spark3 (17, 3.3, org.apache.spark.shuffle.sort.io.LocalDiskShuffleDataIO)
37m 34s
spark3 (17, 3.4, org.apache.spark.shuffle.sort.io.LocalDiskShuffleDataIO)
38m 3s
spark3 (17, 3.5, org.apache.spark.shuffle.sort.io.LocalDiskShuffleDataIO)
37m 49s
spark3 (17, 3.5, org.apache.spark.shuffle.celeborn.CelebornShuffleDataIO)
36m 47s
15h 46m 34s