Skip to content

Commit

Permalink
Avoid dynamic filter current predicate computation when not used
Browse files Browse the repository at this point in the history
`DynamicFilter.getCurrentPredicate` is not free to call. Do not call it
when results wouldn't be used.

The edge case, where `getCurrentPredicate()` returns
`TupleDomain.none()` is not worth optimizing for. In the future, it can
be handled by the engine instead.
  • Loading branch information
findepi committed Nov 25, 2021
1 parent 8274da8 commit ef5084c
Showing 1 changed file with 4 additions and 1 deletion.
Original file line number Diff line number Diff line change
Expand Up @@ -370,7 +370,10 @@ private ListenableFuture<Void> loadPartition(HivePartitionMetadata partition)
TupleDomain<HiveColumnHandle> effectivePredicate = compactEffectivePredicate.transformKeys(HiveColumnHandle.class::cast);

List<HiveColumnHandle> partitionColumns = getPartitionKeyColumnHandles(table, typeManager);
BooleanSupplier partitionMatchSupplier = () -> partitionMatches(partitionColumns, dynamicFilter.getCurrentPredicate(), hivePartition);
BooleanSupplier partitionMatchSupplier =
partitionColumns.stream().noneMatch(dynamicFilter.getColumnsCovered()::contains)
? () -> true
: () -> partitionMatches(partitionColumns, dynamicFilter.getCurrentPredicate(), hivePartition);
if (!partitionMatchSupplier.getAsBoolean()) {
// Avoid listing files and creating splits from a partition if it has been pruned due to dynamic filters
return COMPLETED_FUTURE;
Expand Down

0 comments on commit ef5084c

Please sign in to comment.