Skip to content

Commit

Permalink
Update documentation for enable-coordinator-dynamic-filters-distribution
Browse files Browse the repository at this point in the history
  • Loading branch information
raunaqmorarka authored and sopel39 committed Jul 12, 2021
1 parent 4bab587 commit 90e3fd2
Showing 1 changed file with 12 additions and 6 deletions.
18 changes: 12 additions & 6 deletions docs/src/main/sphinx/admin/dynamic-filtering.rst
Original file line number Diff line number Diff line change
Expand Up @@ -23,11 +23,6 @@ from the processed dimension table on the right side of join. In the case of bro
the runtime predicates generated from this collection are pushed into the local table scan
on the left side of the join running on the same worker.

Dynamic filtering is enabled by default using the ``enable-dynamic-filtering``
configuration property. To disable dynamic filtering, set the configuration
property to ``false``. Alternatively, use the session property
``enable_dynamic_filtering``.

Additionally, these runtime predicates are communicated to the coordinator over the network
so that dynamic filtering can also be performed on the coordinator during enumeration of
table scan splits.
Expand All @@ -36,12 +31,24 @@ For example, in the case of the Hive connector, dynamic filters are used
to skip loading of partitions which don't match the join criteria.
This is known as **dynamic partition pruning**.

After completing the collection of dynamic filters, the coordinator also distributes them
to worker nodes over the network for partitioned joins. This allows push down of dynamic
filters from partitioned joins into the table scans on the left side of that join.
Distribution of dynamic filters from the coordinator to workers is enabled by default.
It can be disabled by setting either the ``enable-coordinator-dynamic-filters-distribution``
configuration property, or the session property
``enable_coordinator_dynamic_filters_distribution`` to ``false``.

The results of dynamic filtering optimization can include the following benefits:

* improved overall query performance
* reduced network traffic between Trino and the data source
* reduced load on the remote data source

Dynamic filtering is enabled by default. It can be disabled by setting either the
``enable-dynamic-filtering`` configuration property, or the session property
``enable_dynamic_filtering`` to ``false``.

Support for push down of dynamic filters is specific to each connector,
and the relevant underlying database or storage system. The documentation for
specific connectors with support for dynamic filtering includes further details,
Expand Down Expand Up @@ -246,7 +253,6 @@ of selected rows from the dimension table.
Limitations
-----------

* Push down of dynamic filters into local table scan on worker nodes is limited to broadcast joins.
* Min-max dynamic filter collection is not supported for ``DOUBLE``, ``REAL`` and unorderable data types.
* Dynamic filtering is not supported for ``DOUBLE`` and ``REAL`` data types when using ``IS NOT DISTINCT FROM`` predicate.
* Dynamic filtering is supported when the join key contains a cast from the build key type to the
Expand Down

0 comments on commit 90e3fd2

Please sign in to comment.