You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
In order to ensure records with logically equivalent keys join successfully when using Schema-Registry-enabled key formats (which may result in different serialized bytes for logically equivalent data keys, due to potential differences in user vs ksqlDB schemas), ksqlDB forces repartitions on both sides of such joins to ensure data are properly co-partitioned (see #6635 for context). However, there's room for improvement since repartitions are currently forced even if not strictly necessary (e.g., if there was already another repartition upstream of the join). We should enhance the join logic to avoid unnecessary repartitions in these cases. Doing so would require passing information from the leaves back towards the root of the join tree, so pre-join repartition nodes will know whether any repartitions have already taken place upstream, thus obviating the need for an additional repartition.
The text was updated successfully, but these errors were encountered:
In order to ensure records with logically equivalent keys join successfully when using Schema-Registry-enabled key formats (which may result in different serialized bytes for logically equivalent data keys, due to potential differences in user vs ksqlDB schemas), ksqlDB forces repartitions on both sides of such joins to ensure data are properly co-partitioned (see #6635 for context). However, there's room for improvement since repartitions are currently forced even if not strictly necessary (e.g., if there was already another repartition upstream of the join). We should enhance the join logic to avoid unnecessary repartitions in these cases. Doing so would require passing information from the leaves back towards the root of the join tree, so pre-join repartition nodes will know whether any repartitions have already taken place upstream, thus obviating the need for an additional repartition.
The text was updated successfully, but these errors were encountered: