Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
[SPARK-50405][SQL] Handle collation type coercion of complex data typ…
…es properly ### What changes were proposed in this pull request? This pull request generalizes collation type coercion to support not just casting all children to a single string type, but also handling complex data types such as structs, maps, and arrays (arrays partially worked already). The core idea is to recursively analyze the entire data type of an expression, annotating each StringType within it with the highest-priority collation and its strength. This annotation propagates upward through the expression tree. Once the root of the expression is reached, the annotations are removed, and the expression is cast to the desired data type. For the root expression `e`, the collation data type context is computed by first calculating the context for all its children and then merging those results into the data type of `e`. ### Why are the changes needed? In #48663, a new approach to calculating collation precedence was introduced. This approach recursively examines the children of an expression and propagates the collation with the highest priority upward. However, the current implementation of collation coercion is limited to determining the StringType that all children should be cast to. This approach falls short when dealing with complex types like structs, maps, and arrays, which can also contain collations. To address this limitation, we need a more general mechanism that allows coercion of any data type, not just simple strings. ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? With new unit tests. ### Was this patch authored or co-authored using generative AI tooling? No. Closes #48936 from stefankandic/collationCoercionComplex. Lead-authored-by: Stefan Kandic <[email protected]> Co-authored-by: Stefan Kandic <[email protected]> Signed-off-by: Max Gekk <[email protected]>
- Loading branch information