[ballista] support date_part and date_turnc ser/de, pass tpch 7 #840
Conversation
| "date_part" | "datepart" => BuiltinScalarFunction::DatePart, | ||
| "date_trunc" | "datetrunc" => BuiltinScalarFunction::DateTrunc, |
There was a problem hiding this comment.
Ballista physical plan ser/de uses function.to_string() to serialize the name, which uses the enum name and resulted in function name without _.
houqp
changed the title
| SHA384 = 31; | ||
| SHA512 = 32; | ||
| LN = 33; | ||
| DATEPART = 27; |
There was a problem hiding this comment.
This makes it a breaking change? Not that it matters much now I guess, but would be good to know when it starts mattering.
There was a problem hiding this comment.
Yep, there is another breaking change in ScalarFunctionNode where i renamed expr field to args. Both of these changes are done purely for improving readability of the code base.
I think we should start to be careful about breaking change when we start to receive interests in using ballista for production PoC. But if any one thinks we shouldn't introduce breaking change for readability even at current stage, I am more than happy to revert these changes.
There was a problem hiding this comment.
I think until we know someone is using the ballista codebase in a way that such a change will affect them, we should not spend extra time striving for backwards compatibility (aka in my opinion this change is fine).
Ballista also looks to me like more of an overall system (aka something people could use directly rather than as a library) which is different than DataFusion. I wonder if internal changes such as these enum values changes are less of an issue with such a system.
There was a problem hiding this comment.
I agree that we don't need to worry too much about breaking changes at this point given the early and experimental nature of Ballista.
| @@ -1140,6 +1140,7 @@ mod tests { | |||
| test_round_trip!(q3, 3); | |||
| test_round_trip!(q5, 5); | |||
| test_round_trip!(q6, 6); | |||
| test_round_trip!(q7, 7); | |||
There was a problem hiding this comment.
What's the status of query 8,9,13,14,19 in Ballista?
Those are now included in the DataFusion CI
There was a problem hiding this comment.
They are all broken in ballista due to different ser/de errors, my plan is to fix them one at a time so we are on par with datafusion.
houqp
added
api change
|
Thanks @houqp |
Rationale for this change
Add date_part and date_trunc ser/de to support tpch 7 in ballista
What changes are included in this PR?
Are there any user-facing changes?