-
Notifications
You must be signed in to change notification settings - Fork 28.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[SPARK-49909][SQL] Fix the pretty name of some expressions #48385
Conversation
+1, LGTM. Merging to master. |
@panbingkun I think we should backport the fix. Could you open separate PRs, please, because your changes cause conflicts in |
Sure, allow me to complete it. |
### What changes were proposed in this pull request? The pr aims to fix the `pretty name` of some `expressions`, includes: `random`, `to_varchar`, `current_database`, `curdate`, `dateadd` and `array_agg`. (PS: The pr is backport branch-3.5, master pr is: #48385) ### Why are the changes needed? The actual function name used does not match the displayed name, as shown below: - Before: <img width="573" alt="image" src="https://github.com/user-attachments/assets/f5785c80-f6cb-494f-a15e-9258eca688a7"> - After: <img width="570" alt="image" src="https://github.com/user-attachments/assets/792a7092-ccbf-49f4-a616-19110e5c2361"> ### Does this PR introduce _any_ user-facing change? Yes, Make the header of the data seen by the end-user from `Spark SQL` consistent with the `actual function name` used. ### How was this patch tested? - Pass GA. - Update existed UT. ### Was this patch authored or co-authored using generative AI tooling? No. Closes #48393 from panbingkun/branch-3.5_SPARK-49909. Authored-by: panbingkun <[email protected]> Signed-off-by: Max Gekk <[email protected]>
### What changes were proposed in this pull request? The pr aims to fix the `pretty name` of some `expressions`, includes: `random`, `to_varchar`, `current_database`, `curdate`, `dateadd` and `array_agg`. (PS: The pr is backport branch-3.4, master pr is: #48385) ### Why are the changes needed? The actual function name used does not match the displayed name, as shown below: - Before: <img width="573" alt="image" src="https://github.com/user-attachments/assets/f5785c80-f6cb-494f-a15e-9258eca688a7"> - After: <img width="570" alt="image" src="https://github.com/user-attachments/assets/792a7092-ccbf-49f4-a616-19110e5c2361"> ### Does this PR introduce _any_ user-facing change? Yes, Make the header of the data seen by the end-user from `Spark SQL` consistent with the `actual function name` used. ### How was this patch tested? - Pass GA. - Update existed UT. ### Was this patch authored or co-authored using generative AI tooling? No. Closes #48396 from panbingkun/branch-3.4_SPARK-49909. Authored-by: panbingkun <[email protected]> Signed-off-by: Max Gekk <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The benefit of this fix is minimal, but it will disrupt the queries of many end users because they rely on the auto-generated alias.
![]() spark-sql (default)> select random("1");
[DATATYPE_MISMATCH.UNEXPECTED_INPUT_TYPE] Cannot resolve "rand(1)" due to data type mismatch: The first parameter requires the ("INT" or "BIGINT") type, however "1" has the type "STRING". SQLSTATE: 42K09; line 1 pos 7;
'Project [unresolvedalias(random(1))]
+- OneRowRelation
|
revert pr: #48530 |
After the master branch merges the revert PR, I will submit PRs to |
### What changes were proposed in this pull request? The pr aims to revert #48385. This reverts commit 52538f0. ### Why are the changes needed? When upgrading spark from `an old version` to `the latest version`, some end-users may rely on the `original schema` (`although it may not be correct`), which can make the `upgrade` very difficult. so, let's first restore it to its original state. ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? Pass GA ### Was this patch authored or co-authored using generative AI tooling? No. Closes #48530 from panbingkun/SPARK-49909_revert. Authored-by: panbingkun <[email protected]> Signed-off-by: Max Gekk <[email protected]>
### What changes were proposed in this pull request? The pr aims to fix the `pretty name` of some `expressions`, includes: `random`, `to_varchar`, `current_database`, `curdate`, `dateadd` and `array_agg`. ### Why are the changes needed? The actual function name used does not match the displayed name, as shown below: - Before: <img width="573" alt="image" src="https://github.com/user-attachments/assets/f5785c80-f6cb-494f-a15e-9258eca688a7"> - After: <img width="570" alt="image" src="https://github.com/user-attachments/assets/792a7092-ccbf-49f4-a616-19110e5c2361"> ### Does this PR introduce _any_ user-facing change? Yes, Make the header of the data seen by the end-user from `Spark SQL` consistent with the `actual function name` used. ### How was this patch tested? - Pass GA. - Update existed UT. ### Was this patch authored or co-authored using generative AI tooling? No. Closes apache#48385 from panbingkun/SPARK-49909. Authored-by: panbingkun <[email protected]> Signed-off-by: Max Gekk <[email protected]>
### What changes were proposed in this pull request? The pr aims to revert apache#48385. This reverts commit 52538f0. ### Why are the changes needed? When upgrading spark from `an old version` to `the latest version`, some end-users may rely on the `original schema` (`although it may not be correct`), which can make the `upgrade` very difficult. so, let's first restore it to its original state. ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? Pass GA ### Was this patch authored or co-authored using generative AI tooling? No. Closes apache#48530 from panbingkun/SPARK-49909_revert. Authored-by: panbingkun <[email protected]> Signed-off-by: Max Gekk <[email protected]>
What changes were proposed in this pull request?
The pr aims to fix the
pretty name
of someexpressions
, includes:random
,to_varchar
,current_database
,curdate
,dateadd
andarray_agg
.Why are the changes needed?
The actual function name used does not match the displayed name, as shown below:
Does this PR introduce any user-facing change?
Yes, Make the header of the data seen by the end-user from
Spark SQL
consistent with theactual function name
used.How was this patch tested?
Was this patch authored or co-authored using generative AI tooling?
No.