-
Notifications
You must be signed in to change notification settings - Fork 198
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Implement Python bindings for BallistaContext #15
Comments
@andygrove hope to work on this, if you don't have any other plan. |
Thank you @djKooks that would be great |
@andygrove
or should I update current datafusion python binding inside existing |
Yes, I think that makes sense. |
I would be interested to hear what others think though. @alamb @Dandandan @jorgecarleitao @houqp do you have an opinion on this? |
I agree that this makes the most sense 👍 Out of curiosity, do the bindings come with the client and executors, or just the client? |
I agree having a separate python binding for Ballista in the location suggested by @djKooks in #15 makes sense to me |
@alamb @jorgecarleitao @andygrove thanks for suggestion 🙇
I think it will be enough to do with client only in first step, but do you have any more suggestion? |
I do not have any more to add here -- since I don't use the python bindings myself I don't have a lot to offer with specifics |
@andygrove @alamb @jorgecarleitao thanks for comment. |
Related to #58 |
This has now been implemented. |
Is your feature request related to a problem or challenge? Please describe what you are trying to do.
We have Python bindings for DataFusion's ExecutionContext. It would be good to also support Ballista's BallistaContext so that we can use Python to run distributed queries.
Describe the solution you'd like
Probably something like this?
Describe alternatives you've considered
Another approach might be to have ballista be an optional feature of DataFusion and then enable new methods on the DataFusion ExecutionContext instead but that would probably result in tons of additional dependencies and blur the lines between DataFusion and Ballista and I think there is a strong case for DataFusion=lib/embedded and Ballista=distributed.
Additional context
N/A
The text was updated successfully, but these errors were encountered: