Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for Jupyter notebooks in Ballista #42

Open
andygrove opened this issue Apr 24, 2021 · 4 comments
Open

Add support for Jupyter notebooks in Ballista #42

andygrove opened this issue Apr 24, 2021 · 4 comments
Labels
enhancement New feature or request

Comments

@andygrove
Copy link
Member

Is your feature request related to a problem or challenge? Please describe what you are trying to do.

Jupyter notebooks are a popular tool for interacting with data science and "Big Data" systems and it would be nice if we could add support for this in Ballista.

Describe the solution you'd like

It probably makes sense to support writing Python code in the notebook, so this would depend on having Python bindings available for DataFusion & Ballista (see https://pypi.org/project/datafusion/).

Describe alternatives you've considered
None

Additional context
None

@andygrove andygrove added the enhancement New feature or request label Apr 24, 2021
@drauschenbach
Copy link

I've investigated Python and JavaScript notebooks for Ballista, and think that Rust notebooks would be the low-hanging fruit and natural starting point:

https://github.com/google/evcxr/blob/main/evcxr_jupyter/README.md

EvCxR supports one-liner crate imports, and that's how the Ballista Client could be integrated, once apache/datafusion#509 and apache/datafusion#234 are addressed.

Since EvCxR already exists, maybe this issue should be renamed to focus more specifically on a Python-specific Jupyter kernel.

@jorgecarleitao
Copy link
Member

An alternative is to add support to Ballista to the Python bindings. I am not sure about the benefit of Rust itself in the notebook; in the notebook people usually favor interpreted languages, that do not have the compiling overhead.

An interface common for DataFusion and Ballista context, e.g. via a trait, and use Box<dyn Context> on the Python binding to use one or the other should be enough?

@xudong963
Copy link
Member

FYI, databend has supportted https://databend.rs/doc/integrations/gui-tool/jupyter#what-is-jupyter-notebook :)

@andygrove andygrove transferred this issue from apache/datafusion May 26, 2022
@nl5887
Copy link
Contributor

nl5887 commented Jun 6, 2022

Related to #58

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

5 participants