-
Notifications
You must be signed in to change notification settings - Fork 1.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add Ballista examples #775
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
the scheduler overview section is very clear 👍
|
||
// execute the query - note that calling collect on the DataFrame | ||
// trait will execute the query with DataFusion so we have to call | ||
// collect on the BallistaContext instead and pass it the DataFusion |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
👍
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM -- documentation for the win!
ballista-examples/Cargo.toml
Outdated
edition = "2018" | ||
publish = false | ||
|
||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I wonder if it is worth adding bin
targets here?
As it is I can't run these examples:
(arrow_dev) alamb@MacBook-Pro:~/Software/arrow-datafusion/ballista-examples$ cargo run
error: a bin target must be available for `cargo run`
Maybe something like
[[bin]]
name = "dataframe"
path = "src/ballista_dataframe.rs"
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I was following the same pattern that we use in datafusion-examples
where we use cargo run --example
rather than cargo run --bin
.
% cargo run --example
error: "--example" takes one argument.
Available examples:
ballista-dataframe
ballista-sql
It is a little odd that we package the examples in their own crate, so maybe packaging them as binaries makes more sense now?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The main motivation from extracting them into a separate folder/crate for datafusion-examples was to reduce the nr of dependencies and compilation time.
Maybe bin works just as well?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I have updated this to use --bin
now.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🎉
Co-authored-by: Andrew Lamb <[email protected]>
Which issue does this PR close?
Closes #774 .
Rationale for this change
Examples help new users get started.
What changes are included in this PR?
Are there any user-facing changes?
No