Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Benchmark subcommand to distinguish between DataFusion and Ballista #402

Merged
merged 5 commits into from
May 25, 2021

Conversation

jgoday
Copy link
Contributor

@jgoday jgoday commented May 23, 2021

Which issue does this PR close?

Closes #401.

Rationale for this change

What changes are included in this PR?

It defines a BenchmarkSubCommandOpt to allow different arguments depending on whether it is run against ballista or datafusion.

Are there any user-facing changes?

It changes tpch bin arugments generated by benchmarks project.
It contains changes in benchmarks/README.md and benchmars/run.sh files.

@codecov-commenter
Copy link

Codecov Report

Merging #402 (c939c19) into master (db4f098) will increase coverage by 0.00%.
The diff coverage is 0.00%.

Impacted file tree graph

@@           Coverage Diff           @@
##           master     #402   +/-   ##
=======================================
  Coverage   74.94%   74.94%           
=======================================
  Files         146      146           
  Lines       24314    24314           
=======================================
+ Hits        18221    18223    +2     
+ Misses       6093     6091    -2     
Impacted Files Coverage Δ
benchmarks/src/bin/tpch.rs 30.82% <0.00%> (ø)
datafusion/src/logical_plan/expr.rs 84.34% <0.00%> (ø)
datafusion/src/physical_plan/hash_aggregate.rs 85.21% <0.00%> (+0.36%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update db4f098...c939c19. Read the comment docs.

@andygrove
Copy link
Member

There are two other files which will need updating:

  • docs/user-guide/src/distributed/raspberrypi.md
  • benchmarks/run.sh

@jgoday
Copy link
Contributor Author

jgoday commented May 24, 2021

There are two other files which will need updating:

* docs/user-guide/src/distributed/raspberrypi.md

* benchmarks/run.sh

Fixed DataFusionBenchmarkOpts name and raspberrypi.md in last commit.

Copy link
Member

@andygrove andygrove left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thanks @jgoday

@andygrove andygrove merged commit ea59d05 into apache:master May 25, 2021
jimexist pushed a commit to jimexist/arrow-datafusion that referenced this pull request May 26, 2021
…pache#402)

* apache#401: Add subcommand to TPC-H benchmark args to distinguish between DataFusion and Ballista

* fix benchmark subcommand name

* Fix lint

* fix benchmark tests using DatafusionBenchmarkOpts

* Fix DataFusionBenchmarkOpts name and update doc
@houqp houqp added ballista datafusion Changes in the datafusion crate labels Jul 29, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
datafusion Changes in the datafusion crate
Projects
None yet
Development

Successfully merging this pull request may close these issues.

TPC-H benchmark should distinguish between DataFusion and Ballista arguments
4 participants