Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding TPCH benchmarks for Sort Merge Join #10092

Merged
merged 6 commits into from
Apr 16, 2024
Merged

Adding TPCH benchmarks for Sort Merge Join #10092

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
c7f272c
Adding TPCH bencmarks for Sort Merge Join
comphead Apr 15, 2024
301c827
Update benchmarks/bench.sh
comphead Apr 16, 2024
3faeec9
fix benches
comphead Apr 16, 2024
7ba1b96
Merge branch 'dev' of https://github.com/comphead/arrow-datafusion in…
comphead Apr 16, 2024
4d671e2
fmt
comphead Apr 16, 2024
a88b278
comments
comphead Apr 16, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 21 additions & 0 deletions benchmarks/bench.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -213,6 +213,12 @@ main() {
tpch_mem10)
run_tpch_mem "10"
;;
tpch_smj)
run_tpch_smj "1"
;;
tpch_smj10)
run_tpch_smj "10"
;;
parquet)
run_parquet
;;
Expand Down Expand Up @@ -320,6 +326,21 @@ run_tpch() {
$CARGO_COMMAND --bin tpch -- benchmark datafusion --iterations 5 --path "${TPCH_DIR}" --format parquet -o ${RESULTS_FILE}
}

# Runs the tpch benchmark with sort merge join
run_tpch_smj() {
SCALE_FACTOR=$1
if [ -z "$SCALE_FACTOR" ] ; then
echo "Internal error: Scale factor not specified"
exit 1
fi
TPCH_DIR="${DATA_DIR}/tpch_sf${SCALE_FACTOR}"

RESULTS_FILE="${RESULTS_DIR}/tpch_smj_sf${SCALE_FACTOR}.json"
echo "RESULTS_FILE: ${RESULTS_FILE}"
echo "Running tpch SMJ benchmark..."
$CARGO_COMMAND --bin tpch -- benchmark datafusion --iterations 5 --path "${TPCH_DIR}" -j false --format parquet -o ${RESULTS_FILE}
comphead marked this conversation as resolved.
Show resolved Hide resolved
}

# Runs the tpch in memory
run_tpch_mem() {
SCALE_FACTOR=$1
Expand Down
10 changes: 9 additions & 1 deletion benchmarks/src/tpch/run.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,9 @@ use datafusion_common::{DEFAULT_CSV_EXTENSION, DEFAULT_PARQUET_EXTENSION};
use log::info;
use structopt::StructOpt;

// hack to avoid `default_value is meaningless for bool` errors
type BoolDefaultTrue = bool;

/// Run the tpch benchmark.
///
/// This benchmarks is derived from the [TPC-H][1] version
Expand Down Expand Up @@ -81,6 +84,10 @@ pub struct RunOpt {
/// Whether to disable collection of statistics (and cost based optimizations) or not.
#[structopt(short = "S", long = "disable-statistics")]
disable_statistics: bool,

/// Whether to disable collection of statistics (and cost based optimizations) or not.
#[structopt(short = "j", long = "hash-join", default_value = "true")]
prefer_hash_join: BoolDefaultTrue,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am worried that this might switch the tpch runs to using SMJ by accident (given your comment above). I started some benchmark runs to see if we can get some data one way or the other

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hash join enabled by default, SMJ is a separate key which is not even documented yet as it still in experimental

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

benchmarks look good to me. ✅

}

const TPCH_QUERY_START_ID: usize = 1;
Expand All @@ -107,10 +114,11 @@ impl RunOpt {
}

async fn benchmark_query(&self, query_id: usize) -> Result<Vec<QueryResult>> {
let config = self
let mut config = self
.common
.config()
.with_collect_statistics(!self.disable_statistics);
config.options_mut().optimizer.prefer_hash_join = self.prefer_hash_join;
let ctx = SessionContext::new_with_config(config);

// register tables
Expand Down
Loading