Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add DataFusion solution #18

Merged
merged 46 commits into from
Jul 3, 2023
Merged
Changes from 1 commit
Commits
Show all changes
46 commits
Select commit Hold shift + click to select a range
3a983fd
Datafusion solution
Dandandan Jan 17, 2021
b1f613e
Datafusion solution
Dandandan Jan 17, 2021
51ce127
Query fix
Dandandan Jan 17, 2021
3343428
Undo change
Dandandan Jan 17, 2021
d1e7ff3
Increase batch size
Dandandan Jan 17, 2021
58be012
Rename to ans
Dandandan Jan 18, 2021
d87c92d
Fix
Dandandan Jan 18, 2021
2b67e2a
Add q7/q10
Dandandan Jan 18, 2021
d217e37
Use multiple threads better
Dandandan Jan 18, 2021
5a3e5ec
Add exec script
Dandandan Jan 18, 2021
f839050
Some cleanup
Dandandan Jan 18, 2021
6cb14f5
Rename
Dandandan Jan 18, 2021
63fe38b
Fix disabled snmalloc
Dandandan Jan 19, 2021
cbecfbc
Use arrow master again
Dandandan Jan 19, 2021
88ba391
Update benchmark code
Dandandan May 19, 2021
fbb50dc
Make queries work again
matthewmturner Dec 25, 2021
20978b7
Add join queries
matthewmturner Dec 25, 2021
4042b3c
group by q8
matthewmturner Dec 26, 2021
2cca309
Add python bindings
matthewmturner Dec 27, 2021
c345446
Fix join and utils
matthewmturner Jan 4, 2022
82f34fb
Remove rust impl and update utilities
matthewmturner Jan 4, 2022
969bf8e
Add datafusion
Jun 13, 2023
852c9bf
Use version
Jun 13, 2023
403f7cc
Correct
Jun 13, 2023
8dbfb97
Add datafusion to csv
Jun 13, 2023
456799c
Adapt
Jun 13, 2023
0e0ebc2
Undo change
Jun 13, 2023
333f360
Fix version script
Jun 13, 2023
a201d65
Add median
Jun 13, 2023
b80c15b
Add regression query
Jun 13, 2023
04544bc
WIP
Jun 13, 2023
496d637
Fix report
Jun 14, 2023
61f895a
Fix query, consistent syntax
Jun 14, 2023
f1b6366
Undo
Jun 14, 2023
0d5443c
Run twice
Jun 15, 2023
8de0fb6
WIP
Jun 21, 2023
b3975b0
WIP
Jun 21, 2023
30db955
WIP
Jun 21, 2023
8f6904e
WIP
Jun 21, 2023
06abfc0
WIP
Jun 21, 2023
d5dd233
WIP
Jun 21, 2023
c1f2ce0
Unroll
Jun 30, 2023
cbc28ae
Merge remote-tracking branch 'origin/master' into datafusion
Jun 30, 2023
7b91cf6
Rev
Jun 30, 2023
e85542c
WIP
Jul 1, 2023
ae0a21f
Join bench
Jul 1, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Use arrow master again
Dandandan committed Jan 19, 2021
commit cbecfbcaf44fc4da591b77fee65000d730c3371b
6 changes: 3 additions & 3 deletions datafusion/Cargo.toml
Original file line number Diff line number Diff line change
@@ -4,11 +4,11 @@ version = "0.1.0"
edition = "2018"

[dependencies]
datafusion = { git = "https://github.com/Dandandan/arrow.git", features = ["simd"], branch="mem_table_repartition" }
arrow = { git = "https://github.com/Dandandan/arrow.git", features = ["simd"], branch="mem_table_repartition"}
datafusion = { git = "https://github.com/apache/arrow.git", features = ["simd"]}
arrow = { git = "https://github.com/apache/arrow.git", features = ["simd"]}
tokio = { version = "0.2", features = ["macros", "rt-core", "rt-threaded"] }
snmalloc-rs = {version = "0.2", features= ["cache-friendly"]}
num_cpus="1.0"
num_cpus = "1.0"

[profile.release]
lto = true
2 changes: 1 addition & 1 deletion datafusion/ver-datafusion.sh
Original file line number Diff line number Diff line change
@@ -1 +1 @@
cargo tree | grep "├── datafusion"
cargo tree | grep "├── datafusion" | cut -d ' ' -f3