Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: optimize window computation with UnsafeRowOpt by removing zipped df #1882

Conversation

tobegit3hub
Copy link
Collaborator

@tobegit3hub tobegit3hub added this to the v0.6 milestone May 25, 2022
@tobegit3hub tobegit3hub self-assigned this May 25, 2022
@github-actions github-actions bot added the batch-engine openmldb batch(offline) engine label May 25, 2022
@codecov
Copy link

codecov bot commented May 25, 2022

Codecov Report

Merging #1882 (06e4181) into main (4cda032) will increase coverage by 0.01%.
The diff coverage is 40.27%.

@@             Coverage Diff              @@
##               main    #1882      +/-   ##
============================================
+ Coverage     75.69%   75.70%   +0.01%     
  Complexity      347      347              
============================================
  Files           613      613              
  Lines        117018   117050      +32     
  Branches       1024     1009      -15     
============================================
+ Hits          88579    88616      +37     
+ Misses        28230    28225       -5     
  Partials        209      209              
Impacted Files Coverage Δ
..._4paradigm/openmldb/batch/utils/HybridseUtil.scala 77.14% <27.77%> (-7.29%) ⬇️
...4paradigm/openmldb/batch/nodes/WindowAggPlan.scala 61.04% <36.11%> (-2.83%) ⬇️
..._4paradigm/openmldb/batch/utils/SparkRowUtil.scala 60.00% <55.55%> (-0.98%) ⬇️
...aradigm/openmldb/batch/window/WindowComputer.scala 81.18% <66.66%> (+13.33%) ⬆️
hybridse/include/base/spin_lock.h 66.66% <0.00%> (-33.34%) ⬇️
src/client/tablet_client.cc 55.11% <0.00%> (-0.23%) ⬇️
src/sdk/sql_cluster_router.cc 54.84% <0.00%> (-0.13%) ⬇️
hybridse/src/vm/runner.cc 67.76% <0.00%> (-0.04%) ⬇️
src/storage/mem_table.cc 89.26% <0.00%> (ø)
... and 8 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 4cda032...06e4181. Read the comment docs.

@github-actions
Copy link
Contributor

github-actions bot commented May 25, 2022

SDK Test Report

  75 files    75 suites   5m 26s ⏱️
163 tests 161 ✔️ 2 💤 0
204 runs  202 ✔️ 2 💤 0

Results for commit 06e4181.

♻️ This comment has been updated with latest results.

@github-actions
Copy link
Contributor

github-actions bot commented May 25, 2022

Linux Test Report

       57 files       187 suites   1h 0m 58s ⏱️
  8 503 tests   8 499 ✔️ 4 💤 0
12 489 runs  12 485 ✔️ 4 💤 0

Results for commit 06e4181.

♻️ This comment has been updated with latest results.

@tobegit3hub tobegit3hub merged commit 250da4e into 4paradigm:main May 31, 2022
@tobegit3hub
Copy link
Collaborator Author

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
batch-engine openmldb batch(offline) engine
Projects
None yet
Development

Successfully merging this pull request may close these issues.

UnsafeRowOpt Spark planner for window to remove zipped dataframe
4 participants