Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: core dump in macos running sql_cluster_test #1892

Merged

Conversation

aceforeverd
Copy link
Collaborator

@aceforeverd aceforeverd commented May 26, 2022

  1. fix a sql_cluster_test core in darwin: when there is no partition data for a table locally, the DistributeWindowIterator::SeekToFirst set it_ still. This cores the program when user want to DistributeWindowIterator::GetKey() because it_ is not null, so it try to get key data from it_
  2. enable more last join and window union test for cluster/request mode
  3. update codecov.yml
  4. improve toydb_run_engine bootstrapping

if case contains resource field
`DistributeWindowIterator::SeekToFirst` will set `it_` even there is no
data locally, this lead to use `it_` instead `kv_it_` to fetch row data
when using `GetKey()`, which will core the test
@github-actions github-actions bot added execute-engine hybridse sql engine storage-engine openmldb storage engine. nameserver & tablet labels May 26, 2022
@aceforeverd aceforeverd requested review from dl239, zhanghaohit, tobegit3hub and vagetablechicken and removed request for dl239 May 26, 2022 09:21
@aceforeverd aceforeverd self-assigned this May 26, 2022
@aceforeverd aceforeverd added this to the v0.6 milestone May 26, 2022
@codecov
Copy link

codecov bot commented May 26, 2022

Codecov Report

Merging #1892 (230e8ed) into main (4cda032) will decrease coverage by 0.00%.
The diff coverage is 79.36%.

@@             Coverage Diff              @@
##               main    #1892      +/-   ##
============================================
- Coverage     75.69%   75.69%   -0.01%     
  Complexity      347      347              
============================================
  Files           613      613              
  Lines        117018   117031      +13     
  Branches       1024     1024              
============================================
+ Hits          88579    88584       +5     
- Misses        28230    28238       +8     
  Partials        209      209              
Impacted Files Coverage Δ
hybridse/include/case/sql_case.h 94.59% <ø> (+7.63%) ⬆️
hybridse/src/case/sql_case.cc 72.44% <50.00%> (-0.18%) ⬇️
src/catalog/distribute_iterator.cc 88.07% <82.00%> (-0.55%) ⬇️
hybridse/src/vm/runner.cc 67.75% <100.00%> (-0.06%) ⬇️
src/catalog/distribute_iterator.h 54.54% <100.00%> (+4.54%) ⬆️
src/catalog/distribute_iterator_test.cc 100.00% <100.00%> (ø)
hybridse/include/base/spin_lock.h 66.66% <0.00%> (-33.34%) ⬇️
src/sdk/db_sdk.cc 62.58% <0.00%> (-4.09%) ⬇️
src/zk/dist_lock.cc 81.81% <0.00%> (-1.52%) ⬇️
src/client/tablet_client.cc 55.11% <0.00%> (-0.23%) ⬇️
... and 10 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 4cda032...230e8ed. Read the comment docs.

@github-actions
Copy link
Contributor

github-actions bot commented May 26, 2022

SDK Test Report

  75 files    75 suites   6m 22s ⏱️
163 tests 161 ✔️ 2 💤 0
204 runs  202 ✔️ 2 💤 0

Results for commit 230e8ed.

♻️ This comment has been updated with latest results.

Copy link
Collaborator

@zhanghaohit zhanghaohit left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions
Copy link
Contributor

github-actions bot commented May 26, 2022

HybridSE Linux Test Report

       67 files       235 suites   6m 7s ⏱️
19 059 tests 19 057 ✔️ 2 💤 0

Results for commit 230e8ed.

♻️ This comment has been updated with latest results.

@github-actions
Copy link
Contributor

github-actions bot commented May 26, 2022

HybridSE Mac Test Report

       67 files       235 suites   6m 42s ⏱️
19 059 tests 19 057 ✔️ 2 💤 0

Results for commit 230e8ed.

♻️ This comment has been updated with latest results.

@github-actions
Copy link
Contributor

github-actions bot commented May 26, 2022

Linux Test Report

       57 files       193 suites   1h 4m 33s ⏱️
  8 673 tests   8 669 ✔️ 4 💤 0
12 744 runs  12 740 ✔️ 4 💤 0

Results for commit 230e8ed.

♻️ This comment has been updated with latest results.

@aceforeverd aceforeverd changed the title fix: core dump in macos running request union on macos fix: core dump in macos running sql_cluster_test May 26, 2022
kv_it_ = kv.second->Traverse(tid_, cur_pid_, index_name_, "", 0, FLAGS_traverse_cnt_limit, count);
if (kv_it_ && kv_it_->Valid()) {
response_vec_.emplace_back(kv_it_->GetResponse());
auto it = kv.second->Traverse(tid_, kv.first, index_name_, "", 0, FLAGS_traverse_cnt_limit, count);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the Seek has the same problem. it's better to check it_ validation in GetKey and GetValue function

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we should ensure only zero or one of it_ and kv_it_ is not null.
So the check here is whether it_ and kv_it_ both not null.
It is it_ and kv_it_ 's duty to check valid itself before GetKey or GetValue

Copy link
Collaborator Author

@aceforeverd aceforeverd May 27, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we should improve type safety for iterators, in both definition and implementation. E.g the GetKey method of WindowIterator should mark as const

@aceforeverd aceforeverd requested a review from dl239 May 27, 2022 09:11
@aceforeverd aceforeverd merged commit 1ffa30f into 4paradigm:main May 28, 2022
@aceforeverd aceforeverd deleted the feat-cluster-request-test-for-last-join branch May 28, 2022 02:21
@aceforeverd
Copy link
Collaborator Author

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
execute-engine hybridse sql engine storage-engine openmldb storage engine. nameserver & tablet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants