Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

*: fix a data race on TestConnExecutionTimeout #35923

Merged
merged 28 commits into from
Jul 11, 2022
Merged

*: fix a data race on TestConnExecutionTimeout #35923

merged 28 commits into from
Jul 11, 2022

Conversation

tiancaiamao
Copy link
Contributor

What problem does this PR solve?

Issue Number: close #35922

Problem Summary:

What is changed and how it works?

This data race is introduced by #35803

There are two problems in that commit:

  • data race between the been-killed session groutine and the Kill() invoker goroutine
  • background expensive query handler might kill long running DDL mistakenly

What's done in pr/35803 is that it add a check in the Kill() function,
and from the stmtctx if it's found to be executing DDL, the kill is changed to CancelDDL.

But this is not thread-safe ... check the stmtctx for it's state is not precise,
because the stmtctx might be constantly changing...
The been-killed session update it's stmtctx and then data race from the checking process.

In this commit, to avoid the data race, I change the logic to "just set the kill flag, and let the callee do the cancel operation"

Check List

Tests

  • Unit test
  • Integration test
  • Manual test (add detailed scripts or steps below)
make failpoint-enable
cd server
go test -race  -run TestConnExecutionTimeout

No data race any more.

  • No code

Side effects

  • Performance regression: Consumes more CPU
  • Performance regression: Consumes more Memory
  • Breaking backward compatibility

Documentation

  • Affects user behaviors
  • Contains syntax changes
  • Contains variable changes
  • Contains experimental features
  • Changes MySQL compatibility

Release note

Please refer to Release Notes Language Style Guide to write a quality release note.

None

@tiancaiamao tiancaiamao requested review from morgo, xhebox and tangenta July 4, 2022 09:47
@ti-chi-bot
Copy link
Member

ti-chi-bot commented Jul 4, 2022 •
edited
Loading

[REVIEW NOTIFICATION]

This pull request has been approved by:

  • tangenta
  • xhebox

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Reviewer can indicate their review by submitting an approval review.
Reviewer can cancel approval by submitting a request changes review.

@ti-chi-bot ti-chi-bot added release-note-none Denotes a PR that doesn't merit a release note. do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Jul 4, 2022
@tiancaiamao tiancaiamao marked this pull request as ready for review July 4, 2022 09:48
@ti-chi-bot ti-chi-bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jul 4, 2022
xhebox
xhebox reviewed Jul 4, 2022
View reviewed changes
ddl/ddl.go Show resolved Hide resolved
@sre-bot
Copy link
Contributor

sre-bot commented Jul 4, 2022 •
edited
Loading

Code Coverage Details: https://codecov.io/github/pingcap/tidb/commit/94d14735418190ca258ee248b04fcfd77f185891

@ti-chi-bot ti-chi-bot added the status/LGT1 Indicates that a PR has LGTM 1. label Jul 4, 2022
@ti-chi-bot ti-chi-bot added status/LGT2 Indicates that a PR has LGTM 2. and removed status/LGT1 Indicates that a PR has LGTM 1. labels Jul 4, 2022
@xhebox
Copy link
Contributor

xhebox commented Jul 4, 2022

/merge

@ti-chi-bot
Copy link
Member

This pull request has been accepted and is ready to merge.

Commit hash: 33f1684

@ti-chi-bot ti-chi-bot added the status/can-merge Indicates a PR has been approved by a committer. label Jul 4, 2022
@tiancaiamao
Copy link
Contributor Author

/run-check_dev_2

2 similar comments
@tiancaiamao
Copy link
Contributor Author

/run-check_dev_2

@tiancaiamao
Copy link
Contributor Author

/run-check_dev_2

@morgo
Copy link
Contributor

morgo commented Jul 4, 2022

LGTM, thanks for fixing!

@hawkingrei
Copy link
Member

/run-check_dev_2

@hawkingrei
Copy link
Member

/merge

@tiancaiamao
Copy link
Contributor Author

/rebuild

@tiancaiamao
Copy link
Contributor Author

/hold

I need to investigate why collation test failed

@ti-chi-bot ti-chi-bot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Jul 6, 2022
@hawkingrei
Copy link
Member

@tiancaiamao It meets a new data race in the tidb server.

#35979

@ti-chi-bot ti-chi-bot removed the status/can-merge Indicates a PR has been approved by a committer. label Jul 6, 2022
@tiancaiamao
Copy link
Contributor Author

/run-check_dev_2

@tiancaiamao
Copy link
Contributor Author

/merge

@ti-chi-bot
Copy link
Member

This pull request has been accepted and is ready to merge.

Commit hash: 771b2e8

@ti-chi-bot ti-chi-bot added the status/can-merge Indicates a PR has been approved by a committer. label Jul 11, 2022
@tiancaiamao
Copy link
Contributor Author

/unhold

@ti-chi-bot ti-chi-bot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Jul 11, 2022
@ti-chi-bot ti-chi-bot merged commit 756a817 into pingcap:master Jul 11, 2022
@tiancaiamao tiancaiamao deleted the race-35922 branch July 11, 2022 05:37
@sre-bot
Copy link
Contributor

sre-bot commented Jul 11, 2022

TiDB MergeCI notify

🔴 Bad News! New failing [1] after this pr merged.
These new failed integration tests seem to be caused by the current PR, please try to fix these new failed integration tests, thanks!

CI Name Result Duration Compare with Parent commit
idc-jenkins-ci/integration-cdc-test 🟥 failed 1, success 34, total 35 25 min New failing
idc-jenkins-ci-tidb/integration-common-test 🟢 all 11 tests passed 30 min Existing passed
idc-jenkins-ci-tidb/common-test 🟢 all 12 tests passed 12 min Existing passed
idc-jenkins-ci-tidb/tics-test 🟢 all 1 tests passed 7 min 2 sec Existing passed
idc-jenkins-ci-tidb/integration-ddl-test 🟢 all 6 tests passed 6 min 23 sec Existing passed
idc-jenkins-ci-tidb/sqllogic-test-2 🟢 all 28 tests passed 5 min 41 sec Existing passed
idc-jenkins-ci-tidb/sqllogic-test-1 🟢 all 26 tests passed 5 min 24 sec Existing passed
idc-jenkins-ci-tidb/mybatis-test 🟢 all 1 tests passed 3 min 11 sec Existing passed
idc-jenkins-ci-tidb/integration-compatibility-test 🟢 all 1 tests passed 2 min 42 sec Existing passed
idc-jenkins-ci-tidb/plugin-test 🟢 build success, plugin test success 4min Existing passed

@tiancaiamao tiancaiamao mentioned this pull request Jul 14, 2022
Closed
This was referenced May 16, 2023
Closed
Merged
@ti-chi-bot ti-chi-bot mentioned this pull request May 17, 2023
Merged
12 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@tangenta tangenta tangenta approved these changes

@xhebox xhebox xhebox approved these changes

@morgo morgo Awaiting requested review from morgo

Assignees
No one assigned
Labels
release-note-none Denotes a PR that doesn't merit a release note. size/M Denotes a PR that changes 30-99 lines, ignoring generated files. status/can-merge Indicates a PR has been approved by a committer. status/LGT2 Indicates that a PR has LGTM 2.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Data race on server TestConnExecutionTimeout function
7 participants
@tiancaiamao @ti-chi-bot @sre-bot @xhebox @morgo @hawkingrei @tangenta