-
Notifications
You must be signed in to change notification settings - Fork 3.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[opt](Nereids) lock table in ascending order of table IDs #45045
base: master
Are you sure you want to change the base?
Conversation
Thank you for your contribution to Apache Doris. Please clearly describe your PR:
|
32b4505
to
37a00a2
Compare
run buildall |
TPC-H: Total hot run time: 39817 ms
|
TPC-DS: Total hot run time: 68792 ms
|
ClickBench: Total hot run time: 32.24 s
|
aa00351
to
d030993
Compare
run buildall |
d030993
to
c50ff85
Compare
run buildall |
TPC-H: Total hot run time: 39865 ms
|
TPC-DS: Total hot run time: 197793 ms
|
ClickBench: Total hot run time: 33.51 s
|
c50ff85
to
fd02503
Compare
run buildall |
TPC-H: Total hot run time: 39926 ms
|
TPC-DS: Total hot run time: 196875 ms
|
ClickBench: Total hot run time: 32.59 s
|
fd02503
to
47a8e07
Compare
run buildall |
TPC-H: Total hot run time: 39999 ms
|
TPC-DS: Total hot run time: 196928 ms
|
ClickBench: Total hot run time: 33.76 s
|
47a8e07
to
888e041
Compare
run buildall |
TPC-H: Total hot run time: 39842 ms
|
TPC-DS: Total hot run time: 196897 ms
|
ClickBench: Total hot run time: 32.01 s
|
888e041
to
5644491
Compare
TPC-H: Total hot run time: 40090 ms
|
TPC-DS: Total hot run time: 190338 ms
|
ClickBench: Total hot run time: 32.48 s
|
0d532a6
to
1aedf9d
Compare
run buildall |
1aedf9d
to
a2ece57
Compare
TPC-H: Total hot run time: 40778 ms
|
TPC-DS: Total hot run time: 194732 ms
|
ClickBench: Total hot run time: 32.62 s
|
d67beb9
to
924a1af
Compare
run buildall |
TPC-H: Total hot run time: 39878 ms
|
TPC-DS: Total hot run time: 197215 ms
|
ClickBench: Total hot run time: 32.98 s
|
924a1af
to
3b9418a
Compare
run buildall |
TPC-H: Total hot run time: 39649 ms
|
TPC-DS: Total hot run time: 189089 ms
|
ClickBench: Total hot run time: 31.83 s
|
3b9418a
to
b30d00a
Compare
run buildall |
b30d00a
to
18bd9e9
Compare
run buildall |
@@ -243,6 +243,7 @@ private List<Pair<List<Comparable>, TRow>> getPartitionInfosInrernal() throws An | |||
List<Pair<List<Comparable>, TRow>> partitionInfos = new ArrayList<Pair<List<Comparable>, TRow>>(); | |||
Map<Long, List<String>> partitionsUnSyncTables = null; | |||
String mtmvPartitionSyncErrorMsg = null; | |||
olapTable.readLock(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
need lock of relatedTable?
trow.addToColumnValue(new TCell().setBoolVal(MTMVPartitionUtil.isMTMVSync(mv))); | ||
if (LOG.isDebugEnabled()) { | ||
LOG.debug("mvend: " + mv.getName()); | ||
mv.readLock(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
need readLock of relatedTable?
What problem does this PR solve?
Problem Summary:
Doris's table locks are fair read-write locks. If two threads acquire read locks on tables in different orders and simultaneously a third thread attempts to acquire a write lock on one of these tables, a deadlock can form between the two threads trying to acquire read locks. This PR changes the lock acquisition order for queries to follow the order of table IDs, ensuring that the lock acquisition order for tables is consistent among different threads.
Release note
Execute table locking operations in ascending order of table IDs
Check List (For Author)
Test
Behavior changed:
Does this need documentation?
Check List (For Reviewer who merge this PR)