etcdserver: significantly reduces start-up time #11779
Conversation
Codecov Report
@@ Coverage Diff @@
## master #11779 +/- ##
==========================================
- Coverage 66.65% 66.18% -0.47%
==========================================
Files 403 403
Lines 36881 36879 -2
==========================================
- Hits 24582 24408 -174
- Misses 10811 10969 +158
- Partials 1488 1502 +14
Continue to review full report at Codecov.
|
|
With 40 million key test data, this pr can reduce the startup time from 5 minutes to 2.5 minutes. @jingyih PTAL. thanks. |
|
cc @gyuho |
|
This feels more like a bug... Let's backport this. As always, awesome work @tangcong ! Thanks. |
Can we also include this in CHANGELOG? :) |
tangcong
force-pushed
the
optimize-etcd-startup-time
branch
from
73c8d71 to
3e64261
Compare
updated. |
etcdserver/backend.go
Outdated
| kv := mvcc.New(cfg.Logger, oldbe, &lease.FakeLessor{}, ci, mvcc.StoreConfig{CompactionBatchLimit: cfg.CompactionBatchLimit}) | ||
| defer kv.Close() | ||
| if snapshot.Metadata.Index <= kv.ConsistentIndex() { | ||
| if snapshot.Metadata.Index <= ci.ConsistentIndex() { |
There was a problem hiding this comment.
we have to make sure the backend has consistentIndexKeyName key，Otherwise, boltdb will throw an exception. recoverSnapshotBackend is only called when wal exists. I think it is safe. how do you think so? @gyuho
There was a problem hiding this comment.
Hmm, this is called before we call mvcc.NewStore...
There was a problem hiding this comment.
Can we do something like
if beExist {
kvindex := srv.kv.ConsistentIndex()There was a problem hiding this comment.
Is there any exception that wal file exists but db does not？
There was a problem hiding this comment.
Can we do something like
if beExist { kvindex := srv.kv.ConsistentIndex()
done,it will make recoverSnapshotBackend more robust,thanks.
There was a problem hiding this comment.
Is there any exception that wal file exists but db does not？
I don't think so, unless user sets up a separate WAL dir, and deletes data dir intentionally.
|
@tangcong Ok, then let's just keep it in master. |
if there are too much keys(> 1 millions),etcd is slow to start. After my investigation, 90% of the time is spent on mvcc restore(rebuild index tree), 9% is spent on open backend db, and because the index tree has a global lock(idx.Insert), there is no room for optimization. however,restore index is called twice, it is unreasonable.we can benefit from pr #11699 and cut time in half.