Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: iavl async pruning (backport #593) #599

Merged
merged 1 commit into from
Apr 8, 2024
Merged

feat: iavl async pruning (backport #593) #599

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ require (
github.com/cosmos/go-bip39 v1.0.0
github.com/cosmos/gogogateway v1.2.0
github.com/cosmos/gogoproto v1.4.10
github.com/cosmos/iavl v1.1.2-0.20240405173644-e52f7630d3b7
github.com/cosmos/iavl v1.1.2-0.20240405172238-7f92c6b356ac
github.com/cosmos/ics23/go v0.10.0
github.com/cosmos/ledger-cosmos-go v0.12.1
github.com/golang/mock v1.6.0
Expand Down
4 changes: 2 additions & 2 deletions go.sum
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -361,8 +361,8 @@ github.com/cosmos/gogogateway v1.2.0/go.mod h1:iQpLkGWxYcnCdz5iAdLcRBSw3h7NXeOkZ
github.com/cosmos/gogoproto v1.4.2/go.mod h1:cLxOsn1ljAHSV527CHOtaIP91kK6cCrZETRBrkzItWU=
github.com/cosmos/gogoproto v1.4.10 h1:QH/yT8X+c0F4ZDacDv3z+xE3WU1P1Z3wQoLMBRJoKuI=
github.com/cosmos/gogoproto v1.4.10/go.mod h1:3aAZzeRWpAwr+SS/LLkICX2/kDFyaYVzckBDzygIxek=
github.com/cosmos/iavl v1.1.2-0.20240405173644-e52f7630d3b7 h1:eGi/RRQ5K+C6J8aCSRYXuS5NFiSe+S46F1zMOEQmykY=
github.com/cosmos/iavl v1.1.2-0.20240405173644-e52f7630d3b7/go.mod h1:jLeUvm6bGT1YutCaL2fIar/8vGUE8cPZvh/gXEWDaDM=
github.com/cosmos/iavl v1.1.2-0.20240405172238-7f92c6b356ac h1:D1OG5ugS4r1Jq8U331gB4mrYsX7JQsasfWkFvdva4KI=
github.com/cosmos/iavl v1.1.2-0.20240405172238-7f92c6b356ac/go.mod h1:jLeUvm6bGT1YutCaL2fIar/8vGUE8cPZvh/gXEWDaDM=
github.com/cosmos/ics23/go v0.10.0 h1:iXqLLgp2Lp+EdpIuwXTYIQU+AiHj9mOC2X9ab++bZDM=
github.com/cosmos/ics23/go v0.10.0/go.mod h1:ZfJSmng/TBNTBkFemHHHj5YY7VAU/MBU980F4VU1NG0=
github.com/cosmos/keyring v1.2.0 h1:8C1lBP9xhImmIabyXW4c3vFjjLiBdGCmfLUfeZlV1Yo=
Expand Down
8 changes: 8 additions & 0 deletions server/mock/store.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,14 @@ func (ms multiStore) Commit() storetypes.CommitID {
panic("not implemented")
}

func (ms multiStore) SetCommitting() {
panic("not implemented")
}

func (ms multiStore) UnsetCommitting() {
panic("not implemented")
}

func (ms multiStore) LastCommitID() storetypes.CommitID {
panic("not implemented")
}
Expand Down
2 changes: 1 addition & 1 deletion simapp/go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,7 @@ require (
github.com/cosmos/go-bip39 v1.0.0 // indirect
github.com/cosmos/gogogateway v1.2.0 // indirect
github.com/cosmos/gogoproto v1.4.10 // indirect
github.com/cosmos/iavl v1.1.2-0.20240405173644-e52f7630d3b7 // indirect
github.com/cosmos/iavl v1.1.2-0.20240405172238-7f92c6b356ac // indirect
github.com/cosmos/ics23/go v0.10.0 // indirect
github.com/cosmos/ledger-cosmos-go v0.12.1 // indirect
github.com/cosmos/rosetta-sdk-go v0.10.0 // indirect
Expand Down
4 changes: 2 additions & 2 deletions simapp/go.sum
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -358,8 +358,8 @@ github.com/cosmos/gogogateway v1.2.0/go.mod h1:iQpLkGWxYcnCdz5iAdLcRBSw3h7NXeOkZ
github.com/cosmos/gogoproto v1.4.2/go.mod h1:cLxOsn1ljAHSV527CHOtaIP91kK6cCrZETRBrkzItWU=
github.com/cosmos/gogoproto v1.4.10 h1:QH/yT8X+c0F4ZDacDv3z+xE3WU1P1Z3wQoLMBRJoKuI=
github.com/cosmos/gogoproto v1.4.10/go.mod h1:3aAZzeRWpAwr+SS/LLkICX2/kDFyaYVzckBDzygIxek=
github.com/cosmos/iavl v1.1.2-0.20240405173644-e52f7630d3b7 h1:eGi/RRQ5K+C6J8aCSRYXuS5NFiSe+S46F1zMOEQmykY=
github.com/cosmos/iavl v1.1.2-0.20240405173644-e52f7630d3b7/go.mod h1:jLeUvm6bGT1YutCaL2fIar/8vGUE8cPZvh/gXEWDaDM=
github.com/cosmos/iavl v1.1.2-0.20240405172238-7f92c6b356ac h1:D1OG5ugS4r1Jq8U331gB4mrYsX7JQsasfWkFvdva4KI=
github.com/cosmos/iavl v1.1.2-0.20240405172238-7f92c6b356ac/go.mod h1:jLeUvm6bGT1YutCaL2fIar/8vGUE8cPZvh/gXEWDaDM=
github.com/cosmos/ics23/go v0.10.0 h1:iXqLLgp2Lp+EdpIuwXTYIQU+AiHj9mOC2X9ab++bZDM=
github.com/cosmos/ics23/go v0.10.0/go.mod h1:ZfJSmng/TBNTBkFemHHHj5YY7VAU/MBU980F4VU1NG0=
github.com/cosmos/keyring v1.2.0 h1:8C1lBP9xhImmIabyXW4c3vFjjLiBdGCmfLUfeZlV1Yo=
Expand Down
14 changes: 13 additions & 1 deletion store/iavl/store.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,8 @@ func LoadStore(db dbm.DB, logger log.Logger, key types.StoreKey, id types.Commit
// provided DB. An error is returned if the version fails to load, or if called with a positive
// version on an empty tree.
func LoadStoreWithInitialVersion(db dbm.DB, logger log.Logger, key types.StoreKey, id types.CommitID, initialVersion uint64, cacheSize int, disableFastNode bool) (types.CommitKVStore, error) {
tree := iavl.NewMutableTree(wrapper.NewIAVLDB(db), cacheSize, disableFastNode, clog.NewNopLogger(), iavl.InitialVersionOption(initialVersion))
// Create a new IAVL tree with the async pruning enabled
tree := iavl.NewMutableTree(wrapper.NewIAVLDB(db), cacheSize, disableFastNode, clog.NewNopLogger(), iavl.InitialVersionOption(initialVersion), iavl.AsyncPruningOption(true))

isUpgradeable, err := tree.IsUpgradeable()
if err != nil {
Expand Down Expand Up @@ -117,6 +118,17 @@ func (st *Store) GetImmutable(version int64) (*Store, error) {
}, nil
}

// SetCommitting marks the store as committing, which will prevent any
// parallel writes to the store. It is referenced in the async pruning.
func (st *Store) SetCommitting() {
st.tree.SetCommitting()
}

// UnsetCommitting marks the store as not committing.
func (st *Store) UnsetCommitting() {
st.tree.UnsetCommitting()
}

// Commit commits the current store state and returns a CommitID with the new
// version and hash.
func (st *Store) Commit() types.CommitID {
Expand Down
10 changes: 10 additions & 0 deletions store/iavl/tree.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,8 @@ type (
Get(key []byte) ([]byte, error)
Set(key, value []byte) (bool, error)
Remove(key []byte) ([]byte, bool, error)
SetCommitting()
UnsetCommitting()
SaveVersion() ([]byte, int64, error)
DeleteVersionsTo(version int64) error
Version() int64
Expand Down Expand Up @@ -55,6 +57,14 @@ func (it *immutableTree) Remove(_ []byte) ([]byte, bool, error) {
panic("cannot call 'Remove' on an immutable IAVL tree")
}

func (it *immutableTree) SetCommitting() {
panic("cannot call 'SetCommitting' on an immutable IAVL tree")
}

func (it *immutableTree) UnsetCommitting() {
panic("cannot call 'UnsetCommitting' on an immutable IAVL tree")
}

func (it *immutableTree) SaveVersion() ([]byte, int64, error) {
panic("cannot call 'SaveVersion' on an immutable IAVL tree")
}
Expand Down
6 changes: 6 additions & 0 deletions store/mem/store.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,12 @@ func (s Store) CacheWrapWithTrace(w io.Writer, tc types.TraceContext) types.Cach
// Commit performs a no-op as entries are persistent between commitments.
func (s *Store) Commit() (id types.CommitID) { return }

// Implements CommitStore.
func (s *Store) SetCommitting() {}

// Implements CommitStore.
func (s *Store) UnsetCommitting() {}

func (s *Store) SetPruning(pruning pruningtypes.PruningOptions) {}

// GetPruning is a no-op as pruning options cannot be directly set on this store.
Expand Down
4 changes: 4 additions & 0 deletions store/rootmulti/dbadapter.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,10 @@ func (cdsa commitDBStoreAdapter) LastCommitID() types.CommitID {
}
}

func (cdsa commitDBStoreAdapter) SetCommitting() {}

func (cdsa commitDBStoreAdapter) UnsetCommitting() {}

func (cdsa commitDBStoreAdapter) SetPruning(_ pruningtypes.PruningOptions) {}

// GetPruning is a no-op as pruning options cannot be directly set on this store.
Expand Down
16 changes: 16 additions & 0 deletions store/rootmulti/store.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -451,7 +451,9 @@
rs.logger.Debug("commit header and version mismatch", "header_height", rs.commitHeader.Height, "version", version)
}

rs.SetCommitting()
rs.lastCommitInfo = commitStores(version, rs.stores, rs.removalMap)
rs.UnsetCommitting()
rs.lastCommitInfo.Timestamp = rs.commitHeader.Time
defer rs.flushMetadata(rs.db, version, rs.lastCommitInfo)

Expand All @@ -477,6 +479,20 @@
}
}

// SetCommitting implements Committer/CommitStore.
func (rs *Store) SetCommitting() {
for _, store := range rs.stores {
store.SetCommitting()
}
Comment on lines +484 to +486

Check warning

Code scanning / CodeQL

Iteration over map Warning

Iteration over map may be a possible source of non-determinism
}

// UnsetCommitting implements Committer/CommitStore.
func (rs *Store) UnsetCommitting() {
for _, store := range rs.stores {
store.UnsetCommitting()
}
Comment on lines +491 to +493

Check warning

Code scanning / CodeQL

Iteration over map Warning

Iteration over map may be a possible source of non-determinism
}

// CacheWrap implements CacheWrapper/Store/CommitStore.
func (rs *Store) CacheWrap() types.CacheWrap {
return rs.CacheMultiStore().(types.CacheWrap)
Expand Down
27 changes: 17 additions & 10 deletions store/rootmulti/store_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -511,6 +511,10 @@ func TestMultiStore_Pruning(t *testing.T) {
ms.Commit()
}

// asyn pruning, simulate the consensus process
time.Sleep(150 * time.Millisecond)
ms.Commit()

for _, v := range tc.saved {
_, err := ms.CacheMultiStoreWithVersion(v)
require.NoError(t, err, "expected no error when loading height: %d", v)
Expand Down Expand Up @@ -548,16 +552,6 @@ func TestMultiStore_Pruning_SameHeightsTwice(t *testing.T) {

require.Equal(t, numVersions, lastCommitInfo.Version)

for v := int64(1); v < numVersions-int64(keepRecent); v++ {
err := ms.LoadVersion(v)
require.Error(t, err, "expected error when loading pruned height: %d", v)
}

for v := int64(numVersions - int64(keepRecent)); v < numVersions; v++ {
err := ms.LoadVersion(v)
require.NoError(t, err, "expected no error when loading height: %d", v)
}

// Get latest
err := ms.LoadVersion(numVersions - 1)
require.NoError(t, err)
Expand All @@ -574,8 +568,19 @@ func TestMultiStore_Pruning_SameHeightsTwice(t *testing.T) {
require.Equal(t, numVersions, lastCommitInfo.Version)

// Ensure that can commit one more height with no panic
time.Sleep(150 * time.Millisecond)
lastCommitInfo = ms.Commit()
require.Equal(t, numVersions+1, lastCommitInfo.Version)

for v := int64(1); v < numVersions-int64(keepRecent); v++ {
err := ms.LoadVersion(v)
require.Error(t, err, "expected error when loading pruned height: %d", v)
}

for v := int64(numVersions - int64(keepRecent)); v < numVersions; v++ {
err := ms.LoadVersion(v)
require.NoError(t, err, "expected no error when loading height: %d", v)
}
}

func TestMultiStore_PruningRestart(t *testing.T) {
Expand Down Expand Up @@ -613,6 +618,8 @@ func TestMultiStore_PruningRestart(t *testing.T) {

// commit one more block and ensure the heights have been pruned
ms.Commit()
time.Sleep(150 * time.Millisecond)
ms.Commit()

actualHeightsToPrune, err = ms.pruningManager.GetFlushAndResetPruningHeights()
require.NoError(t, err)
Expand Down
6 changes: 6 additions & 0 deletions store/transient/store.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,12 @@ func (ts *Store) LastCommitID() (id types.CommitID) {
return
}

// Implements CommitStore
func (ts *Store) SetCommitting() {}

// Implements CommitStore
func (ts *Store) UnsetCommitting() {}

// Implements Store.
func (ts *Store) GetStoreType() types.StoreType {
return types.StoreTypeTransient
Expand Down
2 changes: 2 additions & 0 deletions store/types/store.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,8 @@ type Store interface {
type Committer interface {
Commit() CommitID
LastCommitID() CommitID
SetCommitting()
UnsetCommitting()

SetPruning(pruningtypes.PruningOptions)
GetPruning() pruningtypes.PruningOptions
Expand Down
2 changes: 1 addition & 1 deletion tests/go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,7 @@ require (
github.com/cosmos/cosmos-proto v1.0.0-beta.2 // indirect
github.com/cosmos/go-bip39 v1.0.0 // indirect
github.com/cosmos/gogogateway v1.2.0 // indirect
github.com/cosmos/iavl v1.1.2-0.20240405173644-e52f7630d3b7 // indirect
github.com/cosmos/iavl v1.1.2-0.20240405172238-7f92c6b356ac // indirect
github.com/cosmos/ics23/go v0.10.0 // indirect
github.com/cosmos/ledger-cosmos-go v0.12.2 // indirect
github.com/cosmos/rosetta-sdk-go v0.10.0 // indirect
Expand Down
4 changes: 2 additions & 2 deletions tests/go.sum
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -358,8 +358,8 @@ github.com/cosmos/gogogateway v1.2.0/go.mod h1:iQpLkGWxYcnCdz5iAdLcRBSw3h7NXeOkZ
github.com/cosmos/gogoproto v1.4.2/go.mod h1:cLxOsn1ljAHSV527CHOtaIP91kK6cCrZETRBrkzItWU=
github.com/cosmos/gogoproto v1.4.10 h1:QH/yT8X+c0F4ZDacDv3z+xE3WU1P1Z3wQoLMBRJoKuI=
github.com/cosmos/gogoproto v1.4.10/go.mod h1:3aAZzeRWpAwr+SS/LLkICX2/kDFyaYVzckBDzygIxek=
github.com/cosmos/iavl v1.1.2-0.20240405173644-e52f7630d3b7 h1:eGi/RRQ5K+C6J8aCSRYXuS5NFiSe+S46F1zMOEQmykY=
github.com/cosmos/iavl v1.1.2-0.20240405173644-e52f7630d3b7/go.mod h1:jLeUvm6bGT1YutCaL2fIar/8vGUE8cPZvh/gXEWDaDM=
github.com/cosmos/iavl v1.1.2-0.20240405172238-7f92c6b356ac h1:D1OG5ugS4r1Jq8U331gB4mrYsX7JQsasfWkFvdva4KI=
github.com/cosmos/iavl v1.1.2-0.20240405172238-7f92c6b356ac/go.mod h1:jLeUvm6bGT1YutCaL2fIar/8vGUE8cPZvh/gXEWDaDM=
github.com/cosmos/ics23/go v0.10.0 h1:iXqLLgp2Lp+EdpIuwXTYIQU+AiHj9mOC2X9ab++bZDM=
github.com/cosmos/ics23/go v0.10.0/go.mod h1:ZfJSmng/TBNTBkFemHHHj5YY7VAU/MBU980F4VU1NG0=
github.com/cosmos/keyring v1.2.0 h1:8C1lBP9xhImmIabyXW4c3vFjjLiBdGCmfLUfeZlV1Yo=
Expand Down
Loading