Progress Tracking for In-Place Store Migrations #10689
Labels
Comments
|
Good idea! @tomtau would you like to create a PR? |
I can add some extra logging; for the additional tracking, I'm not sure of what the best way is (store migrations can add their logging individually, but not sure whether it's worth having something generic that works across them) |
19 tasks
tomtau
added a commit
to tomtau/cosmos-sdk
that referenced
this issue
Dec 14, 2021
fixes cosmos#10689 it'd also be good to add more fine-grained tracking of individual migrations, but there doesn't seem to be a quick way of doing it. Perhaps the best is to leave it to each migration implementation to add its own internal logging?
mergify bot
pushed a commit
that referenced
this issue
Dec 14, 2021
## Description Closes: #10689 it'd also be good to add more fine-grained tracking of individual migrations, but there doesn't seem to be a quick way of doing it. Perhaps the best is to leave it to each migration implementation to add its own internal logging? --- ### Author Checklist *All items are required. Please add a note to the item if the item is not applicable and please add links to any relevant follow up issues.* I have... - [x] included the correct [type prefix](https://github.com/commitizen/conventional-commit-types/blob/v3.0.0/index.json) in the PR title - [ ] ~added `!` to the type prefix if API or client breaking change~ - [x] targeted the correct branch (see [PR Targeting](https://github.com/cosmos/cosmos-sdk/blob/master/CONTRIBUTING.md#pr-targeting)) - [x] provided a link to the relevant issue or specification - [x] followed the guidelines for [building modules](https://github.com/cosmos/cosmos-sdk/blob/master/docs/building-modules) - [ ] ~included the necessary unit and integration [tests](https://github.com/cosmos/cosmos-sdk/blob/master/CONTRIBUTING.md#testing)~ - [x] added a changelog entry to `CHANGELOG.md` - [ ] ~included comments for [documenting Go code](https://blog.golang.org/godoc)~ - [ ] ~updated the relevant documentation or specification~ - [x] reviewed "Files changed" and left comments if necessary - [ ] confirmed all CI checks have passed ### Reviewers Checklist *All items are required. Please add a note if the item is not applicable and please add your handle next to the items reviewed if you only reviewed selected items.* I have... - [ ] confirmed the correct [type prefix](https://github.com/commitizen/conventional-commit-types/blob/v3.0.0/index.json) in the PR title - [ ] confirmed `!` in the type prefix if API or client breaking change - [ ] confirmed all author checklist items have been addressed - [ ] reviewed state machine logic - [ ] reviewed API design and naming - [ ] reviewed documentation is accurate - [ ] reviewed tests and test coverage - [ ] manually tested (if applicable)
mergify bot
pushed a commit
that referenced
this issue
Feb 3, 2022
## Description Closes: #10689 it'd also be good to add more fine-grained tracking of individual migrations, but there doesn't seem to be a quick way of doing it. Perhaps the best is to leave it to each migration implementation to add its own internal logging? --- ### Author Checklist *All items are required. Please add a note to the item if the item is not applicable and please add links to any relevant follow up issues.* I have... - [x] included the correct [type prefix](https://github.com/commitizen/conventional-commit-types/blob/v3.0.0/index.json) in the PR title - [ ] ~added `!` to the type prefix if API or client breaking change~ - [x] targeted the correct branch (see [PR Targeting](https://github.com/cosmos/cosmos-sdk/blob/master/CONTRIBUTING.md#pr-targeting)) - [x] provided a link to the relevant issue or specification - [x] followed the guidelines for [building modules](https://github.com/cosmos/cosmos-sdk/blob/master/docs/building-modules) - [ ] ~included the necessary unit and integration [tests](https://github.com/cosmos/cosmos-sdk/blob/master/CONTRIBUTING.md#testing)~ - [x] added a changelog entry to `CHANGELOG.md` - [ ] ~included comments for [documenting Go code](https://blog.golang.org/godoc)~ - [ ] ~updated the relevant documentation or specification~ - [x] reviewed "Files changed" and left comments if necessary - [ ] confirmed all CI checks have passed ### Reviewers Checklist *All items are required. Please add a note if the item is not applicable and please add your handle next to the items reviewed if you only reviewed selected items.* I have... - [ ] confirmed the correct [type prefix](https://github.com/commitizen/conventional-commit-types/blob/v3.0.0/index.json) in the PR title - [ ] confirmed `!` in the type prefix if API or client breaking change - [ ] confirmed all author checklist items have been addressed - [ ] reviewed state machine logic - [ ] reviewed API design and naming - [ ] reviewed documentation is accurate - [ ] reviewed tests and test coverage - [ ] manually tested (if applicable) (cherry picked from commit 8b74157) # Conflicts: # types/module/module.go
tac0turtle
added a commit
that referenced
this issue
Feb 3, 2022
…11107) * feat: extra logging in in-place store migrations (#10768) ## Description Closes: #10689 it'd also be good to add more fine-grained tracking of individual migrations, but there doesn't seem to be a quick way of doing it. Perhaps the best is to leave it to each migration implementation to add its own internal logging? --- ### Author Checklist *All items are required. Please add a note to the item if the item is not applicable and please add links to any relevant follow up issues.* I have... - [x] included the correct [type prefix](https://github.com/commitizen/conventional-commit-types/blob/v3.0.0/index.json) in the PR title - [ ] ~added `!` to the type prefix if API or client breaking change~ - [x] targeted the correct branch (see [PR Targeting](https://github.com/cosmos/cosmos-sdk/blob/master/CONTRIBUTING.md#pr-targeting)) - [x] provided a link to the relevant issue or specification - [x] followed the guidelines for [building modules](https://github.com/cosmos/cosmos-sdk/blob/master/docs/building-modules) - [ ] ~included the necessary unit and integration [tests](https://github.com/cosmos/cosmos-sdk/blob/master/CONTRIBUTING.md#testing)~ - [x] added a changelog entry to `CHANGELOG.md` - [ ] ~included comments for [documenting Go code](https://blog.golang.org/godoc)~ - [ ] ~updated the relevant documentation or specification~ - [x] reviewed "Files changed" and left comments if necessary - [ ] confirmed all CI checks have passed ### Reviewers Checklist *All items are required. Please add a note if the item is not applicable and please add your handle next to the items reviewed if you only reviewed selected items.* I have... - [ ] confirmed the correct [type prefix](https://github.com/commitizen/conventional-commit-types/blob/v3.0.0/index.json) in the PR title - [ ] confirmed `!` in the type prefix if API or client breaking change - [ ] confirmed all author checklist items have been addressed - [ ] reviewed state machine logic - [ ] reviewed API design and naming - [ ] reviewed documentation is accurate - [ ] reviewed tests and test coverage - [ ] manually tested (if applicable) (cherry picked from commit 8b74157) # Conflicts: # types/module/module.go * fix conflict * remove files Co-authored-by: Tomas Tauber <[email protected]> Co-authored-by: marbar3778 <[email protected]>
JimLarson
pushed a commit
to agoric-labs/cosmos-sdk
that referenced
this issue
Jul 7, 2022
…) (cosmos#11107) * feat: extra logging in in-place store migrations (cosmos#10768) ## Description Closes: cosmos#10689 it'd also be good to add more fine-grained tracking of individual migrations, but there doesn't seem to be a quick way of doing it. Perhaps the best is to leave it to each migration implementation to add its own internal logging? --- ### Author Checklist *All items are required. Please add a note to the item if the item is not applicable and please add links to any relevant follow up issues.* I have... - [x] included the correct [type prefix](https://github.com/commitizen/conventional-commit-types/blob/v3.0.0/index.json) in the PR title - [ ] ~added `!` to the type prefix if API or client breaking change~ - [x] targeted the correct branch (see [PR Targeting](https://github.com/cosmos/cosmos-sdk/blob/master/CONTRIBUTING.md#pr-targeting)) - [x] provided a link to the relevant issue or specification - [x] followed the guidelines for [building modules](https://github.com/cosmos/cosmos-sdk/blob/master/docs/building-modules) - [ ] ~included the necessary unit and integration [tests](https://github.com/cosmos/cosmos-sdk/blob/master/CONTRIBUTING.md#testing)~ - [x] added a changelog entry to `CHANGELOG.md` - [ ] ~included comments for [documenting Go code](https://blog.golang.org/godoc)~ - [ ] ~updated the relevant documentation or specification~ - [x] reviewed "Files changed" and left comments if necessary - [ ] confirmed all CI checks have passed ### Reviewers Checklist *All items are required. Please add a note if the item is not applicable and please add your handle next to the items reviewed if you only reviewed selected items.* I have... - [ ] confirmed the correct [type prefix](https://github.com/commitizen/conventional-commit-types/blob/v3.0.0/index.json) in the PR title - [ ] confirmed `!` in the type prefix if API or client breaking change - [ ] confirmed all author checklist items have been addressed - [ ] reviewed state machine logic - [ ] reviewed API design and naming - [ ] reviewed documentation is accurate - [ ] reviewed tests and test coverage - [ ] manually tested (if applicable) (cherry picked from commit 8b74157) # Conflicts: # types/module/module.go * fix conflict * remove files Co-authored-by: Tomas Tauber <[email protected]> Co-authored-by: marbar3778 <[email protected]>
JeancarloBarrios
pushed a commit
to agoric-labs/cosmos-sdk
that referenced
this issue
Sep 28, 2024
…) (cosmos#11107) * feat: extra logging in in-place store migrations (cosmos#10768) ## Description Closes: cosmos#10689 it'd also be good to add more fine-grained tracking of individual migrations, but there doesn't seem to be a quick way of doing it. Perhaps the best is to leave it to each migration implementation to add its own internal logging? --- ### Author Checklist *All items are required. Please add a note to the item if the item is not applicable and please add links to any relevant follow up issues.* I have... - [x] included the correct [type prefix](https://github.com/commitizen/conventional-commit-types/blob/v3.0.0/index.json) in the PR title - [ ] ~added `!` to the type prefix if API or client breaking change~ - [x] targeted the correct branch (see [PR Targeting](https://github.com/cosmos/cosmos-sdk/blob/master/CONTRIBUTING.md#pr-targeting)) - [x] provided a link to the relevant issue or specification - [x] followed the guidelines for [building modules](https://github.com/cosmos/cosmos-sdk/blob/master/docs/building-modules) - [ ] ~included the necessary unit and integration [tests](https://github.com/cosmos/cosmos-sdk/blob/master/CONTRIBUTING.md#testing)~ - [x] added a changelog entry to `CHANGELOG.md` - [ ] ~included comments for [documenting Go code](https://blog.golang.org/godoc)~ - [ ] ~updated the relevant documentation or specification~ - [x] reviewed "Files changed" and left comments if necessary - [ ] confirmed all CI checks have passed ### Reviewers Checklist *All items are required. Please add a note if the item is not applicable and please add your handle next to the items reviewed if you only reviewed selected items.* I have... - [ ] confirmed the correct [type prefix](https://github.com/commitizen/conventional-commit-types/blob/v3.0.0/index.json) in the PR title - [ ] confirmed `!` in the type prefix if API or client breaking change - [ ] confirmed all author checklist items have been addressed - [ ] reviewed state machine logic - [ ] reviewed API design and naming - [ ] reviewed documentation is accurate - [ ] reviewed tests and test coverage - [ ] manually tested (if applicable) (cherry picked from commit 8b74157) # Conflicts: # types/module/module.go * fix conflict * remove files Co-authored-by: Tomas Tauber <[email protected]> Co-authored-by: marbar3778 <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Summary
Right now, there is no feedback on the progress of in-place store migrations during network upgrades, hence node operators may be puzzled whether something went wrong.
Problem Definition
If a store migration is executed on a large state, it could take a long time. On existing large production networks on low spec or archival nodes, this can lead to a node being stuck at a single log entry for hours:
This can be confusing to a node operator who does not know when to expect the upgrade to complete or whether the upgrade is making any progress or something went wrong.
Proposal
It would be good to add some additional logging (e.g. which modules are being migrated) or potentially some new functionality to allow tracking migration progress (e.g. how many entry out of total entries were migrated).
Related: #9928 #10047
For Admin Use
The text was updated successfully, but these errors were encountered: