Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a "compact" command #4701

Closed
arajasek opened this issue Nov 3, 2020 · 6 comments
Closed

Add a "compact" command #4701

arajasek opened this issue Nov 3, 2020 · 6 comments
Assignees
Labels
area/chain Area: Chain area/ux Area: UX effort/hours Effort: Hours good first issue Good for newcomers kind/feature Kind: Feature

Comments

@arajasek
Copy link
Contributor

arajasek commented Nov 3, 2020

Inspired by #4547, our docs currently say:

## Compacting the chain data

It is possible to "prune" the current chain data used by Lotus to reduce the disk footprint of a Lotus node by resyncing from a minimal snapshot. This is a quick overview of the steps:

1. Stop the lotus daemon, i.e. `lotus daemon stop`)
2. Remove the contents of the `datastore/chain/` folder in the Lotus path, i.e. `rm -rf ~/.lotus/datastore/chain/*`.
3. Start the daemon using a minimal snapshot as explained above, i.e. `lotus daemon --import-snapshot https://fil-chain-snapshots-fallback.s3.amazonaws.com/mainnet/minimal_finality_stateroots_latest.car`.

We probably shouldn't be telling people to directly delete folders. Adding some lotus compact command that does this in one step is probably the best thing to do.

@jennijuju
Copy link
Member

Another solution is actually have an option to trigger regular chain datastore pruning

@jennijuju jennijuju added the area/chain Area: Chain label Feb 24, 2021
@arajasek
Copy link
Contributor Author

arajasek commented Mar 1, 2021

This should largely get implemented by #4992

@jennijuju
Copy link
Member

Add documentation for split blockstore, which implements this feature by nature.

@vyzo
Copy link
Contributor

vyzo commented Mar 22, 2021

Note that as part of #5808 I am adding a lotus-shed command to gc the hotstore.
We can do this as an additional command in that vein, that garbage collects the coldstore.
We will have to finish #5808 first however, it still has a couple of small issues with hotstore misses.

@parkan
Copy link

parkan commented Jul 8, 2021

running the daemon with --import-snapshot a second time appears to blow away the chain data and restart the sync, would that be equivalent to the steps in the docs, or is it more destructive?

I'll open another issue to warn on unintentional --import-snapshot if not

(brought to you by restarting the daemon to reload config and unintentionally restarting the sync)

@arajasek
Copy link
Contributor Author

Splitstore is the correct way to do this, we can close this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/chain Area: Chain area/ux Area: UX effort/hours Effort: Hours good first issue Good for newcomers kind/feature Kind: Feature
Projects
None yet
Development

No branches or pull requests

5 participants