Skip to content

Import snapshot directly into hotstore for nodes running splitstore #10554

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
2 of 9 tasks
Tracked by #10886
ZenGround0 opened this issue Mar 23, 2023 · 1 comment · May be fixed by #12803
Open
2 of 9 tasks
Tracked by #10886

Import snapshot directly into hotstore for nodes running splitstore #10554

ZenGround0 opened this issue Mar 23, 2023 · 1 comment · May be fixed by #12803
Labels
area/chain Area: Chain effort/hours Effort: Hours kind/feature Kind: Feature need/team-input Hint: Needs Team Input P2 P2: Should be resolved

Comments

@ZenGround0
Copy link
Contributor

Checklist

  • This is not brainstorming ideas. If you have an idea you'd like to discuss, please open a new discussion on the lotus forum and select the category as Ideas.
  • I have a specific, actionable, and well motivated feature request to propose.

Lotus component

  • lotus daemon - chain sync
  • lotus fvm/fevm - Lotus FVM and FEVM interactions
  • lotus miner/worker - sealing
  • lotus miner - proving(WindowPoSt/WinningPoSt)
  • lotus JSON-RPC API
  • lotus message management (mpool)
  • Other

What is the motivation behind this feature request? Is your feature request related to a problem? Please describe.

Coldstore shouldn't track unwanted state in discard mode. Today we waste ~100 G of disk space no matter what when syncing from snapshot

Describe the solution you'd like

import-snaphsot daemon flag should import the chain into the hotstore if the node is configured to run with splitstore.

Basically repo should create a blockstore using hotstore instead of universal store location if applicable. We might want to make a new blockstore location that automatically does this or we could manually check the repo config during import.

Describe alternatives you've considered

We could also make it easier to purge the coldstore of snapshot state by improving chain prune cold-compact but it would be better to prevent this problem in the first place than to keep it around and also create a solution

Additional context

No response

@TippyFlitsUK TippyFlitsUK added P2 P2: Should be resolved need/team-input Hint: Needs Team Input area/chain Area: Chain and removed need/triage labels Apr 11, 2023
@jennijuju jennijuju added this to the Splitstore Enhancements milestone May 12, 2023
@jennijuju jennijuju removed this from the Splitstore Enhancements milestone May 19, 2023
@jennijuju jennijuju added the effort/hours Effort: Hours label Jun 6, 2023
@ZenGround0
Copy link
Contributor Author

Looking at this more today the problem seems to be that

  1. Node startup depends on initial state in universal blockstore
  2. Splitstore startup depends on node startup

So we have to unravel these dependencies if we want to save the disk space

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/chain Area: Chain effort/hours Effort: Hours kind/feature Kind: Feature need/team-input Hint: Needs Team Input P2 P2: Should be resolved
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants