feat(iroh-bytes): bring back validation #2107
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…a concrete type. This is necessary to send progress senders via actor messages, but we also probably don't want to use a concrete progress sender type in the actor messges.
# Conflicts: # Cargo.lock # iroh-base/Cargo.toml # iroh-bytes/src/store/bao_file.rs # iroh-bytes/src/store/file.rs # iroh-bytes/src/store/file/tests.rs # iroh-bytes/src/store/mem.rs # iroh-bytes/src/store/readonly_mem.rs
BaoFileHandle is exactly the entry.
- separate fsck aka consistency check and validate
rklaehn
force-pushed
the
enhanced-validation
branch
from
a258a3a to
8ef9d24
Compare
rklaehn
force-pushed
the
enhanced-validation
branch
from
8ef9d24 to
f58f071
Compare
# Conflicts: # iroh-bytes/src/store/fs/validate.rs # iroh-cli/src/commands/doctor.rs
it is no longer used in validate
rklaehn
force-pushed
the
enhanced-validation
branch
from
0370ada to
47e321a
Compare
…res-buffered also better comments.
# Conflicts: # iroh-cli/src/commands/blob.rs # iroh/src/client/blobs.rs # iroh/src/node/rpc.rs # iroh/src/rpc_protocol.rs
dignifiedquire
approved these changes
Mar 22, 2024
matheus23
pushed a commit
that referenced
this pull request
Nov 14, 2024
## Description feat(iroh-bytes): bring back validation This brings back validation, and also separates validation and consistency check and validation. Rationale: - consistency check is highly dependent on the store impl. E.g. for an in memory impl it might not even make sense. - validate is much more expensive, but can be mostly implemented in a store impl independent way. It just needs access to the data. consistency check output at very high verbosity: (basically dumps the db and then validates each entry) <img width="1383" alt="image" src="https://github.com/n0-computer/iroh/assets/248257/84463e9a-a671-4c17-9c7d-968308d38f34"> validate output: (validates all complete files, lists ranges for all partial files) <img width="1168" alt="image" src="https://github.com/n0-computer/iroh/assets/248257/bd9336c6-d978-4948-b65c-4fd6b4fd56e8"> ## Notes & open questions - Note: repairing by downgrading complete entries to partial entries is not yet implemented. It is a bit tricky. - Note: both are implemented as a RPC command for a running store, and in iroh doctor for a separate file. ## Change checklist - [ ] Self-review. - [ ] Documentation updates if relevant. - [ ] Tests if relevant.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
feat(iroh-bytes): bring back validation
This brings back validation, and also separates validation and consistency check and validation.
Rationale:
consistency check output at very high verbosity:
(basically dumps the db and then validates each entry)
validate output:
(validates all complete files, lists ranges for all partial files)
Notes & open questions
Change checklist