-
Notifications
You must be signed in to change notification settings - Fork 165
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: add iroh-sync and integrate into iroh node #1333
Commits on Aug 14, 2023
-
Configuration menu - View commit details
-
Copy full SHA for 3340613 - Browse repository at this point
Copy the full SHA 3340613View commit details -
Configuration menu - View commit details
-
Copy full SHA for 867d741 - Browse repository at this point
Copy the full SHA 867d741View commit details -
Configuration menu - View commit details
-
Copy full SHA for f5c51c0 - Browse repository at this point
Copy the full SHA f5c51c0View commit details -
Configuration menu - View commit details
-
Copy full SHA for 230a362 - Browse repository at this point
Copy the full SHA 230a362View commit details -
Configuration menu - View commit details
-
Copy full SHA for 350a486 - Browse repository at this point
Copy the full SHA 350a486View commit details -
Configuration menu - View commit details
-
Copy full SHA for e8e9158 - Browse repository at this point
Copy the full SHA e8e9158View commit details -
feat: WIP integration of sync and bytes
* removes content support from iroh-sync * adds a quick-and-dirty writable database to iroh-bytes (will be replaced with a better generic writable database soon) * adds a `Downloader` to queue get requests for individual hashes from individual peers * adds a `BlobStore` that combines the writable db with the downloader * adds a `Doc` abstraction that combines an iroh-sync `Replica` with a `BlobStore` to download content from peers on-demand * updates the sync repl example to plug it all together * also adds very basic persistence to `Replica` (encode to byte string) and uses this in the repl example
Configuration menu - View commit details
-
Copy full SHA for 9a9266e - Browse repository at this point
Copy the full SHA 9a9266eView commit details -
feat: proper REPL for sync example, and docs store
* make the REPL in the sync example work properly with rustyline for editing and reading input, shell-style argument parsing and clap for parsing commands * add a docs store for opening and closing docs * add author to doc struct
Configuration menu - View commit details
-
Copy full SHA for b1868ad - Browse repository at this point
Copy the full SHA b1868adView commit details -
Configuration menu - View commit details
-
Copy full SHA for 607fd81 - Browse repository at this point
Copy the full SHA 607fd81View commit details -
Configuration menu - View commit details
-
Copy full SHA for a0f96db - Browse repository at this point
Copy the full SHA a0f96dbView commit details -
Configuration menu - View commit details
-
Copy full SHA for e0e062c - Browse repository at this point
Copy the full SHA e0e062cView commit details -
Configuration menu - View commit details
-
Copy full SHA for 6347541 - Browse repository at this point
Copy the full SHA 6347541View commit details -
Configuration menu - View commit details
-
Copy full SHA for 82d4589 - Browse repository at this point
Copy the full SHA 82d4589View commit details -
Configuration menu - View commit details
-
Copy full SHA for 94163c7 - Browse repository at this point
Copy the full SHA 94163c7View commit details -
Configuration menu - View commit details
-
Copy full SHA for 7240fc5 - Browse repository at this point
Copy the full SHA 7240fc5View commit details -
Configuration menu - View commit details
-
Copy full SHA for 069a53b - Browse repository at this point
Copy the full SHA 069a53bView commit details -
fix: remove usage of unbounded channels
uses flume channels to allow for combined sync and async usage
Configuration menu - View commit details
-
Copy full SHA for ce9d809 - Browse repository at this point
Copy the full SHA ce9d809View commit details -
Configuration menu - View commit details
-
Copy full SHA for 2c9aa58 - Browse repository at this point
Copy the full SHA 2c9aa58View commit details -
Configuration menu - View commit details
-
Copy full SHA for 5a627b9 - Browse repository at this point
Copy the full SHA 5a627b9View commit details -
Configuration menu - View commit details
-
Copy full SHA for 5899d0e - Browse repository at this point
Copy the full SHA 5899d0eView commit details -
Configuration menu - View commit details
-
Copy full SHA for 43eff25 - Browse repository at this point
Copy the full SHA 43eff25View commit details -
Configuration menu - View commit details
-
Copy full SHA for 33883f8 - Browse repository at this point
Copy the full SHA 33883f8View commit details -
Configuration menu - View commit details
-
Copy full SHA for b487671 - Browse repository at this point
Copy the full SHA b487671View commit details -
Configuration menu - View commit details
-
Copy full SHA for e56e7aa - Browse repository at this point
Copy the full SHA e56e7aaView commit details -
Configuration menu - View commit details
-
Copy full SHA for b4358dd - Browse repository at this point
Copy the full SHA b4358ddView commit details -
Configuration menu - View commit details
-
Copy full SHA for c700721 - Browse repository at this point
Copy the full SHA c700721View commit details -
Configuration menu - View commit details
-
Copy full SHA for 6bdfaf6 - Browse repository at this point
Copy the full SHA 6bdfaf6View commit details -
Configuration menu - View commit details
-
Copy full SHA for 27e0923 - Browse repository at this point
Copy the full SHA 27e0923View commit details -
Configuration menu - View commit details
-
Copy full SHA for 6745dfa - Browse repository at this point
Copy the full SHA 6745dfaView commit details -
Configuration menu - View commit details
-
Copy full SHA for aeda868 - Browse repository at this point
Copy the full SHA aeda868View commit details -
Configuration menu - View commit details
-
Copy full SHA for 4de81b6 - Browse repository at this point
Copy the full SHA 4de81b6View commit details -
Configuration menu - View commit details
-
Copy full SHA for 494748b - Browse repository at this point
Copy the full SHA 494748bView commit details -
Configuration menu - View commit details
-
Copy full SHA for 193a33a - Browse repository at this point
Copy the full SHA 193a33aView commit details -
Configuration menu - View commit details
-
Copy full SHA for b0a7ab0 - Browse repository at this point
Copy the full SHA b0a7ab0View commit details -
feat(iroh-sync): implement file system backed for documents (#1315)
* start refactoring store into its own module * implement more details * works again * draft fs db and integrate error handling * fill out more of the implemenation * lifetime sadness * self referential fight: Rust 0 - Dig 1 * basic tests and range fixes * introduce Store trait and update tests to test against both impls * implement remove * integrate new storage into the example * implement iterators * fixes and more tests * clippy and deny cleanup
Configuration menu - View commit details
-
Copy full SHA for fb693b1 - Browse repository at this point
Copy the full SHA fb693b1View commit details -
Configuration menu - View commit details
-
Copy full SHA for 56fb90c - Browse repository at this point
Copy the full SHA 56fb90cView commit details -
Configuration menu - View commit details
-
Copy full SHA for e4fca32 - Browse repository at this point
Copy the full SHA e4fca32View commit details -
Configuration menu - View commit details
-
Copy full SHA for b33d3ef - Browse repository at this point
Copy the full SHA b33d3efView commit details -
Configuration menu - View commit details
-
Copy full SHA for cd17db8 - Browse repository at this point
Copy the full SHA cd17db8View commit details -
Configuration menu - View commit details
-
Copy full SHA for 722df34 - Browse repository at this point
Copy the full SHA 722df34View commit details -
Configuration menu - View commit details
-
Copy full SHA for f0d9ef2 - Browse repository at this point
Copy the full SHA f0d9ef2View commit details -
Configuration menu - View commit details
-
Copy full SHA for 111b117 - Browse repository at this point
Copy the full SHA 111b117View commit details -
Configuration menu - View commit details
-
Copy full SHA for 59077b1 - Browse repository at this point
Copy the full SHA 59077b1View commit details -
Configuration menu - View commit details
-
Copy full SHA for 437c031 - Browse repository at this point
Copy the full SHA 437c031View commit details -
Configuration menu - View commit details
-
Copy full SHA for 481b771 - Browse repository at this point
Copy the full SHA 481b771View commit details -
rework abstractions for integration in iroh node
* removed the `Doc`, `DocStore`, `BlobStore` abstractions, they were not the right way for now * Introduce `SyncEngine` that combines the gossip `LiveSync` with on_insert callbacks to download content, to replace the functionality from the above-mentioned abstractions * Rename `iroh_sync::get_replica` to `iroh_sync::open_replica` and expect them to be singletons per namespace * Various other minor fixes and improvements to logging
Configuration menu - View commit details
-
Copy full SHA for 8da512f - Browse repository at this point
Copy the full SHA 8da512fView commit details -
wip: integrate iroh sync and gossip
wip: more methods wip: integration of iroh-sync in iroh changes after rebase continue integration of iroh-sync in iroh node * removed the `Doc`, `DocStore`, `BlobStore` abstractions, they were not the right way for now * Introduce `SyncEngine` that combines the gossip `LiveSync` with on_insert callbacks to download content, to replace the functionality from the above-mentioned abstractions * Add RPC handlers on the `SyncEngine` for the most important operations * Introduce `iroh::client::Iroh`, a wrapper around the RPC client, and `iroh::client::Doc`, a wrapper around RPC client plus document id * Use the new client structs in the CLI * Rename `iroh_sync::get_replica` to `iroh_sync::open_replica` and expect them to be singletons per namespace * Various other minor fixes and improvements to logging
Configuration menu - View commit details
-
Copy full SHA for c76a1da - Browse repository at this point
Copy the full SHA c76a1daView commit details -
Configuration menu - View commit details
-
Copy full SHA for 7784510 - Browse repository at this point
Copy the full SHA 7784510View commit details -
Configuration menu - View commit details
-
Copy full SHA for d1caf19 - Browse repository at this point
Copy the full SHA d1caf19View commit details -
Configuration menu - View commit details
-
Copy full SHA for f80617a - Browse repository at this point
Copy the full SHA f80617aView commit details -
Configuration menu - View commit details
-
Copy full SHA for 134477a - Browse repository at this point
Copy the full SHA 134477aView commit details -
Configuration menu - View commit details
-
Copy full SHA for 5880f50 - Browse repository at this point
Copy the full SHA 5880f50View commit details -
Configuration menu - View commit details
-
Copy full SHA for e911696 - Browse repository at this point
Copy the full SHA e911696View commit details -
Configuration menu - View commit details
-
Copy full SHA for d021455 - Browse repository at this point
Copy the full SHA d021455View commit details -
Configuration menu - View commit details
-
Copy full SHA for c78fb5b - Browse repository at this point
Copy the full SHA c78fb5bView commit details -
Configuration menu - View commit details
-
Copy full SHA for c039f72 - Browse repository at this point
Copy the full SHA c039f72View commit details -
Configuration menu - View commit details
-
Copy full SHA for 910fd87 - Browse repository at this point
Copy the full SHA 910fd87View commit details -
Configuration menu - View commit details
-
Copy full SHA for f03b061 - Browse repository at this point
Copy the full SHA f03b061View commit details -
Configuration menu - View commit details
-
Copy full SHA for 1f864b5 - Browse repository at this point
Copy the full SHA 1f864b5View commit details -
Configuration menu - View commit details
-
Copy full SHA for 961bd3d - Browse repository at this point
Copy the full SHA 961bd3dView commit details -
Configuration menu - View commit details
-
Copy full SHA for 67dd342 - Browse repository at this point
Copy the full SHA 67dd342View commit details -
Configuration menu - View commit details
-
Copy full SHA for cb24396 - Browse repository at this point
Copy the full SHA cb24396View commit details -
Configuration menu - View commit details
-
Copy full SHA for 75679db - Browse repository at this point
Copy the full SHA 75679dbView commit details -
Configuration menu - View commit details
-
Copy full SHA for 69951d2 - Browse repository at this point
Copy the full SHA 69951d2View commit details -
Configuration menu - View commit details
-
Copy full SHA for 4d87cd3 - Browse repository at this point
Copy the full SHA 4d87cd3View commit details -
Configuration menu - View commit details
-
Copy full SHA for 14ce536 - Browse repository at this point
Copy the full SHA 14ce536View commit details -
Configuration menu - View commit details
-
Copy full SHA for 610d09f - Browse repository at this point
Copy the full SHA 610d09fView commit details -
Configuration menu - View commit details
-
Copy full SHA for a16ec9a - Browse repository at this point
Copy the full SHA a16ec9aView commit details -
Configuration menu - View commit details
-
Copy full SHA for ac5240e - Browse repository at this point
Copy the full SHA ac5240eView commit details -
Configuration menu - View commit details
-
Copy full SHA for 3bc1483 - Browse repository at this point
Copy the full SHA 3bc1483View commit details -
Configuration menu - View commit details
-
Copy full SHA for 9d5d06a - Browse repository at this point
Copy the full SHA 9d5d06aView commit details -
Configuration menu - View commit details
-
Copy full SHA for 23d138b - Browse repository at this point
Copy the full SHA 23d138bView commit details -
Configuration menu - View commit details
-
Copy full SHA for cc4c5a9 - Browse repository at this point
Copy the full SHA cc4c5a9View commit details -
Configuration menu - View commit details
-
Copy full SHA for d05e32c - Browse repository at this point
Copy the full SHA d05e32cView commit details -
Configuration menu - View commit details
-
Copy full SHA for ef0892d - Browse repository at this point
Copy the full SHA ef0892dView commit details -
Configuration menu - View commit details
-
Copy full SHA for a285c6d - Browse repository at this point
Copy the full SHA a285c6dView commit details -
Configuration menu - View commit details
-
Copy full SHA for 96c9a43 - Browse repository at this point
Copy the full SHA 96c9a43View commit details -
Configuration menu - View commit details
-
Copy full SHA for fbfdd27 - Browse repository at this point
Copy the full SHA fbfdd27View commit details -
Configuration menu - View commit details
-
Copy full SHA for 6daef61 - Browse repository at this point
Copy the full SHA 6daef61View commit details -
Configuration menu - View commit details
-
Copy full SHA for e596d7b - Browse repository at this point
Copy the full SHA e596d7bView commit details -
Configuration menu - View commit details
-
Copy full SHA for ec4e619 - Browse repository at this point
Copy the full SHA ec4e619View commit details -
Configuration menu - View commit details
-
Copy full SHA for 35f3218 - Browse repository at this point
Copy the full SHA 35f3218View commit details -
Configuration menu - View commit details
-
Copy full SHA for 4429435 - Browse repository at this point
Copy the full SHA 4429435View commit details -
Configuration menu - View commit details
-
Copy full SHA for d15b61d - Browse repository at this point
Copy the full SHA d15b61dView commit details -
Configuration menu - View commit details
-
Copy full SHA for 4c649c1 - Browse repository at this point
Copy the full SHA 4c649c1View commit details -
Configuration menu - View commit details
-
Copy full SHA for ac2bd41 - Browse repository at this point
Copy the full SHA ac2bd41View commit details -
Configuration menu - View commit details
-
Copy full SHA for 07653ca - Browse repository at this point
Copy the full SHA 07653caView commit details -
Configuration menu - View commit details
-
Copy full SHA for 925ff83 - Browse repository at this point
Copy the full SHA 925ff83View commit details
Commits on Aug 15, 2023
-
Configuration menu - View commit details
-
Copy full SHA for f5c0dd9 - Browse repository at this point
Copy the full SHA f5c0dd9View commit details -
Configuration menu - View commit details
-
Copy full SHA for c08fef8 - Browse repository at this point
Copy the full SHA c08fef8View commit details
Commits on Aug 17, 2023
-
Configuration menu - View commit details
-
Copy full SHA for 0287d4d - Browse repository at this point
Copy the full SHA 0287d4dView commit details -
Configuration menu - View commit details
-
Copy full SHA for a8e8093 - Browse repository at this point
Copy the full SHA a8e8093View commit details -
Configuration menu - View commit details
-
Copy full SHA for 839a855 - Browse repository at this point
Copy the full SHA 839a855View commit details -
Configuration menu - View commit details
-
Copy full SHA for cde53c3 - Browse repository at this point
Copy the full SHA cde53c3View commit details -
Configuration menu - View commit details
-
Copy full SHA for fc91c5e - Browse repository at this point
Copy the full SHA fc91c5eView commit details -
Configuration menu - View commit details
-
Copy full SHA for ff90fe6 - Browse repository at this point
Copy the full SHA ff90fe6View commit details -
Configuration menu - View commit details
-
Copy full SHA for 1fb6f8f - Browse repository at this point
Copy the full SHA 1fb6f8fView commit details -
Configuration menu - View commit details
-
Copy full SHA for b666f87 - Browse repository at this point
Copy the full SHA b666f87View commit details -
Configuration menu - View commit details
-
Copy full SHA for 4b50b64 - Browse repository at this point
Copy the full SHA 4b50b64View commit details -
Configuration menu - View commit details
-
Copy full SHA for 337fc1d - Browse repository at this point
Copy the full SHA 337fc1dView commit details -
Configuration menu - View commit details
-
Copy full SHA for 4df7796 - Browse repository at this point
Copy the full SHA 4df7796View commit details -
Configuration menu - View commit details
-
Copy full SHA for b103077 - Browse repository at this point
Copy the full SHA b103077View commit details -
Configuration menu - View commit details
-
Copy full SHA for 036c168 - Browse repository at this point
Copy the full SHA 036c168View commit details -
Configuration menu - View commit details
-
Copy full SHA for b990a72 - Browse repository at this point
Copy the full SHA b990a72View commit details -
Configuration menu - View commit details
-
Copy full SHA for a3bd92e - Browse repository at this point
Copy the full SHA a3bd92eView commit details
Commits on Aug 22, 2023
-
Configuration menu - View commit details
-
Copy full SHA for b2a92f1 - Browse repository at this point
Copy the full SHA b2a92f1View commit details -
Configuration menu - View commit details
-
Copy full SHA for 5bf0e9e - Browse repository at this point
Copy the full SHA 5bf0e9eView commit details -
Configuration menu - View commit details
-
Copy full SHA for 05035e3 - Browse repository at this point
Copy the full SHA 05035e3View commit details -
Configuration menu - View commit details
-
Copy full SHA for e655828 - Browse repository at this point
Copy the full SHA e655828View commit details -
Configuration menu - View commit details
-
Copy full SHA for 6615092 - Browse repository at this point
Copy the full SHA 6615092View commit details -
Configuration menu - View commit details
-
Copy full SHA for 6fa839d - Browse repository at this point
Copy the full SHA 6fa839dView commit details -
Configuration menu - View commit details
-
Copy full SHA for ab2d2db - Browse repository at this point
Copy the full SHA ab2d2dbView commit details -
Configuration menu - View commit details
-
Copy full SHA for 629289b - Browse repository at this point
Copy the full SHA 629289bView commit details -
Configuration menu - View commit details
-
Copy full SHA for 3779d27 - Browse repository at this point
Copy the full SHA 3779d27View commit details -
Configuration menu - View commit details
-
Copy full SHA for f6f9f9c - Browse repository at this point
Copy the full SHA f6f9f9cView commit details
Commits on Aug 23, 2023
-
Configuration menu - View commit details
-
Copy full SHA for 0968384 - Browse repository at this point
Copy the full SHA 0968384View commit details -
Configuration menu - View commit details
-
Copy full SHA for 452059b - Browse repository at this point
Copy the full SHA 452059bView commit details -
Configuration menu - View commit details
-
Copy full SHA for bbdf7bf - Browse repository at this point
Copy the full SHA bbdf7bfView commit details -
Configuration menu - View commit details
-
Copy full SHA for f571631 - Browse repository at this point
Copy the full SHA f571631View commit details -
Configuration menu - View commit details
-
Copy full SHA for f065781 - Browse repository at this point
Copy the full SHA f065781View commit details -
Configuration menu - View commit details
-
Copy full SHA for f00db58 - Browse repository at this point
Copy the full SHA f00db58View commit details -
Configuration menu - View commit details
-
Copy full SHA for 5e12659 - Browse repository at this point
Copy the full SHA 5e12659View commit details -
Configuration menu - View commit details
-
Copy full SHA for 2c540be - Browse repository at this point
Copy the full SHA 2c540beView commit details -
Configuration menu - View commit details
-
Copy full SHA for 136a994 - Browse repository at this point
Copy the full SHA 136a994View commit details -
Configuration menu - View commit details
-
Copy full SHA for 23911fc - Browse repository at this point
Copy the full SHA 23911fcView commit details -
Configuration menu - View commit details
-
Copy full SHA for e44fb83 - Browse repository at this point
Copy the full SHA e44fb83View commit details -
Configuration menu - View commit details
-
Copy full SHA for 0d432da - Browse repository at this point
Copy the full SHA 0d432daView commit details -
Configuration menu - View commit details
-
Copy full SHA for 502da40 - Browse repository at this point
Copy the full SHA 502da40View commit details -
Configuration menu - View commit details
-
Copy full SHA for a63fbf8 - Browse repository at this point
Copy the full SHA a63fbf8View commit details -
Configuration menu - View commit details
-
Copy full SHA for 3339add - Browse repository at this point
Copy the full SHA 3339addView commit details -
Configuration menu - View commit details
-
Copy full SHA for b4d2164 - Browse repository at this point
Copy the full SHA b4d2164View commit details -
Configuration menu - View commit details
-
Copy full SHA for fe781be - Browse repository at this point
Copy the full SHA fe781beView commit details -
Configuration menu - View commit details
-
Copy full SHA for 264dadf - Browse repository at this point
Copy the full SHA 264dadfView commit details -
Configuration menu - View commit details
-
Copy full SHA for 503ff23 - Browse repository at this point
Copy the full SHA 503ff23View commit details
Commits on Aug 24, 2023
-
fix(sync): properly drop rpc subscriptions, add doc status (#1396)
## Description So far in #1333, if a RPC or in-memory client called `doc.subscribe()` the event callback would never be dropped, even if the client did drop the event stream. This PR fixes this, by having the event callbacks return whether the callback should stay active or not. We can't use the removal token here, because calling `LiveSync::unsubscribe` from within the event callback would deadlock the actor. Also adds a a `LiveStatus` to the doc info RPC call. For now only contains the number of subscribers. More info, e.g. on peers, can come later. ## Notes & open questions * As of #1333 and unchanged by this PR: `doc.subscribe` will fail for documents that are not in the `LiveSync` (they are added via `doc.import` or `doc.start_sync`). This is unfortunate, because you'd often want to setup a subscription before starting sync, to catch all events. ## Change checklist - [x] Self-review. - [x] Documentation updates if relevant. - [x] Tests if relevant.
Configuration menu - View commit details
-
Copy full SHA for 0e3f169 - Browse repository at this point
Copy the full SHA 0e3f169View commit details -
Configuration menu - View commit details
-
Copy full SHA for c50bf53 - Browse repository at this point
Copy the full SHA c50bf53View commit details -
refactor(iroh): simplify and cleanup the sync algorithm (#1401)
switches to also use tokio::codec under the hood now. Also introduces a max message length of 1GiB for now.
Configuration menu - View commit details
-
Copy full SHA for d206252 - Browse repository at this point
Copy the full SHA d206252View commit details -
Configuration menu - View commit details
-
Copy full SHA for 8d803b0 - Browse repository at this point
Copy the full SHA 8d803b0View commit details -
Configuration menu - View commit details
-
Copy full SHA for 248f6b3 - Browse repository at this point
Copy the full SHA 248f6b3View commit details -
Configuration menu - View commit details
-
Copy full SHA for 3921eac - Browse repository at this point
Copy the full SHA 3921eacView commit details -
Configuration menu - View commit details
-
Copy full SHA for 39fda80 - Browse repository at this point
Copy the full SHA 39fda80View commit details -
Configuration menu - View commit details
-
Copy full SHA for 7690287 - Browse repository at this point
Copy the full SHA 7690287View commit details -
Configuration menu - View commit details
-
Copy full SHA for 0bc1efb - Browse repository at this point
Copy the full SHA 0bc1efbView commit details -
Configuration menu - View commit details
-
Copy full SHA for b7c7623 - Browse repository at this point
Copy the full SHA b7c7623View commit details