Persistent local state for docs

[Frando]

We will want to persist some instance-local state for each doc. This includes:

• ASAP: Sync status (enabled/disabled) and list of peers to sync with. Either including their addresses, or persisting the addresses at the iroh-net level.
• Local labels for docs / petnames
• Later: Sync policies (ranges to sync). We don't support partial sync yet.
• Later: Download policies (ranges to download)

1. we should do now-ish I think, otherwise restarting a node means that you have to re-join all docs again for sync to work.

How/where do we want to store this info?

• Extend the iroh_sync::Store trait with set_local(namespace: NamespaceId, key: Vec<u8>, value: Vec<u8>) and get_local(namespace: NamespaceId, key: &[u8]) methods, which would persist to the same storage backend as replica data (so redb in the default case).
• Use an actual document, like a "instance config doc", and store its NamespaceId in a file in the IROH_DATA_DIR. Would need some care to not share this accidentally, but might also be a feature.
• Use redb directly in the upper iroh crate to persist this data
• Store in simple files in IROH_DATA_DIR

I'm leaning to the first idea of these, but not super sure yet. Ideas, opinions?

[dignifiedquire]

• Local labels for docs / petnames

Later: this is a can of worms, lets not open it before the release

[dignifiedquire]

• Sync status (enabled/disabled) and list of peers to sync with. Either including their addresses, or persisting the addresses at the iroh-net level.

Lets use a toml file, like a config file to store this. It should be easy to add and remove documents from this using your texteditor.

[b5]

Lets use a toml file, like a config file to store this. It should be easy to add and remove documents from this using your texteditor.

I second this, in particular b/c I don't think we're ready to have iroh nodes auto-subscribe to all documents by default on phones. Using a toml file means it'll be more of a console / CLI thing, and we can decide on if this needs to be baked into the core (thus shipping to all platforms) after we get a better sense of user expectations

[ramfox]

@Frando can we close this in favor of your thorough tracking issue #1490 ?