Gossip pruned

[njgheorghita]

What was wrong?

fixes #981

How was it fixed?

• add content that's dropped from local storage to propagation content vector
• limit offer message in propagation to 64 content keys (according to spec)

To-Do

• Clean up commit history and use conventional commits.

[KolbyML]

[morph-dev]

Nicely done. Couple of suggestion, nothing major (except maybe calculating distance correctly).

[morph-dev]

I think this could be simplified to something like this:

interested_content.sort_by_cached_key(|(key, value)| {
  let key_as_array: [u8; 32] = std::array::from_fn(|i| key[i]);
  XorMetric::distance(&key_as_array, &enr.node_id().raw());
});

While writing this, I realized that we are calculating distance from content key, not content id, which I think is wrong. Is that intentional?
I would say that enrs_and_content should most likely be HashMap<String, Vec<(TContentKey, Vec<u8>)>>.

[njgheorghita]

Ahh, yup my mistake, we should be calculating on the id.

[morph-dev]

I think this can be simplified to:
self.store.insert(...).and(Ok(vec![]))

Optionally and in addition, you can let these functions return dropped content and handle it once in a put function above.

[morph-dev]

Same as state store, this could be:
self.store(...).and(Ok(vec![])).

Or if you prefer this:
self.store(...).map(|_| vec![]).

[morph-dev]

nit: maybe this

Suggested change

	/// Returns a list of keys that were evicted from the store, which are gossiped into the
	/// Returns a list of keys that were evicted from the store, which should be gossiped into the

[morph-dev]

nit: I would still return content_size as well. This should save us some calculations and make sure we are using the right value.

[morph-dev]

nit: Update function documentation.

[morph-dev]

nit: I don't think that you need to declare type here, so simple let mut dropped_content = vec![]; should be enough.

Also, you can do something like this:

let mut dropped_content = if self.pruning_strategy.should_prune(&self.usage_stats) {
    self.prune()?
} else {
    vec![]
}