[WIP] Split trade statistics between recent data and historical data

[chimp1984]

First part: Add pruning support for trade statistics:

We add an interface for prunable PersistableNetworkPayloads so we can
handle it generically in the p2p module. We check at startup if our
persisted data has old elements and remove those. Data we receive from
the seed node at startup will get checked as well for old objects as
well data we receive from the network.

Next steps: Add historical data handling

[sqrrm]

This implementation throws away all old tradestats data and doesn't allow for any historical lookup. What is the plan for allowing historical data?

Instead of throwing data away during pruning it should probably be stored locally as HistoricalTradeStats which would work a bit like the split stores in #4233 except it's not a fixed block, but rather an append store but it can be excluded for all non historical data requests to seeds.

[sqrrm]

Is there a reason to not let this inherit from PersistableNetworkPayload? That would make more sense to me.

[chimp1984]

Good finding. Will change it.

[chimp1984]

This implementation throws away all old tradestats data and doesn't allow for any historical lookup. What is the plan for allowing historical data?

Instead of throwing data away during pruning it should probably be stored locally as HistoricalTradeStats which would work a bit like the split stores in #4233 except it's not a fixed block, but rather an append store but it can be excluded for all non historical data requests to seeds.

Yes, the historical data part is not impl. yet... Not 100% sure about the details how it will be done.

[freimair]

NACK

• pruning will not solve the problem: if bisq grows, we might need to prune after weeks or even days
• your proposal seems like it would constantly shift objects from recent to historical datastore. Wouldn't that mean that the historical data store will grow as fast as the single data store does now?
• adding code does not solve the underlying issue, it is just a risky fix that will be overpowered in months again
• that adds even more complexity right now and down the road with only minimal functional gain
• yet it leaves the existing code, which starts to become overpowered by the requirements and thus, desperately needs fixing, untouched and suffering

Requested Changes: come together in a call and talk about it, and yes, that will be a long one.

[freimair]

ah yes, and your plan sounds a lot like #4233, especially the splitting of data stores (only into 2 and keep updating the historical one)

I, however, do not see how you distinguish between recent and historical data, except by doing it by timestamp (which I believe you guys already agreed that this is NOT the way to go #4233 (comment))

[chimp1984]

Closes as its WIP. Will continue to look into it over the next weeks.