migrations

[dimiandre]

We need a way to migrate tables and update old blocks data, otherwise every time there is a structure change we need to resync the indexer

[rllola]

Hi!

It is a good point. After merging this PR migrations would be not needed for the most part. It will store decoded data in JSON and not in tables anymore.

The only time we would need migrations would be for change in the more meta part of the protocol which we don't expect to change. Also because we expect a lot of data and those data are available, we would prefer re-syncing over migration. It is faster because when syncing we don't have indexes created yet and the data can be extracted from a node so dropping a table doesn't mean they are lost. I also suspect migration over index data to be slower than actually starting from scratch so you don't get anything from this. Of course this could evolve but for now it is not needed.

[opsecx]

We need a way to migrate tables and update old blocks data, otherwise every time there is a structure change we need to resync the indexer

Agree, was discussing this with Lola in discord the other day. Creating a new issue re some of this, may be of help to you too eventually.