NEAR Indexer Framework 0.10.0 release notes

[khorolets]

Probably, will be included in nearcore 1.22.0

• ~ Sep 13, 2021 on testnet
• ~ Sep 20, 2021 on mainnet

Recommended commit hash 13141b4b78b402734a83df41fc4b7feb5b7ced01
At the moment of writing this release notes there is ongoing 1.21.0 release process
NEAR Indexer Framework 0.10.0 won't be included there
Consider waiting for 1.22.0 release candidate to be created

Release notes

• Add additional logs on Indexer Framework start
• Avoid double genesis validation by removing the explicit validation on Indexer instantiation
• Replaced the method how genesis is being read to optimize memory usage

Breaking changes

Since the change of reading genesis method to optimize memory usage. You'd be able to iterate over genesis records with near_config.genesis.for_each_record(|record| {...}).
Nothing is changed for you if your indexer does nothing about genesis records.

Feel free to leave questions in the comments below.

[frol]

@khorolets How big is the RAM usage reduction on testnet and mainnet?

[frol]

Answering my own question here: near/near-indexer-for-explorer#155 (comment)

I tested it with testnet genesis (5.4GB JSON file) on my local machine with local postgres database and it works great!

• peak RAM usage was 4.6GB, but it does not seem to be related to the indexer since while the genesis processing was happening, the RAM usage dropped down to 3.8GB and never go up
• time-wise it looks great!
  • 10 minutes to "Load config from" (not Indexer problem, nearcore does something heavy there)
  • 35 seconds to store accounts from genesis
  • 60 seconds to skip unrelated genesis records
  • 60 seconds to store access keys from genesis
• nearcore is not blocked by this storing the genesis data, which means that the implementation is async-friendly