Eval pipeline

[dhall1995]

Hi there,

Thanks for this piece of work & the associated codebase. It's fantastic!

I was wondering whether you guys have an internal eval pipeline you use during training? Does this just consist of the evaluation scripts included in trainer.py?

What i'm imagining is similar to the notebook tutorials you guys provide but perhaps configurable and outputting some metrics on some standard tasks. It would, for example, be great to recreate the plots you guys provide in the paper which demonstrate model capability scaling with number of cells used in pre-training. In the future these kind of standardised tasks to evaluate on could help decide which extensions are useful or not.