Example/Test Model Benchmarks (Canonical WandB runs)

[cat-state]

🚀 The feature, motivation, and pitch

If we had links to benchmarks for the example (and/or test) models, it would be easier to add new models, and keep track of improvements in method implementations. Additionally, during refactoring, it would allow checking that no performance degrading changes were introduced.

This can be a minimal version of #13

Alternatives

No response

Additional context

No response

[albertsun1]

Hey! I'm new to contributing to trlx, would it be worth for me to give this a go for the ppo/ilql sentiment examples?

[maxreciprocate]

@cat-state something like that? vwxyzjn/cleanrl#307

[cat-state]

@albertsun1

Hey! I'm new to contributing to trlx, would it be worth for me to give this a go for the ppo/ilql sentiment examples?

Sure, although you might need compute?
@reciprocated maybe we should make a single-node config version that can be finetuned on a single gpu fast?

[cat-state]

So I see that WandB actually lists the commit hash used for a run. So if we could find/tag TRLX runs in wandb then each commit could be matched up to a specific state of the repository.

[maxreciprocate]

Sure, although you might need compute? @reciprocated maybe we should make a single-node config version that can be finetuned on a single gpu fast?

fwiw {ppo,ilql}_config.yml were meant to be single gpu, up to a batch_size, since they both use gpt2 small

[maxreciprocate]

Resolved with #357