Data agent benchmarking

[elmutt]

Measures data agent price & market cap responses versus benchmark values from Coingecko, Defillama, etc.

See config.py for prompt & coin list templates

Running:

1. cd submodules/benchmarks
2. Modify config.py with new prompts, coins & error tolerances
3. Run pip install -r requirements.txt
4. Run python benchmarks.py price for price benchmarks
5. Run python benchmarks.py mcap for market cap benchmarks

Considerations

• It runs very slow because of coingecko rate limits. Disabling coingecko adapter lets us reduce the time.sleep() in benchmarks.py and a little bit faster but the data agent can still be rate limited.

• The source of truth asset id is the coingecko id. Any new adapters will need some way of translating the coingecko id if they use something else. For example, the defillama adapter "just works" because they use coingecko ids.