Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update data ingestion for chainsync #1734

Merged
merged 36 commits into from
Nov 16, 2024
Merged

Update data ingestion for chainsync #1734

merged 36 commits into from
Nov 16, 2024

Conversation

slundqui
Copy link
Contributor

@slundqui slundqui commented Nov 15, 2024

Updating data ingestion scripts.

  • Move "run data ingestion in background" functionality out of acquire_data.py logic and into outside script.
  • Adding ability to backfill calculate historical positions.
  • Allow pool info insertion logic to handle non-duplicate data.
  • Paginating event logs when getting from chain.

Copy link

codecov bot commented Nov 15, 2024

Codecov Report

Attention: Patch coverage is 89.70588% with 7 lines in your changes missing coverage. Please review.

Project coverage is 81.12%. Comparing base (401e2d9) to head (84bbc80).
Report is 1 commits behind head on main.

Files with missing lines Patch % Lines
src/agent0/chainsync/exec/acquire_data.py 70.58% 5 Missing ⚠️
src/agent0/chainsync/db/hyperdrive/chain_to_db.py 83.33% 1 Missing ⚠️
src/agent0/chainsync/exec/analyze_data.py 90.00% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1734      +/-   ##
==========================================
+ Coverage   81.06%   81.12%   +0.06%     
==========================================
  Files          96       96              
  Lines        7120     7106      -14     
==========================================
- Hits         5772     5765       -7     
+ Misses       1348     1341       -7     
Flag Coverage Δ
unittests 81.12% <89.70%> (+0.06%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@slundqui slundqui changed the title Update data ingestion script for chainsync Update data ingestion for chainsync Nov 15, 2024
@slundqui slundqui marked this pull request as ready for review November 15, 2024 06:38
@slundqui slundqui enabled auto-merge (squash) November 16, 2024 07:26
@slundqui slundqui merged commit 6ab90fe into delvtech:main Nov 16, 2024
4 checks passed
@slundqui slundqui deleted the db branch November 20, 2024 03:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants