Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Data pipeline to postgres #611

Merged
merged 23 commits into from
Jul 5, 2023
Merged

Data pipeline to postgres #611

merged 23 commits into from
Jul 5, 2023

Conversation

slundqui
Copy link

@slundqui slundqui commented Jul 1, 2023 •
edited
Loading

Reworks acquire_data.py to read/write to/from postgres instead of from file.

@vercel
Copy link

vercel bot commented Jul 1, 2023 •
edited
Loading

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
elf-simulations ✅ Ready (Inspect) Visit Preview 💬 Add feedback Jul 5, 2023 5:11pm

Sheng Lundquist added 19 commits July 3, 2023 18:12
Fixing a bug with getting queried block timestamp instead of latest b…
4ec782f 
…lock timestamp
Removing unused import
603aa78
Using numeric and timestamp types in postgres. Contract interface cas…
f697d53 
…ts results as fixedpoint
Acquire data now successfully fills transactions based on last entry …
e9a7bac 
…into db
Small formatting changes
dc51f53
Using sqlalchemy mapper to define schema
b5d8858
Updating pool info test
589f8e4
Updating simple test
d7bcdf0
Removing unused imports
4a08625
Initial commit for transaction data
3ccb073
Updating schema for transactions
8b48dfe
Fixing an issue with only looking at mined blocks instead of current …
371fa88 
…block
Adding gaseUsed field
26ea403
Renaming varialbe
a9df9f3
Adding tests for transactions. Updating autoincrement to work with te…
71e02f8 
…sts.
Added getter functions from postgres. Added tests
5e56f67
Fixing demo to use database and postgres interface
1120136
Adding block number as index for ticker
9d1bb36
pylint
4ceb668
@slundqui slundqui force-pushed the postgres_updates branch from 411c83d to 4ceb668 Compare July 4, 2023 01:12
@slundqui slundqui marked this pull request as ready for review July 4, 2023 01:12
dpaiton
dpaiton reviewed Jul 4, 2023
View reviewed changes
dpaiton
dpaiton approved these changes Jul 4, 2023
View reviewed changes
Copy link
Contributor

@dpaiton dpaiton left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left some comments, but otherwise it looks great. Did you try syncing this up with the full demo pipeline (devnet -> bot trades -> postgres -> streamlit)? I think that would be a good test to do before merging.

@slundqui
Copy link
Author

slundqui commented Jul 5, 2023

Left some comments, but otherwise it looks great. Did you try syncing this up with the full demo pipeline (devnet -> bot trades -> postgres -> streamlit)? I think that would be a good test to do before merging.

This was tested with the demo pipeline.

@slundqui slundqui merged commit 6d2010c into delvtech:main Jul 5, 2023
@slundqui slundqui deleted the postgres_updates branch July 5, 2023 18:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@dpaiton dpaiton dpaiton approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@slundqui @dpaiton