Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor Staking Models and Create Parquet Stake Processor #647

Open
wants to merge 15 commits into
base: main
Choose a base branch
from

Conversation

dermanyang
Copy link
Contributor

@dermanyang dermanyang commented Dec 13, 2024

Purpose

Leaving in Postgres lookups for delegator_balances in this parquet processor as there aren't sufficient staking transactions to have an impact on performance. Further, the logic is not possible to rewrite without them.

/delegator_pools models

  • Refactor DelegatorPoolBalanceMetadata, PoolBalanceMetadata, DelegatorPoolBalance, CurrentDelegatorPoolBalance to use the postgres/common pattern.
  • DelegatorPool left as-is as all fields are postgres and parquet compatible.

/current_delegated_voter

  • Moved CurrentDelegatorVoter to db/common and left as-is as all fields are postgres and parquet compatible

/delegator_balances models

  • Refactor CurrentDelegatorBalance, DelegatorBalance to use the postgres/common pattern.
  • Moved CurrentDelegatorBalanceQuery to db/common/ but left as-is since we won't have lookups in parquet.

/delegator_activities models

  • Refactor DelegatedStakingActivity to use the postgres/common pattern.

/staking_pool_voter models

  • Moved CurrentStakingPoolVoter to db/common and left as-is as all fields are postgres and parquet compatible

`/proposal_voters models

  • Refactor RawProposalVote to use common/postgres pattern.

Testing

image

TODOs

  • Will migrate additional models as requested from Analytics team

Copy link

codecov bot commented Dec 16, 2024

Codecov Report

Attention: Patch coverage is 60.00000% with 480 lines in your changes missing coverage. Please review.

Project coverage is 47.6%. Comparing base (0cad206) to head (d7e434d).

Files with missing lines Patch % Lines
.../src/parquet_processors/parquet_stake_processor.rs 0.0% 111 Missing ⚠️
...parquet_stake_processor/parquet_stake_extractor.rs 0.0% 78 Missing ⚠️
...b/common/models/stake_models/delegator_balances.rs 86.6% 53 Missing ⚠️
rust/processor/src/processors/stake_processor.rs 60.3% 42 Missing ⚠️
.../models/stake_models/parquet_delegator_balances.rs 0.0% 41 Missing ⚠️
...common/models/stake_models/delegator_activities.rs 61.1% 28 Missing ⚠️
...c/db/common/models/stake_models/proposal_voters.rs 33.3% 22 Missing ⚠️
...db/postgres/models/stake_models/delegator_pools.rs 47.6% 22 Missing ⚠️
rust/sdk-processor/src/parquet_processors/mod.rs 0.0% 20 Missing ⚠️
...odels/stake_models/parquet_delegator_activities.rs 0.0% 19 Missing ⚠️
... and 5 more
Additional details and impacted files
@@           Coverage Diff           @@
##            main    #647     +/-   ##
=======================================
- Coverage   48.0%   47.6%   -0.4%     
=======================================
  Files        232     242     +10     
  Lines      27159   27656    +497     
=======================================
+ Hits       13055   13187    +132     
- Misses     14104   14469    +365     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@dermanyang dermanyang changed the title Refactor Staking Models to postgres, parquet, and common (raw) Refactor Staking Models and Create Parquet Stake Processor Dec 19, 2024
@dermanyang dermanyang marked this pull request as ready for review December 19, 2024 00:18
@dermanyang dermanyang requested a review from yuunlimm December 19, 2024 00:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant