Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Migrate parquet account txn processor to sdk #626

Merged
merged 1 commit into from
Dec 12, 2024
Merged

Migrate parquet account txn processor to sdk #626

merged 1 commit into from
Dec 12, 2024

Conversation

yuunlimm
Copy link
Contributor

@yuunlimm yuunlimm commented Dec 6, 2024 •
edited
Loading

Description

migrating parquet account txn processor to sdk

Test Plan

mroe rows in parquet is expected
Screenshot 2024-12-11 at 4 19 37 PM
Screenshot 2024-12-11 at 4 14 03 PM

This was referenced Dec 6, 2024
Merged
Merged
Merged
@yuunlimm Graphite App
Copy link
Contributor Author

yuunlimm commented Dec 6, 2024 •
edited
Loading

This stack of pull requests is managed by Graphite. Learn more about stacking.

@yuunlimm yuunlimm mentioned this pull request Dec 6, 2024
Merged
@codecov Codecov
Copy link

codecov bot commented Dec 6, 2024 •
edited
Loading

Codecov Report

Attention: Patch coverage is 21.18644% with 186 lines in your changes missing coverage. Please review.

Project coverage is 48.3%. Comparing base (69a0ba1) to head (c8f2719).
Report is 1 commits behind head on main.

Files with missing lines Patch % Lines
...ocessors/parquet_account_transactions_processor.rs 0.0% 104 Missing ⚠️
...rocessor/parquet_account_transactions_extractor.rs 0.0% 52 Missing ⚠️
...unt_transaction_models/raw_account_transactions.rs 80.0% 12 Missing ⚠️
...transaction_models/parquet_account_transactions.rs 0.0% 8 Missing ⚠️
rust/sdk-processor/src/parquet_processors/mod.rs 0.0% 5 Missing ⚠️
...k-processor/src/config/indexer_processor_config.rs 0.0% 3 Missing ⚠️
...r/src/processors/account_transactions_processor.rs 0.0% 1 Missing ⚠️
rust/sdk-processor/src/config/processor_config.rs 50.0% 1 Missing ⚠️
Additional details and impacted files 
@@           Coverage Diff           @@
##            main    #626     +/-   ##
=======================================
- Coverage   48.7%   48.3%   -0.4%     
=======================================
  Files        210     214      +4     
  Lines      26223   26397    +174     
=======================================
+ Hits       12775   12776      +1     
- Misses     13448   13621    +173

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@yuunlimm yuunlimm force-pushed the 12-05-migrate_parquet_txn_metadata_processor_to_sdk branch from 69a1b32 to 19cb208 Compare December 6, 2024 19:52
@yuunlimm yuunlimm force-pushed the 12-06-migrate_parquet_account_txn_processor_to_sdk branch from aed4387 to 63a30df Compare December 6, 2024 19:52
@yuunlimm yuunlimm force-pushed the 12-05-migrate_parquet_txn_metadata_processor_to_sdk branch from 19cb208 to 3bc956d Compare December 6, 2024 20:12
@yuunlimm yuunlimm force-pushed the 12-06-migrate_parquet_account_txn_processor_to_sdk branch from 63a30df to 92335a8 Compare December 6, 2024 20:12
@yuunlimm yuunlimm force-pushed the 12-05-migrate_parquet_txn_metadata_processor_to_sdk branch from 3bc956d to b6dfdc6 Compare December 6, 2024 20:38
@yuunlimm yuunlimm force-pushed the 12-06-migrate_parquet_account_txn_processor_to_sdk branch 4 times, most recently from 0cedb4b to bdf9888 Compare December 9, 2024 17:26
@yuunlimm yuunlimm marked this pull request as ready for review December 9, 2024 20:15
@yuunlimm yuunlimm requested review from dermanyang and rtso December 9, 2024 20:15
@yuunlimm yuunlimm mentioned this pull request Dec 10, 2024
Merged
dermanyang
dermanyang reviewed Dec 10, 2024
View reviewed changes
rust/processor/src/db/common/models/account_transaction_models/raw_account_transactions.rs
/// We will do 1 level of redirection for now (e.g. if it's an object, we will record the owner as account address).
/// We will also consider transactions that the account signed or is part of a multi sig / multi agent.
/// TODO: recursively find the parent account of an object
/// TODO: include table items in the detection path
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are these TODOs still relevant?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just for context, they're from this original PR: aptos-labs/aptos-core#9076

@dermanyang
Copy link
Contributor

did you do a sanity check on number of rows in the equivalent version range on the respective legacy processor?

This was referenced Dec 10, 2024
Merged
Merged
Merged
Merged
@yuunlimm yuunlimm force-pushed the 12-05-migrate_parquet_txn_metadata_processor_to_sdk branch from b6dfdc6 to e1b7479 Compare December 10, 2024 20:09
@yuunlimm yuunlimm force-pushed the 12-06-migrate_parquet_account_txn_processor_to_sdk branch from bdf9888 to 8cebf59 Compare December 10, 2024 20:09
@yuunlimm yuunlimm force-pushed the 12-05-migrate_parquet_txn_metadata_processor_to_sdk branch from e1b7479 to a61e32f Compare December 10, 2024 21:49
@yuunlimm yuunlimm force-pushed the 12-06-migrate_parquet_account_txn_processor_to_sdk branch from 8cebf59 to 60cc57a Compare December 10, 2024 21:49
@yuunlimm yuunlimm force-pushed the 12-05-migrate_parquet_txn_metadata_processor_to_sdk branch from a61e32f to ecfeae8 Compare December 10, 2024 21:52
@yuunlimm yuunlimm force-pushed the 12-06-migrate_parquet_account_txn_processor_to_sdk branch from 60cc57a to 11a7977 Compare December 10, 2024 21:52
This was referenced Dec 10, 2024
Merged
Merged
Merged
Merged
@yuunlimm yuunlimm force-pushed the 12-05-migrate_parquet_txn_metadata_processor_to_sdk branch from ecfeae8 to cc6872f Compare December 11, 2024 21:02
@yuunlimm yuunlimm force-pushed the 12-06-migrate_parquet_account_txn_processor_to_sdk branch from 11a7977 to 106eb92 Compare December 11, 2024 21:02
@yuunlimm yuunlimm force-pushed the 12-05-migrate_parquet_txn_metadata_processor_to_sdk branch from cc6872f to 877caaf Compare December 11, 2024 21:07
@yuunlimm yuunlimm force-pushed the 12-06-migrate_parquet_account_txn_processor_to_sdk branch from 106eb92 to 24ace6a Compare December 11, 2024 21:07
@yuunlimm yuunlimm force-pushed the 12-05-migrate_parquet_txn_metadata_processor_to_sdk branch from 877caaf to f5f38f0 Compare December 11, 2024 22:18
@yuunlimm yuunlimm force-pushed the 12-06-migrate_parquet_account_txn_processor_to_sdk branch from 24ace6a to b0aa109 Compare December 11, 2024 22:18
@yuunlimm yuunlimm force-pushed the 12-05-migrate_parquet_txn_metadata_processor_to_sdk branch from f5f38f0 to d49cc2e Compare December 11, 2024 23:42
@yuunlimm yuunlimm force-pushed the 12-06-migrate_parquet_account_txn_processor_to_sdk branch from b0aa109 to 36d1ed3 Compare December 11, 2024 23:42
@yuunlimm yuunlimm force-pushed the 12-05-migrate_parquet_txn_metadata_processor_to_sdk branch from d49cc2e to 764a852 Compare December 11, 2024 23:53
@yuunlimm yuunlimm force-pushed the 12-06-migrate_parquet_account_txn_processor_to_sdk branch from 36d1ed3 to 5e8bc02 Compare December 11, 2024 23:53
@yuunlimm yuunlimm force-pushed the 12-05-migrate_parquet_txn_metadata_processor_to_sdk branch from 764a852 to c468b4c Compare December 12, 2024 02:23
@yuunlimm yuunlimm force-pushed the 12-06-migrate_parquet_account_txn_processor_to_sdk branch from 5e8bc02 to 08d3a8f Compare December 12, 2024 02:23
@yuunlimm yuunlimm mentioned this pull request Dec 12, 2024
Closed
Base automatically changed from 12-05-migrate_parquet_txn_metadata_processor_to_sdk to 11-26-add_sdk_version_of_parquet_fa_processor December 12, 2024 02:54
@yuunlimm yuunlimm changed the base branch from 11-26-add_sdk_version_of_parquet_fa_processor to main December 12, 2024 03:08
@yuunlimm yuunlimm force-pushed the 12-06-migrate_parquet_account_txn_processor_to_sdk branch from 08d3a8f to c8f2719 Compare December 12, 2024 03:16
@yuunlimm yuunlimm merged commit 789a402 into main Dec 12, 2024
11 checks passed
@yuunlimm Graphite App
Copy link
Contributor Author

Merge activity

  • Dec 11, 10:27 PM EST: A user merged this pull request with Graphite.

@yuunlimm yuunlimm deleted the 12-06-migrate_parquet_account_txn_processor_to_sdk branch December 12, 2024 03:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@dermanyang dermanyang dermanyang approved these changes

@rtso rtso Awaiting requested review from rtso

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@yuunlimm @dermanyang