Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[parquet-sdk][token_v2] migrate TokenDataV2 and CurrentTokenDataV2 #636

Merged
merged 1 commit into from
Dec 16, 2024
Merged

[parquet-sdk][token_v2] migrate TokenDataV2 and CurrentTokenDataV2 #636

merged 1 commit into from
Dec 16, 2024

Conversation

yuunlimm
Copy link
Contributor

@yuunlimm yuunlimm commented Dec 10, 2024 •
edited
Loading

Description

TestPlan

  • Compared the number of rows from DE teams BigQuery table, testing BigQuery table, legacy processor table, and sdk processor table

Current Token Datas

Screenshot 2024-12-11 at 7 56 31 PM
Screenshot 2024-12-11 at 7 56 25 PM

test table

Screenshot 2024-12-11 at 9 40 36 PM

Token Datas

Screenshot 2024-12-11 at 7 55 41 PM

Screenshot 2024-12-11 at 7 55 39 PM

test table

Screenshot 2024-12-11 at 9 39 33 PM

This was referenced Dec 10, 2024
Merged
Merged
Merged
Merged
Merged
Merged
Merged
Merged
@yuunlimm Graphite App
Copy link
Contributor Author

yuunlimm commented Dec 10, 2024 •
edited
Loading

This stack of pull requests is managed by Graphite. Learn more about stacking.

@yuunlimm yuunlimm mentioned this pull request Dec 10, 2024
Merged
@codecov Codecov
Copy link

codecov bot commented Dec 10, 2024 •
edited
Loading

Codecov Report

Attention: Patch coverage is 43.69748% with 201 lines in your changes missing coverage. Please review.

Project coverage is 48.0%. Comparing base (676f4e8) to head (df1767d).
Report is 1 commits behind head on main.

Files with missing lines Patch % Lines
...b/parquet/models/token_v2_models/v2_token_datas.rs 0.0% 67 Missing ⚠️
...t_token_v2_processor/parquet_token_v2_extractor.rs 0.0% 43 Missing ⚠️
...s/parquet_processors/parquet_token_v2_processor.rs 0.0% 28 Missing ⚠️
...ust/processor/src/processors/token_v2_processor.rs 35.8% 25 Missing ⚠️
...processor/src/processors/nft_metadata_processor.rs 0.0% 14 Missing ⚠️
rust/sdk-processor/src/parquet_processors/mod.rs 0.0% 10 Missing ⚠️
...c/parquet_processors/parquet_token_v2_processor.rs 0.0% 9 Missing ⚠️
...quet/models/token_v2_models/v2_token_ownerships.rs 0.0% 5 Missing ⚠️
Additional details and impacted files 
@@           Coverage Diff           @@
##            main    #636     +/-   ##
=======================================
+ Coverage   47.8%   48.0%   +0.1%     
=======================================
  Files        224     225      +1     
  Lines      26873   26918     +45     
=======================================
+ Hits       12866   12924     +58     
+ Misses     14007   13994     -13

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@yuunlimm yuunlimm force-pushed the 12-09-_parquet-sdk_token_v2_migrate_tokenactivityv2 branch from 04cdd33 to 9d696b3 Compare December 10, 2024 20:10
@yuunlimm yuunlimm force-pushed the 12-10-_parquet-sdk_token_v2_migrate_tokendatav2_and_currenttokendatav2 branch from 24a6f77 to cc70a9d Compare December 10, 2024 20:10
@yuunlimm yuunlimm force-pushed the 12-09-_parquet-sdk_token_v2_migrate_tokenactivityv2 branch from 9d696b3 to 8ec8c22 Compare December 10, 2024 21:49
@yuunlimm yuunlimm force-pushed the 12-10-_parquet-sdk_token_v2_migrate_tokendatav2_and_currenttokendatav2 branch from cc70a9d to ae12560 Compare December 10, 2024 21:49
@yuunlimm yuunlimm force-pushed the 12-09-_parquet-sdk_token_v2_migrate_tokenactivityv2 branch from 8ec8c22 to 89d4083 Compare December 10, 2024 21:52
@yuunlimm yuunlimm force-pushed the 12-10-_parquet-sdk_token_v2_migrate_tokendatav2_and_currenttokendatav2 branch from ae12560 to 7a6a7f9 Compare December 10, 2024 21:53
@yuunlimm yuunlimm force-pushed the 12-09-_parquet-sdk_token_v2_migrate_tokenactivityv2 branch from 89d4083 to 1e88d0b Compare December 10, 2024 22:21
@yuunlimm yuunlimm force-pushed the 12-10-_parquet-sdk_token_v2_migrate_tokendatav2_and_currenttokendatav2 branch from 7a6a7f9 to f7d158f Compare December 10, 2024 22:21
@yuunlimm yuunlimm mentioned this pull request Dec 10, 2024
Merged
@yuunlimm yuunlimm force-pushed the 12-09-_parquet-sdk_token_v2_migrate_tokenactivityv2 branch from 1e88d0b to db30b8a Compare December 10, 2024 23:53
@yuunlimm yuunlimm force-pushed the 12-10-_parquet-sdk_token_v2_migrate_tokendatav2_and_currenttokendatav2 branch from f7d158f to 69d5c9b Compare December 10, 2024 23:53
This was referenced Dec 11, 2024
Merged
Merged
Merged
@yuunlimm yuunlimm force-pushed the 12-09-_parquet-sdk_token_v2_migrate_tokenactivityv2 branch from db30b8a to 610b02c Compare December 11, 2024 21:03
@yuunlimm yuunlimm force-pushed the 12-10-_parquet-sdk_token_v2_migrate_tokendatav2_and_currenttokendatav2 branch from 69d5c9b to 3690bb3 Compare December 11, 2024 21:03
@yuunlimm yuunlimm force-pushed the 12-09-_parquet-sdk_token_v2_migrate_tokenactivityv2 branch from 610b02c to 136dcaf Compare December 11, 2024 21:07
@yuunlimm yuunlimm force-pushed the 12-10-_parquet-sdk_token_v2_migrate_tokendatav2_and_currenttokendatav2 branch from 3690bb3 to 1e4e874 Compare December 11, 2024 21:07
@yuunlimm yuunlimm force-pushed the 12-09-_parquet-sdk_token_v2_migrate_tokenactivityv2 branch from 136dcaf to d9654f1 Compare December 11, 2024 22:18
@yuunlimm yuunlimm force-pushed the 12-10-_parquet-sdk_token_v2_migrate_tokendatav2_and_currenttokendatav2 branch from 1e4e874 to ece7950 Compare December 11, 2024 22:18
@yuunlimm yuunlimm force-pushed the 12-09-_parquet-sdk_token_v2_migrate_tokenactivityv2 branch from d9654f1 to 6eadf7a Compare December 11, 2024 23:42
@yuunlimm yuunlimm force-pushed the 12-10-_parquet-sdk_token_v2_migrate_tokendatav2_and_currenttokendatav2 branch from ece7950 to 5193ef9 Compare December 11, 2024 23:42
@yuunlimm yuunlimm force-pushed the 12-09-_parquet-sdk_token_v2_migrate_tokenactivityv2 branch from 6eadf7a to 2448db8 Compare December 11, 2024 23:53
@yuunlimm yuunlimm force-pushed the 12-10-_parquet-sdk_token_v2_migrate_tokendatav2_and_currenttokendatav2 branch from 5193ef9 to e9d7817 Compare December 11, 2024 23:53
@yuunlimm yuunlimm force-pushed the 12-09-_parquet-sdk_token_v2_migrate_tokenactivityv2 branch from 2448db8 to a80d5d2 Compare December 12, 2024 02:23
@yuunlimm yuunlimm mentioned this pull request Dec 12, 2024
Closed
@yuunlimm yuunlimm force-pushed the 12-10-_parquet-sdk_token_v2_migrate_tokendatav2_and_currenttokendatav2 branch from e9d7817 to 5d501e4 Compare December 12, 2024 02:23
@yuunlimm yuunlimm marked this pull request as ready for review December 12, 2024 03:28
@yuunlimm yuunlimm force-pushed the 12-09-_parquet-sdk_token_v2_migrate_tokenactivityv2 branch from a80d5d2 to da99b1b Compare December 12, 2024 03:29
@yuunlimm yuunlimm force-pushed the 12-10-_parquet-sdk_token_v2_migrate_tokendatav2_and_currenttokendatav2 branch from 5d501e4 to c70afcd Compare December 12, 2024 03:30
@yuunlimm yuunlimm requested a review from dermanyang December 12, 2024 05:41
dermanyang
dermanyang reviewed Dec 12, 2024
View reviewed changes
rust/processor/src/db/parquet/models/token_v2_models/v2_token_datas.rs Outdated
supply: Some(raw_item.supply.unwrap().to_string()), // todo: handle error
largest_property_version_v1: raw_item.largest_property_version_v1.unwrap().to_u64(),
token_uri: raw_item.token_uri,
token_properties: canonical_json::to_string(&raw_item.token_properties).unwrap(), // todo: handle error
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems like the error handling should be included in this change. Does it fail silently at the moment?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updated1

@yuunlimm yuunlimm force-pushed the 12-09-_parquet-sdk_token_v2_migrate_tokenactivityv2 branch from da99b1b to 7829964 Compare December 13, 2024 21:40
@yuunlimm yuunlimm force-pushed the 12-10-_parquet-sdk_token_v2_migrate_tokendatav2_and_currenttokendatav2 branch from c70afcd to 6c21d80 Compare December 13, 2024 21:40
@yuunlimm yuunlimm force-pushed the 12-09-_parquet-sdk_token_v2_migrate_tokenactivityv2 branch from 7829964 to 5fb6366 Compare December 13, 2024 23:27
@yuunlimm yuunlimm requested a review from dermanyang December 13, 2024 23:28
Base automatically changed from 12-09-_parquet-sdk_token_v2_migrate_tokenactivityv2 to main December 14, 2024 01:05
@yuunlimm yuunlimm force-pushed the 12-10-_parquet-sdk_token_v2_migrate_tokendatav2_and_currenttokendatav2 branch from 6c21d80 to 4dd8e80 Compare December 16, 2024 16:38
@yuunlimm yuunlimm force-pushed the 12-10-_parquet-sdk_token_v2_migrate_tokendatav2_and_currenttokendatav2 branch from 4dd8e80 to df1767d Compare December 16, 2024 18:01
@yuunlimm yuunlimm merged commit b66e25a into main Dec 16, 2024
11 checks passed
@yuunlimm yuunlimm deleted the 12-10-_parquet-sdk_token_v2_migrate_tokendatav2_and_currenttokendatav2 branch December 16, 2024 18:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@dermanyang dermanyang dermanyang approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@yuunlimm @dermanyang