[parquet-sdk]migrate objects [pt2]

[yuunlimm]

Description

• migrate objects
• for parquet, we want to avoid lookups for deleted resource. so we replaced with default values, which will be handled during the load as a part of qc pipeline.

                 Self {
                        transaction_version: txn_version,
                        write_set_change_index,
                        object_address: resource.address.clone(),
                        owner_address: DELETED_RESOURCE_OWNER_ADDRESS.to_string(),   <-------
                        state_key_hash: resource.state_key_hash.clone(),
                        guid_creation_num: BigDecimal::default(),       <-------
                        allow_ungated_transfer: false,     <-------
                        is_deleted: true,
                        untransferrable: false,          <-------
                        block_timestamp: chrono::NaiveDateTime::default(),      <-------

these fields are the ones from the prev owner

Test Plan

• Compared the number of rows from DE teams BigQuery table, testing BigQuery table, legacy processor table, and sdk processor table

objects

current objects

[yuunlimm]

• [parquet-sdk]migrate objects [pt2] #641 👈 (View in Graphite)
• [parquet-sdk]migrate objects [pt1] #640
• [parquet-sdk][token_v2] migrate token ownership table [pt2] #638
• main

This stack of pull requests is managed by Graphite. Learn more about stacking.

[codecov]

Codecov Report

Attention: Patch coverage is 40.09324% with 257 lines in your changes missing coverage. Please review.

Project coverage is 48.0%. Comparing base (fdfb3a3) to head (a0c8208).
Report is 1 commits behind head on main.

Files with missing lines	Patch %	Lines
...rc/parquet_processors/parquet_objects_processor.rs	0.0%	100 Missing ⚠️
.../src/db/parquet/models/object_models/v2_objects.rs	0.0%	43 Missing ⚠️
...uet_objects_processor/parquet_objects_extractor.rs	0.0%	37 Missing ⚠️
...c/db/common/models/object_models/raw_v2_objects.rs	58.1%	31 Missing ⚠️
rust/processor/src/processors/objects_processor.rs	85.9%	16 Missing ⚠️
rust/sdk-processor/src/parquet_processors/mod.rs	0.0%	8 Missing ⚠️
...ust/processor/src/processors/token_v2_processor.rs	50.0%	7 Missing ⚠️
.../models/token_v2_models/raw_v2_token_ownerships.rs	40.0%	6 Missing ⚠️
...s/parquet_processors/parquet_token_v2_processor.rs	0.0%	3 Missing ⚠️
...k-processor/src/config/indexer_processor_config.rs	0.0%	2 Missing ⚠️
... and 4 more

Additional details and impacted files

@@           Coverage Diff           @@
##            main    #641     +/-   ##
=======================================
+ Coverage   46.6%   48.0%   +1.3%     
=======================================
  Files        229     232      +3     
  Lines      27015   27159    +144     
=======================================
+ Hits       12616   13055    +439     
+ Misses     14399   14104    -295     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[dermanyang]

Spoke offline; we thought it doesn't make sense to have a required query_retries and q_retry_delay when the client isn't present. At the same time, having all as option could lead to some user error. Agreed on introducing a new struct DbConnection as an Optional parameter with all three fields as required.

[yuunlimm]

@ying-w

[dermanyang]

🔥