Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add toolbox for ar compress and uncompress #529

Merged
merged 5 commits into from
Jun 30, 2023
Merged

Conversation

jingchen2222
Copy link
Collaborator

@jingchen2222 jingchen2222 commented Jun 28, 2023

Movivation

In this CR we are going to provide a AR toolbox for AR operations including upload/download , parquet compress and parse. etc.

This is a part of work for coldstart and recover #471


PR-Codex overview

This PR focuses on introducing an ArToolBox module and refactoring the RollupExecutor module.

Detailed summary

  • Added ArToolBox module for handling Arweave related operations.
  • Refactored RollupExecutor to use ArToolBox for compressing and uploading record batches.
  • Removed unused imports and dependencies.

The following files were skipped due to too many changes: src/node/src/ar_toolbox.rs

✨ Ask PR-Codex anything about this PR by commenting with /codex {your question}

@jingchen2222 jingchen2222 requested a review from imotai June 28, 2023 05:31
@jingchen2222 jingchen2222 self-assigned this Jun 28, 2023
src/node/src/ar_toolbox.rs Outdated Show resolved Hide resolved
@codecov-commenter
Copy link

codecov-commenter commented Jun 28, 2023

Codecov Report

❗ No coverage uploaded for pull request base (main@e1250ba). Click here to learn what that means.
The diff coverage is 77.77%.

❗ Your organization is not using the GitHub App Integration. As a result you may experience degraded service beginning May 15th. Please install the Github App Integration for your organization. Read more.

@@           Coverage Diff           @@
##             main     #529   +/-   ##
=======================================
  Coverage        ?   55.03%           
=======================================
  Files           ?       72           
  Lines           ?    13897           
  Branches        ?       61           
=======================================
  Hits            ?     7648           
  Misses          ?     6249           
  Partials        ?        0           
Flag Coverage Δ
rust 55.03% <77.77%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
src/node/src/rollup_executor.rs 30.94% <57.14%> (ø)
src/node/src/ar_toolbox.rs 78.86% <78.86%> (ø)

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

let mut mutations = Vec::new();
let payloads = record_batch
.column_by_name("payload")
.unwrap()
Copy link
Contributor

@imotai imotai Jun 30, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

unwrap will panic if some error.

Copy link
Contributor

@imotai imotai left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@imotai imotai merged commit 65a6c26 into main Jun 30, 2023
@imotai imotai deleted the feat/add_ar_toolbox branch July 24, 2023 01:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants