Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

update arrow-rs deps to latest master #216

Merged
merged 4 commits into from
Apr 28, 2021

Conversation

alamb
Copy link
Contributor

@alamb alamb commented Apr 28, 2021

Update to the latest arrow-rs master along with a script to automate the process

Rationale for this change

Get the latest and greatest Arrow, and add a script to make it easier to do in the future

What changes are included in this PR?

  1. Update to latest arrow + code
  2. Script to automatically update dependencies in the future

Example

python update_arrow_deps.py
Updating files in /Users/alamb/Software/arrow-datafusion to use sha ed00e4d4a160cd5182bfafb81fee2240ec005014
updating /Users/alamb/Software/arrow-datafusion/Cargo.toml
updating /Users/alamb/Software/arrow-datafusion/ballista/rust/core/Cargo.toml
updating /Users/alamb/Software/arrow-datafusion/ballista/rust/scheduler/Cargo.toml
updating /Users/alamb/Software/arrow-datafusion/ballista/rust/executor/Cargo.toml
updating /Users/alamb/Software/arrow-datafusion/ballista/rust/client/Cargo.toml
updating /Users/alamb/Software/arrow-datafusion/datafusion/Cargo.toml
updating /Users/alamb/Software/arrow-datafusion/datafusion-examples/Cargo.toml
updating /Users/alamb/Software/arrow-datafusion/benchmarks/Cargo.toml

@alamb alamb force-pushed the alamb/update_deps_again branch from ce3cb51 to 051d323 Compare April 28, 2021 12:59
@codecov-commenter
Copy link

Codecov Report

Merging #216 (80ac2e9) into master (3371574) will not change coverage.
The diff coverage is n/a.

❗ Current head 80ac2e9 differs from pull request most recent head 8fc1889. Consider uploading reports for the commit 8fc1889 to get more accurate results
Impacted file tree graph

@@           Coverage Diff           @@
##           master     #216   +/-   ##
=======================================
  Coverage   76.39%   76.39%           
=======================================
  Files         134      134           
  Lines       23206    23206           
=======================================
  Hits        17728    17728           
  Misses       5478     5478           

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 3371574...8fc1889. Read the comment docs.

@alamb alamb marked this pull request as ready for review April 28, 2021 14:56
@@ -0,0 +1,83 @@
#!/usr/bin/env python
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am sorry if this causes 👁️ 🩸

Copy link
Member

@jorgecarleitao jorgecarleitao left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thanks a lot for this.

@Dandandan
Copy link
Contributor

Looks good 💯. I am wondering what the process for doing releases to crates.io is going to be? In that case we need to use release versions.

@alamb
Copy link
Contributor Author

alamb commented Apr 28, 2021

@Dandandan -- I would like to get to the point where we do much more frequent arrow releases (like maybe weekly) and then datafusion will just pick them up automatically

I was imagining, for example, releasing 4.1.0 shortly with these fixes.

But I don't really have a plan / sorted out a process for doing this yet -- @andygrove have you given the matter much thought?

@alamb alamb merged commit f43dc44 into apache:master Apr 28, 2021
@alamb alamb deleted the alamb/update_deps_again branch April 28, 2021 16:52
@andygrove
Copy link
Member

@alamb I agree that we want frequent releases. I have not had much time to think about what we do next. We should collaborate on a process in the google doc we have going for the migration as next step?

@alamb
Copy link
Contributor Author

alamb commented Apr 29, 2021

We should collaborate on a process in the google doc we have going for the migration as next step?

I agree @andygrove if you haven't had a chance, when I next get some time I will make a doc

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants