DataFusion 8.0.0 Release

[andygrove]

Is your feature request related to a problem or challenge? Please describe what you are trying to do.
I would like to propose that we release DataFusion 8.0.0 in mid-May. This will be around 3 months after the last major release.

Let's use this issue to coordinate and track subtasks.

Prep tasks

• Update update_datafusion_versions.py to include all crates #2423
• minor: update versions and paths in changelog scripts #2429
• Move data-access crate to same level as other crates? #2424
• datafusion-cli crate has circular dependency #2433
• Create separate ballista-cli crate #2493
• Review and fix up rustdoc warnings? Review docs for each crate - skipped this

Release PR

• Update version numbers in Cargo.toml
• Update version numbers in User Guide and READMEs

Post-release tasks

• Publish user guide
• Prepare release notes draft in Google Doc here
• Publish blog post

[alamb]

I think this is a great idea. Thanks @andygrove

Also related might be #2327 where we were talking about more frequent releases to crates.io -- I don't think it has any bearing on the 8.0 release directly.

Here are some suggested subtasks:

1. Figure out if we want to do a ballista release to crates.io at the same time
2. Enumerate and prepare to release the new subcrates (I think we had datafusion, datafusion-expr, and datafusion-common last release, but we have several new crates this time)
3. Update versions / do dry run of uploading to crates.io

[andygrove]

I created a separate issue for releasing Ballista 0.7.0

[andygrove]

@alamb I was heading towards having separate releases for DataFusion and Ballista until I realized that datafusion-cli depends on both ballista and datafusion which introduces a circular dependency as noted in #2433

I now think we need to release everything at once, with datafusion-cli as the last crate that gets published.

[alamb]

I now think we need to release everything at once, with datafusion-cli as the last crate that gets published.

👍

The situation is non ideal -- hopefully we can find a better way somehow (after the 8.0.0 release)