Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Clean up CI workflows by removing "matrix" strategy, simplifying names #3196

Merged
merged 2 commits into from
Aug 21, 2022
Merged

Clean up CI workflows by removing "matrix" strategy, simplifying names #3196

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
d4a0f5a
Clean up CI workflows by removing "matrix" strategy, simplifying names
alamb Aug 18, 2022
ca349a4
fix
alamb Aug 18, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
105 changes: 33 additions & 72 deletions .github/workflows/rust.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,14 +25,10 @@ on:
jobs:
# Check crate compiles
linux-build-lib:
name: Build Libraries on AMD64 Rust ${{ matrix.rust }}
name: cargo check
runs-on: ubuntu-latest
strategy:
matrix:
arch: [amd64]
rust: [stable]
container:
image: ${{ matrix.arch }}/rust
image: amd64/rust
env:
# Disable full debug symbol generation to speed up CI build and keep memory down
# "1" means line tables only, which is useful for panic tracebacks.
Expand All @@ -49,7 +45,7 @@ jobs:
- name: Setup Rust toolchain
uses: ./.github/actions/setup-builder
with:
rust-version: ${{ matrix.rust }}
rust-version: stable
- name: Check workspace in debug mode
run: |
cargo check
Expand All @@ -65,15 +61,11 @@ jobs:

# test the crate
linux-test:
name: Test Workspace on AMD64 Rust ${{ matrix.rust }}
name: cargo test (amd64)
needs: [linux-build-lib]
runs-on: ubuntu-latest
strategy:
matrix:
arch: [amd64]
rust: [stable]
container:
image: ${{ matrix.arch }}/rust
image: amd64/rust
env:
# Disable full debug symbol generation to speed up CI build and keep memory down
# "1" means line tables only, which is useful for panic tracebacks.
Expand Down Expand Up @@ -101,7 +93,7 @@ jobs:
- name: Setup Rust toolchain
uses: ./.github/actions/setup-builder
with:
rust-version: ${{ matrix.rust }}
rust-version: stable
- name: Run tests
run: |
export PATH=$PATH:$HOME/d/protoc/bin
Expand All @@ -115,7 +107,7 @@ jobs:
cargo run --example avro_sql --features=datafusion/avro

integration-test:
name: "Integration Test"
name: "Compare to postgres"
needs: [linux-build-lib]
runs-on: ubuntu-latest
services:
Expand Down Expand Up @@ -171,12 +163,8 @@ jobs:
POSTGRES_PASSWORD: postgres

windows:
name: Test on Windows Rust ${{ matrix.rust }}
runs-on: ${{ matrix.os }}
strategy:
matrix:
os: [windows-latest]
rust: [stable]
name: cargo test (win64)
runs-on: windows-latest
steps:
- uses: actions/checkout@v3
with:
Expand All @@ -195,8 +183,8 @@ jobs:
# with a OS-dependent path.
- name: Setup Rust toolchain
run: |
rustup toolchain install ${{ matrix.rust }}
rustup default ${{ matrix.rust }}
rustup toolchain install stable
rustup default stable
rustup component add rustfmt
- name: Run tests
shell: bash
Expand All @@ -208,12 +196,8 @@ jobs:
RUSTFLAGS: "-C debuginfo=0"

macos:
name: Test on MacOS Rust ${{ matrix.rust }}
runs-on: ${{ matrix.os }}
strategy:
matrix:
os: [macos-latest]
rust: [stable]
name: cargo test (mac)
runs-on: macos-latest
steps:
- uses: actions/checkout@v3
with:
Expand All @@ -233,8 +217,8 @@ jobs:
# with a OS-dependent path.
- name: Setup Rust toolchain
run: |
rustup toolchain install ${{ matrix.rust }}
rustup default ${{ matrix.rust }}
rustup toolchain install stable
rustup default stable
rustup component add rustfmt
- name: Run tests
shell: bash
Expand All @@ -245,14 +229,11 @@ jobs:
RUSTFLAGS: "-C debuginfo=0"

test-datafusion-pyarrow:
name: cargo test pyarrow (amd64)
needs: [linux-build-lib]
runs-on: ubuntu-latest
strategy:
matrix:
arch: [amd64]
rust: [stable]
container:
image: ${{ matrix.arch }}/rust
image: amd64/rust
env:
# Disable full debug symbol generation to speed up CI build and keep memory down
# "1" means line tables only, which is useful for panic tracebacks.
Expand All @@ -277,7 +258,7 @@ jobs:
- name: Setup Rust toolchain
uses: ./.github/actions/setup-builder
with:
rust-version: ${{ matrix.rust }}
rust-version: stable
- name: Run tests
run: |
cd datafusion
Expand All @@ -299,12 +280,8 @@ jobs:
run: ci/scripts/rust_fmt.sh

coverage:
name: Coverage
name: coverage
runs-on: ubuntu-latest
strategy:
matrix:
arch: [ amd64 ]
rust: [ stable ]
steps:
- uses: actions/checkout@v3
with:
Expand All @@ -321,8 +298,8 @@ jobs:
protoc --version
- name: Setup Rust toolchain
run: |
rustup toolchain install ${{ matrix.rust }}
rustup default ${{ matrix.rust }}
rustup toolchain install stable
rustup default stable
rustup component add rustfmt clippy
- name: Cache Cargo
uses: actions/cache@v3
Expand All @@ -342,15 +319,11 @@ jobs:
run: bash <(curl -s https://codecov.io/bash)

clippy:
name: Clippy
name: clippy
needs: [linux-build-lib]
runs-on: ubuntu-latest
strategy:
matrix:
arch: [amd64]
rust: [stable]
container:
image: ${{ matrix.arch }}/rust
image: amd64/rust
env:
# Disable full debug symbol generation to speed up CI build and keep memory down
# "1" means line tables only, which is useful for panic tracebacks.
Expand All @@ -368,20 +341,16 @@ jobs:
- name: Setup Rust toolchain
uses: ./.github/actions/setup-builder
with:
rust-version: ${{ matrix.rust }}
rust-version: stable
- name: Install Clippy
run: |
rustup component add clippy
- name: Run clippy
run: ci/scripts/rust_clippy.sh

miri-checks:
name: MIRI
name: cargo miri test (amd64)
runs-on: ubuntu-latest
strategy:
matrix:
arch: [amd64]
rust: [nightly-2022-01-17]
steps:
- uses: actions/checkout@v3
with:
Expand All @@ -395,8 +364,8 @@ jobs:
key: ${{ runner.os }}-cargo-miri-${{ hashFiles('**/Cargo.lock') }}
- name: Setup Rust toolchain
run: |
rustup toolchain install ${{ matrix.rust }}
rustup default ${{ matrix.rust }}
rustup toolchain install nightly-2022-01-17
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why use this version？
Is there a possibility that this version needs to be updated later?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we probably pinned the version a while ago when some nightly had issues. I think updating it would be a great idea (and as you say a good follow on PR)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Follow on PR: #3215 (will need some work)

rustup default nightly-2022-01-17
rustup component add rustfmt clippy miri
- name: Run Miri Checks
env:
Expand All @@ -411,15 +380,11 @@ jobs:

# Check answers are correct when hash values collide
hash-collisions:
name: Test Hash Collisions on AMD64 Rust ${{ matrix.rust }}
name: cargo test hash collisions (amd64)
needs: [linux-build-lib]
runs-on: ubuntu-latest
strategy:
matrix:
arch: [amd64]
rust: [stable]
container:
image: ${{ matrix.arch }}/rust
image: amd64/rust
env:
# Disable full debug symbol generation to speed up CI build and keep memory down
# "1" means line tables only, which is useful for panic tracebacks.
Expand All @@ -437,23 +402,19 @@ jobs:
- name: Setup Rust toolchain
uses: ./.github/actions/setup-builder
with:
rust-version: ${{ matrix.rust }}
rust-version: stable
- name: Run tests
run: |
cd datafusion
# Force all hash values to collide
cargo test --all --features=force_hash_collisions

cargo-toml-formatting-checks:
name: Check Cargo.toml formatting
name: check Cargo.toml formatting
needs: [linux-build-lib]
runs-on: ubuntu-latest
strategy:
matrix:
arch: [amd64]
rust: [stable]
container:
image: ${{ matrix.arch }}/rust
image: amd64/rust
env:
# Disable full debug symbol generation to speed up CI build and keep memory down
# "1" means line tables only, which is useful for panic tracebacks.
Expand All @@ -471,7 +432,7 @@ jobs:
- name: Setup Rust toolchain
uses: ./.github/actions/setup-builder
with:
rust-version: ${{ matrix.rust }}
rust-version: stable
- name: Install cargo-tomlfmt
run: |
which cargo-tomlfmt || cargo install cargo-tomlfmt
Expand Down