Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Miri CI check reports status for only arrow-ord #5159

Closed
Jefffrey opened this issue Dec 2, 2023 · 0 comments · Fixed by #5243
Closed

Miri CI check reports status for only arrow-ord #5159

Jefffrey opened this issue Dec 2, 2023 · 0 comments · Fixed by #5243
Labels
bug development-process Related to development process of arrow-rs

Comments

@Jefffrey
Copy link
Contributor

Jefffrey commented Dec 2, 2023

Describe the bug

See here:

#!/bin/bash
#
# Script
#
# Must be run with nightly rust for example
# rustup default nightly
export MIRIFLAGS="-Zmiri-disable-isolation"
cargo miri setup
cargo clean
echo "Starting Arrow MIRI run..."
cargo miri test -p arrow-buffer
cargo miri test -p arrow-data --features ffi
cargo miri test -p arrow-schema --features ffi
cargo miri test -p arrow-array
cargo miri test -p arrow-arith --features simd
cargo miri test -p arrow-ord

Without set -e setting then bash script will continue even if earlier commands fail, with the return value of the script being the last command.

To Reproduce

Expected behavior

Should fail Miri CI check for any failure in the script

Additional context

Could fix by adding set -e at the top, but seems like some Miri checks are failing due to unsupported operations, e.g.

error: unsupported operation: inline assembly is not supported
   --> /home/runner/work/arrow-rs/arrow-rs/arrow-buffer/src/bigint/div.rs:196:9
    |
196 | /         std::arch::asm!(
197 | |             "div {divisor}",
198 | |             divisor = in(reg) divisor,
199 | |             inout("rax") quot,
200 | |             inout("rdx") rem,
201 | |             options(pure, nomem, nostack)
202 | |         );
    | |_________^ inline assembly is not supported
    |
    = help: this is likely not a bug in the program; it indicates that the program performed an operation that the interpreter does not support
    = note: BACKTRACE:
    = note: inside `arrow_buffer::bigint::div::div_rem_word` at /home/runner/work/arrow-rs/arrow-rs/arrow-buffer/src/bigint/div.rs:196:9: 202:10
note: inside closure
   --> /home/runner/work/arrow-rs/arrow-rs/arrow-buffer/src/bigint/div.rs:65:22
    |
65  |         let (q, r) = div_rem_word(rem, *d, divisor);
    |                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    = note: inside closure at /home/runner/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/iter/traits/iterator.rs:855:29: 855:36
    = note: inside `<std::slice::IterMut<'_, u64> as std::iter::DoubleEndedIterator>::rfold::<(), {closure@std::iter::Iterator::for_each::call<&mut u64, {closure@arrow_buffer::bigint::div::div_rem_small<4>::{closure#0}}>::{closure#0}}>` at /home/runner/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/iter/traits/double_ended.rs:307:21: 307:32
    = note: inside `<std::iter::Rev<std::slice::IterMut<'_, u64>> as std::iter::Iterator>::fold::<(), {closure@std::iter::Iterator::for_each::call<&mut u64, {closure@arrow_buffer::bigint::div::div_rem_small<4>::{closure#0}}>::{closure#0}}>` at /home/runner/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/iter/adapters/rev.rs:64:9: 64:33
    = note: inside `<std::iter::Rev<std::slice::IterMut<'_, u64>> as std::iter::Iterator>::for_each::<{closure@arrow_buffer::bigint::div::div_rem_small<4>::{closure#0}}>` at /home/runner/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/iter/traits/iterator.rs:858:9: 858:31
note: inside `arrow_buffer::bigint::div::div_rem_small::<4>`
   --> /home/runner/work/arrow-rs/arrow-rs/arrow-buffer/src/bigint/div.rs:64:5
    |
64  | /     numerator.iter_mut().rev().for_each(|d| {
65  | |         let (q, r) = div_rem_word(rem, *d, divisor);
66  | |         *d = q;
67  | |         rem = r;
68  | |     });
    | |______^
note: inside `arrow_buffer::bigint::div::div_rem::<4>`
   --> /home/runner/work/arrow-rs/arrow-rs/arrow-buffer/src/bigint/div.rs:39:16
    |
39  |         return div_rem_small(numerator, divisor[0]);
    |                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
note: inside `arrow_buffer::i256::div_rem`
   --> /home/runner/work/arrow-rs/arrow-rs/arrow-buffer/src/bigint/mod.rs:444:26
    |
444 |         let (div, rem) = div_rem(&a.as_digits(), &b.as_digits());
    |                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
note: inside `arrow_buffer::i256::wrapping_div`
   --> /home/runner/work/arrow-rs/arrow-rs/arrow-buffer/src/bigint/mod.rs:483:15
    |
483 |         match self.div_rem(other) {
    |               ^^^^^^^^^^^^^^^^^^^
note: inside `<arrow_buffer::i256 as arrow_array::ArrowNativeTypeOp>::div_wrapping`
   --> /home/runner/work/arrow-rs/arrow-rs/arrow-array/src/arithmetic.rs:273:1
    |
273 | native_type_op!(i256, i256::ZERO, i256::ONE);
    | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
note: inside `arithmetic::divide_and_round::<arrow_array::types::Decimal256Type>`
   --> arrow-arith/src/arithmetic.rs:179:13
    |
179 |     let d = input.div_wrapping(div);
    |             ^^^^^^^^^^^^^^^^^^^^^^^
note: inside closure
   --> arrow-arith/src/arithmetic.rs:123:15
    |
123 |         mul = divide_and_round::<Decimal256Type>(mul, divisor);
    |               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
note: inside `arity::try_binary_no_nulls::<&arrow_array::PrimitiveArray<arrow_array::types::Decimal128Type>, &arrow_array::PrimitiveArray<arrow_array::types::Decimal128Type>, {closure@arrow-arith/src/arithmetic.rs:118:56: 118:62}, arrow_array::types::Decimal128Type>`
   --> arrow-arith/src/arity.rs:393:35
    |
393 |             buffer.push_unchecked(op(a.value_unchecked(idx), b.value_unchecked(idx))?);
    |                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
note: inside `arity::try_binary::<&arrow_array::PrimitiveArray<arrow_array::types::Decimal128Type>, &arrow_array::PrimitiveArray<arrow_array::types::Decimal128Type>, {closure@arrow-arith/src/arithmetic.rs:118:56: 118:62}, arrow_array::types::Decimal128Type>`
   --> arrow-arith/src/arity.rs:293:9
    |
293 |         try_binary_no_nulls(len, a, b, op)
    |         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
note: inside `arithmetic::multiply_fixed_point_checked`
   --> arrow-arith/src/arithmetic.rs:118:5
    |
118 | /     try_binary::<_, _, _, Decimal128Type>(left, right, |a, b| {
119 | |         let a = i256::from_i128(a);
120 | |         let b = i256::from_i128(b);
121 | |
...   |
126 | |         })
127 | |     })
    | |______^
note: inside `arithmetic::tests::test_decimal_multiply_allow_precision_loss`
   --> arrow-arith/src/arithmetic.rs:217:22
    |
217 |         let result = multiply_fixed_point_checked(&a, &b, 28).unwrap();
    |                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
note: inside closure
   --> arrow-arith/src/arithmetic.rs:199:52
    |
198 |     #[test]
    |     ------- in this procedural macro expansion
199 |     fn test_decimal_multiply_allow_precision_loss() {
    |                                                    ^
    = note: this error originates in the macro `native_type_op` which comes from the expansion of the attribute macro `test` (in Nightly builds, run with -Z macro-backtrace for more info)

note: some details are omitted, run with `MIRIFLAGS=-Zmiri-backtrace=full` for a verbose backtrace

error: aborting due to 1 previous error; 1 warning emitted

error: test failed, to rerun pass `-p arrow-arith --lib`

Caused by:
  process didn't exit successfully: `/home/runner/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/bin/cargo-miri runner /home/runner/work/arrow-rs/arrow-rs/target/miri/x86_64-unknown-linux-gnu/debug/deps/arrow_arith-935c019359b6f7a3` (exit status: 1)
note: test exited abnormally; to see the full output pass --nocapture to the harness.

Of the crates that run Miri:

# Succeeds
cargo miri test -p arrow-buffer
# Succeeds
cargo miri test -p arrow-data --features ffi
# Succeeds
cargo miri test -p arrow-schema --features ffi
# Fails
cargo miri test -p arrow-array
# Fails
cargo miri test -p arrow-arith --features simd
# Succeeds
cargo miri test -p arrow-ord

Note that parquet crate also suffers failure when running Miri due to unsupported operation #614

@Jefffrey Jefffrey added the bug label Dec 2, 2023
@tustvold tustvold added the development-process Related to development process of arrow-rs label Dec 2, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug development-process Related to development process of arrow-rs
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants