Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Inconsistent output between pretty print and CSV writer for Arrow #3513

Closed
alamb opened this issue Jan 11, 2023 · 1 comment · Fixed by #3514
Closed

Inconsistent output between pretty print and CSV writer for Arrow #3513

alamb opened this issue Jan 11, 2023 · 1 comment · Fixed by #3514
Labels
arrow Changes to the arrow crate bug

Comments

@alamb
Copy link
Contributor

alamb commented Jan 11, 2023

Describe the bug
The formatting of floats as printed by pretty_print_batches and the CSV Writer is inconsistent

To Reproduce

fn main() {

    let arr = Float64Array::from(vec![Some(1.0), None, Some(2.0)]);
    let batch = RecordBatch::try_from_iter(vec![
        ("col", Arc::new(arr) as _)
    ]).unwrap();


    println!("Display");
    println!("{}", pretty_format_batches(&[batch.clone()]).unwrap());

    println!("CSV");
    let mut writer = Writer::new(std::io::stdout());
    writer.write(&batch).unwrap();

}

Produces the output:

Display
+-----+
| col |
+-----+
| 1   |
|     |
| 2   |
+-----+
CSV
col
1.0
""
2.0

Note how pretty print shows 1 and csv shows 1.0

Expected behavior
I expect the displays to be the same (likely by making the csv writer consistent with the pretty printer)

Additional context
See other usecase on on apache/datafusion#4876

I think #3483 from @JayjeetAtGithub covers one potential implementation mechanism

@tustvold
Copy link
Contributor

label_issue.py automatically added labels {'arrow'} from #3514

@tustvold tustvold added the arrow Changes to the arrow crate label Feb 10, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
arrow Changes to the arrow crate bug
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants