Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve error messages for parallel parquet writer "Unable to send array to writer!" #11397

Closed
alamb opened this issue Jul 10, 2024 · 1 comment · Fixed by #11399
Closed

Improve error messages for parallel parquet writer "Unable to send array to writer!" #11397

alamb opened this issue Jul 10, 2024 · 1 comment · Fixed by #11399
Assignees
Labels
enhancement New feature or request

Comments

@alamb
Copy link
Contributor

alamb commented Jul 10, 2024

@devinjdangelo noted on #11345 that if an error happens deep in the parallel parquet writer it often is not returned to the user and instead we get a generic message "Unable to send array to writer!"

          I think we need to update several map_err statements to propagate inner error messages rather than ignore them. E.g. 

col_array_channels[next_channel]
.send(c)
.await
.map_err(|_| {
DataFusionError::Internal("Unable to send array to writer!".into())

change to something like

            col_array_channels[next_channel]
                .send(c)
                .await
                .map_err(|e| internal_datafusion_err!("Unable to send array to writer due to error {e}"))

Originally posted by @devinjdangelo in #11345 (comment)

@alamb alamb changed the title I think we need to update several map_err statements to propagate inner error messages rather than ignore them. E.g. Improve error messages for parallel parquet writer "Unable to send array to writer!" Jul 10, 2024
@alamb alamb added the enhancement New feature or request label Jul 10, 2024
@wiedld
Copy link
Contributor

wiedld commented Jul 10, 2024

take

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants