-
Notifications
You must be signed in to change notification settings - Fork 1.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix CI check when version changes -- remove checked in file that is created by doc example #12034
Conversation
Since this file is produced by https://github.com/apache/datafusion/blob/main/docs/source/library-user-guide/using-the-dataframe-api.md#write-dataframe-to-files, which is runned by this doctest as cargo test --doc 'library_user_guide_dataframe_api' won't it be better to replace it with tempdir rather than adding this file to gitignore? E.g. use datafusion::prelude::*;
use datafusion::error::Result;
use datafusion::dataframe::DataFrameWriteOptions;
#[tokio::main]
async fn main() -> Result<()> {
// Replace with actual target path
let target_path = tempfile::tempdir()?.path().join("example.parquet");
let ctx = SessionContext::new();
// read example.csv file into a DataFrame
let df = ctx.read_csv("tests/data/example.csv", CsvReadOptions::new()).await?;
// stream the contents of the DataFrame to the target file
df.write_parquet(
target_path.to_string_lossy().as_ref(),
DataFrameWriteOptions::new(),
None, // writer_options
).await;
Ok(())
} or to delete the file in the end of the example, like it's done in other doctests, but using tempdir has zero (or close to it) risk that this file will be committed. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🚀
Thank you @alamb
Thank you for the review @korowa |
Which issue does this PR close?
Closes #11892
Rationale for this change
See #11892 -- basically if versions change the CI fails
The problem is that the output parquet file that is written as part of the doc example is checked in, and thus if the actual bytes written by the example change (e.g. when the version number changes) the local file is different too
What changes are included in this PR?
Add it to.gitignore
so it isn't added againAre these changes tested?
Yes by CI
Are there any user-facing changes?
No, this is a test fix only