Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature add weekday temporal kernel #1891

Merged
merged 5 commits into from
Jun 17, 2022

Conversation

nl5887
Copy link
Contributor

@nl5887 nl5887 commented Jun 16, 2022

This feature will add the temporal kernel for weekday (day of week). When added, DOW support can be add to datafusion.

@nl5887 nl5887 force-pushed the feature-add-weekday-temporal-kernel branch from d1a5f47 to b510a7d Compare June 16, 2022 18:12
@github-actions github-actions bot added the arrow Changes to the arrow crate label Jun 16, 2022
@codecov-commenter
Copy link

codecov-commenter commented Jun 16, 2022

Codecov Report

Merging #1891 (a4e3e47) into master (8d787d9) will decrease coverage by 0.00%.
The diff coverage is 93.75%.

@@            Coverage Diff             @@
##           master    #1891      +/-   ##
==========================================
- Coverage   83.48%   83.48%   -0.01%     
==========================================
  Files         201      203       +2     
  Lines       56838    57156     +318     
==========================================
+ Hits        47452    47715     +263     
- Misses       9386     9441      +55     
Impacted Files Coverage Δ
arrow/src/compute/kernels/temporal.rs 94.64% <93.75%> (-2.49%) ⬇️
arrow/src/buffer/mutable.rs 89.13% <0.00%> (-1.41%) ⬇️
arrow/src/datatypes/datatype.rs 65.42% <0.00%> (-0.38%) ⬇️
arrow/src/compute/kernels/filter.rs 88.05% <0.00%> (-0.28%) ⬇️
parquet_derive/src/parquet_field.rs 65.75% <0.00%> (-0.23%) ⬇️
arrow/src/array/array_primitive.rs 94.97% <0.00%> (-0.09%) ⬇️
arrow/src/ffi.rs 86.97% <0.00%> (ø)
arrow/src/pyarrow.rs 0.00% <0.00%> (ø)
arrow/src/array/ffi.rs 100.00% <0.00%> (ø)
... and 40 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 8d787d9...a4e3e47. Read the comment docs.

Copy link
Contributor

@tustvold tustvold left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we get a test or two of this functionality please

Copy link
Contributor

@alamb alamb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good -- thank you for the contribution @nl5887 !

vec![Some(1514764800000), None, Some(1550636625000)].into();

let b = weekday(&a).unwrap();
assert_eq!(1, b.value(1));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Screen Shot 2022-06-16 at 4 08 52 PM

Screen Shot 2022-06-16 at 4 09 47 PM
👍

@alamb
Copy link
Contributor

alamb commented Jun 16, 2022

Looks like at least one test need to be updated:

https://github.com/apache/arrow-rs/runs/6924858466?check_suite_focus=true

Screen Shot 2022-06-16 at 4 11 00 PM

@nl5887
Copy link
Contributor Author

nl5887 commented Jun 17, 2022

Looks like at least one test need to be updated:

https://github.com/apache/arrow-rs/runs/6924858466?check_suite_focus=true

Screen Shot 2022-06-16 at 4 11 00 PM

Thanks, updated the test. Any opinions if we should use the ISO 8601 (similar to chrono-rs) week days (starting at Monday) or the US (starting at Sunday)? Currently it is using ISO (https://docs.rs/chrono/latest/src/chrono/lib.rs.html#646).

@alamb
Copy link
Contributor

alamb commented Jun 17, 2022

Thanks, updated the test. Any opinions if we should use the ISO 8601 (similar to chrono-rs) week days (starting at Monday) or the US (starting at Sunday)? Currently it is using ISO (https://docs.rs/chrono/latest/src/chrono/lib.rs.html#646).

I don't have any particular opinion -- it is likely that whatever we pick will be wrong for some usecases. The best long term is likely to allow the user to specify what behavior they want but I think we can do that in a follow on PR if anyone needs it

@@ -211,6 +211,34 @@ where
Ok(b.finish())
}

/// Extracts the day of week of a given temporal array as an array of integers
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
/// Extracts the day of week of a given temporal array as an array of integers
/// Extracts the day of week of a given temporal array as an array of integers
/// Monday is 0, Tuesday is 1, etc.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

proposed in #1894

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
arrow Changes to the arrow crate
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants