Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: typeset invalid dates errors #1678

Merged
merged 7 commits into from
Dec 4, 2024

Conversation

alexbarros
Copy link
Contributor

@alexbarros alexbarros commented Dec 3, 2024

Fixes conversion errors for DateTime types, when receiving unsupported dates (e.g. '0001-01-01').
With this fix the the invalid dates will be ignored when calculating the metrics that depend on the datetime types, while others (e.g. distinct values) will consider the date without any transformation.

It will also display the amount of invalid values identified
image

@alexbarros alexbarros changed the title Fix/typeset invalid dates errors fix: typeset invalid dates errors Dec 3, 2024
@alexbarros alexbarros requested a review from fabclmnt December 3, 2024 14:18
@alexbarros alexbarros marked this pull request as ready for review December 3, 2024 14:19
@alexbarros alexbarros force-pushed the fix/typeset_invalid_dates_errors branch from 3313919 to 11cd49b Compare December 3, 2024 17:38
Copy link

codecov bot commented Dec 3, 2024

Codecov Report

Attention: Patch coverage is 95.00000% with 1 line in your changes missing coverage. Please review.

Project coverage is 90.27%. Comparing base (9195531) to head (340b47e).
Report is 6 commits behind head on develop.

Files with missing lines Patch % Lines
...ata_profiling/model/pandas/describe_date_pandas.py 90.90% 1 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff             @@
##           develop    #1678      +/-   ##
===========================================
+ Coverage    90.25%   90.27%   +0.01%     
===========================================
  Files          198      198              
  Lines         6477     6496      +19     
===========================================
+ Hits          5846     5864      +18     
- Misses         631      632       +1     
Flag Coverage Δ
py3.8-ubuntu-22.04-pandas 90.27% <95.00%> (+0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@alexbarros alexbarros merged commit 1e8cb89 into develop Dec 4, 2024
14 of 16 checks passed
@alexbarros alexbarros deleted the fix/typeset_invalid_dates_errors branch December 4, 2024 12:20
portellaa pushed a commit that referenced this pull request Dec 8, 2024
* fix: ignore invalid dates during conversion

* fix: apply type conversion to user defined types

* test: add unit test to invalid date type convertion

* fix: add invalid dates to variable info

* fix(linting): code formatting

* test: update unit tests

* fix: rename to_datetime method
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants