Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

COMPAT: Floating point fractional hours for timedeltas gives unexpected precision issue #9048

Closed
Tracked by #7
ahjulstad opened this issue Dec 9, 2014 · 7 comments · Fixed by #53552
Closed
Tracked by #7
Labels
Bug Needs Tests Unit test(s) needed to prevent regressions Timedelta Timedelta data type

Comments

@ahjulstad
Copy link
Contributor

Creating a timedelta with fractional hours gives an unexpected large round-off error.

>>> to_timedelta(1.0/3, unit='h')
Timedelta('0 days 00:19:59.998800')

This is indicates a precision around 2x10^-7 ( 1/(1000_60_60) ), which is less than a single precision float.

@ahjulstad
Copy link
Contributor Author

Test-case code:

        # Tests with fractional hours as input:
        expected = np.array([0, 20*60*1000000000, 30*60*1000000000, 40*60*1000000000], dtype='timedelta64[ns]')
        result = to_timedelta([0., 1.0/3, 0.5, 2.0/3], unit='h', box=False)
        tm.assert_almost_equal(expected, result)

@jreback
Copy link
Contributor

jreback commented Dec 9, 2014

Here is where to change: https://github.com/pydata/pandas/blob/master/pandas/tslib.pyx#L2448

prob need a few tests for this though (and this also affects datetimes as well, e.g. you can use to_datetime in a similar manner). can prob change for h/m I suspect.

@jreback jreback changed the title Floating point fractional hours gives unexpected precision issue COMPAT: Floating point fractional hours for timedeltas gives unexpected precision issue Dec 9, 2014
@jreback jreback added Dtype Conversions Unexpected or buggy dtype conversions Numeric Operations Arithmetic, Comparison, and Logical operations Timedelta Timedelta data type Datetime Datetime data dtype labels Dec 9, 2014
@jreback jreback added this to the 0.16.0 milestone Dec 9, 2014
@ahjulstad
Copy link
Contributor Author

I have had a look at it, but for some reason am struggling with recompiling.
I am using anaconda, with a conda environment for pandas, and pandas installed with
pip install -e ., but python just hangs on import pandas.
Pointers to detailed instructions appreciated.

@jreback
Copy link
Contributor

jreback commented Dec 9, 2014

don't use pip.

This gets you all the deps
conda install pandas

This gets you the build deps
conda install cython libpython nose

To build
python setup.py build

@ahjulstad
Copy link
Contributor Author

It builds, but I cannot load the compiled modules within the git directory without some more magic. After build_ext –inplace, import pandas hangs (eating a lot of memory). I am on Windows using VS2010 compiler. I have limited time the next days to look at this, and I have a full reinstall coming up soon, so I won’t worry too much about it now.

@jreback
Copy link
Contributor

jreback commented Dec 10, 2014

you cannot use the VS 2010 compiler as python doesn't support it (except for >= 3.3.). use the recipe I show above. which works on all pythons and doesn't depend on your environment.

@jreback jreback modified the milestones: 0.16.0, Next Major Release Mar 6, 2015
@mroeschke mroeschke added Bug and removed Datetime Datetime data dtype Dtype Conversions Unexpected or buggy dtype conversions labels Mar 31, 2020
@mroeschke mroeschke removed this from the Contributions Welcome milestone Oct 13, 2022
@jbrockmendel jbrockmendel removed the Numeric Operations Arithmetic, Comparison, and Logical operations label Mar 28, 2023
@jbrockmendel
Copy link
Member

I see much higher precision:

In [11]: pd.to_timedelta(1.0/3, unit='h')
Out[11]: Timedelta('0 days 00:19:59.999999998')

The discussion above makes me think this is either long-sinced fixed of platform-specific. Could use a test.

@jbrockmendel jbrockmendel added the Needs Tests Unit test(s) needed to prevent regressions label Mar 28, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug Needs Tests Unit test(s) needed to prevent regressions Timedelta Timedelta data type
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants