Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Timezone aware Timestamp.dayofyear does not report correct day #13303

Closed
tumalow-will opened this issue May 27, 2016 · 1 comment
Closed

Timezone aware Timestamp.dayofyear does not report correct day #13303

tumalow-will opened this issue May 27, 2016 · 1 comment
Labels
Bug Timezones Timezone data dtype
Milestone

Comments

@tumalow-will
Copy link

tumalow-will commented May 27, 2016

Code Sample, a copy-pastable example if possible

import pandas as PD

time_str = '2015-10-01 23:59:00-04:00'
pd_dto = PD.to_datetime(time_str, utc=True).tz_convert('US/Eastern')

actual_yday = pd_dto.date().timetuple().tm_yday
print 'pandas thinks', pd_dto, 'is the', pd_dto.dayofyear, 'yday'
print 'when really its the', actual_yday

Expected Output

I expect the dayofyear to agree with tm_yday, but they do not.

output of pd.show_versions()

## INSTALLED VERSIONS

commit: None
python: 2.7.10.final.0
python-bits: 64
OS: Windows
OS-release: 8
machine: AMD64
processor: Intel64 Family 6 Model 61 Stepping 4, GenuineIntel
byteorder: little
LC_ALL: None
LANG: None

pandas: 0.16.2
nose: 1.3.7
Cython: 0.22.1
numpy: 1.9.2
scipy: 0.15.1
statsmodels: 0.6.1
IPython: 3.2.0
sphinx: 1.3.1
patsy: 0.3.0
dateutil: 2.4.2
pytz: 2015.4
bottleneck: 1.0.0
tables: 3.2.0
numexpr: 2.4.3
matplotlib: 1.4.3
openpyxl: 1.8.5
xlrd: 0.9.3
xlwt: 1.0.0
xlsxwriter: 0.7.3
lxml: 3.4.4
bs4: 4.3.2
html5lib: None
httplib2: None
apiclient: None
sqlalchemy: 1.0.5
pymysql: None
psycopg2: None

@jreback
Copy link
Contributor

jreback commented May 27, 2016

So this is working properly for DatetimeIndexes; which converts to 'local timestamps' before computing the field accessors. However for Timestamps it is NOT converting and doing this in UTC (which is what things are stored).

Pretty straightforward to fix if you'd like to do a PR. Relevant code is panda/tslib.pyx/Timestamp._get_field

In [37]: pd.to_datetime(['2015-1-01 23:59:00']).tz_localize('US/Eastern')
Out[37]: DatetimeIndex(['2015-01-01 23:59:00-05:00'], dtype='datetime64[ns, US/Eastern]', freq=None)

In [38]: pd.to_datetime(['2015-1-01 23:59:00']).tz_localize('US/Eastern').dayofyear
Out[38]: array([1], dtype=int32)

In [40]: pd.to_datetime(['2015-1-01 23:59:00']).tz_localize('US/Eastern').tz_convert('UTC')
Out[40]: DatetimeIndex(['2015-01-02 04:59:00+00:00'], dtype='datetime64[ns, UTC]', freq=None)

In [41]: pd.to_datetime(['2015-1-01 23:59:00']).tz_localize('US/Eastern').tz_convert('UTC').dayofyear
Out[41]: array([2], dtype=int32)

But not working for Timestamps

In [43]: ts
Out[43]: Timestamp('2015-01-01 23:59:00-0500', tz='US/Eastern')

In [44]: ts.dayofyear
Out[44]: 2

@jreback jreback added this to the Next Major Release milestone May 27, 2016
mroeschke added a commit to mroeschke/pandas that referenced this issue Mar 20, 2017
@jreback jreback modified the milestones: 0.20.0, Next Major Release Mar 20, 2017
mroeschke added a commit to mroeschke/pandas that referenced this issue Mar 20, 2017
…dev#13303)

Add whatsnew

Add tests for datetimeindex and fix Timestamp start/end attributes
AnkurDedania pushed a commit to AnkurDedania/pandas that referenced this issue Mar 21, 2017
…dev#13303)

closes pandas-dev#13303

Previously, calling a date/time attribute with Timestamp that's tz
aware (e.g. `Timestamp('...', tz='...').dayofyear`) would return the
attribute in UTC instead of the local tz.

Author: Matt Roeschke <[email protected]>

Closes pandas-dev#15740 from mroeschke/fix_13303 and squashes the following commits:

b78b333 [Matt Roeschke] BUG: tz aware Timestamp field accessors returns local values (pandas-dev#13303)
mattip pushed a commit to mattip/pandas that referenced this issue Apr 3, 2017
…dev#13303)

closes pandas-dev#13303

Previously, calling a date/time attribute with Timestamp that's tz
aware (e.g. `Timestamp('...', tz='...').dayofyear`) would return the
attribute in UTC instead of the local tz.

Author: Matt Roeschke <[email protected]>

Closes pandas-dev#15740 from mroeschke/fix_13303 and squashes the following commits:

b78b333 [Matt Roeschke] BUG: tz aware Timestamp field accessors returns local values (pandas-dev#13303)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug Timezones Timezone data dtype
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants