Precision support for Timestamp

[wavexx]

Timestamp stores the value in a datetime64[ns] internal representation. For most purposes this is fine, but I'd love if I could choose the internal precision, for example to use datetime64[s] instead.

When I do timeseries manipulations that span more than a day, I generally do not need sub-second resolution and I could definitely use the extended range instead. But, more conveniently, this would also allow to pretty-print the result without a ton of trailing zeros without setting some arbitrary threshold.

Incidentally, this would remove the limit of ~500 years, but it's more of an ergonomic change than an actual need. For example, when I need to perform direct manipulations of the values in a DatetimeIndex, I often have to scale up to seconds if I want to perform meaningful fitting and avoid numerical instability. When plotting, it's an extra indirection I need to care about.

[TomAugspurger]

From what I recall, Timestamps always being nanosecond resolution was an intentional decision that simplified the implementation. We provide Period objects (see here) for representing times outside the 500 years that DatetimeIndex can represent.

When plotting, it's an extra indirection I need to care about.

Is it bar plots that cause you issues? We already nicely format line plots, and I want to rip out that code and pass it up to matplotlib. In the process I hope to fix barplot formatting as well.

[jreback]

there are a bunch of issues w.r.t. support for non-ns M8 resolutions, e.g. #7307, #6741

its a bit non-trivial, but on the roadmap, or course a pull-request would be the best!

[wavexx]

On Sat, Feb 27 2016, Tom Augspurger [email protected] wrote:

From what I recall, Timestamps always being nanosecond resolution was an
intentional decision that simplified the implementation. We provide Period
objects (see here) for representing times outside the 500 years that
DatetimeIndex can represent.

I read them about before and I see how they can be convenient, but
somehow they feel overkill for what should essentially be a fixed
precision multiplier.

It also doesn't map as nicely with numpy's datetime64[x] types, which
pop up more frequently. But maybe Periods could be improved to use them?

When plotting, it's an extra indirection I need to care about.

Is it bar plots that cause you issues? We already nicely format line plots, and
I want to rip out that code and pass it up to matplotlib. In the process I hope
to fix barplot formatting as well.

No, I didn't stumble in plotting issues directly. What I mean is that I
often need to get the values directly out of the series, and in doing so
I generally have to transform from/back to seconds for various reasons.

Incidentally, when I work with experiments from microbiology, I always
use relative times (as in ms from experiment start). Because of this, I
never had much use of the subsecond resolution in timestamps so far.

[jreback]

@wavexx you know Timedeltas are designed exactly for that purpose.

[wavexx]

On Sat, Feb 27 2016, Jeff Reback [email protected] wrote:

@wavexx you know Timedeltas are designed exactly for that purpose.

I really don't feel the need for timedeltas when working in
microbiology. If you work with a fixed unit, you want to use that unit
through-out, including in the internal representation.

I had a few uses of timedeltas before, but it generally involved
timespans longer than a few minutes, where the added formatting made
sense.

[jreback]

duplicate of issues mentioned above.