Addition of support to plot time along the x-axis

[Aniket025]

I was wondering if there is something to support time in dd/mm/yy and hours:minutes: seconds on the x-axis vs some data on the y-axis.

[epezent]

Other people have asked for time-formatted x-axes, so I think it's probably a feature that could be useful and something we can start looking into. I see two paths forward:

1. Expose a public API that allows the user to define their on axes label formatting. I don't know how it would look just yet, but this would be the most flexible option.

2. Support time format directly (in addition to normal and scientific formats). If we go this route, then other questions arise:

• do we treat the user's float* xs input as seconds? what if their data is in minutes or hours?
• are the labels fixed to a certain format (e.g hh::mm::ss.ms) or dynamic based on the range (e.g. switch from hour to min to sec as the user zooms in)

Either way, we will need a more sophisticated grid division/subdivision approach that only subdivides if labels do not overlap, or hides labels when they do overlap. This should be our first focus.

[Aniket025]

I think there can be possible solutions by going with the second option.
People who want to plot will mostly have it as a timestamp which can be converted to whatever the user wishes to have. Also, plotting every individual second on the x-axis won't be a very good option.
We can have a specific label as HH: MM initially and as the user zooms we can add HH:MM:: SS and then HH:MM::SS: MS to it.
I am sharing a snapshot of an application that does the same and we can make something similar along the x-axis.
Please see the images below :

Here they have in HH:MM
Then as we zoom

We can follow a similar approach which shall solve major issues for most people.

[Aniket025]

Also, we might keep the gridlines constant for a frame for better visibility and divide a given time into a specific number of slots and mark their labels to help make it dynamic.

[Prinkesh]

Hi @epezent . Hi have added support for time axis with formatting based on current data range .
Max granularity that one can see is of milliseconds .

Any suggestion on handling nanoseconds timestamps as currently in the api data is passed in float and all calculations are done in floating point . I end up losing precision for timestamps . I was thinking of templatizing the data type but that would be a major change across the library .

[epezent]

@Prinkesh , this looks awesome!! Thanks for taking the lead on this.

I agree, templatizing the data type might be nasty and would necessitate bringing a lot of stuff into the header. There's been some discussion on moving to double internally (#33 (comment)). We could then expose versions of the functions that take double*. Truth be told, it's probably the better choice for most people. floatis just what I gravitated towards when I first started the library.

I can look into this today, if you can put nanoseconds on hold. I'll report back when the change is made. In the meantime, I have some suggestions:

• I see the graph is stretched quite a bit. Was this because the labels were overlapping? We need better label overlapping detection and grid subdividing methods (I mentioned this above).
• The same formatting needs to be applied to the bottom-right cursor indicator.
• What should the base (left-most) granularity be? I think years is excessive and would be annoying. I would say that for most users, HH:MM:SS.MS would be sufficient, but might we need to consider other formats, with flags such as ImPlotAxisFlags_HHMMSSMS and ImPlotAxisFlags_YYYYMMDD?
• We want to use "nice numbers" for the grid divisions and labels. I think the last granularity should either be a multiple of 1,5, or 10 (and maybe 2?). So:

03:55, 02:45:21, 13:22:30 are ok
03.53, 02:45:22, 13:22:36 are not ok

[Prinkesh]

right now i detect the range of data and format the labels as follows

Data bucket Value	Format
num_months	"%Y:%m:%e"
num_hrs	"%e:%I"
num_seconds	"%I:%M"
num_millisecs	"%I:%M:%S.XXX"
num_microsecs	"%S.XXXXXX"
num_nanosecs	XXXXXXXXX

The table is in precedence order . Ex num_months > divisions than 1st row format is applied so on .
We can easily add to years

Further i am adding prefix on Left most part of graph (prefix format would be the upper precedence for the data i.e if data falls in microsec range than prefix will fall in millisecs range])

• I can look into this today, if you can put nanoseconds on hold. I'll report back when the change is made. In the meantime, I have some suggestions:

even double will loose precision . At max microseconds could be supported . else uint64_t is needed . Which requires a lot of change after introducing that type of y and x axis would be different

• I see the graph is stretched quite a bit. Was this because the labels were overlapping? We need better label overlapping detection and grid subdividing methods (I mentioned this above).

No such reason i was experimenting with nanosecond data thats why its stretched. But number of subdivisions should be less from my experimentation in small window size

• We want to use "nice numbers" for the grid divisions and labels. I think the last granularity should either be a multiple of 1,5, or 10 (and maybe 2?). So:

Agree with this . We can pick the first tick as a multiple of some number than rest can be calculated easily .

[epezent]

How likely is it that someone would want to have a plot is zoomable from years/months to nanoseconds? Seems like an extreme use case to me. What we might need to do, is have multiple time formatting options. For example:

ImPlotAxisFlags_YYYYMMDDHH ... data would be interpreted as years, formatted as 2020.05.15
ImPlotAxisFlags_HHMMSS ... data would be interpreted as hours, formatted as 08:24:30
ImPlotAxisFlags_SSMSUSNS ... data would be interpreted as seconds, formatted as as 55.123456789 or 55.123.456.789 (the latter is unconventional but a little easier on the eyes)

[Prinkesh]

How likely is it that someone would want to have a plot is zoomable from years/months to nanoseconds? Seems like an extreme use case to me. What we might need to do, is have multiple time formatting options. For example:

That is true .. This is for the purpose that user doesn't need to specify the data format . Rather they will just specify that x axis is time .

I think we can also solve nanosecond issue but having an time offset .. So if someone is viewing. nanosecond level data . We might give an offset of start time and other data would be relative to that . Only formatter will use the offset time to calculate the formatting value .

[epezent]

That is true .. This is for the purpose that user doesn't need to specify the data format . Rather they will just specify that x axis is time .

Even then, how is data interpreted? Always as seconds I presume?

I do see you point though. It would be nice if it just magically worked without setting different flags.

One thing I want to avoid is changing the plotting API function signature(s) or adding new signatures just for time formatted plots. I'm not sure if your offset approach would be possible then.

[Prinkesh]

Even then, how is data interpreted? Always as seconds I presume?

I was thinking of milliseconds as most use case would be for signal processing / latency analysis / mainly high frequency data . If someone wanted to do analysis on yearly /monthly / daily data / low frequency of data better choice would web based plotting libs .

One thing I want to avoid is changing the plotting API function signature(s) or adding new signatures just for time formatted plots. I'm not sure if your offset approach would be possible then.

We could add something like this probably

ImPlot::SetNextPlotBaseValue(value);

and maintain this base_value while calculating tick_markers ?

[Prinkesh]

Hi @epezent ,

Will you be adding support for double ? I am adding the support till microseconds for now . I was going through supporting double we will have to change ImVec2 as well or add another struct for double as points are stored in that struct .

[epezent]

Yes I’m working on it. And yes it will require a new vec2

[epezent]

I may not be able to finish this today. Perhaps tomorrow. Sorry for the delay.

[epezent]

@Prinkesh , you can give double precision a try on the double branch.

Also, can you please provide input on #36

[epezent]

@Prinkesh, any updates on your PR? If this isn't something you can continue working on, I can try myself. Let me know.

[Prinkesh]

Will update it by tomorrow .

…

On Fri, Jun 5, 2020, 10:47 PM Evan Pezent ***@***.***> wrote: @Prinkesh <https://github.com/Prinkesh>, any updates on your PR? If this isn't something you can continue working on, I can try myself. Let me know. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#34 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABLDNUQDQROB7GDSCK3IZ4TRVESBTANCNFSM4NAELKYQ> .

[Prinkesh]

Here are the snapshot from the implementation . Left most bottom text shows the initial time for the frame and rest of the label shows the part which changes .

[epezent]

This is awesome. Can't wait to try it.

[epezent]

@Prinkesh has a very good PR for this waiting in #55. Hopefully I will be able to merge it soon.

[epezent]

To give an update on this, @Prinkesh's PR was obliterated by the recent refactors. I've pieced it back together for the most part on the time branch. However, it's likely that I will do a total rewrite before this is finished. I'd like to achieve the quality of time formatting displayed in @leeoniya's uplot (see example). Can't say when I'll be done, but it's being worked on.

[hoffstadt]

Good to know this is in the works. Our users have been requesting this features in our wrapping.

[epezent]

I'm happy to share a long overdue update for this! It's been rather challenging to get just right, but I nearly have this working to the level of quality I hope to achieve. Check it out!

A few details:

• Times are interpreted as a UNIX timestamp in seconds (e.g. 1599243545)
• Currently only UTC is supported (no timezones of daylight savings time yet). Leap years are accounted for.
• Internally it uses double (instead of time_t), so subsecond precision is possible down to milliseconds (microseconds has some rounding issues, but I may get it working)
• The unit splits are: year, month, day, hour, minute, second, millisecond, (and possibly microsecond)
• I drew a lot of inspiration from @leeoniya's uPlot and copied his two-level labels best that I could (still need to render the second level label for the left most tick)
• Months and years are subdivided correctly so that each subdivision is on an exact day or month, respectively. This means that subdivisions are not always equally spaced.
• It only works for the x-axis and overrides logarithmic scaling. Currently it is enabled with an ImPlotAxisFlag but I'll probably just make it an ImPlotFlag so people don't try to put it on Y.
• I've only tested on Windows, so if someone can let me know how it works on Linux and macOS, that'd be great. I'm sure there will be some silly time.h differences to take care of.

I still need to do more testing, bug fixing, and code cleanup. But if you are interested, you can check out my progress on the time-axes branch.

Also, thank you @Prinkesh for your initial attempt. I used bits and pieces of your code in my implementation.

[hoffstadt]

Awesome! I will be checking this out later tonight or tomorrow. I have Linux and MacOS machines setup already so I will let you know how that goes. Great work, man!

[leeoniya]

I drew a lot of inspiration from @leeoniya's uPlot and copied his two-level labels best that I could (still need to render the second level label for the left most tick)

the one rough spot in uPlot's config format & impl - and this is a balance between pragmatism and perfection - is that it only tracks 2 levels of context, which means that when you're looking at hour or minute-level resolutions, each rollover renders the full context (year, month, day) even when they remain unchanged, which is often more than necessary: leeoniya/uPlot#284 (comment)

[epezent]

Yea, I noticed that. I couldn't figure out if it was intentional or not, but I didn't mind it either way. I sort of have a similar issue right now where redundant bottom level information is displayed, e.g. 6/16 and 9/19 9:22am here:

[leeoniya]

yep, it's not intentional, per se.

uPlot's config array [1] wouldn't be that difficult to expand to track every level of context, but it would get quite large without some mechanism/abstraction for defining similar behavior for swaths of contexts. i haven't thought of what that may look like and still remain cognitively grokkable.

[1] https://github.com/leeoniya/uPlot/blob/9288f09582610e22b4538f13f3ec968e07456117/src/opts.js#L119-L142

[epezent]

@leeoniya's uPlot OHLC demo recreated in full now with time formatted axes. The only difference seems to be that uPlot can skip Saturday/Sunday. I'm not too worried about implementing that right now.

Still a few things to add and tidy up, but I'm very happy with where it's at so far.

[leeoniya]

nice :)

you should draw the hover bar below behind the markers so it does not affect their color and dilute the good contrast.

it looks like you're doing "major" ticks at month boundaries? which is a bit different than what uPlot does (ordinal/even spacing). this causes your version to have denser labels near the major ticks. (e.g. 3/29 and 4/1). it's probably the right trade-off in most cases. uPlot has no concept of major/minor ticks currently.

btw, i think i have context tracking config array figured out. still finishing it up but it'll land soon if it works out.

[epezent]

That's correct. My implementation sort of works like this:

1. figure out what minor unit should be based on pixel width and plot range
2. get major unit (i.e. next largest unit up from minor unit)
3. floor the plot minimum to the nearest major unit
4. add major ticks in intervals of 1 major unit up until plot maximum, culling those outside the plot range
5. for each major-to-major section, add minor ticks in intervals of n minor units where n is determined from a table of allowable step sizes.

My first naive approach simply subdivided major sections n times, evenly, to get the minor ticks, but what ended up happening is that I'd have a label that wasn't exactly what it said it should be, e.g. "3/29" might not be exactly midnight.

Decided I'd rather have the labels be exact than evenly spaced. For days/months, it's noticeable like you say toward the end of the month when it's on a 7 or 14 day step size. I put in a little logic to hide the last day label if it's going to overlap the next month label. You can see this with 3/30 and 3/31 above.

[leeoniya]

success in leeoniya/uPlot@66d3e13

in https://leeoniya.github.io/uPlot/bench/uPlot.html, spanning the 12am periods while zooming now shows the difference in initial and follow-up label contexts. could still use some tweaking of the defaults but appears to work as intended and is pretty simple.

[marcizhu]

@epezent I can confirm that it works flawlessly on macOS. Good job! :D

[epezent]

This work is complete, merged, and detailed here: #48 (comment)

Woohoo!

[jhgorse]

Folks looking for ImPlotAxisFlags_Time are now looking for ImPlotScale_Time. Documentation still refers to the obsolesced name.

[kylevernyi]

Does anyone know how to set the X axis to interpret the UNIX timestamp as milliseconds instead of seconds? I see from above that it should support milliseconds but none of the examples use it as far as I can tell.