Redux: Issue 815 - Failed to sort by timestamp

[Blatwurst]

My question extends from #815. I added comments to the bottom of that issue, but since it's closed, I'm going to reiterate here...

This question involves sorting time fields, but it seems it is really a general string sort question. I'm using a recent Kibana/ElasticSearh/logstash stack. Sorting my records in Kibana works great when sorting by @timestamp, but that field is very unreadable. So I have a second 'time' field whose values look like this: 14/05/08-13:41:23 (I'll add millis once I get this working). I assume this is just a string field since I've done nothing to indicate otherwise.

Why doesn't this field sort my records properly by date/time even if it's just a string field? It seems to me that the field should sort properly by normal string comparison. Why does it not? The same seems to be true of the example originally posed in issue 815, where the values are very similar to mine. Are string fields not generally sortable without some additional setup?

TIA for any help.

[rashidkpc]

Multi-token string fields are not sortable by Elasticsearch in any predictable way. You'll need to set that field as not_analyzed if you need to sort on it.

[pwaller]

@rashidkpc I'm using @timestamp. Here is the sort by @timestamp in both directions. In both cases, it shows ascending in time, but it starts at a different place. The only thing I changed between these two images is to click the timestamp sort order icon.

/cc @drj11

[drj11]

Well, at least @pwaller's are sorted. When I try and sort my @timestamp (of just 6 items): I get this:

and

again, like @pwaller I just flipped the sort order using the clicky icon thing.

[Aldian-fr]

@pwaller Don't you have several pages of records? In standard configuration there are 5 of them, so when reverting order it probably took records from page 5.

@drj11 are you sure your timestamps are dates and not just strings?

@rashidkpc I don't really understand why an ES call is even needed since the data is already loaded. Couldn't Kibana just perform a pure javascript sort?

[pwaller]

@Aldian-fr uh, in both cases it says:

The text doesn't change.

[drj11]

@Aldian-fr I'm pretty sure they're strings. How should dates be represented? It's all JSON, right? So they pretty much have to be strings. All of these records get their @timestamp field from logstash (which I hope is not surprising).

Are you saying that it's okay for a field that is an ISO 8601 formatted string to not sort correctly?

[Aldian-fr]

@pwaller

But did you use the arrow to navigate to the bottom of page 5? There you should look to compare

@drj11 look at the original question #815 Your dates must me JSON dates and not just strings. Dates exist as a javascript object, it is much more powerful than just strings

[drj11]

@Aldian-fr JSON does not have a representation of dates. JavaScript does, but JSON does not. I'm gonna look at #815.

[pwaller]

I don't understand. Is the button I clicked not a sort button?

[Aldian-fr]

when you make a research, the histogram tells you that there are 51980 results, then kibana says it paged 500 of them and displays the 100 first. But when you sort, you also sort on pages you don't see. Navigate to have the interface tell you

[pwaller]

@Aldian-fr look the numbers in my post - they are both ascending in time. I don't understand what you're saying - is this the intended behaviour? Then how do I use the sort?

Neither of the two states result in the user-intended effect: to show the most recent events first.

[Aldian-fr]

Seems like I misunderstood the problem. Sorry, english is not my mother language and that was not clear even now. But I feel like you would better create a separate topic with an appropriate title like "Kibana displays descending order but sort ascending" or such

[pwaller]

I've opened #1298

[rashidkpc]

This is fixed in Kibana 4