Error error parsing created field: parsing time "1714686445000000000" as "2006-01-02T15:04:05Z07:00": cannot parse "686445000000000" as "-" #2027
Comments
|
@prologic would you perhaps share some sample code and/or more details here that could aid us in reproducing the situation. @CascadingRadium would you look into this please. |
|
All the code is here: https://git.mills.io/yarnsocial/search Unfortunately I found this out by accident, so I haven't had much time to dig into it to build a reproducer. I noticed this in a local dev environment, and just (wrongly) assumed my local index db was corrupt, blew it away and thought nothing of it. But this is happening in my production app at https://search.twtxt.net/ Let me know what I can do to help debug this 🙏 I can probably shove the index somewhere for you? |
|
Meanwhile I'm going to take a copy of the production index and see if the error reproos locally to debug. |
|
hi i have repro'd the issue, will put out the fix soon |
|
fix #2030 |
Addresses #2027 In bleve v2.3.10, [a bug fix](#1868) was introduced where the value of a datetime field in the Fields part of a search hit was made to match the value actually present in the document. - For example if a document A was indexed and had value of a datetime field X to be "2001/08/20 03:00:10", and ifA was returned as part of the search result, A's Fields will have a map which would look like - Before Fix - "X" : "2001-08-20T03:00:10Z" - After fix - "X" : "2001/08/20 03:00:10" This would come to play in case of custom user defined date time parser, and we did this by storing the layout with which the time string was parsed with during indexing. With timestamp support being added at the same time, they did not really have a valid layout to store and hence stored nil. During queries, the layout extracted, if null, was always assumed to be for the timestamp case _but_ the case where an existing index from pre v2.3.10 (which would also return null layout) was not handled and it became a catchall case leading to the datetime string being always returned in a timestamp rather than the expected default RFC3339.
|
Oh wow that was fast! Sorry guys I had to go to bed, I couldn't wait to copy the production DB I have to repro locally which is 2.1G Tar uncompressed 😅 Thanks so much for the fast fix! I assume we'll release/tag a new patch/minor release? |
|
Thank you! This bug appears to be fixed! Example: https://search.twtxt.net/search?q=conv:%23f57rmoq&s=created&s=_id I upgraded to |
|
hi @prologic bleve version v2.4.1 has been released with the fix. please use |
|
I hate to report, but this has happened again. Reproducible currently in prod: https://search.twtxt.net/search?q=Hello+World&f= Notably not all queries are failing. Also of note, no update to the bleve dependency this time. According to Git blame I'm using |
|
@prologic giving us a deterministic test case would be greatly helpful. @CascadingRadium would you take a look into the code to see if if we can spot any other path that could cause this. |
|
Let me know how I can help provide a reproducible test! I can even provide the data files if that helps as there's nothing sensitive in them. It's all publicly accessible anyway. |
|
hi @prologic
this suggests that somehow the code has fallen back to an old version <2.4.1 could you please confirm that all the dependent libraries of your search service are using atleast v2.4.1 |
|
I'm reasonably confident that the only commit that updated bleve from v2.4.0 to v2.4.1 was: https://git.mills.io/yarnsocial/search/commit/3031765b98dd9a6a85affceb72f6d3707704ac96 no changes in bleve dependencies since. |
|
At this point would you recommend I rebuild the index? (I can thankfully) |
|
yea if that is ok |
|
Will do! I'll close this issue. Many thanks! |
Not actually sure where this error is coming from. So asking... Was there a recent change in the way Bleve handles and parses Time fields? I have an existing index that I'd rather not have to nuke and start over 😅
The text was updated successfully, but these errors were encountered: