Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MB-58110: Format dates in search result correctly #1868

Merged
merged 5 commits into from
Aug 25, 2023
Merged

Conversation

CascadingRadium
Copy link
Member

@CascadingRadium CascadingRadium commented Aug 14, 2023

Jira

MB-58110

Description

Currently, the datetime fields stored in the Hits.Fields section of the search result object is always formatted to RFC3339. Therefore, when the user applies a

  • custom date time parser to a field with format X, and stores that field
  • then searches the same field using a date range query, with request.Fields = "*"
  • the result struct always has the field content formatted as RFC3339, which breaks the UX as the user expects the same date to show up in the result that he has made available in the bucket.

Must merge blevesearch/bleve_index_api#37 first

document/field_datetime.go Outdated Show resolved Hide resolved
document/field_datetime.go Show resolved Hide resolved
@CascadingRadium CascadingRadium merged commit c287973 into master Aug 25, 2023
@CascadingRadium CascadingRadium deleted the MB-58110 branch August 25, 2023 19:14
abhinavdangeti added a commit that referenced this pull request Aug 25, 2023
* Save the layout in a custom date time parser, that successfully parses a date time string, as part of a stored field and then use the same layout to format the time object back to its original format upon calling the DateTime() function.

* Address code review comments

---------

Co-authored-by: Abhinav Dangeti <[email protected]>
CascadingRadium added a commit that referenced this pull request Aug 31, 2023
* Save the layout in a custom date time parser, that successfully parses a date time string, as part of a stored field and then use the same layout to format the time object back to its original format upon calling the DateTime() function. 

* Address code review comments

---------

Co-authored-by: Abhinav Dangeti <[email protected]>
CascadingRadium added a commit that referenced this pull request May 3, 2024
Addresses #2027

In bleve v2.3.10, [a bug
fix](#1868) was introduced
where the value of a datetime field in the Fields part of a search hit
was made to match the value actually present in the document.
- For example if a document A was indexed and had value of a datetime
field X to be "2001/08/20 03:00:10", and ifA was returned as part of the
search result, A's Fields will have a map which would look like
- Before Fix 
    - "X" : "2001-08-20T03:00:10Z"
- After fix 
    - "X" : "2001/08/20 03:00:10"

This would come to play in case of custom user defined date time parser,
and we did this by storing the layout with which the time string was
parsed with during indexing. With timestamp support being added at the
same time, they did not really have a valid layout to store and hence
stored nil. During queries, the layout extracted, if null, was always
assumed to be for the timestamp case _but_ the case where an existing
index from pre v2.3.10 (which would also return null layout) was not
handled and it became a catchall case leading to the datetime string
being always returned in a timestamp rather than the expected default
RFC3339.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants