Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add more check logic when processing search request #3870

Merged
merged 3 commits into from
Sep 25, 2023

Conversation

JerryKwan
Copy link
Collaborator

@JerryKwan JerryKwan commented Sep 23, 2023

return error when search with start_timestamp or end_timestamp but timestamp field is not defined in index

Description

Check if timestamp field is not defined but start_timestamp or end_timestamp is set when searching

How was this PR tested?

1, Create index without timestamp field defined


doc_mapping:
  field_mappings:
    - name: title
      type: text
      tokenizer: default
      record: position
      stored: true
    - name: body
      type: text
      tokenizer: default
      record: position
      stored: true
    - name: creationDate
      type: datetime
      fast: true
      input_formats:
        - rfc3339
      precision: seconds
  timestamp_field: creationDate

search_settings:
  default_search_fields: [title, body]

indexing_settings:
  commit_timeout_secs: 10

2, Injest data

./quickwit index ingest --index stackoverflow --input-pathstackoverflow.posts.transformed-10000.json --force

3, Search without start_timestamp specified

./quickwit index search --index stackoverflow --query "search AND engine" --max-hits 1

Got result successfully

{
  "num_hits": 10,
  "hits": [
    {
      "answerId": 16049,
      "body": "Not sure if this is what you are after but you can search source code on Google. Follow the link for a search on 'function:reduce() lang:python' on Google Code search At first glance the following projects use reduce() MoinMoin Zope Numeric ScientificPython etc. etc. but then these are hardly surprising since they are huge projects. The functionality of reduce can be done using function recursion which I guess Guido thought was more explicit. Update: Since Google's Code Search was discontinued on 15-Jan-2012, besides reverting to regular Google searches, there's something called Code Snippets Collection that looks promising. A number of other resources are mentioned in answers this (closed) question Replacement for Google Code Search?. Update 2 (29-May-2017): A good source for Python examples (in open-source code) is the Nullege search engine. ",
      "creationDate": "2008-08-19T12:16:01Z",
      "questionId": 15995,
      "type": "answer",
      "user": "199"
    }
  ],
  "elapsed_time_micros": 106447,
  "errors": []
}

4, Search with start_timestamp

./quickwit index search --index stackoverflow --query "search AND engine" --max-hits 1 --start-timestamp 1218912000

Got error as expected

Command failed: API error: (code=400 Bad Request, message=the timestamp field is not set in index: ["stackoverflow"] definition but start-timestamp or end-timestamp are set in the query)

Caused by:
    (code=400 Bad Request, message=the timestamp field is not set in index: ["stackoverflow"] definition but start-timestamp or end-timestamp are set in the query)

Closes #3167

return error when search with start_timestamp or end_timestamp but
timestamp field is not defined in index
@JerryKwan JerryKwan changed the title add more check logc when processing search request add more check logic when processing search request Sep 23, 2023
@PSeitz
Copy link
Contributor

PSeitz commented Sep 25, 2023

LGTM. Can you add a unit test in quickwit-search/src/tests.rs

@JerryKwan
Copy link
Collaborator Author

@PSeitz
Thanks for reviewing. I will add the test asap

add test case for index without timestamp defined but query with
start_timestamp or end_timestamp
@JerryKwan
Copy link
Collaborator Author

@PSeitz
Test case added, please help to review again, thank you

Copy link
Contributor

@PSeitz PSeitz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice, thanks!

@PSeitz PSeitz enabled auto-merge (squash) September 25, 2023 07:22
@PSeitz PSeitz merged commit 5f8bb5d into quickwit-oss:main Sep 25, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

start_timestamp, end_timestamp parameters should return an error without timestamp field defined
2 participants