Fixed the ef_search default value for faiss HNSW with filters and updated the perf-tool to include Faiss HNSW tests #926

navneet1v · 2023-06-04T06:53:02Z

Description

Fixed the ef_search default value for faiss HNSW with filters and updated the perf-tool to include Faiss HNSW tests

Issues Resolved

Check List

New functionality includes testing.
- All tests pass
New functionality has been documented.
- New functionality has javadoc added
Commits are signed as per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

codecov · 2023-06-04T07:10:33Z

Codecov Report

Merging #926 (6fe46e9) into feature/faiss-filtering (119b8d6) will not change coverage.
The diff coverage is n/a.

❗ Current head 6fe46e9 differs from pull request most recent head 5bb4a0f. Consider uploading reports for the commit 5bb4a0f to get more accurate results

@@                    Coverage Diff                     @@
##             feature/faiss-filtering     #926   +/-   ##
==========================================================
  Coverage                      83.92%   83.92%           
  Complexity                      1092     1092           
==========================================================
  Files                            152      152           
  Lines                           4492     4492           
  Branches                         400      400           
==========================================================
  Hits                            3770     3770           
  Misses                           538      538           
  Partials                         184      184

junqiu-lei · 2023-06-05T21:37:50Z

benchmarks/perf-tool/release-configs/faiss-hnsw/test.yml

+    index_name: target_index
+  - name: create_index
+    index_name: target_index
+    index_spec: /home/ec2-user/[PATH]/index.json


Shall we use /home/ec2-user by a root path parameter instead of hardcoded?

It will require changes in the core perf tool. We can take this as a backlog. It was already like this on the test.yml so I am using it as it is.

martin-gaievski · 2023-06-05T21:55:39Z

benchmarks/perf-tool/okpt/io/config/parsers/test.py

@@ -23,6 +23,7 @@ class TestConfig:
    test_name: str
    test_id: str
    endpoint: str
+    port: int


If port is configurable we need to update description of test params in benchmarks/perf-tool/README

naveentatikonda · 2023-06-05T21:49:30Z

...hmarks/perf-tool/release-configs/faiss-hnsw/filtering/relaxed-filter/relaxed-filter-test.yml

@@ -0,0 +1,35 @@
+endpoint: "navneev.aka.corp.amazon.com"


Do you want to change this to endpoint: [ENDPOINT]

naveentatikonda · 2023-06-05T21:49:58Z

...hmarks/perf-tool/release-configs/faiss-hnsw/filtering/relaxed-filter/relaxed-filter-test.yml

@@ -0,0 +1,35 @@
+endpoint: "navneev.aka.corp.amazon.com"
+port: 9900


Also, the port

port I will remove

If we are using ELB endpoint and if I send the port(80) with endpoint, it doesn't work right ? We need to set it separately like you added before

Yes, we need to set separately.

So, shall we add the placeholder to set port in all .yml files ?

It's fine. not sure how many times we use ELB. Plus I have added things in documentation so we should be good.

naveentatikonda · 2023-06-05T21:51:57Z

...erf-tool/release-configs/faiss-hnsw/filtering/restrictive-filter/restrictive-filter-test.yml

@@ -0,0 +1,37 @@
+endpoint: [ENDPOINT]
+test_name: "Faiss HNSW Restrictive Filter Test"


We need to add port: here and in other .yml files as well. Else, it will pick default port as 80 instead of 9200 for localhost and will fail

making default value as 9200

…ated the perf-tool to include Faiss HNSW tests Signed-off-by: Navneet Verma <[email protected]>

…ated the perf-tool to include Faiss HNSW tests (opensearch-project#926) Signed-off-by: Navneet Verma <[email protected]>

…es include * Enabled the efficient filtering support for Faiss Engine (opensearch-project#907) * Fixed the ef_search default value for faiss HNSW with filters and updated the perf-tool to include Faiss HNSW tests (opensearch-project#926) * Added exact search for cases when filteredIds < k to improve the recall for exact search (opensearch-project#928) * Improved Exact Search to return only K results and added client side latency metric for query Benchmarks (opensearch-project#933) * Added Integration Tests and Unit test for Efficient Filtering for Faiss Engine (opensearch-project#934) Signed-off-by: Navneet Verma <[email protected]>

…es include (#936) * Enabled the efficient filtering support for Faiss Engine (#907) * Fixed the ef_search default value for faiss HNSW with filters and updated the perf-tool to include Faiss HNSW tests (#926) * Added exact search for cases when filteredIds < k to improve the recall for exact search (#928) * Improved Exact Search to return only K results and added client side latency metric for query Benchmarks (#933) * Added Integration Tests and Unit test for Efficient Filtering for Faiss Engine (#934) Signed-off-by: Navneet Verma <[email protected]>

…es include (opensearch-project#936) * Enabled the efficient filtering support for Faiss Engine (opensearch-project#907) * Fixed the ef_search default value for faiss HNSW with filters and updated the perf-tool to include Faiss HNSW tests (opensearch-project#926) * Added exact search for cases when filteredIds < k to improve the recall for exact search (opensearch-project#928) * Improved Exact Search to return only K results and added client side latency metric for query Benchmarks (opensearch-project#933) * Added Integration Tests and Unit test for Efficient Filtering for Faiss Engine (opensearch-project#934) Signed-off-by: Navneet Verma <[email protected]>

…es include (#936) * Enabled the efficient filtering support for Faiss Engine (#907) * Fixed the ef_search default value for faiss HNSW with filters and updated the perf-tool to include Faiss HNSW tests (#926) * Added exact search for cases when filteredIds < k to improve the recall for exact search (#928) * Improved Exact Search to return only K results and added client side latency metric for query Benchmarks (#933) * Added Integration Tests and Unit test for Efficient Filtering for Faiss Engine (#934) Signed-off-by: Navneet Verma <[email protected]>

navneet1v requested review from heemin32, VijayanB, vamshin, jmazanec15, naveentatikonda, junqiu-lei and martin-gaievski as code owners June 4, 2023 06:53

navneet1v added the skip-changelog label Jun 5, 2023

navneet1v force-pushed the feature/faiss-filtering branch from 6fe46e9 to e53070f Compare June 5, 2023 19:17

junqiu-lei reviewed Jun 5, 2023

View reviewed changes

junqiu-lei approved these changes Jun 5, 2023

View reviewed changes

martin-gaievski reviewed Jun 5, 2023

View reviewed changes

naveentatikonda reviewed Jun 5, 2023

View reviewed changes

Fixed the ef_search default value for faiss HNSW with filters and upd…

5bb4a0f

…ated the perf-tool to include Faiss HNSW tests Signed-off-by: Navneet Verma <[email protected]>

navneet1v force-pushed the feature/faiss-filtering branch from e53070f to 5bb4a0f Compare June 5, 2023 22:38

martin-gaievski approved these changes Jun 5, 2023

View reviewed changes

navneet1v merged commit 1135d79 into opensearch-project:feature/faiss-filtering Jun 5, 2023

navneet1v mentioned this pull request Jun 14, 2023

Added support for Efficient Pre-filtering for Faiss Engine #936

Merged

5 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fixed the ef_search default value for faiss HNSW with filters and updated the perf-tool to include Faiss HNSW tests #926

Fixed the ef_search default value for faiss HNSW with filters and updated the perf-tool to include Faiss HNSW tests #926

navneet1v commented Jun 4, 2023

codecov bot commented Jun 4, 2023 •

edited

Loading

junqiu-lei Jun 5, 2023

navneet1v Jun 5, 2023

martin-gaievski Jun 5, 2023

navneet1v Jun 5, 2023

naveentatikonda Jun 5, 2023

navneet1v Jun 5, 2023

naveentatikonda Jun 5, 2023

navneet1v Jun 5, 2023

naveentatikonda Jun 5, 2023

navneet1v Jun 5, 2023

naveentatikonda Jun 5, 2023

navneet1v Jun 5, 2023

naveentatikonda Jun 5, 2023

navneet1v Jun 5, 2023

		@@ -0,0 +1,35 @@
		endpoint: "navneev.aka.corp.amazon.com"
		port: 9900

		@@ -0,0 +1,37 @@
		endpoint: [ENDPOINT]
		test_name: "Faiss HNSW Restrictive Filter Test"

Fixed the ef_search default value for faiss HNSW with filters and updated the perf-tool to include Faiss HNSW tests #926

Fixed the ef_search default value for faiss HNSW with filters and updated the perf-tool to include Faiss HNSW tests #926

Conversation

navneet1v commented Jun 4, 2023

Description

Issues Resolved

Check List

codecov bot commented Jun 4, 2023 • edited Loading

Codecov Report

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

codecov bot commented Jun 4, 2023 •

edited

Loading