Add reference to hybrid mmap extensions setting in k-NN #3262

kolchfa-aws · 2023-03-07T16:31:40Z

Checklist

[x ] By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license and subject to the Developers Certificate of Origin.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Signed-off-by: Fanit Kolchina <[email protected]>

kolchfa-aws · 2023-03-07T16:32:49Z

@martin-gaievski Could you please review this PR for technical accuracy? Also, which versions of the documentation does it make sense to backport this PR to? Thanks!

Signed-off-by: Fanit Kolchina <[email protected]>

martin-gaievski · 2023-03-07T17:28:02Z

_search-plugins/knn/performance-tuning.md

@@ -82,6 +82,12 @@ Take the following steps to improve search performance:

   If your use case is simply to read the IDs and scores of the nearest neighbors, you can disable reading stored fields, which saves time retrieving the vectors from stored fields.

+* **Use `mmap` file I/O**
+
+   For the Lucene-based approximate k-NN search, there is no dedicated cache layer that speeds up read/write operations. Instead, the plugin is relying on the existing caching mechanism in OpenSearch core. In versions 2.4 and below, by default, read/write operations were based on Java NIO, which can be slow depending on the Lucene version and number of segments per shard. Starting with version 2.5, k-NN enables [`mmap`](https://en.wikipedia.org/wiki/Mmap) file I/O by default when the store type is `hybridfs` (the default store type in OpenSearch). This leads to fast file I/O operations and improves the overall performance of both data ingestion and search. The two file extensions specific to vector values that use `mmap` are `.vec` and `.vem`. For more information about these file extensions, see [Lucene documentation](https://lucene.apache.org/core/9_0_0/core/org/apache/lucene/codecs/lucene90/Lucene90HnswVectorsFormat.html).


We probably need to add a note in a passage below that this is only about read/write operations from k-NN and Lucene engine, otherwise it sounds like that whole OpenSearch is using NIO.

In versions 2.4 and below, by default, read/write operations were based on Java NIO, which can be slow depending on the Lucene version and number of segments per shard.

suggested change:

In versions 2.4 and below, by default, read/write operations of Lucene-based approximate k-NN search were based on Java NIO. This can be slow depending on the Lucene version and number of segments per shard.

Signed-off-by: Fanit Kolchina <[email protected]>

cwillum

LGTM

cwillum · 2023-03-07T21:58:45Z

_search-plugins/knn/performance-tuning.md

@@ -82,6 +82,12 @@ Take the following steps to improve search performance:

   If your use case is simply to read the IDs and scores of the nearest neighbors, you can disable reading stored fields, which saves time retrieving the vectors from stored fields.

+* **Use `mmap` file I/O**
+
+   For the Lucene-based approximate k-NN search, there is no dedicated cache layer that speeds up read/write operations. Instead, the plugin is relying on the existing caching mechanism in OpenSearch core. In versions 2.4 and below of the the Lucene-based approximate k-NN search, by default, read/write operations were based on Java NIO, which can be slow depending on the Lucene version and number of segments per shard. Starting with version 2.5, k-NN enables [`mmap`](https://en.wikipedia.org/wiki/Mmap) file I/O by default when the store type is `hybridfs` (the default store type in OpenSearch). This leads to fast file I/O operations and improves the overall performance of both data ingestion and search. The two file extensions specific to vector values that use `mmap` are `.vec` and `.vem`. For more information about these file extensions, see [Lucene documentation](https://lucene.apache.org/core/9_0_0/core/org/apache/lucene/codecs/lucene90/Lucene90HnswVectorsFormat.html).


alternative:
"In Lucene-based approximate k-NN search versions 2.4 and below, read/write operations were based by default on Java NIO ..."

Naarcha-AWS

Small typo. Otherwise LGTM.

_search-plugins/knn/performance-tuning.md

Co-authored-by: Naarcha-AWS <[email protected]>

natebower

@kolchfa-aws Please see my comment and changes and let me know if you have any questions. Thanks!

natebower · 2023-03-08T14:17:32Z

_search-plugins/knn/performance-tuning.md

@@ -82,6 +82,12 @@ Take the following steps to improve search performance:

   If your use case is simply to read the IDs and scores of the nearest neighbors, you can disable reading stored fields, which saves time retrieving the vectors from stored fields.

+* **Use `mmap` file I/O**
+
+   For the Lucene-based approximate k-NN search, there is no dedicated cache layer that speeds up read/write operations. Instead, the plugin is relying on the existing caching mechanism in OpenSearch core. In versions 2.4 and below of the Lucene-based approximate k-NN search, by default, read/write operations were based on Java NIO, which can be slow depending on the Lucene version and number of segments per shard. Starting with version 2.5, k-NN enables [`mmap`](https://en.wikipedia.org/wiki/Mmap) file I/O by default when the store type is `hybridfs` (the default store type in OpenSearch). This leads to fast file I/O operations and improves the overall performance of both data ingestion and search. The two file extensions specific to vector values that use `mmap` are `.vec` and `.vem`. For more information about these file extensions, see [Lucene documentation](https://lucene.apache.org/core/9_0_0/core/org/apache/lucene/codecs/lucene90/Lucene90HnswVectorsFormat.html).


Suggested change

For the Lucene-based approximate k-NN search, there is no dedicated cache layer that speeds up read/write operations. Instead, the plugin is relying on the existing caching mechanism in OpenSearch core. In versions 2.4 and below of the Lucene-based approximate k-NN search, by default, read/write operations were based on Java NIO, which can be slow depending on the Lucene version and number of segments per shard. Starting with version 2.5, k-NN enables [`mmap`](https://en.wikipedia.org/wiki/Mmap) file I/O by default when the store type is `hybridfs` (the default store type in OpenSearch). This leads to fast file I/O operations and improves the overall performance of both data ingestion and search. The two file extensions specific to vector values that use `mmap` are `.vec` and `.vem`. For more information about these file extensions, see [Lucene documentation](https://lucene.apache.org/core/9_0_0/core/org/apache/lucene/codecs/lucene90/Lucene90HnswVectorsFormat.html).

For the Lucene-based approximate k-NN search, there is no dedicated cache layer that speeds up read/write operations. Instead, the plugin relies on the existing caching mechanism in OpenSearch core. In versions 2.4 and earlier of the Lucene-based approximate k-NN search, read/write operations were based on Java NIO by default, which can be slow, depending on the Lucene version and number of segments per shard. Starting with version 2.5, k-NN enables [`mmap`](https://en.wikipedia.org/wiki/Mmap) file I/O by default when the store type is `hybridfs` (the default store type in OpenSearch). This leads to fast file I/O operations and improves the overall performance of both data ingestion and search. The two file extensions specific to vector values that use `mmap` are `.vec` and `.vem`. For more information about these file extensions, see [the Lucene documentation](https://lucene.apache.org/core/9_0_0/core/org/apache/lucene/codecs/lucene90/Lucene90HnswVectorsFormat.html).

In Stavros' blog on approximate k-NN, it's defined as "approximate k-NN (ANN)". Let's confirm whether we should be using this acronym and, if so, define the term the first time it appears on the page and then use just the acronym thereafter.

natebower · 2023-03-08T14:20:48Z

_search-plugins/knn/performance-tuning.md

+
+   For the Lucene-based approximate k-NN search, there is no dedicated cache layer that speeds up read/write operations. Instead, the plugin is relying on the existing caching mechanism in OpenSearch core. In versions 2.4 and below of the Lucene-based approximate k-NN search, by default, read/write operations were based on Java NIO, which can be slow depending on the Lucene version and number of segments per shard. Starting with version 2.5, k-NN enables [`mmap`](https://en.wikipedia.org/wiki/Mmap) file I/O by default when the store type is `hybridfs` (the default store type in OpenSearch). This leads to fast file I/O operations and improves the overall performance of both data ingestion and search. The two file extensions specific to vector values that use `mmap` are `.vec` and `.vem`. For more information about these file extensions, see [Lucene documentation](https://lucene.apache.org/core/9_0_0/core/org/apache/lucene/codecs/lucene90/Lucene90HnswVectorsFormat.html).
+
+   The `mmap` file I/O uses the system file cache rather than memory allocated for Java heap, so no additional allocation is required. To change the default list of extensions set by the plugin, update the `index.store.hybrid.mmap.extensions` setting at the cluster level using the [Cluster Settings API]({{site.url}}{{site.baseurl}}/api-reference/cluster-api/cluster-settings). **Note**: This is an expert-level setting that requires closing the index before updating the setting and reopening it after the update.


Suggested change

The `mmap` file I/O uses the system file cache rather than memory allocated for Java heap, so no additional allocation is required. To change the default list of extensions set by the plugin, update the `index.store.hybrid.mmap.extensions` setting at the cluster level using the [Cluster Settings API]({{site.url}}{{site.baseurl}}/api-reference/cluster-api/cluster-settings). **Note**: This is an expert-level setting that requires closing the index before updating the setting and reopening it after the update.

The `mmap` file I/O uses the system file cache rather than memory allocated for the Java heap, so no additional allocation is required. To change the default list of extensions set by the plugin, update the `index.store.hybrid.mmap.extensions` setting at the cluster level using the [Cluster Settings API]({{site.url}}{{site.baseurl}}/api-reference/cluster-api/cluster-settings). **Note**: This is an expert-level setting that requires closing the index before updating the setting and reopening it after the update.

Signed-off-by: Fanit Kolchina <[email protected]>

kolchfa-aws · 2023-03-16T14:09:19Z

@martin-gaievski Which versions does this apply to? Thanks!

martin-gaievski · 2023-03-16T15:43:31Z

@martin-gaievski Which versions does this apply to? Thanks!

All versions starting from 2.5.0 and above.

* Add reference to hybrid mmap extensions setting in k-NN Signed-off-by: Fanit Kolchina <[email protected]> * Removed extra line Signed-off-by: Fanit Kolchina <[email protected]> * Reworded based on tech review Signed-off-by: Fanit Kolchina <[email protected]> * Update _search-plugins/knn/performance-tuning.md Co-authored-by: Naarcha-AWS <[email protected]> * Implemented editorial comments Signed-off-by: Fanit Kolchina <[email protected]> --------- Signed-off-by: Fanit Kolchina <[email protected]> Co-authored-by: Naarcha-AWS <[email protected]> (cherry picked from commit 492c9ec) Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

* Add reference to hybrid mmap extensions setting in k-NN * Removed extra line * Reworded based on tech review * Update _search-plugins/knn/performance-tuning.md * Implemented editorial comments --------- (cherry picked from commit 492c9ec) Signed-off-by: Fanit Kolchina <[email protected]> Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: Naarcha-AWS <[email protected]>

* Add reference to hybrid mmap extensions setting in k-NN Signed-off-by: Fanit Kolchina <[email protected]> * Removed extra line Signed-off-by: Fanit Kolchina <[email protected]> * Reworded based on tech review Signed-off-by: Fanit Kolchina <[email protected]> * Update _search-plugins/knn/performance-tuning.md Co-authored-by: Naarcha-AWS <[email protected]> * Implemented editorial comments Signed-off-by: Fanit Kolchina <[email protected]> --------- Signed-off-by: Fanit Kolchina <[email protected]> Co-authored-by: Naarcha-AWS <[email protected]>

This reverts commit 2b2b535.

…NN (#3262)"" This reverts commit a2af701.

This reverts commit 2b2b535.

…roject#3262) * Add reference to hybrid mmap extensions setting in k-NN Signed-off-by: Fanit Kolchina <[email protected]> * Removed extra line Signed-off-by: Fanit Kolchina <[email protected]> * Reworded based on tech review Signed-off-by: Fanit Kolchina <[email protected]> * Update _search-plugins/knn/performance-tuning.md Co-authored-by: Naarcha-AWS <[email protected]> * Implemented editorial comments Signed-off-by: Fanit Kolchina <[email protected]> --------- Signed-off-by: Fanit Kolchina <[email protected]> Co-authored-by: Naarcha-AWS <[email protected]>

Add reference to hybrid mmap extensions setting in k-NN

d4cae1c

Signed-off-by: Fanit Kolchina <[email protected]>

kolchfa-aws requested a review from a team as a code owner March 7, 2023 16:31

kolchfa-aws self-assigned this Mar 7, 2023

Removed extra line

bf61502

Signed-off-by: Fanit Kolchina <[email protected]>

martin-gaievski reviewed Mar 7, 2023

View reviewed changes

Reworded based on tech review

a541962

Signed-off-by: Fanit Kolchina <[email protected]>

cwillum approved these changes Mar 7, 2023

View reviewed changes

Naarcha-AWS approved these changes Mar 7, 2023

View reviewed changes

_search-plugins/knn/performance-tuning.md Outdated Show resolved Hide resolved

Update _search-plugins/knn/performance-tuning.md

59a98a2

Co-authored-by: Naarcha-AWS <[email protected]>

natebower reviewed Mar 8, 2023

View reviewed changes

kolchfa-aws added 2 commits March 8, 2023 15:27

Implemented editorial comments

5b64c19

Signed-off-by: Fanit Kolchina <[email protected]>

Resolved conflicts

745337a

Signed-off-by: Fanit Kolchina <[email protected]>

kolchfa-aws added backport 2.5 PR: Backport label for 2.5 backport 2.6 PR: Backport label for 2.6 labels Mar 16, 2023

kolchfa-aws merged commit 492c9ec into main Mar 16, 2023

opensearch-trigger-bot bot mentioned this pull request Mar 16, 2023

[Backport 2.5] Add reference to hybrid mmap extensions setting in k-NN #3476

Merged

opensearch-trigger-bot bot mentioned this pull request Mar 16, 2023

[Backport 2.6] Add reference to hybrid mmap extensions setting in k-NN #3477

Merged

vagimeli added a commit that referenced this pull request May 4, 2023

Revert "Add reference to hybrid mmap extensions setting in k-NN (#3262)"

a2af701

This reverts commit 2b2b535.

vagimeli added a commit that referenced this pull request May 4, 2023

Revert "Revert "Add reference to hybrid mmap extensions setting in k-…

05eef65

…NN (#3262)"" This reverts commit a2af701.

vagimeli added a commit that referenced this pull request May 4, 2023

Revert "Add reference to hybrid mmap extensions setting in k-NN (#3262)"

96b1829

This reverts commit 2b2b535.

kolchfa-aws deleted the mmap-extensions-knn branch March 28, 2024 21:53

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add reference to hybrid mmap extensions setting in k-NN #3262

Add reference to hybrid mmap extensions setting in k-NN #3262

kolchfa-aws commented Mar 7, 2023

kolchfa-aws commented Mar 7, 2023

martin-gaievski Mar 7, 2023

cwillum left a comment

cwillum Mar 7, 2023

Naarcha-AWS left a comment

natebower left a comment

natebower Mar 8, 2023

natebower Mar 8, 2023

natebower Mar 8, 2023

kolchfa-aws commented Mar 16, 2023

martin-gaievski commented Mar 16, 2023


		For the Lucene-based approximate k-NN search, there is no dedicated cache layer that speeds up read/write operations. Instead, the plugin is relying on the existing caching mechanism in OpenSearch core. In versions 2.4 and below of the Lucene-based approximate k-NN search, by default, read/write operations were based on Java NIO, which can be slow depending on the Lucene version and number of segments per shard. Starting with version 2.5, k-NN enables [`mmap`](https://en.wikipedia.org/wiki/Mmap) file I/O by default when the store type is `hybridfs` (the default store type in OpenSearch). This leads to fast file I/O operations and improves the overall performance of both data ingestion and search. The two file extensions specific to vector values that use `mmap` are `.vec` and `.vem`. For more information about these file extensions, see [Lucene documentation](https://lucene.apache.org/core/9_0_0/core/org/apache/lucene/codecs/lucene90/Lucene90HnswVectorsFormat.html).

		The `mmap` file I/O uses the system file cache rather than memory allocated for Java heap, so no additional allocation is required. To change the default list of extensions set by the plugin, update the `index.store.hybrid.mmap.extensions` setting at the cluster level using the [Cluster Settings API]({{site.url}}{{site.baseurl}}/api-reference/cluster-api/cluster-settings). Note: This is an expert-level setting that requires closing the index before updating the setting and reopening it after the update.

Add reference to hybrid mmap extensions setting in k-NN #3262

Add reference to hybrid mmap extensions setting in k-NN #3262

Conversation

kolchfa-aws commented Mar 7, 2023

Checklist

kolchfa-aws commented Mar 7, 2023

martin-gaievski Mar 7, 2023

Choose a reason for hiding this comment

cwillum left a comment

Choose a reason for hiding this comment

cwillum Mar 7, 2023

Choose a reason for hiding this comment

Naarcha-AWS left a comment

Choose a reason for hiding this comment

natebower left a comment

Choose a reason for hiding this comment

natebower Mar 8, 2023

Choose a reason for hiding this comment

natebower Mar 8, 2023

Choose a reason for hiding this comment

natebower Mar 8, 2023

Choose a reason for hiding this comment

kolchfa-aws commented Mar 16, 2023

martin-gaievski commented Mar 16, 2023