Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a note about zstd incompatibility with k-NN and security analytics #5014

Merged
merged 6 commits into from
Sep 22, 2023

Conversation

kolchfa-aws
Copy link
Collaborator

Fixes #5002

Checklist

  • By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license and subject to the Developers Certificate of Origin.
    For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@kolchfa-aws
Copy link
Collaborator Author

@sarthakaggarwal97 Could you review this PR please?

@kolchfa-aws kolchfa-aws added the release-notes PR: Include this PR in the automated release notes label Sep 13, 2023
@hdhalter hdhalter added the 3 - Tech review PR: Tech review in progress label Sep 14, 2023
@sarthakaggarwal97
Copy link
Contributor

LGTM! thanks @kolchfa-aws for taking this up!

@@ -24,6 +24,9 @@ It may be challenging to change the codec setting of an existing index (see [Cha
* `zstd` (OpenSearch 2.9 and later) -- This codec provides significant compression comparable to the `best_compression` codec with reasonable CPU usage and improved indexing and search performance compared to the `default` codec.
* `zstd_no_dict` (OpenSearch 2.9 and later) -- This codec is similar to `zstd` but excludes the dictionary compression feature. It provides faster indexing and search operations compared to `zstd` at the expense of a slightly larger index size.

As of OpenSearch 2.10, the `zstd` and `zstd_no_dict` codecs are not compatible with the [k-NN]({{site.url}}{{site.baseurl}}/search-plugins/knn/index/) and [Security Analytics]({{site.url}}{{site.baseurl}}/security-analytics/index/) plugins.
Copy link
Contributor

@sarthakaggarwal97 sarthakaggarwal97 Sep 14, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we rephrase it to

As of OpenSearch 2.10, the zstd and zstd_no_dict compression codecs can not be used for k-NN and Security Analytics indices.

Copy link
Contributor

@sarthakaggarwal97 sarthakaggarwal97 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor comment, zstd and zstd_no_dict codecs will not be usable for the knn/ security-analytics indices. They can still be installed together

Copy link
Contributor

@sarthakaggarwal97 sarthakaggarwal97 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks @kolchfa-aws , looks good!

Copy link
Collaborator

@natebower natebower left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM with one minor change.

Co-authored-by: Nathan Bower <[email protected]>
Signed-off-by: kolchfa-aws <[email protected]>
@kolchfa-aws kolchfa-aws added 6 - Done but waiting to merge PR: The work is done and ready to merge and removed 3 - Tech review PR: Tech review in progress labels Sep 14, 2023
@kolchfa-aws kolchfa-aws merged commit bf886cb into main Sep 22, 2023
harshavamsi pushed a commit to harshavamsi/documentation-website that referenced this pull request Oct 31, 2023
opensearch-project#5014)

* Add a note about zstd incompatibility with  k-NN and security analytics plugins

Signed-off-by: Fanit Kolchina <[email protected]>

* Capitalization

Signed-off-by: Fanit Kolchina <[email protected]>

* Update _im-plugin/index-codecs.md

Signed-off-by: kolchfa-aws <[email protected]>

* Update _im-plugin/index-codecs.md

Co-authored-by: Nathan Bower <[email protected]>
Signed-off-by: kolchfa-aws <[email protected]>

---------

Signed-off-by: Fanit Kolchina <[email protected]>
Signed-off-by: kolchfa-aws <[email protected]>
Co-authored-by: Nathan Bower <[email protected]>
vagimeli pushed a commit that referenced this pull request Dec 21, 2023
#5014)

* Add a note about zstd incompatibility with  k-NN and security analytics plugins

Signed-off-by: Fanit Kolchina <[email protected]>

* Capitalization

Signed-off-by: Fanit Kolchina <[email protected]>

* Update _im-plugin/index-codecs.md

Signed-off-by: kolchfa-aws <[email protected]>

* Update _im-plugin/index-codecs.md

Co-authored-by: Nathan Bower <[email protected]>
Signed-off-by: kolchfa-aws <[email protected]>

---------

Signed-off-by: Fanit Kolchina <[email protected]>
Signed-off-by: kolchfa-aws <[email protected]>
Co-authored-by: Nathan Bower <[email protected]>
@Naarcha-AWS Naarcha-AWS deleted the zstd-update branch March 28, 2024 23:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
6 - Done but waiting to merge PR: The work is done and ready to merge release-notes PR: Include this PR in the automated release notes v2.10.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[DOC] ZStd Documentation update to exclude knn and correlation indices for v2.10
4 participants