Add Query Insights documentation #6261

ansjcy · 2024-01-25T22:23:55Z

Description

Initial documentation for query insights and top n queries.

Issues Resolved

Closes #6180

Checklist

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license and subject to the Developers Certificate of Origin.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Signed-off-by: Chenyang Ji <[email protected]>

Signed-off-by: Fanit Kolchina <[email protected]>

kolchfa-aws · 2024-01-29T23:13:52Z

@ansjcy I've reviewed the draft. A couple of comments/questions:

I don't think the diagram is necessary for the end user. It seems more appropriate for developers.
Is there any other possibility for exporter other than local index?
What is the repo of this plugin?
For settings, I provided API requests so users can update this on existing clusters. Please verify that the requests are technically accurate.
Thanks!

Signed-off-by: Chenyang Ji <[email protected]>

ansjcy · 2024-02-05T23:12:17Z

Hi @kolchfa-aws , thanks for reviewing this documentation PR for query insights!

I don't think the diagram is necessary for the end user. It seems more appropriate for developers.

Make sense, I removed the diagram in this PR.

Is there any other possibility for exporter other than local index?

Actually we decided to take out exporters in our initial release for query insights, thus I removed all exporter related document in this PR.

What is the repo of this plugin?

This plugin will live in the core OpenSearch repo.

For settings, I provided API requests so users can update this on existing clusters. Please verify that the requests are technically accurate.

Just reviewed, they all make sense to me. Thanks for the update.

kolchfa-aws · 2024-02-06T20:23:59Z

@ansjcy Thank you. Because this plugin will be part of core, will it come with default installation? We have steps to install the plugin so I want to make sure these steps are necessary.

Signed-off-by: Fanit Kolchina <[email protected]>

ansjcy · 2024-02-07T20:24:58Z

it will not be bundled by default for now, we can add it in Additional plugins since that section mentions those plugins can be installed with bin/opensearch-plugin install <plugin-name>.

Signed-off-by: Fanit Kolchina <[email protected]>

kolchfa-aws · 2024-02-07T21:07:24Z

Thanks, @ansjcy! I added installation steps.

…mentation-website into query-insights

natebower

@ansjcy @kolchfa-aws Please see my comments and changes and let me know if you have any questions. Of note, there are some naming/wording issues in this content that make it a little confusing and hard to follow in places. Thanks!

_install-and-configure/configuring-opensearch/plugin-settings.md

_observing-your-data/query-insights/index.md

natebower · 2024-02-08T12:29:37Z

_observing-your-data/query-insights/index.md

+has_toc: false
+---
+
+# Query insights


Global: It's not ideal to have one thing named the "Query Insights plugin" and to then name something within that "query insights" or "query insights features." This can result in reader confusion regarding which one we're referencing as well as awkward phrasing throughout.

_observing-your-data/query-insights/index.md

natebower · 2024-02-08T12:54:09Z

_observing-your-data/query-insights/top-n-queries.md

+
+## Configuring top n size
+
+You can configure the top n size using `search.insights.top_queries.latency.top_n_size`. For example, a cluster with the following configuration will collect the top 10 queries in the specified window size:


Should the end of the first sentence read "the search.insights.top_queries.latency.top_n_size [noun]"? Replace "window size".

natebower · 2024-02-08T12:55:38Z

_observing-your-data/query-insights/top-n-queries.md

+```
+{% include copy-curl.html %}
+
+## Configuring top n size


This wording is also awkward and starts to become confusing when attempting to differentiate the content in this section from the preceding content. In other words, what is a "top n size"?

_observing-your-data/query-insights/top-n-queries.md

Co-authored-by: Nathan Bower <[email protected]> Signed-off-by: kolchfa-aws <[email protected]>

Signed-off-by: Fanit Kolchina <[email protected]>

kolchfa-aws

LGTM. Thank you, @ansjcy!

Signed-off-by: Fanit Kolchina <[email protected]>

…mentation-website into query-insights

Signed-off-by: Fanit Kolchina <[email protected]>

* Query Insights initial documentation Signed-off-by: Chenyang Ji <[email protected]> * Doc review Signed-off-by: Fanit Kolchina <[email protected]> * update endpoints and remove exporters Signed-off-by: Chenyang Ji <[email protected]> * Moved query insights to additional plugins section Signed-off-by: Fanit Kolchina <[email protected]> * Added installation instructions Signed-off-by: Fanit Kolchina <[email protected]> * Apply suggestions from code review Co-authored-by: Nathan Bower <[email protected]> Signed-off-by: kolchfa-aws <[email protected]> * editorial comments Signed-off-by: Fanit Kolchina <[email protected]> * Resolve merge conflicts Signed-off-by: Fanit Kolchina <[email protected]> * merge conflicts and link fix Signed-off-by: Fanit Kolchina <[email protected]> * Fix link Signed-off-by: Fanit Kolchina <[email protected]> --------- Signed-off-by: Chenyang Ji <[email protected]> Signed-off-by: Fanit Kolchina <[email protected]> Signed-off-by: kolchfa-aws <[email protected]> Co-authored-by: Fanit Kolchina <[email protected]> Co-authored-by: kolchfa-aws <[email protected]> Co-authored-by: Nathan Bower <[email protected]> Signed-off-by: Tianjing Li <[email protected]>

Signed-off-by: Tianjing Li <[email protected]> Clarification of Customer Comment # 20230525 (opensearch-project#6374) * 20230525 clarificaton for customer comment on refernes to kibana Signed-off-by: [email protected] <[email protected]> * 20230525 clarificaton for customer comment on refernes to kibana - scentance clarification Signed-off-by: [email protected] <[email protected]> --------- Signed-off-by: [email protected] <[email protected]> Signed-off-by: Tianjing Li <[email protected]> Add searchbp metrics to Performance Analyzer (opensearch-project#5390) * added searchbp metrics Signed-off-by: Heather Halter <[email protected]> * Update reference.md Signed-off-by: Naarcha-AWS <[email protected]> * Update reference.md Signed-off-by: Naarcha-AWS <[email protected]> * Apply suggestions from code review Co-authored-by: Nathan Bower <[email protected]> Signed-off-by: Naarcha-AWS <[email protected]> --------- Signed-off-by: Heather Halter <[email protected]> Signed-off-by: Naarcha-AWS <[email protected]> Co-authored-by: Naarcha-AWS <[email protected]> Co-authored-by: Nathan Bower <[email protected]> Signed-off-by: Tianjing Li <[email protected]> Add documentation for new reranking feature in 2.12 (opensearch-project#6368) * Create reranking.md document new reranking feature in 2.12 Signed-off-by: HenryL27 <[email protected]> * Doc review and address comments Signed-off-by: Fanit Kolchina <[email protected]> * Apply suggestions from code review Co-authored-by: Nathan Bower <[email protected]> Signed-off-by: kolchfa-aws <[email protected]> * Update _search-plugins/search-pipelines/rerank-processor.md Co-authored-by: Nathan Bower <[email protected]> Signed-off-by: kolchfa-aws <[email protected]> * Update _search-plugins/search-pipelines/rerank-processor.md Co-authored-by: Nathan Bower <[email protected]> Signed-off-by: kolchfa-aws <[email protected]> --------- Signed-off-by: HenryL27 <[email protected]> Signed-off-by: Fanit Kolchina <[email protected]> Signed-off-by: kolchfa-aws <[email protected]> Co-authored-by: Fanit Kolchina <[email protected]> Co-authored-by: kolchfa-aws <[email protected]> Co-authored-by: Nathan Bower <[email protected]> Signed-off-by: Tianjing Li <[email protected]> Add Query Insights documentation (opensearch-project#6261) * Query Insights initial documentation Signed-off-by: Chenyang Ji <[email protected]> * Doc review Signed-off-by: Fanit Kolchina <[email protected]> * update endpoints and remove exporters Signed-off-by: Chenyang Ji <[email protected]> * Moved query insights to additional plugins section Signed-off-by: Fanit Kolchina <[email protected]> * Added installation instructions Signed-off-by: Fanit Kolchina <[email protected]> * Apply suggestions from code review Co-authored-by: Nathan Bower <[email protected]> Signed-off-by: kolchfa-aws <[email protected]> * editorial comments Signed-off-by: Fanit Kolchina <[email protected]> * Resolve merge conflicts Signed-off-by: Fanit Kolchina <[email protected]> * merge conflicts and link fix Signed-off-by: Fanit Kolchina <[email protected]> * Fix link Signed-off-by: Fanit Kolchina <[email protected]> --------- Signed-off-by: Chenyang Ji <[email protected]> Signed-off-by: Fanit Kolchina <[email protected]> Signed-off-by: kolchfa-aws <[email protected]> Co-authored-by: Fanit Kolchina <[email protected]> Co-authored-by: kolchfa-aws <[email protected]> Co-authored-by: Nathan Bower <[email protected]> Signed-off-by: Tianjing Li <[email protected]> Add Running a Workload (opensearch-project#6287) * Add Running a Workload draft Signed-off-by: Naarcha-AWS <[email protected]> * Apply suggestions from code review Signed-off-by: Naarcha-AWS <[email protected]> * Update running-workloads.md Signed-off-by: Naarcha-AWS <[email protected]> * Update _benchmark/user-guide/running-workloads.md Signed-off-by: Naarcha-AWS <[email protected]> * Fix link Signed-off-by: Naarcha-AWS <[email protected]> * Add additional missing link Signed-off-by: Naarcha-AWS <[email protected]> * Add running workloads Signed-off-by: Naarcha-AWS <[email protected]> * Apply suggestions from code review Signed-off-by: Naarcha-AWS <[email protected]> * Apply suggestions from code review Co-authored-by: Melissa Vagi <[email protected]> Signed-off-by: Naarcha-AWS <[email protected]> * Apply suggestions from code review Co-authored-by: Nathan Bower <[email protected]> Signed-off-by: Naarcha-AWS <[email protected]> * Add numbered steps Signed-off-by: Naarcha-AWS <[email protected]> --------- Signed-off-by: Naarcha-AWS <[email protected]> Signed-off-by: Naarcha-AWS <[email protected]> Co-authored-by: Melissa Vagi <[email protected]> Co-authored-by: Nathan Bower <[email protected]> Signed-off-by: Tianjing Li <[email protected]>

Clarification of Customer Comment # 20230525 (#6374) * 20230525 clarificaton for customer comment on refernes to kibana * 20230525 clarificaton for customer comment on refernes to kibana - scentance clarification --------- Add searchbp metrics to Performance Analyzer (#5390) * added searchbp metrics * Update reference.md * Update reference.md * Apply suggestions from code review --------- Add documentation for new reranking feature in 2.12 (#6368) * Create reranking.md document new reranking feature in 2.12 * Doc review and address comments * Apply suggestions from code review * Update _search-plugins/search-pipelines/rerank-processor.md * Update _search-plugins/search-pipelines/rerank-processor.md --------- Add Query Insights documentation (#6261) * Query Insights initial documentation * Doc review * update endpoints and remove exporters * Moved query insights to additional plugins section * Added installation instructions * Apply suggestions from code review * editorial comments * Resolve merge conflicts * merge conflicts and link fix * Fix link --------- Add Running a Workload (#6287) * Add Running a Workload draft * Apply suggestions from code review * Update running-workloads.md * Update _benchmark/user-guide/running-workloads.md * Fix link * Add additional missing link * Add running workloads * Apply suggestions from code review * Apply suggestions from code review * Apply suggestions from code review * Add numbered steps --------- Signed-off-by: Naarcha-AWS <[email protected]> Signed-off-by: Naarcha-AWS <[email protected]> Signed-off-by: Tianjing Li <[email protected]> Co-authored-by: Melissa Vagi <[email protected]> Co-authored-by: Nathan Bower <[email protected]> Co-authored-by: kolchfa-aws <[email protected]>

* Query Insights initial documentation Signed-off-by: Chenyang Ji <[email protected]> * Doc review Signed-off-by: Fanit Kolchina <[email protected]> * update endpoints and remove exporters Signed-off-by: Chenyang Ji <[email protected]> * Moved query insights to additional plugins section Signed-off-by: Fanit Kolchina <[email protected]> * Added installation instructions Signed-off-by: Fanit Kolchina <[email protected]> * Apply suggestions from code review Co-authored-by: Nathan Bower <[email protected]> Signed-off-by: kolchfa-aws <[email protected]> * editorial comments Signed-off-by: Fanit Kolchina <[email protected]> * Resolve merge conflicts Signed-off-by: Fanit Kolchina <[email protected]> * merge conflicts and link fix Signed-off-by: Fanit Kolchina <[email protected]> * Fix link Signed-off-by: Fanit Kolchina <[email protected]> --------- Signed-off-by: Chenyang Ji <[email protected]> Signed-off-by: Fanit Kolchina <[email protected]> Signed-off-by: kolchfa-aws <[email protected]> Co-authored-by: Fanit Kolchina <[email protected]> Co-authored-by: kolchfa-aws <[email protected]> Co-authored-by: Nathan Bower <[email protected]>

Clarification of Customer Comment # 20230525 (opensearch-project#6374) * 20230525 clarificaton for customer comment on refernes to kibana * 20230525 clarificaton for customer comment on refernes to kibana - scentance clarification --------- Add searchbp metrics to Performance Analyzer (opensearch-project#5390) * added searchbp metrics * Update reference.md * Update reference.md * Apply suggestions from code review --------- Add documentation for new reranking feature in 2.12 (opensearch-project#6368) * Create reranking.md document new reranking feature in 2.12 * Doc review and address comments * Apply suggestions from code review * Update _search-plugins/search-pipelines/rerank-processor.md * Update _search-plugins/search-pipelines/rerank-processor.md --------- Add Query Insights documentation (opensearch-project#6261) * Query Insights initial documentation * Doc review * update endpoints and remove exporters * Moved query insights to additional plugins section * Added installation instructions * Apply suggestions from code review * editorial comments * Resolve merge conflicts * merge conflicts and link fix * Fix link --------- Add Running a Workload (opensearch-project#6287) * Add Running a Workload draft * Apply suggestions from code review * Update running-workloads.md * Update _benchmark/user-guide/running-workloads.md * Fix link * Add additional missing link * Add running workloads * Apply suggestions from code review * Apply suggestions from code review * Apply suggestions from code review * Add numbered steps --------- Signed-off-by: Naarcha-AWS <[email protected]> Signed-off-by: Naarcha-AWS <[email protected]> Signed-off-by: Tianjing Li <[email protected]> Co-authored-by: Melissa Vagi <[email protected]> Co-authored-by: Nathan Bower <[email protected]> Co-authored-by: kolchfa-aws <[email protected]>

Query Insights initial documentation

94f689e

Signed-off-by: Chenyang Ji <[email protected]>

ansjcy requested review from hdhalter, kolchfa-aws, Naarcha-AWS, vagimeli, AMoo-Miki, natebower and dlvenable as code owners January 25, 2024 22:23

ansjcy changed the title ~~Query Insights initial documentation~~ [DRAFT] Query Insights initial documentation Jan 25, 2024

ansjcy mentioned this pull request Jan 25, 2024

[META] Generic Query Insights Framework opensearch-project/OpenSearch#11522

Open

32 tasks

kolchfa-aws self-assigned this Jan 25, 2024

kolchfa-aws added release-notes PR: Include this PR in the automated release notes v2.12.0 labels Jan 25, 2024

hdhalter added the 4 - Doc review PR: Doc review in progress label Jan 29, 2024

Doc review

c466d4a

Signed-off-by: Fanit Kolchina <[email protected]>

update endpoints and remove exporters

18c9959

Signed-off-by: Chenyang Ji <[email protected]>

hdhalter changed the title ~~[DRAFT] Query Insights initial documentation~~ Add Query Insights initial documentation Feb 7, 2024

kolchfa-aws and others added 2 commits February 7, 2024 13:47

Moved query insights to additional plugins section

28607d3

Signed-off-by: Fanit Kolchina <[email protected]>

Merge branch 'main' into query-insights-documentation

b82330b

Added installation instructions

0fd4191

Signed-off-by: Fanit Kolchina <[email protected]>

Merge branch 'query-insights-documentation' of github.com:ansjcy/docu…

f186bc7

…mentation-website into query-insights

natebower reviewed Feb 8, 2024

View reviewed changes

kolchfa-aws reviewed Feb 8, 2024

View reviewed changes

_observing-your-data/query-insights/top-n-queries.md Outdated Show resolved Hide resolved

Apply suggestions from code review

53c032c

Co-authored-by: Nathan Bower <[email protected]> Signed-off-by: kolchfa-aws <[email protected]>

hdhalter removed the 4 - Doc review PR: Doc review in progress label Feb 8, 2024

hdhalter added the 5 - Editorial review PR: Editorial review in progress label Feb 8, 2024

natebower added 6 - Done but waiting to merge PR: The work is done and ready to merge and removed 5 - Editorial review PR: Editorial review in progress labels Feb 13, 2024

kolchfa-aws and others added 3 commits February 13, 2024 12:02

editorial comments

4f24d1e

Signed-off-by: Fanit Kolchina <[email protected]>

Resolved merge conflicts

4a53088

Signed-off-by: Fanit Kolchina <[email protected]>

Merge branch 'main' into query-insights-documentation

52b383b

kolchfa-aws approved these changes Feb 13, 2024

View reviewed changes

kolchfa-aws added 3 commits February 13, 2024 12:09

Resolve merge conflicts

42b9e34

Signed-off-by: Fanit Kolchina <[email protected]>

Merge branch 'query-insights-documentation' of github.com:ansjcy/docu…

fd90983

…mentation-website into query-insights

merge conflicts and link fix

b3d2581

Signed-off-by: Fanit Kolchina <[email protected]>

hdhalter changed the title ~~Add Query Insights initial documentation~~ Add Query Insights documentation Feb 13, 2024

Fix link

0a79b38

Signed-off-by: Fanit Kolchina <[email protected]>

kolchfa-aws merged commit c87bd64 into opensearch-project:main Feb 13, 2024
2 of 3 checks passed

hdhalter added 3 - Done Issue is done/complete and removed 6 - Done but waiting to merge PR: The work is done and ready to merge labels Feb 13, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add Query Insights documentation #6261

Add Query Insights documentation #6261

ansjcy commented Jan 25, 2024 •

edited by hdhalter

Loading

kolchfa-aws commented Jan 29, 2024

ansjcy commented Feb 5, 2024

kolchfa-aws commented Feb 6, 2024

ansjcy commented Feb 7, 2024

kolchfa-aws commented Feb 7, 2024

natebower left a comment

natebower Feb 8, 2024

natebower Feb 8, 2024

natebower Feb 8, 2024

kolchfa-aws Feb 13, 2024

kolchfa-aws left a comment


		## Configuring top n size

		You can configure the top n size using `search.insights.top_queries.latency.top_n_size`. For example, a cluster with the following configuration will collect the top 10 queries in the specified window size:

Add Query Insights documentation #6261

Add Query Insights documentation #6261

Conversation

ansjcy commented Jan 25, 2024 • edited by hdhalter Loading

Description

Issues Resolved

Checklist

kolchfa-aws commented Jan 29, 2024

ansjcy commented Feb 5, 2024

kolchfa-aws commented Feb 6, 2024

ansjcy commented Feb 7, 2024

kolchfa-aws commented Feb 7, 2024

natebower left a comment

Choose a reason for hiding this comment

natebower Feb 8, 2024

Choose a reason for hiding this comment

natebower Feb 8, 2024

Choose a reason for hiding this comment

natebower Feb 8, 2024

Choose a reason for hiding this comment

kolchfa-aws Feb 13, 2024

Choose a reason for hiding this comment

kolchfa-aws left a comment

Choose a reason for hiding this comment

ansjcy commented Jan 25, 2024 •

edited by hdhalter

Loading