[DOC] Add new documentation about Query Workbench (#5378)

* Rewrite Query Workbench --------- Signed-off-by: Melissa Vagi <[email protected]> Signed-off-by: Fanit Kolchina <[email protected]> Co-authored-by: kolchfa-aws <[email protected]> Co-authored-by: Fanit Kolchina <[email protected]>
opensearch-project · Dec 20, 2023 · 32a1af4 · 32a1af4
1 parent eb207b3
commit 32a1af4
Show file tree

Hide file tree

Showing 14 changed files with 156 additions and 140 deletions.
diff --git a/_dashboards/query-workbench.md b/_dashboards/query-workbench.md
@@ -0,0 +1,118 @@
+---
+layout: default
+title: Query Workbench
+nav_order: 125
+redirect_from:
+  - /search-plugins/sql/workbench/
+---
+
+# Query Workbench
+
+Query Workbench is a tool within OpenSearch Dashboards. You can use Query Workbench to run on-demand [SQL]({{site.url}}{{site.baseurl}}/search-plugins/sql/sql/index/) and [PPL]({{site.url}}{{site.baseurl}}/search-plugins/sql/ppl/index/) queries, translate queries into their equivalent REST API calls, and view and save results in different [response formats]({{site.url}}{{site.baseurl}}/search-plugins/sql/response-formats/). 
+
+A view of the Query Workbench interface within OpenSearch Dashboards is shown in the following image.
+
+<img src="{{site.url}}{{site.baseurl}}/images/dashboards/query-workbench-ui.png" alt="Query Workbench interface within OpenSearch Dashboards"  width="700">
+
+## Prerequisites 
+
+Before getting started, make sure you have [indexed your data]({{site.url}}{{site.baseurl}}/im-plugin/index/). 
+
+For this tutorial, you can index the following sample documents. Alternatively, you can use the [OpenSearch Playground](https://playground.opensearch.org/app/opensearch-query-workbench#/), which has preloaded indexes that you can use to try out Query Workbench.
+
+To index sample documents, send the following [Bulk API]({{site.url}}{{site.baseurl}}/api-reference/document-apis/bulk/) request:
+
+```json
+PUT accounts/_bulk?refresh
+{"index":{"_id":"1"}}
+{"account_number":1,"balance":39225,"firstname":"Amber","lastname":"Duke","age":32,"gender":"M","address":"880 Holmes Lane","employer":"Pyrami","email":"[email protected]","city":"Brogan","state":"IL"}
+{"index":{"_id":"6"}}
+{"account_number":6,"balance":5686,"firstname":"Hattie","lastname":"Bond","age":36,"gender":"M","address":"671 Bristol Street","employer":"Netagy","email":"[email protected]","city":"Dante","state":"TN"}
+{"index":{"_id":"13"}}
+{"account_number":13,"balance":32838,"firstname":"Nanette","lastname":"Bates","age":28,"gender":"F","address":"789 Madison Street","employer":"Quility","email":"[email protected]","city":"Nogal","state":"VA"}
+{"index":{"_id":"18"}}
+{"account_number":18,"balance":4180,"firstname":"Dale","lastname":"Adams","age":33,"gender":"M","address":"467 Hutchinson Court","email":"[email protected]","city":"Orick","state":"MD"}
+```
+{% include copy-curl.html %}
+
+## Running SQL queries within Query Workbench
+
+Follow these steps to learn how to run SQL queries against your OpenSearch data using Query Workbench:
+
+1. Access Query Workbench.
+    - To access Query Workbench, go to OpenSearch Dashboards and choose **OpenSearch Plugins** > **Query Workbench** from the main menu.
+
+2. Run a query.
+    - Select the **SQL** button. In the query editor, type a SQL expression and then select the **Run** button to run the query. 
+
+    The following example query retrieves the first name, last name, and balance from the `accounts` index for accounts with a balance greater than 10,000 and sorts by balance in descending order:
+
+    ```sql
+    SELECT
+      firstname,
+      lastname,
+      balance
+    FROM
+      accounts
+    WHERE
+      balance > 10000
+    ORDER BY
+      balance DESC;
+    ```
+    {% include copy.html %}
+
+3. View the results.
+    - View the results in the **Results** pane, which presents the query output in tabular format. You can filter and download the results as needed.
+
+   The following image shows the query editor pane and results pane for the preceding SQL query:
+
+    <img src="{{site.url}}{{site.baseurl}}/images/dashboards/query-workbench-query-step2.png" alt="Query Workbench SQL query input and results output panes" width="800">
+
+4. Clear the query editor.  
+    - Select the **Clear** button to clear the query editor and run a new query. 
+
+5. Examine how the query is processed.
+    - Select the **Explain** button to examine how OpenSearch processes the query, including the steps involved and order of operations. 
+
+    The following image shows the explanation of the SQL query that was run in step 2.
+
+    <img src="{{site.url}}{{site.baseurl}}/images/dashboards/query-explain.png" alt="Query Workbench SQL query explanation pane" width="500">
+
+## Running PPL queries within Query Workbench
+
+Follow these steps to learn how to run PPL queries against your OpenSearch data using Query Workbench:
+
+1. Access Query Workbench.
+    - To access Query Workbench, go to OpenSearch Dashboards and choose **OpenSearch Plugins** > **Query Workbench** from the main menu.
+
+2. Run a query.
+    - Select the **PPL** button. In the query editor, type a PPL query and then select the **Run** button to run the query. 
+
+    The following is an example query that retrieves the `firstname` and `lastname` fields for documents in the `accounts` index with age greater than `18`:
+
+    ```sql
+    search source=accounts
+    | where age > 18
+    | fields firstname, lastname
+    ```
+    {% include copy.html %}
+
+3. View the results.
+    - View the results in the **Results** pane, which presents the query output in tabular format.
+
+   The following image shows the query editor pane and results pane for the PPL query that was run in step 2:
+
+    <img src="{{site.url}}{{site.baseurl}}/images/dashboards/query-workbench-ppl.png" alt="Query Workbench PPL query input and results output panes">
+
+4. Clear the query editor.  
+    - Select the **Clear** button to clear the query editor and run a new query. 
+
+5. Examine how the query is processed.
+    - Select the **Explain** button to examine how OpenSearch processes the query, including the steps involved and order of operations. 
+
+    The following image shows the explanation of the PPL query that was run in step 2.
+
+    <img src="{{site.url}}{{site.baseurl}}/images/dashboards/query-PPL-explain.png" alt="Query Workbench PPL query explanation pane" width="500">
+
+Query Workbench does not support delete or update operations through SQL or PPL. Access to data is read-only.
+{: .important}
diff --git a/_search-plugins/sql/index.md b/_search-plugins/sql/index.md
@@ -10,8 +10,12 @@ redirect_from:
 
 # SQL and PPL
 
-OpenSearch SQL lets you write queries in SQL rather than the [OpenSearch query domain-specific language (DSL)]({{site.url}}{{site.baseurl}}/opensearch/query-dsl/full-text). If you're already familiar with SQL and don't want to learn the query DSL, this feature is a great option.
+OpenSearch SQL lets you write queries in SQL rather than the [OpenSearch query domain-specific language (DSL)]({{site.url}}{{site.baseurl}}/opensearch/query-dsl/full-text/).
 
-## Contributing
+## Next steps
 
-To get involved and help us improve the SQL plugin, see the [development guide](https://github.com/opensearch-project/sql/blob/main/DEVELOPER_GUIDE.rst) for instructions on setting up your development environment and building the project.
+- Learn about the [SQL and PPL API]({{site.url}}{{site.baseurl}}/search-plugins/sql/sql-ppl-api/).
+- Learn about [using SQL within OpenSearch]({{site.url}}{{site.baseurl}}/search-plugins/sql/sql/index/).
+- Learn about [using PPL within OpenSearch]({{site.url}}{{site.baseurl}}/search-plugins/sql/ppl/index/).
+- Learn about [using Query Workbench for SQL and PPL queries within OpenSearch Dashboards]({{site.url}}{{site.baseurl}}/dashboards/query-workbench/). 
+- Learn more about OpenSearch SQL in the [Developer Guide](https://github.com/opensearch-project/sql/blob/main/DEVELOPER_GUIDE.rst).
diff --git a/_search-plugins/sql/ppl/index.md b/_search-plugins/sql/ppl/index.md
@@ -1,6 +1,6 @@
 ---
 layout: default
-title: PPL &ndash; Piped Processing Language
+title: PPL
 parent: SQL and PPL
 nav_order: 5
 has_children: true
@@ -14,53 +14,34 @@ redirect_from:
   - /search-plugins/ppl/protocol/
 ---
 
-# PPL &ndash; Piped Processing Language
+# PPL
 
-Piped Processing Language (PPL) is a query language that lets you use pipe (`|`) syntax to explore, discover, and query data stored in OpenSearch.
+Piped Processing Language (PPL) is a query language that focuses on processing data in a sequential, step-by-step manner. PPL uses the pipe (`|`) operator to combine commands to find and retrieve data. It is the primary language used with observability in OpenSearch and supports multi-data queries. 
 
-To quickly get up and running with PPL, use **Query Workbench** in OpenSearch Dashboards. To learn more, see [Workbench]({{site.url}}{{site.baseurl}}/search-plugins/sql/workbench/).
+## PPL syntax
 
-The PPL syntax consists of commands delimited by the pipe character (`|`) where data flows from left to right through each pipeline.
+The following example shows the basic PPL syntax:
 
 ```sql
-search command | command 1 | command 2 ...
+search source=<index-name> | <command_1> | <command_2> | ... | <command_n>
 ```
+{% include copy.html %}
 
-You can only use read-only commands like `search`, `where`, `fields`, `rename`, `dedup`, `stats`, `sort`, `eval`, `head`, `top`, and `rare`.
+See [Syntax]({{site.url}}{{site.baseurl}}/search-plugins/sql/ppl/syntax/) for specific PPL syntax examples.
 
-## Quick start
+## PPL commands
 
-To get started with PPL, choose **Dev Tools** in OpenSearch Dashboards and use the `bulk` operation to index some sample data:
+PPL filters, transforms, and aggregates data using a series of commands. See [Commands](/search-plugins/sql/ppl/functions/) for a description and an example of each command.  
 
-```json
-PUT accounts/_bulk?refresh
-{"index":{"_id":"1"}}
-{"account_number":1,"balance":39225,"firstname":"Amber","lastname":"Duke","age":32,"gender":"M","address":"880 Holmes Lane","employer":"Pyrami","email":"[email protected]","city":"Brogan","state":"IL"}
-{"index":{"_id":"6"}}
-{"account_number":6,"balance":5686,"firstname":"Hattie","lastname":"Bond","age":36,"gender":"M","address":"671 Bristol Street","employer":"Netagy","email":"[email protected]","city":"Dante","state":"TN"}
-{"index":{"_id":"13"}}
-{"account_number":13,"balance":32838,"firstname":"Nanette","lastname":"Bates","age":28,"gender":"F","address":"789 Madison Street","employer":"Quility","city":"Nogal","state":"VA"}
-{"index":{"_id":"18"}}
-{"account_number":18,"balance":4180,"firstname":"Dale","lastname":"Adams","age":33,"gender":"M","address":"467 Hutchinson Court","email":"[email protected]","city":"Orick","state":"MD"}
-```
-
-Go to **Query Workbench** and select **PPL**.
-
-The following example returns `firstname` and `lastname` fields for documents in an `accounts` index with `age` greater than 18:
+## Using PPL within OpenSearch 
 
-```sql
-search source=accounts
-| where age > 18
-| fields firstname, lastname
-```
+To use PPL, you must have installed OpenSearch Dashboards. PPL is available within the [Query Workbench tool](https://playground.opensearch.org/app/opensearch-query-workbench#/). See the [Query Workbench]({{site.url}}{{site.baseurl}}/dashboards/query-workbench/) documentation for a tutorial on using PPL within OpenSearch.
 
-#### Example response
+## Developer documentation
 
-firstname | lastname |
-:--- | :--- |
-Amber       | Duke
-Hattie      | Bond
-Nanette     | Bates
-Dale        | Adams
+Developers can find information in the following resources:
 
-![PPL query workbench]({{site.url}}{{site.baseurl}}/images/ppl.png)
+- [Piped Processing Language](https://github.com/opensearch-project/piped-processing-language) specification 
+- [OpenSearch PPL Reference Manual](https://github.com/opensearch-project/sql/blob/main/docs/user/ppl/index.rst) 
+- [Observability](https://github.com/opensearch-project/dashboards-observability/) using [PPL-based visualizations](https://github.com/opensearch-project/dashboards-observability#event-analytics)
+- PPL [Data Types](https://github.com/opensearch-project/sql/blob/main/docs/user/ppl/general/datatypes.rst)
diff --git a/_search-plugins/sql/sql-ppl-api.md b/_search-plugins/sql/sql-ppl-api.md
@@ -1,28 +1,16 @@
 ---
 layout: default
-title: SQL/PPL API
+title: SQL and PPL API
 parent: SQL and PPL
 nav_order: 1
 ---
 
-# SQL/PPL API
+# SQL and PPL API
 
 Use the SQL and PPL API to send queries to the SQL plugin. Use the `_sql` endpoint to send queries in SQL, and the `_ppl` endpoint to send queries in PPL. For both of these, you can also use the `_explain` endpoint to translate your query into [OpenSearch domain-specific language]({{site.url}}{{site.baseurl}}/opensearch/query-dsl/) (DSL) or to troubleshoot errors.
 
----
-
-#### Table of contents
-- TOC
-{:toc}
-
-
----
-
 ## Query API
 
-Introduced 1.0
-{: .label .label-purple }
-
 Sends an SQL/PPL query to the SQL plugin. You can pass the format for the response as a query parameter.
 
 ### Query parameters

diff --git a/_search-plugins/sql/sql/index.md b/_search-plugins/sql/sql/index.md
@@ -12,11 +12,7 @@ redirect_from:
 
 # SQL
 
-## Workbench
-
-The easiest way to get familiar with the SQL plugin is to use **Query Workbench** in OpenSearch Dashboards to test various queries. To learn more, see [Workbench]({{site.url}}{{site.baseurl}}/search-plugins/sql/workbench/).
-
-![OpenSearch Dashboards SQL UI plugin]({{site.url}}{{site.baseurl}}/images/sql.png)
+SQL in OpenSearch bridges the gap between traditional relational database concepts and the flexibility of OpenSearch's document-oriented data storage. This integration gives you the ability to use your SQL knowledge to query, analyze, and extract insights from your OpenSearch data.
 
 ## SQL and OpenSearch terminology
 
@@ -30,7 +26,7 @@ Column | Field
 
 ## REST API
 
-For a complete  REST API reference for the SQL plugin, see [SQL/PPL API]({{site.url}}{{site.baseurl}}/search-plugins/sql/sql-ppl-api). 
+For a complete REST API reference for the SQL plugin, see [SQL/PPL API]({{site.url}}{{site.baseurl}}/search-plugins/sql/sql-ppl-api/). 
 
 To use the SQL plugin with your own applications, send requests to the `_plugins/_sql` endpoint:
 
@@ -40,6 +36,7 @@ POST _plugins/_sql
   "query": "SELECT * FROM my-index LIMIT 50"
 }
 ```
+{% include copy-curl.html %}
 
 You can query multiple indexes by using a comma-separated list:
 
@@ -49,29 +46,33 @@ POST _plugins/_sql
   "query": "SELECT * FROM my-index1,myindex2,myindex3 LIMIT 50"
 }
 ```
+{% include copy-curl.html %}
 
-You can also specify an index pattern with a wildcard expression:
+You can specify an index pattern with a wildcard expression:
 
 ```json
 POST _plugins/_sql
 {
   "query": "SELECT * FROM my-index* LIMIT 50"
 }
 ```
+{% include copy-curl.html %}
 
-To run the above query in the command line, use the [curl](https://curl.haxx.se/) command:
+To run the preceding query in the command line, use the [curl](https://curl.haxx.se/) command:
 
 ```bash
 curl -XPOST https://localhost:9200/_plugins/_sql -u 'admin:admin' -k -H 'Content-Type: application/json' -d '{"query": "SELECT * FROM my-index* LIMIT 50"}'
 ```
+{% include copy.html %}
 
-You can specify the [response format]({{site.url}}{{site.baseurl}}/search-plugins/sql/response-formats) as JDBC, standard OpenSearch JSON, CSV, or raw. By default, queries return data in JDBC format. The following query sets the format to JSON:
+You can specify the [response format]({{site.url}}{{site.baseurl}}/search-plugins/sql/response-formats/) as JDBC, standard OpenSearch JSON, CSV, or raw. By default, queries return data in JDBC format. The following query sets the format to JSON:
 
 ```json
 POST _plugins/_sql?format=json
 {
   "query": "SELECT * FROM my-index LIMIT 50"
 }
 ```
+{% include copy-curl.html %}
 
-See the rest of this guide for more information about request parameters, settings, supported operations, and tools.
+For more information about request parameters, settings, supported operations, and tools, see the related topics under [SQL]({{site.url}}{{site.baseurl}}/search-plugins/sql/sql/index/).
diff --git a/_search-plugins/sql/workbench.md b/_search-plugins/sql/workbench.md
diff --git a/images/dashboards/query-PPL-explain.png b/images/dashboards/query-PPL-explain.png
diff --git a/images/dashboards/query-explain.png b/images/dashboards/query-explain.png
diff --git a/images/dashboards/query-list.png b/images/dashboards/query-list.png
diff --git a/images/dashboards/query-workbench-ppl.png b/images/dashboards/query-workbench-ppl.png
diff --git a/images/dashboards/query-workbench-query-step2.png b/images/dashboards/query-workbench-query-step2.png
diff --git a/images/dashboards/query-workbench-ui.png b/images/dashboards/query-workbench-ui.png
diff --git a/images/query-workbench-UI.png b/images/query-workbench-UI.png
diff --git a/images/sql.png b/images/sql.png