Skip to content

Commit

Permalink
[DOC] Create new documentation about index patterns (#5039)
Browse files Browse the repository at this point in the history
* Create new documentation about index patterns

---------

Signed-off-by: Melissa Vagi <[email protected]>
Co-authored-by: Josh Romero <[email protected]>
Co-authored-by: kolchfa-aws <[email protected]>
Co-authored-by: Nathan Bower <[email protected]>
  • Loading branch information
4 people authored Oct 13, 2023
1 parent 346c52d commit 035debb
Show file tree
Hide file tree
Showing 10 changed files with 88 additions and 16 deletions.
2 changes: 1 addition & 1 deletion _dashboards/discover/time-filter.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
layout: default
title: Time filter
parent: Discover
nav_order: 10
nav_order: 20
redirect_from:
- /dashboards/get-started/time-filter/
---
Expand Down
7 changes: 7 additions & 0 deletions _dashboards/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,13 @@ OpenSearch Dashboards is the user interface that lets you visualize your OpenSea
|---------|-------------|
| [Observability in OpenSearch Dashboards]({{site.url}}{{site.baseurl}}//observing-your-data/index/) | Observe, monitor, and secure data and improve performance across tools and workflows. |


## Dashboards Management

| Concept | Description |
|---------|-------------|
| [Dashboards Management]({{site.url}}{{site.baseurl}}/dashboards/management/management-index/) | Learn about the command center for customizing your OpenSearch Dashboards behavior, creating index patterns, and configuring data sources. |

## Dev Tools

| Concept | Description |
Expand Down
65 changes: 65 additions & 0 deletions _dashboards/management/index-patterns.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
---
layout: default
title: Index patterns
parent: Dashboards Management
nav_order: 10
has_children: true
---

# Index patterns

Index patterns are essential for accessing OpenSearch data. An _index pattern_ is the schema of an OpenSearch index. You need to create index patterns to define how data is retrieved and fields are formatted so that you can query, search, and display data.

An index pattern can point to one or more indexes, data streams, or index aliases. For example, an index pattern can point you to your log data from yesterday or all indexes that contain that data. If you store data in multiple indexes, creating an index pattern enables your visualizations to retrieve data from all indexes that match the index pattern.

## Get started

In this tutorial, you'll learn to create index patterns.

{::nomarkdown}<img src="{{site.url}}{{site.baseurl}}/images/icons/alert-icon.png" class="inline-icon" alt="alert icon" size="m"/>{:/}**Note**<br>
To create or modify index patterns, you must have create, manage, and delete permissions. Contact your administrator for support. For more information, refer to [Multi-tenancy configuration]({{site.url}}{{site.baseurl}}/security/multi-tenancy/multi-tenancy-config/#give-roles-access-to-tenants).
{: .note}

## Prerequisites

Before you can create an index pattern, your data must be indexed. To learn about indexing your data in OpenSearch, see [Managing indexes]({{site.url}}{{site.baseurl}}/im-plugin/index/).

## Best practices

Consider the following best practices when creating index patterns:

- **Make your index patterns specific.** Instead of creating an index pattern that matches all indexes, create an index pattern that matches all indexes starting with a certain prefix, for example, `my-index-`. The more specific your index patterns, the better it will be to query and analyze your data.
- **Use wildcards sparingly.** Wildcards can be useful for matching multiple indexes, but they can also make it more difficult to manage your index patterns. Try to use wildcards as specifically as possible.
- **Test your index patterns.** Make sure to test your index patterns to ensure that they match the correct indexes.

## Creating an index pattern

If you added sample data, you have index patterns that you can use to analyze that data. To create an index pattern for your own data, follow these steps.

### Step 1: Define the index pattern

1. Go to OpenSearch Dashboards, and select **Management** > **Dashboards Management** > **Index patterns**.
2. Select **Create index pattern**.
3. From the **Create index pattern** window, define the index pattern by entering a name for your index pattern in the **Index pattern name** field. Dashboards automatically adds a wildcard, `*`, once you start typing. Using a wildcard is helpful for matching an index pattern to multiple sources or indexes. A dropdown list displaying all the indexes that match your index pattern appears when you start typing.
4. Select **Next step**.

An example of step 1 is shown in the following image. Note that the index pattern `security*` matches three indexes. By defining the pattern with a wildcard `*`, you can query and visualize all the data in your indexes.

<img src="{{site.url}}{{site.baseurl}}/images/dashboards/index-patterns-step1.png" alt="Index pattern step 1 UI " width="700"/>

### Step 2: Configure the settings

1. Select `@timestamp` from the dropdown menu to specify the time field for OpenSearch to use when filtering documents based on time. Selecting this time filter determines which field the time filter is applied to. It can be the timestamp of a request or any relevant timestamp field. If you don't want to use a time filter, select that option from the dropdown menu. If you select this option, OpenSearch returns all of the data in the indexes that match the pattern.

2. Select **Create index pattern.** An example is shown in the following image.

<img src="{{site.url}}{{site.baseurl}}/images/dashboards/index-pattern-step2.png" alt="Index pattern step 2 UI " width="700"/>

Once the index pattern has been created, you can view the mapping of the matching indexes. Within the table, you can see the list of fields, along with their data type and properties. An example is shown in the following image.

<img src="{{site.url}}{{site.baseurl}}/images/dashboards/index-pattern-table.png" alt="Index pattern table UI " width="700"/>

## Next steps

- [Understand your data through visuals]({{site.url}}{{site.baseurl}}/dashboards/visualize/viz-index/).
- [Dig into your data]({{site.url}}{{site.baseurl}}/dashboards/discover/index-discover/).
2 changes: 1 addition & 1 deletion _dashboards/management/management-index.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ Dashboards Management serves as the command center for customizing OpenSearch Da

The following applications are available in Dashboards Management:

- **Index Patterns:** To access OpenSearch data, you need to create an index pattern so that you can select the data you want to use and define the properties of the fields. An index pattern can point to one or more indexes, data streams, or index aliases.
- **[Index Patterns]({{site.url}}{{site.baseurl}}/dashboards/management/index-patterns/):** To access OpenSearch data, you need to create an index pattern so that you can select the data you want to use and define the properties of the fields. The Index Pattern tool gives you the ability to create an index pattern from within the UI. Index patterns point to one or more indexes, data streams, or index aliases.
- **[Data Sources]({{site.url}}{{site.baseurl}}/dashboards/management/multi-data-sources/):** The Data Sources tool is used to configure and manage the data sources that OpenSearch uses to collect and analyze data. You can use the tool to specify the source configuration in your copy of the [OpenSearch Dashboards configuration file]({{site.url}}{{site.baseurl}}https://github.com/opensearch-project/OpenSearch-Dashboards/blob/main/config/opensearch_dashboards.yml).
- **Saved Objects:** The Saved Objects tool helps you organize and manage your saved objects. Saved objects are files that store data, such as dashboards, visualizations, and maps, for later use.
- **[Advanced Settings]({{site.url}}{{site.baseurl}}/dashboards/management/advanced-settings/):** The Advanced Settings tool gives you the flexibility to personalize the behavior of OpenSearch Dashboards. The tool is divided into settings sections, such as General, Accessibility, and Notifications, and you can use it to customize and optimize many of your Dashboards settings.
28 changes: 14 additions & 14 deletions _dashboards/management/multi-data-sources.md
Original file line number Diff line number Diff line change
@@ -1,10 +1,9 @@
---
layout: default
title: Multiple data sources
parent: Dashboards Management
nav_order: 20
redirect_from:
- /dashboards/discover/multi-data-sources/
parent: Index patterns
grand_parent: Dashboards Management
nav_order: 10
---

# Multiple data sources
Expand All @@ -25,7 +24,7 @@ To enable multiple data sources:
4. Restart the Dashboards container.
5. Verify that the configuration settings were created and configured properly by connecting to Dashboards and viewing the **Dashboards Management** console. **Data Sources** appears in the sidebar, as shown in the following image.

<img src="{{site.url}}{{site.baseurl}}/images/dashboards/dashboards-mgmt.png" alt="Data sources sidebar on the Dashboards Management interface" width="40%">
<img src="{{site.url}}{{site.baseurl}}/images/dashboards/dashboards-mgmt.png" alt="Data sources sidebar on the Dashboards Management interface" width="700">

## Creating a data source connection

Expand Down Expand Up @@ -73,18 +72,19 @@ To delete the data source connection, select the trash can icon ({::nomarkdown}<

Use index patterns to access the OpenSearch data that you want to explore. Learn how to load your own data and create an index pattern in the following steps. This tutorial uses the preconfigured index pattern `opensearch_dashboards_sample_data_ecommerce Default`.

1. In the Dashboards console, select **Index Patterns** and then **Create index pattern**.
2. Select **Use external data source connection**.
3. Start typing in the **Search data sources** field to search for the data source you created earlier and then select the data source and **Next step**.
1. In the Dashboards console, choose **Index Patterns** and then **Create index pattern**.
2. Choose **Use external data source connection**.
3. Start typing in the **Search data sources** field to search for the data source you created earlier, select the data source, and then select **Next step**.
4. Add an **Index pattern name** to define the index pattern and then choose **Next step**.
5. Choose an option for the **Time field** and then select **Create index pattern**.
1. Go to OpenSearch Dashboards and select **Management** > **Dashboards Management** > **Index patterns**.
2. Select **Create index pattern**.
3. Select **Use external data source connection**.
4. Start typing in the **Search data sources** field to search for the data source you created earlier and then select the data source and **Next step**.
5. In the Dashboards console, choose **Index Patterns** and then **Create index pattern**.
6. Choose **Use external data source connection**.
7. Start typing in the **Search data sources** field to search for the data source you created earlier, select the data source, and then select **Next step**.
8. Add an **Index pattern name** to define the index pattern and then choose **Next step**.
9. Choose an option for the **Time field** and then select **Create index pattern**.

## Searching data

Before you start searching for data, set up the time filter. The sample index pattern used for this tutorial contains time-based data. You can set a time filter that displays only the data within a specified time range, and you can choose the time filter to change the time range or select a specific time range in the histogram.
The sample index pattern used for this tutorial contains time-based data. You can set a time filter that displays only the data within a specified time range, and you can choose the time filter to change the time range or select a specific time range in the histogram.

### Setting the time filter

Expand Down
Binary file added images/dashboards/create-index-pattern.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added images/dashboards/index-pattern-step2.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added images/dashboards/index-pattern-table.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added images/dashboards/index-patterns-step1.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added images/dashboards/index-patterns-ui.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.

0 comments on commit 035debb

Please sign in to comment.