Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[DOC] Fix format issue and refresh content #5477

Merged
merged 13 commits into from
Nov 2, 2023
108 changes: 49 additions & 59 deletions _dashboards/management/multi-data-sources.md
Original file line number Diff line number Diff line change
@@ -1,93 +1,82 @@
---
layout: default
title: Connecting multiple data sources
title: Configuring and using multiple data sources
parent: Data sources
nav_order: 10
redirect_from:
- /dashboards/discover/multi-data-sources/
---

# Connecting multiple data sources
# Configuring and using multiple data sources

Check failure on line 10 in _dashboards/management/multi-data-sources.md

View workflow job for this annotation

GitHub Actions / vale

[vale] _dashboards/management/multi-data-sources.md#L10

[OpenSearch.HeadingCapitalization] 'Configuring and using multiple data sources' is a heading and should be in sentence case.
Raw output
{"message": "[OpenSearch.HeadingCapitalization] 'Configuring and using multiple data sources' is a heading and should be in sentence case.", "location": {"path": "_dashboards/management/multi-data-sources.md", "range": {"start": {"line": 10, "column": 3}}}, "severity": "ERROR"}

Learn how to enable the `data_source` setting in Dashboards; add credentials, data source connections, and index patterns; and combine visualizations in a single dashboard in this tutorial.
The multiple data sources feature in OpenSearch Dashboards provides for the capability to ingest, process, and analyze data from a variety of sources. This feature is configured within the **Dashboards Management** > **Data sources** app.
vagimeli marked this conversation as resolved.
Show resolved Hide resolved

## Modify the YAML file settings for multiple data sources
A view of the main screen is shown in the following image.
vagimeli marked this conversation as resolved.
Show resolved Hide resolved

This tutorial uses a preconfigured data source and index pattern for which you aren’t required to configure settings. However, you have to enable the multiple data sources feature because it is disabled by default.
<img src="{{site.url}}{{site.baseurl}}/images/dashboards/data-sources-management.png" alt="Dashboards Management Data sources main screen" width="700">

To enable multiple data sources:
## Getting started

1. Navigate to your Dashboards home directory, for example, in Docker, `/usr/share/opensearch-dashboards/config`.
2. Open your local copy of the Dashboards configuration file, `opensearch_dashboards.yml`. If you don't have a copy, [`opensearch_dashboards.yml`](https://github.com/opensearch-project/OpenSearch-Dashboards/blob/main/config/opensearch_dashboards.yml) is available on GitHub.
3. Set `data_source.enabled:` to `true` and save the YAML file.
4. Restart the Dashboards container.
5. Verify that the configuration settings were created and configured properly by connecting to Dashboards and viewing the **Dashboards Management** navigation menu. **Data sources** appears in the sidebar, as shown in the following image.
The following tutorial guides you through configuring and using multiple data sources.

<img src="{{site.url}}{{site.baseurl}}/images/dashboards/dashboards-mgmt.png" alt="Data sources sidebar on the Dashboards Management interface" width="700">
### Step 1: Modify the YAML file settings

Check failure on line 22 in _dashboards/management/multi-data-sources.md

View workflow job for this annotation

GitHub Actions / vale

[vale] _dashboards/management/multi-data-sources.md#L22

[OpenSearch.HeadingCapitalization] 'Step 1: Modify the YAML file settings' is a heading and should be in sentence case.
Raw output
{"message": "[OpenSearch.HeadingCapitalization] 'Step 1: Modify the YAML file settings' is a heading and should be in sentence case.", "location": {"path": "_dashboards/management/multi-data-sources.md", "range": {"start": {"line": 22, "column": 5}}}, "severity": "ERROR"}

## Select multiple data sources by using the Dev Tools console
To use the multiple data sources feature, the setting must be enabled. It is disabled by default. To enable multiple data sources:
vagimeli marked this conversation as resolved.
Show resolved Hide resolved

<<<<<<< HEAD
<<<<<<< HEAD
A data source connection specifies the parameters needed to connect to a data source. These parameters form a connection string for the data source. In Dashboards, you can add new data source connections or edit existing connections.
1. Open your local copy of the OpenSearch Dashboards configuration file, `opensearch_dashboards.yml`. If you don't have a copy, [`opensearch_dashboards.yml`](https://github.com/opensearch-project/OpenSearch-Dashboards/blob/main/config/opensearch_dashboards.yml) is available on GitHub.
2. Set `data_source.enabled:` to `true` and save the YAML file.
3. Restart the OpenSearch Dashboards container.
4. Verify that the configuration settings were configured properly by connecting to OpenSearch Dashboards and viewing the **Dashboards Management** navigation menu. **Data sources** appears in the sidebar. You'll see a view similar to the following image.

To create a new data source connection:
<img src="{{site.url}}{{site.baseurl}}/images/dashboards/multidatasources.png" alt="Data sources in sidebar within Dashboards Management" width="700">

1. Go to [`http://localhost:5601`](http://localhost:5601/) and log in with the username `admin` and password `admin`. If you’re running the Security plugin, go to [`https://localhost:5601`](https://localhost:5601/).
2. From the OpenSearch Dashboards main menu, select **Dashboards Management** > **Data sources** > **Create data source connection**.
3. Add information to each field to configure **Connection Details** and **Authentication Method**.

Under **Connection Details**, enter a title and endpoint URL. For this tutorial, use the URL `http://localhost:5601/app/management/opensearch-dashboards/dataSources`. Entering a description is optional.
### Step 2: Create a new data source connection

Under **Authentication Method**, select an authentication method from the dropdown list. Once an authentication method is selected, the applicable fields for that method appear. You can then enter the required details. The authentication method options are:
- **No authentication**: No authentication is used to connect to the data source.
- **Username & Password**: A basic username and password are used to connect to the data source.
- **AWS SigV4**: An AWS Signature Version 4 authenticating request is used to connect to the data source. AWS Signature Version 4 requires an access key and a secret key.
A data source connection specifies the parameters needed to connect to a data source. These parameters form a connection string for the data source.

To create a new data source connection:

For AWS Signature Version 4 authentication, first specify the **Region**. Next, select the OpenSearch service in the **Service Name** list. The options are **Amazon OpenSearch Service** and **Amazon OpenSearch Serverless**. Last, enter the **Access Key** and **Secret Key** for authorization. For an example setup, see the following image.
1. From the OpenSearch Dashboards main menu, select **Dashboards Management** > **Data sources** > **Create data source connection**.
2. Add the required information to each field to configure **Connection Details** and **Authentication Method**.

- Under **Connection Details**, enter a title and endpoint URL. For this tutorial, use the URL `http://localhost:5601/app/management/opensearch-dashboards/dataSources`. Entering a description is optional.

<img src="{{site.url}}{{site.baseurl}}/images/dashboards/SigV4-setup.png" alt="AWS Signature Version 4 auth type setup" width="40%">
- Under **Authentication Method**, select an authentication method from the dropdown list. Once an authentication method is selected, the applicable fields for that method appear. You can then enter the required details. The authentication method options are:
- **No authentication**: No authentication is used to connect to the data source.
- **Username & Password**: A basic username and password are used to connect to the data source.

Check warning on line 46 in _dashboards/management/multi-data-sources.md

View workflow job for this annotation

GitHub Actions / vale

[vale] _dashboards/management/multi-data-sources.md#L46

[OpenSearch.Ampersand] Don't use an ampersand in place of 'and' in documentation.
Raw output
{"message": "[OpenSearch.Ampersand] Don't use an ampersand in place of 'and' in documentation.", "location": {"path": "_dashboards/management/multi-data-sources.md", "range": {"start": {"line": 46, "column": 18}}}, "severity": "WARNING"}
- **AWS SigV4**: An AWS Signature Version 4 authenticating request is used to connect to the data source. AWS Signature Version 4 requires an access key and a secret key.

Check warning on line 47 in _dashboards/management/multi-data-sources.md

View workflow job for this annotation

GitHub Actions / vale

[vale] _dashboards/management/multi-data-sources.md#L47

[OpenSearch.SignatureV4] 'AWS Signature Version 4': Use 'AWS Signature Version 4' instead of 'AWS SigV4' on first appearance. Then, Signature Version 4 may be used. Only use SigV4 when space is limited.
Raw output
{"message": "[OpenSearch.SignatureV4] 'AWS Signature Version 4': Use 'AWS Signature Version 4' instead of 'AWS SigV4' on first appearance. Then, Signature Version 4 may be used. Only use SigV4 when space is limited.", "location": {"path": "_dashboards/management/multi-data-sources.md", "range": {"start": {"line": 47, "column": 11}}}, "severity": "WARNING"}
- For AWS Signature Version 4 authentication, first specify the **Region**. Next, select the OpenSearch service in the **Service Name** list. The options are **Amazon OpenSearch Service** and **Amazon OpenSearch Serverless**. Last, enter the **Access Key** and **Secret Key** for authorization.

For information about available AWS Regions for AWS accounts, see [Available Regions](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-regions-availability-zones.html#concepts-available-regions). For more information about AWS Signature Version 4 authentication requests, see [Authenticating Requests (AWS Signature Version 4)](https://docs.aws.amazon.com/AmazonS3/latest/API/sig-v4-authenticating-requests.html).
{: .note }

After you have entered the appropriate details in all of the required fields, the **Test connection** and **Create data source** buttons become active. You can select **Test connection** to confirm that the connection is valid.
{: .note}

4. Select **Create data source** to save your settings. The connection is created. The active window returns to the **Data Sources** main page, and the new connection appears in the list of data sources.
- After you have entered the appropriate details in all of the required fields, the **Test connection** and **Create data source** buttons become active. You can select **Test connection** to confirm that the connection is valid.

5. Delete the data source connection by selecting the check box to the left of the title and then choosing **Delete 1 connection**. Selecting multiple check boxes for multiple connections is supported.
3. Select **Create data source** to save your settings. The connection is created. The active window returns to the **Data Sources** main page, and the new connection appears in the list of data sources.

### Editing and updating a data source connection
4. Edit or update a data source connection.

To make changes to the data source connection, select a connection in the list on the **Data Sources** main page. The **Connection Details** window opens.
- To make changes to the data source connection, select a connection in the list on the **Data Sources** main page. The **Connection Details** window opens.

To make changes to **Connection Details**, edit one or both of the **Title** and **Description** fields and select **Save changes** in the lower-right corner of the screen. You can also cancel changes here. To change the **Authentication Method**, choose a different authentication method, enter your credentials (if applicable), and then select **Save changes** in the lower-right corner of the screen. The changes are saved.
- To make changes to **Connection Details**, edit one or both of the **Title** and **Description** fields and select **Save changes** in the lower-right corner of the screen. You can also cancel changes here. To change the **Authentication Method**, choose a different authentication method, enter your credentials (if applicable), and then select **Save changes** in the lower-right corner of the screen. The changes are saved.

When **Username & Password** is the selected authentication method, you can update the password by choosing **Update stored password** next to the **Password** field. In the pop-up window, enter a new password in the first field and then enter it again in the second field to confirm. Select **Update stored password** in the pop-up window. The new password is saved. Select **Test connection** to confirm that the connection is valid.
- When **Username & Password** is the selected authentication method, you can update the password by choosing **Update stored password** next to the **Password** field. In the pop-up window, enter a new password in the first field and then enter it again in the second field to confirm. Select **Update stored password** in the pop-up window. The new password is saved. Select **Test connection** to confirm that the connection is valid.

Check warning on line 63 in _dashboards/management/multi-data-sources.md

View workflow job for this annotation

GitHub Actions / vale

[vale] _dashboards/management/multi-data-sources.md#L63

[OpenSearch.Ampersand] Don't use an ampersand in place of 'and' in documentation.
Raw output
{"message": "[OpenSearch.Ampersand] Don't use an ampersand in place of 'and' in documentation.", "location": {"path": "_dashboards/management/multi-data-sources.md", "range": {"start": {"line": 63, "column": 25}}}, "severity": "WARNING"}

When **AWS SigV4** is the selected authentication method, you can update the credentials by selecting **Update stored AWS credential**. In the pop-up window, enter a new access key in the first field and a new secret key in the second field. Select **Update stored AWS credential** in the pop-up window. The new credentials are saved. Select **Test connection** in the upper-right corner of the screen to confirm that the connection is valid.
- When **AWS SigV4** is the selected authentication method, you can update the credentials by selecting **Update stored AWS credential**. In the pop-up window, enter a new access key in the first field and a new secret key in the second field. Select **Update stored AWS credential** in the pop-up window. The new credentials are saved. Select **Test connection** in the upper-right corner of the screen to confirm that the connection is valid.

Check warning on line 65 in _dashboards/management/multi-data-sources.md

View workflow job for this annotation

GitHub Actions / vale

[vale] _dashboards/management/multi-data-sources.md#L65

[OpenSearch.SignatureV4] 'AWS Signature Version 4': Use 'AWS Signature Version 4' instead of 'AWS SigV4' on first appearance. Then, Signature Version 4 may be used. Only use SigV4 when space is limited.
Raw output
{"message": "[OpenSearch.SignatureV4] 'AWS Signature Version 4': Use 'AWS Signature Version 4' instead of 'AWS SigV4' on first appearance. Then, Signature Version 4 may be used. Only use SigV4 when space is limited.", "location": {"path": "_dashboards/management/multi-data-sources.md", "range": {"start": {"line": 65, "column": 18}}}, "severity": "WARNING"}

To delete the data source connection, select the trash can icon ({::nomarkdown}<img src="{{site.url}}{{site.baseurl}}/images/dashboards/trash-can-icon.png" class="inline-icon" alt="trash can icon"/>{:/}).
5. Delete the data source connection by selecting the check box to the left of the title and then choosing **Delete 1 connection**. Selecting multiple check boxes for multiple connections is supported. Alternatively, select the trash can icon ({::nomarkdown}<img src="{{site.url}}{{site.baseurl}}/images/dashboards/trash-can-icon.png" class="inline-icon" alt="trash can icon"/>{:/}).

## Creating an index pattern
An example data source connection is shown in the following image.

Use index patterns to access the OpenSearch data that you want to explore. Learn how to load your own data and create an index pattern in the following steps. This tutorial uses the preconfigured index pattern `opensearch_dashboards_sample_data_ecommerce Default`.
<img src="{{site.url}}{{site.baseurl}}/images/dashboards/data-source-connection.png" alt="Data source connection screen" width="700">

1. Go to OpenSearch Dashboards and select **Management** > **Dashboards Management** > **Index patterns**.
2. Select **Create index pattern**.
3. Select **Use external data source connection**.
4. Start typing in the **Search data sources** field to search for the data source you created earlier and then select the data source and **Next step**.
5. In the Dashboards console, choose **Index Patterns** and then **Create index pattern**.
6. Choose **Use external data source connection**.
7. Start typing in the **Search data sources** field to search for the data source you created earlier, select the data source, and then select **Next step**.
8. Add an **Index pattern name** to define the index pattern and then choose **Next step**.
9. Choose an option for the **Time field** and then select **Create index pattern**.
### Step 3: Creating an index pattern

## Searching data
If you are unfamiliar with creating index patterns within OpenSearch Dashboards, see [Index patterns]({{site.url}}{{site.baseurl}}/dashboards/management/index-patterns/).

The sample index pattern used for this tutorial contains time-based data. You can set a time filter that displays only the data within a specified time range, and you can choose the time filter to change the time range or select a specific time range in the histogram.
### Step 4: Searching the data

### Setting the time filter
You can set a time filter that displays only the data within a specified time range, and you can choose the time filter to change the time range or select a specific time range in the histogram.

To set the time filter:

Expand All @@ -97,21 +86,21 @@
4. Change start or end times by selecting the start or end time in the search bar.
5. In the pop-up window, choose **Absolute**, **Relative**, or **Now** and then specify the date.

### Selecting a time range from the histogram

To select a time range for the histogram, choose from the following options:

* Select the bar that represents the time range you want to zoom in on.
* Select the bar and drag to view a specific time range. You must start the selection with the cursor over the background of the chart (the cursor changes to a plus sign when you hover over a valid start point).
* Select the dropdown and then select an interval.

## Selecting multiple data sources in the Dev Tools console
## Selecting multiple data sources through the Dev Tools console

Alternatively, you can select multiple data sources through the Dev Tools console. This option provides for working with a broader range of data and gaining deeper insight into your code and applications.

Selecting multiple data sources in the Dev Tools console allows you to work with a broader range of data and gain deeper insight into your code and applications. Watch the video to see it in action, and then try it out in the following steps.
Watch the following 10-second video to see it in action.

<img src="{{site.url}}{{site.baseurl}}/images/dashboards/multidata-dev-tools.gif" alt="Multiple data sources in Dev Tools demo">{: .img-fluid}

To select a data source by using the Dev Tools console, follow these steps:
To select a data source through the Dev Tools console, follow these steps:

1. Locate your copy of `opensearch_dashboards.yml` and open it in the editor of your choice.
2. Set `data_source.enabled` to `true`.
Expand All @@ -121,13 +110,14 @@
```json
GET /_cat/indices
```
{% include copy-curl.html %}

5. From the **DataSource** dropdown menu, select a data source and then query the source.
6. Repeat the preceding steps for each data source you want to select.

## Limitations

This feature is still under development, so there are some limitations:
This feature has some limitations:

* The multiple data sources feature is supported for index-pattern-based visualizations only.
* The visualization types Time Series Visual Builder (TSVB), Vega and Vega-Lite, and timeline are not supported.
Expand Down
Binary file added images/dashboards/data-source-connection.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added images/dashboards/data-sources-management.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added images/dashboards/multidatasources.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.