Skip to content

Commit

Permalink
[DOC] Fix format issue and refresh content (opensearch-project#5477)
Browse files Browse the repository at this point in the history 
* Fix format issue and refresh content

---------

Signed-off-by: Melissa Vagi <[email protected]>
Co-authored-by: Heather Halter <[email protected]>
  • Loading branch information
@vagimeli @hdhalter @stephen-crawford
2 people authored and stephen-crawford committed Nov 3, 2023
1 parent 16cd0a2 commit 8f2d841
Show file tree
Hide file tree
Showing 4 changed files with 52 additions and 76 deletions.
128 changes: 52 additions & 76 deletions _dashboards/management/multi-data-sources.md
View file
Original file line number Diff line number Diff line change
@@ -1,117 +1,83 @@
---
layout: default
title: Connecting multiple data sources
title: Configuring and using multiple data sources
parent: Data sources
nav_order: 10
redirect_from:
- /dashboards/discover/multi-data-sources/
---

# Connecting multiple data sources
# Configuring and using multiple data sources

Learn how to enable the `data_source` setting in Dashboards; add credentials, data source connections, and index patterns; and combine visualizations in a single dashboard in this tutorial.
You can ingest, process, and analyze data from multiple data sources in OpenSearch Dashboards. You configure the data sources in the **Dashboards Management** > **Data sources** app, as shown in the following image.

## Modify the YAML file settings for multiple data sources

This tutorial uses a preconfigured data source and index pattern for which you aren’t required to configure settings. However, you have to enable the multiple data sources feature because it is disabled by default.
<img src="{{site.url}}{{site.baseurl}}/images/dashboards/data-sources-management.png" alt="Dashboards Management Data sources main screen" width="700">

To enable multiple data sources:
## Getting started

1. Navigate to your Dashboards home directory, for example, in Docker, `/usr/share/opensearch-dashboards/config`.
2. Open your local copy of the Dashboards configuration file, `opensearch_dashboards.yml`. If you don't have a copy, [`opensearch_dashboards.yml`](https://github.com/opensearch-project/OpenSearch-Dashboards/blob/main/config/opensearch_dashboards.yml) is available on GitHub.
3. Set `data_source.enabled:` to `true` and save the YAML file.
4. Restart the Dashboards container.
5. Verify that the configuration settings were created and configured properly by connecting to Dashboards and viewing the **Dashboards Management** navigation menu. **Data sources** appears in the sidebar, as shown in the following image.
The following tutorial guides you through configuring and using multiple data sources.

<img src="{{site.url}}{{site.baseurl}}/images/dashboards/dashboards-mgmt.png" alt="Data sources sidebar on the Dashboards Management interface" width="700">
### Step 1: Modify the YAML file settings

## Select multiple data sources by using the Dev Tools console
To use multiple data sources, you must enable the `data_source.enabled` setting. It is disabled by default. To enable multiple data sources:

<<<<<<< HEAD
<<<<<<< HEAD
A data source connection specifies the parameters needed to connect to a data source. These parameters form a connection string for the data source. In Dashboards, you can add new data source connections or edit existing connections.
1. Open your local copy of the OpenSearch Dashboards configuration file, `opensearch_dashboards.yml`. If you don't have a copy, [`opensearch_dashboards.yml`](https://github.com/opensearch-project/OpenSearch-Dashboards/blob/main/config/opensearch_dashboards.yml) is available on GitHub.
2. Set `data_source.enabled:` to `true` and save the YAML file.
3. Restart the OpenSearch Dashboards container.
4. Verify that the configuration settings were configured properly by connecting to OpenSearch Dashboards and viewing the **Dashboards Management** navigation menu. **Data sources** appears in the sidebar. You'll see a view similar to the following image.

To create a new data source connection:
<img src="{{site.url}}{{site.baseurl}}/images/dashboards/multidatasources.png" alt="Data sources in sidebar within Dashboards Management" width="700">

1. Go to [`http://localhost:5601`](http://localhost:5601/) and log in with the username `admin` and password `admin`. If you’re running the Security plugin, go to [`https://localhost:5601`](https://localhost:5601/).
2. From the OpenSearch Dashboards main menu, select **Dashboards Management** > **Data sources** > **Create data source connection**.
3. Add information to each field to configure **Connection Details** and **Authentication Method**.

Under **Connection Details**, enter a title and endpoint URL. For this tutorial, use the URL `http://localhost:5601/app/management/opensearch-dashboards/dataSources`. Entering a description is optional.
### Step 2: Create a new data source connection

Under **Authentication Method**, select an authentication method from the dropdown list. Once an authentication method is selected, the applicable fields for that method appear. You can then enter the required details. The authentication method options are:
- **No authentication**: No authentication is used to connect to the data source.
- **Username & Password**: A basic username and password are used to connect to the data source.
- **AWS SigV4**: An AWS Signature Version 4 authenticating request is used to connect to the data source. AWS Signature Version 4 requires an access key and a secret key.
A data source connection specifies the parameters needed to connect to a data source. These parameters form a connection string for the data source.

For AWS Signature Version 4 authentication, first specify the **Region**. Next, select the OpenSearch service in the **Service Name** list. The options are **Amazon OpenSearch Service** and **Amazon OpenSearch Serverless**. Last, enter the **Access Key** and **Secret Key** for authorization. For an example setup, see the following image.
To create a new data source connection:

1. From the OpenSearch Dashboards main menu, select **Dashboards Management** > **Data sources** > **Create data source connection**.
2. Add the required information to each field to configure **Connection Details** and **Authentication Method**.

- Under **Connection Details**, enter a title and endpoint URL. For this tutorial, use the URL `http://localhost:5601/app/management/opensearch-dashboards/dataSources`. Entering a description is optional.

<img src="{{site.url}}{{site.baseurl}}/images/dashboards/SigV4-setup.png" alt="AWS Signature Version 4 auth type setup" width="40%">
- Under **Authentication Method**, select an authentication method from the dropdown list. Once an authentication method is selected, the applicable fields for that method appear. You can then enter the required details. The authentication method options are:
- **No authentication**: No authentication is used to connect to the data source.
- **Username & Password**: A basic username and password are used to connect to the data source.
- **AWS SigV4**: An AWS Signature Version 4 authenticating request is used to connect to the data source. AWS Signature Version 4 requires an access key and a secret key.
- For AWS Signature Version 4 authentication, first specify the **Region**. Next, select the OpenSearch service in the **Service Name** list. The options are **Amazon OpenSearch Service** and **Amazon OpenSearch Serverless**. Last, enter the **Access Key** and **Secret Key** for authorization.

For information about available AWS Regions for AWS accounts, see [Available Regions](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-regions-availability-zones.html#concepts-available-regions). For more information about AWS Signature Version 4 authentication requests, see [Authenticating Requests (AWS Signature Version 4)](https://docs.aws.amazon.com/AmazonS3/latest/API/sig-v4-authenticating-requests.html).
{: .note }

After you have entered the appropriate details in all of the required fields, the **Test connection** and **Create data source** buttons become active. You can select **Test connection** to confirm that the connection is valid.

4. Select **Create data source** to save your settings. The connection is created. The active window returns to the **Data Sources** main page, and the new connection appears in the list of data sources.

5. Delete the data source connection by selecting the check box to the left of the title and then choosing **Delete 1 connection**. Selecting multiple check boxes for multiple connections is supported.

### Editing and updating a data source connection
{: .note}

To make changes to the data source connection, select a connection in the list on the **Data Sources** main page. The **Connection Details** window opens.
- After you have entered the appropriate details in all of the required fields, the **Test connection** and **Create data source** buttons become active. You can select **Test connection** to confirm that the connection is valid.

To make changes to **Connection Details**, edit one or both of the **Title** and **Description** fields and select **Save changes** in the lower-right corner of the screen. You can also cancel changes here. To change the **Authentication Method**, choose a different authentication method, enter your credentials (if applicable), and then select **Save changes** in the lower-right corner of the screen. The changes are saved.
3. Select **Create data source** to save your settings. The connection is created. The active window returns to the **Data Sources** main page, and the new connection appears in the list of data sources.

When **Username & Password** is the selected authentication method, you can update the password by choosing **Update stored password** next to the **Password** field. In the pop-up window, enter a new password in the first field and then enter it again in the second field to confirm. Select **Update stored password** in the pop-up window. The new password is saved. Select **Test connection** to confirm that the connection is valid.
4. Edit or update a data source connection.

When **AWS SigV4** is the selected authentication method, you can update the credentials by selecting **Update stored AWS credential**. In the pop-up window, enter a new access key in the first field and a new secret key in the second field. Select **Update stored AWS credential** in the pop-up window. The new credentials are saved. Select **Test connection** in the upper-right corner of the screen to confirm that the connection is valid.
- To make changes to the data source connection, select a connection in the list on the **Data Sources** main page. The **Connection Details** window opens.

To delete the data source connection, select the trash can icon ({::nomarkdown}<img src="{{site.url}}{{site.baseurl}}/images/dashboards/trash-can-icon.png" class="inline-icon" alt="trash can icon"/>{:/}).
- To make changes to **Connection Details**, edit one or both of the **Title** and **Description** fields and select **Save changes** in the lower-right corner of the screen. You can also cancel changes here. To change the **Authentication Method**, choose a different authentication method, enter your credentials (if applicable), and then select **Save changes** in the lower-right corner of the screen. The changes are saved.

## Creating an index pattern
- When **Username & Password** is the selected authentication method, you can update the password by choosing **Update stored password** next to the **Password** field. In the pop-up window, enter a new password in the first field and then enter it again in the second field to confirm. Select **Update stored password** in the pop-up window. The new password is saved. Select **Test connection** to confirm that the connection is valid.

Use index patterns to access the OpenSearch data that you want to explore. Learn how to load your own data and create an index pattern in the following steps. This tutorial uses the preconfigured index pattern `opensearch_dashboards_sample_data_ecommerce Default`.
- When **AWS SigV4** is the selected authentication method, you can update the credentials by selecting **Update stored AWS credential**. In the pop-up window, enter a new access key in the first field and a new secret key in the second field. Select **Update stored AWS credential** in the pop-up window. The new credentials are saved. Select **Test connection** in the upper-right corner of the screen to confirm that the connection is valid.

1. Go to OpenSearch Dashboards and select **Management** > **Dashboards Management** > **Index patterns**.
2. Select **Create index pattern**.
3. Select **Use external data source connection**.
4. Start typing in the **Search data sources** field to search for the data source you created earlier and then select the data source and **Next step**.
5. In the Dashboards console, choose **Index Patterns** and then **Create index pattern**.
6. Choose **Use external data source connection**.
7. Start typing in the **Search data sources** field to search for the data source you created earlier, select the data source, and then select **Next step**.
8. Add an **Index pattern name** to define the index pattern and then choose **Next step**.
9. Choose an option for the **Time field** and then select **Create index pattern**.
5. Delete the data source connection by selecting the check box to the left of the title and then choosing **Delete 1 connection**. Selecting multiple check boxes for multiple connections is supported. Alternatively, select the trash can icon ({::nomarkdown}<img src="{{site.url}}{{site.baseurl}}/images/dashboards/trash-can-icon.png" class="inline-icon" alt="trash can icon"/>{:/}).

## Searching data
An example data source connection screen is shown in the following image.

The sample index pattern used for this tutorial contains time-based data. You can set a time filter that displays only the data within a specified time range, and you can choose the time filter to change the time range or select a specific time range in the histogram.
<img src="{{site.url}}{{site.baseurl}}/images/dashboards/data-source-connection.png" alt="Data source connection screen" width="700">

### Setting the time filter
### Selecting multiple data sources through the Dev Tools console

To set the time filter:
Alternatively, you can select multiple data sources through the [Dev Tools]({{site.url}}{{site.baseurl}}/dashboards/dev-tools/index-dev/) console. This option provides for working with a broader range of data and gaining deeper insight into your code and applications.

1. In the Dashboards console, select **Discover** and confirm that the index pattern being used is `opensearch_dashboards_sample_data_ecommerce`.
2. Select the calendar icon ({::nomarkdown}<img src="{{site.url}}{{site.baseurl}}/images/icons/calendar-icon.png" class="inline-icon" alt="calendar icon"/>{:/}) to change the time field. The default time period is **Last 15 minutes**.
3. Change the time field to a particular time period, for example, **Last 7 days**, and then select **Refresh**.
4. Change start or end times by selecting the start or end time in the search bar.
5. In the pop-up window, choose **Absolute**, **Relative**, or **Now** and then specify the date.

### Selecting a time range from the histogram

To select a time range for the histogram, choose from the following options:

* Select the bar that represents the time range you want to zoom in on.
* Select the bar and drag to view a specific time range. You must start the selection with the cursor over the background of the chart (the cursor changes to a plus sign when you hover over a valid start point).
* Select the dropdown and then select an interval.

## Selecting multiple data sources in the Dev Tools console

Selecting multiple data sources in the Dev Tools console allows you to work with a broader range of data and gain deeper insight into your code and applications. Watch the video to see it in action, and then try it out in the following steps.
Watch the following 10-second video to see it in action.

<img src="{{site.url}}{{site.baseurl}}/images/dashboards/multidata-dev-tools.gif" alt="Multiple data sources in Dev Tools demo">{: .img-fluid}

To select a data source by using the Dev Tools console, follow these steps:
To select a data source through the Dev Tools console, follow these steps:

1. Locate your copy of `opensearch_dashboards.yml` and open it in the editor of your choice.
2. Set `data_source.enabled` to `true`.
Expand All @@ -121,13 +87,23 @@ To select a data source by using the Dev Tools console, follow these steps:
```json
GET /_cat/indices
```
{% include copy-curl.html %}

5. From the **DataSource** dropdown menu, select a data source and then query the source.
6. Repeat the preceding steps for each data source you want to select.

## Next steps

Once you've configured your multiple data sources, you can start exploring that data. See the following resources to learn more:

- Learn about [managing index patterns]({{site.url}}{{site.baseurl}}/dashboards/management/index-patterns/) through OpenSearch Dashboards.
- Learn about [indexing data using Index Management]({{site.url}}{{site.baseurl}}/dashboards/im-dashboards/index/) through OpenSearch Dashboards.
- Learn about how to [connect OpenSearch and Amazon S3 through OpenSearch Dashboards]({{site.url}}{{site.baseurl}}/dashboards/management/S3-data-source/).
- Learn about the [Integrations tool]({{site.url}}{{site.baseurl}}/integrations/index/), which gives you the flexibility to use various data ingestion methods and connect data from the Dashboards UI.

## Limitations

This feature is still under development, so there are some limitations:
This feature has some limitations:

* The multiple data sources feature is supported for index-pattern-based visualizations only.
* The visualization types Time Series Visual Builder (TSVB), Vega and Vega-Lite, and timeline are not supported.
Expand Down
Binary file added images/dashboards/data-source-connection.png
View file
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added images/dashboards/data-sources-management.png
View file
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added images/dashboards/multidatasources.png
View file
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.

0 comments on commit 8f2d841

Please sign in to comment.