Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Data Explorer] Readme #5273

Merged
merged 3 commits into from
Dec 14, 2023
Merged
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
79 changes: 72 additions & 7 deletions src/plugins/data_explorer/README.md
Original file line number Diff line number Diff line change
@@ -1,11 +1,76 @@
# dataExplorer
# Data Explorer

A OpenSearch Dashboards plugin
## Overview

---
Data Explorer is an integral enhancement of the OpenSearch Dashboards that seeks to consolidate various data exploration facets into a unified platform. Built to provide an efficient data exploration experience, Data Explorer merges capabilities of different applications like Discover, Visbuilder, and Event Analytics into a singular platform.

## Key Features

1. **Unified Data Exploration**: Data Explorer acts as a consolidated platform for all data exploration tasks, aiming to provide users with a seamless and efficient environment.
2. **Extensibility**: Provides an architecture that allows existing exploration apps to migrate with minimal changes.
3. **Shared Utilities**: Offers components and utilities that can be shared across different views.

## Architecture and Integration

Data Explorer, at its core, is a shell for data exploration views. Here's a breakdown of the architecture and how it manages responsibilities:

### Data Explorer Responsibilities:

1. **Data Source**: Acts as the central point for the data source being explored.
2. **View Registry**: Allows apps to register themselves as views and manages their display based on user selection.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For now it seems to have many data source , but stands for different meanings:

  1. multi datasources: enable OSD to connect to different OS clusters by specifying endpoint url and credentials.
  2. data source: the index pattern or index for retrieving data.

Shall we align on unified terms on this?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

I think we could have a terminology reference doc in the repo.

  • Multi-Datasources (OpenSearch Clusters):
    Term: "Connected Clusters" or "Cluster Sources"
    Description: Use this term to refer specifically to the ability of OSD to connect to multiple OpenSearch clusters. Each "Connected Cluster" or "Cluster Source" represents a distinct OpenSearch cluster, identifiable by its endpoint URL and credentials. This feature allows OSD to retrieve and manage data from multiple clusters within the same OSD instance.

  • Data Source (Index Patterns and Indices):
    Term: "Index Patterns"
    Description: This term should be used when referring to the specific index patterns or indices in OpenSearch that are used for data retrieval. "Data Indexes" or "Index Patterns" are the structures or patterns within an individual OpenSearch cluster that OSD queries to fetch and visualize data.

But may be not in this PR. I will approve this PR and we could start discussing on having a terminology reference.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Data source here is meant to talk about all data sources, including multi data sources, and even non OpenSearch data sources.

"Data sources: indexes/index patterns" is only for now but it's going to change soon, which is why the naming of the drop downs was changed in anticipation of that.

3. **State Management**: Provides shared state management and hooks for underlying apps to register their state reducers.
4. **Shared Utilities**: Contains components and utilities that can be reused by various views.

### View Responsibilities:

1. **Metadata Storage**: Handles the logic for storing metadata and its retrieval (Saved objects).
2. **Data Fetching**: Manages the logic for fetching data from the data source.
3. **View specific state management**: Handles view-specific state management and hooks into Data Explorer's state management.
4. **Nav Options & Search/Query Bar**: Manages the navigation options, time filter, and search bar.
5. **View Specific Logic**: Contains view-specific rendering and application logic.
6. **Embeddables**: Responsible for registering their embeddables.

## Development
### Migrating Existing Applications:

Existing applications can migrate their data exploration views to Data Explorer. Such migrations involve:

1. Registering the application as a view.
2. Using Data Explorer's state management and data source.
3. Modifying routes to utilize Data Explorer's routes.
4. Adapting the UI to match Data Explorer's panel and workspace components.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we change workspace to canvas?

#### Routing:

Existing routes for each view are expected to redirect to new routes prefixed with `/data_explorer`. E.g., existing Discover route will redirect to `/data_explorer/discover`.

### View Registry:

For an application to be registered as a view within Data Explorer, it needs to adhere to the following data model:

```ts
interface ViewDefinition<T = any> {
readonly id: string;
readonly title: string;
readonly ui?: {
defaults: DefaultViewState | (() => DefaultViewState) | (() => Promise<DefaultViewState>);
slice: Slice<T>;
};
readonly Canvas: LazyExoticComponent<(props: ViewProps) => React.ReactElement>;
readonly Panel: LazyExoticComponent<(props: ViewProps) => React.ReactElement>;
readonly Context: LazyExoticComponent<
(props: React.PropsWithChildren<ViewProps>) => React.ReactElement
>;
readonly defaultPath: string;
readonly appExtentions: {
savedObject: {
docTypes: [string];
toListItem: (obj: { id: string; title: string }) => ViewListItem;
};
};
readonly shouldShow?: (state: any) => boolean;
}
```

---

See the [OpenSearch Dashboards contributing
guide](https://github.com/opensearch-project/OpenSearch-Dashboards/blob/main/CONTRIBUTING.md) for instructions
setting up your development environment.
Original proposal: https://github.com/opensearch-project/OpenSearch-Dashboards/issues/4165
Loading