-
Notifications
You must be signed in to change notification settings - Fork 916
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Data Explorer] Readme #5273
Merged
ananzh
merged 3 commits into
opensearch-project:main
from
ashwin-pc:chore/data-explorer-docs
Dec 14, 2023
Merged
[Data Explorer] Readme #5273
Changes from 1 commit
Commits
Show all changes
3 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,11 +1,76 @@ | ||
# dataExplorer | ||
# Data Explorer | ||
|
||
A OpenSearch Dashboards plugin | ||
## Overview | ||
|
||
--- | ||
Data Explorer is an integral enhancement of the OpenSearch Dashboards that seeks to consolidate various data exploration facets into a unified platform. Built to provide an efficient data exploration experience, Data Explorer merges capabilities of different applications like Discover, Visbuilder, and Event Analytics into a singular platform. | ||
|
||
## Key Features | ||
|
||
1. **Unified Data Exploration**: Data Explorer acts as a consolidated platform for all data exploration tasks, aiming to provide users with a seamless and efficient environment. | ||
2. **Extensibility**: Provides an architecture that allows existing exploration apps to migrate with minimal changes. | ||
3. **Shared Utilities**: Offers components and utilities that can be shared across different views. | ||
|
||
## Architecture and Integration | ||
|
||
Data Explorer, at its core, is a shell for data exploration views. Here's a breakdown of the architecture and how it manages responsibilities: | ||
|
||
### Data Explorer Responsibilities: | ||
|
||
1. **Data Source**: Acts as the central point for the data source being explored. | ||
2. **View Registry**: Allows apps to register themselves as views and manages their display based on user selection. | ||
3. **State Management**: Provides shared state management and hooks for underlying apps to register their state reducers. | ||
4. **Shared Utilities**: Contains components and utilities that can be reused by various views. | ||
|
||
### View Responsibilities: | ||
|
||
1. **Metadata Storage**: Handles the logic for storing metadata and its retrieval (Saved objects). | ||
2. **Data Fetching**: Manages the logic for fetching data from the data source. | ||
3. **View specific state management**: Handles view-specific state management and hooks into Data Explorer's state management. | ||
4. **Nav Options & Search/Query Bar**: Manages the navigation options, time filter, and search bar. | ||
5. **View Specific Logic**: Contains view-specific rendering and application logic. | ||
6. **Embeddables**: Responsible for registering their embeddables. | ||
|
||
## Development | ||
### Migrating Existing Applications: | ||
|
||
Existing applications can migrate their data exploration views to Data Explorer. Such migrations involve: | ||
|
||
1. Registering the application as a view. | ||
2. Using Data Explorer's state management and data source. | ||
3. Modifying routes to utilize Data Explorer's routes. | ||
4. Adapting the UI to match Data Explorer's panel and workspace components. | ||
|
||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. should we change |
||
#### Routing: | ||
|
||
Existing routes for each view are expected to redirect to new routes prefixed with `/data_explorer`. E.g., existing Discover route will redirect to `/data_explorer/discover`. | ||
|
||
### View Registry: | ||
|
||
For an application to be registered as a view within Data Explorer, it needs to adhere to the following data model: | ||
|
||
```ts | ||
interface ViewDefinition<T = any> { | ||
readonly id: string; | ||
readonly title: string; | ||
readonly ui?: { | ||
defaults: DefaultViewState | (() => DefaultViewState) | (() => Promise<DefaultViewState>); | ||
slice: Slice<T>; | ||
}; | ||
readonly Canvas: LazyExoticComponent<(props: ViewProps) => React.ReactElement>; | ||
readonly Panel: LazyExoticComponent<(props: ViewProps) => React.ReactElement>; | ||
readonly Context: LazyExoticComponent< | ||
(props: React.PropsWithChildren<ViewProps>) => React.ReactElement | ||
>; | ||
readonly defaultPath: string; | ||
readonly appExtentions: { | ||
savedObject: { | ||
docTypes: [string]; | ||
toListItem: (obj: { id: string; title: string }) => ViewListItem; | ||
}; | ||
}; | ||
readonly shouldShow?: (state: any) => boolean; | ||
} | ||
``` | ||
|
||
--- | ||
|
||
See the [OpenSearch Dashboards contributing | ||
guide](https://github.com/opensearch-project/OpenSearch-Dashboards/blob/main/CONTRIBUTING.md) for instructions | ||
setting up your development environment. | ||
Original proposal: https://github.com/opensearch-project/OpenSearch-Dashboards/issues/4165 |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
For now it seems to have many
data source
, but stands for different meanings:Shall we align on unified terms on this?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
+1
I think we could have a terminology reference doc in the repo.
Multi-Datasources (OpenSearch Clusters):
Term: "Connected Clusters" or "Cluster Sources"
Description: Use this term to refer specifically to the ability of OSD to connect to multiple OpenSearch clusters. Each "Connected Cluster" or "Cluster Source" represents a distinct OpenSearch cluster, identifiable by its endpoint URL and credentials. This feature allows OSD to retrieve and manage data from multiple clusters within the same OSD instance.
Data Source (Index Patterns and Indices):
Term: "Index Patterns"
Description: This term should be used when referring to the specific index patterns or indices in OpenSearch that are used for data retrieval. "Data Indexes" or "Index Patterns" are the structures or patterns within an individual OpenSearch cluster that OSD queries to fetch and visualize data.
But may be not in this PR. I will approve this PR and we could start discussing on having a terminology reference.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Data source here is meant to talk about all data sources, including multi data sources, and even non OpenSearch data sources.
"Data sources: indexes/index patterns" is only for now but it's going to change soon, which is why the naming of the drop downs was changed in anticipation of that.