[DESIGN][Table] new table visualization plugin

[ananzh]

Statement

Currently, table visualization is written in Angular. Angular is not supported since 12/2021 and cause high severity CVE issues. Since we decided to remove angular from current code base, table visualization needs to be refactored ore re-written in ReAct. Meanwhile, our current table visualization is old and we see requests to update.

Scope

• Minimal learning
• Easy to use
• Progress not perfection

Note: This design is not final.

Solutions

deangular current vis_type_table

pros:

• No down time.

cons:

• Need understanding on angular which might take much longer time than a rewrite

react + OUI Table

pros:

• No learning curve: no usage difference for customers
• Support splitted grids.

cons:

• Can't implement new features: adjustable columns, toolbar and etc.

react + OUI Datagrid

pros:

• Datagrid supports sorting. So we don't need to maintain the sort state
• Datagrid includes more feature possibilities: adjustable columns, toolbar and etc
• Better rendering: can auto-dectect schemas and do sorting and pagination in memory

cons:

• Datagrid doesn't support split grids. When we have split table (rows/columns), we need to render multiple datagrid components

Recommended Design

The recommendation design is to rewrite table visualization in OpenSearch Dashboards using react and OUI Datagrid.

Same user flow

same way to create table visualization

• on home page, click the side bar menu and click visualize
• on the top right of the visualizations list, click Create visualization
• choose Data Table as the visualization type and then choose the index pattern
• a table visualization should be rendered based on the default settings

same way to access table visualization

Saved table should be able to access from:

• the list on the Visualization page
• the saved object (Stack Management --> Saved objects)
• Dashboard page

Features

To smooth the user experience, this new table visualization is created from the same location and will keep all the previous features. More specifically, we are proposing the following key features for OpenSearch Dashboards Table Visualization:

sort the column

• the look is different. we won't see black up-pointing/down-pointing triangle.

pagination

• pagination should be shown properly at the same location (bottom right of the table)

export table as csv

• keep supporting the two formats: raw and formatted

split table in rows and columns

show total

• In Options, support show total option and metric (Sum, Average, Min, Max, Count). Should show a total of the column

show partial rows

• In Options, support Show partial rows option. When there are too many columns in the table, table should be truncated and display partial rows

show metrics for every bucket/level

• In Options, support Show metrics for every bucket/level option. If this option is chosen, table vis should add a metric for each bucket/level.

show percentage column

• In Options, support Percentage column option and default is Don't show
• Once the Percentage column option is chosen, table vis should add a percentage column

[optional] adjust column width

• support column width adjustment

Architechure

Workflow

The new table visualization will use the following workflow:

• table should be an embeddable visualization on the Visualization page
• embeddable table visualization is reloaded once there is a state change (time range, metric aggregation, bucket and etc)
• VisualizeEmbeddable will update the expression ExpressionLoader. We still use expressions plugin to fetch data. The state changes will make a new expression which will be used to create a query to fetch data from OS
• After the data is back, visConfig and visData are updated and expressions plugin start the render process
• Table rendering is called and Table Visualization will be rendered

Saved Object

Components

Future scope

Removing angular and rewriting table visualization is our first step to bring customer a modernized table experience. We want the future table visualization is to be completely customer driven.

I see there are a lot of requests of pivot Table. I think this is definitely something we should consider. I see there are contributors implementation based on original angular base. I would like to fetch some help from our community to try similar features our table visualization.

Meanwhile, would like more community feedbacks to complete the design of future Table Visualization. Please comment here.

[ananzh]

Tasks

• cleaning current table vis code
  Issue: [Table Visualization] remove current table visualization and legacy code #2212
  PR: [Table Visualization] remove legacy table and add table visualization #2279

• add table visualization plugin in react
  Issue: [Table Visualization] add table visualization plugin in react #2213
  PR: [Table Visualization] remove legacy table and add table visualization #2279

• add OUI Datagrid component for table vis
  Issue: [Table Visualization] add OUI Datagrid component for table vis #2305
  PR:
  restore sorting [Table Visualization] make table vis column sortable #2502
  restore pagination [Table Visualization] restore pagination to table vis #2461
  format columns to make it render correctly [Table Visualization] restore datagrid columns #2411
  make column resizable [Table Visualization] make table vis column resizable #2464

• Table viz to export table as CSV and split table
  Issue: [Table Visualization] Table viz to export table as CSV and split table #2379

• add unit tests, improve local state management and research on unique id
  Issue: [Follow-up][Table Visualization] Replace old table vis, add tests, improve local state management #2551

[ananzh]

Table vis has done implementation. Close this Research issue.