-
Notifications
You must be signed in to change notification settings - Fork 8.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Logs Explorer] Add support for virtual columns in the data table #171844
Labels
Comments
Pinging @elastic/obs-ux-logs-team (Team:obs-ux-logs) |
Example of Custom Renderer used by the Cloud Security Team- Lines 71 to 82 in 0d17a94
|
This was referenced Dec 19, 2023
achyutjhunjhunwala
added a commit
that referenced
this issue
Jan 11, 2024
… virtual column (#173732) ## Summary - Closes #171844 - Closes #171726 - Closes #173825 This PR does 4 things - Customisation point to inject customCellRenderers in the Data Table. - Adding the 1st virtual column - Content and removal of Message column - Add cell actions for virtual columns so that the custom fields can be filtered for - Renders similar content on the Flyout instead of Message ### To make this PR easy to review #### Why Kibana Operations - This PR initially increased the bundle size causing the values to go beyond the boundaries set in `limits.yml`. This has been fixed now after suggestion from Kibana Ops team #### For Kibana Data Discovery Team - Changes done on Unified Data Table - Moving around types to make them more generic and reusable - Split the 2 components SourceDocument and SourceDocumentPopover out of the 1 single large file `ger_render_cell_value.tsx` they were in. This allows me to export them and lazy load the components in my plugin. - Passed additional parameters to `externalCustomRenderers` function #### For Kibana Data Discovery Team - Changes done on Discover Plugin - Currently the discover plugin was not passing the registered customisation for `externalCustomRenderers`. Simply added the hook to retrieve the registered customisation and pass it as props. - Added new customisation type for `CellRendererCustomisation` #### For Logs UX Team - Changes done on Log Explorer - Registered a new customisation for custom Virtual columns. - Rendering logic for Log Level and Message with fallbacks - When Message and its fallbacks are not present, we must render the whole document. Used extracted logic to display complete document as it appears in current Discover. - When a JSON column is expanded (expandCellAction), it should render a JSON viewer as it currently does in Discover - Items like Log Level which are rendered inside a Chip should allow contextual actions like Filter In, Filter Out and Copy ## Want to run this PR locally ``` // Checkout this PR gh pr checkout 173732 // Start Kibana locally yarn start // Start ES locally yarn es snapshot // Generate dummy data node scripts/synthtrace simple_logs.ts --clean ``` ## Live Demo can be seen [here](https://achyutjhunjhunwala-d-pr173732.kb.us-west2.gcp.elastic-cloud.com/app/observability-log-explorer/?pageState=(breakdownField:log.level,columns:!((field:service.name,width:240),(field:host.name,width:320),(field:content)),controls:(data_stream.namespace:(mode:include,selection:(selectedOptions:!(),type:options))),datasetSelection:(selectionType:all),filters:!(),query:(language:kuery,query:%27%27),refreshInterval:(pause:!t,value:5000),rowHeight:0,rowsPerPage:100,time:(from:%272024-01-05T14:39:41.024Z%27,to:%272024-01-05T14:54:43.378Z%27),v:1)) ## Demo ![Virtual Column - Content](https://github.com/elastic/kibana/assets/7416358/db5977cd-1342-4dce-bfe9-bad3ee42487b) ## What's pending - Basically why is the PR still in Draft Stage - [x] Fix colors for logs levels after confirmation from Isa and hide level when not present - [x] Fix JSON rendering on the content cell when message field is not present so that it looks like exactly how it looks in Discover - [x] Add E2E tests - [x] Update DEMO with a gif rather than a screenshot --------- Co-authored-by: kibanamachine <[email protected]>
delanni
pushed a commit
to delanni/kibana
that referenced
this issue
Jan 11, 2024
… virtual column (elastic#173732) ## Summary - Closes elastic#171844 - Closes elastic#171726 - Closes elastic#173825 This PR does 4 things - Customisation point to inject customCellRenderers in the Data Table. - Adding the 1st virtual column - Content and removal of Message column - Add cell actions for virtual columns so that the custom fields can be filtered for - Renders similar content on the Flyout instead of Message ### To make this PR easy to review #### Why Kibana Operations - This PR initially increased the bundle size causing the values to go beyond the boundaries set in `limits.yml`. This has been fixed now after suggestion from Kibana Ops team #### For Kibana Data Discovery Team - Changes done on Unified Data Table - Moving around types to make them more generic and reusable - Split the 2 components SourceDocument and SourceDocumentPopover out of the 1 single large file `ger_render_cell_value.tsx` they were in. This allows me to export them and lazy load the components in my plugin. - Passed additional parameters to `externalCustomRenderers` function #### For Kibana Data Discovery Team - Changes done on Discover Plugin - Currently the discover plugin was not passing the registered customisation for `externalCustomRenderers`. Simply added the hook to retrieve the registered customisation and pass it as props. - Added new customisation type for `CellRendererCustomisation` #### For Logs UX Team - Changes done on Log Explorer - Registered a new customisation for custom Virtual columns. - Rendering logic for Log Level and Message with fallbacks - When Message and its fallbacks are not present, we must render the whole document. Used extracted logic to display complete document as it appears in current Discover. - When a JSON column is expanded (expandCellAction), it should render a JSON viewer as it currently does in Discover - Items like Log Level which are rendered inside a Chip should allow contextual actions like Filter In, Filter Out and Copy ## Want to run this PR locally ``` // Checkout this PR gh pr checkout 173732 // Start Kibana locally yarn start // Start ES locally yarn es snapshot // Generate dummy data node scripts/synthtrace simple_logs.ts --clean ``` ## Live Demo can be seen [here](https://achyutjhunjhunwala-d-pr173732.kb.us-west2.gcp.elastic-cloud.com/app/observability-log-explorer/?pageState=(breakdownField:log.level,columns:!((field:service.name,width:240),(field:host.name,width:320),(field:content)),controls:(data_stream.namespace:(mode:include,selection:(selectedOptions:!(),type:options))),datasetSelection:(selectionType:all),filters:!(),query:(language:kuery,query:%27%27),refreshInterval:(pause:!t,value:5000),rowHeight:0,rowsPerPage:100,time:(from:%272024-01-05T14:39:41.024Z%27,to:%272024-01-05T14:54:43.378Z%27),v:1)) ## Demo ![Virtual Column - Content](https://github.com/elastic/kibana/assets/7416358/db5977cd-1342-4dce-bfe9-bad3ee42487b) ## What's pending - Basically why is the PR still in Draft Stage - [x] Fix colors for logs levels after confirmation from Isa and hide level when not present - [x] Fix JSON rendering on the content cell when message field is not present so that it looks like exactly how it looks in Discover - [x] Add E2E tests - [x] Update DEMO with a gif rather than a screenshot --------- Co-authored-by: kibanamachine <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Labels
📓 Summary
Virtual columns will be introduced in Logs Explorer whose values are created on the fly and are not part of the mapping, aiming to provide useful information to users which is easier to consume.
As part of this work, Discover will be updated to support the virtual columns in the data table.
Related to #169874.
✔️ Acceptance criteria
💡 Implementation hints
Inject the columns into the Discover state
The source of truth for the columns displayed in the document table is the
column
property on the Discovper app state. For every entry (except for_source
) the grid will try to render a column, regardless of whether it's part of the search hit or not. The particular rendering is delegated to the renderer as described in the next section. The log explorer already sets the default columns upon initialization, so this should work analogously.Define an external custom renderer
The unified data grid used in Discover has a default renderer for all cells, which applies field formatter overrides defined in the data view and falls back to field-type-specific formatters otherwise. In addition, the unified data grid has a property
externalCustomRenderers
using which cell renderer overrides can be defined programmatically for columns of a certain name. It is currently not yet exposed via a customization point in the main Discover component, but once it is the Log Explorer could pass renderers that match the names of the virtual columns injected above.The renderer is responsible for both the "cell representation", which should be optimized for space efficiency as well as the "popover view", which offers room for more elaborate representations. The two cases are distinguished via the
isDetails
property.The text was updated successfully, but these errors were encountered: