Skip to content

Commit

Permalink
[Logs Explorer] Add customisation for virtual columns and add the 1st…
Browse files Browse the repository at this point in the history
… virtual column (#173732)

## Summary

- Closes #171844
- Closes #171726
- Closes  #173825

This PR does 4 things

- Customisation point to inject customCellRenderers in the Data Table.
- Adding the 1st virtual column - Content and removal of Message column
- Add cell actions for virtual columns so that the custom fields can be
filtered for
- Renders similar content on the Flyout instead of Message

### To make this PR easy to review

#### Why Kibana Operations

- This PR initially increased the bundle size causing the values to go
beyond the boundaries set in `limits.yml`. This has been fixed now after
suggestion from Kibana Ops team

#### For Kibana Data Discovery Team - Changes done on Unified Data Table

- Moving around types to make them more generic and reusable
- Split the 2 components SourceDocument and SourceDocumentPopover out of
the 1 single large file `ger_render_cell_value.tsx` they were in. This
allows me to export them and lazy load the components in my plugin.
- Passed additional parameters to `externalCustomRenderers` function

#### For Kibana Data Discovery Team - Changes done on Discover Plugin

- Currently the discover plugin was not passing the registered
customisation for `externalCustomRenderers`. Simply added the hook to
retrieve the registered customisation and pass it as props.
- Added new customisation type for `CellRendererCustomisation`

#### For Logs UX Team - Changes done on Log Explorer

- Registered a new customisation for custom Virtual columns.
- Rendering logic for Log Level and Message with fallbacks
- When Message and its fallbacks are not present, we must render the
whole document. Used extracted logic to display complete document as it
appears in current Discover.
- When a JSON column is expanded (expandCellAction), it should render a
JSON viewer as it currently does in Discover
- Items like Log Level which are rendered inside a Chip should allow
contextual actions like Filter In, Filter Out and Copy

## Want to run this PR locally

```
// Checkout this PR
gh pr checkout 173732

// Start Kibana locally
yarn start 

// Start ES locally
yarn es snapshot

// Generate dummy data
node scripts/synthtrace simple_logs.ts --clean

```

## Live Demo can be seen
[here](https://achyutjhunjhunwala-d-pr173732.kb.us-west2.gcp.elastic-cloud.com/app/observability-log-explorer/?pageState=(breakdownField:log.level,columns:!((field:service.name,width:240),(field:host.name,width:320),(field:content)),controls:(data_stream.namespace:(mode:include,selection:(selectedOptions:!(),type:options))),datasetSelection:(selectionType:all),filters:!(),query:(language:kuery,query:%27%27),refreshInterval:(pause:!t,value:5000),rowHeight:0,rowsPerPage:100,time:(from:%272024-01-05T14:39:41.024Z%27,to:%272024-01-05T14:54:43.378Z%27),v:1))

## Demo

![Virtual Column -
Content](https://github.com/elastic/kibana/assets/7416358/db5977cd-1342-4dce-bfe9-bad3ee42487b)

## What's pending - Basically why is the PR still in Draft Stage

- [x] Fix colors for logs levels after confirmation from Isa and hide
level when not present
- [x] Fix JSON rendering on the content cell when message field is not
present so that it looks like exactly how it looks in Discover
- [x] Add E2E tests
- [x] Update DEMO with a gif rather than a screenshot

---------

Co-authored-by: kibanamachine <[email protected]>
  • Loading branch information
achyutjhunjhunwala and kibanamachine authored Jan 11, 2024
1 parent 9be03fe commit eaf5871
Show file tree
Hide file tree
Showing 44 changed files with 2,012 additions and 760 deletions.
2 changes: 2 additions & 0 deletions packages/kbn-apm-synthtrace-client/src/lib/logs/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,8 @@ export type LogDocument = Fields &
'data_stream.type': string;
'data_stream.dataset': string;
message?: string;
'error.message'?: string;
'event.original'?: string;
'event.dataset': string;
'log.level'?: string;
'host.name'?: string;
Expand Down
128 changes: 126 additions & 2 deletions packages/kbn-apm-synthtrace/src/scenarios/simple_logs.ts
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ const scenario: Scenario<LogDocument> = async (runOptions) => {
.interval('1m')
.rate(1)
.generator((timestamp) => {
return Array(20)
return Array(3)
.fill(0)
.map(() => {
const index = Math.floor(Math.random() * 3);
Expand All @@ -63,9 +63,133 @@ const scenario: Scenario<LogDocument> = async (runOptions) => {
});
});

const logsWithNoLogLevel = range
.interval('1m')
.rate(1)
.generator((timestamp) => {
return Array(3)
.fill(0)
.map(() => {
const index = Math.floor(Math.random() * 3);
return log
.create()
.service(SERVICE_NAMES[index])
.defaults({
'trace.id': generateShortId(),
'error.message': MESSAGE_LOG_LEVELS[index].message,
'agent.name': 'synth-agent',
'orchestrator.cluster.name': CLUSTER[index].clusterName,
'orchestrator.cluster.id': CLUSTER[index].clusterId,
'orchestrator.resource.id': generateShortId(),
'cloud.provider': CLOUD_PROVIDERS[Math.floor(Math.random() * 3)],
'cloud.region': CLOUD_REGION[index],
'cloud.availability_zone': `${CLOUD_REGION[index]}a`,
'cloud.project.id': generateShortId(),
'cloud.instance.id': generateShortId(),
'log.file.path': `/logs/${generateLongId()}/error.txt`,
})
.timestamp(timestamp);
});
});

const logsWithErrorMessage = range
.interval('1m')
.rate(1)
.generator((timestamp) => {
return Array(3)
.fill(0)
.map(() => {
const index = Math.floor(Math.random() * 3);
return log
.create()
.logLevel(MESSAGE_LOG_LEVELS[index].level)
.service(SERVICE_NAMES[index])
.defaults({
'trace.id': generateShortId(),
'error.message': MESSAGE_LOG_LEVELS[index].message,
'agent.name': 'synth-agent',
'orchestrator.cluster.name': CLUSTER[index].clusterName,
'orchestrator.cluster.id': CLUSTER[index].clusterId,
'orchestrator.resource.id': generateShortId(),
'cloud.provider': CLOUD_PROVIDERS[Math.floor(Math.random() * 3)],
'cloud.region': CLOUD_REGION[index],
'cloud.availability_zone': `${CLOUD_REGION[index]}a`,
'cloud.project.id': generateShortId(),
'cloud.instance.id': generateShortId(),
'log.file.path': `/logs/${generateLongId()}/error.txt`,
})
.timestamp(timestamp);
});
});

const logsWithEventMessage = range
.interval('1m')
.rate(1)
.generator((timestamp) => {
return Array(3)
.fill(0)
.map(() => {
const index = Math.floor(Math.random() * 3);
return log
.create()
.logLevel(MESSAGE_LOG_LEVELS[index].level)
.service(SERVICE_NAMES[index])
.defaults({
'trace.id': generateShortId(),
'event.original': MESSAGE_LOG_LEVELS[index].message,
'agent.name': 'synth-agent',
'orchestrator.cluster.name': CLUSTER[index].clusterName,
'orchestrator.cluster.id': CLUSTER[index].clusterId,
'orchestrator.resource.id': generateShortId(),
'cloud.provider': CLOUD_PROVIDERS[Math.floor(Math.random() * 3)],
'cloud.region': CLOUD_REGION[index],
'cloud.availability_zone': `${CLOUD_REGION[index]}a`,
'cloud.project.id': generateShortId(),
'cloud.instance.id': generateShortId(),
'log.file.path': `/logs/${generateLongId()}/error.txt`,
})
.timestamp(timestamp);
});
});

const logsWithNoMessage = range
.interval('1m')
.rate(1)
.generator((timestamp) => {
return Array(3)
.fill(0)
.map(() => {
const index = Math.floor(Math.random() * 3);
return log
.create()
.logLevel(MESSAGE_LOG_LEVELS[index].level)
.service(SERVICE_NAMES[index])
.defaults({
'trace.id': generateShortId(),
'agent.name': 'synth-agent',
'orchestrator.cluster.name': CLUSTER[index].clusterName,
'orchestrator.cluster.id': CLUSTER[index].clusterId,
'orchestrator.resource.id': generateShortId(),
'cloud.provider': CLOUD_PROVIDERS[Math.floor(Math.random() * 3)],
'cloud.region': CLOUD_REGION[index],
'cloud.availability_zone': `${CLOUD_REGION[index]}a`,
'cloud.project.id': generateShortId(),
'cloud.instance.id': generateShortId(),
'log.file.path': `/logs/${generateLongId()}/error.txt`,
})
.timestamp(timestamp);
});
});

return withClient(
logsEsClient,
logger.perf('generating_logs', () => logs)
logger.perf('generating_logs', () => [
logs,
logsWithNoLogLevel,
logsWithErrorMessage,
logsWithEventMessage,
logsWithNoMessage,
])
);
},
};
Expand Down
4 changes: 2 additions & 2 deletions packages/kbn-optimizer/limits.yml
Original file line number Diff line number Diff line change
Expand Up @@ -85,12 +85,12 @@ pageLoadAssetSize:
kibanaUsageCollection: 16463
kibanaUtils: 79713
kubernetesSecurity: 77234
lens: 43000
lens: 57135
licenseManagement: 41817
licensing: 29004
links: 44490
lists: 22900
logExplorer: 54342
logExplorer: 44977
logsShared: 281060
logstash: 53548
management: 46112
Expand Down
9 changes: 3 additions & 6 deletions packages/kbn-unified-data-table/src/components/data_table.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,6 @@ import {
EuiDataGridInMemory,
EuiDataGridControlColumn,
EuiDataGridCustomBodyProps,
EuiDataGridCellValueElementProps,
EuiDataGridCustomToolbarProps,
EuiDataGridToolBarVisibilityOptions,
EuiDataGridToolBarVisibilityDisplaySelectorOptions,
Expand All @@ -47,10 +46,11 @@ import type { FieldFormatsStart } from '@kbn/field-formats-plugin/public';
import type { ThemeServiceStart } from '@kbn/react-kibana-context-common';
import type { DataPublicPluginStart } from '@kbn/data-plugin/public';
import type { DocViewFilterFn } from '@kbn/unified-doc-viewer/types';
import type {
import {
UnifiedDataTableSettings,
ValueToStringConverter,
DataTableColumnTypes,
CustomCellRenderer,
} from '../types';
import { getDisplayedColumns } from '../utils/columns';
import { convertValueToString } from '../utils/convert_value_to_string';
Expand Down Expand Up @@ -324,10 +324,7 @@ export interface UnifiedDataTableProps {
/**
* An optional settings for a specified fields rendering like links. Applied only for the listed fields rendering.
*/
externalCustomRenderers?: Record<
string,
(props: EuiDataGridCellValueElementProps) => React.ReactNode
>;
externalCustomRenderers?: CustomCellRenderer;
/**
* Name of the UnifiedDataTable consumer component or application
*/
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0 and the Server Side Public License, v 1; you may not use this file except
* in compliance with, at your election, the Elastic License 2.0 or the Server
* Side Public License, v 1.
*/

import React from 'react';

export const DataTablePopoverCellValue = ({ children }: { children: React.ReactNode }) => {
return <span className="unifiedDataTable__cellPopoverValue eui-textBreakWord">{children}</span>;
};

// eslint-disable-next-line import/no-default-export
export default DataTablePopoverCellValue;
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0 and the Server Side Public License, v 1; you may not use this file except
* in compliance with, at your election, the Elastic License 2.0 or the Server
* Side Public License, v 1.
*/

import { dataViewMock } from '@kbn/discover-utils/src/__mocks__';
import { FieldFormatsStart } from '@kbn/field-formats-plugin/public';
import { mountWithIntl } from '@kbn/test-jest-helpers';
import React from 'react';
import SourceDocument from './source_document';
import type { EsHitRecord } from '@kbn/discover-utils/src/types';
import { buildDataTableRecord } from '@kbn/discover-utils';

const mockServices = {
fieldFormats: {
getDefaultInstance: jest.fn(() => ({ convert: (value: unknown) => (value ? value : '-') })),
},
};

const rowsSource: EsHitRecord[] = [
{
_id: '1',
_index: 'test',
_score: 1,
_source: { bytes: 100, extension: '.gz' },
highlight: {
extension: ['@kibana-highlighted-field.gz@/kibana-highlighted-field'],
},
},
];

const build = (hit: EsHitRecord) => buildDataTableRecord(hit, dataViewMock);

describe('Unified data table source document cell rendering', function () {
it('renders a description list for source type documents', () => {
const rows = rowsSource.map(build);

const component = mountWithIntl(
<SourceDocument
useTopLevelObjectColumns={false}
row={rows[0]}
dataView={dataViewMock}
columnId="_source"
fieldFormats={mockServices.fieldFormats as unknown as FieldFormatsStart}
shouldShowFieldHandler={() => false}
maxEntries={100}
isPlainRecord={true}
/>
);
expect(component.html()).toMatchInlineSnapshot(
`"<dl class=\\"euiDescriptionList unifiedDataTable__descriptionList unifiedDataTable__cellValue css-id58dh-euiDescriptionList-inline-left\\" data-test-subj=\\"discoverCellDescriptionList\\" data-type=\\"inline\\"><dt class=\\"euiDescriptionList__title unifiedDataTable__descriptionListTitle css-1fizlic-euiDescriptionList__title-inline-compressed\\">_index</dt><dd class=\\"euiDescriptionList__description unifiedDataTable__descriptionListDescription css-11rdew2-euiDescriptionList__description-inline-compressed\\">test</dd><dt class=\\"euiDescriptionList__title unifiedDataTable__descriptionListTitle css-1fizlic-euiDescriptionList__title-inline-compressed\\">_score</dt><dd class=\\"euiDescriptionList__description unifiedDataTable__descriptionListDescription css-11rdew2-euiDescriptionList__description-inline-compressed\\">1</dd></dl>"`
);
});
});
Loading

0 comments on commit eaf5871

Please sign in to comment.