Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Logs UI] Show highlighted log entries in the minimap #40745

Merged
merged 73 commits into from
Aug 9, 2019
Merged
Show file tree
Hide file tree
Changes from 72 commits
Commits
Show all changes
73 commits
Select commit Hold shift + click to select a range
bd2e87a
Commit to allow opening the PR
Kerry350 Jun 25, 2019
c54da94
Add start of the highlights menu
Kerry350 Jun 25, 2019
0b7420a
Add terms input
Kerry350 Jun 25, 2019
a762d02
Add a container for highlights state
Kerry350 Jun 25, 2019
494604c
Start to hook up highlights state effects, and Bridge with Redux
Kerry350 Jun 25, 2019
3f2bc2e
Move highlighted logs to a separate graphql field
weltenwort Jun 24, 2019
be180ea
Add highlights argument to unit tests
weltenwort Jun 25, 2019
abb640b
Add initial draft of a log highlights hook
weltenwort Jun 25, 2019
ad207ea
Finish hooking up hook state ready for GraphQL query
Kerry350 Jun 26, 2019
0a43161
Merge branch '22916-log-highlighting' of github.com:Kerry350/kibana i…
Kerry350 Jun 26, 2019
87347d3
Take startKey and endKey from overall collection of log entries
Kerry350 Jun 26, 2019
e4b95f6
Add sourceVersion
Kerry350 Jun 26, 2019
b778172
Fetch the data
Kerry350 Jun 26, 2019
653d468
Use columns ids instead of column indices
weltenwort Jun 27, 2019
1302ebd
merge highlights into stream WIP
weltenwort Jun 27, 2019
9565afd
Merge remote-tracking branch 'upstream/master' into 22916-log-highlig…
Kerry350 Jun 27, 2019
2a96ef4
Add highlighting for the message column type
weltenwort Jun 27, 2019
9c03ced
Use the serialized filter query
weltenwort Jun 27, 2019
1ce84a4
Merge branch '22916-log-highlighting' of github.com:Kerry350/kibana i…
Kerry350 Jun 28, 2019
4f1bc75
UX / Performance improvements
Kerry350 Jun 28, 2019
d647570
Tweaks
Kerry350 Jun 28, 2019
4ecb669
Merge remote-tracking branch 'upstream/master' into 22916-log-highlig…
Kerry350 Jun 28, 2019
32e2073
Menu changes
Kerry350 Jun 28, 2019
f1f75f2
Add highlighting for field columns
weltenwort Jun 28, 2019
7405d16
Use phrase query for highlighting
weltenwort Jun 28, 2019
556a349
Factor out commonly used visibility state
weltenwort Jun 28, 2019
534301b
Fix debounce to support changing onChange prop
weltenwort Jun 28, 2019
c478def
Move clear highlight button inline
weltenwort Jun 28, 2019
bea0187
Translate labels
weltenwort Jun 28, 2019
f816eef
Rely on popover ownfocus prop to focus the input
weltenwort Jun 28, 2019
8eb3425
Rename function to avoid i18n checker false positive
weltenwort Jun 28, 2019
aaad877
Merge branch 'master' into 22916-log-highlighting
weltenwort Jul 1, 2019
39776df
Merge remote-tracking branch 'upstream/master' into 22916-log-highlig…
Kerry350 Jul 2, 2019
56a4240
Rough and ready WIP next / previous
Kerry350 Jul 2, 2019
945a7d1
Merge remote-tracking branch 'upstream/master' into 22916-log-highlig…
Kerry350 Jul 2, 2019
5fd27bc
Add `countBefore` and `countAfter` highlight args
weltenwort Jul 2, 2019
980ffd1
Merge remote-tracking branch 'upstream/master' into 22916-log-highlig…
Kerry350 Jul 3, 2019
ee0ab64
Fix value used
Kerry350 Jul 3, 2019
7d2f70b
Merge branch '22916-log-highlighting-phase-2' of github.com:Kerry350/…
Kerry350 Jul 3, 2019
6400f48
Add a check for entry
Kerry350 Jul 3, 2019
56183ad
Rearrange icons
Kerry350 Jul 3, 2019
a8ebc1f
0 falsey blurgh
Kerry350 Jul 3, 2019
4f41b7e
Organise into separate sub-hooks for clarity
Kerry350 Jul 3, 2019
4810f54
Add comment
Kerry350 Jul 3, 2019
56ac1d8
Use filter and highlight queries also beyond interval
weltenwort Jul 3, 2019
3ddec5f
Refine behaviour of next / previous
Kerry350 Jul 4, 2019
abf61a8
Add log highlights api tests
weltenwort Jul 4, 2019
d8ed09d
Fix a few runtime and compiler errors and linter warnings
weltenwort Jul 4, 2019
dd64ad3
Refactor prev/next jumping to avoid index errors
weltenwort Jul 8, 2019
837d9cc
Use nearest index for initial jump
weltenwort Jul 8, 2019
b40b720
Include element type in translation id
weltenwort Jul 9, 2019
04c4209
Disable clear button when nothing to clear
weltenwort Jul 9, 2019
fa44967
Merge branch 'master' into 22916-log-highlighting-phase-2
weltenwort Jul 9, 2019
96e9dce
Merge remote-tracking branch 'upstream/master' into 22916-log-highlig…
Kerry350 Jul 15, 2019
ac4275a
Initial summary highlights state
Kerry350 Jul 15, 2019
92d9185
Amend component usage
Kerry350 Jul 15, 2019
fd8bc5c
Comment out load call until query exists
Kerry350 Jul 15, 2019
1eb703c
Add graphql api for summary highlights
weltenwort Jul 16, 2019
92af625
Add the client-side log summary highlight query
weltenwort Jul 17, 2019
aacd3bf
Directly use log view configuration without a bridge
weltenwort Jul 17, 2019
48195f1
Rename log entry highlight hook to maintain symmetry
weltenwort Jul 17, 2019
4eeefe4
Re-enable highlight rendering in the minimap
weltenwort Jul 17, 2019
08aed09
Keep the time cursor from intercepting clicks
weltenwort Jul 17, 2019
eec2ab2
tweak minimap highlighting colors and interaction
weltenwort Jul 17, 2019
aba3d69
Merge branch 'master' into 22916-log-highlighting-phase-3
weltenwort Jul 17, 2019
e32c02f
Merge branch 'master' into 22916-log-highlighting-phase-3
weltenwort Jul 24, 2019
c7ff7ed
Clean up after merge
weltenwort Jul 24, 2019
16988eb
Remove commented-out code
weltenwort Jul 24, 2019
67d4571
Change cursor to hint at minimap highlighting interaction
weltenwort Jul 24, 2019
90dc47a
Merge branch 'master' into 22916-log-highlighting-phase-3
weltenwort Aug 7, 2019
eec2358
Restrict highlight query filter to relevant fields
weltenwort Aug 7, 2019
037b347
Add explanatory error message
weltenwort Aug 7, 2019
2820f7f
Merge branch 'master' into 22916-log-highlighting-phase-3
weltenwort Aug 9, 2019
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
87 changes: 87 additions & 0 deletions x-pack/legacy/plugins/infra/common/graphql/types.ts
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,8 @@ export interface InfraSource {
logEntryHighlights: InfraLogEntryInterval[];
/** A consecutive span of summary buckets within an interval */
logSummaryBetween: InfraLogSummaryInterval;
/** Spans of summary highlight buckets within an interval */
logSummaryHighlightsBetween: InfraLogSummaryHighlightInterval[];

logItem: InfraLogItem;
/** A snapshot of nodes */
Expand Down Expand Up @@ -224,6 +226,30 @@ export interface InfraLogSummaryBucket {
/** The number of entries inside the bucket */
entriesCount: number;
}
/** A consecutive sequence of log summary highlight buckets */
export interface InfraLogSummaryHighlightInterval {
/** The millisecond timestamp corresponding to the start of the interval covered by the summary */
start?: number | null;
/** The millisecond timestamp corresponding to the end of the interval covered by the summary */
end?: number | null;
/** The query the log entries were filtered by */
filterQuery?: string | null;
/** The query the log entries were highlighted with */
highlightQuery?: string | null;
/** A list of the log entries */
buckets: InfraLogSummaryHighlightBucket[];
}
/** A log summary highlight bucket */
export interface InfraLogSummaryHighlightBucket {
/** The start timestamp of the bucket */
start: number;
/** The end timestamp of the bucket */
end: number;
/** The number of highlighted entries inside the bucket */
entriesCount: number;
/** The time key of a representative of the highlighted log entries in this bucket */
representativeKey: InfraTimeKey;
}

export interface InfraLogItem {
/** The ID of the document */
Expand Down Expand Up @@ -446,6 +472,18 @@ export interface LogSummaryBetweenInfraSourceArgs {
/** The query to filter the log entries by */
filterQuery?: string | null;
}
export interface LogSummaryHighlightsBetweenInfraSourceArgs {
/** The millisecond timestamp that corresponds to the start of the interval */
start: number;
/** The millisecond timestamp that corresponds to the end of the interval */
end: number;
/** The size of each bucket in milliseconds */
bucketSize: number;
/** The query to filter the log entries by */
filterQuery?: string | null;
/** The highlighting to apply to the log entries */
highlightQueries: string[];
}
export interface LogItemInfraSourceArgs {
id: string;
}
Expand Down Expand Up @@ -657,6 +695,55 @@ export namespace LogEntryHighlightsQuery {
export type Entries = InfraLogEntryHighlightFields.Fragment;
}

export namespace LogSummaryHighlightsQuery {
export type Variables = {
sourceId?: string | null;
start: number;
end: number;
bucketSize: number;
highlightQueries: string[];
filterQuery?: string | null;
};

export type Query = {
__typename?: 'Query';

source: Source;
};

export type Source = {
__typename?: 'InfraSource';

id: string;

logSummaryHighlightsBetween: LogSummaryHighlightsBetween[];
};

export type LogSummaryHighlightsBetween = {
__typename?: 'InfraLogSummaryHighlightInterval';

start?: number | null;

end?: number | null;

buckets: Buckets[];
};

export type Buckets = {
__typename?: 'InfraLogSummaryHighlightBucket';

start: number;

end: number;

entriesCount: number;

representativeKey: RepresentativeKey;
};

export type RepresentativeKey = InfraTimeKeyFields.Fragment;
}

export namespace LogSummary {
export type Variables = {
sourceId?: string | null;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -9,12 +9,11 @@ import * as React from 'react';

import euiStyled from '../../../../../../common/eui_styled_components';
import { LogEntryTime } from '../../../../common/log_entry';
// import { SearchSummaryBucket } from '../../../../common/log_search_summary';
import { DensityChart } from './density_chart';
import { HighlightedInterval } from './highlighted_interval';
// import { SearchMarkers } from './search_markers';
import { SearchMarkers } from './search_markers';
import { TimeRuler } from './time_ruler';
import { SummaryBucket } from './types';
import { SummaryBucket, SummaryHighlightBucket } from './types';

interface LogMinimapProps {
className?: string;
Expand All @@ -26,7 +25,7 @@ interface LogMinimapProps {
jumpToTarget: (params: LogEntryTime) => any;
intervalSize: number;
summaryBuckets: SummaryBucket[];
// searchSummaryBuckets?: SearchSummaryBucket[];
summaryHighlightBuckets?: SummaryHighlightBucket[];
target: number | null;
width: number;
}
Expand Down Expand Up @@ -81,9 +80,9 @@ export class LogMinimap extends React.Component<LogMinimapProps, LogMinimapState
className,
height,
highlightedInterval,
// jumpToTarget,
jumpToTarget,
summaryBuckets,
// searchSummaryBuckets,
summaryHighlightBuckets,
width,
} = this.props;

Expand Down Expand Up @@ -119,17 +118,17 @@ export class LogMinimap extends React.Component<LogMinimapProps, LogMinimapState
width={width}
/>
) : null}
<TimeCursor x1={0} x2={width} y1={timeCursorY} y2={timeCursorY} />
{/* <g transform={`translate(${width * 0.5}, 0)`}>
<g transform={`translate(${width * 0.5}, 0)`}>
<SearchMarkers
buckets={searchSummaryBuckets || []}
buckets={summaryHighlightBuckets || []}
start={minTime}
end={maxTime}
width={width / 2}
height={height}
jumpToTarget={jumpToTarget}
/>
</g> */}
</g>
<TimeCursor x1={0} x2={width} y1={timeCursorY} y2={timeCursorY} />
</MinimapWrapper>
);
}
Expand All @@ -145,6 +144,7 @@ const MinimapBorder = euiStyled.line`
`;

const TimeCursor = euiStyled.line`
pointer-events: none;
stroke-width: 1px;
stroke: ${props =>
props.theme.darkMode
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -9,11 +9,11 @@ import * as React from 'react';

import euiStyled, { keyframes } from '../../../../../../common/eui_styled_components';
import { LogEntryTime } from '../../../../common/log_entry';
import { SearchSummaryBucket } from '../../../../common/log_search_summary';
import { SearchMarkerTooltip } from './search_marker_tooltip';
import { SummaryHighlightBucket } from './types';

interface SearchMarkerProps {
bucket: SearchSummaryBucket;
bucket: SummaryHighlightBucket;
height: number;
width: number;
jumpToTarget: (target: LogEntryTime) => void;
Expand All @@ -31,7 +31,7 @@ export class SearchMarker extends React.PureComponent<SearchMarkerProps, SearchM
public handleClick: React.MouseEventHandler<SVGGElement> = evt => {
evt.stopPropagation();

this.props.jumpToTarget(this.props.bucket.representative.fields);
this.props.jumpToTarget(this.props.bucket.representativeKey);
};

public handleMouseEnter: React.MouseEventHandler<SVGGElement> = evt => {
Expand All @@ -51,7 +51,7 @@ export class SearchMarker extends React.PureComponent<SearchMarkerProps, SearchM
const { hoveredPosition } = this.state;

const bulge =
bucket.count > 1 ? (
bucket.entriesCount > 1 ? (
<SearchMarkerForegroundRect x="-2" y="-2" width="4" height={height + 2} rx="2" ry="2" />
) : (
<>
Expand All @@ -73,9 +73,9 @@ export class SearchMarker extends React.PureComponent<SearchMarkerProps, SearchM
<SearchMarkerTooltip markerPosition={hoveredPosition}>
<FormattedMessage
id="xpack.infra.logs.searchResultTooltip"
defaultMessage="{bucketCount, plural, one {# search result} other {# search results}}"
defaultMessage="{bucketCount, plural, one {# highlighted entry} other {# highlighted entries}}"
values={{
bucketCount: bucket.count,
bucketCount: bucket.entriesCount,
}}
/>
</SearchMarkerTooltip>
Expand Down Expand Up @@ -107,15 +107,16 @@ const SearchMarkerGroup = euiStyled.g`
`;

const SearchMarkerBackgroundRect = euiStyled.rect`
fill: ${props => props.theme.eui.euiColorSecondary};
fill: ${props => props.theme.eui.euiColorAccent};
opacity: 0;
transition: opacity ${props => props.theme.eui.euiAnimSpeedNormal} ease-in;
cursor: pointer;

${SearchMarkerGroup}:hover & {
opacity: 0.2;
opacity: 0.3;
}
`;

const SearchMarkerForegroundRect = euiStyled.rect`
fill: ${props => props.theme.eui.euiColorSecondary};
fill: ${props => props.theme.eui.euiColorAccent};
`;
Original file line number Diff line number Diff line change
Expand Up @@ -9,11 +9,11 @@ import { scaleTime } from 'd3-scale';
import * as React from 'react';

import { LogEntryTime } from '../../../../common/log_entry';
import { SearchSummaryBucket } from '../../../../common/log_search_summary';
import { SearchMarker } from './search_marker';
import { SummaryHighlightBucket } from './types';

interface SearchMarkersProps {
buckets: SearchSummaryBucket[];
buckets: SummaryHighlightBucket[];
className?: string;
end: number;
start: number;
Expand All @@ -38,7 +38,10 @@ export class SearchMarkers extends React.PureComponent<SearchMarkersProps, {}> {
return (
<g className={classes}>
{buckets.map(bucket => (
<g key={bucket.representative.gid} transform={`translate(0, ${yScale(bucket.start)})`}>
<g
key={`${bucket.representativeKey.time}:${bucket.representativeKey.tiebreaker}`}
transform={`translate(0, ${yScale(bucket.start)})`}
>
<SearchMarker
bucket={bucket}
height={yScale(bucket.end) - yScale(bucket.start)}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,7 @@ const TimeRulerTickLabel = euiStyled.text`
font-size: ${props => props.theme.eui.euiFontSizeXS};
line-height: ${props => props.theme.eui.euiLineHeight};
fill: ${props => props.theme.eui.textColors.subdued};
pointer-events: none;
`;

const TimeRulerGridLine = euiStyled.line`
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,14 @@
* you may not use this file except in compliance with the Elastic License.
*/

import { TimeKey } from '../../../../common/time';

export interface SummaryBucket {
start: number;
end: number;
entriesCount: number;
}

export interface SummaryHighlightBucket extends SummaryBucket {
representativeKey: TimeKey;
}
Original file line number Diff line number Diff line change
Expand Up @@ -11,11 +11,11 @@ import { LogEntryHighlightsQuery } from '../../../graphql/types';
import { DependencyError, useApolloClient } from '../../../utils/apollo_context';
import { LogEntryHighlightsMap } from '../../../utils/log_entry';
import { useTrackedPromise } from '../../../utils/use_tracked_promise';
import { logEntryHighlightsQuery } from './log_highlights.gql_query';
import { logEntryHighlightsQuery } from './log_entry_highlights.gql_query';

export type LogEntryHighlights = LogEntryHighlightsQuery.Query['source']['logEntryHighlights'];

export const useHighlightsFetcher = (
export const useLogEntryHighlights = (
sourceId: string,
sourceVersion: string | undefined,
startKey: TimeKey | null,
Expand All @@ -24,9 +24,7 @@ export const useHighlightsFetcher = (
highlightTerms: string[]
) => {
const apolloClient = useApolloClient();
const [logEntryHighlights, setLogEntryHighlights] = useState<LogEntryHighlights | undefined>(
undefined
);
const [logEntryHighlights, setLogEntryHighlights] = useState<LogEntryHighlights>([]);
const [loadLogEntryHighlightsRequest, loadLogEntryHighlights] = useTrackedPromise(
{
cancelPreviousOn: 'resolution',
Expand All @@ -35,7 +33,7 @@ export const useHighlightsFetcher = (
throw new DependencyError('Failed to load source: No apollo client available.');
}
if (!startKey || !endKey || !highlightTerms.length) {
throw new Error();
throw new Error('Skipping request: Insufficient parameters');
}

return await apolloClient.query<
Expand All @@ -51,9 +49,7 @@ export const useHighlightsFetcher = (
filterQuery,
highlights: [
{
query: JSON.stringify({
multi_match: { query: highlightTerms[0], type: 'phrase', lenient: true },
}),
query: highlightTerms[0],
countBefore: 1,
countAfter: 1,
},
Expand All @@ -69,7 +65,7 @@ export const useHighlightsFetcher = (
);

useEffect(() => {
setLogEntryHighlights(undefined);
setLogEntryHighlights([]);
}, [highlightTerms]);

useEffect(() => {
Expand All @@ -80,29 +76,24 @@ export const useHighlightsFetcher = (
) {
loadLogEntryHighlights();
} else {
setLogEntryHighlights(undefined);
setLogEntryHighlights([]);
}
}, [highlightTerms, startKey, endKey, filterQuery, sourceVersion]);

const logEntryHighlightsById = useMemo(
() =>
logEntryHighlights
? logEntryHighlights.reduce<LogEntryHighlightsMap>(
(accumulatedLogEntryHighlightsById, { entries }) => {
return entries.reduce<LogEntryHighlightsMap>(
(singleHighlightLogEntriesById, entry) => {
const highlightsForId = singleHighlightLogEntriesById[entry.gid] || [];
return {
...singleHighlightLogEntriesById,
[entry.gid]: [...highlightsForId, entry],
};
},
accumulatedLogEntryHighlightsById
);
},
{}
)
: {},
logEntryHighlights.reduce<LogEntryHighlightsMap>(
(accumulatedLogEntryHighlightsById, { entries }) => {
return entries.reduce<LogEntryHighlightsMap>((singleHighlightLogEntriesById, entry) => {
const highlightsForId = singleHighlightLogEntriesById[entry.gid] || [];
return {
...singleHighlightLogEntriesById,
[entry.gid]: [...highlightsForId, entry],
};
}, accumulatedLogEntryHighlightsById);
},
{}
),
[logEntryHighlights]
);

Expand Down
Loading