Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Exploratory View] Mobile experience #99565

Merged
merged 48 commits into from
Jun 18, 2021
Merged
Show file tree
Hide file tree
Changes from 23 commits
Commits
Show all changes
48 commits
Select commit Hold shift + click to select a range
5461cf8
wip
shahzad31 Apr 28, 2021
195ad59
Merge branch 'master' into mobile-experience
shahzad31 Apr 30, 2021
9e261b8
Merge branch 'master' into mobile-experience
shahzad31 May 7, 2021
9626988
wip
shahzad31 May 7, 2021
850f555
Merge branch 'master' into mobile-experience
kibanamachine May 27, 2021
b522c08
Merge branch 'master' into mobile-experience
shahzad31 Jun 2, 2021
720c68f
update
shahzad31 Jun 2, 2021
e8a678f
updated agent filter name to 'iOS/swift'
bryce-b Jun 2, 2021
5a260b2
Merge pull request #4 from bryce-b/mobile-experience
shahzad31 Jun 3, 2021
e2a6e1b
Merge branch 'master' into mobile-experience
shahzad31 Jun 8, 2021
32d36c2
Merge branch 'mobile-experience' of https://github.com/shahzad31/kiba…
shahzad31 Jun 8, 2021
9884d27
update memory and cpu
shahzad31 Jun 8, 2021
487e9c4
fix
shahzad31 Jun 8, 2021
24768ea
Merge branch 'master' into mobile-experience
kibanamachine Jun 9, 2021
41a76f3
Merge branch 'master' into mobile-experience
shahzad31 Jun 9, 2021
9ea07ec
Merge branch 'master' into mobile-experience
shahzad31 Jun 10, 2021
f71aa48
fix types
shahzad31 Jun 10, 2021
dde04c3
Merge branch 'mobile-experience' of https://github.com/shahzad31/kiba…
shahzad31 Jun 10, 2021
2cb4f1a
fix type and filters
shahzad31 Jun 10, 2021
51569fa
Merge branch 'master' into mobile-experience
shahzad31 Jun 10, 2021
e54969c
fix image
shahzad31 Jun 10, 2021
1631c07
Merge branch 'master' into mobile-experience
shahzad31 Jun 10, 2021
1f9c6ad
added mobile experience
shahzad31 Jun 10, 2021
17786ca
Merge branch 'master' into mobile-experience
shahzad31 Jun 14, 2021
9421c50
resolve conflicts
shahzad31 Jun 14, 2021
5192aa0
refactor code for has data
shahzad31 Jun 14, 2021
716184f
added throughput and distribution configs for mobile data
AlexanderWert Jun 15, 2021
6a1390d
Merge pull request #5 from AlexanderWert/mobile-exploratory-view
shahzad31 Jun 15, 2021
3a8bb82
Merge branch 'master' into mobile-experience
shahzad31 Jun 15, 2021
48ece1f
update
shahzad31 Jun 15, 2021
26de779
Merge branch 'master' into mobile-experience
shahzad31 Jun 15, 2021
6754f80
fix types
shahzad31 Jun 15, 2021
541df52
Merge branch 'master' into mobile-experience
shahzad31 Jun 16, 2021
fe641a0
added environment
shahzad31 Jun 16, 2021
423fba9
fix test
shahzad31 Jun 16, 2021
a802015
fix type
shahzad31 Jun 16, 2021
dfc1690
use i18n
shahzad31 Jun 16, 2021
8ab2ddd
Merge branch 'master' into mobile-experience
shahzad31 Jun 16, 2021
7f3bbc3
fix i18n
shahzad31 Jun 16, 2021
b1005cd
Added mobile device distribution config + fixed mobile field names
AlexanderWert Jun 17, 2021
c949862
fixing chart type options and label for distribution
AlexanderWert Jun 17, 2021
ec009fb
Merge pull request #6 from AlexanderWert/mobile-exploratory-view
shahzad31 Jun 17, 2021
2a9e540
Merge branch 'master' into mobile-experience
shahzad31 Jun 17, 2021
22ef586
fix types
shahzad31 Jun 17, 2021
ebf4ec5
fix 18n
shahzad31 Jun 17, 2021
0ecb538
fix test
shahzad31 Jun 17, 2021
2289c58
fix lint
shahzad31 Jun 17, 2021
067a7c3
Merge branch 'master' into mobile-experience
shahzad31 Jun 17, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -53,10 +53,8 @@ export const fetchObservabilityOverviewPageData = async ({
};

export async function getHasData() {
const res = await callApmApi({
return await callApmApi({
endpoint: 'GET /api/apm/observability_overview/has_data',
signal: null,
});

return res.hasData;
}
10 changes: 8 additions & 2 deletions x-pack/plugins/apm/server/lib/observability_overview/has_data.ts
Original file line number Diff line number Diff line change
Expand Up @@ -28,9 +28,15 @@ export function getHasData({ setup }: { setup: Setup }) {
};

const response = await apmEventClient.search(params);
return response.hits.total.value > 0;
return {
hasData: response.hits.total.value > 0,
indices: setup.indices['apm_oss.transactionIndices']!,
Copy link
Member

@sorenlouv sorenlouv Jun 10, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems a little odd to return the indices from this endpoint, since it's called has_data. If we really want to do it, shouldn't this include all of the indices that are being queried?

Eg:

Suggested change
indices: setup.indices['apm_oss.transactionIndices']!,
indices: [
setup.indices[ProcessorEvent.transaction],
setup.indices[ProcessorEvent.error],
setup.indices[ProcessorEvent.metric]
],

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ideally i think we should return info like this from all endpoints. I will modify it to return whole APM indices settings, instead of just transaction.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why should every API return this info? We already have an API (GET /api/apm/settings/apm-index-settings) where you can get this.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i meant in a sense of visiblity perspective, the way we are doing that inspect stuff for debugging purpose, i think it's no harm if we return indices information as an API response, it instanlty give you a view, against which indices that API got executed.

};
} catch (e) {
return false;
return {
hasData: false,
indices: setup.indices['apm_oss.transactionIndices']!,
};
}
});
}
3 changes: 1 addition & 2 deletions x-pack/plugins/apm/server/routes/observability_overview.ts
Original file line number Diff line number Diff line change
Expand Up @@ -21,8 +21,7 @@ const observabilityOverviewHasDataRoute = createApmServerRoute({
options: { tags: ['access:apm'] },
handler: async (resources) => {
const setup = await setupRequest(resources);
const res = await getHasData({ setup });
return { hasData: res };
return await getHasData({ setup });
},
});

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,11 @@
*/

import { FieldFormat } from '../../types';
import { TRANSACTION_DURATION } from '../constants/elasticsearch_fieldnames';
import {
METRIC_SYSTEM_CPU_USAGE,
METRIC_SYSTEM_MEMORY_USAGE,
TRANSACTION_DURATION,
} from '../constants/elasticsearch_fieldnames';

export const apmFieldFormats: FieldFormat[] = [
{
Expand All @@ -21,4 +25,12 @@ export const apmFieldFormats: FieldFormat[] = [
},
},
},
{
field: METRIC_SYSTEM_MEMORY_USAGE,
format: { id: 'bytes', params: {} },
},
{
field: METRIC_SYSTEM_CPU_USAGE,
format: { id: 'percent', params: {} },
},
];
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ import {
LOGS_FREQUENCY_LABEL,
MEMORY_USAGE_LABEL,
METRIC_LABEL,
MOBILE_RESPONSE_LABEL,
MONITOR_DURATION_LABEL,
MONITOR_ID_LABEL,
MONITOR_NAME_LABEL,
Expand Down Expand Up @@ -80,6 +81,16 @@ export const FieldLabels: Record<string, string> = {

'performance.metric': METRIC_LABEL,
'Business.KPI': KPI_LABEL,

'labels.net_connection_carrier_name': 'Carrier Name',
shahzad31 marked this conversation as resolved.
Show resolved Hide resolved
'http.request.method': 'Request Method',
'labels.net_connection_type': 'Connection Type',
'host.os.full': 'Host OS',
'service.version': 'Service Version',
'host.os.platform': 'OS Platform',
'labels.device_model': ' Device Model',
// eslint-disable-next-line @typescript-eslint/naming-convention
'labels.net_connection_carrier_isoCountryCode': 'Carrier Location',
};

export const DataViewLabels: Record<ReportViewTypeId, string> = {
Expand All @@ -94,6 +105,7 @@ export const DataViewLabels: Record<ReportViewTypeId, string> = {
mem: MEMORY_USAGE_LABEL,
nwk: NETWORK_ACTIVITY_LABEL,
cwv: CORE_WEB_VITALS_LABEL,
mkpi: MOBILE_RESPONSE_LABEL,
};

export const ReportToDataTypeMap: Record<ReportViewTypeId, AppDataType> = {
Expand All @@ -108,6 +120,7 @@ export const ReportToDataTypeMap: Record<ReportViewTypeId, AppDataType> = {
logs: 'infra_logs',
cpu: 'infra_metrics',
cwv: 'ux',
mkpi: 'apm',
};

export const USE_BREAK_DOWN_COLUMN = 'USE_BREAK_DOWN_COLUMN';
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -86,6 +86,8 @@ export const ERROR_PAGE_URL = 'error.page.url';

// METRICS
export const METRIC_SYSTEM_FREE_MEMORY = 'system.memory.actual.free';
export const METRIC_SYSTEM_MEMORY_USAGE = 'system.memory.usage';
export const METRIC_SYSTEM_CPU_USAGE = 'system.cpu.usage';
export const METRIC_SYSTEM_TOTAL_MEMORY = 'system.memory.total';
export const METRIC_SYSTEM_CPU_PERCENT = 'system.cpu.total.norm.pct';
export const METRIC_PROCESS_CPU_PERCENT = 'system.process.cpu.total.norm.pct';
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -208,6 +208,13 @@ export const CORE_WEB_VITALS_LABEL = i18n.translate(
}
);

export const MOBILE_RESPONSE_LABEL = i18n.translate(
'xpack.observability.expView.fieldLabels.mobileReponse',
{
defaultMessage: 'Mobile response',
}
);

export const MEMORY_USAGE_LABEL = i18n.translate(
'xpack.observability.expView.fieldLabels.memoryUsage',
{
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ import { getNetworkActivityLensConfig } from './metrics/network_activity_config'
import { getLogsFrequencyLensConfig } from './logs/logs_frequency_config';
import { IIndexPattern } from '../../../../../../../../src/plugins/data/common/index_patterns';
import { getCoreWebVitalsConfig } from './rum/core_web_vitals_config';
import { getMobileKPIConfig } from './mobile/kpi_over_time_config';

interface Props {
reportType: keyof typeof ReportViewTypes;
Expand All @@ -37,6 +38,8 @@ export const getDefaultConfigs = ({ reportType, seriesId, indexPattern }: Props)
return getMonitorDurationConfig({ seriesId, indexPattern });
case 'uptime-pings':
return getMonitorPingsConfig({ seriesId, indexPattern });
case 'mobile-kpi':
return getMobileKPIConfig({ seriesId, indexPattern });
case 'service-latency':
return getServiceLatencyLensConfig({ seriesId, indexPattern });
case 'service-throughput':
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,90 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/

import { ConfigProps, DataSeries } from '../../types';
import { FieldLabels, OPERATION_COLUMN } from '../constants';
import { buildPhraseFilter } from '../utils';
import {
METRIC_SYSTEM_CPU_USAGE,
METRIC_SYSTEM_MEMORY_USAGE,
SERVICE_NAME,
TRANSACTION_DURATION,
} from '../constants/elasticsearch_fieldnames';

export function getMobileKPIConfig({ seriesId, indexPattern }: ConfigProps): DataSeries {
return {
id: seriesId,
reportType: 'service-latency',
defaultSeriesType: 'line',
seriesTypes: ['line', 'bar'],
xAxisColumn: {
sourceField: '@timestamp',
},
yAxisColumns: [
{
sourceField: 'business.kpi',
operationType: 'median',
},
],
hasOperationType: true,
defaultFilters: [
'labels.net_connection_carrier_name',
'labels.device_model',
'labels.net_connection_type',
'host.os.platform',
'host.os.full',
'service.version',
],
breakdowns: [
'labels.net_connection_carrier_name',
'labels.device_model',
'labels.net_connection_type',
'host.os.platform',
'host.os.full',
'service.version',
'labels.net_connection_carrier_isoCountryCode',
],
filters: [...buildPhraseFilter('agent.name', 'iOS/swift', indexPattern)],
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we probably needs to discuss with APM-UI team on what other default filters we need to add here to fetch apm ios data.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

possibly also open-telemetry/swift

labels: {
...FieldLabels,
[TRANSACTION_DURATION]: 'Response latency',
[SERVICE_NAME]: 'Mobile app',
[METRIC_SYSTEM_MEMORY_USAGE]: 'Memory usage',
[METRIC_SYSTEM_CPU_USAGE]: 'CPU usage',
},
reportDefinitions: [
{
field: SERVICE_NAME,
required: true,
},
{
field: 'business.kpi',
custom: true,
options: [
{
label: 'Response latency',
field: TRANSACTION_DURATION,
id: TRANSACTION_DURATION,
columnType: OPERATION_COLUMN,
},
{
label: 'Memory Usage',
field: METRIC_SYSTEM_MEMORY_USAGE,
id: METRIC_SYSTEM_MEMORY_USAGE,
columnType: OPERATION_COLUMN,
},
{
label: 'CPU Usage',
field: METRIC_SYSTEM_CPU_USAGE,
id: METRIC_SYSTEM_CPU_USAGE,
columnType: OPERATION_COLUMN,
},
],
},
],
};
}
Original file line number Diff line number Diff line change
Expand Up @@ -41,14 +41,15 @@ export function IndexPatternContextProvider({ children }: ProviderProps) {
synthetics: null,
ux: null,
apm: null,
mobile: null,
} as HasAppDataState);

const {
services: { data },
} = useKibana<ObservabilityPublicPluginsStart>();

const checkIfAppHasData = async (dataType: AppDataType) => {
const handler = getDataHandler(dataType);
const handler = getDataHandler(dataType === 'mobile' ? 'apm' : dataType);
return handler?.hasData();
};

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ import { ReportToDataTypeMap } from '../../configurations/constants';
export const dataTypes: Array<{ id: AppDataType; label: string }> = [
{ id: 'synthetics', label: 'Synthetic Monitoring' },
{ id: 'ux', label: 'User Experience (RUM)' },
{ id: 'mobile', label: 'Mobile Experience' },
// { id: 'infra_logs', label: 'Logs' },
// { id: 'infra_metrics', label: 'Metrics' },
// { id: 'apm', label: 'APM' },
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ export function ReportFilters({
<SeriesFilter
series={dataViewSeries}
defaultFilters={dataViewSeries.defaultFilters}
filters={dataViewSeries.filters}
seriesId={seriesId}
isNew={true}
/>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,12 @@ export const ReportTypes: Record<AppDataType, Array<{ id: ReportViewTypeId; labe
{ id: 'kpi', label: 'KPI over time' },
{ id: 'cwv', label: 'Core Web Vitals' },
],
mobile: [
{
id: 'mkpi',
label: 'KPI over time',
},
],
apm: [
{ id: 'svl', label: 'Latency' },
{ id: 'tpt', label: 'Throughput' },
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ describe('FilterExpanded', function () {
label={'Browser Family'}
field={USER_AGENT_NAME}
goBack={jest.fn()}
filters={[]}
/>,
{ initSeries }
);
Expand All @@ -38,6 +39,7 @@ describe('FilterExpanded', function () {
label={'Browser Family'}
field={USER_AGENT_NAME}
goBack={goBack}
filters={[]}
/>,
{ initSeries }
);
Expand All @@ -61,6 +63,7 @@ describe('FilterExpanded', function () {
label={'Browser Family'}
field={USER_AGENT_NAME}
goBack={goBack}
filters={[]}
/>,
{ initSeries }
);
Expand All @@ -84,6 +87,7 @@ describe('FilterExpanded', function () {
label={'Browser Family'}
field={USER_AGENT_NAME}
goBack={jest.fn()}
filters={[]}
/>,
{ initSeries }
);
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,16 +6,20 @@
*/

import React, { useState, Fragment } from 'react';
import { EuiFieldSearch, EuiSpacer, EuiButtonEmpty, EuiFilterGroup } from '@elastic/eui';
import { EuiFieldSearch, EuiSpacer, EuiButtonEmpty, EuiFilterGroup, EuiText } from '@elastic/eui';
import styled from 'styled-components';
import { rgba } from 'polished';
import { i18n } from '@kbn/i18n';
import { QueryDslQueryContainer } from '@elastic/elasticsearch/api/types';
import { useAppIndexPatternContext } from '../../hooks/use_app_index_pattern';
import { useSeriesStorage } from '../../hooks/use_series_storage';
import { UrlFilter } from '../../types';
import { DataSeries, UrlFilter } from '../../types';
import { FilterValueButton } from './filter_value_btn';
import { useValuesList } from '../../../../../hooks/use_values_list';
import { euiStyled } from '../../../../../../../../../src/plugins/kibana_react/common';
import { ESFilter } from '../../../../../../../../../typings/elasticsearch';
import { PersistableFilter } from '../../../../../../../lens/common';
import { ExistsFilter } from '../../../../../../../../../src/plugins/data/common/es_query/filters';

interface Props {
seriesId: string;
Expand All @@ -24,9 +28,18 @@ interface Props {
isNegated?: boolean;
goBack: () => void;
nestedField?: string;
filters: DataSeries['filters'];
}

export function FilterExpanded({ seriesId, field, label, goBack, nestedField, isNegated }: Props) {
export function FilterExpanded({
seriesId,
field,
label,
goBack,
nestedField,
isNegated,
filters: defaultFilters,
}: Props) {
const { indexPattern } = useAppIndexPatternContext();

const [value, setValue] = useState('');
Expand All @@ -37,12 +50,25 @@ export function FilterExpanded({ seriesId, field, label, goBack, nestedField, is

const series = getSeries(seriesId);

const queryFilters: ESFilter[] = [];

defaultFilters?.forEach((qFilter: PersistableFilter | ExistsFilter) => {
if (qFilter.query) {
queryFilters.push(qFilter.query);
}
const asExistFilter = qFilter as ExistsFilter;
if (asExistFilter?.exists) {
queryFilters.push(asExistFilter.exists as QueryDslQueryContainer);
}
});

const { values, loading } = useValuesList({
query: value,
indexPattern,
sourceField: field,
time: series.time,
keepHistory: true,
filters: queryFilters,
});

const filters = series?.filters ?? [];
Expand Down Expand Up @@ -70,6 +96,13 @@ export function FilterExpanded({ seriesId, field, label, goBack, nestedField, is
/>
<EuiSpacer size="s" />
<ListWrapper>
{displayValues.length === 0 && !loading && (
<EuiText>
{i18n.translate('xpack.observability.filters.expanded.noFilter', {
defaultMessage: 'No filters found.',
})}
</EuiText>
)}
{displayValues.map((opt) => (
<Fragment key={opt}>
<EuiFilterGroup fullWidth={true} color="primary">
Expand Down
Loading