[APM] Enable metric-powered ui

[sorenlouv]

Closes #92024

[elasticmachine]

Pinging @elastic/apm-ui (Team:apm)

[smith]

[sorenlouv]

This seems like too big a difference to be right.

I'm wondering how trustworthy the metric documents in the archive are. Perhaps some improvements were made to metric doc generation after we built the archive?

@dgieselaar any idea?

[dgieselaar]

is this the bug we've found with _doc_count?

[sorenlouv]

Not sure. Has that been fixed and would rebuilding the archives fix this?

[dgieselaar]

It's an es bug on the search / aggregation side, so rebuilding the archive won't matter. Not sure what the status is of the bug fix.

[dgieselaar]

It's fixed and backported to 7.x 11 days ago so I assumed it made 7.14

[dgieselaar]

It's a missing filter query, it aggregates over all metric documents, not just the transaction metrics one. I've pushed a fix.

[sorenlouv]

@elasticmachine merge upstream

[sorenlouv]

Don't we have a metricset.name that we can automatically add, similar to how we filter by processor.event when using the apm event client?

[sorenlouv]

Eg changing getProcessorEventForAggregatedTransactions to getApmEventForAggregatedTransactions:

export function getApmEventForAggregatedTransactions(
  searchAggregatedTransactions: boolean
): MetricsetName.transactionMetric | ProcessorEvent.transaction {
  return searchAggregatedTransactions
    ? MetricsetName.transactionMetric
    : ProcessorEvent.transaction;
}

and use it like:

const params = {
    apm: {
      events: [
        getApmEventForAggregatedTransactions(
          searchAggregatedTransactions
        )
      ],
    },
    body: {},
}

apmEventClient.search(operationName, params)

where apm.events is a union of ProcessorEvent and MetricsetName

[dgieselaar]

Opened #105713 to track this.

[kibanamachine]

💛 Build succeeded, but was flaky

• continuous-integration/kibana-ci/pull-request

• Commit: 43e551b

• Storybooks Preview

• Flaky suites:

  • xpack-kibana-ciGroup6

---

Test Failures

Kibana Pipeline / general / X-Pack API Integration Tests.x-pack/test/api_integration/apis/security_solution/hosts·ts.apis SecuritySolution Endpoints hosts Make sure that we get Last Seen for a Host without docValueFields

Link to Jenkins

Standard Out

Failed Tests Reporter:
  - Test has not failed recently on tracked branches

[00:00:00]       │
[00:00:00]         └-: apis
[00:00:00]           └-> "before all" hook in "apis"
[00:07:54]           └-: SecuritySolution Endpoints
[00:07:54]             └-> "before all" hook in "SecuritySolution Endpoints"
[00:08:25]             └-: hosts
[00:08:25]               └-> "before all" hook for "Make sure that we get Hosts Table data"
[00:08:25]               └-> "before all" hook for "Make sure that we get Hosts Table data"
[00:08:25]                 │ info [x-pack/test/functional/es_archives/auditbeat/hosts] Loading "mappings.json"
[00:08:25]                 │ info [x-pack/test/functional/es_archives/auditbeat/hosts] Loading "data.json.gz"
[00:08:25]                 │ info [o.e.c.m.MetadataCreateIndexService] [node-01] [auditbeat-8.0.0-2019.02.19-000001] creating index, cause [api], templates [], shards [1]/[1]
[00:08:25]                 │ info [x-pack/test/functional/es_archives/auditbeat/hosts] Created index "auditbeat-8.0.0-2019.02.19-000001"
[00:08:25]                 │ debg [x-pack/test/functional/es_archives/auditbeat/hosts] "auditbeat-8.0.0-2019.02.19-000001" settings {"index":{"lifecycle":{"name":"auditbeat-8.0.0","rollover_alias":"auditbeat-8.0.0"},"mapping":{"total_fields":{"limit":"10000"}},"number_of_replicas":"1","number_of_shards":"1","query":{"default_field":["tags","message","agent.version","agent.name","agent.type","agent.id","agent.ephemeral_id","client.address","client.mac","client.domain","client.geo.continent_name","client.geo.country_name","client.geo.region_name","client.geo.city_name","client.geo.country_iso_code","client.geo.region_iso_code","client.geo.name","cloud.provider","cloud.availability_zone","cloud.region","cloud.instance.id","cloud.instance.name","cloud.machine.type","cloud.account.id","container.runtime","container.id","container.image.name","container.image.tag","container.name","destination.address","destination.mac","destination.domain","destination.geo.continent_name","destination.geo.country_name","destination.geo.region_name","destination.geo.city_name","destination.geo.country_iso_code","destination.geo.region_iso_code","destination.geo.name","ecs.version","error.id","error.message","error.code","event.id","event.kind","event.category","event.action","event.outcome","event.type","event.module","event.dataset","event.hash","event.timezone","file.path","file.target_path","file.extension","file.type","file.device","file.inode","file.uid","file.owner","file.gid","file.group","file.mode","group.id","group.name","host.hostname","host.name","host.id","host.mac","host.type","host.architecture","host.os.platform","host.os.name","host.os.full","host.os.family","host.os.version","host.os.kernel","host.geo.continent_name","host.geo.country_name","host.geo.region_name","host.geo.city_name","host.geo.country_iso_code","host.geo.region_iso_code","host.geo.name","http.request.method","http.request.body.content","http.request.referrer","http.response.body.content","http.version","log.level","network.name","network.type","network.iana_number","network.transport","network.application","network.protocol","network.direction","network.community_id","observer.mac","observer.hostname","observer.vendor","observer.version","observer.serial_number","observer.type","observer.os.platform","observer.os.name","observer.os.full","observer.os.family","observer.os.version","observer.os.kernel","observer.geo.continent_name","observer.geo.country_name","observer.geo.region_name","observer.geo.city_name","observer.geo.country_iso_code","observer.geo.region_iso_code","observer.geo.name","organization.name","organization.id","os.platform","os.name","os.full","os.family","os.version","os.kernel","process.name","process.args","process.executable","process.title","process.working_directory","server.address","server.mac","server.domain","server.geo.continent_name","server.geo.country_name","server.geo.region_name","server.geo.city_name","server.geo.country_iso_code","server.geo.region_iso_code","server.geo.name","service.id","service.name","service.type","service.state","service.version","service.ephemeral_id","source.address","source.mac","source.domain","source.geo.continent_name","source.geo.country_name","source.geo.region_name","source.geo.city_name","source.geo.country_iso_code","source.geo.region_iso_code","source.geo.name","url.original","url.full","url.scheme","url.domain","url.path","url.query","url.fragment","url.username","url.password","user.id","user.name","user.full_name","user.email","user.hash","user.group.id","user.group.name","user_agent.original","user_agent.name","user_agent.version","user_agent.device.name","user_agent.os.platform","user_agent.os.name","user_agent.os.full","user_agent.os.family","user_agent.os.version","user_agent.os.kernel","agent.hostname","error.type","cloud.project.id","kubernetes.pod.name","kubernetes.pod.uid","kubernetes.namespace","kubernetes.node.name","kubernetes.container.name","kubernetes.container.image","file.origin","raw","file.selinux.user","file.selinux.role","file.selinux.domain","file.selinux.level","user.audit.id","user.audit.name","user.effective.id","user.effective.name","user.effective.group.id","user.effective.group.name","user.filesystem.id","user.filesystem.name","user.filesystem.group.id","user.filesystem.group.name","user.saved.id","user.saved.name","user.saved.group.id","user.saved.group.name","user.selinux.user","user.selinux.role","user.selinux.domain","user.selinux.level","user.selinux.category","source.path","destination.path","auditd.message_type","auditd.session","auditd.result","auditd.summary.actor.primary","auditd.summary.actor.secondary","auditd.summary.object.type","auditd.summary.object.primary","auditd.summary.object.secondary","auditd.summary.how","auditd.paths.inode","auditd.paths.dev","auditd.paths.obj_user","auditd.paths.obj_role","auditd.paths.obj_domain","auditd.paths.obj_level","auditd.paths.objtype","auditd.paths.ouid","auditd.paths.rdev","auditd.paths.nametype","auditd.paths.ogid","auditd.paths.item","auditd.paths.mode","auditd.paths.name","auditd.data.action","auditd.data.minor","auditd.data.acct","auditd.data.addr","auditd.data.cipher","auditd.data.id","auditd.data.entries","auditd.data.kind","auditd.data.ksize","auditd.data.spid","auditd.data.arch","auditd.data.argc","auditd.data.major","auditd.data.unit","auditd.data.table","auditd.data.terminal","auditd.data.grantors","auditd.data.direction","auditd.data.op","auditd.data.tty","auditd.data.syscall","auditd.data.data","auditd.data.family","auditd.data.mac","auditd.data.pfs","auditd.data.items","auditd.data.a0","auditd.data.a1","auditd.data.a2","auditd.data.a3","auditd.data.hostname","auditd.data.lport","auditd.data.rport","auditd.data.exit","auditd.data.fp","auditd.data.laddr","auditd.data.sport","auditd.data.capability","auditd.data.nargs","auditd.data.new-enabled","auditd.data.audit_backlog_limit","auditd.data.dir","auditd.data.cap_pe","auditd.data.model","auditd.data.new_pp","auditd.data.old-enabled","auditd.data.oauid","auditd.data.old","auditd.data.banners","auditd.data.feature","auditd.data.vm-ctx","auditd.data.opid","auditd.data.seperms","auditd.data.seresult","auditd.data.new-rng","auditd.data.old-net","auditd.data.sigev_signo","auditd.data.ino","auditd.data.old_enforcing","auditd.data.old-vcpu","auditd.data.range","auditd.data.res","auditd.data.added","auditd.data.fam","auditd.data.nlnk-pid","auditd.data.subj","auditd.data.a[0-3]","auditd.data.cgroup","auditd.data.kernel","auditd.data.ocomm","auditd.data.new-net","auditd.data.permissive","auditd.data.class","auditd.data.compat","auditd.data.fi","auditd.data.changed","auditd.data.msg","auditd.data.dport","auditd.data.new-seuser","auditd.data.invalid_context","auditd.data.dmac","auditd.data.ipx-net","auditd.data.iuid","auditd.data.macproto","auditd.data.obj","auditd.data.ipid","auditd.data.new-fs","auditd.data.vm-pid","auditd.data.cap_pi","auditd.data.old-auid","auditd.data.oses","auditd.data.fd","auditd.data.igid","auditd.data.new-disk","auditd.data.parent","auditd.data.len","auditd.data.oflag","auditd.data.uuid","auditd.data.code","auditd.data.nlnk-grp","auditd.data.cap_fp","auditd.data.new-mem","auditd.data.seperm","auditd.data.enforcing","auditd.data.new-chardev","auditd.data.old-rng","auditd.data.outif","auditd.data.cmd","auditd.data.hook","auditd.data.new-level","auditd.data.sauid","auditd.data.sig","auditd.data.audit_backlog_wait_time","auditd.data.printer","auditd.data.old-mem","auditd.data.perm","auditd.data.old_pi","auditd.data.state","auditd.data.format","auditd.data.new_gid","auditd.data.tcontext","auditd.data.maj","auditd.data.watch","auditd.data.device","auditd.data.grp","auditd.data.bool","auditd.data.icmp_type","auditd.data.new_lock","auditd.data.old_prom","auditd.data.acl","auditd.data.ip","auditd.data.new_pi","auditd.data.default-context","auditd.data.inode_gid","auditd.data.new-log_passwd","auditd.data.new_pe","auditd.data.selected-context","auditd.data.cap_fver","auditd.data.file","auditd.data.net","auditd.data.virt","auditd.data.cap_pp","auditd.data.old-range","auditd.data.resrc","auditd.data.new-range","auditd.data.obj_gid","auditd.data.proto","auditd.data.old-disk","auditd.data.audit_failure","auditd.data.inif","auditd.data.vm","auditd.data.flags","auditd.data.nlnk-fam","auditd.data.old-fs","auditd.data.old-ses","auditd.data.seqno","auditd.data.fver","auditd.data.qbytes","auditd.data.seuser","auditd.data.cap_fe","auditd.data.new-vcpu","auditd.data.old-level","auditd.data.old_pp","auditd.data.daddr","auditd.data.old-role","auditd.data.ioctlcmd","auditd.data.smac","auditd.data.apparmor","auditd.data.fe","auditd.data.perm_mask","auditd.data.ses","auditd.data.cap_fi","auditd.data.obj_uid","auditd.data.reason","auditd.data.list","auditd.data.old_lock","auditd.data.bus","auditd.data.old_pe","auditd.data.new-role","auditd.data.prom","auditd.data.uri","auditd.data.audit_enabled","auditd.data.old-log_passwd","auditd.data.old-seuser","auditd.data.per","auditd.data.scontext","auditd.data.tclass","auditd.data.ver","auditd.data.new","auditd.data.val","auditd.data.img-ctx","auditd.data.old-chardev","auditd.data.old_val","auditd.data.success","auditd.data.inode_uid","auditd.data.removed","auditd.data.socket.port","auditd.data.socket.saddr","auditd.data.socket.addr","auditd.data.socket.family","auditd.data.socket.path","geoip.continent_name","geoip.city_name","geoip.region_name","geoip.country_iso_code","hash.blake2b_256","hash.blake2b_384","hash.blake2b_512","hash.md5","hash.sha1","hash.sha224","hash.sha256","hash.sha384","hash.sha3_224","hash.sha3_256","hash.sha3_384","hash.sha3_512","hash.sha512","hash.sha512_224","hash.sha512_256","hash.xxh64","event.origin","user.entity_id","user.terminal","process.entity_id","socket.entity_id","system.audit.host.timezone.name","system.audit.host.hostname","system.audit.host.id","system.audit.host.architecture","system.audit.host.mac","system.audit.host.os.platform","system.audit.host.os.name","system.audit.host.os.family","system.audit.host.os.version","system.audit.host.os.kernel","system.audit.package.entity_id","system.audit.package.name","system.audit.package.version","system.audit.package.release","system.audit.package.arch","system.audit.package.license","system.audit.package.summary","system.audit.package.url","system.audit.user.name","system.audit.user.uid","system.audit.user.gid","system.audit.user.dir","system.audit.user.shell","system.audit.user.user_information","system.audit.user.password.type","fields.*"]},"refresh_interval":"5s"}}
[00:08:25]                 │ info [x-pack/test/functional/es_archives/auditbeat/hosts] Indexed 1751 docs into "auditbeat-8.0.0-2019.02.19-000001"
[00:08:25]                 │ info [x-pack/test/functional/es_archives/auditbeat/hosts] Indexed 1 docs into "winlogbeat-8.0.0-2019.02.19-000001"
[00:08:25]               └-> Make sure that we get Hosts Table data
[00:08:25]                 └-> "before each" hook: global before each for "Make sure that we get Hosts Table data"
[00:08:25]                 └- ✓ pass  (42ms) "apis SecuritySolution Endpoints hosts Make sure that we get Hosts Table data"
[00:08:25]               └-> Make sure that pagination is working in Hosts Table query
[00:08:25]                 └-> "before each" hook: global before each for "Make sure that pagination is working in Hosts Table query"
[00:08:25]                 └- ✓ pass  (38ms) "apis SecuritySolution Endpoints hosts Make sure that pagination is working in Hosts Table query"
[00:08:25]               └-> Make sure that we get Host details data
[00:08:25]                 └-> "before each" hook: global before each for "Make sure that we get Host details data"
[00:08:25]                 └- ✓ pass  (45ms) "apis SecuritySolution Endpoints hosts Make sure that we get Host details data"
[00:08:25]               └-> Make sure that we get First Seen for a Host without docValueFields
[00:08:25]                 └-> "before each" hook: global before each for "Make sure that we get First Seen for a Host without docValueFields"
[00:08:25]                 └- ✓ pass  (31ms) "apis SecuritySolution Endpoints hosts Make sure that we get First Seen for a Host without docValueFields"
[00:08:25]               └-> Make sure that we get Last Seen for a Host without docValueFields
[00:08:25]                 └-> "before each" hook: global before each for "Make sure that we get Last Seen for a Host without docValueFields"
[00:08:25]                 └- ✖ fail: apis SecuritySolution Endpoints hosts Make sure that we get Last Seen for a Host without docValueFields
[00:08:25]                 │      Error: expected null to sort of equal '2019-02-19T20:42:33.561Z'
[00:08:25]                 │       at Assertion.assert (/dev/shm/workspace/parallel/24/kibana/node_modules/@kbn/expect/expect.js:100:11)
[00:08:25]                 │       at Assertion.eql (/dev/shm/workspace/parallel/24/kibana/node_modules/@kbn/expect/expect.js:244:8)
[00:08:25]                 │       at Context.<anonymous> (test/api_integration/apis/security_solution/hosts.ts:177:45)
[00:08:25]                 │       at Object.apply (/dev/shm/workspace/parallel/24/kibana/node_modules/@kbn/test/target_node/functional_test_runner/lib/mocha/wrap_function.js:87:16)
[00:08:25]                 │ 
[00:08:25]                 │ 

Stack Trace

Error: expected null to sort of equal '2019-02-19T20:42:33.561Z'
    at Assertion.assert (/dev/shm/workspace/parallel/24/kibana/node_modules/@kbn/expect/expect.js:100:11)
    at Assertion.eql (/dev/shm/workspace/parallel/24/kibana/node_modules/@kbn/expect/expect.js:244:8)
    at Context.<anonymous> (test/api_integration/apis/security_solution/hosts.ts:177:45)
    at Object.apply (/dev/shm/workspace/parallel/24/kibana/node_modules/@kbn/test/target_node/functional_test_runner/lib/mocha/wrap_function.js:87:16) {
  actual: null,
  expected: '2019-02-19T20:42:33.561Z',
  showDiff: true
}

---

Metrics [docs]

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
apm	4.3MB	4.3MB	-212.0B

History

• 💔 Build #138752 failed 3a87e90
• 💔 Build #138100 failed 20259c2
• 💚 Build #137740 succeeded b116870
• 💚 Build #137411 succeeded 41eaa2e
• 💔 Build #137298 failed 5f457a9

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @dgieselaar

[kibanamachine]

💚 Backport successful

Status	Branch	Result
✅	7.x

This backport PR will be merged automatically after passing CI.