[APM] Collect total number of service groups

[kpatticha]

Summary

part of #128621

Collects the total number of service groups.

total field is added in the service_groups object

...
"service_groups": {
   "kuery_fields": [ 
      "service.language.name",
      "agent.name",
      "service.environment"
    ],
  "total": 3
},
...

How to test locally

1. request GET {KIBANA_HOST}/internal/apm/debug-telemetry
2. look for service_groups

[elasticmachine]

Pinging @elastic/apm-ui (Team:APM)

[gbamparop]

LGTM! Thanks for adding tests

[sorenlouv]

It looks like this is actually retrieving the service groups instead of doing a count. Is that not possible with saved objects?

[sorenlouv]

Ah.. I see we retrieve the kuery field. Are we able to reduce the response from ES to just this field then?
(Ideally we'd do a terms agg on kuery to avoid retrieving duplicates).

[sorenlouv]

Doing a terms agg with a size of 500 will also return more unique results (up to 500 to be exact) since it does not include duplicates.

[kpatticha]

For the kuery I think we don't want to avoid duplicates. We want to collect all the possible kuery_fields

[dgieselaar]

What is the added value of two kuery strings that are the same? Are you counting the number of times a field is being used?

[dgieselaar]

One question though, does this collect telemetry for all spaces?

[kpatticha]

I don't understand why you don't want to avoid kuery duplicates

In order to see which fields are most often used.

there's really no point in having service groups with the exact same kuery as others

that's right but there might be service groups that use same fields but different kuery.

For example 2 service groups

{
  ...
  "groupName": "production java",
  "kuery": "service.environement: 'production' or agent.name:'java' ",
  ...
},

{
  ...
  "groupName": "stage envs",
  "kuery": "service.environement: 'stage' ",
  ...
},

we'll collect

kuery_fields: ['service.environement', 'service.environement', 'agent.name',]

[dgieselaar]

@kpatticha Elasticsearch essentially deduplicates non-unique values anyway. I don't think there's a point in storing a single value multiple times.

But more importantly, can you make sure this telemetry is collected for all spaces?

[kpatticha]

oh! I wasn't aware about this.

😱 that's good catch.

savedObjectsClient.find searches in the default space. Let me fix it in a follow up

[kpatticha]

follow up PR - #142856

[sorenlouv]

Looks ok. I added a comment but it might be more of an issue with saved objects than anything else.

[kibana-ci]

💚 Build Succeeded

• Buildkite Build
• Commit: 72b91db

Metrics [docs]

✅ unchanged

History

• 💔 Build #78473 failed fbe2b18
• 💔 Build #78442 failed 8ba068b

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream