[Telemetry] Fully migrate to new platform

[Bamieh]

Telemetry migrated to new platform #35403, but is using the old plugin system until the following features are introduced in the new platform:

• Dealing with plugin Hacks.

   uiExports: {
     hacks: [
       'plugins/telemetry/hacks/telemetry_opt_in',
       'plugins/telemetry/hacks/telemetry_trigger',
     ],
   }
  

• Core exposing Banner

• Replace legacy collectorSet with the NP metrics plugin

   init(server) {
     server.usage.collectorSet(...)
   }
  

  collectorSets in the New Platform #46924

• Dealing with management sections

   uiExports: {
     managementSections: ['plugins/telemetry/views/management'],
   }
  

• SavedObjects server-side client Server saved objects client through request context #44143

CC @skaapgif

[elasticmachine]

Pinging @elastic/kibana-stack-services

[elasticmachine]

Pinging @elastic/kibana-platform

[joshdover]

* [ ]  Dealing with plugin Hacks.
  ```
   uiExports: {
     hacks: [
       'plugins/telemetry/hacks/telemetry_opt_in',
       'plugins/telemetry/hacks/telemetry_trigger',
     ],
   }
  ```

All plugins in the new platform run on all pages, meaning that all plugins have their setup and start methods called. Plugins are not applications, but they may register zero or more applications and only one application is ever displayed on the page at a time. Is there something else that is needed?

* [ ]  Core exposing collector sets
  ```
   init(server) {
     server.usage.collectorSet(...)
   }
  ```
  
  
  If `collectorSet` is exposed from the telemetry plugin, then telemetry will be required from many plugins; Do you think this is OK? or do we keep them separate like the current situation?

I would expect telemetry to be an optionalPlugin dependency for all plugins that integrate with it. That way an admin could disable it and be sure that telemetry code is not running at all without breaking any plugins.

[Bamieh]

For usage collectors, I believe its a good idea to keep them in core as they are being used by almost every plugin to register collectors, plus it is used by monitoring to track kibana stats (for both telemetry and monitoring) and used by the kibana_stats beat that hits the /stats endpoint to track kibana stats.

It can be moved to telemetry but I'm not sure if we'd want that?

[joshdover]

Any reason it couldn't be an OSS plugin that both telemetry and monitoring could depend on?

[Bamieh]

I have a PR open to move telemetry to OSS #45769 so we can put it there, but my concern is that we're gonna require telemetry as optional dep in every plugin so that might not be ideal + we'll have to change lots of code in lots of places.

[Bamieh]

Places where usage.collectorSet is consumed:

• '/api/stats' this is used by kibana metric beats to collect kibana stats and report them to the user. Located in src/legacy/server/status/routes/api/register_stats.js
• monitoring plugin located in xpack.
• telemetry plugin soon in src.

These are the places collectors are consumed. Collectors are registered in many plugins. Ideally I'd vote to keep it separate from telemetry as we can potentially use them in more places other than telemetry in addition to metric beat and monitoring.

Even if the user disables telemetry, i believe we still want to have those collectors running as they do now as they might be consumed in other places.

[rudolf]

I've updated the description with a link to our collectorSet discussion in #46924

[elasticmachine]

Pinging @elastic/pulse (Team:Pulse)