Multiple managers for Kubernetes

[moolitayer]

WAT ?

• Monitoring = Metrics & Alerts
• Container providers should have a main manager ContainerManager and a secondary manager MonitoringManager linked by parant_manager_id.
• All monitoring workers should move under the MonitoringManager
• There should be two possible implementations for the Monitoring Manager living side by side. Hawkular and Prometheus. The implementation should be different files.

WHY ?

Move closer to other providers where separate entities in the provider are separate managers. Better worker management. Examples:

• If the hawkular authentication status is bad, we should stop the related workers: metrics(currently).
• We are adding a new event listener from hawkular alerts on top of the kubernetes event listener. That would belong under the Monitoring manager.

HOW ?

(In merge order)

• Add monitoring manager manageiq/#15354
• Add kubernetes monitoring manager manageiq-providers-kubernetes/#38
• Add openshift monitoring manager manageiq-providers-openshift/#21
• Adapt manageiq to new managers manageiq/15506
• Add prometheus event collection for Kubernetes under the new manager manageiq-providers-kubernetes/#40
• Same for OepnShift manageiq-providers-openshift/#22
• Switch existing monitoring workers (hawkular metrics collector) to use the new manager
• Add a new event catcher
• Make the kubernetes monitor have two implementations: one for hawkular and one for prometheus (in separate files)

prometheus related work

ManageIQ/manageiq#15473
ManageIQ/manageiq-ui-classic#1640

[cben]

1. Does this involve upgrading k8s and openshift from being managers (EMS) to Providers? Or will there be some sibling manager -> manager link? 2. Will the metrics Endpoints eventually move into the monitoring manager? IOW how will API look? (not opposed to changing, just curious)

[moolitayer]

1. Does this involve upgrading k8s and openshift from being managers (EMS)
   to Providers?
   Or will there be some sibling manager -> manager link?

Turns out both these approaches exist in ManageIQ[1][2][3]. After consultation with @blomquisg and @Ladas the manager + parent_manager route seems more productive for us.

The foreman example uses a provider, the other two a link between managers;
[1] https://github.com/ManageIQ/manageiq-providers-amazon/blob/087f2bb574304ce915945425f8e91d0252fc492a/app/models/manageiq/providers/amazon/cloud_manager.rb#L69
[2] https://github.com/ManageIQ/manageiq-providers-azure/blob/7bfc70f2a76090c80de55d714eb0c90f82bd2384/app/models/manageiq/providers/azure/cloud_manager.rb#L34
[3] https://github.com/ManageIQ/manageiq-providers-foreman/blob/1a9257535c6b60183a07141dc08996411f263e7a/app/models/manageiq/providers/foreman/provider.rb#L23

2. Will the metrics Endpoints eventually move into the monitoring manager?
   IOW how will API look? (not opposed to changing, just curious)

Don't know yet. I will probably start by taking a look at what other providers are doing.

[blomquisg]

It's worth pointing out that we're discussing the Provider/Manager relationship: http://talk.manageiq.org/t/rethinking-providers-and-managers/2494

I'm not sure exactly all the overlaps with this yet. But, I'm sure there's going to be some overlaps.

Also, the notion of controlling workers via Managers will hopefully be a thing of the past as we dive more and more in the ReArch ("H" Release).

/cc @Fryguy @gtanzillo, @agrare, @durandom

[miq-bot]

This issue has been automatically marked as stale because it has not been updated for at least 6 months.

If you can still reproduce this issue on the current release or on master, please reply with all of the information you have about it in order to keep the issue open.

Thank you for all your contributions!

[agrare]

Kubernetes has a monitoring manager now so I'm going to say this is done. Please reopen and update the description if you want to do more here.