Adding Prometheus and Grafana #1837
Conversation
ghost
assigned 
|
Thanks @ritazh! Could you describe the UX for opting into having this extension installed on your acs-engine deployed cluster? |
| # prometheus-grafana Extension | ||
|
|
||
|
|
||
| This is the prometheus-grafana extension. Add this extension to your json file as shown below to automatically enable prometheus and grafana in your new Kubernetes cluster. |
There was a problem hiding this comment.
Could you kindly replace "your json file" with "the api model you pass as input into acs-engine"
|
@ritazh Ignore that, I see the api model pattern documented in the README! |
|
@jackfrancis To opt out of this feature, you simple do not include |
| ``` | ||
| $ kubectl get pods --show-all | ||
|
|
||
| $ NAMESPACE=default |
There was a problem hiding this comment.
Is this configurable? If not it probably should be.
There was a problem hiding this comment.
Currently it is not configurable. Definitely agreed that the end user should be able to specify a namespace.
|
@tstringer Would you add an example https://github.com/Azure/acs-engine/blob/master/examples/extensions/kubernetes.oms.json |
| { | ||
| "name": "prometheus-grafana-k8s", | ||
| "version": "v1", | ||
| "rootURL": "https://raw.githubusercontent.com/ritazh/acs-engine/feat-monitor/" |
There was a problem hiding this comment.
What's the purpose of the rootURL property? We probably want it to refer to something a little more persistent.
There was a problem hiding this comment.
That's where it'll pull the extension itself from. And absolutely that should be pointing to the upstream repo, but we were using @ritazh's fork to test this out.
This is a chicken and the egg thing. What is usually done to test out an extension in a fork before being merged upstream?
There was a problem hiding this comment.
Understood. Because this is an optional property, can we remove it from the examples/ api model, or is there value in configuring your cluster to include this extension with the rootURL property properly defined?
There was a problem hiding this comment.
rootURL is used by dta = strings.Replace(dta, "EXTENSION_URL_REPLACE", extensionProfile.RootURL, -1) in engine.go and EXTENSION_URL_REPLACE is used in template-link.json as:
"templateLink": {
"uri": "EXTENSION_URL_REPLACEextensions/prometheus-grafana-k8s/v1/template.json",
"contentVersion": "1.0.0.0"
},
|
@jackfrancis absolutely, we can add a sample api model config to the repo. That'll be done shortly. |
|
@jchauncey this pr now includes support to specify a non-default namespace to put these components in. Thanks for pointing that out! |
|
@jackfrancis Please could you also review this PR which adds Prometheus instrumentation to Azure API calls kubernetes/kubernetes#56286? |
|
@cosmincojocar do metrics like this - |
|
The metric will record the duration of the Azure API call (in this case list public IP addresses) and also if the call was successful or an error. The metrics are grouped per subscription and resource group. |
What this PR does / why we need it:
Adding Prometheus and Grafana to acs-engine to enable out of the box k8s monitoring experience
Special notes for your reviewer:
We have decided to create a new extension for this purpose to fully utilize the existing Prometheus and Grafana helm charts to manage releases and updates of existing charts. The new
prometheus-grafana-k8sextension installs the helm cli and installs the Prometheus and Grafana charts.cc @tstringer