Link scope-ui graphs clickable to prometheus queries #2664

rndstr · 2017-06-29T10:53:12Z

scope-app:

Adds -app.metrics-graph cli flag for configuring the base url to
use for graph links; supports :orgID and :query placeholders
Assigns query URLs to existing metrics and creates an empty metric if it's missing

scope-ui:

Extends <CloudFeature /> with option alwaysShow
Adds <CloudLink /> to simplify routing when in cloud vs not in cloud
Links metric graphs in the ui's node details view for all k8s
toplogies and containers so far

TODO

receive design suggestions
deal with overflow items (remove dead code in case not in use)
verify naming of pods in k8s for queries
~~integration tests~~ (postponed)
lint errors

As long as there is no --app.metrics-graph configured, this features is invisible.

Screenshots

prog/main.go

@@ -353,6 +354,7 @@ func setupFlags(flags *flags) {
 	flag.IntVar(&flags.app.memcachedCompressionLevel, "app.memcached.compression", gzip.DefaultCompression, "How much to compress reports stored in memcached.")
 	flag.StringVar(&flags.app.userIDHeader, "app.userid.header", "", "HTTP header to use as userid")
 	flag.BoolVar(&flags.app.externalUI, "app.externalUI", false, "Point to externally hosted static UI assets")
+	flag.StringVar(&flags.app.metricsGraphURL, "app.metrics-graph", "", "Enable extended metrics graph by providing a templated URL (supports :orgID and :query)")


render/detailed/links.go

+	}
+	params := &queryParams{[]cell{{[]string{query}}}}
+
+	bs, err := json.Marshal(params)


2opremio · 2017-06-29T11:52:47Z

Please fix the linting errors.

bboreham · 2017-07-04T09:10:49Z

I note that #2620 anticipates some further additions, e.g. StatefulSet/PetSet. Should we also anticipate them here?

render/detailed/links.go

+	// Prometheus queries for topologies
+	topologyQueries = map[string]map[string]*template.Template{
+		report.Pod: {
+			docker.MemoryUsage:   prepareTemplate(`sum(container_memory_usage_bytes{pod_name="{{.Label}}"})`),


render/detailed/links.go

+	"github.com/ugorji/go/codec"
+)
+
+// MetricLink describes a link referencing a metric.


render/detailed/links.go

+
+var (
+	// As configured by the user
+	metricsGraphURL = ""


davkal

LGTM, only minor issues.

UX has been discussed separately. This is a good first step.

client/app/scripts/components/node-details/node-details-health-link-item.js

+
+  onClick() {
+    trackMixpanelEvent('scope.node.metric.click', {
+      layout: GRAPH_VIEW_MODE,


client/app/scripts/components/node-details/node-details-health.js

@@ -20,7 +21,12 @@ export default class NodeDetailsHealth extends React.Component {
  }

  render() {
-    const metrics = this.props.metrics || [];
+    const {
+      metrics = makeList(),


client/app/scripts/components/node-details/node-details-table-node-metric-link.js

+
+  onClick() {
+    trackMixpanelEvent('scope.node.metric.click', {
+      layout: TABLE_VIEW_MODE,


client/app/scripts/components/node-details/node-details-table-node-metric-link.js

+    });
+  }
+
+  static dismissEvent(ev) {


client/app/scripts/components/node-details/node-details-table.js

@@ -109,7 +109,7 @@ function getSortedNodes(nodes, sortedByHeader, sortedDesc) {
  const getValue = getValueForSortedBy(sortedByHeader);
  const withAndWithoutValues = groupBy(nodes, (n) => {
    const v = getValue(n);
-    return v !== null && v !== undefined ? 'withValues' : 'withoutValues';
+    return !n.valueEmpty && v !== null && v !== undefined ? 'withValues' : 'withoutValues';


client/app/scripts/utils/color-utils.js

+
+export function darkenColor(c) {
+  let color = hsl(c);
+  if (hsl.l < 0.5) {


client/package.json

    "d3-zoom": "1.1.4",
    "dagre": "0.7.4",
    "debug": "2.6.6",
    "filesize": "3.5.9",
+    "filter-invalid-dom-props": "^2.0.0",


client/app/scripts/components/node-details.js

+            <NodeDetailsHealth
+              metrics={details.metrics}
+              topologyId={topologyId}
+              nodeColor={nodeColor}


render/detailed/links.go

+		}
+
+		bs.Reset()
+		if err := tpl.Execute(&bs, summary); err != nil {


render/detailed/links.go

+		return ""
+	}
+
+	if metricsGraphURL[len(metricsGraphURL)-1] != '/' {


client/app/styles/_base.scss

@@ -12,6 +12,10 @@
 // TODO: Remove this line once Service UI CONFIGURE button stops being added to Scope.
 .scope-wrapper .setup-nav-button { display: none; }

+a {
+  text-decoration: none;


rndstr · 2017-08-03T11:36:03Z

@2opremio @davkal @bboreham PTAL?

scope-app: - Adds `-app.metrics-graph` cli flag for configuring the base url to use for graph links; supports `:orgID` and `:query` placeholders - Renders `metric_links` in node detail API response scope-ui: - Extends `<CloudFeature />` with option `alwaysShow` and adds boolean `isCloud` property - Links metric graphs in the ui's node details view for all k8s toplogies; or displays placeholder graph if no metrics available

The initial idea was to keep it separate since the unattached links were also to be displayed distinctively from the metrics. With the new design, unattached links are rendered in the same list as metrics with attached links. Therefore, we treat unattached metric links as an empty metric.

This reverts commit 1a00288.

And more DRY.

Closes #2753

This reverts commit 4e73801.

Fixes #2740

See remix-run/react-router@fbc109c

davkal · 2017-08-15T13:26:00Z

Latest JS changes LGTM. Cant wait to see it on dev!

render/detailed/connections.go

 	output := []Connection{}
 	for row, count := range c.counts {
 		// Use MakeNodeSummary to render the id and label of this node
 		// TODO(paulbellamy): Would be cleaner if we hade just a
 		// MakeNodeID(ns[row.remoteNodeID]). As we don't need the whole summary.
-		summary, _ := MakeNodeSummary(r, ns[row.remoteNodeID])
+		summary, _ := MakeNodeSummary(r, ns[row.remoteNodeID], metricsGraphURL)


render/detailed/links.go

@@ -15,7 +15,7 @@ import (
 // MetricLink describes a URL referencing a metric.
 type MetricLink struct {
 	// References the metric id
-	ID       string `json:"id,omitempty"`
+	ID       string `json:"id"`


rndstr changed the title ~~Link scope-ui graphs clickable to prometheus queries~~ [WIP] Link scope-ui graphs clickable to prometheus queries Jun 29, 2017

2opremio reviewed Jun 29, 2017

View reviewed changes

render/detailed/links.go Outdated

}

params := &queryParams{[]cell{{[]string{query}}}}

bs, err := json.Marshal(params)

This comment was marked as abuse.

Sign in to view

bboreham requested changes Jul 4, 2017

View reviewed changes

rndstr force-pushed the scope-cortex-ui-link-promquery branch 3 times, most recently from 9c3e7c9 to b43dfd2 Compare July 12, 2017 10:47

This was referenced Jul 14, 2017

show historical metrics #1619

Closed

metrics aggregation #1635

Open

Show aggregated process metrics in containers/k8s pods #1257

Open

rndstr force-pushed the scope-cortex-ui-link-promquery branch 2 times, most recently from 388a599 to 9c57959 Compare July 24, 2017 12:09

rndstr mentioned this pull request Jul 24, 2017

Scope/Cortex: get rid of singleton metricsGraphURL variable #2740

Closed

rndstr changed the title ~~[WIP] Link scope-ui graphs clickable to prometheus queries~~ Link scope-ui graphs clickable to prometheus queries Jul 24, 2017

davkal reviewed Jul 27, 2017

View reviewed changes

client/app/scripts/components/node-details.js Outdated

<NodeDetailsHealth

metrics={details.metrics}

topologyId={topologyId}

nodeColor={nodeColor}

This comment was marked as abuse.

Sign in to view

This comment was marked as abuse.

Sign in to view

2opremio reviewed Jul 27, 2017

View reviewed changes

render/detailed/links.go Outdated

}

bs.Reset()

if err := tpl.Execute(&bs, summary); err != nil {

This comment was marked as abuse.

Sign in to view

This comment was marked as abuse.

Sign in to view

2opremio reviewed Jul 27, 2017

View reviewed changes

render/detailed/links.go

return ""

}

if metricsGraphURL[len(metricsGraphURL)-1] != '/' {

This comment was marked as abuse.

Sign in to view

This comment was marked as abuse.

Sign in to view

rndstr force-pushed the scope-cortex-ui-link-promquery branch 3 times, most recently from f1be917 to 21d909d Compare August 1, 2017 10:30

rndstr commented Aug 3, 2017

View reviewed changes

client/app/styles/_base.scss

@@ -12,6 +12,10 @@

// TODO: Remove this line once Service UI CONFIGURE button stops being added to Scope.

.scope-wrapper .setup-nav-button { display: none; }

a {

text-decoration: none;

This comment was marked as abuse.

Sign in to view

rndstr force-pushed the scope-cortex-ui-link-promquery branch 2 times, most recently from 6931aa6 to 69c4430 Compare August 14, 2017 14:04

rndstr added 3 commits August 14, 2017 18:46

Fix linting, cleanup, and add example to flag

14bf0c6

Remove unnecessary fixture dependency of tests

aff3295

rndstr added 22 commits August 14, 2017 18:46

New design for hover states and overflow handling

4985b45

Expand on the prom query source in comments

c895b78

Fix linting errors

6ac2479

Avoid global by passing metricsGraphURL down

b52701b

Update Deployment and DaemonSet queries

42f2054

Switch to migrated rules in kube-state-metrics

3722d55

Pass layout when tracking mixpanel event

9372498

Revert "Avoid global by passing metricsGraphURL down"

b34e111

This reverts commit 1a00288.

Fix query typo

34dd118

Color metric label on hover

484b18e

JS feedback from David

92d4341

Replace templates with simple string replacement

8a07205

Do not mix immutablejs with plain array functions

9c610bf

No preview for overflow health items

60a04db

Use percentage and MB for CPU/Memory urls

ca45b90

And more DRY.

Pass timetravel timestamp to cortex in deeplink

0c167e0

Closes #2753

Remove grouped containers since queries are incorrect

bdab286

Revert "Revert "Avoid global by passing metricsGraphURL down""

bcb0694

This reverts commit 4e73801.

Get rid of metricsGraphURL singleton

f6af178

Fixes #2740

Introduce WebReporter to hold web-related options

4779557

Update tests to new code

b47cfbd

rndstr force-pushed the scope-cortex-ui-link-promquery branch from 69c4430 to b47cfbd Compare August 14, 2017 17:46

rndstr added 2 commits August 14, 2017 19:10

Decode WS with %20 instead +

c059f9a

See remix-run/react-router@fbc109c

Disable net traffic metric links

44c23fa

bboreham approved these changes Aug 15, 2017

View reviewed changes

rndstr merged commit 0d381a3 into master Aug 15, 2017

rndstr deleted the scope-cortex-ui-link-promquery branch August 15, 2017 17:56

bboreham mentioned this pull request Oct 13, 2019

Replace Scope metrics timeseries with Prometheus metrics #3710

Open

Link scope-ui graphs clickable to prometheus queries #2664

Link scope-ui graphs clickable to prometheus queries #2664

Conversation

rndstr commented Jun 29, 2017 • edited Loading

This comment was marked as abuse.

This comment was marked as abuse.

2opremio commented Jun 29, 2017

bboreham commented Jul 4, 2017

This comment was marked as abuse.

This comment was marked as abuse.

This comment was marked as abuse.

This comment was marked as abuse.

This comment was marked as abuse.

This comment was marked as abuse.

This comment was marked as abuse.

This comment was marked as abuse.

This comment was marked as abuse.

This comment was marked as abuse.

davkal left a comment

Choose a reason for hiding this comment

This comment was marked as abuse.

This comment was marked as abuse.

This comment was marked as abuse.

This comment was marked as abuse.

This comment was marked as abuse.

This comment was marked as abuse.

This comment was marked as abuse.

This comment was marked as abuse.

This comment was marked as abuse.

This comment was marked as abuse.

This comment was marked as abuse.

This comment was marked as abuse.

This comment was marked as abuse.

This comment was marked as abuse.

This comment was marked as abuse.

This comment was marked as abuse.

This comment was marked as abuse.

This comment was marked as abuse.

This comment was marked as abuse.

This comment was marked as abuse.

rndstr commented Aug 3, 2017

davkal commented Aug 15, 2017

This comment was marked as abuse.

This comment was marked as abuse.

rndstr commented Jun 29, 2017 •

edited

Loading