[stable/grafana] Support Provisioner Configuration #3514

BugRoger · 2018-02-01T14:05:17Z

This PR extends the stable/grafana chart to allow for outside configuration of provisioners.

With Grafana 5 it will be possible to manage datasources and dashboards using provisioners. Adding a provisioner is done by adding one or more yaml configuration files in the provisioning/datasources and provisioning/dashboards directory. See also: http://docs.grafana.org/administration/provisioning/

For datasources the file can contain a list of datasources that will be added or updated during start-up. If the datasource already exists, Grafana will update it to match the configuration file.

For dashboards the files contain the configuration for provisioning from external resources. It currently only support reading dashboards from file but more provisioners are in the works.

This functionality partly clashes with the jobs provided by this Helm chart. In order to support both Grafana 4/5 in the same chart, I think, it is a sensible approach to allow for both provisioning mechanisms. In the absence of more sophisticated provisioners (that allow the decoupling from Helm and dashboards) it is anyway required to load the dashboard from the filesystem and inject it using the existing mechanism.

BugRoger · 2018-02-01T15:05:28Z

/assign @linki

BugRoger · 2018-02-04T11:42:23Z

/assign @prydonius

BugRoger · 2018-02-04T11:43:01Z

/unassign @linki

NicolasT · 2018-02-16T22:04:50Z

Looking forward to this to land. Too bad though that creating a default datasource for e.g. Prometheus deployed by the stable Chart (in a Chart having both Grafana and Prometheus as requirements) still won't be possible using this approach, because its service name/DNS entry depends on the Helm release name 😞

monotek · 2018-02-19T15:11:26Z

@BugRoger
Looking forward to see this merged. Would you mind to fix the the merge conflict?

k8s-ci-robot · 2018-02-19T19:23:49Z

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: BugRoger
To fully approve this pull request, please assign additional approvers.
We suggest the following additional approver: prydonius

Assign the PR to them by writing /assign @prydonius in a comment when ready.

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

BugRoger · 2018-02-19T19:35:45Z

Fixed. It was just some other change increasing the version

BugRoger · 2018-02-19T19:37:34Z

/assign @prydonius

pesimon · 2018-03-01T12:38:09Z

Can confirm that it works like a charm and grafana 5.0.0 was just released
https://github.com/grafana/grafana/releases

unguiculus · 2018-03-01T16:48:02Z

stable/grafana/templates/deployment.yaml

@@ -83,6 +83,14 @@ spec:
          resources:
 {{ toYaml .Values.server.resources | indent 12 }}
          volumeMounts:
+            {{- if .Values.provisioningDatasourcesFiles }}
+            - name: provisioning-volume 
+              mountPath: {{ default "/var/lib/grafana/provisioning" .Values.server.provisioningLocalPath }}/datasources


Don't use the default function. Defaults come from values.yaml anyways which is better. Apply to all your changes.

I figured this is common practice in the chart. So, let's leave it for now. Would be cool to get that fixed in a separate PR.

jdumars · 2018-03-02T16:55:03Z

/ok-to-test

monotek · 2018-03-02T17:38:30Z

/retest

jpds · 2018-03-05T14:08:30Z

This merge proposal doesn't actually change the default version to 5.0.0 (relevant issue: #3937).

sstarcher · 2018-03-08T16:59:41Z

This should bump the version to 5.0.1 and add the AppVersion to the charts.yaml

unguiculus · 2018-03-09T20:54:53Z

stable/grafana/templates/deployment.yaml

@@ -83,6 +83,14 @@ spec:
          resources:
 {{ toYaml .Values.server.resources | indent 12 }}
          volumeMounts:
+            {{- if .Values.provisioningDatasourcesFiles }}
+            - name: provisioning-volume 
+              mountPath: {{ default "/var/lib/grafana/provisioning" .Values.server.provisioningLocalPath }}/datasources


I figured this is common practice in the chart. So, let's leave it for now. Would be cool to get that fixed in a separate PR.

sstarcher · 2018-03-22T10:51:27Z

@BugRoger could you fix the linting issues from the circleci build

richerve · 2018-03-22T22:54:18Z

@BugRoger are you still working on this, if not I can help

richerve · 2018-03-23T09:06:37Z

In my opinion is better to bump the minor version and remove at least the support to configure datasources using the Job, it is a hack that is difficult to maintain and not needed anymore. In general the dashboards Job will also not be needed but I think is handy to have the functionality to import remote dashboards.

About bumping the minor version (or major) of the Chart, Grafana is not a project that has Long Term Support releases so keeping a Chart that is compatible with version 4.6.x and another compatible with >= 5 is not a big issue I think, eventually the Chart versions compatible with Grafana <= 4 will not be updated anymore and this will happen sooner than later.

I want to use this Chart on the kube_prometheus "meta Chart" in prometheus-operator/prometheus-operator#1127, they are maintaining what is basically a copy of the stable Chart but adding grafana-watcher that essentially serves the same purpose as the Jobs here, both cases are hacks that with Grafana >= 5 are not required.

BugRoger · 2018-03-23T14:38:59Z

Fixed the linter nitpicks. Bumped the version. Rebased on master.

richerve · 2018-03-28T09:24:34Z

Any way to move this forward?

monotek · 2018-03-28T15:50:28Z

@BugRoger

Please update to the newest Grafana 5.0.4 Docker image as the older images don't work with Kubernetes 1.7.14, 1.8.9, or 1.9.4. This would probably also fix the build.

See: grafana/grafana-docker#140

BugRoger · 2018-03-28T18:53:13Z

Rebased. Updated Image to 5.0.4. Added appVersion to make Linter happy. Tests are green.

richerve · 2018-03-29T21:48:00Z

@BugRoger, sadly again conflicting files. I think this should be coordinated with someone that can merge, do the fix and immediately apply it. I'm waiting for this PR to be merged to work on the integration in https://github.com/coreos/kube-prometheus

knopkodav · 2018-03-30T18:52:48Z

The same issue on 1.9.6-gke.0

matthope · 2018-04-04T08:14:11Z

How does this compare to #4541 ?

richerve · 2018-04-04T10:01:08Z

@matthope In concept both are the same in approach are different, this PR include changes to support the new Grafana >= 5 config but also maintains backward compatibility. The PR on #4541 removes all backwards compatibility.

@BugRoger friendly ping about updating the branch.

rtluckie · 2018-04-05T14:07:54Z

please check out #4713 before merging.

BugRoger · 2018-04-09T12:27:53Z

It seems i can't fix this. There's a deadlock between the different checks.

The linter requires me to set the version to version in chart.yaml to 0.8.5. This causes a conflict against master. That fails the e2e tests.

I can either resolve the conflict or fix the version.

sstarcher · 2018-04-09T12:32:00Z

master is already at 0.8.5. You need to rebase and bump it to 0.8.6 or 0.9.0

BugRoger · 2018-04-09T16:03:47Z

Ah, rebased and bumped the version again....

unguiculus · 2018-04-10T15:21:20Z

/hold

unguiculus · 2018-04-10T15:21:36Z

I'm inclined to close this in favor of #4713.

sstarcher · 2018-04-10T15:36:47Z

@unguiculus if #4713 is merged it would make this PR no longer needed since #4713 is such a large overhaul and simplification.

I have not tested #4713, but the changes themselves are heading in a good direction.

unguiculus · 2018-04-10T17:27:21Z

Closing in favor of #4713. Thanks @BugRoger.

k8s-ci-robot added cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Feb 1, 2018

BugRoger force-pushed the master branch from a4ba3d0 to 31faaf0 Compare February 1, 2018 14:08

k8s-ci-robot assigned linki Feb 1, 2018

k8s-ci-robot assigned prydonius Feb 4, 2018

k8s-ci-robot unassigned linki Feb 4, 2018

unguiculus suggested changes Mar 1, 2018

View reviewed changes

k8s-ci-robot removed the needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. label Mar 2, 2018

jgoclawski mentioned this pull request Mar 9, 2018

[stable/grafana] Improve importing dashboards. #3577

Merged

unguiculus suggested changes Mar 9, 2018

View reviewed changes

unguiculus approved these changes Mar 9, 2018

View reviewed changes

BugRoger force-pushed the master branch from f88ce8e to cfb2967 Compare March 23, 2018 14:38

BugRoger force-pushed the master branch from cfb2967 to 82adfa0 Compare March 23, 2018 14:41

BugRoger force-pushed the master branch 4 times, most recently from 9f99687 to fc91d68 Compare March 28, 2018 18:49

jgoclawski mentioned this pull request Apr 8, 2018

Grafana version 0.8.5 fails to deploy dashboards using dashboard-import-job on k8s v1.9.4 #4708

Closed

BugRoger force-pushed the master branch 3 times, most recently from a856b34 to baa8cfc Compare April 9, 2018 12:21

richerve mentioned this pull request Apr 9, 2018

[stable/grafana] Utilize 5.x datasource and dashboard import tooling and general refactor #4713

Merged

BugRoger force-pushed the master branch from baa8cfc to 99be764 Compare April 9, 2018 16:01

extends chart to allow configuration of provisioners

b3559aa

BugRoger force-pushed the master branch from 99be764 to b3559aa Compare April 9, 2018 16:02

k8s-ci-robot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Apr 10, 2018

unguiculus closed this Apr 10, 2018

[stable/grafana] Support Provisioner Configuration #3514

[stable/grafana] Support Provisioner Configuration #3514

Conversation

BugRoger commented Feb 1, 2018

BugRoger commented Feb 1, 2018

BugRoger commented Feb 4, 2018

BugRoger commented Feb 4, 2018

NicolasT commented Feb 16, 2018

monotek commented Feb 19, 2018

k8s-ci-robot commented Feb 19, 2018

BugRoger commented Feb 19, 2018

BugRoger commented Feb 19, 2018

pesimon commented Mar 1, 2018

unguiculus Mar 1, 2018

Choose a reason for hiding this comment

unguiculus Mar 9, 2018

Choose a reason for hiding this comment

jdumars commented Mar 2, 2018

monotek commented Mar 2, 2018

jpds commented Mar 5, 2018

sstarcher commented Mar 8, 2018

unguiculus Mar 9, 2018

Choose a reason for hiding this comment

sstarcher commented Mar 22, 2018

richerve commented Mar 22, 2018

richerve commented Mar 23, 2018 • edited Loading

BugRoger commented Mar 23, 2018

richerve commented Mar 28, 2018

monotek commented Mar 28, 2018 • edited Loading

BugRoger commented Mar 28, 2018

richerve commented Mar 29, 2018

knopkodav commented Mar 30, 2018

matthope commented Apr 4, 2018

richerve commented Apr 4, 2018

rtluckie commented Apr 5, 2018

BugRoger commented Apr 9, 2018

sstarcher commented Apr 9, 2018 • edited Loading

BugRoger commented Apr 9, 2018

unguiculus commented Apr 10, 2018

unguiculus commented Apr 10, 2018

sstarcher commented Apr 10, 2018

unguiculus commented Apr 10, 2018

richerve commented Mar 23, 2018 •

edited

Loading

monotek commented Mar 28, 2018 •

edited

Loading

sstarcher commented Apr 9, 2018 •

edited

Loading