[docs][chuck] Example YAML generation using ksonnet and kubecfg #58

jesscodez · 2017-08-18T17:42:56Z

This is heavily based off of @chuckha 's PR #52. (See the latest commit for the changes that I made).
cc @timothysc @kensimon PTAL

It is dependent on the pending merge of another PR I made yesterday (ksonnet/ksonnet-lib#67), which adds kubecfg to the ksonnet-lib Docker image. (The new image will also need to be pushed to the public registry).

This PR incorporates that new kubecfg functionality to:

Output YAML instead of JSON
Output pretty-printed YAML/JSON for the multiline values in the ConfigMaps
Generate an all-in-one YAML manifest (as well as component manifests)
Use an extVar (env variable $RBAC_ENABLED) to determine if RBAC resource definitions should be included in the generated YAML files.

It also fixes some minor bugs from Chuck's commit, and I have confirmed that it works on clusters with and without RBAC apigroups enabled.

jesscodez · 2017-08-18T17:43:24Z

Oh I also updated docs accordingly, let me know if you have any changes in mind.

jesscodez · 2017-08-18T17:49:16Z

Makefile

 # Note the only reason we are creating this is because upstream
 # does not yet publish a released e2e container
 # https://github.com/kubernetes/kubernetes/issues/47920

+EXAMPLE_FILES = $(wildcard examples/ksonnet/components/*.jsonnet)
+EXAMPLE_OUTPUT = examples/quickstart/aggregate.yaml $(patsubst examples/ksonnet/components/%.jsonnet,examples/quickstart/components/%.yaml,$(EXAMPLE_FILES))
+KSONNET_BUILD_IMAGE = ksonnet:test # TODO: CHANGE ONCE NEW KSONNET IMAGE IS PUSHED


reminder to self

chuckha · 2017-08-18T18:00:17Z

examples/ksonnet/aggregate.jsonnet

+local configmaps = import "examples/ksonnet/components/10-configmaps.jsonnet";
+local pod = import "examples/ksonnet/components/20-pod.jsonnet";
+
+rbac + configmaps + pod


does kubecfg take a k8s v1.List object and turn it into --- delimited yaml objects? :mindblown:

heh I was so pleasantly surprised with all of kubecfg's functionality

timothysc · 2017-08-21T13:54:51Z

@chuckha @kensimon I'll let you two see this through.

ncdc · 2017-08-21T15:26:03Z

README.md

@@ -32,19 +32,31 @@ This guide executes a Sonobuoy run on your cluster, and records the following re

 * *You should have `kubectl` installed.* If not, follow the instructions for [installing via Homebrew (MacOS)][6] or [building the binary (Linux)][7].

+* *You should have `docker` installed (minimum 17.05).* Docker images are used to generate the YAML examples, based on your cluster's RBAC settings.


Why the 17.05 requirement?

@ncdc I saw that on Docker multi-stage build page (https://docs.docker.com/engine/userguide/eng-image/multistage-build/), but if that version is a potential blocker for folks, I can repurpose the Dockerfile?

err clarification: the Dockerfile for ksonnet

Oh, you're doing multistage builds. I know Fedora's newest is 1.13 (still on the previous versioning scheme). Might be an issue.

ah ok, thanks for the heads up---the ksonnet PR is on hold til Alex gets back tomorrow, but if I finish the Ark PR, I'll come back and try to take out that dependency

We're not doing multi-stage builds yet, please remove the requirement.

ah my bad -- the 17.05 req is for building images, not running them. The ksonnet image has a multi-stage build but since we're pulling from a public image registry it should be fine. I'll remove this.

chuckha · 2017-08-21T19:29:57Z

Were you able to test the quickstart files without RBAC enabled?

jesscodez · 2017-08-21T19:32:52Z

@chuckha yup I disabled RBAC, sonobuoy worked fine (as long as I didn't include the manifests for RBAC components).

chuckha

This looks good to me!

timothysc · 2017-08-23T21:48:18Z

@abiogenesis-now please rebase and we can go ahead and merge.

jesscodez · 2017-08-23T22:29:15Z

will rebase soon!

jesscodez · 2017-08-24T01:27:12Z

@timothysc @kensimon rebased, but
(1) I think my branch name might be breaking the Jenkins build? Not sure how to best fix
(2) I ran both Sonobuoy master and my branch, and encountered the following error with both:

Have you seen this before/do you know what might be causing it? The systemd_logs pods keep launching and dying.

kensimon · 2017-08-24T16:52:01Z

Looks like the git describe we're doing in the Makefile to pick an image tag is choking on the slash in the source branch (chuck/jsonnet). I'll take a look at that.

For your screenshot, I'd need more context for that error, a 409 happens when you're uploading something twice.

jesscodez · 2017-08-24T17:07:52Z

👍 let me know if it's easier/simpler to somehow rename this remote branch (is that possible without having to recreate the PR?) thanks!

Will try to dig up some more useful logs

kensimon · 2017-08-24T18:31:04Z

run tests

kensimon · 2017-08-24T18:32:17Z

Bah, I hit the "Update branch" button in github hoping it would give me an option to rebase, but it created a merge commit (which breaks the signoff checker.) @abiogenesis-now could you rebase?

jesscodez · 2017-08-24T18:35:32Z

👍 can do, also making some small docs revisions so please hold

also I'm no longer encountering the 409 error I screenshotted above--I'm not sure if I accidentally had too many sonobuoy instances running (???) from improper cleanup or some such thing. Confirmed that this code works on my quickstart cluster and my non-RBAC cluster.

Closes issue #5. - Adds jsonnet to generate the quickstart examples - Removes the YAML - Adds the generated JSON - Docs are updated to reflect JSON files vs YAML - Adds a Makefile target, `generate-examples` Signed-off-by: Chuck Ha <[email protected]>

Signed-off-by: Jessica Yao <[email protected]>

jesscodez · 2017-08-24T18:55:29Z

@kensimon This is done on my end, feel free to merge if it looks good to you and the build passes!

[docs][chuck] Example YAML generation using ksonnet and kubecfg Signed-off-by: Jesse Hamilton [email protected]

[docs][chuck] Example YAML generation using ksonnet and kubecfg Signed-off-by: Jesse Hamilton [email protected] Signed-off-by: Jesse Hamilton [email protected]

jesscodez commented Aug 18, 2017

View reviewed changes

chuckha reviewed Aug 18, 2017

View reviewed changes

timothysc requested a review from kensimon August 21, 2017 13:53

timothysc added this to the v0.9.0 milestone Aug 21, 2017

timothysc added area/documentation Related to project documentation p1-important labels Aug 21, 2017

jesscodez mentioned this pull request Aug 21, 2017

Example of using ThirdPartyResources instead of CustomResourceDefinit… vmware-tanzu/velero#36

Closed

ncdc reviewed Aug 21, 2017

View reviewed changes

chuckha approved these changes Aug 22, 2017

View reviewed changes

Chuck Ha and others added 2 commits August 24, 2017 11:52

Use ksonnet to generate examples

0a2dbb4

Closes issue #5. - Adds jsonnet to generate the quickstart examples - Removes the YAML - Adds the generated JSON - Docs are updated to reflect JSON files vs YAML - Adds a Makefile target, `generate-examples` Signed-off-by: Chuck Ha <[email protected]>

Fix jsonnet to output prettier yaml and include rbac_enabled variable

d3336c0

Signed-off-by: Jessica Yao <[email protected]>

jesscodez changed the title ~~[hold][docs][chuck] Example YAML generation using ksonnet and kubecfg~~ [docs][chuck] Example YAML generation using ksonnet and kubecfg Aug 24, 2017

kensimon merged commit ed6b294 into vmware-tanzu:master Aug 24, 2017

jesscodez deleted the chuck/jsonnet branch August 24, 2017 18:56

This was referenced Aug 24, 2017

quickstart needs explanation #46

Closed

There should be an "all-in-one" YAML file #23

Closed

[quickstart] Support non-rbac clusters #53

Closed

jhamilton1 pushed a commit to jhamilton1/sonobuoy that referenced this pull request Jun 27, 2018

Merge pull request vmware-tanzu#58 from abiogenesis-now/chuck/jsonnet

e5b9fab

[docs][chuck] Example YAML generation using ksonnet and kubecfg Signed-off-by: Jesse Hamilton [email protected]

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[docs][chuck] Example YAML generation using ksonnet and kubecfg #58

[docs][chuck] Example YAML generation using ksonnet and kubecfg #58

jesscodez commented Aug 18, 2017

jesscodez commented Aug 18, 2017

jesscodez Aug 18, 2017

chuckha Aug 18, 2017

jesscodez Aug 18, 2017

timothysc commented Aug 21, 2017

ncdc Aug 21, 2017

jesscodez Aug 21, 2017

jesscodez Aug 21, 2017

ncdc Aug 21, 2017

jesscodez Aug 21, 2017

timothysc Aug 23, 2017

jesscodez Aug 23, 2017

chuckha commented Aug 21, 2017

jesscodez commented Aug 21, 2017

chuckha left a comment

timothysc commented Aug 23, 2017

jesscodez commented Aug 23, 2017

jesscodez commented Aug 24, 2017

kensimon commented Aug 24, 2017

jesscodez commented Aug 24, 2017

kensimon commented Aug 24, 2017

kensimon commented Aug 24, 2017

jesscodez commented Aug 24, 2017

jesscodez commented Aug 24, 2017

		@@ -32,19 +32,31 @@ This guide executes a Sonobuoy run on your cluster, and records the following re

		* You should have `kubectl` installed. If not, follow the instructions for [installing via Homebrew (MacOS)][6] or [building the binary (Linux)][7].

		* You should have `docker` installed (minimum 17.05). Docker images are used to generate the YAML examples, based on your cluster's RBAC settings.

[docs][chuck] Example YAML generation using ksonnet and kubecfg #58

[docs][chuck] Example YAML generation using ksonnet and kubecfg #58

Conversation

jesscodez commented Aug 18, 2017

jesscodez commented Aug 18, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

timothysc commented Aug 21, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

chuckha commented Aug 21, 2017

jesscodez commented Aug 21, 2017

chuckha left a comment

Choose a reason for hiding this comment

timothysc commented Aug 23, 2017

jesscodez commented Aug 23, 2017

jesscodez commented Aug 24, 2017

kensimon commented Aug 24, 2017

jesscodez commented Aug 24, 2017

kensimon commented Aug 24, 2017

kensimon commented Aug 24, 2017

jesscodez commented Aug 24, 2017

jesscodez commented Aug 24, 2017