Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Generate deployment manifests from helm chart #475

Merged
merged 10 commits into from
May 4, 2020

Conversation

krmichel
Copy link
Contributor

@krmichel krmichel commented Mar 26, 2020

/feature

This fixes #305, fixes #469

  • Helm
    • Update helpers to have a subset of standard labels to use as selectorLabels
    • Split apart rbac and service account templates to be single resources to deal with inconsistency in helm template
    • Update/Add labels on resources to use helpers
    • Replace deprecated serviceAccount with serviceAccountName
    • Add minor special handling if the release name is kustomize
  • Manifest
    • Started generating manifests from helm chart
      • Split apart rbac manifest as part of the generation process
    • Updated kustomization.yaml files
  • Makefile
    • Added target to download helm
    • Added generate-kustomize target

Tested by generating manifests with kubectl kustomize for stable, alpha, and dev before generating manifests and then again after. These manifests were compared to look for only expected differences (new labels and small reconciliations).

@k8s-ci-robot k8s-ci-robot added cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Mar 26, 2020
@k8s-ci-robot
Copy link
Contributor

Hi @krmichel. Thanks for your PR.

I'm waiting for a kubernetes-sigs member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@k8s-ci-robot k8s-ci-robot added the size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. label Mar 26, 2020
@krmichel
Copy link
Contributor Author

/assign @jsafrane

@krmichel
Copy link
Contributor Author

@leakingtapan, Here is the PR for generating the manifests from the helm chart.

@coveralls
Copy link

coveralls commented Mar 26, 2020

Pull Request Test Coverage Report for Build 1091

  • 0 of 0 changed or added relevant lines in 0 files are covered.
  • No unchanged relevant lines lost coverage.
  • Overall coverage remained the same at 78.871%

Totals Coverage Status
Change from base Build 1089: 0.0%
Covered Lines: 1411
Relevant Lines: 1789

💛 - Coveralls

@leakingtapan
Copy link
Contributor

/ok-to-test

@k8s-ci-robot k8s-ci-robot added ok-to-test Indicates a non-member PR verified by an org member that is safe to test. and removed needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Mar 26, 2020
@krmichel
Copy link
Contributor Author

krmichel commented Mar 26, 2020

This fixes #469

@k8s-ci-robot k8s-ci-robot added needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. and removed needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. labels Mar 29, 2020
@krmichel
Copy link
Contributor Author

This has passed tests. @bertinatto , @d-nishi , @leakingtapan: anyone want to take a look at it?

@bertinatto
Copy link
Member

Thank you for your PR, @krmichel! I'll leave this to @leakingtapan as he is our Helm expert.

@krmichel
Copy link
Contributor Author

@leakingtapan Any chance you can take a look at this soon?

Copy link
Contributor

@leakingtapan leakingtapan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@krmichel sorry for the late reply. This is awesome job! Left some comments.

docs/README.md Show resolved Hide resolved
@@ -84,3 +91,24 @@ push-release:
.PHONY: push
push:
docker push $(IMAGE):latest

.PHONY: generate-kustomize
generate-kustomize: bin/helm
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems a bit over complicated. Is it possible to simply this step? I imagine it will be hard to update this list of command, whenever there is some changes or refactoring to the helm templates. I think it is okay to restructure the layer out of kustomize file structure to make the generation earlier and more maintainable. Maybe combine some of the kustomize files into one for each layer? As they are all auto generated.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I don't like it much either. What I ran into is that when running helm template for files with more than one resource those resources weren't rendered in a consistent order. I could run the target twice in a row and get the same resources but in a different order which makes it difficult to manage in source control. I would be open if anyone has a way to produce reliably reproducible output and keep the alpha features separate in the folder structure.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Given the limitation with helm. I think I can live with it for now, and we can follow up with this as a separate issue.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just noticed this. It should have been fixed in Helm 3.1.0: helm/helm#6842. Weird you were seeing this with Helm 3.1.2 here.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I saw that helm issue when I was working on this but still saw issues with resources moving around in the processed output on multiple runs without changing anything.

deploy/kubernetes/base/node.yaml Outdated Show resolved Hide resolved
@k8s-ci-robot k8s-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Apr 27, 2020
@k8s-ci-robot k8s-ci-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label May 3, 2020
@krmichel
Copy link
Contributor Author

krmichel commented May 3, 2020

/retest

@leakingtapan
Copy link
Contributor

/lgtm
/approve

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label May 4, 2020
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: krmichel, leakingtapan

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label May 4, 2020
@k8s-ci-robot k8s-ci-robot merged commit fa0de91 into kubernetes-sigs:master May 4, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. lgtm "Looks good to me", indicates that a PR is ready to be merged. ok-to-test Indicates a non-member PR verified by an org member that is safe to test. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files.
Projects
None yet
7 participants