Introduce upgrade --from-manifests flag

[siggy]

The linkerd upgrade command read the control-plane's config from
Kubernetes, which required the environment to be configured to connect
to the appropriate k8s cluster.

Introduce a linkerd upgrade --from-manifests flag, allowing the user
to feed the output of linkerd install into the upgrade command.

Fixes #2629

Signed-off-by: Andrew Seigner [email protected]

Relates to #2637.

Notes

Note that the first commit, 5fb1c0c, contains all the code changes. The second commit simply renames test_helper.go to fake.go.

Examples

Install flow

linkerd install > install_manifests.yaml
kubectl apply -f install_manifests.yaml
linkerd upgrade --from-manifests install_manifests.yaml

Validate linkerd upgrade command works the same with and without --from-manifests

linkerd install | kubectl apply -f -

diff <(
  bin/linkerd upgrade
) <(
  kubectl -n linkerd get cm/linkerd-config secrets/linkerd-identity-issuer -oyaml |
    bin/linkerd upgrade --from-manifests -
)

[siggy]

@jon-walton if you have a moment, please take this out for a spin, thanks!

[l5d-bot]

Integration test results for 2dc3488: success 🎉
Log output: https://gist.github.com/2668e8b197c6a5e1a952b1face3d17f6

[jon-walton]

awesome, that works 👍 thanks! 🎉

[olix0r]

Nice clean change!

One question for @grampelberg: if we're supporting a file-based workflow, we probably need to update documentation to address the storage of the issuer credentials.

[olix0r]

tioli: named params might be easier to read here

[grampelberg]

@olix0r because they show up in the manifest?

[alpeb]

Very elegant approach, looks great to me! 👍

[ihcsim]

Just wanna confirm that we are ok with writing the issuer's secret to disk.

[siggy]

@ihcsim Re: writing secrets to disk, I think that's a good question for @grampelberg, and if the answer is "yes", maybe deserves treatment in our docs.

[olix0r]

@grampelberg right. Ideally, we'd encrypt that part of the manifest and have --from-manifest decrypt the secrets

[ihcsim]

fwiw, I think all #2629 cares about is the linkerd-config config map.

[l5d-bot]

Integration test results for 3ee494a: success 🎉
Log output: https://gist.github.com/362877c549463cb48f27ac6752d32be6

[alpeb]

Another possibility would be to add a note in the Upgrade CLI docs, telling people they can store all the resources in disk, but discouraging keeping the Secret one. Then we could have fetchIdentityValues() not fail when the Secret is not found when using --from-manifests. In my brief testing this doesn't break anything.
One downside I can see is when trying to do kubectl apply --prune with resource files in disk that don't match what there is in the cluster, one has to be careful not to delete the Secret in the cluster.

[jon-walton]

This is an interesting point.

However, when using a gitops workflow, a linkerd install --flags > install.yaml would end up in git and deployed by the deployment controller (in my case, ArgoCD). if the secret is kept out of the manifest, we'll need linkerd to create the secret and store it in a k8s secret object at runtime

[grampelberg]

Is there another option to writing the secret to disk? It'd need to live somewhere ...

From a docs perspective, it is pretty easy to suggest piping through gpg or the like before committing the file.

[ihcsim]

if the secret is kept out of the manifest, we'll need linkerd to create the secret and store it in a k8s secret object at runtime

The (identity) secret is created during installation, and remained untouched, in the cluster, during an upgrade.

[siggy]

Going to merge this as-is. I agree storing secrets in the clear on disk is not great, but also agree we can guide users via docs on best practices, such as @grampelberg's gpg approach.

[grampelberg]

Can we open an issue on the website repo to track writing up a blurb about managing the manifest?

[siggy]

@grampelberg filed linkerd/website#275