Optimize stack configuration step #650
Assignees
Labels
Milestone
Comments
EronWright
changed the title
|
Added to epic #586 |
5 tasks
blampe
added
kind/engineering
blampe
added a commit
that referenced
this issue
Oct 17, 2024
We currently issue one `SetAllConfig` RPC for each user-specified config. This is slow but it has important correctness guarantees: 1. The order we apply config matters -- if the user specifies `foo: foo` followed by `foo: bar`, the net result must always be `foo: bar`. 2. The Pulumi CLI (and therefore Automation API) only allows specifying `--path` on an all-or-nothing basis. This is bad for us because we potentially have a blend of path and non-path keys. Ideally we would be able to supply all of our configs to the automation API in a single call, and in the case where _all_ of our config keys are path-like (or all are not path-like) we actually can do that because we no longer have limitation (2). This PR makes that possible in the general case by transforming our config keys in a way that allows us to treat them as if they are all path-like. In particular: * The agent's `SetAllConfig` handler is modified to take a list of configs instead of a map in order to preserve config order. The top-level `path` param is also removed and handled on a per-key basis. * While resolving configs, we escape any non-path keys so subsequent path parsing treats them as verbatim. For example `foo.bar` gets escaped as `["foo.bar"]`. * We can then supply all of our keys at once to Automation API with `Path: true`. * If there are no configs to set then the operator doesn't invoke `SetAllConfig`. Fixes #650
github-project-automation
bot
moved this from In Progress
to Done
in Pulumi Kubernetes Operator v2
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
The Workspace controller is expected to configure the stack with the supplied configuration values. The current implementation calls the
SetAllConfigRPC method for each config item, and this is proving to be slow. Would be nice to set in bulk, but be considerate about the ordering of the config elements.Also, consider enhancing the Workspace config block to natively understand configmap/secret refs. Today, the Stack controller handles references via general extensibility, i.e. via the pod template. A first-class option would be better.
The text was updated successfully, but these errors were encountered: