Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(core): inherit Build action mode from dependant Deploy action #5589

Merged
merged 6 commits into from
Dec 22, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions core/src/actions/types.ts
Original file line number Diff line number Diff line change
Expand Up @@ -137,6 +137,11 @@ export interface ActionModes {
local?: boolean
}

export const ALL_ACTION_MODES_SUPPORTED: ActionModes = {
sync: true,
local: true,
}

export type ActionMode = keyof ActionModes | "default"

export type ActionModeMap = {
Expand Down
4 changes: 2 additions & 2 deletions core/src/commands/deploy.ts
Original file line number Diff line number Diff line change
Expand Up @@ -194,8 +194,8 @@ export class DeployCommand extends Command<Args, Opts> {

const actionModes: ActionModeMap = {
// Support a single empty value (which comes across as an empty list) as equivalent to '*'
local: opts["local-mode"]?.length === 0 ? ["*"] : opts["local-mode"]?.map((s) => "deploy." + s),
sync: opts.sync?.length === 0 ? ["*"] : opts.sync?.map((s) => "deploy." + s),
local: opts["local-mode"]?.length === 0 ? ["deploy.*"] : opts["local-mode"]?.map((s) => "deploy." + s),
sync: opts.sync?.length === 0 ? ["deploy.*"] : opts.sync?.map((s) => "deploy." + s),
}

const graph = await garden.getConfigGraph({ log, emit: true, actionModes })
Expand Down
8 changes: 6 additions & 2 deletions core/src/config/template-contexts/actions.ts
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ import type { ActionConfig, Action, ExecutedAction, ResolvedAction } from "../..
import type { ActionMode } from "../../actions/types.js"
import type { Garden } from "../../garden.js"
import type { GardenModule } from "../../types/module.js"
import { deline } from "../../util/string.js"
import { dedent, deline } from "../../util/string.js"
import type { DeepPrimitiveMap, PrimitiveMap } from "../common.js"
import { joi, joiIdentifier, joiIdentifierMap, joiPrimitive, joiVariables } from "../common.js"
import type { ProviderMap } from "../provider.js"
Expand Down Expand Up @@ -40,7 +40,11 @@ const actionModeSchema = joi
.default("default")
.allow("default", "sync", "local")
.description(
"The mode that the action should be executed in (e.g. 'sync' or 'local' for Deploy actions). Set to 'default' if no special mode is being used."
dedent`
The mode that the action should be executed in (e.g. 'sync' or 'local' for Deploy actions). Set to 'default' if no special mode is being used.

Build actions inherit the mode from Deploy actions that depend on them. E.g. If a Deploy action is in 'sync' mode and depends on a Build action, the Build action will inherit the 'sync' mode setting from the Deploy action. This enables installing different tools that may be necessary for different development modes.
`
)
.example("sync")

Expand Down
91 changes: 76 additions & 15 deletions core/src/graph/actions.ts
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ import type {
Executed,
Resolved,
} from "../actions/types.js"
import { actionKinds } from "../actions/types.js"
import { ALL_ACTION_MODES_SUPPORTED, actionKinds } from "../actions/types.js"
import {
actionReferenceToString,
addActionDependency,
Expand Down Expand Up @@ -59,7 +59,7 @@ import { mergeVariables } from "./common.js"
import type { ConfigGraph } from "./config-graph.js"
import { MutableConfigGraph } from "./config-graph.js"
import type { ModuleGraph } from "./modules.js"
import type { MaybeUndefined } from "../util/util.js"
import { isTruthy, type MaybeUndefined } from "../util/util.js"
import minimatch from "minimatch"
import type { ConfigContext } from "../config/template-contexts/base.js"
import type { LinkedSource, LinkedSourceMap } from "../config-store/local.js"
Expand Down Expand Up @@ -138,19 +138,74 @@ export const actionConfigsToGraph = profileAsync(async function actionConfigsToG
// Doing this in two steps makes the code a bit less readable, but it's worth it for the performance boost.
const preprocessResults: { [key: string]: PreprocessActionResult } = {}
const computedActionModes: { [key: string]: { mode: ActionMode; explicitMode: boolean } } = {}
await Promise.all(
Object.entries(configsByKey).map(async ([key, config]) => {
const { mode, explicitMode } = getActionMode(config, actionModes, log)
computedActionModes[key] = { mode, explicitMode }
preprocessResults[key] = await preprocessActionConfig({
garden,
config,
router,
log,
mode,

const preprocessActions = async (predicate: (config: ActionConfig) => boolean = () => true) => {
return await Promise.all(
Object.entries(configsByKey).map(async ([key, config]) => {
if (!predicate(config)) {
return
}

const { mode, explicitMode } = getActionMode(config, actionModes, log)
computedActionModes[key] = { mode, explicitMode }
preprocessResults[key] = await preprocessActionConfig({
garden,
config,
router,
log,
mode,
})
})
})
)
)
}

// First preprocess only the Deploy actions, so we can infer the mode of Build actions that are used by them.
await preprocessActions((config) => config.kind === "Deploy")

// This enables users to use `this.mode` in Build action configs, such that `this.mode == "sync"`
// when a Deploy action that uses the Build action is in sync mode.
//
// The proper solution to this would involve e.g. parametrized actions, or injecting a separate Build action
// with `this.mode` set to the Deploy action's mode before resolution (both would need to be thought out carefully).
const actionTypes = await garden.getActionTypes()
const buildModeOverrides: Record<string, { mode: ActionMode; overriddenByDeploy: string }> = {}
for (const [key, res] of Object.entries(preprocessResults)) {
const config = res.config
const { mode } = computedActionModes[key]
if (config.kind === "Deploy" && mode !== "default") {
const definition = actionTypes[config.kind][config.type]?.spec
const buildDeps = dependenciesFromActionConfig(
log,
config,
configsByKey,
definition,
res.templateContext,
actionTypes
)
const referencedBuildNames = [config.build, ...buildDeps.map((d) => d.name)].filter(isTruthy)
for (const buildName of referencedBuildNames) {
const buildKey = actionReferenceToString({ kind: "Build", name: buildName })
if (buildModeOverrides[buildKey]) {
const prev = buildModeOverrides[buildKey]
log.warn(dedent`
Using mode ${styles.highlight(prev.mode)} for Build ${styles.highlight(buildName)} as requested by\
the Deploy ${styles.highlight(prev.overriddenByDeploy)}.

Ignoring request by Deploy ${styles.highlight(config.name)} to use mode ${styles.highlight(mode)}.
`)
}
actionModes[mode] = [buildKey, ...(actionModes[mode] || [])]
buildModeOverrides[buildKey] = {
mode,
overriddenByDeploy: config.name,
}
}
}
}

// Preprocess all remaining actions (Deploy actions are preprocessed above)
// We are preprocessing actions in two batches so we can infer the mode of Build actions that are used by Deploy actions. See the comments above.
await preprocessActions((config) => config.kind !== "Deploy")

// Optimize file scanning by avoiding unnecessarily broad scans when project is not in repo root.
const preprocessedConfigs = Object.values(preprocessResults).map((r) => r.config)
Expand Down Expand Up @@ -714,7 +769,13 @@ export const preprocessActionConfig = profileAsync(async function preprocessActi

resolveTemplates()

const { config: updatedConfig, supportedModes } = await router.configureAction({ config, log })
const configureActionResult = await router.configureAction({ config, log })

const { config: updatedConfig } = configureActionResult

// NOTE: Build actions inherit the supported modes of the Deploy actions that use them
const supportedModes: ActionModes =
config.kind === "Build" ? ALL_ACTION_MODES_SUPPORTED : configureActionResult.supportedModes

// -> Throw if trying to modify no-template fields
for (const field of noTemplateFields) {
Expand Down
45 changes: 45 additions & 0 deletions core/test/unit/src/actions/action-configs-to-graph.ts
Original file line number Diff line number Diff line change
Expand Up @@ -840,6 +840,51 @@ describe("actionConfigsToGraph", () => {
expect(action.mode()).to.equal("local")
})

it("deploy action mode overrides the mode of a dependency build action", async () => {
const graph = await actionConfigsToGraph({
garden,
log,
groupConfigs: [],
configs: [
{
kind: "Deploy",
type: "test",
name: "foo",
timeout: DEFAULT_DEPLOY_TIMEOUT_SEC,
variables: {},
dependencies: [{ kind: "Build", name: "foo" }],
internal: {
basePath: tmpDir.path,
},
spec: {},
},
{
kind: "Build",
type: "test",
name: "foo",
timeout: DEFAULT_DEPLOY_TIMEOUT_SEC,
variables: {},
internal: {
basePath: tmpDir.path,
},
spec: {},
},
],
moduleGraph: new ModuleGraph([], {}),
linkedSources: {},
actionModes: {
local: ["deploy.*"],
},
environmentName: garden.environmentName,
})

const deploy = graph.getDeploy("foo")
expect(deploy.mode()).to.equal("local")

const build = graph.getBuild("foo")
expect(build.mode()).to.equal("local")
})

it("throws if an unknown action kind is given", async () => {
await expectError(
() =>
Expand Down
2 changes: 2 additions & 0 deletions docs/reference/action-types/Build/container.md
Original file line number Diff line number Diff line change
Expand Up @@ -410,6 +410,8 @@ my-variable: ${actions.build.my-build.sourcePath}

The mode that the action should be executed in (e.g. 'sync' or 'local' for Deploy actions). Set to 'default' if no special mode is being used.

Build actions inherit the mode from Deploy actions that depend on them. E.g. If a Deploy action is in 'sync' mode and depends on a Build action, the Build action will inherit the 'sync' mode setting from the Deploy action. This enables installing different tools that may be necessary for different development modes.

| Type | Default |
| -------- | ----------- |
| `string` | `"default"` |
Expand Down
2 changes: 2 additions & 0 deletions docs/reference/action-types/Build/exec.md
Original file line number Diff line number Diff line change
Expand Up @@ -392,6 +392,8 @@ my-variable: ${actions.build.my-build.sourcePath}

The mode that the action should be executed in (e.g. 'sync' or 'local' for Deploy actions). Set to 'default' if no special mode is being used.

Build actions inherit the mode from Deploy actions that depend on them. E.g. If a Deploy action is in 'sync' mode and depends on a Build action, the Build action will inherit the 'sync' mode setting from the Deploy action. This enables installing different tools that may be necessary for different development modes.

| Type | Default |
| -------- | ----------- |
| `string` | `"default"` |
Expand Down
2 changes: 2 additions & 0 deletions docs/reference/action-types/Build/jib-container.md
Original file line number Diff line number Diff line change
Expand Up @@ -569,6 +569,8 @@ my-variable: ${actions.build.my-build.sourcePath}

The mode that the action should be executed in (e.g. 'sync' or 'local' for Deploy actions). Set to 'default' if no special mode is being used.

Build actions inherit the mode from Deploy actions that depend on them. E.g. If a Deploy action is in 'sync' mode and depends on a Build action, the Build action will inherit the 'sync' mode setting from the Deploy action. This enables installing different tools that may be necessary for different development modes.

| Type | Default |
| -------- | ----------- |
| `string` | `"default"` |
Expand Down
2 changes: 2 additions & 0 deletions docs/reference/action-types/Deploy/configmap.md
Original file line number Diff line number Diff line change
Expand Up @@ -321,6 +321,8 @@ my-variable: ${actions.deploy.my-deploy.sourcePath}

The mode that the action should be executed in (e.g. 'sync' or 'local' for Deploy actions). Set to 'default' if no special mode is being used.

Build actions inherit the mode from Deploy actions that depend on them. E.g. If a Deploy action is in 'sync' mode and depends on a Build action, the Build action will inherit the 'sync' mode setting from the Deploy action. This enables installing different tools that may be necessary for different development modes.

| Type | Default |
| -------- | ----------- |
| `string` | `"default"` |
Expand Down
2 changes: 2 additions & 0 deletions docs/reference/action-types/Deploy/container.md
Original file line number Diff line number Diff line change
Expand Up @@ -1191,6 +1191,8 @@ my-variable: ${actions.deploy.my-deploy.sourcePath}

The mode that the action should be executed in (e.g. 'sync' or 'local' for Deploy actions). Set to 'default' if no special mode is being used.

Build actions inherit the mode from Deploy actions that depend on them. E.g. If a Deploy action is in 'sync' mode and depends on a Build action, the Build action will inherit the 'sync' mode setting from the Deploy action. This enables installing different tools that may be necessary for different development modes.

| Type | Default |
| -------- | ----------- |
| `string` | `"default"` |
Expand Down
2 changes: 2 additions & 0 deletions docs/reference/action-types/Deploy/exec.md
Original file line number Diff line number Diff line change
Expand Up @@ -391,6 +391,8 @@ my-variable: ${actions.deploy.my-deploy.sourcePath}

The mode that the action should be executed in (e.g. 'sync' or 'local' for Deploy actions). Set to 'default' if no special mode is being used.

Build actions inherit the mode from Deploy actions that depend on them. E.g. If a Deploy action is in 'sync' mode and depends on a Build action, the Build action will inherit the 'sync' mode setting from the Deploy action. This enables installing different tools that may be necessary for different development modes.

| Type | Default |
| -------- | ----------- |
| `string` | `"default"` |
Expand Down
2 changes: 2 additions & 0 deletions docs/reference/action-types/Deploy/helm.md
Original file line number Diff line number Diff line change
Expand Up @@ -1045,6 +1045,8 @@ my-variable: ${actions.deploy.my-deploy.sourcePath}

The mode that the action should be executed in (e.g. 'sync' or 'local' for Deploy actions). Set to 'default' if no special mode is being used.

Build actions inherit the mode from Deploy actions that depend on them. E.g. If a Deploy action is in 'sync' mode and depends on a Build action, the Build action will inherit the 'sync' mode setting from the Deploy action. This enables installing different tools that may be necessary for different development modes.

| Type | Default |
| -------- | ----------- |
| `string` | `"default"` |
Expand Down
2 changes: 2 additions & 0 deletions docs/reference/action-types/Deploy/kubernetes.md
Original file line number Diff line number Diff line change
Expand Up @@ -1082,6 +1082,8 @@ my-variable: ${actions.deploy.my-deploy.sourcePath}

The mode that the action should be executed in (e.g. 'sync' or 'local' for Deploy actions). Set to 'default' if no special mode is being used.

Build actions inherit the mode from Deploy actions that depend on them. E.g. If a Deploy action is in 'sync' mode and depends on a Build action, the Build action will inherit the 'sync' mode setting from the Deploy action. This enables installing different tools that may be necessary for different development modes.

| Type | Default |
| -------- | ----------- |
| `string` | `"default"` |
Expand Down
2 changes: 2 additions & 0 deletions docs/reference/action-types/Deploy/persistentvolumeclaim.md
Original file line number Diff line number Diff line change
Expand Up @@ -491,6 +491,8 @@ my-variable: ${actions.deploy.my-deploy.sourcePath}

The mode that the action should be executed in (e.g. 'sync' or 'local' for Deploy actions). Set to 'default' if no special mode is being used.

Build actions inherit the mode from Deploy actions that depend on them. E.g. If a Deploy action is in 'sync' mode and depends on a Build action, the Build action will inherit the 'sync' mode setting from the Deploy action. This enables installing different tools that may be necessary for different development modes.

| Type | Default |
| -------- | ----------- |
| `string` | `"default"` |
Expand Down
2 changes: 2 additions & 0 deletions docs/reference/action-types/Deploy/pulumi.md
Original file line number Diff line number Diff line change
Expand Up @@ -467,6 +467,8 @@ my-variable: ${actions.deploy.my-deploy.sourcePath}

The mode that the action should be executed in (e.g. 'sync' or 'local' for Deploy actions). Set to 'default' if no special mode is being used.

Build actions inherit the mode from Deploy actions that depend on them. E.g. If a Deploy action is in 'sync' mode and depends on a Build action, the Build action will inherit the 'sync' mode setting from the Deploy action. This enables installing different tools that may be necessary for different development modes.

| Type | Default |
| -------- | ----------- |
| `string` | `"default"` |
Expand Down
2 changes: 2 additions & 0 deletions docs/reference/action-types/Deploy/terraform.md
Original file line number Diff line number Diff line change
Expand Up @@ -378,6 +378,8 @@ my-variable: ${actions.deploy.my-deploy.sourcePath}

The mode that the action should be executed in (e.g. 'sync' or 'local' for Deploy actions). Set to 'default' if no special mode is being used.

Build actions inherit the mode from Deploy actions that depend on them. E.g. If a Deploy action is in 'sync' mode and depends on a Build action, the Build action will inherit the 'sync' mode setting from the Deploy action. This enables installing different tools that may be necessary for different development modes.

| Type | Default |
| -------- | ----------- |
| `string` | `"default"` |
Expand Down
2 changes: 2 additions & 0 deletions docs/reference/action-types/Run/container.md
Original file line number Diff line number Diff line change
Expand Up @@ -618,6 +618,8 @@ my-variable: ${actions.run.my-run.sourcePath}

The mode that the action should be executed in (e.g. 'sync' or 'local' for Deploy actions). Set to 'default' if no special mode is being used.

Build actions inherit the mode from Deploy actions that depend on them. E.g. If a Deploy action is in 'sync' mode and depends on a Build action, the Build action will inherit the 'sync' mode setting from the Deploy action. This enables installing different tools that may be necessary for different development modes.

| Type | Default |
| -------- | ----------- |
| `string` | `"default"` |
Expand Down
2 changes: 2 additions & 0 deletions docs/reference/action-types/Run/exec.md
Original file line number Diff line number Diff line change
Expand Up @@ -371,6 +371,8 @@ my-variable: ${actions.run.my-run.sourcePath}

The mode that the action should be executed in (e.g. 'sync' or 'local' for Deploy actions). Set to 'default' if no special mode is being used.

Build actions inherit the mode from Deploy actions that depend on them. E.g. If a Deploy action is in 'sync' mode and depends on a Build action, the Build action will inherit the 'sync' mode setting from the Deploy action. This enables installing different tools that may be necessary for different development modes.

| Type | Default |
| -------- | ----------- |
| `string` | `"default"` |
Expand Down
2 changes: 2 additions & 0 deletions docs/reference/action-types/Run/helm-pod.md
Original file line number Diff line number Diff line change
Expand Up @@ -642,6 +642,8 @@ my-variable: ${actions.run.my-run.sourcePath}

The mode that the action should be executed in (e.g. 'sync' or 'local' for Deploy actions). Set to 'default' if no special mode is being used.

Build actions inherit the mode from Deploy actions that depend on them. E.g. If a Deploy action is in 'sync' mode and depends on a Build action, the Build action will inherit the 'sync' mode setting from the Deploy action. This enables installing different tools that may be necessary for different development modes.

| Type | Default |
| -------- | ----------- |
| `string` | `"default"` |
Expand Down
2 changes: 2 additions & 0 deletions docs/reference/action-types/Run/kubernetes-exec.md
Original file line number Diff line number Diff line change
Expand Up @@ -414,6 +414,8 @@ my-variable: ${actions.run.my-run.sourcePath}

The mode that the action should be executed in (e.g. 'sync' or 'local' for Deploy actions). Set to 'default' if no special mode is being used.

Build actions inherit the mode from Deploy actions that depend on them. E.g. If a Deploy action is in 'sync' mode and depends on a Build action, the Build action will inherit the 'sync' mode setting from the Deploy action. This enables installing different tools that may be necessary for different development modes.

| Type | Default |
| -------- | ----------- |
| `string` | `"default"` |
Expand Down
2 changes: 2 additions & 0 deletions docs/reference/action-types/Run/kubernetes-pod.md
Original file line number Diff line number Diff line change
Expand Up @@ -707,6 +707,8 @@ my-variable: ${actions.run.my-run.sourcePath}

The mode that the action should be executed in (e.g. 'sync' or 'local' for Deploy actions). Set to 'default' if no special mode is being used.

Build actions inherit the mode from Deploy actions that depend on them. E.g. If a Deploy action is in 'sync' mode and depends on a Build action, the Build action will inherit the 'sync' mode setting from the Deploy action. This enables installing different tools that may be necessary for different development modes.

| Type | Default |
| -------- | ----------- |
| `string` | `"default"` |
Expand Down
2 changes: 2 additions & 0 deletions docs/reference/action-types/Test/conftest-helm.md
Original file line number Diff line number Diff line change
Expand Up @@ -368,6 +368,8 @@ my-variable: ${actions.test.my-test.sourcePath}

The mode that the action should be executed in (e.g. 'sync' or 'local' for Deploy actions). Set to 'default' if no special mode is being used.

Build actions inherit the mode from Deploy actions that depend on them. E.g. If a Deploy action is in 'sync' mode and depends on a Build action, the Build action will inherit the 'sync' mode setting from the Deploy action. This enables installing different tools that may be necessary for different development modes.

| Type | Default |
| -------- | ----------- |
| `string` | `"default"` |
Expand Down
2 changes: 2 additions & 0 deletions docs/reference/action-types/Test/conftest.md
Original file line number Diff line number Diff line change
Expand Up @@ -356,6 +356,8 @@ my-variable: ${actions.test.my-test.sourcePath}

The mode that the action should be executed in (e.g. 'sync' or 'local' for Deploy actions). Set to 'default' if no special mode is being used.

Build actions inherit the mode from Deploy actions that depend on them. E.g. If a Deploy action is in 'sync' mode and depends on a Build action, the Build action will inherit the 'sync' mode setting from the Deploy action. This enables installing different tools that may be necessary for different development modes.

| Type | Default |
| -------- | ----------- |
| `string` | `"default"` |
Expand Down
Loading