Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add webhook for manually approving traffic weight increase #849

Merged
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions artifacts/flagger/crd.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -940,6 +940,7 @@ spec:
- post-rollout
- event
- rollback
- confirm-traffic-increase
url:
description: URL address of this webhook
type: string
Expand Down
1 change: 1 addition & 0 deletions charts/flagger/crds/crd.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -940,6 +940,7 @@ spec:
- post-rollout
- event
- rollback
- confirm-traffic-increase
url:
description: URL address of this webhook
type: string
Expand Down
9 changes: 8 additions & 1 deletion docs/gitbook/usage/webhooks.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,9 @@ There are several types of hooks:
* **rollout** hooks are executed during the analysis on each iteration before the metric checks.
If a rollout hook call fails the canary advancement is paused and eventfully rolled back.

* **confirm-traffic-increase** hooks are executed right before the weight on the canary is increased. The canary
advancement is paused until this hook returns HTTP 200.

* **confirm-promotion** hooks are executed before the promotion step.
The canary promotion is paused until the hooks return HTTP 200.
While the promotion is paused, Flagger will continue to run the metrics checks and rollout hooks.
Expand Down Expand Up @@ -51,6 +54,9 @@ Spec:
timeout: 15s
metadata:
cmd: "hey -z 1m -q 5 -c 2 http://podinfo-canary.test:9898/"
- name: "traffic increase gate"
type: confirm-traffic-increase
url: http://flagger-loadtester.test/gate/approve
- name: "promotion gate"
type: confirm-promotion
url: http://flagger-loadtester.test/gate/approve
Expand Down Expand Up @@ -347,7 +353,8 @@ the web-hook will try to call Concord before timing out (Default is 30s).
## Manual Gating

For manual approval of a canary deployment you can use the `confirm-rollout` and `confirm-promotion` webhooks.
The confirmation rollout hooks are executed before the pre-rollout hooks.
The confirmation rollout hooks are executed before the pre-rollout hooks. For manually approving traffic weight increase,
you can use the `confirm-traffic-increase` webhook.
Flagger will halt the canary traffic shifting and analysis until the confirm webhook returns HTTP status 200.

For manual rollback of a canary deployment you can use the `rollback` webhook.
Expand Down
2 changes: 2 additions & 0 deletions pkg/apis/flagger/v1beta1/canary.go
Original file line number Diff line number Diff line change
Expand Up @@ -314,6 +314,8 @@ const (
EventHook HookType = "event"
// RollbackHook rollback canary analysis if webhook returns HTTP 200
RollbackHook HookType = "rollback"
// ConfirmTrafficIncreaseHook increases traffic weight if webhook returns HTTP 200
ConfirmTrafficIncreaseHook = "confirm-traffic-increase"
)

// CanaryWebhook holds the reference to external checks used for canary analysis
Expand Down
6 changes: 6 additions & 0 deletions pkg/controller/scheduler.go
Original file line number Diff line number Diff line change
Expand Up @@ -412,6 +412,12 @@ func (c *Controller) advanceCanary(name string, namespace string) {

// strategy: Canary progressive traffic increase
if c.nextStepWeight(cd, canaryWeight) > 0 {
// run hook only if traffic is not mirrored
if !mirrored {
if promote := c.runConfirmTrafficIncreaseHooks(cd); !promote {
return
}
}
c.runCanary(cd, canaryController, meshRouter, mirrored, canaryWeight, primaryWeight, maxWeight)
}

Expand Down
16 changes: 16 additions & 0 deletions pkg/controller/scheduler_hooks.go
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,22 @@ import (
"github.com/fluxcd/flagger/pkg/canary"
)

func (c *Controller) runConfirmTrafficIncreaseHooks(canary *flaggerv1.Canary) bool {
for _, webhook := range canary.GetAnalysis().Webhooks {
if webhook.Type == flaggerv1.ConfirmTrafficIncreaseHook {
err := CallWebhook(canary.Name, canary.Namespace, flaggerv1.CanaryPhaseProgressing, webhook)
if err != nil {
c.recordEventWarningf(canary, "Halt %s.%s advancement waiting for traffic increase approval %s",
canary.Name, canary.Namespace, webhook.Name)
c.alert(canary, "Canary traffic increase is waiting for approval.", false, flaggerv1.SeverityWarn)
return false
}
c.recordEventInfof(canary, "Confirm-traffic-increase check %s passed", webhook.Name)
}
}
return true
}

func (c *Controller) runConfirmRolloutHooks(canary *flaggerv1.Canary, canaryController canary.Controller) bool {
for _, webhook := range canary.GetAnalysis().Webhooks {
if webhook.Type == flaggerv1.ConfirmRolloutHook {
Expand Down