Failed
=== RUN   TestCanarySuite/TestCanaryDynamicStableScale
time="2024-02-14T00:28:31Z" level=info msg="Deleting e2e-test-name=TestCanaryDynamicStableScale"
W0214 00:28:31.950092   21758 warnings.go:70] policy/v1beta1 PodDisruptionBudget is deprecated in v1.21+, unavailable in v1.25+; use policy/v1 PodDisruptionBudget
W0214 00:28:31.952711   21758 warnings.go:70] policy/v1beta1 PodDisruptionBudget is deprecated in v1.21+, unavailable in v1.25+; use policy/v1 PodDisruptionBudget
time="2024-02-14T00:28:32Z" level=info msg="service/dynamic-stable-scale-root created\n" rollout=dynamic-stable-scale
time="2024-02-14T00:28:32Z" level=info msg="service/dynamic-stable-scale-canary created\n" rollout=dynamic-stable-scale
time="2024-02-14T00:28:32Z" level=info msg="service/dynamic-stable-scale-stable created\n" rollout=dynamic-stable-scale
time="2024-02-14T00:28:32Z" level=info msg="ingress.networking.k8s.io/dynamic-stable-scale-ingress created\n" rollout=dynamic-stable-scale
time="2024-02-14T00:28:33Z" level=info msg="rollout.argoproj.io/dynamic-stable-scale created\n" rollout=dynamic-stable-scale
time="2024-02-14T00:28:34Z" level=info msg="Marked 4 revision 1 pods as ready" rollout=dynamic-stable-scale
time="2024-02-14T00:28:34Z" level=info msg="Waiting for condition: status=Healthy" rollout=dynamic-stable-scale
time="2024-02-14T00:28:37Z" level=info msg="Condition 'status=Healthy' met after 3s" rollout=dynamic-stable-scale
time="2024-02-14T00:28:37Z" level=info msg="Updated rollout pod spec: 2024-02-14T00:28:37.83571815Z" rollout=dynamic-stable-scale
time="2024-02-14T00:28:38Z" level=info msg="Marked 1 revision 2 pods as ready" rollout=dynamic-stable-scale
time="2024-02-14T00:28:38Z" level=info msg="Waiting for condition: status=Paused" rollout=dynamic-stable-scale
time="2024-02-14T00:28:39Z" level=info msg="Condition 'status=Paused' met after 0s" rollout=dynamic-stable-scale
time="2024-02-14T00:28:39Z" level=info msg="Sleeping 2s" rollout=dynamic-stable-scale
time="2024-02-14T00:28:41Z" level=info msg="Pod expectation 'revision:1 pod count == 3' met" rollout=dynamic-stable-scale
time="2024-02-14T00:28:41Z" level=info msg="Pod expectation 'revision:2 pod count == 1' met" rollout=dynamic-stable-scale
time="2024-02-14T00:28:41Z" level=info msg="Promoted rollout" rollout=dynamic-stable-scale
time="2024-02-14T00:28:42Z" level=info msg="Marked 2 revision 2 pods as ready" rollout=dynamic-stable-scale
time="2024-02-14T00:28:42Z" level=info msg="Waiting for condition: status.currentStepIndex=3" rollout=dynamic-stable-scale
time="2024-02-14T00:28:44Z" level=info msg="Condition 'status.currentStepIndex=3' met after 2s" rollout=dynamic-stable-scale
time="2024-02-14T00:28:44Z" level=info msg="Sleeping 2s" rollout=dynamic-stable-scale
time="2024-02-14T00:28:46Z" level=info msg="Pod expectation 'revision:1 pod count == 1' met" rollout=dynamic-stable-scale
time="2024-02-14T00:28:46Z" level=info msg="Pod expectation 'revision:2 pod count == 3' met" rollout=dynamic-stable-scale
time="2024-02-14T00:28:46Z" level=info msg="Aborted rollout" rollout=dynamic-stable-scale
time="2024-02-14T00:28:47Z" level=info msg="Marked 2 revision 1 pods as ready" rollout=dynamic-stable-scale
time="2024-02-14T00:28:47Z" level=info msg="Waiting for condition: rev=2 podcount=1" rollout=dynamic-stable-scale
time="2024-02-14T00:28:49Z" level=info msg="Condition 'rev=2 podcount=1' met after 1s" rollout=dynamic-stable-scale
time="2024-02-14T00:28:49Z" level=info msg="Pod expectation 'revision:1 pod count == 4' met" rollout=dynamic-stable-scale
time="2024-02-14T00:28:50Z" level=info msg="Marked 1 revision 1 pods as ready" rollout=dynamic-stable-scale
time="2024-02-14T00:28:50Z" level=info msg="Waiting for condition: rev=2 podcount=0" rollout=dynamic-stable-scale
time="2024-02-14T00:28:51Z" level=info msg="Condition 'rev=2 podcount=0' met after 0s" rollout=dynamic-stable-scale
    canary_test.go:638: Expected dynamic-stable-scale-canary selector: map[app:dynamic-stable-scale rollouts-pod-template-hash:868d98995b]. Actual: map[app:dynamic-stable-scale rollouts-pod-template-hash:5689547dfd]
Name:            dynamic-stable-scale
Namespace:       default
Status:          �[31m✖�[0m Degraded
Message:         RolloutAborted: Rollout aborted update to revision 2
Strategy:        Canary
  Step:          0/4
  SetWeight:     0
  ActualWeight:  0
Images:          nginx:1.19-alpine (�[32mstable�[0m)
Replicas:
  Desired:       4
  Current:       4
  Updated:       0
  Ready:         4
  Available:     4

NAME                                              KIND        STATUS         AGE  INFO
⟳ dynamic-stable-scale                            Rollout     �[31m✖�[0m Degraded     18s  
├──# revision:2                                                                   
│  └──⧉ �[33mdynamic-stable-scale-5689547dfd�[0m           ReplicaSet  �[0m•�[0m ScaledDown   14s  �[33mcanary�[0m
│     └──□ dynamic-stable-scale-5689547dfd-z2vs8  Pod         �[94m◌�[0m Terminating  14s  ready:1/1
└──# revision:1                                                                   
   └──⧉ �[32mdynamic-stable-scale-868d98995b�[0m           ReplicaSet  �[32m✔�[0m Healthy      18s  �[32mstable�[0m
      ├──□ dynamic-stable-scale-868d98995b-x9t4c  Pod         �[32m✔�[0m Running      18s  ready:1/1
      ├──□ dynamic-stable-scale-868d98995b-gp6gf  Pod         �[32m✔�[0m Running      5s   ready:1/1
      ├──□ dynamic-stable-scale-868d98995b-l7gkh  Pod         �[32m✔�[0m Running      5s   ready:1/1
      └──□ dynamic-stable-scale-868d98995b-q4zf7  Pod         �[32m✔�[0m Running      5s   ready:1/1

---
apiVersion: argoproj.io/v1alpha1
kind: Rollout
metadata:
  annotations:
    rollout.argoproj.io/revision: "2"
  creationTimestamp: "2024-02-14T00:28:33Z"
  generation: 2
  labels:
    argo-rollouts.argoproj.io/controller-instance-id: argo-rollouts-e2e
    e2e-test-name: TestCanaryDynamicStableScale
  name: dynamic-stable-scale
  namespace: default
  resourceVersion: "6415"
  uid: 369f4b29-401d-490a-9aa2-437cb73140fc
spec:
  replicas: 4
  selector:
    matchLabels:
      app: dynamic-stable-scale
  strategy:
    canary:
      canaryService: dynamic-stable-scale-canary
      dynamicStableScale: true
      stableService: dynamic-stable-scale-stable
      steps:
      - setWeight: 25
      - pause: {}
      - setWeight: 75
      - pause: {}
      trafficRouting:
        alb:
          ingress: dynamic-stable-scale-ingress
          rootService: dynamic-stable-scale-root
          servicePort: 80
  template:
    metadata:
      annotations:
        update: "2024-02-14T00:28:37.83571815Z"
      creationTimestamp: null
      labels:
        app: dynamic-stable-scale
    spec:
      containers:
      - image: nginx:1.19-alpine
        name: dynamic-stable-scale
        ports:
        - containerPort: 80
          name: http
          protocol: TCP
        resources:
          requests:
            cpu: 5m
            memory: 16Mi
      readinessGates:
      - conditionType: argoproj.io/e2e-readiness
status:
  HPAReplicas: 5
  abort: true
  abortedAt: "2024-02-14T00:28:46Z"
  availableReplicas: 5
  blueGreen: {}
  canary:
    weights:
      canary:
        podTemplateHash: 5689547dfd
        serviceName: dynamic-stable-scale-canary
        weight: 0
      stable:
        podTemplateHash: 868d98995b
        serviceName: dynamic-stable-scale-stable
        weight: 100
  conditions:
  - lastTransitionTime: "2024-02-14T00:28:37Z"
    lastUpdateTime: "2024-02-14T00:28:37Z"
    message: Rollout has minimum availability
    reason: AvailableReason
    status: "True"
    type: Available
  - lastTransitionTime: "2024-02-14T00:28:37Z"
    lastUpdateTime: "2024-02-14T00:28:37Z"
    message: Rollout is not healthy
    reason: RolloutHealthy
    status: "False"
    type: Healthy
  - lastTransitionTime: "2024-02-14T00:28:37Z"
    lastUpdateTime: "2024-02-14T00:28:37Z"
    message: RolloutCompleted
    reason: RolloutCompleted
    status: "False"
    type: Completed
  - lastTransitionTime: "2024-02-14T00:28:44Z"
    lastUpdateTime: "2024-02-14T00:28:44Z"
    message: Rollout is paused
    reason: RolloutPaused
    status: "True"
    type: Paused
  - lastTransitionTime: "2024-02-14T00:28:46Z"
    lastUpdateTime: "2024-02-14T00:28:46Z"
    message: Rollout aborted update to revision 2
    reason: RolloutAborted
    status: "False"
    type: Progressing
  currentPodHash: 5689547dfd
  currentStepHash: 7c5b668566
  currentStepIndex: 0
  message: 'RolloutAborted: Rollout aborted update to revision 2'
  observedGeneration: "2"
  phase: Degraded
  readyReplicas: 5
  replicas: 5
  selector: app=dynamic-stable-scale
  stableRS: 868d98995b
  updatedReplicas: 1

2024-02-14T00:28:33Z         Normal    RolloutAddedToInformer    Rollout resource added to informer: default/dynamic-stable-scale
2024-02-14T00:28:33Z         Normal    RolloutNotCompleted       Rollout not completed, started update to revision 1 (868d98995b)
2024-02-14T00:28:33Z         Normal    RolloutUpdated            Rollout updated to revision 1
2024-02-14T00:28:33Z         Normal    NewReplicaSetCreated      Created ReplicaSet dynamic-stable-scale-868d98995b (revision 1)
2024-02-14T00:28:33Z         Normal    PatchingALBIngress        Updating Ingress `dynamic-stable-scale-ingress` to desiredWeight '0'
2024-02-14T00:28:33Z (x2)    Normal    TrafficWeightUpdated      Traffic weight updated to 0
2024-02-14T00:28:33Z         Normal    ScalingReplicaSet         Scaled up ReplicaSet dynamic-stable-scale-868d98995b (revision 1) from 0 to 4
2024-02-14T00:28:33Z         Normal    RolloutCompleted          Rollout completed update to revision 1 (868d98995b): Initial deploy
2024-02-14T00:28:37Z         Normal    SwitchService             Switched selector for service 'dynamic-stable-scale-stable' from '' to '868d98995b'
2024-02-14T00:28:37Z         Normal    SwitchService             Switched selector for service 'dynamic-stable-scale-canary' from '' to '868d98995b'
2024-02-14T00:28:37Z         Normal    RolloutNotCompleted       Rollout not completed, started update to revision 2 (5689547dfd)
2024-02-14T00:28:37Z         Normal    RolloutUpdated            Rollout updated to revision 2
2024-02-14T00:28:37Z         Normal    NewReplicaSetCreated      Created ReplicaSet dynamic-stable-scale-5689547dfd (revision 2)
2024-02-14T00:28:37Z (x3)    Normal    TrafficWeightUpdated      Traffic weight updated 
2024-02-14T00:28:37Z         Normal    ScalingReplicaSet         Scaled up ReplicaSet dynamic-stable-scale-5689547dfd (revision 2) from 0 to 1
2024-02-14T00:28:39Z         Normal    SwitchService             Switched selector for service 'dynamic-stable-scale-canary' from '868d98995b' to '5689547dfd'
2024-02-14T00:28:39Z         Normal    PatchingALBIngress        Updating Ingress `dynamic-stable-scale-ingress` to desiredWeight '25'
2024-02-14T00:28:39Z         Normal    TrafficWeightUpdated      Traffic weight updated from 0 to 25
2024-02-14T00:28:39Z         Normal    RolloutStepCompleted      Rollout step 1/4 completed (setWeight: 25)
2024-02-14T00:28:39Z         Normal    ScalingReplicaSet         Scaled down ReplicaSet dynamic-stable-scale-868d98995b (revision 1) from 4 to 3
2024-02-14T00:28:39Z         Normal    RolloutPaused             Rollout is paused (CanaryPauseStep)
2024-02-14T00:28:41Z         Normal    RolloutStepCompleted      Rollout step 2/4 completed (pause)

time="2024-02-14T00:28:51Z" level=info msg="Deleting e2e-test-name=TestCanaryDynamicStableScale"
W0214 00:28:51.934915   21758 warnings.go:70] policy/v1beta1 PodDisruptionBudget is deprecated in v1.21+, unavailable in v1.25+; use policy/v1 PodDisruptionBudget
W0214 00:28:51.939620   21758 warnings.go:70] policy/v1beta1 PodDisruptionBudget is deprecated in v1.21+, unavailable in v1.25+; use policy/v1 PodDisruptionBudget
    --- FAIL: TestCanarySuite/TestCanaryDynamicStableScale (20.07s)

Failed
=== RUN   TestCanarySuite
time="2024-02-14T00:28:31Z" level=info msg="analysistemplate.argoproj.io/sleep-job created\n"
--- FAIL: TestCanarySuite (358.85s)

Failed
=== RUN   TestAnalysisSuite/TestCanaryBackgroundAnalysis
time="2024-02-14T00:22:30Z" level=info msg="Deleting e2e-test-name=TestCanaryBackgroundAnalysis"
time="2024-02-14T00:22:31Z" level=info msg="rollout.argoproj.io/rollout-background-analysis created\n" rollout=rollout-background-analysis
time="2024-02-14T00:22:31Z" level=info msg="Waiting for condition: status=Healthy" rollout=rollout-background-analysis
time="2024-02-14T00:22:33Z" level=info msg="Condition 'status=Healthy' met after 2s" rollout=rollout-background-analysis
time="2024-02-14T00:22:33Z" level=info msg="AnalysisRun expectation 'analysisrun count == 0' met" rollout=rollout-background-analysis
time="2024-02-14T00:22:33Z" level=info msg="Updated rollout pod spec: 2024-02-14T00:22:33.201563082Z" rollout=rollout-background-analysis
time="2024-02-14T00:22:33Z" level=info msg="Waiting for condition: status=Paused" rollout=rollout-background-analysis
time="2024-02-14T00:22:35Z" level=info msg="Condition 'status=Paused' met after 2s" rollout=rollout-background-analysis
time="2024-02-14T00:22:35Z" level=info msg="AnalysisRun expectation 'analysisrun count == 1' met" rollout=rollout-background-analysis
time="2024-02-14T00:22:35Z" level=info msg="Background AnalysisRun expectation 'background analysis phase == Running' met" rollout=rollout-background-analysis
time="2024-02-14T00:22:35Z" level=info msg="Promoted rollout" rollout=rollout-background-analysis
time="2024-02-14T00:22:35Z" level=info msg="Waiting for condition: status=Healthy" rollout=rollout-background-analysis
    when.go:455: timeout after 2m0s waiting for condition status=Healthy
Name:            rollout-background-analysis
Namespace:       default
Status:          �[31m✖�[0m Degraded
Message:         RolloutAborted: Rollout aborted update to revision 2: Metric "web" assessed Error due to consecutiveErrors (5) > consecutiveErrorLimit (4): "Error Message: received non 2xx response code: 401"
Strategy:        Canary
  Step:          0/2
  SetWeight:     0
  ActualWeight:  0
Images:          nginx:1.19-alpine (�[32mstable�[0m)
Replicas:
  Desired:       1
  Current:       1
  Updated:       0
  Ready:         1
  Available:     1

NAME                                                    KIND         STATUS        AGE   INFO
⟳ rollout-background-analysis                           Rollout      �[31m✖�[0m Degraded    2m4s  
├──# revision:2                                                                          
│  ├──⧉ �[33mrollout-background-analysis-8459477799�[0m          ReplicaSet   �[0m•�[0m ScaledDown  2m2s  �[33mcanary�[0m
│  └──α �[0mrollout-background-analysis-8459477799-2�[0m        AnalysisRun  �[31m⚠�[0m Error       2m2s  �[31m⚠�[0m 5
└──# revision:1                                                                          
   └──⧉ �[32mrollout-background-analysis-fc7fcff6d�[0m           ReplicaSet   �[32m✔�[0m Healthy     2m4s  �[32mstable�[0m
      └──□ rollout-background-analysis-fc7fcff6d-dgb8x  Pod          �[32m✔�[0m Running     2m4s  ready:1/1

---
apiVersion: argoproj.io/v1alpha1
kind: Rollout
metadata:
  annotations:
    rollout.argoproj.io/revision: "2"
  creationTimestamp: "2024-02-14T00:22:31Z"
  generation: 3
  labels:
    argo-rollouts.argoproj.io/controller-instance-id: argo-rollouts-e2e
    e2e-test-name: TestCanaryBackgroundAnalysis
  name: rollout-background-analysis
  namespace: default
  resourceVersion: "2493"
  uid: 4f166d09-d9a5-494a-a7c8-61bab83c7385
spec:
  replicas: 1
  selector:
    matchLabels:
      app: rollout-background-analysis
  strategy:
    canary:
      analysis:
        analysisRunMetadata: {}
        templates:
        - templateName: web-background
      steps:
      - setWeight: 10
      - pause: {}
  template:
    metadata:
      annotations:
        update: "2024-02-14T00:22:33.201563082Z"
      creationTimestamp: null
      labels:
        app: rollout-background-analysis
    spec:
      containers:
      - image: nginx:1.19-alpine
        name: rollouts-demo
        resources:
          requests:
            cpu: 5m
            memory: 16Mi
status:
  HPAReplicas: 1
  abort: true
  abortedAt: "2024-02-14T00:23:13Z"
  availableReplicas: 1
  blueGreen: {}
  canary:
    currentBackgroundAnalysisRunStatus:
      message: 'Metric "web" assessed Error due to consecutiveErrors (5) > consecutiveErrorLimit
        (4): "Error Message: received non 2xx response code: 401"'
      name: rollout-background-analysis-8459477799-2
      status: Error
  conditions:
  - lastTransitionTime: "2024-02-14T00:22:33Z"
    lastUpdateTime: "2024-02-14T00:22:33Z"
    message: Rollout has minimum availability
    reason: AvailableReason
    status: "True"
    type: Available
  - lastTransitionTime: "2024-02-14T00:22:33Z"
    lastUpdateTime: "2024-02-14T00:22:33Z"
    message: Rollout is not healthy
    reason: RolloutHealthy
    status: "False"
    type: Healthy
  - lastTransitionTime: "2024-02-14T00:22:33Z"
    lastUpdateTime: "2024-02-14T00:22:33Z"
    message: RolloutCompleted
    reason: RolloutCompleted
    status: "False"
    type: Completed
  - lastTransitionTime: "2024-02-14T00:22:35Z"
    lastUpdateTime: "2024-02-14T00:22:35Z"
    message: Rollout is paused
    reason: RolloutPaused
    status: "True"
    type: Paused
  - lastTransitionTime: "2024-02-14T00:23:13Z"
    lastUpdateTime: "2024-02-14T00:23:13Z"
    message: 'Rollout aborted update to revision 2: Metric "web" assessed Error due
      to consecutiveErrors (5) > consecutiveErrorLimit (4): "Error Message: received
      non 2xx response code: 401"'
    reason: RolloutAborted
    status: "False"
    type: Progressing
  currentPodHash: "8459477799"
  currentStepHash: 849b756fd4
  currentStepIndex: 0
  message: 'RolloutAborted: Rollout aborted update to revision 2: Metric "web" assessed
    Error due to consecutiveErrors (5) > consecutiveErrorLimit (4): "Error Message:
    received non 2xx response code: 401"'
  observedGeneration: "3"
  phase: Degraded
  readyReplicas: 1
  replicas: 1
  selector: app=rollout-background-analysis
  stableRS: fc7fcff6d

2024-02-14T00:22:31Z    Normal     RolloutAddedToInformer    Rollout resource added to informer: default/rollout-background-analysis
2024-02-14T00:22:31Z    Normal     RolloutNotCompleted       Rollout not completed, started update to revision 1 (fc7fcff6d)
2024-02-14T00:22:31Z    Normal     RolloutUpdated            Rollout updated to revision 1
2024-02-14T00:22:31Z    Normal     NewReplicaSetCreated      Created ReplicaSet rollout-background-analysis-fc7fcff6d (revision 1)
2024-02-14T00:22:31Z    Normal     ScalingReplicaSet         Scaled up ReplicaSet rollout-background-analysis-fc7fcff6d (revision 1) from 0 to 1
2024-02-14T00:22:31Z    Normal     RolloutCompleted          Rollout completed update to revision 1 (fc7fcff6d): Initial deploy
2024-02-14T00:22:33Z    Normal     RolloutNotCompleted       Rollout not completed, started update to revision 2 (8459477799)
2024-02-14T00:22:33Z    Normal     RolloutUpdated            Rollout updated to revision 2
2024-02-14T00:22:33Z    Normal     NewReplicaSetCreated      Created ReplicaSet rollout-background-analysis-8459477799 (revision 2)
2024-02-14T00:22:33Z    Normal     AnalysisRunRunning        Background Analysis Run 'rollout-background-analysis-8459477799-2' Status New: 'Running' Previous: 'NoPreviousStatus'
2024-02-14T00:22:33Z    Normal     ScalingReplicaSet         Scaled up ReplicaSet rollout-background-analysis-8459477799 (revision 2) from 0 to 1
2024-02-14T00:22:35Z    Normal     RolloutStepCompleted      Rollout step 1/2 completed (setWeight: 10)
2024-02-14T00:22:35Z    Normal     RolloutPaused             Rollout is paused (CanaryPauseStep)
2024-02-14T00:23:13Z    Warning    AnalysisRunError          Background Analysis Run 'rollout-background-analysis-8459477799-2' Status New: 'Error' Previous: 'Running'
2024-02-14T00:23:13Z    Warning    RolloutAborted            Rollout aborted update to revision 2: Metric "web" assessed Error due to consecutiveErrors (5) > consecutiveErrorLimit (4): "Error Message: received non 2xx response code: 401"
2024-02-14T00:23:13Z    Normal     ScalingReplicaSet         Scaled down ReplicaSet rollout-background-analysis-8459477799 (revision 2) from 1 to 0

time="2024-02-14T00:24:35Z" level=info msg="Deleting e2e-test-name=TestCanaryBackgroundAnalysis"
    --- FAIL: TestAnalysisSuite/TestCanaryBackgroundAnalysis (124.57s)

Failed
=== RUN   TestAnalysisSuite
time="2024-02-14T00:18:50Z" level=info msg="analysistemplate.argoproj.io/web-background created\n"
time="2024-02-14T00:18:50Z" level=info msg="analysistemplate.argoproj.io/sleep-job created\n"
time="2024-02-14T00:18:50Z" level=info msg="analysistemplate.argoproj.io/multiple-job created\n"
time="2024-02-14T00:18:51Z" level=info msg="analysistemplate.argoproj.io/multiple-job-fail created\n"
--- FAIL: TestAnalysisSuite (387.39s)