feat: report status when stack is locked

[rquitales]

Proposed changes

• Surfaces Locked Stack Errors:

  • The agent server now returns a structured response instead of an error when the Pulumi CLI returns a 409 (Conflict) error.
  • Clients can now determine well known Pulumi CLI errors through the PulumiErrorInfo protobuf message

• Improves Stack CR Status Updates:

  • The error message from UpdateCR.status.message is now correctly propagated to StackCR.status.lastUpdate.message.
  • Ensures locked stack errors are surfaced in the Stack CR's status subresource.

Example Stack CR Status Block

status:
    conditions:
    - lastTransitionTime: "2025-02-06T00:11:09Z"
      message: reconciliation is in progress
      reason: NotReadyInProgress
      status: "False"
      type: Ready
    - lastTransitionTime: "2025-02-06T00:11:09Z"
      message: 4 update failure(s)
      reason: RetryingAfterFailure
      status: "True"
      type: Reconciling
    lastUpdate:
      failures: 4
      generation: 4
      lastAttemptedCommit: sha256:f335a9e0bc445b0dbe3187371f56017bcdd66e23b68c6eda54910eeb48d5e3a0
      lastResyncTime: "2025-02-06T00:26:33Z"
      lastSuccessfulCommit: sha256:f335a9e0bc445b0dbe3187371f56017bcdd66e23b68c6eda54910eeb48d5e3a0
      message: Another update is currently in progress
      name: nginx-stack-194d8a6b139
      state: failed
      type: up
    observedGeneration: 4
    outputs:
      availableReplicas: 1

Example Update CR Status Block

status:
  conditions:
  - lastTransitionTime: "2025-02-06T00:26:33Z"
    message: ""
    observedGeneration: 1
    reason: Complete
    status: "False"
    type: Progressing
  - lastTransitionTime: "2025-02-06T00:26:33Z"
    message: Another update is currently in progress
    observedGeneration: 1
    reason: StackLocked
    status: "True"
    type: Failed
  - lastTransitionTime: "2025-02-06T00:26:33Z"
    message: ""
    observedGeneration: 1
    reason: Updated
    status: "True"
    type: Complete
  endTime: "1970-01-01T00:00:00Z"
  message: Another update is currently in progress
  observedGeneration: 1
  startTime: "1970-01-01T00:00:00Z"

Testing

• Added envtests to validate that statuses are correctly surfaced.
• Manually validated on a GKE cluster.

Related Issues

Fixes: #806
Fixes: #736

[rquitales]

This change is part of the following stack:

• feat: add WorkspaceReclaimPolicy to Stack spec #804
  • feat: report status when stack is locked #807 ◀
    • fix: do not delete workspace pod on authz errors #805

_{Change managed by git-spice.}

[codecov]

Codecov Report

Attention: Patch coverage is 54.79452% with 33 lines in your changes missing coverage. Please review.

Project coverage is 51.48%. Comparing base (e46b376) to head (a3788c0).
Report is 2 commits behind head on master.

Files with missing lines	Patch %	Lines
...ator/internal/controller/auto/update_controller.go	64.00%	17 Missing and 1 partial ⚠️
agent/pkg/server/pulumi_errors.go	35.71%	8 Missing and 1 partial ⚠️
agent/pkg/server/server.go	25.00%	6 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master     #807      +/-   ##
==========================================
+ Coverage   51.38%   51.48%   +0.10%     
==========================================
  Files          30       31       +1     
  Lines        4231     4296      +65     
==========================================
+ Hits         2174     2212      +38     
- Misses       1870     1895      +25     
- Partials      187      189       +2     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[rquitales]

This is the core logic for determining if we have a structured error, and uses it for surfacing information about Pulumi errors back to the user.

[EronWright]

Looking really good

[EronWright]

LGTM