[BUG] Workload identity has broken after warning and upgrade to latest Kubernetes AKS cluster version #3546
Labels
resolution/answer-provided
Provided answer to issue, question or feedback.
Comments
|
@xtianus79 It's not really a bug, but rather a redesign. The need for the pod label was announced last year ~6 weeks in advance (Release 2022-12-04). You can find more about reasoning behind the change here: |
|
Action required from @Azure/aks-pm |
|
Issue needing attention of @Azure/aks-leads |
CocoWang-wql
added
resolution/answer-provided
|
Thanks for reaching out. I'm closing this issue as it was marked with "Answer Provided" and it hasn't had activity for 2 days. |
ghost
locked as resolved and limited conversation to collaborators
This issue was closed.
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Describe the bug
I got an email back in Janurary 1/24/23 to be exact, that I needed to add the
azure.workload.identity/use: "true"label in your pod labels as soon as possible.I also, got the email that I needed to upgrade my cluster (not a fan auto upgrading) that's scary per se. From this version
AKS is retiring v1.23.x on 2 April 2023BTW thank you very much for the emails they were apt.
So, I upgraded my AKS and an engineer reported that the service layer related to the workload identity pod was down. So I investigated. Also to note, the reason why I'm explaining the story this way is because the documentation seems to be different compared to the code I was running before and it's confusing me on what I should focus on to correct the issue. Let me explain.
I updated the cluster and went back to the workload identity email and documentation.
It now seems that the label of
azure.workload.identity/use: "true"should be put in the pod instead of theServiceAccountwhich is where I had the label before. Noted here.Side question, I can't find that service account anywhere. Where is that exactly/how do you find that? Is it in AKS or another resource group. Anyway, I had my Yaml file.
The other file which is the pod yaml does not have the
azure.workload.identity/use: "true"in it BUT the new documentation is showing that is where it needs to be.My question is, should I remove the label from the service account, reestablish it, "I'd prefer not to do that but if I have to I have to" and then add the label to the pod layer?
It's just difficult to tell because I'm seeing the new documentation and not an upgrade instruction.
Thanks.
Here is the error message I'm getting.
{"status-code":500,"message":"Request failed with status code 500","system-response":"<!DOCTYPE html>\n<html lang=\"en\">\n<head>\n<meta charset=\"utf-8\">\n<title>Error</title>\n</head>\n<body>\n<pre>FetchError: request to http://xxxxxxxx-service:zzzz/xxxxxxxxx?xxxxxxx=xxxxxxxxxxx failed, reason: connect ECONNREFUSED xxxxxxx:xxxxxx<br> at ClientRequest.<anonymous> (fxxxxxxxxxxx)<br> at ClientRequest.emit (node:events:513:28)<br> at xxxxxxxx (node:_http_client:494:9)<br> at Socket.emit (node:events:513:28)<br> at emitErrorNT (node:internal/streams/destroy:157:8)<br> at emitErrorCloseNT (node:internal/streams/destroy:122:3)<br> at processTicksAndRejections (node:internal/process/task_queues:83:21)</pre>\n</body>\n</html>\n","time-stamp":"2023-03-17T11:40:09.918Z"}The text was updated successfully, but these errors were encountered: