[Release-1.27] - Traefik manifest has wrong location for podAnnotations

[brandond]

Backport fix for Traefik manifest has wrong location for podAnnotations

• Traefik manifest has wrong location for podAnnotations #9919

[ShylajaDevadiga]

Validated using latest commit id 2d48b19 on release-1.27 branch

Environment Details

Infrastructure
Cloud EC2 instance

Node(s) CPU architecture, OS, and Version:

cat /etc/os-release
NAME="SLES"
VERSION="15-SP5"
VERSION_ID="15.5"
PRETTY_NAME="SUSE Linux Enterprise Server 15 SP5"

Cluster Configuration:
Single node

Config.yaml:

cat /etc/rancher/k3s/config,yaml
write-kubeconfig-mode: "0644"
tls-san:
  - fake.fqdn.value
cluster-init: true

Steps to reproduce the issue and validate the fix

1. Install k3s
2. Check valuesCintent in traefik manifests

Replication results:

ec2-user@ip-172-31-13-122:~> k3s -v
k3s version v1.27.12+k3s1 (78ad5756)
go version go1.21.8
ec2-user@ip-172-31-13-122:~> sudo cat  /var/lib/rancher/k3s/server/manifests/traefik.yaml
---
apiVersion: helm.cattle.io/v1
kind: HelmChart
metadata:
  name: traefik-crd
  namespace: kube-system
spec:
  chart: https://%{KUBERNETES_API}%/static/charts/traefik-crd-25.0.2+up25.0.0.tgz
---
apiVersion: helm.cattle.io/v1
kind: HelmChart
metadata:
  name: traefik
  namespace: kube-system
spec:
  chart: https://%{KUBERNETES_API}%/static/charts/traefik-25.0.2+up25.0.0.tgz
  set:
    global.systemDefaultRegistry: ""
  valuesContent: |-
    podAnnotations:
      prometheus.io/port: "8082"
      prometheus.io/scrape: "true"
    providers:
      kubernetesIngress:
        publishedService:
          enabled: true
    priorityClassName: "system-cluster-critical"
    image:
      repository: "rancher/mirrored-library-traefik"
      tag: "2.10.5"
    tolerations:
    - key: "CriticalAddonsOnly"
      operator: "Exists"
    - key: "node-role.kubernetes.io/control-plane"
      operator: "Exists"
      effect: "NoSchedule"
    - key: "node-role.kubernetes.io/master"
      operator: "Exists"
      effect: "NoSchedule"
    service:
      ipFamilyPolicy: "PreferDualStack"

Validation results:
podAnnotations is nested as deployment.podAnnotations.

ec2-user@ip-172-31-1-234:~> k3s -v
k3s version v1.27.12+k3s-2d48b196 (2d48b196)
go version go1.21.8
ec2-user@ip-172-31-1-234:~> sudo chmod 644 /etc/rancher/k3s/k3s.yaml 
ec2-user@ip-172-31-1-234:~> sudo cat  /var/lib/rancher/k3s/server/manifests/traefik.yaml
---
apiVersion: helm.cattle.io/v1
kind: HelmChart
metadata:
  name: traefik-crd
  namespace: kube-system
spec:
  chart: https://%{KUBERNETES_API}%/static/charts/traefik-crd-25.0.3+up25.0.0.tgz
---
apiVersion: helm.cattle.io/v1
kind: HelmChart
metadata:
  name: traefik
  namespace: kube-system
spec:
  chart: https://%{KUBERNETES_API}%/static/charts/traefik-25.0.3+up25.0.0.tgz
  set:
    global.systemDefaultRegistry: ""
  valuesContent: |-
    deployment:
      podAnnotations:
        prometheus.io/port: "8082"
        prometheus.io/scrape: "true"
    providers:
      kubernetesIngress:
        publishedService:
          enabled: true
    priorityClassName: "system-cluster-critical"
    image:
      repository: "rancher/mirrored-library-traefik"
      tag: "2.10.7"
    tolerations:
    - key: "CriticalAddonsOnly"
      operator: "Exists"
    - key: "node-role.kubernetes.io/control-plane"
      operator: "Exists"
      effect: "NoSchedule"
    - key: "node-role.kubernetes.io/master"
      operator: "Exists"
      effect: "NoSchedule"
    service:
      ipFamilyPolicy: "PreferDualStack"