-
Notifications
You must be signed in to change notification settings - Fork 893
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fixed apiEnablements unordered issue #1304
fixed apiEnablements unordered issue #1304
Conversation
205932b
to
abfac96
Compare
Signed-off-by: pigletfly <[email protected]>
abfac96
to
2d797ac
Compare
Hi @pigletfly , I tested on my local without this pr, and found that the sequence of the contents of the Have you ever come across any reason that made the order inconsistency? |
yes, I run |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If that's the case, it's necessary to sort it.
@Garrybest Have you ever met this issue? This is the no-harm and insurance patch, but I want to figure out the root cause before merging this. |
Yes, I have met the same issue. I have a cluster of version 1.16.6, it shows different status:
apiEnablements:
- groupVersion: v1
resources:
- kind: Binding
name: bindings
- kind: ComponentStatus
name: componentstatuses
- kind: ConfigMap
name: configmaps
- kind: Endpoints
name: endpoints
- kind: Event
name: events
- kind: LimitRange
name: limitranges
- kind: Namespace
name: namespaces
- kind: Node
name: nodes
- kind: PersistentVolumeClaim
name: persistentvolumeclaims
- kind: PersistentVolume
name: persistentvolumes
- kind: Pod
name: pods
- kind: PodTemplate
name: podtemplates
- kind: ReplicationController
name: replicationcontrollers
- kind: ResourceQuota
name: resourcequotas
- kind: Secret
name: secrets
- kind: ServiceAccount
name: serviceaccounts
- kind: Service
name: services
- groupVersion: apiregistration.k8s.io/v1
resources:
- kind: APIService
name: apiservices
- groupVersion: apiregistration.k8s.io/v1beta1
resources:
- kind: APIService
name: apiservices
- groupVersion: extensions/v1beta1
resources:
- kind: Ingress
name: ingresses
- groupVersion: apps/v1
resources:
- kind: ControllerRevision
name: controllerrevisions
- kind: DaemonSet
name: daemonsets
- kind: Deployment
name: deployments
- kind: ReplicaSet
name: replicasets
- kind: StatefulSet
name: statefulsets
- groupVersion: events.k8s.io/v1beta1
resources:
- kind: Event
name: events
- groupVersion: authentication.k8s.io/v1
resources:
- kind: TokenReview
name: tokenreviews
- groupVersion: authentication.k8s.io/v1beta1
resources:
- kind: TokenReview
name: tokenreviews
- groupVersion: authorization.k8s.io/v1
resources:
- kind: LocalSubjectAccessReview
name: localsubjectaccessreviews
- kind: SelfSubjectAccessReview
name: selfsubjectaccessreviews
- kind: SelfSubjectRulesReview
name: selfsubjectrulesreviews
- kind: SubjectAccessReview
name: subjectaccessreviews
- groupVersion: authorization.k8s.io/v1beta1
resources:
- kind: LocalSubjectAccessReview
name: localsubjectaccessreviews
- kind: SelfSubjectAccessReview
name: selfsubjectaccessreviews
- kind: SelfSubjectRulesReview
name: selfsubjectrulesreviews
- kind: SubjectAccessReview
name: subjectaccessreviews
- groupVersion: autoscaling/v1
resources:
- kind: HorizontalPodAutoscaler
name: horizontalpodautoscalers
- groupVersion: autoscaling/v2beta1
resources:
- kind: HorizontalPodAutoscaler
name: horizontalpodautoscalers
- groupVersion: autoscaling/v2beta2
resources:
- kind: HorizontalPodAutoscaler
name: horizontalpodautoscalers
- groupVersion: batch/v1
resources:
- kind: Job
name: jobs
- groupVersion: batch/v1beta1
resources:
- kind: CronJob
name: cronjobs
- groupVersion: certificates.k8s.io/v1beta1
resources:
- kind: CertificateSigningRequest
name: certificatesigningrequests
- groupVersion: networking.k8s.io/v1
resources:
- kind: NetworkPolicy
name: networkpolicies
- groupVersion: networking.k8s.io/v1beta1
resources:
- kind: Ingress
name: ingresses
- groupVersion: policy/v1beta1
resources:
- kind: PodDisruptionBudget
name: poddisruptionbudgets
- kind: PodSecurityPolicy
name: podsecuritypolicies
- groupVersion: rbac.authorization.k8s.io/v1
resources:
- kind: ClusterRoleBinding
name: clusterrolebindings
- kind: ClusterRole
name: clusterroles
- kind: RoleBinding
name: rolebindings
- kind: Role
name: roles
- groupVersion: rbac.authorization.k8s.io/v1beta1
resources:
- kind: ClusterRoleBinding
name: clusterrolebindings
- kind: ClusterRole
name: clusterroles
- kind: RoleBinding
name: rolebindings
- kind: Role
name: roles
- groupVersion: storage.k8s.io/v1
resources:
- kind: StorageClass
name: storageclasses
- kind: VolumeAttachment
name: volumeattachments
- groupVersion: storage.k8s.io/v1beta1
resources:
- kind: CSIDriver
name: csidrivers
- kind: CSINode
name: csinodes
- kind: StorageClass
name: storageclasses
- kind: VolumeAttachment
name: volumeattachments
- groupVersion: admissionregistration.k8s.io/v1
resources:
- kind: MutatingWebhookConfiguration
name: mutatingwebhookconfigurations
- kind: ValidatingWebhookConfiguration
name: validatingwebhookconfigurations
- groupVersion: admissionregistration.k8s.io/v1beta1
resources:
- kind: MutatingWebhookConfiguration
name: mutatingwebhookconfigurations
- kind: ValidatingWebhookConfiguration
name: validatingwebhookconfigurations
- groupVersion: apiextensions.k8s.io/v1
resources:
- kind: CustomResourceDefinition
name: customresourcedefinitions
- groupVersion: apiextensions.k8s.io/v1beta1
resources:
- kind: CustomResourceDefinition
name: customresourcedefinitions
- groupVersion: scheduling.k8s.io/v1
resources:
- kind: PriorityClass
name: priorityclasses
- groupVersion: scheduling.k8s.io/v1beta1
resources:
- kind: PriorityClass
name: priorityclasses
- groupVersion: coordination.k8s.io/v1
resources:
- kind: Lease
name: leases
- groupVersion: coordination.k8s.io/v1beta1
resources:
- kind: Lease
name: leases
- groupVersion: node.k8s.io/v1beta1
resources:
- kind: RuntimeClass
name: runtimeclasses
- groupVersion: apps.tkestack.io/v1
resources:
- kind: TApp
name: tapps
- groupVersion: kubeflow.org/v1
resources:
- kind: TFJob
name: tfjobs
- groupVersion: kubeflow.org/v1alpha2
resources:
- kind: MPIJob
name: mpijobs
- groupVersion: kyverno.io/v1
resources:
- kind: GenerateRequest
name: generaterequests
- kind: ClusterPolicy
name: clusterpolicies
- kind: Policy
name: policies
- groupVersion: kyverno.io/v1alpha2
resources:
- kind: ClusterReportChangeRequest
name: clusterreportchangerequests
- kind: ReportChangeRequest
name: reportchangerequests
- groupVersion: lbcf.tkestack.io/v1
resources:
- kind: Bind
name: binds
- groupVersion: lbcf.tkestack.io/v1beta1
resources:
- kind: BackendGroup
name: backendgroups
- kind: LoadBalancerDriver
name: loadbalancerdrivers
- kind: LoadBalancer
name: loadbalancers
- kind: BackendRecord
name: backendrecords
- groupVersion: monitoring.coreos.com/v1
resources:
- kind: ThanosRuler
name: thanosrulers
- kind: Probe
name: probes
- kind: PrometheusRule
name: prometheusrules
- kind: PodMonitor
name: podmonitors
- kind: ServiceMonitor
name: servicemonitors
- kind: Alertmanager
name: alertmanagers
- kind: Prometheus
name: prometheuses
- groupVersion: monitoring.coreos.com/v1alpha1
resources:
- kind: AlertmanagerConfig
name: alertmanagerconfigs
- groupVersion: scheduling.tkestack.io/v1
resources:
- kind: TaskGroup
name: taskgroups
- groupVersion: storage.tkestack.io/v1
resources:
- kind: CSI
name: csis
- groupVersion: apps.kruise.io/v1beta1
resources:
- kind: StatefulSet
name: statefulsets
- groupVersion: apps.kruise.io/v1alpha1
resources:
- kind: AdvancedCronJob
name: advancedcronjobs
- kind: SidecarSet
name: sidecarsets
- kind: ImagePullJob
name: imagepulljobs
- kind: StatefulSet
name: statefulsets
- kind: NodeImage
name: nodeimages
- kind: DaemonSet
name: daemonsets
- kind: UnitedDeployment
name: uniteddeployments
- kind: BroadcastJob
name: broadcastjobs
- kind: CloneSet
name: clonesets
- kind: ContainerRecreateRequest
name: containerrecreaterequests
- groupVersion: data.fluid.io/v1alpha1
resources:
- kind: GooseFSRuntime
name: goosefsruntimes
- kind: DataBackup
name: databackups
- kind: Dataset
name: datasets
- kind: DataLoad
name: dataloads
- kind: JindoRuntime
name: jindoruntimes
- kind: AlluxioRuntime
name: alluxioruntimes
- groupVersion: kstone.tkestack.io/v1alpha1
resources:
- kind: EtcdCluster
name: etcdclusters
- kind: EtcdMonitor
name: etcdmonitors
- groupVersion: scheduling.incubator.k8s.io/v1alpha1
resources:
- kind: PodGroup
name: podgroups
- groupVersion: snapshot.storage.k8s.io/v1alpha1
resources:
- kind: VolumeSnapshotClass
name: volumesnapshotclasses
- kind: VolumeSnapshot
name: volumesnapshots
- kind: VolumeSnapshotContent
name: volumesnapshotcontents
- groupVersion: wgpolicyk8s.io/v1alpha2
resources:
- kind: ClusterPolicyReport
name: clusterpolicyreports
- kind: PolicyReport
name: policyreports
- groupVersion: flinkoperator.k8s.io/v1beta1
resources:
- kind: FlinkCluster
name: flinkclusters
- groupVersion: etcd.database.coreos.com/v1beta2
resources:
- kind: EtcdBackup
name: etcdbackups
- groupVersion: custom.metrics.k8s.io/v1beta2
- groupVersion: custom.metrics.k8s.io/v1beta1
- groupVersion: external.metrics.k8s.io/v1beta1
resources:
- kind: ExternalMetricValueList
name: prometheus_notifications_queue_length status:
apiEnablements:
- groupVersion: v1
resources:
- kind: Binding
name: bindings
- kind: ComponentStatus
name: componentstatuses
- kind: ConfigMap
name: configmaps
- kind: Endpoints
name: endpoints
- kind: Event
name: events
- kind: LimitRange
name: limitranges
- kind: Namespace
name: namespaces
- kind: Node
name: nodes
- kind: PersistentVolumeClaim
name: persistentvolumeclaims
- kind: PersistentVolume
name: persistentvolumes
- kind: Pod
name: pods
- kind: PodTemplate
name: podtemplates
- kind: ReplicationController
name: replicationcontrollers
- kind: ResourceQuota
name: resourcequotas
- kind: Secret
name: secrets
- kind: ServiceAccount
name: serviceaccounts
- kind: Service
name: services
- groupVersion: apiregistration.k8s.io/v1
resources:
- kind: APIService
name: apiservices
- groupVersion: apiregistration.k8s.io/v1beta1
resources:
- kind: APIService
name: apiservices
- groupVersion: extensions/v1beta1
resources:
- kind: Ingress
name: ingresses
- groupVersion: apps/v1
resources:
- kind: ControllerRevision
name: controllerrevisions
- kind: DaemonSet
name: daemonsets
- kind: Deployment
name: deployments
- kind: ReplicaSet
name: replicasets
- kind: StatefulSet
name: statefulsets
- groupVersion: events.k8s.io/v1beta1
resources:
- kind: Event
name: events
- groupVersion: authentication.k8s.io/v1
resources:
- kind: TokenReview
name: tokenreviews
- groupVersion: authentication.k8s.io/v1beta1
resources:
- kind: TokenReview
name: tokenreviews
- groupVersion: authorization.k8s.io/v1
resources:
- kind: LocalSubjectAccessReview
name: localsubjectaccessreviews
- kind: SelfSubjectAccessReview
name: selfsubjectaccessreviews
- kind: SelfSubjectRulesReview
name: selfsubjectrulesreviews
- kind: SubjectAccessReview
name: subjectaccessreviews
- groupVersion: authorization.k8s.io/v1beta1
resources:
- kind: LocalSubjectAccessReview
name: localsubjectaccessreviews
- kind: SelfSubjectAccessReview
name: selfsubjectaccessreviews
- kind: SelfSubjectRulesReview
name: selfsubjectrulesreviews
- kind: SubjectAccessReview
name: subjectaccessreviews
- groupVersion: autoscaling/v1
resources:
- kind: HorizontalPodAutoscaler
name: horizontalpodautoscalers
- groupVersion: autoscaling/v2beta1
resources:
- kind: HorizontalPodAutoscaler
name: horizontalpodautoscalers
- groupVersion: autoscaling/v2beta2
resources:
- kind: HorizontalPodAutoscaler
name: horizontalpodautoscalers
- groupVersion: batch/v1
resources:
- kind: Job
name: jobs
- groupVersion: batch/v1beta1
resources:
- kind: CronJob
name: cronjobs
- groupVersion: certificates.k8s.io/v1beta1
resources:
- kind: CertificateSigningRequest
name: certificatesigningrequests
- groupVersion: networking.k8s.io/v1
resources:
- kind: NetworkPolicy
name: networkpolicies
- groupVersion: networking.k8s.io/v1beta1
resources:
- kind: Ingress
name: ingresses
- groupVersion: policy/v1beta1
resources:
- kind: PodDisruptionBudget
name: poddisruptionbudgets
- kind: PodSecurityPolicy
name: podsecuritypolicies
- groupVersion: rbac.authorization.k8s.io/v1
resources:
- kind: ClusterRoleBinding
name: clusterrolebindings
- kind: ClusterRole
name: clusterroles
- kind: RoleBinding
name: rolebindings
- kind: Role
name: roles
- groupVersion: rbac.authorization.k8s.io/v1beta1
resources:
- kind: ClusterRoleBinding
name: clusterrolebindings
- kind: ClusterRole
name: clusterroles
- kind: RoleBinding
name: rolebindings
- kind: Role
name: roles
- groupVersion: storage.k8s.io/v1
resources:
- kind: StorageClass
name: storageclasses
- kind: VolumeAttachment
name: volumeattachments
- groupVersion: storage.k8s.io/v1beta1
resources:
- kind: CSIDriver
name: csidrivers
- kind: CSINode
name: csinodes
- kind: StorageClass
name: storageclasses
- kind: VolumeAttachment
name: volumeattachments
- groupVersion: admissionregistration.k8s.io/v1
resources:
- kind: MutatingWebhookConfiguration
name: mutatingwebhookconfigurations
- kind: ValidatingWebhookConfiguration
name: validatingwebhookconfigurations
- groupVersion: admissionregistration.k8s.io/v1beta1
resources:
- kind: MutatingWebhookConfiguration
name: mutatingwebhookconfigurations
- kind: ValidatingWebhookConfiguration
name: validatingwebhookconfigurations
- groupVersion: apiextensions.k8s.io/v1
resources:
- kind: CustomResourceDefinition
name: customresourcedefinitions
- groupVersion: apiextensions.k8s.io/v1beta1
resources:
- kind: CustomResourceDefinition
name: customresourcedefinitions
- groupVersion: scheduling.k8s.io/v1
resources:
- kind: PriorityClass
name: priorityclasses
- groupVersion: scheduling.k8s.io/v1beta1
resources:
- kind: PriorityClass
name: priorityclasses
- groupVersion: coordination.k8s.io/v1
resources:
- kind: Lease
name: leases
- groupVersion: coordination.k8s.io/v1beta1
resources:
- kind: Lease
name: leases
- groupVersion: node.k8s.io/v1beta1
resources:
- kind: RuntimeClass
name: runtimeclasses
- groupVersion: apps.tkestack.io/v1
resources:
- kind: TApp
name: tapps
- groupVersion: kubeflow.org/v1
resources:
- kind: TFJob
name: tfjobs
- groupVersion: kubeflow.org/v1alpha2
resources:
- kind: MPIJob
name: mpijobs
- groupVersion: kyverno.io/v1
resources:
- kind: GenerateRequest
name: generaterequests
- kind: ClusterPolicy
name: clusterpolicies
- kind: Policy
name: policies
- groupVersion: kyverno.io/v1alpha2
resources:
- kind: ReportChangeRequest
name: reportchangerequests
- kind: ClusterReportChangeRequest
name: clusterreportchangerequests
- groupVersion: lbcf.tkestack.io/v1
resources:
- kind: Bind
name: binds
- groupVersion: lbcf.tkestack.io/v1beta1
resources:
- kind: BackendGroup
name: backendgroups
- kind: LoadBalancerDriver
name: loadbalancerdrivers
- kind: LoadBalancer
name: loadbalancers
- kind: BackendRecord
name: backendrecords
- groupVersion: monitoring.coreos.com/v1
resources:
- kind: ServiceMonitor
name: servicemonitors
- kind: Prometheus
name: prometheuses
- kind: Alertmanager
name: alertmanagers
- kind: ThanosRuler
name: thanosrulers
- kind: PrometheusRule
name: prometheusrules
- kind: PodMonitor
name: podmonitors
- kind: Probe
name: probes
- groupVersion: monitoring.coreos.com/v1alpha1
resources:
- kind: AlertmanagerConfig
name: alertmanagerconfigs
- groupVersion: scheduling.tkestack.io/v1
resources:
- kind: TaskGroup
name: taskgroups
- groupVersion: storage.tkestack.io/v1
resources:
- kind: CSI
name: csis
- groupVersion: apps.kruise.io/v1beta1
resources:
- kind: StatefulSet
name: statefulsets
- groupVersion: apps.kruise.io/v1alpha1
resources:
- kind: DaemonSet
name: daemonsets
- kind: BroadcastJob
name: broadcastjobs
- kind: CloneSet
name: clonesets
- kind: UnitedDeployment
name: uniteddeployments
- kind: ContainerRecreateRequest
name: containerrecreaterequests
- kind: AdvancedCronJob
name: advancedcronjobs
- kind: SidecarSet
name: sidecarsets
- kind: StatefulSet
name: statefulsets
- kind: NodeImage
name: nodeimages
- kind: ImagePullJob
name: imagepulljobs
- groupVersion: data.fluid.io/v1alpha1
resources:
- kind: GooseFSRuntime
name: goosefsruntimes
- kind: DataBackup
name: databackups
- kind: Dataset
name: datasets
- kind: DataLoad
name: dataloads
- kind: JindoRuntime
name: jindoruntimes
- kind: AlluxioRuntime
name: alluxioruntimes
- groupVersion: kstone.tkestack.io/v1alpha1
resources:
- kind: EtcdMonitor
name: etcdmonitors
- kind: EtcdCluster
name: etcdclusters
- groupVersion: scheduling.incubator.k8s.io/v1alpha1
resources:
- kind: PodGroup
name: podgroups
- groupVersion: snapshot.storage.k8s.io/v1alpha1
resources:
- kind: VolumeSnapshot
name: volumesnapshots
- kind: VolumeSnapshotContent
name: volumesnapshotcontents
- kind: VolumeSnapshotClass
name: volumesnapshotclasses
- groupVersion: wgpolicyk8s.io/v1alpha2
resources:
- kind: PolicyReport
name: policyreports
- kind: ClusterPolicyReport
name: clusterpolicyreports
- groupVersion: flinkoperator.k8s.io/v1beta1
resources:
- kind: FlinkCluster
name: flinkclusters
- groupVersion: etcd.database.coreos.com/v1beta2
resources:
- kind: EtcdBackup
name: etcdbackups
- groupVersion: custom.metrics.k8s.io/v1beta2
- groupVersion: custom.metrics.k8s.io/v1beta1
- groupVersion: external.metrics.k8s.io/v1beta1
resources:
- kind: ExternalMetricValueList
name: prometheus_notifications_queue_length |
It seems like only the resources may have a different sequence. |
Yes, the apigroup are sorted but the resources defined by |
/retitle fixed apiEnablements unordered issue |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
/lgtm
/approve
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: RainbowMango The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
@pigletfly @Garrybest Do you think this PR should cherry-pick to |
By the way, I tried to figure out the root cause, echoing some of my founding here. For the Kubernetes native APIs, the groups are ordered. Karmada retrieves the APIs by No clue about the CRD resources yet. |
IMO, there is no need to rebase |
It seems like the CRD resource does not keep the consistency. |
What type of PR is this?
/kind cleanup
What this PR does / why we need it:
if apiResourceList is not sorted, the cluster object will keep updating.
Which issue(s) this PR fixes:
Fixes #
Special notes for your reviewer:
Does this PR introduce a user-facing change?: