2 default StorageClasses were found

[thbkrkr]

I just created a new 1.14 GKE cluster and I could not create an ES cluster because 2 default StorageClasses were found:

1m  Warning  FailedCreate  statefulset/z-es-master                                  
  create Pod z-es-master-0 in StatefulSet z-es-master failed error: 
    Failed to create PVC elasticsearch-data-z-es-master-0: 
      persistentvolumeclaims "elasticsearch-data-bzz-es-master-0" is forbidden: 
        Internal error occurred: 2 default StorageClasses were found

We failed to change the default storage class because the existing storage class standard uses the storageclass.kubernetes.io/is-default-class annotation while we update the storageclass.beta.kubernetes.io/is-default-class annotation to false.

At the end, we have two storage classes with storageclass.kubernetes.io/is-default-class: "true".

> k get sc standard -o yaml                                                                                                                                
allowVolumeExpansion: true                                                                                                                                                            
apiVersion: storage.k8s.io/v1                                                                                                                                                         
kind: StorageClass                                                                                                                                                                    
metadata:                                                                                                                                                                             
  annotations:                                                                                                                                                                        
    storageclass.beta.kubernetes.io/is-default-class: "false"                                                                                                                         
    storageclass.kubernetes.io/is-default-class: "true"
...
> k get sc standard-customized -o yaml                                                                                                                     
allowVolumeExpansion: true                       
apiVersion: storage.k8s.io/v1                                                                                                                                                         
kind: StorageClass                                                                        
metadata:                                                                                                                                                                             
  annotations:                                                                                                                                                                        
    kubectl.kubernetes.io/last-applied-configuration: {...}
    storageclass.kubernetes.io/is-default-class: "true"
...

In CI, the job cloud-on-k8s-stack fails for the same reason (https://devops-ci.elastic.co/view/cloud-on-k8s/job/cloud-on-k8s-stack/32/console).