[stable/mongdb-replicaset] mongodb chart in dns race condition

[aba182]

BUG REPORT

Version of Helm and Kubernetes:

# helm version
Client: &version.Version{SemVer:"v2.8.2", GitCommit:"a80231648a1473929271764b920a8e346f6de844", GitTreeState:"clean"}
Server: &version.Version{SemVer:"v2.8.2", GitCommit:"a80231648a1473929271764b920a8e346f6de844", GitTreeState:"clean"}

# kubectl version
Client Version: version.Info{Major:"1", Minor:"9", GitVersion:"v1.9.4", GitCommit:"bee2d1505c4fe820744d26d41ecd3fdd4a3d6546", GitTreeState:"clean", BuildDate:"2018-03-12T16:29:47Z", GoVersion:"go1.9.3", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"9", GitVersion:"v1.9.4", GitCommit:"bee2d1505c4fe820744d26d41ecd3fdd4a3d6546", GitTreeState:"clean", BuildDate:"2018-03-12T16:21:35Z", GoVersion:"go1.9.3", Compiler:"gc", Platform:"linux/amd64"}

Which chart:
mongodb replicaset https://github.com/kubernetes/charts/tree/master/stable/mongodb-replicaset

What happened:

Service can't create DNS entry so bootstrap system can't find dns entry which blocks deployment.

What you expected to happen:
DNS to be created and peer-finder to do DNS lookup and config magic

How to reproduce it (as minimally and precisely as possible):
helm install --name sonar stable/mongodb-replicaset -f values.yml

cat values.yml
persistentVolume:
  enabled: true
  storageClass: "glusterfs-storage"
  accessModes:
    - ReadWriteOnce
  size: 200Gi
  annotations: {volume.beta.kubernetes.io/storage-class: glusterfs-storage}

Logs:
kube DNS:

kubectl logs kube-dns-7b4c7c64df-fdczh -n kube-system kubedns
...
I0425 15:47:53.857281       1 dns.go:555] Could not find endpoints for service "sonar-mongodb-replicaset" in namespace "default". DNS records will be created once endpoints show up.

bootstrap:

# kubectl logs sonar-mongodb-replicaset-0 bootstrap
2018/04/25 15:49:10 lookup sonar-mongodb-replicaset on 10.96.0.10:53: read udp 10.244.9.6:49023->10.96.0.10:53: i/o timeout

Service is present with endpoint:

# kubectl describe svc sonar-mongodb-replicaset
Name:              sonar-mongodb-replicaset
Namespace:         default
Labels:            app=mongodb-replicaset
                   chart=mongodb-replicaset-3.3.0
                   heritage=Tiller
                   release=sonar
Annotations:       service.alpha.kubernetes.io/tolerate-unready-endpoints=true
Selector:          app=mongodb-replicaset,release=sonar
Type:              ClusterIP
IP:                None
Port:              peer  27017/TCP
TargetPort:        27017/TCP
Endpoints:         10.244.9.6:27017
Session Affinity:  None
Events:            <none>

[aba182]

Never mind, this is caused by internal DNS issues with my cluster. Not sure on resolution but pretty sure it isn't chart related.

[aba182]

For anyone who ends up here by chance. This issue was caused by the changing of kubeadm serviceCIDR but the kubelet systemd config is hard coded. Need to change to my defined location fix:

kubernetes/kubeadm#83