[GLBC] Surface event when front-end not created

[bowei]

From @pijusn on May 4, 2017 11:2

I recently ran following Shell script with given Kubernetes definitions and after some time, GLBC was created but it did not get any front-end created.

set -ex

read -p "Press any key to start..." -sn1

gcloud config set project echo-test-166405
gcloud container clusters create echo-cluster --machine-type="g1-small" --num-nodes=1

echo "Cluster created"

openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout /tmp/tls.key -out /tmp/tls.crt -subj "/C=LT"
kubectl create secret tls echoserver-tls --key /tmp/tls.key --cert /tmp/tls.crt

# Note: At this point I just waited a minute to make sure it's all OK.
echo "Self-generated certificate created"
read -p "Press any key to continue... " -sn1

kubectl apply -f echoserver/00-namespace.yaml
kubectl apply -f echoserver/deployment.yaml
kubectl apply -f echoserver/service.yaml
kubectl apply -f echoserver/ingress-tls.yaml

Here are all YAML files merged into one:

apiVersion: v1
kind: Namespace
metadata:
  name: echoserver
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: echoserver
  namespace: echoserver
spec:
  replicas: 1
  template:
    metadata:
      labels:
        app: echoserver
    spec:
      containers:
      - image: gcr.io/google_containers/echoserver:1.0
        imagePullPolicy: Always
        name: echoserver
        ports:
        - containerPort: 8080
---
apiVersion: v1
kind: Service
metadata:
  name: echoserver
  namespace: echoserver
spec:
  ports:
  - port: 80
    targetPort: 8080
    protocol: TCP
  type: NodePort
  selector:
    app: echoserver
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: echoserver
  namespace: echoserver
  annotations:
    kubernetes.io/tls-acme: "true"
    kubernetes.io/ingress.class: "gce"
    kubernetes.io/ingress.allow-http: "false"
spec:
  tls:
  - hosts:
    - echo.pijusn.eu
    secretName: echoserver-tls
  rules:
  - host: echo.pijusn.eu
    http:
      paths:
      - backend:
          serviceName: echoserver
          servicePort: 80

I expected HTTPS front-end to be created. It was a fresh project, fresh cluster so no quotes were kicking in.

After I removed kubernetes.io/ingress.allow-http: "false" it did create an HTTP front-end but still did not create HTTPS one.

This seems like an issue. Also, if you have ideas where to look for an error message or something (why it failed to create it) - please share.

Copied from original issue: kubernetes/ingress-nginx#686

[bowei]

From @tonglil on May 8, 2017 21:50

Ok, so there are a few things here that result in you seeing what you see.

Firstly, kube-lego needs to be install in your cluster for the tls-acme annotation to work (for getting a cert from letsencrypt).

Secondly, because there is no cert in the cluster (from kube-lego) AND http is disabled, then no front-end will be created.

Once you allow http by setting ingress.allow-http to true (default setting), then it will create the http front end, but again, kube-lego is not setup to create the cert for the https front end.

---

@nicksardo if you are comfortable, I would like to contribute to this repository by labeling issues nginx or gce.

[bowei]

From @nicksardo on May 8, 2017 22:6

@tonglil Correct me if I'm wrong, but I believe for labeling power you must be an org member and get write permission for the repo. With the start of the governance, I think all org changes are frozen for the time being.

[bowei]

From @tonglil on May 8, 2017 22:14

Gotcha, no worries. Happy to apply again another time.

[bowei]

From @pijusn on May 9, 2017 4:33

@tonglil I actually used a self-signed certificate. See (you can also find it in the Shell script I pasted earlier):

openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout /tmp/tls.key -out /tmp/tls.crt -subj "/C=LT"
kubectl create secret tls echoserver-tls --key /tmp/tls.key --cert /tmp/tls.crt

I would expect Kubernetes not to depend on kube-lego. To be specific, I would expect it to create an HTTPS frontend with the provided certificate even if it's not trusted, is issued for a different domain or whatever.

I now know that kube-lego does not support it but (correct me if I'm wrong) ACME server ignores certificate when using HTTPS. That's a nice way to kick-start LB setup. To achieve it, ingress controller should support such scenarios.

[bowei]

From @tonglil on May 9, 2017 5:12

@pijusn oops, sorry I missed that part.

What happens if you do this:

spec:
  tls:
-  - hosts:
-    - echo.pijusn.eu
-    secretName: echoserver-tls
+  - secretName: echoserver-tls

You can also see if there are any events emitted by the Ingress with kubectl describe ingress echoserver.

[bowei]

From @nicksardo on May 9, 2017 5:37

I'm betting you're creating the secret in the default namespace but the ingress exists in echoserver. It really should be surfaced as an event when the tls cert cannot be fetched.

[bowei]

From @pijusn on May 9, 2017 6:10

@nicksardo I think you are right. That would certainly make sense. I will verify it some time this week. Thank you for pointing it out.

[bowei]

From @nicksardo on May 10, 2017 17:16

Let's use this issue to track surfacing an event when cert lookup fails.

In the future, stack overflow would be a better medium for this question.

[bowei]

From @sebbov on September 15, 2017 16:54

+1 I also hit this issue and was at a loss to find out what the issue was. The target proxy and global forwarding rule would just not be created. Users should have visibility into any errors in ingress => GCP object creation pipeline.

FWIW, in my case, the TLS secret did exist in the correct namespace, but because of the way I had created it, a botched attempt at copying it from another cluster's secret, it was in an incorrect state.

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

Prevent issues from auto-closing with an /lifecycle frozen comment.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or @fejta.
/lifecycle stale

[fejta-bot]

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle rotten
/remove-lifecycle stale

[tonglil]

/close