Skip to content

Latest commit

 

History

History
252 lines (230 loc) · 17.7 KB

k8s-ingress.md

File metadata and controls

252 lines (230 loc) · 17.7 KB

K8S Ingress Introduction

k8s services can be exposed to external (outside of the cluster) in many ways. Popular ways are https://kubernetes.io/docs/concepts/services-networking/ingress/#alternatives. Ingress is another way to expose the service to external. Ingress provides layer 7 load balancing whereas the others provide layer 4 load balancing.

Types of ingress

  1. Single Service:
    This type is as equivalent to exposing the services with other ways.

    Sample yaml to create single-service:
        apiVersion: extensions/v1beta1
        kind: Ingress
        metadata:
            name: single-service
        spec:
            backend:
                serviceName: nginx
                servicePort: 80

    All the traffic would be routed to nginx service.

  2. Simple fanout:
    Based on the url path, the traffic would be routed to desired service.

    Sample yaml to create simple-fanout:
        apiVersion: extensions/v1beta1
        kind: ingress
        metadata:
            name: simple-fanout
        spec:
            rules:
                -   http:
                        paths:
                         -  path: /qa.html
                             backend:
                                 serviceName: qa-webserver
                                 servicePort: 80
                         -  path: /dev.html
                             backend:
                                 serviceName: dev-webserver
                                 servicePort: 80

    If the incoming traffic's url path matches to qa.html, the traffic would be routed to qa-webserver service. If the incoming traffic ul matches to dev.html, the traffic's would be touted to dev-webserver.

  3. Name based Virtual Hosting:
    Multiple host names use the same ip address. Based on the Host, traffic would be routed to desired service.

    Sample yaml to create name based virtual-hosting:
        apiVersion: extensions/v1beta1
        kind: ingress
        metadata:
            name: virtual-host
        spec:
            rules:
                - host: maps.google.com
                   http
                        paths:
                        - backend:
                              serviceName: google-maps-webserver
                              servicePort: 80
                - host: images.google.com
                   http
                        paths:
                        - backend:
                              serviceName: google-images-webserver
                              servicePort: 80

    If the incoming traffic's host header matches to maps.google.com, the traffic would be routed to google-maps-webserver. If the incoming traffic's host header matches to images.google.com, the traffic would be routed to google-images-webserver.

For more information please refer https://kubernetes.io/docs/concepts/services-networking/ingress/

Ingress in Contrail

Ingress is implemented through load balancer feature in contrail. [Please refer contrail feature guide for more information on contrail load balancer]. Whenever ingress is configured in k8s, contrail-kube-manager creates the load balancer object in contrail-controller. Contrail service contrail-svc-monitor listens for the load balancer objects and launches the haproxy with appropriate configuration based on the ingress spec rules in active-standby mode in two active compute nodes.

Detailed Explanation with simple-fanout example:

Simple-fanout yaml:

        apiVersion: extensions/v1beta1
        kind: ingress
        metadata:
            name: simple-fanout
        spec:
            rules:
                - http:
                      paths:
                         - path: /qa.html
                            backend:
                               serviceName: qa-webserver
                               servicePort: 80
                      paths:
                         - path: /dev.html
                            backend:
                               serviceName: dev-webserver
                               servicePort: 80

Creating simple-fanout ingress in k8s:

        Kubectl create –f simple-fanout.yaml –n <namespace_name>

Simple-fanout in k8s with contrail:

Image of k8s-ingress-simple-fanout

K8s-events:

    Kubectl connects to kube-api-server and creates the simps-fanout ingress in k8s.

Contrail-events:

Contrail-kube-manager:

Contrail-kube-manager register for event notification for interested resources with kube-api-server which includes ingress. When ingress object is created/modified/deleted in kube-api-server, It will send an event notification to contrail-kibe-manager. Contrail-kube-manager creates load balancer, virtual-machine-interface for load balancer and instance-ip for virtual-machine-interface from pod-ipam in cluster-network. If the external(public) pool available, it allocates external-ip and associate to the virtual-machine-interface. The fq_name of the external pool is [<public_network_project>,__public__,__fip_pool_public__]. Finally it updates the cluseter-ip and external-ip in k8s.

Contrail-svc-monitor:

Contrail-svc-monitor listens for event notifications for contrail load balancer objects. When contrail-svc-monitor gets the create load balancer event from contrail-api-server, it generates the haproxy configuration file with simple-fanout ingress spec rules, finds two active compute nodes for active-standby mode and sends to contrail-vrouter-agents as service-instances through control-node to launch haproxy with generated haproxy config

Contrail-vrouter-agent:

Contrail-vrouter-agent uses the opencontrail_vrouter_netns script to launch the haproxy inside the network namespace with generated haproxy configuration.

Viewing the ingress information:

k8s:
To View the http rules:

    kubectl describe ing/simple-fanout –n <namespace_name>

    Name: simple-fanout
    Namespace: default
    Address:  10.84.59.22,10.47.255.248
    Default backend: default-http-backend:80 (<none>)
    Rules:
                  Host      Path          Backends
               ------    ---------   ---------------
                 *
                             /qa.html      qa-webserver:80 (<none>)
                             /dev.html      dev-webserver:80 (<none>)
    Annotations:
    No events.

To View the ip information:

    kubectl get ing/simple-fanout -o wide
    NAME                HOSTS    ADDRESS                            PORTS    AGE
    simple-fanout    *           10.84.59.22,10.47.255.248  80            50s

Contrail:
Contrail-api:

http://<contrail-controller-node>:8082/loadbalancer/<ingress_uuid> | python -m json.tool

contrail-control:

http://<contrail-controller-node>:8083/Snh_IFMapTableShowReq?table_name=service_instance&search_string=

contrail-vrouter-agent:
ifmap_agent:

http://<conril-compute-node>:8085/Snh_ShowIFMapAgentReq?table_name=service_instance&node_sub_string=&link_type_sub_string=&link_node_sub_string=

opencontrail_vrouter_netns:

http://<contrail-compute-node>:8085/Snh_ServiceInstanceReq?uuid=

haproxy configuration:

Haproxy configuration is stored in /var/lib/contrail/loadbalancer/ haproxy/<ingress_uuid>/haproxy.conf

Haproxy configuration for simple-fanout:

global

daemon

user haproxy

group haproxy

log /dev/log local0

log /dev/log local1 notice

tune.ssl.default-dh-param 2048

ssl-default-bind-ciphers ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:ECDH+3DES:DH+3DES:RSA+AESGCM:RSA+AES:RSA+3DES:!aNULL:!MD5:!DSS

ulimit-n 200000

maxconn 65000

stats socket /var/lib/contrail/loadbalancer/haproxy/aa13aa8d-1faf-11e7-a423-00259030b0fe/haproxy.sock mode 0666 level user

defaults

log global

retries 3

option redispatch

timeout connect 5000

timeout client 300000

timeout server 300000

frontend e602faab-00e2-4ee2-b618-a420d8bb5128

option tcplog

bind 10.47.255.248:80

mode http

option forwardfor

acl 8dc82a38-d61b-4853-96f2-e82c0ff66801_path path /qa.html

use_backend 8dc82a38-d61b-4853-96f2-e82c0ff66801 if 8dc82a38-d61b-4853-96f2-e82c0ff66801_path

acl 56c180d1-9205-47d2-8eb7-4dba6ed5a6ae_path path /dev.html

use_backend 56c180d1-9205-47d2-8eb7-4dba6ed5a6ae if 56c180d1-9205-47d2-8eb7-4dba6ed5a6ae_path

backend 8dc82a38-d61b-4853-96f2-e82c0ff66801

mode http

balance roundrobin

option forwardfor

server c3ee3971-6207-4239-8112-615c1741a3d5 10.96.13.170:80 weight 1

backend 56c180d1-9205-47d2-8eb7-4dba6ed5a6ae

mode http

balance roundrobin

option forwardfor

server 671990da-51c5-46ea-b995-e3648f1bf2b7 10.98.199.61:80 weight 1

Specifying the external-ip in yaml:

An option is given to provide the external-ip in the yaml file. If the external-ip is provided, contrail-kube-manager uses the external-ip otherwise it allocates from the external (public) pool if it is available

Simple-fanout with external(public) ip:

        apiVersion: extensions/v1beta1
        kind: ingress
        metadata:
            name: simple-fanout
            annotations:
                externalIP: 10.84.59.22
        spec:
            rules:
            -   http:
                    paths:
                     -  path: /qa.html
                         backend:
                             serviceName: qa-webserver
                             servicePort: 80
                     -  path: /dev.html
                         backend:
                             serviceName: dev-webserver
                             servicePort: 80