Feat | add SLB network plugin

[chrisliu1995]

Background 背景说明

Cloud load balancer, as a very mature network cloud product, has been well known by developers and has been widely used. However, in game scenarios, due to the stateful nature of game servers, user traffic cannot be balanced to different game servers, which runs counter to the concept of Service in Kubernetes.

The Service matches the corresponding Pod, and balances the traffic carried by the LB to different pods. As shown in the figure below, the port corresponding to the Service is 80, and the targetPort is 80. Only one port is opened on the LB.

云负载均衡器(LoadBalancer)作为极为成熟的网络云产品已经被开发者熟知，得到了广泛地应用。然而在游戏场景下，由于游戏服有状态的特性，用户流量是无法均衡到不同的游戏服上的，这与Kubernetes中Service的概念背道而驰。

Service匹配上对应的Pod，将LB承载的流量均衡到不同的pod上，如下图所示，Service对应的port是80，targetPort是80，LB实际上只开放了一个端口。

In the game server scenario, a single LB should open different ports and forward the traffic to the corresponding Pod. As shown in the figure below, the traffic is forwarded from port 555 of LB to port 80 of pod0, from port 556 of LB to port 80 of pod1, and from port 557 of LB to port 80 of pod2. This way of using LB is what the game server needs.

而在游戏服场景下，单个LB应该开放不同端口，将流量转发到对应的Pod上，如下图所示，LB的555端口转发到pod0的80端口；LB的556端口转发到pod1的80端口；LB的557端口转发到pod2的80端口。这种LB的使用方式才是游戏服所需要的。

How to use

Using OKG [cloud provider & network plugin mechanism](#15) , used as follows:

Specify network configuration when deploying GameServerSet:

cat <<EOF | kubectl apply -f -
apiVersion: game.kruise.io/v1alpha1
kind: GameServerSet
metadata:
  name: gs-slb
  namespace: default
spec:
  replicas: 1
  updateStrategy:
    rollingUpdate:
      podUpdatePolicy: InPlaceIfPossible
  network:
    networkType: AlibabaCloud-SLB
    networkConf:
    - name: SlbIds
      #Fill in Alibaba Cloud LoadBalancer Id here
      value: "lb-xxxxxxxxxxxxxxxxx"
    - name: PortProtocols
      #Fill in the exposed ports and their corresponding protocols here. 
      #If there are multiple ports, the format is as follows: {port1}/{protocol1},{port2}/{protocol2}...
      #If the protocol is not filled in, the default is TCP
      value: "80"
    - name: Fixed
      #Fill in here whether a fixed IP is required [optional] ; Default is false
      value: "false"
  gameServerTemplate:
    spec:
      containers:
        - image: registry.cn-hangzhou.aliyuncs.com/gs-demo/gameserver:network
          name: gameserver
EOF

Check network status in GameServer:

networkStatus:
    createTime: "2022-11-24T01:27:30Z"
    currentNetworkState: Ready
    desiredNetworkState: Ready
    externalAddresses:
    - ip: 47.97.167.217
      ports:
      - name: "80"
        port: "611"
        protocol: TCP
    internalAddresses:
    - ip: 172.16.0.17
      ports:
      - name: "80"
        port: "80"
        protocol: TCP
    lastTransitionTime: "2022-11-24T01:27:30Z"
    networkType: Ali-SLB

Detailed

Design Overview 设计概述

ACK(Alibaba Cloud Container Service for Kubernetes) supports the mechanism of SLB multiplexing in k8s. Different SVCs can use different ports of the same SLB. According to this, the Ali-SLB network plugin will record the port assignments corresponding to each SLB. For game servers that specify the network type as Ali-SLB, the Ali-SLB network plugin will automatically allocate a port and create a service object, after the public network IP in the svc ingress field is successfully created, the GameServer network is in the Ready state, and the process is completed.

阿里云容器服务支持在k8s中对SLB复用的机制，不同的svc可以使用同一个SLB的不同端口。由此，Ali-SLB network plugin将记录各SLB对应的端口分配情况，对于指定了网络类型为Ali-SLB，Ali-SLB网络插件将会自动分配一个端口并创建一个service对象，待svc ingress字段的公网IP创建成功后，GameServer的网络处于Ready状态，该过程执行完成。

Fixed-IP 固定IP

When Fixed is specified as true in the network configuration of GameServerSet, the fixed IP function will take effect. Even if the Pod is deleted and rebuilt, the traffic path of the game server from SLB port to Pod port will not change.

When creating SVC, set the ownerReference of SVC according to Fixed. When Fixed is true, the owner of SVC is GameServerSet, and the SVC will be deleted only when GameServerSet is deleted; when Fixed is false, the owner of SVC is pod, and SVC will also be deleted when pod is deleted.

让GameServerSet的网络参数中指定了Fixed为true时，固定IP功能生效，即使Pod被删除重建，对于该游戏服的访问链路不会发生改变，外部IP、端口与内部IP、端口的映射关系维持固定。

在创建SVC时，根据Fixed设置SVC的ownerReference。当Fixed为true时，SVC的owner为GameServerSet，只有GameServerSet删除时该SVC才会被删除，与Pod的生命周期无关；当Fixed为false时，SVC的owner为Pod，当Pod被删除时SVC也将被删除。

Network Isolation 网络隔离

The Ali-SLB network plug-in provides network isolation. Even when the Pod is Ready, it can also remove the external network of the game server.

When the networkDisabeld field of GameServer.Spec is specified as true, the Ali-SLB network plugin will isolate the game server from the network, change the SVC network type from LoadBalancer to ClusterIP. This function is suitable for scenarios such as testing the game server after the game server updated when not reopened for players, cutting off traffic when the game server is abnorma.

Ali-SLB网络插件在SVC层面提供了网络隔离的能力，即使Pod为Ready时，也可以实现游戏服的对外网络的摘除。

在GameServer.Spec的networkDisabeld字段指定为true时，Ali-SLB网络插件将对该游戏服进行网络隔离，将对应SVC的网络类型由LoadBalancer改为ClusterIP，切断外部访问的流量。此功能适用于游戏服更新完成后测试游戏测试通过后再开服、以及游戏服异常时等切断流量等场景。

[ringtail]

networkType: Ali-SLB， Maybe alibabacloud-shared-slb would be better.

[ringtail]

alibabacloud-shared-slb should use one single slb as gateway. But for large scale game. slb pool would be better. So if this plugin only support one shared slb. alibabacloud-shared-slb would be a better name.

[chrisliu1995]

alibabacloud-shared-slb should use one single slb as gateway. But for large scale game. slb pool would be better. So if this plugin only support one shared slb. alibabacloud-shared-slb would be a better name.

User can pass different lb-id on different GameServerSet workloads, which means user can decide whether to use shared slb or not.

Plus, in the future, we will support that user can decide whether to use multi lb-id on one GameServerSet workload. So maybe alibabacloud-slb would be better?