Add ali cloud VPC network support

Signed-off-by: yaoyao.xyy <[email protected]>

Documentation/alicloud-vpc-backend.md

@@ -0,0 +1,67 @@
+# AliCloud VPC Backend for Flannel
+
+When running in an AliCloud VPC, we recommend using the ali-vpc backend to build your network. This prevent from package encapsulation compare to overlay network and achieve maximum performance with IP routes. Also, there is no need for an separate flannel interface to be created.
+In order to run flannel on AliCloud we need first create an [AliCloud VPC Network](https://vpc.console.aliyun.com/#/vpc/cn-hangzhou/list)
+
+### Create VPC network
+Navigate to AliCloud VPC Network list page, then click [create vpc network] button.
+![vpc](img/ali-create-vpc.png)
+
+- Set vpc name with some meaningful string.
+- Choose a subnet for the VPC. There are three subnet for you to select which is 192.168.0.0/16 172.16.0.0/12 10.0.0.0/8. Choose one according to your cluster size.
+- Click create and wait for ready.
+
+### Create switch
+Click manager switch to navigate to switch list page, and create a switch.
+
+- Set switch name to a meaningful string.
+- Choose one AV Zone where you want to run your ECS
+- Set up a subnet which should be contained in your VPC subnet. Here we set subnet as 192.168.0.0/16.
+- Confirm Creating.
+
+### Create instance
+Create an instance whose network type is VPC and then add the instance to your previous VPC network. Note: The ECS you created must sit in the same AV zone with your previous created switch.
+![create instance](img/ali-create-instance.png)
+
+- Select the proper VPC network.
+
+### Get your own ACCESS_KEY_ID and ACCESS_KEY_SECRET. 
+Click [find key](https://ak-console.aliyun.com/#/accesskey)
+
+![create key](img/ali-create-key.png)
+
+- If you havent create any key yet, just click [create key secret] to create a new one.
+- take a note of AccessKeyId and AccessKeySecret for further use.
+
+### Go ahead and launch the instance! 
+
+All that’s left now is to start etcd, publish the network configuration and run the flannel daemon. 
+First, SSH into `instance-1`:
+
+- Start etcd:
+
+```
+$ etcd2 -advertise-client-urls http://$INTERNAL_IP:2379 -listen-client-urls http://0.0.0.0:2379
+```
+- Publish configuration in etcd (ensure that the network range does not overlap with the one configured for the VPC)
+
+```
+$ etcdctl set /coreos.com/network/config '{"Network":"10.24.0.0/16", "Backend": {"Type": "ali-vpc"}}'
+```
+- Fetch the latest release using wget from https://github.com/coreos/flannel/ 
+- make dist/flanneld
+- Run flannel daemon:
+
+```
+sudo ./flanneld --etcd-endpoints=http://127.0.0.1:2379
+```
+
+Next, create and connect to a clone of `instance-1`.
+Run flannel with the `--etcd-endpoints` flag set to the *internal* IP of the instance running etcd.
+
+Confirm that the subnet route table has entries for the lease acquired by each of the subnets.
+
+![router-confirm](img/ali-vpc-confirm.png)
+### Limitations
+
+Keep in mind that the AliCloud VPC [limits](https://vpc.console.aliyun.com/#/vpc/cn-hangzhou/detail/vpc-bp11xpfe5ev6wvhfb14b6/router) the number of entries per route table to 50. If you require more routes, request a quota increase or simply switch to the VXLAN backend.

Documentation/kube-flannel-aliyun.yml

@@ -0,0 +1,86 @@
+---
+kind: ConfigMap
+apiVersion: v1
+metadata:
+  name: kube-flannel-cfg
+  namespace: kube-system
+  labels:
+    tier: node
+    app: flannel
+data:
+  cni-conf.json: |
+    {
+      "name": "cbr0",
+      "type": "flannel",
+      "delegate": {
+        "isDefaultGateway": true
+      }
+    }
+  net-conf.json: |
+    {
+      "Network": "10.24.0.0/16",
+      "Backend": {
+        "Type": "ali-vpc"
+      }
+    }
+---
+apiVersion: extensions/v1beta1
+kind: DaemonSet
+metadata:
+  name: kube-flannel-ds
+  namespace: kube-system
+  labels:
+    tier: node
+    app: flannel
+spec:
+  template:
+    metadata:
+      labels:
+        tier: node
+        app: flannel
+    spec:
+      hostNetwork: true
+      nodeSelector:
+        beta.kubernetes.io/arch: amd64
+      containers:
+      - name: kube-flannel
+        image: registry.cn-hangzhou.aliyuncs.com/google-containers/flannel-git:v0.6.2-ubuntu
+        command: [ "/opt/bin/flanneld", "--ip-masq", "--kube-subnet-mgr" ]
+        securityContext:
+          privileged: true
+        env:
+        - name: POD_NAME
+          valueFrom:
+            fieldRef:
+              fieldPath: metadata.name
+        - name: POD_NAMESPACE
+          valueFrom:
+            fieldRef:
+              fieldPath: metadata.namespace
+        - name: ACCESS_KEY_ID
+          value: [replace with your own key]
+        - name: ACCESS_KEY_SECRET
+          value: [replaces with your own secret]
+        volumeMounts:
+        - name: run
+          mountPath: /run
+        - name: flannel-cfg
+          mountPath: /etc/kube-flannel/
+      - name: install-cni
+        image: registry.cn-hangzhou.aliyuncs.com/google-containers/flannel-git:v0.6.2-ubuntu
+        command: [ "/bin/sh", "-c", "set -e -x; cp -f /etc/kube-flannel/cni-conf.json /etc/cni/net.d/10-flannel.conf; while true; do sleep 3600; done" ]
+        volumeMounts:
+        - name: cni
+          mountPath: /etc/cni/net.d
+        - name: flannel-cfg
+          mountPath: /etc/kube-flannel/
+      volumes:
+        - name: run
+          hostPath:
+            path: /run
+        - name: cni
+          hostPath:
+            path: /etc/cni/net.d
+        - name: flannel-cfg
+          configMap:
+            name: kube-flannel-cfg

README.md

@@ -95,6 +95,15 @@ This is the only mandatory key.
 * alloc: only perform subnet allocation (no forwarding of data packets).
   * `Type` (string): `alloc`
 
+* ali-vpc: create IP routes in a [alicloud VPC route table](https://vpc.console.aliyun.com)
+  * Requirements:
+    * Running on an ECS instance that is in an Alicloud VPC.
+    * Permission require accessid and keysecret 
+  * `Type` (string): `ali-vpc`
+  * `AccessKeyID` (string): api access key id. can also be configure with environment ACCESS_KEY_ID
+  * `AccessKeySecret` (string): api access key secret.can also be configure with environment ACCESS_KEY_SECRET
+  Note: Currently, AliVPC limit the number of entries per route table to 50.  
+
 ### Example configuration JSON
 
 The following configuration illustrates the use of most options with `udp` backend.

backend/alivpc/alivpc.go

@@ -0,0 +1,198 @@
+// Copyright 2015 flannel authors
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package alivpc
+
+import (
+	"encoding/json"
+	"fmt"
+	"os"
+
+	log "github.com/golang/glog"
+	"golang.org/x/net/context"
+
+	"github.com/coreos/flannel/backend"
+	"github.com/coreos/flannel/pkg/ip"
+	"github.com/coreos/flannel/subnet"
+	"github.com/denverdino/aliyungo/common"
+	"github.com/denverdino/aliyungo/ecs"
+	"github.com/denverdino/aliyungo/metadata"
+)
+
+func init() {
+	backend.Register("ali-vpc", New)
+}
+
+type AliVpcBackend struct {
+	sm       subnet.Manager
+	extIface *backend.ExternalInterface
+}
+
+func New(sm subnet.Manager, extIface *backend.ExternalInterface) (backend.Backend, error) {
+	be := AliVpcBackend{
+		sm:       sm,
+		extIface: extIface,
+	}
+	return &be, nil
+}
+
+func (be *AliVpcBackend) Run(ctx context.Context) {
+	<-ctx.Done()
+}
+
+func (be *AliVpcBackend) RegisterNetwork(ctx context.Context, network string, config *subnet.Config) (backend.Network, error) {
+	// 1. Parse our configuration
+	cfg := struct {
+		AccessKeyID     string
+		AccessKeySecret string
+	}{}
+
+	if len(config.Backend) > 0 {
+		if err := json.Unmarshal(config.Backend, &cfg); err != nil {
+			return nil, fmt.Errorf("error decoding VPC backend config: %v", err)
+		}
+	}
+	log.Infof("Unmarshal Configure : %v\n", cfg)
+
+	// 2. Acquire the lease form subnet manager
+	attrs := subnet.LeaseAttrs{
+		PublicIP: ip.FromIP(be.extIface.ExtAddr),
+	}
+
+	l, err := be.sm.AcquireLease(ctx, network, &attrs)
+	switch err {
+	case nil:
+
+	case context.Canceled, context.DeadlineExceeded:
+		return nil, err
+
+	default:
+		return nil, fmt.Errorf("failed to acquire lease: %v", err)
+	}
+	if cfg.AccessKeyID == "" || cfg.AccessKeySecret == "" {
+		cfg.AccessKeyID = os.Getenv("ACCESS_KEY_ID")
+		cfg.AccessKeySecret = os.Getenv("ACCESS_KEY_SECRET")
+
+		if cfg.AccessKeyID == "" || cfg.AccessKeySecret == "" {
+			return nil, fmt.Errorf("ACCESS_KEY_ID and ACCESS_KEY_SECRET must be provided! ")
+		}
+	}
+
+	meta := metadata.NewMetaData(nil)
+	REGION, err := meta.Region()
+	if err != nil {
+		return nil, err
+	}
+	instanceid, err := meta.InstanceID()
+	if err != nil {
+		return nil, err
+	}
+	VpcID, err := meta.VpcID()
+	if err != nil {
+		return nil, err
+	}
+
+	c := ecs.NewClient(cfg.AccessKeyID, cfg.AccessKeySecret)
+
+	vpc, _, err := c.DescribeVpcs(&ecs.DescribeVpcsArgs{
+		RegionId: common.Region(REGION),
+		VpcId:    VpcID,
+	})
+	if err != nil || len(vpc) <= 0 {
+		log.Errorf("Error DescribeVpcs: %s . \n", getErrorString(err))
+		return nil, err
+	}
+
+	vroute, _, err := c.DescribeVRouters(&ecs.DescribeVRoutersArgs{
+		VRouterId: vpc[0].VRouterId,
+		RegionId:  common.Region(REGION),
+	})
+	if err != nil || len(vroute) <= 0 {
+		log.Errorf("Error DescribeVRouters: %s .\n", getErrorString(err))
+		return nil, err
+	}
+	vRouterId := vroute[0].VRouterId
+	rTableId := vroute[0].RouteTableIds.RouteTableId[0]
+
+	rtables, _, err := c.DescribeRouteTables(&ecs.DescribeRouteTablesArgs{
+		VRouterId:    vRouterId,
+		RouteTableId: rTableId,
+	})
+	if err != nil || len(rtables) <= 0 {
+		log.Errorf("Error DescribeRouteTables: %s.\n", err.Error())
+		return nil, err
+	}
+
+	route := &ecs.CreateRouteEntryArgs{
+		DestinationCidrBlock: l.Subnet.String(),
+		NextHopType:          ecs.NextHopIntance,
+		NextHopId:            instanceid,
+		ClientToken:          "",
+		RouteTableId:         rTableId,
+	}
+	if err := be.recreateRoute(c, rtables[0], route); err != nil {
+		return nil, err
+	}
+
+	if err := c.WaitForAllRouteEntriesAvailable(vRouterId, rTableId, 60); err != nil {
+		return nil, err
+	}
+	return &backend.SimpleNetwork{
+		SubnetLease: l,
+		ExtIface:    be.extIface,
+	}, nil
+}
+
+func (be *AliVpcBackend) recreateRoute(c *ecs.Client, table ecs.RouteTableSetType, route *ecs.CreateRouteEntryArgs) error {
+	exist := false
+	for _, e := range table.RouteEntrys.RouteEntry {
+		if e.RouteTableId == route.RouteTableId &&
+			e.Type == ecs.RouteTableCustom &&
+			e.InstanceId == route.NextHopId {
+
+			if e.DestinationCidrBlock == route.DestinationCidrBlock &&
+				e.Status == ecs.RouteEntryStatusAvailable {
+				exist = true
+				log.Infof("Keep target entry: rtableid=%s, CIDR=%s, NextHop=%s \n", e.RouteTableId, e.DestinationCidrBlock, e.InstanceId)
+				continue
+			}
+			// Fix: here we delete all the route which targeted to us(instance) except the specified route.
+			// That means only one CIDR was allowed to target to the instance. Think if We need to change this
+			// to adapt to multi CIDR and deal with unavailable route entry.
+			if err := c.DeleteRouteEntry(&ecs.DeleteRouteEntryArgs{
+				RouteTableId:         route.RouteTableId,
+				DestinationCidrBlock: e.DestinationCidrBlock,
+				NextHopId:            route.NextHopId,
+			}); err != nil {
+				return err
+			}
+
+			log.Infof("Remove old route entry: rtableid=%s, CIDR=%s, NextHop=%s \n", e.RouteTableId, e.DestinationCidrBlock, e.InstanceId)
+			continue
+		}
+
+		log.Infof("Keep route entry: rtableid=%s, CIDR=%s, NextHop=%s \n", e.RouteTableId, e.DestinationCidrBlock, e.InstanceId)
+	}
+	if !exist {
+		return c.CreateRouteEntry(route)
+	}
+	return nil
+}
+
+func getErrorString(e error) string {
+	if e == nil {
+		return ""
+	}
+	return e.Error()
+}

main.go

@@ -40,6 +40,7 @@ import (
 	_ "github.com/coreos/flannel/backend/hostgw"
 	_ "github.com/coreos/flannel/backend/udp"
 	_ "github.com/coreos/flannel/backend/vxlan"
+	_ "github.com/coreos/flannel/backend/alivpc"
 )
 
 type CmdLineOpts struct {