Flannel should respond to k8s node delete events by cleaning up lease

[davidmccormick]

Expected Behavior

When a node a purposely removed/deleted from a k8s cluster, we want its network lease to be freed up immediately rather than having to wait for lease to expire. We have a number of large clusters with a reduced pod network/17 which means we are running out of leases when we roll/upgrade them. We feel that it is a waste to keep leases around for ephemeral nodes that are not going to return when we need faster lease recycling.

Current Behavior

Tested on kubernetes 1.8.4 with flannel 0.9.1. When I run kubectl delete node ABC then ABC is removed from k8s but checking etcd contents the lease remains behind.

Possible Solution

Does kubernetes pass a delete event through or could flannel watch for node delete events like an operator?

Steps to Reproduce (for bugs)

1. use kubectl and get nodes list.
2. use etcdctl (v2 api) to list the leases under /coreos.com/network/subnets/*.
3. use kubectl delete node ABC to delete one of the nodes.
4. use etcdctl to list the leases again.
5. check that lease for deleted node has been removed.

Context

Rolling out updates to large clusters or clusters with few leases left is problematic because nodes are terminated and replacements spun up but flannel runs of leases to give out and so the new nodes are unusable - whilst there are a number of leases blocked until their 24h expiration times out.

Your Environment

• Flannel version: 0.9.1
• Backend used vxlan
• Etcd version: v3.2.10
• Kubernetes version (if used): 1.8.4.coreos
• Operating System and version: CoreOS (beta branch)
• Link to your project (optional):

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.