Flux probably makes too many API server requests

[samb1729]

In cluster/kubernetes/kubernetes.go, we fetch a set of namespaces from the Kubernetes API server and then perform 4 requests per namespace. This means that the number of requests scales linearly with the number of namespaces in the cluster.

Given that kubectl get pod --all-namespaces performs a single request rather than one per namespace, I think we can do the same in Flux:

$ kubectl get pod --all-namespaces --v=6
I0405 20:05:44.121200   57585 loader.go:357] Config loaded from file /Users/sam/.kube/config
I0405 20:05:44.147190   57585 round_trippers.go:436] GET https://192.168.99.100:8443/api/v1/pods?limit=500 200 OK in 15 milliseconds
I0405 20:05:44.162286   57585 round_trippers.go:436] GET https://192.168.99.100:8443/swagger-2.0.0.pb-v1 304 Not Modified in 1 milliseconds
NAMESPACE     NAME                          READY     STATUS    RESTARTS   AGE
default       helloworld-5c59ff79fc-bpljt   2/2       Running   0          54m
default       nginx-7d58bc98fc-sxrsp        1/1       Running   1          16h
kube-system   kube-addon-manager-minikube   1/1       Running   5          4d
kube-system   kube-dns-54cccfbdf8-f6qj6     3/3       Running   15         4d
kube-system   storage-provisioner           1/1       Running   5          4d
test          helloworld-5c59ff79fc-z6xrx   2/2       Running   2          16h

[squaremo]

I don't think this is a very urgent problem for responsiveness. If the number of API requests is a problem (though it's quite conservatively rate limited), then a local cache might be the way to solve it -- see #471.

[kingdonb]

With Flux v1 in maintenance, scalability issues are definitely considered out of scope now.