add more description for whole multus.conf to clarify its requirements

[nordysu]

@s1061123 add:
This issue is to address that current README.md improvement so that reader don't misunderstand that minimum requirements for multus.conf, especially in https://github.com/intel/multus-cni#configuring-multus-to-use-the-kubeconfig
------------------------------ Original description --------------------
I'm using master branch to deploy multus. When i set the config file with:

{
    "name": "node-cni-network",
    "type": "multus",
    "kubeconfig": "/etc/kubernetes/node-kubeconfig.yaml"
}

I cannot start a pod with network annotation.

From logs, it said that at least one delegate must be specified.

Do I have to set a default delegate to use multus?

[s1061123]

Right. multus itself does not provide any network interface to container.
Pod needs network interface at least one interface and that is the default delegate.

We also plan to set default network as network attachment definition in future, but it is not yet, so
could you please add default delegate into your config?

[nordysu]

@s1061123, I got your point, that is: Currently we must set a default delegate, no matter pod have annotation of network or not.

What i am trying to do is use annotations to ensure every pod have one or more interface. Further more, I may need some pods have master network with flannel, and some other pods have master network with macvlan. Is that possible?

[s1061123]

@soyuutaku I am not clear that the difference between default delegate and master network in your comment. I feel master network and default delegate points same network and this master network is connected to kubernetes API network, too.

Currently multus provides to add one ore more 'extra networks' with one default delegates to the pod.
Kubernetes needs to access the pod and default delegates is used for that, so we cannot remove default delegates (= master network) and Kubernetes provides only one network for that, hence we cannot do that you mentioned (provide two or more master network in a cluster).

[rkamudhan]

some how related to #131 issue as well.. Good to see this conversation.

[nordysu]

@s1061123. My understanding is follow steps below, I can have two pods with only one network interface, and the interfaces are in different networks. Is that right?

1. let the config file as below, and than make sure every pod have annotations of network name by crd.

{
    "name": "node-cni-network",
    "type": "multus",
    "kubeconfig": "/etc/kubernetes/node-kubeconfig.yaml"
}

2. start up a pod as:

apiVersion: v1
kind: Pod
metadata:
  name: samplepod1
  annotations:
    k8s.v1.cni.cncf.io/networks: macvlan
spec:
  containers:
...

3. start up another pod as:

apiVersion: v1
kind: Pod
metadata:
  name: samplepod2
  annotations:
    k8s.v1.cni.cncf.io/networks: flannel
spec:
  containers:

...

[s1061123]

@soyuutaku Let me explain what the default delegate is.

Let's see following 'just-simple-pod' and imagine we run it in 'usual k8s, without multus'.

apiVersion: v1
kind: Pod
metadata:
  name: just-simple-pod
spec:
  containers:
...

At that time, the pod, just-simple-pod will have one interface, from '/etc/cni/net.d'. This interface is used to reach outside (i.e. internet) and also kubernetes will use it for managing. Yeah, kubernetes (i.e. kubelet/apiserver) requires the connection to the pod.

Multus does not change the concept. So we need a 'interface for the connection to kubernetes API' and that is the default delegate (or master network). So you need to add default delegates in '/etc/cni/net.d/multus.conf'. You can current cni plugin in /etc/cni/net.d.

[nordysu]

@s1061123 , I get your point. Kubernetes need pod have at least one interface so that kubernetes can manage it.

I wonder if kubernetes can manage pods on different network? Let say Pod A have flannel as default delegate, and Pod B have macvlan as default delegate.

If kubernetes need all pods have a unify interface to be default delegate, then this section of Readme file may misleading users by given a config file without delegate.

[s1061123]

@soyuutaku Multus also supports to load network config from json file in 'defaultConfDir', /etc/cni/multus/net.d. In this case, but your mention to document seems to be good suggestion to clarify configuration.

So can we change the issue subject to clarify it, as "add more description for whole multus.conf to clarify its requirements" or some stuff?

[nordysu]

@s1061123 Sounds good. We need some more description for multus.conf.

[s1061123]

I've changed the subject and add some small description in the top.
@rkamudhan , I suppose the section is from old multus document (guess v1.0 or v1.1) and it may be deleted for now.
Could you please double check about it?

[s1061123]

Close the issue because #191 update the document up-to-date.