Default network readiness enhancement

[dougbtv]

Uses the alternate network readiness methodology, section 6.1.1 of the de-facto standard.

In short:

If the implementation chooses this method, it must then block any pod network attachment/detachment operations until the cluster-wide default network is ready, or until a suitable timeout is reached. The recommended timeout value is 2 minutes.

This method waits in a loop for 2 minutes (by default) looking for a configuration file from the default network and is otherwise configurable (e.g. for which default network configuration file to look for an for the timeout length).

More details available in README.md changes.

[dougbtv]

@rkamudhan -- do you have a suggestion for a retry library to use?

I looked into using the one you had used for the annotation re-try, but, apparently it seems to be for conflicting writes, or at least that's what I found looking at this method: https://github.com/kubernetes/client-go/blob/master/util/retry/util.go#L45-L47

I have a rebase, but, I don't have a refactor here yet.

[dougbtv]

Note to self: spec updated to say "not a default 2 minute wait" -- we'll just have an exponential back-off, and keep on running.

[rkamudhan]

package type

import (
        ......
	"time"

	"k8s.io/apimachinery/pkg/util/wait"
        .......
)

.... bla bla code...

var DefaultReadinessBackoff = wait.Backoff{
      // We can change the values here
	Steps:    4, 
	Duration: 10 * time.Millisecond,
	Factor:   5.0,
	Jitter:   0.1,
}

.... bla bla code...


err := wait.ExponentialBackoff(backoff, func() (bool, error) {
		err := findmeDefaultNetwork(n.DefaultFilename)
		switch {
		case err == nil:
			return true, nil
		default:
			return false, err
		}
	})
	return err

func youFoundmeDefaultNetwork() {
}

.... bla bla code...

Refer: https://godoc.org/k8s.io/apimachinery/pkg/util/wait

[rkamudhan]

We can use https://godoc.org/k8s.io/apimachinery/pkg/util/wait backoff here

[dougbtv]

Sweet! Kural thank you for the suggestion, and the awesome code example -- really appreciated.

[dougbtv]

Thanks again on input for the wait library in apimachinery. Happy to take any input on the default backoff parameters, and also if any should be exposed in the configurations.

[rkamudhan]

yes yes. We can take backoff time from user as well.. But it is advance feature, we can do it later.. no issues..

[rkamudhan]

@dougbtv how the delegate information is transferred from the default file to RawDelegates ?

[rkamudhan]

If we have defaultNetworkFile, then the delegate is not a must right ?

[rkamudhan]

I think this PR is WIP

[dougbtv]

Idea is to set 250 millis, and factor of 4, for .25 seconds, 1 second, 4 second, 16 secons....

[dougbtv]

Latest commit pushed has decided on back off.

[dougbtv]

rename defaultnetworkfile to readinessindicatorfile

[s1061123]

Looks good to me, Doug! Thank you for your update!

[dougbtv]

Idea: We run this as a startup option -- that is, this happens to look for a file (or CRD object) before Multus runs, so you say run multus as ./multus --is-default-ready /target/file.name or ./multus --is-default-ready my-crd-name and it reports back saying that it's ready or not. So that we don't have the entrypoint script runs first before this happens, this might be better for configuration management, especially relative to an operator-style management of cni plugin configuration.

cc: @s1061123 @dcbw

[dougbtv]

@s1061123 -- give this a look, I have it rebased to fix up the conflicts! Thanks for letting me know about it.

[coveralls]

Pull Request Test Coverage Report for Build 416

• 8 of 10 (80.0%) changed or added relevant lines in 2 files are covered.
• No unchanged relevant lines lost coverage.
• Overall coverage increased (+0.4%) to 46.465%

---

Changes Missing Coverage	Covered Lines	Changed/Added Lines	%
multus/multus.go	5	7	71.43%

Totals
Change from base Build 415:	0.4%
Covered Lines:	414
Relevant Lines:	891

---

💛 - Coveralls

[s1061123]

@dougbtv , done! Thank you for your long-time work!