[feat] Add retries for HTTP operations

[james0209]

What

Use go-retryablehttp for retryable network operations.

Why

closes #140

[radeksimko]

Thanks for the PR.

Could we keep the HTTP client logic in that httpclient package and also retain the logic which adds the User-Agent header?

For example:

// NewHTTPClient provides a pre-configured http.Client
// e.g. with relevant User-Agent header
func NewHTTPClient() *http.Client {
	client := retryablehttp.NewClient().StandardClient()
	client.Transport = &userAgentRoundTripper{
		userAgent: fmt.Sprintf("hc-install/%s", version.Version()),
		inner:     client.Transport,
	}
	return client
}

[james0209]

Thanks for the PR.

Could we keep the HTTP client logic in that httpclient package and also retain the logic which adds the User-Agent header?

For example:

// NewHTTPClient provides a pre-configured http.Client
// e.g. with relevant User-Agent header
func NewHTTPClient() *http.Client {
	client := retryablehttp.NewClient().StandardClient()
	client.Transport = &userAgentRoundTripper{
		userAgent: fmt.Sprintf("hc-install/%s", version.Version()),
		inner:     client.Transport,
	}
	return client
}

@radeksimko I've got no problem putting it in the httpclient 🙂

Although from what I can tell, the UserAgent is not used currently. It is set on something called cli, but then only the client is returned and used. UserAgent is only used to be assigned to cli but cli is never returned or used anywhere - only client.

// NewHTTPClient provides a pre-configured http.Client
// e.g. with relevant User-Agent header
func NewHTTPClient() *http.Client {
	client := cleanhttp.DefaultClient()

	userAgent := fmt.Sprintf("hc-install/%s", version.Version())

	cli := cleanhttp.DefaultPooledClient()
	cli.Transport = &userAgentRoundTripper{
		userAgent: userAgent,
		inner:     cli.Transport,
	}

	return client
}

I could do something like you suggested above if you would like UserAgent to actually be utilized?

[radeksimko]

Although from what I can tell, the UserAgent is not used currently.

Yes, I noticed it isn't used - that was certainly not intentional - i.e. that's a bug that would be worth addressing also.

I could do something like you suggested above if you would like UserAgent to actually be utilized?

Yes please.

[radeksimko]

LGTM