Image digest HEAD checking does not respect proxy settings

[pneuschwander]

Describe the bug

We operate a VM that can only access the internet through our corporate proxy. Therefore we configure http_proxy/HTTP_PROXY/https_proxy/HTTPS_PROXY/no_proxy/NO_PROXY accordingly. But it seems that Watchtower does not consider that environmental variable / the proxy configuration. As a result it fails when doing the head request and falls back to regular pull.

level=warning msg="Could not do a head request, falling back to regular pull."

There is already a ticket about it but it has been closed by the author as he no longer uses that setup. see #394
We are currently using that setup and try to make it work.
To Reproduce

0. Be in an environment that allows internet access only through a corporate proxy.
1. Spin up Watchtower with appropriate http_proxy ... env variables, set interval to a short time
2. Wait until first execution happened
3. Check logs

Expected behavior

Watchtower uses head request to check for new images instead of falling back to regular pull

Screenshots

Environment

Docker 20.10.6 on Debian Buster with amd64

Logs from running watchtower with the --debug option

time="2021-04-22T14:27:50Z" level=debug msg="Checking containers for updated images",
time="2021-04-22T14:27:50Z" level=debug msg="Retrieving running containers",
time="2021-04-22T14:27:50Z" level=debug msg="Trying to load authentication credentials." container=/infrastructure_watchtower_1 image="containrrr/watchtower:latest",
time="2021-04-22T14:27:50Z" level=debug msg="No credentials for containrrr found" config_file=/config.json,
time="2021-04-22T14:27:50Z" level=debug msg="Got image name: containrrr/watchtower:latest",
time="2021-04-22T14:27:50Z" level=debug msg="Checking if pull is needed" container=/infrastructure_watchtower_1 image="containrrr/watchtower:latest",
time="2021-04-22T14:27:50Z" level=debug msg="Building challenge URL" URL="https://index.docker.io/v2/",
time="2021-04-22T14:27:50Z" level=debug msg="Got response to challenge request" header="Bearer realm=\"https://auth.docker.io/token\",service=\"registry.docker.io\"" status="401 Unauthorized",
time="2021-04-22T14:27:50Z" level=debug msg="Checking challenge header content" realm="https://auth.docker.io/token" service=registry.docker.io,
time="2021-04-22T14:27:50Z" level=debug msg="Setting scope for auth token" image=containrrr/watchtower scope="repository:containrrr/watchtower:pull",
time="2021-04-22T14:27:50Z" level=debug msg="No credentials found.",
time="2021-04-22T14:27:51Z" level=debug msg="Parsing image ref" host=index.docker.io image=containrrr/watchtower normalized="docker.io/containrrr/watchtower:latest" tag=latest,
time="2021-04-22T14:27:51Z" level=debug msg="Doing a HEAD request to fetch a digest" url="https://index.docker.io/v2/containrrr/watchtower/manifests/latest",
time="2021-04-22T14:27:51Z" level=warning msg="Could not do a head request, falling back to regular pull." container=/infrastructure_watchtower_1 image="containrrr/watchtower:latest",
time="2021-04-22T14:27:51Z" level=debug msg="Reason: Head \"https://index.docker.io/v2/containrrr/watchtower/manifests/latest\": dial tcp 34.195.201.174:443: connect: connection refused",
time="2021-04-22T14:27:51Z" level=debug msg="Pulling image" container=/infrastructure_watchtower_1 image="containrrr/watchtower:latest",
time="2021-04-22T14:27:52Z" level=debug msg="No new images found for /infrastructure_watchtower_1"

Additional context

[github-actions]

Hi there! 👋🏼 As you're new to this repo, we'd like to suggest that you read our code of conduct as well as our contribution guidelines. Thanks a bunch for opening your first issue! 🙏

[simskij]

Interesting! I really thought we took this into consideration. Thank you for the report!

[piksel]

Yep, found the issue.

pkg/registry/digest.GetDigest instantiates it's own http.Transport instead of using the default one. This causes things like proxy auto-detection to not work correctly.
It seems like the reason for doing so is to bypass TLS checking, which I don't see any reason to do anyway. It also has some other problems that might be the cause of some of the errors we've been seeing...

[ghost]

To avoid important communication to get lost in a closed issues no one monitors, I'll go ahead and lock this issue. If you want to continue the discussion, please open a new issue. Thank you! 🙏🏼