Possible issue with "VS Tools for Docker" when there's a network proxy to access Docker Hub

[CESARDELATORRE]

Looks like VS Tools for Docker can have issues when there's a network proxy in between the dev machine and Docker Hub.
However, the Docker CLI, just by using "docker pull" looks like that works ok.
so the issue might be just related to VS Tools for Docker?

Original thread from:
#107 (comment)

sg1970 commented 22 hours ago • edited
This is just not proxy aware un-fortunately like most tech comming out these days making me wonder if Iam the only one that works behind a proxy !!! The project builds fine but wont run when I hit F5 as it runs into a proxy error. I get the error (see below) on the Docker Compose project.

I tried these things :

Set Proxy credentials on the docker settings
set http_proxy and https_proxy env variables.
And finally out of desperation tried this as well --> https://stackoverflow.com/a/12634725
But no joy so far!

I suspect I need to edit Dockerfile or docker-compose.yaml files to pass proxy creds in clear text ?

--- Error Details ---

The remote server returned an error: (407) Proxy Authentication Required

Severity Code Description Project File Line Suppression State Error MSB4018 The "EnsureVsDbgExists" task failed unexpectedly. System.Net.Http.HttpRequestException: An error occurred while sending the request. ---> System.Net.WebException: The remote server returned an error: (407) Proxy Authentication Required. at System.Net.HttpWebRequest.EndGetResponse(IAsyncResult asyncResult) at System.Net.Http.HttpClientHandler.GetResponseCallback(IAsyncResult ar) --- End of inner exception stack trace --- at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Microsoft.DotNet.Docker.BuildTasks.VsDbgHelper.<>c__DisplayClass3_0.<b__0>d.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Microsoft.DotNet.Docker.BuildTasks.VsDbgHelper.d__2.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Microsoft.DotNet.Docker.BuildTasks.DockerBaseTask.Execute() at Microsoft.Build.BackEnd.TaskExecutionHost.Microsoft.Build.BackEnd.ITaskExecutionHost.Execute() at Microsoft.Build.BackEnd.TaskBuilder.d__26.MoveNext() docker-compose C:\Program Files (x86)\Microsoft Visual Studio\2017\Professional\MSBuild\Microsoft\VisualStudio\v15.0\Docker\Microsoft.VisualStudio.Docker.Compose.targets 185
@CESARDELATORRE
Owner
CESARDELATORRE commented 5 hours ago
@sg1970 This is related to your Docker environment configuration since you need to use a proxy.
Basically, you need to have access to https://hub.docker.com/ in order to pull/download the base Docker images.
You can try to manually pull/download the base images you see on every dockerfile, or the infrastructure containers, like running the following Docker CLI commands from the command prompt (once Docker is installed):

docker pull microsoft/aspnetcore:1.1.2
docker pull microsoft/mssql-server-linux
docker pull mongo
docker pull redis
docker pull rabbitmq
Here's some info about it, from Docker:
Proxy configuration
If you are behind an HTTP proxy server, for example in corporate settings, before open a connect to registry, you may need to configure the Docker daemon’s proxy settings, using the HTTP_PROXY, HTTPS_PROXY, and NO_PROXY environment variables. To set these environment variables on a host using systemd, refer to the control and configure Docker with systemd for variables configuration.

Also, see this thread at stackoverflow
https://stackoverflow.com/questions/23111631/cannot-download-docker-images-behind-a-proxy

I cannot test it because we don't use any traditional proxy in MSFT Corpnet.
In any case, you are experiencing a Docker environment problem, so please, research in Docker's sites, as Docker CLI does support to work with proxies, ok?
Thank you.
@sg1970

sg1970 commented 24 minutes ago • edited
@CESARDELATORRE Thanks for the quick response. Greatly appreciate that !

I have access to docker hub (thru the same corporate proxy)
I can manually download the images from command prompt (see logs below)
So I was able to successfully pull all Docker images you listed
But still stuck with the same Proxy ... It just wont run thru Visual Studio !

I realize that it has nothing to do with your code (Works from home) but something to think about.

---Manual Docker Pull ---
C:>docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
d4w/nsenter latest 9e4f13a0901e 9 months ago 83.8 kB

C:>docker pull microsoft/aspnetcore:1.1.2
1.1.2: Pulling from microsoft/aspnetcore
9f0706ba7422: Pull complete
8c906b05891b: Pull complete
7378286116c1: Pull complete
6c5e797d4ef1: Pull complete
b660c1be1245: Pull complete
Digest: sha256:8dc2cad5c561aaf44fdc963f5cd8ac36e53b52dae595a50b4212930cca533b5a
Status: Downloaded newer image for microsoft/aspnetcore:1.1.2

C:>docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
microsoft/aspnetcore 1.1.2 7a07a406c6af About an hour ago 305 MB
d4w/nsenter latest 9e4f13a0901e 9 months ago 83.8 kB

C:>
@CESARDELATORRE
Owner
CESARDELATORRE commented 3 minutes ago
@sg1970 Interesting... If it is about Visual Studio and not really about Docker CLI, I'll ping the VS Docker Tools team guys. What is your email? Please, send me an email to CESARDL at Microsoft.com so I copy you into the thread I'll open with the VS Docker Tools team guys.

[dbreshears]

Our Docker Tools extension is downloading a script in order to install the debugger which at first glance appears to be the issue. We will need to look into how to address the proxy issue with this.

As a workaround, can you try downloading the file from https://aka.ms/getvsdbgps1 and place it in the %temp% folder. If we find that file sitting there we won’t attempt to download it again.

[sg1970]

@dbreshears thanks for the quick response.

1. Copied the .ps1 script to my %temp% folder
2. Unblocked it (right click unblock)
3. Closed VS and re-opened in elevated mode
4. Opened my Solution and hit F5 with Docker Compose Project set as the startup proj.

There is progress but unfortunately I have a new problem (missing a directory ) :
Starting: "docker" exec -i ac81399cf74c /clrdbg/vsdbg/vsdbg --interpreter=mi rpc error: code = 2 desc = oci runtime error: exec failed: container_linux.go:247: starting container process caused "exec: \"/clrdbg/vsdbg/vsdbg\": stat /clrdbg/vsdbg/vsdbg: no such file or directory" "docker" exited with code 126 (0x7E).

thats the error I get on VS output window. ( see attached snapshots ).

Please note that I had already manually downloaded all the Docker images needed by my solution.

[sg1970]

@dbreshears quick update: Iam able to reproduce this error on a simple Helloworld webapi project also (so that rules out any code issues).

[dbreshears]

@sg1970 . Which build of VS do you have from help about?

[dbreshears]

@sg1970. I see that others have hit this and mentioned that restarting Docker has fixed things. Can you try that? dotnet/dotnet-docker#162 . Also do you have any files under %userprofile%\clrdbg\vsdbg folder? If not perhaps the script in the %temp% folder is not able to download the files via the proxy issue?

[sg1970]

@dbreshears I tried re-starting Docker. It did not make any difference. However I notice that the %userprofile%\clrdbg\vsdbg folder did not exist. only the clrdbg folder exists and it was empty. I manually created the vsdbg folder and re-ran the app. It failed with same error msg but I see 2 files : success_rid.txt and success_version.txt.

My VS Version = 15.2(26430.13)
Docker version = 17.03 .1-ce-win12(12058)

Also the powershell file (GetClrDbg.ps1) listed in the thread you linked is different from the one you asked me to download (GetVsDbg.ps1 )

Thanks for all the help. Greatly appreciated.

[sg1970]

@dbreshears I figured out a work around based on the code in the GetVsDbg.ps1 powershell script. I realized that it was downloading files needed for vsdbg but the URL to download is constructed at runtime based on 2 params (RID and Version). So basically it comes down to stitiching that URL and downloading the .zip file and extracting it to %userprofile%\clrdbg\vsdbg.

Steps:

1. if the vsdbg folder is missing under %userprofile%\clrdbg\ create it
2. Hit F5 from VS2017 with docker compose set as startup proj.
3. You will now find 2 files under %userprofile%\clrdbg\vsdbg\ ( success_rid.txt and success_version.txt)
4. Construct the URL like so: https://vsdebugger.azureedge.net/vsdbg-VERSION/vsdbg-RID.zip

value of Version is from success_version.txt ( Replace dots(.) with comma(,) )
RID is from succes_rid.txt and dont forget the .zip at the end.

So in my case
Vesrion = 15.1.10518.1 which converts to 15-1-10518-1
RID = debian.8-x64.

So URL = https://vsdebugger.azureedge.net/vsdbg-15-1-10518-1/vsdbg-debian.8-x64.zip

Now manually download that zip and extract to %userprofile%\clrdbg\vsdbg

Then hit F5 and it should work.

HTH.

Thanks for your help!

[dbreshears]

@sg1970. Glad you were able to get this to work. How did you download the .zip file, through a browser? Also, the original error you received attempting to download the GetVSDbg.ps1 was 407 - Proxy authentication required, do you know if that was the same error you received when running that PS1 script. Wondering if you had tried to run it manually otherwise assume it was just hidden by the F5 of the Docker project

[dbreshears]

@sg1970. Would be interested to know if you were able to download https://aka.ms/getvsdbgps1 via Internet Explorer. It seems that WebClient would use the same proxy settings as IE if interpreting this doc correctly ->https://msdn.microsoft.com/en-us/library/system.net.webclient.proxy(v=vs.110).aspx

[sg1970]

@dbreshears Yes I downloaded it from my browser (Chrome) from behind the same proxy. But the URL you have in your last post is incorrect. you had it right on the post you made on Thursday. So the URL that works is: https://aka.ms/getvsdbgps1.

However I never got it to work thru Visual Studio 2017 so that is still an issue.

[misha-fm]

@dbreshears @sg1970 had the same issue in our office network behind the proxy. It turns out that WebClient doesn't take the environment proxy settings. The solution that solved that problem was to modify the script GetVSDbg.ps1 in %TMP% folder:
replace the line 57:
$zipStream = (New-Object System.Net.WebClient).OpenRead($url)
with these 3 lines:
$webClient = New-Object System.Net.WebClient
$webClient.Proxy.Credentials =[System.Net.CredentialCache]::DefaultNetworkCredentials
$zipStream = $webClient.OpenRead($url).

Then run F5 and it should work

If it doesn't, the problem can be the following: when you run debug before, the container was already created, so when you try to run debug (after changing the script), it downloads the folder, but the container isn't recreated, and doesn't have the mounted volume with clrdbg folder. So you need to remove the container (docker rm -f <container_id>) and run debug again.

[misha-fm]

But there still is a problem at some workstations (at my computer I can't reproduce the bug): sometimes (after reboot, or after restarting VS), the modified script get's replaced with the original one (which doesn't take credentials from Windows), so when the debug is run, the folder gets deleted (by the script), then the script cannot download the archive with clrdbg folder from internet (due to proxy issue) and the "F5-process" fails with the error described in the topic starter. So we need to modify the script at %TMP% once again, and then run F5 once again. It works fine until the modified script gets replaced with the original one, and so on...

[CESARDELATORRE]

Thanks for the workarounds. I'm pointing to this thread from the WiKi until this issue is solved from VS2017 Tools for Docker.

[mvelosop]

@CESARDELATORRE, I have no way to check this, do you know if it's been solved?

[mvelosop]

Closing the issue for now, it looks like the proxy and other networking issues have been solved with the recent release of Docker for Windows 18.03.1-ce.

[los93sol]

It would be nice if there was better error reporting than "An error occurred while sending the request". What request? To where? From what? What was the error?

[mvelosop]

Hi, @los93sol,

Did you get the logs? There's usually a lot of information there!

[los93sol]

Not sure where the logs would be but this issue is really annoying, it's intermittent and things will go from working fine to all of a sudden I can't run debug from VS. Usually a restart of VS fixes the issue, but after the latest update I have been having to restart the entire box.

[los93sol]

Still having this issue on VS19 Preview

[los93sol]

Did a Wireshark capture and I can see my proxy credentials are not being passed and I'm getting back a 407 from the proxy when VS is trying to access aka.ms Is anyone going to acknowledge this is a real issue and actually fix it instead of just closing the issue? Something is clearly not right with this as it's not picking up the proxy settings from the machine!

[mvelosop]

Hi @los93sol, I'm sorry I can't help you better here, but this is focused on eShopOnContainers issues and this seems to be a environment issue.

I did a quick search and found another repo focused on VS Tools for Docker, that might be a better place to ask, it looks like they use that repo only to keep track of issues.

https://github.com/Microsoft/DockerTools

Hope this helps.

[dbreshears]

@los93sol, we believe this is fixed in the 15.9.10 serviciing updated. I updated the issue with that comment in https://github.com/Microsoft/DockerTools