-
Notifications
You must be signed in to change notification settings - Fork 3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Support for web proxies is broken in pip 20.3 #9190
Comments
Thanks for filing this! Could you please provide the output of the pip list -o and pip install XYZ commands on pip 20.3 with -v, and post the output contrasting that with pip 20.2.* with -v? |
Here is the requested output. I have removed more than 1000 uninteresting lines.
|
Many colleagues had the issue, but I did not on my own workstation, and we still don't understand why. I had a hunch it had something to do with window's SSL library as I have a Windows 1903 and the colleagues have 1803. But it may not be related after all? |
Having exactly this problem since 20.3 upgrade. I analyze log and pip never tries to reach the defined proxy server port at all |
I can confirm the issue, proxy environment variable
|
Since those configs come from requests (or urllib3?), this may be related to the vendor upgrade. |
One way to verify this would be to run pip 20.3 uses the following versions: pip/src/pip/_vendor/vendor.txt Lines 14 to 18 in 30eeb9c
pip 20.2.4: pip/src/pip/_vendor/vendor.txt Lines 14 to 18 in 57dd580
|
It seems like this is indeed a urllib3 change (urllib3/urllib3#2075). Maybe we should downgrade urllib3 to |
People affected by this issue: Can you provide the URL you’re using as your proxy for us to validate whether this is indeed causing the issue? |
I encountered this issue in our setup and rolling back to use 20.3b1 works fine. Below is the issue with 20.3:
Our proxy is |
I got the above error with —proxy set to our “https” proxy URL. Changed it to “http” and it works fine. |
The new flow for Removing these 2 lines makes everything work again:
I have no idea what's the purpose of this, and I couldn't find any explanation in the code or the commit message.
EDIT: I now think that the code is fine, even if a bit unclear. Changing the proxy from "https" to "http" is the correct solution. |
same here. pip version 20.3.1
|
Hi, we have identified this is caused by a change in the underlying urllib3 implementation. urllib3 maintainers have commented on #8785, which is the caused by the same change, so I will make that issue the canonical thread for this. |
Oops, thanks for the correction! |
The fix suggested by @tomerv solved my problem. The two lines are in question are lines 961 and 962 of connectionpool.py. You should apply this fix to all instances of connectionpool.py that contain these two lines. One instance will be in the pip package and another instance will be in urllib3 package if you have it installed. |
The bug still persists at version 21.0.1 when using pip behind a proxy. It's quite annoying to downgrade to pip 20.2.3 every time I need to create a new venv. The solution given by @tomerv works!! I also add
just after line 962 at This issue really should be addressed for the next version of pip. |
Sorry for the tone, but you started this first. Maybe you should contribute something for the next version of pip, if you know what needs changing and how? |
I actually don't think any longer that there's a bug here. As I commented in the referenced issue (#9216 (comment)) the error is on the user's side, using "https://" when "http://" should be used. Therefore, the fix I suggested before would be incorrect. I do think that pip should provide a better error message in this case, if possible. Sadly, I don't have the expertise to do that. |
Thank you @tomerv for the reply. I understand your point, and I agree that changing from "https://" to "http://" should be the best option, therefore it seems not to be a working one. I have run the following command:
and that's the error:
The problem is caused because The error is clear. The problem is the SSL certificate since it understands I'm using HTTPS. It seems that the parameter |
Okay, I would like to make some clarifications. The problem seems to be related to how pip is getting the proxy. Passing a proxy parameter to pip installation has no effect, instead, pip gets it from the system (Windows in my case). Doing To deploy the application to the server (Centos) also doesn't work. I use Ansible for that and I pass the proxy to it. But somehow pip fails to get the proxy from the --proxy parameter. So, I would say the error lies there. As a user, I would expect pip to use whatever address I pass to the proxy parameter, and not get the proxy from the system var. |
@tomerv I have found the problem. It's actually within pip at I'm using pip 21.0.1 to debug. The problem is caused because the proxy set by --proxy in the pip method is not being passed corrected to the request of urllib package. This is the original request method of PipSession: def request(self, method, url, *args, **kwargs):
# Allow setting a default timeout on a session
kwargs.setdefault("timeout", self.timeout)
# Dispatch the actual request
return super().request(method, url, *args, **kwargs) If you make a breakpoint there you gonna see that self.proxies is corrected setted from --proxies, however, when the call to the parent class is made the proxies get lost and urllib set it from the system, which then gets the wrong scheme (https instead of http). An easy fix I suggest is to add the following line at the request method in the file def request(self, method, url, *args, **kwargs):
# Allow setting a default timeout on a session
kwargs.setdefault("timeout", self.timeout)
kwargs.setdefault("proxies", self.proxies) # fix problem with proxies.
# Dispatch the actual request
return super().request(method, url, *args, **kwargs) This way the proxies are carried out correctly to the request, if no proxy is set an empty dict is passed. So now if you run I hope this will help to remove this bug in next versions of pip asap. |
Environment
I am behind a corporate firewall with an authenticating proxy that requires both a user ID and a password.
Description
pip 20.2.4 supported the proxy but pip 20.3 does not. I tried both setting the environment variables and
using the --proxy command line argument, but neither worked in pip 20.3. pip 20.2.4 works fine even if
no --proxy argument is used and no proxy-related environment variables are set.
Expected behavior
pip 20.3 should support the proxy in the same way that pip 20.2.4 does.
How to Reproduce
See the output below. I set up a virtual environment, and updated pip to version 20.2.4, demonstrated
that there was no bug, then updated pip to version 20.3 and demonstrated the bug
Output
The text was updated successfully, but these errors were encountered: