Option index-url = for specific commands doesn't set index-url to PyPI

[RagBlufThim]

• Pip version: 18.1 (and others, see below)
• Python version: 3.7.2 (and others, see below)
• Operating system: Windows 7 (and others, see below)

In the [global] section of my pip.ini I set index-url to a non-standard value. In the sections for some commands ([download], [list], [search]) I use index-url = to reset the index-url for these commands to the standard value (PyPI). Since the last update of my python environment that doesn't work anymore:

• pip download tries to access the non-standard index-url given in the [global] section, not PyPI. The same is true for the version check of pip at the end of the command execution.
• pip list tries to access the non-standard index-url given in the [global] section, not PyPI, for the version check of pip at the end of the command execution.
• Strangely, pip search works as expected.

I can reproduce this new and unexpected behaviour with

• pip 18.1 - Python 3.7.2 - Windows 7
• pip 10.0.1 - Python 3.7.1 - Windows 10
• pip 18.1 - Python 3.6.4 - Windows 7

Everything used to work and still works as expected for

• pip 9.0.1 - Python 3.6.4 - Windows 7
• pip 9.0.3 - Python 2.7.15 - Windows 7

All five configurations mentioned use the same pip.ini.

I had a look at the release notes of pip 10 and a few of the issues mentioned there. I couldn't find anything that indicates that the change of behaviour is intended.
I also searched the issues for "index-url =", but none of the search results seems to be about the problem I encounter.

Here are some details / examples (pip 18.1, Python 3.7.2, Windows 7):

C:\>py -m pip config list
download.dest='E:\\zuInst.pip\\download'
download.index-url=''
download.no-index='false'
global.index-url='-'
global.log='E:\\zuInst.pip\\pip.log'
global.no-cache-dir='false'
global.no-index='true'
install.find-links='\nE:\\zuInst-erl\\PyPIP\nE:\\zuInst.pip\\wheel'
install.only-binary=':all:'
install.upgrade-strategy='only-if-needed'
list.find-links='\nE:\\zuInst-erl\\PyPIP'
list.format='columns'
list.index-url=''
list.no-index='false'
search.find-links='\nE:\\zuInst-erl\\PyPIP'
search.index-url=''
search.no-index='false'
wheel.find-links='\nE:\\zuInst.pip\\download'
wheel.wheel-dir='E:\\zuInst.pip\\wheel'

C:\>py -m pip download wheel
Looking in indexes: -
Collecting wheel
  Url '-/wheel/' is ignored. It is either a non-existing path or lacks a specific scheme.
  Could not find a version that satisfies the requirement wheel (from versions:)
No matching distribution found for wheel
Url '-/pip/' is ignored. It is either a non-existing path or lacks a specific scheme.

C:\>py -m pip list
Package    Version
---------- ----------
numpy      1.15.4+mkl
Pillow     5.4.1
pip        18.1
pywin32    224
scipy      1.2.0
setuptools 40.6.2
Url '-/pip/' is ignored. It is either a non-existing path or lacks a specific scheme.

Here in contrast a version that works (pip 9.0.3, Python 2.7.15, Windows 7, same pip.ini as above):

C:\>py -2 -m pip download wheel
Collecting wheel
  Downloading https://files.pythonhosted.org/packages/ff/47/1dfa4795e24fd6f93d5d58602dd716c3f101cfd5a77cd9acbe519b44a0a9/wheel-0.32.3-py2.py3-none-any.whl
  Saved e:\zuinst.pip\download\wheel-0.32.3-py2.py3-none-any.whl
Successfully downloaded wheel
You are using pip version 9.0.3, however version 18.1 is available.
You should consider upgrading via the 'python -m pip install --upgrade pip' command.

C:\>py -2 -m pip list
Package    Version
---------- -------
ipython    0.13
numpy      1.9.2
oletools   0.50
PIL        1.1.7
pip        9.0.3
py2exe     0.6.9
pyreadline 2.1
pywin32    224
setuptools 39.0.1
wheel      0.29.0
You are using pip version 9.0.3, however version 18.1 is available.
You should consider upgrading via the 'python -m pip install --upgrade pip' command.

[cjerdonek]

Is this related? #5528 It looks like it could be related to a change in how empty strings are processed.

[cjerdonek]

It looks like this behavior change is a result of this PR ("Add a configuration command"), which first appeared in version 18.1: #4240

You can see that the following lines were introduced in this PR:

for section_key, val in self.config.items():
    # ignore empty values
    if not val:
        logger.debug(
            "Ignoring configuration key '%s' as it's value is empty.",
            section_key
        )
        continue

Here is a link to the line in the PR itself: https://github.com/pypa/pip/pull/4240/files#diff-e3e54473df8cc86ee86078f3eba8a131R161

[pfmoore]

It looks like @cjerdonek's diagnosis is correct. However, I don't think that using an empty value to mean "reset to the default value" has ever been documented or supported behaviour. I'm not saying it's not something useful that maybe we should support, but I'd view it as a new feature, not a bug. (And maybe there's a better way to spell "reset to default" than a blank value).

[RagBlufThim]

Thank you for the explanations.

However, I don't think that using an empty value to mean "reset to the default value" has ever been documented or supported behaviour.

https://pip.pypa.io/en/stable/user_guide/#config-file says:
pip allows you to set all command line option defaults in a standard ini style config file.
...
The names of the settings are derived from the long command line option

https://pip.pypa.io/en/stable/reference/pip_download/ says about options:
-i, --index-url
Base URL of Python Package Index (default https://pypi.org/simple).

The latter made me think, when I created my pip.ini some years ago, that using --index-url without value sets PyPI as index-url, and that happened to work in the config file (with pip 9).
I just tried it on the command line, but there --index-url without value doesn't work.
So it seems to have been sheer luck that my pip.ini used to work in the first place.

I'm not saying it's not something useful that maybe we should support, but I'd view it as a new feature, not a bug.

How do I turn this bug report into a feature request?

In the meantime, is there any better idea than to change all lines index-url = to
index-url = https://pypi.org/simple in my pip.ini?

[pfmoore]

How do I turn this bug report into a feature request?

We don't really distinguish in the tracker, so it's fine as it is.

In the meantime, is there any better idea than to change all lines index-url = to index-url = https://pypi.org/simple in my pip.ini?

Not really, sorry. Although you could switch the logic round and leave [global] empty and put the setting in those places where you want it (presumably [install] and maybe [wheel] - are there others?) I'm not sure that's any easier though.

(I'm not entirely sure why you'd want to use different index-urls depending on the command, so it's possible there's another way to achieve what you want without hitting this question at all, but that's a separate point).

[rseuchter]

I just ran into this after using pip behind a corporate Artifactory acting as a proxy and was able to analyse it a bit better.

Interestingly, pip search uses --index to specify a custom index whereas pip install uses --index-url.

This in turn means that setting (replaced actual URL)

[global]
index = https://example.com/artifactory/pypi

works for pip search but not for pip install. Using index-url instead however works for pip install but not for pip search. The documentation indicates to use index-url and is thereby partially wrong.

A workaround is to simply specify both index and index-url in your config file:

[global]
index = https://example.com/artifactory/pypi
index-url = https://example.com/artifactory/pypi

Alas, here comes the fun part when you use the CLI (using an empty config):

/ # pip config list
/ # pip install --index https://example.com/artifactory/pypi flake8
Looking in indexes: https://example.com/artifactory/pypi
/ # pip install --index-url https://example.com/artifactory/pypi flake8
Looking in indexes: https://example.com/artifactory/pypi

install command eats both --index and --index-url! (search however, doesn't)

My pip version details:

/ # pip debug
WARNING: This command is only meant for debugging. Do not use this with automation for parsing and getting these details, since the output and options of this command may change without notice.
pip version: pip 20.0.2 from /usr/lib/python3.8/site-packages/pip (python 3.8)
sys.version: 3.8.2 (default, Feb 29 2020, 17:03:31)
[GCC 9.2.0]
sys.executable: /usr/bin/python3.8
sys.getdefaultencoding: utf-8
sys.getfilesystemencoding: utf-8
locale.getpreferredencoding: UTF-8
sys.platform: linux
sys.implementation:
  name: cpython
'cert' config value: global
REQUESTS_CA_BUNDLE: None
CURL_CA_BUNDLE: None
pip._vendor.certifi.where(): /usr/lib/python3.8/site-packages/pip/_vendor/certifi/cacert.pem
Compatible tags: 30
  cp38-cp38-linux_x86_64
  cp38-abi3-linux_x86_64
  cp38-none-linux_x86_64
  cp37-abi3-linux_x86_64
  cp36-abi3-linux_x86_64
  cp35-abi3-linux_x86_64
  cp34-abi3-linux_x86_64
  cp33-abi3-linux_x86_64
  cp32-abi3-linux_x86_64
  py38-none-linux_x86_64
  ...
  [First 10 tags shown. Pass --verbose to show all.]

[uranusjr]

IMO it’d be worthwhile to rename pip search --index to match other commands. @rseuchter would you mind opening a separate issue to discuss this?

[rseuchter]

Yeah, later when I find the time.

[xavfernandez]

IMO it’d be worthwhile to rename pip search --index to match other commands. @rseuchter would you mind opening a separate issue to discuss this?

cf #589 & #395 for context.

[rseuchter]

@xavfernandez Thank you for doing the research. I'll refrain from opening yet another bug as it would be an exact duplicate of #589.

However, I can't tell if that is the same as #395. I have a feeling that after about 9 years there's no chance of getting that fixed anytime soon?! Maybe the documentation should be clearer and indicate these problems.

Here's my suggestion:

• If somebody found the time and confirmed my workaround works then it could be included in the docs.
• Also, maybe @RagBlufThim can check if it helps with this problem.