Allow PEP 508 URL spec as editable

[uranusjr]

See #1289 (comment)

[sbidoul]

Looks good, thanks!.

[uranusjr]

Updated for review comments.

[sbidoul]

Do we have a natural place in the documentation for that ?

[uranusjr]

There’s a section in pip install on editable installs, I’ll update it to use the PEP 508 format. (Should we keep the #egg= format? I’m inclined not to because our next step is to deprecate it.)

[sbidoul]

Should we keep the #egg= format? I’m inclined not to because our next step is to deprecate it.

👍 to drop #egg= references from the doc now. That's a good way to check that we have an alternative for all documented use cases.

[sbidoul]

Hm, on second look... this will let non VCS URLs go through ? So we need to merge #9436 first and see how we can do the link.is_vcs here too.

[uranusjr]

Oh, good point, Requirement() does not check if the URL is actually valid.

[uranusjr]

One thing we ca do here is to add an extra check, and only treat the string as PEP 508 if the URL part contains ://. This means pseudo URL strings will automatically fall back to the old parsing logic, and can be handled independently to this new logic.

[uranusjr]

@sbidoul I added an additional check to prevent pseudo-URLs from going through here. Do you think it’d help?

[sbidoul]

@uranusjr I have not tested (I whish I had more time for pip...) but this version would accept any URLs ? Editables URLs must be file:// or vcs+...:// urls only, correct ? And even file URLs must point to a local directory, not a file.

[uranusjr]

Correct. There are already checks in a later stage to error out on invalid URL values, so I think it’s fine to pass them through here.

pip/src/pip/_internal/req/constructors.py

Lines 116 to 119 in baaf66f

	raise InstallationError(
	f'{editable_req} is not a valid editable requirement. '
	f'It should either be a path to a local project or a VCS URL '
	f'(beginning with {backends}).'

[uranusjr]

I did a mass find-and-replace on the documentation (and rewrote a couple of paragraphs) to ditch egg= for PEP 508. The documentation now contains no mention of egg= at all.

[BrownTruck]

Hello!

I am an automated bot and I have noticed that this pull request is not currently able to be merged. If you are able to either merge the master branch into this pull request or rebase this pull request against master then it will be eligible for code review and hopefully merging!

[sbidoul]

@uranusjr this does not seem to work, unfortunately:

$ pip install -e "pip-test-package @ git+https://github.com/pypa/pip-test-package" 
ERROR: Could not detect requirement name for 'git+https://github.com/pypa/pip-test-package', please specify one with #egg=your_package_name

Somehow parse_editable is being invoked twice: first with name @ url and then a second time with url only.

[uranusjr]

Thanks for catching this. A code path is building editables incorrectly with a bare URL instead of the whole requirement string. This “works” right now because the URL is the requirement string, but not anymore.

[uranusjr]

Hmm, turns out packaging actually contains some restriction on the URL and does not accept git+file:///.... This means it’s very difficult to write tests for pip install -e "NAME @ VCS+URL" without depending on network access. I’ve opened pypa/packaging#412 for this.

---

Update: Need to first resolve pypa/packaging#264.

[BrownTruck]

Hello!

I am an automated bot and I have noticed that this pull request is not currently able to be merged. If you are able to either merge the master branch into this pull request or rebase this pull request against master then it will be eligible for code review and hopefully merging!

[pradyunsg]

FWIW, I suggest filing a new PR for this, since our CI pipeline changed significantly; and the existing test results would likely contribute to "noise" in the checks view. :)

[uranusjr]

The needed upstream fix in packaging is also taking a long time, and there’s no point working on this before that’s done.

[sbidoul]

there’s no point working on this before that’s done.

Why ? Implementing pip install -e "pip-test-package @ git+https://github.com/pypa/pip-test-package" is not blocked by the packaging issue. I personally could live with a test that needs the network for this.

[uranusjr]

My wording was bad, it’s more like I don’t think it’s a good investment of my time right now 😓