Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How to run pip from sources (fixes #5747) #5752

Closed
wants to merge 6 commits into from
Closed

How to run pip from sources (fixes #5747) #5752

Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
240f9dc
How to run pip from sources (fixes #5747)
techtonik Sep 3, 2018
1f68c1f
Pagure link is not used even if Sphinx doesn't complain
techtonik Sep 3, 2018
d16d77d
Create 5747.feature
techtonik Sep 3, 2018
7d4f7e3
Rename 5747.feature to 5747.doc
techtonik Sep 3, 2018
1e2a5c4
Cloning code is too obvious, captn
techtonik Sep 3, 2018
80e7374
Point to tests doc, because running pip is insufficient for them
techtonik Sep 3, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
56 changes: 55 additions & 1 deletion docs/html/development/getting-started.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,61 @@ This document is meant to get you setup to work on pip and to act as a guide and
reference to the the development setup. If you face any issues during this
process, please `open an issue`_ about it on the issue tracker.

Development tools
Running Developent Version
==========================
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You don't say why anyone would want to do this (apart from running the tests, which as you say this approach doesn't work for...)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I skipped the intro, because I feel like this part should be in TL;DR section. People who don't needed it can easily skip this section. In fact, I think the auditory of development docs are only two groups of people - newcomers (setup), and occasional contributors who forget things (me).

My goal is to have a quick reference for rebasing and manual testing that my patches still work. For that I need only a single python src/pip line. But I would like to help people coming for an easy or good-first-issue from funnels (marketing) like OpenHatch and upcoming HacktoberFest to have more fun with pip than virtualenv. Just grab someone experienced from Ruby or Node.js background and ask them to run pip from sources. These are the people that I have in mind.


Clone git repository and change dir into it:

.. code-block:: console

$ git clone https://github.com/pypa/pip
$ cd pip

Then run `python src/pip`:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

there needs to be a big fat warning here on why this works and that this absolutely doenst work for running the testsuite that ensure pip works ^^

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The explanation why it works is below. And there is no visible way to run tests without tox, so people have to read about tox below anyway.


.. code-block:: console

$ python src/pip

That's it!

You may want to skip to next chapter, or read on for full details.

Running `src/pip` will load all `pip` modules and
`dependencies <https://github.com/pypa/pip/tree/master/src/pip/_vendor)>`_
from the clone directory. This way they do not need to be installed on your
system. The process of bundling dependencies with a project sources is called
`vendoring`, but because there is no PEP for that, other Python won't work
this way and still require `pip` for installation.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This makes no sense. Why would there be a PEP for vendoring? What do you mean by "other Python won't work this way"?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I said that to warn people that vendoring in pip is probably a hack, and not something supported by language like vendor directories in Go or something approved by community as a general practice. I do this, because Python community is lawful good when it comes to proven things (e.g. nobody do monkey-patching, even if it is possible) and usually devise a PEP to mark something as generally worthy to let people know and allow tools to build around that.

If there is a way for other project to adopt pip-style vendoring for their own project, I would be happy to replace that paragraph with a link to the tutorial, because it is a valuable feature. For me it was a major selling point to contribute to Spyder, SCons and quite a few other projects.


If you want to practice `virtualenv` development, then `pip` inside can be
relinked directly to development sources by installing it in
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is very confusingly worded. If you're just saying "this is how you use virtualenv for development" then fine, but it doesn't read like that. And anyway, using virtualenv and editable installs for development isn't a workflow that's specifically for pip, so while this section might be helpful for the odd person who doesn't know how to do it, it's not really needed in general.

Also, this is much more reliable than your proposed "just run pip" approach, so I'd rather see this as the default approach, with your suggestion only recommended for people who (for whatever reason - I can't honestly think of any reasonable ones) don't want to use a virtualenv.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't like the wording too, but I want to explain the how --editable option works. The way it is explained by pip is by vague reference to setuptools documentation:

-e, --editable <path/url>   Install a project in editable mode (i.e. setuptools "develop mode") from a local
                          project path or a VCS url.

Which look like undocumented - you won't find find this mode in setuptools docs, which refer to easy_install docs, where no "develop mode" either - https://setuptools.readthedocs.io/en/latest/search.html?q=%22develop+mode%22 The problem is that you're not only exposing people to virtualenv, but also to decades of Python packaging problem with keyword like setuptools and easy_install that pip and wheels are trying to solve. It will take days to learn and it is not for Millenials who need immediate gratification to keep them on track (speaking of myself). I mean you guys is the only reason I get back to this. )

Last time I tried to use virtualenv I ended up setting up virtualenvwrapper, and I tell you - there is some big reason why so many virtualenv forks/alternatives exists. So before I edit this text, maybe I can hear arguments why would anyone recommend virtualenv for pip development? The question is two fold:

  1. Why recommend a tool like virtualenv at all if pip just works out of the box?
  2. Why recommend old virtualenv and not more modern virtualenvwrapper or alternatives?

When you say this is much more reliable - it doesn't sound like an argument to me. Why it is much more reliable? Let's make technical doc technical. It is development doc asking for the whole gory details after all. My argument that you don't need to be exposed to all this complexity, because there is no way you can mess with your system dependencies while developing pip. You may use virtualenv for test run pip install, but that's another story - "How to create virtualenv with custom pip?" where you probably want to keep separate envs for every pip modification to be able to compare state and debug later.

Sorry, I am in a rush again. Can't smile. :)

`development mode`. `pip install -e .` does that. This needs to be done only
once.

Full setup for `virtualenv` in `devpip` subdir:

.. code-block:: console

$ virtualenv .devpip
$ source .devpip/bin/activate
# ^^^ Linux only, on Windows run `.devpip/Scripts/activate.bat`
(.devpip)$ pip install -e .

To avoid activating `virtualenv` every time to test `pip`, you can use full path
to the executable to run development version:

.. code-block:: console

$ .devpip/bin/pip --version
pip 18.1.dev0 from /home/techtonik/p/pip/src/pip (python 2.7)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor point: It would be better to show examples using Python 3.x. Python 2 is not something we should be encouraging people to use.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep. Will do. But I will have to use python3 name, because python on Ubuntu 18.04 still points to 2nd version.


$ pip --version
pip 9.0.1 from /usr/lib/python2.7/dist-packages (python 2.7)
$ python src/pip --version
pip 18.1.dev0 from /home/techtonik/p/pip/src/pip (python 2.7)

Development Tools
=================

pip uses :pypi:`tox` for testing against multiple different Python environments
Expand Down
1 change: 1 addition & 0 deletions news/5747.doc
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Describe how to run `pip` from source, and how it vendors dependencies.