Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

The --target option clashes with other command line flags and config files #4390

Closed
dstufft opened this issue Mar 30, 2017 · 28 comments · Fixed by #9636
Closed

The --target option clashes with other command line flags and config files #4390

dstufft opened this issue Mar 30, 2017 · 28 comments · Fixed by #9636
Labels
C: editable Editable installations C: target pip install's --target option's behaviour handling project: setuptools Related to setuptools

Comments

@dstufft
Copy link
Member

dstufft commented Mar 30, 2017

The --target option clashes with several other command like flags and config files like --user and distutils setup. Ideally we should handle this far more gracefully.

See also #3826, #4106, #562, #4139

@halms
Copy link

halms commented Apr 1, 2017

I have made an attempt to fix the behavior of --target in PR #4111.

@AlexeySanko
Copy link

#4111 was rebased to #4557

@jedie
Copy link

jedie commented Mar 18, 2018

Current pull request is: #4557 isn't it?

@denisglotov
Copy link

denisglotov commented Aug 5, 2018

The bug disappears with the latest pip 18.0.

@cjerdonek cjerdonek added the C: editable Editable installations label Aug 5, 2018
@Waseem-Q
Copy link

This related issue still appears with pip 18.0
pypa/setuptools#392

@AlJohri
Copy link

AlJohri commented Sep 6, 2018

Can confirm, this issue still persists.

@revmischa
Copy link

Getting this error when installing with pipenv a local -e module:

  Running setup.py develop for Flask-Serverless
    Complete output from command /Users/cyber/.virtualenvs/jbwidgets-Jr4Btlt8/bin/python3.7 -c "import setuptools, tokenize;__file__=
'/Users/cyber/dev/flask-serverless/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close(
);exec(compile(code, __file__, 'exec'))" --no-user-cfg develop --no-deps --home=/private/var/folders/n8/rmzyc9yx2zvbc978_p6y_by40000g
n/T/pip-target-1lg_9i3s:
    usage: -c [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
       or: -c --help [cmd1 cmd2 ...]
       or: -c --help-commands
       or: -c cmd --help
    
    error: option --home not recognized

@okgolove
Copy link

okgolove commented Nov 7, 2018

It still doesn't work.

@tedivm
Copy link

tedivm commented Mar 29, 2019

Does anyone have any advice on how to work around this issue? It's particularly problematic when dealing with private repositories (since the -e flag breaks things), and this issue essentially makes it impossible to work with private repositories on serverless setups.

@cankarm
Copy link

cankarm commented Apr 12, 2019

I have the same issue which breaks the deployment of serverless functions. The workaround with setting the prefix= to blanck in ~/.pydistutils.cfg does not work. Does anyone have another idea?

@rouge8
Copy link
Contributor

rouge8 commented Apr 12, 2019

#6008 might fix some of this. Not sure what needs to be done on that or if it just needs to be reviewed?

@rdlou
Copy link

rdlou commented May 30, 2019

Is there an update for this? its pretty essential for creating zips to deploy on aws lambda

@shlomizadok
Copy link

Solved this for the time being on Ubuntu 18.04.2 LTS with
setting the prefix= to blank in /root/.pydistutils.cfg
and sudo pip install <Package> --target <dir>

@uranusjr
Copy link
Member

@joinsito It is impossible to suggest workarounds without information on your runtime configurations and environment.

@erindru
Copy link

erindru commented Apr 20, 2020

@joinsito I was trying to package libraries for AWS Lambda and was getting this issue. Their documentation had a helpful suggestion:

In order for --target to work on Debian-based systems like Ubuntu, you may also need to pass the --system flag to prevent distutils errors.

So by adding the --system flag to the list of arguments, I was able to resolve my particular issue

@nuno-andre
Copy link

@lorenzoLectrefy Maybe this could be a workaround for your case:

pip wheel \
  --src . \
  --editable git+ssh://[email protected]/myOrg/myPackage@master#egg=my-package

@zooba
Copy link
Contributor

zooba commented Jul 7, 2020

I just posted (half) a PR that enables at least some of this. Not sure if it's what the team had in mind, but I'm open to making changes.

PR #8555

@pradyunsg
Copy link
Member

This is also covered by #7828.

@zooba
Copy link
Contributor

zooba commented Jul 8, 2020

This is also covered by #7828.

Not quite. My change makes it so that --target on the command line will override the PIP_USER environment variable (also --prefix, which now I think about it isn't quite right?).

But #7828 is a good looking change to take as well. We'll just have to update the "check for incompatible options" part of it to allow CLI flags to override other defaults.

@ghost
Copy link

ghost commented Feb 5, 2021

I had to combine the --target and --editable options when I needed to package my custom code shared among my AWS Lambda Functions as a Lambda layer, and ended up with an error error: option --home not recognized.

My workaround so far is

  1. Create a wheel with --editable,

    pip wheel --editable .

    Suppose the output is my-module-x.y.z.whl.

  2. Install from the wheel created at the step 1 (i.e., my-module-x.y.z.whl) with --target.

    pip install --target ../package/python my-module-x.y.z.whl

@dwt
Copy link
Contributor

dwt commented Feb 11, 2021

I don't pretend to understand all the history in this bug - but it seems sensible to convert the --target $folder option to --install-dir=$folder, when calling setup.py develop instead of providing --home would provide a good enough workaround for this? Is there any reason not to do this?

@RonnyPfannschmidt
Copy link
Contributor

time available and the fact that until recently this was using easy_install which is not compatible with --home

dwt added a commit to dwt/pip that referenced this issue Feb 20, 2021
… understand --home but instead requires the --install-dir option.

Parially fixes pypa#4390
@dwt
Copy link
Contributor

dwt commented Feb 20, 2021

@RonnyPfannschmidt I've created a pull request that implements the fix by renaming the argument handed to setup.py. What do you think?

@RonnyPfannschmidt
Copy link
Contributor

Looks like a breaking change

@dwt
Copy link
Contributor

dwt commented Feb 20, 2021

@RonnyPfannschmidt I'm not sure I understand - the behaviour was broken before, as in using --target together with --editable exploded while installing. Is that what you mean?

@uranusjr uranusjr added project: setuptools Related to setuptools and removed S: needs triage Issues/PRs that need to be triaged labels Feb 28, 2021
@uranusjr
Copy link
Member

This is not yet fixed since we still need some changes in setuptools, but I’m closing since there’s nothing more to do at pip’s side. Kindly continue the discussion on setuptools’s issue tracker, and open new issues if it turns out pip needs to do something more about this.

inmantaci added a commit to inmanta/inmanta-core that referenced this issue Apr 27, 2021
Bumps [pip](https://github.com/pypa/pip) from 21.0.1 to 21.1.
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a href="https://github.com/pypa/pip/blob/main/NEWS.rst">pip's changelog</a>.</em></p>
<blockquote>
<h1>21.1 (2021-04-24)</h1>
<h2>Process</h2>
<ul>
<li>Start installation scheme migration from <code>distutils</code> to <code>sysconfig</code>. A
warning is implemented to detect differences between the two implementations to
encourage user reports, so we can avoid breakages before they happen.</li>
</ul>
<h2>Features</h2>
<ul>
<li>Add the ability for the new resolver to process URL constraints. (<code>[#8253](pypa/pip#8253) &lt;https://github.com/pypa/pip/issues/8253&gt;</code>_)</li>
<li>Add a feature <code>--use-feature=in-tree-build</code> to build local projects in-place
when installing. This is expected to become the default behavior in pip 21.3;
see <code>Installing from local packages &lt;https://pip.pypa.io/en/stable/user_guide/#installing-from-local-packages&gt;</code>_
for more information. (<code>[#9091](pypa/pip#9091) &lt;https://github.com/pypa/pip/issues/9091&gt;</code>_)</li>
<li>Bring back the &quot;(from versions: ...)&quot; message, that was shown on resolution failures. (<code>[#9139](pypa/pip#9139) &lt;https://github.com/pypa/pip/issues/9139&gt;</code>_)</li>
<li>Add support for editable installs for project with only setup.cfg files. (<code>[#9547](pypa/pip#9547) &lt;https://github.com/pypa/pip/issues/9547&gt;</code>_)</li>
<li>Improve performance when picking the best file from indexes during <code>pip install</code>. (<code>[#9748](pypa/pip#9748) &lt;https://github.com/pypa/pip/issues/9748&gt;</code>_)</li>
<li>Warn instead of erroring out when doing a PEP 517 build in presence of
<code>--build-option</code>. Warn when doing a PEP 517 build in presence of
<code>--global-option</code>. (<code>[#9774](pypa/pip#9774) &lt;https://github.com/pypa/pip/issues/9774&gt;</code>_)</li>
</ul>
<h2>Bug Fixes</h2>
<ul>
<li>Fixed <code>--target</code> to work with <code>--editable</code> installs. (<code>[#4390](pypa/pip#4390) &lt;https://github.com/pypa/pip/issues/4390&gt;</code>_)</li>
<li>Add a warning, discouraging the usage of pip as root, outside a virtual environment. (<code>[#6409](pypa/pip#6409) &lt;https://github.com/pypa/pip/issues/6409&gt;</code>_)</li>
<li>Ignore <code>.dist-info</code> directories if the stem is not a valid Python distribution
name, so they don't show up in e.g. <code>pip freeze</code>. (<code>[#7269](pypa/pip#7269) &lt;https://github.com/pypa/pip/issues/7269&gt;</code>_)</li>
<li>Only query the keyring for URLs that actually trigger error 401.
This prevents an unnecessary keyring unlock prompt on every pip install
invocation (even with default index URL which is not password protected). (<code>[#8090](pypa/pip#8090) &lt;https://github.com/pypa/pip/issues/8090&gt;</code>_)</li>
<li>Prevent packages already-installed alongside with pip to be injected into an
isolated build environment during build-time dependency population. (<code>[#8214](pypa/pip#8214) &lt;https://github.com/pypa/pip/issues/8214&gt;</code>_)</li>
<li>Fix <code>pip freeze</code> permission denied error in order to display an understandable error message and offer solutions. (<code>[#8418](pypa/pip#8418) &lt;https://github.com/pypa/pip/issues/8418&gt;</code>_)</li>
<li>Correctly uninstall script files (from setuptools' <code>scripts</code> argument), when installed with <code>--user</code>. (<code>[#8733](pypa/pip#8733) &lt;https://github.com/pypa/pip/issues/8733&gt;</code>_)</li>
<li>New resolver: When a requirement is requested both via a direct URL
(<code>req @ URL</code>) and via version specifier with extras (<code>req[extra]</code>), the
resolver will now be able to use the URL to correctly resolve the requirement
with extras. (<code>[#8785](pypa/pip#8785) &lt;https://github.com/pypa/pip/issues/8785&gt;</code>_)</li>
<li>New resolver: Show relevant entries from user-supplied constraint files in the
error message to improve debuggability. (<code>[#9300](pypa/pip#9300) &lt;https://github.com/pypa/pip/issues/9300&gt;</code>_)</li>
<li>Avoid parsing version to make the version check more robust against lousily
debundled downstream distributions. (<code>[#9348](pypa/pip#9348) &lt;https://github.com/pypa/pip/issues/9348&gt;</code>_)</li>
<li><code>--user</code> is no longer suggested incorrectly when pip fails with a permission
error in a virtual environment. (<code>[#9409](pypa/pip#9409) &lt;https://github.com/pypa/pip/issues/9409&gt;</code>_)</li>
<li>Fix incorrect reporting on <code>Requires-Python</code> conflicts. (<code>[#9541](pypa/pip#9541) &lt;https://github.com/pypa/pip/issues/9541&gt;</code>_)</li>
</ul>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a href="https://github.com/pypa/pip/commit/2b2a268d25963727c2a1c805de8f0246b9cd63f6"><code>2b2a268</code></a> Bump for release</li>
<li><a href="https://github.com/pypa/pip/commit/ea761a6575f37b90cf89035ee8be3808cf872184"><code>ea761a6</code></a> Update AUTHORS.txt</li>
<li><a href="https://github.com/pypa/pip/commit/2edd3fdf2af2f09dce5085ef0eb54684b4f9bc04"><code>2edd3fd</code></a> Postpone a deprecation to 21.2</li>
<li><a href="https://github.com/pypa/pip/commit/3cccfbf169bd35133ee25d2543659b9c1e262f8c"><code>3cccfbf</code></a> Rename mislabeled news fragment</li>
<li><a href="https://github.com/pypa/pip/commit/21cd124b5d40b510295c201b9152a65ac3337a37"><code>21cd124</code></a> Fix NEWS.rst placeholder position</li>
<li><a href="https://github.com/pypa/pip/commit/e46bdda9711392fec0c45c1175bae6db847cb30b"><code>e46bdda</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/pypa/pip/issues/9827">#9827</a> from pradyunsg/fix-git-improper-tag-handling</li>
<li><a href="https://github.com/pypa/pip/commit/0e4938d269815a5bf1dd8c16e851cb1199fc5317"><code>0e4938d</code></a> 📰</li>
<li><a href="https://github.com/pypa/pip/commit/ca832b2836e0bffa7cf95589acdcd71230f5834e"><code>ca832b2</code></a> Don't split git references on unicode separators</li>
<li><a href="https://github.com/pypa/pip/commit/1320bac4ff80d76b8fba2c8b4b4614a40fb9c6c3"><code>1320bac</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/pypa/pip/issues/9814">#9814</a> from pradyunsg/revamp-ci-apr-2021-v2</li>
<li><a href="https://github.com/pypa/pip/commit/e9cc23ffd97cb6d66d32dc3ec27cf832524bb33d"><code>e9cc23f</code></a> Skip checks on PRs only</li>
<li>Additional commits viewable in <a href="https://github.com/pypa/pip/compare/21.0.1...21.1">compare view</a></li>
</ul>
</details>
<br />

[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=pip&package-manager=pip&previous-version=21.0.1&new-version=21.1)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually

</details>
inmantaci added a commit to inmanta/inmanta-core that referenced this issue May 18, 2021
Bumps [pip](https://github.com/pypa/pip) from 21.0.1 to 21.1.1.
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a href="https://github.com/pypa/pip/blob/main/NEWS.rst">pip's changelog</a>.</em></p>
<blockquote>
<h1>21.1.1 (2021-04-30)</h1>
<h2>Deprecations and Removals</h2>
<ul>
<li>Temporarily set the new &quot;Value for ... does not match&quot; location warnings level
to <em>DEBUG</em>, to hide them from casual users. This prepares pip 21.1 for CPython
inclusion, while pip maintainers digest the first intake of location mismatch
issues for the <code>distutils</code>-<code>sysconfig</code> transition. (<code>[#9912](pypa/pip#9912) &lt;https://github.com/pypa/pip/issues/9912&gt;</code>_)</li>
</ul>
<h2>Bug Fixes</h2>
<ul>
<li>This change fixes a bug on Python <!-- raw HTML omitted -->`_)</li>
<li>Fix compatibility between distutils and sysconfig when the project name is unknown outside of a virtual environment. (<code>[#9838](pypa/pip#9838) &lt;https://github.com/pypa/pip/issues/9838&gt;</code>_)</li>
<li>Fix Python 3.6 compatibility when a PEP 517 build requirement itself needs to be
built in an isolated environment. (<code>[#9878](pypa/pip#9878) &lt;https://github.com/pypa/pip/issues/9878&gt;</code>_)</li>
</ul>
<h1>21.1 (2021-04-24)</h1>
<h2>Process</h2>
<ul>
<li>Start installation scheme migration from <code>distutils</code> to <code>sysconfig</code>. A
warning is implemented to detect differences between the two implementations to
encourage user reports, so we can avoid breakages before they happen.</li>
</ul>
<h2>Features</h2>
<ul>
<li>Add the ability for the new resolver to process URL constraints. (<code>[#8253](pypa/pip#8253) &lt;https://github.com/pypa/pip/issues/8253&gt;</code>_)</li>
<li>Add a feature <code>--use-feature=in-tree-build</code> to build local projects in-place
when installing. This is expected to become the default behavior in pip 21.3;
see <code>Installing from local packages &lt;https://pip.pypa.io/en/stable/user_guide/#installing-from-local-packages&gt;</code>_
for more information. (<code>[#9091](pypa/pip#9091) &lt;https://github.com/pypa/pip/issues/9091&gt;</code>_)</li>
<li>Bring back the &quot;(from versions: ...)&quot; message, that was shown on resolution failures. (<code>[#9139](pypa/pip#9139) &lt;https://github.com/pypa/pip/issues/9139&gt;</code>_)</li>
<li>Add support for editable installs for project with only setup.cfg files. (<code>[#9547](pypa/pip#9547) &lt;https://github.com/pypa/pip/issues/9547&gt;</code>_)</li>
<li>Improve performance when picking the best file from indexes during <code>pip install</code>. (<code>[#9748](pypa/pip#9748) &lt;https://github.com/pypa/pip/issues/9748&gt;</code>_)</li>
<li>Warn instead of erroring out when doing a PEP 517 build in presence of
<code>--build-option</code>. Warn when doing a PEP 517 build in presence of
<code>--global-option</code>. (<code>[#9774](pypa/pip#9774) &lt;https://github.com/pypa/pip/issues/9774&gt;</code>_)</li>
</ul>
<h2>Bug Fixes</h2>
<ul>
<li>Fixed <code>--target</code> to work with <code>--editable</code> installs. (<code>[#4390](pypa/pip#4390) &lt;https://github.com/pypa/pip/issues/4390&gt;</code>_)</li>
<li>Add a warning, discouraging the usage of pip as root, outside a virtual environment. (<code>[#6409](pypa/pip#6409) &lt;https://github.com/pypa/pip/issues/6409&gt;</code>_)</li>
</ul>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a href="https://github.com/pypa/pip/commit/c53d88c4c374523425f4db6bef949090764465c0"><code>c53d88c</code></a> Bump for release</li>
<li><a href="https://github.com/pypa/pip/commit/4417e7f4bef2b2c70767c1dbfe72f82cc6b7b83f"><code>4417e7f</code></a> Update AUTHORS.txt</li>
<li><a href="https://github.com/pypa/pip/commit/0c29bfe48e650c5a428b77eba4af7f067c019cc8"><code>0c29bfe</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/pypa/pip/issues/9912">#9912</a> from uranusjr/sysconfig-remove-warning-for-python-re...</li>
<li><a href="https://github.com/pypa/pip/commit/f56ec327b92ebe836e63e07cb2449a20e09dec38"><code>f56ec32</code></a> Make location mismatch messages DEBUG level</li>
<li><a href="https://github.com/pypa/pip/commit/999b121402302a00b235a0d443f5661b69d6fd60"><code>999b121</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/pypa/pip/issues/9883">#9883</a> from uranusjr/isolated-pip-py36-compat</li>
<li><a href="https://github.com/pypa/pip/commit/f88420319db450aefbed1500f04e31be46874aaf"><code>f884203</code></a> Fallback to self-invoke via directory on 3.6</li>
<li><a href="https://github.com/pypa/pip/commit/7a77484a492c8f1e1f5ef24eaf71a43df9ea47eb"><code>7a77484</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/pypa/pip/issues/9835">#9835</a> from jamescurtin/9831-bugfix</li>
<li><a href="https://github.com/pypa/pip/commit/914bcc3dba88179f4e88ce90b63660474ba795cd"><code>914bcc3</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/pypa/pip/issues/9838">#9838</a> from uranusjr/sysconfig-header-with-none-project</li>
<li><a href="https://github.com/pypa/pip/commit/2a009a0b8a5d8d03117897f0f11f9c1dcf2a4b5a"><code>2a009a0</code></a> Better explanatory comment</li>
<li><a href="https://github.com/pypa/pip/commit/e7b1722efeaf4ff403142847ce1b52caedd5efcd"><code>e7b1722</code></a> Set dist_name to UNKNOWN when empty outside venv</li>
<li>Additional commits viewable in <a href="https://github.com/pypa/pip/compare/21.0.1...21.1.1">compare view</a></li>
</ul>
</details>
<br />

[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=pip&package-manager=pip&previous-version=21.0.1&new-version=21.1.1)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually

</details>
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Sep 30, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
C: editable Editable installations C: target pip install's --target option's behaviour handling project: setuptools Related to setuptools
Projects
None yet