Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Immitate pip install --no-deps / Add --only-root #3377

Closed
2 tasks done
Natureshadow opened this issue Nov 18, 2020 · 19 comments · Fixed by #5783
Closed
2 tasks done

Immitate pip install --no-deps / Add --only-root #3377

Natureshadow opened this issue Nov 18, 2020 · 19 comments · Fixed by #5783
Labels
kind/feature Feature requests/implementations

Comments

@Natureshadow
Copy link

Natureshadow commented Nov 18, 2020

  • I have searched the issues of this repo and believe that this is not a duplicate.
  • I have searched the documentation and believe that my question is not covered.

Feature Request

It would be great if Poetry could support installing only the root of the project, but leave all dependencies untouched, much like the --no-deps option to pip does.

The use case is a development environment where the code being worked on is spread across several packages, and I want to keep several of them editable in the same poetry environment without having to temporarily change/remove a dependency from pyproject.toml.

Poetry ould optionally still check whether the installed packages fulfill the dependencies or not, but with such an option that developers will explicitly use, this is not necessary I suppose.

@Natureshadow Natureshadow added kind/feature Feature requests/implementations status/triage This issue needs to be triaged labels Nov 18, 2020
@sinoroc
Copy link

sinoroc commented Nov 18, 2020

To make sure I understand, you have tried the following and it does not fit your workflow, right?

[tool.poetry.dependencies]
# ...
library = "^0.1.0"

[tool.poetry.dev-dependencies]
library = {path = "../library", develop = true}

@Natureshadow
Copy link
Author

Well, this is a nice hack, but it would require one of manipulating pyproject.toml every time I start developing on such a situation, or forcing relative paths on every developer in the project (given that the library is, or might be, in a different Git repository/whatever).

@wom
Copy link

wom commented Apr 7, 2021

I'd like to add a +1 here; sometimes upstream packages are too conservative when pinning dependencies. I'd rather avoid having to add a pip install --no-dep <package==... after my initial poetry install.

@cglacet
Copy link

cglacet commented Jun 8, 2021

Does anyone found a way to prevent install sub-dependencies? It would also be interesting when trying to generate documentation for another package (where you only want to be able to read the package's code).

My current fix is to put all the concerned package in my dev-dependencies, then run poetry with --no-dev and use pip to install the dependencies from a (git) submodule directory.

In my CI job, that looks like:

poetry install --no-dev -vv
git submodule sync --recursive
git submodule update --init --recursive
poetry run python -m pip install --no-deps submodules/*

@Badg
Copy link

Badg commented Oct 8, 2021

Just adding another +1 for a different use case: multi-stage installations for quicker docker builds. Basically, the logic is, at my dayjob we build a new docker image for every code change, which means docker build time can be pretty important. Our application code changes a lot more frequently than our dependencies, so we split the install into two different phases: first, the dependencies, and then, the root project. That way, when all we have is changes in the application code, the docker image builder doesn't need to reinstall dependencies on every build. Hopefully that explanation makes sense (if you're familiar with docker anyways).

The relevant bit of our dockerfile currently looks like this (side note: I personally would rather be distributing the application package as a first-class python package hosted on a private package index, but that's a completely separate conversation that has nothing to do with poetry!):

# Copy and install the project definition and deps first, so that we
# don't need to re-run this step if we only changed our application code.
# Note that this is installed into a clean virtualenv
COPY pyproject.toml .
COPY poetry.lock .
RUN poetry install --no-root --no-dev

# copy the rest and install our application package
COPY . .
RUN poetry run pip install --no-dependencies .

What I'd love to see instead, is something like this:

COPY pyproject.toml .
COPY poetry.lock .
RUN poetry install --no-root --no-dev
COPY . .
RUN poetry install --only-root

Edit: I'm stupid busy right now, so I can't put together a PR, but if in a month or two I have some free time and nobody else has closed this, would the maintainers be open to a PR?

@cmin764
Copy link

cmin764 commented Feb 1, 2022

I'm interested into this!

Use case: installing locally for development a package that requires a main dependency which isn't available anymore given the upgraded Python interpreter version for this local case only. Therefore I'm manually installing another version of the dependency, one that is available but can't continue on poetry install the project itself because that specific pinned down dependency version isn't satisfied. (and I don't want to edit pyproject.toml or the lock file neither)

@iamhatesz
Copy link

iamhatesz commented Feb 7, 2022

@Badg 100% agree with this use case. I noticed the same issue in our setup, where we have a tons of heavy ML libraries, which we'd like to avoid downloading each time.

@caniko
Copy link

caniko commented May 9, 2022

+1

@abn
Copy link
Member

abn commented May 9, 2022

Poetry on master should make this possible, albeit, via misfeature.

poetry install --only=non-existent-group

@Maciej-Zwolinski
Copy link

Yet Another Use Case:

Since poetry does not recognize torch==1.10.2+cu113 as satisfying torch>=1.10.2 it'd be great to force poetry to ignore dependencies for packages that require specific versions of torch.

@MikiGrit
Copy link

I'm also interested in this functionality.

Our usecase... pytorch-toolbelt depends on any opencv package and have specified opencv-python as a dependency. This is unfortunately problem for us since the opencv-python package have also dependency on OpenGL library which is difficult to install in our Gitlab CI. On the other hand it would be perfectly fine to install non-opengl variant (opencv-python-headless) ourselves but we also need a way how to tell poetry that we don't want to install dependencies for pytorch-toolbelt.

@cristianmtr
Copy link

cristianmtr commented Jan 9, 2024

Is this possible now? Couldn't figure it out from perusing the docs. We hit the problem:

  • we have project A
  • project A has a dependency on project B
  • proj B has strict dependencies (versions) on projects C, D, etc.
  • in proj A we don't want to install the specific versions of projects C, D, etc. that proj. B requires, but use other versions

Currently we solve this by having a two-step process (in proj. A):

pip install -e .[dev,whatever,etc.]
pip install --no-deps projB

It would be nice to find a packaging system that can add the --no-deps functionality in the requirements file itself (setup.py, project.toml, etc.)

@sinoroc
Copy link

sinoroc commented Jan 9, 2024

@cristianmtr There is now poetry install --only-root:

@cristianmtr
Copy link

@cristianmtr There is now poetry install --only-root:

* https://python-poetry.org/docs/cli/#options-2

* [Add `--only-root` option to install command, excluding all dependencies. #5783](https://github.com/python-poetry/poetry/pull/5783)

But this is only for the installation of the main project (project A) itself, not for the installation of dependencies (project B).

How can I add a dependency in my pyproject.toml and specify that I want to do --only-root for that dependency (project B in my above example).

@charlienewey-odin
Copy link

charlienewey-odin commented Jan 10, 2024

As @Maciej-Zwolinski has said, this poses difficulties for when certain libraries have unusual package suffixes.

For example, the official NVIDIA PyTorch image (nvcr.io/nvidia/pytorch:23.12-py3) comes preconfigured with at least 3 packages that cause errors:

numba = "0.57.1+1.g4157f3379"
torch = "2.2.0a0+81ea7a4"
torchvision = "0.17.0a0"

This makes poetry a bit frustrating to use in some edge cases, because poetry doesn't detect them as "valid"

@Natureshadow
Copy link
Author

Natureshadow commented Jan 10, 2024 via email

@caniko
Copy link

caniko commented Jan 10, 2024

I would recommend opening a separate issue for this problem

@sinoroc
Copy link

sinoroc commented Jan 10, 2024

But this is only for the installation of the main project (project A) itself, not for the installation of dependencies (project B).

If it is for dependencies, then yes it needs a different GitHub issue, yes.

It seems for such cases, an override mechanism for recursive dependencies would be more appropriate.

I started a discussion on this here (not limited to Poetry): https://discuss.python.org/t/proposal-overrides-for-installers/23666

Copy link

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Feb 29, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
kind/feature Feature requests/implementations
Projects
None yet
Development

Successfully merging a pull request may close this issue.