How can I specify differing sets of dependency extras for each of my project's extras?

[kurtiss]

• I have searched the issues of this repo and believe that this is not a duplicate.
• I have searched the documentation and believe that my question is not covered.

Question

How can I specify differing sets of dependency extras for each of my project's extras? This is something that is relatively trivial to do with setup.py. To clarify, I would like to be able to specify the following dependency graph:

myproject[feature-a] depends on [dependency-one, dependency-two[feature-a]]
myproject[feature-b] depends on [dependency-one, dependency-two[feature-b]]

I'll propose a couple ways that pyproject.toml might model these graphs, but the core value of these examples is to highlight an apparent deficiency with the current model.

1. By allowing dependency aliases that specify distinct sets of extras. For example:

[tool.poetry.dependencies]
…
dependency-two-a = { git = "git+ssh://[email protected]/projects/dependency-two.git",optional = true,extras = ["feature-a"], provides="dependency-two" }
dependency-two-b = { git = "git+ssh://[email protected]/projects/dependency-two.git",optional = true,extras = ["feature-b"], provides="dependency-two" }
…

[tool.poetry.extras]
feature-a = ["dependency-one", "dependency-two-a"]
feature-b = ["dependency-one", "dependency-two-b"]

2. By treating the set of extras in the dependencies section as a superset that can be further filtered down when the dependency is referenced in the extras section. For example:

[tool.poetry.dependencies]
…
dependency-two = { git = "git+ssh://[email protected]/projects/dependency-two.git",optional = true,extras = ["feature-a", "feature-b"]}
…

[tool.poetry.extras]
feature-a = ["dependency-one", "dependency-two[feature-a]"]
feature-b = ["dependency-one", "dependency-two[feature-b]"]

Thanks for your time!

[kurtiss]

@sdispater Gently bumping this – is this use case compelling enough to warrant some attention?

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[kurtiss]

Bumping again, because the issue is outstanding and marks a flaw in the way dependencies are modeled, preventing reasonable use of this application. Hope somebody can take a look at this.

[wshayes]

Looks like you can install multiple dependencies in a single set using the following approach:

poetry add -E api fastapi
poetry add -E api gunicorn
poetry add -E api uvicorn

resulting in the following changes to the pyproject.toml file:

fastapi = {version = "^0.61.0", extras = ["api"]}
gunicorn = {version = "^20.0.4", extras = ["api"]}
uvicorn = {version = "^0.11.8", extras = ["api"]}

But running poetry update; poetry install -E API doesn't work yet.

You need to add the following entry to the pyproject.toml file:

[tool.poetry.extras]
api = ["fastapi^0.61.0", "gunicorn^20.0.4", "uvicorn^0.11.8"]

and then poetry update; poetry install -E API works fine.

[micuda]

Thanks for creating this issue, @kurtiss. We have the same problem. I would prefer option (2) proposed by @kurtiss, because it feels more intuitive - this was the first thing I tried.

What do you think, @sdispater?

@wshayes you are describing a different use-case here.

[graingert]

I'm looking to port celery from setuptools to poetry and have this problem also

See #4313

[Qsi1984]

I reported a possible solution using markers here: #4313 (comment).
Maybe this would be a (temporary) solution in your cases too.

[DaanRademaker]

Is this still being worken on? It'quite important to decrease size of package installs for other packages that depend on packages build with poetry.

[blaise17]

I have a subtly different use case, which I'm not sure works with the currently proposed solutions without some modification.
Specifically, I need the dependency without extras to not be optional, but the dependency with extras to be optional. I also need one or more of the extras for the dependency to not be optional, while one or more still are optional.
Would it be possible to specify the optional flag per extra?

For example:

[tool.poetry.dependencies]
…
dependency-one = {version = "^1.2.0", extras = [{extra="feature-a", optional = true}, "feature-b"] }
dependency-two = {version = "^1.2.0", extras = [{extra="feature-a", optional = true}] }
…

[tool.poetry.extras]
feature-a = ["dependency-one[feature-a]", "dependency-two[feature-a]"]

[sorasful]

Hi, do we have workarounds this ?

We do have the same problem, we want to be able to specify extras that depends on each other to avoid redundancy but they are not getting installed.

Thanks!

[sorasful]

So basically, what we did is we created another pyproject.toml which we edit and we use the syntax :

[tool.poetry.extras]
mylib1 = [
    "oauthlib",
    "requests_oauthlib",
]
mylib2 = [
    "mylib1 ",
    "Flask",
    "Fastapi",
]

And we created a script that generates a pyproject.toml file and when it does, it repaces "mylib1" by all the requirements in it. For example we would have something like that :

[tool.poetry.extras]
mylib1 = [
    "oauthlib",
    "requests_oauthlib",
]
mylib2 = [
    "oauthlib",
    "requests_oauthlib",
    "Flask",
    "Fastapi",
]

We added this in a pre-commit, and it's done.
It's not ideal, but it's a workaround.