Rewrite relative package dependencies as regular dependencies

[tnielens]

Is your feature request related to a problem? Please describe.
I have a multi package repository.

- package-a/
- package-b/
- package-c/
...

These packages can depend on one another in an acyclic fashion. These packages can also depend on 3rd party dependencies. All packages are versioned and published together.
I'd like to publish each of these packages separately to my package index and have the relative dependencies turned into regular transitive dependencies. The current behavior I observe with this plugin is that transitive dependencies are copied entirely in the wheel instead of being referred to by package coordinates in the wheel metadata.

Describe the solution you'd like
Publish the packages separately instead of folding everything into a fat package.

Describe alternatives you've considered
None.

Additional context
I'm coming from the jvm world where publishing modules this way is common. At my orgnanisation, we maintain several multi module repositories and publish all modules separately with the same version.

[DavidVujic]

Hi @tnielens, thank you for reporting this issue!

Can you explain a bit further? I don't understand the problem. Do you have examples of pyproject.toml to share?

[tnielens]

I recommend the long thread in issue python-poetry/poetry#2270 :).
In some of the POCs listed in the issue, dependencies between published packages are done with regular poetry dependencies (as opposed to package relative references used by poetry-multiproject-plugin).

From the example above:

pyproject.toml        -> root aggregating project
package-a/            -> subackage a
package-a/package-a/  -> sources a here
package-a/pyproject.toml
package-b/            -> subpackage b
package-b/package-b/  -> sources b here
package-b/pyproject.toml

The way I undestand the doc of this plugin, if package-a depends on package-b, it must be declared the following way in package-a/pyproject.toml.

[tool.poetry]
packages = [
    { include = "package-a", from = "." },
    { include = "package-b", from = "../package-b" } 
]

If the package-a wheel is built, it will inline package-b. But I'd like the wheel package-anot to inline package-b, but to declare a dependency on it as a separate published package.

This can be almost achieved by using poetry path dependencies as in

[tool.poetry.dependencies]
package_b = {path = "../package-b", develop = true}

But the issue with this second approach is that the sdist and wheel built will include a relative reference to package-b although I'd like to substitute it with a regular dependency with the project monoversion.

See python-poetry/poetry#2270 (comment) .

[DavidVujic]

Thank you for the clarification!

The use case for this plugin is to share code between services, apps or libraries. This can be done in several ways but it is recommended that the actual Python code to be shared lives outside of the project(s) themselves as suggested in the docs.

About packages:
I think this might be the confusing naming convention that is used in the Python community. packages, as used in the [tool.poetry] section is about the Python code put in folders (also known as namespace packages) - i.e. your Python code.

The things that are added in the [tool.poetry.dependencies] section are third-party libraries. This is usually the things you install from PyPI, but can also be added (installed) from a local folder or from a git repository. Also, a library defines its own third-party dependencies. These things - the libraries - aren't handled by the MultiProject plugin, but Poetry does this thing pretty well already.

[DavidVujic]

I'll close this one, because I believe there is no action to be taken right now. Please reopen this issue if you don't agree and have ideas on how to proceed 🙏