How is the module directory name (src) being specified?

[charlesreid1]

Guide link

https://packaging.python.org/en/latest/tutorials/packaging-projects/

Problem description

The Packaging Python Projects tutorial walks through creating a Python package with the following directory structure:

packaging_tutorial/
├── LICENSE
├── pyproject.toml
├── README.md
├── src/
│   └── example_package_YOUR_USERNAME_HERE/
│       ├── __init__.py
│       └── example.py
└── tests/

However, the src/ directory, which is the location of the module code, is not specified anywhere in pyproject.toml. There is apparently no way for pyproject.toml to know the source code is in src.

So, how does pyproject.toml know to find the source code in src?

Secondly, if I wanted to use a different package structure, where the source code is in src/ but without the extra directory named example_package_YOUR_LUSERNAME_HERE, how would I do that with pyproject.toml?

packaging_tutorial/
├── LICENSE
├── pyproject.toml
├── README.md
├── src/
│   └── __init__.py
│   └── example.py
└── tests/

I have been searching for more information about how to specify the package_dir option, which is how you would normally do this in a setup.py-based project, as per a pypa example, but I can't find any infromation about package_dir options in pyproject.toml.

What has me extremely confused is this prior issue: #518 (comment)

That person was working through the exact same tutorial, and references a package_dir option that doesn't seem to be present. Any assistance would be appreciated.

[merwok]

how does pyproject.toml know to find the source code in src?

That depends on the specific build tool used. Some may require src (for reasons explained in https://hynek.me/articles/testing-packaging/ and pypa/packaging.python.org#320), some may allow it and auto-detect, or allow and require some config (setuptools used to require something like packages = find: + where = src).

if I wanted to use a different package structure, where the source code is in src/ but without the extra directory named example_package_YOUR_LUSERNAME_HERE

What is the goal or use case of that?

[henryiii]

All four tools mentioned in the tutorial (hatchling, hatchling, flit, and pdm-pep517) will look for "src/<package-name>" in addition to "<package-name>" without extra configuration. Details vary a bit, but they all work. Other tools, like trampolim and whey, do not, and have to be configured.

if I wanted to use a different package structure, where the source code is in src/ but without the extra directory named example_package_YOUR_LUSERNAME_HERE

You really, really don't want to do this except in very special situations. The contents of this directory get installed directly into site-packages, so the only valid use cases I can think of would be if you are distributing a single file <package-name>.py or if you are distributing multiple packages, like pytest & _pytest or setuptools & pkgconfig, but this is rare for a new package.

[kalvdans]

All four tools mentioned in the tutorial (hatchling, hatchling, flit, and pdm-pep517) will look for "src/<package-name>" in addition to "<package-name>" without extra configuration. Details vary a bit, but they all work. Other tools, like trampolim and whey, do not, and have to be configured.

Sorry to revive this old thread, but where do I list which <package-name> I want to package? It seems to work for me by magic with neither the src nor the package-name specified. (I have set a name under [project] in my pyproject.toml which differs from the Python package name.)

[henryiii]

It’s the name. If your package name and your name of your module do not match, then there are ways to still do it, but that’s going to be specific to the backend. But you only should do this in very special situations, as it will almost always cause problems if the pip install name and import name don’t match. Such as seven or eight years later, getting into a fight over the person who does own the PyPI name matching your import. I’ve had to help with one of those. ;)

setuptools tries to find all packages, and just rules out some common names like “test”. Hatchling requires:

[tool.hatch.build.targets.wheel]
packages = ["src/foo"]

Other backends vary.

[kalvdans]

Thanks for your insight! I found that setuptools default hardcode src directory, and rules out directories without Python files in them, that's why it magically worked for us.