requires_dist is null in json api when sdist is uploaded before wheel package

[dkliban]

Your Environment

Thank you for taking the time to report an issue.

To more efficiently resolve this issue, we'd like to know some basic information about your system and setup.

1. Your operating system: Fedora 33

2. Version of python you are running:

python --version

Python 3.9.4

3. How did you install twine? Did you use your operating system's package manager or pip or something else?

pip

4. Version of twine you have installed (include complete output of):

twine --version

twine version 3.4.1 (importlib_metadata: 4.0.1, pkginfo: 1.7.0, requests: 2.24.0, requests-toolbelt: 0.9.1, tqdm: 4.60.0)

5. Which package repository are you targeting?

If you're having issues uploading a specific package, you must include a copy of the following:

• The package's PKG-INFO file
• A redacted version of your .pypirc file (REMOVE ALL USERNAMES & PASSWORDS BEFORE UPLOADING)

The Issue

When I upload the sdist of the package first, the requires_dist field in the JSON API is set to null. When I upload the wheel package first, the requires_dist field in the JSON API is set correctly.

This is an example of the sdist being uploaded first: https://test.pypi.org/pypi/gerrods-test/0.0.1/json
This is an example of the wheel being uploaded first: https://test.pypi.org/pypi/gerrods-test/0.0.2/json

Steps to Reproduce

Upload sdist before upload the wheel package.

This issue is masked when twine upload dist/* is called because the wheel is always uploaded first..

[bhrutledge]

This is new territory for me; thanks for the detailed writeup. One thing that wasn't clear to me was the end result of the requires_dist field not being set, but it looks that maps to the install_requires option in setuptools, so I assume it means your package's dependencies don't get installed.

Some searching yielded some existing issues for this, making me think the issue is not in Twine:

pypa/setuptools#1805
pypi/warehouse#6231
https://discuss.python.org/t/2620

As you noted, this isn't an issue in practice, because Twine will always upload the wheel first:

twine/twine/commands/__init__.py

Line 48 in 2afd7ad

return _group_wheel_files_first(uploads)

When I upload the sdist of the package first

Why are you doing that? 😉

[dkliban]

This is new territory for me; thanks for the detailed writeup. One thing that wasn't clear to me was the end result of the requires_dist field not being set, but it looks that maps to the install_requires option in setuptools, so I assume it means your package's dependencies don't get installed.

Some searching yielded some existing issues for this, making me think the issue is not in Twine:

pypa/setuptools#1805
pypa/warehouse#6231
https://discuss.python.org/t/2620

As you noted, this isn't an issue in practice, because Twine will always upload the wheel first:

twine/twine/commands/__init__.py

Line 48 in 2afd7ad

return _group_wheel_files_first(uploads)

When I upload the sdist of the package first

Why are you doing that? wink

I am building 2 packages as part of our release workflow. The server package and the client package. These assets are then tested and published to PyPI. The packages are both in the dist directory. I break up the upload of the server and client into separate jobs. That's why I don't specify dist/*. Should I not be storing the packages in the same dist directory?

[sigmavirus24]

That's why I don't specify dist/*. Should I not be storing the packages in the same dist directory?

If you're using the same auth to upload both and not project specific tokens, you do not need to run separate commands. Even if you really wanted to continue doing that, you could do dist/server* and dist/client* (as examples) instead of trying to specify the files yourself

[bhrutledge]

I am building 2 packages as part of our release workflow.

I'm curious about your configuration; do you have a repo that you can share? I've generally assumed one Project per directory:

another practical way to define projects currently is something that contains a pyproject.toml, setup.py, or setup.cfg file at the root of the project source directory.

So, in my head, you'd have a client and server Project, each of which generates two Packages: a Source and a Wheel.

That said, I echo @sigmavirus24 comment.

[sigmavirus24]

I've generally assumed one Project per directory:

I'm imagining something like

lib/
server/
  - setup.cfg
client/
  - setup.cfg
dist/

Where both rely on modules in lib. I could envision having automation that builds those from the common root and puts them all in the dist/ directory. I've seen patterns like this before as well

[bhrutledge]

I'm going to close this due to inactivity, and because it doesn't seem like an issue with Twine. We can reopen it if necessary