Unable to upload packages using token auth due to 403 errors

[antoniovazquezblanco]

Your Environment

1. Your operating system: Windows 10

2. Version of python you are running: Python 3.7.4

3. How did you install twine? Did you use your operating system's package manager or pip or something else? pip

4. Version of twine you have installed (include complete output of): twine version 3.1.1 (pkginfo: 1.5.0.1, requests: 2.22.0, setuptools: 40.8.0, requests-toolbelt: 0.9.1, tqdm: 4.42.1)

5. Which package repository are you targeting?
   Ideally twine default repo (I believe it is https://upload.pypi.org/legacy/) but due to failures I am testing against https://test.pypi.org/legacy/.

PKG-INFO.txt

I am not using a .pypirc file but rather setting TWINE_USERNAME and TWINE_PASSWORD environment variables in my Gitlab CI runner.

The Issue

I am unable to perform an upload using a token. I always get a 403 error. I have tryed using the -u and -p parameters in order to see if that worked but this was unsucessful also.

twine upload --repository-url https://test.pypi.org/legacy/ -u __token__ -p pypi-tokentokentokentoken...token  dist/*
Uploading distributions to https://test.pypi.org/legacy/
Uploading django-fe-manager-0.0.3.tar.gz
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 13.1k/13.1k [00:00<00:00, 16.3kB/s] 
NOTE: Try --verbose to see response content.
HTTPError: 403 Client Error: Invalid or non-existent authentication information. for url: https://test.pypi.org/legacy/

I have tryed to generate other tokens and did not suceed.

Steps to Reproduce

Just try to upload a package using a token.

[bhrutledge]

At first glance, the command-line looks correct.

Pardon the question, but are you sure you're using the correct token? For https://test.pypi.org/legacy/, it should be listed on https://test.pypi.org/manage/account/, under "API tokens". Can you provide a screenshot of that section?

[antoniovazquezblanco]

Thank you for the pointer. I did not know that the test repo required a separate account. I have created an account and succesfully uploaded to test using command line arguments. Given that, I've deleted my old tokens in the real pypi index and created a new one. I managed to succesfully upload my package via command line in Windows 10.

However, after updating gitlab runner variables to the correct values I could not upload the package. It seems that gitlab was not setting my environment variables as the pipeline I was running on was not a "protected" job and twine was not warning me that it did not have any credentials to use.

It would be nice to have a feature that warns you about not having credentials before getting a 403 error.

Please feel free to close the issue if the feature above is not desired.

Thank you for the support and fast response.

[bhrutledge]

I'm glad you were able to figure it out.

It would be nice to have a feature that warns you about not having credentials before getting a 403 error.

I read that as "Twine could give a more helpful message than 403 Client Error". I've been thinking along these lines for awhile, after seeing other folks have issues (e.g. #424). I think it's worth a separate issue, but I'm going to leave this open for now as a reference.

Also, maybe Twine's documentation could be more explicit about using tokens and the distinction between PyPI and Test PyPI. Thus far we've been relying on the Packaging User Guide and the PyPI help.

Suggestions welcome.

[antoniovazquezblanco]

What I meant is that twine did not have a configuration file, I had not passed any user credentials and there were no environment variables related to authentication. In that case, when a 403 error is returned by the server, it could be useful for the user to know that no credentials were set. I was believing that they were set via env variables but due to configuration issues, it was not the case.

Thank you!

[deveshks]

Hi @bhrutledge

I wanted to contribute towards fixing this.

I think we can raise a custom exception at https://github.com/pypa/twine/blob/master/twine/utils.py#L144 like AuthenticationException with a custom message.

[bhrutledge]

@deveshks Thanks for offering to help! I haven't really thought about this since February, and your comment reminded me that I've been wanting to write up an issue to refactor/normalize how Twine handles custom exceptions. So, while I think something AuthenticationException is a good idea, I'm a little reluctant to bolt it on at the moment.

However, I also don't want to stifle your interest. So, could you propose what message(s) would get displayed to the user?

A related issue is #424, in which folks also seem to be confused about needing different credentials for PyPI and TestPyPI.

[deveshks]

@deveshks Thanks for offering to help! I haven't really thought about this since February, and your comment reminded me that I've been wanting to write up an issue to refactor/normalize how Twine handles custom exceptions. So, while I think something AuthenticationException is a good idea, I'm a little reluctant to bolt it on at the moment.

Hi @bhrutledge ,
Thank you for your response. Yes, we can certainly improve on how to handle custom exceptions than the current mechanism of comparing error codes and strings against hardcoded constants. And I think since we only have two such comparisons, it's a good time to design such a thing and add this new exception onto that. I would also be interested in implementing such a thing.

However, I also don't want to stifle your interest. So, could you propose what message(s) would get displayed to the user?

From what I understand, we can either provide username and password explicitly in twine upload, or we can set the username as __token__ and provide the api token as the password, we can divide the error messages as either Unable to authenticate against <repo url> for username <user>. Please check your entered password, or Unable to authenticate against <repo url> with the provided api token. Please ensure that the api token is correctly entered.

I think that we can also read these details from the env variables TWINE_PASSWORD or the keyring for them, so we can customize our exceptions based on that knowledge, or maybe not.

A related issue is #424, in which folks also seem to be confused about needing different credentials for PyPI and TestPyPI.

This can be resolved if for e.g. we have a way to 1. Find if the provided user exists for either PyPI/TestPyPI. and 2. If the provided user exists but the authentication info is wrong.

If we encourage them to use keyring to store these for different repo urls, we can perform 1 and 2 on the keyring backend instead. From what I understood TWINE_USERNAME and TWINE_PASSWORD should only allow setting these for one repo url, so these won't be helpful here. A .pypirc file can also be used locally, but you mention here #424 (comment) that the last two options are not recommended.

Also I got to know about twine from @brainwane over at the pypa-dev IRC channel, so I would also want to thank her for introducing me to the project.

I have made some contributions to the pip repo in the past few weeks, and I am willing to take on some of the outstanding issues here as well :)

[bhrutledge]

@deveshks I started to write up an issue to add a custom exception as we've discussed, but then I discovered that the 403 response from PyPI now includes a URL for more information:

Invalid or non-existent authentication information. See https://test.pypi.org/help/#invalid-auth for details.

I'm inclined to rely on that, rather than duplicate the content here. That said, I've opened pypi/warehouse#7812 to add a little more to the PyPI help, and #587 to make Twine's formatting clearer.

With that in mind, I'm going to close this issue, though if folks feel strongly otherwise, it can be re-opened.