Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

uv pip install/compile fails on AWS codeartifact repo #1388

Closed
raylu opened this issue Feb 15, 2024 · 13 comments · Fixed by #1413
Closed

uv pip install/compile fails on AWS codeartifact repo #1388

raylu opened this issue Feb 15, 2024 · 13 comments · Fixed by #1413
Labels
bug Something isn't working registry Related to package indexes and registries

Comments

@raylu
Copy link

raylu commented Feb 15, 2024 •
edited
Loading

pip works:

$ pip install pip --index-url 'https://aws:[password]@[account].d.codeartifact.us-west-2.amazonaws.com/pypi/shared-packages-pypi/simple' -U
Looking in indexes: https://aws:****@[account].d.codeartifact.us-west-2.amazonaws.com/pypi/shared-packages-pypi/simple
Collecting pip
  Downloading https://[account].d.codeartifact.us-west-2.amazonaws.com/pypi/shared-packages-pypi/simple/pip/24.0/pip-24.0-py3-none-any.whl (2.1 MB)
     |████████████████████████████████| 2.1 MB 1.2 MB/s 
Installing collected packages: pip
Successfully installed pip-24.0

but uv doesn't:

$ uv pip install pip --index-url 'https://aws:[password]@[account].d.codeartifact.us-west-2.amazonaws.com/pypi/shared-packages-pypi/simple'
error: Failed to download: pip==24.0
  Caused by: HTTP status client error (404 Not Found) for url (https://[account].d.codeartifact.us-west-2.amazonaws.com/pypi/shared-packages-pypi/simple/24.0/pip-24.0-py3-none-any.whl#sha256=ba0d021a166865d2265246961bec0152ff124de910c5cc39f1156ce3fa7c69dc)

notice that pip is using /simple/pip/24.0 but uv is using /simple/24.0

related (but different URL patterns): #1362, #1371
based on #651, do you already have a AWS codeartifact repo handy to test with?
@zanieb zanieb added the bug Something isn't working label Feb 15, 2024
@jasonprado
Copy link

I'm seeing similar 404s on a jfrog.io repository

@zanieb zanieb added the registry Related to package indexes and registries label Feb 15, 2024
@zanieb
Copy link
Member

zanieb commented Feb 16, 2024

Does this work if you include the /pip in your index URL?

We do not have a CodeArtifact repo to test with yet.

@charliermarsh
Copy link
Member

I can set one up for us, important to have.

@raylu
Copy link
Author

raylu commented Feb 16, 2024

sorry, maybe I should have picked a less confusing package to install. but /pip there is the name of the package

$ pip install jinja2 --index-url 'https://aws:[password]@[account].d.codeartifact.us-west-2.amazonaws.com/pypi/shared-packages-pypi/simple'
Looking in indexes: https://aws:****@[account].d.codeartifact.us-west-2.amazonaws.com/pypi/shared-packages-pypi/simple
Collecting jinja2
  Downloading https://[account].d.codeartifact.us-west-2.amazonaws.com/pypi/shared-packages-pypi/simple/jinja2/3.1.3/Jinja2-3.1.3-py3-none-any.whl (133 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 133.2/133.2 kB 2.1 MB/s eta 0:00:00
Collecting MarkupSafe>=2.0 (from jinja2)
  Downloading https://[account].d.codeartifact.us-west-2.amazonaws.com/pypi/shared-packages-pypi/simple/markupsafe/2.1.5/MarkupSafe-2.1.5-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (26 kB)
Installing collected packages: MarkupSafe, jinja2
Successfully installed MarkupSafe-2.1.5 jinja2-3.1.3

$ uv pip install jinja2 --index-url 'https://aws:[password]@[account].d.codeartifact.us-west-2.amazonaws.com/pypi/shared-packages-pypi/simple'
error: Failed to download: jinja2==3.1.3
  Caused by: HTTP status client error (404 Not Found) for url (https://[account].d.codeartifact.us-west-2.amazonaws.com/pypi/shared-packages-pypi/simple/3.1.3/Jinja2-3.1.3-py3-none-any.whl#sha256=7d6d50dd97d52cbc355597bd845fabfbac3f551e1f99619e39a35ce8c370b5fa)

@zanieb
Copy link
Member

zanieb commented Feb 16, 2024

Ahh thank you!

So to highlight the difference here:

pip: ../simple/jinja2/3.1.3/Jinja2-3.1.3-py3-none-any.whl
uv:  ../simple/3.1.3/Jinja2-3.1.3-py3-none-any.whl

It looks like we're failing to include the package name.

@charliermarsh
Copy link
Member

My guess is that the index returns relative URLs, and we're making them relative to the index rather than to the page that we requested.

@charliermarsh
Copy link
Member

@raylu - Is there any way you can confirm for me, if you request the https://aws:[password]@[account].d.codeartifact.us-west-2.amazonaws.com/pypi/shared-packages-pypi/simple/flask/?format=application/vnd.pypi.simple.v1+json or similar, are the URLs relative? And are they relative to /simple/flask?

@charliermarsh charliermarsh self-assigned this Feb 16, 2024
@raylu
Copy link
Author

raylu commented Feb 16, 2024 

$ curl 'https://aws:[password]@[account].d.codeartifact.us-west-2.amazonaws.com/pypi/shared-packages-pypi/simple/flask/?format=application/vnd.pypi.simple.v1+json'
<!DOCTYPE html>
<html>
  <head>
    <title>Links for flask</title>
  </head>
  <body>
    <h1>Links for flask</h1>
      <a href="0.1/Flask-0.1.tar.gz#sha256=9da884457e910bf0847d396cb4b778ad9f3c3d17db1c5997cb861937bd284237"  data-gpg-sig="false" >Flask-0.1.tar.gz</a>
      <br/>
      <a href="0.10.1/Flask-0.10.1.tar.gz#sha256=4c83829ff83d408b5e1d4995472265411d2c414112298f2eb4b359d9e4563373"  data-gpg-sig="false" >Flask-0.10.1.tar.gz</a>
      <br/>
[...]
      <br/>
      <a href="3.0.1/flask-3.0.1.tar.gz#sha256=6489f51bb3666def6f314e15f19d50a1869a19ae0e8c9a3641ffe66c77d42403" data-requires-python="&gt;=3.8" data-gpg-sig="false" >flask-3.0.1.tar.gz</a>
      <br/>
      <a href="3.0.2/flask-3.0.2-py3-none-any.whl#sha256=3232e0e9c850d781933cf0207523d1ece087eb8d87b23777ae38456e2fbe7c6e" data-requires-python="&gt;=3.8" data-gpg-sig="false" >flask-3.0.2-py3-none-any.whl</a>
      <br/>
      <a href="3.0.2/flask-3.0.2.tar.gz#sha256=822c03f4b799204250a7ee84b1eddc40665395333973dfb9deebfe425fefcb7d" data-requires-python="&gt;=3.8" data-gpg-sig="false" >flask-3.0.2.tar.gz</a>
      <br/>
  </body>
</html>

I also tried with -H 'Accept: application/vnd.pypi.simple.latest+json' but got HTTP/2 406
so it looks like AWS codeartifact only does the HTML format (or I'm requesting it wrong)

@charliermarsh
Copy link
Member

Thanks, that's great! I assume there's no <base> tag in the file?

@zanieb
Copy link
Member

zanieb commented Feb 16, 2024

Exploring in #1413

@zanieb zanieb mentioned this issue Feb 16, 2024
Merged
@charliermarsh charliermarsh removed their assignment Feb 16, 2024
@raylu
Copy link
Author

raylu commented Feb 16, 2024

yeah, no <base> (the only thing I cut out is a bunch of versions in the middle)

@charliermarsh
Copy link
Member

👍 I think we've pinpointed the problem, thank you!

@zanieb
Copy link
Member

zanieb commented Feb 16, 2024

Should be fixed in 829a1ed

zanieb added a commit that referenced this issue Feb 16, 2024
@zanieb
Fix broken URLs parsed from relative paths in registries (#1413)
0bfce35 
Closes #1388

Fixes incorrect handling of relative paths returned by indexes without
an explicit `<base>`.

`Url.join` will drop the last segment in an url e.g. `http://foo/bar` ->
`http://foo/baz` if there is not a trailing slash but what we want is
`http://foo/bar/baz`. We don't add the trailing `/` in
`base_url_join_relative` because flat indexes are `http://foo/bar.html`
and we _want_ `bar.html` to be replaced.
@inigohidalgo inigohidalgo mentioned this issue Feb 16, 2024
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working registry Related to package indexes and registries
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Fix broken URLs parsed from relative paths in registries astral-sh/uv
4 participants
@jasonprado @raylu @charliermarsh @zanieb