Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Clean up gensim project on PyPI #3237

Closed
mpenkov opened this issue Sep 17, 2021 · 6 comments
Closed

Clean up gensim project on PyPI #3237

mpenkov opened this issue Sep 17, 2021 · 6 comments
Assignees
Labels
housekeeping internal tasks and processes impact HIGH Show-stopper for affected users

Comments

@mpenkov
Copy link
Collaborator

mpenkov commented Sep 17, 2021

We've hit the 10GB limit for projects on PyPI. This is preventing some of the 4.1.2 wheels from being uploaded.

https://pypi.org/help/#project-size-limit

We need to clean up the project area before we can upload new wheels.

@mpenkov mpenkov added housekeeping internal tasks and processes impact HIGH Show-stopper for affected users labels Sep 17, 2021
@mpenkov
Copy link
Collaborator Author

mpenkov commented Sep 17, 2021

I think the safest thing to do immediately is to delete all 4.1.0 and 4.1.1 wheels because they are buggy. That would allow the 4.1.2 wheels to be uploaded properly.

I'd do this myself, but I don't have the permissions.

@piskvorky
Copy link
Owner

piskvorky commented Sep 17, 2021

Oh wow. That's weird, where did 10 GB go? I'll have a look. Thanks for reporting!

EDIT: OK, I see. We build wheels for 4 platforms x 4 Python versions = 16 wheels per release. Each wheel is 25 MB (why so large?), so that's > 400 MB per release! I can see 10 GB disappearing quickly.

How do other projects with extensive wheel support (scikit-learn?) solve this?

@piskvorky
Copy link
Owner

piskvorky commented Sep 17, 2021

@mpenkov I requested a quota increase on PyPI: pypi/support#1315

I also deleted 4.0.0rc1 and 4.0.0b0 to free up some PyPI space now. Hopefully noone was using those…

@mpenkov
Copy link
Collaborator Author

mpenkov commented Sep 17, 2021

OK, thank you.

I uploaded the remaining wheels, so the 4.1.2 release contains all the wheels that it needs now, including aarm64.

@piskvorky
Copy link
Owner

piskvorky commented Sep 17, 2021

Are you sure? Github still reports 4.1.1 as the latest:
https://github.com/RaRe-Technologies/gensim/releases/

PyPI seems fine, thanks a lot!

@mpenkov
Copy link
Collaborator Author

mpenkov commented Sep 18, 2021

Not sure why it was missing, but I manually pushed the tag and updated the release, so it's there now.

https://github.com/RaRe-Technologies/gensim/releases/tag/4.1.2

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
housekeeping internal tasks and processes impact HIGH Show-stopper for affected users
Projects
None yet
Development

No branches or pull requests

2 participants