Implement PEP 592: Yanked Releases #5838

dstufft · 2019-05-12T17:24:04Z

Update the Release model to support marking a release as yanked.
Update the simple API so that it exposes the data-yanked attribute.
~~Provide an API for yanking/unyanking a release.~~
Tests

nlhkabu · 2019-09-15T11:06:10Z

Note:
If / when this PR is re-reviewed, we will need to ensure that the templates are appropriately translated.

brainwane · 2020-01-27T17:21:57Z

This feature would be helpful in improving pip's test automation so it would be great if you could prioritize finishing this WIP.

dstufft · 2020-01-27T17:36:59Z

I am unlikely to find time in the interim to finish this, though I think we could deploy this without an API and just have it be a UI only feature for now.

brainwane · 2020-04-02T21:41:52Z

@ewdurbin @di @dstufft hey - as Ernest and @pradyunsg and I talked about in a meeting about the pip resolver work, getting this implemented might help us smooth the second-order effects of the resolver rollout, so if you could help get this finished and merged, we'd appreciate that!

di · 2020-04-02T21:50:08Z

@nlhkabu, could you take a look at 1d7824d if you have a second? Without this change, the callout blocks were appearing above the modals.

nlhkabu · 2020-04-03T18:55:57Z

@di 1d7824d looks fine to me.

I have updated the copy in a couple of places (fixed formatting, typo), and changed the icon:

Hope this is ok @dstufft

ewdurbin · 2020-04-03T19:35:47Z

The value of yanked be added to the JSON documents for projects/releases, rather than hiding them.

dstufft

I think my biggest thing is I'm not sure that we want yanked to function as hiding releases from the UI (but maybe we do, I'm not sure?).

My gut instinct tells me we probably want this behavior:

On the unversioned URL for a project, we won't pick a yanked version, and would instead act as if there were no versions.
- One could argue that it's better to just display a yanked version as a last ditch effort to display something, I'm not sure though?
On a versioned URL, yanked releases will show up normally, except there is some marker on the page to indict this release has been yanked (likely some sort of red warning banner?).
On the list of all versions, the yanked releases still show up, but are marked in some fashion to show that they have been yanked (greyed out to demphasize? red? idk).

Fundamentally, I don't think we want to reimplement the hidden release option from legacy PyPI, this is largely about making it so that people can prevent a version from being installed unless absolutely needed, it isn't about hiding it like it never existed and I think that being able to introspect it in the UI still is still useful.

dstufft · 2020-04-03T20:08:44Z

warehouse/packaging/models.py

-                    (Release.project == self)
-                    & (Release.canonical_version == canonical_version)
+                    Release.project == self,
+                    Release.yanked.is_(False),


Do we want to do this? This will make it so yanked versions can't be viewed online at all right?

di · 2020-04-03T20:22:46Z

I see where you're coming from @dstufft. My thought is that if we want people to use yanks instead of deleting releases, we need to make yanks behave just like deleting releases, except you can still pin to them. People delete releases for a lot of reasons: bad long_description, typos, etc. Ideally in those cases, maintainers can still "remove" a release from the public eye, but anyone already using it can continue to pin to it.

Another issue this helps avoid is requiring average users to understand what a yanked release even is. If a user is looking at project page for some-project and they see that the latest release, 1.0.0 is marked as "yanked" with a big red box, but it's still there, they're probably not going to understand what's happening when they pip install some-project and get version 0.9.9 instead of 1.0.0. If the project page just shows the un-yanked releases though, nothing changes here, and they never need to know that 1.0.0 even exists.

dstufft · 2020-04-03T20:30:26Z

See I almost consider it the other way, people are still using versions of pip that don't support yanked, so they're likely to be very confused if they pip install some-project with a version that supports it, and then a version that doesn't support it and get different results. I presume most of them are going to go to PyPI at that point and then will start looking around trying to figure out why they're getting different behavior, and keeping the versions all in the UI, but appropriately marking the yanked ones as yanked will help them debug what is happening.

I also don't think that the use case for yanked needs to behave "just as if it was deleted". I do think that for the unversioned URL we should certainly not select a yanked version (possibly unless there ONLY exist yanked versions?), so someone would have to explictly seek out a yanked version in order to even see one exists.

If we do that, we might possibly want to mark a yanked release's mage as noindex as well, so that it won't show up in google's search either.

di · 2020-04-03T20:39:53Z

Hmm, I see your point, and I think you're probably right. I hadn't considered the difference in behaviors between clients that do/don't support yanked releases.

di · 2020-04-04T19:31:20Z

@dstufft I think I addressed all your feedback here. Yanked releases will show up everywhere except:

Project.latest_version will not return a yanked release as the latest
The project detail page will not show a yanked release as the latest
/pypi/<project_name>/json will not use a yanked release as the latest
Search reindexing will not include yanked releases

warehouse/templates/legacy/api/simple/detail.html

warehouse/templates/pages/help.html

pradyunsg

I feel like it'd be nice to require users to put in something as "reason for yanking" (picking from a bunch of fixed choices, for example) since installers can present that information to the user (pip prints a warning containing the message), and having it be compulsory from the start would help avoid a potential transition step if we add it later.

Then again, it's much easier to do a smoother transition on web application than a command line application, so it's perfectly fine if y'all think it's OK to not start with that... :)

warehouse/templates/email/unyanked-project-release/body.html

warehouse/templates/email/unyanked-project-release/body.txt

dstufft

I can't approve this because it's my own PR, but we'll just pretend I did.

dstufft · 2020-04-20T17:18:25Z

warehouse/templates/manage/release.html

+      This release will still be installable for users pinning to this exact version, e.g. when using <code>{{ project_name }}=={{ version }}</code>.
+    {% endtrans %}
+    {% trans href="https://www.python.org/dev/peps/pep-0592/" %}
+      For more information, see <a href="{{ href }}">PEP 592</a>.


This doesn't need to block this PR, but I've wondered if our propensity for linking to PEPs is the best user experience we can provide here, or if we should figure out a real documentation pipeline for Warehouse (not really counting what exists now since it's primarily developer facing documentation, not user facing) and start moving to that.

I think I agree, we are sort of hitting the limits of what we can document inline and in /help.

I filed #7817 - thanks for the idea, @dstufft

This allows yanked releases to show up in the UI with the exception of: * Project.latest_version will not return a yanked release as the latest * The project detail page will not show a yanked release as the latest * /pypi/<project_name>/json will not use a yanked release as the latest * Search reindexing will not include yanked releases

Co-Authored-By: Pradyun Gedam <[email protected]>

docs/api-reference/json.rst

pradyunsg · 2020-04-22T22:43:42Z

YAY! Thanks for all of your work on this @nlhkabu @di @dstufft @ewdurbin! ^>^

dstufft mentioned this pull request May 12, 2019

Implement PEP 592 -- Yanking of Releases/Files #5837

Closed

brainwane mentioned this pull request Jun 12, 2019

Increasing pip's & PyPI's metadata strictness pypa/packaging-problems#264

Open

brainwane mentioned this pull request Jun 22, 2019

Add support for yanked files from a repository (PEP 592) pypa/pip#6633

Closed

pradyunsg mentioned this pull request Jan 21, 2020

Rename pip._internal.distributions.{source -> sdist} pypa/pip#7621

Merged

di force-pushed the yanked branch from f5e1449 to 5293058 Compare March 13, 2020 21:14

di changed the title ~~[WIP] Implement PEP 592: Yanked Releases~~ Implement PEP 592: Yanked Releases Mar 13, 2020

di requested a review from ewdurbin March 13, 2020 21:15

di force-pushed the yanked branch from ff7709a to 4cad002 Compare April 2, 2020 21:48

di force-pushed the yanked branch from ae0c843 to 9b264ad Compare April 3, 2020 19:32

dstufft commented Apr 3, 2020

View reviewed changes

di force-pushed the yanked branch 2 times, most recently from 468f70a to 9fa75ad Compare April 4, 2020 19:28

dstufft commented Apr 4, 2020

View reviewed changes

warehouse/templates/legacy/api/simple/detail.html Outdated Show resolved Hide resolved

warehouse/templates/pages/help.html Outdated Show resolved Hide resolved

pradyunsg reviewed Apr 4, 2020

View reviewed changes

di force-pushed the yanked branch 2 times, most recently from ff61e7a to 63334ac Compare April 6, 2020 14:36

di mentioned this pull request Apr 13, 2020

Package names that replace stdlib names #7789

Open

dstufft commented Apr 20, 2020

View reviewed changes

brainwane mentioned this pull request Apr 20, 2020

improve pipeline for user-facing feature-specific documentation #7817

Open

di and others added 19 commits April 20, 2020 12:47

Tests

a12482d

Updates from pypi#7689

9501a8a

Update icon and copy

101abee

Update un-yank icon as well

6e5eac0

Update migration

eb54532

Update email templates to remove translations

d99fc09

Only reindex non-yanked releases

fd2102f

Don't filter yanked releases from JSON API

9d863ed

Show yanked status in JSON API

9497b35

Only show 'Releases' header when there are releases

128a826

Show badge for yanked release

790996d

Add warning label on detail page for yanked releases

31a588a

Add help section on yanked releases

27c5538

Update PEP number on help page

6bdb3a3

Update tests

d6b8ce8

Update conditional in simple/detail.html

cfce65a

Apply suggestions from code review

32a295a

Co-Authored-By: Pradyun Gedam <[email protected]>

Update translations

295873f

di force-pushed the yanked branch from 63334ac to 295873f Compare April 20, 2020 17:48

Update migration

b6894f7

ewdurbin approved these changes Apr 21, 2020

View reviewed changes

di added 2 commits April 22, 2020 15:19

Merge branch 'master' into yanked

2d4af02

Merge branch 'master' into yanked

a0e79cb

merwok reviewed Apr 22, 2020

View reviewed changes

docs/api-reference/json.rst Show resolved Hide resolved

Add missing commas

f25af63

di merged commit 69ce3dd into pypi:master Apr 22, 2020

brainwane mentioned this pull request Apr 23, 2020

Add mention of PEP 592 for simple repo API spec pypa/packaging.python.org#723

Merged

pradyunsg mentioned this pull request Jan 31, 2022

[DEPRECATION] Moving away from html5lib to html.parser pypa/pip#10825

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Implement PEP 592: Yanked Releases #5838

Implement PEP 592: Yanked Releases #5838

dstufft commented May 12, 2019 •

edited by di

Loading

nlhkabu commented Sep 15, 2019

brainwane commented Jan 27, 2020

dstufft commented Jan 27, 2020

brainwane commented Apr 2, 2020

di commented Apr 2, 2020

nlhkabu commented Apr 3, 2020 •

edited

Loading

ewdurbin commented Apr 3, 2020

dstufft left a comment

dstufft Apr 3, 2020

di commented Apr 3, 2020

dstufft commented Apr 3, 2020

di commented Apr 3, 2020

di commented Apr 4, 2020

pradyunsg left a comment •

edited

Loading

dstufft left a comment

dstufft Apr 20, 2020

di Apr 20, 2020

brainwane Apr 20, 2020

pradyunsg commented Apr 22, 2020

Implement PEP 592: Yanked Releases #5838

Implement PEP 592: Yanked Releases #5838

Conversation

dstufft commented May 12, 2019 • edited by di Loading

nlhkabu commented Sep 15, 2019

brainwane commented Jan 27, 2020

dstufft commented Jan 27, 2020

brainwane commented Apr 2, 2020

di commented Apr 2, 2020

nlhkabu commented Apr 3, 2020 • edited Loading

ewdurbin commented Apr 3, 2020

dstufft left a comment

Choose a reason for hiding this comment

dstufft Apr 3, 2020

Choose a reason for hiding this comment

di commented Apr 3, 2020

dstufft commented Apr 3, 2020

di commented Apr 3, 2020

di commented Apr 4, 2020

pradyunsg left a comment • edited Loading

Choose a reason for hiding this comment

dstufft left a comment

Choose a reason for hiding this comment

dstufft Apr 20, 2020

Choose a reason for hiding this comment

di Apr 20, 2020

Choose a reason for hiding this comment

brainwane Apr 20, 2020

Choose a reason for hiding this comment

pradyunsg commented Apr 22, 2020

dstufft commented May 12, 2019 •

edited by di

Loading

nlhkabu commented Apr 3, 2020 •

edited

Loading

pradyunsg left a comment •

edited

Loading