Discussion of long term methods of addressing 2FA Account Recovery Requests

[ewdurbin]

Opening this as a follow on from #679 for @pypa/warehouse-team to discuss possible methods for addressing 2FA requests, which were honestly an overwhelming task.

There were some suggestions in the thread that may be worth bringing over to this discussion, but to start things off

Here is the basic flow I use:

1. Look up User in PyPI admin
2. Ensure that the report and status of their 2FA enrollment match up. If alternative 2FA method exists (TOTP, WebAuthn, or Recovery Codes) notify user to use one of those.
3. Determine what projects the User has Owner/Maintainer status on. If no projects are associated with the User, immediately perform 2FA reset.
4. Determine how many unique public source code repositories are associated with the Users Projects based on releases of the Project from before the request. For each security boundary such as a GitHub org or code hosting service.
5. Send email template below requesting that a branch with a randomly generated secret is pushed to a repository in each security boundary.
6. Once the user has responded to confirm that the branches have been pushed, go through and verify that each branch exists on the source code repository from the metadata of a release from before the request.
7. Issue 2FA reset and password reset for the issue.

Template:

SUBJECT: PyPI Account Recovery Request: USERNAME

Hello,

An account recovery request was filed at GHLINK for the PyPI username USERNAME which has this address (EMAIL) as it’s primary email.

In order to verify the request, please push a branch with the name TOKEN to the public source code repository for each of the following projects then respond to this email:

- PROJECT
- PROJECT

The branch does not need to include any commits or changes and can be deleted after it’s been verified.

After that, I’ll reset 2FA for your account and issue a password reset.

-Ee W. Durbin III
Director of Infrastructure
Python Software Foundation

[ewjoachim]

Thank you Ee for taking the time to bring this up !

I'd like to summarize publicly the ideas that I suggested when we discussed this issue earlier:

• In your process, steps 1, 2, 3a, 4, 5 and 6 can be done by non-admin moderator users, with enough documentation so that an admin can do 3b and/or 7. We could also allow non-admin users to perform 3b (disabling 2fa when user has not uploaded anything). This could be a way to spread the load between more people, but there's not many active moderators so it wouldn't help by much.
• While originally, there was no recovery codes, there are now, which means we could decide that it's user's responsibility to make sure they do have the recovery codes, and explicitly end the GitHub procedure for cases where 2FA was lost. Identically, Warehouse supports multiple emails, and user have the necessary tools to ensure that they will not be locked out even if they lose access to one email address. This would involve changes in wording and maybe specific communications, maybe periodic emails or something. @di started exploring it for 2FA in Encourage generation of 2FA recovery codes warehouse#8897

Of course, we can imagine exceptions in cases where not making one would have an effect on the Python community at large but it's not that simple. The more popular a package is, the more breaking it would be if its maintainers couldn't upload it anymore and had to switch to a new name, but at the same time, the more attractive it would be for hackers and the more catastrophic handing it over to the wrong person would be. Has the case ever happened of all owners of a popular package loosing access to their PyPI account ?

With solution 2 (the one I'd favor), we're making Warehouse a little bit less user friendly, but more secure, in that we're removing official ways to bypass our own security measures. I'd find this reassuring.

[di]

Send email template below requesting that a branch with a randomly generated secret is pushed to a repository in each security boundary.

Does this need to be a secret or an email? Otherwise, it seems like everything except for step 7 could be done in the issue thread by moderators, at which point it could be transferred to admins for reset (similar to PEP 541 requests).

[ewdurbin]

Send email template below requesting that a branch with a randomly generated secret is pushed to a repository in each security boundary.

Does this need to be a secret or an email? Otherwise, it seems like everything except for step 7 could be done in the issue thread by moderators, at which point it could be transferred to admins for reset (similar to PEP 541 requests).

It is delivered to the primary email address on the account via email to confirm that the person requesting here controls the account there. If we don’t perform the request via email it doesn’t offer the chance for a invalid request to be flagged by the real owner.

[ewjoachim]

That being said, if we want to continue down this path, we could make a one-click button in the admin that sends the email and adds the secret string to the security logs that are displayed on the page. And add a saved reply in GitHub so that we can quickly answer on the ticket.

[ewdurbin]

import secrets
import requests

from warehouse.accounts.models import User

admin = db.query(User).filter(User.username == "ewdurbin").one()
ip_address = "[redacted]"

from sqlalchemy import func, literal

from warehouse.packaging.models import Project, ProhibitedProjectName
from warehouse.utils.project import remove_project


class FakeReq:
    user = admin
    db = db
    remote_addr = ip_address


request = FakeReq()

TMPL = """
To: {primary_email}
Cc: {cc_email}

PyPI Account Recovery Request: {username}

Hello,

An account recovery request was filed at {ghlink} for the PyPI username {username} which has this address ({primary_email}) as it’s primary email.

In order to verify the request, please push a branch with the name {token} to the public source code repository for the {project_name} project and respond to this email.

The branch does not need to include any commits or changes and can be deleted after it’s been verified.

After that, I’ll reset 2FA for your account and issue a password reset.

-Ee Durbin
Director of Infrastructure
Python Software Foundation
"""

TMPL_NO_PUB = """
To: {primary_email}
Cc: {cc_email}

PyPI Account Recovery Request: {username}

Hello,

An account recovery request was filed at {ghlink} for the PyPI username {username} which has this address ({primary_email}) as it’s primary email.

Because there are no projects with public source code repositories associated with your account, we are unable to utilize our normal verification process to proceed.

Please respond to this email confirming intent to reset 2FA.

After that, and a 7 day waiting period, I’ll reset 2FA for your account and issue a password reset.

-Ee Durbin
Director of Infrastructure
Python Software Foundation
"""

def generate():
    username = input('username: ')
    ghlink = input('GitHub Issue Link: ')
    user = db.query(User).filter(User.username == username).one()
    if len(user.projects) > 0:
        options = {}
        for i, project in enumerate([p for p in user.projects if len(p.releases)>0]):
            options[i] = project
            print(f'{i}\t{project.name}\t{len(project.releases)}\t{str(project.latest_version[1])}')
            for k, v in project.releases[0].urls.items():
                try:
                    status_code = requests.get(v).status_code
                except:
                    status_code = 404
                print(f'\t\t- {k}: {v} {status_code}')
        if len(options) > 0:
            try:
                choice = options[int(input('choose a project: '))]
            except ValueError:
                choice = None
            token = secrets.token_urlsafe().replace('-','').replace('_','')[:16]
            primary_email = user.primary_email.email
            cc_email = ', '.join([email.email for email in user.emails if not email.primary])
            if choice is not None:
                print(TMPL.format(primary_email=primary_email, cc_email=cc_email, username=user.username, ghlink=ghlink, token=token, project_name=choice.name))
            else:
                print(TMPL_NO_PUB.format(primary_email=primary_email, cc_email=cc_email, username=user.username, ghlink=ghlink, token=token, project_name=None))
        else:
            print("no projects with releases!")
    else:
        print("no projects!")

g = generate

[ewdurbin]

Just dumping the tooling I use to generate the email templates and tokens, in case anyone wanted to implement an admin button.

[pradyunsg]

Until we have the admin button implemented, we can also start making progress on the backlog today by having a workflow that requires less admin time and can be collaboratively used. I propose that:

• We add a "Pending password reset" column to the Account Recovery board.
• Give all moderators and admins access to a shared inbox for an @pypi.org email, to send and view recovery emails.

With that, the process is basically:

flowchart TD
A[[New ticket]] --> B{{user exists}}
B --> |no| close
B --> |yes| C{{reported status matches 2FA enrollment}}
C --> |no| close
C --> |yes| D{{Does the user have other 2FA methods?}}
D --> |yes| E[Ask the user to use the alternative methods.]
E --> |has access| close
E --> |no access| F
D --> |no| F{{Does the user have any projects registered?}}
F --> |no| L[Make a comment\nthat no projects are registered]
L --> admin-reset[Mark for an admin to reset the password]
F --> |yes| G("Determine all 'security boundaries'\n(eg: GitHub org or code hosting service)\nbased on public source code repositories\n associated with the Users' Projects, from releases\nof the Project from before the request")
G --> H[For each security boundary,\nrequest a branch to be pushed to a repository\nwith a unique randomly generated secret\nvia the registered email]
H --> |user confirms that they've pushed branches| J[Validate that the branches are pushed]
J --> K[Make a comment\nlisting the validated security boundaries]
K --> admin-reset
admin-reset --> resolved
style close fill:#f99
style admin-reset fill:#ff9
style resolved fill:#9f9

Loading

(yay, mermaid support on GitHub)

The only thing that needs the admin bit on PyPI is the admin-reset, who should be able to action on the issue based on the final comment in the issue. If we can't have a shared inbox, that'd make this less appealing. OTOH, we can work around that by sharing the tokens on one of our shared moderator+admin non-public discussion forums, or by having the moderator/admin who sent the verification emails assign themselves on the issue making it clear who knows the verification information and is responsible for validating it.

[pradyunsg]

I’ve gone ahead and pinned this for visibility. It’s still a locked issue.

[ewdurbin]

The PSF has hired a full-time support role, and 2FA/Account Recovery request are now being processed consistently. Closing.