2FA/API tokens: staging/testing rollout

[brainwane]

What's the problem this feature will solve?
To finish #996 (see #5567), we need to test MFA with real users on real packages; asking them to spin up dev environments is too hard and won't help multi-maintainer projects reason well about what MFA policies they want to set up.

Describe the solution you'd like
My tentative suggestion is:

• We first roll out MFA on test.pypi.org and publicize it to project maintainers for a designated 2-week testing period, and we allot extra time for dealing with support requests during that time
• We then roll it out to pypi.org and announce it on the PyPI announcement email list, PSF blog, etc.

Additional context

• If things go awry during initial testing, is there any chance we will need to wipe tokens from users' accounts?
• Are there particular categories of user we need to make sure we get in the beta test? Ideas that come to mind for me: Windows, mobile, users of particular TOTP implementations and U2F implementations, people on very slow connections, people who habitually block a lot of cookies/JS, maintainers who maintain 20+ projects, people with very old and weird PyPI accounts (e.g., we do not have verified email for them, their passwords do not adhere to current policy), users with upwards of 10 MFA methods they want to add to their accounts, multi-maintainer projects, multi-owner projects, organizations where users share an auth token within a group.
• Is test.pypi.org the right place for this? Should we spin up some other instance for this particular kind of testing, since maintainers do use Test PyPI for real uploads (of packages that need testing)?

Cc @ewdurbin .

[di]

If we want to roll out to TestPyPI first, we'll probably want to feature-flag this feature, which would be a bit of work.

It might be preferable to do a true "beta" on PyPI instead, with a limited group of users. This could just be a boolean field on the User model that enables the feature, which we set in the admin UI. And we could just skip this check and enable it for all TestPyPI users, and eventually remove it once we've rolled everything out.

This could allow us to do a timeline like:

• 1 week period on TestPyPI for everyone
• 2 week beta on PyPI for select users
• Open to all users

If things go awry during initial testing, is there any chance we will need to wipe tokens from users' accounts?

Possibly? I think we should be explicit in our "beta" announcement that this might happen, so users aren't surprised.

Are there particular categories of user we need to make sure we get in the beta test?

I think all the groups you listed would be ideal.

people who habitually block a lot of cookies/JS

This is a good point -- it looks pretty likely that our dependency on qrious means that this won't work for users blocking JavaScript. We probably need to test this and add a fallback, or maybe revisit this dependency. (cc @woodruffw)

Is test.pypi.org the right place for this?

I think it is. We don't really claim that TestPyPI should be depended on for much.

[ewdurbin]

If we want to roll out to TestPyPI first, we'll probably want to feature-flag this feature, which would be a bit of work.

It might be preferable to do a true "beta" on PyPI instead, with a limited group of users. This could just be a boolean field on the User model that enables the feature, which we set in the admin UI. And we could just skip this check and enable it for all TestPyPI users, and eventually remove it once we've rolled everything out.

This could allow us to do a timeline like:

• 1 week period on TestPyPI for everyone
• 2 week beta on PyPI for select users
• Open to all users

+1 to this approach.

[woodruffw]

This is a good point -- it looks pretty likely that our dependency on qrious means that this won't work for users blocking JavaScript. We probably need to test this and add a fallback, or maybe revisit this dependency.

It should be relatively easy to add a fallback here: we can display the otpauth:// URL directly, and the user can copy-and-paste that into their TOTP application or use their QR generator of choice. Right now that URL is only available via the aria-label of the canvas that we render the QR in, but that wouldn't be hard to change at all. (Sidenote: I'm not a frontend developer at all, so I have no idea how to check whether a user has disabled Javascript. Is it just <noscript>?)

This could allow us to do a timeline like:

• 1 week period on TestPyPI for everyone
• 2 week beta on PyPI for select users
• Open to all users

This looks good to me!

[brainwane]

@nlhkabu - does this feel ok to you? If so then I will declare consensus and start planning this, writing sets of "hey come test this" announcement text, etc.

[nlhkabu]

Hi @brainwane - sounds very reasonable to me.

I'd like to run some direct user tests alongside open testing - sessions where I am able to observe users and understand anywhere they get stuck.

I've set up a Google form for registering users for this. @brainwane would you be able to:

1. review the form
2. add a link to it in your announcement

To be clear, I don't think that manual user testing should block the launch of 2FA, unless I find any major technical issues that are not raised by open beta testing.

[di]

@nlhkabu One small comment on the form: might want to ask for their PyPI username as well, so we can enable 2FA for them during the private beta.

[di]

#5567 is close to being done. I'm going to plan on merging it on or before Friday, May 3rd, which would give us the following timeline:

• On or before 5/3 - Merge 2FA (TOTP) support #5567
• 5/3 - enable 2FA across all of TestPyPI, send announcements
• 5/3-5/20 - private beta on PyPI, add users as necessary
• 5/20 - enable for all PyPI users

[brainwane]

OK, draft announcements:

For distutils-sig/discuss.python.org and other longform places:

Subject line: PyPI two-factor auth (2FA) trial May 3-20

Dear PyPI users:

To increase the security of PyPI downloads, we're beginning to introduce two-factor authentication (2FA) as a login security option, and want project maintainers and owners to start testing it.

Starting this Friday, May 3rd, you'll be able to use 2FA on Test PyPI. And if you'd like to try 2FA on official PyPI, please fill out this Google form so we can invite you to the private beta, which we plan to hold 3-20 May.

PyPI currently supports a single 2FA method: generating a code through a Time-based One-time Password (TOTP) application. After you set up 2FA on your PyPI account, then you must provide a TOTP (along with your username and password) to log in. Therefore, to use 2FA on PyPI, you'll need to provision an application (usually a mobile phone app) in order to generate authentication codes; our our testing wiki page gives you suggestions and pointers.

This change only applies to the login step, not package uploads.

More details at our testing wiki page.

During this testing period, if things go awry, there's a chance we will need to wipe tokens from users' accounts, so if you choose to try it, please be forewarned. We suggest you make sure you have a PyPI-verified email address on your user account before trying the feature, to make potential account recovery smoother.

And please let us know if you run into glitches.

We expect to end this testing period on May 20th, then enable the optional 2FA feature for all PyPI users, and move on to working on WebAuthn support.

Thanks to the Open Technology Fund for funding this work. More progress reports at the Packaging Working Group's wiki page.

-the PyPI team

for pypi-announce:

Subject line: PyPI two-factor auth (2FA) trial May 3-20

Dear PyPI users:

To increase the security of PyPI downloads, we're beginning to introduce two-factor authentication (2FA) as a login security option, and want project maintainers and owners to start testing it.

Starting this Friday, May 3rd, you'll be able to use 2FA on Test PyPI. And if you'd like to try 2FA on official PyPI, please fill out this Google form so we can invite you to the private beta, which we plan to hold 3-20 May.

More details at our testing wiki page.

We expect to end this testing period on May 20th, then enable the 2FA feature for all PyPI users, and move on to working on WebAuthn support.

Thanks to the Open Technology Fund for funding this work. More progress reports at our wiki page.

-the PyPI team

[brainwane]

(I need to add a note calling out that this doesn’t affect the upload endpoint currently.)

[nlhkabu]

Looks good to me @brainwane - thank you. You can probably change this sentence to be shorter:

If you set up 2FA on your PyPI account, you must provide a second method of identity verification (other than your username and password) to log in.

[mlissner]

First:

And if you'd like to try 2FA on official PyPI, please fill out this Google form so we can invite you to the private beta, which we plan to hold 3-20 May.

Should this say that it's really more of a user test? It's not just joining a beta, the plan is to sit with these folks for an hour, right? I eagerly clicked the link, then was disappointed since I didn't have the hour.

Second: Is there a plan to require 2FA eventually, or, if there isn't a plan, is it worth threatening that that might happen eventually? I think it makes sense to have 2FA for software repos like PyPi, but I know requiring it would be difficult. Maybe there's a middle ground to start pushing for.

[di]

@mlissner You don't need to be available for that hour to be in the beta, just respond "no". :)

[mlissner]

I'm like 10% sure that changed? Anyway, good!

[brainwane]

When we flipped the switch for this feature on Test PyPI & canon PyPI, we did not initially update the user model so that existing Test PyPI accounts had the feature on. Ernest (I believe) ran some SQL to update all the Test PyPI accounts yesterday and turn on the flag.

Right now, new Test PyPI accounts (as in, created since that time yesterday) do not have the flag set so they don't see 2FA in their account settings.

I'd like @ewdurbin to run the "turn this on for everybody on Test PyPI" SQL again just to get that taken care of, then figure out what to do next.

I think @di told me that turning it on by default for all Test PyPI accounts would also turn it on for all canon PyPI accounts (does this mean we ought to have a better feature flag system in general, or just for this feature?) (and I presume that constraint would also stop us from doing something to fix this like: verifying the email address triggers turning on 2FA support).