Add support for API keys

[edmorley]

A scary number of people embed their PyPI username and password in their Travis config (using Travis encrypted variables), to enable automatic releases for certain branches (Travis even has a guide for it).

In addition, the packaging docs example encourages users to save their password in plaintext on disk in their .pypirc (they can of course use twine's password prompting, but I wonder how many read that far, rather than just copy the example verbatim?)

Whilst in an ideal world credentials of any form wouldn't be saved unencrypted to disk (or given to a third-party such as Travis) and instead users prompted every time - I don't think this is realistic in practice.

API keys would offer the following advantages:

1. Higher-entropy credentials that are guaranteed to have not been reused on multiple sites.
2. The ability to give the API key a smaller permissions scope than that of the owner's username/password. For example an API key would not be permitted to change a user's listed GPG key or in the future, their 2FA settings. Or an API key could be limited to a specific package.
3. Since this would be separate from the existing username/password auth, a signing based approach (eg HMAC) could be used, without breaking older clients. This would ensure that if a connection was MiTMed (eg due to a protocol or client exploit), the API key itself would still remain secure.
4. Eventually support could be dropped for the password field in .pypirc, leaving a much safer choice between password prompting every time, or creating an API key that could be saved to disk.
5. If/when support is added for 2FA, users who need to automate PyPI uploads won't have to forgo 2FA for their whole account. They could instead choose to just create a 2FA-circumventing API key for just the one package that needs uploads in automation.

Many thanks :-)

(I've filed this against warehouse since I'm presuming this is beyond the scope of maintenance-only changes being made to the old PyPI codebase)

[dstufft]

This is another thing I've been wanting to do, but is likely a post-launch task. I'm a bit on the fence of how to exactly handle them, but one option I've been thinking about is instead of API keys, using client certificates for TLS which would give built in support for a signing based approach, high entropy, allow it to be used for all uploads (could store it password protected typically, and just offer a password-less option for automation), and expiration of the token.

One problem with this is that it would mean we can't route uploads through our CDN, however uploads don't really gain anything by going through the CDN (and in fact, it's a bit harmful since uploads need a longer timeout than normal requests, we're forced to have high, 20+ second timeouts on upload).

I've also considered something like OAuth here instead of just an API Key which would solve a lot of these problems as well, in addition to making it possible to securely grant other projects the ability to modify just one package (or one scope inside of that package).

There's also the likely future signing tool, TUF, where we could just enforce that all uploads must be signed by a valid key for that author, and use that key as the authentication.

A lot of different options here, which is another reason why it's likely a post-launch task :)

[lukesneeringer]

I really really want to get my PyPI information out of CI. At the risk of responding to a years-old thread...I want to volunteer to do this work (as well as #996). :-)

At this point, Warehouse is launched (albeit in beta), and the legacy upload endpoint is deprecated. I assert it would be reasonable to add this, although others who have actually been thinking about this for more than a few days might know better than I (so feel free to chime in and tell me!).

Before reading @dstufft's comments above, my thought was, "Implement API keys", but I have nothing against the idea of certificates.

Here is what I think is needed (sub out "key" below with "certificate" if we go that route):

• The ability to create a new key.
• The ability to invalidate a key.
• The ability to see when each existing key was last used.
• The ability to scope a key's permissions to only be able to publish particular packages.

Would anyone have any objection to my taking some time to scope this out further, with an eye to getting the work in soon-ish? (Since I am new around here, it would probably require some review cycles from @ewdurbin, @dstufft, etc.)

[brainwane]

Thanks for your note, @lukesneeringer, and sorry for the slow response! Thank you for volunteering to do this work!

As I think you know, but just for context for future folks finding this discussion, the folks working on Warehouse have gotten funding to concentrate on improving and deploying Warehouse, and have kicked off work towards our development roadmap -- the most urgent task is to improve Warehouse to the point where we can redirect pypi.python.org to pypi.org so the site is more sustainable and reliable.

So that's what Ernest, Dustin, and Nicole have been concentrating on and will be concentrating on for the next few months. But I'm putting your suggestion on the agenda for a sync-up meeting we'll have tomorrow, and we'll have more thoughts for you then.

Also, Ernest wants to help folks get started as new Warehouse contributors, and has 30-minute 1:1 slots available each week, in case that's something you, or someone you know, is interested in doing.

Thanks again! Talk with you soon.

[lukesneeringer]

But I'm putting your suggestion on the agenda for a sync-up meeting we'll have tomorrow, and we'll have more thoughts for you then.

Sounds good.

My guess is that this is probably work that can be done in parallel to the Warehouse improvements. The trick would be that the keys would not work on legacy PyPI, and therefore anyone using the legacy URL would not be able to use them. (However, I suppose it might be the case that review cycles or whatnot would not be available.)

Also, Ernest wants to help folks get started as new Warehouse contributors, and has 30-minute 1:1 slots available each week, in case that's something you, or someone you know, is interested in doing.

Yep -- we already did that. :-)

[brainwane]

@lukesneeringer Oh great, glad you and Ernest have already started working together!

In our meeting today we said "yay" about you working on this! Please go ahead and start scoping it out and let us know your thought process as you work. I could imagine you finding the SimplySecure resources useful on a UX level.

We also decided that, as a new feature, this belongs in a future milestone. But we will do our level best to review your work as you have it!

Could I please ask you to also comment at #996 to mention there that you're working on it?

Also, will you be at the PyCon sprints?

[lukesneeringer]

That sounds good.
I currently plan to be at only day one of PyCon sprints, but I have not booked plane tickets yet, so that is mutable.