Skip to content

Add a guide about publishing dists via GH Actions #647

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
40 changes: 40 additions & 0 deletions source/guides/github-actions-ci-cd-sample/publish-to-test-pypi.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
name: Publish Python 🐍 distributions 📦 to PyPI and TestPyPI

on: push

jobs:
build-n-publish:
name: Build and publish Python 🐍 distributions 📦 to PyPI and TestPyPI
runs-on: ubuntu-18.04

steps:
- uses: actions/checkout@master
- name: Set up Python 3.7
uses: actions/setup-python@v1
with:
version: 3.7
- name: Install pep517
run: >-
python -m
pip install
pep517
--user
- name: Build a binary wheel and a source tarball
run: >-
python -m
pep517.build
--source
--binary
--out-dir dist/
.
# Actually publish to PyPI/TestPyPI
- name: Publish distribution 📦 to Test PyPI
uses: pypa/gh-action-pypi-publish@master
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Once there's a release, include the version number here.

We strongly recommend that you include the version of the action you are using by specifying a Git ref, SHA, or Docker tag number. If you don't specify a version, it could break your workflows or cause unexpected behavior when the action owner publishes an update.

  • Using the commit SHA of a released action version is the safest for stability and security.
  • Using the specific major action version allows you to receive critical fixes and security patches while still maintaining compatibility. It also assures that your workflow should still work.
  • Using the master branch of an action may be convenient, but if someone releases a new major version with a breaking change, your workflow could break.

https://help.github.com/en/articles/workflow-syntax-for-github-actions#jobsjob_idstepsuses

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, I'd also mention it somewhere in the README of the action itself.

with:
password: ${{ secrets.test_pypi_password }}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
password: ${{ secrets.test_pypi_password }}
password: ${{ secrets.testpypi_password }}

repository_url: https://test.pypi.org/legacy/
- name: Publish distribution 📦 to PyPI
if: startsWith(github.event.ref, 'refs/tags')
uses: pypa/gh-action-pypi-publish@master
with:
password: ${{ secrets.pypi_password }}
1 change: 1 addition & 0 deletions source/guides/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -29,3 +29,4 @@ introduction to packaging, see :doc:`/tutorials/index`.
migrating-to-pypi-org
using-testpypi
making-a-pypi-friendly-readme
publishing-package-distribution-releases-using-github-actions-ci-cd-workflows
Original file line number Diff line number Diff line change
@@ -0,0 +1,154 @@
=============================================================================
Publishing package distribution releases using GitHub Actions CI/CD workflows
=============================================================================

`GitHub Actions CI/CD`_ allows you to run a series of commands
whenever an event occurs on the GitHub platform. One
popular choice is having a workflow that's triggered by a
``push`` event.
This guide shows you how to publish a Python distribution
whenever a tagged commit is pushed.
It will use the `pypa/gh-action-pypi-publish GitHub Action`_ https://github.com/marketplace/actions/pypi-publish

.. attention::

This guide *assumes* that you already have a project that
you know how to build distributions for and *it lives on GitHub*.

.. warning::

At the time of writing, `GitHub Actions CI/CD`_
is in public beta. If you don't have it enabled,
you should `join the waitlist`_ to gain access.


Saving credentials on GitHub
============================

In this guide, we'll demonstrate uploading to both
PyPI and TestPyPI, meaning that we'll have two separate sets
of credentials. And we'll need to save them in the GitHub repository
settings.

Let's begin! 🚀

1. Go to https://pypi.org/manage/account/#api-tokens and
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe put the Test PyPI instructions before production PyPI?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think users usually have an account on prod, but rarely on test...

create a new `API token`_. If you have the project on PyPI
already, limit the token scope to just that project.
You can call it something like
``GitHub Actions CI/CD — project-org/project-repo``
in order for it to be easily distinguishable in the token
list.
**Don't close the page just yet — you won't see that token
again.**
2. In a separate browser tab or window, go to the ``Settings``
tab of your target repository and then click on `Secrets`_
in the left sidebar.
3. Create a new secret called ``pypi_password`` and copy-paste
the token from the fist step.
4. Now, go to https://test.pypi.org/manage/account/#api-tokens
and repeat the steps. Save that TestPyPI token on GitHub
as ``test_pypi_password``.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
as ``test_pypi_password``.
as ``testpypi_password``.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that it's better readable to maintain snake_case here.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is testpypi_password the snake_case version of TestPyPI (no space)?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TestPyPI is CamelCase version and it's clearly visible that test is a separate word, even though it's spelled together in the capitalized version. So I consider test_pypi_password a proper snake_case version.


.. attention::

If you don't have a TestPyPI account, you'll need to
create it. It's not the same as a regular PyPI account.


Creating a workflow definition
==============================

GitHub CI/CD workflows are declared in YAML files stored in the
``.github/workflows/`` directory of your repository.

Let's create a ``.github/workflows/publish-to-test-pypi.yml``
file.

Start it with a meaningful name and define the event that
should make GitHub run this workflow:

.. literalinclude:: github-actions-ci-cd-sample/publish-to-test-pypi.yml
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
.. literalinclude:: github-actions-ci-cd-sample/publish-to-test-pypi.yml
.. literalinclude:: github-actions-ci-cd-sample/publish-to-testpypi.yml

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think, it's better readable to keep the separation.

:language: yaml
:end-before: jobs:


Defining a workflow job environment
===================================

Now, let's add initial setup for our job. It's a process that
will execute commands that we'll define later.
In this guide, we'll use Ubuntu 18.04:

.. literalinclude:: github-actions-ci-cd-sample/publish-to-test-pypi.yml
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
.. literalinclude:: github-actions-ci-cd-sample/publish-to-test-pypi.yml
.. literalinclude:: github-actions-ci-cd-sample/publish-to-testpypi.yml

:language: yaml
:start-after: on:
:end-before: steps:


Checking out the project and building distributions
===================================================

Then, add the following under the ``build-n-publish`` section:

.. literalinclude:: github-actions-ci-cd-sample/publish-to-test-pypi.yml
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
.. literalinclude:: github-actions-ci-cd-sample/publish-to-test-pypi.yml
.. literalinclude:: github-actions-ci-cd-sample/publish-to-testpypi.yml

:language: yaml
:start-after: runs-on:
:end-before: Install pep517

This will download your repository into the CI runner and then
install and activate Python 3.7.

And now we can build dists from source. In this example, we'll
use ``pep517`` package, assuming that your project has a
``pyproject.toml`` properly set up (see
:pep:`517`/:pep:`518`).
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe this would be a good place to explain what the pep517 package is? Most users haven't seen this before.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I wasn't sure about this. AFAIK it's not really intended for end-users but it was an easy way to inject "some way" of building dists as I'm trying not to focus on building in this guide and point users to their favorite building methods...


.. tip::

You can use any other method for building distributions as long as
it produces ready-to-upload artifacts saved into the
``dist/`` folder.

So add this to the steps list:

.. literalinclude:: github-actions-ci-cd-sample/publish-to-test-pypi.yml
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
.. literalinclude:: github-actions-ci-cd-sample/publish-to-test-pypi.yml
.. literalinclude:: github-actions-ci-cd-sample/publish-to-testpypi.yml

:language: yaml
:start-after: version: 3.7
:end-before: Actually publish to PyPI/TestPyPI


Publishing the distribution to PyPI and TestPyPI
================================================

Finally, add the following steps at the end:

.. literalinclude:: github-actions-ci-cd-sample/publish-to-test-pypi.yml
:language: yaml
:start-after: Actually publish to PyPI/TestPyPI

These two steps use the `pypa/gh-action-pypi-publish`_ GitHub
Action: the first one uploads contents of the ``dist/`` folder
into TestPyPI unconditionally and the second does that to
PyPI, but only if the current commit is tagged.


That's all, folks!
==================

Now, whenever you push a tagged commit to your Git repository remote
on GitHub, this workflow will publish it to PyPI.
And it'll publish any push to TestPyPI which is useful for
providing test builds to your alpha users as well as making
sure that your release pipeline remains healthy!


.. _API token: https://pypi.org/help/#apitoken
.. _GitHub Actions CI/CD: https://github.com/features/actions
.. _join the waitlist: https://github.com/features/actions/signup
.. _pypa/gh-action-pypi-publish:
https://github.com/pypa/gh-action-pypi-publish
.. _`pypa/gh-action-pypi-publish GitHub Action`:
https://github.com/marketplace/actions/pypi-publish
.. _Secrets:
https://help.github.com/en/articles/virtual-environments-for-github-actions#creating-and-using-secrets-encrypted-variables