Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Step 4: A PEP517-styled quickstart user guide and room for collaboration #2146

Merged
merged 7 commits into from
May 26, 2020
Merged

Step 4: A PEP517-styled quickstart user guide and room for collaboration #2146

Changes from 5 commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
04dbe6f
docs: WIP update quickstart guide
alvyjudy May 18, 2020
cdf8524
docs: made quickstart pkg section declarative
alvyjudy May 20, 2020
780b91b
docs: changed some mistaken explanation in quickst
alvyjudy May 25, 2020
8dbae6e
docs: update quickstart
alvyjudy May 25, 2020
b2af2ea
docs: update quickstart
alvyjudy May 25, 2020
992ecc0
docs: proper invocation of pep517
alvyjudy May 25, 2020
b13e4a2
docs: add dev mode section in quickstart
alvyjudy May 26, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
278 changes: 128 additions & 150 deletions docs/userguide/quickstart.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,211 +2,189 @@
``setuptools`` Quickstart
==========================

.. contents::

Installation
============

.. _Installing Packages: https://packaging.python.org/tutorials/installing-packages/

To install the latest version of setuptools, use::

pip install --upgrade setuptools

Refer to `Installing Packages`_ guide for more information.

Python packaging at a glance
============================
The landscape of Python packaging is shifting and ``Setuptools`` has evolved to
only provide backend support, no longer being the de-facto packaging tool in
the market. All python package must provide a ``pyproject.toml`` and specify
the backend (build system) it wants to use. The distribution can then
be generated with whatever tools that provides a ``build sdist``-alike
functionality. While this may appear cumbersome, given the added pieces,
it in fact tremendously enhances the portability of your package. The
change is driven under `PEP 517 <https://www.python.org/dev/peps/pep-0517/#
build-requirements>``. To learn more about Python packaging in general,
navigate to the `bottom <Resources on python packaging>`_ of this page.


Basic Use
=========
For basic use of setuptools, you will need a ``pyproject.toml`` with the
exact following info, which declares you want to use ``setuptools`` to
package your project:

.. code-block:: toml

[build-system]
requires = ["setuptools", "wheel"]
build-backend = "setuptools.build_meta"

Then, you will need a ``setup.cfg`` to specify your package information,
such as metadata, contents, dependencies, etc. Here we demonstrate the minimum

.. code-block:: ini

[metadata]
name = "mypackage"
version = 0.0.1

For basic use of setuptools, just import things from setuptools. Here's a
minimal setup script using setuptools::
[options]
packages = "mypackage"
install_requires =
requests
importlib; python_version == "2.6"

from setuptools import setup, find_packages
setup(
name="HelloWorld",
version="0.1",
packages=find_packages(),
)
This is what your project would look like::

As you can see, it doesn't take much to use setuptools in a project.
Run that script in your project folder, alongside the Python packages
you have developed.
~/mypackage/
pyproject.toml
setup.cfg
mypackage/__init__.py

Invoke that script to produce distributions and automatically include all
packages in the directory where the setup.py lives. See the `Command
Reference`_ section below to see what commands you can give to this setup
script. For example, to produce a source distribution, simply invoke::
Then, you need an installer, such as `pep517 <https://pypi.org/project/pep517/>`_
which you can obtain via ``pip install pep517``. After downloading it, invoke
the installer::

setup.py sdist
pep517 build
alvyjudy marked this conversation as resolved.
Show resolved Hide resolved

You now have your distribution ready (e.g. a ``tar.gz`` file and a ``.whl``
file in the ``dist`` directory), which you can upload to PyPI!

Of course, before you release your project to PyPI, you'll want to add a bit
more information to your setup script to help people find or learn about your
project. And maybe your project will have grown by then to include a few
dependencies, and perhaps some data files and scripts::

from setuptools import setup, find_packages
setup(
name="HelloWorld",
version="0.1",
packages=find_packages(),
scripts=["say_hello.py"],

# Project uses reStructuredText, so ensure that the docutils get
# installed or upgraded on the target machine
install_requires=["docutils>=0.3"],

package_data={
# If any package contains *.txt or *.rst files, include them:
"": ["*.txt", "*.rst"],
# And include any *.msg files found in the "hello" package, too:
"hello": ["*.msg"],
},

# metadata to display on PyPI
author="Me",
author_email="me@example.com",
description="This is an Example Package",
keywords="hello world example examples",
url="http://example.com/HelloWorld/", # project home page, if any
project_urls={
"Bug Tracker": "https://bugs.example.com/HelloWorld/",
"Documentation": "https://docs.example.com/HelloWorld/",
"Source Code": "https://code.example.com/HelloWorld/",
},
classifiers=[
"License :: OSI Approved :: Python Software Foundation License"
]

# could also include long_description, download_url, etc.
)
dependencies, and perhaps some data files and scripts. In the next few section,
we will walk through those additional but essential information you need
to specify to properly package your project.


Automatic package discovery
===========================

For simple projects, it's usually easy enough to manually add packages to
the ``packages`` argument of ``setup()``. However, for very large projects
, it can be a big burden to keep the package list updated. setuptools therefore
provides tools to ease the burden.
the ``packages`` keyword in ``setup.cfg``. However, for very large projects
, it can be a big burden to keep the package list updated. ``setuptools``
therefore provides two convenient tools to ease the burden: ``find: `` and
``find_namespace: ``. To use it in your project:

.. code-block:: ini

``find_packages()`` takes a source directory and two lists of package name
patterns to exclude and include. It then walks the target directory, filtering
by inclusion patterns, and return a list of Python packages (any directory).
Finally, exclusion patterns are applied to remove matching packages.
[options]
packages = find:

For example::
#...
from setuptools import find_packages()
[options.packages.find] #optional
include=pkg1, pkg2
exclude=pk3, pk4

setup(
#...,
packages = find_packages()
)
When you pass the above information, alongside other necessary ones,
``setuptools`` walks through the directory specified in ``where`` (omitted
here as the package reside in current directory) and filters the packages
it can find following the ``include`` (default to none), then remove
those that match the ``exclude`` and return a list of Python packages. Note
that each entry in the ``[options.packages.find]`` is optional. The above
setup also allows you to adopt a ``src/`` layout. For more details and advanced
use, go to :ref:`package_discovery`

For more details and advanced use, go to :ref:`package_discovery`

Entry points and automatic script creation
===========================================

Setuptools support automatic creation of scripts upon installation, that runs
code within your package if you specify them with the ``entry_point`` keyword.
This is what allows you to run commands like ``pip install`` instead of having
to type ``python -m pip install``. To accomplish this, consider the following
example::

setup(
#....
entry_points={
"console_scripts": [
"foo = my_package.some_module:main_func",
],
}
)

When this project is installed, a ``foo`` script will be installed and will
invoke the ``main_func`` when called by the user. For detailed usage, including
managing the additional or optional dependencies, go to :ref:`entry_point`.
to type ``python -m pip install``. To accomplish this, add the entry_points
keyword in your ``setup.cfg``:

.. code-block:: ini

[options]
entry_points =
[console_script]
main = mypkg:some_func

When this project is installed, a ``main`` script will be installed and will
invoke the ``some_func`` in the ``__init__.py`` file when called by the user.
For detailed usage, including managing the additional or optional dependencies,
go to :ref:`entry_point`.


Dependency management
=====================

``setuptools`` supports automatically installing dependencies when a package is
installed. The simplest way to include requirement specifiers is to use the
``install_requires`` argument to ``setup()``. It takes a string or list of
strings containing requirement specifiers::
``install_requires`` argument to ``setup.cfg``. It takes a string or list of
strings containing requirement specifiers (A version specifier is one of the
operators <, >, <=, >=, == or !=, followed by a version identifier):

.. code-block:: ini

setup(
#...
install_requires = "docutils >= 0.3"
)
[options]
install_requires =
docutils >= 0.3
requests <= 0.4

When your project is installed, either by using pip, ``setup.py install``,
or ``setup.py develop``, all of the dependencies not already installed will
be located (via PyPI), downloaded, built (if necessary), and installed.
When your project is installed, all of the dependencies not already installed
will be located (via PyPI), downloaded, built (if necessary), and installed.
This, of course, is a simplified scenarios. ``setuptools`` also provide
additional keywords such as ``setup_requires`` that allows you to install
dependencies before running the script, and ``extras_requires`` that take
care of those needed by automatically generated scripts. It also provides
mechanisms to handle dependencies that are not in PyPI. For more advanced use,
see :ref:`dependency_management`

For more advanced use, see :ref:`dependency_management`

Including Data Files
====================

The distutils have traditionally allowed installation of "data files", which
are placed in a platform-specific location. Setuptools offers three ways to
specify data files to be included in your packages. For the simpliest use, you
can simply use the ``include_package_data`` keyword e.g.::
can simply use the ``include_package_data`` keyword:

.. code-block:: ini

setup(
...
include_package_data=True
)
[options]
include_package_data = True

This tells setuptools to install any data files it finds in your packages.
The data files must be specified via the distutils' ``MANIFEST.in`` file.

For more details, see :ref:`datafiles`

Development mode
alvyjudy marked this conversation as resolved.
Show resolved Hide resolved
================

Setuptools allows you to deploy your projects for use in a common directory or
staging area, but without copying any files. Thus, you can edit each project's
code in its checkout directory, and only need to run build commands when you
change a project's C extensions or similarly compiled files.

To do this, use the ``setup.py develop`` command. It works very similarly to
``setup.py install``, except that it doesn't actually install anything.
Instead, it creates a special ``.egg-link`` file in the deployment directory,
that links to your project's source code. And, if your deployment directory is
Python's ``site-packages`` directory, it will also update the
``easy-install.pth`` file to include your project's source code, thereby making
it available on ``sys.path`` for all programs using that Python installation.

for more information, go to :ref:`development_mode`

Distributing a ``setuptools``-based project
===========================================
Before you begin, make sure you have the latest versions of setuptools and wheel::

pip install --upgrade setuptools wheel

To build a setuptools project, run this command from the same directory where
setup.py is located::

setup.py sdist bdist_wheel

This will generate distribution archives in the `dist` directory.

Before you upload the generated archives make sure you're registered on
https://test.pypi.org/account/register/. You will also need to verify your email
to be able to upload any packages.
You should install twine to be able to upload packages::

pip install --upgrade twine

Now, to upload these archives, run::
Uploading your package to PyPI
==============================
After generating the distribution files, next step would be to upload your
distribution so others can use it. This functionality is provided by
``twine <https://pypi.org/project/twine/>`` and we will only demonstrate the
basic use here.

twine upload --repository-url https://test.pypi.org/legacy/ dist/*

To install your newly uploaded package ``example_pkg``, you can use pip::
Transitioning from ``setup.py`` to ``setup.cfg``
==================================================
To avoid executing arbitary scripts and boilerplate code, we are transitioning
into a full-fledged ``setup.cfg`` to declare your package information instead
of running ``setup()``. This inevitably brings challenges due to a different
syntax. Here we provide a quick guide to understanding how ``setup.cfg`` is
parsed by ``setuptool`` to ease the pain of transition.

pip install --index-url https://test.pypi.org/simple/ example_pkg

The next following sections will walk you through all of the available functions
``setuptools`` offers in excrutiating details (including those already mentioned)
for more advanced use.
Resources on Python packaging
=============================
Packaging in Python is hard. Here we provide a list of links for those that
want to learn more.