Skip to content

Conversation

@jsf9k
Copy link
Member

@jsf9k jsf9k commented Oct 28, 2025

🗣 Description

This pull request updates our project packaging configuration from using a setup.py file to using a pyproject.toml file with a setuptools build backend.

See also cisagov/skeleton-generic#235.

💭 Motivation and context

Resolves #163. Resolves #136.

🧪 Testing

All automated tests pass.

✅ Pre-approval checklist

  • This PR has an informative and human-readable title.
  • Changes are limited to a single goal - eschew scope creep!
  • All relevant type-of-change labels have been added.
  • I have read the CONTRIBUTING document.
  • These code changes follow cisagov code standards.
  • All new and existing tests pass.
  • Bump major, minor, patch, pre-release, and/or build versions as appropriate via the bump_version script if this repository is versioned and the changes in this PR warrant a version bump.
  • Create a pre-release (necessary if and only if the pre-release version was bumped).

✅ Pre-merge checklist

  • Finalize version.

✅ Post-merge checklist

  • Create a release (necessary if and only if the version was bumped).

@jsf9k jsf9k self-assigned this Oct 28, 2025
@jsf9k jsf9k added the improvement This issue or pull request will add or improve functionality, maintainability, or ease of use label Oct 28, 2025
@jsf9k jsf9k moved this to In Progress in Next Kraken Oct 28, 2025
@github-actions github-actions bot added dependencies Pull requests that update a dependency file python Pull requests that update Python code labels Oct 28, 2025
@jsf9k jsf9k force-pushed the improvement/use-pyproject-toml-file branch from 540a8db to 336618a Compare October 28, 2025 17:26
@jsf9k jsf9k moved this from In progress to Review in progress in Skeleton Maintenance Oct 28, 2025
@jsf9k jsf9k added the kraken 🐙 This pull request is ready to merge during the next Lineage Kraken release label Oct 28, 2025
@github-actions github-actions bot added the test This issue or pull request adds or otherwise modifies test code label Oct 28, 2025
@jsf9k
Copy link
Member Author

jsf9k commented Oct 28, 2025

@mcdonnnj - I'm not sure what kind of version bump to apply here, if any.

@jsf9k
Copy link
Member Author

jsf9k commented Oct 28, 2025

The flake8 config could be moved to the pyproject.toml file as well, but that would require the flake8-pyproject Python package. There is also the complication that the flake8 configuration comes from upstream. See cisagov/skeleton-generic#234.

The bandit config is only used when pre-commit runs bandit against the Python test code; otherwise, we use the default configuration for bandit. For completeness, also see cisagov/skeleton-generic#233.

@jsf9k jsf9k marked this pull request as ready for review October 28, 2025 17:59
@jsf9k jsf9k requested review from dav3r and mcdonnnj as code owners October 28, 2025 17:59
@jsf9k
Copy link
Member Author

jsf9k commented Oct 28, 2025

I realize that we want wheel and pip to be the latest versions, but should we now remove setuptools from setup-env? If so then this would be an upstream change in cisagov/skeleton-generic.

Copy link
Member

@dav3r dav3r left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This all looks like a solid start to me. Thanks for getting us switched over to pyproject.toml. 🎉

@dav3r dav3r requested a review from Copilot October 29, 2025 19:22
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This pull request migrates the project from a setup.py-based build configuration to a modern pyproject.toml-based configuration using the Hatchling build backend. This modernization consolidates configuration files and follows current Python packaging best practices (PEP 517/518).

  • Migrates from setup.py to pyproject.toml with Hatchling as the build backend
  • Consolidates pytest.ini and .isort.cfg configurations into pyproject.toml
  • Updates GitHub labeler configuration to reference the new configuration file

Reviewed Changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
setup.py Removed legacy setup.py file (116 lines)
pytest.ini Removed standalone pytest configuration file
.isort.cfg Removed standalone isort configuration file
pyproject.toml Added comprehensive project configuration including build system, dependencies, tool configurations, and metadata
.github/labeler.yml Updated file references from setup.py, .isort.cfg, and pytest.ini to pyproject.toml

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@jsf9k
Copy link
Member Author

jsf9k commented Nov 5, 2025

@mcdonnnj - Any thoughts here? Are you still researching the current landscape of build backends?

@jsf9k
Copy link
Member Author

jsf9k commented Nov 6, 2025

In this PR I moved the isort config to pyproject.toml, but just now I realized that this file is actually inherited from cisagov/skeleton-generic. As a result I have dropped commit 1ae44fe.

Do we want to leave the .isort.cfg file as it is in cisagov/skeleton-generic or create a pyproject.toml in that repo to contain that tool's configuration? I'm for the latter, since that would make things cleaner here.

jsf9k added 3 commits November 6, 2025 11:55
This is because this file now contains configurations for test tools.

Also remove test tool config files that no longer exist.
These may be of use to folks who are editing pyproject.toml for a
descendant of this skeleton repository.
@jsf9k jsf9k force-pushed the improvement/use-pyproject-toml-file branch from 901f748 to 3b8d5fc Compare November 6, 2025 16:55
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
@dav3r
Copy link
Member

dav3r commented Nov 6, 2025

Do we want to leave the .isort.cfg file as it is in cisagov/skeleton-generic or create a pyproject.toml in that repo to contain that tool's configuration? I'm for the latter, since that would make things cleaner here.

I get what you are saying and see the appeal of getting rid of .isort.cfg, but I don't love the idea of having a python-specific file (pyproject.toml) in a non-python repo (skeleton-generic). @mcdonnnj, what do you think here?

@jsf9k
Copy link
Member Author

jsf9k commented Nov 6, 2025

Do we want to leave the .isort.cfg file as it is in cisagov/skeleton-generic or create a pyproject.toml in that repo to contain that tool's configuration? I'm for the latter, since that would make things cleaner here.

I get what you are saying and see the appeal of getting rid of .isort.cfg, but I don't love the idea of having a python-specific file (pyproject.toml) in a non-python repo (skeleton-generic). @mcdonnnj, what do you think here?

In a way you already do, since .isort.cfg is present. isort only sorts Python imports. That sounds pretty Python-specific to me.

The pyproject.toml in cisagov/skeleton-generic wouldn't contain a [project] section, just a [tool.isort] section.

@dav3r
Copy link
Member

dav3r commented Nov 6, 2025

In a way you already do, since .isort.cfg is present. isort only sorts Python imports. That sounds pretty Python-specific to me.

LOL, I forgot about that. In that case, I agree with your plan to replace .isort.cfg with a section in pyproject.toml.

This gets rid of the .bandit.yml file that was being used only against
the test code.
@jsf9k
Copy link
Member Author

jsf9k commented Nov 9, 2025

@mcdonnnj - At one point you said that if we used setuptools as the Python build backend then we shouldn't get rid of the setup.py file. Why was that? I don't think the setup.py file is necessary unless you're compiling native code. If you insist, I could add a simple setup.py like this as a placeholder:

import setuptools

setuptools.setup()

I don't see the point of that, though.

Also, it appears that hatchling doesn't support native code associated with Python libraries very well. Maybe that's one reason to use setuptools instead. As far as I can tell, setuptools is still the most common build backend and a lot of folks are saying the fact that the documentation appears to promote hatchling instead is misleading. Projects like numpy that have a lot of compiled native code tend to use something like PDM.

I'm kind of of the opinion that until we have native extensions it's a tossup between hatchling and setuptools.

I think this PR would change only slightly if we used setuptools as the build backend instead of hatchling. I'm happy to make those changes if that's where you're leaning.

@jsf9k
Copy link
Member Author

jsf9k commented Nov 10, 2025

FYI, here are the necessary changes for moving to setuptools as the build backend:

diff --git a/pyproject.toml b/pyproject.toml
index 370ba01..727434b 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -1,9 +1,9 @@
 # For more information about configuring project metadata for the
-# hatch build backend, please see
-# https://hatch.pypa.io/latest/config/metadata/
+# setuptools build backend, please see
+# https://setuptools.pypa.io/en/latest/userguide/pyproject_config.html
 [build-system]
-build-backend = "hatchling.build"
-requires = ["hatchling"]
+build-backend = "setuptools.build_meta"
+requires = ["setuptools"]
 
 [project]
 authors = [
@@ -38,11 +38,10 @@ dependencies = [
     "schema",
 ]
 description = "Example Python library"
-dynamic = ["version"]
+dynamic = ["readme", "version"]
 keywords = ["skeleton"]
 license = "CC0-1.0"
 name = "example"
-readme = "README.md"
 requires-python = ">=3.9"
 
 [project.optional-dependencies]
@@ -74,12 +73,12 @@ issues = "https://github.com/cisagov/skeleton-python-library/issues"
 mission = "https://www.cisa.gov/cybersecurity"
 source = "https://github.com/cisagov/skeleton-python-library"
 
-[tool.hatch.version]
-# Versions should comply with PEP440
-path = "src/example/_version.py"
-
 [tool.pytest.ini_options]
 # Increase verbosity, display extra test summary info for tests that
 # did not pass, display code coverage results, and enable debug
 # logging.
 addopts = "--verbose -ra --cov --log-cli-level=DEBUG"
+
+[tool.setuptools.dynamic]
+readme = {file = ["README.md"], content-type = "text/markdown"}
+version = {attr = "example._version.__version__"}

@jsf9k
Copy link
Member Author

jsf9k commented Nov 10, 2025

In case it helps, this is how I cope with the current state of Python packaging:
image

@mcdonnnj
Copy link
Member

@mcdonnnj - At one point you said that if we used setuptools as the Python build backend then we shouldn't get rid of the setup.py file. Why was that? I don't think the setup.py file is necessary unless you're compiling native code.

Keeping the setup.py file is the recommendation of PyPA in their guide to modernizing setup.py projects and it has not been deprecated.

@jsf9k
Copy link
Member Author

jsf9k commented Nov 10, 2025

@mcdonnnj - At one point you said that if we used setuptools as the Python build backend then we shouldn't get rid of the setup.py file. Why was that? I don't think the setup.py file is necessary unless you're compiling native code.

Keeping the setup.py file is the recommendation of PyPA in their guide to modernizing setup.py projects and it has not been deprecated.

Yes, in the first link they say it can coexist but nowhere do they say it must be present. My point is that if everything can be moved into the pyproject.toml file (as I show above) then there is no need for it. It is just a dumb placeholder file.

@jsf9k
Copy link
Member Author

jsf9k commented Nov 10, 2025

I don't think the setup.py file is necessary unless you're compiling native code.

Actually, it isn't even necessary if you're compiling native code. So I don't see any reason to keep a setup.py file around.

The setuptools build backend:
- Supports native extensions, in contrast with the hatchling build
backend
- Is the most commonly used build backend among projects on PyPI
- Is an officially supported build backend from Python, in contrast
with the hatchling build backend
@jsf9k
Copy link
Member Author

jsf9k commented Nov 10, 2025

FYI, here are the necessary changes for moving to setuptools as the build backend:

diff --git a/pyproject.toml b/pyproject.toml
index 370ba01..727434b 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -1,9 +1,9 @@
 # For more information about configuring project metadata for the
-# hatch build backend, please see
-# https://hatch.pypa.io/latest/config/metadata/
+# setuptools build backend, please see
+# https://setuptools.pypa.io/en/latest/userguide/pyproject_config.html
 [build-system]
-build-backend = "hatchling.build"
-requires = ["hatchling"]
+build-backend = "setuptools.build_meta"
+requires = ["setuptools"]
 
 [project]
 authors = [
@@ -38,11 +38,10 @@ dependencies = [
     "schema",
 ]
 description = "Example Python library"
-dynamic = ["version"]
+dynamic = ["readme", "version"]
 keywords = ["skeleton"]
 license = "CC0-1.0"
 name = "example"
-readme = "README.md"
 requires-python = ">=3.9"
 
 [project.optional-dependencies]
@@ -74,12 +73,12 @@ issues = "https://github.com/cisagov/skeleton-python-library/issues"
 mission = "https://www.cisa.gov/cybersecurity"
 source = "https://github.com/cisagov/skeleton-python-library"
 
-[tool.hatch.version]
-# Versions should comply with PEP440
-path = "src/example/_version.py"
-
 [tool.pytest.ini_options]
 # Increase verbosity, display extra test summary info for tests that
 # did not pass, display code coverage results, and enable debug
 # logging.
 addopts = "--verbose -ra --cov --log-cli-level=DEBUG"
+
+[tool.setuptools.dynamic]
+readme = {file = ["README.md"], content-type = "text/markdown"}
+version = {attr = "example._version.__version__"}

Please see commit 461f872.

@jsf9k
Copy link
Member Author

jsf9k commented Nov 10, 2025

@mcdonnnj - At one point you said that if we used setuptools as the Python build backend then we shouldn't get rid of the setup.py file. Why was that? I don't think the setup.py file is necessary unless you're compiling native code. If you insist, I could add a simple setup.py like this as a placeholder:

import setuptools

setuptools.setup()

I don't see the point of that, though.

Also, it appears that hatchling doesn't support native code associated with Python libraries very well. Maybe that's one reason to use setuptools instead. As far as I can tell, setuptools is still the most common build backend and a lot of folks are saying the fact that the documentation appears to promote hatchling instead is misleading. Projects like numpy that have a lot of compiled native code tend to use something like PDM.

I'm kind of of the opinion that until we have native extensions it's a tossup between hatchling and setuptools.

I think this PR would change only slightly if we used setuptools as the build backend instead of hatchling. I'm happy to make those changes if that's where you're leaning.

Please see commit 461f872.

This ensures that the data file(s) are incorporated into the wheels
that are built.
This directory can be created by `pip install .`.
@jsf9k jsf9k requested a review from dav3r November 12, 2025 14:50
Copy link
Member

@dav3r dav3r left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This all looks reasonable to me. 👍

@jsf9k
Copy link
Member Author

jsf9k commented Nov 12, 2025

I realize that we want wheel and pip to be the latest versions, but should we now remove setuptools from setup-env? If so then this would be an upstream change in cisagov/skeleton-generic.

This comment is no longer applicable since we moved to using setuptools as our build backend.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

dependencies Pull requests that update a dependency file improvement This issue or pull request will add or improve functionality, maintainability, or ease of use kraken 🐙 This pull request is ready to merge during the next Lineage Kraken release python Pull requests that update Python code test This issue or pull request adds or otherwise modifies test code

Projects

Status: In Progress
Status: Review in progress

Development

Successfully merging this pull request may close these issues.

Upgrade from setup.py to pyproject.toml Setup.py link returns 404 error

4 participants