packaging: Switch from poetry to uv

[myme]

Description

This PR switches novem-python over to use uv for development and packaging. This change is motivated by poetry2nix being abandoned in favor of uv2nix. That said, uv has been gaining popularity as a faster and "better" package manager for Python outside of the nix ecosystem.

Installation

uv installation instructions can be found here: https://docs.astral.sh/uv/#installation

Examples

Python 3.9:

2025-12-15-155800.webm

Python 3.13:

2025-12-15-160005.webm

Pre-commit hooks:

CI workflow:

Nix check:

[sondove]

Magic, got it.

Does this mean nix workflows are back on the table?

[myme]

Magic, got it.

Does this mean nix workflows are back on the table?

Yes. Last commit enables it again.

[bjornars]

Magic!

My only gripe is that I think this is a complete "relock" of the dependencies - so we've implicitly upgraded all kinds of minor versions. If it works, it's probably ok?

My experiments led me to uvx migrate-to-uv which migrated poetry.lock file over - but maybe that introduces other problems, idk.

[myme]

Magic!

My only gripe is that I think this is a complete "relock" of the dependencies - so we've implicitly upgraded all kinds of minor versions. If it works, it's probably ok?

My experiments led me to uvx migrate-to-uv which migrated poetry.lock file over - but maybe that introduces other problems, idk.

Yes. I wanted to resolve that a bit better myself and tried uvx migrate-to-uv. It did complain about some of our deps:

But it did produce an uv.lock in the end.

I had Claude write me a comparison of poetry.lock vs. uv.lock:

❯ python compare_locks.py
======================================================================
PACKAGE VERSION COMPARISON: poetry.lock (main) vs uv.lock (current)
======================================================================

### VERSION DIFFERENCES (23 packages)
----------------------------------------------------------------------
Package                             poetry.lock     uv.lock
----------------------------------------------------------------------
certifi                             2025.7.14       2025.11.12
cfgv                                3.4.0           3.4.0, 3.5.0
charset-normalizer                  3.4.2           3.4.4
click                               8.1.8, 8.2.1    8.1.8, 8.3.1
coverage                            7.9.2           7.10.7, 7.13.0
distlib                             0.3.9           0.4.0
exceptiongroup                      1.3.0           1.3.1
filelock                            3.18.0          3.19.1, 3.20.0
identify                            2.6.12          2.6.15
idna                                3.10            3.11
iniconfig                           2.1.0           2.1.0, 2.3.0
librt                               0.7.3           0.7.4
mypy                                1.19.0          1.19.1
numpy                               2.0.2, 2.3.1    2.0.2, 2.2.6, 2.3.5
pandas                              2.3.1           2.3.3
pandas-stubs                        2.2.2.240807, 2.3.0.250703 2.2.2.240807, 2.3.3.251201
platformdirs                        4.3.8           4.4.0, 4.5.1
pyyaml                              6.0.2           6.0.3
tomli                               2.2.1           2.3.0
types-pytz                          2025.2.0.20250516 2025.2.0.20251108
types-requests                      2.32.4.20250611 2.32.4.20250913
tzdata                              2025.2          2025.3
virtualenv                          20.31.2         20.35.4

### ONLY IN uv.lock (1 packages)
----------------------------------------------------------------------
  novem: 0.5.6

### SAME VERSION (31 packages)
----------------------------------------------------------------------
  annotated-types: 0.7.0
  black: 24.10.0
  colorama: 0.4.6
  flake8: 7.3.0
  importlib-metadata: 8.7.0
  isort: 6.1.0
  mccabe: 0.7.0
  mypy-extensions: 1.1.0
  nodeenv: 1.9.1
  packaging: 25.0
  pathspec: 0.12.1
  pluggy: 1.6.0
  pre-commit: 4.3.0, 4.5.0
  pycodestyle: 2.14.0
  pydantic: 2.12.5
  pydantic-core: 2.41.5
  pyfakefs: 5.10.2
  pyflakes: 3.4.0
  pygments: 2.19.2
  pyreadline3: 3.5.4
  pytest: 8.4.2
  pytest-cov: 5.0.0
  python-dateutil: 2.9.0.post0
  pytz: 2025.2
  requests: 2.32.5
  requests-mock: 1.12.1
  six: 1.17.0
  typing-extensions: 4.15.0
  typing-inspection: 0.4.2
  urllib3: 2.6.2
  zipp: 3.23.0

======================================================================
SUMMARY
======================================================================
  Total packages in poetry.lock (main): 54
  Total packages in uv.lock (current):  55
  Packages with version differences:    23
  Packages only in uv.lock:             1
  Packages only in poetry.lock:         0
  Packages with same version:           31

certifi, charset-normalizer, idna are all requests dependencies and will be updated in client software. However, I do consider requests so thoroughly tested and used that bumping these should be fairly safe. Other direct production dependencies that haven't changed are urlib3, typing-extensions and packaging.

No new versions are older than those provided by our old poetry.lock. I think the remainder of the updated versions are transitive test/dev dependencies, and shouldn't matter all that much.

[myme]

Here's the compare_locks.py script:

#!/usr/bin/env python3
"""Compare package versions between uv.lock and poetry.lock (from main branch)."""

import re
import subprocess
import sys
from collections import defaultdict


def parse_uv_lock(filepath: str) -> dict[str, set[str]]:
    """Parse uv.lock file and extract package names and versions."""
    packages = defaultdict(set)

    with open(filepath) as f:
        content = f.read()

    # uv.lock format: [[package]] blocks with name = "..." and version = "..."
    # Split by [[package]] markers
    package_blocks = re.split(r'\[\[package\]\]', content)

    for block in package_blocks[1:]:  # Skip the header before first package
        name_match = re.search(r'^name\s*=\s*"([^"]+)"', block, re.MULTILINE)
        version_match = re.search(r'^version\s*=\s*"([^"]+)"', block, re.MULTILINE)

        if name_match and version_match:
            name = name_match.group(1)
            version = version_match.group(1)
            packages[name].add(version)

    return packages


def parse_poetry_lock(content: str) -> dict[str, set[str]]:
    """Parse poetry.lock content and extract package names and versions."""
    packages = defaultdict(set)

    # poetry.lock format: [[package]] blocks with name = "..." and version = "..."
    package_blocks = re.split(r'\[\[package\]\]', content)

    for block in package_blocks[1:]:  # Skip the header before first package
        name_match = re.search(r'^name\s*=\s*"([^"]+)"', block, re.MULTILINE)
        version_match = re.search(r'^version\s*=\s*"([^"]+)"', block, re.MULTILINE)

        if name_match and version_match:
            name = name_match.group(1)
            version = version_match.group(1)
            packages[name].add(version)

    return packages


def get_poetry_lock_from_main() -> str:
    """Get poetry.lock content from the main branch."""
    result = subprocess.run(
        ['git', 'show', 'main:poetry.lock'],
        capture_output=True,
        text=True
    )
    if result.returncode != 0:
        print(f"Error fetching poetry.lock from main: {result.stderr}", file=sys.stderr)
        sys.exit(1)
    return result.stdout


def main():
    # Parse both lock files
    uv_packages = parse_uv_lock('uv.lock')
    poetry_content = get_poetry_lock_from_main()
    poetry_packages = parse_poetry_lock(poetry_content)

    # Normalize package names (replace - with _ for comparison)
    def normalize(name: str) -> str:
        return name.lower().replace('-', '_')

    uv_normalized = {normalize(k): (k, v) for k, v in uv_packages.items()}
    poetry_normalized = {normalize(k): (k, v) for k, v in poetry_packages.items()}

    all_packages = sorted(set(uv_normalized.keys()) | set(poetry_normalized.keys()))

    # Categorize differences
    only_in_uv = []
    only_in_poetry = []
    version_diff = []
    same_version = []

    for norm_name in all_packages:
        in_uv = norm_name in uv_normalized
        in_poetry = norm_name in poetry_normalized

        if in_uv and not in_poetry:
            name, versions = uv_normalized[norm_name]
            only_in_uv.append((name, versions))
        elif in_poetry and not in_uv:
            name, versions = poetry_normalized[norm_name]
            only_in_poetry.append((name, versions))
        else:
            uv_name, uv_versions = uv_normalized[norm_name]
            poetry_name, poetry_versions = poetry_normalized[norm_name]

            if uv_versions != poetry_versions:
                version_diff.append((uv_name, poetry_versions, uv_versions))
            else:
                same_version.append((uv_name, uv_versions))

    # Print results
    print("=" * 70)
    print("PACKAGE VERSION COMPARISON: poetry.lock (main) vs uv.lock (current)")
    print("=" * 70)

    if version_diff:
        print(f"\n### VERSION DIFFERENCES ({len(version_diff)} packages)")
        print("-" * 70)
        print(f"{'Package':<35} {'poetry.lock':<15} {'uv.lock':<15}")
        print("-" * 70)
        for name, poetry_vers, uv_vers in sorted(version_diff):
            poetry_str = ', '.join(sorted(poetry_vers))
            uv_str = ', '.join(sorted(uv_vers))
            print(f"{name:<35} {poetry_str:<15} {uv_str:<15}")

    if only_in_uv:
        print(f"\n### ONLY IN uv.lock ({len(only_in_uv)} packages)")
        print("-" * 70)
        for name, versions in sorted(only_in_uv):
            ver_str = ', '.join(sorted(versions))
            print(f"  {name}: {ver_str}")

    if only_in_poetry:
        print(f"\n### ONLY IN poetry.lock ({len(only_in_poetry)} packages)")
        print("-" * 70)
        for name, versions in sorted(only_in_poetry):
            ver_str = ', '.join(sorted(versions))
            print(f"  {name}: {ver_str}")

    if same_version:
        print(f"\n### SAME VERSION ({len(same_version)} packages)")
        print("-" * 70)
        for name, versions in sorted(same_version):
            ver_str = ', '.join(sorted(versions))
            print(f"  {name}: {ver_str}")

    # Summary
    print("\n" + "=" * 70)
    print("SUMMARY")
    print("=" * 70)
    print(f"  Total packages in poetry.lock (main): {len(poetry_packages)}")
    print(f"  Total packages in uv.lock (current):  {len(uv_packages)}")
    print(f"  Packages with version differences:    {len(version_diff)}")
    print(f"  Packages only in uv.lock:             {len(only_in_uv)}")
    print(f"  Packages only in poetry.lock:         {len(only_in_poetry)}")
    print(f"  Packages with same version:           {len(same_version)}")


if __name__ == '__main__':
    main()

[sondove]

Seems pretty solid to me