Setuptools-scm 0.9.3 breaks a number of distributions

[potiuk]

The 0.9. released about an hour ago breaks a number of distributions built from sdist. Detected by "canary" build of Apache airflow:

https://github.com/apache/airflow/actions/runs/16759485885/job/47451369503

For example: lazy-object-proxy:

  × Failed to build `lazy-object-proxy==1.7.1`
    ├─▶ The build backend returned an error
    ╰─▶ Call to `setuptools.build_meta:__legacy__.build_wheel` failed (exit
        status: 1)
  
        [stderr]
        toml section missing 'pyproject.toml does not contain a
        tool.setuptools_scm section'
        Traceback (most recent call last):
          File
        "/root/.cache/uv/builds-v0/.tmpX6p75h/lib/python3.12/site-packages/setuptools_scm/_integration/pyproject_reading.py",
        line 71, in read_pyproject
            section = defn.get("tool", {})[tool_name]
                      ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^
        KeyError: 'setuptools_scm'
        toml section missing 'pyproject.toml does not contain a
        tool.setuptools_scm section'
        Traceback (most recent call last):
          File
        "/root/.cache/uv/builds-v0/.tmpX6p75h/lib/python3.12/site-packages/setuptools_scm/_integration/pyproject_reading.py",
        line 71, in read_pyproject
            section = defn.get("tool", {})[tool_name]
                      ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^
        KeyError: 'setuptools_scm'
        Traceback (most recent call last):
          File "<string>", line 11, in <module>
          File
        "/root/.cache/uv/builds-v0/.tmpX6p75h/lib/python3.12/site-packages/setuptools/build_meta.py",
        line 432, in build_wheel
            return _build(['bdist_wheel'])
                   ^^^^^^^^^^^^^^^^^^^^^^^
          File
        "/root/.cache/uv/builds-v0/.tmpX6p75h/lib/python3.12/site-packages/setuptools/build_meta.py",
        line 423, in _build
            return self._build_with_temp_dir(
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^
          File
        "/root/.cache/uv/builds-v0/.tmpX6p75h/lib/python3.12/site-packages/setuptools/build_meta.py",
        line 404, in _build_with_temp_dir
            self.run_setup()
          File
        "/root/.cache/uv/builds-v0/.tmpX6p75h/lib/python3.12/site-packages/setuptools/build_meta.py",
        line 512, in run_setup
            super().run_setup(setup_script=setup_script)
          File
        "/root/.cache/uv/builds-v0/.tmpX6p75h/lib/python3.12/site-packages/setuptools/build_meta.py",
        line 317, in run_setup
            exec(code, locals())
          File "<string>", line 73, in <module>
          File
        "/root/.cache/uv/builds-v0/.tmpX6p75h/lib/python3.12/site-packages/setuptools/__init__.py",
        line 115, in setup
            return distutils.core.setup(**attrs)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
          File
        "/root/.cache/uv/builds-v0/.tmpX6p75h/lib/python3.12/site-packages/setuptools/_distutils/core.py",
        line 148, in setup
            _setup_distribution = dist = klass(attrs)
                                         ^^^^^^^^^^^^
          File
        "/root/.cache/uv/builds-v0/.tmpX6p75h/lib/python3.12/site-packages/setuptools/dist.py",
        line 321, in __init__
            _Distribution.__init__(self, dist_attrs)
          File
        "/root/.cache/uv/builds-v0/.tmpX6p75h/lib/python3.12/site-packages/setuptools/_distutils/dist.py",
        line 309, in __init__
            self.finalize_options()
          File
        "/root/.cache/uv/builds-v0/.tmpX6p75h/lib/python3.12/site-packages/setuptools/dist.py",
        line 784, in finalize_options
            ep(self)
          File
        "/root/.cache/uv/builds-v0/.tmpX6p75h/lib/python3.12/site-packages/setuptools/dist.py",
        line 804, in _finalize_setup_keywords
            ep.load()(self, ep.name, value)
          File
        "/root/.cache/uv/builds-v0/.tmpX6p75h/lib/python3.12/site-packages/setuptools_scm/_integration/setuptools.py",
        line 131, in version_keyword
            config = _config.Configuration.from_file(
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
          File
        "/root/.cache/uv/builds-v0/.tmpX6p75h/lib/python3.12/site-packages/setuptools_scm/_config.py",
        line 296, in from_file
            pyproject_data = _read_pyproject(
                             ^^^^^^^^^^^^^^^^
          File
        "/root/.cache/uv/builds-v0/.tmpX6p75h/lib/python3.12/site-packages/setuptools_scm/_integration/pyproject_reading.py",
        line 95, in read_pyproject
            pyproject_data.verify_dynamic_version_when_required()
          File
        "/root/.cache/uv/builds-v0/.tmpX6p75h/lib/python3.12/site-packages/setuptools_scm/_integration/pyproject_reading.py",
        line 38, in verify_dynamic_version_when_required
            raise ValueError(
        ValueError: pyproject.toml: setuptools-scm is present in
        [build-system].requires but dynamic=['version'] is not set in
        [project]. Either add dynamic=['version'] to [project] or add a
        [tool.setuptools_scm] section.
  
        hint: This usually indicates a problem with the package or the build
        environment.
    help: `lazy-object-proxy` (v1.7.1) was included because
          `apache-airflow-providers-mysql:dev` (v6.3.3) depends on
          `apache-airflow-task-sdk` which depends on `apache-airflow-core`
          (v3.1.0) which depends on `lazy-object-proxy`

or xmlsec:

 #47 234.4   × Failed to build `xmlsec==1.3.16`
  #47 234.4   ├─▶ The build backend returned an error
  #47 234.4   ╰─▶ Call to `setuptools.build_meta:__legacy__.build_wheel` failed (exit
  #47 234.4       status: 1)
  #47 234.4 
  #47 234.4       [stderr]
  #47 234.4       toml section missing 'pyproject.toml does not contain a
  #47 234.4       tool.setuptools_scm section'
  #47 234.4       Traceback (most recent call last):
  #47 234.4         File
  #47 234.4       "/root/.cache/uv/builds-v0/.tmpUsP5sY/lib/python3.10/site-packages/setuptools_scm/_integration/pyproject_reading.py",
  #47 234.4       line 71, in read_pyproject
  #47 234.4           section = defn.get("tool", {})[tool_name]
  #47 234.4       KeyError: 'setuptools_scm'
  #47 234.4       Traceback (most recent call last):
  #47 234.4         File "<string>", line 11, in <module>
  #47 234.4         File
  #47 234.4       "/root/.cache/uv/builds-v0/.tmpUsP5sY/lib/python3.10/site-packages/setuptools/build_meta.py",
  #47 234.4       line 432, in build_wheel
  #47 234.4           return _build(['bdist_wheel'])
  #47 234.4         File
  #47 234.4       "/root/.cache/uv/builds-v0/.tmpUsP5sY/lib/python3.10/site-packages/setuptools/build_meta.py",
  #47 234.4       line 423, in _build
  #47 234.4           return self._build_with_temp_dir(
  #47 234.4         File
  #47 234.4       "/root/.cache/uv/builds-v0/.tmpUsP5sY/lib/python3.10/site-packages/setuptools/build_meta.py",
  #47 234.4       line 404, in _build_with_temp_dir
  #47 234.4           self.run_setup()
  #47 234.4         File
  #47 234.4       "/root/.cache/uv/builds-v0/.tmpUsP5sY/lib/python3.10/site-packages/setuptools/build_meta.py",
  #47 234.4       line 512, in run_setup
  #47 234.4           super().run_setup(setup_script=setup_script)
  #47 234.4         File
  #47 234.4       "/root/.cache/uv/builds-v0/.tmpUsP5sY/lib/python3.10/site-packages/setuptools/build_meta.py",
  #47 234.4       line 317, in run_setup
  #47 234.4           exec(code, locals())
  #47 234.4         File "<string>", line 591, in <module>
  #47 234.4         File
  #47 234.4       "/root/.cache/uv/builds-v0/.tmpUsP5sY/lib/python3.10/site-packages/setuptools/__init__.py",
  #47 234.4       line 115, in setup
  #47 234.4           return distutils.core.setup(**attrs)
  #47 234.4         File
  #47 234.4       "/root/.cache/uv/builds-v0/.tmpUsP5sY/lib/python3.10/site-packages/setuptools/_distutils/core.py",
  #47 234.4       line 148, in setup
  #47 234.4           _setup_distribution = dist = klass(attrs)
  #47 234.4         File
  #47 234.4       "/root/.cache/uv/builds-v0/.tmpUsP5sY/lib/python3.10/site-packages/setuptools/dist.py",
  #47 234.4       line 321, in __init__
  #47 234.4           _Distribution.__init__(self, dist_attrs)
  #47 234.4         File
  #47 234.4       "/root/.cache/uv/builds-v0/.tmpUsP5sY/lib/python3.10/site-packages/setuptools/_distutils/dist.py",
  #47 234.4       line 309, in __init__
  #47 234.4           self.finalize_options()
  #47 234.4         File
  #47 234.4       "/root/.cache/uv/builds-v0/.tmpUsP5sY/lib/python3.10/site-packages/setuptools/dist.py",
  #47 234.4       line 784, in finalize_options
  #47 234.4           ep(self)
  #47 234.4         File
  #47 234.4       "/root/.cache/uv/builds-v0/.tmpUsP5sY/lib/python3.10/site-packages/setuptools/dist.py",
  #47 234.4       line 804, in _finalize_setup_keywords
  #47 234.4           ep.load()(self, ep.name, value)
  #47 234.4         File
  #47 234.4       "/root/.cache/uv/builds-v0/.tmpUsP5sY/lib/python3.10/site-packages/setuptools_scm/_integration/setuptools.py",
  #47 234.4       line 131, in version_keyword
  #47 234.4           config = _config.Configuration.from_file(
  #47 234.4         File
  #47 234.4       "/root/.cache/uv/builds-v0/.tmpUsP5sY/lib/python3.10/site-packages/setuptools_scm/_config.py",
  #47 234.4       line 296, in from_file
  #47 234.4           pyproject_data = _read_pyproject(
  #47 234.4         File
  #47 234.4       "/root/.cache/uv/builds-v0/.tmpUsP5sY/lib/python3.10/site-packages/setuptools_scm/_integration/pyproject_reading.py",
  #47 234.4       line 95, in read_pyproject
  #47 234.4           pyproject_data.verify_dynamic_version_when_required()
  #47 234.4         File
  #47 234.4       "/root/.cache/uv/builds-v0/.tmpUsP5sY/lib/python3.10/site-packages/setuptools_scm/_integration/pyproject_reading.py",
  #47 234.4       line 38, in verify_dynamic_version_when_required
  #47 234.4           raise ValueError(
  #47 234.4       ValueError: pyproject.toml: setuptools-scm is present in
  #47 234.4       [build-system].requires but dynamic=['version'] is not set in
  #47 234.4       [project]. Either add dynamic=['version'] to [project] or add a
  #47 234.4       [tool.setuptools_scm] section.
  #47 234.4 
  #47 234.4       hint: This usually indicates a problem with the package or the build
  #47 234.4       environment.
  #47 234.4   help: `xmlsec` (v1.3.16) was included because
  #47 234.4         `apache-airflow-providers-amazon[python3-saml]` (v9.11.0) depends
  #47 234.4         on `xmlsec`
  #47 ERROR: process "/bin/bash -o pipefail -o errexit -o nounset -o nolog -c bash /scripts/docker/install_airflow_when_building_images.sh" did not complete successfully: exit code: 1

[potiuk]

I think this is pretty serious - and you will be flooded with issues. I suggest to yank 0.9.* and investigate later.

[timabbott]

I investigated this because Zulip's CI suddenly started failing today; apparently xmlsec no longer builds via uv. Issue is here, reported by yet another person: xmlsec/python-xmlsec#366

I agree with the assessment that it seems likely that a lot of packages were not prepared for this transition.

[potiuk]

Looks like the problem is that https://github.com/pypa/setuptools-scm/pull/1190/files#diff-1991e6eaee1fcf864f5534ddf2df3eeb6199b72376af2fdf2c2896c301532f6dR32 requires that pyproject.toml in the project must have dynamic version set in pyproject.toml when setuptools-scm is used, but it does not take into account projects that havae botth setup.py and pyproject.toml and they only use pyproject.toml for static check configuration (this is what xmlsec does).

[sfc-gh-yuwang]

We are running into the same issue, anyone have any suggestion of how to temoprarily work around this? I tried pin setuptools-scm version but does not help

[arthur-st]

I don't think this can be worked around, as packages using setuptools-scm for the build process will be creating an isolated build environment where they pull their build dependencies in. I guess maybe it is possible to work around this if you have a Python private registry and yoink the >=9.0 versions “locally” there, but I'm not sure if build environments will respect your private index.

[WSH032]

If I understand correctly, [build-system.requires] can actually be overridden: https://discuss.python.org/t/no-way-to-pin-build-dependencies/29833.

[sizehnde]

If you find yourself in a situation where you need to explicitly pin setuptools-scm<9.0.3 and regenerate pip-compile created requirements.txt files, but you can't do that because the temporary build environment itself pulls in 9.0.3 transitively through lazy-object-proxy==1.9.0, you can do this:

echo "setuptools_scm<9.0.3" > reqs.txt
PIP_CONSTRAINT="reqs.txt" pip-compile --generate-hashes --output-file=requirements.txt requirements.in 

[maxzintel]

xmlsec v1.3.14 appears to work for us, but not 13, 15, or 16.

[maxzintel]

xmlsec v1.3.14 appears to work for us, but not 13, 15, or 16.

As did

RUN echo "setuptools-scm<9" > /tmp/constraints.txt
RUN PIP_CONSTRAINT=/tmp/constraints.txt pip install --no-cache-dir lxml==4.9.1 xmlsec==1.3.13

as a workaround

[belimawr]

This seems to be affecting the CI for Beats (https://github.com/elastic/beats/) as well :/

  │ exit code: 1
  ╰─> [40 lines of output]
      toml section missing 'pyproject.toml does not contain a tool.setuptools_scm section'
      Traceback (most recent call last):
        File "/tmp/pip-build-env-63fsh64h/overlay/lib/python3.10/site-packages/setuptools_scm/_integration/pyproject_reading.py", line 71, in read_pyproject
          section = defn.get("tool", {})[tool_name]
      KeyError: 'setuptools_scm'
      Traceback (most recent call last):
        File "/home/runner/work/beats/beats/build/python-env/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
          main()
        File "/home/runner/work/beats/beats/build/python-env/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
        File "/home/runner/work/beats/beats/build/python-env/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 149, in prepare_metadata_for_build_wheel
          return hook(metadata_directory, config_settings)
        File "/tmp/pip-build-env-63fsh64h/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 374, in prepare_metadata_for_build_wheel
          self.run_setup()
        File "/tmp/pip-build-env-63fsh64h/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 512, in run_setup
          super().run_setup(setup_script=setup_script)
        File "/tmp/pip-build-env-63fsh64h/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 317, in run_setup
          exec(code, locals())
        File "<string>", line 61, in <module>
        File "/tmp/pip-build-env-63fsh64h/overlay/lib/python3.10/site-packages/setuptools/__init__.py", line 115, in setup
          return distutils.core.setup(**attrs)
        File "/tmp/pip-build-env-63fsh64h/overlay/lib/python3.10/site-packages/setuptools/_distutils/core.py", line 148, in setup
          _setup_distribution = dist = klass(attrs)
        File "/tmp/pip-build-env-63fsh64h/overlay/lib/python3.10/site-packages/setuptools/dist.py", line 321, in __init__
          _Distribution.__init__(self, dist_attrs)
        File "/tmp/pip-build-env-63fsh64h/overlay/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 309, in __init__
          self.finalize_options()
        File "/tmp/pip-build-env-63fsh64h/overlay/lib/python3.10/site-packages/setuptools/dist.py", line 784, in finalize_options
          ep(self)
        File "/tmp/pip-build-env-63fsh64h/overlay/lib/python3.10/site-packages/setuptools/dist.py", line 804, in _finalize_setup_keywords
          ep.load()(self, ep.name, value)
        File "/tmp/pip-build-env-63fsh64h/overlay/lib/python3.10/site-packages/setuptools_scm/_integration/setuptools.py", line 131, in version_keyword
          config = _config.Configuration.from_file(
        File "/tmp/pip-build-env-63fsh64h/overlay/lib/python3.10/site-packages/setuptools_scm/_config.py", line 296, in from_file
          pyproject_data = _read_pyproject(
        File "/tmp/pip-build-env-63fsh64h/overlay/lib/python3.10/site-packages/setuptools_scm/_integration/pyproject_reading.py", line 95, in read_pyproject
          pyproject_data.verify_dynamic_version_when_required()
        File "/tmp/pip-build-env-63fsh64h/overlay/lib/python3.10/site-packages/setuptools_scm/_integration/pyproject_reading.py", line 38, in verify_dynamic_version_when_required
          raise ValueError(
      ValueError: pyproject.toml: setuptools-scm is present in [build-system].requires but dynamic=['version'] is not set in [project]. Either add dynamic=['version'] to [project] or add a [tool.setuptools_scm] section.
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

Notice:  A new release of pip is available: 23.0.1 -> 25.2
Notice:  To update, run: pip install --upgrade pip

I manged to track it to installing pytlint 2.4.4

python3 -m venv foo
source foo/bin/activate
python -m pip install  pylint==2.4.4

[BwL1289]

To chime in here... this should get yanked.