Extract version from source file?

[ChrisBarker-NOAA]

@henryiii told me that being able to extract the version from the source was on the list, but not yet done.

But I don't see an issue on it.

How can I help? It seems like pretty straightforward code to write, once I know where to put it.

NOTE: I'd like to see two options:

1. setuptools-like version from a code file:

version = {attr = "my_package.VERSION"}

2. setuptools_scm style: writing to a dynamically generated file:

write_to = "src/binary4fun/_version.py"

In this case, unlike setuptools_scm, I'd like to be able to be flexible about where the version comes from -- e.g. it could be extracted from SCM, or simply hard-coded in the pyproject.toml

[bennyrowland]

Hi Chris,
This seems like two orthogonal suggestions to me:

1. determining the version
2. outputting the determined version to a file

We already support running setuptools_scm itself (see https://github.com/scikit-build/scikit-build-core/blob/main/docs/configuration.md#dynamic-metadata).

The way to handle a new dynamic version approach would be to create a new dynamic metadata plugin (whether this would be part of scikit-build-core or a separate package would be up to @henryiii). This could look like (in pyproject.toml):

[tool.scikit-build]
metadata.version.provider = "metadata_plugin"
metadata.version.attr = "my_package.VERSION"

or alternatively

[tool.scikit-build]
metadata.version.provider = "metadata_plugin"

[tool.metadata-plugin]
attr = "my_package.VERSION"

if you wanted to separate out the config for the plugin.

Generating a file containing the version info in some way is really separate from working out what the version should be. setuptools_scm bundles the two components together, and so another plugin could do the same, but maybe there is some value in having a separate component that can write the version into a file independently of how the version is worked out. One simple way for end users to do this without changing scikit-build-core would be to do it in CMake, using the SKBUILD_PROJECT_VERSION variable which is injected into CMake by scikit-build-core. Writing that into a template file or similar would be very easy and flexible. Adding something to the docs on how to do that could be useful.

[ChrisBarker-NOAA]

determining the version
outputting the determined version to a file

Absolutely -- definitely want to separate those!

setuptools_scm bundles the two components together

it bundles everything together -- I don't think we should follow that model :-)

a separate component that can write the version into a file independently of how the version is worked out.

yes!

One simple way for end users to do this without changing scikit-build-core would be to do it in CMake, using the SKBUILD_PROJECT_VERSION variable which is injected into CMake by scikit-build-core. Writing that into a template file or similar would be very easy and flexible.

hmm, a good way to prototype it in any case -- I'll try to take a stab at that when I get a chance.

[henryiii]

Adding the ability to read the version from a file would be easy - I'd just do something like this:

# src/scikit_build_core/metadata/regex.py
from __future__ import annotations

__all__ = ["dynamic_metadata"]


def __dir__() -> list[str]:
    return __all__


def dynamic_metadata(
    fields: frozenset[str],
    settings: dict[str, object] | None = None,
) -> dict[str, str | dict[str, str | None]]:
    # this is a classic implementation, waiting for the release of
    # vcs-versioning and an improved public interface

    if fields != {"version"}:
        msg = "Only the 'version' field is supported for now"
        raise ValueError(msg)

    if set(settings) > {"file", "regex"} :
        msg = "Only file and regex supported"
        raise ValueError(msg)

    if "file" not in settings:
        msg = "Must specify a file to search for the version"
        raise ValueError(msg)

    if "regex" not in settings:
       # Inspired by https://github.com/pypa/hatch/blob/6b40d5bb0e26e8646fbe507bdb87aa6012fe539b/backend/src/hatchling/version/core.py#L6C19-L6C80
       regex = r"""^__version__ *= *(['"])(?P<version>.+?)\2"""
       # Maybe we could have a custom one if `CMakeLists.txt/*.cmake` was specified in file?

    version = # apply regex to file and collect the version

    return {"version": version}

The reason I haven't is because we need to restructure this API before standardizing it. There are two issues: one, it's really hard to match options with each requested field - the backend is responsible for splitting up the calls. Two, it's missing the dynamic setting from metadata 2.4 (which I'd forgotten about since not much supported it when I wrote it, not sure if much does now). I think the new API would look something like this:

def dynamic_metadata(
    field: str,
    settings: dict[str, object] | None = None,
) -> dict[str, str | dict[str, str | None]]:
    # ...
    return {field: ..., "dynamic": False}

An extension would be responsible for caching access multiple calls with differing fields (and maybe settings), but most extensions wouldn't care, and I think it's doable for the ones that do (like a hypothetical "cmake" extension that reads values from the CMake FileAPI). And it makes settings better (per extension already, no backend work), and it makes it easier to add extra output fields if any were needed. dynamic can't be a return field, so this would be the metadata 2.2 value.

If someone wants to implement this before I refactor the structure, though, go ahead. :) I don't mind updating three extensions instead of two when rewriting this. I'd be happy to have it built-in for now; it won't require any extra dependencies. We can factor it out later. I don't want external or local plugins just yet, since the API is changing soon.

[henryiii]

Having a way to write metadata information to files (potentially only generated files) would be a good separate feature, though for version it's actually easy to do from CMake currently, since CMake can see the version. Being able to read from those locations when building from SDist isn't that important, I think, since I think we will (eventually?) be able to rely on metadata 2.2 and with that you can tell if you need to regenerate the values or use them from SDist. (Also, in the case of version, it's not allowed to be dynamic in metadata 2.2, which would mean you could always use it when building from SDist).

[ChrisBarker-NOAA]

Thanks!

Is there now an example anywhere of using this for the version?

(in my case, I'd like to extract it from the packages __init__.py -- but another example would be fine.)

[henryiii]

https://scikit-build-core.readthedocs.io/en/latest/configuration.html#dynamic-metadata

Click on Regex. (Needs git main for now, release soonish)

[ChrisBarker-NOAA]

Awesome, thanks!