Use gh-action-pypi-publish v1 release branch, tp.

[mjpieters]

The release branch makes sure future releases don't fail when newer metadata versions are used or where the PyPI publishing tooling needs to be updated to match changes in PyPI itself.

Also, switch to using trusted publishing, where GitHub signs wheels so users can know that the wheels where built by the official CyLP github repository.

Important

Trusted publishing requires changes to the CyLP PyPI project, see Adding a Trusted Publisher to an existing PyPI project in the PyPI documentation.

Once this workflow is shown to be working, you'll also want to remove the PYPI_API_TOKEN secret from this project, and revoke that token in PyPI.

Fixes #214

[mjpieters]

The Trusted Publisher setup in PyPI is very easy, all you need is the following four pieces of information:

• Owner: coin-or
• Repository name: CyLP
• Worflow name: cibuildwheel.yml
• Environment name: pypi

Fill these in on the CyLP Publishing settings page.

The workflow uses a new GitHub deployment environment named pypi. You can add additional checks and restrictions to that environment (either create it before running this workflow or run it once and have it automatically created).

You could first test the workflow by temporarily having it push to test.pypi.org instead:

  with:
    repository-url: https://test.pypi.org/legacy/

provided you first configure trusted publishing on test.pypi.org first. I'd also change the GHA deployment environment name to testpypi, for good measure.

[tkralphs]

OK, thanks, @mjpieters. I saw the message about the legacy publishing in the out and I will merge this PR to address that, but I got sidetracked running down the other error that is indicated there. It looks like there is more going on than just the legacy publishing. The error message seemed to indicate some problem with the metadata.

Looking into that, it seems we should either specify the metadata fields in pyproject.toml (and not in setup .py) or declare them as dynamic in pyproject.toml (obviously, I'm not that up-to-date on recent changes to packaging). After some trial and error, I moved metdata specification to pyproject.toml, which seems to be the best practice (I eventually discovered validate-pyproject, which I should have used from the beginning instead of repeatedly trying to make a new release).

Anyway, that did not fix the error. I downloaded the macOS wheels locally and tried to read their metadata with wheel2json. I get an error parsing the metadata, which does not seem like it should happen:

wheel2json cylp-0.93.0-cp310-cp310-macosx_10_9_x86_64.whl
...
<snip>
...
    raise errors.DuplicateFieldError(name)
headerparser.errors.DuplicateFieldError: Header field 'Generator' occurs more than once

After a lot of Googling and asking AI, I finally, I just opened up the .whl file and looked at its contents. Indeed, cylp-0.93.0.dist-info/WHEEL has the contents

Wheel-Version: 1.0
Generator: setuptools (80.9.0)
Root-Is-Purelib: false
Tag: cp311-cp311-macosx_10_9_x86_64
Generator: delocate 0.13.0

I can see in the logs that delocate is being used to make the libraries relocatable, but everything I read says that it should not add this extra entry to the WHEEL file.

Anyway, I will merge your PRs and see what happens, but I don't see how it can change the contents of the .whl file itself. Do you have any idea about this?

[tkralphs]

@mjpieters, I just pushed a commit that will try to publish to test.pypi.org, as you suggested. Let's see.

[mjpieters]

There is issue with the metadata. I had verified the wheels with the most recent twine release and it's all fine.

The error is purely due to the old tooling not knowing what to do with the Metadata version 2.4 data.

The wheels also work perfectly, I had no issue installing the macos arm64 versions and run my project tests against them.

I can't comment on the use of delocate but trust cibuildwheels to get this right. There are thousands of projects relying on their output including numpy and scipy themselves and if cibuildwheels got this wrong it would already have been fixed. 😄

[mjpieters]

I just pushed a commit that will try to publish to test.pypi.org, as you suggested. Let's see.

It's working! :-)

[mjpieters]

@tkralphs: On test.pypi you can see that the wheels all have full attestations attached too now.

I do note that the Documentation="https://coin-or.github.io/cylp" link found in pyproject.toml does not exist. Is that intentional?

[tkralphs]

OK, great. Add this to the pantheon of incredibly unhelpful error messages. And to the list of times I ignored the elephant in the room :). It still seems a bit weird that wheel2json fails to parse the metadata on the macOS wheels. Everything I could find on-line says that Generator cannot be listed twice. But I certainly know that lots of big projects are using cibuildwheels and that it must be OK.

Out of curiosity, why do you need

url: https://pypi.org/p/cylp

The documentation here doesn't add it.

[mjpieters]

Out of curiosity, why do you need

url: https://pypi.org/p/cylp

The documentation here doesn't add it.

The action documentation does. When you include a URL, GitHub will display it in relevant locations such as the deployments list for the testpypi environment, and if a pull request triggered the deployment, in the pull request time line, etc.

[tkralphs]

I do note that the Documentation="https://coin-or.github.io/cylp" link found in pyproject.toml does not exist. Is that intentional?

Ugh, I should have tested. The case is wrong. It should be https://coin-or.github.io/CyLP.

[tkralphs]

I cancelled the workflow and fixed that typo. Let's hope this is the last thing!

[tkralphs]

OK, all good now! Thanks for the help!

[mjpieters]

OK, all good now! Thanks for the help!

It's all great! I can see the new release on PyPI, and you can see that the pypi deployments page here on GH includes the environment URL set in the workflow. :-)

With the release/v1 branch you should be set for a while now, hopefully for a long time, fingers crossed!

[tkralphs]

Thanks so much.