ci: test pypy upstream #3743

henryiii · 2022-02-16T16:54:23Z

Description

Adding a nox test for PyPy upstream before their next release. I think this works on 3.7 and 3.8, though we should see if we still need our workarounds, and fails on PyPy 3.9, but haven't really looked into it more than just adding the tests. One test was regularly xpassing for me. CC @mattip.

Suggested changelog entry:

* Support testing development versions of PyPy (UNIX only) via nox

henryiii · 2022-02-16T16:59:37Z

Hmmm, CMake failing to find PyPy 3.9 but reporting 3.9 is a valid version was not what I was expecting. That works correctly locally.

henryiii · 2022-02-16T17:02:36Z

This is the local error:

nox > cmake --build .nox/pypy_upstream-3-9/tmp --target pytest
[0/2] Re-checking globbed directories...
[0/1] cd /Users/henryschreiner/git/software/pybind11/.nox/pypy_upstream-3-9/tmp/tests && /Users...tests/test_union.py /Users/henryschreiner/git/software/pybind11/tests/test_virtual_functions.py
libc++abi: terminating with uncaught exception of type pybind11::error_already_set: IndentationError: ('unexpected indent', ('<string>', 2, 1, '        class pybind11_static_property(property):\n', 2))

This actually seems correct, the code has a leading indent. Not sure why PyPy < 3.9 likes it, actually.

Skylion007 · 2022-02-16T17:12:08Z

Okay, so we should definitely make a PR to fix that leading indent then.

henryiii · 2022-02-16T17:18:46Z

With that fix, Python tests pass, then the CMake tests fails with:

======== CMake output     ======
Found pybind11: /Users/henryschreiner/git/software/pybind11/.nox/pypy_upstream-3-9/tmp/mock_install/include (found version "2.10.0dev1")
Found pybind11 v2.10.0 dev1: /Users/henryschreiner/git/software/pybind11/.nox/pypy_upstream-3-9/tmp/mock_install/include;/Users/henryschreiner/git/software/pybind11/.nox/pypy_upstream-3-9/tmp/pypy-c-jit-104891-17e3c4e34398-osx64/include/pypy3.9/
Configuring done
Generating done
Build files have been written to: /Users/henryschreiner/git/software/pybind11/.nox/pypy_upstream-3-9/tmp/tests/test_cmake_build/installed_function
======== End CMake output ======
Change Dir: /Users/henryschreiner/git/software/pybind11/.nox/pypy_upstream-3-9/tmp/tests/test_cmake_build/installed_function

Run Build Command(s):/Users/henryschreiner/git/software/pybind11/.nox/pypy_upstream-3-9/bin/ninja check_installed_function && [1/2] Linking CXX shared library test_cmake_build.pypy39-pp73-darwin.so
FAILED: test_cmake_build.pypy39-pp73-darwin.so
: && /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++ -O3 -DNDEBUG -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.1.sdk -dynamiclib -Wl,-headerpad_max_install_names  -o test_cmake_build.pypy39-pp73-darwin.so -install_name @rpath/test_cmake_build.pypy39-pp73-darwin.so CMakeFiles/test_installed_function.dir/Users/henryschreiner/git/software/pybind11/tests/test_cmake_build/main.cpp.o  /usr/local/lib/libpypy3-c.dylib && :
Undefined symbols for architecture x86_64:
  "_PyPyCMethod_New", referenced from:
      pybind11::cpp_function::initialize_generic(std::__1::unique_ptr<pybind11::detail::function_record, pybind11::cpp_function::InitializingFunctionRecordDeleter>&&, char const*, std::type_info const* const*, unsigned long) in main.cpp.o
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
ninja: build stopped: subcommand failed.

Skylion007 · 2022-02-16T20:08:42Z

Nice, PyPy, 3.7 and 3.8 are working now.

henryiii · 2022-02-16T20:17:47Z

3.7 and 3.8 were always working. It's just 3.9 that's broken - via CMake on CI and by _PyPyCMethod_New missing (have no idea what that is) locally - only in one of the three CMake compile tests, I think.

noxfile.py

tests/test_exceptions.py

mattip · 2022-02-16T21:22:28Z

As far as I can tell, PyPy3.9 has the include file pypy_decl.h with #define PyCMethod_New PyPyCMethod_New, and CPython has the include file methodobject.h with PyAPI_FUNC(PyObject *) PyCMethod_New(PyMethodDef *, PyObject *, .... I wonder where _PyPyCMethod_New with the underline is coming from?

mattip · 2022-02-16T23:46:19Z

I think CMAKE is not finding the include files nor the library for PyPy3.9.

henryiii · 2022-02-17T03:11:54Z

Yes, but it is finding it for 3.8 & 3.7, AND it is finding it locally on my Mac for 3.9, even though everything should be identical - same pypy download, same cmake, etc...

mattip · 2022-02-17T14:09:58Z

it is finding it locally on my Mac for 3.9,

Do the tests pass?

henryiii · 2022-02-17T18:15:09Z

The normal tests pass, then the build tests fail with the error listed above.

mattip · 2022-02-17T18:22:44Z

What would be the easiest way to run those specific tests locally?

mattip · 2022-02-17T18:34:26Z

Got it how to run tests. Locally on Ubuntu 20.04, pipx run nox -s 'pypy_upstream(3.9)' from this branch fails the test_python_alreadyset_in_destructor test.

Edit: clarify which "it" I got.

mattip · 2022-02-17T19:02:15Z

I am not seeing any failure with _PyPyCMethod_New.

henryiii · 2022-02-18T22:06:25Z

Let me revert the last commit, that should get past that test.

tests: fix leading indent in PyPy code Update noxfile.py Co-authored-by: Matti Picus <matti.picus@gmail.com> Update tests/test_exceptions.py Co-authored-by: Matti Picus <matti.picus@gmail.com> Revert "Update tests/test_exceptions.py" This reverts commit af3061d.

henryiii · 2022-03-06T05:33:14Z

Ahaha, I bet this is what's breaking CMake: scikit-build/scikit-build#673 (at least it's a start)

DarkWingMcQuack · 2022-03-08T22:11:15Z

This is the local error:

nox > cmake --build .nox/pypy_upstream-3-9/tmp --target pytest
[0/2] Re-checking globbed directories...
[0/1] cd /Users/henryschreiner/git/software/pybind11/.nox/pypy_upstream-3-9/tmp/tests && /Users...tests/test_union.py /Users/henryschreiner/git/software/pybind11/tests/test_virtual_functions.py
libc++abi: terminating with uncaught exception of type pybind11::error_already_set: IndentationError: ('unexpected indent', ('<string>', 2, 1, '        class pybind11_static_property(property):\n', 2))

This actually seems correct, the code has a leading indent. Not sure why PyPy < 3.9 likes it, actually.

I have this bug in a project currently which i try porting to pypy. Is there any fix for this currently?
Using python 3.8 is not an option unfortunatly for me.
This is the only place where i found this error, so i thought i would just ask here

mattip · 2022-03-09T06:50:03Z

@DarkWingMcQuack the indent is fixed on HEAD. You can install pybind11 from git until the fix is released: pip install git+https://github.com/pybind/pybind11.git.

henryiii · 2022-03-09T14:44:59Z

It should also be available in the v2.9 branch. Hopefully we can cut a new patch release soon.

henryiii · 2022-03-09T14:46:28Z

The library name change in 3.9 is also likely breaking things, though, possibly including all older versions of CMake's FindPython support. I want to look into that before making a release. Might not be fixable from our end, though, for CMake.

mattip · 2022-03-09T16:38:15Z

The library name change in 3.9 is also likely breaking things

bummer.

nachovizzo · 2022-03-18T13:57:23Z

I'm also seeing this error on a CI build, for Python 3.9

[root@bbf72c22780d ~]# "${PYBIN}/pytest" --capture=sys /io/
================================================ test session starts ================================================
platform linux -- Python 3.9.10[pypy-7.3.8-final], pytest-7.1.1, pluggy-1.0.0
rootdir: /io
collecting ... terminate called after throwing an instance of 'pybind11::error_already_set'
  what():  IndentationError: ('unexpected indent', ('<string>', 2, 1, '        class pybind11_static_property(property):\n', 2))
Fatal Python error: Aborted

mattip · 2022-03-18T15:22:29Z

@nachovizzo The indent is fixed on HEAD. You can install pybind11 from git until the fix is released: pip install git+https://github.com/pybind/pybind11.git.

nachovizzo · 2022-03-18T15:59:22Z

@nachovizzo The indent is fixed on HEAD. You can install pybind11 from git until the fix is released: pip install git+https://github.com/pybind/pybind11.git.

Great, already try this and fix the problem. Looking forward to a new release ;)

henryiii added the python dev Working on development versions of Python label Feb 16, 2022

mattip reviewed Feb 16, 2022

View reviewed changes

noxfile.py Outdated Show resolved Hide resolved

mattip reviewed Feb 16, 2022

View reviewed changes

tests/test_exceptions.py Show resolved Hide resolved

henryiii mentioned this pull request Mar 1, 2022

ci: fix PyPy #3768

Merged

henryiii force-pushed the henryiii/ci/pypyupstream branch from 7482ae9 to 89b8d8a Compare March 1, 2022 18:04

ci: include PyPy 3.9

f2026d5

Tobias-Fischer mentioned this pull request Nov 21, 2022

Rebuild for PyPy3.8 and PyPy3.9 conda-forge/pytinyrenderer-feedstock#1

Open

rwgk mentioned this pull request Feb 10, 2023

FWD pybind11 google/pybind11clif#3743

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ci: test pypy upstream #3743

ci: test pypy upstream #3743

henryiii commented Feb 16, 2022

henryiii commented Feb 16, 2022 •

edited

Loading

henryiii commented Feb 16, 2022 •

edited

Loading

Skylion007 commented Feb 16, 2022

henryiii commented Feb 16, 2022

Skylion007 commented Feb 16, 2022

henryiii commented Feb 16, 2022

mattip commented Feb 16, 2022

mattip commented Feb 16, 2022

henryiii commented Feb 17, 2022

mattip commented Feb 17, 2022

henryiii commented Feb 17, 2022

mattip commented Feb 17, 2022

mattip commented Feb 17, 2022 •

edited

Loading

mattip commented Feb 17, 2022

henryiii commented Feb 18, 2022

henryiii commented Mar 6, 2022

DarkWingMcQuack commented Mar 8, 2022 •

edited

Loading

mattip commented Mar 9, 2022

henryiii commented Mar 9, 2022

henryiii commented Mar 9, 2022

mattip commented Mar 9, 2022

nachovizzo commented Mar 18, 2022

mattip commented Mar 18, 2022

nachovizzo commented Mar 18, 2022

ci: test pypy upstream #3743

Are you sure you want to change the base?

ci: test pypy upstream #3743

Conversation

henryiii commented Feb 16, 2022

Description

Suggested changelog entry:

henryiii commented Feb 16, 2022 • edited Loading

henryiii commented Feb 16, 2022 • edited Loading

Skylion007 commented Feb 16, 2022

henryiii commented Feb 16, 2022

Skylion007 commented Feb 16, 2022

henryiii commented Feb 16, 2022

mattip commented Feb 16, 2022

mattip commented Feb 16, 2022

henryiii commented Feb 17, 2022

mattip commented Feb 17, 2022

henryiii commented Feb 17, 2022

mattip commented Feb 17, 2022

mattip commented Feb 17, 2022 • edited Loading

mattip commented Feb 17, 2022

henryiii commented Feb 18, 2022

henryiii commented Mar 6, 2022

DarkWingMcQuack commented Mar 8, 2022 • edited Loading

mattip commented Mar 9, 2022

henryiii commented Mar 9, 2022

henryiii commented Mar 9, 2022

mattip commented Mar 9, 2022

nachovizzo commented Mar 18, 2022

mattip commented Mar 18, 2022

nachovizzo commented Mar 18, 2022

henryiii commented Feb 16, 2022 •

edited

Loading

henryiii commented Feb 16, 2022 •

edited

Loading

mattip commented Feb 17, 2022 •

edited

Loading

DarkWingMcQuack commented Mar 8, 2022 •

edited

Loading