[Feature request] Support for Nvidia (former Portland Group) compilers

[andriish]

Hi all,

I've tried to compile some Python bindings with the Nvidia compiler (https://developer.nvidia.com/hpc-sdk) and it turns out it more or less works with a tiny exception that one should guard the occurrences of
std::exception_ptr and std::current_exception() with something like

#if !defined (__PGIC__)
....
# endif

I've used 6 such guards in total. Four in the details/internals.h and two in the pybind11.h

I was wondering if that would be interesting for the developers to have this modifications and support the NVidia compiler suite in this way.

Best regards,
Andrii

[henryiii]

Is there a way to test builds on them (ideally on GitHub Actions) (a docker container or such?)

[andriish]

Hi @henryiii ,

I have a native installation (v20), but it looks the contaners are available as well
https://ngc.nvidia.com/catalog/containers/hpc:pgi-compilers (at least v19).

Best regards,

Andrii

[YannickJadoul]

one should guard the occurrences of std::exception_ptr and std::current_exception() with something like

Is this safe to do? I thought pybind11 does depend on usage of these two symbols (e.g., for custom exception translators). So that means that we should disable all functionality using this?

[andriish]

Hi @YannickJadoul ,

So that means that we should disable all functionality using this?

Only for this particular stl implementation as this implementation does not have the neede classes (if i'm not mistaken).

Optionally one can provide a simplistic implementation for these classes. Or even dummies.

Best regards,
Andrii

[henryiii]

Running this:

docker run --rm -it -v $PWD:/pybind11 nvcr.io/hpc/pgi-compilers:ce
apt-get update && apt-get install -y python3-dev python3-pip python3-pytest
wget -qO- "https://cmake.org/files/v3.18/cmake-3.18.1-Linux-x86_64.tar.gz" | tar --strip-components=1 -xz -C /usr/local
cmake -S pybind11/ -B build
cmake --build build

(Eek, one layer alone of that docker image is over 5GB!)

Produces this: (click to expand)

cmroot@9a2d2cfec530:/# cmake -S pybind11/ -B build
-- The CXX compiler identification is PGI 19.10.0
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /opt/pgi/linux86-64-llvm/19.10/bin/pgc++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- pybind11 v2.6.0 dev
-- CMake 3.18.1
-- Found PythonInterp: /usr/bin/python3.6 (found version "3.6.9")
-- Found PythonLibs: /usr/lib/x86_64-linux-gnu/libpython3.6m.so
-- Setting tests build type to MinSizeRel as none was specified
-- Building tests WITHOUT Eigen, use -DDOWNLOAD_EIGEN on CMake 3.11+ to download
-- Could NOT find Boost (missing: Boost_INCLUDE_DIR) (Required is at least version "1.56")
-- Catch not detected. Interpreter tests will be skipped. Install Catch headers manually or use `cmake -DDOWNLOAD_CATCH=ON` to fetch them automatically.
-- Configuring done
-- Generating done
-- Build files have been written to: /build
root@9a2d2cfec530:/# cmake --build build
Scanning dependencies of target pybind11_cross_module_tests
[  2%] Building CXX object tests/CMakeFiles/pybind11_cross_module_tests.dir/pybind11_cross_module_tests.cpp.o
pgc++-Warning--Mipa is not supported for this target

At end of source: warning: routine is both "inline" and "noinline"

[  4%] Linking CXX shared module /pybind11/tests/pybind11_cross_module_tests.cpython-36m-x86_64-linux-gnu.so
pgc++-Warning--Mipa is not supported for this target

[  4%] Built target pybind11_cross_module_tests
Scanning dependencies of target pybind11_tests
[  7%] Building CXX object tests/CMakeFiles/pybind11_tests.dir/pybind11_tests.cpp.o
pgc++-Warning--Mipa is not supported for this target

At end of source: warning: routine is both "inline" and "noinline"

[  9%] Building CXX object tests/CMakeFiles/pybind11_tests.dir/test_async.cpp.o
pgc++-Warning--Mipa is not supported for this target

At end of source: warning: routine is both "inline" and "noinline"

[ 11%] Building CXX object tests/CMakeFiles/pybind11_tests.dir/test_buffers.cpp.o
pgc++-Warning--Mipa is not supported for this target

At end of source: warning: routine is both "inline" and "noinline"

[ 14%] Building CXX object tests/CMakeFiles/pybind11_tests.dir/test_builtin_casters.cpp.o
pgc++-Warning--Mipa is not supported for this target

At end of source: warning: routine is both "inline" and "noinline"

[ 16%] Building CXX object tests/CMakeFiles/pybind11_tests.dir/test_call_policies.cpp.o
pgc++-Warning--Mipa is not supported for this target

At end of source: warning: routine is both "inline" and "noinline"

[ 19%] Building CXX object tests/CMakeFiles/pybind11_tests.dir/test_callbacks.cpp.o
pgc++-Warning--Mipa is not supported for this target

At end of source: warning: routine is both "inline" and "noinline"

[ 21%] Building CXX object tests/CMakeFiles/pybind11_tests.dir/test_chrono.cpp.o
pgc++-Warning--Mipa is not supported for this target

At end of source: warning: routine is both "inline" and "noinline"

[ 23%] Building CXX object tests/CMakeFiles/pybind11_tests.dir/test_class.cpp.o
pgc++-Warning--Mipa is not supported for this target

At end of source: warning: routine is both "inline" and "noinline"

[ 26%] Building CXX object tests/CMakeFiles/pybind11_tests.dir/test_constants_and_functions.cpp.o
pgc++-Warning--Mipa is not supported for this target

"/pybind11/tests/test_constants_and_functions.cpp", line 125: warning:
          incompatible exception specifications
      m.def("f3", f3);
                  ^

At end of source: warning: routine is both "inline" and "noinline"

[ 28%] Building CXX object tests/CMakeFiles/pybind11_tests.dir/test_copy_move.cpp.o
pgc++-Warning--Mipa is not supported for this target

At end of source: warning: routine is both "inline" and "noinline"

[ 30%] Building CXX object tests/CMakeFiles/pybind11_tests.dir/test_custom_type_casters.cpp.o
pgc++-Warning--Mipa is not supported for this target

At end of source: warning: routine is both "inline" and "noinline"

[ 33%] Building CXX object tests/CMakeFiles/pybind11_tests.dir/test_docstring_options.cpp.o
pgc++-Warning--Mipa is not supported for this target

At end of source: warning: routine is both "inline" and "noinline"

[ 35%] Building CXX object tests/CMakeFiles/pybind11_tests.dir/test_enum.cpp.o
pgc++-Warning--Mipa is not supported for this target

At end of source: warning: routine is both "inline" and "noinline"

[ 38%] Building CXX object tests/CMakeFiles/pybind11_tests.dir/test_eval.cpp.o
pgc++-Warning--Mipa is not supported for this target

At end of source: warning: routine is both "inline" and "noinline"

[ 40%] Building CXX object tests/CMakeFiles/pybind11_tests.dir/test_exceptions.cpp.o
pgc++-Warning--Mipa is not supported for this target

"/pybind11/tests/test_exceptions.cpp", line 202: warning: statement is
          unreachable
          return false;
          ^

At end of source: warning: routine is both "inline" and "noinline"

[ 42%] Building CXX object tests/CMakeFiles/pybind11_tests.dir/test_factory_constructors.cpp.o
pgc++-Warning--Mipa is not supported for this target

At end of source: warning: routine is both "inline" and "noinline"

[ 45%] Building CXX object tests/CMakeFiles/pybind11_tests.dir/test_gil_scoped.cpp.o
pgc++-Warning--Mipa is not supported for this target

At end of source: warning: routine is both "inline" and "noinline"

[ 47%] Building CXX object tests/CMakeFiles/pybind11_tests.dir/test_iostream.cpp.o
pgc++-Warning--Mipa is not supported for this target

At end of source: warning: routine is both "inline" and "noinline"

[ 50%] Building CXX object tests/CMakeFiles/pybind11_tests.dir/test_kwargs_and_defaults.cpp.o
pgc++-Warning--Mipa is not supported for this target

At end of source: warning: routine is both "inline" and "noinline"

[ 52%] Building CXX object tests/CMakeFiles/pybind11_tests.dir/test_local_bindings.cpp.o
pgc++-Warning--Mipa is not supported for this target

At end of source: warning: routine is both "inline" and "noinline"

[ 54%] Building CXX object tests/CMakeFiles/pybind11_tests.dir/test_methods_and_attributes.cpp.o
pgc++-Warning--Mipa is not supported for this target

At end of source: warning: routine is both "inline" and "noinline"

[ 57%] Building CXX object tests/CMakeFiles/pybind11_tests.dir/test_modules.cpp.o
pgc++-Warning--Mipa is not supported for this target

At end of source: warning: routine is both "inline" and "noinline"

[ 59%] Building CXX object tests/CMakeFiles/pybind11_tests.dir/test_multiple_inheritance.cpp.o
pgc++-Warning--Mipa is not supported for this target

At end of source: warning: routine is both "inline" and "noinline"

[ 61%] Building CXX object tests/CMakeFiles/pybind11_tests.dir/test_numpy_array.cpp.o
pgc++-Warning--Mipa is not supported for this target

At end of source: warning: routine is both "inline" and "noinline"

[ 64%] Building CXX object tests/CMakeFiles/pybind11_tests.dir/test_numpy_dtypes.cpp.o
pgc++-Warning--Mipa is not supported for this target

"/pybind11/tests/test_numpy_dtypes.cpp", line 273: warning: offsetof applied to
          non-POD types is nonstandard
      PYBIND11_NUMPY_DTYPE(ComplexStruct, cflt, cdbl);
      ^

"/pybind11/tests/test_numpy_dtypes.cpp", line 273: warning: offsetof applied to
          non-POD types is nonstandard
      PYBIND11_NUMPY_DTYPE(ComplexStruct, cflt, cdbl);
      ^

At end of source: warning: routine is both "inline" and "noinline"

[ 66%] Building CXX object tests/CMakeFiles/pybind11_tests.dir/test_numpy_vectorize.cpp.o
pgc++-Warning--Mipa is not supported for this target

At end of source: warning: routine is both "inline" and "noinline"

[ 69%] Building CXX object tests/CMakeFiles/pybind11_tests.dir/test_opaque_types.cpp.o
pgc++-Warning--Mipa is not supported for this target

At end of source: warning: routine is both "inline" and "noinline"

[ 71%] Building CXX object tests/CMakeFiles/pybind11_tests.dir/test_operator_overloading.cpp.o
pgc++-Warning--Mipa is not supported for this target

At end of source: warning: routine is both "inline" and "noinline"

[ 73%] Building CXX object tests/CMakeFiles/pybind11_tests.dir/test_pickling.cpp.o
pgc++-Warning--Mipa is not supported for this target

At end of source: warning: routine is both "inline" and "noinline"

[ 76%] Building CXX object tests/CMakeFiles/pybind11_tests.dir/test_pytypes.cpp.o
pgc++-Warning--Mipa is not supported for this target

At end of source: warning: routine is both "inline" and "noinline"

[ 78%] Building CXX object tests/CMakeFiles/pybind11_tests.dir/test_sequences_and_iterators.cpp.o
pgc++-Warning--Mipa is not supported for this target

At end of source: warning: routine is both "inline" and "noinline"

[ 80%] Building CXX object tests/CMakeFiles/pybind11_tests.dir/test_smart_ptr.cpp.o
pgc++-Warning--Mipa is not supported for this target

At end of source: warning: routine is both "inline" and "noinline"

[ 83%] Building CXX object tests/CMakeFiles/pybind11_tests.dir/test_stl.cpp.o
pgc++-Warning--Mipa is not supported for this target

At end of source: warning: routine is both "inline" and "noinline"

[ 85%] Building CXX object tests/CMakeFiles/pybind11_tests.dir/test_stl_binders.cpp.o
pgc++-Warning--Mipa is not supported for this target

At end of source: warning: routine is both "inline" and "noinline"

[ 88%] Building CXX object tests/CMakeFiles/pybind11_tests.dir/test_tagbased_polymorphic.cpp.o
pgc++-Warning--Mipa is not supported for this target

At end of source: warning: routine is both "inline" and "noinline"

[ 90%] Building CXX object tests/CMakeFiles/pybind11_tests.dir/test_union.cpp.o
pgc++-Warning--Mipa is not supported for this target

At end of source: warning: routine is both "inline" and "noinline"

[ 92%] Building CXX object tests/CMakeFiles/pybind11_tests.dir/test_virtual_functions.cpp.o
pgc++-Warning--Mipa is not supported for this target

At end of source: warning: routine is both "inline" and "noinline"

"/pybind11/include/pybind11/cast.h", line 1730: error: function
          "NonCopyable::NonCopyable(const NonCopyable &)" (declared at line 104
          of "/pybind11/tests/test_virtual_functions.cpp") cannot be referenced
          -- it is a deleted function
      return cast_op<T>(load_type<T>(handle));
             ^
          detected during:
            instantiation of "T pybind11::cast<T,<unnamed>>(const
                      pybind11::handle &) [with T=NonCopyable, <unnamed>=0]" at
                      line 1785
            instantiation of
                      "std::enable_if_t<pybind11::detail::negation<pybind11::de
                      tail::any_of<pybind11::detail::move_always<T, void>,
                      pybind11::detail::move_if_unreferenced<T, void>>>::value,
                      T> pybind11::cast<T>(pybind11::object &&) [with
                      T=NonCopyable]" at line 1815
            instantiation of "std::enable_if_t<<expression>, T>
                      pybind11::detail::cast_safe<T>(pybind11::object &&) [with
                      T=NonCopyable]" at line 144 of
                      "/pybind11/tests/test_virtual_functions.cpp"

1 error detected in the compilation of "/pybind11/tests/test_virtual_functions.cpp".
tests/CMakeFiles/pybind11_tests.dir/build.make:549: recipe for target 'tests/CMakeFiles/pybind11_tests.dir/test_virtual_functions.cpp.o' failed
make[2]: *** [tests/CMakeFiles/pybind11_tests.dir/test_virtual_functions.cpp.o] Error 2
CMakeFiles/Makefile2:250: recipe for target 'tests/CMakeFiles/pybind11_tests.dir/all' failed
make[1]: *** [tests/CMakeFiles/pybind11_tests.dir/all] Error 2
Makefile:148: recipe for target 'all' failed
make: *** [all] Error 2

[henryiii]

And it put tests/pybind11_cross_module_tests.cpython-36m-x86_64-linux-gnu.so in the source directory outside the docker container because #2353 hasn't been merged yet. 😠

[andriish]

Hi @henryiii ,

OK, earlier I haven't compiled the test suite, just the code I needed.

Now I've tried the test suite with nvcc 20.7.
Apart from #if defined(__PGIC__) I've added -Wc,--pending_instantiations=0 (maximum number of recursive template instantiations, or 0 for unlimited) and -D__GCC_ATOMIC_TEST_AND_SET_TRUEVAL=1 ( old system) to the compiler options.
With these modifications, I was able to compile the whole the test suite but
test_exceptions.cpp
test_eigen.cpp
test_virtual_functions.cpp
test_smart_ptr.cpp.

Best regards,

Andrii

[andriish]

Hi @henryiii ,

is there any progress with this issue?

If you and other authors are inclined to have Nvidia support, the suggested trivial "fix" from me could be a first step.
It is better than nothing... I can also make a merge request with these trivial "fixes". Albeit I'm sure you can reproduce them in some minutes.

If the Nvidia support is not important in the moment, it is better to close this issue and reopen it in the future if there will be support form the authors.

Best regards,

Andrii

[YannickJadoul]

Have a look at the recent open pull requests (https://github.com/pybind/pybind11/pulls): #2461

[henryiii]

The PR would need to add testing as well - probably something that looks exactly like the PR for NVCC; a few disabled tests if using PGI compilers, and a few ifs, and a docker-based test run.

[andriish]

Hi @YannickJadoul , @henryiii ,

#2461

Ok, it looks I've missed the begging of party.
Thanks!

a few disabled tests if using PGI compilers

Yes.

and a few ifs,

Yes.
But are you sure that the correct identification is __CUDACC_ ?
Having CUDA is great, but wouldn't that make sense to have Nvidia (PGI) compillers w/o CUDA first, i.e.
__PGIC__ ?
I'm not sure that defined __CUDACC__ always means that __PGIC__ is defined.

Best regards,

Andrii

[henryiii]

The PR above is for NVCC, not PGI. But something similar might work for PGI. You just would not need to activate the CUDA language to test with PGI, since it's a valid C++ compiler.

You can compile CUDA with NVCC, Clang, and (apparently) now PGI - though in this case I think it's still NVCC, just with PGI as the host, instead of the normal system compiler (GCC). So the fix is important if the two are going to be mixed, but still separate from supporting PGI compilers.

[henryiii]

wouldn't that make sense to have Nvidia (PGI) compillers w/o CUDA first

NVCC is used heavily in Python, and PGI barely at all. PyTorch, etc. all compile with NVCC (because you have to for CUDA; Clang support is quite new), but I don't think any of them use PGI. Usually the host compiler matches your Python install; I don't know where you would get a PGI compiled Python stack.

Doesn't mean it shouldn't be supported, but it is where priorities are coming from.

[andriish]

Hi @henryiii ,

The PR above is for NVCC

OK, I see. The reference to the #2461 confused me a bit.

but it is where priorities are coming from.

Exactly what I had in mind when created this issue. nvc++ and nvc from the new NVidia HPC SDK are PGI ^1 , so very-very likely many people will use them together with nvcc just because it is from the same SDK by NVidia and is available for all platforms. In this sense 'Nvidia C++ compiler' aka 'PGI C++ compiler' is the new 'normal' compiler for CUDA.

And that looks good, as for me: "free", Windows/Linux/Mac support, C/C++ and Fortran.

Best regards,

Andrii

^1 And define macro __PGIC__.