Can't build boost with python 3 on ubuntu 18.04

[hstarmans]

I am able to build and run boost with python 2 on ubuntu 18.04. I am not able to build and run boost with python 3.
My steps are as follows;

1. Download boost
   I tried both boost 1.71 and 1.67 both in the end fail. Example here is given for 1.71.

cd /home/user/
wget https://dl.bintray.com/boostorg/release/1.71.0/source/boost_1_71_0.tar.bz2
tar --bzip2 -xf boost_1_71_0.tar.bz2

2. Build boost against python 3
   The prefix folder is where the python library gets written

cd /home/user/boost_1_71_0
./bootstrap.sh -with-libraries=python -with-python-version=3.6 --prefix=/home/user/temp/
./b2 &./b2 install

4. Download this library

cd /home/user/
git clone https://github.com/TNG/boost-python-examples

5. Create a virtual environment so python points to python 3

cd boost-python-examples
python3 -m venv environment
source /environment/bin/activate

6. Run cmake with flags
   I read this stack over flow post
   As a result I run the following commands;

set -x
rm -rf build
mkdir build
cd build
cmake -DPYTHON_VERSION_SUFFIX=-py3 -DCMAKE_BUILD_TYPE=DEBUG -DBOOST_ROOT=/home/user/temp/ -DBoost_NO_SYSTEM_PATHS=TRUE ..

Result -- Could NOT find Boost
Also tried -DBOOST_ROOT=/home/user/temp/boost_1_71_0 and several other options to no avail.

I have also tried to modify the cmakecache.txt from the python 2 configuration into variables suited for python 3.
This eventually leads to the error;

relocation R_X86_64_PC32 against symbol `PyLong_Type' can not be used when making a shared object; recompile with -fPIC

I also obtain this error when i tried to run examples using bjam.

[Achimh3011]

Did you check which python version suffix was used in the Boost compilation (if any)? They are not standardized and might even be omitted in manual compilation of the library.

[hstarmans]

Thanks for the quick reply. I made it work in python 3 by modifying the cmakecache.
I have also fixed it by modifying the CMakeList.txt. I believe there is a bug; see next post.

First i remove all instances of boost and made sure cmake could not find any instance.

dpkg -S /usr/include/boost/version.hpp

I removed all these packages. I installed the Ubuntu default boost packages.

sudo apt install libboost-dev libboost-all-dev

Create a directory, run cmake and then make. It detects python 2 and boost. I checked I was able to run examples for this python 2 version.
I then made a new directory ran cmake again

mkdir build2 && cd build 2 && cmake ..

I hacked the cache of cmake CMakeCache.txt in the build folder and than ran make; it worked.

# There is data before this but not relevant so I didn't post it on Github

########################
# EXTERNAL cache entries
########################

//The directory containing a CMake configuration file for Boost.
Boost_DIR:PATH=Boost_DIR-NOTFOUND

//Path to a file.
Boost_INCLUDE_DIR:PATH=/usr/include

//Boost library directory DEBUG
Boost_LIBRARY_DIR_DEBUG:PATH=/usr/lib/x86_64-linux-gnu

//Boost library directory RELEASE
Boost_LIBRARY_DIR_RELEASE:PATH=/usr/lib/x86_64-linux-gnu

//Boost python library (debug)
Boost_PYTHON_LIBRARY_DEBUG:FILEPATH=/usr/lib/x86_64-linux-gnu/libboost_python3.so

//Boost python library (release)
Boost_PYTHON_LIBRARY_RELEASE:FILEPATH=/usr/lib/x86_64-linux-gnu/libboost_python3.so

//Path to a program.
CMAKE_AR:FILEPATH=/usr/bin/ar

//Choose the type of build, options are: None(CMAKE_CXX_FLAGS or
// CMAKE_C_FLAGS used) Debug Release RelWithDebInfo MinSizeRel.
CMAKE_BUILD_TYPE:STRING=

//Enable/Disable color output during build.
CMAKE_COLOR_MAKEFILE:BOOL=ON

//CXX compiler
CMAKE_CXX_COMPILER:FILEPATH=/usr/bin/c++

//A wrapper around 'ar' adding the appropriate '--plugin' option
// for the GCC compiler
CMAKE_CXX_COMPILER_AR:FILEPATH=/usr/bin/gcc-ar-7

//A wrapper around 'ranlib' adding the appropriate '--plugin' option
// for the GCC compiler
CMAKE_CXX_COMPILER_RANLIB:FILEPATH=/usr/bin/gcc-ranlib-7

//Flags used by the compiler during all build types.
CMAKE_CXX_FLAGS:STRING=

//Flags used by the compiler during debug builds.
CMAKE_CXX_FLAGS_DEBUG:STRING=-g

//Flags used by the compiler during release builds for minimum
// size.
CMAKE_CXX_FLAGS_MINSIZEREL:STRING=-Os -DNDEBUG

//Flags used by the compiler during release builds.
CMAKE_CXX_FLAGS_RELEASE:STRING=-O3 -DNDEBUG

//Flags used by the compiler during release builds with debug info.
CMAKE_CXX_FLAGS_RELWITHDEBINFO:STRING=-O2 -g -DNDEBUG

//C compiler
CMAKE_C_COMPILER:FILEPATH=/usr/bin/cc

//A wrapper around 'ar' adding the appropriate '--plugin' option
// for the GCC compiler
CMAKE_C_COMPILER_AR:FILEPATH=/usr/bin/gcc-ar-7

//A wrapper around 'ranlib' adding the appropriate '--plugin' option
// for the GCC compiler
CMAKE_C_COMPILER_RANLIB:FILEPATH=/usr/bin/gcc-ranlib-7

//Flags used by the compiler during all build types.
CMAKE_C_FLAGS:STRING=

//Flags used by the compiler during debug builds.
CMAKE_C_FLAGS_DEBUG:STRING=-g

//Flags used by the compiler during release builds for minimum
// size.
CMAKE_C_FLAGS_MINSIZEREL:STRING=-Os -DNDEBUG

//Flags used by the compiler during release builds.
CMAKE_C_FLAGS_RELEASE:STRING=-O3 -DNDEBUG

//Flags used by the compiler during release builds with debug info.
CMAKE_C_FLAGS_RELWITHDEBINFO:STRING=-O2 -g -DNDEBUG

//Flags used by the linker.
CMAKE_EXE_LINKER_FLAGS:STRING=

//Flags used by the linker during debug builds.
CMAKE_EXE_LINKER_FLAGS_DEBUG:STRING=

//Flags used by the linker during release minsize builds.
CMAKE_EXE_LINKER_FLAGS_MINSIZEREL:STRING=

//Flags used by the linker during release builds.
CMAKE_EXE_LINKER_FLAGS_RELEASE:STRING=

//Flags used by the linker during Release with Debug Info builds.
CMAKE_EXE_LINKER_FLAGS_RELWITHDEBINFO:STRING=

//Enable/Disable output of compile commands during generation.
CMAKE_EXPORT_COMPILE_COMMANDS:BOOL=OFF

//Install path prefix, prepended onto install directories.
CMAKE_INSTALL_PREFIX:PATH=/usr/local

//Path to a program.
CMAKE_LINKER:FILEPATH=/usr/bin/ld

//Path to a program.
CMAKE_MAKE_PROGRAM:FILEPATH=/usr/bin/make

//Flags used by the linker during the creation of modules.
CMAKE_MODULE_LINKER_FLAGS:STRING=

//Flags used by the linker during debug builds.
CMAKE_MODULE_LINKER_FLAGS_DEBUG:STRING=

//Flags used by the linker during release minsize builds.
CMAKE_MODULE_LINKER_FLAGS_MINSIZEREL:STRING=

//Flags used by the linker during release builds.
CMAKE_MODULE_LINKER_FLAGS_RELEASE:STRING=

//Flags used by the linker during Release with Debug Info builds.
CMAKE_MODULE_LINKER_FLAGS_RELWITHDEBINFO:STRING=

//Path to a program.
CMAKE_NM:FILEPATH=/usr/bin/nm

//Path to a program.
CMAKE_OBJCOPY:FILEPATH=/usr/bin/objcopy

//Path to a program.
CMAKE_OBJDUMP:FILEPATH=/usr/bin/objdump

//Value Computed by CMake
CMAKE_PROJECT_NAME:STATIC=Project

//Path to a program.
CMAKE_RANLIB:FILEPATH=/usr/bin/ranlib

//Flags used by the linker during the creation of dll's.
CMAKE_SHARED_LINKER_FLAGS:STRING=

//Flags used by the linker during debug builds.
CMAKE_SHARED_LINKER_FLAGS_DEBUG:STRING=

//Flags used by the linker during release minsize builds.
CMAKE_SHARED_LINKER_FLAGS_MINSIZEREL:STRING=

//Flags used by the linker during release builds.
CMAKE_SHARED_LINKER_FLAGS_RELEASE:STRING=

//Flags used by the linker during Release with Debug Info builds.
CMAKE_SHARED_LINKER_FLAGS_RELWITHDEBINFO:STRING=

//If set, runtime paths are not added when installing shared libraries,
// but are added when building.
CMAKE_SKIP_INSTALL_RPATH:BOOL=NO

//If set, runtime paths are not added when using shared libraries.
CMAKE_SKIP_RPATH:BOOL=NO

//Flags used by the linker during the creation of static libraries.
CMAKE_STATIC_LINKER_FLAGS:STRING=

//Flags used by the linker during debug builds.
CMAKE_STATIC_LINKER_FLAGS_DEBUG:STRING=

//Flags used by the linker during release minsize builds.
CMAKE_STATIC_LINKER_FLAGS_MINSIZEREL:STRING=

//Flags used by the linker during release builds.
CMAKE_STATIC_LINKER_FLAGS_RELEASE:STRING=

//Flags used by the linker during Release with Debug Info builds.
CMAKE_STATIC_LINKER_FLAGS_RELWITHDEBINFO:STRING=

//Path to a program.
CMAKE_STRIP:FILEPATH=/usr/bin/strip

//If this value is on, makefiles will be generated without the
// .SILENT directive, and all commands will be echoed to the console
// during the make.  This is useful for debugging only. With Visual
// Studio IDE projects all commands are done without /nologo.
CMAKE_VERBOSE_MAKEFILE:BOOL=FALSE

//Add module classes
PYTHON_ENABLE_MODULE_classes:BOOL=ON

//Add module ctor
PYTHON_ENABLE_MODULE_ctor:BOOL=ON

//Add module hello
PYTHON_ENABLE_MODULE_hello:BOOL=ON

//Add module inheritance
PYTHON_ENABLE_MODULE_inheritance:BOOL=ON

//Add module iterators
PYTHON_ENABLE_MODULE_iterators:BOOL=ON

//Add module member
PYTHON_ENABLE_MODULE_member:BOOL=ON

//Add module myexceptions
PYTHON_ENABLE_MODULE_myexceptions:BOOL=ON

//Add module mymodule
PYTHON_ENABLE_MODULE_mymodule:BOOL=ON

//Add module operators
PYTHON_ENABLE_MODULE_operators:BOOL=ON

//Add module overload
PYTHON_ENABLE_MODULE_overload:BOOL=ON

//Add module policies
PYTHON_ENABLE_MODULE_policies:BOOL=ON

//Add module virtual
PYTHON_ENABLE_MODULE_virtual:BOOL=ON

//Path to a program.
PYTHON_EXECUTABLE:FILEPATH=/usr/bin/python3

//Path to a file.
PYTHON_INCLUDE_DIR:PATH=/usr/include/python3.6

//Path to a library.
PYTHON_LIBRARY:FILEPATH=/usr/lib/x86_64-linux-gnu/libpython3.6m.so

//Path to a library.
PYTHON_LIBRARY_DEBUG:FILEPATH=PYTHON_LIBRARY_DEBUG-NOTFOUND

//Add module classes shared
PYTHON_MODULE_classes_BUILD_SHARED:BOOL=ON

//Add module ctor shared
PYTHON_MODULE_ctor_BUILD_SHARED:BOOL=ON

//Add module hello shared
PYTHON_MODULE_hello_BUILD_SHARED:BOOL=ON

//Add module inheritance shared
PYTHON_MODULE_inheritance_BUILD_SHARED:BOOL=ON

//Add module iterators shared
PYTHON_MODULE_iterators_BUILD_SHARED:BOOL=ON

//Add module member shared
PYTHON_MODULE_member_BUILD_SHARED:BOOL=ON

//Add module myexceptions shared
PYTHON_MODULE_myexceptions_BUILD_SHARED:BOOL=ON

//Add module mymodule shared
PYTHON_MODULE_mymodule_BUILD_SHARED:BOOL=ON

//Add module operators shared
PYTHON_MODULE_operators_BUILD_SHARED:BOOL=ON

//Add module overload shared
PYTHON_MODULE_overload_BUILD_SHARED:BOOL=ON

//Add module policies shared
PYTHON_MODULE_policies_BUILD_SHARED:BOOL=ON

//Add module virtual shared
PYTHON_MODULE_virtual_BUILD_SHARED:BOOL=ON

//Value Computed by CMake
Project_BINARY_DIR:STATIC=/home/starmans/boost-python-examples/build2

//Value Computed by CMake
Project_SOURCE_DIR:STATIC=/home/starmans/boost-python-examples

//Dependencies for the target
hello_LIB_DEPENDS:STATIC=general;/usr/lib/x86_64-linux-gnu/libboost_python3.so;general;/usr/lib/x86_64-linux-gnu/libpython3.6m.so;

//I have remove the other targets and everything which comes after this.. To minimize output on //Github

So the remaining questions is how do you get cmake into coming up with this output !?

[hstarmans]

Okay let's conclude this. Let's have a look at the provided CMakeList.txt.
In line 3 it states;

FIND_PACKAGE(PythonInterp)

The problem with this is that it will default to Python 2 and never Python 3. Creating a virtual environment to circumvent this is a bit hacky.
I modified this to

FIND_PACKAGE(PythonInterp 3)

Then as I am on unix and not apple I arrive at line 8. Here we have the following;

FIND_PACKAGE(Boost COMPONENTS python${PYTHON_VERSION_SUFFIX})

There is an obvious typo here, it should be;

FIND_PACKAGE(Boost COMPONENTS python${PYTHON_VERSION_MAJOR}

The installation works with these two modifications.

The question that remains is can't you pass a flag to cmake that it looks for python 2 or 3?
Personally, I think it should default to Python 3. If it can not find python3 it should look for python 2.

[Achimh3011]

So for Ubuntu 18.04 the version suffix is simply a 3? Then that needs to be considered in build.sh. If you want, you could add a switch for enforcing python3 there as well.

[hstarmans]

A suffix is a letter or a group of letters attached to the end of a word to form a new word. So Python 3.7 here 3 is version major and 7 is the suffix. This mistake isn't in the other cases where you have;

FIND_PACKAGE(Boost COMPONENTS python${PYTHON_VERSION_MAJOR}${PYTHON_VERSION_MINOR})

In the build.sh I would remove -DPYTHON_VERSION_SUFFIX

cmake  -DCMAKE_BUILD_TYPE=DEBUG .. && make && make test

I would start the cmake file with the code shown below, as it defaults to 3.

FIND_PACKAGE(PythonInterp 3)
     if (NOT PYTHONINTERP_FOUND)
          FIND_PACKAGE(PythonInterp)

[hstarmans]

The issue is closed from my side, I have decided to move to pybind as it is more lightweight than boost. I could also find a cmake, setup.py example. This works for now.