Build python with --enable-shared

[eric-wieser]

This enables python packages that embed python into binaries to be documented with readthedocs.

This is in response to cocotb/cocotb#1569, which we concluded was due to python being built without --enable-shared.

Taken from https://github.com/pyenv/pyenv/wiki#how-to-build-cpython-with---enable-shared

[ericholscher]

@eric-wieser I assume you've already tried this, but have you looked into mocking the imports? We really try to avoid needing complex local environments for doc generation, when at all possible.

[eric-wieser]

Mocking the imports isn't really an option, because the failure happens at build time not import time.

We opted for using conda for our docs, which has a python binary built with --enable-shared by default.
That solved our problem, but I would have slightly preferred to stick with pip / pyenv.

The main argument for doing this is that system python installations are typically built with this flag present - you can check with import sysconfig; print(sysconfig.get_config_vars()['Py_ENABLE_SHARED'])

[ericholscher]

@eric-wieser well the goal of mocking would be to not have to install your library, but to be able to import it with mocked dependencies. It definitely doesn't work everywhere.

Thanks for the info, it will help us with considering this PR 👍

[eric-wieser]

Well the library in question is one we hope to generate documentation for - mocking it would lose all the docstrings.

There are things we've considered in our project to avoid needing this, which I'm the long run made sense - but this PR or switching to conda were a lot easier.

[humitos]

The main argument for doing this is that system python installations are typically built with this flag present - you can check with import sysconfig; print(sysconfig.get_config_vars()['Py_ENABLE_SHARED'])

I double-checked this and I can confirm that the Python version that comes with Ubuntu 20.04 LTS has this enabled and the Python we are compiling ourselves don't. So, I'm fine adding this flag to the new Docker images in #166

[humitos]

Hi @eric-wieser! In #166 we implemented the ability to install extra tools --which is still in beta. Together with that change, now, we are also compiling Python with --enable-shared. Would you like to give it a try and let us know if it works? You need to add something like this to your .readthedocs.yaml file:

build:
  os: "ubuntu-20.04"
  tools:
    python: "3.9"

Note that if you are already defining the Python version via python.version you have to remove it from there.

Thanks!