ERROR: Could not build wheels for bottleneck which use PEP 517 and cannot be installed directly

[AlysonR]

I get the errors
ERROR: Failed building wheel
ERROR: Could not build wheels for bottleneck which use PEP 517 and cannot be installed directly

I have python-devel installed, my wheel is updated to the current version, pip is updated to the current version, pep517 is installed

I saw this was an issue before that was closed after a false answer was given that was never proven to work (installing developers tools). I will not use conda or anaconda so please do not suggest that I install those to solve this problem.

[qwhelan]

Could you please provide exact versions of each of those dependencies, as well as the operating system?

[ArasAzimi]

@qwhelan I had this same issue:

Here are some dependencies and the virtual env I see the error in:

-Ubuntu 18.04.4 LTS
-5.3.0-62-generic
-x86_64
-Python 3.7.8
-pip 20.1.1

virtualenv -p python3.7 venv
source venv/bin/activate
pip install --upgrade setuptools wheel
pip install numpy==1.18.1
pip install fastai

This seems to be working with python 3.7.3 ( I had a mac system working and when I checked the python version it had 3.7.3). On the same Ubuntu system as above when I downgraded to python 3.7.3 The error no longer showed up. I thought I will post here in case it can help to find out the root cause. Also tried with python 3.8.3 and had the error.

[qwhelan]

@ArasAzimi Thank you, I will attempt to reproduce locally.

[Zeitsperre]

You might not find this useful, but we've noted the same failures on Python3.6 builds (on Travis CI) of the master branch of Bottleneck for the project xclim. I've opened a PR on there to try and figure out the cause: Ouranosinc/xclim#517

Feel free to check out the build statuses on that PR; they seem to be exactly as @AlysonR describes.

[aulemahal]

I believe I have found the source of the error, at least in my configuration. It is due to the specific of virtual environment creation with python -m venv.

When creating a virtual environment with venv into ${venv}, a ${venv}/include folder is created and sys.prefix == ${venv}. However, in opposition to virtualenv, the include folder is empty, it exists solely for files created by packages installed within the environment. Source : PEP 405.

The include folders needed by bottleneck are still accessible, but their path has to be constructed relative to sys.base_prefix instead, to get the system files. Changing:

bottleneck/bottleneck/src/bn_config.py

Line 30 in f2bc792

for prefix in [sys.prefix, sys.exec_prefix]:

to

    for prefix in [sys.prefix, sys.exec_prefix, sys.base_prefix]:

fixes the problem.

Of course, I have pythonX.X-dev installed (on ubuntu) for this to work.

EDIT: I have python 3.7.9.

[AndyWangSFU]

This is really a weird issue. It happens in my local machine, and happens again in my poetry virtual machine... (BTW they have different python versions)

[nocluebutalotofit]

On macOS Big Sur 11.2.3 I get the following error running pip install:

$ pip install Bottleneck==1.3.2

Collecting Bottleneck==1.3.2
Using cached Bottleneck-1.3.2.tar.gz (88 kB)
Installing build dependencies ... done
Getting requirements to build wheel ... done
Preparing wheel metadata ... done
Collecting numpy
Using cached numpy-1.20.2-cp38-cp38-macosx_10_9_x86_64.whl (16.0 MB)
Building wheels for collected packages: Bottleneck
Building wheel for Bottleneck (PEP 517) ... error
ERROR: Command errored out with exit status 1:
command: /Users/daniel/PycharmProjects/machine-learning-for-trading_book/venv/bin/python /Users/daniel/PycharmProjects/machine-learning-for-trading_book/venv/lib/python3.8/site-packages/pip/_vendor/pep517/in_process/in_process.py build_wheel /var/folders/hh/ll8cmcb100v7lbrfh1n1t96m0000gn/T/tmpd5gppf3b
cwd: /private/var/folders/hh/ll8cmcb100v7lbrfh1n1t96m0000gn/T/pip-install-082v_gn/bottleneck_c8de5d9f837747fba5e8843c02aa359c
Complete output (127 lines):
running bdist_wheel
running build
running build_py
creating build
creating build/lib.macosx-10.14.6-x86_64-3.8
creating build/lib.macosx-10.14.6-x86_64-3.8/bottleneck
copying bottleneck/_version.py -> build/lib.macosx-10.14.6-x86_64-3.8/bottleneck
copying bottleneck/init.py -> build/lib.macosx-10.14.6-x86_64-3.8/bottleneck
copying bottleneck/_pytesttester.py -> build/lib.macosx-10.14.6-x86_64-3.8/bottleneck
creating build/lib.macosx-10.14.6-x86_64-3.8/bottleneck/benchmark
copying bottleneck/benchmark/bench_detailed.py -> build/lib.macosx-10.14.6-x86_64-3.8/bottleneck/benchmark
copying bottleneck/benchmark/autotimeit.py -> build/lib.macosx-10.14.6-x86_64-3.8/bottleneck/benchmark
copying bottleneck/benchmark/init.py -> build/lib.macosx-10.14.6-x86_64-3.8/bottleneck/benchmark
copying bottleneck/benchmark/bench.py -> build/lib.macosx-10.14.6-x86_64-3.8/bottleneck/benchmark
creating build/lib.macosx-10.14.6-x86_64-3.8/bottleneck/slow
copying bottleneck/slow/reduce.py -> build/lib.macosx-10.14.6-x86_64-3.8/bottleneck/slow
copying bottleneck/slow/init.py -> build/lib.macosx-10.14.6-x86_64-3.8/bottleneck/slow
copying bottleneck/slow/nonreduce.py -> build/lib.macosx-10.14.6-x86_64-3.8/bottleneck/slow
copying bottleneck/slow/move.py -> build/lib.macosx-10.14.6-x86_64-3.8/bottleneck/slow
copying bottleneck/slow/nonreduce_axis.py -> build/lib.macosx-10.14.6-x86_64-3.8/bottleneck/slow
creating build/lib.macosx-10.14.6-x86_64-3.8/bottleneck/tests
copying bottleneck/tests/nonreduce_axis_test.py -> build/lib.macosx-10.14.6-x86_64-3.8/bottleneck/tests
copying bottleneck/tests/scalar_input_test.py -> build/lib.macosx-10.14.6-x86_64-3.8/bottleneck/tests
copying bottleneck/tests/reduce_test.py -> build/lib.macosx-10.14.6-x86_64-3.8/bottleneck/tests
copying bottleneck/tests/util.py -> build/lib.macosx-10.14.6-x86_64-3.8/bottleneck/tests
copying bottleneck/tests/move_test.py -> build/lib.macosx-10.14.6-x86_64-3.8/bottleneck/tests
copying bottleneck/tests/init.py -> build/lib.macosx-10.14.6-x86_64-3.8/bottleneck/tests
copying bottleneck/tests/input_modification_test.py -> build/lib.macosx-10.14.6-x86_64-3.8/bottleneck/tests
copying bottleneck/tests/nonreduce_test.py -> build/lib.macosx-10.14.6-x86_64-3.8/bottleneck/tests
copying bottleneck/tests/list_input_test.py -> build/lib.macosx-10.14.6-x86_64-3.8/bottleneck/tests
copying bottleneck/tests/memory_test.py -> build/lib.macosx-10.14.6-x86_64-3.8/bottleneck/tests
creating build/lib.macosx-10.14.6-x86_64-3.8/bottleneck/src
copying bottleneck/src/bn_config.py -> build/lib.macosx-10.14.6-x86_64-3.8/bottleneck/src
copying bottleneck/src/init.py -> build/lib.macosx-10.14.6-x86_64-3.8/bottleneck/src
copying bottleneck/src/bn_template.py -> build/lib.macosx-10.14.6-x86_64-3.8/bottleneck/src
UPDATING build/lib.macosx-10.14.6-x86_64-3.8/bottleneck/_version.py
set build/lib.macosx-10.14.6-x86_64-3.8/bottleneck/_version.py to '1.3.2'
running build_ext
running config
compiling '_configtest.c':

#pragma clang diagnostic error "-Wattributes"

int attribute((optimize("O3"))) have_attribute_optimize_opt_3(void*);

int main(void)
{
return 0;
}

x86_64-apple-darwin13.4.0-clang -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -iwithsysroot/System/Library/Frameworks/System.framework/PrivateHeaders -iwithsysroot/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.8/Headers -arch arm64 -arch x86_64 -Werror=implicit-function-declaration -march=core2 -mtune=haswell -mssse3 -ftree-vectorize -fPIC -fPIE -fstack-protector-strong -O2 -pipe -isystem /Users/daniel/opt/anaconda3/include -D_FORTIFY_SOURCE=2 -isystem /Users/daniel/opt/anaconda3/include -c _configtest.c -o _configtest.o
clang-11: error: the clang compiler does not support '-march=core2'
failure.
removing: _configtest.c _configtest.o
compiling '_configtest.c':

#ifndef __cplusplus
static inline int static_func (void)
{
return 0;
}
inline int nostatic_func (void)
{
return 0;
}
#endif
int main(void) {
int r1 = static_func();
int r2 = nostatic_func();
return r1 + r2;
}

x86_64-apple-darwin13.4.0-clang -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -iwithsysroot/System/Library/Frameworks/System.framework/PrivateHeaders -iwithsysroot/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.8/Headers -arch arm64 -arch x86_64 -Werror=implicit-function-declaration -march=core2 -mtune=haswell -mssse3 -ftree-vectorize -fPIC -fPIE -fstack-protector-strong -O2 -pipe -isystem /Users/daniel/opt/anaconda3/include -D_FORTIFY_SOURCE=2 -isystem /Users/daniel/opt/anaconda3/include -c _configtest.c -o _configtest.o
clang-11: error: the clang compiler does not support '-march=core2'
failure.
removing: _configtest.c _configtest.o
compiling '_configtest.c':

#ifndef __cplusplus
static inline int static_func (void)
{
return 0;
}
inline int nostatic_func (void)
{
return 0;
}
#endif
int main(void) {
int r1 = static_func();
int r2 = nostatic_func();
return r1 + r2;
}

x86_64-apple-darwin13.4.0-clang -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -iwithsysroot/System/Library/Frameworks/System.framework/PrivateHeaders -iwithsysroot/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.8/Headers -arch arm64 -arch x86_64 -Werror=implicit-function-declaration -march=core2 -mtune=haswell -mssse3 -ftree-vectorize -fPIC -fPIE -fstack-protector-strong -O2 -pipe -isystem /Users/daniel/opt/anaconda3/include -D_FORTIFY_SOURCE=2 -isystem /Users/daniel/opt/anaconda3/include -c _configtest.c -o _configtest.o
clang-11: error: the clang compiler does not support '-march=core2'
failure.
removing: _configtest.c _configtest.o
compiling '_configtest.c':

#ifndef __cplusplus
static __inline int static_func (void)
{
return 0;
}
__inline int nostatic_func (void)
{
return 0;
}
#endif
int main(void) {
int r1 = static_func();
int r2 = nostatic_func();
return r1 + r2;
}

x86_64-apple-darwin13.4.0-clang -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -iwithsysroot/System/Library/Frameworks/System.framework/PrivateHeaders -iwithsysroot/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.8/Headers -arch arm64 -arch x86_64 -Werror=implicit-function-declaration -march=core2 -mtune=haswell -mssse3 -ftree-vectorize -fPIC -fPIE -fstack-protector-strong -O2 -pipe -isystem /Users/daniel/opt/anaconda3/include -D_FORTIFY_SOURCE=2 -isystem /Users/daniel/opt/anaconda3/include -c _configtest.c -o _configtest.o
clang-11: error: the clang compiler does not support '-march=core2'
failure.
removing: _configtest.c _configtest.o
building 'bottleneck.reduce' extension
creating build/temp.macosx-10.14.6-x86_64-3.8
creating build/temp.macosx-10.14.6-x86_64-3.8/bottleneck
creating build/temp.macosx-10.14.6-x86_64-3.8/bottleneck/src
x86_64-apple-darwin13.4.0-clang -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -iwithsysroot/System/Library/Frameworks/System.framework/PrivateHeaders -iwithsysroot/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.8/Headers -arch arm64 -arch x86_64 -Werror=implicit-function-declaration -march=core2 -mtune=haswell -mssse3 -ftree-vectorize -fPIC -fPIE -fstack-protector-strong -O2 -pipe -isystem /Users/daniel/opt/anaconda3/include -D_FORTIFY_SOURCE=2 -isystem /Users/daniel/opt/anaconda3/include -I/private/var/folders/hh/ll8cmcb100v7lbrfh1n1t96m0000gn/T/pip-build-env-6idcz14p/overlay/lib/python3.8/site-packages/numpy/core/include -I/Users/daniel/PycharmProjects/machine-learning-for-trading_book/venv/include -I/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.8/include/python3.8 -Ibottleneck/src -c bottleneck/src/reduce.c -o build/temp.macosx-10.14.6-x86_64-3.8/bottleneck/src/reduce.o -O2
clang-11: error: the clang compiler does not support '-march=core2'
error: command 'x86_64-apple-darwin13.4.0-clang' failed with exit status 1

ERROR: Failed building wheel for Bottleneck
Failed to build Bottleneck
ERROR: Could not build wheels for Bottleneck which use PEP 517 and cannot be installed directly

[bkaankuguoglu]

Is there any update regarding this issue? So far I've tried the following but none worked:

• Installing the package with the most recent pip and setuptools
• Installing with --no-binary mode pip install bottleneck --no-binary :all:
• Changing the line 30 in bottleneck/bottleneck/src/bn_config.py file to:
  for prefix in [sys.prefix, sys.exec_prefix, sys.base_prefix]: and for prefix in [sys.prefix, sys.exec_prefix, <path_to_venv>]:

[brentrichards]

After many re-installs, running:
pip install --upgrade setuptools wheel
fixed the issue for me. (On Windows Dev edition).

[profxj]

Running on a VM via kubernetes with conda installed using

# Miniconda
ENV PATH="/root/miniconda3/bin:${PATH}"
ARG PATH="/root/miniconda3/bin:${PATH}"
RUN wget https://repo.anaconda.com/miniconda/Miniconda3-py38_4.9.2-Linux-x86_64.sh \
    && mkdir /root/.conda \
    && bash Miniconda3-py38_4.9.2-Linux-x86_64.sh -b \
    && rm -f Miniconda3-py38_4.9.2-Linux-x86_64.sh 

fails to Install, even after the

pip install --upgrade setuptools wheel

approach suggested above.

The final bit of the crash:

  gcc -pthread -B /root/miniconda3/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -c _configtest.c -o _configtest.o
  failure.
  removing: _configtest.c _configtest.o
  building 'bottleneck.reduce' extension
  creating build/temp.linux-x86_64-3.8
  creating build/temp.linux-x86_64-3.8/bottleneck
  creating build/temp.linux-x86_64-3.8/bottleneck/src
  gcc -pthread -B /root/miniconda3/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/tmp/pip-build-env-xf8ufge7/overlay/lib/python3.8/site-packages/numpy/core/include -I/root/miniconda3/include/python3.8 -Ibottleneck/src -c bottleneck/src/reduce.c -o build/temp.linux-x86_64-3.8/bottleneck/src/reduce.o -O2
  error: command 'gcc' failed with exit status 1
  ----------------------------------------
  ERROR: Failed building wheel for bottleneck
Failed to build bottleneck
ERROR: Could not build wheels for bottleneck which use PEP 517 and cannot be installed directly

[brentrichards]

On Windows, I finally realised that I needed a background tool to build the Wheel, that is not installed by default (and the message doen't hint at). I installed Visual Studio Build tools, and it all worked.

[franciscorodriguez92]

Hi!

Installing gcc solved the problem for me (Ubuntu 20.04):

sudo apt-get install gcc