Python 3.8 package installation fails in Docker Build

[justineyster]

Description

Cannot install confluent-kafka on docker image running Python 3.8 image. Was forced to roll back to Python 3.7

How to reproduce

Attempt to build a docker image with python:3 or python 3.8 as the base image. We used pipenv in our project. pip install confluent-kafka will result in the same error: ERROR: ERROR: Package installation failed...

Within the stack trace: #error "confluent-kafka-python requires librdkafka v1.0.0 or later. Install the latest version of librdkafka from the Confluent repositories, see http://docs.confluent.io/current/installation.html"'

Full stack of failed package installation: https://gist.github.com/justineyster/736b6dfa384d32bf1d5333ddd3b307a8

In an attempt to mitigate, I successfully installed librdkafka with apt-get update && apt-get install librdkafka before running the install but the failure and error message was the same.

The only thing that fixed the error was rolling our Docker image back to python:3.7.

Checklist

Please provide the following information:

• confluent-kafka-python and librdkafka version (confluent_kafka.version() and confluent_kafka.libversion()): latest of both
• Apache Kafka broker version: unsure
• Client configuration: {...}
• Operating system: python:3 Docker image, based on Debian
• Provide client logs (with 'debug': '..' as necessary)
• Provide broker log excerpts
• Critical issue

[antonioIrizar]

I fixed it using a recent version of librdkafka, because apt install old version.

I add the following code on docker and it works:

RUN wget https://github.com/edenhill/librdkafka/archive/v1.2.0.tar.gz && tar xvzf v1.2.0.tar.gz && cd librdkafka-1.2.0/ && ./configure && make && make install && ldconfig

[moorchegue]

Are binaries no longer included into the package?

Similar issue: #45

[joekohlsdorf]

They are included but there are no wheels for Python 3.8 yet.

[stevanmilic]

This can be closed, new version 1.3 has wheels for Python 3.8.

[rnpridgeon]

As @stevanmilic mentioned we have released wheels for 3.8. With that said we are still sorting out some ci build issues with appveyor so windows 3.8 wheels are still a bit out.

[justineyster]

This issue is back in the latest release.

[Bennyweasl]

Just came across this today

[rnpridgeon]

I'm unable to reproduce using the precompiled binary wheels with the latest release. Please open a new issue with a complete set of details.

cat Dockerfile 
FROM python:3.8.0
RUN pip install confluent_kafka
CMD python -c 'import confluent_kafka as ck; print("py: {}, c: {}".format(ck.version(), ck.libversion()))'

docker build .
Sending build context to Docker daemon  2.048kB
Step 1/3 : FROM python:3.8.0
 ---> 0a3a95c81a2b
Step 2/3 : RUN pip install confluent_kafka
 ---> Running in d691d21b23ab
Collecting confluent_kafka
  Downloading https://files.pythonhosted.org/packages/6e/37/9d46fd1cc3112c11d57dfd54c0e0a86bec4d95d56c4dfe0454fa2818eb87/confluent_kafka-1.3.0-cp38-cp38-manylinux2010_x86_64.whl (9.1MB)
Installing collected packages: confluent-kafka
Successfully installed confluent-kafka-1.3.0
WARNING: You are using pip version 19.3.1; however, version 20.0.2 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
Removing intermediate container d691d21b23ab
 ---> 251f125a41d5
Step 3/3 : CMD python -c 'import confluent_kafka as ck; print("py: {}, c: {}".format(ck.version(), ck.libversion()))'
 ---> Running in 958861d06e70
Removing intermediate container 958861d06e70
 ---> a926c6178b38
Successfully built a926c6178b38
MacBook-Pro-4:py38_docker ryan$ docker run -it a926c6178b38
py: ('1.3.0', 16973824), c: ('1.3.0', 16974079)