Undefined Symbol: rd_kafka_producev , Python 3.4

[digiman999]

Importing in PyCharm & Bash with Python3.4 gives

Traceback (most recent call last):
  File "generic.py", line 1, in <module>
    from confluent_kafka import Producer
  File "/usr/lib64/python3.4/site-packages/confluent_kafka/__init__.py", line 2, in <module>
    from .cimpl import (Consumer,  # noqa
ImportError: /usr/lib64/python3.4/site-packages/confluent_kafka/cimpl.cpython-34m.so: undefined symbol: rd_kafka_producev

The setup:

• Python 3.4 (v2 & 3.6 also installed)
• librdkafka 0.9.5 (via yum)
• librdkafka-devel: 0.11.0_confluent3.3.0 (via yum)
• Centos 7

This issue says the librdkafka version needs updating. I'm using the latest librdkafka & -devel.

Any help much appreciated as the package looks great.

[edenhill]

My guess is that the confluent_kafka Python module was built with librdkafka >=v0.9.4, but during runtime the linker is finding an older outdated librdkafka.so file <0.9.4.

Search your runtime system for librdkafka.so* and remove any that looks like it could be outdated (or all, and then reinstall the propers) and try again.

We're actively working on providing autonomous wheels for linux and osx which bundle librdkafka.

[lafolle]

Just went through the same issue (though, with Go client.
There were 2 versions in my system:

1. /usr/lib/x86_64-linux-gnu/librdkafka.so.1
2. /usr/local/lib/librdkafka.so.1

It turned out, the binary generated was loading first.

lafolle@yoss:~/src/kafkatest/cmd/pro$ ldd pro
        linux-vdso.so.1 =>  (0x00007ffda9758000)
        librdkafka.so.1 => /usr/lib/x86_64-linux-gnu/librdkafka.so.1 (0x00007f9786d56000)
        libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f9786b38000)

Solution was to remove first, and set LD_LIBRARY_PATH to /usr/local/lib and rebuild.

Thanks @edenhill .

[digiman999]

As @edenhill suggested, I searched for librdkafka.so* and had two versions. Deleting the older version made it work. Much appreciated.

[edenhill]

See #184 (comment)