[Bug] cpp client segmentation fault

[zhcn]

### Search before asking

• I searched in the issues and found nothing similar.

Version

gcc version 4.8.2 20140120 (Red Hat 4.8.2-16) (GCC)
pulsar cpp client version 2.11

Minimal reproduce step

#include <pulsar/Client.h>
using namespace pulsar;
using namespace std;

void func() {
    shared_ptr<string> t = make_shared<string>("");;// when I add this line, this program will core(even this func never been called), after remove this line, this program works fine. use shared_ptr<string>t(new string("")) also works fine
}

int main(){
    ReportClient j;
    shared_ptr<string> t;
    Client client("pulsar://xxx");
    Producer producer;
    Result result = client.createProducer("topic_xxx", producer);
    if (result != ResultOk) {
        std::cout << "Error creating producer: " << result << std::endl;
        return -1;
    }
}


g++ report_main.cpp  -I../libs/pulsar/include -lpulsar

What did you expect to see?

works

What did you see instead?

Program received signal SIGSEGV, Segmentation fault.
0x00007ffff5c325f4 in __memcpy_ssse3_back () from /lib64/libc.so.6
Missing separate debuginfos, use: debuginfo-install glibc-2.17-196.tl2.3.x86_64 libgcc-7.3.1-6.tl2.x86_64 libstdc++-7.3.1-6.tl2.x86_64 zlib-1.2.7-15.el7.x86_64
(gdb) bt
#0  0x00007ffff5c325f4 in __memcpy_ssse3_back () from /lib64/libc.so.6
#1  0x000000000040f247 in std::char_traits<char>::copy (__s1=0x7ffff487d028 "\270\247h", __s2=0x68a538 "\270\247h", __n=6666176)
    at /usr/lib/gcc/x86_64-redhat-linux/7/../../../../include/c++/7/bits/char_traits.h:350
#2  0x000000000040fbf5 in std::string::_M_copy (__d=0x7ffff487d028 "\270\247h", __s=0x68a538 "\270\247h", __n=6666176)
    at /usr/lib/gcc/x86_64-redhat-linux/7/../../../../include/c++/7/bits/basic_string.h:3363
#3  0x0000000000420c9a in std::string::_M_mutate (this=0x68a530, __pos=0, __len1=0, __len2=0)
    at /usr/lib/gcc/x86_64-redhat-linux/7/../../../../include/c++/7/bits/basic_string.tcc:934
#4  0x00007ffff7b24437 in std::string::_M_leak_hard() () from /lib64/libstdc++.so.6
#5  0x00007ffff7b2451b in std::string::operator[](unsigned long) () from /lib64/libstdc++.so.6
#6  0x00007ffff6b6584e in pulsar::SharedBuffer::SharedBuffer (this=0x688ec0, size=65536)
    at /usr/src/debug/apache-pulsar-2.11.0-SNAPSHOT-src/pulsar-client-cpp/lib/SharedBuffer.h:221
#7  0x00007ffff6b652d6 in pulsar::SharedBuffer::allocate (size=65536)
    at /usr/src/debug/apache-pulsar-2.11.0-SNAPSHOT-src/pulsar-client-cpp/lib/SharedBuffer.h:62
#8  0x00007ffff6b48a18 in pulsar::ClientConnection::ClientConnection (this=0x688e00, 
    logicalAddress="pulsar://ss-share2-pulsar-discovery-1.tencent-distribute.com:6650,ss-share2-pulsar-discovery-2.tencent-distribute.com:6650,ss-share2-pulsar-discovery-3.tencent-distribute.com:6650", 
    physicalAddress="pulsar://ss-share2-pulsar-discovery-1.tencent-distribute.com:6650,ss-share2-pulsar-discovery-2.tencent-distribute.com:6650,ss-share2-pulsar-discovery-3.tencent-distribute.com:6650", executor=std::shared_ptr (count 6668440, weak -1) 0x7fffffffdd00, 
    clientConfiguration=..., authentication=warning: RTTI symbol not found for class 'std::_Sp_counted_ptr<AuthDisabled*, (__gnu_cxx::_Lock_policy)2>'
warning: RTTI symbol not found for class 'std::_Sp_counted_ptr<AuthDisabled*, (__gnu_cxx::_Lock_policy)2>'
std::shared_ptr (count 3, weak 0) 0x67afd0)
    at /usr/src/debug/apache-pulsar-2.11.0-SNAPSHOT-src/pulsar-client-cpp/lib/ClientConnection.cc:176
#9  0x00007ffff6c1dadc in pulsar::ConnectionPool::getConnectionAsync (this=0x687bf0, 
    logicalAddress="pulsar://ss-share2-pulsar-discovery-1.tencent-distribute.com:6650,ss-share2-pulsar-discovery-2.tencent-distribute.com:6650,ss-share2-pulsar-discovery-3.tencent-distribute.com:6650", 
    physicalAddress="pulsar://ss-share2-pulsar-discovery-1.tencent-distribute.com:6650,ss-share2-pulsar-discovery-2.tencent-distribute.com:6650,ss-share2-pulsar-discovery-3.tencent-distribute.com:6650")
    at /usr/src/debug/apache-pulsar-2.11.0-SNAPSHOT-src/pulsar-client-cpp/lib/ConnectionPool.cc:94
#10 0x00007ffff6b2ca2f in pulsar::BinaryProtoLookupService::getPartitionMetadataAsync (this=0x687ef8, topicName=warning: RTTI symbol not found for class 'std::_Sp_counted_ptr<pulsar::TopicName*, (__gnu_cxx::_Lock_policy)2>'
warning: RTTI symbol not found for class 'std::_Sp_counted_ptr<pulsar::TopicName*, (__gnu_cxx::_Lock_policy)2>'

    std::shared_ptr (count 2, weak 0) 0x688250)
    at /usr/src/debug/apache-pulsar-2.11.0-SNAPSHOT-src/pulsar-client-cpp/lib/BinaryProtoLookupService.cc:76
#11 0x00007ffff6bc6a24 in pulsar::ClientImpl::createProducerAsync(std::string const&, pulsar::ProducerConfiguration, std::function<void (pulsar::Result, pulsar::Producer)>) (this=0x687ae8, 
    topic="pcg_pcgptc8218943_cloudrenderpaascost/pcg_pcgptc8218943_cloudrenderpaascost/pcg_pcgptc8218943_cloudrenderpaascost", conf=..., 
    callback=...) at /usr/src/debug/apache-pulsar-2.11.0-SNAPSHOT-src/pulsar-client-cpp/lib/ClientImpl.cc:171
#12 0x00007ffff6b3bc5f in pulsar::Client::createProducerAsync(std::string const&, pulsar::ProducerConfiguration, std::function<void (pulsar::Result, pulsar::Producer)>) (this=0x7fffffffe140, 
    topic="pcg_pcgptc8218943_cloudrenderpaascost/pcg_pcgptc8218943_cloudrenderpaascost/pcg_pcgptc8218943_cloudrenderpaascost", conf=..., 
    callback=...) at /usr/src/debug/apache-pulsar-2.11.0-SNAPSHOT-src/pulsar-client-cpp/lib/Client.cc:65
#13 0x00007ffff6b3bab2 in pulsar::Client::createProducer (this=0x7fffffffe140, 
    topic="pcg_pcgptc8218943_cloudrenderpaascost/pcg_pcgptc8218943_cloudrenderpaascost/pcg_pcgptc8218943_cloudrenderpaascost", conf=..., 
    producer=...) at /usr/src/debug/apache-pulsar-2.11.0-SNAPSHOT-src/pulsar-client-cpp/lib/Client.cc:53
#14 0x00007ffff6b3b9fd in pulsar::Client::createProducer (this=0x7fffffffe140, 
    topic="pcg_pcgptc8218943_cloudrenderpaascost/pcg_pcgptc8218943_cloudrenderpaascost/pcg_pcgptc8218943_cloudrenderpaascost", 
---Type <return> to continue, or q <return> to quit---
    producer=...) at /usr/src/debug/apache-pulsar-2.11.0-SNAPSHOT-src/pulsar-client-cpp/lib/Client.cc:47
#15 0x000000000040eb9c in main () at test/report_main.cpp:30

Anything else?

No response

Are you willing to submit a PR?

• I'm willing to submit a PR!

[zhcn]

I found that when I use make_shared in ReportClient, the program will core, remove make_shared, it works fine

[tisonkun]

@zhcn code doesn't render well. Could you wrap them inside a code block?

[zhcn]

@zhcn code doesn't render well. Could you wrap them inside a code block?

ok

[zhcn]

can you see anything wrong？ @tisonkun

[zhcn]

more infomation is here https://stackoverflow.com/questions/73554350/make-sharedstring-segment-fault#comment129889831_73554350

[zhcn]

I use apache-pulsar-client-2.10.1-1.x86_64.rpm download from web, it also cause segment fault

[BewareMyPower]

How did you install the Pulsar library? Compiled from source or just installed from yum?

I see you used the RPM package and your G++ version is 7.

#3  0x0000000000420c9a in std::string::_M_mutate (this=0x68a530, __pos=0, __len1=0, __len2=0)
    at /usr/lib/gcc/x86_64-redhat-linux/7/../../../../include/c++/7/bits/basic_string.tcc:934

Could you try the default GCC (4.8.2)?

[BewareMyPower]

Could you give a complete example? Your code cannot be compiled.

I just downloaded the latest code in branch-2.11 and built the Pulsar C++ client from source. The following example works well.

#include <pulsar/Client.h>
using namespace pulsar;

void func() {
    auto t = std::make_shared<std::string>("");
    std::cout << t << std::endl;
}

int main() {
    func();
    Client client("pulsar://localhost:6650");
    std::string topic = "my-topic";
    Producer producer;
    auto result = client.createProducer(topic, producer);
    if (result != ResultOk) {
        std::cout << "Error creating producer: " << result << std::endl;
        return 1;
    }
    MessageId messageId;
    producer.send(MessageBuilder().setContent("hello").build(), messageId);
    client.close();
    return 0;
}

Outputs:

0x12b9c08
2022-08-31 13:34:21.891 INFO  [140604211804928] ClientConnection:189 | [<none> -> pulsar://localhost:6650] Create ClientConnection, timeout=10000
2022-08-31 13:34:21.891 INFO  [140604211804928] ConnectionPool:96 | Created connection for pulsar://localhost:6650
2022-08-31 13:34:21.896 INFO  [140604079314688] ClientConnection:375 | [127.0.0.1:51650 -> 127.0.0.1:6650] Connected to broker
2022-08-31 13:34:21.899 INFO  [140604079314688] HandlerBase:61 | [persistent://public/default/my-topic, ] Getting connection from pool
2022-08-31 13:34:21.903 INFO  [140604079314688] ProducerImpl:186 | [persistent://public/default/my-topic, ] Created producer on broker [127.0.0.1:51650 -> 127.0.0.1:6650] 
2022-08-31 13:34:21.907 INFO  [140604211804928] ClientImpl:507 | Closing Pulsar client with 1 producers and 0 consumers
2022-08-31 13:34:21.908 INFO  [140604211804928] ProducerImpl:657 | [persistent://public/default/my-topic, standalone-0-2] Closing producer for topic persistent://public/default/my-topic
2022-08-31 13:34:21.909 INFO  [140604079314688] ProducerImpl:697 | [persistent://public/default/my-topic, standalone-0-2] Closed producer 0
2022-08-31 13:34:21.909 INFO  [140604060354304] ClientConnection:1560 | [127.0.0.1:51650 -> 127.0.0.1:6650] Connection closed
2022-08-31 13:34:21.909 INFO  [140604060354304] ClientConnection:263 | [127.0.0.1:51650 -> 127.0.0.1:6650] Destroyed connection
2022-08-31 13:34:21.910 INFO  [140604211804928] ProducerImpl:620 | Producer - [persistent://public/default/my-topic, standalone-0-2] , [batchMessageContainer = { BatchMessageContainer [size = 0] [bytes = 0] [maxSize = 1000] [maxBytes = 131072] [topicName = persistent://public/default/my-topic] [numberOfBatchesSent_ = 2] [averageBatchSize_ = 0.5] }]

The dependencies are:

-- Found OpenSSL: /usr/lib64/libcrypto.so (found version "1.0.2k")  
Failed to find Protobuf in config mode, try to find it from system path
-- Protobuf_LIBRARIES: /usr/local/lib/libprotobuf.so
-- Protobuf_INCLUDE_DIRS: /usr/local/include
-- Found CURL: /usr/lib64/libcurl.so (found version "7.29.0")  
-- Found Boost: /usr/include (found version "1.53.0")  
-- Linking with Boost:System
-- Using Boost::Regex
-- Found Boost: /usr/include (found version "1.53.0") found components: system regex 

I installed Protobuf 3.20.0 from source.

The GCC version:

$ g++ --version
g++ (GCC) 4.8.5 20150623 (Red Hat 4.8.5-44)
Copyright (C) 2015 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

[BewareMyPower]

I also tried the 2.10.1 RPM package and it works well with GCC 4.8.

lrwxrwxrwx 1 root root       19 Aug 31 13:48 /usr/lib/libpulsar.so -> libpulsar.so.2.10.1
-rwxr-xr-x 1 root root 10636144 Jun 13 07:57 /usr/lib/libpulsar.so.2.10.1
lrwxrwxrwx 1 root root       24 Aug 31 13:48 /usr/lib/libpulsarnossl.so -> libpulsarnossl.so.2.10.1
-rwxr-xr-x 1 root root  7569800 Jun 13 07:57 /usr/lib/libpulsarnossl.so.2.10.1

g++ SampleProducer.cc -std=c++11 -I /pulsar/pulsar-client-cpp/include -L /usr/lib -lpulsar
export LD_LIBRARY_PATH=/usr/lib:$LD_LIBRARY_PATH
./a.out

Outputs:

0x16536c8
2022-08-31 13:56:48.473 INFO  [139834303919872] ExecutorService:41 | Run io_service in a single thread
2022-08-31 13:56:48.473 INFO  [139834337716672] ClientConnection:189 | [<none> -> pulsar://localhost:6650] Create ClientConnection, timeout=10000
2022-08-31 13:56:48.473 INFO  [139834337716672] ConnectionPool:96 | Created connection for pulsar://localhost:6650
2022-08-31 13:56:48.474 INFO  [139834303919872] ClientConnection:375 | [127.0.0.1:51658 -> 127.0.0.1:6650] Connected to broker
2022-08-31 13:56:48.575 INFO  [139834303919872] HandlerBase:64 | [persistent://public/default/my-topic, ] Getting connection from pool
2022-08-31 13:56:48.901 INFO  [139834303919872] ProducerImpl:189 | [persistent://public/default/my-topic, ] Created producer on broker [127.0.0.1:51658 -> 127.0.0.1:6650] 
2022-08-31 13:56:49.008 INFO  [139834337716672] ClientImpl:498 | Closing Pulsar client with 1 producers and 0 consumers
2022-08-31 13:56:49.008 INFO  [139834337716672] ProducerImpl:661 | [persistent://public/default/my-topic, standalone-0-0] Closing producer for topic persistent://public/default/my-topic
2022-08-31 13:56:49.016 INFO  [139834303919872] ProducerImpl:704 | [persistent://public/default/my-topic, standalone-0-0] Closed producer
2022-08-31 13:56:49.017 INFO  [139834284959488] ClientConnection:1560 | [127.0.0.1:51658 -> 127.0.0.1:6650] Connection closed
2022-08-31 13:56:49.017 INFO  [139834284959488] ClientConnection:263 | [127.0.0.1:51658 -> 127.0.0.1:6650] Destroyed connection
2022-08-31 13:56:49.018 INFO  [139834303919872] ExecutorService:47 | Event loop of ExecutorService exits successfully
2022-08-31 13:56:49.018 INFO  [139834337716672] ProducerImpl:628 | Producer - [persistent://public/default/my-topic, standalone-0-0] , [batchMessageContainer = { BatchMessageContainer [size = 0] [bytes = 0] [maxSize = 1000] [maxBytes = 131072] [topicName = persistent://public/default/my-topic] [numberOfBatchesSent_ = 2] [averageBatchSize_ = 0.5] }]

I suspect there are multiple GCC compilers in your environment and you didn't set the related environment variables (PATH and LD_LIBRARY_PATH) well. Could you show the ldd result of your executable file and the GCC version?

ldd <your-executable>
g++ --version

I just remember a thing. The RPM package was compiled from GCC 4.8.5, i.e. when you use the library with a GCC >= 5, you must add the -D_GLIBCXX_USE_CXX11_ABI=0 to fix the ABI compatibility issue. See here for details.

[zhcn]

@BewareMyPower

g++ --version
g++ (GCC) 7.3.1 20180303 (Red Hat 7.3.1-6)
Copyright © 2017 Free Software Foundation, Inc.

ldd shared
      linux-vdso.so.1 =>  (0x00007ffc661d8000)
        /$LIB/libonion.so => /lib64/libonion.so (0x00007ff948ae0000)
        libpulsar.so.2.10.1 => /lib/libpulsar.so.2.10.1 (0x00007ff947db6000)
        libstdc++.so.6 => /lib64/libstdc++.so.6 (0x00007ff947a2b000)
        libm.so.6 => /lib64/libm.so.6 (0x00007ff947729000)
        libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007ff947512000)
        libc.so.6 => /lib64/libc.so.6 (0x00007ff94714e000)
        libdl.so.2 => /lib64/libdl.so.2 (0x00007ff946f4a000)
        librt.so.1 => /lib64/librt.so.1 (0x00007ff946d42000)
        libpthread.so.0 => /lib64/libpthread.so.0 (0x00007ff946b26000)
        /lib64/ld-linux-x86-64.so.2 (0x00007ff9489c7000)

[zhcn]

Could you give a complete example? Your code cannot be compiled.

I just downloaded the latest code in branch-2.11 and built the Pulsar C++ client from source. The following example works well.

#include <pulsar/Client.h>
using namespace pulsar;

void func() {
    auto t = std::make_shared<std::string>("");
    std::cout << t << std::endl;
}

int main() {
    func();
    Client client("pulsar://localhost:6650");
    std::string topic = "my-topic";
    Producer producer;
    auto result = client.createProducer(topic, producer);
    if (result != ResultOk) {
        std::cout << "Error creating producer: " << result << std::endl;
        return 1;
    }
    MessageId messageId;
    producer.send(MessageBuilder().setContent("hello").build(), messageId);
    client.close();
    return 0;
}

Outputs:

0x12b9c08
2022-08-31 13:34:21.891 INFO  [140604211804928] ClientConnection:189 | [<none> -> pulsar://localhost:6650] Create ClientConnection, timeout=10000
2022-08-31 13:34:21.891 INFO  [140604211804928] ConnectionPool:96 | Created connection for pulsar://localhost:6650
2022-08-31 13:34:21.896 INFO  [140604079314688] ClientConnection:375 | [127.0.0.1:51650 -> 127.0.0.1:6650] Connected to broker
2022-08-31 13:34:21.899 INFO  [140604079314688] HandlerBase:61 | [persistent://public/default/my-topic, ] Getting connection from pool
2022-08-31 13:34:21.903 INFO  [140604079314688] ProducerImpl:186 | [persistent://public/default/my-topic, ] Created producer on broker [127.0.0.1:51650 -> 127.0.0.1:6650] 
2022-08-31 13:34:21.907 INFO  [140604211804928] ClientImpl:507 | Closing Pulsar client with 1 producers and 0 consumers
2022-08-31 13:34:21.908 INFO  [140604211804928] ProducerImpl:657 | [persistent://public/default/my-topic, standalone-0-2] Closing producer for topic persistent://public/default/my-topic
2022-08-31 13:34:21.909 INFO  [140604079314688] ProducerImpl:697 | [persistent://public/default/my-topic, standalone-0-2] Closed producer 0
2022-08-31 13:34:21.909 INFO  [140604060354304] ClientConnection:1560 | [127.0.0.1:51650 -> 127.0.0.1:6650] Connection closed
2022-08-31 13:34:21.909 INFO  [140604060354304] ClientConnection:263 | [127.0.0.1:51650 -> 127.0.0.1:6650] Destroyed connection
2022-08-31 13:34:21.910 INFO  [140604211804928] ProducerImpl:620 | Producer - [persistent://public/default/my-topic, standalone-0-2] , [batchMessageContainer = { BatchMessageContainer [size = 0] [bytes = 0] [maxSize = 1000] [maxBytes = 131072] [topicName = persistent://public/default/my-topic] [numberOfBatchesSent_ = 2] [averageBatchSize_ = 0.5] }]

The dependencies are:

-- Found OpenSSL: /usr/lib64/libcrypto.so (found version "1.0.2k")  
Failed to find Protobuf in config mode, try to find it from system path
-- Protobuf_LIBRARIES: /usr/local/lib/libprotobuf.so
-- Protobuf_INCLUDE_DIRS: /usr/local/include
-- Found CURL: /usr/lib64/libcurl.so (found version "7.29.0")  
-- Found Boost: /usr/include (found version "1.53.0")  
-- Linking with Boost:System
-- Using Boost::Regex
-- Found Boost: /usr/include (found version "1.53.0") found components: system regex 

I installed Protobuf 3.20.0 from source.

The GCC version:

$ g++ --version
g++ (GCC) 4.8.5 20150623 (Red Hat 4.8.5-44)
Copyright (C) 2015 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

#include <pulsar/Client.h>
#include <iostream>
#include <memory>

void func() {
    std::shared_ptr<std::string> t = std::make_shared<std::string>("");
}

int main(){
    pulsar::Client client("pulsar://localhost:6650");
    pulsar::Producer producer;
    pulsar::Result result = client.createProducer("my-topic", producer);
    if (result != pulsar::ResultOk) {
        std::cout << "Error creating producer: " << result << std::endl;
        return -1;
    }
    return 0;
}