There may be deadlock when shutdown using http client.

[owent]

Describe your environment

OS: Linux

Steps to reproduce

• Using http client
• Call Shutdown when process is ready to exit.

What is the expected behavior?

Shutdown successful.

What is the actual behavior?

There will be deadlock some times.

Additional context

Thread 1:

#0  0x00007fb823d3f54d in __lll_lock_wait () from /lib64/libpthread.so.0
#1  0x00007fb823d3aeb6 in _L_lock_941 () from /lib64/libpthread.so.0
#2  0x00007fb823d3adaf in pthread_mutex_lock () from /lib64/libpthread.so.0
#3  0x0000000001becf7f in __gthread_mutex_lock (__mutex=0x5fede0a1808) at /usr/include/c++/4.8.2/x86_64-redhat-linux/bits/gthr-default.h:748
#4  __gthread_recursive_mutex_lock (__mutex=0x5fede0a1808) at /usr/include/c++/4.8.2/x86_64-redhat-linux/bits/gthr-default.h:810
#5  lock (this=0x5fede0a1808) at /usr/include/c++/4.8.2/mutex:175
#6  lock_guard (__m=..., this=<synthetic pointer>) at /usr/include/c++/4.8.2/mutex:414
#7  opentelemetry::v1::ext::http::client::curl::HttpClient::ScheduleAbortSession (this=0x5fede0a1798, session_id=2992)
    at /data/devops/workspace/p-4943d876cb734a609b55ad5c740c36ff/src/server/pub9/third_party/packages/opentelemetry-cpp-v1.10.0/ext/src/http/client/curl/http_client_curl.cc:413
#8  0x0000000001befbc4 in opentelemetry::v1::ext::http::client::curl::HttpOperation::Abort (this=<optimized out>)
    at /data/devops/workspace/p-4943d876cb734a609b55ad5c740c36ff/src/server/pub9/third_party/packages/opentelemetry-cpp-v1.10.0/ext/src/http/client/curl/http_operation_curl.cc:1201
#9  0x0000000001becec2 in opentelemetry::v1::ext::http::client::curl::Session::CancelSession (this=0x5ff9c090c88)
    at /data/devops/workspace/p-4943d876cb734a609b55ad5c740c36ff/src/server/pub9/third_party/packages/opentelemetry-cpp-v1.10.0/ext/src/http/client/curl/http_client_curl.cc:98
#10 0x0000000001bea69a in opentelemetry::v1::ext::http::client::curl::HttpClient::CancelAllSessions (this=0x5fede0a1798)
    at /data/devops/workspace/p-4943d876cb734a609b55ad5c740c36ff/src/server/pub9/third_party/packages/opentelemetry-cpp-v1.10.0/ext/src/http/client/curl/http_client_curl.cc:198
#11 0x0000000001bdff77 in opentelemetry::v1::exporter::otlp::OtlpHttpClient::Shutdown (this=0x5fede4406c0, timeout=...)
    at /data/devops/workspace/p-4943d876cb734a609b55ad5c740c36ff/src/server/pub9/third_party/packages/opentelemetry-cpp-v1.10.0/exporters/otlp/src/otlp_http_client.cc:840
#12 0x0000000001c08b2b in opentelemetry::v1::sdk::logs::BatchLogRecordProcessor::Shutdown (this=0x5fede1b48c0, timeout=...)
    at /data/devops/workspace/p-4943d876cb734a609b55ad5c740c36ff/src/server/pub9/third_party/packages/opentelemetry-cpp-v1.10.0/sdk/src/logs/batch_log_record_processor.cc:309
#13 0x0000000001c06750 in opentelemetry::v1::sdk::logs::MultiLogRecordProcessor::Shutdown (this=<optimized out>, timeout=...)
    at /data/devops/workspace/p-4943d876cb734a609b55ad5c740c36ff/src/server/pub9/third_party/packages/opentelemetry-cpp-v1.10.0/sdk/src/logs/multi_log_record_processor.cc:132
#14 0x0000000001c01f37 in opentelemetry::v1::sdk::logs::LoggerProvider::Shutdown (this=<optimized out>)
    at /data/devops/workspace/p-4943d876cb734a609b55ad5c740c36ff/src/server/pub9/third_party/packages/opentelemetry-cpp-v1.10.0/sdk/src/logs/logger_provider.cc:119
#15 0x00000000019049ff in operator() (__closure=<optimized out>, provider=...)

Thread 2:

#0  0x00007fb823d3f54d in __lll_lock_wait () from /lib64/libpthread.so.0
#1  0x00007fb823d3ae9b in _L_lock_883 () from /lib64/libpthread.so.0
#2  0x00007fb823d3ad68 in pthread_mutex_lock () from /lib64/libpthread.so.0
#3  0x0000000001bec4aa in __gthread_mutex_lock (__mutex=0x5fede0a17e0) at /usr/include/c++/4.8.2/x86_64-redhat-linux/bits/gthr-default.h:748
#4  lock (this=0x5fede0a17e0) at /usr/include/c++/4.8.2/mutex:134
#5  lock_guard (__m=..., this=<synthetic pointer>) at /usr/include/c++/4.8.2/mutex:414
#6  opentelemetry::v1::ext::http::client::curl::HttpClient::doRemoveSessions (this=this@entry=0x5fede0a1798)
    at /data/devops/workspace/p-4943d876cb734a609b55ad5c740c36ff/src/server/pub9/third_party/packages/opentelemetry-cpp-v1.10.0/ext/src/http/client/curl/http_client_curl.cc:525
#7  0x0000000001bec93a in operator() (__closure=<optimized out>, self=0x5fede0a1798)
    at /data/devops/workspace/p-4943d876cb734a609b55ad5c740c36ff/src/server/pub9/third_party/packages/opentelemetry-cpp-v1.10.0/ext/src/http/client/curl/http_client_curl.cc:338
#8  _M_invoke<0ul> (this=<optimized out>) at /usr/include/c++/4.8.2/functional:1732
#9  operator() (this=<optimized out>) at /usr/include/c++/4.8.2/functional:1720
#10 std::thread::_Impl<std::_Bind_simple<opentelemetry::v1::ext::http::client::curl::HttpClient::MaybeSpawnBackgroundThread()::__lambda8(opentelemetry::v1::ext::http::client::curl::HttpClient*)> >::_M_run(void) (this=<optimized out>) at /usr/include/c++/4.8.2/thread:115
#11 0x00007fb8234bc340 in ?? () from /lib64/libstdc++.so.6
#12 0x00007fb823d38ea5 in start_thread () from /lib64/libpthread.so.0
#13 0x00007fb822e35b0d in clone () from /lib64/libc.so.6