Consider the performance harm of synchronized exporters

[taozhijiang]

Hi, I have tested and deployed the latest v1.2.0 C++ SDK in our production environments. Our cloud infrastructure provider supports HTTP OTLP exporter for the Tracer function.

I set the sample_ratio to 0.01, otherwise the Tracer will greately affect our service performance. However, when analyse the tracer data detailly, we find some slow handled requests, and most time are token by span->End() call actually.

Span::End() -> MultiSpanProcessor::OnEnd() -> SimpleSpanProcessor::OnEnd()

We can see, SimpleSpanProcessor::End() calls exporter with SpinLockMutex, and will call OtlpHttpExporter with http request and wait for the response synchronously. At this time, our business main procedure will hang for these operations.
I think http requests are inefficient and slow. So the export operation should better be executed by additional threads asynchronously, I known create additional theads in SDK is not a very graceful idea, so hope for better solutions here.

[esigo]

Hi @taozhijiang, thanks to reaching out to us. There is already a PR (#1209) by @owent under review.

[lalitb]

Also to add, @taozhijiang - did you consider using BatchSpanProcessor, as this performs upload in a separate thread, and so allows the application to continue with business logic without waiting for the upload to complete.

[taozhijiang]

Also to add, @taozhijiang - did you consider using BatchSpanProcessor, as this performs upload in a separate thread, and so allows the application to continue with business logic without waiting for the upload to complete.

Thanks for the suggestion, I will try this later.