Reuse HTTP connection between trace flushes

[marcotc]

I think it would be a bit safer to implement a retry logic first, as unforeseen persistent connections issues would be mitigated by a retry mechanism

This PR changes the Tracing HTTP transport adapter (lib/ddtrace/transport/http/adapters/net.rb) to reuse its HTTP connection internally.

Previously, we created a new connection and closed it on every trace batch flush, which can happens as frequently as once per second.

This PR does not close the connection, and instead reuses it for successive flushes.

If the remote connection errs out (e.g. other side closes the connection), the connection will be retried internally by Net::HTTP.

Benchmarking

Invoke the tracer transport pipeline, flushing a single trace containing 50 spans. The tail end of the pipeline will invoke the HTTP client modified in this PR.

Results in "transport flushes per second".

Before

     http_transport     441.739  (±15.2%) i/s -     29.486k in  70.012534s

After

     http_transport     774.359  (± 5.3%) i/s -     54.072k in  70.056594s

Reusing the HTTP connection is 1.75x faster then creating a brand new connection every time.

[marcotc]

Moved to a shared spec helper, now that checking for leaked FDs is needed in multiple places.

[marcotc]

I decided not to use this approach for HTTP testing, but I feel like this refactor is still good.

[marcotc]

I should move this out of this PR.

[marcotc]

We have this quite clever monkey patch to expose private members for testing, but is actually never used.

This kind of test-specific, internal access to the tracer (specifically monitoring if the AsyncTransport has finished flushing) was the best way to write reliable tests around HTTP connection reuse.

But polluting the main class space with methods that won't actually be present when users use ddtrace in production is scary, since we could write features that use methods that are not available in production, but will always succeed in our test suite.

Because of this, I've moved this monkey patching to a refinement, so that it only applies to the current spec file under test, never leaking to lib/ code.

Because the file changed so much, git thinks I completely deleted it (which I kind of did, for most of the unused features). spec/support/test_access_refinement.rb contains these new refinements.

[codecov-commenter]

Codecov Report

Merging #2227 (6fe546d) into master (9dbd6d5) will decrease coverage by 0.00%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master    #2227      +/-   ##
==========================================
- Coverage   98.32%   98.32%   -0.01%     
==========================================
  Files        1120     1125       +5     
  Lines       60401    60703     +302     
==========================================
+ Hits        59389    59685     +296     
- Misses       1012     1018       +6     

Impacted Files	Coverage Δ
lib/datadog/appsec/extensions.rb	85.91% <0.00%> (-2.15%)	⬇️
lib/datadog/appsec/response.rb	86.66% <0.00%> (-1.80%)	⬇️
lib/datadog/appsec/processor.rb	91.75% <0.00%> (-1.32%)	⬇️
...dog/profiling/collectors/cpu_and_wall_time_spec.rb	97.48% <0.00%> (-0.42%)	⬇️
spec/datadog/integration_spec.rb	97.40% <0.00%> (-0.13%)	⬇️
lib/datadog/core/configuration/settings.rb	99.18% <0.00%> (-0.01%)	⬇️
spec/support/faux_transport.rb	100.00% <0.00%> (ø)
lib/datadog/core/metrics/ext.rb	100.00% <0.00%> (ø)
spec/ddtrace/release_gem_spec.rb	100.00% <0.00%> (ø)
lib/datadog/appsec/configuration.rb	100.00% <0.00%> (ø)
... and 32 more

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more