Extract Writer from Tracer

[delner]

Currently our Datadog::Writer lives inside Datadog::Tracer. This has a few drawbacks:

1. Creates tight coupling between "generating traces" and "writing traces"
   • To create a tracer, you must also create and configure a writer.
2. Creating tracers implicitly creates writers with threads & background processes
   • These threads have the potential of hanging around in applications and test suites

The goal of this PR is to decouple "tracing" from "writing", by creating a simple pub-sub event.

Tracer#trace_completed
        |
     (trace)
        |
        V
   Writer#write
        |
        V
Writer#flush_completed
        |
   (responses)
        |
        V
PrioritySampler#update

• Datadog::Tracer now triggers a :trace_completed event with the trace that finished.
• Datadog::Writer now subscribes to that event and ingests the trace for writing.
• Datadog::Writer now triggers a :flush_completed event with responses from agent.
• Datadog.tracer.sampler now subscribes to that event and ingests the responses to update the priority sampler.

This pattern has some nice benefits for the tracer overall:

1. Each component is simpler to test with less coupling
2. Multiple writers can subscribe to a tracer (you could conceivably have both an agent and file writer)
3. Tracing in CI can be much more lightweight (reroute traces to a test buffer instead of creating writer with threads)
4. Easier to swap alternative writers in (e.g. synchronous vs asynchronous, etc)
5. Much more similar to a ractorized message pattern, which may ease a migration to that implementation in the future.

[delner]

Thus far I have the core & its test suite updated. Tests for each integration still have to be updated, which is WIP.

[ivoanjo]

I'll be honest: This is a pretty big change and I don't feel at all confident that I can review it properly. Is there a way we can break this down into smaller steps perhaps?

[marcotc]

Despite the large number of files touched, the core changes to the core tracer components are surprisingly non-complex.
I really like this change, and I think it's worth pursuing its completion.

[delner]

@ivoanjo Totally understand how this PR can be overwhelming... it is a lot of lines changed. I think the more complicated part of all this is more in the test suite, which hopefully makes the operational code (in the lib) more manageable to review.

I'm still trying to get all the tests to pass (having trouble with some expectations around tracer defaults implicitly written into some of the tests) but I will look for opportunities to simplify/split parts, if possible, to make it smaller. I might be able to do this with adding the subscription events, or some of the Minitest coverage that migrated into RSpec. Much of it will have to remain in one PR I suspect.

[ivoanjo]

(Removed myself for now otherwise this always shows up on my "reviews assigned to you list" -- do please re-add me when you start tackling this again :D )