Is your feature request related to a problem? Please describe.

Our application spends a considerable amount of time (p50 10ms, p95 25ms) adding tags to traces on every single request. The tags are essential to our observability, but adding them slows down requests for users.

Describe the goal of the feature

The goal of the feature is to make it easy for developers to add tags to a trace after the response is sent to the user. This could be accomplished using rack after-reply procs (supported by puma, unicorn, maybe others?). We were able to do this but it involved monkey patching the rack request span to defer finishing:

class << span
  alias_method :original_finish, :finish

  def finish(_end_time = nil)
    # Record the time that the span is finished so we can pass it along later
    @finished_at = Datadog::Core::Utils::Time.now.utc
  end

  def deferred_finish
    original_finish(@finished_at)
  end
end

class AfterReplyMiddleware
  ...
  def call(env)
    env['rack.after_reply'] ||= []
    env['rack.after_reply'] << -> do
      span = env['datadog.rack_request_span']
      do_tagging(span, env)
      span.deferred_finish
    end
    @app.call(env)
  end
end

Ideally the developer could configure a proc to run (e.g. for tagging) after the response is sent to user, but before the span is sent to DataDog.

Describe alternatives you've considered

If the proc could execute on the background thread that writes the traces, that would also speed up response times, but that would require the developer to consider the thread safety of the proc (I'm pretty sure our tagging code is not thread safe)