Skip to content

Starting spans in MongoDB instrumentation takes a long time due to IP lookups #2050

@krosen040

Description

@krosen040

We're using Spring Cloud Sleuth 3.0.3, which uses the MongoDB instrumentation from openzipkin/brave, but I'm filing this issue in this project because I noticed that this is about to change (see PR #2044), and the problematic code still remains.

In TraceMongoCommandListener, each time a new span is started the IP of the Mongo host is resolved, see here. Sometimes this can take a substantial amount of time. We have services where the request latency increased by up to 100ms per request when adding Spring Cloud Sleuth. When disabling the MongoDB instrumentation, it went down to normal levels.

I think the results of the hostname to IP lookups in InetAddress are cached, but at least in our environment that expires in a short amount of time (30 seconds). This problem can therefore become especially noticeable for services that receive few requests, since the cache won't be utilized.

I'm not sure what the best solution to this is, but a few ideas:

  • Make setting remoteIpAndPort opt-in or opt-out, using some property
  • Don't resolve the IP, just set the hostname as some tag (I'm not very familiar with the Spring Cloud Sleuth/Brave APIs, but I know that OpenTracing and OpenTelemetry define standard tags for the hostname)

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions