Skip to content

Conversation

@kmahar
Copy link
Contributor

@kmahar kmahar commented Oct 7, 2020

This is a longstanding inefficiency in the driver, but only came up as a problem when using the new BSON library.

Even if the user has no handlers registered, we create a corresponding Swift event, which in the case of e.g. a command started event for a large command is quite expensive (especially now that - at least for the moment - by default we validate incoming documents.)

This PR amends that by introducing a new concept of a monitoring "component" for each event type. Depending on the component of the event to be published, we inspect the client to see if any corresponding handlers exist.

I am feeling like this file is getting a little wild, using a lot of different types and protocols, but at the same time I couldn't figure out a clean way to accomplish this without the addition of the new type. I technically could have checked if the type conformed to CommandEventProtocol or not and acted accordingly but that felt kind of hacky. If you have any ideas let me know.

Note that as part of this I also resolved SWIFT-524 (see inline comment about test changes).

This problem might have somewhat "gone away" if we stopped validating the command documents (which we should probably also do), but this was a very quick win that I think is worth implementing regardless.

Comparison of runtimes, in seconds:

Benchmark Old New Unoptimized With this fix
Small doc bulk insert 10k copies .096 4258.673 0.424
Large doc bulk insert 10 copies 0.332 59.637 2.79
small doc insertOne 10k times 3.258 8.276 4.481
findOne by _id 10k times 4.07 229.698 14.597
find() and empty cursor, 10k small docs 0.468 10436.246 1.025
find() and empty cursor, 10 large docs 0.032 105.365 3.568

Note that the first and second to last ones literally got 10,000 times faster...!! My spawn host was working hard last night 🙂

I need to do some further investigation into what the remaining bottlenecks for these benchmarks are, which should be much easier now that their runtimes are significantly lower. I suspect validation continues to be a main culprit, which we still encounter via e.g. parsing reply documents and reading documents from cursors. I will probably end up writing a couple of new small benchmarks that specifically test BSON validation in order to speed up iterating on that code.

let event = type.init(mongocEvent: mongocEvent)

// TODO: SWIFT-524: remove workaround for CDRIVER-3256
if let tdChanged = event as? TopologyDescriptionChangedEvent,
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this CDRIVER ticket was resolved as part of 1.17.

}

expect(receivedEvents.count).to(equal(4))
expect(receivedEvents.count).to(equal(5))
Copy link
Contributor Author

@kmahar kmahar Oct 7, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so, the CDRIVER ticket mentioned above only actually resolved the duplicate server changed event issue. The topology changed one still existed. however, according to DRIVERS-944, publishing the event is consistent with the SDAM monitoring tests.

I'm kind of 50/50 on whether to make the change, because as Kevin points out in the comment, there is no visible change to the user from the topology description. But to not publish it would be inconsistent with the spec tests and what other drivers seem to do if they pass the tests.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah I kind of agree that we probably shouldn't emit events in this case, but also think for simplicity and conformity we should just do what the tests require. If that ticket ever gets resolved in the other direction we could just stop emitting the event--I don't think it would constitute a breaking behavioral change.

@kmahar kmahar requested a review from patrickfreed October 7, 2020 03:36
@codecov-io
Copy link

codecov-io commented Oct 7, 2020

Codecov Report

❗ No coverage uploaded for pull request base (master@bf20ebb). Click here to learn what that means.
The diff coverage is 92.30%.

Impacted file tree graph

@@            Coverage Diff            @@
##             master     #531   +/-   ##
=========================================
  Coverage          ?   77.09%           
=========================================
  Files             ?      132           
  Lines             ?    14095           
  Branches          ?        0           
=========================================
  Hits              ?    10866           
  Misses            ?     3229           
  Partials          ?        0           
Impacted Files Coverage Δ
Sources/MongoSwift/APM.swift 85.77% <86.95%> (ø)
Tests/MongoSwiftSyncTests/SDAMTests.swift 65.67% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update bf20ebb...3e9d0bc. Read the comment docs.

}

expect(receivedEvents.count).to(equal(4))
expect(receivedEvents.count).to(equal(5))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah I kind of agree that we probably shouldn't emit events in this case, but also think for simplicity and conformity we should just do what the tests require. If that ticket ever gets resolved in the other direction we could just stop emitting the event--I don't think it would constitute a breaking behavioral change.

@kmahar kmahar merged commit 1953dcc into master Oct 7, 2020
@kmahar kmahar deleted the SWIFT-1015/lazy-event-creation branch October 7, 2020 16:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants