Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make ConcurrencyDetector optional #24125

Merged
merged 4 commits into from
Feb 19, 2021
Merged

Make ConcurrencyDetector optional #24125

merged 4 commits into from
Feb 19, 2021

Conversation

roji
Copy link
Member

@roji roji commented Feb 11, 2021

ConcurrencyDetector uses an AsyncLocal to allow reentrant async usages; AsyncLocal triggers lots of ExecutionAsync allocations, plus the bool that we set gets boxed internally:

ConcurrencyDetector

A quick search in the codebase shows that we don't actually need the reentrancy.

OK yeah, Lazy_load_one_to_one_reference_with_recursive_property requires reentrancy, as well as queries being executed in the top-most projection (e.g. Select_nested_projection, Top_level_projection_track_entities_before_passing_to_client_method), which I think also require MARS.

@roji roji linked an issue Feb 11, 2021 that may be closed by this pull request
@roji roji force-pushed the ConcurrencyDetectorSpeedup branch 2 times, most recently from d9152ca to b6e407f Compare February 11, 2021 17:10
@roji
Copy link
Member Author

roji commented Feb 11, 2021

FYI removing this AsyncLocal is worth almost 5% RPS in TE fortunes perf (from 74,909 RPS to 77,149).

@roji
Copy link
Member Author

roji commented Feb 12, 2021

Design decision: add a configuration flag to remove concurrency detection entirely (not just reentrancy).

@roji roji removed the needs-design label Feb 12, 2021
@roji roji closed this Feb 13, 2021
@roji roji reopened this Feb 13, 2021
@roji roji force-pushed the ConcurrencyDetectorSpeedup branch 2 times, most recently from 46a33c2 to 81f50d7 Compare February 16, 2021 11:02
@roji roji changed the title Remove reentrancy support from ConcurrencyDetector Make ConcurrencyDetector optional Feb 16, 2021
@roji roji force-pushed the ConcurrencyDetectorSpeedup branch 6 times, most recently from 9bac471 to 5f6916a Compare February 18, 2021 10:50
@roji roji marked this pull request as ready for review February 18, 2021 17:54
@roji
Copy link
Member Author

roji commented Feb 18, 2021

This adds the concurrency detector flag as a core singleton option, similar to detailed errors. An alternative would be to just register a different no-op implementation of IConcurrencyDetector, but it seems a bit better to bake the flag into the querying enumerable compiled query (we could in theory even remove the check entirely).

namespace Microsoft.EntityFrameworkCore
{
[Collection("ConcurrencyDetector")]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Avoid usage of collection and assign different store name in derived types. Then your cosmos test issue would also be resolved.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't we want to minimize the number of databases we create though?

Re the cosmos issue, I think the fix in this PR should go in anyway, since it makes the cosmos testing infrastructure behave more like the other providers'.

Basically I don't really care so much :)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am fine either way, so far we have gone with different names rather than using collection (perhaps because model always differed). It is trade-off of new pattern vs additional cost. May be @AndriySvyryd can suggest, or go with whatever you feel like.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With the Cosmos fix you should be able to use the same store name and avoid using collection, right?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note that we have SaveChanges which actually changes the database; without the collection, wouldn't the two classes run in parallel and therefore interfere with one another?

I can have the SaveChanges tests from the different classes look for different rows, so that even if they run in parallel they don't interfere, but it may be safer for the future to just prevent parallelization for this specific test suite... Let me know what you prefer and I'll do it.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we can look for different rows and avoid this then it would be better. Imagine some random intern in future removing collection from here trying to improve test perf. It passes locally but failed the official build. 😉

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Random future intern fiddling around with our tests makes me scared! Will change.

@roji roji requested a review from smitpatel February 18, 2021 21:22
@ghost
Copy link

ghost commented Feb 19, 2021

Hello @roji!

Because this pull request has the auto-merge label, I will be glad to assist with helping to merge this pull request once all check-in policies pass.

p.s. you can customize the way I help with merging this pull request, such as holding this pull request until a specific person approves. Simply @mention me (@msftbot) and give me an instruction to get started! Learn more here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Improve EF Core (non-tracking) query performance on TechEmpower Fortunes
3 participants