Skip to content

Carry over version map size to prevent excessive resizing #27516

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Nov 24, 2017
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,14 @@ public abstract class ConcurrentCollections {
* Creates a new CHM with an aggressive concurrency level, aimed at high concurrent update rate long living maps.
*/
public static <K, V> ConcurrentMap<K, V> newConcurrentMapWithAggressiveConcurrency() {
return new ConcurrentHashMap<>(16, 0.75f, aggressiveConcurrencyLevel);
return newConcurrentMapWithAggressiveConcurrency(16);
}

/**
* Creates a new CHM with an aggressive concurrency level, aimed at high concurrent update rate long living maps.
*/
public static <K, V> ConcurrentMap<K, V> newConcurrentMapWithAggressiveConcurrency(int initalCapacity) {
return new ConcurrentHashMap<>(initalCapacity, 0.75f, aggressiveConcurrencyLevel);
}

public static <K, V> ConcurrentMap<K, V> newConcurrentMap() {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -102,7 +102,7 @@ public void beforeRefresh() throws IOException {
// map. While reopen is running, any lookup will first
// try this new map, then fallback to old, then to the
// current searcher:
maps = new Maps(ConcurrentCollections.<BytesRef,VersionValue>newConcurrentMapWithAggressiveConcurrency(), maps.current);
maps = new Maps(ConcurrentCollections.newConcurrentMapWithAggressiveConcurrency(maps.current.size()), maps.current);

// This is not 100% correct, since concurrent indexing ops can change these counters in between our execution of the previous
// line and this one, but that should be minor, and the error won't accumulate over time:
Expand All @@ -117,7 +117,7 @@ public void afterRefresh(boolean didRefresh) throws IOException {
// case. This is because we assign new maps (in beforeRefresh) slightly before Lucene actually flushes any segments for the
// reopen, and so any concurrent indexing requests can still sneak in a few additions to that current map that are in fact reflected
// in the previous reader. We don't touch tombstones here: they expire on their own index.gc_deletes timeframe:
maps = new Maps(maps.current, ConcurrentCollections.<BytesRef,VersionValue>newConcurrentMapWithAggressiveConcurrency());
maps = new Maps(maps.current, Collections.emptyMap());
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm probably lacking the larger context here but is it ok to change the implementation class from ConcurrentHashMap to HashMap here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah it's an immutable map and we should never ever modify it too.

}

/** Returns the live version (add or delete) for this uid. */
Expand Down