Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Optimize Adaptive Server Selection #13952

Merged
merged 12 commits into from
Sep 12, 2024
Prev Previous commit
Next Next commit
move hashset
  • Loading branch information
praveenc7 committed Sep 9, 2024
commit dda2ddc3a709a7af893eb334076a4ffeb1e535c0
Original file line number Diff line number Diff line change
Expand Up @@ -140,6 +140,7 @@ private Pair<Map<String, String>, Map<String, String>> selectServersUsingAdaptiv
Map<String, String> segmentToSelectedInstanceMap = new HashMap<>(HashUtil.getHashMapCapacity(segments.size()));
// No need to adjust this map per total segment numbers, as optional segments should be empty most of the time.
Map<String, String> optionalSegmentToInstanceMap = new HashMap<>();
Set<String> selectedServers = new HashSet<>();
for (String segment : segments) {
// NOTE: candidates can be null when there is no enabled instances for the segment, or the instance selector has
// not been updated (we update all components for routing in sequence)
Expand All @@ -156,6 +157,11 @@ private Pair<Map<String, String>, Map<String, String>> selectServersUsingAdaptiv
if (!serverRankMap.isEmpty()) {
int bestRank = Integer.MAX_VALUE;
for (SegmentInstanceCandidate candidate : candidates) {
// If the candidate is already selected, implying that it is the best candidate for the segment, select it and exit the loop.
if (selectedServers.contains(candidate.getInstance())) {
selectedInstance = candidate;
break;
}
Integer rank = serverRankMap.get(candidate.getInstance());
if (rank == null) {
// Let's use the round-robin approach until stats for all servers are populated.
Expand All @@ -168,6 +174,8 @@ private Pair<Map<String, String>, Map<String, String>> selectServersUsingAdaptiv
}
}
}
// Add the selected instance to the set.
selectedServers.add(selectedInstance.getInstance());
// This can only be offline when it is a new segment. And such segment is marked as optional segment so that
// broker or server can skip it upon any issue to process it.
if (selectedInstance.isOnline()) {
Expand Down