Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Tiered Caching] Bug fix for IndicesRequestCache StaleKey management #13070

Merged
Merged
Changes from 1 commit
Commits
Show all changes
42 commits
Select commit Hold shift + click to select a range
2911c25
Update IndicesRequestCache.java
kiranprakash154 Apr 3, 2024
05610d3
Update IndicesRequestCacheTests.java
kiranprakash154 Apr 3, 2024
917df93
Update CHANGELOG.md
kiranprakash154 Apr 3, 2024
ac1464f
revert
kiranprakash154 Apr 4, 2024
f2d72f2
revert
kiranprakash154 Apr 4, 2024
148ff74
Update IndicesRequestCache.java
kiranprakash154 Apr 4, 2024
d8451c2
Update IndicesRequestCacheTests.java
kiranprakash154 Apr 4, 2024
ad88936
Update IndicesRequestCache.java
kiranprakash154 Apr 4, 2024
87e5c46
Update IndicesRequestCacheTests.java
kiranprakash154 Apr 4, 2024
6b36ee6
Update IndicesRequestCache.java
kiranprakash154 Apr 8, 2024
bebfc3a
Update IndicesRequestCacheTests.java
kiranprakash154 Apr 8, 2024
4179f46
Merge branch 'main' into kp/stalekey-status-check
kiranprakash154 Apr 8, 2024
f42ebb8
Update IndicesRequestCacheTests.java
kiranprakash154 Apr 8, 2024
56f4e1f
Update IndicesRequestCacheTests.java
kiranprakash154 Apr 8, 2024
6630cfc
Update IndicesRequestCacheTests.java
kiranprakash154 Apr 8, 2024
5de80d0
Update IndicesRequestCache.java
kiranprakash154 Apr 8, 2024
139cbfc
Update IndicesRequestCacheTests.java
kiranprakash154 Apr 8, 2024
eaeba38
Update IndicesRequestCacheTests.java
kiranprakash154 Apr 9, 2024
6122aa2
Update IndicesRequestCache.java
kiranprakash154 Apr 9, 2024
e870a98
Update IndicesRequestCacheTests.java
kiranprakash154 Apr 9, 2024
f658de6
Merge branch 'main' into kp/stalekey-status-check
kiranprakash154 Apr 9, 2024
bed1121
Update IndicesRequestCache.java
kiranprakash154 Apr 9, 2024
e178ea7
Update IndicesRequestCacheTests.java
kiranprakash154 Apr 9, 2024
e14a54c
Merge branch 'kp/stalekey-status-check' of https://github.com/kiranpr…
kiranprakash154 Apr 9, 2024
3a2e852
code comments only
kiranprakash154 Apr 10, 2024
5989e1d
docs changes
kiranprakash154 Apr 10, 2024
68b949b
Update CHANGELOG.md
kiranprakash154 Apr 10, 2024
dd91613
revert catching AlreadyClosedException
kiranprakash154 Apr 11, 2024
707f8bd
assert
kiranprakash154 Apr 11, 2024
00dee8d
Merge branch 'main' into kp/stalekey-status-check
kiranprakash154 Apr 12, 2024
70cc990
conflicts
kiranprakash154 Apr 12, 2024
b6b5f2b
Update IndicesRequestCacheTests.java
kiranprakash154 Apr 12, 2024
4c69187
Update IndicesRequestCache.java
kiranprakash154 Apr 12, 2024
31d9c54
address comments
kiranprakash154 Apr 12, 2024
9aeb82d
Update IndicesRequestCache.java
kiranprakash154 Apr 12, 2024
8bfa6c4
Update IndicesRequestCache.java
kiranprakash154 Apr 12, 2024
e37282f
Merge branch 'main' into kp/stalekey-status-check
kiranprakash154 Apr 23, 2024
d5cdf50
address conflicts
kiranprakash154 Apr 23, 2024
9068227
spotless apply
kiranprakash154 Apr 23, 2024
5a15c87
address comments
kiranprakash154 Apr 23, 2024
430e57c
update code comments
kiranprakash154 Apr 23, 2024
581ea2a
address bug & add tests
kiranprakash154 Apr 24, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Update IndicesRequestCache.java
Signed-off-by: Kiran Prakash <awskiran@amazon.com>
  • Loading branch information
kiranprakash154 committed Apr 4, 2024
commit 148ff743c176f804dd49750f64cb688b4c32f8df
Original file line number Diff line number Diff line change
Expand Up @@ -523,15 +523,30 @@ private void updateCleanupKeyToCountMapOnCacheEviction(CleanupKey cleanupKey) {
}
ShardId shardId = indexShard.shardId();

cleanupKeyToCountMap.computeIfPresent(shardId, (shard, keyCountMap) -> {
keyCountMap.computeIfPresent(cleanupKey.readerCacheKeyId, (key, currentValue) -> {
// decrement the stale key count
staleKeysCount.decrementAndGet();
int newValue = currentValue - 1;
// Remove the key if the new value is zero by returning null; otherwise, update with the new value.
cleanupKeyToCountMap.computeIfPresent(shardId, (id, stringIntegerConcurrentMap) -> {
stringIntegerConcurrentMap.compute(cleanupKey.readerCacheKeyId, (readerCacheKeyId, count) -> {
// Check if the key is currently present
if (count != null) {
// The key does not exist, so this is already accounted in the staleKeysCount
// and hence needs to be decremented
staleKeysCount.decrementAndGet();
}

// Regardless of whether the key was initially present, we perform the decrement operation
// Calculate the new value assuming a null count as zero to handle non-existent keys
int newValue = (count == null ? 0 : count) - 1;

// If the new value is 0, remove the entry by returning null
return newValue == 0 ? null : newValue;
});
return keyCountMap;

// return null and remove the shardId entry if the shardId has no other entries
if (stringIntegerConcurrentMap.isEmpty()) {
return null;
}

// Return the modified map to ensure it gets updated
return stringIntegerConcurrentMap;
});
}

Expand Down Expand Up @@ -577,7 +592,7 @@ private void incrementStaleKeysCount(CleanupKey cleanupKey) {
return null;
}
}
// Return the modified countMap
// Return the modified countMap to retain updates
return countMap;
});
}
Expand Down