Skip to content

YTDB-165: Ytdb 165 add non throwing get to cas array clean rebased#671

Open
artem (logart) wants to merge 11 commits into
developfrom
ytdb-165-add-non-throwing-get-to-cas-array-clean-rebased
Open

YTDB-165: Ytdb 165 add non throwing get to cas array clean rebased#671
artem (logart) wants to merge 11 commits into
developfrom
ytdb-165-add-non-throwing-get-to-cas-array-clean-rebased

Conversation

@logart
Copy link
Copy Markdown
Collaborator

Motivation:

While implementing #YTDB-165, I have a place where I do casArray.get(pageIndex) to check if the page is in the readCache already.

This operation fails for all new pages since they are not in the array yet.

Based on the logic of the method, I will then cache those pages and add them to the array.
Solution:

Add a flag (I know this is bad, but I feel like copying the method is even worse...) to the get method of CASObjectArray, so it will define either throw on out of bound (array behaviour) or return null (map behaviour).

An alternative solution would be to create an atomic operation. Something along the lines like getOrSet which will get the existing value or (if it is missing) set a computed one.

https://youtrack.jetbrains.com/issue/YTDB-495/Make-CASObjectArray-behave-like-a-map

@github-actions github-actions Bot changed the title Ytdb 165 add non throwing get to cas array clean rebased YTDB-165: Ytdb 165 add non throwing get to cas array clean rebased Feb 19, 2026
@gemini-code-assist
Copy link
Copy Markdown
Contributor

Summary of Changes

Hello artem (@logart), I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request refines the CASObjectArray to offer more flexible retrieval methods, moving beyond strict array-like behavior to incorporate map-like characteristics. The primary goal is to enable checking for the presence of an element at a given index without necessarily throwing an exception if the index is out of bounds, which is particularly useful for optimizing cache interactions. This change introduces a configurable get operation and a dedicated getOrEmpty method, complemented by new constructors and a defaultPlaceholder for improved usability. Robust multi-threaded tests have been added to validate the concurrent integrity of these new functionalities.

Highlights

  • Enhanced CASObjectArray Get Behavior: The CASObjectArray class now includes an overloaded get method that accepts a throwOnOutOfBounds flag, allowing callers to control whether an ArrayIndexOutOfBoundsException is thrown or a default placeholder is returned when accessing an out-of-bounds index. A new getOrEmpty method was also added for convenience, which will return a default placeholder (or null) without throwing an exception.
  • Default Placeholder Support: A defaultPlaceholder field was introduced in CASObjectArray, along with new constructors, to allow specifying a default value to be returned by getOrEmpty when an element is not found. An overloaded set method was also added to utilize this default placeholder.
  • New Multi-threaded Tests: A new test file, CASObjectArrayMTTest.java, was added to provide comprehensive multi-threaded tests for add, set, and compareAndSet operations using the Fray testing framework, ensuring the concurrency safety of the enhanced CASObjectArray.
  • Build Configuration Update: The pom.xml file was updated to integrate the Fray Maven plugin and its runtime and JUnit dependencies, enabling the execution of the new multi-threaded tests.
Changelog
  • core/pom.xml
    • Added fray-plugins-maven plugin to the build section.
    • Included fray-runtime and fray-junit dependencies with test scope.
  • core/src/main/java/com/jetbrains/youtrackdb/internal/common/concur/collection/CASObjectArray.java
    • Introduced defaultPlaceholder field.
    • Added two new constructors: a no-argument constructor and one accepting a defaultPlaceholder.
    • Added an overloaded set method that uses the defaultPlaceholder.
    • Refactored the get(int index) method to call a new private get(int index, boolean throwOnOutOfBounds) method.
    • Implemented getOrEmpty(int index) method to return the defaultPlaceholder (or null) for out-of-bounds indices.
  • core/src/test/java/com/jetbrains/youtrackdb/internal/common/concur/collection/CASObjectArrayMTTest.java
    • Added new file CASObjectArrayMTTest.java.
    • Implemented addShouldReturnCorrectIndex concurrency test.
    • Implemented setShouldEventuallyUpdateArray concurrency test.
    • Implemented compareAndSetShouldUpdateOnlyOnce concurrency test.
  • core/src/test/java/com/jetbrains/youtrackdb/internal/common/concur/collection/CASObjectArrayTest.java
    • Added shouldThrowIfIndexIsOutOfBounds test case.
    • Added shouldReturnNullIfElementDoesNotExistsLikeAMap test case.
    • Added shouldReturnDefaultPlaceholderIfElementDoesNotExists test case.
Activity
  • The pull request was initiated by logart.
  • The motivation for this change stems from the need to check for page existence in a readCache (related to YTDB-165) without encountering ArrayIndexOutOfBoundsException for new pages.
  • The author considered and discussed alternative solutions, such as copying the method or implementing an atomic getOrSet operation.
  • The changes align with the broader goal of making CASObjectArray behave more like a map, as suggested in YTDB-495.
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment Gemini (@gemini-code-assist) Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on Gemini (@gemini-code-assist) comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request modifies CASObjectArray to allow non-throwing get operations and introduces new test dependencies along with a multi-threaded test class. The primary modification involves adding a defaultPlaceholder field and a getOrEmpty method, along with an overloaded get method that accepts a throwOnOutOfBounds boolean, enhancing flexibility in element retrieval. However, the new get and set methods lack proper validation for negative indices, potentially leading to infinite loops and Denial of Service. Additionally, the default constructor introduces a regression where the set(int, T) method reliably crashes with a NullPointerException due to an uninitialized placeholder. Ensure explicit bounds checking is added and the default placeholder is safely handled.

Comment on lines +115 to 123
private T get(int index, boolean throwOnOutOfBounds) {
final var size = this.size.get();

if (size <= index) {
throw new ArrayIndexOutOfBoundsException("Requested " + index + ", size is " + size);
if (throwOnOutOfBounds) {
throw new ArrayIndexOutOfBoundsException("Requested " + index + ", size is " + size);
}
return defaultPlaceholder;
}
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

security-high high

The get method (and by extension getOrEmpty) does not properly validate that the index parameter is non-negative, potentially leading to a Denial of Service. The existing check if (size <= index) fails for negative indices because size is always non-negative. For index < -1, the code calculates a containerIndex of 31 and enters an infinite loop of Thread.yield() while waiting for a container that will never be created. A check for index < 0 should be added to ensure the method fails safely for all out-of-bounds indices.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is fixed right in the beginning of the method.

Comment on lines +107 to +108
public T getOrEmpty(int index) {
return get(index, false);
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The getOrEmpty method provides a convenient way to retrieve an element without throwing an ArrayIndexOutOfBoundsException, returning the defaultPlaceholder instead. This aligns with the motivation of the PR.

Comment on lines 111 to +112
public T get(int index) {
return get(index, true);
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The original get method is now a wrapper around the new private get method, ensuring backward compatibility while leveraging the new functionality.

Comment on lines +207 to +211
@Test
public void shouldThrowIfIndexIsOutOfBounds() {
final var array = new CASObjectArray<Integer>();

Assert.assertThrows(IndexOutOfBoundsException.class, () -> array.get(0));
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

This test case correctly verifies that the default get method still throws IndexOutOfBoundsException when the index is out of bounds, maintaining the original behavior.

Comment on lines +214 to +218
@Test
public void shouldReturnNullIfElementDoesNotExistsLikeAMap() {
final var array = new CASObjectArray<Integer>();

Assert.assertNull(array.getOrEmpty(0));
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

This test confirms that getOrEmpty returns null when no default placeholder is specified and the index is out of bounds, which is the expected map-like behavior.

Comment on lines +221 to +226
@Test
public void shouldReturnDefaultPlaceholderIfElementDoesNotExists() {
Integer defaultPlaceholder = (int) (Math.random() * Integer.MAX_VALUE);
final var array = new CASObjectArray<>(defaultPlaceholder);

Assert.assertSame(defaultPlaceholder, array.getOrEmpty(0));
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

This test ensures that getOrEmpty returns the specified defaultPlaceholder when the index is out of bounds, validating the new constructor and behavior.

@github-actions
Copy link
Copy Markdown

github-actions Bot commented Feb 19, 2026

Qodana for JVM

46 new problems were found

Inspection name Severity Problems
Local variable type can be omitted 🔶 Warning 9
Default annotation parameter value 🔶 Warning 6
Statement lambda can be replaced with expression lambda 🔶 Warning 1
Spelling ◽️ Notice 27
Grammar ◽️ Notice 2
Duplicated code fragment ◽️ Notice 1

💡 Qodana analysis was run in the pull request mode: only the changed files were checked
☁️ View the detailed Qodana report

Contact Qodana team

Contact us at qodana-support@jetbrains.com

public void set(int index, T value) {
if (defaultPlaceholder == null) {
throw new UnsupportedOperationException(
"set without placeholder only works with new CASObjectArray(defaultPlaceholder) constructed object."
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

String concatenation produces ...object.Please use...

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, I don't get what should use instead.

return container.compareAndSet(indexInsideContainer, oldValue, value);
}

public T getOrEmpty(int index) {
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

getOrDefault is more appropriate


@Test
public void shouldReturnDefaultPlaceholderIfElementDoesNotExists() {
Integer defaultPlaceholder = (int) (Math.random() * Integer.MAX_VALUE);
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That is non-reproducable test as seed is not printed.

private final T defaultPlaceholder;

public CASObjectArray() {
this(null);
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

seems like null can not be passed, does not it ?

Copy link
Copy Markdown
Collaborator Author

@logart artem (logart) Feb 24, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It could, by default we are setting null for the new element(legacy behaviour which I want to keep).
That's why when you use set(index, value, placholder) method the placeholder is mandatory and non null.
With the new api I've added you either need to:

  1. Use non default constructor with placeholder, this way you are free to use both set(index, value) and set(index, value, placeholder) methods, or
  2. Use default constructor like it used to be, but then set(index, value) will throw an exception since null placeholder is indeed not allowed.

I've thought about this design for a while, and I don't really like it, but I don't know how to make this change backward compatible in other way.

Comment thread core/pom.xml Outdated
</configuration>
</plugin>
<plugin>
<groupId>org.pastalab.fray.maven</groupId>
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The core module's surefire is configured with surefire-junit47 , are fray tests really executed

@logart artem (logart) force-pushed the ytdb-165-add-non-throwing-get-to-cas-array-clean-rebased branch 3 times, most recently from 068af4e to df30def Compare February 27, 2026 10:19
@github-actions
Copy link
Copy Markdown

github-actions Bot commented Feb 27, 2026

Coverage Gate Results

Thresholds: 85% line, 70% branch

Line Coverage: ✅ 89.5% (17/19 lines)

File Coverage Uncovered Lines
core/src/main/java/com/jetbrains/youtrackdb/internal/common/concur/collection/CASObjectArray.java ✅ 89.5% (17/19) 51-52

Branch Coverage: ✅ 83.3% (10/12 branches)

File Coverage Lines with Uncovered Branches
core/src/main/java/com/jetbrains/youtrackdb/internal/common/concur/collection/CASObjectArray.java ✅ 83.3% (10/12) 47, 106

@github-actions
Copy link
Copy Markdown

github-actions Bot commented Feb 27, 2026

Mutation Testing Gate Results

Threshold: 85%

✅ No mutations found — mutation gate skipped.

@andrii0lomakin
Copy link
Copy Markdown
Collaborator

artem (@logart) do we need this one ?

@logart artem (logart) force-pushed the ytdb-165-add-non-throwing-get-to-cas-array-clean-rebased branch from df30def to 4008d6d Compare March 3, 2026 09:14
@logart
Copy link
Copy Markdown
Collaborator Author

Andrii Lomakin (@andrii0lomakin) not sure how to reply to specific comment, so adding a link: #671 (comment)

If you mean PR, yes, I am integrating your vmlense configuration from develop and checking if all my tests pass.

@logart artem (logart) force-pushed the ytdb-165-add-non-throwing-get-to-cas-array-clean-rebased branch from 4008d6d to 322da95 Compare March 3, 2026 09:35
artem.loginov added 10 commits April 22, 2026 12:01
Add default placeholder to simplify null/placeholder comparison.
Add a check on correct set usage with defaultPlaceholder value.
Add more concurrent tests.
Fix issue found by review.
Negative index hang check.
Replace fray with vmlens.
Revert junit 5 changes from pom.
Fix review comments.
Rename getOrEmpty to getOrDefault.
Don't use string concatenation.
Print value of defaultPlaceholder if test fails.
Do not run vmlense and jacoco together. This cause false positive raises on the vmlense side.
@logart artem (logart) force-pushed the ytdb-165-add-non-throwing-get-to-cas-array-clean-rebased branch from e35566c to fa5faad Compare April 22, 2026 10:14
Remove changes not related to my PR.
Combine existing and my own tests in com.jetbrains.youtrackdb.internal.common.concur.collection.CASObjectArrayMTTest
@github-actions
Copy link
Copy Markdown

Test Count Gate Results

Tolerance: 5% drop allowed per module

Overall: ✅ 19959 tests (baseline: 19944, +15)

Module Baseline Current Change Status
core 9396 9411 +15
docker-tests 1891 1891 +0
embedded 1931 1931 +0
examples 6 6 +0
gremlin-annotations 30 30 +0
jmh-ldbc 39 39 +0
server 5504 5504 +0
tests 1147 1147 +0

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants