IllegalArgumentException when scroll ID has a node no longer part of the Cluster

[anuragrai16]

This pull request improves the handling of invalid or outdated scroll IDs in OpenSearch by refining the exception thrown when a referenced node is no longer part of the cluster.

Scroll IDs in OpenSearch are not simple unique identifiers—they are Base64-encoded payloads containing references to target nodes and shard-local searcher context IDs. When a client sends a scroll ID referencing a node that no longer exists (e.g., due to node replacement or cluster changes) or to a different cluster that does not have the node, the coordinator currently treats this as an internal server error (HTTP 500), assuming it to be a server-side issue.

However, in most cases, the issue lies with the scroll ID itself being invalid or stale, not with the server. This PR modifies the behavior such that when the coordinator fails to locate the referenced node in the cluster state, it now throws an IllegalArgumentException. This exception is correctly mapped to an HTTP 400 (Bad Request), providing more accurate feedback to the client.

Related Issues

Check List

• Functionality includes testing.
• API changes companion pull request created, if applicable.
• Public documentation issue/PR created, if applicable.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[github-actions]

❕ Gradle check result for 272f928: UNSTABLE

Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 72.73%. Comparing base (8f310f5) to head (45c6f76).
⚠️ Report is 5 commits behind head on main.

Additional details and impacted files

@@             Coverage Diff              @@
##               main   #19031      +/-   ##
============================================
- Coverage     72.89%   72.73%   -0.16%     
+ Complexity    69405    69325      -80     
============================================
  Files          5647     5647              
  Lines        319084   319085       +1     
  Branches      46157    46157              
============================================
- Hits         232594   232088     -506     
- Misses        67700    68225     +525     
+ Partials      18790    18772      -18     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[github-actions]

❌ Gradle check result for e50833f: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[msfroh]

@anuragrai16 -- Can you please update the PR description to capture the real problem?

Specifically, scroll IDs are not just unique identifiers. They are a Base64-encoded payload with target nodes and shard-local searcher context IDs. Anyone can send a scroll ID that mentions a node that doesn't exist (either because it never existed or because the node has been replaced) and the coordinator will throw an exception. OpenSearch currently assumes that it's a server error (i.e. not finding that node in the cluster state is a bug), when it's far more likely that the scroll ID is somehow invalid (e.g. it was created before some nodes were replaced).

[github-actions]

❌ Gradle check result for 5f04d21: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

✅ Gradle check result for 1e560bd: SUCCESS

[github-actions]

✅ Gradle check result for 45c6f76: SUCCESS