Introduced new setting search.query.max_query_string_heap_size

[nibix]

Description

This introduces a new cluster wide setting search.query.max_query_string_heap_size which can be used to limit the max heap utilization of instantiated Lucene-style query strings. This complements the search.query.max_query_string_length setting.

A couple of considerations:

• The default value is 50 MB, which should be a generous limit. When considering this value one has to keep in mind that a single search operation instantiates the query strings per searched shard. Thus, several instances might be created in parallel. Thus, this value should be chosen not too close to the total heap size of the cluster.
• The code that estimates the heap size uses a heuristic value for the size of a single member field query object of 180 bytes. This seemed to be a good fit for our tests. The class RamUsageEstimator would also offer the possibility to use reflective methods to gain more precise figures for the object sizes. However, I am not sure if this might have performance impacts. Thus, I have not used this for now.

Check List

• Functionality includes testing.
• API changes companion pull request created, if applicable.
• Public documentation issue/PR created, if applicable.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[github-actions]

❌ Gradle check result for a39b97b: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

❌ Gradle check result for 626b486: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

❌ Gradle check result for 027860e: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

✅ Gradle check result for 7818306: SUCCESS

[codecov]

Codecov Report

❌ Patch coverage is 65.95745% with 16 lines in your changes missing coverage. Please review.
✅ Project coverage is 73.06%. Comparing base (48b08fb) to head (7818306).

Files with missing lines	Patch %	Lines
...pensearch/index/search/QueryStringQueryParser.java	62.79%	14 Missing and 2 partials ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##               main   #19493      +/-   ##
============================================
- Coverage     73.08%   73.06%   -0.02%     
- Complexity    70470    70484      +14     
============================================
  Files          5712     5712              
  Lines        322762   322788      +26     
  Branches      46743    46746       +3     
============================================
- Hits         235877   235856      -21     
- Misses        67912    67931      +19     
- Partials      18973    19001      +28     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.