[Security Solution] Support more than 10k rules on Coverage Overview Dashboard #160698
Open
Description
Epic: https://github.com/elastic/security-team/issues/2905 (internal)
Relates to: #155184
Summary
Coverage Overview API can't handle more than 10k rules so the response is limited by this number. Add support for the higher number of rules.
Details
Coverage Overview API uses rulesClient.find() under the hood to fetch and handle the rules. The problem with that it uses ES Search API so requesting data outside of the result window (index.max_result_window) leads to an error
illegal_argument_exception: Result window is too large, from + size must be less than or equal to: [10000] but was [20000]. See the scroll api for a more efficient way to request large data sets. This limit can be set by changing the [index.max_result_window] index level setting.
A recommended way to overcome such a problem is to use Point in time API. rulesClient doesn't support it so it requires either extending the rulesClient implementation or find another way. There is a high chance it will be fixed by #155184 as it uses the same approach to fetch the rules.
Metadata
Assignees
Labels
Security Solution Detection Rule Management areaSecurity Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc.Security Detection Rule Management TeamSecurity Detection Response TeamFixes for quality problems that affect the customer experienceAddressing this issue will have a medium level of impact on the quality/strength of our product.