[8.17] [Security Solution] Added concurrency limits and request throttling to prebuilt rule routes (#209551) #210773
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…o prebuilt rule routes (elastic#209551) **Resolves: elastic#208357 **Resolves: elastic#208355 ## Summary To prevent possible OOM errors, we need to limit concurrent requests to prebuilt rule routes (see attached tickets for more details). - `installation/_perform` and `upgrade/_perform` endpoints - Concurrency is limited to one parallel call. If another call is made simultaneously, the server responds with 429 Too Many Requests. - On the front end, all rule install and upgrade operations are retried in case of a 429 response. This ensures proper handling when a user clicks multiple times an update or install rule buttons - `prebuilt_rules/_bootstrap` endpoint - Install prebuilt rules and endpoint packages sequentially instead of in parallel to prevent from having them both downloaded into memory simultaneously. - Added a 30-minute socket timeout to prevent the proxy from closing the connection while rule installation is in progress. - Introduced a `throttleRequests` wrapper, ensuring the endpoint handler is called only once when multiple concurrent requests are received. - The first request triggers the handler, while subsequent requests wait for the first one to complete and reuse its result. - This prevents costly prebuilt rule package installation from running in parallel. - Reusing the response ensures the frontend correctly invalidates cached prebuilt rule queries. Since concurrent frontend requests should receive the same installed package information, responding with 421 and using the retry logic as in cases above is not an option here because the second request would receive a package installation skipped response leading to no cache invalidation. - `installation/_review` and `upgrade/_review` endpoints - Concurrency is limited to one parallel call. If another call is made simultaneously, the server responds with 429 Too Many Requests. - On the front end, all rule install and upgrade operations are retried in case of a 429 response. This ensures proper handling when a user clicks multiple times an update or install rule buttons (cherry picked from commit c5557f3) # Conflicts: # x-pack/plugins/security_solution/public/detection_engine/rule_management/api/hooks/prebuilt_rules/capped_exponential_backoff.ts # x-pack/plugins/security_solution/public/detection_engine/rule_management/api/hooks/prebuilt_rules/retry_on_rate_limited_error.ts # x-pack/plugins/security_solution/public/detection_engine/rule_management/api/hooks/prebuilt_rules/use_bootstrap_prebuilt_rules.ts # x-pack/plugins/security_solution/public/detection_engine/rule_management/api/hooks/prebuilt_rules/use_perform_specific_rules_upgrade_mutation.ts # x-pack/plugins/security_solution/public/detection_engine/rule_management/api/hooks/use_bootstrap_prebuilt_rules.ts # x-pack/plugins/security_solution/server/lib/detection_engine/prebuilt_rules/api/bootstrap_prebuilt_rules/bootstrap_prebuilt_rules_handler.ts # x-pack/plugins/security_solution/server/lib/detection_engine/prebuilt_rules/api/perform_rule_upgrade/perform_rule_upgrade_route.ts # x-pack/plugins/security_solution/server/lib/detection_engine/prebuilt_rules/api/review_rule_installation/review_rule_installation_handler.ts # x-pack/plugins/security_solution/server/lib/detection_engine/prebuilt_rules/api/review_rule_installation/review_rule_installation_route.ts # x-pack/plugins/security_solution/server/lib/detection_engine/prebuilt_rules/api/review_rule_upgrade/review_rule_upgrade_handler.ts # x-pack/plugins/security_solution/server/lib/detection_engine/prebuilt_rules/api/review_rule_upgrade/review_rule_upgrade_route.ts # x-pack/plugins/security_solution/server/utils/throttle_requests.test.ts # x-pack/plugins/security_solution/server/utils/throttle_requests.ts # x-pack/solutions/security/plugins/security_solution/public/detection_engine/rule_management/api/hooks/use_bootstrap_prebuilt_rules.ts # x-pack/solutions/security/plugins/security_solution/public/detection_engine/rule_management_ui/components/rules_table/upgrade_prebuilt_rules_table/translations.tsx # x-pack/solutions/security/plugins/security_solution/public/detection_engine/rule_management_ui/components/rules_table/upgrade_prebuilt_rules_table/upgrade_prebuilt_rules_table_context.tsx
maximpn
approved these changes
Feb 12, 2025
💚 Build Succeeded
Metrics [docs]Module Count
Async chunks
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Backport
This will backport the following commits from
mainto8.17:Questions ?
Please refer to the Backport tool documentation
\n\n### Questions ?\nPlease refer to the [Backport tool\ndocumentation](https://github.com/sqren/backport)\n\n\n\nCo-authored-by: Dmitrii Shevchenko "}},{"branch":"8.18","label":"v8.18.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"url":"https://github.com//pull/210640","number":210640,"state":"MERGED","mergeCommit":{"sha":"2bd85b19aa84575ec0745ba2ad24d4c7718d824f","message":"[8.18] [Security Solution] Added concurrency limits and request throttling to prebuilt rule routes (#209551) (#210640)\n\n# Backport\n\nThis will backport the following commits from `main` to `8.18`:\n- [[Security Solution] Added concurrency limits and request throttling\nto prebuilt rule routes\n(#209551)](https://github.com//pull/209551)\n\n\n\n### Questions ?\nPlease refer to the [Backport tool\ndocumentation](https://github.com/sqren/backport)\n\n\n\nCo-authored-by: Dmitrii Shevchenko "}},{"branch":"main","label":"v9.1.0","branchLabelMappingKey":"^v9.1.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com//pull/209551","number":209551,"mergeCommit":{"message":"[Security Solution] Added concurrency limits and request throttling to prebuilt rule routes (#209551)\n\n**Resolves: https://github.com//issues/208357**\n**Resolves: https://github.com//issues/208355**\n\n## Summary \n\nTo prevent possible OOM errors, we need to limit concurrent requests to\nprebuilt rule routes (see attached tickets for more details).\n\n- `installation/_perform` and `upgrade/_perform` endpoints\n- Concurrency is limited to one parallel call. If another call is made\nsimultaneously, the server responds with 429 Too Many Requests.\n- On the front end, all rule install and upgrade operations are retried\nin case of a 429 response. This ensures proper handling when a user\nclicks multiple times an update or install rule buttons\n\n- `prebuilt_rules/_bootstrap` endpoint\n- Install prebuilt rules and endpoint packages sequentially instead of\nin parallel to prevent from having them both downloaded into memory\nsimultaneously.\n- Added a 30-minute socket timeout to prevent the proxy from closing the\nconnection while rule installation is in progress.\n- Introduced a `throttleRequests` wrapper, ensuring the endpoint handler\nis called only once when multiple concurrent requests are received.\n- The first request triggers the handler, while subsequent requests wait\nfor the first one to complete and reuse its result.\n- This prevents costly prebuilt rule package installation from running\nin parallel.\n- Reusing the response ensures the frontend correctly invalidates cached\nprebuilt rule queries. Since concurrent frontend requests should receive\nthe same installed package information, responding with 421 and using\nthe retry logic as in cases above is not an option here because the\nsecond request would receive a package installation skipped response\nleading to no cache invalidation.\n\n- `installation/_review` and `upgrade/_review` endpoints\n- Concurrency is limited to one parallel call. If another call is made\nsimultaneously, the server responds with 429 Too Many Requests.\n- On the front end, all rule install and upgrade operations are retried\nin case of a 429 response. This ensures proper handling when a user\nclicks multiple times an update or install rule buttons","sha":"c5557f33213f699acd9bb656af9166b1449d18f9"}},{"branch":"8.x","label":"v8.19.0","branchLabelMappingKey":"^v8.19.0$","isSourceBranch":false,"url":"https://github.com//pull/210641","number":210641,"state":"MERGED","mergeCommit":{"sha":"8018c82f7d92b1d4798dea3c0a677bae740d4f8a","message":"[8.x] [Security Solution] Added concurrency limits and request throttling to prebuilt rule routes (#209551) (#210641)\n\n# Backport\n\nThis will backport the following commits from `main` to `8.x`:\n- [[Security Solution] Added concurrency limits and request throttling\nto prebuilt rule routes\n(#209551)](https://github.com//pull/209551)\n\n\n\n### Questions ?\nPlease refer to the [Backport tool\ndocumentation](https://github.com/sqren/backport)\n\n\n\nCo-authored-by: Dmitrii Shevchenko "}},{"branch":"8.17","label":"v8.17.3","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"}]}] BACKPORT-->