Skip to content

Commit edf1ec0

Browse files
[8.18] Rule gaps and manual rule runs (#6649)
* First draft * Formatting * Some deduping * Revisions * New images * image updates * Minor edits * em dash * Moved more content around * Tweak * Grammar fix * Missing space * Update docs/detections/rules-ui-monitor.asciidoc Co-authored-by: natasha-moore-elastic <137783811+natasha-moore-elastic@users.noreply.github.com> * Update docs/detections/rules-ui-monitor.asciidoc Co-authored-by: natasha-moore-elastic <137783811+natasha-moore-elastic@users.noreply.github.com> * Update docs/detections/rules-ui-monitor.asciidoc Co-authored-by: natasha-moore-elastic <137783811+natasha-moore-elastic@users.noreply.github.com> * Update docs/detections/rules-ui-monitor.asciidoc * Update docs/detections/rules-ui-manage.asciidoc * Update docs/detections/rules-ui-monitor.asciidoc * Feedback from technical review * Update docs/detections/rules-ui-monitor.asciidoc * Update docs/detections/rules-ui-manage.asciidoc * Kseniia's feedback * One more change * revert changes * uppercase * Table name --------- Co-authored-by: natasha-moore-elastic <137783811+natasha-moore-elastic@users.noreply.github.com>
1 parent 2f30f4c commit edf1ec0

File tree

5 files changed

+60
-28
lines changed

5 files changed

+60
-28
lines changed

docs/detections/images/gaps-table.png

248 KB
Loading
28.2 KB
Loading
56.5 KB
Loading

docs/detections/rules-ui-manage.asciidoc

Lines changed: 7 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -109,9 +109,7 @@ NOTE: When duplicating a rule with exceptions, you can choose to duplicate the r
109109
[[manually-run-rules]]
110110
=== Run rules manually
111111

112-
beta::[]
113-
114-
Manually run enabled rules for a specified period of time for testing purposes or additional rule coverage.
112+
Manually run enabled rules for a specified period of time to deliberately test them, provide additional rule coverage, or fill gaps in rule executions.
115113

116114
IMPORTANT: Before manually running rules, make sure you properly understand and plan for rule dependencies. Incorrect scheduling can lead to inconsistent rule results.
117115

@@ -121,18 +119,19 @@ IMPORTANT: Before manually running rules, make sure you properly understand and
121119
* Select all the rules you want to manually run, select the **Bulk actions** menu, then select **Manual run**.
122120
. Specify when the manual run starts and ends. The default selection is the current day starting three hours in the past. The rule will search for events during the selected time range.
123121
. Click **Run** to manually run the rule.
124-
+
125-
NOTE: Manual runs can produce multiple rule executions. This is determined by the manual run's time range and the rule's execution schedule.
126122

127-
The manual run's details are shown in the <<manual-runs-table,Manual runs>> table on the *Execution results* tab. Changes you make to the manual run or rule settings will display in the Manual runs table after the current run completes.
123+
The rule will run over the time range that you selected. Note that all <<rule-notifications,rule actions>> will also be activated, except for **Summary of alerts** actions that run at a custom frequency.
124+
125+
Go to the <<manual-runs-table>> on the **Execution results** tab to track the manual rule executions. If you manually ran the rule over a gap, you can also monitor the gap fill's progress from the <<gaps-table>>.
128126

129127
[NOTE]
130128
=====
131129
Be mindful of the following:
132130
133-
* Rule actions are not activated during manual runs.
131+
* Any changes that you make to the manual run or rule settings will display in the Manual runs table after the current run completes.
134132
* Except for threshold rules, duplicate alerts aren't created if you manually run a rule during a time range that was already covered by a scheduled run.
135-
* Manual runs are executed with low priority and limited concurrency, meaning they might take longer to complete. This can be especially apparent for rules requiring multiple executions.
133+
* Manually running a custom query rule with suppression may incorrectly inflate the number of suppressed alerts.
134+
136135
=====
137136

138137
[float]

docs/detections/rules-ui-monitor.asciidoc

Lines changed: 53 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -20,8 +20,7 @@ Refer to the <<troubleshoot-signals>> section below for strategies on adjusting
2020
[[rule-monitoring-tab]]
2121
=== Rule Monitoring tab
2222

23-
To view a summary of all rule executions, including the most recent failures and execution
24-
times, select the *Rule Monitoring* tab on the *Rules* page. To access the tab, find **Detection rules (SIEM)** in the navigation menu or look for “Detection rules (SIEM)” using the {kibana-ref}/introduction.html#kibana-navigation-search[global search field], then go to the *Rule Monitoring* tab.
23+
To view a summary of all rule executions (including the most recent failures, execution times, and gaps in rule executions), select the *Rule Monitoring* tab on the *Rules* page. To access the tab, find **Detection rules (SIEM)** in the navigation menu or look for “Detection rules (SIEM)” using the {kibana-ref}/introduction.html#kibana-navigation-search[global search field], then go to the *Rule Monitoring* tab.
2524

2625
[role="screenshot"]
2726
image::images/monitor-table.png[]
@@ -32,18 +31,28 @@ TIP: To sort the rules list, click any column header. To sort in descending orde
3231

3332
For detailed information on a rule, the alerts it generated, and associated errors, click on its name in the table. This also allows you to perform the same actions that are available on the <<rules-ui-management, **Installed Rules** tab>>, such as modifying or deleting rules, activating or deactivating rules, exporting or importing rules, and duplicating prebuilt rules.
3433

34+
For information about rule execution gaps (which are periods of time when a rule didn't run), use the panel above the table. The time filter on the left allows you to select a time range for viewing gap data. The **Total rules with gaps:** field tells you how many rules have unfilled or partially filled gaps within the selected time range. The **Only rules with gaps** filter on the right lets you only display rules with unfilled or partially filled gaps.
35+
36+
Within the table, the **Last Gap (if any)** column conveys how long the most recent gap for a rule lasted. The **Unfilled gaps duration** column shows whether a rule still has gaps and provides a total sum of the remaining unfilled or partially filled gaps. The total sum can change based on the time range that you select in the panel above the table. If a rule has no gaps, the columns display a dash (`––`).
37+
38+
TIP: For a detailed view of a rule's gaps, go to the **Execution results** tab and check the <<gaps-table>>.
39+
3540
[float]
3641
[[rule-execution-logs]]
37-
=== Execution results
42+
=== Execution results tab
3843

39-
Each detection rule execution is logged, including the execution type, the execution's success or failure, any warning or error messages, how long it took to search for data, create alerts, and complete. This can help you troubleshoot a particular rule if it isn't behaving as expected (for example, if it isn't creating alerts or takes a long time to run).
44+
From the **Execution results** tab, you can access the rule’s execution log, monitor and address gaps in a rule's execution schedule, and check manual runs for the rule. To find the tab, click the rule's name to open its details, then scroll down.
4045

41-
To access a rule's execution log, click the rule's name to open its details, then scroll down and select the **Execution results** tab. Within the Execution log table, you can click the arrow at the end of a row to expand a long warning or error message.
46+
[float]
47+
[[execution-results-tab]]
48+
==== Execution log table
49+
50+
Each detection rule execution is logged, including the execution type, the execution's success or failure, any warning or error messages, how long it took to search for data, create alerts, and complete. This can help you troubleshoot a particular rule if it isn't behaving as expected (for example, if it isn't creating alerts or takes a long time to run).
4251

4352
[role="screenshot"]
4453
image::images/rule-execution-logs.png[Execution log table on the rule execution results tab]
4554

46-
You can hover over each column heading to display a tooltip about that column's data. Click a column heading to sort the table by that column.
55+
You can hover over each column heading to display a tooltip about that column's data. Click a column heading to sort the table by that column. Within the Execution log table, you can click the arrow at the end of a row to expand a long warning or error message.
4756

4857
Use these controls to filter what's included in the logs table:
4958

@@ -65,32 +74,56 @@ Use these controls to filter what's included in the logs table:
6574
* The *Actions* column allows you to show alerts generated from a given rule execution. Click the filter icon (image:images/filter-icon.png[Filter icon,18,17]) to create a global search filter based on the rule execution's ID value. This replaces any previously applied filters, changes the global date and time range to 24 hours before and after the rule execution, and displays a confirmation notification. You can revert this action by clicking *Restore previous filters* in the notification.
6675

6776
[float]
68-
[[manual-runs-table]]
69-
==== Manual runs table
77+
[[gaps-table]]
78+
==== Gaps table
7079

7180
beta::[]
7281

73-
Each manual run can produce multiple rule executions, depending on the time range of the run and the rule's execution schedule. These details are shown in the Manual runs table.
82+
Gaps in rule executions are periods of time where a rule didn’t run. They can be caused by various disruptions, including system updates, rule failures, or simply turning off a rule. Addressing gaps is essential for maintaining consistent coverage and avoiding missed alerts.
7483

75-
To access the table, navigate to the detection rules page, click the rule's name to open its details, then scroll down and select the **Execution results** tab. Scroll down again to find the Manual runs table.
84+
TIP: Refer to the <<troubleshoot-gaps>> section for strategies for avoiding gaps.
7685

77-
To stop an active run, go to the appropriate row and click **Stop run** in the **Actions** column. Completed rule executions for each manual run are logged in the Execution log table.
86+
Use the information in the Gaps table to assess the scope and severity of rule execution gaps. To control what's shown in the table, you can filter the table by gap status, select a time range for viewing gap data, and sort multiple columns.
7887

7988
[role="screenshot"]
80-
image::images/manual-rule-run-table.png[Manual rule runs table on the rule execution results tab]
89+
image::images/gaps-table.png[Gaps table on the rule execution results tab]
8190

82-
The Manual runs table displays important details such as:
91+
The Gaps table has the following columns:
8392

84-
* The status of each manual run:
85-
** **Pending**: The rule is not yet running.
86-
** **Running**: The rule is executing during the time range you specified. Some rules, such as indicator match rules, can take longer to run.
87-
** **Error**: The rule's configuration is preventing it from running correctly. For example, the rule's conditions cannot be validated.
93+
* **Status**: The current state of the gap. It can be `Filled`, `Partially filled`, or `Unfilled`.
94+
* **Detected at**: The date and time the gap was first discovered.
95+
* **Manual fill tasks**: The status of the manual run that’s filling the gap. For more details about the manual run, refer to its entry in the <<manual-runs-table,Manual runs table>>.
96+
* **Event time covered**: How much progress the manual run has made filling the gap.
97+
+
98+
NOTE: If you stop a manual run that hasn't finished filling a gap, the gap’s status will be set to `Partially filled`. To fill the remaining gap, you can select the **Fill remaining gap** action or <<manually-run-rules,manually run>> the rule over the gap's time frame.
99+
+
100+
* **Range**: When the gap started and ended.
101+
* **Total gap duration**: How long the gap lasted.
102+
* **Actions**: The actions that you can take for the gap. They can be **Fill gap** (starts a manual run to fill the gap) or **Fill remaining gap** (starts a manual run that fills the leftover portion of the gap).
88103

89-
* When a manual run started and the time in which it will run
104+
[float]
105+
[[manual-runs-table]]
106+
==== Manual runs table
107+
108+
You can <<manually-run-rules,manually run>> enabled rules for a specified period of time to deliberately test them, provide additional rule coverage, or fill gaps in rule executions. Each manual run can produce multiple rule executions, depending on the time range of the run and the rule's execution schedule.
109+
110+
NOTE: Manual runs are executed with low priority and limited concurrency, meaning they might take longer to complete. This can be especially apparent for rules requiring multiple executions.
90111

91-
* The number of rule executions that are failing, pending, running, and completed for a manual run
112+
The Manual runs table tracks manual rule executions and provides important details such as:
113+
114+
* The total number of rule executions that the manual run will produce and how many are failing, pending, running, and completed.
115+
* When the manual run started and the time range that it will cover.
116+
+
117+
NOTE: To stop an active run, go to the appropriate row in the table and click **Stop run** in the **Actions** column. Completed rule executions for each manual run are logged in the Execution log table.
118+
+
119+
* The status of each manual run:
120+
** `Pending`: The rule is not yet running.
121+
** `Running`: The rule is executing during the time range you specified. Some rule types, such as indicator match rules, can take longer to run.
122+
** `Error`: The rule's configuration is preventing it from running correctly. For example, the rule's conditions cannot be validated.
123+
124+
[role="screenshot"]
125+
image::images/manual-rule-run-table.png[Manual rule runs table on the rule execution results tab]
92126

93-
* The total number of rule executions that are occurring for a manual run
94127

95128
[float]
96129
[[troubleshoot-signals]]

0 commit comments

Comments
 (0)