Skip to content

Commit 1afe89b

Browse files
benskelkerbenskelker
authored
[Docs]Detections and Alerts UI (#73)
* dtections-ui-overview * initial overview draft * typo * restructuring for dedicated alerts section * rewording * exceptions from alerts * adds new rule options * adds new action text placeholder * restructer * structure, exceptions and building-blocks * minor edits * adds exceptions * exceptions cont * exceptions correction * more stuff * proofing and whatnot * terminology * nested exception conditions * typo * typo - thanks Nate * corrections - round 1 * add nested conditions example * typo * editing * more proofing * updates ex example * adds promoted endpoint events * typo * corrections after review * corrections
1 parent 2051071 commit 1afe89b

23 files changed

+787
-306
lines changed

‎docs/siem/detections/alerts-ui-manage.asciidoc‎

Lines changed: 106 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,106 @@
1+
[[alerts-ui-manage]]
2+
[role="xpack"]
3+
== Managing detection alerts
4+
5+
The Detections page displays all <<detection-alert-def, detection alerts>>.
6+
From the Alerts table, you can change an alert's status, and start
7+
investigating and analyzing alerts in Timeline.
8+
9+
TIP: From Timeline, you can <<cases-ui-open, create cases>> to track issues and
10+
share information with colleagues.
11+
12+
To view detection alerts created by a specific rule, you can:
13+
14+
* Filter for a specific rule in the KQL bar (for example,
15+
`signal.rule.name :"SSH (Secure Shell) from the Internet"`).
16+
* View detection alerts in the *Rule details* page (click
17+
*Manage detection rules* -> rule name in the *All rules* table).
18+
19+
NOTE: KQL autocomplete for `.siem-signals-*` indices is available on the
20+
*Detections* and *Rule details* pages, and in Timeline when either `All` or
21+
`Detection alerts` is selected.
22+
23+
TIP: Use the icons in the upper left corner of the Alerts table to customize
24+
displayed columns and row renderers, and view the table in full screen mode.
25+
26+
[float]
27+
[[detection-alert-status]]
28+
=== Change alert statuses
29+
30+
You can set an alert's status to indicate whether it needs to be investigated
31+
(`Open`), is under active investigation (`In progress`), or resolved
32+
(`Closed`). By default, the Alerts table displays open alerts. To view alerts
33+
with other statuses, click *In progress* or *Closed*.
34+
35+
To change alert statuses, either:
36+
37+
* In the alert's row, click the *more options* icon, and then select the
38+
required status (*Mark in progress*, *Close alert*, or *Open alert*).
39+
* In the Alerts table, select all the alerts you want to change, and then select
40+
*Take action* -> *Close selected*, *Open selected*, or *Mark in progress*.
41+
42+
[float]
43+
[[signals-to-timelines]]
44+
=== Send alerts to Timeline
45+
46+
To view an alert in Timeline, click the *Investigate in timeline* icon.
47+
48+
TIP: When you send an alert generated by a
49+
<<rules-ui-create, threshold rule>> to Timeline, all matching events are
50+
listed in the Timeline, even ones that did not reach the threshold value. For
51+
example, if you have an alert generated by a threshold rule that detects 10
52+
failed login attempts, when you send that alert to Timeline all failed login
53+
attempts detected by the rule are listed.
54+
55+
If the rule that generated the alert uses a Timeline template, when you
56+
investigate the alert in Timeline, the dropzone query values defined in the
57+
template are replaced with their corresponding alert values.
58+
59+
// * `host.name`
60+
// * `host.hostname`
61+
// * `host.domain`
62+
// * `host.id`
63+
// * `host.ip`
64+
// * `client.ip`
65+
// * `destination.ip`
66+
// * `server.ip`
67+
// * `source.ip`
68+
// * `network.community_id`
69+
// * `user.name`
70+
// * `process.name`
71+
72+
*Example*
73+
74+
This Timeline template uses the `host.name: "{host.name}"` dropzone filter in
75+
the rule. When alerts generated by the rule are investigated in Timeline, the
76+
`{host.name}` value is replaced with the alert's `host.name` value. If the
77+
alerts's `host.name` value is `Windows-ArsenalFC`, the Timeline dropzone query
78+
is `host.name: "Windows-ArsenalFC"`.
79+
80+
NOTE: See <<timelines-ui>> for information on creating Timelines and Timeline
81+
templates. For information on how to add Timeline templates to rules, see
82+
<<rules-ui-create>>.
83+
84+
[float]
85+
[[add-exception-from-alerts]]
86+
=== Add rule exceptions
87+
88+
You can add exceptions to the rule that generated the alert directly from the
89+
Alerts table. Exceptions prevent a rule from generating alerts even when its
90+
criteria are met.
91+
92+
To add an exception, click the actions icon (three dots) and then select
93+
_Add exception_.
94+
95+
For information about exceptions and how to use them, see
96+
<<detections-ui-exceptions>>.
97+
98+
[float]
99+
[[alerts-to-resolver]]
100+
=== Visually analyze process relationships.
101+
102+
For process events received from the Elastic Endpoint agent, you can open a
103+
visual mapping of the relationships and hierarchy connecting related processes.
104+
105+
To visualize process relationships, click the *Analyze event* icon. For more
106+
information, see Ben xref.

‎docs/siem/detections/api/rules-api-create.asciidoc‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -377,7 +377,7 @@ used as an identifier across systems
377377
* `{{context.rule.saved_id}}`: Saved search ID
378378
* `{{context.rule.severity}}`: Rule severity
379379
* `{{context.rule.threat}}`: Rule threat framework
380-
* `{{context.rule.threshold}}`: Rule threshold (threshold rules only)
380+
* `{{context.rule.threshold}}`: Rule threshold values (threshold rules only)
381381
* `{{context.rule.timeline_id}}`: Associated timeline ID
382382
* `{{context.rule.timeline_title}}`: Associated timeline name
383383
* `{{context.rule.type}}`: Rule type

‎docs/siem/detections/building-block-rule.asciidoc‎

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,31 @@
1+
[[building-block-rule]]
2+
[role="xpack"]
3+
== About building-block rules
4+
5+
Create building-block rules when you do not want to see their generated alerts
6+
in the UI. This is useful when you want:
7+
8+
* A record of low-risk alerts without producing noise in the Alerts table.
9+
* Rules that execute on the alert indices (`.siem-signals-<kibana space>-*`).
10+
You can then use building-block rules to create hidden alerts that act as a
11+
basis for an 'ordinary' rule to generate visible alerts.
12+
13+
[float]
14+
=== Set up rules that run on alert indices
15+
16+
To create a rule that searches alert indices, in the *Index patterns* field,
17+
add the index pattern for alert indices:
18+
19+
[role="screenshot"]
20+
image::images/alert-indices-ui.png[]
21+
22+
[float]
23+
24+
=== View building-block alerts in the UI
25+
26+
. Go to *Security* -> *Detections*
27+
. In the Alert table, select _Additional filters_ ->
28+
_Include building-block alerts_.
29+
30+
NOTE: On a building-block Rule details page, the rule's alerts are displayed (by
31+
default, _Include building-block alerts_ is selected).

‎docs/siem/detections/detection-engine-intro.asciidoc‎

Lines changed: 78 additions & 105 deletions
Original file line numberDiff line numberDiff line change
@@ -1,26 +1,60 @@
11
[[detection-engine-overview]]
22
[role="xpack"]
33

4-
= Detections (beta)
4+
= Detections and Alerts (beta)
55

66
beta[]
77

8-
The SIEM Detections feature automatically searches for threats and creates
9-
signals when they are detected. Signal detection rules define the conditions
10-
for creating signals. Additionally, you can use the {kib}
11-
{kibana-ref}/alerting-getting-started.html[Alerting and Actions]
12-
framework to send notifications via other systems, such as email and Slack,
13-
when signals are generated.
8+
Use the Detections feature to create and manage rules, and view the alerts
9+
these rules create. Rules periodically search indices (such as `endgame-*` and
10+
`filebeat-*`) for suspicious source events, and create alerts when a rule's
11+
conditions are met. When an alert is created, its status is `Open`. To help
12+
track investigations, an alert's status can be set as `Open`, `In progress`, or
13+
`Closed` (see <<detection-alert-status>>).
1414

15-
NOTE: To use {kib} Alerting for signal notifications, you need the
15+
[role="screenshot"]
16+
image::images/detections-ui.png[]
17+
18+
In addition to creating <<rules-ui-create, your own rules>>, enable
19+
<<load-prebuilt-rules, Elastic prebuilt rules>> to immediately start detecting
20+
suspicious activity. For detailed information on all the prebuilt rules, see the
21+
<<prebuilt-rules>> section. Once the prebuilt rules are loaded and
22+
running, <<tuning-detection-signals>> and <<detections-ui-exceptions>> explain
23+
how to modify the rules to reduce false positives and get a better set of
24+
actionable alerts. You can also use exceptions and value lists when creating or
25+
modifying your own rules.
26+
27+
There are two special prebuilt rules you need to know about:
28+
29+
* <<elastic-endpoint-prebuilt-rule, *Elastic Endpoint Security*>>:
30+
Automatically creates an alert from all incoming Elastic Endpoint alerts. To
31+
receive Elastic Endpoint alerts, you must install the Endpoint agent on your
32+
hosts (BEN: see xref).
33+
+
34+
When this rule is enabled, the following Endpoint events are displayed as
35+
detection alerts:
36+
+
37+
** Malware Prevention Alert
38+
** Malware Detection Alert
39+
+
40+
NOTE: When you load the prebuilt rules, this is the only rule that is enabled
41+
by default.
42+
43+
* <<external-alerts, *External Alerts*>>: Automatically creates an alert for
44+
all incoming third-party system alerts (for example, Suricata alerts).
45+
46+
If you want to receive notifications via external systems, such as Slack or
47+
email, when alerts are created, use the {kib}
48+
{kibana-ref}/alerting-getting-started.html[Alerting and Actions] framework.
49+
50+
NOTE: To use {kib} Alerting for detection alert notifications, you need the
1651
https://www.elastic.co/subscriptions[appropriate license].
1752

18-
The {siem-app} comes with <<prebuilt-rules, prebuilt rules>> that search for
19-
suspicious activity on your network and hosts. For information on how to
20-
optimize the prebuilt rules, see <<tuning-detection-signals>>. You can also
21-
<<rules-ui-create, create your own rules>>.
53+
After rules have started running, you can monitor their executions to verify
54+
they are functioning correctly, as well as view, manage, and troubleshoot
55+
alerts (see <<alerts-ui-manage>> and <<alerts-ui-monitor>>).
2256

23-
You can manage detection rules and signals via the UI or the
57+
You can create and manage rules and alerts via the UI or the
2458
<<rule-api-overview, Detections API>>.
2559

2660
[IMPORTANT]
@@ -29,106 +63,45 @@ To make sure you can access Detections and manage rules, see
2963
<<detections-permissions>>.
3064
==============
3165

32-
[role="screenshot"]
33-
image::detections-ui.png[]
34-
3566
[float]
3667
[[det-engine-terminology]]
3768
== Terminology
3869

39-
Rules::
40-
Perform background tasks to detect suspicious activity. There are two types of
41-
rules:
42-
43-
* Query-based rules search indices for documents that match their queries at
44-
scheduled intervals. When a document matches, a signal is created.
45-
* {ml-cap} rules create a signal when a {ml} job discovers an anomaly score
46-
above a rule's defined threshold.
47-
48-
Signals::
49-
Always refer to {siem-soln} produced detections. Signals are never received
50-
from external systems. When a rule's conditions are met, the {siem-app}
51-
writes one or more signals to an Elasticsearch `signals` index.
70+
Actions::
71+
Sends notifications via other systems when a detection alert is created, such
72+
as email, Slack, PagerDuty, and Webhook.
73+
74+
[[detection-alert-def]]
75+
Detection alerts::
76+
{es-sec} produced alerts. Detection alerts are never received from external
77+
systems. When a rule's conditions are met, {es-sec} writes a detection alert to
78+
an Elasticsearch `.signals` index.
5279
+
5380
[NOTE]
5481
==============
55-
Signal indices are created for each {kib} space. The naming convention is:
56-
`.siem-signals-<space name>`. For the default space, the signals index is named
57-
`.siem-signals-default`.
82+
Detection alert indices are created for each {kib} space. The naming convention
83+
is: `.siem-signals-<space name>`. For the default space, the alerts index is
84+
named `.siem-signals-default`.
5885
==============
5986

60-
Alerts and events::
61-
Always refer to data the {siem-app} receives from external systems, such as
62-
Elastic Endpoint and Suricata.
87+
Detection rules::
88+
Background tasks that run periodically and produce alerts when suspicious
89+
activity is detected.
6390

64-
Actions::
65-
Used to send notifications via other systems when a signal is created, such as
66-
email, Slack, PagerDuty, and Webhook.
67-
68-
[float]
69-
== Signals and external alerts
91+
Endpoint exceptions::
92+
<<term-exceptions, Exceptions>> added to both rules and Endpoint agents on
93+
hosts. Endpoint exceptions can only be added when:
7094

71-
The Detections page displays all signals and alerts. To view signals created
72-
by a rule, you can:
95+
* Endpoint agents are installed on the hosts.
96+
* The Elastic Endpoint Security rule is activated.
7397

74-
* Filter for a specific rule in the KQL bar (for example,
75-
`signal.rule.name :"SSH (Secure Shell) from the Internet"`).
76-
* View signals in the *Rule details* page (click
77-
*Manage signal detection rules* -> rule name in the *All rules* table).
98+
[[term-exceptions]]
99+
Exceptions::
100+
Added to rules to prevent specific source event field values from generating
101+
alerts.
78102

79-
NOTE: KQL autocomplete for `.siem-signals-*` indices is available on the
80-
*Detections* and *Rule details* pages, and in Timeline when either `All events`
81-
or `Signal events` is selected.
82-
83-
To view alerts from external data shippers, click *External alerts*.
84-
85-
[float]
86-
=== Open and close signals
87-
88-
You can close signals to indicate they do not need any further investigation.
89-
By default, the All signals table displays open signals. To view closed
90-
signals, click *Closed signals*.
91-
92-
To open and close signals, either:
93-
94-
* Click the *Close/Open signal* icon.
95-
* Select the signals you want to open or close and then click
96-
*Close/Open selected*.
97-
98-
[float]
99-
[[signals-to-timelines]]
100-
=== Investigate signals in Timeline
101-
102-
To investigate a signal in Timeline, click the *Investigate in timeline*
103-
icon.
104-
105-
If the rule that generated the signal uses a timeline template, when you
106-
investigate the signal in Timeline, the following dropzone query values are
107-
replaced with their corresponding signal values:
108-
109-
* `host.name`
110-
* `host.hostname`
111-
* `host.domain`
112-
* `host.id`
113-
* `host.ip`
114-
* `client.ip`
115-
* `destination.ip`
116-
* `server.ip`
117-
* `source.ip`
118-
* `network.community_id`
119-
* `user.name`
120-
* `process.name`
121-
122-
*Example*
123-
124-
The timeline template used in the rule has this dropzone query:
125-
`host.name: "Linux-LiverpoolFC"`. When signals generated by the rule are
126-
investigated in Timeline, the `host.name` value is replaced with the signal's
127-
`host.name` value. If the signal's `host.name` value is `Windows-ArsenalFC`,
128-
the timeline dropzone query is `host.name: "Windows-ArsenalFC"`.
129-
130-
NOTE: For information on how to add timeline templates to rules, see
131-
<<create-rule-ui>>.
103+
External alerts::
104+
Alerts {es-sec} receives from external systems, such as Suricata.
132105

133106
[float]
134107
[[detections-permissions]]
@@ -150,14 +123,14 @@ of at least 32 characters. For example:
150123

151124
For *all* deployments (on-premises and hosted):
152125

153-
* To view signals and detection rules, you must have at least:
126+
* To view detection rules and alerts, you must have at least:
154127
** `read` permissions for the `.siem-signals-<space name>` index, where
155128
`<space name>` is the name of the {kib} space you are using to view Detections
156129
(see {ref}/security-privileges.html#privileges-list-indices[Indices privileges]).
157-
** {kib} space `Read` privileges for the `SIEM` feature (see
130+
** {kib} space `Read` privileges for the `Security` feature (see
158131
{kibana-ref}/xpack-spaces.html#spaces-control-user-access[Feature access based on user privileges]).
159132
* To create and modify detection rules, you must have:
160-
** {kib} space `All` privileges for the `SIEM` feature (see
133+
** {kib} space `All` privileges for the `Security` feature (see
161134
{kibana-ref}/xpack-spaces.html#spaces-control-user-access[Feature access based on user privileges]).
162135
** Write permissions for the `.siem-signals-<space name>` index, such as
163136
`create` `create_doc`, `write`, `index`, and `all`
@@ -173,11 +146,11 @@ when you try to open the *Detections* page.
173146
*`Let’s set up your detection engine`*
174147

175148
If you see this message, a user with these privileges must visit the
176-
*Detections* page before you can view signals and rules:
149+
*Detections* page before you can view detection rules and alerts:
177150

178151
* The `manage` cluster privilege (see {ref}/security-privileges.html[{es} security privileges]).
179152
* The `create_index` index privilege (see {ref}/security-privileges.html[{es} security privileges]).
180-
* {kib} space `All` privileges for the `SIEM` feature (see
153+
* {kib} space `All` privileges for the `Security` feature (see
181154
{kibana-ref}/xpack-spaces.html#spaces-control-user-access[Feature access based on user privileges]).
182155

183156
NOTE: For *on-premises* {stack} deployments only, this message may be displayed

0 commit comments

Comments
 (0)