-
Notifications
You must be signed in to change notification settings - Fork 231
Event extraction rules #6118
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Event extraction rules #6118
Changes from all commits
Commits
Show all changes
22 commits
Select commit
Hold shift + click to select a range
edaa50d
Event Extraction Rules
JV0812 3d8328b
delete duplicate files
JV0812 d2ddc68
added API docs
JV0812 ea7505e
minor fix
JV0812 6666f0f
added beta tag
JV0812 4f860c7
Merge branch 'main' into event-extraction-rules
JV0812 3a94efc
Update docs/api/event-extraction-rules.md
JV0812 61b9100
Update docs/manage/event-extraction-rules.md
JV0812 4e2dc10
Update docs/manage/event-extraction-rules.md
JV0812 88ca786
Update docs/manage/event-extraction-rules.md
JV0812 29bdae6
Update docs/manage/event-extraction-rules.md
JV0812 8aa9f67
Update docs/manage/event-extraction-rules.md
JV0812 bb3396c
Update docs/manage/event-extraction-rules.md
JV0812 6c10ad7
Update docs/manage/event-extraction-rules.md
JV0812 6df46d5
Update docs/manage/event-extraction-rules.md
JV0812 fe4d805
Update docs/manage/event-extraction-rules.md
JV0812 ed8eb8f
Update docs/manage/event-extraction-rules.md
JV0812 5080c09
Update docs/manage/event-extraction-rules.md
JV0812 323f92d
Update docs/manage/event-extraction-rules.md
JV0812 294d787
minor fix
JV0812 486212c
added limitations
JV0812 a40a913
Merge branch 'main' into event-extraction-rules
JV0812 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,44 @@ | ||
| --- | ||
| id: event-extraction-rules | ||
| title: Event Analytics Management APIs | ||
| sidebar_label: Event Extraction Rules | ||
| description: Use Event Analytics Management APIs to configure Event Extraction Rules. | ||
| --- | ||
|
|
||
| <head> | ||
| <meta name="robots" content="noindex" /> | ||
| </head> | ||
|
|
||
| <p><a href={useBaseUrl('docs/beta')}><span className="beta">Beta</span></a></p> | ||
|
|
||
| import useBaseUrl from '@docusaurus/useBaseUrl'; | ||
| import ApiIntro from '../reuse/api-intro.md'; | ||
| import ApiRoles from '../reuse/api-roles.md'; | ||
|
|
||
| <img src={useBaseUrl('img/icons/operations/rules.png')} alt="Thumbnail icon" width="50"/> | ||
|
|
||
| The Event Analytics Management API allows you to configure event extraction rules from HTTP endpoints. For more information, refer to [Event Extraction Rules](/docs/manage/event-extraction-rules). | ||
|
|
||
| ## Documentation | ||
|
|
||
| <ApiIntro/> | ||
|
|
||
| | Deployment | Documentation URL | | ||
| |:-- |:-- | | ||
| | AU | https://api.au.sumologic.com/docs/#tag/eventAnalytics | | ||
| | CA | https://api.ca.sumologic.com/docs/#tag/eventAnalytics | | ||
| | DE | https://api.de.sumologic.com/docs/#tag/eventAnalytics | | ||
| | EU | https://api.eu.sumologic.com/docs/#tag/eventAnalytics | | ||
| | FED | https://api.fed.sumologic.com/docs/#tag/eventAnalytics | | ||
| | JP | https://api.jp.sumologic.com/docs/#tag/eventAnalytics | | ||
| | KR | https://api.kr.sumologic.com/docs/#tag/eventAnalytics | | ||
| | US1 | https://api.sumologic.com/docs/#tag/eventAnalytics | | ||
| | US2 | https://api.us2.sumologic.com/docs/#tag/eventAnalytics | | ||
|
|
||
| ## Required role capabilities | ||
|
|
||
| <ApiRoles/> | ||
|
|
||
| * [Data Management](/docs/manage/users-roles/roles/role-capabilities/#data-management) | ||
| * Manage Event Extraction Rules | ||
| * View Event Extraction Rules | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,101 @@ | ||
| --- | ||
| id: event-extraction-rules | ||
| title: Event Extraction Rules | ||
| description: Learn how to use Sumi Logic event extraction rules. | ||
| --- | ||
|
|
||
| <head> | ||
| <meta name="robots" content="noindex" /> | ||
| </head> | ||
|
|
||
| <p><a href={useBaseUrl('docs/beta')}><span className="beta">Beta</span></a></p> | ||
|
|
||
| import useBaseUrl from '@docusaurus/useBaseUrl'; | ||
|
|
||
| Event Extraction Rules enables you to automatically extract, correlate, and enrich events directly from log data, making them available for event querying and analysis. By reducing noise and highlighting meaningful events, this capability accelerates troubleshooting and root cause analysis with minimal manual effort. You can precisely control how events are classified by configuring event type and priority, ensuring that the most impactful events are surfaced and clearly represented within log searches for faster, more informed insights. | ||
|
|
||
| ## Create an Event Extraction Rule | ||
|
|
||
| You can create an event extraction rule of your own from scratch by following the instructions below. | ||
|
|
||
| :::note | ||
| You need the `Manage Event Extraction Rules` [role capability](/docs/manage/users-roles/roles/role-capabilities/) to create an event extraction rule. | ||
| ::: | ||
|
|
||
| 1. [**New UI**](/docs/get-started/sumo-logic-ui). To access the Event Extraction Rules page, in the main Sumo Logic menu select **Data Management**, and then under **Logs** select **Event Extraction Rules**. You can also click the **Go To...** menu at the top of the screen and select **Event Extraction Rules**. <br/>[**Classic UI**](/docs/get-started/sumo-logic-ui-classic). In the main Sumo Logic menu, select **Manage Data > Logs > Event Extraction Rules**. | ||
| 1. Click the **+ Add Event Extraction Rule** button on the top right of the table.<br/><img src={useBaseUrl('img/manage/event-extraction-rule/event-extraction-rule.png')} alt="event-extraction-rule" style={{border: '1px solid gray'}} width="800"/> | ||
| 1. Enter the following options in the **Create New Event Extraction Rule** page:<br/><img src={useBaseUrl('img/manage/event-extraction-rule/create-event-extraction-rule.png')} alt="create-event-extraction-rule" style={{border: '1px solid gray'}} width="500"/> | ||
| 1. **Log Query**. Enter the log search query for the event to filter the logs. Optimize queries by limiting log volume, parsing and extracting only required fields, and using the fields operator to return only the data needed for event correlation and visualization. | ||
| 1. **Preview**. Click the **Preview Log Messages** button to preview the log messages for the query entered, | ||
| 1. **Event Configuration**: | ||
| 1. **Event Name**. A unique name for the event. | ||
| 1. **Event Description (optional)**. An optional text field to provide additional context about the event—such as its purpose, expected behavior, or when it should occur. Helps to understand the significance of the event. | ||
| 1. **Event Source**. Specify where the event originates from. This helps you to categorize and track events across different data sources. | ||
| 1. **Event Priority**. You can select Low, Medium, or High depending on the importance of the event. | ||
| 1. **Event Type**. Defines the category of the event - Deployment, Feature Flag Change, Infrastructure Change, or Configuration Change. This helps you in filtering, grouping, and analyzing events based on their nature. | ||
| 1. **Timeline Preview**. This previews how event markers will display in the histogram timeline on the logs page when this rule is active. This marker also displays the event type, source, and priority details. | ||
| 1. **Advanced Settings (optional)**. Use this section if you want to compare values from parsed event fields with fields in incoming log messages. When the selected values match, the system displays a visual marker to highlight the match. | ||
| 1. **Event Record Field**. Choose the field from the event record that you want to compare against incoming log data. | ||
| 1. **Match Type**. **Exact Match** is selected by default. This option creates a marker when the value in the incoming log exactly matches the value specified in the event record field. | ||
| 1. **Log Message Field**. Select the field from the incoming log message that should be compared with the chosen event record field. | ||
| 1. **Rule Details**. Enter the rule name and rule description (optional) of your choice that makes it easy to identify the rule. | ||
|
|
||
| :::info | ||
| When an Event Extraction Rule is created, events only from previous seven days are automatically backfilled into the event index. | ||
| ::: | ||
|
|
||
| ## Search for user data events | ||
|
|
||
| Searching the user data events is the same as running a normal search against your ingested data. You specify the `_index` metadata field with `sumologic_userdata_events`. | ||
|
|
||
| For example, to search for system events: | ||
|
|
||
| 1. In the Search page, enter the following: `_index=sumologic_userdata_events`. | ||
| :::info | ||
| Make sure to enter the query exactly as shown. Changing any part of the query renders it ineffective. | ||
| ::: | ||
| 1. Choose the time range for the events that you'd like to review. | ||
| 1. Click **Start** to run the search. | ||
|
|
||
| :::note | ||
| Add the `_eventExtractionRuleID` field to view the event ID against each log message. | ||
| ::: | ||
|
|
||
| ## Edit a rule | ||
|
|
||
| To edit the existing event extraction rule, follow the below steps: | ||
|
|
||
| 1. [**New UI**](/docs/get-started/sumo-logic-ui). To access the Event Extraction Rules page, in the main Sumo Logic menu select **Data Management**, and then under **Logs** select **Event Extraction Rules**. You can also click the **Go To...** menu at the top of the screen and select **Event Extraction Rules**. <br/>[**Classic UI**](/docs/get-started/sumo-logic-ui-classic). In the main Sumo Logic menu, select **Manage Data > Logs > Event Extraction Rules**. | ||
| 1. Navigate to the respective event rule which you wish to edit. | ||
| 1. On the left pane, click **Edit** button.<br/><img src={useBaseUrl('img/manage/event-extraction-rule/edit-event-extraction-rules.png')} alt="edit-event-extraction-rule" style={{border: '1px solid gray'}} width="400"/> | ||
| 1. In the event extraction rule editing pane, perform the required editing and click **Submit** to save the changes. | ||
|
|
||
| ## Duplicate a rule | ||
|
|
||
| To duplicate the existing event extraction rule, follow the below steps: | ||
|
|
||
| 1. [**New UI**](/docs/get-started/sumo-logic-ui). To access the Event Extraction Rules page, in the main Sumo Logic menu select **Data Management**, and then under **Logs** select **Event Extraction Rules**. You can also click the **Go To...** menu at the top of the screen and select **Event Extraction Rules**. <br/>[**Classic UI**](/docs/get-started/sumo-logic-ui-classic). In the main Sumo Logic menu, select **Manage Data > Logs > Event Extraction Rules**. | ||
| 1. Navigate to the respective event rule which you wish to duplicate. | ||
| 1. On the left pane, click **Duplicate** button.<br/><img src={useBaseUrl('img/manage/event-extraction-rule/duplicate-event-extraction-rules.png')} alt="duplicate-event-extraction-rule" style={{border: '1px solid gray'}} width="400"/> | ||
| 1. In the event extraction rule editing pane, perform the required editing and click **Submit** to duplicate the changes. | ||
|
|
||
| ## Delete a rule | ||
|
|
||
| To delete the existing event extraction rule, follow the below steps: | ||
|
|
||
| 1. [**New UI**](/docs/get-started/sumo-logic-ui). To access the Event Extraction Rules page, in the main Sumo Logic menu select **Data Management**, and then under **Logs** select **Event Extraction Rules**. You can also click the **Go To...** menu at the top of the screen and select **Event Extraction Rules**. <br/>[**Classic UI**](/docs/get-started/sumo-logic-ui-classic). In the main Sumo Logic menu, select **Manage Data > Logs > Event Extraction Rules**. | ||
| 1. Navigate to the respective event rule which you wish to edit. | ||
| 1. On the left pane, click **Delete** button.<br/><img src={useBaseUrl('img/manage/event-extraction-rule/delete-event-extraction-rules.png')} alt="delete-event-extraction-rule" style={{border: '1px solid gray'}} width="400"/> | ||
| 1. In the **Delete [rule name] item** pop-up, click on **Delete**.<br/><img src={useBaseUrl('img/manage/event-extraction-rule/delete-confirm-event-extraction-rule.png')} alt="delete-confirm-event-extraction-rule" style={{border: '1px solid gray'}} width="400"/> | ||
|
|
||
| ## Limitations | ||
|
|
||
| - You can create a maximum of 50 event extraction rules. | ||
| - For any query, a maximum of five event markers will be displayed in the histogram, regardless of the selected time range. | ||
|
|
||
| ## Operational considerations | ||
|
|
||
| - To restrict user access to extracted events, you can deny access to the `sumologic_userdata_events` index for specific roles. Ensure that you have the **[Usage Management](/docs/manage/users-roles/roles/role-capabilities/#user-management)** capability enabled, as it is required to configure index-level access restrictions. | ||
| - An Event Extraction Rule can generate a maximum of 1,000 events per hour. If this limit is exceeded, the rule may be automatically disabled. To re-enable the rule, review and refine the rule query to reduce the event volume. | ||
| - Audit logs for all create, read, update, and delete (CRUD) actions performed on Event Extraction Rules are available in the `_index=sumologic_audit_events ` and `_sourcecategory=eventExtractionRule`. | ||
| - System-generated events can be viewed by querying the `_index=sumologic_system_events` and `_sourcecategory=eventExtractionRule`, allowing you to identify errors and take appropriate corrective actions. |
Binary file added
BIN
+156 KB
static/img/manage/event-extraction-rule/create-event-extraction-rule.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added
BIN
+39.7 KB
static/img/manage/event-extraction-rule/delete-confirm-event-extraction-rule.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added
BIN
+92.8 KB
static/img/manage/event-extraction-rule/delete-event-extraction-rules.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added
BIN
+93 KB
static/img/manage/event-extraction-rule/duplicate-event-extraction-rules.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added
BIN
+93 KB
static/img/manage/event-extraction-rule/edit-event-extraction-rules.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.