Skip to content

Revises event.created and expands with event.start and event.end #80

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 2 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -174,6 +174,8 @@ The event fields are used for context information about the data itself.
| <a name="event.version"></a>event.version | The version field contains the version an event for ECS adheres to.<br/>This field should be provided as part of each event to make it possible to detect to which ECS version an event belongs.<br/>event.version is a required field and must exist in all events. It describes which ECS version the event adheres to.<br/>The current version is 0.1.0. | keyword | | `0.1.0` |
| <a name="event.duration"></a>event.duration | Duration of the event in nanoseconds. | long | | |
| <a name="event.created"></a>event.created | event.created contains the date when the event was created.<br/>This timestamp is distinct from @timestamp in that @timestamp contains the processed timestamp. For logs these two timestamps can be different as the timestamp in the log line and when the event is read for example by Filebeat are not identical. `@timestamp` must contain the timestamp extracted from the log line, event.created when the log line is read. The same could apply to package capturing where @timestamp contains the timestamp extracted from the network package and event.created when the event was created.<br/>In case the two timestamps are identical, @timestamp should be used. | date | | |
| <a name="event.start"></a>event.start | event.start contains the date when the event started.<br/>This timestamp is distinct from @timestamp in that @timestamp contains the processed timestamp. For logs these two timestamps can be different as the timestamp in the log line and when the event is read for example by Filebeat are not identical. `@timestamp` must contain the timestamp extracted from the log line, event.start when the log line is read. The same could apply to package capturing where @timestamp contains the timestamp extracted from the network package and event.created when the event was created.<br/>In case the two timestamps are identical, @timestamp should be used. | date | | |
| <a name="event.end"></a>event.end | event.end contains the date when the event ended. | date | | |
| <a name="event.risk_score"></a>event.risk_score | Risk score value of the event. | float | | |


Expand Down
2 changes: 2 additions & 0 deletions schema.csv
Original file line number Diff line number Diff line change
Expand Up @@ -42,12 +42,14 @@ event.category,keyword,0,metrics
event.created,date,0,
event.dataset,keyword,0,stats
event.duration,long,0,
event.end,date,0,
event.hash,keyword,1,123456789012345678901234567890ABCD
event.id,keyword,1,8a4f500d
event.module,keyword,0,mysql
event.raw,keyword,1,Sep 19 08:26:10 host CEF:0&#124;Security&#124; threatmanager&#124;1.0&#124;100&#124; worm successfully stopped&#124;10&#124;src=10.0.0.1 dst=2.1.2.2spt=1232
event.risk_score,float,0,
event.severity,long,1,7
event.start,date,0,
event.type,keyword,0,nginx-stats-metrics
event.version,keyword,0,0.1.0
file.ctime,date,0,
Expand Down
21 changes: 21 additions & 0 deletions schemas/event.yml
Original file line number Diff line number Diff line change
Expand Up @@ -128,6 +128,27 @@

In case the two timestamps are identical, @timestamp should be used.

- name: start
type: date
description: >
event.start contains the date when the event started.

This timestamp is distinct from @timestamp in that @timestamp contains
the processed timestamp. For logs these two timestamps can be different
as the timestamp in the log line and when the event is read for example
by Filebeat are not identical. `@timestamp` must contain the timestamp
extracted from the log line, event.start when the log line is read.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For me that is event.created.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would not explain event.start in terms of the phenomenon of reading log files.

I think it would be more beneficial to specifically explain start/end specifically in terms of the observation of long running phenomena, like flows.

Some points I'd try to hit in this description and/or end:

  • start and end usually go hand in hand. Although there may be exceptions (missing field) in the case of partial flows.
  • These are to describe potentially long running events. start can actually be hours before @timestamp, if the observed phenomenon started a long time ago and was just reported.
  • I would relate this description to event.duration.

To @ruflin's point, having 2/3 fields out of start, end and duration is enough to derive the 3rd fields. I'd rather have 3/3, but we may (or may not) want to mention that 2/3 can be enough?

The same could apply to package capturing where @timestamp contains the
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"package capturing" => "packet capture" (same correction on line below as well)

timestamp extracted from the network package and event.created when the
event was created.

In case the two timestamps are identical, @timestamp should be used.

- name: end
type: date
description: >
event.end contains the date when the event ended.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You have a nice description in the PR of this with an example. Could you add this here?


- name: risk_score
type: float
description: >
Expand Down
7 changes: 7 additions & 0 deletions template.json
Original file line number Diff line number Diff line change
Expand Up @@ -222,6 +222,9 @@
"duration": {
"type": "long"
},
"end": {
"type": "date"
},
"hash": {
"ignore_above": 1024,
"type": "keyword"
Expand All @@ -246,6 +249,9 @@
"severity": {
"type": "long"
},
"start": {
"type": "date"
},
"type": {
"ignore_above": 1024,
"type": "keyword"
Expand Down Expand Up @@ -655,6 +661,7 @@
"properties": {
"certificates": {
"doc_values": false,
"ignore_above": 1024,
"type": "keyword"
},
"ciphersuite": {
Expand Down