Source/target and diff modeling

[andrewstucki]

This is a more generic application of modeling either "source/target" style events or "diff" events that I'm spinning off from #589

As I initially mentioned (#589 (comment)). There are a number of things that ECS should support modeling including things like:

• setuid/setgid operations
• file modification events (i.e. renames, permissions)
• IPC calls
• network requests/flows (current use of source and destination)
• user modifications
• process execs
• registry modifications
• windows source/target audit log info

Overall these more or less fall into two categories:

1. Modelling communication between two like things (network connections, IPCs, windows audit log)
2. Modification events

Currently the way ECS has started to approach this is to make fields that are specific to each of these domains, i.e. source/destination are currently for network modeling only, and then there's also client/server

I'm advocating for adopting a more generic field set that allows you to do generic source/target or diff modeling which would essentially allow you to embed any other field set in it.

For example--something like origin and target (sad that source/destination is already taken):

For file modifications:

origin.file.name = "foo"
target.file.name = "bar"

For process execs:

origin.process.path = "foo"
target.process.path = "bar"

For network requests (slightly difficult because of the lack of port info):

origin.host.ip = "foo"
target.host.ip = "bar"

For user modification (maybe by another user baz who did the modification?):

user.name = "baz"
origin.user.name = "foo"
target.user.name = "bar"

Thoughts?

[janniten]

For user modification (maybe by another user baz who did the modification?):

user.name = "baz"
origin.user.name = "foo"
target.user.name = "bar"

Hi @andrewstucki,
baz is the user performing the modification?
Thank you

[andrewstucki]

@janniten that was the idea in that particular example, but the desire was just to highlight the need to model any sort of diffing or source/destination style events in a sizeable portion of the existing field sets out there.

I'm fairly open to ideas on naming/implementation (would this be at the top-level as I suggested or embedded inside each top-level field set, etc.), but just calling out the need to solve this issue for a number of use-cases so that we don't just solve the same problem a myriad of different ways as they arise. So, any suggestions/opinions on how we should go about this are more than welcome!

[rw-access]

Cross-process activity could also be modeled this way. This could include injection, credential access/handle opening, etc.

source.process.name = "totallynotmalware.exe"
target.process.name = "lsass.exe"

I'd prefer that versus nesting source/target under process. Then you have two mega namespaces source and target without needing to pollute everything with a source and target.

You could also argue that source is implicit for the existing namespaces. The above example could be equivalently written as:

process.name = "totallynotmalware.exe"
target.process.name = "lsass.exe"

The big advantage is that you can now build aggregations on those better. For instance, "what are the counts of the categories of events for cmd.exe?" You don't want to search search.process.name and process.name, and that will also make it tricky to assign them into buckets.

process.name == "cmd.exe" // instead of : process.name == "cmd.exe" or source.process.name == "cmd.exe"
| count event.category

[andrewstucki]

I totally agree about the implicit nature of some things, and am slightly less concerned with the proposed origin field set than the target field set. The thing that this would preclude us from doing though is using something like the origin field set to model the original state of a diff operation from a third entity (i.e. the example of the user who modifies a different user's name).

At least in our particular use case for security related stuff, I think that there's a ton of utility introducing the target field set even if we weren't able to introduce origin. But I'd like people with other use-cases to chime in on the utility of these constructs as well.

[rw-access]

@andrewstucki just to make sure I'm tracking your example well, does this reflect your example?
User Alice renamed Bob to Robert.

{
  "user": {"name": "alice"},
  "origin": {"user": {"name": "bob"}},
  "target": {"user": {"name": "robert"}}
}

This maps most closely to a Event ID of 4781 for Windows Event Logs (Security)

Subject:

   Security ID:  ACME\Alice
   Account Name:  Alice
   Account Domain:  ACME
   Logon ID:  0x1f40f

Target Account:

   Security ID:  ACME\robert
   Account Domain:  ACME
   Old Account Name: bob
   New Account Name: robert

Additional Information:

   Privileges:  -

[marshallmain]

Using origin and target to represent the old vs new state of a single user (or object in general) seems like a big semantic change from the process semantics where the origin process is generally taking action on the target process.

If I started out using origin.process and target.process and tried to apply that logic to origin.user and target.user I would expect origin.user to be taking an action on target.user, whereas the suggestion here is origin.user is transforming into target.user.

I definitely want to add the capability to describe an original state that has transformed to a new state but conflating it with an object taking action on a different object seems confusing to me.