[ECS] Multiple users in an event proposal

[webmat]

After reviewing proposals and discussions, new and old, private and public (#117, #234, #589, #678, elastic/beats#9963, elastic/beats#10111, elastic/beats#10192), here's a high level proposal. I'd like to get feedback on this.

When user.* is populated at the root of an event, it's meant to capture the user performing an action, or the only user relevant to an event, when that's the situation.

The goal of this proposal is to solve the following use cases:

1. User A creates or deletes User B (no prior or post state)
2. User A modifies User B (there's a prior and a post state)
3. User A assumes the identity of User B (or fails to)

One thing to note, is that with real-time privilege escalation (e.g. sudo, MacOS modal) or auditing frameworks like auditd, it's useful to be able to identify all 3 of these additional users concurrently, in a single event.

Here are locations I propose we could nest the user schema, to represent various use cases:

• user.effective.*
• user.target.*
• user.new.*

Note that in each of these cases, we are nesting the user schema as a new name, similar to what we do with process.parent. To give a concrete example the effective user name would be at user.effective.name (not at user.effective.user.name).

user.effective.*

Used to identify the final identity, when logging in remotely as another user, or escalating privileges.

Simplified privilege escalation example, where "alice" escalates privileges to "root":

{
  "user": {
    "name": "alice",
    "effective": {
      "name": "root"
    }
  }
}

user.target.*

Used to identify the user being modified, or targeted by an action.

Simplified user management example, where "root" modifies user "bob":

{
  "user": {
    "name": "root",
    "target": {
      "name": "bob"
    }
  }
}

user.new.*

Used to identify what was changed about the user. Ideally this would only capture the difference, not repeat all values.

Simplified user management example, when an existing user is modified, for example changing user name. Here user "bob" is renamed "foo" by "root":

{
  "user": {
    "name": "root",
    "target": {
      "name": "bob"
    },
    "new": {
      "name": "foo"
    }
  }
}

Note: The creation of a new user would only use user.target, not user.new. user.new should only be used.

Putting it all together

If it's possible to determine who was the original user prior to privilege escalation, a full event could look like this:

{
  "user": {
    "name": "alice",
    "effective": {
      "name": "root"
    },
    "target": {
      "name": "bob"
    }
  }
}

The action being observed should be captured in event.action.

If the event is user management, the category fields should be event.category: [ "iam" ] and event.type: [ "user" ] with a secondary entry in event.type, whichever is more appropriate: "creation", "deletion" or "change".

If the event is authentication and privilege escalation, event.category: [ "authentication" ], event.type: ["start"] and event.outcome either "success" or "failure".

Discussion points

Nowhere in the above plan do we use the current nestings of the user field set: client.user, destination.user, host.user, server.user, source.user.

I'm wondering if people are using host.user or if we should deprecate it for removal in ECS 2.0.

I'd also like to know if the 4 nestings under the network-related field sets are currently being used. I realize that folks may have used for example source.user and destination.user to model remote logons or the "acting" vs "target" users, with ECS specifying nothing yet on how these interactions should be modelled until now. I'd like to get your input if the plan above makes sense.

Are there other uses that are actually network-related for source.user and destination.user?

[webmat]

Ping @MikePaquette @dainperkins @dcode @andrewstucki @leehinman @rw-access @marshallmain @janniten @willemdh @neu5ron @vbohata

Let's have a virtual party! #socialdistancing :-)

[willemdh]

@webmat Really good and clear proposal!

We are not using host.user.

panw.panos contain source.user.name and destination.user.name. There is no need for source|destination.user.new|effective|target there afaik.

[neu5ron]

Ping @MikePaquette @dainperkins @dcode @andrewstucki @leehinman @rw-access @marshallmain @janniten @willemdh @neu5ron @vbohata

Let's have a virtual party! #socialdistancing :-)

i'm in :)

[janniten]

Hi all!
I'm agree with @willemdh. This approach is very clear and represents very well all the cases we'have been discussing. I love it!
We are not using the host.user neither.
Regarding to the network-related field, for the Fortinet logs we are mapping, we only use source.user and destination.user. Same case that panw.panos

[webmat]

@janniten @leehinman Do you have insights you could share about the semantics of using source.user and destination.user? Would you say this maps with "3. User A assumes the identity of User B (or fails to)"

I may not have made this explicit enough in the user.effective description (I've now updated the issue body). But that scenario is meant to represent:

• local privilege escalation like su or sudo
• remote logon: "bob" logs on to machine B as "alice"

So I'm wondering if we should deprecate those, or clarify differences in semantics.

@janniten I'd like to see Fortinet events samples, if you can share any.

Looking at the Beats PANOS logs, it looks like it's only populating the source.user.* (and client.user.*) side of the equation, never the destination.user.*. So looking at this, I wonder if these are simply the "doer" in the event and hence should be at the top level, instead of nested there. WDYT @leehinman?

[willemdh]

@webmat Just checked our panw logs and about 1/4 of the logs containing source.user.name have a destination.user.name.

This depends on what the user-id agents saw as the last logon to the destination IP.
So in fact this information is not always correct, as the event could be about traffic to a system which has multiple users or might even have nothing to do with any users logged in the destination ip.

[janniten]

Hi @webmat, I think the case "3. User A assumes the identity of User B (or fails to)"
is best represented using the user.name and user.effective rather than using the source.user and destination.user

Attached, some Fortinet logs samples
samples.txt

[leehinman]

@webmat I like the proposal. Couple questions.

Would it be correct to say that user.effective would be used when we know that the event represents some kind of attempted transition across an identity boundary. Examples of identity boundary transitions could be:

• changing identity on host (sudo, su, Run as)
• Running a setuid/setgid program
• Login (ssh, Kerberos cross realm auth, etc) where we know existing identity and identity you are hoping to become
• Application dropping privileges (Sendmail becoming user to deliver mail)
• Basic Auth in HTTP Request (if we know your current user info)

This wouldn't cover situations where we are establishing your identity for the first time or we don't have initial identity information.

• First login to workstation
• Firewall observing Basic Auth in HTTP Request

Also it looks like we would only support one identity boundary transition per event? So you wouldn't capture the following as a single event:

User Alice on machine A runs:
ssh bob@b.elastic.co "sudo useradd malory"

I think one identity transition per event is good, I'm just making sure I'm reading it right.

I can't think of an example where we couldn't replace source.user with user and destination.user with user.effective

[rw-access]

I just created the issue #810 for capturing information about Windows logon sessions and tokens and ran into this problem there, too. You can have multiple tokens in a single event, so whatever we do here should also be done there.

I realized yet another case for the multiple X in an event. We currently have process and process.parent, but do we also need to add process.target? That would refer to cross-process events. For example, mimikatz.exe opening a handle and reading memory from lsass.exe.

I don't think we need to pollute this issue with that specific scenario, but I figured it would be good to bring attention to the fact that "multiple users in an event" is just one example of "multiple X in an event", and the decision we make here should be easy enough to apply in multiple places.

For that reason, I'm wondering if it could make sense to have a target fieldset where we can map user, process, token, etc. underneath that. I'm not sure which is more clear to end users, but I'm not attached to either way: user.target, process.target, token.target vs target.user, target.process, target.token, etc.

Edit: There are also some cases where you could have three different X's in an event. For example, the acting process, the logging process and the target process. s/process/user happens as well.

[janniten]

I just created an issue for capturing information about Windows logon sessions and tokens and ran into this issue there, too. You can have multiple tokens in a single event, so whatever we do here should also be done there.

@rw-access, Can you provide more information about this issue? Thank you

[rw-access]

Edited my comment above to add a link to #810, which contains the extra context. I think many of the concerns are still separate from this issue, and I don't want to hijack this thread, so any token- or session-specific discussion would be perfect on #810

[webmat]

@leehinman Yes, I agree with your examples. Whenever we can identify the prior identity (user at top level) and the ultimate identity , we should capture the former at user, and the latter at user.effective.

Note that I don't think this should set an undue burden on event sources to keep track of users over successive events. I think it's fine to track users across events. So there's 2 situations I see here:

• when using an audit framework like auditd, if you can always determine who's the "real" user behind a privileged command, or the real user prior to switching user, then by all means let's always fill {"user": { "name"... "effective": {"name" ...}}
• if the event source sees one transition (e.g. sudo su to root), then sees "root" doing stuff without the audit/original user, I think it's fine to capture only the transition event with {"user": { "name"... "effective": {"name" ...}} and the following ones with {"user": { "name": "root"... }}
  • I think people are used to tracing user activity across events like that, when there's no audit framework. @dcode @neu5ron thoughts?

Hopefully the semantics work for setuid/setgid as well. I think they do, but please let me know if you see potential edge cases.

Here's how I see the 3 levels example you're giving with ssh bob@b.elastic.co "sudo useradd malory":

• we have "alice" running the command "ssh bob@b.elastic.co ..." on host a, and therefore emitted by this host.
  • {"user": { "name": "alice"... }, "destination": { "user": {"name": "bob" ...}}}
• Then on host b we have user "bob" running the command sudo useradd malory. So this one is emitted by host b.
  • {"user": { "name": "alice"..., "effective": {"name": "bob" ...}, "target": {"name": "mallory" ...}}

WDYT?

I agree with @willemdh: there's no intention of nesting all of that new user mega-structure (user.target, user.effective...) in all places where user is currently reused. This bit is tricky, though, because I do want to drag along some of the nesting (user.group) but not all (the new ones proposed here)