Add entity to the top level namespaces in which it's being used #2556

mjwolf · 2025-10-22T21:11:27Z

1. What does this PR do?

Add entity to the top level namespace of which it's used in sub-types.

For example, we want user.target.entity fields to be present. user.entity itself is a user type, so entity must be reused on the user type.

This also removes custom generator code which was being used to workaround the fact that this wasn't true before.

2. Which ECS fields are affected/introduced?

3. Why is this change necessary?

Fixes #2555

4. Have you added/updated documentation?

YES

5. Have you built ECS and committed any newly generated files?

YES

6. Have you run the ECS validation tests locally?

YES

7. Anything else for the reviewers?

Commit Message

Add entity to the top level namespace of which it's used in sub-types.

For example, we want user.target.entity fields to be present. user.entity itself is a user type, so entity must be reused on the user type.

This also removes custom generator code which was being used to workaround the fact that this wasn't true before.

Add entity to the top level namespace of which it's used in sub-types. For example, we want `user.target.entity` fields to be present. `user.entity` itself is a `user` type, so `entity` must be reused on the user type. This also removes custom generator code which was being used to workaround the fact that this wasn't true before.

github-actions · 2025-10-22T21:11:43Z

Documentation changes preview: https://docs-v3-preview.elastic.dev/elastic/ecs/pull/2556/reference/

github-actions · 2025-10-22T21:11:43Z

🤖 GitHub comments

Expand to view the GitHub comments

Just comment with:

run docs-build : Re-trigger the docs validation. (use unformatted text in the comment!)

github-actions · 2025-10-22T21:12:51Z

🔍 Preview links for changed docs

trisch-me · 2025-10-23T13:39:25Z

docs/reference/ecs-entity.md

 ## Field reuse [_field_reuse]

 The `entity` fields are expected to be nested at:



cloud.entity is missing

docs/reference/ecs-entity.md

schemas/subsets/main.yml

schemas/entity.yml

kgeller

Left a couple questions

kgeller · 2025-10-23T19:04:45Z

generated/csv/fields.csv

 9.3.0-dev,true,cloud,cloud.target.provider,keyword,extended,,aws,Name of the cloud provider.
 9.3.0-dev,true,cloud,cloud.target.region,keyword,extended,,us-east-1,"Region in which this host, resource, or service is located."
 9.3.0-dev,true,cloud,cloud.target.service.name,keyword,extended,,lambda,The cloud service name.
+9.3.0-dev,true,cloud,cloud.target.target.entity.attributes,object,extended,,,A set of static or semi-static attributes of the entity.


Looks like we are getting target duplicated in the output

cloud.target.target.entity.attributes

kgeller · 2025-10-23T19:07:13Z

generated/csv/fields.csv

 9.3.0-dev,true,user,user.roles,keyword,extended,array,"[""kibana_admin"", ""reporting_user""]",Array of user roles at the time of the event.
 9.3.0-dev,true,user,user.target.domain,keyword,extended,,,Name of the directory the user is a member of.
 9.3.0-dev,true,user,user.target.email,keyword,extended,,,User email address.
-9.3.0-dev,true,user,user.target.entity.attributes,object,extended,,,A set of static or semi-static attributes of the entity.


Looks like all the user.* entity related fields are removed?

Re-organize the finalizer to consider order first, and process self-nested and foreign reuse by order level, rather than doing all foreign reuse first, and then self-nested reuse second. This also collects self-nested fields by order level, to enable the above.

mjwolf · 2025-10-23T22:46:22Z

This is a summary of the changes I made in this PR now:

Removed entity as a top-level field
Added entity as a field on top-level objects:
- cloud.entity.*
- user.entity.*
- service.entity.*
- host.entity.*
- orchestrator.entity.* (this is new. It's requested in the RFC but was missed before)
Added with self-nested objects:
- cloud.origin.entity.*
- cloud.target.entity.*
- user.changes.entity.*
- user.effective.entity.*
- user.target.entity.*
- service.origin.entity.*
- service.target.entity.*
Reverted the changes I made to the generator scripts previously for entity, which enabled the incorrect reuse that was fixed with this
Modified finalizer.py, to consider order level first, and process foreign and self-nested reuse at the same time for each level. This was needed to allow both self-nested and foreign reuse to work with entity.

@nick-alayil, can you confirm if these changes are correct and what you want for entity? entity is added to some other objects as a consequence of being added to the top-level field (i.e. user.effective is a user type, and since entity is now a field on user, it's also added at user.effective.entity). It could be changed, if you don't want entity on these fields

nick-alayil · 2025-10-24T02:01:42Z

@nick-alayil, can you confirm if these changes are correct and what you want for entity?

Thanks @mjwolf for making these changes! From this ticket’s context, everything looks good. I was mainly aiming to get user.entity.* and service.entity.* added.

There are a few other fields missing from a Graph Viz perspective, but I think those should be handled separately from this PR:

host.target.entity.* needed for Graph Viz, but the host FieldSet doesn’t currently have a target nested under host.
generic.entity.* and generic.target.entity.* also needed for Graph Viz, though the generic FieldSet doesn’t exist yet. This would introduce a new root-level object and likely requires a separate discussion/PR.

It could be changed, if you don't want entity on these fields

Not required (user.changes.entity.*, user.effective.entity.*) from the Graph Viz perspective. Similarly, *.origin.entity.* fields (like cloud.origin.entity.* and service.origin.entity.*) are also not needed.

kgeller · 2025-10-24T14:29:48Z

generated/csv/fields.csv

+9.3.0-dev,true,user,user.changes.risk.calculated_level,keyword,extended,,High,A risk classification level calculated by an internal system as part of entity analytics and entity risk scoring.
+9.3.0-dev,true,user,user.changes.risk.calculated_score,float,extended,,880.73,A risk classification score calculated by an internal system as part of entity analytics and entity risk scoring.
+9.3.0-dev,true,user,user.changes.risk.calculated_score_norm,float,extended,,88.73,A normalized risk score calculated by an internal system.
+9.3.0-dev,true,user,user.changes.risk.static_level,keyword,extended,,High,"A risk classification level obtained from outside the system, such as from some external Threat Intelligence Platform."
+9.3.0-dev,true,user,user.changes.risk.static_score,float,extended,,830.0,"A risk classification score obtained from outside the system, such as from some external Threat Intelligence Platform."
+9.3.0-dev,true,user,user.changes.risk.static_score_norm,float,extended,,83.0,A normalized risk score calculated by an external system.


did we mean for user.changes.risk.* to be added?

kgeller · 2025-10-24T14:30:23Z

generated/csv/fields.csv

+9.3.0-dev,true,user,user.effective.risk.calculated_level,keyword,extended,,High,A risk classification level calculated by an internal system as part of entity analytics and entity risk scoring.
+9.3.0-dev,true,user,user.effective.risk.calculated_score,float,extended,,880.73,A risk classification score calculated by an internal system as part of entity analytics and entity risk scoring.
+9.3.0-dev,true,user,user.effective.risk.calculated_score_norm,float,extended,,88.73,A normalized risk score calculated by an internal system.
+9.3.0-dev,true,user,user.effective.risk.static_level,keyword,extended,,High,"A risk classification level obtained from outside the system, such as from some external Threat Intelligence Platform."
+9.3.0-dev,true,user,user.effective.risk.static_score,float,extended,,830.0,"A risk classification score obtained from outside the system, such as from some external Threat Intelligence Platform."
+9.3.0-dev,true,user,user.effective.risk.static_score_norm,float,extended,,83.0,A normalized risk score calculated by an external system.


same with user.effective.risk.*?

mjwolf · 2025-10-24T14:58:14Z

experimental/generated/csv/fields.csv

 9.3.0-dev+exp,true,user,user.target.id,keyword,core,,S-1-5-21-202424912787-2692429404-2351956786-1000,Unique identifier of the user.
 9.3.0-dev+exp,true,user,user.target.name,keyword,core,,a.einstein,Short name or login of the user.
 9.3.0-dev+exp,true,user,user.target.name.text,match_only_text,core,,a.einstein,Short name or login of the user.
+9.3.0-dev+exp,true,user,user.target.risk.calculated_level,keyword,extended,,High,A risk classification level calculated by an internal system as part of entity analytics and entity risk scoring.


These new risk fields should have existed previously. risk is a field in user type, and since user.target is a user type, user.target.risk should have exist. This is fixed by the change to finalizer ordering that fixed entity fields in the same situation.

…tic#2556) Add entity to the top level namespace of which it's used in sub-types. For example, we want user.target.entity fields to be present. user.entity itself is a user type, so entity must be reused on the user type. This also removes custom generator code which was being used to workaround the fact that this wasn't true before.

… (#2557) Add entity to the top level namespace of which it's used in sub-types. For example, we want user.target.entity fields to be present. user.entity itself is a user type, so entity must be reused on the user type. This also removes custom generator code which was being used to workaround the fact that this wasn't true before.

mjwolf mentioned this pull request Oct 22, 2025

Entity subdomain doesn’t exist in top level domains #2555

Closed

trisch-me reviewed Oct 23, 2025

View reviewed changes

docs/reference/ecs-entity.md Outdated Show resolved Hide resolved

mjwolf marked this pull request as ready for review October 23, 2025 16:04

mjwolf requested a review from a team as a code owner October 23, 2025 16:04

mjwolf added 2 commits October 23, 2025 10:13

Remove entity as a top-level field

57e3813

Add

8ba0393

mjwolf requested a review from trisch-me October 23, 2025 17:24

kgeller reviewed Oct 23, 2025

View reviewed changes

schemas/subsets/main.yml Outdated Show resolved Hide resolved

kgeller reviewed Oct 23, 2025

View reviewed changes

schemas/entity.yml Show resolved Hide resolved

kgeller reviewed Oct 23, 2025

View reviewed changes

Remove commented code

3dc652e

mjwolf requested a review from kgeller October 23, 2025 19:01

kgeller reviewed Oct 23, 2025

View reviewed changes

mjwolf added 4 commits October 23, 2025 14:35

Remove double nesting of target

2635952

Add entity to correct top-level objects

a31ca11

fix lint errors

4f237fc

mjwolf requested a review from kgeller October 23, 2025 22:50

Remove debug code

dc466f9

kgeller reviewed Oct 24, 2025

View reviewed changes

mjwolf commented Oct 24, 2025

View reviewed changes

kgeller approved these changes Oct 24, 2025

View reviewed changes

mjwolf merged commit 5db6546 into elastic:main Oct 24, 2025
7 checks passed

mjwolf mentioned this pull request Oct 24, 2025

[9.2 forward port] Add entity to the top level namespaces in which it's being used (#2556) #2557

Merged

		## Field reuse [_field_reuse]

		The `entity` fields are expected to be nested at:

Uh oh!

Add entity to the top level namespaces in which it's being used #2556

Add entity to the top level namespaces in which it's being used #2556

Uh oh!

Conversation

mjwolf commented Oct 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

1. What does this PR do?

2. Which ECS fields are affected/introduced?

3. Why is this change necessary?

4. Have you added/updated documentation?

5. Have you built ECS and committed any newly generated files?

6. Have you run the ECS validation tests locally?

7. Anything else for the reviewers?

Commit Message

Uh oh!

github-actions bot commented Oct 22, 2025

Uh oh!

github-actions bot commented Oct 22, 2025

🤖 GitHub comments

Uh oh!

github-actions bot commented Oct 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔍 Preview links for changed docs

Uh oh!

trisch-me Oct 23, 2025

Choose a reason for hiding this comment

Uh oh!

mjwolf Oct 23, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

kgeller left a comment

Choose a reason for hiding this comment

Uh oh!

kgeller Oct 23, 2025

Choose a reason for hiding this comment

Uh oh!

kgeller Oct 23, 2025

Choose a reason for hiding this comment

Uh oh!

mjwolf commented Oct 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

nick-alayil commented Oct 24, 2025

Uh oh!

kgeller Oct 24, 2025

Choose a reason for hiding this comment

Uh oh!

kgeller Oct 24, 2025

Choose a reason for hiding this comment

Uh oh!

mjwolf Oct 24, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

mjwolf commented Oct 22, 2025 •

edited

Loading

github-actions bot commented Oct 22, 2025 •

edited

Loading

mjwolf commented Oct 23, 2025 •

edited

Loading