[auditd_manager] Update fields and sample_event.json #12541
Conversation
| - name: auditd.data.* | ||
| description: Auditd related data | ||
| type: flattened | ||
| type: keyword |
There was a problem hiding this comment.
If this field is declared as flattened, this is not created as mapping currently.
Probably, because there are other fields auditd.data.xxx.
If it set as a dynamic template, it keeps all the fields above plus the ones not able to match (auditd.data.a0-N)
There was a problem hiding this comment.
If flattened is expected maybe this should be defined without the wildcard:
- name: auditd.data
description: Auditd related data
type: flattened
There was a problem hiding this comment.
If that definition is changed as:
- name: auditd.data
description: Auditd related data
type: flattenedThis mapping is not created. I think this is caused because there are other definitions present for auditd.data.<field> fields. For instance auditd.data.action (as keyword).
If it is required to be flattened, I'm afraid it should be needed to remove all the auditd.data.<field> definitions.
There was a problem hiding this comment.
This mapping is not created. I think this is caused because there are other definitions present for auditd.data. fields. For instance auditd.data.action (as keyword).
On what version of the stack? This looks like a case of elastic/kibana#204104
There was a problem hiding this comment.
I did run the tests with 8.16.2 (Kibana version set in the manifest).
Just tested with 8.18.0-SNAPSHOT and it does not generate the flattened type for auditd.data neither.
Complete mapping for `auditd.data` (8.18.0-SNAPSHOT)
"auditd": {
"properties": {
"data": {
"properties": {
"acct": {
"type": "keyword",
"ignore_above": 1024
},
"acl": {
"type": "keyword",
"ignore_above": 1024
},
"action": {
"type": "keyword",
"ignore_above": 1024
},
"added": {
"type": "long"
},
"addr": {
"type": "keyword",
"ignore_above": 1024
},
"apparmor": {
"type": "keyword",
"ignore_above": 1024
},
"arch": {
"type": "keyword",
"ignore_above": 1024
},
"argc": {
"type": "long"
},
"audit_backlog_limit": {
"type": "keyword",
"ignore_above": 1024
},
"audit_backlog_wait_time": {
"type": "keyword",
"ignore_above": 1024
},
"audit_enabled": {
"type": "keyword",
"ignore_above": 1024
},
"audit_failure": {
"type": "keyword",
"ignore_above": 1024
},
"audit_pid": {
"type": "long"
},
"auid": {
"type": "keyword",
"ignore_above": 1024
},
"banners": {
"type": "keyword",
"ignore_above": 1024
},
"bool": {
"type": "keyword",
"ignore_above": 1024
},
"bus": {
"type": "keyword",
"ignore_above": 1024
},
"cap_fe": {
"type": "keyword",
"ignore_above": 1024
},
"cap_fi": {
"type": "keyword",
"ignore_above": 1024
},
"cap_fp": {
"type": "keyword",
"ignore_above": 1024
},
"cap_fver": {
"type": "keyword",
"ignore_above": 1024
},
"cap_pe": {
"type": "keyword",
"ignore_above": 1024
},
"cap_pi": {
"type": "keyword",
"ignore_above": 1024
},
"cap_pp": {
"type": "keyword",
"ignore_above": 1024
},
"capability": {
"type": "keyword",
"ignore_above": 1024
},
"cgroup": {
"type": "keyword",
"ignore_above": 1024
},
"changed": {
"type": "long"
},
"cipher": {
"type": "keyword",
"ignore_above": 1024
},
"class": {
"type": "keyword",
"ignore_above": 1024
},
"cmd": {
"type": "keyword",
"ignore_above": 1024
},
"code": {
"type": "keyword",
"ignore_above": 1024
},
"compat": {
"type": "keyword",
"ignore_above": 1024
},
"daddr": {
"type": "ip"
},
"data": {
"type": "keyword",
"ignore_above": 1024
},
"default_context": {
"type": "keyword",
"ignore_above": 1024
},
"device": {
"type": "keyword",
"ignore_above": 1024
},
"dir": {
"type": "keyword",
"ignore_above": 1024
},
"direction": {
"type": "keyword",
"ignore_above": 1024
},
"dmac": {
"type": "keyword",
"ignore_above": 1024
},
"dport": {
"type": "long"
},
"enforcing": {
"type": "keyword",
"ignore_above": 1024
},
"entries": {
"type": "long"
},
"exit": {
"type": "keyword",
"ignore_above": 1024
},
"fam": {
"type": "keyword",
"ignore_above": 1024
},
"family": {
"type": "keyword",
"ignore_above": 1024
},
"fd": {
"type": "keyword",
"ignore_above": 1024
},
"fe": {
"type": "keyword",
"ignore_above": 1024
},
"feature": {
"type": "keyword",
"ignore_above": 1024
},
"fi": {
"type": "keyword",
"ignore_above": 1024
},
"file": {
"type": "keyword",
"ignore_above": 1024
},
"flags": {
"type": "keyword",
"ignore_above": 1024
},
"format": {
"type": "keyword",
"ignore_above": 1024
},
"fp": {
"type": "keyword",
"ignore_above": 1024
},
"frootid": {
"type": "keyword",
"ignore_above": 1024
},
"fver": {
"type": "keyword",
"ignore_above": 1024
},
"grantors": {
"type": "keyword",
"ignore_above": 1024
},
"grp": {
"type": "keyword",
"ignore_above": 1024
},
"hook": {
"type": "keyword",
"ignore_above": 1024
},
"hostname": {
"type": "keyword",
"ignore_above": 1024
},
"icmp_type": {
"type": "keyword",
"ignore_above": 1024
},
"id": {
"type": "keyword",
"ignore_above": 1024
},
"igid": {
"type": "keyword",
"ignore_above": 1024
},
"img_ctx": {
"type": "keyword",
"ignore_above": 1024
},
"inif": {
"type": "keyword",
"ignore_above": 1024
},
"ino": {
"type": "keyword",
"ignore_above": 1024
},
"inode_gid": {
"type": "keyword",
"ignore_above": 1024
},
"inode_uid": {
"type": "keyword",
"ignore_above": 1024
},
"invalid_context": {
"type": "keyword",
"ignore_above": 1024
},
"ioctlcmd": {
"type": "keyword",
"ignore_above": 1024
},
"ip": {
"type": "ip"
},
"ipid": {
"type": "keyword",
"ignore_above": 1024
},
"ipx_net": {
"type": "keyword",
"ignore_above": 1024
},
"items": {
"type": "long"
},
"iuid": {
"type": "keyword",
"ignore_above": 1024
},
"kernel": {
"type": "keyword",
"ignore_above": 1024
},
"kind": {
"type": "keyword",
"ignore_above": 1024
},
"ksize": {
"type": "keyword",
"ignore_above": 1024
},
"laddr": {
"type": "keyword",
"ignore_above": 1024
},
"len": {
"type": "keyword",
"ignore_above": 1024
},
"list": {
"type": "keyword",
"ignore_above": 1024
},
"lport": {
"type": "long"
},
"mac": {
"type": "keyword",
"ignore_above": 1024
},
"macproto": {
"type": "keyword",
"ignore_above": 1024
},
"maj": {
"type": "keyword",
"ignore_above": 1024
},
"major": {
"type": "keyword",
"ignore_above": 1024
},
"minor": {
"type": "keyword",
"ignore_above": 1024
},
"model": {
"type": "keyword",
"ignore_above": 1024
},
"msg": {
"type": "keyword",
"ignore_above": 1024
},
"nargs": {
"type": "long"
},
"net": {
"type": "keyword",
"ignore_above": 1024
},
"new": {
"type": "keyword",
"ignore_above": 1024
},
"new_chardev": {
"type": "keyword",
"ignore_above": 1024
},
"new_disk": {
"type": "keyword",
"ignore_above": 1024
},
"new_enabled": {
"type": "keyword",
"ignore_above": 1024
},
"new_fs": {
"type": "keyword",
"ignore_above": 1024
},
"new_gid": {
"type": "keyword",
"ignore_above": 1024
},
"new_level": {
"type": "keyword",
"ignore_above": 1024
},
"new_lock": {
"type": "keyword",
"ignore_above": 1024
},
"new_log_passwd": {
"type": "keyword",
"ignore_above": 1024
},
"new_mem": {
"type": "keyword",
"ignore_above": 1024
},
"new_net": {
"type": "keyword",
"ignore_above": 1024
},
"new_pe": {
"type": "keyword",
"ignore_above": 1024
},
"new_pi": {
"type": "keyword",
"ignore_above": 1024
},
"new_pp": {
"type": "keyword",
"ignore_above": 1024
},
"new_range": {
"type": "keyword",
"ignore_above": 1024
},
"new_rng": {
"type": "keyword",
"ignore_above": 1024
},
"new_role": {
"type": "keyword",
"ignore_above": 1024
},
"new_ses": {
"type": "keyword",
"ignore_above": 1024
},
"new_seuser": {
"type": "keyword",
"ignore_above": 1024
},
"new_vcpu": {
"type": "long"
},
"nlnk_fam": {
"type": "keyword",
"ignore_above": 1024
},
"nlnk_grp": {
"type": "keyword",
"ignore_above": 1024
},
"nlnk_pid": {
"type": "long"
},
"oauid": {
"type": "keyword",
"ignore_above": 1024
},
"obj": {
"type": "keyword",
"ignore_above": 1024
},
"obj_gid": {
"type": "keyword",
"ignore_above": 1024
},
"obj_uid": {
"type": "keyword",
"ignore_above": 1024
},
"ocomm": {
"type": "keyword",
"ignore_above": 1024
},
"oflag": {
"type": "keyword",
"ignore_above": 1024
},
"old": {
"type": "keyword",
"ignore_above": 1024
},
"old_auid": {
"type": "keyword",
"ignore_above": 1024
},
"old_chardev": {
"type": "keyword",
"ignore_above": 1024
},
"old_disk": {
"type": "keyword",
"ignore_above": 1024
},
"old_enabled": {
"type": "keyword",
"ignore_above": 1024
},
"old_enforcing": {
"type": "keyword",
"ignore_above": 1024
},
"old_fs": {
"type": "keyword",
"ignore_above": 1024
},
"old_level": {
"type": "keyword",
"ignore_above": 1024
},
"old_lock": {
"type": "keyword",
"ignore_above": 1024
},
"old_log_passwd": {
"type": "keyword",
"ignore_above": 1024
},
"old_mem": {
"type": "keyword",
"ignore_above": 1024
},
"old_net": {
"type": "keyword",
"ignore_above": 1024
},
"old_pa": {
"type": "keyword",
"ignore_above": 1024
},
"old_pe": {
"type": "keyword",
"ignore_above": 1024
},
"old_pi": {
"type": "keyword",
"ignore_above": 1024
},
"old_pp": {
"type": "keyword",
"ignore_above": 1024
},
"old_prom": {
"type": "keyword",
"ignore_above": 1024
},
"old_range": {
"type": "keyword",
"ignore_above": 1024
},
"old_rng": {
"type": "keyword",
"ignore_above": 1024
},
"old_role": {
"type": "keyword",
"ignore_above": 1024
},
"old_ses": {
"type": "keyword",
"ignore_above": 1024
},
"old_seuser": {
"type": "keyword",
"ignore_above": 1024
},
"old_val": {
"type": "keyword",
"ignore_above": 1024
},
"old_vcpu": {
"type": "long"
},
"op": {
"type": "keyword",
"ignore_above": 1024
},
"opid": {
"type": "long"
},
"oses": {
"type": "keyword",
"ignore_above": 1024
},
"outif": {
"type": "keyword",
"ignore_above": 1024
},
"pa": {
"type": "keyword",
"ignore_above": 1024
},
"parent": {
"type": "keyword",
"ignore_above": 1024
},
"pe": {
"type": "keyword",
"ignore_above": 1024
},
"per": {
"type": "keyword",
"ignore_above": 1024
},
"perm": {
"type": "keyword",
"ignore_above": 1024
},
"perm_mask": {
"type": "keyword",
"ignore_above": 1024
},
"permissive": {
"type": "keyword",
"ignore_above": 1024
},
"pfs": {
"type": "keyword",
"ignore_above": 1024
},
"pi": {
"type": "keyword",
"ignore_above": 1024
},
"pp": {
"type": "keyword",
"ignore_above": 1024
},
"printer": {
"type": "keyword",
"ignore_above": 1024
},
"prom": {
"type": "keyword",
"ignore_above": 1024
},
"proto": {
"type": "keyword",
"ignore_above": 1024
},
"qbytes": {
"type": "keyword",
"ignore_above": 1024
},
"range": {
"type": "keyword",
"ignore_above": 1024
},
"reason": {
"type": "keyword",
"ignore_above": 1024
},
"removed": {
"type": "long"
},
"res": {
"type": "keyword",
"ignore_above": 1024
},
"reset": {
"type": "keyword",
"ignore_above": 1024
},
"resrc": {
"type": "keyword",
"ignore_above": 1024
},
"result": {
"type": "keyword",
"ignore_above": 1024
},
"rport": {
"type": "long"
},
"sauid": {
"type": "keyword",
"ignore_above": 1024
},
"scontext": {
"type": "keyword",
"ignore_above": 1024
},
"selected_context": {
"type": "keyword",
"ignore_above": 1024
},
"seperm": {
"type": "keyword",
"ignore_above": 1024
},
"seperms": {
"type": "keyword",
"ignore_above": 1024
},
"seqno": {
"type": "long"
},
"seresult": {
"type": "keyword",
"ignore_above": 1024
},
"ses": {
"type": "keyword",
"ignore_above": 1024
},
"seuser": {
"type": "keyword",
"ignore_above": 1024
},
"sig": {
"type": "keyword",
"ignore_above": 1024
},
"sigev_signo": {
"type": "keyword",
"ignore_above": 1024
},
"smac": {
"type": "keyword",
"ignore_above": 1024
},
"socket": {
"properties": {
"addr": {
"type": "keyword",
"ignore_above": 1024
},
"family": {
"type": "keyword",
"ignore_above": 1024
},
"path": {
"type": "keyword",
"ignore_above": 1024
},
"port": {
"type": "long"
},
"saddr": {
"type": "keyword",
"ignore_above": 1024
}
}
},
"spid": {
"type": "long"
},
"sport": {
"type": "long"
},
"state": {
"type": "keyword",
"ignore_above": 1024
},
"subj": {
"type": "keyword",
"ignore_above": 1024
},
"success": {
"type": "keyword",
"ignore_above": 1024
},
"syscall": {
"type": "keyword",
"ignore_above": 1024
},
"table": {
"type": "keyword",
"ignore_above": 1024
},
"tclass": {
"type": "keyword",
"ignore_above": 1024
},
"tcontext": {
"type": "keyword",
"ignore_above": 1024
},
"terminal": {
"type": "keyword",
"ignore_above": 1024
},
"tty": {
"type": "keyword",
"ignore_above": 1024
},
"unit": {
"type": "keyword",
"ignore_above": 1024
},
"uri": {
"type": "keyword",
"ignore_above": 1024
},
"uuid": {
"type": "keyword",
"ignore_above": 1024
},
"val": {
"type": "keyword",
"ignore_above": 1024
},
"ver": {
"type": "keyword",
"ignore_above": 1024
},
"virt": {
"type": "keyword",
"ignore_above": 1024
},
"vm": {
"type": "keyword",
"ignore_above": 1024
},
"vm_ctx": {
"type": "keyword",
"ignore_above": 1024
},
"vm_pid": {
"type": "long"
},
"watch": {
"type": "keyword",
"ignore_above": 1024
}
}
}
}
}Just realized that there are at least another group socket under auditd.data:
- name: auditd.data.socket.port
description: The port number.
type: long
- name: auditd.data.socket.saddr
description: The raw socket address structure.
type: keyword
- name: auditd.data.socket.addr
description: The remote address.
type: keyword
- name: auditd.data.socket.family
description: The socket family (unix, ipv4, ipv6, netlink).
type: keyword
- name: auditd.data.socket.path
description: This is the path associated with a unix socket.
type: keywordCould this auditd.data.socket field cause that it is better then set a dynamic template (keyword) instead for the missing fields (e.g. auditd.data.a0) ?
There was a problem hiding this comment.
Yeah, it is probably not possible to merge a flattened with a group. The fleet code definitely does not support it, it only merges groups, objects and maybe nested. It would probably not make sense to merge a flattened with other types.
So we should probably avoid flattened in this case, using a wildcard and considering subobjects: false.
There was a problem hiding this comment.
Keeping all the definitions present in the file (except auditd.data.a0-N), I think of these two options to update the definition for auditd.data.*:
- Mapping all strings to keyword
- name: auditd.data.*
description: Auditd related data
type: keyword- Mapping everything as a keyword (even fields whose type would be
long)
- name: auditd.data.*
description: Auditd related data
type: object
object_type: keyword
object_type_mapping_type: "*"Maybe better use the first option, and if it is required, add new corresponding definitions in the future, as it happens for auditd.data.socket.port for instance. To have a better control in the fields. WDYT?
| # this mapping does not generate a dynamic template, and the expected fields do not match | ||
| # should it be kept for documentation purposes? | ||
| # - name: auditd.data.a0-N | ||
| # description: the arguments to a syscall | ||
| # type: keyword |
There was a problem hiding this comment.
What should it be done for this field definition ? Just remove it?
There was a problem hiding this comment.
Would it work with a auditd.data.a* definition, specially to keep the documentation?
| # this mapping does not generate a dynamic template, and the expected fields do not match | |
| # should it be kept for documentation purposes? | |
| # - name: auditd.data.a0-N | |
| # description: the arguments to a syscall | |
| # type: keyword | |
| - name: auditd.data.a* | |
| description: the arguments to a syscall | |
| type: keyword |
Though this would match also things that are not arguments.
So maybe this can be removed, yes.
| "a3": "0", | ||
| "arch": "x86_64", | ||
| "audit_pid": "22501", | ||
| "audit_pid": 2532842, |
There was a problem hiding this comment.
It was required to re-generate the sample_event since this field is now a long (as in the field definition).
andrewkroh
added
bugfix
🚀 Benchmarks reportTo see the full report comment with |
andrewkroh
added
Integration:1password
mrodm
force-pushed
the
update_auditd_manager_mappings
branch
from
e32c560 to
f54601d
Compare
mrodm
added
Integration:auditd_manager
mrodm
force-pushed
the
update_auditd_manager_mappings
branch
2 times, most recently
from
1e8b1b3 to
af08a22
Compare
…41bb116f37d13f2d4e04502d0
(cherry picked from commit fa96beb)
mrodm
force-pushed
the
update_auditd_manager_mappings
branch
from
6b08b22 to
d2ba547
Compare
This reverts commit d2ba547.
| @@ -757,7 +757,7 @@ teardown_test_package() { | |||
| } | |||
|
|
|||
| list_all_directories() { | |||
| find . -maxdepth 1 -mindepth 1 -type d | xargs -I {} basename {} | sort | |||
| find . -maxdepth 1 -mindepth 1 -type d | xargs -I {} basename {} | sort |grep auditd_manager | |||
There was a problem hiding this comment.
To be removed changes in .buildkite/ folder.
mrodm
added
the
Team:Security-Linux Platform
|
Pinging @elastic/sec-linux-platform (Team:Security-Linux Platform) |
|
Hi @elastic/sec-linux-platform, Thanks in advance! |
| @@ -737,6 +739,6 @@ | |||
| type: keyword | |||
| - name: auditd.data.result | |||
| type: keyword | |||
| - name: auditd.data | |||
| - name: auditd.data.* | |||
There was a problem hiding this comment.
My interpretation of the original intent with auditd.data.a0-N is to path_match auditd.data.a* and set a dynamic keyword. So I am thinking we should make it auditd.data.a*. Ideally we could use a match regex to only map auditd.data.a\d+ to keyword, this way if there happens be some non-argument field we get alerted to it not having a specific mapping during testing.
There should be specific mappings for the other audit.data fields already. 🤞
There was a problem hiding this comment.
In that case, it would be needed to keep these two field definitions @andrewkroh :
- name: auditd.data.a*
description: the arguments to a syscall
type: keyword
- name: auditd.data.*
description: Auditd related data
type: keywordUpdated in 1aa7413
Currently, the second one auditd.data.* is still needed. If it is not added, there is an error in system tests:
[0] field "auditd.data.subj_user" is undefined
For the auditd.data.* field definition, it cannot be set flattened type as auditd.data (see https://github.com/elastic/integrations/pull/12541/files#r1946774552).
There was a problem hiding this comment.
[0] field "auditd.data.subj_user" is undefined
It does look like we are missing mappings for the SELinux subject properties like seen in this data.
Here are some additions that should cover these missing fields. Then auditd.data.* won't be needed for the tests to pass. 🤞
- name: auditd.data.subj_user
type: keyword
description: >
The SELinux user identity. This represents the SELinux user role that is
assigned to the subject (user or process) performing an action. It's part
of the SELinux security context and is used to enforce policies that
restrict what actions a subject can perform.
- name: auditd.data.subj_role
type: keyword
description: >
The SELinux role associated with the subject. The role determines the
capabilities a subject has within a given SELinux policy. Roles are used
to define higher-level security attributes in the context of the system's
security policies.
- name: auditd.data.subj_domain
type: keyword
description: >
The SELinux domain or type assigned to the subject. The domain specifies
the type of resource or process the subject is interacting with, helping
enforce domain-based access controls, which are crucial in limiting resource
access.
- name: auditd.data.subj_level
type: keyword
description: >
The SELinux sensitivity level for the subject. It indicates the security
classification level, like `s0` or `s2`, that defines how data or processes
are handled based on confidentiality and integrity levels within the system.
- name: auditd.data.subj_category
type: keyword
description: >
The SELinux category associated with the subject. It helps further refine
the level of access by classifying subjects into categories for
multi-level security (MLS). Categories are often used to label data
with additional attributes, like "high" or "low," enhancing granularity.
|
💚 Build Succeeded
History
cc @mrodm |
Proposed commit message
Update field definitions to be valid when
elastic-packageuses validation based on mappings.Errors can be found in this PR: https://buildkite.com/elastic/integrations/builds/21222
To solve these issues this PR :
flattenedtype by a dynamic template where all string fields are converted to keywordauditd.data.a0-Nsince this is not taken into account.Checklist
changelog.ymlfile.Author's Checklist
elastic-packageenabled mappings: https://buildkite.com/elastic/integrations/builds/21793.buildkitefolder andgo.mod/go.sumfiles.auditd.data.a0-N).How to test this PR locally
Run elastic-package with mappings validation enabled and run system tests:
Related issues