Description
openedon May 17, 2024
Description
Related to #108469
To be able to troubleshoot ingestion issues, it's helpful to know what changed around the time problems started. A full version control system of ES objects is an aspirational goal here, but as a low hanging fruit the meta information about when an ES object was created, last changed and by whom would give a lot of the value without big investments.
Scope
For index templates, component templates and ingest pipelines, track created_at
, modified_at
and modified_by
and return as part of the respective APIs to retrieve these objects:
GET _ingest/pipeline/my-pipeline
{
"my-pipeline": {
"processors": [
...
],
"_meta": {
...
},
"created_at": "2025-05-05T00:00:00",
"modified_at": "2025-05-05T00:00:00",
"modified_by": "user_xyz",
}
}
GET _component_template/my-template
{
"component_templates": [
{
"name": "my-template",
"component_template": {
"template": {
...
},
"version": 3,
"_meta": {
...
},
"created_at": "2025-05-05T00:00:00",
"modified_at": "2025-05-05T00:00:00",
"modified_by": "user_xyz",
}
}
]
}
GET _index_template/my-template
{
"index_templates": [
{
"name": "my-template",
"index_template": {
...,
"created_at": "2025-05-05T00:00:00",
"modified_at": "2025-05-05T00:00:00",
"modified_by": "user_xyz",
}
}
]
}
It's not possible to write these properties.
Considerations
Merge with _meta
The existing _meta
object is user-controlled. An alternative approach would be to add these pieces of information to this object and override whatever the user set manually. While cleaner in the sense of not introducing more properties on these objects, this has some downsides:
- Information can potentially be falsified
- Might interfere with custom user-defined tracking solutions - breaking change?
Permissions
This approach might leak usernames to other users that normally wouldn't have access to them. This could be counteracted by only returning modified_by
if the user has the required permissions to read user data in the first place.