Closed
Description
#76885 introduced the possibility of using ECS Grok patterns instead of the legacy ones.
We should switch to using these in the text structure plugin and for the Grok patterns we add to categorization results.
It looks like some new date formats exist in the latest set of Grok patterns - certainly for newer versions of Tomcat and Catalina, possibly others - we should add those to the timestamp format finder too.
- Add
ecs_compatibility
option to_text_structure/find_structure
endpoint, defaultdisabled
, and change that endpoint to use ECS Grok patterns if it's set tov1
. This may also necessitate making the timestamp format finder aware of two different Grok patterns per timestamp format, and then having it use the appropriate one depending on whether ECS Grok patterns are in use (investigation required). - Change UI to set
ecs_compatibility
tov1
when calling_text_structure/find_structure
. [ML] Passecs_compatibility=v1
when calling ES find file structure API kibana#138428 - Have a look through the ECS Grok patterns that were added in ECS support for Grok processor #76885 and see if there are any new timestamp formats that didn't exist in the original Grok patterns. Maybe Tomcat and Catalina have some new ones, maybe others. If any are found add configs for them to the timestamp format finder in
_text_structure/find_structure
. - Change the Grok pattern creator for
_ml/anomaly_detectors/<job_id>/results/categories
to always use ECS Grok patterns - this change can be made unconditionally without keeping a BWC option for the old Grok patterns, as the functionality is experimental. [ML] Get categories endpoint to use ECS Grok patterns #89386