Skip to content

event.duration takes a significative amount of disk space #31574

Open

Description

While analyzing disk space used by data collected by Metricbeat and stored in indexes with TSDB and synthetic _source enabled (elastic/elasticsearch#85649), @nik9000 found that event.duration takes up to 16.7% of the disk space.

This field is automatically added by Metricbeat, by default, with the duration of the fetch operation.

I guess that the main purpose of this field is to monitor or debug metrics collection itself, but this may not be so useful for the final users of most modules.

Being in Metricbeat, this is also added to metrics documents collected by Agent.

We should reconsider this field, or disable it by default.

cc @ruflin

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Assignees

No one assigned

    Labels

    Team:Elastic-Agent-Data-PlaneLabel for the Agent Data Plane teamTeam:Obs-DCLabel for the Data Collection teambugdiscussIssue needs further discussion.

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions