Skip to content

Find a way to Deduplicate Index Settings #78892

Open
@original-brownbear

Description

@original-brownbear

Relates #77466

Unlike the mapping metadata which we store in compressed+serialized for on this master node, we do store the settings for each index as deserialized Settings object. This can be a significant source of heap usage if there's a large number of indices with non-trivial settings. An example of this would be the audit-beats template which contains a fairly long list of field names in index.query.default_field. In this example, handling 10k audit-beat indices takes almost 500MB of master heap just for storing the duplicate lists of field names in Settings instances.

I will look for an easy win here, it shouldn't be too hard to deduplicate these in some form when building index metadata.

Metadata

Metadata

Labels

:Distributed Coordination/Cluster CoordinationCluster formation and cluster state publication, including cluster membership and fault detection.>enhancementTeam:Distributed (Obsolete)Meta label for distributed team (obsolete). Replaced by Distributed Indexing/Coordination.

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions