Document index.load_fixed_bitset_filters_eagerly

[ppf2]

FixedBitSetFilterCache is a data structure that is loaded eagerly in memory (by default) to support nested query/filter and nested aggregations. However, the problem is that it can cause it to use too much heap for it is loaded for all nested fields (regardless of whether these fields are being used). To prevent this from happening, a common configuration workaround is to set index.load_fixed_bitset_filters_eagerly: false in the yml of the nodes and restart them to prevent the nodes from running OOM when attempting to eagerly load the fixedbitsets.

At this point, I have seen enough occurrences of this in the field which necessitates the setting of index.load_fixed_bitset_filters_eagerly: false to prevent the nodes from running out of memory. It may make sense to start documenting this setting unless there is a better solution we are already planning :)

[pickypg]

Related to #10224

[ppf2]

Maybe we can also look into having ES set index.load_fixed_bitset_filters_eagerly automatically on startup (eg. by collecting stats on the number of nested fields on startup, or other relevant metrics) and making the decision on whether it is appropriate to load them eagerly vs. lazily based on the amount of heap available, etc.. Currently, users tend to find out that they have to set this option after their nodes have run out of memory.

[clintongormley]

I think we should document this setting, but not add heuristics. Instead, we'll add a soft limit to the number of nested docs (#14983) and look at ways of removing the need for the fixed bitsets (#14919 (comment))

[pickypg]

I don't think that this was intentionally closed.