Project: Fields

[stacey-gammon]

Project Fields

Access field values using the search fields response, not source.

Intro

It's common for Kibana developers to access field values from _source. This was made very apparent during the runtime fields mapping investigation.

This should be considered an anti-pattern, instead the new fields parameter on search requests should be used.

Differences

There are some differences to be aware of when switching over:

1. Unmapped fields will not be returned in the fields array (but are in _source).
2. Runtime fields and field aliases will be returned in the fields array (but are not in _source).
3. Object arrays are currently flattened. We have an opportunity to change this.
4. Nested objects are currently flattened but the plan is to unflatten them.
5. Multi-field sub fields will report values, where as in _source, only the parent field has a value and all sub fields work like empty fields.
6. Field values may be formatted differently. For example the number "1.5" ingested into source will be returned as "1" if the field is mapped to type long.
7. TODO: Disabled fields? Disabled fields are returned by source, are they returned in the fields array? This would be a problem, they want them to be displayed. They aren't searchable or filterable, but useful to grab values from them.

TODO: clarification on when _source is appropriate. e.g. drilled into views like in Uptime, Discover "raw" view.

Work items

TODO: get links for these items.

• Support fields parameter in search services. [Data Services] Support fields parameter in high level search apis #77241 Owner: @ppisljar
• Switch Discover over. Owner: @kertal
• Switch Lens over. Owner: @wylieconlon
• Switch Logs UI over. Owner: @weltenwort
• Switch timeline over. Owner: @XavierM
• Switch ML over. Owner: @peteharverson
• Switch maps over. Owner: @thomasneirynck
• Switch Synthetics over. Owner: @andrewvc (They own the mapping so no motivation to switch over, but also no problem switching over. 7.11 booked. Possibly 7.12? If _source goes away, they would have to switch over). Does not use data search services.

8.0

Starting in 8.0, I think we should consider not returning _source by default because it encourages developers to access field values from it.

[stacey-gammon]

Work is complete, closing the project