Add Data field descriptions

[rayafratkina]

Describe a specific use case for the feature:
In complex deployments of Elastic stack, there is often a need to annotate fields for end users beyond just the name and type since the user who create visualization and explore data in discover may not be deeply familiar with the schema or the schema is too complex and even knowledgeable users may benefit from reminders.

Describe the feature:
Idea is to add a Description field to Index pattern management. This will be optional.
There are 2 ways we can use it

1. Add a toooltip to the field list in Lens/Discover.
2. Create a read-only "Data dictionary" page that would be available to anyone with view Discover/Dashboard permissions.
   User will be able to pick the index pattern and see a table showing field name, type, and description. Ideally the table will be searchable and filterable based on keyword and type.

Open questions/other considerations:

• What about localization? Assuming we would not support translating the descriptions explicitly, but it would be possible to have different descriptions per space if we implement this on the Kibana index patterns.

cc @alexfrancoeur @linyaru @timductive @alexh97 @VijayDoshi

[elasticmachine]

Pinging @elastic/kibana-app-services (Team:AppServices)

[VijayDoshi]

+1 to this. It is very common for users to need more information about a field than what is possible in the name to give them confidence in their analysis. Suggest limiting the description to a reasonable number of characters - perhaps 255.

[alexfrancoeur]

++ I think this would be useful for a number of reasons. At some point, we probably want to connect with the Elasticsearch team to see if Elasticsearch admins can ship standardized descriptions independent of index patterns, but having the framework in place to start adding descriptions to tooltips, stand alone views and other areas within Kibana will be necessary to make these data dictionaries more prominent.

@elastic-jb maybe this is something we can sync with the team on? I imagine a number of edge cases will pop up, but overall, it feels like lower hanging fruit to augment index patterns.

[alexfrancoeur]

@stevewritescode you might be interested in following this feature

[timroes]

Linking this since very slightly related: #17087 This issue here is about allowing free form semantical descriptions on fields, the linked issue about structured semantic data. No duplicate just linking for better findability.

[rschirin]

hey there, I saw that my request has been closed and "merged" with this one. I thought to a tooltip since I got feedback from analysts in my office

[rayafratkina]

@rschirin sorry, looks like 2 different changes to your issue conflated: in this comment @kertal mentioned that we have recently added ability to customize field names in Management.

This issue will be used going forward to track adding an additional Descriptions field. It is not done yet and something we are talking about doing in the future.

[rschirin]

@rschirin sorry, looks like 2 different changes to your issue conflated: in this comment @kertal mentioned that we have recently added ability to customize field names in Management.

This issue will be used going forward to track adding an additional Descriptions field. It is not done yet and something we are talking about doing in the future.

Personally, I think that just the capacity to change/add/modify/customize a field name is not what I was looking for.
The description of a field's name could not be summarized with other 2 words. That is the reason I think that currently Kibana is missing of a tooltip or glossary.
As I described in my scenario, if we use an acronyn we cannot explain it in a short field name.

[elastic-jb]

@cjcenizal this sounds very similar to the idea we were brainstorming late last year. I think there is a ton of value especially in context of runtime fields. Who created it? When was the formula changed? Even just the acronym expansion idea is very useful.

[cjcenizal]

Yes @elastic-jb, if I recall correctly we were thinking that it'd be useful to support extensive annotations on runtime fields so that you could record the reason behind your changes to any given field. This would help multiple analysts collaborating on the same index patterns communicate as they iterate on the data together.