[discuss] Add unique identifiers to English responses

[joshdover]

In order to support localization efforts in Kibana, I'd like Elasticsearch to provide stable identifiers in responses in addition to English strings that Kibana can use to lookup the corresponding localized string to display to end users in their chosen language. Types of English strings Elasticsearch provides now: error messages, deprecation warnings, ...?

There many ways we could chose to format this identifier, but instead of using error code numbers, I propose that we use human-readable, namespaced strings. For example, in the Deprecations Info API (GET _xpack/migration/deprecations) we could add an attribute to each returned deprecation ("errors" in this sense) like so:

{
  "level" : "info",
  "message" : "Coercion of boolean fields",
  "messageId": "xpack.migration.deprecations.coercionBooleanFields.message",  // added
  "url" : "https://www.elastic.co/guide/en/elasticsearch/reference/6.0/breaking_60_indices_changes.html",
  "details" : "more information, like which nodes, indices, or settings are to blame"
  "detailsId": "xpack.migration.deprecations.coercionBooleanFields.details"  // added
}

Kibana can then use the identifiers to lookup localized versions of the string and use the provided English string as a fallback.

Ideally whatever we do in this Deprecations API we can use as a pattern going forward, so I'd like to start a discussion about whether or not this is a pattern that seems adaptable to the rest of Elasticsearch or not.

cc @cjcenizal @sebelga @talevy @azasypkin

[talevy]

I am new to localization strategies, but it may make sense to bubble up within a general API interaction with ES. The Deprecation API probably won't be the last API to be localized by Kibana. May make sense to explore more general rendering strategies on the ES side, like creating a common locale flag that will add additional locale-specific metadata into the responses

[elasticmachine]

Pinging @elastic/es-core-infra

[sebelga]

@joshdover Yes, this is something that would greatly help us in the UI. I already mentioned having error codes coming back in the errors returned by ES as they would not only help guiding the user to the proper action/screen to fix the error, but it would also help us for the translations of the error messages.

messageId could work for the same purpose. We would not be able to use this messageId as translation key for the translation engine to pick it up, right? Meaning: the translation engine need the key to be in plain string in the ui code, is that correct?

[joshdover]

@talevy I like the idea of having a common URL param that ES would use to tell whether or ES should return translation string information to the client.

@sebelga If I understand you correctly, yes the UI uses these static translation identifiers to lookup the localized string in the user's preferred language. Here's an example:

https://github.com/elastic/kibana/blob/cc07aa29d5c59efd116cd523c8ea5999dabe4258/src/core_plugins/table_vis/public/table_vis.js#L55-L57

[sebelga]

@joshdover I was referring that we would not be able to do this:

const elasticSearchResponse = await someApi();
return i18n.translate(elasticSearchResponse.messageId, {
      defaultMessage: elasticSearchResponse.message,
}),

[azasypkin]

@joshdover I was referring that we would not be able to do this:

const elasticSearchResponse = await someApi();
return i18n.translate(elasticSearchResponse.messageId, {
      defaultMessage: elasticSearchResponse.message,
}),

Yeah, this won't work. If we want ES messages to be included into Kibana i18n workflow we need to do something like this (to be able to extract message-id <---> message pairs during static analysis):

const ES_MESSAGES = new Map([
  ['xpack.migration.deprecations.coercionBooleanFields.message', 
     i18n.translate('xpack.migration.deprecations.coercionBooleanFields.message', {
      defaultMessage: 'Coercion of boolean fields',
    })
  ]
]);
...............
const elasticSearchResponse = await someApi();
return ES_MESSAGES.get(elasticSearchResponse.messageId) || elasticSearchResponse.message;

May make sense to explore more general rendering strategies on the ES side, like creating a common locale flag that will add additional locale-specific metadata into the responses

If I understand your idea correctly, then it would make Kibana life easier since we'll be just passing locale within request and displaying what ES returns, but this will require ES to implement their own i18n workflow: extract all labels that needs to be translated, share them with translation vendors and incorporate translations back to ES code/distribution. We're going to use this approach with 3rd party services Kibana interacts with (e.g. map and tile services) though.

[cjcenizal]

The workflow @azasypkin describes is what I had in mind, too. Just having a consistent identifier for error messages would be tremendously helpful. I don't think we need ES to handle the localization aspect, though I don't think that option is precluded if we want to consider adding that at a later stage.

[sebelga]

@azasypkin Yes that's what I thought, it'd be nice to not have to duplicate the translation key and have the i18n extractor take into account this use case. Parsing something like this

const ES_MESSAGES = new Map([
  [
     'xpack.migration.deprecations.coercionBooleanFields.message', // key
     'Coercion of {someValue} fields' // default message
  ]
]);

And then be able to use it i in i18n.translate() passing the messageId or errorCode from ES

[azasypkin]

Parsing something like this

Well, I'm concerned about several things here:

• It's pretty hard to differentiate this map from generic Map<string, string>, so our label extraction tools may flag too many false positives
• While it's trivial to parse code like this i18n.translate(response.messageId, { defaultMessage: ES_MESSAGES.get(response.messageId) }) it may be a way to complex to properly interpret this and make sure that i18n.translate is called with something that makes sense and not random variables during static analysis (e.g. typos or just wrong usage)
• It's not clear how to define optional description (aka additional context that is provided to translators together with the label itself) with such approach

it'd be nice to not have to duplicate the translation key

I'd like to get more context on what user-facing messages we intend to receive from ES side first, but I have a feeling that message IDs returned from ES (if any) may not map 1:1 to label IDs we use in Kibana. Label IDs are supposed to be used by translators to get additional context about the part of the app or flow where the label is used and how to find it in the app itself. I can potentially imagine the case when the same ES API is used in two different contexts in Kibana and hence messages may require different IDs, be a part of other label/text etc... Very-very synthetic example:

const ES_MESSAGES = new Map([
  ['es.api.search.notFound', 
    i18n.translate('xpack.security.userSearch.userNotFound', {
      defaultMessage: 'User not found',
    })
  ]
]);
.....
const ES_MESSAGES = new Map([
  ['es.api.search.notFound', 
     i18n.translate('kbn.discovery.search.noResultsFound', {
      defaultMessage: 'Your search - {userQuery} - did not match any documents. ',
      values: { userQuery }
    })
  ]
]);

[sebelga]

@azasypkin I totally agree. I was a bit distracted by the fact that in our discussion the error codes coming from ES where so similar to a i18n translation key.
Maybe we'd prefer another format, like the ones introduced in Node.js? https://nodejs.org/api/errors.html#errors_node_js_error_codes

[joshdover]

I see the problems then, I did not realize the static analysis requirement of the localization keys, but that makes sense now.

With the context above, it seems that generic ES error codes may be best. This could allow us to map a generic error code to an application/context-specific translation and give us a lot more flexibility to change these without relying on ES changes. I would expect that error codes may also help us guide the user in what needs to be fixed to continue in many of our features.

I think the deprecations API need for this is a bit different in that it is very specific, and I think using that as a starting place for how to do this for all of ES generally was a mistake on my part. For the time being, we've decided not to translate these deprecation messages and only translate the UI strings for the upgrade assistant.