Cast exception error while using the get api

[cristi-]

Elasticsearch version (bin/elasticsearch --version):

7.6.0

Plugins installed: [analysis-icu analysis-kuramoji es-restlog es-shopname-tokenizer repository-gcs]

JVM version (java -version):
openjdk version "13.0.2" 2020-01-14

OS version (uname -a if on a Unix-like system):
Linux elastic2-es-data-14 4.14.138+ #1 SMP Tue Sep 3 02:58:08 PDT 2019 x86_64 x86_64 x86_64 GNU/Linux

Description of the problem including expected versus actual behavior:

From time to time, while retrieving documents using the GET Api from a index that receives no search requests, we see the following exception:

Exception in thread "main" NotSerializableExceptionWrapper[class_cast_exception: class java.lang.String cannot be cast to class java.lang.Number (java.lang.String and java.lang.Number are in module java.base of loader 'bootstrap')]
	at org.elasticsearch.index.mapper.NumberFieldMapper$NumberFieldType.valueForDisplay(NumberFieldMapper.java:966)
	at org.elasticsearch.index.fieldvisitor.FieldsVisitor.postProcess(FieldsVisitor.java:104)
	at org.elasticsearch.index.get.ShardGetService.innerGetLoadFromStoredFields(ShardGetService.java:284)
	at org.elasticsearch.index.get.ShardGetService.innerGet(ShardGetService.java:194)
	at org.elasticsearch.index.get.ShardGetService.get(ShardGetService.java:104)
	at org.elasticsearch.index.get.ShardGetService.get(ShardGetService.java:95)
	at org.elasticsearch.action.get.TransportGetAction.shardOperation(TransportGetAction.java:106)
	at org.elasticsearch.action.get.TransportGetAction.shardOperation(TransportGetAction.java:45)
	at org.elasticsearch.action.support.single.shard.TransportSingleShardAction.lambda$asyncShardOperation$0(TransportSingleShardAction.java:110)
	at org.elasticsearch.action.ActionRunnable.lambda$supply$0(ActionRunnable.java:58)
	at org.elasticsearch.action.ActionRunnable$2.doRun(ActionRunnable.java:73)
	at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:692)
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.lang.Thread.run(Thread.java:830)

The index has the following mapping:

{ - 
  "index-timestamps-1589491666": { - 
    "mappings": { - 
      "properties": { - 
        "timestamp": { - 
          "type": "date",
          "store": true,
          "format": "dateOptionalTime"
        },
        "value": { - 
          "type": "long",
          "store": true
        }
      }
    }
  }
}

The changes added in #48843 used the following logic to parse data from translog files: https://github.com/elastic/elasticsearch/pull/48843/files#diff-04af9607316b0935c611e5ebfd5e7c61R256-R265

However in https://github.com/apache/lucene-solr/blob/a6a02ac0e5116f4cf061fd3afeb930c3cc379f5a/lucene/core/src/java/org/apache/lucene/document/Field.java#L252-L264 stringValue() will return the fieldsData.toString(); even for numeric values.

This string value reaches valueForDisplay() where the exception occurs:

Steps to reproduce:

Unfortunately I don't have a easy way to reproduce this right now.

Provide logs (if relevant):

[elasticmachine]

Pinging @elastic/es-search (:Search/Search)

[elasticmachine]

Pinging @elastic/es-distributed (:Distributed/Engine)

[jtibshirani]

Thank you @cristi- for the detailed report. Since you've already dug in and identified the likely cause (which is great), would you be interested in submitting a PR? No worries if not, I can submit a fix soon.

[cristi-]

@jtibshirani thank you for the suggestion, I can try opening a PR to fix this. I was thinking of checking numeric types first and then strings as the final check. Would that be the right direction? I'll try to add a test too.