Closed
Description
Elasticsearch version (bin/elasticsearch --version): 6.3.2
Plugins installed: []
JVM version (java -version): 1.8.0_171
OS version (uname -a if on a Unix-like system): Alpine 3.8
Description of the problem including expected versus actual behavior:
Prerequisites to reproduce the problem:
- a mapping with a double nested structure, where at least two fields are present (in the deepest nested object)
- instances where only one of these two fields appear (so one of the two fields is never present in the index)
- a tophits aggregation where you request the field that never appears as one of the source fields.
The result is a failing shard with a nullpointer as reason. You don't see this problem when you only have one level of nesting or when there are instances that contain both fields.
Steps to reproduce:
- Create the index + mapping:
curl -X PUT "localhost:9200/test?pretty" -H 'Content-Type: application/json' -d'
{
"settings" : {
"number_of_shards" : 1
},
"mappings" : {
"_doc" : {
"properties" : {
"a" : {
"type" : "nested" ,
"include_in_root": true,
"properties": {
"b": {
"type" : "nested" ,
"include_in_root": true,
"properties": {
"c": {
"type": "keyword"
},
"d": {
"type": "keyword"
}
}
}
}
}
}
}
}
}
'
- Add one document (without the 'd' field):
curl -X PUT "localhost:9200/test/_doc/1?pretty&refresh" -H 'Content-Type: application/json' -d'
{
"a" : [
{
"b": [
{
"c": "foo1"
},
{
"c": "foo2"
}
]
}
]
}
'
- Execute the tophits aggregation, where you ask to include the 'd' field
curl -X POST "localhost:9200/test/_search?pretty" -H 'Content-Type: application/json' -d'
{
"aggs":{
"a":{
"nested":{
"path":"a"
},
"aggregations":{
"b":{
"nested":{
"path":"a.b"
},
"aggregations":{
"tophits":{
"top_hits":{
"from":0,
"size":1,
"_source":{
"includes":[
"a.b.d"
],
"excludes":[]
}
}
}
}
}
}
}
}
}
'
The result of the last command is as follows:
{
"error" : {
"root_cause" : [
{
"type" : "null_pointer_exception",
"reason" : null
}
],
"type" : "search_phase_execution_exception",
"reason" : "all shards failed",
"phase" : "query",
"grouped" : true,
"failed_shards" : [
{
"shard" : 0,
"index" : "test",
"node" : "1lzL7jmZS0W3JNkjfe0h5g",
"reason" : {
"type" : "null_pointer_exception",
"reason" : null
}
}
],
"caused_by" : {
"type" : "null_pointer_exception",
"reason" : null,
"caused_by" : {
"type" : "null_pointer_exception",
"reason" : null
}
}
},
"status" : 500
}
Provide logs (if relevant):
Additional logs seen in the server logs:
[2018-08-27T11:20:18,534][WARN ][r.suppressed ] path: /test/_search, params: {pretty=, index=test}
org.elasticsearch.action.search.SearchPhaseExecutionException: all shards failed
at org.elasticsearch.action.search.AbstractSearchAsyncAction.onPhaseFailure(AbstractSearchAsyncAction.java:288) ~[elasticsearch-6.3.2.jar:6.3.2]
at org.elasticsearch.action.search.AbstractSearchAsyncAction.executeNextPhase(AbstractSearchAsyncAction.java:128) ~[elasticsearch-6.3.2.jar:6.3.2]
at org.elasticsearch.action.search.AbstractSearchAsyncAction.onPhaseDone(AbstractSearchAsyncAction.java:249) ~[elasticsearch-6.3.2.jar:6.3.2]
at org.elasticsearch.action.search.InitialSearchPhase.onShardFailure(InitialSearchPhase.java:101) ~[elasticsearch-6.3.2.jar:6.3.2]
at org.elasticsearch.action.search.InitialSearchPhase.access$100(InitialSearchPhase.java:48) ~[elasticsearch-6.3.2.jar:6.3.2]
at org.elasticsearch.action.search.InitialSearchPhase$2.lambda$onFailure$1(InitialSearchPhase.java:222) ~[elasticsearch-6.3.2.jar:6.3.2]
at org.elasticsearch.action.search.InitialSearchPhase.maybeFork(InitialSearchPhase.java:176) [elasticsearch-6.3.2.jar:6.3.2]
at org.elasticsearch.action.search.InitialSearchPhase.access$000(InitialSearchPhase.java:48) [elasticsearch-6.3.2.jar:6.3.2]
at org.elasticsearch.action.search.InitialSearchPhase$2.onFailure(InitialSearchPhase.java:222) [elasticsearch-6.3.2.jar:6.3.2]
at org.elasticsearch.action.search.SearchExecutionStatsCollector.onFailure(SearchExecutionStatsCollector.java:73) [elasticsearch-6.3.2.jar:6.3.2]
at org.elasticsearch.action.ActionListenerResponseHandler.handleException(ActionListenerResponseHandler.java:51) [elasticsearch-6.3.2.jar:6.3.2]
at org.elasticsearch.action.search.SearchTransportService$ConnectionCountingHandler.handleException(SearchTransportService.java:527) [elasticsearch-6.3.2.jar:6.3.2]
at org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleException(TransportService.java:1103) [elasticsearch-6.3.2.jar:6.3.2]
at org.elasticsearch.transport.TransportService$DirectResponseChannel.processException(TransportService.java:1196) [elasticsearch-6.3.2.jar:6.3.2]
at org.elasticsearch.transport.TransportService$DirectResponseChannel.sendResponse(TransportService.java:1180) [elasticsearch-6.3.2.jar:6.3.2]
at org.elasticsearch.transport.TaskTransportChannel.sendResponse(TaskTransportChannel.java:66) [elasticsearch-6.3.2.jar:6.3.2]
at org.elasticsearch.action.search.SearchTransportService$6$1.onFailure(SearchTransportService.java:385) [elasticsearch-6.3.2.jar:6.3.2]
at org.elasticsearch.search.SearchService$2.onFailure(SearchService.java:341) [elasticsearch-6.3.2.jar:6.3.2]
at org.elasticsearch.search.SearchService$2.onResponse(SearchService.java:335) [elasticsearch-6.3.2.jar:6.3.2]
at org.elasticsearch.search.SearchService$2.onResponse(SearchService.java:329) [elasticsearch-6.3.2.jar:6.3.2]
at org.elasticsearch.search.SearchService$3.doRun(SearchService.java:1019) [elasticsearch-6.3.2.jar:6.3.2]
at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:725) [elasticsearch-6.3.2.jar:6.3.2]
at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [elasticsearch-6.3.2.jar:6.3.2]
at org.elasticsearch.common.util.concurrent.TimedRunnable.doRun(TimedRunnable.java:41) [elasticsearch-6.3.2.jar:6.3.2]
at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [elasticsearch-6.3.2.jar:6.3.2]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_171]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_171]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_171]
Caused by: org.elasticsearch.ElasticsearchException$1
at org.elasticsearch.ElasticsearchException.guessRootCauses(ElasticsearchException.java:658) ~[elasticsearch-6.3.2.jar:6.3.2]
at org.elasticsearch.action.search.AbstractSearchAsyncAction.executeNextPhase(AbstractSearchAsyncAction.java:126) ~[elasticsearch-6.3.2.jar:6.3.2]
... 26 more
Caused by: java.lang.NullPointerException
at org.elasticsearch.search.fetch.subphase.FetchSourceSubPhase.getNestedSource(FetchSourceSubPhase.java:83) ~[elasticsearch-6.3.2.jar:6.3.2]
at org.elasticsearch.search.fetch.subphase.FetchSourceSubPhase.hitExecute(FetchSourceSubPhase.java:58) ~[elasticsearch-6.3.2.jar:6.3.2]
at org.elasticsearch.search.fetch.FetchPhase.execute(FetchPhase.java:162) ~[elasticsearch-6.3.2.jar:6.3.2]
at org.elasticsearch.search.aggregations.metrics.tophits.TopHitsAggregator.buildAggregation(TopHitsAggregator.java:168) ~[elasticsearch-6.3.2.jar:6.3.2]
at org.elasticsearch.search.aggregations.bucket.BucketsAggregator.bucketAggregations(BucketsAggregator.java:128) ~[elasticsearch-6.3.2.jar:6.3.2]
at org.elasticsearch.search.aggregations.bucket.nested.NestedAggregator.buildAggregation(NestedAggregator.java:127) ~[elasticsearch-6.3.2.jar:6.3.2]
at org.elasticsearch.search.aggregations.bucket.BucketsAggregator.bucketAggregations(BucketsAggregator.java:128) ~[elasticsearch-6.3.2.jar:6.3.2]
at org.elasticsearch.search.aggregations.bucket.nested.NestedAggregator.buildAggregation(NestedAggregator.java:127) ~[elasticsearch-6.3.2.jar:6.3.2]
at org.elasticsearch.search.aggregations.AggregationPhase.execute(AggregationPhase.java:130) ~[elasticsearch-6.3.2.jar:6.3.2]
at org.elasticsearch.search.query.QueryPhase.execute(QueryPhase.java:113) ~[elasticsearch-6.3.2.jar:6.3.2]
at org.elasticsearch.search.SearchService.loadOrExecuteQueryPhase(SearchService.java:324) ~[elasticsearch-6.3.2.jar:6.3.2]
at org.elasticsearch.search.SearchService.executeQueryPhase(SearchService.java:357) ~[elasticsearch-6.3.2.jar:6.3.2]
at org.elasticsearch.search.SearchService$2.onResponse(SearchService.java:333) ~[elasticsearch-6.3.2.jar:6.3.2]
... 9 more