NullPointerException with intervals query

[stefanobranco]

Elasticsearch version: 7.0.0

Plugins installed: None

JVM version: 1.8.0_171

OS version: Windows Server 2016

Description of the problem including expected versus actual behavior:

Elastic/Lucene throws a NullPointerException when an interval query is being executed while one of the matches is a stopword (stopword filter configured for the field).

Steps to reproduce:

1. Create a new index with a language analyzer active for a specific field:

PUT interval_test
{
  "mappings": {
    "properties": {
      "content": {
        "type": "text",
        "analyzer": "english"
      }
    }
  }
}

2. Index a new document containing text with at least one stopword

PUT interval_test/_doc/1
{
  "content": "the pioneer"
}

3. Execute an intervals query where one of the match queries is a stopword

POST interval_test/_search
{
  "query": {
    "intervals" : {
      "content" : {
        "all_of" : {
          "intervals" : [
            {
              "all_of" : {
                "intervals" : [
                  { "match" : { "query" : "the" } },
                  { "match" : { "query" : "pioneer" } }
                ]
              }
            }
          ]
        }
      }
    }
  }
}

Provide logs (if relevant):

Caused by: java.lang.NullPointerException
	at org.apache.lucene.search.intervals.ConjunctionIntervalsSource.intervals(ConjunctionIntervalsSource.java:78) ~[lucene-sandbox-8.0.0.jar:8.0.0 2ae4746365c1ee72a0047ced7610b2096e438979 - jimczi - 2019-03-08 12:00:11]
	at org.apache.lucene.search.intervals.IntervalQuery$IntervalWeight.scorer(IntervalQuery.java:184) ~[lucene-sandbox-8.0.0.jar:8.0.0 2ae4746365c1ee72a0047ced7610b2096e438979 - jimczi - 2019-03-08 12:00:11]
	at org.apache.lucene.search.Weight.scorerSupplier(Weight.java:143) ~[lucene-core-8.0.0.jar:8.0.0 2ae4746365c1ee72a0047ced7610b2096e438979 - jimczi - 2019-03-08 11:58:55]
	at org.apache.lucene.search.BooleanWeight.scorerSupplier(BooleanWeight.java:374) ~[lucene-core-8.0.0.jar:8.0.0 2ae4746365c1ee72a0047ced7610b2096e438979 - jimczi - 2019-03-08 11:58:55]
	at org.apache.lucene.search.BooleanWeight.scorerSupplier(BooleanWeight.java:374) ~[lucene-core-8.0.0.jar:8.0.0 2ae4746365c1ee72a0047ced7610b2096e438979 - jimczi - 2019-03-08 11:58:55]
	at org.apache.lucene.search.BooleanWeight.scorer(BooleanWeight.java:340) ~[lucene-core-8.0.0.jar:8.0.0 2ae4746365c1ee72a0047ced7610b2096e438979 - jimczi - 2019-03-08 11:58:55]
	at org.elasticsearch.common.lucene.search.function.FunctionScoreQuery$CustomBoostFactorWeight.functionScorer(FunctionScoreQuery.java:260) ~[elasticsearch-7.0.0.jar:7.0.0]
	at org.elasticsearch.common.lucene.search.function.FunctionScoreQuery$CustomBoostFactorWeight.scorer(FunctionScoreQuery.java:282) ~[elasticsearch-7.0.0.jar:7.0.0]
	at org.apache.lucene.search.Weight.scorerSupplier(Weight.java:143) ~[lucene-core-8.0.0.jar:8.0.0 2ae4746365c1ee72a0047ced7610b2096e438979 - jimczi - 2019-03-08 11:58:55]
	at org.apache.lucene.search.BooleanWeight.scorerSupplier(BooleanWeight.java:374) ~[lucene-core-8.0.0.jar:8.0.0 2ae4746365c1ee72a0047ced7610b2096e438979 - jimczi - 2019-03-08 11:58:55]
	at org.apache.lucene.search.BooleanWeight.scorer(BooleanWeight.java:340) ~[lucene-core-8.0.0.jar:8.0.0 2ae4746365c1ee72a0047ced7610b2096e438979 - jimczi - 2019-03-08 11:58:55]
	at org.apache.lucene.search.Weight.bulkScorer(Weight.java:177) ~[lucene-core-8.0.0.jar:8.0.0 2ae4746365c1ee72a0047ced7610b2096e438979 - jimczi - 2019-03-08 11:58:55]
	at org.apache.lucene.search.BooleanWeight.bulkScorer(BooleanWeight.java:334) ~[lucene-core-8.0.0.jar:8.0.0 2ae4746365c1ee72a0047ced7610b2096e438979 - jimczi - 2019-03-08 11:58:55]
	at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:649) ~[lucene-core-8.0.0.jar:8.0.0 2ae4746365c1ee72a0047ced7610b2096e438979 - jimczi - 2019-03-08 11:58:55]
	at org.elasticsearch.search.internal.ContextIndexSearcher.search(ContextIndexSearcher.java:177) ~[elasticsearch-7.0.0.jar:7.0.0]
	at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:443) ~[lucene-core-8.0.0.jar:8.0.0 2ae4746365c1ee72a0047ced7610b2096e438979 - jimczi - 2019-03-08 11:58:55]
	at org.elasticsearch.search.query.QueryPhase.execute(QueryPhase.java:275) ~[elasticsearch-7.0.0.jar:7.0.0]
	at org.elasticsearch.search.query.QueryPhase.execute(QueryPhase.java:115) ~[elasticsearch-7.0.0.jar:7.0.0]
	at org.elasticsearch.search.SearchService.loadOrExecuteQueryPhase(SearchService.java:349) ~[elasticsearch-7.0.0.jar:7.0.0]
	at org.elasticsearch.search.SearchService.executeQueryPhase(SearchService.java:393) ~[elasticsearch-7.0.0.jar:7.0.0]
	at org.elasticsearch.search.SearchService.access$100(SearchService.java:124) ~[elasticsearch-7.0.0.jar:7.0.0]
	at org.elasticsearch.search.SearchService$2.onResponse(SearchService.java:358) ~[elasticsearch-7.0.0.jar:7.0.0]

[elasticmachine]

Pinging @elastic/es-search

[romseygeek]

This is a bug in IntervalBuilder#analyzeText, which is returning null instead of NO_INTERVALS. I'm not quite sure if NO_INTERVALS is the correct thing to return here though, as this may lead to some unexpected behaviour - for example, the query in the issue description would end up returning no documents, but an equivalent query asking for a match on the pioneer would return docs, as the stopword gap gets reworked into a widening of the non-stopword interval. Maybe the best response is actually to throw an exception if a match query ends up with no tokens after analysis? What do you think @jimczi?

[jimczi]

I think it's ok if the query in the issue description ends up returning no documents. We have the same problematic with the normal match query which returns a MatchNoDocsQuery by default when all terms are removed from the analysis. However we also have an option to ignore the query completely:
https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-match-query.html#query-dsl-match-query-zero
Would it be difficult to add this option in the match interval query ? I also wonder if we could improve the documentation to ensure that users don't split their match queries like in the issue description. I guess that this is the legacy of the non-analyzed span queries so we should make it clear (if it's not already the case) that block of texts that must appear together should not be splitted.

[romseygeek]

Would it be difficult to add this option in the match interval query ?

We could add something like ignore_empty to the all_of interval builder, but I think documentation is probably the way to go. I'll open a PR to fix the immediate bug, and work on the docs as well.