New query flag: allow_partial_results with default set to true.

[markharwood]

There are various reasons today why a search could fail to produce all of the required results:

1. A failure to execute within a timeout associated with the search
2. Missing shards (ie a red index)
3. Failed shards - e.g. an error parsing the query

Until now elasticsearch has attempted to to return partial results under these circumstances and report the partial status in flags in the search response e.g. timed_out:true.
Sadly, today we expect most client applications will pay little attention to these flags and present partial results to clients with no indication that the results are incomplete.

This issue proposes we fix this by providing a new cluster level setting called default_allow_partial_results flag which is set to false and will throw an error to the client application in the event of any of the above 3 conditions. This can be overridden in individual search requests using a allow_partial_results flag.

[jpountz]

This makes sense to me, though I'm not sure a cluster setting is required. We could just default to false and expect users who are not happy with this behaviour to override it in the request?

[markharwood]

We could just default to false and expect users who are not happy with this behaviour to override it in the request?

My assumption was that a server-side setting would be useful to those existing installations where you didn't want to rewrite all client applications that were knowingly used to consuming partial results. (Not sure how many of those exist though!)

[markharwood]

We discussed this further and agreed a cluster-level setting would provide administrators with a useful BWC tool for existing apps if we decide to default searches to throwing errors on partial results.
We are still undecided if the default will be changed to throw errors but having a cluster-level setting will help mitigate the impact of such a decision.