MultiSearcher Filtering with more than one index is broken [LUCENE-958]

[asfimport]

Using filtering on MultiSarcher with more than one index is not working right.

Probable the best way to describe the problem is to give a small example:
Suppose we have 2 Indexes with two documents each:

Index1 {Doc(id:"A", content:".."), Doc(id:"B", content:"..")}
Index2 {Doc(id:"C", content:".."), Doc(id:"D", content:"..")}

When executing a query that returns all the documents and applying a QueryFilter that generated the following BitSet (1, 0, 0, 0), it should filter all the documents but first; the real results are: Doc(id:"A") and Doc(id:"B"), this because the MultiSearcher applies the BitSet filter from the bitset's position 0 for each Index.

The right way is to split the bitset in peaces: Index1 to get the bites form 0-1 and Index2 bites from 2-3.

---

Migrated from LUCENE-958 by Ion Badita, resolved Jul 13 2007

[asfimport]

Yonik Seeley (@yonik) (migrated from JIRA)

Actually, I retract that I think it's broken.
Filter.getBits() will be called for each sub-index, so a new BitSet should be returned specific to each sub-index. Having a Filter implementation that returns the same BitSet for each sub index is incorrect.

[asfimport]

Nicolas Lalevée (migrated from JIRA)

I don't understand the invalid status of this issue. If the filter provided to the multisearcher have to implement itself the document offset handling, then there an API issue : MultiSearcher implements Searchable.

[asfimport]

Yonik Seeley (@yonik) (migrated from JIRA)

Think of a Filter as symbolic rather than concrete (e.g. the set of documents matching the query "inStock:true")
Filter.getBits is passed an IndexReader as an argument, and returns the actual ids (via a BitSet) for that reader only.
Thus, an implementation that returns the same BitSet regardless of what IndexReader it is passed is not going to work correctly.

[asfimport]

Nicolas Lalevée (migrated from JIRA)

I had the same feeling as you Ion because we both generate the filter's bitset outside of the filter itself, assuming that the filter will always be used with the correct IndexReader.

But as pointed by Yonik, the Filter API expect the filter to give a bitset corresponding to provided reader. So the API actually is consistent.