You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
CRITICAL: random-100-match-kw-small-vocab declares vector_count 100 for a 1M corpus — shared-corpus mode uploads 100 docs and reports success #224
The two small-vocab entries point at a 1,000,000-point corpus (random_keywords_1m_vocab_10) while declaring vector_count: 100. Their siblings on the same corpus family declare 1000000 correctly. This looks like a copy-paste from the genuine random-100 entry directly above.
vector_count is not decorative.src/bin/vector_db_benchmark/engine/redis.rs:1901-1922 uses it as the shared-corpus completeness gate:
ifletSome(expected) = dataset.config.vector_count.filter(|&n| n > 0){
With shared-corpus mode enabled (redis.rs:249, driven by shared_key_prefix()), the upload is skipped as soon as existing >= expected — i.e. after 100 keys — and prints "Shared corpus already populated … skipping re-upload". The sweep then measures recall over ~100 documents against 1M-point ground truth. Recall collapses to ~0 with no error surfaced.
Fix: set both to 1000000. Then add a guard so this class cannot recur — e.g. validate vector_count against the actual corpus on first download, or fail loudly when the completeness gate is satisfied by an implausibly small fraction of a declared corpus.
Two more vector_count values are wrong in the benign (over-declaring) direction, measured from the on-disk NPY headers — worth fixing in the same PR since --list-datasets reports them:
arxiv-titles-384-angular-*: declared 2205995, actual (2138591, 384)
h-and-m-2048-angular-*: declared 105542, actual (105100, 2048)
Activity
fix(datasets): never trust a declared count because a directory happe…
fix(skip-upload): probe before fetching, measure sparse instead of tr…
test(qdrant): cover the multivector vector_count cross-check (round 5)
feat(msmarco): add uniformly-sampled (crc32) twins of the three variants
feat(datasets): add MS MARCO v2.1 + Cohere embed-v3 (vectors + real d…