-
Notifications
You must be signed in to change notification settings - Fork 132
Open
Labels
bugSomething isn't workingSomething isn't working
Description
AnnIVFFlat
test configuration 21 with { 1000, 10000, 2051, 16, 40, 1024, distance::InnerProduct, 1,0,0}
shows very low recall. This test becomes flaky depending on the architecture used and tests start to fail with recall values below the threshold. We believe this is mathematical property of the test configuration and propose to skip this test until we find a better parameter set.
[ RUN ] AnnIVFFlatTest/AnnIVFFlatTestF_half.AnnIVFFlat/21
[ 331][14:12:46:144510][info ] Recall = 0.030875 (494/16000), the error is 0.9% above the threshold (eps = 0.005000).
/home/coder/cuvs/cpp/tests/neighbors/ann_ivf_flat/../ann_ivf_flat.cuh:260: Failure
Value of: eval_neighbours(indices_naive, indices_ivfflat, distances_naive, distances_ivfflat, ps.num_queries, ps.k, eps, min_recall)
Actual: false (actual recall (0.030875) is lower than the minimum expected recall (0.0390625); eps = 0.004999999888241291. )
Expected: true
[ 331][14:12:47:605797][info ] Recall = 0.071750 (1148/16000), the error is 3.4% below the threshold (eps = 0.005000).
[ FAILED ] AnnIVFFlatTest/AnnIVFFlatTestF_half.AnnIVFFlat/21, where GetParam() = { 1000, 10000, 2051, 16, 40, 1024, distance::InnerProduct, 1,0,0}
(1920 ms)
Metadata
Metadata
Assignees
Labels
bugSomething isn't workingSomething isn't working
Type
Projects
Status
Todo