Summary
After calling compact(), get(key) returns another key's vector. No removals are required —
compact() on a freshly built index with zero deletions is enough to break retrieval for every key
sampled.
The graph itself survives (search self-recall stays ~99%), so this is specifically the key→vector
retrieval path: slot reordering performed by compact() is not reflected in the lookup that get()
resolves through.
compact() is not exposed on the Python Index wrapper, but it is bound on the compiled module
(python/lib.cpp:1480, i.def("compact", &compact_index<index_at>, ...)), so it is reachable as
index._compiled.compact(threads=...). That makes it an attractive-looking trap for anyone who goes
looking for a way to repair a degraded graph.
Reproduction
Pure usearch, stock MetricKind.Hamming, zero removals:
"""compact() leaves get() returning another key's vector."""
import numpy as np
from usearch.index import Index, MetricKind, ScalarKind
N = 2_000
rng = np.random.default_rng(7)
vecs = rng.integers(0, 256, size=(N, 32), dtype=np.uint8)
keys = np.arange(1, N + 1, dtype=np.uint64)
owner = {vecs[i].tobytes(): i + 1 for i in range(N)}
index = Index(ndim=256, metric=MetricKind.Hamming, dtype=ScalarKind.B1)
index.add(keys, vecs)
def report(label):
print(label)
for k in (1, 2, 3, 500, 1000, 2000):
got = np.asarray(index.get(int(k)), dtype=np.uint8).ravel()
print(f" get({k:>4}) returns the vector of key {owner.get(got.tobytes(), '<unknown>')}")
report("before compact():")
index._compiled.compact(threads=0) # not exposed on the Index wrapper
report("\nafter compact():")
hits = sum(
int(np.atleast_1d(index.search(vecs[k - 1], 1).keys).ravel()[0]) == k for k in range(1, 201)
)
print(f"\nsearch self-recall after compact(): {hits}/200 (graph itself is fine)")
print(f"index size: {len(index)}")
Output:
before compact():
get( 1) returns the vector of key 1
get( 2) returns the vector of key 2
get( 3) returns the vector of key 3
get( 500) returns the vector of key 500
get(1000) returns the vector of key 1000
get(2000) returns the vector of key 2000
after compact():
get( 1) returns the vector of key 52
get( 2) returns the vector of key 302
get( 3) returns the vector of key 1802
get( 500) returns the vector of key 1992
get(1000) returns the vector of key 111
get(2000) returns the vector of key 1541
search self-recall after compact(): 197/200 (graph itself is fine)
index size: 2000
Every sampled key returns a wrong vector. len(index) and contains() remain correct, so the
breakage is silent — a caller that uses get() to read vectors back gets plausible-looking but
wrong data with no error.
Suspected mechanism
index_dense_gt::compact() reorders nodes and rewrites neighbor links to match the new slot
assignments (and updates entry_slot_). The key→slot mapping used by get() appears not to be
rebuilt with the same track_slot_change remapping, so keys resolve to stale slot numbers that now
hold different vectors.
Suggested resolution
Either fix the remapping so compact() keeps get() consistent and then expose it on the Python
Index wrapper, or — if it is known-incomplete — remove/guard the _compiled.compact binding so it
cannot be reached by accident.
A working compact() matters beyond this bug: it is the natural repair for the graph-reachability
defect in the companion issue below.
Environment
- usearch-iscc 2.24.5 (reports
usearch.__version__ == 2.24.0)
- Python 3.12.0, Windows 10 x86_64
Companion issue
Graph reachability lost after remove()+add() churn — see #8
Downstream tracking issue: iscc/iscc-usearch#30
Summary
After calling
compact(),get(key)returns another key's vector. No removals are required —compact()on a freshly built index with zero deletions is enough to break retrieval for every keysampled.
The graph itself survives (search self-recall stays ~99%), so this is specifically the key→vector
retrieval path: slot reordering performed by
compact()is not reflected in the lookup thatget()resolves through.
compact()is not exposed on the PythonIndexwrapper, but it is bound on the compiled module(
python/lib.cpp:1480,i.def("compact", &compact_index<index_at>, ...)), so it is reachable asindex._compiled.compact(threads=...). That makes it an attractive-looking trap for anyone who goeslooking for a way to repair a degraded graph.
Reproduction
Pure usearch, stock
MetricKind.Hamming, zero removals:Output:
Every sampled key returns a wrong vector.
len(index)andcontains()remain correct, so thebreakage is silent — a caller that uses
get()to read vectors back gets plausible-looking butwrong data with no error.
Suspected mechanism
index_dense_gt::compact()reorders nodes and rewrites neighbor links to match the new slotassignments (and updates
entry_slot_). The key→slot mapping used byget()appears not to berebuilt with the same
track_slot_changeremapping, so keys resolve to stale slot numbers that nowhold different vectors.
Suggested resolution
Either fix the remapping so
compact()keepsget()consistent and then expose it on the PythonIndexwrapper, or — if it is known-incomplete — remove/guard the_compiled.compactbinding so itcannot be reached by accident.
A working
compact()matters beyond this bug: it is the natural repair for the graph-reachabilitydefect in the companion issue below.
Environment
usearch.__version__ == 2.24.0)Companion issue
Graph reachability lost after remove()+add() churn — see #8
Downstream tracking issue: iscc/iscc-usearch#30