Idea
Support running embeddinggemma-300m at a truncated MRL dimension (512/256/128) to shrink the embedding cache and speed up cosine scans.
Current state
- No support anywhere: no
truncate_dim at model construction or encode, no config knob, and the SentenceTransformer kwargs are assembled internally with no pass-through, so it cannot be injected.
- Both built-in profiles run at full 768 dims; every calibration manifest records
dimension: 768.
- Existing dimension machinery is integrity-only and already acts as a safety net:
_validate_embedding_matrix rejects mismatched dims, the embedding cache discards hits whose dimensionality cannot match, and shard loads detect a dimension change outright — a truncated run would cold-start the cache rather than mix vector spaces.
Constraints (why this is a calibrated addition, not a flag)
- Model-gated:
embeddinggemma-300m is MRL-trained (768 → 512/256/128 officially supported); gte-modernbert-base is not, so truncating it is unsupported by the checkpoint and must stay impossible.
- Truncation shifts the cosine scale, so every per-language duplicate gate and search default measured at 768 is invalid at a truncated dim. Shipping this requires a full
scripts/sweep_semantic_thresholds.py recalibration per supported dim, with reports/manifests recording the dim they were swept at (dimension is already in the manifest).
- The truncated dim must join the embedding-space identity / cache key, and
_uncalibrated_gate_reasons must treat a non-default dim like a custom prompt or alternate task: no calibrated gate applies, explicit thresholds required.
Payoff assessment
Truncation does not speed up inference — the full model still runs; only the kept output dims shrink. The wins are cache footprint (~3x smaller at 256) and pair-scan memory/speed, which are minor at typical corpus sizes where model inference dominates. Not worth building until cache size is a real complaint; recorded here so the design constraints are not rediscovered.
Idea
Support running
embeddinggemma-300mat a truncated MRL dimension (512/256/128) to shrink the embedding cache and speed up cosine scans.Current state
truncate_dimat model construction or encode, no config knob, and theSentenceTransformerkwargs are assembled internally with no pass-through, so it cannot be injected.dimension: 768._validate_embedding_matrixrejects mismatched dims, the embedding cache discards hits whose dimensionality cannot match, and shard loads detect a dimension change outright — a truncated run would cold-start the cache rather than mix vector spaces.Constraints (why this is a calibrated addition, not a flag)
embeddinggemma-300mis MRL-trained (768 → 512/256/128 officially supported);gte-modernbert-baseis not, so truncating it is unsupported by the checkpoint and must stay impossible.scripts/sweep_semantic_thresholds.pyrecalibration per supported dim, with reports/manifests recording the dim they were swept at (dimensionis already in the manifest)._uncalibrated_gate_reasonsmust treat a non-default dim like a custom prompt or alternate task: no calibrated gate applies, explicit thresholds required.Payoff assessment
Truncation does not speed up inference — the full model still runs; only the kept output dims shrink. The wins are cache footprint (~3x smaller at 256) and pair-scan memory/speed, which are minor at typical corpus sizes where model inference dominates. Not worth building until cache size is a real complaint; recorded here so the design constraints are not rediscovered.