Implement the vchord / pgvector support

[qdrddr]

Implemented the vchord + pgvector support plan from code_update_plan.md. Here are the changes made:

1. Added _detect_vector_extension() helper function in hindsight-api/hindsight_api/migrations.py (lines 36-65)

• Detects available vector extension: vchord (preferred) or pgvector
• Raises RuntimeError if neither extension is found
• Includes debug logging for the detected extension

2. Updated ensure_embedding_dimension() function in migrations.py

• Line 397-398: Added extension detection after engine creation
• Lines 447-464: Updated index drop logic to handle both HNSW and vchordrq indexes
• Lines 472-492: Updated index creation to conditionally create:
• vchordrq index with vector_l2_ops for vchord
• hnsw index with vector_cosine_ops for pgvector
• Added debug logging for which index type was created

3. Updated 5a366d414dce_initial_schema.py migration (lines 203-228)

• Added inline extension detection
• Conditional index creation for the memory_units table:
• vchordrq for vchord
• HNSW for pgvector

4. Updated n9i0j1k2l3m4_learnings_and_pinned_reflections.py migration

• Added from sqlalchemy import text import
• Added _detect_vector_extension() helper function (lines 31-48)
• Added extension detection in upgrade() function (line 56)
• Conditional index creation for learning table (lines 85-95)
• Conditional index creation for pinned_reflections table (lines 131-141)

Key Features:

• Backward compatible - Existing pgvector deployments work unchanged
• Automatic detection - Detects vchord first, falls back to pgvector
• High-dimensional support - vchord supports embeddings 65K dimensions (e.g., text-embedding-3-large with 3072 dims)
• Debug logging - Logs which extension is detected on startup
• Minimal changes - Only index creation/dropping differs between extensions
• No model changes - Vector type works with both extensions

All code passes linting checks and has no syntax errors.

[nicoloboschi]

Should use detect extension method?

[qdrddr]

I just wanted to preserve backward compatibility with PGvector for whatever reason in case you want to have it (for example, for licensing reasons):

• When PGVector -> use HNSW
• When VectorChord -> use vchordrq index

If you want to completely switch from PGVector to VectorChord, then no need for extention detection.

[qdrddr]

Added _detect_vector_extension() function @nicoloboschi