[Bug]: Map database too large (450MB+ on large monorepos)

[NeelPrime]

Summary

On large monorepos, .arbiterx/map.db grows to 450MB+, causing slow queries, disk waste, and sluggish MCP tool responses.

Expected Size

Repo size	Expected DB	Actual
10K files	50-100MB	—
50K files	~200MB	450MB+

Likely Causes

1. Full docstrings stored — multi-line docstrings stored verbatim for every symbol, creating large text blobs
2. Variable declarations indexed — every x = 5 creates a symbol row. Should only store functions/classes/methods
3. MAX_FILE_SIZE too high — minified/generated JS files (500KB+) getting parsed and all their symbols stored
4. Signatures too verbose — multi-line function signatures stored in full
5. Edges table explosion — every function call in every file = a row. Large repos can have millions
6. No VACUUM after re-index — deleted rows leave dead space in SQLite

Suggested Fixes

High impact:

• Truncate docstrings to 200 chars max
• Skip variable/assignment symbols (only index functions, classes, methods, interfaces, types)
• Lower MAX_FILE_SIZE to 500KB
• Run VACUUM after index_repo completes

Medium impact:

• Truncate signatures to first line only (120 chars)
• Add a --compact flag that stores only names + file locations (no signatures/docstrings)
• Deduplicate edges (same source→target in same file = one row)

Low impact:

• Compress the SQLite file (not worth the query overhead)
• Store symbols in a separate file per language (over-engineering)

Diagnosis Commands

arbiterx map --status
sqlite3 .arbiterx/map.db "SELECT COUNT(*) FROM files; SELECT COUNT(*) FROM symbols; SELECT COUNT(*) FROM edges;"
sqlite3 .arbiterx/map.db "SELECT SUM(LENGTH(docstring)) FROM symbols;"
sqlite3 .arbiterx/map.db "SELECT SUM(LENGTH(signature)) FROM symbols;"
sqlite3 .arbiterx/map.db "SELECT kind, COUNT(*) FROM symbols GROUP BY kind ORDER BY COUNT(*) DESC;"

Target

A 50K-file monorepo should produce a map.db under 100MB while retaining all useful symbol information for queries.