The gap
What separates this tool from name-matching heuristics is that it validates against data. That is the thesis of the whole project.
The public corpus cannot exercise it. A published structure.sql has no rows, so make benchmark measures exactly one thing: whether the right candidate was raised. Verdicts, BROKEN, the orphan counts, and the rule that no false positive may ever be confirmed are covered only by integration fixtures — about a dozen scenarios built by hand, by someone who already knew the answer.
The practical consequence is visible in the published numbers. Discourse raises 396 candidates against 12 recoverable keys, 390 of them outside the truth set. The README argues, correctly in principle, that a candidate outside the truth set is not a false positive — a real relationship that was never declared is the product, not an error. But nobody knows what fraction of those 390 would survive validation, because there is nothing to validate against. Today that is reasoning, not measurement.
Proposal
Generate data into the corpus schemas:
- Populate primary keys.
- Populate foreign key columns from the true relationships.
- Inject a controlled percentage of orphans, so
BROKEN has something to find.
Then the benchmark measures verdicts end to end, precision becomes a publishable number, and the zero-confirmed-false-positive rule is exercised at the scale of 1,857 keys instead of a dozen fixtures.
The trap to design around
Independent random fill will overstate precision. A spurious candidate whose columns are filled independently has near-zero containment and is rejected trivially. Real data is not like that: two small-integer status columns, two code columns ranging 1–10, a pair of enums — accidental containment is common, and it is exactly the case where a wrong confirmation would happen.
So the generator has to inject accidental containment on purpose: overlapping value domains between columns that are not related. Without that, the precision number will be beautiful and false, which is worse than not having it.
Why this first
It is the largest measurable gap in the project, and it unblocks the two-pass bootstrap (#2 in the roadmap), which cannot be verified against a corpus with no rows.
The gap
What separates this tool from name-matching heuristics is that it validates against data. That is the thesis of the whole project.
The public corpus cannot exercise it. A published
structure.sqlhas no rows, somake benchmarkmeasures exactly one thing: whether the right candidate was raised. Verdicts,BROKEN, the orphan counts, and the rule that no false positive may ever be confirmed are covered only by integration fixtures — about a dozen scenarios built by hand, by someone who already knew the answer.The practical consequence is visible in the published numbers. Discourse raises 396 candidates against 12 recoverable keys, 390 of them outside the truth set. The README argues, correctly in principle, that a candidate outside the truth set is not a false positive — a real relationship that was never declared is the product, not an error. But nobody knows what fraction of those 390 would survive validation, because there is nothing to validate against. Today that is reasoning, not measurement.
Proposal
Generate data into the corpus schemas:
BROKENhas something to find.Then the benchmark measures verdicts end to end, precision becomes a publishable number, and the zero-confirmed-false-positive rule is exercised at the scale of 1,857 keys instead of a dozen fixtures.
The trap to design around
Independent random fill will overstate precision. A spurious candidate whose columns are filled independently has near-zero containment and is rejected trivially. Real data is not like that: two small-integer status columns, two code columns ranging 1–10, a pair of enums — accidental containment is common, and it is exactly the case where a wrong confirmation would happen.
So the generator has to inject accidental containment on purpose: overlapping value domains between columns that are not related. Without that, the precision number will be beautiful and false, which is worse than not having it.
Why this first
It is the largest measurable gap in the project, and it unblocks the two-pass bootstrap (#2 in the roadmap), which cannot be verified against a corpus with no rows.