Skip to content

feat(capability-matrix): strict cross-SDK parity score + coverage scope - #63

Merged
grdsdev merged 9 commits into
mainfrom
claude/parity-score-compute-publish-9c1f01
Jul 15, 2026
Merged

feat(capability-matrix): strict cross-SDK parity score + coverage scope#63
grdsdev merged 9 commits into
mainfrom
claude/parity-score-compute-publish-9c1f01

Conversation

@grdsdev

@grdsdev grdsdev commented Jul 15, 2026

Copy link
Copy Markdown
Collaborator

Summary

Implements SDK-995 — compute and publish the first cross-SDK feature parity score.

  • Replaces the old fuzzy average "parity" with the ticket's strict formula: a feature only counts toward overall if it's exactly implemented in every core SDK (javascript, flutter, python, swift). partially_implemented no longer earns credit toward this metric.
  • C#, Go, and Kotlin are community-maintained/in active development — excluded from the strict parity math, but still shown in the matrix table and keep their own perLanguage completion score (informational, unchanged).
  • Adds a new counter-metric, coverage scope: % of "done" (implemented/partial) cells in core SDKs that have a registered symbols list, i.e. how much of what's marked complete has real code evidence attached.
  • Drops the unused per-feature parity column from the site table (perFeature field removed entirely).
  • No new infra — ships on the existing hourly aggregate-capabilities.yml + deploy-pages.yml pipeline, so it goes live on supabase.github.io/sdk automatically.

Design spec: docs/superpowers/specs/2026-07-15-first-parity-score-design.md (local, gitignored — not in this diff).

Verified locally

Built the site against live compliance data fetched from all 6 available SDK repos (Kotlin has no sdk-compliance.yaml yet):

  • Overall parity: 58% (125/215 features implemented across all 4 core SDKs)
  • Coverage scope: 32%
  • Confirmed no leftover .th-parity/.parity-cell in rendered HTML, both new/existing badges render correctly.

Test plan

  • npm test — 182/182 passing
  • npm run typecheck — clean
  • npm run build-site against real fetched compliance data, manually reviewed in browser
  • grep -rn "perFeature" — zero remaining references

grdsdev added 5 commits July 15, 2026 11:09
@grdsdev
feat(capability-matrix): add CORE_LANGUAGES and coverageScope to Pari…
7f5e863 
…tyReport

- Add CORE_LANGUAGES constant and CoreLanguage type for the subset of language SDKs used in parity computation
- Replace perFeature field in ParityReport with coverageScope: number
- Expected: report.ts and generate-site.ts will fail typecheck until fixed in tasks 2 and 3
@grdsdev
feat(capability-matrix): strict cross-SDK parity score + coverage scope
c5fd4c4
@grdsdev
fix(capability-matrix): drop perFeature reintroduced during Task 2
1acade1 
Task 1 removed perFeature from ParityReport per plan; Task 2's implementer
re-added it (in both types.ts and report.ts) to make the whole-project
typecheck pass early, which was never a Task 2 requirement — generate-site.ts
still references parity.perFeature until Task 3 removes that column. Revert
to the plan's exact shape; the remaining generate-site.ts typecheck failure
is expected until Task 3.
@grdsdev
feat(capability-matrix): drop per-feature parity column, add coverage…
a382f4a 
… scope badge

- Remove perFeature parity column from feature table
- Add Coverage scope badge to header
- Remove dead .th-parity and .parity-cell CSS rules
- Reduce table colspan from LANGUAGES.length + 2 to LANGUAGES.length + 1
@grdsdev
chore(capability-matrix): drop dead parity-color CSS and unused CoreL…
263af5a 
…anguage type

Bare .parity-high/mid/low rules were unreachable after the per-feature
column was removed — only the higher-specificity .sdk-score.* and
.area-card-score.* combinators are ever matched. CoreLanguage was exported
but never consumed anywhere.
@grdsdev
grdsdev requested a review from a team as a code owner July 15, 2026 14:49
QuintinWillison
QuintinWillison approved these changes Jul 15, 2026
View reviewed changes
Comment thread scripts/capability-matrix/src/report.ts Outdated
grdsdev added 4 commits July 15, 2026 13:25
@grdsdev
feat(capability-matrix): explain overall parity and coverage scope on…
95cf4da 
… site

Adds a hover tooltip on each badge plus a short note under them, so
"58%"/"32%" don't read as mysterious numbers — states plainly that both
are computed only across the 4 core SDKs and what counts as a pass.
@grdsdev
refactor(capability-matrix): make parity/coverage explanations always…
f3b56ce 
…-visible

Replace the hover-only badge tooltips with metric cards that show the
description text directly under each value — no longer hidden behind a
hover. Header restructured: title row on top, metrics row below it,
then the SDK score cards.
@grdsdev
fix(capability-matrix): coverageScope only counts exactly implemented
6e45061 
partially_implemented was being treated as "done" for coverage scope
via the shared isDone() helper (same as perLanguage's legacy semantics),
but a partial implementation isn't really done — per review feedback on
#63, a symbol-linked partial claim shouldn't score the same as a
symbol-linked full one. perLanguage keeps its existing isDone-based
behavior unchanged; only coverageScope now requires status ===
"implemented".
@grdsdev
fix(capability-matrix): perLanguage no longer credits partially_imple…
5f0bbe7 
…mented

Drops isDone() entirely — every metric (overall, perLanguage, coverageScope)
now requires the exact "implemented" status. Per discussion on #63,
partial credit conflated "claims something is done" with "actually done",
which no longer applies anywhere in this pipeline.
@grdsdev
grdsdev merged commit de1abe1 into main Jul 15, 2026
3 checks passed
@grdsdev
grdsdev deleted the claude/parity-score-compute-publish-9c1f01 branch July 15, 2026 16:46
@grdsdev grdsdev mentioned this pull request Jul 15, 2026
Merged
2 tasks
grdsdev added a commit that referenced this pull request Jul 16, 2026
@grdsdev
fix(capability-matrix): correct coverage scope metric description (#64)
5daeaf5 
The "Coverage scope" card said features "marked implemented/partial"
but the underlying computeParity() logic only counts strictly
`implemented` status, per the strict cross-SDK parity change in #63.
Reword the text to match the actual logic instead of the logic.

Also fix missing leading "%" to match the other metric card's style.
@supabase-releaser supabase-releaser Bot mentioned this pull request Aug 12, 2026
Merged
grdsdev pushed a commit that referenced this pull request Aug 12, 2026
@supabase-releaser
chore(main): release 1.0.0 (#81)
93b5258 
🤖 I have created a release *beep* *boop*
---


## [1.0.0](v1.0.0...v1.0.0)
(2026-08-12)


### ⚠ BREAKING CHANGES

* reconcile capability matrix inconsistencies from skill audit
([#74](#74))

### Features

* add capabilities based on supabase-js public methods
([#19](#19))
([8b07e38](8b07e38))
* add capability-matrix maintenance skill
([#73](#73))
([070e2c9](070e2c9))
* add review-spec and review-spec-compliance skills
([#6](#6))
([b2646c1](b2646c1))
* add SDK implementation status matrix to README and skill
([#4](#4))
([753f06c](753f06c))
* **api-check:** include file path and line number in compliance failure
messages ([#45](#45))
([2f4be47](2f4be47))
* **auth:** add sign-out reason capability
([#47](#47))
([51a3abd](51a3abd))
* canonical SDK capability matrix
([#8](#8))
([215bc3e](215bc3e))
* **capability-matrix:** strict cross-SDK parity score + coverage scope
([#63](#63))
([de1abe1](de1abe1))
* CI check 1 — block PRs adding public API not in capability matrix
([#31](#31))
([293440c](293440c))
* **compliance:** list undeclared features after validation
([#48](#48))
([29f396a](29f396a))
* **compliance:** list undeclared features as notes after validation
([29f396a](29f396a))
* **compliance:** split symbol evidence from symbol coverage
([#75](#75))
([abc8e71](abc8e71))
* initial SDK specs, skills, and install script
([e662b17](e662b17))
* move SDK compliance to per-repo files
([#15](#15))
([4d32675](4d32675))
* **parsers:** add Dart public API parser via package:analyzer
(alternative to [#35](#35))
([#41](#41))
([e3ba07a](e3ba07a))
* **parsers:** add griffe-based Python public API surface parser
([#36](#36))
([c44f836](c44f836))
* **parsers:** replace Swift regex parser with swift-symbolgraph-extract
([#38](#38))
([80529a7](80529a7))
* **realtime:** add multiple postgres_changes filters capability
([#70](#70))
([825ab0c](825ab0c))
* reconcile capability matrix inconsistencies from skill audit
([#74](#74))
([9c53a70](9c53a70))
* rename sdk-parse-ignore to .sdk-parse-ignore
([#37](#37))
([4f4ab61](4f4ab61))
* render symbol names as clickable links in capability matrix
([#14](#14))
([dcaf122](dcaf122))
* show feature description as visible sub-text in capability matrix
([#13](#13))
([3a750ab](3a750ab))
* **site:** serve compliance.json with precomputed parity from GitHub
Pages ([#46](#46))
([0d9106c](0d9106c))
* **storage:** add purge_cache and purge_bucket_cache canonical
capabilities ([#44](#44))
([9a6f864](9a6f864))
* **storage:** add storage.errors.error_codes capability
([#71](#71))
([fabb9a7](fabb9a7))
* **storage:** add the five missing Iceberg catalog capabilities
([#76](#76))
([c3c8f9e](c3c8f9e))


### Bug Fixes

* **aggregate:** point csharp and go SDKs at correct repo slugs
([#62](#62))
([8b7320f](8b7320f))
* **aggregate:** point kotlin SDK at supabase-community/supabase-kt
([#61](#61))
([b1a99e9](b1a99e9))
* **capability-matrix:** correct coverage scope metric description
([#64](#64))
([5daeaf5](5daeaf5))
* **ci:** repair python pipeline and simplify sdk-compliance workflow
([#42](#42))
([2140f44](2140f44))
* **ci:** use nx to run docs:json so workspace deps are built first
([#56](#56))
([9b449bb](9b449bb))
* **dart-extractor:** exclude
[@internal-annotated](https://github.com/internal-annotated) symbols
from the public API surface
([9bd358e](9bd358e))
* **dart-extractor:** exclude [@internal](https://github.com/internal)
symbols from the public API surface
([#54](#54))
([9bd358e](9bd358e))
* remove broken sticky thead, add scroll-margin-top and group-row
borders ([#10](#10))
([68761cd](68761cd))

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).

Co-authored-by: supabase-releaser[bot] <223506987+supabase-releaser[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@QuintinWillison QuintinWillison QuintinWillison approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@grdsdev @QuintinWillison