Skip to content

feat(backend): Ohio outside-group funders/industries (PR 8) - #546

Merged
shu1513 merged 2 commits into
mainfrom
claude/ohio-finance-pr8
Aug 6, 2026
Merged

shu1513 merged 2 commits into
mainfrom
claude/ohio-finance-pr8

Conversation

@shu1513

@shu1513 shu1513 commented Aug 6, 2026 •

Copy link
Copy Markdown
Owner

Eighth PR of the Ohio campaign-finance plan (ohio_plan.md, scope item #3): who funds the committees behind a candidate's outside spending — donor + industry breakdowns, wired into the existing committee-label / industry-label manual queues. Matrix row: standard donor/industry outside breakdowns (maryland/tennessee pattern).

What's here

ohioOutsideGroupContributionAggregator.ts — maryland pattern, adapted to Ohio bulk rows:

  • Spender matching by numeric MASTER_KEY (never name); cycle window RPT_YEAR ∈ {Y−1, Y}.
  • Donor vocabulary pinned from the real files: 31-A / 31-E / 31-J-1 count; 31-A-2 (other income) and the party-only 31-CC (restricted fund) are known-excluded; anything else fails closed into an unknown counter. The real 2026 PAC/party files have zero unknowns.
  • Organization donors come from the explicit NON_INDIVIDUAL column — no name heuristics, individuals never become labels.
  • Real-data finding: union PACs put transaction descriptions in NON_INDIVIDUAL ("CONTRIBUTION FROM DUES MONEY" $76K, "TRANSFER OF MEMBERSHIP DUES" $855K). A value is rejected (counted) only when it both starts like a generic description and carries dues/transfer context — entity-bearing values ("OHIO AFL-CIO/DUES MONEY") keep their verbatim label.

Batch sync — after a year's outside aggregation succeeds, one filtered pass over PAC_CON_{Y-1,Y} + PPC_CON_{Y-1,Y} keeps only rows for 31-U spender committees, then slices per candidate. A funder artifact failure only disables this leg (funders null → the writer keeps stored breakdown rows; year summary carries fundersAvailable/fundersError) — the outside totals still sync. Candidate-committee 31-U spenders are documented as not covered (their receipts live in the ~90 MB CAC_CON files; no bundle-covered spender is one on the real 2026 files).

Sync — maryland's industry enrichment verbatim: the aggregator's static industry rows are discarded and rebuilt from merged classification state (rules + cached DB rows + manual verdicts), so a manual industry label always wins, and every unresolved donor persists a classification_source='unknown' row. No AI classifier is constructed by default.

Queues: zero new code. The committee-label due list already reads Ohio outside groups through the merged read path (live since PR 7), and the industry-label due list reads exactly the 'unknown' classification rows this PR persists; OHIO_SOS was already in FINANCE_SUMMARY_SOURCES.

Verification

  • npm run typecheck + npm test green (6,332 passed; +14 new: 11 aggregator, 3 sync/batch).
  • Real-data smoke over the cached 305 MB cycle: 1,290 filtered funder rows → 6 org-donor rows (RYAN B WALKER CPA INC $300 for NFIB; Libertarian Party state fund $19,000 + Butler County LP $394.50), 1,218 individuals skipped, 4 dues descriptions rejected, 62 excluded-form rows, 0 unknown forms.
  • Full non-dry batch run on the local DB (migration 210 applied): 25/25 synced with PR 7's exact direct totals (Acton receipts $15,898,732.10 / cash $8,136,281.98); the stale bundle correctly disabled the outside and funder legs with stored rows preserved.
  • Coverage note: the big super PACs (V-PAC, $8.66M) have no rows in PAC_CON at all — funder coverage is structurally partial and empty is correct (never fabricate).

🤖 Generated with Claude Code

Summary by CodeRabbit

  • New Features

    • Added Ohio outside-group funder aggregation for candidate finance data.
    • Added organizational donor filtering, support/oppose categorization, industry classification, and contribution totals.
    • Added donor breakdown limits and classification handling.
    • Reported funder availability, processing results, and contribution counts.
  • Bug Fixes

    • Prevented invalid, individual, or unmatched contributions from affecting funder totals.
    • Preserved existing breakdowns when funder records are unavailable and removed stale data after empty updates.
  • Documentation

    • Updated the Ohio implementation plan with completed functionality and validation details.

Adds the funder leg of Ohio campaign finance (ohio_plan.md #3):

- ohioOutsideGroupContributionAggregator: maryland pattern (donor +
  industry), MASTER_KEY matching, RPT_YEAR cycle window, pinned donor
  vocabulary (31-A/31-E/31-J-1; 31-A-2 + 31-CC known-excluded, unknowns
  fail closed), organization donors from the explicit NON_INDIVIDUAL
  column, and a pinned non-entity filter for union-PAC dues descriptions.
- batchSync: one filtered pass over PAC_CON/PPC_CON per year for the
  31-U spender committees, sliced per candidate; a funder artifact
  failure only disables this leg (stored breakdowns preserved).
- sync: maryland industry enrichment (rules + cached/manual DB rows, no
  AI classifier by default), classification rows persisted so the manual
  industry-label queue picks unresolved donors up; committee-label queue
  needed no new code.

Real-data smoke over the cached 2026 cycle: 1,290 filtered rows, 0
unknown forms, 6 org donors, 1,218 individuals skipped, 4 dues
descriptions rejected; full local batch run 25/25 with PR 7's exact
direct totals and the stale bundle correctly disabling the outside and
funder legs.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Aug 6, 2026 •

Copy link
Copy Markdown

Review Change Stack

📝 Walkthrough

Walkthrough

The Ohio finance pipeline now ingests PAC and party contributions for outside-group funder aggregation. Candidate sync enriches funder breakdowns with industry classifications, preserves unavailable data, removes stale empty results, and reports processing diagnostics.

Changes

Ohio outside-group funders

Layer / File(s) Summary
Funder aggregation contracts and filtering
backend/src/pipeline/ohioFinance/ohioOutsideGroupContributionAggregator.ts, backend/tests/pipeline/ohioFinance/ohioOutsideGroupContributionAggregator.test.ts, backend/src/pipeline/ohioFinance/index.ts
The aggregator validates inputs, filters eligible organizational contributions, builds donor and industry breakdowns, and reports match statistics. The barrel module exports the aggregator. Tests cover filtering, classifications, thresholds, sorting, and invalid data.
Contribution ingestion and batch integration
backend/src/pipeline/ohioFinance/ohioCandidateFinanceBatchSync.ts, backend/tests/pipeline/ohioFinance/ohioCandidateFinanceBatchSync.test.ts
The batch sync loads PAC and party artifacts, filters rows to 31-U spender committees, runs funder aggregation, and reports availability and contribution-row counts. Tests cover populated and unavailable artifacts.
Classification and snapshot persistence
backend/src/pipeline/ohioFinance/ohioCandidateFinanceSync.ts, backend/tests/pipeline/ohioFinance/ohioCandidateFinanceSync.test.ts, ohio_plan.md
Candidate sync validates classification settings, enriches and merges outside breakdowns, preserves unavailable funder data, deletes stale empty breakdowns, and returns diagnostics. The plan records the completed implementation.

Estimated code review effort: 4 (Complex) | ~45 minutes

Sequence Diagram(s)

sequenceDiagram
  participant OhioSoSArtifacts
  participant BatchSync
  participant FunderAggregator
  participant CandidateSync
  participant SnapshotStore
  OhioSoSArtifacts->>BatchSync: provide PAC and party contribution artifacts
  BatchSync->>FunderAggregator: pass filtered 31-U contribution rows
  FunderAggregator->>BatchSync: return funder breakdowns and row counts
  BatchSync->>CandidateSync: provide outside finance and funder data
  CandidateSync->>SnapshotStore: persist enriched outside breakdowns
Loading

Possibly related PRs

  • shu1513/voteapp#538: Adds the Ohio SoS bulk contribution artifacts used by this funder aggregation flow.
  • shu1513/voteapp#542: Provides the Ohio 31-U outside-spending aggregation integrated with this flow.
  • shu1513/voteapp#545: Introduces the Ohio finance sync functions extended by this change.
🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and concisely describes the Ohio outside-group funder and industry changes delivered by the pull request.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches 💡 1
📝 Generate docstrings 💡
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch claude/ohio-finance-pr8

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

…ping, de-binary the aggregator

- The funder aggregator no longer caps donor rows: the sync layer
  classifies EVERY donor, rebuilds industry totals from all of them, and
  only then caps the persisted donor display rows
  (outsideMaxDonorBreakdownsPerGroup, default 50). The old order rebuilt
  a >50-donor group's industries from just the top 50 donors.
- Replace five literal NUL bytes in the aggregator's key helpers with
  textual escapes - Git treated the file as binary, hiding its whole
  diff from review.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@backend/src/pipeline/ohioFinance/ohioOutsideGroupContributionAggregator.ts`:
- Around line 270-275: Update the row validation condition in the Ohio
outside-group contribution aggregator to reject non-finite and fractional
amountCents values, while preserving the existing positive-value requirement.
Also reject non-finite reportYear values before applying the
cycleStartYear/electionYear bounds so every invalid row increments
unusableRowCount.
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 0f0afdd4-f07a-4b9d-b63c-c8c2c3192642

📥 Commits

Reviewing files that changed from the base of the PR and between 73da493 and 893c950.

📒 Files selected for processing (5)
  • backend/src/pipeline/ohioFinance/ohioCandidateFinanceSync.ts
  • backend/src/pipeline/ohioFinance/ohioOutsideGroupContributionAggregator.ts
  • backend/tests/pipeline/ohioFinance/ohioCandidateFinanceSync.test.ts
  • backend/tests/pipeline/ohioFinance/ohioOutsideGroupContributionAggregator.test.ts
  • ohio_plan.md
🚧 Files skipped from review as they are similar to previous changes (2)
  • backend/tests/pipeline/ohioFinance/ohioOutsideGroupContributionAggregator.test.ts
  • ohio_plan.md

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant