Repository navigation
feat(backend): Ohio finance PR 7 — sync, batchSync, scheduler, npm scripts - #545
Merged
Merged
Conversation
…ripts
Per-candidate sync (ohioCandidateFinanceSync.ts) is a write step that takes
aggregation results rather than raw rows: the batch layer owns the single
stream over the ~90 MB CAC_CON files (plan decision 10). The committee comes
from the due row's active link; outsideFinance null writes NULL outside
totals (writer preserveWhenNull) and leaves stored outside groups untouched,
while an empty list clears stale rows.
Batch sync (ohioCandidateFinanceBatchSync.ts): shared due-list factory over
the oh_ tables; auto-link phase reuses the PR 5 walker with the cached
active-candidate list; per election year one direct accumulator per distinct
linked committee fed by one CAC_CON_{Y-1,Y} pass, cover files loaded once
per run, and one outside-spending aggregation over (name, office)-deduped
targets so a candidate due for both primary and general cannot quarantine
their own rows as ambiguous. Missing direct artifacts fail the year's rows;
missing outside artifacts only disable the outside leg, surfaced in a
per-year availability summary.
readOhioSos31uDetailBundle accepts both the version-1 payload and the
2026-08-04 spike checkpoint format, keeping the existing 305 MB artifact
cache usable without another attended Chrome session.
BullMQ scheduler (queue ohio_candidate_finance_sync_maintenance, daily cron
default 55 9 * * * UTC) plus sync-due / scheduler:upsert / scheduler:worker
/ scheduler:trigger scripts and the ohio-candidates:finance npm block,
maryland as template.
Verified: typecheck + full backend suite green (6,289 tests; 17 new), and an
end-to-end dry run over the real cached cycle reproduces the PR 6 smoke
exactly - Acton oppose $8,211,114.50, Ramaswamy support $600,000 with cover
receipts $55,081,342.57 beating the $30.3M itemized sum, Stephens oppose
$217,704.16; 13 reports reconciled, 0 quarantined.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
Warning Review limit reached
Next review available in: 50 minutes You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (4)
📝 WalkthroughWalkthroughAdded Ohio candidate finance synchronization with cached artifact parsing, direct and outside-spending aggregation, snapshot persistence, due-row batch processing, BullMQ scheduling, worker execution, CLI commands, and comprehensive tests. ChangesOhio finance synchronization
Estimated code review effort: 4 (Complex) | ~60 minutes Sequence Diagram(s)sequenceDiagram
participant Operator
participant SchedulerCLI
participant BullMQ
participant OhioFinanceWorker
participant OhioCandidateFinanceBatchSync
participant PostgreSQL
Operator->>SchedulerCLI: configure or trigger finance sync
SchedulerCLI->>BullMQ: upsert or enqueue job
BullMQ->>OhioFinanceWorker: deliver finance job
OhioFinanceWorker->>OhioCandidateFinanceBatchSync: run due-row synchronization
OhioCandidateFinanceBatchSync->>PostgreSQL: read due rows and write snapshots
PostgreSQL-->>OhioCandidateFinanceBatchSync: return synchronization results
OhioCandidateFinanceBatchSync-->>OhioFinanceWorker: return job result
Possibly related PRs
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches 💡 1📝 Generate docstrings 💡
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Owner
Author
|
@coderabbitai full review |
✅ Action performedFull review finished. |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
🧹 Nitpick comments (5)
backend/src/pipeline/ohioFinance/ohioSosArtifactAcquisition.ts (1)
502-507: 🗄️ Data Integrity & Integration | 🔵 Trivial | ⚡ Quick winThe version-1 branch casts row contents without validating them.
The header comment states the reader is fail-closed and that any shape surprise throws. The version-1 branch checks only that
reportKeyis a string androwsis an array. It then casts the elements withreport.rows as OhioSos31uDetailRow[]. A damaged or hand-edited cache entry, for exampleamountCentsstored as"1000"instead of a number, passes this check.
aggregateOhioOutsideSpendingthen evaluatesrow.amountCents <= 0and adds the value to the report total. A string value would produce wrong reconciliation totals instead of an error, which contradicts the fail-closed policy. The legacy branch does not have this gap, becauseparseOhioSos31uDetailTablevalidates every cell.Consider validating the numeric and direction fields per row before returning them.
♻️ Suggested per-row validation
return payload.reports.map((report) => { if (typeof report.reportKey !== "string" || !Array.isArray(report.rows)) { throw new Error(`Ohio SoS 31-U detail bundle at ${detailPath} has a malformed report entry`); } - return { reportKey: report.reportKey, rows: report.rows as OhioSos31uDetailRow[] }; + const rows = report.rows.map((row, index) => { + const candidate = row as Partial<OhioSos31uDetailRow>; + if ( + typeof candidate.reportKey !== "string" || + (candidate.amountCents !== null && typeof candidate.amountCents !== "number") || + (candidate.year !== null && typeof candidate.year !== "number") + ) { + throw new Error( + `Ohio SoS 31-U detail bundle at ${detailPath} report ${report.reportKey} row ${index + 1} is malformed` + ); + } + return candidate as OhioSos31uDetailRow; + }); + return { reportKey: report.reportKey, rows }; });🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@backend/src/pipeline/ohioFinance/ohioSosArtifactAcquisition.ts` around lines 502 - 507, Update the version-1 report handling around the report mapping to validate every row’s numeric and direction fields before returning it, rather than relying on the OhioSos31uDetailRow cast. Reject malformed values, including numeric fields stored as strings, by throwing an error that preserves the fail-closed behavior; keep valid rows unchanged and continue returning the validated reportKey and rows.backend/tests/pipeline/ohioFinance/ohioSosArtifactAcquisition.test.ts (1)
382-396: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winAdd coverage for the malformed-entry throws.
The tests cover the happy paths, the cycle mismatch, and the unrecognized shape. They do not cover the two malformed-entry throws in
readOhioSos31uDetailBundle(source lines 504 and 513). They also do not pin the behavior for a payload that declaresversion: 1but omitsreports; that payload reaches the "unrecognized format" throw rather than a malformed-entry error.✅ Suggested added cases
it("rejects a version-1 report entry without a string reportKey", async () => { const cacheDir = await makeCacheDir(); await writeBundle(cacheDir, { version: 1, cycleYear: 2026, reports: [{ reportKey: 100, rows: [] }] }); await expect(readOhioSos31uDetailBundle({ cacheDir, cycleYear: 2026 })).rejects.toThrow( /malformed report entry/ ); }); it("rejects a legacy entry without a string key", async () => { const cacheDir = await makeCacheDir(); await writeBundle(cacheDir, { rows: [{ key: 512315395, rows: [] }] }); await expect(readOhioSos31uDetailBundle({ cacheDir, cycleYear: 2026 })).rejects.toThrow( /malformed legacy report entry/ ); }); it("rejects a version-1 payload without a reports array", async () => { const cacheDir = await makeCacheDir(); await writeBundle(cacheDir, { version: 1, cycleYear: 2026 }); await expect(readOhioSos31uDetailBundle({ cacheDir, cycleYear: 2026 })).rejects.toThrow( /unrecognized format/ ); });🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@backend/tests/pipeline/ohioFinance/ohioSosArtifactAcquisition.test.ts` around lines 382 - 396, Add tests in the existing ohioSos31uDetailBundle suite for both malformed-entry branches: a version-1 report with a non-string reportKey must reject with “malformed report entry,” and a legacy entry with a non-string key must reject with “malformed legacy report entry.” Also cover a version-1 payload missing its reports array and assert it rejects with “unrecognized format,” preserving the documented classification.backend/src/pipeline/ohioFinance/ohioCandidateFinanceBatchSync.ts (2)
333-333: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winThe broad catch reports data-corruption errors as unavailable artifacts.
The try block spans the artifact reads and the aggregation call.
aggregateOhioOutsideSpendingthrows on conditions that are not missing files, for exampleDuplicate Ohio 31-U detail report key: ...and a target without acandidateKey. The catch reports every such failure asOhio SoS outside-spending artifacts unavailable, setsavailable: false, and lets the run preserve stale outside totals.The
errorfield still carries the real message, so the failure is observable. The log line misattributes the cause, which slows diagnosis of upstream corruption.Consider limiting the try to the artifact loads at lines 334-355 and letting aggregation errors surface with their own handling, or reword the warning so it does not assert a cause it has not established.
Also applies to: 407-417
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@backend/src/pipeline/ohioFinance/ohioCandidateFinanceBatchSync.ts` at line 333, Limit the try/catch around the artifact-loading operations in the Ohio candidate finance sync, such as the loads near the artifact read block, so only missing or unavailable artifacts produce the unavailable warning and available:false result. Let aggregation errors from aggregateOhioOutsideSpending, including duplicate keys or missing candidateKey failures, propagate through their existing handling without being relabeled as artifact unavailability; apply the same correction to the corresponding catch block near the later sync path.
267-294: 🚀 Performance & Scalability | 🔵 Trivial | ⚡ Quick winKey each direct contribution row by its master key before aggregating.
aggregateDirectForYearcallsaccumulator.add(row)for every committee accumulator on each contribution row beforeaddfilters byrow.masterKey.trim() !== committeeId. BuildMappertransactionYearwithinput.committeeIdsonce, collect it, then stream bothCAC_CONfiles and route rows with a keyed accumulator lookup. Add an emptycommitteeIds/accumulators guard to skip both files when no committee ids are linked for the year.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@backend/src/pipeline/ohioFinance/ohioCandidateFinanceBatchSync.ts` around lines 267 - 294, The contribution sync in aggregateDirectForYear currently scans every row through every accumulator; build a per-transactionYear Map keyed by trimmed master key from input.committeeIds, return early when no committee IDs/accumulators exist, and route each streamed CAC_CON row only to the accumulator matching its trimmed row.masterKey instead of iterating accumulators.values().backend/src/scheduler/ohioCandidateFinanceSyncScheduler.ts (1)
135-149: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winConsider not persisting
forceinto the recurring scheduler payload.
forceis stored in the repeatable job data. Every later daily run then callsisOhioCampaignFinanceSyncEnabled(true)and bypassesOHIO_CAMPAIGN_FINANCE_SYNC_ENABLED(seebackend/src/config/featureFlags.tslines 242-247). An operator who runsohio-candidates:finance:scheduler:upsert -- --forceonce therefore disables the sync kill switch for the daily schedule until the next upsert. The state is not visible from the flag env vars.If
forceis intended only as a one-shot operator override, restrict it to the manual enqueue path and drop it from the recurring payload.♻️ Proposed change to keep the kill switch effective for daily runs
data: { dryRun: Boolean(jobData.dryRun), - force: Boolean(jobData.force), maxCandidates: jobData.maxCandidates,🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@backend/src/scheduler/ohioCandidateFinanceSyncScheduler.ts` around lines 135 - 149, Remove force from the recurring job data created by the scheduler upsert flow around OHIO_CANDIDATE_FINANCE_SYNC_JOB_NAME, so repeatable daily runs do not persist a one-shot override. Keep force available only in the manual enqueue path, while preserving the other scheduler payload fields unchanged.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@backend/src/pipeline/ohioFinance/ohioSosArtifactAcquisition.ts`:
- Around line 489-494: Update aggregateOutsideForYear and the cached
detail-bundle parsing flow to avoid readFile/JSON.parse loading the entire 31-U
artifact into memory; use streaming or chunked parsing while preserving the
existing report and row data behavior. Ensure row parsing caps legacy raw-cell
replication, and avoid retaining parsed report collections alongside stacked
rows and cover rows.
In `@backend/src/scripts/triggerOhioCandidateFinanceSync.ts`:
- Around line 9-30: Extract parseFlagValue and parsePositiveIntegerFlag into a
shared module imported by syncDueOhioCandidateFinance.ts,
triggerOhioCandidateFinanceSync.ts, and
upsertOhioCandidateFinanceSyncScheduler.ts. Replace the local parsers in
triggerOhioCandidateFinanceSync.ts (lines 9-30) and
upsertOhioCandidateFinanceSyncScheduler.ts (lines 10-31), passing each script’s
known boolean and value flag names; retain validateKnownFlags and duplicate-flag
rejection in the shared implementation.
---
Nitpick comments:
In `@backend/src/pipeline/ohioFinance/ohioCandidateFinanceBatchSync.ts`:
- Line 333: Limit the try/catch around the artifact-loading operations in the
Ohio candidate finance sync, such as the loads near the artifact read block, so
only missing or unavailable artifacts produce the unavailable warning and
available:false result. Let aggregation errors from
aggregateOhioOutsideSpending, including duplicate keys or missing candidateKey
failures, propagate through their existing handling without being relabeled as
artifact unavailability; apply the same correction to the corresponding catch
block near the later sync path.
- Around line 267-294: The contribution sync in aggregateDirectForYear currently
scans every row through every accumulator; build a per-transactionYear Map keyed
by trimmed master key from input.committeeIds, return early when no committee
IDs/accumulators exist, and route each streamed CAC_CON row only to the
accumulator matching its trimmed row.masterKey instead of iterating
accumulators.values().
In `@backend/src/pipeline/ohioFinance/ohioSosArtifactAcquisition.ts`:
- Around line 502-507: Update the version-1 report handling around the report
mapping to validate every row’s numeric and direction fields before returning
it, rather than relying on the OhioSos31uDetailRow cast. Reject malformed
values, including numeric fields stored as strings, by throwing an error that
preserves the fail-closed behavior; keep valid rows unchanged and continue
returning the validated reportKey and rows.
In `@backend/src/scheduler/ohioCandidateFinanceSyncScheduler.ts`:
- Around line 135-149: Remove force from the recurring job data created by the
scheduler upsert flow around OHIO_CANDIDATE_FINANCE_SYNC_JOB_NAME, so repeatable
daily runs do not persist a one-shot override. Keep force available only in the
manual enqueue path, while preserving the other scheduler payload fields
unchanged.
In `@backend/tests/pipeline/ohioFinance/ohioSosArtifactAcquisition.test.ts`:
- Around line 382-396: Add tests in the existing ohioSos31uDetailBundle suite
for both malformed-entry branches: a version-1 report with a non-string
reportKey must reject with “malformed report entry,” and a legacy entry with a
non-string key must reject with “malformed legacy report entry.” Also cover a
version-1 payload missing its reports array and assert it rejects with
“unrecognized format,” preserving the documented classification.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro Plus
Run ID: b2582fc9-d268-4814-8ae8-37578bf79c6d
📒 Files selected for processing (14)
backend/package.jsonbackend/src/pipeline/ohioFinance/index.tsbackend/src/pipeline/ohioFinance/ohioCandidateFinanceBatchSync.tsbackend/src/pipeline/ohioFinance/ohioCandidateFinanceSync.tsbackend/src/pipeline/ohioFinance/ohioSosArtifactAcquisition.tsbackend/src/scheduler/ohioCandidateFinanceSyncScheduler.tsbackend/src/scripts/runOhioCandidateFinanceSyncSchedulerWorker.tsbackend/src/scripts/syncDueOhioCandidateFinance.tsbackend/src/scripts/triggerOhioCandidateFinanceSync.tsbackend/src/scripts/upsertOhioCandidateFinanceSyncScheduler.tsbackend/tests/pipeline/ohioFinance/ohioCandidateFinanceBatchSync.test.tsbackend/tests/pipeline/ohioFinance/ohioCandidateFinanceSync.test.tsbackend/tests/pipeline/ohioFinance/ohioSosArtifactAcquisition.test.tsohio_plan.md
…rgets, fail-closed reader, cover-IE zero Four fixes from the Codex/CodeRabbit review of PR #545: - A 31-U detail bundle missing report keys the annual files discovered now fails the year closed to outside-unavailable instead of publishing partial totals and zeros. The real cache's spike bundle misses 15 keys carrying ~$5.9M of annual 31-U money, so this was production-relevant. - The outside target dedupe keys on candidateId + office, not display name: one person due for primary and general still shares a target, while two different people with the same name stay separate and the aggregator quarantines that name as ambiguous instead of paying both. - The version-1 bundle reader validates every field the aggregator consumes (direction pinned to support/oppose/null, integer cents, matching report keys) instead of casting - a bogus direction would have landed in the aggregator's oppose branch. - Cover-IE gate (leftover from the #542 review): a blank VALUE_IND_EXPENDITURES on an otherwise-filled cover row now counts as zero and feeds the third reconciliation leg; fully-blank rows stay ignored like missing ones (shared isBlankOhioSosCoverPageRow). A probe of all 28 cycle report keys found 13 exact IE matches, 15 no-cover-row, and 0 blank-on-filled, so this is added protection with no behavior change on the real files. Also: direct contribution rows route to their accumulator by MASTER_KEY instead of the rows x committees scan, and the outside catch warning no longer claims "artifacts unavailable" for what may be an aggregation failure. Declined from review: streaming JSON for the bundle (9.8 KB file - the 305 MB figure is the whole cache), shared CLI flag parser, and dropping force from the recurring scheduler payload (both verbatim the maryland/all-states template; repo-wide questions). Verified: typecheck green, 6,299 tests pass (10 new), and the real-data dry run now reports outside unavailable on the stale spike bundle with direct totals unchanged. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…, provenance, strict flags All four Codex findings fixed: - Cache integrity: every cached stream passes a manifest gate first. A file whose size no longer matches its manifest (torn copy, outside edit, truncation on a row boundary the parser cannot see) fails the year instead of publishing from it. A file with no manifest streams with a warning - there is nothing to verify against, and the extant spike cache carries no manifests; a strict ready-only gate would brick it while detecting nothing more. - Same-name ambiguity guard: the outside target universe is now the year's FULL set of active oh_candidate_finance_links (new per-year query), not the stale-filtered, 25-row due page. A same-name double is visible to the aggregator even when not due in the same run, so attribution no longer depends on sync timing. Unlinked same-name candidates remain the residual blind spot; auto-link runs first. - Link provenance: link_source rides in the due row (the established alaska/tennessee/virginia linkColumns pattern) and the sync writes it back unchanged - a manual link stays manual instead of being rewritten as sos_bulk_export, which would have lost provenance and exposed the manual pin to auto-link supersession. - CLI safety: trigger and scheduler-upsert scripts now validate known flags and reject duplicates exactly like sync-due, so a typo such as --dryrun fails loudly instead of silently enqueueing a real write. Verified: typecheck green, 6,303 tests pass (4 new), real-data dry run unchanged (direct totals identical; manifest-less cache warns per file; outside still honestly unavailable on the stale spike bundle). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@backend/src/scripts/triggerOhioCandidateFinanceSync.ts`:
- Around line 21-30: Update validateKnownFlags to explicitly reject or correctly
parse inline assignments for the boolean flags in KNOWN_BOOLEAN_FLAGS,
especially --dry-run and --force; ensure downstream checks such as
args.includes("--dry-run") cannot treat an assigned boolean form as a real sync.
Preserve existing validation for value flags and standalone boolean flags.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro Plus
Run ID: 56093dc4-0d9f-427a-9f6e-8f837d94b2df
📒 Files selected for processing (13)
backend/src/pipeline/ohioFinance/ohioCandidateFinanceBatchSync.tsbackend/src/pipeline/ohioFinance/ohioCandidateFinanceSync.tsbackend/src/pipeline/ohioFinance/ohioDirectContributionAggregator.tsbackend/src/pipeline/ohioFinance/ohioOutsideSpendingAggregator.tsbackend/src/pipeline/ohioFinance/ohioSosArtifactAcquisition.tsbackend/src/pipeline/ohioFinance/ohioSosBulkFiles.tsbackend/src/scripts/triggerOhioCandidateFinanceSync.tsbackend/src/scripts/upsertOhioCandidateFinanceSyncScheduler.tsbackend/tests/pipeline/ohioFinance/ohioCandidateFinanceBatchSync.test.tsbackend/tests/pipeline/ohioFinance/ohioCandidateFinanceSync.test.tsbackend/tests/pipeline/ohioFinance/ohioOutsideSpendingAggregator.test.tsbackend/tests/pipeline/ohioFinance/ohioSosArtifactAcquisition.test.tsohio_plan.md
🚧 Files skipped from review as they are similar to previous changes (6)
- backend/tests/pipeline/ohioFinance/ohioCandidateFinanceSync.test.ts
- backend/src/pipeline/ohioFinance/ohioCandidateFinanceSync.ts
- backend/src/pipeline/ohioFinance/ohioSosArtifactAcquisition.ts
- ohio_plan.md
- backend/src/scripts/upsertOhioCandidateFinanceSyncScheduler.ts
- backend/src/pipeline/ohioFinance/ohioCandidateFinanceBatchSync.ts
CodeRabbit round 3, one valid finding (the other two comments were stale
duplicates of already-addressed rounds): --dry-run=true passed
validateKnownFlags (the name before "=" matches the boolean set) but the
later args.includes("--dry-run") check returned false, so the command
would run or enqueue a REAL sync instead of a dry run. Boolean flags now
reject any inline "=value" form in all three scripts (sync-due, trigger,
scheduler-upsert), and a shared parameterized test file pins the strict
contract: unknown flag, boolean-with-value, repeated flag, and
non-positive integer all throw.
Verified: typecheck green, 6,318 tests pass (15 new).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Ohio campaign-finance PR 7 of 9 (
ohio_plan.md): sync + batchSync + scheduler + npm scripts, maryland block as template.What's here
ohioCandidateFinanceSync.ts— per-candidate write step. Takes aggregation results, not raw rows: the batch layer owns the single stream over the ~90 MB CAC_CON files (decision 10). Committee identity is the due row's active link (the due list only returns linked candidates, so no resolver call).outsideFinance: nullwrites NULL outside totals — the writer'spreserveWhenNullpolicy keeps stored values — and leaves outside-group rows untouched; an empty groups list clears stale rows.ohioCandidateFinanceBatchSync.ts— due list via the shared factory over theoh_tables; auto-link phase reuses the PR 5 walker with the cached active-candidate list; per election year one direct accumulator per distinct linked committee fed by oneCAC_CON_{Y-1,Y}pass, cover files loaded once per run, and one outside-spending aggregation over (candidate name, office)-deduped targets — without the dedupe, a candidate due for both primary and general would appear twice and quarantine every row aimed at them as ambiguous.outsideAggregationByYearsummary for the PR 9 live-run report.readOhioSos31uDetailBundle— reads the 31-U detail bundle fail-closed, accepting both the version-1 payload PR 4 writes and the 2026-08-04 spike checkpoint format, so the existing 305 MB cache works without another attended Chrome session.ohio_candidate_finance_sync_maintenance, daily cron default55 9 * * *UTC),sync-due/scheduler:upsert/scheduler:worker/scheduler:triggerscripts, and theohio-candidates:finance:*npm block.Verification
npm run typecheck+npm testgreen inbackend/(6,289 tests; 17 new: 6 sync, 7 batchSync, 4 bundle reader).🤖 Generated with Claude Code
Summary by CodeRabbit
New Features
Documentation