Repository navigation
fix(backend): discover Ohio SoS artifacts by product label, not file name - #551
Conversation
…name
The file-transfer page lists product labels ("Candidate
Contributions--2026") and never publishes file names, but discovery
searched each listing row for a *.CSV name. Every one of the 17 required
artifacts was therefore reported missing and no refresh could run.
Each product now maps to its exact label on its own list page, and the
cache file name is derived from the matched product, so the download
plan, download loop, and manifest code are unchanged.
Exactness is load-bearing: the NEW tab lists per-committee files whose
labels differ from the statewide annual only by a committee name between
single dashes ("Candidate Contributions-DAVE YOST FOR OHIO-2026" vs
"Candidate Contributions--2026"), so a prefix match would install one
committee's file as the whole state's. Tests pin all 11 label forms
verbatim from the live portal plus that trap.
Verified end to end against the portal: 17/17 artifacts resolved, all 28
Form 31-U detail reports fetched, 189/189 candidates synced.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (3)
📝 WalkthroughWalkthroughOhio SoS artifact discovery now matches exact normalized portal labels to required products and transaction years. The pipeline derives cache filenames from resolved metadata, skips unrelated listings, and records the completed live refresh and synchronization results. ChangesOhio SoS artifact discovery
Estimated code review effort: 3 (Moderate) | ~20 minutes Sequence Diagram(s)sequenceDiagram
participant OhioSoSPortal
participant LabelResolver
participant ArtifactAcquisition
OhioSoSPortal->>LabelResolver: listing cells and list type
LabelResolver->>ArtifactAcquisition: product key and transaction year
ArtifactAcquisition->>ArtifactAcquisition: derive cache filename
Possibly related PRs
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
@coderabbitai full review |
✅ Action performedFull review finished. |
Found during the first attended live run: the Ohio raw-data refresh could not resolve a single artifact.
The bug
listOhioSosPortalFilessearched every listing row for a*.CSVfile name. The file-transfer page doesn't publish file names — it lists product labels:Active Candidate ListCandidate Contributions--2026PAC COVER PAGESResult: 303 rows listed across the four tabs, zero matched, all 17 required artifacts reported missing. The label-discovery path had never completed against the portal — the acquisition spike downloaded through a different route, so PR 4 shipped this untested against live HTML.
The fix
Each product maps to its exact label on its own list page (
ohioSosProductFromListingRow). The cache file name is then derived from the matched product, soplanOhioSosCycleDownloads, the download loop, and the manifest code are untouched.Exactness and tab-scoping are load-bearing. The
NEWtab lists per-committee files whose labels differ from the statewide annual only by a committee name between single dashes:A prefix or substring match would silently install one candidate's file as the entire state's contributions. Matching is full-string, case-insensitive (the portal renders
PAC COVER PAGESin caps), and scoped to the product's own list type. Tests pin all 11 label forms verbatim from the live portal, plus the per-committee trap from both theNEWandCANtabs.Verification (live, attended)
missing_file_names: [], ids matching the spike's recorded map.481068415−$86,455.17,506135809−$418.40) and quarantine per decision 4 instead of publishing.npm run typecheck+npm testgreen (6,362 passed).One data note worth recording:
CAC_CON_2026.CSVdownloaded at 133 MB / 593,131 rows against a listed 91,023 KB. Verified legitimate rather than assumed — every row isRPT_YEAR=2026across 341 distinct committees, and the row count scales with the byte count, so it's a post-filing-deadline batch and the portal's size column is stale.🤖 Generated with Claude Code
Summary by CodeRabbit
Bug Fixes
Documentation