[codex] strengthen event ingestion validation#8
Conversation
|
@codex review
|
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 9dce0d5ea2
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
@codex review
|
|
Codex Review: Didn't find any major issues. What shall we delve into next? ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
Summary
Fixes #3.
This PR keeps scope limited to input validation and error clarity for local file ingestion. It strengthens
load_eventsso the repository's two supported runtime formats,.jsonland.csv, fail more predictably when input is malformed or when required event fields are missing.Before this change, JSONL parsing already rejected invalid JSON lines, but CSV parsing mostly surfaced raw pandas exceptions, and required-field validation only checked whether a column existed anywhere in the frame. That meant a mixed JSONL or CSV file could still pass validation even if an individual record was missing a required field value, because the column existed due to other rows.
The fix does three focused things:
ValueErrorthat includes the file path.jsonland.csvinputs must providetimestamp,event_type,source,target, andstatusfor every row/record, not just somewhere in the tableI also made a one-line README clarification so the documented required fields match the stricter runtime behavior for both supported formats.
Validation
pytest tests/test_io.pypytestNotes
I also attempted
python -m pip install -e .in this environment, but it timed out locally before completion. The repository's existing editable install was already available, and the full test suite still passed against the checked-out code.