Skip to content

Fix SimpleOutput annotation cleanup for failing contains items#602

Open
staging-devin-ai-integration[bot] wants to merge 1 commit intomainfrom
evalon/blaze-the--8df94c68
Open

Fix SimpleOutput annotation cleanup for failing contains items#602
staging-devin-ai-integration[bot] wants to merge 1 commit intomainfrom
evalon/blaze-the--8df94c68

Conversation

@staging-devin-ai-integration
Copy link

Fix SimpleOutput cleanup for failing contains item annotations

Summary

This PR fixes a SimpleOutput annotation retention bug for contains subschema failures.

Previously, when a contains item failed (and the failure was masked from output), SimpleOutput could keep annotations (e.g. title) for that failing item because cleanup was skipped before the early return. The fix adds targeted cleanup for masked contains failures and scopes that cleanup using both:

  • evaluation path context, and
  • instance location prefix

so annotations from failing items are dropped without removing valid annotations from matching items or the root contains annotation.

Also adds a regression test for:

  • schema with contains: { "type": "number", "title": "Test" }
  • instance [ "foo", 42, true ]

and verifies only the root /contains annotation and the /1 title annotation remain.

Review & Testing Checklist for Human

  • Verify the new masked-failure cleanup logic in SimpleOutput is correct for contains and does not unintentionally change behavior for other masked keywords (anyOf/oneOf/not/if).
  • Sanity-check the starts_with_initial + size() + instance_location.starts_with(...) combination to confirm it removes only sibling annotations for the failing item, not parent/root annotations.
  • Run the JSON Schema CLI reproduction from the issue and confirm annotations for /0 and /2 are no longer emitted while /1 is retained.

Recommended human test plan:

  1. Run the new unit test and surrounding compiler_output_simple tests.
  2. Run the CLI example from the report with --verbose.
  3. Spot-check an existing contains annotation case (including unevaluatedItems interactions) to ensure no regressions.

Notes

Co-Authored-By: bot_apk <apk@cognition.ai>
@staging-devin-ai-integration
Copy link
Author

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment and CI monitoring

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

0 participants