Skip to content

fix: make critic response parsing None-safe#219

Open
zjw49246 wants to merge 1 commit into
llmsresearch:mainfrom
zjw49246:fix/critic-none-safe-parse
Open

fix: make critic response parsing None-safe#219
zjw49246 wants to merge 1 commit into
llmsresearch:mainfrom
zjw49246:fix/critic-none-safe-parse

Conversation

@zjw49246
Copy link
Copy Markdown

What does this PR do?

When a VLM provider returns None (e.g. Gemini empty response, OpenRouter refusal with content: null), extract_json() crashes with AttributeError: 'NoneType' object has no attribute 'strip' and CriticAgent._parse_response() propagates the crash. This PR adds None guards at both layers so the critic degrades gracefully to an empty CritiqueResult instead of crashing.

This is defense-in-depth for #40 — complements the provider-level fix in #55 and the pipeline-level fallback in #146 by handling the case at the parsing layer itself.

Type of change

  • Bug fix
  • New feature
  • Reference dataset addition
  • Documentation update
  • Refactor (no functional change)
  • New provider support

Changes made

  • paperbanana/core/utils.py: Added early if not text: return None guard in extract_json(), updated type hint to str | None
  • paperbanana/agents/critic.py: Added explicit None check at the top of _parse_response(), updated type hint to str | None
  • tests/test_agents/test_critic_parse.py (new): 8 tests covering extract_json (None, empty, valid, truncated) and CriticAgent._parse_response (None, empty, valid JSON, no-suggestions, truncated JSON)

How to test

  1. pytest tests/test_agents/test_critic_parse.py -v — all 8 tests pass
  2. Specifically test_none_response_returns_empty_critique verifies that _parse_response(None) returns a CritiqueResult with needs_revision=False instead of crashing

Checklist

  • pytest tests/ -v passes
  • ruff check paperbanana/ mcp_server/ tests/ scripts/ passes
  • I've added/updated tests for new functionality (if applicable)
  • I've updated documentation (if applicable)
  • For reference dataset additions: verified methodology text matches diagram, aspect ratio is within [1.5, 2.5], metadata.json is complete

When Gemini returns an empty response (response.text is None),
extract_json() crashed with AttributeError on text.strip(). The critic
agent now handles None gracefully, returning an empty CritiqueResult
instead of raising.

This is defense-in-depth for the thinking model issue (llmsresearch#40) — even if
the RuntimeError from gemini.py is caught by tenacity retry and all
retries fail, the critic won't crash with an unhelpful AttributeError.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Copy link
Copy Markdown
Member

@dippatel1994 dippatel1994 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Clean fix. None-guard added at both the call site (CriticAgent._parse_response) and the underlying utility (extract_json), so any other caller that hits a None or empty VLM response gets the safe path too. Type hints updated to str | None on both, which makes the contract explicit.

Test coverage is exactly what I'd ask for: None, empty string, valid object, valid array, truncated JSON — at both layers. The "truncated JSON returns empty critique" test is particularly nice because it matches a real failure mode from #40.

Approving — this directly addresses the silent revision-loop bypass described in issue #40 and we've seen similar reports from #213.

One small nit (non-blocking, fine in a follow-up): the logger.warning("Critic received None response from VLM") line would be more useful with structured fields (e.g. vlm_provider=self.vlm_provider.name) so it's filterable in production logs. Not a merge blocker.

Thanks for the contribution.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants