normalize messages from sub-LLM calls to prevent errors#664
Conversation
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
| # Check if content is a nested message dict (has 'role' and 'content' keys) | ||
| # This happens when model passes message dicts to llm_batch instead of strings | ||
| if "role" in content and "content" in content: | ||
| msg_copy["content"] = content["content"] |
There was a problem hiding this comment.
Nested content extraction skips further normalization checks
When extracting inner content from a nested message dict (one with both role and content keys), the extracted content["content"] is assigned directly without further normalization. If the inner content is itself a malformed dict (e.g., a content part object with type key, or another nested message), it won't be wrapped in an array or recursively normalized. This means the final content could still be an invalid bare dict, violating the stated invariant that the API expects content to be a string, array of objects, or None.
There was a problem hiding this comment.
Seems overly defensive, I've never seen that happen. Also, maybe the models just shouldn't nest too deep, and failing on this edgecase is fine.
2 participants
Description
The sub-LLMs can receive more than just strings from the RLM. This PR introduces a normalization step of these messages, so that sub-LLMs work in all valid cases, which they didn't before.
Type of Change
Testing
uv run pytestlocally.Checklist
Additional Notes
Note
Improves robustness of sub-LLM chat calls by sanitizing message content formats.
_normalize_message_content()inverifiers/envs/experimental/rlm_env.pyto coercecontentinto API-accepted forms (extract nested{role, content}, wrap{type: ...}content-part objects, fallback wrap unknown dicts)_call_sub_llm_api()instead of rawmessagesto avoid malformed payload errorsWritten by Cursor Bugbot for commit c5b18a8. This will update automatically on new commits. Configure here.