Skip to content

Conversation

@maxkahan
Copy link
Contributor

@maxkahan maxkahan commented Nov 27, 2025

Fixes instruction following issue

Summary by CodeRabbit

  • New Features

    • Dynamic model selection with personality-aware assistant behavior.
    • Structured instruction objects for more consistent LLM responses.
  • Configuration Changes

    • Turn-detection parameters updated to use pre-speech buffer and speech probability thresholds.
  • Improvements

    • Better conversation-history normalization for more coherent context.
    • Call lifecycle now relies on agent flow (no immediate canned join response).
  • Bug Fixes

    • Removed small post-turn STT cleanup/delay that affected transcript finalization.

✏️ Tip: You can customize this high-level summary in your review settings.

@coderabbitai
Copy link

coderabbitai bot commented Nov 27, 2025

Warning

Rate limit exceeded

@maxkahan has exceeded the limit for the number of commits or files that can be reviewed per hour. Please wait 3 minutes and 51 seconds before requesting another review.

⌛ How to resolve this issue?

After the wait time has elapsed, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout.

Please see our FAQ for further information.

📥 Commits

Reviewing files that changed from the base of the PR and between 5163248 and e4d6b32.

📒 Files selected for processing (2)
  • plugins/elevenlabs/example/elevenlabs_example.py (0 hunks)
  • plugins/openrouter/vision_agents/plugins/openrouter/openrouter_llm.py (2 hunks)

Walkthrough

Refactors OpenRouter LLM to accept structured Instructions, injects those instructions as a system message, rebuilds conversation history before responses, updates example to use model-driven personality mapping and new turn-detection params, and removes a small post-turn STT cleanup and micro-delay; other edits are formatting/test adjustments.

Changes

Cohort / File(s) Summary
OpenRouter example
plugins/openrouter/example/openrouter_example.py
Adds dynamic model→personality mapping, instantiates LLM with chosen model, builds multi-line instructions including personality and a six-word constraint, replaces turn-detection params (buffer_in_seconds/confidence_thresholdpre_speech_buffer_ms/speech_probability_threshold), and removes an explicit simple_response in join_call.
OpenRouter tests
plugins/openrouter/tests/test_openrouter_llm.py
Test now wraps instruction text in an Instructions object before calling LLM.set_instructions, removes skip, and expands prompt text used in simple_response.
OpenRouter LLM implementation
plugins/openrouter/vision_agents/plugins/openrouter/openrouter_llm.py
Adds _instructions: str field and overrides create_response to prepend instructions as a system message (normalizes input to list, temporarily clears _instructions to avoid duplication). Reworks add_conversation_history to construct a normalized messages list from conversation history plus new input and persist normalized messages into conversation state.
STT / agent minor edits & formatting
agents-core/vision_agents/core/agents/agents.py, agents-core/vision_agents/core/stt/stt.py, plugins/deepgram/vision_agents/plugins/deepgram/deepgram_stt.py, plugins/elevenlabs/vision_agents/plugins/elevenlabs/stt.py, tests/test_transcript_buffer.py, plugins/elevenlabs/tests/test_elevenlabs_stt.py
Removed a post-turn STT buffer clear + 0.02s delay from TurnEndedEvent handling; other changes are reformatting/argument reflow or minor test formatting (no behavior change).
Project metadata
pyproject.toml
Removed trailing blank line after [tool.hatch.version] block (formatting only).

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20–25 minutes

  • Inspect create_response override for correct input normalization, temporary _instructions clearing/restoration, and no instruction duplication across branches.
  • Verify add_conversation_history preserves message order, role/content mapping, and updates in-memory conversation consistently.
  • Check example model→personality mapping and instruction formatting for correctness and edge cases (unknown models).
  • Confirm removal of STT post-turn clear/delay doesn't break downstream turn-finalization logic or tests that assume that timing.

Possibly related PRs

Suggested reviewers

  • d3xvn
  • tschellenbach

Poem

The system-message settles like a thin black veil,
Instructions stitched in the dark between the names,
A personality named and pinned like a moth,
Conversation bones laid out and seamed,
Silence loses its small tidy ritual, and waits.

Pre-merge checks and finishing touches

❌ Failed checks (1 warning, 1 inconclusive)
Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 69.23% which is insufficient. The required threshold is 80.00%. You can run @coderabbitai generate docstrings to improve docstring coverage.
Title check ❓ Inconclusive The title 'Fix openrouter' is vague and does not clearly convey the main change; it lacks specificity about what is being fixed. Consider using a more descriptive title such as 'Fix OpenRouter instruction following by updating LLM set_instructions to accept Instructions object' to better communicate the primary change.
✅ Passed checks (1 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🧹 Nitpick comments (1)
plugins/openrouter/example/openrouter_example.py (1)

50-54: Consider stripping leading whitespace from the instructions.

The multi-line f-string includes indentation whitespace that will be passed to the LLM. While most LLMs handle this gracefully, cleaner formatting is preferable.

-        instructions=f"""
-        Answer in 6 words or less.
-        {personality}
-        """,
+        instructions=f"Answer in 6 words or less. {personality}",

Or use textwrap.dedent() if you prefer the multi-line format for readability.

📜 Review details

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Disabled knowledge base sources:

  • Linear integration is disabled by default for public repositories

You can enable these sources in your CodeRabbit configuration.

📥 Commits

Reviewing files that changed from the base of the PR and between 74b64ed and a77e13d.

📒 Files selected for processing (4)
  • plugins/openrouter/example/openrouter_example.py (1 hunks)
  • plugins/openrouter/tests/test_openrouter_llm.py (2 hunks)
  • plugins/openrouter/vision_agents/plugins/openrouter/openrouter_llm.py (1 hunks)
  • pyproject.toml (0 hunks)
💤 Files with no reviewable changes (1)
  • pyproject.toml
🧰 Additional context used
📓 Path-based instructions (2)
**/*.py

📄 CodeRabbit inference engine (.cursor/rules/python.mdc)

**/*.py: Never adjust sys.path in Python code
Never write except Exception as e - use specific exception handling
Avoid using getattr, hasattr, delattr and setattr; prefer normal attribute access in Python
Docstrings should follow the Google style guide for docstrings

Files:

  • plugins/openrouter/vision_agents/plugins/openrouter/openrouter_llm.py
  • plugins/openrouter/example/openrouter_example.py
  • plugins/openrouter/tests/test_openrouter_llm.py
**/*test*.py

📄 CodeRabbit inference engine (.cursor/rules/python.mdc)

**/*test*.py: Never mock in tests; use pytest for testing
Mark integration tests with @pytest.mark.integration decorator
@pytest.mark.asyncio is not needed - it is automatic

Files:

  • plugins/openrouter/tests/test_openrouter_llm.py
🧬 Code graph analysis (2)
plugins/openrouter/vision_agents/plugins/openrouter/openrouter_llm.py (1)
plugins/openai/vision_agents/plugins/openai/openai_llm.py (2)
  • create_response (112-205)
  • add_conversation_history (108-110)
plugins/openrouter/tests/test_openrouter_llm.py (2)
agents-core/vision_agents/core/instructions.py (1)
  • Instructions (15-115)
agents-core/vision_agents/core/llm/llm.py (1)
  • set_instructions (179-193)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (6)
  • GitHub Check: unit / Test "not integration"
  • GitHub Check: unit / Mypy
  • GitHub Check: unit / Ruff
  • GitHub Check: unit / Mypy
  • GitHub Check: unit / Ruff
  • GitHub Check: unit / Test "not integration"
🔇 Additional comments (4)
plugins/openrouter/vision_agents/plugins/openrouter/openrouter_llm.py (1)

48-77: LGTM! Clean workaround for OpenRouter's API constraints.

The override correctly handles the absence of an instructions parameter by converting it to a system message. The try/finally block ensures _instructions is always restored, even if the parent call raises an exception.

plugins/openrouter/tests/test_openrouter_llm.py (1)

9-9: LGTM! Test correctly adopts Instructions class.

The test now properly uses the Instructions wrapper for system instructions, aligning with the structured instruction handling introduced in the codebase. The extended prompt description provides clearer context for the expected "NL" response.

Also applies to: 146-146, 149-149

plugins/openrouter/example/openrouter_example.py (2)

57-59: Parameter names are correct and align with the SmartTurnDetection API.

The parameters pre_speech_buffer_ms and speech_probability_threshold match the actual SmartTurnDetection.__init__ signature. The values (2000ms pre-speech buffer and 0.9 probability threshold) are reasonable and higher than the defaults (200ms and 0.5 respectively).


33-45: Code is correct; model identifier is valid.

The "google/gemini-2.5-flash" model is confirmed to exist in the OpenRouter API. The personality mapping logic on lines 33–45 is clear, readable, and follows Python conventions with no coding guideline violations. No changes required.

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

♻️ Duplicate comments (1)
plugins/openrouter/vision_agents/plugins/openrouter/openrouter_llm.py (1)

90-113: Fix conversation-history role corruption and remove hasattr per guidelines

When rebuilding messages from self._conversation.messages, the fallback branch still hardcodes "role": "user" for non-dict originals. This mislabels assistant/system turns as user messages and corrupts context. The checks also continue to use hasattr, which conflicts with the project’s Python guidelines.

You can both preserve roles and avoid hasattr by caching _conversation via normal attribute access in a try/except and extracting m.role when available:

-        # Build the message list
-        messages = []
-
-        if hasattr(self, "_conversation") and self._conversation:
-            # Extract serializable message content from conversation history
-            for m in self._conversation.messages:
-                if isinstance(m.original, dict):
-                    messages.append(m.original)
-                else:
-                    # For non-dict originals, use the normalized content
-                    messages.append(
-                        {"content": m.content, "role": "user", "type": "message"}
-                    )
+        # Build the message list
+        messages = []
+
+        conversation = None
+        try:
+            conversation = self._conversation  # type: ignore[attr-defined]
+        except AttributeError:
+            pass
+
+        if conversation:
+            # Extract serializable message content from conversation history
+            for m in conversation.messages:
+                if isinstance(m.original, dict):
+                    messages.append(m.original)
+                else:
+                    # For non-dict originals, preserve the role when available
+                    role = "user"
+                    try:
+                        role = m.role  # type: ignore[attr-defined]
+                    except AttributeError:
+                        pass
+                    messages.append(
+                        {"content": m.content, "role": role, "type": "message"}
+                    )
@@
-        # Add new messages
-        messages.extend(new_messages)
-        kwargs["input"] = messages
-
-        # Add messages to conversation
-        if hasattr(self, "_conversation") and self._conversation:
-            normalized_messages = self._normalize_message(new_messages)
-            for msg in normalized_messages:
-                self._conversation.messages.append(msg)
+        # Add new messages
+        messages.extend(new_messages)
+        kwargs["input"] = messages
+
+        # Add messages to conversation
+        if conversation:
+            normalized_messages = self._normalize_message(new_messages)
+            for msg in normalized_messages:
+                conversation.messages.append(msg)

This keeps history roles accurate and removes hasattr usage in favor of normal attribute access with AttributeError handling, in line with the coding guidelines.

🧹 Nitpick comments (1)
plugins/openrouter/vision_agents/plugins/openrouter/openrouter_llm.py (1)

19-19: _instructions should allow None to match usage and avoid type ignores

_instructions is annotated as str but later set to None in create_response, requiring # type: ignore[assignment]. Make the type explicitly nullable and drop the ignore to better reflect actual usage.

-    _instructions: str
+    _instructions: str | None
...
-        # Temporarily clear instructions so parent doesn't add it
-        self._instructions, original = None, self._instructions  # type: ignore[assignment]
+        # Temporarily clear instructions so parent doesn't add it
+        self._instructions, original = None, self._instructions
📜 Review details

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Disabled knowledge base sources:

  • Linear integration is disabled by default for public repositories

You can enable these sources in your CodeRabbit configuration.

📥 Commits

Reviewing files that changed from the base of the PR and between baa83cf and 5163248.

📒 Files selected for processing (1)
  • plugins/openrouter/vision_agents/plugins/openrouter/openrouter_llm.py (2 hunks)
🧰 Additional context used
📓 Path-based instructions (1)
**/*.py

📄 CodeRabbit inference engine (.cursor/rules/python.mdc)

**/*.py: Never adjust sys.path in Python code
Never write except Exception as e - use specific exception handling
Avoid using getattr, hasattr, delattr and setattr; prefer normal attribute access in Python
Docstrings should follow the Google style guide for docstrings

Files:

  • plugins/openrouter/vision_agents/plugins/openrouter/openrouter_llm.py
🧬 Code graph analysis (1)
plugins/openrouter/vision_agents/plugins/openrouter/openrouter_llm.py (1)
plugins/openai/vision_agents/plugins/openai/openai_llm.py (2)
  • create_response (112-205)
  • add_conversation_history (108-110)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (4)
  • GitHub Check: unit / Test "not integration"
  • GitHub Check: unit / Mypy
  • GitHub Check: unit / Test "not integration"
  • GitHub Check: unit / Mypy
🔇 Additional comments (1)
plugins/openrouter/vision_agents/plugins/openrouter/openrouter_llm.py (1)

50-82: Confirm _instructions is a rendered string before injecting as system content

This override assumes self._instructions can be placed directly in {"content": self._instructions, ...}. With the move to structured Instructions, if _instructions is an object (not a plain string), this may break JSON serialization or inject an unintended representation.

Please verify that by the time create_response runs, _instructions is already rendered to a string suitable as system content (e.g., via a render/format method in LLM.set_instructions). If not, add the appropriate conversion here before building the system message.

@maxkahan maxkahan merged commit 4f6a325 into main Nov 27, 2025
8 checks passed
@maxkahan maxkahan deleted the fix-openrouter branch November 27, 2025 18:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants