Skip to content

Comments

re-raise auth errs + fix overlong prompt err for openrouter#813

Merged
willccbb merged 5 commits intomainfrom
raise-on-auth-err
Feb 2, 2026
Merged

re-raise auth errs + fix overlong prompt err for openrouter#813
willccbb merged 5 commits intomainfrom
raise-on-auth-err

Conversation

@mikasenghaas
Copy link
Member

@mikasenghaas mikasenghaas commented Feb 2, 2026

Description

one small convenience feature + buf fix:

  1. we now reraise OAI's AuthenticationError which will immediately kill the environment process (e.g. in vf-eval the server exits) which is the more desirable "fail" loudly behavior as its completely unrecoverable. currently, we run through all rollouts requests and mark an authentication in each
  2. fix the string matching of vf.OverlongPromptError to also work for OpenRouter which had this endpoint's context [...] instead of this model's context [...] which we were matching before

Type of Change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Documentation update
  • Test improvement

Testing

  • All existing tests pass when running uv run pytest locally.
  • New tests have been added to cover the changes

Checklist

  • My code follows the style guidelines of this project as outlined in AGENTS.md
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • Any dependent changes have been merged and published

Additional Notes


Note

Low Risk
Low risk: localized error-handling changes around model API calls; primary impact is different exception propagation/typing on auth and bad-request failures.

Overview
Model-call error handling in Environment.get_model_response is adjusted to fail fast on openai.AuthenticationError by re-raising it instead of wrapping it, so unrecoverable auth failures terminate immediately.

Overlong-prompt detection is broadened by matching more generic context-length phrases (not tied to "this model's" wording), and BadRequestErrors that aren’t context-length related are now consistently wrapped as vf.ModelError.

Written by Cursor Bugbot for commit 0d4b766. This will update automatically on new commits. Configure here.

@mikasenghaas mikasenghaas marked this pull request as ready for review February 2, 2026 16:46
@mikasenghaas mikasenghaas requested a review from willccbb February 2, 2026 16:46
Copy link

@cursor cursor bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Bugbot Autofix is OFF. To automatically fix reported issues with Cloud Agents, enable Autofix in the Cursor dashboard.

@willccbb willccbb merged commit 1776624 into main Feb 2, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants