Skip to content

Conversation

@lseguin1337
Copy link

Motivation

When model providers throttle requests due to rate limiting, developers need a consistent way to detect and handle these scenarios. This enables implementing retry strategies, backoff mechanisms, or graceful degradation in applications.

Changes

Introduces ModelThrottleError, a new error type thrown when rate limiting is detected:

Before:

// Bedrock: Error was thrown as raw event data
// OpenAI: Error was propagated as-is, requiring inspection of error properties

After:

import { Agent, ModelThrottleError } from '@strands-agents/sdk'

try {
  for await (const event of agent.run('Generate a long story')) {
    // handle events
  }
} catch (error) {
  if (error instanceof ModelThrottleError) {
    // Implement retry logic with backoff
    console.log('Rate limited, retrying after delay...')
  }
}

Provider-specific behavior

  • BedrockModel: Detects throttlingException events during streaming
  • OpenAIModel: Detects HTTP 429 status, rate_limit_exceeded error codes, and common rate limit message patterns

Closes #424

pgrayy
pgrayy previously approved these changes Jan 26, 2026
OPENAI_RATE_LIMIT_PATTERNS.some((pattern) => err.message?.toLowerCase().includes(pattern))
) {
logger.warn('OpenAI threw rate limit error')
throw new ModelThrottleError(err.message ?? 'Request was throttled by the model provider', { cause: err })
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you elaborate on the purpose of { cause: err } and why you pass it in here but not for BedrockModel?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the goal was the forward the original error, but maybe it's not necessary. Let me know I can remove it

Introduces ModelThrottledError, a new error type thrown when rate limiting is detected.

Provider-specific behavior:
- BedrockModel: Detects throttlingException events during streaming
- OpenAIModel: Detects HTTP 429 status, rate_limit_exceeded error codes, and common rate limit message patterns

Closes strands-agents#424
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[V1] Models - ModelThrottledError Handling

2 participants