Skip to content

ci: skip live tests on 429 (shared-key rate-limit resilience)#38

Merged
karlwaldman merged 1 commit into
mainfrom
ci/live-skip-on-429
Jun 22, 2026
Merged

ci: skip live tests on 429 (shared-key rate-limit resilience)#38
karlwaldman merged 1 commit into
mainfrom
ci/live-skip-on-429

Conversation

@karlwaldman

Copy link
Copy Markdown
Member

Problem

The live/integration tests hit the real API with a single 1-request/second key shared across SDK repos in CI. Cross-repo contention produces HTTP 429 (RateLimitError) responses that red-flag otherwise-green code.

Change

Add a live_call fixture in tests/integration/conftest.py that:

  • Spaces live calls >= 1.1s apart (respecting the 1 req/sec shared key).
  • Converts any 429 (RateLimitError) into a pytest.skip("rate-limited (shared CI key) - skipping live assertion").
  • Re-raises all non-429 errors unchanged so real failures still surface (pytest.skip raises Skipped, a BaseException, so it also propagates correctly past the timeout test's except Exception).

Every live API call in test_live_api.py and test_historical_endpoints.py is routed through this helper. The concurrent stress test wraps its future.result() collection in the same 429->skip guard.

Preserved behavior

  • Existing skip-if-no-OILPRICEAPI_KEY behavior is unchanged.
  • No version bump. No source (oilpriceapi/) changes — tests only.

Gates (all green locally)

  • mypy oilpriceapi/ --ignore-missing-imports -> Success
  • ruff check oilpriceapi/ -> All checks passed
  • pytest tests/ --ignore=tests/integration --ignore=tests/contract -m 'not slow' --cov=oilpriceapi -> 311 passed, 9 skipped, coverage 59.33% (>50%)

🤖 Generated with Claude Code

The integration/live tests hit the real API with a single 1-request/second
key that is shared across SDK repos in CI. Cross-repo contention produces
HTTP 429 (RateLimitError) responses that fail otherwise-green code.

Add a `live_call` fixture in tests/integration/conftest.py that:
- spaces live calls >= 1.1s apart (1 req/sec shared key), and
- converts any 429 (RateLimitError) into a pytest.skip while re-raising
  all other errors unchanged so real failures still surface.

Route every live API call in test_live_api.py and
test_historical_endpoints.py through this helper. The existing
skip-if-no-OILPRICEAPI_KEY behavior is preserved. No version bump.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Jun 22, 2026

Copy link
Copy Markdown

Warning

Review limit reached

@karlwaldman, we couldn't start this review because you've reached your PR review rate limit.

More reviews will be available in 56 minutes and 11 seconds. Learn how PR review limits work.

Your organization has used up its prepaid credits, and credit purchases are no longer available. Enable the review add-on in the billing tab to keep reviews running — you're only billed for reviews past your plan's rate limits ($0.25/file).

⌛ How to resolve this issue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based credits.

🚦 How do rate limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan refill rate.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, the refill rate gradually slows as usage increases. The highest same-day bursts are limited more strictly.

Please see our Fair Usage Limits Policy for further information.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: c6d29471-a7ba-4aab-bacc-054be5bdb819

📥 Commits

Reviewing files that changed from the base of the PR and between 81e24b1 and 38b423c.

📒 Files selected for processing (3)
  • tests/integration/conftest.py
  • tests/integration/test_historical_endpoints.py
  • tests/integration/test_live_api.py
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch ci/live-skip-on-429

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@karlwaldman karlwaldman merged commit 7c7b777 into main Jun 22, 2026
6 checks passed
@karlwaldman karlwaldman deleted the ci/live-skip-on-429 branch June 22, 2026 09:15
karlwaldman added a commit that referenced this pull request Jul 3, 2026
The integration/live tests hit the real API with a single 1-request/second
key that is shared across SDK repos in CI. Cross-repo contention produces
HTTP 429 (RateLimitError) responses that fail otherwise-green code.

Add a `live_call` fixture in tests/integration/conftest.py that:
- spaces live calls >= 1.1s apart (1 req/sec shared key), and
- converts any 429 (RateLimitError) into a pytest.skip while re-raising
  all other errors unchanged so real failures still surface.

Route every live API call in test_live_api.py and
test_historical_endpoints.py through this helper. The existing
skip-if-no-OILPRICEAPI_KEY behavior is preserved. No version bump.

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant