Skip to content

Fix flaky hosted API tests: unify SQLite timeout, increase httpx timeout, fix missing await#21105

Draft
devin-ai-integration[bot] wants to merge 1 commit intomainfrom
devin/1773342657-fix-flaky-hosted-api-tests
Draft

Fix flaky hosted API tests: unify SQLite timeout, increase httpx timeout, fix missing await#21105
devin-ai-integration[bot] wants to merge 1 commit intomainfrom
devin/1773342657-fix-flaky-hosted-api-tests

Conversation

@devin-ai-integration
Copy link
Contributor

Fixes two flaky test failures observed on main (workflow run, commit 11576ed):

  • test_get_runs_in_queue_concurrency_limit_and_limit[1]httpx.ReadError caused by sqlite3.OperationalError: database is locked
  • test_delete_task_run_deletes_logshttpx.ConnectTimeout

Changes

1. Unify SQLite busy_timeout to 60s for all modes (configurations.py)

The test-mode SQLite busy_timeout was 30s while production used 60s. Under heavy parallel test execution (pytest-xdist), the hosted API server subprocess and test process share the same SQLite file, and lock contention can exceed 30s. The connect_args timeout is also bumped from 30s to 60s to match.

2. Increase hosted_api_client httpx timeout to 30s (tests/fixtures/api.py)

The default httpx timeout of 5s is too aggressive when the hosted server is slow to respond under CI load. This is a broad fix that helps all tests using hosted_api_client.

3. Fix missing await on asyncio.sleep(1) (test_task_runs.py)

test_delete_task_run_deletes_logs had asyncio.sleep(1) without await, which creates a coroutine object that's immediately discarded. This turned the polling loop into a tight busy-loop hammering the database instead of yielding between retries.

Review checklist for humans

  • Verify the 30s httpx timeout for hosted_api_client is appropriate — not so generous that it masks real server issues, but note it's still well under the 90s pytest-timeout
  • The PRAGMA busy_timeout is now unconditionally 60s (no test/prod branch), while connect_args timeout still has a test-mode branch — confirm this asymmetry is acceptable
  • The await asyncio.sleep(1) fix is clearly correct but verify the surrounding polling logic still has the right semantics

Checklist

  • This pull request references any related issue by including "closes <link to issue>"
    • If no issue exists and your change is not a small fix, please create an issue first.
  • If this pull request adds new functionality, it includes unit tests that cover the changes
  • If this pull request removes docs files, it includes redirect settings in mint.json.
  • If this pull request adds functions or classes, it includes helpful docstrings.

Link to Devin Session | Requested by: bot_apk (apk@cognition.ai)

…out, fix missing await

Three root causes addressed for flaky test failures on main:

1. SQLite busy_timeout was 30s in test mode vs 60s in production. Under
   heavy parallel test execution (pytest-xdist), the hosted API server
   subprocess and test process compete for the same SQLite file, causing
   lock contention that can exceed 30s. Unified to 60s for all modes.

2. hosted_api_client used the default httpx timeout of 5s. When the hosted
   server is slow to respond under CI load (e.g., waiting on SQLite locks),
   5s is insufficient. Increased to 30s.

3. test_delete_task_run_deletes_logs had a missing 'await' on
   asyncio.sleep(1), creating a tight polling loop that hammered the DB
   instead of yielding between retries.

Failing tests:
- test_get_runs_in_queue_concurrency_limit_and_limit[1] (httpx.ReadError)
- test_delete_task_run_deletes_logs (httpx.ConnectTimeout)

Co-authored-by: bot_apk <apk@cognition.ai>
@devin-ai-integration
Copy link
Contributor Author

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment and CI monitoring

@devin-ai-integration devin-ai-integration bot added the development Tech debt, refactors, CI, tests, and other related work. label Mar 12, 2026
@codspeed-hq
Copy link

codspeed-hq bot commented Mar 12, 2026

Merging this PR will not alter performance

✅ 2 untouched benchmarks


Comparing devin/1773342657-fix-flaky-hosted-api-tests (8b8a90e) with main (5188608)

Open in CodSpeed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

development Tech debt, refactors, CI, tests, and other related work.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

0 participants