Skip to content

Conversation

@bradenrayhorn
Copy link
Contributor

This fixes an issue that can occur when using multiple sqlitemigration pools attempt to connect to the same database file concurrently. When opening a sqlitemigration pool while another pool is writing to the database, the pool may fail to startup with a SQLITE_BUSY error (shown as database is locked).

In my use case, I had some end-to-end tests running that started an API server and modified the database using a CLI. Both the CLI and the API server started a sqlitemigration pool and occasionally the CLI calls would fail due to this issue.

From my testing, the culprit is the ensureAppID function, which opens a deferred transaction that will later be upgraded to a write transaction. At the point of upgrading to a write transaction, if the database is locked, SQLite will fail immediately with SQLITE_BUSY [1] [2] and the busy timeout will have no effect, causing the pool to fail to start.

If ensureAppID is switched use an immediate transaction, that tells SQLite this is a write transaction. SQLite will attempt to lock the database immediately. If a lock cannot be acquired, the busy timeout is used allowing the transaction to wait until it can get a lock, thus allowing the pool to start successfully.

I've also included a test case. Please let me know on feedback on the test, it is a bit unwieldy. I'm not sure if it fits with the standards of this project.

@zombiezen zombiezen linked an issue May 4, 2024 that may be closed by this pull request
@zombiezen zombiezen merged commit 59bb121 into zombiezen:main May 4, 2024
@zombiezen
Copy link
Owner

This is great, thank you for the investigation and the fix!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Unexpected error: "migrate database: sqlite: step: database is locked"

2 participants