Stop spawning long-lasting threads for libsql connections

[MarinPostma]

This PR removes the need to spawn long-lasting threads for libsql connections. This reduces the number of threads required by sqld, since it can now reuse threads from the thread pool for connections.

Lock stealing

Previously, every connection spawned a blocking thread that kept track of the transaction state and timed out and rolled if a transaction didn't complete in time.

Avoiding to spawn a new thread means that we need a new mechanism to timeout a transaction. This PR introduces a lock-stealing mechanism.
The lock-stealing algorithm only monitors write locks, meaning RO transactions can continue indefinitely. Another change is that a lock is stolen i.if another connection claims it. Therefore, a write connection can go indefinitely until another connection steals the lock, guaranteeing it had at least TXN_TIMEOUT_DURATION to perform work.

Here is a detailed description of the algorithm:

• all connections to a database share a TxnState, that contains a TxnSlot
• when a connection acquires a write lock to the database, this is detected by monitoring the state of the connection before and after the call thanks to sqlite3_txn_state()
• if the connection acquired a write lock (txn state none/read -> write), a new txn slot is created. A clone of the TxnSlot is placed in the TxnState shared with other connections to this database, while another clone is kept in the transaction state. The TxnSlot contains the instant the txn should timeout, a is_stolen flag, and a pointer to the connection currently holding the lock.
• when another connection attempts to acquire the lock, the busy_handler callback will be called. The callback is being passed to the TxnState for the connection. The handler looks at the current slot to determine when the current txn will timeout and waits for that instant before retrying. The waiting handler can also be notified that the transaction has been finished early.
• If the handler waits until the txn timeout and isn't notified of the termination of the txn, it will attempt to steal the lock. This is done by calling rollback on the slot's txn, and marking the slot as stolen.
• When a connection notices that its slot has been stolen, it returns a timed-out error to the subsequent request.

[psarna]

Nice, I really like the idea, but there's one more thing to consider. Previous transaction time limit was also an implicit backpressure/guardrails mechanism, so that e.g. if somebody opens a single transaction and then keeps writing for ages, they'd just get a timeout and learn that they need to write in smaller bits. After the patch, users could technically keep writing, with their data not getting checkpointed (because of the write lock), and not backed up I guess, since it's not a full transaction. It's not very critical, because it's just an antipattern to have super-long-standing transactions, but maybe we should impose some kind of high limit anyway? So that people that hold a transaction for a full minute know that they're very likely doing something wrong.

[psarna]

Sounds like we can have a fast path here where timeout_at is already in the past, that doesn't need to run within tokio::block_on and use the runtime in any way, right? And we can just steal the lock immediately. It's technically just an optimization, but I suggest we add it early, if we have an opportunity to run sync code without blocking on an async reactor

[psarna]

Ugh, I just wrote an elaborate paragraph why I think we may need acquire/release on load/store of is_stolen respectively`, only to read your comment about the connection lock above. Nvm 😇

[MarinPostma]

That's a relatively fix I think, we just have to check that we haven't timed out the current transaction in the connection. For checkpointing, we can also have it steal the lock when necessary. I'll do that :) @psarna

[psarna]

Btw, our checkpointing connection is not preemptible, since it needs to finish, right? In that case we should clearly log and measure how much time a checkpoint takes, since it's likely to be a source of SQLITE_BUSY for users

[MarinPostma]

This is an argument for checkpointing often

[MarinPostma]

In that case, we should clearly log and measure

we already do that :)

[penberg]

Looks good, ship it, @MarinPostma! 🚀