New sync client doesn't recover from a connection failure on Android

[Radiokot]

Hey guys, so I briefly tried newClientImplementation and found out that it doesn't recover from a connection failure, meaning it doesn't retry uploading the transaction if the first attempt failed due to missing data connection. I also tried switching newClientImplementation and method individually, and the issue occurs only with newClientImplementation = true so apparently nothing to do with ConnectionMethod.WebSocket.

To reproduce:

• Write to the database while offline
• Wait for the sync attempt to fail
• Enable the data connection
• Wait for the changes to appear in Supabase

With the stable client, you have the following:

... do the write ...
Error in streamingSync: Unable to resolve host
... enable data connection ...
Could not apply checkpoint. Waiting for next sync complete line
... Supabase connector is called, transaction uploaded ...
validated checkpoint Checkpoint(...)

With the new client, you have this:

... do the write ...
Error in streamingSync: Unable to resolve host
... enable data connection ...
Could not apply checkpoint. Waiting for next sync complete line
[SQLite EXEC] BEGIN TRANSACTION
powersync_control(line_text, {"token_expires_in":602356})
[SQLite EXEC] SELECT powersync_control(?, ?) AS r
control result: []
... nothing else happens ...

Force-closing and re-opening the app doesn't help. The only way to get the transaction uploaded is to make another one, on this exact device with the data connection enabled – then the sync kicks back and the failed transaction gets uploaded.

Thanks.

[simolus3]

Thanks for the report! This is exactly the kind of issue we were hoping to find by releasing the client as an experimental option :) It will be fixed by the linked PR.

[Radiokot]

Hey @simolus3 ,

I've just tested version 1.2.1. It got better, but as unfortunate as it may be, the issue is still present when ConnectionMethod.WebSocket is used. I found out that the connection status check in SyncStream works for HTTP, but gives false negative result for the socket.

powersync-kotlin/core/src/commonMain/kotlin/com/powersync/sync/SyncStream.kt

Line 127 in 4bfaa3b

if (!status.connected || !isUploadingCrud.compareAndSet(null, thisIteration)) {

[simolus3]

Version 1.2.2 should fix this issue for websockets as well, thanks again for the follow-up!

[Radiokot]

Hey Simon, thanks for working on the fix! We're close, but... not yet 🥲 Your fix makes it possible to resume sync on app force re-start, but the new client still doesn't recover from the connection loss with WebSockets (didn't try with HTTP). If you disable the data connection after some time after the app start, the sync stream dies and can only be resumed with app force re-start. Here's how I tested it:

With the stable client:

• Start the app with enabled data connection
• Wait for the "validated checkpoint Checkpoint..." in the log
• Wait 10 sec
• Disable data connection
• See "Error in streamingSync" in the log every 5 seconds
• Enable data connection
• Write to the local DB or to the Supabase
• See the changes synchronized

With the new client:

• Start the app with enabled data connection
• Wait for the "Validated and applied checkpoint" in the log
• Wait 10 sec if it is a debuggable build, 2 minutes if it is a profileable/release build
• Disable data connection
• See no errors in logs
• Enable data connection
• Write to the local DB or to the Supabase
• See no changes synchronized

[Radiokot]

Having that it takes longer to fail in a non-debuggable build, I thought that it perhaps was some kind of memory issue and the sync stream dies regardless of the data connection, but it doesn't. If you keep the data connection enabled, even after 2 minutes you can do some writes and see the changes synchronized.

[simolus3]

This is very interesting, thanks for sharing. I wonder if this may be due to the keepalive handling in RSocket, but either way there's more work for us here.