Skip to content

chore(ntx-builder): DataStore & DB writers refactor #1689

Merged
Mirko-von-Leipzig merged 4 commits intonextfrom
santiagopittella-ntx-builder-datastore-refactor-v2
Feb 23, 2026
Merged

chore(ntx-builder): DataStore & DB writers refactor #1689
Mirko-von-Leipzig merged 4 commits intonextfrom
santiagopittella-ntx-builder-datastore-refactor-v2

Conversation

@SantiagoPittella
Copy link
Collaborator

closes #1657
closes #1628

This PR includes two major changes:

  1. Persistent note script cache.
    Adds a note_script table to the database so that fetched note scripts survive restarts.

  2. Centralized DB writes via actor notification channel.
    In the phase 2 PR, when we switched from in-memory to the SQLite db, actors were still writing to the DB to mark notes as failing. In this PR I added a channel between actor and coordinator so that the actor can emit an event to the coordinator when it is failing to consume a note.

Comment on lines 101 to 102
/// Receiver for notifications from account actors (e.g., note failures).
notification_rx: mpsc::UnboundedReceiver<ActorNotification>,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this should be bounded - actors can wait to submit.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we limit it to 1 message at a time?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The actor needs to wait until the notification has been handled in order to proceed. So having it be a single one at a time makes sense.

So we might also want a return oneshot channel for the actor to know its been completed.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hmmm in that case we should keep one receiver per actor in the coordinator, since oneshot is single producer, right?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are several patterns we can use here; but the one I'm proposing looks like this:

struct Coordinator {
    rx_notes_failed: tokio::sync::mpsc::Receiver<
        // Notes that failed.
        HashSet<NoteId>,
        // Return channel to inform the note sender that processing is done.
        tokio::sync::oneshot::Sender<()>,
    >,
    ...
}

impl Coordinator {
    fn handle_note_failure(&mut self, notes: HashSet<NoteId>, notify: oneshot::Sender<()> {
        self.db.increment_note_failures(notes).await;
        let _ = notify.send().await;
    }
}

struct Actor {
    tx_notes_failed: tokio::sync::mpsc::Sender < ..as above..>,
}

impl Actor {
    // Channel closure handling elided as demo code.
    async fn process_note_failure(&self, notes: HashSet<NoteId>) {
        let (tx, rx) = tokio::sync::oneshot::channel();

        let _ = self.tx_notes_failed.send(notes, tx).await;
        // Wait for coordinator to inform us that processing completed.
        // If we don't wait, we risk a data race with the note failure
        // being updated in the database.
        rx.await;
    }
}

There are alternatives, but this has the advantage that the oneshot return channels usage is obviously tied to the note failure request. You could have an additional channel per actor, or try to re-use the notification channel, but then you run into other problems e.g. that channel will have other signals on it already maybe.

@SantiagoPittella SantiagoPittella force-pushed the santiagopittella-ntx-builder-datastore-refactor-v2 branch 2 times, most recently from ce60fd3 to 931937b Compare February 19, 2026 20:43
@SantiagoPittella SantiagoPittella added the no changelog This PR does not require an entry in the `CHANGELOG.md` file label Feb 19, 2026
Base automatically changed from santiagopittella-ntx-builder-state-mgmt-migration-v2 to next February 20, 2026 06:14
@SantiagoPittella SantiagoPittella force-pushed the santiagopittella-ntx-builder-datastore-refactor-v2 branch from 931937b to 9198f08 Compare February 20, 2026 13:24
@SantiagoPittella
Copy link
Collaborator Author

@Mirko-von-Leipzig should I wait for any other review to merge this one?

@Mirko-von-Leipzig Mirko-von-Leipzig merged commit e2d908b into next Feb 23, 2026
30 of 31 checks passed
@Mirko-von-Leipzig Mirko-von-Leipzig deleted the santiagopittella-ntx-builder-datastore-refactor-v2 branch February 23, 2026 11:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

no changelog This PR does not require an entry in the `CHANGELOG.md` file

Projects

None yet

Development

Successfully merging this pull request may close these issues.

NTX Builder state (phase 3): Replace DataStore impl with DB queries Persist ntx state

2 participants