Skip to content

Async recipient-side of static invoice server #3618

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 14 commits into
base: main
Choose a base branch
from

Conversation

valentinewallace
Copy link
Contributor

@valentinewallace valentinewallace commented Feb 24, 2025

As part of being an async recipient, we need to interactively build an offer and static invoice with an always-online node that will serve static invoices on our behalf in response to invoice requests from payers.

While users could build this invoice manually, the plan is for LDK to automatically build it for them using onion messages. See this doc for more details on the protocol. Here we implement the client side of the linked protocol.

See lightning/bolts#1149 for more information on async payments.

Partially addresses #2298

@valentinewallace
Copy link
Contributor Author

Will go through the commits in a bit more detail before taking this out of draft, but conceptual feedback or feedback on the protocol itself is welcome, or the way the code is organized overall. It does add a significant amount of code to ChannelManager currently.

Copy link
Collaborator

@TheBlueMatt TheBlueMatt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmmm, I wonder if we shouldn't allow the client to cache N offers rather than only 1. I worry a bit about the privacy implications of having One Offer that gets reused across different contexts.

@valentinewallace
Copy link
Contributor Author

Hmmm, I wonder if we shouldn't allow the client to cache N offers rather than only 1. I worry a bit about the privacy implications of having One Offer that gets reused across different contexts.

I think that makes sense, so they would interactively build and cache a few and then randomly(?) return one of them on get_cached_async_receive_offer?

It seems reasonable to save for follow-up although I could adapt the AsyncReceiveOffer cache struct serialization for this now.

@TheBlueMatt
Copy link
Collaborator

TheBlueMatt commented Feb 25, 2025

I think that makes sense, so they would interactively build and cache a few and then randomly(?) return one of them on get_cached_async_receive_offer?

Yea, I dunno what to do for the fetch'er, maybe we just expose the whole list?

It seems reasonable to save for follow-up although I could adapt the AsyncReceiveOffer cache struct serialization for this now.

Makes sense, tho I imagine it would be a rather trivial diff, no?

@jkczyz jkczyz self-requested a review February 27, 2025 18:10
@valentinewallace valentinewallace added the weekly goal Someone wants to land this this week label Feb 27, 2025
Comment on lines +568 to +592
const IV_BYTES: &[u8; IV_LEN] = b"LDK Offer Paths~";
let mut hmac = expanded_key.hmac_for_offer();
hmac.input(IV_BYTES);
hmac.input(&nonce.0);
hmac.input(ASYNC_PAYMENTS_OFFER_PATHS_INPUT);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need to include path_absolute_expiry?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought the nonce/IV was sufficient but I'm not certain. @TheBlueMatt would it be an improvement to commit to the expiry in the hmac? IIUC the path still can't be re-purposed...

@valentinewallace valentinewallace marked this pull request as ready for review March 4, 2025 21:14
@valentinewallace
Copy link
Contributor Author

Going to base this on #3640. Will finish updating the ser macros based on those changes and push updates here after finishing some review.

@valentinewallace valentinewallace removed the weekly goal Someone wants to land this this week label Mar 4, 2025
Copy link
Contributor

@vincenzopalazzo vincenzopalazzo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added a couple of comments that I find out while working on the CI failure in #3593

@valentinewallace
Copy link
Contributor Author

Pushed some updates after moving the async receive offer cache into the new OffersMessageFlow struct added in #3639.

Copy link
Contributor

@elnosh elnosh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

New to the codebase but interested in following async payments. From reading the explanation in the commit messages, the protocol/flow between the async recipient and the always-online node to build the static invoice and offer made sense. Overall the code changes look good to me.

Comment on lines +41 to +44
fn handle_offer_paths_request(
&self, message: OfferPathsRequest, context: AsyncPaymentsContext,
responder: Option<Responder>,
) -> Option<(OfferPaths, ResponseInstruction)>;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see it is similar to other message handler traits in the OnionMessenger but I was wondering why return Options in these handle_ methods instead of Results?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good question, I wrote that code forever ago but I think it was just consistency with the other onion message handler traits at the time. Fine to switch if reviewers prefer, although I might punt since the handle_held_htlc_available instance within the async payments trait is pre-existing...

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The original OffersMessageHandler handles all offers-related message types in one method where not responding with a message is typical for handling Bolt12Invoice. Similarly, a message may not have a reply path and thus wouldn't have a responder. So the interface was more general.

For handler traits that have a method per message, it may make sense to limit the interface if there is a restricted set of possibilities. That said, I'm not sure if we want to use Result as the responding behavior should really be defined by the handler. For instance, an error handling an InvoiceRequest could mean either the corresponding Offer could not be authenticated -- and thus no response should be sent -- or the sender made an error -- in which case an InvoiceError should be sent as a response.

Somewhat related: for Responder, OnionMessenger could simply not call the handler if the message doesn't have a reply path but we are expecting one. I don't have any strong opinions yet on whether Option should be used in that case.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That said, I'm not sure if we want to use Result as the responding behavior should really be defined by the handler.

In the async payments case, as well, if we receive an offer_paths message we may have enough offers cached already and choose not to respond, which is expected behavior. So the signature may end up as Result<Option<ServeStaticInvoice>, ()>. That's not necessarily bad, but it's less clean than one might assume if we switched to using a Result.

Comment on lines +257 to +264
Self::OfferPathsRequest(_) => OFFER_PATHS_REQ_TLV_TYPE,
Self::OfferPaths(msg) => msg.tlv_type(),
Self::ServeStaticInvoice(msg) => msg.tlv_type(),
Self::StaticInvoicePersisted(_) => INVOICE_PERSISTED_TLV_TYPE,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do some use the const directly here and others get the const set through the tlv_type on the msg?

Copy link
Contributor Author

@valentinewallace valentinewallace May 19, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The variants that return consts correspond to messages that don't implement the OnionMessageContents trait, so they don't have the tlv_type method available. Looks like docs are a bit lacking here but the OnionMessageContents trait implementation seems to only be needed for onion messages that are sent in direct response to other onion messages.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't it more consistent to implement OnionMessageContents anyway?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't mind doing that, may want to save for follow-up though since it's pre-existing for a bunch of other messages too. Or could add better documentation on OnionMessageContents.

@valentinewallace valentinewallace force-pushed the 2025-02-static-inv-server-client branch from 5455d55 to f8023ca Compare May 21, 2025 00:11
@valentinewallace
Copy link
Contributor Author

Rebased on the latest version of #3639

Copy link

codecov bot commented May 21, 2025

Codecov Report

Attention: Patch coverage is 29.41176% with 120 lines in your changes missing coverage. Please review.

Project coverage is 89.80%. Comparing base (7b62ea4) to head (6fe5f86).
Report is 2 commits behind head on main.

Files with missing lines Patch % Lines
lightning/src/ln/channelmanager.rs 20.89% 53 Missing ⚠️
lightning/src/onion_message/async_payments.rs 0.00% 29 Missing ⚠️
lightning/src/onion_message/functional_tests.rs 0.00% 20 Missing ⚠️
lightning/src/ln/peer_handler.rs 0.00% 17 Missing ⚠️
lightning/src/offers/async_receive_offer_cache.rs 95.45% 0 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #3618      +/-   ##
==========================================
- Coverage   89.91%   89.80%   -0.11%     
==========================================
  Files         160      161       +1     
  Lines      129307   129476     +169     
  Branches   129307   129476     +169     
==========================================
+ Hits       116262   116275      +13     
- Misses      10355    10502     +147     
- Partials     2690     2699       +9     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@valentinewallace valentinewallace force-pushed the 2025-02-static-inv-server-client branch 3 times, most recently from d1cc154 to 68fd751 Compare May 22, 2025 22:52
@valentinewallace
Copy link
Contributor Author

Pushed some minor fixes for CI.

@valentinewallace
Copy link
Contributor Author

Updated in response to to @joostjager's comments. Link to diff

@valentinewallace valentinewallace changed the title Client-side of static invoice server Async recipient-side of static invoice server Jun 2, 2025
@valentinewallace valentinewallace force-pushed the 2025-02-static-inv-server-client branch from ec93a15 to 7823551 Compare June 2, 2025 17:44
@valentinewallace
Copy link
Contributor Author

Rebased on main after #3639 landed

@valentinewallace valentinewallace force-pushed the 2025-02-static-inv-server-client branch from 7823551 to 66d7b4f Compare June 2, 2025 17:54
@valentinewallace
Copy link
Contributor Author

valentinewallace commented Jun 2, 2025

Clarified a few more docs I noticed. Link to diff

Edit: pushed another CI fix

@valentinewallace valentinewallace force-pushed the 2025-02-static-inv-server-client branch from 66d7b4f to 6adac0b Compare June 2, 2025 18:01
Copy link
Contributor

@joostjager joostjager left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall, this is well executed. However, as we also discussed offline, I still can't shake the feeling that this approach may be too far ahead of where we are right now.

The use case is spontaneous payments of variable amounts to offline recipients. If the current state of Lightning is LSPs with client nodes without support for blinded paths, and this is intended for that environment, then there must be a simpler way to achieve the same goal. Ultimately, the payer only needs to know the LSP node and the recipient node. With that information, they can send an onion message to the LSP, wait for the release, and then send the HTLC.

@@ -15,7 +15,8 @@ use lightning::ln::peer_handler::IgnoringMessageHandler;
use lightning::ln::script::ShutdownScript;
use lightning::offers::invoice::UnsignedBolt12Invoice;
use lightning::onion_message::async_payments::{
AsyncPaymentsMessageHandler, HeldHtlcAvailable, ReleaseHeldHtlc,
AsyncPaymentsMessageHandler, HeldHtlcAvailable, OfferPaths, OfferPathsRequest, ReleaseHeldHtlc,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Commit message reads very well. Some of that valuable info might even be moved into code comments?

Comment on lines +257 to +264
Self::OfferPathsRequest(_) => OFFER_PATHS_REQ_TLV_TYPE,
Self::OfferPaths(msg) => msg.tlv_type(),
Self::ServeStaticInvoice(msg) => msg.tlv_type(),
Self::StaticInvoicePersisted(_) => INVOICE_PERSISTED_TLV_TYPE,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't it more consistent to implement OnionMessageContents anyway?

/// Will only be set if [`UserConfig::paths_to_static_invoice_server`] is set and we succeeded in
/// interactively building a [`StaticInvoice`] with the static invoice server.
#[cfg(async_payments)]
pub fn get_cached_async_receive_offers(&self) -> Vec<Offer> {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tests in the follow up look good indeed.

As part of being an async recipient, we need to support interactively building
an offer and static invoice with an always-online node that will serve static
invoices on our behalf.

Add a config field containing blinded message paths that async recipients can
use to request blinded paths that will be included in their offer. Payers will
forward invoice requests over the paths returned by the server, and receive a
static invoice in response if the recipient is offline.
@valentinewallace valentinewallace force-pushed the 2025-02-static-inv-server-client branch from 6adac0b to 128208f Compare June 4, 2025 20:11
@valentinewallace
Copy link
Contributor Author

Rebased after #3767 landed, haven't addressed feedback yet

Because async recipients are not online to respond to invoice requests,
the plan is for another node on the network that is always-online to serve
static invoices on their behalf.

The protocol is as follows:
- Recipient is configured with blinded message paths to reach the static invoice
  server
- On startup, recipient requests blinded message paths for inclusion in their
  offer from the static invoice server over the configured paths
- Server replies with offer paths for the recipient
- Recipient builds their offer using these paths and the corresponding static
  invoice and replies with the invoice
- Server persists the invoice and confirms that they've persisted it, causing
  the recipient to cache the interactively built offer for use

At pay-time, the payer sends an invoice request to the static invoice server,
who replies with the static invoice after forwarding the invreq to the
recipient (to give them a chance to provide a fresh invoice in case they're
online).

Here we add the requisite trait methods and onion messages to support this
protocol.

An alterate design could be for the async recipient to publish static invoices
directly without a preceding offer, e.g. on their website. Some drawbacks of
this design include:
1) No fallback to regular BOLT 12 in the case that the recipient happens to be online
at pay-time. Falling back to regular BOLT 12 allows the recipient to provide a fresh
invoice and regain the proof-of-payment property
2) Static invoices don't fit in a QR code
3) No automatic rotation of the static invoice, which is useful in the case that payment
paths become outdated due to changing fees, etc
In future commits, as part of being an async recipient, we will interactively
build offers and static invoices with an always-online node that will serve
static invoices on our behalf.

Once an offer is built and the static invoice is confirmed as persisted by the
server, we will use the new offer cache added here to save the invoice metadata
and the offer in ChannelManager, though the OffersMessageFlow is responsible
for keeping the cache updated.

We want to cache and persist these offers so we always have them at the ready,
we don't want to begin the process of interactively building an offer the
moment it is needed. The offers are likely to be long-lived so caching them
avoids having to keep interactively rebuilding them after every restart.
As an async recipient, we need to interactively build static invoices that an
always-online node will serve to payers on our behalf.

At the start of this process, we send a requests for paths to include in our
offers to the always-online node on startup and refresh the cached offers when
they expire.
We previously weren't dropping the cache lock if no new offers were needed, then
attempted to acquire it again in a later method
We're gonna expose these for testing later so move them
@valentinewallace valentinewallace force-pushed the 2025-02-static-inv-server-client branch from 128208f to 4f8d15c Compare June 5, 2025 22:48
@valentinewallace
Copy link
Contributor Author

valentinewallace commented Jun 5, 2025

Mostly updated, need to fix linting. I added fixups for a few things to call attention/add a short explanation. Diff

Edit: fixed linting, diff

As an async recipient, we need to interactively build a static invoice that an
always-online node will serve to payers on our behalf.

As part of this process, the static invoice server sends us blinded message
paths to include in our offer so they'll receive invoice requests from senders
trying to pay us while we're offline. On receipt of these paths, create an
offer and static invoice and send the invoice back to the server so they can
provide the invoice to payers.
As an async recipient, we need to interactively build a static invoice that an
always-online node will serve on our behalf.

Once this invoice is built and persisted by the static invoice server, they
will send us a confirmation onion message. At this time, cache the
corresponding offer and mark it as ready to receive async payments.
As an async recipient, we need to interactively build offers and corresponding
static invoices, the latter of which an always-online node will serve to payers
on our behalf.

Offers may be very long-lived and have a longer expiration than their
corresponding static invoice. Therefore, persist a fresh invoice with the
static invoice server when the current invoice gets close to expiration.
Over the past several commits we've implemented interactively building an async
receive offer with a static invoice server that will service invoice requests
on our behalf as an async recipient.

Here we add an API to retrieve the resulting offers so we can receive payments
when we're offline.
@valentinewallace valentinewallace force-pushed the 2025-02-static-inv-server-client branch from 4f8d15c to 6fe5f86 Compare June 5, 2025 23:00
Comment on lines +250 to +251
// Update the invoice if it expires in less than a day, as long as the offer has a longer
// expiry than that.
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Question for reviewers: I'm not sure it makes sense to only update a static invoice if it expires in less than a day. That doesn't seem like enough time. But it only lives for 2 weeks anyway... Should we maybe make it longer lived, or replace it once it expires in a week, or?

Copy link
Contributor

@joostjager joostjager left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did another pass. Still not at the bottom of this PR in terms of understanding all the details though.

// TLV record types for the `onionmsg_tlv` TLV stream as defined in BOLT 4.
// TODO: document static invoice server onion message payload types in a bLIP.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove before merge I think?

/// [`StaticInvoicePersisted`] to the recipient to confirm that the offer corresponding to the
/// invoice is ready to receive async payments.
pub invoice: StaticInvoice,
// TODO: include blinded paths to forward the invreq to the async recipient
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reminder to address?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I originally tracked this for follow-up when there were more items left out of this PR, but at this point it's basically the only thing left and it's pretty minor. Are you fine if I just address this todo in this PR?

@@ -14264,6 +14266,7 @@ where
(15, inbound_payment_id_secret, option),
(17, in_flight_monitor_updates, option),
(19, peer_storage_dir, optional_vec),
(21, async_receive_offer_cache, (default_value, async_receive_offer_cache)),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This cache seems so far away from channel manager core responsibilities. Can't it be persisted elsewhere / separately?

I also thought that for fixing the force closures, we wanted to move away from channel mgr persistence completely and reconstruct everything from the monitors.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, it crossed my mind that this might come up when we address removing ChannelManager persistence :( Will start an offline discussion for this next week, not sure what the best solution is. We could have a separate method to retrieve the cache for persistence. Don't think this should be a blocker since it's an optional field that can easily be removed, but it'd be nice to have a path forward.

fn write<W: Writer>(&self, w: &mut W) -> Result<(), io::Error> {
write_tlv_fields!(w, {
(0, self.offers, required_vec),
// offer paths request retry info always resets on restart
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The reasoning in https://github.com/lightningdevkit/rust-lightning/pull/3819/files#r2126342283 also applies here. In this code base, do we want to separate runtime state from persistent state, or does it not matter?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not really following that reasoning tbh, we have other places in the codebase where things reset on restart within persisted state.

let elapsed = duration_since_epoch.saturating_sub(offer_created_at).as_secs();

// If an offer is in the last 10% of its lifespan, it's expiring soon.
elapsed.saturating_mul(10) < offer_lifespan.saturating_mul(9)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Constant?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you think that would improve the code here? It's just a way of checking whether 90%+ of the offer lifespan has passed without using floating point arithmetic. {ELAPSED,LIFESPAN}_MULTIPLIER doesn't seem like an improvement to me, let me know if you have ideas though.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought just a percentage or fraction defined somewhere. Not critical ofc.

.new_offers_requested(duration_since_epoch);

let message = AsyncPaymentsMessage::OfferPathsRequest(OfferPathsRequest {});
enqueue_onion_message_with_reply_paths(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is more a general onion messaging question. If none of the nodes on the network relay onion messages, will this still try and work for direct connects for both the request and reply?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IIUC yeah, because we generate an event PeerConnectionNeeded to tell the user to connect to the intro node directly.

@@ -12842,7 +12842,30 @@ where
fn handle_offer_paths(
&self, _message: OfferPaths, _context: AsyncPaymentsContext, _responder: Option<Responder>,
) -> Option<(ServeStaticInvoice, ResponseInstruction)> {
None
#[cfg(async_payments)]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe I am still surprised that this lives in channelmanager. Didn't look into the flow refactoring yet, but I thought the goal was to keep high level stuff like offers out of chan mgr?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, from my view this this just calls into the flow so nothing substantial is really living in ChannelManager. The manager still receives offers messages in general because it needs to add to the outbound_payments module when we initiate a payment to an offer, but the bulk of the logic now lives in the OffersMessageFlow.

#[cfg(async_payments)]
{
let should_persist = self.flow.handle_static_invoice_persisted(_context);
let _persistence_guard =
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure how this works exactly, but can't the call to notify be skipped completely if not should_persist?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants