Adding support for main bolt12 commands to NIP-47

[21M4TW]

This is a request to add support for Bolt12 to NWC, such that a Lightning client can use NWC to generate and use Bolt12 offers, fetch and pay Bolt12 invoices, thus benefiting from the improvements over Bolt11.

[TheBlueMatt]

Cool! Sadly I think there's some confusion around the "API" for BOLT 12.

[daywalker90]

@TheBlueMatt is correct when it comes to bolt12 invoices. They should be kept out completely and only deal with offers. That means paying an offer not bolt12 invoice and dropping the fetch_invoice method completely. I was working on a PR like this aswell: daywalker90#1

[21M4TW]

@TheBlueMatt is correct when it comes to bolt12 invoices. They should be kept out completely and only deal with offers. That means paying an offer not bolt12 invoice and dropping the fetch_invoice method completely. I was working on a PR like this aswell: daywalker90#1

Thanks! As I mentioned I am also fine with dropping the fetch_invoice method. It could be replaced by a modified pay_invoice command that supports offer strings and also payer_note, and the addition of a get_offer_info command that provides the main decoded offer information. I will modify the PR to reflect this.

[21M4TW]

@TheBlueMatt is correct when it comes to bolt12 invoices. They should be kept out completely and only deal with offers. That means paying an offer not bolt12 invoice and dropping the fetch_invoice method completely. I was working on a PR like this aswell: daywalker90#1

Ok so I did remove the fetch_invoice command and made the changes required to make it work with bolt12. I have also added support for the offer issuer field, as well as other currencies when making and decoding offers. The offer_issuer, offer_id and payer_note fields are now part of the results when listing transactions, when applicable.

[TheBlueMatt]

I really hate that the API allows you to list invoices/offers which have been issued (as invoices/offers should be static and should never have any state on the node at all!) but there's no reason to leave the BOLT 12 API hobbled compared to the BOLT 11 API.

[TheBlueMatt]

#1952 (comment)

[21M4TW]

I was thinking to simply use a single flag parameter for everything, e.g. ONCHAIN|BOLT12|ARK, BOLT11|BOLT12, ANY (all bits set), etc

So every NWC client needs to be aware of every time of payment instruction? I don't buy that this makes sense.

No they don't have to. If a client just wants to disable BOLT11, it can simply use ANY&~BOLT11

[TheBlueMatt]

No they don't have to. If a client just wants to disable BOLT11, it can simply use ANY&~BOLT11

Yay parsers lol. Again we're getting into complexifying the API without a demonstrated reason. This isn't a hill I'm gonna die on but I don't buy any of the speculation you've provided as justification for adding the complexity. If some wallet runs into scalability challenges, (a) they should fix them! and (b) we can add filtering later.

[matbalez]

I can imagine many experiences that might involve a user or application developer wanting to make it easy to send a pre-defined amount of bitcoin to their generic BOLT12 offer (contained in their BIP-353 payment instructions)

The things you describe here appear to be about the sending-side, rather than the receiving-side. The current discussion is about fetching payment instructions, ie receiving.

Be that as it may, i'm curious about whether or not the things I describe will be possible once this PR is merged? If not, how might we make those possible in NWC?

[TheBlueMatt]

Be that as it may, i'm curious about whether or not the things I describe will be possible once this PR is merged? If not, how might we make those possible in NWC?

Yes, I believe so. All of the things you describe would be covered with a simple "make a payment to a given BIP-321" API, as long as that API allows for (a) pre-authorization (I believe NWC already does) and (b) including a recipient message in the payment (which presumably wallets would ignore for payment instructions that cannot accept recipient messages, eg on-chain payments). There's a separate discussion to be had as to whether the sending API should support sending directly to BIP 353s rather than requiring the NWC client do the 353 -> 321 resolution, but that can happen later IMO.

[vitorpamplona]

I think my overall point, here, is that the point of NWC generally is to communicate receiving payment instructions, pay payment instructions, and list historical transactions. The format for bitcoin payment instructions is BIP 321, so orienting NWC around BIP 321 as the way to communicate payment instructions over NWC seems logical. This PR, instead, attempts to build NWC support for each potential new payment instruction format via a new, parallel, protocol that every NWC client and server have to add support for. If we do this every time there's a new payment instruction format, NWC will never be a usable protocol. If we try to orient ourselves towards "bitcoin" rather than "low-level lightning-specific stuff" we end up with NWC being a fairly straightforward protocol with trivial extensibility and the ability to use new payment instruction formats even if the NWC client doesn't support a given format.

Frankly, BIP 321 feels like the same thing as the JSON here but in a URI format. There are variables for a bunch of different standards (parsing to other parsers) that we all need to code just to understand what it is (where to get the lnbc value vs uri value, message, etc) in our clients. Remember, Nostr clients are not necessarily using any bitcoin dependencies, like an assembler/parser for that URI. Many times these things are coded by hand. If you are telling me that there is no way around having to code parsers for every single Bitcoin encoding format then so be it, but hopefully this spec can minimize such things and just focus on ease of use.

For instance, up to now, I have not seen the need to support silent payment workflows in NIP-47. Using BIP 321 would inherently make us aware of such a thing and having to be mindful that a BIP321 can come with a silent payment in a ways that our apps don't have screens for.

On the new offer flow, we could move some of the Req/Resp params to a BIP321 scheme, but other params like single_use/used don't seem to have a place in BIP321. So, the JSON format will still be needed on top of BIP321. Which means that now things just got more complex because we have to parse BIP321 instead of getting BIP321 inner variables for free in the JSON.

[TheBlueMatt]

Frankly, BIP 321 feels like the same thing as the JSON here but in a URI format.

Sure, there's no huge gap between bitcoin:addr?lightning=bolt11 and { "on-chain": "addr", "lightning": "bolt11" }. The reason to use 321 is that every Bitcoin wallet has had support for parsing it since 2012 or so, its the underlying format of Bitcoin QR codes (allowing a NWC client to simply take the NWC response and encode it in a QR code, without any bitcoin-specific logic), etc. I don't see any reason to prefer JSON for such an encoding.

There are variables for a bunch of different standards (parsing to other parsers) that we all need to code just to understand what it is (where to get the lnbc value vs uri value, message, etc) in our clients. Remember, Nostr clients are not necessarily using any bitcoin dependencies, like an assembler/parser for that URI.

You have it inverted - part of the point of using BIP 321 is that it can be a "black box payment instruction format". NWC can be built around receiving and paying BIP 321s without ever having any kind of parser or ability to understand them! They can scan a QR code to pay, build a QR code to receive, and pass the instructions around without any concept of Bitcoin. The same is very much not true if we list out a bunch of addresses for various protocols in the JSON.

Many times these things are coded by hand. If you are telling me that there is no way around having to code parsers for every single Bitcoin encoding format then so be it, but hopefully this spec can minimize such things and just focus on ease of use.

I think you may be assuming that if an NWC client receives "bitcoin:on-chain?lightning=ln&ar=ark-addr" they have to somehow understand how to parse/validate ark-addr. They absolutely do not, they should ignore it if they are paying directly or simply pass the full URI to the payer if they are able.

For instance, up to now, I have not seen the need to support silent payment workflows in NIP-47. Using BIP 321 would inherently make us aware of such a thing and having to be mindful that a BIP321 can come with a silent payment in a ways that our apps don't have screens for.

Can you describe a concrete reason for this concern? If an NWC client is accepting payment on behalf of an NWC server/wallet it should take the URI and pass it to the payer, without thinking about what is in it - that's explicitly not its job. If it is the payer or communicating via some other protocol, it will naturally extract only the payment instruction subset that it wants.

On the new offer flow, we could move some of the Req/Resp params to a BIP321 scheme, but other params like single_use/used don't seem to have a place in BIP321. So, the JSON format will still be needed on top of BIP321.

Right, I wasn't suggesting we only send BIP 321s over the wire, indeed that makes no sense. Rather, any place we have a BOLT 12 offer in the spec, we instead replace it with a BIP 321 as the "payment instruction format".

Which means that now things just got more complex because we have to parse BIP321 instead of getting BIP321 inner variables for free in the JSON.

No you don't, see above.

[vitorpamplona]

I think you may be assuming that if an NWC client receives "bitcoin:on-chain?lightning=ln&ar=ark-addr" they have to somehow understand how to parse/validate ark-addr.

That's why Amethyst has to do today with lightning. We have to display it before the user pays it. So, I need to understand whatever encode comes back and prepare my user to see something, approve and then send to the NWC connected wallet to pay.

For lightning, for instance, we have to support getting invoices from the user's LUD16 which is an API on itself, we need to check if the resulting invoice matches the value we asked for (which we need to decode it) then we need to pass it along to the NWC wallet in a new payment and then check the preimage when it comes back. LNWithdrawn for instance, has a slightly different flow.

We do the same for Cashu tokens and their Mint APIs. We will have to do the same for offers from this PR.

So, to us, each of these protocols requires about 1-2 weeks of development to understand their specifics of each user flow and integrate the appropriate parsers, validators and, most importantly screens, etc.

[21M4TW]

No they don't have to. If a client just wants to disable BOLT11, it can simply use ANY&~BOLT11

Yay parsers lol. Again we're getting into complexifying the API without a demonstrated reason. This isn't a hill I'm gonna die on but I don't buy any of the speculation you've provided as justification for adding the complexity. If some wallet runs into scalability challenges, (a) they should fix them! and (b) we can add filtering later.

Sorry I don't understand why you like parsing so much? Usually, similarly to what Vitor is saying I think, I prefer avoiding it if unnecessary. Setting a bit flag seems better to me than relying on string parsing? And it has benefits on the node side as well, not just on the wallet side...

[TheBlueMatt]

So, to us, each of these protocols requires about 1-2 weeks of development to understand their specifics of each user flow and integrate the appropriate parsers, validators and, most importantly screens, etc.

I'm really confused, honestly. Can you describe what it is you want to do that requires you need to parse opaque payment instructions. I think you're thinking that somehow an NWC client that sees bitcoin:... has to think about it, but not only do they not need to, they should not.

Imagine I'm building a wallet frontend (Zeus, as an example) that is an NWC client. I want to display the "receive page". I ask the NWC server for the "payment instruction string" and when I receive it I shove it in a QR code. I do not parse it, I do not try to figure out what's in there, I just display it. Now instead of Cashu taking 1-2 weeks of development its there, transparently, and I change zero code!

If you're a wallet wishing to pay an NWC server, well then you already have logic to parse BIP 321. There's no new code required there - you parse it when you see a QR code, you parse it when you get a link, the code is already there.

Sorry I don't understand why you like parsing so much? Usually, similarly to what Vitor is saying I think, I prefer avoiding it if unnecessary. Setting a bit flag seems better to me than relying on string parsing? And it has benefits on the node side as well, not just on the wallet side...

Heh, sorry my parsers reference there was in reference to you saying we'd pass a string like ANY&~BOLT11 which now implies that NWC server have to build parsers for the specific boolean language to select which subset of things to remove. Like I mentioned several times in this thread, the point of using BIP 321 is specifically to avoid new parsers or logic around new protocols, not to add any.

[TheBlueMatt]

Given how long this conversation has gone on, it seems likely that a live call might be a more productive way to resolve this. Would y'all be interested in hopping on a call after the holidays? I could also schedule something this week but I imagine that might be hard for some.

[21M4TW]

Heh, sorry my parsers reference there was in reference to you saying we'd pass a string like ANY&~BOLT11 which now implies that NWC server have to build parsers for the specific boolean language to select which subset of things to remove. Like I mentioned several times in this thread, the point of using BIP 321 is specifically to avoid new parsers or logic around new protocols, not to add any.

I don't see how this should require parsing on the node side either? For example, if the server is a lightning node supporting both Bolt11 invoices and Bolt12 offers, all its NWC plugin would need to do when it is composing the BIP321 string is

if(FLAGS&BOLT11) {
   Generate Bolt11 invoice
   Append to BIP 321 string
}

if(FLAGS&BOLT12) {
   Generate Bolt12 offer
   Append to BIP 321 string
}

[21M4TW]

Given how long this conversation has gone on, it seems likely that a live call might be a more productive way to resolve this. Would y'all be interested in hopping on a call after the holidays? I could also schedule something this week but I imagine that might be hard for some.

Sure. It can work for me after the holidays yes

[vitorpamplona]

I'm really confused, honestly. Can you describe what it is you want to do that requires you need to parse opaque payment instructions. I think you're thinking that somehow an NWC client that sees bitcoin:... has to think about it, but not only do they not need to, they should not.

This will never be the case because both the application and the user confirm the transaction before sending for payment, and in order to confirm it, the app/user has to know what is actually happening... at least, value, from, to, messages, etc. Every nostr client that I know of parses lightning APIs/encoders today to perform simple zaps.

Let's say I want to zap @TheBlueMatt 12 sats with a message.

1. Load the user's profile event and get the LUD16 or LUD06 address
2. Decode that address format
3. Create a ZapRequest Nostr event
4. Hit the addresses' .well-known/lnurlp, parse the resulting JSON, and check if it allows zaps, min/max and comment size
5. Use the callback url to assemble the GET call to build the invoice.
6. Parse the result of that API for the pr property with the BOLT11-encoded invoice
7. Parse the Bolt 11 invoice to verify if the server gave me an invoice with the right amount
8. If pr is not available, parse the other parts of the JSON to figure out which JSON property has the error description
9. If bolt 11 is valid and with the right amoung, assemble the NWC payment request
10. Send the payment request to the NWC relay and keeps waiting for a response.
11. Get the payment response from NWC, and parse it to confirm the zap on the screen.
12. Once the ZapEvent arrives, use the Bolt11 invoice as a key to remove the NWC payment and not double-count the total Zaps a post received.

So, Amethyst has to:

• Understand/Parse LUD16 and LUD06
• Understand/Parse the .well-know/lnurlp format
• Assemble the LNURL REST call to build the invoice
• Parse the LNURL result to get the Bolt11 invoice or address/show the error
• Parse the Bolt11 invoice to get the value and check if it is the same. If not deal with it (either it didn't change much and it is ok, or show to the user for additional approval)
• Assemble and Parse the NWC formats.

If bolt 11 invoices show on posts, we do 7-12 as well. Withdraws are not are intensive as this, but we still parse. When Cashu tokens appear, the user has to receive them somewhere, usually to the LUD 16 address. So, Amethyst has to decode tokens (cashuA, cashuB), calculate fees, assemble and parse "melts" into the address (which also needs to create the ln invoice)

NWC will never be a simple "just pass whatever string comes from one server to the NWC relay, and you are done".

[TheBlueMatt]

I don't see how this should require parsing on the node side either? For example, if the server is a lightning node supporting both Bolt11 invoices and Bolt12 offers, all its NWC plugin would need to do when it is composing the BIP321 string is

if(FLAGS&BOLT11)

Yes, you have to somehow parse the FLAGS set :). But, again, not gonna die on this hill, its not critical.

This will never be the case because both the application and the user confirm the transaction before sending for payment, and in order to confirm it, the app/user has to know what is actually happening... at least, value, from, to, messages, etc. Every nostr client that I know of parses lightning APIs/encoders today to perform simple zaps.

Right, we're talking about two totally different use-cases for NWC. I was referring to the "wallet remote control" use case (eg something like Zeus). In this world you are doing things like talking to the NWC server to fetch payment instructions so that you can embed them in a QR code. In that case, you want a BIP 321.

You're talking about zaps, which isn't a part of NWC and is unrelated to NWC. Yes, when you want to zap you have to do a whole pile of things to figure out how to build the payment request for nostr stuff (yea, zaps were kinda terribly designed, with BOLT 12 we have a chance to redo that and make them 5x simpler, luckily), but once you have the BOLT 11 and you just want to pay it, whether NWC uses per-payment-type instructions or BIP 321 URIs barely impacts you at all. The only difference is whether you add bitcoin:?lightning= in front. Given using BIP 321s (a) improves compatibility with other future payment protocols transparently to some NWC clients, (b) avoids extra parsing and formatting in some cases, that seems like a great tradeoff!

[vitorpamplona]

I was referring to the "wallet remote control" use case

Even in this case (e.g. Alby Go), if I paste a BIP 321 uri, I want alby go do show me the amount I am paying. If it doesn't have an amount, the app needs to allow me to input it. So the app must parse all sub-encodings inside 321 and allow me to review what I am doing. Which means each encoding might have slightly different things to confirm with the user and re-encode for payment.

[TheBlueMatt]

Even in this case (e.g. Alby Go), if I paste a BIP 321 uri, I want alby go do show me the amount I am paying. If it doesn't have an amount, the app needs to allow me to input it.

This is also unrelated to NWC? If a wallet is parsing bitcoin payment instructions, it already parses BIP 321 (as that's the standard format for QR codes and link-opens).

I agree the amount-ful/amount-less distinction is an important part of a Bitcoin wallet's sending flow, and maybe it makes sense for NWC to have a way to ask the NWC server to tell us the amount required for some payment instructions, removing the need for the sender to support parsing anything.

That said, its worth pointing out that there is still value in having the NWC protocol be BIP 321-based here - if a wallet remote control app scans a QR code and can parse the BOLT 11/12 field/amount parameter it can figure out which flow to use. When it then tells the NWC server to pay, it can provide the full BIP 321 string as well as the amount it wants to pay (either to validate that the amount it parsed is being used or to community the amount the user picked). This allows the NWC server to support bitcoin payment methods that the NWC client/wallet remote control app isn't aware of at all. The QR code can include Cashu/Ark/Spark instructions that will get used seamlessly without the NWC client being aware of what they even are, and certainly without have any idea how to parse them!

So the app must parse all sub-encodings inside 321 and allow me to review what I am doing. Which means each encoding might have slightly different things to confirm with the user and re-encode for payment.

Nope! See above.

[vitorpamplona]

I just want to say that I got nothing against BIP 321, bolt 12, or silent payments. My points are strictly about simplicity. Nostr Wallet Connect was created exactly because it was too hard to implement simple payment flows directly with nodes. The more we expose the inner workings of bitcoin and lightning, the worse this spec gets. At some point this will be as complicated as coding a direct call. I don't know if there is a way to do what you want without exposing so much, but keep pushing for simplicity in these schemas.

Keep in mind that we don't need to put EVERYTHING in NIP-47. It is healthy to break things down into multiple NIPs to allow clients and wallets to choose to support different flows and/or help them build support over time, starting with simple specs and growing from there.

[TheBlueMatt]

We're absolutely entirely on the same page on goals, my goal is to keep the protocol as simple as possible and to make it as easy as possible to implement simple payment flows that Just Work. Using BIP (3)21 entirely avoids exposing the inner workings of Bitcoin/lightning/Ark/Spark, whereas your approach of putting everything in new NIPs for NWC is exposing the inner workings of Bitcoin/lightning/Ark/Spark. I don't think continuing to argue over text is bearing fruit, though - let's schedule a call for Jan 5th? Happy to do any time US-east.

[21M4TW]

I don't have anything against BIP 321 either. I agree with should have a call. Anytime Jan 5th would be perfect for me.

[21M4TW]

@TheBlueMatt , @vitorpamplona : Are we meeting today about this? Thanks!

[TheBlueMatt]

Oops, sorry, was a little over-committed with start of the year stuff. I'm free to meet any time today or thursday.

[21M4TW]

Oops, sorry, was a little over-committed with start of the year stuff. I'm free to meet any time today or thursday.

I am available today as well. Thursday would be more difficult for me. Have you hear back from @vitorpamplona ?

[TheBlueMatt]

I have not. I'm also pretty free next week mon/tues.

[vitorpamplona]

Hopefully, @rolznz can join the call as he is likely to have more usage for this than Amethyst

[21M4TW]

I have not. I'm also pretty free next week mon/tues.

Same for me. Mon/Tues is normally the easiest on my side.

[rolznz]

Monday/Tuesday is good for me too. I am in Asia time, so US-east morning time (10AM ET?) would be best for me if that works for you guys.

[TheBlueMatt]

Monday 10am ET?

[21M4TW]

Monday 10am ET?

That is perfect for me too

[TheBlueMatt]

NWC Future Payment Instructions
Monday, January 12 · 10:00 – 11:00am
Time zone: America/New_York
Google Meet joining info
Video call link: https://meet.google.com/wke-aqdf-aoc
Or dial: ‪(US) +1 559-854-1433‬ PIN: ‪772 010 435‬#
More phone numbers: https://tel.meet/wke-aqdf-aoc?pin=5426189608691