When the `p` multiplier is used, the amount MUST be divisible
by 10 since the resolution used internally is millisatoshi.

This addresses but does not close lightning#692.

Tabs or spaces ? Spaces seems to largely beat tabs in this files (and more globally in the repo).

We simply specify, in each case, where they will appear ("Context").

Because `globalfeatures` is already in use, we fold that into the
renamed `localfeatures` field to unify them (now called `features`),
but dissuade further use.

Note also: we REQUIRE minimal `features` field in
channel_announcement, since otherwise both sides of channel will not
agree and not be able to create their signatures!

Consider these theoretical future features:

`opt_dlog_chan`: a new channel type which uses a new discrete log HTLC
type, but can't support traditional HTLC:

* `init`: presents as odd (optional) or even (if traditional channels
  not supported)
* `node_announcement`: the same as above, so you can seek suitable peers.
* `channel_announcement`: presents as even (compulsory), since users need
  to use the new HTLCs.

`opt_wumbochan`: a node which allows channels > 2^24 satoshis:

* `init`: presents as odd (optional), or maybe even (if you only want
  giant channels)
* `node_announcement`: the same as above, so you can seek suitable peers.
* `channel_announcement`: not present, since size of channel indicates
  capacity.

`opt_wumbohtlc`: a channel which allows HTLCs > 2^32 millisatoshis:

* `init`: presents as odd (optional), or even (compulsory)
* `node_announcement`: the same as above, so you can seek suitable peers.
* `channel_announcement`: odd (optional) since you can use the channel
  without understanding what this option means.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Co-Authored-By: Bastien Teinturier <31281497+t-bast@users.noreply.github.com>

The feature fields refer to the properties of the channel/node, not the
message itself, so we can still propagate them (and should, to avoid
splitting the network).

If we want to make an incompatible announcement message, we'll use a
different type, or insert an even TLV type.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

A bit less dense, but avoids a separate feature space.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

I recently made a cut & paste bug with the protocol tests, and
paid an HTLC of amount 100M msat, but with only a 1M msat `amt_to_forward`
in the hop_data.  To my surprise, it was accepted.

This is because we allow overpaying the routing fee (considered 0
for the final hop).  This doesn't make sense for the final hop: anything
but exact equality implies a bug, or that the previous node took the
wrong amount from the payment.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

We also define what the basic_mpp feature means in an invoice, by
reference to the next commit.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

This also defines the TLV format for payment_secret; the two are intertwined.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

This means the BOLT11 invoice must offer it (we already say it must
set the field if it offers it), and that the receiving node must
require it (again, we already say it must check it if it requires it).

Without the payment_secret, MPP payments are especially vulnerable to
probing attacks: unlike normal payments (with amounts) they can be
detected with 1msat payment probes.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

*Require* payment_secret for multi-part payments

BOLT 4 explicitly indicates var_onion_optin may appear in a BOLT 11
invoice, however, BOLT 9 only indicates it is available in init and
node_announcement contextx. Resolve this conflict in favor of BOLT 4
as there doesn't seem to be much reason to *not* allow it in BOLT
11 invoices.

This appears to have been an oversight in the flat features spec,
and is somewhat implicitly relied on for several new feature bits -
if var_onion_optin is set on a node_announcement (its not allowed
on a channel_announcement), then trying to route through that node
using the pre-tlv formt is somewhat nonsensical, and should be
forbidden.

Resolve two spec oddities regarding new features.

Update a requirement that was missed in lightning#711

…lightning#722)

As reading of commit 6729755 shows, `final_expiry_too_soon` was
17, not PERM|17.

Note that because we folded a previously non-permanent failure into
the now-permanent PERM|15 failure code, modifications to payment
algorithms may now be needed to specificalyl detect this case,
otherwise payment algorithms may give up in some edge cases where
blocks are mined while payments are in-transit between sender and
receiver.

This commit:
 - Adds a new Dependencies column to the BOLT 9 feature table
   populated with existing feature dependencies.
 - Requires that all valid feature vectors set transitive dependencies.
 - Requires checking transitive dependencies when validating init
   messages and payment request.
 - Removes transitive feature requiremetns from the BOLT 11 writer, now
   that they are implicit by needing to comply with the BOLT 9 origin
   requirements.

As a final step, we now can remove several of the BOLT 11 writer's
requirements now that it builds on BOLT 9's, particularly:
 - setting the even bit if a feature is required.
 - only setting a feature if the node supports a given feature.

The lone requirement that remains pertains to setting the `s` value if
and only if the `payment_secret` feature is set.

In this commit, we modify the existing instructions to create the Sphinx
packet to no longer start out with a zero initialize set of 1366 bytes.
Instead, we now instruct the sender to use _random_ bytes derived from a
CSPRG. This fixes a recently discovered privacy leak that allows an
adversarial exit hop to ascertain a lower bound on the true path length.

Note that this doesn't affect packet processing, so this is a backwards
compatible change. Only clients need to update in order to avoid this
privacy leak.

After this change is applied, the test vectors as is don't match the
spec, as they're created using the original all zero starting bytes. We
can either update these with our specified set of random bytes, or leave
them as is, as they're fully deterministic as is.

An alternative path would be to generate more random bytes from the
shared secret as we do elsewhere (the chacha based CSPRNG).

… padding

Add missing parenthesis in link.

Specify that resolution of amount is msat

As more nodes on the network use timestamp_filter to block gossip
floods, we've seen some propagation problems.  This should avoid it
(and is implemented now by c-lightning).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

... to make it consistent with the `where_to_put_secret` above.

…lightning#742)

ECDSA signatures in Bitcoin are DER-encoded but public keys are not.

The compressed format for public keys is for example standardized in
Sections 2.3.3 and 2.3.4 of

  Standards for Efficient Cryptography, SEC 1: Elliptic Curve
  Cryptography, Certicom Research, Version 2, 2009,
  https://www.secg.org/sec1-v2.pdf

Add option to open large channels (wumbo).

Add networks to init message via TLV extension.
This prevents accidentally connecting a testnet node to mainnet or the other way around.

Use `hmac` (lower-case) in packet fields to stay consistent with other fields.

We are using lexixographic order (which is the same as big-endian
when inputs have the same size, such as compressed public keys).

I initially used JSON, but it was not human-writable.  The simple
DSL is much more human-writable, and readable.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

Works with current c-lightning master branch, as long as it's built with
--enable-developer.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

Very simple successful open.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

Does a simple dust HTLC and a non-dust HTLC.  Both fail.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

This helps for testing that we handle it correctly.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

Preliminary parsing for subtypes. Probably need something on the
unpacking side also.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

Useful for gossip tests, antonym of maybe-send.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

Unpacking and comparison for subtypes.

make sure that the funds have hit the wallet before continuing
node operation

Adds a command flag for command flags. Passed down to the
runner on start. Ex:

   -f max-allowed=0 -f no-fuzz=true

if the values are empty, match them up

For dual-funded tests, we need a way to verify that witnesses are
the correct length instead of the correct contents. This is a bit
'complicated' because witnesses are in the range of 71-73 bytes long.

This patch allows a byte length (or range) to be specified by using
`*73` or `*71-73`, as in the case for a range of valid lengths.

New test runner event, gives the node under test time to confirm
that they do in fact have funds in their wallet before proceeding.

Allows dual-funding tests to specify a different set of outputs to
be used as inputs for the funding transaction

This reverts commit 09d9c80.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

…cts.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

Otherwise combinatorial explosion buries us in gossip alternatives.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

We still treat them as blobs of hex, but we don't base it on the suffix.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

now that we support dual_fund and static_remotekey, we needed
to update the init tests

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

You can still override on cmdline.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

Feature bit is now 13 (0x80000), and we include the
my_current_per_commitment_point with option_static_remotekey; we just
ignore it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

They worked for about two weeks after I wrote them!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

Will be used later for fundchannel process

Allow for the test to specify the feerate for a fundchannel open

Build out the command for fundchannel, originates open_channel
from node under test

If we exit before fundchannel has finished, it shows errors; let's
swallow the errors if they're happening on/after shutdown

We need to make sure that any fundchannel process is killed by stop,
so that we can run multiple fundchannel attempts in the same go

Also removes 'shutdown(wait=False)' since it doesn't do anything,
really.

Simple 'open channel' check from the opener's perspective.  Uses the
'new' fundchannel impl for c-lightning.

Moving c-lightning over to 'ground' sigs means that it'd be more
flexible if the protocol test framework allowed for dynamically
generating or verifying sigs based on a hash + private key.

Uses ':' notation as a separator so that `htlc_signature` list
parsing will still work without modification