Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Test init networks TLV #13

Open
wants to merge 110 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
110 commits
Select commit Hold shift + click to select a range
a5b3818
Specify that resolution of amount is msat
Sword-Smith Nov 14, 2019
efd16b9
fix <id> tag
nayuta-ueno Nov 14, 2019
3a0a7fd
remove `funding_locked` future section
nayuta-ueno Nov 12, 2019
b84d09e
bolt07: remove trailing tabs
darosior Oct 1, 2019
5f57ee3
BOLT-02: Fix link to channel_id section (#704)
ysangkok Nov 22, 2019
206084c
BOLT 9: flatten feature fields.
rustyrussell Sep 17, 2019
6502e30
BOLT 7: always propagate announcements with unknown features.
rustyrussell Sep 17, 2019
8e69306
BOLT 11: use the same features for BOLT11 invoices as for others.
rustyrussell Oct 30, 2019
0fb66ca
Minor href fix in Contents of 07-routing-gossip.md (#707)
arvchristos Dec 9, 2019
2422630
BOLT 4: don't allow a "fee" for the final node.
rustyrussell Dec 3, 2019
b2a3c2f
BOLT 9: Add feature bits for payment_secret and basic_mpp.
rustyrussell Nov 26, 2019
5776d2a
BOLT 11: Add payment_secret field (compulsory for new invoices).
rustyrussell Nov 26, 2019
4c3d016
BOLT 4: Multi-part payments.
rustyrussell Nov 26, 2019
6ad8ee4
BOLT 4/11: require payment_secret for multi-part payments.
rustyrussell Dec 4, 2019
138ee87
Merge pull request #712 from rustyrussell/rebased-amp
cfromknecht Dec 16, 2019
44c5fa1
Resolve conflict between BOLT 4&9 re: var_onion_optin context
TheBlueMatt Dec 24, 2019
5abee4d
Do not allow routing to a node with unkown feature bits set.
TheBlueMatt Dec 24, 2019
5f2c0ef
Merge pull request #723 from TheBlueMatt/master
cfromknecht Jan 8, 2020
f219ee0
#711 don't allow a "fee" for the final node. (#718)
nayuta-ueno Jan 8, 2020
b62c878
BOLT09: fix incorrect hyperlink text for option_mpp (#725)
cfromknecht Jan 10, 2020
11f6017
04-onion-routing.md: Fix factual error about `final_expiry_too_soon`.…
ZmnSCPxj Jan 10, 2020
4c638b7
09+11: require transitive feature dependencies
cfromknecht Jan 17, 2020
1259f8f
BOLT11: set TLV bit in payment secret test vectors
cfromknecht Jan 17, 2020
53653e5
BOLT 04: add missing subsections to ToC
cfromknecht Jan 17, 2020
c3a8e5e
BOLT11: simplify existing writer feature requirements
cfromknecht Jan 17, 2020
caca437
BOLT 1: Define custom message type range
t-bast Nov 21, 2019
798ff4b
fixup! Specify that resolution of amount is msat
cdecker Jan 21, 2020
29f1386
fixup! Specify that resolution of amount is msat
cdecker Jan 21, 2020
8dd0b75
BOLT-04: modify Sphinx packet construction to use starting random bytes
Roasbeef Nov 6, 2019
7c1edeb
bolt04: minor JSON fix and generate the exact number of bytes for the…
cdecker Jan 21, 2020
75f46ba
Update 01-messaging.md (#732)
kiminuo Jan 29, 2020
17df7f2
Merge pull request #700 from Sword-Smith/patch-3
cdecker Jan 29, 2020
0bb69d3
Fix bad wording of amount requirement in Bolt11 (#733)
Sword-Smith Jan 31, 2020
458b0d3
BOLT 7: be more aggressive about sending our own gossip.
rustyrussell Oct 14, 2019
2afe097
Fix a typo in `insert_secret` pseudo code (#741)
Feb 14, 2020
fb7102e
Remove reference to DER encoding for public keys in compressed format…
real-or-random Feb 17, 2020
3847935
Single-option large channel proposal (#596)
araspitzu Feb 18, 2020
7b01692
BOLT 1: add networks to init message. (#682)
rustyrussell Feb 18, 2020
a2afdfd
Keep `hmac` case consistent (#547)
OrfeasLitos Feb 18, 2020
dcbf858
Clarify numerical comparison of pubkeys (#743)
t-bast Feb 18, 2020
eccb27b
tests: spec for test vectors, and a simple example.
rustyrussell Aug 5, 2019
077af84
tests: c-lightning support.
rustyrussell Aug 5, 2019
d940f77
tests: channel opening test.
rustyrussell Aug 5, 2019
8e46a95
tests: HTLC-adding test.
rustyrussell Aug 5, 2019
04eb8e3
tests: add reconnection tests to add-htlc.
rustyrussell Aug 5, 2019
d67cf5a
tests: htlc success tests.
rustyrussell Aug 5, 2019
73a04e7
tests: make to_self_delay different between peers.
rustyrussell Aug 5, 2019
8684b8b
tests/events: split htlc-add and htlc-fail tests.
rustyrussell Aug 5, 2019
abb41ed
tests/events: test unknown messages.
rustyrussell Aug 5, 2019
b6bfb47
fixup: unpack handle TLV fields
niftynei Aug 12, 2019
d250ebe
rfc-tests: handle subtype parsing
niftynei Aug 12, 2019
97eb45a
WIP: handle unpack subtypes
niftynei Aug 12, 2019
20ed1ad
Keep flake8 happy.
rustyrussell Aug 15, 2019
06b9ec3
tests: add must-not-send
rustyrussell Aug 15, 2019
6dfb24b
protocol tests: handle unpack subtypes
niftynei Aug 3, 2019
20c1304
protcol tests: for c-lightning driver, ensure funds in wallet
niftynei Aug 6, 2019
6592287
protocol tests: allow implementation specific flags to be passed in
niftynei Aug 6, 2019
d33705e
subtypes: match empty values
niftynei Aug 13, 2019
1c74067
rebase errors, fixup
niftynei Aug 16, 2019
afb87dc
Don't count arrays of bytes as an integer type
niftynei Aug 16, 2019
da66e11
Allow check for field length instead of contents
niftynei Aug 16, 2019
1a863ac
new event 'wait-funds'
niftynei Aug 16, 2019
3b1a9a0
Move funding output block broadcast into tests
niftynei Aug 17, 2019
eebe7ae
Revert "new event 'wait-funds'"
rustyrussell Aug 28, 2019
50213ad
test-spec.md: update with length options.
rustyrussell Aug 28, 2019
5cabd01
tools/test-events.py: handle indention which occurs at EOF.
rustyrussell Aug 28, 2019
00b5f5f
fixup invalidation logic
rustyrussell Aug 28, 2019
a959ecc
tools/test-events-clightning.py: give error location if node disconne…
rustyrussell Aug 28, 2019
ad8ff1c
fix up 'conn=' parsing.
rustyrussell Aug 28, 2019
3edaa8a
give proper information when unknown event type
rustyrussell Aug 28, 2019
cbdca18
Implement 'Any order'.
rustyrussell Aug 28, 2019
487cc61
handle tlvs in a slightly less hacky way.
rustyrussell Aug 29, 2019
86926b0
Fix parsing of signature hex fields.
rustyrussell Aug 29, 2019
e7380c2
Add setup.incl which contains convenience vars.
rustyrussell Aug 29, 2019
46c4139
Simple gossip tests, part 1.
rustyrussell Aug 29, 2019
7e39c2c
Basic gossip_timestamp_filter tests.
rustyrussell Aug 29, 2019
4f3fe68
Basic query_channel_range tests.
rustyrussell Aug 29, 2019
0fab8a6
basic query_short_channel_ids tests.
rustyrussell Aug 29, 2019
841a4a4
query_short_channel_ids tests for option_gossip_queries_ex.
rustyrussell Aug 29, 2019
fabde2b
query_channel_range tests for option_gossip_queries_ex
rustyrussell Aug 29, 2019
fd408dc
tests: add tests for option_static_remotekey.
rustyrussell Aug 29, 2019
f99c7ab
featurebits: update feature bit tests
niftynei Oct 10, 2019
a42c38a
test-events.py: Fix printing of unexpected fields.
rustyrussell Nov 12, 2019
4a9720e
tools/test-events-clightning: autodetect supported features.
rustyrussell Nov 12, 2019
a811778
tools/test-events-clightning: use modern -dev options
rustyrussell Nov 12, 2019
0e98ecb
tests: fix option_static_remotekey
rustyrussell Nov 12, 2019
5e9c614
tools/test-events-clightning.py: fix time so gossip tests work.
rustyrussell Nov 12, 2019
254ad12
tests: Add shutdown for executor
niftynei Oct 31, 2019
b8af700
tests: add new error type for internal errors
niftynei Oct 31, 2019
01dbfc6
tests: add feerate to fundchannel command
niftynei Oct 31, 2019
4b476db
test: add fundchannel implementation to c-lightning runner
niftynei Oct 31, 2019
3fc25c0
test: cleaner shutdown of fundchannel on exit
niftynei Oct 31, 2019
be5ddaa
tests: kill fundchannel on stop
niftynei Nov 1, 2019
a0276fc
use dev-force-tmp-channel-id, needed for opener tests
niftynei Nov 4, 2019
647da25
tests: test case for basic opening a channel (opener)
niftynei Nov 6, 2019
50de0f7
tests: add capability to dynamically generate/verify sigs
niftynei Nov 13, 2019
6a3237c
tests: update signatures to use SIG() notation
niftynei Nov 13, 2019
cc29195
tests: consolidate openchannel tests; use SIG
niftynei Nov 13, 2019
e43238d
tools: update c-lightning runner for modern per-network paths.
rustyrussell Dec 2, 2019
2049a6e
tests: update bolt1-01-init "high-numbered" features now 18 and 19 ar…
rustyrussell Dec 2, 2019
8d5ef1e
tools: fix test-events-clightning.py funchannel.
rustyrussell Dec 2, 2019
7a01332
tests: Fix broken links in tests/events/README.md
jkczyz Dec 2, 2019
58d9b9f
tests: Fix grammatical and consistency errors
jkczyz Dec 3, 2019
a6c90a8
tests: Fix `Any order` and `One of` event examples
jkczyz Dec 3, 2019
d73824a
tests: Fix INPUT_EVENT specification
jkczyz Dec 3, 2019
6ff22bc
events: rename 'localfeatures' to 'features'
darosior Dec 25, 2019
46fdceb
BOLT 1: add networks to init message.
rustyrussell Oct 3, 2019
6bf9e63
tests: add the network's chain_hash to the test spec
darosior Dec 25, 2019
92f3d9f
tools/test-events-clightning.py: correct an indent error
darosior Dec 25, 2019
66f2f6d
events: add tests for init tlvs
darosior Dec 25, 2019
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .aspell.en.pws
Original file line number Diff line number Diff line change
Expand Up @@ -367,7 +367,12 @@ fips
rfc
varint
CompactSize
multipath
mpp
tlvs
snprintf
GitHub
IRC
bitmasks
CSPRNG
lexicographically
55 changes: 42 additions & 13 deletions 01-messaging.md
Original file line number Diff line number Diff line change
Expand Up @@ -51,12 +51,13 @@ A receiving node:
- upon receiving a message of _even_, unknown type:
- MUST fail the channels.

The messages are grouped logically into four groups, ordered by the most significant bit that is set:
The messages are grouped logically into five groups, ordered by the most significant bit that is set:

- Setup & Control (types `0`-`31`): messages related to connection setup, control, supported features, and error reporting (described below)
- Channel (types `32`-`127`): messages used to setup and tear down micropayment channels (described in [BOLT #2](02-peer-protocol.md))
- Commitment (types `128`-`255`): messages related to updating the current commitment transaction, which includes adding, revoking, and settling HTLCs as well as updating fees and exchanging signatures (described in [BOLT #2](02-peer-protocol.md))
- Routing (types `256`-`511`): messages containing node and channel announcements, as well as any active route exploration (described in [BOLT #7](07-routing-gossip.md))
- Custom (types `32768`-`65535`): experimental and application-specific messages

The size of the message is required by the transport layer to fit into a 2-byte unsigned int; therefore, the maximum possible size is 65535 bytes.

Expand All @@ -66,6 +67,13 @@ A node:
- MUST fail the channels.
- that negotiates an option in this specification:
- MUST include all the fields annotated with that option.
- When defining custom messages:
- SHOULD pick a random `type` to avoid collision with other custom types.
- SHOULD pick a `type` that doesn't conflict with other experiments listed in [this issue](https://github.com/lightningnetwork/lightning-rfc/issues/716).
- SHOULD pick an odd `type` identifiers when regular nodes should ignore the
additional data.
- SHOULD pick an even `type` identifiers when regular nodes should reject
the message and close the connection.

### Rationale

Expand Down Expand Up @@ -100,7 +108,7 @@ A `tlv_record` represents a single field, encoded in the form:
A `varint` is a variable-length, unsigned integer encoding using the
[BigSize](#appendix-a-bigsize-test-vectors) format, which resembles the bitcoin
CompactSize encoding but uses big-endian for multi-byte values rather than
little-endian.
little-endian.

A `tlv_stream` is a series of (possibly zero) `tlv_record`s, represented as the
concatenation of the encoded `tlv_record`s. When used to extend existing
Expand Down Expand Up @@ -214,7 +222,7 @@ integers can be omitted:
The following convenience types are also defined:

* `chain_hash`: a 32-byte chain identifier (see [BOLT #0](00-introduction.md#glossary-and-terminology-guide))
* `channel_id`: a 32-byte channel_id (see [BOLT #2](02-peer-protocol.md#definition-of-channel-id)
* `channel_id`: a 32-byte channel_id (see [BOLT #2](02-peer-protocol.md#definition-of-channel-id))
* `sha256`: a 32-byte SHA2-256 hash
* `signature`: a 64-byte bitcoin Elliptic Curve signature
* `point`: a 33-byte Elliptic Curve point (compressed encoding as per [SEC 1 standard](http://www.secg.org/sec1-v2.pdf#subsubsection.2.3.3))
Expand All @@ -226,45 +234,66 @@ The following convenience types are also defined:

Once authentication is complete, the first message reveals the features supported or required by this node, even if this is a reconnection.

[BOLT #9](09-features.md) specifies lists of global and local features. Each feature is generally represented in `globalfeatures` or `localfeatures` by 2 bits. The least-significant bit is numbered 0, which is _even_, and the next most significant bit is numbered 1, which is _odd_.
[BOLT #9](09-features.md) specifies lists of features. Each feature is generally represented by 2 bits. The least-significant bit is numbered 0, which is _even_, and the next most significant bit is numbered 1, which is _odd_. For historical reasons, features are divided into global and local feature bitmasks.

Both fields `globalfeatures` and `localfeatures` MUST be padded to bytes with 0s.
The `features` field MUST be padded to bytes with 0s.

1. type: 16 (`init`)
2. data:
* [`u16`:`gflen`]
* [`gflen*byte`:`globalfeatures`]
* [`u16`:`lflen`]
* [`lflen*byte`:`localfeatures`]
* [`u16`:`flen`]
* [`flen*byte`:`features`]
* [`init_tlvs`:`tlvs`]

The 2-byte `gflen` and `lflen` fields indicate the number of bytes in the immediately following field.
1. tlvs: `init_tlvs`
2. types:
1. type: 1 (`networks`)
2. data:
* [`...*chain_hash`:`chains`]


The optional `networks` indicates the chains the node is interested in.

#### Requirements

The sending node:
- MUST send `init` as the first Lightning message for any connection.
- MUST set feature bits as defined in [BOLT #9](09-features.md).
- MUST set any undefined feature bits to 0.
- SHOULD use the minimum lengths required to represent the feature fields.
- SHOULD NOT set features greater than 13 in `globalfeatures`.
- SHOULD use the minimum length required to represent the `features` field.
- SHOULD set `networks` to all chains it will gossip or open channels for.

The receiving node:
- MUST wait to receive `init` before sending any other messages.
- MUST combine (logical OR) the two feature bitmaps into one logical `features` map.
- MUST respond to known feature bits as specified in [BOLT #9](09-features.md).
- upon receiving unknown _odd_ feature bits that are non-zero:
- MUST ignore the bit.
- upon receiving unknown _even_ feature bits that are non-zero:
- MUST fail the connection.
- upon receiving `networks` containing no common chains
- MAY fail the connection.
- if the feature vector does not set all known, transitive dependencies:
- MUST fail the connection.
- upon receiving `networks` containing no common chains
- MAY fail the connection.

#### Rationale

There used to be two feature bitfields here, but for backwards compatibility they're now
combined into one.

This semantic allows both future incompatible changes and future backward compatible changes. Bits should generally be assigned in pairs, in order that optional features may later become compulsory.

Nodes wait for receipt of the other's features to simplify error
diagnosis when features are incompatible.

The feature masks are split into local features (which only affect the
protocol between these two nodes) and global features (which can affect
HTLCs and are thus also advertised to other nodes).
Since all networks share the same port, but most implementations only
support a single network, the `networks` fields avoids nodes
erroneously believing they will receive updates about their preferred
network, or that they can open channels.

### The `error` Message

Expand Down Expand Up @@ -875,7 +904,7 @@ failure:

## References

1. <a id="reference-2">http://www.unicode.org/charts/PDF/U2600.pdf</a>
1. <a id="reference-1">http://www.unicode.org/charts/PDF/U2600.pdf</a>

## Authors

Expand Down
32 changes: 13 additions & 19 deletions 02-peer-protocol.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ operation, and closing.
# Table of Contents

* [Channel](#channel)
* [Definition of `channel_id`](#definition-of-channel-id)
* [Definition of `channel_id`](#definition-of-channel_id)
* [Channel Establishment](#channel-establishment)
* [The `open_channel` Message](#the-open_channel-message)
* [The `accept_channel` Message](#the-accept_channel-message)
Expand Down Expand Up @@ -187,16 +187,22 @@ The `shutdown_scriptpubkey` allows the sending node to commit to where
funds will go on mutual close, which the remote node should enforce
even if a node is compromised later.

[ FIXME: Describe dangerous feature bit for larger channel amounts. ]
The `option_support_large_channel` is a feature used to let everyone
know this node will accept `funding_satoshis` greater than or equal to 2^24.
Since it's broadcast in the `node_announcement` message other nodes can use it to identify peers
willing to accept large channel even before exchanging the `init` message with them.

#### Requirements

The sending node:
- MUST ensure the `chain_hash` value identifies the chain it wishes to open the channel within.
- MUST ensure `temporary_channel_id` is unique from any other channel ID with the same peer.
- MUST set `funding_satoshis` to less than 2^24 satoshi.
- if both nodes advertised `option_support_large_channel`:
- MAY set `funding_satoshis` greater than or equal to 2^24 satoshi.
- otherwise:
- MUST set `funding_satoshis` to less than 2^24 satoshi.
- MUST set `push_msat` to equal or less than 1000 * `funding_satoshis`.
- MUST set `funding_pubkey`, `revocation_basepoint`, `htlc_basepoint`, `payment_basepoint`, and `delayed_payment_basepoint` to valid DER-encoded, compressed, secp256k1 pubkeys.
- MUST set `funding_pubkey`, `revocation_basepoint`, `htlc_basepoint`, `payment_basepoint`, and `delayed_payment_basepoint` to valid secp256k1 pubkeys in compressed format.
- MUST set `first_per_commitment_point` to the per-commitment point to be used for the initial commitment transaction, derived as specified in [BOLT #3](03-transactions.md#per-commitment-secret-requirements).
- MUST set `channel_reserve_satoshis` greater than or equal to `dust_limit_satoshis`.
- MUST set undefined bits in `channel_flags` to 0.
Expand Down Expand Up @@ -234,19 +240,18 @@ The receiving node MUST fail the channel if:
- `max_accepted_htlcs` is greater than 483.
- it considers `feerate_per_kw` too small for timely processing or unreasonably large.
- `funding_pubkey`, `revocation_basepoint`, `htlc_basepoint`, `payment_basepoint`, or `delayed_payment_basepoint`
are not valid DER-encoded compressed secp256k1 pubkeys.
are not valid secp256k1 pubkeys in compressed format.
- `dust_limit_satoshis` is greater than `channel_reserve_satoshis`.
- the funder's amount for the initial commitment transaction is not sufficient for full [fee payment](03-transactions.md#fee-payment).
- both `to_local` and `to_remote` amounts for the initial commitment transaction are less than or equal to `channel_reserve_satoshis` (see [BOLT 3](03-transactions.md#commitment-transaction-outputs)).
- `funding_satoshis` is greater than or equal to 2^24 and the receiver does not support `option_support_large_channel`.

The receiving node MUST NOT:
- consider funds received, using `push_msat`, to be received until the funding transaction has reached sufficient depth.

#### Rationale

The requirement for `funding_satoshi` to be less than 2^24 satoshi is a temporary self-imposed limit while implementations are not yet considered stable.
It can be lifted at any point in time, or adjusted for other currencies, since it is solely enforced by the endpoints of a channel.
Specifically, [the routing gossip protocol](07-routing-gossip.md) does not discard channels that have a larger capacity.
The requirement for `funding_satoshis` to be less than 2^24 satoshi was a temporary self-imposed limit while implementations were not yet considered stable, it can be lifted by advertising `option_support_large_channel`.

The *channel reserve* is specified by the peer's `channel_reserve_satoshis`: 1% of the channel total is suggested. Each side of a channel maintains this reserve so it always has something to lose if it were to try to broadcast an old, revoked commitment transaction. Initially, this reserve may not be met, as only one side has funds; but the protocol ensures that there is always progress toward meeting this reserve, and once met, it is maintained.

Expand All @@ -264,12 +269,6 @@ are above both `dust_limit_satoshis`.

Details for how to handle a channel failure can be found in [BOLT 5:Failing a Channel](05-onchain.md#failing-a-channel).

#### Future

It would be easy to have a local feature bit which indicated that a
receiving node was prepared to fund a channel, which would reverse this
protocol.

### The `accept_channel` Message

This message contains information about a node and indicates its
Expand Down Expand Up @@ -425,11 +424,6 @@ funds are at risk. If the fundee were to remember the channel forever, this
would create a Denial of Service risk; therefore, forgetting it is recommended
(even if the promise of `push_msat` is significant).

#### Future

An SPV proof could be added and block hashes could be routed in separate
messages.

## Channel Close

Nodes can negotiate a mutual close of the connection, which unlike a
Expand Down
4 changes: 2 additions & 2 deletions 03-transactions.md
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,7 @@ Most transaction outputs used here are pay-to-witness-script-hash<sup>[BIP141](h

`2 <pubkey1> <pubkey2> 2 OP_CHECKMULTISIG`

* Where `pubkey1` is the numerically lesser of the two DER-encoded `funding_pubkey` and where `pubkey2` is the numerically greater of the two.
* Where `pubkey1` is the lexicographically lesser of the two `funding_pubkey` in compressed format, and where `pubkey2` is the lexicographically greater of the two.

## Commitment Transaction

Expand Down Expand Up @@ -575,7 +575,7 @@ A double-check, that all previous secrets derive correctly, is needed;
if this check fails, the secrets were not generated from the same seed:

insert_secret(secret, I):
B = where_to_put_secret(secret, I)
B = where_to_put_secret(I)

# This tracks the index of the secret in each bucket across the traversal.
for b in 0 to B:
Expand Down
Loading