Skip to content

Commit 3cf46fd

Browse files
t-bastt-bast
committed
Advertize known compression algorithms
Add a tlv field in `init` to list supported compression algorithms. This compression will be used in several places, currently in extended gossip queries. Fixes #811
1 parent 0dc3af8 commit 3cf46fd

File tree

3 files changed

+53
-26
lines changed

3 files changed

+53
-26
lines changed

‎01-messaging.md‎

Lines changed: 21 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -264,10 +264,20 @@ The `features` field MUST be padded to bytes with 0s.
264264
1. type: 1 (`networks`)
265265
2. data:
266266
* [`...*chain_hash`:`chains`]
267-
267+
1. type: 3 (`compression_algorithms`)
268+
2. data:
269+
* [`...*byte`:`supported_algorithms`]
268270

269271
The optional `networks` indicates the chains the node is interested in.
270272

273+
The optional `compression_algorithms` is a bitfield indicating what compression
274+
algorithms the node supports for [gossip queries](07-routing-gossip.md#query-messages).
275+
276+
| Bit Position | Algorithm |
277+
| ------------- | ---------------------------------------------------- |
278+
| 0 | Uncompressed |
279+
| 1 | [zlib_deflate](https://www.ietf.org/rfc/rfc1950.txt) |
280+
271281
#### Requirements
272282

273283
The sending node:
@@ -277,6 +287,8 @@ The sending node:
277287
- SHOULD NOT set features greater than 13 in `globalfeatures`.
278288
- SHOULD use the minimum length required to represent the `features` field.
279289
- SHOULD set `networks` to all chains it will gossip or open channels for.
290+
- if it sets the `option_compression` feature bit:
291+
- MUST set the `compression_algorithms` field
280292

281293
The receiving node:
282294
- MUST wait to receive `init` before sending any other messages.
@@ -290,6 +302,9 @@ The receiving node:
290302
- MAY fail the connection.
291303
- if the feature vector does not set all known, transitive dependencies:
292304
- MUST fail the connection.
305+
- if it has no `compression_algorithms` in common with the sending node:
306+
- MUST NOT exchange messages that require compression (e.g. gossip queries).
307+
- MAY fail the connection.
293308

294309
#### Rationale
295310

@@ -459,6 +474,7 @@ multi-byte values with big-endian.
459474
Values encoded with BigSize will produce an encoding of either 1, 3, 5, or 9
460475
bytes depending on the size of the integer. The encoding is a piece-wise
461476
function that takes a `uint64` value `x` and produces:
477+
462478
```
463479
uint8(x) if x < 0xfd
464480
0xfd + be16(uint16(x)) if x < 0x10000
@@ -477,6 +493,7 @@ decoded with BigSize should be checked to ensure they are minimally encoded.
477493
### BigSize Decoding Tests
478494

479495
The following is an example of how to execute the BigSize decoding tests.
496+
480497
```golang
481498
func testReadBigSize(t *testing.T, test bigSizeTest) {
482499
var buf [8]byte
@@ -499,6 +516,7 @@ func testReadBigSize(t *testing.T, test bigSizeTest) {
499516
```
500517

501518
A correct implementation should pass against these test vectors:
519+
502520
```json
503521
[
504522
{
@@ -607,6 +625,7 @@ A correct implementation should pass against these test vectors:
607625
### BigSize Encoding Tests
608626

609627
The following is an example of how to execute the BigSize encoding tests.
628+
610629
```golang
611630
func testWriteBigSize(t *testing.T, test bigSizeTest) {
612631
var (
@@ -627,6 +646,7 @@ func testWriteBigSize(t *testing.T, test bigSizeTest) {
627646
```
628647

629648
A correct implementation should pass against the following test vectors:
649+
630650
```json
631651
[
632652
{

‎07-routing-gossip.md‎

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -566,9 +566,12 @@ request what gossip should be received.
566566
There are several messages which contain a long array of
567567
`short_channel_id`s (called `encoded_short_ids`) so we utilize a
568568
simple compression scheme: the first byte indicates the encoding, the
569-
rest contains the data.
569+
rest contains the data. The compression algorithms supported by each
570+
node are advertised in the `init` message's `compression_algorithms`
571+
tlv field.
570572

571573
Encoding types:
574+
572575
* `0`: uncompressed array of `short_channel_id` types, in ascending order.
573576
* `1`: array of `short_channel_id` types, in ascending order, compressed with zlib deflate<sup>[1](#reference-1)</sup>
574577

@@ -634,6 +637,7 @@ The sender:
634637
- MUST set `chain_hash` to the 32-byte hash that uniquely identifies the chain
635638
that the `short_channel_id`s refer to.
636639
- MUST set the first byte of `encoded_short_ids` to the encoding type.
640+
- MUST NOT use an encoding not supported by the remote peer.
637641
- MUST encode a whole number of `short_channel_id`s to `encoded_short_ids`
638642
- MAY send this if it receives a `channel_update` for a
639643
`short_channel_id` for which it has no `channel_announcement`.

‎09-features.md‎

Lines changed: 27 additions & 24 deletions
Original file line numberDiff line numberDiff line change
@@ -19,28 +19,30 @@ for use of the channel, so the presentation of those features depends
1919
on the feature itself.
2020

2121
The Context column decodes as follows:
22+
2223
* `I`: presented in the `init` message.
2324
* `N`: presented in the `node_announcement` messages
2425
* `C`: presented in the `channel_announcement` message.
2526
* `C-`: presented in the `channel_announcement` message, but always odd (optional).
2627
* `C+`: presented in the `channel_announcement` message, but always even (required).
2728
* `9`: presented in [BOLT 11](11-payment-encoding.md) invoices.
2829

29-
| Bits | Name | Description | Context | Dependencies | Link |
30-
|-------|----------------------------------|-----------------------------------------------------------|----------|-------------------|---------------------------------------|
31-
| 0/1 | `option_data_loss_protect` | Requires or supports extra `channel_reestablish` fields | IN | | [BOLT #2][bolt02-retransmit] |
32-
| 3 | `initial_routing_sync` | Sending node needs a complete routing information dump | I | | [BOLT #7][bolt07-sync] |
33-
| 4/5 | `option_upfront_shutdown_script` | Commits to a shutdown scriptpubkey when opening channel | IN | | [BOLT #2][bolt02-open] |
34-
| 6/7 | `gossip_queries` | More sophisticated gossip control | IN | | [BOLT #7][bolt07-query] |
35-
| 8/9 | `var_onion_optin` | Requires/supports variable-length routing onion payloads | IN9 | | [Routing Onion Specification][bolt04] |
36-
| 10/11 | `gossip_queries_ex` | Gossip queries can include additional information | IN | `gossip_queries` | [BOLT #7][bolt07-query] |
37-
| 12/13 | `option_static_remotekey` | Static key for remote output | IN | | [BOLT #3](03-transactions.md) |
38-
| 14/15 | `payment_secret` | Node supports `payment_secret` field | IN9 | `var_onion_optin` | [Routing Onion Specification][bolt04] |
39-
| 16/17 | `basic_mpp` | Node can receive basic multi-part payments | IN9 | `payment_secret` | [BOLT #4][bolt04-mpp] |
40-
| 18/19 | `option_support_large_channel` | Can create large channels | IN | | [BOLT #2](02-peer-protocol.md#the-open_channel-message) |
41-
| 20/21 | `option_anchor_outputs` | Anchor outputs | IN | `option_static_remotekey` | [BOLT #3](03-transactions.md) |
42-
| 22/23 | `option_anchors_zero_fee_htlc_tx` | Anchor commitment type with zero fee HTLC transactions | IN | | [BOLT #3][bolt03-htlc-tx], [lightning-dev][ml-sighash-single-harmful]|
43-
| 26/27 | `option_shutdown_anysegwit` | Future segwit versions allowed in `shutdown` | IN | | [BOLT #2][bolt02-shutdown] |
30+
| Bits | Name | Description | Context | Dependencies | Link |
31+
|-------|-----------------------------------|-----------------------------------------------------------|----------|---------------------------|-----------------------------------------------------------------------|
32+
| 0/1 | `option_data_loss_protect` | Requires or supports extra `channel_reestablish` fields | IN | | [BOLT #2][bolt02-retransmit] |
33+
| 3 | `initial_routing_sync` | Sending node needs a complete routing information dump | I | | [BOLT #7][bolt07-sync] |
34+
| 4/5 | `option_upfront_shutdown_script` | Commits to a shutdown scriptpubkey when opening channel | IN | | [BOLT #2][bolt02-open] |
35+
| 6/7 | `gossip_queries` | More sophisticated gossip control | IN | | [BOLT #7][bolt07-query] |
36+
| 8/9 | `var_onion_optin` | Requires/supports variable-length routing onion payloads | IN9 | | [Routing Onion Specification][bolt04] |
37+
| 10/11 | `gossip_queries_ex` | Gossip queries can include additional information | IN | `gossip_queries` | [BOLT #7][bolt07-query] |
38+
| 12/13 | `option_static_remotekey` | Static key for remote output | IN | | [BOLT #3](03-transactions.md) |
39+
| 14/15 | `payment_secret` | Node supports `payment_secret` field | IN9 | `var_onion_optin` | [Routing Onion Specification][bolt04] |
40+
| 16/17 | `basic_mpp` | Node can receive basic multi-part payments | IN9 | `payment_secret` | [BOLT #4][bolt04-mpp] |
41+
| 18/19 | `option_support_large_channel` | Can create large channels | IN | | [BOLT #2](02-peer-protocol.md#the-open_channel-message) |
42+
| 20/21 | `option_anchor_outputs` | Anchor outputs | IN | `option_static_remotekey` | [BOLT #3](03-transactions.md) |
43+
| 22/23 | `option_anchors_zero_fee_htlc_tx` | Anchor commitment type with zero fee HTLC transactions | IN | | [BOLT #3][bolt03-htlc-tx], [lightning-dev][ml-sighash-single-harmful] |
44+
| 26/27 | `option_shutdown_anysegwit` | Future segwit versions allowed in `shutdown` | IN | | [BOLT #2][bolt02-shutdown] |
45+
| 32/33 | `option_compression` | Compression algorithms advertised in `init` | IN | | [BOLT #1](01-messaging.md#the-init-message) |
4446

4547
## Definitions
4648

@@ -49,15 +51,16 @@ We define `option_anchors` as `option_anchor_outputs || option_anchors_zero_fee_
4951
## Requirements
5052

5153
The origin node:
52-
* If it supports a feature above, SHOULD set the corresponding odd
53-
bit in all feature fields indicated by the Context column unless
54-
indicated that it must set the even feature bit instead.
55-
* If it requires a feature above, MUST set the corresponding even
56-
feature bit in all feature fields indicated by the Context column,
57-
unless indicated that it must set the odd feature bit instead.
58-
* MUST NOT set feature bits it does not support.
59-
* MUST NOT set feature bits in fields not specified by the table above.
60-
* MUST set all transitive feature dependencies.
54+
55+
* If it supports a feature above, SHOULD set the corresponding odd
56+
bit in all feature fields indicated by the Context column unless
57+
indicated that it must set the even feature bit instead.
58+
* If it requires a feature above, MUST set the corresponding even
59+
feature bit in all feature fields indicated by the Context column,
60+
unless indicated that it must set the odd feature bit instead.
61+
* MUST NOT set feature bits it does not support.
62+
* MUST NOT set feature bits in fields not specified by the table above.
63+
* MUST set all transitive feature dependencies.
6164

6265
The requirements for receiving specific bits are defined in the linked sections in the table above.
6366
The requirements for feature bits that are not defined

0 commit comments

Comments
 (0)