Skip to content

Conversation

@lontivero
Copy link
Contributor

This is for keeping some progress and discussions about the coming light client (bip 157 and bip 158). Note that Bitcoin Core has a first on-review PR for Golomb-Rice filters (bitcoin/bitcoin#12254).

The idea is starting with the low-hanging fruit such as the more relevant p2p protocol message. Once it is compatible with btcd nodes, we could use it in WasabiWallet (far future)

@NicolasDorier
Copy link
Collaborator

Wow this is awesome! will read more on this.

@lontivero
Copy link
Contributor Author

Bitcoin Core team members are discussing about what info to include into the filters, how many filter types should exist and how to verify the filters are correct. Additionally depending on the amount and type of data, the P value (used to set the FP rate) could need to be recalculated.
https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2018-May/015972.html

@knocte
Copy link
Contributor

knocte commented Mar 16, 2020

Just curious, did BIP157 and BIP158 implementations ever got included into BitcoinCore?

@lontivero
Copy link
Contributor Author

Just curious, did BIP157 and BIP158 implementations ever got included into BitcoinCore?

  • BIP158: yes
  • BIP157: no

public enum FilterType : byte
{
Basic = (0x00),
Extended = (0x01)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What does extended filter hold?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cfilter is sent in response to getcfilters, one for each block in the
requested range.
cfcheckpt is sent in response to getcfcheckpt. The filter headers included are the set of all filter headers on the requested chain where the height is a positive multiple of 1,000.
cfheaders is sent in response to getcfheaders. Instead of including the filter headers themselves, the response includes one filter header and a sequence of filter hashes, from which the headers can be derived. This has the benefit that the client can verify the binding links between the headers
Contains what is common to all compact filters
queries:

* FilterType
* StartHeight
* StopHash
getcfilters is used to request the compact filters of a particular type for a particular range of blocks.
getcfheaders is used to request verifiable filter headers for a range of blocks.
getcfcheckpt is used to request filter headers at evenly spaced intervals over a range of blocks. Clients may use filter hashes from getcfheaders to connect these checkpoints,
@lontivero lontivero force-pushed the light-client-p2p-protocol branch from fdb8ca8 to aee05b8 Compare February 4, 2021 04:57
@lontivero lontivero changed the title [WIP] Light client p2p protocol Light client p2p protocol Feb 4, 2021
@lontivero lontivero marked this pull request as draft February 4, 2021 04:58
@lontivero
Copy link
Contributor Author

@NicolasDorier I would like to continue with this PR but I hit a wall that kept me blocked for months. I have implemented the p2p messages and I can see the network traffic is okay and that Bitcoin Core receives and replay okay so, that part is correct and works perfect.

The problem I have is I am not sure how to implement the compact filters and compact filter headers. Considering the bip 157 recommends to synchronize the block headers first, should implement this using something like the CompactFiltersChainBehavior? or should I modify the ChainBehavior? Do you think it is correct to reuse the SlimChain to keep the chain of compact filter headers?

I would appreciate if you could take a look at the bip 157 and provide me some guidance.


Btw, the commits that starts with [WIP] are just garbage but I pushed them to give you an idea of what I am talking about.

@lontivero
Copy link
Contributor Author

It seems the fact this PR is 401 has bad karma and will not be authorized. cuack!

Idk when i will find the time to continue with this so, what do you think about removing the [WIP] commits? Would you merge it?

@lontivero
Copy link
Contributor Author

@NicolasDorier I hate to break your zen but as every year, since 2018, I feel the need to go back to this PR.

What do you think about removing the [WIP] commits? Would you merge it? Thank you.

@NicolasDorier
Copy link
Collaborator

well I completely forgot this PR, if you think it's ready to merge let's merg eit.

}

//[Fact]
//[Trait("Protocol", "Protocol")]
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove this so the CI can test it.

@lontivero lontivero force-pushed the light-client-p2p-protocol branch from aee05b8 to e3cdfc4 Compare July 21, 2021 13:52
@lontivero lontivero marked this pull request as ready for review July 21, 2021 14:05
@lontivero
Copy link
Contributor Author

The PR contains only the protocol messages (payload). I removed the CompactFiltersChainBehavior and the integration test because I don't know how to implement it correctly.

@NicolasDorier NicolasDorier merged commit e144629 into MetacoSA:master Jul 24, 2021
@knocte
Copy link
Contributor

knocte commented Jul 26, 2021

@lontivero just curious, how far is this for a light-node to not depend on, for exampe, electrum servers?

@lontivero
Copy link
Contributor Author

It is still far from that because the bip recommends to download the block headers chain first and then download the compact filters chain.

You can take a look at this two commits in order to have an idea about how to make a naive implementation of this:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants