Troubleshooting Failed Bitcoin-related Test Cases in Compiling lnd from Source

[mohamedawnallah]

Hey everyone,

I'm in the process of compiling lnd from the source by following the instructions in the INSTALL.md doc. Everything seemed to install correctly, but I encountered issues when running the test cases using make check. All the tests passed except the ones associated with bitcoind. These failures seem to be related to the configuration and connection issues with bitcoind.

I've placed the necessary bitcoin-core binaries (such as bitcoind) in /usr/local/bin located in the $PATH. These binaries were downloaded from the latest version of bitcoin core 26.0, specifically using the version bitcoin-26.0-arm64-apple-darwin.tar.gz.

The error log I received after running make check includes messages like:

ok  	github.com/lightningnetwork/lnd/lnwallet/btcwallet	(cached)
--- FAIL: TestFilteredChainView (10.42s)
    interface_test.go:1010: Testing 'bitcoind_zmq' implementation of FilteredChainView
    interface_test.go:1022: unable to make chain view: unable to establish connection to bitcoind: method did not return within the timeout
FAIL
FAIL	github.com/lightningnetwork/lnd/routing/chainview	10.435s
FAIL
ok  	github.com/lightningnetwork/lnd/routing	(cached)
ok  	github.com/lightningnetwork/lnd/autopilot	(cached)
ok  	github.com/lightningnetwork/lnd/brontide	(cached)
ok  	github.com/lightningnetwork/lnd/cert	(cached)
--- FAIL: TestHistoricalConfDetailsTxIndex (0.00s)
    --- FAIL: TestHistoricalConfDetailsTxIndex/rpc_polling_enabled (10.32s)
        bitcoind_test.go:126: unable to establish connection to bitcoind: method did not return within the timeout
    --- FAIL: TestHistoricalConfDetailsTxIndex/rpc_polling_disabled (10.36s)
        bitcoind_test.go:126: unable to establish connection to bitcoind: method did not return within the timeout
--- FAIL: TestHistoricalConfDetailsNoTxIndex (0.00s)
    --- FAIL: TestHistoricalConfDetailsNoTxIndex/rpc_polling_enabled (10.37s)
        bitcoind_test.go:222: unable to establish connection to bitcoind: method did not return within the timeout
    --- FAIL: TestHistoricalConfDetailsNoTxIndex/rpc_polling_disabled (10.40s)
        bitcoind_test.go:222: unable to establish connection to bitcoind: method did not return within the timeout
FAIL
FAIL	github.com/lightningnetwork/lnd/chainntnfs/bitcoindnotify	20.776s
FAIL

My bitcoin.conf file is configured for regtest as follows:

# Generated by https://jlopp.github.io/bitcoin-core-config-generator/

# This config should be placed in following path:
# $HOME/Library/Application Support/Bitcoin/bitcoin.conf

regtest=1
server=1
daemon=1
debug=1
printtoconsole=1
debuglogfile=debug.log

# [Sections]
# Most options automatically apply to mainnet, testnet, and regtest networks.
# If you want to confine an option to just one network, you should add it in the relevant section.
# EXCEPTIONS: The options addnode, connect, port, bind, rpcport, rpcbind and wallet
# only apply to mainnet unless they appear in the appropriate section below.

# Options only for mainnet
[main]
prune=1000MB
rpcport=8332
rpcbind=0.0.0.0 # Note: For Testing purposes
rpcallowip=0.0.0.0/0 # Note: For Testing purposes
rpcuser=mainnet # Note: For Testing purposes
rpcpassword=mainnet # Note: For Testing purposes

# Options only for testnet
[test]

# Options only for regtest
[regtest]
fallbackfee=0.000001
port=18444
noconnect=1
dnsseed=0
dns=0
upnp=0
onlynet=ipv4
rpcport=18443
rpcbind=0.0.0.0 # Note: For Testing purposes
rpcallowip=0.0.0.0/0 # Note: For Testing purposes
rpcuser=regtest # Note: For Testing purposes
rpcpassword=regtest # Note: For Testing purposes
zmqpubrawblock=tcp://127.0.0.1:28332
zmqpubrawtx=tcp://127.0.0.1:28333

I'm running this on macOS m1 with the following system information:

Darwin Mohameds-MacBook-Air.local 23.0.0 Darwin Kernel Version 23.0.0: Fri Sep 15 14:41:34 PDT 2023; root:xnu-10002.1.13~1/RELEASE_ARM64_T8103 arm64

Any suggestions or insights on how to resolve these bitcoind-related test failures would be greatly appreciated! 🙏

[guggero]

Hmm, interesting. This might be an issue with v26.0 of bitcoind. I just ran the same test locally with v26.0 and got the same failure.
Or maybe our tests are just flaky (unfortunately some still are). Let me re-try with an earlier version of bitcoind to get some more info.

[guggero]

I haven't seen this dial unix //var/.../blocks.socket: connect: invalid argument before. But after googling a bit, I found this: containers/podman#13609 (comment)

So this seems to be some kind of length restriction. Do you know why things are written to var/folders/nw/qczy4tjj333g09tcy197q3s40000gn/T on your system instead of just /tmp/?

[mohamedawnallah]

I don't know actually I see these kind of issues for the docker containers engine because there are setting to disable saving/sharing data to /var/folders'. That said running the test cases using make check` wouldn't require docker engine to be installed. I've also spent considerable amount of time to know why this happen but actually failed. Also I have found similar discussion on Lightning Network related to the same issue but not answered yet:
#7015

[mohamedawnallah]

@guggero That said I run the lnd node from the source and I didn't face issues. I'm wondering if fixing this issue is not necessary to contribute to lnd repo ?

[mohamedawnallah]

I've just solved the issue using the sudo make check command it seems a permission issue. Thanks so much @guggero for pointing me in the right direction 🙏. Now I'm gonna pick up good first issues

[guggero]

Okay, good to know! I'm also working on a fix for bitcoind 26 which generally doesn't seem to work with unix sockets currently. Should have the PR up today or so.

I'm wondering if fixing this issue is not necessary to contribute to lnd repo?

No, as mentioned above, if you create a PR, the full CI pipeline will run on it, running all those checks for you. Though if you are a first-time contributor, someone needs to manually approve the run of the CI, so you can save a lot of time by running the checks on your local machine first.

[mohamedawnallah]

Thanks a lot for your follow-up!

[guggero]

Can you try #8273 if that allows you to run make check successfully on your machine now (without sudo which shouldn't be needed and hints at some issue with the way your machine or Golang installation is set up)?

[mohamedawnallah]

Hey @guggero sorry it seems I missed this notification. I'm gonna try it now and let you know!