"bwallet-cli admin wallets" Error: socket hang up

[sulliwane]

I'm running a bcoin node locally (testnet, --prune, synced).

Here is my bcoin.conf:

# main, testnet
network: testnet

# Data directory
prefix: /data

# Enable flags
use-workers: true

log-file: true
log-level: info
max-files: 8192

http-host: ::

api-key: password2

Here is my wallet.conf:

network: testnet
prefix: /data
http-host: ::

# Security settings
api-key: password
wallet-auth: true

This command is NOT working:

BCOIN_URI=http://localhost:18334 BCOIN_API_KEY=password BCOIN_NETWORK=testnet bwallet-cli admin wallets

But this command is working:

BCOIN_URI=http://localhost:18332 BCOIN_API_KEY=password2 BCOIN_NETWORK=testnet bcoin-cli info

So it seems like the wallet service is not up...I'm starting bcoin with this command BCOIN_CONFIG=/data/bcoin.conf /bcoin/bin/bcoin --spv...is there a seperate command to start the wallet service?

Many thanks!

[sulliwane]

related: #634

I tried using WALLET_CONFIG=/data/wallet.conf then removed it and tried putting wallet.conf in /data/testnet/wallet.conf, and now I get this error:

Error: Status code: 403.

here is the log on the node side:

[debug] (http) Request for method=GET path=/wallet (::ffff:172.21.0.1).

[sulliwane]

Seems like this guy is having similar issue : #493 (comment)

[sulliwane]

I commented out this: # wallet-auth: true in wallet.conf (by referring to this doc page: https://github.com/bcoin-org/bcoin/blob/master/docs/Configuration.md) and it's now working!

[pinheadmz]

Glad you solved your issue, @sulliwane ! A bit more info for you: wallet-auth means a token is required (in addition to the API key) to execute wallet commands.

Another thing I noticed about your setup is you are missing the endpoint /wallet for the wallet-server curl commands. See this section of the docs about the url to use: http://bcoin.io/api-docs/?shell--curl#the-walletdb-and-object

[sulliwane]

Hey @pinheadmz , I'm not quite sure what you mean by "missing the endpoint /wallet" ?

I just did that:

const newWallet = await walletClient.createWallet('1', {
      witness: false,
      watchOnly: true,
      accountKey: 'tpubDDYmpeYPBzPC2...',
    });
console.log(await wallet.getInfo())

But my balance shows { tx: 0, coin: 0, unconfirmed: 0, confirmed: 0 } (even after starting a rescan), whereas on the copay app I used to create a testnet wallet, I have 0.25 testnetBTC.

Are these two things related?

Do I need the extra step of running wallet.importPublic('default', xpubkey) ?

Thank you for the clarification!

[sulliwane]

I just received a new tx from a faucet (0.01BTC) and these appear correctly: { tx: 1, coin: 1, unconfirmed: 1000000, confirmed: 0 }. My node is running with --spv, but i did start a rescan...any idea why it's discovered the previous tx to this address ?

[pinheadmz]

rescan in SPV mode does nothing. Was this new transaction broadcast after you have already created your watch-only wallet? That is the intended behavior :-)

To recover the older transactions, you may need to run reset which rewinds the chain. Its quite fast in SPV mode: bcoin-cli reset <height>

[sulliwane]

Yes! the watch-only wallet was already created. So it's working as expected. So If my current understanding is correct, I have two options:

1. run bcoin node in --spv mode, and start a bcoin-cli reset <height> each time a new watch-only xpubkey wallet is created.
2. run bcoin node in full mode...it has all tx history of the universe, including the few one related to my wallets.

I'm not sure about --prune mode, offering any advantages over these two solutions...

My app is very simple, watch-only service, xpubkey only new wallets, in the most lightweight way possible. The best would have been not to maintain the bcoin node myself, but seems I have no choice ;)

[pinheadmz]

run bcoin node in --spv mode, and start a bcoin-cli reset each time a new watch-only xpubkey wallet is created.

You only need to reset SPV (or rescan Full) if you're importing an xpub WHICH HAS ALREADY BEEN USED ON THE BLOCKCHAIN. If it is a brand new key that has never received a transaction (and you are sure 😉) you do not to reset or rescan, just import the xpub and wait for new transactions!

run bcoin node in full mode...it has all tx history of the universe, including the few one related to my wallets.

Yes, you're correct. If you import an xpub that has already been used on chain, you'll need to run a rescan <height> and if you know WHEN the xpub was generated and used for the first time, you can start you rescan from a much shallower <height> and only scan blocks you need to. Actually the <height> parameter applies to reset command as well.

I'm not sure about --prune mode, offering any advantages over these two solutions...

Yeah I kinda agree. Pruned nodes are more secure and private because they verify everything and rely on nothing. They maintain a complete UTXO set and so they offer some more features than SPV, but if you ever think you will have the need to look up transactions in historical blocks, a pruned node will be useless there.

[sulliwane]

thanks a tone for your detailed answer.

Right, I did not clarify this => 90% of new wallets will have previous/existing tx onchain, made before I start tracking them. But, I'm only interested in knowing the latest state of balance (but I guess I can't avoid scanning all the history to get this information).

You mean

• bcoin-cli reset <height> with bcoin running in spv, and
• bwallet-cli rescan <height> with bcoin running in full mode, correct?

I understand rescan (asking for wallet service to rescan history), but reset is commanding the bitcoin node so what the logic behind the scene? Is it that it's forcing the node to replay all transactions since <height> and as walletdb is watching it, wallet service will get the correct final balance?

[pinheadmz]

Yeah you are correct! Because SPV has no block history on disk, the only way to recover old tx history is with reset <height>, which effectively sets the node's current blockchain tip to <height>, then it resumes syncing to the network like normal -- asking peers for blockheaders, etc -- starting from that point in history.

This would be TERRIBLE practice for a full node, because it would require immense bandwidth and processing every single time. However, in SPV it is a somewhat trivial operation because it only downloads and processes 80-byte block headers 😄 and then whatever transactions are requested by the bloom filter.

If you are deploying a production application with multiple users all running wallets on the same node, I would advise against using SPV. For a busy application you'd probably want to even bump UP the disk usage by running a full node with --index=tx and --index--address so you wouldn't even have to rescan the chain, you could use the database indexes created by bcoin.

[sulliwane]

hum, I see. very interesting! I may indeed go for a full node with the options you mention, so I don't need to rescan. Any idea of what would be roughly the disk space requirements when using --index=tx and --index--address for the main bitcoin chain?

Under the hood, is bcoin spawning a bitcoind node? Or is bcoin a full JS bitcoin node in its own?

[pinheadmz]

bcoin is a completely separate implementation of the Bitcoin protocol written in JavaScript :-)

This post goes in to a lot of detail about our disk usage and ways to improve it:
#585

Unfortunately at this time a full node plus indexing probably takes up about ~500 GB

For comparison, my Bitcoin Core full node takes up about 232 GB, some ~20 GB of which appears to be index files.

[sulliwane]

Ok, I see. 500 GB doesn't sound like very lightweight :) 232GB neither anyway...

Out of curiosity, is it possible to connect the wallet service to a bitcoin core full node instead of the bcoin JS node? Or is bcoin JS node providing some API layers that are not provided by vanilla bitcoind node? (sorry for the neophyte questions here, I'm much more familiar with the way ethereum nodes work).

I was expecting to find some centralized services (like Infura for eth) publicly exposing bitcoin core node API (using one standardized API layer everyone would agree on). Running a full node is often a deal breaker for developers.

[pinheadmz]

Yeah there are tons of API services, most block explorers offer (paid) high volume endpoints. You might also look for Insight servers or even try the Electrum protocol -- which is basically SPV but wrapped in some extra security features.

Just keep in mind there is no security or privacy unless you are verifying the blockchain yourself!