Cyphernode v0.9.0-rc.4

This release is mostly performance improvements and software upgrades.

This release contributors: @phillamy @Kexkey

New

• bitcoin_gettxoutproof endpoint
• bitcoin_generatetoaddress endpoint

Improvements

• More and better tests
• Configured Bitcoin network and port are now available in installation info (displayed in the Welcome app)
• Containers environment variables are now segregated in separate env files instead of directly in docker-compose file
• Wallet events (walletnotify) are now published to a MQTT's topic and handled much better
• On tx event, tx information is now sent to Cyphernode by the Bitcoin Node instead of being pulled by Cyphernode
• Block events (blocknotify) are now published to a MQTT's topic and handled much better
• On block event, block information is now sent to Cyphernode by the Bitcoin Node instead of being pulled by Cyphernode
• In regtest, blocks are mined on startup when there's no funds in spending wallet
• Now serving HTTP (instead of self-signed HTTPS) when traefik is accessed via Tor
• Proxy container now Debian-based
• 60-sec timeout on locks when processing new tx confirmations
• Better error handling
• Batching: we flag as DEFECT the batch transactions when bitcoind times out
• Defaulting the spending transactions to "ECONOMICAL"
• Core Lightning now uses PostgreSQL instead of SQLite3
• Turned on Offers (bolt12) and Anchor Channels in CLN
• Upgraded Traefik from 1.7 to 2.6
• Upgraded Tor from 0.4.5.6 to 0.4.7.8
• Upgraded Bitcoin Core from v22.0 to v24.0.1
• Upgraded Core Lightning from v0.10.2 to v23.11.2
• Upgraded Spark Wallet from v0.3.0 to v0.3.1
• Upgraded Batcher from v0.2.0 to v0.2.1
• Upgraded Specter from v1.7.1 to v1.14.3

Fixes

• Permission problems during setup on Linux
• torgen build fixed
• Fixes for CLN breaking changes (amount_msat...)
• Removed deprecated legacypay from ln_pay
• Minor fixes here and there

Cyphernode v0.8.1

This is a minor release of a few fixes and improvements:

• Fixed: full compatibility with docker-compose was lost in v0.8.0 (thanks @BHodl)

• Improved: ln_pay now pays using the pay plugin first (with MPP), then legacypay on failure

• Improved: moved env variables from docker-compose.yaml to env files, for proxycron only for now (thanks @phillamy)

• Small improvements in the startup scripts

• Removed inserting previous txs in database when computing a tx fees

Cyphernode v0.8.0

Say hello to PostgreSQL! We moved from SQLite3 to PostgreSQL to take advantage of its enterprise-class features. Here are some of our motivations:

• Better overall performance
• Easier to implement replicas / distributed redundancy
• Running in an independent container: can be used by other containers as well
• More/better administration tools
• Easier to configure C-lightning to use PostgreSQL
• Future development

All of that may also be possible with SQLite3, but with a lot more work.

If you have an existing Cyphernode installation with existing data, Cyphernode will take care of the migration: we built all the required ETL scripts that will hopefully flawlessly move your current instance to the new DBMS.

There are also several improvements and new features in this release. Thanks go to @pablof7z @phillamy and @schulterklopfer for their valuable contributions, feedbacks and inputs!

New features

• PostgreSQL: migrating from SQLite3 to PostgreSQL
  • Automatic migration from current SQLite3 to new PostgreSQL (ETL)
  • New Indexes
  • Separate container
• Support for labels when:
  • watching addresses
  • getting new addresses
• New ln_paystatus endpoint
• New validateaddress endpoint
• New deriveindex_bitcoind endpoint (20x faster than Pycoin), also supports ypub/upub and zpub/vpub notations!
• New derivepubpath_bitcoind (20x faster than Pycoin), also supports ypub/upub and zpub/vpub notations!

Fixes and improvements

• Refactoring of manage_missed_conf and confirmation management
• ln_pay now first pays using legacy_pay (MPP disabled) and on failure (for routing reasons), retry with the pay plugin (MPP enabled by default)
• Small fixes in ln_pay
• Small fixes in ln_delinvoice
• Small fixes in ln_connectfund
• Small fixes in LN webhooks
• ln_listpays can now take a bolt11 string argument
• Sometimes, Gatekeeper was not compliant to JWT: now it is but still compatible with previous buggy version
• Fixed CN client examples
• Gatekeeper now returns 401 Unauthorized on authentication error and 403 Forbidden on authorization error
• Gatekeeper now waits for the Proxy to be ready before listening to requests
• More graceful shutdown on certain containers
• Docker now uses the helloworld endpoint to check Proxy's health
• Better way to determine slow machine during setup
• Better tests when starting up
• Fixed a bug when running Cyphernode as current user instead of dedicated user
• When trying to add a batcher that already exists (same label), it will now modify existing one
• Got rid of the full rawtx from the database! Let's use Bitcoin Core if needed
• helloworld endpoint now returns a JSON compliant response
• Added and improved tests:
  • api_auth_docker/tests/test-gatekeeper.sh
  • proxy_docker/app/tests/test-manage-missed.sh
  • proxy_docker/app/tests/test-batching.sh
  • proxy_docker/app/tests/test-derive.sh
  • proxy_docker/app/tests/test-watchpub32.sh
  • proxy_docker/app/tests/test-watches.sh
• Fixed typos and improved clarity in messages
• Bump ws from 5.2.2 to 5.2.3 in /cyphernodeconf_docker
• Bump path-parse from 1.0.6 to 1.0.7 in /cyphernodeconf_docker
• Bump tmpl from 1.0.4 to 1.0.5 in /cyphernodeconf_docker
• Bump validator from 10.11.0 to 13.7.0 in /cyphernodeconf_docker
• Code cleaning

Upgrades

• C-lightning from v0.10.0 to v0.10.2
• Bitcoin Core from v0.21.1 to v22.0

Cypherapps

• Batcher from v0.1.2 to v0.2.0
• Spark Wallet from v0.2.17 to v0.3.0
• Specter from v1.3.1 to v1.7.1

Cyphernode v0.7.0

This is more a technical release than a feature one. You as a user will not see much differences with previous release, except probably for Specter. Nonetheless an important one!

Fixes and improvements

• Docker Swarm deployment more robust, will not deploy on a distributed Swarm to allow volume sharing (@schulterklopfer);
• Tor tests on startup now use check.torproject.org Onion address;
• Since Bitcoin Core v0.21.0, simply having a wallet in bitcoin.conf does not create it -- we now need to create them when bitcoind starts up if they don't already exist (thanks for your first contributions @phillamy!);
• Small fixes in removefrombatch and watchtxid;
• Now using new key server for Tor's GPG signature verification;
• Some cleaning of useless stuff.

Upgrades

• Cyphernode's glibc base image: glibc from 2.29 to 2.31;
• Bitcoin Core from 0.20.1 to 0.21.1;
• C-Lightning from 0.9.1 to 0.10.0;
• Tor from 0.4.4.5 to 0.4.5.6;
• Python 3.6 to 3.8;
• Normalized Alpine to version 3.12.

CypherApps

• Specter Desktop from v0.9.2 to v1.3.1;
• Batcher from v0.1.1 to v0.1.2.

Enjoy!

Cyphernode v0.6.0

A small release with a big new feature! Cyphernode now supports integration of Specter Desktop, an amazing web GUI for Bitcoin Core with hardware wallet support! https://github.com/cryptoadvance/specter-desktop

Installation notes: if you already were using the batcher cypherapp, please move/rename the dist/apps/ directory, so that the new cypherapps branch is cloned during setup. Then copy and modify your batcher docker-compose.yaml file and data/ dir in dist/apps/batcher/.

What's new

• Specter Desktop as an optional feature -- easily accessible from the Cyphernode Welcome App.

What's improved

• The batcher cypherapp is now optional;
• Added blockfilterindex to Bitcoin Core configuration;
• Tor container is much smaller;
• Upgraded Spark Wallet to v0.2.17;
• Upgraded Welcome App to v0.2.4;
• Upgraded Batcher to v0.1.1.

What's fixed

• Unwatching an address without supplying the webhook URLs didn't unwatch the address! Now it does;
• Spark Wallet was deployed even when LN was disabled. Fixed!

Cyphernode v0.5.0

Cyphernode and batching

The main focus of this release is on improving the batching functionalities. It is now much more convenient to batch multiple payments using Cyphernode, especially combined with the new Batcher cypherapp (to be released soon).

Special thanks to @xsats for his suggestions, discussions and for testing the batching endpoints and the Batcher cypherapp. Big thanks to @Tomtibo for his contribution to improving the documentation and report and suggestions to the xpub watching features.

What's New

• unwatchtxid: it was possible to watch a txid, now it's also possible to unwatch a previously watched txid;
• bitcoin_estimatesmartfee: get the Bitcoin Core estimatesmartfee response;
• createbatcher: create a batching template;
• updatebatcher: change a batching template;
• addtobatch: queue a payment to an ongoing batch;
• removefrombatch: dequeue a previously added payment from an ongoing batch;
• listbatchers: list of batching templates with info;
• getbatcher: get current state/summary of the requested batching template;
• getbatchdetails: get the state as well as all the outputs of a batch, past or ongoing.

What's Improved

• Log path configuration: a path has been added to the config where the log files are stored, instead of relying entirely on Docker's logs;
• Multi watches on the same address: now possible to have multiple watches with different webhooks on the same address;
• Multi watches on the same txid: now possible to have multiple watches with different webhooks on the same txid;
• Unwatch by id: now possible to unwatch by id instead of address; useful when having multiple watches on the same address;
• More info in watchtxid webhook body: the returned information now has full tx details;
• Better doc.

What's Fixed

• replaceable now boolean, was 0 or 1 before;
• Fixed txid vs hash in multiple webhooks/getters;
• Several small fixes and improvements.

Upgrades

• Bitcoin Core v0.20.1
• C-Lightning v0.9.1
• Spark Wallet v0.2.16
• Tor v0.4.4.5

Cyphernode v0.4.0

This release adds useful LN features and fixes small glitches.

Thanks to @gabidi for your great contribution!

What's new

• ln_listpeers: Calls listpeers from lightningd. Returns the list of peers and the channels opened with them, even for currently offline peers.
• ln_getroute: Ln nodeId to find route to.
• ln_listpays: Calls listpays from lightningd. Returns list of payments made.
• ln_listfunds: Calls listfunds from lightningd. Returns list of funds in outputs and channels.
• ln_withdraw: Withdraw outputs from c-lightning wallet to a destination of choice with confirmation targeting fees. Can either supply satoshi in sats (integer), BTC (8 point decimal) or set all to true to withdraw all outputs.
• Bitcoin Core upgraded to v0.19.1
• C-Lightning upgraded to v0.8.2
• Tor upgraded to version 0.4.3.5

What's fixed

• watchtxid crash
• OpenAPI docs fixed for the get_txns_spending endpoint (thanks @nk1tz)
• Fix in ln_pay when expected_msatoshi was null
• Fixed race condition when the same watch was called more than once and was confirmed at the same time... the webhooks were then not called.

Cyphernode v0.3.2

Mostly bug fixes and improvements.

What's new:

• Added confTarget arg to the spend endpoint (thanks @FrancisPouliot and @laurent80 !) (32300fe)
• Added replaceable and subtractfeefromamount args to the spend endpoint (874c42d)

What's improved:

• Watched Bitcoin address is validated before watching it (96a830f)
• Added details on spend response: address, amount, firstseen, size, vsize, replaceable and fee (9d4c703) (e9321b6)

What's fixed:

• Callbacks were Torified when Tor was activated, even when we didn't want it (thanks @Tomtibo !) (ed06a00)
• Proxychains output was appearing in OTS operation responses (0a5526e)
• Fixed requesthandler to support multi-line bodies on requests (0a5526e)
• When Torified, Bitcoin node now also uses clearnet when allowed (7b2004f)
• Bump acorn from 5.7.3 to 5.7.4 in /cyphernodeconf_docker (#177)

Cyphernode v0.3.1

Minor release, small enhancements and bug fixes.

• Added a QR Code of Onion v3 address at the end of startup when Tor is activated (#166)
• Give more time to bitcoin and lightning to shut down before kill (#172)
• In newblock published message, blockheight should be an integer (#169)
• Fixed temp rawtx file name conflicts (#170)
• Bugfix/txidasstring (#171)
• When RBF, multiple txid linked to same address, must SELECT DISTINCT cuz listunspent errors (#167)

Cyphernode v0.3.0

Big thanks to the contributors of this release of Cyphernode:

• @mayankchhabra
• @schulterklopfer
• @gabidi
• @cryptoskillz
• @FrancisPouliot
• @Kexkey

What's new

Tor support

• Tor socks5 proxy
• Configurable Tor gateway for:
  • OTS stamp, upgrade and verify requests (Tor or clearnet)
  • OTS callbacks/webhooks (Tor or clearnet)
  • Address watch callbacks/webhooks (Tor or clearnet)
  • TXID watch callbacks/webhooks (Tor or clearnet)
  • Bitcoin Node networking (Tor and/or clearnet)
  • Lightning Node networking (Tor and/or clearnet)
• Configurable Tor Hidden Services for:
  • Traefik web proxy
  • Bitcoin Node
  • Lightning Node
  • For more privacy, each has its own v3 onion address
• Spark Wallet gets its rates through Tor when installed, clearnet otherwise

New endpoints

• get_txns_by_watchlabel to get a list of transactions for a specific xpub (used with the xpub watching feature)
• get_unused_addresses_by_watchlabel to get a list of unused addresses for a specific xpub, in the watching window (used with the xpub watching feature)
• getblockhash to get the block hash by block height
• get_txns_spending to get a list of transactions in your spending wallet
• getbalances to get extended balance information of your spending wallet

Added functionalities

• A base64-encoded eventMessage field can be passed to the watch endpoint, that will be published to the tx_confirmation topic on 0-conf and 1-conf, along with the TXID, address, vout index, amount and number of confirmations
• A base64-encoded eventMessage field can be passed to the spend endpoint, that will be published to the spend topic when spending, along with the TXID, address and amount
• A message is now published to the newblock topic when a new block is mined, containing the block hash and the block height
• Support for any amount on ln_create_invoice (for donation, for example)
• ln_pay will use the supplied expected_msatoshi argument when there's no amount in the bolt11 invoice
• Callbacks/webhooks now optional on address watchers (eventMessage useful here)
• Callbacks/webhooks now optional on LN invoice payments

What's improved

• Upgraded Bitcoin Core to version 0.19.0.1
• Upgraded C-Lightning to version 0.8.0
• Upgraded Spark Wallet to version 0.2.12
• Upgraded multiple containers to Alpine 3.11.0
• Better deployment mechanism (dependency waiting, readiness states)
• Better handling of transaction confirmations
• Broker (publisher/subscriber system) now reachable by the CypherApps -- your web apps can now benefit from the events published by Cyphernode (notifier, tx_confirmation, spend, newblock)
• Cypherapps can now decide to accept or not HTTP; the redirection to HTTPS by Traefik is not forced anymore