Skip to content

Commit

Permalink
[Bitcoin/Rust]: Finalize Bitcoin V2 refactoring (#3848)
Browse files Browse the repository at this point in the history
* [Bitcoin/V2]: Add a `rust/frameworks` directory, move `tw_utxo` there

* [Bitcoin/V2]: Add Bitcoin Legacy address implementation

* [Bitcoin/V2]: Add `StandardBitcoinAddres`

* [Bitcoin/V2]: Add segwit address

* Switch to TW Bitcoin Address implementation in `BitcoinEntry`

* [Bitcoin/V2]: Implement signAsDER and verifyAsDER used in Bitcoin

* [BitcoinV2]: Add standard Unsigned and Signed Transaction declarations

* Add and implement Bitcoin encoding for Signed Transaction

* [BitcoinV2]: Refactor `TWDerivation` to avoid Coin names

* [BitcoinV2]: Refactor preimage hashing

* Delegate transaction preimage hashing to `LegacyPreimage`, `Witness0Preimage` helpers

* [BitcoinV2]: Design a `TransactionSigner` API

* [BitcoinV2]: Design an API for `TransactionPlanner`, `TransactionBuilder`

* add Script check methods

* cargo fmt

* make signing_method optional, try to automatically determine signing method if unavailable

* start expanding verify_signatures

* add schnorr module to tw_keypair, wip

* expand KeyPair impl for schnorr

* expand enums in tw module with Schnorr

* simplify schnorr module structure

* complete SigningKeyTrait and VerifyingKeyTrait impl for KeyPair

* complete verify_signatures mechanism

* adjust some types, complete compile method

* add test interface for signer, wip

* handle Schnorr variant in tw_keypair

* ignore the recovery byte in ECDSA signature

* update comment

* specify the keys explicitly in tests

* continue debugging

* remove UseDefault from SighashBase, use All as default, fix sighash calculation

* formatting

* add Witness type, correctly consensus-encode items

* fix P2WPKH scriptPubkey generation

* add standard OPCodes and add convenience functions for Script

* complete standard Script builders and checkers

* update script submodule structures

* build claim scripts for standardized transaction variants

* fix Script method call

* update comments, use H264 for new_p2pk instead of PublicKey type

* small cleanup to push_slice

* fix typo

* rename Sighash to SighashType

* expand TransactionBuilder

* adjust builder and add test, wip

* update TransactionBuilder, init test, wip

* complete build_legacy_tx test

* assert in build_legacy_tx

* add BitcoinEcdsaSignature type, serialize correctly for claim script

* add SER_SIZE const to BitcoinEcdsaSignature

* rename TransactionSigner to SighashComputer

* add a ClaimBuilder

* rename some types

* add tests for building Segwit transactions

* delete playgrounds

* track transaction builder test

* add fee calculation traits, implement for standard Transaction, Inputs and Outputs

* implement TransactionFee::fee

* implement CompactInteger::serialized_len

* consider Segwith flag and marker when calculating weight/size/fee

* add select_inputs logic to TransactionFee, handle vbyte/size calculation edge case

* expand tests with size, vsize and weight calculation

* add fee() calculation tests

* add input selection tests, wip

* add base_size to TransactionFee trait, implement method

* move input selection logic out of TransactionFee trait and to ComputedTransaction

* add change amount related methods

* continue with change output build pattern, wip

* mvoe utxo input selector to separate module

* complete test for utxo selection with change output generation

* add note slight fee difference

* extra comments

* introduce CompiledSelectionBuilder

* add p2tr condition builder functions

* add builders for input, claim and output P2TR-key-path scripts

* track taproot1_sighash, wip

* extra conditions

* add tests for Taproot inputs, spending conditions and claims. wip

* continue on Taproot sighash calculation, wip

* continuing with Taproot Sighash debugging...

* add mockup of TapsighHash

* extend test, wip

* remove unused fields in UtxoPreiamgeArgs

* implement Hasher::TapSighash variant

* rename TapSighash prefix const

* seperate sighash methods on PreimageInterface into legacy/segwit and taproot

* move spent_* hashing methods to TransactionHasher

* adjust TapSighash workflow to other variants

* move tx_hasher and sighash_ty to a per-input basis

* add const for Annex indicator

* include leaf_hash and separator as part of TapSighash

* construct Taproot script-path spending info and merkle root, wip

* add p2tr_script_path spending script builder

* init asset/ module, move brc20 and ordinals there

* adjust brc20 and ordinal code for new changes, add brc20_transfer input builder

* separate input builder into individual modules

* fix and format

* remove unused DEFAULT_TX_HASHER in utxo_selector

* build brc20 spending input

* build brc20 transfer output

* add tweak support to tw_keypair

* build brc20 transfer output

* add support for non aux rand data when signing for schnorr, complete P2TR key-path test

* add test for BRC20 transfer output, fix issue with payload

* testing brc20 transfer reveal, wip

* finalize brc20 reveal construction with tests

* get rid of dbg!

* add custom script builders

* wipe tw_bitcoin's preimage_hashes_impl, start with utxo builder migration

* handle custom script proto builder

* start migrating output proto to builder matcher, wip

* use SelectionBuilder

* expand migration of tw_bitcoin preimage_hashes_impl

* create dummy claims for preiamge_hashes_impl

* add and handle a UseAll selector

* complete construction of proto outputs

* add init module with proto_input_to_native function

* implement proto_output_to_native

* process change output

* handle custom script_pubkey outputs

* fix signature issue for p2tr_script_path, expand p2wsh impl

* set send_to_address tests to ignore for now

* remove ignore attribute for ordinals and brc20, all related tests pass

* derive P2PKH and P2SH from base58 address

* derive P2WSH and P2WPKH from bech32 address

* move address to native derivation logic to individual function

* init taproot module with TaprootAddress

* implement p2tr_with_coin_and_prefix for TaprootAddress, derive from StandardBitcoinAddress

* fix tests for reference changes

* distinguish between Segwit and Taproot for StandardBitcoinAddress

* derive Taproot program from address, do extra witness program check

* complete builders for P2WPKH and P2WSH derived from address string

* complete legacy P2PKH and P2SH builders derived from address string

* move address derivation logic to address_to_native

* complete proto builders for p2pkh and p2wpkh from_hash variants

* expand output builder for P2SH, both redeem script and hash

* expand P2SH input claiming, wip

* set ignore attribute to p2sh tests for now

* add option to provide sigs to proto_input_to_native, deprecate some legacy code

* construct proto return value for compile method

* implement txid and fix ownership issue with proto return structure

* update tw_bitcoin calls in wallet_core_rs for legacy FFI functions

* update p2tr_script_path test module, use updated tw_utxo crate

* adjust legacy_script tests to tw_utxo changes

* remove unused imports

* remove print/dbg statements

* remove proto interface for tw_utxo

* add proto_input_to_spending_data function

* rename to proto_sighash_to_sig

* sign sighashes and compile final transaction in tw_bitcon entry

* formatting

* add todo to fn txid

* [Bitcoin]: Cleanups

* [Bitcoin]: Fix most of clippy warnings

* [Bitcoin]: Refactor schnorr private and public keys

* [Bitcoin]: Fix BRC20 tests

* [Bitcoin]: Fix txid

* [Bitcoin]: Temporary add `BitcoinV3.proto` - refactored API

* [Bitcoin]: Deprecate tw_bitcoin/src/modules

* [Bitcoin]: Add `TransactionProtobuf` module

* [Bitcoin]: Use `TWError<SigningError>` in tw_utxo

* [Bitcoin]: Add `UtxoProtobuf` builder

* Merge `SigningMethod::TaprootAll` and `SigningMethod::TaprootOnePrevout`
* Temporary remove `OrdinalInscription`

* [Bitcoin]: Add a `SpendingDataConstructor` pattern

* Refactor `UtxoBuilder` to create a corresponding `SpendingDataConstructor`
* Slightly refactor `BitcoinV3.proto` API

* [Bitcoin]: Finalize UTXO selectors

* Add `MaxSelector`, add `SigningInput::max_amount_output`
* Refactor `SelectorBuilder` as `ExactSelector`
* Add `DustFilter` and `DustPolicy`
* Slightly refactor `BitcoinV3.proto` API

* [Bitcoin]: Finalize UTXO selectors, split `SighashComputer`

* Add `MaxSelector`, add `SigningInput::max_amount_output`
* Refactor `SelectorBuilder` as `ExactSelector`
* Add `DustFilter` and `DustPolicy`
* Slightly refactor `BitcoinV3.proto` API
* Split `SighashComputer` into several modules
* Finalize `TxCompiler`
* Finalize `SighashVerifier`

* [Bitcoin]: Add `TxSigner`

* [Bitcoin]: Add `OutputProtobuf` builder

* Fix Segwit address to not allow witness version > 0

* [Bitcoin]: Refactor `TaprootAddress` and `SegwitAddress`

* [Bitcoin]: Add transaction signer

* [Bitcoin]: Add transaction planner

* [Bitcoin]: Add transaction compiler

* [Bitcoin]: Make `TxPlanner` generic

* [Bitcoin]: Some clean ups

* [Bitcoin]: Remove legacy BRC20 API

* [Bitcoin]: Add BRC20 test

* Add default chain info if not provided
* Add default sequence if not provided

* [Bitcoin]: Add P2PKH test

* [Bitcoin]: Fix final fee calculation

* [Bitcoin]: Add p2tr_key_path

* [Bitcoin]: Move unit tests to `tw_any_coin/tests/chains/bitcoin`

* [Bitcoin]: Fix `send_to_address` tests

* [Bitcoin]: Fix broken tests

* [Bitcoin]: Move UTXO selection to `tw_any_coin/tests`

* [Bitcoin]: Add exact selector with and without change test

* [Bitcoin]: Check transaction weight

* Add more exact_selector tests

* [Bitcoin]: Add more selector tests, increase code coverage

* [Bitcoin]: Add p2pkh compile test

* [Bitcoin]: Add mainnet BCH test

* Fix fork_id sighashing

* [Bitcoin]: Add send to p2sh and p2wsh address tests

* Add p2wpkh signing test
* Set V1 default transaction version

* [Bitcoin]: Add OP_RETURN test

* [Bitcoin]: Allow to pass a DER encoded signature to `TransactionCompiler`

* [Bitcoin]: Test Bitcoin addresses

* [Bitcoin]: Add BRC20 compile tests

* [Bitcoin]: Fix transaction weight

* [Bitcoin]: Remove legacy FFI TWBitcoinFeeCalculateFee

* Return an error if `is_it_brc_operation` is true

* [Bitcoin]: Add P2SH output signing test

* [Bitcoin]: Add p2tr_script_path output test

* Reorg p2tr_key_path tests to cover multiple protobuf types

* [Bitcoin]: Fix p2tr custom script

* [Bitcoin]: Fix P2PK

* Extend P2PKH test to cover all protobuf inputs
* Add P2WSH output test
* Add sighash single test

* [Bitcoin]: Fix some TODOs

* [Bitcoin]: Finalize BitcoinV2

* Remove `Bitcoin.SigningInput.is_it_brc_operation` and `Bitcoin.SigningInput.planning_v2`
* TODO adopt C++ BRC20 tests for BitcoinV2

* [Bitcoin]: Fix one of Android tests

* Add Rust `test_bitcoin_sign_brc20_transfer` test

* [Bitcoin]: Uncomment out Android tests

* [Bitcoin]: Fix iOS tests

* Remove NFT data from Android and iOS test suits

* [Bitcoin]: Fix WASM BRC20 tests

* [CI] Trigger CI

* [Bitcoin]: Minor fixes

* [Bitcoin]: Fix Rust WASM tests

* [Bitcoin]: Add real P2TR transaction tests

* [TON]: Add P2TR test in WASM

---------

Co-authored-by: lamafab <42901763+lamafab@users.noreply.github.com>
  • Loading branch information
satoshiotomakan and lamafab authored Jul 15, 2024
1 parent 9373610 commit 221584f
Show file tree
Hide file tree
Showing 237 changed files with 12,669 additions and 9,400 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -52,17 +52,17 @@ class TestCoinType {
fun testDerivationPath() {
var res = CoinType.createFromValue(CoinType.BITCOIN.value()).derivationPath().toString()
assertEquals(res, "m/84'/0'/0'/0/0")
res = CoinType.createFromValue(CoinType.BITCOIN.value()).derivationPathWithDerivation(Derivation.BITCOINLEGACY).toString()
res = CoinType.createFromValue(CoinType.BITCOIN.value()).derivationPathWithDerivation(Derivation.LEGACY).toString()
assertEquals(res, "m/44'/0'/0'/0/0")
res = CoinType.createFromValue(CoinType.SOLANA.value()).derivationPathWithDerivation(Derivation.SOLANASOLANA).toString()
res = CoinType.createFromValue(CoinType.SOLANA.value()).derivationPathWithDerivation(Derivation.SOLANA).toString()
assertEquals(res, "m/44'/501'/0'/0'")
}

@Test
fun testDeriveAddressFromPublicKeyAndDerivation() {
val publicKey = PublicKey("0279BE667EF9DCBBAC55A06295CE870B07029BFCDB2DCE28D959F2815B16F81798".toHexByteArray(), PublicKeyType.SECP256K1)

val address = CoinType.BITCOIN.deriveAddressFromPublicKeyAndDerivation(publicKey, Derivation.BITCOINSEGWIT)
val address = CoinType.BITCOIN.deriveAddressFromPublicKeyAndDerivation(publicKey, Derivation.SEGWIT)
assertEquals(address, "bc1qw508d6qejxtdg4y5r3zarvary0c5xw7kv8f3t4")
}
}

This file was deleted.

Large diffs are not rendered by default.

Original file line number Diff line number Diff line change
Expand Up @@ -53,10 +53,10 @@ class TestAnyAddress {
fun testCreateWithPublicKeyDerivation() {
val coin = CoinType.BITCOIN
val pubkey = PublicKey(any_address_test_pubkey.toHexByteArray(), PublicKeyType.SECP256K1)
val address1 = AnyAddress(pubkey, coin, Derivation.BITCOINSEGWIT)
val address1 = AnyAddress(pubkey, coin, Derivation.SEGWIT)
assertEquals(address1.description(), any_address_test_address)

val address2 = AnyAddress(pubkey, coin, Derivation.BITCOINLEGACY)
val address2 = AnyAddress(pubkey, coin, Derivation.LEGACY)
assertEquals(address2.description(), "1JvRfEQFv5q5qy9uTSAezH7kVQf4hqnHXx")
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -113,13 +113,13 @@ class TestHDWallet {
val coin = CoinType.BITCOIN
val wallet = HDWallet(words, password)

val key1 = wallet.getKeyDerivation(coin, Derivation.BITCOINSEGWIT)
val key1 = wallet.getKeyDerivation(coin, Derivation.SEGWIT)
assertEquals(key1.data().toHex(), "0x1901b5994f075af71397f65bd68a9fff8d3025d65f5a2c731cf90f5e259d6aac")

val key2 = wallet.getKeyDerivation(coin, Derivation.BITCOINLEGACY)
val key2 = wallet.getKeyDerivation(coin, Derivation.LEGACY)
assertEquals(key2.data().toHex(), "0x28071bf4e2b0340db41b807ed8a5514139e5d6427ff9d58dbd22b7ed187103a4")

val key3 = wallet.getKeyDerivation(coin, Derivation.BITCOINTESTNET)
val key3 = wallet.getKeyDerivation(coin, Derivation.TESTNET)
assertEquals(key3.data().toHex(), "0xca5845e1b43e3adf577b7f110b60596479425695005a594c88f9901c3afe864f")
}

Expand All @@ -137,13 +137,13 @@ class TestHDWallet {
val coin = CoinType.BITCOIN
val wallet = HDWallet(words, password)

val address1 = wallet.getAddressDerivation(coin, Derivation.BITCOINSEGWIT)
val address1 = wallet.getAddressDerivation(coin, Derivation.SEGWIT)
assertEquals(address1, "bc1qumwjg8danv2vm29lp5swdux4r60ezptzz7ce85")

val address2 = wallet.getAddressDerivation(coin, Derivation.BITCOINLEGACY)
val address2 = wallet.getAddressDerivation(coin, Derivation.LEGACY)
assertEquals(address2, "1PeUvjuxyf31aJKX6kCXuaqxhmG78ZUdL1")

val address3 = wallet.getAddressDerivation(coin, Derivation.BITCOINTESTNET)
val address3 = wallet.getAddressDerivation(coin, Derivation.TESTNET)
assertEquals(address3, "tb1qwgpxgwn33z3ke9s7q65l976pseh4edrzfmyvl0")
}

Expand Down
5 changes: 3 additions & 2 deletions codegen/bin/coins
Original file line number Diff line number Diff line change
Expand Up @@ -31,9 +31,9 @@ def self.derivation_name(deriv)
deriv['name'].downcase
end

def self.derivation_enum_name(deriv, coin)
def self.derivation_enum_name(deriv)
return "TWDerivationDefault" if deriv['name'].nil?
"TWDerivation" + format_name(coin['name']) + camel_case(deriv['name'])
"TWDerivation" + camel_case(deriv['name'])
end

def self.coin_img(coin)
Expand All @@ -53,6 +53,7 @@ coins = JSON.parse(json_string).sort_by { |x| x['coinId'] }

# used in some cases for numbering enum values
enum_count = 0
derivations = {}

erbs = [
{'template' => 'TWDerivation.h.erb', 'folder' => 'include/TrustWalletCore', 'file' => 'TWDerivation.h'},
Expand Down
2 changes: 1 addition & 1 deletion codegen/lib/templates/CoinInfoData.cpp.erb
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,7 @@ const CoinInfo getCoinInfo(TWCoinType coin) {
TWCurve<%= format_name(coin['curve']) %>,
{
<% coin['derivation'].each do |deriv| -%>{
<%= derivation_enum_name(deriv, coin) %>,
<%= derivation_enum_name(deriv) %>,
"<%= deriv['path'] %>",
"<%= derivation_name(deriv) %>",
TWHDVersion<% if deriv['xpub'].nil? -%>None<% else -%><%= format_name(deriv['xpub']) %><% end -%>,
Expand Down
7 changes: 3 additions & 4 deletions codegen/lib/templates/TWDerivation.h.erb
Original file line number Diff line number Diff line change
Expand Up @@ -18,11 +18,10 @@ enum TWDerivation {
TWDerivationCustom = 1, // custom, for any coin
<% enum_count += 1 -%>
<% coins.each do |coin| -%>
<% if coin['derivation'].count > 1 -%>
<% coin['derivation'].each_with_index do |deriv, index| -%>
<% if index > 0 or !deriv['name'].nil? -%>
<%= derivation_enum_name(deriv, coin) %> = <% enum_count += 1 -%><%= enum_count %>,
<% end -%>
<% if !deriv['name'].nil? and !derivations.has_key?(deriv['name']) -%>
<% derivations[deriv['name']] = true -%>
<%= derivation_enum_name(deriv) %> = <% enum_count += 1 -%><%= enum_count %>,
<% end -%>
<% end -%>
<% end -%>
Expand Down
23 changes: 0 additions & 23 deletions include/TrustWalletCore/TWBitcoinFee.h

This file was deleted.

20 changes: 0 additions & 20 deletions include/TrustWalletCore/TWBitcoinScript.h
Original file line number Diff line number Diff line change
Expand Up @@ -191,26 +191,6 @@ struct TWBitcoinScript* _Nonnull TWBitcoinScriptBuildPayToWitnessPubkeyHash(TWDa
TW_EXPORT_STATIC_METHOD
struct TWBitcoinScript* _Nonnull TWBitcoinScriptBuildPayToWitnessScriptHash(TWData* _Nonnull scriptHash);

/// Builds the Ordinals inscripton for BRC20 transfer.
///
/// \param ticker ticker of the brc20
/// \param amount uint64 transfer amount
/// \param pubkey Non-null pointer to a pubkey
/// \note Must be deleted with \TWBitcoinScriptDelete
/// \return A pointer to the built script
TW_EXPORT_STATIC_METHOD
TWData* _Nonnull TWBitcoinScriptBuildBRC20InscribeTransfer(TWString* _Nonnull ticker, TWString* _Nonnull amount, TWData* _Nonnull pubkey);

/// Builds the Ordinals inscripton for NFT construction.
///
/// \param mimeType the MIME type of the payload
/// \param payload the payload to inscribe
/// \param pubkey Non-null pointer to a pubkey
/// \note Must be deleted with \TWBitcoinScriptDelete
/// \return A pointer to the built script
TW_EXPORT_STATIC_METHOD
TWData* _Nonnull TWBitcoinScriptBuildOrdinalNftInscription(TWString* _Nonnull mimeType, TWData* _Nonnull payload, TWData* _Nonnull pubkey);

/// Builds a appropriate lock script for the given address..
///
/// \param address Non-null pointer to an address
Expand Down
38 changes: 38 additions & 0 deletions registry.json
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,7 @@
"blockchain": "Bitcoin",
"derivation": [
{
"name": "segwit",
"path": "m/84'/2'/0'/0/0",
"xpub": "zpub",
"xprv": "zprv"
Expand Down Expand Up @@ -159,9 +160,16 @@
"blockchain": "Bitcoin",
"derivation": [
{
"name": "segwit",
"path": "m/84'/14'/0'/0/0",
"xpub": "zpub",
"xprv": "zprv"
},
{
"name": "legacy",
"path": "m/44'/14'/0'/0/0",
"xpub": "xpub",
"xprv": "xprv"
}
],
"curve": "secp256k1",
Expand Down Expand Up @@ -225,6 +233,7 @@
"blockchain": "Bitcoin",
"derivation": [
{
"name": "segwit",
"path": "m/84'/20'/0'/0/0",
"xpub": "zpub",
"xprv": "zprv"
Expand Down Expand Up @@ -258,9 +267,16 @@
"blockchain": "Bitcoin",
"derivation": [
{
"name": "legacy",
"path": "m/44'/22'/0'/0/0",
"xpub": "xpub",
"xprv": "xprv"
},
{
"name": "segwit",
"path": "m/84'/22'/0'/0/0",
"xpub": "zpub",
"xprv": "zprv"
}
],
"curve": "secp256k1",
Expand Down Expand Up @@ -324,9 +340,16 @@
"blockchain": "Bitcoin",
"derivation": [
{
"name": "segwit",
"path": "m/84'/57'/0'/0/0",
"xpub": "zpub",
"xprv": "zprv"
},
{
"name": "legacy",
"path": "m/44'/57'/0'/0/0",
"xpub": "xpub",
"xprv": "xprv"
}
],
"curve": "secp256k1",
Expand Down Expand Up @@ -1517,9 +1540,16 @@
"blockchain": "Bitcoin",
"derivation": [
{
"name": "segwit",
"path": "m/84'/156'/0'/0/0",
"xpub": "zpub",
"xprv": "zprv"
},
{
"name": "legacy",
"path": "m/44'/156'/0'/0/0",
"xpub": "xpub",
"xprv": "xprv"
}
],
"curve": "secp256k1",
Expand Down Expand Up @@ -2842,9 +2872,16 @@
"blockchain": "Bitcoin",
"derivation": [
{
"name": "legacy",
"path": "m/44'/2301'/0'/0/0",
"xpub": "xpub",
"xprv": "xprv"
},
{
"name": "segwit",
"path": "m/84'/2301'/0'/0/0",
"xpub": "zpub",
"xprv": "zprv"
}
],
"curve": "secp256k1",
Expand Down Expand Up @@ -4243,6 +4280,7 @@
"blockchain": "Bitcoin",
"derivation": [
{
"name": "segwit",
"path": "m/44'/105105'/0'/0/0",
"xpub": "xpub",
"xprv": "xprv"
Expand Down
Loading

0 comments on commit 221584f

Please sign in to comment.