Merge #515: [0.17] Cleaned up CA

ac64280 Set correct ports for Liquid network params (Steven Roose)
6491b4d Use upstream Shuffle which doesn't cause DEBUG panic (Gregory Sanders)
de56cd8 move fee estimation test to extended tests, takes too long (Gregory Sanders)
1f062da Fix compilation of certain builds (Gregory Sanders)
7d86c7d remove leftover asset debug printing (Gregory Sanders)
1fec77f Add elements transaction unit test data to EXTRA_DIST (Gregory Sanders)
665d9eb Whitespace and tab linting (Gregory Sanders)
3055245 Spelling linter (Gregory Sanders)
0148024 Functional test linting (Gregory Sanders)
242515f LogPrints need \n (Gregory Sanders)
b4c9ccb remove duplicate includes in test_bitcoin (Gregory Sanders)
8371cfc Fixup blind.h guard (Gregory Sanders)
bc9a610 Fixup wallet log print (Gregory Sanders)
f56de12 Remove all but one circular dep: coinselection <-> wallet (Gregory Sanders)
5c3481a Lint RPC args (Gregory Sanders)
04a8f2e Document blindedprefix arg (Gregory Sanders)
a687dd9 Add assets and issuance functional tests (Steven Roose)
96af43b Add confidential transactions functional test (Steven Roose)
39020cd Adapt existing functional tests to CA (Steven Roose)
6113bf8 [BROKEN] Add fee outputs to functional tests (Steven Roose)
b48aea8 [BROKEN] Account for assets in all balance inspection in tests (Steven Roose)
43f8b45 [BROKEN] Adapt bitcoin_functional tests to CA (Steven Roose)
5f9f578 [BROKEN] Unbreak Qt code after CA (Steven Roose)
f7920d7 [BROKEN] Add issuance wallet RPCs (Steven Roose)
a361dda [BROKEN] Add CA wallet RPCs (Steven Roose)
b6397d6 [BROKEN] Adapt wallet unit tests to CA (Steven Roose)
d3ab44d [BROKEN] Adapt wallet to CA (Steven Roose)
49e8083 [BROKEN] Add raw issuance RPCs (Steven Roose)
a03239d [BROKEN] Add raw tx RPCs for CA (Steven Roose)
6038c37 [BROKEN] Add unit test for blinding logic (Steven Roose)
aa33202 [BROKEN] Add blinding logic (Steven Roose)
cd56626 [BROKEN] Adapt existing unit tests to CA (Steven Roose)
f4553d4 [BROKEN] Adapt existing benchmarks to CA (Steven Roose)
703db5b [BROKEN] Adapt existing RPCs to CA (Steven Roose)
6493ad9 Adapt bitcoin-tx to CA (Steven Roose)
0b50661 [BROKEN] Add CA validation (Steven Roose)
d53479c [BROKEN] Introduce CA in data structures (Steven Roose)
04d0691 Rename con_elementswitness to con_elementsmode (Steven Roose)
484d83c Add new arguments (Steven Roose)
fde9fe1 Add basic issuance boilerplate (Steven Roose)
f81122f Add asset and value blinding functionality (Steven Roose)
8b8256d Add ConfidentialCommitment-based types (Steven Roose)
d0322f7 Add initial issuance boilerplate (Steven Roose)
fd411bb Enable surjection proof module in secp256k1-zkp (Steven Roose)
84954c8 Add CAssetsDir (Steven Roose)
10773c6 Add CAsset and CAmountMap (Steven Roose)
14af0b6 Implement blech32 addresses, expose over RPC (Gregory Sanders)
0f28780 Add blech32 implementation and API (Steven Roose)
9cb2fa0 Add blech32 python implementation with diff annotation with segwit_addr.py (Gregory Sanders)
17a22f1 Add support for blinded addresses (Steven Roose)
00fba77 Add liquidv1 chainparams (Gregory Sanders)
045e615 Add CKey::ECDH (Steven Roose)
aa375e6 Replace CScriptID and CKeyID in CTxDestination with dedicated type (Gregory Sanders)
3e7cff1 Disable tests that are not relevant for Elements (Steven Roose)
3a0b3b7 Add tweakfedpeg and compatbility test (Gregory Sanders)
36a5d5d Fix bug in calculate_contract for Liquid (Steven Roose)
77ba5ee RPC params convertion cleanup (Steven Roose)
ce19a5f Add secure reporting document, link in README (Gregory Sanders)

Pull request description:

  Cleaned up version of #503.

Tree-SHA512: fb82a182a49384dd4be5838c147f6049f3b554f6d861681df46d83819c39c460b469cb68a8f6c47a2e8d3c17e439fea8818f8bca03b200027e6a127875376892

Author: stevenroose

Makefile.am

@@ -269,6 +269,8 @@ EXTRA_DIST += \
     test/util/data/txcreatesignv1.hex \
     test/util/data/txcreatesignv1.json \
     test/util/data/txcreatesignv2.hex \
+    test/util/data/txcreatemultisig4_elements.hex \
+    test/util/data/txcreatemultisig4_elements.json \
     test/util/rpcauth-test.py
 
 CLEANFILES = $(OSX_DMG) $(BITCOIN_WIN_INSTALLER)

README.md

@@ -74,3 +74,7 @@ Translations are periodically pulled from Transifex and merged into the git repo
 pull from Transifex would automatically overwrite them again.
 
 Translators should also subscribe to the [mailing list](https://groups.google.com/forum/#!forum/bitcoin-translators).
+
+Secure Reporting
+------------------
+See [our vulnerability reporting guide](SECURITY.md)

SECURITY.md

@@ -256,7 +256,7 @@ if test "x$enable_debug" = xyes; then
   # Prefer -Og, fall back to -O0 if that is unavailable.
   AX_CHECK_COMPILE_FLAG(
     [-Og],
-    [[DEBUG_CXXFLAGS="$DEBUG_CXXFLAGS -Og"]],
+    [[DEBUG_CXXFLAGS="$DEBUG_CXXFLAGS -O0"]],
     [AX_CHECK_COMPILE_FLAG([-O0],[[DEBUG_CXXFLAGS="$DEBUG_CXXFLAGS -O0"]],,[[$CXXFLAG_WERROR]])],
     [[$CXXFLAG_WERROR]])
 
@@ -1459,7 +1459,7 @@ if test x$need_bundled_univalue = xyes; then
   AC_CONFIG_SUBDIRS([src/univalue])
 fi
 
-ac_configure_args="${ac_configure_args} --disable-shared --with-pic --with-bignum=no --enable-module-recovery --disable-jni --enable-experimental --enable-module-whitelist --enable-module-rangeproof --enable-module-generator"
+ac_configure_args="${ac_configure_args} --disable-shared --with-pic --with-bignum=no --enable-module-recovery --disable-jni --enable-experimental --enable-module-whitelist --enable-module-rangeproof --enable-module-generator --enable-module-surjectionproof --enable-module-ecdh"
 AC_CONFIG_SUBDIRS([src/secp256k1])
 
 AC_OUTPUT

configure.ac

@@ -92,8 +92,12 @@ endif
 BITCOIN_CORE_H = \
   addrdb.h \
   addrman.h \
+  asset.h \
+  assetsdir.h \
   base58.h \
   bech32.h \
+  blech32.h \
+  blind.h \
   bloom.h \
   blockencodings.h \
   blockfilter.h \
@@ -111,6 +115,7 @@ BITCOIN_CORE_H = \
   compat/endian.h \
   compat/sanity.h \
   compressor.h \
+  confidential_validation.h \
   consensus/consensus.h \
   consensus/tx_verify.h \
   core_io.h \
@@ -126,6 +131,7 @@ BITCOIN_CORE_H = \
   interfaces/handler.h \
   interfaces/node.h \
   interfaces/wallet.h \
+  issuance.h \
   key.h \
   key_io.h \
   keystore.h \
@@ -223,12 +229,14 @@ libbitcoin_server_a_CXXFLAGS = $(AM_CXXFLAGS) $(PIE_FLAGS)
 libbitcoin_server_a_SOURCES = \
   addrdb.cpp \
   addrman.cpp \
+  assetsdir.cpp \
   bloom.cpp \
   blockencodings.cpp \
   blockfilter.cpp \
   block_proof.cpp \
   chain.cpp \
   checkpoints.cpp \
+  confidential_validation.cpp \
   consensus/tx_verify.cpp \
   httprpc.cpp \
   httpserver.cpp \
@@ -290,6 +298,7 @@ endif
 libbitcoin_wallet_a_CPPFLAGS = $(AM_CPPFLAGS) $(BITCOIN_INCLUDES)
 libbitcoin_wallet_a_CXXFLAGS = $(AM_CXXFLAGS) $(PIE_FLAGS)
 libbitcoin_wallet_a_SOURCES = \
+  assetsdir.cpp \
   interfaces/wallet.cpp \
   wallet/coincontrol.cpp \
   wallet/crypter.cpp \
@@ -358,6 +367,7 @@ libbitcoin_consensus_a_SOURCES = \
   amount.h \
   arith_uint256.cpp \
   arith_uint256.h \
+  asset.cpp \
   consensus/merkle.cpp \
   consensus/merkle.h \
   consensus/params.h \
@@ -367,6 +377,8 @@ libbitcoin_consensus_a_SOURCES = \
   prevector.h \
   primitives/block.cpp \
   primitives/block.h \
+  primitives/confidential.cpp \
+  primitives/confidential.h \
   primitives/txwitness.cpp \
   primitives/txwitness.h \
   primitives/transaction.cpp \
@@ -403,11 +415,14 @@ libbitcoin_common_a_CXXFLAGS = $(AM_CXXFLAGS) $(PIE_FLAGS)
 libbitcoin_common_a_SOURCES = \
   base58.cpp \
   bech32.cpp \
+  blech32.cpp \
+  blind.cpp \
   chainparams.cpp \
   coins.cpp \
   compressor.cpp \
   core_read.cpp \
   core_write.cpp \
+  issuance.cpp \
   key.cpp \
   key_io.cpp \
   keystore.cpp \

src/Makefile.am

@@ -96,7 +96,8 @@ BITCOIN_TESTS =\
   test/util_tests.cpp \
   test/validation_block_tests.cpp \
   test/versionbits_tests.cpp \
-  test/pegin_spent_tests.cpp
+  test/pegin_spent_tests.cpp \
+  test/blind_tests.cpp
 # ELEMENTS IN THE END
 
 if ENABLE_PROPERTY_TESTS

src/Makefile.test.include

@@ -0,0 +1,149 @@
+
+#include <asset.h>
+
+CAmountMap& operator+=(CAmountMap& a, const CAmountMap& b)
+{
+    for(std::map<CAsset, CAmount>::const_iterator it = b.begin(); it != b.end(); ++it)
+        a[it->first] += it->second;
+    return a;
+}
+
+CAmountMap& operator-=(CAmountMap& a, const CAmountMap& b)
+{
+    for(std::map<CAsset, CAmount>::const_iterator it = b.begin(); it != b.end(); ++it)
+        a[it->first] -= it->second;
+    return a;
+}
+
+CAmountMap operator+(const CAmountMap& a, const CAmountMap& b)
+{
+    CAmountMap c;
+    for(std::map<CAsset, CAmount>::const_iterator it = a.begin(); it != a.end(); ++it)
+        c[it->first] += it->second;
+    for(std::map<CAsset, CAmount>::const_iterator it = b.begin(); it != b.end(); ++it)
+        c[it->first] += it->second;
+    return c;
+}
+
+CAmountMap operator-(const CAmountMap& a, const CAmountMap& b)
+{
+    CAmountMap c;
+    for(std::map<CAsset, CAmount>::const_iterator it = a.begin(); it != a.end(); ++it)
+        c[it->first] += it->second;
+    for(std::map<CAsset, CAmount>::const_iterator it = b.begin(); it != b.end(); ++it)
+        c[it->first] -= it->second;
+    return c;
+}
+
+bool operator<(const CAmountMap& a, const CAmountMap& b)
+{
+    bool smallerElement = false;
+    for(std::map<CAsset, CAmount>::const_iterator it = b.begin(); it != b.end(); ++it) {
+        CAmount aValue = a.count(it->first) ? a.find(it->first)->second : 0;
+        if (aValue > it->second)
+            return false;
+        if (aValue < it->second)
+            smallerElement = true;
+    }
+    for(std::map<CAsset, CAmount>::const_iterator it = a.begin(); it != a.end(); ++it) {
+        CAmount bValue = b.count(it->first) ? b.find(it->first)->second : 0;
+        if (it->second > bValue)
+            return false;
+        if (it->second < bValue)
+            smallerElement = true;
+    }
+    return smallerElement;
+}
+
+bool operator<=(const CAmountMap& a, const CAmountMap& b)
+{
+    for(std::map<CAsset, CAmount>::const_iterator it = b.begin(); it != b.end(); ++it) {
+        CAmount aValue = a.count(it->first) ? a.find(it->first)->second : 0;
+        if (aValue > it->second)
+            return false;
+    }
+    for(std::map<CAsset, CAmount>::const_iterator it = a.begin(); it != a.end(); ++it) {
+        CAmount bValue = b.count(it->first) ? b.find(it->first)->second : 0;
+        if (it->second > bValue)
+            return false;
+    }
+    return true;
+}
+
+bool operator>(const CAmountMap& a, const CAmountMap& b)
+{
+    bool largerElement = false;
+    for(std::map<CAsset, CAmount>::const_iterator it = b.begin(); it != b.end(); ++it) {
+        CAmount aValue = a.count(it->first) ? a.find(it->first)->second : 0;
+        if (aValue < it->second)
+            return false;
+        if (aValue > it->second)
+            largerElement = true;
+    }
+    for(std::map<CAsset, CAmount>::const_iterator it = a.begin(); it != a.end(); ++it) {
+        CAmount bValue = b.count(it->first) ? b.find(it->first)->second : 0;
+        if (it->second < bValue)
+            return false;
+        if (it->second > bValue)
+            largerElement = true;
+    }
+    return largerElement;
+}
+
+bool operator>=(const CAmountMap& a, const CAmountMap& b)
+{
+    for(std::map<CAsset, CAmount>::const_iterator it = b.begin(); it != b.end(); ++it) {
+        if ((a.count(it->first) ? a.find(it->first)->second : 0) < it->second)
+            return false;
+    }
+    for(std::map<CAsset, CAmount>::const_iterator it = a.begin(); it != a.end(); ++it) {
+        if (it->second < (b.count(it->first) ? b.find(it->first)->second : 0))
+            return false;
+    }
+    return true;
+}
+
+bool operator==(const CAmountMap& a, const CAmountMap& b)
+{
+    for(std::map<CAsset, CAmount>::const_iterator it = a.begin(); it != a.end(); ++it) {
+        if ((b.count(it->first) ? b.find(it->first)->second : 0) != it->second)
+            return false;
+    }
+    for(std::map<CAsset, CAmount>::const_iterator it = b.begin(); it != b.end(); ++it) {
+        if ((a.count(it->first) ? a.find(it->first)->second : 0) != it->second)
+            return false;
+    }
+    return true;
+}
+
+bool operator!=(const CAmountMap& a, const CAmountMap& b)
+{
+    return !(a == b);
+}
+
+bool hasNegativeValue(const CAmountMap& amount)
+{
+    for(std::map<CAsset, CAmount>::const_iterator it = amount.begin(); it != amount.end(); ++it) {
+        if (it->second < 0)
+            return true;
+    }
+    return false;
+}
+
+bool hasNonPostiveValue(const CAmountMap& amount)
+{
+    for(std::map<CAsset, CAmount>::const_iterator it = amount.begin(); it != amount.end(); ++it) {
+        if (it->second <= 0)
+            return true;
+    }
+    return false;
+}
+
+CAmount valueFor(const CAmountMap& mapValue, const CAsset& asset) {
+    CAmountMap::const_iterator it = mapValue.find(asset);
+    if (it != mapValue.end()) {
+        return it->second;
+    } else {
+        return CAmount(0);
+    }
+}

src/asset.cpp

@@ -0,0 +1,108 @@
+
+#ifndef BITCOIN_ASSET_H
+#define BITCOIN_ASSET_H
+
+#include <uint256.h>
+
+#include <amount.h>
+#include <serialize.h>
+
+/**
+ *  Native Asset Issuance
+ *
+ *  An asset identifier tag, a 256 bits serialized hash (sha256) defined
+ *  by the issuance transaction from which the output’s coins are derived.
+ *  Each output contains coins from a single asset/currency.
+ *  For the host currency, the similarly-calculated hash of the chain’s genesis
+ *  block is used instead.
+**/
+struct CAsset {
+    uint256 id;
+
+    CAsset() { }
+    explicit CAsset(const uint256& idIn) : id(idIn) { }
+    explicit CAsset(const std::vector<unsigned char>& vchIDIn) : id(vchIDIn) { }
+
+    ADD_SERIALIZE_METHODS;
+
+    template <typename Stream, typename Operation>
+    inline void SerializationOp(Stream& s, Operation ser_action) {
+        READWRITE(id);
+    }
+
+    bool IsNull() const { return id.IsNull(); }
+    void SetNull() { id.SetNull(); }
+
+    unsigned char* begin() { return id.begin(); }
+    unsigned char* end() { return id.end(); }
+    const unsigned char* begin() const { return id.begin(); }
+    const unsigned char* end() const { return id.end(); }
+
+    std::string GetHex() const { return id.GetHex(); }
+    void SetHex(const std::string& str) { id.SetHex(str); }
+
+    friend bool operator==(const CAsset& a, const CAsset& b)
+    {
+        return a.id == b.id;
+    }
+
+    friend bool operator!=(const CAsset& a, const CAsset& b)
+    {
+        return !(a == b);
+    }
+
+    friend bool operator<(const CAsset& a, const CAsset& b)
+    {
+        return a.id < b.id;
+    }
+};
+
+/** Used for consensus fee and general wallet accounting*/
+typedef std::map<CAsset, CAmount> CAmountMap;
+
+CAmountMap& operator+=(CAmountMap& a, const CAmountMap& b);
+CAmountMap& operator-=(CAmountMap& a, const CAmountMap& b);
+CAmountMap operator+(const CAmountMap& a, const CAmountMap& b);
+CAmountMap operator-(const CAmountMap& a, const CAmountMap& b);
+
+// WARNING: Comparisons are only looking for *complete* ordering.
+// For strict inequality checks, if any entry would fail the non-strict
+// inequality, the comparison will fail. Therefore it is possible
+// that all inequality comparison checks may fail.
+// Therefore if >/< fails against a CAmountMap(), this means there
+// are all zeroes or one or more negative values.
+//
+// Examples: 1A + 2B <= 1A + 2B + 1C
+//      and  1A + 2B <  1A + 2B + 1C
+//                   but
+//           !(1A + 2B == 1A + 2B + 1C)
+//-------------------------------------
+//           1A + 2B == 1A + 2B
+//      and  1A + 2B <= 1A + 2B
+//                   but
+//           !(1A + 2B < 1A + 2B)
+//-------------------------------------
+//           !(1A + 2B == 2B - 1C)
+//           !(1A + 2B >= 2B - 1C)
+//                     ...
+//           !(1A + 2B < 2B - 1C)
+//      and   1A + 2B != 2B - 1C
+bool operator<(const CAmountMap& a, const CAmountMap& b);
+bool operator<=(const CAmountMap& a, const CAmountMap& b);
+bool operator>(const CAmountMap& a, const CAmountMap& b);
+bool operator>=(const CAmountMap& a, const CAmountMap& b);
+bool operator==(const CAmountMap& a, const CAmountMap& b);
+bool operator!=(const CAmountMap& a, const CAmountMap& b);
+
+inline bool MoneyRange(const CAmountMap& mapValue) {
+    for(CAmountMap::const_iterator it = mapValue.begin(); it != mapValue.end(); it++) {
+        if (it->second < 0 || it->second > MAX_MONEY) {
+            return false;
+        }
+    }
+   return true;
+}
+
+CAmount valueFor(const CAmountMap& mapValue, const CAsset& asset);
+
+#endif //  BITCOIN_AMOUNT_H