Sync with the main repo

.cirrus.yml

@@ -90,7 +90,7 @@ task:
     QT_DOWNLOAD_URL: 'https://download.qt.io/official_releases/qt/5.12/5.12.11/single/qt-everywhere-src-5.12.11.zip'
     QT_LOCAL_PATH: 'C:\qt-everywhere-src-5.12.11.zip'
     QT_SOURCE_DIR: 'C:\qt-everywhere-src-5.12.11'
-    QTBASEDIR: 'C:\Qt5.12.11_x64_static_vs2019_160900'
+    QTBASEDIR: 'C:\Qt_static'
     x64_NATIVE_TOOLS: '"C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Auxiliary\Build\vcvars64.bat"'
     IgnoreWarnIntDirInTempDetected: 'true'
   merge_script:

build_msvc/README.md

@@ -3,67 +3,65 @@ Building Bitcoin Core with Visual Studio
 
 Introduction
 ---------------------
-Solution and project files to build the Bitcoin Core applications `msbuild` or Visual Studio can be found in the `build_msvc` directory. The build has been tested with Visual Studio 2019 (building with earlier versions of Visual Studio should not be expected to work).
+Solution and project files to build Bitcoin Core with `msbuild` or Visual Studio can be found in the `build_msvc` directory. The build has been tested with Visual Studio 2019 (building with earlier versions of Visual Studio should not be expected to work).
 
-Building with Visual Studio is an alternative to the Linux based [cross-compiler build](https://github.com/bitcoin/bitcoin/blob/master/doc/build-windows.md).
+To build Bitcoin Core from the command-line, it is sufficient to only install the Visual Studio Build Tools component.
 
-Quick Start
----------------------
-The minimal steps required to build Bitcoin Core with the msbuild toolchain are below. More detailed instructions are contained in the following sections.
+Building with Visual Studio is an alternative to the Linux based [cross-compiler build](../doc/build-windows.md).
 
-```
-cd build_msvc
-py -3 msvc-autogen.py
-msbuild /m bitcoin.sln /p:Platform=x64 /p:Configuration=Release /t:build
-```
 
-Dependencies
+Prerequisites
 ---------------------
-A number of [open source libraries](https://github.com/bitcoin/bitcoin/blob/master/doc/dependencies.md) are required in order to be able to build Bitcoin Core.
+To build [dependencies](../doc/dependencies.md) (except for [Qt](#qt)),
+the default approach is to use the [vcpkg](https://docs.microsoft.com/en-us/cpp/vcpkg) package manager from Microsoft:
 
-Options for installing the dependencies in a Visual Studio compatible manner are:
+1. [Install](https://vcpkg.io/en/getting-started.html) vcpkg.
 
-- Use Microsoft's [vcpkg](https://docs.microsoft.com/en-us/cpp/vcpkg) to download the source packages and build locally. This is the recommended approach.
-- Download the source code, build each dependency, add the required include paths, link libraries and binary tools to the Visual Studio project files.
-- Use [nuget](https://www.nuget.org/) packages with the understanding that any binary files have been compiled by an untrusted third party.
+2. By default, vcpkg makes both `release` and `debug` builds for each package.
+To save build time and disk space, one could skip `debug` builds (example uses PowerShell):
+```powershell
 
-The [external dependencies](https://github.com/bitcoin/bitcoin/blob/master/doc/dependencies.md) required for building are listed in the `build_msvc/vcpkg.json` file. To ensure `msbuild` project files automatically install the `vcpkg` dependencies use:
-
-```
-vcpkg integrate install
+Add-Content -Path "vcpkg\triplets\x64-windows-static.cmake" -Value "set(VCPKG_BUILD_TYPE release)"
 ```
 
 Qt
 ---------------------
-In order to build Bitcoin Core a static build of Qt is required. The runtime library version (e.g. v142) and platform type (x86 or x64) must also match.
+To build Bitcoin Core with the GUI, a static build of Qt is required.
+
+1. Download a single ZIP archive of Qt source code from https://download.qt.io/official_releases/qt/ (e.g., [`qt-everywhere-src-5.12.11.zip`](https://download.qt.io/official_releases/qt/5.12/5.12.11/single/qt-everywhere-src-5.12.11.zip)), and expand it into a dedicated folder. The following instructions assume that this folder is `C:\dev\qt-source`.
+
+2. Open "x64 Native Tools Command Prompt for VS 2019", and input the following commands:
+```cmd
+cd C:\dev\qt-source
+mkdir build
+cd build
+..\configure -release -silent -opensource -confirm-license -opengl desktop -no-shared -static -static-runtime -mp -qt-zlib -qt-pcre -qt-libpng -no-libjpeg -nomake examples -nomake tests -nomake tools -no-dbus -no-libudev -no-icu -no-gtk -no-opengles3 -no-angle -no-sql-sqlite -no-sql-odbc -no-sqlite -no-libudev -no-vulkan -skip qt3d -skip qtactiveqt -skip qtandroidextras -skip qtcanvas3d -skip qtcharts -skip qtconnectivity -skip qtdatavis3d -skip qtdeclarative -skip qtdoc -skip qtgamepad -skip qtgraphicaleffects -skip qtimageformats -skip qtlocation -skip qtmacextras -skip qtmultimedia -skip qtnetworkauth -skip qtpurchasing -skip qtquickcontrols -skip qtquickcontrols2 -skip qtscript -skip qtscxml -skip qtsensors -skip qtserialbus -skip qtserialport -skip qtspeech -skip qtvirtualkeyboard -skip qtwayland -skip qtwebchannel -skip qtwebengine -skip qtwebsockets -skip qtwebview -skip qtx11extras -skip qtxmlpatterns -no-openssl -no-feature-sql -no-feature-sqlmodel -prefix C:\Qt_static
+nmake
+nmake install
+```
 
-Some prebuilt x64 versions of Qt can be downloaded from [here](https://github.com/sipsorcery/qt_win_binary/releases). Please be aware these downloads are NOT officially sanctioned by Bitcoin Core and are provided for developer convenience only. They should NOT be used for builds that will be used in a production environment or with real funds.
+One could speed up building with [`jom`](https://wiki.qt.io/Jom), a replacement for `nmake` which makes use of all CPU cores.
 
-To determine which Qt prebuilt version to download open the `.cirrus.yml` file and note the `QT_DOWNLOAD_URL`. When extracting the zip file the destination path must be set to `C:\`. This is due to the way that Qt includes, libraries and tools use internal paths.
+To build Bitcoin Core without Qt, unload or disable the `bitcoin-qt`, `libbitcoin_qt` and `test_bitcoin-qt` projects.
 
-To build Bitcoin Core without Qt unload or disable the `bitcoin-qt`, `libbitcoin_qt` and `test_bitcoin-qt` projects.
 
 Building
 ---------------------
-The instructions below use `vcpkg` to install the dependencies.
-
-- Install [`vcpkg`](https://github.com/Microsoft/vcpkg).
-
-- Use Python to generate `*.vcxproj` from Makefile
+1. Use Python to generate `*.vcxproj` from Makefile:
 
 ```
 PS >py -3 msvc-autogen.py
 ```
 
-- An optional step is to adjust the settings in the `build_msvc` directory and the `common.init.vcxproj` file. This project file contains settings that are common to all projects such as the runtime library version and target Windows SDK version. The Qt directories can also be set.
+2. An optional step is to adjust the settings in the `build_msvc` directory and the `common.init.vcxproj` file. This project file contains settings that are common to all projects such as the runtime library version and target Windows SDK version. The Qt directories can also be set. To specify a non-default path to a static Qt package directory, use the `QTBASEDIR` environment variable.
 
-- To build from the command line with the Visual Studio 2019 toolchain use:
+3. To build from the command-line with the Visual Studio 2019 toolchain use:
 
-```
-msbuild /m bitcoin.sln /p:Platform=x64 /p:Configuration=Release /t:build
+```cmd
+msbuild -property:Configuration=Release -maxCpuCount -verbosity:minimal bitcoin.sln
 ```
 
-- Alternatively, open the `build_msvc/bitcoin.sln` file in Visual Studio 2019.
+Alternatively, open the `build_msvc/bitcoin.sln` file in Visual Studio 2019.
 
 Security
 ---------------------

build_msvc/common.qt.init.vcxproj

@@ -2,7 +2,7 @@
 <Project xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
 
   <PropertyGroup Label="QtGlobals">
-    <QtBaseDir>C:\Qt5.12.11_x64_static_vs2019_160900</QtBaseDir>
+    <QtBaseDir Condition="'$(QTBASEDIR)' == ''">C:\Qt_static</QtBaseDir>
     <QtPluginsLibraryDir>$(QtBaseDir)\plugins</QtPluginsLibraryDir>
     <QtLibraryDir>$(QtBaseDir)\lib</QtLibraryDir>
     <QtIncludeDir>$(QtBaseDir)\include</QtIncludeDir>

ci/test/00_setup_env_i686_multiprocess.sh

@@ -14,4 +14,5 @@ export DEP_OPTS="DEBUG=1 MULTIPROCESS=1"
 export GOAL="install"
 export BITCOIN_CONFIG="--enable-debug CC='clang -m32' CXX='clang++ -m32' LDFLAGS='--rtlib=compiler-rt -lgcc_s'"
 export TEST_RUNNER_ENV="BITCOIND=bitcoin-node"
+export TEST_RUNNER_EXTRA="--nosandbox"
 export PIP_PACKAGES="lief"

configure.ac

@@ -71,6 +71,12 @@ case $host in
   ;;
 esac
 
+AC_ARG_WITH([seccomp],
+  [AS_HELP_STRING([--with-seccomp],
+  [enable experimental syscall sandbox feature (-sandbox), default is yes if seccomp-bpf is detected under Linux x86_64])],
+  [seccomp_found=$withval],
+  [seccomp_found=auto])
+
 dnl Require C++17 compiler (no GNU extensions)
 AX_CXX_COMPILE_STDCXX([17], [noext], [mandatory])
 
@@ -101,7 +107,7 @@ AC_PATH_TOOL(GCOV, gcov)
 AC_PATH_TOOL(LLVM_COV, llvm-cov)
 AC_PATH_PROG(LCOV, lcov)
 dnl Python 3.6 is specified in .python-version and should be used if available, see doc/dependencies.md
-AC_PATH_PROGS([PYTHON], [python3.6 python3.7 python3.8 python3.9 python3 python])
+AC_PATH_PROGS([PYTHON], [python3.6 python3.7 python3.8 python3.9, python3.10, python3 python])
 AC_PATH_PROG(GENHTML, genhtml)
 AC_PATH_PROG([GIT], [git])
 AC_PATH_PROG(CCACHE,ccache)
@@ -1432,6 +1438,36 @@ if test "x$use_external_signer" != xno; then
 fi
 AM_CONDITIONAL([ENABLE_EXTERNAL_SIGNER], [test "x$use_external_signer" = "xyes"])
 
+dnl Do not compile with syscall sandbox support when compiling under the sanitizers.
+dnl The sanitizers introduce use of syscalls that are not typically used in bitcoind
+dnl (such as execve when the sanitizers execute llvm-symbolizer).
+if test x$use_sanitizers != x; then
+  AC_MSG_WARN(Specifying --with-sanitizers forces --without-seccomp since the sanitizers introduce use of syscalls not allowed by the bitcoind syscall sandbox (-sandbox=<mode>).)
+  seccomp_found=no
+fi
+if test "x$seccomp_found" != "xno"; then
+  AC_MSG_CHECKING([for seccomp-bpf (Linux x86-64)])
+  AC_PREPROC_IFELSE([AC_LANG_PROGRAM([[
+      @%:@include <linux/seccomp.h>
+    ]], [[
+      #if !defined(__x86_64__)
+      #  error Syscall sandbox is an experimental feature currently available only under Linux x86-64.
+      #endif
+    ]])],[
+      AC_MSG_RESULT(yes)
+      seccomp_found="yes"
+      AC_DEFINE(USE_SYSCALL_SANDBOX, 1, [Define this symbol to build with syscall sandbox support.])
+    ],[
+      AC_MSG_RESULT(no)
+      seccomp_found="no"
+  ])
+fi
+dnl Currently only enable -sandbox=<mode> feature if seccomp is found.
+dnl In the future, sandboxing could be also be supported with other
+dnl sandboxing mechanisms besides seccomp.
+use_syscall_sandbox=$seccomp_found
+AM_CONDITIONAL([ENABLE_SYSCALL_SANDBOX], [test "x$use_syscall_sandbox" != "xno"])
+
 dnl Check for reduced exports
 if test x$use_reduce_exports = xyes; then
   AX_CHECK_COMPILE_FLAG([-fvisibility=hidden],[CXXFLAGS="$CXXFLAGS -fvisibility=hidden"],
@@ -1925,6 +1961,7 @@ echo
 echo "Options used to compile and link:"
 echo "  external signer = $use_external_signer"
 echo "  multiprocess    = $build_multiprocess"
+echo "  with experimental syscall sandbox support = $use_syscall_sandbox"
 echo "  with libs       = $build_bitcoin_libs"
 echo "  with wallet     = $enable_wallet"
 if test "x$enable_wallet" != "xno"; then

contrib/signet/getcoins.py

@@ -4,18 +4,77 @@
 # file COPYING or http://www.opensource.org/licenses/mit-license.php.
 
 import argparse
+import io
+import requests
 import subprocess
 import sys
-import requests
 
 DEFAULT_GLOBAL_FAUCET = 'https://signetfaucet.com/claim'
+DEFAULT_GLOBAL_CAPTCHA = 'https://signetfaucet.com/captcha'
 GLOBAL_FIRST_BLOCK_HASH = '00000086d6b2636cb2a392d45edc4ec544a10024d30141c9adf4bfd9de533b53'
 
+# braille unicode block
+BASE = 0x2800
+BIT_PER_PIXEL = [
+    [0x01, 0x08],
+    [0x02, 0x10],
+    [0x04, 0x20],
+    [0x40, 0x80],
+]
+BW = 2
+BH = 4
+
+# imagemagick or compatible fork (used for converting SVG)
+CONVERT = 'convert'
+
+class PPMImage:
+    '''
+    Load a PPM image (Pillow-ish API).
+    '''
+    def __init__(self, f):
+        if f.readline() != b'P6\n':
+            raise ValueError('Invalid ppm format: header')
+        line = f.readline()
+        (width, height) = (int(x) for x in line.rstrip().split(b' '))
+        if f.readline() != b'255\n':
+            raise ValueError('Invalid ppm format: color depth')
+        data = f.read(width * height * 3)
+        stride = width * 3
+        self.size = (width, height)
+        self._grid = [[tuple(data[stride * y + 3 * x:stride * y + 3 * (x + 1)]) for x in range(width)] for y in range(height)]
+
+    def getpixel(self, pos):
+        return self._grid[pos[1]][pos[0]]
+
+def print_image(img, threshold=128):
+    '''Print black-and-white image to terminal in braille unicode characters.'''
+    x_blocks = (img.size[0] + BW - 1) // BW
+    y_blocks = (img.size[1] + BH - 1) // BH
+
+    for yb in range(y_blocks):
+        line = []
+        for xb in range(x_blocks):
+            ch = BASE
+            for y in range(BH):
+                for x in range(BW):
+                    try:
+                        val = img.getpixel((xb * BW + x, yb * BH + y))
+                    except IndexError:
+                        pass
+                    else:
+                        if val[0] < threshold:
+                            ch |= BIT_PER_PIXEL[y][x]
+            line.append(chr(ch))
+        print(''.join(line))
+
 parser = argparse.ArgumentParser(description='Script to get coins from a faucet.', epilog='You may need to start with double-dash (--) when providing bitcoin-cli arguments.')
 parser.add_argument('-c', '--cmd', dest='cmd', default='bitcoin-cli', help='bitcoin-cli command to use')
 parser.add_argument('-f', '--faucet', dest='faucet', default=DEFAULT_GLOBAL_FAUCET, help='URL of the faucet')
+parser.add_argument('-g', '--captcha', dest='captcha', default=DEFAULT_GLOBAL_CAPTCHA, help='URL of the faucet captcha, or empty if no captcha is needed')
 parser.add_argument('-a', '--addr', dest='addr', default='', help='Bitcoin address to which the faucet should send')
 parser.add_argument('-p', '--password', dest='password', default='', help='Faucet password, if any')
+parser.add_argument('-n', '--amount', dest='amount', default='0.001', help='Amount to request (0.001-0.1, default is 0.001)')
+parser.add_argument('-i', '--imagemagick', dest='imagemagick', default=CONVERT, help='Path to imagemagick convert utility')
 parser.add_argument('bitcoin_cli_args', nargs='*', help='Arguments to pass on to bitcoin-cli (default: -signet)')
 
 args = parser.parse_args()
@@ -43,14 +102,43 @@ def bitcoin_cli(rpc_command_and_params):
     if curr_signet_hash != GLOBAL_FIRST_BLOCK_HASH:
         print('The global faucet cannot be used with a custom Signet network. Please use the global signet or setup your custom faucet to use this functionality.\n')
         exit(1)
+else:
+    # For custom faucets, don't request captcha by default.
+    if args.captcha == DEFAULT_GLOBAL_CAPTCHA:
+        args.captcha = ''
 
 if args.addr == '':
     # get address for receiving coins
     args.addr = bitcoin_cli(['getnewaddress', 'faucet', 'bech32'])
 
-data = {'address': args.addr, 'password': args.password}
+data = {'address': args.addr, 'password': args.password, 'amount': args.amount}
+
+# Store cookies
+# for debugging: print(session.cookies.get_dict())
+session = requests.Session()
+
+if args.captcha != '': # Retrieve a captcha
+    try:
+        res = session.get(args.captcha)
+    except:
+        print('Unexpected error when contacting faucet:', sys.exc_info()[0])
+        exit(1)
+
+    # Convert SVG image to PPM, and load it
+    try:
+        rv = subprocess.run([args.imagemagick, '-', '-depth', '8', 'ppm:-'], input=res.content, check=True, capture_output=True)
+    except FileNotFoundError:
+        print('The binary', args.imagemagick, 'could not be found. Please make sure ImageMagick (or a compatible fork) is installed and that the correct path is specified.')
+        exit(1)
+    img = PPMImage(io.BytesIO(rv.stdout))
+
+    # Terminal interaction
+    print_image(img)
+    print('Enter captcha: ', end='')
+    data['captcha'] = input()
+
 try:
-    res = requests.post(args.faucet, data=data)
+    res = session.post(args.faucet, data=data)
 except:
     print('Unexpected error when contacting faucet:', sys.exc_info()[0])
     exit(1)

doc/release-notes-22539.md

@@ -0,0 +1,8 @@
+Notable changes
+===============
+
+P2P and network changes
+-----------------------
+
+- Fee estimation now takes the feerate of replacement (RBF) transactions into
+  account.

src/Makefile.am

@@ -117,6 +117,7 @@ endif
 BITCOIN_CORE_H = \
   addrdb.h \
   addrman.h \
+  addrman_impl.h \
   attributes.h \
   banman.h \
   base58.h \
@@ -261,6 +262,7 @@ BITCOIN_CORE_H = \
   util/sock.h \
   util/spanparsing.h \
   util/string.h \
+  util/syscall_sandbox.h \
   util/system.h \
   util/thread.h \
   util/threadnames.h \
@@ -498,9 +500,9 @@ crypto_libbitcoin_crypto_shani_a_SOURCES = crypto/sha256_shani.cpp
 libbitcoin_consensus_a_CPPFLAGS = $(AM_CPPFLAGS) $(BITCOIN_INCLUDES)
 libbitcoin_consensus_a_CXXFLAGS = $(AM_CXXFLAGS) $(PIE_FLAGS)
 libbitcoin_consensus_a_SOURCES = \
-  amount.h \
   arith_uint256.cpp \
   arith_uint256.h \
+  consensus/amount.h \
   consensus/merkle.cpp \
   consensus/merkle.h \
   consensus/params.h \
@@ -611,6 +613,7 @@ libbitcoin_util_a_SOURCES = \
   util/spanparsing.cpp \
   util/strencodings.cpp \
   util/string.cpp \
+  util/syscall_sandbox.cpp \
   util/time.cpp \
   util/tokenpipe.cpp \
   $(BITCOIN_CORE_H)

src/Makefile.test.include

@@ -119,8 +119,9 @@ BITCOIN_TESTS =\
   test/sanity_tests.cpp \
   test/scheduler_tests.cpp \
   test/script_p2sh_tests.cpp \
-  test/script_tests.cpp \
+  test/script_parse_tests.cpp \
   test/script_standard_tests.cpp \
+  test/script_tests.cpp \
   test/scriptnum_tests.cpp \
   test/serfloat_tests.cpp \
   test/serialize_tests.cpp \