Merge dashpay#6802: backport: 0.25 batch 587

9e0f19f Merge bitcoin#28512: doc: Be vague instead of wrong about MALLOC_ARENA_MAX (PastaBot)
d04979f Merge bitcoin#29821: fuzz: Some `test/fuzz/test_runner.py` improvements (PastaBot)
c9ffe01 Merge bitcoin#27423: ci: fix git dubious permissions error (PastaBot)
d4b420f Merge bitcoin#29805: test: Fix debug recommendation in argsman_tests (PastaBot)
78a5017 Merge bitcoin#29776: ThreadSanitizer: Fix bitcoin#29767 (PastaBot)
6c24671 Merge bitcoin#29687: cli: improve bitcoin-cli error when not connected (PastaBot)
4f1d6e7 Merge bitcoin#29764: doc: Suggest installing dev packages for debian/ubuntu qt5 build (PastaBot)
6a5e1c2 Merge bitcoin#29094: ci: Better tidy errors (PastaBot)
08eb23b Merge bitcoin#27919: ci: Run fuzz target even if input folder is empty (PastaBot)

Pull request description:

  ## Issue being fixed or feature implemented
  batch of backports

  ## What was done?

  ## How Has This Been Tested?

  ## Breaking Changes

  ## Checklist:
    _Go over all the following points, and put an `x` in all the boxes that apply._
  - [ ] I have performed a self-review of my own code
  - [ ] I have commented my code, particularly in hard-to-understand areas
  - [ ] I have added or updated relevant unit/integration/functional/e2e tests
  - [ ] I have made corresponding changes to the documentation
  - [x] I have assigned this pull request to a milestone _(for repository code-owners and collaborators only)_

ACKs for top commit:
  knst:
    utACK 9e0f19f

Tree-SHA512: 1c16cd9402cec0490df96044e98a33e87b221a162885ad3bf87ef641af9cbd5fa9a7179bc13fccf6ff14cf22ba967fc8f992e5c9cc486fc04287a50bcd16579c

Author: PastaPastaPasta

ci/dash/lint-tidy.sh

@@ -12,7 +12,11 @@ set -eo pipefail
 #          that *before* running this script.
 
 cd "${BASE_ROOT_DIR}/build-ci/dashcore-${BUILD_TARGET}/src"
-( run-clang-tidy -quiet "${MAKEJOBS}" ) | grep -C5 "error"
+if ! ( run-clang-tidy -quiet "${MAKEJOBS}" | tee tmp.tidy-out.txt ); then
+  grep -C5 "error: " tmp.tidy-out.txt
+  echo "^^^ ⚠️ Failure generated from clang-tidy"
+  false
+fi
 
 cd "${BASE_ROOT_DIR}/build-ci/dashcore-${BUILD_TARGET}"
 iwyu_tool.py \

ci/test/04_install.sh

@@ -61,6 +61,9 @@ if [ -z "$DANGER_RUN_CI_ON_HOST" ]; then
   docker exec "$DOCKER_ID" chown -R "$LOCAL_USER":"$LOCAL_USER" "${BASE_ROOT_DIR}"
   export DOCKER_CI_CMD_PREFIX_ROOT="docker exec -u 0 $DOCKER_ID"
   export DOCKER_CI_CMD_PREFIX="docker exec -u $LOCAL_UID $DOCKER_ID"
+  # Fixes permission issues when there is a container UID/GID mismatch with the owner
+  # of the mounted dash src dir.
+  $DOCKER_CI_CMD_PREFIX_ROOT git config --global --add safe.directory "*"
 else
   echo "Running on host system without docker wrapper"
 fi

doc/build-unix.md

@@ -107,7 +107,7 @@ To build without GUI pass `--without-gui`.
 To build with Qt 5 you need the following:
 
 ```sh
-sudo apt-get install libqt5gui5 libqt5core5a libqt5dbus5 qttools5-dev qttools5-dev-tools
+sudo apt-get install qtbase5-dev qttools5-dev qttools5-dev-tools
 ```
 
 Additionally, to support Wayland protocol for modern desktop environments:

doc/reduce-memory.md

@@ -43,7 +43,7 @@ threads take up 8MiB for the thread stack on a 64-bit system, and 4MiB in a
 
 ## Linux specific
 
-By default, since glibc `2.10`, the C library will create up to two heap arenas per core. This is known to cause excessive memory usage in some scenarios. To avoid this make a script that sets `MALLOC_ARENA_MAX` before starting dashd:
+By default, glibc's implementation of `malloc` may use more than one arena. This is known to cause excessive memory usage in some scenarios. To avoid this, make a script that sets `MALLOC_ARENA_MAX` before starting dashd:
 
 ```bash
 #!/usr/bin/env bash

src/bitcoin-cli.cpp

@@ -828,7 +828,10 @@ static UniValue CallRPC(BaseRequestHandler* rh, const std::string& strMethod, co
         if (response.error != -1) {
             responseErrorMessage = strprintf(" (error code %d - \"%s\")", response.error, http_errorstring(response.error));
         }
-        throw CConnectionFailed(strprintf("Could not connect to the server %s:%d%s\n\nMake sure the dashd server is running and that you are connecting to the correct RPC port.", host, port, responseErrorMessage));
+        throw CConnectionFailed(strprintf("Could not connect to the server %s:%d%s\n\n"
+                    "Make sure the dashd server is running and that you are connecting to the correct RPC port.\n"
+                    "Use \"dash-cli -help\" for more info.",
+                    host, port, responseErrorMessage));
     } else if (response.status == HTTP_UNAUTHORIZED) {
         if (failedToGetAuthCookie) {
             throw std::runtime_error(strprintf(

src/index/base.cpp

@@ -151,9 +151,9 @@ void BaseIndex::ThreadSync()
                 const CBlockIndex* pindex_next = NextSyncBlock(pindex, m_chainstate->m_chain);
                 if (!pindex_next) {
                     SetBestBlockIndex(pindex);
-                    m_synced = true;
                     // No need to handle errors in Commit. See rationale above.
                     Commit();
+                    m_synced = true;
                     break;
                 }
                 if (pindex_next->pprev != pindex && !Rewind(pindex, pindex_next->pprev)) {

src/test/argsman_tests.cpp

@@ -903,7 +903,7 @@ BOOST_FIXTURE_TEST_CASE(util_ArgsMerge, ArgsMergeTestingSetup)
 
     // If check below fails, should manually dump the results with:
     //
-    //   ARGS_MERGE_TEST_OUT=results.txt ./test_dash --run_test=util_tests/util_ArgsMerge
+    //   ARGS_MERGE_TEST_OUT=results.txt ./test_dash --run_test=argsman_tests/util_ArgsMerge
     //
     // And verify diff against previous results to make sure the changes are expected.
     //
@@ -1006,7 +1006,7 @@ BOOST_FIXTURE_TEST_CASE(util_ChainMerge, ChainMergeTestingSetup)
 
     // If check below fails, should manually dump the results with:
     //
-    //   CHAIN_MERGE_TEST_OUT=results.txt ./test_dash --run_test=util_tests/util_ChainMerge
+    //   CHAIN_MERGE_TEST_OUT=results.txt ./test_dash --run_test=argsman_tests/util_ChainMerge
     //
     // And verify diff against previous results to make sure the changes are expected.
     //

test/fuzz/test_runner.py

@@ -6,6 +6,7 @@
 """
 
 from concurrent.futures import ThreadPoolExecutor, as_completed
+from pathlib import Path
 import argparse
 import configparser
 import logging
@@ -41,6 +42,11 @@ def main():
         action='store_true',
         help='If true, run fuzzing binaries under the valgrind memory error detector',
     )
+    parser.add_argument(
+        "--empty_min_time",
+        type=int,
+        help="If set, run at least this long, if the existing fuzz inputs directory is empty.",
+    )
     parser.add_argument(
         '-x',
         '--exclude',
@@ -76,6 +82,7 @@ def main():
     )
 
     args = parser.parse_args()
+    args.corpus_dir = Path(args.corpus_dir)
 
     # Set up logging
     logging.basicConfig(
@@ -92,9 +99,11 @@ def main():
         logging.error("Must have fuzz targets built")
         sys.exit(1)
 
+    fuzz_bin=os.getenv("DASHFUZZ", default=os.path.join(config["environment"]["BUILDDIR"], 'src', 'test', 'fuzz', 'fuzz'))
+
     # Build list of tests
     test_list_all = parse_test_list(
-        fuzz_bin=os.path.join(config["environment"]["BUILDDIR"], 'src', 'test', 'fuzz', 'fuzz'),
+        fuzz_bin=fuzz_bin,
         source_dir=config['environment']['SRCDIR'],
     )
 
@@ -139,7 +148,7 @@ def main():
     try:
         help_output = subprocess.run(
             args=[
-                os.path.join(config["environment"]["BUILDDIR"], 'src', 'test', 'fuzz', 'fuzz'),
+                fuzz_bin,
                 '-help=1',
             ],
             env=get_fuzz_env(target=test_list_selection[0], source_dir=config['environment']['SRCDIR']),
@@ -160,7 +169,7 @@ def main():
             return generate_corpus(
                 fuzz_pool=fuzz_pool,
                 src_dir=config['environment']['SRCDIR'],
-                build_dir=config["environment"]["BUILDDIR"],
+                fuzz_bin=fuzz_bin,
                 corpus_dir=args.corpus_dir,
                 targets=test_list_selection,
             )
@@ -171,7 +180,7 @@ def main():
                 corpus=args.corpus_dir,
                 test_list=test_list_selection,
                 src_dir=config['environment']['SRCDIR'],
-                build_dir=config["environment"]["BUILDDIR"],
+                fuzz_bin=fuzz_bin,
                 merge_dir=args.m_dir,
             )
             return
@@ -181,12 +190,13 @@ def main():
             corpus=args.corpus_dir,
             test_list=test_list_selection,
             src_dir=config['environment']['SRCDIR'],
-            build_dir=config["environment"]["BUILDDIR"],
+            fuzz_bin=fuzz_bin,
             use_valgrind=args.valgrind,
+            empty_min_time=args.empty_min_time,
         )
 
 
-def generate_corpus(*, fuzz_pool, src_dir, build_dir, corpus_dir, targets):
+def generate_corpus(*, fuzz_pool, src_dir, fuzz_bin, corpus_dir, targets):
     """Generates new corpus.
 
     Run {targets} without input, and outputs the generated corpus to
@@ -212,7 +222,7 @@ def job(command, t):
         os.makedirs(target_corpus_dir, exist_ok=True)
         use_value_profile = int(random.random() < .3)
         command = [
-            os.path.join(build_dir, 'src', 'test', 'fuzz', 'fuzz'),
+            fuzz_bin,
             "-rss_limit_mb=8000",
             "-max_total_time=6000",
             "-reload=0",
@@ -225,12 +235,12 @@ def job(command, t):
         future.result()
 
 
-def merge_inputs(*, fuzz_pool, corpus, test_list, src_dir, build_dir, merge_dir):
+def merge_inputs(*, fuzz_pool, corpus, test_list, src_dir, fuzz_bin, merge_dir):
     logging.info("Merge the inputs from the passed dir into the corpus_dir. Passed dir {}".format(merge_dir))
     jobs = []
     for t in test_list:
         args = [
-            os.path.join(build_dir, 'src', 'test', 'fuzz', 'fuzz'),
+            fuzz_bin,
             '-rss_limit_mb=8000',
             '-set_cover_merge=1',
             # set_cover_merge is used instead of -merge=1 to reduce the overall
@@ -263,16 +273,22 @@ def job(t, args):
         future.result()
 
 
-def run_once(*, fuzz_pool, corpus, test_list, src_dir, build_dir, use_valgrind):
+def run_once(*, fuzz_pool, corpus, test_list, src_dir, fuzz_bin, use_valgrind, empty_min_time):
     jobs = []
     for t in test_list:
-        corpus_path = os.path.join(corpus, t)
+        corpus_path = corpus / t
         os.makedirs(corpus_path, exist_ok=True)
         args = [
-            os.path.join(build_dir, 'src', 'test', 'fuzz', 'fuzz'),
-            '-runs=1',
-            corpus_path,
+            fuzz_bin,
         ]
+        empty_dir = not any(corpus_path.iterdir())
+        if empty_min_time and empty_dir:
+            args += [f"-max_total_time={empty_min_time}"]
+        else:
+            args += [
+                "-runs=1",
+                corpus_path,
+            ]
         if use_valgrind:
             args = ['valgrind', '--quiet', '--error-exitcode=1'] + args