Skip to content

Conversation

@UdjinM6
Copy link

@UdjinM6 UdjinM6 commented Aug 23, 2024

Issue being fixed or feature implemented

Txout-s in creditOutputs for AssetLock txes should be shown the way txout-s are shown in other places. We should not be using debug strings there.

Example: getrawtransaction 50757f651f335e22c5a810bd05c1e5aac0d95b132f6454e2a72683f88e3983f3 1

develop:

  "assetLockTx": {
    "version": 1,
    "creditOutputs": [
      "CTxOut(nValue=0.01000000, scriptPubKey=76a914cdfca4ae1cf2333056659a2c)"
    ]
  },

This PR:

  "assetLockTx": {
    "version": 1,
    "creditOutputs": [
      {
        "value": 0.01000000,
        "valueSat": 1000000,
        "scriptPubKey": {
          "asm": "OP_DUP OP_HASH160 cdfca4ae1cf2333056659a2c8dc656f36d228402 OP_EQUALVERIFY OP_CHECKSIG",
          "hex": "76a914cdfca4ae1cf2333056659a2c8dc656f36d22840288ac",
          "address": "yf6c2VSpWGXUgmjQSHRpfEcTPsbqN4oL4c",
          "type": "pubkeyhash"
        }
      }
    ]
  },

kudos to @coolaj86 for finding the issue

What was done?

Change CAssetLockPayload::ToJson() output to be closer to TxToUniv()

NOTE: refactor: use key_to_p2pkh_script in more places commit is a bit unrelated but I decided to add it anyway to make it easier to follow assetlock creation vs getrawtransaction rpc check.

How Has This Been Tested?

Try example above, run tests

Breaking Changes

RPC output is different for AssetLock txes

Checklist:

  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have added or updated relevant unit/integration/functional/e2e tests
  • I have made corresponding changes to the documentation
  • I have assigned this pull request to a milestone

@UdjinM6 UdjinM6 added RPC Some notable changes to RPC params/behaviour/descriptions backport-candidate-21.1.x labels Aug 23, 2024
@UdjinM6 UdjinM6 force-pushed the fix_assetlock_outputs_json branch 2 times, most recently from ae9bb25 to ba78bd2 Compare August 23, 2024 21:08
@coolaj86
Copy link

Thank you @UdjinM6!

@UdjinM6 UdjinM6 added this to the 21.2 milestone Aug 23, 2024
@UdjinM6 UdjinM6 force-pushed the fix_assetlock_outputs_json branch from ba78bd2 to e54fe42 Compare August 23, 2024 22:04
@PastaPastaPasta
Copy link
Member

This is a breaking change, also not a fix really but a feat I guess? As such, can't merge in pre-v22, cannot backport

we could merge in e54fe42 earlier ofc

@UdjinM6
Copy link
Author

UdjinM6 commented Aug 25, 2024

In no other places (in RPC) txouts are shown like that. The data is incomplete so you could get the same scriptPubKey string for two different actual scriptPubKey-s. If you use that for some logic in your app things will get messy one day. Basically, the way it works right now makes RPC output unusable/dangerous which is clearly a bug that must be fixed asap imo.

#include <primitives/transaction.h>
#include <gsl/pointers.h>

#include <core_io.h>
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could you add here a forward declaration of ScriptPubKeyToUniv or do something else?

To avoid:

"core_io -> evo/assetlocktx -> core_io"

@PastaPastaPasta
Copy link
Member

In no other places (in RPC) txouts are shown like that. The data is incomplete so you could get the same scriptPubKey string for two different actual scriptPubKey-s. If you use that for some logic in your app things will get messy one day. Basically, the way it works right now makes RPC output unusable/dangerous which is clearly a bug that must be fixed asap imo.

Is it truly inherently broken? I see no reason why platform for example (or anyone else) couldn't be using / relying on it as it right now.

Can you explain further how/why it is incomplete / inherently broken?

@UdjinM6
Copy link
Author

UdjinM6 commented Aug 27, 2024

Can you explain further how/why it is incomplete / inherently broken?

pls see PR description, tl;dr:
scriptPubKey shown in RPC results in develop:
76a914cdfca4ae1cf2333056659a2c
actual scriptPubKey:
76a914cdfca4ae1cf2333056659a2c8dc656f36d22840288ac

knst
knst previously approved these changes Aug 27, 2024
Copy link
Collaborator

@knst knst left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

utACK b330318

@PastaPastaPasta
Copy link
Member

PastaPastaPasta commented Sep 4, 2024

But it's still a breaking change @UdjinM6, platform could be parsing the amount component CTxOut(nValue=0.01000000 and using that somehow. We could ask @shumkov to review and if they sign off on this not being a breaking change for them, then I guess we could merge it in (still concerns me though because still stuff like platform-explorer, mnowatch, misc. explorer could be using it), but generally, we need to be very careful about merging stuff that could in theory break platform.

@shumkov
Copy link

shumkov commented Sep 4, 2024

It doesn't look like a breaking change for Drive, since it uses only height and hex.

I herby temporarily release you from your obligation to block all PRs which result in breaking changes in minor versions. This temporary release shall only be for the PR 6229. I herby solemnly swear to release you from all potential blame arising from any issues related to platform as a result of merging this PR.

@PastaPastaPasta
Copy link
Member

@UdjinM6 I think with release notes documenting this change, I can merge it for 21.2

@UdjinM6 UdjinM6 force-pushed the fix_assetlock_outputs_json branch from 90e6410 to 9876c2d Compare September 4, 2024 19:33
Copy link
Member

@PastaPastaPasta PastaPastaPasta left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

utACK 9876c2d

@PastaPastaPasta PastaPastaPasta merged commit b38c4d3 into dashpay:develop Sep 5, 2024
@thephez
Copy link
Collaborator

thephez commented Sep 5, 2024

Good thing these messages have an edit log @shumkov 😂

It doesn't look like a breaking change for Drive, since it uses only height and hex.

I herby temporarily release you from your obligation to block all PRs which result in breaking changes in minor versions. This temporary release shall only be for the PR 6229. I herby solemnly swear to release you from all potential blame arising from any issues related to platform as a result of merging this PR.

image

PastaPastaPasta added a commit to PastaPastaPasta/dash that referenced this pull request Oct 22, 2024
…should be an array of objects, not debug strings

9876c2d docs: add partial release notes (UdjinM6)
b330318 refactor: drop circular dependency (UdjinM6)
e54fe42 refactor: use `key_to_p2pkh_script` in more places (UdjinM6)
3ed6246 test: check `creditOutputs` format (UdjinM6)
ba0e645 fix: `creditOutputs` in AssetLock tx json output should be an array of objects, not debug strings (UdjinM6)

Pull request description:

  ## Issue being fixed or feature implemented
  Txout-s in `creditOutputs` for AssetLock txes should be shown the way txout-s are shown in other places. We should not be using debug strings there.

  Example: `getrawtransaction 50757f651f335e22c5a810bd05c1e5aac0d95b132f6454e2a72683f88e3983f3 1`

  develop:
  ```
    "assetLockTx": {
      "version": 1,
      "creditOutputs": [
        "CTxOut(nValue=0.01000000, scriptPubKey=76a914cdfca4ae1cf2333056659a2c)"
      ]
    },
  ```
  This PR:
  ```
    "assetLockTx": {
      "version": 1,
      "creditOutputs": [
        {
          "value": 0.01000000,
          "valueSat": 1000000,
          "scriptPubKey": {
            "asm": "OP_DUP OP_HASH160 cdfca4ae1cf2333056659a2c8dc656f36d228402 OP_EQUALVERIFY OP_CHECKSIG",
            "hex": "76a914cdfca4ae1cf2333056659a2c8dc656f36d22840288ac",
            "address": "yf6c2VSpWGXUgmjQSHRpfEcTPsbqN4oL4c",
            "type": "pubkeyhash"
          }
        }
      ]
    },
  ```
  kudos to @coolaj86 for finding the issue

  ## What was done?
  Change `CAssetLockPayload::ToJson()` output to be closer to [`TxToUniv()`](https://github.com/dashpay/dash/blob/develop/src/core_write.cpp#L262-L272)

  NOTE: `refactor: use key_to_p2pkh_script in more places` commit is a bit unrelated but I decided to add it anyway to make it easier to follow assetlock creation vs getrawtransaction rpc check.

  ## How Has This Been Tested?
  Try example above, run tests

  ## Breaking Changes
  RPC output is different for AssetLock txes

  ## Checklist:
  - [x] I have performed a self-review of my own code
  - [ ] I have commented my code, particularly in hard-to-understand areas
  - [ ] I have added or updated relevant unit/integration/functional/e2e tests
  - [ ] I have made corresponding changes to the documentation
  - [ ] I have assigned this pull request to a milestone

ACKs for top commit:
  PastaPastaPasta:
    utACK 9876c2d

Tree-SHA512: 158c98ac9e4979bb29c4f54cb1b71806f22aaec92218d92cd2b2e9b9f74df721563e7a6c5f517ea358ac74659fa79f51d1b683002a1cdceb1b8ee80f8fd79375
@UdjinM6 UdjinM6 modified the milestones: 21.2, 21.1.1 Oct 22, 2024
PastaPastaPasta added a commit that referenced this pull request Oct 22, 2024
d627a6e chore: bump version to 21.1.1 (pasta)
5f9700c docs: release notes for v21.1.1 (pasta)
1c00726 Merge #6277: chore: add builder key for kittywhiskers (pasta)
a2bc0f1 Merge #6290: chore: update pasta gpg key to reflect new subkeys (pasta)
167608c Merge #6338: ci: attest results of guix builds (pasta)
6fb4e49 Merge #6197: ci: always build guix, save artifacts (pasta)
c0ca93c Merge #6340: fix: make 6336 compile in v21.1.x branch, using older CHECK_NONFATAL functionality (pasta)
bb96df4 Merge #6336: fix: rpc getblock and getblockstats for blocks with withdrawal transactions (asset unlock) (pasta)
8e70262 Merge #6131: feat: make a support of Qt app to show Platform transfer Tx (pasta)
80ed279 Merge #6328: backport: bitcoin#30131, bitcoin#23258, bitcoin#30504 - fix bild for Ubuntu 24.10 + clang (pasta)
bd772fb Merge #6229: fix: `creditOutputs` in AssetLock tx json output should be an array of objects, not debug strings (pasta)
9bf39a9 Merge #6222: fix: adjust payee predictions after mn_rr activation, add tests (pasta)
87bebfc Merge #6219: fix: correct is_snapshot_cs in VerifyDB (pasta)
a4e6b8a Merge #6208: fix: persist coinjoin denoms options from gui over restarts (pasta)

Pull request description:

  ## Issue being fixed or feature implemented
  See commits, backports, release notes, version bump

  ## What was done?

  ## How Has This Been Tested?

  ## Breaking Changes

  ## Checklist:
    _Go over all the following points, and put an `x` in all the boxes that apply._
  - [ ] I have performed a self-review of my own code
  - [ ] I have commented my code, particularly in hard-to-understand areas
  - [ ] I have added or updated relevant unit/integration/functional/e2e tests
  - [ ] I have made corresponding changes to the documentation
  - [ ] I have assigned this pull request to a milestone _(for repository code-owners and collaborators only)_

ACKs for top commit:
  knst:
    utACK d627a6e
  kwvg:
    ACK d627a6e
  UdjinM6:
    utACK d627a6e
  ogabrielides:
    utACK d627a6e

Tree-SHA512: cde7e40760e16e9f48da8149c3742d18a34029b057405e4d55b87110da96acbcd19b47280451dd7b5ad1ccfc91fde655452cf5f0f0d1e01a41b4c685337c64b8
PastaPastaPasta added a commit that referenced this pull request Apr 27, 2025
…ray-bounds` and `-Wdangling-reference`, re-enable `-Werror` for GCC

40caa7d revert: don't build using `-Werror` when using GCC (Kittywhiskers Van Gogh)
96e494c build: constrain `-Wstringop-over{read,flow}` suppression to GCC only (Kittywhiskers Van Gogh)
6bff03b build: don't error on `-Wattributes` with GCC <14 (Kittywhiskers Van Gogh)
9aab60b build: don't error on `-Warray-bounds` and `-Wdangling-reference` with GCC (Kittywhiskers Van Gogh)
18fcb26 fix: resolve `-Wunused-function` warning (Kittywhiskers Van Gogh)
7a65315 fix: resolve `-Wmaybe-uninitialized` warnings (Kittywhiskers Van Gogh)
b208018 fix: resolve `-Wignored-qualifiers` warning (Kittywhiskers Van Gogh)
7dd0daa fix: resolve `-Woverloaded-virtual` warning (Kittywhiskers Van Gogh)
c587838 refactor: use `GetTransactionBlock()` as narrow `GetTransaction()` proxy (Kittywhiskers Van Gogh)
d9828a9 refactor: move `ToJson` definitions relied by `TxToUniv` to source file (Kittywhiskers Van Gogh)

Pull request description:

  ## Additional Information

  * Depends on #6637

  * Depends on #6638

  * `TxToUniv()` is a function defined in `core_write.cpp`, which is a part of `libbitcoin_common` that relies on `ToJson()` definitions in various classes (`CCbTx`, `CProRegTx` and others) to print out of contents of transaction payload. The problem is, those various classes have their source files a part of `libbitcoin_server`, so when binaries like `dash-tx` (which do not use `libbitcoin_server`) need to be linked, the necessary `ToJson()` implementations, were they defined in source files, wouldn't be found and this would result in a linking error.

    This is worked around by defining them in the header instead but this creates a new problem where these headers are included by `core_write` but constructing the UniValue objects returned by `ToJson()` needs something from `core_write` (i.e. there is a circular dependency).

    This was worked around in [dash#6229](#6229) ([commit](b330318)) with redefinition, which was fine but is now verboten with `-Wredundant-decls` enforcement.

    * To work around this, all `ToJson()` definitions from those headers relied on by `TxToUniv()` have been moved to their own source file (`evo/core_write.cpp`) and included in `libbitcoin_common`. This takes care of the linking issue and avoids circular dependencies.

  * `GetTransaction()` is extensively used in Dash-specific code and a redefinition in `validation.cpp` is used to reduce the amount of reported circular dependencies, removing that circular dependency in order to satisfy `-Wredundant-decls` results in the following (see below).

    <details>

    <summary>Circular dependencies:</summary>

    Removed:

    ```
    "evo/deterministicmns -> validation -> txmempool -> evo/deterministicmns"
    ```

    Added:

    ```
    "node/transaction -> validation -> node/transaction"
    "evo/deterministicmns -> node/transaction -> net_processing -> evo/deterministicmns"
    "evo/deterministicmns -> node/transaction -> node/blockstorage -> evo/deterministicmns"
    "evo/deterministicmns -> node/transaction -> node/context -> evo/deterministicmns"
    "evo/deterministicmns -> node/transaction -> txmempool -> evo/deterministicmns"
    "evo/chainhelper -> llmq/chainlocks -> node/transaction -> node/context -> evo/chainhelper"
    "evo/deterministicmns -> node/transaction -> net_processing -> evo/mnauth -> evo/deterministicmns"
    "evo/deterministicmns -> node/transaction -> node/context -> governance/governance -> evo/deterministicmns"
    "evo/deterministicmns -> node/transaction -> net_processing -> llmq/dkgsession -> evo/deterministicmns"
    "evo/deterministicmns -> node/transaction -> net_processing -> llmq/dkgsessionmgr -> evo/deterministicmns"
    "evo/deterministicmns -> node/transaction -> net_processing -> llmq/quorums -> evo/deterministicmns"
    "evo/deterministicmns -> node/transaction -> node/blockstorage -> masternode/node -> evo/deterministicmns"
    "governance/governance -> governance/object -> node/transaction -> node/context -> governance/governance"
    "llmq/chainlocks -> node/transaction -> node/context -> llmq/context -> llmq/chainlocks"
    "llmq/context -> llmq/instantsend -> node/transaction -> node/context -> llmq/context"
    "coinjoin/context -> coinjoin/server -> evo/deterministicmns -> node/transaction -> node/context -> coinjoin/context"
    "evo/cbtx -> evo/deterministicmns -> node/transaction -> node/context -> evo/creditpool -> evo/cbtx"
    "evo/deterministicmns -> node/transaction -> net_processing -> llmq/dkgsession -> llmq/debug -> evo/deterministicmns"
    "evo/deterministicmns -> node/transaction -> node/context -> evo/mnhftx -> llmq/signing -> llmq/signing_shares -> evo/deterministicmns"
    ```

    Diff: **+18**

    </details>

    To avoid this, alongside some header adjustments, a proxy to `GetTransaction()` has been introduced in `evo/chainhelper.cpp`, `GetTransactionBlock()`. This _reduces_ the number of circular dependencies (see below).

    <details>

    <summary>Circular dependencies:</summary>

    Removed:

    ```
    "consensus/tx_verify -> evo/assetlocktx -> validation -> consensus/tx_verify"
    "consensus/tx_verify -> evo/assetlocktx -> validation -> txmempool -> consensus/tx_verify"
    "evo/assetlocktx -> validation -> txmempool -> evo/assetlocktx"
    "evo/deterministicmns -> validation -> evo/deterministicmns"
    "evo/deterministicmns -> validation -> txmempool -> evo/deterministicmns"
    ```

    Added:

    ```
    "evo/chainhelper -> llmq/chainlocks -> evo/chainhelper"
    "evo/chainhelper -> llmq/instantsend -> evo/chainhelper"
    "evo/chainhelper -> evo/specialtxman -> evo/deterministicmns -> evo/chainhelper"
    "evo/chainhelper -> node/transaction -> node/context -> evo/chainhelper"
    "evo/deterministicmns -> llmq/commitment -> validation -> evo/deterministicmns"
    "consensus/tx_verify -> evo/assetlocktx -> llmq/commitment -> validation -> consensus/tx_verify"
    "evo/assetlocktx -> llmq/signing -> net_processing -> txmempool -> evo/assetlocktx"
    "evo/chainhelper -> masternode/payments -> governance/classes -> governance/object -> evo/chainhelper"
    "evo/deterministicmns -> llmq/commitment -> validation -> txmempool -> evo/deterministicmns"
    "consensus/tx_verify -> evo/assetlocktx -> llmq/signing -> net_processing -> txmempool -> consensus/tx_verify"
    ```

    Diff: **+5**

    </details>

    To differentiate it from `GetTransaction()`, since all Dash-specific invocations either rely on the transaction index or the mempool, we can safely trim down the number of arguments and add a fast-fail.

  * Even with `-Werror` enabled, we have to downgrade `-Warray-bounds` to warnings (i.e. use `-Wno-error=array-bounds`) as there are two sources of the warning, `immer` (see [arximboldi/immer#223](arximboldi/immer#223)) and serialization code (fixed with [bitcoin#30765](bitcoin#30765)). As due to the former, we need to suppress it anyways, the latter has not been backported in the interest of brevity.

  * Similar to the above, we have to downgrade `-Wdangling-reference` as it is caused by UniValue-adjacent code that is fixed by [bitcoin#27605](bitcoin#27605) but would require an out-of-order backport that pulls in multiple dependent PRs in order to be complete (or alternatively, backport it as-is and then make future backporting annoying). The simpler solution to downgrade the warning was opted for instead.

  * We need to annotate `CHDPubKey::nVersion` with `[[maybe_unused]]` to avoid upsetting Clang due to `-Wunused-private-field`. GCC 11 doesn't emit a warning for that variable and therefore, finds the annotation unnecessary, triggering an error due to `-Wattributes` ([build](https://gitlab.com/dashpay/dash/-/jobs/9777475240#L1581)).

    Since both Clang 18 and GCC 14 agree with the `-Wunused-private-field` assessment, we are better off downgrading `-Wattributes` to a warning for GCC 11.

  ## Breaking Changes

  None expected.

  ## Checklist

  - [x] I have performed a self-review of my own code
  - [x] I have commented my code, particularly in hard-to-understand areas **(note: N/A)**
  - [x] I have added or updated relevant unit/integration/functional/e2e tests **(note: N/A)**
  - [x] I have made corresponding changes to the documentation **(note: N/A)**
  - [x] I have assigned this pull request to a milestone _(for repository code-owners and collaborators only)_

ACKs for top commit:
  knst:
    ACK 40caa7d
  UdjinM6:
    utACK 40caa7d

Tree-SHA512: 4fb383f4e2142db24ffedffd023cc075804c07e499abfcfc5708047d64fd981599cdb7479d92f501bedede6d94b76cc16f77f1372dd5bce7e1906ddb374679c3
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

RPC Some notable changes to RPC params/behaviour/descriptions

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants