Skip to content

CAT in Tapscript (BIP-347) #29247

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 3 commits into
base: master
Choose a base branch
from

Conversation

0xBEEFCAF3
Copy link

@0xBEEFCAF3 0xBEEFCAF3 commented Jan 14, 2024

CAT in Tapscript

This PR provides the necessary code to enable the opcode OP_CAT in Tapscript as specified in BIP-347: OP_CAT in Tapscript and BIN-2024-0001,

Important: This PR does not include miner activation functionality. This means that merging this PR into bitcoin-core will not make OP_CAT functional in Bitcoin.
If this PR is merged it is not a signal of community consensus around activating CAT nor be read as a portent about the activation process or timeline.
The PR is not a stand-in for consensus around such decisions.

This PR includes:

  • An implementation of the Tapscript opcode OP_CAT along with the flags SCRIPT_VERIFY_OP_CAT and SCRIPT_VERIFY_DISCOURAGE_OP_CAT.
  • Integration and unit tests which ensure that our CAT implementation works as expected. This includes ensuring that CAT never uses more that 520 bytes of stack memory. A full description of what these tests cover is given below.
  • An improvement to src/test/script_tests.cpp JSON script format enabling the rapid creation of new Tapscript unit tests.

Activation on Bitcoin-inquisition (signet)

Bitcoin-inquisition PR 37 which contained our implementation of OP_CAT was merged into bitcoin-inquisition (signet) on Apr 25 2024, released as bitcoin-inquisition release 25.2 on Apr 26 2024. and activated in bitcoin-inquisition Apr 30 2024. The bitcoin-inquisition PR was reviewed in the PR review club (transcript of discussion). Since Apr 30 2024 there have been many OP_CAT transactions created and spent on signet.

The code merged into bitcoin-inquisition differs from this PR, as we have removed the consensus logic which was used to activate it on signet.

OP_CAT Tapscript Implementation

We implement OP_CAT as a new Tapscript op code by redefining the opcode OP_SUCCESS126 (126 in decimal and 0x7e in hexadecimal). This is the same opcode value used by the original OP_CAT.

When evaluated, the OP_CAT instruction:

  1. Pops the top two values off the stack,
  2. concatenates the popped values together in stack order,
  3. and then pushes the concatenated value on the top of the stack.

OP_CAT fails if there are fewer than two values on the stack or if a concatenated value would have a combined size greater than the maximum script element size of 520 bytes.

See BIP 347 for a deeper description.

Errors thrown

If evaluated OP_CAT can throw the following errors:

  • If at time of evaluation the stack has fewer than two elements we throw the error: SCRIPT_ERR_INVALID_STACK_OPERATION
  • If at time of evaluation the top two stack elements have a combined size greater than MAX_SCRIPT_ELEMENT_SIZE (520 bytes) we throw the error: SCRIPT_ERR_PUSH_SIZE.

Script verification flags

While this PR does not contain any miner signaling and activation logic and can not activate OP_CAT, it does contain two flags which future activation logic could set to control activation of OP_CAT.

  • SCRIPT_VERIFY_OP_CAT IF a bitcoin node has this set to true, then it treat OP_CAT enabled for Tapscript. That is, OP_SUCCESS126 will be redefining to OP_CAT in Tapscript. If this was set to true at the consensus level this would cause a soft fork.

SCRIPT_VERIFY_DISCOURAGE_OP_CAT When set to true, a node receiving any Tapscript transaction containing the opcode OP_CAT or OP_SUCCESS126 will reject the transaction throwing the error SCRIPT_ERR_DISCOURAGE_OP_CAT but not banning the node which relayed the transaction. This prevents nodes from relaying transactions with OP_CAT. This is equivalent to the behavior of SCRIPT_ERR_DISCOURAGE_OP_CAT = true when SCRIPT_VERIFY_OP_CAT = false as OP_CAT is an OP_SUCCESS (OP_SUCCESS126).

This is how these two flags are intended to be used to ensure a smooth soft fork.

Stage SCRIPT_VERIFY OP_CAT SCRIPT_VERIFY DISCOURAGE_OP_CAT Status
1.Default False True This does not represent any change in behavior of the bitcoin-core node.
2.Upgrading True True OP_CAT is activating.
3.Upgraded True False It is clear that OP_CAT has been activated and the network has been upgraded.

The flag SCRIPT_ERR_DISCOURAGE_OP_CAT provides a window of time for the network to fully activate, before nodes will relay or accept transactions containing OP_CAT in their mem pools.

Tests

This PR contains a suite of script tests to ensure that OP_CAT functions as expected. In the JSON script tests (script_tests.json), we test:

  • That if OP_CAT is not activated that there are no changes to bitcoin consensus.
  • That regardless of the activation or non-activation of OP_CAT in Tapscript, pre-Tapscript scripts, i.e. Bitcoin scripts, have no changes in behavior.
  • That OP_CAT when evaluated throws the expected error when there are less than two elements on the stack.
  • That OP_CAT when evaluated in a variety of circumstances and edge cases, successfully concatenates elements of the stack. This includes:
    • multiple calls to OP_CAT in a row,
    • evaluating OP_CAT inside of a IF conditional,
    • evaluating OP_CAT on zero size stack elements, random and large stack elements,
    • evaluating OP_CAT on values being moved to and from the alt stack,
    • and checking that when evaluated OP_CAT concatenates the elements in the expected order.

All of these tests are designed to cover the happy path of OP_CAT, the various errors which OP_CAT can throw and all the corner cases between those two outcomes.

Additionally we include three tests outside of the JSON script tests.

  • cat_simple and cat_empty_stack are designed to test OP_CAT outside of the JSON serialization regime. Ensure that we catch bugs that we might miss in the JSON script tests due to a bug introduced at the JSON serialization layer.
  • cat_dup_test enumerates all stack element sizes from 1 to 522 bytes and then enumerates up to 10 repetitions of OP\_DUP OP\_CAT. It then tests if the stack element would exceed 520 bytes and if so did OP_CAT throw the error SCRIPT_ERR_PUSH_SIZE. This allows us to be certain that OP_CAT will not introduce any OP\_DUP OP\_CAT memory exhaustion attacks.

Better Tapscript tests in JSON script tests

While writing these JSON script tests (script_tests.json) we ran into the following problem. The JSON script tests are simple and easy to write for pre-Tapscript scripts, but adding or changing a Tapscript test requires substantial work per test.
Consider the following pre-tapscript test:

["'aa' 'bb'", "CAT 0x4c 0x02 0xaabb EQUAL", "P2SH,STRICTENC", "DISABLED_OPCODE", "CAT disabled"]

whereas a Tapscript test for the same script (annotated with comments for better readability) would look like:

[
    [
        "aa",
        "bb",
        "7e4c02aabb87", // output script
        "c0d6889cb081036e0faefa3a35157ad71086b123b2b144b649798b494c300a961d", // control block
        0.00000001
    ],
    "",
    "0x51 0x20 0x15048ed3a65748549c27b671936987093cf73a4c9cb18522a74fb9553060ca99", // Tapscript output
    "P2SH,WITNESS,TAPROOT",
    "OK",
    "TAPSCRIPT CATs aa and bb together and checks if EQUAL to aabb"
]

Computing the Tapscript output, such as 0x51 0x20 0x15048ed3a65748549c27b671936987093cf73a4c9cb18522a74fb9553060ca99, requires writing custom code and running it for each test. The same is true for the Tapscript control block, such as c0d6889cb081036e0faefa3a35157ad71086b123b2b144b649798b494c300a961d. If a test is changed or updated new outputs and control blocks must be computed. The complexity of doing this is likely the reason that no one has added any Tapscript tests to JSON script tests until this PR.

In this PR we address this issue by adding the following improvements to JSON script tests:

  • Adding simple macros ("#SCRIPT# and #CONTROLBLOCK#) that allow the script test parser to automatically generate and inject a valid Tapscript output and control block to be computed automatically from the JSON script.
  • Allowing Tapscript scripts to use the human readable strings like pre-script scripts by marking the location of the script in the witness stack using #SCRIPT#. This transforms the unreadable script 7e4c02aabb87 into #SCRIPT# CAT 0x4c 0x02 0xaabb EQUAL.

This results in the following JSON script test which is far easier to write and easier to read.

[
    [
        "aa",
        "bb",
        "#SCRIPT# CAT",
        "#CONTROLBLOCK#",
        0.00000001
    ],
    "",
    "0x51 0x20 #TAPROOTOUTPUT#",
    "P2SH,WITNESS,TAPROOT,OP_CAT",
    "OK",
    "TAPSCRIPT Test of OP_CAT flag by calling CAT on two elements. TAPSCRIPT_OP_CAT flag is set so CAT is executed."
],

@DrahtBot
Copy link
Contributor

DrahtBot commented Jan 14, 2024

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Code Coverage & Benchmarks

For details see: https://corecheck.dev/bitcoin/bitcoin/pulls/29247.

Reviews

See the guideline for information on the review process.
A summary of reviews will appear here.

Conflicts

Reviewers, this pull request conflicts with the following ones:

  • #32453 ([Policy] Discourage Unsigned Annexes by JeremyRubin)
  • #32247 (BIP-348 (OP_CHECKSIGFROMSTACK) (regtest only) by jamesob)
  • #31989 (BIP-119 (OP_CHECKTEMPLATEVERIFY) (regtest only) by jamesob)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

LLM Linter (✨ experimental)

Possible typos and grammar issues:

  • rest -> test [typo]

@DrahtBot DrahtBot changed the title Reenable OP_CAT Reenable OP_CAT Jan 14, 2024
@DrahtBot
Copy link
Contributor

🚧 At least one of the CI tasks failed. Make sure to run all tests locally, according to the
documentation.

Possibly this is due to a silent merge conflict (the changes in this pull request being
incompatible with the current code in the target branch). If so, make sure to rebase on the latest
commit of the target branch.

Leave a comment here, if you need help tracking down a confusing failure.

Debug: https://github.com/bitcoin/bitcoin/runs/20473822722

@0xBEEFCAF3
Copy link
Author

Drafting until I get the inquisition PR approved and I can get the builds passing.

@moonsettler
Copy link

Some of us have been playing with the idea of neutered CAT: instead of the MAX_SCRIPT_ELEMENT_SIZE (520 bytes) the maximum output size would be 80 bytes.

Rationale:
It explicitly disables open ended second order effects by removing the ability to assemble a SIGHASH or CTV template on the stack for more detailed introspection than intended. If such introspection behavior is desired in the future it can be explicitly enabled by specialized and more efficient opcodes.

Combined with CSFS it can be used as a signed datacarrier, for which the 'standard' limit is 80 bytes, it also has to be smaller than 84 bytes which is required for building CTV templates on the stack, and larger than 64/65/72 bytes which are respectively needed for:

  • Merkle inclusion: 2x 20/32 byte hashes
  • LN-symmetry (LNhance): 2x 32 byte hashes
  • Separate sig: 64/72 bytes (0-conf bonds, staking contracts)

Could be a livable compromise between the conservatives that want to preserve certain characteristics of bitcoin and the prometheans who want to give the developers more practical and useful tools to build with.

@bigspider
Copy link

Rationale: It explicitly disables open ended second order effects by removing the ability to assemble a SIGHASH or CTV template on the stack for more detailed introspection than intended. If such introspection behavior is desired in the future it can be explicitly enabled by specialized and more efficient opcodes.

Script is already expressive enough to (awkwardly, expensively) compute arbitrary SHA256 hashes on the stack, except that the result would be broken in smaller pieces of at most 4 bytes. With CAT, those can be concatenated to get a single 32-byte result.

Therefore, neutering CATs does not achieve the desired result of preventing the CHECKSIG tricks, unless you limit the length of the result to less than 32 bytes - which would also neuter most of the utility of the opcode.

@0xBEEFCAF3 0xBEEFCAF3 force-pushed the arm/re-enable-opcat branch 4 times, most recently from b9c67da to f1fd2b6 Compare May 27, 2024 18:07
@0xBEEFCAF3 0xBEEFCAF3 marked this pull request as ready for review May 27, 2024 19:28
@0xBEEFCAF3 0xBEEFCAF3 force-pushed the arm/re-enable-opcat branch from f1fd2b6 to 6b259bd Compare August 27, 2024 12:43
@DrahtBot
Copy link
Contributor

🚧 At least one of the CI tasks failed.
Debug: https://github.com/bitcoin/bitcoin/runs/29308305455

Hints

Make sure to run all tests locally, according to the documentation.

The failure may happen due to a number of reasons, for example:

  • Possibly due to a silent merge conflict (the changes in this pull request being
    incompatible with the current code in the target branch). If so, make sure to rebase on the latest
    commit of the target branch.

  • A sanitizer issue, which can only be found by compiling with the sanitizer and running the
    affected test.

  • An intermittent issue.

Leave a comment here, if you need help tracking down a confusing failure.

@DrahtBot DrahtBot mentioned this pull request Aug 28, 2024
@0xBEEFCAF3 0xBEEFCAF3 force-pushed the arm/re-enable-opcat branch from 6b259bd to a04c22b Compare August 29, 2024 01:50
@0xBEEFCAF3 0xBEEFCAF3 changed the title Reenable OP_CAT CAT in Tapscript (BIP-347) Aug 29, 2024
glozow added a commit that referenced this pull request Apr 21, 2025
e3d7533 test: improves tapscript unit tests (Ethan Heilman)
3e16708 test: Ensures test fails if witness is not hex (Ethan Heilman)

Pull request description:

  This commit creates new test utilities for future Taproot script tests within script_tests.json. The key features of this commit are the addition of three new tags: `#SCRIPT#`, `#CONTROLBLOCK#`, and `#TAPROOTOUTPUT#`. These tags streamline the test creation process by eliminating the need to manually generate these components outside the test suite.

  * `#SCRIPT#`: Parses Tapscript and outputs a byte string of opcodes.
  * `#CONTROLBLOCK#`: Automatically generates the control block for a given Taproot output.
  * `#TAPROOTOUTPUT#`: Generates the final Taproot scriptPubKey.

  This code was originally part of the OP_CAT PR #29247 but was pulled out into a separate PR to reduce the rebase treadmill for the OP_CAT PR.

  Additionally this PR adds a check to ensure that if the witness data can not be parsed as hex the test fails. Prior to this PR, the test code would fail silently and set the values it couldn't parse as empty stack elements. This fix was suggested by @instagibbs.

  ## Rationale

  While writing JSON script tests (script_tests.json) for #29247 we ran into the following problem. The JSON script tests are simple and easy to write for pre-Tapscript scripts, but adding or changing a Tapscript test requires substantial work per test. Consider the following pre-tapscript test:

  ```
  ["'aa' 'bb'", "CAT 0x4c 0x02 0xaabb EQUAL", "P2SH,STRICTENC", "DISABLED_OPCODE", "CAT disabled"]
  ````

  whereas a Tapscript test for the same script (annotated with comments for better readability) would look like:

  ```
  [
      [
          "aa",
          "bb",
          "7e4c02aabb87", // output script
          "c0d6889cb081036e0faefa3a35157ad71086b123b2b144b649798b494c300a961d", // control block
          0.00000001
      ],
      "",
      "0x51 0x20 0x15048ed3a65748549c27b671936987093cf73a4c9cb18522a74fb9553060ca99", // Tapscript output
      "P2SH,WITNESS,TAPROOT",
      "OK",
      "TAPSCRIPT CATs aa and bb together and checks if EQUAL to aabb"
  ]
  ```

  Computing the Tapscript output, such as `0x51 0x20 0x15048ed3a65748549c27b671936987093cf73a4c9cb18522a74fb9553060ca99`, requires writing custom code and running it for each test. The same is true for the Tapscript control block, such as `c0d6889cb081036e0faefa3a35157ad71086b123b2b144b649798b494c300a961d`. If a test is changed or updated new outputs and control blocks must be computed. The complexity of doing this is likely the reason that no one has added any Tapscript tests to JSON script tests until this PR.

  In this PR we address this issue by adding the following improvements to JSON script tests:

  Adding simple macros ("#SCRIPT# and #CONTROLBLOCK#) that allow the script test parser to automatically generate and inject a valid Tapscript output and control block to be computed automatically from the JSON script.
  Allowing Tapscript scripts to use the human readable strings like pre-script scripts by marking the location of the script in the witness stack using #SCRIPT#. This transforms the unreadable script 7e4c02aabb87 into #SCRIPT# CAT 0x4c 0x02 0xaabb EQUAL.
  This results in the following JSON script test which is far easier to write and easier to read.

  ```
  [
      [
          "aa",
          "bb",
          "#SCRIPT# CAT",
          "#CONTROLBLOCK#",
          0.00000001
      ],
      "",
      "0x51 0x20 #TAPROOTOUTPUT#",
      "P2SH,WITNESS,TAPROOT,OP_CAT",
      "OK",
      "TAPSCRIPT Test of OP_CAT flag by calling CAT on two elements. TAPSCRIPT_OP_CAT flag is set so CAT is executed."
  ],
  ```

ACKs for top commit:
  instagibbs:
    reACK e3d7533
  sipa:
    utACK e3d7533
  janb84:
    Re ACK [e3d7533](e3d7533)

Tree-SHA512: 948c3ec28a4b2b222c2d77e48918ed19d298b51d64662fc20959073edd9978fc796516a392da9755a7e173f556e3021816dc6ce8eb3ed16bbe0fa6ebc574fd48
@AlexSQY
Copy link

AlexSQY commented May 7, 2025

What about SCRIPT_VERIFY OP_CAT = False and SCRIPT_VERIFY DISCOURAGE_OP_CAT = False?
This combination is not described in the "soft fork" table. Is it part of the tests?

@EthanHeilman
Copy link
Contributor

EthanHeilman commented May 7, 2025

@AlexSQY That's an interesting question.
Current Bitcoin behavior is OP_CAT = False and DISCOURAGE_OP_CAT = False. I don't believe we would ever want to set DISCOURAGE_OP_CAT = False while OP_CAT = False since a disabled opcode should be discouraged.

@AlexSQY
Copy link

AlexSQY commented May 8, 2025

@EthanHeilman , thanks for the feedback.
I understand this combination should be discouraged.
My concern, perhaps not justified, was about this negative case not being documented/tested, thus wondering about a potential edge case.
Found out those tapscript test cases but I do not know if "not set" is exactly same as "set to false" (all those variables set to false by default?):
https://github.com/bitcoin/bitcoin/pull/29247/files#diff-ef0159635ee0fee8864cd9570a04337cabb63e9c49a9e96d7b746194ecaa427d

PS: DISCOURAGE_OP_SUCCESS also introduces more combinations one with the others (2 tested as part of the actual script_tests.json, nominal cases only?)

@EthanHeilman
Copy link
Contributor

@AlexSQY I see your point. Let me see if there is anything that can be done to ensure it is never the case that OP_CAT = False && DISCOURAGE_OP_CAT = False since that should never happen outside of tests.

The behavior is what you see in that test is the expected behavior for DISCOURAGE_OP_CAT = False and OP_CAT = False,
since OP_CAT = False the CAT opcode is an OP_SUCCESS so it returns success (OK).

@0xBEEFCAF3 What do you think?

@0xBEEFCAF3
Copy link
Author

Let me see if there is anything that can be done to ensure it is never the case that OP_CAT = False && DISCOURAGE_OP_CAT = False since that should never happen outside of tests.

By default, the SCRIPT_VERIFY_DISCOURAGE_OP_CAT bit is set, and we do not expose this as a configurable mempool policy. The only practical way for node operators to unset this bit (SCRIPT_VERIFY_DISCOURAGE_OP_CAT = False) is by compiling a custom binary.

That said, @AlexSQY, you're right -- this edge case deserves documentation. If I recall correctly, it’s being tested here, where neither the discourage bit nor the script verify bit is set. In that case, both are false, and 0x7e just behaves as OP_SUCCESS.

0xBEEFCAF3 and others added 3 commits May 8, 2025 23:01
Implement OP_CAT as a new Tapscript op code by redefining the opcode OP_SUCCESS126 (126 in decimal and 0x7e in hexadecimal).
This is the same opcode value used by the original OP_CAT.

When evaluated, the OP_CAT instruction:

Pops the top two values off the stack,
concatenates the popped values together in stack order,
and then pushes the concatenated value on the top of the stack.
OP_CAT fails if there are fewer than two values on the stack or if a concatenated value would have a combined size greater than the maximum script element size of 520 bytes.

See [BIP
347](https://github.com/bitcoin/bips/blob/master/bip-0347.mediawiki) for a deeper description.
Co-authored-by: Ethan Heilman <ethan.r.heilman@gmail.com>
The goal of this functional test is to ensure OP_CAT spends are still
disabled by default in segwitv0 and legacy spends. Spending such inputs
should result in `mandatory-script-verify-flag-failed (Attempted to use a disabled opcode)`.
While spending OP_CAT inputs in tapscript should be discouraged under
the default `STANDARD_SCRIPT_VERIFY_FLAGS`.
@ajtowns
Copy link
Contributor

ajtowns commented May 12, 2025

Current Bitcoin behavior is OP_CAT = False and DISCOURAGE_OP_CAT = False.

DISCOURAGE_OP_SUCCESS in current bitcoin covers the behaviour of DISCOURAGE_OP_CAT.

OP_CAT=false and DISCOURAGE_OP_CAT=false are used for validating blocks prior to activation of OP_CAT, so this isn't an edge case as far as I can see? OP_CAT=true and DISCOURAGE_OP_CAT=true should be an edge case, but it doesn't make sense of course: "disallow OP_CAT, but if it were allowed, which it's not, enforce its rules".

I think the table should look more like:

Phase Mode SV_OP_CAT SV_DISCOURAGE_OP_CAT
Pre-activation Transaction relay ❌ (false) ✔️ (true)
Pre-activation Block validation ❌ (false) ❌ (false)
Post-activation Transaction relay ✔️ (true) ❌ (false)
Post-activation Block validation ✔️ (true) ❌ (false)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

10 participants