Add predicate support

[ra0x3]

• In order to complete Add example of indexing Script log #881 we need to create a standard for including Witness data in a Transaction - where this Witness data can be filtered on a Transaction level downstream

[ra0x3]

@lostman

• I see grantees are still asking if we can index predicates How to index transaction of predicate execution? #1155
• Did you ever do anything here?

[ra0x3]

@Voxelot

• This keeps getting lost in the shuffle 😅 trying to get it going again
• Continuing the conversation from How to index transaction of predicate execution? #1155
• I think the only thing blocking implementation of @dmihal's comment here is the definition of "watches the predicate address"
  • Not sure if this would be an actual Address in a Receipt, or if like we previously discussed, this "predicate address" would actually be some type of standardized byte-array in the Witness data of a transaction

[Voxelot]

In this case, it would be filtering predicate inputs by the owner address.

Here's the overall flow that I'm picturing, given that we want to have handlers for specific witnesses and coin owners:

1. user creates a script tx to open a position on spark, which is represented as a coin output. This tx also includes witness data to signal to the indexer that this output needs to be watched.
2. As part of the standard we may want to include the relevant tx output indexes in the witness data of the predicates that need to be watched, as well as a commitment (i.e. hash) of the predicate template used, and also any compile time constants used in the predicate construction.
3. In the witness data handler, it will subscribe/filter based on the hash of the predicate template to know which tx outputs are of interest to the application, and then dynamically register a listener for each coin owner referred to by the witness data. Some extra validation would need to be done here to avoid accidentally indexing fraudulently constructed witnesses trying to spoof the spark indexer. Will expand on this in more detail later.
4. When the output coin is spent in the input of a later transaction (i.e. a fill order), then the indexer would automatically pick it up by filtering on the owner field and call the appropriate handler.

Witness validation procedure:

1. The indexer module would need to be configured with a set of known trusted predicate templates. By predicate templates, I'm referring to predicate bytecode that hasn't had its configurable constants swapped in yet. This configuration would also need to know the schema of configurable constants based on the predicate abi in order to help out with parsing this witness data and some codegen for swapping in the configurable constants.
2. To validate the witness, extract the predicate template hash from the witness data to lookup the appropriate predicate template configured in step 1. If the hash doesn't match a known predicate template then it should be rejected.
3. Fetch all the configurable constants from the witness data and swap them into the predicate template: https://docs.fuel.network/docs/fuels-rs/predicates/#configurable-constants
4. Convert the resultant predicate bytecode into an owner/address, ie. https://github.com/FuelLabs/fuel-vm/blob/master/fuel-tx/src/transaction/types/input.rs#L794
5. Then compare the owner computed by the indexer against the owner on the transaction, if they the match then it is safe to index.

Here's an example of what I'm thinking this standardized witness format could be to enable validation:

Field	Type	Description
PredicateTemplateHash	Bytes32	The hash of the base predicate template
OutputIndex	u8	The output index corresponding to the predicate that was created on this tx
ConfigurationSchema	Any	The schema of configurable constants that this predicate uses, autogenerated from the abi

[Voxelot]

Here's a more concrete description of how this could work, broken down step by step. There's definitely some lower-level details I'm missing but this should give a rough sketch of the general usage & flow.

User flow: Opening and fulfilling a market order

Setup:

1. Define a predicate for orders containing the following configurable constants:
   1. The asset pair
   2. The price
   3. The order side
   4. The address that can either cancel the order or recieve the fill amount
2. Compile the predicate using forc to get a predicate template
3. Register predicate template in the indexer manifest with the following details:
   1. The predicate template bytecode
      1. This is hashed for quick lookups
   2. The predicate ABI (which includes configurable constants)

User A opens a position to buy eth:

1. user substitutes order parameters into the order predicate template using configurable constants
   1. pair: ETH-USDC
   2. price: $2,000
   3. side: buy
   4. address: 0xF1A2B3...
2. user submits a script transaction to place market order with the following:
   1. A $2,000 USDC coin input for buying ETH
   2. A $2,000 USDC coin output, with the owner set as the predicate root of the configured predicate from the previous step.
   3. A witness containing metadata about the order so that others may find it, containing
      1. The index of the predicate coin output on the transaction
      2. The hash of the predicate template that was used (before substituting their custom config parameters)
      3. All the configuration parameters used for setting up the predicate (pair, price, side & address)
3. The indexer adds the users market order to an orderbook
   1. The indexer detects the predicate template hash in the witness data
   2. The indexer authenticates that this predicate is relevant to the application
      1. The predicate template hash in the witness data matches what was configured for this indexer (Setup 3.1.1)
      2. The configurable constants are injected into the predicate template configured for this indexer (Setup 3.1)
      3. The predicate root is calculated
      4. If the calculated predicate root matches the owner of the output referred to in 2.3.1, then this witness is authenticated
   3. The indexer calls a handler registered for this predicate template:

      #[indexer(predicate_created)]
      handle_new_market_order(output: Output, order: Order) {
          // the output is pulled from the transaction using the output idx specified in the witness data
          if order.pair == "ETH-USDC" {
              ETH_USDC_BOOK.insert(output.utxo_id(), order);
          }
      }
      

   4. The UtxoId is also tracked by the indexer to call a different handler when the predicate is spent

User B fulfills market order by selling eth

1. Using the indexer API, user B finds user A's buy order for ETH by scanning the ETH_USDC_BOOK table
2. User B fulfills buy order by submitting a script transaction
   1. A coin input with 1 ETH
   2. The predicate input created by user A
   3. A coin output assigning 1 ETH to user A's address defined in the witness
   4. A coin output claiming $2,000 USDC to user B's wallet from the predicate input
3. The indexer detects user A's predicate is spent and removes it from the orderbook:

     // the indexer remembers the utxo id of the predicate and it's associated metadata
     // during the predicate_created step so that this corresponding handler can be called
     #[indexer(predicate_spent)]
     handle_fulfilled_market_order(input: Input, order: Order) {
         if order.pair == "ETH-USDC" {
             ETH_USDC_BOOK.remove(input.utxo_id());
         }
     }
   

[chlenc]

Hi guys, I am happy to participate in this discussion in any case, if you need any help tag me 🫡

[ra0x3]

Temporarily blocked by release of fuels 0.50