CHIP-0052: Partial Offers

[Yakuhito]

No description provided.

[Yakuhito]

The CI of the 'partial' repo has a step that outputs costs for all the test cases:

Note: I'm editing this post to keep costs up-to-date with the master branch.

+-------------------------------------------------+------------+---+----------+----------+----------+
| Cost statistics for Partial Offer (XCH for CAT) |            |   |          |          |          |
+-------------------------------------------------+------------+---+----------+----------+----------+
| label                                           | avg        | n | min      | max      | median   |
+-------------------------------------------------+------------+---+----------+----------+----------+
| partial_fill                                    | 59921970.0 | 4 | 57800318 | 62043622 | 59921970 |
+-------------------------------------------------+------------+---+----------+----------+----------+
| full_fill                                       | 58157953.0 | 4 | 56036301 | 60279605 | 58157953 |
+-------------------------------------------------+------------+---+----------+----------+----------+

+--------------------------------------------------+------------+---+----------+----------+----------+
| Cost statistics for Partial Offer (XCH for rCAT) |            |   |          |          |          |
+--------------------------------------------------+------------+---+----------+----------+----------+
| label                                            | avg        | n | min      | max      | median   |
+--------------------------------------------------+------------+---+----------+----------+----------+
| partial_fill                                     | 70226875.0 | 4 | 68105223 | 72348527 | 70226875 |
+--------------------------------------------------+------------+---+----------+----------+----------+
| full_fill                                        | 68462858.0 | 4 | 66341206 | 70584510 | 68462858 |
+--------------------------------------------------+------------+---+----------+----------+----------+

+-------------------------------------------------+------------+---+----------+----------+----------+
| Cost statistics for Partial Offer (CAT for XCH) |            |   |          |          |          |
+-------------------------------------------------+------------+---+----------+----------+----------+
| label                                           | avg        | n | min      | max      | median   |
+-------------------------------------------------+------------+---+----------+----------+----------+
| partial_fill                                    | 87675942.0 | 4 | 87565854 | 87786030 | 87675942 |
+-------------------------------------------------+------------+---+----------+----------+----------+
| full_fill                                       | 85904782.0 | 4 | 85794694 | 86014870 | 85904782 |
+-------------------------------------------------+------------+---+----------+----------+----------+

+--------------------------------------------------+-------------+---+-----------+-----------+-----------+
| Cost statistics for Partial Offer (rCAT for XCH) |             |   |           |           |           |
+--------------------------------------------------+-------------+---+-----------+-----------+-----------+
| label                                            | avg         | n | min       | max       | median    |
+--------------------------------------------------+-------------+---+-----------+-----------+-----------+
| partial_fill                                     | 110988952.0 | 4 | 110877384 | 111100520 | 110988952 |
+--------------------------------------------------+-------------+---+-----------+-----------+-----------+
| full_fill                                        | 109198932.0 | 4 | 109087364 | 109310500 | 109198932 |
+--------------------------------------------------+-------------+---+-----------+-----------+-----------+

+--------------------------------------------------+-------------+---+-----------+-----------+-----------+
| Cost statistics for Partial Offer (CAT for rCAT) |             |   |           |           |           |
+--------------------------------------------------+-------------+---+-----------+-----------+-----------+
| label                                            | avg         | n | min       | max       | median    |
+--------------------------------------------------+-------------+---+-----------+-----------+-----------+
| partial_fill                                     | 124943591.0 | 4 | 122817031 | 127070151 | 124943591 |
+--------------------------------------------------+-------------+---+-----------+-----------+-----------+
| full_fill                                        | 123172431.0 | 4 | 121045871 | 125298991 | 123172431 |
+--------------------------------------------------+-------------+---+-----------+-----------+-----------+

+-------------------------------------------------+-------------+---+-----------+-----------+-----------+
| Cost statistics for Partial Offer (CAT for CAT) |             |   |           |           |           |
+-------------------------------------------------+-------------+---+-----------+-----------+-----------+
| label                                           | avg         | n | min       | max       | median    |
+-------------------------------------------------+-------------+---+-----------+-----------+-----------+
| partial_fill                                    | 114638686.0 | 4 | 112512126 | 116765246 | 114638686 |
+-------------------------------------------------+-------------+---+-----------+-----------+-----------+
| full_fill                                       | 112867526.0 | 4 | 110740966 | 114994086 | 112867526 |
+-------------------------------------------------+-------------+---+-----------+-----------+-----------+

+---------------------------------------------------+-------------+---+-----------+-----------+-----------+
| Cost statistics for Partial Offer (rCAT for rCAT) |             |   |           |           |           |
+---------------------------------------------------+-------------+---+-----------+-----------+-----------+
| label                                             | avg         | n | min       | max       | median    |
+---------------------------------------------------+-------------+---+-----------+-----------+-----------+
| partial_fill                                      | 148256601.0 | 4 | 146128561 | 150384641 | 148256601 |
+---------------------------------------------------+-------------+---+-----------+-----------+-----------+
| full_fill                                         | 146466581.0 | 4 | 144338541 | 148594621 | 146466581 |
+---------------------------------------------------+-------------+---+-----------+-----------+-----------+

+--------------------------------------------------+-------------+---+-----------+-----------+-----------+
| Cost statistics for Partial Offer (rCAT for CAT) |             |   |           |           |           |
+--------------------------------------------------+-------------+---+-----------+-----------+-----------+
| label                                            | avg         | n | min       | max       | median    |
+--------------------------------------------------+-------------+---+-----------+-----------+-----------+
| partial_fill                                     | 137951696.0 | 4 | 135823656 | 140079736 | 137951696 |
+--------------------------------------------------+-------------+---+-----------+-----------+-----------+
| full_fill                                        | 136161676.0 | 4 | 134033636 | 138289716 | 136161676 |
+--------------------------------------------------+-------------+---+-----------+-----------+-----------+

[Yakuhito]

There is also this summary available regarding some design choices made for this standard.

[SlowestTimelord]

This is really important functionality! Thanks for getting it out there @Yakuhito and CNI team! And also shout out to chia-wallet-sdk

[danieljperry]

This CHIP is now a Draft. Please leave your reviews here, and feel free to discuss it in the #chips channel of our Discord.

[judeallred]

This is incredible and I'm a huge fan of bringing this to CHIA. I think it will make DEX and offer experiences much easier to use.

A few questions / notes:

• What is the behavior when, in a given block, there are multiple takers to a partial offer? My assumption is that they will all be filled unless they overflow the amount of assets provided by the offer, in which case a subset (determined by the order chosen by the farmer / influenced by fee) will succeed and the remainder will fail. Some test case examples:

  • Buyers A, B, and C each seek to take 1/3rd of the offer. Buyer D seeks to take 1/2 of an offer. All takers submit within the same block.
    • Possible resolutions seem to be:
    • [A, B, C] fill, D does not.
    • D plus one of [A, B, C] fill
    • Only one fills, the rest bounce (not desired) or postpone until the next block (suboptimal)
    • Two of [A, B, C] fill, and D fills. At least one of them only has a partial fill.

• Does the partial fill behavior only apply to the maker side of the offer, or does it apply to the taker side as well?

  • As a taker, if I try to take 10% of a partial offer, am I guaranteed a "fill-or-kill" behavior, or is it possible that I can receive less than 10%?
    • As a taker, I feel like I'd expect a fill-or-kill behavior on my side of the experience, especially because of the minimum spend and fee dynamic.
    • I suppose optimally, as a taker of a partial offer I'd like to be able to specify if I am willing to accept a partial fill vs a complete fill.

• Is the partial offer ID persistent throughout multiple partial fills, or does the offer ID mutate as the underlying coin changes? I assume it's consistent (so that e.g. on Dexie I could look up a partial offer by id and see its chain of events, or put that ID into a wallet like Sage and accept it without having to find the latest offer id)

• I'm interested in others thoughts on if 'partial' is the best prefix for this. It feels like a broad term which might apply better to some other future primitive that could exist in Chia-- I wonder if partial-offer or something of that nature might be a better choice? It may be perfectly fine as-is.

[Yakuhito]

Hey Jude, thank you a lot for the quesitons/notes. Much appreciated!

What is the behavior when, in a given block, there are multiple takers to a partial offer?

Right now, wallets would need to use the Replace-by-Fee mechanism to allow multiple people to fill the same partial offer in the same block. When A spends the partial coin to partially fill the offer, the coin creates a new partial coin with the remainder amount - B would need to spend the newly-created coin, meaning that B needs to include A's bundle in the new mempool item to fill the offer (or wait until the mempool item is confirmed).

In the end, a farmer will only include valid bundles in a block. The 'simple' scenario is that A broadcasts a bundle, then B broadcasts a new mempool item that is a superset of A's, and C comes along and uses RBF on the [A, B] mempool item. The farmer may then include [A, B, C] in the next block. If D submits their bundle around the same time as B, the options become [A, B, C] and [A, D] - some farmers will consider [A, D] valid and that [A, B, C] is conflicting, while others will consider [A, B, C] valid (due to mempool propagation). Retries & how to handle those cases would be left up to individual wallets.

Does the partial fill behavior only apply to the maker side of the offer, or does it apply to the taker side as well?

Partial offers are 'compatible' with normal offers. If you use a normal offer, which I assume would be the default for wallets, you get 'fill-or-kill'. As you noted, you may also create a 'matching' partial offer to accept partial fills.

Is the partial offer ID persistent throughout multiple partial fills, or does the offer ID mutate as the underlying coin changes?

Usually, offer IDs were either the hash of the 'offer1...' string (current Dexie behavior) or the hash of the spend bundle you get when you decode them (reference wallet behavior). Both change with every partial offer fill, but it's easy to follow the partial offer on-chain as a partial fill transaction leads to a new coin being created, up until the offer is cancelled/expires/is fully filled.

I'm interested in others thoughts on if 'partial' is the best prefix for this.

Interesting point - everyone I talked to until now assumed the prefix would be partial. Curious to see what others think as well - to me, partialoffer seems a bit long. Changing the prefix is easy in the code; we just need to make sure the CHIP is consistent so we don't end up with more than one prefix.

[hoffmang9]

Arguably these are dynamic offers.

[danieljperry]

Yak will present the details of this CHIP over a Zoom call on Monday, September 29. See the CHIPs channel of our Discord for more details.

[judeallred]

What would you estimate the scope would be to support multiple assets (still restricted to xch, cat, and rcat) on either side of the offer?

I'm thinking about the case of liquidity pool tokens ala TibetSwap. It seems like one could have a partial offer which understands the arithmetic of the ratios between the assets ( 1 TIBET-DBX-XCH -> [0.00593 XCH, 0.84 DBX])

It's a little more complicated (perhaps uncomfortably so) to allow [multi-asset] -> [multi-asset], but having it permitting multiple assets on one (but not both) sides of the offer might be a nice addition.

[judeallred]

How do partial offers handle arithmetic underflow errors?
For example if I had an offer of [1 XCH => 1 DBX], since the granularity of XCH is 1,000,000,000,000 but the granularity of DBX is 1,000, how does the partial offer resolve the the lower granularity?

test cases:
Given 1XCH -> 1DBX partial offer

• Take 100 mojos. Outcome? (expected: failure to create spend bundle, or 0 DBX + 100 mojos as change)
• Take 0.1XCH + 100mojos. Outcome? (expected: 0.1DBX + 100 mojos as change)
• A takes (0.5xch + 10 mojos), submits to mempool. B takes (0.5xch + 10 mojos), RBF's A's submission and submits to mempool. Outcome?

Do partial offers guard against accepting them with values greater than what's expected?
test case:

• given 1XCH -> 1DBX partial offer
  • take 10 XCH.
    expected: Either fails to create spend bundle, or successfully purchases 1 DBX and receives 9 XCH as change

[edit: correct stated precision of DBX]

[Yakuhito]

What would you estimate the scope would be to support multiple assets (still restricted to xch, cat, and rcat) on either side of the offer?

The current partial puzzle is just 438 bytes long - it's specialized, which allows it to be tiny. Adding one-for-many (i.e., taker has to pay more than one asset to fill the partial offer) would remove a lot of the optimizations in the puzzle - for example, you'd need a recursive function that creates announcement asserts for each asset, which would also make the environment harder to access (it'd have several arguments). I'd expect the puzzle to at least double in size, which is a bad trade-off given that most usage will involve one asset being traded for another. The smaller the puzzle, the more partial offer spends we can fit in a block, and the more efficient the market is overall.

To offer multiple assets, the complexity would also grow a lot (not just the puzzle size) - I'd design it so the partial offers would be spent in a ring, with the lead coin asserting that the other side paid the fair amount. Each CAT/rCAT needs to be its own coin, so the coins would need to coordinate with each other (or at least assert the same nonce without being vulnerable to nonce replay between two separate partial offers). Alternatively, you could have a singleton owning all assets.

In both cases, supporting multiple assets leads to the 'one asset for one asset' case being more expensive because the puzzle has to include the logic for supporting multiple assets (even if it's not used). I think that the right approach is to keep this optimized puzzle for partial offers, and make another one for multi-asset support should the need arise in the future. To borrow an idea from @Rigidity, wallets adding multi-asset support will have to make a lot of updates anyway, so using a different puzzle shouldn't make a difference in adoption.

I'm thinking about the case of liquidity pool tokens ala TibetSwap. It seems like one could have a partial offer which understands the arithmetic of the ratios between the assets ( 1 TIBET-DBX-XCH -> [0.00593 XCH, 0.84 DBX])

I don't think this is a use case - when you add assets to TibetSwap, you add 50% of value in XCH and 50% in the CAT at the pool's current price. It doesn't matter how much liquidity you add - the ratio ('price') is the same. I don't see value in using partial offers there because successfully adding some liquidity means you would've been able to add 100% of the liquidity as well.

[Yakuhito]

The logic is here. Precision is left flexible, but usually the values are in mojos. The rounding is always done in the partial coin's favor (so the taker can lower their bid up until the 'discrepancy' is 0). The amount the partial coin will give the taker is based to what the taker gives (other_asset_amount).

To clarify, in the example below, other_asset_amount is the amount of XCH the taker gives to fill the partial offer. The partial offer coin is a CAT (DBX).

For example if I had an offer of [1 XCH => 1 DBX

1 XCH - 10^18 mojos ; 1 DBX = 1000 mojos
PRECISION = 10^12 ; PRICE_PRECISION = 1000

Take 100 mojos. Outcome?

(/ (* other_asset_amount PRICE_PRECISION) PRECISION) = (/ (* 100 1000) 10^12) = 0 DBX mojos go to the taker

Take 0.1XCH + 100mojos.

If you give the partial offer coin the whole value, it will happily swallow the change:
(/ (* other_asset_amount PRICE_PRECISION) PRECISION) = (/ (* 100000000100 1000) 10^12) = 100 DBX mojos (0.1 DBX) go to the taker
In this case, the driver/wallet would 'reverse' the equation and know to give just 0.1 XCH, which results in the same CAT amount being given to the taker:
(/ (* other_asset_amount PRICE_PRECISION) PRECISION) = (/ (* 100000000000 1000) 10^12) = 100 DBX mojos

A takes (0.5xch + 10 mojos), submits to mempool. B takes (0.5xch + 10 mojos), RBF's A's submission and submits to mempool. Outcome?

The drivers catch that - or the maker has 20 extra mojos. The main idea is that the taker can arrive at the minimum amount they need to offer to get n DBX.

given 1XCH -> 1DBX partial offer, take 10 XCH.

Driver/wallet would limit the amount to 1 XCH. But, just for fun, let's assume they won't. Then the partial coin will send the following amount to the taker:
(/ (* other_asset_amount PRICE_PRECISION) PRECISION) = (/ (* 10^13 1000) 10^12) = 10000 DBX mojos (10 DBX)
The partial offer coin will not recreate itself. Assuming no one filled the partial offer before, a 1 DBX partial coin asserted 10 XCH were paid to the maker and created 10 DBX worth of value (going to the taker). The taker could make this work by spending 9 DBX so the deltas add up - otherwise, the transaction would fail.

[judeallred]

@Yakuhito Thank you for your thoughtful and thorough answers!

[danieljperry]

In case you missed it, here is Yak's discussion of this CHIP:

https://youtu.be/cVhQf1rP5XM

[SlowestTimelord]

So if I understand correctly, the initial offer is off-chain but subsequent partial offers are on-chain. It sounds to me like there's an opportunity here to more properly define an on-chain offer standard even for the initial offer or normal offers in general. The major benefit would be on-chain discovery and being able to replace the current "counter offer" functionality that's sharing off-chain data (plus a minor side benefit of a shorter offer string). The obvious downside is the inefficient use of blockspace but do you think it's worth explicitly defining that standard anyhow?

To build on Rigidity's question in the call, a nice property of partial offers would be easy discoverability by wallets (without needing to do offer backups) so that expired partial offers could just be displayed and spent alongside other "regular" coins without a need to explicitly cancel them by spending them to wallet controlled puzzle hash. I understand this is possible now but it sounds like it requires the wallet to at least know about the initial offer first.

I also think it's worth considering different terms here as to not overload "partial offers":

1. The name of the standard: e.g. partial fills, partial offers
2. An offer file that allows partial fills: e.g. dynamic offer, divisible offer
3. The subsequent on-chain coins: e.g. partial offer coin, remainder coin, fragment coin
4. The offer file created from the on-chain coins: e.g. partial offer, remainder offer, fragment offer

[Yakuhito]

Hey stl, thank you for your comment.

It sounds to me like there's an opportunity here to more properly define an on-chain offer standard even for the initial offer or normal offers in general. The major benefit would be on-chain discovery and being able to replace the current "counter offer" functionality that's sharing off-chain data (plus a minor side benefit of a shorter offer string).

I would bring up the "x% orders remain unfulfilled in traditional markets." argument - having to post offers of any kind on-chain without knowing if they'll be filled at all seems like a very big barrier, especially under fee pressure conditions. We can always come and change this in the future (it doesn't change the Chialisp itself; it's just a matter of consensus/ecosystem support - and could pass as a separate CHIP), but, for now, I think supporting on-chain creation requires a lot of effort for something that feels like a step backwards. One of the main advantages of both types of offers is that they don't require transactions until a counterparty is found - if that weren't the case, I think the best method to swap two assets is to deposit concentrated liquidity in a v3 AMM and withdraw your LP after the price has moved over your position (assuming such an AMM would exist on Chia).

a nice property of partial offers would be easy discoverability by wallets (without needing to do offer backups) so that expired partial offers could just be displayed and spent alongside other "regular" coins without a need to explicitly cancel them by spending them to wallet controlled puzzle hash

This is a very good point, and I agree. I had a call with @Rigidity this morning and we think it's worth it to hint the initial partial offer coin to the wallet. It would add ~100 bytes to the initial transaction, but would allow the wallet to discover all the partial offers that have been accepted at least once by solely looking at the blockchain (no partial offer exports needed). I'll add this to the CHIP shortly.

I also think it's worth considering different terms here as to not overload "partial offers"

Naming in the context of this standard is something I should've thought about sooner. I've certainly changed the terms quite a bit since I first started talking about this standard. Here's how I would call each term now:
1 - Partial offers CHIP or CHIP-0052
2 - Partial offers. I've heard a lot of good suggestions, but they all ultimately come with some possible confusions. To a certain extent, the community (and wallets) will need to help educate newcomers about the functionality of partial offers, much like they already do when it comes to normal offers. My 2 other favorite names are 'orders' and 'dynamic offers.' - but, again, I like almost all suggestions and think we could proceed with any.
3 - Partial offer coins
4 - Partial offer string - although a lot of people will just not say "string," much like they do when they create/upload/ask for offers (not offer strings)

[Yakuhito]

To those following, I would like to propose changing the puzzle of partial offers from Chialisp to Rue. A lot of effort has gone into optimizing the partial offer Chialisp puzzle - however, the Rue-produced CLVM output is ~96% of the Chialisp-produced one (17 less bytes; ~200k cost saved in benchmarks) while being functionally equivalent. This is thanks to Rue's additional optimization methods - some of which cannot be replicated in Chialisp (read more).

[SlowestTimelord]

I would bring up the "x% orders remain unfulfilled in traditional markets." argument - having to post offers of any kind on-chain without knowing if they'll be filled at all seems like a very big barrier, especially under fee pressure conditions. We can always come and change this in the future (it doesn't change the Chialisp itself; it's just a matter of consensus/ecosystem support - and could pass as a separate CHIP), but, for now, I think supporting on-chain creation requires a lot of effort for something that feels like a step backwards. One of the main advantages of both types of offers is that they don't require transactions until a counterparty is found - if that weren't the case, I think the best method to swap two assets is to deposit concentrated liquidity in a v3 AMM and withdraw your LP after the price has moved over your position (assuming such an AMM would exist on Chia).

I agree on-chain initial offers make little sense in most circumstances but I do think there's value in standardizing it (in case anybody does want to do it -- e.g. counter offers) but also agree that should be a separate CHIP.

This is a very good point, and I agree. I had a call with @Rigidity this morning and we think it's worth it to hint the initial partial offer coin to the wallet. It would add ~100 bytes to the initial transaction, but would allow the wallet to discover all the partial offers that have been accepted at least once by solely looking at the blockchain (no partial offer exports needed). I'll add this to the CHIP shortly.

Nice! I think the added bytes is definitely a worthwhile trade off to add hinting.

To those following, I would like to propose changing the puzzle of partial offers from Chialisp to Rue. A lot of effort has gone into optimizing the partial offer Chialisp puzzle - however, the Rue-produced CLVM output is ~96% of the Chialisp-produced one (17 less bytes; ~200k cost saved in benchmarks) while being functionally equivalent. This is thanks to Rue's additional optimization methods - some of which cannot be replicated in Chialisp (read more).

Amazing, shoutout @Rigidity!

[danieljperry]

Yak has answered all questions so far, and he has updated the CHIP to include a Rue version of the puzzle.
This CHIP is therefore now in Review. Please leave your reviews here.