-
Notifications
You must be signed in to change notification settings - Fork 79
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add orderbook #1429
Add orderbook #1429
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good! Definitely going in the right direction.
pallets/order-book/src/lib.rs
Outdated
ensure!( | ||
T::ReserveCurrency::can_reserve( | ||
&account_id, | ||
T::Fees::fee_value(T::OrderFeeKey::get()) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could be a conflict if we allowed trading_curreny
= reserve_curreny
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah, yeah it does look like that could result in a failure later at the second reserve in the extrinsic if the asset out is also the reserve currency. Looking into that now.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's updated to check the fee currency against the asset out currency, and consolidate the reserves when they match.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Currently updating other reserve/unreserve locations to make sure correct currency type is reserved/unreserved when the outgoing currency matches the fees so the mechanism is always consistent (i.e. so we don't have any reserves done through native balances trait impl that are undone via orml impl.)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
All reserve/unreserve should be covered now, cleaning it up, adding additional coverage around this as well now.
pallets/order-book/src/lib.rs
Outdated
pub fn get_account_orders( | ||
account_id: T::AccountId, | ||
) -> Result< | ||
sp_std::vec::Vec<( | ||
T::Hash, | ||
Order<T::Hash, T::AccountId, T::AssetCurrencyId, T::ForeignCurrencyBalance>, | ||
)>, | ||
Error<T>, | ||
> { | ||
Ok(<UserOrders<T>>::iter_prefix(account_id).collect()) | ||
} | ||
|
||
/// Get all orders | ||
/// Provided for frontend to grab all open orders | ||
pub fn get_all_orders() -> Result< | ||
sp_std::vec::Vec<( | ||
T::Hash, | ||
Order<T::Hash, T::AccountId, T::AssetCurrencyId, T::ForeignCurrencyBalance>, | ||
)>, | ||
Error<T>, | ||
> { | ||
Ok(<Orders<T>>::iter().collect()) | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You might want to use this for a RPC to let it be queried by the front end easier, right? But can't you just access the storage items Orders
and UserOrders
directly instead of using an RPC? I always think if you have complex computations, it's cool to provide a RPC for it, but just querying the storage might not be a big deal for the front end. In addition: If you access the functions inside the runtime, I think you know that iter...().collect()
is a bad practice and usually leads to overweight of blocks. But I assume the functions will never be called inside the runtime.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That's a very good point, as it is only for frontend it won't be called in the runtime, and therefore won't affect blockweights, but definitely a good thing to check! :)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Agree with @Chralt98, I would remove these methods to avoid bad usage from the runtime. If they're needed for testing, the tester can call iter_prefix()
or iter()
directly.
Forcing to use iter
or iter_prefix
the user can see that this implies a lot of read accesses.
pallets/order-book/src/lib.rs
Outdated
#[pallet::call_index(2)] | ||
// dummy weight for now | ||
#[pallet::weight(10_000 + T::DbWeight::get().reads_writes(2, 2).ref_time())] | ||
pub fn fill_order_full(origin: OriginFor<T>, order_id: T::Hash) -> DispatchResult { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Are there plans to partially fill orders? I mean assume one big ask order comes in at the best price (lowest ask). Now a small fish can't really buy at this price, because the order is too big for the small fish.
The other case is the trickier: Assume there are many small ask orders at the best price (lowest ask). Now a big whale places a market order and buys all small ask orders. First, this often creates slippage, as you might know already.. Second, there might be not enough open ask orders to fill the big bid of the whale. Third, the whale needs to transact with many small fish accounts to fulfil the small orders. This leads to a high computational demand and therefore high transaction costs.
So is your plan to just allow the orders to be filled 100% to mitigate this and sacrifice that a small fish can not fill the possible big order?
<T as frame_system::Config>::Hash: PartialEq<<T as frame_system::Config>::Hash>, | ||
{ | ||
/// Create an order, with the minimum fulfillment amount set to the buy | ||
/// amount, as the first iteration will not have partial fulfillment |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Oh I saw this after this message.
But what are your plans to realise not partial fulfillment, but the other way around: If a big order needs to eat many small orders?
Assume there are many small ask orders at the best price (lowest ask). Now a big whale places a market order and buys all small ask orders. First, this often creates slippage, as you might know already.. Second, there might be not enough open ask orders to fill the big bid of the whale. Third, the whale needs to transact with many small fish accounts to fulfil the small orders. This leads to a high computational demand and therefore high transaction costs.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thats a good question. The initial plan was to match orders of corresponding currency pairs and compatible prices and min fulfillment amounts, however there was some concern with the weight of matching those up, it's something I believe we plan on revisiting in the future.
}) | ||
); | ||
}); | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I know this is wip, but I assume you know that a fill_order_full
test is missing.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, I had not gotten to that test by then. I was aware and it is added now, but thanks for looking out for that! :)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think the error from this slack issue comes because cfg-mocks
and/or cfg-test-utils
dont have their runtime-benchmarks
feature added
0e66add
to
6df9f8d
Compare
Still reviewing on my end, but putting it in review now as it would be good to get it merged sooner if possible to have available for frontend. I would like to extend tests and benchmarks more, but that might be better to do in second PR. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Fast first review with some nitpick comments and questions to understand this better. I would make a second deep review focusing in the main methods and implementation logic.
But everything seems clear and well structured!
libs/traits/src/lib.rs
Outdated
/// be inmediately and completelly fullfilled. If there was already an | ||
/// active order with the same account currencies, the order is | ||
/// increased/decreased and the same order id is returned. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Then, why do we need an update_order()
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
update_order
allows accounts to update active orders they have, and can also be used when we add partial fulfillment.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
But given that comment, it seems like place_order
does the same if the order already exits.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah, the current place_order
creates a new order instead of updating the order, the docstring wasn't updated to match the behaviour, but it might be better to have behaviour match docstring.
@wischli would it be better for place_order
to update an existing order for an account/currency pair?
If that's the case it would make sense to switch order id back to hash, but without nonce--this should be a simple change though.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Verified current behaviour is good, and updated docstring to make sure it's accurate.
cfg_test_utils::mocks::orml_asset_registry::impl_mock_registry! { | ||
RegistryMock, | ||
CurrencyId, | ||
ForeignCurrencyBalance, | ||
CustomMetadata | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could this be migrated to a pallet mock?
cc @mustermeiszer
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
of course. Was the old approach before we had your mock builder.
pallets/order-book/src/lib.rs
Outdated
pub fn get_account_orders( | ||
account_id: T::AccountId, | ||
) -> Result< | ||
sp_std::vec::Vec<( | ||
T::Hash, | ||
Order<T::Hash, T::AccountId, T::AssetCurrencyId, T::ForeignCurrencyBalance>, | ||
)>, | ||
Error<T>, | ||
> { | ||
Ok(<UserOrders<T>>::iter_prefix(account_id).collect()) | ||
} | ||
|
||
/// Get all orders | ||
/// Provided for frontend to grab all open orders | ||
pub fn get_all_orders() -> Result< | ||
sp_std::vec::Vec<( | ||
T::Hash, | ||
Order<T::Hash, T::AccountId, T::AssetCurrencyId, T::ForeignCurrencyBalance>, | ||
)>, | ||
Error<T>, | ||
> { | ||
Ok(<Orders<T>>::iter().collect()) | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Agree with @Chralt98, I would remove these methods to avoid bad usage from the runtime. If they're needed for testing, the tester can call iter_prefix()
or iter()
directly.
Forcing to use iter
or iter_prefix
the user can see that this implies a lot of read accesses.
pallets/order-book/src/lib.rs
Outdated
ensure!( | ||
T::ReserveCurrency::can_reserve( | ||
&account, | ||
T::Fees::fee_value(T::OrderFeeKey::get()) | ||
), | ||
Error::<T>::InsufficientReserveFunds, | ||
); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think you could reserve directly without this check. If you can not reserve, you will get an error
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I was thinking it would be cheaper in cases where reserve was not available to try and check before reserve, but we can also remove if wanted as well.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
No strong opinion here 👍🏻. It's just I'm always checking lines that could be removed 😆.
To give my reasoning, I think in case of an error, the weight of this call will be the same, which is higher than the computation time the node wasted on it for an error. The weight will be added to the current block weight as if the call was ok. So, from my understanding, we would not be saving computation or making it cheaper.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
All of the reserve checks except the reserve check before the transfers to ensure enough funds can actually be transferred were removed. Also just Found/fixed an error with fund check & added tests when double checking again for extra superfluous checks.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Second review round, mostly questions to understand it better and some suggestions. I would appreciate another eyes on this 🙌🏻.
Thanks for your work on this! It's pretty complex and seems like you've thought in weird border cases.
/// amount, as the first iteration will not have partial fulfillment | ||
#[pallet::call_index(0)] | ||
#[pallet::weight(T::Weights::create_order_v1())] | ||
pub fn create_order_v1( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Curious about why _v1()
here
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah, the initial orders are pretty simple, it's currently just specifying a buy price with full fulfillment, but we're going to have partial fulfillment as well in the future, so I wanted to make sure that we could add that functionality in the future without having to modify an existing extrinsic.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ok 👍🏻 . Anyways, from my understanding, I think it's not a problem if the existing extrinsic is modified later, communicating a break API change (or internal change)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I would also change that. We could rather have another create_order_with_partial
or so in the future
pallets/order-book/src/lib.rs
Outdated
/// Cancel an existing order that had been created by calling account. | ||
#[pallet::call_index(1)] | ||
#[pallet::weight(T::Weights::user_cancel_order())] | ||
pub fn user_cancel_order(origin: OriginFor<T>, order_id: T::Hash) -> DispatchResult { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe cancel_order
without user_
prefix (?). Is it added because of name collision with the trait?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah, it's the name collision with the trait, so wanted to specify that this would be the one callable by users.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think it is totally fine having both names at the same time. If the user wants to call the trait once, they can use <Self as Trait>::cancel_order()
. I would also say that user_
is a generic word, it changes depending on the context: every call has a "user" who calls it. So not sure if it express the idea you want.
Anyway, nitpick things not mandatory!
&account_id, | ||
sell_amount, | ||
)?; | ||
Self::remove_order(order.order_id)?; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's probable the front-end wants, sooner or later, to show the closed orders. We should probably add a new "closed orders" storage to allow front-end people to fetch it and show the user that a previous order was already fulfilled and closed.
cc @mustermeiszer @wischli
Anyways, not blocking, I think it's something we can add later in case of needing it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I am a little concerend about storage in this case. But I agree, that it would be nice.
pallets/order-book/src/lib.rs
Outdated
T::TradeableAsset::unreserve( | ||
order.asset_out_id, | ||
&order.placing_account, | ||
order.max_sell_amount, | ||
); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Out of curiosity, why reserve/unreserve
and not hold/release
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I was using reserve
/unreserve
as those where provided by orml via MultiReservableCurrency
, which seemed the safest bet for foreign asset trades. But hold
/release
do seem more preferable here.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not sure exactly what the difference is, TBH. Is it the same thing, or one of them does some transfer under the hood?
pallets/order-book/src/lib.rs
Outdated
Error::<T>::InvalidAssetId | ||
); | ||
<NonceStore<T>>::try_mutate(|n| { | ||
*n = n.ensure_add(T::Nonce::one())?; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Totally nitpick: there is a ensure_add_assign
for these cases
0a2ea5a
to
fd754cc
Compare
8696ba5
to
31e65bf
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Approved! We can merge it to unlock Apps team. Thanks for your work on this Thea! 🙌🏻
Anyways, I would love to have other eyes on it, because I'm not fully inmersed in this topic. Would be great to have a postmorten review maybe from @mustermeiszer @wischli
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe I am missing something, but I think we are missing some essential decimal conversion.
Furthermore, I am not understanding sell_price_limit
. ^^ I need an example with numbers here. @thea-leake
/// Returns the order id created with by this buy order if it could not | ||
/// be immediately and completely fulfilled. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is misleading. IF the order is to be fulfilled directly, we can not return an OrderId
but the return type is Result<OrderId, DispatchError>
. We should either change the docs or change the return type.
/// `sell_price_limit` defines the lowest price acceptable for | ||
/// `currency_in` currency when buying with `currency_out`. This |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Would love to have an example here with numbers. ^^
cfg_test_utils::mocks::orml_asset_registry::impl_mock_registry! { | ||
RegistryMock, | ||
CurrencyId, | ||
ForeignCurrencyBalance, | ||
CustomMetadata | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
of course. Was the old approach before we had your mock builder.
/// amount, as the first iteration will not have partial fulfillment | ||
#[pallet::call_index(0)] | ||
#[pallet::weight(T::Weights::create_order_v1())] | ||
pub fn create_order_v1( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I would also change that. We could rather have another create_order_with_partial
or so in the future
/// Fill an existing order, fulfilling the entire order. | ||
#[pallet::call_index(2)] | ||
#[pallet::weight(T::Weights::fill_order_full())] | ||
pub fn fill_order_full(origin: OriginFor<T>, order_id: T::OrderIdNonce) -> DispatchResult { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@Chralt98 do you have an idea on how to realise the whale-eat-small
situation?
I am curious what thoughts you already had on this and what is to be considered to make this efficient?
Do you think a fill_orders(origin: OriginFor<T>, order_ids: Vec<T::OrderIdNonce>)
would be sufficient? The whale could compute off-chain what is the best. We could even do fill_orders_partial(origin: OriginFor<T>, order_ids: Vec<(T::OrderIdNonce, Perquintill)>)
allowing to partially fulfill orders.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
A dedicated runtime api for computing the needed T::OrderIdNonce
s could provide an easy and long lasting API for front-ends/
/// Swap tokens buying a `buy_amount` of `currency_in` using the | ||
/// `currency_out` tokens. The implementator of this method should know | ||
/// the current market rate between those two currencies. | ||
/// `sell_price_limit` defines the lowest price acceptable for |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Shouldn't this be the hightest
accetable price?
/// Swap tokens buying a `buy_amount` of `currency_in` using the | ||
/// `currency_out` tokens. The implementator of this method should know | ||
/// the current market rate between those two currencies. | ||
/// `sell_price_limit` defines the lowest price acceptable for |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think I am not understanding the sell_price_limit
. Would this be the minimum amount of currency_in
a user wants to get for 1
unit of currency_out
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I understand sell_price_limit
as a way of protecting my order against a big conversion ratio.
So, if I want to buy 10 selling 20, it will be ok if sell_price_limit
is 30, but if the conversion drops to 0.25, then I would not sell 40 to buy 10. (double-check this @thea-leake)
Would this be the minimum amount of currency_in a user wants to get for 1 unit of currency_out?
I understand define it as the maximum amount of currency_out
I would give for getting the buy_amount
of currency_in
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
as we use it as buy_amount.ensure_mul(sell_price_limit)
I think it is not as you described it above, right? In your example it would rather be
buy_amount = 10.00
sell_price_limit = 0.5
But I think we are having a logical bug here.
type TradeableAsset: MultiReservableCurrency< | ||
Self::AccountId, | ||
Balance = <Self as pallet::Config>::ForeignCurrencyBalance, | ||
CurrencyId = Self::AssetCurrencyId, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think this should be a Substrate native trait instead of orml one.
/// Type for currency orders can be made for | ||
type TradeableAsset: MultiReservableCurrency< | ||
Self::AccountId, | ||
Balance = <Self as pallet::Config>::ForeignCurrencyBalance, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The Balance
type should be equal for Reservable
and ForeignCurrency
// might need decimals from currency, but should hopefully be able to use FP | ||
// price/amounts from FP balance | ||
|
||
let sell_amount = order.buy_amount.ensure_mul(order.price)?; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
IIUC we are missing decimal conversion here. WDYT?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes... I think you're right. If used Balance
here have different decimal representation, we should unify them.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah, I think you're right, as we're dealing with the overall balances and not the adjusted precision from the registry.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Regarding the conversions, in the Oracle PR, I add a module to perform that kind of conversions. Maybe you can extend that module @thea-leake
with a balance_to_balance()
conversion: https://github.com/centrifuge/centrifuge-chain/pull/1474/files#diff-833d286c0e9655fa1b12da1bceabc7ff8988d1c74b7149be553320af61b78b54
Hopefully this PR is merged today or tomorrow
I also think we should make If I get it right, this limit is saying. I want at least |
From my understanding of this:
|
Agreed! I think we use as
|
Yeah, I agree it should be a rate; I think having the prices as Fixed Points would be much easier to reason about from a usage perspective. |
The |
Description
Adds the orderbook pallet.
Changes and Descriptions
Adds pallet providing an orderbook to exchange assets.
Adds pallet to dev runtime, and dev runtime weights.
Checklist: