[fastx client api] Expand the Rust Client API

[patrickkuo]

The current client API supports transfer, receive and get balance

It should be expended to support all the operations supported by OrderKind.

Also get_spendable_amount should be replaced by get_possesed_objects as we no longer have the concept of balance in the system.

The expended Client should contains the following functions :

fn transfer_to_fastpay(
    &mut self, 
    object_id: ObjectID, 
    recipient: FastPayAddress, 
    user_data: UserData
) -> AsyncResult<CertifiedOrder, Error>

fn transfer_to_primary(
    &mut self,
    object_id: ObjectID,
    recipient: PrimaryAddress,
    user_data: UserData,
) -> AsyncResult<CertifiedOrder, Error>

fn publish_module(
    &mut self, 
    gas_payment: ObjectRef, 
    module_bytes: Dec<u8>
) -> AsyncResult<CertifiedOrder, Error>

fn call_module(
    &mut self, 
    module: ModuleId,
    function: Identifier,
    type_arguments: Vec<TypeTag>,
    gas_payment: ObjectRef,
    object_arguments: Vec<ObjectRef>,
    pure_arguments: Vec<Vec<u8>>,
    gas_budget: u64,
) -> AsyncResult<CertifiedOrder, Error>

fn receive_from_fastpay(
    &mut self,
    certificate: CertifiedOrder,
) -> AsyncResult<(), Error> 

fn get_possesed_objects(&mut self) -> AsyncResult<Vec<ObjectRef>, Error>

[gdanezis]

A few comments on this Task:

• I think that 'get_own_objects' might be preferable to 'get_possessed_objects' - shorter and less confusing.
• We clearly need to re-think 'transfer_to_fastpay' and 'receive_from_fastpay' what are these meant to do (there is no more fastpay now). Do we use them to init objects?
• Similarly there is no primary so 'transfer_to_primary' may not make sense.
• This leaves us with the core 'publish_module' and 'call_module' which we for sure need.
• What about reading object state? I expected a 'read_object' that returns the state of the object?
• What about requesting certificates for sync?

This PR is about the Rust implementation. Probably we should make separate PRs about exposing writes / reads through REST APIs, and then JS/Python/Curl libraries using them to support end clients and integrations. Probably we populate #67 and #68 .

[patrickkuo]

Agree @gdanezis! We definitely need a way to read object state, I had a similar conversation with @oxade yesterday about this.

I really should have written down some user stories and the interaction between client and the platform, I will do it here now.

1. As a user, I want to publish an object to FastX

   • User can publish object to FastX using fn publish_object(object_bytes : Vec<u8>) -> AsyncResult<CertifiedOrder, Error>
   • An object_id will be generated using the hash of the object_bytes.
   • The object_bytes will be stored in client state for later retrieval (Or should it be transferred to authorities? Or CDN?).
   • The object_id will be sent to the authorities in a Transfer Order.
   • A certificate will be returned back to the client with the object id (CertifiedOrder)

2. As a user, I want to transfer object to another user.

   • User can transfer object using fn transfer(object_id:ObjectID,recipient: FastPayAddress)
   • A certificate will be returned back to the client (CertifiedOrder)
   • User can pass on the certificate to another user off-platform(?).

3. As a user, I want to sync my client with authorities so my states and certificates are up-to-date.

   • The client will communicate with the authorities to get the latest certificates that's owned by the user (Is it owner's responsibility to maintain the up-to-date list of object_id or should authorities maintain a map of owner address -> object_id ?).

4. As a user, I want to view my inventory.

   • User can retrieve list of object_id for all the objects they own get_own_objects()
   • a list of object_ids will be returned to the user from the local client_state

5. As a user, I want to download my object

   • User can retrieve the object using read_object(object_id)
   • object_bytes will be returned from local storage (or from Authorities? CDN?)

Some questions:

• How do we handle init object? I believe the object_id will need to be generated from the object_bytes, but do we need to send it to the authorities to verify? Sending over the object_bytes will probably impact performance? Or do we trust client will do the right thing?
• I believe authorities will store all the object_ids and certificates, and the Client will be able to retrieve everything from authorities in case of data lost? Is this the right assumption?

So I think the scope of work for this issue are:

1. remove transfer_to_fastpay, transfer_to_primary, receive_from_fastpay from Client
2. implement publish_object, transfer_object, publish_module, call_module, sync_client_state, get_own_objects, read_object in Client
3. add client sync support in authority (?)
4. figure out where to store and download object_bytes.

These are written based on my understanding of the codebase, please do correct me if there are anything doesn't look right to you.

[gdanezis]

That is very helpful @patrickkuo many thanks.

On 1. I think it is interesting to offer an 'uncontrolled' issuance of objects (but charge gas fees for this). This would allow people to issue 'static' objects, that may only be transferred. Its a nice 'hello world' application of NFTs. Maybe we can add some provenance to them -- ie keep track of the creator, besides the owner. But probably not much else. These are the objects we can also transfer with 2. without triggering a move call?

However, for most objects with move logic I foresee a controlled issuance. In this case a game dev uploads a move contract. One or more of the methods of the move contract allows people to issue NFTs (in a controlled way), and these NFTs can be mutated, through further calls to the contract. Composition means contracts call contracts etc. This is the 'publish_module' and 'call_module' story.

How do we handle init object?

What is the init object?

I believe the object_id will need to be generated from the object_bytes, but do we need to send it to the authorities to verify?

Ah, creating an object needs to involve another mutable object (the gas fees object is enough), and then the system / anyone can derive the new object_id cryptographically. This is deterministic, and designed to have uniqueness properties, so it cannot be arbitrary or provided by the client.

In practice this means we have a 'Issue(gas_object, data) -> new_nft_object' call.

Sending over the object_bytes will probably impact performance?

Maybe we want to charge gas according to the data stored, maybe we want to initially put a limit? I vote for making people pay for bytes stored.

Or do we trust client will do the right thing?

We, as a rule, do not when it impacts the security of others :)

[sblackshear]

Thanks for kicking off this discussion + great thoughts! Let me share what I had in mind for the first story ("publish an object"), since it's related to the programmability story.

As a user, I want to publish an object to FastX

1. The user first writes a Move module MyObject defining the schema of the object (via a Move struct), the creation API (e.g., maybe you have to pay $10 to create a MyObject), the operations allowed on it (e.g., updates to fields), etc. A simple example might be:

module FastX::Natural {
  /// An object wrapping a natural number
  struct Natural { id: ID, n: u64 }

  public fun publish(n: u64, ctx: &mut TxContext) {
    let natural = Natural { id: TxContext::new_id(ctx), n };
    Transfer::transfer(n, TxContext::sender(ctx))
  }
}

, and a more complex one might be like https://github.com/MystenLabs/fastnft/blob/main/fastx_programmability/framework/examples/CombinableObjects.move.

2. Alternatively (and this is probably more common), the use selects a Move module MyObject that's already published on-chain (e.g., a module that mimics ERC721 NFT's)

3. The user sends a Call order invoking MyObject::publish with the appropriate arguments (e.g., n for our Natural example), then shares it with enough authorities to create a certificate.

4. The user shares the certificate with an authority. The processing logic runs the call, which creates a Natural with a fresh object ID, then tranfers is to the transaction sender. The postprocessing of the call here creates a Rust Object::Move wrapping the Move value, and assigns its owner field to the transaction sender.

5. The authority returns the object ID(s) created or mutated via executing the certificate to the user.

6. If the user wants to see the object itself, they can use the "download object" flow you describe applied to the object ID's from the previous step.

How do we handle init object? I believe the object_id will need to be generated from the object_bytes, but do we need to send it to the authorities to verify?

I think the flow above covers this (but let me know if I'm wrong)--TxContext knows how to generate fresh ID's using the hash of the current transaction.

[sblackshear]

On 1. I think it is interesting to offer an 'uncontrolled' issuance of objects (but charge gas fees for this). This would allow people to issue 'static' objects, that may only be transferred. Its a nice 'hello world' application of NFTs. Maybe we can add some provenance to them -- ie keep track of the creator, besides the owner. But probably not much else. These are the objects we can also transfer with 2. without triggering a move call?

I think this is a very interesting and important question. My take on this: everything should be a Move object, but we should expose "native" order types for key operations like Issue, Transfer, and BatchIssue implemented in pure Rust (i.e., they don't involve a Move call) that can create and operate on certain Move types only (e.g., fungible tokens, ERC721-compatible NFT's). The restriction would would work by asserting that the Object type_ is an allowed one.

My reasoning:

• This allows common operations to be fast, which is crucially important
• However, we still get programmability on objects created/transferred through the fast API's--they're on equal footing with objects created via Move calls. It would be unfortunate if you have to choose between your NFT's being fast/cheap and being programmable--ideally, I should be able to cheaply airdrop 10K NFT's in a single tx via BatchIssue, but then let the recipients use them in a game that requires programmability, put them up for sale, atomically swap them, etc.
• This ensures that the client story for "how do I interpret the object bytes handed to me by read_object" is the same for all object types. If you let a user publish raw bytes, there's no canonical way for other users to find the schema that lets them interpret these bytes. But Move objects all carry their schema in the type_ field, which can be used to deserialize the object, display it in your favorite tree-structured format (YAML, JSON, ...), etc. Also, we have tools for generating object bindings from Move type definitions in Rust, JS, Python, Java, and a few other langs which clients can take advantage of in writing application code that manipulates their objects.

[sblackshear]

Maybe we want to charge gas according to the data stored, maybe we want to initially put a limit? I vote for making people pay for bytes stored.

Fully agree with this (charging an a stringent initial limit), both for DDOS protection and (perhaps even more so) for content moderation reasons. I'm very concerned about folks placing illegal and copyrighted content en masse on FastNFT, which could place the platform under a lot of scrutiny + direct legal action toward the logical place (us).

Of course, IPFS and other platforms already have to deal with this problem a bit (e.g., https://ipfs.io/legal/), so we can/should understand and mimic their approaches. Dfinity also ran into this recently--someone uploaded a an emulator of a Mario game and Nintendo sent them an infringement notice. What happened there + the follow-up is interesting (although the user made it easy for them by voluntarily taking down the offending code).

However, I suspect that this isn't that big of a problem on these platforms only because:

• it's really expensive to store/compute anything
• the platforms are not at all popular compared to any conventional internet service

If we solve either or both of these problems, I think we'll have to be much more clever in our approach to decentralized content management. Would love to be able to punt on this problem in the short term via stringent storage limits (or other hacks) if we can...

[sblackshear]

We clearly need to re-think 'transfer_to_fastpay' and 'receive_from_fastpay' what are these meant to do (there is no more fastpay now). Do we use them to init objects?

I think it is probably safe to get rid of these

[sblackshear]

User can pass on the certificate to another user off-platform(?).

I think this is one way to solve the problem, but we should strive for an approach that does not require pairwise communication between users (since establishing such channels is a challenging problem, particularly between clients that may go offline frequently). One idea that has been kicked around (though is unfortunately not written up in detail anywhere yet) is an event system with topics and a conventional pub/sub architecture. A client with address A would (e.g) subscribe to (e.g.) to transfer events involving A (or maybe just all events involving A), and thus find out about objects/payments received from the pub/sub system instead of directly from the sender.

A beginning of this idea was implemented in George's fastnft-exec prototype, but there's still a lot to be done in designing the event system itself from the programmability, authority persistent store, and pub/sub message bus perspectives. Ideas definitely welcome!

[gdanezis]

Also, here are some very old notes about client interfaces:
https://github.com/MystenLabs/fastnftexec-experimental/wiki/Client-Interfaces-for-fastNFT

One thing that I have not seen discussed is (1) streaming interfaces where users can 'register' and receive updates, which also relate to (2) events generated by fastnft contracts (we do not have these quite yet).

[oxade]

@sblackshear @gdanezis I like the idea of an eventing/streaming system (very exciting). But curious how important is this feature right now? Should we dig deeper?

[gdanezis]

My intuition is: lets get an end-to-end flow to work first, then we handle streaming / event indexing. @sblackshear ?

[sblackshear]

My intuition is: lets get an end-to-end flow to work first, then we handle streaming / event indexing. @sblackshear ?

100% agree!