diff --git a/README.md b/README.md index ecdaf7baa..2ac517436 100644 --- a/README.md +++ b/README.md @@ -10,28 +10,29 @@ Changes to the protocol specification and standards are called NEAR Enhancement ## NEPs -| NEP # | Title | Author | Status | -| ----------------------------------------------------------------- |------------------------------------------|---------------------------------------------|--------| -| [0001](https://github.com/near/NEPs/blob/master/neps/nep-0001.md) | NEP Purpose and Guidelines | @jlogelin | Living | -| [0021](https://github.com/near/NEPs/blob/master/neps/nep-0021.md) | Fungible Token Standard (Deprecated) | @evgenykuzyakov | Final | -| [0141](https://github.com/near/NEPs/blob/master/neps/nep-0141.md) | Fungible Token Standard | @evgenykuzyakov @oysterpack | Final | -| [0145](https://github.com/near/NEPs/blob/master/neps/nep-0145.md) | Storage Management | @evgenykuzyakov | Final | -| [0148](https://github.com/near/NEPs/blob/master/neps/nep-0148.md) | Fungible Token Metadata | @robert-zaremba @evgenykuzyakov @oysterpack | Final | -| [0171](https://github.com/near/NEPs/blob/master/neps/nep-0171.md) | Non Fungible Token Standard | @mikedotexe @evgenykuzyakov @oysterpack | Final | -| [0177](https://github.com/near/NEPs/blob/master/neps/nep-0177.md) | Non Fungible Token Metadata | @chadoh @mikedotexe | Final | -| [0178](https://github.com/near/NEPs/blob/master/neps/nep-0178.md) | Non Fungible Token Approval Management | @chadoh @thor314 | Final | -| [0181](https://github.com/near/NEPs/blob/master/neps/nep-0181.md) | Non Fungible Token Enumeration | @chadoh @thor314 | Final | -| [0199](https://github.com/near/NEPs/blob/master/neps/nep-0199.md) | Non Fungible Token Royalties and Payouts | @thor314 @mattlockyer | Final | -| [0245](https://github.com/near/NEPs/blob/master/neps/nep-0245.md) | Multi Token Standard | @zcstarr @riqi @jriemann @marcos.sun | Review | -| [0264](https://github.com/near/NEPs/blob/master/neps/nep-0264.md) | Promise Gas Weights | @austinabell | Final | -| [0297](https://github.com/near/NEPs/blob/master/neps/nep-0297.md) | Events Standard | @telezhnaya | Final | -| [0330](https://github.com/near/NEPs/blob/master/neps/nep-0330.md) | Source Metadata | @BenKurrek | Review | -| [0366](https://github.com/near/NEPs/blob/master/neps/nep-0366.md) | Meta Transactions | @ilblackdragon @e-uleyskiy @fadeevab | Draft | -| [0399](https://github.com/near/NEPs/blob/master/neps/nep-0399.md) | Flat Storage | @Longarithm @mzhangmzz | Review | -| [0448](https://github.com/near/NEPs/blob/master/neps/nep-0448.md) | Zero-balance Accounts | @bowenwang1996 | Final | -| [0455](https://github.com/near/NEPs/blob/master/neps/nep-0455.md) | Parameter Compute Costs | @akashin @jakmeier | Final | -| [0514](https://github.com/near/NEPs/blob/master/neps/nep-0514.md) | Fewer Block Producer Seats in `testnet` | @nikurt | Final | - +| NEP # | Title | Author | Status | +| ----------------------------------------------------------------- | ---------------------------------------- | -------------------------------------------- | ---------- | +| [0001](https://github.com/near/NEPs/blob/master/neps/nep-0001.md) | NEP Purpose and Guidelines | @jlogelin | Living | +| [0021](https://github.com/near/NEPs/blob/master/neps/nep-0021.md) | Fungible Token Standard (Deprecated) | @evgenykuzyakov | Deprecated | +| [0141](https://github.com/near/NEPs/blob/master/neps/nep-0141.md) | Fungible Token Standard | @evgenykuzyakov @oysterpack, @robert-zaremba | Final | +| [0145](https://github.com/near/NEPs/blob/master/neps/nep-0145.md) | Storage Management | @evgenykuzyakov | Final | +| [0148](https://github.com/near/NEPs/blob/master/neps/nep-0148.md) | Fungible Token Metadata | @robert-zaremba @evgenykuzyakov @oysterpack | Final | +| [0171](https://github.com/near/NEPs/blob/master/neps/nep-0171.md) | Non Fungible Token Standard | @mikedotexe @evgenykuzyakov @oysterpack | Final | +| [0177](https://github.com/near/NEPs/blob/master/neps/nep-0177.md) | Non Fungible Token Metadata | @chadoh @mikedotexe | Final | +| [0178](https://github.com/near/NEPs/blob/master/neps/nep-0178.md) | Non Fungible Token Approval Management | @chadoh @thor314 | Final | +| [0181](https://github.com/near/NEPs/blob/master/neps/nep-0181.md) | Non Fungible Token Enumeration | @chadoh @thor314 | Final | +| [0199](https://github.com/near/NEPs/blob/master/neps/nep-0199.md) | Non Fungible Token Royalties and Payouts | @thor314 @mattlockyer | Final | +| [0245](https://github.com/near/NEPs/blob/master/neps/nep-0245.md) | Multi Token Standard | @zcstarr @riqi @jriemann @marcos.sun | Review | +| [0264](https://github.com/near/NEPs/blob/master/neps/nep-0264.md) | Promise Gas Weights | @austinabell | Final | +| [0297](https://github.com/near/NEPs/blob/master/neps/nep-0297.md) | Events Standard | @telezhnaya | Final | +| [0330](https://github.com/near/NEPs/blob/master/neps/nep-0330.md) | Source Metadata | @BenKurrek | Review | +| [0366](https://github.com/near/NEPs/blob/master/neps/nep-0366.md) | Meta Transactions | @ilblackdragon @e-uleyskiy @fadeevab | Final | +| [0393](https://github.com/near/NEPs/blob/master/neps/nep-0393.md) | Sould Bound Token (SBT) | @robert-zaremba | Final | +| [0399](https://github.com/near/NEPs/blob/master/neps/nep-0399.md) | Flat Storage | @Longarithm @mzhangmzz | Final | +| [0448](https://github.com/near/NEPs/blob/master/neps/nep-0448.md) | Zero-balance Accounts | @bowenwang1996 | Final | +| [0452](https://github.com/near/NEPs/blob/master/neps/nep-0452.md) | Linkdrop Standard | @benkurrek @miyachi | Final | +| [0455](https://github.com/near/NEPs/blob/master/neps/nep-0455.md) | Parameter Compute Costs | @akashin @jakmeier | Final | +| [0514](https://github.com/near/NEPs/blob/master/neps/nep-0514.md) | Fewer Block Producer Seats in `testnet` | @nikurt | Final | ## Specification diff --git a/neps/nep-0330.md b/neps/nep-0330.md index 0d56e4018..6615165c4 100644 --- a/neps/nep-0330.md +++ b/neps/nep-0330.md @@ -1,25 +1,29 @@ --- NEP: 330 Title: Source Metadata -Author: Ben Kurrek , Osman Abdelnasir +Author: Ben Kurrek , Osman Abdelnasir , Andrey Gruzdev <@canvi>, Alexey Zenin <@alexthebuildr> DiscussionsTo: https://github.com/near/NEPs/discussions/329 Status: Approved Type: Standards Track Category: Contract -Version: 1.1.0 +Version: 1.2.0 Created: 27-Feb-2022 -Updated: 13-Jan-2023 +Updated: 19-Feb-2023 --- ## Summary -The contract source metadata is a standard interface that allows auditing and viewing source code for a deployed smart contract. Implementation of this standard is purely optional but is recommended for developers whose contracts are open source. +The contract source metadata represents a standardized interface designed to facilitate the auditing and inspection of source code associated with a deployed smart contract. Adoption of this standard remains discretionary; however, it is strongly advocated for developers who maintain an open-source approach to their contracts. This initiative promotes greater accountability and transparency within the ecosystem, encouraging best practices in contract development and deployment. ## Motivation -There is no trivial way of finding the source code or author of a deployed smart contract. Having a standard that outlines how to view the source code of an arbitrary smart contract creates an environment of openness and collaboration. +The incorporation of metadata facilitates the discovery and validation of deployed source code, thereby significantly reducing the requisite level of trust during code integration or interaction processes. -Additionally, we would like for wallets and dApps to be able to parse this information and determine which methods they are able to call and render UIs that provide that functionality. +The absence of an accepted protocol for identifying the source code or author contact details of a deployed smart contract presents a challenge. Establishing a standardized framework for accessing the source code of any given smart contract would foster a culture of transparency and collaborative engagement. + +Moreover, the current landscape does not offer a straightforward mechanism to verify the authenticity of a smart contract's deployed source code against its deployed version. To address this issue, it is imperative that metadata includes specific details that enable contract verification through reproducible builds. + +Furthermore, it is desirable for users and dApps to possess the capability to interpret this metadata, thereby identifying executable methods and generating UIs that facilitate such functionalities. This also extends to acquiring comprehensive insights into potential future modifications by the contract or its developers, enhancing overall system transparency and user trust. The initial discussion can be found [here](https://github.com/near/NEPs/discussions/329). @@ -31,22 +35,31 @@ There is a lot of information that can be held about a contract. Ultimately, we Successful implementations of this standard will introduce a new (`ContractSourceMetadata`) struct that will hold all the necessary information to be queried for. This struct will be kept on the contract level. -The metadata will include three optional fields: +The metadata will include optional fields: -- `version`: a string that references the specific commit hash or version of the code that is currently deployed on-chain. This can be included regardless of whether or not the contract is open-sourced and can also be used for organizational purposes. -- `link`: a string that references the link to the open-source code. This can be anything such as Github or a CID to somewhere on IPFS. +- `version`: a string that references the specific commit ID or a tag of the code currently deployed on-chain. Examples: `"v0.8.1"`, `"a80bc29"`. +- `link`: an URL to the currently deployed code. It must include version or a tag if using a GitHub or a GitLab link. Examples: "https://github.com/near/near-cli-rs/releases/tag/v0.8.1", "https://github.com/near/cargo-near-new-project-template/tree/9c16aaff3c0fe5bda4d8ffb418c4bb2b535eb420" or an IPFS CID. - `standards`: a list of objects (see type definition below) that enumerates the NEPs supported by the contract. If this extension is supported, it is advised to also include NEP-330 version 1.1.0 in the list (`{standard: "nep330", version: "1.1.0"}`). +- `build_info`: a build details object (see type definition below) that contains all the necessary information about how the contract was built, making it possible for others to reproduce the same WASM of this contract. ```ts type ContractSourceMetadata = { - version: string|null, // optional, commit hash being used for the currently deployed wasm. If the contract is not open-sourced, this could also be a numbering system for internal organization / tracking such as "1.0.0" and "2.1.0". - link: string|null, // optional, link to open source code such as a Github repository or a CID to somewhere on IPFS. - standards: Standard[]|null, // optional, standards and extensions implemented in the currently deployed wasm e.g. [{standard: "nep330", version: "1.1.0"},{standard: "nep141", version: "1.0.0"}]. + version: string|null, // optional, commit hash being used for the currently deployed WASM. If the contract is not open-sourced, this could also be a numbering system for internal organization / tracking such as "1.0.0" and "2.1.0". + link: string|null, // optional, link to open source code such as a Github repository or a CID to somewhere on IPFS, e.g., "https://github.com/near/cargo-near-new-project-template/tree/9c16aaff3c0fe5bda4d8ffb418c4bb2b535eb420" + standards: Standard[]|null, // optional, standards and extensions implemented in the currently deployed WASM, e.g., [{standard: "nep330", version: "1.1.0"},{standard: "nep141", version: "1.0.0"}]. + build_info: BuildInfo|null, // optional, details that are required for contract WASM reproducibility. } type Standard { - standard: string, // standard name e.g. "nep141" - version: string, // semantic version number of the Standard e.g. "1.0.0" + standard: string, // standard name, e.g., "nep141" + version: string, // semantic version number of the Standard, e.g., "1.0.0" +} + +type BuildInfo { + build_environment: string, // reference to a reproducible build environment docker image, e.g., "docker.io/sourcescan/cargo-near@sha256:bf488476d9c4e49e36862bbdef2c595f88d34a295fd551cc65dc291553849471" or something else pointing to the build environment. + source_code_snapshot: string, // reference to the source code snapshot that was used to build the contract, e.g., "git+https://github.com/near/cargo-near-new-project-template.git#9c16aaff3c0fe5bda4d8ffb418c4bb2b535eb420" or "ipfs://". + contract_path: string|null, // relative path to contract crate within the source code, e.g., "contracts/contract-one". Often, it is the root of the repository, so can be omitted. + build_command: string[], // the exact command that was used to build the contract, with all the flags, e.g., ["cargo", "near", "build", "--no-abi"]. } ``` @@ -56,54 +69,70 @@ In order to view this information, contracts must include a getter which will re function contract_source_metadata(): ContractSourceMetadata {} ``` +### Ensuring WASM Reproducibility + +#### Build Environment Docker Image + +When using a Docker image as a reference, it's important to specify the digest of the image to ensure reproducibility, since a tag could be reassigned to a different image. + +### Paths Inside Docker Image + +During the build, paths from the source of the build as well as the location of the cargo registry could be saved into WASM, which affects reproducibility. Therefore, we need to ensure that everyone uses the same paths inside the Docker image. We propose using the following paths: + +- `/home/near/code` - Mounting volume from the host system containing the source code. +- `/home/near/.cargo` - Cargo registry. + +#### Cargo.lock + +It is important to have `Cargo.lock` inside the source code snapshot to ensure reproducibility. Example: https://github.com/near/core-contracts. + ## Reference Implementation -As an example, say there was an NFT contract deployed on-chain which was currently using the commit hash `39f2d2646f2f60e18ab53337501370dc02a5661c` and had its open source code located at `https://github.com/near-examples/nft-tutorial`. This contract would then include a struct which has the following fields: +As an example, consider a contract located at the root path of the repository, which was deployed using the `cargo near deploy --no-abi` and environment docker image `sourcescan/cargo-near@sha256:bf488476d9c4e49e36862bbdef2c595f88d34a295fd551cc65dc291553849471`. Its latest commit hash is `9c16aaff3c0fe5bda4d8ffb418c4bb2b535eb420`, and its open-source code can be found at `https://github.com/near/cargo-near-new-project-template`. This contract would then include a struct with the following fields: ```ts type ContractSourceMetadata = { - version: "39f2d2646f2f60e18ab53337501370dc02a5661c" - link: "https://github.com/near-examples/nft-tutorial", - standards: [ - { - standard: "nep330", - version: "1.1.0" - }, - { - standard: "nep171", - version: "1.0.0" - }, - { - standard: "nep177", - version: "2.0.0" + version: "1.0.0", + link: "https://github.com/near/cargo-near-new-project-template/tree/9c16aaff3c0fe5bda4d8ffb418c4bb2b535eb420", + standards: [ + { + standard: "nep330", + version: "1.1.0" + } + ], + build_info: { + build_environment: "docker.io/sourcescan/cargo-near@sha256:bf488476d9c4e49e36862bbdef2c595f88d34a295fd551cc65dc291553849471", + source_code_snapshot: "git+https://github.com/near/cargo-near-new-project-template.git#9c16aaff3c0fe5bda4d8ffb418c4bb2b535eb420", + contract_path: ".", + build_command: ["cargo", "near", "deploy", "--no-abi"] } - ] } + ``` -If someone were to call the view function `contract_metadata`, the contract would return: +Calling the view function `contract_source_metadata`, the contract would return: ```bash { - version: "39f2d2646f2f60e18ab53337501370dc02a5661c" - link: "https://github.com/near-examples/nft-tutorial", + version: "1.0.0" + link: "https://github.com/near/cargo-near-new-project-template/tree/9c16aaff3c0fe5bda4d8ffb418c4bb2b535eb420", standards: [ { standard: "nep330", version: "1.1.0" - }, - { - standard: "nep171", - version: "1.0.0" - }, - { - standard: "nep177", - version: "2.0.0" } - ] + ], + build_info: { + build_environment: "docker.io/sourcescan/cargo-near@sha256:bf488476d9c4e49e36862bbdef2c595f88d34a295fd551cc65dc291553849471", + source_code_snapshot: "git+https://github.com/near/cargo-near-new-project-template.git#9c16aaff3c0fe5bda4d8ffb418c4bb2b535eb420", + contract_path: ".", + build_command: ["cargo", "near", "deploy", "--no-abi"] + } } ``` +This could be used by SourceScan to reproduce the same WASM using the build details and to verify the on-chain WASM code with the reproduced one. + An example implementation can be seen below. ```rust @@ -113,17 +142,26 @@ pub struct Contract { pub contract_metadata: ContractSourceMetadata } -// Standard structure +/// NEP supported by the contract. pub struct Standard { pub standard: String, pub version: String } +/// BuildInfo structure +pub struct BuildInfo { + pub build_environment: String, + pub source_code_snapshot: String, + pub contract_path: Option, + pub build_command: Vec, +} + /// Contract metadata structure pub struct ContractSourceMetadata { - pub version: String, - pub link: String, - pub standards: Vec + pub version: Option, + pub link: Option, + pub standards: Option>, + pub build_info: Option, } /// Minimum Viable Interface @@ -168,6 +206,13 @@ The extension NEP-351 that added Contract Metadata to this NEP-330 was approved | 2 | NEP-330 and NEP-351 should be included in the list of the supported NEPs | There seems to be a general agreement that it is a good default, so NEP was updated | Resolved | | 3 | JSON Event could be beneficial, so tooling can react to the changes in the supported standards | It is outside the scope of this NEP. Also, list of supported standards only changes with contract re-deployment, so tooling can track DEPLOY_CODE events and check the list of supported standards when new code is deployed | Won’t fix | +### 1.2.0 - Build Details Extension + +The NEP extension adds build details to the contract metadata, containing necessary information about how the contract was built. This makes it possible for others to reproduce the same WASM of this contract. The idea first appeared in the [cargo-near SourceScan integration thread](https://github.com/near/cargo-near/issues/131). + +#### Benefits + +- This NEP extension gives developers the capability to save all the required build details, making it possible to reproduce the same WASM code in the future. This ensures greater consistency in contracts and the ability to verify source code. With the assistance of tools like SourceScan and cargo-near, the development process on NEAR becomes significantly easier ## Copyright diff --git a/neps/nep-0393.md b/neps/nep-0393.md index 9c10b0469..567a3ebe6 100644 --- a/neps/nep-0393.md +++ b/neps/nep-0393.md @@ -269,8 +269,9 @@ The Soulbound Token follows the NFT [NEP-171](https://github.com/near/NEPs/blob/ All time related attributes are defined in milliseconds (as per NEP-171). ```rust -/// ContractMetadata defines contract wide attributes, which describes the whole contract. -pub struct ContractMetadata { +/// IssuerMetadata defines contract wide attributes, which describes the whole contract. +/// Must be provided by the Issuer contract. See the `SBTIssuer` trait. +pub struct IssuerMetadata { /// Version with namespace, example: "sbt-1.0.0". Required. pub spec: String, /// Issuer Name, required, ex. "Mosaics" @@ -290,6 +291,25 @@ pub struct ContractMetadata { pub reference_hash: Option, } +/// ClassMetadata defines SBT class wide attributes, which are shared and default to all SBTs of +/// the given class. Must be provided by the Issuer contract. See the `SBTIssuer` trait. +pub struct ClassMetadata { + /// Issuer class name. Required to be not empty. + pub name: String, + /// If defined, should be used instead of `IssuerMetadata::symbol`. + pub symbol: Option, + /// An URL to an Icon. To protect fellow developers from unintentionally triggering any + /// SSRF vulnerabilities with URL parsers, we don't allow to set an image bytes here. + /// If it doesn't start with a scheme (eg: https://) then `IssuerMetadata::base_uri` + /// should be prepended. + pub icon: Option, + /// JSON or an URL to a JSON file with more info. If it doesn't start with a scheme + /// (eg: https://) then base_uri should be prepended. + pub reference: Option, + /// Base64-encoded sha256 hash of JSON from reference field. Required if `reference` is included. + pub reference_hash: Option, +} + /// TokenMetadata defines attributes for each SBT token. pub struct TokenMetadata { pub class: ClassId, // token class. Required. Must be non zero. @@ -448,12 +468,18 @@ Example **Soul Transfer** interface: ### SBT Issuer interface -SBTContract is the minimum required interface to be implemented by issuer. Other methods, such as a mint function, which requests the registry to proceed with token minting, is specific to an Issuer implementation (similarly, mint is not part of the FT standard). +SBTIssuer is the minimum required interface to be implemented by issuer. Other methods, such as a mint function, which requests the registry to proceed with token minting, is specific to an Issuer implementation (similarly, mint is not part of the FT standard). + +The issuer must provide metadata object of the Issuer. Optionally, Issuer can also provide metadata object for each token class. +Issuer level (contract) metadata, must provide information common to all tokens and all classes defined by the issuer. Class level metadata, must provide information common to all tokens of a given class. Information should be deduplicated and denormalized whenever possible. +Example: The issuer can set a default icon for all tokens (SBT) using `IssuerMetadata::icon` and additionally it can customize an icon of a particular token via `TokenMetadata::icon`. ```rust -pub trait SBTContract { - /// returns contract metadata - fn sbt_metadata(&self) -> ContractMetadata; +pub trait SBTIssuer { + /// Returns contract metadata. + fn sbt_metadata(&self) -> IssuerMetadata; + /// Returns SBT class metadata, or `None` if the class is not found. + fn sbt_class_metadata(&self, class: ClassId) -> Option; } ``` @@ -594,7 +620,7 @@ trait SBT { ## Reference Implementation -- Common [type definitions](https://github.com/near-ndc/i-am-human/tree/main/contracts/sbt) (events, traits). +- Common [type definitions](https://github.com/near-ndc/i-am-human/tree/master/contracts/sbt) (events, traits). - [I Am Human](https://github.com/near-ndc/i-am-human) registry and issuers. ## Consequences @@ -667,6 +693,24 @@ The Contract Standards Working Group members approved this NEP on June 30, 2023 | 9 | [Privacy](https://github.com/near/NEPs/pull/393/#issuecomment-1504309947) | Concerns have been addressed: [comment-1](https://github.com/near/NEPs/pull/393/#issuecomment-1504485420) and [comment2](https://github.com/near/NEPs/pull/393/#issuecomment-1505958549) | resolved | | 10 | @frol [suggested](https://github.com/near/NEPs/pull/393/#discussion_r1247879778) to use a struct in `sbt_recover` and `sbt_soul_transfer`. | Motivation to use pair `(number, bool)` rather than follow a common Iterator Pattern. Rust uses `Option` type for that, that works perfectly for languages with native Option type, but creates a "null" problem for anything else. Other common way to implement Iterator is the presented pair, which doesn't require extra type definition and reduces code size. | new | + +### v1.1.0 + + +In v1.0.0 we defined Issuer (an entity authorized to mint SBTs in the registry) and SBT Class. We also defined Issuer Metadata and Token Metadata, but we didn't provide interface for class metadata. This was implemented in the reference implementation (in one of the subsequent revisions), but was not backported to the NEP. This update: + +- Fixes the name of the issuer interface from `SBTContract` to `SBTIssuer`. The original name is wrong and we oversight it in reviews. We talk everywhere about the issuer entity and issuer contract (even the header is SBT Issuer interface). +- Renames `ContractMetadata` to `IssuerMetadata`. +- Adds `ClassMetadata` struct and `sbt_class_metadata` function to the `SBTIssuer` interface. + +Reference implementation: [ContractMetadata, ClassMetadata, TokenMetadata](https://github.com/near-ndc/i-am-human/blob/registry/v1.8.0/contracts/sbt/src/metadata.rs#L18) and [SBTIssuer interface](https://github.com/near-ndc/i-am-human/blob/registry/v1.8.0/contracts/sbt/src/lib.rs#L49). + +#### Benefits + +- Improves the documentation and meaning of the issuer entity. +- Adds missing `ClassMetadata`. +- Improves issuer, class and token metadata documentation. + ## Copyright [Creative Commons Attribution 4.0 International Public License (CC BY 4.0)](https://creativecommons.org/licenses/by/4.0/) diff --git a/neps/nep-0413.md b/neps/nep-0413.md index 54da3906e..ef6230fc5 100644 --- a/neps/nep-0413.md +++ b/neps/nep-0413.md @@ -159,7 +159,7 @@ Non-web Wallets, such as [Ledger](https://www.ledger.com) can directly return th ## References -A full example on how to implement the `signMessage` method can be [found here](https://github.com/gagdiez/near-login/blob/main/tests/authentication/auth.ava.ts#L27-#L65). +A full example on how to implement the `signMessage` method can be [found here](https://github.com/gagdiez/near-login/blob/1650e25080ab2e8a8c508638a9ba9e9732e76036/server/tests/wallet.ts#L60-L77). ## Drawbacks diff --git a/neps/nep-0452.md b/neps/nep-0452.md new file mode 100644 index 000000000..c5df9f258 --- /dev/null +++ b/neps/nep-0452.md @@ -0,0 +1,243 @@ +--- +NEP: 452 +Title: Linkdrop Standard +Author: Ben Kurrek , Ken Miyachi +DiscussionsTo: https://gov.near.org/t/official-linkdrop-standard/32463/1 +Status: Final +Type: Standards Track +Category: Contract +Version: 1.0.0 +Created: 24-Jan-2023 +Updated: 19-Apr-2023 +--- + +## Summary + +A standard interface for linkdrops that support $NEAR, fungible tokens, non-fungible tokens, and is extensible to support new types in the future. + +Linkdrops are a simple way to send assets to someone by providing them with a link. This link can be embedded into a QR code, sent via email, text or any other means. Within the link, there is a private key that allows the holder to call a method that can create an account and send it assets. Alternatively, if the holder has an account, the assets can be sent there as well. + +By definition, anyone with an access key can interact with the blockchain and since there is a private key embedded in the link, this removes the need for the end-user to have a wallet. + +## Motivation + +Linkdrops are an extremely powerful tool that enable seamless onboarding and instant crypto experiences with the click of a link. The original [near-linkdrop](https://github.com/near/near-linkdrop) contract provides a minimal interface allowing users to embed $NEAR within an access key and create a simple Web2 style link that can then be used as a means of onboarding. This simple $NEAR linkdrop is not enough as many artists, developers, event coordinators, and applications want to drop more digital assets such as NFTs, FTs, tickets etc. + +As linkdrop implementations start to push the boundaries of what’s possible, new data structures, methods, and interfaces are being developed. There needs to be a standard data model and interface put into place to ensure assets can be claimed independent of the contract they came from. If not, integrating any application with linkdrops will require customized solutions, which would become cumbersome for the developer and deteriorate the user onboarding experience. The linkdrop standard addresses these issues by providing a simple and extensible standard data model and interface. + +The initial discussion can be found [here](https://gov.near.org/t/official-linkdrop-standard/32463/1). + +## Specification + +### Example Scenarios + +_Pre-requisite Steps_: Linkdrop creation: +The linkdrop creator that has an account with some $NEAR: + +- creates a keypair locally (`pubKey1`, `privKey1`). (The keypair is not written to chain at this time) +- calls a method on a contract that implements the linkdrop standard in order to create the drop. The `pubKey1` and desired $NEAR amount are both passed in as arguments. +- The contract maps the `pubKey1` to the desired balance for the linkdrop (`KeyInfo` record). +- The contract then adds the `pubKey1` as a function call access key with the ability to call `claim` and `create_account_and_claim`. This means that anyone with the `privKey1` (see above), can sign a transaction on behalf of the contract (signer id set to contract id) with a function call to call one of the mentioned functions to claim the assets. + +#### Claiming a linkdrop without a NEAR Account + +A user with _no_ account can claim the assets associated with an existing public key, already registered in the linkdrop contract: + +- generates a new keypair (`pubKey2`, `privKey2`) locally. (This new keypair is not written to chain) +- chooses a new account ID such as benji.near. +- calls `create_account_and_claim`. The transaction is signed on behalf of the linkdrop contract (`signer_id` is set to the contract address) using `privKey1`. + - the args of this function call will contain both `pubKey2` (which will be used to create a full access key for the new account) and the account ID itself. + - the linkdrop contract will delete the access key associated with `pubKey1` so that it cannot be used again. + - the linkdrop contract will create the new account and transfer the funds to it alongside any other assets. +- the user will be able to sign transactions on behalf of the new account using `privKey2`. + +#### Claiming a linkdrop with a NEAR Account + +A user with an _existing_ account can claim the assets with an existing public key, already registered in the linkdrop contract: + +- calls `claim`. The transaction is signed on behalf of the linkdrop contract (`signer_id` is set to the contract address) using `privKey1`. + - the args of this function call will simply contain the user's existing account ID. + - the linkdrop contract will delete the access key associated with `pubKey1` so that it cannot be used again. + - the linkdrop contract will transfer the funds to that account alongside any other assets. + +```ts +/// Information about a specific public key. +type KeyInfo = { + /// How much Gas should be attached when the key is used to call `claim` or `create_account_and_claim`. + /// It is up to the smart contract developer to calculate the required gas (which can be done either automatically on the contract or on the client-side). + required_gas: string, + + /// yoctoNEAR$ amount that will be sent to the account that claims the linkdrop (either new or existing) + /// when the key is successfully used. + yoctonear: string, + + /// If using the NFT standard extension, a set of NFTData can be linked to the public key + /// indicating that all those assets will be sent to the account that claims the linkdrop (either new or + /// existing) when the key is successfully used. + nft_list: NFTData[] | null, + + /// If using the FT standard extension, a set of FTData can be linked to the public key + /// indicating that all those assets will be sent to the account that claims the linkdrop (either new or + /// existing) when the key is successfully used. + ft_list: FTData[] | null + + /// ... other types can be introduced and the standard is easily extendable. +} + + +/// Data outlining a specific Non-Fungible Token that should be sent to the claiming account +/// (either new or existing) when a key is successfully used. +type NFTData = { + /// the id of the token to transfer + token_id: string, + + /// The valid NEAR account indicating the Non-Fungible Token contract. + contract_id: string +} + + +/// Data outlining Fungible Tokens that should be sent to the claiming account +/// (either new or existing) when a key is successfully used. +type FTData = { + /// The number of tokens to transfer, wrapped in quotes and treated + /// like a string, although the number will be stored as an unsigned integer + /// with 128 bits. + amount: string, + + /// The valid NEAR account indicating the Fungible Token contract. + contract_id: string +} + +/****************/ +/* VIEW METHODS */ +/****************/ + +/// Allows you to query for the amount of $NEAR tokens contained in a linkdrop corresponding to a given public key. +/// +/// Requirements: +/// * Panics if the key does not exist. +/// +/// Arguments: +/// * `key` the public counterpart of the key used to sign, expressed as a string with format ":" (e.g. "ed25519:6TupyNrcHGTt5XRLmHTc2KGaiSbjhQi1KHtCXTgbcr4Y") +/// +/// Returns a string representing the $yoctoNEAR amount associated with a given public key +function get_key_balance(key: string) -> string; + +/// Allows you to query for the `KeyInfo` corresponding to a given public key. This method is preferred over `get_key_balance` as it provides more information about the key. +/// +/// Requirements: +/// * Panics if the key does not exist. +/// +/// Arguments: +/// * `key` the public counterpart of the key used to sign, expressed as a string with format ":" (e.g. "ed25519:6TupyNrcHGTt5XRLmHTc2KGaiSbjhQi1KHtCXTgbcr4Y") +/// +/// Returns `KeyInfo` associated with a given public key +function get_key_information(key: string) -> KeyInfo; + +/******************/ +/* CHANGE METHODS */ +/******************/ + +/// Transfer all assets linked to the signer’s public key to an `account_id`. +/// If the transfer fails for whatever reason, it is up to the smart contract developer to +/// choose what should happen. For example, the contract can choose to keep the assets +/// or send them back to the original linkdrop creator. +/// +/// Requirements: +/// * The predecessor account *MUST* be the current contract ID. +/// * The `account_id` MUST be an *initialized* NEAR account. +/// * The assets being sent *MUST* be associated with the signer’s public key. +/// * The assets *MUST* be sent to the `account_id` passed in. +/// +/// Arguments: +/// * `account_id` the account that should receive the linkdrop assets. +/// +/// Returns `true` if the claim was successful meaning all assets were sent to the `account_id`. +function claim(account_id: string) -> Promise; + +/// Creates a new NEAR account and transfers all assets linked to the signer’s public key to +/// the *newly created account*. If the transfer fails for whatever reason, it is up to the +/// smart contract developer to choose what should happen. For example, the contract can +/// choose to keep the assets or return them to the original linkdrop creator. +/// +/// Requirements: +/// * The predecessor account *MUST* be the current contract ID. +/// * The assets being sent *MUST* be associated with the signer’s public key. +/// * The assets *MUST* be sent to the `new_account_id` passed in. +/// * The newly created account *MUST* have a new access key added to its account (either +/// full or limited access) in the same receipt that the account was created in. +/// * The Public key must be in a binary format with base58 string serialization with human-readable curve. +/// The key types currently supported are secp256k1 and ed25519. Ed25519 public keys accepted are 32 bytes +/// and secp256k1 keys are the uncompressed 64 format. +/// +/// Arguments: +/// * `new_account_id`: the valid NEAR account which is being created and should +/// receive the linkdrop assets +/// * `new_public_key`: the valid public key that should be used for the access key added to the newly created account (serialized with borsh). +/// +/// Returns `true` if the claim was successful meaning the `new_account_id` was created and all assets were sent to it. +function create_account_and_claim(new_account_id: string, new_public_key: string) -> Promise; +``` + +## Reference Implementation + +Below are some references for linkdrop contracts. + +- [Link Drop Contract](https://github.com/near/near-linkdrop) + +- [Keypom Contract](https://github.com/keypom/keypom) + +## Security Implications + +1. Linkdrop Creation + Linkdrop creation involves creating keypairs that, when used, have access to assets such as $NEAR, FTs, NFTs, etc. These keys should be limited access and restricted to specific functionality. For example, they should only have permission to call `claim` and `create_account_and_claim`. Since the keys allow the holder to sign transactions on behalf of the linkdrop contract, without the proper security measures, they could be used in a malicious manner (for example executing private methods or owner-only functions). + + Another important security implication of linkdrop creation is to ensure that only one key is mapped to a set of assets at any given time. Externally, assets such as FTs, and NFTs belong to the overall linkdrop contract account rather than a specific access key. It is important to ensure that specific keys can only claim assets that they are mapped to. + +2. Linkdrop Key Management + Key management is a critical safety component of linkdrops. The linkdrop contract should implement a key management strategy for keys such that a reentrancy attack does not occur. For example, one strategy may be to "lock" or mark a key as "in transfer" such that it cannot be used again until the transfer is complete. + +3. Asset Refunds & Failed Claims + Given that linkdrops could contain multiple different assets such as NFTs, or fungible tokens, sending assets might happen across multiple blocks. If the claim was unsuccessful (such as passing in an invalid account ID), it is important to ensure that all state is properly managed and assets are optionally refunded depending on the linkdrop contract's implementation. + +4. Fungible Tokens & Future Data + Fungible token contracts require that anyone receiving tokens must be registered. For this reason, it is important to ensure that storage for accounts claiming linkdrops is paid for. This concept can be extended to any future data types that may be added. You must ensure that all the pre-requisite conditions have been met for the asset that is being transferred. + +5. Tokens Properly Sent to Linkdrop Contract + Since the linkdrop contract facilitates the transfer of assets including NFTs, and FTs, it is important to ensure that those tokens have been properly sent to the linkdrop contract prior to claiming. In addition, since all the tokens are in a shared pool, you must ensure that the linkdrop contract cannot claim assets that do not belong to the key that is being used to claim. + +It is also important to note that not every linkdrop is valid. Drops can expire, funds can be lazily sent to the contract (as seen in the case of fungible and non-fungible tokens) and the supply can be limited. + +## Alternatives + +#### Why is this design the best in the space of possible designs? + +This design allows for flexibility and extensibility of the standard while providing a set of criteria that cover the majority of current linkdrop use cases. The design was heavily inspired by current, functional NEPs such as the Fungible Token and Non-Fungible Token standards. + +#### What other designs have been considered and what is the rationale for not choosing them? + +A generic data struct that all drop types needed to inherit from. This struct contained a name and some metadata in the form of stringified JSON. This made it easily extensible for any new types down the road. The rationale for not choosing this design was both simplicity and flexibility. Having one data struct requires keys to be of one type only. In reality, there can be many at once. In addition, having a generic, open-ended metadata field could lead to many interpretations and different designs. We chose to use a KeyInfo struct that can be easily extensible and can cover all use-cases by having optional vectors of different data types. The proposed standard is simple, supports drops with multiple assets, and is backwards compatible with all previous linkdrops, and can be extended very easily. + +A standard linkdrop creation interface. A standardized linkdrop creation interface would provide data models and functions to ensure linkdrops were created and stored in a specific format. The rationale for not choosing this design was that is was too restrictive. Standardizing linkdrop creation adds complexity and reduces flexibility by restricting linkdrop creators in the process in which linkdrops are created, and potentially limiting linkdrop functionality. The functionality of the linkdrop creation, such as refunding of assets, access keys, and batch creation, should be chosen by the linkdrop creator and live within the linkdrop creator platform. Further, linkdrop creation is often not displayed to end users and there is not an inherent value proposition for a standardized linkdrop creation interface from a client perspective. + +#### What is the impact of not doing this? + +The impact of not doing this is creating a fragmented ecosystem of linkdrops, increasing the friction for user onboarding. Linkdrop claim pages (e.g. wallet providers) would have to implement custom integrations for every linkdrop provider platform. Inherently this would lead to a bad user experience when new users are onboarding and interacting with linkdrops in general. + +## Future possibilities + +- Linkdrop creation interface + +- Bulk linkdrop management (create, update, claim) + +- Function call data types (allowing for funder defined functions to be executed when a linkdrop is claimed) + +- Optional configurations added to KeyInfo which can include multi-usekeys, time-based claiming etc… + +- Standard process for how links connect to claim pages (i.e a standardized URL such as an app’s baseUrl/contractId= [LINKDROP_CONTRACT]&secretKey=[SECRET_KEY] + +- Standard for deleting keys and refunding assets. + +## Copyright + +Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/). diff --git a/neps/nep-0455.md b/neps/nep-0455.md index 9699f1458..5cd9bddfc 100644 --- a/neps/nep-0455.md +++ b/neps/nep-0455.md @@ -155,24 +155,24 @@ By default, the parameter compute cost matches the corresponding gas cost. Compute costs should be applicable to all gas parameters, specifically including: -- [`ExtCosts`](https://cs.github.com/near/nearcore/blob/6e08a41084c632010b1d4c42132ad58ecf1398a2/core/primitives-core/src/config.rs#L377) -- [`ActionCosts`](https://cs.github.com/near/nearcore/blob/6e08a41084c632010b1d4c42132ad58ecf1398a2/core/primitives-core/src/config.rs#L456) +- [`ExtCosts`](https://github.com/near/nearcore/blob/6e08a41084c632010b1d4c42132ad58ecf1398a2/core/primitives-core/src/config.rs#L377) +- [`ActionCosts`](https://github.com/near/nearcore/blob/6e08a41084c632010b1d4c42132ad58ecf1398a2/core/primitives-core/src/config.rs#L456) Changes necessary to support `ExtCosts`: -1. Track compute usage in [`GasCounter`](https://cs.github.com/near/nearcore/blob/51670e593a3741342a1abc40bb65e29ba0e1b026/runtime/near-vm-logic/src/gas_counter.rs#L47) struct -2. Track compute usage in [`VMOutcome`](https://cs.github.com/near/nearcore/blob/056c62183e31e64cd6cacfc923a357775bc2b5c9/runtime/near-vm-logic/src/logic.rs#L2868) struct (alongside `burnt_gas` and `used_gas`) -3. Store compute usage in [`ActionResult`](https://cs.github.com/near/nearcore/blob/6d2f3fcdd8512e0071847b9d2ca10fb0268f469e/runtime/runtime/src/lib.rs#L129) and aggregate it across multiple actions by modifying [`ActionResult::merge`](https://cs.github.com/near/nearcore/blob/6d2f3fcdd8512e0071847b9d2ca10fb0268f469e/runtime/runtime/src/lib.rs#L141) -4. Store compute costs in [`ExecutionOutcome`](https://cs.github.com/near/nearcore/blob/578983c8df9cc36508da2fb4a205c852e92b211a/runtime/runtime/src/lib.rs#L266) and [aggregate them across all transactions](https://cs.github.com/near/nearcore/blob/578983c8df9cc36508da2fb4a205c852e92b211a/runtime/runtime/src/lib.rs#L1279) -5. Enforce the chunk compute limit when the chunk is [applied](https://cs.github.com/near/nearcore/blob/6d2f3fcdd8512e0071847b9d2ca10fb0268f469e/runtime/runtime/src/lib.rs#L1325) +1. Track compute usage in [`GasCounter`](https://github.com/near/nearcore/blob/51670e593a3741342a1abc40bb65e29ba0e1b026/runtime/near-vm-logic/src/gas_counter.rs#L47) struct +2. Track compute usage in [`VMOutcome`](https://github.com/near/nearcore/blob/056c62183e31e64cd6cacfc923a357775bc2b5c9/runtime/near-vm-logic/src/logic.rs#L2868) struct (alongside `burnt_gas` and `used_gas`) +3. Store compute usage in [`ActionResult`](https://github.com/near/nearcore/blob/6d2f3fcdd8512e0071847b9d2ca10fb0268f469e/runtime/runtime/src/lib.rs#L129) and aggregate it across multiple actions by modifying [`ActionResult::merge`](https://github.com/near/nearcore/blob/6d2f3fcdd8512e0071847b9d2ca10fb0268f469e/runtime/runtime/src/lib.rs#L141) +4. Store compute costs in [`ExecutionOutcome`](https://github.com/near/nearcore/blob/578983c8df9cc36508da2fb4a205c852e92b211a/runtime/runtime/src/lib.rs#L266) and [aggregate them across all transactions](https://github.com/near/nearcore/blob/578983c8df9cc36508da2fb4a205c852e92b211a/runtime/runtime/src/lib.rs#L1279) +5. Enforce the chunk compute limit when the chunk is [applied](https://github.com/near/nearcore/blob/6d2f3fcdd8512e0071847b9d2ca10fb0268f469e/runtime/runtime/src/lib.rs#L1325) Additional changes necessary to support `ActionCosts`: -1. Return compute costs from [`total_send_fees`](https://cs.github.com/near/nearcore/blob/578983c8df9cc36508da2fb4a205c852e92b211a/runtime/runtime/src/config.rs#L71) -2. Store aggregate compute cost in [`TransactionCost`](https://cs.github.com/near/nearcore/blob/578983c8df9cc36508da2fb4a205c852e92b211a/runtime/runtime/src/config.rs#L22) struct -3. Propagate compute costs to [`VerificationResult`](https://cs.github.com/near/nearcore/blob/578983c8df9cc36508da2fb4a205c852e92b211a/runtime/runtime/src/verifier.rs#L330) +1. Return compute costs from [`total_send_fees`](https://github.com/near/nearcore/blob/578983c8df9cc36508da2fb4a205c852e92b211a/runtime/runtime/src/config.rs#L71) +2. Store aggregate compute cost in [`TransactionCost`](https://github.com/near/nearcore/blob/578983c8df9cc36508da2fb4a205c852e92b211a/runtime/runtime/src/config.rs#L22) struct +3. Propagate compute costs to [`VerificationResult`](https://github.com/near/nearcore/blob/578983c8df9cc36508da2fb4a205c852e92b211a/runtime/runtime/src/verifier.rs#L330) -Additionaly, the gas price computation will need to be adjusted in [`compute_new_gas_price`](https://cs.github.com/near/nearcore/blob/578983c8df9cc36508da2fb4a205c852e92b211a/core/primitives/src/block.rs#L328) to use compute cost instead of gas cost. +Additionaly, the gas price computation will need to be adjusted in [`compute_new_gas_price`](https://github.com/near/nearcore/blob/578983c8df9cc36508da2fb4a205c852e92b211a/core/primitives/src/block.rs#L328) to use compute cost instead of gas cost. ## Security Implications diff --git a/neps/nep-519-yield-execution.md b/neps/nep-519-yield-execution.md new file mode 100644 index 000000000..e636d0df3 --- /dev/null +++ b/neps/nep-519-yield-execution.md @@ -0,0 +1,191 @@ +--- +NEP: 519 +Title: Yield Execution +Authors: Akhi Singhania ; Saketh Are +Status: Draft +DiscussionsTo: https://github.com/near/NEPs/pull/519 +Type: Protocol +Version: 0.0.0 +Created: 2023-11-17 +LastUpdated: 2023-11-20 +--- + +## Summary + +Today, when a smart contract is called by a user or another contract, it has no sensible way to delay responding to the caller till it has observed another future transaction. This proposal introduces this possibility into the NEAR protocol. + +## Motivation + +There exist some situations where when a smart contract on NEAR is called, it will only be able to provide an answer at some time in the future. The callee needs a way to defer replying to the caller while the response is being prepared. + +Examples include a smart contract (`S`) that provides the MPC signing capability. It relies on indexers external to the NEAR protocol for computing the signatures. The rough steps are: + +1. Signer contract provides a function `fn sign_payload(Payload, ...)`. +2. When called, the contract defers replying to the caller. +3. External indexers are monitoring the transactions on the contract; they observe the new signing request, compute a signature, and call another function `fn signature_available(Signature, ...)` on the signer contract. +4. The signer contract validates the signature and replies to the original caller. + +Today, the NEAR protocol has no sensible way to defer replying to the caller in step 2 above. This proposal proposes adding two following new host functions to the NEAR protocol: + +- `promise_yield_create`: allows setting up a continuation function that should only be executed after `promise_yield_resume` is invoked. Together with `promise_return` this allows delaying the reply to the caller; +- `promise_yield_resume`: indicates to the protocol that the continuation to the yield may now be executed. + +If these two host functions were available, then `promise_yield_create` would be used to implement step 2 above and `promise_yield_resume` would be used for step 3 of the motivating example above. + +## Specification + +The proposal is to add the following host functions to the NEAR protocol: + + +```rust +/// Smart contracts can use this host function along with +/// `promise_yield_resume()` to delay replying to their caller for up to 200 +/// blocks. This host function allows the contract to provide a callback to the +/// protocol that will be executed after either contract calls +/// `promise_yield_resume()` or after 200 blocks have been executed. The +/// callback then has the opportunity to either reply to the caller or to delay +/// replying again. +/// +/// `method_name_len` and `method_name_ptr`: Identify the callback method that +/// should be executed either after the contract calls `promise_yield_resume()` +/// or after 200 blocks have been executed. +/// +/// `arguments_len` and `arguments_ptr` provide an initial blob of arguments +/// that will be passed to the callback. These will be available via the +/// `input` host function. +/// +/// `gas`: Similar to the `gas` parameter in +/// [promise_create](https://github.com/near/nearcore/blob/a908de36ab6f75eb130447a5788007e26d05f93e/runtime/near-vm-runner/src/logic/logic.rs#L1281), +/// the `gas` parameter is a prepayment for the gas that would be used to +/// execute the callback. +/// +/// `gas_weight`: Similar to the `gas_weight` parameter in +/// [promise_batch_action_function_call_weight](https://github.com/near/nearcore/blob/a908de36ab6f75eb130447a5788007e26d05f93e/runtime/near-vm-runner/src/logic/logic.rs#L1699), +/// this improves the devX for the smart contract. It allows a contract to +/// specify a portion of the remaining gas for executing the callback instead of +/// specifying a precise amount. +/// +/// `register_id`: is used to identify the register that will be filled with a +/// unique resumption token. This token is used with `promise_yield_resume` to +/// resolve the continuation receipt set up by this function. +/// +/// Return value: u64: Similar to the +/// [promise_create](https://github.com/near/nearcore/blob/a908de36ab6f75eb130447a5788007e26d05f93e/runtime/near-vm-runner/src/logic/logic.rs#L1281) +/// host function, this function also create a promise and returns an index to +/// the promise. This index can be used to create a chain of promises. +pub fn promise_yield_create( + method_name_len: u64, + method_name_ptr: u64, + arguments_len: u64, + arguments_ptr: u64, + gas: u64, + gas_weight: u64, + register_id: u64, +) -> u64; + +/// See `promise_yield_create()` for more details. This host function can be +/// used to resolve the continuation that was set up by +/// `promise_yield_create()`. The contract calling this function must be the +/// same contract that called `promise_yield_create()` earlier. This host +/// function cannot be called for the same resumption token twice or if the +/// callback specified in `promise_yield_create()` has already executed. +/// +/// `data_id_len` and `data_it_ptr`: Used to pass the unique resumption token +/// that was returned to the smart contract in the `promise_yield_create()` +/// function (via the register). +/// +/// `payload_len` and `payload_ptr`: the smart contract can provide an +/// additional optional blob of arguments that should be passed to the callback +/// that will be resumed. These are available via the `promise_result` host +/// function. +/// +/// This function can be called multiple times with the same data id. If it is +/// called successfully multiple times, then the implementation guarantees that +/// the yielded callback will execute with one of the successfully submitted +/// payloads. If submission was successful, then `1` is returned. Otherwise +/// (e.g. if the yield receipt has already timed out or the yielded callback has +/// already been executed) `0` will be returned, indicating that this payload +/// could not be submitted successfully. +pub fn promise_yield_resume( + data_id_len: u64, + data_id_ptr: u64, + payload_len: u64, + payload_ptr: u64, +) -> u32; +``` + +## Reference Implementation + +The reference implementation against the nearcore repository can be found in this [PR](https://github.com/near/nearcore/pull/10415). + +## Security Implications + +Some potential security issues have been identified and are covered below: + +- Smart contracts using this functionality have to be careful not to let just any party trigger a call to `promise_yield_resume`. In the example above, it is possible that a malicious actor may pretend to be an external signer and call the `signature_available()` function with an incorrect signature. Hence contracts should be taking precautions by only letting select callers call the function (by using [this](https://github.com/aurora-is-near/near-plugins/blob/master/near-plugins/src/access_controllable.rs) service for example) and validating the payload before acting upon it. +- This mechanism introduces a new way to create delayed receipts in the protocol. When the protocol is under conditions of congestion, this mechanism could be used to further aggravate the situation. This is deemed as not a terrible issue as the existing mechanisms of using promises and etc. can also be used to further exacerbate the situation. + +## Alternatives + +Two alternatives have been identified. + +### Self calls to delay replying + +In the `fn sign_payload(Payload, ...)` function, instead of calling `yield`, the contract can keep calling itself in a loop till external indexer replies with the signature. This would work but would be very fragile and expensive. The contract would have to pay for all the calls and function executions while it is waiting for the response. Also depending on the congestion on the network; if the shard is not busy at all, some self calls could happen within the same block meaning that the contract might not actually wait for as long as it hoped for and if the network is very busy then the call from the external indexer might be arbitrarily delayed. + +### Change the flow of calls + +The general flow of cross contract calls in NEAR is that a contract `A` sends a request to another contract `B` to perform a service and `B` replies to `A` with the response. This flow could be altered. When a contract `A` calls `B` to perform a service, `B` could respond with a "promise to call it later with the answer". Then when the signature is eventually available, `B` can then send `A` a request with the signature. + +There are some problems with this approach though. After the change of flow of calls; `B` is now going to be paying for gas for various executions that `A` should have been paying for. Due to bugs or malicious intent, `B` could forget to call `A` with the signature. If `A` is calling `B` deep in a call tree and `B` replies to it without actually providing an answer, then `A` would need a mechanism to keep the call tree alive while it waits for `B` to call it with the signature in effect running into the same problem that this NEP is attempting to solve. + +## Future possibilities + +One potential future possibility is to allow contracts to specify how long the protocol should wait (up to a certain limit) for the contract to call `promise_yield_resume`. If contracts specify a smaller value, they would potentially be charged a smaller gas fee. This would make contracts more efficient. This enhancement does lead to a more complex implementation and could even allow malicious contracts to more easily concentrate a lot of callbacks to occur at the same time increasing the congestion on the network. Hence, we decided not to include this feature for the time being. + +## Consequences + +[This section describes the consequences, after applying the decision. All consequences should be summarized here, not just the "positive" ones. Record any concerns raised throughout the NEP discussion.] + +### Positive + +- p1 + +### Neutral + +- n1 + +### Negative + +- n1 + +### Backwards Compatibility + +We believe this can be implemented with full backwards compatibility. + +## Changelog + +### 1.0.0 - Initial Version + +> Placeholder for the context about when and who approved this NEP version. + +#### Benefits + +> List of benefits filled by the Subject Matter Experts while reviewing this version: + +- Benefit 1 +- Benefit 2 + +#### Concerns + +> Template for Subject Matter Experts review for this version: +> Status: New | Ongoing | Resolved + +| # | Concern | Resolution | Status | +| --: | :------ | :--------- | -----: | +| 1 | | | | +| 2 | | | | + +## Copyright + +Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/). diff --git a/specs/BlockchainLayer/EpochManager/Epoch.md b/specs/BlockchainLayer/EpochManager/Epoch.md index 371e81e5a..bab363b5c 100644 --- a/specs/BlockchainLayer/EpochManager/Epoch.md +++ b/specs/BlockchainLayer/EpochManager/Epoch.md @@ -10,29 +10,42 @@ Within one epoch, validator assignment is based on block height: each height has each height and shard have a chunk producer. ### End of an epoch -A block is defined to be the last block in its epoch if it's the genesis block or if the following conditions are met: -- Let `estimated_next_epoch_start = first_block_in_epoch.height + epoch_length` -- `block.height + 1 >= estimated_next_epoch_start` + +Let `estimated_next_epoch_start = first_block_in_epoch.height + epoch_length` + +A `block` is defined to be the last block in its epoch if it's the genesis block or if the following condition is met: + - `block.last_finalized_height + 3 >= estimated_next_epoch_start` `epoch_length` is defined in `genesis_config` and has a value of `43200` height delta on mainnet (12 hours at 1 block per second). +Since every final block must have two more blocks on top of it, it means that the last block in an epoch will have a height of at least `block.last_finalized_height + 2`, so for the last block it holds that `block.height + 1 >= estimated_next_epoch_start`. Its height will be at least `estimated_next_epoch_start - 1`. + +Note that an epoch only ends when there is a final block above a certain height. If there are no final blocks, the epoch will be stretched until the required final block appears. An epoch can potentially be longer than `epoch_length`. + +![Diagram of epoch end](epoch_end_diagram.png) + ### EpochHeight + Epochs on one chain can be identified by height, which is defined the following way: + - Special epoch that contains genesis block only: undefined - Epoch starting from the block that's after genesis: `0` (NOTE: in current implementation it's `1` due to a bug, so there are two epochs with height `1`) - Following epochs: height of the parent epoch plus one ### Epoch id + Every block stores the id of its epoch - `epoch_id`. Epoch id is defined as + - For special genesis block epoch it's `0` - For epoch with height `0` it's `0` (NOTE: the first two epochs use the same epoch id) - For epoch with height `1` it's the hash of genesis block - For epoch with height `T+2` it's the hash of the last block in epoch `T` -### End of an epoch +### Epoch end + - After processing the last block of epoch `T`, `EpochManager` aggregates information from block of the epoch, and computes validator set for epoch `T+2`. This process is described in [EpochManager](EpochManager.md). - After that, the validator set rotates to epoch `T+1`, and the next block is produced by the validator from the new set diff --git a/specs/BlockchainLayer/EpochManager/epoch_end_diagram.png b/specs/BlockchainLayer/EpochManager/epoch_end_diagram.png new file mode 100644 index 000000000..c02f9f3cc Binary files /dev/null and b/specs/BlockchainLayer/EpochManager/epoch_end_diagram.png differ diff --git a/specs/ChainSpec/Consensus.md b/specs/ChainSpec/Consensus.md index 8651d49dd..5daa2d3e1 100644 --- a/specs/ChainSpec/Consensus.md +++ b/specs/ChainSpec/Consensus.md @@ -195,7 +195,9 @@ Now back to the theorem. Without loss of generality, assume that $\operatorname{ ## Liveness -See the proof of liveness in [near.ai/doomslug](https://near.ai/doomslug). The consensus in this section differs in that it requires two consecutive blocks with endorsements. The proof in the linked paper trivially extends, by observing that once the delay is sufficiently long for a honest block producer to collect enough endorsements, the next block producer ought to have enough time to collect all the endorsements too. +See the proof of liveness in [Doomslug Whitepaper](https://discovery-domain.org/papers/doomslug.pdf) and the recent [Nightshade](https://discovery-domain.org/papers/nightshade.pdf) sharding protocol. + +The consensus in this section differs in that it requires two consecutive blocks with endorsements. The proof in the linked paper trivially extends, by observing that once the delay is sufficiently long for a honest block producer to collect enough endorsements, the next block producer ought to have enough time to collect all the endorsements too. ## Approval condition diff --git a/specs/ChainSpec/SelectingBlockProducers.md b/specs/ChainSpec/SelectingBlockProducers.md index 875ac9fa9..a8711e92f 100644 --- a/specs/ChainSpec/SelectingBlockProducers.md +++ b/specs/ChainSpec/SelectingBlockProducers.md @@ -25,17 +25,18 @@ block producers to be the one responsible for producing the chunk/block at each shard. There are several desiderata for these algorithms: -* Larger stakes should be preferred (more staked tokens means more security) -* The frequency with which a given participant is selected to produce a particular chunk/block is + +- Larger stakes should be preferred (more staked tokens means more security) +- The frequency with which a given participant is selected to produce a particular chunk/block is proportional to that participant's stake -* All participants selected as chunk/block producers should be selected to produce at least one +- All participants selected as chunk/block producers should be selected to produce at least one chunk/block during the epoch -* It should be possible to determine which chunk/block producer is supposed to produce the +- It should be possible to determine which chunk/block producer is supposed to produce the chunk/block at height $h$, for any $h$ within the epoch, in constant time -* The block producer chosen at height $h$ should have been a chunk producer for some shard at +- The block producer chosen at height $h$ should have been a chunk producer for some shard at height $h - 1$, this minimizes network communication between chunk producers and block producers -* The number of distinct chunk-only/block producers should be as large as is allowed by the +- The number of distinct chunk-only/block producers should be as large as is allowed by the scalability in the consensus algorithm (too large and the system would be too slow, too small and the system would be too centralized) $^{\dagger}$ @@ -46,16 +47,16 @@ There are several desiderata for these algorithms: ## Assumptions -* The maximum number of distinct chunk-only producers and block producers supported by the consensus +- The maximum number of distinct chunk-only producers and block producers supported by the consensus algorithm is a fixed constant. This will be a parameter of the protocol itself (i.e. all nodes must agree on the constant). In this document, we will denote the maximum number of chunk-only producers as `MAX_NUM_CP` and the maximum number of block producers by `MAX_NUM_BP`. -* The minimum number of blocks in the epoch is known at the time of block producer selection. This +- The minimum number of blocks in the epoch is known at the time of block producer selection. This minimum does not need to be incredibly accurate, but we will assume it is within a factor of 2 of the actual number of blocks in the epoch. In this document we will refer to this as the "length of the epoch", denoted by `epoch_length`. -* To meet the requirement that any chosen validator will be selected to produce at least one - chunk/block in the epoch, we assume it is acceptable for the probability of this *not* happening +- To meet the requirement that any chosen validator will be selected to produce at least one + chunk/block in the epoch, we assume it is acceptable for the probability of this _not_ happening to be sufficiently low. Let `PROBABILITY_NEVER_SELECTED` be a protocol constant which gives the maximum allowable probability that the chunk-only/block producer with the least stake will never be selected to produce a chunk/block during the epoch. We will additionally assume the chunk/block @@ -66,27 +67,30 @@ There are several desiderata for these algorithms: stake (what stake is "relevant" depends on whether the validator is a chunk-only producer or a block producer; more details below). Hence, the algorithm will enforce the condition $(1 - (s_\text{min} / S))^\text{epoch\_length} < \text{PROBABILITY\_NEVER\_SELECTED}$. - -In mainnet and testnet, `epoch_length` is set to `43200`. Let $\text{PROBABILITY\_NEVER\_SELECTED}=0.001$, + +In mainnet and testnet, `epoch_length` is set to `43200`. Let $\text{PROBABILITY\_NEVER\_SELECTED}=0.001$, we obtain, $s_\text{min} / S = 160/1000,000$. ## Algorithm for selecting block and chunk producers -A potential validator cannot specify whether they want to become a block producer or a chunk-only producer. -There is only one type of proposal. The same algorithm is used for selecting block producers and chunk producers, -but with different thresholds. The threshold for becoming block producers is higher, so if a node is selected as a block + +A potential validator cannot specify whether they want to become a block producer or a chunk-only producer. +There is only one type of proposal. The same algorithm is used for selecting block producers and chunk producers, +but with different thresholds. The threshold for becoming block producers is higher, so if a node is selected as a block producer, it will also be a chunk producer, but not the other way around. Validators who are selected as chunk producers but not block producers are chunk-only producers. ### select_validators + ### Input -* `max_num_validators: u16` max number of validators to be selected -* `min_stake_fraction: Ratio` minimum stake ratio for selected validator -* `validator_proposals: Vec` (proposed stakes for the next epoch from nodes sending + +- `max_num_validators: u16` max number of validators to be selected +- `min_stake_fraction: Ratio` minimum stake ratio for selected validator +- `validator_proposals: Vec` (proposed stakes for the next epoch from nodes sending staking transactions) ### Output -* `(validators: Vec, sampler: WeightedIndex)` +- `(validators: Vec, sampler: WeightedIndex)` ### Steps @@ -110,10 +114,12 @@ return (validators, validator_sampler) ``` ### Algorithm for selecting block producers + ### Input -* `MAX_NUM_BP: u16` Max number of block producers, see Assumptions -* `min_stake_fraction: Ratio` $s_\text{min} / S$, see Assumptions -* `validator_proposals: Vec` (proposed stakes for the next epoch from nodes sending + +- `MAX_NUM_BP: u16` Max number of block producers, see Assumptions +- `min_stake_fraction: Ratio` $s_\text{min} / S$, see Assumptions +- `validator_proposals: Vec` (proposed stakes for the next epoch from nodes sending staking transactions) ```python @@ -121,42 +127,46 @@ select_validators(MAX_NUM_BP, min_stake_fraction, validator_proposals) ``` ### Algorithm for selecting chunk producers + ### Input -* `MAX_NUM_CP: u16` max number of chunk producers, see Assumptions` -* `min_stake_fraction: Ratio` $s_\text{min} / S$, see Assumptions -* `num_shards: u64` number of shards -* `validator_proposals: Vec` (proposed stakes for the next epoch from nodes sending + +- `MAX_NUM_CP: u16` max number of chunk producers, see Assumptions` +- `min_stake_fraction: Ratio` $s_\text{min} / S$, see Assumptions +- `num_shards: u64` number of shards +- `validator_proposals: Vec` (proposed stakes for the next epoch from nodes sending staking transactions) ```python select_validators(MAX_NUM_CP, min_stake_fraction/num_shards, validator_proposals) ``` -The reasoning for using `min_stake_fraction/num_shards` as the threshold here is that -we will assign chunk producers to shards later and the algorithm (described below) will try to assign + +The reasoning for using `min_stake_fraction/num_shards` as the threshold here is that +we will assign chunk producers to shards later and the algorithm (described below) will try to assign them in a way that the total stake in each shard is distributed as evenly as possible. So the total stake in each shard will be roughly be `total_stake_all_chunk_producers / num_shards`. ## Algorithm for assigning chunk producers to shards + Note that block producers are a subset of chunk producers, so this algorithm will also assign block producers -to shards. This also means that a block producer may only be assigned to a subset of shards. For the security of -the protocol, all block producers must track all shards, even if they are not assigned to produce chunks for all shards. -We enforce that in the implementation level, not the protocol level. A validator node will panic if it doesn't track all -shards. +to shards. This also means that a block producer may only be assigned to a subset of shards. For the security of +the protocol, all block producers must track all shards, even if they are not assigned to produce chunks for all shards. +We enforce that in the implementation level, not the protocol level. A validator node will panic if it doesn't track all +shards. ### Input -* `chunk_producers: Vec` -* `num_shards: usize` -* `min_validators_per_shard: usize` +- `chunk_producers: Vec` +- `num_shards: usize` +- `min_validators_per_shard: usize` ### Output -* `validator_shard_assignments: Vec>` +- `validator_shard_assignments: Vec>` - $i$-th element gives the validators assigned to shard $i$ ### Steps -* While any shard has fewer than `min_validators_per_shard` validators assigned to it: +- While any shard has fewer than `min_validators_per_shard` validators assigned to it: - Let `cp_i` be the next element of `chunk_producers` (cycle back to the beginning as needed) - Note: if there are more shards than chunk producers, then some chunk producers will be assigned to multiple shards. This is undesirable because we want each chunk-only producer @@ -167,12 +177,12 @@ shards. - Let `shard_id` be the shard with the fewest number of assigned validators such that `cp_i` has not been assigned to `shard_id` - Assign `cp_i` to `shard_id` -* While there are any validators which have not been assigned to any shard: +- While there are any validators which have not been assigned to any shard: - Let `cp_i` be the next validator not assigned to any shard - Let `shard_id` be the shard with the least total stake (total stake = sum of stakes of all validators assigned to that shard) - Assign `cp_i` to `shard_id` -* Return the shard assignments +- Return the shard assignments In addition to the above description, we have a [proof-of-concept (PoC) on GitHub](https://github.com/birchmd/bp-shard-assign-poc). Note: this PoC has not been updated since @@ -181,11 +191,13 @@ the same algorithm works to assign chunk producers to shards; it is only a matte variables referencing "block producers" to reference "chunk producers" instead. ## Algorithm for sampling validators proportional to stake + We sample validators with probability proportional to their stake using the following data structure. -* `weighted_sampler: WeightedIndex` + +- `weighted_sampler: WeightedIndex` - Allow $O(1)$ sampling - This structure will be based on the - [WeightedIndex](https://rust-random.github.io/rand/rand/distributions/weighted/alias_method/struct.WeightedIndex.html) + [WeightedIndex](https://rust-random.github.io/rand/rand/distributions/struct.WeightedIndex.html) implementation (see a description of [Vose's Alias Method](https://en.wikipedia.org/wiki/Alias_method) for details) @@ -194,14 +206,14 @@ algorithms for selecting a specific block producer and chunk producer at each he ### Input -* `rng_seed: [u8; 32]` +- `rng_seed: [u8; 32]` - See usages of this algorithm below to see how this seed is generated -* `validators: Vec` -* `sampler: WeightedIndex` +- `validators: Vec` +- `sampler: WeightedIndex` ### Output -* `selection: ValidatorStake` +- `selection: ValidatorStake` ### Steps @@ -228,18 +240,18 @@ return validators[index] ### Input -* `h: BlockHeight` +- `h: BlockHeight` - Height to compute the block producer for - Only heights within the epoch corresponding to the given block producers make sense as input -* `block_producers: Vec` (output from above) -* `block_producer_sampler: WeightedIndex` -* `epoch_rng_seed: [u8; 32]` +- `block_producers: Vec` (output from above) +- `block_producer_sampler: WeightedIndex` +- `epoch_rng_seed: [u8; 32]` - Fixed seed for the epoch determined from Verified Random Function (VRF) output of last block in the previous epoch ### Output -* `block_producer: ValidatorStake` +- `block_producer: ValidatorStake` ### Steps @@ -256,14 +268,14 @@ return select_validator(rng_seed=block_seed, validators=block_producers, sampler ### Input -* (same inputs as selection of block producer at height h) -* `num_shards: usize` -* `chunk_producer_sampler: Vec` (outputs from chunk-only producer selection) -* `validator_shard_assignments: Vec>` +- (same inputs as selection of block producer at height h) +- `num_shards: usize` +- `chunk_producer_sampler: Vec` (outputs from chunk-only producer selection) +- `validator_shard_assignments: Vec>` ### Output -* `chunk_producers: Vec` +- `chunk_producers: Vec` - `i`th element gives the validator that will produce the chunk for shard `i`. Note: at least one of these will be a block producer, while others will be chunk-only producers. diff --git a/specs/DataStructures/Transaction.md b/specs/DataStructures/Transaction.md index 13ebed4bd..692147a5b 100644 --- a/specs/DataStructures/Transaction.md +++ b/specs/DataStructures/Transaction.md @@ -1 +1,3 @@ # Transaction + +See [Transactions](../RuntimeSpec/Transactions.md) documentation in the [Runtime Specification](../RuntimeSpec) section. diff --git a/specs/Proposals/0033-economics.md b/specs/Proposals/0033-economics.md index 349628f85..d70e727cb 100644 --- a/specs/Proposals/0033-economics.md +++ b/specs/Proposals/0033-economics.md @@ -4,40 +4,46 @@ - Issue(s): link to relevant issues in relevant repos (not required). # Summary + [summary]: #summary -Adding economics specification for NEAR Protocol based on the economics paper - https://near.org/papers/economics-in-sharded-blockchain/ +Adding economics specification for NEAR Protocol based on the NEAR whitepaper - https://pages.near.org/papers/the-official-near-white-paper/#economics # Motivation + [motivation]: #motivation Currently, the specification is defined by the implementation in https://github.com/near/nearcore. This codifies all the parameters and formulas and defines main concepts. # Guide-level explanation + [guide-level-explanation]: #guide-level-explanation The goal is to build a set of specs about NEAR token economics, for analysts and adopters, to simplify their understanding of the protocol and its game-theoretical dynamics. This initial release will be oriented to validators and staking in general. # Reference-level explanation + [reference-level-explanation]: #reference-level-explanation This part of the documentation is self-contained. It may provide material for third-party research papers, and spreadsheet analysis. # Drawbacks + [drawbacks]: #drawbacks We might just put this in the NEAR docs. # Rationale and alternatives -[rationale-and-alternatives]: #rationale-and-alternatives +[rationale-and-alternatives]: #rationale-and-alternatives # Unresolved questions -[unresolved-questions]: #unresolved-questions +[unresolved-questions]: #unresolved-questions # Future possibilities + [future-possibilities]: #future-possibilities -This is an open document which may be used by NEAR's community to pull request a new economic policy. Having a formal document also for non-technical aspects opens new opportunities for the governance. +This is an open document which may be used by NEAR's community to pull request a new economic policy. Having a formal document also for non-technical aspects opens new opportunities for the governance.