JSON interface description by axic · Pull Request #1387 · argotorg/solidity

axic · 2016-11-16T13:05:46Z

Follow up #1129. (Moved from #645 to a common branch.)

Todo:

axic · 2016-11-16T16:19:10Z

docs/using-the-compiler.rst

+        abi: [ /* abi definition */ ],
+        userdoc: [],
+        devdoc: [],
+        natspec: [ /* user documentation comments */ ]


Removing natspec from here, only keeping userdoc / devdoc.

axic · 2016-11-16T16:23:49Z

docs/using-the-compiler.rst

+        "dir/file.sol": {keccax: "0xabc..."},
+        "xkcd": {swarm: "0x456..."}
+      },
+      // This is a subset of the regular compiler input


settings is up to the intrepretation of the compiler defined by the language and compiler fields.

axic · 2016-11-16T16:25:09Z

docs/using-the-compiler.rst

+"official") at a registry contract.
+
+    {
+      version: "1",


Add comment: this is the version of the metadata spec.

axic · 2016-11-16T16:25:39Z

docs/using-the-compiler.rst

+    {
+      version: "1",
+      language: "Solidity",
+      compiler: {


compiler is language specific.

axic · 2016-11-16T16:25:56Z

docs/using-the-compiler.rst

+      language: "Solidity",
+      compiler: {
+        commit: "55db20e32c97098d13230ab7500758e8e3b31d64",
+        version: "soljson-2313-2016-12-12",


Must equal to the compiler semver version. Required field.

axic · 2016-11-16T16:26:16Z

docs/using-the-compiler.rst

+      version: "1",
+      language: "Solidity",
+      compiler: {
+        commit: "55db20e32c97098d13230ab7500758e8e3b31d64",


Remove, the semver version includes the commit.

axic · 2016-11-16T16:27:56Z

docs/using-the-compiler.rst

+      compiler: {
+        commit: "55db20e32c97098d13230ab7500758e8e3b31d64",
+        version: "soljson-2313-2016-12-12",
+        keccak: "0x123..."


keccak256 hash of the actual compiler. Optional field.

axic · 2016-11-16T16:28:10Z

docs/using-the-compiler.rst

+
+    {
+      version: "1",
+      language: "Solidity",


Required field.

axic · 2016-11-16T16:39:14Z

docs/using-the-compiler.rst

+      // This is a subset of the regular compiler input
+      sources:
+      {
+        "abc": {keccak: "0x456..."}, // here, sources are always given by hash


The supported fields are:

keccak256: a hash of the file. Recommended in every case, but if content is specified it has very little benefit.

content: the actual file content as a string.

url: a way to retrieve the file. It can be (but not limited to):

http:// or https:// (ie. github)

bzz:// or bzzr://

ipfs:

if you have url specified and content is not specified it should be mandated to have keccak256.

The key (filename) is an identifier for a source file. Cannot contain a colon. It used by fields in the settings.

axic · 2016-11-16T16:49:39Z

docs/using-the-compiler.rst

+        optimizer: {enabled: true, runs: 500},
+        compilationTarget: "myFile.sol:MyContract",
+        libraries: {
+          "def:MyLib": "0x123123..."


Add comment that entries are source keys and contract names plus the on-chain address of the contract.

Split out to be a map of key and library:

libraries: { "def": { "MyLib": "0x123123..." } }

We currently do not have file-based libraries, that should be changed in the near future, but we might also just allow libraries: { "MyLib": "0x123"} for now.

axic · 2016-11-16T16:55:23Z

docs/using-the-compiler.rst

+    {
+      sources:
+      {
+        // the keys here are the "global" names of the source files, imports can use other files via remappings (see below)


What applies to sources in metadata applies here. Two differences:

keccak256 is used to verify the content after acquiring it

file: as a url is supported

How about supporting keccak256 and urls. The compiler will then (in some defined order) query the caller for each url and hash, until the file is retrieved or the options are exhausted.

axic · 2016-11-16T16:56:00Z

docs/using-the-compiler.rst

+Input Description
+-----------------
+
+QUESTION: How to specific file-reading callback? - probably not as part of json input


The read callback will receive the url as specified in the sources list.

axic · 2016-11-16T16:59:33Z

docs/using-the-compiler.rst

+              message: "Invalid keyword" // mandatory
+          }
+      ]
+      contracts: {


This contains all the compiled outputs. It can be limited/filtered by the compilationTarget setting.

axic · 2016-11-16T17:06:28Z

docs/using-the-compiler.rst

+        // (axic) what is remapping doing exactly?
+        optimizer: {enabled: true, runs: 500},
+        // if given, only compiles this contract, can also be an array. If only a contract name is given, tries to find it if unique.
+        compilationTarget: "myFile.sol:MyContract",


compilationTarget:

can be an array

if no contract name is specified, all contracts in that file are considered

it can refer to a key and contract name which is not part of the sources, but will be loaded during parsing the sources

We remove this, as it is superseded by outputSelection.

axic · 2016-11-16T17:08:57Z

docs/using-the-compiler.rst

+        },
+        // The following can be used to restrict the fields the compiler will output.
+        // (axic)
+        outputSelection: [


Merging compilation target and output selection:

"*": { "*": [ "abi", "asm" ] } "def": { "MyContract": [ "abi", "opcodes" ] } "*": { "*": [ "srcmap" ] } "*": { "": [ "ast", "formal" ] }

Empty contract name refers to the file itself, while the star refers to all of the contracts.

outputSelection is a hashmap of the above. If no outputSelection is defined, then the compiler loads and does type checking, but will not generate any outputs apart from errors.

ast and formal are file-level and if they are needed, an additional empty-string entry is needed.

axic · 2016-11-16T17:25:21Z

docs/using-the-compiler.rst

+      sourceList: ["source1.sol", "source2.sol"], // this is important for source references both in the ast as well as in the srcmap in the contract
+      sources: {
+        "source1.sol": {
+          "AST": { ... }


The fields are:

keccak256

id (used in source maps)

ast

@chriseth if the AST is a JSON object, what is the hash referring to?

axic · 2016-11-16T17:25:33Z

docs/using-the-compiler.rst

+      formal: {
+        "why3": "..."
+      },
+      sourceList: ["source1.sol", "source2.sol"], // this is important for source references both in the ast as well as in the srcmap in the contract


This is superseded by the id in sources.

axic · 2016-11-16T17:34:14Z

docs/using-the-compiler.rst

+            "abi", "evm.assembly", "evm.bytecode", ..., "why3", "ewasm.wasm"
+        ]
+        outputSelection: {
+        abi,asm,ast,bin,bin-runtime,clone-bin,devdoc,interface,opcodes,srcmap,srcmap-runtime,userdoc


Output types:

abi

ast (as a JSON object)

evm.assembly

...

To be defined soon.

ast - AST of all source files

why3 - Translated source suitable for formal

abi - ABI

userdoc - User Doc

devdoc - Dev Doc

metadata - Metadata

evm.ir - New assembly format before desugaring (not supported atm)

evm.assembly - New assembly format after desugaring (not supported atm)

evm.asm - current asm

evm.asmJSON - current asmjson

evm.opcodes - current opcodes

evm.methodIdentifiers - the current "hashes" / "functionHashes"

evm.gasEstimates - current --gas and "gasEstimates"

evm.bytecode - current --bin or bytecode

evm.deployedBytecode - current --bin-runtime or runtimeBytecode

--clone-bin is removed.

axic · 2016-11-16T17:35:03Z

docs/using-the-compiler.rst

+      {
+        remappings: [":g/dir"],
+        optimizer: {enabled: true, runs: 500},
+        compilationTarget: "myFile.sol:MyContract",


Split out to a be a map of key and contract:

compilationTarget: { "myFile.sol": "MyContract }

axic · 2016-11-17T10:30:56Z

@chriseth @VoR0220 updated the metadata output.

axic · 2016-11-17T10:31:56Z

docs/using-the-compiler.rst

+Note that the actual bytecode is not part of the metadata because the hash
+of the metadata structure will be included in the bytecode itself.
+
+This requires the compiler to be able to compute the hash of its own binary,


This section should be removed.

axic · 2016-11-17T10:37:25Z

docs/using-the-compiler.rst

+              }
+          },
+          functionHashes:
+          metadata: // see below


Mention that, contrary to the rest of the fields, this is not a JSON object, but a stringified JSON. This is because serialisation differences might result in different output and thus metadata hash.

I wonder if we should call this metadata (which is kind of vague) or come with an actual name for it.

This should actually refer to the metadata section in miscellaneous.rst (once #1386 is merged).

chriseth · 2016-11-17T16:46:26Z

docs/using-the-compiler.rst

+      {
+        "myFile.sol": {
+          "keccak256": "0x123...",
+          "url": "bzzr://0x56..."


I think this does usually not have a 0x prefix.

chriseth · 2016-11-23T16:02:19Z

docs/using-the-compiler.rst

+      language: "Solidity",
+      // Required field, the contents are specific to the language
+      compiler: {
+        name: "solc",


This is not part of the implementation.

pipermerriam · 2017-02-09T15:41:04Z

🎂 !!!! This is great. Thank you for all the work you've put into this. It's going to be a major improvement.

axic added in progress labels Nov 16, 2016

This was referenced Nov 16, 2016

JSON output: better error reporting #778

Closed

Provide gas estimates in combined json #542

Closed

chriseth mentioned this pull request Nov 16, 2016

Document input description and metadata output. #645

Closed

axic commented Nov 16, 2016

View reviewed changes

docs/using-the-compiler.rst Outdated

{

version: "1",

language: "Solidity",

compiler: {

Copy link

Contributor Author

axic Nov 16, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

compiler is language specific.

axic commented Nov 16, 2016

View reviewed changes

docs/using-the-compiler.rst Outdated

{

version: "1",

language: "Solidity",

Copy link

Contributor Author

axic Nov 16, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Required field.

axic commented Nov 16, 2016

View reviewed changes

axic commented Nov 17, 2016

View reviewed changes

axic force-pushed the json-interface branch from 6b2c6bb to a5cc932 Compare November 17, 2016 16:38

chriseth reviewed Nov 17, 2016

View reviewed changes

chriseth reviewed Nov 23, 2016

View reviewed changes

axic added 19 commits February 8, 2017 21:53

The metadata section has been moved, make only a reference to it

d9f14e7

Place into a code block

720cf20

Update sources definition based on the metadata

6e2cc08

Update settings section

82c0e4d

Update output selection

4b5639b

Add metadata.useLiteralContent option

cbb6686

Change layout and include API, Input, Output sections

d46ec20

Include pseudo-code of compiler API

21a0228

Update errors output

627a2ce

Make contracts output two-level

4b19f56

Updated EVM output

c217bc2

Error list should have sourceLocation

a3340e2

Support linkReferences with length specified

9fc017d

Explain every contract output field

9fa54db

Update the AST output

96677cd

Move file into sourceLocation

10d3a59

Include language field in the JSON

749db76

Drop the legacy assembly output

cd81e58

Simplify the compiler API section (and remove pseudo code)

dc431fe

axic force-pushed the json-interface branch from 7829834 to dc431fe Compare February 8, 2017 23:42

chriseth approved these changes Feb 9, 2017

View reviewed changes

chriseth merged commit 4c8d818 into develop Feb 9, 2017

chriseth removed the in progress label Feb 9, 2017

axic deleted the json-interface branch February 9, 2017 12:25

This was referenced Feb 9, 2017

JSON output: empty fields #758

Closed

Export unlinked objects via json output #863

Closed

JSON input: control the required outputs #752

Closed

Define "json input/output" #1129

Closed

Impossible to differentiate link references in bytecode #1645

Closed

Conversation

axic commented Nov 16, 2016 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

axic Nov 16, 2016 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

axic Nov 16, 2016 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

axic Nov 16, 2016 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

axic Nov 16, 2016 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

axic Dec 23, 2016 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

axic commented Nov 17, 2016

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

axic commented Nov 16, 2016 •

edited

Loading

axic Nov 16, 2016 •

edited

Loading

axic Nov 16, 2016 •

edited

Loading

axic Nov 16, 2016 •

edited

Loading

axic Nov 16, 2016 •

edited

Loading

axic Dec 23, 2016 •

edited

Loading