WIP:New import ast

[djudjuu]

Current status:

• --import-ast option in commandline able to import and compile all kinds of AST-nodes (exception: inlineAssembly)

• testuite that verifies

  • that an imported and exported AST is the same as original import
  • that the bytecodes of a compiled original .sol file is the same as the exported & imported JSON (see bug 1 below)

The testsuite runs on all contracts defined in /test/libsolidity/ASTJson/ (barring few tiny modifications) and 10 more contracts that include things not covered in there, such as (if, while, emit,...)

This PR introduces some changes to the contract metadata too, the compiler now
- defines new input-language solidityAST, and
- [sources][sourceName][content] is the compactPrint of the Json used as source (needs testing)

left TODOs:

• 1.) add support for inlineAssembly
• 2.) configure ASTImporter to use the evm-version specified in the JSON
• 3.) write another helper-function bytecodeWithoutAnyMetadata() that extracts the metadata of imported contracts too

bugs to fix:

• I can not use dev::test::bytecodeSansMetadata even though I am importing it and declaring the namespace (see ASTImportTest.cpp::run())

• some importStatements are not processed correctly (contract A is B,C,D,F {} will generate wrong order of dependencies, see /test/libsolidity/ASTImport/import.sol

• address(this) does not get processed correctly. -> referencedDeclarations will be a newly created one (see /test/libsolidity/ASTImport/address_payable.json) (see comment below for more details)

More about the testsuite

The testsuite is made of .sol files from which a combined-json is created with

solc --combined-json ast,compact_format --pretty-json feature.sol > feature.json

feature.json is then imported into the compiler, analyzed & compiled. Then a new json combined-json object is generated including an AST and the bytecode. It should be the same as the originally imported.

It can be run like this:

../scripts/soltest.sh --show-progress true -t "ASTImport" --no-ipc --no-smt

• a helper script to create the combined-json command on all .sol files: here

• and to extract the resulting Jsons from the output in case of there was an error: here

(check the comments for detailed instructions how to use the scripts)

[djudjuu]

bug3: address(this) has a wrong "referencedDeclaration"

steps to reproduce

• uncomment those lines and run the test
• expectedResult: the json of [functioncall][arguments][identifier][referencedDeclaration]: 66 (or some other low number
• obtained Result will be 144, (or somehting equally high)

suspected source

"referencedDeclaration" is produced from this:

IdentifierAnnotation& Identifier::annotation() const
{
	if (!m_annotation)
		m_annotation = new IdentifierAnnotation();
	return dynamic_cast<IdentifierAnnotation&>(*m_annotation);
}

so when there is no annotation, it will just produce a new one...

ideas about fixing it

Another thing i think i understood is that the annotations are set in the analyze()-step by the NameAndTypeResolver, which uses m_sourceOrder which in turn is filled in resolveImports() (called first thing in CompilerStack::analyze())

resolveImports relies on m_sources, which is not filled (instead I created m_sourceJsons), but it also refers to annotation() in its toposort()-function. Is it creating those in that step?

So possible avenues for solving it could be to mdofify m_sourceJsons so that it can be used to fill m_sourceOrder??