raw AST is missing from output (only shows ape-adjusted AST)

[0xalpharush]

Environment information

• OS: macOS
• Python Version: 3.10
• ape and plugin versions:

$ ape --version
0.6.15 # I also tried 0.6.16.dev9+gabe4e96b

$ ape plugins list
Installed Plugins:
  solidity    0.6.7
  hardhat     0.6.12
  vyper       0.6.9

• Contents of your ape-config.yaml (NOTE: do not post anything private like RPC urls or secrets!):

$ cat ape-config.yaml
name: yearn-v3

plugins:
  - name: solidity
    version: 0.6.7
  - name: vyper
    version: 0.6.9
  - name: hardhat
    version: 0.6.12

default_ecosystem: ethereum
dependencies:
  - name: openzeppelin
    github: OpenZeppelin/openzeppelin-contracts
    version: 4.7.3

solidity:
  import_remapping:
    - "@openzeppelin/contracts=openzeppelin/v4.7.3"

ethereum:
  local:
    default_provider: hardhat

What went wrong?

I am trying to parse the build manifest for ape support in https://github.com/crytic/crytic-compile, but the compiler info is missing
Reproduce with:

$ git clone https://github.com/0xalpharush/yearn-vaults-v3 && cd yearn-vaults-v3
$ python3 -m venv venv
$ source venv/bin/activate
$ python3 -m pip install -r requirements.txt
$ yarn
$ ape plugins install .
$ ape compile
$ cat .build/__local__.json | jq 'keys'
[
  "contractTypes",
  "manifest",
  "sources"
]

With python:

$ python
>>> from ethpm_types import PackageManifest
>>> import json
>>> print(PackageManifest.parse_obj(json.load(open(".build/__local__.json"))).compilers)
None

This field is missing https://github.com/ApeWorX/ethpm-types/blob/dffcb6b2fe17f858c95d07499aafe1d66e939dfa/ethpm_types/manifest.py#L124

How can it be fixed?

I think it may need to be added here

ape/src/ape/managers/project/types.py

Line 177 in abe4e96

def create_manifest(

[Aviksaikat]

hmm, it seems like there are many bugs in ape compile

[fubuloubu]

hmm, it seems like there are many bugs in ape compile

It does need some attention at the moment, however this specific issue should be easy to fix if you want to give it a shot @Aviksaikat

Will help add Ape support to Slither

[0xalpharush]

I mentioned it on Telegram but I think the ASTs are also manipulated internally by Ape and that output is what's provided. At least for slither, we need the original output of vyper/ solc

[Aviksaikat]

I will look into this as soon as the remove functionality implementation is done. Just bear with me.

[fubuloubu]

I will look into this as soon as the remove functionality implementation is done. Just bear with me.

No worries! Thank you for working on all of this, we appreciate your contributions!

[antazoey]

I mentioned it on Telegram but I think the ASTs are also manipulated internally by Ape and that output is what's provided. At least for slither, we need the original output of vyper/ solc

Slither is going to have to be integrated directly with the ape-vyper and ape-solidity plugins

[antazoey]

Thus far, compiler info has only been needed for publishing and not local development, so it is included in the get_compiler_settings() method:

In [6]: project.extract_manifest().dict().keys()
Out[6]: dict_keys(['manifest', 'meta', 'sources', 'contractTypes', 'compilers', 'deployments', 'buildDependencies'])

[0xalpharush]

Slither is going to have to be integrated directly with the ape-vyper and ape-solidity plugins

We generally invoke commands like hardhat/forge build rather than taking a dependency on these frameworks. We need a way to have the original artifacts (as if we invoked vyper/solc directly) from the compiler by version akin to the artifact format that hardhat and foundry provide.

[fubuloubu]

I think we can build in these as local links to the files in the contracts/ folder

If you just read and resolve the links (no dependency on Ape, just use ethpm-types in crytic-compile) you will get the full artifact set including sources

---

Edit. @antazoey to be clearer in .build/__local__.json:

• sources has a bunch of local links to files in project's` contracts folder
• contractTypes can local link to .build/{ContractTypeName}.json encoded ContractType objects (not sure how it's currently being done)
• AST in contractTypes can use the original output from the compiler? and then we would read and modify it within ape?
• compilers needs to get filled in
• publishing would just be resolving all local links together into the one file (or eventually publishing each local link to a separate CID and updating the local links to CID links instead in the cached manifest)

[antazoey]

Is ethpmtypes ok with links? We have to update there. But i like this plan!

[antazoey]

Related, pcamp and ast stuff should be links as well to keep the size down

[fubuloubu]

Is ethpmtypes ok with links? We have to update there. But i like this plan!

There is a method to resolve links in Source: https://github.com/ApeWorX/ethpm-types/blob/main/ethpm_types/source.py#L240

[fubuloubu]

Related, pcamp and ast stuff should be links as well to keep the size down

Good point, they can be additional local links within the ContractType object? (might be a breaking change there)

[antazoey]

since they are optional, we could exclude them only from the final build i suppose

[antazoey]

fixed in #1746

[0xalpharush]

Should I open a separate issue for #1590 (comment) or is this also fixed?