Stop crashing on encounters with non-truffle projects

[cgewecke]

Resolves #40

[codecov-io]

Codecov Report

Merging #41 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master      #41   +/-   ##
=======================================
  Coverage   97.56%   97.56%           
=======================================
  Files           6        6           
  Lines         329      329           
  Branches       78       78           
=======================================
  Hits          321      321           
  Misses          8        8

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 1c2d328...c7a2376. Read the comment docs.

[travs]

@cgewecke
Does solidity-coverage support non-truffle projects?
We (melon protocol) are migrating away from truffle at the moment (moving to dapp).

Not entirely sure how to configure that if it is supported.

[cgewecke]

@travs Cool, if you move we will support you.

I'm also not sure how to configure it. In principle all we require is that the files to instrument be stored in a contracts folder somewhere and that the tests use testrpc-sc as a provider. Obviously the no-config/auto-launch is oriented towards truffle users but .solcover.js lets you:

• externalize the testrpc-sc launch if necessary by setting norpc to true
• run an arbitrary test command (not truffle) with testCommand

app.js also modularises each piece of the run sequence so you could even write a little JS runner for dapp similar to the current version of bin/exec.js which is very simple.

Would be great to see a dapphub example in the FAQ.

[travs]

@cgewecke
Cool! Would be great to try this.

Will be looking into this tomorrow and later this week, but one question I have right away is: can we use src/ instead of contracts/? Since dapp seems to support src/ only (may be an option to implement this change on their side as well).

[cgewecke]

contracts should definitely be made configurable then. . . that's simple. A far larger issue is that dapp uses ethrun to execute the code. At the moment we have hacked testrpc / ethereumjs-vm so that it logs to file all the coverage events we insert as part of the instrumentation process. At the end of the run we read that file to generate the report.

Either dapp needs to allow any blockchain client or someone will need to hack ethrun.

Out of curiosity is there a discussion anywhere in the issues at Melon about the pros and cons of this platform switch? Are you going to push engineering resources into dapp?

[travs]

@cgewecke
Ok got it. Seems like a fair amount of work to implement this if I understand correctly.

The only mention in the issues that I can recall is here, but it is not an extensive discussion.

We plan on using a combination of dapp + our own scripts going forward for deployment, testing, etc.
But we have no agreement as such to help engineer dapp itself, beyond providing a large project to test the framework against.

[cgewecke]

@travs The changes to the testrpc vm are not too difficult.

• Read from a file that lists all of the events we've injected into the solidity.
• Write to a file when one of those events turns up at the LOG opcode.

We also raised the default transaction gasLimit at testrpc and it has it's own option to set an arbitrary gas limit. This is necessary since running the instrumentation consumes lots of gas.

This project is all volunteer - realistically you or Maker would need to maintain an ethrun fork, (or branch?) that had these features. You'd be welcome to maintain it here if that works for you but it's not necessary.

And if you want help debugging I'm very happy to help. Definitely into more than one dev platform.

[mbrock]

Hey, I'm one of the DappHub developers.

It would be great to have coverage support for dapp and ds-test.

The ability to test ds-test projects with TestRPC would be good, and that would enable the use of solidity-coverage...

But we're also interested in building coverage tracking into our debugging EVM (https://github.com/dapphub/hevm). It already supports source maps and so I think we could pretty straightforwardly implement coverage tracking on the instruction level, without any Solidity instrumentation.

We'll be thinking about it. Thanks for the explanations and offer to help!

[cgewecke]

@mbrock Hi. Wow - great! That's good news.

We've been talking about refactoring and taking a similar approach in issue #111 here - definitely seems like the right way.