SmartBugs is an execution framework aiming at simplifying the execution of analysis tools on datasets of smart contracts.
-
A plugin system to easily add new analysis tools, based on Docker images;
-
Parallel execution of the tools to speed up the execution time;
-
An output mechanism that normalizes the way the tools are outputting the results, and simplifies the process of the output across tools.
-
Automatic detection and download of the correct version of the Solidity compiler as required by the contract under analysis.
- HoneyBadger
- Maian
- Manticore
- Mythril
- Osiris
- Oyente
- Securify
- Slither
- Smartcheck
- Solhint
- Conkas
Once you have Docker and Python3 installed your system, follow the steps:
- Clone SmartBugs's repository:
git clone https://github.com/smartbugs/smartbugs.git
- Install all the Python requirements:
pip3 install -r requirements.txt
SmartBugs provides a command-line interface that can be used as follows:
smartBugs.py [-h, --help]
--list tools # list all the tools available
--list datasets # list all the datasets available
--dataset DATASET # the name of the dataset to analyze (e.g. reentrancy)
--file FILES # the paths to the folder(s) or the Solidity contract(s) to analyze
--tool TOOLS # the list of tools to use for the analysis (all to use all of them)
--info TOOL # show information about tool
--skip-existing # skip the execution that already has results
--processes PROCESSES # the number of process to use during the analysis (by default 1)
--output-version # specifies SmartBugs' output version {v1 (Json), v2 (SARIF), all}
--aggregate-sarif # aggregates SARIF output per analysed file
--unique-sarif-output # aggregates all analysis in a single file
--import-path PATH # defines project's root directory so that analysis tools are able to import from other filesFor example, we can analyse all contracts labelled with type reentrancy with the tool oyente by executing:
python3 smartBugs.py --tool oyente --dataset reentrancyTo analyze a specific file (or folder), we can use the option --file. For example, to run all the tools on the file dataset/reentrancy/simple_dao.sol, we can run:
python3 smartBugs.py --tool all --file dataset/reentrancy/simple_dao.solBy default, results will be placed in the directory results.
We make available three smart contract datasets with SmartBugs:
- SB Curated: a curated dataset that contains 143 annotated contracts with 208 tagged vulnerabilities that can be used to evaluate the accuracy of analysis tools.
- SB Wild: a dataset with 47,398 unique contract from the Ethereum network (for details on how they were collected, see the ICSE 2020 paper)
- SolidiFI Benchmark: a remote dataset of contracts injected with 9369 bugs of 7 different types.
SB Curated provides a collection of vulnerable Solidity smart contracts organized according to the DASP taxonomy. It is available in the dataset repository.
| Vulnerability | Description | Level |
|---|---|---|
| Reentrancy | Reentrant function calls make a contract to behave in an unexpected way | Solidity |
| Access Control | Failure to use function modifiers or use of tx.origin | Solidity |
| Arithmetic | Integer over/underflows | Solidity |
| Unchecked Low Level Calls | call(), callcode(), delegatecall() or send() fails and it is not checked | Solidity |
| Denial Of Service | The contract is overwhelmed with time-consuming computations | Solidity |
| Bad Randomness | Malicious miner biases the outcome | Blockchain |
| Front Running | Two dependent transactions that invoke the same contract are included in one block | Blockchain |
| Time Manipulation | The timestamp of the block is manipulated by the miner | Blockchain |
| Short Addresses | EVM itself accepts incorrectly padded arguments | EVM |
| Unknown Unknowns | Vulnerabilities not identified in DASP 10 | N.A |
SB Wild is available in a separated repository due to its size: https://github.com/smartbugs/smartbugs-wild
You can set any git repository as a remote dataset. Smartbugs is distributed with Ghaleb and Pattabiraman's SolidiFI Benchmark, a dataset of buggy contracts injected with 9369 bugs of 7 different types: reentrancy, timestamp dependency, unhandled exceptions, unchecked send, TOD, integer overflow/underflow, and use of tx.origin.
To add new remote datasets, update the configuration file dataset.yaml with
the location of the dataset (url), the local directory where the dataset will be located (local_dir),
and any relevant subsets (if any). As an example, here's the configuration for SolidiFI:
solidiFI:
- url: git@github.com:smartbugs/SolidiFI-benchmark.git
- local_dir: dataset/solidiFI
- subsets: # Accessed as solidiFI/name
- overflow_underflow: buggy_contracts/Overflow-Underflow
- reentrancy: buggy_contracts/Re-entrancy
- tod: buggy_contracts/TOD
- timestamp_dependency: buggy_contracts/Timestamp-Dependency
- unchecked_send: buggy_contracts/Unchecked-Send
- unhandled_exceptions: buggy_contracts/Unhandled-Exceptions
- tx_origin: buggy_contracts/tx.origin
With this configuration, if we want to run slither in the remote sub-directory buggy_contracts/tx.origin,
we can run:
python3 smartBugs.py --tool slither --dataset solidiFI/tx_originTo run it in the entire dataset, use solidiFI instead of solidiFI/tx_origin.
When we use a remote dataset for the first time, we are asked to confirm the creation of the local copy.
- SmartBugs was used to analyze 47,587 smart contracts (work published at ICSE 2020). These contracts are available in a separate repository. The results are also in their own repository.
- SmartBugs was used to evaluate a simple extension of Smartcheck (work published at ASE 2020, Tool Demo Track)
- ... you are more than welcome to add your own work here!
The license in the file LICENSE applies to all the files in this repository,
except for all the smart contracts in the dataset folder.
The smart contracts in this folder are
publicly available, were obtained using the Etherscan APIs, and retain their
original licenses. Please contact us for any additional questions.