Some IDA Python scripts for auto-analysis and a Hive-plot visualizer.

IDA Python Scripts

The scripts in this repo are an extraction from a larger set of scripts used for semi-automated analysis in security assurance of Irdeto's anti-reverse-engineering technology. In that environment we developed other, more detailed, analyses on-top-of what you find in this repo. What we have published here are some simple, but also hopefully useful scripts. The scripts were developed over the period spanning 2016-2018; tempered by valuable feedback from Irdeto Ottawa Security Assurance teammates @JonathanBeverley and @kennethblite.

The IDA Python parts of the scripts use the idascript framework from @devttys0 to enable both command-line and IDA Python shell command invocation. There are a couple features in this repo (as mentioned above, there are more advanced analysis features internally which we won't be releasing):

• batch+parallel analysis of a directory tree of binaries
• batch+parallel analysis of the binaries in an APK
• automating rizzo (also from @devttys0) analysis
• using IDA for strings :)

Hiveplot Binary Visualizer

Alongside those scripts above, we have also published an interactive hive-plot tool for visualizing the cross-references in a given IDA Pro .idb.

Hive plots are an idea first described by Krzywinski, Birol, Jones and Marra, please see http://www.hiveplot.com/. Mike Bostock (the author of d3.js) made a hive-plot plugin and a demo based on rendering of references in source code, please see https://bost.ocks.org/mike/hive/.

We have built on-top of the d3-hive.js plugin and added some UI elements to make contextual analysis of binaries easier as well as adding dynamic adjustments to weights and other visual elements as one zooms through the plots.

Samples

Here are some sample hive plots rendered from samples in https://github.com/maldroid/samples and https://github.com/JonathanSalwan/binary-samples. These were produced by running the test/run_tests.sh script:

Installing

You will need:

• bash, with at least the utilities readlink and find
  • on OSX you'll need to install gnu coreutils to get greadlink installed
• GNU parallel
• IDA (6.95 tested) with the following installed:
  • IDAPython (2.7 tested)
  • idascript https://github.com/devttys0/idascript (275be937d98da280fecdaf623fcc3c60ff1237e4 tested)
  • Sark https://github.com/tmr232/Sark (1.0 tested)
• apktool https://github.com/iBotPeaches/Apktool (2.2.1 tested)

For the hive-plots (viz/ stuff), you will also need:

• node.js (v6.9.1 tested)
• curl
• a browser to view the .html files (chrome tested)

With all of those installed, clone the repo and run a script. You will then need to run npm install in the viz/ subdirectory.

The Scripts

There are both .py and shell-scripts in this repo. The .py files are all idascript-based IDA Python scripts; the shell scripts mostly call-out to those to enable batch processing.

Shell Script Name	Description
apk-recipe	An all-in-one script to unpack an apk, find all .dex and binaries and analyze them all in parallel.
get-analysis-targets	get-analysis-targets <dirs...> to recursively descend and extract all ELF and Mach-o files to named dirs in the current directory. 'Thinning' the macho files into individual architectures
analyze-all	analyze-all <target...> to recursively descend and produce .idb for every ELF in parallel
unpack-all	unpack-all <packages...> to unpack all .ipa and .apk to named dirs in the current directory.
riz-all	riz-all <target...> to recursively descend and produce .idb for every .idb in parallel
viz/do	creates a d3.js based vizualization given a json export of cross-references
viz/*{.js,.css,.html}	other parts of the vizualization scripts, mostly dependencies for the d3.js client-side scripts.

IDA Python idascript Name	Description
rename_default_functions_with_a_prefix.py	Useful when preparing rizzo signatures, gives all subXXXX functions a prefix so that they are not ignored by rizzo
idb2riz.py	simple idascript to run RizzoBuild() from the rizzo.py plugin on a X.idb to produce X.riz
just_analysis.py	a 'stub' idascript for use by auto-analyze
viz/all-crossrefs-idb.py	a simple script, to be used with the other script in viz/ , that exports all the static analysis cross-references from an .idb to a json structure

Usage

IDA Python Scripts

A good candidate for using auto-analysis scritps is apk-recipe which will unpack and auto-analyze all binaries (.so and .dex in an apk).

$ git clone https://github.com/cloakware-ctf/idascripts.git
[...]
$ mkdir -p target-apk/downloads
$ mv target.apk target-apk/downloads/
$ cd target-apk/
$ ../idascripts/apk-recipe .

At this point any and all binaries in the .apk will have been extracted and auto-analyzed by IDA Pro. All .idb databases will be in well-named folders under analysis. Note that the script will execute the IDA Pro analysis in parallel (using GNU parallel) to save time.

Hiveplot Binary Visualizer

The other interesting feature of these scripts is to produce hive-plot vizualizations using viz/do. This script will hunt for any IDA pro databases and create hive plot vizualiations of the xrefs therein. Continuing the above analysis:

./viz/do analysis/

At this point there will be a standalone .html file containing an interactive hive-plot for each .idb in analysis/.

Future Work

Hive Plots:

• Add a duplicate axis (like in Mike Bostock's example) for cases where we want to see intra-group links
• Auto-orient the axes so that the most-central axis is most-horizontal
• Enable dragging of axis to rotate axis
• Enable dragging nodes to re-order
• For panel-plots (not pictured in this repo) keep pan independent, but still tie scale
• Use tt for node names
• add floating labels at border of visible areas on edges when they are selected or active
• dark theme

Pull requests are welcome.