Automated End-To-End Testing (Regression Tests)

[nforbus]

Pursue a method to do end-to-end testing, as a means to treat the randstruct plugin as a blackbox.

[nforbus]

Modified the poc/Makefile to include a tests flag. It creates two outputs, one with the plugin in use, and one without the plugin in use.

You can then run the test code that Cole added, select the filenames of the two output files, and it will output the difference between the two files in llvm IR.

[jeffreytakahashi]

Now looking at LLDB testing!
https://lldb.llvm.org/tutorial.html

As it turns out, the entire LLDB API is available via Python. This is the future:
https://lldb.llvm.org/python_reference/index.html

[nforbus]

Updated the makefile commands to show the variable names for better readability. Cannot use the same method to create more human-readable test results though.

Embedding within CMake still to happen, more research needed because idk jack about CMake.

[jcantrell]

Figured out how to use lldb to find locations of struct memories to compare their locations when compiled with and without the plugin.

[jcantrell]

Working on a python script to automate this check and output 1 or 0 depending on if the structs were shuffled.

[tim-pugh]

We'll be attending a unit testing workshop I've scheduled from 6:30pm until 8pm Jan 29th, 2019 at the Engineering Building. Google cal invite should be in the mail. We'll show up a hour early to prepare as a team.

[jcantrell]

The plan (as of Jan. 30) is to write a python script https://lldb.llvm.org/python-reference.html, https://lldb.llvm.org/varformats.html that wraps lldb, and use lldb's features to analyze order of structs of a test program or set of test programs.
Useful lldb commands:
target create "program"
process launch --stop-at-entry
thread continue
b filename:linenum (sets a breakpoint)
frame variable varname
frame var -L
l
l -