feature: Print Debug output when using Arbitrary

[ParkMyCar]

When fuzzing with libfuzzer and using arbitrary::Arbitrary, if you find a failing input then cargo fuzz prints the Debug representation of your Arbitrary struct. This is quite useful because it's easier to create a unit test and repro the failing scenario.

Could afl.rs add a similar feature that when using Arbitrary it'll output the Debug representation of your failing struct?

[smoelius]

In principle, it could, but I think this would require significant changes to afl.rs.

Right now, AFL's output directory is opaque to afl.rs. A change like this would require afl.rs to have some knowledge of that directory's structure, which would add maintenance costs to afl.rs.

Also, as far as I am aware, there is no easy way to tell when AFL has found a crash, e.g., via a return code. You can pass --run-until-crash to AFL, but you still won't know whether AFL exited because of a crash or (say) a timeout. (I would love to be corrected on this.) So some ad hoc method would be needed to distinguish such cases.

For myself, I would lean toward not making a change like this. I personally like the idea of afl.rs being a lightweight wrapper around AFL, and of other tools adding functionality on top of afl.rs. (At the risk of sounding like a sales pitch, test-fuzz is a wrapper around afl.rs that does something like you are describing. test-fuzz doesn't use arbitrary, though.)

You're probably already aware of this, but for your specific problem, you should be able to:

    fuzz!(|my_struct: MyStruct| {
        println!("{:?}", my_struct);
        ...
    }

And then:

target/debug/my_target < output/crashes/crashing_input

I know this is not ideal, though.

[smoelius]

Actually, I think I was being a little rash.

Looking more closely at how libfuzzer does this, I think it would be possible to incorporate something similar to afl.rs without running into the problems I mentioned.

One thing we might do different is, rather than write the file at the target's entry point, we might write it only after a panic occurs (here maybe?). Then, when AFL exits, cargo-afl could check whether the file is non-empty and, if so, print it out. I think this would circumvent the "not knowing when you've found a crash" problem.

Does this idea sound reasonable to you, @ParkMyCar? Is it something you would like to try to tackle?