If an input crashes the fuzzer, the coverage task exits

[jotaylo]

Information

• Onefuzz version: 7.0.0
• OS: Windows

I have a set of inputs, some of which crash my fuzzer when run in the context of a coverage task. When this happens, the coverage recording stops, and no other inputs get processed for coverage.

[2023-04-18 22:00:32.426955100 UTC] INFO: agent ready, dispatching task
[2023-04-18 22:00:32.427126300 UTC] task_start: event_type coverage
[2023-04-18 22:00:32.427161700 UTC] INFO: starting coverage task
[2023-04-18 22:01:08.593162400 UTC] WARN: error recording coverage for input = task_readonlyinputs_0\000d6e2ba72cb88fac1da4b61252fb6a90169178: coverage recording failed, child status = exit code: 0xc0000005
[2023-04-18 22:01:11.649161700 UTC] ERROR: error running task: failed to record coverage for input = task_readonlyinputs_0\000d6e2ba72cb88fac1da4b61252fb6a90169178 after 2 attempts

 

Caused by:
    coverage recording failed, child status = exit code: 0xc0000005

1. Can the error message be changed from "child status" to "target.exe status" or similar, it took me a while to figure out what the error meant.
2. After failing to record coverage for this input, it should continue on to processing the next input, not exit.

AB#140609

[ranweiler]

The behavior requested here is definitely ideal. I'll add some historical context on the current behavior, since a change to address this issue should be sensitive to the original motivations.

What happens right now (fail the task after exhausting retries for any single input) is by design. This is due to the current absence of built-in, per-task "push to user" alerting mechanisms. In some cases, such alerting could be built using webhooks, but doesn't ship by default, and there aren't typically webhooks at this granular a level.A

Multiple coverage recording errors typically indicate a task target configuration error whose observability is program path-dependent (e.g. does not fail until a doomed dynamic loading call), or represent a OneFuzz error of some sort (e.g. when intrusively setting software breakpoints).

To properly implement a resolution for this issue, it would be good to durably capture that the total coverage does not include some set of inputs. For example, one solution could be introducing a no_coverage container, similar to the no_repro container for crash analysis. When coverage recording fails and exhausts the retry limit, the failing input could be written to the no_coverage container for later analysis. Webhooks could naturally be associated with this, in the same way they already are with no_repro container updates.