This document explains the concept of a fuzz target and gives advice on how to make your fuzz targets really good.
A fuzz target is a function that accepts an array of bytes and does something interesting with these bytes using the API under test. We define a specific signature for fuzz targets:
// fuzz_target.cc
extern "C" int LLVMFuzzerTestOneInput(const uint8_t *Data, size_t Size) {
DoSomethingInterestingWithMyAPI(Data, Size);
return 0; // Non-zero return values are reserved for future use.
}Note that the interface for this function is C, and so it can be implemented in any programming language that supports C interface. Despite the name referencing the LLVM libFuzzer fuzz targets are independent from a particular fuzzing engine and can be used with fuzzing, symbolic execution, or any other form of testing.
The basic things to remember about a fuzz target:
- The fuzzing engine will execute it many times with different inputs in the same process.
- It must tolerate any kind of input (empty, huge, malformed, etc).
- It must not
exit()orabort()on any input (if it does, it's a bug). - It may use threads but ideally all threads should be joined at the end of the function.
- It must be as deterministic as possible. Non-determinism (e.g. random decisions not based on the input bytes) will make fuzzing inefficient.
- It must be fast. Try avoiding cubic or greater complexity, logging, or excessive memory consumption.
- Ideally, it should not modify any global state (although that’s not strict).
- Usually, the narrower the target the better. E.g. if your target can parse several data formats, split it into several targets, one per format.
More details follow.
A good fuzz target participates in the project's continuous testing:
- It resides in the same repository as the code and other tests.
- It is being compiled with available/applicable sanitizers as part of the usual testing process and linked e.g. with this standalone runner or similar.
- It is being executed as part of the usual testing process using the seed corpus as inputs.
- Inputs that trigger bugs on the fuzz target are added to the seed corpus as a form of regression testing.
Sometimes an API under test needs to be initialized.
The simplest and the recommended way is to have a statically initialized global object inside LLVMFuzzerTestOneInput:
// fuzz_target.cc
extern "C" int LLVMFuzzerTestOneInput(const uint8_t *Data, size_t Size) {
static bool Initialized = DoInitialization();
...Or (less preferable) in the global scope if that works for you:
// fuzz_target.cc
static bool Initialized = DoInitialization();
extern "C" int LLVMFuzzerTestOneInput(const uint8_t *Data, size_t Size) {
...A fuzz target needs to be deterministic, i.e. given the same input it should
have the same behavior.
This means, for example, that a fuzz target should not use rand() or any other
source of randomness.
Fuzzing is a search algorithm that requires many iterations, and so a good fuzz target should be very fast.
A typical good fuzz target will have an order of 1000 executions per second per one CPU core on average (exec/s) or more. For lightweight targets, 10000 exec/s or more.
If your fuzz target has less than 10 exec/s you are probably doing something wrong. We recommend to profile fuzz targets and eliminate any obvious hot spots.
For CPU-efficient fuzzing a good fuzz target should consume less RAM than it is available on a given (virtual) machine per one CPU core. There is no one-size-fits-all RAM threshold, but as of 2019 a typical good fuzz target would consume less than 1.5Gb.
Ideally, a fuzz target should not modify a global state because otherwise the fuzz target may be non-deterministic.
It may not be possible in every case
(strictly speaking, even calling malloc() modifies a global state).
A good fuzz target should not have any
- timeouts (inputs that take too long to process),
- OOMs (input that cause the fuzz target to consume too much RAM),
- shallow (easily discoverable) bugs. Otherwise fuzzing will stall quickly.
In most cases a good fuzz target should be accompanied with a seed corpus, which is a set of representative inputs for the fuzz target. These inputs combined should cover large portions of the API under test, ideally achieving 100% coverage (different coverage metrics can be applied, e.g. block coverage or edge coverage, depending on a specific case).
Avoid large seed inputs when smaller inputs are sufficient for providing the same coverage.
A seed corpus is stored as a directory where every individual file represents one input, subdirectories are allowed.
When fixing a bug or adding a new functionality to the API, don't forget to extend the seed corpus. Monitor the code coverage achieved by the corpus and try to keep it close to 100%.
It is often insufficient to have a seed corpus with good code coverage to claim good fuzzability, i.e. the ability of a fuzzing engine to discover many code paths in the API under test.
For example, imagine we are fuzzing an API that consumes an encrypted input, and we have a comprehensive seed corpus with such encrypted inputs. This seed corpus will provide good code coverage, but any mutation of the inputs will be rejected early as broken.
So, it is important to ensure that the fuzz target can discover a large subset of reachable control flow edges without using the seed corpus. Tools such as Clang's source-based code coverage can assist with this process.
If fuzzing a given target without a seed corpus for, say, a billion iterations, does not provide coverage comparable to a good seed corpus, consider
- Splitting the target (see Large APIs)
- Using dictionaries
- Using Structure-Aware Fuzzing
If your API consumes inputs of a specific size(s) the best way is to express it in the fuzz targer, like this:
// fuzz_target.cc
extern "C" int LLVMFuzzerTestOneInput(const uint8_t *Data, size_t Size) {
if (Size > kMaxSize || Size < kMinSize) return 0;The fuzzing engine will periodically create mutations with a wrong size, but it is not going to hurt the fuzzing efficiency too much since most of the time will be spent executing valid inputs.
If your API can consume arbitrarily large inputs (e.g. a video codec) but you are confident that inputs of a certain size are sufficient to trigger all functionality (and bugs), you may also limit the input size the same way. In case you are not sure, you may create two targets, one with the limit (fast) and one without (slower, but potentially with more coverage).
If the input type for a fuzz target relies on certain keywords (or magic values) using a dictionary of such keywords might be beneficial.
In future we expect the fuzzing engines to become smarter and stop relying on dictionaries, but today dictionaries are often very practical.
A good fuzz target does not use I/O:
- Avoid debug output to
stderrorstdoutas it slows down fuzzing. - Avoid reading from disk other than during one-time initialization.
- Avoid writing to disk.
It might be tempting to use fmemopen to fuzz APIs that
consume FILE*, but using fmemopen may inhibit important search algorithms in the fuzzing engine.
Prefer using direct in-memory APIs whenever possible.
Of course, if the API under test is built around accessing FILE*,
and there is no in-memory counterpart, then fmemopen might be the only choice.
If you absolutely need to fuzz with threads, minimize the number of threads spawned by the fuzz target on every input (thread creation is slow).
If your API uses thread pools, you may create the thread pool during the one-time initialization, but make sure the workers are idle between the executions of the fuzz target (i.e. when the fuzzing engine is not running the fuzz target in the main threads, other threads should be sleeping).
Fuzzing is a search algorithm with super-linear complexity,
i.e. for a fuzz target with N control flow edges the time it takes to discover
most edges is much greater than O(N), most likely, at least O(N^3).
For any large API prefer to create several fuzz targets that cover subsets of the API. However, having a fuzz target for the full API is sometimes also useful.
Any API with more than 20000-30000 reachable control flow edges should probably be considered large.
Example: TODO add links to the pdfium fuzz target and sub targets.
Avoid linking unreachable code into your fuzz target. Even if some code linked to the fuzzer binary is never executed, it may still slow down fuzzing.
It is often desirable to split the input into two or more independent parts (sub-inputs). See split-inputs for recommended options.
In many cases the API under test consumes highly structured inputs, e.g. compressed, encrypted, checksum-protected, serialization formats, database query languages, programming languages, etc. Consider Structure-Aware Fuzzing for such targets.