fuzzing: Update to afl++, adding persisten-fuzzing example

[Teufelchen1]

Contribution description

Hello! 🦐

This upgrades the fuzzing infrastructure from AFL to AFL++.

The PR also adds an example usage of AFL++ persistent fuzzing, which increases the execution speed by about 4500%; from ~1800 exec/s to 81.000 exec/s, at least on my machine :). I picked, once again, the uri_parser as the example use-case.
To enable persistent fuzzing, one has to use afl-clang-fast instead of the old afl-gcc, I adjusted the Makefile accordingly - which might break backwards compatibility with certain afl-gcc-only setups. All fuzzing examples present in RIOT work fine with this change, with the exception of gcoap, where a small change of the used modules was necessary.

Testing procedure

Testing is done as usual with fuzzing:
cd RIOT/fuzzing/<target>/
make all-asan
make fuzz

[benpicco]

Why is this needed now?

[Teufelchen1]

Thanks for waiting.

Otherwise coap_resources and coap_resources_numof are not defined. See /sys/net/application_layer/nanocoap/nanocoap.c:448. The file gets pulled in as soon as you use the gcoap module.

With normal llvm/gcc this isn't an issue, only AFL++ is affected.

As far as I understand it, this is usually not a problem as the code paths containing the undefined values aren't used and get removed during build. AFL++ seems to do this differently and immediately complains about the undefined references even though they aren't used.

[Teufelchen1]

Mhm sounds a bit vague, to make my explanation clear:

You can comment out coap_well_known_core_default_handler and coap_handle_req in nanocoap.c and still compile & run examples/gcoap/ using gcc/llvm. In this modified state, the gcoap fuzzing examples also builds fine via AFL++ without the nanocoap_resources module.

[benpicco]

that's a bit concerning because the if (IS_USED(MODULE_…)) { pattern is used quite a lot in RIOT.
What is the error you are getting?

[Teufelchen1]

/usr/bin/ld: RIOT/fuzzing/gcoap/bin/native/nanocoap/nanocoap.o: in function `coap_handle_req':
RIOT/sys/net/application_layer/nanocoap/nanocoap.c:449: undefined reference to `coap_resources_numof'
/usr/bin/ld: RIOT/sys/net/application_layer/nanocoap/nanocoap.c:448: undefined reference to `coap_resources'
/usr/bin/ld: RIOT/fuzzing/gcoap/bin/native/nanocoap/nanocoap.o: in function `coap_well_known_core_default_handler':
RIOT/sys/net/application_layer/nanocoap/nanocoap.c:(.text.coap_well_known_core_default_handler[coap_well_known_core_default_handler]+0x387): undefined reference to `coap_resources_numof'
/usr/bin/ld: RIOT/sys/net/application_layer/nanocoap/nanocoap.c:1262: undefined reference to `coap_resources'

[benpicco]

Ah I see what's happening, looks like --gc-sections does not happen with AFL++.

In the 'old style' applications were expected to define coap_resources/coap_resources_numof, now with nanocoap_resources this is an XFA and the definitions are inside nanoCoAP. But to not break applications we keep support for the legacy definitions unless the application requests nanocoap_resources.

The GCoAP fuzzing example doesn't make use of nanoCoAP resources at all (as GCoAP has it's own way to define resources as a linked list of resource arrays) but the functions that would interact with nanocoap_resources are still compiled as they are part of common code, they are just not used by GCoAP so they are normally discarded on linking.

[riot-ci]

Murdock results

❌ FAILED

4c696ac fuzzing: Update to afl++, adding persisten-fuzzing example

Success	Failures	Total	Runtime
2339	0	6934	04m:05s

Artifacts

[benpicco]

CI is not happy - does this need an update of the CI riotdocker?

[Teufelchen1]

CI is not happy - does this need an update of the CI riotdocker?

Not sure, it is by design that the macros e.g. __AFL_FUZZ_INIT() are not known outside of AFL. Correct me if I'm wrong but we do not use AFL(any version) within CI right?
The best course of action might be to disable building this fuzzing harness during CI?

[benpicco]

we do not use AFL(any version) within CI right?

The fuzzing examples get build by CI - and they should be, otherwise they will just bitrot away. If we need to install AFL++ on CI, we should just do that by adding it to the riotdocker Dockerfile.

[Teufelchen1]

Remind myself to check in on this again