Add implementation for perfetto project.

[fmayer]

This delegates to scripts in our repo to make changes easier.

Tested using

python infra/helper.py build_image perfetto
python infra/helper.py build_fuzzers --sanitizer address perfetto
python infra/helper.py run_fuzzer perfetto unwinding_fuzzer

[inferno-chromium]

can you please fix the issue with cpu_reader_fuzzer, this is blocking builds from being archived.

python infra/helper.py check_build perfetto

BAD BUILD: /out/cpu_reader_fuzzer seems to have either startup crash or exit:
INFO: Seed: 1005731611
INFO: Loaded 1 modules   (26191 inline 8-bit counters): 26191 [0xa33ef0, 0xa3a53f), 
INFO: Loaded 1 PC tables (26191 PCs): 26191 [0xa3a540,0xaa0a30), 
INFO: -max_len is not provided; libFuzzer will not generate inputs larger than 4096 bytes
proto_translation_table. PERFETTO_CHECK(!page_header.empty())
AddressSanitizer:DEADLYSIGNAL
=================================================================
==117==ERROR: AddressSanitizer: ILL on unknown address 0x0000006357b8 (pc 0x0000006357b8 bp 0x7ffcf8c971b0 sp 0x7ffcf8c97060 T0)
SCARINESS: 10 (signal)
    #0 0x6357b7 in perfetto::ProtoTranslationTable::Create(perfetto::FtraceProcfs const*, std::__1::vector<perfetto::Event, std::__1::allocator<perfetto::Event> >, std::__1::vector<perfetto::Field, std::__1::allocator<perfetto::Field> >) /work/build/../../src/perfetto/src/traced/probes/ftrace/proto_translation_table.cc
    #1 0x64ac16 in perfetto::GetTable(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) /work/build/../../src/perfetto/src/traced/probes/ftrace/test/cpu_reader_support.cc:38:18
    #2 0x5a47af in perfetto::FuzzCpuReaderParsePage(unsigned char const*, unsigned long) /work/build/../../src/perfetto/src/traced/probes/ftrace/cpu_reader_fuzzer.cc:46:34
    #3 0x5a4e40 in LLVMFuzzerTestOneInput /work/build/../../src/perfetto/src/traced/probes/ftrace/cpu_reader_fuzzer.cc:71:3

Also fixed our docs to explicitly point towards doing this step in 6dc263e

[inferno-chromium]

Also, looks like your build script is sending exit code 1 which is cause build step to fail.

Step #2: Already have image: gcr.io/oss-fuzz/perfetto
Finished Step #2
ERROR
ERROR: build step 2 "gcr.io/oss-fuzz/perfetto" failed: exit status 1

See https://oss-fuzz-build-logs.storage.googleapis.com/index.html, click on Perfetto.

[fmayer]

Sorry about that. Fixed the bad build for cpu_reader_fuzzer.

I am not sure what causes the error in Step #2? The actions in build.sh seem to be executed in Step #1 from what I can tell. Step #2 only seems to say "Already have image: gcr.io/oss-fuzz/perfetto", rather than run our build script. I do not get this error locally

$ python infra/helper.py build_image perfetto
python infra/helper.py build_fuzzers --sanitizer address perfetto
python infra/helper.py check_build perfetto
...
Successfully built 67d2e1fab4cd
Successfully tagged gcr.io/oss-fuzz/perfetto:latest
...
Running: docker run --rm -i --privileged -e FUZZING_ENGINE=libfuzzer -e SANITIZER=address -v /usr/local/google/home/fmayer/oss-fuzz/build/out/perfetto:/out -t gcr.io/oss-fuzz-base/base-runner test_all
INFO: performing bad build checks for /out/cpu_reader_fuzzer.
INFO: performing bad build checks for /out/unwinding_fuzzer.
INFO: performing bad build checks for /out/end_to_end_shared_memory_fuzzer.
INFO: performing bad build checks for /out/buffered_frame_deserializer_fuzzer.
4 fuzzers total, 0 seem to be broken (0%).
Check build passed.

[Dor1s]

Step #2 builds the image, i.e. python infra/helper.py build_image perfetto.

[Dor1s]

Sorry, building the image is the second step, but it's Step #1 as the first step is Step #0. So, Step #3 is just a single bash command srcmap > /workspace/srcmap.json && cat /workspace/srcmap.json. The fact that it fails doesn't make any sense to me.

[Dor1s]

A wild guess is that the git installed in perfetto's Dockerfile might be causing some issues with generating the srcmap.json. Removed in 1d9dc4c, let's see if it helps.

[Dor1s]

Hm, nope.

[Dor1s]

Now I suspect that the problem is that srcmap prints an empty output. @segfaulthunter what does tools/install-build-deps do? Can it be overwriting some important tools / utilities? srcmap implementation is available here: https://github.com/google/oss-fuzz/blob/7b179a4a243556a318db1e7e491488079c1e1b72/infra/base-images/base-builder/srcmap

[Dor1s]

Chatted offline with @segfaulthunter. Indeed, tools/install-build-deps was a culprit, as it was downloading some git repositories which didn't have the origin set up. @segfaulthunter is fixing that upstream, and I've landed 2d1a956 to make the errors from srcmap more obvious.

[Dor1s]

Good news is that we've fixed the build. Bad news is that there is now a build check failure:

 Step #5: INFO: performing bad build checks for /workspace/out/address/cpu_reader_fuzzer. I 
 Step #5: INFO: performing bad build checks for /workspace/out/address/buffered_frame_deserializer_fuzzer. I 
 Step #5: INFO: performing bad build checks for /workspace/out/address/unwinding_fuzzer. I 
 Step #5: INFO: performing bad build checks for /workspace/out/address/end_to_end_shared_memory_fuzzer. I 
 Cloud Build GetBuild 8f6138c4-7561-4203-9cb5-fdc33f97c4c3 {"@type":"type.googleapis.com/google.cloud.audit.AuditLog","authenticationInfo":{},"requestMetadata":{"callerIp":"35.202.133.208","callerSuppliedUserAgent":"google-api-python-client/1.7.0 (gzip),gzip(gfe),gzip(gfe)"},"serviceName":"cloudbuild.googleapis.com","methodName":"google.devtools.cloudbuild.… I  Cloud Build GetBuild 8f6138c4-7561-4203-9cb5-fdc33f97c4c3 
 Cloud Build GetBuild 8f6138c4-7561-4203-9cb5-fdc33f97c4c3 {"@type":"type.googleapis.com/google.cloud.audit.AuditLog","authenticationInfo":{},"requestMetadata":{"callerIp":"35.202.133.208","callerSuppliedUserAgent":"google-api-python-client/1.7.0 (gzip),gzip(gfe),gzip(gfe)"},"serviceName":"cloudbuild.googleapis.com","methodName":"google.devtools.cloudbuild.… I  Cloud Build GetBuild 8f6138c4-7561-4203-9cb5-fdc33f97c4c3 
 Step #5: Broken fuzz targets (1): I 
 Step #5: cpu_reader_fuzzer: I 
 Step #5: BAD BUILD: /workspace/out/address/cpu_reader_fuzzer seems to have either startup crash or exit: I 
 Step #5: INFO: Seed: 2335547660 I 
 Step #5: INFO: Loaded 1 modules   (26205 inline 8-bit counters): 26205 [0xa33ef0, 0xa3a54d),  I 
 Step #5: INFO: Loaded 1 PC tables (26205 PCs): 26205 [0xa3a550,0xaa0b20),  I 
 Step #5: INFO: -max_len is not provided; libFuzzer will not generate inputs larger than 4096 bytes I 
 Step #5: �[90mcpu_reader_fuzzer.cc:50 �[0m �[31mCould not read table. This fuzzer must be run in the root directory. (errno: 2, No such file or directory)�[0m I 
 Step #5: AddressSanitizer:DEADLYSIGNAL I 
 Step #5: ================================================================= I 
 Step #5: ==114==ERROR: AddressSanitizer: ILL on unknown address 0x0000005a4bfb (pc 0x0000005a4bfb bp 0x7ffcb1e242f0 sp 0x7ffcb1e24220 T0) I 
 Step #5: SCARINESS: 10 (signal) I 
 Step #5:     #0 0x5a4bfa in perfetto::FuzzCpuReaderParsePage(unsigned char const*, unsigned long) /work/build/../../src/perfetto/src/traced/probes/ftrace/cpu_reader_fuzzer.cc:48:5 I 
 Step #5:     #1 0x5a4e40 in LLVMFuzzerTestOneInput /work/build/../../src/perfetto/src/traced/probes/ftrace/cpu_reader_fuzzer.cc:71:3 I 
 Step #5:     #2 0x549885 in fuzzer::Fuzzer::ExecuteCallback(unsigned char const*, unsigned long) /src/libfuzzer/FuzzerLoop.cpp:571:15 I 
 Step #5:     #3 0x54c63a in fuzzer::Fuzzer::ReadAndExecuteSeedCorpora(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, fuzzer::fuzzer_allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&) /src/libfuzzer/FuzzerLoop.cpp:761:3 I 
 Step #5:     #4 0x54de58 in fuzzer::Fuzzer::Loop(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, fuzzer::fuzzer_allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&) /src/libfuzzer/FuzzerLoop.cpp:806:3 I 
 Step #5:     #5 0x52b651 in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) /src/libfuzzer/FuzzerDriver.cpp:764:6 I 
 Step #5:     #6 0x51e70c in main /src/libfuzzer/FuzzerMain.cpp:20:10 I 
 Step #5:     #7 0x7f0fcf61a82f in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x2082f) I 
 Step #5:     #8 0x41ebc8 in _start (out/address/cpu_reader_fuzzer+0x41ebc8) I 
 Step #5:  I 
 Step #5: AddressSanitizer can not provide additional info. I 
 Step #5: SUMMARY: AddressSanitizer: ILL /work/build/../../src/perfetto/src/traced/probes/ftrace/cpu_reader_fuzzer.cc:48:5 in perfetto::FuzzCpuReaderParsePage(unsigned char const*, unsigned long) I 
 Step #5: ==114==ABORTING I 
 Step #5: MS: 0 ; base unit: 0000000000000000000000000000000000000000 I 
 Step #5:  I 
 Step #5:  I 
 Step #5: artifact_prefix='./'; Test unit written to ./crash-da39a3ee5e6b4b0d3255bfef95601890afd80709 I 
 Step #5: Base64:  I 
 Step #5: ERROR: 25% of fuzz targets seem to be broken. See the list above for a detailed information. I 
 Finished Step #5 I 

To reproduce, please follow https://github.com/google/oss-fuzz/blob/master/docs/reproducing.md#reproducing-build-failures

[Dor1s]

@segfaulthunter thanks for the follow up fixes, now the build is failing with a bunch of errors of the following type:

 Step #12: [342/632] python ../../src/perfetto/gn/standalone/build_tool_wrapper.py ./gcc_like_host/protoc --proto_path ../../src/perfetto/protos --cpp_out gen/protos ../../src/perfetto/protos/perfetto/common/commit_data_request.proto ../../src/perfetto/protos/perfetto/common/sys_stats_counters.proto I 
 Step #12: FAILED: gen/protos/perfetto/common/commit_data_request.pb.h gen/protos/perfetto/common/commit_data_request.pb.cc gen/protos/perfetto/common/sys_stats_counters.pb.h gen/protos/perfetto/common/sys_stats_counters.pb.cc  I 
 Step #12: python ../../src/perfetto/gn/standalone/build_tool_wrapper.py ./gcc_like_host/protoc --proto_path ../../src/perfetto/protos --cpp_out gen/protos ../../src/perfetto/protos/perfetto/common/commit_data_request.proto ../../src/perfetto/protos/perfetto/common/sys_stats_counters.proto I 
 Step #12: AddressSanitizer:DEADLYSIGNAL I 
 Step #12: ================================================================= I 
 Step #12: ==1250==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000000 (pc 0x000000000000 bp 0x00000000015c sp 0x7ffef874e338 T0) I 
 Step #12: ==1250==Hint: pc points to the zero page. I 
 Step #12: ==1250==The signal is caused by a READ memory access. I 
 Step #12: ==1250==Hint: address points to the zero page. I 
 Step #12:  I 
 Step #12: AddressSanitizer can not provide additional info. I 
 Step #12: SUMMARY: AddressSanitizer: SEGV (<unknown module>)  I 
 Step #12: ==1250==ABORTING I 
 Step #12: ninja: build stopped: subcommand failed. I 

To reproduce it locally, you would need to specify AFL fuzzing engine:

infra/helper.py build_fuzzers --engine=afl perfetto

[fmayer]

I restricted to only libfuzzer in #2021 while we look into this issue on our side. Thanks for the follow up!