Add option to print virtual address to trace.py

[mirek26]

We'd like to be able to symbolize stacks with external symbolizers like addr2line or llvm-symbolizer to get more information including inlined functions or line numbers. This is adding a simple option to print the virtual address.

Happy to add this as an option to bpf.sym() if that's preferred

[yonghong-song]

Could you add or modify an existing example in trace_example.txt? This way, maybe more people will see it and use it. Could you also change the man page at bcc/man/man8 directory? Thanks!

[mirek26]

Thanks for looking! Updated the man page and added an example to trace_example.txt - there was no example showing -K/-U either, so I tried to choose something reasonable. If you have some better ideas let me know.

[palmtenor]

The use case we have for this option is to tail the outputted virtual addresses, resolve them and log through a different pipeline.

Do you feel it's reasonable to have this option lead to only output virtual addresses instead of in addition to string symbols? The reason is that BCC's default symbol resolution behavior is quite expensive (look for debuginfo first, does CRC calculation for debuginfo). Doing those on large binaries such as HHVM that we use for web server often take couple of seconds with a lot of CPU and memory.

Do you feel that only outputting virtual addresses is a useful behavior for the tool in general, that we could add this option for?

[yonghong-song]

[buildbot, ok to test]

[yonghong-song]

virtual address output is also useful when:
. symbol resolution yields unknown. user may try to use the virtual address to find out why?
. if the same function appears twice inside another function, virtual address may help you
decipher which one is actually in the stack.
I hacked trace.py file exactly for this reason in the past.

Regarding whether this option -a should output only virtual address without symbol resolution, I think addr+sym is useful and we should preserve. It may remove the redundant work since symbol resolution will be done externally. But can we attempt to improve this perspective in bcc first?

[mirek26]

From bcc's perspective I think addr+sym makes more sense. For our use case, printing only addresses would be better, but we'd probably need another flag for that and I don't want to overcomplicate this code. (Then we could take this even further and add "-F ip,sym,dso" as in perf...)

We use this only for ad-hoc debugging and profiling, so I don't think it's a big issue. Or maybe I just haven't run into this being a problem yet :)

[yonghong-song]

LGTM.

cc @brendangregg @goldshtn