macro benchmarks

[realark]

• Base perf project and jetty test project
• Script for running performance tests against different agent versions

[tylerbenson]

Can I suggest moving this project to be a sibling of the benchmark tests under dd-java-agent?

How long does a test iteration take? I'd suggest updating the example to include agent versions in dd-java-agent/benchmark/releases

[realark]

Sure I'm fine with moving the project below dd-java-agent. I've updated the readme to point people to the releases dir.

Each test endpoint takes about 2 minutes (30s warmup, 90s test); total test time of 10 minutes per agent.

I also added the ability to customize the resources per machine. Defaults settings are in perf-test-default-settings.rc.

[tylerbenson]

Looks good. What do the results look like? What kind of impact are we seeing?

[realark]

The results look very good. At 1ms and above the agent has no impact on the throughput.

For the <1MS worst-case scenario (average 60 microseconds latency per request) the agent is noticeable at 20% reduction in throughput. I'm not surprised to see that; we're essentially measuring nothing so our presence is more obvious.

That leaves the question open about the performance between 60us and 1ms response times, but I wasn't sure if that data was interesting enough to collect.