[Feature] Custom Serialization code needs performance tests

[peternied]

What is the bug?
The Custom Serialization logic was added to the security plugin codebase in an effort to improve serialization speed in different scenarios; however, a scenario was found where there was a significant regression [1], there should be a suite of tests that could catch and/or add check to ensure this doesn't happen in the future.

How can one reproduce the bug?
Inspect the gradle tasks and pull request checks for tests that measure the runtime performance of the serialization code in the Security Plugin - don't find any.

What is the expected behavior?
There should be a set of performance test cases that can validate the baseline and the behavior with the feature enabled to be sure no significant latency increases occur.

Do you have any additional context?

• [1] Coming from [BUG] DLS performance has regressed with new serialization format #3776
• POC for a performance test main...peternied:security:dls-perf

[Bukhtawar]

We have a benchmarks directory for JMH benchmarks, might be a good starting place for such optimisations

[mgodwan]

I think irrespective of custom serialization, it will be good to have JMH.

I've a commit which I can help to productionize: main...mgodwan:security:jmh

[cwperks]

[Triage] Thank you for filing @peternied. Having performance tests in general for optimizations will help measure the benefit of PRs that provide optimizations w/o any end user change.

[shikharj05]

Custom serialization is removed from security #5339