test: add pika benchmark tools (#2663)

* add pika benchmark

---------

Co-authored-by: liuyuecai <liuyuecai@360.cn>

tools/pika_benchmark/README.md

@@ -0,0 +1,55 @@
+This tool provides stress testing commands for commonly used indicators, and supports one-click generation of visual statistical charts in SVG format from stress testing results to improve stress testing efficiency. The usage steps are as follows:
+
+1、First of all, the stress testing tool uses Redis’ official memtier_benchmark, so you must choose to install this tool. For the installation process, please refer to the official documentation: [https://github.com/RedisLabs/memtier_benchmark](https://github.com /RedisLabs/memtier_benchmark).
+
+2、Next, start a Pika process, and then execute the stress test script to perform the stress test:
+
+```shell
+sh pika_benchmark.sh -host 127.0.0.1 -port 9221
+```
+Currently the following test parameters are supported:
+```shell
+-host <host>       Server hostname, default: 127.0.0.1
+-port <port>       Server port, default: 9221
+-requests <requests>   Number of requests, default: 10000
+-clients <clients>    Number of concurrent clients, default: 50
+-threads <threads>    Number of threads, default: 4
+-dataSize <dataSize>   Data size, default: 32
+```
+
+3、After the stress test is completed, parse and format the stress test data. First execute go build to compile and convert the program:
+```shell
+go build parser.go
+```
+
+Next, execute the program to format the stress test data. If the output folder does not exist, it needs to be created manually in advance:
+```shell
+mkdir -p parsed_data
+
+./parser -in_dir=$(pwd)/bench_data -out_dir=$(pwd)/parsed_data
+```
+
+4、Use a python script to generate a statistical chart image from the data:
+```shell
+sh gen_chart.sh
+```
+
+After the execution is completed, four svg files will be produced in the ./charts directory. You can directly open them with a browser to see the effect.
+
+5、Currently, the following four pressure measurement charts can be automatically generated:
+
+5.1 Percentile delay statistics chart of commonly used commands:
+
+![img.png](img/cmd_latency.png)
+
+5.2 OPS statistics chart of commonly used commands:
+
+![img.png](img/cmd_ops.png)
+
+5.3 Percentile delay statistics chart under different reading and writing scenarios:
+
+![img.png](img/rw_latency.png)
+
+5.4 OPS statistics chart under different reading and writing scenarios:
+
+![img_1.png](img/rw_ops.png)

tools/pika_benchmark/README_ZH.md

@@ -0,0 +1,52 @@
+本工具提供了常用指标的压测命令，并支持将压测结果一键生成可视化的 svg 格式的统计图，提升压测效率。使用步骤如下：
+
+1、首先，压测工具使用了 Redis 官方的 memtier_benchmark，所以要选把这个工具安装好，安装流程参考官方文档：[https://github.com/RedisLabs/memtier_benchmark](https://github.com/RedisLabs/memtier_benchmark)。
+
+2、接下来启动一个 Pika 进程，然后执行压测脚本进行压测：
+```shell
+sh pika_benchmark.sh -host 127.0.0.1 -port 9221
+```
+目前可支持以下测试参数：
+```shell
+-host <host>       Server hostname, default: 127.0.0.1
+-port <port>       Server port, default: 9221
+-requests <requests>   Number of requests, default: 10000
+-clients <clients>    Number of concurrent clients, default: 50
+-threads <threads>    Number of threads, default: 4
+-dataSize <dataSize>   Data size, default: 32
+```
+
+3、压测完成后，对压测数据进行解析和格式化。首先执行 go build 编译转换程序：
+```shell
+go build parser.go
+```
+接下来执行程序对压测数据进行格式化，如果输出文件夹不存在需要提前手动创建：
+```shell
+mkdir -p parsed_data
+
+./parser -in_dir=$(pwd)/bench_data -out_dir=$(pwd)/parsed_data
+```
+
+4、使用 python 脚本将数据生成统计图图片：
+```shell
+sh gen_chart.sh
+```
+执行完成后，会在 ./charts 目录生产四个 svg 文件，直接使用浏览器打开即可看到效果。
+
+5、目前可以自动生成以下四种压测图表：
+
+5.1 常用命令的百分位延时统计图：
+
+![img.png](img/cmd_latency.png)
+
+5.2 常用命令的 OPS 统计图：
+
+![img.png](img/cmd_ops.png)
+
+5.3 不同读写场景下的百分位延时统计图：
+
+![img.png](img/rw_latency.png)
+
+5.4 不同读写场景下的 OPS 统计图：
+
+![img_1.png](img/rw_ops.png)

tools/pika_benchmark/gen_chart.py

@@ -0,0 +1,136 @@
+#!/usr/bin/env python3
+
+import argparse
+import json
+import pygal
+from os import path
+from os import walk
+from pygal.style import Style
+
+chartStyle = Style(
+    background='transparent',
+    plot_background='transparent',
+    font_family='googlefont:Montserrat',
+    # colors=('#D8365D', '#78365D')
+    colors=('#66CC69', '#173361', '#D8365D'),
+    # colors=('#66CC69', '#667C69', '#173361', '#D8365D', '#78365D'),
+)
+
+# theme = pygal.style.CleanStyle
+theme = chartStyle
+fill = False
+
+
+def create_quantile_chart(workload, title, y_label, time_series):
+    import math
+    chart = pygal.XY(style=theme, dots_size=0.5,
+                     legend_at_bottom=True,
+                     truncate_legend=37,
+                     x_value_formatter=lambda x: '{:,.2f} %'.format(
+                         100.0 - (100.0 / (10 ** x))),
+                     show_dots=False, fill=fill,
+                     stroke_style={'width': 2},
+                     print_values=True, print_values_position='top',
+                     show_y_guides=True, show_x_guides=False)
+    chart.title = title
+    # chart.stroke = False
+
+    chart.human_readable = True
+    chart.y_title = y_label
+    chart.x_labels = [0.30103, 1, 2, 3]
+
+    for label, values, opts in time_series:
+        values = sorted((float(x), y) for x, y in values.items())
+        xy_values = [(math.log10(100 / (100 - x)), y)
+                     for x, y in values if x <= 99.9]
+        chart.add(label, xy_values, stroke_style=opts)
+
+    chart.render_to_file('%s/%s.svg' % (args.outPath, workload))
+
+
+def create_bar_chart(workload, title, y_label, x_label, data):
+    chart = pygal.Bar(
+        style=theme, dots_size=1, show_dots=False, stroke_style={'width': 2}, fill=fill,
+        show_legend=False, show_x_guides=False, show_y_guides=False, print_values_position='top',
+        print_values=True, show_y_labels=True, show_x_labels=True,
+    )
+    chart.title = title
+    chart.x_labels = x_label
+    chart.y_title = y_label
+    chart.value_formatter = lambda y: "{:,.0f}".format(y)
+
+    for label, points in data.items():
+        chart.add(label, points)
+    print("workload", workload)
+    chart.render_to_file('%s/%s.svg' % (args.outPath, workload))
+
+
+#     chart.render_to_file('%s.svg' % workload)
+
+if __name__ == "__main__":
+    parser = argparse.ArgumentParser(
+        description='Plot Kafka OpenMessaging Benchmark results')
+    parser.add_argument('--filePath', dest='filePath', required=True, type=str,
+                        help='Explicitly specify result files to plot')
+    parser.add_argument('--prefix', dest='prefix', help='prefix of filename')
+    parser.add_argument('--outPath', dest='outPath', help='out path to save the plot')
+    args = parser.parse_args()
+
+    aggregate = []
+
+    for (dirpath, dirnames, filenames) in walk(args.filePath):
+        for file in filenames:
+            file_path = path.join(dirpath, file)
+            data = json.load(open(file_path))
+            data['file'] = file
+            aggregate.append(data)
+
+    opsPerSes = []
+    latencyMap = []
+
+    drivers = []
+
+    pub_rate_avg = {}
+    pub_rate_avg["Throughput (MB/s)"] = []
+
+    colors = ['#2a6e3f', '#ee7959', '#ffee6f', '#e94829', '#667C69', '#173361', '#D8365D', '#33A1C9', '#e47690',
+              '#FF5733', '#fac03d', "#f091a0"]
+
+    # Aggregate across all runs
+    count = 0
+    for data in aggregate:
+
+        if ('opsPerSes' in data and data['opsPerSes'] is not None):
+            opsPerSes.append(data['opsPerSes'])
+
+        if ('latencyMap' in data and data['latencyMap'] is not None):
+            latencyMap.append(data['latencyMap'])
+
+        drivers.append(data['file'])
+
+        if ('opsPerSes' in data and data['opsPerSes'] is not None):
+            pub_rate_avg["Throughput (MB/s)"].append(
+                {
+                    'value': data['opsPerSes'],
+                    'color': colors[count]
+                })
+            count = count + 1
+
+    # Parse plot options
+    opts = []
+    for driver in drivers:
+        opts.append({})
+
+    # Generate publish rate bar-chart
+    svg = f'pika-{args.prefix}-ops'
+    print(pub_rate_avg)
+    if ("Throughput (MB/s)" in pub_rate_avg and len(pub_rate_avg["Throughput (MB/s)"]) > 0):
+        create_bar_chart(svg, 'Cmd Ops', 'Ops/second',
+                         drivers, pub_rate_avg)
+
+    if (len(latencyMap) > 0):
+        time_series = zip(drivers, latencyMap, opts)
+        svg = f'pika-{args.prefix}-latency-quantile'
+        create_quantile_chart(svg, 'Latency Quantiles',
+                              y_label='Latency (ms)',
+                              time_series=time_series)

tools/pika_benchmark/gen_chart.sh

@@ -0,0 +1,8 @@
+#!/bin/bash
+
+mkdir -p $(pwd)/charts
+
+python3  gen_chart.py --filePath $(pwd)/parsed_data/cmd_ops --prefix=cmd --outPath=$(pwd)/charts
+python3  gen_chart.py --filePath $(pwd)/parsed_data/cmd_latency --prefix=cmd --outPath=$(pwd)/charts
+python3  gen_chart.py --filePath $(pwd)/parsed_data/rw_ops --prefix=rw --outPath=$(pwd)/charts
+python3  gen_chart.py --filePath $(pwd)/parsed_data/rw_latency --prefix=rw --outPath=$(pwd)/charts

tools/pika_benchmark/go.mod

@@ -0,0 +1,3 @@
+module pika-benchmark
+
+go 1.19