grpcreplay

GrpcReplay is a network monitoring tool that can record your grpc traffic (Unary RPC) and use it for grayscale testing, stress testing or traffic analysis.

• 中文 README

Feature

• support filter
• support to parse Protobuf, requires grpc reflection GRPC Server Reflection Protocol
• Supports various input and output plugins
• Supports multiple encoding forms of gRPC requests (can be easily extended)
• Support gRPC request replay and replay at multiple speeds

Test

Python/Java/Golang gRPC all passed the test

Python Demo

• demo
• how to use demo

Java Demo

• demo
• how to use demo

Golang Demo

• demo

Compile

install libpcap

Ubuntu

apt-get install -y libpcap-dev

Centos

yum install -y libpcap-devel

Mac

brew install libpcap

compile

make build

Principle

1. Since gRPC uses Hpack to compress the header, in order to solve this problem, a mechanism similar to tcpkill is used to kill the old connection and force the client to initiate a new connection.
2. Use gRPC's reflection mechanism to get the definition of Message so that gRPC requests can be parsed.

Architecture

Notice

1. Temporarily only supports h2c, not h2
2. The current gRPC encoding only supports Protobuf. refer to encoding
3. The gRPC server needs to enable reflection GRPC Server Reflection Protocol
4. Only supports Unary RPC, not Streaming RPC
5. Root permissions required on macOS

sudo -s

Usage

Capture gRPC request on "0.0.0.0:35001" and print in console

./grpcr --input-raw="0.0.0.0:35001" --output-stdout

Capture gRPC request on "127.0.0.1:35001" and print in console

./grpcr --input-raw="127.0.0.1:35001" --output-stdout

Capture the gRPC request on "127.0.0.1:35001", send it to "127.0.0.1:35002", and print it in the console

./grpcr --input-raw="127.0.0.1:35001" --output-stdout --output-grpc="grpc://127.0.0.1:35002"

Set the value of codec, optional value: "simple" | "json"

./grpcr --input-raw="127.0.0.1:35001" --output-stdout --codec="simple"

Capture the gRPC request on "127.0.0.1:35001" and record it in a folder. The file is divided according to the maximum limit of 500MB and compressed. Note: The directory must already exist and be writeable.

./grpcr --input-raw="127.0.0.1:35001" --output-file-directory="/tmp/mycapture" --output-file-max-size=500

Read gRPC requests from a folder and replay them. gRPC requests will be sent to "127.0.0.1:35002" and printed in the console

./grpcr --input-file-directory="/tmp/mycapture" --output-stdout --output-grpc="grpc://127.0.0.1:35002"

Hint: You can use input-file-replay-speed to speed up the replay

--input-file-replay-speed=10

Capture gRPC requests on "127.0.0.1:35001", keep only requests whose method suffix is Time, and print them in the console

./grpcr --input-raw="127.0.0.1:35001" --output-stdout --include-filter-method-match=".*Time$"

Capture the gRPC request on "127.0.0.1:35001" and send it to the test topic of RocketMQ

./grpcr --input-raw="127.0.0.1:35001" --output-rocketmq-name-server="192.168.2.100:9876" --output-rocketmq-topic="test"

Get the gRPC request from RocketMQ, send it to "127.0.0.1:35001", and print it in the console

./grpcr --input-rocketmq-name-server="192.168.2.100:9876" --input-rocketmq-topic="test" --output-stdout --output-grpc="grpc://127.0.0.1 :35001"

The captured request looks like

{
	"headers": {
		":authority": "localhost:35001",
		":method": "POST",
		":path": "/SearchService/Search",
		":scheme": "http",
		"content-type": "application/grpc",
		"te": "trailers",
		"testkey1": "testvalue1",
		"testkey2": "testvalue2",
		"user-agent": "grpc-go/1.48.0"
	},
	"method": "/SearchService/Search",
	"request": "{\"staffName\":\"zhangsan\",\"gender\":true,\"age\":405084}"
}

Debug

Set the log level Optional value: debug | info | warn | error

export SIMPLE_LOG_LEVEL=debug

Dependency

This project uses google/gopacket and therefore depends on libpcap

Thanks

inspired by fullstorydev/grpcurl and buger/goreplay

TODO

• 1)Discover all connections associated with the target port
• 2)Use bypass blocking to end these connections one by one
• 3)Grab the request on the target port and parse it
• 4)GRPC request replay
• 5)Supports writing GRPC requests to the console
• 6)Support for writing GRPC requests to files
• 7)Support for writing GRPC requests to kafka
• 8)Support for writing GRPC requests to RocketMQ
• 9)Support for reading GRPC requests from files
• 10)Support reading GRPC requests from kafka
• 11)Support for reading GRPC requests from RocketMQ
• 12)Support custom filter
• 13)support TLS
• 14)Optimize the processing speed of output_grpc

donate