wkbench

wkbench is a composable benchmark toolkit for WuKongIM and related black-box messaging workloads.

This repository starts the v2 architecture: a small kernel runs scenario graphs, while independent units provide capabilities through versioned ports. Units do not import each other. Composition happens only in scenario YAML.

Quick Start

Run the dry-run group SEND example:

GOWORK=off go run ./cmd/wkbench run -scenario ./examples/group-send.yaml

Explain a scenario graph before running it:

GOWORK=off go run ./cmd/wkbench explain -scenario ./examples/group-send.yaml
GOWORK=off go run ./cmd/wkbench explain -scenario ./examples/group-send.yaml -format json

explain validates specs and wiring, then prints the execution order and resolved input bindings. It does not run units, create reports, or touch target services.

Plan deterministic unit work before running it:

GOWORK=off go run ./cmd/wkbench plan -scenario ./examples/group-send.yaml
GOWORK=off go run ./cmd/wkbench plan -scenario ./examples/group-send.yaml -format json

plan validates the scenario and calls each unit's Plan phase, then prints per-unit plan status and shard counts. It does not run units, publish outputs, write reports, or touch target services.

List built-in units:

GOWORK=off go run ./cmd/wkbench list-units

wkbench loads bundled official plugins by default for data, control-plane, and migrated background units. Use -no-official-plugins before the command when you need to inspect only the host-local runtime units:

GOWORK=off go run ./cmd/wkbench -no-official-plugins list-units

External Plugins

External plugins are standalone executables. Generate a starter plugin, build it, register it in .wkbench/plugins.yaml, then run normal scenarios:

GOWORK=off go run ./cmd/wkbench plugin init \
  -dir /tmp/acme-wkbench-plugin \
  -module example.com/acme/wkbench-plugin \
  -name acme.echo
cd /tmp/acme-wkbench-plugin
go test ./...
go build -o ./bin/acme-echo-plugin ./cmd/acme-echo-plugin
cd -
GOWORK=off go run ./cmd/wkbench plugin add acme.echo /tmp/acme-wkbench-plugin/bin/acme-echo-plugin
GOWORK=off go run ./cmd/wkbench plugin doctor
GOWORK=off go run ./cmd/wkbench list-units

You can still load a plugin for one command with the global -plugin flag:

GOWORK=off go build -o /tmp/wkbench-demo-plugin ./plugins/demo/cmd/wkbench-demo-plugin
GOWORK=off go run ./cmd/wkbench -plugin /tmp/wkbench-demo-plugin list-units
GOWORK=off go run ./cmd/wkbench -plugin /tmp/wkbench-demo-plugin validate -scenario ./examples/plugin-echo.yaml
GOWORK=off go run ./cmd/wkbench -plugin /tmp/wkbench-demo-plugin run -scenario ./examples/plugin-echo.yaml

Scenario YAML can reference external units as <plugin-name>:<kind>, for example wkbench.demo:demo.echo/v1. list-units, validate, explain, plan, and run automatically load bundled official plugins and enabled project plugins. Official data, control-plane, and migrated background units use the same stdio RPC path as third-party plugins by default; host-local units remain for capability ports, local resources, and token-source interfaces. See docs/plugin-authoring.md for authoring details, plugin config, and Phase 1 limits.

Create a new unit skeleton:

GOWORK=off go run ./cmd/wkbench new-unit -kind demo.group_send_probe/v1 -dir ./units/demo/group_send_probe
GOWORK=off go test ./units/demo/group_send_probe

Run tests:

GOWORK=off go test ./...

Current Units

Bundled official plugins provide these data, control-plane, and background units by default:

• core.static_groups/v1: produces deterministic in-memory group channels.
• identity.pool/v1: produces deterministic user/device identities.
• identity.person_pairs/v1: produces deterministic person-channel send targets.
• wukongim.target/v1: describes and probes black-box WuKongIM endpoints.
• wukongim.prepare_group_channels/v1: prepares group channels and subscribers through /bench/v1.
• wukongim.metrics_collector/v1: scrapes target metrics as a background unit.
• report.assert/v1: asserts traffic summary values.

The host keeps these local until plugin RPC supports capability ports, token-source interfaces, and local resources:

• core.fake_group_sender/v1: produces a fake WKProto group sender for examples and tests.
• core.fake_message_sender/v1: produces a fake generic WKProto message sender for dry-run examples and tests.
• wukongim.prepare_tokens/v1: prepares user tokens through /bench/v1/users/tokens.
• wkproto.session_pool/v1: opens real WKProto sessions and provides legacy port.wkproto.group_sender/v1 senders plus generic port.wkproto.message_sender/v1 senders for traffic.send/v1.
• traffic.group_send/v1: sends group messages through port.wkproto.group_sender/v1.
• traffic.send/v1: sends protocol messages through port.wkproto.message_sender/v1 and measures SEND -> SENDACK latency.

Validate the real WuKongIM example without connecting:

GOWORK=off go run ./cmd/wkbench validate -scenario ./examples/wukongim-group-send.yaml

Inspect the same scenario's graph without connecting:

GOWORK=off go run ./cmd/wkbench explain -scenario ./examples/wukongim-group-send.yaml
GOWORK=off go run ./cmd/wkbench explain -scenario ./examples/wukongim-group-send.yaml -format json

Plan the same scenario without connecting:

GOWORK=off go run ./cmd/wkbench plan -scenario ./examples/wukongim-group-send.yaml
GOWORK=off go run ./cmd/wkbench plan -scenario ./examples/wukongim-group-send.yaml -format json

Validate, inspect, and plan the mixed group/person send-rate scenario without connecting:

GOWORK=off go run ./cmd/wkbench validate -scenario ./examples/wukongim-send-rate-mixed.yaml
GOWORK=off go run ./cmd/wkbench explain -scenario ./examples/wukongim-send-rate-mixed.yaml
GOWORK=off go run ./cmd/wkbench plan -scenario ./examples/wukongim-send-rate-mixed.yaml

Run the single-node WuKongIM smoke after starting a target with bench API enabled:

./scripts/smoke-wukongim-single-node.sh

Run the mixed group/person send-rate smoke against the same target:

./scripts/smoke-wukongim-send-rate-mixed.sh

Override the scenario path for either smoke script when needed:

WKBENCH_SCENARIO=/path/to/scenario.yaml ./scripts/smoke-wukongim-single-node.sh
WKBENCH_SCENARIO=/path/to/scenario.yaml ./scripts/smoke-wukongim-send-rate-mixed.sh

Start a local WuKongIM v2 three-node target from this repository and run the mixed send-rate scenario against all three nodes:

./scripts/start-wukongimv2-three-nodes.sh --clean
GOWORK=off go run ./cmd/wkbench run -scenario ./examples/wukongim-three-node-send-rate-mixed.yaml

Sweep a three-node target to find the highest passing send-link QPS:

./scripts/bench-wukongim-three-node-send-rate-sweep.sh \
  --mode mixed \
  --rates 100,200,500 \
  --duration 2m \
  --no-start-target

In mixed mode, the script runs person and group sub-scenarios concurrently and reports both per-workload rows plus an aggregate total row.

For an end-to-end local run that starts and stops the three-node target:

./scripts/bench-wukongim-three-node-send-rate-sweep.sh \
  --mode mixed \
  --rates 10,20 \
  --duration 5s \
  --start-target \
  --clean-target

Architecture Notes

• benchkit/contract defines the stable Unit API.
• benchkit/ports/* defines shared capability contracts.
• benchkit/kernel validates graph wiring, auto-connects unique matching ports, plans, and runs units.
• cmd/wkbench assembles the distribution by loading bundled official plugins and registering host-local runtime units.

See docs/design/wkbench-v2-unit-architecture.md and docs/unit-standard.md.