[benchmarking] Capture cluster hardware density frontiers and Prometheus server telemetry in Locust runner #1590

Background & Problem

According to the Actor Density Benchmark Specs, key evaluation targets are active actors per node, per vCPU, per GB RAM, and the actor-to-pod (A/P) bin-packing ratio (defined as active concurrent actors / worker pod count).

Currently:

1. benchmarking/locust/runner.py only records client-side request latencies and does not discover cluster hardware capacity (nodes, vCPUs, RAM allocatable).
2. During high-density benchmark runs, client requests may appear successful while server-side actors are queued or experiencing memory pressure. We need ground-truth telemetry from Prometheus (concurrent active/suspended actors, PSI pressure stalls, snapshot sizes, and ateapi throughput) captured alongside client metrics.

Proposed Changes

1. Cluster Hardware Discovery: Add RBAC in locust.yaml allowing the runner to query nodes and compute cluster density frontiers (actors/node, actors/vCPU, actors/GB RAM, and the steady-state A/P ratio: active_actors / worker_pods) written into stats.jsonl.
2. Prometheus Telemetry Harvester (server_telemetry.py): Query in-cluster Prometheus at the end of benchmark trials for ground-truth actor packing, PSI kernel pressure, and snapshot performance, outputting to server_summary.json and stats.jsonl.
3. Preserve status.json contract ({"locust_exit_code": 0, "stats_generated": true}) to maintain compatibility with test harnesses and CI orchestrators.

References

• Agent Substrate: Actor Density Benchmark Specs
• Substrate Benchmark Telemetry & Metrics Enhancement

cc Max Smythe (@maxsmythe) Haowei Cai (Roy) (@roycaihw) Aditya Shantanu (@aditya-shantanu)