Skip to content

Conversation

@enisdenjo
Copy link
Member

Related graphql-hive/gateway#1629

Also

  • Updated Hive Router and Grafbase versions to latest
  • No JIT on Hive Gateway with Bun

@github-actions
Copy link

github-actions bot commented Oct 26, 2025

💻 Website Preview

The latest changes are available as preview in: https://107e3e0e.federation-gateway-benchmark.pages.dev

@github-actions
Copy link

github-actions bot commented Oct 26, 2025

Overview for: constant-vus-over-time

This scenario runs 4 subgraphs and a GraphQL gateway with Federation spec, and runs a heavy query. It's being executed with a constant amount of VUs over a fixed amount of time. It measure things like memory usage, CPU usage, average RPS. It also includes a summary of the entire execution, and metrics information about HTTP execution times.

This scenario was running 50 VUs over 60s

Comparison

Comparison

Gateway RPS ⬇️ Requests Duration Notes
hive-router 1687 101661 total, 0 failed avg: 29ms, p95: 51ms
grafbase 1551 93594 total, 0 failed avg: 32ms, p95: 56ms
cosmo 697 42145 total, 0 failed avg: 71ms, p95: 104ms
hive-gateway-bun 571 34712 total, 0 failed avg: 86ms, p95: 113ms
hive-gateway 556 33837 total, 0 failed avg: 88ms, p95: 118ms
apollo-router 378 23033 total, 0 failed avg: 130ms, p95: 169ms
apollo-gateway 128 7834 total, 0 failed avg: 384ms, p95: 452ms
Summary for: `hive-router`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 304683      ✗ 0     
     data_received..................: 8.9 GB  148 MB/s
     data_sent......................: 118 MB  2.0 MB/s
     http_req_blocked...............: avg=5.39µs   min=1.22µs  med=2.58µs  max=9.48ms   p(90)=3.69µs   p(95)=4.38µs   p(99.9)=86.44µs
     http_req_connecting............: avg=2.34µs   min=0s      med=0s      max=9.45ms   p(90)=0s       p(95)=0s       p(99.9)=0s     
     http_req_duration..............: avg=29.26ms  min=1.95ms  med=26.93ms max=454.1ms  p(90)=44.36ms  p(95)=50.63ms  p(99.9)=85.94ms
       { expected_response:true }...: avg=29.26ms  min=1.95ms  med=26.93ms max=454.1ms  p(90)=44.36ms  p(95)=50.63ms  p(99.9)=85.94ms
     http_req_failed................: 0.00%   ✓ 0           ✗ 101661
     http_req_receiving.............: avg=117.31µs min=24.27µs med=43.61µs max=102.4ms  p(90)=103.08µs p(95)=332µs    p(99.9)=12.96ms
     http_req_sending...............: avg=103.58µs min=4.99µs  med=10.27µs max=345.29ms p(90)=30.24µs  p(95)=123.74µs p(99.9)=14.64ms
     http_req_tls_handshaking.......: avg=0s       min=0s      med=0s      max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s     
     http_req_waiting...............: avg=29.03ms  min=1.91ms  med=26.76ms max=452.69ms p(90)=43.99ms  p(95)=50.24ms  p(99.9)=84.49ms
     http_reqs......................: 101661  1687.135577/s
     iteration_duration.............: avg=29.53ms  min=5.09ms  med=27.18ms max=482.94ms p(90)=44.6ms   p(95)=50.9ms   p(99.9)=86.69ms
     iterations.....................: 101561  1685.476007/s
     success_rate...................: 100.00% ✓ 101561      ✗ 0     
     vus............................: 50      min=50        max=50  
     vus_max........................: 50      min=50        max=50  

Performance Overview

Performance Overview

HTTP Overview

HTTP Overview
Summary for: `grafbase`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 280482      ✗ 0    
     data_received..................: 8.2 GB  136 MB/s
     data_sent......................: 109 MB  1.8 MB/s
     http_req_blocked...............: avg=6.44µs   min=1.28µs  med=3.14µs  max=9.7ms    p(90)=4.49µs   p(95)=5.55µs   p(99.9)=124.18µs
     http_req_connecting............: avg=2.58µs   min=0s      med=0s      max=9.67ms   p(90)=0s       p(95)=0s       p(99.9)=0s      
     http_req_duration..............: avg=31.76ms  min=2.69ms  med=28.44ms max=482.19ms p(90)=48.8ms   p(95)=55.83ms  p(99.9)=96.33ms 
       { expected_response:true }...: avg=31.76ms  min=2.69ms  med=28.44ms max=482.19ms p(90)=48.8ms   p(95)=55.83ms  p(99.9)=96.33ms 
     http_req_failed................: 0.00%   ✓ 0           ✗ 93594
     http_req_receiving.............: avg=157.52µs min=27.74µs med=49.62µs max=111.98ms p(90)=145.73µs p(95)=406.47µs p(99.9)=17.73ms 
     http_req_sending...............: avg=123.37µs min=5.58µs  med=11.98µs max=428ms    p(90)=38.55µs  p(95)=144.29µs p(99.9)=17.16ms 
     http_req_tls_handshaking.......: avg=0s       min=0s      med=0s      max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s      
     http_req_waiting...............: avg=31.48ms  min=2.64ms  med=28.25ms max=464.71ms p(90)=48.29ms  p(95)=55.23ms  p(99.9)=89.96ms 
     http_reqs......................: 93594   1551.202164/s
     iteration_duration.............: avg=32.07ms  min=7.52ms  med=28.73ms max=514.47ms p(90)=49.09ms  p(95)=56.11ms  p(99.9)=97.73ms 
     iterations.....................: 93494   1549.544791/s
     success_rate...................: 100.00% ✓ 93494       ✗ 0    
     vus............................: 50      min=50        max=50 
     vus_max........................: 50      min=50        max=50 

Performance Overview

Performance Overview

HTTP Overview

HTTP Overview
Summary for: `cosmo`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 126135     ✗ 0    
     data_received..................: 3.7 GB  61 MB/s
     data_sent......................: 49 MB   811 kB/s
     http_req_blocked...............: avg=9.92µs   min=1.26µs  med=2.75µs  max=9.85ms   p(90)=3.83µs   p(95)=4.73µs   p(99.9)=2.31ms  
     http_req_connecting............: avg=6.19µs   min=0s      med=0s      max=9.82ms   p(90)=0s       p(95)=0s       p(99.9)=1.71ms  
     http_req_duration..............: avg=70.91ms  min=2.78ms  med=69.57ms max=487.18ms p(90)=96.46ms  p(95)=104.09ms p(99.9)=277.43ms
       { expected_response:true }...: avg=70.91ms  min=2.78ms  med=69.57ms max=487.18ms p(90)=96.46ms  p(95)=104.09ms p(99.9)=277.43ms
     http_req_failed................: 0.00%   ✓ 0          ✗ 42145
     http_req_receiving.............: avg=144.04µs min=28.08µs med=69.09µs max=62.47ms  p(90)=137.38µs p(95)=363.09µs p(99.9)=11.54ms 
     http_req_sending...............: avg=59.88µs  min=5.64µs  med=11.05µs max=334.96ms p(90)=28.36µs  p(95)=120.14µs p(99.9)=3.68ms  
     http_req_tls_handshaking.......: avg=0s       min=0s      med=0s      max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s      
     http_req_waiting...............: avg=70.71ms  min=2.7ms   med=69.38ms max=467.01ms p(90)=96.25ms  p(95)=103.85ms p(99.9)=265.93ms
     http_reqs......................: 42145   697.136385/s
     iteration_duration.............: avg=71.38ms  min=6.88ms  med=69.88ms max=527.03ms p(90)=96.8ms   p(95)=104.41ms p(99.9)=292.32ms
     iterations.....................: 42045   695.482247/s
     success_rate...................: 100.00% ✓ 42045      ✗ 0    
     vus............................: 50      min=50       max=50 
     vus_max........................: 50      min=50       max=50 

Performance Overview

Performance Overview

HTTP Overview

HTTP Overview
Summary for: `hive-gateway-bun`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 103836     ✗ 0    
     data_received..................: 3.0 GB  50 MB/s
     data_sent......................: 40 MB   665 kB/s
     http_req_blocked...............: avg=12.42µs  min=1.25µs  med=2.86µs  max=11.52ms  p(90)=4.52µs   p(95)=5.71µs   p(99.9)=3.27ms  
     http_req_connecting............: avg=8.7µs    min=0s      med=0s      max=11.48ms  p(90)=0s       p(95)=0s       p(99.9)=3.23ms  
     http_req_duration..............: avg=86.16ms  min=5.33ms  med=84.43ms max=590.86ms p(90)=104.26ms p(95)=112.57ms p(99.9)=362.97ms
       { expected_response:true }...: avg=86.16ms  min=5.33ms  med=84.43ms max=590.86ms p(90)=104.26ms p(95)=112.57ms p(99.9)=362.97ms
     http_req_failed................: 0.00%   ✓ 0          ✗ 34712
     http_req_receiving.............: avg=103.62µs min=26.75µs med=50.6µs  max=33.52ms  p(90)=112.26µs p(95)=311.26µs p(99.9)=5.44ms  
     http_req_sending...............: avg=79.2µs   min=5.7µs   med=11.32µs max=450.81ms p(90)=32.3µs   p(95)=134.58µs p(99.9)=5.89ms  
     http_req_tls_handshaking.......: avg=0s       min=0s      med=0s      max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s      
     http_req_waiting...............: avg=85.97ms  min=5.24ms  med=84.29ms max=574.68ms p(90)=104.11ms p(95)=112.32ms p(99.9)=355.01ms
     http_reqs......................: 34712   571.746228/s
     iteration_duration.............: avg=86.73ms  min=13.65ms med=84.77ms max=620.16ms p(90)=104.6ms  p(95)=112.95ms p(99.9)=385.1ms 
     iterations.....................: 34612   570.099114/s
     success_rate...................: 100.00% ✓ 34612      ✗ 0    
     vus............................: 50      min=50       max=50 
     vus_max........................: 50      min=50       max=50 

Performance Overview

Performance Overview

HTTP Overview

HTTP Overview
Summary for: `hive-gateway`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 101211     ✗ 0    
     data_received..................: 3.0 GB  49 MB/s
     data_sent......................: 39 MB   648 kB/s
     http_req_blocked...............: avg=13.15µs  min=1.26µs  med=2.99µs  max=11.57ms  p(90)=4.8µs    p(95)=6.1µs    p(99.9)=3.54ms  
     http_req_connecting............: avg=9.21µs   min=0s      med=0s      max=11.54ms  p(90)=0s       p(95)=0s       p(99.9)=3.51ms  
     http_req_duration..............: avg=88.41ms  min=5.31ms  med=86.18ms max=648.07ms p(90)=108.76ms p(95)=118.39ms p(99.9)=333.84ms
       { expected_response:true }...: avg=88.41ms  min=5.31ms  med=86.18ms max=648.07ms p(90)=108.76ms p(95)=118.39ms p(99.9)=333.84ms
     http_req_failed................: 0.00%   ✓ 0          ✗ 33837
     http_req_receiving.............: avg=104.05µs min=28.36µs med=53.25µs max=44.77ms  p(90)=115.22µs p(95)=248.85µs p(99.9)=5.09ms  
     http_req_sending...............: avg=74.26µs  min=5.6µs   med=11.72µs max=230.62ms p(90)=32.13µs  p(95)=134.02µs p(99.9)=5.39ms  
     http_req_tls_handshaking.......: avg=0s       min=0s      med=0s      max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s      
     http_req_waiting...............: avg=88.23ms  min=5.26ms  med=86.05ms max=647.25ms p(90)=108.56ms p(95)=118.03ms p(99.9)=329.5ms 
     http_reqs......................: 33837   556.999496/s
     iteration_duration.............: avg=89ms     min=17.63ms med=86.51ms max=683.18ms p(90)=109.1ms  p(95)=118.76ms p(99.9)=355.08ms
     iterations.....................: 33737   555.353371/s
     success_rate...................: 100.00% ✓ 33737      ✗ 0    
     vus............................: 50      min=50       max=50 
     vus_max........................: 50      min=50       max=50 

Performance Overview

Performance Overview

HTTP Overview

HTTP Overview
Summary for: `apollo-router`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 68799      ✗ 0    
     data_received..................: 2.0 GB  33 MB/s
     data_sent......................: 27 MB   441 kB/s
     http_req_blocked...............: avg=13.98µs  min=1.96µs  med=3.01µs   max=9.6ms    p(90)=4.39µs   p(95)=5.14µs   p(99.9)=5.11ms  
     http_req_connecting............: avg=10.36µs  min=0s      med=0s       max=9.56ms   p(90)=0s       p(95)=0s       p(99.9)=5.1ms   
     http_req_duration..............: avg=130.05ms min=6.13ms  med=129.18ms max=595.36ms p(90)=160.17ms p(95)=169.43ms p(99.9)=422.19ms
       { expected_response:true }...: avg=130.05ms min=6.13ms  med=129.18ms max=595.36ms p(90)=160.17ms p(95)=169.43ms p(99.9)=422.19ms
     http_req_failed................: 0.00%   ✓ 0          ✗ 23033
     http_req_receiving.............: avg=93.66µs  min=34.24µs med=53.22µs  max=358.85ms p(90)=99.67µs  p(95)=126.5µs  p(99.9)=1.67ms  
     http_req_sending...............: avg=68.22µs  min=7.84µs  med=11.62µs  max=382.84ms p(90)=19.99µs  p(95)=36.72µs  p(99.9)=2.21ms  
     http_req_tls_handshaking.......: avg=0s       min=0s      med=0s       max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s      
     http_req_waiting...............: avg=129.89ms min=6.08ms  med=129.07ms max=577.39ms p(90)=160.03ms p(95)=169.27ms p(99.9)=412.2ms 
     http_reqs......................: 23033   378.795453/s
     iteration_duration.............: avg=130.91ms min=23.07ms med=129.59ms max=648.51ms p(90)=160.52ms p(95)=169.82ms p(99.9)=428.07ms
     iterations.....................: 22933   377.150875/s
     success_rate...................: 100.00% ✓ 22933      ✗ 0    
     vus............................: 50      min=50       max=50 
     vus_max........................: 50      min=50       max=50 

Performance Overview

Performance Overview

HTTP Overview

HTTP Overview
Summary for: `apollo-gateway`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 23202      ✗ 0   
     data_received..................: 688 MB  11 MB/s
     data_sent......................: 9.1 MB  149 kB/s
     http_req_blocked...............: avg=31.88µs  min=1.22µs   med=3.19µs   max=9.24ms   p(90)=4.67µs   p(95)=5.35µs   p(99.9)=7.61ms  
     http_req_connecting............: avg=28.4µs   min=0s       med=0s       max=9.21ms   p(90)=0s       p(95)=0s       p(99.9)=7.58ms  
     http_req_duration..............: avg=383.89ms min=7.04ms   med=383.4ms  max=717.3ms  p(90)=435.26ms p(95)=452.04ms p(99.9)=646.73ms
       { expected_response:true }...: avg=383.89ms min=7.04ms   med=383.4ms  max=717.3ms  p(90)=435.26ms p(95)=452.04ms p(99.9)=646.73ms
     http_req_failed................: 0.00%   ✓ 0          ✗ 7834
     http_req_receiving.............: avg=82.33µs  min=24.41µs  med=53.08µs  max=100.01ms p(90)=86.44µs  p(95)=99.16µs  p(99.9)=553.78µs
     http_req_sending...............: avg=101.28µs min=5.33µs   med=12.17µs  max=195.03ms p(90)=18.14µs  p(95)=21.39µs  p(99.9)=4.59ms  
     http_req_tls_handshaking.......: avg=0s       min=0s       med=0s       max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s      
     http_req_waiting...............: avg=383.7ms  min=6.99ms   med=383.28ms max=703.08ms p(90)=435.07ms p(95)=451.9ms  p(99.9)=645.59ms
     http_reqs......................: 7834    128.241308/s
     iteration_duration.............: avg=389.1ms  min=161.44ms med=383.91ms max=731.22ms p(90)=436.24ms p(95)=452.58ms p(99.9)=660.04ms
     iterations.....................: 7734    126.604325/s
     success_rate...................: 100.00% ✓ 7734       ✗ 0   
     vus............................: 32      min=32       max=50
     vus_max........................: 50      min=50       max=50

Performance Overview

Performance Overview

HTTP Overview

HTTP Overview

@github-actions
Copy link

github-actions bot commented Oct 26, 2025

Overview for: ramping-vus

This scenario runs 4 subgraphs and a GraphQL gateway with Federation spec, and runs a heavy query. We are running a heavy load of concurrent VUs to measure response time and other stats, during stress. It measure things like memory usage, CPU usage, response times. It also includes a summary of the entire execution, and metrics information about HTTP execution times.

This scenario was running 500 VUs over 60s

Comparison

Comparison

Gateway RPS ⬇️ Requests Duration Notes
hive-router 1737 108476 total, 0 failed avg: 128ms, p95: 305ms
grafbase 1516 96461 total, 0 failed avg: 144ms, p95: 331ms
cosmo 679 43892 total, 0 failed avg: 320ms, p95: 751ms
hive-gateway 565 37350 total, 0 failed avg: 376ms, p95: 803ms ❌ non-compatible response structure (3)
hive-gateway-bun 557 37004 total, 0 failed avg: 380ms, p95: 798ms ❌ non-compatible response structure (1)
apollo-router 393 26291 total, 0 failed avg: 536ms, p95: 1204ms ❌ 20 unexpected GraphQL errors
apollo-gateway 148 10345 total, 0 failed avg: 1252ms, p95: 2632ms
Summary for: `hive-router`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 322428      ✗ 0     
     data_received..................: 9.5 GB  152 MB/s
     data_sent......................: 126 MB  2.0 MB/s
     http_req_blocked...............: avg=529.98µs min=1.18µs  med=2.64µs   max=364.21ms p(90)=4.07µs   p(95)=4.87µs   p(99.9)=182.56ms
     http_req_connecting............: avg=526.33µs min=0s      med=0s       max=364.15ms p(90)=0s       p(95)=0s       p(99.9)=182.48ms
     http_req_duration..............: avg=128.32ms min=1.74ms  med=117.51ms max=443.76ms p(90)=255.12ms p(95)=304.99ms p(99.9)=396.35ms
       { expected_response:true }...: avg=128.32ms min=1.74ms  med=117.51ms max=443.76ms p(90)=255.12ms p(95)=304.99ms p(99.9)=396.35ms
     http_req_failed................: 0.00%   ✓ 0           ✗ 108476
     http_req_receiving.............: avg=537.92µs min=22.89µs med=42.76µs  max=129.94ms p(90)=109.44µs p(95)=400.27µs p(99.9)=74.01ms 
     http_req_sending...............: avg=454.18µs min=5.28µs  med=10.51µs  max=133.64ms p(90)=25.78µs  p(95)=129.46µs p(99.9)=70.55ms 
     http_req_tls_handshaking.......: avg=0s       min=0s      med=0s       max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s      
     http_req_waiting...............: avg=127.33ms min=1.69ms  med=116.58ms max=416.44ms p(90)=252.79ms p(95)=302.74ms p(99.9)=394.07ms
     http_reqs......................: 108476  1737.578541/s
     iteration_duration.............: avg=130.37ms min=1.91ms  med=119.16ms max=802.22ms p(90)=257.71ms p(95)=308.1ms  p(99.9)=449.9ms 
     iterations.....................: 107476  1721.560449/s
     success_rate...................: 100.00% ✓ 107476      ✗ 0     
     vus............................: 79      min=0         max=493 
     vus_max........................: 500     min=500       max=500 

Performance Overview

Performance Overview

HTTP Overview

HTTP Overview
Summary for: `grafbase`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 286383      ✗ 0    
     data_received..................: 8.5 GB  133 MB/s
     data_sent......................: 112 MB  1.8 MB/s
     http_req_blocked...............: avg=678.6µs  min=1.16µs  med=2.92µs   max=379.66ms p(90)=4.23µs   p(95)=5.2µs    p(99.9)=215.22ms
     http_req_connecting............: avg=674.37µs min=0s      med=0s       max=379.61ms p(90)=0s       p(95)=0s       p(99.9)=215.15ms
     http_req_duration..............: avg=144.27ms min=2.79ms  med=134.56ms max=481.7ms  p(90)=284.31ms p(95)=330.81ms p(99.9)=423.87ms
       { expected_response:true }...: avg=144.27ms min=2.79ms  med=134.56ms max=481.7ms  p(90)=284.31ms p(95)=330.81ms p(99.9)=423.87ms
     http_req_failed................: 0.00%   ✓ 0           ✗ 96461
     http_req_receiving.............: avg=647.17µs min=24.65µs med=46.92µs  max=146.94ms p(90)=107.46µs p(95)=424.49µs p(99.9)=86.08ms 
     http_req_sending...............: avg=605.82µs min=5.34µs  med=11.67µs  max=143.13ms p(90)=30.72µs  p(95)=135.89µs p(99.9)=87.75ms 
     http_req_tls_handshaking.......: avg=0s       min=0s      med=0s       max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s      
     http_req_waiting...............: avg=143.02ms min=2.73ms  med=133.19ms max=437.35ms p(90)=281.17ms p(95)=328.15ms p(99.9)=418.1ms 
     http_reqs......................: 96461   1516.236176/s
     iteration_duration.............: avg=146.83ms min=3.23ms  med=136.65ms max=872.82ms p(90)=287.46ms p(95)=333.86ms p(99.9)=504.49ms
     iterations.....................: 95461   1500.517531/s
     success_rate...................: 100.00% ✓ 95461       ✗ 0    
     vus............................: 88      min=0         max=499
     vus_max........................: 500     min=500       max=500

Performance Overview

Performance Overview

HTTP Overview

HTTP Overview
Summary for: `cosmo`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 128676     ✗ 0    
     data_received..................: 3.9 GB  60 MB/s
     data_sent......................: 51 MB   792 kB/s
     http_req_blocked...............: avg=142.66µs min=1.21µs  med=2.72µs   max=204.61ms p(90)=4.12µs   p(95)=6.12µs   p(99.9)=50.47ms 
     http_req_connecting............: avg=137.54µs min=0s      med=0s       max=173.99ms p(90)=0s       p(95)=0s       p(99.9)=50.43ms 
     http_req_duration..............: avg=319.88ms min=3.01ms  med=290.74ms max=1.4s     p(90)=655.77ms p(95)=750.98ms p(99.9)=1.13s   
       { expected_response:true }...: avg=319.88ms min=3.01ms  med=290.74ms max=1.4s     p(90)=655.77ms p(95)=750.98ms p(99.9)=1.13s   
     http_req_failed................: 0.00%   ✓ 0          ✗ 43892
     http_req_receiving.............: avg=923.13µs min=27.82µs med=58.31µs  max=224.03ms p(90)=197.21µs p(95)=504.09µs p(99.9)=139.02ms
     http_req_sending...............: avg=204.92µs min=5.6µs   med=10.93µs  max=117.15ms p(90)=29.98µs  p(95)=131.02µs p(99.9)=43.75ms 
     http_req_tls_handshaking.......: avg=0s       min=0s      med=0s       max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s      
     http_req_waiting...............: avg=318.75ms min=2.92ms  med=290.09ms max=1.4s     p(90)=652.25ms p(95)=748.86ms p(99.9)=1.13s   
     http_reqs......................: 43892   679.774317/s
     iteration_duration.............: avg=327.94ms min=3.34ms  med=299.12ms max=1.4s     p(90)=660.5ms  p(95)=755.72ms p(99.9)=1.13s   
     iterations.....................: 42892   664.286887/s
     success_rate...................: 100.00% ✓ 42892      ✗ 0    
     vus............................: 90      min=0        max=499
     vus_max........................: 500     min=500      max=500

Performance Overview

Performance Overview

HTTP Overview

HTTP Overview
Summary for: `hive-gateway`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✗ valid response structure
      ↳  99% — ✓ 36347 / ✗ 3

     checks.........................: 99.99% ✓ 109047     ✗ 3    
     data_received..................: 3.3 GB 50 MB/s
     data_sent......................: 44 MB  659 kB/s
     http_req_blocked...............: avg=161.07µs min=1.27µs  med=2.68µs   max=252.25ms p(90)=4.23µs   p(95)=5.96µs   p(99.9)=46.72ms
     http_req_connecting............: avg=152.31µs min=0s      med=0s       max=252.08ms p(90)=0s       p(95)=0s       p(99.9)=46.67ms
     http_req_duration..............: avg=376.21ms min=4.89ms  med=358.56ms max=1.53s    p(90)=710.86ms p(95)=803.43ms p(99.9)=1.08s  
       { expected_response:true }...: avg=376.21ms min=4.89ms  med=358.56ms max=1.53s    p(90)=710.86ms p(95)=803.43ms p(99.9)=1.08s  
     http_req_failed................: 0.00%  ✓ 0          ✗ 37350
     http_req_receiving.............: avg=230.47µs min=26.35µs med=43.43µs  max=181.03ms p(90)=93.23µs  p(95)=364.89µs p(99.9)=39.15ms
     http_req_sending...............: avg=283.22µs min=5.59µs  med=10.65µs  max=184.42ms p(90)=27.83µs  p(95)=129.33µs p(99.9)=48.94ms
     http_req_tls_handshaking.......: avg=0s       min=0s      med=0s       max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s     
     http_req_waiting...............: avg=375.7ms  min=4.84ms  med=358.16ms max=1.53s    p(90)=709.59ms p(95)=802.64ms p(99.9)=1.08s  
     http_reqs......................: 37350  565.454906/s
     iteration_duration.............: avg=387.15ms min=5.07ms  med=369.01ms max=1.77s    p(90)=716.86ms p(95)=806.78ms p(99.9)=1.09s  
     iterations.....................: 36350  550.315551/s
     success_rate...................: 99.99% ✓ 36347      ✗ 3    
     vus............................: 64     min=0        max=497
     vus_max........................: 500    min=500      max=500

Performance Overview

Performance Overview

HTTP Overview

HTTP Overview
Summary for: `hive-gateway-bun`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✗ valid response structure
      ↳  99% — ✓ 36003 / ✗ 1

     checks.........................: 99.99% ✓ 108011     ✗ 1    
     data_received..................: 3.3 GB 49 MB/s
     data_sent......................: 43 MB  650 kB/s
     http_req_blocked...............: avg=194.9µs  min=1.19µs  med=2.77µs   max=247.2ms  p(90)=4.62µs   p(95)=6.92µs   p(99.9)=61.49ms
     http_req_connecting............: avg=190.31µs min=0s      med=0s       max=247.13ms p(90)=0s       p(95)=0s       p(99.9)=61.34ms
     http_req_duration..............: avg=379.77ms min=5ms     med=368.49ms max=1.37s    p(90)=715.42ms p(95)=797.53ms p(99.9)=1.12s  
       { expected_response:true }...: avg=379.77ms min=5ms     med=368.49ms max=1.37s    p(90)=715.42ms p(95)=797.53ms p(99.9)=1.12s  
     http_req_failed................: 0.00%  ✓ 0          ✗ 37004
     http_req_receiving.............: avg=232.64µs min=27.08µs med=46.14µs  max=125.65ms p(90)=101.02µs p(95)=411.05µs p(99.9)=36.7ms 
     http_req_sending...............: avg=323.98µs min=5.99µs  med=11.12µs  max=176.32ms p(90)=33.15µs  p(95)=138.64µs p(99.9)=60.39ms
     http_req_tls_handshaking.......: avg=0s       min=0s      med=0s       max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s     
     http_req_waiting...............: avg=379.22ms min=4.95ms  med=368.08ms max=1.37s    p(90)=714.45ms p(95)=796.27ms p(99.9)=1.12s  
     http_reqs......................: 37004  557.965812/s
     iteration_duration.............: avg=390.99ms min=5.62ms  med=377.87ms max=1.4s     p(90)=719.72ms p(95)=801.63ms p(99.9)=1.19s  
     iterations.....................: 36004  542.887285/s
     success_rate...................: 99.99% ✓ 36003      ✗ 1    
     vus............................: 76     min=0        max=498
     vus_max........................: 500    min=500      max=500

Performance Overview

Performance Overview

HTTP Overview

HTTP Overview
Summary for: `apollo-router`

K6 Output

     ✓ response code was 200
     ✗ no graphql errors
      ↳  99% — ✓ 25271 / ✗ 20
     ✓ valid response structure

     checks.........................: 99.97% ✓ 75853      ✗ 20   
     data_received..................: 2.3 GB 35 MB/s
     data_sent......................: 31 MB  459 kB/s
     http_req_blocked...............: avg=62.99µs  min=1.4µs   med=3.02µs   max=85.53ms p(90)=4.42µs  p(95)=5.64µs  p(99.9)=15.11ms
     http_req_connecting............: avg=56.39µs  min=0s      med=0s       max=85.35ms p(90)=0s      p(95)=0s      p(99.9)=14.51ms
     http_req_duration..............: avg=536.19ms min=5.93ms  med=491.08ms max=2s      p(90)=1.08s   p(95)=1.2s    p(99.9)=1.69s  
       { expected_response:true }...: avg=536.19ms min=5.93ms  med=491.08ms max=2s      p(90)=1.08s   p(95)=1.2s    p(99.9)=1.69s  
     http_req_failed................: 0.00%  ✓ 0          ✗ 26291
     http_req_receiving.............: avg=98.88µs  min=25.99µs med=51.21µs  max=80.47ms p(90)=84.19µs p(95)=138.8µs p(99.9)=5.55ms 
     http_req_sending...............: avg=73.96µs  min=5.91µs  med=11.16µs  max=64.28ms p(90)=19.91µs p(95)=74.17µs p(99.9)=11.64ms
     http_req_tls_handshaking.......: avg=0s       min=0s      med=0s       max=0s      p(90)=0s      p(95)=0s      p(99.9)=0s     
     http_req_waiting...............: avg=536.02ms min=5.87ms  med=490.86ms max=2s      p(90)=1.08s   p(95)=1.2s    p(99.9)=1.69s  
     http_reqs......................: 26291  393.909113/s
     iteration_duration.............: avg=557.71ms min=6.12ms  med=516.62ms max=2s      p(90)=1.08s   p(95)=1.21s   p(99.9)=1.7s   
     iterations.....................: 25290  378.91147/s
     success_rate...................: 99.92% ✓ 25270      ✗ 20   
     vus............................: 101    min=0        max=500
     vus_max........................: 500    min=500      max=500

Performance Overview

Performance Overview

HTTP Overview

HTTP Overview
Summary for: `apollo-gateway`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 28035      ✗ 0    
     data_received..................: 909 MB  13 MB/s
     data_sent......................: 12 MB   178 kB/s
     http_req_blocked...............: avg=35.29µs min=1.27µs  med=3.64µs  max=39.73ms p(90)=6.36µs   p(95)=16.99µs  p(99.9)=5.99ms
     http_req_connecting............: avg=28.74µs min=0s      med=0s      max=39.67ms p(90)=0s       p(95)=0s       p(99.9)=5ms   
     http_req_duration..............: avg=1.25s   min=7.61ms  med=1.22s   max=16.64s  p(90)=2.23s    p(95)=2.63s    p(99.9)=15.03s
       { expected_response:true }...: avg=1.25s   min=7.61ms  med=1.22s   max=16.64s  p(90)=2.23s    p(95)=2.63s    p(99.9)=15.03s
     http_req_failed................: 0.00%   ✓ 0          ✗ 10345
     http_req_receiving.............: avg=84.16µs min=28.72µs med=63.34µs max=8.61ms  p(90)=115.95µs p(95)=145.65µs p(99.9)=1.46ms
     http_req_sending...............: avg=45.85µs min=6.22µs  med=14.02µs max=33.03ms p(90)=32.36µs  p(95)=50.05µs  p(99.9)=7.77ms
     http_req_tls_handshaking.......: avg=0s      min=0s      med=0s      max=0s      p(90)=0s       p(95)=0s       p(99.9)=0s    
     http_req_waiting...............: avg=1.25s   min=7.55ms  med=1.22s   max=16.64s  p(90)=2.23s    p(95)=2.63s    p(99.9)=15.03s
     http_reqs......................: 10345   148.995142/s
     iteration_duration.............: avg=1.38s   min=7.8ms   med=1.33s   max=16.65s  p(90)=2.27s    p(95)=2.71s    p(99.9)=15.13s
     iterations.....................: 9345    134.592518/s
     success_rate...................: 100.00% ✓ 9345       ✗ 0    
     vus............................: 93      min=0        max=500
     vus_max........................: 500     min=500      max=500

Performance Overview

Performance Overview

HTTP Overview

HTTP Overview

@github-actions
Copy link

github-actions bot commented Oct 26, 2025

Overview for: constant-vus-subgraphs-delay

This scenario runs 4 subgraphs and a GraphQL gateway with Federation spec, and runs a heavy query. It's being executed with a constant amount of VUs over a fixed amount of time. It measure things like memory usage, CPU usage, average RPS. It also includes a summary of the entire execution, and metrics information about HTTP execution times.

This scenario was running 50 VUs over 60s

Comparison

Comparison

Gateway RPS ⬇️ Requests Duration Notes
grafbase 171 13892 total, 0 failed avg: 217ms, p95: 228ms
hive-router 166 13492 total, 0 failed avg: 224ms, p95: 235ms
cosmo 158 12879 total, 0 failed avg: 235ms, p95: 257ms
hive-gateway-bun 140 11399 total, 0 failed avg: 265ms, p95: 275ms
hive-gateway 139 11349 total, 0 failed avg: 266ms, p95: 276ms
apollo-router 129 10501 total, 0 failed avg: 288ms, p95: 336ms
apollo-gateway 89 7346 total, 0 failed avg: 412ms, p95: 451ms
Summary for: `grafbase`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 41376      ✗ 0    
     data_received..................: 1.2 GB  15 MB/s
     data_sent......................: 16 MB   200 kB/s
     http_req_blocked...............: avg=19.25µs  min=1.06µs   med=2.01µs   max=9.29ms   p(90)=3.62µs   p(95)=5.13µs   p(99.9)=6.74ms  
     http_req_connecting............: avg=16.65µs  min=0s       med=0s       max=9.24ms   p(90)=0s       p(95)=0s       p(99.9)=6.66ms  
     http_req_duration..............: avg=217.35ms min=165.27ms med=216.52ms max=603.18ms p(90)=225.66ms p(95)=228.42ms p(99.9)=508.73ms
       { expected_response:true }...: avg=217.35ms min=165.27ms med=216.52ms max=603.18ms p(90)=225.66ms p(95)=228.42ms p(99.9)=508.73ms
     http_req_failed................: 0.00%   ✓ 0          ✗ 13892
     http_req_receiving.............: avg=83.96µs  min=23.38µs  med=44.4µs   max=141.05ms p(90)=90.79µs  p(95)=253.47µs p(99.9)=2.07ms  
     http_req_sending...............: avg=102.66µs min=4.43µs   med=8.02µs   max=352.68ms p(90)=23.2µs   p(95)=91.97µs  p(99.9)=6.07ms  
     http_req_tls_handshaking.......: avg=0s       min=0s       med=0s       max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s      
     http_req_waiting...............: avg=217.17ms min=165.18ms med=216.43ms max=590.08ms p(90)=225.55ms p(95)=228.22ms p(99.9)=479.3ms 
     http_reqs......................: 13892   171.621005/s
     iteration_duration.............: avg=217.64ms min=165.49ms med=216.75ms max=608.55ms p(90)=225.89ms p(95)=228.66ms p(99.9)=518.13ms
     iterations.....................: 13792   170.38561/s
     success_rate...................: 100.00% ✓ 13792      ✗ 0    
     vus............................: 50      min=0        max=50 
     vus_max........................: 50      min=50       max=50 

Performance Overview

Performance Overview

HTTP Overview

HTTP Overview
Summary for: `hive-router`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 40176      ✗ 0    
     data_received..................: 1.2 GB  15 MB/s
     data_sent......................: 16 MB   194 kB/s
     http_req_blocked...............: avg=15.22µs  min=1.03µs   med=2.23µs   max=7.02ms   p(90)=3.39µs   p(95)=4.78µs   p(99.9)=4.98ms  
     http_req_connecting............: avg=12.67µs  min=0s       med=0s       max=6.99ms   p(90)=0s       p(95)=0s       p(99.9)=4.96ms  
     http_req_duration..............: avg=224ms    min=160.1ms  med=223.12ms max=598.68ms p(90)=231.95ms p(95)=234.63ms p(99.9)=507.99ms
       { expected_response:true }...: avg=224ms    min=160.1ms  med=223.12ms max=598.68ms p(90)=231.95ms p(95)=234.63ms p(99.9)=507.99ms
     http_req_failed................: 0.00%   ✓ 0          ✗ 13492
     http_req_receiving.............: avg=77.5µs   min=19.97µs  med=40.7µs   max=17.64ms  p(90)=88.49µs  p(95)=245.1µs  p(99.9)=4.92ms  
     http_req_sending...............: avg=90.25µs  min=4.31µs   med=8.63µs   max=352.36ms p(90)=21.16µs  p(95)=92.58µs  p(99.9)=5.65ms  
     http_req_tls_handshaking.......: avg=0s       min=0s       med=0s       max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s      
     http_req_waiting...............: avg=223.83ms min=160.04ms med=223.02ms max=598.04ms p(90)=231.8ms  p(95)=234.45ms p(99.9)=498.82ms
     http_reqs......................: 13492   166.714021/s
     iteration_duration.............: avg=224.34ms min=160.29ms med=223.36ms max=617.17ms p(90)=232.15ms p(95)=234.82ms p(99.9)=517.12ms
     iterations.....................: 13392   165.47837/s
     success_rate...................: 100.00% ✓ 13392      ✗ 0    
     vus............................: 50      min=0        max=50 
     vus_max........................: 50      min=50       max=50 

Performance Overview

Performance Overview

HTTP Overview

HTTP Overview
Summary for: `cosmo`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 38337      ✗ 0    
     data_received..................: 1.1 GB  14 MB/s
     data_sent......................: 15 MB   185 kB/s
     http_req_blocked...............: avg=18.14µs  min=1.24µs   med=2.83µs   max=7.53ms   p(90)=4.47µs   p(95)=5.81µs   p(99.9)=5.65ms  
     http_req_connecting............: avg=14.62µs  min=0s       med=0s       max=7.5ms    p(90)=0s       p(95)=0s       p(99.9)=5.63ms  
     http_req_duration..............: avg=234.81ms min=133.82ms med=234.58ms max=623.68ms p(90)=251.33ms p(95)=256.97ms p(99.9)=517.04ms
       { expected_response:true }...: avg=234.81ms min=133.82ms med=234.58ms max=623.68ms p(90)=251.33ms p(95)=256.97ms p(99.9)=517.04ms
     http_req_failed................: 0.00%   ✓ 0          ✗ 12879
     http_req_receiving.............: avg=244.82µs min=28.04µs  med=54.13µs  max=240.58ms p(90)=133.61µs p(95)=409.99µs p(99.9)=24.45ms 
     http_req_sending...............: avg=116.09µs min=5.26µs   med=10.56µs  max=236.93ms p(90)=22.21µs  p(95)=118.89µs p(99.9)=3.43ms  
     http_req_tls_handshaking.......: avg=0s       min=0s       med=0s       max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s      
     http_req_waiting...............: avg=234.45ms min=133.7ms  med=234.35ms max=622.88ms p(90)=250.97ms p(95)=256.44ms p(99.9)=516.34ms
     http_reqs......................: 12879   158.718581/s
     iteration_duration.............: avg=235.32ms min=134.1ms  med=234.96ms max=640.83ms p(90)=251.64ms p(95)=257.29ms p(99.9)=533.27ms
     iterations.....................: 12779   157.486198/s
     success_rate...................: 100.00% ✓ 12779      ✗ 0    
     vus............................: 50      min=0        max=50 
     vus_max........................: 50      min=50       max=50 

Performance Overview

Performance Overview

HTTP Overview

HTTP Overview
Summary for: `hive-gateway-bun`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 33897      ✗ 0    
     data_received..................: 1.0 GB  12 MB/s
     data_sent......................: 13 MB   163 kB/s
     http_req_blocked...............: avg=28.63µs  min=1.15µs   med=2.52µs   max=11.6ms   p(90)=3.92µs   p(95)=4.91µs   p(99.9)=9.11ms  
     http_req_connecting............: avg=25.47µs  min=0s       med=0s       max=11.56ms  p(90)=0s       p(95)=0s       p(99.9)=9.06ms  
     http_req_duration..............: avg=265.14ms min=146.04ms med=264.14ms max=704.72ms p(90)=272.25ms p(95)=275.47ms p(99.9)=608.24ms
       { expected_response:true }...: avg=265.14ms min=146.04ms med=264.14ms max=704.72ms p(90)=272.25ms p(95)=275.47ms p(99.9)=608.24ms
     http_req_failed................: 0.00%   ✓ 0          ✗ 11399
     http_req_receiving.............: avg=77.63µs  min=23.76µs  med=44.93µs  max=6.58ms   p(90)=93.25µs  p(95)=304.29µs p(99.9)=1.32ms  
     http_req_sending...............: avg=99.52µs  min=5.33µs   med=10.6µs   max=201.89ms p(90)=36.11µs  p(95)=119.22µs p(99.9)=6.48ms  
     http_req_tls_handshaking.......: avg=0s       min=0s       med=0s       max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s      
     http_req_waiting...............: avg=264.96ms min=145.97ms med=264.03ms max=703.77ms p(90)=272.01ms p(95)=275.2ms  p(99.9)=607.14ms
     http_reqs......................: 11399   140.173707/s
     iteration_duration.............: avg=265.96ms min=146.19ms med=264.43ms max=722.72ms p(90)=272.52ms p(95)=275.77ms p(99.9)=628.69ms
     iterations.....................: 11299   138.944005/s
     success_rate...................: 100.00% ✓ 11299      ✗ 0    
     vus............................: 50      min=0        max=50 
     vus_max........................: 50      min=50       max=50 

Performance Overview

Performance Overview

HTTP Overview

HTTP Overview
Summary for: `hive-gateway`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 33747      ✗ 0    
     data_received..................: 997 MB  12 MB/s
     data_sent......................: 13 MB   162 kB/s
     http_req_blocked...............: avg=28.84µs  min=1.18µs   med=2.5µs    max=11.77ms  p(90)=3.92µs   p(95)=5.05µs   p(99.9)=9.23ms  
     http_req_connecting............: avg=25.65µs  min=0s       med=0s       max=11.73ms  p(90)=0s       p(95)=0s       p(99.9)=9.19ms  
     http_req_duration..............: avg=266.19ms min=179.34ms med=265.06ms max=653.85ms p(90)=272.81ms p(95)=275.59ms p(99.9)=568.3ms 
       { expected_response:true }...: avg=266.19ms min=179.34ms med=265.06ms max=653.85ms p(90)=272.81ms p(95)=275.59ms p(99.9)=568.3ms 
     http_req_failed................: 0.00%   ✓ 0          ✗ 11349
     http_req_receiving.............: avg=80.71µs  min=24.35µs  med=45.71µs  max=9.99ms   p(90)=94.32µs  p(95)=306.62µs p(99.9)=1.86ms  
     http_req_sending...............: avg=127.02µs min=5.51µs   med=10.57µs  max=167.25ms p(90)=36.03µs  p(95)=127.16µs p(99.9)=6.49ms  
     http_req_tls_handshaking.......: avg=0s       min=0s       med=0s       max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s      
     http_req_waiting...............: avg=265.98ms min=176.93ms med=264.95ms max=653.06ms p(90)=272.62ms p(95)=275.38ms p(99.9)=567.17ms
     http_reqs......................: 11349   139.598074/s
     iteration_duration.............: avg=267.01ms min=179.49ms med=265.35ms max=685.82ms p(90)=273.08ms p(95)=275.88ms p(99.9)=605.05ms
     iterations.....................: 11249   138.368027/s
     success_rate...................: 100.00% ✓ 11249      ✗ 0    
     vus............................: 50      min=0        max=50 
     vus_max........................: 50      min=50       max=50 

Performance Overview

Performance Overview

HTTP Overview

HTTP Overview
Summary for: `apollo-router`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 31203      ✗ 0    
     data_received..................: 921 MB  11 MB/s
     data_sent......................: 12 MB   150 kB/s
     http_req_blocked...............: avg=22.3µs   min=1.68µs   med=2.58µs   max=7.75ms   p(90)=3.78µs   p(95)=4.52µs   p(99.9)=6.19ms  
     http_req_connecting............: avg=19.07µs  min=0s       med=0s       max=7.7ms    p(90)=0s       p(95)=0s       p(99.9)=6.17ms  
     http_req_duration..............: avg=287.96ms min=185.84ms med=287.5ms  max=705.66ms p(90)=325.98ms p(95)=335.93ms p(99.9)=624.79ms
       { expected_response:true }...: avg=287.96ms min=185.84ms med=287.5ms  max=705.66ms p(90)=325.98ms p(95)=335.93ms p(99.9)=624.79ms
     http_req_failed................: 0.00%   ✓ 0          ✗ 10501
     http_req_receiving.............: avg=93.61µs  min=29.26µs  med=48.71µs  max=162.51ms p(90)=94.75µs  p(95)=163.61µs p(99.9)=1.41ms  
     http_req_sending...............: avg=84.91µs  min=6.89µs   med=10.55µs  max=326.61ms p(90)=19.58µs  p(95)=123.26µs p(99.9)=3.62ms  
     http_req_tls_handshaking.......: avg=0s       min=0s       med=0s       max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s      
     http_req_waiting...............: avg=287.78ms min=185.78ms med=287.38ms max=691.05ms p(90)=325.83ms p(95)=335.68ms p(99.9)=612.6ms 
     http_reqs......................: 10501   129.069298/s
     iteration_duration.............: avg=289.09ms min=186.09ms med=288.14ms max=739.21ms p(90)=326.42ms p(95)=336.45ms p(99.9)=650.19ms
     iterations.....................: 10401   127.840183/s
     success_rate...................: 100.00% ✓ 10401      ✗ 0    
     vus............................: 50      min=0        max=50 
     vus_max........................: 50      min=50       max=50 

Performance Overview

Performance Overview

HTTP Overview

HTTP Overview
Summary for: `apollo-gateway`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 21738     ✗ 0   
     data_received..................: 645 MB  7.9 MB/s
     data_sent......................: 8.5 MB  105 kB/s
     http_req_blocked...............: avg=33.15µs  min=1.2µs    med=3.42µs   max=8.68ms   p(90)=5.23µs   p(95)=5.86µs   p(99.9)=7.35ms  
     http_req_connecting............: avg=29.35µs  min=0s       med=0s       max=8.63ms   p(90)=0s       p(95)=0s       p(99.9)=7.32ms  
     http_req_duration..............: avg=412.12ms min=212.19ms med=413.4ms  max=831.11ms p(90)=438.73ms p(95)=451.04ms p(99.9)=764.59ms
       { expected_response:true }...: avg=412.12ms min=212.19ms med=413.4ms  max=831.11ms p(90)=438.73ms p(95)=451.04ms p(99.9)=764.59ms
     http_req_failed................: 0.00%   ✓ 0         ✗ 7346
     http_req_receiving.............: avg=68.16µs  min=27.82µs  med=58.58µs  max=2.09ms   p(90)=97.34µs  p(95)=111.23µs p(99.9)=822.63µs
     http_req_sending...............: avg=94.12µs  min=5.29µs   med=12.99µs  max=201.48ms p(90)=20.58µs  p(95)=24.22µs  p(99.9)=5.88ms  
     http_req_tls_handshaking.......: avg=0s       min=0s       med=0s       max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s      
     http_req_waiting...............: avg=411.96ms min=212.09ms med=413.31ms max=830.2ms  p(90)=438.58ms p(95)=450.92ms p(99.9)=763.69ms
     http_reqs......................: 7346    89.925298/s
     iteration_duration.............: avg=415.23ms min=292.25ms med=413.84ms max=868.78ms p(90)=439.28ms p(95)=451.44ms p(99.9)=797.02ms
     iterations.....................: 7246    88.701158/s
     success_rate...................: 100.00% ✓ 7246      ✗ 0   
     vus............................: 50      min=0       max=50
     vus_max........................: 50      min=50      max=50

Performance Overview

Performance Overview

HTTP Overview

HTTP Overview

@github-actions
Copy link

github-actions bot commented Oct 26, 2025

Overview for: constant-vus-subgraphs-delay-resources

This scenario runs 4 subgraphs and a GraphQL gateway with Federation spec, and runs a heavy query. It's being executed with a constant amount of VUs over a fixed amount of time. It measure things like memory usage, CPU usage, average RPS. It also includes a summary of the entire execution, and metrics information about HTTP execution times.

This scenario was running 50 VUs over 60s

Comparison

Comparison

Gateway RPS ⬇️ Requests Duration Notes
grafbase 173 14040 total, 0 failed avg: 215ms, p95: 227ms
hive-router 169 13690 total, 0 failed avg: 221ms, p95: 231ms
cosmo 167 13593 total, 0 failed avg: 222ms, p95: 245ms
hive-gateway-bun 147 11981 total, 0 failed avg: 252ms, p95: 268ms
hive-gateway 146 11968 total, 0 failed avg: 253ms, p95: 267ms
apollo-router 135 11040 total, 0 failed avg: 274ms, p95: 319ms
apollo-gateway 124 10150 total, 0 failed avg: 298ms, p95: 328ms
Summary for: `grafbase`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 41820      ✗ 0    
     data_received..................: 1.2 GB  15 MB/s
     data_sent......................: 16 MB   202 kB/s
     http_req_blocked...............: avg=21.04µs  min=1.18µs   med=1.86µs   max=10.14ms  p(90)=3.27µs   p(95)=4.85µs   p(99.9)=7.6ms   
     http_req_connecting............: avg=18.52µs  min=0s       med=0s       max=10.1ms   p(90)=0s       p(95)=0s       p(99.9)=7.58ms  
     http_req_duration..............: avg=215.36ms min=128.45ms med=214.63ms max=615.54ms p(90)=224.01ms p(95)=226.77ms p(99.9)=511.16ms
       { expected_response:true }...: avg=215.36ms min=128.45ms med=214.63ms max=615.54ms p(90)=224.01ms p(95)=226.77ms p(99.9)=511.16ms
     http_req_failed................: 0.00%   ✓ 0          ✗ 14040
     http_req_receiving.............: avg=110.88µs min=24.63µs  med=44.84µs  max=380.27ms p(90)=92.57µs  p(95)=249.72µs p(99.9)=5.78ms  
     http_req_sending...............: avg=103.24µs min=5.5µs    med=8.3µs    max=341.93ms p(90)=27.49µs  p(95)=95.17µs  p(99.9)=9.31ms  
     http_req_tls_handshaking.......: avg=0s       min=0s       med=0s       max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s      
     http_req_waiting...............: avg=215.15ms min=128.4ms  med=214.52ms max=588.86ms p(90)=223.83ms p(95)=226.58ms p(99.9)=486.43ms
     http_reqs......................: 14040   173.341305/s
     iteration_duration.............: avg=215.69ms min=128.69ms med=214.87ms max=635.72ms p(90)=224.2ms  p(95)=227ms    p(99.9)=525.95ms
     iterations.....................: 13940   172.10668/s
     success_rate...................: 100.00% ✓ 13940      ✗ 0    
     vus............................: 50      min=0        max=50 
     vus_max........................: 50      min=50       max=50 

Performance Overview

Performance Overview

HTTP Overview

HTTP Overview
Summary for: `hive-router`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 40770      ✗ 0    
     data_received..................: 1.2 GB  15 MB/s
     data_sent......................: 16 MB   197 kB/s
     http_req_blocked...............: avg=20.82µs  min=1.08µs   med=1.9µs    max=10.15ms  p(90)=3.05µs   p(95)=4.23µs   p(99.9)=7.47ms  
     http_req_connecting............: avg=18.42µs  min=0s       med=0s       max=10.11ms  p(90)=0s       p(95)=0s       p(99.9)=7.45ms  
     http_req_duration..............: avg=221.02ms min=116.74ms med=220.07ms max=614.17ms p(90)=228.5ms  p(95)=231.34ms p(99.9)=522.17ms
       { expected_response:true }...: avg=221.02ms min=116.74ms med=220.07ms max=614.17ms p(90)=228.5ms  p(95)=231.34ms p(99.9)=522.17ms
     http_req_failed................: 0.00%   ✓ 0          ✗ 13690
     http_req_receiving.............: avg=91.31µs  min=23.54µs  med=40.44µs  max=177.12ms p(90)=84.05µs  p(95)=244.35µs p(99.9)=5.22ms  
     http_req_sending...............: avg=137.68µs min=5.37µs   med=8.18µs   max=377.81ms p(90)=27.51µs  p(95)=95.53µs  p(99.9)=6.42ms  
     http_req_tls_handshaking.......: avg=0s       min=0s       med=0s       max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s      
     http_req_waiting...............: avg=220.79ms min=114.55ms med=219.97ms max=602.18ms p(90)=228.32ms p(95)=231.05ms p(99.9)=495.07ms
     http_reqs......................: 13690   169.080256/s
     iteration_duration.............: avg=221.38ms min=116.89ms med=220.33ms max=627.53ms p(90)=228.69ms p(95)=231.51ms p(99.9)=537.42ms
     iterations.....................: 13590   167.845192/s
     success_rate...................: 100.00% ✓ 13590      ✗ 0    
     vus............................: 50      min=0        max=50 
     vus_max........................: 50      min=50       max=50 

Performance Overview

Performance Overview

HTTP Overview

HTTP Overview
Summary for: `cosmo`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 40479      ✗ 0    
     data_received..................: 1.2 GB  15 MB/s
     data_sent......................: 16 MB   195 kB/s
     http_req_blocked...............: avg=22.32µs  min=1.38µs   med=2.64µs   max=10.06ms  p(90)=3.88µs   p(95)=4.83µs   p(99.9)=7.56ms  
     http_req_connecting............: avg=18.91µs  min=0s       med=0s       max=10.03ms  p(90)=0s       p(95)=0s       p(99.9)=7.55ms  
     http_req_duration..............: avg=222.45ms min=155.76ms med=221.38ms max=635.81ms p(90)=239.27ms p(95)=245.14ms p(99.9)=519.22ms
       { expected_response:true }...: avg=222.45ms min=155.76ms med=221.38ms max=635.81ms p(90)=239.27ms p(95)=245.14ms p(99.9)=519.22ms
     http_req_failed................: 0.00%   ✓ 0          ✗ 13593
     http_req_receiving.............: avg=442.85µs min=31.91µs  med=60.84µs  max=84.39ms  p(90)=326.96µs p(95)=751.6µs  p(99.9)=29.55ms 
     http_req_sending...............: avg=80.12µs  min=7.06µs   med=10.87µs  max=186.65ms p(90)=31.24µs  p(95)=120.17µs p(99.9)=4.2ms   
     http_req_tls_handshaking.......: avg=0s       min=0s       med=0s       max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s      
     http_req_waiting...............: avg=221.93ms min=155.68ms med=220.9ms  max=634.49ms p(90)=238.63ms p(95)=244.5ms  p(99.9)=518.06ms
     http_reqs......................: 13593   167.654777/s
     iteration_duration.............: avg=222.89ms min=156ms    med=221.73ms max=667.59ms p(90)=239.56ms p(95)=245.49ms p(99.9)=551.87ms
     iterations.....................: 13493   166.421387/s
     success_rate...................: 100.00% ✓ 13493      ✗ 0    
     vus............................: 50      min=0        max=50 
     vus_max........................: 50      min=50       max=50 

Performance Overview

Performance Overview

HTTP Overview

HTTP Overview
Summary for: `hive-gateway-bun`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 35643      ✗ 0    
     data_received..................: 1.1 GB  13 MB/s
     data_sent......................: 14 MB   171 kB/s
     http_req_blocked...............: avg=25.12µs  min=1.11µs   med=2.38µs   max=10.73ms  p(90)=4.01µs   p(95)=5.2µs    p(99.9)=8.13ms  
     http_req_connecting............: avg=22µs     min=0s       med=0s       max=10.69ms  p(90)=0s       p(95)=0s       p(99.9)=8.11ms  
     http_req_duration..............: avg=252.46ms min=188.6ms  med=250.29ms max=717.9ms  p(90)=262.06ms p(95)=267.67ms p(99.9)=604.35ms
       { expected_response:true }...: avg=252.46ms min=188.6ms  med=250.29ms max=717.9ms  p(90)=262.06ms p(95)=267.67ms p(99.9)=604.35ms
     http_req_failed................: 0.00%   ✓ 0          ✗ 11981
     http_req_receiving.............: avg=77.04µs  min=24.78µs  med=44.47µs  max=7.96ms   p(90)=94.6µs   p(95)=242.35µs p(99.9)=2.26ms  
     http_req_sending...............: avg=122.08µs min=5.19µs   med=10µs     max=250.41ms p(90)=34.86µs  p(95)=113.65µs p(99.9)=4.57ms  
     http_req_tls_handshaking.......: avg=0s       min=0s       med=0s       max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s      
     http_req_waiting...............: avg=252.26ms min=188.55ms med=250.19ms max=717.2ms  p(90)=261.94ms p(95)=267.49ms p(99.9)=602.81ms
     http_reqs......................: 11981   147.25829/s
     iteration_duration.............: avg=253.09ms min=188.85ms med=250.58ms max=726.07ms p(90)=262.32ms p(95)=268.01ms p(99.9)=621.39ms
     iterations.....................: 11881   146.029191/s
     success_rate...................: 100.00% ✓ 11881      ✗ 0    
     vus............................: 50      min=0        max=50 
     vus_max........................: 50      min=50       max=50 

Performance Overview

Performance Overview

HTTP Overview

HTTP Overview
Summary for: `hive-gateway`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 35604      ✗ 0    
     data_received..................: 1.1 GB  13 MB/s
     data_sent......................: 14 MB   171 kB/s
     http_req_blocked...............: avg=25.36µs  min=1.17µs   med=2.55µs   max=11.23ms  p(90)=4.18µs   p(95)=5.29µs   p(99.9)=8.31ms  
     http_req_connecting............: avg=21.95µs  min=0s       med=0s       max=11.19ms  p(90)=0s       p(95)=0s       p(99.9)=8.21ms  
     http_req_duration..............: avg=252.54ms min=195.18ms med=250.12ms max=764.03ms p(90)=260.83ms p(95)=266.66ms p(99.9)=657.56ms
       { expected_response:true }...: avg=252.54ms min=195.18ms med=250.12ms max=764.03ms p(90)=260.83ms p(95)=266.66ms p(99.9)=657.56ms
     http_req_failed................: 0.00%   ✓ 0          ✗ 11968
     http_req_receiving.............: avg=79.7µs   min=26.38µs  med=46.9µs   max=12.31ms  p(90)=98.48µs  p(95)=158.29µs p(99.9)=2.97ms  
     http_req_sending...............: avg=146.94µs min=5.3µs    med=10.29µs  max=258.84ms p(90)=33.29µs  p(95)=98.75µs  p(99.9)=6.86ms  
     http_req_tls_handshaking.......: avg=0s       min=0s       med=0s       max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s      
     http_req_waiting...............: avg=252.31ms min=195.11ms med=250.01ms max=762.85ms p(90)=260.67ms p(95)=266.5ms  p(99.9)=656.12ms
     http_reqs......................: 11968   146.868301/s
     iteration_duration.............: avg=253.23ms min=195.32ms med=250.41ms max=788.28ms p(90)=261.14ms p(95)=267ms    p(99.9)=680.92ms
     iterations.....................: 11868   145.641126/s
     success_rate...................: 100.00% ✓ 11868      ✗ 0    
     vus............................: 50      min=0        max=50 
     vus_max........................: 50      min=50       max=50 

Performance Overview

Performance Overview

HTTP Overview

HTTP Overview
Summary for: `apollo-router`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 32820      ✗ 0    
     data_received..................: 969 MB  12 MB/s
     data_sent......................: 13 MB   158 kB/s
     http_req_blocked...............: avg=23.31µs  min=1.39µs   med=2.82µs   max=7.92ms   p(90)=4.56µs   p(95)=5.73µs   p(99.9)=6.29ms  
     http_req_connecting............: avg=18.48µs  min=0s       med=0s       max=7.89ms   p(90)=0s       p(95)=0s       p(99.9)=6.27ms  
     http_req_duration..............: avg=273.58ms min=168.87ms med=273.36ms max=698.79ms p(90)=308.23ms p(95)=318.58ms p(99.9)=615.3ms 
       { expected_response:true }...: avg=273.58ms min=168.87ms med=273.36ms max=698.79ms p(90)=308.23ms p(95)=318.58ms p(99.9)=615.3ms 
     http_req_failed................: 0.00%   ✓ 0          ✗ 11040
     http_req_receiving.............: avg=140.34µs min=30.63µs  med=55.32µs  max=137.53ms p(90)=156.41µs p(95)=453.07µs p(99.9)=4.72ms  
     http_req_sending...............: avg=142.23µs min=5.84µs   med=11.5µs   max=256.8ms  p(90)=46.02µs  p(95)=142.99µs p(99.9)=4.74ms  
     http_req_tls_handshaking.......: avg=0s       min=0s       med=0s       max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s      
     http_req_waiting...............: avg=273.3ms  min=168.77ms med=273.13ms max=682.51ms p(90)=307.93ms p(95)=318.18ms p(99.9)=601.44ms
     http_reqs......................: 11040   135.432253/s
     iteration_duration.............: avg=274.55ms min=169.13ms med=274.06ms max=718.53ms p(90)=308.75ms p(95)=318.99ms p(99.9)=629.08ms
     iterations.....................: 10940   134.205512/s
     success_rate...................: 100.00% ✓ 10940      ✗ 0    
     vus............................: 50      min=0        max=50 
     vus_max........................: 50      min=50       max=50 

Performance Overview

Performance Overview

HTTP Overview

HTTP Overview
Summary for: `apollo-gateway`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 30150      ✗ 0    
     data_received..................: 892 MB  11 MB/s
     data_sent......................: 12 MB   145 kB/s
     http_req_blocked...............: avg=34.23µs  min=1.37µs   med=3.26µs   max=11.46ms  p(90)=4.84µs   p(95)=5.71µs   p(99.9)=9.32ms  
     http_req_connecting............: avg=28.89µs  min=0s       med=0s       max=11.43ms  p(90)=0s       p(95)=0s       p(99.9)=9.25ms  
     http_req_duration..............: avg=297.93ms min=210.26ms med=296.08ms max=818.54ms p(90)=320.04ms p(95)=327.88ms p(99.9)=710.03ms
       { expected_response:true }...: avg=297.93ms min=210.26ms med=296.08ms max=818.54ms p(90)=320.04ms p(95)=327.88ms p(99.9)=710.03ms
     http_req_failed................: 0.00%   ✓ 0          ✗ 10150
     http_req_receiving.............: avg=76.57µs  min=29.93µs  med=57.43µs  max=5.41ms   p(90)=106.9µs  p(95)=127.24µs p(99.9)=1.9ms   
     http_req_sending...............: avg=89.71µs  min=6.33µs   med=12.88µs  max=241.15ms p(90)=23.27µs  p(95)=39.67µs  p(99.9)=3ms     
     http_req_tls_handshaking.......: avg=0s       min=0s       med=0s       max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s      
     http_req_waiting...............: avg=297.77ms min=210.18ms med=295.97ms max=817.69ms p(90)=319.9ms  p(95)=327.74ms p(99.9)=708.83ms
     http_reqs......................: 10150   124.414692/s
     iteration_duration.............: avg=299.3ms  min=221.71ms med=296.55ms max=857.84ms p(90)=320.46ms p(95)=328.37ms p(99.9)=738.88ms
     iterations.....................: 10050   123.188932/s
     success_rate...................: 100.00% ✓ 10050      ✗ 0    
     vus............................: 50      min=0        max=50 
     vus_max........................: 50      min=50       max=50 

Performance Overview

Performance Overview

HTTP Overview

HTTP Overview

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants