Skip to content

Conversation

@kamilkisiela
Copy link
Contributor

@kamilkisiela kamilkisiela commented Oct 28, 2025

This commit introduces a new script to detect performance regressions. It compares the 'http_reqs' rate metric from the PR summary against the main branch summary and determines if there is a regression of more than 2%.

Also, the commit adds .gitignore entries, a script to detect regressions, and changes to the CI workflow to execute the benchmarks and compare the results.

@github-actions
Copy link

github-actions bot commented Oct 28, 2025

🐋 This PR was built and pushed to the following Docker images:

Image Names: ghcr.io/graphql-hive/router

Platforms: linux/amd64,linux/arm64

Image Tags: ghcr.io/graphql-hive/router:pr-531 ghcr.io/graphql-hive/router:sha-52abb5a

Docker metadata
{
"buildx.build.ref": "builder-a45b52bd-33cb-4549-99e9-7def76a6d499/builder-a45b52bd-33cb-4549-99e9-7def76a6d4990/97w5vilsbswt2552d4mps7185",
"containerimage.descriptor": {
  "mediaType": "application/vnd.oci.image.index.v1+json",
  "digest": "sha256:a7b181d78ec616d8c5279c1916fc135fdf3f6394b1b1b5a8d4d0b2fa5adf7f60",
  "size": 1609
},
"containerimage.digest": "sha256:a7b181d78ec616d8c5279c1916fc135fdf3f6394b1b1b5a8d4d0b2fa5adf7f60",
"image.name": "ghcr.io/graphql-hive/router:pr-531,ghcr.io/graphql-hive/router:sha-52abb5a"
}

@github-actions
Copy link

github-actions bot commented Oct 28, 2025

k6-benchmark results

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     █ setup

     checks.........................: 100.00% ✓ 226365      ✗ 0    
     data_received..................: 6.6 GB  220 MB/s
     data_sent......................: 89 MB   2.9 MB/s
     http_req_blocked...............: avg=3.11µs   min=639ns   med=1.65µs  max=11.54ms  p(90)=2.39µs  p(95)=2.77µs  
     http_req_connecting............: avg=647ns    min=0s      med=0s      max=2.07ms   p(90)=0s      p(95)=0s      
     http_req_duration..............: avg=19.45ms  min=2.06ms  med=18.53ms max=95.78ms  p(90)=26.67ms p(95)=29.73ms 
       { expected_response:true }...: avg=19.45ms  min=2.06ms  med=18.53ms max=95.78ms  p(90)=26.67ms p(95)=29.73ms 
     http_req_failed................: 0.00%   ✓ 0           ✗ 75475
     http_req_receiving.............: avg=131.82µs min=27.74µs med=44.61µs max=62ms     p(90)=79.96µs p(95)=368.81µs
     http_req_sending...............: avg=20.64µs  min=4.34µs  med=10.15µs max=18.68ms  p(90)=14.75µs p(95)=20.77µs 
     http_req_tls_handshaking.......: avg=0s       min=0s      med=0s      max=0s       p(90)=0s      p(95)=0s      
     http_req_waiting...............: avg=19.29ms  min=2ms     med=18.41ms max=73.42ms  p(90)=26.43ms p(95)=29.46ms 
     http_reqs......................: 75475   2509.870296/s
     iteration_duration.............: avg=19.87ms  min=5.63ms  med=18.85ms max=228.91ms p(90)=27.06ms p(95)=30.17ms 
     iterations.....................: 75455   2509.205209/s
     vus............................: 50      min=50        max=50 
     vus_max........................: 50      min=50        max=50 

This commit introduces a new script to detect performance regressions.
It compares the 'http_reqs' rate metric from the PR summary against the
main branch summary and determines if there is a regression of more than
5%.

Also, the commit adds .gitignore entries, a script to detect
regressions, and changes to the CI workflow to execute the benchmarks
and compare the results.
@kamilkisiela kamilkisiela force-pushed the kamil-benchmark-against-main branch from d1333c1 to 4d9015d Compare October 28, 2025 10:18
@kamilkisiela kamilkisiela marked this pull request as ready for review October 28, 2025 10:19
@kamilkisiela kamilkisiela enabled auto-merge (squash) October 28, 2025 10:51
@kamilkisiela kamilkisiela disabled auto-merge October 28, 2025 10:51
@kamilkisiela kamilkisiela enabled auto-merge (squash) October 28, 2025 10:53
if: github.event_name == 'pull_request'
uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5
with:
ref: main
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we do not chain PR a lot, but this should really be ${{ github.base_ref }}

@kamilkisiela kamilkisiela merged commit bee241d into main Oct 28, 2025
16 checks passed
@kamilkisiela kamilkisiela deleted the kamil-benchmark-against-main branch October 28, 2025 10:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants