Closed
Description
In this pipeline run of the NFR test results on edge: https://github.com/nginxinc/nginx-gateway-fabric/actions/runs/10872403318, the scale test for UpstreamServers encountered an error when ran on Plus.
NGF Error:
{
"level": "error",
"ts": "2024-09-15T19:17:23Z",
"logger": "eventLoop.eventHandler",
"msg": "couldn't update upstream via the API, reloading configuration instead",
"batchID": 231,
"upstreamName": "scale_backend_80",
"error": "failed to update servers of scale_backend_80 upstream: failed to add 10.120.10.168:8080 server to scale_backend_80 upstream: expected 201 response, got 500. error.status=500; error.text=upstream memory exhausted; error.code=UpstreamOutOfMemory; request_id=12c0924ef4cfa2569609b05e32b8261b; href=https://nginx.org/en/docs/http/ngx_http_api_module.html",
"stacktrace": "github.com/nginxinc/nginx-gateway-fabric/internal/mode/static.(*eventHandlerImpl).updateUpstreamServers\n\t/home/runner/work/nginx-gateway-fabric/nginx-gateway-fabric/internal/mode/static/handler.go:374\ngithub.com/nginxinc/nginx-gateway-fabric/internal/mode/static.(*eventHandlerImpl).HandleEventBatch\n\t/home/runner/work/nginx-gateway-fabric/nginx-gateway-fabric/internal/mode/static/handler.go:200\ngithub.com/nginxinc/nginx-gateway-fabric/internal/framework/events.(*EventLoop).Start.func1.1\n\t/home/runner/work/nginx-gateway-fabric/nginx-gateway-fabric/internal/framework/events/loop.go:74"
}
Nginx Plus Error:
2024/09/15 19:17:23 [crit] 121#121: ngx_slab_alloc() failed: no memory in upstream zone "scale_backend_80"
2024/09/15 19:17:23 [crit] 23#23: ngx_slab_alloc() failed: no memory in upstream zone "scale_backend_80"
The error is the same as the one described in #2023
#2439 should have fixed this issue.
The full NFR test results can be found in this PR: #2554
Acceptance
- When the test is run ten times, there are no reported failures.
Metadata
Metadata
Assignees
Labels
Type
Projects
Status
✅ Done