Skip to content

Automate Reconfiguration Performance Test #2313

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 42 commits into from
Aug 12, 2024
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
42 commits
Select commit Hold shift + click to select a range
57a4073
Add code implementation of resource deploying scripts
bjee19 Jul 8, 2024
5e12b36
Add functions to check for resources and clean them up
bjee19 Jul 10, 2024
8a04efc
Add results template
bjee19 Jul 10, 2024
d233c92
Add basic prometheus metrics
bjee19 Jul 12, 2024
d2ad309
Add queries
bjee19 Jul 12, 2024
9358660
Add queries and results
bjee19 Jul 17, 2024
4da88d5
Add time to ready total and average
bjee19 Jul 29, 2024
946a547
Add logic documentation for time to ready average
bjee19 Jul 29, 2024
4c7aad1
Add test run on gke
bjee19 Jul 30, 2024
6f94138
Add test description
bjee19 Jul 30, 2024
25e5502
Remove completed TODO
bjee19 Jul 30, 2024
898b94c
Sort imports
bjee19 Aug 1, 2024
b99121f
Change timeouts for creating resources
bjee19 Aug 1, 2024
0d0f29b
Remove debugging print lines
bjee19 Aug 1, 2024
107898b
Add test results from 150 numResource run
bjee19 Aug 1, 2024
7f38b8f
Remove certificate namespace
bjee19 Aug 2, 2024
20a1f17
Delete entire namespace instead of specific manifest resources
bjee19 Aug 3, 2024
43d2b85
Change comparison on HTTPRoutes to be equal operator
bjee19 Aug 5, 2024
377778e
Add units to results
bjee19 Aug 5, 2024
bfe2f13
Refactor reconfig namespace to be declared in beforeeach
bjee19 Aug 5, 2024
e7709d7
Refactor cleanupResources and cleanup after each test
bjee19 Aug 5, 2024
714be9f
Add reconfiguration test to nfr checks in system suite test
bjee19 Aug 5, 2024
bc4862d
Close out file and redeploy NGF for future tests
bjee19 Aug 5, 2024
0ee057f
Add constant maxResourceCount
bjee19 Aug 5, 2024
97b819a
Refactor setup steps
bjee19 Aug 6, 2024
531bc4e
Refactor ginkgo test specs
bjee19 Aug 6, 2024
509cbdd
Refactor scale test to include framework queries
bjee19 Aug 7, 2024
dc8125f
Refactor blind sleep to instead wait for pods to be ready
bjee19 Aug 7, 2024
410d68d
Add and use ApplyFromBuffer to create unique resources
bjee19 Aug 8, 2024
0d3c604
Delete old reconfiguration files
bjee19 Aug 8, 2024
85b24ef
Revert "Delete old reconfiguration files"
bjee19 Aug 8, 2024
03ed0bc
Revert full delete of reconfiguration files and keep results files
bjee19 Aug 8, 2024
3bf27e2
Elaborate on test description
bjee19 Aug 8, 2024
35159e9
Add note on node size
bjee19 Aug 8, 2024
fe1f414
Change package to main
bjee19 Aug 8, 2024
9842b1e
Refactor queries.go function layout to be more readable
bjee19 Aug 9, 2024
2d753a4
Remove unnecessary return and correct placement of creation of resour…
bjee19 Aug 9, 2024
cdf4387
Use time constant layout when parsing logs
bjee19 Aug 9, 2024
c67265c
Move queries to different file and add descriptions to functions
bjee19 Aug 9, 2024
e1720b6
Add FIXME on issues
bjee19 Aug 12, 2024
685295d
Add latest results
bjee19 Aug 12, 2024
683cd36
Remove latest results
bjee19 Aug 12, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Add test run on gke
  • Loading branch information
bjee19 committed Aug 12, 2024
commit 4c7aad137fbbb4ac5a5974b7cea856e3c38b74ed
101 changes: 101 additions & 0 deletions tests/results/reconfig/v1.3.0/1.3.0-oss.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,101 @@
# Results

## Test environment

NGINX Plus: false

GKE Cluster:

- Node count: 3
- k8s version: v1.29.6-gke.1038001
- vCPUs per node: 2
- RAM per node: 4019168Ki
- Max pods per node: 110
- Zone: us-central1-c
- Instance Type: e2-medium

## Test 1 NumResources 30

### Reloads and Time to Ready

- TimeToReadyTotal: 1
- TimeToReadyAvgSingle: < 1
- NGINX Reloads: 2
- NGINX Reload Average Time: 97
- Reload distribution:
- 500ms: 2
- 1000ms: 2
- 5000ms: 2
- 10000ms: 2
- 30000ms: 2
- +Infms: 2

### Event Batch Processing

- Event Batch Total: 6
- Event Batch Processing Average Time: 70ms
- Event Batch Processing distribution:
- 500ms: 6
- 1000ms: 6
- 5000ms: 6
- 10000ms: 6
- 30000ms: 6
- +Infms: 6


## Test 2 NumResources 30

### Reloads and Time to Ready

- TimeToReadyTotal: 10
- TimeToReadyAvgSingle: < 1
- NGINX Reloads: 63
- NGINX Reload Average Time: 129
- Reload distribution:
- 500ms: 63
- 1000ms: 63
- 5000ms: 63
- 10000ms: 63
- 30000ms: 63
- +Infms: 63

### Event Batch Processing

- Event Batch Total: 339
- Event Batch Processing Average Time: 29ms
- Event Batch Processing distribution:
- 500ms: 339
- 1000ms: 339
- 5000ms: 339
- 10000ms: 339
- 30000ms: 339
- +Infms: 339


## Test 3 NumResources 30

### Reloads and Time to Ready

- TimeToReadyTotal: < 1
- TimeToReadyAvgSingle: < 1
- NGINX Reloads: 66
- NGINX Reload Average Time: 131
- Reload distribution:
- 500ms: 66
- 1000ms: 66
- 5000ms: 66
- 10000ms: 66
- 30000ms: 66
- +Infms: 66

### Event Batch Processing

- Event Batch Total: 338
- Event Batch Processing Average Time: 26ms
- Event Batch Processing distribution:
- 500ms: 338
- 1000ms: 338
- 5000ms: 338
- 10000ms: 338
- 30000ms: 338
- +Infms: 338
39 changes: 36 additions & 3 deletions tests/suite/reconfig_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -496,7 +496,7 @@ var _ = Describe("Reconfiguration Performance Testing", Ordered, Label("reconfig
cleanupResources(30)
}

It("test 1", func() {
It("test 1 - 30 resources", func() {
timeToReadyStartingLogSubstring := "Starting NGINX Gateway Fabric"

runTestWithMetrics("1",
Expand All @@ -507,7 +507,18 @@ var _ = Describe("Reconfiguration Performance Testing", Ordered, Label("reconfig
)
})

It("test 2", func() {
//It("test 1 - 150 resources", func() {
// timeToReadyStartingLogSubstring := "Starting NGINX Gateway Fabric"
//
// runTestWithMetrics("1",
// 150,
// createResourcesGWLast,
// false,
// timeToReadyStartingLogSubstring,
// )
//})

It("test 2 - 30 resources", func() {
timeToReadyStartingLogSubstring := "Reconciling the resource\",\"controller\":\"httproute\""

runTestWithMetrics("2",
Expand All @@ -518,7 +529,18 @@ var _ = Describe("Reconfiguration Performance Testing", Ordered, Label("reconfig
)
})

It("test 3", func() {
//It("test 2 - 150 resources", func() {
// timeToReadyStartingLogSubstring := "Reconciling the resource\",\"controller\":\"httproute\""
//
// runTestWithMetrics("2",
// 150,
// createResourcesRoutesLast,
// true,
// timeToReadyStartingLogSubstring,
// )
//})

It("test 3 - 30 resources", func() {
timeToReadyStartingLogSubstring := "Reconciling the resource\",\"controller\":\"gateway\""

runTestWithMetrics("3",
Expand All @@ -528,6 +550,17 @@ var _ = Describe("Reconfiguration Performance Testing", Ordered, Label("reconfig
timeToReadyStartingLogSubstring,
)
})

//It("test 3 - 150 resources", func() {
// timeToReadyStartingLogSubstring := "Reconciling the resource\",\"controller\":\"gateway\""
//
// runTestWithMetrics("3",
// 150,
// createResourcesGWLast,
// true,
// timeToReadyStartingLogSubstring,
// )
//})
})

type reconfigTestResults struct {
Expand Down