Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Profile/backend/profiling #291

Merged
merged 6 commits into from
Aug 10, 2024
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Next Next commit
chore: added some methods for prof and benchmarking
  • Loading branch information
waveyboym committed Jul 30, 2024
commit ae065db4c64590a80d7de3eac7f0a79165e21deb
85 changes: 85 additions & 0 deletions .github/workflows/load-test-system.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,85 @@
name: Load Test System 🏋️💪

# trigger action to run once a week on Sunday at 12:00 AM
on:
schedule:
- cron: "0 0 * * 0"

workflow_dispatch:

defaults:
run:
working-directory: testing/load

jobs:
load_test_backend:
name: 🏋️💪 Load Test Backend 🔌
runs-on: ubuntu-latest

steps:
- name: ⬇️ Checkout repository
uses: actions/checkout@v4

- name: 🏗 Install k6
run: |
sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys C5AD17C747E3415A3642D57D77C6C491D6AC1D69
echo "deb https://dl.k6.io/deb stable main" | sudo tee /etc/apt/sources.list.d/k6.list
sudo apt-get update
sudo apt-get install k6

- name: 👩‍💻 Login to k6
run: k6 login cloud --token ${{ secrets.K6_CLOUD_TOKEN }}

- name: 🏋️💪 Run load test
run: k6 cloud backend.js

- name: ✅ Load test completed
run: echo "Load test completed successfully!"

load_test_web:
name: 🏋️💪 Load Test Web 🌐
runs-on: ubuntu-latest

steps:
- name: ⬇️ Checkout repository
uses: actions/checkout@v4

- name: 🏗 Install k6
run: |
sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys C5AD17C747E3415A3642D57D77C6C491D6AC1D69
echo "deb https://dl.k6.io/deb stable main" | sudo tee /etc/apt/sources.list.d/k6.list
sudo apt-get update
sudo apt-get install k6

- name: 👩‍💻 Login to k6
run: k6 login cloud --token ${{ secrets.K6_CLOUD_TOKEN }}

- name: 🏋️💪 Run load test
run: k6 cloud web.js

- name: ✅ Load test completed
run: echo "Load test completed successfully!"

load_test_landing_page:
name: 🏋️💪 Load Test Landing Page 🏠
runs-on: ubuntu-latest

steps:
- name: ⬇️ Checkout repository
uses: actions/checkout@v4

- name: 🏗 Install k6
run: |
sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys C5AD17C747E3415A3642D57D77C6C491D6AC1D69
echo "deb https://dl.k6.io/deb stable main" | sudo tee /etc/apt/sources.list.d/k6.list
sudo apt-get update
sudo apt-get install k6

- name: 👩‍💻 Login to k6
run: k6 login cloud --token ${{ secrets.K6_CLOUD_TOKEN }}

- name: 🏋️💪 Run load test
run: k6 cloud landing-page.js

- name: ✅ Load test completed
run: echo "Load test completed successfully!"
2 changes: 0 additions & 2 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,5 +1,3 @@
/benchmarks

documentation/SRS-Docs/SRS.aux
documentation/SRS-Docs/SRS.fdb_latexmk
documentation/SRS-Docs/SRS.fls
Expand Down
8 changes: 8 additions & 0 deletions occupi-backend/cmd/occupi-backend/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,9 @@ package main

import (
"flag"
"log"
"net/http"
_ "net/http/pprof"
"time"

"github.com/gin-contrib/cors"
Expand Down Expand Up @@ -167,6 +170,11 @@ func runServer(ginRouter *gin.Engine) {
logrus.Infof("Server running with cert file: %s", certFile)
logrus.Infof("Server running with key file: %s", keyFile)

// profile the server using pprof
go func() {
log.Println(http.ListenAndServe("localhost:6060", nil))
}()

// Listening on the port with TLS if env is prod or dev.deployed
if configs.GetEnv() == "prod" || configs.GetEnv() == "devdeployed" {
if err := ginRouter.RunTLS(":"+configs.GetPort(), certFile, keyFile); err != nil {
Expand Down
16 changes: 16 additions & 0 deletions testing/benchmarks/profile-usage
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
go tool pprof http://localhost:6060/debug/pprof/profile?seconds=30

At terminal (pprof) is showing type help it will give the list of pprof commands as below

1. top: It displays a list of the top functions that consume CPU time. It shows which functions are the most CPU-intensive.
2. list <function>: listcommand followed by the name of a function to display the source code for that function, highlighting the lines where the most CPU time is spent.
3. web: This command generates an interactive graphical visualization of the profile data. It opens a web browser with a graphical representation of the call graph, making it easier to identify performance bottlenecks.
4. web list: This command combines the web and list commands. It generates a web-based visualisation and allows you to click on functions in the visualisation to see their source code with highlighted hotspots.
5. peek <function>: The peek command displays a summary of the profile data for a specific function, including the percentage of total CPU time it consumes and the number of times it was called.
6. disasm <function>: The disasm command displays the assembly code for a specific function. This can be useful for low-level performance analysis.
7. pdf: The pdf command generates a PDF file containing the call graph visualisation. This is useful for sharing profiling results or documenting performance improvements.
8. text: The text command displays the profile data in text form, showing the top functions and their CPU usage. It's a simple textual representation of the profiling data.
9. topN <N>: Use topN followed by a number (e.g., top10) to display the top N functions consuming CPU time. This can help you focus on the most significant bottlenecks.
10. raw: The raw command displays the raw profiling data in a machine-readable format. This is useful for advanced analysis or automation.
11. tags: The tags command displays all available pprof tags in the profile data. Tags can provide additional context for profiling results.
12. quit or exit: Use either of these commands to exit the pprof interactive session.
59 changes: 59 additions & 0 deletions testing/load/test.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
import http from 'k6/http';
import { sleep } from 'k6';

export const options = {
// A number specifying the number of VUs to run concurrently.
vus: 10,
// A string specifying the total duration of the test run.
duration: '30s',

// The following section contains configuration options for execution of this
// test script in Grafana Cloud.
//
// See https://grafana.com/docs/grafana-cloud/k6/get-started/run-cloud-tests-from-the-cli/
// to learn about authoring and running k6 test scripts in Grafana k6 Cloud.
//
cloud: {
// The ID of the project to which the test is assigned in the k6 Cloud UI.
// By default tests are executed in default project.
projectID: 3707181,
// The name of the test in the k6 Cloud UI.
// Test runs with the same name will be grouped.
name: "Testing occupi endpoints to full fill performance requirements",
},

// Uncomment this section to enable the use of Browser API in your tests.
//
// See https://grafana.com/docs/k6/latest/using-k6-browser/running-browser-tests/ to learn more
// about using Browser API in your test scripts.
//
// scenarios: {
// // The scenario name appears in the result summary, tags, and so on.
// // You can give the scenario any name, as long as each name in the script is unique.
// ui: {
// // Executor is a mandatory parameter for browser-based tests.
// // Shared iterations in this case tells k6 to reuse VUs to execute iterations.
// //
// // See https://grafana.com/docs/k6/latest/using-k6/scenarios/executors/ for other executor types.
// executor: 'shared-iterations',
// options: {
// browser: {
// // This is a mandatory parameter that instructs k6 to launch and
// // connect to a chromium-based browser, and use it to run UI-based
// // tests.
// type: 'chromium',
// },
// },
// },
// }
};

// The function that defines VU logic.
//
// See https://grafana.com/docs/k6/latest/examples/get-started-with-k6/ to learn more
// about authoring k6 scripts.
//
export default function() {
http.get('https://test.k6.io');
sleep(1);
}
9 changes: 9 additions & 0 deletions testing/security/nikto-commands
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
git clone https://github.com/sullo/nikto.git
cd nikto
docker build -t sullo/nikto .
# Call it without arguments to display the full help
docker run --rm sullo/nikto
# Basic usage
docker run --rm sullo/nikto -h http://www.example.com
# To save the report in a specific format, mount /tmp as a volume:
docker run --rm -v $(pwd):/tmp sullo/nikto -h http://www.example.com -o /tmp/out.json