STRESS.md

UNDER CONSTRUCTION

First, create a Kubernetes cluster

Install Keep

gcloud config set project keep-dev-429814 gcloud container clusters get-credentials keep-stress --zone us-central1-c --project keep-dev-429814 helm repo add keephq https://keephq.github.io/helm-charts helm pull keephq/keep

create the namespace

kubectl create namespace keep

install keep

helm install keep keephq/keep --namespace keep

from local

helm install keep ./charts/keep --namespace keep

kubectl -n keep describe pod keep-backend-697f6b946f-v2jxp kubectl -n keep logs keep-frontend-577fdf5497-r8ht9

Import alerts

uninstall

helm uninstall keep --namespace keep

kubectl -n keep exec -it keep-backend-64c4d7ddb7-7p5q5 /bin/bash

copy the db

kubectl -n keep exec -it keep-database-86dd6b6775-92sz4 /bin/bash kubectl -n keep cp ./keep.sql keep-database-659c69689-vxhkz:/tmp/keep.sql kubectl -n keep exec -it keep-database-659c69689-vxhkz -- bash -c "mysql -u root keep < /tmp/keep.sql"

exec into the pod

kubectl -n keep exec -it keep-database-86dd6b6775-92sz4 -- /bin/bash

import

kubectl -n keep exec -it keep-database-659c69689-vxhkz -- bash -c "mysql -u root keep < /tmp/keep.sql"

No Load

500k alerts - 1Gi/250m cpu: get_last_alerts 2 minutes and 30 seconds

Keep Backend Workers get a timeout after one minute (status code 500 for preset and alert endpoints)

500k alerts - 2Gi/500m cpu:

• default mysql: get_last_alerts 1 minutes and 30 seconds
• innodb_buffer_pool_size = 4294967296: 25 seconds, 3 seconds after cache

500k alerts - 4Gi/1 cpu: get_last_alerts 2 minutes and 30 seconds

500k alerts - 8Gi/1 cpu: get_last_alerts 2 minutes and 30 seconds

Load 10 alerts per minute

Load 100 alerts per minute

Load 1000 alerts per minute

1M alerts

Load 10 alerts per minute

Load 100 alerts per minute

Load 1000 alerts per minute