Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Locust testing scripts and configurations #50

Merged
merged 7 commits into from
Dec 2, 2024
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Refactor Locust testing to single test type
  • Loading branch information
milank94 committed Dec 2, 2024
commit be8504264409ea8bcccbd8b9cab378bc784b5370
24 changes: 4 additions & 20 deletions locust/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,30 +8,14 @@ In your Python environment with model dependencies installed, install the Locust
pip install -r requirements.txt
```

## Test Types
## Load Test

### Static Test
Run a test using a dataset of variable input prompts generated by [generate_prompts](./../utils/prompt_generation.py#250).

Run a basic test with a single user and a predefined input prompt.

- Prompt: "What is in Austin Texas?"
- Prompt Length: 7 tokens
- Output Length: 128 tokens

Command to run the static test:

```bash
locust --config locust_static.conf
```

### Dynamic Test

Run a dynamic test using a dataset of variable input prompts generated by [generate_prompts](./../utils/prompt_generation.py#250).

Command to run the dynamic test:
Command to run the load test:

```bash
locust --config locust_dynamic.conf
locust --config locust_config.conf
```

## Test Configuration
Expand Down
1 change: 0 additions & 1 deletion locust/locust_dynamic.conf → locust/locust_config.conf
Original file line number Diff line number Diff line change
Expand Up @@ -4,4 +4,3 @@ host = http://localhost:7000
users = 32
spawn-rate = 1
run-time = 3m
tags = [dynamic]
7 changes: 0 additions & 7 deletions locust/locust_static.conf

This file was deleted.

20 changes: 4 additions & 16 deletions locust/locustfile.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
import jwt

from data_reader import DataReader
from locust import FastHttpUser, events, tag, task
from locust import FastHttpUser, events, task

# Constants for timeouts and API configuration
NETWORK_TIMEOUT = 300.0
Expand Down Expand Up @@ -39,13 +39,9 @@ def get_authorization():

# Event listener to load custom data before tests start
@events.test_start.add_listener
def load_custom_data(environment, **kwargs):
def load_custom_data(**kwargs):
global data_iter
if "dynamic" in environment.parsed_options.tags:
data_iter = DataReader()
print("Dynamic data loaded.")
else:
print("Dynamic data not loaded (no 'dynamic' tag).")
data_iter = DataReader()


class ServeUser(FastHttpUser):
Expand All @@ -64,16 +60,8 @@ def post_request(self, prompt: str, max_tokens: int):
response = self.client.post(API_ENDPOINT, json=json_data, headers=self.headers)
return response

@tag("static")
@task
def basic_test(self):
"""Test using a static prompt."""
prompt = "What is in Austin Texas?"
self.post_request(prompt, max_tokens=128)

@tag("dynamic")
@task
def dataset_test(self):
"""Test using dynamic prompts from a data iterator."""
"""Test using generated prompts from a data iterator."""
prompt = next(data_iter)
self.post_request(prompt, max_tokens=128)