Skip to content

[Makefile] Load Test Helpers for the poral #329

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
Olshansk wants to merge 2 commits into
base: main
Choose a base branch
from
Draft

[Makefile] Load Test Helpers for the poral #329

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
5231fca
Quick first iteration
Olshansk Jul 1, 2025
40796a7
Merge branch 'main' into load_test_improvements
Olshansk Jul 6, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions docusaurus/docs/develop/path/7_e2e_and_load_tests.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ description: End-to-End & Load Tests for PATH

## tl;dr How do I run the tests?

### 1. **Run E2E tests**
### 1. **Run E2E tests**

```bash
# Shannon E2E tests with all service IDs
Expand Down Expand Up @@ -111,9 +111,9 @@ PATH E2E tests support two distinct modes of operation:

| Configuration File | E2E Test (Required?) | Load Test (Required?) | Default available? |
| -------------------------------------------------- | :------------------: | :-------------------: | :-----------------------------------------: |
| `./e2e/config/.morse.config.yaml` (for Morse) | ✅ | ❌ | ❌ |
| `./e2e/config/.shannon.config.yaml` (for Shannon) | ✅ | ❌ | ❌ |
| `./e2e/config/.e2e_load_test.config.yaml` (custom) | ❌ | ✅ | `e2e/config/e2e_load_test.config.tmpl.yaml` |
| `./e2e/config/.morse.config.yaml` (for Morse) | ✅ | ❌ | ❌ |
| `./e2e/config/.shannon.config.yaml` (for Shannon) | ✅ | ❌ | ❌ |
| `./e2e/config/.e2e_load_test.config.yaml` (custom) | ❌ | ✅ | `e2e/config/e2e_load_test.config.tmpl.yaml` |

:::tip Populate Configs

Expand Down
87 changes: 87 additions & 0 deletions makefiles/test.mk
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,93 @@ load_test: ## Run a Shannon load test with specified service IDs (e.g. make load
fi
@(cd e2e && TEST_MODE=load TEST_PROTOCOL=shannon TEST_SERVICE_IDS=$(filter-out $@,$(MAKECMDGOALS)) go test -v -tags=e2e -count=1 -run Test_PATH_E2E)


##################
### Portal Tests ###
##################

.PHONY: make_load_test_portal_local_path
Copy link
Contributor

@commoddity commoddity Jul 1, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The main issue here is that in local load testing PATH has no concept of a "Portal App ID" so this target doesn't really make sense.

For testing against production the concept is valid however.

This could, of course, change if/when we update the load tests to run against PATH + GUARD locally, but even then the default GUARD configuration uses only Envoy's API key auth, not our custom Grove-legacy Portal App ID/Secret Key auth.

In order for Portal App ID to make sense for local testing we would need to update:

  1. Load tests to run against PATH + GUARD (currently they just run against a local PATH binary with no auth)
  2. GUARD to run with Grove-legacy auth (PEAS, GUARD chart set to use Grove auth) (currently GUARD uses Envoy API key auth by default, not Grove-legacy auth)

make_load_test_portal_local_path: ## Run a Shannon load test against local PATH with Portal credentials
@echo "📋 Copying archival config..."
@cp e2e/config/.grove.archival.e2e_load_test.config.yaml e2e/config/.e2e_load_test.config.yaml
@echo "✅ Config copied successfully"
@echo ""
@echo "🔍 Checking environment variables..."
@if [ -z "$(PORTAL_APPLICATION_ID)" ]; then \
echo "❌ Error: PORTAL_APPLICATION_ID environment variable is not set"; \
echo " 🔐 Grove employees: Get credentials from https://share.1password.com/s#x4x-cWnkf4GUw50BwV166aWEEevZV1nmaO2RxJssWjg"; \
echo " 💡 Set with: export PORTAL_APPLICATION_ID=your_app_id"; \
exit 1; \
fi
@if [ -z "$(PORTAL_API_KEY)" ]; then \
echo "❌ Error: PORTAL_API_KEY environment variable is not set"; \
echo " 🔐 Grove employees: Get credentials from https://share.1password.com/s#x4x-cWnkf4GUw50BwV166aWEEevZV1nmaO2RxJssWjg"; \
echo " 💡 Set with: export PORTAL_API_KEY=your_api_key"; \
exit 1; \
fi
@echo "✅ Environment variables are set"
@echo ""
@echo "⚙️ Configuring for LOCAL PATH..."
@if [[ "$$(uname)" == "Darwin" ]]; then \
sed -i '' 's/portal_application_id: ""/portal_application_id: "$(PORTAL_APPLICATION_ID)"/; s/portal_api_key: ""/portal_api_key: "test_api_key"/' e2e/config/.e2e_load_test.config.yaml; \
else \
sed -i 's/portal_application_id: ""/portal_application_id: "$(PORTAL_APPLICATION_ID)"/; s/portal_api_key: ""/portal_api_key: "test_api_key"/' e2e/config/.e2e_load_test.config.yaml; \
fi
@echo "✅ Config updated for local PATH"
@echo ""
@if [ "$(filter-out $@,$(MAKECMDGOALS))" = "" ]; then \
echo "🚀 Running load test with default service IDs: bsc,eth,pocket,poly,gnosis,xrpl_evm_test"; \
(cd e2e && TEST_MODE=load TEST_PROTOCOL=shannon TEST_SERVICE_IDS=bsc,eth,pocket,poly,gnosis,xrpl_evm_test go test -v -tags=e2e -count=1 -run Test_PATH_E2E); \
else \
echo "🚀 Running load test with service IDs: $(filter-out $@,$(MAKECMDGOALS))"; \
(cd e2e && TEST_MODE=load TEST_PROTOCOL=shannon TEST_SERVICE_IDS=$(filter-out $@,$(MAKECMDGOALS)) go test -v -tags=e2e -count=1 -run Test_PATH_E2E); \
fi

.PHONY: make_load_test_portal_prod_path
make_load_test_portal_prod_path: ## Run a Shannon load test against production PATH with Portal credentials
@echo "📋 Copying archival config..."
@cp e2e/config/.grove.archival.e2e_load_test.config.yaml e2e/config/.e2e_load_test.config.yaml
@echo "✅ Config copied successfully"
@echo ""
@echo "🔍 Checking environment variables..."
@if [ -z "$(PORTAL_APPLICATION_ID)" ]; then \
echo "❌ Error: PORTAL_APPLICATION_ID environment variable is not set"; \
echo " 🔐 Grove employees: Get credentials from https://share.1password.com/s#x4x-cWnkf4GUw50BwV166aWEEevZV1nmaO2RxJssWjg"; \
echo " 💡 Set with: export PORTAL_APPLICATION_ID=your_app_id"; \
exit 1; \
fi
@if [ -z "$(PORTAL_API_KEY)" ]; then \
echo "❌ Error: PORTAL_API_KEY environment variable is not set"; \
echo " 🔐 Grove employees: Get credentials from https://share.1password.com/s#x4x-cWnkf4GUw50BwV166aWEEevZV1nmaO2RxJssWjg"; \
echo " 💡 Set with: export PORTAL_API_KEY=your_api_key"; \
exit 1; \
fi
@echo "✅ Environment variables are set"
@echo ""
@echo "⚙️ Configuring for PRODUCTION PATH..."
@if [[ "$$(uname)" == "Darwin" ]]; then \
sed -i '' 's/portal_application_id: ""/portal_application_id: "$(PORTAL_APPLICATION_ID)"/; s/portal_api_key: ""/portal_api_key: "$(PORTAL_API_KEY)"/' e2e/config/.e2e_load_test.config.yaml; \
else \
sed -i 's/portal_application_id: ""/portal_application_id: "$(PORTAL_APPLICATION_ID)"/; s/portal_api_key: ""/portal_api_key: "$(PORTAL_API_KEY)"/' e2e/config/.e2e_load_test.config.yaml; \
fi
@echo "✅ Config updated for production PATH"
@echo ""
@if [ "$(filter-out $@,$(MAKECMDGOALS))" = "" ]; then \
echo "🚀 Running load test with default service IDs: bsc,eth,pocket,poly,gnosis,xrpl_evm_test"; \
(cd e2e && TEST_MODE=load TEST_PROTOCOL=shannon TEST_SERVICE_IDS=bsc,eth,pocket,poly,gnosis,xrpl_evm_test go test -v -tags=e2e -count=1 -run Test_PATH_E2E); \
else \
echo "🚀 Running load test with service IDs: $(filter-out $@,$(MAKECMDGOALS))"; \
(cd e2e && TEST_MODE=load TEST_PROTOCOL=shannon TEST_SERVICE_IDS=$(filter-out $@,$(MAKECMDGOALS)) go test -v -tags=e2e -count=1 -run Test_PATH_E2E); \
fi

# Allow service IDs to be passed as arguments
%:
@:

##################
### Morse Tests ###
##################

# Targets are also provided to run a morse load test, which use the `morse_load_test` targets

.PHONY: morse_load_test_all
Expand Down