Skip to content

Optionally use different host internal and host ip for networking in dev environments. #562

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 3 commits into
base: main
Choose a base branch
from
Open

Optionally use different host internal and host ip for networking in dev environments. #562

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
716b59d
Optionally use different host internal and host ip for networking.
dkennetzoracle Sep 4, 2025
672ca7b
FIX: Adding file back in config change.
dkennetzoracle Sep 5, 2025
16452f8
Configurable logs dir in dev config.
dkennetzoracle Sep 5, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -133,6 +133,7 @@ grafana-data
prom_data
arch_log/
arch_logs/
arch/logs/
crates/*/target/
crates/target/
build.log
Expand Down
22 changes: 17 additions & 5 deletions arch/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,10 +6,19 @@
$ rustup target add wasm32-wasip1
```

## Building
## Container Runtime Support

```sh
$ cargo build --target wasm32-wasip1 --release
# Build the gateway WASM modules (must be run from the crates directory)
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can revert this, but cargo build --target wasm32-wasip1 --release this failed on my machine, but building these packages individually worked. I can revert this, I just wanted to draw attention to it.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That is because llm_gateway and prompt_gateway binaries are loaded in envoy which only supports wasm32-wasip1 runtime. But for brightstaff there is no such limitation as runs as individual server.

$ cd ../crates
$ cargo build --target wasm32-wasip1 --release -p llm_gateway
$ cargo build --target wasm32-wasip1 --release -p prompt_gateway
```

Or build both at once:
```sh
$ cd ../crates
$ cargo build --target wasm32-wasip1 --release -p llm_gateway -p prompt_gateway
```

## Testing
Expand All @@ -25,12 +34,15 @@ $ cargo test

- Build filter binary,
```
$ cargo build --target wasm32-wasip1 --release
$ cd ../crates
$ cargo build --target wasm32-wasip1 --release -p llm_gateway -p prompt_gateway
```
- Start envoy with arch_config.yaml and test,
```
$ docker compose -f docker-compose.dev.yaml up archgw
```
- dev version of docker-compose file uses following files that are mounted inside the container. That means no docker rebuild is needed if any of these files change. Just restart the container and chagne will be picked up,
- dev version of docker-compose file uses following files that are mounted inside the container. That means no docker rebuild is needed if any of these files change. Just restart the container and change will be picked up,
- envoy.template.yaml
- intelligent_prompt_gateway.wasm
- llm_gateway.wasm
- prompt_gateway.wasm
- logs/ directory (for container logs)
9 changes: 6 additions & 3 deletions arch/docker-compose.dev.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,11 +15,14 @@ services:
- ./tools/cli/config_generator.py:/app/config_generator.py
- ../crates/target/wasm32-wasip1/release/llm_gateway.wasm:/etc/envoy/proxy-wasm-plugins/llm_gateway.wasm
- ../crates/target/wasm32-wasip1/release/prompt_gateway.wasm:/etc/envoy/proxy-wasm-plugins/prompt_gateway.wasm
- ~/archgw_logs:/var/log/
# Log volume location is now configurable via ARCHGW_LOGS_DIR (default: ~/archgw_logs)
- ${ARCHGW_LOGS_DIR:-~/archgw_logs}:/var/log/
extra_hosts:
- "host.docker.internal:host-gateway"
- "${HOST_INTERNAL:-host.docker.internal}:${HOST_IP:-host-gateway}"

environment:
- OPENAI_API_KEY=${OPENAI_API_KEY:?error}
- MISTRAL_API_KEY=${MISTRAL_API_KEY:?error}
- OTEL_TRACING_HTTP_ENDPOINT=http://host.docker.internal:4318/v1/traces
- OTEL_TRACING_HTTP_ENDPOINT=http://${HOST_INTERNAL:-host.docker.internal}:4318/v1/traces
- MODEL_SERVER_PORT=${MODEL_SERVER_PORT:-51000}
- HOST_INTERNAL=${HOST_INTERNAL:-host.docker.internal}
2 changes: 1 addition & 1 deletion arch/envoy.template.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -685,7 +685,7 @@ static_resources:
- endpoint:
address:
socket_address:
address: host.docker.internal
address: $HOST_INTERNAL
port_value: $MODEL_SERVER_PORT
hostname: {{ internal_cluster }}
{% endfor %}
Expand Down
3 changes: 2 additions & 1 deletion arch/tools/cli/docker_cli.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
import subprocess
import json
import sys
import os
import requests

from cli.consts import (
Expand Down Expand Up @@ -74,7 +75,7 @@ def docker_start_archgw_detached(
*volume_mappings_args,
*env_args,
"--add-host",
"host.docker.internal:host-gateway",
f"{os.getenv('HOST_INTERNAL', 'host.docker.internal')}:{os.getenv('HOST_IP', 'host-gateway')}",
ARCHGW_DOCKER_IMAGE,
]

Expand Down
5 changes: 3 additions & 2 deletions arch/tools/cli/main.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -105,7 +105,7 @@ def build(service):
"-t",
f"{ARCHGW_DOCKER_IMAGE}",
".",
"--add-host=host.docker.internal:host-gateway",
f"--add-host={os.getenv('HOST_INTERNAL', 'host.docker.internal')}:{os.getenv('HOST_IP', 'host-gateway')}",
],
check=True,
)
Expand Down Expand Up @@ -197,8 +197,9 @@ def up(file, path, service, foreground):

# Set the ARCH_CONFIG_FILE environment variable
env_stage = {
"OTEL_TRACING_HTTP_ENDPOINT": "http://host.docker.internal:4318/v1/traces",
"OTEL_TRACING_HTTP_ENDPOINT": f"http://{os.getenv('HOST_INTERNAL', 'host.docker.internal')}:4318/v1/traces",
"MODEL_SERVER_PORT": os.getenv("MODEL_SERVER_PORT", "51000"),
"HOST_INTERNAL": os.getenv("HOST_INTERNAL", "host.docker.internal"),
}
env = os.environ.copy()
# check if access_keys are preesnt in the config file
Expand Down
Loading