Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add YoloV4 inference server #52

Merged
merged 11 commits into from
Dec 23, 2024
Prev Previous commit
Use full core grid for all wormhole devices (n150 and n300)
bgoelTT committed Dec 23, 2024
commit 5733897abef7e1e523bac48941d85b6bf355aff5
2 changes: 1 addition & 1 deletion tt-metal-yolov4/README.md
Original file line number Diff line number Diff line change
@@ -22,7 +22,7 @@ This will start the default Docker container with the entrypoint command set to
## Development
Inside the container you can then start the server with:
```bash
docker compose --env-file tt-metal-yolov4/.env.default -f tt-metal-yolov4/docker-compose.yaml run --rm inference_server /bin/bash
docker compose --env-file tt-metal-yolov4/.env.default -f tt-metal-yolov4/docker-compose.yaml run --rm --build inference_server /bin/bash
```

Inside the container, run `cd ~/app/server` to navigate to the server implementation.
5 changes: 4 additions & 1 deletion tt-metal-yolov4/server/fast_api_yolov4.py
Original file line number Diff line number Diff line change
@@ -65,9 +65,12 @@ def load_class_names(namesfile):
] == "wormhole_b0_80_arch_eth_dispatch.yaml":
print("WH_ARCH_YAML:", os.environ.get("WH_ARCH_YAML"))
device_id = 0
dispatch_core_config = ttnn.DispatchCoreConfig(
get_dispatch_core_type(), ttnn.DispatchCoreAxis.ROW
)
device = ttnn.CreateDevice(
device_id,
dispatch_core_type=get_dispatch_core_type(),
dispatch_core_config=dispatch_core_config,
l1_small_size=24576,
trace_region_size=3211264,
num_command_queues=2,
3 changes: 1 addition & 2 deletions tt-metal-yolov4/yolov4.src.Dockerfile
Original file line number Diff line number Diff line change
@@ -78,8 +78,7 @@ RUN /bin/bash -c "source ${PYTHON_ENV_DIR}/bin/activate \
&& pip3 install git+https://github.com/tenstorrent/tt-smi"

# runtime required for tt-metal on WH
# UNCOMMENT WHEN TESTING n300
# ENV WH_ARCH_YAML=wormhole_b0_80_arch_eth_dispatch.yaml
ENV WH_ARCH_YAML=wormhole_b0_80_arch_eth_dispatch.yaml

WORKDIR ${HOME_DIR}