Search results

Capture tt-metal and tt-NN loguru logs in inference server python log files

enhancementNew feature or requestNew feature or request

Status: Open.

#13 In tenstorrent/tt-inference-server;

· tstescoTT opened on Sep 25, 2024

Add status messaging and endpoint to allow for client-side users to reason about model initialization and life cycle.

enhancementNew feature or requestNew feature or request

Status: Open.

#17 In tenstorrent/tt-inference-server;

· tstescoTT opened on Sep 26, 2024

vLLM run script prefill + decode trace pre-capture to avoid TTFT on first completions being unexpectedly high or stalling

enhancementNew feature or requestNew feature or request

Status: Open.

#56 In tenstorrent/tt-inference-server;

· tstescoTT opened on Dec 12, 2024

Provide example chat template usage

documentationImprovements or additions to documentationImprovements or additions to documentation

enhancementNew feature or requestNew feature or request

Status: Open.

#36 In tenstorrent/tt-inference-server;

· tstescoTT opened on Nov 15, 2024

Verify device topology automatically

enhancementNew feature or requestNew feature or request

Status: Open.

#74 In tenstorrent/tt-inference-server;

· tstescoTT opened on Jan 21, 2025

MESH_DEVICE management for Llama 3.x implementations

enhancementNew feature or requestNew feature or request

Status: Open.

#73 In tenstorrent/tt-inference-server;

· tstescoTT opened on Jan 21, 2025

Setup.sh check and error message for HF 401 error when gated repo (e.g. Llama 3.x repos) does not have authorization.

enhancementNew feature or requestNew feature or request

Status: Open.

#75 In tenstorrent/tt-inference-server;

· tstescoTT opened on Jan 21, 2025

setup.sh to check host for running out of disk space when setting up new model

enhancementNew feature or requestNew feature or request

Status: Open.

#76 In tenstorrent/tt-inference-server;

· tstescoTT opened on Jan 22, 2025

Add evals support for 3.x meta Llama eval datasets in lm-evaluation-harness

enhancementNew feature or requestNew feature or request

Status: Open.

#81 In tenstorrent/tt-inference-server;

· tstescoTT opened on Jan 28, 2025