Search results

Capture tt-metal and tt-NN loguru logs in inference server python log files

enhancementNew feature or requestNew feature or request

Status: Open.

#13 In tenstorrent/tt-inference-server;

· tstescoTT opened on Sep 25, 2024

Add status messaging and endpoint to allow for client-side users to reason about model initialization and life cycle.

enhancementNew feature or requestNew feature or request

Status: Open.

#17 In tenstorrent/tt-inference-server;

· tstescoTT opened on Sep 26, 2024

Provide example chat template usage

documentationImprovements or additions to documentationImprovements or additions to documentation

enhancementNew feature or requestNew feature or request

Status: Open.

#36 In tenstorrent/tt-inference-server;

· tstescoTT opened on Nov 15, 2024

MESH_DEVICE management for Llama 3.x implementations

enhancementNew feature or requestNew feature or request

Status: Open.

#73 In tenstorrent/tt-inference-server;

· tstescoTT opened on Jan 21, 2025

Verify device topology automatically

enhancementNew feature or requestNew feature or request

Status: Open.

#74 In tenstorrent/tt-inference-server;

· tstescoTT opened on Jan 21, 2025

Add evals support for 3.x meta Llama eval datasets in lm-evaluation-harness

enhancementNew feature or requestNew feature or request

Status: Open.

#81 In tenstorrent/tt-inference-server;

· tstescoTT opened on Jan 28, 2025

pass vLLM KWArgs to run script

enhancementNew feature or requestNew feature or request

Status: Open.

#105 In tenstorrent/tt-inference-server;

· tstescoTT opened on Feb 19, 2025

run.sh script to codify the docker run commands and do validation and error handling

enhancementNew feature or requestNew feature or request

Status: Open.

#115 In tenstorrent/tt-inference-server;

· tstescoTT opened on Feb 24, 2025

Benchmark performance run script

enhancementNew feature or requestNew feature or request

Model readiness suite

Status: Open.

#117 In tenstorrent/tt-inference-server;

· tstescoTT opened on Feb 24, 2025

LLM Eval run script

enhancementNew feature or requestNew feature or request

Model readiness suite

Status: Open.

#120 In tenstorrent/tt-inference-server;

· tstescoTT opened on Feb 25, 2025

Model and vLLM test script

enhancementNew feature or requestNew feature or request

Model readiness suite

Status: Open.

#122 In tenstorrent/tt-inference-server;

· tstescoTT opened on Feb 25, 2025

Setup tool to run in tt-metal dev env

enhancementNew feature or requestNew feature or request

Model readiness suite

Status: Open.

#123 In tenstorrent/tt-inference-server;

· tstescoTT opened on Feb 25, 2025