Search results

Add API authorization to YOLOv4 server

enhancementNew feature or requestNew feature or request

Status: Open.

Feature

#64 In tenstorrent/tt-inference-server;

· bgoelTT opened on Dec 24, 2024

Add health check to YOLOv4 server

enhancementNew feature or requestNew feature or request

Status: Open.

Feature

#63 In tenstorrent/tt-inference-server;

· bgoelTT opened on Dec 24, 2024

vLLM+LLama3.1-70B docker container built from scratch caused an exception in load_model

Status: Open.

#62 In tenstorrent/tt-inference-server;

· yasuhiroitoTT opened on Dec 24, 2024

vLLM run script prefill + decode trace pre-capture to avoid TTFT on first completions being unexpectedly high or stalling

enhancementNew feature or requestNew feature or request

Status: Open.

#56 In tenstorrent/tt-inference-server;

· tstescoTT opened on Dec 12, 2024

Missing --max_prompt_length argument running example_requests_client_alpaca_eval.py

Status: Open.

#51 In tenstorrent/tt-inference-server;

· milank94 opened on Dec 2, 2024

Initial vLLM setup fails due to missing HuggingFace permissions

bugSomething isn't workingSomething isn't working

Status: Open.

#37 In tenstorrent/tt-inference-server;

· milank94 opened on Nov 15, 2024

Provide example chat template usage

documentationImprovements or additions to documentationImprovements or additions to documentation

enhancementNew feature or requestNew feature or request

Status: Open.

#36 In tenstorrent/tt-inference-server;

· tstescoTT opened on Nov 15, 2024

Add tt-metal-falcon-7b from tt studio into tt-inference-server

Status: Open.

#34 In tenstorrent/tt-inference-server;

· anirudTT opened on Nov 8, 2024

Docker run support for HF_TOKEN authentication using env var pass in

enhancementNew feature or requestNew feature or request

Status: Open.

#23 In tenstorrent/tt-inference-server;

· tstescoTT opened on Oct 24, 2024

Add status messaging and endpoint to allow for client-side users to reason about model initialization and life cycle.

enhancementNew feature or requestNew feature or request

Status: Open.

#17 In tenstorrent/tt-inference-server;

· tstescoTT opened on Sep 26, 2024

llama model install script support for llama CLI and huggingface hub

enhancementNew feature or requestNew feature or request

Status: Open.

#14 In tenstorrent/tt-inference-server;

· tstescoTT opened on Sep 26, 2024

Capture tt-metal and tt-NN loguru logs in inference server python log files

enhancementNew feature or requestNew feature or request

Status: Open.

#13 In tenstorrent/tt-inference-server;

· tstescoTT opened on Sep 25, 2024