Search results

Got a wrong result with DeepSeek-Distill-Qwen-7b while running vllm serving with OMP_NUM_THREADS=16

Status: Open.

#491 In intel/xFasterTransformer;

· shanzhou2186 opened on Feb 18, 2025

[Feature] Support Medusa decode (Medusa: Simple Framework for Accelerating LLM Generation with Multiple Decoding Heads)

Status: Open.

#489 In intel/xFasterTransformer;

· yimuu opened on Jan 24, 2025

2 comments

Older gcc support

Status: Open.

#486 In intel/xFasterTransformer;

· Nor7th opened on Dec 5, 2024

qwen2.5 run error

Status: Open.

#482 In intel/xFasterTransformer;

· wzg-zhuo opened on Oct 30, 2024

1 comment

Qwen2.5-0.5B-Instruct quantization with gptq error

Status: Open.

#480 In intel/xFasterTransformer;

· wcollin opened on Oct 14, 2024

1 comment

oneDNN and XFT performance

Status: Open.

#471 In intel/xFasterTransformer;

· LittleNoob2333 opened on Jul 22, 2024

Can we summarize the meanings of data type like bf16_fp16?, for example, what's activation data type and output data type, what's the computing instruction?

documentationImprovements or additions to documentationImprovements or additions to documentation

Status: Open.

#414 In intel/xFasterTransformer;

· heagoo opened on May 21, 2024

1 comment

[output issue] found mistakes in llama-3-70b output by bf16_int4 during benchmark

Status: Open.

#413 In intel/xFasterTransformer;

· intelyoungway opened on May 21, 2024

1 comment

performance issue for opt-1.3b with BS=1 BF16

Status: Open.

#339 In intel/xFasterTransformer;

· bin1guo opened on Apr 23, 2024

1 comment

chatglm3 6b error

Status: Open.

#335 In intel/xFasterTransformer;

· zhm-algo opened on Apr 22, 2024

1 comment

[bug] Segmentation fault occurs at large batch sizes

Status: Open.

#140 In intel/xFasterTransformer;

· aurora327 opened on Dec 20, 2023

1 comment

run webdemo meet error when set dytpe bf16_int8 and bf16_fp16

bugSomething isn't workingSomething isn't working

Status: Open.

#59 In intel/xFasterTransformer;

· zhm-algo opened on Nov 16, 2023

2 comments

run llama2 in two CPU meet error when set dtype int8 and bf16_int8

bugSomething isn't workingSomething isn't working

Status: Open.

#56 In intel/xFasterTransformer;

· zhm-algo opened on Nov 15, 2023

1 comment