projection

Command

1. Simpily run with run_projection.py and the results will be saved to folder "data".

   python run_projection.py

2. Run with jupyter notebook: run_projection.ipynb for simpily visualization.

Example

Compute

Llama2-7B

• Overall Projection (more details in table)

• Attn Projection Prefill | Decode

• FFN Projection Prefill | Decode

Llama2-13B

• Overall Projection (more details in table)

• Attn Projection Prefill | Decode

• FFN Projection Prefill | Decode

Qwen-7B

• Overall Projection (more details in table)

• Attn Projection Prefill | Decode

• FFN Projection Prefill | Decode

Qwen-14B

• Overall Projection (more details in table)

• Attn Projection Prefill | Decode

• FFN Projection Prefill | Decode

Mixtral-8x7B

• Overall Projection (more details in table)

• Attn Projection Prefill | Decode

• FFN Projection Prefill | Decode

Todo

1. Currently only cover single card perf projection, will support multi-card / multi-node.
2. Only cover Llama2-7B, Qwen-7B and Mixtral-8x7B, will cover more models.