-
Notifications
You must be signed in to change notification settings - Fork 1
Open
Description
To use Flash Attention in the executor, it needs to be started with Flash Attention as an additional dependency like:
[[executors]]
class = "train"
name = "diloco-transformer"
runtime = "process"
cmd = "uv"
args = [
"run",
"--python",
"3.12",
"--no-project",
"--with", "https://github.com/mjun0812/flash-attention-prebuild-wheels/releases/download/v0.7.2/flash_attn-2.8.3+cu128torch2.8-cp313-cp313-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl",
"--",
"accelerate",
"launch",
"--config-file",
"accelerate.yaml",
"-m", "hypha.accelerate_executor.training",
"--socket",
"{SOCKET_PATH}",
"--work-dir",
"{WORK_DIR}",
"--job",
"{JOB_JSON}",
]Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels