Reproducible instructions to install flash attention on all working clusters

[tjhunter]

Is your feature request related to a problem? Please describe.

We do not have a standard way to install flash attention. This is a source of bugs and an issue for onboarding new people on the project.

Task:

• write a SOP for compiling/installing flash attention on HPC2020 (ideally it should also work on the other clusters)

Describe the solution you'd like

No response

Describe alternatives you've considered

No response

Additional context

No response

Organisation

No response

[tjhunter]

These instructions should also be relevant to pin the version of pytorch:
https://docs.astral.sh/uv/guides/integration/pytorch/#installing-pytorch