KEP-2401: Determine the tag for torchtune trainer & Add support for multiple accelerators

[Electronic-Waste]

What you would like to be added?

As we discussed in #2516 (comment), we need to:

• Decide what is included in the tag of torchtune-trainer, like the version of pytorch, torchtune, CUDA, cuDNN.
• Add support for multiple accelerators in torchtune-trainer, like amd and ascend GPU, and reflect it in the image tag

/cc @kubeflow/wg-training-leads @astefanutti @deepanker13 @saileshd1402 @seanlaii

Why is this needed?

This will help users get clear information of our torchtune-trainer image.

Love this feature?

Give it a 👍 We prioritize the features with most 👍

[Electronic-Waste]

/remove-label lifecycle/needs-triage
/area llm

[Harshal292004]

/assign

[Electronic-Waste]

@Harshal292004 In fact, this issue is pending and we need to accomplish other issues first:)

[Harshal292004]

@Electronic-Waste which issues should I work on instead of this

[Harshal292004]

@Electronic-Waste I have the entire summer I can contribute rigorously

[Harshal292004]

@Electronic-Waste Any list of priority issues?

[Electronic-Waste]

@Harshal292004 I'm working on the main framework of LLM Trainer V2. And now it's blocked in: #2522. You can take other issues not relevant to KEP-2401, so that you don't need to wait:)