[Tracking issue] General dataset support

[qgallouedec]

The aim is for all trainers to apply the same procedure in their init function:

• if needed, apply the chat template, then
• if needed, tokenize.

Support todo:

Standard dataset

• BCOTrainer
• CPOTrainer
• DPOTrainer
• GKDTrainer (same as SFTTrainer)
• IterativeSFTTrainer removed
• KTOTrainer
• NashMDTrainer
• OnlineDPOTrainer
• ORPOTrainer
• PPOTrainer
• RewardTrainer [RewardTrainer] Tokenize inputs within trainer #2102
• RLOOTrainer 🔥 [Refactor] RLOOTrainer #3801
• SFTTrainer (could be previously achieved via "dataset_text_field") Default dataset_text_field to "text" #2078; 🔬 SFT simplification #2405
• XPOTrainer

Conversational dataset

• BCOTrainer BCOTrainer conversational dataset support #2107
• CPOTrainer Conversational dataset support for CPOTrainer #2144
• DPOTrainer Conversational dataset support for DPOTrainer #2131
• GKDTrainer
• IterativeSFTTrainer removed
• KTOTrainer Conversational dataset support for KTOTrainer #2248
• NashMDTrainer Conversational dataset support for Online DPO #2075
• OnlineDPOTrainer Conversational dataset support for Online DPO #2075
• ORPOTrainer Conversational dataset support for ORPOTrainer #2184
• PPOTrainer
• RewardTrainer [RewardTrainer] Tokenize inputs within trainer #2102
• RLOOTrainer 🔥 [Refactor] RLOOTrainer #3801
• SFTTrainer (yes, via get_formatting_func_from_dataset for now, needs refactoring); refactor in 🔬 SFT simplification #2405
• XPOTrainer Conversational dataset support for Online DPO #2075

Misc

• Update docs/dataset_format.mdx