[KTO] learning rate recomentations for kto

[kashif]

What does this PR do?

recommendation for the KTO learning rate to be small. See: https://x.com/ethayarajh/status/1834780782126153763

[qgallouedec]

What do you think about overriding the default lr of TrainingArgument in this case? It seems to be more convenient for the user.

cc @kawine @corbt

@dataclass
class KTOConfig(TrainingArguments):
    ...
    learning_rate: float = 5e-7

[kawine]

thanks for taking the initiative @qgallouedec ! I just had one suggestion re: the docs.

[kashif]

@qgallouedec done

[qgallouedec]

Thanks, for the record, we should extend this to all our configs, when necessary