HPO: keep the original logic if there's only one process, pass the tr…

[sywangyi]

…ial to trainer

 need to find out solution for following cases
 *if we need to use trial in model_init, how to do it for non-main rank, sync the model with rank0 in app?
 *how to use optuna prune feature for DDP, if we do it in rank0, how does other rank know it.

Signed-off-by: Wang, Yi A yi.a.wang@intel.com

What does this PR do?

Fixes # (issue)

Before submitting

• This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
• Did you read the contributor guideline,
  Pull Request section?
• Was this discussed/approved via a Github issue or the forum? Please add a link
  to it if that's the case.
• Did you make sure to update the documentation with your changes? Here are the
  documentation guidelines, and
  here are tips on formatting docstrings.
• Did you write any new necessary tests?

Who can review?

• trainer: @sgugger

[sywangyi]

@yao-matrix @sgugger please review the patch.

[HuggingFaceDocBuilderDev]

The documentation is not available anymore as the PR was closed or merged.

[sywangyi]

need to find out solution for following cases
*if we need to use trial in model_init, how to do it for non-main rank, sync the model with rank0 in app?
*how to use optuna prune feature for DDP, if we do it in rank0, how does other rank know it.

[sgugger]

Good catch!