[s2s] 1 GPU test for run_distributed_eval

[sshleifer]

Add test coverage for run_distributed_eval.py that can run on 1 GPU.
The command:

python -m torch.distributed.launch --nproc_per_node=1 run_distributed_eval.py --model_name Helsinki-NLP/opus-mt-en-ro --save_dir opus_wmt_en_ro_gens --data_dir wmt_en_ro

works on 1 GPU.

After adding test coverage, we could try to improve API consistency between run_distributed_eval.py and run_eval.py .

[sshleifer]

wdyt @stas00

[stas00]

I will work on that, thank you.

[stas00]

A minor correction to the command (corrected data_dir):

python -m torch.distributed.launch --nproc_per_node=2 run_distributed_eval.py --model_name Helsinki-NLP/opus-mt-en-ro --save_dir test_data/opus_wmt_en_ro_gens --data_dir test_data/wmt_en_ro

Question: why only 1 gpu? we currently don't have it tested at all.

[sshleifer]

I thought 1 GPU test coverage would be runnable in current CI/by more users.
But if much easier to test 2 gpu/easy to add test for 2 GPU that is great!

[stas00]

Bottom line - run with as many GPUs as available.

Thank you for clarifying.