[trainer] fractional epoch

[stas00]

Running

export BS=20; rm -r output_dir; CUDA_VISIBLE_DEVICES=0,1 PYTHONPATH=../../src USE_TF=0   ./finetune_trainer.py --model_name_or_path t5-small --output_dir output_dir --adam_eps 1e-06 --data_dir wmt_en_ro --do_eval --do_predict --do_train --evaluation_strategy=steps --freeze_embeds --label_smoothing 0.1 --learning_rate 3e-5 --logging_first_step --logging_steps 1000 --max_source_length 128 --max_target_length 128 --num_train_epochs 1 --overwrite_output_dir --per_device_eval_batch_size $BS --per_device_train_batch_size $BS --predict_with_generate --eval_steps 25000  --sortish_sampler --task translation_en_to_ro --test_max_target_length 128 --val_max_target_length 128 --warmup_steps 5 --n_train 100 --n_val 100 --n_test 100  --fp16 --save_steps 1

on master, gives:

{'loss': 14.846837043762207, 'learning_rate': 6e-06, 'epoch': 0.3333333333333333} 

epoch can't be fractional.

[stas00]

The fraction (and float) 'epoch': 0.3333333333333333 comes from here:

transformers/src/transformers/trainer.py

Line 899 in 1c19b42

self.state.epoch = epoch + (step + 1) / steps_in_epoch

@sgugger - is this by design

or should it be ceil: 'epoch': 1

                    self.state.epoch = math.ceil( epoch + (step + 1) / steps_in_epoch)

or floor: 'epoch': 0

                    self.state.epoch = math.floor( epoch + (step + 1) / steps_in_epoch) 

'epoch': 0.3333333333333333 is telling me it's somewhere in the first epoch but isn't done yet?

Perhaps it's just fine, it's just very odd to see epoch not being an int.

Thanks.

[sgugger]

This is not a bug, it's completely normal. See the dcoumentation of TrainerState.epoch.

[stas00]

Ah, OK. Some rounding then perhaps - 0.3333333333333333 is just too loud. 2 decimals?

[sgugger]

Sure, there is no formatting at all for those results, but we can add some.

[stas00]

Anything else to format while I'm at it? loss I guess - 4 decimals, right?

{'loss': 14.846837043762207, 'learning_rate': 6e-06, 'epoch': 0.3333333333333333}

[sgugger]

I don't see why not.

[vkaul11]

num_train_epochs=1 and 'epoch': 0.3333333333333333 can that happen ? In my case also the epochs stop at 0.57 and I don't get any print later?