trainer.test() with given checkpoint logs last epoch instead of checkpoint epoch

[markussteindl]

Bug description

Testing from a given checkpoint leads to logging the epoch number of the last checkpoint instead of the checkpoint specified:

trainer = Trainer(..., max_epochs=10)
lightning_module = MyLightningModule(...)
datamodule = MyDatamodule()

trainer.fit(lightning_module , datamodule=datamodule)

trainer.test(lightning_module , datamodule=datamodule, ckpt_path="last")     # <-- ok: logs correct epoch and step
ckpt_path="/.../checkpoints/epoch=2-step=396.ckpt"
trainer.test(lightning_module , datamodule=datamodule, ckpt_path=ckpt_path)  # <-- incorrect: logs last epoch and step

The second test logs epoch 10 instead of epoch 2. Similarly, the step number of the second test is incorrect.

What version are you seeing the problem on?

v2.2.1

[heth27]

I guess this could be caused by the same as #18060. The checkpoint callback is not the last callback called, and thus some loop counter are not updated. Have a look at the fields mentioned in #18060 (comment) and see if this explains the behavior you notice, it might also offer you a workaround.