Calling module.log(...) within a callback fails

[ananthsub]

🐛 Bug

Calling pl_module.log(...) within a Callback fails, even though this is recommended by the documentation here: https://pytorch-lightning.readthedocs.io/en/latest/loggers.html#logging-from-a-callback

Error

  File "my_callback_file.py", line XX, in on_validation_epoch_end
    pl_module.log_dict(my_metrics_dict)
  File "/home/local/USHERBROOKE/pain5474/opt/miniconda3/envs/cav/lib/python3.8/site-packages/pytorch_lightning/core/lightning.py", line 287, in log_dict
    self.log(
  File "/home/local/USHERBROOKE/pain5474/opt/miniconda3/envs/cav/lib/python3.8/site-packages/pytorch_lightning/core/lightning.py", line 233, in log
    self._results.log(
  File "/home/local/USHERBROOKE/pain5474/opt/miniconda3/envs/cav/lib/python3.8/site-packages/pytorch_lightning/core/step_result.py", line 171, in log
    self.__set_meta(
  File "/home/local/USHERBROOKE/pain5474/opt/miniconda3/envs/cav/lib/python3.8/site-packages/pytorch_lightning/core/step_result.py", line 217, in __set_meta
    _internal = self['meta']['_internal']
KeyError: '_internal'
python-BaseException

cc @nathanpainchaud

This is happening on master

Expected behavior

We can log from callbacks using the lightning module

Environment

Happening on PyTorch Lightning master

[SkafteNicki]

@ananthsub I just tried on master and cannot reproduce (I think it was solved yesterday, as I could reproduce 2 days ago).

[nathanpainchaud]

@SkafteNicki The issue was created by @ananthsub based on a question/issue I initially raised in the slack. I'll try to see if the bug is now resolved on master for me (it was still present as of yesterday afternoon) and I'll update you here as soon as I can.

Edit: @SkafteNicki The issue persists on my end, after having just pulled the master. I'll try to write a test today that can reliably reproduce this behavior, and I'll open a draft PR and link it here once I do.