adding tensorboard loss tracking for training and evaluation

[Mogady]

I saw multiple discussions on adding loss tracking and tensor-board integration, however no pull requests have been made in that direction. 1021, 510, 336 and recently I needed that also.

I added a loss tracking module for both training and evaluation.

For evaluation:
since the code allow for custom evaluators, a LossEvaluator class is added which allow for calculating the loss and writing to TFboard logs beside the CSV file.

For training:

• I had to change in the .fit() signature but not much, just adding a parameter tensorboard_params: Dict[str, object] = {'log': True, 'steps': 1000, 'loss': True, 'lr': False, 'grad_hist': False, 'grad_stats': False} for controlling it.
• I used the steps parameter as a flag for when to log the training loss.

I understand this is not general enough, and i tried to not change the code signature that much, but hopefully it can serve as a reference or a basis for a better integration in the next versions.

[mi1ebsco]

Seems there are lots of whitespace differences that makes this PR difficult to review. Perhaps remove the whitespace changes you made and this might expedite the review process?

[Mogady]

Seems there are lots of whitespace differences that makes this PR difficult to review. Perhaps remove the whitespace changes you made and this might expedite the review process?

Yes, sorry about that, it should be more clear now

[nreimers]

Thanks for this PR.

There is a PR #1606 that adds a generic way to track loss that is also compatible with e.g. W&B. My plan is to add this generic framework first and then have classes / callbacks that can be used to track with tensorboard / W&B etc.

[Mogady]

Thanks for this PR.

There is a PR #1606 that adds a generic way to track loss that is also compatible with e.g. W&B. My plan is to add this generic framework first and then have classes / callbacks that can be used to track with tensorboard / W&B etc.

Great, W&B is definitely better and easier, I couldn't use it because of the license, but if you are giving us access to the values to design any custom dashboard then it would be perfect.