Open
Description
During training, Caffe periodically dumps the contents of output blobs, in a one-value-per-line format. This conveniently allows multiple losses or other metrics to be monitored.
A mistake in net specification may cause an internal data blob to be a disconnected top, i.e., an output blob. Caffe will happily dump billions of values to the log in this case, not stopping until disk is full.
(Yes, this happened to me today, so I am making a reminder here to fix it.)