Description
I'm a little lost on using a custom loss function and would appreciate some help.
I have a network which is doing some regression and have been using the MAERegressionOutput with some success.
net = CreateNetwork( ... )
lossNet = MAERegressionOutput( "maeLoss", net, idealResult );
When I use MAERegressionOutput things are fairly simple, we just do
- bind: auto *exec = lossNet.SimpleBind(ctx, args);
- upload input batch data to the right arrays.
- forward pass: exec->Forward(true);
- backward pass: exec->Backward();
If I want metrics on that batch, or to visualise the result of that batch, I can just get the outputs from exec:
mx::NDArray ND = exec->outputs[0].Copy( mx::Context::cpu() );
These outputs would be the same whether I use lossNet or just net. But what about when I use a custom loss? I had understood that if I wanted to have both the output of the custom loss and the output of the network then I would need to use the Group method:
net = CreateNetwork( ... )
lossNet = Group( { MakeLoss( MyCustomLoss( net, idealData ) ), net } )
And indeed, if I do that then exec->outputs has two symbols of the sizes I would expect.
However, I'm a little unclear on how to perform the backward pass.
exec->Backward(); This just gives an error complaining about grad_heads, but
exec->Backward(exec->outputs[0]); is not syntactically correct.
What should I be giving to exec->Backward?