Skip to content
This repository has been archived by the owner on Nov 17, 2023. It is now read-only.
This repository has been archived by the owner on Nov 17, 2023. It is now read-only.

C++, custom loss and network output? #18221

Open
@mureva

Description

I'm a little lost on using a custom loss function and would appreciate some help.

I have a network which is doing some regression and have been using the MAERegressionOutput with some success.
net = CreateNetwork( ... )
lossNet = MAERegressionOutput( "maeLoss", net, idealResult );

When I use MAERegressionOutput things are fairly simple, we just do

  1. bind: auto *exec = lossNet.SimpleBind(ctx, args);
  2. upload input batch data to the right arrays.
  3. forward pass: exec->Forward(true);
  4. backward pass: exec->Backward();

If I want metrics on that batch, or to visualise the result of that batch, I can just get the outputs from exec:
mx::NDArray ND = exec->outputs[0].Copy( mx::Context::cpu() );

These outputs would be the same whether I use lossNet or just net. But what about when I use a custom loss? I had understood that if I wanted to have both the output of the custom loss and the output of the network then I would need to use the Group method:

net = CreateNetwork( ... )
lossNet = Group( { MakeLoss( MyCustomLoss( net, idealData ) ), net } )

And indeed, if I do that then exec->outputs has two symbols of the sizes I would expect.
However, I'm a little unclear on how to perform the backward pass.

exec->Backward(); This just gives an error complaining about grad_heads, but
exec->Backward(exec->outputs[0]); is not syntactically correct.

What should I be giving to exec->Backward?

Metadata

Assignees

No one assigned

    Labels

    C++Related to C++

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions