Lambda Rewards

[AntoinePrv]

Describe the problem or improvement suggested

Many simple reward functions have the following pattern

class SomeReward {

    void before_reset(...) {
        metric = 0;
    }

    double extract(...) {
        auto new_value = SCIPsomething(...);
        auto diff = new_value - metric;
        metric = new_value;
        return diff;
    }

private:
    type metric = 0;
};

Which get redundant and could be automated.

Describe the solution you would like

A LambdaDiffReward function that automates the following, where the type of metric is computed from the return type of SCIPsomething.

Describe alternatives you have considered

Interestingly, when we have SomeReward, we can get back to SCIPsomething by using reward.cumsum().
In Numpy, there is also a np.diff that does the opposite of np.cumsum, i.e. compute the finite difference.

Perhaps, we could have a LambdaReward that does not compute differences and simply return the output

class LambdaReward {

    double extract(...) { return SCIPsomething(...); }

};

As well as a reward.diff() that build the difference.

Additional context

We should be consistent on whether reward functions compute finite differences or not.
Using .cumsum() .diff() we can easily switch from one to the other.