Evaluation for EarlyTerminated update

[Yabbanootje]

I think I found a small bug. I noticed a difference when evaluating my PPOEarlyTerminated agent and when rendering it, namely that in only the evaluation the agent is stopped when it goes above the cost limit. When looking at the code in evaluator.py I saw that in the render() function the line:
done = bool(terminated or truncated)
comes before the lines:
if ( 'EarlyTerminated' in self._cfgs['algo'] and ep_cost >= self._cfgs.algo_cfgs.cost_limit ): terminated = torch.as_tensor(True)
For the evaluate() function the order is switched around, such that it does stop the episode sooner. I would personally think that during evaluation and rendering the EarlyTerminated models should not be stopped. But in any case I do not think that it was intended that the two versions would differ in this regard. Hope this helps.

[Gaiejj]

Thanks for your mentioning! I think it is absolutely right. Frankly speaking, the current implementation of the evaluator is somewhat redundant and complicated. In future versions, we will bind the evaluation logic to the algorithm, allowing the evaluator to simply call the corresponding module instead of redefining the logic. Thank you again for pointing this out.