WIP: PETS algorithm from facebook/mbrl

[albheim]

PR Checklist

• Update NEWS.md?

Base implementation of PETS (see also facebookresearch/mbrl-lib) that is currently not working. I might not have a lot of time for this in the near future so I thought I could at least put it up here in case anyone was interested.

There are a few things we can discuss on how to implement the best, for example

• How to allow for checking reward and termination of states estimated by the model.
  • Currently implemented this as a custom wrapper around the env that overrides reward and is_terminated with kwargs for state/action that default to the expected values if not set. Seems reasonably clean and was the most general way I could come up with, but I'm very open to other suggestions.
• Should clamp be done in GaussianNetwork rather than separately implemented, and if so maybe we should allow setting the desired clamping method?
  • Currently I changed to allow for setting clamping method, but I'm leaning towards having the user setting it as a last layer in the network definition might be clearer?
• Do we want some overarching optimizer abstraction to allow for connecting general optimizers?
  • The facebook mbrl repo has both some general optimizer for single step I think, and then a trajectoryoptimizer for full trajectories. Seemed overkill now for the first thing, but maybe would be nice to aim for something more general similar to that?

[jeremiahpslewis]

Closing this for now, feel free to reopen at a later point in time.