Compatibility with OpenAI gym

[renjieli08]

Hi,

I was wondering if botorch can be used with openAI gym? Basically, my blackbox function is realized by an environment built with openAI gym, and i'd like to optimize certain output features of this environment via botorch (i.e. Bayesian Optimization).
A link to gym is here: https://github.com/openai/gym

Thanks in advance!

[Balandat]

Sure, BoTorch doesn’t care what it optimizes. Whether it is a good fit depends on how many parameters you want to optimize over and how long it takes to run one evaluation of that parameter setting - can you elaborate on that?

If you just want to use Bayesian Optimization (and don’t want to get too deep in the weeds) I recommend using Ax (ax.dev), which is a lot more accessible and easier to use than BoTorch (which is focused more on Bayesian Optimization researchers).

[renjieli08]

Hi Max,

Appreciate the reply. Actually, I was a bit inaccurate in my original post and I should clarify that the blackbox function is actually an external simulation software which is generally very expensive to evaluate (a single simulation takes about 3 mins). The simulation software, which models and simulates photonic circuits, has a built-in python API which I believe can be hooked up with botorch (correct me if I'm wrong). In fact, I have successfully tested the software's python API with pytorch, tensorflow, openAI gym and Ray Rllib.

The simulation software is a blackbox because we don't really care about what it does behind the scenes as long as it gives me the correct output values given certain input parameters.

So, I guess the key question here is whether botorch supports optimizing a photonic circuit whose objectives are evaluated by a blackbox simulation software with a built-in python API. I think the answer to the questions is YES but I am not entirely sure.

Thanks in advance.

[saitcakmak]

Hi @Arcadianlee.

I guess the key question here is whether botorch supports optimizing a photonic circuit whose objectives are evaluated by a blackbox simulation software with a built-in python API.

The answer is yes. BoTorch only requires that you can take the candidates it generates, x, and provide it with a corresponding observation, y = f(x). The same is true for Ax, which is built on BoTorch and handles many details needed to ensure a successful BO run under the hood. Unless you're interested in implementing a custom model or acquisition function, I'd recommend using Ax following this tutorial: https://ax.dev/tutorials/gpei_hartmann_service.html