Add maximum variance ("maxvar") acquisition functions for active learning

[matthewcarbone]

🚀 Feature Request

Currently botorch implements the UpperConfidenceBound analytic and MC acquisition functions in e.g. botorch.acquisition.UpperConfidenceBound. This is useful but it might be nice to have a pure "active learning" acquisition function instead of just setting a large beta value.

Motivation

It's true this isn't really "true Bayesian optimization", but I think there is a community that will find this useful. It kinda feels like botorch should have this.

Pitch

Describe the solution you'd like

Implementing a "hacked version" of UCB which does this.

Describe alternatives you've considered

You can simply set the control parameter beta=1e10 or something large to drown out the contribution from the mean, but this feels pretty sloppy to me.

Are you willing to open a pull request?

Yes absolutely! I can do this for both analytic and MC. Unfortunately no.

Additional context

N/A

[saitcakmak]

Hi @x94carbone. I am not that familiar with active learning applications, so I'll let someone else with more context address this. I just wanted to point out that we do support qNegIntegratedPosteriorVariance as a pure-exploration acquisition function.

[Balandat]

It's true this isn't really "true Bayesian optimization", but I think there is a community that will find this useful. It kinda feels like botorch should have this.

Agreed, I think the active learning aspect is interesting and important and we're happy to accept suggestions - and even happier to accept PRs :) These would naturally go into the active_learning submodule.

Just so I understand, this would basically just return the posterior variance and then optimizing the acquisition function would mean finding the point with highest posterior variance?

[matthewcarbone]

@saitcakmak Ah I see. Is this in principle different than if we set the UCB acquisition function beta to some large number? I do see it comes from botorch.acquisition.active_learning, which I've played around with some time ago, but I don't recall it working as smoothly out of the box as just "hacking" UCB. Perhaps I should revisit though, I don't want to duplicate your efforts.

@Balandat That is correct!

Edit: actually at the very least I do think this will add something: an analytic version of a "pure exploration" acquisition function.

[saitcakmak]

Is this in principle different than if we set the UCB acquisition function beta to some large number?

I think it is closer in spirit to a look-ahead type acquisition function. Rather than greedily picking the point with the maximum variance, it tries to maximize the reduction in the total (integrated) variance in the search space from evaluating a given point. In that sense, it explicitly considers the effects beyond the candidate itself, which is different than what UCB does.

[matthewcarbone]

I think it is closer in spirit to a look-ahead type acquisition function. Rather than greedily picking the point with the maximum variance, it tries to maximize the reduction in the total (integrated) variance in the search space from evaluating a given point. In that sense, it explicitly considers the effects beyond the candidate itself, which is different than what UCB does.

This does sound different then. Perhaps MaxVar/qMaxVar is simpler, but it does sound like a new feature.

What I can say is that most have heard of MaxVar. I think qNegIntegratedPosteriorVariance is a bit more complex, at least for what I know some of my colleagues use/need.

What do you think? Would you like me to open a PR for this?

[Balandat]

What do you think? Would you like me to open a PR for this?

I think MaxVar is a bit of a misnomer, since the acquisition function is really just posterior variance at the query point(s). So it should be (q)PosteriorVariance - the "max" is just b/c we actually optimize it (that's why we call things PosteriorMean rather than MaxPosteriorMean - semantics). We could add this to the active_learning module if you think having this would be helpful for folks.

[matthewcarbone]

I think MaxVar is a bit of a misnomer, since the acquisition function is really just posterior variance at the query point(s). So it should be (q)PosteriorVariance - the "max" is just b/c we actually optimize it (that's why we call things PosteriorMean rather than MaxPosteriorMean - semantics). We could add this to the active_learning module if you think having this would be helpful for folks.

Sure fair enough, that's consistent with the other botorch methods. I'll implement and open a PR a bit later. 👍

[eytan]

One could add an acquisition function that just maximizes posterior variance. This is often used as a baseline in active learning papers and generally performs the worst. I would highly recommend not using this acquisition function for any actual active learning tasks. Integrated mean squared error (which is pretty much the same as integrated posterior variance) is commonly used in active learning within the simulation community.

…

On Wed, Aug 24, 2022 at 2:34 PM Matthew Carbone ***@***.***> wrote: I think MaxVar is a bit of a misnomer, since the acquisition function is really just posterior variance at the query point(s). So it should be (q)PosteriorVariance - the "max" is just b/c we actually optimize it (that's why we call things PosteriorMean rather than MaxPosteriorMean - semantics). We could add this to the active_learning module if you think having this would be helpful for folks. Sure fair enough, that's consistent with the other botorch methods. I'll implement and open a PR a bit later. 👍 — Reply to this email directly, view it on GitHub <#1366 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAW34J3YZPUW6LPJ2WVP7TV2ZTKXANCNFSM57QEDASA> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[matthewcarbone]

@eytan sure, it's usually just a baseline, but it's an important one, and sometimes it's good enough.

[matthewcarbone]

All, I don't feel comfortable signing Meta's required form to open the PR (given my position it's not entirely obvious that I can, I'm no lawyer and I don't pretend to be). I apologize, I didn't realize there was a required "extra step" at first.

Anyways, this is more or less what I had in mind:

class PosteriorVariance(AnalyticAcquisitionFunction):
    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)

    @t_batch_mode_transform(expected_q=1)
    def forward(self, X):
        posterior = self.model.posterior(
            X=X,
            posterior_transform=self.posterior_transform
        )
        mean = posterior.mean
        variance = posterior.variance
        view_shape = (
            mean.shape[:-2] if mean.shape[-2] == 1 else mean.shape[:-1]
        )
        return variance.view(view_shape)

class qPosteriorVariance(MCAcquisitionFunction):
    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)

    @concatenate_pending_points
    @t_batch_mode_transform()
    def forward(self, X):
        posterior = self.model.posterior(
            X=X,
            posterior_transform=self.posterior_transform
        )
        objective = self.objective(self.sampler(posterior), X=X)
        mu = objective.mean(dim=0)
        samples = (objective - mu).abs()
        return samples.max(dim=-1)[0].mean(dim=0)

Please feel free to implement something like the following classes if you feel the meet the bar (or to edit them in ways you feel appropriate). They're largely just built on top of the UCB acquisition functions in your code. You should be able to add this to botorch.acquisition.active_learning with no hiccups.

[takafusui]

Hi, does anybody have a sample code to do active learning using BoTorch so that we increase the global model accuracy? I want to focus on the pure-exploration part, and I am also reluctant to set a large beta in the UpperConfidenceBound analytic acquisition function. I realized the qNegIntegratedPosteriorVariance acquisition function, so I highly welcome something with the qNegIntegratedPosteriorVariance acquisition function as well. Thank you in advance.

[matthewcarbone]

@takafusui incidentally, this

I am also reluctant to set a large beta in the UpperConfidenceBound analytic acquisition function

Does actually work just fine. I usually use beta=100 or so for "active learning". For properly scaled inputs/outputs this seems to be sufficient.

[takafusui]

Hi @matthewcarbone, thank you for your comment. What did you mean 'properly scaled inputs/outputs'? For instance, I usually standardize inputs when fitting a GP model, then scale back outputs when computing posterior mean and variance.