Question about acquisition function at certain candidate point

[MJeong19]

Currently, I am coding an acquisition function for multi-objective Bayesian optimization.
I am trying to obtain the mean(mu) and standard deviation(sigma) of posterior model at certain candidate point in candidate set X.
I tried mu_i=mu(X[i]) and sigma_i=sigma(X[i]), but it didn't work.
I would like to know how can I obtain mean and the standard deviation at certain candidate point.

    def __init__(
        self,
        model: Model,
        train_Y: Tensor,
        ref_point: list[float],
        partitioning: NondominatedPartitioning,
        posterior_transform: PosteriorTransform | None = None,
        beta = 0.01,   
    ) -> None:
        
        if len(ref_point) != partitioning.num_outcomes:
            raise ValueError(
                "The length of the reference point must match the number of outcomes. "
                f"Got ref_point with {len(ref_point)} elements, but expected "
                f"{partitioning.num_outcomes}."
            )
        self.ref=ref_point
        ref_point = torch.tensor(
            ref_point,
            dtype=partitioning.pareto_Y.dtype,
            device=partitioning.pareto_Y.device,
        )
        better_than_ref = (partitioning.pareto_Y > ref_point).all(dim=1)
        if not better_than_ref.any() and partitioning.pareto_Y.shape[0] > 0:
            raise ValueError(
                "At least one pareto point must be better than the reference point."
            )
        super().__init__(model=model, posterior_transform=posterior_transform)
        self.beta=beta
        self.trainy=train_Y
        self.n_obj=partitioning.num_outcomes


def forward(self, X: Tensor) -> Tensor:
        posterior = self.model.posterior(
            X, posterior_transform=self.posterior_transform
        )
        n_obj=self.n_obj
        ref_point=self.ref
        trainy=self.trainy
        n_pts = X.shape[0]
        ucbhvi = np.zeros(n_pts)
        mu = posterior.mean #1*m tensor
        sigma = posterior.variance.clamp_min(1e-9).sqrt()
        for i in range(n_pts):
            ucbhvi_pt = np.zeros(n_obj)
            mu_i=mu(X[i])
            sigma_i=sigma(X[i])
            for j in range(n_obj):            
                ucbhvi_pt[j] = (mu_i[j] - np.sqrt(self.beta)*sigma_i[j]).numpy()

[Balandat]

The mu and sigma in this code are just pytorch tensors, the order of the elements corresponds to the inputs to the Model.posterior() call. So you just need simple indexing: mu[i] and sigma[i].

[MJeong19]

Thank you for answering my question.
I have one more question about acquisition function.
After solving previous error,
an error "Can't call numpy on Tensor that requires gradient." occured due to gradient function in mu and sigma.

So, I removed gradient function of mu and sigma by detach().
However, after this, another error occured:

I would like to know how can I solve this error.

Also, I code the acquisition function base on the code of EHVI,
and I would like to know whether mu and sigma in EHVI code have gradient functions.
Here is part of the acquisition function code I coded:

def forward(self, X: Tensor) -> Tensor:
        posterior = self.model.posterior(
            X, posterior_transform=self.posterior_transform
        )
        ref_point=self.ref
        n_obj=self.n_obj
        Trainy=self.trainy
        trainy=Trainy.numpy()
        n_pts = X.shape[0]
        ucbhvi = np.zeros(n_pts)
        mu = posterior.mean #1*m tensor
        sigma = posterior.variance.clamp_min(1e-9).sqrt()
        Mu=mu.detach()
        Sigma=sigma.detach()
        A=[-15.0, -15.0, -15.0, -15.0, -15.0, -15.0] #lower limit
        for i in range(n_pts):
            ucbhvi_pt = np.zeros(n_obj)
            Mui=Mu[i]
            mu_i=Mui.numpy()
            mu_i=mu_i[0]
            Sigmai=Sigma[i]
            sigma_i=Sigmai.numpy()
            sigma_i=sigma_i[0]
            for j in range(n_obj):
            
                ucbhvi_pt[j] = (mu_i[j] - np.sqrt(self.beta)*sigma_i[j])

            ucbhvi_pt = np.where(ucbhvi_pt > A, ucbhvi_pt, A)
        
        #check if point is outside of ref point
            if np.any(ucbhvi_pt > ref_point):
                ucbhvi[i] = 0

            else:
                #add the uhvi_pt to the list of points
                points = np.vstack((trainy,np.atleast_2d(ucbhvi_pt)))
                hv = pg.hypervolume(points)

                #calculate the exclusive contribution to the hypervolume from our point
                ucbhvi[i] = hv.exclusive(len(points)-1, ref_point)

        Ucbhvi=torch.tensor(ucbhvi, dtype=torch.float64)
        return Ucbhvi

[Balandat]

I would like to know whether mu and sigma in EHVI code have gradient functions.

Not quite sure what specific mu and sigma you mean, but generally yes if they're involved in the acquisition function computation of EHVI they do, that allows us to use gradient-based optimization for the acquisition functions.

Why are you trying to detach / numpify the Sigma in your code in the first place? It seems that you may have some more basic questions about differential programming and how auto-differentiation in pytorch works?

[MJeong19]

I was trying to calculate the hypervolume by pygmo.hypervolume.
( I tried

bd = DominatedPartitioning(ref_point=Ref_point, Y=trainy)
volume = bd.compute_hypervolume().item()

,
but it only returned the result 0)
The input of pg.hypervolume was array.
Because of that, I tried to numpify the sigma in my code.

[sdaulton]

You'll want to pass in tensors for the ref_point and Y arguments to DominatedPartitioning.__init__. This method assumes all objectives are maximized, so you may want to check that there are values of Y that dominate the reference point. Otherwise HV will be zero.