[Feature Request] Multitask prediction from prior

[AdrianSosic]

🚀 Feature Request

The current API of MultiTaskGP could be broadened to also allow making predictions for a completely new task (i.e. one for which there is not a single observation available yet). Currently, this is not possible because the tasks to be modeled are extracted from the training data set, which does not yet contain the index of the new task.

Motivation

Is your feature request related to a problem? Please describe.

A common situation in experimentation is that you have conducted some experiments on tasks X, Y and then would like to optimize a new task Z. While at the transition point, you have no data for Z available and hence it is impossible to estimate any correlation (say, posterior estimates for an IndexKernel), it is still useful to consider data gathered for X and Y because it can unveil general trends that can help optimizing Z.

Hence, having the option to add an "out-of-data" task would be useful. Of course, one single additional task would suffice here because the predictions for all unseen tasks would be identical.

Pitch

Describe the solution you'd like

The extension can be easily implemented by extending the function signature, allowing to modify the content of all_tasks in a reasonable way. There are several possibilities one can consider, but given the fact that the existing tasks are provided in form of arbitrary integers, it seems most natural to me to provide the index of the "blind" task in one form or the other. Open for discussion 🙃

Are you willing to open a pull request? (See CONTRIBUTING)
I'm more than happy to open a PR that adds the feature, provided we align on the desired interface 👍🏼

[saitcakmak]

Hi @AdrianSosic. I've been experimenting with a similar idea, by allowing a custom all_tasks input as you suggested. The mechanics of training & prediction work rather smoothly. However, since there is no data for this new task, its task covariance parameters do not get trained from their randomly initialized values, while the parameters for other tasks do. This makes the correlation between this task and others rather arbitrary (50-50 positive or negative), unless we impose some strong priors on the task kernel to keep things positively correlated. I've tried this with both MultiTaskGP and LCEMGP (which defines the task kernel using explicit embeddings rather than using IndexKernel, and ends up with positive correlation by default). I can expose my local changes in a PR for you

[AdrianSosic]

Hi @saitcakmak, great to hear 😃 I guess we can then join forces in conceptualizing/implementing the feature 👍🏼

Thanks also for your summary. However, I'd like to go two steps back and first make sure we are on the same page. I haven't yet looked too deep into what is really happening behind the scenes, so I can only speculate what I would expect to happen based on the modeling assumptions:

You are saying that the values of the new task do not get trained. I guess the mechanism that applies in the standard setting is ML/MAP inference, depending on whether a prior is registered? For ML, I would agree, and likewise for any flat prior. But if we use, for example, a (non-flat) LKJ prior, shouldn't the correlation values of the new task get updated? Because they would be pulled both by the prior and indirectly by the evidence for the observed task through the prior? Haven't verified this in the code, though. Perhaps you can share your insights?

And then second, you seem to indicate that you consider it a problem that the correlation values can get negative. But I'm not sure if I can follow here. If some of the tasks ARE negatively correlated, don't see a reason why this would become problematic. Can you elaborate?

[saitcakmak]

Hi @AdrianSosic. I just exported the changes in #2271.

I haven't got a chance to play around with the priors yet, but I agree with your point. A carefully designed prior should help in achieving the desired task correlation. My intended use case for this is in initializing new experiments using data from recently completed experiments -- a form of transfer learning. In that setting, I do want positive correlation but I also don't want perfect correlation -- so that there's more uncertainty in the model than a single-task model would have.

I'll try to experiment with this idea in between other things. Please let me know if you find some priors that work well!

[AdrianSosic]

Hi @saitcakmak, I think then we both agree upon the underlying mechanism we envision 👍🏼

Still, there is one subtlety I would like to discuss. The "transfer learning" goal you have in mind is exactly the same for which I need the new feature. However, I would disagree with the statement that positive correlation is always needed for that setting. As already mentioned in my previous answer, I think the general case may contain both negatively and positively correlated source tasks, and in such a case, a prior supporting both negative and positive correlations seems natural. Restricting to positive correlations seems like a strong context-specific assumption and I would not consider it part of the PR. In fact, the PR itself wouldn't even need to bother about the priors at all but just needs enable passing unseen task indices.

Hence, to me, the changes you did already cover most of what is necessary. I think the only open questions are:

• if this is the interface we want to keep.
• if everything works for all possible inputs, i.e. when the training data contains indices that are not contained in the given list AND/OR the given list contains indices that are not in the data.

What do you think?

[AdrianSosic]

Hi @saitcakmak, it seems that you have continued to work on PR in the meantime? Are there any updates from your end? Just wondering what is the fastest way to proceed here...

[saitcakmak]

Hi @AdrianSosic. Sorry, I forgot to respond to your previous message. I did some clean up on the PR & fixed some test failures. I'll ping the team to get a review & get it landed.

Re positive vs negative correlation: I agree that if you have multiple source tasks to transfer from, some of those may be negatively correlated. I was initially trying this with a single tasks from a synthetic problem. If I have a single source I want to transfer from, it makes sense to me to expect positive correlation (since I have a prior assumption that the source task is correlated with the new task I aim to optimize). I played around with the LKJ prior a bunch and I couldn't get it to enforce a positive correlation. Regardless of the prior parameters I used, I could get both positive & negative correlation by simply re-fitting the model on the same data.

Getting back to the PR itself: The PR simply allows creating a task kernel that supports more tasks than what is in the data. In that sense, it does not have any opinions on the particular covariance structure, so the user can modify the covariance to fit their needs.

if this is the interface we want to keep.

Given the flexibility it adds with limited code change, I think we can keep it as long as it serves the use case.

if everything works for all possible inputs, i.e. when the training data contains indices that are not contained in the given list AND/OR the given list contains indices that are not in the data.

This setup wouldn't work if the training data contains indices that are not in the given list. The IndexKernel would error out on evaluation if the index is out of bounds. So, this setup will only support creating a more permissive kernel that supports more indices than you have in the data.

[Balandat]

Can we consider this issue closed with the PR having gone in?