Home

IBS: Frequently Asked Questions

This FAQ is curated by Luigi Acerbi, and in constant expansion.

For a IBS tutorial and example, see ibs_example.m (in MATLAB); other languages to be added.

If you have questions not covered here, please feel free to ask me at luigi.acerbi@helsinki.fi (putting 'IBS' in the subject of the email).

Acknowlegments: Most of the questions currently answered here originated in a live Q&A session with the Ma lab, and thanks to Hsin-Hung Li for taking notes.

General

• Is it okay to stop the IBS algorithm for one trial after a fixed number of samples (e.g., 20)?

  No, this is not okay in the sense that by doing it one would be essentially reverting IBS to be a fixed-sampling method, with all the associated problems discussed in the paper. A more principled way is to put an early-stopping threshold on the log-likelihood, as described in the paper.

• Is it important to provide the standard deviation of the IBS estimator to the optimization/inference algorithm for every parameter combination evaluated?

  It depends:

  • If you are optimizing the target log-likelihood (e.g., for maximum-likelihood or maximum-a-posteriori estimation) then it might help but it is not necessary because the IBS estimator variance, somewhat surprisingly, is nearly constant across the parameter space. However, the BADS optimizer (which we recommend to use in combination with IBS) does not currently support user-provided, input-dependent noise; so in that case it is not even an option.
  • If you are performing Bayesian inference, for example using the VBMC toolbox, then it is necessary to provide the standard deviation of the IBS estimate to the algorithm. Bayesian inference is very sensitive to noisy estimates of the log-likelihood (or log-posterior), so it is important to provide the inference algorithm with all available information about the magnitude of the observation noise.