Importance weighted moment matching #25
Description
https://arxiv.org/abs/1906.08850 introduced an adaptive importance sampling method called Importance Weighted Moment Matching. Given a target distribution p, it takes Monte Carlo samples θ from a distribution q, the log density function of q, and a function h whose expectation one wants to take. It then alternates between transforming the Monte Carlo samples to θ* (effectively, modifying the proposal distribution) and computing the Pareto shape diagnostic k until k ≤ 0.7. The transformations used are affine and are chosen to match the first two moments of the proposal distribution to the first 2 importance-weighted moments.
The method returns either the estimated expectation 𝔼ₚ[h(θ)] or, perhaps, h(θ*) and w(θ*), as well as the shape diagnostic. It would be handy also to return the sequence of affine transformations (or their composition) used. Like #21, this motivates some rethinking of the API.