Support warm-starting for optimization-based emulator models #306
Description
Some optimization-based emulator models (e.g., Neural Processes, SVGPs) could benefit from faster convergence when refitting to new data by retaining learned parameters instead of reinitializing. However, in some cases, reinitialization may be useful to avoid getting stuck in suboptimal local minima. This consideration does not apply to all emulator models (e.g., exact GPs have closed-form updates and do not require iterative optimization).
For models where warm-starting is beneficial/applicable, I suggest adding a warm: bool argument to the child class’s fit(x, y). This could be implemented either via fit(x, y, **kwargs) in the base class for flexibility or explicitly as fit(x, y, warm) in applicable child classes, or just as a flag attribute in the child class, keeping the fit(x, y) API.