Feature Request: parallel Nelder-Mead

[itsdfish]

Nelder-Mead is often used for slow models, sometimes requiring multiple seconds per evaluation. Such cases would greatly benefit from a parallel implementation of the Nelder-Mead method. It appears that a few parallel implementations exist, which are listed below.

http://link.springer.com/article/10.1007/s10614-007-9094-2

http://link.springer.com/article/10.1007/s10614-013-9377-8

[pkofod]

Should be relatively easy to implement. I don't have time right now. Others might, or I might return to it at a later point in time. I will have to read the papers to say anything qualified on the matter :) However, I can say that benchmarking NelderMead on the large CUTEst problems is currently not possible, as it takes too long, so it would be great to improve on that!

[johnmyleswhite]

I would say that we shouldn't work on this until Julia's threading support is finished. I think it would be a mistake to do this using distributed computing by default.

[pkofod]

This is mentioned in #326 so closing this, as there is no plan here.

[structurale]

Curious if the parallel optim ever got implemented.