EnsembleMLP and EnsembleFeedForwardNN to enable parallelization of ensemble models

[DomInvivo]

Changelogs

The goal is to enable an ensemble of MLP's that are trained simultaneously, and where the linear operations are parallelized. This could prove useful (or not) for fine-tuning since we observe large discrepancies between various runs on small datasets.

Features added

• EnsembleFeedForwardNN to enable ensembles in the main architecture or for fine-tuning, with a reduction option to pool all ensembles.
• EnsembleMLP to enable small ensemble MLPs to be used inside models, with a reduction option to pool all ensembles.
• EnsembleFCLayer to enable complex layers with norms, activations, etc. with ensembles
• EnsembleLinear, used by all the classes above, to enable computing multiple linear layers in parallel

These do not allow true Ensembles, as they are averaged together before computing the loss. In ensembles, each model should have it's own loss. This could be implemented in MetricWrapper

---

Checklist:

• Was this PR discussed in an issue? It is recommended to first discuss a new feature into a GitHub issue before opening a PR.
• Add tests to cover the fixed bug(s) or the new introduced feature(s) (if appropriate).
• Update the API documentation is a new function is added, or an existing one is deleted.
• Write concise and explanatory changelogs above.
• If possible, assign one of the following labels to the PR: feature, fix or test (or ask a maintainer to do it for you).

---

discussion related to that PR

[codecov]

Codecov Report

Merging #491 (62ab1ad) into main (639b3f8) will decrease coverage by 0.07%.
The diff coverage is 66.82%.

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #491      +/-   ##
==========================================
- Coverage   71.42%   71.35%   -0.07%     
==========================================
  Files          93       94       +1     
  Lines        8529     8718     +189     
==========================================
+ Hits         6092     6221     +129     
- Misses       2437     2497      +60     

Flag	Coverage Δ
unittests	71.35% <66.82%> (-0.07%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components	Coverage Δ
ipu	49.14% <ø> (ø)