Handle cases where no males/females exist in a *virtual* subpopulation?

[ashander]

I'm attempting to deal with virtual subpopulations where there are no male or females using an approach similar to your suggestion in #15 and running into problems (related to weighting; see also #12 ).

Essentially, I can control the next generation size (as here with demoModel) and even the offspring number and sequencing (although here that is static) but when multiple virtual subpopulations exist in the same physical subpop, I'm running into issues due to weighting --- the virtual subpopulation with no females still attempts to produce offspring because weights are assigned based on current size (see debug output below) and so assigns one offspring to each VSP (VSP sizes in subpop 0 is 1, 1).

Is there some weighting scheme (or other strategy) that will work to assign zero offspring to VSPs with no males/females?

import simuPOP as sim
numoff = 1  # Global used in in demoModel and evolve

def demoModel(pop):
    sim.stat(pop, numOfMales=True, vars=['numOfMales_sp', 'numOfFemales_sp'])
    m_f = [(pop.dvars(i).numOfMales, pop.dvars(i).numOfFemales) for i in range(pop.numSubPop())]
    return [f * numoff if m > 0 else 0 for m,f in m_f]

sim.turnOnDebug('DBG_ALL')
sim.setOptions(seed=112)
initial_size = 7
p = sim.Population(initial_size, infoFields=['mark'])
sim.initInfo(p, [1, 1, 1, 2, 2, 2, 2], infoFields='mark')
sim.initSex(p, sex=[sim.MALE, sim.FEMALE, sim.FEMALE, sim.MALE, sim.MALE, sim.MALE, sim.MALE, ])
p.setVirtualSplitter(sim.InfoSplitter('mark', ranges=((1, 2), (2, 3))))
p.evolve(
    preOps=sim.Dumper(structure = False),
    matingScheme=sim.HeteroMating(
        [sim.HomoMating(
            chooser = sim.CombinedParentsChooser(
                sim.RandomParentChooser(replacement=True, sexChoice=sim.MALE_ONLY),
                sim.SequentialParentChooser(sexChoice=sim.FEMALE_ONLY),
            ),
            generator = sim.OffspringGenerator( ops = [
                sim.MendelianGenoTransmitter(),
            ],
                numOffspring=numoff),
            subPops=[(sim.ALL_AVAIL, 0), (sim.ALL_AVAIL, 1)],
            weight=1,
        )],
        subPopSize=demoModel,
    ),
    postOps=sim.Dumper(structure = False),
    gen=1,
)

Output

::::::::::::::::::::::::::::::::--
Constructor of population is called
Inc ref of 2 to 9
individual size is 24+1*0
, infoPtr: 8, GenoPtr: 8, Flag: 1, plus genoStru
genoSize 0
Analyzed string is >
Filename is 
Analyzed string is >
Filename is 
Creating Simulator 
Constructor of population is called
Inc ref of 1 to 6
individual size is 24+1*0
, infoPtr: 8, GenoPtr: 8, Flag: 1, plus genoStru
genoSize 0
Constructor of population is called
Inc ref of 1 to 7
individual size is 24+1*0
, infoPtr: 8, GenoPtr: 8, Flag: 1, plus genoStru
genoSize 0
Simulator created
Start evolution.: 2e-06
SubPopulation 0 (), 7 Individuals:
   0: MU  |  |  1
   1: FU  |  |  1
   2: FU  |  |  1
   3: MU  |  |  2
   4: MU  |  |  2
   5: MU  |  |  2
   6: MU  |  |  2

Applied <simuPOP.Dumper>: 0.000186
Start mating at generation 0: 1.3e-05
Dec ref of 1 to 6
Inc ref of 2 to 10
New subpop size 2
Positive mating scheme weights: 1, 1
Negative mating scheme weights: 0, 0
VSP sizes in subpop 0 is 1, 1
Mating is done in single-thread mode
Mating scheme 1 in subpopulation 0 failed to produce 1 offspring.

[BoPeng]

It seems that I have somehow missed this ticket. Let me know if you still need help. Basically, the weight design does not consider proportion of males and females within VSPs so the only way to avoid this is to check VSPs subpopulation before mating.

[ashander]

Thanks! I should've replied that we solved it another way (with a custom parent chooser that does essentially what you're suggesting).

But, it does seem like a somewhat fundamental issue that many folks may run into. Are there large obstacles to a low-level patch that would correct the weight design for cases like my example above? If you don't see any, I could try to contribute one =)

[BoPeng]

There are a few problems that make it difficult to produce a general solution:

1. Even if VSP does not have any male or female, mating can still happen if a non-sexual mating scheme is used, or if a conditional mating scheme is used to perform selfing in such cases to mimic the behavior of some plant populations. Basically, such cases can be intentional.
2. We cannot delay the problem to the mating stage because a pre-specified number of offspring (calculated from weight) has to be generated to populate the offspring subpopulation.

So an option is needed here and it has to determine how the size of offspring populations is determined. For example,

1. weight_by=POPSIZE: current behavior
2. BY_MALE: only count male individuals in the parental subpopulation
3. BY_FEMALE: only count female individuals in the parental subpopulation
4. BY_PAIR: determine the number of "mating" parents by number of pairs, namely min(male, female).

The last option would treat SP (or VSP) with no male or no female as having zero mating parents and produce zero offspring. However, it can be confusing to use BY_MALE with weight=-1 because weight=-1 was used to generate the same number of offspring as parents, and now it would be the same number of offspring as fathers (so weight=-2 makes more sense).

[BoPeng]

OK, this is what has been implemented:

As a special case that can be quite annoying during the simulation of small populations, a (virtual) subpopulation can have no male and/or female. If the parental (virtual) subpopulation is empty, it will produce no offspring regardless of its weight. However, if the parental (virtual) subpopulation is not empty, it will be expected to produce some offspring, which is not possible if a sexual mating scheme is used. In this case, you can use a parameter weightBy to specify how parental (virtual) population sizes are calculated. This parameter accepts values ANY_SEX (default), MALE_ONLY, FEMALE_ONLY, PAIR_ONLY, and use all individuals, number of male individuals, number of female individuals, and number of male/female pairs (basically the less of numbers of males and females) as the size of parental (virtual) subpopulation, respectively. When weightBy=PAIR_ONLY is used, parental (virtual) subpopulations with only males or females will appear to be empty and produce no offspring. Note that in this mode (also MALE_ONLY and FEMALE_ONLY), the perceived parental population sizes are no longer the actual parental population sizes so you might need to adjust parameter weight (e.g. weight=-2) to produce correct number of offspring.

[rougerbaptiste]

Hello,
I feel like the parameter weightBy is intended to be optional, though I cannot manage to make my code work without including it (i.e. as it was with the 1.1.8.3 version of simuPOP). Is it meant to be mandatory ?
Thank you for your answer.

[BoPeng]

Yes, weightBy is meant to be optional and has a default value ANYSEX (f5a22bb#diff-b43b9f8d4f6b1e382f8e2a7c160c1921R1730). Could you provide an example where weightBy is required? Note that the latest version of simuPOP is 1.1.9 but this should not affect this particular feature.

[BoPeng]

At least the basic HeteroMating example https://github.com/BoPeng/simuPOP/blob/master/docs/HaplodiploidMating.py works without this option.

[rougerbaptiste]

Indeed, this example works also on my setup.
This is how the mating scheme is defined in my code :

mating = simu.HeteroMating([
            simu.SelfMating(ops=[
                simu.Recombinator(rates=self.metaPop.calcRecombinaison(), 
                             loci=range(len(self.metaPop.getMarkers())-1))],
                 weight=self.metaPop.getSelfing()),
            simu.HermaphroditicMating(ops=[
                   simu.Recombinator(rates=self.metaPop.calcRecombinaison(), 
                       loci=range(len(self.metaPop.getMarkers())-1))], 
                weight=1-self.metaPop.getSelfing())
              ], 
         subPopSize=self.reproduction)

I don't know how to adapt it to this example... And not sure this is actually relevant to this thread.
I hope everything is quite clear with this code. Let me know if I should further describe it.
The error I get (and which I had not with version 1.1.8.3) is : ValueError: mating.cpp: 1703 Overall weight is zero

[BoPeng]

Could you run

turnOnDebug('DBG_DEVEL')

before the evolve function and report the output of this debug message?

So, did you say the problem would disappear if you added weightBy somewhere to your example? How did you add it?