[RF] Cannot generate nested RooSimultaneous from prototype category data

[elusian]

• Checked for duplicates

Describe the bug

RooSimultaneous has a feature where if the internal pdfs are themselves RooSimulteneous, the pdf is flattened in a single level RooSimulteneous with a RooSuperCategory as index.
When generating such a pdf from prototype data, the prototype needs to contain all the subcategories of the super-category, and it does so by checking the super-category servers.
However, recently RooSuperCategory was changed to contain a RooMultiCategory internally, and the only reported direct server is the internal multi-category. This leads to a wrong generation (the prototype data is ignored, the gen context refers to the current labels)

Expected behavior

Proto data works even with nested RooSimultaneous

To Reproduce

import ROOT

x = ROOT.RooRealVar('x', '', 0, 1)

c1 = ROOT.RooCategory('c1', '', {'c11': 0})
c2 = ROOT.RooCategory('c2', '', {'c21': 0})

u = ROOT.RooUniform('u', '', x)
s1 = ROOT.RooSimultaneous('s1', '', {'c11': u}, c1)
s2 = ROOT.RooSimultaneous('s2', '', {'c21': s1}, c2)

proto = ROOT.RooDataSet('proto', '', {c1, c2})
for i in range(100):
    proto.add({c1, c2})

proto.table(c1).Print("V")
proto.table(c2).Print("V")

ROOT.RooMsgService.instance().addStream(ROOT.RooFit.INFO)

# correct
context = s1.genContext(ROOT.RooArgSet(x), proto, ROOT.nullptr, True)
print(context.ClassName())
context.Print()

# not a RooSimGenContext
context = s2.genContext(ROOT.RooArgSet(x), proto, ROOT.nullptr, True)
print(context.ClassName())
context.Print()

# internal super-cat
print(s2.indexCat().GetName(), s2.indexCat().ClassName())

# the server is the internal multi-cat
print([cat.GetName() for cat in s2.indexCat().servers()])

Setup

Head of ROOT 6.26 from LCG dev4

Additional context

[guitargeek]

Hi! Thanks for reporting. Before trying to fix this, I tried to figure out what you meant by "recently" to know how much a solution would need to be backported. ROOT 6.20 was the last release where this worked. The reproducer has to be adjusted to the old PyROOT though:

import ROOT

x = ROOT.RooRealVar("x", "", 0, 1)

c1 = ROOT.RooCategory("c1", "")
c1.defineType("c11", 0)
c2 = ROOT.RooCategory("c2", "")
c2.defineType("c21", 0)

u = ROOT.RooUniform("u", "", ROOT.RooArgSet(x))

# s1 = ROOT.RooSimultaneous('s1', "", c1)
# s1.addPdf(u, "c11")
# s2 = ROOT.RooSimultaneous('s2', "", c2)
# s2.addPdf(u, "c21")

pdf_map_1 = ROOT.std.map("string,RooAbsPdf*")()
pdf_map_1.insert(ROOT.std.pair("string,RooAbsPdf*")("c11", u))
s1 = ROOT.RooSimultaneous("s1", "", pdf_map_1, c1)

pdf_map_2 = ROOT.std.map("string,RooAbsPdf*")()
pdf_map_2.insert(ROOT.std.pair("string,RooAbsPdf*")("c21", s1))
s2 = ROOT.RooSimultaneous("s2", "", pdf_map_2, c2)

categories = ROOT.RooArgSet(c1, c2)
proto = ROOT.RooDataSet("proto", "", categories)
for i in range(100):
    proto.add(categories)

proto.table(c1).Print("V")
proto.table(c2).Print("V")

ROOT.RooMsgService.instance().addStream(ROOT.RooFit.INFO)

# correct
context = s1.genContext(ROOT.RooArgSet(x), proto, ROOT.nullptr, True)
print(context.ClassName())
context.Print()

# not a RooSimGenContext
context = s2.genContext(ROOT.RooArgSet(x), proto, ROOT.nullptr, True)
print(context.ClassName())
context.Print()

# internal super-cat
print(s2.indexCat().GetName(), s2.indexCat().ClassName())

# the server is the internal multi-cat
print([cat.GetName() for cat in s2.indexCat().servers()])

[elusian]

Right, I remembered the change to be more recent and separate from the category modernization, but it seems I was mistaken and it's all in the same PR #5502 from 2020.
Sorry for the confusion

[guitargeek]

Alright, I have opened a PR to fix this, and it will be part of ROOT 6.28.

Would it also benefit you if the fix is ported pack to the next 6.26 patch release? I would like to do this only if necessary, because after all the fix reshuffels quite a bit of code.