Provide an option to generate 1D-folded nodesets

[kcgthb]

Hi there,

Would it make sense to provide an option to fold node lists into 1-dimension nodesets only? ND folding is awesome, but a lot of existing tools don't understand 2+ dimensions nodesets, they just expect one numerical range. And it would be useful to be able to feed them nodesets in a format they understand.

That could also be seen as a 1.6 compatibility option.

Thanks!

[degremont]

I like the idea to be able to fold on 1-D mode when you need to reuse this nodeset with another tool which does not support double folded nodeset. +1
This makes sense.

[thiell]

Yeah this makes sense that's irritating!! :)

Below are some first thoughts...

So what's the library user status around this? RangeSet and RangeSetND objects have a dim() accessor, that returns 0 for an empty RangeSet(ND), 1 for a non-empty RangeSet and >=1 for non-empty RangeSetND (always >1 when used by NodeSet).

There is no direct dim() notion at the NodeSet object level, as NodeSet can handle different nD patterns, think of one NodeSet being "login[1-2],gpu-3-[1-16]".... both dim 1 and 2.

String nodesets are parsed by the NodeSet parser that return a list of patterns kept by NodeSet, eg. "login%s", "gpu-%s-%s". Each pattern has a corresponding RangeSet or RangeSetND object with the right DIM.

[PLAN A]

One way to implement what we want is to modify the parser so that patterns become restricted to 1D like "gpu-3-%s" where only RangeSet objects are being used.

Pro's:

• That's a 1.6-like implementation.

Con's:

• It might change quite a lot of thing to the current code however, as the code has evolved a lot.
• Also, then, once parsed, the NodeSet cannot easily become again nD-enabled.

[PLAN B]

I would perhaps prefer to see something like:

• We keep current parser, initializing RangeSet and/or RangeSetND objects
• Add method NodeSet.set_max_dim(n) or NodeSet.max_dim = n

And str(nodeset) magically returns the good selected-nD folding.

Pro's:

• API rocks
• Clean implementation

Con's:

• Harder to implement ?

Let me know if you have an idea of coding PLAN C, D... Z!

[kcgthb]

I like plan B much better! It would be a shame to lose the ability to go from 1D to ND again once parsed.

[degremont]

I'm on the same line. This limitation should be applied only when casting to string. Go for Plan B.
I'm ok if you want to support "1 or ALL dim" mode only. If this helps making the patch.

You will probably have to do some expand when casting to string

[thiell]

So I've finalized a patch but not using max_dim as initially planned. I added a fold_axis variable to NodeSet (well to NodeSetBase to be precise), and the value can be any iterable over integers defining a set of axis (or dimension indices) where folding rangesets is allowed. I guess this covers all possible wanted cases, even advanced ones. Aurélien, as you said, I needed to do some expand in __str__...

I also added a --axis option to nodeset (that just passes the value to NodeSet's fold_axis), here is a simple example of use that will be pushed to rtfd:

$ nodeset --axis=1 -f node1-ib0 node2-ib0 node1-ib1 node2-ib1
node[1-2]-ib0,node[1-2]-ib1

$ nodeset --axis=2 -f node1-ib0 node2-ib0 node1-ib1 node2-ib1
node1-ib[0-1],node2-ib[0-1]

Isn't that nice ?

Patch proposal available here https://review.gerrithub.io/244809

(example edited after 1-n index change for --axis option, instead of 0 to n-1)

[thiell]

New patchset includes --axis=-n to count backward, mainly for --axis=-1 to fold last dim only (Kilian's idea).