Support for symlinks in generators.yaml

[mzuenni]

The new spec makes the usage of .statement, and *.download way more verbose. Therefore, it would be nice if we could specify symlinks in the generators.yaml. More specifically, it would be nice if instead of only allowing to "hardcoded" file content by extension we could also say that it is supposed to be a symlink to another file (It is sufficient to only allow symlinking to other known extensions). The question is: how should the yaml syntax for this look like?

Option 1 (tags):

- testcase:
    generate: gen.py {seed}
    in.statement: "blub"
    in.download: !!link in.statement #!!link ensures that we can distinguish this from the string "in.statement". 

Option 2 (map):

- testcase:
    generate: gen.py {seed}
    in.statement: "blub"
    in.download: {link: in.statement}

[RagnarGrootKoerkamp]

Is it an option to auto-infer all the required files and have some defaults to mostly avoid manual linking?

Although i guess we made things explicit for a reason. I like the typing idea; makes me wonder if we could use similar patterns elsewhere.

But the map is probably more consistent with the rest, so no strong opinion.

[mzuenni]

Yeah, I also think that option 1 does not fit what we have so far.

Option 3 (abuse of notation?):

- testcase:
    generate: gen.py {seed}
    in.statement: "blub"
    in.download: [in.statement] # becomes a list with a single string which we can again distinguish due to type

Any opinion on option 2 vs 3 @RagnarGrootKoerkamp @mpsijm

I like that 3 is shorter, but I think it's bad design...

[mpsijm]

I'd prefer option 2, for the reasons you mentioned 🙂

I did not follow everything of the discussion, but the spec now also requires the existence of .in.{statement,download} in certain scenarios, right? We probably want to throw errors when the generated files do/will not adhere to these rules. Because of this, I also think that making things explicit is best (which will be easier with this symlink suggestion).