vdk-meta-jobs: Meta Jobs DAG validation

[yonitoo]

Introduce a DagValidator class that properly validates the Meta Jobs DAG before it is built. It checks for:

• DAG structure (spelling mistakes, all the types of the job dict keys)
• duplicate jobs
• DAG cycles

Rework the test_meta_job tests by introducing a TestMetaJob class that contains all the existing ones as well as some new validation-related ones.

Tested-by: all the existing tests pass and introduced some new unit tests for the different cases

Signed-off-by: Yoan Salambashev ysalambashev@vmware.com

[gabrielgeorgiev1]

looks good overall, I'm approving it but I'd still appreciate responses for my comments

[antoniivanov]

In addition to checking for job structure, there are a few other validations you may want to consider on DAG level . (if you are planning to add them in separate PR , then you can ignore this comment)

1. Validate that there are not cycles
2. As the DAG is a list of jobs it's possible to have duplicate job names. This can lead to confusion and unintended behavior

You can use graphlib.TopologicalSorter for that (at least for 1.)

[yonitoo]

Added 2 new classes:

• DagValidator: the validation logic is moved there
• TestMetaJob: cleaned up the mess around test_meta_job unit tests

The only thing left to be done now is to add topological sorter validation.
@tozka 2. is handled by the check that checks for already validated jobs but will also be handled by the topo sorter (I think so)

[antoniivanov]

Functionality looks pretty good. I have mostly clean code suggestion.

[antoniivanov]

LGTM.

Please go over the message description. Make sure they are clear to someone without context . They know what the problem is and how to solve it.