How to extract state-fluents and non-fluents that affect a particular state-fluent's next state.

[vshlroot]

Hi authors,

First of all congratulations on building this great repo. I need a bit of help in below scenario,

Problem Description:
I need to write a function that extracts all grounded state-fluents, non-fluents, and ground actions that affect a particular state-fluent's next state. In general, such state-fluents and actions can be extracted from the DBN/cpf. But I am not sure how to extract the non-fluents.

To take an example, let's see the domain AcademicAdivisin_mdp_2014 (given at link) Here we have below cpf for a state-fluent passed'(?c):

passed'(?c) = if (takeCourse(?c) ^ ~passed(?c)) // If take a course and not already passed
then [ if (~exists_{?c2 : course} PREREQ(?c2,?c))
then Bernoulli( PRIOR_PROB_PASS_NO_PREREQ(?c) )
else Bernoulli((PRIOR_PROB_PASS(?c)) +
(1 - PRIOR_PROB_PASS(?c))*[[sum_{?c2 : course} (PREREQ(?c2,?c) ^ passed(?c2))]
/ [1 + sum_{?c2 : course} PREREQ(?c2,?c)]]) ]
else
passed(?c); // Value persists if course not taken or already passed

What is needed:
Now, I want to write a function get_sfnfac(sf) that takes in a state-fluent as input and returns all state-fluents, actions and non-fluents that can possibly affect its next state.
Considering instance 1 of this domain (given at link,

1. get_sfnfac(passed___CS11) should return:
   {passed___CS11, takeCourse___c11, PRIOR_PROB_PASS_NO_PREREQ___CS11, PRIOR_PROB_PASS___CS11}
   as it CS11 has no prereqs.
2. But for get_sfnfac(passed___CS21) should return:
   {passed___CS21, takeCourse___c21, , PRIOR_PROB_PASS_NO_PREREQ___CS21, PRIOR_PROB_PASS___CS21, PREREQ(CS11, CS21), PREREQ(CS12, CS21)}
   as CS21 has two prereqs CS11 and CS12.

What I have tried so far:
I have tried to look at files in pyRDDLGym/core/parser/ but couldn't find a direct function that I can use.

Any pointers are appreciated.

Thanks,
Vishal

[vshlroot]

I would like to mention that when using the older Java-based RDDLSim, we developed a parser that extracted state and action fluents from a DBN (graphviz file), but it did not capture non-fluents.

While generating a DBN graphviz file and parsing it to obtain state-fluent dependencies is an option, this approach is slower and overlooks non-fluents. A more efficient and complete solution would be to utilize functions from the RDDLgrounder (or a similar class) in pyRDDLGym to manage both state and non-fluents (SF and NF) during or immediately after grounding the CPF.

[mike-gimelfarb]

Sorry for the late reply. I think the most streamlined way to handle dependencies with non-fluents is to convert them to state-fluents. Specifically, change your non-fluent pvariables to state-fluent pvariables with no-op transitions, e.g. non-fluent-var' = non-fluent-var;
You can run an xadd compilation on this modified domain using the symbolic toolkit. Please feel free to submit an issue there if you have problems generating the DBN.

On the issue of performance, unfortunately I do not see how you can achieve this correctly without going through the xadd compilation. The XADD should be asymptotically the best we can currently do time complexity-wise.

[vshlroot]

Thank you for you reply.

Making non-fluents as state-fluents is a hack that I was hoping not to resort to. I was wondering if there is any systematic way to do it.

Can you kindly point me to some code/function on: how to get state-fluent to state-fluent dependencies?
I understand it can be fetched from the DBN after it is created. But by any chance do you keep an intermediate dictionary/list of state-fluents that occur on right hand side of a grounded CPF of a given state-variable?

[mike-gimelfarb]

The compiler does currently process and maintain quite a lot of information, including maintaining a (bipartite) graph of dependencies between state fluents. The only way I could currently suggest is to instantiate and use the dependency/level analysis here by passing in your grounded RDDL model object.

Specifically, you can run the build_call_graph function to get a dictionary mapping next state fluent names to their dependencies in the cpf expression. There is a working example here

However, please note that you won't get the same compact representation as you would get with a full DBN analysis so I'm not sure I would recommend this avenue (e.g. if you multiply a fluent by zero, it will still count as dependency). This approach also currently does not store non-fluent variables in the graph to save on memory, but you can rename non-fluents to state fluents to achieve what you are after. If you wish this functionality to be added into the code as an extra feature, if you absolutely cannot use the translation trick as discussed, please feel free to submit a PR (alternatively, if you wish me to look at it, please let me know, although my time to develop and test is more limited nowadays).