You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
In our test_rf_... tests, we want to run the code with multiple backends (e.g. PT and RETURNN net dict (TF)), and then compare the results.
This is straightforward for all deterministic computations. However, for random, we need some special care. Even if the underlying algo is deterministic, it will be different across backends. So the only reasonable thing is that we must make sure that for such tests, random returns the same values in each run.
This is mostly straightforward, except when it comes to control flow (rf.Cond/rf.Loop, see #1282). Ignoring control flow, you could assume that in each framework, the number of calls to random and also the order, would be equal. So that in the first framework, we can cache the results from the N calls, and in the next framework, we can just return them.
With control flow, the number of calls will not be the same, esp not with eager vs graph-based. How to solve this? Maybe it anyway needs to be handled at runtime? So e.g. using some custom EvalLayer in case of RETURNN net dict, which in turn uses some custom tf.py_func or so.
The text was updated successfully, but these errors were encountered:
This is for the new RETURNN frontend (#1120).
In our
test_rf_...
tests, we want to run the code with multiple backends (e.g. PT and RETURNN net dict (TF)), and then compare the results.This is straightforward for all deterministic computations. However, for
random
, we need some special care. Even if the underlying algo is deterministic, it will be different across backends. So the only reasonable thing is that we must make sure that for such tests,random
returns the same values in each run.This is mostly straightforward, except when it comes to control flow (
rf.Cond
/rf.Loop
, see #1282). Ignoring control flow, you could assume that in each framework, the number of calls torandom
and also the order, would be equal. So that in the first framework, we can cache the results from the N calls, and in the next framework, we can just return them.With control flow, the number of calls will not be the same, esp not with eager vs graph-based. How to solve this? Maybe it anyway needs to be handled at runtime? So e.g. using some custom
EvalLayer
in case of RETURNN net dict, which in turn uses some customtf.py_func
or so.The text was updated successfully, but these errors were encountered: