memory issue in evaluation

[mohsen-ghaffari1992]

The simple maze trials are now longer which results in heap issues while evaluating. To see the results
testOnly symsim.examples.concrete.simplemaze.SarsaExperiments
If you did not see the heap issue, please try again. Because it is appearing when the learned policy has situation that timeout happens.

[wasowski]

I must have a better JVM or more RAM, because this does not run out of memory for me. I will try to see whether I can make it run longer.

[wasowski]

@mohsen-ghaffari1992 I am confused. I am looking into src/test/scala/symsim/examples/concrete/simplemaze/SarsaExperiments.scala, but this file seems to run only 5 evaluation episodes. So I cannot understand how this can run out of memory for you?

[mohsen-ghaffari1992]

@wasowski
I think you missed something. We translate all the Q-tables to policies. So, we evaluate it as long as number of episodes.
Five, is the number of times that each policy will be evaluated.

[wasowski]

A ok. This makes sense. So for 10 000 episodes you get a crash during evaluation? (I can see that learning is still run in this example).

[mohsen-ghaffari1992]

Exactly!
As I said, it is possible that you evaluate few times and it does not crush. This is because of not facing many times with policies that most stop due to timeout. If this happens (most of the time agent stocks and need to wait until timeout), then heap get full and crashes.