[rllib] Observations are forced to be a float numpy array?

[deeplearningrobotics]

What is the problem?

Ray version 1.0.1 with TF 2.2

I have complex observations containing of strings but also float arrays. It seems RLlib only accepts float numpy arrays or dicts/lists of those as observations from a custom environment.

Does some documentation exist on how to get complex non-float observations to work?
Either complex observations always end up as a float tensor or RLlib throws an error because it assumes that the observations are a numpy array.

Reproduction (REQUIRED)

I will work on providing some standalone example as soon as I can. But this requires a bit more work as I need to provide a custom environment and a custom model.

[richardliaw]

cc @sven1977

[Bam4d]

@deeplearningrobotics I have been working around this by putting observations that are not tensors into the 'info' dict and then retrieving them later on in the policy by overriding compute_actions_from_input_dict. Its pretty hacky but it works 🚀

[maxsnijders]

@Bam4d if you have any details on how to get the info dict inside compute_actions_from_input_dict then that would be very helpful! There's a kwarg for info_dict, but if I breakpoint there it doesn't seem like the "info" that's returned from step() by the environment through it's last return value is present.

[Bam4d]

I think this bug was fixed in one of the latest versions. 1.6.0 perhaps