carla client doesn't wait for server to start even with set_timeout

[qhaas]

When I use set_timeout with the CARLA PythonAPI, I expect it to periodically check to see if the carla server specified port (default 2000) is receiving connections and if so, connect to it before the timeout is over and proceed. It doesn't appear to do so... if the carla server isn't up/running before the carla client is called, it appears to simply fail after the timeout... consider the following two scripts to launch a CARLA 0.9.10 server/client pair, with a slightly modified tutorial.py. One attempts to start the client immediately after the server, the other sleeps for several seconds to give the server time to spin up:

$ grep client.set_timeout PythonAPI/examples/tutorial.py 
        client.set_timeout(6.0)

$ cat testSetTimeout.sh
#!/bin/bash
./CarlaUE4.sh -opengl &
python3 PythonAPI/examples/tutorial.py

$ cat testSetTimeoutSleep.sh
#!/bin/bash
./CarlaUE4.sh -opengl &
sleep 6
python3 PythonAPI/examples/tutorial.py

The first one fails, despite giving plenty of time for the server to start. After killing the server started by the first script, the 2nd script succeeds:

$ ./testSetTimeout.sh
...
RuntimeError: time-out of 6000ms while waiting for the simulator, make sure the simulator is ready and connected to localhost:2000
...

$ ./testSetTimeoutSleep.sh
4.24.3-0+++UE4+Release-4.24 518 0
Disabling core dumps.
created vehicle.volkswagen.t2
created sensor.camera.depth
moved vehicle to Location(x=47.603519, y=-43.829613, z=0.003256)
created vehicle.mustang.mustang
created vehicle.dodge_charger.police
...

This could be related to Issue #3592 and Issue #3430

[qhaas]

Seems to still occur in CARLA 0.9.11

[qhaas]

This appears to be an issue with the underlying rpclib dependency and requires the entire client to be re-instantiated to proceed if the server isn't up/running/ready to receive connections when the client reaches out to it.

[qhaas]

We implemented a workaround where we basically just try/catch the timeout in the CARLA side of the C++ code, re-initialize the rpc client, and retry a few times. The patch file for CARLA 0.9.8 is below, but this is a stopgap since this inability to recover from a late starting server likely should be addressed at the rpclib level, so not going to create a merge request.

Speaking of rpclib, given carla has forked rpclib, likely due to upstream abandonment. @bernatx it might be worth examining another user's fork for fixes, but it doesn't look like this particular shortcoming has been addressed in that fork either (based on commit logs).

carla_issue_3707.txt

[corkyw10]

@bernatx are you able to follow up on this please?

[qhaas]

It would appear that the maintainer of the original rpclib is back and looking for someone to takeover.

[sztomi]

@qhaas to be clear: there is a new maintainer (@qchateau) tending for the project (merging PRs and bugfixes mostly). I opened that issue because this maintenance doesn't cover new development - in case someone is interested in that, they would be welcome as another maintainer.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.