Question: possible to refresh asyncResult for new client without holding msg_ids?

[histed]

I'm using a client that connects to the hub via ssh. That ssh connection sometimes times out.
Right now, to reconnect, I start a new client, hold a list of all jobs' msg_ids, and rebuild the asyncresults by querying the hub database by msg_id.

Is it possible to do the same thing without writing code to keep track of all the msg_ids and refresh each async result? This is not that hard when kept in a single list, but it's a pain when nested lists of jobs are needed, or jobs need restarting (and thus msg_ids change).

Is there a way to reconnect an existing client?
Or a way to tell an async result to associate itself to a new client?

Thanks.

[minrk]

In general, no, if you want to preserve an association of msg_ids from one session to the next, you need to keep track of them.

However, zmq automatically reconnects, so if the peer becomes available again, it will re-establish the connection. If SSH decided to terminate the tunnel, then recreating the same tunnel should allow it to be all reconnected. There probably isn't a great API for doing that, since the tunneled ports aren't stored, if I recall. If you created the ssh tunnel(s) manually, and modified the connection file to point to local ports, then it wouldn't be IPython creating the tunnels, it would be you, and you could kill/restart the ssh tunnels and zmq's reconnect might be happier that way.

[histed]

The manual ssh-tunneling is a nice trick, I'll have to try it.

Otherwise, though (if I'm not mistaken) that means that if you're running jobs that take more than a few min, or your network is unreliable at all, you need to work with msg_ids and pretty much skip AsyncResults (i.e. use client.get_result(msg_id).get() not asyncresult.get()) , and you need to use a remote database.
Which is too bad, because AsyncResults have some advantages.

Can you add a method of making AsyncResults robust across network hiccups to the feature request queue? Sounds like it might be difficult, so I'm not holding my breath. Thanks.

[minrk]

Typically, zmq is robust across network hiccups, it's ssh that seems to be causing a problem here. But I will look into a manual reconnect method.

[histed]

Great. My use case is usually: run client on laptop in a notebook, start jobs, wait several hours to days -- with periodic interaction with client to collect outputs and start new contingent jobs (laptop may suspend and/or move networks), retrieve results.
Thanks for your time. ipyparallel is quite nice.

[minrk]

For those really long cases, I think perhaps making the persist msg_ids to disk and restore AsyncResults from msg_ids a better experience would be the way to go. Then you don't have to leave a process running at all.

[histed]

Sure - make sure dill/pickle can serialize AsyncResults, and then add an AsyncResult.renew() method that uses its msg_id to recover itself from the hub if the client is dead?

Also, probably related to zmq allowing reconnections, if the client ssh tunnel times out, asyncresult.get() and stderr and stdout just fail silently - they never change state. That took me by surprise at first.

Thanks