crew_controller_sge() does not work in newer version

[psychelzh]

Currently the folllowing errors found:

Error in dial(sock, url = url, autostart = asyncdial || NA, tls = tls,  : 
  6 | Connection refused
Calls: <Anonymous> ... do.call -> <Anonymous> -> dial_and_sync_socket -> dial
Execution halted

Do you have any clue on this?

[psychelzh]

I have upgraded {nannext} and {mirai} packages, too, but no luck. At last, I have to resort to targets::tar_make_future(), which works as expected.

[psychelzh]

Maybe this is related to the configuration of our server, because it works when I submit jobs using another node.

[wlandau]

#1 (comment) might help, it could be a matter of supplying the right host name. You might also try the troubleshooting advice at #36 (comment), especially the part about trying to use mirai on the cluster without crew.

I just installed the current CRAN versions of packages (nanonext 0.13.0, mirai 0.12.1, crew 0.9.0, crew.cluster 0.3.0, targets 1.5.1) and tested this SGE pipeline. Everything still works on my company's SGE cluster. So either the error you describe has something to do with your instance of SGE or the specific pipeline you are running. reprexes are always helpful.

[wlandau]

Another thing to try is to make sure your cluster workers in your SGE jobs are using the same packages versions as the local R session. This is especially important for mirai and nanonext. So I would manually start an SGE job and make it print out the mirai and nanonext versions to confirm that they are what you would expect.