Redirect of stdout to stderr

[lunactic]

Expected Behavior

Tell us what should happen

I am using nodejs / typescript to run the cwltool with code like this:

conn.exec(command, (err: Error, stream: ssh.ClientChannel) => {
  (err) {
    reject(err);
   }
  stream.on('close', (code, signal) => {
    Logger.log("debug", "Stream :: close :: code: " + code + ", signal: " + signal, "DockerManagement::runDockerImageSSH");
  }).on('data', (data) => {
    Logger.log("debug", "STDOUT: " + data, "DockerManagement::runDockerImageSSH");
  }).stderr.on('data', (data) => {
    Logger.log("error", "STDERR: " + data, "DockerManagement::runDockerImageSSH");
  });
});

I would expect that logging information about running the cwltool would be logged for me stating with "STDOUT", as it would be logged to the standard output.

Actual Behavior

When I run the cwltool like this:

cwltool --outdir /data_test/results/fuzzybighearteddotterel/data_0/ --debug --tmp-outdir-prefix /data_test/output/ --tmpdir-prefix /data_test/tmp/ --workdir /input /data_test/executables/evaluation/icdar2017hisdoclayoutcomplayoutevaluationtest/1/8d7ba5fdcdedc7908370924b366d266fc4c996e2.cwl /data_test/results/fuzzybighearteddotterel/data_0/data_0.yaml

My logging output looks like this:

[2017-08-28 11:37:05.514] [ERROR] DIVAServices - STDERR: Resolved '/data_test/executables/evaluation/icdar2017hisdoclayoutcomplayoutevaluationtest/1/8d7ba5fdcdedc7908370924b366d266fc4c996e2.cwl' to 'file:///data_test/executables/evaluation/icdar2017hisdoclayoutcomplayoutevaluationtest/1/8d7ba5fdcdedc7908370924b366d266fc4c996e2.cwl'
 [DockerManagement::runDockerImageSSH]
[2017-08-28 11:37:06.096] [ERROR] DIVAServices - STDERR: [job 8d7ba5fdcdedc7908370924b366d266fc4c996e2.cwl] initializing from file:///data_test/executables/evaluation/icdar2017hisdoclayoutcomplayoutevaluationtest/1/8d7ba5fdcdedc7908370924b366d266fc4c996e2.cwl
 [DockerManagement::runDockerImageSSH]

So all logging information is sent to the stderr output. This seems to be the default behaviour as it is set here:

cwltool/cwltool/job.py

Line 458 in 9e1112f

stdout = sys.stderr

I wonder if this behaviour is intentional, and what would be the proper way to handle this differently.

Your Environment

• cwltool version: 1.0.20170824143724

[osallou]

+1 , would expect to redirect job to stdout not stderr. stderr should only container ERROR related info. Furthermore some tools (galaxy...) consider a job failure if there is something is stderr (though can be configured not to do so...).
Same in other places

cwltool/cwltool/job.py

Line 524 in cf18555

stdout=sys.stderr, # The nested script will output the paths to the correct files if they need

[mr-c]

Hello @lunactic and thank you for your issue report.

The only output to stdout from a CWL runner should be the resulting output object. Here is a meta-description of our expectations for cwl-runners: https://github.com/common-workflow-language/common-workflow-language/pull/278/files#diff-ee814a9c027fc9750beb075c283a973cR41

If you need to process the output from the underlying tool you are welcome to capture the tool's stdout via the CWL tool definition itself: http://www.commonwl.org/user_guide/05-stdout/

[lunactic]

Thank you for this answer. With this I was actually able to figure out what I needed to do .