Improve behavior of error reporting

[gaow]

Currently by default running DSC on a local computer hides stderr and stdout. We should write them to files and point users to them when there are errors. We need to also clean them up if they turn out to be empty to prevent leaving behind us a trail of trivial err and out files.

Also we need to make the rerun prompt Rscript <path to script> more obvious -- even suggest in the prompt that users should open up the file and interact with it.

[pcarbo]

Currently by default running DSC on a local computer hides stderr and stdout.

@gaow Do you mean stderr and stdout from running the individual module scripts?

[gaow]

Do you mean stderr and stdout from running the individual module scripts?

Yes. For the default verbosity level we currently hide both of them, because otherwise the screen got flooded by mostly irrelevant info from all concurrent jobs.

[pcarbo]

@gaow For debugging, it would be great to have stderr and stdout logged for all all module instances, perhaps using the naming scheme used for the .Rds files, with a flag to turn off logging separately for stdout and stderr. Do you think this is possible?

[gaow]

perhaps using the naming scheme used for the .Rds files, with a flag to turn off logging separately for stdout and stderr. Do you think this is possible?

Yes it is. Except I'm a bit reluctant to add a new command flag. I had always being trying to keep dsc command options minimal. I'm thinking just keep them in file regardless, but ideally automatically clean up empty ones. Now that we have separate modules in different folder this should not be too bad in terms of file counts and users can always safely rm -f *.stderr *.stdout.

[pcarbo]

@gaow Having an additional flag is less important.

[gaow]

Done. This change relies on the current SoS master (a new SoS release is also soon to come). Documentation "debug tips" will need an update.