--pack should preserve @namespace

[anton-khodak]

---

Expected Behavior

cwltool should resolve file formats specified as links (e.g., after --pack-ing a workflow)

Steps to reproduce

cd /path/to/common-workflow-language/v1.0/v1.0
cwltool --pack formattest2.cwl > formattest2-packed.cwl
cwltool formattest2-packed.cwl formattest2-job.json

Expected output:

{
    "output": {
        "format": "http://edamontology.org/format_2330", 
        "checksum": "sha1$97fe1b50b4582cebc7d853796ebd62e3e163aa3f", 
        "basename": "output.txt", 
        "location": "file:///Users/anton/Programming/CWL/cwltool/cwltool/schemas/v1.0/output.txt", 
        "path": "/Users/anton/Programming/CWL/cwltool/cwltool/schemas/v1.0/output.txt", 
        "class": "File", 
        "size": 1111
    }
}

Produced output:

{
    "output": {
        "format": "edam:format_2330", 
        "checksum": "sha1$97fe1b50b4582cebc7d853796ebd62e3e163aa3f", 
        "basename": "output.txt", 
        "location": "file:///Users/anton/Programming/CWL/cwltool/cwltool/schemas/v1.0/output.txt", 
        "path": "/Users/anton/Programming/CWL/cwltool/cwltool/schemas/v1.0/output.txt", 
        "class": "File", 
        "size": 1111
    }
}

Your Environment

• cwltool version:
  1.0.20171107133715

[tetron]

It should have lifted the $namespaces section, so the loader of the packed file is responsible for resolving it (by usual rules), is that missing?

[mr-c]

Here's the packed version from cwltool 1.0.20171205210500

{
    "class": "CommandLineTool", 
    "doc": "Reverse each line using the `rev` command", 
    "hints": [
        {
            "dockerPull": "debian:stretch-slim", 
            "class": "DockerRequirement"
        }
    ], 
    "inputs": [
        {
            "type": "File", 
            "inputBinding": {}, 
            "format": "http://edamontology.org/format_2330", 
            "id": "#main/input"
        }
    ], 
    "outputs": [
        {
            "type": "File", 
            "outputBinding": {
                "glob": "output.txt"
            }, 
            "format": "$(inputs.input.format)", 
            "id": "#main/output"
        }
    ], 
    "baseCommand": "rev", 
    "stdout": "output.txt", 
    "id": "#main", 
    "cwlVersion": "v1.0"
}

So no, the namespace section isn't being copied over during --pack

FYI: the input object is using edam but it doesn't define it:
(formattest2-job.json)

{
    "input": {
        "class": "File",
        "location": "ref.fasta",
        "format": "edam:format_1929"
    }
}

Adding a @namespaces section to the input object doesn't change the final output when using the --pack'd version.

Reviewing the spec I'm not seeing a lot of guidance on how to process the input object with regard to namespaces.

[mr-c]

So perhaps the real question is, do users need to duplicate @namespaces in input objects to make use of them?

If not, then --pack needs to preserve @namespaces and not collapse them.

[mr-c]

The chosen solution was to allow the namespace collapsing (as that might be different for different sub components of the CWL description) but to propagate the top level @namespaces so that it would be available to input documents.