Propose two VGP workflows generating customised browsers

[fubar2]

Draft of two VGP workflows for review and suggestions.

Basic tracks documentation draft

Dimers, coverage, SV documentation draft

[fubar2]

Now that tests pass using a simple has_size on the output zip, will start adding more assertions about the contents to each one.

[github-actions]

Test Results (powered by Planemo)

Test Summary

Test State	Count
Total	0
Passed	0
Error	0
Failure	0
Skipped	0

[bgruening]

@fubar2 I don't think any tests have run. I found this error in the logs:

Traceback (most recent call last):
  File "/tmp/tmpeh7k1bw5/galaxy-dev/lib/galaxy/workflow/run.py", line 467, in replacement_for_connection
    replacement = step_outputs[output_name]
                  ~~~~~~~~~~~~^^^^^^^^^^^^^
KeyError: 'output'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/tmp/tmpeh7k1bw5/galaxy-dev/lib/galaxy/workflow/run.py", line 238, in invoke
    incomplete_or_none = self._invoke_step(workflow_invocation_step)
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/tmp/tmpeh7k1bw5/galaxy-dev/lib/galaxy/workflow/run.py", line 322, in _invoke_step
    incomplete_or_none = invocation_step.workflow_step.module.execute(
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/tmp/tmpeh7k1bw5/galaxy-dev/lib/galaxy/workflow/modules.py", line 2144, in execute
    collection_info = self.compute_collection_info(progress, step, all_inputs)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/tmp/tmpeh7k1bw5/galaxy-dev/lib/galaxy/workflow/modules.py", line 517, in compute_collection_info
    collections_to_match = self._find_collections_to_match(progress, step, all_inputs)
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/tmp/tmpeh7k1bw5/galaxy-dev/lib/galaxy/workflow/modules.py", line 537, in _find_collections_to_match
    data = progress.replacement_for_input(self.trans, step, input_dict)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/tmp/tmpeh7k1bw5/galaxy-dev/lib/galaxy/workflow/run.py", line 442, in replacement_for_input
    replacement = self.replacement_for_connection(connection[0], is_data=is_data)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/tmp/tmpeh7k1bw5/galaxy-dev/lib/galaxy/workflow/run.py", line 469, in replacement_for_connection
    raise modules.FailWorkflowEvaluation(
galaxy.workflow.modules.FailWorkflowEvaluation

Can you check in the best-practice wizard in the wf-editor that everything is green and well annotated.

[fubar2]

The test outputs look fine. https://vgp.usegalaxy.org/u/fubar/h/ebphib-dimer-coverage-robot-test
The planemo copy of the workflow seems to have an error about a legacy parameter - I'll double check that I am testing what I think I am.

[fubar2]

Got another one from the other workflow. In a subworkflow.
Will update the copy I am using to test and see if that helps.
I cannot currently see any Planemo outputs from CI - github times out and throws a unicorn.

galaxy.workflow.run ERROR 2024-10-10 08:56:50,088 [pN:main.1,p:5757,tN:WorkflowRequestMonitor.monitor_thread] Failed to schedule Workflow[id=8,name=TreeValGal_anno_bed_optional], problem occurred on WorkflowStep[index=17,type=tool,label=None,uuid=c53a0cf0-5b7d-49b8-a8fb-26a3adfef04c,id=108].
Traceback (most recent call last):
  File "/tmp/tmpep3m9rtm/galaxy-dev/lib/galaxy/workflow/run.py", line 467, in replacement_for_connection
    replacement = step_outputs[output_name]
                  ~~~~~~~~~~~~^^^^^^^^^^^^^
KeyError: 'output'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/tmp/tmpep3m9rtm/galaxy-dev/lib/galaxy/workflow/run.py", line 238, in invoke
    incomplete_or_none = self._invoke_step(workflow_invocation_step)
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/tmp/tmpep3m9rtm/galaxy-dev/lib/galaxy/workflow/run.py", line 322, in _invoke_step
    incomplete_or_none = invocation_step.workflow_step.module.execute(
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/tmp/tmpep3m9rtm/galaxy-dev/lib/galaxy/workflow/modules.py", line 2144, in execute
    collection_info = self.compute_collection_info(progress, step, all_inputs)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/tmp/tmpep3m9rtm/galaxy-dev/lib/galaxy/workflow/modules.py", line 517, in compute_collection_info
    collections_to_match = self._find_collections_to_match(progress, step, all_inputs)
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/tmp/tmpep3m9rtm/galaxy-dev/lib/galaxy/workflow/modules.py", line 537, in _find_collections_to_match
    data = progress.replacement_for_input(self.trans, step, input_dict)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/tmp/tmpep3m9rtm/galaxy-dev/lib/galaxy/workflow/run.py", line 442, in replacement_for_input
    replacement = self.replacement_for_connection(connection[0], is_data=is_data)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/tmp/tmpep3m9rtm/galaxy-dev/lib/galaxy/workflow/run.py", line 469, in replacement_for_connection
    raise modules.FailWorkflowEvaluation(
galaxy.workflow.modules.FailWorkflowEvaluation

[fubar2]

Still seeing errors like

Traceback (most recent call last):
  File "/tmp/tmp0jlhb0vq/galaxy-dev/lib/galaxy/workflow/run.py", line 467, in replacement_for_connection
    replacement = step_outputs[output_name]
                  ~~~~~~~~~~~~^^^^^^^^^^^^^
KeyError: 'output'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/tmp/tmp0jlhb0vq/galaxy-dev/lib/galaxy/workflow/run.py", line 238, in invoke
    incomplete_or_none = self._invoke_step(workflow_invocation_step)
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/tmp/tmp0jlhb0vq/galaxy-dev/lib/galaxy/workflow/run.py", line 322, in _invoke_step
    incomplete_or_none = invocation_step.workflow_step.module.execute(
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/tmp/tmp0jlhb0vq/galaxy-dev/lib/galaxy/workflow/modules.py", line 2144, in execute
    collection_info = self.compute_collection_info(progress, step, all_inputs)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/tmp/tmp0jlhb0vq/galaxy-dev/lib/galaxy/workflow/modules.py", line 517, in compute_collection_info
    collections_to_match = self._find_collections_to_match(progress, step, all_inputs)
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/tmp/tmp0jlhb0vq/galaxy-dev/lib/galaxy/workflow/modules.py", line 537, in _find_collections_to_match
    data = progress.replacement_for_input(self.trans, step, input_dict)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/tmp/tmp0jlhb0vq/galaxy-dev/lib/galaxy/workflow/run.py", line 442, in replacement_for_input
    replacement = self.replacement_for_connection(connection[0], is_data=is_data)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/tmp/tmp0jlhb0vq/galaxy-dev/lib/galaxy/workflow/run.py", line 469, in replacement_for_connection
    raise modules.FailWorkflowEvaluation(
galaxy.workflow.modules.FailWorkflowEvaluation

[fubar2]

@bgruening: Just ran that descriptive one from the CL. It worked and the output in history https://vgp.usegalaxy.org/u/fubar/h/ebphib-descriptive-robot-test-1 is https://vgp.usegalaxy.org/datasets/f9cad7b01a472135fb471c9626540288/preview which works as expected with the absurdly small test inputs.
Seems the workflow and planemo tests are good so not sure what's next.
Should I start by splitting into two PR to help simplify the CI testing situation?
Advice appreciated.

[fubar2]

ping @mvdbeek @Delphine-L : What's the correct protocol for IWC PR review once they pass tests?
Review, comments and suggestions welcomed for this please.