Propose two VGP workflows generating customised browsers #557

fubar2 · 2024-10-09T23:07:41Z

Draft of two VGP workflows for review and suggestions.

Dimers, coverage, SV documentation draft

fubar2 · 2024-10-10T02:30:36Z

Now that tests pass using a simple has_size on the output zip, will start adding more assertions about the contents to each one.

github-actions · 2024-10-10T04:25:45Z

Test Results (powered by Planemo)

Test Summary

Test State	Count
Total	0
Passed	0
Error	0
Failure	0
Skipped	0

bgruening · 2024-10-10T06:11:42Z

@fubar2 I don't think any tests have run. I found this error in the logs:

Traceback (most recent call last):
  File "/tmp/tmpeh7k1bw5/galaxy-dev/lib/galaxy/workflow/run.py", line 467, in replacement_for_connection
    replacement = step_outputs[output_name]
                  ~~~~~~~~~~~~^^^^^^^^^^^^^
KeyError: 'output'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/tmp/tmpeh7k1bw5/galaxy-dev/lib/galaxy/workflow/run.py", line 238, in invoke
    incomplete_or_none = self._invoke_step(workflow_invocation_step)
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/tmp/tmpeh7k1bw5/galaxy-dev/lib/galaxy/workflow/run.py", line 322, in _invoke_step
    incomplete_or_none = invocation_step.workflow_step.module.execute(
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/tmp/tmpeh7k1bw5/galaxy-dev/lib/galaxy/workflow/modules.py", line 2144, in execute
    collection_info = self.compute_collection_info(progress, step, all_inputs)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/tmp/tmpeh7k1bw5/galaxy-dev/lib/galaxy/workflow/modules.py", line 517, in compute_collection_info
    collections_to_match = self._find_collections_to_match(progress, step, all_inputs)
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/tmp/tmpeh7k1bw5/galaxy-dev/lib/galaxy/workflow/modules.py", line 537, in _find_collections_to_match
    data = progress.replacement_for_input(self.trans, step, input_dict)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/tmp/tmpeh7k1bw5/galaxy-dev/lib/galaxy/workflow/run.py", line 442, in replacement_for_input
    replacement = self.replacement_for_connection(connection[0], is_data=is_data)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/tmp/tmpeh7k1bw5/galaxy-dev/lib/galaxy/workflow/run.py", line 469, in replacement_for_connection
    raise modules.FailWorkflowEvaluation(
galaxy.workflow.modules.FailWorkflowEvaluation

Can you check in the best-practice wizard in the wf-editor that everything is green and well annotated.

fubar2 · 2024-10-10T07:27:35Z

The test outputs look fine. https://vgp.usegalaxy.org/u/fubar/h/ebphib-dimer-coverage-robot-test
The planemo copy of the workflow seems to have an error about a legacy parameter - I'll double check that I am testing what I think I am.

…eems like a good one...

fubar2 · 2024-10-10T09:26:28Z

Got another one from the other workflow. In a subworkflow.
Will update the copy I am using to test and see if that helps.
I cannot currently see any Planemo outputs from CI - github times out and throws a unicorn.

galaxy.workflow.run ERROR 2024-10-10 08:56:50,088 [pN:main.1,p:5757,tN:WorkflowRequestMonitor.monitor_thread] Failed to schedule Workflow[id=8,name=TreeValGal_anno_bed_optional], problem occurred on WorkflowStep[index=17,type=tool,label=None,uuid=c53a0cf0-5b7d-49b8-a8fb-26a3adfef04c,id=108].
Traceback (most recent call last):
  File "/tmp/tmpep3m9rtm/galaxy-dev/lib/galaxy/workflow/run.py", line 467, in replacement_for_connection
    replacement = step_outputs[output_name]
                  ~~~~~~~~~~~~^^^^^^^^^^^^^
KeyError: 'output'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/tmp/tmpep3m9rtm/galaxy-dev/lib/galaxy/workflow/run.py", line 238, in invoke
    incomplete_or_none = self._invoke_step(workflow_invocation_step)
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/tmp/tmpep3m9rtm/galaxy-dev/lib/galaxy/workflow/run.py", line 322, in _invoke_step
    incomplete_or_none = invocation_step.workflow_step.module.execute(
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/tmp/tmpep3m9rtm/galaxy-dev/lib/galaxy/workflow/modules.py", line 2144, in execute
    collection_info = self.compute_collection_info(progress, step, all_inputs)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/tmp/tmpep3m9rtm/galaxy-dev/lib/galaxy/workflow/modules.py", line 517, in compute_collection_info
    collections_to_match = self._find_collections_to_match(progress, step, all_inputs)
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/tmp/tmpep3m9rtm/galaxy-dev/lib/galaxy/workflow/modules.py", line 537, in _find_collections_to_match
    data = progress.replacement_for_input(self.trans, step, input_dict)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/tmp/tmpep3m9rtm/galaxy-dev/lib/galaxy/workflow/run.py", line 442, in replacement_for_input
    replacement = self.replacement_for_connection(connection[0], is_data=is_data)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/tmp/tmpep3m9rtm/galaxy-dev/lib/galaxy/workflow/run.py", line 469, in replacement_for_connection
    raise modules.FailWorkflowEvaluation(
galaxy.workflow.modules.FailWorkflowEvaluation

fubar2 · 2024-10-10T10:16:52Z

Still seeing errors like

Traceback (most recent call last):
  File "/tmp/tmp0jlhb0vq/galaxy-dev/lib/galaxy/workflow/run.py", line 467, in replacement_for_connection
    replacement = step_outputs[output_name]
                  ~~~~~~~~~~~~^^^^^^^^^^^^^
KeyError: 'output'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/tmp/tmp0jlhb0vq/galaxy-dev/lib/galaxy/workflow/run.py", line 238, in invoke
    incomplete_or_none = self._invoke_step(workflow_invocation_step)
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/tmp/tmp0jlhb0vq/galaxy-dev/lib/galaxy/workflow/run.py", line 322, in _invoke_step
    incomplete_or_none = invocation_step.workflow_step.module.execute(
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/tmp/tmp0jlhb0vq/galaxy-dev/lib/galaxy/workflow/modules.py", line 2144, in execute
    collection_info = self.compute_collection_info(progress, step, all_inputs)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/tmp/tmp0jlhb0vq/galaxy-dev/lib/galaxy/workflow/modules.py", line 517, in compute_collection_info
    collections_to_match = self._find_collections_to_match(progress, step, all_inputs)
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/tmp/tmp0jlhb0vq/galaxy-dev/lib/galaxy/workflow/modules.py", line 537, in _find_collections_to_match
    data = progress.replacement_for_input(self.trans, step, input_dict)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/tmp/tmp0jlhb0vq/galaxy-dev/lib/galaxy/workflow/run.py", line 442, in replacement_for_input
    replacement = self.replacement_for_connection(connection[0], is_data=is_data)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/tmp/tmp0jlhb0vq/galaxy-dev/lib/galaxy/workflow/run.py", line 469, in replacement_for_connection
    raise modules.FailWorkflowEvaluation(
galaxy.workflow.modules.FailWorkflowEvaluation

fubar2 · 2024-10-10T10:29:10Z

@bgruening: Just ran that descriptive one from the CL. It worked and the output in history https://vgp.usegalaxy.org/u/fubar/h/ebphib-descriptive-robot-test-1 is https://vgp.usegalaxy.org/datasets/f9cad7b01a472135fb471c9626540288/preview which works as expected with the absurdly small test inputs.
Seems the workflow and planemo tests are good so not sure what's next.
Should I start by splitting into two PR to help simplify the CI testing situation?
Advice appreciated.

fubar2 · 2024-10-16T22:08:53Z

ping @mvdbeek @Delphine-L : What's the correct protocol for IWC PR review once they pass tests?
Review, comments and suggestions welcomed for this please.

github-actions · 2024-12-02T11:18:56Z

Test Results (powered by Planemo)

Test Summary

Test State	Count
Total	4
Passed	2
Error	1
Failure	1
Skipped	0

Errored Tests

❌ EBPhib-precuration.ga_0

Failed Tests

❌ EBPhib-big.ga_0

Passed Tests

✅ EBPhib-descriptive.ga_0
✅ EBPhib-dimer-coverage.ga_0

github-actions · 2024-12-03T00:53:09Z

Test Results (powered by Planemo)

Test Summary

Test State	Count
Total	4
Passed	4
Error	0
Failure	0
Skipped	0

Passed Tests

✅ EBPhib-big.ga_0
✅ EBPhib-descriptive.ga_0
✅ EBPhib-dimer-coverage.ga_0
✅ EBPhib-precuration.ga_0

fubar2 added 12 commits October 9, 2024 14:01

trying to recover after some git tomfoolery

888ed08

nearly got test for treevalgal working

822f52c

got EBPhisb-descriptive tests working - just fixing a name change...

2b8ea13

update readme

f85ea39

fixing dimer tests

1da5d6c

both EBPhisb workflows pass tests at least...

6d128b4

repair .docstore to match current names

8ba1579

odd - .dockstore not updated

8e2ffae

remove bogus copy of a workflow

53e6bf8

remove old planemo json

3663285

RTM! Adding release to .ga files

e4d687b

add changelogs

fd0256d

fubar2 marked this pull request as draft October 10, 2024 03:46

rename to EBP haplotype interactive browser

fd054f6

Ugh. These tested fine but forgot to push.

46f694f

fubar2 added 3 commits October 10, 2024 19:02

Fixed a problem using a ${} parameter. Not liked. Pity. Handy as.

9c71b3d

add release back into the newly refreshed fixed dimer workflow

cf1185c

Have a strange error so replacing the test descriptive ga with what s…

fc776c6

…eems like a good one...

fubar2 mentioned this pull request Oct 10, 2024

How does one define inputs for TreeValGal? #554

Closed

fubar2 mentioned this pull request Oct 11, 2024

Add the two current workflows to this fork Delphine-L/iwc#11

Open

Merge branch 'galaxyproject:main' into EBPhisb

bf460ec

fubar2 closed this Oct 13, 2024

fubar2 deleted the EBPhisb branch October 13, 2024 11:38

fubar2 restored the EBPhisb branch October 13, 2024 11:41

fubar2 reopened this Oct 13, 2024

fubar2 marked this pull request as ready for review October 16, 2024 22:03

fubar2 and others added 3 commits November 30, 2024 15:20

Merge branch 'galaxyproject:main' into EBPhisb

11a106f

adding a new tiny precuration workflow for Kirsty

13f7304

fix tests

6745db9

both wf tests pass locally

4b4f8a6

fubar2 added 3 commits December 3, 2024 11:57

remove descriptive and dimer EBPhib workflows - now part of the big one

0aa3ab3

add readme

9899b17

forgot to add readme for big

a4c1ff9

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Propose two VGP workflows generating customised browsers #557

Propose two VGP workflows generating customised browsers #557

fubar2 commented Oct 9, 2024 •

edited

Loading

fubar2 commented Oct 10, 2024

github-actions bot commented Oct 10, 2024

bgruening commented Oct 10, 2024

fubar2 commented Oct 10, 2024

fubar2 commented Oct 10, 2024

fubar2 commented Oct 10, 2024

fubar2 commented Oct 10, 2024 •

edited

Loading

fubar2 commented Oct 16, 2024 •

edited

Loading

github-actions bot commented Dec 2, 2024

github-actions bot commented Dec 3, 2024

Propose two VGP workflows generating customised browsers #557

Are you sure you want to change the base?

Propose two VGP workflows generating customised browsers #557

Conversation

fubar2 commented Oct 9, 2024 • edited Loading

fubar2 commented Oct 10, 2024

github-actions bot commented Oct 10, 2024

Test Results (powered by Planemo)

Test Summary

bgruening commented Oct 10, 2024

fubar2 commented Oct 10, 2024

fubar2 commented Oct 10, 2024

fubar2 commented Oct 10, 2024

fubar2 commented Oct 10, 2024 • edited Loading

fubar2 commented Oct 16, 2024 • edited Loading

github-actions bot commented Dec 2, 2024

Test Results (powered by Planemo)

Test Summary

github-actions bot commented Dec 3, 2024

Test Results (powered by Planemo)

Test Summary

fubar2 commented Oct 9, 2024 •

edited

Loading

fubar2 commented Oct 10, 2024 •

edited

Loading

fubar2 commented Oct 16, 2024 •

edited

Loading