Implement shed serve and test commands, other enhancements.

[jmchilton]

• Implement shed_test command that installs the latest installable revision of specified repositories and tests them. Closes Implement shed_test command. #176. a48d811
• Implement shed_serve command that likewise installs the latest installable revision of specified repositories into a fresh Galaxy instance and serves it. 74e3d64
• Enhance a variety of commands to allow specifying multiple paths (test, serve, shed_*). Closes Many more commands should allow specifing multiple directories. #150. 2bd5dfc
• Refactorings to enable this and reduce existing duplication. daa4f59, 4116999

Requires merging of:

• Implement a --failed flag for the test command. #210
• Allow config of shed tool conf used with run_tests.sh -installed. galaxy#267

[hexylena]

@jmchilton does this get us closer to #19 ?

[jmchilton]

Nope... this is almost exactly the opposite of that. Planemo was originally about fighting the machine and has totally devolved into just another a cog 😡.

I have been making slow progress on #19 (for instance https://github.com/galaxyproject/planemo/tree/shed2tap2). I tried to work on this again this weekend and I am having real problems synchronizing how Python is handled in the Tool Shed with Homebrew.

[hexylena]

Ahah. Gotcha. From the description of "installs the latest installable revision of specified repositories into a fresh Galaxy instance and serves it" it sounded a lot like we'd be getting close to testing tools with dependencies.

[jmchilton]

This does test tools with dependencies - but once they have been uploaded into the tool shed not before. This can be thought of a stand-alone variant of Galaxy's install and test framework.

[hexylena]

So, if I'm reading this right, I can use shed_test for testing packages, and then shed_serve to upload them all to a Galaxy, and somehow then use a test to use that same Galaxy with the installed packages?

[jmchilton]

Not quite - a potential workflow would be something like this:

• Use test and lint to develop tools locally.
• Use serve to review tool. Ready to publish...
• Use shed_init to create .shed.yml.
• Write tool_dependencies.xml.
• Use shed_upload --force_repository_creation --shed_target testtoolshed to populate the tool on the test tool shed.
• Use shed_test --target testtoolshed to test published tools. This will start a Galaxy in the background, install the tool(s) from the test tool shed and its dependencies, and run the tests with tool shed dependency resolution (instead of local resolution configured with test).
• Publish to main tool shed.

I don't think I would use shed_serve in a typical workflow - but if you want to do serve but with tool shed installed tools instead of local copies - that is what it would do. (More for consistency and as a stepping stone to ``shed_test`.)

The goal with #19 is more to have a test that works like shed_test but without having to upload the packages to the tool shed and install and test them. If you had a really robust variant of #19 you can imagine you could eliminate publishing to the test tool shed and running shed_test from this workflow - you would be confident the thing you upload passes the tests before publishing them.

[hexylena]

As always I'm thinking from the perspective of CI servers... :)

So I could do:

• launch a TS in a docker container (@bgruening, we can do this yeah?)
• Use shed_upload --force_repository_creation --shed_target $docker_ts to populate the tool on the local tool shed.
• Use shed_test --target $docker_ts to test published tools. This will start a Galaxy in the background, install the tool(s) from the test tool shed and its dependencies, and run the tests with tool shed dependency resolution (instead of local resolution configured with test).

[jmchilton]

Well again - if we could do the mythical #19 you wouldn't need the tool shed - planemo+galaxy would be enough.

This stop gap is so I have something useful to show at the GCC but doesn't help with your exact scenario. It does help with other potential CI jobs - for instance you can imagine automatically pushing everything committed to github to the test tool shed and if the tests pass there with shed_test then promoting them to the main tool shed. Or giving the repository a green light for publication in a UI built on something like #177.

But yes - this doesn't help at all with testing pull requests say - without combining it with something like #119 or "capsules".

[hexylena]

Sure, not needing a TS would be absolutely brilliant, but I'm willing to use whatever hacks/stopgaps I can in the meantime.

for instance you can imagine automatically pushing everything committed to github to the test tool shed and if the tests pass there with shed_test then promoting them to the main tool shed

definitely going to do that for galaxyproject/tools-iuc#129

I'll play around with the new commands soon and see what I can do, I really, really, really want Jenkins/Planemo tests to start being green across the board rather than "oh, this test is failing because the bedtools from brew is a different patch version than the one in the TS"

[bgruening]

@erasche yes we have a TS in a docker container if you are interested.
@jmchilton probably know my plan to strip out the relevant parts of parsing & installing tool_dependencies into it's own library. But this hack is great and serves us well over the next month I guess!

Thanks John!
Bjoern

[hexylena]

@bgruening wunderbar. I'll test this out wednesday and see if I can get jenkins to handle complete testing now. Ahhh so excited.