Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow unambiguous specification of requirement names #3117

Closed
wants to merge 1 commit into from

Conversation

jmchilton
Copy link
Member

@jmchilton jmchilton commented Nov 2, 2016

This adapts CWL-style SoftwareRequirement specs to solve 83% of #1927 (the various fringe problems caused by tool shed and conda package names not matching exactly). Here I'm trying to implement the CWL specification in a way that helps solve this in Galaxy. It is a delicate balancing act aimed to upset as many interested parties as I can.

To understand the problem - consider the blast+ requirement found in the Galaxy wrappers. It looks something like this:

<requirement type="package" version="2.2.31>blast+</requirement>

Great, that works for Galaxy and Tool Shed packages. It doesn't work for bioconda at all. I think this problem is fairly uncommon - most packages have simple names shared across Debian, Brew, Conda, etc... - but it does happen in some cases that there are inconsistencies. Some people have taken to duplicating the requirement - this is bad and should not be done since they are mutually exclusive and Galaxy will attempt to resolve both.

This PR introduces the following syntax for tools with profile >= 16.10:

<requirement type="package" version="2.2.31" name="blast+">
    <specification uri="https://anaconda.org/bioconda/blast" />
    <specification uri="https://packages.debian.org/sid/ncbi-blast+" version="2.2.31-3" />
</requirement>

This allows finer grain resolution of the requirement without sacrificing the abstract name at the top. It allows the name and the version to be adapted by resolvers as needed (hopefully rarely so).

This syntax is the future facing one, but obviously this tool would not work on older Galaxy versions. To remedy this - an alternative syntax can be used for tools targetting Galaxy versions pre-16.10:

<requirement type="package"
    version="2.2"
    specification_uris="https://anaconda.org/bioconda/blast,https://packages.debian.org/sid/[email protected]"
>blast+</requirement>

This syntax sucks - but it does give newer Galaxies the ability to resolve the specifications without breaking the more simple functionality for older Galaxies. Example tools demonstrating both syntax types are available in the PR.

For more information on the CWL side of this - checkout the discussion on common-workflow-language/cwltool#214. The CWL specification information is defined at http://www.commonwl.org/v1.0/CommandLineTool.html#SoftwarePackage.

I've stubbed out the documentation changes in case we don't reach consensus on this. I will be happy to update the docs with these details if the is a broad consensus that this is the least objectionable design.

Update x2: I've tweaked the XML examples a bit after initial PR creation.

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
This adapts CWL-style ``SoftwareRequirement`` ``specs`` to solve galaxyproject#1927. Here I'm trying to implement the CWL specification in a way that helps enable the feasibility of Conda packaging in Galaxy. It is a delicate balancing act aimed to upset as many interested parties as I can.

To understand the problem - consider the ``blast+`` requirement found in the Galaxy wrappers. It looks something like this:

```
<requirement type="package" version="2.2.31" name="blast+">
```

Great, that works for Galaxy and Tool Shed packages. It doesn't work for bioconda at all. I think this problem is fairly uncommon - most packages have simple names shared across Debian, Brew, Conda, etc... - but it does happen in some cases that there are inconsistencies. Some people have taken to duplicating the requirement - this is bad and should not be done since they are mutually exclusive and Galaxy will attempt to resolve both.

This introduces the following syntax for tools with profile >= 16.10:

```
<requirement type="package" version="2.2.31" name="blast+">
    <specification uri="https://anaconda.org/bioconda/blast" />
    <specification uri="https://packages.debian.org/sid/ncbi-blast+" version="2.2.31-3" />
</requirement>
```

This allows finer grain resolution of the requirement without sacrificing the abstract name at the top. It allows the name and the version to be adapted by resolvers as needed (hopefully rarely so).

This syntax is the future facing one, but obviously this tool would not work on older Galaxy versions. To remedy this - an alternative syntax can be used for tools targetting Galaxy verions pre-16.10:

```
<requirement type="package" version="2.2" specification_uris="https://anaconda.org/bioconda/[email protected],https://packages.debian.org/jessie/[email protected]">blast+</requirement>
```

This syntax sucks - but it does give newer Galaxies the ability to resolve the specifications without breaking the more simple functionality for older Galaxies.

For more information on the CWL side of this - checkout the discussion on common-workflow-language/cwltool#214. The CWL specification information is defined at http://www.commonwl.org/v1.0/CommandLineTool.html#SoftwarePackage.
@mvdbeek
Copy link
Member

mvdbeek commented Nov 2, 2016

I think this is a great solution to a very annoying problem, I don't see any problem with this. I vote 👍 for the approach.

@bgruening
Copy link
Member

bgruening commented Nov 3, 2016

This is not an easy syntax and makes tool dev even harder, usability again degrades here :(
On a more technical note this change does not solve the need for a requirement file which we would actually need at some point, if we change the syntax I would rather see a solution that solves "all" our current problems.

Imho, currently we are facing this problem because we are in a transition phase but once the dust is settled I don't think we have this problem - instead we have a solid solution where we support Conda, Docker, Singularity etc with one set of binaries - reproducible between technologies. This PR is also in this regard a step back imho.

@jmchilton
Copy link
Member Author

@bgruening How can this make anything harder - it is optional, it is only for corner cases, and there is no alternative for this currently. If you'd like to propose an alternative syntax that would be fantastic - this feels like a hit and run without one.

I'm strongly opposed to exposing details about Galaxy internals - such as resolver type - in the XML as has been previously suggested. James has likewise rebuffed suggestions that would brew/conda specific tags in the XML - it is meant to be abstract.

If you don't think tool authors should target other package managers beside conda also - that is fine make it part of the IUC standards not to. But Galaxy is meant to be a generic, plugin driven platform - if someone wants to take it a go build a toolset based around Debian they should be able to. You preference for conda shouldn't stop them from doing that.

@bgruening I beg you - if you want to be only conda - you have so much power to push that. This syntax doesn't need to appear in any of your tools or your training materials or in the IUC repo - but it is important for the platform as a platform.

]]></command>
<requirements>
<!-- Demonstrate backward-compatible-ish specification_uri syntax. -->
<requirement type="package" version="2.2" specification_uris="https://anaconda.org/bioconda/[email protected],https://packages.debian.org/jessie/[email protected]">blast+</requirement>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So theoretically this wouldn't lint in older Galaxies, right?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Galaxy doesn't lint stuff - planemo does. Older versions of Galaxy would load and use this requirement I think but to pass linting would require a Planemo upgrade post merge (as with all XSD changes now).

Copy link
Member

@mvdbeek mvdbeek Nov 3, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It does miss the name attribute though, right?
nevermind, had the wrong syntax in my head

Copy link
Member

@martenson martenson Nov 3, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Which is fine with me. IOW specification_uris param is unknown and ignored by older Galaxies.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes.

@jxtx
Copy link
Contributor

jxtx commented Nov 3, 2016

My concern is that this puts information about specific ways of resolving dependencies into the tool configuration file. So if we introduce additional resolvers in the future, every tool config needs to change. Or if a local maintainer wants to use a different approach (modules, rpms, ...) they need to modify their local tool configs.

You could have a mapping file to address that of course that maps abstract requirements to modules, rpm, ...

But once you start down that road why not just use that approach for all resolvers? Yes, it needs to be maintained separately. But I still feel like that is the better compromise.

@martenson
Copy link
Member

I really thought there must be some people already maintaining this sort of list which we could just enhance with TS package column. I did not find it yet though.

@jmchilton
Copy link
Member Author

jmchilton commented Nov 3, 2016

@jxtx I would argue that this doesn't expose details of "how" to resolve things - it should be seen as clarifying the name of the software with unamibigous URIs. It is clarifying the name of the package - not how to install it. In most cases this won't be needed.

I also want to make it clear - that this could one day map to a biotools URI for the package - and that biotools could host a mapping - a truly cross platform mapping - of URIs to package manager names. This would mean we wouldn't need to update each edge case tool for each resolver - we just need to work with biotools.

This isn't a Galaxy-specific problem - we shouldn't be building Galaxy-specific mappings. We should be building bridges and working with broader communities. That is why I think synchronizing how we do this between Galaxy tools and CWL tools is important and why getting what we do here in galaxy-lib to be the "de facto" standard for CWL.

@jmchilton jmchilton changed the title Allow finer grain specification of requirements. Allow unambiguous specification of requirement names Nov 3, 2016
@martenson
Copy link
Member

martenson commented Nov 3, 2016

biotools being https://bio.tools/ ? Can you please provide a link where is this effort described (package's uri unification)?

@jmchilton
Copy link
Member Author

@martenson - Indeed that is the effort I'm referring to. It is the Elixir effort to create a tool (in the abstract - not Galaxy or CWL sense) registry. I have no link for specific plans around URIs and mapping - this "plan" stems from informal conversations I've been tangentially been involved with.

Is there any disagreement from any party that this mapping would be better maintained by an effort like that? If no, is approval of this PR contingent on such an effort being written up or implemented?

@mr-c
Copy link
Contributor

mr-c commented Nov 3, 2016

@martenson Here is the relevant part of the design document for the biotoolsSchema 2.0 https://docs.google.com/document/d/1tqw7FELV4F_qzrTA9KpVYoORAeFPyY1ZOjaGTPN2H1E/edit#heading=h.20wnowszz93y

@jxtx
Copy link
Contributor

jxtx commented Nov 3, 2016

it should be seen as clarifying the name of the software with unamibious URIs. It is clarifying the name of the package - not how to install it. In most cases this won't be needed.

Yes, it is specifying what the name of the package is for some chosen set of specific packaging providers.

I also want to make it clear - that this could one day map to a biotools URI for the package - and that biotools could host a mapping - a truly cross platform mapping - of URIs to package manager names. This would mean we wouldn't need to update each edge case tool for each resolver - we just need to work with biotools.

But we would have to update every tool in the world to have the biotools URIs first no?

This isn't a Galaxy-specific problem - we shouldn't be building Galaxy-specific mappings.

Probably not. I'm not arguing the mappings should be Galaxy specific. I am suggesting they do not being in the tool configurations. We could quickly end up with a list of half a dozen specs in every tool. Do we duplicate the entire specs record for every tool that uses samtools?

@jmchilton
Copy link
Member Author

Yes, it is specifying what the name of the package is for some chosen set of specific packaging providers.

I don't think this is true. The example shows two - but if something like software identifiers or biotools handles many package managers for us - then this isn't a problem.

But we would have to update every tool in the world to have the biotools URIs first no?

I retain the position that this is fringe problem and only required when there are ambiguities. This is relatively few tools. For these tools though yes - it would be best if they have biotools URIs. I imagine all best practice tools in Galaxy should rely on tools that have biotools entries.

Probably not. I'm not arguing the mappings should be Galaxy specific. I am suggesting they do not being in the tool configurations. We could quickly end up with a list of half a dozen specs in every tool. Do we duplicate the entire specs record for every tool that uses samtools?

Or just one for bioconda now and one for biotools or identifiers.org as those entities become useful. If you are on board for a non-Galaxy specific mapping - then why not a URI key into this mapping? I guess we could do something more concise but aren't the advantages of working with CWL on this worth doing it that way. It seems worth fighting for a broader than Galaxy solution.

If I'm going to loose on this - do you imagine a path forward for mappings that aren't specific to Galaxy? Should we develop a registry or something? Will enough people in our community care about something besides Conda to make it worth the effort?

@jxtx
Copy link
Contributor

jxtx commented Nov 3, 2016

I don't think this is true. The example shows two - but if something like software identifiers or biotools handles many package managers for us - then this isn't a problem.

I'm uncomfortable with an argument along the lines of "yes this could enable complexity but it won't because someday we will all adopt a single unified package registry that everyone loves". We've rarely if ever seen this work. Okay, you like bio.tools. Meanwhile NIH is also talking about doing or doing this under BD2K.

Or just one for bioconda now and one for biotools or identifiers.org as those entities become useful. If you are on board for a non-Galaxy specific mapping - then why not a URI key into this mapping?

Because there is no obvious choice for what this mapping should be. A layer of indirections lets of have the flexibility of allowing people to use different mappings, even changing mappings as this problem evolves, without needing to modify the tools all the time.

@jmchilton
Copy link
Member Author

We've rarely if ever seen this work.

This reads to me like "It hasn't worked before so we are going to do it ourselves instead of trying to get it to work with others." You know I'm skeptical of these efforts - I'm not super confident bio.tools is going to work - but I think it is a better attempt then doing it ourselves.

Meanwhile NIH is also talking about doing or doing this under BD2K.

I thought this idea was rejected? It would be very disappointing to see this proceed. Regardless - I also do prefer bio.tools - because people in our community believe in it like @mr-c and @bgruening and because I think it is being based on existing long running efforts.

Because there is no obvious choice for what this mapping should be.

I know you believe bioconda is the future of Galaxy - so that would seem to be a pretty obvious choice right there. I've also made a case here for bio.tools. Some desire to hedge on fringe of some of these things doesn't warrant doing this mapping ourselves I don't think.

Doing the mapping ourselves has the problem that shipping the mapping with Galaxy means it will be out of date frequently. The inferior implementation and deployment aspects of that should be enough to prefer this implementation.

If we do decide we are going to do the mapping ourselves - I'd propose that internally we continue down this path but we represent it as a Galaxy URI. So for instance we can automatically convert internally something like:

<requirement type="package" version="2.2.31">blast+</requirement>

into:

<requirement type="package" version="2.2.31" name="blast+">
    <specification uri="galaxy_tool_requirements://blast+" />
</requirement>

This way we can at least share an implementation of this resolution with CWL and whatever mapping we do ourselves can be shared with the CWL community. In other words - if we insist on competing with them I'd like to compete with them in a way that allows them to use our mapping and if our mapping.

Is there still an objection to that?

@jmchilton
Copy link
Member Author

@martenson, @nsoranzo, @lparsons - you have all made the case I think that annotating these things in the tool are better than a mapping file. I didn't agree with this in the abstract and I tried to split the baby with this PR, but it looks like I failed to convince @jxtx that some degree of indirect annotation is better than a mapping file. Are any of convinced by this discussion that the mapping file is superior to in-tool annotations? Are we any closer to reaching a consensus on this?

@hmenager
Copy link
Contributor

hmenager commented Nov 3, 2016

I agree with you @jmchilton on trying to make this requirements management as independent from Galaxy as possible. I think that it would be much more compatible with local infrastructures (such as Pasteur), where the same tools have to be made available on Galaxy and on the command line.
I'm wondering whether this could be a three parties solution:

  • bio.tools defines a unique ID (including version) for a tool
  • galaxy xml syntax allows to include this ID in the requirements of a tool
  • packaging systems (say debian, conda, docker, etc.) allows to include this same ID.

CC @joncison and @osallou for comments, opinions, etc.

@jxtx
Copy link
Contributor

jxtx commented Nov 3, 2016

I tend to be very conservative when it comes to the Galaxy tool configuration
syntax. Almost anything else we do in Galaxy can be changed later with
little consequence. Tool configs on the other hand are our major contract
with the world. Anything we do there requires careful consideration of the
cost in terms of long term support and complexity for our developers.

So while I of course support the idea of a single registry for resolving
all tools, it doesn't exist yet. So yes, my tendency is to hedge with a
layer of indirection.

When I look at the specs proposal I still see a lot of potential for
accumulation of various resolver specifications and a lot of work for
maintainers of those tools in the long run. A tool author would need to
think about all of these different potential packaging systems URI schemes.

I guess this fails two tests for me:

  1. Tool author shouldn't need to know how Galaxy provides the requested
    dependencies
  2. Galaxy instance maintainers should never need to edit a tool config file

I think this is a good discussion and I don't think it should be rushed to
a solution.

On Thu, Nov 3, 2016 at 3:53 PM John Chilton [email protected]
wrote:

We've rarely if ever seen this work.

This reads to me like "It hasn't worked before so we are going to do it
ourselves instead of trying to get it to work with others." You know I'm
skeptical of these efforts - I'm not super confident bio.tools is going to
work - but I think it is a better attempt then doing it ourselves.

Meanwhile NIH is also talking about doing or doing this under BD2K.

I thought this idea was rejected? It would be very disappointing to see
this proceed. Regardless - I also do prefer bio.tools - because people in
our community believe in it like @mr-c https://github.com/mr-c and
@bgruening https://github.com/bgruening and because I think it is being
based on existing long running efforts.

Because there is no obvious choice for what this mapping should be.

I know you believe bioconda is the future of Galaxy - so that would seem
to be a pretty obvious choice right there. I've also made a case here for
bio.tools. Some desire to hedge on fringe of some of these things doesn't
warrant doing this mapping ourselves I don't think.

Doing the mapping ourselves has the problem that shipping the mapping with
Galaxy means it will be out of date frequently. The inferior implementation
and deployment aspects of that should be enough to prefer this
implementation.

If we do decide we are going to do the mapping ourselves - I'd propose
that internally we continue down this path but we represent it as a Galaxy
URI. So for instance we can automatically convert internally something like:

blast+

into:

This way we can at least share an implementation of this resolution with
CWL and whatever mapping we do ourselves can be shared with the CWL
community. In other words - if we insist on competing with them I'd like to
compete with them in a way that allows them to use our mapping and if our
mapping.

Is there still an objection to that?


You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#3117 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAE4ZTNfYzvmuawmWBD9fgLo9mnNgH4Wks5q6jvQgaJpZM4KniRi
.

@jxtx
Copy link
Contributor

jxtx commented Nov 3, 2016

This reads to me like "It hasn't worked before so we are going to do it ourselves instead of trying to get it to work with others." You know I'm skeptical of these efforts - I'm not super confident bio.tools is going to work - but I think it is a better attempt then doing it ourselves.

Trying to get it to work with others sounds great, but the order seems strange. Why not work with bio.tools and bioconda and other packagers to get a registry working, and then convince Galaxy and CWL to buy in?

If the goal is to get to a single registry decoupled from tool developers and frameworks, I'm not yet convinced coupling our tool configs to a bunch of specific resolvers is a step toward that goal.

@jmchilton
Copy link
Member Author

I guess this fails two tests for me:

  1. Tool author shouldn't need to know how Galaxy provides the requested dependencies
  2. Galaxy instance maintainers should never need to edit a tool config file

As unrealistic as 1 is - I actually agree completely. Obviously I agree on 2 also. I think this is a solid - the most solid - approach to deal with both of these issues. I think the status quo tool infrastructure fails these tests much more than implementing this PR. At least we agree on values 😞.

@lparsons
Copy link
Contributor

lparsons commented Nov 4, 2016

I agree with the two main objectives:

  1. Tool authors should not need to know how Galaxy provides dependencies
  2. Galaxy admins (instance maintainers) should never need to edit a tool config file

Thinking about these led me to the conclusion that it is the responsibility of the Galaxy admin to provide the mapping of tools to dependencies. They decide what package management system(s) are in use and Galaxy admins must have the ability to provide a custom mapping if required. However, Galaxy should suggest and use a default mapping when possible. Therefore I propose the following:

  1. Tools must provide a minimal definition in the form of a requirements spec that includes a package name and version. For example:

    <requirement type="package" version="2.2.31">blast+</requirement>
    
  2. Galaxy attempts to determine an appropriate mapping to a dependency given the configured dependency resolvers.

  3. The Galaxy administrator is free to provide a custom mapping whenever they feel appropriate by specifying: galaxy tool (optional), tool package name, tool package version, dependency resolver, resolver package name, resolver package version.

The first two pieces are already implemented, which leaves the third piece which must be in the control of the the Galaxy admin. The method used to implement the custom mapping could therefore be stored in the database or a config file.

@bgruening
Copy link
Member

@lparsons this is exactly what I was trying to formulate in a additional response. We do exactly this in job_conf.xml for tool resources and it makes a lot of sense to have this mapping configurable for an admin and not as burden for the tool developer.

@jmchilton

@bgruening How can this make anything harder - it is optional, it is only for corner cases, and there is no alternative for this currently. If you'd like to propose an alternative syntax that would be fantastic - this feels like a hit and run without one.

There is nothing like an optional feature. Let's consider a use-case:
People want to use modules and the requirement name of Galaxy does not fit the module name. Now you have two solutions hack tools to use this proposed feature internally or try to get this change upstream. People will try to upstream these changes (I would do) or fork tools to get this in. People can use this optional feature to tweak there specific use-case but upstream can not support all of them (https://en.wikipedia.org/wiki/List_of_software_package_management_systems). So how do people maintain this? Even if upstream would maintain this, which is a huge burden for tool devs, how do you ensure reproducibility across instances?

I'm strongly opposed to exposing details about Galaxy internals - such as resolver type - in the XML as has been previously suggested.

Agree and I was always on your side here.

If you don't think tool authors should target other package managers beside conda also - that is fine make it part of the IUC standards not to. But Galaxy is meant to be a generic, plugin driven platform - if someone wants to take it a go build a toolset based around Debian they should be able to. You preference for conda shouldn't stop them from doing that.

This is what you read in my sentence, but this is not what I wanted to say. I think we have a very compelling architecture in the making, that enables a plenitude of different technologies to resolve the exact same binary. If you want to support even more resolvers, that's fine but don't do it on the tool level, this is not good for tool devs.

@bgruening I beg you - if you want to be only conda - you have so much power to push that. This syntax doesn't need to appear in any of your tools or your training materials or in the IUC repo - but it is important for the platform as a platform.

I have no power and no time at all ... but yes these changes will appear in IUC and any other repository because this is how things should work. If there is a feature people will use it and they should upstream it. Please no forks because of using/not using optional features. So in the end we need to deal with it.

Concerning the bio.tools discussion: We are working on this. We have organised hackathons with the bio.tools people we are participating on the discussion about a unique tool id, it's not there yet and it is complicated and implementation without a real working system is not a good argument. If there is a unique tool ID we can and should use this, but time will tell and as it stands it will be pretty easy to use this ID with the current system. Please note that we are mass-registering all tools for Galaxy in the registry to you could in principle use bio.tools as ID mapping service without this extra PR here. Not saying we should do this - we need to discuss this with all people involved just saying that this should be possible with the current ideas floating around in bio.tools.

Currently, if we want to have more resolvers we should put this on the admin side very much like we do with tool resources.

Regarding your explicit implementation:

<requirement type="package" version="2.2.31" name="blast+">
    <specification uri="https://anaconda.org/bioconda/blast" />
    <specification uri="https://packages.debian.org/sid/ncbi-blast+" version="2.2.31-3" />
</requirement>

In bio.tools a URL is for several reasons considered as bad practice and we should see what fits better to describe a tool. This particular example with sid in it's name makes my reproducible heart cry :(

@jmchilton jmchilton closed this Nov 4, 2016
@jmchilton
Copy link
Member Author

@lparsons I don't disagree with you in the least. It saddens me that this PR was construed by some to imply that admins should alter tools to adapt to their infrastructure (point 2). This is almost certainly a total communication failure on my part, because that would be contrary to everything I've ever tried to do with tool dependencies. I don't know how to proceed with this PR - but I'll implement the ability to adapt those things as you described first to clear this up. That will make the environment module resolver in particular much more useful.

@lparsons
Copy link
Contributor

lparsons commented Nov 4, 2016

@jmchilton Don't be sad, communication is hard, and I for one gained considerable clarity through this discussion.

@joncison
Copy link

joncison commented Nov 7, 2016

Hi chaps

This is hard problem, and I appreciate the discussions. I'll chip in a few things from the bio.tools side:

  • short-form ID format is biotools:tooldID/versionID where toolID and versionID are user-specified, URL-safe, length-restricted versions of the user-supplied tool name and version number. toolID limited to 12 chars, versionID limited to 6 chars (these limits based on analysis of real tool names but not yet cast in stone - let me know).
  • persistent URL format is https://bio.tools/tool/toolID/version/versionID e.g. https://bio.tools/tool/SignalP/version/4.1
  • bio.tools is aiming for a comprehensive catalogue of tools, but including links to binaries, containers, VMs, packages etc. I'd welcome very much feedback / requirements on our "candidate stable schema", which we're beginning to put into production this month:
    https://docs.google.com/document/d/1tqw7FELV4F_qzrTA9KpVYoORAeFPyY1ZOjaGTPN2H1E/edit
    https://github.com/bio-tools/biotoolsschema
  • ELIXIR and thus bio.tools is supposed exactly to solve some "infrastructure" problems e.g. mapping as discussed here, allowing other projects to concentrate on their core missions
  • but Rome wasn't built in a day, so I'd beg patience (and your requirements) as we chip away at the problems

Cheers!

@mr-c
Copy link
Contributor

mr-c commented Nov 14, 2016

Hello all, in response to this PR I've proposed some clarifications to how the CWL specifies software: https://github.com/common-workflow-language/common-workflow-language/pull/343/files#diff-d073fd17f9716e916e182a3a9592b388L696

In summary: no specification of what software corresponds to a particular CWL description is required, if one does specify then only a name is required. A portable identifier is recommended, especially if the name conflicts with other packages or is inconsistently specified. That identifier can be mapped to specific mechanisms by the CWL implementation. Additionally allowed are identifiers pointing at specific mechanism from the generic (bioconda, Debian) to the site specific (Environment Modules).

Feedback is very welcome -- and structural changes are still a possibility in later releases of CWL.

jmchilton added a commit to jmchilton/galaxy that referenced this pull request Jan 18, 2017
Per discussion on galaxyproject#3117 - some people were in favor of the idea but some were opposed. I think it is still important to implement this for CWL tools and useful from a library perspective. This just removes changes to Galaxy tool XML.
jmchilton added a commit to jmchilton/galaxy that referenced this pull request Jan 23, 2017
Per discussion on galaxyproject#3117 - some people were in favor of the idea but some were opposed. I think it is still important to implement this for CWL tools and useful from a library perspective. This just removes changes to Galaxy tool XML.
jmchilton added a commit to jmchilton/galaxy that referenced this pull request Jan 23, 2017
Per discussion on galaxyproject#3117 - some people were in favor of the idea but some were opposed. I think it is still important to implement this for CWL tools and useful from a library perspective. This just removes changes to Galaxy tool XML.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

9 participants