Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

need method for outputting pipeline CWL input format #150

Open
stevekm opened this issue Feb 24, 2020 · 0 comments
Open

need method for outputting pipeline CWL input format #150

stevekm opened this issue Feb 24, 2020 · 0 comments

Comments

@stevekm
Copy link
Member

stevekm commented Feb 24, 2020

Right now, the datastructure that pipeline Operators generate from the get_jobs method is not the same as the CWL input for the pipeline. In particular, "file" entries are represented like this in the Operator's datastructure:

{'class': 'File', 'location': 'bid:///c26e1cf4-0455-4ad7-b2b1-8996cb60214d'}

or like this:

{'class': 'File', 'location': 'juno:///path/to/foo.txt'}

However, the actual CWL representation looks like this:

{'class': 'File', 'path': '/path/to/foo.txt'}

Currently, Beagle is automatically converting the former two syntaxes into the latter one after the job is submitted (? @sivkovic please feel free to correct or clarify this). Also, if the user is manually creating the input JSON to pass to the Beagle API to run a pipeline, it must be in the former two syntaxes, and not the latter syntax.

This makes development and manual run creation difficult since the datastructure that you pass to the CWL no longer matches the format used by Beagle, so you cannot verify that your Bealge data is correct by running it against the CWL directly. Similarly, when building a new operator, you cannot manually check that your output JSON format is correct since it will not be accepted by cwltool, etc., due to the format discrepancies.

It would be helpful to be able to somehow output the exact CWL input JSON format that an operator would be creating in order to validate against the CWL pipeline input format.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant