Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

make a local pipeline runner #311

Closed
ambrosejcarr opened this issue Jul 9, 2018 · 1 comment · Fixed by #1192 or #1160
Closed

make a local pipeline runner #311

ambrosejcarr opened this issue Jul 9, 2018 · 1 comment · Fixed by #1192 or #1160
Assignees
@ttung
Milestone
0.1.0

Comments

@ambrosejcarr
Copy link
Member

Starfish users will expect to be able to process multiple FOVs efficiently, but at the moment we are focusing on single FOVs.

The simplest multi-field of view orchestration would be to process them sequentially with a for loop. More sophisticated would be a multi-processing based approach. Finally, we will want to link with the HCA cromwell/wdl architecture.

Design an API to enable these kinds of multi-field of view processing. This will interact with #310 and depends on #302
@ambrosejcarr ambrosejcarr mentioned this issue Jul 9, 2018
Closed
1 task
@ttung ttung self-assigned this Aug 7, 2018
@neuromusic neuromusic added this to the 0.1.0 milestone Dec 19, 2018
@neuromusic neuromusic modified the milestones: 0.2.0, 0.1.0 Mar 13, 2019
@neuromusic neuromusic reopened this Mar 18, 2019
@neuromusic neuromusic changed the title ENH: Draft API for multi-FOV execution in starfish make a local pipeline runner Mar 18, 2019
@ttung ttung mentioned this issue Mar 28, 2019
Closed
@ttung ttung mentioned this issue Apr 10, 2019
Merged
@ttung
Copy link
Collaborator

ttung commented Apr 10, 2019

@berl Would love your feedback if you get a chance. https://github.com/spacetx/starfish/pull/1160/files#diff-7fddd31cfc49ca7f1ce96c1e0d2cc138 is an example of a recipe.

ttung pushed a commit to ttung/starfish that referenced this issue Apr 11, 2019
Recipe runner CLI entry point
a52a19e 
1. Add the recipe runner CLI entry point.
2. Add a recipe for the ISS pipeline and a test that exercises the recipe.

Test plan: `pytest starfish/test/full_pipelines/recipe`

Depends on spacetx#1117

Fixes spacetx#311
@ttung ttung mentioned this issue Apr 11, 2019
Closed
@ttung ttung mentioned this issue Apr 16, 2019
Merged
ttung added a commit that referenced this issue Apr 23, 2019
@ttung
Recipe and recipe execution (#1192)
b64bb0f 
Builds on top of #1097 to construct and run recipes with one or more runnables.

A recipe wires together one or more runnables.  Inputs to recipes are available to the recipe itself as the magic variable name "file_inputs".  A runnable is invoked by the magic function "compute".  Anything written to the magic variable "file_outputs" is matched and written out to a set of output paths.

Fixes #311
ttung added a commit that referenced this issue Apr 23, 2019
@ttung
Recipe runner CLI entry point (#1160)
efa9d78 
1. Add the recipe runner CLI entry point.
2. Add a recipe for the ISS pipeline and a test that exercises the recipe.

Test plan: `pytest starfish/test/full_pipelines/recipe`

Depends on #1117

Fixes #311
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@ttung ttung

Labels
None yet
Projects
None yet
Milestone
0.1.0
Development

Successfully merging a pull request may close this issue.

Recipe and recipe execution spacetx/starfish
Recipe runner CLI entry point spacetx/starfish
3 participants
@ttung @neuromusic @ambrosejcarr