Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Recipe and recipe execution #1192

Merged
merged 1 commit into from
Apr 23, 2019
Merged

Recipe and recipe execution #1192

merged 1 commit into from
Apr 23, 2019

Conversation

ttung
Copy link
Collaborator

@ttung ttung commented Apr 16, 2019

Builds on top of #1163 to construct and run recipes with one or more runnables.

A recipe wires together one or more runnables. Inputs to recipes are available to the recipe itself as the magic variable name "file_inputs". A runnable is invoked by the magic function "compute". Anything written to the magic variable "file_outputs" is matched and written out to a set of output paths.

This is a continuation of #1117
Fixes #311

@codecov-io
Copy link

codecov-io commented Apr 16, 2019

Codecov Report

Merging #1192 into master will increase coverage by 0.23%.
The diff coverage is 95.87%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #1192      +/-   ##
==========================================
+ Coverage   88.42%   88.66%   +0.23%     
==========================================
  Files         136      137       +1     
  Lines        5071     5168      +97     
==========================================
+ Hits         4484     4582      +98     
+ Misses        587      586       -1
Impacted Files Coverage Δ
starfish/recipe/__init__.py 100% <100%> (ø) ⬆️
starfish/recipe/recipe.py 95.83% <95.83%> (ø)
starfish/recipe/runnable.py 95.55% <0%> (+1.11%) ⬆️
starfish/recipe/filesystem.py 95.91% <0%> (+8.16%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 8557657...a780b19. Read the comment docs.

ordered_sequence = OrderedSequence()
file_outputs: MutableMapping[int, Runnable] = {}
recipe_scope = {
"Axes": Axes,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we need this info?

@ttung ttung force-pushed the tonytung-recipe branch from f09f72c to 77c21ec Compare April 17, 2019 02:39
@ttung ttung force-pushed the tonytung-runnable branch from 62a9474 to aeeec4f Compare April 17, 2019 03:38
@ttung ttung force-pushed the tonytung-recipe branch 3 times, most recently from e5c2ef4 to b821904 Compare April 17, 2019 07:23
@ttung ttung force-pushed the tonytung-runnable branch from 967c275 to af72897 Compare April 17, 2019 07:26
@ttung ttung force-pushed the tonytung-recipe branch from b821904 to 23843f5 Compare April 17, 2019 07:26
@ttung ttung force-pushed the tonytung-runnable branch from af72897 to 8741e29 Compare April 17, 2019 17:29
@ttung ttung force-pushed the tonytung-recipe branch 3 times, most recently from c97d827 to 78ffea6 Compare April 18, 2019 03:33
@ttung ttung force-pushed the tonytung-runnable branch from 30131d5 to 65c9fcb Compare April 18, 2019 05:22
@ttung ttung force-pushed the tonytung-recipe branch 2 times, most recently from 2821750 to c472b10 Compare April 18, 2019 19:13
@ttung ttung force-pushed the tonytung-runnable branch from ae2e012 to e7afa06 Compare April 18, 2019 19:57
@ttung ttung force-pushed the tonytung-recipe branch from c472b10 to ac7794c Compare April 18, 2019 19:57
@ttung ttung force-pushed the tonytung-runnable branch from e7afa06 to e8d86b0 Compare April 18, 2019 19:58
@ttung ttung force-pushed the tonytung-recipe branch 3 times, most recently from a2452c4 to 744112b Compare April 18, 2019 22:19
@ttung ttung force-pushed the tonytung-runnable branch from e8d86b0 to 8407918 Compare April 19, 2019 05:41
@ttung ttung force-pushed the tonytung-recipe branch from 744112b to 5ddf958 Compare April 19, 2019 05:41
@ttung ttung force-pushed the tonytung-runnable branch from 8407918 to ec49c0e Compare April 19, 2019 05:59
@ttung ttung force-pushed the tonytung-recipe branch from d93822a to 8c9add1 Compare April 19, 2019 06:47
@ttung ttung force-pushed the tonytung-runnable branch from 2f827bf to 62b1ebb Compare April 19, 2019 16:10
@ttung ttung force-pushed the tonytung-recipe branch 4 times, most recently from f1808b6 to bcd4ed7 Compare April 19, 2019 20:30
Copy link
Member

@ambrosejcarr ambrosejcarr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good. Where are the docs? ;-)

@ttung ttung force-pushed the tonytung-recipe branch 5 times, most recently from cb07126 to 920b571 Compare April 21, 2019 17:11
Copy link
Member

@ambrosejcarr ambrosejcarr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great. Did I miss documentation of the deviations from python-style code (e.g. the @ symbol)?

@ttung ttung force-pushed the tonytung-recipe branch 2 times, most recently from 05caa43 to c7ef79c Compare April 22, 2019 00:10
@ttung ttung force-pushed the tonytung-runnable branch from 2f42b0f to 4cfc692 Compare April 22, 2019 00:11
@ttung ttung force-pushed the tonytung-recipe branch from c7ef79c to 239c0af Compare April 22, 2019 00:11
@ttung
Copy link
Collaborator Author

ttung commented Apr 22, 2019

Looks great. Did I miss documentation of the deviations from python-style code (e.g. the @ symbol)?

you did. that is a CLI-ism. Not a recipe thing.

@ttung ttung force-pushed the tonytung-recipe branch 4 times, most recently from e8bb437 to 1038482 Compare April 22, 2019 19:29
@ttung ttung changed the base branch from tonytung-runnable to master April 22, 2019 19:30
@ttung ttung force-pushed the tonytung-recipe branch 3 times, most recently from ec53c50 to 61dd567 Compare April 22, 2019 21:50
Builds on top of #1097 to construct and run recipes with one or more runnables.

A recipe wires together one or more runnables.  Inputs to recipes are available to the recipe itself as the magic variable name "file_inputs".  A runnable is invoked by the magic function "compute".  Anything written to the magic variable "file_outputs" is matched and written out to a set of output paths.

Fixes #311
@ttung ttung force-pushed the tonytung-recipe branch from 61dd567 to a780b19 Compare April 23, 2019 00:38
@ttung ttung merged commit b64bb0f into master Apr 23, 2019
@ttung ttung deleted the tonytung-recipe branch April 23, 2019 05:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

make a local pipeline runner
4 participants