Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Test case for multiprocessing + imagestack #1589

Merged
merged 1 commit into from
Oct 10, 2019
Merged

Conversation

ttung
Copy link
Collaborator

@ttung ttung commented Sep 30, 2019

We want to support multiprocessing + imagestack for two use cases:

  1. algorithms that do not release the GIL do not run well under multithreading.
  2. it allows us to parallelize across FOVs in an interesting way [1]

This adds a test case that we can pass an ImageStack to a child process for processing.

Note that this will not pass travis until spacetx/slicedimage#125 is landed and released.

Test plan: this is the test plan
Depends on #1586

[1] Ideally, we should make it such that the child workers cooperatively share the compute resources. This is not done today. See #1588 for what needs to be done next.

@ttung ttung requested a review from shanaxel42 September 30, 2019 16:23
@codecov-io
Copy link

codecov-io commented Sep 30, 2019

Codecov Report

Merging #1589 into master will not change coverage.
The diff coverage is n/a.

Impacted file tree graph

@@           Coverage Diff           @@
##           master    #1589   +/-   ##
=======================================
  Coverage   88.04%   88.04%           
=======================================
  Files         151      151           
  Lines        5171     5171           
=======================================
  Hits         4553     4553           
  Misses        618      618

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c6d7c44...1753094. Read the comment docs.

@ttung ttung force-pushed the tonytung-multiprocessing branch 2 times, most recently from e9012c2 to d66dfb1 Compare October 1, 2019 02:40
@ttung ttung changed the base branch from tonytung-no-script to master October 1, 2019 03:17
@ttung ttung force-pushed the tonytung-multiprocessing branch 4 times, most recently from 1a63fb1 to df81012 Compare October 4, 2019 17:53
@ttung ttung force-pushed the tonytung-multiprocessing branch 7 times, most recently from 2fbbf98 to 9b90b32 Compare October 10, 2019 06:18
We want to support multiprocessing + imagestack for two use cases:

1. algorithms that do not release the GIL do not run well under multithreading.
2. it allows us to parallelize across FOVs in an interesting way [1]

This adds a test case that we can pass an ImageStack to a child process for processing.

Note that this will not pass travis until spacetx/slicedimage#125 is landed and released.

Test plan: this is the test plan
Depends on #1586

[1] Ideally, we should make it such that the child workers cooperatively share the compute resources.  This is not done today.  See #1588 for what needs to be done next.
@ttung ttung force-pushed the tonytung-multiprocessing branch from 9b90b32 to 1753094 Compare October 10, 2019 06:19
@ttung ttung merged commit 58eb7ac into master Oct 10, 2019
@ttung ttung deleted the tonytung-multiprocessing branch October 10, 2019 08:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants