Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ImageStack select on Physical Coordinates #1147

Merged
merged 10 commits into from
Apr 17, 2019
Merged

Conversation

shanaxel42
Copy link
Collaborator

Per the discussion in #1136 I think the correct path going forward is to maintain the xc/yc/zc coordinate arrays on our ImageStack object and provide both an Imagestck.sel_by_physical_coords() method and a convert_coords_to_indices() helper method to help index ImageStacks by physical coordinate values or ranges. This PR adds both and some tests.

@@ -383,14 +383,30 @@ def sel(self, indexers: Mapping[Axes, Union[int, tuple]]):
ImageStack :
a new image stack indexed by given value or range.
"""

# convert indexers to Dict[str, (int/slice)] format
# TODO shanaxel42 check if this can be changed to xarray.copy(deep=false)
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Checked, cannot do regular copy get an xarray error.

@shanaxel42 shanaxel42 requested review from ttung and ambrosejcarr April 8, 2019 17:45
@codecov-io
Copy link

codecov-io commented Apr 8, 2019

Codecov Report

Merging #1147 into master will increase coverage by 0.03%.
The diff coverage is 94.28%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #1147      +/-   ##
==========================================
+ Coverage   88.41%   88.44%   +0.03%     
==========================================
  Files         133      133              
  Lines        4953     4984      +31     
==========================================
+ Hits         4379     4408      +29     
- Misses        574      576       +2
Impacted Files Coverage Δ
starfish/imagestack/imagestack.py 93.88% <100%> (+0.15%) ⬆️
starfish/imagestack/indexing_utils.py 94.87% <92.59%> (-5.13%) ⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 664aa70...96026c3. Read the comment docs.

starfish/imagestack/imagestack.py Outdated Show resolved Hide resolved
starfish/imagestack/imagestack.py Outdated Show resolved Hide resolved
starfish/imagestack/indexing_utils.py Outdated Show resolved Hide resolved
starfish/imagestack/indexing_utils.py Outdated Show resolved Hide resolved
assert iu.find_nearest(stack.xarray[Coordinates.X.value], -5) == 0


def test_convert_coords_to_indices():
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we add a test that does both coordinate and index selection?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

added!

@shanaxel42 shanaxel42 requested a review from ttung April 10, 2019 21:37
@ttung ttung closed this Apr 11, 2019
@ttung ttung deleted the saxelrod-multiple-sel branch April 11, 2019 00:01
@ttung ttung restored the saxelrod-multiple-sel branch April 11, 2019 07:31
@ttung ttung reopened this Apr 11, 2019
@codecov-io
Copy link

Codecov Report

Merging #1147 into master will decrease coverage by 3.52%.
The diff coverage is 92.59%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #1147      +/-   ##
==========================================
- Coverage   89.08%   85.55%   -3.53%     
==========================================
  Files         128      128              
  Lines        4891     4915      +24     
==========================================
- Hits         4357     4205     -152     
- Misses        534      710     +176
Impacted Files Coverage Δ
starfish/imagestack/imagestack.py 93.49% <100%> (+0.05%) ⬆️
starfish/imagestack/indexing_utils.py 94.59% <91.66%> (-5.41%) ⬇️
...rfish/test/full_pipelines/cli/test_dartfish_cli.py 38.88% <0%> (-61.12%) ⬇️
starfish/util/indirectfile/_imagestack.py 57.89% <0%> (-42.11%) ⬇️
starfish/test/full_pipelines/cli/_base_cli_test.py 52.63% <0%> (-31.58%) ⬇️
starfish/test/full_pipelines/cli/test_iss.py 70% <0%> (-30%) ⬇️
starfish/util/indirectfile/_codebook.py 60% <0%> (-26.67%) ⬇️
...sh/spots/_decoder/per_round_max_channel_decoder.py 76.92% <0%> (-23.08%) ⬇️
starfish/image/_segmentation/_base.py 74.28% <0%> (-22.86%) ⬇️
starfish/spacetx_format/cli.py 61.7% <0%> (-22.35%) ⬇️
... and 24 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 169fe89...1d22d23. Read the comment docs.

@codecov-io
Copy link

Codecov Report

Merging #1147 into master will increase coverage by 0.01%.
The diff coverage is 92.59%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #1147      +/-   ##
==========================================
+ Coverage   89.08%   89.09%   +0.01%     
==========================================
  Files         128      128              
  Lines        4891     4915      +24     
==========================================
+ Hits         4357     4379      +22     
- Misses        534      536       +2
Impacted Files Coverage Δ
starfish/imagestack/imagestack.py 93.49% <100%> (+0.05%) ⬆️
starfish/imagestack/indexing_utils.py 94.59% <91.66%> (-5.41%) ⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 169fe89...1d22d23. Read the comment docs.

Copy link
Collaborator

@ttung ttung left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suspect this will break for images with noncontiguous labels. This is because you're getting the index offsets for the slice operation, but you are using .sel. .sel operates on labels.

At the very least, we should add a test to verify that noncontiguous labels + sel_by_physical_coords works.

stack = deepcopy(self)
selector = indexing_utils.convert_to_selector(indexers)
stack._data._data = indexing_utils.index_keep_dimensions(self.xarray, selector)
return stack

def sel_by_physical_coords(
self, indexers: Mapping[Coordinates, Union[Number, Tuple[Number, Number]]]):
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you find yourself repeating yourself a ton about Union[Number, Tuple[Number, Number]], you can create a type alias like how we created a type alias for Number.


Parameters
----------
indexers : Dict[Axes, (float/tuple)]:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
indexers : Dict[Axes, (float/tuple)]:
indexers : Mapping[Coordinates, Union[Number, Tuple[Number, Number]]]:

----------
array: xr.DataArray
The xarray with both physical and positional coordinates.
indexers: Mapping[Coordinates, Union[int, tuple]]
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
indexers: Mapping[Coordinates, Union[int, tuple]]
indexers: Mapping[Coordinates, Union[Number, Tuple[Number, Number]]]

data: xr.DataArray, indexers: Mapping[str, Union[int, slice]]) -> xr.DataArray:
def convert_coords_to_indices(array: xr.DataArray,
indexers: Mapping[Coordinates, Union[Number, Tuple[Number, Number]]]
) -> Dict[Axes, Union[int, tuple]]:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
) -> Dict[Axes, Union[int, tuple]]:
) -> Mapping[Axes, Union[float, Tuple[Number, Number]]]:

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this method can't return a float since we're converting to positional indexers.


Returns
-------
Mapping[Axes, Union[int, tuple]]:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Mapping[Axes, Union[int, tuple]]:
Mapping[Axes, Union[float, Tuple[Number, Number]]]:


Parameters
----------
array: xr.DataArray
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: @ambrosejcarr always wrote the parameters docs with a space before and after the colon, so I just copied him.

starfish/imagestack/indexing_utils.py Show resolved Hide resolved
@shanaxel42
Copy link
Collaborator Author

I suspect this will break for images with noncontiguous labels. This is because you're getting the index offsets for the slice operation, but you are using .sel. .sel operates on labels.

At the very least, we should add a test to verify that noncontiguous labels + sel_by_physical_coords works.

Wait why? I think this might only happen in the Z scenario. But since x/y are unlabeled sel and isel are effectively the same

@ttung
Copy link
Collaborator

ttung commented Apr 12, 2019

Wait why? I think this might only happen in the Z scenario. But since x/y are unlabeled sel and isel are effectively the same

yes, I think you're right.

@shanaxel42 shanaxel42 requested a review from ttung April 15, 2019 16:37
Copy link
Collaborator

@ttung ttung left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

pls see comments, rest lgtm.


Parameters
----------
indexers : Dict[Axes, (int/tuple)]
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is inconsistent with the typing in your function declaration.

>>> stack = ImageStack.synthetic_stack(5, 5, 15, 200, 200)
>>> stack
<starfish.ImageStack (r: 5, c: 5, z: 15, y: 200, x: 200)>
>>> stack.sel({Axes.ROUND: (1, None), Axes.CH: 0, Axes.ZPLANE: 0})
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
>>> stack.sel({Axes.ROUND: (1, None), Axes.CH: 0, Axes.ZPLANE: 0})
>>> stack.isel({Axes.ROUND: (1, None), Axes.CH: 0, Axes.ZPLANE: 0})

<starfish.ImageStack (r: 5, c: 5, z: 15, y: 200, x: 200)>
>>> stack.sel({Axes.ROUND: (1, None), Axes.CH: 0, Axes.ZPLANE: 0})
<starfish.ImageStack (r: 4, c: 1, z: 1, y: 200, x: 200)>
>>> stack.sel({Axes.ROUND: 0, Axes.CH: 0, Axes.ZPLANE: 1,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
>>> stack.sel({Axes.ROUND: 0, Axes.CH: 0, Axes.ZPLANE: 1,
>>> stack.isel({Axes.ROUND: 0, Axes.CH: 0, Axes.ZPLANE: 1,

data: xr.DataArray, indexers: Mapping[str, Union[int, slice]]) -> xr.DataArray:
def convert_coords_to_indices(array: xr.DataArray,
indexers: Mapping[Coordinates, Union[Number, Tuple[Number, Number]]]
) -> Dict[Axes, Union[int, Tuple[Number, Number]]]:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
) -> Dict[Axes, Union[int, Tuple[Number, Number]]]:
) -> Mapping[Axes, Union[int, Tuple[Number, Number]]]:

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

out of curiosity why do you prefer Mapping to Dict? Mapping always give me a lot of lint troubles

array: xr.DataArray
The array to do lookups in.

value: Union[float, tuple]
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
value: Union[float, tuple]
value : Union[Number, Tuple[Number, Number]]


def find_nearest(array: xr.DataArray,
value: Union[Number, Tuple[Number, Number]]
) -> Union[int, Tuple[Number, Number]]:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
) -> Union[int, Tuple[Number, Number]]:
) -> Union[int, Tuple[int, int]]:


Returns
-------
Union[int, tuple]:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Union[int, tuple]:
Union[int, Tuple[int, int]]:

@shanaxel42 shanaxel42 merged commit 3f00fe0 into master Apr 17, 2019
@shanaxel42 shanaxel42 deleted the saxelrod-multiple-sel branch April 17, 2019 22:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants