feat(ngff zarr): add multi channel support #480

PaulHax · 2022-05-04T22:07:56Z

Somehow these hacks work for all the datasets I've loaded except for the 2 lowest resolution scales in oeway's astronaut where the chunks span components.

thewtex · 2022-05-05T01:17:49Z

const chunkSize = toDimensionArray(['c', 'x', 'y', 'z'], info.chunkSize)

Instead of ['c', 'x', 'y', 'z'], this should be info.dims now, correct?

PaulHax · 2022-05-06T00:37:16Z

Works on oeway's astronaut now. Some fishy stuff with the x chunk index to shift down x rows:

pixelArray[
                cc +
                  zz * pixelStrides.z +
                  (yy + x) * pixelStrides.y +
                  xx * pixelStrides.x
              ] =

oeway · 2022-05-06T17:53:47Z

Thanks for working on this! Looking forward it! FYI: I am currently making a browser file system that can handle large files backed by indexeddb. Would be nice to test using pyodide/jupyterlite + zarr-python to read in-browser large files, then pipe the data as a zarr store to itk-vtk-viewer!

thewtex · 2022-05-06T22:54:33Z

src/IO/ImageDataFromChunks.worker.js

+    }
+
+    const chunkSize = Object.fromEntries(info.chunkSize)
+    const chunkStrides = {


The chunk strides for each dimension will be based on the input MultiscaleSpatialImage dims and their order: maybe a slice here on the dims array starting on the dim index and a reduce based on the chunkSize.

thewtex

🦖 🌮 🎉

Looks awesome!

Steps for optimization noted inline that I think we should add before merging.

thewtex · 2022-05-10T13:37:10Z

src/IO/ImageDataFromChunks.worker.js

+      for (let cc = itStart.c; cc < itEnd.c; cc++) {
+        for (let zz = itStart.z; zz < itEnd.z; zz++) {
+          for (let yy = itStart.y; yy < itEnd.y; yy++) {
+            for (let xx = itStart.x; xx < itEnd.x; xx++) {
+              pixelArray[
+                cc +
+                  xx * pixelStrides.x +
+                  yy * pixelStrides.y +
+                  zz * pixelStrides.z
+              ] =
+                chunk[
+                  // subtract x * chunkSize.x from xx to start at beginning of chunk despite itStart.x
+                  (xx - x * chunkSize.x) * chunkStrides.x +
+                    (yy - y * chunkSize.y) * chunkStrides.y +
+                    (zz - z * chunkSize.z) * chunkStrides.z +
+                    (cc - c * chunkSize.c) * chunkStrides.c
+                ]
+            } // column
+          } // row
+        } // slice
+      } // component
+    } // chunk


Optimizations to add to this critical piece:

[ ] 1. Three versions of the loop based on the dim order. a) tczyx, b) tzyxc c) other. a) for most OME-NGFF, b for the interleaved channel c) in the wild, e.g. it is also common to have tzcyz. c) can be like it is now. a) and b) will further be optimized in the following steps.

[ ] 2. For the conditions where we can use them in the inner-most loop, use .subarray and .set for getting and setting the contiguous region.

[ ] 3. Cache the offsets in the intermediate loops.

Added caching of offsets. Do we have any example images of non tczyx dim ordering we would like to support? I have not tested any...

Also, webpack dev server dishes out test/data/input

thewtex · 2022-05-12T16:11:26Z

Looking good. Can we base and add tests for the new datasets?

PaulHax · 2022-05-12T22:21:11Z

Looking good. Can we base and add tests for the new datasets?

Added tests for a selection of the working images.

Some issues:

All the first_instar_brains: .zattrs 'axes' metadata order does not match the array's .zattrs "_ARRAY_DIMENSIONS". I'll change code to use _ARRAY_DIMENSIONS.
first_instar_brain_czyx.zarr/.zattrs file is empty {}
Most of the idr datasets have their dtype endianness the way we don't support, >u2 and >f4. Not sure what to do there.

thewtex · 2022-05-13T17:07:34Z

For the first two, we should fix the data generator. Could you please create issues on the multi-scale-spatial-image repository, and assign it to me?

For the endianness issue, we should transform an chunk array from big to little before loading into the itk.Image. Could you please look for a JS library to do that and apply when required?

Chrome --no-sandbox Thread --dockered flag through Karma to toggle some tests

thewtex

Looking good.

thewtex · 2022-05-15T19:54:05Z

src/IO/dtypeUtils.js

+  ['f8', 'getFloat64'],
+])
+
+export const ElementGetter = (dtype, buffer) => {


For efficiency, let's have this operate over the chunk TypedArray instead of the element

Where to do this optimization? Current in worker flow: Dataview(chunkBuffer).getX(index, endiness) -> IJKTypedArray.

PaulHax · 2022-05-16T21:17:25Z

[ ] 2. For the conditions where we can use them in the inner-most loop, use .subarray and .set for getting and setting the contiguous region.

Implemented the above with guidance from the old code. Good speedup with first_instar_brain_zyxc.zarr highest resolution taking the worker ~11ms vs old way of ~26ms.

thewtex

Turbo-tacular!~ 🚀 👨‍🎤

Please take a look at a comment inline, follow-up as needed, and merge this puppy. 🥺

thewtex · 2022-05-17T02:16:40Z

src/IO/ZarrMultiscaleSpatialImage.js

  dataset,
 }) => {
  const dims =
-    pixelArrayAttrs?._ARRAY_DIMENSIONS ?? // xarray


Should we just change the order here -- first try / trust axes, then _ARRAY_DIMENSIONS?

Seems that many NGFFs don't have a foo.zarr/s0/.zattrs where _ARRAY_DIMENSIONS lives. To avoid 404s the code is not fetching the scale level .zattrs, so no _ARRAY_DIMENSIONS to fall back to.

oeway · 2022-05-18T13:54:04Z

Hi @PaulHax @thewtex Awesome to see this PR get merged! Well done, can't wait to try it.

It looks like the master branch CI is failing and that it explains why I cannot already show the multi-channel zarr images.

github-actions · 2022-05-18T19:41:17Z

🎉 This PR is included in version 12.3.0 🎉

The release is available on:

Your semantic-release bot 📦🚀

thewtex · 2022-05-18T19:50:17Z

Hi @oeway CI was given some ❤️ , v12.3.0 is now out with these improvements, please give it a whirl!

PaulHax requested a review from thewtex May 4, 2022 22:07

PaulHax force-pushed the zarr-channels branch from e983dd5 to 3bd4576 Compare May 6, 2022 00:32

PaulHax force-pushed the zarr-channels branch 2 times, most recently from 4a11f39 to 54457d8 Compare May 6, 2022 16:43

thewtex reviewed May 6, 2022

View reviewed changes

PaulHax force-pushed the zarr-channels branch 2 times, most recently from ced60eb to 0822171 Compare May 9, 2022 13:16

PaulHax marked this pull request as ready for review May 9, 2022 13:16

thewtex requested changes May 10, 2022

View reviewed changes

PaulHax force-pushed the zarr-channels branch from 4a1ec1e to e70b2b4 Compare May 11, 2022 21:29

PaulHax added 6 commits May 12, 2022 11:14

feat(ngff zarr): add multi channel support

17f9f40

feat(ngff zarr): better multichannel support

2dc2e4d

fix(MultiscaleSpatialImage): add chunk buffers to worker transferables

a3f9641

perf(ImageDataFromChunks): cache offsets in intermediate loops

8714a3e

feat(ngff zarr): some support for arbitrary dimension orders

ec4411a

fix(ngff zarr): allow for relative to document paths

371d170

Also, webpack dev server dishes out test/data/input

PaulHax force-pushed the zarr-channels branch from 9dfbfe5 to 371d170 Compare May 12, 2022 17:03

chore(zarr tests): add tests for de-chunked zarr images

d3a6ad6

PaulHax force-pushed the zarr-channels branch 2 times, most recently from 152d4fe to 761e8b7 Compare May 13, 2022 16:48

feat(ngff zarr): prefer _ARRAY_DIMENSIONS over axes. Test changes.

0d23145

Chrome --no-sandbox Thread --dockered flag through Karma to toggle some tests

PaulHax force-pushed the zarr-channels branch from 761e8b7 to 0d23145 Compare May 13, 2022 19:52

feat(ngff zarr): support big endian dtypes

fb4f9fc

PaulHax mentioned this pull request May 13, 2022

NGFF axes order and _ARRAY_DIMENSIONS order don't match spatial-image/multiscale-spatial-image#41

Open

thewtex requested changes May 15, 2022

View reviewed changes

PaulHax force-pushed the zarr-channels branch 3 times, most recently from b10d70c to 0e2b82d Compare May 16, 2022 15:53

fix(ngff zarr): only use axes metadata, no _ARRAY_DIMENSIONS

fbcf4a4

PaulHax force-pushed the zarr-channels branch from 0e2b82d to ad8b0b4 Compare May 16, 2022 21:08

thewtex approved these changes May 17, 2022

View reviewed changes

PaulHax force-pushed the zarr-channels branch from ad8b0b4 to 9f644dd Compare May 17, 2022 12:27

perf(ngff zarr): input data interleaves components then copy by row

1c99a87

PaulHax force-pushed the zarr-channels branch from 9f644dd to 1c99a87 Compare May 17, 2022 12:33

fix(ngff zarr): don't assume Uint8 in row copy mode

f80bb83

thewtex merged commit aef6541 into Kitware:master May 17, 2022

github-actions bot added the released label May 18, 2022

thewtex mentioned this pull request May 18, 2022

Add ome-zarr support #315

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(ngff zarr): add multi channel support #480

feat(ngff zarr): add multi channel support #480

PaulHax commented May 4, 2022

thewtex commented May 5, 2022

PaulHax commented May 6, 2022

oeway commented May 6, 2022

thewtex May 6, 2022

thewtex left a comment

thewtex May 10, 2022 •

edited

Loading

PaulHax May 10, 2022

thewtex commented May 12, 2022

PaulHax commented May 12, 2022

thewtex commented May 13, 2022

thewtex left a comment

thewtex May 15, 2022

PaulHax May 16, 2022

PaulHax commented May 16, 2022

thewtex left a comment

thewtex May 17, 2022

PaulHax May 17, 2022

oeway commented May 18, 2022

github-actions bot commented May 18, 2022

thewtex commented May 18, 2022

feat(ngff zarr): add multi channel support #480

feat(ngff zarr): add multi channel support #480

Conversation

PaulHax commented May 4, 2022

thewtex commented May 5, 2022

PaulHax commented May 6, 2022

oeway commented May 6, 2022

thewtex May 6, 2022

Choose a reason for hiding this comment

thewtex left a comment

Choose a reason for hiding this comment

thewtex May 10, 2022 • edited Loading

Choose a reason for hiding this comment

PaulHax May 10, 2022

Choose a reason for hiding this comment

thewtex commented May 12, 2022

PaulHax commented May 12, 2022

thewtex commented May 13, 2022

thewtex left a comment

Choose a reason for hiding this comment

thewtex May 15, 2022

Choose a reason for hiding this comment

PaulHax May 16, 2022

Choose a reason for hiding this comment

PaulHax commented May 16, 2022

thewtex left a comment

Choose a reason for hiding this comment

thewtex May 17, 2022

Choose a reason for hiding this comment

PaulHax May 17, 2022

Choose a reason for hiding this comment

oeway commented May 18, 2022

github-actions bot commented May 18, 2022

thewtex commented May 18, 2022

thewtex May 10, 2022 •

edited

Loading