Reading nd datasets (#966)

* Started debuging for reading nd datasets.

* Implementation of VecNInt and NDBoundingBox.

* Rename VecInt and fix some issues.

* Work on import of existing zarr dataset.

* Add nd_bounding_box to properties of Layer.

* Update hooks in properties.py to make import of 4d zarr datasets possible.

* Add axis_order and index of additional axes to nd_bounding box creation.

* Working on reading nd data with pims images.

* Modify pims images to support more _iter_dims.

* Add method for expected bounding box instead of expected shape in pims images.

* Adding functions for editing ndboundingbox in 3d and update import from_images.

* Propagade nd-bounding box to different methods that take care of writing the dataset.

* Update object unstructuring for json and start implementing nd array access.

* Adapt array classes and add axes information to ArrayInfo.

* Updated zarr writing for nd arrays. Axes order still get mixed up between old BoundingBoxes and new NDBoundingBoxes.

* Adapted buffered_slice_reader and test behaviour with different tif images.

* Adding testdata and fix bugs with axes operation for writing zarr array.

* Fixing issues with axis order.

* Rewrite buffered_slice_writer and fix bugs to get old tests working.

* Fix chunking in buffered slice writer.

* Fix issues with old tests.

* Working on fixing all tests.

* Fixing issues with alignment, writing with offsets and different mags.

* Fix reading without parameters for nd datasets and addint test.

* Fix issue with empty additionalAxes in json and some minor issues.

* Move script reading_nd_data to examples in docs and implement some feedback.

* Do some formatting and implement requested changes.

* Update naming for accessing attributes of nd bounding boxes.

* Fix buffered_slice_writer for different axis and typechecking.

* Fix issue with ensure_size in initialization.

* Adapt pims_images to support xyz images with channels and timepoint.

* run formatter

* Fix issues with failed tests and add comments.

* Fix statement with wrong VecInt initialization.

* Insert previously deleted assertions.

* Add docstrings and use bounding box for read in nd case instead of VecInts.

* Implement requested changes.

* Changes init of NDBoundingBoxes.

* Add converter for VecInt attributes of nd bounding box to pass typechecks.

* Enhance documentation.

* Update Changelog.md

docs/mkdocs.yml

@@ -97,6 +97,7 @@ nav:
               - webknossos-py/examples/download_tiff_stack.md
               - webknossos-py/examples/remote_datasets.md
               - webknossos-py/examples/zarr_and_dask.md
+              - webknossos-py/examples/convert_4d_tiff.md
           - Annotation Examples:
               - webknossos-py/examples/apply_merger_mode.md
               - webknossos-py/examples/learned_segmenter.md
@@ -112,8 +113,10 @@ nav:
           - Overview: api/webknossos.md
           - Geometry:
               - BoundingBox: api/webknossos/geometry/bounding_box.md
+              - NDBoundingBox: api/webknossos/geometry/nd_bounding_box.md
               - Mag: api/webknossos/geometry/mag.md
               - Vec3Int: api/webknossos/geometry/vec3_int.md
+              - VecInt: api/webknossos/geometry/vec_int.md
           - Dataset:
               - Dataset: api/webknossos/dataset/dataset.md
               - Layer: api/webknossos/dataset/layer.md

docs/src/webknossos-py/examples/convert_4d_tiff.md

@@ -0,0 +1,13 @@
+# Convert 4D Tiff
+
+This example demonstrates the basic interactions with Datasets that have more than three dimensions. 
+
+In order to manipulate 4D data in WEBKNOSSOS, we first convert the 4D Tiff dataset into a Zarr3 dataset. This conversion is achieved using the [from_images method](../../api/webknossos/dataset/dataset.md#Dataset.from_images).
+
+Once the dataset is converted, we can access specific layers and views, [read data](../../api/webknossos/dataset/mag_view.md#MagView.read) from a defined bounding box, and [write data](../../api/webknossos/dataset/mag_view.md#MagView.write) to a different position within the dataset. The [NDBoundingBox](../../api/webknossos/geometry/nd_bounding_box.md#NDBoundingBox) is utilized to select a 4D region of the dataset.
+
+```python
+--8<--
+webknossos/examples/convert_4d_tiff.py
+--8<--
+```

webknossos/Changelog.md

@@ -21,6 +21,7 @@ For upgrade instructions, please check the respective _Breaking Changes_ section
 - The rules for naming the layers have been tightened to match the allowed layer names on webknossos. [#1016](https://github.com/scalableminds/webknossos-libs/pull/1016)
 - Replaced PyLint linter + black formatter with Ruff for development. [#1013](https://github.com/scalableminds/webknossos-libs/pull/1013)
 - The remote operations now use the WEBKNOSSOS API version 6. [#1018](https://github.com/scalableminds/webknossos-libs/pull/1018)
+- The conversion of 4D Tiff files to a Zarr3 Dataset is possible. NDBoundingBoxes and VecInt classes are introduced to support working with more than 3 dimensions. [#966](https://github.com/scalableminds/webknossos-libs/pull/966)
 
 ### Fixed
 

webknossos/examples/convert_4d_tiff.py

@@ -0,0 +1,48 @@
+from pathlib import Path
+
+import webknossos as wk
+
+
+def main() -> None:
+    # Create a WEBKNOSSOS dataset from a 4D tiff image
+    dataset = wk.Dataset.from_images(
+        Path(__file__).parent.parent / "testdata" / "4D" / "4D_series",
+        "testoutput/4D_series",
+        voxel_size=(10, 10, 10),
+        data_format="zarr3",
+        use_bioformats=True,
+    )
+
+    # Access the first color layer and the Mag 1 view of this layer
+    layer = dataset.get_color_layers()[0]
+    mag_view = layer.get_finest_mag()
+
+    # To get the bounding box of the dataset use layer.bounding_box
+    # -> NDBoundingBox(topleft=(0, 0, 0, 0), size=(7, 5, 167, 439), axes=('t', 'z', 'y', 'x'))
+
+    # Read all data of the dataset
+    data = mag_view.read()
+    # data.shape -> (1, 7, 5, 167, 439) # first value is the channel dimension
+
+    # Read data for a specific time point (t=3) of the dataset
+    data = mag_view.read(
+        absolute_bounding_box=layer.bounding_box.with_bounds("t", 3, 1)
+    )
+    # data.shape -> (1, 1, 5, 167, 439)
+
+    # Create a NDBoundingBox to read data from a specific region of the dataset
+    read_bbox = wk.NDBoundingBox(
+        topleft=(2, 0, 67, 39),
+        size=(2, 5, 100, 400),
+        axes=("t", "z", "y", "x"),
+        index=(1, 2, 3, 4),
+    )
+    data = mag_view.read(absolute_bounding_box=read_bbox)
+    # data.shape -> (1, 2, 5, 100, 400) # first value is the channel dimension
+
+    # Write some data to a given position
+    mag_view.write(data, absolute_bounding_box=read_bbox.offset((2, 0, 0, 0)))
+
+
+if __name__ == "__main__":
+    main()

webknossos/tests/dataset/test_add_layer_from_images.py

@@ -43,6 +43,31 @@ def test_compare_tifffile(tmp_path: Path) -> None:
         assert np.array_equal(data[:, :, z_index], comparison_slice)
 
 
+def test_compare_nd_tifffile(tmp_path: Path) -> None:
+    ds = wk.Dataset(tmp_path, (1, 1, 1))
+    layer = ds.add_layer_from_images(
+        "testdata/4D/4D_series/4D-series.ome.tif",
+        layer_name="color",
+        category="color",
+        topleft=(100, 100, 55),
+        use_bioformats=True,
+        data_format="zarr3",
+        chunk_shape=(8, 8, 8),
+        chunks_per_shard=(8, 8, 8),
+    )
+    assert layer.bounding_box.topleft == wk.VecInt(
+        0, 55, 100, 100, axes=("t", "z", "y", "x")
+    )
+    assert layer.bounding_box.size == wk.VecInt(
+        7, 5, 167, 439, axes=("t", "z", "y", "x")
+    )
+    read_with_tifffile_reader = TiffFile(
+        "testdata/4D/4D_series/4D-series.ome.tif"
+    ).asarray()
+    read_first_channel_from_dataset = layer.get_finest_mag().read()[0]
+    assert np.array_equal(read_with_tifffile_reader, read_first_channel_from_dataset)
+
+
 REPO_IMAGES_ARGS: List[
     Tuple[Union[str, List[Path]], Dict[str, Any], str, int, Tuple[int, int, int]]
 ] = [
@@ -205,15 +230,6 @@ def download_and_unpack(
         (192, 128, 9),
         1,
     ),
-    (
-        "https://samples.scif.io/sdub.zip",
-        "sdub*.pic",
-        {"allow_multiple_layers": True},
-        "uint8",
-        1,
-        (192, 128, 9),
-        12,
-    ),
     (
         "https://samples.scif.io/test-avi.zip",
         "t1-rendering.avi",

webknossos/webknossos/_nml/parameters.py

@@ -3,7 +3,7 @@
 
 from loxun import XmlWriter
 
-from ..geometry import BoundingBox
+from ..geometry import BoundingBox, NDBoundingBox
 from ..geometry.bounding_box import _DEFAULT_BBOX_NAME
 from .utils import Vector3, enforce_not_null, filter_none_values
 
@@ -22,22 +22,24 @@ class Parameters(NamedTuple):
     editPosition: Optional[Vector3] = None
     editRotation: Optional[Vector3] = None
     zoomLevel: Optional[float] = None
-    taskBoundingBox: Optional[BoundingBox] = None
-    userBoundingBoxes: Optional[List[BoundingBox]] = None
+    taskBoundingBox: Optional[NDBoundingBox] = None
+    userBoundingBoxes: Optional[List[NDBoundingBox]] = None
 
     def _dump_bounding_box(
         self,
         xf: XmlWriter,
-        bounding_box: BoundingBox,
+        bounding_box: NDBoundingBox,
         tag_name: str,
         bbox_id: Optional[int],  # user bounding boxes need an id
     ) -> None:
         color = bounding_box.color or DEFAULT_BOUNDING_BOX_COLOR
 
         attributes = {
-            "name": _DEFAULT_BBOX_NAME
-            if bounding_box.name is None
-            else str(bounding_box.name),
+            "name": (
+                _DEFAULT_BBOX_NAME
+                if bounding_box.name is None
+                else str(bounding_box.name)
+            ),
             "isVisible": "true" if bounding_box.is_visible else "false",
             "color.r": str(color[0]),
             "color.g": str(color[1]),
@@ -136,7 +138,7 @@ def _dump(self, xf: XmlWriter) -> None:
         xf.endTag()  # parameters
 
     @classmethod
-    def _parse_bounding_box(cls, bounding_box_element: Element) -> BoundingBox:
+    def _parse_bounding_box(cls, bounding_box_element: Element) -> NDBoundingBox:
         topleft = (
             int(bounding_box_element.get("topLeftX", 0)),
             int(bounding_box_element.get("topLeftY", 0)),
@@ -165,14 +167,16 @@ def _parse_bounding_box(cls, bounding_box_element: Element) -> BoundingBox:
         )
 
     @classmethod
-    def _parse_user_bounding_boxes(cls, nml_parameters: Element) -> List[BoundingBox]:
+    def _parse_user_bounding_boxes(cls, nml_parameters: Element) -> List[NDBoundingBox]:
         if nml_parameters.find("userBoundingBox") is None:
             return []
         bb_elements = nml_parameters.findall("userBoundingBox")
         return [cls._parse_bounding_box(bb_element) for bb_element in bb_elements]
 
     @classmethod
-    def _parse_task_bounding_box(cls, nml_parameters: Element) -> Optional[BoundingBox]:
+    def _parse_task_bounding_box(
+        cls, nml_parameters: Element
+    ) -> Optional[NDBoundingBox]:
         bb_element = nml_parameters.find("taskBoundingBox")
         if bb_element is not None:
             return cls._parse_bounding_box(bb_element)

webknossos/webknossos/annotation/annotation.py

@@ -60,7 +60,7 @@
 )
 from ..dataset.defaults import PROPERTIES_FILE_NAME
 from ..dataset.properties import DatasetProperties, dataset_converter
-from ..geometry import BoundingBox, Vec3Int
+from ..geometry import NDBoundingBox, Vec3Int
 from ..skeleton import Skeleton
 from ..utils import time_since_epoch_in_ms, warn_deprecated
 from ._nml_conversion import annotation_to_nml, nml_to_skeleton
@@ -124,8 +124,8 @@ class Annotation:
     edit_rotation: Optional[Vector3] = None
     zoom_level: Optional[float] = None
     metadata: Dict[str, str] = attr.Factory(dict)
-    task_bounding_box: Optional[BoundingBox] = None
-    user_bounding_boxes: List[BoundingBox] = attr.Factory(list)
+    task_bounding_box: Optional[NDBoundingBox] = None
+    user_bounding_boxes: List[NDBoundingBox] = attr.Factory(list)
     _volume_layers: List[_VolumeLayer] = attr.field(factory=list, init=False)
 
     @classmethod
@@ -474,7 +474,7 @@ def _load_from_zip(cls, content: Union[str, PathLike, BinaryIO]) -> "Annotation"
         assert len(nml_paths) > 0, "Couldn't find an nml file in the supplied zip-file."
         assert (
             len(nml_paths) == 1
-        ), f"There must be exactly one nml file in the zip-file, buf found {len(nml_paths)}."
+        ), f"There must be exactly one nml file in the zip-file, but found {len(nml_paths)}."
         with nml_paths[0].open(mode="rb") as f:
             return cls._load_from_nml(nml_paths[0].stem, f, possible_volume_paths=paths)
 

webknossos/webknossos/cli/convert_knossos.py

@@ -151,15 +151,15 @@ def convert_cube_job(
     time_start(f"Converting of {target_view.bounding_box}")
     cube_size = cast(Tuple[int, int, int], (KNOSSOS_CUBE_EDGE_LEN,) * 3)
 
-    offset = target_view.bounding_box.in_mag(target_view.mag).topleft
-    size = target_view.bounding_box.in_mag(target_view.mag).size
+    offset = target_view.bounding_box.in_mag(target_view.mag).topleft_xyz
+    size = target_view.bounding_box.in_mag(target_view.mag).size_xyz
     buffer = np.zeros(size.to_tuple(), dtype=target_view.get_dtype())
     with open_knossos(source_knossos_info) as source_knossos:
         for x in range(0, size.x, KNOSSOS_CUBE_EDGE_LEN):
             for y in range(0, size.y, KNOSSOS_CUBE_EDGE_LEN):
                 for z in range(0, size.z, KNOSSOS_CUBE_EDGE_LEN):
                     cube_data = source_knossos.read(
-                        (offset + Vec3Int(x, y, z)).to_tuple(), cube_size
+                        Vec3Int(offset + (x, y, z)).to_tuple(), cube_size
                     )
                     buffer[
                         x : (x + KNOSSOS_CUBE_EDGE_LEN),

webknossos/webknossos/cli/export_wkw_as_tiff.py

@@ -262,7 +262,7 @@ def main(
 
     mag_view = Dataset.open(source).get_layer(layer_name).get_mag(mag)
 
-    bbox = mag_view.bounding_box if bbox is None else bbox
+    bbox = BoundingBox.from_ndbbox(mag_view.bounding_box) if bbox is None else bbox
 
     logging.info("Starting tiff export for bounding box: %s", bbox)
     executor_args = Namespace(