Code replication errors in a different machine - ValueError: cannot reshape array of size 532480 into shape (64,64,128)

[Mohinta2892]

Error

Traceback (most recent call last):
  File "/home/tom/.local/lib/python3.10/site-packages/gunpowder/nodes/batch_provider.py", line 193, in request_batch
    batch = self.provide(upstream_request)
  File "/home/tom/.local/lib/python3.10/site-packages/gunpowder/nodes/zarr_source.py", line 151, in provide
    self.__read(data_file, self.datasets[array_key], dataset_roi),
  File "/home/tom/.local/lib/python3.10/site-packages/gunpowder/nodes/zarr_source.py", line 231, in __read
    array = np.asarray(data_file[ds_name][(slice(None),) * c + roi.to_slices()])
  File "/home/tom/.local/lib/python3.10/site-packages/zarr/core.py", line 798, in __getitem__
    result = self.get_orthogonal_selection(pure_selection, fields=fields)
  File "/home/tom/.local/lib/python3.10/site-packages/zarr/core.py", line 1080, in get_orthogonal_selection
    return self._get_selection(indexer=indexer, out=out, fields=fields)
  File "/home/tom/.local/lib/python3.10/site-packages/zarr/core.py", line 1343, in _get_selection
    self._chunk_getitems(
  File "/home/tom/.local/lib/python3.10/site-packages/zarr/core.py", line 2183, in _chunk_getitems
    self._process_chunk(
  File "/home/tom/.local/lib/python3.10/site-packages/zarr/core.py", line 2096, in _process_chunk
    chunk = self._decode_chunk(cdata)
  File "/home/tom/.local/lib/python3.10/site-packages/zarr/core.py", line 2377, in _decode_chunk
    chunk = chunk.reshape(expected_shape or self._chunks, order=self._order)
ValueError: cannot reshape array of size 532480 into shape (64,64,128)

Exception in pipeline:
ZarrSource[/home/tom/Documents/Internship/Samia/catena/data/TOM/data_3d/test/samia.zarr] -> ZarrWrite -> Normalize -> IntensityScaleShift -> Unsqueeze -> Stack -> Predict -> Squeeze -> Squeeze -> ZarrWrite -> Scan
while trying to process request

Solution:

Your zarr has been constructed incorrectly. We will add a test_zarr.py soon, as a preprocessing step.
You must ensure that the data is 3D in (z,y,x) and the chunk size of the input data is >= the input_shape you provide in the config files. This is because when reading data from zarrs they are read as chunks. If data is already == input_shape, reading data in should be faster. However, having chunks that are massive in size also beats this point. As then multiple workers might be competing to read from the same chunk or read partial data. You can use always use smaller chunks (64,64,64) but ensure that the input_shape/input_size remains a 3D multiple of the chunk_size and the voxel_size.