Skip to content

Commit

Permalink
Avoid writing empty chunks
Browse files Browse the repository at this point in the history
  • Loading branch information
constantinpape committed Nov 13, 2020
1 parent c7b6782 commit 5cddf27
Showing 1 changed file with 6 additions and 1 deletion.
7 changes: 6 additions & 1 deletion data-conversion/to_ome_zarr.py
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,12 @@ def copy_dataset(ds_in, ds_out, n_threads):
n_blocks = len(blocks)

def _copy_chunk(block):
ds_out[block] = ds_in[block]
# make sure we don't copy empty blocks; I don't know
# if zarr makes sure not to write them out
data_in = ds_in[block]
if data_in.sum() == 0:
return
ds_out[block] = data_in

with futures.ThreadPoolExecutor(n_threads) as tp:
list(tqdm(tp.map(_copy_chunk, blocks), total=n_blocks))
Expand Down

0 comments on commit 5cddf27

Please sign in to comment.