Skip to content

Commit

Permalink
use store dimension separtor in DirectoryStore.listdir (#1335)
Browse files Browse the repository at this point in the history
* use store dimension separtor in DirectoryStore.listdir

NestedDirectoryStore which inherits from DirectoryStore
only supports '/' as a dimension separator. However
listdir uses the parent DirectoryStore.listdir which
produces keys with an incorrect separator '.'

Fixes #1334

* update release note
  • Loading branch information
braingram authored Jan 26, 2023
1 parent 0bf0b3b commit 6f11ae7
Show file tree
Hide file tree
Showing 3 changed files with 12 additions and 1 deletion.
2 changes: 2 additions & 0 deletions docs/release.rst
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,8 @@ Bug fixes

* Ensure contiguous data is give to ``FSStore``. Only copying if needed.
By :user:`Mads R. B. Kristensen <madsbk>` :issue:`1285`.
* NestedDirectoryStore.listdir now returns chunk keys with the correct '/' dimension_separator.
By :user:`Brett Graham <braingram>` :issue:`1334`.

.. _release_2.13.6:

Expand Down
4 changes: 3 additions & 1 deletion zarr/storage.py
Original file line number Diff line number Diff line change
Expand Up @@ -1204,7 +1204,9 @@ def _nested_listdir(self, path=None):
for file_name in file_names:
file_path = os.path.join(dir_path, file_name)
rel_path = file_path.split(root_path + os.path.sep)[1]
new_children.append(rel_path.replace(os.path.sep, '.'))
new_children.append(rel_path.replace(
os.path.sep,
self._dimension_separator or '.'))
else:
new_children.append(entry)
return sorted(new_children)
Expand Down
7 changes: 7 additions & 0 deletions zarr/tests/test_storage.py
Original file line number Diff line number Diff line change
Expand Up @@ -1442,6 +1442,13 @@ def test_chunk_nesting(self):
store[self.root + '42'] = b'zzz'
assert b'zzz' == store[self.root + '42']

def test_listdir(self):
store = self.create_store()
z = zarr.zeros((10, 10), chunks=(5, 5), store=store)
z[:] = 1 # write to all chunks
for k in store.listdir():
assert store.get(k) is not None


class TestNestedDirectoryStoreNone:

Expand Down

0 comments on commit 6f11ae7

Please sign in to comment.