Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Parse chunk shape to check for float values #2535

Merged
merged 4 commits into from
Dec 5, 2024
Merged

Parse chunk shape to check for float values #2535

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
ec79b87
Test that shapes and chunk shapes containing floats raise a TypeError
faymanns Dec 5, 2024
7c1e34c
Parse chunk shape
faymanns Dec 5, 2024
a1a3f48
Move check for floats from create to normalize_chunks
faymanns Dec 5, 2024
dfb4709
Merge branch 'main' into main
jhamman Dec 5, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions src/zarr/core/array.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -461,8 +461,11 @@ async def create(
raise ValueError("Only one of chunk_shape or chunks can be provided.")

if chunks:
chunks = parse_shapelike(chunks)
_chunks = normalize_chunks(chunks, shape, dtype_parsed.itemsize)
else:
if chunk_shape:
chunk_shape = parse_shapelike(chunk_shape)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suggest we move this check into normalize_chunks. Here's where the behavior you're looking to change is:

zarr-python/src/zarr/core/chunk_grids.py

Lines 116 to 117 in 2fe12a7

if isinstance(chunks, numbers.Integral):
chunks = tuple(int(chunks) for _ in shape)

If you look at normalize_chunks, you'll see why Zarr is more permissive on chunk shape than array shape. I'm not opposed to us disallowing floats but we need to be careful keep some of the other behaviors in normalize_chunks.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That seems like a better place indeed! I've updated the PR.

_chunks = normalize_chunks(chunk_shape, shape, dtype_parsed.itemsize)

result: AsyncArray[ArrayV3Metadata] | AsyncArray[ArrayV2Metadata]
Expand Down
8 changes: 8 additions & 0 deletions tests/test_api.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,14 @@ def test_create_array(memory_store: Store) -> None:
assert z.shape == (400,)
assert z.chunks == (40,)

# create array with float shape
with pytest.raises(TypeError):
z = create(shape=(400.5, 100), store=store, overwrite=True)

# create array with float chunk shape
with pytest.raises(TypeError):
z = create(shape=(400, 100), chunks=(16, 16.5), store=store, overwrite=True)


@pytest.mark.parametrize("path", ["foo", "/", "/foo", "///foo/bar"])
@pytest.mark.parametrize("node_type", ["array", "group"])
Expand Down
Loading