-
-
Notifications
You must be signed in to change notification settings - Fork 1.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Handle empty containers in zarr chunk checks #5526
Conversation
@jhamman Ready for review, though not urgent. Mostly, I'm curious if this the right way to go about solving the linked issue, or if there is something deeper in xarray to update. |
Thanks @chrisroat for the PR. This looks great. I do think we should add an entry in |
In my original PR, I wrote:
I just want to be sure we think this is the correct fix, and there won't be any unintended consequences. I'm really not familiar with the interaction of chunks between zarr, dask, and xarray. I don't feel 100% sure this is the correct solution, though it fixes the immediate solution. |
Thanks @chrisroat - let's see if @rabernat or @shoyer has an opinion here then. |
Is this change still of potential value? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@chrisroat - thanks so much for your contribution! Sorry that I never reviewed this PR until now.
LGTM.
@@ -93,7 +93,7 @@ def _determine_zarr_chunks(enc_chunks, var_chunks, ndim, name, safe_chunks): | |||
|
|||
# if there are no chunks in encoding and the variable data is a numpy | |||
# array, then we let zarr use its own heuristics to pick the chunks | |||
if var_chunks is None and enc_chunks is None: | |||
if not var_chunks and not enc_chunks: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just for my understanding, could you explain the difference in behavior between var_chunks is None
and not var_chunks
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I believe I hit this when var_chunks was an empty list instead of None
:
>>> var_chunks=[]
>>> not var_chunks
True
>>> var_chunks is None
False
>>>
I do not know whey the use case I mention above uses an empty list (while the code was designed to expect None). I am not sure if there would be undesired knock-on effects of this change. Thoughts?
I can still add a whats-new if that is desired.
Thanks @chrisroat Sorry this took so long to get merged |
pre-commit run --all-files
User visible changes (including notable bug fixes) are documented inwhats-new.rst
New functions/methods are listed inapi.rst
Continuation of #5019, which closed during the master/main switch.