Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cache pre-existing Zarr arrays in Zarr backend #9861

Merged
merged 48 commits into from
Dec 18, 2024
Merged
Changes from 1 commit
Commits
Show all changes
48 commits
Select commit Hold shift + click to select a range
9503c71
cache array keys on ZarrStore instances
d-v-b Dec 4, 2024
94b48ac
refactor get_array_keys
d-v-b Dec 6, 2024
2dacb2a
add test for get_array_keys
d-v-b Dec 6, 2024
c7cfacf
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Dec 6, 2024
b3cf2e5
lint
d-v-b Dec 6, 2024
d9bd7fd
Merge branch 'perf/cache-array-keys' of https://github.com/d-v-b/xarr…
d-v-b Dec 6, 2024
8d47dd3
Merge branch 'main' of https://www.github.com/pydata/xarray into perf…
d-v-b Dec 6, 2024
bec8b42
add type annotation for _cache
d-v-b Dec 6, 2024
61fe759
make type hint accurate
d-v-b Dec 6, 2024
0952c93
make type hint pass mypy
d-v-b Dec 6, 2024
96ddcf4
make type hint pass mypy, correctly
d-v-b Dec 6, 2024
60761a8
Update xarray/backends/zarr.py
d-v-b Dec 9, 2024
f82659d
Merge branch 'main' of https://github.com/pydata/xarray into perf/cac…
d-v-b Dec 10, 2024
107c4e7
use roundtrip method instead of explicit alternative
d-v-b Dec 10, 2024
2d3c13a
cache members instead of just array keys
d-v-b Dec 10, 2024
be5e389
adjust test to members caching
d-v-b Dec 10, 2024
1eaf3ea
Merge branch 'perf/cache-array-keys' of https://github.com/d-v-b/xarr…
d-v-b Dec 10, 2024
9d147b9
refactor members cache
d-v-b Dec 10, 2024
bdb20a6
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Dec 10, 2024
6578caa
explictly revert changes to test_write_store
d-v-b Dec 10, 2024
9596bcf
Merge branch 'perf/cache-array-keys' of https://github.com/d-v-b/xarr…
d-v-b Dec 10, 2024
df4274f
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Dec 10, 2024
5bf7aff
indent correctly
d-v-b Dec 10, 2024
df51855
Merge branch 'perf/cache-array-keys' of https://github.com/d-v-b/xarr…
d-v-b Dec 10, 2024
5a818a5
update instrumented tests, remove cache update functionality, and set…
d-v-b Dec 10, 2024
c18f81b
update instrumented tests, remove cache update functionality, and set…
d-v-b Dec 10, 2024
7b13099
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Dec 10, 2024
8925020
lint
d-v-b Dec 10, 2024
5efbe30
Merge branch 'perf/cache-array-keys' of https://github.com/d-v-b/xarr…
d-v-b Dec 10, 2024
192fc8b
update tests
d-v-b Dec 10, 2024
2ce5b2e
make check_requests more permissive
d-v-b Dec 10, 2024
87a40c7
update instrumented tests for zarr==2.18
d-v-b Dec 10, 2024
7e79b79
Merge branch 'main' into perf/cache-array-keys
dcherian Dec 11, 2024
d987529
Update xarray/backends/zarr.py
d-v-b Dec 14, 2024
169b736
Update xarray/backends/zarr.py
d-v-b Dec 14, 2024
e20d11d
Update xarray/backends/zarr.py
d-v-b Dec 14, 2024
a1c25f2
Update xarray/backends/zarr.py
d-v-b Dec 14, 2024
f8d78cb
Update xarray/backends/zarr.py
d-v-b Dec 16, 2024
650937e
doc: add whats new content
d-v-b Dec 17, 2024
fcf2d64
fixup
d-v-b Dec 17, 2024
bf64fd2
Merge branch 'main' of https://www.github.com/pydata/xarray into perf…
d-v-b Dec 17, 2024
dcecf43
update whats new
d-v-b Dec 17, 2024
d0bb5a2
fixup
d-v-b Dec 17, 2024
65cadcb
remove vestigial cache_array_keys
d-v-b Dec 17, 2024
e8cd3c8
make cache_members default to True
d-v-b Dec 17, 2024
8d31f0e
tweak
dcherian Dec 17, 2024
3d15d3d
Remove from public API
dcherian Dec 17, 2024
f1ef905
Merge branch 'main' into perf/cache-array-keys
dcherian Dec 18, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
update whats new
d-v-b committed Dec 17, 2024
commit dcecf43167e1a7af03d89f1e930799957dcf0c98
2 changes: 1 addition & 1 deletion doc/whats-new.rst
Original file line number Diff line number Diff line change
@@ -24,14 +24,14 @@ New Features
- Better support wrapping additional array types (e.g. ``cupy`` or ``jax``) by calling generalized
duck array operations throughout more xarray methods. (:issue:`7848`, :pull:`9798`).
By `Sam Levang <https://github.com/slevang>`_.

- Better performance for reading Zarr arrays in the ``ZarrStore`` class by caching the state of Zarr
storage and avoiding redundant IO operations. Usage of the cache can be controlled via the
``cache_members`` parameter to ``ZarrStore``. When ``cache_members`` is ``True`` (the default), the
``ZarrStore`` stores a snapshot of names and metadata of the in-scope Zarr arrays; this cache
is then used when iterating over those Zarr arrays, which avoids IO operations and thereby reduces
latency. (:issue:`9853`, :pull:`9861`). By `Davis Bennett <https://github.com/d-v-b>`_.

new parameter, ``cache_members``.
- Add ``unit`` - keyword argument to :py:func:`date_range` and ``microsecond`` parsing to
iso8601-parser (:pull:`9885`).
By `Kai Mühlbauer <https://github.com/kmuehlbauer>`_.

Unchanged files with check annotations Beta

self.save(
expected, store_target=store, consolidated=False, **self.version_kwargs
)
with pytest.warns(

Check failure on line 2331 in xarray/tests/test_backends.py

GitHub Actions / macos-latest py3.10

TestZarrWriteEmpty.test_read_non_consolidated_warning[2] Failed: DID NOT WARN. No warnings of type (<class 'RuntimeWarning'>,) were emitted. Emitted warnings: [].

Check failure on line 2331 in xarray/tests/test_backends.py

GitHub Actions / ubuntu-latest py3.10 min-all-deps

TestZarrDictStore.test_read_non_consolidated_warning[2] Failed: DID NOT WARN. No warnings of type (<class 'RuntimeWarning'>,) were emitted. Emitted warnings: [].

Check failure on line 2331 in xarray/tests/test_backends.py

GitHub Actions / macos-latest py3.12

TestZarrWriteEmpty.test_read_non_consolidated_warning[2] Failed: DID NOT WARN. No warnings of type (<class 'RuntimeWarning'>,) were emitted. Emitted warnings: [].

Check failure on line 2331 in xarray/tests/test_backends.py

GitHub Actions / ubuntu-latest py3.11 all-but-dask

TestZarrDictStore.test_read_non_consolidated_warning[2] Failed: DID NOT WARN. No warnings of type (<class 'RuntimeWarning'>,) were emitted. Emitted warnings: [].

Check failure on line 2331 in xarray/tests/test_backends.py

GitHub Actions / ubuntu-latest py3.12 all-but-numba

TestZarrWriteEmpty.test_read_non_consolidated_warning[2] Failed: DID NOT WARN. No warnings of type (<class 'RuntimeWarning'>,) were emitted. Emitted warnings: [].

Check failure on line 2331 in xarray/tests/test_backends.py

GitHub Actions / ubuntu-latest py3.10

TestZarrWriteEmpty.test_read_non_consolidated_warning[2] Failed: DID NOT WARN. No warnings of type (<class 'RuntimeWarning'>,) were emitted. Emitted warnings: [].

Check failure on line 2331 in xarray/tests/test_backends.py

GitHub Actions / ubuntu-latest py3.12

TestZarrWriteEmpty.test_read_non_consolidated_warning[2] Failed: DID NOT WARN. No warnings of type (<class 'RuntimeWarning'>,) were emitted. Emitted warnings: [].
RuntimeWarning,
match="Failed to open Zarr store with consolidated",
):
with pytest.raises(
FileNotFoundError, match="(No such file or directory|Unable to find group)"
):
xr.open_zarr(f"{uuid.uuid4()}")

Check failure on line 2342 in xarray/tests/test_backends.py

GitHub Actions / macos-latest py3.10

TestZarrWriteEmpty.test_non_existent_store[2] TypeError: ZarrBackendEntrypoint.open_dataset() got an unexpected keyword argument 'cache_members'

Check failure on line 2342 in xarray/tests/test_backends.py

GitHub Actions / ubuntu-latest py3.10 min-all-deps

TestZarrDictStore.test_non_existent_store[2] TypeError: ZarrBackendEntrypoint.open_dataset() got an unexpected keyword argument 'cache_members'

Check failure on line 2342 in xarray/tests/test_backends.py

GitHub Actions / macos-latest py3.12

TestZarrWriteEmpty.test_non_existent_store[2] TypeError: ZarrBackendEntrypoint.open_dataset() got an unexpected keyword argument 'cache_members'

Check failure on line 2342 in xarray/tests/test_backends.py

GitHub Actions / ubuntu-latest py3.11 all-but-dask

TestZarrDictStore.test_non_existent_store[2] TypeError: ZarrBackendEntrypoint.open_dataset() got an unexpected keyword argument 'cache_members'

Check failure on line 2342 in xarray/tests/test_backends.py

GitHub Actions / ubuntu-latest py3.12 all-but-numba

TestZarrWriteEmpty.test_non_existent_store[2] TypeError: ZarrBackendEntrypoint.open_dataset() got an unexpected keyword argument 'cache_members'

Check failure on line 2342 in xarray/tests/test_backends.py

GitHub Actions / ubuntu-latest py3.10

TestZarrWriteEmpty.test_non_existent_store[2] TypeError: ZarrBackendEntrypoint.open_dataset() got an unexpected keyword argument 'cache_members'

Check failure on line 2342 in xarray/tests/test_backends.py

GitHub Actions / ubuntu-latest py3.12

TestZarrWriteEmpty.test_non_existent_store[2] TypeError: ZarrBackendEntrypoint.open_dataset() got an unexpected keyword argument 'cache_members'
@pytest.mark.skipif(has_zarr_v3, reason="chunk_store not implemented in zarr v3")
def test_with_chunkstore(self) -> None:
**self.version_kwargs,
)
if compute:
with xr.open_zarr(

Check failure on line 3015 in xarray/tests/test_backends.py

GitHub Actions / macos-latest py3.10

TestZarrWriteEmpty.test_write_region[2-False-False-True-False] TypeError: ZarrBackendEntrypoint.open_dataset() got an unexpected keyword argument 'cache_members'

Check failure on line 3015 in xarray/tests/test_backends.py

GitHub Actions / macos-latest py3.10

TestZarrWriteEmpty.test_write_region[2-False-False-True-True] TypeError: ZarrBackendEntrypoint.open_dataset() got an unexpected keyword argument 'cache_members'

Check failure on line 3015 in xarray/tests/test_backends.py

GitHub Actions / macos-latest py3.10

TestZarrWriteEmpty.test_write_region[2-False-False-True-None] TypeError: ZarrBackendEntrypoint.open_dataset() got an unexpected keyword argument 'cache_members'

Check failure on line 3015 in xarray/tests/test_backends.py

GitHub Actions / macos-latest py3.12

TestZarrWriteEmpty.test_write_region[2-False-False-True-False] TypeError: ZarrBackendEntrypoint.open_dataset() got an unexpected keyword argument 'cache_members'

Check failure on line 3015 in xarray/tests/test_backends.py

GitHub Actions / macos-latest py3.12

TestZarrWriteEmpty.test_write_region[2-False-False-True-True] TypeError: ZarrBackendEntrypoint.open_dataset() got an unexpected keyword argument 'cache_members'

Check failure on line 3015 in xarray/tests/test_backends.py

GitHub Actions / macos-latest py3.12

TestZarrWriteEmpty.test_write_region[2-False-False-True-None] TypeError: ZarrBackendEntrypoint.open_dataset() got an unexpected keyword argument 'cache_members'

Check failure on line 3015 in xarray/tests/test_backends.py

GitHub Actions / ubuntu-latest py3.12 all-but-numba

TestZarrWriteEmpty.test_write_region[2-False-False-True-False] TypeError: ZarrBackendEntrypoint.open_dataset() got an unexpected keyword argument 'cache_members'

Check failure on line 3015 in xarray/tests/test_backends.py

GitHub Actions / ubuntu-latest py3.12 all-but-numba

TestZarrWriteEmpty.test_write_region[2-False-False-True-True] TypeError: ZarrBackendEntrypoint.open_dataset() got an unexpected keyword argument 'cache_members'

Check failure on line 3015 in xarray/tests/test_backends.py

GitHub Actions / ubuntu-latest py3.12 all-but-numba

TestZarrWriteEmpty.test_write_region[2-False-False-True-None] TypeError: ZarrBackendEntrypoint.open_dataset() got an unexpected keyword argument 'cache_members'

Check failure on line 3015 in xarray/tests/test_backends.py

GitHub Actions / ubuntu-latest py3.10

TestZarrWriteEmpty.test_write_region[2-False-False-True-False] TypeError: ZarrBackendEntrypoint.open_dataset() got an unexpected keyword argument 'cache_members'

Check failure on line 3015 in xarray/tests/test_backends.py

GitHub Actions / ubuntu-latest py3.10

TestZarrWriteEmpty.test_write_region[2-False-False-True-True] TypeError: ZarrBackendEntrypoint.open_dataset() got an unexpected keyword argument 'cache_members'

Check failure on line 3015 in xarray/tests/test_backends.py

GitHub Actions / ubuntu-latest py3.10

TestZarrWriteEmpty.test_write_region[2-False-False-True-None] TypeError: ZarrBackendEntrypoint.open_dataset() got an unexpected keyword argument 'cache_members'

Check failure on line 3015 in xarray/tests/test_backends.py

GitHub Actions / ubuntu-latest py3.12

TestZarrWriteEmpty.test_write_region[2-False-False-True-False] TypeError: ZarrBackendEntrypoint.open_dataset() got an unexpected keyword argument 'cache_members'

Check failure on line 3015 in xarray/tests/test_backends.py

GitHub Actions / ubuntu-latest py3.12

TestZarrWriteEmpty.test_write_region[2-False-False-True-True] TypeError: ZarrBackendEntrypoint.open_dataset() got an unexpected keyword argument 'cache_members'

Check failure on line 3015 in xarray/tests/test_backends.py

GitHub Actions / ubuntu-latest py3.12

TestZarrWriteEmpty.test_write_region[2-False-False-True-None] TypeError: ZarrBackendEntrypoint.open_dataset() got an unexpected keyword argument 'cache_members'
store, consolidated=consolidated, **self.version_kwargs
) as actual:
assert_identical(actual, zeros)
write_empty_chunks=write_empty,
**self.version_kwargs,
)
with xr.open_zarr(

Check failure on line 3028 in xarray/tests/test_backends.py

GitHub Actions / macos-latest py3.10

TestZarrWriteEmpty.test_write_region[2-False-False-False-False] TypeError: ZarrBackendEntrypoint.open_dataset() got an unexpected keyword argument 'cache_members'

Check failure on line 3028 in xarray/tests/test_backends.py

GitHub Actions / macos-latest py3.10

TestZarrWriteEmpty.test_write_region[2-False-False-False-True] TypeError: ZarrBackendEntrypoint.open_dataset() got an unexpected keyword argument 'cache_members'

Check failure on line 3028 in xarray/tests/test_backends.py

GitHub Actions / macos-latest py3.10

TestZarrWriteEmpty.test_write_region[2-False-False-False-None] TypeError: ZarrBackendEntrypoint.open_dataset() got an unexpected keyword argument 'cache_members'

Check failure on line 3028 in xarray/tests/test_backends.py

GitHub Actions / ubuntu-latest py3.10 min-all-deps

TestZarrDictStore.test_write_region[2-False-False-False-False] TypeError: ZarrBackendEntrypoint.open_dataset() got an unexpected keyword argument 'cache_members'

Check failure on line 3028 in xarray/tests/test_backends.py

GitHub Actions / ubuntu-latest py3.10 min-all-deps

TestZarrDictStore.test_write_region[2-False-False-False-True] TypeError: ZarrBackendEntrypoint.open_dataset() got an unexpected keyword argument 'cache_members'

Check failure on line 3028 in xarray/tests/test_backends.py

GitHub Actions / ubuntu-latest py3.10 min-all-deps

TestZarrDictStore.test_write_region[2-False-False-False-None] TypeError: ZarrBackendEntrypoint.open_dataset() got an unexpected keyword argument 'cache_members'

Check failure on line 3028 in xarray/tests/test_backends.py

GitHub Actions / macos-latest py3.12

TestZarrWriteEmpty.test_write_region[2-False-False-False-False] TypeError: ZarrBackendEntrypoint.open_dataset() got an unexpected keyword argument 'cache_members'

Check failure on line 3028 in xarray/tests/test_backends.py

GitHub Actions / macos-latest py3.12

TestZarrWriteEmpty.test_write_region[2-False-False-False-True] TypeError: ZarrBackendEntrypoint.open_dataset() got an unexpected keyword argument 'cache_members'

Check failure on line 3028 in xarray/tests/test_backends.py

GitHub Actions / macos-latest py3.12

TestZarrWriteEmpty.test_write_region[2-False-False-False-None] TypeError: ZarrBackendEntrypoint.open_dataset() got an unexpected keyword argument 'cache_members'

Check failure on line 3028 in xarray/tests/test_backends.py

GitHub Actions / ubuntu-latest py3.12 all-but-numba

TestZarrWriteEmpty.test_write_region[2-False-False-False-False] TypeError: ZarrBackendEntrypoint.open_dataset() got an unexpected keyword argument 'cache_members'

Check failure on line 3028 in xarray/tests/test_backends.py

GitHub Actions / ubuntu-latest py3.12 all-but-numba

TestZarrWriteEmpty.test_write_region[2-False-False-False-True] TypeError: ZarrBackendEntrypoint.open_dataset() got an unexpected keyword argument 'cache_members'

Check failure on line 3028 in xarray/tests/test_backends.py

GitHub Actions / ubuntu-latest py3.12 all-but-numba

TestZarrWriteEmpty.test_write_region[2-False-False-False-None] TypeError: ZarrBackendEntrypoint.open_dataset() got an unexpected keyword argument 'cache_members'

Check failure on line 3028 in xarray/tests/test_backends.py

GitHub Actions / ubuntu-latest py3.10

TestZarrWriteEmpty.test_write_region[2-False-False-False-False] TypeError: ZarrBackendEntrypoint.open_dataset() got an unexpected keyword argument 'cache_members'

Check failure on line 3028 in xarray/tests/test_backends.py

GitHub Actions / ubuntu-latest py3.10

TestZarrWriteEmpty.test_write_region[2-False-False-False-True] TypeError: ZarrBackendEntrypoint.open_dataset() got an unexpected keyword argument 'cache_members'

Check failure on line 3028 in xarray/tests/test_backends.py

GitHub Actions / ubuntu-latest py3.10

TestZarrWriteEmpty.test_write_region[2-False-False-False-None] TypeError: ZarrBackendEntrypoint.open_dataset() got an unexpected keyword argument 'cache_members'

Check failure on line 3028 in xarray/tests/test_backends.py

GitHub Actions / ubuntu-latest py3.12

TestZarrWriteEmpty.test_write_region[2-False-False-False-False] TypeError: ZarrBackendEntrypoint.open_dataset() got an unexpected keyword argument 'cache_members'

Check failure on line 3028 in xarray/tests/test_backends.py

GitHub Actions / ubuntu-latest py3.12

TestZarrWriteEmpty.test_write_region[2-False-False-False-True] TypeError: ZarrBackendEntrypoint.open_dataset() got an unexpected keyword argument 'cache_members'

Check failure on line 3028 in xarray/tests/test_backends.py

GitHub Actions / ubuntu-latest py3.12

TestZarrWriteEmpty.test_write_region[2-False-False-False-None] TypeError: ZarrBackendEntrypoint.open_dataset() got an unexpected keyword argument 'cache_members'
store, consolidated=consolidated, **self.version_kwargs
) as actual:
assert_identical(actual, nonzeros)
ds = create_test_data()
with self.create_zarr_target() as store_target:
ds.to_zarr(store_target, **self.version_kwargs)
ds_a = xr.open_zarr(store_target, **self.version_kwargs)

Check failure on line 3212 in xarray/tests/test_backends.py

GitHub Actions / ubuntu-latest py3.11 all-but-dask

TestZarrWriteEmpty.test_open_zarr_use_cftime[2] TypeError: ZarrBackendEntrypoint.open_dataset() got an unexpected keyword argument 'cache_members'
assert_identical(ds, ds_a)
ds_b = xr.open_zarr(store_target, use_cftime=True, **self.version_kwargs)
assert xr.coding.times.contains_cftime_datetimes(ds_b.time.variable)
# NOTE: using self.roundtrip, which uses open_dataset, will not trigger the bug.
with self.create_zarr_target() as initial_store:
ds.to_zarr(initial_store, mode="w", **self.version_kwargs)
ds1 = xr.open_zarr(initial_store, **self.version_kwargs)

Check failure on line 3224 in xarray/tests/test_backends.py

GitHub Actions / ubuntu-latest py3.11 all-but-dask

TestZarrWriteEmpty.test_write_read_select_write[2] TypeError: ZarrBackendEntrypoint.open_dataset() got an unexpected keyword argument 'cache_members'
# Combination of where+squeeze triggers error on write.
ds_sel = ds1.where(ds1.coords["dim3"] == "a", drop=True).squeeze("dim3")
ds = obj if isinstance(obj, Dataset) else obj.to_dataset()
with self.create_zarr_target() as store_target:
ds.to_zarr(store_target, **self.version_kwargs)
assert_identical(ds, xr.open_zarr(store_target, **self.version_kwargs))

Check failure on line 3239 in xarray/tests/test_backends.py

GitHub Actions / ubuntu-latest py3.11 all-but-dask

TestZarrWriteEmpty.test_attributes[2-obj0] TypeError: ZarrBackendEntrypoint.open_dataset() got an unexpected keyword argument 'cache_members'

Check failure on line 3239 in xarray/tests/test_backends.py

GitHub Actions / ubuntu-latest py3.11 all-but-dask

TestZarrWriteEmpty.test_attributes[2-obj1] TypeError: ZarrBackendEntrypoint.open_dataset() got an unexpected keyword argument 'cache_members'
obj.attrs["bad"] = DataArray()
ds = obj if isinstance(obj, Dataset) else obj.to_dataset()
ds.to_zarr(store, mode="w")
# We expect this to request array metadata information, so call_count should be == 1,
xrds = xr.open_zarr(store)

Check failure on line 3708 in xarray/tests/test_backends.py

GitHub Actions / ubuntu-latest py3.10 min-all-deps

TestZarrWriteEmpty.test_avoid_excess_metadata_calls[2] TypeError: ZarrBackendEntrypoint.open_dataset() got an unexpected keyword argument 'cache_members'

Check failure on line 3708 in xarray/tests/test_backends.py

GitHub Actions / ubuntu-latest py3.11 all-but-dask

TestZarrWriteEmpty.test_avoid_excess_metadata_calls[2] TypeError: ZarrBackendEntrypoint.open_dataset() got an unexpected keyword argument 'cache_members'
call_count = mock.call_count
assert call_count == 1
with pytest.warns(FutureWarning, match="zarr_version"):
ds.to_zarr(store=store, zarr_version=2)
with pytest.warns(FutureWarning, match="zarr_version"):

Check failure on line 3742 in xarray/tests/test_backends.py

GitHub Actions / ubuntu-latest py3.10 min-all-deps

test_zarr_version_deprecated Failed: DID NOT WARN. No warnings of type (<class 'FutureWarning'>,) were emitted. Emitted warnings: [].

Check failure on line 3742 in xarray/tests/test_backends.py

GitHub Actions / ubuntu-latest py3.11 all-but-dask

test_zarr_version_deprecated Failed: DID NOT WARN. No warnings of type (<class 'FutureWarning'>,) were emitted. Emitted warnings: [].
xr.open_zarr(store=store, zarr_version=2)
with pytest.raises(ValueError, match="zarr_format"):
def test_open_nczarr(self) -> None:
with create_tmp_file(suffix=".zarr") as tmp:
expected = self._create_nczarr(tmp)
actual = xr.open_zarr(tmp, consolidated=False)

Check failure on line 6070 in xarray/tests/test_backends.py

GitHub Actions / macos-latest py3.10

TestNCZarr.test_open_nczarr TypeError: ZarrBackendEntrypoint.open_dataset() got an unexpected keyword argument 'cache_members'

Check failure on line 6070 in xarray/tests/test_backends.py

GitHub Actions / macos-latest py3.12

TestNCZarr.test_open_nczarr TypeError: ZarrBackendEntrypoint.open_dataset() got an unexpected keyword argument 'cache_members'

Check failure on line 6070 in xarray/tests/test_backends.py

GitHub Actions / ubuntu-latest py3.12 all-but-numba

TestNCZarr.test_open_nczarr TypeError: ZarrBackendEntrypoint.open_dataset() got an unexpected keyword argument 'cache_members'

Check failure on line 6070 in xarray/tests/test_backends.py

GitHub Actions / ubuntu-latest py3.10

TestNCZarr.test_open_nczarr TypeError: ZarrBackendEntrypoint.open_dataset() got an unexpected keyword argument 'cache_members'

Check failure on line 6070 in xarray/tests/test_backends.py

GitHub Actions / ubuntu-latest py3.12

TestNCZarr.test_open_nczarr TypeError: ZarrBackendEntrypoint.open_dataset() got an unexpected keyword argument 'cache_members'
assert_identical(expected, actual)
def test_overwriting_nczarr(self) -> None:
ds = self._create_nczarr(tmp)
expected = ds[["var1"]]
expected.to_zarr(tmp, mode="w")
actual = xr.open_zarr(tmp, consolidated=False)

Check failure on line 6078 in xarray/tests/test_backends.py

GitHub Actions / macos-latest py3.10

TestNCZarr.test_overwriting_nczarr TypeError: ZarrBackendEntrypoint.open_dataset() got an unexpected keyword argument 'cache_members'

Check failure on line 6078 in xarray/tests/test_backends.py

GitHub Actions / macos-latest py3.12

TestNCZarr.test_overwriting_nczarr TypeError: ZarrBackendEntrypoint.open_dataset() got an unexpected keyword argument 'cache_members'

Check failure on line 6078 in xarray/tests/test_backends.py

GitHub Actions / ubuntu-latest py3.12 all-but-numba

TestNCZarr.test_overwriting_nczarr TypeError: ZarrBackendEntrypoint.open_dataset() got an unexpected keyword argument 'cache_members'

Check failure on line 6078 in xarray/tests/test_backends.py

GitHub Actions / ubuntu-latest py3.10

TestNCZarr.test_overwriting_nczarr TypeError: ZarrBackendEntrypoint.open_dataset() got an unexpected keyword argument 'cache_members'

Check failure on line 6078 in xarray/tests/test_backends.py

GitHub Actions / ubuntu-latest py3.12

TestNCZarr.test_overwriting_nczarr TypeError: ZarrBackendEntrypoint.open_dataset() got an unexpected keyword argument 'cache_members'
assert_identical(expected, actual)
@pytest.mark.parametrize("mode", ["a", "r+"])
ds_region = 1 + ds.isel(x=slice(2, 4), y=slice(6, 8))
ds_region.to_zarr(tmp_path / "test.zarr", region="auto")
ds_updated = xr.open_zarr(tmp_path / "test.zarr")

Check failure on line 6135 in xarray/tests/test_backends.py

GitHub Actions / ubuntu-latest py3.10 min-all-deps

TestZarrRegionAuto.test_zarr_region_auto_all[2] TypeError: ZarrBackendEntrypoint.open_dataset() got an unexpected keyword argument 'cache_members'

Check failure on line 6135 in xarray/tests/test_backends.py

GitHub Actions / ubuntu-latest py3.11 all-but-dask

TestZarrRegionAuto.test_zarr_region_auto_all[2] TypeError: ZarrBackendEntrypoint.open_dataset() got an unexpected keyword argument 'cache_members'
expected = ds.copy()
expected["test"][2:4, 6:8] += 1
tmp_path / "test.zarr", region={"x": "auto", "y": slice(6, 8)}
)
ds_updated = xr.open_zarr(tmp_path / "test.zarr")

Check failure on line 6161 in xarray/tests/test_backends.py

GitHub Actions / ubuntu-latest py3.10 min-all-deps

TestZarrRegionAuto.test_zarr_region_auto_mixed[2] TypeError: ZarrBackendEntrypoint.open_dataset() got an unexpected keyword argument 'cache_members'

Check failure on line 6161 in xarray/tests/test_backends.py

GitHub Actions / ubuntu-latest py3.11 all-but-dask

TestZarrRegionAuto.test_zarr_region_auto_mixed[2] TypeError: ZarrBackendEntrypoint.open_dataset() got an unexpected keyword argument 'cache_members'
expected = ds.copy()
expected["test"][2:4, 6:8] += 1
da.isel(x=slice(7, 11)).chunk(x=(3, 1)).to_zarr(
store, safe_chunks=True, append_dim="x"
)
assert xr.open_zarr(store)["foo"].equals(da.isel(x=slice(0, 11)))

Check failure on line 6417 in xarray/tests/test_backends.py

GitHub Actions / ubuntu-latest py3.10 min-all-deps

test_zarr_safe_chunk_append_dim[2] TypeError: ZarrBackendEntrypoint.open_dataset() got an unexpected keyword argument 'cache_members'
da.isel(x=slice(0, 7)).to_zarr(store, safe_chunks=True, mode="w")
# If the first chunk is of the size of the border size + N * zchunk then it is valid