7 tests fail #1819

yurivict · 2024-04-26T04:01:35Z

Zarr version

2.17.2

Numcodecs version

Numcodecs

Python Version

3.9

Operating System

FreeBSD 14

Installation

port

Description

========================================================================================= FAILURES =========================================================================================
_____________________________________________________________________________ TestNestedFSStore.test_hierarchy _____________________________________________________________________________
[gw2] freebsd14 -- Python 3.9.18 /usr/local/bin/python3.9

self = <zarr.tests.test_storage.TestNestedFSStore object at 0x1453af27310>

    def test_hierarchy(self):
        # setup
        store = self.create_store()
        store[self.root + "a"] = b"aaa"
        store[self.root + "b"] = b"bbb"
        store[self.root + "c/d"] = b"ddd"
        store[self.root + "c/e/f"] = b"fff"
        store[self.root + "c/e/g"] = b"ggg"
    
        # check keys
        assert self.root + "a" in store
        assert self.root + "b" in store
        assert self.root + "c/d" in store
        assert self.root + "c/e/f" in store
        assert self.root + "c/e/g" in store
        assert self.root + "c" not in store
        assert self.root + "c/" not in store
        assert self.root + "c/e" not in store
        assert self.root + "c/e/" not in store
        assert self.root + "c/d/x" not in store
    
        # check __getitem__
        with pytest.raises(KeyError):
            store[self.root + "c"]
        with pytest.raises(KeyError):
            store[self.root + "c/e"]
        with pytest.raises(KeyError):
            store[self.root + "c/d/x"]
    
        # test getsize (optional)
        if hasattr(store, "getsize"):
            # TODO: proper behavior of getsize?
            #       v3 returns size of all nested arrays, not just the
            #       size of the arrays in the current folder.
            if self.version == 2:
                assert 6 == store.getsize()
            else:
                assert 15 == store.getsize()
            assert 3 == store.getsize("a")
            assert 3 == store.getsize("b")
            if self.version == 2:
                assert 3 == store.getsize("c")
            else:
                assert 9 == store.getsize("c")
            assert 3 == store.getsize("c/d")
            assert 6 == store.getsize("c/e")
            assert 3 == store.getsize("c/e/f")
            assert 3 == store.getsize("c/e/g")
            # non-existent paths
            assert 0 == store.getsize("x")
            assert 0 == store.getsize("a/x")
            assert 0 == store.getsize("c/x")
            assert 0 == store.getsize("c/x/y")
            assert 0 == store.getsize("c/d/y")
            assert 0 == store.getsize("c/d/y/z")
    
            # access item via full path
            assert 3 == store.getsize(self.root + "a")
    
        # test listdir (optional)
        if hasattr(store, "listdir"):
            assert {"a", "b", "c"} == set(store.listdir(self.root))
            assert {"d", "e"} == set(store.listdir(self.root + "c"))
            assert {"f", "g"} == set(store.listdir(self.root + "c/e"))
            # no exception raised if path does not exist or is leaf
            assert [] == store.listdir(self.root + "x")
            assert [] == store.listdir(self.root + "a/x")
            assert [] == store.listdir(self.root + "c/x")
            assert [] == store.listdir(self.root + "c/x/y")
            assert [] == store.listdir(self.root + "c/d/y")
            assert [] == store.listdir(self.root + "c/d/y/z")
>           assert [] == store.listdir(self.root + "c/e/f")
E           AssertionError: assert [] == ['f']
E             Right contains one more item: 'f'
E             Full diff:
E             - ['f']
E             + []

zarr/tests/test_storage.py:409: AssertionError
_________________________________________________________________________________ TestFSStore.test_complex _________________________________________________________________________________
[gw1] freebsd14 -- Python 3.9.18 /usr/local/bin/python3.9

self = <zarr.tests.test_storage.TestFSStore object at 0x35cf7d40f8b0>

    def test_complex(self):
        path1 = tempfile.mkdtemp()
        path2 = tempfile.mkdtemp()
        store = self.create_store(
            path="simplecache::file://" + path1,
            simplecache={"same_names": True, "cache_storage": path2},
        )
        assert not store
        assert not os.listdir(path1)
        assert not os.listdir(path2)
        store[self.root + "foo"] = b"hello"
        assert "foo" in os.listdir(str(path1) + "/" + self.root)
        assert self.root + "foo" in store
>       assert not os.listdir(str(path2))
E       AssertionError: assert not ['foo']
E        +  where ['foo'] = <built-in function listdir>('/tmp/tmpglsphvne')
E        +    where <built-in function listdir> = os.listdir
E        +    and   '/tmp/tmpglsphvne' = str('/tmp/tmpglsphvne')

zarr/tests/test_storage.py:1155: AssertionError
________________________________________________________________________________ TestFSStore.test_hierarchy ________________________________________________________________________________
[gw1] freebsd14 -- Python 3.9.18 /usr/local/bin/python3.9

self = <zarr.tests.test_storage.TestFSStore object at 0x35cf7bcd6d00>

    def test_hierarchy(self):
        # setup
        store = self.create_store()
        store[self.root + "a"] = b"aaa"
        store[self.root + "b"] = b"bbb"
        store[self.root + "c/d"] = b"ddd"
        store[self.root + "c/e/f"] = b"fff"
        store[self.root + "c/e/g"] = b"ggg"
    
        # check keys
        assert self.root + "a" in store
        assert self.root + "b" in store
        assert self.root + "c/d" in store
        assert self.root + "c/e/f" in store
        assert self.root + "c/e/g" in store
        assert self.root + "c" not in store
        assert self.root + "c/" not in store
        assert self.root + "c/e" not in store
        assert self.root + "c/e/" not in store
        assert self.root + "c/d/x" not in store
    
        # check __getitem__
        with pytest.raises(KeyError):
            store[self.root + "c"]
        with pytest.raises(KeyError):
            store[self.root + "c/e"]
        with pytest.raises(KeyError):
            store[self.root + "c/d/x"]
    
        # test getsize (optional)
        if hasattr(store, "getsize"):
            # TODO: proper behavior of getsize?
            #       v3 returns size of all nested arrays, not just the
            #       size of the arrays in the current folder.
            if self.version == 2:
                assert 6 == store.getsize()
            else:
                assert 15 == store.getsize()
            assert 3 == store.getsize("a")
            assert 3 == store.getsize("b")
            if self.version == 2:
                assert 3 == store.getsize("c")
            else:
                assert 9 == store.getsize("c")
            assert 3 == store.getsize("c/d")
            assert 6 == store.getsize("c/e")
            assert 3 == store.getsize("c/e/f")
            assert 3 == store.getsize("c/e/g")
            # non-existent paths
            assert 0 == store.getsize("x")
            assert 0 == store.getsize("a/x")
            assert 0 == store.getsize("c/x")
            assert 0 == store.getsize("c/x/y")
            assert 0 == store.getsize("c/d/y")
            assert 0 == store.getsize("c/d/y/z")
    
            # access item via full path
            assert 3 == store.getsize(self.root + "a")
    
        # test listdir (optional)
        if hasattr(store, "listdir"):
            assert {"a", "b", "c"} == set(store.listdir(self.root))
            assert {"d", "e"} == set(store.listdir(self.root + "c"))
            assert {"f", "g"} == set(store.listdir(self.root + "c/e"))
            # no exception raised if path does not exist or is leaf
            assert [] == store.listdir(self.root + "x")
            assert [] == store.listdir(self.root + "a/x")
            assert [] == store.listdir(self.root + "c/x")
            assert [] == store.listdir(self.root + "c/x/y")
            assert [] == store.listdir(self.root + "c/d/y")
            assert [] == store.listdir(self.root + "c/d/y/z")
>           assert [] == store.listdir(self.root + "c/e/f")
E           AssertionError: assert [] == ['f']
E             Right contains one more item: 'f'
E             Full diff:
E             - ['f']
E             + []

zarr/tests/test_storage.py:409: AssertionError
________________________________________________________________________ TestFSStoreWithKeySeparator.test_hierarchy ________________________________________________________________________
[gw2] freebsd14 -- Python 3.9.18 /usr/local/bin/python3.9

self = <zarr.tests.test_storage.TestFSStoreWithKeySeparator object at 0x1453afb09d0>

    def test_hierarchy(self):
        # setup
        store = self.create_store()
        store[self.root + "a"] = b"aaa"
        store[self.root + "b"] = b"bbb"
        store[self.root + "c/d"] = b"ddd"
        store[self.root + "c/e/f"] = b"fff"
        store[self.root + "c/e/g"] = b"ggg"
    
        # check keys
        assert self.root + "a" in store
        assert self.root + "b" in store
        assert self.root + "c/d" in store
        assert self.root + "c/e/f" in store
        assert self.root + "c/e/g" in store
        assert self.root + "c" not in store
        assert self.root + "c/" not in store
        assert self.root + "c/e" not in store
        assert self.root + "c/e/" not in store
        assert self.root + "c/d/x" not in store
    
        # check __getitem__
        with pytest.raises(KeyError):
            store[self.root + "c"]
        with pytest.raises(KeyError):
            store[self.root + "c/e"]
        with pytest.raises(KeyError):
            store[self.root + "c/d/x"]
    
        # test getsize (optional)
        if hasattr(store, "getsize"):
            # TODO: proper behavior of getsize?
            #       v3 returns size of all nested arrays, not just the
            #       size of the arrays in the current folder.
            if self.version == 2:
                assert 6 == store.getsize()
            else:
                assert 15 == store.getsize()
            assert 3 == store.getsize("a")
            assert 3 == store.getsize("b")
            if self.version == 2:
                assert 3 == store.getsize("c")
            else:
                assert 9 == store.getsize("c")
            assert 3 == store.getsize("c/d")
            assert 6 == store.getsize("c/e")
            assert 3 == store.getsize("c/e/f")
            assert 3 == store.getsize("c/e/g")
            # non-existent paths
            assert 0 == store.getsize("x")
            assert 0 == store.getsize("a/x")
            assert 0 == store.getsize("c/x")
            assert 0 == store.getsize("c/x/y")
            assert 0 == store.getsize("c/d/y")
            assert 0 == store.getsize("c/d/y/z")
    
            # access item via full path
            assert 3 == store.getsize(self.root + "a")
    
        # test listdir (optional)
        if hasattr(store, "listdir"):
            assert {"a", "b", "c"} == set(store.listdir(self.root))
            assert {"d", "e"} == set(store.listdir(self.root + "c"))
            assert {"f", "g"} == set(store.listdir(self.root + "c/e"))
            # no exception raised if path does not exist or is leaf
            assert [] == store.listdir(self.root + "x")
            assert [] == store.listdir(self.root + "a/x")
            assert [] == store.listdir(self.root + "c/x")
            assert [] == store.listdir(self.root + "c/x/y")
            assert [] == store.listdir(self.root + "c/d/y")
            assert [] == store.listdir(self.root + "c/d/y/z")
>           assert [] == store.listdir(self.root + "c/e/f")
E           AssertionError: assert [] == ['f']
E             Right contains one more item: 'f'
E             Full diff:
E             - ['f']
E             + []

zarr/tests/test_storage.py:409: AssertionError
________________________________________________________________________________ TestN5FSStore.test_complex ________________________________________________________________________________
[gw1] freebsd14 -- Python 3.9.18 /usr/local/bin/python3.9

self = <zarr.tests.test_storage.TestN5FSStore object at 0x35cf7cf2a4c0>

    def test_complex(self):
        path1 = tempfile.mkdtemp()
        path2 = tempfile.mkdtemp()
        store = self.create_store(
            path="simplecache::file://" + path1,
            simplecache={"same_names": True, "cache_storage": path2},
        )
        assert not store
        assert not os.listdir(path1)
        assert not os.listdir(path2)
        store[self.root + "foo"] = b"hello"
        assert "foo" in os.listdir(str(path1) + "/" + self.root)
        assert self.root + "foo" in store
>       assert not os.listdir(str(path2))
E       AssertionError: assert not ['foo']
E        +  where ['foo'] = <built-in function listdir>('/tmp/tmp54ti_hui')
E        +    where <built-in function listdir> = os.listdir
E        +    and   '/tmp/tmp54ti_hui' = str('/tmp/tmp54ti_hui')

zarr/tests/test_storage.py:1155: AssertionError
_______________________________________________________________________________ TestN5FSStore.test_hierarchy _______________________________________________________________________________
[gw1] freebsd14 -- Python 3.9.18 /usr/local/bin/python3.9

self = <zarr.tests.test_storage.TestN5FSStore object at 0x35cf7c43d550>

    def test_hierarchy(self):
        # setup
        store = self.create_store()
        store[self.root + "a"] = b"aaa"
        store[self.root + "b"] = b"bbb"
        store[self.root + "c/d"] = b"ddd"
        store[self.root + "c/e/f"] = b"fff"
        store[self.root + "c/e/g"] = b"ggg"
    
        # check keys
        assert self.root + "a" in store
        assert self.root + "b" in store
        assert self.root + "c/d" in store
        assert self.root + "c/e/f" in store
        assert self.root + "c/e/g" in store
        assert self.root + "c" not in store
        assert self.root + "c/" not in store
        assert self.root + "c/e" not in store
        assert self.root + "c/e/" not in store
        assert self.root + "c/d/x" not in store
    
        # check __getitem__
        with pytest.raises(KeyError):
            store[self.root + "c"]
        with pytest.raises(KeyError):
            store[self.root + "c/e"]
        with pytest.raises(KeyError):
            store[self.root + "c/d/x"]
    
        # test getsize (optional)
        if hasattr(store, "getsize"):
            # TODO: proper behavior of getsize?
            #       v3 returns size of all nested arrays, not just the
            #       size of the arrays in the current folder.
            if self.version == 2:
                assert 6 == store.getsize()
            else:
                assert 15 == store.getsize()
            assert 3 == store.getsize("a")
            assert 3 == store.getsize("b")
            if self.version == 2:
                assert 3 == store.getsize("c")
            else:
                assert 9 == store.getsize("c")
            assert 3 == store.getsize("c/d")
            assert 6 == store.getsize("c/e")
            assert 3 == store.getsize("c/e/f")
            assert 3 == store.getsize("c/e/g")
            # non-existent paths
            assert 0 == store.getsize("x")
            assert 0 == store.getsize("a/x")
            assert 0 == store.getsize("c/x")
            assert 0 == store.getsize("c/x/y")
            assert 0 == store.getsize("c/d/y")
            assert 0 == store.getsize("c/d/y/z")
    
            # access item via full path
            assert 3 == store.getsize(self.root + "a")
    
        # test listdir (optional)
        if hasattr(store, "listdir"):
            assert {"a", "b", "c"} == set(store.listdir(self.root))
            assert {"d", "e"} == set(store.listdir(self.root + "c"))
            assert {"f", "g"} == set(store.listdir(self.root + "c/e"))
            # no exception raised if path does not exist or is leaf
            assert [] == store.listdir(self.root + "x")
            assert [] == store.listdir(self.root + "a/x")
            assert [] == store.listdir(self.root + "c/x")
            assert [] == store.listdir(self.root + "c/x/y")
            assert [] == store.listdir(self.root + "c/d/y")
            assert [] == store.listdir(self.root + "c/d/y/z")
>           assert [] == store.listdir(self.root + "c/e/f")
E           AssertionError: assert [] == ['f']
E             Right contains one more item: 'f'
E             Full diff:
E             - ['f']
E             + []

zarr/tests/test_storage.py:409: AssertionError
_________________________________________________________________________ TestFSStoreFromFilesystem.test_hierarchy _________________________________________________________________________
[gw0] freebsd14 -- Python 3.9.18 /usr/local/bin/python3.9

self = <zarr.tests.test_storage.TestFSStoreFromFilesystem object at 0xfa69af959a0>

    def test_hierarchy(self):
        # setup
        store = self.create_store()
        store[self.root + "a"] = b"aaa"
        store[self.root + "b"] = b"bbb"
        store[self.root + "c/d"] = b"ddd"
        store[self.root + "c/e/f"] = b"fff"
        store[self.root + "c/e/g"] = b"ggg"
    
        # check keys
        assert self.root + "a" in store
        assert self.root + "b" in store
        assert self.root + "c/d" in store
        assert self.root + "c/e/f" in store
        assert self.root + "c/e/g" in store
        assert self.root + "c" not in store
        assert self.root + "c/" not in store
        assert self.root + "c/e" not in store
        assert self.root + "c/e/" not in store
        assert self.root + "c/d/x" not in store
    
        # check __getitem__
        with pytest.raises(KeyError):
            store[self.root + "c"]
        with pytest.raises(KeyError):
            store[self.root + "c/e"]
        with pytest.raises(KeyError):
            store[self.root + "c/d/x"]
    
        # test getsize (optional)
        if hasattr(store, "getsize"):
            # TODO: proper behavior of getsize?
            #       v3 returns size of all nested arrays, not just the
            #       size of the arrays in the current folder.
            if self.version == 2:
                assert 6 == store.getsize()
            else:
                assert 15 == store.getsize()
            assert 3 == store.getsize("a")
            assert 3 == store.getsize("b")
            if self.version == 2:
                assert 3 == store.getsize("c")
            else:
                assert 9 == store.getsize("c")
            assert 3 == store.getsize("c/d")
            assert 6 == store.getsize("c/e")
            assert 3 == store.getsize("c/e/f")
            assert 3 == store.getsize("c/e/g")
            # non-existent paths
            assert 0 == store.getsize("x")
            assert 0 == store.getsize("a/x")
            assert 0 == store.getsize("c/x")
            assert 0 == store.getsize("c/x/y")
            assert 0 == store.getsize("c/d/y")
            assert 0 == store.getsize("c/d/y/z")
    
            # access item via full path
            assert 3 == store.getsize(self.root + "a")
    
        # test listdir (optional)
        if hasattr(store, "listdir"):
            assert {"a", "b", "c"} == set(store.listdir(self.root))
            assert {"d", "e"} == set(store.listdir(self.root + "c"))
            assert {"f", "g"} == set(store.listdir(self.root + "c/e"))
            # no exception raised if path does not exist or is leaf
            assert [] == store.listdir(self.root + "x")
            assert [] == store.listdir(self.root + "a/x")
            assert [] == store.listdir(self.root + "c/x")
            assert [] == store.listdir(self.root + "c/x/y")
            assert [] == store.listdir(self.root + "c/d/y")
            assert [] == store.listdir(self.root + "c/d/y/z")
>           assert [] == store.listdir(self.root + "c/e/f")
E           AssertionError: assert [] == ['f']
E             Right contains one more item: 'f'
E             Full diff:
E             - ['f']
E             + []

zarr/tests/test_storage.py:409: AssertionError

Steps to reproduce

run tests

Additional output

No response

The text was updated successfully, but these errors were encountered:

avalentino · 2024-05-01T10:42:36Z

This issue is also reproducible on debian sid

avalentino · 2024-05-01T16:05:08Z

Apparently the issue is triggered by the update of fsspec from v2024.2.0 to v2024.3.1

jhamman · 2024-05-17T16:29:53Z

See #1877 (comment) for a diagnosis of what is going on here. Marking this as help-wanted.

martindurant · 2024-06-07T17:01:40Z

Damn but I thought I had already commented on these in a different thread. Now to find it...

jhamman · 2024-06-07T17:04:24Z

You did (#1877 (comment)), but that ticket was closed by dependabot.

QuLogic · 2024-06-14T20:52:59Z

This is also a dupe of #1679 (or vice versa).

…r-developers#1679) This is adapted from the fixes that were rolled into zarr-developers#1785 for the v3 branch. Signed-off-by: Adam Williamson <[email protected]>

yurivict · 2024-09-19T03:35:02Z

2.18.3 has 32 test failures: https://freebsd.org/~yuri/py-zarr-2.18.3-tests.log

avalentino · 2024-11-27T07:44:46Z

Please find below an update from a recent test on Debian Sid.
Please not that, in addition to the fsspec related ones, there is also an error in test_array_with_delta_filter that started appearing recently.
It could be related to some update in dependencies.
Shall I open a separate ticked for it?

python3.12 -m pytest --pyargs zarr
============================= test session starts ==============================
platform linux -- Python 3.12.7, pytest-8.3.3, pluggy-1.5.0
Matplotlib: 3.8.3
Freetype: 2.13.3
benchmark: 5.1.0 (defaults: timer=time.perf_counter disable_gc=False min_rounds=5 min_time=0.000005 max_time=1.0 calibration_precision=10 warmup=False warmup_iterations=100000)
rootdir: /home/antonio/debian/git/build-area/zarr-2.18.3+ds/.pybuild/cpython3_3.12_zarr/build
configfile: pyproject.toml
plugins: remotedata-0.4.1, console-scripts-1.4.1, mock-3.14.0, lazy-fixtures-1.1.1, astropy-0.11.0, instafail-0.5.0, socket-0.7.0, doctestplus-1.3.0, mpl-0.17.0, anyio-4.6.2, timeout-2.3.1, filter-subpackage-0.2.0, recording-0.13.2, subtests-0.13.1, cov-5.0.0, astropy-header-0.2.2, xdist-3.6.1, hypothesis-6.119.3, arraydiff-0.6.1, benchmark-5.1.0, requests_mock-1.12.1, typeguard-4.4.1
collected 5174 items

zarr/tests/test_attrs.py ....................                            [  0%]

[...]

zarr/tests/test_util.py ..................                               [100%]

=================================== FAILURES ===================================
_________________________ test_array_with_delta_filter _________________________

    def test_array_with_delta_filter():
        # setup
        astype = "u1"
        dtype = "i8"
        filters = [Delta(astype=astype, dtype=dtype)]
        data = np.arange(100, dtype=dtype)
    
        for compressor in compressors:
>           a = array(data, chunks=10, compressor=compressor, filters=filters)

zarr/tests/test_filters.py:40: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
zarr/creation.py:444: in array
    z[...] = data
zarr/core.py:1449: in __setitem__
    self.set_basic_selection(pure_selection, value, fields=fields)
zarr/core.py:1545: in set_basic_selection
    return self._set_basic_selection_nd(selection, value, fields=fields)
zarr/core.py:1935: in _set_basic_selection_nd
    self._set_selection(indexer, value, fields=fields)
zarr/core.py:1988: in _set_selection
    self._chunk_setitem(chunk_coords, chunk_selection, chunk_value, fields=fields)
zarr/core.py:2261: in _chunk_setitem
    self._chunk_setitem_nosync(chunk_coords, chunk_selection, value, fields=fields)
zarr/core.py:2271: in _chunk_setitem_nosync
    self.chunk_store[ckey] = self._encode_chunk(cdata)
zarr/core.py:2387: in _encode_chunk
    chunk = f.encode(chunk)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = Delta(dtype='<i8', astype='|u1')
buf = array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])

    def encode(self, buf):
        # normalise input
        arr = ensure_ndarray(buf).view(self.dtype)
    
        # flatten to simplify implementation
        arr = arr.reshape(-1, order='A')
    
        # setup encoded output
        enc = np.empty_like(arr, dtype=self.astype)
    
        # set first element
        enc[0] = arr[0]
    
        # compute differences
        # using np.subtract for in-place operations
        if arr.dtype == bool:
            np.not_equal(arr[1:], arr[:-1], out=enc[1:])
        else:
>           np.subtract(arr[1:], arr[:-1], out=enc[1:])
E           numpy.core._exceptions._UFuncOutputCastingError: Cannot cast ufunc 'subtract' output from dtype('int64') to dtype('uint8') with casting rule 'same_kind'

/usr/lib/python3/dist-packages/numcodecs/delta.py:70: UFuncTypeError
__________________________ TestFSStore.test_hierarchy __________________________

self = <zarr.tests.test_storage.TestFSStore object at 0x764c45b5f4d0>

    def test_hierarchy(self):
        # setup
        store = self.create_store()
        store[self.root + "a"] = b"aaa"
        store[self.root + "b"] = b"bbb"
        store[self.root + "c/d"] = b"ddd"
        store[self.root + "c/e/f"] = b"fff"
        store[self.root + "c/e/g"] = b"ggg"
    
        # check keys
        assert self.root + "a" in store
        assert self.root + "b" in store
        assert self.root + "c/d" in store
        assert self.root + "c/e/f" in store
        assert self.root + "c/e/g" in store
        assert self.root + "c" not in store
        assert self.root + "c/" not in store
        assert self.root + "c/e" not in store
        assert self.root + "c/e/" not in store
        assert self.root + "c/d/x" not in store
    
        # check __getitem__
        with pytest.raises(KeyError):
            store[self.root + "c"]
        with pytest.raises(KeyError):
            store[self.root + "c/e"]
        with pytest.raises(KeyError):
            store[self.root + "c/d/x"]
    
        # test getsize (optional)
        if hasattr(store, "getsize"):
            # TODO: proper behavior of getsize?
            #       v3 returns size of all nested arrays, not just the
            #       size of the arrays in the current folder.
            if self.version == 2:
                assert 6 == store.getsize()
            else:
                assert 15 == store.getsize()
            assert 3 == store.getsize("a")
            assert 3 == store.getsize("b")
            if self.version == 2:
                assert 3 == store.getsize("c")
            else:
                assert 9 == store.getsize("c")
            assert 3 == store.getsize("c/d")
            assert 6 == store.getsize("c/e")
            assert 3 == store.getsize("c/e/f")
            assert 3 == store.getsize("c/e/g")
            # non-existent paths
            assert 0 == store.getsize("x")
            assert 0 == store.getsize("a/x")
            assert 0 == store.getsize("c/x")
            assert 0 == store.getsize("c/x/y")
            assert 0 == store.getsize("c/d/y")
            assert 0 == store.getsize("c/d/y/z")
    
            # access item via full path
            assert 3 == store.getsize(self.root + "a")
    
        # test listdir (optional)
        if hasattr(store, "listdir"):
            assert {"a", "b", "c"} == set(store.listdir(self.root))
            assert {"d", "e"} == set(store.listdir(self.root + "c"))
            assert {"f", "g"} == set(store.listdir(self.root + "c/e"))
            # no exception raised if path does not exist or is leaf
            assert [] == store.listdir(self.root + "x")
            assert [] == store.listdir(self.root + "a/x")
            assert [] == store.listdir(self.root + "c/x")
            assert [] == store.listdir(self.root + "c/x/y")
            assert [] == store.listdir(self.root + "c/d/y")
            assert [] == store.listdir(self.root + "c/d/y/z")
>           assert [] == store.listdir(self.root + "c/e/f")
E           AssertionError: assert [] == ['f']
E             
E             Right contains one more item: 'f'
E             Use -v to get more diff

zarr/tests/test_storage.py:409: AssertionError
___________________________ TestFSStore.test_complex ___________________________

self = <zarr.tests.test_storage.TestFSStore object at 0x764c45b97c20>

    def test_complex(self):
        path1 = tempfile.mkdtemp()
        path2 = tempfile.mkdtemp()
        store = self.create_store(
            path="simplecache::file://" + path1,
            simplecache={"same_names": True, "cache_storage": path2},
        )
        assert not store
        assert not os.listdir(path1)
        assert not os.listdir(path2)
        store[self.root + "foo"] = b"hello"
        assert "foo" in os.listdir(str(path1) + "/" + self.root)
        assert self.root + "foo" in store
>       assert not os.listdir(str(path2))
E       AssertionError: assert not ['foo']
E        +  where ['foo'] = <built-in function listdir>('/tmp/tmp2i9jx98l')
E        +    where <built-in function listdir> = os.listdir
E        +    and   '/tmp/tmp2i9jx98l' = str('/tmp/tmp2i9jx98l')

zarr/tests/test_storage.py:1155: AssertionError
__________________ TestFSStoreWithKeySeparator.test_hierarchy __________________

self = <zarr.tests.test_storage.TestFSStoreWithKeySeparator object at 0x764c45b26810>

    def test_hierarchy(self):
        # setup
        store = self.create_store()
        store[self.root + "a"] = b"aaa"
        store[self.root + "b"] = b"bbb"
        store[self.root + "c/d"] = b"ddd"
        store[self.root + "c/e/f"] = b"fff"
        store[self.root + "c/e/g"] = b"ggg"
    
        # check keys
        assert self.root + "a" in store
        assert self.root + "b" in store
        assert self.root + "c/d" in store
        assert self.root + "c/e/f" in store
        assert self.root + "c/e/g" in store
        assert self.root + "c" not in store
        assert self.root + "c/" not in store
        assert self.root + "c/e" not in store
        assert self.root + "c/e/" not in store
        assert self.root + "c/d/x" not in store
    
        # check __getitem__
        with pytest.raises(KeyError):
            store[self.root + "c"]
        with pytest.raises(KeyError):
            store[self.root + "c/e"]
        with pytest.raises(KeyError):
            store[self.root + "c/d/x"]
    
        # test getsize (optional)
        if hasattr(store, "getsize"):
            # TODO: proper behavior of getsize?
            #       v3 returns size of all nested arrays, not just the
            #       size of the arrays in the current folder.
            if self.version == 2:
                assert 6 == store.getsize()
            else:
                assert 15 == store.getsize()
            assert 3 == store.getsize("a")
            assert 3 == store.getsize("b")
            if self.version == 2:
                assert 3 == store.getsize("c")
            else:
                assert 9 == store.getsize("c")
            assert 3 == store.getsize("c/d")
            assert 6 == store.getsize("c/e")
            assert 3 == store.getsize("c/e/f")
            assert 3 == store.getsize("c/e/g")
            # non-existent paths
            assert 0 == store.getsize("x")
            assert 0 == store.getsize("a/x")
            assert 0 == store.getsize("c/x")
            assert 0 == store.getsize("c/x/y")
            assert 0 == store.getsize("c/d/y")
            assert 0 == store.getsize("c/d/y/z")
    
            # access item via full path
            assert 3 == store.getsize(self.root + "a")
    
        # test listdir (optional)
        if hasattr(store, "listdir"):
            assert {"a", "b", "c"} == set(store.listdir(self.root))
            assert {"d", "e"} == set(store.listdir(self.root + "c"))
            assert {"f", "g"} == set(store.listdir(self.root + "c/e"))
            # no exception raised if path does not exist or is leaf
            assert [] == store.listdir(self.root + "x")
            assert [] == store.listdir(self.root + "a/x")
            assert [] == store.listdir(self.root + "c/x")
            assert [] == store.listdir(self.root + "c/x/y")
            assert [] == store.listdir(self.root + "c/d/y")
            assert [] == store.listdir(self.root + "c/d/y/z")
>           assert [] == store.listdir(self.root + "c/e/f")
E           AssertionError: assert [] == ['f']
E             
E             Right contains one more item: 'f'
E             Use -v to get more diff

zarr/tests/test_storage.py:409: AssertionError
___________________ TestFSStoreFromFilesystem.test_hierarchy ___________________

self = <zarr.tests.test_storage.TestFSStoreFromFilesystem object at 0x764c45bb6120>

    def test_hierarchy(self):
        # setup
        store = self.create_store()
        store[self.root + "a"] = b"aaa"
        store[self.root + "b"] = b"bbb"
        store[self.root + "c/d"] = b"ddd"
        store[self.root + "c/e/f"] = b"fff"
        store[self.root + "c/e/g"] = b"ggg"
    
        # check keys
        assert self.root + "a" in store
        assert self.root + "b" in store
        assert self.root + "c/d" in store
        assert self.root + "c/e/f" in store
        assert self.root + "c/e/g" in store
        assert self.root + "c" not in store
        assert self.root + "c/" not in store
        assert self.root + "c/e" not in store
        assert self.root + "c/e/" not in store
        assert self.root + "c/d/x" not in store
    
        # check __getitem__
        with pytest.raises(KeyError):
            store[self.root + "c"]
        with pytest.raises(KeyError):
            store[self.root + "c/e"]
        with pytest.raises(KeyError):
            store[self.root + "c/d/x"]
    
        # test getsize (optional)
        if hasattr(store, "getsize"):
            # TODO: proper behavior of getsize?
            #       v3 returns size of all nested arrays, not just the
            #       size of the arrays in the current folder.
            if self.version == 2:
                assert 6 == store.getsize()
            else:
                assert 15 == store.getsize()
            assert 3 == store.getsize("a")
            assert 3 == store.getsize("b")
            if self.version == 2:
                assert 3 == store.getsize("c")
            else:
                assert 9 == store.getsize("c")
            assert 3 == store.getsize("c/d")
            assert 6 == store.getsize("c/e")
            assert 3 == store.getsize("c/e/f")
            assert 3 == store.getsize("c/e/g")
            # non-existent paths
            assert 0 == store.getsize("x")
            assert 0 == store.getsize("a/x")
            assert 0 == store.getsize("c/x")
            assert 0 == store.getsize("c/x/y")
            assert 0 == store.getsize("c/d/y")
            assert 0 == store.getsize("c/d/y/z")
    
            # access item via full path
            assert 3 == store.getsize(self.root + "a")
    
        # test listdir (optional)
        if hasattr(store, "listdir"):
            assert {"a", "b", "c"} == set(store.listdir(self.root))
            assert {"d", "e"} == set(store.listdir(self.root + "c"))
            assert {"f", "g"} == set(store.listdir(self.root + "c/e"))
            # no exception raised if path does not exist or is leaf
            assert [] == store.listdir(self.root + "x")
            assert [] == store.listdir(self.root + "a/x")
            assert [] == store.listdir(self.root + "c/x")
            assert [] == store.listdir(self.root + "c/x/y")
            assert [] == store.listdir(self.root + "c/d/y")
            assert [] == store.listdir(self.root + "c/d/y/z")
>           assert [] == store.listdir(self.root + "c/e/f")
E           AssertionError: assert [] == ['f']
E             
E             Right contains one more item: 'f'
E             Use -v to get more diff

zarr/tests/test_storage.py:409: AssertionError
_________________________ TestN5FSStore.test_hierarchy _________________________

self = <zarr.tests.test_storage.TestN5FSStore object at 0x764c45b47e30>

    def test_hierarchy(self):
        # setup
        store = self.create_store()
        store[self.root + "a"] = b"aaa"
        store[self.root + "b"] = b"bbb"
        store[self.root + "c/d"] = b"ddd"
        store[self.root + "c/e/f"] = b"fff"
        store[self.root + "c/e/g"] = b"ggg"
    
        # check keys
        assert self.root + "a" in store
        assert self.root + "b" in store
        assert self.root + "c/d" in store
        assert self.root + "c/e/f" in store
        assert self.root + "c/e/g" in store
        assert self.root + "c" not in store
        assert self.root + "c/" not in store
        assert self.root + "c/e" not in store
        assert self.root + "c/e/" not in store
        assert self.root + "c/d/x" not in store
    
        # check __getitem__
        with pytest.raises(KeyError):
            store[self.root + "c"]
        with pytest.raises(KeyError):
            store[self.root + "c/e"]
        with pytest.raises(KeyError):
            store[self.root + "c/d/x"]
    
        # test getsize (optional)
        if hasattr(store, "getsize"):
            # TODO: proper behavior of getsize?
            #       v3 returns size of all nested arrays, not just the
            #       size of the arrays in the current folder.
            if self.version == 2:
                assert 6 == store.getsize()
            else:
                assert 15 == store.getsize()
            assert 3 == store.getsize("a")
            assert 3 == store.getsize("b")
            if self.version == 2:
                assert 3 == store.getsize("c")
            else:
                assert 9 == store.getsize("c")
            assert 3 == store.getsize("c/d")
            assert 6 == store.getsize("c/e")
            assert 3 == store.getsize("c/e/f")
            assert 3 == store.getsize("c/e/g")
            # non-existent paths
            assert 0 == store.getsize("x")
            assert 0 == store.getsize("a/x")
            assert 0 == store.getsize("c/x")
            assert 0 == store.getsize("c/x/y")
            assert 0 == store.getsize("c/d/y")
            assert 0 == store.getsize("c/d/y/z")
    
            # access item via full path
            assert 3 == store.getsize(self.root + "a")
    
        # test listdir (optional)
        if hasattr(store, "listdir"):
            assert {"a", "b", "c"} == set(store.listdir(self.root))
            assert {"d", "e"} == set(store.listdir(self.root + "c"))
            assert {"f", "g"} == set(store.listdir(self.root + "c/e"))
            # no exception raised if path does not exist or is leaf
            assert [] == store.listdir(self.root + "x")
            assert [] == store.listdir(self.root + "a/x")
            assert [] == store.listdir(self.root + "c/x")
            assert [] == store.listdir(self.root + "c/x/y")
            assert [] == store.listdir(self.root + "c/d/y")
            assert [] == store.listdir(self.root + "c/d/y/z")
>           assert [] == store.listdir(self.root + "c/e/f")
E           AssertionError: assert [] == ['f']
E             
E             Right contains one more item: 'f'
E             Use -v to get more diff

zarr/tests/test_storage.py:409: AssertionError
__________________________ TestN5FSStore.test_complex __________________________

self = <zarr.tests.test_storage.TestN5FSStore object at 0x764c45b445c0>

    def test_complex(self):
        path1 = tempfile.mkdtemp()
        path2 = tempfile.mkdtemp()
        store = self.create_store(
            path="simplecache::file://" + path1,
            simplecache={"same_names": True, "cache_storage": path2},
        )
        assert not store
        assert not os.listdir(path1)
        assert not os.listdir(path2)
        store[self.root + "foo"] = b"hello"
        assert "foo" in os.listdir(str(path1) + "/" + self.root)
        assert self.root + "foo" in store
>       assert not os.listdir(str(path2))
E       AssertionError: assert not ['foo']
E        +  where ['foo'] = <built-in function listdir>('/tmp/tmpz0cyks9e')
E        +    where <built-in function listdir> = os.listdir
E        +    and   '/tmp/tmpz0cyks9e' = str('/tmp/tmpz0cyks9e')

zarr/tests/test_storage.py:1155: AssertionError
_______________________ TestNestedFSStore.test_hierarchy _______________________

self = <zarr.tests.test_storage.TestNestedFSStore object at 0x764c45b67740>

    def test_hierarchy(self):
        # setup
        store = self.create_store()
        store[self.root + "a"] = b"aaa"
        store[self.root + "b"] = b"bbb"
        store[self.root + "c/d"] = b"ddd"
        store[self.root + "c/e/f"] = b"fff"
        store[self.root + "c/e/g"] = b"ggg"
    
        # check keys
        assert self.root + "a" in store
        assert self.root + "b" in store
        assert self.root + "c/d" in store
        assert self.root + "c/e/f" in store
        assert self.root + "c/e/g" in store
        assert self.root + "c" not in store
        assert self.root + "c/" not in store
        assert self.root + "c/e" not in store
        assert self.root + "c/e/" not in store
        assert self.root + "c/d/x" not in store
    
        # check __getitem__
        with pytest.raises(KeyError):
            store[self.root + "c"]
        with pytest.raises(KeyError):
            store[self.root + "c/e"]
        with pytest.raises(KeyError):
            store[self.root + "c/d/x"]
    
        # test getsize (optional)
        if hasattr(store, "getsize"):
            # TODO: proper behavior of getsize?
            #       v3 returns size of all nested arrays, not just the
            #       size of the arrays in the current folder.
            if self.version == 2:
                assert 6 == store.getsize()
            else:
                assert 15 == store.getsize()
            assert 3 == store.getsize("a")
            assert 3 == store.getsize("b")
            if self.version == 2:
                assert 3 == store.getsize("c")
            else:
                assert 9 == store.getsize("c")
            assert 3 == store.getsize("c/d")
            assert 6 == store.getsize("c/e")
            assert 3 == store.getsize("c/e/f")
            assert 3 == store.getsize("c/e/g")
            # non-existent paths
            assert 0 == store.getsize("x")
            assert 0 == store.getsize("a/x")
            assert 0 == store.getsize("c/x")
            assert 0 == store.getsize("c/x/y")
            assert 0 == store.getsize("c/d/y")
            assert 0 == store.getsize("c/d/y/z")
    
            # access item via full path
            assert 3 == store.getsize(self.root + "a")
    
        # test listdir (optional)
        if hasattr(store, "listdir"):
            assert {"a", "b", "c"} == set(store.listdir(self.root))
            assert {"d", "e"} == set(store.listdir(self.root + "c"))
            assert {"f", "g"} == set(store.listdir(self.root + "c/e"))
            # no exception raised if path does not exist or is leaf
            assert [] == store.listdir(self.root + "x")
            assert [] == store.listdir(self.root + "a/x")
            assert [] == store.listdir(self.root + "c/x")
            assert [] == store.listdir(self.root + "c/x/y")
            assert [] == store.listdir(self.root + "c/d/y")
            assert [] == store.listdir(self.root + "c/d/y/z")
>           assert [] == store.listdir(self.root + "c/e/f")
E           AssertionError: assert [] == ['f']
E             
E             Right contains one more item: 'f'
E             Use -v to get more diff

zarr/tests/test_storage.py:409: AssertionError
=============================== warnings summary ===============================
zarr/tests/test_storage.py::TestZipStore::test_store_and_retrieve_ndarray
  /usr/lib/python3.12/zipfile/__init__.py:1607: UserWarning: Duplicate name: 'foo'
    return self._open_to_write(zinfo, force_zip64=force_zip64)

zarr/tests/test_sync.py: 32 warnings
  /usr/lib/python3.12/multiprocessing/popen_fork.py:66: DeprecationWarning: This process (pid=46096) is multi-threaded, use of fork() may lead to deadlocks in the child.
    self.pid = os.fork()

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
============================= slowest 10 durations =============================

[...]

=========================== short test summary info ============================
FAILED zarr/tests/test_filters.py::test_array_with_delta_filter - numpy.core....
FAILED zarr/tests/test_storage.py::TestFSStore::test_hierarchy - AssertionErr...
FAILED zarr/tests/test_storage.py::TestFSStore::test_complex - AssertionError...
FAILED zarr/tests/test_storage.py::TestFSStoreWithKeySeparator::test_hierarchy
FAILED zarr/tests/test_storage.py::TestFSStoreFromFilesystem::test_hierarchy
FAILED zarr/tests/test_storage.py::TestN5FSStore::test_hierarchy - AssertionE...
FAILED zarr/tests/test_storage.py::TestN5FSStore::test_complex - AssertionErr...
FAILED zarr/tests/test_storage.py::TestNestedFSStore::test_hierarchy - Assert...
= 8 failed, 2580 passed, 2581 skipped, 5 xfailed, 33 warnings in 116.89s (0:01:56) =

QuLogic · 2024-12-23T23:47:20Z

Please not that, in addition to the fsspec related ones, there is also an error in test_array_with_delta_filter that started appearing recently.

This is a NumPy 2 change, I think.

yurivict added the bug Potential issues with the zarr-python library label Apr 26, 2024

jhamman added the help wanted Issue could use help from someone with familiarity on the topic label May 17, 2024

yurivict mentioned this issue May 27, 2024

7 tests fail #1916

Closed

jhamman mentioned this issue Jun 7, 2024

Basic working FsspecStore #1785

Merged

6 tasks

martindurant mentioned this issue Jul 20, 2024

Hierarchy test fail with fsspec 2024.2.0 #1679

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

7 tests fail #1819

7 tests fail #1819

yurivict commented Apr 26, 2024

avalentino commented May 1, 2024

avalentino commented May 1, 2024

jhamman commented May 17, 2024

martindurant commented Jun 7, 2024

jhamman commented Jun 7, 2024

QuLogic commented Jun 14, 2024

yurivict commented Sep 19, 2024

avalentino commented Nov 27, 2024

QuLogic commented Dec 23, 2024

7 tests fail #1819

7 tests fail #1819

Comments

yurivict commented Apr 26, 2024

Zarr version

Numcodecs version

Python Version

Operating System

Installation

Description

Steps to reproduce

Additional output

avalentino commented May 1, 2024

avalentino commented May 1, 2024

jhamman commented May 17, 2024

martindurant commented Jun 7, 2024

jhamman commented Jun 7, 2024

QuLogic commented Jun 14, 2024

yurivict commented Sep 19, 2024

avalentino commented Nov 27, 2024

QuLogic commented Dec 23, 2024