Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pandas <1.0.5 is not compatible with numpy 1.20 #617

Closed
5 tasks done
rly opened this issue May 19, 2021 · 0 comments · Fixed by #618
Closed
5 tasks done

pandas <1.0.5 is not compatible with numpy 1.20 #617

rly opened this issue May 19, 2021 · 0 comments · Fixed by #618
Labels
category: bug errors in the code or code behavior

Comments

@rly
Copy link
Contributor

rly commented May 19, 2021

Description

With pandas==0.25.3 and numpy==1.20.3, python test.py reveals 9 errors, such as:

======================================================================
ERROR: test_getitem_int (common.test_table.DynamicTableRegionRoundTrip)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/mnt/c/Users/Ryan/Documents/NWB/hdmf/tests/unit/common/test_table.py", line 918, in test_getitem_int
    rec = self._getitem(0)
  File "/mnt/c/Users/Ryan/Documents/NWB/hdmf/tests/unit/common/test_table.py", line 891, in _getitem
    return table[arg]
  File "/home/rly/miniconda3/envs/test/lib/python3.8/site-packages/hdmf/common/table.py", line 784, in __getitem__
    ret = self.get(key)
  File "/home/rly/miniconda3/envs/test/lib/python3.8/site-packages/hdmf/common/table.py", line 838, in get
    ret[name] = col.get(arg, df=df, **kwargs)
  File "/home/rly/miniconda3/envs/test/lib/python3.8/site-packages/hdmf/common/table.py", line 1091, in get
    ret = self.table.get(ret, df=df, **kwargs)
  File "/home/rly/miniconda3/envs/test/lib/python3.8/site-packages/hdmf/common/table.py", line 890, in get
    ret = pd.DataFrame(retdf, index=pd.Index(name=self.id.name, data=id_index))
  File "/home/rly/miniconda3/envs/test/lib/python3.8/site-packages/pandas/core/frame.py", line 411, in __init__
    mgr = init_dict(data, index, columns, dtype=dtype)
  File "/home/rly/miniconda3/envs/test/lib/python3.8/site-packages/pandas/core/internals/construction.py", line 257, in init_dict
    return arrays_to_mgr(arrays, data_names, index, columns, dtype=dtype)
  File "/home/rly/miniconda3/envs/test/lib/python3.8/site-packages/pandas/core/internals/construction.py", line 82, in arrays_to_mgr
    arrays = _homogenize(arrays, index, dtype)
  File "/home/rly/miniconda3/envs/test/lib/python3.8/site-packages/pandas/core/internals/construction.py", line 322, in _homogenize
    val = sanitize_array(
  File "/home/rly/miniconda3/envs/test/lib/python3.8/site-packages/pandas/core/internals/construction.py", line 712, in sanitize_array
    subarr = construct_1d_arraylike_from_scalar(value, len(index), dtype)
  File "/home/rly/miniconda3/envs/test/lib/python3.8/site-packages/pandas/core/dtypes/cast.py", line 1233, in construct_1d_arraylike_from_scalar
    subarr = np.empty(length, dtype=dtype)
TypeError: Cannot interpret '<attribute 'dtype' of 'numpy.generic' objects>' as a data type

There are also many DeprecationWarnings between pandas and numpy such as:

/home/rly/miniconda3/envs/test/lib/python3.8/site-packages/pandas/core/indexes/base.py:459: DeprecationWarning: `np.bool` is a deprecated alias for the builtin `bool`. To silence this warning, use `bool` by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use `np.bool_` here.
Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
/home/rly/miniconda3/envs/test/lib/python3.8/site-packages/pandas/core/internals/construction.py:588: DeprecationWarning: `np.object` is a deprecated alias for the builtin `object`. To silence this warning, use `object` by itself. Doing this will not modify any behavior and is safe.
Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations

See also numpy/numpy#18355

We can either set an upper bound on numpy to <1.20 or set a lower bound on pandas to >=1.0.5. Or just note this as an incompatible environment configuration in the README. I would prefer to upgrade pandas requirements so that we continue to get the latest updates with numpy.

Steps to Reproduce

Environment

Python Executable: Conda
Python Version: 3.8
Operating System: Windows, Linux
HDMF Version: 2.5.5

Checklist

  • Have you ensured the bug was not already reported ?
  • Have you included a brief and descriptive title?
  • Have you included a clear description of the problem you are trying to solve?
  • Have you included a minimal code snippet that reproduces the issue you are encountering?
  • Have you checked our Contributing document?
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
category: bug errors in the code or code behavior
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant