Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Min requirements CI failing on Windows #1190

Closed
rly opened this issue Feb 11, 2020 · 1 comment · Fixed by #1194
Closed

Min requirements CI failing on Windows #1190

rly opened this issue Feb 11, 2020 · 1 comment · Fixed by #1194
Labels
category: bug errors in the code or code behavior

Comments

@rly
Copy link
Contributor

rly commented Feb 11, 2020

The Azure test for python 3.5 using the minimum requirements is failing due to the following error. It happens non-deterministically. It is probably a weird interaction between h5py, windows, and pandas.

hdf5.test_nwbfile.TestElectrodes
    +----------------+---------------------------------------------------------+-------------+
    |   Test name    |                          Output                         |    Status   |
    +----------------+---------------------------------------------------------+-------------+
    | test_roundtrip | ValueError: cannot expose native-only dtype 'g' in non- | error       |
    |                | native byte order '<' via buffer interface              |             |
    |                |                                                         |             |
    +----------------+---------------------------------------------------------+-------------+
ERROR in test test_roundtrip (hdf5.test_nwbfile.TestElectrodes):
Traceback (most recent call last):
File "D:\a\1\s\tests\integration\hdf5\test_nwbfile.py", line 459, in test_roundtrip
  pd.testing.assert_frame_equal(self.read_container[0].drop('group', axis=1),
File "D:\a\1\s\.tox\py35-min-req\lib\site-packages\pandas\core\frame.py", line 3694, in drop
  errors=errors)
File "D:\a\1\s\.tox\py35-min-req\lib\site-packages\pandas\core\generic.py", line 3108, in drop
  obj = obj._drop_axis(labels, axis, level=level, errors=errors)
File "D:\a\1\s\.tox\py35-min-req\lib\site-packages\pandas\core\generic.py", line 3141, in _drop_axis
  dropped = self.reindex(**{axis_name: new_axis})
File "D:\a\1\s\.tox\py35-min-req\lib\site-packages\pandas\util\_decorators.py", line 186, in wrapper
  return func(*args, **kwargs)
File "D:\a\1\s\.tox\py35-min-req\lib\site-packages\pandas\core\frame.py", line 3563, in reindex
  return super(DataFrame, self).reindex(**kwargs)
File "D:\a\1\s\.tox\py35-min-req\lib\site-packages\pandas\core\generic.py", line 3685, in reindex
  fill_value, copy).__finalize__(self)
File "D:\a\1\s\.tox\py35-min-req\lib\site-packages\pandas\core\frame.py", line 3493, in _reindex_axes
  fill_value, limit, tolerance)
File "D:\a\1\s\.tox\py35-min-req\lib\site-packages\pandas\core\frame.py", line 3518, in _reindex_columns
  allow_dups=False)
File "D:\a\1\s\.tox\py35-min-req\lib\site-packages\pandas\core\generic.py", line 3806, in _reindex_with_indexers
  copy=copy)
File "D:\a\1\s\.tox\py35-min-req\lib\site-packages\pandas\core\internals.py", line 4421, in reindex_indexer
  fill_tuple=(fill_value,))
File "D:\a\1\s\.tox\py35-min-req\lib\site-packages\pandas\core\internals.py", line 4501, in _slice_take_blocks_ax0
  fill_tuple=None))
File "D:\a\1\s\.tox\py35-min-req\lib\site-packages\pandas\core\internals.py", line 1254, in take_nd
  allow_fill=False)
File "D:\a\1\s\.tox\py35-min-req\lib\site-packages\pandas\core\algorithms.py", line 1656, in take_nd
  func(arr, indexer, out, fill_value)
File "pandas\_libs\algos_take_helper.pxi", line 4629, in pandas._libs.algos.take_2d_axis1_float64_float64
File "stringsource", line 653, in View.MemoryView.memoryview_cwrapper
File "stringsource", line 348, in View.MemoryView.memoryview.__cinit__
ValueError: cannot expose native-only dtype 'g' in non-native byte order '<' via buffer interface
@rly rly added the category: bug errors in the code or code behavior label Feb 11, 2020
rly added a commit that referenced this issue Feb 11, 2020
@rly
Copy link
Contributor Author

rly commented Feb 20, 2020

This error is apparently resolved in h5py 2.10. It is due (again) to an h5py/Windows on Python 3.5 interaction where floating point values are read stochastically as np.float128 (dtype code 'g') instead of np.float64, and pandas is not happy with that. See h5py/h5py#1051 and h5py/h5py#1134 for more information.

I cannot think of a clean way to resolve it in HDMF, so in the meantime, I am going to make the minimum requirements CI run on Python 3.6 rather than 3.5. Python 3.5 end of life is 2020-09-13. Since Python 3.6 has been out for three years and Python 3.8 is out now, I think we can safely start encouraging users to upgrade from Python 3.5.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
category: bug errors in the code or code behavior
Projects
None yet
1 participant