RuntimeError: Can't rename attribute (record is already in B-tree)

[cgohlke]

I get one test error on Windows after rebuilding h5py-2.9.0 with Cython-0.29.6 and HDF5-1.10.5 (any Python version).

To assist reproducing bugs, please include the following:

• Operating System: Windows 10
• Python version: any
• Where Python was acquired: Python.org
• h5py version: 2.9.0
• HDF5 version: 1.10.5
• The full traceback/stack trace shown (if it appears)

Summary of the h5py configuration
---------------------------------

h5py    2.9.0
HDF5    1.10.5
Python  3.7.2 (tags/v3.7.2:9a3ffc0492, Dec 23 2018, 23:09:28) [MSC v.1916 64 bit (AMD64)]
sys.platform    win32
sys.maxsize     9223372036854775807
numpy   1.15.4

...........E...........................................x.................................s......................................x....................................sX:\Python37\lib\site-packages\h5py\_hl\files.py:258: H5pyDeprecationWarning: File.fid has been deprecated. Use File.id instead.
  "Use File.id instead.", H5pyDeprecationWarning)
...s......ss.ss...............................................................................ss...................ssssss.....................................................................x....x.........................x......x..................................................ssss............................................................
======================================================================
ERROR: test_track_order (h5py.tests.old.test_attrs.TestTrackOrder)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "X:\Python37\lib\site-packages\h5py\tests\old\test_attrs.py", line 180, in test_track_order
    attrs = self.fill_attrs(track_order=True)  # creation order
  File "X:\Python37\lib\site-packages\h5py\tests\old\test_attrs.py", line 174, in fill_attrs
    attrs[str(i)] = i
  File "h5py\_objects.pyx", line 54, in h5py._objects.with_phil.wrapper
  File "h5py\_objects.pyx", line 55, in h5py._objects.with_phil.wrapper
  File "X:\Python37\lib\site-packages\h5py\_hl\attrs.py", line 95, in __setitem__
    self.create(name, data=value, dtype=base.guess_dtype(value))
  File "X:\Python37\lib\site-packages\h5py\_hl\attrs.py", line 210, in create
    h5a.rename(self._id, self._e(tempname), self._e(name))
  File "h5py\_objects.pyx", line 54, in h5py._objects.with_phil.wrapper
  File "h5py\_objects.pyx", line 55, in h5py._objects.with_phil.wrapper
  File "h5py\h5a.pyx", line 118, in h5py.h5a.rename
RuntimeError: Can't rename attribute (record is already in B-tree)

----------------------------------------------------------------------
Ran 509 tests in 2.525s

[greyltc]

I see the exact same error in Arch Linux. The bug title should probably be changed to something like "HDF5-1.10.5 breaks h5py"

[aparamon]

Hmm, it's another manifestation of HDFFV-10579:
#1098 (comment)

Please put some pressure on HDF Group via forum; we cannot fix it in h5py, unfortunately.

[greyltc]

So,

1. The h5py developers are unaffiliated with and unsupported by the HDF5 developers.
2. The HDF5 developers have made a stable release that breaks h5py.
3. There is no reasonable change that can be made to h5py to fix this breakage.

Am I understanding this correctly?

[epourmal]

Hello, h5py developers!

I just answered in another thread about our plans to address the issue (in HDF5 1.12.0 planned for May 2019 and in HDF5 1.10.6 planned for November 2019).

The HDF Group developers try to support h5py community as much as we can. I don't think we break h5py code constantly, but I agree it became more often in the past two years due to our stretched resources. I hope we should find ways to work together and address the issues as soon as HDF5 libraries under development break h5py code.

Does h5py community has CI running that uses HDF5 code under development?

FYI: Attributes creation ordering, tracking, dense storage, etc. were introduce in HDF5 1.8.0 (2008). The bug was filed in September 2018. i.e., 10 years later.

Our Bitbucket is open. We have public CDash ; the results of h5py CI can be posted there.

Please contact me directly [email protected] if you are interested in discussing what h5py developers and The HDF Group could do to catch and address the issues in a timely manner and not when HDF5 is released.

Thank you!

Elena

[tacaswell]

This thread has gotten a bit contentious. I believe everyone here is operating in good faith but with constrained resources. Things fall through the cracks despite everyone's best intentions and efforts.

---

It looks like we had a version check on the failing test

h5py/h5py/tests/old/test_attrs.py

Lines 181 to 183 in 0cc2c24

	@ut.skipUnless(h5py.version.hdf5_version_tuple >= (1, 10, 5), 'HDF5 1.10.5 required')
	# https://forum.hdfgroup.org/t/bug-h5arename-fails-unexpectedly/4881
	def test_track_order(self):

being optimistic that it would be fixed in 1.10.5.

The quick fix for this is to bump the version check.

---

Currently h5py CI tests the development branch of h5py against a range of released versions of hdf5 [1] and install hdf5 from source if we have to [2]. I think it is reasonable to add a allowed-fail to our test suite that will install from a development version. Can we have some guidance on what branch we should pull from or is there a weekly/nightly snapshot we should pull instead? It seems like a good idea to push to the hdf5 cdash from that test (maybe not on every PR build?). travis has a way to encode whatever secrets we would need to be able to push out to the hdf5 cdash server.

However, we do not regularly (re)test the released version of h5py, it may make sense for hdfgoup to include a matrix of {py27, py36, py37}x{h5py28, h5py29} in the hdf5 test suite.

@epourmal I'll also follow up via email.

[1] https://github.com/h5py/h5py/blob/master/.travis.yml
[2] https://github.com/h5py/h5py/blob/master/ci/travis/get_hdf5_if_needed.sh

[tacaswell]

@cgohlke You may want to apply the patch from #1186 . Does not really fix the problem, but gets back to where we were with hdf5 1.10.4.

[kmuehlbauer]

See upstream HDFGroup/hdf5#1388 for details.