Rockstar halos of rockstar main version cannot be opened

[arkordt]

Bug report

Cannot load rockstar binary file in revision 2, version 0.99.9-RC3+.

Digging into the commit history of https://github.com/yt-project/yt/blob/main/yt/frontends/rockstar/definitions.py, the halo definition of revision 2 seems to suitable for the rockstar-galaxies version https://github.com/jwise77/rockstar-galaxies but not the base version. If the respective lines in definitions.py are commented out, loading data works fine. When using #4997, loading data works again, but probably only accidentally. I did not spend any more time on identifying differences there except that the field ('halos', 'halfmass_radius') is missing (it is only included in the rockstar base version).

Suggested solution: the rockstar galaxies version prints version 0.99.9-RC3-newA into the respective header field. This could be used in #4997 as an additional source of information. However, the halo_dts in definitions.py can no longer be constructed from revision number only.

If #4997 will be merged, the I am willing to contribute for solving this issue.

Code for reproduction

Download the data from http://use.yt/upload/ae37f6a2 and unpack the tar.gz archive

import yt
ds = yt.load('[path to uncompressed data]/halos_168.0.bin')
ds.all_data()

Actual outcome

/Users/aronkordt/code/yt/yt/frontends/rockstar/data_structures.py:42: RuntimeWarning: overflow encountered in scalar multiply
  expected_end = self._member_offset + 8 * self._Npart.sum()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/aronkordt/code/yt/yt/data_objects/static_output.py", line 1190, in all_data
    self.index
  File "/Users/aronkordt/code/yt/yt/data_objects/static_output.py", line 604, in index
    self._instantiated_index = self._index_class(
                               ^^^^^^^^^^^^^^^^^^
  File "/Users/aronkordt/code/yt/yt/geometry/particle_geometry_handler.py", line 72, in __init__
    super().__init__(ds, dataset_type)
  File "/Users/aronkordt/code/yt/yt/geometry/geometry_handler.py", line 46, in __init__
    self._detect_output_fields()
  File "/Users/aronkordt/code/yt/yt/geometry/particle_geometry_handler.py", line 360, in _detect_output_fields
    pcounts = self._get_particle_type_counts()
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/aronkordt/code/yt/yt/geometry/particle_geometry_handler.py", line 86, in _get_particle_type_counts
    for df in self.data_files:
              ^^^^^^^^^^^^^^^
  File "/Users/aronkordt/code/yt/yt/geometry/particle_geometry_handler.py", line 106, in data_files
    self._setup_filenames()
  File "/Users/aronkordt/code/yt/yt/geometry/particle_geometry_handler.py", line 140, in _setup_filenames
    df = cls(self.dataset, self.io, _filename, fi, (start, end))
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/aronkordt/code/yt/yt/frontends/rockstar/data_structures.py", line 44, in __init__
    raise RuntimeError(
RuntimeError: File size 258208 does not match expected size -1643523978899944832.

Expected outcome

yt : [INFO     ] 2024-10-26 11:24:32,538 Allocating for 1.291e+04 particles
Loading particle index: 100%|████████████████████████████████████████████████████████████| 32/32 [00:00<00:00, 9879.12it/s]
YTRegion (halos_168.0.bin): , center=[1.0551713e+23 1.0551713e+23 1.0551713e+23] cm, left_edge=[0. 0. 0.] cm, right_edge=[2.11034261e+23 2.11034261e+23 2.11034261e+23] cm

Version Information

• Operating System: MacOS Sonoma 14.6.1, or RedHat 8.5.1
• Python Version: 3.12.6
• yt version: current main branch
• Other Libraries (if applicable):

yt installed from source