IndexError: list index out of range in geometry reading

[akhoubani]

We encountered an IndexError: list index out of range in geometry reading for loading nek5000 field files. readnek works fine for some dumped field files, while fails for some other files with the same geometry. I share two field files, one working and one broken.

This field file works fine.
cbox0.f00298.txt

While the following gives the error.
cbox0.f00299.txt

home/Dev/miniconda3/lib/python3.8/site-packages/pymech/neksuite.py in readnek(fname)
    139     # read geometry
    140     for iel in elmap:
--> 141         el = data.elem[iel - 1]
    142         for idim in range(var[0]):  # if var[0] == 0, geometry is not read
    143             read_file_into_data(el.pos, idim)

IndexError: list index out of range

[ashwinvis]

Can you check using the visnek command and Paraview if the files can be viewed?

[akhoubani]

Yes, they can be viewed by Paraview.

[akhoubani]

We should consider if we put .xyz. = f in the .usr file, the geometry will not be written except for the first dumped file, but the Paraview can read the geometry (mesh). It means, Paraview only needs one file for geometry.

Pymech considers the geometry for each different dumped file. If it would be an option that Pymech only reads the first dumped file for geometry, I think it can solve the issue.

[guillaumechauvat]

Pymech will only read the geometry if it's present in the file. There is usually no problem reading a file containing only velocity.

If you look at the first 132 bytes of the file representing the header, it contains something like "XU", or "XUP" or similar if there is a geometry, and "U" or "UP" or similar if there isn't.

Your problem looks like it has to do with the ordering of elements. There is a reordering going on inside Nek that I don't really understand, you can have two files with the same mesh but the elements are in a different order. This is what the elmap is for.

[guillaumechauvat]

If you want to use the geometry in file 1 for all subsequent files, you can do it manually: get the mesh from file 1, store it and use it for processing other files. There is no notion of file set right now in pymech.

[akhoubani]

I am changing the mesh in the usrdat2 subroutine of .usr file.

I am not sure, but maybe it could be related to not calling gen_re2(0) subroutine after modifying the mesh. I am investigating it. I'll let you know.

Edit: It is not related to gen_re2(0) subroutine.

[akhoubani]

Something strange is happening. When I run a simulation on 1 node and 10 cores of DELL C6320 I don't encounter this problem, while for 1 node and 20 cores, I have this issue of IndexError: list index out of range.

[guillaumechauvat]

The element map (the permutation table describing in which order the elements are stored) seems broken in your second file.

In your first file (00298), it is:

601, 602, 603, 604, 633, 634, 635, 636, 637, 665, 666, 667, 668, 670, 697, 698, 699, 700, 701, 702, 703, 704, 729, 730, 731, 732, 733, 734, 735, 736, 761, 762, 763, 764, 765, 766, 767, 768, 409, 410, 411, 412, 441, 442, 443, 444, 473, 474, 475, 476, 505, 506, 507, 508, 537, 538, 539, 540, 541, 542, 569, 570, 571, 572, 573, 574, 605, 606, 607, 608, 638, 639, 640, 669, 671, 672, 413, 414, 415, 416, 445, 446, 447, 448, 477, 478, 479, 480, 509, 510, 511, 512, 543, 544, 575, 576, 593, 594, 625, 626, 657, 658, 689, 690, 691, 692, 721, 722, 723, 724, 753, 754, 755, 756, 497, 498, 529, 530, 561, 562, 563, 564, 595, 596, 597, 598, 599, 600, 627, 628, 629, 630, 631, 632, 659, 660, 661, 662, 663, 664, 693, 694, 695, 696, 725, 726, 727, 728, 757, 758, 759, 760, 401, 402, 403, 404, 405, 406, 433, 434, 435, 436, 437, 438, 439, 440, 465, 466, 467, 468, 469, 470, 471, 472, 499, 500, 501, 502, 503, 504, 531, 532, 533, 534, 535, 536, 565, 566, 567, 568, 217, 218, 219, 220, 249, 250, 251, 252, 281, 282, 283, 284, 313, 314, 315, 316, 317, 318, 319, 320, 345, 346, 347, 348, 349, 350, 351, 352, 377, 378, 379, 380, 381, 382, 383, 384, 407, 408, 25, 26, 27, 28, 57, 58, 59, 60, 89, 90, 91, 92, 121, 122, 123, 124, 153, 154, 155, 156, 185, 186, 187, 188, 189, 190, 221, 222, 223, 224, 253, 254, 255, 256, 285, 286, 287, 288, 29, 30, 31, 32, 61, 62, 63, 64, 93, 94, 95, 96, 125, 126, 127, 128, 157, 158, 159, 160, 191, 192, 241, 242, 273, 274, 305, 306, 307, 308, 337, 338, 339, 340, 369, 370, 371, 372, 113, 114, 145, 146, 177, 178, 209, 210, 211, 212, 213, 214, 215, 216, 243, 244, 245, 246, 247, 248, 275, 276, 277, 278, 279, 280, 309, 310, 311, 312, 341, 342, 343, 344, 373, 374, 375, 376, 17, 18, 19, 20, 21, 22, 49, 50, 51, 52, 53, 54, 55, 81, 82, 83, 84, 85, 86, 88, 115, 116, 117, 118, 119, 120, 147, 148, 149, 150, 151, 152, 179, 180, 181, 182, 183, 184, 23, 24, 56, 87, 585, 586, 587, 588, 617, 618, 619, 620, 649, 650, 651, 652, 681, 682, 683, 684, 685, 686, 713, 714, 715, 716, 717, 718, 719, 720, 745, 746, 747, 748, 749, 750, 751, 752, 393, 394, 395, 396, 425, 426, 427, 428, 457, 458, 459, 460, 489, 490, 491, 492, 521, 522, 523, 524, 553, 554, 555, 556, 589, 590, 591, 592, 621, 622, 623, 624, 653, 654, 655, 656, 687, 688, 397, 398, 399, 400, 429, 430, 431, 432, 461, 462, 463, 464, 493, 494, 495, 496, 525, 526, 527, 528, 557, 558, 559, 560, 609, 641, 642, 673, 674, 675, 676, 705, 706, 707, 708, 737, 738, 739, 740, 481, 482, 513, 514, 545, 546, 577, 578, 579, 580, 581, 582, 583, 584, 610, 611, 612, 613, 614, 615, 616, 643, 644, 645, 646, 647, 648, 677, 678, 679, 680, 709, 710, 711, 712, 741, 742, 743, 744, 385, 386, 387, 388, 389, 390, 417, 418, 419, 420, 421, 422, 423, 449, 450, 451, 452, 453, 454, 455, 456, 483, 484, 485, 486, 487, 488, 515, 516, 517, 518, 519, 520, 547, 548, 549, 550, 551, 552, 201, 202, 203, 204, 233, 234, 235, 236, 265, 266, 267, 268, 297, 298, 299, 300, 301, 302, 303, 304, 329, 330, 331, 332, 333, 334, 335, 336, 361, 362, 363, 364, 365, 366, 367, 368, 391, 392, 424, 9, 10, 11, 12, 41, 42, 43, 44, 73, 74, 75, 76, 105, 106, 107, 108, 137, 138, 139, 140, 141, 169, 170, 171, 172, 173, 174, 205, 206, 207, 208, 237, 238, 239, 240, 269, 270, 271, 272, 13, 14, 15, 16, 45, 46, 47, 48, 77, 78, 79, 80, 109, 110, 111, 112, 142, 143, 144, 175, 176, 193, 194, 225, 226, 257, 258, 289, 290, 291, 292, 321, 322, 323, 324, 353, 354, 355, 356, 97, 98, 129, 130, 131, 161, 162, 163, 164, 195, 196, 197, 198, 199, 200, 227, 228, 229, 230, 231, 232, 259, 260, 261, 262, 263, 264, 293, 294, 295, 296, 325, 326, 327, 328, 357, 358, 359, 360, 1, 2, 3, 4, 5, 6, 7, 8, 33, 34, 35, 36, 37, 38, 39, 40, 65, 66, 67, 68, 69, 70, 71, 72, 99, 100, 101, 102, 103, 104, 132, 133, 134, 135, 136, 165, 166, 167, 168

Here, each number between 0 and 767 is present exactly once.

In your second file, it is instead

601, 602, 603, 604, 633, 634, 635, 636, 637, 665, 666, 667, 668, 670, 697, 698, 699, 700, 701, 702, 703, 704, 729, 730, 731, 732, 733, 734, 735, 736, 761, 762, 763, 764, 765, 766, 767, 768, 409, 410, 411, 412, 441, 442, 443, 444, 473, 474, 475, 476, 505, 506, 507, 508, 537, 538, 539, 540, 541, 542, 569, 570, 571, 572, 573, 574, 605, 606, 607, 608, 638, 639, 640, 669, 671, 672, 413, 414, 415, 416, 445, 446, 447, 448, 477, 478, 479, 480, 509, 510, 511, 512, 543, 544, 575, 576, 593, 594, 625, 626, 657, 658, 689, 690, 691, 692, 721, 722, 723, 724, 753, 754, 755, 756, 497, 498, 529, 530, 561, 562, 563, 564, 595, 596, 597, 598, 599, 600, 627, 628, 629, 630, 631, 632, 659, 660, 661, 662, 663, 664, 693, 694, 695, 696, 725, 726, 727, 728, 757, 758, 759, 760, 401, 402, 403, 404, 405, 406, 433, 434, 435, 436, 437, 438, 439, 440, 465, 466, 467, 468, 469, 470, 471, 472, 499, 500, 501, 502, 503, 504, 531, 532, 533, 534, 535, 536, 565, 566, 567, 568, 217, 218, 219, 220, 249, 250, 251, 252, 281, 282, 283, 284, 313, 314, 315, 316, 317, 318, 319, 320, 345, 346, 347, 348, 349, 350, 351, 352, 377, 378, 379, 380, 381, 382, 383, 384, 407, 408, 25, 26, 27, 28, 57, 58, 59, 60, 89, 90, 91, 92, 121, 122, 123, 124, 153, 154, 155, 156, 185, 186, 187, 188, 189, 190, 221, 222, 223, 224, 253, 254, 255, 256, 285, 286, 287, 288, 29, 30, 31, 32, 61, 62, 63, 64, 93, 94, 95, 96, 125, 126, 127, 128, 157, 158, 159, 160, 191, 192, 241, 242, 273, 274, 305, 306, 307, 308, 337, 338, 339, 340, 369, 370, 371, 372, 113, 114, 145, 146, 177, 178, 209, 210, 211, 212, 213, 214, 215, 216, 243, 244, 245, 246, 247, 248, 275, 276, 277, 278, 279, 280, 309, 310, 311, 312, 341, 342, 343, 344, 373, 374, 375, 376, 17, 18, 19, 20, 21, 22, 49, 50, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 2577, 0, 1870113760, 32563, 1870113760, 32563, 0, 0, 0, 0, 65536, 1, 0, 0, 176, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 6, 7, 8, 33, 34, 35, 36, 37, 38, 39, 40, 65, 66, 67, 68, 69, 70, 71, 72, 99, 100, 101, 102, 103, 104, 132, 133, 134, 135, 136, 165, 166, 167, 168

Note all the zeros and very large numbers in the second half...

[guillaumechauvat]

I checked with hexdump, there seems to be a lot of zeros in the file itself, so I don't think it's a problem with pymech. But we could add a check and display a better error message.

It's really confusing though. I've never seen it before. Is this straight out of Nek5000? Could you please share your .usr file?

[akhoubani]

I checked with hexdump, there seems to be a lot of zeros in the file itself, so I don't think it's a problem with pymech. But we could add a check and display a better error message.

It's really confusing though. I've never seen it before. Is this straight out of Nek5000? Could you please share your .usr file?

Thank you very much.

Yes, it is strange that we have this reordering of elements. You could see .usr file here. In fact we are using snek5000-cbox.

[ashwinvis]

@akhoubani Did you encounter this error again? Should we prioritize it or fix it later?

[akhoubani]

@ashwinvis Because I encountered this error for simulations on 20 cores (2*10), I limited the simulations to 10 cores. It would be nice to prioritize it.

However, I encountered another error for simulations done on 20 cores related to the dataset when using pymech.dataset.open_dataset("cbox0.f00030").

File ~/useful/project/20CONVECTION/miniconda3/lib/python3.9/site-packages/pymech/dataset.py:36, in open_dataset(path, **kwargs)
     33 else:
     34     raise NotImplementedError(f"Filetype: {path.suffix} is not supported.")
---> 36 return _open(path, **kwargs)

File ~/useful/project/20CONVECTION/miniconda3/lib/python3.9/site-packages/pymech/dataset.py:152, in _open_nek_dataset(path)
    150 elements = field.elem
    151 elem_stores = [_NekDataStore(elem) for elem in elements]
--> 152 elem_dsets = [
    153     xr.Dataset.load_store(store).set_coords(store.axes) for store in elem_stores
    154 ]
    156 # See: https://github.com/MITgcm/xmitgcm/pull/200
    157 if xr.__version__ < "0.15.2":

File ~/useful/project/20CONVECTION/miniconda3/lib/python3.9/site-packages/pymech/dataset.py:153, in <listcomp>(.0)
    150 elements = field.elem
    151 elem_stores = [_NekDataStore(elem) for elem in elements]
    152 elem_dsets = [
--> 153     xr.Dataset.load_store(store).set_coords(store.axes) for store in elem_stores
    154 ]
    156 # See: https://github.com/MITgcm/xmitgcm/pull/200
    157 if xr.__version__ < "0.15.2":

File ~/useful/project/20CONVECTION/miniconda3/lib/python3.9/site-packages/xarray/core/dataset.py:770, in Dataset.load_store(cls, store, decoder)
    768 if decoder:
    769     variables, attributes = decoder(variables, attributes)
--> 770 obj = cls(variables, attrs=attributes)
    771 obj.set_close(store.close)
    772 return obj

File ~/useful/project/20CONVECTION/miniconda3/lib/python3.9/site-packages/xarray/core/dataset.py:750, in Dataset.__init__(self, data_vars, coords, attrs)
    747 if isinstance(coords, Dataset):
    748     coords = coords.variables
--> 750 variables, coord_names, dims, indexes, _ = merge_data_and_coords(
    751     data_vars, coords, compat="broadcast_equals"
    752 )
    754 self._attrs = dict(attrs) if attrs is not None else None
    755 self._close = None

File ~/useful/project/20CONVECTION/miniconda3/lib/python3.9/site-packages/xarray/core/merge.py:483, in merge_data_and_coords(data, coords, compat, join)
    481 explicit_coords = coords.keys()
    482 indexes = dict(_extract_indexes_from_coords(coords))
--> 483 return merge_core(
    484     objects, compat, join, explicit_coords=explicit_coords, indexes=indexes
    485 )

File ~/useful/project/20CONVECTION/miniconda3/lib/python3.9/site-packages/xarray/core/merge.py:640, in merge_core(objects, compat, join, combine_attrs, priority_arg, explicit_coords, indexes, fill_value)
    635 variables, out_indexes = merge_collected(
    636     collected, prioritized, compat=compat, combine_attrs=combine_attrs
    637 )
    638 assert_unique_multiindex_level_names(variables)
--> 640 dims = calculate_dimensions(variables)
    642 coord_names, noncoord_names = determine_coords(coerced)
    643 if explicit_coords is not None:

File ~/useful/project/20CONVECTION/miniconda3/lib/python3.9/site-packages/xarray/core/dataset.py:204, in calculate_dimensions(variables)
    202             last_used[dim] = k
    203         elif dims[dim] != size:
--> 204             raise ValueError(
    205                 f"conflicting sizes for dimension {dim!r}: "
    206                 f"length {size} on {k!r} and length {dims[dim]} on {last_used!r}"
    207             )
    208 return dims

ValueError: conflicting sizes for dimension 'y': length 10 on 'xmesh' and length 1 on {'x': 'x', 'y': 'y', 'z': 'z'}

[akhoubani]

I share three field files for the same simulation:

Without error:
cbox0.f00027.txt

Error corresponding to conflicting sizes:
cbox0.f00028.txt

Error corresponding to index:
cbox0.f00029.txt

[ashwinvis]

@akhoubani I have to de-prioritize this issue and #56 . Hopefully these are very rare.