resolve index file madness?

[edwardhartnett]

wgrib2 has an index format: https://www.cpc.ncep.noaa.gov/products/wesley/wgrib2/grb2_mk_inv.html

NCEPLIBS-g2/g2c has an index format: https://noaa-emc.github.io/NCEPLIBS-g2/g2index_8F90.html#ae24e3633f74e1d49d787fa19b07f716c

After doing a bunch of work on that I saw the netCDF-Java solution, which was just to take the whole GRIB2 file except the data sections. This has the advantage of preserving all metadata, even metadata which is newly developed by WMO. In other words, it's a forwards-compatible solution, which is very attractive. And it means the same code can be used to read the index file as the data file, which is nice.

@webisu what does your index file contain? Is it documented anywhere?

[webisu]

The wgrib2 index is simply the text output wgrib2 FILE.grb >FILE.idx The index file and be augmented by other quantities, ex wgrib2 FILE.grb -center >FILE.idx Wgrib2 has options like -if STRING -not_if STRING -elseif STRING -match STRING -not STRING The STRING is compared with the match_inv. So the match_inv is like a super index that is generated internally. The various string comparisons assume that STRING is a regex. Since regex meta-characters can be a problem for some operations, a *_fs does a "fixed string" comparison. (No meta-characters). Nomads uses the wgrib2 index files. Having text based index files has worked out well. The public have taken the simple provided shell script and made customized versions for their requirements. Wgrib2 can use differently that than other programs. It can be used to direct the order that the grib messages are read. To only process HGT wgrib2 FILE.grb | grep HGT | wgrib2 -i FILE.grb -grib HGT.grb To sort records wgrib2 FILE.grb | (sort program) | wgrib2 -i FILE.grb -grib SORTED.grb

…

On Thu, Jul 25, 2024 at 11:10 AM Edward Hartnett ***@***.***> wrote: wgrib2 has an index format: https://www.cpc.ncep.noaa.gov/products/wesley/wgrib2/grb2_mk_inv.html NCEPLIBS-g2/g2c has an index format: https://noaa-emc.github.io/NCEPLIBS-g2/g2index_8F90.html#ae24e3633f74e1d49d787fa19b07f716c After doing a bunch of work on that I saw the netCDF-Java solution, which was just to take the whole GRIB2 file except the data sections. This has the advantage of preserving all metadata, even metadata which is newly developed by WMO. In other words, it's a forwards-compatible solution, which is very attractive. And it means the same code can be used to read the index file as the data file, which is nice. @webisu <https://github.com/webisu> what does your index file contain? Is it documented anywhere? — Reply to this email directly, view it on GitHub <#736>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AEIB7ZRWGA6MTXSJK5MRID3ZOEIPJAVCNFSM6AAAAABLOYL4DOVHI2DSMVQWIX3LMV43ASLTON2WKOZSGQZTAMRWGE2DGNY> . You are receiving this because you were assigned.Message ID: ***@***.***>