Opening of ENVI stack with long line in header fails

[felixcremer]

I am not sure, whether this is an ArchGDAL or an GDAL issue, but opening here as I encountered it while using ArchGDAL.
Opening an ENVI stack with a very long line in the header file, fails with

AG.read("/home/crem_fe/Daten/S1_Hidalgo_timestack_20km_VV___lin_20_test_tandemdem12")
ERROR: GDALError (CE_Failure, code 1):
	Maximum number of characters allowed reached.

Stacktrace:
 [1] gdaljl_errorhandler(::GDAL.CPLErr, ::Int32, ::Cstring) at /home/crem_fe/.julia/packages/GDAL/cghbO/src/error.jl:36
 [2] gdalopenex(::String, ::Int64, ::Ptr{Cstring}, ::Ptr{Cstring}, ::Ptr{Cstring}) at /home/crem_fe/.julia/packages/GDAL/cghbO/src/gdal_h.jl:472
 [3] #read#25 at /home/crem_fe/.julia/packages/ArchGDAL/JhHWz/src/dataset.jl:388 [inlined]
 [4] read(::String) at /home/crem_fe/.julia/packages/ArchGDAL/JhHWz/src/dataset.jl:388
 [5] top-level scope at REPL[4]:1

I can open the same file with the python GDAL bindings.
There was an issue in GDAL for this
OSGeo/gdal#506 but this should be fixed by now.

If I manually insert line breaks between the bandnames the file can also be opened in ArchGDAL, but it would be nice, I won't have to do this.

[visr]

Did you try this in Python with or without UseExceptions() as described in https://gdal.org/api/python_gotchas.html#python-bindings-do-not-raise-exceptions-unless-you-explicitly-call-useexceptions?

Because it may just that we throw an error where Python doesn't (but probably should).

[felixcremer]

I just tested it with UseExceptions() in python and it can be opened.

Is there a way to test the underlying GDAL installation?
Trying to use the binary in the artifacts folder leads to linking issues and I am not sure, what I should do about them.

[visr]

Ok, interesting.

Is there a way to test the underlying GDAL installation?

I wanted to link to the GDAL.jl readme for this, but it is outdated now as well: JuliaGeo/GDAL.jl#86

But if you do this through the Artifacts, it should not have any linking issues.

[felixcremer]

I tested the underlying GDAL installation by

run(`$(GDAL_jll.gdalinfo_path_path) $path`)

before this works, you need to set your LD_LIBRARY_PATH to the lib folder of your PROJ_jll artifacts_dir.
If I do this with my global gdalinfo I get the correct band names as a description, but If I does this with the julia installed GDAL, the descreptions are empty.
I also checked it from outside of julia by using the gdalinfo in the artifacts_dir of GDAL.
Therefore, I think this is some build issue of GDAL, but I am not sure where this happens.

[visr]

There was an issue in GDAL for this OSGeo/gdal#506 but this should be fixed by now.

This only increased the maximum line length from 256 to 1024. How long is your header line?

[felixcremer]

I just tested it and it seems as if opening the stack breaks as soon as the header line is longer than 10000 characters.
Opening the following stack breaks for me, If I combine the three band names lines.
small_envistack.zip
But I can open this with ArchGDAL as soon as I break the line into two with less then 10000 characters.
But this would indicate, that this is not related to OSGeo/gdal#506.

[visr]

Ha ok. Did you check if the 1024 has since been updated in GDAL?

[felixcremer]

I was wrong, the issue OSGeo/gdal#506 is the same error, but for a different dataset.
The line limit is set for the ENVI stacks here and it seems to be 10000:
https://github.com/OSGeo/gdal/blob/9bad539ca0bd003da39286d78498681c36e9bdbd/gdal/frmts/raw/envidataset.cpp#L1935
I still don't understand why I could have opened it in python.
Unfortunately, I can't test it in python at the moment.

[visr]

Ok, but I guesss it is fair to say that the error is expected here then, right? Should we close this issue? I don't think there is much we can do here, if this is what GDAL does.