Can't read part with intersecting bounds

[vincentsarago]

reported in developmentseed/titiler#640 (reply in thread)

I'm getting that error consistently with the following url:

https://{something}/cog/bbox/-72.39661926483915,-14.198810143003932,-72.26331881334586,-14.066961918019524/595x842.tif?url=https://{something}/mantis/l1c/2024/08/03/MANTIS_L1C_021900730139_20240803T151319_20240803T151321/TCI_COG.tiff

I've tried several things:
- Adjusting bbox values
- Not adding bbox
- Using query parameters (such as zoom, max_size, nodata, etc)

Nothings seems to be working.

The info endpoint returns the following:

Driver: GTiff/GeoTIFF
Files: mantis.tif
Size is 7532, 7428
Coordinate System is:
GEOGCRS["WGS 84",
    DATUM["World Geodetic System 1984",
        ELLIPSOID["WGS 84",6378137,298.257223563,
            LENGTHUNIT["metre",1]]],
    PRIMEM["Greenwich",0,
        ANGLEUNIT["degree",0.0174532925199433]],
    CS[ellipsoidal,2],
        AXIS["geodetic latitude (Lat)",north,
            ORDER[1],
            ANGLEUNIT["degree",0.0174532925199433]],
        AXIS["geodetic longitude (Lon)",east,
            ORDER[2],
            ANGLEUNIT["degree",0.0174532925199433]],
    ID["EPSG",4326]]
Data axis to CRS axis mapping: 2,1
GeoTransform =
  -72.39661926483915, 1.523991680507554e-05, 2.492379929651992e-06
  -14.16868824977417, -3.999189223282901e-06, 1.369498273487359e-05
Metadata:
  AREA_OR_POINT=Area
Image Structure Metadata:
  COMPRESSION=LZW
  INTERLEAVE=PIXEL
  LAYOUT=COG
Corner Coordinates:
Upper Left  ( -72.3966193, -14.1686882) ( 72d23'47.83"W, 14d10' 7.28"S)
Lower Left  ( -72.3781059, -14.0669619) ( 72d22'41.18"W, 14d 4' 1.06"S)
Upper Right ( -72.2818322, -14.1988101) ( 72d16'54.60"W, 14d11'55.72"S)
Lower Right ( -72.2633188, -14.0970838) ( 72d15'47.95"W, 14d 5'49.50"S)
Center      ( -72.3299690, -14.1328860) ( 72d19'47.89"W, 14d 7'58.39"S)
Band 1 Block=512x512 Type=Byte, ColorInterp=Undefined
  NoData Value=0
  Overviews: 3766x3714, 1883x1857, 941x928, 470x464
Band 2 Block=512x512 Type=Byte, ColorInterp=Undefined
  NoData Value=0
  Overviews: 3766x3714, 1883x1857, 941x928, 470x464
Band 3 Block=512x512 Type=Byte, ColorInterp=Undefined
  NoData Value=0
  Overviews: 3766x3714, 1883x1857, 941x928, 470x464
  
  The bounds endpoint returns the following:
  
  {'bounds': [-72.39661926483915, -14.198810143003932, -72.26331881334586, -14.066961918019524]}
  
  The statistics endpoint returns the following:
  
  {'b1': {'min': 2.0, 'max': 255.0, 'mean': 120.4366812365454, 'count': 946702.0, 'sum': 114017647.0, 'std': 37.32024805775883, 'median': 121.0, 'majority': 121.0, 'minority': 2.0, 'unique': 254.0, 'histogram': [[6759.0, 25467.0, 83966.0, 184967.0, 259873.0, 216997.0, 111765.0, 41689.0, 13035.0, 2184.0], [2.0, 27.3, 52.6, 77.9, 103.2, 128.5, 153.8, 179.1, 204.4, 229.70000000000002, 255.0]], 'valid_percent': 91.54, 'masked_pixels': 87538.0, 'valid_pixels': 946702.0, 'percentile_2': 44.0, 'percentile_98': 201.0}, 'b2': {'min': 1.0, 'max': 255.0, 'mean': 111.99736559128426, 'count': 946702.0, 'sum': 106028130.0, 'std': 34.50480622081493, 'median': 113.0, 'majority': 118.0, 'minority': 1.0, 'unique': 255.0, 'histogram': [[7252.0, 31979.0, 112193.0, 207180.0, 284018.0, 204372.0, 72250.0, 21912.0, 4764.0, 782.0], [1.0, 26.4, 51.8, 77.19999999999999, 102.6, 128.0, 153.39999999999998, 178.79999999999998, 204.2, 229.6, 255.0]], 'valid_percent': 91.54, 'masked_pixels': 87538.0, 'valid_pixels': 946702.0, 'percentile_2': 40.0, 'percentile_98': 185.0}, 'b3': {'min': 1.0, 'max': 255.0, 'mean': 107.7169785212242, 'count': 946702.0, 'sum': 101975879.0, 'std': 32.5602988773908, 'median': 109.0, 'majority': 115.0, 'minority': 247.0, 'unique': 255.0, 'histogram': [[8383.0, 36193.0, 122873.0, 228991.0, 295296.0, 189146.0, 51156.0, 12027.0, 2297.0, 340.0], [1.0, 26.4, 51.8, 77.19999999999999, 102.6, 128.0, 153.39999999999998, 178.79999999999998, 204.2, 229.6, 255.0]], 'valid_percent': 91.54, 'masked_pixels': 87538.0, 'valid_pixels': 946702.0, 'percentile_2': 38.0, 'percentile_98': 174.0}}
  
  The preview endpoint works fine!
  

Any idea of what might be happening?

ps - If I use only the last part of the url (https://{something}/mantis/l1c/2024/08/03/MANTIS_L1C_021900730139_20240803T151319_20240803T151321/TCI_COG.tiff) the image is properly downloaded

Thank you in advance :)

_Originally posted by @MisterTiago in https://github.com/developmentseed/titiler/discussions/640#discussioncomment-10636946_

Issue: User is trying to read bbox with bounds intersecting the database bounds (boundless read)

Solutions:

• user WarpedVRT
• customize the windows.from_bounds method from rasterio to avoid causing issue

cc @MisterTiago

[vincentsarago]

🤔 I think I miss understood the issue.

• the bbox passed by the user doesn't cross the dataset bounds (it's the dataset bounds 😓)
• when using a bounds that is fully contained with the dataset bounds it also fails

from affine import Affine
from rasterio.windows import from_bounds

transform = Affine(
    1.5239916805075538e-05, 2.4923799296519917e-06, -72.39661926483915,
    -3.999189223282901e-06, 1.3694982734873585e-05, -14.168688249774165,
    0.0, 0.0, 1.0,
)

dataset_bounds = [-72.39661926483915, -14.198810143003932, -72.26331881334586, -14.066961918019524]
from_bounds(*dataset_bounds, transform)
>> WindowError: Bounds and transform are inconsistent

within_bounds = [-72.30, -14.15, -72.28, -14.10]
from_bounds(*within_bounds, transform)
>> WindowError: Bounds and transform are inconsistent

Notes:

• There is something going on with the transform of the dataset, the transform shows there is rotation and I'm
• This might be a rasterio bug

[MisterTiago]

Hi @vincentsarago,

Should this be an issue opened in rasterio repo?

[vincentsarago]

Already opened an issue rasterio/rasterio#3176

[MisterTiago]

Hi @vincentsarago ,

I noticed the issue you opened is now closed. I was on vacations last week so I'm a bit lost here haha

Would you mind putting me up to date on this issue please?

Thank you :)

[vincentsarago]

@MisterTiago the issue needs more deep diving to really understand what's going on.

There is something with the file geo transform but I can't be 100% sure that's the behaviour is expected or not/

How did the file was created?

[MisterTiago]

"How did the file was created?"

Are you asking for the packages used in creating the file or something else? The file was created using a script that uses rasterio (amongst other things)

[vincentsarago]

Are you asking for the packages used in creating the file or something else? The file was created using a script that uses rasterio (amongst other things)

Yes, the fact the transform is uncommon (having rotation) is making things a bit weird.

[MisterTiago]

Do you want to know how the transform is being calculated for this image?

[MisterTiago]

HI @vincentsarago ,

Apologies for taking so long to reply, been caught on other issues.

Regarding the transform, we use a simple pinhole camera model which generates GCPs for the 4 corners from which the affine is made.

Does this answer your question?

[MisterTiago]

Hi @vincentsarago ,

Did you have time to take a deeper look into the issue?

[vincentsarago]

Hi @MisterTiago

Sadly I did not. Got a lot of projects work at the moment so open-source maintenance is not the priority sadly.

[MisterTiago]

Do you think there's someone we can tag that might be able to look into this?

[vincentsarago]

not that I can think off

I still think there is something with rasterio as I've tried to explained in rasterio/rasterio#3176

But my understanding of Affine transform is limited so if you or if you find someone with knowledge in affine transform, maybe you can see if there is really an issue with rasterio, or if it's with your file.

I would also try to replicate what you're trying to do with gdalwarp command (which is a nice way to tell that maybe rasterio has a bug)

[MisterTiago]

Thank you for the feedback @vincentsarago!

I'll have your comment into consideration. I appreciate the help you provided so far and best of luck with your current projects :)

[MisterTiago]

Hi @vincentsarago,

Just want to let you know that I've applied the command gdalwarp -t_srs EPSG:4326 -of COG \<input\> \<output\> and my action now works with the output image.