Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Investigate higher HEAD/GET Request with GDAL >3.7 #697

Open
vincentsarago opened this issue Apr 18, 2024 · 12 comments · Fixed by OSGeo/gdal#10957
Open

Investigate higher HEAD/GET Request with GDAL >3.7 #697

vincentsarago opened this issue Apr 18, 2024 · 12 comments · Fixed by OSGeo/gdal#10957
Labels
bug

Comments

@vincentsarago
Copy link
Member

with https://github.com/cogeotiff/rio-tiler/blob/main/tests/benchmarks/requests.py we test some tile request fetching and use tilebench to monitor the number or GET/HEAD requests made.

with the latest commits, this test is failing

rio-tiler/tests/benchmarks/requests.py

Lines 35 to 67 in 4d95c28

def test_tile_read():
"""Tile Read tests."""
@profile(
kernels=True,
config={
"GDAL_HTTP_MERGE_CONSECUTIVE_RANGES": "YES",
"GDAL_DISABLE_READDIR_ON_OPEN": "EMPTY_DIR",
"GDAL_INGESTED_BYTES_AT_OPEN": 32768,
"CPL_VSIL_CURL_NON_CACHED": f"/vsicurl/{dataset_url}",
},
quiet=True,
add_to_return=True,
)
def tile(src_path: str, x: int, y: int, z: int):
with Reader(src_path) as src_dst:
return src_dst.tile(x, y, z, tilesize=256)
_, stats = tile(
dataset_url,
493,
741,
11,
)
assert stats["HEAD"]["count"] == 1
assert stats["GET"]["count"] == 3
assert stats["GET"]["bytes"] == 2293760
assert stats["GET"]["ranges"] == [
"0-32767",
"9011200-10158079",
"11075584-12189695",
]
assert stats["WarpKernels"]

👇 this used to return HEAD: 1 and GET 3 🤷

tilebench profile https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif --tile 11-493-741 | jq
{
  "HEAD": {
    "count": 3
  },
  "GET": {
    "count": 4,
    "bytes": 2293760,
    "ranges": [
      "0-32767",
      "9011200-10158079",
      "11075584-12189695"
    ]
  },
  "Timing": 3.494839668273926
}
@vincentsarago
Copy link
Member Author

vincentsarago commented Apr 18, 2024 •
edited
Loading

Rasterio 1.3.9 / GDAL 3.6.4

tilebench profile https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif --tile 11-493-741 | jq
{
  "HEAD": {
    "count": 1
  },
  "GET": {
    "count": 3,
    "bytes": 2293760,
    "ranges": [
      "0-32767",
      "9011200-10158079",
      "11075584-12189695"
    ]
  },
  "Timing": 2.6649599075317383
}

Rasterio 1.3.10 / GDAL 3.7.3

{
  "HEAD": {
    "count": 3
  },
  "GET": {
    "count": 4,
    "bytes": 2293760,
    "ranges": [
      "0-32767",
      "9011200-10158079",
      "11075584-12189695"
    ]
  },
  "Timing": 3.2888236045837402
}

Rasterio 1.3.10 / GDAL 3.8.4

 tilebench profile https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif --tile 11-493-741 | jq
{
  "HEAD": {
    "count": 3
  },
  "GET": {
    "count": 4,
    "bytes": 2293760,
    "ranges": [
      "0-32767",
      "9011200-10158079",
      "11075584-12189695"
    ]
  },
  "Timing": 2.7414448261260986
}

CURL Logs

tilebench profile https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif --tile 11-493-741 --add-stdout | jq -r '.logs[]' | grep "CURL_INFO_HEADER_OUT"
CPLE_None in CURL_INFO_HEADER_OUT: HEAD /sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif HTTP/1.1
CPLE_None in CURL_INFO_HEADER_OUT: GET /sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif HTTP/1.1
CPLE_None in CURL_INFO_HEADER_OUT: GET /sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/ HTTP/1.1
CPLE_None in CURL_INFO_HEADER_OUT: HEAD /sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.xml HTTP/1.1
CPLE_None in CURL_INFO_HEADER_OUT: HEAD /sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.XML HTTP/1.1
CPLE_None in CURL_INFO_HEADER_OUT: GET /sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif HTTP/1.1
CPLE_None in CURL_INFO_HEADER_OUT: GET /sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif HTTP/1.1

tilebench profile https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif --tile 11-493-741 --add-stdout | jq '.logs'
[
  "Entering env context: <rasterio.env.Env object at 0x13fa59460>",
  "Starting outermost env",
  "No GDAL environment exists",
  "New GDAL environment <rasterio._env.GDALEnv object at 0x13fa599d0> created",
  "Installing FilePath filesystem handler plugin...",
  "GDAL_DATA found in environment.",
  "PROJ_LIB found in environment.",
  "Started GDALEnv: self=<rasterio._env.GDALEnv object at 0x13fa599d0>.",
  "Entered env context: <rasterio.env.Env object at 0x13fa59460>",
  "Got a copy of environment <rasterio._env.GDALEnv object at 0x13fa599d0> options",
  "Entering env context: <rasterio.env.Env object at 0x13fad4820>",
  "Got a copy of environment <rasterio._env.GDALEnv object at 0x13fa599d0> options",
  "Entered env context: <rasterio.env.Env object at 0x13fad4820>",
  "Sharing flag: 0",
  "CPLE_None in HTTP: libcurl/8.4.0 (SecureTransport) LibreSSL/3.3.6 zlib/1.2.12 nghttp2/1.58.0",
  "CPLE_None in HTTP: GDAL was built against curl 7.87.0, but is running against 8.4.0.",
  "CPLE_None in CURL_INFO_TEXT: Couldn't find host sentinel-cogs.s3.us-west-2.amazonaws.com in the .netrc file; using defaults",
  "CPLE_None in CURL_INFO_TEXT:   Trying 52.218.232.177:443...",
  "CPLE_None in CURL_INFO_TEXT: Connected to sentinel-cogs.s3.us-west-2.amazonaws.com (52.218.232.177) port 443",
  "CPLE_None in CURL_INFO_TEXT: ALPN: curl offers h2,http/1.1",
  "CPLE_None in CURL_INFO_TEXT: (304) (OUT), TLS handshake, Client hello (1):",
  "CPLE_None in CURL_INFO_TEXT:  CAfile: /Users/vincentsarago/Dev/venv/py39/lib/python3.9/site-packages/certifi/cacert.pem",
  "CPLE_None in CURL_INFO_TEXT:  CApath: none",
  "CPLE_None in CURL_INFO_TEXT: (304) (IN), TLS handshake, Server hello (2):",
  "CPLE_None in CURL_INFO_TEXT: TLSv1.2 (IN), TLS handshake, Certificate (11):",
  "CPLE_None in CURL_INFO_TEXT: TLSv1.2 (IN), TLS handshake, Server key exchange (12):",
  "CPLE_None in CURL_INFO_TEXT: TLSv1.2 (IN), TLS handshake, Server finished (14):",
  "CPLE_None in CURL_INFO_TEXT: TLSv1.2 (OUT), TLS handshake, Client key exchange (16):",
  "CPLE_None in CURL_INFO_TEXT: TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):",
  "CPLE_None in CURL_INFO_TEXT: TLSv1.2 (OUT), TLS handshake, Finished (20):",
  "CPLE_None in CURL_INFO_TEXT: TLSv1.2 (IN), TLS change cipher, Change cipher spec (1):",
  "CPLE_None in CURL_INFO_TEXT: TLSv1.2 (IN), TLS handshake, Finished (20):",
  "CPLE_None in CURL_INFO_TEXT: SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256",
  "CPLE_None in CURL_INFO_TEXT: ALPN: server accepted http/1.1",
  "CPLE_None in CURL_INFO_TEXT: Server certificate:",
  "CPLE_None in CURL_INFO_TEXT:  subject: CN=*.s3-us-west-2.amazonaws.com",
  "CPLE_None in CURL_INFO_TEXT:  start date: Mar 15 00:00:00 2024 GMT",
  "CPLE_None in CURL_INFO_TEXT:  expire date: Feb 15 23:59:59 2025 GMT",
  "CPLE_None in CURL_INFO_TEXT:  subjectAltName: host \"sentinel-cogs.s3.us-west-2.amazonaws.com\" matched cert's \"*.s3.us-west-2.amazonaws.com\"",
  "CPLE_None in CURL_INFO_TEXT:  issuer: C=US; O=Amazon; CN=Amazon RSA 2048 M01",
  "CPLE_None in CURL_INFO_TEXT:  SSL certificate verify ok.",
  "CPLE_None in CURL_INFO_TEXT: using HTTP/1.1",
  "CPLE_None in CURL_INFO_HEADER_OUT: HEAD /sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif HTTP/1.1",
  "Host: sentinel-cogs.s3.us-west-2.amazonaws.com",
  "User-Agent: GDAL/3.7.3",
  "Accept: */*",
  "",
  "CPLE_None in CURL_INFO_HEADER_IN: HTTP/1.1 200 OK",
  "CPLE_None in CURL_INFO_HEADER_IN: x-amz-id-2: glv7Xi1AW/YS7UJsgsQAoYokXq/gB35c8LMEJYo2FLKpI+o1tR6xsyfWKou6TgrfZgcuY2ooDYQ=",
  "CPLE_None in CURL_INFO_HEADER_IN: x-amz-request-id: 2VMQ56MHK9DHVN5M",
  "CPLE_None in CURL_INFO_HEADER_IN: Date: Thu, 18 Apr 2024 13:54:32 GMT",
  "CPLE_None in CURL_INFO_HEADER_IN: Last-Modified: Mon, 09 Oct 2023 01:23:09 GMT",
  "CPLE_None in CURL_INFO_HEADER_IN: ETag: \"3e23880cd7ac8a32ae71c55ad4f85c5d-24\"",
  "CPLE_None in CURL_INFO_HEADER_IN: x-amz-storage-class: INTELLIGENT_TIERING",
  "CPLE_None in CURL_INFO_HEADER_IN: x-amz-server-side-encryption: AES256",
  "CPLE_None in CURL_INFO_HEADER_IN: Cache-Control: public, max-age=31536000, immutable",
  "CPLE_None in CURL_INFO_HEADER_IN: Accept-Ranges: bytes",
  "CPLE_None in CURL_INFO_HEADER_IN: Content-Type: image/tiff; application=geotiff; profile=cloud-optimized",
  "CPLE_None in CURL_INFO_HEADER_IN: Server: AmazonS3",
  "CPLE_None in CURL_INFO_HEADER_IN: Content-Length: 200959614",
  "CPLE_None in CURL_INFO_HEADER_IN: ",
  "CPLE_None in CURL_INFO_TEXT: Connection #0 to host sentinel-cogs.s3.us-west-2.amazonaws.com left intact",
  "CPLE_None in VSICURL: GetFileSize(https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif)=200959614  response_code=200",
  "CPLE_None in VSICURL: Downloading 0-32767 (https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif)...",
  "CPLE_None in CURL_INFO_TEXT: Couldn't find host sentinel-cogs.s3.us-west-2.amazonaws.com in the .netrc file; using defaults",
  "CPLE_None in CURL_INFO_TEXT: Found bundle for host: 0x13f8f81e0 [serially]",
  "CPLE_None in CURL_INFO_TEXT: Can not multiplex, even if we wanted to",
  "CPLE_None in CURL_INFO_TEXT: Re-using existing connection with host sentinel-cogs.s3.us-west-2.amazonaws.com",
  "CPLE_None in CURL_INFO_HEADER_OUT: GET /sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif HTTP/1.1",
  "Host: sentinel-cogs.s3.us-west-2.amazonaws.com",
  "User-Agent: GDAL/3.7.3",
  "Accept: */*",
  "Range: bytes=0-32767",
  "",
  "CPLE_None in CURL_INFO_HEADER_IN: HTTP/1.1 206 Partial Content",
  "CPLE_None in CURL_INFO_HEADER_IN: x-amz-id-2: pTwUP0oEk8NsXKYMEHOeoWlAF5VlM6s/F4HLQPcdYMBbkPD13uqFkzOzdrCLv/JkFVg1LMJdeIc=",
  "CPLE_None in CURL_INFO_HEADER_IN: x-amz-request-id: 2VMYTCRG7WAEDXPJ",
  "CPLE_None in CURL_INFO_HEADER_IN: Date: Thu, 18 Apr 2024 13:54:32 GMT",
  "CPLE_None in CURL_INFO_HEADER_IN: Last-Modified: Mon, 09 Oct 2023 01:23:09 GMT",
  "CPLE_None in CURL_INFO_HEADER_IN: ETag: \"3e23880cd7ac8a32ae71c55ad4f85c5d-24\"",
  "CPLE_None in CURL_INFO_HEADER_IN: x-amz-storage-class: INTELLIGENT_TIERING",
  "CPLE_None in CURL_INFO_HEADER_IN: x-amz-server-side-encryption: AES256",
  "CPLE_None in CURL_INFO_HEADER_IN: Cache-Control: public, max-age=31536000, immutable",
  "CPLE_None in CURL_INFO_HEADER_IN: Accept-Ranges: bytes",
  "CPLE_None in CURL_INFO_HEADER_IN: Content-Range: bytes 0-32767/200959614",
  "CPLE_None in CURL_INFO_HEADER_IN: Content-Type: image/tiff; application=geotiff; profile=cloud-optimized",
  "CPLE_None in CURL_INFO_HEADER_IN: Server: AmazonS3",
  "CPLE_None in CURL_INFO_HEADER_IN: Content-Length: 32768",
  "CPLE_None in CURL_INFO_HEADER_IN: ",
  "CPLE_None in CURL_INFO_TEXT: Connection #0 to host sentinel-cogs.s3.us-west-2.amazonaws.com left intact",
  "CPLE_None in VSICURL: Got response_code=206",
  "CPLE_None in GDAL: GDALOpen(/vsicurl/https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif, this=0x11f249ca0) succeeds as GTiff.",
  "Nodata success: 1, Nodata value: 0.000000",
  "Nodata success: 1, Nodata value: 0.000000",
  "Nodata success: 1, Nodata value: 0.000000",
  "Dataset <open DatasetReader name='https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif' mode='r'> is started.",
  "Exiting env context: <rasterio.env.Env object at 0x13fad4820>",
  "Cleared existing <rasterio._env.GDALEnv object at 0x13fa599d0> options",
  "Stopped GDALEnv <rasterio._env.GDALEnv object at 0x13fa599d0>.",
  "No GDAL environment exists",
  "New GDAL environment <rasterio._env.GDALEnv object at 0x13fa599d0> created",
  "GDAL_DATA found in environment.",
  "PROJ_LIB found in environment.",
  "Started GDALEnv: self=<rasterio._env.GDALEnv object at 0x13fa599d0>.",
  "Exited env context: <rasterio.env.Env object at 0x13fad4820>",
  "CPLE_None in GTiff: ScanDirectories()",
  "CPLE_None in GTiff: Opened 5490x5490 overview.",
  "CPLE_None in GTiff: Opened 2745x2745 overview.",
  "CPLE_None in GTiff: Opened 1373x1373 overview.",
  "CPLE_None in GTiff: Opened 687x687 overview.",
  "Matched. confidence=100, c_code=b'3857', c_name=b'EPSG'",
  "Matched. confidence=100, c_code=b'32615', c_name=b'EPSG'",
  "CPLE_None in VRT: No valid sources found for band in VRT file ",
  "CPLE_None in GDAL: GDALOpen(<VRTDataset rasterYSize=\"10980\" rasterXSize=\"10980\"><VRTRasterBand /><SRS>PROJCS[\"WGS 84 / UTM zone 15N\",GEOGCS[\"WGS 84\",DATUM[\"WGS_1984\",SPHEROID[\"WGS 84\",6378137,298.257223563,AUTHORITY[\"EPSG\",\"7030\"]],AUTHORITY[\"EPSG\",\"6326\"]],PRIMEM[\"Greenwich\",0,AUTHORITY[\"EPSG\",\"8901\"]],UNIT[\"degree\",0.0174532925199433,AUTHORITY[\"EPSG\",\"9122\"]],AUTHORITY[\"EPSG\",\"4326\"]],PROJECTION[\"Transverse_Mercator\"],PARAMETER[\"latitude_of_origin\",0],PARAMETER[\"central_meridian\",-93],PARAMETER[\"scale_factor\",0.9996],PARAMETER[\"false_easting\",500000],PARAMETER[\"false_northing\",0],UNIT[\"metre\",1,AUTHORITY[\"EPSG\",\"9001\"]],AXIS[\"Easting\",EAST],AXIS[\"Northing\",NORTH],AUTHORITY[\"EPSG\",\"32615\"]]</SRS><GeoTransform>399960.0,10.0,0.0,5000040.0,0.0,-10.0</GeoTransform></VRTDataset>, this=0x13fc0a1f0) succeeds as VRT.",
  "CPLE_None in GDAL: Computing area of interest: -94.2725, 44.1585, -92.8758, 45.1538",
  "Created exact transformer",
  "CPLE_None in GDAL: GDALClose(<VRTDataset rasterYSize=\"10980\" rasterXSize=\"10980\"><VRTRasterBand /><SRS>PROJCS[\"WGS 84 / UTM zone 15N\",GEOGCS[\"WGS 84\",DATUM[\"WGS_1984\",SPHEROID[\"WGS 84\",6378137,298.257223563,AUTHORITY[\"EPSG\",\"7030\"]],AUTHORITY[\"EPSG\",\"6326\"]],PRIMEM[\"Greenwich\",0,AUTHORITY[\"EPSG\",\"8901\"]],UNIT[\"degree\",0.0174532925199433,AUTHORITY[\"EPSG\",\"9122\"]],AUTHORITY[\"EPSG\",\"4326\"]],PROJECTION[\"Transverse_Mercator\"],PARAMETER[\"latitude_of_origin\",0],PARAMETER[\"central_meridian\",-93],PARAMETER[\"scale_factor\",0.9996],PARAMETER[\"false_easting\",500000],PARAMETER[\"false_northing\",0],UNIT[\"metre\",1,AUTHORITY[\"EPSG\",\"9001\"]],AXIS[\"Easting\",EAST],AXIS[\"Northing\",NORTH],AUTHORITY[\"EPSG\",\"32615\"]]</SRS><GeoTransform>399960.0,10.0,0.0,5000040.0,0.0,-10.0</GeoTransform></VRTDataset>, this=0x13fc0a1f0)",
  "Exported CRS to WKT.",
  "Warp_extras: {'init_dest': 'NO_DATA'}",
  "Nodata success: 1, Nodata value: 0.000000",
  "Nodata success: 1, Nodata value: 0.000000",
  "Nodata success: 1, Nodata value: 0.000000",
  "Dataset <closed WarpedVRT name='WarpedVRT(https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif)' mode='r'> is started.",
  "Output nodata value read from file: 0.0",
  "Output nodata value read from file: 0.0",
  "Output nodata value read from file: 0.0",
  "Output nodata values: [0.0, 0.0, 0.0]",
  "all_valid: False",
  "mask_flags: ([<MaskFlags.nodata: 8>], [<MaskFlags.nodata: 8>], [<MaskFlags.nodata: 8>])",
  "Jump straight to _read()",
  "Window: None",
  "IO window xoff=0.0 yoff=0.0 width=1391.0 height=1391.0",
  "CPLE_None in GDAL: GDALOverviewDataset(/vsicurl/https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif, this=0x13fc0a1f0) creation.",
  "CPLE_None in VSICURL: GetFileList(/vsicurl/https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A)",
  "CPLE_None in CURL_INFO_TEXT: Couldn't find host sentinel-cogs.s3.us-west-2.amazonaws.com in the .netrc file; using defaults",
  "CPLE_None in CURL_INFO_TEXT: Found bundle for host: 0x13f8f81e0 [serially]",
  "CPLE_None in CURL_INFO_TEXT: Can not multiplex, even if we wanted to",
  "CPLE_None in CURL_INFO_TEXT: Re-using existing connection with host sentinel-cogs.s3.us-west-2.amazonaws.com",
  "CPLE_None in CURL_INFO_HEADER_OUT: GET /sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/ HTTP/1.1",
  "Host: sentinel-cogs.s3.us-west-2.amazonaws.com",
  "User-Agent: GDAL/3.7.3",
  "Accept: */*",
  "",
  "CPLE_None in CURL_INFO_HEADER_IN: HTTP/1.1 404 Not Found",
  "CPLE_None in CURL_INFO_HEADER_IN: x-amz-request-id: EJXKH2VG0DDEA2TP",
  "CPLE_None in CURL_INFO_HEADER_IN: x-amz-id-2: 9ZbkWtfMg77R0o/nF4+/8n2OmcPdpK0ibQYNx5hoUXfPkKO3L4KlHK5qiimT/t5SmuSFoajsV5k=",
  "CPLE_None in CURL_INFO_HEADER_IN: Content-Type: application/xml",
  "CPLE_None in CURL_INFO_HEADER_IN: Transfer-Encoding: chunked",
  "CPLE_None in CURL_INFO_HEADER_IN: Date: Thu, 18 Apr 2024 13:54:31 GMT",
  "CPLE_None in CURL_INFO_HEADER_IN: Server: AmazonS3",
  "CPLE_None in CURL_INFO_HEADER_IN: ",
  "CPLE_None in CURL_INFO_TEXT: Connection #0 to host sentinel-cogs.s3.us-west-2.amazonaws.com left intact",
  "CPLE_None in CURL_INFO_TEXT: Couldn't find host sentinel-cogs.s3.us-west-2.amazonaws.com in the .netrc file; using defaults",
  "CPLE_None in CURL_INFO_TEXT: Found bundle for host: 0x13f8f81e0 [serially]",
  "CPLE_None in CURL_INFO_TEXT: Can not multiplex, even if we wanted to",
  "CPLE_None in CURL_INFO_TEXT: Re-using existing connection with host sentinel-cogs.s3.us-west-2.amazonaws.com",
  "CPLE_None in CURL_INFO_HEADER_OUT: HEAD /sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.xml HTTP/1.1",
  "Host: sentinel-cogs.s3.us-west-2.amazonaws.com",
  "User-Agent: GDAL/3.7.3",
  "Accept: */*",
  "",
  "CPLE_None in CURL_INFO_HEADER_IN: HTTP/1.1 404 Not Found",
  "CPLE_None in CURL_INFO_HEADER_IN: x-amz-request-id: EJXNYD4M1D6W2XSH",
  "CPLE_None in CURL_INFO_HEADER_IN: x-amz-id-2: ww4bqRaqhxyjtj+DTfhePvtfVUJnIYdVC8HlpSTobCbJFGRomigMMM06NOTZZnCZ2DC8ntOMK1Y=",
  "CPLE_None in CURL_INFO_HEADER_IN: Content-Type: application/xml",
  "CPLE_None in CURL_INFO_HEADER_IN: Date: Thu, 18 Apr 2024 13:54:31 GMT",
  "CPLE_None in CURL_INFO_HEADER_IN: Server: AmazonS3",
  "CPLE_None in CURL_INFO_HEADER_IN: ",
  "CPLE_None in CURL_INFO_TEXT: Connection #0 to host sentinel-cogs.s3.us-west-2.amazonaws.com left intact",
  "CPLE_None in VSICURL: GetFileSize(https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.xml): response_code=404, server error msg=HTTP/1.1 404 Not Found",
  "x-amz-request-id: EJXNYD4M1D6W2XSH",
  "x-amz-id-2: ww4bqRaqhxyjtj+DTfhePvtfVUJnIYdVC8HlpSTobCbJFGRomigMMM06NOTZZnCZ2DC8ntOMK1Y=",
  "Content-Type: application/xml",
  "Date: Thu, 18 Apr 2024 13:54:31 GMT",
  "Server: AmazonS3",
  "",
  "",
  "CPLE_None in CURL_INFO_TEXT: Couldn't find host sentinel-cogs.s3.us-west-2.amazonaws.com in the .netrc file; using defaults",
  "CPLE_None in CURL_INFO_TEXT: Found bundle for host: 0x13f8f81e0 [serially]",
  "CPLE_None in CURL_INFO_TEXT: Can not multiplex, even if we wanted to",
  "CPLE_None in CURL_INFO_TEXT: Re-using existing connection with host sentinel-cogs.s3.us-west-2.amazonaws.com",
  "CPLE_None in CURL_INFO_HEADER_OUT: HEAD /sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.XML HTTP/1.1",
  "Host: sentinel-cogs.s3.us-west-2.amazonaws.com",
  "User-Agent: GDAL/3.7.3",
  "Accept: */*",
  "",
  "CPLE_None in CURL_INFO_HEADER_IN: HTTP/1.1 404 Not Found",
  "CPLE_None in CURL_INFO_HEADER_IN: x-amz-request-id: EJXQCKA8MA3349AW",
  "CPLE_None in CURL_INFO_HEADER_IN: x-amz-id-2: Lbj0y6wdZs3xlpQjojuE0A7vFO/skRqxCSFwJRMcbFT7Uj8jRRp0D+hCtYazzSXB2Rax1RghP7k=",
  "CPLE_None in CURL_INFO_HEADER_IN: Content-Type: application/xml",
  "CPLE_None in CURL_INFO_HEADER_IN: Date: Thu, 18 Apr 2024 13:54:31 GMT",
  "CPLE_None in CURL_INFO_HEADER_IN: Server: AmazonS3",
  "CPLE_None in CURL_INFO_HEADER_IN: ",
  "CPLE_None in CURL_INFO_TEXT: Connection #0 to host sentinel-cogs.s3.us-west-2.amazonaws.com left intact",
  "CPLE_None in VSICURL: GetFileSize(https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.XML): response_code=404, server error msg=HTTP/1.1 404 Not Found",
  "x-amz-request-id: EJXQCKA8MA3349AW",
  "x-amz-id-2: Lbj0y6wdZs3xlpQjojuE0A7vFO/skRqxCSFwJRMcbFT7Uj8jRRp0D+hCtYazzSXB2Rax1RghP7k=",
  "Content-Type: application/xml",
  "Date: Thu, 18 Apr 2024 13:54:31 GMT",
  "Server: AmazonS3",
  "",
  "",
  "CPLE_None in GDAL: GDALOverviewDataset(/vsicurl/https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif, this=0x11f416860) creation.",
  "CPLE_None in GDAL: GDALOverviewDataset(/vsicurl/https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif, this=0x13fc349c0) creation.",
  "CPLE_None in GDAL: GDALOverviewDataset(/vsicurl/https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif, this=0x13fc35350) creation.",
  "CPLE_None in GDAL: GDALOverviewDataset(/vsicurl/https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif, this=0x13fc48aa0) creation.",
  "CPLE_None in GDAL: GDALOverviewDataset(/vsicurl/https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif, this=0x13fc534c0) creation.",
  "CPLE_None in GDAL: GDAL_CACHEMAX = 12288 MB",
  "CPLE_None in VSICURL: Downloading 9011200-10158079 (https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif)...",
  "CPLE_None in CURL_INFO_TEXT: Couldn't find host sentinel-cogs.s3.us-west-2.amazonaws.com in the .netrc file; using defaults",
  "CPLE_None in CURL_INFO_TEXT: Found bundle for host: 0x13f8f81e0 [serially]",
  "CPLE_None in CURL_INFO_TEXT: Can not multiplex, even if we wanted to",
  "CPLE_None in CURL_INFO_TEXT: Re-using existing connection with host sentinel-cogs.s3.us-west-2.amazonaws.com",
  "CPLE_None in CURL_INFO_HEADER_OUT: GET /sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif HTTP/1.1",
  "Host: sentinel-cogs.s3.us-west-2.amazonaws.com",
  "User-Agent: GDAL/3.7.3",
  "Accept: */*",
  "Range: bytes=9011200-10158079",
  "",
  "CPLE_None in CURL_INFO_HEADER_IN: HTTP/1.1 206 Partial Content",
  "CPLE_None in CURL_INFO_HEADER_IN: x-amz-id-2: +NsF6qFLnSt0sbQ9WvuJl3iuifVS2Ggbah7FMHrObNpF9hc3IDG7IJPjkuFm8VlBdq8/m4pEpHE=",
  "CPLE_None in CURL_INFO_HEADER_IN: x-amz-request-id: EJXS85WYC3GDRM76",
  "CPLE_None in CURL_INFO_HEADER_IN: Date: Thu, 18 Apr 2024 13:54:33 GMT",
  "CPLE_None in CURL_INFO_HEADER_IN: Last-Modified: Mon, 09 Oct 2023 01:23:09 GMT",
  "CPLE_None in CURL_INFO_HEADER_IN: ETag: \"3e23880cd7ac8a32ae71c55ad4f85c5d-24\"",
  "CPLE_None in CURL_INFO_HEADER_IN: x-amz-storage-class: INTELLIGENT_TIERING",
  "CPLE_None in CURL_INFO_HEADER_IN: x-amz-server-side-encryption: AES256",
  "CPLE_None in CURL_INFO_HEADER_IN: Cache-Control: public, max-age=31536000, immutable",
  "CPLE_None in CURL_INFO_HEADER_IN: Accept-Ranges: bytes",
  "CPLE_None in CURL_INFO_HEADER_IN: Content-Range: bytes 9011200-10158079/200959614",
  "CPLE_None in CURL_INFO_HEADER_IN: Content-Type: image/tiff; application=geotiff; profile=cloud-optimized",
  "CPLE_None in CURL_INFO_HEADER_IN: Server: AmazonS3",
  "CPLE_None in CURL_INFO_HEADER_IN: Content-Length: 1146880",
  "CPLE_None in CURL_INFO_HEADER_IN: ",
  "CPLE_None in CURL_INFO_TEXT: Connection #0 to host sentinel-cogs.s3.us-west-2.amazonaws.com left intact",
  "CPLE_None in VSICURL: Got response_code=206",
  "CPLE_None in GDAL: GDALWarpKernel()::GWKNearestByte() Src=1824,1911,351x130 Dst=0,0,348x128",
  "CPLE_None in VSICURL: Downloading 11075584-12189695 (https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif)...",
  "CPLE_None in CURL_INFO_TEXT: Couldn't find host sentinel-cogs.s3.us-west-2.amazonaws.com in the .netrc file; using defaults",
  "CPLE_None in CURL_INFO_TEXT: Found bundle for host: 0x13f8f81e0 [serially]",
  "CPLE_None in CURL_INFO_TEXT: Can not multiplex, even if we wanted to",
  "CPLE_None in CURL_INFO_TEXT: Re-using existing connection with host sentinel-cogs.s3.us-west-2.amazonaws.com",
  "CPLE_None in CURL_INFO_HEADER_OUT: GET /sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif HTTP/1.1",
  "Host: sentinel-cogs.s3.us-west-2.amazonaws.com",
  "User-Agent: GDAL/3.7.3",
  "Accept: */*",
  "Range: bytes=11075584-12189695",
  "",
  "CPLE_None in CURL_INFO_HEADER_IN: HTTP/1.1 206 Partial Content",
  "CPLE_None in CURL_INFO_HEADER_IN: x-amz-id-2: dBrzELUWdBHGmu7gG4b5UgZ1SgZfl3nn4OF+bQjnPGRxHmUnjcHE24thEHncL6q6j2AixFF3GYI=",
  "CPLE_None in CURL_INFO_HEADER_IN: x-amz-request-id: 4B1HMCNGJGFWVKXB",
  "CPLE_None in CURL_INFO_HEADER_IN: Date: Thu, 18 Apr 2024 13:54:34 GMT",
  "CPLE_None in CURL_INFO_HEADER_IN: Last-Modified: Mon, 09 Oct 2023 01:23:09 GMT",
  "CPLE_None in CURL_INFO_HEADER_IN: ETag: \"3e23880cd7ac8a32ae71c55ad4f85c5d-24\"",
  "CPLE_None in CURL_INFO_HEADER_IN: x-amz-storage-class: INTELLIGENT_TIERING",
  "CPLE_None in CURL_INFO_HEADER_IN: x-amz-server-side-encryption: AES256",
  "CPLE_None in CURL_INFO_HEADER_IN: Cache-Control: public, max-age=31536000, immutable",
  "CPLE_None in CURL_INFO_HEADER_IN: Accept-Ranges: bytes",
  "CPLE_None in CURL_INFO_HEADER_IN: Content-Range: bytes 11075584-12189695/200959614",
  "CPLE_None in CURL_INFO_HEADER_IN: Content-Type: image/tiff; application=geotiff; profile=cloud-optimized",
  "CPLE_None in CURL_INFO_HEADER_IN: Server: AmazonS3",
  "CPLE_None in CURL_INFO_HEADER_IN: Content-Length: 1114112",
  "CPLE_None in CURL_INFO_HEADER_IN: ",
  "CPLE_None in CURL_INFO_TEXT: Connection #0 to host sentinel-cogs.s3.us-west-2.amazonaws.com left intact",
  "CPLE_None in VSICURL: Got response_code=206",
  "CPLE_None in GDAL: GDALWarpKernel()::GWKNearestByte() Src=1824,2039,351x130 Dst=0,128,348x128",
  "CPLE_None in GDAL: GDALWarpKernel()::GWKNearestByte() Src=1823,2167,352x94 Dst=0,256,348x92",
  "IO window xoff=0.0 yoff=0.0 width=1391.0 height=1391.0",
  "CPLE_None in GDAL: GDALClose(/vsicurl/https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif, this=0x11f249ca0)",
  "Exiting env context: <rasterio.env.Env object at 0x13fa59460>",
  "Cleared existing <rasterio._env.GDALEnv object at 0x13fa599d0> options",
  "Stopped GDALEnv <rasterio._env.GDALEnv object at 0x13fa599d0>.",
  "Exiting outermost env",
  "Exited env context: <rasterio.env.Env object at 0x13fa59460>"
]

@vincentsarago
Copy link
Member Author

Some weird stuff from the logs:

  • we have 2 HEAD (GetFileSize) requests to .xml and .XMLfiles
  • we have 1 GET (GetFileList) request

while we have GDAL_DISABLE_READDIR_ON_OPEN=EMPTY_DIR

CPLE_None in VSICURL: GetFileSize(https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif)=200959614  response_code=200
CPLE_None in VSICURL: GetFileList(/vsicurl/https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A)
CPLE_None in VSICURL: GetFileSize(https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.xml): response_code=404, server error msg=HTTP/1.1 404 Not Found
CPLE_None in VSICURL: GetFileSize(https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.XML): response_code=404, server error msg=HTTP/1.1 404 Not Found

@vincentsarago vincentsarago mentioned this issue Apr 18, 2024
Closed
@rouault
Copy link

rouault commented Apr 18, 2024

I don't confirm that using plain GDAL command line (3.8.5 or master):

$ CPL_VSIL_SHOW_NETWORK_STATS=YES GDAL_DISABLE_READDIR_ON_OPEN=EMPTY_DIR CPL_DEBUG=ON gdalinfo /vsicurl/https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif
HTTP: libcurl/7.68.0 GnuTLS/3.6.13 zlib/1.2.11 brotli/1.0.7 libidn2/2.2.0 libpsl/0.21.0 (+libidn2/2.2.0) libssh/0.9.3/openssl/zlib nghttp2/1.40.0 librtmp/2.3
VSICURL: GetFileSize(https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif)=200959614  response_code=200
VSICURL: Downloading 0-16383 (https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif)...
VSICURL: Got response_code=206
GDAL: GDALOpen(/vsicurl/https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif, this=0x55daa42fd160) succeeds as GTiff.
GDAL: GDALDefaultOverviews::OverviewScan()
GTiff: ScanDirectories()
GTiff: Opened 5490x5490 overview.
GTiff: Opened 2745x2745 overview.
GTiff: Opened 1373x1373 overview.
GTiff: Opened 687x687 overview.
OGR_CT: Source CRS: 'EPSG:32615'
OGR_CT: Target CRS: 'GEOGCRS["WGS 84",DATUM["World Geodetic System 1984",ELLIPSOID["WGS 84",6378137,298.257223563,LENGTHUNIT["metre",1]],ID["EPSG",6326]],PRIMEM["Greenwich",0,ANGLEUNIT["degree",0.0174532925199433],ID["EPSG",8901]],CS[ellipsoidal,2],AXIS["geodetic latitude (Lat)",north,ORDER[1],ANGLEUNIT["degree",0.0174532925199433,ID["EPSG",9122]]],AXIS["geodetic longitude (Lon)",east,ORDER[2],ANGLEUNIT["degree",0.0174532925199433,ID["EPSG",9122]]]]'
Driver: GTiff/GeoTIFF
Files: /vsicurl/https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif
Size is 10980, 10980
Coordinate System is:
PROJCRS["WGS 84 / UTM zone 15N",
    BASEGEOGCRS["WGS 84",
        DATUM["World Geodetic System 1984",
            ELLIPSOID["WGS 84",6378137,298.257223563,
                LENGTHUNIT["metre",1]]],
        PRIMEM["Greenwich",0,
            ANGLEUNIT["degree",0.0174532925199433]],
        ID["EPSG",4326]],
    CONVERSION["UTM zone 15N",
        METHOD["Transverse Mercator",
            ID["EPSG",9807]],
        PARAMETER["Latitude of natural origin",0,
            ANGLEUNIT["degree",0.0174532925199433],
            ID["EPSG",8801]],
        PARAMETER["Longitude of natural origin",-93,
            ANGLEUNIT["degree",0.0174532925199433],
            ID["EPSG",8802]],
        PARAMETER["Scale factor at natural origin",0.9996,
            SCALEUNIT["unity",1],
            ID["EPSG",8805]],
        PARAMETER["False easting",500000,
            LENGTHUNIT["metre",1],
            ID["EPSG",8806]],
        PARAMETER["False northing",0,
            LENGTHUNIT["metre",1],
            ID["EPSG",8807]]],
    CS[Cartesian,2],
        AXIS["(E)",east,
            ORDER[1],
            LENGTHUNIT["metre",1]],
        AXIS["(N)",north,
            ORDER[2],
            LENGTHUNIT["metre",1]],
    USAGE[
        SCOPE["Navigation and medium accuracy spatial referencing."],
        AREA["Between 96°W and 90°W, northern hemisphere between equator and 84°N, onshore and offshore. Canada - Manitoba; Nunavut; Ontario. Ecuador -Galapagos. Guatemala. Mexico. United States (USA)."],
        BBOX[0,-96,84,-90]],
    ID["EPSG",32615]]
Data axis to CRS axis mapping: 1,2
Origin = (399960.000000000000000,5000040.000000000000000)
Pixel Size = (10.000000000000000,-10.000000000000000)
Metadata:
  OVR_RESAMPLING_ALG=AVERAGE
  AREA_OR_POINT=Area
Image Structure Metadata:
  COMPRESSION=DEFLATE
  INTERLEAVE=PIXEL
  PREDICTOR=2
Corner Coordinates:
Upper Left  (  399960.000, 5000040.000) ( 94d16'21.18"W, 45d 8'48.29"N)
Lower Left  (  399960.000, 4890240.000) ( 94d15' 4.02"W, 44d 9'30.66"N)
Upper Right (  509760.000, 5000040.000) ( 92d52'33.00"W, 45d 9'13.57"N)
Lower Right (  509760.000, 4890240.000) ( 92d52'40.53"W, 44d 9'55.08"N)
Center      (  454860.000, 4945140.000) ( 93d34' 9.68"W, 44d39'29.53"N)
Band 1 Block=1024x1024 Type=Byte, ColorInterp=Red
  NoData Value=0
  Overviews: 5490x5490, 2745x2745, 1373x1373, 687x687
Band 2 Block=1024x1024 Type=Byte, ColorInterp=Green
  NoData Value=0
  Overviews: 5490x5490, 2745x2745, 1373x1373, 687x687
Band 3 Block=1024x1024 Type=Byte, ColorInterp=Blue
  NoData Value=0
  Overviews: 5490x5490, 2745x2745, 1373x1373, 687x687
GDAL: GDALClose(/vsicurl/https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif, this=0x55daa42fd160)
GDAL: In GDALDestroy - unloading GDAL shared library.
Network statistics:
{
  "methods":{
    "HEAD":{
      "count":1
    },
    "GET":{
      "count":1,
      "downloaded_bytes":16384
    }
  },
  "handlers":{
    "vsicurl":{
      "methods":{
        "HEAD":{
          "count":1
        },
        "GET":{
          "count":1,
          "downloaded_bytes":16384
        }
      },
      "files":{
        "\/vsicurl\/https:\/\/sentinel-cogs.s3.us-west-2.amazonaws.com\/sentinel-s2-l2a-cogs\/15\/T\/VK\/2023\/10\/S2B_15TVK_20231008_0_L2A\/TCI.tif":{
          "methods":{
            "GET":{
              "count":1,
              "downloaded_bytes":16384
            }
          },
          "actions":{
            "Read":{
              "methods":{
                "GET":{
                  "count":1,
                  "downloaded_bytes":16384
                }
              }
            }
          }
        }
      },
      "actions":{
        "Stat":{
          "methods":{
            "HEAD":{
              "count":1
            }
          },
          "files":{
            "\/vsicurl\/https:\/\/sentinel-cogs.s3.us-west-2.amazonaws.com\/sentinel-s2-l2a-cogs\/15\/T\/VK\/2023\/10\/S2B_15TVK_20231008_0_L2A\/TCI.tif":{
              "methods":{
                "HEAD":{
                  "count":1
                }
              }
            }
          }
        }
      }
    }
  }
}

@vincentsarago
Copy link
Member Author

For some reason doing INFO works fine, it's when reading the data that we get additional HEAD/GET

info is tested here and works

rio-tiler/tests/benchmarks/requests.py

Lines 14 to 36 in 473de24

def test_info():
"""Info should only GET the header."""
@profile(
kernels=True,
config={
"GDAL_HTTP_MERGE_CONSECUTIVE_RANGES": "YES",
"GDAL_DISABLE_READDIR_ON_OPEN": "EMPTY_DIR",
"GDAL_INGESTED_BYTES_AT_OPEN": 32768,
},
quiet=True,
add_to_return=True,
)
def info(src_path: str):
with Reader(src_path) as src_dst:
return src_dst.info()
_, stats = info(dataset_url)
assert stats["HEAD"]["count"] == 1
assert stats["GET"]["count"] == 1
assert stats["GET"]["bytes"] == 32768
assert stats["GET"]["ranges"] == ["0-32767"]
assert not stats["WarpKernels"]

@vincentsarago
Copy link
Member Author

well gdalwarp gives the expected result

CPL_VSIL_SHOW_NETWORK_STATS=YES GDAL_DISABLE_READDIR_ON_OPEN=EMPTY_DIR CPL_DEBUG=ON gdalwarp -q /vsicurl/https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif out.tif -t_srs EPSG:3857 -ts 256 256 -te -10390543.876974 5518141.945963 -10370975.997733 5537709.825204 -overwrite
HTTP: libcurl/8.4.0 (SecureTransport) LibreSSL/3.3.6 zlib/1.2.12 nghttp2/1.58.0
VSICURL: GetFileSize(https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif)=200959614  response_code=200
VSICURL: Downloading 0-32767 (https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif)...
VSICURL: Got response_code=206
GDAL: GDALOpen(/vsicurl/https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif, this=0x10fe23d90) succeeds as GTiff.
GDAL: Using GTiff driver
GDAL: Computing area of interest: -94.2725, 44.1585, -92.8758, 45.1538
GDAL: GDALDriver::Create(GTiff,out.tif,256,256,3,Byte,0x10ffdced0)
WARP: Copying metadata from first source to destination dataset
GTiff: ScanDirectories()
GTiff: Opened 5490x5490 overview.
GTiff: Opened 2745x2745 overview.
GTiff: Opened 1373x1373 overview.
GTiff: Opened 687x687 overview.
WARP: Selecting overview level 1 for /vsicurl/https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif
GDAL: GDALOverviewDataset(/vsicurl/https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif, this=0x13100c690) creation.
WARP: srcNoData=0.000000 dstNoData=0.000000
WARP: calling GDALSetRasterNoDataValue() for band#0
WARP: srcNoData=0.000000 dstNoData=0.000000
WARP: calling GDALSetRasterNoDataValue() for band#1
WARP: srcNoData=0.000000 dstNoData=0.000000
WARP: calling GDALSetRasterNoDataValue() for band#2
GDALWARP: Defining SKIP_NOSOURCE=YES
GDAL: Computing area of interest: -94.2725, 44.1585, -92.8758, 45.1538
VSICURL: Downloading 9019424-10144695 (https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif)...
VSICURL: Downloading 11076968-12175156 (https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif)...
VSICURL: Download completed
GDAL: GDAL_CACHEMAX = 12288 MB
GDAL: GDALWarpKernel()::GWKNearestByte() Src=1823,1911,352x350 Dst=0,0,256x256
GTiff: Adjusted bytes to write from 7680 to 4608.
GDAL: GDALClose(out.tif, this=0x131005630)
GDAL: GDALClose(/vsicurl/https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif, this=0x10fe23d90)
Network statistics:
{
  "methods":{
    "HEAD":{
      "count":1
    },
    "GET":{
      "count":2,
      "downloaded_bytes":2256229
    }
  },
  "handlers":{
    "vsicurl":{
      "methods":{
        "HEAD":{
          "count":1
        },
        "GET":{
          "count":2,
          "downloaded_bytes":2256229
        }
      },
      "files":{
        "\/vsicurl\/https:\/\/sentinel-cogs.s3.us-west-2.amazonaws.com\/sentinel-s2-l2a-cogs\/15\/T\/VK\/2023\/10\/S2B_15TVK_20231008_0_L2A\/TCI.tif":{
          "methods":{
            "GET":{
              "count":2,
              "downloaded_bytes":2256229
            }
          },
          "actions":{
            "Read":{
              "methods":{
                "GET":{
                  "count":1,
                  "downloaded_bytes":32768
                }
              }
            },
            "ReadMultiRange":{
              "methods":{
                "GET":{
                  "count":1,
                  "downloaded_bytes":2223461
                }
              }
            }
          }
        }
      },
      "actions":{
        "Stat":{
          "methods":{
            "HEAD":{
              "count":1
            }
          },
          "files":{
            "\/vsicurl\/https:\/\/sentinel-cogs.s3.us-west-2.amazonaws.com\/sentinel-s2-l2a-cogs\/15\/T\/VK\/2023\/10\/S2B_15TVK_20231008_0_L2A\/TCI.tif":{
              "methods":{
                "HEAD":{
                  "count":1
                }
              }
            }
          }
        }
      }
    }
  }
}

😬

@rouault
Copy link

rouault commented Apr 18, 2024

you may want to break into GTiffDataset::LookForProjectionFromXML() . Normally papszSiblingFiles = GetSiblingFiles() should be such that papszSiblingFiles != nullptr and papszSiblingFiles[0] == nullptr if GDAL_DISABLE_READDIR_ON_OPEN=EMPTY_DIR

@vincentsarago
Copy link
Member Author

just to summarize:

with gdalwarp we can 1 HEAD + 2 GET request, while with rio-tiler/rasterio we get 3 HEAD / 4 GET 😬

@vincentsarago
Copy link
Member Author

@rouault is LookForProjectionFromXML something that changed between gdal 3.6 and 3.7?

@rouault
Copy link

rouault commented Apr 19, 2024

is LookForProjectionFromXML something that changed between gdal 3.6 and 3.7?

it has been introduced in 3.7

@vincentsarago
Copy link
Member Author

I'm going back to this and to me there seems to be a bug somewhere in GDAL

Rasterio (1.4.1, GDAL=3.9.2) + WarpedVRT

import logging

import rasterio
from rasterio.vrt import WarpedVRT

logging.basicConfig(level=logging.DEBUG)

src_path = "https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif"

with rasterio.Env(GDAL_DISABLE_READDIR_ON_OPEN="EMPTY_DIR", GDAL_INGESTED_BYTES_AT_OPEN=32768, GDAL_HTTP_MERGE_CONSECUTIVE_RANGES="YES", CPL_DEBUG="ON", CPL_CURL_VERBOSE="YES"):
    with rasterio.open(src_path) as src:
        with WarpedVRT(src, dst_crs="epsg:3857") as vrt:
            vrt.read(indexes=1, out_shape=(1, vrt.height//5, vrt.width//5))

logs

DEBUG:rasterio.env:Entering env context: <rasterio.env.Env object at 0x103ead2e0>
DEBUG:rasterio.env:Starting outermost env
DEBUG:rasterio.env:No GDAL environment exists
DEBUG:rasterio.env:New GDAL environment <rasterio._env.GDALEnv object at 0x10bdbdfa0> created
DEBUG:rasterio._filepath:Installing FilePath filesystem handler plugin...
DEBUG:rasterio._env:GDAL_DATA found in environment.
DEBUG:rasterio._env:PROJ_LIB found in environment.
DEBUG:rasterio._env:Started GDALEnv: self=<rasterio._env.GDALEnv object at 0x10bdbdfa0>.
DEBUG:rasterio.env:Entered env context: <rasterio.env.Env object at 0x103ead2e0>
DEBUG:rasterio.env:Got a copy of environment <rasterio._env.GDALEnv object at 0x10bdbdfa0> options
DEBUG:rasterio.env:Entering env context: <rasterio.env.Env object at 0x10be4d8e0>
DEBUG:rasterio.env:Got a copy of environment <rasterio._env.GDALEnv object at 0x10bdbdfa0> options
DEBUG:rasterio.env:Entered env context: <rasterio.env.Env object at 0x10be4d8e0>
DEBUG:rasterio._base:Sharing flag: 0
DEBUG:rasterio._env:CPLE_None in HTTP: libcurl/8.8.0 OpenSSL/1.1.1w zlib/1.2.12 nghttp2/1.46.0
DEBUG:rasterio._env:CPLE_None in CURL_INFO_TEXT: Couldn't find host sentinel-cogs.s3.us-west-2.amazonaws.com in the .netrc file; using defaults
DEBUG:rasterio._env:CPLE_None in CURL_INFO_TEXT: Host sentinel-cogs.s3.us-west-2.amazonaws.com:443 was resolved.
DEBUG:rasterio._env:CPLE_None in CURL_INFO_TEXT: IPv6: (none)
DEBUG:rasterio._env:CPLE_None in CURL_INFO_TEXT: IPv4: 3.5.84.124, 52.92.209.90, 52.92.235.82, 3.5.80.209, 3.5.80.200, 52.218.246.41, 3.5.86.120, 52.92.212.234
DEBUG:rasterio._env:CPLE_None in CURL_INFO_TEXT:   Trying 3.5.84.124:443...
DEBUG:rasterio._env:CPLE_None in CURL_INFO_TEXT: Connected to sentinel-cogs.s3.us-west-2.amazonaws.com (3.5.84.124) port 443
DEBUG:rasterio._env:CPLE_None in CURL_INFO_TEXT: ALPN: curl offers h2,http/1.1
DEBUG:rasterio._env:CPLE_None in CURL_INFO_TEXT: TLSv1.3 (OUT), TLS handshake, Client hello (1):
DEBUG:rasterio._env:CPLE_None in CURL_INFO_TEXT:  CAfile: [/Users/vincentsarago/Dev/venv/py39/lib/python3.9/site-packages/certifi/cacert.pem](https://file+.vscode-resource.vscode-cdn.net/Users/vincentsarago/Dev/venv/py39/lib/python3.9/site-packages/certifi/cacert.pem)
DEBUG:rasterio._env:CPLE_None in CURL_INFO_TEXT:  CApath: none
DEBUG:rasterio._env:CPLE_None in CURL_INFO_TEXT: TLSv1.3 (IN), TLS handshake, Server hello (2):
DEBUG:rasterio._env:CPLE_None in CURL_INFO_TEXT: TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
DEBUG:rasterio._env:CPLE_None in CURL_INFO_TEXT: TLSv1.3 (IN), TLS handshake, Certificate (11):
DEBUG:rasterio._env:CPLE_None in CURL_INFO_TEXT: TLSv1.3 (IN), TLS handshake, CERT verify (15):
DEBUG:rasterio._env:CPLE_None in CURL_INFO_TEXT: TLSv1.3 (IN), TLS handshake, Finished (20):
DEBUG:rasterio._env:CPLE_None in CURL_INFO_TEXT: TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
DEBUG:rasterio._env:CPLE_None in CURL_INFO_TEXT: TLSv1.3 (OUT), TLS handshake, Finished (20):
DEBUG:rasterio._env:CPLE_None in CURL_INFO_TEXT: SSL connection using TLSv1.3 [/](https://file+.vscode-resource.vscode-cdn.net/) TLS_AES_128_GCM_SHA256 [/](https://file+.vscode-resource.vscode-cdn.net/) [blank] [/](https://file+.vscode-resource.vscode-cdn.net/) UNDEF
DEBUG:rasterio._env:CPLE_None in CURL_INFO_TEXT: ALPN: server accepted http/1.1
DEBUG:rasterio._env:CPLE_None in CURL_INFO_TEXT: Server certificate:
DEBUG:rasterio._env:CPLE_None in CURL_INFO_TEXT:  subject: CN=*.s3-us-west-2.amazonaws.com
DEBUG:rasterio._env:CPLE_None in CURL_INFO_TEXT:  start date: Sep 14 00:00:00 2024 GMT
DEBUG:rasterio._env:CPLE_None in CURL_INFO_TEXT:  expire date: Aug 29 23:59:59 2025 GMT
DEBUG:rasterio._env:CPLE_None in CURL_INFO_TEXT:  subjectAltName: host "sentinel-cogs.s3.us-west-2.amazonaws.com" matched cert's "*.s3.us-west-2.amazonaws.com"
DEBUG:rasterio._env:CPLE_None in CURL_INFO_TEXT:  issuer: C=US; O=Amazon; CN=Amazon RSA 2048 M01
DEBUG:rasterio._env:CPLE_None in CURL_INFO_TEXT:  SSL certificate verify ok.
DEBUG:rasterio._env:CPLE_None in CURL_INFO_TEXT:   Certificate level 0: Public key type ? (2048/112 Bits/secBits), signed using sha256WithRSAEncryption
DEBUG:rasterio._env:CPLE_None in CURL_INFO_TEXT:   Certificate level 1: Public key type ? (2048/112 Bits/secBits), signed using sha256WithRSAEncryption
DEBUG:rasterio._env:CPLE_None in CURL_INFO_TEXT:   Certificate level 2: Public key type ? (2048/112 Bits/secBits), signed using sha256WithRSAEncryption
DEBUG:rasterio._env:CPLE_None in CURL_INFO_TEXT: using HTTP/1.x
DEBUG:rasterio._env:CPLE_None in CURL_INFO_HEADER_OUT: HEAD [/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif](https://file+.vscode-resource.vscode-cdn.net/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif) HTTP/1.1
Host: sentinel-cogs.s3.us-west-2.amazonaws.com
User-Agent: GDAL/3.9.2
Accept: */*

DEBUG:rasterio._env:CPLE_None in CURL_INFO_TEXT: Request completely sent off
DEBUG:rasterio._env:CPLE_None in CURL_INFO_HEADER_IN: HTTP/1.1 200 OK
DEBUG:rasterio._env:CPLE_None in CURL_INFO_HEADER_IN: x-amz-id-2: 66ZukBWtLNWUF+BSUWt3j8cFljhBQOTe77fxbWCjR7Nlc2FSHEtka2GSAdtyURpFIrzytM6T2qTXMIB7Rndz1A==
DEBUG:rasterio._env:CPLE_None in CURL_INFO_HEADER_IN: x-amz-request-id: 04M946PJ3Z8KRAHG
DEBUG:rasterio._env:CPLE_None in CURL_INFO_HEADER_IN: Date: Wed, 02 Oct 2024 08:52:19 GMT
DEBUG:rasterio._env:CPLE_None in CURL_INFO_HEADER_IN: Last-Modified: Mon, 09 Oct 2023 01:23:09 GMT
DEBUG:rasterio._env:CPLE_None in CURL_INFO_HEADER_IN: ETag: "3e23880cd7ac8a32ae71c55ad4f85c5d-24"
DEBUG:rasterio._env:CPLE_None in CURL_INFO_HEADER_IN: x-amz-storage-class: INTELLIGENT_TIERING
DEBUG:rasterio._env:CPLE_None in CURL_INFO_HEADER_IN: x-amz-server-side-encryption: AES256
DEBUG:rasterio._env:CPLE_None in CURL_INFO_HEADER_IN: Cache-Control: public, max-age=31536000, immutable
DEBUG:rasterio._env:CPLE_None in CURL_INFO_HEADER_IN: Accept-Ranges: bytes
DEBUG:rasterio._env:CPLE_None in CURL_INFO_HEADER_IN: Content-Type: image/tiff; application=geotiff; profile=cloud-optimized
DEBUG:rasterio._env:CPLE_None in CURL_INFO_HEADER_IN: Server: AmazonS3
DEBUG:rasterio._env:CPLE_None in CURL_INFO_HEADER_IN: Content-Length: 200959614
DEBUG:rasterio._env:CPLE_None in CURL_INFO_HEADER_IN: 
DEBUG:rasterio._env:CPLE_None in CURL_INFO_TEXT: Connection #0 to host sentinel-cogs.s3.us-west-2.amazonaws.com left intact
DEBUG:rasterio._env:CPLE_None in VSICURL: GetFileSize(https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif)=200959614  response_code=200
DEBUG:rasterio._env:CPLE_None in VSICURL: Downloading 0-32767 (https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif)...
DEBUG:rasterio._env:CPLE_None in CURL_INFO_TEXT: Couldn't find host sentinel-cogs.s3.us-west-2.amazonaws.com in the .netrc file; using defaults
DEBUG:rasterio._env:CPLE_None in CURL_INFO_TEXT: Found bundle for host: 0x11e2cfb30 [serially]
DEBUG:rasterio._env:CPLE_None in CURL_INFO_TEXT: Can not multiplex, even if we wanted to
DEBUG:rasterio._env:CPLE_None in CURL_INFO_TEXT: Re-using existing connection with host sentinel-cogs.s3.us-west-2.amazonaws.com
DEBUG:rasterio._env:CPLE_None in CURL_INFO_HEADER_OUT: GET [/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif](https://file+.vscode-resource.vscode-cdn.net/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif) HTTP/1.1
Host: sentinel-cogs.s3.us-west-2.amazonaws.com
User-Agent: GDAL/3.9.2
Accept: */*
Range: bytes=0-32767

DEBUG:rasterio._env:CPLE_None in CURL_INFO_TEXT: Request completely sent off
DEBUG:rasterio._env:CPLE_None in CURL_INFO_HEADER_IN: HTTP/1.1 206 Partial Content
DEBUG:rasterio._env:CPLE_None in CURL_INFO_HEADER_IN: x-amz-id-2: TBEzLxd3o5rOMfIOKBl52WyRHPU85i6pqIx/nvSPCiB3SHJTt/XNUFxm8Phv2i7nRn32kK6VFlJEM8W45QOSTw==
DEBUG:rasterio._env:CPLE_None in CURL_INFO_HEADER_IN: x-amz-request-id: 04MA0QVA6SDSNJ1N
DEBUG:rasterio._env:CPLE_None in CURL_INFO_HEADER_IN: Date: Wed, 02 Oct 2024 08:52:19 GMT
DEBUG:rasterio._env:CPLE_None in CURL_INFO_HEADER_IN: Last-Modified: Mon, 09 Oct 2023 01:23:09 GMT
DEBUG:rasterio._env:CPLE_None in CURL_INFO_HEADER_IN: ETag: "3e23880cd7ac8a32ae71c55ad4f85c5d-24"
DEBUG:rasterio._env:CPLE_None in CURL_INFO_HEADER_IN: x-amz-storage-class: INTELLIGENT_TIERING
DEBUG:rasterio._env:CPLE_None in CURL_INFO_HEADER_IN: x-amz-server-side-encryption: AES256
DEBUG:rasterio._env:CPLE_None in CURL_INFO_HEADER_IN: Cache-Control: public, max-age=31536000, immutable
DEBUG:rasterio._env:CPLE_None in CURL_INFO_HEADER_IN: Accept-Ranges: bytes
DEBUG:rasterio._env:CPLE_None in CURL_INFO_HEADER_IN: Content-Range: bytes 0-32767/200959614
DEBUG:rasterio._env:CPLE_None in CURL_INFO_HEADER_IN: Content-Type: image/tiff; application=geotiff; profile=cloud-optimized
DEBUG:rasterio._env:CPLE_None in CURL_INFO_HEADER_IN: Server: AmazonS3
DEBUG:rasterio._env:CPLE_None in CURL_INFO_HEADER_IN: Content-Length: 32768
DEBUG:rasterio._env:CPLE_None in CURL_INFO_HEADER_IN: 
DEBUG:rasterio._env:CPLE_None in CURL_INFO_TEXT: Connection #0 to host sentinel-cogs.s3.us-west-2.amazonaws.com left intact
DEBUG:rasterio._env:CPLE_None in VSICURL: Got response_code=206
DEBUG:rasterio._env:CPLE_None in GDAL: GDALOpen(/vsicurl/https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif, this=0x13de06f60) succeeds as GTiff.
DEBUG:rasterio._base:Nodata success: 1, Nodata value: 0.000000
DEBUG:rasterio._base:Nodata success: 1, Nodata value: 0.000000
DEBUG:rasterio._base:Nodata success: 1, Nodata value: 0.000000
DEBUG:rasterio._base:Dataset <open DatasetReader name='https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif' mode='r'> is started.
DEBUG:rasterio.env:Exiting env context: <rasterio.env.Env object at 0x10be4d8e0>
DEBUG:rasterio.env:Cleared existing <rasterio._env.GDALEnv object at 0x10bdbdfa0> options
DEBUG:rasterio._env:Stopped GDALEnv <rasterio._env.GDALEnv object at 0x10bdbdfa0>.
DEBUG:rasterio.env:No GDAL environment exists
DEBUG:rasterio.env:New GDAL environment <rasterio._env.GDALEnv object at 0x10bdbdfa0> created
DEBUG:rasterio._env:GDAL_DATA found in environment.
DEBUG:rasterio._env:PROJ_LIB found in environment.
DEBUG:rasterio._env:Started GDALEnv: self=<rasterio._env.GDALEnv object at 0x10bdbdfa0>.
DEBUG:rasterio.env:Exited env context: <rasterio.env.Env object at 0x10be4d8e0>
DEBUG:rasterio._warp:Exported CRS to WKT.
DEBUG:rasterio._warp:Warp_extras: {'init_dest': 'NO_DATA'}
DEBUG:rasterio._env:CPLE_None in GDAL: Computing area of interest: -94.2725, 44.1585, -92.8758, 45.1538
DEBUG:rasterio._base:Nodata success: 1, Nodata value: 0.000000
DEBUG:rasterio._base:Nodata success: 1, Nodata value: 0.000000
DEBUG:rasterio._base:Nodata success: 1, Nodata value: 0.000000
DEBUG:rasterio._base:Dataset <closed WarpedVRT name='WarpedVRT(https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif)' mode='r'> is started.
DEBUG:rasterio._io:Output nodata value read from file: 0.0
DEBUG:rasterio._io:Output nodata values: [0.0]
DEBUG:rasterio._io:all_valid: False
DEBUG:rasterio._io:mask_flags: ([<MaskFlags.nodata: 8>], [<MaskFlags.nodata: 8>], [<MaskFlags.nodata: 8>])
DEBUG:rasterio._io:Jump straight to _read()
DEBUG:rasterio._io:Window: None
DEBUG:rasterio._io:IO window xoff=0.0 yoff=0.0 width=11055.0 height=11075.0
DEBUG:rasterio._err:GTiff: ScanDirectories()
DEBUG:rasterio._err:GTiff: Opened 5490x5490 overview.
DEBUG:rasterio._err:GTiff: Opened 2745x2745 overview.
DEBUG:rasterio._err:GTiff: Opened 1373x1373 overview.
DEBUG:rasterio._err:GTiff: Opened 687x687 overview.
DEBUG:rasterio._err:GDAL: GDALOverviewDataset(/vsicurl/https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif, this=0x11e4b4f30) creation.
DEBUG:rasterio._err:VSICURL: GetFileList(/vsicurl/https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A)
DEBUG:rasterio._err:CURL_INFO_TEXT: Couldn't find host sentinel-cogs.s3.us-west-2.amazonaws.com in the .netrc file; using defaults
DEBUG:rasterio._err:CURL_INFO_TEXT: Found bundle for host: 0x11e2cfb30 [serially]
DEBUG:rasterio._err:CURL_INFO_TEXT: Can not multiplex, even if we wanted to
DEBUG:rasterio._err:CURL_INFO_TEXT: Re-using existing connection with host sentinel-cogs.s3.us-west-2.amazonaws.com
DEBUG:rasterio._err:CURL_INFO_HEADER_OUT: GET [/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/](https://file+.vscode-resource.vscode-cdn.net/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/) HTTP/1.1
Host: sentinel-cogs.s3.us-west-2.amazonaws.com
User-Agent: GDAL/3.9.2
Accept: */*

DEBUG:rasterio._err:CURL_INFO_TEXT: Request completely sent off
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: HTTP/1.1 404 Not Found
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: x-amz-request-id: 04M3W5D1X1QG1QGH
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: x-amz-id-2: SWw3TihMYc69BHEGWgTDst0ipH9aYZKc31WNlx9XyBX6+EU9bUH+3JlcttL5NhleQQLJp2i5SwVJXzYqRei6NA==
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Content-Type: application/xml
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Transfer-Encoding: chunked
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Date: Wed, 02 Oct 2024 08:52:18 GMT
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Server: AmazonS3
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: 
DEBUG:rasterio._err:CURL_INFO_TEXT: Connection #0 to host sentinel-cogs.s3.us-west-2.amazonaws.com left intact
DEBUG:rasterio._err:CURL_INFO_TEXT: Couldn't find host sentinel-cogs.s3.us-west-2.amazonaws.com in the .netrc file; using defaults
DEBUG:rasterio._err:CURL_INFO_TEXT: Found bundle for host: 0x11e2cfb30 [serially]
DEBUG:rasterio._err:CURL_INFO_TEXT: Can not multiplex, even if we wanted to
DEBUG:rasterio._err:CURL_INFO_TEXT: Re-using existing connection with host sentinel-cogs.s3.us-west-2.amazonaws.com
DEBUG:rasterio._err:CURL_INFO_HEADER_OUT: HEAD [/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.xml](https://file+.vscode-resource.vscode-cdn.net/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.xml) HTTP/1.1
Host: sentinel-cogs.s3.us-west-2.amazonaws.com
User-Agent: GDAL/3.9.2
Accept: */*

DEBUG:rasterio._err:CURL_INFO_TEXT: Request completely sent off
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: HTTP/1.1 404 Not Found
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: x-amz-request-id: 39Q0YGBWGZG9N8W9
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: x-amz-id-2: edsMl8Micg3K9YVsD6fZAUmphAeuYjtLNt2FzSu9+LKIyEnKMVzns8T+MXGkluiHb5jb7/f4ndx3lJJNPoSctQ==
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Content-Type: application/xml
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Date: Wed, 02 Oct 2024 08:52:18 GMT
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Server: AmazonS3
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: 
DEBUG:rasterio._err:CURL_INFO_TEXT: Connection #0 to host sentinel-cogs.s3.us-west-2.amazonaws.com left intact
DEBUG:rasterio._err:VSICURL: GetFileSize(https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.xml): response_code=404, server error msg=HTTP/1.1 404 Not Found
x-amz-request-id: 39Q0YGBWGZG9N8W9
x-amz-id-2: edsMl8Micg3K9YVsD6fZAUmphAeuYjtLNt2FzSu9+LKIyEnKMVzns8T+MXGkluiHb5jb7/f4ndx3lJJNPoSctQ==
Content-Type: application/xml
Date: Wed, 02 Oct 2024 08:52:18 GMT
Server: AmazonS3


DEBUG:rasterio._err:CURL_INFO_TEXT: Couldn't find host sentinel-cogs.s3.us-west-2.amazonaws.com in the .netrc file; using defaults
DEBUG:rasterio._err:CURL_INFO_TEXT: Found bundle for host: 0x11e2cfb30 [serially]
DEBUG:rasterio._err:CURL_INFO_TEXT: Can not multiplex, even if we wanted to
DEBUG:rasterio._err:CURL_INFO_TEXT: Re-using existing connection with host sentinel-cogs.s3.us-west-2.amazonaws.com
DEBUG:rasterio._err:CURL_INFO_HEADER_OUT: HEAD [/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.XML](https://file+.vscode-resource.vscode-cdn.net/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.XML) HTTP/1.1
Host: sentinel-cogs.s3.us-west-2.amazonaws.com
User-Agent: GDAL/3.9.2
Accept: */*

DEBUG:rasterio._err:CURL_INFO_TEXT: Request completely sent off
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: HTTP/1.1 404 Not Found
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: x-amz-request-id: 39Q5MNF1V3W72Z28
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: x-amz-id-2: cPhXCpikJGdDx1TmISOlQWMRG8OfAKyE08AO6zCln+QFJ5EUIDq3VT/OR1UfKPyyITiDtfZ3lNDnmU9Uea2UcQ==
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Content-Type: application/xml
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Date: Wed, 02 Oct 2024 08:52:18 GMT
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Server: AmazonS3
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: 
DEBUG:rasterio._err:CURL_INFO_TEXT: Connection #0 to host sentinel-cogs.s3.us-west-2.amazonaws.com left intact
DEBUG:rasterio._err:VSICURL: GetFileSize(https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.XML): response_code=404, server error msg=HTTP/1.1 404 Not Found
x-amz-request-id: 39Q5MNF1V3W72Z28
x-amz-id-2: cPhXCpikJGdDx1TmISOlQWMRG8OfAKyE08AO6zCln+QFJ5EUIDq3VT/OR1UfKPyyITiDtfZ3lNDnmU9Uea2UcQ==
Content-Type: application/xml
Date: Wed, 02 Oct 2024 08:52:18 GMT
Server: AmazonS3


DEBUG:rasterio._err:GDAL: GDALOverviewDataset(/vsicurl/https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif, this=0x11e4b6860) creation.
DEBUG:rasterio._err:GDAL: GDALOverviewDataset(/vsicurl/https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif, this=0x11e4bb390) creation.
DEBUG:rasterio._err:GDAL: GDALOverviewDataset(/vsicurl/https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif, this=0x10c216510) creation.
DEBUG:rasterio._err:GDAL: Selecting overview 2764 x 2769
DEBUG:rasterio._err:GDAL: GDALOverviewDataset(/vsicurl/https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif, this=0x11e319550) creation.
DEBUG:rasterio._err:GDAL: GDALOverviewDataset(/vsicurl/https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif, this=0x11e4cedb0) creation.
DEBUG:rasterio._err:GDAL: GDAL_CACHEMAX = 12288 MB
DEBUG:rasterio._err:VSICURL: Downloading 3096576-3112959 (https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif)...
DEBUG:rasterio._err:CURL_INFO_TEXT: Couldn't find host sentinel-cogs.s3.us-west-2.amazonaws.com in the .netrc file; using defaults
DEBUG:rasterio._err:CURL_INFO_TEXT: Found bundle for host: 0x11e2cfb30 [serially]
DEBUG:rasterio._err:CURL_INFO_TEXT: Can not multiplex, even if we wanted to
DEBUG:rasterio._err:CURL_INFO_TEXT: Re-using existing connection with host sentinel-cogs.s3.us-west-2.amazonaws.com
DEBUG:rasterio._err:CURL_INFO_HEADER_OUT: GET [/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif](https://file+.vscode-resource.vscode-cdn.net/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif) HTTP/1.1
Host: sentinel-cogs.s3.us-west-2.amazonaws.com
User-Agent: GDAL/3.9.2
Accept: */*
Range: bytes=3096576-3112959

DEBUG:rasterio._err:CURL_INFO_TEXT: Request completely sent off
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: HTTP/1.1 206 Partial Content
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: x-amz-id-2: 6n/vbvz2oquK4DZ5+Mw1zpJeZD9NT7yt2cBN5woxpSn5+CfMHRRdd0kjHtT57ItyALyRLlH90RZtfD4kpBdQPQ==
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: x-amz-request-id: 39Q5X6179ZERSXSV
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Date: Wed, 02 Oct 2024 08:52:20 GMT
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Last-Modified: Mon, 09 Oct 2023 01:23:09 GMT
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: ETag: "3e23880cd7ac8a32ae71c55ad4f85c5d-24"
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: x-amz-storage-class: INTELLIGENT_TIERING
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: x-amz-server-side-encryption: AES256
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Cache-Control: public, max-age=31536000, immutable
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Accept-Ranges: bytes
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Content-Range: bytes 3096576-3112959/200959614
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Content-Type: image/tiff; application=geotiff; profile=cloud-optimized
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Server: AmazonS3
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Content-Length: 16384
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: 
DEBUG:rasterio._err:CURL_INFO_TEXT: Connection #0 to host sentinel-cogs.s3.us-west-2.amazonaws.com left intact
DEBUG:rasterio._err:VSICURL: Got response_code=206
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=0,0,509x115 Dst=0,0,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=507,0,511x120 Dst=512,0,512x128
DEBUG:rasterio._err:VSICURL: Downloading 3112960-3309567 (https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif)...
DEBUG:rasterio._err:CURL_INFO_TEXT: Couldn't find host sentinel-cogs.s3.us-west-2.amazonaws.com in the .netrc file; using defaults
DEBUG:rasterio._err:CURL_INFO_TEXT: Found bundle for host: 0x11e2cfb30 [serially]
DEBUG:rasterio._err:CURL_INFO_TEXT: Can not multiplex, even if we wanted to
DEBUG:rasterio._err:CURL_INFO_TEXT: Re-using existing connection with host sentinel-cogs.s3.us-west-2.amazonaws.com
DEBUG:rasterio._err:CURL_INFO_HEADER_OUT: GET [/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif](https://file+.vscode-resource.vscode-cdn.net/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif) HTTP/1.1
Host: sentinel-cogs.s3.us-west-2.amazonaws.com
User-Agent: GDAL/3.9.2
Accept: */*
Range: bytes=3112960-3309567

DEBUG:rasterio._err:CURL_INFO_TEXT: Request completely sent off
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: HTTP/1.1 206 Partial Content
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: x-amz-id-2: 6GHyhzRGc8oWWQGG7SMKOEpWhq0qGTCwGppQCwweQh/4NSqBnBvymMhpnoyARnG8WemCvkj0RW4ygGRpORhWwg==
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: x-amz-request-id: 39QB0EX4HEZM4MND
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Date: Wed, 02 Oct 2024 08:52:20 GMT
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Last-Modified: Mon, 09 Oct 2023 01:23:09 GMT
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: ETag: "3e23880cd7ac8a32ae71c55ad4f85c5d-24"
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: x-amz-storage-class: INTELLIGENT_TIERING
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: x-amz-server-side-encryption: AES256
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Cache-Control: public, max-age=31536000, immutable
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Accept-Ranges: bytes
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Content-Range: bytes 3112960-3309567/200959614
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Content-Type: image/tiff; application=geotiff; profile=cloud-optimized
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Server: AmazonS3
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Content-Length: 196608
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: 
DEBUG:rasterio._err:CURL_INFO_TEXT: Connection #0 to host sentinel-cogs.s3.us-west-2.amazonaws.com left intact
DEBUG:rasterio._err:VSICURL: Got response_code=206
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=1015,0,511x124 Dst=1024,0,512x128
DEBUG:rasterio._err:VSICURL: Downloading 3309568-3948543 (https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif)...
DEBUG:rasterio._err:CURL_INFO_TEXT: Couldn't find host sentinel-cogs.s3.us-west-2.amazonaws.com in the .netrc file; using defaults
DEBUG:rasterio._err:CURL_INFO_TEXT: Found bundle for host: 0x11e2cfb30 [serially]
DEBUG:rasterio._err:CURL_INFO_TEXT: Can not multiplex, even if we wanted to
DEBUG:rasterio._err:CURL_INFO_TEXT: Re-using existing connection with host sentinel-cogs.s3.us-west-2.amazonaws.com
DEBUG:rasterio._err:CURL_INFO_HEADER_OUT: GET [/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif](https://file+.vscode-resource.vscode-cdn.net/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif) HTTP/1.1
Host: sentinel-cogs.s3.us-west-2.amazonaws.com
User-Agent: GDAL/3.9.2
Accept: */*
Range: bytes=3309568-3948543

DEBUG:rasterio._err:CURL_INFO_TEXT: Request completely sent off
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: HTTP/1.1 206 Partial Content
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: x-amz-id-2: yqgPY6dA7F2DbbL4C+JqpsA9M8GccY9rC1vkN0TiFcPqtjtnxMxM7YQkdRfD5nWV9gISKhcTO8omelWhSNRyIw==
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: x-amz-request-id: 87PS0WKV3DTGYJCY
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Date: Wed, 02 Oct 2024 08:52:21 GMT
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Last-Modified: Mon, 09 Oct 2023 01:23:09 GMT
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: ETag: "3e23880cd7ac8a32ae71c55ad4f85c5d-24"
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: x-amz-storage-class: INTELLIGENT_TIERING
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: x-amz-server-side-encryption: AES256
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Cache-Control: public, max-age=31536000, immutable
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Accept-Ranges: bytes
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Content-Range: bytes 3309568-3948543/200959614
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Content-Type: image/tiff; application=geotiff; profile=cloud-optimized
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Server: AmazonS3
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Content-Length: 638976
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: 
DEBUG:rasterio._err:CURL_INFO_TEXT: Connection #0 to host sentinel-cogs.s3.us-west-2.amazonaws.com left intact
DEBUG:rasterio._err:VSICURL: Got response_code=206
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=1524,0,510x127 Dst=1536,0,512x128
DEBUG:rasterio._err:VSICURL: Downloading 3948544-5226495 (https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif)...
DEBUG:rasterio._err:CURL_INFO_TEXT: Couldn't find host sentinel-cogs.s3.us-west-2.amazonaws.com in the .netrc file; using defaults
DEBUG:rasterio._err:CURL_INFO_TEXT: Found bundle for host: 0x11e2cfb30 [serially]
DEBUG:rasterio._err:CURL_INFO_TEXT: Can not multiplex, even if we wanted to
DEBUG:rasterio._err:CURL_INFO_TEXT: Re-using existing connection with host sentinel-cogs.s3.us-west-2.amazonaws.com
DEBUG:rasterio._err:CURL_INFO_HEADER_OUT: GET [/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif](https://file+.vscode-resource.vscode-cdn.net/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif) HTTP/1.1
Host: sentinel-cogs.s3.us-west-2.amazonaws.com
User-Agent: GDAL/3.9.2
Accept: */*
Range: bytes=3948544-5226495

DEBUG:rasterio._err:CURL_INFO_TEXT: Request completely sent off
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: HTTP/1.1 206 Partial Content
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: x-amz-id-2: hgo/SksLd2yLV14hkryD2kRDi6AhnyDyamg60BHBnUaP9E4r0MDn4cDGsIwhUuYK+eADCkHT/GJgnTsK6nE1CA==
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: x-amz-request-id: 87PG3YQ9RDSFSVVP
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Date: Wed, 02 Oct 2024 08:52:21 GMT
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Last-Modified: Mon, 09 Oct 2023 01:23:09 GMT
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: ETag: "3e23880cd7ac8a32ae71c55ad4f85c5d-24"
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: x-amz-storage-class: INTELLIGENT_TIERING
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: x-amz-server-side-encryption: AES256
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Cache-Control: public, max-age=31536000, immutable
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Accept-Ranges: bytes
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Content-Range: bytes 3948544-5226495/200959614
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Content-Type: image/tiff; application=geotiff; profile=cloud-optimized
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Server: AmazonS3
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Content-Length: 1277952
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: 
DEBUG:rasterio._err:CURL_INFO_TEXT: Connection #0 to host sentinel-cogs.s3.us-west-2.amazonaws.com left intact
DEBUG:rasterio._err:VSICURL: Got response_code=206
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=2033,0,510x127 Dst=2048,0,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=2542,0,203x127 Dst=2560,0,204x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=0,107,508x135 Dst=0,128,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=505,114,511x133 Dst=512,128,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=1014,119,511x132 Dst=1024,128,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=1523,123,511x130 Dst=1536,128,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=2033,126,510x128 Dst=2048,128,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=2542,126,203x128 Dst=2560,128,204x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=0,233,506x135 Dst=0,256,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=503,241,512x133 Dst=512,256,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=1013,246,511x132 Dst=1024,256,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=1523,250,511x130 Dst=1536,256,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=2032,252,511x129 Dst=2048,256,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=2542,253,203x128 Dst=2560,256,204x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=0,360,504x136 Dst=0,384,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=502,367,512x134 Dst=512,384,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=1012,373,512x132 Dst=1024,384,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=1522,377,511x130 Dst=1536,384,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=2032,379,511x129 Dst=2048,384,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=2542,380,203x128 Dst=2560,384,204x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=0,487,503x136 Dst=0,512,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=500,495,513x133 Dst=512,512,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=1011,500,512x132 Dst=1024,512,512x128
DEBUG:rasterio._err:VSICURL: Downloading 5226496-7782399 (https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif)...
DEBUG:rasterio._err:CURL_INFO_TEXT: Couldn't find host sentinel-cogs.s3.us-west-2.amazonaws.com in the .netrc file; using defaults
DEBUG:rasterio._err:CURL_INFO_TEXT: Found bundle for host: 0x11e2cfb30 [serially]
DEBUG:rasterio._err:CURL_INFO_TEXT: Can not multiplex, even if we wanted to
DEBUG:rasterio._err:CURL_INFO_TEXT: Re-using existing connection with host sentinel-cogs.s3.us-west-2.amazonaws.com
DEBUG:rasterio._err:CURL_INFO_HEADER_OUT: GET [/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif](https://file+.vscode-resource.vscode-cdn.net/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif) HTTP/1.1
Host: sentinel-cogs.s3.us-west-2.amazonaws.com
User-Agent: GDAL/3.9.2
Accept: */*
Range: bytes=5226496-7782399

DEBUG:rasterio._err:CURL_INFO_TEXT: Request completely sent off
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: HTTP/1.1 206 Partial Content
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: x-amz-id-2: Knzkc7DKDTFgOy0+3db6tnzTCXfKw+V1NO33XW3qdsVN/DbpdXZXrgIjdWWQVKp7cA9LCLtNLwPTnaW4CkVfRQ==
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: x-amz-request-id: 3RTD6FNTYN4SPTAB
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Date: Wed, 02 Oct 2024 08:52:22 GMT
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Last-Modified: Mon, 09 Oct 2023 01:23:09 GMT
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: ETag: "3e23880cd7ac8a32ae71c55ad4f85c5d-24"
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: x-amz-storage-class: INTELLIGENT_TIERING
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: x-amz-server-side-encryption: AES256
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Cache-Control: public, max-age=31536000, immutable
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Accept-Ranges: bytes
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Content-Range: bytes 5226496-7782399/200959614
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Content-Type: image/tiff; application=geotiff; profile=cloud-optimized
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Server: AmazonS3
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Content-Length: 2555904
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: 
DEBUG:rasterio._err:CURL_INFO_TEXT: Connection #0 to host sentinel-cogs.s3.us-west-2.amazonaws.com left intact
DEBUG:rasterio._err:VSICURL: Got response_code=206
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=1521,504,512x130 Dst=1536,512,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=2032,506,511x129 Dst=2048,512,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=2542,507,203x128 Dst=2560,512,204x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=0,614,501x136 Dst=0,640,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=499,622,513x133 Dst=512,640,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=1010,627,512x132 Dst=1024,640,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=1520,631,513x131 Dst=1536,640,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=2031,633,512x129 Dst=2048,640,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=2542,634,203x128 Dst=2560,640,204x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=0,742,500x135 Dst=0,768,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=497,749,514x134 Dst=512,768,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=1008,754,513x133 Dst=1024,768,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=1520,758,512x131 Dst=1536,768,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=2031,761,512x129 Dst=2048,768,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=2542,761,203x129 Dst=2560,768,204x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=0,869,498x136 Dst=0,896,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=496,876,513x134 Dst=512,896,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=1007,882,514x132 Dst=1024,896,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=1519,886,513x130 Dst=1536,896,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=2030,888,513x129 Dst=2048,896,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=2542,889,203x128 Dst=2560,896,204x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=0,996,497x136 Dst=0,1024,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=494,1004,514x134 Dst=512,1024,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=1006,1009,514x133 Dst=1024,1024,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=1518,1013,513x131 Dst=1536,1024,512x128
DEBUG:rasterio._err:VSICURL: Downloading 7782400-10338303 (https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif)...
DEBUG:rasterio._err:CURL_INFO_TEXT: Couldn't find host sentinel-cogs.s3.us-west-2.amazonaws.com in the .netrc file; using defaults
DEBUG:rasterio._err:CURL_INFO_TEXT: Found bundle for host: 0x11e2cfb30 [serially]
DEBUG:rasterio._err:CURL_INFO_TEXT: Can not multiplex, even if we wanted to
DEBUG:rasterio._err:CURL_INFO_TEXT: Re-using existing connection with host sentinel-cogs.s3.us-west-2.amazonaws.com
DEBUG:rasterio._err:CURL_INFO_HEADER_OUT: GET [/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif](https://file+.vscode-resource.vscode-cdn.net/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif) HTTP/1.1
Host: sentinel-cogs.s3.us-west-2.amazonaws.com
User-Agent: GDAL/3.9.2
Accept: */*
Range: bytes=7782400-10338303

DEBUG:rasterio._err:CURL_INFO_TEXT: Request completely sent off
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: HTTP/1.1 206 Partial Content
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: x-amz-id-2: XAY9dwZ6cgnWABIB8CPEhAIn+mCl6tr5wODh+ydISgMNGwdfH1cp6b/SNsPaEmehIRDRhaRSeNcsIoTCnXvfKA==
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: x-amz-request-id: BV7W5P11Z1QSEW27
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Date: Wed, 02 Oct 2024 08:52:24 GMT
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Last-Modified: Mon, 09 Oct 2023 01:23:09 GMT
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: ETag: "3e23880cd7ac8a32ae71c55ad4f85c5d-24"
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: x-amz-storage-class: INTELLIGENT_TIERING
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: x-amz-server-side-encryption: AES256
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Cache-Control: public, max-age=31536000, immutable
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Accept-Ranges: bytes
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Content-Range: bytes 7782400-10338303/200959614
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Content-Type: image/tiff; application=geotiff; profile=cloud-optimized
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Server: AmazonS3
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Content-Length: 2555904
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: 
DEBUG:rasterio._err:CURL_INFO_TEXT: Connection #0 to host sentinel-cogs.s3.us-west-2.amazonaws.com left intact
DEBUG:rasterio._err:VSICURL: Got response_code=206
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=2030,1015,513x130 Dst=2048,1024,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=2542,1016,203x129 Dst=2560,1024,204x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=0,1124,495x136 Dst=0,1152,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=492,1131,515x134 Dst=512,1152,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=1005,1137,514x132 Dst=1024,1152,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=1517,1141,514x131 Dst=1536,1152,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=2030,1143,513x129 Dst=2048,1152,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=2542,1143,203x129 Dst=2560,1152,204x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=0,1252,493x136 Dst=0,1280,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=491,1259,515x134 Dst=512,1280,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=1004,1264,514x133 Dst=1024,1280,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=1516,1268,515x131 Dst=1536,1280,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=2029,1271,514x129 Dst=2048,1280,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=2542,1271,203x129 Dst=2560,1280,204x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=0,1379,492x136 Dst=0,1408,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=489,1387,516x134 Dst=512,1408,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=1002,1392,515x133 Dst=1024,1408,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=1516,1396,514x131 Dst=1536,1408,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=2029,1398,514x130 Dst=2048,1408,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=2542,1399,203x129 Dst=2560,1408,204x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=0,1507,490x136 Dst=0,1536,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=488,1514,515x135 Dst=512,1536,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=1001,1520,516x133 Dst=1024,1536,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=1515,1524,515x131 Dst=1536,1536,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=2029,1526,514x130 Dst=2048,1536,512x128
DEBUG:rasterio._err:VSICURL: Downloading 10338304-12894207 (https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif)...
DEBUG:rasterio._err:CURL_INFO_TEXT: Couldn't find host sentinel-cogs.s3.us-west-2.amazonaws.com in the .netrc file; using defaults
DEBUG:rasterio._err:CURL_INFO_TEXT: Found bundle for host: 0x11e2cfb30 [serially]
DEBUG:rasterio._err:CURL_INFO_TEXT: Can not multiplex, even if we wanted to
DEBUG:rasterio._err:CURL_INFO_TEXT: Re-using existing connection with host sentinel-cogs.s3.us-west-2.amazonaws.com
DEBUG:rasterio._err:CURL_INFO_HEADER_OUT: GET [/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif](https://file+.vscode-resource.vscode-cdn.net/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif) HTTP/1.1
Host: sentinel-cogs.s3.us-west-2.amazonaws.com
User-Agent: GDAL/3.9.2
Accept: */*
Range: bytes=10338304-12894207

DEBUG:rasterio._err:CURL_INFO_TEXT: Request completely sent off
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: HTTP/1.1 206 Partial Content
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: x-amz-id-2: e4pv6zaja1zW0J4XYxBisnAKWyTauRyNx712+0fFACOu3pOc4yanIfVTh8OvvIxh6s7sb23EAdQyQjvLMx4QeA==
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: x-amz-request-id: JNYCDH9Z4BH47JEH
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Date: Wed, 02 Oct 2024 08:52:25 GMT
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Last-Modified: Mon, 09 Oct 2023 01:23:09 GMT
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: ETag: "3e23880cd7ac8a32ae71c55ad4f85c5d-24"
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: x-amz-storage-class: INTELLIGENT_TIERING
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: x-amz-server-side-encryption: AES256
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Cache-Control: public, max-age=31536000, immutable
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Accept-Ranges: bytes
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Content-Range: bytes 10338304-12894207/200959614
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Content-Type: image/tiff; application=geotiff; profile=cloud-optimized
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Server: AmazonS3
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Content-Length: 2555904
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: 
DEBUG:rasterio._err:CURL_INFO_TEXT: Connection #0 to host sentinel-cogs.s3.us-west-2.amazonaws.com left intact
DEBUG:rasterio._err:VSICURL: Got response_code=206
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=2542,1527,203x129 Dst=2560,1536,204x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=0,1635,489x136 Dst=0,1664,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=486,1642,516x135 Dst=512,1664,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=1000,1648,516x133 Dst=1024,1664,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=1514,1652,516x131 Dst=1536,1664,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=2028,1654,515x130 Dst=2048,1664,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=2542,1655,203x129 Dst=2560,1664,204x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=0,1763,487x136 Dst=0,1792,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=484,1770,517x135 Dst=512,1792,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=999,1776,516x133 Dst=1024,1792,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=1513,1780,516x131 Dst=1536,1792,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=2028,1782,515x130 Dst=2048,1792,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=2542,1783,203x129 Dst=2560,1792,204x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=0,1891,485x137 Dst=0,1920,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=483,1898,517x135 Dst=512,1920,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=998,1904,516x133 Dst=1024,1920,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=1513,1908,516x132 Dst=1536,1920,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=2028,1910,515x130 Dst=2048,1920,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=2542,1911,203x129 Dst=2560,1920,204x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=0,2020,484x136 Dst=0,2048,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=481,2027,518x135 Dst=512,2048,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=997,2032,517x134 Dst=1024,2048,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=1512,2036,517x132 Dst=1536,2048,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=2027,2039,516x130 Dst=2048,2048,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=2542,2039,203x130 Dst=2560,2048,204x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=0,2148,482x136 Dst=0,2176,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=480,2155,518x135 Dst=512,2176,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=995,2161,518x133 Dst=1024,2176,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=1511,2165,517x131 Dst=1536,2176,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=2027,2167,516x130 Dst=2048,2176,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=2542,2167,203x130 Dst=2560,2176,204x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=0,2276,481x137 Dst=0,2304,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=478,2283,518x136 Dst=512,2304,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=994,2289,518x134 Dst=1024,2304,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=1510,2293,518x132 Dst=1536,2304,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=2026,2295,518x131 Dst=2048,2304,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=2542,2296,203x130 Dst=2560,2304,204x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=0,2405,479x137 Dst=0,2432,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=476,2412,519x135 Dst=512,2432,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=993,2418,518x133 Dst=1024,2432,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=1509,2422,518x131 Dst=1536,2432,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=2026,2424,518x130 Dst=2048,2432,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=2543,2424,202x130 Dst=2560,2432,204x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=0,2533,477x137 Dst=0,2560,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=475,2541,519x135 Dst=512,2560,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=992,2546,518x134 Dst=1024,2560,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=1509,2550,518x132 Dst=1536,2560,512x128
DEBUG:rasterio._err:VSICURL: Downloading 12894208-15450111 (https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif)...
DEBUG:rasterio._err:CURL_INFO_TEXT: Couldn't find host sentinel-cogs.s3.us-west-2.amazonaws.com in the .netrc file; using defaults
DEBUG:rasterio._err:CURL_INFO_TEXT: Found bundle for host: 0x11e2cfb30 [serially]
DEBUG:rasterio._err:CURL_INFO_TEXT: Can not multiplex, even if we wanted to
DEBUG:rasterio._err:CURL_INFO_TEXT: Re-using existing connection with host sentinel-cogs.s3.us-west-2.amazonaws.com
DEBUG:rasterio._err:CURL_INFO_HEADER_OUT: GET [/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif](https://file+.vscode-resource.vscode-cdn.net/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif) HTTP/1.1
Host: sentinel-cogs.s3.us-west-2.amazonaws.com
User-Agent: GDAL/3.9.2
Accept: */*
Range: bytes=12894208-15450111

DEBUG:rasterio._err:CURL_INFO_TEXT: Request completely sent off
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: HTTP/1.1 206 Partial Content
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: x-amz-id-2: oakZsSEL5Nty2AuT0Tghf6F4EPMC3cOkQVIEHBJF7CTvZ8coj8cYBjaDpZspjbuklLkkMGuRb5Z0/Zsmk2Zn4w==
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: x-amz-request-id: X56SHGFYSG8N7ND9
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Date: Wed, 02 Oct 2024 08:52:26 GMT
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Last-Modified: Mon, 09 Oct 2023 01:23:09 GMT
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: ETag: "3e23880cd7ac8a32ae71c55ad4f85c5d-24"
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: x-amz-storage-class: INTELLIGENT_TIERING
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: x-amz-server-side-encryption: AES256
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Cache-Control: public, max-age=31536000, immutable
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Accept-Ranges: bytes
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Content-Range: bytes 12894208-15450111/200959614
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Content-Type: image/tiff; application=geotiff; profile=cloud-optimized
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Server: AmazonS3
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Content-Length: 2555904
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: 
DEBUG:rasterio._err:CURL_INFO_TEXT: Connection #0 to host sentinel-cogs.s3.us-west-2.amazonaws.com left intact
DEBUG:rasterio._err:VSICURL: Got response_code=206
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=2026,2552,518x131 Dst=2048,2560,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=2543,2553,202x130 Dst=2560,2560,204x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=0,2662,476x83 Dst=0,2688,512x81
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=474,2669,519x76 Dst=512,2688,512x81
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=991,2675,519x70 Dst=1024,2688,512x81
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=1508,2679,519x66 Dst=1536,2688,512x81
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=2025,2681,519x64 Dst=2048,2688,512x81
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=2543,2682,202x63 Dst=2560,2688,204x81
DEBUG:rasterio._env:CPLE_None in GDAL: GDALClose(/vsicurl/https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif, this=0x13de06f60)
DEBUG:rasterio.env:Exiting env context: <rasterio.env.Env object at 0x103ead2e0>
DEBUG:rasterio.env:Cleared existing <rasterio._env.GDALEnv object at 0x10bdbdfa0> options
DEBUG:rasterio._env:Stopped GDALEnv <rasterio._env.GDALEnv object at 0x10bdbdfa0>.
DEBUG:rasterio.env:Exiting outermost env
DEBUG:rasterio.env:Exited env context: <rasterio.env.Env object at 0x103ead2e0>

@rouault I don't see any mentione of GDAL_DISABLE_READDIR_ON_OPEN in https://github.com/OSGeo/gdal/blob/96608941276cc29a4c17b55d453ab8bd616e34dd/frmts/gtiff/gtiffdataset_read.cpp#L5828-L5854

what I don't understand is why gdalwarp doesn't result in the same behaviour

GDAL 3.8.5 (gdalwarp)

GDAL_DISABLE_READDIR_ON_OPEN=EMPTY_DIR GDAL_INGESTED_BYTES_AT_OPEN=32768 GDAL_HTTP_MERGE_CONSECUTIVE_RANGES=YES CPL_DEBUG=ON CPL_CURL_VERBOSE=YES gdalwarp -q /vsicurl/https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif out.tif -b 1 -t_srs EPSG:3857 -ts 2215 2211 -overwrite
HTTP: libcurl/8.4.0 (SecureTransport) LibreSSL/3.3.6 zlib/1.2.12 nghttp2/1.58.0
CURL_INFO_TEXT: Couldn't find host sentinel-cogs.s3.us-west-2.amazonaws.com in the .netrc file; using defaults
CURL_INFO_TEXT:   Trying 52.92.129.34:443...
CURL_INFO_TEXT: Connected to sentinel-cogs.s3.us-west-2.amazonaws.com (52.92.129.34) port 443
CURL_INFO_TEXT: ALPN: curl offers h2,http/1.1
CURL_INFO_TEXT: (304) (OUT), TLS handshake, Client hello (1):
CURL_INFO_TEXT:  CAfile: /etc/ssl/cert.pem
CURL_INFO_TEXT:  CApath: none
CURL_INFO_TEXT: (304) (IN), TLS handshake, Server hello (2):
CURL_INFO_TEXT: (304) (IN), TLS handshake, Unknown (8):
CURL_INFO_TEXT: (304) (IN), TLS handshake, Certificate (11):
CURL_INFO_TEXT: (304) (IN), TLS handshake, CERT verify (15):
CURL_INFO_TEXT: (304) (IN), TLS handshake, Finished (20):
CURL_INFO_TEXT: (304) (OUT), TLS handshake, Finished (20):
CURL_INFO_TEXT: SSL connection using TLSv1.3 / AEAD-AES128-GCM-SHA256
CURL_INFO_TEXT: ALPN: server accepted http/1.1
CURL_INFO_TEXT: Server certificate:
CURL_INFO_TEXT:  subject: CN=*.s3-us-west-2.amazonaws.com
CURL_INFO_TEXT:  start date: Sep 14 00:00:00 2024 GMT
CURL_INFO_TEXT:  expire date: Aug 29 23:59:59 2025 GMT
CURL_INFO_TEXT:  subjectAltName: host "sentinel-cogs.s3.us-west-2.amazonaws.com" matched cert's "*.s3.us-west-2.amazonaws.com"
CURL_INFO_TEXT:  issuer: C=US; O=Amazon; CN=Amazon RSA 2048 M01
CURL_INFO_TEXT:  SSL certificate verify ok.
CURL_INFO_TEXT: using HTTP/1.1
CURL_INFO_HEADER_OUT: HEAD /sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif HTTP/1.1
Host: sentinel-cogs.s3.us-west-2.amazonaws.com
User-Agent: GDAL/3.8.5
Accept: */*

CURL_INFO_HEADER_IN: HTTP/1.1 200 OK
CURL_INFO_HEADER_IN: x-amz-id-2: IHhHcFOmwMl5CwUr/ArLtiOnzqWsBK+yFiUaR1rUNgfsI4XF1jyoAgUrqZNSLwRBpyQZy+8NNh8=
CURL_INFO_HEADER_IN: x-amz-request-id: W45JSEN9T8VZGB6P
CURL_INFO_HEADER_IN: Date: Wed, 02 Oct 2024 08:51:25 GMT
CURL_INFO_HEADER_IN: Last-Modified: Mon, 09 Oct 2023 01:23:09 GMT
CURL_INFO_HEADER_IN: ETag: "3e23880cd7ac8a32ae71c55ad4f85c5d-24"
CURL_INFO_HEADER_IN: x-amz-storage-class: INTELLIGENT_TIERING
CURL_INFO_HEADER_IN: x-amz-server-side-encryption: AES256
CURL_INFO_HEADER_IN: Cache-Control: public, max-age=31536000, immutable
CURL_INFO_HEADER_IN: Accept-Ranges: bytes
CURL_INFO_HEADER_IN: Content-Type: image/tiff; application=geotiff; profile=cloud-optimized
CURL_INFO_HEADER_IN: Server: AmazonS3
CURL_INFO_HEADER_IN: Content-Length: 200959614
CURL_INFO_HEADER_IN: 
CURL_INFO_TEXT: Connection #0 to host sentinel-cogs.s3.us-west-2.amazonaws.com left intact
VSICURL: GetFileSize(https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif)=200959614  response_code=200
VSICURL: Downloading 0-32767 (https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif)...
CURL_INFO_TEXT: Couldn't find host sentinel-cogs.s3.us-west-2.amazonaws.com in the .netrc file; using defaults
CURL_INFO_TEXT: Found bundle for host: 0x111f3f360 [serially]
CURL_INFO_TEXT: Can not multiplex, even if we wanted to
CURL_INFO_TEXT: Re-using existing connection with host sentinel-cogs.s3.us-west-2.amazonaws.com
CURL_INFO_HEADER_OUT: GET /sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif HTTP/1.1
Host: sentinel-cogs.s3.us-west-2.amazonaws.com
User-Agent: GDAL/3.8.5
Accept: */*
Range: bytes=0-32767

CURL_INFO_HEADER_IN: HTTP/1.1 206 Partial Content
CURL_INFO_HEADER_IN: x-amz-id-2: aWHsd1h8TIJ1yXExGjnPQvf80Ai2gHjgfEzSwpY0740oDPSLUP8YD0qqCj43x3Q8ohS4fCZLJwA=
CURL_INFO_HEADER_IN: x-amz-request-id: W45M68E8CENK6F25
CURL_INFO_HEADER_IN: Date: Wed, 02 Oct 2024 08:51:25 GMT
CURL_INFO_HEADER_IN: Last-Modified: Mon, 09 Oct 2023 01:23:09 GMT
CURL_INFO_HEADER_IN: ETag: "3e23880cd7ac8a32ae71c55ad4f85c5d-24"
CURL_INFO_HEADER_IN: x-amz-storage-class: INTELLIGENT_TIERING
CURL_INFO_HEADER_IN: x-amz-server-side-encryption: AES256
CURL_INFO_HEADER_IN: Cache-Control: public, max-age=31536000, immutable
CURL_INFO_HEADER_IN: Accept-Ranges: bytes
CURL_INFO_HEADER_IN: Content-Range: bytes 0-32767/200959614
CURL_INFO_HEADER_IN: Content-Type: image/tiff; application=geotiff; profile=cloud-optimized
CURL_INFO_HEADER_IN: Server: AmazonS3
CURL_INFO_HEADER_IN: Content-Length: 32768
CURL_INFO_HEADER_IN: 
CURL_INFO_TEXT: Connection #0 to host sentinel-cogs.s3.us-west-2.amazonaws.com left intact
VSICURL: Got response_code=206
GDAL: GDALOpen(/vsicurl/https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif, this=0x112804fe0) succeeds as GTiff.
GDAL: GDALOpen(out.tif, this=0x121f62210) succeeds as GTiff.
GDAL: GDALClose(out.tif, this=0x121f62210)
GDAL: Using GTiff driver
GDAL: GDALOpen(out.tif, this=0x121f61ae0) succeeds as GTiff.
GDAL: GDALDefaultOverviews::OverviewScan()
GDAL: GDALClose(out.tif, this=0x121f61ae0)
GDAL: GDALOpen(/vsicurl/https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif, this=0x111fcea10) succeeds as GTiff.
GDAL: GDALDefaultOverviews::OverviewScan()
GDAL: GDALClose(/vsicurl/https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif, this=0x111fcea10)
GDAL: Computing area of interest: -94.2725, 44.1585, -92.8758, 45.1538
GDAL: QuietDelete(out.tif) invoking Delete()
GDAL: GDALOpen(out.tif, this=0x121f67700) succeeds as GTiff.
GDAL: GDALDefaultOverviews::OverviewScan()
GDAL: GDALClose(out.tif, this=0x121f67700)
GDAL: GDALDriver::Create(GTiff,out.tif,2215,2211,1,Byte,0x0)
WARP: Copying metadata from first source to destination dataset
GTiff: ScanDirectories()
GTiff: Opened 5490x5490 overview.
GTiff: Opened 2745x2745 overview.
GTiff: Opened 1373x1373 overview.
GTiff: Opened 687x687 overview.
WARP: Selecting overview level 1 for /vsicurl/https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif
GDAL: GDALOverviewDataset(/vsicurl/https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif, this=0x12282baa0) creation.
WARP: srcNoData=0.000000 dstNoData=0.000000
WARP: calling GDALSetRasterNoDataValue() for band#0
GDALWARP: Defining SKIP_NOSOURCE=YES
GDAL: Computing area of interest: -94.2725, 44.1585, -92.8758, 45.1538
VSICURL: Downloading 3096576-13139967 (https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif)...
CURL_INFO_TEXT: Couldn't find host sentinel-cogs.s3.us-west-2.amazonaws.com in the .netrc file; using defaults
CURL_INFO_TEXT: Found bundle for host: 0x111f3f360 [serially]
CURL_INFO_TEXT: Can not multiplex, even if we wanted to
CURL_INFO_TEXT: Re-using existing connection with host sentinel-cogs.s3.us-west-2.amazonaws.com
CURL_INFO_HEADER_OUT: GET /sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif HTTP/1.1
Host: sentinel-cogs.s3.us-west-2.amazonaws.com
User-Agent: GDAL/3.8.5
Accept: */*
Range: bytes=3096576-13139967

CURL_INFO_HEADER_IN: HTTP/1.1 206 Partial Content
CURL_INFO_HEADER_IN: x-amz-id-2: daI+K5fgNIqmSsx/+eJOL9nP0WR221sRPb9cBKyZq4OMnjSbMG/KKuPGLRAXhkWf6rUVf7yZLP4=
CURL_INFO_HEADER_IN: x-amz-request-id: DCC9TEV1T906696B
CURL_INFO_HEADER_IN: Date: Wed, 02 Oct 2024 08:51:26 GMT
CURL_INFO_HEADER_IN: Last-Modified: Mon, 09 Oct 2023 01:23:09 GMT
CURL_INFO_HEADER_IN: ETag: "3e23880cd7ac8a32ae71c55ad4f85c5d-24"
CURL_INFO_HEADER_IN: x-amz-storage-class: INTELLIGENT_TIERING
CURL_INFO_HEADER_IN: x-amz-server-side-encryption: AES256
CURL_INFO_HEADER_IN: Cache-Control: public, max-age=31536000, immutable
CURL_INFO_HEADER_IN: Accept-Ranges: bytes
CURL_INFO_HEADER_IN: Content-Range: bytes 3096576-13139967/200959614
CURL_INFO_HEADER_IN: Content-Type: image/tiff; application=geotiff; profile=cloud-optimized
CURL_INFO_HEADER_IN: Server: AmazonS3
CURL_INFO_HEADER_IN: Content-Length: 10043392
CURL_INFO_HEADER_IN: 
CURL_INFO_TEXT: Connection #0 to host sentinel-cogs.s3.us-west-2.amazonaws.com left intact
VSICURL: Got response_code=206
GDAL: GDAL_CACHEMAX = 12288 MB
GDAL: GDALWarpKernel()::GWKNearestByte() Src=0,0,2745x2745 Dst=0,0,2215x2211
GDAL: GDALClose(out.tif, this=0x122825390)
GDAL: GDALClose(/vsicurl/https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif, this=0x112804fe0)

We can also see that rasterio doesn't merge the Range request

I've tested with both gdal 3.8 and 3.9

This was referenced Oct 2, 2024
Closed
Merged
rouault added a commit to rouault/gdal that referenced this issue Oct 7, 2024
@rouault
GTiff: use main dataset GetSiblingFiles() for overviews
90cc612 
Fixes cogeotiff/rio-tiler#697
@rouault rouault mentioned this issue Oct 7, 2024
Merged
@rouault
Copy link

rouault commented Oct 7, 2024

should be resolved per OSGeo/gdal#10957

@vincentsarago
Copy link
Member Author

Merci @rouault 🙏

rouault added a commit to rouault/gdal that referenced this issue Oct 7, 2024
@rouault
GTiff: use main dataset GetSiblingFiles() for overviews
fc7e43e 
Fixes cogeotiff/rio-tiler#697
@four43 four43 mentioned this issue Nov 11, 2024
Merged
5 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

GTiff: use main dataset GetSiblingFiles() for overviews rouault/gdal
2 participants
@rouault @vincentsarago