Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Forward rasterio env options in multibase Reader #623

Open
vincentsarago opened this issue Jul 10, 2023 · 2 comments
Open

Forward rasterio env options in multibase Reader #623

vincentsarago opened this issue Jul 10, 2023 · 2 comments

Comments

@vincentsarago
Copy link
Member

in MultiBaseReader (STAC) we initiate a new rasterio.Env to be able to set the Header size from each file. Sadly when we do this we loose any env options passed in a top level env

with rasterio.Env(
    session=AWSSession(
        aws_access_key_id="MyDevseedId",
        aws_secret_access_key="MyDevseedKey",
    )
):
    with rasterio.open("s3://ds-satellite/cogs/NaturalEarth/world_grey.tif") as src:
        print(src.profile)
    
    with rasterio.Env():
        with rasterio.open("s3://ds-satellite/cogs/NaturalEarth/world_grey_1024_512.tif") as src:
            print(src.profile)

{'driver': 'GTiff', 'dtype': 'uint8', 'nodata': None, 'width': 21580, 'height': 10780, 'count': 3, 'crs': CRS.from_epsg(4326), 'transform': Affine(0.01666666666667, 0.0, -179.8333333333333,
       0.0, -0.01666666666667, 89.83333333333331), 'blockxsize': 128, 'blockysize': 128, 'tiled': True, 'compress': 'jpeg', 'interleave': 'pixel', 'photometric': 'ycbcr'}

rasterio/_base.pyx in rasterio._base.DatasetBase.__init__()

RasterioIOError: Access Denied

First reported in NASA-IMPACT/veda-backend#192

@vincentsarago
Copy link
Member Author

😭 This is also true for mosaic

from rio_tiler.tasks import create_tasks, filter_tasks
import rasterio
from rasterio.session import AWSSession


with rasterio.Env(
    session=AWSSession(
        aws_access_key_id="MyDevseedId",
        aws_secret_access_key="MyDevseedKey",
    ),
):
    paths = [
        "s3://ds-satellite/cogs/NaturalEarth/world_grey.tif",
        "s3://ds-satellite/cogs/NaturalEarth/world_grey_512_256.tif",
    ]
    
    def reader(path):
        with rasterio.open(path) as src:
            return src.profile
        
    tasks = filter_tasks(create_tasks(reader, paths, threads=2))

>> RasterioIOError: Access Denied

I always assumed that rasterio was picking the envs from top level but no. It will create a new Env per thread, I guess this is why stackstac does https://github.com/gjoseph92/stackstac/blob/main/stackstac/rio_reader.py

cc @ranchodeluxe @sharkinsspatial

@ranchodeluxe
Copy link

@vincentsarago : given the lack of motion on rasterio/rasterio#2866 is there a fix you're working on here now? Or is this something I can take swing at when I get a chance?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants