use factory from titiler.xarray #72

hrodmn · 2024-11-26T12:04:11Z

Now we can import features from titiler.xarray and just make a few modifications instead of defining custom methods for everything 🥳

reader.py just adds a cache to the default titiler.xarray.io.Reader
factory.py just adds /variables and /histogram endpoints to titiler.xarray.factory.TilerFactory, and has the custom /map endpoint template.

There are a few things that will change in this application as a result, though.

dropped features

If we want to keep either of these we will need to keep the custom factory methods rather than recycle the existing factory methods from titiler.xarray and rio-tiler:

No more support for reference files (use xarray.open_zarr and make aiohttp and s3fs optional titiler#1016 (comment))
No more multiscale zarr group zoom level functionality (Add Xarray sub-package titiler#1013 (comment))

smaller changes

tileMatrixSetId is now a required parameter for most endpoints (removed default of WebMercatorQuad)
the minzoom and maxzoom parameters have been removed from the /info response
band_metadata and band_description are no longer empty in /info response

The bounds in the tilejson.json response are no longer capped to -180, -90, 180, 90 because this chunk is not defined in the rio-tiler tilejson method

titiler-xarray/titiler/xarray/factory.py

Lines 413 to 416 in 1f31282

    
           minx, miny, maxx, maxy = zip( 
        
               [-180, -90, 180, 90], list(src_dst.geographic_bounds) 
        
           ) 
        
           bounds = [max(minx), max(miny), min(maxx), min(maxy)]

I will need to update the test expectations to match these feature changes but want to check with others first!

Unfortunately the renaming operation (titiler/xarray -> titiler/xarray_api makes the PR diff less useful than it could be for factory.py :/ so I will post a diff here:

factory.py diff

3,4c3
< from dataclasses import dataclass
< from typing import Dict, List, Literal, Optional, Type, Union
---
> from typing import List, Literal, Optional, Type, Union
9,11c8,9
< from fastapi import Depends, Path, Query
< from pydantic import conint
< from rio_tiler.models import Info
---
> from attrs import define
> from fastapi import Depends, Query
13c11
< from starlette.responses import HTMLResponse, Response
---
> from starlette.responses import HTMLResponse
17,19c15
< from titiler.core.dependencies import ColorFormulaParams
< from titiler.core.factory import BaseTilerFactory, img_endpoint_params
< from titiler.core.models.mapbox import TileJSON
---
> from titiler.core.dependencies import ColorFormulaParams, DefaultDependency
22,23c18,20
< from titiler.core.utils import render_image
< from titiler.xarray.reader import ZarrReader
---
> from titiler.xarray.dependencies import XarrayIOParams, XarrayParams
> from titiler.xarray.factory import TilerFactory as BaseTilerFactory
> from titiler.xarray_api.reader import XarrayReader
42,44c39,41
< @dataclass
< class ZarrTilerFactory(BaseTilerFactory):
<     """Zarr Tiler Factory."""
---
> @define(kw_only=True)
> class XarrayTilerFactory(BaseTilerFactory):
>     """Xarray Tiler Factory."""
46c43,44
<     reader: Type[ZarrReader] = ZarrReader
---
>     reader: Type[XarrayReader] = XarrayReader
>     reader_dependency: Type[DefaultDependency] = XarrayParams
49a48,52
>         super().register_routes()
>         self.variables()
> 
>     def variables(self) -> None:
>         """Register /variables endpoint"""
56,84c59,61
<         def variable_endpoint(
<             url: Annotated[str, Query(description="Dataset URL")],
<             group: Annotated[
<                 Optional[int],
<                 Query(
<                     description="Select a specific zarr group from a zarr hierarchy. Could be associated with a zoom level or dataset."
<                 ),
<             ] = None,
<             reference: Annotated[
<                 Optional[bool],
<                 Query(
<                     title="reference",
<                     description="Whether the dataset is a kerchunk reference",
<                 ),
<             ] = False,
<             decode_times: Annotated[
<                 Optional[bool],
<                 Query(
<                     title="decode_times",
<                     description="Whether to decode times",
<                 ),
<             ] = True,
<             consolidated: Annotated[
<                 Optional[bool],
<                 Query(
<                     title="consolidated",
<                     description="Whether to expect and open zarr store with consolidated metadata",
<                 ),
<             ] = True,
---
>         def get_variables(
>             src_path=Depends(self.path_dependency),
>             io_params=Depends(XarrayIOParams),
87,88c64,67
<             return self.reader.list_variables(
<                 url, group=group, reference=reference, consolidated=consolidated
---
>             return XarrayReader.list_variables(
>                 src_path=src_path,
>                 group=io_params.group,
>                 decode_times=io_params.decode_times,
91,423c70,72
<         @self.router.get(
<             "/info",
<             response_model=Info,
<             response_model_exclude_none=True,
<             response_class=JSONResponse,
<             responses={200: {"description": "Return dataset's basic info."}},
<         )
<         def info_endpoint(
<             url: Annotated[str, Query(description="Dataset URL")],
<             variable: Annotated[
<                 str,
<                 Query(description="Xarray Variable"),
<             ],
<             group: Annotated[
<                 Optional[int],
<                 Query(
<                     description="Select a specific zarr group from a zarr hierarchy, can be for pyramids or datasets. Can be used to open a dataset in HDF5 files."
<                 ),
<             ] = None,
<             reference: Annotated[
<                 bool,
<                 Query(
<                     title="reference",
<                     description="Whether the dataset is a kerchunk reference",
<                 ),
<             ] = False,
<             decode_times: Annotated[
<                 bool,
<                 Query(
<                     title="decode_times",
<                     description="Whether to decode times",
<                 ),
<             ] = True,
<             drop_dim: Annotated[
<                 Optional[str],
<                 Query(description="Dimension to drop"),
<             ] = None,
<             show_times: Annotated[
<                 Optional[bool],
<                 Query(description="Show info about the time dimension"),
<             ] = None,
<             consolidated: Annotated[
<                 Optional[bool],
<                 Query(
<                     title="consolidated",
<                     description="Whether to expect and open zarr store with consolidated metadata",
<                 ),
<             ] = True,
<         ) -> Info:
<             """Return dataset's basic info."""
<             with self.reader(
<                 url,
<                 variable=variable,
<                 group=group,
<                 reference=reference,
<                 decode_times=decode_times,
<                 drop_dim=drop_dim,
<                 consolidated=consolidated,
<             ) as src_dst:
<                 info = src_dst.info().model_dump()
<                 if show_times and "time" in src_dst.input.dims:
<                     times = [str(x.data) for x in src_dst.input.time]
<                     info["count"] = len(times)
<                     info["times"] = times
< 
<             return Info(**info)
< 
<         @self.router.get(r"/tiles/{z}/{x}/{y}", **img_endpoint_params)
<         @self.router.get(r"/tiles/{z}/{x}/{y}.{format}", **img_endpoint_params)
<         @self.router.get(r"/tiles/{z}/{x}/{y}@{scale}x", **img_endpoint_params)
<         @self.router.get(r"/tiles/{z}/{x}/{y}@{scale}x.{format}", **img_endpoint_params)
<         @self.router.get(r"/tiles/{tileMatrixSetId}/{z}/{x}/{y}", **img_endpoint_params)
<         @self.router.get(
<             r"/tiles/{tileMatrixSetId}/{z}/{x}/{y}.{format}", **img_endpoint_params
<         )
<         @self.router.get(
<             r"/tiles/{tileMatrixSetId}/{z}/{x}/{y}@{scale}x", **img_endpoint_params
<         )
<         @self.router.get(
<             r"/tiles/{tileMatrixSetId}/{z}/{x}/{y}@{scale}x.{format}",
<             **img_endpoint_params,
<         )
<         def tiles_endpoint(  # type: ignore
<             z: Annotated[
<                 int,
<                 Path(
<                     description="Identifier (Z) selecting one of the scales defined in the TileMatrixSet and representing the scaleDenominator the tile.",
<                 ),
<             ],
<             x: Annotated[
<                 int,
<                 Path(
<                     description="Column (X) index of the tile on the selected TileMatrix. It cannot exceed the MatrixHeight-1 for the selected TileMatrix.",
<                 ),
<             ],
<             y: Annotated[
<                 int,
<                 Path(
<                     description="Row (Y) index of the tile on the selected TileMatrix. It cannot exceed the MatrixWidth-1 for the selected TileMatrix.",
<                 ),
<             ],
<             url: Annotated[str, Query(description="Dataset URL")],
<             variable: Annotated[
<                 str,
<                 Query(description="Xarray Variable"),
<             ],
<             tileMatrixSetId: Annotated[  # type: ignore
<                 Literal[tuple(self.supported_tms.list())],
<                 f"Identifier selecting one of the TileMatrixSetId supported (default: '{self.default_tms}')",
<             ] = self.default_tms,
<             scale: Annotated[  # type: ignore
<                 conint(gt=0, le=4), "Tile size scale. 1=256x256, 2=512x512..."
<             ] = 1,
<             format: Annotated[
<                 ImageType,
<                 "Default will be automatically defined if the output image needs a mask (png) or not (jpeg).",
<             ] = None,
<             multiscale: Annotated[
<                 bool,
<                 Query(
<                     title="multiscale",
<                     description="Whether the dataset has multiscale groups (Zoom levels)",
<                 ),
<             ] = False,
<             reference: Annotated[
<                 bool,
<                 Query(
<                     title="reference",
<                     description="Whether the dataset is a kerchunk reference",
<                 ),
<             ] = False,
<             decode_times: Annotated[
<                 bool,
<                 Query(
<                     title="decode_times",
<                     description="Whether to decode times",
<                 ),
<             ] = True,
<             drop_dim: Annotated[
<                 Optional[str],
<                 Query(description="Dimension to drop"),
<             ] = None,
<             datetime: Annotated[
<                 Optional[str], Query(description="Slice of time to read (if available)")
<             ] = None,
<             post_process=Depends(self.process_dependency),
<             rescale=Depends(self.rescale_dependency),
<             color_formula=Depends(ColorFormulaParams),
<             colormap=Depends(self.colormap_dependency),
<             render_params=Depends(self.render_dependency),
<             consolidated: Annotated[
<                 Optional[bool],
<                 Query(
<                     title="consolidated",
<                     description="Whether to expect and open zarr store with consolidated metadata",
<                 ),
<             ] = True,
<             nodata=Depends(nodata_dependency),
<         ) -> Response:
<             """Create map tile from a dataset."""
<             tms = self.supported_tms.get(tileMatrixSetId)
<             with self.reader(
<                 url,
<                 variable=variable,
<                 group=z if multiscale else None,
<                 reference=reference,
<                 decode_times=decode_times,
<                 drop_dim=drop_dim,
<                 datetime=datetime,
<                 tms=tms,
<                 consolidated=consolidated,
<             ) as src_dst:
<                 image = src_dst.tile(
<                     x,
<                     y,
<                     z,
<                     tilesize=scale * 256,
<                     nodata=nodata if nodata is not None else src_dst.input.rio.nodata,
<                 )
< 
<             if post_process:
<                 image = post_process(image)
< 
<             if rescale:
<                 image.rescale(rescale)
< 
<             if color_formula:
<                 image.apply_color_formula(color_formula)
< 
<             content, media_type = render_image(
<                 image,
<                 output_format=format,
<                 colormap=colormap,
<                 **render_params,
<             )
< 
<             return Response(content, media_type=media_type)
< 
<         @self.router.get(
<             "/tilejson.json",
<             response_model=TileJSON,
<             responses={200: {"description": "Return a tilejson"}},
<             response_model_exclude_none=True,
<         )
<         @self.router.get(
<             "/{tileMatrixSetId}/tilejson.json",
<             response_model=TileJSON,
<             responses={200: {"description": "Return a tilejson"}},
<             response_model_exclude_none=True,
<         )
<         def tilejson_endpoint(  # type: ignore
<             request: Request,
<             url: Annotated[str, Query(description="Dataset URL")],
<             variable: Annotated[
<                 str,
<                 Query(description="Xarray Variable"),
<             ],
<             tileMatrixSetId: Annotated[  # type: ignore
<                 Literal[tuple(self.supported_tms.list())],
<                 f"Identifier selecting one of the TileMatrixSetId supported (default: '{self.default_tms}')",
<             ] = self.default_tms,
<             group: Annotated[
<                 Optional[int],
<                 Query(
<                     description="Select a specific zarr group from a zarr hierarchy, can be for pyramids or datasets. Can be used to open a dataset in HDF5 files."
<                 ),
<             ] = None,
<             reference: Annotated[
<                 bool,
<                 Query(
<                     title="reference",
<                     description="Whether the dataset is a kerchunk reference",
<                 ),
<             ] = False,
<             decode_times: Annotated[
<                 bool,
<                 Query(
<                     title="decode_times",
<                     description="Whether to decode times",
<                 ),
<             ] = True,
<             drop_dim: Annotated[
<                 Optional[str],
<                 Query(description="Dimension to drop"),
<             ] = None,
<             datetime: Annotated[
<                 Optional[str], Query(description="Slice of time to read (if available)")
<             ] = None,
<             tile_format: Annotated[
<                 Optional[ImageType],
<                 Query(
<                     description="Default will be automatically defined if the output image needs a mask (png) or not (jpeg).",
<                 ),
<             ] = None,
<             tile_scale: Annotated[
<                 int,
<                 Query(
<                     gt=0, lt=4, description="Tile size scale. 1=256x256, 2=512x512..."
<                 ),
<             ] = 1,
<             minzoom: Annotated[
<                 Optional[int],
<                 Query(description="Overwrite default minzoom."),
<             ] = None,
<             maxzoom: Annotated[
<                 Optional[int],
<                 Query(description="Overwrite default maxzoom."),
<             ] = None,
<             post_process=Depends(self.process_dependency),
<             rescale=Depends(self.rescale_dependency),
<             color_formula=Depends(ColorFormulaParams),
<             colormap=Depends(self.colormap_dependency),
<             render_params=Depends(self.render_dependency),
<             consolidated: Annotated[
<                 Optional[bool],
<                 Query(
<                     title="consolidated",
<                     description="Whether to expect and open zarr store with consolidated metadata",
<                 ),
<             ] = True,
<             nodata=Depends(nodata_dependency),
<         ) -> Dict:
<             """Return TileJSON document for a dataset."""
<             route_params = {
<                 "z": "{z}",
<                 "x": "{x}",
<                 "y": "{y}",
<                 "scale": tile_scale,
<                 "tileMatrixSetId": tileMatrixSetId,
<             }
<             if tile_format:
<                 route_params["format"] = tile_format.value
<             tiles_url = self.url_for(request, "tiles_endpoint", **route_params)
< 
<             qs_key_to_remove = [
<                 "tilematrixsetid",
<                 "tile_format",
<                 "tile_scale",
<                 "minzoom",
<                 "maxzoom",
<                 "group",
<             ]
<             qs = [
<                 (key, value)
<                 for (key, value) in request.query_params._list
<                 if key.lower() not in qs_key_to_remove
<             ]
<             if qs:
<                 tiles_url += f"?{urlencode(qs)}"
< 
<             tms = self.supported_tms.get(tileMatrixSetId)
< 
<             with self.reader(
<                 url,
<                 variable=variable,
<                 group=group,
<                 reference=reference,
<                 decode_times=decode_times,
<                 tms=tms,
<                 consolidated=consolidated,
<             ) as src_dst:
<                 # see https://github.com/corteva/rioxarray/issues/645
<                 minx, miny, maxx, maxy = zip(
<                     [-180, -90, 180, 90], list(src_dst.geographic_bounds)
<                 )
<                 bounds = [max(minx), max(miny), min(maxx), min(maxy)]
< 
<                 return {
<                     "bounds": bounds,
<                     "minzoom": minzoom if minzoom is not None else src_dst.minzoom,
<                     "maxzoom": maxzoom if maxzoom is not None else src_dst.maxzoom,
<                     "tiles": [tiles_url],
<                 }
---
>     def statistics(self) -> None:
>         """Register /statistics and /histogram endpoints"""
>         super().statistics()
432,456c81,82
<             url: Annotated[str, Query(description="Dataset URL")],
<             variable: Annotated[
<                 str,
<                 Query(description="Xarray Variable"),
<             ],
<             reference: Annotated[
<                 bool,
<                 Query(
<                     title="reference",
<                     description="Whether the dataset is a kerchunk reference",
<                 ),
<             ] = False,
<             consolidated: Annotated[
<                 bool,
<                 Query(
<                     title="consolidated",
<                     description="Whether to expect a consolidated dataset",
<                 ),
<             ] = True,
<             group: Annotated[
<                 Optional[int],
<                 Query(
<                     description="Select a specific zarr group from a zarr hierarchy, can be for pyramids or datasets. Can be used to open a dataset in HDF5 files."
<                 ),
<             ] = None,
---
>             src_path=Depends(self.path_dependency),
>             reader_params=Depends(self.reader_dependency),
459,463c85,89
<                 url,
<                 variable=variable,
<                 reference=reference,
<                 consolidated=consolidated,
<                 group=group,
---
>                 src_path=src_path,
>                 variable=reader_params.variable,
>                 group=reader_params.group,
>                 decode_times=reader_params.decode_times,
>                 datetime=reader_params.datetime,
477c103,105
<         @self.router.get("/map", response_class=HTMLResponse)
---
>     def map_viewer(self) -> None:
>         """Register /map endpoints"""
> 
481d108
<             url: Annotated[Optional[str], Query(description="Dataset URL")] = None,
484,485c111,113
<                 f"Identifier selecting one of the TileMatrixSetId supported (default: '{self.default_tms}')",
<             ] = self.default_tms,
---
>                 "Identifier selecting one of the supported TileMatrixSetIds",
>             ],
>             url: Annotated[Optional[str], Query(description="Dataset URL")] = None,
496,502d123
<             reference: Annotated[
<                 bool,
<                 Query(
<                     title="reference",
<                     description="Whether the dataset is a kerchunk reference",
<                 ),
<             ] = False,
545,547c166,167
<             templates = Jinja2Templates(
<                 directory="",
<                 loader=jinja2.ChoiceLoader([jinja2.PackageLoader(__package__, ".")]),
---
>             jinja2_env = jinja2.Environment(
>                 loader=jinja2.ChoiceLoader([jinja2.PackageLoader(__package__, ".")])
548a169,170
>             templates = Jinja2Templates(env=jinja2_env)
> 
551c173
<                     request, "tilejson_endpoint", tileMatrixSetId=tileMatrixSetId
---
>                     request, "tilejson", tileMatrixSetId=tileMatrixSetId

Resolves #71

src/titiler/xarray_api/factory.py

vincentsarago · 2024-11-26T12:45:59Z

src/titiler/xarray_api/factory.py

+            variable: Annotated[
+                Optional[str],
+                Query(description="Xarray Variable"),
+            ] = None,
+            group: Annotated[
+                Optional[int],
+                Query(
+                    description="Select a specific zarr group from a zarr hierarchy, can be for pyramids or datasets. Can be used to open a dataset in HDF5 files."
+                ),
+            ] = None,
+            decode_times: Annotated[
+                bool,
+                Query(
+                    title="decode_times",
+                    description="Whether to decode times",
+                ),
+            ] = True,
+            drop_dim: Annotated[
+                Optional[str],
+                Query(description="Dimension to drop"),
+            ] = None,
+            datetime: Annotated[
+                Optional[str], Query(description="Slice of time to read (if available)")
+            ] = None,


this is within the reader_dependency https://github.com/developmentseed/titiler/blob/a1955706f02671cefac7e2806e43bab46f2a04dd/src/titiler/xarray/titiler/xarray/dependencies.py#L14-L56

yes but I need the url parameter to be Optional for this endpoint. is there a way to override the type of a particular parameter in a dependency?

when no url is provided the user gets the interactive form instead of the actual map

is there a way to override the type of a particular parameter in a dependency?

Yes but not if it's only for a specific endpoint, you'll need to hardcode things directly at the endpoint level

I wasn't talking about the url parameter but maybe a better way of doing this would be to have separate endpoints for both webpage!

Yeah it might make more sense to have two endpoints like /map and /map-builder instead of this conditional logic.

src/titiler/xarray_api/factory.py

abarciauskas-bgse · 2024-11-27T16:43:00Z

pyproject.toml

-name = "titiler.xarray"
-description = "TiTiler extension for xarray."
+name = "titiler-multidim"
+description = "TiTiler application extension for titiler.xarray."


Suggested change

description = "TiTiler application extension for titiler.xarray."

description = "TiTiler application using titiler.xarray."

?

abarciauskas-bgse · 2024-11-27T16:45:50Z

src/titiler/multidim/factory.py

+from titiler.core.dependencies import ColorFormulaParams, DefaultDependency
+from titiler.core.resources.enums import ImageType
+from titiler.core.resources.responses import JSONResponse
+from titiler.multidim.reader import XarrayReader


non-blocking but now that I'm looking at this I am wondering if the naming convention should help distinguish local application dependencies vs imported dependencies... like titiler_app.multidim|xarray or titiler.app.multidim|xarrat

abarciauskas-bgse · 2024-11-27T16:47:41Z

src/titiler/multidim/map-form.html

+                document.addEventListener('DOMContentLoaded', function() {
+                    var colormapDropdown = document.getElementById("colormap-dropdown");
+
+                    // Fetch colormaps from API


abarciauskas-bgse · 2024-11-27T16:49:51Z

src/titiler/multidim/map-form.html

-                <label for="reference">Is this URL for a reference file (i.e. <a href="https://fsspec.github.io/kerchunk">kerchunk</a>)?:</label>
-                <br />
-                <br />
-                <input type="checkbox" id="consolidated" name="consolidated" checked>


Just noting that I think it's ok if we remove the consolidated parameter as by default xarray will try and read consolidated and fallback to reading unconsolidated (https://docs.xarray.dev/en/v2023.06.0/generated/xarray.open_zarr.html)

j08lue · 2024-11-27T22:05:08Z

No more support for reference files

I see this was discussed here: developmentseed/titiler#1016 (comment)

So no more Kerchunk support? While I see how virtual Zarr is the better approach, I think some projects that are looking to use TiTiler for multidim datasets are still using Kerchunk.

For example on the UKRI / UK EO DataHub, this may be data they need to load with TiTiler: https://radiantearth.github.io/stac-browser/#/external/eocis.org/stac/collections/eocis-lst-slstrA-day

Example reference file: https://eocis.org/data/eocis-lst-slstrA-day-kerchunk/2024/10/EOCIS_-LST-L3C-LST-SLSTRA-0.01deg_1DAILY_DAY-20241020000000-fv4.00-kerchunk.json

Would there be an easy enough way for them to enable this in a custom runtime / fork of TiTiler-Xarray, if they need it?

Not a blocker for merging this, but we should point out ways forward to them.

I see VirtualiZarr and Icechunk have docs on migrating from Kerchunk / creating virtual datasets:

👌

hrodmn · 2024-11-28T02:20:06Z

Thanks for the reviews @j08lue and @abarciauskas-bgse!

Would there be an easy enough way for them to enable this in a custom runtime / fork of TiTiler-Xarray, if they need it?

It don't think it would be too hard to keep that feature working in this application! If it is important to any users right now we can define a custom version of open_xarray_dataset rather than importing it from titiler.xarray.

It will be very simple if we can identify kerchunk reference files from the url parameter. Can we assume that any file with a .json file extension will be a kerchunk reference? If so, we would only need to add some logic to a custom opener function. If we truly need an additional argument like reference we would need to redefine all of the factory endpoints to use it :/.

j08lue · 2024-11-28T08:56:15Z

It will be very simple if we can identify kerchunk reference files from the url parameter

If it is simple, either via .json or the reference:// prefix that @vincentsarago suggested here, I think we should add it for now.

Maybe mark Kerchunk support for deprecation already, to nudge existing users time to move to Icechunk or so.

I guess this adds a few dependencies, though, which would be great to be able to avoid longer-term.

vincentsarago · 2024-11-28T09:39:12Z

Just a reminder, we've drop kerchunk reference support in titiler-xarray because it needed some nested option (e.g you could have your kerchunk on one bucket in S3 but the netcdf on another bucket but maybe private, meaning that you need to be able to define both kwargs when using fsspec, maybe I'm wrong but that what I understood at first.

We could re-add it back but with the assumption that the reference file will always sit next to the data

abarciauskas-bgse · 2024-12-04T17:04:49Z

@j08lue I discussed briefly with @hrodmn - I agree we should add back support so we can support the UKRI / UK EO DataHub datasets. However I'm not sure it makes sense to add it to this application specifically since the deployment will only work with datasets in S3 (due to the lambda's subnet configuration), so even if we had back reference support it won't work for the example https://eocis.org/data/eocis-lst-slstrA-day-kerchunk/2024/10/EOCIS_-LST-L3C-LST-SLSTRA-0.01deg_1DAILY_DAY-20241020000000-fv4.00-kerchunk.json. When would you like to demonstrate this functionality to UKRI / UK EO DataHub? I think it would make the most sense to create a separate demo deployment for them which has this reference support.

j08lue · 2024-12-04T20:58:37Z

create a separate demo deployment for them which has this reference support.

Thanks for being so considerate. The UK EODH engineering team is deploying their own instance of TiTiler-Xarray (in their k8s cluster), so we do not need a (demo) deployment that works for their datasets with references.

It would just be great if they could keep using TiTiler-Xarray and did not need to build their own separate application for reference support. Or if, that we could describe how that is done.

If I understand you correctly, TiTiler-Xarray will get back support for references, but it is a matter of the deployment to make sure the service has access to the relevant resources.

dwest77a · 2024-12-06T15:46:17Z

zarr-developers/VirtualiZarr#321 Added this issue for reference with using VirtualiZarr to create Icechunk stores.

There is also an issue converting an existing Kerchunk file to Icechunk using VirtualiZarr due to 'inline' data, where chunks are written as base64 chunks into the file, instead of as a reference. Conversion is not currently possible for files where this is the case - which covers a significant portion of the CEDA kerchunk files.

hrodmn added 3 commits November 26, 2024 05:35

use factory from titiler.xarray

039b8c8

fix map endpoint tests

4bfe7cb

remove print statements

4f443aa

vincentsarago reviewed Nov 26, 2024

View reviewed changes

src/titiler/xarray_api/factory.py Outdated Show resolved Hide resolved

use self.reader

b92e068

vincentsarago reviewed Nov 26, 2024

View reviewed changes

src/titiler/xarray_api/factory.py Outdated Show resolved Hide resolved

vincentsarago reviewed Nov 26, 2024

View reviewed changes

src/titiler/xarray_api/factory.py Outdated Show resolved Hide resolved

hrodmn added 4 commits November 27, 2024 05:57

remove redundant nodata_dependency

74d46b9

update test expectations

6d51287

clean up bounds in text fixtures

a389a45

xarray_api -> multidim

144e9b2

hrodmn marked this pull request as ready for review November 27, 2024 13:31

hrodmn requested a review from abarciauskas-bgse November 27, 2024 13:31

abarciauskas-bgse reviewed Nov 27, 2024

View reviewed changes

abarciauskas-bgse approved these changes Nov 27, 2024

View reviewed changes

j08lue mentioned this pull request Nov 28, 2024

Deploy demo of dynamic tiles for Xarray-read data formats with Data Access BB EOEPCA/data-access#95

Open

1 task

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

use factory from titiler.xarray #72

use factory from titiler.xarray #72

hrodmn commented Nov 26, 2024 •

edited

Loading

vincentsarago Nov 26, 2024 •

edited

Loading

hrodmn Nov 26, 2024

vincentsarago Nov 26, 2024

hrodmn Nov 26, 2024

abarciauskas-bgse Nov 27, 2024

abarciauskas-bgse Nov 27, 2024

abarciauskas-bgse Nov 27, 2024

abarciauskas-bgse Nov 27, 2024

j08lue commented Nov 27, 2024 •

edited

Loading

hrodmn commented Nov 28, 2024 •

edited

Loading

j08lue commented Nov 28, 2024 •

edited

Loading

vincentsarago commented Nov 28, 2024

abarciauskas-bgse commented Dec 4, 2024

j08lue commented Dec 4, 2024

dwest77a commented Dec 6, 2024 •

edited

Loading

	minx, miny, maxx, maxy = zip(
	[-180, -90, 180, 90], list(src_dst.geographic_bounds)
	)
	bounds = [max(minx), max(miny), min(maxx), min(maxy)]

	description = "TiTiler application extension for titiler.xarray."
	description = "TiTiler application using titiler.xarray."

use factory from titiler.xarray #72

Are you sure you want to change the base?

use factory from titiler.xarray #72

Conversation

hrodmn commented Nov 26, 2024 • edited Loading

dropped features

smaller changes

vincentsarago Nov 26, 2024 • edited Loading

Choose a reason for hiding this comment

hrodmn Nov 26, 2024

Choose a reason for hiding this comment

vincentsarago Nov 26, 2024

Choose a reason for hiding this comment

hrodmn Nov 26, 2024

Choose a reason for hiding this comment

abarciauskas-bgse Nov 27, 2024

Choose a reason for hiding this comment

abarciauskas-bgse Nov 27, 2024

Choose a reason for hiding this comment

abarciauskas-bgse Nov 27, 2024

Choose a reason for hiding this comment

abarciauskas-bgse Nov 27, 2024

Choose a reason for hiding this comment

j08lue commented Nov 27, 2024 • edited Loading

hrodmn commented Nov 28, 2024 • edited Loading

j08lue commented Nov 28, 2024 • edited Loading

vincentsarago commented Nov 28, 2024

abarciauskas-bgse commented Dec 4, 2024

j08lue commented Dec 4, 2024

dwest77a commented Dec 6, 2024 • edited Loading

hrodmn commented Nov 26, 2024 •

edited

Loading

vincentsarago Nov 26, 2024 •

edited

Loading

j08lue commented Nov 27, 2024 •

edited

Loading

hrodmn commented Nov 28, 2024 •

edited

Loading

j08lue commented Nov 28, 2024 •

edited

Loading

dwest77a commented Dec 6, 2024 •

edited

Loading