From 95e67b6dcaae357e8c5059757299d3605e155d1f Mon Sep 17 00:00:00 2001 From: Jessica Scheick Date: Mon, 22 Jul 2024 12:19:15 -0400 Subject: [PATCH] add backend intro and how-to diagram (#9175) * add backend intro and how-to diagram * update what's new * fix link style [skip-ci] * update numpy nan syntax to address docs build fail [skip-ci] * improve some spacing on diagram * fix items not rendering properly due to typos [skip-ci] * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update python syntax * use html code blocks to fix spacing [skip-ci] * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * suppress ipython block output * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * change to okexcept [skip-ci] * re-add supress so error output is ignored [skip-ci] * Update doc/user-guide/io.rst * add per-line links to diagram [skip-ci] --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Maximilian Roos <5635139+max-sixty@users.noreply.github.com> --- doc/internals/how-to-add-new-backend.rst | 2 +- doc/user-guide/io.rst | 75 ++++++++++++++++++++++++ doc/whats-new.rst | 4 +- 3 files changed, 79 insertions(+), 2 deletions(-) diff --git a/doc/internals/how-to-add-new-backend.rst b/doc/internals/how-to-add-new-backend.rst index 4352dd3df5b..a979abe34e2 100644 --- a/doc/internals/how-to-add-new-backend.rst +++ b/doc/internals/how-to-add-new-backend.rst @@ -4,7 +4,7 @@ How to add a new backend ------------------------ Adding a new backend for read support to Xarray does not require -to integrate any code in Xarray; all you need to do is: +one to integrate any code in Xarray; all you need to do is: - Create a class that inherits from Xarray :py:class:`~xarray.backends.BackendEntrypoint` and implements the method ``open_dataset`` see :ref:`RST backend_entrypoint` diff --git a/doc/user-guide/io.rst b/doc/user-guide/io.rst index da414bc383e..85c47334858 100644 --- a/doc/user-guide/io.rst +++ b/doc/user-guide/io.rst @@ -19,6 +19,81 @@ format (recommended). np.random.seed(123456) +You can `read different types of files `_ +in `xr.open_dataset` by specifying the engine to be used: + +.. ipython:: python + :okexcept: + :suppress: + + import xarray as xr + + xr.open_dataset("my_file.grib", engine="cfgrib") + +The "engine" provides a set of instructions that tells xarray how +to read the data and pack them into a `dataset` (or `dataarray`). +These instructions are stored in an underlying "backend". + +Xarray comes with several backends that cover many common data formats. +Many more backends are available via external libraries, or you can `write your own `_. +This diagram aims to help you determine - based on the format of the file you'd like to read - +which type of backend you're using and how to use it. + +Text and boxes are clickable for more information. +Following the diagram is detailed information on many popular backends. +You can learn more about using and developing backends in the +`Xarray tutorial JupyterBook `_. + +.. mermaid:: + :alt: Flowchart illustrating how to choose the right backend engine to read your data + + flowchart LR + built-in-eng["""Is your data stored in one of these formats? + - netCDF4 (netcdf4) + - netCDF3 (scipy) + - Zarr (zarr) + - DODS/OPeNDAP (pydap) + - HDF5 (h5netcdf) + """] + + built-in("""You're in luck! Xarray bundles a backend for this format. + Open data using xr.open_dataset(). We recommend + always setting the engine you want to use.""") + + installed-eng["""One of these formats? + - GRIB (cfgrib) + - TileDB (tiledb) + - GeoTIFF, JPEG-2000, ESRI-hdf (rioxarray, via GDAL) + - Sentinel-1 SAFE (xarray-sentinel) + """] + + installed("""Install the package indicated in parentheses to your + Python environment. Restart the kernel and use + xr.open_dataset(files, engine='rioxarray').""") + + other("""Ask around to see if someone in your data community + has created an Xarray backend for your data type. + If not, you may need to create your own or consider + exporting your data to a more common format.""") + + built-in-eng -->|Yes| built-in + built-in-eng -->|No| installed-eng + + installed-eng -->|Yes| installed + installed-eng -->|No| other + + click built-in-eng "https://docs.xarray.dev/en/stable/getting-started-guide/faq.html#how-do-i-open-format-x-file-as-an-xarray-dataset" + click other "https://docs.xarray.dev/en/stable/internals/how-to-add-new-backend.html" + + classDef quesNodefmt fill:#9DEEF4,stroke:#206C89,text-align:left + class built-in-eng,installed-eng quesNodefmt + + classDef ansNodefmt fill:#FFAA05,stroke:#E37F17,text-align:left,white-space:nowrap + class built-in,installed,other ansNodefmt + + linkStyle default font-size:20pt,color:#206C89 + + .. _io.netcdf: netCDF diff --git a/doc/whats-new.rst b/doc/whats-new.rst index 30df104f168..fe678e7b7ee 100644 --- a/doc/whats-new.rst +++ b/doc/whats-new.rst @@ -84,7 +84,9 @@ Bug fixes Documentation ~~~~~~~~~~~~~ -- Adds a flow-chart diagram to help users navigate help resources (`Discussion #8990 `_). +- Adds intro to backend section of docs, including a flow-chart to navigate types of backends (:pull:`9175`). + By `Jessica Scheick `_. +- Adds a flow-chart diagram to help users navigate help resources (`Discussion #8990 `_, :pull:`9147`). By `Jessica Scheick `_. - Improvements to Zarr & chunking docs (:pull:`9139`, :pull:`9140`, :pull:`9132`) By `Maximilian Roos `_.