Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Clean up band handling/filtering #76

Closed
soxofaan opened this issue Sep 4, 2019 · 5 comments
Closed

Clean up band handling/filtering #76

soxofaan opened this issue Sep 4, 2019 · 5 comments

Comments

@soxofaan
Copy link
Member

soxofaan commented Sep 4, 2019

The python client is currently quite confusing concerning band handling/filtering:

  • ImageCollection.band_filter(bands) expects a list of band names according to the docs. However the only way I actually get this function to work is with a single integer 0. When specifying a list of strings I get java.lang.String cannot be cast to java.lang.Integer, when specifying a single string I get error ('band must be an int, tuple, or list. Recieved', <class 'str'>, 'instead.'), when specifying an int other than 0 I get Band 1 does not exist
  • ImageCollection.band(name) expects a band name (string) and works properly
  • the current (0.4.2) spec defines actually the process filter_bands (not band_filter) and defines three possible arguments for band selection: bands, common_names and wavelengths https://open-eo.github.io/openeo-api/processreference/#filter_bands

I think it's best to get these things more in harmony

soxofaan added a commit to soxofaan/openeo-python-client that referenced this issue Sep 5, 2019
soxofaan added a commit to soxofaan/openeo-python-client that referenced this issue Sep 5, 2019
soxofaan added a commit to soxofaan/openeo-python-client that referenced this issue Sep 5, 2019
soxofaan added a commit to soxofaan/openeo-python-client that referenced this issue Sep 11, 2019
soxofaan added a commit to soxofaan/openeo-python-client that referenced this issue Sep 11, 2019
soxofaan added a commit to soxofaan/openeo-python-client that referenced this issue Sep 11, 2019
Allows to get a image collection band using common name ("red", "nir", ...): e.g. `im.band("red")`

related to Open-EO#76 Open-EO#77 and EP-3053
soxofaan added a commit to soxofaan/openeo-python-client that referenced this issue Sep 12, 2019
Allows to get a image collection band using common name ("red", "nir", ...): e.g. `im.band("red")`

related to Open-EO#76 Open-EO#77 and EP-3053
soxofaan added a commit to soxofaan/openeo-python-client that referenced this issue Sep 12, 2019
soxofaan added a commit to soxofaan/openeo-python-client that referenced this issue Sep 12, 2019
soxofaan added a commit to soxofaan/openeo-python-client that referenced this issue Sep 12, 2019
Allows to get a image collection band using common name ("red", "nir", ...): e.g. `im.band("red")`

related to Open-EO#76 Open-EO#77 and EP-3053
soxofaan added a commit to soxofaan/openeo-python-client that referenced this issue Sep 12, 2019
Allows to get a image collection band using common name ("red", "nir", ...): e.g. `im.band("red")`

related to Open-EO#76 Open-EO#77 and EP-3053
soxofaan added a commit to soxofaan/openeo-python-client that referenced this issue Sep 13, 2019
soxofaan added a commit to soxofaan/openeo-python-client that referenced this issue Sep 13, 2019
soxofaan added a commit to soxofaan/openeo-python-client that referenced this issue Sep 13, 2019
Allows to get a image collection band using common name ("red", "nir", ...): e.g. `im.band("red")`

related to Open-EO#76 Open-EO#77 and EP-3053
soxofaan added a commit to soxofaan/openeo-python-client that referenced this issue Sep 16, 2019
soxofaan added a commit to soxofaan/openeo-python-client that referenced this issue Sep 16, 2019
soxofaan added a commit to soxofaan/openeo-python-client that referenced this issue Sep 16, 2019
Allows to get a image collection band using common name ("red", "nir", ...): e.g. `im.band("red")`

related to Open-EO#76 Open-EO#77 and EP-3053
soxofaan added a commit to Open-EO/openeo-geopyspark-driver that referenced this issue Nov 5, 2019
jdries added a commit that referenced this issue Nov 21, 2019
…ters are applied, this way we might succeed in providing the right index to array_element
jdries added a commit that referenced this issue Nov 21, 2019
@Jaapel
Copy link

Jaapel commented Sep 21, 2021

Not sure what the progress of this ticket is, but I still receive errors when using .filter_bands(str). Luckily, the .band(str) function works well.

@soxofaan
Copy link
Member Author

Most original issues should be covered by now.

but I still receive errors when using .filter_bands(str)
what kind of errors and can you provide the steps to reproduce?

@Jaapel
Copy link

Jaapel commented Sep 21, 2021

from openeo import connect, Connection
from openeo.rest.datacube import DataCube
from typing import Dict, Union, List
import pathlib

vito_url: str = "https://openeo.vito.be/openeo/1.0"
con: Connection = connect(vito_url)
con.authenticate_oidc(provider_id="egi")

out_dir = pathlib.Path("output")
out_dir.mkdir(parents=True, exist_ok=True)

denia_harbour_bbox: Dict[str, Union[float, str]] = {"west": 0.10594089795383788, "east": 0.12937267590793944, "south": 38.83464299556706, "north": 38.85035302841166, "crs": "EPSG:4326"}
temporal_extent: List[str] = ["2020-08-01", "2021-01-01"]

collection = ("LANDSAT8_L1C", ["B06", "B05", "B03"])
band_names: str = ["swir1", "nir", "green"]
dc: DataCube = con.load_collection(
        collection_id=collection[0],
        spatial_extent=denia_harbour_bbox,
        temporal_extent=temporal_extent,
        bands=collection[1]
    ).add_dimension(name="source_name", label=collection[0], type="other") \
    .rename_labels(dimension="bands", source=collection[1], target=band_names)

green: DataCube = dc.filter_bands("green")
nir: DataCube = dc.filter_bands("nir")
ndwi: DataCube = (green - nir) / (green + nir)

ndwi.download(out_dir / "test.nc", format="netcdf")

results in:
OpenEoApiError: [400] unknown: Band 1 does not exist

@soxofaan
Copy link
Member Author

@Jaapel :

First of all, your script seems to work against the dev instance of the VITO backend (https://openeo-dev.vito.be/openeo/1.0).

Secondly and more importantly, the way you are computing ndwi here is not recommended.
You should use .bands() instead of .filter_bands().
.bands() is intended to do (spectral) "band math": doing calculations with values from spectral bands expressed with python math symbols (- and / in your case):

green: DataCube = dc.band("green")
nir: DataCube = dc.band("nir")
ndwi: DataCube = (green - nir) / (green + nir)

This will be translated to a reduce_dimension operation along the "bands" dimension, so that your 4D datacube (t,x,y,bands) is converted to a 3D data cube (t,x,y), holding the ndwi values, which is what you want.

With filter_bands(), there is no reduce_dimension, but the math operations are currently translated to a merge_cubes process with an overlap resolver. And while this appears to work on our dev instance, it should actually not work according to the spec. I'll probably have to file a separate bug for this.

@soxofaan
Copy link
Member Author

I think it's fine to close this old/outdated issue now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants