Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding webmap links to an item #15

Closed
pedro-cf opened this issue Apr 24, 2024 · 28 comments
Closed

Adding webmap links to an item #15

pedro-cf opened this issue Apr 24, 2024 · 28 comments

Comments

@pedro-cf
Copy link
Collaborator

Is your feature request related to a problem? Please describe.
Greetings I was doing some tests and trying to create an item that included links with this schema for WMS/WMTS/etc services (https://github.com/stac-extensions/web-map-links):

{
  "stac_version": "1.0.0",
	"collection": "test-collection",
  "stac_extensions": [
    "https://stac-extensions.github.io/web-map-links/v1.2.0/schema.json"
  ],
  "type": "Feature",
  "id": "item",
  "bbox": [
    172.9,
    1.3,
    173,
    1.4
  ],
  "geometry": {
    "type": "Polygon",
    "coordinates": [
      [
        [
          172.9,
          1.3
        ],
        [
          173,
          1.3
        ],
        [
          173,
          1.4
        ],
        [
          172.9,
          1.4
        ],
        [
          172.9,
          1.3
        ]
      ]
    ]
  },
  "properties": {
    "datetime": "2020-12-11T22:38:32Z"
  },
  "attribution": "© 2022 The Example Corp.",
  "links": [
    {
      "href": "https://example.com/examples/item.json",
      "rel": "self"
    },
    {
      "href": "https://maps.example.com/wmts",
      "rel": "wmts",
      "type": "image/png",
      "title": "RGB composite visualized through a WMTS",
      "wmts:layer": [
        "streets",
        "satellite"
      ]
    },
    {
      "href": "https://maps.example.com/wms",
      "rel": "wms",
      "type": "image/png",
      "title": "RGB composite visualized through a WMS",
      "wms:layers": [
        "rgb"
      ],
      "wms:transparent": true
    },
    {
      "href": "https://maps.example.com/xyz/{z}/{x}/{y}.jpg",
      "rel": "xyz",
      "type": "image/jpeg",
      "title": "RGB composite visualized through a XYZ"
    },
    {
      "href": "https://maps.example.com/item/tilejson.json",
      "rel": "tilejson",
      "title": "TileJSON",
      "type": "application/json"
    },
    {
      "href": "https://storage.googleapis.com/open-cogs/planet-stac/cocabamba-peru/3d-geofox.ai/3DTiles/tileset.json",
      "rel": "3d-tiles",
      "title": "3D Tiles",
      "type": "application/json"
    },
    {
      "href": "https://maps.example.com/item/example.pmtiles",
      "rel": "pmtiles",
      "title": "PMTiles",
      "type": "application/vnd.pmtiles",
      "pmtiles:layers": [
        "streets"
      ]
    }
  ],
  "assets": {}
}

Here is the actual item created in the database after a POST /collections/test-collection/items:

{
	"type": "Feature",
	"stac_version": "1.0.0",
	"stac_extensions": [
		"https://stac-extensions.github.io/web-map-links/v1.2.0/schema.json"
	],
	"id": "item",
	"geometry": {
		"type": "Polygon",
		"coordinates": [
			[
				[
					172.9,
					1.3
				],
				[
					173,
					1.3
				],
				[
					173,
					1.4
				],
				[
					172.9,
					1.4
				],
				[
					172.9,
					1.3
				]
			]
		]
	},
	"bbox": [
		172.9,
		1.3,
		173.0,
		1.4
	],
	"properties": {
		"datetime": "2020-12-11T22:38:32Z",
		"created": "2024-04-24T09:54:54.384886Z",
		"updated": "2024-04-24T09:54:54.384886Z"
	},
	"links": [
		{
			"rel": "self",
			"type": "application/geo+json",
			"href": "http://172.16.102.18:8086/collections/test-collection/items/item"
		},
		{
			"rel": "parent",
			"type": "application/json",
			"href": "http://172.16.102.18:8086/collections/test-collection"
		},
		{
			"rel": "collection",
			"type": "application/json",
			"href": "http://172.16.102.18:8086/collections/test-collection"
		},
		{
			"rel": "root",
			"type": "application/json",
			"href": "http://172.16.102.18:8086/"
		}
	],
	"assets": {},
	"collection": "test-collection"
}

It seems these links are being ignored? I'm a bit lost. Do I need to add a conformance class (don't know how) or does this require addition development ? or is it a bug?

@jonhealy1

@pedro-cf pedro-cf changed the title Adding a Conformance Class Adding webmap links to an item Apr 24, 2024
@jonhealy1
Copy link
Contributor

I feel like this is a bug but I'm not entirely sure - the best might be to run stac-fastapi-elasticsearch

@pedro-cf
Copy link
Collaborator Author

pedro-cf commented Apr 24, 2024

I feel like this is a bug but I'm not entirely sure - the best might be to run stac-fastapi-elasticsearch

Confirmed. Works just fine in https://github.com/stac-utils/stac-fastapi-elasticsearch-opensearch

Output from sfeo POST http://localhost:8080/collections/test-collection/items:

{
	"type": "Feature",
	"stac_version": "1.0.0",
	"stac_extensions": [
		"https://stac-extensions.github.io/web-map-links/v1.2.0/schema.json"
	],
	"id": "item",
	"geometry": {
		"type": "Polygon",
		"coordinates": [
			[
				[
					172.9,
					1.3
				],
				[
					173,
					1.3
				],
				[
					173,
					1.4
				],
				[
					172.9,
					1.4
				],
				[
					172.9,
					1.3
				]
			]
		]
	},
	"bbox": [
		172.9,
		1.3,
		173.0,
		1.4
	],
	"properties": {
		"datetime": "2020-12-11T22:38:32Z",
		"created": "2024-04-24T16:21:54.865138Z",
		"updated": "2024-04-24T16:21:54.865138Z"
	},
	"links": [
		{
			"href": "https://maps.example.com/wmts",
			"rel": "wmts",
			"type": "image/png",
			"title": "RGB composite visualized through a WMTS",
			"wmts:layer": [
				"streets",
				"satellite"
			]
		},
		{
			"href": "https://maps.example.com/wms",
			"rel": "wms",
			"type": "image/png",
			"title": "RGB composite visualized through a WMS",
			"wms:layers": [
				"rgb"
			],
			"wms:transparent": true
		},
		{
			"href": "https://maps.example.com/xyz/{z}/{x}/{y}.jpg",
			"rel": "xyz",
			"type": "image/jpeg",
			"title": "RGB composite visualized through a XYZ"
		},
		{
			"href": "https://maps.example.com/item/tilejson.json",
			"rel": "tilejson",
			"title": "TileJSON",
			"type": "application/json"
		},
		{
			"href": "https://storage.googleapis.com/open-cogs/planet-stac/cocabamba-peru/3d-geofox.ai/3DTiles/tileset.json",
			"rel": "3d-tiles",
			"title": "3D Tiles",
			"type": "application/json"
		},
		{
			"href": "https://maps.example.com/item/example.pmtiles",
			"rel": "pmtiles",
			"title": "PMTiles",
			"type": "application/vnd.pmtiles",
			"pmtiles:layers": [
				"streets"
			]
		}
	],
	"assets": {},
	"collection": "test-collection"
}

Any idea what might be causing it? @jonhealy1

@jonhealy1
Copy link
Contributor

I will have a look. stac-fastapi-mongo uses the core library from sfeos. sfeos is much more developed.

@pedro-cf
Copy link
Collaborator Author

pedro-cf commented Apr 24, 2024

It seems the issue is here:

mongo_item = self.item_serializer.stac_to_db(item, base_url)

@pedro-cf
Copy link
Collaborator Author

pedro-cf commented Apr 24, 2024

@jonhealy1 I tried upgrading stac-fastapi.core

2.1.0 -> 2.2.0 : fixes the issue and tests are 100% passing:
image

2.1.0 -> 2.3.0 : fixes the issue but causes 9 test errors:
image

Would you say it is safe to upgrade to 2.2.0 ?

@jonhealy1
Copy link
Contributor

If there are no errors it's good. Just make a pr to upgrade to v2.2.0. I will then relase a new version of stac-fastapi-mongo

@pedro-cf
Copy link
Collaborator Author

pedro-cf commented Apr 25, 2024

I would like to upgrade the stac-fastapi.core to 2.3.0

I've made this fork/branch: https://github.com/pedro-cf/stac-fastapi-mongo/tree/stac-fastapi.core-2.3.0

I've fixed 5 of the 9 tests. The errors were mostly due to this big change inte stac-fastapi.core:

PUT /collections is now PUT /collections/{collection_id}

I've also added this to the basic_auth tests to prevent awkawardness when calling tests outside of the github workflow, like with make test:

    if not os.getenv("BASIC_AUTH"):
        pytest.skip()

I can't seem to understand how to fix these remaining tests:

image

When running stac_fastapi/tests/api/test_api.py::test_datetime_non_interval:

async def test_datetime_non_interval(app_client, ctx):
dt_formats = [
"2020-02-12T12:30:22+00:00",
"2020-02-12T12:30:22.00Z",
"2020-02-12T12:30:22Z",
"2020-02-12T12:30:22.00+00:00",
]
for dt in dt_formats:
params = {
"datetime": dt,
"collections": [ctx.item["collection"]],
}
resp = await app_client.post("/search", json=params)
assert resp.status_code == 200
resp_json = resp.json()
# datetime is returned in this format "2020-02-12T12:30:22Z"
assert resp_json["features"][0]["properties"]["datetime"][0:19] == dt[0:19]

resp_json has no features and fails with IndexError: list index out of range
When I replicate this test in a real environment I don't run into this error.

I don't really understand the intricacies of conftest.py and ctx and app_client.

Maybe you can help me on understanding why app_client and app_client.post("/search", json=params) is acting differently?

@jonhealy1
Copy link
Contributor

The fourth test is a test we were struggling with that has now been fixed in stac-fastapi but the changes are not in the core library yet. There is an issue here that I made: stac-utils/stac-fastapi-elasticsearch-opensearch#230

@jonhealy1
Copy link
Contributor

It's interesting that all of the tests failing are datetime associated

@jonhealy1
Copy link
Contributor

It's also weird that they weren't failing before and the same tests pass in sfeos without any changes

@jonhealy1
Copy link
Contributor

I'm starting it now going to see what I can figure out.

@pedro-cf
Copy link
Collaborator Author

@jonhealy1 I think it's something to the with the test_item.json

I tried creating the item with the file itself:

curl --request POST \
  --url http://localhost:8084/collections/test-collection/items \
  --header 'Content-Type: application/json' \
  --data @./stac_fastapi/tests/data/test_item.json

and I tried creating the item by copy pasting the output from resp = await app_client.post("/search", json={}) in the tests...

and when I take what's stored in the database and json diffcheck the two:

image

@pedro-cf
Copy link
Collaborator Author

pedro-cf commented Apr 25, 2024

This curl is suppost to create the same item in the DB, but the datetime isn't being converted to the Mongo date format, it's say the same 2020-02-12T12:30:22+00:00:

curl --request POST --url http://localhost:8084/collections/test-collection/items --header 'Content-Type: application/json' --data '{"type":"Feature","geometry":{"type":"Polygon","coordinates":[[[152.15052873427666,-33.82243006904891],[150.1000346138806,-34.257132625788756],[149.5776607193635,-32.514709769700254],[151.6262528041627,-32.08081674221862],[152.15052873427666,-33.82243006904891]]]},"properties":{"datetime":"2020-02-12T12:30:22+00:00"},"id":"test-item","bbox":[149.57574,-34.25796,152.15194,-32.07915],"stac_version":"1.0.0","assets":{"ANG":{"title":"Angle Coefficients File","description":"Collection 2 Level-1 Angle Coefficients File (ANG)","href":"https://landsateuwest.blob.core.windows.net/landsat-c2/level-2/standard/oli-tirs/2021/108/066/LC08_L2SP_108066_20210712_20210720_02_T1/LC08_L2SP_108066_20210712_20210720_02_T1_ANG.txt","type":"text/plain"},"SR_B1":{"title":"Coastal/Aerosol Band (B1)","description":"Collection 2 Level-2 Coastal/Aerosol Band (B1) Surface Reflectance","gsd":30,"href":"https://landsateuwest.blob.core.windows.net/landsat-c2/level-2/standard/oli-tirs/2021/108/066/LC08_L2SP_108066_20210712_20210720_02_T1/LC08_L2SP_108066_20210712_20210720_02_T1_SR_B1.TIF","type":"image/tiff; application=geotiff; profile=cloud-optimized","eo:bands":[{"gsd":30,"name":"SR_B1","common_name":"coastal","center_wavelength":0.44,"full_width_half_max":0.02}],"proj:shape":[7731,7591],"proj:transform":[30,0,304185,0,-30,-843585]},"SR_B2":{"title":"Blue Band (B2)","description":"Collection 2 Level-2 Blue Band (B2) Surface Reflectance","gsd":30,"href":"https://landsateuwest.blob.core.windows.net/landsat-c2/level-2/standard/oli-tirs/2021/108/066/LC08_L2SP_108066_20210712_20210720_02_T1/LC08_L2SP_108066_20210712_20210720_02_T1_SR_B2.TIF","type":"image/tiff; application=geotiff; profile=cloud-optimized","eo:bands":[{"gsd":30,"name":"SR_B2","common_name":"blue","center_wavelength":0.48,"full_width_half_max":0.06}],"proj:shape":[7731,7591],"proj:transform":[30,0,304185,0,-30,-843585]},"SR_B3":{"title":"Green Band (B3)","description":"Collection 2 Level-2 Green Band (B3) Surface Reflectance","gsd":30,"href":"https://landsateuwest.blob.core.windows.net/landsat-c2/level-2/standard/oli-tirs/2021/108/066/LC08_L2SP_108066_20210712_20210720_02_T1/LC08_L2SP_108066_20210712_20210720_02_T1_SR_B3.TIF","type":"image/tiff; application=geotiff; profile=cloud-optimized","eo:bands":[{"gsd":30,"name":"SR_B3","common_name":"green","center_wavelength":0.56,"full_width_half_max":0.06}],"proj:shape":[7731,7591],"proj:transform":[30,0,304185,0,-30,-843585]},"SR_B4":{"title":"Red Band (B4)","description":"Collection 2 Level-2 Red Band (B4) Surface Reflectance","gsd":30,"href":"https://landsateuwest.blob.core.windows.net/landsat-c2/level-2/standard/oli-tirs/2021/108/066/LC08_L2SP_108066_20210712_20210720_02_T1/LC08_L2SP_108066_20210712_20210720_02_T1_SR_B4.TIF","type":"image/tiff; application=geotiff; profile=cloud-optimized","eo:bands":[{"gsd":30,"name":"SR_B4","common_name":"red","center_wavelength":0.65,"full_width_half_max":0.04}],"proj:shape":[7731,7591],"proj:transform":[30,0,304185,0,-30,-843585]},"SR_B5":{"title":"Near Infrared Band 0.8 (B5)","description":"Collection 2 Level-2 Near Infrared Band 0.8 (B5) Surface Reflectance","gsd":30,"href":"https://landsateuwest.blob.core.windows.net/landsat-c2/level-2/standard/oli-tirs/2021/108/066/LC08_L2SP_108066_20210712_20210720_02_T1/LC08_L2SP_108066_20210712_20210720_02_T1_SR_B5.TIF","type":"image/tiff; application=geotiff; profile=cloud-optimized","eo:bands":[{"gsd":30,"name":"SR_B5","common_name":"nir08","center_wavelength":0.86,"full_width_half_max":0.03}],"proj:shape":[7731,7591],"proj:transform":[30,0,304185,0,-30,-843585]},"SR_B6":{"title":"Short-wave Infrared Band 1.6 (B6)","description":"Collection 2 Level-2 Short-wave Infrared Band 1.6 (B6) Surface Reflectance","gsd":30,"href":"https://landsateuwest.blob.core.windows.net/landsat-c2/level-2/standard/oli-tirs/2021/108/066/LC08_L2SP_108066_20210712_20210720_02_T1/LC08_L2SP_108066_20210712_20210720_02_T1_SR_B6.TIF","type":"image/tiff; application=geotiff; profile=cloud-optimized","eo:bands":[{"gsd":30,"name":"SR_B6","common_name":"swir16","center_wavelength":1.6,"full_width_half_max":0.08}],"proj:shape":[7731,7591],"proj:transform":[30,0,304185,0,-30,-843585]},"SR_B7":{"title":"Short-wave Infrared Band 2.2 (B7)","description":"Collection 2 Level-2 Short-wave Infrared Band 2.2 (B7) Surface Reflectance","gsd":30,"href":"https://landsateuwest.blob.core.windows.net/landsat-c2/level-2/standard/oli-tirs/2021/108/066/LC08_L2SP_108066_20210712_20210720_02_T1/LC08_L2SP_108066_20210712_20210720_02_T1_SR_B7.TIF","type":"image/tiff; application=geotiff; profile=cloud-optimized","eo:bands":[{"gsd":30,"name":"SR_B7","common_name":"swir22","center_wavelength":2.2,"full_width_half_max":0.2}],"proj:shape":[7731,7591],"proj:transform":[30,0,304185,0,-30,-843585]},"ST_QA":{"title":"Surface Temperature Quality Assessment Band","description":"Landsat Collection 2 Level-2 Surface Temperature Band Surface Temperature Product","gsd":30,"href":"https://landsateuwest.blob.core.windows.net/landsat-c2/level-2/standard/oli-tirs/2021/108/066/LC08_L2SP_108066_20210712_20210720_02_T1/LC08_L2SP_108066_20210712_20210720_02_T1_ST_QA.TIF","type":"image/tiff; application=geotiff; profile=cloud-optimized","proj:shape":[7731,7591],"proj:transform":[30,0,304185,0,-30,-843585]},"ST_B10":{"title":"Surface Temperature Band (B10)","description":"Landsat Collection 2 Level-2 Surface Temperature Band (B10) Surface Temperature Product","gsd":100,"href":"https://landsateuwest.blob.core.windows.net/landsat-c2/level-2/standard/oli-tirs/2021/108/066/LC08_L2SP_108066_20210712_20210720_02_T1/LC08_L2SP_108066_20210712_20210720_02_T1_ST_B10.TIF","type":"image/tiff; application=geotiff; profile=cloud-optimized","eo:bands":[{"gsd":100,"name":"ST_B10","common_name":"lwir11","center_wavelength":10.9,"full_width_half_max":0.8}],"proj:shape":[7731,7591],"proj:transform":[30,0,304185,0,-30,-843585]},"MTL.txt":{"title":"Product Metadata File","description":"Collection 2 Level-1 Product Metadata File (MTL)","href":"https://landsateuwest.blob.core.windows.net/landsat-c2/level-2/standard/oli-tirs/2021/108/066/LC08_L2SP_108066_20210712_20210720_02_T1/LC08_L2SP_108066_20210712_20210720_02_T1_MTL.txt","type":"text/plain"},"MTL.xml":{"title":"Product Metadata File (xml)","description":"Collection 2 Level-1 Product Metadata File (xml)","href":"https://landsateuwest.blob.core.windows.net/landsat-c2/level-2/standard/oli-tirs/2021/108/066/LC08_L2SP_108066_20210712_20210720_02_T1/LC08_L2SP_108066_20210712_20210720_02_T1_MTL.xml","type":"application/xml"},"ST_DRAD":{"title":"Downwelled Radiance Band","description":"Landsat Collection 2 Level-2 Downwelled Radiance Band Surface Temperature Product","gsd":30,"href":"https://landsateuwest.blob.core.windows.net/landsat-c2/level-2/standard/oli-tirs/2021/108/066/LC08_L2SP_108066_20210712_20210720_02_T1/LC08_L2SP_108066_20210712_20210720_02_T1_ST_DRAD.TIF","type":"image/tiff; application=geotiff; profile=cloud-optimized","eo:bands":[{"gsd":30,"name":"ST_DRAD","description":"downwelled radiance"}],"proj:shape":[7731,7591],"proj:transform":[30,0,304185,0,-30,-843585]},"ST_EMIS":{"title":"Emissivity Band","description":"Landsat Collection 2 Level-2 Emissivity Band Surface Temperature Product","gsd":30,"href":"https://landsateuwest.blob.core.windows.net/landsat-c2/level-2/standard/oli-tirs/2021/108/066/LC08_L2SP_108066_20210712_20210720_02_T1/LC08_L2SP_108066_20210712_20210720_02_T1_ST_EMIS.TIF","type":"image/tiff; application=geotiff; profile=cloud-optimized","eo:bands":[{"gsd":30,"name":"ST_EMIS","description":"emissivity"}],"proj:shape":[7731,7591],"proj:transform":[30,0,304185,0,-30,-843585]},"ST_EMSD":{"title":"Emissivity Standard Deviation Band","description":"Landsat Collection 2 Level-2 Emissivity Standard Deviation Band Surface Temperature Product","gsd":30,"href":"https://landsateuwest.blob.core.windows.net/landsat-c2/level-2/standard/oli-tirs/2021/108/066/LC08_L2SP_108066_20210712_20210720_02_T1/LC08_L2SP_108066_20210712_20210720_02_T1_ST_EMSD.TIF","type":"image/tiff; application=geotiff; profile=cloud-optimized","eo:bands":[{"gsd":30,"name":"ST_EMSD","description":"emissivity standard deviation"}],"proj:shape":[7731,7591],"proj:transform":[30,0,304185,0,-30,-843585]}},"links":[{"href":"http://test-server/collections/test-collection/items/test-item","rel":"self","type":"application/geo+json"},{"href":"http://test-server/collections/test-collection","rel":"parent","type":"application/json"},{"href":"http://test-server/collections/test-collection","rel":"collection","type":"application/json"},{"href":"http://test-server/","rel":"root","type":"application/json"}],"collection":"test-collection"}'

@jonhealy1
Copy link
Contributor

Is it something to do with how mongo stores the date?

@pedro-cf
Copy link
Collaborator Author

The datetimes are being stored as strings in the database not as ISODate and the /search mechanism seems to be working with strings aswell.

The test_item.json creates an item in the db with the following format:

image

The stac_fastapi/tests/api/test_api.py::test_datetime_non_intervaltest runs the following first query:

Query:  {'$and': [{'collection': {'$in': ['test-collection']}}, {'properties.datetime': {'$gte': '2020-02-12T12:30:22+00:00'}}, {'properties.datetime': {'$lte': '2020-02-12T12:30:22+00:00'}}]}

As you see the date formats are different, and since the comparison is being done bettween date strings with different formats it doesn't work so the result is 0 features.

@pedro-cf
Copy link
Collaborator Author

Animation

@jonhealy1
Copy link
Contributor

I hacked this repo together pretty quickly. The previous releases are here and they are probably generally better. I wanted to show that we could build another backend from the sfeos core library.

@pedro-cf
Copy link
Collaborator Author

I hacked this repo together pretty quickly. The previous releases are here and they are probably generally better. I wanted to show that we could build another backend from the sfeos core library.

  1. But it's ok to focus on this repo and try to improve right ?

  2. How does SFEOs handle different datetime formats and does it store datetimes in a generic format ? I don't know much about elasticsearch...

  3. Do you think the datetimes should be stored as ISODates in MongoDB?

if yes:

  1. Do we just try to parse every field with dateutil.parser perhaps ? or just specific fields (datetime,start_datetime,end_datetime,created,updated)?

  2. Do we enforce ISODate formats only ?

@jonhealy1
Copy link
Contributor

I do like mongo a lot and I have used mongo for clients when I had the choice at my old job. I think the project is worth improving but almost everyone I have talked to in geospatial seems to have an opposite opinion. I was working on stac fastapi mongo first but no one was paying attention then I did elasticsearch and people were like wow - elasticsearch.

@pedro-cf
Copy link
Collaborator Author

I do like mongo a lot and I have used mongo for clients when I had the choice at my old job. I think the project is worth improving but almost everyone I have talked to in geospatial seems to have an opposite opinion. I was working on stac fastapi mongo first but no one was paying attention then I did elasticsearch and people were like wow - elasticsearch.

I think mongo has it's advantages, it's cheap, easy to setup managed services in the cloud and works well on-premises.

@jonhealy1
Copy link
Contributor

I attended a conference and Paul Ramsey, the guy who did A LOT of the geospatial stuff for postgis, asked the creator of stac-fastapi-pgstac how come they didn't build it in mongo. I thought that was pretty cool. The datetime stuff really confuses me. I haven't worked with it a lot. Everything I have done is just: 2020-03-01T00:00:00Z

@jonhealy1
Copy link
Contributor

jonhealy1 commented Apr 25, 2024

I didn't know anything about es when I started stac-fatsapi-es. Phil Varner stepped in to help out with that part later on behalf of a big company because they were wondering about the performance implications

@jonhealy1
Copy link
Contributor

I still don't know that much about es or mongo ....

@pedro-cf
Copy link
Collaborator Author

pedro-cf commented Apr 25, 2024

I think the big issue is a managed service for elasticsearch is very expensive on Azure/AWS and I'm not even sure if it would connect seamlessly with the sfeos-app. Have u tried deploying sfeos on azure f.e. connected with a Azure Elastic Search managed service?

@jonhealy1
Copy link
Contributor

There is opensearch I guess. I had always used the free tier on mongo atlas but then recently we moved a service to aws documentdb which is compatible. I have never had the opportunity to run stac-fastapi-es in the cloud. I'm pretty sure it would be fine.

@jonhealy1
Copy link
Contributor

Honestly it may be more productive for you to look at stac-fastapi-es. You should get more attention for your contributions. You are a very good programmer. I know it doesn't have a lot of stars etc. but I know of a few very big companies who use it or who have considered it. It would be good to move the basic auth stuff there for example. I haven't used Azure much - a little in school. I'm definitely down for continuing to work on this project don't get me wrong.

@jonhealy1
Copy link
Contributor

mongo atlas has serverless options which aws documentdb doesn't have. I wanted to use it for a non-profit we were working for but our pm thought otherwise.

@pedro-cf pedro-cf mentioned this issue Apr 25, 2024
4 tasks
@pedro-cf
Copy link
Collaborator Author

pedro-cf commented Apr 25, 2024

One of the many things changed in from 2.2.0 to 2.3.0:

POST /search

{
	"collections": [
		"test-collection"
	],
	"datetime": "2020-02-12T12:30:22Z/.."
}

Query in 2.2.0:

{'$and': [{'collection': {'$in': ['test-collection']}}, {'properties.datetime': {'$gte': '2020-02-12T12:30:22Z'}}, {'properties.datetime': {'$lte': '2200-12-01T12:31:12Z'}}]}

Query in 2.3.0:

{'$and': [{'collection': {'$in': ['test-collection']}}, {'properties.datetime': {'$gte': '2020-02-12T12:30:22.000Z'}}, {'properties.datetime': {'$lte': None}}]}

this query won't work with pymongo

@pedro-cf pedro-cf mentioned this issue Apr 25, 2024
4 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants