Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Remote UDP extension #515 #538

Merged
merged 4 commits into from
Jul 1, 2024
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,11 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

## Unreleased / Draft

### Added

- **New extensions:**
- [Remote UDP Extension](./extensions/remote-udp/README.md)

### Fixed

- `GET /file_formats`: Base paramater on top of normal JSON Schema, not Process JSON Schema
Expand Down
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,7 @@ See also the [changelog](CHANGELOG.md) and the [milestones](https://github.com/O
| ---------------------------------------------- | ------- | ------------ | ----------- |
| [Commercial Data](extensions/commercial-data/) | 0.1.0 | experimental | Provides an interface for discovering, ordering and using commercial data. |
| [Federation](extensions/federation/) | 0.1.0 | experimental | Covers federation aspects, i.e. where multiple back-ends are exposed as a single API. |
| [Remote UDP](extensions/remote-udp/) | 0.1.0 | experimental | Load user-defined processes that are hosted externally through the process namespace into process graphs. |

## Repository

Expand Down
65 changes: 65 additions & 0 deletions extensions/remote-udp/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
# Remote UDP Extension

The openEO API is a specification for interoperable cloud-based processing of large Earth observation datasets.

This extension enables user to load user-defined processes that are hosted on other hosts through the process namespace into process graphs.
m-mohr marked this conversation as resolved.
Show resolved Hide resolved

- Version: **0.1.0**
- Stability: **experimental**
- Conformance class: `https://api.openeo.org/extensions/remote-udp/0.1.0`

## Justification

The openEO API defines the `namespace` property in a process node of a process graph as follows:
> The following options are predefined by the openEO API, but additional namespaces may be introduced by back-ends or in a future version of the API.
> * `null` [...]
> * `backend` [...]
> * `user` [...]

This makes it possible that this extension adds additional allowed values to the `namespace` property.
m-mohr marked this conversation as resolved.
Show resolved Hide resolved

## Specification

This extension extends the `namespace` property of process graph nodes so that it accepts **absolute** URL with the protocols `https` (**recommended**) and `http` (discouraged). The URLs specified MUST either return

1. a single process (compatible to the endpoint `GET /process_graphs/{process_graph_id}`)
2. a list of processes (compatible to the endpoint `GET /process_graphs`)

If a URL is provided for the `namespace` property, the `id` property of the process graph node MUST be set to the `id` of the process.
m-mohr marked this conversation as resolved.
Show resolved Hide resolved
m-mohr marked this conversation as resolved.
Show resolved Hide resolved

### Compatibility

Compatible means in this context that the requests and responses must comply to the openEO API specification with the following exceptions:

- The `Authorization` header is not required and will usually not be sent.
m-mohr marked this conversation as resolved.
Show resolved Hide resolved
- Lists of processes MUST NOT paginate and the full process description MUST be provided for each process (i.e., the recommendation to omit large properties such as `process_graph` doesn't apply).
m-mohr marked this conversation as resolved.
Show resolved Hide resolved

### Client Considerations

Clients MUST only offer this functionality to users if the conformance class of this extension is listed in the `conformsTo` property of the `GET /` endpoint.
m-mohr marked this conversation as resolved.
Show resolved Hide resolved

The protocol `http` is discouraged for URLs as web-based clients may not be able to retrieve HTTP URLs from a HTTPS context.
For the same reason it is also RECOMMENDED to enable CORS for all URLs.
m-mohr marked this conversation as resolved.
Show resolved Hide resolved

### Error Handling

The following error SHOULD be reported if the namespace can't be resolved:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What if the URL can be resolved, but the document does not fit? e.g. it's not JSON, it does not follow expected schema, it doesn't contain a process_graph property, ...?

Shouldn't this also be a ProcessNamespaceInvalid?
Or would that be for standard errors like ProcessInvalid, ProcessGraphMissing? Note however that these don't have placeholders like for a reason, so that could lead to poor error messages

Copy link
Member Author

@m-mohr m-mohr Jul 1, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I guess that should fallback to the standard errors. Generally, back-ends can also define own error messages and codes. If specific common cases occur that are not covered and need a reason, we can fine-tune this later. Please give feedback :-)


- Code: `ProcessNamespaceInvalid`
- Message: `The value passed for namespace '{namespace}' in process '{process}' is invalid: {reason}`
- HTTP Status Code: 400

## Example

An exemplary process graph node:

```json
{
"process_id": "echo",
"namespace": "https://hub.openeo.org/processes/echo",
"arguments": {
"message": "Hello World"
},
m-mohr marked this conversation as resolved.
Show resolved Hide resolved
"result": true
}
```
Loading