diff --git a/README.md b/README.md index 5bf2ca89..250005f4 100644 --- a/README.md +++ b/README.md @@ -20,29 +20,29 @@ All changes made to the specification can be reviewed in the [GitHub repository] ## Abstract The __Dataspace Protocol__ is a set of specifications designed to facilitate interoperable data sharing between entities governed by usage control and based on Web technologies. These specifications -define the schemas and protocols required for entities to publish data, negotiate usage agreements, and access data as part of a federation of technical systems termed a -__*dataspace*__. +define the schemas and protocols required for entities to publish data, negotiate [Agreements](./model/terminology.md#agreement), and access data as part of a federation of technical systems termed a +[Dataspace](./model/terminology.md#dataspace). ## Introduction -Sharing data between autonomous entities requires the provision of metadata to facilitate the transfer of datasets by making use of a data transfer (or application layer) protocol. +Sharing data between autonomous entities requires the provision of metadata to facilitate the transfer of [Datasets](./model/terminology.md#dataset) by making use of a data transfer (or application layer) protocol. The __Dataspace Protocol__ defines how this metadata is provisioned: -1. How datasets are deployed as [DCAT Catalogs](https://www.w3.org/TR/vocab-dcat-3/) and usage control is expressed as [ODRL Policies](https://www.w3.org/TR/odrl-model/). -2. How contract agreements that govern data usage are syntactically expressed and electronically negotiated. -3. How datasets are accessed using data transfer protocols. +1. How [Datasets](./model/terminology.md#dataset) are deployed as [DCAT Catalogs](https://www.w3.org/TR/vocab-dcat-3/#Class:Catalog) and usage control is expressed as [ODRL Policies](https://www.w3.org/TR/odrl-model/). +2. How [Agreements](./model/terminology.md#agreement) that govern data usage are syntactically expressed and electronically negotiated. +3. How [Datasets](./model/terminology.md#dataset) are accessed using Transfer Process Protocols. These specifications build on protocols located in the [ISO OSI model (ISO/IEC 7498-1:1994)](https://www.iso.org/standard/20269.html) layers, like HTTPS. The purpose of this specification is to define interactions between systems independent of such protocols, but describing how to implement it in an unambiguous and extensible way. -To do so, the messages that are exchanged during the process are described in this specification and the states and their transitions are specified as state machines, based on the key terms and concepts of a data space. -On this foundation the binding to data transfer protocols, like HTTPS, is described. +To do so, the messages that are exchanged during the process are described in this specification and the states and their transitions are specified as state machines, based on the key terms and concepts of a [Dataspace](./model/terminology.md#dataspace). +On this foundation the binding to Transfer Process Protocols, like HTTPS, is described. The specifications are organized into the following documents: * __*Dataspace Model*__ and __*Dataspace Terminology*__ documents that define key terms. -* __*Catalog Protocol*__ and __*Catalog HTTPS Binding*__ documents that define how DCAT Catalogs are published and accessed as HTTPS endpoints respectively. -* __*Contract Negotiation Protocol*__ and __*Contract Negotiation HTTPS Binding*__ documents that define how contract negotiations are conducted and requested via HTTPS endpoints. -* __*Transfer Process Protocol*__ and __*Transfer Process HTTPS Binding*__ documents that define how transfer processes using a given data transfer protocol are governed via HTTPS +* __*Catalog Protocol*__ and __*Catalog HTTPS Binding*__ documents that define how [DCAT Catalogs](https://www.w3.org/TR/vocab-dcat-3/#Class:Catalog) are published and accessed as HTTPS endpoints respectively. +* __*Contract Negotiation Protocol*__ and __*Contract Negotiation HTTPS Binding*__ documents that define how [Contract Negotiations](./model/terminology.md#contract-negotiation) are conducted and requested via HTTPS endpoints. +* __*Transfer Process Protocol*__ and __*Transfer Process HTTPS Binding*__ documents that define how [Transfer Processes](./model/terminology.md#transfer-process) using a given data transfer protocol are governed via HTTPS endpoints. This specification does not cover the data transfer process as such. @@ -51,21 +51,21 @@ As an implication, the data transfer can be conducted in a separated process if ### Context of this specification -The __Dataspace Protocol__ is used in the context of data spaces as described and defined in the subsequent sections with the purpose to support interoperability. -In this context, the specification provides fundamental technical interoperability for participants in data spaces and therefore the protocol specified here is required to join any data space as specified [here](). -Beyond the technical interoperability measures described in this specification, semantic interoperability should also be addressed by the participants. On the perspective of the data space, interoperability needs to be addressed also on the level of trust, on organizational level and on legal level. -The aspect of cross data space communication is not subject of this document, as this is addressed by the data spaces' organizational and legal agreements. +The __Dataspace Protocol__ is used in the context of [Dataspaces](./model/terminology.md#dataspace) as described and defined in the subsequent sections with the purpose to support interoperability. +In this context, the specification provides fundamental technical interoperability for [Participants](./model/terminology.md#participant) in [Dataspaces](./model/terminology.md#dataspace) and therefore the protocol specified here is required to join any [Dataspace](./model/terminology.md#dataspace). +Beyond the technical interoperability measures described in this specification, semantic interoperability should also be addressed by the [Participants](./model/terminology.md#participant). On the perspective of the [Dataspace](./model/terminology.md#dataspace), interoperability needs to be addressed also on the level of trust, on organizational level and on legal level. +The aspect of cross [Dataspace](./model/terminology.md#dataspace) communication is not subject of this document, as this is addressed by the [Dataspaces'](./model/terminology.md#dataspace) organizational and legal agreements. -The interaction of participants in a data space is conducted by the participant agents, so-called Connectors, which implement the protocols described above. -While most interactions take place between Connectors, some interactions with other systems are required. +The interaction of [Participants](./model/terminology.md#participant) in a [Dataspace](./model/terminology.md#dataspace) is conducted by the [Participant Agents](./model/terminology.md#participant-agent), so-called [Connectors](./model/terminology.md#connector--data-service-), which implement the protocols described above. +While most interactions take place between [Connectors](./model/terminology.md#connector--data-service-), some interactions with other systems are required. The figure below provides an overview on the context of this specification. -An Identity Provider realizes the required interfaces and provides required information to implement Trust Framework of a data space. -The validation of the identity of a given participant agent and the validation of additional claims is the fundamental mechanism. The structure and content of such claims and identity may vary between different data spaces, as well as the structure of such an Identity Provider, e.g. a centralized system, a decentralized system or a federated system. +An [Identity Provider](./model/terminology.md#identity-provider) realizes the required interfaces and provides required information to implement Trust Framework of a [Dataspace](./model/terminology.md#dataspace). +The validation of the identity of a given [Participant Agent](./model/terminology.md#participant-agent) and the validation of additional claims is the fundamental mechanism. The structure and content of such claims and identity may vary between different [Dataspaces](./model/terminology.md#dataspace), as well as the structure of such an Identity Provider, e.g. a centralized system, a decentralized system or a federated system. -A connector will implement additional internal functionalities, like monitoring or Policy Engines, as appropriate. It is not covered by this specification, if a connector implements such or how. +A [Connector](./model/terminology.md#connector--data-service-) will implement additional internal functionalities, like monitoring or policy engines, as appropriate. It is not covered by this specification, if a [Connector](./model/terminology.md#connector--data-service-) implements such or how. -The same applies for the data, which is transferred between the systems. While this document does not define the transport protocol, the structure, syntax and semantics of the data, a specification for those aspects is required and subject to the agreements of the participants or the data space. +The same applies for the data, which is transferred between the systems. While this document does not define the transport protocol, the structure, syntax and semantics of the data, a specification for those aspects is required and subject to the agreements of the [Participants](./model/terminology.md#participant) or the [Dataspace](./model/terminology.md#dataspace). ![Overview on protocol and context](./resources/figures/ProtocolOverview.png) diff --git a/best.practices/README.md b/best.practices/README.md index 7848adc0..162f8156 100644 --- a/best.practices/README.md +++ b/best.practices/README.md @@ -25,7 +25,7 @@ The Dataspace Protocol is under development and the working group is active on t The remainder of the document is structured as follows: -* [**Related documents:**](./related.documents.md) provides useful resources for understanding the overarching concepts of **Dataspaces** and other foundational documents. +* [**Related documents:**](./related.documents.md) provides useful resources for understanding the overarching concepts of [Dataspaces](../model/terminology.md#dataspace) and other foundational documents. * [**Extensions:**](./extensions.md) How to extend the Dataspace Protocol. * [**Examples:**](./examples.md) Living examples to foster the understanding of the Dataspace Protocol. * [**How to create Bindings**](./bindings.md) diff --git a/catalog/catalog.binding.https.md b/catalog/catalog.binding.https.md index fff4ce01..4ad6b0be 100644 --- a/catalog/catalog.binding.https.md +++ b/catalog/catalog.binding.https.md @@ -10,14 +10,14 @@ The OpenAPI definitions for this specification can be accessed [here](TBD). ### 2.1 Prerequisites -1. The `` notation indicates the base URL for a catalog service endpoint. For example, if the base catalog URL is `api.example.com`, the URL `https:///catalog/request` +1. The `` notation indicates the base URL for a [Catalog Service](../model/terminology.md#catalog-service) endpoint. For example, if the base [Catalog](../model/terminology.md#catalog) URL is `api.example.com`, the URL `https:///catalog/request` will map to `https//api.example.com/catalog/request`. 2. All request and response messages must use the `application/json` media type. ### 2.2 CatalogError -In the event of a request error, the catalog service must return an appropriate HTTP code and a [CatalogError](./catalog.protocol.md#) in the response body. +In the event of a request error, the [Catalog Service](../model/terminology.md#catalog-service) must return an appropriate HTTP code and a [CatalogError](./catalog.protocol.md#) in the response body. | Field | Type | Description | |---------|---------------|-------------------------------------------------------------| @@ -42,15 +42,15 @@ Authorization: ... } ``` -The `Authorization` header is optional if the catalog service does not require authorization. If present, the contents of the `Authorization` header are detailed in the +The `Authorization` header is optional if the [Catalog Service](../model/terminology.md#catalog-service) does not require authorization. If present, the contents of the `Authorization` header are detailed in the [Authorization section](#31-authorization). -The `filter` property is optional. If present, the `filter` property can contain an implementation-specific filter expression or query to be executed as part of the catalog +The `filter` property is optional. If present, the `filter` property can contain an implementation-specific filter expression or query to be executed as part of the [Catalog](../model/terminology.md#catalog) request. #### 2.3.2 OK (200) Response -If the request is successful, the catalog service must return a response body containing a [Catalog](./message/catalog.json) which is a profiled DCAT `Catalog` type +If the request is successful, the [Catalog Service](../model/terminology.md#catalog-service) must return a response body containing a [Catalog](./message/catalog.json) which is a profiled [DCAT `Catalog`](https://www.w3.org/TR/vocab-dcat-3/#Class:Catalog) type described by the [Catalog Protocol Specification](catalog.protocol.md). ### 2.4 The `catalog/datasets/{id}` endpoint @@ -66,19 +66,19 @@ Authorization: ... ``` -The `Authorization` header is optional if the catalog service does not require authorization. If present, the contents of the `Authorization` header are detailed in the +The `Authorization` header is optional if the [Catalog Service](../model/terminology.md#catalog-service) does not require authorization. If present, the contents of the `Authorization` header are detailed in the [Authorization section](#31-authorization). #### 2.4.2 OK (200) Response -If the request is successful, the catalog service must return a response body containing a [Dataset](./message/dataset.json) which is a DCAT `Dataset` type +If the request is successful, the [Catalog Service](../model/terminology.md#catalog-service) must return a response body containing a [Dataset](./message/dataset.json) which is a [DCAT `Dataset`](https://www.w3.org/TR/vocab-dcat-3/#Class:Dataset) type described by the [Catalog Protocol Specification](catalog.protocol.md). ## 3 Technical Considerations ### 3.1 Authorization -A catalog service may require authorization. If the catalog service requires authorization, requests must include an HTTP `Authorization` header with a token. The semantics of +A [Catalog Service](../model/terminology.md#catalog-service) may require authorization. If the [Catalog Service](../model/terminology.md#catalog-service) requires authorization, requests must include an HTTP `Authorization` header with a token. The semantics of such tokens are not part of this specification. ### 3.2 Versioning @@ -87,7 +87,7 @@ such tokens are not part of this specification. ### 3.3 Pagination -A catalog service may paginate the results of a `CatalogRequestMessage`. Pagination data is specified using [Web Linking](https://datatracker.ietf.org/doc/html/rfc5988) +A [Catalog Service](../model/terminology.md#catalog-service) may paginate the results of a `CatalogRequestMessage`. Pagination data is specified using [Web Linking](https://datatracker.ietf.org/doc/html/rfc5988) and the HTTP `Link` header. The `Link` header will contain URLs for navigating to previous and subsequent results. The following request sequence demonstrates pagination: ``` @@ -125,12 +125,12 @@ Link: ; rel="previous" ### 3.4 Compression -Catalog services MAY compress responses to a catalog request by setting the `Content-Encoding` header to `gzip` as described in +[Catalog Services](../model/terminology.md#catalog-service) MAY compress responses to a [Catalog](../model/terminology.md#catalog) request by setting the `Content-Encoding` header to `gzip` as described in the [HTTP 1.1 Specification](https://www.rfc-editor.org/rfc/rfc9110.html#name-gzip-coding). ## 4 The Well-Known Proof Metadata Endpoint -When an implementation supports protected _datasets_, it may offer a proof metadata endpoint clients can use to determine proof requirements. If the implementation +When an implementation supports protected [Datasets](../model/terminology.md#dataset), it may offer a proof metadata endpoint clients can use to determine proof requirements. If the implementation offers a proof data endpoint, it must use the `dspace-trust` [Well-Known Uniform Resource Identifier](https://www.rfc-editor.org/rfc/rfc8615.html) at the top of the path hierarchy: @@ -138,5 +138,5 @@ path hierarchy: The contents of the response is a JSON object defined by individual trust specifications and not defined here. -Note that if multiple connectors are hosted under the same base URL, a path segment appended to the base well-known URL can be used, for example, +Note that if multiple [Connectors](../model/terminology.md#connector--data-service-) are hosted under the same base URL, a path segment appended to the base well-known URL can be used, for example, `https://example.com/.well-known/dspace-trust/connector1.` diff --git a/catalog/catalog.protocol.md b/catalog/catalog.protocol.md index 28265be8..031552d3 100644 --- a/catalog/catalog.protocol.md +++ b/catalog/catalog.protocol.md @@ -2,16 +2,16 @@ ## 1 Introduction: Terms -This document outlines the catalog protocol. The following terms are used: +This document outlines the Catalog Protocol. The following terms are used: - A _**message type**_ defines the structure of a _message_. - A _**message**_ is an instantiation of a _message type_. -- A _**catalog**_ is a [DCAT catalog](https://www.w3.org/TR/vocab-dcat-3/) offered by a _provider_. -- a _**catalog service**_ is a provider participant agent that advertises offered datasets. -- A _**consumer**_ is a participant agent that requests access to an offered datasets. +- A _**catalog**_ is a [DCAT Catalog](https://www.w3.org/TR/vocab-dcat-3/#Class:Catalog) offered by a _provider_. +- a _**catalog service**_ is a provider [Participant Agent](../model/terminology.md#participant-agent) that advertises offered [Datasets](../model/terminology.md#dataset). +- A _**consumer**_ is a [Participant Agent](../model/terminology.md#participant-agent) that requests access to an offered [Datasets](../model/terminology.md#dataset). -The catalog protocol defines a how a `Catalog` is requested from a catalog service by a consumer using an abstract message exchange format. The concrete message exchange wire -format is defined in binding specifications. +The Catalog Protocol defines a how a [Catalog](../model/terminology.md#catalog) is requested from a [Catalog Service](../model/terminology.md#catalog-service) by a consumer using an abstract message exchange format. The concrete message exchange wire +format is defined in Catalog Binding Specifications. ## 2 Message Types @@ -26,19 +26,19 @@ Future IDS specifications may define additional optional serialization formats. **Example**: [CatalogRequestMessage](./message/catalog-request-message.json) -**Response**: [Catalog](#22-catalog) containing the [DCAT catalog](https://www.w3.org/TR/vocab-dcat-3/#Class:Catalog). +**Response**: [Catalog](#22-catalog) containing the [DCAT Catalog](https://www.w3.org/TR/vocab-dcat-3/#Class:Catalog). **Schema**: [CatalogRequestMessageShape](./message/shape/catalog-request-message-shape.ttl) and the [CatalogRequestMessage JSON Schema](./message/schema/catalog-request-message-schema.json) #### Description -The `CatalogRequestMessage` is message sent by a consumer to a catalog service. The catalog service must respond with a `Catalog,` which is a +The `CatalogRequestMessage` is message sent by a consumer to a [Catalog Service](../model/terminology.md#catalog-service). The [Catalog Service](../model/terminology.md#catalog-service) must respond with a [Catalog](../model/terminology.md#catalog), which is a valid instance of a [DCAT Catalog](https://www.w3.org/TR/vocab-dcat-3/#Class:Catalog). -The `CatalogRequestMessage` may have a `filter` property which contains an implementation-specific query or filter expression type supported by the catalog service. +The `CatalogRequestMessage` may have a `filter` property which contains an implementation-specific query or filter expression type supported by the [Catalog Service](../model/terminology.md#catalog-service). -The catalog service may require an authorization token. Details for including that token can be found in the relevant catalog binding specification. Similarly, pagination may -be defined in the relevant catalog binding specification. +The [Catalog Service](../model/terminology.md#catalog-service) may require an authorization token. Details for including that token can be found in the relevant Catalog Binding Specification. Similarly, pagination may +be defined in the relevant Catalog Binding Specification. ### 2.2 Catalog @@ -56,7 +56,7 @@ be defined in the relevant catalog binding specification. #### Description -The catalog contains all [Datasets](#31-dataset) which the requester shall see. +The [Catalog](../model/terminology.md#catalog) contains all [Datasets](#31-dataset) which the requester shall see. ### 2.3 CatalogError @@ -73,7 +73,7 @@ The catalog contains all [Datasets](#31-dataset) which the requester shall see. #### Description -A Catalog Error Message is used when an error occurred after a `CatalogRequestMessage` and the provider can not provide its catalog to the requester. +A Catalog Error Message is used when an error occurred after a `CatalogRequestMessage` and the provider can not provide its [Catalog](../model/terminology.md#catalog) to the requester. ### 2.4 DatasetRequestMessage @@ -81,18 +81,18 @@ A Catalog Error Message is used when an error occurred after a `CatalogRequestMe **Example**: [DatasetRequestMessage](./message/dataset-request-message.json) -**Response**: [Dataset](#22-catalog) containing the [DCAT Dataset](https://www.w3.org/TR/vocab-dcat-3/#Class:Dataset). +**Response**: [Dataset](#25-dataset) containing the [DCAT Dataset](https://www.w3.org/TR/vocab-dcat-3/#Class:Dataset). **Schema**: [DatasetRequestMessageShape](./message/shape/dataset-request-message-shape.ttl) and the [DatasetRequestMessage JSON Schema](./message/schema/dataset-request-message-schema.json) #### Description -The `DatasetRequestMessage` is message sent by a consumer to a catalog service. The catalog service must respond with a `Dataset,` which is a +The `DatasetRequestMessage` is message sent by a consumer to a [Catalog Service](../model/terminology.md#catalog-service). The [Catalog Service](../model/terminology.md#catalog-service) must respond with a `Dataset,` which is a valid instance of a [DCAT Dataset](https://www.w3.org/TR/vocab-dcat-3/#Class:Dataset). -The `DatasetRequestMessage` must have a `dataset` property which contains the id of the dataset. +The `DatasetRequestMessage` must have a [Dataset](../model/terminology.md#dataset) property which contains the id of the [Dataset](../model/terminology.md#dataset). -The catalog service may require an authorization token. Details for including that token can be found in the relevant catalog binding specification. +The [Catalog Service](../model/terminology.md#catalog-service) may require an authorization token. Details for including that token can be found in the relevant Catalog Binding Specification. ### 2.5 Dataset @@ -110,33 +110,33 @@ The catalog service may require an authorization token. Details for including th ## 3 DCAT Vocabulary Mapping -This section describes how the IDS Information Model maps to DCAT resources. +This section describes how the DSP Information Model maps to DCAT resources. ### 3.1 Dataset -A `Dataset` is a [DCAT Dataset](https://www.w3.org/TR/vocab-dcat-3/#Class:Dataset) with the following attributes: +A [Dataset](../model/terminology.md#dataset) is a [DCAT Dataset](https://www.w3.org/TR/vocab-dcat-3/#Class:Dataset) with the following attributes: #### 3.1.1 odrl:hasPolicy -A `Dataset` must have 1..N `hasPolicy` attributes that contain an ODRL `Offer` defining the usage control policy associated with the dataset. Offers must NOT contain any -target attributes. The target of an offer is the associated dataset. +A [Dataset](../model/terminology.md#dataset) must have 1..N `hasPolicy` attributes that contain an [ODRL `Offer`](https://www.w3.org/TR/odrl-vocab/#term-Offer) defining the [Usage Policy](../model/terminology.md#policy) associated with the [Dataset](../model/terminology.md#dataset). Offers must NOT contain any +target attributes. The target of an [Offer](../model/terminology.md#offer) is the associated [Dataset](../model/terminology.md#dataset). -> Note: As `odrl:hasPolicy rdfs:domain odrl:Asset`, each `Dataset` is also an `odrl:Asset` from an ODRL perspective. +> Note: As `odrl:hasPolicy rdfs:domain odrl:Asset`, each [Dataset](../model/terminology.md#dataset) is also an `odrl:Asset` from an ODRL perspective. ### 3.2 Distributions -A dataset may contain 0..N [DCAT Distributions](https://www.w3.org/TR/vocab-dcat-3/#Class:Distribution). Each distribution must have at least one `DataService` which specifies where -the distribution is obtained. Specifically, a `DataService` specifies the endpoint for initiating a `ContractNegotiation` and `TransferProcess`. +A [Dataset](../model/terminology.md#dataset) may contain 0..N [DCAT Distributions](https://www.w3.org/TR/vocab-dcat-3/#Class:Distribution). Each distribution must have at least one `DataService` which specifies where +the distribution is obtained. Specifically, a `DataService` specifies the endpoint for initiating a [Contract Negotiation](../model/terminology.md#contract-negotiation) and [Transfer Process](../model/terminology.md#transfer-process). -A Distribution may have 0..N `hasPolicy` attributes that contain an ODRL `Offer` defining the usage control policy associated with the dataset and this explicit `Distribution`. -Offers must NOT contain any target attributes. The target of an offer is the dataset that contains the distribution. +A Distribution may have 0..N `hasPolicy` attributes that contain an [ODRL `Offer`](https://www.w3.org/TR/odrl-vocab/#term-Offer) defining the [Usage Policy](../model/terminology.md#policy) associated with the [Dataset](../model/terminology.md#dataset) and this explicit `Distribution`. +[Offers](../model/terminology.md#offer) must NOT contain any target attributes. The target of an [Offer](../model/terminology.md#offer) is the [Dataset](../model/terminology.md#dataset) that contains the distribution. Support for `hasPolicy` attributes on a `Distribution` is optional. Implementations may choose not to support this feature, in which case they should return an appropriate error message to clients. ### 3.3 DataService -A DataService may specify an IDS service endpoint such as a `Connector`. +A DataService may specify an IDS service endpoint such as a [Connector](../model/terminology.md#connector--data-service-). #### 3.3.1 dspace:dataServiceType @@ -153,51 +153,51 @@ The following table lists well-know IDS endpoint types: | Value | Description | |---------------|----------------------| -| dspace:connector | A Connector endpoint | +| dspace:connector | A [Connector](../model/terminology.md#connector--data-service-) endpoint | | | | #### 3.3.2 dcat:servesDataset -Note that the property `dcat:servesDataset` should be omitted from the `DataService` since `DataSets` are included as top-level entries. Clients are not required to process the +Note that the property `dcat:servesDataset` should be omitted from the `DataService` since [Datasets](../model/terminology.md#dataset) are included as top-level entries. Clients are not required to process the contents of `dcat:servesDataset`. ## 4 Technical Considerations ### 4.1 Queries and Filter Expressions -A _**catalog service**_ may support catalog queries or filter expressions as an implementation-specific feature. However, it is expected that query capabilities will be implemented +A [Catalog Service](../model/terminology.md#catalog-service) may support [Catalog](../model/terminology.md#catalog) queries or filter expressions as an implementation-specific feature. However, it is expected that query capabilities will be implemented by the consumer against the results of a `CatalogRequestMessage,` as the latter is an RDF vocabulary. Client-side querying can be scaled by periodically crawling provider catalog -services, caching the results, and executing queries against the locally-stored catalogs. +services, caching the results, and executing queries against the locally-stored [Catalogs](../model/terminology.md#catalog). ### 4.2 Replication Protocol -The catalog protocol is designed to be used by federated services without the need for a replication protocol. Each consumer is responsible for issuing requests -to 1..N catalog services, and managing the results. It follows that a specific replication protocol is not needed, or more precisely, each consumer replicates data from catalog +The Catalog Protocol is designed to be used by federated services without the need for a replication protocol. Each consumer is responsible for issuing requests +to 1..N [Catalog Services](../model/terminology.md#catalog-service), and managing the results. It follows that a specific replication protocol is not needed, or more precisely, each consumer replicates data from catalog services by issuing `CatalogRequestMessages`. -The discovery protocol adopted by a particular dataspace defines how a consumer discovers catalog services. +The discovery protocol adopted by a particular [Dataspace](../model/terminology.md#dataspace) defines how a consumer discovers [Catalog Services](../model/terminology.md#catalog-service). ### 4.3 Security -It is expected (although not required) that catalog services implement access control. A catalog as well as individual catalog _datasets_ may be restricted to trusted parties. -The catalog service may require consumers to include a security token along with a `CatalogRequestMessage.` The specifics of how this is done can be found in the relevant -catalog binding specification. The semantics of such tokens are not part of this specification. +It is expected (although not required) that [Catalog Services](../model/terminology.md#catalog-service) implement access control. A [Catalog](../model/terminology.md#catalog) as well as individual [Datasets](../model/terminology.md#dataset) may be restricted to trusted parties. +The [Catalog Service](../model/terminology.md#catalog-service) may require consumers to include a security token along with a `CatalogRequestMessage.` The specifics of how this is done can be found in the relevant +Catalog Binding Specification. The semantics of such tokens are not part of this specification. #### 4.3.1 The Proof Metadata Endpoint -When a catalog contains protected _datasets_ the provider has two options: include all _datasets_ in the catalog response and restrict access when a contract is negotiated; -or, require one or more proofs when the catalog request is made and filter the _datasets_ accordingly. The latter option requires a mechanism for clients to discover -the type of proofs that may be presented at request time. The specifics of proof types and presenting a proof during a catalog request is outside the scope of the -Dataspace Protocol Specifications. However, binding specifications should define a proof data endpoint for obtaining this information. +When a [Catalog](../model/terminology.md#catalog) contains protected [Datasets](../model/terminology.md#dataset) the provider has two options: include all [Datasets](../model/terminology.md#dataset) in the [Catalog](../model/terminology.md#catalog) response and restrict access when a contract is negotiated; +or, require one or more proofs when the [Catalog](../model/terminology.md#catalog) request is made and filter the [Datasets](../model/terminology.md#dataset) accordingly. The latter option requires a mechanism for clients to discover +the type of proofs that may be presented at request time. The specifics of proof types and presenting a proof during a [Catalog](../model/terminology.md#catalog) request is outside the scope of the +Dataspace Protocol Specifications. However, Catalog Binding Specifications should define a proof data endpoint for obtaining this information. ### 4.4 Catalog Brokers -A dataspace may include _**catalog brokers**_. A catalog broker is a consumer that has trusted access to 1..N upstream catalog services and advertises their respective catalogs as -a single catalog service. The broker is expected to honor upstream access control requirements. +A [Dataspace](../model/terminology.md#dataspace) may include _**catalog brokers**_. A catalog broker is a consumer that has trusted access to 1..N upstream [Catalog Services](../model/terminology.md#catalog-service) and advertises their respective [Catalogs](../model/terminology.md#catalog) as +a single [Catalog Service](../model/terminology.md#catalog-service). The broker is expected to honor upstream access control requirements. ## 5 DCAT and ODRL Profiles -The catalog is a DCAT catalog with the following restrictions: +The [Catalog](../model/terminology.md#catalog) is a [DCAT Catalog](https://www.w3.org/TR/vocab-dcat-3/#Class:Catalog) with the following restrictions: -1. Each ODRL `Offer` must be unique to a dataset since the target of the offer is derived from its enclosing context. -2. Each ODRL `Offer` must NOT include an explicit `target` attribute. +1. Each [ODRL `Offer`](https://www.w3.org/TR/odrl-vocab/#term-Offer) must be unique to a [Dataset](../model/terminology.md#dataset) since the target of the [Offer](../model/terminology.md#offer) is derived from its enclosing context. +2. Each [ODRL `Offer`](https://www.w3.org/TR/odrl-vocab/#term-Offer) must NOT include an explicit `target` attribute. diff --git a/catalog/message/example/dcat.distribution.example.option1.json b/catalog/message/example/dcat.distribution.example.option1.json index f1f8e8f3..749ee86d 100644 --- a/catalog/message/example/dcat.distribution.example.option1.json +++ b/catalog/message/example/dcat.distribution.example.option1.json @@ -14,7 +14,7 @@ } ], "dcat:keyword": [ - "data space", + "dataspace", "dcat dataset", "collection" ], diff --git a/catalog/message/example/dcat.distribution.example.option2.json b/catalog/message/example/dcat.distribution.example.option2.json index 567a19c0..a5400689 100644 --- a/catalog/message/example/dcat.distribution.example.option2.json +++ b/catalog/message/example/dcat.distribution.example.option2.json @@ -14,7 +14,7 @@ } ], "dcat:keyword": [ - "data space", + "dataspace", "dcat dataset", "collection" ], diff --git a/common/common.binding.https.md b/common/common.binding.https.md index a99c3bd3..373bb534 100644 --- a/common/common.binding.https.md +++ b/common/common.binding.https.md @@ -9,5 +9,5 @@ path hierarchy: The contents of the response is a JSON object defined in the [Dataspace Protocol](./common.protocol.md#exposure-of-dataspace-protocol-versions). -Note that if multiple connectors are hosted under the same base URL, a path segment appended to the base well-known URL can be used, for example, +Note that if multiple [Connectors](../model/terminology.md#connector--data-service-) are hosted under the same base URL, a path segment appended to the base well-known URL can be used, for example, `https://example.com/.well-known/dspace-version/connector1.` \ No newline at end of file diff --git a/common/common.protocol.md b/common/common.protocol.md index ba8ecf9c..6f204f37 100644 --- a/common/common.protocol.md +++ b/common/common.protocol.md @@ -2,9 +2,9 @@ ## Exposure of Dataspace Protocol Versions -Connectors implementing the Dataspace Protocol may operate on different versions. Therefore, it is necessary that they can discover the supported versions of each other reliably and unambiguously. Each Connector must expose information of at least one Dataspace Protocol Version it supports. The specifics of how this information is obtained its defined by specific protocol bindings. +[Connectors](../model/terminology.md#connector--data-service-) implementing the Dataspace Protocol may operate on different versions. Therefore, it is necessary that they can discover the supported versions of each other reliably and unambiguously. Each [Connector](../model/terminology.md#connector--data-service-) must expose information of at least one Dataspace Protocol Version it supports. The specifics of how this information is obtained its defined by specific protocol bindings. -A Connector must respond to a respective request by providing a JSON-LD object containing an array of supported versions with at least one item. The item connects the version tag (`version` attribute) with the absolute URL path segment of the root path for all endpoints of this version. The following example specifies that this Connector offers version `1.0` endpoints at `/some/path/v1`. +A [Connector](../model/terminology.md#connector--data-service-) must respond to a respective request by providing a JSON-LD object containing an array of supported versions with at least one item. The item connects the version tag (`version` attribute) with the absolute URL path segment of the root path for all endpoints of this version. The following example specifies that this [Connector](../model/terminology.md#connector--data-service-) offers version `1.0` endpoints at `/some/path/v1`. ``` { @@ -20,4 +20,4 @@ A Connector must respond to a respective request by providing a JSON-LD object c This data object must comply to the [JSON Schema](schema/version-schema.json) and the [SHACL Shape](shape/version-shape.ttl). -The requesting Connector may select from the endpoints in the response. If the Connector can't identify a matching Dataspace Protocol Version, it must terminate the communication. \ No newline at end of file +The requesting [Connector](../model/terminology.md#connector--data-service-) may select from the endpoints in the response. If the [Connector](../model/terminology.md#connector--data-service-) can't identify a matching Dataspace Protocol Version, it must terminate the communication. \ No newline at end of file diff --git a/model/model.md b/model/model.md index 2e18d693..5ad9430e 100644 --- a/model/model.md +++ b/model/model.md @@ -6,72 +6,72 @@ The following sections outline the Dataspace Information Model, which form the f ### 2.1 Dataspace Entity Relationships -The relationships between the primary dataspace entities are defined as follows: +The relationships between the primary [Dataspace](./terminology.md#dataspace) entities are defined as follows: ![](./m.dataspace.relationships.png) Note that all relationships are multiplicities unless specified. -- A `Dataspace Authority` manages one or more `Dataspaces`. This will include participant registration and may entail mandating business and/or requirements. For example, a - Dataspace Authority may require participants to obtain some form of business certification. A Dataspace authority may also impose technical requirements such as support for the +- A [Dataspace Authority](./terminology.md#dataspace-authority) manages one or more [Dataspaces](./terminology.md#dataspace). This will include [Participant](./terminology.md#participant) registration and may entail mandating business and/or requirements. For example, a + [Dataspace Authority](./terminology.md#dataspace-authority) may require [Participants](./terminology.md#participant) to obtain some form of business certification. A [Dataspace Authority](./terminology.md#dataspace-authority) may also impose technical requirements such as support for the technical enforcement of specific usage policies. -- A `Participant` is a member of one or more `Dataspaces`. A participant registers `Participant Agents` that perform tasks on its behalf. -- A `Participant Agent` performs tasks such as publishing a catalog or engaging in a dataset transfer. In order to accomplish these tasks, a participant agent may - use a _**verifiable presentation**_ generated from a _**credential**_ obtained from a third-party issuer. A participant agent may also use an _**ID token**_ issued by a - third-party identity provider. Note that a participant agent is a logical construct and does not necessarily correspond to a single runtime process. -- An `Identity Provider` is a trust anchor that generates `ID tokens` used to verify the identity of a `Participant Agent`. Multiple identity providers may operate in - a dataspace. The types and semantics of ID tokens are not part of this specification. An identity provider may be a third-party or a participant itself (for example, in the case +- A [Participant](./terminology.md#participant) is a member of one or more [Dataspaces](./terminology.md#dataspace). A [Participant](./terminology.md#participant) registers [Participant Agents](./terminology.md#participant-agent) that perform tasks on its behalf. +- A [Participant Agent](./terminology.md#participant-agent) performs tasks such as publishing a [Catalog](./terminology.md#catalog) or engaging in a [Transfer Process](./terminology.md#transfer-process). In order to accomplish these tasks, a [Participant Agent](./terminology.md#participant-agent) may + use a _**verifiable presentation**_ generated from a _**credential**_ obtained from a third-party [Credential Issuer](./terminology.md#credential-issuer). A [Participant Agent](./terminology.md#participant-agent) may also use an _**ID token**_ issued by a + third-party [Identity Provider](./terminology.md#identity-provider). Note that a [Participant Agent](./terminology.md#participant-agent) is a logical construct and does not necessarily correspond to a single runtime process. +- An [Identity Provider](./terminology.md#identity-provider) is a trust anchor that generates `ID tokens` used to verify the identity of a [Participant Agent](./terminology.md#participant-agent). Multiple identity providers may operate in + a [Dataspace](./terminology.md#dataspace). The types and semantics of ID tokens are not part of this specification. An [Identity Provider](./terminology.md#identity-provider) may be a third-party or a [Participant](./terminology.md#participant) itself (for example, in the case of decentralized identifiers). -- A `Credential Issuer` issues _verifiable credentials_ used by participant agents to allow access to datasets and verify usage control. +- A [Credential Issuer](./terminology.md#credential-issuer) issues _verifiable credentials_ used by [Participant Agents](./terminology.md#participant-agent) to allow access to [Datasets](./terminology.md#dataset) and verify usage control. -The diagram below depicts the relationships between `ParticipantAgent` types: +The diagram below depicts the relationships between [Participant Agent](./terminology.md#participant-agent) types: ![](./m.participant.entities.png) -- A `CatalogService` is a `Participant Agent` that makes a [DCAT Catalog](https://www.w3.org/TR/vocab-dcat-3/#Class:Catalog) available to other participants. -- A `Catalog` contains one or more `Datasets`, which are [DCAT Datasets](https://www.w3.org/TR/vocab-dcat-3/#Class:Dataset). A `Catalog` also contains **_at least one_** - [DCAT DataService](https://www.w3.org/TR/vocab-dcat-3/#Class:Data_Service) that references a `Connector` where datasets may be obtained. -- A `Dataset` has **_at least one_** `Offer`, which is an [ODRL Offer](https://www.w3.org/TR/odrl-model/#policy-offer) describing the _usage control policy_ associated with the dataset. -- A `Connector` is a `Participant Agent` that performs `Contract Negotiation` and `Transfer Process` operations with another connector. An outcome of a `ContractNegotiation` may - be the production of an `Agreement`, which is an [ODRL Agreement](https://www.w3.org/TR/odrl-model/#policy-agreement) defining the _usage control policy_ agreed to for a dataset. +- A [Catalog Service](./terminology.md#catalog-service) is a [Participant Agent](./terminology.md#participant-agent) that makes a [DCAT Catalog](https://www.w3.org/TR/vocab-dcat-3/#Class:Catalog) available to other [Participants](./terminology.md#participant). +- A [Catalog](./terminology.md#catalog) contains one or more [Datasets](./terminology.md#dataset), which are [DCAT Datasets](https://www.w3.org/TR/vocab-dcat-3/#Class:Dataset). A [Catalog](./terminology.md#catalog) also contains **_at least one_** + [DCAT DataService](https://www.w3.org/TR/vocab-dcat-3/#Class:Data_Service) that references a [Connector](./terminology.md#connector--data-service-) where [Datasets](./terminology.md#dataset) may be obtained. +- A [Dataset](./terminology.md#dataset) has **_at least one_** [Offer](./terminology.md#offer), which is an [ODRL Offer](https://www.w3.org/TR/odrl-model/#policy-offer) describing the [Usage Policy](./terminology.md#policy) associated with the [Dataset](./terminology.md#dataset). +- A [Connector](./terminology.md#connector--data-service-) is a [Participant Agent](./terminology.md#participant-agent) that performs [Contract Negotiation](./terminology.md#contract-negotiation) and [Transfer Process](./terminology.md#transfer-process) operations with another [Connector](./terminology.md#connector--data-service-). An outcome of a [Contract Negotiation](./terminology.md#contract-negotiation) may + be the production of an [Agreement](./terminology.md#agreement), which is an [ODRL Agreement](https://www.w3.org/TR/odrl-model/#policy-agreement) defining the [Usage Policy](./terminology.md#policy) agreed to for a [Dataset](./terminology.md#dataset). ## 2.2 Classes -Not all dataspace entities have a concrete _technical_ materialization; some entities may exist as purely logical constructs. For example, a `Dataspace Authority` -and `Participant Agent` have no representation in the protocol message flows that constitute dataspace interactions. This section outlines the classes that comprise the concrete +Not all [Dataspace](./terminology.md#dataspace) entities have a concrete _technical_ materialization; some entities may exist as purely logical constructs. For example, a [Dataspace Authority](./terminology.md#dataspace-authority) +and [Participant Agent](./terminology.md#participant-agent) have no representation in the protocol message flows that constitute [Dataspace](./terminology.md#dataspace) interactions. This section outlines the classes that comprise the concrete elements of the model, i.e. those that are represented in protocol message flows. **_Note 1:_** -The classes and definitions used in the dataspace protocol are reused from different standards and specifications as much as possible, in particular, DCAT and ODRL. As, however, the external definitions allow different interpretations or provide more attributes than required, the dataspace protocol is leveraging _profiles_ of the original definitions rather than the complete original expressiveness. A _profile_ in this sense is a restriction or subset of an external definition, enforcing that every occurance of an externally defined class is always conformant with the original definition. However, not every standard-compliant class might be compliant to the dataspace profile. +The classes and definitions used in the Dataspace Protocol are reused from different standards and specifications as much as possible, in particular, [DCAT](https://www.w3.org/TR/vocab-dcat-3) and [ODRL](https://www.w3.org/TR/odrl/). As, however, the external definitions allow different interpretations or provide more attributes than required, the dataspace protocol is leveraging _profiles_ of the original definitions rather than the complete original expressiveness. A _profile_ in this sense is a restriction or subset of an external definition, enforcing that every occurance of an externally defined class is always conformant with the original definition. However, not every standard-compliant class might be compliant to the dataspace profile. ### 2.2.1 Catalog -A `Catalog` is a [DCAT Catalog](https://www.w3.org/TR/vocab-dcat-3/#Class:Catalog) with the following attributes: +A [Catalog](./terminology.md#catalog) is a [DCAT Catalog](https://www.w3.org/TR/vocab-dcat-3/#Class:Catalog) with the following attributes: -- 0..N `Datasets`. Since a catalog may be dynamically generated for a request based on the requesting participant's credentials it is possible for it to contain 0 matching - datasets. (DCAT PROFILE) -- 1..N [DCAT DataService](https://www.w3.org/TR/vocab-dcat-3/#Class:Data_Service) that references a `Connector` where datasets may be obtained. (DCAT PROFILE) +- 0..N [Datasets](./terminology.md#dataset). Since a [Catalog](./terminology.md#catalog) may be dynamically generated for a request based on the requesting [Participant's](./terminology.md#participant) credentials it is possible for it to contain 0 matching + [Datasets](./terminology.md#dataset). (DCAT PROFILE) +- 1..N [DCAT DataService](https://www.w3.org/TR/vocab-dcat-3/#Class:Data_Service) that references a [Connector](./terminology.md#connector--data-service-) where [Datasets](./terminology.md#dataset) may be obtained. (DCAT PROFILE) ### 2.2.2 Dataset -A `Dataset` is a [DCAT Dataset](https://www.w3.org/TR/vocab-dcat-3/#Class:Dataset) with the following attributes: +A [Dataset](./terminology.md#dataset) is a [DCAT Dataset](https://www.w3.org/TR/vocab-dcat-3/#Class:Dataset) with the following attributes: -- 1..N `hasPolicy` attributes that contain an ODRL `Offer` defining the usage control policy associated with the dataset. **_Offers must NOT contain any target attributes. The - target of an offer is the associated dataset._** (ODRL PROFILE) +- 1..N `hasPolicy` attributes that contain an [ODRL `Offer`](https://www.w3.org/TR/odrl-vocab/#term-Offer) defining the [Usage Policy](./terminology.md#policy) associated with the [Dataset](./terminology.md#dataset). **_Offers must NOT contain any target attributes. The + target of an [Offer](./terminology.md#offer) is the associated [Dataset](./terminology.md#dataset)._** (ODRL PROFILE) - 1..N [DCAT Distributions](https://www.w3.org/TR/vocab-dcat-3/#Class:Distribution). Each distribution must have at least one `DataService` which specifies where the distribution - is obtained. Specifically, a `DataService` specifies the endpoint for initiating a `ContractNegotiation` and `TransferProcess`. (DCAT PROFILE) + is obtained. Specifically, a `DataService` specifies the endpoint for initiating a [Contract Negotiation](./terminology.md#contract-negotiation) and [Transfer Process](./terminology.md#transfer-process). (DCAT PROFILE) ### 2.2.3 Offer -An `Offer` is an [ODRL Offer](https://www.w3.org/TR/odrl-model/#policy-offer) with the following attributes: +An [Offer](./terminology.md#offer) is an [ODRL Offer](https://www.w3.org/TR/odrl-model/#policy-offer) with the following attributes: - An ODRL `uid` is represented as an "@id" that is a unique UUID. (ODRL PROFILE) -- The `Offer` must be unique to a dataset since the target of the offer is derived from its enclosing context. -- The `Offer` must NOT include an explicit `target` attribute. +- The [Offer](./terminology.md#offer) must be unique to a [Dataset](./terminology.md#dataset) since the target of the [Offer](./terminology.md#offer) is derived from its enclosing context. +- The [Offer](./terminology.md#offer) must NOT include an explicit `target` attribute. ## 2.2.4 Agreement -An `Agreement` is an [ODRL Agreement](https://www.w3.org/TR/odrl-model/#policy-agreement) with the following attributes: +An [Agreement](./terminology.md#agreement) is an [ODRL Agreement](https://www.w3.org/TR/odrl-model/#policy-agreement) with the following attributes: -- The `Agreement` class must include one `target` attribute that is the UUID of the dataset the agreement is associated with. An agreement is therefore associated with **EXACTLY - ONE** dataset. (ODRL PROFILE) +- The [Agreement](./terminology.md#agreement) class must include one `target` attribute that is the UUID of the [Dataset](./terminology.md#dataset) the [Agreement](./terminology.md#agreement) is associated with. An [Agreement](./terminology.md#agreement) is therefore associated with **EXACTLY + ONE** [Dataset](./terminology.md#dataset). (ODRL PROFILE) diff --git a/model/terminology.md b/model/terminology.md index eb48e4d0..6e99fe21 100644 --- a/model/terminology.md +++ b/model/terminology.md @@ -1,72 +1,72 @@ # 1 Terminology -This and the following section defines the core concepts, entities, and relationships that underpin a `Dataspace`. +This and the following section defines the core concepts, entities, and relationships that underpin a __dataspace__. ## Dataspace -A `Dataspace` is a set of technical services that facilitate interoperable `Dataset` sharing between entities. +A `Dataspace` is a set of technical services that facilitate interoperable [Dataset](#dataset) sharing between entities. -## DataspaceAuthority +## Dataspace Authority -A `DataspaceAuthority` is an entity that manages a `Dataspace`. +A `Dataspace Authority` is an entity that manages a [Dataspace](#dataspace). ## Participant -A `Participant` is a `Dataspace` member that provides and/or consumes `Datasets`. +A `Participant` is a [Dataspace](#dataspace) member that provides and/or consumes [Datasets](#dataset). -## ParticipantAgent +## Participant Agent -A `ParticipantAgent` is a technology system that performs operations on behalf of a `Participant`. +A `Participant Agent` is a technology system that performs operations on behalf of a [Participant](#participant). -## IdentityProvider +## Identity Provider -An `IdentityProvider` is a trusted technology system that creates, maintains, and manages identity information for a `Participant` and `ParticipantAgents`. +An `Identity Provider` is a trusted technology system that creates, maintains, and manages identity information for a [Participant](#participant) and [Participant Agents](#participant-agent). -## CredentialIssuer +## Credential Issuer -A `CredentialIssuer` is a trusted technology system that issues verifiable credentials for a `Participant` and `ParticipantAgents`. +A `Credential Issuer` is a trusted technology system that issues verifiable credentials for a [Participant](#participant) and [Participant Agents](#participant-agent). ## Observability, Traceability and Audit Logging -`Observability, Traceability and Audit Logging` of transactions, e.g. `Contract Negotiation`, `Data Transfer` and enforcement of access policies or usage policies, in a `Dataspace` can be a requirement. +`Observability, Traceability and Audit Logging` of transactions, e.g. [Contract Negotiation](#contract-negotiation), [Transfer Process](#transfer-process) and enforcement of access policies or usage policies, in a [Dataspace](#dataspace) can be a requirement. If a trusted technology system is required that records and verifies those domain events. This is not in the scope of the current version of the document and is subject of future work. -## DataspaceRegistrationService +## Dataspace Registration Service -A `DataspaceRegistrationService` is a technology system that maintains the state of `Participants` in a `Dataspace`. +A `Dataspace Registration Service` is a technology system that maintains the state of [Participants](#participant) in a [Dataspace](#dataspace). ## Dataset -Data or a technical service that can be shared by a `Participant`. +Data or a technical service that can be shared by a [Participant](#participant). ## Policy -A set of rules, duties, and obligations that define the terms of use for a `Dataset`. +A set of rules, duties, and obligations that define the terms of use for a [Dataset](#dataset). Also referred to as `Usage Policy`. ## Offer -A concrete `Policy` associated with a specific `Dataset`. +A concrete [Policy](#policy) associated with a specific [Dataset](#dataset). ## Agreement -A concrete `Policy` associated with a specific `Dataset` that has been signed by both the provider and consumer `Participants`. +A concrete [Policy](#policy) associated with a specific [Dataset](#dataset) that has been signed by both the provider and consumer [Participants](#participant). ## Catalog -A collection of entries representing `Datasets` and their `Offers` that is advertised by a provider `Participant`. +A collection of entries representing [Datasets](#dataset) and their [Offers](#offer) that is advertised by a provider [Participant](#participant). -## CatalogService +## Catalog Service -A `ParticipantAgent` that makes a `Catalog` accessible to `Participants`. +A [Participant Agent](#participant-agent) that makes a [Catalog](#catalog) accessible to [Participants](#participant). -## Connector (DataService) +## Connector (Data Service) -A `ParticipantAgent` that produces `Agreements` and manages `Dataset` sharing. +A [Participant Agent](#participant-agent) that produces [Agreements](#agreement) and manages [Dataset](#dataset) sharing. ## Contract Negotiation -A set of interactions between a provider `Connector` and consumer `Connector` that establish an `Agreement`. +A set of interactions between a provider [Connector](#connector--data-service-) and consumer [Connector](#connector--data-service-) that establish an [Agreement](#agreement). -## Dataset Transfer +## Transfer Process -A set of interactions between a provider `Connector` and consumer `Connector` that give access to a `Dataset` under the terms of an `Agreement`. +A set of interactions between a provider [Connector](#connector--data-service-) and consumer [Connector](#connector--data-service-) that give access to a [Dataset](#dataset) under the terms of an `[Agreement](#agreement). diff --git a/negotiation/contract.negotiation.binding.https.md b/negotiation/contract.negotiation.binding.https.md index 87a1ba04..dab2425c 100644 --- a/negotiation/contract.negotiation.binding.https.md +++ b/negotiation/contract.negotiation.binding.https.md @@ -10,32 +10,32 @@ The OpenAPI definitions for this specification can be accessed [here](TBD). ### 2.1 Prerequisites -1. The `` notation indicates the base URL for a connector endpoint. For example, if the base connector URL is `connector.example.com`, the +1. The `` notation indicates the base URL for a [Connector](../model/terminology.md#connector--data-service-) endpoint. For example, if the base [Connector](../model/terminology.md#connector--data-service-) URL is `connector.example.com`, the URL `https:///negotiation/request` will map to `https//connector.example.com/negotiation/request`. 2. All request and response messages must use the `application/json` media type. ### 2.2 Contract Negotiation Error -In the event of a client request error, the connector must return an appropriate HTTP 4xx client error code. If an error body is returned it must be +In the event of a client request error, the [Connector](../model/terminology.md#connector--data-service-) must return an appropriate HTTP 4xx client error code. If an error body is returned it must be a [ContractNegotiationError](./message/contract-negotiation-error.json) with the following properties: -| Field | Type | Description | -|-------------|---------------|-------------------------------------------------------------| -| consumerPid | UUID | The contract negotiation unique id on consumer side. | -| providerPid | UUID | The contract negotiation unique id on provider side. | -| code | string | An optional implementation-specific error code. | -| reasons | Array[object] | An optional array of implementation-specific error objects. | +| Field | Type | Description | +|-------------|---------------|------------------------------------------------------------------------------------------------------| +| consumerPid | UUID | The [Contract Negotiation](../model/terminology.md#contract-negotiation) unique id on consumer side. | +| providerPid | UUID | The [Contract Negotiation](../model/terminology.md#contract-negotiation) unique id on provider side. | +| code | string | An optional implementation-specific error code. | +| reasons | Array[object] | An optional array of implementation-specific error objects. | ### 2.2.1 State transition errors -If a client or provider connector makes a request that results in an invalid contract negotiation state transition as defined by the Contract Negotiation Protocol, it must return +If a client or provider [Connector](../model/terminology.md#connector--data-service-) makes a request that results in an invalid [Contract Negotiation's](./contract.negotiation.protocol.md#ack---contractnegotiation) state transition as defined by the Contract Negotiation Protocol, it must return an HTTP code 400 (Bad Request) with an `ContractNegotiationError` in the response body. ### 2.3 Authorization All requests should use the `Authorization` header to include an authorization token. The semantics of such tokens are not part of this specification. The `Authorization` HTTP -header is optional if the connector does not require authorization. +header is optional if the [Connector](../model/terminology.md#connector--data-service-) does not require authorization. ### 2.4 The provider `negotiations` resource @@ -48,7 +48,7 @@ Authorization: ... ``` -If the negotiation is found and the client is authorized, the provider connector must return an HTTP 200 (OK) response and a body containing +If the [Contract Negotiation](../model/terminology.md#contract-negotiation) is found and the client is authorized, the provider [Connector](../model/terminology.md#connector--data-service-) must return an HTTP 200 (OK) response and a body containing the [ContractNegotiation](./message/contract-negotiation.json): ``` @@ -63,13 +63,13 @@ the [ContractNegotiation](./message/contract-negotiation.json): Predefined states are: `REQUESTED`, `OFFERED`, `ACCEPTED`, `AGREED`, `VERIFIED`, `FINALIZED`, and `TERMINATED`. -If the negotiation does not exist or the client is not authorized, the provider connector must return an HTTP 404 (Not Found) response. +If the [Contract Negotiation](../model/terminology.md#contract-negotiation) does not exist or the client is not authorized, the provider [Connector](../model/terminology.md#connector--data-service-) must return an HTTP 404 (Not Found) response. ### 2.5 The provider `negotiations/request` resource #### 2.5.1 POST -A contract negotiation is started and placed in the `REQUESTED` state when a consumer POSTs +A [Contract Negotiation](../model/terminology.md#contract-negotiation) is started and placed in the `REQUESTED` state when a consumer POSTs a [ContractRequestMessage](./message/contract-request-message_initial.json) to `negotiations/request`: ``` @@ -87,13 +87,13 @@ Authorization: ... } ``` -The `callbackAddress` property specifies the base endpoint `URL` where the client receives messages associated with the contract negotiation. Support for the `HTTPS` scheme is +The `callbackAddress` property specifies the base endpoint `URL` where the client receives messages associated with the [Contract Negotiation](../model/terminology.md#contract-negotiation). Support for the `HTTPS` scheme is required. Implementations may optionally support other URL schemes. -Callback messages will be sent to paths under the base URL as described by this specification. Note that provider connectors should properly handle the cases where a trailing `/` +Callback messages will be sent to paths under the base URL as described by this specification. Note that provider [Connectors](../model/terminology.md#connector--data-service-) should properly handle the cases where a trailing `/` is included with or absent from the `callbackAddress` when resolving full URL. -The provider connector must return an HTTP 201 (Created) response with a body containing +The provider [Connector](../model/terminology.md#connector--data-service-) must return an HTTP 201 (Created) response with a body containing the [ContractNegotiation](./message/contract-negotiation.json): ``` @@ -110,7 +110,7 @@ the [ContractNegotiation](./message/contract-negotiation.json): #### 2.6.1 POST -A consumer may make an offer by POSTing a [ContractRequestMessage](./message/contract-request-message.json) to `negotiations/:providerPid/request`: +A consumer may make an [Offer](../model/terminology.md#offer) by POSTing a [ContractRequestMessage](./message/contract-request-message.json) to `negotiations/:providerPid/request`: ``` POST https://connector.provider.com/negotiations/urn:uuid:dcbf434c-eacf-4582-9a02-f8dd50120fd3/offers @@ -132,24 +132,24 @@ Authorization: ... The consumer must include the `providerPid` and `consumerPid`. The consumer must include either the `offer` or `offerId` property. -If the message is successfully processed, the provider connector must return and HTTP 200 (OK) response. The response body is not specified and clients are not required to process +If the message is successfully processed, the provider [Connector](../model/terminology.md#connector--data-service-) must return and HTTP 200 (OK) response. The response body is not specified and clients are not required to process it. ### 2.7 The provider `negotiations/:providerPid/events` resource #### 2.7.1 POST -A consumer connector can POST a [ContractNegotiationEventMessage](./message/contract-negotiation-event-message.json) to `negotiations/:providerPid/events` to accept the current -provider contract offer. If the negotiation state is successfully transitioned, the provider must return HTTP code 200 (OK). The response body is not specified and clients are not +A consumer [Connector](../model/terminology.md#connector--data-service-) can POST a [ContractNegotiationEventMessage](./message/contract-negotiation-event-message.json) to `negotiations/:providerPid/events` to accept the current +provider [Offer](../model/terminology.md#offer). If the [Contract Negotiation's](./contract.negotiation.protocol.md#ack---contractnegotiation) state is successfully transitioned, the provider must return HTTP code 200 (OK). The response body is not specified and clients are not required to process it. -If the current contract offer was created by the consumer, the provider must return HTTP code 400 (Bad Request) with an `NegotiationErrorMessage` in the response body. +If the current [Offer](../model/terminology.md#offer) was created by the consumer, the provider must return HTTP code 400 (Bad Request) with an `NegotiationErrorMessage` in the response body. ### 2.8 The provider `negotiations/:providerPid/agreement/verification` resource #### 2.8.1 POST -The consumer connector can POST a [ContractAgreementVerificationMessage](./message/contract-agreement-verification-message.json) to verify an agreement. If the negotiation state is +The consumer [Connector](../model/terminology.md#connector--data-service-) can POST a [ContractAgreementVerificationMessage](./message/contract-agreement-verification-message.json) to verify an [Agreement](../model/terminology.md#agreement). If the [Contract Negotiation's](./contract.negotiation.protocol.md#ack---contractnegotiation) state is successfully transitioned, the provider must return HTTP code 200 (OK). The response body is not specified and clients are not required to process it. ``` @@ -170,14 +170,14 @@ Authorization: ... #### 2.9.1 POST -The consumer connector can POST a [ContractNegotiationTerminationMessage](./message/contract-negotiation-termination-message.json) to terminate a negotiation. If the negotiation +The consumer [Connector](../model/terminology.md#connector--data-service-) can POST a [ContractNegotiationTerminationMessage](./message/contract-negotiation-termination-message.json) to terminate a [Contract Negotiation](../model/terminology.md#contract-negotiation). If the [Contract Negotiation's](../model/terminology.md#contract-negotiation) state is successfully transitioned, the provider must return HTTP code 200 (OK). The response body is not specified and clients are not required to process it. ## 3 Consumer Callback Path Bindings ### 3.1 Prerequisites -All callback paths are relative to the `callbackAddress` base URL specified in the `ContractRequestMessage` that initiated a contract negotiation. For example, if +All callback paths are relative to the `callbackAddress` base URL specified in the `ContractRequestMessage` that initiated a [Contract Negotiation](../model/terminology.md#contract-negotiation). For example, if the `callbackAddress` is specified as `https://connector.consumer/callback` and a callback path binding is `negotiations/:consumerPid/offers`, the resolved URL will be `https://connector.consumer.com/callback/negotiations/:consumerPid/offers`. @@ -185,7 +185,7 @@ be `https://connector.consumer.com/callback/negotiations/:consumerPid/offers`. #### 3.2.1 POST -A contract offer is started and placed in the `OFFERED` state when a provider POSTs a +A [Contract Negotiation](../model/terminology.md#contract-negotiation) is started and placed in the `OFFERED` state when a provider POSTs a [ContractOfferMessage](./message/contract-offer-message_initial.json) to `negotiations/offers`: ``` @@ -207,11 +207,11 @@ Authorization: ... } ``` -The `callbackAddress` property specifies the base endpoint URL where the client receives messages associated with the contract negotiation. Support for the HTTPS scheme is required. Implementations may optionally support other URL schemes. +The `callbackAddress` property specifies the base endpoint URL where the client receives messages associated with the [Contract Negotiation](../model/terminology.md#contract-negotiation). Support for the HTTPS scheme is required. Implementations may optionally support other URL schemes. -Callback messages will be sent to paths under the base URL as described by this specification. Note that consumer connectors should properly handle the cases where a trailing / is included with or absent from the callbackAddress when resolving full URL. +Callback messages will be sent to paths under the base URL as described by this specification. Note that consumer [Connectors](../model/terminology.md#connector--data-service-) should properly handle the cases where a trailing / is included with or absent from the callbackAddress when resolving full URL. -The consumer connector must return an HTTP 201 (Created) response with a body containing the `ContractNegotiation`: +The consumer [Connector](../model/terminology.md#connector--data-service-) must return an HTTP 201 (Created) response with a body containing the [Contract Negotiation](./contract.negotiation.protocol.md#ack---contractnegotiation): ``` { @@ -227,7 +227,7 @@ The consumer connector must return an HTTP 201 (Created) response with a body co #### 3.3.1 POST -A provider may make an offer by POSTing a [ContractOfferMessage](./message/contract-offer-message.json) to the `negotiations/:consumerPid/offers` callback: +A provider may make an [Offer](../model/terminology.md#offer) by POSTing a [ContractOfferMessage](./message/contract-offer-message.json) to the `negotiations/:consumerPid/offers` callback: ``` POST https://connector.consumer.com/callback/negotiations/urn:uuid:32541fe6-c580-409e-85a8-8a9a32fbe833/offers @@ -247,15 +247,15 @@ Authorization: ... } ``` -If the message is successfully processed, the consumer provider connector must return an HTTP 200 (OK) response. The response body is not specified and clients are not required to +If the message is successfully processed, the consumer provider [Connector](../model/terminology.md#connector--data-service-) must return an HTTP 200 (OK) response. The response body is not specified and clients are not required to process it. ### 3.4 The consumer `negotiations/:consumerPid/agreement` resource #### 3.4.1 POST -The provider connector can POST a [ContractAgreementMessage](./message/contract-agreement-message.json) to the `negotiations/:consumerPid/agreement` callback to create an agreement. If the -negotiation state is successfully transitioned, the consumer must return HTTP code 200 (OK). The response body is not specified and clients are not required to process it. +The provider [Connector](../model/terminology.md#connector--data-service-) can POST a [ContractAgreementMessage](./message/contract-agreement-message.json) to the `negotiations/:consumerPid/agreement` callback to create an [Agreement](../model/terminology.md#agreement). If the +[Contract Negotiation's](./contract.negotiation.protocol.md#ack---contractnegotiation) state is successfully transitioned, the consumer must return HTTP code 200 (OK). The response body is not specified and clients are not required to process it. ``` POST https://connector.consumer.com/negotiations/urn:uuid:32541fe6-c580-409e-85a8-8a9a32fbe833/agreement @@ -282,12 +282,12 @@ Authorization: ... #### 3.5.1 POST A provider can POST a [ContractNegotiationEventMessage](./message/contract-negotiation-event-message.json) to the `negotiations/:consumerPid/events` callback with an `eventType` -of `FINALIZED` to finalize a contract agreement. If the negotiation state is successfully transitioned, the consumer must return HTTP code 200 (OK). The response body is not +of `FINALIZED` to finalize an [Agreement](../model/terminology.md#agreement). If the [Contract Negotiation's](./contract.negotiation.protocol.md#ack---contractnegotiation) state is successfully transitioned, the consumer must return HTTP code 200 (OK). The response body is not specified and clients are not required to process it. ### 3.6 The consumer `negotiations/:consumerPid/termination` resource #### 3.6.1 POST -The provider connector can POST a [ContractNegotiationTerminationMessage](./message/contract-negotiation-termination-message.json) to terminate a negotiation. If the negotiation +The provider [Connector](../model/terminology.md#connector--data-service-) can POST a [ContractNegotiationTerminationMessage](./message/contract-negotiation-termination-message.json) to terminate a [Contract Negotiation](../model/terminology.md#contract-negotiation). If the [Contract Negotiation's](./contract.negotiation.protocol.md#ack---contractnegotiation) state is successfully transitioned, the consumer must return HTTP code 200 (OK). The response body is not specified and clients are not required to process it. diff --git a/negotiation/contract.negotiation.protocol.md b/negotiation/contract.negotiation.protocol.md index 14fdad32..090c1b8f 100644 --- a/negotiation/contract.negotiation.protocol.md +++ b/negotiation/contract.negotiation.protocol.md @@ -2,32 +2,32 @@ ## Introduction: Terms -This document outlines the key elements of the contract negotiation protocol. The following terms are used: +This document outlines the key elements of the Contract Negotiation Protocol. The following terms are used: - A _**message type**_ defines the structure of a _message_. - A _**message**_ is an instantiation of a _message type_. - The _**contract negotiation protocol**_ is the set of allowable message type sequences and is defined as a state machine (CNP-SM). - A _**contract negotiation (CN)**_ is an instantiation of the CNP-SM. -- A _**provider**_ is a participant agent that offers a dataset. -- A _**consumer**_ is a participant agent that requests access to an offered dataset. +- A _**provider**_ is a [Participant Agent](../model/terminology.md#participant-agent) that offers a [Dataset](../model/terminology.md#dataset). +- A _**consumer**_ is a [Participant Agent](../model/terminology.md#participant-agent) that requests access to an offered [Dataset](../model/terminology.md#dataset). ## Contract Negotiation Protocol -A contract negotiation (CN) involves two parties, a _provider_ that offers one or more datasets under a usage contract and _consumer_ that requests datasets. +A [Contract Negotiation](../model/terminology.md#contract-negotiation) (CN) involves two parties, a _provider_ that offers one or more [Datasets](../model/terminology.md#dataset) under a usage contract and _consumer_ that requests [Datasets](../model/terminology.md#dataset). A CN is uniquely identified through an [IRI](https://www.w3.org/International/articles/idn-and-iri/). Each CN requires a newly generated IRI, which may not be used in a CN after a terminal state has been reached. A CN progresses through a series of states, which are tracked by the provider and consumer using messages. A CN transitions to a state in response to an acknowledged message from the counter-party. Both parties have the same state of the CN. In case the states differ, the CN is terminated and a new CN has to be initiated. The CN states are: -- **REQUESTED** - A contract for a dataset has been requested by the consumer based on an offer and the provider has sent an ACK response. -- **OFFERED** - The provider has sent a contract offer to the consumer and the consumer has sent an ACK response. -- **ACCEPTED** - The consumer has accepted the latest contract offer and the provider has sent an ACK response. -- **AGREED** - The provider has accepted the latest contract offer, sent an agreement to the consumer, and the consumer has sent an ACK response. -- **VERIFIED** - The consumer has sent an agreement verification to the provider and the provider has sent an ACK response. -- **FINALIZED** - The provider has sent a finalization message including his own agreement verification to the consumer and the consumer has sent an ACK response. Data is +- **REQUESTED** - A contract for a [Dataset](../model/terminology.md#dataset) has been requested by the consumer based on an [Offer](../model/terminology.md#offer) and the provider has sent an ACK response. +- **OFFERED** - The provider has sent an [Offer](../model/terminology.md#offer) to the consumer and the consumer has sent an ACK response. +- **ACCEPTED** - The consumer has accepted the latest [Offer](../model/terminology.md#offer) and the provider has sent an ACK response. +- **AGREED** - The provider has accepted the latest [Offer](../model/terminology.md#offer), sent an [Agreement](../model/terminology.md#agreement) to the consumer, and the consumer has sent an ACK response. +- **VERIFIED** - The consumer has sent an [Agreement](../model/terminology.md#agreement) verification to the provider and the provider has sent an ACK response. +- **FINALIZED** - The provider has sent a finalization message including his own [Agreement](../model/terminology.md#agreement) verification to the consumer and the consumer has sent an ACK response. Data is now available to the consumer. -- **TERMINATED** - The provider or consumer has placed the contract negotiation in a terminated state. A termination message has been sent by either of the participants and the +- **TERMINATED** - The provider or consumer has placed the [Contract Negotiation](../model/terminology.md#contract-negotiation) in a terminated state. A termination message has been sent by either of the [Participants](../model/terminology.md#participant) and the other has sent an ACK response. This is a terminal state. ### Contract Negotiation State Machine @@ -46,9 +46,9 @@ The CN state machine is transitioned upon receipt and acknowledgement of a messa ### Notes - Concrete wire formats are defined by the protocol binding, e.g. HTTPS. -- All policy types (Offer, Agreement) must contain an unique identifier in the form of a URI. GUIDs can also be used in the form of URNs, for instance following the +- All [Policy](../model/terminology.md#policy) types ([Offer](../model/terminology.md#offer), [Agreement](../model/terminology.md#agreement)) must contain an unique identifier in the form of a URI. GUIDs can also be used in the form of URNs, for instance following the pattern . -- An ODRL Agreement must have a target property containing the dataset id. +- An [ODRL Agreement](https://www.w3.org/TR/odrl-vocab/#term-Agreement) must have a target property containing the [Dataset](../model/terminology.md#dataset) id. ### 1. ContractRequestMessage @@ -66,17 +66,17 @@ The CN state machine is transitioned upon receipt and acknowledgement of a messa #### Description -The `ContractRequestMessage` is sent by a consumer to initiate a contract negotiation or to respond to a `ContractOfferMessage` sent by a provider. +The `ContractRequestMessage` is sent by a consumer to initiate a [Contract Negotiation](../model/terminology.md#contract-negotiation) or to respond to a `ContractOfferMessage` sent by a provider. #### Notes -- The consumer must include an `offer` property, which itself must have a `@id` property. If the message includes a `providerPid` property, the request will be associated with an existing contract - negotiation and a consumer offer will be created using either the `offer` or `offer.@id` properties. If the message does not include a `providerPid`, a new contract negotiation +- The consumer must include an `offer` property, which itself must have a `@id` property. If the message includes a `providerPid` property, the request will be associated with an existing [Contract Negotiation](../model/terminology.md#contract-negotiation) + and a consumer [Offer](../model/terminology.md#offer) will be created using either the `offer` or `offer.@id` properties. If the message does not include a `providerPid`, a new [Contract Negotiation](../model/terminology.md#contract-negotiation) will be created on provider side using either the `offer` or `offer.@id` properties and the provider selects an appropriate `providerPid`. -- An `offer.@id` will generally refer to an offer contained in a catalog. If the provider is not aware of the `offer.@id` value, it must respond with an error message. +- An `offer.@id` will generally refer to an [Offer](../model/terminology.md#offer) contained in a [Catalog](../model/terminology.md#catalog). If the provider is not aware of the `offer.@id` value, it must respond with an error message. -- The dataset id is not technically required but included to avoid an error where the offer is associated with a different data set. +- The [Dataset](../model/terminology.md#dataset) id is not technically required but included to avoid an error where the [Offer](../model/terminology.md#offer) is associated with a different data set. - `callbackAddress` is a URL indicating where messages to the consumer should be sent in asynchronous settings. If the address is not understood, the provider MUST return an UNRECOVERABLE error. @@ -98,16 +98,16 @@ The `ContractRequestMessage` is sent by a consumer to initiate a contract negoti #### Description -The `ContractOfferMessage` is sent by a provider to initiate a contract negotiation or to respond to a `ContractRequestMessage` sent by a consumer. +The `ContractOfferMessage` is sent by a provider to initiate a [Contract Negotiation](../model/terminology.md#contract-negotiation) or to respond to a `ContractRequestMessage` sent by a consumer. ### Notes -If the message includes a `consumerPid` property, the request will be associated with an existing contract negotiation. If the message does not include a `consumerPid`, a new contract negotiation +If the message includes a `consumerPid` property, the request will be associated with an existing [Contract Negotiation](../model/terminology.md#contract-negotiation). If the message does not include a `consumerPid`, a new [Contract Negotiation](../model/terminology.md#contract-negotiation) will be created on consumer side and the consumer selects an appropriate `consumerPid`. #### Notes -- The dataset id is not required but can be included when the provider initiates a contract negotiation. +- The [Dataset](../model/terminology.md#dataset) id is not required but can be included when the provider initiates a [Contract Negotiation](../model/terminology.md#contract-negotiation). ### 3. ContractAgreementMessage @@ -125,18 +125,18 @@ will be created on consumer side and the consumer selects an appropriate `consum #### Description -The `ContractAgreementMessage` is sent by a provider when it agrees to a contract. It contains the complete contract agreement. +The `ContractAgreementMessage` is sent by a provider when it agrees to a contract. It contains the complete [Agreement](../model/terminology.md#agreement). A `ContractAgreementMessage` must contain a `consumerPid` and a `providerPid`. -A `ContractAgreementMessage` must contain an ODRL `Agreement`. +A `ContractAgreementMessage` must contain an [ODRL `Agreement`](https://www.w3.org/TR/odrl-vocab/#term-Agreement). -An `Agreement` must contain a `dspace:timestamp` property defined as an XSD DateTime type. +An [Agreement](../model/terminology.md#agreement) must contain a `dspace:timestamp` property defined as an XSD DateTime type. -An `Agreement` must contain a `dspace:consumerId` and `dspace:providerId`. The contents of these -properties are a dataspace-specific unique identifier of the contract agreement parties. Note that these -identifiers are not necessarily the same as the identifiers of the participant agents negotiating the -contract (i.e. the "connectors"). +An [Agreement](../model/terminology.md#agreement) must contain a `dspace:consumerId` and `dspace:providerId`. The contents of these +properties are a [Dataspace](../model/terminology.md#dataspace)-specific unique identifier of the [Agreement](../model/terminology.md#agreement) parties. Note that these +identifiers are not necessarily the same as the identifiers of the [Participant Agents](../model/terminology.md#participant-agent) negotiating the +contract (i.e. the [Connectors](../model/terminology.md#connector--data-service-)). ### 4. ContractAgreementVerificationMessage @@ -155,7 +155,7 @@ contract (i.e. the "connectors"). #### Description -The `ContractAgreementVerificationMessage` is sent by a consumer to verify the acceptance of a contract agreement. A provider responds with an error if the contract cannot be +The `ContractAgreementVerificationMessage` is sent by a consumer to verify the acceptance of an [Agreement](../model/terminology.md#agreement). A provider responds with an error if the contract cannot be validated or is incorrect. A `ContractAgreementVerificationMessage` must contain a `consumerPid` and a `providerPid`. @@ -177,7 +177,7 @@ A `ContractAgreementVerificationMessage` must contain a `consumerPid` and a `pro #### Description -When the `ContractNegotiationEventMessage` is sent by a provider with an `eventType` property set to `FINALIZED`, a contract agreement has been finalized and the associated dataset +When the `ContractNegotiationEventMessage` is sent by a provider with an `eventType` property set to `FINALIZED`, an [Agreement](../model/terminology.md#agreement) has been finalized and the associated [Dataset](../model/terminology.md#dataset) is accessible. The state machine is transitioned to the `FINALIZED` state. Other event types may be defined in the future. A consumer responds with an error if the contract cannot be validated or is incorrect. @@ -187,8 +187,8 @@ When the `ContractNegotiationEventMessage` is sent by a consumer with an `eventT It is an error for a provider to send a `ContractNegotiationEventMessage` with an eventType `ACCEPTED` to the consumer. -Note that contract events are not intended for propagation of agreement state after a contract negotiation has entered a terminal state. It is considered an error for a consumer or -provider to send a contract negotiation event after the negotiation state machine has entered a terminal state. +Note that [Contract Negotiation](#ack---contractnegotiation) events are not intended for propagation of [Agreement](../model/terminology.md#agreement) state after a [Contract Negotiation](../model/terminology.md#contract-negotiation) has entered a terminal state. It is considered an error for a consumer or +provider to send an event after the [Contract Negotiation's](../model/terminology.md#contract-negotiation) state machine has entered a terminal state. A `ContractNegotiationEventMessage` must contain a `consumerPid` and a `providerPid`. @@ -208,15 +208,15 @@ A `ContractNegotiationEventMessage` must contain a `consumerPid` and a `provider #### Description -The `ContractNegotiationTerminationMessage` is sent by a consumer or provider indicating it has cancelled the negotiation sequence. The message can be sent at any state of a negotiation +The `ContractNegotiationTerminationMessage` is sent by a consumer or provider indicating it has cancelled the [Contract Negotiation](../model/terminology.md#contract-negotiation) sequence. The message can be sent at any state of a [Contract Negotiation](../model/terminology.md#contract-negotiation) without providing an explanation. Nevertheless, the sender may provide a description to help the receiver. A `ContractNegotiationTerminationMessage` must contain a `consumerPid` and a `providerPid`. #### Notes -- A contract negotiation may be terminated for a variety of reasons, for example, an unrecoverable error was encountered or one of the parties no longer wishes to continue. A - connector's operator may remove terminated contract negotiation resources after it has reached the terminated state. +- A [Contract Negotiation](../model/terminology.md#contract-negotiation) may be terminated for a variety of reasons, for example, an unrecoverable error was encountered or one of the parties no longer wishes to continue. A + [Connector's](../model/terminology.md#connector--data-service-) operator may remove terminated [Contract Negotiation](../model/terminology.md#contract-negotiation) resources after it has reached the terminated state. - If an error is received in response to a `ContractNegotiationTerminationMessage`, the sending party may choose to ignore the error. diff --git a/transfer/transfer.process.binding.https.md b/transfer/transfer.process.binding.https.md index 1bb96f08..a1d7f5e0 100644 --- a/transfer/transfer.process.binding.https.md +++ b/transfer/transfer.process.binding.https.md @@ -10,32 +10,32 @@ The OpenAPI definitions for this specification can be accessed [here](TBD). ### 2.1 Prerequisites -1. The `` notation indicates the base URL for a connector endpoint. For example, if the base connector URL is `connector.example.com`, the +1. The `` notation indicates the base URL for a [Connector](../model/terminology.md#connector--data-service-) endpoint. For example, if the base [Connector](../model/terminology.md#connector--data-service-) URL is `connector.example.com`, the URL `https:///transfers/request` will map to `https//connector.example.com/transfers/request`. 2. All request and response messages must use the `application/json` media type. ### 2.2 TransferError -In the event of a client request error, the connector must return an appropriate HTTP 4xxx client error code. If an error body is returned it must be +In the event of a client request error, the [Connector](../model/terminology.md#connector--data-service-) must return an appropriate HTTP 4xxx client error code. If an error body is returned it must be a [TransferError](./message/transfer-error.json) with the following properties: | Field | Type | Description | |-------------|---------------|-------------------------------------------------------------| -| consumerPid | UUID | The transfer process unique id on consumer side. | -| providerPid | UUID | The transfer process unique id on provider side. | +| consumerPid | UUID | The [Transfer Process](../model/terminology.md#transfer-process) unique id on consumer side. | +| providerPid | UUID | The [Transfer Process](../model/terminology.md#transfer-process) unique id on provider side. | | code | string | An optional implementation-specific error code. | | reasons | Array[object] | An optional array of implementation-specific error objects. | ### 2.2.1 State transition errors -If a client or provider connector makes a request that results in an invalid transfer process state transition as defined by the Transfer Process Protocol, it must return +If a client or provider [Connector](../model/terminology.md#connector--data-service-) makes a request that results in an invalid [Transfer Process](../model/terminology.md#transfer-process) state transition as defined by the Transfer Process Protocol, it must return an HTTP code 400 (Bad Request) with an `TransferError` in the response body. ### 2.3 Authorization All requests should use the `Authorization` header to include an authorization token. The semantics of such tokens are not part of this specification. The `Authorization` HTTP -header is optional if the connector does not require authorization. +header is optional if the [Connector](../model/terminology.md#connector--data-service-) does not require authorization. ### 2.4 The provider `transfers` resource @@ -48,7 +48,7 @@ Authorization: ... ``` -If the transfer process is found and the client is authorized, the provider connector must return an HTTP 200 (OK) response and a body containing +If the [Transfer Process](./transfer.process.protocol.md#ack---transferprocess) is found and the client is authorized, the provider [Connector](../model/terminology.md#connector--data-service-) must return an HTTP 200 (OK) response and a body containing the [TransferProcess](./message/transfer-process.json): ``` @@ -63,13 +63,13 @@ the [TransferProcess](./message/transfer-process.json): Predefined states are: `REQUESTED`, `STARTED`, `SUSPENDED`, `REQUESTED`, `COMPLETED`, and `TERMINATED`. -If the transfer process does not exist or the client is not authorized, the provider connector must return an HTTP 404 (Not Found) response. +If the [Transfer Process](./transfer.process.protocol.md#ack---transferprocess) does not exist or the client is not authorized, the provider [Connector](../model/terminology.md#connector--data-service-) must return an HTTP 404 (Not Found) response. ### 2.5 The provider `transfers/request` resource #### 2.5.1 POST -A transfer process is started and placed in the `REQUESTED` state when a consumer POSTs a [TransferRequestMessage](./transfer.process.protocol.md#1-transferrequestmessage) +A [Transfer Process](../model/terminology.md#transfer-process) is started and placed in the `REQUESTED` state when a consumer POSTs a [TransferRequestMessage](./transfer.process.protocol.md#1-transferrequestmessage) to `transfers/request`: ``` @@ -99,13 +99,13 @@ to `transfers/request`: } ``` -The `callbackAddress` property specifies the base endpoint `URL` where the client receives messages associated with the transfer process. Support for the `HTTPS` scheme is +The `callbackAddress` property specifies the base endpoint `URL` where the client receives messages associated with the [Transfer Process](../model/terminology.md#transfer-process). Support for the `HTTPS` scheme is required. Implementations may optionally support other URL schemes. -Callback messages will be sent to paths under the base URL as described by this specification. Note that provider connectors should properly handle the cases where a trailing `/` +Callback messages will be sent to paths under the base URL as described by this specification. Note that provider [Connectors](../model/terminology.md#connector--data-service-) should properly handle the cases where a trailing `/` is included with or absent from the `callbackAddress` when resolving full URL. -The provider connector must return an HTTP 201 (Created) response with a body containing +The provider [Connector](../model/terminology.md#connector--data-service-) must return an HTTP 201 (Created) response with a body containing the [TransferProcess](./message/transfer-process.json) message: ``` @@ -123,35 +123,35 @@ the [TransferProcess](./message/transfer-process.json) message: #### 2.6.1 POST -The consumer connector can POST a [TransferStartMessage](./message/transfer-start-message.json) to attempt to start a transfer process after it has been suspended. If the transfer -process state is successfully transitioned, the provider must return HTTP code 200 (OK). The response body is not specified and clients are not required to process it. +The consumer [Connector](../model/terminology.md#connector--data-service-) can POST a [TransferStartMessage](./message/transfer-start-message.json) to attempt to start a [Transfer Process](../model/terminology.md#transfer-process) after it has been suspended. If the [Transfer +Process's](transfer.process.protocol.md#ack---transferprocess) state is successfully transitioned, the provider must return HTTP code 200 (OK). The response body is not specified and clients are not required to process it. ### 2.7 The provider `transfers/:providerPid/completion` resource #### 2.7.1 POST -The consumer connector can POST a [TransferCompletionMessage](./message/transfer-completion-message.json) to complete a transfer process. If the transfer -process state is successfully transitioned, the provider must return HTTP code 200 (OK). The response body is not specified and clients are not required to process it. +The consumer [Connector](../model/terminology.md#connector--data-service-) can POST a [TransferCompletionMessage](./message/transfer-completion-message.json) to complete a [Transfer Process](../model/terminology.md#transfer-process). If the [Transfer +Process's](transfer.process.protocol.md#ack---transferprocess) state is successfully transitioned, the provider must return HTTP code 200 (OK). The response body is not specified and clients are not required to process it. ### 2.8 The provider `transfers/:providerPid/termination` resource #### 2.8.1 POST -The consumer connector can POST a [TransferTerminationMessage](./message/transfer-termination-message.json) to terminate a transfer process. If the transfer -process state is successfully transitioned, the provider must return HTTP code 200 (OK). The response body is not specified and clients are not required to process it. +The consumer [Connector](../model/terminology.md#connector--data-service-) can POST a [TransferTerminationMessage](./message/transfer-termination-message.json) to terminate a [Transfer Process](../model/terminology.md#transfer-process). If the [Transfer +Process's](transfer.process.protocol.md#ack---transferprocess) state is successfully transitioned, the provider must return HTTP code 200 (OK). The response body is not specified and clients are not required to process it. ### 2.9 The provider `transfers/:providerPid/suspension` resource #### 2.9.1 POST -The consumer connector can POST a [TransferSuspensionMessage](./message/transfer-suspension-message.json) to suspend a transfer process. If the transfer -process state is successfully transitioned, the provider must return HTTP code 200 (OK). The response body is not specified and clients are not required to process it. +The consumer [Connector](../model/terminology.md#connector--data-service-) can POST a [TransferSuspensionMessage](./message/transfer-suspension-message.json) to suspend a [Transfer Process](../model/terminology.md#transfer-process). If the [Transfer +Process's](transfer.process.protocol.md#ack---transferprocess) state is successfully transitioned, the provider must return HTTP code 200 (OK). The response body is not specified and clients are not required to process it. ## 3 Consumer Callback Path Bindings ### 3.1 Prerequisites -All callback paths are relative to the `callbackAddress` base URL specified in the `TransferRequestMessage` that initiated a transfer process. For example, if +All callback paths are relative to the `callbackAddress` base URL specified in the `TransferRequestMessage` that initiated a [Transfer Process](../model/terminology.md#transfer-process). For example, if the `callbackAddress` is specified as `https://connector.consumer/callback` and a callback path binding is `transfers/:consumerPid/start`, the resolved URL will be `https://connector.consumer.com/callback/transfers/:consumerPid/start`. @@ -159,26 +159,26 @@ be `https://connector.consumer.com/callback/transfers/:consumerPid/start`. #### 3.2.1 POST -The provider connector can POST a [TransferStartMessage](./message/transfer-start-message.json) to indicate the start of a transfer process. If the transfer -process state is successfully transitioned, the consumer must return HTTP code 200 (OK). The response body is not specified and clients are not required to process it. +The provider [Connector](../model/terminology.md#connector--data-service-) can POST a [TransferStartMessage](./message/transfer-start-message.json) to indicate the start of a [Transfer Process](../model/terminology.md#transfer-process). If the [Transfer +Process's](transfer.process.protocol.md#ack---transferprocess) state is successfully transitioned, the consumer must return HTTP code 200 (OK). The response body is not specified and clients are not required to process it. ### 3.3 The consumer `transfers/:consumerPid/completion` resource #### 3.3.1 POST -The provider connector can POST a [TransferCompletionMessage](./message/transfer-completion-message.json) to complete a transfer process. If the transfer -process state is successfully transitioned, the consumer must return HTTP code 200 (OK). The response body is not specified and clients are not required to process it. +The provider [Connector](../model/terminology.md#connector--data-service-) can POST a [TransferCompletionMessage](./message/transfer-completion-message.json) to complete a [Transfer Process](../model/terminology.md#transfer-process). If the [Transfer +Process's](transfer.process.protocol.md#ack---transferprocess) state is successfully transitioned, the consumer must return HTTP code 200 (OK). The response body is not specified and clients are not required to process it. ### 3.4 The consumer `transfers/:consumerPid/termination` resource #### 3.4.1 POST -The provider connector can POST a [TransferTerminationMessage](./message/transfer-termination-message.json) to terminate a transfer process. If the transfer -process state is successfully transitioned, the consumer must return HTTP code 200 (OK). The response body is not specified and clients are not required to process it. +The provider [Connector](../model/terminology.md#connector--data-service-) can POST a [TransferTerminationMessage](./message/transfer-termination-message.json) to terminate a [Transfer Process](../model/terminology.md#transfer-process). If the [Transfer +Process's](transfer.process.protocol.md#ack---transferprocess) state is successfully transitioned, the consumer must return HTTP code 200 (OK). The response body is not specified and clients are not required to process it. ### 3.5 The consumer `transfers/:consumerPid/suspension` resource #### 3.5.1 POST -The provider connector can POST a [TransferSuspensionMessage](./message/transfer-suspension-message.json) to suspend a transfer process. If the transfer -process state is successfully transitioned, the consumer must return HTTP code 200 (OK). The response body is not specified and clients are not required to process it. +The provider [Connector](../model/terminology.md#connector--data-service-) can POST a [TransferSuspensionMessage](./message/transfer-suspension-message.json) to suspend a [Transfer Process](../model/terminology.md#transfer-process). If the [Transfer +Process's](transfer.process.protocol.md#ack---transferprocess) state is successfully transitioned, the consumer must return HTTP code 200 (OK). The response body is not specified and clients are not required to process it. diff --git a/transfer/transfer.process.protocol.md b/transfer/transfer.process.protocol.md index b55e91e5..7d0ae142 100644 --- a/transfer/transfer.process.protocol.md +++ b/transfer/transfer.process.protocol.md @@ -2,27 +2,27 @@ ## Introduction: Terms -This document outlines the key elements of the transfer process protocol. The following terms are used: +This document outlines the key elements of the Transfer Process Protocol. The following terms are used: - A _**message type**_ defines the structure of a _message_. - A _**message**_ is an instantiation of a _message type_. - The _**transfer process protocol**_ is the set of allowable message type sequences and is defined as a state machine (TP-SM). - A _**transfer process (TP)**_ is an instantiation of the CNP-TP. -- A _**provider**_ is a participant agent that offers a dataset. -- A _**consumer**_ is a participant agent that requests access to an offered dataset. -- A _**Connector**_ is a `PariticipantAgent` that produces `Agreements` and manages `Dataset` sharing. -- An _**Dataset**_ is data or a service a provider grants access to. -- An _**Agreement**_ is a result of a [Contract Negotiation](../negotiation/contract.negotiation.protocol.md) and is associated with _exactly one_ `Dataset`. +- A _**provider**_ is a [Participant Agent](../model/terminology.md#participant-agent) that offers a [Dataset](../model/terminology.md#dataset). +- A _**consumer**_ is a [Participant Agent](../model/terminology.md#participant-agent) that requests access to an offered [Dataset](../model/terminology.md#dataset). +- A _**Connector**_ is a [Participant Agent](../model/terminology.md#participant-agent) that produces [Agreements](../model/terminology.md#agreement) and manages [Dataset](../model/terminology.md#dataset) sharing. +- A _**Dataset**_ is data or a service a provider grants access to. +- An _**Agreement**_ is a result of a [Contract Negotiation](../negotiation/contract.negotiation.protocol.md) and is associated with _exactly one_ [Dataset](../model/terminology.md#dataset). ## Transfer Process Protocol -A transfer process (TP) involves two parties, a _provider_ that offers one or more datasets under a usage policy and _consumer_ that requests datasets. A TP progresses through +A [Transfer Process](../model/terminology.md#transfer-process) (TP) involves two parties, a _provider_ that offers one or more [Datasets](../model/terminology.md#dataset) under a [Usage Policy](../model/terminology.md#policy) and _consumer_ that requests [Datasets](../model/terminology.md#dataset). A TP progresses through a series of states, which are tracked by the provider and consumer using messages. A TP transitions to a state in response to a message from the counter-party. ### Connector Components: Control and Data Planes -A TP is managed by a `Connector`. The connector consists of two logical components, a `Control Plane` and a `Data Plane`. The control plane serves as a coordinating layer that -receives counter-party messages and manages the TP state. The data plane performs the actual transfer of data using a wire protocol. Both participants run control and data +A TP is managed by a [Connector](../model/terminology.md#connector--data-service-). The [Connector](../model/terminology.md#connector--data-service-) consists of two logical components, a `Control Plane` and a `Data Plane`. The control plane serves as a coordinating layer that +receives counter-party messages and manages the TP state. The data plane performs the actual transfer of data using a wire protocol. Both [Participants](../model/terminology.md#participant) run control and data planes. It is important to note that the control and data planes are logical constructs. Implementations may choose to deploy both components within a single process or across @@ -30,7 +30,7 @@ heterogeneous clusters. ### Dataset Transfer Types -Dataset transfers are characterized as `push` or `pull` transfers and it's data is either `finite` or `non-finite`. This section describes the difference between these types. +[Dataset](../model/terminology.md#dataset) transfers are characterized as `push` or `pull` transfers and it's data is either `finite` or `non-finite`. This section describes the difference between these types. #### Push Transfer @@ -49,18 +49,18 @@ message, the consumer can request the data from the provider-specified endpoint. #### Finite and Non-Finite Data Data may be `finite` or `non-finite.` Finite data is data that is defined by a finite set, for example, machine learning data or images. After finite data transmission has -finished, the transfer process is completed. Non-finite data is data that is defined by an infinite set or has no specified end, for example streams or an API endpoint. With +finished, the [Transfer Process](../model/terminology.md#transfer-process) is completed. Non-finite data is data that is defined by an infinite set or has no specified end, for example streams or an API endpoint. With non-finite data, a TP will continue indefinitely until either the consumer or provider explicitly terminates the transmission. ### Transfer Process States The TP states are: -- **REQUESTED** - A dataset has been requested under an `Agreement` by the consumer and the provider has sent an ACK response. -- **STARTED** - The dataset is available for access by the consumer or the provider has begun pushing the data to the consumer endpoint. +- **REQUESTED** - A [Dataset](../model/terminology.md#dataset) has been requested under an [Agreement](../model/terminology.md#agreement) by the consumer and the provider has sent an ACK response. +- **STARTED** - The [Dataset](../model/terminology.md#dataset) is available for access by the consumer or the provider has begun pushing the data to the consumer endpoint. - **COMPLETED** - The transfer has been completed by either the consumer or the provider. - **SUSPENDED** - The transfer has been suspended by the consumer or the provider. -- **TERMINATED** - The transfer process has been terminated by the consumer or the provider. +- **TERMINATED** - The [Transfer Process](../model/terminology.md#transfer-process) has been terminated by the consumer or the provider. ### Transfer Process State Machine @@ -84,21 +84,21 @@ The TP states are: #### Description -The `TransferRequestMessage` is sent by a consumer to initiate a transfer process. +The `TransferRequestMessage` is sent by a consumer to initiate a [Transfer Process](../model/terminology.md#transfer-process). #### Notes - The `consumerPid` property refers to the transfer id on consumer side. -- The `agreementId` property refers to an existing contract agreement between the consumer and provider. -- The `dct:format` property is a format specified by a `Distribution` for the `Dataset` associated with the agreement. This is generally obtained from the provider `Catalog`. +- The `agreementId` property refers to an existing [Agreement](../model/terminology.md#agreement) between the consumer and provider. +- The `dct:format` property is a format specified by a `Distribution` for the [Dataset](../model/terminology.md#dataset) associated with the [Agreement](../model/terminology.md#agreement). This is generally obtained from the provider [Catalog](../model/terminology.md#catalog). - The `dataAddress` property must only be provided if the `dct:format` requires a push transfer. - `callbackAddress` is a URI indicating where messages to the consumer should be sent. If the address is not understood, the provider MUST return an UNRECOVERABLE error. Providers should implement idempotent behavior for `TransferRequestMessage` based on the value of `dspace:consumerPid`. Providers may choose to implement idempotent behavior for a certain period of -time. For example, until a transfer processes has completed and been archived after an implementation-specific expiration period. If a request for the given `dspace:consumerPid` has already been +time. For example, until a [Transfer Process](../model/terminology.md#transfer-process) has completed and been archived after an implementation-specific expiration period. If a request for the given `dspace:consumerPid` has already been received *and* the same consumer sent the original message, the provider should respond with an appropriate `DataAddressMessage`. -Once a transfer process have been created, all associated callback messages must include a `dspace:consumerPid` and `dspace:providerPid`. +Once a [Transfer Process](../model/terminology.md#transfer-process) have been created, all associated callback messages must include a `dspace:consumerPid` and `dspace:providerPid`. Providers must include a `dspace:consumerPid` and a `dspace:providerPid` property in the `TransferProcess`. @@ -123,11 +123,11 @@ Providers must include a `dspace:consumerPid` and a `dspace:providerPid` propert #### Description -The `TransferStartMessage` is sent by the provider to indicate the dataset transfer has been initiated. +The `TransferStartMessage` is sent by the provider to indicate the [Dataset](../model/terminology.md#dataset) transfer has been initiated. #### Notes -- The `dataAddress` is only provided if the current transfer is a pull transfer and contains a transport-specific endpoint address for obtaining the dataset. It may include a temporary authorization via the `dspace:endpointProperties` property. +- The `dataAddress` is only provided if the current transfer is a pull transfer and contains a transport-specific endpoint address for obtaining the [Dataset](../model/terminology.md#dataset). It may include a temporary authorization via the `dspace:endpointProperties` property. ### 3. TransferSuspensionMessage @@ -145,7 +145,7 @@ The `TransferStartMessage` is sent by the provider to indicate the dataset trans #### Description -The `TransferSuspensionMessage` is sent by the provider or consumer when either of them needs to temporarily suspend the data transfer. +The `TransferSuspensionMessage` is sent by the provider or consumer when either of them needs to temporarily suspend the [Transfer Process](../model/terminology.md#transfer-process). ### 4. TransferCompletionMessage @@ -163,7 +163,7 @@ The `TransferSuspensionMessage` is sent by the provider or consumer when either #### Description -The `TransferCompletionMessage` is sent by the provider or consumer when a dataset transfer has completed. Note that some data plane implementations may optimize completion +The `TransferCompletionMessage` is sent by the provider or consumer when a [Dataset](../model/terminology.md#dataset) transfer has completed. Note that some data plane implementations may optimize completion notification by performing it as part of its wire protocol. In those cases, a `TransferCompletionMessage` message does not need to be sent. ### 5. TransferTerminationMessage @@ -182,7 +182,7 @@ notification by performing it as part of its wire protocol. In those cases, a `T #### Description -The `TransferTerminationMessage` is sent by the provider or consumer at any point except a terminal state to indicate the data transfer process should stop and be placed in +The `TransferTerminationMessage` is sent by the provider or consumer at any point except a terminal state to indicate the [Transfer Process](../model/terminology.md#transfer-process) should stop and be placed in a terminal state. If the termination was due to an error, the sender may include error information.