Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Chore/edc 0.7.0 infra #351

Merged
merged 21 commits into from
May 21, 2024
Merged
Show file tree
Hide file tree
Changes from 5 commits
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
5c72cbd
chore(edc): version increase wip
tom-rm-meyer-ISST Apr 23, 2024
14632d0
refactor(bdrs): move to infrastructure docker
tom-rm-meyer-ISST Apr 25, 2024
e30b7a6
refactor(docker-compose-edc): refactor to get to old state
tom-rm-meyer-ISST Apr 29, 2024
652e876
refactor(local): cleanup bdrs and so on
tom-rm-meyer-ISST Apr 29, 2024
ed119a2
feat(iam-mock): added iam-mock to infrastructure
tom-rm-meyer-ISST Apr 29, 2024
0041b97
feat(iam-mock): added non-root user to dockerfile
tom-rm-meyer-ISST Apr 30, 2024
7ce0213
chore: updated frontend license file
tom-rm-meyer-ISST Apr 30, 2024
85b3a3a
refactor(local): made dtr and edc of customer / supplier share one db…
tom-rm-meyer-ISST Apr 30, 2024
6f5db7f
refactor(local): made dtr and edc of customer / supplier share one db…
tom-rm-meyer-ISST May 3, 2024
19bdf5c
feat(iam-mock): added sts edr refresh workflow mock
tom-rm-meyer-ISST May 6, 2024
057a2f0
ci(.tractusx): exclude iam-mock Docker from qGate check as not distri…
tom-rm-meyer-ISST May 7, 2024
fdfaf8f
refactor(local deployment): removed comments and made mock-util only …
tom-rm-meyer-ISST May 14, 2024
4e56c84
refactor(generate-keys.sh): generate file seed-bdrs.sh
tom-rm-meyer-ISST May 14, 2024
60fa49f
chore(mock-util-service): updated dependencies
tom-rm-meyer-ISST May 14, 2024
14106c6
Merge branch 'main' into chore/edc-0.7.0-infra
tom-rm-meyer-ISST May 15, 2024
60c191d
chore: updated frontend dependencies
tom-rm-meyer-ISST May 15, 2024
815fd42
fix(iam-mock): corrected claims in credential flow
tom-rm-meyer-ISST May 16, 2024
b268dd7
fix(local/docker-compose.yaml): corrected version of dtr-supplier (me…
tom-rm-meyer-ISST May 16, 2024
7f42181
chore(tractus-x-edc/docker-compose.yaml): bump version to 0.7.1
tom-rm-meyer-ISST May 16, 2024
daae340
Merge branch 'main' into chore/edc-0.7.0-infra
tom-rm-meyer-ISST May 17, 2024
ec0b456
chore: remove unused edc compose and notes during migration
tom-rm-meyer-ISST May 21, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
72 changes: 50 additions & 22 deletions local/INSTALL.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,28 +2,34 @@

## Additional information for Windows users

If you want to use the local setup on a Windows machine, it seems advisable to use the Windows subsystem for Linux (WSL).
An installation guide can be found [here](https://learn.microsoft.com/en-us/windows/wsl/install).
If you want to use the local setup on a Windows machine, it seems advisable to use the Windows subsystem for Linux (
WSL).
An installation guide can be found [here](https://learn.microsoft.com/en-us/windows/wsl/install).

Also see this [additional information about using Docker in combination with WSL2](https://docs.docker.com/desktop/wsl/)

## Initial Setup
In case you had any previous installations of this project on your machine, it is advisable to remove them via the script

In case you had any previous installations of this project on your machine, it is advisable to remove them via the
script
(see below in the Notes on debugging section).

Run the following script to generate the necessary keys. It will also create an .env file in the ./local folder.
Make sure to have `openssl` and `jq` installed in your shell. openssl is pre-installed on most operating systems. jq can be
Run the following script to generate the necessary keys. It will also create an .env file in the ./local folder.
Make sure to have `openssl` and `jq` installed in your shell. openssl is pre-installed on most operating systems. jq can
be
installed via the usual installation repositories, see [here](https://jqlang.github.io/jq/download/)

```shell
cd local
sh generate-keys.sh
```

## Build
If you are doing a fresh install and everytime you edited the code of the PURIS frontend or backend you have to create a
new build of docker images for the PURIS frontend/backend.

For creating a docker image of the frontend, navigate your shell to the frontend folder and run
If you are doing a fresh install and everytime you edited the code of the PURIS frontend or backend you have to create a
new build of docker images for the PURIS frontend/backend.

For creating a docker image of the frontend, navigate your shell to the frontend folder and run

```
docker build -t puris-frontend:dev .
Expand All @@ -37,50 +43,72 @@ docker build -t puris-backend:dev .

Please see the INSTALL.md documents in the [frontend](../frontend/INSTALL.md) and [backend](../backend/INSTALL.md)

The default image tag is 'dev'. Remember to also adjust the tag in the docker-compose.yaml if you want to use different
tags.
The default image tag is 'dev'. Remember to also adjust the tag in the docker-compose.yaml if you want to use different
tags.

## Start
First start the infrastructure by navigating your shell to the local folder and running

First start the infrastructure by navigating your shell to the local folder and running

```shell
docker compose -f docker-compose-infrastructure.yaml up
```
After the MIW container has finished booting, use this script (also in the local folder) to initialise two wallets for customer and supplier:

Note: sh init-wallets.sh is temporarily not needed
After the MIW container has finished booting, use this script (also in the local folder) to initialise two wallets for
customer and supplier:

```shell
sh init-wallets.sh
```
Then start the PURIS demonstrator containers via:

After starting the central infrastructure, initialize the bdrs-service. To do so, in file `seed-bdrs.sh` set the key
to the value of `EDC_API_PW` from `.env` file generated earlier.

```shell
sh seed-brds.sh
```

Then start the PURIS demonstrator containers via:

```shell
docker compose up
```
Wait for the startup and visit http://localhost:3000/ for the customer's frontend or http://localhost:3001/ for the supplier side.

Whenever you have edited the source code in the frontend or backend and you want to test these changes, we recommend that you
stop all the containers, which are not part of the infrastructure, by deleting the volumes, i.e. run
Wait for the startup and visit http://localhost:3000/ for the customer's frontend or http://localhost:3001/ for the
supplier side.

Whenever you have edited the source code in the frontend or backend and you want to test these changes, we recommend
that you
stop all the containers, which are not part of the infrastructure, by deleting the volumes, i.e. run

```
docker compose down -v
```

Then, build a new docker image for the respective component (frontend and/or backend) as described above and then restart via
Then, build a new docker image for the respective component (frontend and/or backend) as described above and then
restart via

```shell
docker compose up
```

In general, it is not necessary to restart the infrastructure, if you had to rebuild frontend or backend.
However, in rare cases there may be issues with the MIW. If this
happens, you should use the cleanup script as mentioned in the debugging section below and then repeat the above-mentioned
steps beginning with the Initial Setup section.
In general, it is not necessary to restart the infrastructure, if you had to rebuild frontend or backend.
However, in rare cases there may be issues with the MIW. If this
happens, you should use the cleanup script as mentioned in the debugging section below and then repeat the
above-mentioned
steps beginning with the Initial Setup section.

## Notes on debugging

### Vault & Certs

When having problems with the certs or the vault, one may need to delete the vault container.
The following script stops all infrastructure containers as well as the PURIS demonstrator containers:
The following script stops all infrastructure containers as well as the PURIS demonstrator containers:

```shell
cd local
sh cleanup.sh
```

Then start your containers again with the aforementioned commands.
188 changes: 188 additions & 0 deletions local/MIGRATION_NOTES.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,188 @@
# Policy Definition -> still in alignment

Old

```json
{
"@context": {
"@vocab": "https://w3id.org/edc/v0.0.1/ns/",
"odrl": "http://www.w3.org/ns/odrl/2/"
},
"@type": "PolicyDefinitionRequestDto",
"@id": "{{POLICY_ID}}",
"policy": {
"@type": "Policy",
"odrl:permission": [
{
"odrl:action": "USE",
"odrl:constraint": {
"@type": "LogicalConstraint",
"odrl:or": [
{
"@type": "Constraint",
"odrl:leftOperand": "BusinessPartnerNumber",
"odrl:operator": {
"@id": "odrl:eq"
},
"odrl:rightOperand": "{{SUPPLIER_BPNL}}"
}
]
}
}
]
}
}
```

error

```json
[
{
"message": "https://w3id.org/edc/v0.0.1/ns/policy/@type was expected to be http://www.w3.org/ns/odrl/2/Set but it was not",
"type": "ValidationFailure",
"path": "https://w3id.org/edc/v0.0.1/ns/policy/@type",
"invalidValue": [
"https://w3id.org/edc/v0.0.1/ns/Policy"
]
}
]
```

policy.@type = "odrl:Set"

Catalog Request needs `counterPartyId`

# bdrs

Calls needed:

- management -> create bpn directory
- BPN-Directory -> map of bpn and did reachable

The EDC needs to self-IATP to get a `MembershipCredential` to use the BDRS

DIDs are build following JsonWebKey2020
DID-ID like did:web:name-to-use

Credential Service
mock: https://github.com/eclipse-tractusx/tractusx-edc/blob/main/edc-extensions/bdrs-client/src/test/java/org/eclipse/tractusx/edc/identity/mapper/BdrsClientImplComponentTest.java

Update Cache -> why does it need a bearer token with the membershipCredToken sent to /bpn-directory?
https://github.com/eclipse-tractusx/tractusx-edc/blob/main/edc-extensions/bdrs-client/src/main/java/org/eclipse/tractusx/edc/identity/mapper/BdrsClientImpl.java#L92

Dids seem to
be [dependent on the hosting companies' url](https://github.com/eclipse-tractusx/identity-trust/blob/main/specifications/tx.dataspace.topology.md)

Seems like:

- MIW / DIM are credential services
- Portal + DIM are issuer services

A client uses a token during a request, to grant access to specific resources

- verifier uses it to request the vp
- the CS endpoint is resolved using bdrs
- **What's the bearer access scope**

access scopes

- org.eclipse.tractusx.vc.type:Member:read
- org.eclipse.tractusx.vc.id:uuid:read -> give access to verifieable credential by id

Endoints:

- POST presentations/query
- uses OAuth2 scopes that need to be mapped to presentation definition
- storage api credentials

https://github.com/eclipse-edc/Connector/blob/4fd16b8e34d685239ea40fc3d8e9b02cc8ccf323/core/common/token-core/src/main/java/org/eclipse/edc/token/TokenValidationServiceImpl.java#L54

- a key is somehow resolved. This may be from the did.json

Following
this [test](https://github.com/eclipse-tractusx/tractusx-edc/blob/main/edc-extensions/bdrs-client/src/test/java/org/eclipse/tractusx/edc/identity/mapper/BdrsClientImplComponentTest.java),

- the VC is signed by the issuer
- the VP is signed by the holder

Following Tractus-X Connector Setup

- DIM = your wallet that already contains VCs -> encapsulated STS and CS
- Credential Service = get your own VP to hand over. (something like the miw)
- SecureTokenService = get auth for something and then request presentation

# Updates EDR

edr callback payload

```json
{
"id": "3099e0f1-e255-4a00-8a8b-8ec5c16e8758",
"at": 1714325393313,
"payload": {
"transferProcessId": "07231854-112b-45bb-957b-4fb01dc2718f",
"callbackAddresses": [
{
"uri": "http://mock-util-service:80/edr-log",
"events": [
"transfer.process.started"
],
"transactional": false,
"authKey": "None",
"authCodeId": "None"
}
],
"assetId": "ASSET_1",
"type": "CONSUMER",
"contractId": "54dd6fe4-7a4e-4de6-b7b8-2f131fc99f79",
"dataAddress": {
"properties": {
"process_id": "6570b7a5-7df9-42be-9fd7-80f200427fc3",
"participant_id": "BPNL1234567890ZZ",
"asset_id": "ASSET_1",
"https://w3id.org/edc/v0.0.1/ns/endpointType": "https://w3id.org/idsa/v4.1/HTTP",
"https://w3id.org/tractusx/auth/refreshEndpoint": "http://customer-data-plane:8285/api/public",
"https://w3id.org/tractusx/auth/audience": "did:web:mock-util-service/supplier",
"agreement_id": "54dd6fe4-7a4e-4de6-b7b8-2f131fc99f79",
"flow_type": "PULL",
"https://w3id.org/edc/v0.0.1/ns/type": "https://w3id.org/idsa/v4.1/HTTP",
"https://w3id.org/edc/v0.0.1/ns/endpoint": "http://customer-data-plane:8285/api/public",
"https://w3id.org/tractusx/auth/refreshToken": "eyJraWQiOiJjdXN0b21lci1jZXJ0IiwiYWxnIjoiUlMyNTYifQ.eyJleHAiOjE3MTQzMjU2OTMsImlhdCI6MTcxNDMyNTM5MywianRpIjoiMGY2YzM4NjItOGYxZS00YzU1LWIwMzEtNGMzM2NhZWIxMzY5In0.L_r5a_hZY3aFYw4SYOoV_Ct5yWuDJBRwPeujAPKv8aPVB_buRZHDPwwnrlYAIWa4j4QIiKjmMMFQN7NUi56tIYr3An3KGwfycekCAS5CSMMAx7x6In5JTRPyyBEi897gjXYGHDlfFa_j7G5bG4__InwDt5HF_2_BKTrPMGEEGL62pAm2cm9qfZJCNJx2R6tnkSymlR0E6Dju2FsCWiOIbYlPP6JHjDkU9aKRIv6l_n0HodRUELBLKBGi565O5zwkec9sNxYdv4mTwskU4IMOvGJPNgHE3QKpzyPCIl7CzVJICCaMszl698rAp9BYP0tokUNj8yNAKbR5ZutYFnAwSA",
"https://w3id.org/tractusx/auth/expiresIn": "300",
"https://w3id.org/edc/v0.0.1/ns/authorization": "eyJraWQiOiJjdXN0b21lci1jZXJ0IiwiYWxnIjoiUlMyNTYifQ.eyJpc3MiOiJCUE5MNDQ0NDQ0NDQ0NFhYIiwiYXVkIjoiQlBOTDEyMzQ1Njc4OTBaWiIsInN1YiI6IkJQTkw0NDQ0NDQ0NDQ0WFgiLCJleHAiOjE3MTQzMjU2OTMsImlhdCI6MTcxNDMyNTM5MywianRpIjoiMzMwMjhjZDEtMTVlZC00Njk1LWE0NjMtNDc2MTJlNmZhNDk5In0.AP8BY0gjnKFxeswCPRaalKPD-nyLtXqe8hpEQH_CcWoN48KLXLJzgyQXo04WtcCPe7QBU0dyOd9UBi71tmxPNNACLRg_HZVmAFfRZWSkCY9pr-sreChP0EJcTT7AXgHnBIT0mKZbcQ_8b8g9BI-nS43eAd52I_WAg6oTK5hvyMOha7H-HvPeyNDGPA5QQ2RKuf3JKEw-26RALZdgkLz0VDjHd9CMDJJC0nvkbzP928LvzmLs8r-e1YFJwFtZ-ipVlxb7OiFrg7UeAwwb46spi2epMj3Px1QLXrd-Fd9skV2Iw8PugPIUFm5ehyK2d5mQYB4waAm5kEmgVVLLvwVX8A",
"https://w3id.org/tractusx/auth/refreshAudience": "did:web:mock-util-service/supplier"
}
}
},
"type": "TransferProcessStarted"
}
```

Get against EDR API after Transfer Process:
`{{SUPPLIER_EDC}}/{{MANAGEMENT_PATH}}/v2/transferprocesses/{{TRANSFER_PROCESS_ID}}`
Will be loaded lazily

```json
{
"@type": "DataAddress",
"endpointType": "https://w3id.org/idsa/v4.1/HTTP",
"tx-auth:refreshEndpoint": "http://customer-data-plane:8285/api/public",
"tx-auth:audience": "did:web:mock-util-service/supplier",
"type": "https://w3id.org/idsa/v4.1/HTTP",
"endpoint": "http://customer-data-plane:8285/api/public",
"tx-auth:refreshToken": "eyJraWQiOiJjdXN0b21lci1jZXJ0IiwiYWxnIjoiUlMyNTYifQ.eyJleHAiOjE3MTQzMjU2OTMsImlhdCI6MTcxNDMyNTM5MywianRpIjoiMGY2YzM4NjItOGYxZS00YzU1LWIwMzEtNGMzM2NhZWIxMzY5In0.L_r5a_hZY3aFYw4SYOoV_Ct5yWuDJBRwPeujAPKv8aPVB_buRZHDPwwnrlYAIWa4j4QIiKjmMMFQN7NUi56tIYr3An3KGwfycekCAS5CSMMAx7x6In5JTRPyyBEi897gjXYGHDlfFa_j7G5bG4__InwDt5HF_2_BKTrPMGEEGL62pAm2cm9qfZJCNJx2R6tnkSymlR0E6Dju2FsCWiOIbYlPP6JHjDkU9aKRIv6l_n0HodRUELBLKBGi565O5zwkec9sNxYdv4mTwskU4IMOvGJPNgHE3QKpzyPCIl7CzVJICCaMszl698rAp9BYP0tokUNj8yNAKbR5ZutYFnAwSA",
"tx-auth:expiresIn": "300",
// use Header Authorization <authorization token>
"authorization": "eyJraWQiOiJjdXN0b21lci1jZXJ0IiwiYWxnIjoiUlMyNTYifQ.eyJpc3MiOiJCUE5MNDQ0NDQ0NDQ0NFhYIiwiYXVkIjoiQlBOTDEyMzQ1Njc4OTBaWiIsInN1YiI6IkJQTkw0NDQ0NDQ0NDQ0WFgiLCJleHAiOjE3MTQzMjU2OTMsImlhdCI6MTcxNDMyNTM5MywianRpIjoiMzMwMjhjZDEtMTVlZC00Njk1LWE0NjMtNDc2MTJlNmZhNDk5In0.AP8BY0gjnKFxeswCPRaalKPD-nyLtXqe8hpEQH_CcWoN48KLXLJzgyQXo04WtcCPe7QBU0dyOd9UBi71tmxPNNACLRg_HZVmAFfRZWSkCY9pr-sreChP0EJcTT7AXgHnBIT0mKZbcQ_8b8g9BI-nS43eAd52I_WAg6oTK5hvyMOha7H-HvPeyNDGPA5QQ2RKuf3JKEw-26RALZdgkLz0VDjHd9CMDJJC0nvkbzP928LvzmLs8r-e1YFJwFtZ-ipVlxb7OiFrg7UeAwwb46spi2epMj3Px1QLXrd-Fd9skV2Iw8PugPIUFm5ehyK2d5mQYB4waAm5kEmgVVLLvwVX8A",
"tx-auth:refreshAudience": "did:web:mock-util-service/supplier",
"@context": {
"@vocab": "https://w3id.org/edc/v0.0.1/ns/",
"edc": "https://w3id.org/edc/v0.0.1/ns/",
"tx": "https://w3id.org/tractusx/v0.0.1/ns/",
"tx-auth": "https://w3id.org/tractusx/auth/",
"cx-policy": "https://w3id.org/catenax/policy/",
"odrl": "http://www.w3.org/ns/odrl/2/"
}
}
```
9 changes: 9 additions & 0 deletions local/bdrs/application.properties
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
edc.api.auth.key=${EDC_API_PW}
web.http.management.port=8581
web.http.management.path=/api/management
web.http.directory.port=8582
web.http.directory.path=/api/directory
# looking up DIDs should not use https
edc.iam.did.web.use.https=false
edc.iam.trusted-issuer.issuer.id=did:web:mock-util-service/trusted-issuer
_level=DEBUG
2 changes: 2 additions & 0 deletions local/cleanup.sh
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,8 @@
docker compose down -v
docker compose -f docker-compose-infrastructure.yaml down -v
docker image rm local-vault
docker image rm local-mock-util-service
rm .env
rm ./vault/secrets -r
rm ./iam-mock/keys -r
echo "Deleted .env and vault/secrets"
Loading
Loading