Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Data Provisioning by Trace-X #746

Merged
merged 54 commits into from
Mar 14, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
54 commits
Select commit Hold shift + click to select a range
5949acb
feature(tx-backend): #536 draft subsequential flow
ds-ext-sceronik Feb 29, 2024
af55c68
feature(tx-backend): #536 integration test for subsequential flow
ds-ext-sceronik Mar 5, 2024
fed4af0
feature(tx-backend): #536 integration test for subsequential flow
ds-ext-sceronik Mar 5, 2024
ef657dc
feature(tx-backend): #536 merge main into branch and resolve conflict
ds-ext-sceronik Mar 5, 2024
322f671
feature(tx-backend): #536 testing env config
ds-ext-sceronik Mar 5, 2024
a407e10
feature(tx-backend): #536 testing test with override
ds-ext-sceronik Mar 5, 2024
01122a3
feature(tx-backend): #536 testing with adjustments to lib code
ds-ext-sceronik Mar 6, 2024
14cab7e
feature(tx-backend): #536 testing with adjustments to lib code
ds-ext-sceronik Mar 6, 2024
206ac80
feature(tx-backend): #536 testing with adjustments to lib code
ds-ext-sceronik Mar 6, 2024
06158f1
feature(tx-backend): #536 change shell descriptor path used in service
ds-ext-sceronik Mar 6, 2024
10aafdc
feature(tx-backend): #536 change shell descriptor path used in service
ds-ext-sceronik Mar 6, 2024
01a9d45
feature(tx-backend): #536 submodel asset base url change
ds-ext-sceronik Mar 6, 2024
fe7f6e0
feature(tx-backend): #536 submodel asset base url change
ds-ext-sceronik Mar 6, 2024
31fcdbb
feature(tx-backend): #536 change shortId mapping to be null safe
ds-ext-sceronik Mar 7, 2024
811c000
feature(tx-backend): #536 adjust subjectIds
ds-ext-sceronik Mar 7, 2024
af1211f
feature(tx-backend): #536 adjust subjectIds
ds-ext-sceronik Mar 7, 2024
9ddaad7
feature(tx-backend): #536 submodel asset id randomly generated urn:uuid
ds-ext-sceronik Mar 8, 2024
1a59410
feature(tx-backend): #536 merge main
ds-ext-sceronik Mar 8, 2024
ee285b4
feature(tx-backend): #536 merge main
ds-ext-sceronik Mar 8, 2024
415c236
feature(tx-backend): #536 adjustments to IRS lib
ds-ext-sceronik Mar 11, 2024
bebd622
Merge branch 'main' into feature/536-subsequential-flow-in-trace-x
ds-ext-sceronik Mar 11, 2024
477fed0
feature(tx-backend): #536 deployment adjustments
ds-ext-sceronik Mar 11, 2024
39964cf
feature(tx-backend): #536 remove test classes
ds-ext-sceronik Mar 11, 2024
b404281
feature(tx-backend): #536 adjustments to the flow
ds-ext-sceronik Mar 12, 2024
0bffc71
feature(tx-backend): #536 merge main into the branch
ds-ext-sceronik Mar 12, 2024
0d02218
feature(tx-backend): #536 adjust code to create policies from policy …
ds-ext-sceronik Mar 12, 2024
a7a6f0f
feature(tx-backend): #536 cleanup code and adjust for testing externa…
ds-ext-sceronik Mar 12, 2024
c78367a
feature(tx-backend): #536 remove comment
ds-ext-sceronik Mar 12, 2024
0ca983b
feature(tx-backend): #536 Missing import state in Import State Response
ds-ext-sceronik Mar 12, 2024
4777ec6
feature(tx-backend): #536 Missing import state in Import State Response
ds-ext-sceronik Mar 12, 2024
91febf2
feature(tx-backend): #536 test
ds-ext-sceronik Mar 12, 2024
aaac175
feature(tx-backend): #536 test
ds-ext-sceronik Mar 12, 2024
7132874
feature(tx-backend): #536 test
ds-ext-sceronik Mar 12, 2024
2ace6cc
feature(tx-backend): #536 test specified external subjects
ds-ext-sceronik Mar 12, 2024
516e42c
feature(tx-backend): #536 adjust code to create externalSubjectIds fr…
ds-ext-sceronik Mar 13, 2024
a1f385a
feature(tx-backend): #536 merge main
ds-ext-sceronik Mar 13, 2024
41f4b3d
feature(tx-backend): #536 fix Value injection on allowedBpns
ds-ext-sceronik Mar 13, 2024
fe9be22
feature(tx-backend): #536 fix merge main
ds-ext-sceronik Mar 13, 2024
c619b98
feature(tx-backend): #536 add allowed BPNS to deployment to map from …
ds-ext-sceronik Mar 13, 2024
3af89eb
feature(tx-backend): #536 call synchronize assets after publishing
ds-ext-sceronik Mar 13, 2024
a799f49
feature(tx-backend): #536 commit new exception
ds-ext-sceronik Mar 13, 2024
24b7fed
feature(tx-backend): #536 added more logging to cron job
ds-ext-sceronik Mar 13, 2024
b8352ed
feature(tx-backend): #536 documentation and adjustment to controller …
ds-ext-sceronik Mar 13, 2024
8f968d0
feature(tx-backend): #536 merge main
ds-ext-sceronik Mar 13, 2024
33c196e
feature(tx-backend): #536 after review
ds-ext-sceronik Mar 13, 2024
2653410
feature(tx-backend): #536 after review
ds-ext-sceronik Mar 13, 2024
fe58d3a
feature(tx-backend): #536 sonar issues
ds-ext-sceronik Mar 13, 2024
5cdaf34
feature(tx-backend): #536 fix lint issue
ds-ext-sceronik Mar 13, 2024
17e9b66
feature(tx-backend): #536 handle error state
ds-ext-sceronik Mar 14, 2024
42d4f4f
feature(tx-backend): #536 hendle error on EDC creation
ds-ext-sceronik Mar 14, 2024
e9596cb
feature(tx-backend): #536 fix scenarion number
ds-ext-sceronik Mar 14, 2024
36102f8
feature(tx-backend): #536 remove edcNotification related services and…
ds-ext-sceronik Mar 14, 2024
14d360f
Merge branch 'main' into feature/536-subsequential-flow-in-trace-x
ds-ext-sceronik Mar 14, 2024
60c0d0a
Merge pull request #1023 from catenax-ng/feature/536-subsequential-fl…
ds-mwesener Mar 14, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 6 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,17 +12,22 @@ _**For better traceability add the corresponding GitHub issue number in each cha
### Added
- #421 Added contract agreement view
- #515 Service Unavailable Response on Notification failure
- #536 Added import state PUBLISHED_TO_CX in frontend
- #536 Added import state PUBLISHED_TO_CORE_SERVICES in frontend
- #420 add /contracts api to fetch contract agreement information from EDC for assets
- Added a step to the pull-request-backend.yml which checks if the pom.xml(root) properties have some versions ending with -SNAPSHOT
- Added a PostConstruct method in PolicyStartUpConfig to allow Integration tests to run without errors in stack traces.
- #536 added new ImportState to asset PUBLISHED_TO_CORE_SERVICES indicating edc assets and dtr shells were created for given asset
- #536 added cron job responsible to publish assets in PUBLISHED_TO_CORE_SERVICES import state to edc and dtr

### Changed
- Updated RELEASE.md to the latest release guide (added more steps)
- #515 Fixed notification toast click area
- #625 increased height of tables
- #423 Moved errorMessages from investigation/alert to notification list
- Updated COMPATIBILITY.md matrix adding release C-X 24.3 and 23.12
- #536 rework /policies to respond with policies from the IRS policy store
- #536 sync assets logic was adjusted to create IRS jobs only for assets that are not in TRANSIENT or IN_SYNC states
- #536 Updated Arc42 documentation and user-manual with publish assets informations

### Removed
- #625 Removed the header and breadcrumbs section from app layout
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -98,6 +98,12 @@ spec:
value: {{ .Values.edc.apiKey | quote }}
- name: EDC_PROVIDER_URL
value: {{ .Values.edc.providerUrl | quote }}
- name: EDC_PROVIDER_DATAPLANE_URL
value: {{ .Values.edc.providerDataplaneUrl | quote }}
- name: REGISTRY_URL_WITH_PATH
value: {{ .Values.registry.urlWithPath | quote }}
- name: REGISTRY_ALLOWED_BPNS
value: {{ .Values.registry.allowedBpns | quote }}
- name: SUBMODEL_URL
value: {{ .Values.submodel.baseUrl | quote }}
- name: IRS_URL
Expand Down
1 change: 1 addition & 0 deletions charts/traceability-foss/charts/backend/values.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -169,6 +169,7 @@ irs:
baseUrl: "https://replace.me"
registry:
urlWithPath: "https://replace.me"
allowedBpns: "replace me"
portal:
baseUrl: "https://replace.me"

Expand Down
3 changes: 3 additions & 0 deletions charts/traceability-foss/values.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -293,6 +293,8 @@ backend:
callbackUrl: "CHANGEME" # example: http://<irs-helm.nameOverride>:8181/internal/endpoint-data-reference
callbackUrlEdcClient: "CHANGEME" # example: https://<backend.ingress.hosts>/api/internal/endpoint-data-reference
dataEndpointUrl: "CHANGEME" # example: https://<tractusx-connector.controlplane.ingress.hosts>/management"
partsProviderControlplaneUrl: "CHANGEME" # host of the parts provider EDC
partsProviderDataplaneUrl: "CHANGEME"

discoveryfinder:
baseUrl: "CHANGEME" # example: https://discoveryfinder.net/discoveryfinder/api/administration/connectors/discovery/search
Expand All @@ -301,6 +303,7 @@ backend:
baseUrl: "https://replace.me" # https://<irs-helm.ingress.host>
registry:
urlWithPath: "https://replace.me" # digitalTwinRegistry /semantics/registry/api/v3.0
allowedBpns: "BPN1,BPN2" # "," separated list of allowed bpns for creating shells
portal:
baseUrl: "https://replace.me"

Expand Down
1 change: 0 additions & 1 deletion docs/RELEASE.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,6 @@
</div>

## Release an app

Prerequisite:
Make sure eclipse / catena git repositories are in sync

Expand Down
11 changes: 7 additions & 4 deletions docs/src/docs/arc42/runtime-view/data-provisioning.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

This sequence diagrams describes the process of importing data from a Trace-X Dataformat

Modul 1
== Modul 1

Data will be imported by the Trace-X Frontend into Trace-X backend and will be persisted as asset by a Trace-X instance in a transient state.
The raw data which is needed for the shared services (DTR / EDC) will be persisted as well.
Expand All @@ -13,7 +13,7 @@ The raw data which is needed for the shared services (DTR / EDC) will be persist
include::../../../uml-diagrams/arc42/runtime-view/data-provisioning/trace-x-data-import-interface-modul1-sequence.puml[]
....

Modul 2 - DRAFT
== Modul 2

The frontend is able to select assets and publish / syncronize them with the shared services. DTR / EDC / Submodel API.

Expand All @@ -23,7 +23,7 @@ The frontend is able to select assets and publish / syncronize them with the sha
include::../../../uml-diagrams/arc42/runtime-view/data-provisioning/trace-x-data-import-interface-modul2-sequence.puml[]
....

Modul 3 - DRAFT
== Modul 3

The backend is able to persist the data in the DTR / EDC and allows to use IRS for resolving assets.

Expand All @@ -33,6 +33,9 @@ The backend is able to persist the data in the DTR / EDC and allows to use IRS f
include::../../../uml-diagrams/arc42/runtime-view/data-provisioning/trace-x-data-import-interface-modul3-sequence.puml[]
....

TODO: Add all scenarios for data-provisioning

include::data-provisioning/return-import-report.adoc[leveloffset=+1]

include::data-provisioning/publish-assets.adoc[leveloffset=+1]

include::data-provisioning/publish-assets-error.adoc[leveloffset=+1]
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
= Scenario 3: Publish assets Error on EDC or DTR

This section describes user interaction when publishing assets fails due to EDC or DTR error ( for example services are unavailable )

[plantuml,target=import-report-receive,format=svg]
....
include::../../../../uml-diagrams/arc42/runtime-view/data-provisioning/publish-assets-error.puml[]
....

== Overview

When a user publishes assets, TraceX-FOSS checks if the user has an adequate role ('ROLE_ADMIN').
If yes, then endpoint starts to publish assets to network.
If any of required Services are not available or returns Error response upon executing flow assets are set to ERROR state and user can retry publishing them at any time when services are available
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
= Scenario 2: Publish assets

This section describes user interaction when publishing assets

[plantuml,target=import-report-receive,format=svg]
....
include::../../../../uml-diagrams/arc42/runtime-view/data-provisioning/publish-assets.puml[]
....

== Overview

When a user publishes assets, TraceX-FOSS checks if the user has an adequate role ('ROLE_ADMIN').
If yes, then endpoint starts to publish assets to network.
5 changes: 4 additions & 1 deletion docs/src/docs/user/user-manual.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -199,7 +199,10 @@ The following table explains the different import state an asset can have:
|Asset is uploaded but not synchronized with the Item Relationship Service (IRS).

|in_synchronization
|Asset is in the process of synchronizing with the IRS.
|Asset is ready to be published.

|published_to_core_services
|Asset is published, EDC assets, DTR shell, Submodel are created

|persistent
|Asset is successfully synchronized with the IRS.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ TraceX -> TraceX: Extract globalAssetId as CatenaX UUID
TraceX -> TraceX: Filter for new CX UUIDs

loop for each new CX_UUID
TraceX -> IRS: Register job to get submodel data
TraceX -> IRS: Register job for assets not in TRANSIENT or IN_SYNC import states to get submodel data
rnote right TraceX
aspects, globalAssetId, bomLifecycle, collectAspects, direction, depth
end rnote
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
@startuml
skinparam monochrome true
skinparam shadowing false
autonumber "<b>[000]"

actor TraceXApiConsumer
activate TraceXApiConsumer

box "Trace-X FOSS" #LightGrey
participant TraceX
activate TraceX

TraceXApiConsumer -> TraceX : POST /assets/publish
TraceX -> TraceX : Module 2 fails to create EDC Assets or DTR shells
TraceXApiConsumer -> TraceXApiConsumer : GET /assets
TraceXApiConsumer -> TraceX : POST /assets/publish
TraceX -> TraceX : Module 2 process successfully
TraceXApiConsumer -> TraceXApiConsumer : GET /assets




@enduml
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
@startuml
skinparam monochrome true
skinparam shadowing false
autonumber "<b>[000]"

actor TraceXApiConsumer
activate TraceXApiConsumer

box "Trace-X FOSS" #LightGrey
participant TraceX
activate TraceX

TraceXApiConsumer -> TraceX : POST /assets/publish
TraceX -> TraceX : Module 2
TraceXApiConsumer -> TraceXApiConsumer : GET /assets





@enduml
Original file line number Diff line number Diff line change
@@ -1,15 +1,17 @@
@startuml
participant FE
participant BE
autonumber "<b>[000]"

FE -> BE: [001] request assets: GET/assetsAsxxx
BE --> FE: [002] return assets_as_built OR assets_as_planned
FE -> FE: [003] present assets
FE -> BE: [004] select assets to synchronize: GET/policies
BE --> FE: [005] return policies
FE -> FE: [006] open detailview & assign policy (via dropdown)
FE -> BE: [007] register assets for publishing: POST/assets/sync
BE --> FE: [008] update asset state to IN_SYNC
BE -> BE: [008] trigger 'publish AAS Workflow' (Job scheduler)
FE -> FE: [009] refresh of FE view
FE -> BE: request assets: GET/assetsAsxxx
BE --> FE: return assets_as_built OR assets_as_planned
FE -> FE: present assets
FE -> BE: select assets to synchronize: GET/policies
BE --> FE: return policies
FE -> FE: open detailview & assign policy (via dropdown)
FE -> BE: register assets for publishing: POST/assets/publish
BE -> BE: update asset state to IN_SYNC
BE -> BE: trigger 'publish AAS Workflow' (Job scheduler)
BE -> BE: trigger 'data consumption process'
FE -> FE: refresh of FE view
@enduml
Original file line number Diff line number Diff line change
Expand Up @@ -3,19 +3,32 @@ participant BE
participant EDC
participant Registry
participant Submodels
participant Irs
autonumber "<b>[000]"

BE ->> BE: [001] scheduler job
BE ->> BE: [002] receive list of IN_SYNC_assets
BE ->> EDC: [003] create asset in EDC: POST/create/asset
EDC -->> BE: [004] response
BE ->> EDC: [005] create policy in EDC: POST/create/policy
EDC -->> BE: [006] response
BE ->> EDC: [007] create contract in EDC: POST/create/contract
EDC -->> BE: [008] response
BE ->> Submodels: [009] create submodel: POST/submodel
Submodels -->> BE: [010]
BE ->> Registry: [011] register shell in registry: POST/semantics/registry
Registry -->> BE: [012]
BE ->> BE: [013] update asset state PUBLISHED_TO_CX
BE ->> BE: [014] trigger IRS sync
BE ->> BE: scheduler job
BE ->> BE: receive list of IN_SYNC assets
BE ->> Irs: get policy for assets from policy store: GET/irs/policies
Irs -->> BE: response
BE ->> EDC: create policy in EDC: POST/create/policy
EDC -->> BE: response
BE ->> EDC: create DTR asset in EDC: POST/create/asset
EDC -->> BE: response
BE ->> EDC: create DTR contract in EDC: POST/create/contract
EDC -->> BE: response
BE ->> EDC: create Submodel asset in EDC: POST/create/asset
EDC -->> BE: response
BE ->> EDC: create Submodel contract in EDC: POST/create/contract
EDC -->> BE: response
BE ->> Submodels: create submodel: POST/submodel
Submodels -->> BE:
BE ->> Registry: [017] register shell in registry: POST/semantics/registry
alt Successful DTR shell creation
Registry -->> BE: shell created
BE ->> BE: update asset state PUBLISHED_TO_CORE_SERVICES
BE ->> BE: trigger IRS sync
else DTR shell was not created
Registry -->> BE: create shell not successful or service unavailable
BE ->> BE: update asset state ERROR
end
@enduml
Original file line number Diff line number Diff line change
Expand Up @@ -66,7 +66,7 @@ export const mockBmwAssets = [
'receivedQualityAlertIdsInStatusActive': [],
'sentQualityInvestigationIdsInStatusActive': [],
'receivedQualityInvestigationIdsInStatusActive': [],
'importState': 'PUBLISHED_TO_CX',
'importState': 'PUBLISHED_TO_CORE_SERVICES',
'importNote': 'This is a test import note.'
},
{
Expand Down
4 changes: 2 additions & 2 deletions frontend/src/app/modules/page/parts/model/parts.model.ts
Original file line number Diff line number Diff line change
Expand Up @@ -176,7 +176,7 @@ export enum ImportState {
IN_SYNCHRONIZATION = "IN_SYNCHRONIZATION",
ERROR = "ERROR",
UNSET = "UNSET",
PUBLISHED_TO_CX="PUBLISHED_TO_CX"
PUBLISHED_TO_CORE_SERVICES="PUBLISHED_TO_CORE_SERVICES"
}

export enum ImportStateInCamelCase {
Expand All @@ -185,7 +185,7 @@ export enum ImportStateInCamelCase {
IN_SYNCHRONIZATION = "In Synchronization",
ERROR = "Error",
UNSET = "Unset",
PUBLISHED_TO_CX="Published to CX"
PUBLISHED_TO_CORE_SERVICES="Published to Core Services"
}

export enum FilterOperator {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@
background-color: rgb(255,236,189);
}

&__PUBLISHED_TO_CX {
&__PUBLISHED_TO_CORE_SERVICES {
background-color: rgb(218, 245, 255);
}

Expand Down
4 changes: 2 additions & 2 deletions frontend/src/assets/locales/de/common.json
Original file line number Diff line number Diff line change
Expand Up @@ -145,7 +145,7 @@
"IN_SYNCHRONIZATION": "IN SYNCHRONISATION",
"ERROR": "FEHLER",
"UNSET": "UNDEFINIERT",
"PUBLISHED_TO_CX": "VERÖFFENTLICHT"
"PUBLISHED_TO_CORE_SERVICES": "VERÖFFENTLICHT"
},
"severity": {
"MINOR": "Gering",
Expand Down Expand Up @@ -336,7 +336,7 @@
"IN_SYNCHRONIZATION": "In Synchronisation",
"ERROR": "Fehler",
"UNSET": "Undefiniert",
"PUBLISHED_TO_CX": "Veröffentlicht"
"PUBLISHED_TO_CORE_SERVICES": "Veröffentlicht"
}
},
"parts": {
Expand Down
4 changes: 2 additions & 2 deletions frontend/src/assets/locales/en/common.json
Original file line number Diff line number Diff line change
Expand Up @@ -143,7 +143,7 @@
"IN_SYNCHRONIZATION": "IN SYNCHRONIZATION",
"ERROR": "ERROR",
"UNSET": "UNSET",
"PUBLISHED_TO_CX": "PUBLISHED"
"PUBLISHED_TO_CORE_SERVICES": "PUBLISHED"
},
"severity": {
"MINOR": "Minor",
Expand Down Expand Up @@ -338,7 +338,7 @@
"IN_SYNCHRONIZATION": "In Synchronization",
"ERROR": "Error",
"UNSET": "Unset",
"PUBLISHED_TO_CX": "Veröffentlicht"
"PUBLISHED_TO_CORE_SERVICES": "Veröffentlicht"

}
},
Expand Down
Loading
Loading