Skip to content

Latest commit

 

History

History
515 lines (406 loc) · 21.9 KB

INSTALL.md

File metadata and controls

515 lines (406 loc) · 21.9 KB

INSTALL

BPDM offers Helm Charts, Dockerfiles and configuration files to support the installation process of the BPDM applications. The following chapters show how to install the applications in different scenarios.

Local Installation

Requirements

  • JAVA 21
  • Maven (3.9 supported)
  • Docker Engine (tested on 26.1.2)
  • Docker Compose (tested on 2.27.0)

Default Installation

BPDM services require a PostgreSQL database and Keycloak server to run. Navigate to the root folder of the BPDM repository. Then set up the necessary dependencies by using the provided Docker Compose file:

docker compose -f docker/compose/dependencies/docker-compose.yml up -d

This will run a Postgres and Keycloak with an initial realm already configured to be used by the BPDM services. You will need to pass the necessary client secrets when starting the BPDM services. Those client secrets are generated by Keycloak after the first time setup. Therefore, we will log in to the Keycloak server, look-up the secrets and write them into environment variables that are read by the BPDM services.

  1. Log in to Keycloaks admin console with the admin credentials (default in the Docker Compose file): http://localhost:8180/admin/master/console
  2. Navigate to realm "Cx-Central"
  3. Navigate to the following clients and write the client secret to the environment variable:
Client Environment Variable
DUMMY-ORCHESTRATOR-TASK_PROCESSOR BPDM_DUMMY_ORCH_CLIENT_SECRET
GATE-ORCHESTRATOR-TASK_CREATOR BPDM_GATE_ORCH_CLIENT_SECRET
GATE-POOL-SHARING_MEMBER BPDM_GATE_POOL_CLIENT_SECRET
POOL-ORCHESTRATOR-TASK-PROCESSOR BPDM_POOL_ORCH_CLIENT_SECRET

This information on what client expects which environment variable is also available in the application properties files of the respective BPDM applications (like for the gate) Now, make sure that client secrets are available in the expected environment variables:

export BPDM_DUMMY_ORCH_CLIENT_SECRET=...
export BPDM_GATE_ORCH_CLIENT_SECRET=...
export BPDM_GATE_POOL_CLIENT_SECRET=...
export BPDM_POOL_ORCH_CLIENT_SECRET=...

After this step, we can finally build, install the BPDM applications.

mvn clean install

Each BPDM application can now be run separately. We will run each application after another. For this, we navigate into the application's subfolder and run the application with the spring-boot run goal.

cd bpdm-orchestrator
mvn spring-boot:run
cd ../bpdm-cleaning-service-dummy
mvn spring-boot:run
cd ../bpdm-pool
mvn spring-boot:run
cd ../bpdm-gate
mvn spring-boot:run

After this you have full-fledged running BPDM system. As a next step on how to pass business partner data and create golden records you can have a look at the api documentation of this repository.

Insecure Installation

You may want to perform a quick installation in which security is not necessary. In this case, you don't need to configure any environment variables. This means you can directly run the BPDM applications after you set up the necessary dependencies. Make sure to disable authentication requirements by using the provided no-auth profile when running the applications:

mvn spring-boot:run -Dspring.profiles.active=no-auth

Helm Charts

Installation of BPDM applications with the Helm Charts has the most software requirements but is the qickest way to set up a running system.

Requirements

  • kubectl (1.30 supported)
  • Docker Engine (tested on 26.1.2)
  • Minikube (tested on 1.33.0)
  • Helm (tested on 3.14.4)

Default Installation

Navigate to the projects root folder. Then install a new release of BPDM on your default namespace via helm:

helm install bpdm ./charts/bpdm

This will install the BPDM applications with its own Postgres and Keycloak in default values. Mind that this will also install the applications with default passwords.

Override Default Secrets

It is good practice to overwrite the default secrets and passwords that are used in the BPDM Charts. For this, you can first define a bunch of environment variables holding new secret values and use them later during deployment:

helm install bpdm \
    --set-value postgres.auth.password=$BPDM_POSTGRES \
    --set-value keycloak.auth.adminPassword=$BPDM_KEYCLOAK_ADMIN \
    --set-value keycloak.bpdm.realm.clientSecrets.cleaningDummyOrchestrator=$BPDM_DUMMY_ORCH_CLIENT_SECRET \
    --set-value keycloak.bpdm.realm.clientSecrets.poolOrchestrator=$BPDM_POOL_ORCH_CLIENT_SECRET \
    --set-value keycloak.bpdm.realm.clientSecrets.gateOrchestrator=$BPDM_GATE_ORCH_CLIENT_SECRET \
    --set-value keycloak.bpdm.realm.clientSecrets.gatePool=$BPDM_GATE_POOL_CLIENT_SECRET \
    --set-value bpdm-gate.applicationSecrets.bpdm.client.orchestrator.registration=$BPDM_GATE_ORCH_CLIENT_SECRET \
    --set-value bpdm-gate.applicationSecrets.bpdm.client.pool.registration=$BPDM_GATE_POOL_CLIENT_SECRET \
    --set-value bpdm-pool.applicationSecrets.bpdm.client.orchestrator.registration=$BPDM_POOL_ORCH_CLIENT_SECRET \
    --set-value bpdm-cleaning-service-dummy.applicationSecrets.bpdm.client.orchestrator.registration=$BPDM_DUMMY_ORCH_CLIENT_SECRET\
 .charts/bpdm/bpdm

Insecure Installation

For non-production purposes you may want to install BPDM applications that are not authenticated. All BPDM applications offer a Spring profile to quickly remove all authentication configuration for their APIs and client connections. In this case you can also disable the Keycloak dependency from being deployed.

helm install bpdm \
    --set-value keycloak.enabled=false
    --set-value bpdm-gate.profiles=["no-auth"] \
    --set-value bpdm-orchestrator.profiles=["no-auth"] \
    --set-value bpdm-pool.profiles=["no-auth"] \
    --set-value bpdm-cleaning-service-dummy.profiles=["no-auth"] 
 .charts/bpdm/bpdm

You can also more fine-granulary remove authentication on APIs and BPDM client connections. You can refer to the no-auth profile configurations (for example that of the Gate) as a documentation.

Use External Dependencies

The BPDM Charts deploy their own PostgreSQL and Keycloak dependencies. However, for production it is recommended to host dedicated Postgres and Keycloak instances with which the BPDM applications should connect to.

Additional Requirements

  • Postgres (15.4.0 supported)
  • Keycloak (22.0.3 supported)

Installation

In this case, you can disable the dependencies and configure the connection to external systems in the application configuration.

helm install bpdm \
    --set-value keycloak.enabled=false
    --set-value postgres.enabled=false
    --set-value bpdm-gate.applicationConfig.bpdm.datasource.host=external-db \
    --set-value bpdm-gate.applicationConfig.bpdm.security.auth-server-url=http://external-keycloak \
    --set-value bpdm-pool.applicationConfig.bpdm.datasource.host=external-db \
    --set-value bpdm-pool.applicationConfig.bpdm.security.auth-server-url=http://external-keycloak \
    --set-value bpdm-orchestrator.applicationConfig.bpdm.security.auth-server-url=http://external-keycloak \
    --set-value bpdm-cleaning-service-dummy.applicationConfig.bpdm.client.orchestrator.provider.issuer-uri= http://external-keycloak/realms/CX-Central \
 .charts/bpdm/bpdm

Fine-granular Configuration

You can configure all BPDM applications over Helm values more fine-granulary via the applicationConfig and applicationSecrets. Values under these groups are directly injected as application properties in the deployed containers.

As a reference of what can be changed have a look at the respective application properties files of each application:

Post-Run Configuration

For the most part BPDM applications don't need to be further configured after they started. The BPDM Pool is an exception to that rule. The Pool offers endpoints for operators to regulate available metadata. Metadata are supporting business partner information like legal forms, identifier types or administrative level 1 areas. For example, by adding a range of legal forms operators are able to determine the available legal forms business partners in the Pool can have.

Typically, business partner data references metadata via technical keys. If a technical key does not exist in the respective metadata the Pool rejects the record.

Administrative level 1 areas follows the ISO 3166-2 norm and is filled by default. Such metadata does not need to be added by the operator.

The use case Postman collection shows which metadata can be added and how that is done. Refer to use cases for Operators(O):

  • CL: Shows how to add available legal forms
  • CIL: Shows how to add available identifier types used for legal entities
  • CIA: Shows how to add available identifier types used for addresses

Please note that in the current implementation there are no endpoints to delete metadata. Deletions would need to be done directly in the database.

EDC Installation

This section shows how to make your BPDM Gate and Pool APIs available over an EDC. This documentation assumes that you already have running BPDM and EDC deployments. For deploying an EDC please consult the documentation on the EDC repository.

Requirements

  • Running BPDM applications
  • Running EDC (0.7.3 supported)

Installation

The general idea of configuring data offers for BPDM is to assets which grant access to a portion of the BPDM APIs. Which API resources are accessible over an asset is determined by the purposes defined in the BPDM framework agreement. For some purposes you may need to access business partner output data from the BPDM Gate for example but won't have access to the input data. Blueprints for such assets are documented in this POSTMAN collection. Accompanying the asset definitions are Policy and Contract Definition blueprints. Except for a general Access Policy those blueprints are grouped by purpose.

After all assets, policies and contract definitions are configured a sharing company's EDC now can query its available assets and the contract under which they are exposed.

Creating Offers

Following are specific instructions on how to create offers to expose the BPDM APIs over EDC. Note that the instructions reference definitions taken from the previously references Postman collection.

ReadAccessPoolForCatenaXMember

This offer allows a company to access the Pool for reading golden record data of all Catena-X members.

  1. Create a policy of type HasBusinessPartnerNumber for the company's BPNL (if it not yet exists)
  2. Create a policy of type AcceptPurpose with usage purpose for the pool (if not yet exists)
  3. Create a technical user with role Pool Cx Member and the company's BPN identity
  4. Create a ReadAccessPoolForCatenaXMember asset with the created technical user for client credentials
  5. Create a contract definition ReadAccessPoolForCatenaXMember referencing the created asset

FullAccessGateInputForSharingMember

This offer allows a company to access a Gate API for uploading business partner data and sending it to the golden record process.

  1. Create a policy of type HasBusinessPartnerNumber for the company's BPNL (if it not already exists)
  2. Create a policy of type AcceptPurpose with upload usage purpose for the gate (if it not already exists)
  3. Create a technical user with role Gate Input Manager and the company's BPN identity
  4. Create a FullAccessGateInputForSharingMember asset with the created technical user for client credentials
  5. Create a contract definition FullAccessGateInputForSharingMember referencing the created asset

ReadAccessGateInputForSharingMember

This offer allows a company to access a Gate API for reading business partner data that has been uploaded by the company. This offer explicitly does not grant access to read the golden record output for the uploaded business partner data.

  1. Create a policy of type HasBusinessPartnerNumber for the company's BPNL (if it not already exists)
  2. Create a policy of type AcceptPurpose with download usage purpose for the gate (if it not already exists)
  3. Create a technical user with role Gate Input Consumer and the company's BPN identity
  4. Create a ReadAccessGateInputForSharingMember asset with the created technical user for client credentials
  5. Create a contract definition ReadAccessGateInputForSharingMember referencing the created asset

ReadAccessGateOutputForSharingMember

This offer allows a company to access a Gate API for reading the golden record output of business partner data previously shared to the golden record process. This offer does not grant access to the uploaded business partner input data the golden record output is based on.

  1. Create a policy of type HasBusinessPartnerNumber for the company's BPNL (if it not already exists)
  2. Create a policy of type AcceptPurpose with download usage purpose for the gate (if it not already exists)
  3. Create a technical user with role Gate Output Consumer and the company's BPN identity
  4. Create a ReadAccessGateOutputForSharingMember asset with the created technical user for client credentials
  5. Create a contract definition ReadAccessGateOutputForSharingMember referencing the created asset

Portal Configuration

This section explains how BPDM needs to operated on the Tractus-X Portal.

At the moment we assume that the Catena-X operator is also the golden record process provider. This means the operator has the Admin role on the Portal.

The following instructions assume you are using the BPDM helm chart to deploy the BPDM services.

Deploy the initial golden record process components

  1. Disable the own Keycloak dependency
  2. Set the authentication server to Central-IDP
  3. Provide the client credentials of pre-existing Central-IDP technical users to the BPDM components to establish communication between the components
  4. Overwrite the standard client-ids to match the one in Central-IDP
  5. Overwrite the BPDM Orchestrator default permission names to be compatible with the Central-IDP (see technical debts)
  6. Expose the Pool over ingress on context path pool to make it available to the Portal
  7. Expose the Portal Gate over ingress on context path companies/test-company to make it available to the Portal
  8. Configure the Gate to not automatically share newly uploaded business partner data to the golden record process
  9. Configure the Gate to have no owner as it needs to be used by multiple companies to manage their own sites and addresses
# Helm Values Overwrite

keycloak:
  enabled: false

bpdm-pool:
  ingress:
    enabled: true
    annotations:
      nginx.ingress.kubernetes.io/rewrite-target: "/$2"
      nginx.ingress.kubernetes.io/use-regex: "true"
      nginx.ingress.kubernetes.io/x-forwarded-prefix: "/pool"
    hosts:
      - host: "bpdm-host-name"
        paths:
          - path: "/pool(/|$)(.*)"
            pathType: "ImplementationSpecific"
  applicationConfig:
    server:
    # App should take the x-forward-header-prefix into account for Swagger-UI and redirects to work correctly
    forward-headers-strategy: "FRAMEWORK"
    bpdm:
      security:
        auth-server-url: "http://central-idp-host-name/auth"
        client-id: "Cl7-CX-BPDM"
      client:
        orchestrator:
          registration:
            client-id: "sa-cl7-cx-5"
  applicationSecrets:
    bpdm:
      client:
        orchestrator:
          registration:
            client-secret: "*****"

bpdm-gate:
  ingress:
    enabled: true
    annotations:
        nginx.ingress.kubernetes.io/rewrite-target: "/$2"
        nginx.ingress.kubernetes.io/use-regex: "true"
        nginx.ingress.kubernetes.io/x-forwarded-prefix: "/companies/test-company"
    hosts:
        - host: "bpdm-host-name"
          paths:
            - path: "/companies/test-company(/|$)(.*)"
              pathType: "ImplementationSpecific"
  applicationConfig:
    server:
    # App should take the x-forward-header-prefix into account for Swagger-UI and redirects to work correctly
    forward-headers-strategy: "FRAMEWORK"
    bpdm:
      bpn:
        owner-bpn-l:
      tasks:
        creation:
          fromSharingMember:
            starts-as-ready: false
      security:
        auth-server-url: "http://central-idp-host-name/auth"
        client-id: "Cl16-CX-BPDMGate"
      client:
        orchestrator:
          registration:
            client-id: "sa-cl7-cx-5"
        pool:
          registration:
            client-id: "sa-cl7-cx-5"
  applicationSecrets:
    bpdm:
      client:
        orchestrator:
          registration:
            client-secret: "*****"
        pool:
          registration:
            client-secret: "*****"

bpdm-orchestrator:
  applicationConfig:
    bpdm:
      security:
        auth-server-url: "http://central-idp-host-name/auth"
        client-id: "Cl7-CX-BPDM"
        permissions:
          createTask: "write_partner"
          readTask: "write_partner"
          reservation:
            clean: "write_partner"
            cleanAndSync: "write_partner"
            poolSync: "write_partner"
          result:
            clean: "write_partner"
            cleanAndSync: "write_partner"
            poolSync: "write_partner"

bpdm-cleaning-service-dummy:
  applicationConfig:
    bpdm:
      client:
        orchestrator:
          provider:
            issuer-uri: "https://central-idp-host-name/auth/realms/CX-Central"
          registration:
            client-id: "sa-cl7-cx-5"
  applicationSecrets:
    bpdm:
      client:
        orchestrator:
          registration:
            client-secret: "*****"

Create BPDM marketplace services

For giving companies access to the golden record process and/or Pool the operator needs to create appropriate EDC offers. The EDC offers should provide access to the Pool API and Gate API. Companies should not access the APIs directly over technical users but rather use the EDC offers.

The most important pre-requisite for creating such EDC offers are technical users which have the correct roles and BPN identities. The Portal supports creating such technical users by creating marketplace services that companies can subscribe to. After the company subscribed to a service a technical user will be created automatically by the Portal containing the BPN identity of the subscribing company and the roles defined in the marketplace service.

For BPDM the operator needs to create 3 services:

  1. Business Partner Members Pool Service: Access to the Catena-X member golden records
  2. Sharing Member Upload Service: Upload company business partner data and share it to the golden record process
  3. Sharing Member Download Service: Download golden records for previously shared business partner data

When creating the service, the operator needs to specify with which role the technical user should be initialized after a company subscribes to it:

  1. Business Partner Members Pool Service: Pool Cx Member
  2. Sharing Member Upload Service: Gate Input Manager
  3. Sharing Member Download Service: Gate Output Consumer

Manage BPDM service subscriptions

When a company subscribes to a BPDM service the operator needs to perform the following tasks:

  1. Create a new Gate deployment (if it is a sharing member service)
  2. Create an EDC offer based on the service using the created technical user

If a new Gate has to be deployed it needs to integrate into the existing BPDM components:

  1. Disable all components except the Gate to only deploy a new Gate instance
  2. Make sure that the Gate is accessible to your EDC (either by ingress or internal host name)
  3. Set the subscribing company's BPNL as the owner of that Gate
  4. Optionally configure whether uploaded business partner data should be automatically shared to the golden record process according to the subscribing companies wishes
  5. Set the Gate's Orchestrator and Pool client base-urls so that the Gate can find the existing components
# Helm Values Overwrite

keycloak:
  enabled: false

bpdm-pool:
  enabled: false

bpdm-gate:
  applicationConfig:
    bpdm:
      bpn:
        owner-bpn-l: BPNLXXXXXXXXXXXX
      tasks:
        creation:
          fromSharingMember:
            starts-as-ready: false/true
      security:
        auth-server-url: "http://central-idp-host-name/auth"
        client-id: "Cl16-CX-BPDMGate"
      client:
        orchestrator:
          base-url: "orchestrator-api-url"
          registration:
            client-id: "sa-cl7-cx-5"
        pool:
          base-url: "pool-api-url"
          registration:
            client-id: "sa-cl7-cx-5"
  applicationSecrets:
    bpdm:
      client:
        orchestrator:
          registration:
            client-secret: "*****"
        pool:
          registration:
            client-secret: "*****"

bpdm-orchestrator:
  enabled: false

bpdm-cleaning-service-dummy:
  enabled: false

NOTICE

This work is licensed under the Apache-2.0.

  • SPDX-License-Identifier: Apache-2.0
  • SPDX-FileCopyrightText: 2023,2024 ZF Friedrichshafen AG
  • SPDX-FileCopyrightText: 2023,2024 SAP SE
  • SPDX-FileCopyrightText: 2023,2024 Bayerische Motoren Werke Aktiengesellschaft (BMW AG)
  • SPDX-FileCopyrightText: 2023,2024 Mercedes Benz Group
  • SPDX-FileCopyrightText: 2023,2024 Robert Bosch GmbH
  • SPDX-FileCopyrightText: 2023,2024 Schaeffler AG
  • SPDX-FileCopyrightText: 2023,2024 Contributors to the Eclipse Foundation
  • Source URL: https://github.com/eclipse-tractusx/bpdm